Barcelona, Spain | September 3, 2018 MobileHCI 2018 Workshop on Socio-Technical Aspects of Text Entry Text Entry for People with Mild Cognitive Impairments Abstract Ryan Gibson Augmentative and Alternative Communication Mark D Dunlop technologies have largely focussed on people with severe motor impairments or people who cannot speak. Majed Al Khan In this position paper we wish to discuss how text entry Gennaro Imperatore can better support people with mild cognitive Computer and Information Sciences impairments, what contexts text entry matters for them University of Strathclyde and how studies could take their needs into account. Glasgow G1 1XH, Scotland UK Author Keywords ryan.gibson@strath.ac.uk Text entry; assistive technology; cognitive impairment gennaro.imperatore@strath.ac.uk ACM Classification Keywords majed.khan@strath.ac.uk K.4.2 Social Issues: Assistive technologies for persons mark.dunlop@strath.ac.uk with disabilities Introduction The field of text entry for persons with intellectual disabilities has been under researched in recent years compared to that of other vulnerable groups such as visually impaired users (e.g. ​[2]​[9,12]​). Many recommendations for making technologies accessible to Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed people with learning disabilities focus on the utilisation for profit or commercial advantage and that copies bear this notice and the full citation of easy read methods. For example, proloquo2go on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the owner/author(s). allows the user to select options consisting of a Copyright held by the owner/author(s). combination of images and simplified text in order to MobileHCI, 2018 Barcelona, Spain. formulate a sentence ​[11]​. The selected options are 28 Barcelona, Spain | September 3, 2018 MobileHCI 2018 Workshop on Socio-Technical Aspects of Text Entry then played back to ensure the needs of the individual models, and interaction modalities. The authors found are made known. 61 distinct text entry methods over the past 30 years and evaluated these against a set of measurables Williams and Henig found a potential issue with this ranging from the target group to the selection approach when exploring the accessibility of online techniques, language models and modalities used. content ​[13]​. They found that a considerable number of participants would either have difficulty Selection techniques were grouped into 3 categories. understanding the meaning conveyed by the embedded “Direct Selection” involves the user choosing a images or would interpret them in different manners. particular key from a limited set and typically includes As such, those participants with a greater three techniques for reducing the length of such sets: understanding of language relied upon the chording keyboards, ambiguous keyboards, and accompanying text to navigate through the interface. encoding. Ambiguous keyboards are the most This highlights the need to develop such resources in commonly used technique with this population and conjunction with the views of target stakeholders to involves grouping letters of the alphabet into one key. ensure they are understood as intended and have the Multiple taps are then required to select the desired desired effect. character. Furthermore, the inclusion of images results in longer Reducing the number of keys improves access for pages that may require scrolling to view all the content people with motor control difficulties but often comes at available. People with learning disabilities have an a cost of increased cognitive load and decreased entry increased chance of living in a home seriously affected speed. A former chief executive of a learning disability by poverty ​[3]​ and may therefore suffer from digital charity, interviewed by Gibson et al. [10], also exclusion. Consequently, this population may be suggested that people with short attention spans may unfamiliar with technologically specific actions such as have difficulty completing tasks that contain additional scrolling or pinching- as highlighted by Williams and steps: ​“Again it would depend on how easy they were Henig who found that some participants were unaware to use but the quicker the better I would say. The that “invisible” content existed. shorter the better in terms of how much time someone would have to [complete it]. So easy to use People with learning disabilities are highly absolutely…as few kinds of steps in the process, as few heterogeneous in nature and often have additional clicks in the process as possible.” impairments that affect aspects such as their linguistic and motor abilities ​[1]​. Significant research has been “Scanning” is used when a very low number of keys are conducted into text entry techniques for people with available to the user (typically one or two). The poor fine-motor skills. Polacek et al. ​[8]​ has conducted technique typically includes a sequential highlighting an extensive overview of the common techniques used algorithm that presents options to the user until the by this population including: selection of keys, desired item is selected, in this case a character or approaches to character layouts, use of language group of characters. “Row-column” scanning is one of 29 Barcelona, Spain | September 3, 2018 MobileHCI 2018 Workshop on Socio-Technical Aspects of Text Entry the simplest techniques used in which potential items meaning the sequence of operations required to type a are organised within a matrix. The algorithm will character) the sequence of operations required to enter sequentially highlight the rows first until a selection has text remain consistent throughout. In comparison, been made before the items in the selected row are dynamic distributions alter the sequence of actions linearly scanned. More complex scanning techniques required to enter text depending on the current context have been described in depth by Polacek et al. ​[8]​, e.g. – for example altering the sequence of letters on each scanning ambiguous keyboards ​[5]​. Scanning solutions key based on the current written context. This may be tend to be focussed on people with very limited motion cognitively demanding for people with learning as they are particularly slow input techniques, for disabilities - a population that often requires a people with learning difficulties they could also cause consistent and predictable approach to communicating interaction problems as short-term memory may be or navigating across user interfaces, as discussed by stressed by the slow nature of the input. one of the experts interviewed by Gibson et al., [10]; ​“I suppose that good practice would say you should “Pointing and gestures” involves using non-traditional always take a consistent approach to your methods to select options, such as pointing devices communication style with people [who have learning controlled by trackballs, joysticks, head tracking, disabilities].” eye-gazing software etc. Some of these solutions, however, do not support direct selection e.g. Language models are the final techniques characterised eye-gazing. 3 common techniques are used to by Polacek et al. ​[8]​ and are a means of characterising overcome this issue. “Dwell-time” supports selection language in a structured and consistent way. Almost when the cursor rests within a predefined radius for a all text entry methods use a language model as a select period of time. “Multimodal Interaction” involves means of predicting the intended input of the user. 3 the use of various modalities to confirm a selection. essential approaches were discovered by the authors: This may include actions such as head movements, syntactic, semantic and statistical. Syntactic and speech recognition, non-verbal vocal commands etc. semantic approaches store rules either in probability “Gestural input” involves the transformation of strokes, tables or as a grammar and the difference between the made via the pointing device, into text or through two lies in the categorisation of words (syntactic or dynamic interaction (e.g. ​[9]​). While necessary for semantic categorization). The statistical approach those with learning difficulties who cannot use predicts input based on historical statistics of usage, touchscreens or physical keyboards these are not typically as word or letter n-grams. The order of the suitable for others. model further refers to the longest n-gram contained in the language model and the probability of the next Character layout involves detecting the optimal layout items is extracted from the model based on already of characters, or sequences of characters, used to written n - 1 items. People with mild learning maximize the stakeholders type rate. They may difficulties tend to have reduced vocabulary and may generally be divided into two categories: static and have difficulties with spelling and grammatical dynamic. During static distributions (distributions 30 Barcelona, Spain | September 3, 2018 MobileHCI 2018 Workshop on Socio-Technical Aspects of Text Entry construction. Targeted language models may ● how text entry can better support people with considerably help input but would lead to interaction mild cognitive impairments, style challenges such as: inexperience in using modern ● what contexts text entry matters for them, touchscreen interaction modalities; and the potentially ● and how studies could take their needs into excessive cognitive load placed on stakeholders due to account. word corrections/suggestions. In particular mild cognitive impairments can lead to The work by Polacek et al. ​[8]​ is certainly a starting slow entry rates, forgetting the context of messages, point for exploring text entry methods for people with reduced social awareness/empathy and difficulty in learning disabilities. The paper highlights the various remembering words. techniques used in existing resources and discusses Cognitive impairments are also often tied with other how these affect people with significant motor impediments to fluid text entry such as reduced vision impairments (a condition prevalent throughout the and motor control difficulties (e.g. post stroke). How do learning disability population). It would be interesting these co-limiters impact? In particular short term to discuss further how these techniques may be memory reductions could seriously impact the ability to adapted to suit the complex needs of people with use slow text input methods or exploit autocorrect learning disabilities, particularly how they may address suggestions. the cognitive deficiencies present throughout. Text entry study formats also need adjusted for people Another solution is to borrow from the field of AAC. with mild learning difficulties. Copy tasks may overly Traditionally AAC and Text Entry have largely been challenge short-term memory limitations while considered as separate fields, however as pointed by composition tasks may be difficult on topics that the [14] AAC and Text Entry share the goal of improving participants are not comfortable with. the communication experience of users. Fried-Olsen et al. [15] argue that AAC have a great potential to help Finally, can we develop general purpose text entry users with neurodegenerative diseases. The great methods that (semi-)automatically adapt to the computational power available today and the individual abilities and restrictions of users? affordability of smart devices could lead to Text Entry/AAC systems which could adapt to both stable Biographies Ryan Gibson has just completed his MPhil on support and declining cognitive impairments. for people with cognitive impairments in preparation for Finally Norman and Alm [16] show the promise of AAC medical appointments ​[10]​. He has now started a PhD systems to help those affected by dementia. between the Digital Health & Wellbeing and Data Distinct text entry needs of people with Analytics & Mobile Interaction groups at Strathclyde. cognitive impairments In this position paper we wish to discuss: 31 Barcelona, Spain | September 3, 2018 MobileHCI 2018 Workshop on Socio-Technical Aspects of Text Entry Gennaro Imperatore recently completed his PhD on Interaction​ 8, 2: 20–46. generative AAC for people with speech production 7. Craig O’Neil, Mark D. Dunlop, and Andrew Kerr. 2015. problems as an after effect of a stroke ​[4]​. Supporting Sit-To-Stand Rehabilitation Using Smartphone Sensors and Arduino Haptic Feedback Majed Al Khan is conducting a PhD on supporting Modules. ​Adjunct Proceedings of MobileHCI ’15.​ navigation and independent movement of people with Down’s Syndrome using smarter mobile and wearable 8. Ondrej Polacek, Adam J. Sporka, and Pavel Slavik. 2015. Text input for motor-impaired people. ​Universal technologies. Access in the Information Society​ 16, 1: 51–72. Mark Dunlop is a senior lecturer and leads Data 9. Daniel Rough, Keith Vertanen, and Per Ola Kristensson. Analytics & Mobile Interaction groups at Strathclyde. He 2014. An evaluation of Dasher with a high-performance has a long history of research in mobile text entry, language model as a gaze communication method. conducted studies with older adults (e.g. ​[6]​ ) and Proc AVI ’14.​ investigated support apps for stroke survivors ​[7]​. 10. Ryan Colin Gibson, Matt-Mouley Bouamrane, Mark Dunlop. 2018, Mobile Support for Adults with Mild References Learning Disabilities during Clinical Consultations. ​Proc MobileHCI 2018,​ ACM. 1. Franca Garzotto and Roberto Gonella. 2011. An open-ended tangible environment for disabled 11. Samuel Sennott and Adam Bowker. 2009. Autism, children’s learning. ​Proc - IDC ’11​. AAC, and Proloquo2Go. ​Perspectives on Augmentative and Alternative Communication​ 18, 4: 137. 2. Tiago Guerreiro, Paulo Lagoá, Hugo Nicolau, Pedro Santana, and Joaquim Jorge. 2008. Mobile text-entry 12. Caleb Southern, James Clawson, Brian Frey, Gregory models for people with disabilities. ​Proc ECCE ’08​. Abowd, and Mario Romero. 2012. An evaluation of BrailleTouch. ​Proceedings of MobileHCI ’12​. 3. Pauline Heslop and Anna Marriott. 2015. Making a difference - the impact of the Confidential Inquiry into 13. Peter Williams and Christian Hennig. 2015. Optimising premature deaths of people with learning disabilities. web site designs for people with learning disabilities. British Journal of Learning Disabilities​ 43, 2: 142–149. Journal of research in special educational needs, ​15(1​) 4. Gennaro Imperatore and Mark D. Dunlop. 2015. An 14. Johansen, S., Hansen, P., 2006. Augmentative and Investigation into Appropriation of Portable Smart Alternative Communication: The Future of Text on Devices by Users with Aphasia. ​Proc - ASSETS ’15​. the Move. Univers Access Inf Soc 5, 125–149. 5. I. Scott Mackenzie, I. Scott Mackenzie, and Torsten 15. Fried-Oken, M., Mooney, A., Peters, B., 2015. Felzer. 2010. SAK: Scanning Ambiguous Keyboard for Supporting communication for patients with Efficient One-Key Text Entry. ​ACM transactions on computer-human interaction, 1 ​ 7, 3: 1–39. neurodegenerative disease. NeuroRehabilitation 37. 6. Emma Nicol, Andreas Komninos, and Mark D. Dunlop. 16. Arnott, J.L., Alm, N., 2016. How Can We Develop 2016. A Participatory Design and Formal Study AAC for Dementia?, in: Computers Helping People Investigation into Mobile Text Entry for Older Adults. with Special Needs, LNCS. Springer. International Journal of Mobile Human Computer 32