=Paper=
{{Paper
|id=Vol-2183/position5
|storemode=property
|title=Text Entry for People with Mild Cognitive Impairments
|pdfUrl=https://ceur-ws.org/Vol-2183/position5.pdf
|volume=Vol-2183
|authors=Ryan Gibson,Mark D Dunlop,Majed Al Khan,Gennaro Imperatore
}}
==Text Entry for People with Mild Cognitive Impairments==
https://ceur-ws.org/Vol-2183/position5.pdf
Barcelona, Spain | September 3, 2018 MobileHCI 2018 Workshop on Socio-Technical Aspects of Text Entry
Text Entry for People with Mild
Cognitive Impairments
Abstract
Ryan Gibson Augmentative and Alternative Communication
Mark D Dunlop technologies have largely focussed on people with
severe motor impairments or people who cannot speak.
Majed Al Khan
In this position paper we wish to discuss how text entry
Gennaro Imperatore
can better support people with mild cognitive
Computer and Information Sciences impairments, what contexts text entry matters for them
University of Strathclyde and how studies could take their needs into account.
Glasgow G1 1XH, Scotland UK
Author Keywords
ryan.gibson@strath.ac.uk Text entry; assistive technology; cognitive impairment
gennaro.imperatore@strath.ac.uk
ACM Classification Keywords
majed.khan@strath.ac.uk
K.4.2 Social Issues: Assistive technologies for persons
mark.dunlop@strath.ac.uk with disabilities
Introduction
The field of text entry for persons with intellectual
disabilities has been under researched in recent years
compared to that of other vulnerable groups such as
visually impaired users (e.g. [2][9,12]). Many
recommendations for making technologies accessible to
Permission to make digital or hard copies of part or all of this work for personal or
classroom use is granted without fee provided that copies are not made or distributed
people with learning disabilities focus on the utilisation
for profit or commercial advantage and that copies bear this notice and the full citation of easy read methods. For example, proloquo2go
on the first page. Copyrights for third-party components of this work must be honored.
For all other uses, contact the owner/author(s).
allows the user to select options consisting of a
Copyright held by the owner/author(s). combination of images and simplified text in order to
MobileHCI, 2018 Barcelona, Spain. formulate a sentence [11]. The selected options are
28
Barcelona, Spain | September 3, 2018 MobileHCI 2018 Workshop on Socio-Technical Aspects of Text Entry
then played back to ensure the needs of the individual models, and interaction modalities. The authors found
are made known. 61 distinct text entry methods over the past 30 years
and evaluated these against a set of measurables
Williams and Henig found a potential issue with this
ranging from the target group to the selection
approach when exploring the accessibility of online
techniques, language models and modalities used.
content [13]. They found that a considerable number
of participants would either have difficulty Selection techniques were grouped into 3 categories.
understanding the meaning conveyed by the embedded “Direct Selection” involves the user choosing a
images or would interpret them in different manners. particular key from a limited set and typically includes
As such, those participants with a greater three techniques for reducing the length of such sets:
understanding of language relied upon the chording keyboards, ambiguous keyboards, and
accompanying text to navigate through the interface. encoding. Ambiguous keyboards are the most
This highlights the need to develop such resources in commonly used technique with this population and
conjunction with the views of target stakeholders to involves grouping letters of the alphabet into one key.
ensure they are understood as intended and have the Multiple taps are then required to select the desired
desired effect. character.
Furthermore, the inclusion of images results in longer Reducing the number of keys improves access for
pages that may require scrolling to view all the content people with motor control difficulties but often comes at
available. People with learning disabilities have an a cost of increased cognitive load and decreased entry
increased chance of living in a home seriously affected speed. A former chief executive of a learning disability
by poverty [3] and may therefore suffer from digital charity, interviewed by Gibson et al. [10], also
exclusion. Consequently, this population may be suggested that people with short attention spans may
unfamiliar with technologically specific actions such as have difficulty completing tasks that contain additional
scrolling or pinching- as highlighted by Williams and steps: “Again it would depend on how easy they were
Henig who found that some participants were unaware to use but the quicker the better I would say. The
that “invisible” content existed. shorter the better in terms of how much time someone
would have to [complete it]. So easy to use
People with learning disabilities are highly
absolutely…as few kinds of steps in the process, as few
heterogeneous in nature and often have additional
clicks in the process as possible.”
impairments that affect aspects such as their linguistic
and motor abilities [1]. Significant research has been “Scanning” is used when a very low number of keys are
conducted into text entry techniques for people with available to the user (typically one or two). The
poor fine-motor skills. Polacek et al. [8] has conducted technique typically includes a sequential highlighting
an extensive overview of the common techniques used algorithm that presents options to the user until the
by this population including: selection of keys, desired item is selected, in this case a character or
approaches to character layouts, use of language group of characters. “Row-column” scanning is one of
29
Barcelona, Spain | September 3, 2018 MobileHCI 2018 Workshop on Socio-Technical Aspects of Text Entry
the simplest techniques used in which potential items meaning the sequence of operations required to type a
are organised within a matrix. The algorithm will character) the sequence of operations required to enter
sequentially highlight the rows first until a selection has text remain consistent throughout. In comparison,
been made before the items in the selected row are dynamic distributions alter the sequence of actions
linearly scanned. More complex scanning techniques required to enter text depending on the current context
have been described in depth by Polacek et al. [8], e.g. – for example altering the sequence of letters on each
scanning ambiguous keyboards [5]. Scanning solutions key based on the current written context. This may be
tend to be focussed on people with very limited motion cognitively demanding for people with learning
as they are particularly slow input techniques, for disabilities - a population that often requires a
people with learning difficulties they could also cause consistent and predictable approach to communicating
interaction problems as short-term memory may be or navigating across user interfaces, as discussed by
stressed by the slow nature of the input. one of the experts interviewed by Gibson et al., [10]; “I
suppose that good practice would say you should
“Pointing and gestures” involves using non-traditional
always take a consistent approach to your
methods to select options, such as pointing devices
communication style with people [who have learning
controlled by trackballs, joysticks, head tracking,
disabilities].”
eye-gazing software etc. Some of these solutions,
however, do not support direct selection e.g. Language models are the final techniques characterised
eye-gazing. 3 common techniques are used to by Polacek et al. [8] and are a means of characterising
overcome this issue. “Dwell-time” supports selection language in a structured and consistent way. Almost
when the cursor rests within a predefined radius for a all text entry methods use a language model as a
select period of time. “Multimodal Interaction” involves means of predicting the intended input of the user. 3
the use of various modalities to confirm a selection. essential approaches were discovered by the authors:
This may include actions such as head movements, syntactic, semantic and statistical. Syntactic and
speech recognition, non-verbal vocal commands etc. semantic approaches store rules either in probability
“Gestural input” involves the transformation of strokes, tables or as a grammar and the difference between the
made via the pointing device, into text or through two lies in the categorisation of words (syntactic or
dynamic interaction (e.g. [9]). While necessary for semantic categorization). The statistical approach
those with learning difficulties who cannot use predicts input based on historical statistics of usage,
touchscreens or physical keyboards these are not typically as word or letter n-grams. The order of the
suitable for others. model further refers to the longest n-gram contained in
the language model and the probability of the next
Character layout involves detecting the optimal layout
items is extracted from the model based on already
of characters, or sequences of characters, used to
written n - 1 items. People with mild learning
maximize the stakeholders type rate. They may
difficulties tend to have reduced vocabulary and may
generally be divided into two categories: static and
have difficulties with spelling and grammatical
dynamic. During static distributions (distributions
30
Barcelona, Spain | September 3, 2018 MobileHCI 2018 Workshop on Socio-Technical Aspects of Text Entry
construction. Targeted language models may ● how text entry can better support people with
considerably help input but would lead to interaction mild cognitive impairments,
style challenges such as: inexperience in using modern ● what contexts text entry matters for them,
touchscreen interaction modalities; and the potentially ● and how studies could take their needs into
excessive cognitive load placed on stakeholders due to account.
word corrections/suggestions.
In particular mild cognitive impairments can lead to
The work by Polacek et al. [8] is certainly a starting slow entry rates, forgetting the context of messages,
point for exploring text entry methods for people with reduced social awareness/empathy and difficulty in
learning disabilities. The paper highlights the various remembering words.
techniques used in existing resources and discusses
Cognitive impairments are also often tied with other
how these affect people with significant motor
impediments to fluid text entry such as reduced vision
impairments (a condition prevalent throughout the
and motor control difficulties (e.g. post stroke). How do
learning disability population). It would be interesting
these co-limiters impact? In particular short term
to discuss further how these techniques may be
memory reductions could seriously impact the ability to
adapted to suit the complex needs of people with
use slow text input methods or exploit autocorrect
learning disabilities, particularly how they may address
suggestions.
the cognitive deficiencies present throughout.
Text entry study formats also need adjusted for people
Another solution is to borrow from the field of AAC.
with mild learning difficulties. Copy tasks may overly
Traditionally AAC and Text Entry have largely been
challenge short-term memory limitations while
considered as separate fields, however as pointed by
composition tasks may be difficult on topics that the
[14] AAC and Text Entry share the goal of improving
participants are not comfortable with.
the communication experience of users. Fried-Olsen et
al. [15] argue that AAC have a great potential to help Finally, can we develop general purpose text entry
users with neurodegenerative diseases. The great methods that (semi-)automatically adapt to the
computational power available today and the individual abilities and restrictions of users?
affordability of smart devices could lead to Text
Entry/AAC systems which could adapt to both stable
Biographies
Ryan Gibson has just completed his MPhil on support
and declining cognitive impairments.
for people with cognitive impairments in preparation for
Finally Norman and Alm [16] show the promise of AAC
medical appointments [10]. He has now started a PhD
systems to help those affected by dementia.
between the Digital Health & Wellbeing and Data
Distinct text entry needs of people with Analytics & Mobile Interaction groups at Strathclyde.
cognitive impairments
In this position paper we wish to discuss:
31
Barcelona, Spain | September 3, 2018 MobileHCI 2018 Workshop on Socio-Technical Aspects of Text Entry
Gennaro Imperatore recently completed his PhD on Interaction 8, 2: 20–46.
generative AAC for people with speech production
7. Craig O’Neil, Mark D. Dunlop, and Andrew Kerr. 2015.
problems as an after effect of a stroke [4]. Supporting Sit-To-Stand Rehabilitation Using
Smartphone Sensors and Arduino Haptic Feedback
Majed Al Khan is conducting a PhD on supporting
Modules. Adjunct Proceedings of MobileHCI ’15.
navigation and independent movement of people with
Down’s Syndrome using smarter mobile and wearable 8. Ondrej Polacek, Adam J. Sporka, and Pavel Slavik.
2015. Text input for motor-impaired people. Universal
technologies.
Access in the Information Society 16, 1: 51–72.
Mark Dunlop is a senior lecturer and leads Data
9. Daniel Rough, Keith Vertanen, and Per Ola Kristensson.
Analytics & Mobile Interaction groups at Strathclyde. He 2014. An evaluation of Dasher with a high-performance
has a long history of research in mobile text entry, language model as a gaze communication method.
conducted studies with older adults (e.g. [6] ) and Proc AVI ’14.
investigated support apps for stroke survivors [7]. 10. Ryan Colin Gibson, Matt-Mouley Bouamrane, Mark
Dunlop. 2018, Mobile Support for Adults with Mild
References Learning Disabilities during Clinical Consultations. Proc
MobileHCI 2018, ACM.
1. Franca Garzotto and Roberto Gonella. 2011. An
open-ended tangible environment for disabled 11. Samuel Sennott and Adam Bowker. 2009. Autism,
children’s learning. Proc - IDC ’11. AAC, and Proloquo2Go. Perspectives on Augmentative
and Alternative Communication 18, 4: 137.
2. Tiago Guerreiro, Paulo Lagoá, Hugo Nicolau, Pedro
Santana, and Joaquim Jorge. 2008. Mobile text-entry 12. Caleb Southern, James Clawson, Brian Frey, Gregory
models for people with disabilities. Proc ECCE ’08. Abowd, and Mario Romero. 2012. An evaluation of
BrailleTouch. Proceedings of MobileHCI ’12.
3. Pauline Heslop and Anna Marriott. 2015. Making a
difference - the impact of the Confidential Inquiry into 13. Peter Williams and Christian Hennig. 2015. Optimising
premature deaths of people with learning disabilities. web site designs for people with learning disabilities.
British Journal of Learning Disabilities 43, 2: 142–149. Journal of research in special educational needs, 15(1)
4. Gennaro Imperatore and Mark D. Dunlop. 2015. An 14. Johansen, S., Hansen, P., 2006. Augmentative and
Investigation into Appropriation of Portable Smart Alternative Communication: The Future of Text on
Devices by Users with Aphasia. Proc - ASSETS ’15. the Move. Univers Access Inf Soc 5, 125–149.
5. I. Scott Mackenzie, I. Scott Mackenzie, and Torsten
15. Fried-Oken, M., Mooney, A., Peters, B., 2015.
Felzer. 2010. SAK: Scanning Ambiguous Keyboard for
Supporting communication for patients with
Efficient One-Key Text Entry. ACM transactions on
computer-human interaction, 1 7, 3: 1–39. neurodegenerative disease. NeuroRehabilitation 37.
6. Emma Nicol, Andreas Komninos, and Mark D. Dunlop. 16. Arnott, J.L., Alm, N., 2016. How Can We Develop
2016. A Participatory Design and Formal Study AAC for Dementia?, in: Computers Helping People
Investigation into Mobile Text Entry for Older Adults. with Special Needs, LNCS. Springer.
International Journal of Mobile Human Computer
32