=Paper=
{{Paper
|id=Vol-3404/keynote1
|storemode=property
|title=Human-Computer Interface Design of a Communication Aid Software System for People with Parkinson’s Disease
|pdfUrl=https://ceur-ws.org/Vol-3404/paper8.pdf
|volume=Vol-3404
|authors=Yohan Guerrier,Sophie Lepreux,Christophe Kolski,Véronique Delcroix,Káthia Marçal de Oliveira
|dblpUrl=https://dblp.org/rec/conf/eics/GuerrierLKDO22
}}
==Human-Computer Interface Design of a Communication Aid Software System for People with Parkinson’s Disease==
https://ceur-ws.org/Vol-3404/paper8.pdf
Human-Computer Interface Design of a Communication Aid
Software System for People with Parkinson’s Disease
Yohan Guerrier1, Sophie Lepreux1, Christophe Kolski1, Véronique Delcroix1,
and Káthia Marçal de Oliveira 1
1
Univ. Polytechnique Hauts-de-France, CNRS, UMR 8201 LAMIH, F-59313 Valenciennes, France
Abstract
Easy to use, intuitive and adaptive; with these three requirements defined by a neurologist we
started the development of a communication aid software system for people with Parkinson's
disease, named ParkinsonCom. We understood that the Human-Computer Interaction (HCI)
would be a key issue. Following HCI design principles, we decided to design the user
interfaces, make mock-ups and validate them with users before implementation. This paper
presents how this process was accomplished in a totally distant way (by internet) due to the
confinement, and the results obtained.
Keywords
Communication, Assistive technology, disability, Parkinson 1
1. Introduction
Parkinson's disease is the second most common neurodegenerative disease in the world. The quality
of life and social participation of people with Parkinson's disease (PD) are severely affected, not only
by motor symptoms, but especially by non-motor symptoms such as communication disorders that
impair speech intelligibility, the ability to express emotional states and, consequently, social
relationships. In order to improve this situation, we have been working on the development of a
communication aid software system for people with PD in the ParkinsonCom project [1].
Using a user-centered design approach [2], we developed a first prototype. To that end, a process of
design, mockup and validation of the user interface was performed. Our main challenge for this was to
accomplish all these issues remotely via internet meetings due to COVID's health crisis. Following the
standard NF EN ISO 9241-210 (2011) [3] concerning the user-centered process, a survey was carried
out to identify the main difficulties encountered by people having PD. Next, we made interviews with
experts (11 persons with PD, 7 spouses of persons with PD and 3 neurologists). After extracting the
essential needs of people with PD, we defined the priorities about functionalities to be implemented in
our prototype:
• Make a dialog - made for PD patients to communicate with people, they can type a dialogue
using the keyboard and the pictograms suggested while typing in text area. Those dialogs can
be reused later by accessing the appropriate menu.
• Make a dialog based on suggested ones from library: we propose some ready dialogs that can
be used, modified or completed.
• Use a dialog - saved dialogs or proposed ones can be modified or read by voice synthesis.
• Short request - Request for help to a specific person by a call, especially in off mode (a period
of blockage or discomfort) [4]. Besides, users can access this screen to entertain themselves by
listening to music via an audio player, or listen to funny stories.
Engineering Interactive Computing Systems for People with Disabilities, June 21–06, 2022, Sophia-Antipolis, France
EMAIL: yohan.guerrier@uphf.fr (A. 1); sophie.lepreux@uphf.fr (A. 2); christophe.kolski@uphf.fr (A. 3); veronique.delcroix@uphf.fr (A.
4); kathia.oliveira@uphf.fr (A. 5) ORCID: 0000-0003-1040-0799 (A. 1); 0000-0002-0582-7993 (A. 2); 0000-0002-7881-6435 (A. 3); 0000-
0001-6367-078X (A. 4); 0000-0001-8146-5966 (A. 5)
© 2022 Copyright for this paper by its authors.
Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).
CEUR
Wor
Pr
ks
hop
oceedi
ngs
ht
I
tp:
//
ceur
-
SSN1613-
ws
.or
0073
g
CEUR Workshop Proceedings (CEUR-WS.org)
With those defined functionalities, we had to design the user interfaces before the final
implementation. Our main difficulty was how to define user interfaces without meeting the users. With
the period of confinement, any face-to-face interaction was prohibited, especially with users with a risk
profile (elderly people with some illness). How to guarantee simple and intuitive interfaces? We decided
therefore to set up a workshop with psychologists and psycho-educators who were partners in the
project to draw these user interfaces; then, to make mockups of them; and finally, to present and validate
them with final users (people with Parkinson's disease) remotely.
This article reports this experience of defining user interfaces for the ParkinsonCom software.
Section 2 presents a brief background, before describing the workshop performed for brainstorming
about the user interface design (section 3) and the mockups defined from the results of the workshop
(section 4). Section 5 presents the results of the validation of the mockups with people with Parkinson.
Finally, section 6 shows our next steps.
2. Background
Convinced that we should use a user-centered approach, we asked ourselves about which techniques
and procedures to apply and which technologies to use. To answer this question, we have performed a
rapid review [5] following the guidelines defined by [6] with the objective of analyzing the studies
performed for communication tools or in the field of Parkinson Disease.
Applying a specific search string (("user-centered" OR "human-centered" OR "user-centered" OR
"human-centered") AND ("communication" OR "conversation" OR "ICT" OR "parkinson")) in
SCOPUS database, we obtained 1576 documents, that after the elimination of duplicates, resulted in
1,548 documents to analyze. With a reading procedure searching only articles for our objective we
obtained 67 final articles.
Figure 1 shows the different approaches of user-centered design to capture the needs of future users,
as well as to proceed with the design and evaluation. The three most used methods are interviews,
prototype testing and workshops.
Figure 1: User-centered design approaches
Of the 29 articles mentioning the use of interviews, we found that they were used with Parkinson's
patients, people with Autism Spectrum Disorder (ASD), people with mental disabilities, elderly people,
and people with speech impairments in general. The majority of the articles concerned applications
related to communication support (e.g. [7], [8]). Of the 17 articles dedicated to prototype testing, the
approach most often used is to create a prototype to identify needs, then test and improve it. We also
identified 17 articles that cite the use of workshops to identify needs and requirements. Experts and
future users usually participate in those workshops to discuss an application under development.
The rapid review showed also that the most used technologies are smartphones (in 32 of the selected
articles), tablets (20) and computers (17). If we consider only the works for people with PD, the most
used technological support is the tablet (10 of 13). These technologies are primarily used through the
touch interaction mode in a large majority of cases (37). In some of these works the alternative and
augmented communication (AAC) is presented as an effective interaction mode. For instance, Ilyas [9]
developed a communication tool (pad communication) for people with disabilities, particularly having
speech inhibition to communicate. Grigis and Lazzari [10] showed the importance of the AAC on tablets
in the life of persons with severe communication disabilities. Finally, the research project presented in
[11] aimed to understand the needs for the design of a physician-patient communication visualization
system, as well as to develop a prototype.
3. Workshops with Project Team
Based on the results of the rapid review and following the advices of the neurologists, we decided
to develop the tool for tablets and use AAC. In order to establish the user interface layout, two design
workshops were organized with psychologists, psycho-educators, HCI specialists and an expert in
communication tools for people with disabilities. These allowed a number of suggestions to be made
regarding the functioning and presentation of the system.
The first workshop focused on the main features such as:
1. Starting the software;
2. Preparing a dialogue from words/pictograms;
3. Preparing sentences or word lists from existing sentences;
4. Make a request when in off mode.
This allowed for discussions about certain choices and items position, which also demonstrated the
need to make available choices that contribute to the personalization of the support system.
Various interface elements were discussed and drawings were made (See examples in Figure 2), for
instance:
• the decision of whether the application should be started in the user's ON or OFF mode, or whether
we should leave the choice to each;
• to support “prepare a dialogue” functionality, a draw was proposed focusing on the whole
sentences recorded (by default or by the person). Then the preparation of a dialogue could consist
in proposing sequences of words by following tree structures. The addition of one's own words
could be done by manual additions, downloads from the internet or by taking personal
photographs.
• One of the proposed draws shows how the user interface could look like to prepare the dialogue
from sentences, words and pictograms. This mockup shows the importance of providing recording
capabilities at any time, and that words can be added at any time. We have noted the importance
of allowing the users to choose the voice of the speech synthesizer if they cannot record their own
voice.
• In another draw, everyday sentences are pre-configured so that they can be easily adapted to the
context. If the user chooses "Hello X, how are you?", the interface suggests replacing X by names,
nicknames of people around him or her. It also suggests a follow-up so that the conversation is
dynamic and not just a closed response.
Figure 2: Examples of three very different technics for interface drawings used during the workshop
The second workshop was positioned around three main ideas:
1. We have lists of words, sentences, pictograms already recorded: how to use them?
2. We can think in the form of tasks that can be carried out by people.
3. Propose models using playful mechanisms.
These discussions sought to identify how to motivate the user with Parkinson's disease to use the
tool to prepare dialogues. The idea would be to do it in a playful way, as if it was a game. The user, as
player, gives a beginning of a sentence, ending with an important word that is missing, the system tries
to propose solutions (continuation of the sentence), and the player chooses a solution. If the word under
the end of the sentence is not in the proposals, you can go to a help, not necessarily with words, but
which can contain pictograms. It is always necessary to be able to go to a category directory to find
something. We must also be able to complete the content of this directory, if a word is missing. Scoring
and medal mechanisms common in online games can be used for users who make more dialogues,
enrich the general dialogues base, and so on.
4. Mockups Design
Based on the discussions and results of the workshop, we created mockups for the "Make a
Dialogue" functionality by presenting different options for Human-Computer Interaction (See for
example, Figure 3 and Figure 4).
(a) Landscape user interface with horizontally aligned buttons
(b) Portrait user interface with vertically aligned buttons
Figure 3: User interface for “Make a dialogue” functionality
(a) Only word (b) Only pictogram
(c) Both pictogram + word
Figure 4: Different options to support “Make a dialogue” functionality
5. Evaluation with people with Parkinson’s disease
We performed mockup evaluations with 16 volunteer persons with Parkinson's disease (8 French
and 8 Belgian). Each interview lasted on average one hour. The interviews had two objectives. The
first objective was to present the project and its progress. The second objective was to identify the
preferences on some options for Human-Computer Interaction of the users concerning:
• button placement (horizontally aligned at the top or vertically on the side);
• the positioning of the interfaces on the tablet (landscape or portrait);
• the way suggestions are presented to complete the text (text only, pictogram only or
pictogram and text);
• the type of keyboard (classic or swiftkey which presents word suggestions);
• the choice of a symbol for the identification that they are in off mode and therefore, the
choice to make a simple request (thumb down, smile, sad or text).
In addition, we sought to identify whether the chosen features would meet expectations and
suggestions for prototype development. Figure 5 shows the percentages of the different options on the
presented human-machine interaction elements. We can see that the chosen elements are:
(a) the screen presented in landscape (75%);
(b) the buttons aligned horizontally at the top (63%);
(c) the swiftkey keyboard (81%);
(d) the use of pictograms followed by word(s) for the proposals to complete the text (50%);
(e) the thumb for the indication of off mode and therefore to allow to "make a request" (44%).
Figure 5: Options on Human-Computer interaction elements
6. Conclusion
This paper reports the user-centered design of the user interfaces of a communication aid software
system for people with Parkinson 's disease. After this design, a first prototype of the system was
developed and it was evaluated by final users. A new version of the system is under development to
integrate gamification elements and personalization options.
7. Acknowledgements
ParkinsonCom project is developed with the support of the European Regional Development Fund
(Interreg V France-Wallonie-Vlaanderen) and the Agency for quality life AVIQ (l’Agence pour une
Vie de Qualité) from Wallonia, for which the authors are deeply grateful.
8. References
[1] K.M. Oliveira, E. Batselé, S. Lepreux, E. Buchet, C. Kolski, M. Boutiflat, V. Delcroix, H. Geurts,
K. Apedo, L. Dehon, H. Ezzedine, Y. Guerrier, M. Haelewyck, N. Jura, P. Pudlo, Y. Rekik.
ParkinsonCom Project: Towards a Software Communication Tool for People with Parkinson's
Disease. In: International Conference on Human-Computer Interaction Springer, 2021, pp. 418-
428.
[2] S. Lepreux, K. Apedo, K. Oliveira. Vers une conception centrée sur l'utilisateur ayant un profil
évolutif : Une étude de cas avec des personnes atteintes de la maladie de Parkinson. In: Proceedings
of the 31st Conference on l'Interaction Homme-Machine: Adjunct, ACM, 2021.
[3] NF EN ISO 9241-210 (2011-01-01) Ergonomics of human-system interaction - Part 210 : human-
centred design for interactive systems (2011).
[4] J. Whitfield, A. Reif, and A. Goberman. Voicing contrast of stop consonant production in the
speech of individuals with Parkinson disease ON and OFF dopaminergic medication. Clinical
linguistics and phonetics 32, 7, 2018, pp. 587–59.
[5] Y. Guerrier, K. Oliveira, C. Kolski, A. Lepreux, K. Apedo, V. Delcroix, H. Ezzedine. Une étude
systématique pour la conception d'un systéme d'aide à la communication pour les personnes
atteintes de la Maladie de Parkinson. In: Actes du 39ème Congrès INFORSID, INFormatique des
Organisations et Systèmes d'Information et de Décision, 2021, pp. 71-86
[6] A. C. Tricco, E. V. Langlois, S. E.Straus et al. Rapid reviews to strengthen health policy and
systems: a practical guide. World Health Organization, Geneva, 2017.
[7] S. E.Martins, P. Maldonado. Cubo: Communication System for Children with Autism Spectrum
Disorders. In: International Conference on Applied Human Factors and Ergonomics (pp. 353-365).
Springer, 2019.
[8] K. Wołk, A.Wołk, W. Glinkowski. A cross-lingual mobile medical communication system
prototype for foreigners and subjects with speech, hearing, and mental disabilities based on
pictograms. Computational and mathematical methods in medicine, 2017.
[9] C.M.A. Ilyas, K. Rodil, M. Rehm. Developing a User-Centred Communication Pad for Cognitive
and Physical Impaired People. In: Brooks, A., Brooks, E. (eds) Interactivity, Game Creation,
Design, Learning, and Innovation. Lecture Notes of the Institute for Computer Sciences, Social
Informatics and Telecommunications Engineering, vol 328. Springer, 2020.
[10] D. Grigis, M, Lazzari. Augmentative and alternative communication on tablet to help persons with
severe disabilities. In: Proceedings of the Biannual Conference of the Italian Chapter of SIGCHI
(CHItaly '13). Association for Computing Machinery, 2013, pp. 1–4.
[11] F. Lin, C. Lin, Y.J. Hsueh, C. Lee, C. Hsieh. Graphical Tools for Doctor-Patient Communication:
An App Prototype Design in Children's Pain Management. HCI, 2016.