=Paper=
{{Paper
|id=Vol-2524/paper16
|storemode=property
|title=An intuitive hardware layout for personalized augmentative and alternative communication systems
|pdfUrl=https://ceur-ws.org/Vol-2524/paper16.pdf
|volume=Vol-2524
|authors=Francesco Davide Cascone,Giuseppe Di Gironimo,Antonio Gloria,Massimo Martorelli,Antonio Lanzotti
|dblpUrl=https://dblp.org/rec/conf/psychobit/CasconeGGML19
}}
==An intuitive hardware layout for personalized augmentative and alternative communication systems==
https://ceur-ws.org/Vol-2524/paper16.pdf
An intuitive hardware layout for personalized
Augmentative and Alternative Communication Systems
Francesco Davide Cascone1, Giuseppe Di Gironimo1, Antonio Gloria2,
Massimo Martorelli1, Antonio Lanzotti1
1 Department of Industrial Engineering, Fraunhofer JL IDEAS, University of Naples Federico
II, Naples, Italy
2 Institute of Polymers, Composites and Biomaterials, National Research Council of Italy, Na-
ples, Italy
francescodavide.cascone@unina.it
Abstract. The complexity of the interaction between user and computer can limit
usability in products. When products are aimed at individuals with disability, the
complexity increases the cognitive load and can reduce performances. The iden-
tification of interaction models and usability issues plays a role in product devel-
opment as it enables designers to reduce this complexity. Methodology to iden-
tify lacking areas in products are required and permits to correct issues leading
to an improvement of performances. A custom Augmentative and Alternative
Communication system was developed for a student of the University of Naples
Federico II. The user has complex communication needs and motor impairments
and requires a personalized device to communicate. To promote an efficient in-
teraction, hardware and software interfaces were personalized. Several studies
were conducted: a usability evaluation, determination of the learning rate and
Hardware/Software layout optimization were used to reduce the cognitive de-
mands required by the system in its functioning. In this paper the HW layout
optimization is investigated and strategies to reduce the cognitive load modifying
order and position of the sensors of the input peripherals are provided.
Keywords: Augmentative and Alternative Communication, Usability Testing,
Human-Computer Interaction.
1 Introduction
A challenge of Human-Computer Interaction (HCI) is to ensure design for all [1].
The complexity of interaction acts as a barrier in designing “usable” products [2]. The
reduction of the complexity can be a main issue in the development of interfaces [3].
As the cognitive demand increases due to poor designed products, the relationship be-
tween hardware and software interfaces should be analyzed to minimize cognitive load
and improve performance of the user. These barriers limit the overall performance of
Copyright © 2019 for this paper by its authors. Use permitted under Creative Com-
mons License Attribution 4.0 International (CC BY 4.0).
2
the user leading to an inefficient interaction [4]. To achieve this result, methodologies
to recognize interface issues, quantify performance and apply corrective actions are
required. The complexity underlying the interaction can be understood by means of
specific tools: participatory design and observational studies. In addition, a Usability
Test can provide a good understanding of the interaction model. Due to the specific
modes of interaction, interfaces aimed at individuals with disability are often personal-
ized and these tools should be tailored before being applied. Also, the improvement of
performances is an aspect related to personalization. A usability assessment can be used
to identifying barriers in products.
The project team developed an Augmentative and Alternative Communication (AAC)
System for a student of STEM of University of Naples Federico II with motor impair-
ments and speech disorder. The areas interested in the product development are sensory,
motor and cognitive. In sensory area the response time and feedbacks were analysed.
In motor area ergonomics analysis and personalization of the interface were carried out.
In cognitive area two main attributes were analysed: training and operational skills. The
level of training of the user was determined using learning curves. The operational skills
were determined through task analysis. From the experiment challenges in the field of
personalized AAC were identified: (a) personalization of the off-the-shelves products;
(b) the reduction of the gap between user’s needs and commercial offers; (c) the reduc-
tion of development times; (d) the search for barriers in the intervention; (e) the quan-
tification of times associated to prototyping, response of the user, evaluation of prod-
ucts and training.
The case study was supported by an Analytic Hierarchy Process (AHP) and Multiple-
Criteria Decision Analysis (MCDA) approach. [5]
2 Usability Evaluation and Interface Optimization
A linear additive evaluation model was assumed to identify the lacking area in which
it is possible to provide corrective actions. The Usability Index was decomposed into
three dimensions: effectiveness, efficiency and satisfaction. Each of these dimensions
were broken down into usability functions: Number of Errors (NE) and Task Comple-
tion (TC) in effectiveness dimension, Number of Operations (NO) and Time (T) in
efficiency dimension and Post-Session Ratings (PSR) in satisfaction dimension. By
recorded video tests, judgments of an expert panel and questionnaires measures for each
function were obtained. With a linear additive evaluation model, the measures can be
combined using their normalized values into an overall value to obtain weights for the
usability attributes [6]. From the analysis of data and observational studies the issues
were identified and classified into three areas (cognitive, physical and operational). For
each issue a root cause was identified. Corrective actions were provided using TRIZ-
like methodology and solutions were validated by an expert panel. From this analysis,
a typical error not directly related to spastic events or dystonia was found. This error
was defined “uncertainty” and represents an unintentional behavior that could be traced
back to the absence of intuitiveness of the hardware layout, i.e. position and order of
the sensors of the input device. Since the number of errors is a function that increases
3
operations, this error also results in an increase of the number of movements and, there-
fore, in physical effort. This function must be contained within an acceptable limit and
the design team should make the AAC-System error-tolerant [7]. To reduce errors, a
study of the interaction is required and strategies to reduce thinking activities in favor
of automatic behaviors should be considered. Furthermore, these strategies play a role
in reducing the learning time and promote learnability as it is a usability attribute that
influences the overall performance. [8]
The test system used during the experimentation (fig. 1) consisted of four switches and
one bending sensor. The switches act as a navigation set, while the bending sensor em-
ulates return key. The user moves into a grid containing symbols (fig. 2) of alphabetical,
functions, special characters and numbers type. The arrangement of the sensors onto an
arc curve avoids false pressure and it has showed to be more ergonomic than the ar-
rangement on a line. The hardware configuration proposed is showed in fig. 1. This
layout helps the body segment to move onto a specific path. The body segment used by
the user is the right arm. The actuation of sensors is made by a pressure of the finger or
the hit of the hand.
Fig. 1. The personalized AAC System described in 2.
The first and the second sensor are classified primary sensors as they emulate right
and down keys. The third and fourth are classified assistive sensors as they recovery
from errors. The fifth sensor emulates the return key (fig. 3). This layout was deter-
mined by means of observational study and task analysis. The main task consists of
four operations: (1) locate the symbols in the grid, (2) position in the row, (3) position
in the column and (4) press confirm.
Fig. 2. Schematic of the grid layout
4
With regards to the mode of input of the grid in the configuration of the home cell in
the left corner and in an error-less scenario, the layout in fig. 3 is supposed to reduce
the range of movement as the switches down/right are used in the interaction and lay
on a minimum length path for the body segment of the user. However, the proposed
layout does not verify the hypothesis of the design team as the user made a large amount
of errors during the experimentation.
Fig. 3. Schematic of the input peripherals
During the design phase a code was developed to assess the NOs of two arrays of char-
acters distributed into the grid (fig.5). The code takes as input samples of texts in the
form of dialogues. An alphabetic grid and re-ordered grid were tested. The re-ordered
grid was obtained considering the distribution of characters in fig. 5b. In the re-ordered
grid a save of operations of 70% was reached, compared to the alphabetical one. The
communication rate of the user was estimated dividing the number of characters to
compose a task-sentence by the time of observation. The mean value ranged in four to
five characters per minute. This value represents a low value for the design team and
can be improved. E.g., the user spent on average about 250 operations to compose a
sentence of 20 characters and a time of four to five minutes [6]. A re-ordered layout
could improve the communication rate at the expense of complexity of the interaction.
Fig. 4. Alphabetical and re-ordered layouts (a), distribution of characters at different sample
sizes (b)
5
In the experimental version of the system, the size of the grid was set to 5x4 with the
main characters on a first grid and access to a second grid for the latter characters as it
maintains the alphabetical order and the most frequent symbols (a, e, i, o) are on the
shortest path (respectively, NO from four to seven) (fig. 4). The input mode was set on
Home Vertex, with the Home cell on the left corner.
From the previously reported results, it appears that the re-ordered grid has greater gains
in terms of number of operations, but it is cognitively more complex for the user. An
alphabetical grid of the appropriate size provides frequent characters, vocals, on the
first line, as in fig. 2. From the analysis of path, the entry mode chosen is in the config-
uration home cell on the first alphabetic character instead of the left corner cell because
it allows the user to move in all directions of the grid and has a good saving on the
number of operations compared to other entry modes.
3 Understanding the interaction and correcting issues
It was found from the Analysis of Data obtained during the Usability Assessment
that the increment of NO, which contributes to physical effort as it is correlated to mo-
tion of the body segment of the user, increases NE. The reduction of NO can be obtained
modifying the SW interface. Therefore, both NO and NE are influenced by the HW
interface. To reduce the human error caused by a bad ordering of the sensors and to
obtain an increase of performance the HW layout should be modified consequentially.
By means of observational studies, the error modes of the user in the interaction with
the HW/SW interface were found. [9] Even if the initial layout has proven advantages
in reducing some type of error, e.g. false pressure, it can cause issues in the operational
and cognitive area.
The errors made by the user were divided into four categories: (1) blink, (2) slip, (3)
lift and (4) uncertainty. A blink error occurs in a corner position of the grid when the
user presses multiple times the same switch and it can be caused by spastic events. A
slip occurs when the user loses the target cell and goes beyond it, pressing multiple
times the same switch in a non-corner position. This error can be caused even by spas-
ticity or operational issues. The slip error can be corrected introducing a delay in the
signal acquisition that prevent the spasticity cause. The correction of slip error spastic-
ity-caused enables the experimenter to analyze other root causes in the operational area.
A lift is a raise of the arm over the bending sensor and can be corrected modifying the
position of the sensor to an ergonomic position. Uncertainty is the name given to the
wrong planning of the action. In this case, the user doesn’t identify the right sequence
of switches while performing actions, which cause unwilled actions. Also, intersecting
paths were found. To consider issues of these intersection paths that causes uncertainty,
the HW layout should be modified. In addition, the optimal layout should verify the
conditions of minimum number of errors generated and a minimum number of move-
ments and it should also improve automatic behavior. The actions the user can perform
on the system are: (a) characters selection, (b) “speak” cell selection, (c) slip correction,
(d) selection of an autocompletion cell (e) mistake correction and (f) access to the sec-
ond grid. The overall possible combinations using five switches are 5!=120. The
6
combinations are reduced aggregating the sensors into two modules in which one con-
tains a navigation set constituted by four switches and the other containing key-return
constituted by the bending sensor. The return key can be positioned on the right or left
of the first module, leading to (4!)*2=48 combinations which 24 are for right-handed
and the latter for left-handed. Once the position of the key-return module is set up,
knowing the body segment used, the number of possible HW are fixed. This set of
combinations can be reduced to six considering a “natural layout”. These six combina-
tions indicated as “natural layout” maintain natural directions. For these combinations
there is no reversal right-left and up-down (Tab. 1), regarding to a specific mode of
entry that considers one arm.
Table 1. The combination set referred to a natural layout
# First Second Third Fourth
1 Down Left Right Up
2 Down Up Left Right
3 Down Left Up Right
4 Left Down Right Up
5 Left Down Up Right
6 Left Right Down Up
From the gathered data, weights were assumed for each of these actions. Regarding
to the SW issues found, the hardware layout can be modified to obtain a reduction of
uncertainty. The actions were divided into active actions, as (a) and (b), and corrective
actions as actions to recover or correct errors, as (c) to (f). From the combination set
reported in tab. 1, the chosen combination is the number three as it can decouple move-
ment (clockwise/counterclockwise) of active and corrective actions (Tab.2).
Table 2. Actions performed on the system by the user, their weights, directions and curves
Action Weight Position Movement Shape Curve
Selection 1 P1 Clockwise Arc C1
Confirm 0.8 P1 Counterclockwise Arc C2
Autocompletion 0.3 P1-P2 Counterclockwise Line C3
Mistake correction 0.5 P2 Counterclockwise Line C4
Read 0.3 P1 Counterclockwise Line C4
Slips Horizontal 0.4 P1-P2 Both Line C5
Slips Vertical 0.5 P1-P2 Both Line C6
Second Grid 0.4 P2-P1 Counterclockwise Line C7
The actions of selection and confirm, in an error-less scenario, are performed se-
quentially. When error doesn’t occur the sequence of selection and confirm, laying on
the same curve, are performed in two steps: the first is clockwise and the second is
counterclockwise.
7
The rest positions were chosen by the design team (identifying two ergonomics posi-
tion P1 and P2) and sensors were placed on two type of curves: in the arc curve the user
performs a rotation of the body segment around the elbow, while in the line curve the
user performs a translation from position P1 to P2. The movement imposed by a rule
should increase compatibility between performing actions and automatic behavior.
At least, to guarantee the convergence of the body segment from all positions to the
return-key, the surface area of the sensor has been adequately scaled.
In fig. 5 is shown the resulting HW layout.
Fig. 5. Action paths in performing tasks
4 Conclusion
The showed HW/SW layouts are more easily to learn and recall by the user. The
decoupling of actions should prevent the user from errors as uncertainty, but tests
should be carried out confirming the feasibility.
A layout “easy to memorize” could be called “natural” as it reduces the training
associated to its use and doesn’t alter directions, even without written tag on switches.
The advantages of a natural design could be a reduction of the orientation of move-
ment (clockwise or counterclockwise), reduction of the time related to thinking and
perception, reduction of cognitive demands, imposing interaction by rule compared to
designs which rely on the intuitiveness of the user and can cause confusion due to a
wrong programming of the action.
8
The layout can have benefits also on the learnability, as a usability attribute of the
product. In the product development, learnability is an aspect which should be taken
into account as it has a positive impact in reducing time of design and experimentation.
The use of a layout easily to understand and memorize could also reduce errors as it
reduces the uncertainty of the user during the task.
Acknowledgements
This study was supported by the project INCLUDE all (INnovative methods for de-
sign and manufaCturing of personaLized interfaces for all stUDEnts) financed by the
University of Naples Federico II. A special thanks to the SiNapSI center of the Univer-
sity of Naples Federico II, center for the inclusion of all students with disability and
temporary disorders.
References
1. Dix, A.: Human–computer interaction, foundations and new paradigms. Journal of Visual
Languages & Computing 42, 122–134 (2017).
2. Pullin G., Treviranus, J., Patel R., Higginbotham J.: Designing interaction, voice, and inclu-
sion in AAC research, Augmentative and Alternative Communication, 33 (2017)
3. Dietz A., Weissling K., Griffith J., McKelvey M., Macke D.: The Impact of Interface Design
During an Initial High-Technology AAC Experience: A Collective Case Study of People
with Aphasia, Augmentative and Alternative Communication, 30 (2014)
4. Light J., McNaughton D.: Putting people first: Re-Thinking the Role of Technology in Agu-
mentative and Alternative Communication Intervention, Augmentative and Alternative
Communication 29, 299-309 (2013)
5. Lanzotti A., Martorelli M., Papa S., Sicignano G., Pepino A., Orrei S., Ciaramella S., Del
Giudice D.M., Gloria A., Gerbino S.: Participatory design of a personalized device for a
student with spastic quadriplegia at Federico II, Advances in Engineering 7, 339-349 (2018)
6. Cascone F.D., Martorelli M., Gloria A., Papa S., Lanzotti A.: Towards the development of
interfaces for students with speech disorder and motor impairments, Procedia Manufacturing
(2019)
7. Dix, A.: Human-Computer Interaction, third edition, Pearson (2003)
8. Cascone, F. D., Martorelli M., Gloria A., Papa S., Lanzotti A.: The role of learnability in the
development of interfaces in a complex case of disability; Proceedings of the Virtual Con-
cept International Workshop on Educational Innovation in Engineering and Sciences Mex-
ico, Monterrey, (2018)
9. Reason J.: Human Error, Cambridge University Press (1991)