=Paper=
{{Paper
|id=Vol-3053/sample-7col
|storemode=property
|title=Empowering Young Adults with Intellectual Disability to Design Smart Interactive Experiences
|pdfUrl=https://ceur-ws.org/Vol-3053/paper_6.pdf
|volume=Vol-3053
|authors=Diego Morra,Giulia Cosentino,Mirko Gelsomini,Maristella Matera,Marco Mores
|dblpUrl=https://dblp.org/rec/conf/interact/MorraCGMM21
}}
==Empowering Young Adults with Intellectual Disability to Design Smart Interactive Experiences==
https://ceur-ws.org/Vol-3053/paper_6.pdf
Empowering Young Adults with Intellectual Disability
to Design Smart Interactive Experiences
Diego Morra1 , Giulia Cosentino2 , Mirko Gelsomini1 , Maristella Matera1 and
Marco Mores3
1
Politecnico di Milano, Milan, Italy
2
Norwegian University of Science and Technology (NTNU), Trondheim, Norway
3
Fraternità e Amicizia, Milan, Italy
Abstract
This paper reports on a co-design toolkit that aims to support young adults with Intellectual Disability
(ID) to design smart objects that can enhance situations of their everyday life. The design experience that
the toolkit can support allows subjects to feel engaged and empowered in solving concrete problems; it
thus favours reflection and can be used also for educational purposes.
Keywords
Intellectual Disability, Co-Design, Tangible UIs, Card Toolkits
1. Introduction
The work presented in this paper wants to address the following research question: Can a
structured co-design method, based on the adoption of physical and digital material, empower
individuals with Intellectual Disability to reflect on and ideate smart outdoor experiences? To
our knowledge, these aspects have scarcely been addressed in previous studies. The literature
mainly reports on the adoption of combined physical-digital interfaces as a means to support
learning experiences. Our work focuses instead on using this technology to make people
with Intellectual Disability (ID) protagonists in the ideation of smart outdoor experiences for
themselves. These consist in orchestrations of smart devices and behaviors situated outdoors,
which can occur in situations users might encounter in their everyday life [1].
For this purpose, we propose COBO (COllaborative BOard), a phygital toolkit that we designed
by involving a group of young adults with ID and special-education professionals of a social-care
center [2]. Inspired to SNaP [3], a collaborative card-based board game addressing the co-design
of smart objects, COBO introduces a paradigm that combines multimedia elements (images,
animations, sounds) shown on a tablet device, and physical items (deck of cards, an interactive
board, some smart objects) that are manipulated by the users. This material is used within a
EMPATHY: Empowering People in Dealing with Internet of Things Ecosystems. Workshop co-located with INTERACT
2021, August 30, 2021, Bari, Italy
Envelope-Open diego.morra@polimi.it (D. Morra); giulia.cosentino@ntnu.it (G. Cosentino); mirko.gelsomini@polimi.it
(M. Gelsomini); maristella.matera@polimi.it (M. Matera); marco.mores@fraternitaeamicizia.it (M. Mores)
Orcid 0000-0001-7275-6219 (D. Morra); 0000-0001-9223-0943 (G. Cosentino); 0000-0001-8421-6850 (M. Gelsomini);
0000-0003-0552-8624 (M. Matera)
© 2021 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).
CEUR
Workshop
Proceedings
http://ceur-ws.org
ISSN 1613-0073
CEUR Workshop Proceedings (CEUR-WS.org)
flow of structured activities that favours users’ reflection, group discussion and collaboration.
Overall, the participants are guided in the ideation of smart interactive artefacts.
2. Rationale and Background
Intellectual Disability (ID) is a condition arising during the developmental period characterized
by often co-occurring challenges in the cognitive, social, communicative, motor, behavioral
and emotional spheres. The disorder includes deficits in intellectual functioning (e.g., reason-
ing, problem solving, planning, abstract thinking, judgment, academic learning) and adaptive
behavior (communication, social participation), affecting autonomy in everyday life [4].
The literature highlights how involving subjects with ID in co-design methods combined
with the exploitation of adequate communication channels (manual activities, small games,
interactive tasks) can lead to effective education processes [5]. The research discussed in this
paper addresses the design of a co-design toolkit that was conducted in collaboration with young
adults living with ID and their special-education professionals attending Fraternità & Amicizia
(F&A), an accredited private non-profit organization that manages social day-care centers in
Milan (Italy), offering numerous services to people with ID. The design activities conducted at
F&A aimed to understand how a card-based toolkit for smart object design, SNaP, could be used
to empower users with ID to design their smart objects for outdoor activities. SNaP [3] is a
collaborative game, with ideas taken from traditional board game design. It originally addressed
children aged from 11 to 14 years old, but it has also proved to be effective with other classes of
users (e.g., university students and adults). It aims to make participants “protagonists” in the
ideation and reflections concerning smart interactive experiences. The SNaP version used for
the study reported in this paper focuses on experiences for enhancing outdoors activities, e.g.,
at the park. The material consists of three decks of cards and a game board that aim to inspire
the design of augmented environment objects. During the game-play, players have the role
of Designers. The game is facilitated by a moderator, with the role of Mayor. After different
phases of card acquisition and discussion, all players “win” the game collaboratively; the game
ends when each of them has designed at least an interactive object fulfilling his/her mission.
3. COBO
COBO (COllaborative BOard) is the toolkit resulting from the design activities conducted at F&A
in collaboration with 4 young adults with ID and their special-education professionals. COBO
proposes the same game dynamics as SNaP, but it introduces new elements: an interactive board,
showing the effect of card combinations, a Web application, running on a tablet and guiding the
players in the different steps of the game, and augmented cards, equipped with a 2D code that
can be read by visual recognition software embedded in the Web application.
Figure 1 illustrates the components of the interactive board. Some actuator bases positioned
in the upper side of the board (Figure 1.1) enable a multisensory feedback representing the
effect of card combinations. Their role is to detect different plugged-in objects (elements of a
park environment) that can be inserted on the top through a jack connection. For each object,
the bases can generate sound, light and vibration output.
Figure 1: The interactive board prototype.
On the front side (Figure 1.2), the interactive board then integrates physical input devices and
sensors: a button, a switch, a rotary potentiometer, a distance sensor, a brightness and a motion
sensor. The interactive board also includes a housing slot for a 10-inches tablet (Figure 1.3) for
the execution of the Web application. All these components are controlled by an Arduino Mega
2560 board, which is connected to the tablet through an ESP8266 WiFi Module. The external
input and output components are instead directly connected to the board.
The cards and the paper board have been re-designed to give a new visual identity to the game
(Figure 2). In addition, on the back, each card has now an AZTEC code that can be recognized
by the Web application.
The Web application then includes functionality for the visual card recognition, voice-based
UI, and speech recording.
During the game, the players move along the board by rolling the dice and picking up cards, in
order to come up with ideas of interactive experiences. Players are supported by the interactive
board according to a structured flow of activities.
The moderator first prepares the playground and positions the deck of cards. The role of
the moderator (Major) is complemented by Virginia, an avatar/conversational agent provided
by the Web application, which supports players both in learning the game rules and in the
crucial phases of ideation through card combination. Virginia is depicted as a woman university
researcher acting as a reference point for interactions. She can assist the players through visual
Figure 2: COBO restyled cards and game board.
and auditory channels, activated by pressing an always-active help button.
When the player collects enough cards, she can have COBO recognizing them by simply
placing their AZTEC code in front of the tablet camera. When the card code is recognized, the
card name is told and its virtual representation is shown on the screen.
Once a combination of 3 cards is added (input, environment and output), the user can plug
the environment physical objects into the actuator bases; while interacting with buttons and
sensors, the user sees in real-time a “live” version of the card combination effect, which is
conveyed through the multisensory functions of the board.
Players can finally voice-record the idea they had in mind. Once the registration is completed,
it can be re-played and saved for future listening. In case of any failure (e.g., wrong mounting
of the objects on the interactive bases) the application is able to guide the players to undertake
the right actions through spoken and written recommendations.
4. Conclusion and Future Work
The activities conducted to design and evaluate COBO allowed us to collect several insights,
which can be found in [2]. Overall, the process helped us identify tangible and digital interactive
material as a viable solution to enhance participants’ ability to understand the co-desing method
and ideate. The final version of the interactive board seemed to allow the participants to
enhance their self-accomplishment. Their response, which we observed during a final evaluation
workshop [2], suggests that COBO could help enhance both the learning and the creativity
process. Our future work will focus on systematically assessing the observed benefits through
user studies involving a larger sample of users and in a longer period. A long-term goal could
be to investigate the use of COBO to help people with ID improve their communication and
logic skills (e.g., [6]), and to enhance their creative process, which is usually lacking.
References
[1] C. Ardito, P. Buono, G. Desolda, M. Matera, From smart objects to smart experiences:
An end-user development approach, Int. J. Hum. Comput. Stud. 114 (2018) 51–68. URL:
https://doi.org/10.1016/j.ijhcs.2017.12.002. doi:10.1016/j.ijhcs.2017.12.002 .
[2] G. Cosentino, D. Morra, M. Gelsomini, M. Matera, M. Mores, Cobo: A card-based toolkit
for co-designingsmart outdoor experiences with people withintellectual disability, in: Proc.
of Interact 2021, Bari, Italy, September 2021, 2021.
[3] R. Gennari, M. Matera, A. Melonio, E. Roumelioti, A Board-Game for Co-Designing Smart
Nature Environments in Workshops with Children, in: End-User Development - 7th
International Symposium, IS-EUD 2019, Hatfield, UK, July 10-12, 2019, Proceedings, is-EUD
2019, Springer, 2019, pp. 132–148. doi:10.1007/978- 3- 030- 24781- 2\_9 .
[4] American Psychiatric Association, Diagnostic and statistical manual of mental disorders
(DSM-5), American Psychiatric Pub., 2013.
[5] L. Sitbon, S. Farhin, Co-designing interactive applications with adults with intellectual
disability: a case study, in: Proc. of the 29th Australian conference on Computer-Human
Interaction, 2017, pp. 487–491.
[6] K. Ellis, E. Dao, O. Smith, S. Lindsay, P. Olivier, Tapeblocks: A making toolkit for people
living with intellectual disabilities, in: Proc. of CHI 2021, CHI ’21, ACM, New York, NY,
USA, 2021. URL: https://doi.org/10.1145/3411764.3445647.