Empowering Young Adults with Intellectual Disability to Design Smart Interactive Experiences Diego Morra1 , Giulia Cosentino2 , Mirko Gelsomini1 , Maristella Matera1 and Marco Mores3 1 Politecnico di Milano, Milan, Italy 2 Norwegian University of Science and Technology (NTNU), Trondheim, Norway 3 Fraternità e Amicizia, Milan, Italy Abstract This paper reports on a co-design toolkit that aims to support young adults with Intellectual Disability (ID) to design smart objects that can enhance situations of their everyday life. The design experience that the toolkit can support allows subjects to feel engaged and empowered in solving concrete problems; it thus favours reflection and can be used also for educational purposes. Keywords Intellectual Disability, Co-Design, Tangible UIs, Card Toolkits 1. Introduction The work presented in this paper wants to address the following research question: Can a structured co-design method, based on the adoption of physical and digital material, empower individuals with Intellectual Disability to reflect on and ideate smart outdoor experiences? To our knowledge, these aspects have scarcely been addressed in previous studies. The literature mainly reports on the adoption of combined physical-digital interfaces as a means to support learning experiences. Our work focuses instead on using this technology to make people with Intellectual Disability (ID) protagonists in the ideation of smart outdoor experiences for themselves. These consist in orchestrations of smart devices and behaviors situated outdoors, which can occur in situations users might encounter in their everyday life [1]. For this purpose, we propose COBO (COllaborative BOard), a phygital toolkit that we designed by involving a group of young adults with ID and special-education professionals of a social-care center [2]. Inspired to SNaP [3], a collaborative card-based board game addressing the co-design of smart objects, COBO introduces a paradigm that combines multimedia elements (images, animations, sounds) shown on a tablet device, and physical items (deck of cards, an interactive board, some smart objects) that are manipulated by the users. This material is used within a EMPATHY: Empowering People in Dealing with Internet of Things Ecosystems. Workshop co-located with INTERACT 2021, August 30, 2021, Bari, Italy Envelope-Open diego.morra@polimi.it (D. Morra); giulia.cosentino@ntnu.it (G. Cosentino); mirko.gelsomini@polimi.it (M. Gelsomini); maristella.matera@polimi.it (M. Matera); marco.mores@fraternitaeamicizia.it (M. Mores) Orcid 0000-0001-7275-6219 (D. Morra); 0000-0001-9223-0943 (G. Cosentino); 0000-0001-8421-6850 (M. Gelsomini); 0000-0003-0552-8624 (M. Matera) © 2021 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). CEUR Workshop Proceedings http://ceur-ws.org ISSN 1613-0073 CEUR Workshop Proceedings (CEUR-WS.org) flow of structured activities that favours users’ reflection, group discussion and collaboration. Overall, the participants are guided in the ideation of smart interactive artefacts. 2. Rationale and Background Intellectual Disability (ID) is a condition arising during the developmental period characterized by often co-occurring challenges in the cognitive, social, communicative, motor, behavioral and emotional spheres. The disorder includes deficits in intellectual functioning (e.g., reason- ing, problem solving, planning, abstract thinking, judgment, academic learning) and adaptive behavior (communication, social participation), affecting autonomy in everyday life [4]. The literature highlights how involving subjects with ID in co-design methods combined with the exploitation of adequate communication channels (manual activities, small games, interactive tasks) can lead to effective education processes [5]. The research discussed in this paper addresses the design of a co-design toolkit that was conducted in collaboration with young adults living with ID and their special-education professionals attending Fraternità & Amicizia (F&A), an accredited private non-profit organization that manages social day-care centers in Milan (Italy), offering numerous services to people with ID. The design activities conducted at F&A aimed to understand how a card-based toolkit for smart object design, SNaP, could be used to empower users with ID to design their smart objects for outdoor activities. SNaP [3] is a collaborative game, with ideas taken from traditional board game design. It originally addressed children aged from 11 to 14 years old, but it has also proved to be effective with other classes of users (e.g., university students and adults). It aims to make participants “protagonists” in the ideation and reflections concerning smart interactive experiences. The SNaP version used for the study reported in this paper focuses on experiences for enhancing outdoors activities, e.g., at the park. The material consists of three decks of cards and a game board that aim to inspire the design of augmented environment objects. During the game-play, players have the role of Designers. The game is facilitated by a moderator, with the role of Mayor. After different phases of card acquisition and discussion, all players “win” the game collaboratively; the game ends when each of them has designed at least an interactive object fulfilling his/her mission. 3. COBO COBO (COllaborative BOard) is the toolkit resulting from the design activities conducted at F&A in collaboration with 4 young adults with ID and their special-education professionals. COBO proposes the same game dynamics as SNaP, but it introduces new elements: an interactive board, showing the effect of card combinations, a Web application, running on a tablet and guiding the players in the different steps of the game, and augmented cards, equipped with a 2D code that can be read by visual recognition software embedded in the Web application. Figure 1 illustrates the components of the interactive board. Some actuator bases positioned in the upper side of the board (Figure 1.1) enable a multisensory feedback representing the effect of card combinations. Their role is to detect different plugged-in objects (elements of a park environment) that can be inserted on the top through a jack connection. For each object, the bases can generate sound, light and vibration output. Figure 1: The interactive board prototype. On the front side (Figure 1.2), the interactive board then integrates physical input devices and sensors: a button, a switch, a rotary potentiometer, a distance sensor, a brightness and a motion sensor. The interactive board also includes a housing slot for a 10-inches tablet (Figure 1.3) for the execution of the Web application. All these components are controlled by an Arduino Mega 2560 board, which is connected to the tablet through an ESP8266 WiFi Module. The external input and output components are instead directly connected to the board. The cards and the paper board have been re-designed to give a new visual identity to the game (Figure 2). In addition, on the back, each card has now an AZTEC code that can be recognized by the Web application. The Web application then includes functionality for the visual card recognition, voice-based UI, and speech recording. During the game, the players move along the board by rolling the dice and picking up cards, in order to come up with ideas of interactive experiences. Players are supported by the interactive board according to a structured flow of activities. The moderator first prepares the playground and positions the deck of cards. The role of the moderator (Major) is complemented by Virginia, an avatar/conversational agent provided by the Web application, which supports players both in learning the game rules and in the crucial phases of ideation through card combination. Virginia is depicted as a woman university researcher acting as a reference point for interactions. She can assist the players through visual Figure 2: COBO restyled cards and game board. and auditory channels, activated by pressing an always-active help button. When the player collects enough cards, she can have COBO recognizing them by simply placing their AZTEC code in front of the tablet camera. When the card code is recognized, the card name is told and its virtual representation is shown on the screen. Once a combination of 3 cards is added (input, environment and output), the user can plug the environment physical objects into the actuator bases; while interacting with buttons and sensors, the user sees in real-time a “live” version of the card combination effect, which is conveyed through the multisensory functions of the board. Players can finally voice-record the idea they had in mind. Once the registration is completed, it can be re-played and saved for future listening. In case of any failure (e.g., wrong mounting of the objects on the interactive bases) the application is able to guide the players to undertake the right actions through spoken and written recommendations. 4. Conclusion and Future Work The activities conducted to design and evaluate COBO allowed us to collect several insights, which can be found in [2]. Overall, the process helped us identify tangible and digital interactive material as a viable solution to enhance participants’ ability to understand the co-desing method and ideate. The final version of the interactive board seemed to allow the participants to enhance their self-accomplishment. Their response, which we observed during a final evaluation workshop [2], suggests that COBO could help enhance both the learning and the creativity process. Our future work will focus on systematically assessing the observed benefits through user studies involving a larger sample of users and in a longer period. A long-term goal could be to investigate the use of COBO to help people with ID improve their communication and logic skills (e.g., [6]), and to enhance their creative process, which is usually lacking. References [1] C. Ardito, P. Buono, G. Desolda, M. Matera, From smart objects to smart experiences: An end-user development approach, Int. J. Hum. Comput. Stud. 114 (2018) 51–68. URL: https://doi.org/10.1016/j.ijhcs.2017.12.002. doi:10.1016/j.ijhcs.2017.12.002 . [2] G. Cosentino, D. Morra, M. Gelsomini, M. Matera, M. Mores, Cobo: A card-based toolkit for co-designingsmart outdoor experiences with people withintellectual disability, in: Proc. of Interact 2021, Bari, Italy, September 2021, 2021. [3] R. Gennari, M. Matera, A. Melonio, E. Roumelioti, A Board-Game for Co-Designing Smart Nature Environments in Workshops with Children, in: End-User Development - 7th International Symposium, IS-EUD 2019, Hatfield, UK, July 10-12, 2019, Proceedings, is-EUD 2019, Springer, 2019, pp. 132–148. doi:10.1007/978- 3- 030- 24781- 2\_9 . [4] American Psychiatric Association, Diagnostic and statistical manual of mental disorders (DSM-5), American Psychiatric Pub., 2013. [5] L. Sitbon, S. Farhin, Co-designing interactive applications with adults with intellectual disability: a case study, in: Proc. of the 29th Australian conference on Computer-Human Interaction, 2017, pp. 487–491. [6] K. Ellis, E. Dao, O. Smith, S. Lindsay, P. Olivier, Tapeblocks: A making toolkit for people living with intellectual disabilities, in: Proc. of CHI 2021, CHI ’21, ACM, New York, NY, USA, 2021. URL: https://doi.org/10.1145/3411764.3445647.