=Paper=
{{Paper
|id=Vol-2702/sample-6col
|storemode=property
|title=Empowering caregivers in multisensory smart spaces for education and therapy
|pdfUrl=https://ceur-ws.org/Vol-2702/EMPATHY_2020_paper_6.pdf
|volume=Vol-2702
|authors=Giulia Cosentino,Mattia Gianotti,Fabiano Riccardi,Franca Garzotto
|dblpUrl=https://dblp.org/rec/conf/avi/CosentinoGRG20
}}
==Empowering caregivers in multisensory smart spaces for education and therapy==
https://ceur-ws.org/Vol-2702/EMPATHY_2020_paper_6.pdf
Empowering caregivers in multisensory smart spaces for
education and therapy
Giulia Cosentinoa , Mattia Gianottia , Fabiano Riccardia and Franca Garzottoa
a
Politecnico di Milano, Milan, Italy
Abstract
Interactive Smart Spaces (ISSs) are physical installations equipped with digitally enriched embedded devices
that can sense the users’ presence and actions and generate stimulation for multiple senses. Our research fo-
cuses on ISSs for education and rehabilitation. In these domains, there is a strong need of tuning and controlling
the User Experience to address the specific needs of each user or user group, a process that, in principle, should
be performed by caregivers and in real-time. Nowadays, several examples of ISSs exist but they do not provide
control interfaces that are powerful for customization purposes yet simple enough for caregivers, who typically
have a limited technological know-how.
Keywords
Smart space, User Experience Design, Caregivers Empowerment, Grafical User Interface,
1. Introduction
The increasing debate about personalized and multi-sensory educational processes, arisen in the lit-
erature in the latest years, pointed out the noteworthy potentialities that sensory and synesthetic
experience can have in helping children, especially those with special educational needs, to improve
their abilities. Most multi-sensory approaches take shape in dedicated spaces called "Interactive Smart
Spaces" (ISSs) - rooms equipped with items that provide gentle stimulation of different senses while of-
fering a non-threatening, relaxing, but also an educational, environment. An ISS is an inter-connected
structure grounded on the theories of embodied cognition and sensory integration, on the develop-
ment of cognitive skills such as mental imagery and implicit memory, reasoning and problem solving
sometimes through the concept of "playful learning" [1, 2]. We developed the Magic Room (see Fig-
ures 1), an innovative, unconventional IoT enabled ISS, evolution of a previous work called P3S [5].
The Magic Room is a room-size interactive MSE that enables new forms of playful interventions for
children. This installation has been designed to let children with NDD and more in general children
with Special Education Needs in the school environment experience playful and learning activities
improving their quality of life and inclusion in the society through playful learning experiences. The
Magic Room has been designed to transform almost any room in a digitally semi-autonomous and
affordable MSE if compared to commercially available solutions. Magic Room is easily extensible in
quantity and category of smart appliances available in the environment, available stimuli that can be
detected or provided by the system and experiences available to the children and caregiver in terms
of content or mechanics of interaction.
EMPATHY: Empowering People in Dealing with Internet of Things Ecosystems. Workshop co-located with AVI 2020, Island of
Ischia, Italy
email: giulia.cosentino@polimi.it (G. Cosentino); mattia.gianotti@polimi.it (M. Gianotti); fabiano.riccardi.polimi.it (F.
Riccardi); franca.garzotto@polimi.it (F. Garzotto)
orcid: 0000-0002-8560-4924 (G. Cosentino); 0000-0001-6035-3367 (M. Gianotti); 0000-0001-5510-3240 (F. Riccardi);
0000-0003-4905-7166 (F. Garzotto)
© 2020 Copyright for this paper by its authors.
Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).
CEUR
Workshop
Proceedings
http://ceur-ws.org
ISSN 1613-0073 CEUR Workshop Proceedings (CEUR-WS.org)
Figure 1: Magic Room
Several factors influence children with disability environment, one of the most important is the
caregiver presence: a caregiver should promote autonomy (all activities that an individual does through-
out a day that are necessary and/or meaningful to that person)[3]. Using interactive technologies such
as ISS is considered an approach that does not replace current therapies, but can support caregivers
in their work with children. Given the novelty of these technologies, caregivers have an even more
important role as they are required to manage the children’s experience in the ISS. The greater the
understanding of the context where the child is inserted, the more caregivers are motivated. The total
delegation of the system, instead, can cause pressure on the caregivers, who are therefore less moti-
vated. To enable the caregivers to use the ISS, it is necessary to build an accessible and scalable tool
that allows them to easily take advantage of all the possibilities that the technology offers. Usually,
the absence of an easy to use and ecologic control is one of the most relevant causes that could lead
to the non-use of the ISS, leaving an environment designed for social usage in need of high skilled
technicians.
2. Related work
Researchers have already developed different solutions of multi-sensory environments, such as Snoeze-
len, MEDIATE, Lands of fog, and P3S. Snoezelen offers a relaxed atmosphere with pleasant surround-
ings, soothing sounds, captivating aromas, tactile experiences, massage and vibration, vibrosonic sen-
sations, and gentle movement [4]. MEDIATE stimulates children through visual, tactile, and aural
channels and allows them to express themselves through body movements, but does not support in-
teraction with objects [5]. Lands of Fog is a full-body interaction system based on a projection on
the floor where children can explore and discover unique characters, objects, and events [6]. In P3S
(Playful Supervised Smart Spaces), smart objects are integrated with interactive multimedia content
immersed in physical space, through light effects on a "luminous carpet "and animations (stories and
games) projected on the wall and the floor [7].
MEDIATE [5],Snoezelen [4] and Lands of Fog [6] do not present any type of control interface,
the system is independently managed. P3S [7] allow the caregiver a certain level of configuration
and control. Most of these systems leave the educator in a pure observer role or acting on physical
switches to change the state of the environment, and the children often explore the environment
freely.
Figure 2: Caregivers’ tablet interface
3. The Customization Tool
In close cooperation with 20 teachers from two primary schools in Milan and 10 psychologists from
two local therapeutic centers, we did a workshop to define the user experience in the Magic Room. The
co-design with the experts aimed at increasing user satisfaction by improving usability and ease of use
provided in the interaction between the caregivers and the system. During the workshop, we wanted
to understand their technology knowledge level and how they imagined children experience related
to educational needs with particular attention with children’s with special needs and, therefore, the
identification of specific requirements to be addressed in the project.
We opted to design and develop a solution that allows teachers and therapists to use the room
intuitively via tablet interface (see Figures 2). The system provides a set of customizable Activities each
one devoted to a subset of specific purposes, among which the caregiver can choose and personalize.
Activities can be organized in playlists, called Experiences. The system is mainly autonomous during
its execution, letting the caregiver spend quality time with the pupils and not worrying about the
system management: the educator is in charge only on deciding which game to play, to tailor it
according to the situation and children, and then start it. However, we decided to let the caregiver
the possibility to control the Activity in real-time and to manage the turn-taking. To perform all such
tasks the caregivers’ web interface is divided into three key moments (see Figures 3 and 4): Create,
Play and Live.
In the Create, educators can customize the activities’ parameters and save them for future reuse
and or pack a set of those in an Experience in advance. In Play, they can select which Activities or
Experiences to do in the session and eventually which player may participate. Additionally, before
playing, the educator can modify the Activity’s parameters to tailor them to the children’s needs
better. In Live, they can control the flow of the Activity during execution.
4. The Authoring Tool
The previously described Create allows to personalize some parameters such as level difficulty, en-
abled stimuli, themes, but it does not allow the structural modifications of Activities. Create can be
considered as a Customization Tool, but we observed that in some situations, caregivers would obtain
a wider expressiveness power. This increases the challenges because there is the need to find the
appropriate trade-off between expressiveness and an intuitive interface. Observing the normal rou-
Figure 3: Customization tool (Play)
Figure 4: Customization tool (Create)
tine of caregiver, we had observed that storytelling is still widely used and that the amount of know
story is huge (i.e., all caregivers can tell different stories to their pupils). For these reasons, we decide
to design an Authoring Tool showed in Figure 5 to create stories benefiting of all the multi-sensory
interactions and stimuli available in our Smart Space. The development of this tool required the con-
ceptualization of the main aspects of a storytelling activity (Figure 6). It leads the design of the user
interface and establishing a sound vocabulary for the caregivers.
The Magic Room is able to provide Stimuli and sense Events derived from user interaction and
system triggers. A Stimulus is composed of a type and a set of parameters to personalize it (i.e., the
light’s color). Examples of stimulus are: showing multimedia content on the projectors in a specific
position, turning on the bubble machine, controlling a LED in the fin of a smart dolphin. The stimulus
can be combined in serial or parallel. A delay function is available to adjust the flow of the stimuli.
An Event is described by the type of interaction required, the locus of the interaction (in terms of,
possibly, both of position in the room or sensible area over a smart object), and a condition to be
satisfied on the interaction.
A Story is composed by a sequence of Scenes. A Scene represents a relevant and self-contained
moment of the storytelling. The Scene can be arranged accordingly to the traditional peculiarity of
flows, including branches and loops: this enables interactive stories, with different paths and endings.
Figure 5: Authoring tool
Figure 6: Model describing a storytelling Activity.
Each scene is composed of three steps: Opening, Acting, and Closing. Opening describes a set of Stimuli
that introduce the Scene to the children. Acting is the central part of the Scene, and it is composed of
Rules. A Rule consists of an Event and a Reward. When a Rule is triggered, the others are deactivated.
A Reward is provided only at the completion of the corresponding Event before the Closing and is
comprised of Stimuli. Finally, each Event is associated with a "flow instruction," which determines
the next Scene. Closing, likewise Opening, describes final stimuli the children receive before the next
Scene. Closings are independent of the triggered Event.
Based on this model, a graphical application has been implemented to support the authoring of
stories for the Magic Room, without requiring any programming skills. Once the story has been
defined, the system generates a machine-readable code for the activity engine, which can interpret
and execute the story’s specifications.
5. Conclusion and Future Work
The Authoring Tool described in this paper has been designed as an evolution of the previous gener-
ation of the Magic Room [7], in order to expand the expressive power of the newer environment. We
are now planning to perform a Usability study to assess the actual effectiveness of the tool with respect
to the background and characteristics of the caregivers who need to configure the Magic Room.
References
[1] M. Wilson, Six views of embodied cognition, Psychonomic bulletin & review 9 (2002) 625–636.
[2] M. Resnick, Playful learning and creative societies, Education Update 8 (2003) 1–2.
[3] M. Stevenson, The influences on the play of infants and toddlers, The ecological context of
children’s play (1989) 84–103.
[4] G. Lancioni, A. Cuvo, M. O’reilly, Snoezelen: an overview of research with people with develop-
mental disabilities and dementia, Disability and rehabilitation 24 (2002) 175–184.
[5] N. Pares, P. Masri, G. Van Wolferen, C. Creed, Achieving dialogue with children with severe
autism in an adaptive multisensory interaction: the" mediate" project, IEEE Transactions on
Visualization and Computer Graphics 11 (2005) 734–743.
[6] J. Mora-Guiard, C. Crowell, N. Pares, P. Heaton, Lands of fog: helping children with autism in
social interaction through a full-body interactive experience, in: Proceedings of the the 15th
international conference on interaction design and children, 2016, pp. 262–274.
[7] F. Garzotto, M. Gelsomini, Magic room: A smart space for children with neurodevelopmental
disorder, IEEE Pervasive Computing 17 (2018) 38–48.