Interactive Smart Spaces (ISSs) are physical installations equipped with digitally enriched embedded devices that can sense the users' presence and actions and generate stimulation for multiple senses. Our research focuses on ISSs for education and rehabilitation. In these domains, there is a strong need of tuning and controlling the User Experience to address the specific needs of each user or user group, a process that, in principle, should be performed by caregivers and in real-time. Nowadays, several examples of ISSs exist but they do not provide control interfaces that are powerful for customization purposes yet simple enough for caregivers, who typically have a limited technological know-how.
Several factors influence children with disability environment, one of the most important is the
caregiver presence: a caregiver should promote autonomy (all activities that an individual does
throughout a day that are necessary and/or meaningful to that person)[
Researchers have already developed diferent solutions of multi-sensory environments, such as
Snoezelen, MEDIATE, Lands of fog, and P3S. Snoezelen ofers a relaxed atmosphere with pleasant
surroundings, soothing sounds, captivating aromas, tactile experiences, massage and vibration, vibrosonic
sensations, and gentle movement [
MEDIATE [
In close cooperation with 20 teachers from two primary schools in Milan and 10 psychologists from two local therapeutic centers, we did a workshop to define the user experience in the Magic Room. The co-design with the experts aimed at increasing user satisfaction by improving usability and ease of use provided in the interaction between the caregivers and the system. During the workshop, we wanted to understand their technology knowledge level and how they imagined children experience related to educational needs with particular attention with children’s with special needs and, therefore, the identification of specific requirements to be addressed in the project.
We opted to design and develop a solution that allows teachers and therapists to use the room intuitively via tablet interface (see Figures 2). The system provides a set of customizable Activities each one devoted to a subset of specific purposes, among which the caregiver can choose and personalize. Activities can be organized in playlists, called Experiences. The system is mainly autonomous during its execution, letting the caregiver spend quality time with the pupils and not worrying about the system management: the educator is in charge only on deciding which game to play, to tailor it according to the situation and children, and then start it. However, we decided to let the caregiver the possibility to control the Activity in real-time and to manage the turn-taking. To perform all such tasks the caregivers’ web interface is divided into three key moments (see Figures 3 and 4): Create, Play and Live.
In the Create, educators can customize the activities’ parameters and save them for future reuse and or pack a set of those in an Experience in advance. In Play, they can select which Activities or Experiences to do in the session and eventually which player may participate. Additionally, before playing, the educator can modify the Activity’s parameters to tailor them to the children’s needs better. In Live, they can control the flow of the Activity during execution.
The previously described Create allows to personalize some parameters such as level dificulty, enabled stimuli, themes, but it does not allow the structural modifications of Activities. Create can be considered as a Customization Tool, but we observed that in some situations, caregivers would obtain a wider expressiveness power. This increases the challenges because there is the need to find the appropriate trade-of between expressiveness and an intuitive interface. Observing the normal routine of caregiver, we had observed that storytelling is still widely used and that the amount of know story is huge (i.e., all caregivers can tell diferent stories to their pupils). For these reasons, we decide to design an Authoring Tool showed in Figure 5 to create stories benefiting of all the multi-sensory interactions and stimuli available in our Smart Space. The development of this tool required the conceptualization of the main aspects of a storytelling activity (Figure 6). It leads the design of the user interface and establishing a sound vocabulary for the caregivers.
The Magic Room is able to provide Stimuli and sense Events derived from user interaction and system triggers. A Stimulus is composed of a type and a set of parameters to personalize it (i.e., the light’s color). Examples of stimulus are: showing multimedia content on the projectors in a specific position, turning on the bubble machine, controlling a LED in the fin of a smart dolphin. The stimulus can be combined in serial or parallel. A delay function is available to adjust the flow of the stimuli. An Event is described by the type of interaction required, the locus of the interaction (in terms of, possibly, both of position in the room or sensible area over a smart object), and a condition to be satisfied on the interaction.
A Story is composed by a sequence of Scenes. A Scene represents a relevant and self-contained moment of the storytelling. The Scene can be arranged accordingly to the traditional peculiarity of lfows, including branches and loops: this enables interactive stories, with diferent paths and endings.
Each scene is composed of three steps: Opening, Acting, and Closing. Opening describes a set of Stimuli that introduce the Scene to the children. Acting is the central part of the Scene, and it is composed of Rules. A Rule consists of an Event and a Reward. When a Rule is triggered, the others are deactivated. A Reward is provided only at the completion of the corresponding Event before the Closing and is comprised of Stimuli. Finally, each Event is associated with a "flow instruction," which determines the next Scene. Closing, likewise Opening, describes final stimuli the children receive before the next Scene. Closings are independent of the triggered Event.
Based on this model, a graphical application has been implemented to support the authoring of stories for the Magic Room, without requiring any programming skills. Once the story has been defined, the system generates a machine-readable code for the activity engine, which can interpret and execute the story’s specifications.
The Authoring Tool described in this paper has been designed as an evolution of the previous
generation of the Magic Room [