In recent years the number of ICT-based solutions for the Ambient/Active Assisted Living (AAL) has grown continuously. Such technologies need to be validated before being used on a large scale to help people to live independently and longer in their preferred environment. However, the testing of ICT solutions to manage smart homes requires huge resources, since the tests need to be conducted for a long time, with real human inhabitants, taking into account different kind of impairments and different economical conditions. In this paper, we present the use of a 3D simulator for the AAL: as a use case, we describe how the simulator can be used to interact with a real mobile application to manage a smart home, using the app to control a “virtual smart home”.
Over the years a growing number of ICT-based solutions has been proposed to
address the main objective of the Ambient Assisted Living (AAL): to extend
the time people can live in their home independently. Researchers, companies
and end-user organizations are focusing on building smart homes, by equipping
patients’ home with sensor and actuator networks. The term “smart home”
defines a dwelling equipped with technology to monitor its inhabitants and to
ensure their independence and good health [
Unfortunately, the testing of software systems to control smart homes and to process data requires enormous resources in terms of time, work and money, since tests need to be conducted: – with real human inhabitants, – in different environmental situations, – taking into account different kinds of impairments, – under different economical capabilities and conditions.
In order to speed up the development and testing of ICT technologies and tools for the AAL, we propose the use of a virtual environment, i.e. a 3D simulator that provides interfaces to virtual sensors and actuators; such a simulator could allow to perform the testing of software solutions for the AAL; ideally, the tested software systems could be migrated in a transparent way to a real smart home at the end of the tests. 1.1
We present a mobile application to manage a smart home and we describe how
it can interact with a virtual environment, using the AAL simulator outlined
in [
In facts, the real mobile application communicates with the simulated environment (receiving values from sensors and sending commands to the actuators) by means of TCP/IP sockets, allowing the decoupling of the software development from the hardware development. Thus, the mobile application could be migrated in a real environment if the real sensors have the same interface of the simulated ones and are able to communicate via the TCP/IP protocol. 1.2
The remainder of the paper is as follows: section 2 reports related works on AAL mobile applications and other approaches to the use of virtual environments within the AAL domain. Section 3 describes the implemented mobile application and shows the interaction with our proof-of-concept simulator. Finally, section 4 draws the conclusions of this work and highlights future works. 2
The increasing success of mobile devices has relevant effects even in the
healthcare and assistance sector: entire surveys are dedicated to mobile-based based
assistive technologies and mobile devices are crucial in the trend toward more
personalized care [
Despite the potential of ICT applications in the AAL, many difficulties
encumber the testing of software systems in real home environments. Kormanyos
and Pataki [
Of course, such tasks require many resources; moreover, to collect data, the
system developers should directly follow patients (living with them) or patients
themselves should record their activities: the normal flow of actions is influenced.
Thus, authors propose a model to represent human behaviours in a simulated
environment in order to generate data for activity recognition algorithms. By
providing distinct models for the human behaviours, the environment and the
sensor networks, authors implemented a tool able to generate textual logs about
variables such as bed pressure and unwashed dishes in the sink; the tool can also
simulate different kinds of humans. Even in [
Beyond simulation for testing purposes, virtual environments are used in the
design of AAL platforms: Van’t Klooster et al. [
A real mobile application in a simulated environment Similarly to what happens in the robotics field, in which several simulation environments are available, we want to test IT systems for the AAL in a 3D virtual environment that provides APIs to the interfaces of sensors and actuators available in the market; this approach allows: – to speed up the development of software systems, by decoupling hardware from software; in facts, real tests can result in the need to modify or even redesign a component. With the simulator this process is faster and real tests can be conducted in more advanced phases; – to easily migrate software systems from the simulator to the real world.
This can be an advantage for both the development of a system and its maintenance (as the migration can be also in the opposite direction); – to execute tests in an economically sustainable way.
The addition of the 3D feature plays an important role to allow interactions, as those typical of AAL applications. Moreover it allows designers and developers to interact on the fly with the simulation environment, giving them the chance to generate unexpected events or move objects during the execution. In the following subsections we highlight: the mobile application implemented to ease the control of a smart home (3.1), the tools used to develop the simulator (3.2), the interaction between the mobile application on a real smartphone and the simulator (3.3). 3.1
The mobile application allows to manage a smart home. Thus, a smartphone becomes a real remote control, equipped with sensors, that interacts with the home environment of the assisted person: through the smartphone interface, the assisted person can manually control lights, doors, windows, temperature and more.
Figure 1 shows some screenshots of the pages of the mobile application. The first one (Figure 1a) is the main page: it allows the access to all the pages for the interaction with the listed controls. We designed the Graphical User Interface (GUI) to be user friendly, with minimal graphics and large icons, taking into account visually impaired or disabled patients.
The stylized light bulb controls the lights of the environment (Figure 1c). Figure 2 shows an example of interaction: one can turn the light on in a room by touching the icon that corresponds to that room. The interface provides an immediate feedback of the status of the lights in the home: a yellow background of the icon of a light indicates that it is turned on. The ambient light sensor allows to automatically detect the lighting and to turn the light on if needed: this kind of behaviour is present when the slider, in the light interface, is activated. The sensor can be used even to turn the flashlight on in case of unexpected blackout.
Automatic doors and windows can be controlled through the respective stylized icons (Figure 1b-d). Even in this case (Figure 1d) there is an immediate feedback of the status of the home: if the background of an icon is blue, the correspondent window is open.
Through the thermometer icon, the assisted person can set the desired temperature inside the home: intuitively, the blue and red icons decrease and increase the preferred value (Figure 1e). (a) Home page (b) Doors control
(c) Lights control (d) Windows control (e) Thermostat
(f) Emergency calls (g) Safe number call (h) Voice commands
In the SOS page (Figure 1f), the patient can quickly send emergency or familiar calls; in the second case (Figure 1g) either a normal call or a skype call can be selected.
One of the most important features, specifically addressed to visually impaired patients, is represented by the icon with the stylized microphone (Figure 1h): it allows to use the smartphone speech recognition system to send voice commands to the home. In facts, we mapped all the commands available in the interface, adding also the possibility to open/close all the windows or turn the light on/off with a single command. 3.2
For the simulation of the virtual environment we used different softwares. To represent the home environment we used Sweet Home 3D3, a free interior design application to draw the plan of a house and to arrange the furniture in a 3D
model. It allows to easily create and export in Blender4 models of domestic environments, in which the different sensors and actuators can be placed.
To implement our simulation, we used Morse5, the Modular Robots Open Simulation Engine. It is an open-source robotics simulator based on the Blender game engine. The architecture is based on components able to simulate sensors, actuators and robots; its structure is flexible, allowing to specify a level of abstraction of the simulation according to the needs, and modular because it is able to interact with any middleware used in the robotics field, without imposing a standard to which others must adapt.
Within the 3D home environment, we represented the patient with an avatar on a wheelchair equipped with sensors as described in the next subsection. 3.3
The virtual domestic environment can be manually controlled by a user through the mobile application: the communication between the simulation in Morse and the mobile application on a real smartphone uses TCP/IP sockets. This allows to simulate a real world scenario where the domestic Wi-Fi network can be used to take advantage, anywhere in the environment, of all the services offered by the application and the smart home. Each actuator and each sensor are associated to a thread in order to send commands and retrieve values.
For an accurate interaction some sensors need to be simulated. For example, the possibility to adjust the ambient temperature is essential in order to ensure the maximum comfort within the house. The simulated temperature sensor emulates a thermometer, measuring the temperature with respect to the distance
from heat sources. It defines a default temperature throughout the scenario, which is affected by local heat sources. The temperature rises exponentially when the distance between the sensor and the heat source decreases. Its equation is given by: temperature = Def aulT emperature+Ps F ireT emperature(s)∗e(−α∗distance(s))
We placed the temperature sensor on the wheelchair of the patient, in order to ensure the possibility to control the perceived temperature at any given point using a digital thermometer. One of the functionalities of the mobile application is to control the thermometer, and thus we modeled this kind of interaction.
In addition, we simulated motion sensors; beyond complex applications for activity detection and recognition, they are essential even in simple tasks, such as turning on certain lights (entrance, rooms, etc.) only when it is actually needed; similar checks can be applied also to the climate system. Since in Morse there are no motion sensors as those described, we simulated it using a SICK sensor, made available by the software. It is a laser scanner which works by generating a series of rays in predefined directions, and by using the collision system of the physics engine to detect whether any active object is found within a certain distance from the origin of the sensor. We used the simulated motion sensors to localize the assisted person inside the home. They were placed in strategic points of the house to try to get through each of them the maximum possible coverage.
Beside the simulated sensors, we used the sensor available on the smartphone, transforming it in a real remote control. The available sensors include: – ambient light sensor; – accelerometer, gyroscope and GPS; – microphone.
The ambient light sensor is able to detect changes in light: hence, we used it to automatically activate the lights in the room, inside the virtual environment, where the avatar of the assisted person is located; in case of emergency as a blackout, it can activate the flashlight of the camera. The accelerometer, gyroscope and GPS can be used for fall detection, indoor and outdoor localization of the patient’s wheelchair and accidental situation. Finally, through the use of the microphone and speech recognition on the smartphone operating system, voice commands can be sent to the system that manages the house. 4
In this paper, we described the interaction between a mobile application to manage a smart home, running on a real smartphone, and a virtual home environment, implemented within the Morse robotics simulator6. The developed mobile
http://www.youtube.com/watch?v=zXEpShRNGuo available on
youtube:
software should be easily migrated in real environments, since the interaction
with the simulator is based on TCP/IP sockets: the only requirement is that
the simulated sensors should provide the same interface as the sensors available
in the market (being able to operate in a TCP/IP network). More in general,
the used simulator is intended to develop and test also intelligent systems, able
to manage a smart home and execute plan to ensure the safety of the assisted
person, as described in [
Of course, more qualitative and quantitative tests on the simulator are needed. The objective of future work could be the design and development of a simulator specifically dedicated to the AAL: it would be an effective means to develop and test the proposed AAL solutions, simulating the interfaces to real sensors and actuators, and representing human behaviours through virtual avatars. In our vision, such a simulator should allow AAL researchers and organizations to cooperate in enabling people to live in their preferred environment as long as possible.