Using 3D simulators for the Ambient Assisted Living

Abstract

Ambient Assisted Living (AAL) and Ambient Intelligence (AmI) aim at building safe, smart and interconnected environments around people. Nevertheless, the testing phase of intelligent software systems managing smart homes, in real environments, requires an enormous effort in terms of time, work and money. With this paper, we discuss the possibility to use 3D virtual environments during the developing and preliminary tests of such systems, as it happens in robotics with 3D simulators. Even if this approach cannot totally replace the need of real user trials, it can speed up the implementation of prototypes, decoupling software from hardware. We also present a brief description of a proof-of-concept, showing some of the benefits described.

Introduction

– with real human inhabitants with different kinds of impairments; – in different environmental situations; – taking into account diverse economical conditions.

Such trials will be also extremely expensive, especially for the development of software prototypes that would require the hardware for a great number of tests; furthermore the result of this effort could be the need to modify or redesign some components (and thus to repeat tests).

A virtual 3D environment, simulating an interface with necessary sensors and actuators could be an effective tool to develop, simulate and test intelligent systems with the aim to extend the time people can (autonomously) live in their preferred environment.

This paper discusses the possibility to use 3D virtual environments and simulators for the AAL, analysing challenges and potential advantages. Using virtual environments to accurately reproduce real environments would allow a partial decoupling of the software development from the hardware development. Moreover it would improve the economical sustainability of the development of software to control smart homes, speeding up the implementation of prototypes and allowing the use of real environments only for release candidate versions. Finally it would provide an environment where researchers in the AAL could easily collaborate and combine their findings.

The paper is structured as follows: section 2 lists some current applications of virtual words in the AAL domain; section 3 discusses the challenges in the use of virtual environments to test intelligent software systems for the AAL; section 4 describes the implementation of a preliminary proof-of-concept, based on a robotics simulator; finally, section 5 concludes the paper, suggesting future research directions. 2

Related Works

Most of the current applications of virtual worlds in the AAL are addressed to motor and cognitive rehabilitation [ 4 ] and to improve social inclusion [ 5 ]. In particular serious gaming has been widely adopted in rehabilitation. Indeed, typical game features (as providing challenges and goals, stimulating curiosity, cooperation, competition [ 6 ]) enhance user engagement and intrinsic motivation: thus serious games can support users in developing their skills, in learning and in experiencing situations that are impossible (e.g. for economical reasons) in the real world [ 7, 8 ]. For these reasons, virtual reality and serious games are also used in therapies for pain management [ 9 ]. Beyond rehabilitation, a promising direction is the use of serious gaming for the acceptance of facility automation and smart homes [ 10 ]. Moreover the use of Interactive Scenario Visualization (ISV) can lead to the clarification of system functionalities, as well as to gain stakeholders’ feedback [ 11 ], especially in the design phase. 3

3D Virtual Environments for the AAL

The use of 3D virtual environments to develop intelligent software systems to manage virtual homes is inspired by the robotics field, where a relevant number of 3D software simulators is available. The basic concept is to use a 3D simulator that exposes the interfaces to sensors and actuators (through suitable libraries) available in the market and that allows to create a virtual home environment: here, intelligent software systems could be tested to evaluate their behaviour, even when facing unexpected events. Ideally, the intelligent systems should be migrated transparently in a real environment with the sensors and actuators previously simulated.

To this point, one might argue that there is no need to use the 3D for AAL simulations. Nevertheless, the 3D feature gives not only the involvement typical of graphical effects, but should provide to system developers the possibility to interact directly with the simulations, for example moving objects on the fly. With a reliable physics engine (as those typical of computer games), developers can simulate unexpected events and thus understand if the system behaves correctly.

In a complete 3D simulator for the AAL, also human behaviours should be represented: the involved research topics are similar to the inclusion of human reasoning mechanism in computer games and serious games, by exploiting the results of the Artificial Intelligence (AI) field [ 12 ]: – the selection of the suitable AI approach to model and imitate human behaviours especially taking into account the need of human like responsiveness and communication. – The influence of the adoption of strong AI techniques to imitate human reasoning on the simulator design, since a well identified problem in computer game design is the introduction of the AI in the last phases. – The definition of suitable goals and metrics to test intelligent software systems for the AAL.

A promising direction is the design and implementation of virtual characters as BDI agents (see, for instance, [ 13 ] for the development of BDI based NonPlayer Characters (NPC) in computer games). Although this approach is ideal to implement virtual characters capable of carrying out long-term autonomous actions, it leads to several sub-problems: balancing between proactive and reactive behaviours, scheduling properly goals on the basis of the application domain, representing the environment in a symbolic manner and translating this representation in the interaction with the simulator engine.

To obtain the advantages described in the introduction section, a clear requirements analysis (as suggested in [ 14 ]) is needed: firstly system developers, as the prominent users of 3D simulators, and their needs have to be taken into account; moreover patients, their relatives, physicians and health operators with their needs about AAL systems and their personal experience may give a crucial contribution in the identification of the simulation scenarios. 4

A proof-of-concept example

To show some of the benefits, we implemented a proof-of-concept, using the multi-agent expert system described in [ 15 ]. To create the 3D environment, we used the outdoor multi-robot simulators Gazebo [ 16 ] and Morse [ 17 ]. Figure 1 shows the virtual domestic environment composed by four rooms.

Thanks to the CameraSensor and the RaySensor classes provided by Gazebo, we simulated two scenarios to test the behaviour of the multi-agent system controlling the home: in the first the expert system has to aerate the rooms by opening the windows, but keeping closed the one in the room where the assisted person actually stays; moreover it has to close and open windows coherently with the movements of the patient, using data provided by ray sensors. In the second scenario the system has to detect the fall of the patient, sending an alarm. To implement these scenarios, we added some agents to the architecture in [ 15 ]: two Ambient Agents (the Gateway and the Camera Agents, to control the ray and the camera sensors) and two Actuator Agents (the WindowActuator and the CameraActuator Agents to send commands to the windows and the video cameras in the environment). We carried out tests also using the Morse simulator. We added also a wheelchair, temperature sensors, gas detectors and light sensors. We tested the control of the home through a mobile application using both touch and vocal commands1.

Beyond the implemented scenarios, the communication between the software agents and the sensors in the 3D environment highlights the potential of the simulations: being based on the TCP/IP protocol, each agent creates a socket channel to send commands to the sensors. Thus, if virtual sensors provide the same interface as off-the-shelf sensors (as those implemented in Gazebo and in similar robotics simulators), the developed agents can be easily migrated in real environments. This is an advantage also when a real system is running: new agents can be implemented in virtual environments and added to the real one; moreover existing ones can be migrated in the virtual environment for maintenance. 5

Conclusions

We discussed on the opportunity to simulate virtual 3D environments to design and develop intelligent software system to control smart homes for the AAL. Even if such an approach cannot completely replace user trials before the everyday use, it can speed up the development of prototypes as well as simplify the maintenance and the update of existing systems.

Instead of using a robotics simulator, as in our simple proof-of-concept, a dedicated AAL simulator (with a physics engine) could be developed, in order to adhere to specific AAL needs: it could be an ideal platform to combine the efforts of the AAL research community for the development of assistive technologies. 1 For a video of the Morse simulation and the mobile application see http://www.youtube.com/watch?v=zXEpShRNGuo

References

1. World Health Organization: Global health and ageing . ( 2011 )

2. Sun , H., De Florio , V. , Gui , N. , Blondia , C. : Promises and challenges of ambient assisted living systems . In: Information Technology: New Generations , 2009 . ITNG ' 09 . Sixth International Conference on. ( 2009 ) 1201 - 1207

3. Sadri , F. : Ambient intelligence: A survey . ACM Computing Surveys 43 ( 4 ) ( 2011 ) 1 - 66

4. Rego , P. , Moreira , P. , Reis , L. : Serious games for rehabilitation: A survey and a classification towards a taxonomy . In: Information Systems and Technologies (CISTI) , 2010 5th Iberian Conference on. ( 2010 ) 1 - 6

5. Cascado , D. , Romero , S.J. , Hors , S. , Brasero , A. , Fernandez-Luque , L. , Sevillano , J.L. : Virtual worlds to enhance ambient-assisted living . In: Annual International Conference of the IEEE Engineering in Medicine and Biology Society . ( 2010 ) 212 - 215

6. Malone , T.W. , Lepper , M.R. : Making learning fun: A taxonomy of intrinsic motivations for learning . In: Aptitude, learning, and instruction 3 . ( 1987 ) 223 - 253

7. Susi , T. , Johannesson , M. , Backlund , P. : Serious games: an overview . Technical Report HS-IKI-TR-07-001 , School of Humanities and Informatics, University of Sko¨vde ( 2007 )

8. Connolly , T.M. , Boyle , E.A. , MacArthur , E., Hainey , T. , Boyle , J.M.: A systematic literature review of empirical evidence on computer games and serious games . Computers & Education 59 ( 2 ) ( 2012 ) 661 - 686

9. Simmonds , M. , Zikos , D. : Computer games to decrease pain and improve mood and movement . In: Proceedings of the the 7th ACM International Conference on PErvasive Technologies Related to Assistive Environments (PETRA 2014 ). ( 2014 )

10. Leonhardt , S. , Kassel , S. , Randow , A. , Teich , T. : Learning in the context of an ambient assisted living apartment: Including methods of serious gaming . In Motta, G., Wu , B., eds.: Software Engineering Education for a Global E-Service Economy. State of the Art, Trends and Developments . Progress in IS. Springer International Publishing ( 2014 ) 49 - 55

11. Van't Klooster , J.W., Van Beijnum , B.J. , Eliens , A. , Hermens , H.: Interactive scenario visualization for user-based service development . In: Proceedings of the 2012 International Conference on Collaboration Technologies and Systems , CTS 2012 . ( 2012 ) 498 - 503

12. Dignum , F. : Agents for games and simulations . Autonomous Agents and MultiAgent Systems 24 ( 2 ) ( 2012 ) 217 - 220

13. Palazzo , L. , Dolcini , G. , Claudi , A. , Biancucci , G. , Sernani , P. , Ippoliti , L. , Salladini , L. , Dragoni , A.F. : Spyke3D: A new computer games oriented BDI agent framework . In: Computer Games: AI, Animation, Mobile, Interactive Multimedia, Educational & Serious Games (CGAMES), 2013 18th International Conference on, IEEE ( 2013 ) 49 - 53

14. Calvaresi , D. , Sturm , A. , Yu , E. , Dragoni , A.F. : Exploring domain requirements and technology solutions: a goal modeling approach . In: Proceedings of the 7th International iStar Workshop (iStar14) . ( 2014 )

15. Sernani , P. , Palazzo , L. , Claudi , A. , Dolcini , G. , Biancucci , G. , Trentalange , G. , Dragoni , A.F. : Virtual carer: A BDI agent system for Ambient Assisted Living . In: 2nd International Workshop on Artificial Intelligence and NetMedicine (NetMed 2013 ). ( 2013 ) 14 - 23

16. Gazebo: Robot simulation made easy . Retrieved May 20 , 2014 , from http://gazebosim.org ( 2014 )

17. Morse : the modular openrobots simulation engine . Retrieved May 20 , 2014 , from http://www.openrobots.org/wiki/morse/ ( 2014 )