The use of virtual reality games, known as \exergaming", is gaining more and more interest as a mobilization tool and as a key piece in the delivery of quality health, especially in elderly people. Mobility tracking of elderly people facilitates the extraction of useful spatiotemporal characteristics regarding their activities and behavior at home. Currently, the analysis of human mobility is based on expensive technologies. In this paper, we propose a semantic interoperability agent which exploits mobility tracking and spatiotemporal characteristics to extract human pro ling and give incentives for mobilization at home. The agent exploits an extended ontology which facilitates the collation of evidence for the e ects of exergaming on the movement control of older adults. In order to provide personalized monitoring services, a number of rules are individually de ned to generate incentives. To evaluate the proposed semantic interoperability agent, human mobility data are collected and analyzed based on daily activities, their duration and mobility patterns. We show that the proposed agent is robust enough for activity classi cation, and that the recommendations for mobilization are accurate. We further demonstrate the agent's potential in useful knowledge inference regarding personalized elderly people home care.
The increasing trend in the number of elderly people is a major public health challenge. Home support is an important preoccupation for the elderly and their families. It is known that physical activity is important for older adults at any age and health status, i.e. from a 50-year-old up to a 80-year-old.
However the home is not a place without risk for the older adults. The means
to enhance the health and quality of life by motivating them for physical training
in an entertaining way with real-time interventions are very limited. A
nowadays popular solution for enhancing the physical activity of older adults is to
provide them with computer games which are played via body movement and
thus have the inherent e ect of unobtrusive physical exercise, i.e. the so-called
\exergaming" [
The game must be designed for non-skilled users in order to be accepted by older adults. This includes clear user interfaces, suitable game topics, avoidance of small objects and the encouragement by visual and auditive feedback. Also the social factor and personal preferences, e.g. by providing multi-user games, variety, etc., have to be considered. A very crucial requirement is to mind the mobility of the individual user, as many age-related processes have an impact on the ability to move which may di er from the skills needed to play a game. Common age-related changes are decrements in balance, gait, strength, impacts on visual and hearing senses as well as impairment of memory, attention and vigilance. Additional aspects to be addressed are the longer reaction, the overall movement times and the increased risk of falling. The high importance of this personalization aspect is given by its high correlation with the older adult's motivation to play and hence with the acceptability of exergaming, i.e. if a frail 80-year-old is confronted with games that require unachievable movements, she will feel over strained and soon lose interest. On the other hand, if a healthy 65-year-old is confronted with unchallenging game tasks, she will feel bored and will lose interest as well.
The required personalization can be achieved by manually con guring the exergaming platform based on the supposed mobility of the older adult, but this has many drawbacks, e.g it is hard to assess the mobility beforehand, mobility can change over time, and the individual preferences are not considered. Hence, what is needed is a platform that automatically and continuously adapts to the user's preferences, skills and mobility. Despite the ongoing research and development in the area of exergaming, performed both by industry and the research community, the problem of personalized recommendations are not properly addressed.
The goal of this work is to contribute towards the personalization of elderly
people home care [
Several works present literature reviews of data-driven health care systems.
Zenunia et al. [
Other approaches consider spatiotemporal data analysis, information and
images collected from wearables and other monitoring devices. Temporal aspects
to model behavior are considered by Floeck et al. [
Another category, to which the present work most closely relates, involves
ontology-driven approaches for health care services. A HealthIoT ontology is
proposed by Rhayem et al. [
Several methods address health care services from an applied perspective
based on gami cation and applications. An application for mobile devices,
developed for the Android platform in the JAVA programming language and XML
markup to identify the frailty phenotype among the elderly was proposed by
Silva dos Santos et al. [
From the point of sociology view, Wanka and Gallistl [
Although the current approaches of data-driven health care systems use monitoring devices, ambiguous semantics and data curation from diverse sources, each individual work merely focuses on a single one of them. Compared to the aforementioned approaches, the proposed method di ers in that it exploits user-driven behavioral characteristics using spatiotemporal information to provide concrete and targeted recommendations. It also infers useful personalized knowledge which is built incrementally from individual users movements and behavior at home and is delivered back to the end-user in the form of recommendations and incentives for exercise. This work introduces a semantic interoperability agent which promotes easy data exchange with existing and other systems and e ciently blends data-driven and semantic-aware health services to fuel personalized interventions which improve the self-esteem and the quality of elderly people life. A fringe bene t of the agent is that in this way, it supports several frameworks as it is based on standardized data format and provides the exibility to be built on top existing and future systems. 3
The semantic interoperability agent proposed in this paper consists of 4 components: the Ontology, the Data Alignment, the Personalized Recommendations and the Interfaces. The framework is realized by means of a knowledge base which is built incrementally from user-driven behavioral characteristics and is used to store information that corresponds to the movement sequences, along with interfaces that provide mechanisms for accessing and updating this information.
The Ontology contains the entities and relations describing the movement sequences and the type of movements. The Data Alignment consists of a set of classes which model the Ontology and specify the conditions which trigger recommendations. The Personalized Recommendations consist of a set of methods which provide a rule based mechanism that generates interventions and incentives for elderly mobilization according to their mobility conditions and a knowledge base which is built incrementally from their behavioral characteristics. The Interfaces consist of a set of methods and REST APIs which provide the means to retrieve and update entities of the Ontology and expose the movement sequences as they have been processed by the other components of the agent.
This semantic interoperability agent automatically serves as a middleware which provides knowledge gained during the execution history through the REST APIs. If, for example, a set of movement patterns have been observed, these sequences are stored to a centralized database and shared through all the instantiations of the entire framework by the respective API. The semantic interoperability is realized through the Ontology, Data Alignment and Interfaces that are combined in order to turn domain speci c data into domain agnostic across di erent services. Figure 2 shows the functionality of the semantic interoperability agent, how it interacts with the mobility tracking module and how it produces recommendations for personalized exercises and incentives for mobilization through the Interfaces. Therefore, knowledge incrementally gained information is derived by semantic-aware movement sequences, stored in the database and exposed by the Interfaces.
Fig. 2. The Semantic Interoperability Agent.
The input to the semantic interoperability agent comprises sparse mobility tracking data in the form of spatiotemporal sequences. Using linear interpolation between consecutive samples, we derive the mobility tracks of each person. A mobility track is modeled as a list of spatiotemporal points M = fp0; : : : ; png with pi = hxi; yi; tii and xi; yi 2 R; ti 2 R+ for i = 0; 1; : : : ; n and t0 < t1 < t2 < : : : < tn. These tracks are susceptible to noise, as they are a ected by a measurement error and a sampling error due to the variable sampling rate. The output of the semantic interoperability agent is a set of recommendations R, modeled as a set R = fid; ActivityT ypeg, where id corresponds to a user identi er and ActivityT ype corresponds to the category of the personalized activity proposed each time to her. This activity may result in a walking game or a mind game having as a sequence either a light mobilization or a mental exercise in the form of recommendations to elderly people.
The tracking of the user is realized by 3D sensors, happens throughout the day and covers individual tracks. The individual tracks are stored and periodically sent to the semantic interoperability agent for further analysis to gain information of the user's mobility by extracting whenever the person is moving within the room, the duration, distance, type and velocity of her movement. End-users data have been anonymized by removing their personal and identi able information. The following information is extracted from the stored tracks and is taken into account in the rules de nition and enforcement for the provision of recommendations in each individual person: { The Active Time in Room (ATR). The duration of every track is summed up and divided through the estimated time that the user has actually been in the room during the day. This leads to a relative amount of active time within the observed room by also capturing the distance and duration. { The Active Time of Day (ATD). The observed tracks are divided into timeslots of one hour to give a statistical overview of the user's most active/inactive times during the day in the form of a histogram. { The Average Gait Velocity (AGV). The observed tracks are segmented and ltered into straight parts to calculate the average gait velocity on straight paths. { The Average Stand up Time (AST). When the person stands up (i.e. a track is recognized from within the scene) the time from sitting to standing is measured. { The Average Walking Time (AWT). When the person walks (i.e. a track is recognized from within the scene) the duration of her walking is measured.
This mobility tracking information is analyzed to create a spatiotemporal behavior model which shows where in the room the user stays most of the time. This is analyzed over a long time period (i.e. 3 months) and is compared to previous behavior recordings to determine changes in the user's behavior.
UniversAAL [
As an outcome of this stage, a formal conceptual model to de ne individual elderly pro les is achieved in which data provided by the di erent sources participating in the mobility tracking process can be mapped. This is achieved through a exible and extendable model for both data-in-motion and data-at-rest, which can be further exploited across multiple processing components.
The main purpose of the Ontology is to model all possible movement sequences that are of interest to the semantic interoperability agent and thus the entire framework. The ontology is developed in order to cover sequences of movements and has been designed in such a way to allow easy extension, thus facilitating easy future modeling of di erent movement sequences, that are needed due to the discovery of new e cient movement patterns.
All the implemented classes follow the hierarchical structure proposed by
UniversAAL [
The OWL-DL language (an OWL { Ontology Web Language { sublanguage)
was chosen to describe the ontology model [
The main purpose of the Data Alignment is to map the mobility tracking data provided as input to each class of the Ontology and nally to JAVA objects of the semantic interoperability agent. The JAVA objects contain the relevant elds and methods needed for representing the corresponding classes and some helper methods. They are all serializable, so that they can be promptly converted to JSON format and communicated via the endpoints of the Interfaces. Except from the mapping of Ontology classes to JAVA objects the mapping of JAVA objects to database tables is needed as well, in order to store all the required information into the centralized database. A JAVA API serves as consumer which acquires data that are generated by a scheduler and stores them in the centralized database for further process and usage by the facets of the Interfaces. Mapping JAVA objects to database tables is implemented via the JAVA Persistence API (JPA). The JPA API allows to map, store, update and retrieve data from the centralized database to JAVA objects and vice versa.
In order to provide personalized recommendations, apart from an instance of the
elderly movement ontology, a number of rules are individually de ned for each
person. These rules take into account the duration, the distance, the velocity
and the kind of elderlies activity extracted from the stored tracks (i.e. active
time, gait velocity, stand-up or walking time), as presented in Section 3.2. By
using these rules, the behavior of individuals are expressed inside the domain
and thus can be used to express individual recommendations according to their
movement conditions. In fact, rule-based systems have been extensively used
in applications that require personalized services [
The steps of the algorithm are listed in Algorithm 1. Speci cally, the Semantic Interoperability Agent takes as input Mobility Tracks M and gives as output a set of Personalized Recommendations R. For each mobility track (Lines 3 - 18), the algorithm looks for people's presence in a room and a set of movements by using speci c temporal criteria (e.g. dates) (Lines 4 - 7). Then, the algorithm records in the M ovingSequence the set of walking movement, dates and types (Line 8). If the set of M ovingSequence is empty, the algorithm returns a set of personalized recommendations regarding games which include walking exercises in order to mobilize the elderly people (Line 13). On the contrary, if the set of M ovingSequence is not empty, the algorithm returns a set of personalized recommendations regarding mind games (Line 16).
As interoperability and personalized services are one of the primary design and evaluation goals, we need to ensure that the Semantic Interoperability Agent achieves abundant communication and interfaces among sensors, software and tools. This is related with all the data needed to support the respective decision support systems. Also, it achieves interoperability of the solution with external services as the data are made available in a standardized format that can be read and used by other systems.
The e ciency of the Semantic Interoperability Agent has been evaluated in terms of technology acceptance and incentives for exergaming. We conducted a survey interviewing end-users by both using quantitative and qualitative questions. In this section, we focus on the respective questions which concern the Semantic Interoperability Agent and especially the technology use and acceptance, their attitude towards exergaming, the accuracy of personalized recommendations and the e ciency of motives for physical activities. We also denote some evaluation criteria regarding mental, functional and general health of endusers. Mental health refers to psychological and social well-being condition of end-users, functional health refers to the ability of end-users to do the activities they need to do and general health refers to end-users who are generally healthy. We interviewed 201 end-users who are coming from Austria (i.e 100) and Netherlands (i.e. 101). Their average age is approximately around 77 years old where 33% of them are men and 67% of them are women. Also, 55% of end-users live in single households. All the end-users gave their consent for the participation in the survey and no personal data were collected. We used a 5points scale questionnaire in which we either measured and evaluated end-users agreement (from 1 to 5, e.g. ranging from strongly disagree to strongly agree), or satisfaction (from 1 to 5, e.g. ranging from very dissatis ed to very satis ed).
Fig. 4. Quantitative and Qualitative Results.
Figure 4 shows that the end-users exhibit wide acceptance (more than 68.5%) of the system and believe that its use has signi cantly contributed in their mental, functional and general health (more than 72%). However, the greater incentives have been received by persons who do not exercise in their real life, so the agent demonstrates a better potential in the elderly people of 75 years old or more (30%). At the same time, the end-users who would not adopt exergaming are either too t and already prefer a more active way of life or too sick and thus an alternative option should be taken into account. Besides, the end-users who proved not be such motivated (i.e. not interested, too boring) by exergaming concern a small part of the interviewed of about 3%.
The purpose of this work is to o er an interoperability solution which is easily accessible and stores meaningful information driven by ontologies to provide personalized recommendations to elderly people in exergaming. We extended a data model described by a standardized ontology which is familiar to domain experts. Having a clear model contributed to identify rules and provide personalized interventions.
In a nutshell, the semantic interoperability agent exposes data in a standardized format and supports interoperability with existing AAL systems and external services. The agent serves as middleware by taking into consideration mobility behaviors. It drives personalized recommendations to the end-users in their private homes by increasing their self-esteem and thus their quality of life.
In the near future, we plan to experiment with the proposed agent in the context of online methods. As it is becoming increasingly easier to gain access to mobility data sources, such an agent could improve the process of enhancing, combining and enriching disparate data sources and optimize the every day life of elderly people through interactive interventions. To this e ect, we are investigating automatic methods to infer useful semantic knowledge from diverse data sources with variable characteristics.
This work was supported by EnterTrain project which received fund under the AAL Programme (Project No. AAL-2015-056).
Karagiorgou et al.