Social-aware and context-aware multi-sensor fall detection platform Femke De Backere, Femke Ongenae, Floris Van den Abeele, Jeroen Hoebeke, Stijn Verstichel, Ann Ackaert, and Filip De Turck Department of Information Technology (INTEC), Ghent University - iMinds, Gaston Crommenlaan 8 bus 201, B-9050 Ghent, Belgium Femke.DeBackere@intec.ugent.be Abstract. A social- and context-aware multi-sensor platform is pre- sented, which integrates information of fall detection systems and sen- sors at the home of the elderly, by using an ontology. This integrated contextual information allows to automatically and continuously assess the fall risk of the elderly, to more accurately detect falls and identify false alarms and to automatically notify the appropriate caregiver. Keywords: Fall Detection & Risk Assessment, Semantic, Context-aware 1 Introduction For elderly fall incidents are often life-changing events that might lead to degra- dation or loss of autonomy. More than half of the elderly living in nursing homes and about one third of the elderly living at home fall at least once a year [4]. 10 to 15% of them suffer severe injuries [6]. Psychological consequences can also not be underestimated. Reliable fall detection and prevention are thus necessities. The fall risks, e.g., impaired mobility and gait, are assessed by formal care- givers with standardized tests on predefined, long time intervals. As a result, targeted measures and advice are formulated. Attempts to automate this assess- ment and follow-up through domotic and monitoring systems are limited and not integrated [6]. Several systems exist to detect falls, e.g., Personal Alarm Sys- tems (PAS), accelerometers, video cameras and micro arrays. However, the PAS is often not worn and false alarms and undetected falls regularly occur [1, 3, 6]. When the help desk is notified of a fall, a predefined ordered list is used to assign it. The current context of the (in)formal caregivers, e.g., location or availability, is thus not taken into account, resulting in unnecessary delays and distractions. The FallRisk project1 aims to develop a social- and context-aware multi- sensor framework to: 1) automatically assess the potential fall risks and the compliance of the elderly to advice by monitoring his or her behavior, 2) reduce the amount of undetected calls and false alarms by combining the information gathered by the plethora of fall detection systems and sensors installed at the home of the elderly, and 3) automatically select the (in)formal caregivers to 1 http://www.iminds.be/en/research/overview-projects/p/detail/fallrisk-2 2 Temp/Humidity/Light sensor Motion sensors Datasources MCI, MCK MCD MCI OCarePlatform Local Gateway Controllers D MC MCI, MCK CD M Patient Residence Pressure Formal Informal sensors Caregivers Caregivers Fig. 1: General architecture of the fall detection platform whom an alarm or notification should be sent based on the current context. To realize this goal, the framework integrates the heterogeneous and voluminous raw data gathered by all the devices in an ontology. Based on this integrated view of the current context, more intelligent algorithms can be defined. 2 General Architecture of the FallRisk system To detect falls and assess the fall risk, sensors are installed within the home of the elderly as shown in Figure 1. Three types of context information are captured: 1) the environment, e.g., by temperature, humidity and light sensors, 2) the activity level of the elderly, e.g., by passive infrared motion sensors, and 3) position information, e.g., by pressure sensors. Fall detection and risk assessment systems are also integrated. Finally, data is gathered from the elderly’s smartphone. Note that, not all devices should be installed. Depending on the needs and preferences of the elderly, the most appropriate ones should be selected that complement each other and lead to a reliable set-up for continuous monitoring. The raw Care Data (CD) generated by the devices is gathered on the Local Gateway at the residence. To save bandwidth, the gateway already processes some information, e.g., the video images. To provide a back-up plan when the connection to the servers is lost, the Local Gateway is able to do some rigorous analysis of the data. Finally, the gateway transforms CD into Meta Care Data (MCD) by enriching it with, e.g., timestamps, identifiers and location. The Controllers manage the connections between the OCarePlatform and the clients providing MCD, i.e., Local Gateways, caregivers’ smartphones and databases of the care organizations. The MCD is back-upped within the Data- sources. These also store all static information related to the elderly and care- givers. The Controllers transform the MCD to Meta Care Information (MCI) by tagging it with one or more Meta Care Concepts (MCC). The OCarePlatform uses the MCC to identify the corresponding ontological concept such that the MCI can be correctly integrated into the ontology. The MCI is sent to the OCarePlatform, which infers new Meta Care Knowl- edge (MCK) by using ontology-based reasoning. Derived knowledge, concerning contextual information and fall estimation and detection, is sent back to the 3 integratedInto System isObservationOf isPartOf xsd:float hasValue Sensorboard Sensor Device FallDetectionSystem Actuator Pressure Motion Humidity Light Temperature Call Portable TV Camera Chair Camera Micro PAS Light Sensor Sensor Sensor Sensor Sensor Button Phone Lift System Array hasUnit hasPrevious hasUnit xsd:string xsd:string Symptom Observation Event hasContext Entity Observation hasValue xsd:float hasFault hasSymptom has External CallButton Light Pressure Fall Humidity Motion Fault Solution Solution Temperature Status Intensity Status Observation Observation Observation Observation Observation Observation Observation hasLocation Furniture hasLocation Status hasStatus Location hasProfile Profile belongsTo hasLocation System Bed Status Zone hasCentre Coordinate BasicProfile RiskProfile xsd:int Coordinate Person Status hasDegree Staircase Room Hallway Biological Sociological Psychological Medical Behavioral Task Trust Person Profile Profile Profile RiskProfile RiskProfile Status Relationship belongsTo hasRole associated Gender Nationality Language Impatient FallRisk Action hasTrustRelationship Role hasStatus ownedBy hasCompetence belongsTo Process Informal inNeedOf hasStatus Patient Family Formal Competence Competence Task Caregiver Caregiver hasPriority Action Nurse Doctor Priority assignedTo PlannedTask requiresAction UnplannedTask TakingStairs performedBy Notification hasReason FallReason Reason hasReason Call Fig. 2: Prevalent concepts of the ambient-aware continuous care ontology for fall risk estimation and fall detection Controllers. They are responsible for notifying the correct caregiver(s) or the emergency response center based on the derived knowledge. 3 The OCarePlatform The OCarePlatform facilitates the intelligent and coordinated integration, anal- ysis, combination and efficient usage of the large amount of MCI sent by the Controllers by using ontologies. To model the knowledge pertaining to fall de- tection and risk assessment, the Ambient-Aware Continuous Care Ontology (AC- CIO) [5], was extended as shown in Figure 2. As shown in Figure 3, MCI enters the OCarePlatform through the Context Provider Services, which transform it to ontology individuals by analyzing the associated MCCs. As these map on ontological concepts, the Context Provider Services know which type of individuals need to be created and how they should be created by analyzing the axioms defined in the ontologies. The OCarePlatform is developed as a modular platform, consisting of an extensible set of MCI Services. These are the brains of the platform. They process the large amount of data in an efficient and manageable manner. Each service has a specific task, which can be implemented by 1) specifying axioms in the ontology to classify the incoming information and link it to appropriate action, 2) adopting rule engines to perform more complex analyses, and 3) integrating proprietary algorithms. Some example services are shown in Figure 3. Consequently, there is a need for an intelligent filtering system, capable of sending only that data to the MCI Services in which they are interested at that time. For this, the Semantic Communication Bus SCB [2] was designed, which uses the extended ACCIO ontology to filter the data based on its semantics, instead of on syntactical text patterns. Ontological data is published onto the SCB by Context Provider Services by using the Context Manager. The MCI Services use this Context Manager to specify the context they are interested 4 Stairs Monitoring MCI Service Fall Detection MCI Service Context Interpreter Context Interpreter Reasoner Core Reasoner Core Notification MCI Ontologies Ontologies Service import import Application Domain Domain Logic Specific Specific Ontologies Ontologies publish push MCI, publish MCI, push ... publish MCI, push MCI, MCK MCK MCK MCI, MCK MCK MCI, MCK Context Manager Context Manager Context Manager set filter rules & push set filter rules & push set filter rules & push publish MCI, MCK MCI, MCK publish MCI, MCK MCI, MCK publish MCI, MCK MCI, MCK Semantic Core Ontologies Communication Context Disseminator Cache Bus (SCB) publish MCI Context Manager Context Provider Services MCI Fig. 3: Architecture of the OCarePlatform in, by defining ontological filtering rules and registering them with the Context Disseminator. This allows to reduce the amount of data that is forwarded to the MCI Services, which prevents them from being flooded with huge amounts of data. It also facilitates an agile approach, where new services can easily be deployed or duplicated for scalability and redundancy. For example, the Fall Risk MCI Service registers the following rule: Event and hasContext some ((Action and (isPerformedBy some (hasRole some Patient))) LichtIntensityObservation) It is important to note that all conclusions, called MCK, drawn by the MCI services are put back on the SCB. In this way, conclusions drawn by one MCI service can be used by a second MCI service as additional situational informa- tion. As such, the OCarePlatform supports the composition of complex services from a set of smaller services in a loosely coupled manner. The simple services perform specific reasoning tasks in parallel and notify their conclusions to other services, which have expressed an interest in this kind of information. Acknowledgment FallRisk is funded by iMinds and IWT and involves COM- meto, Televic Healthcare, TP Vision, Verhaert and Wit-Gele Kruis Limburg. References 1. Debard, G., Karsmakers, P., Deschodt, M., et. al: Camera-based Fall Detection using Multiple Features validated with Real Life Video. In: Proc. of International Conference on Intelligent Environments. pp. 441–450 (2011) 2. Famaey, J., Latré, S., Strassner, J., et. al: An Ontology-Driven Semantic Bus for Autonomic Communication Elements. In: Proc. of the 5th IEEE international con- ference on Modelling autonomic communication environments. pp. 37–50 (2010) 3. Li, Y., , Ho, K.C., Popescu, M.: A Microphone Array System for Automatic Fall Detection. IEEE Transactions on Biomedical Engineering 59(5), 1291–1301 (2012) 4. Milisen, K., Detroch, E., Bellens, K., et. al: Falls among Community-Dwelling El- derly: A Pilot Study of Prevalence, Circumstances and Consequences in Flanders. Tijdschr Gerontol Geriatr 35(1), 15–20 (2004) 5. Ongenae, F., Bleumes, L., Sulmon, N., et. al: Participatory Design of a Continu- ous Care Ontology: Towards a User-Driven Ontology Engineering Methodology. In: Proc. of International conference on Knowledge Engineering and Ontology Devel- opment. pp. 81–90 (2011) 6. Vlaskamp, F., van der Heijden, J.: SeniorWatch: Fall Detector: A Cooperation be- tween The United Kingdom and The Netherlands. Tech. rep. (December 2001)