Automatic Textual Care Record Generation for Smart Nursing

Abstract

We developed a system that automatically generates care records from raw level sensor data that monitor vital sign of residents. The system first analyzes time-series data and extracts important features, which then translated into the resident's activity and health status, from which the system summarizes the natural language description for the care records. All relevant knowledge for this translation is represented in ontology language OWL. We evaluated the system by inputting actual sensor data The vital data filled in the care record includes body temperature, blood pressure, SPO2, heart rate, respiratory rate, and weight. The amount of food and water consumed should also be entered. ADL includes bathing or excretion, and the place where the excretion was performed are needed. Residents' vital data can be obtained by a body temperature sensor and microwave sensor. ADL can be recognized by location sensors such as RFID and BLE. The degree of care level can be recognized from the proximity of caretaker and caregiver. Vital sign at bed time is also available by a mat sensor.

Introduction

The objective of smart nursing is to establish a collaboration framework by human and system to achieve wellbeing of both caretaker and caregiver. We have developed IoT-based ADL monitoring system for nursing home, which is used for detecting abnormality such as falling and stroking (Oishi and Numao 2018) , and its application of FIM measuring (Oishi and Numao 2019) . We focus on caregiver’s wellbeing: A lot of nursing staffs are suffering from overload. According to Oita-prefecture’s survey, more than 80% of staff’s overwork is spent for documentation of care record. We analyze the care records and develop a system which automatically generates the description from resident’s vital data and ADL. The technical challenge is off course, the translation of raw level sensor data into high level description of resident’s status, and summarizing in natural language. Another challenge is how to represent the translation knowledge, because many kinds of knowledge should be used in the way. There are mainly 2 approaches to deal with the problem, end-to-end translation by machine learning and step-wise translation by rule-based system. The former one is simple but needs a large amount of training data. The latter one does not require training data but needs to build a knowledge base. In this paper, we use the W3 standard ontology language OWL to represent 3 knowledges: (i) interpretation of time-series sensor data, (ii) recognition of ADL and health status, (iii) translation into natural language description for the care record. We use the ontology mapping to translate sensor level status to ADL status and the semantic reasoner to diagnose a possible disease from the status. Measurement of vital data is important for maintaining the health of residents. By taking regular measurements, it is possible to detect changes in the health status of residents from the difference between normal and abnormal conditions, leading to prevention of disease and early detection. Therefore, it is important to verbalize what kind of abnormality compared to normality, what kind of illness can occur when the abnormality is seen by residents and how to deal with after extracting abnormality from time series data. In order to achieve this, it is necessary to have knowledge about the definition of normality and abnormality, phenomena that can occur at the time of abnormality, and how to deal with them for each domain of vital data. By recording whether resident can perform basic actions independently or how much care services is required, coordination between caregivers such as transfer can be performed smoothly. Therefore, it is important to verbalize the contents from the results of ADL recognition, the place, the time, and the degree of care services. In order to achieve, it is necessary to define the location and time of each ADL and the care services for it.

Architecture of Ontology-based Text

Summarization

In this research, we propose a system that automatically generates sentences from time-series data. Our system is able to analyze and verbalize time-series data using do-mainspecific knowledge described in ontologies. As shown in the ifgure 1, our system takes time-series data and domain name as input and outputs a sentence summa-rizing the input data. The flow of the proposed system is as follows. 1. (Off-line) Constructing domain-specific ontologies and a time-series structure ontology related to the in-put domain name. 2. Merging time-series structure ontologies and domainspecific ontologies. 3. Performing change point detection for time-series data input, and features of the input data are extracted and symbolized. 4. Reasoning in the merged ontologies (2) by inputting the symbolized features (3) as OWL Individual, then determine the class that the input feature belongs to.

Ontology Building

As off-line process, domain-specific ontology and timeseries structure ontology are constructed by using OWL language. Domain-specific ontology defines the terminology and their relationships. For example, vital-sign ontology deifnes the property, disease, and person as a class and their relationships are defined by object property (Figure 2). Timeseries structure ontology is built based on TimeseriesML.

Feature Extraction

Change point detection is performed on the input timeseries data, and the time-series where the change occurred is extract-ed. After that, the start time of the partial timeseries, its length, and the value of the interval are obtained and symbolized to extract the features of the input data. Figure 3 shows the body movement by mat sensor.

Text Generation

The domain-specific ontology and time-series structure ontology are merged. After that, inference is performed on the merged ontologies by inputting the symbolized features as OWL individual. From the result, output sentence is generated by assembling the properties of classes that the individual belongs to. Assembling is also controlled by ontology rule; thus, no templates are necessary

Experiment

The time-series data of body temperature, heart rate, and respiration rate were input to the system, and text is generated, for example: THERE IS A POSSIBLE OF SLEEP APNEA SYNDROME BECAUSE RESPIRATORY RATE WAS UNDER 12 DURING SLEEP FOR 15 SECONDS FROM 01:00:00 on JULY 2, 2020 TO 01:00:15 ON JULY 2, 2020.

We also compared the sentences described in the generated care record with previous studies, and confirmed the generation of textual summaries reflecting domain-specific knowledge. Since our system is verbalized for each time-series data, the correlation of each domain is not described in the generated sentence. Therefore, we analyze the correlation of each domain from multiple time-series data, and aim to relfect the result in sentences

Acknowledgments

This work was supported by JSPS KAKENHI Grant Number JP20H04289 ”Functional Independence Measurement System based on ADL Ontology for Aged Person”

References

Oishi , N.; and Numao, M. 2018 . Active Online Learning Architecture for Multimodal Sensor-based ADL Recognition .

Oishi , N.; and Numao, M. 2019 . Measuring Functional Independence of an Aged Person with a Combination of Machine Learning and Logical Reasoning .