=Paper=
{{Paper
|id=Vol-3276/SSS-22_FinalPaper_84
|storemode=property
|title=Automatic Textual Care Record Generation for Smart
Nursing
|pdfUrl=https://ceur-ws.org/Vol-3276/SSS-22_FinalPaper_84.pdf
|volume=Vol-3276
|authors=Hayate Kondo,Masayuki Numao
}}
==Automatic Textual Care Record Generation for Smart
Nursing==
https://ceur-ws.org/Vol-3276/SSS-22_FinalPaper_84.pdf
Automatic Textual Care Record Generation for Smart Nursing
Hayate Kondo and Masayuki Numao
Department of Communication Engineering and Informatics
The University of Electro-Communications
1-5-1, Chofugaoka, Chofu-shi
Tokyo 182-8585, JAPAN
numao@cs.uec.ac.jp
Abstract Source Data and Interpretation Knowledge
We developed a system that automatically generates care The vital data filled in the care record includes body temper-
records from raw level sensor data that monitor vital sign of ature, blood pressure, SPO2, heart rate, respiratory rate, and
residents. The system first analyzes time-series data and ex- weight. The amount of food and water consumed should also
tracts important features, which then translated into the resi- be entered. ADL includes bathing or excretion, and the place
dent’s activity and health status, from which the system sum- where the excretion was performed are needed. Residents’
marizes the natural language description for the care records. vital data can be obtained by a body temperature sensor and
All relevant knowledge for this translation is represented in
microwave sensor. ADL can be recognized by location sen-
ontology language OWL. We evaluated the system by in-
putting actual sensor data sors such as RFID and BLE. The degree of care level can be
recognized from the proximity of caretaker and caregiver.
Vital sign at bed time is also available by a mat sensor.
Introduction
The objective of smart nursing is to establish a collaboration Required Knowledge
framework by human and system to achieve wellbeing of Measurement of vital data is important for maintaining the
both caretaker and caregiver. We have developed IoT-based health of residents. By taking regular measurements, it is
ADL monitoring system for nursing home, which is used for possible to detect changes in the health status of residents
detecting abnormality such as falling and stroking(Oishi and from the difference between normal and abnormal condi-
Numao 2018), and its application of FIM measuring(Oishi tions, leading to prevention of disease and early detection.
and Numao 2019). We focus on caregiver’s wellbeing: A lot Therefore, it is important to verbalize what kind of abnor-
of nursing staffs are suffering from overload. According to mality compared to normality, what kind of illness can occur
Oita-prefecture’s survey, more than 80% of staff’s overwork when the abnormality is seen by residents and how to deal
is spent for documentation of care record. We analyze the with after extracting abnormality from time series data. In
care records and develop a system which automatically gen- order to achieve this, it is necessary to have knowledge about
erates the description from resident’s vital data and ADL. the definition of normality and abnormality, phenomena that
The technical challenge is off course, the translation of raw can occur at the time of abnormality, and how to deal with
level sensor data into high level description of resident’s sta- them for each domain of vital data. By recording whether
tus, and summarizing in natural language. Another challenge resident can perform basic actions independently or how
is how to represent the translation knowledge, because many much care services is required, coordination between care-
kinds of knowledge should be used in the way. There are givers such as transfer can be performed smoothly. There-
mainly 2 approaches to deal with the problem, end-to-end fore, it is important to verbalize the contents from the results
translation by machine learning and step-wise translation by of ADL recognition, the place, the time, and the degree of
rule-based system. The former one is simple but needs a care services. In order to achieve, it is necessary to define
large amount of training data. The latter one does not require the location and time of each ADL and the care services for
training data but needs to build a knowledge base. In this pa- it.
per, we use the W3 standard ontology language OWL to rep-
resent 3 knowledges: (i) interpretation of time-series sensor
data, (ii) recognition of ADL and health status, (iii) transla- Architecture of Ontology-based Text
tion into natural language description for the care record. We Summarization
use the ontology mapping to translate sensor level status to In this research, we propose a system that automatically gen-
ADL status and the semantic reasoner to diagnose a possible erates sentences from time-series data. Our system is able
disease from the status. to analyze and verbalize time-series data using do-main-
___________________________________ specific knowledge described in ontologies. As shown in the
In T. Kido, K. Takadama (Eds.), Proceedings of the AAAI 2022 Spring Symposium figure 1, our system takes time-series data and domain name
“How Fair is Fair? Achieving Wellbeing AI”, Stanford University, Palo Alto, California,
USA, March 21–23, 2022. Copyright © 2022 for this paper by its authors. Use permitted
under Creative Commons License Attribution 4.0 International (CC BY 4.0).
41
as input and outputs a sentence summa-rizing the input data. Feature Extraction
The flow of the proposed system is as follows. Change point detection is performed on the input time-
1. (Off-line) Constructing domain-specific ontologies and a series data, and the time-series where the change occurred
time-series structure ontology related to the in-put do- is extract-ed. After that, the start time of the partial time-
main name. series, its length, and the value of the interval are obtained
2. Merging time-series structure ontologies and domain- and symbolized to extract the features of the input data. Fig-
specific ontologies. ure 3 shows the body movement by mat sensor.
3. Performing change point detection for time-series data
input, and features of the input data are extracted and
symbolized.
4. Reasoning in the merged ontologies (2) by inputting the
symbolized features (3) as OWL Individual, then deter-
mine the class that the input feature belongs to.
5. Generating the text by assembling the class attribute.
There are mainly 3 modules are involved: Ontology Figure 3: Body Movement Detection
Build-ing, Feature Extraction, and Text Generation.
Text Generation
The domain-specific ontology and time-series structure on-
tology are merged. After that, inference is performed on the
merged ontologies by inputting the symbolized features as
OWL individual. From the result, output sentence is gener-
ated by assembling the properties of classes that the individ-
ual belongs to. Assembling is also controlled by ontology
rule; thus, no templates are necessary
Experiment
The time-series data of body temperature, heart rate, and res-
piration rate were input to the system, and text is generated,
for example:
THERE IS A POSSIBLE OF SLEEP APNEA SYNDROME
BECAUSE RESPIRATORY RATE WAS UNDER 12 DUR-
ING SLEEP FOR 15 SECONDS FROM 01:00:00 on JULY
2, 2020 TO 01:00:15 ON JULY 2, 2020.
Figure 1: System Configuration
We also compared the sentences described in the generated
care record with previous studies, and confirmed the genera-
tion of textual summaries reflecting domain-specific knowl-
Ontology Building edge. Since our system is verbalized for each time-series
As off-line process, domain-specific ontology and time- data, the correlation of each domain is not described in the
series structure ontology are constructed by using OWL lan- generated sentence. Therefore, we analyze the correlation of
guage. Domain-specific ontology defines the terminology each domain from multiple time-series data, and aim to re-
and their relationships. For example, vital-sign ontology de- flect the result in sentences
fines the property, disease, and person as a class and their re-
lationships are defined by object property (Figure 2). Time- References
series structure ontology is built based on TimeseriesML. Oishi, N.; and Numao, M. 2018. Active Online Learning
Architecture for Multimodal Sensor-based ADL Recogni-
tion.
Oishi, N.; and Numao, M. 2019. Measuring Functional
Independence of an Aged Person with a Combination of
Machine Learning and Logical Reasoning.
Acknowledgments
This work was supported by JSPS KAKENHI Grant Num-
ber JP20H04289 ”Functional Independence Measurement
Figure 2: Vitalsign Ontology
System based on ADL Ontology for Aged Person”
42