This paper presents the MARIO's CGA (Comprehensive Geriatric Assessment) module for the Kompa¨ı platform that aims at autonomously performing and managing the execution of specific tests required in the CGA process. The application relies on the CGA ontology, which is part of the Mario Ontology Network (MON)4, as a support to the execution of the assessment process and a reference model for storing test information.
Dementia is a broad category of brain diseases which cause a gradual decrease in the ability to speak, think and remember thus affecting a person’s daily functioning. These impairments are often accompanied by loneliness, isolation and depression. Unfortunately, the number of people with dementia (PWD) is increasing and is expected to reach 80 million, by 2040. Current health care strategies are insufficient to combat this phenomena hence, ICT solutions and companion robots are considered key technologies for mitigating its effect.
MARIO5 is a companion robot (based on the KOMPAI¨ platform) that contributes to address this problem. It relies on the Semantic Web for the background knowledge supporting its understanding and dialoguing capabilities. Among its functionalities, MARIO assists caregivers and physicians in performing the Comprehensive Geriatric Assessment (CGA) of PWD, a clinical procedure for assessing the medical, psychosocial, functional and environmental status of a PWD.
This is not the first attempt to support health professionals in this procedure, in fact
they increasingly use ICT tools and devices during the performance of the CGA for
recording test results and calculate the corresponding scores. Specific software
applications are available for supporting the clinicians in the evaluation and calculation of
the MPI (Multidimensional Prognostic Index) and related scores, such as the
CalculateMPI tool and the iMPI application for iOS-based devices. However, it has been observed
that these devices and the need to interact with them to input information can represent
a “communication barrier” between the caregiver and the patient during clinical
interviews. The lack of visual contact with the caregiver can further increase stress and
anxiety in frail elderly patients undergoing a cognitive evaluation whose results may
potentially impact on their autonomy. MARIO’s solution proposes a completely different
4 http://www.ontologydesignpatterns.org/ont/mario/mario.owl
5 http://www.mario-project.eu/portal/
approach: to make the patient interact with a robot that can autonomously perform the
procedure. In a similar direction, as follow up of the ECHORD++ challenge focused on
Robotics for the CGA6, the ASSESSTRONIC project and the CLARC framework [
The Comprehensive Geriatric Assessment. The CGA is a diagnostic process that aims to collect and to analyse data to determine the medical, psychosocial, functional and environmental status and problems of an elderly patient. Its ultimate goal is to define an overall, personalized plan for treatment and long-term care. The CGA procedures conceptually include three main phases: (i) Clinical Interview, to gather preliminary information about patient’s health status, by interacting with the patient and his/her relatives; (ii) Multidimensional Assessment, to assess the functional mental and social status of the patient, through multidimensional tests; (iii) Care Plan Definition and Review, based on the previous phases a patient-specific care plan is defined and implemented. The multidimensional assessment phase is the core of the CGA process and represents a critical, time consuming activity for the caregivers. Physicians rely on a set of assessment tools and standardized rating scales to evaluate patient’s functional abilities, physical and mental health, and cognitive status. Two main classes of tests are performed: (i) Questionnaire-based tests: standardized clinical questionnaires (e.g., about his/her daily life and ability to autonomously perform specific activities). Depending on the answers, a score is given to the patient and evaluated according to a reference rating scale. (ii) Observation-based activity performance tests: specific physical activities (such as getting up and walking for a short path) are observed to rate the patient as for gait/balance assessment and fall risk assessment.
The introduction of a robotic solution able of autonomously performing parts of a CGA is expected to reduce the direct involvement of health professionals in the timeconsuming data collection tasks, as well as the perceived tiredness resulting from the performance of repetitive tests. As a result, this will enable them to concentrate their efforts on the interpretation of the results and the elaboration of personalized care plans. 2
By exploiting the underlying robotic platform and its sensors and I/O devices, the CGA module is required to enable MARIO to manage autonomously the execution of some CGA tests. To this end, the robotic CGA module is required to: (i) initiate and undertake a dialogue-based interaction with the patient to perform questionnaire-based selected tests; (ii) monitor patient’s motion behaviour under specific constraints and execution settings; (iii) complement the natural language interface with a graphical user interface and touch-based interaction modality to gather patient’s input. Sensorimotor capabilities are considered as a viable solution for further improving the user experience during the assessment procedures. This includes the ability of MARIO to orient itself towards the user, as well as the ability to dynamically approach the patient and adjust its position and distance depending on the test to be performed.
Architecture. A diagram sketching the architectural components of the CGA module is shown in Figure 1 and the main components are described as follows. The Caregiver Vocal answer
Kinect device MARIO(IONNCTLOULDOINGGYCNGEATWMOORDKULAENADNKDNPOPWDLBE-DRGDFE)BASE
On screen Q&A joints/skeleton tracking
Interface a Web-based Graphical User Interface, designed to allow the authorized clinicians to configure patient’s profile and CGA sessions, trigger and monitor their executions, and access generated health reports and scores resulting from the assessments. The Session and State Manager manages the overall execution of CGA sessions, coordinating the scheduling of the configured tests. The MPI Calculator is responsible for calculating the overall Multidimensional Prognostic Index, taking into account the scores and rating scales resulting from the execution of the assessment tests.
The Questionnaire-based Test Executor is in charge of the engaging and the execution of questionnaire-driven tests that are part of the assessment process. The dialogue flow is driven by the robot (i.e., the interaction is system-initiated) and unfolds on the basis of a continuous question-answer interaction pattern. For a specific test, the corresponding questionnaire script is derived from its description and representation retrieved from the Knowledge Base. Questions are thus formulated in a way that induces a restriction on the answers space. The advantages of this approach are twofold: on one side, providing the user with a limited set of possible answers (typically restricted to “yes/no” options) aims at reducing the cognitive load for the patient in the questionanswer process; on the other side, this reduces the interpretation dimensions that have to be considered when natural language understanding techniques are used. If the system is not able to understand and interpret the answer, the question is posed again.
The CGA Answers Understander takes as input the textual representation of patient’s utterances, as provided by the MARIO Speech-to-Text subsystem. From the analysis of video and audio recording of CGA-sessions emerged that the usage of a restricted vocabulary and keyword-spotting techniques can be effective in supporting predefined dialogues where the interaction is driven by the system for eliciting specific information from the user through a set of questions. The actual interpretation strategy directly depends on the question classification and corresponding answer type, namely: yes-no or Wh -questions. (i) Yes-no questions. The Yes-No questions cover most of the items in the CGA questionnaires, the patient’s answers are matched against regular expression patterns that aim at capturing both positive and negative answers. The patterns were built by exploiting existing linguistic resources, in particular the Paraphrase Database (PPDB)7, an automatically extracted multilingual database containing paraphrases in 16 different languages. PPDB has been re-engineered in RDF8 according to the recently introduced PPDB ontology9. (ii) Wh-Questions. In the case of Wh-questions, which cover most of the items in the SPMSQ10 (e.g., ”What day of the week is it today?”, ”Who is the Pope now?”), the understanding process maps to the task of comparing patient’s answers with known properties of named entities retrieved from the MARIO’s KB as well as commonsense KBs. The matching process relies on specialised understanding functions that restrict the recognition and interpretation to specific domains, such as dates and numbers (used for example when the user is asked to perform basic math calculations as part of the SPMSQ questionnaire).
CGA ontology module. The contribution of the CGA ontology11 is twofold. On the
one hand, the ontology supports the execution of the assessment by providing a
reference model for storing test information (such as questions, expected answer etc.). On
the other hand, it allows to store the data resulting from the patient’s assessments. To
the best of our knowledge there is no other ontology able to represents the results of an
execution of the CGA. Some ontologies have been proposed for supporting the
(general) medical assessment process [
8 PPDB-RDF SPARQL endpoint, http://w3id.org/framester/sparql 9 PPDB ontology, http://w3id.org/ppdb/ontology/ppdb.owl 10 SPSMQ stands for Short Portable Mental Status Questionnaire. 11 http://www.ontologydesignpatterns.org/ont/mario/cga.owl 12 Since cga:GeriatricAssessment specialises the class action:Action
cga:ClinicalTest clinicalact:hasMember
cga:GeriatricAssessment clinicalact:hasMember cga:assessesPatient action:Action healthrole:Patient cga:CGA time:TempotrimaleE:a1ntTtiimtye action:Task cga:Question action:executesTask cga:correctResponse rdfs:Literal clinicalact:hasMember cga:score rdfs:Literal
cga:hasText language:Text cga:score rdfs:Literal action:byAgent
action:Agent cga:toQuestion cga:anscwgear:rAdnfss:Lwiteerral assessment is to be executed, we implemented the Ontology Design Pattern Task Execution. The action cga:GeriatricAssessment executes a cga:ClinicalTest which provides a “description” of how the assessment has to be executed. A cga:ClinicalTest can be composed of other clinical tests or some cga:Question. Furthermore the CGA ontology allows to store information about the answers (i.e. cga:Answer) provided by a patient to reply to a question. 3
The MARIO’s CGA module described in this paper is being validated for acceptability with patients in two different dementia care settings in Ireland and Italy. It will be tested and validated with patients in hospital settings and in nursing facility settings. The outcome and continuous feedback provided by trial activities will further contribute to the refinement and evolution of the CGA module.
Acknowledgements. The research leading to these results has received funding from the European Union Horizons 2020 the Framework Programme for Research and Innovation (2014-2020) under grant agreement 643808 Project MARIO ”Managing active and healthy aging with use of caring service robots”.