This paper presents and motivates the construction of an ontological module for physiotherapist sessions. The work is developed as part of the research project SORTT (an IoT-Serviced And Ontology-based Remote human digital Twin for physioTherapy) and the model is conceived as an extension of the DOLCE ontology. The overall goal is to enhance patient supervision by healthcare professionals reducing the need for costly infrastructures, and making this kind of health service accessible to a larger number of patients, in particular obese and diabetic patients. After discussing the literature, the paper discusses the knowledge acquired via interviews with experts in the domain, and structures a module in line with the IoT system envisioned in the project. The result is a further step towards the development of reliable and flexible health IoT- and ontology-based systems.
This contribution presents the research project SORTT (an IoT-Serviced And Ontology-based Remote human digital Twin for physioTherapy) and focuses on related research achievements about the extension of the DOLCE ontology to implement it. SORTT is a two-year research project funded by the Italian Ministry of Universities and Research (MUR) through the PRIN 2022 PNRR call, which proposes an IoT-based framework to enhance patient supervision by healthcare professionals, lessening the problem of traditional approaches that require costly infrastructures.
Over the past decades, we have witnessed a steady increase in life expectancy. As a result, old age is increasingly considered an important phase of life with studies on dedicated activities and well-being. Significant progress has been made in scientific research, particularly in psychology and medicine. Culturally, it is increasingly recognized as a stage of life that should be nurtured and well-being experienced to the fullest. At the same time, especially in Western countries—where the effects of economic prosperity have not adequately accounted for certain unhealthy habits—age-related disorders have been rising both in number and severity. Poor nutrition and lack of physical activity contribute significantly to this trend.
Aging, combined with unhealthy lifestyle choices, contributes to the development of high-risk metabolic conditions such as obesity and type 2 diabetes. Across Europe, these disorders have reached epidemic levels, with physical activity playing a crucial role in enhancing motor function and improving patients’ physiological and stress-related health profiles. However, access to preventive and tailored motor rehabilitation remains limited, as such programs are typically offered only in specialized centers under the supervision of healthcare professionals. This restriction significantly reduces their accessibility, underscoring the need for innovative solutions that eliminate reliance on costly infrastructure and broaden availability, as claimed in the SORTT research project document.
At the same time, other phenomena have marked the past decade, including the COVID-19 pandemic, which, among its various side effects, has accelerated the development and enhancement of remote activities and the supporting technologies. Although these technologies were already available, they were deployed only in exceptional, rather than routine, circumstances. This shift has driven a new focus on tele-technologies and their various applications, integrating them into everyday use—despite the long-term impact of systematically reduced direct human interaction remaining unclear.
A significant portion of these advancements has been directed toward remote patient management, particularly during hospital discharge phases by medical staff. Often, this remote support is aimed at rehabilitation activities, both cognitive and physical—a topic widely explored in scientific literature. Simultaneously, the rapid development of remote technologies, the Internet of Things (IoT), Artificial Intelligence (AI), and smart support systems has continued at an intense pace. This evolution has involved not only software but also interface tools and hardware, providing end users with increasingly diverse configurations designed for maximum flexibility in use. Through the SORTT research project, we aim to enhance and facilitate access to appropriate care, even within a remote setting. Our approach introduces an innovative model that reduces dependence on expensive infrastructure while expanding the capacity of healthcare professionals to oversee a greater number of patients.
To achieve this, the project focuses on integrating real-time and semi-autonomous monitoring, motor training, and rehabilitation supervision directly within the patient’s home. A key challenge in this research is the dynamic collection and processing of physio-metabolic and inertial data at runtime, aligning this information with diagnostic concepts that reflect the patient’s evolving health status and individual needs. SORTT seeks to bridge remotely acquired physio-metabolic data—gathered using advanced Internet of Things (IoT) and edge computing technologies—with an ontology-based system that represents the healthcare professional’s understanding of the patient. At its core, SORTT envisions a smart environment where multiple sensors capture a range of motor, physiological, and stress-related conditions. This setup enables accurate performance assessment and monitoring of the patient throughout a structured motor rehabilitation session. The smart environment comprises wearable sensors, integrating an IoT system alongside an ontology-driven framework for modeling patient data.
Several research projects have been andd are targeting this field. The SORTT project itself was initiated alongside the I-TROPHYTS project, both focusing on technological solutions to enhance remote rehabilitation. The I-TROPHYTS and SORTT projects investigate “intelligent” frameworks based on: IoT solutions [1] and ontological modeling [2, 3, 4] possibly integrated in a robotic architecture. The two research projects offer an homogeneous and partly shared bulk of technologies and aims including: (i) a common approach to the physical performance; (ii) a common source of financing (the Italian MUR/PRIN/PNRR Agency) and overlapping research groups; (iii) solutions for health care problems typically involving families and social communities.
More specifically, I-TROPHYTS aims at increasing the number of patients followed by a single medical professional, leveraging on the use of ontological modeling, IoT communications and humanoid robotics to dispense with the physical presence of the physiotherapist. SORTT aims to involve patients having difficulties reaching and participating in standard medical facilities, leveraging the use of ontological modeling and IoT to remotely monitor rehabilitation activities (either at home or in rehabilitation facilities), and give rise to a sort of digital twin of the subject. In these projects, ontological modeling is intended to integrate data-centric and medical-centric aspects for a remote controlled and safe running of physiotherapy sessions. The goal is to continuously collect and analyze patients’ data from sensors via machine learning during an exercise session, to integrate these data to evaluate the health status of the patient, and to make this integrated information accessible to therapists and robotic devices at runtime, possibly alerting to warning conditions or poor executions.
To evaluate the quality of the exercise execution and to detect the occurrence of undesired situations, the ontology should classify and store the different types of exercises, the information on how they should be performed, the status of the patient, the safe range of medical parameters, and the list of joints and tissues whose activation is the target of each exercise.
The innovation we present here lies in the implementation of an ontology-based model system for this purpose, integrated with this type of technology, specifically aimed at developing an (admittedly limited) Ontology-Based Human Digital Twin, OHDT. Previously, the ontological method has been applied only partially to the process—either in the collection of patient data or in interface support—but, to the best of our knowledge, not in the semantic control of the IoT system and LLM applied to telerehabilitation, as we will explore in the next section.
This paper focuses on the organization of the ontology and the data in SORTT. In the next session we review the literature in the field; in Section 3 we report the interaction with partners and the interview with a physiotherapist from the CMG health center; Section 4 describes the setting of the ontology and how to deal with qualities; in Section 5 we discuss the extension of the DOLCE ontology; the final considerations of Session 6 conclude the paper.
In the last decades, healthcare support has emerged as a significant societal challenge. Enhancing quality of life not only benefits individuals affected by various conditions but also contributes to overall physical and psychological well-being. Given the recent improvements of technology, there is growing interest in applying tele-monitoring systems and cutting-edge technologies to provide physical assistance. These innovations have been developed in various forms and applications. Our focus in this paper is on the use of foundational and applied ontology in medical, healthcare, and rehabilitation fields. Ontologies have been increasingly employed in the healthcare domain to structure complex knowledge, support decisionmaking, and enhance data interoperability. Several works have focused on developing ontology-based tools in this domain, whether motivated by recreational purposes [5] or care in relation to maintaining wellness or recovery following episodes such as heart attacks and traumas. Approaches to support rehabilitation exercises (not necessarily motor) or managing related knowledge may exploit the use of applied ontologies. These uses of ontology are often guided by pre-existing modules, especially where reference has to be made to already organized domain knowledge. Usually, the ontology is aimed at implementing support for planning and data management in protocols and scenario building. For example, the mixed-initiative planning (MIP) system SLOTH [5] is an assistance planning approach to support users in developing workout plans tailored to their fitness goals (modeled as abstract tasks). Through ontological reasoning capabilities, the system can determine whether a particular collection of workouts satisfies the specified requirements and subsequently generate a structured plan offering a large tailored set of workouts. The knowledge framework in SLOTH reasons on the ontology, which provides an effective formalism for representing conceptual hierarchies and inter-relationships among domain-specific concepts [5] while a statistically-driven approach, called language understanding intelligent service (LUIS), handles complex and natural user inputs. In the context of neuropediatric physiotherapy, [6] shows that ontology is helpful not only to categorize collected data, and analyze relations connecting them, but can also be a learning tool for medical students. Ontologies are also applied to manage patient data and assist in treatment planning on type 2 diabetes mellitus [7]. In [8] the formal model OPTImAL is introduced to address factors influencing cardiovascular patients’ adherence to physical activity, suggesting that ontologies can have a role in behavioral and treatment domains.
In the area of home-based rehabilitation, [9] introduces an ontology for cerebral palsy rehabilitation that integrates various treatment elements and aims at facilitating specialists’ decision-making when prescribing therapy exercises to patients. Physical therapy is for enhancing the functional abilities of individuals with disabilities or physical impairments: the authors reuse existing ontologies in rehabilitation, such as the RehabilitationTreatment Specification System (RTSS) and the Ontology for Neurological Rehabilitation (NeuRO). Logical rules are designed to automate decision making processes within the ontology, ensuring that patients with limited functional abilities (as indicated by a low assessment score) are identified for environmental modifications to support their rehabilitation needs. It can also be implemented to monitor rehabilitation exercises done from home. Regarding physiotherapy, [10] developes TRAK, an ontology aimed at organizing knowledge for the rehabilitation of knee conditions. The works [11] and [12] investigate the use of ontology to support rehabilitation for individuals affected by musculoskeletal disorders, with the goal of helping patients regain their range of motion. The papers focus on upper-limb rehabilitation and are aided by a robotic system that guides and assists patients in performing correctly the prescribed exercises.
The approach of Berges et al. [13], which provides an ontology for physiotherapy aimed at systematically gathering and structuring patient-related knowledge and data, is particularly relevant for the project SORTT. This work focuses on organizing the information that constitutes a patient’s physiotherapy record, ensuring it is effectively modeled for easy consultation. Another key aspect of this research is the recognition of the importance of dynamically tracking goal achievement to support the patient’s recovery process over time. This includes managing both recommended and contraindicated exercises based on the patient’s condition, thereby assisting in the design of treatment plans and guiding physiotherapists in their decision-making during therapy sessions [13]. The semantic and ontological approach in this study is specifically designed to aid physiotherapists in various routine tasks, such as recording and retrieving patient information, structuring physiotherapy protocols for specific disorders, and selecting appropriate exercises for each treatment phase. In essence, this research focuses on organizing data, structuring patient-specific knowledge, and planning therapeutic interventions to support physiotherapists. The study relies on the TRHONT (Telerehabilitation Ontology), an application ontology that integrates the KIRESONT and GLENONT modules, along with additional physiotherapy-related data, all built upon the foundational FMA ontology (Foundational Model of Anatomy) [14]. Their objective is primarily to build a queryable database and organize exercise sets by constructing the patient’s history to support the physiotherapist. There seems to be no clear intention to use the ontology in an active and autonomous planning mode, which is deferred to the physiotherapist’s decision, and above all there is no reference to a foundational ontology for modeling relations and remote semantic control of the patient’s action. Also, this approach does not consider to link data and sensors from which they originate. Moreover, the representation of the "record" associated with a patient (that collects all the information about that patient) seems a sort of large reification of all the information available about the patient, in the sense that they "subdivide" the record according to the CQs. Looking at the CQs, it seems that each of these is aimed at identifying some property of the patient, of the patient’s movements, etc. Relatively to [13], the aim of SORTT is to develop a more comprehensive and foundational system which includes also an epistemological perspective: data is collected by instruments, which have characteristics that qualify the data they produce; data can be conflicting and may require to be mediated/cleaned up; IoT systems change over time and data should not be merged without due analysis of the capacity of each IoT system versioning.
Regarding the more terminological part related to the types of exercises, the exercises used, the protocols to verify the progress of a certain patient, the telerehabilitation system, an interesting work is [15]. It implements a system called KiReS, a Kinect-based system which covers, on the one hand, the needs of physiotherapists and, on the other hand, the needs of the patients, by providing them an intuitive and encouraging interface for performing exercises, which also gives useful feedback to enhance the rehabilitation process. On effective remote interventions, this study focuses on a Kinect-based algorithm for monitoring physical rehabilitation exercises. It highlights the effort to capture an agent’s motion using Microsoft’s Kinect tool (now discontinued). This research has relevant overlaps with our study, particularly in its approach to shoulder joint exercises. It defines initial and final positions, as well as movement trajectories, utilizing a statistical observation framework to assess execution accuracy through extrinsic semantic analysis. However, from the system description, the semantic reasoning does not aim to evaluate the correctness of the exercise [15]. It concentrates on movement capture, statistically processing a high record of experimentally constructed material to be able to deduce the correctness of the movement expressed by the patient, in which, based on certain algorithms on partially quantitative and partially qualitative information, a metric is determined to establish what type of exercise is being performed and the deviation from what is recorded as correct, thus directly expressing exercise monitoring on a statistical (not semantic) basis. The difference is relevant. In a remote physiotherapy rehabilitation session, it may be necessary and useful for greater patient protection and better performance of the designed system to implement a double control - statistical via sensor and semantic through ontology - anyway to be designed as superordinate, in a control room position to support the physiotherapist who, in synchronous or asynchronous mode, evaluates the quality of the exercise and the overall physiological data of the patient and the general progress of the session.
Another interesting use of ontology for physical exercise is on helping in the annotation and explication of instructional texts of physical exercises. In [16] the design of an ontology is proposed which incorporates the information on the human body movement that is helpful to analyze implicit constituents in exercise instructions. Like the previous, this approach is also not aligned with a foundational ontology. The approach in SORTT differs also because it focuses on an intrinsic interpretation of the human body engaged in exercise. We emphasize the relationships between exercise-related qualities and affected anatomical parts, directly anchoring them to the foundational DOLCE ontology. Additionally, we have developed a tailored extension to identify quality and relational spaces. In Section 4, we provide a detailed overview of our system’s construction, demonstrating how our ontology-based approach to developing an OHDT represents an innovative contribution to digital twin healthcare literature, particularly suited to medical applications where transparency and cognitive accessibility are crucial.
To ground our ontology model in actual clinical practice, beside a literature survey on the relevant topics, we conducted interviews with domain experts (physiotherapist and medical doctor) from the Centro Medico Galliera s.r.l. (CMG), a center specialized in physiotherapy treatments and involved in the project as expertise provider. The aim was to elicit the aspects that, theoretically but especially practically, they consider and monitor before and during a physiotherapist session.
The therapist begins each treatment plan by gathering the patient’s medical history and asking questions about the patient’s lifestyle, clinical framework, work, and physical activity. The gathered knowledge helps to identify possible risk factors and to adapt treatment accordingly. Based on this knowledge and the overall goal, a physiotherapist assigns specific exercises, because textually "it is rare for two patients with the same issue to receive identical treatments or exercises." This emphasizes the need for an ontology able to accommodate personalization in the treatment plans.
During exercises, the physiotherapist constantly checks the state of the patient to ensure there are no alarming signs of distress. You can tell visually if a patient is having some kind of problem (like tachycardia or difficulty breathing) through signs like cyanotic lips, excessive sweating, dizziness, and so on. There is a shared agreement that without an expert in the room, these observational cues need to be substituted by sensors that measure vital signs like heart rate, blood pressure and oxygen saturation, therefore the ontological model needs to account for this type of information and the IoT system should be able to measure (perhaps indirectly, depending on the available sensors) such parameters. Throughout the therapy process, patient performance is constantly evaluated: “It often happens that the exercises planned for the patient need to be modified,” depending on how the patient performs the exercises during the session. If a therapist is not present to check the overall performance of the patient, the model needs to acquire and store performance data and make it accessible to a professional that can adjust, simplify or change the exercises as needed. Indeed, each exercise has its own evaluation criteria with which to establish whether it has been executed correctly or not. These criteria ensure that the intended muscles are activated as needed while avoiding ineffective or harmful movements. A domain ontology for this activities should have the capacity to store the data that identifies correct performances of each exercise, and connect this to the data derived from the sensors making it possible to check whether the ongoing execution is as expected.
The interview with experts revealed a complex and articulated picture of rehabilitation practice, providing valuable insights for the development of an ontological model. As anticipated, the preparation of a physiotherapy session begins with an accurate patient anamnesis. Physiotherapists emphasize the importance of collecting information about the patient’s history, lifestyle, and specific characteristics. This initial phase includes mechanical and functional tests that allow for establishing the actual state, clear objectives and targeted action programs. A fundamental aspect that emerged is that exercises are built specifically for each patient, even when dealing with the same clinical problem: the evaluation process is systematic and includes examination of body structures to identify any alterations, assessment of articular or muscular motor deficits.
Documentation plays a crucial role: physiotherapists maintain structured lists of questions for anamnesis and use personalized tables for data collection. Following the protocol, the date, patient conditions, and exercise evolution are systematically recorded, including specific parameters such as the bending angle measured with an ad hoc goniometer. During the sessions, evaluating the patient becomes a continuous and multimodal process. From the second session onward, physiotherapists primarily use visual assessment, observing how the patient enters, walks, sits, evaluating skin color and asking the key question: "how are you?". Verbal communication is a very important elicitation step in this process and it is still unclear up to which level it can be substituted in the IoT system. This aspect requires further analysis since physiotherapists interpret not only what the patient says, but also how they say it and how they behave. Elements such as dizziness, fatigue, visual alterations, or intercurrent episodes (viruses, accidents) may radically change the session approach.
During an exercise execution, physiotherapists constantly monitor various parameters. Breathing emerges as an element of crucial importance which cannot be overestimated. As the experts explain, you must monitor how the patient breathes by watching the abdomen, especially in obese patients but also in normal-weight ones. Incorrect breathing can lead to apnea, reducing oxygenation and compromising exercise effectiveness. Warning signs that require particular attention include: dizziness (requiring immediate interruption, controlled breathing and evaluation), breathlessness (requiring immediate pause), body temperature alterations (particularly after surgical operations), nervousness or agitation (especially in neurological patients), nausea or even vomiting (requiring immediate interruption).
Pain management represents a central aspect of physiotherapy practice. Professionals use standardized pain scales (VAS or NRS) from 1 to 10, and when a patient reports a level 7 or higher, they proceed with immediate evaluation to decide whether to interrupt, change the load, or suspend the exercise. Absolute priorities requiring immediate attention are: pain, dizziness, nausea, and vomiting. These "red flags" indicate that something is not working and requires thorough investigation.
An interesting aspect concerns the relationship between personalization and exercise standardization. While there are absolute principles for correct exercise execution (for example, an exercise must be done in axis, off-axis indicating a problem), patients may present characteristics that require adaptations. Anamnesis serves precisely to create a tailored program, considering factors such as patient’s body or previous surgical interventions. For example, for a patient with hip prosthesis, certain exercises might not be appropriate and require variations or complete substitutions.
Important considerations emerge regarding the ethical aspects of the profession. It’s not always appropriate to report sensor-detected data to the patient, as this could generate anxiety or panic. The decision on what to communicate and what not falls within professional ethics and depends on the specific situation. Empathy and patient motivation play fundamental roles in treatment effectiveness. Physiotherapists must balance the need to be transparent with that of maintaining a positive and motivating therapeutic environment.
As this report shows, the integration of tele-rehabilitation systems in physiotherapy for obese and diabetic patients with remote IoT control presents several challenges. In summary, the collected data suggests that the system should be able to: adapt in real-time to patient conditions; monitor vital and biomechanical parameters simultaneously; interpret verbal and non-verbal signals; personalize exercises based on anamnesis and previous performance; manage emergency situations with appropriate safety protocols.
Building an ontological partial human digital twin requires to represent the integrity of the patient characteristics, values, and physiology merging information coming from sensors. Also, the system has to master knowledge of the physiotherapist that is today tacit, and recognize the signs that require to stop, to repeat an exercise correcting the execution, or to pause. The contribution of the ontology here is both on correctly interpreting the data coming from sensors, and on acting based on this interpretation.
The CMG center granted us access to recordings of several exercise executions to observe and model the ‘standard’ exercise. To deepen the domain knowledge we referred to domain literature (as in Section 2) and physiotherapy manuals (e.g., [17]). It is important to combine technological precision with human sensitivity, creating systems that can support physiotherapists without replacing the essential human element of the therapeutic relationship.
From the information collected from the literature and direct interactions with experts, one useful application of an ontology in this domain is in supporting the physiotherapist in making decisions and planning physical sessions.
We gathered enough information that indicate how the ontology can help in the planning of the sessions, the management of patient’s data, the evolution of the patient’s clinical situation, and consequently the variation of the exercise program. Once an exercise is fixed, there are no factors related to the patient’s medical history that may influence how that exercise should be monitored; in fact, an exercise must be carried out in a specific way, which is the same for everyone. For example, if an abduction exercise requires raising the arms to 90 degrees and one person raises them to 75 degrees while another raises them to 100 degrees, this does not change the fact that the exercise has been performed correctly in both cases, even if to varying degrees of “quality”. Indeed, the key is to verify that the specific muscles which are the target of that exercise are effectively activated during the exercise execution.
As suggested by physiotherapists, it is essential to define a threshold below which the exercise becomes ineffective (e.g., if the arms do not reach 50 degrees). If the patient cannot exceed this threshold, a different exercise should be assigned. However, it would still be necessary to distinguish between a person who reaches 45 degrees because it is their physical limit and another who reaches 45 degrees due to fatigue or lack of motivation to exert themselves.
Another goal is to support the comprehensive representation of the information related to the execution of physical therapy exercises, from ideal movements and sequences of exercises to actual patient performance and status. A therapeutic program may have different goals. One goal could be to strengthen muscles, while another could be to improve flexibility or breathing capacity. The combination of exercise and goal determines how the exercise should be evaluated. In the first case, the number of repetitions is increased, while in the second, the degrees of movement achieved by the patient are monitored. So, again, the patient’s medical history determines a goal, which in turn determines the exercise. We concentrated on establishing a classification of goals to decide whether to focus on monitoring muscle movement or achieving a specific position. We started from the scenario that was intended to be modeled: a patient doing physiotherapy, performing a given exercise. The first thing to note is that the patient starts in an assigned position which is described to him/her (standing at a certain point, sitting on a given chair etc.), which needs to be stored in the ontology (in Fig. 1 an image taken from an abduction exercise with the persons raising their right arms laterally at 90° degrees).
Now we focus on how a specific exercise can be characterized in terms of the qualities of its instances and of the participants to its instances as envisioned in our approach. We focus on simple exercises: a person sitting on a chair with her arms down along the body slowly raises her right arm, without bending it, with a 180-degree lateral rotation up to above her head. For the sake of simplicity, we cover only the part of the exercise consisting in the lateral movement of the arm: a shoulder abduction that moves the arm in the frontal plane with a range of approximately 180 degrees. The movement starts at 0 degrees when the arm is positioned at the side of the body, progresses to 90 degrees when the arm is horizontal at shoulder height (as shown in Fig. 1), and reaches 180 degrees when the arm is completely raised overhead. Other exercises, like when the arm performs an anterior movement, (shoulder flexion) can be modeled similarly. Given that there are several types of information to manage, we have identified four key elements (classes) around which to model the rest of the ontology: the body parts, the body qualities, the sensor data, and the exercises.
The human body. The body of the patient is taken as a whole and, given the modeling context, three types of parts are considered: tissues, organs (like muscles and bones) and joints (articulations). The class of tissues can be further divided into soft (primarily, those of muscle) and hard (primarily, those of bones). Coherently, the organs here considered are the bones and the muscles. Joints are complexes of muscles and bones with constraints on relative movements. The idea is to be able to indicate which tissues, organs and joints are the focus of a movement in an exercise.
The body qualities. These describe the status of the body and of the parts as indicated above. Here we ifnd information like temperature, blood pressure, muscle tension and position. Depending on the IoT system one implements, some of these data are taken directly from sensors and others are the output of Machine Learning (ML), e.g., usually temperature is measured directly while the body or joint position are calculated via ML (in ways that depend on the IoT system).
The sensor data. This part of the ontology collects information about the characteristics of the devices in the IoT system and the implemented data processing algorithms.
The exercises. This class in the ontology stores information about the exercises a patient may be asked to perform. Each exercise is identified by some features: the initial position of the body at the start of the exercise, the final position, the trajectory drawn by the relevant limbs between the starting and the ending times and, depending on the exercise, some intermediate positions that must be assumed in between.
These different classes are connected by dedicated relations in the ontology. For instance, the position assumed by a patient at the start of an exercise is compared with the position that is expected at the beginning of that exercise. To do this we take from the class of body qualities the position assumed by the patient at the beginning of the exercise (e.g., computed by the IoT system) and compare it with the position stored in the exercise class, where there are the expected positions. Further relevant positions can be checked similarly to establish whether the exercise is executed correctly or not. Overall, this data together with data about the patient’s health gives a snapshot of the patient’s status and activity during the session. Before concentrating on the technical details of the ontology, we give an example of how we model therapy exercises through the classes that we added. The details and axioms defining these classes are laid out in the following paragraph. Consider a specific exercise movement, for instance, the exercise which consists in Rising the Right Arm from a Sitting Position. Call RRASP the class which collects all the executions of this exercise. By definition, RRASP is a subclass of the class EXERCISE (informally, the latter collects all the executions of all exercises). We want to model a specific execution of this exercise by our target patient, to monitor how well this exercise is done, and to control her health status during the execution. As anticipated, a member of RRASP is an execution of the exercise described above, i.e., the event in which a human body executes all the actions (in the given order) that compose that type of exercise. The human body is a member of the class BODY, a subclass of BIOLOGICAL ENTITY. We now concentrate on the execution of exercise RRASP performed by a person, we call her Sara, on February 10, we call it Sara-rrasp-10/02. Note that Sara in this discussion is, to all effects, not a person but just a human body. This is obviously a simplification. In particular, the human body Sara participates in the event Sara-rrasp-10/02 and is the only human body participating in it. The human body Sara has physical qualities that are physiologically relevant, such as body TEMPERATURE and BLOOD PRESSURE. Others are relative to the movements she does, such as POSITION and MUSCULAR TENSION. These four are all subcategories of the category PHYSICAL QUALITY and, together with many more that we do not consider, are qualities of Sara’s body or of its parts. Sara’s qualities persist throughout the ◦ ED
PD
Q PED
NPED
AS
EV
STV
TQ PQ
AQ
AB R TR M
F
POB
NPOB ACH
ACC
ST PRO TRJ TL SL TMP
POS
PRS
TNS PR
AR APO NAPO
MOB SOB
EXC
TMPr POSr PRSr TNSr S TRJr T BIO
JNT DEV
ASO NASO BDY ORG
TSS
SAG
SC performance, but their values may change as she moves. If Sara-rrasp-10/02 was performed between 16:00 and 16:05, her temperature Sara-temperature, may have value 36.5° at the beginning of the exercise (at 16:00), and value 36.7° at the end of the exercise (at 16:05). These values correspond to positions in the quality space of temperature, which we call TEMPERATURE REGION. The value of a quality, in this case of temperature, can be a point (in the ideal case) but usually it is an interval (the range that takes into account the tolerance of the sensor) since data is epistemic information with some level of uncertainty. More generally, all qualities that are measured are treated in this fashion.
When performing the exercise, Sara’s right arm movement corresponds to a 3D path in space, which we assume is elaborated by the IoT system, and is stored in the category TRAJECTORY of the ontology. The trajectory of Sara’s body is a quality of Sara’s performance and takes value in a dedicated trajectory space, called TRAJECTORY REGION. The TRAJECTORY REGION also contains the ideal path for the exercise so that the two can be compared. (In our view this comparison is done via machine learning, but it is useful to keep this information in the ontology.)
Every exercise requires moving certain parts of Sara’s body. To keep track of the purpose of each exercise, we introduce the JOINT, ORGAN, and TISSUE categories, which are subcategories of BIOLOGICAL ENTITY. Each exercise is associated with the information of which tissues (e.g. muscle tissue), organs (e.g. muscle), and joint (e.g. shoulder) should be activated and related parameters.
The sensors used to collect data about Sara’s movements and conditions form a subcategory of the category MEASURING DEVICE. Information about the sensors (and the algorithms used to elaborate the data) is collected because it can be useful to indicate the precision and reliability of the data in the IoT system. This allows also to compare exercise recordings that use distinct IoT systems (due, for instance, to a change or upgrade of the system).
In the next section, we present the extension of the DOLCE ontology with the categories to model this exercise and its monitoring values and spaces of qualities.
The domain ontology we have developed is conceived as an extension of the Descriptive Ontology for Linguistic and Cognitive Engineering (DOLCE) [18, 19]. DOLCE is a foundational ontology that aims to capture the ontological categories that emerge in a cognitive view of reality. Indeed, it is inspired by natural language and common sense, making it a particularly suitable approach in modeling knowledge about people and standard practices.
The management of qualities is particularly important in this work. Referring to the example about angles in the previous section, we see that their classification is relative to initial/final positions and detected/computed trajectories, moreover the values may be collected by different sensors and elaborated by different algorithms. DOLCE provides a flexible way to represent individual qualities and their values. In particular, the same individual quality can be evaluated in different measurement spaces (called quality spaces) with distinct structures such as discrete, continuous, topological, or metrical. In an epistemological perspective, the values included in the spaces, as well as their structure, may depend on the resolution of the available sensors. This flexibility further amplifies the possibility to expand the monitoring perspective and to compare and integrate data from different IoT systems.
As we have seen, while the criteria for the correctness of an exercise are general, the ability to execute an exercise, and to execute it correctly, depends on the individual patient [20]. To complete an exercise correctly, a muscle must be activated to a certain tension; however, the choice of a specific exercise, number of repetitions, and tolerance levels are not independent of the patient’s age, conditions, and pain levels [21]. For exercises on joints (e.g., raising the left arm), the possibility of performing the exercise correctly depends on the patient’s condition. The quality spaces associated with a category of exercises (e.g., raising the left arm) can be further qualified by the exercise’s evaluation criteria, allowing to decide whether a given execution corresponds to a good performance or else relatively to the specific objective for that patient. This enables conceptual and implementation simplification by avoiding having to hypothesize a different category of exercises for every possible situation and patient (e.g., the patient’s initial state and the goal/objective of the physiotherapy cycle). Used in this way, quality spaces can structure a set of data in the needed manner for remote monitoring and for decision-making in planning the rehabilitation program, or simply for maintaining the predefined level of wellness in the patients.
Fig. 2 illustrates the DOLCE taxonomy extended by new categories (in red and underlined) which has been added to represent different types of exercises and their participants, as well as some relevant participant qualities.
Given the aim of the ontology and the relevance that the position of the body has during the physiotherapist session, a quality POSITION (POS) is included as subcategory of PHYSICAL QUALITY (PQ), i.e., as a quality of physical objects. To maintain a certain position and to change position, various parts of the human body are involved. To store this information and to be able to identify which parts of the body are engaged, we add two subcategories of DOLCE’s NON-AGENTIVE PHYSICAL OBJECT (NAPO), namely MEASUREMENT DEVICE (DEV) and BIOLOGICAL ENTITY (BIO). This means that the IoT and the human body (including its parts) are included in the ontology as physical objects with qualities relevant for the physiotherapist purposes. Thus, they are here included as mere technological and mere biological entities, they are not full-fledged agents (the ontology remains open and can be expanded adding categories for these agents, if needed).
DEV collects the physical entities that produce the data used by the system. It is useful to represent the provenance of the data by connecting the values of the qualities of participants with the sensor network that evaluates them. Measurement devices could consist of both physical and software components, e.g., a set of sensors together with the algorithms that elaborate their outputs to provide a measurement value. As a simplification hypothesis, we consider neither the structure nor the software components of devices.
BIO is the subcategory BIOLOGICAL ENTITY. Here we are interested in HUMAN BODY (BDY), and some kinds of parts of bodies, namely, TISSUE (TSS), ORGAN (ORG) and JOINT (JNT). We focus mostly on two types of tissue: bone tissue and muscle tissue, which constitute the bone and the muscle organs. Organs, in turn, are part of the joints, which are structures made of different organs: primarily bones and muscles but one can expand it to include tendons, cartilage and so on. To make a movement, the body changes a joint status by using muscles via varying the tension in the corresponding tissues. To keep track of this, we add another subcategory of PHYSICAL QUALITY (PQ) called TENSION (TNS). After a movement the patient moves into another position. In transitioning from one position to another, the body follows a trajectory. It is important to trace the trajectory because, depending on the trajectory’s path, the therapist can determine whether the exercise was performed correctly or not. This kind of qualities of events is called TRJ and is a subcategory of DOLCE’S TEMPORAL QUALITY (TQ).
The representational mechanism of qualities and quality spaces present in DOLCE is very useful to talk about the structure of the values of a given property (e.g., temperatures can be linear while the positions of bodies are typically structured in a multi-dimensional space, which in turn can be topological, metrical, geometrical etc.). The idea is that an entity can have several individual qualities—each one capturing a specific aspect of such entity, e.g., its temperature, position, etc.—that are located in one (or multiple) qualities spaces structuring the values of the aspect represented by that quality. For instance, to represent the specific temperature of a body b, we introduce the individual quality tb (representing the temperature of b) and associate it to a location in the temperature space (called region in DOLCE), say 37C. In DOLCE, the link between the individual quality tb and b is called (direct) quality(-of) (DQT) while the link between tb and 37C is called quale(-of).
First, observe that when two different bodies b and b′ have the same temperature of 37C, there are two different individual qualities tb of b and tb′ of b′, respectively (formally, DQT(tb, b) and DQT(tb′, b′)), which are both located in the same region 37C. Individual qualities are always linked to (inhere in) a single entity, independently of their associated location. Second, DOLCE distinguishes physical and temporal qualities (PQ vs. TQ). Physical qualities are linked to Physical Endurants (PED), i.e., commonsense objects like bodies, persons, sensors, tables, etc., while temporal qualities are linked to Perdurants (PD), i.e., events or processes, like the specific execution of an exercise by a person. Since objects can change their properties through time—i.e., their qualities can change values—the location relation has a temporal argument for physical qualities (in this case it is called tQL), which is not needed for temporal qualities (in this case it is called QL). Third, the link between quality and quality spaces in DOLCE allows to represent ‘partial’ information about (a quality of) an entity. For instance, a given thermometer could have a coarse resolution, i.e., it clusters into a single measurement output an interval of temperature values, e.g., it is not able to systematically distinguish 37.01C from 36.99C. The idea is (i) to represent the maximal resolution via the atomic regions of a space (regions that cannot be further decomposed); and (ii) to allow a quality to be located at a larger region, representing the fact that the available information is limited in terms of precision. We consider a few subcategories of PQ useful for the examples we will take into account: TEMPERATURE (TMP), POSITION (POS), BLOOD PRESSURE (PRS), and MUSCULAR TENSION (TNS). Concerning temporal qualities, the subclass of TQ named TRAJECTORY (TRJ) is enough, see Fig. 2.
As required by DOLCE, these new kinds of qualities must be linked to at least a quality space where the qualities are located, i.e., take values. The spaces to locate TMP-, POS-, PRS-, and TNS-qualities are called TMPR, POSR, PRSR, and TNSR, respectively. They are all subcategories of PHYSICAL REGION (PR). The space TRJR to locate TRJ is a subcategory of TEMPORAL REGION (TR), see Fig. 2.
Another category of interest is EXERCISE (EXC), a subcategory of DOLCE’s ACCOMPLISHMENT (ACC). ACC is a subcategory of PERDURANT (PD) whose instances are characterized by the fact that they (i) can be divided into smaller perdurants and (ii) the aggregation of two accomplishments of a certain type is not an accomplishment of the same type. In fact, a sequence of exercises of some type is not an exercise of the same type (e.g., two different marathons are not a marathon), and exercises are made from various steps. Regarding time, which characterizes qualities of events, DOLCE includes the category TEMPORAL INTERVAL (T) but does not commit to a specific temporal structure. For our scenario, it seems enough to adopt a notion of time which is atomic, discrete, and linear.
A characteristic of perdurants (e.g., events, processes, states) is that they can be ‘sliced’ through time, e.g., the first, middle and the last part of a perdurant are all perdurants (perhaps of different types, e.g., the ifrst quarter of a basketball match is still a perdurant but not a basketball match). This characteristic of perdurants allows to introduce a link between the trajectory of a perdurant and the position of the maximal physical object participating in it. To simplify the scenario, consider a perdurant e which is the execution of a specific exercise having a single and constant physical participant, say body b (a scenario that is quite normal for the type of exercises we are considering). The idea is that, the positions of b during e and the trajectory of e are interlinked, i.e., the trajectory of the whole execution e is the ‘sequence’ of the positions of the body b during such execution. More specifically, given an atomic temporal interval t (at which e exists), the trajectory of the temporal slice s of e at t corresponds to the position p of b at t, i.e., p is the projection of the trajectory of s into the space of positions.
Another useful extension of DOLCE concerns the possibility to understand the location in a quality space not as purely ontological ‘truth’ but as the result of an empirical measurement. The idea is to introduce an epistemological version of the quale relations QL and tQL by adding an argument representing the measurement device used to locate the quality in the space (i.e., the measurement device that generates the data). Then, QLE(q, r, d) stands for “according to the measurement device d, the quality q is located at the region r.” Similarly for tQLE(q, r,t, d) where t is a temporal interval. In case of simple devices like a thermometer or a blood pressure sensor, the link between the output of the device and the location in the temperature (or pressure) space is quite direct. One still has to take into account the resolution/precision of the device that could require to represent the measurement by considering a non atomic region in the space. In this way, one can represent the fact that, modulo the device’s precision, we don’t know the exact value of the quality, we only know that such value is included within a given range.
More difficult is the case of complex quality spaces that elaborate and integrate data coming from several sensors. In our scenario, this can be modeled combining positions and trajectories. First, the notion of position of a physical object seems intuitively connected that of shape. The way shapes can be described and structured has been studied since the 1980s, and in general it requires a quite complex and expressive framework. In the specific case of bodies, the relevant information is given by the relations between the joints, in terms of the tension of muscles, etc.
For example, the body’s features are collected and computed during the data collection phase using dedicated devices. Transmissions occur at a given frequency according to a predefined schedule. The sensors from which raw data are collected register, for example: (i) acceleration; (ii) rotation; (iii) heart rate; (iv) blood pressure; (v) temperature, etc. The collected data relate to the categories of (a) Exercise, which represents the type of exercise performed (here too, following common research development agreement, shoulder flexion and adduction exercises have been tested); (b) Side, which identifies the body side; (c) Timestamp; (d) Time, the number of seconds of the session; (e) Number of repetitions; ( f ) Degrees of the angle between the body and the arm; (g) Direction, as well as heart rate and heart rate frequency measurements and so on. The data collected and analyzed through dedicated devices are processed using machine learning where needed. Traditionally, machine learning algorithms have not been able to incorporate background domain knowledge, and work with a sequence of instances, so there exists the issue of learning from more complex data and recognizing consistencies, discrepancies, misalignments, and anomalies among the data. Here the ontology is integrated as a flexible wrapper for more efficient machine learning and knowledge inferences [22].
Provided devices that measure such relationships are present, the ontology can infer the position of the body from the available data. Nonetheless, it is hard to precisely establish the complex rules that must hold between joints and it is even harder to fully represent them in a formal and computationally feasible way. Delegating this to machine learning algorithms, one may avoid the difcfiulties above by considering the following strategy: (i) individuate the body positions relevant for the scenario (formally represented in the system via regions in the quality space of positions); (ii) use examples of such positions via supervised learning; (iii) fix the ‘precision’ of the system composed of the sensors and the machine learning mechanism by evaluating the discriminatory power of the system. One can then map the outputs of the systems to the regions of the position space without describing them in terms of simpler data. This makes also possible to consider systems combining different sensors and algorithms within the same position space. The approach works for trajectories also. Here, it seems that a given (type) of exercise is in a one-to-one relationship with a given trajectory. Thus, in this case the trajectory can be seen as a definitional parameter for the exercise.
It is natural to represent a type of exercise as the collection of all the possible executions of such exercise. Such a type is then a subcategory of perdurants whose instances have in common the pattern of the exercise (but we would no need to characterise the pattern). A simplifying assumption, given the scenarios we address, is that all exercises have a single body as maximal participant.
This contribution discussed the SORTT project and, more precisely, the development of a DOLCE module to allow remote supervision of patients by healthcare professionals. The research project aims at facilitating the participation in physiotherapy sessions and rehabilitation activities especially for obese and diabetic patients. The construction of an ontology-based partial human digital twin serves to manage the domain knowledge involved in the process, to monitor the safety of the sessions as well as the correctness in performing the exercises.
Our analysis of the state of the art highlighted how attention to medical and wellness support solutions (both therapeutic and maintenance, or even recreational) for remote patient control is growing, and technological advances in software and hardware allows for innovative solutions and systems. At the same time, it emerges that the use of applied ontology is increasingly considered in these domains and in managing related knowledge.
We have seen that the domain is articulated and fairly complex to model. Our work focused on isolating and defining the ontology’s purpose and focus. The work started from the problem of modeling a typical exercise relying on observations, registered sessions, existing literature and expert interviews. This led to propose an extension of the DOLCE ontology for which we gave the main motivations and intended use.
At the best of our knowledge, the literature to model and control this kind of scenario presents gaps between theoretical and computational considerations, since these two aspects have not yet been addressed simultaneously. We also note that the discussion regarding quality spaces, as anchored to the organization of the DOLCE ontology, adds a further level of flexibility with respect to proposals in the literature.
Overall, the interaction between ontology and machine learning described in this paper is still under investigation, the SORTT system is in a development phase, and further data have to be shared and tested. The different sensors can express diverse conditions that need to be tested and better aligned with each other. Machine learning operates the extraction of raw data, while the ontology enables the coexistence within the same world view of data derived from different sensors, creating a coherent and consistent synthesis, verifying logical consistency and signaling discrepancies between detected conditions that may indicate, for example, health conditions, poor exercise execution, or sensor malfunction. The relationship between ontology and machine learning can be described as a hierarchical organization where the ontology is positioned at a control and verification level of the knowledge emerging from sensors and the initial ML processing, in order to guarantee effectiveness and safety during the remote control of physiotherapy sessions.
The next steps include the construction of a demo, verification of the feasibility, and acquisition of further feedback from medical and physiotherapist personnel once the ontology is suitably populated. While the development of a complete tool for effective and safe execution of rehabilitation activities requires further work, the material here presented increases awareness of the complexity of these systems, and proposes a more inclusive and flexible modeling approach, features that are highly relevant in our aging society.
This work made use of DeepL Write for: Grammar and spelling check, Paraphrase and reword. The authors reviewed and edited the output as needed and take full responsibility for the publication’s content.
This research was supported in part by projects PRIN PNRR SORTT (Prot. P2022H74YP) and PRIN I-TROPHYTS (Prot. 20224TAETP).
[1] L. Fan, Usage of narrowband internet of things in smart medicine and construction of robotic rehabilitation system, IEEE Access 10 (2021) 6246–6259. [2] S. Borgo, An ontological view of components and interactions in behaviorally adaptive systems,
Journal of Integrated Design and Process Science 23 (2019) 17–35.