Explainable AI aims at building intelligent systems that are able to provide a clear, and human understandable, justification of their decisions. This holds for both rule-based and data-driven methods. In management of chronic diseases, the users of such systems are patients that follow strict dietary rules to manage such diseases. After receiving the input of the intake food, the system performs reasoning to understand whether the users follow an unhealthy behaviour. Successively, the system has to communicate the results in a clear and effective way, that is, the output message has to persuade users to follow the right dietary rules. In this paper, we address the main challenges to build such systems: i) the natural language generation of messages that explain the reasoner inconsistency; ii) the effectiveness of such messages at persuading the users. Results prove that the persuasive explanations are able to reduce the unhealthy users' behaviours.
Explainable Artificial Intelligence (XAI) aims at explaining the algorithmic decisions of
AI solutions with non-technical terms in order to make these decision trusted and easily
understandable by humans [
An interesting domain for XAI is healthcare, in particular the management of chronic diseases such as heart disease, cancer and diabetes. These are responsible for approximately 70% of deaths in Europe and U.S. each year and they account for about 75% of the health spending1. Such chronic diseases can be largely preventable by eating healthy, exercising regularly, avoiding smoking, and receiving preventive services. Prevention would help people stay healthy, avoid or delay the onset of diseases, and keep diseases they already have far from becoming worse or debilitating; it would also help people lead productive lives and reduce the costs of public health. The challenges of an explainable system that supports users in following an healthy behaviour are: i) the ability of providing a clear and comprehensible message regarding user’s behaviour, and ii) the effectiveness of the message to persuade the user at adopting an healthy lifestyle. This is fundamental as often people do not know the importance of following diet rules, hence they may not be sufficiently motivated to adopt healthy behaviors. Differently from the case of the power system, here the message must be persuasive and personalized in order to keep people engaged in using the system.
In this paper we present a XAI system based on logical reasoning that supports the
monitoring of users’ behaviors and persuades them to follow healthy lifestyles2. The
concepts and rules of healthy behaviors are formalized as a Tbox of the HeLiS
ontology [
The rest of the paper follows with Section 2 that provides a state-of-the-art of techniques for generating explanations from reasoning inferences. Section 3 shows the reasoning process that checks if a user has a healthy dietary behaviour. Section 4 describes the developed template system for the automatic generation of natural language persuasive explanations. Section 5 presents the Key To Health project in which we deployed the system, whereas Section 6 shows its evaluation. Section 7 concludes the paper. 1 http://www.who.int/nmh/publications/ncd report full en.pdf 2 This work is compliant with good research practice standards. More details at: http://ec.europa.eu/research/participants/data/ref/fp7/89888/ethics-for-researchers en.pdf http://www.who.int/medicines/areas/quality safety/safety efficacy/gcp1.pdf 2
Explainable Artificial Intelligence (XAI) generally relates to strategies able to provide
human-understandable descriptions of learning algorithms usually perceived as black
boxes by users [
One of the first user studies dealing with explanations for entailments of OWL
ontologies was performed by [
In order to gain an understanding of how OWL users interact with ontology axioms
and constructors, the work proposed in [
Besides justifications, formal proofs are considered to be the most prevalent
alternative form of explanation for logic-based knowledge bases. In [
Finally, as ontologies are often considered to be technical artifacts akin to software,
we may regard ontology and justification comprehension as analogous to software
comprehension. There has been a significant amount of work on predicting the complexity
of understanding and the ease of maintaining software. In particular, seminal work
described in [
In summary, there has been a wide range of approaches to explanation in the areas of ontologies, logics, and software comprehension, with some user studies that aim at evaluating the effects of supporting techniques. However, to date there have been no studies dealing directly with the impact on users’ behaviors of explanations from OWL ontologies such as the one presented in this paper. 3
The explainable model implemented within the HORUS.AI platform relies on two main
components: the HeLiS ontology [
In particular, the HeLiS ontology contains knowledge about the dietary (i.e. taxonomy of food categories and food compositions) and physical activities (i.e. effort needed for accomplishing a specific activity) domains.
Monitoring knowledge defines in the Tbox the set of rules enabling the monitoring task and the detection of undesired behaviors (hereafter called “violations”). User knowledge defines in the Abox the concepts describing user profiles and the data populating the knowledge base, i.e., food consumed and activities performed by users.
An undesired behaviour given by the union of Tbox and populated Abox will trigger
a logical inconsistency of the monitoring knowledge that has to be explained. In this
paper, we do not present the full modeling process and the content of HeLiS. The reader
can refer to [
The second component is the reasoner. Reasoning in HORUS.AI has the goal of
verifying if user’s dietary actions are consistent with the monitoring rules defined by
domain experts, detecting and possibly materializing violations in the knowledge base,
upon which further actions may be taken. Reasoning is triggered each time a user’s
profile or associated data are added or modified in the system, and also at specific points
in time such as the end of a day or week, to check a user’s behavior in such
timespans. We implement reasoning in HORUS.AI using RDFpro [
Figure 1 summarizes the knowledge graph generated by the reasoner. In order to
understand the remaining of the section, individuals of type Violation contain all
information about the unhealthy behaviors detected by the reasoner from user’s data.
While, individuals of type MonitoringRule contain information about the
recommendations that users should follow. The descriptions of the other concepts are in [
We organize the reasoning in two phases: offline and online. The offline phase consists in an one-time processing of the static part of the ontology (monitoring rules, food, and nutrients). This is performed to materialize the ontology deductive closure, based on OWL 2 RL and some additional pre-processing rules that identify the most specific types of each Nutrient individual (this information greatly helps in aggregating their amounts). Whereas, during the online phase, each time the reasoning is triggered (e.g., a new meal is entered), the user data is merged with the closed ontology and the deductive closure of the rules is computed. This process can be performed both on a per-user basis or globally on the whole knowledge base. The resulting Violation individuals and their RDF descriptions are then stored back in the knowledge base. The generation of each Violation individual is performed in two steps. First, information inferred by aggregating the domain, monitoring, and user knowledge is used for instantiating the Violation individual. Second, accessory information is integrated into the Violation individual for supporting the Natural Language Generation module in the generation of the explanation concerning the detected violation. Accessory information includes, for example, references to other individuals of the ontology enabling the access to the positive and negative aspects associated with the food category, or the number of times that the specific rule has been violated. This kind of information can be used for deciding the enforcement level of the persuasion contained within the generated messages.
The result of the reasoning activity is a set of structured packages containing information about the detected unhealthy behaviors. By considering as example the dietary domain, each package contains: (i) the list of meals that contributed to generate the violation; (ii) the actual quantity, for a specific food category, provided by the user; (iii) the expected quantity for the same food category; (iv) the violation level (this value gives a dimension of the violation, the higher the gap between the actual and the expected values is, the higher the value of the violation level parameter will be); and, (v) the violation history: the reasoner computes this value in order to provide a recidivism index about how a user is inclined to violate specific rules. This information, together with the identifiers of the violated rule and user, the rule priority, and the reference of the food (or food category, or nutrient) violated by the user, is sent to the persuasive explanation component that elaborates these packages and decides which information to use for generating the feedback sent to the user. An example of violation instance represented by using the JSON format is shown in Figure 2. violation: [ userId: fb267 violationId: violation_fb267 ruleId: MR-MEDITERRANEAN-028-QB meal: MEAL-58ccf3cbfd110f24e59eeced history: 1 expectedQuantity: 200 quantity: 300 unit: ml level: 1 timestamp: 1491063927420 priority: 1 rule: MR-MEDITERRANEAN-001-GWEEK entity: SweetBeveragesAndJuices entityType: FOODCATEGORY startTime: 1491043927420 endTime: 1491063927420 constraint: less goal: MEDITERRANEAN-GOAL-D-190
Here we present a method that performs a linguistic realization of the violation beans of Figure 2 that is useful as motivational message. This realization has to be human understandable and convince users to avoid undesired behaviours that trigger such inconsistencies. Therefore, we need i) a persuasive framework that helps users in conduct a good dietary behaviour (Section 4.1); ii) an effective natural language generator method that translates the logical language of the reasoning results (Section 4.2). Both components need the HeLiS ontology to retrieve the necessary data. Figure 3 shows the architecture of our method. The core part relies on templates (a grammar) that encode the several parts (feedback, arguments and suggestion) of a persuasion message. The terminal symbols of these templates are organized according to a hierarchy where the most specific terms are related to specific persuasion strategies. A filler layer manages the filling of the terminal symbols into the templates. Once the templates are filled, a sentence realizer generates natural language sentences that respect the grammatical rules of a target language (here Italian). 4.1
We inspired our work from the theoretical framework in [
Persuasion Strategy The violation bean of Figure 2 contains all the information
explaining the inconsistency of the user’s dietary behaviour with respect to the HeLiS
ontology. In addition, at the end of a day/week many of this beans can be generated.
However, a long list of these beans is understandable mainly by the domain experts
and, most of all, it does not prevent the user to avoid such an erroneous behaviour. A
persuasion strategy addresses this challenge by considering the right timing for sending
the bean, the choice of the violation bean to send to the user (not covered in [
The timing represents the event prompting the creation of a new message. Message generation can be triggered by specific events (e.g., the generation of a new violation bean) or by temporal events. In particular, our system works with three kinds of events: – events related to user’s habits and behavior (i.e., the generated violations); – time scheduling: the need to send particular information to the user at specific time of the day or of the week; – localization: the third event triggering the generation of a message after recognizing that the user is in a specific place (e.g., near a vending machine).
The first kind of events is directly triggered by the detection (through the logical reasoning process of Section 3) of a violation; hence, those information are used for generating the persuasive explanation. The second and third kinds of events, instead, generate persuasive explanations by starting from a pool of past violations.
Once a list of violation beans has been generated, a choice of the violation is performed to avoid annoying the user with too many and repetitive messages. If the list of violations is empty, the system infers that the user adopted a healthy behavior so it sends messages with “positive” reinforcing feedback. If such list is not empty, the system sends a message regarding only one violation to provide the user with varied content about different aspects of a correct behavior. The violation is chosen according to (i) its priority, (ii) the number of times it was committed (see the history parameter in Figure 2), and (iii) the number of times the same violation was the object of a message. For example, if a message discouraging to drink sweet beverages has already been sent in the last 4 days, the next highest priority violation bean not sent recently is chosen.
Once a violation bean is selected, a persuasion strategy computes the intention (or
aim) the persuasive message should convey. According to [
Persuasion Content The content of the message is the information the message has to convey to the user. The content generation is the filling of the feedback, argument, suggestion components: Feedback is the part of the message that informs the user about the unhealthy behavior.
Feedback is generated considering data included in the selected violation: the entity of the violation represents the object of the feedback, whereas the level of violation (the deviation between the expected food quantity and the actually one) is used to represent the severity of the incorrect behavior. Feedback contains also information about timing to report the moment in which violation was committed. Argument is the part of the message that informs the user about the possible consequences of a behavior. For example, in the case of diet recommendations, the argument consists of two parts: i) information about nutrients contained in the food intake that caused the violation and ii) information about consequences that nutrients have on health. Consequences imply the positive or negative aspects of nutrients according to the encourage or discourage intention, respectively.
Suggestion this part is the solution proposed to the user in order to motivate him/her to change his behavior. This suggestion informs the user about the alternative and healthy behavior that he/she can adopt.
The representation regards the format of the content to present to the users. We focus on a natural language representation, however, the persuasive framework deals also with audio or visual formats, for example we can use hGraphs (http://hgraph.org/). 4.2
We describe the process of generating the persuasive explanation starting from the received violation bean, the chosen strategy (here encourage or discourage) and HeLiS. As shown in Figure 3, the natural language generation of the content is performed with templates. This is due to the fact that it is very difficult to build a big and tailored dataset of persuasion sentences to perform the linguistic realization with deep learning techniques. In addition, we need the total control on the generated output as wrong indications could lead to serious problems in the healthcare domain. Morevoer, our template system is devised to allow the dynamic construction of tailored sentences thus avoiding standard canned texts. Here, we encode the feedback, argument and suggestion components with some templates, i.e., a grammar with nonterminal/terminal symbols and production rules. The terminal symbols are selected in the filler layer module to fill the nonterminal ones according to the violation, the strategy and HeLiS. Once the templates are filled, they are sent to a sentence realizer that adjusts the raw sentence according to the syntax rules of the selected natural language, here Italian.
The Template System The template system is the organization of the templates according to the presence of nonterminal/terminal symbols and the persuasion strategy. They are organized in layers. The first is the structure of the feedback, argument and suggestion components. It is encoded as a set of production rules between generic nonterminal symbols, Table 1. The second layer consists of production rules between nonterminal and terminal symbols about the domain. This regards the content of the templates, see Table 2. The third layer contains rules between nonterminal and more specific terminal symbols related to the chosen persuasion strategy, Table 3. This decoupling of the templates structure from their content allows the portability of the templates. Indeed, the first layer could be adapted in other domains with other languages with very low effort. Indeed, our target language is Italian but the templates are the same for English and we here just translate the terminal symbols. On the other hand, if a different persuasion strategy needs to be adapted this reflects only the last layer. 1) Structure of the feedback template: feedback := temporal adv + feed verb + adj + quantity + food entity 2) Structure of the argument template: argument := intro + food ent category + verb adj + food property + conseq verb + consequence 3) Structure of the suggestion template: suggestion := intro + food entity + alternative
Table 1 shows the structure of the feedback, argument and suggestion components. This is the concatenation (symbol +) of some nonterminal symbols that are filled with the terminal ones of tables 2 and 3. The filling can be direct (see intro symbol of Table 2) or dependent from other data such as the violation or HeLiS. This dependency needs to be computed by the filler layer module and it can be just a query to HeLiS or could require more complex operations. For example, the symbols food entity or food ent category are filled with the corresponding HeLiS labels retrieved by using the field entity of Figure 2. Some nonterminal symbols (e.g., the feed verb) can be dependent from the verb and its tense: e.g., beverages imply the use of the verb “to drink” while for solid food we used “to eat”. To increase the variety of the message the verbs “to consume” and “to intake” are also used. Simple past tense is used when violation is related to specific moments (“Today you did not eat enough vegetables”), while simple present continuous is used when the violation is related to a period of time not yet ended (“This week you are drinking a lot of fruit juice”). The filling of 1) Terminal symbols for the feedback template: temporal adv := ["today"|"in the last seven days"]violation feed verb := ["to eat"|"to consume"|"to intake"|"to drink"]violation, tense food entity := []violation, HeLiS 2) Terminal symbols for the argument template: intro := "do you know that" food ent category := []violation, HeLiS other symbols can require more complex operations as long as we are processing the most specific layers of the template system. Indeed, the symbols of Table 3 needs the computation of the strategy. This is given by the field constraint in the violation bean: a “less” constraint (fruitjuice <= 200ml) refers to an excess of this food and this behaviour has to be discouraged. A “greater” constraints (vegetables >= 200g) implies an insufficient amount of this food and this behaviour has to be encouraged. Therefore, a “less” constraint will trigger a discourage strategy, whereas a “greater” constraint will trigger an encourage strategy with the consequent choice of the right terminal symbols in the third template layer. Other template filling could require meta-reasoning strategies to identify the appropriate content that can depend on qualitative properties of food, user profile, other specific violations, and the history of messages sent. This can be noticed in the choice of alternative foods for the suggestion template. HeLiS provides foods that are valid alternatives to the consumed food (e.g., similar-taste relation, list of nutrients, consequences on user health). Then, these alternatives are filtered according to the user profile: even if fish is an alternative to legumes it will not be proposed to vegetarians. Moreover, foods that can cause a violation of “less” or “equal” constraints cannot be suggested, e.g., meat cannot be recommended as alternative to cheese if the user has already eaten its maximum quantity. Finally, control on messages history is performed to avoid the repetitiveness of the message content.
The Sentence Realizer Our system creates the message directly in the desired
language through the Sentence Realizer (SR). The SR takes in as input the filled templates
for the feedback, argument and suggestion components and generates a complex and
well-formed sentence according to the grammar rules of the target language, putting
spaces, capitol letters and choosing the correct inflected forms of the lemmas. In
particular, the Italian language is morphologically richer than English and it entails additional
linguistic resources management to harmonize the various parts of the sentences. To this
end, the SR implements a morphological engine based on Morph-it!, a morphological
resource for the Italian language [
Systems for personalized healthy lifestyle recommendations fall in the broad area of decision support. The goal of these systems is to help and guide users in taking healthyinformed decisions about their lifestyle, on aspects such as food consumption. Such systems have to take a decision (e.g., suggesting conscious and healthy food consumption), similarly as a human expert would do, based on available data (e.g., nutrients ingested in the last meals, user health conditions), and to communicate these decisions to the users according to their preferred means and modalities.
As a specific case study, the presented system has been implemented into our HORUS.AI platform and deployed and evaluated in the context of the project Key to Health in workplace health promotion (WHP) inside our institution (Fondazione Bruno Kessler, FBK). WHP, defined as the combined efforts of employers, employees, and society to improve the mental and physical health and well-being of people at work3, aims at preventing the onset of chronic diseases related to an incorrect lifestyle through organizational interventions directed to workers. Actions concern the promotion of correct diet, physical activity, and social and individual well-being, as well as the discouragement of bad habits, such as smoking and alcohol consumption. Within the Key to Health project, HORUS.AI has been used by 120 FBK’s workers (both researchers and employers) as a tool to persuade and motivate them to follow WHP dietary recommendations. Table 4 shows main demographic information concerning the users involved in the performed evaluation campaign. All users were in good health. Indeed, in this first pilot we decided to not involve people affected by chronic diseases or other diseases. 6
In this Section, we report the evaluation activities we performed within our use case
by adopting the HORUS.AI platform. The evaluation we propose is twofold. First, we
present the validation performed by the domain experts with respect to the correctness
and appropriateness of the generated messages (Section 6.1). This validation aims to
verify that the explanations provided by the system are coherent with respect to the
detected unhealthy behaviors. Second, we discuss the effectiveness of generated
explanations on users’ behaviors (Section 6.2) by showing how the use of explanations
resulted more helpful with respect to a control group of users received punctual
feedback without any detail. The evaluation of reasoning performance is out of scope of this
paper. The reader may find these details in [
Explanation Templates Validation. Three experts 4 have been involved for modeling the templates adopted for generating the explanations. As it has been explained in Section 3, explanations are generated by starting from a finite set of templates that are combined together according to the information contained in the violation packages created by the reasoner. For example, given the category contained in the violation and the violation level, templates concerning the positive or negative properties of the specific food category are connected with verbs and adjectives for shaping the final message. The set of message templates has been validated by the experts that verified the grammatical and content correctness of each template.
Appropriateness of Explanations. The second validation task, where experts were involved, concerned the appropriateness of the messages generated with respect to the violations detected by the reasoner. In order to perform this validation, we performed the following steps: 1. we built data packages representing combinations of meals that should trigger, for each rule contained in the system, the detection of the corresponding violation; 2. we verified that the reasoner correctly detected the violation associated with a given data package; 3. we checked, together with the experts, the appropriateness of the explanation generated with respect to each detected violation.
The analysis of the pairs violation-explanation triggered slight revisions of the linguistic fragments. In particular, some verbs and adjectives used in the fragments were changed to better contextualize the messages. 4 All experts are dietitians and well-being coaches of our local healthcare department. 6.2
The second evaluation concerned the effectiveness analysis of generated explanations on the user study designed within the Key to Health project. The user study consisted in providing to a group of users a mobile application we created based on the services included into the HORUS.AI platform. We analyzed the usage of a mobile application connected with our platform for seven weeks by monitoring the information provided by the users and the associated violations. Our goal was to measure the effectiveness of the explanations generated by our platform by observing the evolution of the number of detected violations. The 120 users involved in the Key to Health project have been split in two groups. A first group of 92 users received the whole persuasive messages generated by using the template system. Whereas a second group of 28 users, that was our control group, did not receive any composition of feedback, argument and suggestion, but only canned text messages notifying when a rule was violated. The expectation was to find a higher decrease in the number of violations through the time by the users receiving persuasive messages.
Results concerning the evolution of the violation numbers are presented in Figure 4. We considered three different kinds of dietary rules: – QB-Rules: these rules define the right amount of a specific food category that should be consumed in a meal. – DAY-Rules: these rules define the maximum (or minimum) amount (or portion) of a specific food category that can be consumed during a single day. – WEEK-Rules: these rules define the maximum (or minimum) amount (or portion) of a specific food category that can be consumed during a week.
The three graphs show the average number of violations per user related to the
QBRules, DAY-Rules, and WEEK-Rules sets respectively. The blue line represents the
number of violations, while the red line the average standard deviation observed for
each single event. Then, the green line represents the average number of violations
generated by the control group and the orange one the associated standard deviation. As
mentioned earlier, QB-Rules are verified every time a user stores a meal within the
platform; DAY-Rules are verified at the end of the day; while WEEK-Rules are
verified at the end of each week. The increasing trend of the gap between the blue and
green lines demonstrates the positive impact of the persuasive messages sent to users.
We can observe how for the QB-Rules the average number of violations is below 1:0
after the first 7 weeks of the project. This means that some users started to follow all
the guidelines about what to consume during a single meal. A positive result has been
obtained also for the DAY-Rules and the WEEK-Rules. In particular, for what concerns
DAY-Rules the average number of violations per user at the end of the observed period
is acceptable by considering that it drops of about 67%. For the WEEK-Rules, however,
the drop remained limited. By considering the standard deviation lines, we can
appreciate how both lines remain contained within low bounds and after a more in depth
analysis of the data, we did not observe the presence of outliers.
We presented an explainable AI system supporting the users in following an healthy
diet. The system checks the presence of unhealthy behaviours based on the food
consumed by users. We discussed in particular the role of the natural language generation
component and how it exploits information inferred by the reasoner for generating
contextual effective explanations. We evaluated our system in a real-world context by
discussing the effectiveness of using persuasive explanations with respect to canned texts.
Results demonstrated how persuasive explanations allows the user to follow an healthy
dietary behaviour. Moreover, the modular template systems allows the dynamic
construction of natural language sentences and the templates portability in other domains.
As future work, the persuasive explanations of user’ behavior will be used in a
Computational Persuasion framework [