=Paper=
{{Paper
|id=Vol-2884/paper_124
|storemode=property
|title=Towards Context-aware Knowledge Entailment from Health Conversations
|pdfUrl=https://ceur-ws.org/Vol-2884/paper_124.pdf
|volume=Vol-2884
|authors=Saeedeh Shekarpour,Faisal Alshargi,Mohammadjafar Shekarpour
}}
==Towards Context-aware Knowledge Entailment from Health Conversations==
https://ceur-ws.org/Vol-2884/paper_124.pdf
Towards Context-aware Knowledge Entailment from Health Conversations
Saeedeh Shekarpour,1 , Faisal Alshargi,2 , Mohammadjafar Shekarpour
1
University of Dayton, Dayton, United States
2
University of Leipzig, Leipzig, Germany
sshekarpour1@udayton.org, alshargi@informatik.uni-leipzig.de, mj.shekarpour@gmail.com
Abstract conversations between a bot (i.e., conversational agent)
and a user is a major challenge. Knowledge entail-
Despite the competitive efforts of leading companies, ment mainly implies entailing facts that demonstrate
cognitive technologies such as chatbot technologies still
have limited cognitive capabilities. One of the major
opinions, beliefs, expressions, requests, and feelings
challenges that they face is knowledge entailment from of a particular user about a particular target (object)
the ongoing conversations with a user. Knowledge en- from conversations. These entailed pieces of knowl-
tailment implies entailing facts that indicate opinions, edge will evolve the background knowledge about the
beliefs, expressions, requests, and feelings of a particu- user. Richer background knowledge will extend the
lar user about a particular target during conversations. future contextual inference and reasoning capabilities
The entailed pieces of knowledge will evolve the back- of cognitive technologies built up on top. Although
ground knowledge graph of cognitive technologies and the Natural Language Processing (NLP) community
advance their contextual inference and reasoning ca- deals with the Recognizing Textual Entailment (RTE)
pabilities. Although the Natural Language Processing task, it is treated in a static manner where the prede-
(NLP) community deals with the Recognizing Textual
Entailment (RTE) task, it is treated in a static manner
fined hypothesis is typically fed to the learning model,
where the predefined hypothesis is typically fed to the and then the model decides whether it is entailment
learning model, and then the model decides whether it or not. However, the knowledge entailment task, par-
is an entailment or not. However, since the discourse of ticularly in our context (conversations being dynamic,
conversations is dynamic and unpredictable, the tradi- unpredictable, and complex), requires a model for the
tional RTE approach does not suffice in the context of purposes of not only learning entailments but also for
conversational agents. In this vision paper, we demon- inferring all possible hypotheses (including entailing,
strate our work in progress as to inject background contradicting, etc). Recently, the combination of knowl-
knowledge into machine learning approaches where it edge representation and machine learning has been in
entails facts using domain-specific ontologies and con- the center of attention towards reaching an explain-
textualized knowledge. Further, we propose investigat-
ing solutions for extending or transferring this approach
able, accountable, and fair AI which will exhibit more
to other domains. We frame our discussion in a case robust intelligence and reliable capabilities (Holzinger
study related to mental health conversations. et al. 2017; Samek, Wiegand, and Müller 2017). Knowl-
edge representation provides essential conceptualiza-
tion (domain ontology), contextual entities, associated
Introduction facts, and, more importantly, relations between enti-
Knowledge entailment has applicability in various cog- ties and concepts. In this paper, we describe our work
nitive technologies such as conversational AI inter- in progress as it proposes an ontology-based knowl-
faces (chatbot technologies), which recently gained the edge entailment approach over the discourse of con-
competitive efforts of leading companies. The existing versations. We demonstrate our envisioned plan in an
implementations of this technology have limited cog- illustrative mode to display the open research areas re-
nitive capabilities where they fail to perceive users’ quired the future attention of the community. This pa-
opinions, beliefs, expressions, requests, and feelings. per is organized as follows: Section 2 discusses the lim-
Knowledge entailment (also known as knowledge per- itations of the state-of-the-art. Section 3 presents the
ception) primarily from the text and secondarily from problem statement, followed by Section 4, which show-
cases a case study. Next, we demonstrate our ultimate
AAAI Fall 2020 Symposium on AI for Social Good. Copyright envisioned plan. We close with the remarks related
© 2020 for this paper by its authors. Use permitted under Cre- to the applicabilities of our knowledge entailment ap-
ative Commons License Attribution 4.0 International (CC BY proach in chatbot technologies.
4.0).
Figure 1: Samples of data for RTE task, two given premises with the possible hypotheses. The RTE model determines
whether a given hypothesis is an Entailment (E), or Contradiction (C) or Neural (N).
Limitations of the state-of-the-art Thus, they fail to overcome unpredicted situations. The
RTE (Dagan et al. 2010) is a task in NLP where it deter- machine learning approaches are solely data-driven.
mines whether two given sentences (i) contradict each Advancing them with explicit knowledge will result
other, (ii) are semantically unrelated to each other, in faster convergence on sparse data. Furthermore, it
or (iii) one of them (premise) entails the other one makes them explainable, compliant to the domain, and
(hypothesis). Figure 1 showcases multiple examples. more robust against noise. Figure 2A shows the static
For instance for the given premise An older man is RTE task which predicts (discriminates) the proper
drinking orange juice at a restaurant, label (entailment, contradiction, and neutral) for the
three hypotheses are listed. The first premise A man given input text (premise) and the given hypothesis.
is drinking juice. is an entailment (E) of In this scenario the input hypothesis is supposed to
the premises whereas the second hypothesis Two be given by the user. In our envisioned model (Figure
women are at a restaurant drinking wine 2B) there is no need to worry about the hypothesis be-
is a contradiction C and the third one A man in a cause they are automatically fed to a generative model
restaurant is waiting for his meal to using the existing facts from the background knowl-
arrive. is neutral (N). The work presented in edge graph. We plan to extend a knowledge entailment
(Bowman et al. 2015) was published in the Stanford approach (which is a neural network approach) fed
Natural Language Inference (SNLI) corpus, which is with domain-specific ontologies to contextual as well
far larger than all of the other existing resources of its as personalized knowledge graphs; it will not only be a
type. It contains more than 500K pairs of sentences, data-driven approach but also a knowledge-driven ap-
which are annotated using the labels E (entailment), proach. To present a clear and practical vision of our
C (contradiction), and N (neutral). The RTE task is proposed scenario, we frame a case study on the health
substantially important in information extraction, text domain which entails knowledge from the conversa-
summarization, classification, and machine transla- tions about mental health.
tion. The NLP community deals with static RTE tasks
where the hypothesis is fed to the learning model,
and then the model decides it is an entailment or not, A. Recognizing Textual Entailment
while in scenarios such as knowledge entailment from Input Text (premise) 1. Entailment
conversations, we have to develop a generative model Prediction
(Discriminative)
prediction 2. Contradiction
3. Neutral
Model
where the model can dynamically generate hypotheses User
Hypothesis
and there is no predefined hypothesis.
Problem Statement Input Text (premise)
User
Generative Model
In the era of contemporary conversational AI, the first
generating
Background
Knowledge
major deficiency attributes to the lack of a convinc- Background Knowledge
(Ontology + Data )
Entailment 1
ing approach for knowledge entailment from conver- Entailment 2
Entailment 3
sations, e.g., whether or not a chatbot learns the user B. Knowledge Entailment ...
by entailing knowledge from ongoing conversations, Figure 1: Part (A) is a textual entailment module, which is a discriminative model,
and evolves its underlying knowledge for future con- Figure 2: Part (A) is a textual entailment module,
and Part (B) is a knowledge entailment module, which is a generative model.
versation management. The second deficiency is that which is a discriminative model, and Part (B) is a
the available approaches are solely data-driven ap- knowledge entailment module, which is a generative
proaches (i.e., machine learning approaches) or rule- model.
based approaches, and in both cases, the inference ca-
pabilities are limited to the underlying data and rules.
Case Study Personalized Healthcare Knowledge Graph (PHKG):
Figure 3 demonstrates our expectations from a knowl- The work presented in (Gyrard et al. 2018) introduces
edge entailment approach over the conversations. PHKG, which is described as a representation of all
There is a given excerpt from our underlying conversa- relevant medical knowledge and personal data for a
tion dataset (will be introduced in the following). This patient. PHKG can support the development of inno-
excerpt shows semantics referring to insomnia which is vative applications such as digital personalized coach
a subject question in most of the questionnaires (such applications that can keep patients informed, help to
as PHQ-8 and PHQ-9) for depression disorder. How- manage their chronic condition, and empower physi-
ever, entailing these semantics requires considering the cians to make effective decisions on health-related is-
indicators of insomnia, in addition to the contextual in- sues or receive timely alerts as needed through contin-
formation from several lines in the course of the con- uous monitoring. Typically, PHKG formalizes medical
versation. Considering a given question from PHQ-9 information in terms of relevant relationships between
inquiring about the status of the patient’s sleep, our entities. For instance, a knowledge graph (KG) for
expectation is that our envisioning approach can entail asthma can describe causes, symptoms, and treatments
the piece of knowledge that “the patient has a sleep dis- for asthma, and a PHKG can be the subgraph contain-
order often”. In the following, we introduce the sources ing just those causes, symptoms, and treatments that
of knowledge which will be incorporated in our ap- are applicable to a given patient. In our case study,
proach. PHKG is limited to the knowledge about the patient
which is determined from conversations.
Conversation Excerpt
Ellie: how easy is it for you to get a good night's sleep
Envisioned Plan
Participant: it's pretty good eh somewhat
Ellie: What are you like when you don't sleep well
Figure 4 schematically shows our envisioned plan as
Participant: I'm tired and I kind of fall asleep during class and whatnot
Ellie: do you feel that way often
the given data in the background knowledge graphs
Participant: yeah it's my fault though (personalized health graph and contextualized graph).
Ellie: hm when was the last time that happened
Participant: um probably today Then, our knowledge entailment approach will drive
...
further knowledge from conversations. The third two
validation and quality assurance strategies will be ap-
PHQ9
plied to determine whether entailed knowledge is valid
Trouble falling or staying asleep, or
sleeping too much? Knowledge Entailment
or not. This step might rely on manual approaches such
1. Not at all Patient has sleep disorder nearly often
as crowd-sourcing or automatic approaches such as
2.
3.
Several days
More than half the days
graph completion and reasoning to validate entailed
4. nearly Every Day
knowledge. Finally, the newly entailed knowledge is
added to PHKG and contextualized graphs. Having an
Figure 2: A conversation excerpt, with the entailed statement based on the background iteration over this cycle or upcoming conversation will
Figure 3: A conversation excerpt,
knowledge from PHQ9. with the entailed
help to both entail further knowledge or augment our
statement based on the background knowledge from entailment approach.
PHQ9.
PHQ-9 Ontology and Lexicon: The Diagnostic and Input conversation and background knowledge 1
conversation to
Statistical Manual of Mental Disorders (DSM) (Asso- knowledge
ciation et al. 2013) suggests that clinical depression can Contextualized
Knowledge 2 Entailment Module
be diagnosed through the presence of a set of symp- Conversation
toms over a fixed period of time. The PHQ-9 (Löwe Personalized
Knowledge
Knowledge
Entailment
et al. 2004) is a nine-item depression scale that incorpo- Graph
rates DSM-V. It can be utilized to screen, diagnose, and
measure the severity of depression. We are building an validation
ontology from PHQ-9 where it incorporates all the con-
Augmentation and Quality
knowledge complesion 3
Automatic (Graph Completion,
cepts, depression symptoms and relevant phrases. Reasoning)
Manual (Crowd Sourcing)
Dataset: The Distress Analysis Interview Cor-
pus Wizard-of-Oz (DAIC-WoZ) interview database
(Gratch et al. 2014; DeVault et al. 2014) consists of clin- Figure 4: The process of entailing new knowledge from
ical diagnostic interviews designed to support the di- conversations, validating that, and adding to the per-
agnosis of psychological disorders such as anxiety, de- sonalized knowledge graph.
pression, and post-traumatic stress disorder. This cor-
pus (DAIC) comprises recorded interviews between a Our model will entail triples (subject-predicate-
patient (participant) and a computerized animated vir- object) from conversations where the subject is the on-
tual interviewer "Ellie". It contains data from 189 inter- going user, and predicates and objects represent opin-
views, including transcripts, audio, and video record- ions, beliefs, expressions, requests, and feelings be-
ings, and PHQ depression questionnaire responses. longing to the user. Figure 5 demonstrates the trans-
formation of the input text into a graph that contains Bordes, A.; Boureau, Y.-L.; and Weston, J. 2016. Learn-
all the entailed facts about the patient. We assume that ing end-to-end goal-oriented dialog. arXiv preprint
all the required relations (predicates) and possible ob- arXiv:1605.07683 .
jects are declared in the domain ontology. If we develop Bowman, S. R.; Angeli, G.; Potts, C.; and Manning, C. D.
an attention model that is fed with the context (cog- 2015. A large annotated corpus for learning natural lan-
nitive ontology, context, and personalized knowledge) guage inference. arXiv preprint arXiv:1508.05326 .
along with the user utterance then possibly one or mul-
tiple relations and objects acquire higher weight (atten- Dagan, I.; Dolan, B.; Magnini, B.; and Roth, D. 2010.
tion) – meaning they are entailed from the input utter- Recognizing textual entailment: Rational, evaluation
ance. Furthermore, the higher the volume of conversa- and approaches–erratum. Natural Language Engineer-
tions with the user, the better the context-aware knowl- ing 16(1): 105–105.
edge entailment. The key novel part of this work is DeVault, D.; Artstein, R.; Benn, G.; Dey, T.; Fast, E.;
combining domain knowledge representation and ma- Gainer, A.; Georgila, K.; Gratch, J.; Hartholt, A.; Lhom-
chine learning approaches to provide robust, explain- met, M.; et al. 2014. SimSensei Kiosk: A virtual human
able, and context-aware solutions. interviewer for healthcare decision support. In Proceed-
ings of the 2014 international conference on Autonomous
Not at all
agents and multi-agent systems, 1061–1068.
little interest or pleasure in doing things
Gratch, J.; Artstein, R.; Lucas, G. M.; Stratou, G.;
Conversation Sample
1
Scherer, S.; Nazarian, A.; Wood, R.; Boberg, J.; DeVault,
D.; Marsella, S.; et al. 2014. The distress analysis inter-
Ellie: how easy is it for you to get a good night's sleep 2
often
Participant: it's pretty good eh somewhat feeling down, depressed
view corpus of human and computer interviews. In
Ellie: What are you like when you don't sleep well
Patient
Participant: I'm tired and I kind of fall asleep during class and whatnot
Ellie: do you feel that way often sleeping too much
Participant: yeah it's my fault though
Ellie: hm when was the last time that happened
trouble falling asleep
Not at all
LREC, 3123–3128. Citeseer.
Participant: um probably today
...
every day Gyrard, A.; Gaur, M.; Shekarpour, S.; Thirunarayan, K.;
and Sheth, A. 2018. Personalized Health Knowledge
Figure 5: The process of transforming a given ongoing conversation to a graph containing entailed
facts about the patient. Graph. In ISWC 2018 Contextualized Knowledge Graph
Figure 5: The process of transforming a given ongoing Workshop.
conversation to a graph containing entailed facts about
the patient. Holzinger, A.; Biemann, C.; Pattichis, C. S.; and Kell,
D. B. 2017. What do we need to build explainable
AI systems for the medical domain? arXiv preprint
arXiv:1712.09923 .
Applicability in Chatbot Technology
A chatbot is typically an Artificial Intelligence (AI)- Lee, D.; Oh, K.-J.; and Choi, H.-J. 2017. The chat-
based application designed to simulate a conversa- bot feels you-a counseling service using emotional re-
tion with human users in a continuous and common sponse generation. In 2017 IEEE International Conference
sense manner (Lee, Oh, and Choi 2017). This assis- on Big Data and Smart Computing (BigComp), 437–440.
tance can reduce the cognitive load for the user, espe- IEEE.
cially in high-pressure situations such as surgical oper- Li, X.; Lipton, Z. C.; Dhingra, B.; Li, L.; Gao, J.; and
ations, battlefields, and disaster preparedness and re- Chen, Y.-N. 2016. A user simulator for task-completion
sponse. Furthermore, it is promising and effective in dialogues. arXiv preprint arXiv:1612.05688 .
everyday life activities, such as retail, travel, news, and Li, X.; Wang, Y.; Sun, S.; Panda, S.; Liu, J.; and Gao, J.
entertainment. However, despite the recent competi- 2018. Microsoft dialogue challenge: Building end-to-
tive efforts and investments of leading companies (e.g., end task-completion dialogue systems. arXiv preprint
Facebook (Messenger), Microsoft (Cortana), Apple arXiv:1807.11125 .
(Siri), Google (Duplex), WeChat, and Slack), the exist-
ing implementations do not provide impressive cogni- Löwe, B.; Kroenke, K.; Herzog, W.; and Gräfe, K. 2004.
tive capabilities. For example, state-of-the-art chatbots Measuring depression outcome with a brief self-report
still struggle with simple conversational domains, such instrument: sensitivity to change of the Patient Health
as task ordering (Microsoft challenge (Li et al. 2018, Questionnaire (PHQ-9). Journal of affective disorders
2016), bAbI project of Facebook (Bordes, Boureau, and 81(1): 61–66.
Weston 2016; Weston et al. 2015)), and is still far from Samek, W.; Wiegand, T.; and Müller, K.-R. 2017. Ex-
complicated conversations in a variety of domains. Our plainable artificial intelligence: Understanding, visual-
proposed work, if successful, is a complementary step izing and interpreting deep learning models. arXiv
for future chatbot technologies. preprint arXiv:1708.08296 .
Weston, J.; Bordes, A.; Chopra, S.; Rush, A. M.; van Mer-
References riënboer, B.; Joulin, A.; and Mikolov, T. 2015. Towards
Association, D.-. A. P.; et al. 2013. Diagnostic and sta- ai-complete question answering: A set of prerequisite
tistical manual of mental disorders. Arlington: American toy tasks. arXiv preprint arXiv:1502.05698 .
Psychiatric Publishing .