=Paper=
{{Paper
|id=Vol-2427/paper14
|storemode=property
|title=An Ontology-Powered Dialogue Engine For Patient Communication of Vaccines
|pdfUrl=https://ceur-ws.org/Vol-2427/SEPDA_2019_paper_14.pdf
|volume=Vol-2427
|authors=Muhammad Amith,Rebecca Lin,Licong Cui,Dennis Wang,Anna Zhu,Grace Xiong,Hua Xu,Kirk Roberts,Cui Tao
|dblpUrl=https://dblp.org/rec/conf/semweb/AmithLCWZX0RT19
}}
==An Ontology-Powered Dialogue Engine For Patient Communication of Vaccines==
https://ceur-ws.org/Vol-2427/SEPDA_2019_paper_14.pdf
An Ontology-Powered Dialogue Engine For
Patient Communication of Vaccines
Muhammad Amith1 , Rebecca Lin2 , Licong Cui1 , Dennis Wang3 , Anna Zhu4 ,
Grace Xiong3 , Hua Xu1 , Kirk Roberts1 , and Cui Tao1
1
The University of Texas Health Science Center at Houston, Houston, TX 77030
2
Johns Hopkins University, Baltimore, MD
3
The University of Texas, Austin, TX
4
Southern Methodist University, Dallas, TX
cui.tao@uth.tmc.edu
Abstract. In this study, we introduce an ontology-driven software en-
gine to provide dialogue interaction functionality for a conversational
agent for HPV vaccine counseling. Currently, the HPV vaccination rates
are low that risks unprotected individuals at being infected with HPV,
a virus that leads to life-threatening cancers. In addition, we developed
a question answering subsystem to support the dialogue engine. In this
paper, we discuss our design and development of an ontology-driven dia-
logue engine that uses the Patient Health Information Dialogue Ontology,
an ontology that we previously developed, and a question answering sub-
system based on various previous methods to supplement the dialogue
engine’s interaction with the user. Our next step is to test the functional
ability of the ontology-driven software components and deploy the engine
in a live environment to be integrated with a speech interface.
Keywords: Ontology · Dialogue Management · Question Answering ·
Vaccines · Conversational Agent
1 Introduction
Speech is the most natural and effective way for us to communicate. Through
speech, we can communicate a lot of information in very little time compared
to printed material [16, 6, 13]. Face-to-face communication between a health
provider and patient is an important factor in improving the health outcome
of consumers. This is particularly beneficial in patient-provider communication
for the human papillomavirus (HPV) vaccine, an effective vaccine that prevents
adulthood cancers. Research has shown that provider communication could po-
tentially increase the vaccine uptake substantially [10]. In addition, the Presi-
dent’s Cancer Council recommends provider communication to improve uptake
rates [18]. However, HPV vaccine rates are below the 80% target coverage rate
[20]. This is compounded with the presumption that health providers deal with
Copyright c 2019 for this paper by its authors. Use permitted under Creative Com-
mons License Attribution 4.0 International (CC BY 4.0).
compressed time to thoroughly discuss the HPV vaccine and answer their ques-
tions, and only a third of patients partake in discussion for the HPV vaccine [10].
A dialogue system is a computer-based agent that converses with human users
using either text or speech. Our experimental proposition is a speech-enabled
dialogue system embodied in a software agent that could inhabit a clinical en-
vironment. This agent could administer the communication task of counseling
patients on the HPV vaccine.
Aside from being expressive terminologies in the biomedical field, ontologies
can provide support for autonomous software agents [12], akin to Tim Berners-
Lee’s vision for ontologies in agents [4]. Take the classic knowledge pyramid,
applied in an agent-based use case (Figure 1). An artist playing music emits
audio noise (Noise) that can be translated into digital format by a robot’s analog-
to-digital converter (Data). The digital data can be further processed by the
machine’s speech recognition software to convert the digital data to information
– string text (Information). However, the machine needs to know the rules on
how to react and behave when presented with information (Knowledge). Within
this example, ontologies occupy a unique role for software agents in the evolution
of information on the knowledge pyramid [9]. AT
N
IO
TA
DA
RM
E
DG
FO
LE
IN
OW
NOISE
KN
M
DO
IS
W
Are you
gonna
????
go my
way? …Leave it for
the humans
Ontologies
(C) STARS AND STRIPES/MICHAEL ABRAMS.
Used with permission. Speech recognition
(machine learning)
Analog to
digital
converter
Fig. 1. Application of the knowledge pyramid for agents.
We discuss our prototype software engine, named the Conversational On-
tology Operator (COO), that utilizes an ontology for dialogue management to
coordinate the conversational behavior. This engine is a prototype that we plan
to integrate into an embodied agent to provide the autonomous interaction in-
telligence to discuss health information with a patient. This engine will not only
coordinate the dialogue exchanges with the user, but also answer vaccine ques-
tions from the user. This task is facilitated by a question answering subsystem
for ontologies that we call Frankenstein 5 Ontology Question Answering for User-
5
The inspiration behind the humorous name is due its patchwork of methods and
ideas from various classic QA for ontologies (NLP-Reduce, FREyA, PANTO, etc.).
Centric Systems (FOQUS). This QA system will utilize our previously developed
VISO-HPV [21] as a knowledge base to query.
2 Materials and Method
We first collected data from a Wizard of OZ experiment [8], and that data led to
the development of the ontology-driven dialogue system engine. The following
subsections describes our development endeavor.
Data collection: The genesis of this work began with simulation studies involving
potential participants (n = 16) who fit the demographic that the conversational
agent (CA) is targeted for – parents with at least one child under 18 [2]. The
simulation was a Wizard of OZ experiment that mimicked the CA using an iPad
tablet and a desktop application that transmitted an operators’ utterance to
the tablet, while masquerading as an autonomous CA. We used the dialogue
exchanges recorded in a chat log and the dialogue script to analyze unique inter-
action patterns and parse out participant questions (53 in total). The utterances
and interaction patterns helped us generate an application ontology for CA for
health – Patient Health Information Dialogue Ontology (PHIDO).
Patient Health Information Dialogue Ontology: We produced an ontology for
health dialogue management, called PHIDO [1], that can facilitate dialogue flow
and contextual dialogue information for a software agent. PHIDO provides the
concepts to create a framework of health counseling between human and ma-
chine. We used PHIDO to create a reusable model of a HPV vaccine counseling
encounter. PHIDO describes the various utterance and speech task classes, and
their object and data property links, to coordinate the dialogue. This model con-
tains three basic speech tasks that can be linked together to form a discussion.
This ontology can later be integrated with health intervention models to build
upon and validate these models.
Conversational Ontology Operator: Our previous steps culminated in the de-
velopment of Conversational Ontology Operator (COO), a software engine that
manages the dialogue for the agent. COO implements a transition mechanism
coordinated by PHIDO (Figure 2). To summarize, COO implements a contin-
uous loop where it first queries for the current position of the dialogue based
on a data property (hasFocus) (Part 1 of Figure 2). Afterwards, it queries for
the next utterance instances and collects their data (2 of Figure 2). If the utter-
ance instance is a system-related utterance, the agent will communicate with the
participant (3 of Figure 2), or if it is a participant utterance it will determine
what type of utterance the user spoke (i.e. using the data associated with the
utterance instance) (4 of Figure 2). Lastly, COO will update the position of the
dialogue (hasFocus) and repeat (5 of Figure 2).
Fig. 2. UML sequence diagram outlining the dialogue transition sequence implemented
in the COO engine.
Frankenstein Question Answering for User-Centric Systems: As an accessory
with COO, we co-developed a subsystem for question answering (QA) to re-
spond to consumer questions during the automated counseling. Using a domain
ontology, this QA subsystem called Frankenstein Question Answering for User-
Centric Systems (FOQUS) will query an answer from a natural language question
expressed by the user. The question’s noun phrases (NP) and verb phrases (VP)
are extracted, including its question type determined by keyword-based iden-
tification. The domain ontology’s Object Property, Data Property, and Class
Assertions are parsed and then compared with the NP and VP for similarity.
A score is assigned for each assertion axiom (triple). Various rules are applied
to find the top ranking triples, and from those selected triples, we compose a
natural language response using simple rules for aggregating and compounding
triples. See Figure 3 of the Appendix which outlines the question answering
method.
3 Discussion
COO and FOQUS were developed using Java 8, with Eclipse RDF4j [7], OWL-
API [14], extJWNL [3], Stanford CoreNLP [17] and HermiT reasoning [11] li-
braries. For FOQUS, similarity methods employed string-based matching from
SimMetrics [15] and vector-based comparisons using Numberbatch [19].
Our next endeavor with this project is to test the functionality of both the
dialogue engine and the question answering system. Most of the dialogue inter-
action is primarily communicating the singular pieces of information about the
HPV and HPV vaccine. We will focus on the core dialogue exchange which is the
communication of health information to the user as our test example. To assess
the COO engine’s interaction, we will observe if the system can impart a piece
of health information (HPV vaccine-related) to the user, coordinate question
answering, and transition the conversation to discuss a health topic.
To test FOQUS, we used questions asked during our simulated experiment
with participants. In total, we collected 53 questions that range from age ap-
propriateness for the vaccine, gender-related questions, cost, etc. Some of the
questions may have been mis-transcribed from speech recognition, yet we kept
it as is to imitate how the live system would process the question. Because of
the possibility of mis-recognition of the utterances, FOQUS relies on the salient
terms of the question (noun and verb phrases) to retrieve an answer. Enlisted
evaluators will be asked to evaluate the question and answer pairs based on two
criteria: the acceptability of the answer for the questions (on a 5 point Likert
scale) and whether the answer matches the question (2=yes, 1=partial, 0=no).
The first criterion aims to help us analyze the presentation and composition of
the answer from triples. The second criterion helps us to determine if FOQUS
can answer the question with some degree of relevancy. We calculated Cohen
Kappa’s inter-rater reliability [5] for both of these criterion to determine the
effectiveness of FOQUS.
4 Conclusion
We put forth an ontology-driven dialogue engine to provide an automated HPV
vaccine counseling experience between a patient and a conversational agent.
This paper presents our prototype ontology-driven dialogue engine (COO) with
question-answering facilities (FOQUS). COO uses a previously developed dia-
logue ontology called PHIDO to direct and manage the interaction of the con-
versational agent for HPV vaccine counseling, and FOQUS uses our previously
developed VISO-HPV to answer potential patient questions during the auto-
mated counseling experience. Our next step is to evaluate COO and FOQUS by
demonstrating COO’s ability to fulfill functional use cases and FOQUS’s ability
to answer sample questions from a simulated study. Our next goal is to deploy
and test the software engine with potential users and assess its performance for
possible use in a clinical environment.
Acknowledgments Research was supported by the UTHealth Innovation for
Cancer Prevention Research Training Program (Cancer Prevention and Research
Institute of Texas grant # RP160015), the National Library of Medicine of
the National Institutes of Health under Award Numbers R01LM011829 and
R00LM012104, and the National Institute of Allergy and Infectious Diseases of
the National Institutes of Health under Award Number R01AI130460.
Disclosures Dr. Hua Xu and The University of Texas Health Science Center
at Houston have research-related financial interests in Melax Technologies, Inc.
References
1. Amith, M., Roberts, K., Tao, C.: Conceiving an application ontology to model
patient human papillomavirus vaccine counseling for dialogue management. BMC
Bioinformatics (In Press)
2. Amith, M., Zhu, A., Cunningham, R., Lin, R., Savas, L., Shay, L., Chen, Y., Gong,
Y., Boom, J., Roberts, K., Tao, C.: Early Usability Assessment of a Conversational
Agent for HPV Vaccination. Studies in Health Technology and Informatics pp. 17–
23 (2019). https://doi.org/10.3233/978-1-61499-951-5-17
3. Autayeu, A.: extJWNL, http://extjwnl.sourceforge.net/
4. Berners-Lee, T., Hendler, J., Lassila, O., et al.: The semantic web. Scientific amer-
ican 284(5), 28–37 (2001)
5. Cohen, J.: A coefficient of agreement for nominal scales. Educational and psycho-
logical measurement 20(1), 37–46 (1960)
6. Damianos, L., Loehr, D., Burke, C., Hansen, S., Viszmeg, M.: The msiia experi-
ment: Using speech to enhance human performance on a cognitive task. Interna-
tional Journal of Speech Technology 6(2), 133–144 (2003)
7. Eclipse Foundation: Eclipse RDF4J, https://rdf4j.eclipse.org/
8. Fraser, N.M., Gilbert, G.N.: Simulating speech systems. Computer Speech & Lan-
guage 5(1), 81–99 (1991)
9. Giarratano, J.C., Riley, G.: Expert Systems: Principles and Programming. Course
Technology (2004)
10. Gilkey, M.B., Calo, W.A., Moss, J.L., Shah, P.D., Marciniak,
M.W., Brewer, N.T.: Provider communication and HPV vacci-
nation: The impact of recommendation quality. Vaccine 34(9),
1187–1192 (Feb 2016). https://doi.org/10.1016/j.vaccine.2016.01.023,
https://linkinghub.elsevier.com/retrieve/pii/S0264410X1600058X
11. Glimm, B., Horrocks, I., Motik, B., Stoilos, G., Wang, Z.: Hermit: an owl 2 reasoner.
Journal of Automated Reasoning 53(3), 245–269 (2014)
12. Hadzic, M., Chang, E., Dillon, T., Wongthongtham, P.: Ontology-based multi-
agent systems. Springer (2009)
13. Harris, S., Biermann, A.W.: Mouse selection versus voice selection of menu items.
International Journal of Speech Technology 5(4), 389–402 (2002)
14. Horridge, M., Bechhofer, S.: The owl api: A java api for owl ontologies. Semantic
Web 2(1), 11–21 (2011)
15. Korstanje, M.: SimMetrics (2019), https://github.com/Simmetrics/simmetrics
16. Litman, D.J., Rosé, C.P., Forbes-Riley, K., VanLehn, K., Bhembe, D., Silliman, S.:
Spoken versus typed human and computer dialogue tutoring. International Journal
of Artificial Intelligence in Education 16(2), 145–170 (2006)
17. Manning, C., Surdeanu, M., Bauer, J., Finkel, J., Bethard, S., McClosky, D.: The
stanford corenlp natural language processing toolkit. In: Proceedings of 52nd an-
nual meeting of the association for computational linguistics: system demonstra-
tions. pp. 55–60 (2014)
18. Rimer, B., Harper, H., Witte, O.: Accelerating hpv vaccine uptake: urgency for
action to prevent cancer; a report to the president of the united states from the
president’s cancer panel. National Cancer Institute, Bethesda, MD (2014)
19. Speer, R., Lowry-Duda, J.: Conceptnet at semeval-2017 task 2: Extending word em-
beddings with multilingual relational knowledge. arXiv preprint arXiv:1704.03560
(2017)
20. U.S. Department of Health and Human Services, Office of Disease Prevention and
Health Promotion: Healthy people 2020. Immunization and Infectious Diseases
National Snapshots (2016)
21. Wang, D., Cunningham, R.M., Boom, J., Amith, M., Tao, C.: Towards a HPV Vac-
cine Knowledgebase For Patient Education Content. Studies in Health Technology
and Informatics (2016)
Appendix
Fig. 3. UML activity diagram describing FOQUS’ process implementation.