=Paper=
{{Paper
|id=Vol-1733/paper-02
|storemode=property
|title=Ontology-Based Dialogue Systems for Improved Patient HPV Vaccine Knowledge and Perception
|pdfUrl=https://ceur-ws.org/Vol-1733/paper-02.pdf
|volume=Vol-1733
|authors=Muhammad Amith
|dblpUrl=https://dblp.org/rec/conf/semweb/Amith16
}}
==Ontology-Based Dialogue Systems for Improved Patient HPV Vaccine Knowledge and Perception==
https://ceur-ws.org/Vol-1733/paper-02.pdf
Ontology-based dialogue systems for improved
patient HPV vaccine knowledge and perception
Muhammad “Tuan” Amith, MS1
School of Biomedical Informatics, University of Texas Health Science Center,
Houston TX 08544, USA,
muhammad.f.amith@uth.tmc.edu
Abstract. In Sir Tim Berners-Lee’s seminal article that introduce his
vision of the semantic web, one of the use-cases described was a health-
related example where health consumers utilized intelligent hand-held
devices that aggregated and exchanged health data from the semantic
web. Presently, the majority of health consumers and patients rely on
personal technology and the web to find information and to make per-
sonal health decisions. This proposal aims to contribute towards that use-
case, specifically in the “hot-bed” issue of human papillomavirus (HPV)
vaccine. The HPV vaccine targets young adults and teens to protect
against life-threatening cancers, yet a segment of the public has reser-
vations against the vaccine. I propose an interactive dialogue agent that
harness patient-level vaccine information encoded in an ontology that
can be “talked to” with a natural language interface using utterances.
I aim to pilot this technology in a clinic to assess if patient knowledge
about HPV and the vaccine is increased, and if their attitude toward the
vaccine is modified as a result of using the interactive agent.
Keywords: ontology, natural language processing, dialogue system, con-
versational agents, vaccine, ontology learning, question-answering
1 Problem Statement
At 79 million reported cases, human papillomavirus (HPV) infection is one of
the most common sexually transmitted infection, and per year, according to the
Centers for Disease and Control, 14 million more infections will be reported [10].
For many of the life threatening cancers, like cervical cancer, the HPV virus
(Type 16 and 18) is attributed to HPV infection. The human papillomavirus
(HPV) vaccine is noted to be ∼100% effective against the HPV virus [9], and
yet the United States’ population uptake for HPV vaccine is at a low 28% [11],
short of the 80% uptake goal [43]. Irrespective of the efficacy of the HPV vac-
cine, it has yet to be fully embraced by the public, patients, and even healthcare
providers [18]. Most HPV information interventions are paper-based in the form
of brochures, flyers, etc. [16], yet, patients ignore, misunderstands, or the con-
tent is not effective [28, 41, 26, 27, 33, 1]. There is strong evidence that patient-
provider vaccine counseling has a positive influence in affecting knowledge and
vaccination rates [33, 34, 24].
2
1.1 Research Question:
I postulate, will an ontology-driven health conversational agent improve the vac-
cine literacy of patients and/or have an influence on patients’ intention to vac-
cinate? This research will theorize that an interactive agent, verbally communi-
cating HPV vaccine knowledge, will result in the likelihood in users opting for
the HPV vaccine, improve their vaccine literacy, and improve patient satisfaction
and the patient-provider relationship. Using an ontological knowledge-base, we
can aggregate disparate and complex health information using OWL/RDF, and
also contribute to validated, comprehensive HPV vaccine knowledge for linked
open data. Because ontological knowledge-base will be semantically enriched and
readable by machines, this will allow users to utilize natural language utterance
to query information without any background expertise in complex query lan-
guages. This will provide a natural language interface where patients can easily
interact with a knowledge-base and get precise information, and perhaps, fos-
ter further applicable research for consumer usability and interaction with the
semantic web.
2 Related Work
Several studies reveal that oral communication in health literacy is as important
as written communication when it comes to delivering health information for
patients [36, 39, 40]. Patients are also likely to trust face-to-face interaction to
acquire health information [19]. Personalizing health information that is esoteric
or abstract for patients with low health literacy benefits their comprehension
and understanding [37]. A study from genetic counseling research on patient
communication discovered that patients appear to learn better with interactiv-
ity during a counseling session [37]. Patients who ask questions are also likely
to attain better health information from their provider [42, 45]. Also, patients
favored dialogue with health provider that is uncomplicated and responses that
swiftly addresses their concern [35, 13].
As it stands, there are providers who could educate patients about the HPV
vaccine. Yet this is an added burden to the provider. One of them is being a
good communicator, especially to patients who have low health literacy [17, 38,
15]. Another burden is taking the role of being health educator [47], even in the
difficult situation of talking about sexual-related matters with both the parent
and the teenage child [14, 21]. More importantly, there is not enough time to
discuss and counsel the patient about the HPV vaccine [17, 44]. This proposal
submits the possibility of delegating and automating the communication and ed-
ucation task to an interactive kiosk or tablet application that would be available
for patients, similar to a “Siri”-like experience. The possible benefit other than
cost effectiveness [3] is facilitating the interactive dialogue that could approxi-
mate a real conversation [32], mimicking the personal interaction that patients
and health consumers desire [22, 2]. Also using machine intelligence, the system
can strategize how and when information is provided [32]. The research have
noted that face-to-face counseling between patient and provider has an influence
3
in accepting vaccination for the patient or the patient’s parent [25, 34]. Specif-
ically, the verbal mode of the health dialogue system can offer opportunities
to enhance interactivity between patient and provider, such as using machine
intelligence for decision making and coordination of content delivery, utilizing
interpersonal cues to imitate human conversation and improve communication
efforts with non-experts [32].
3 Approach & Hypotheses
I intend to design and develop a interactive dialogue system specifically for
patient question-answering for patient-level vaccine information and to affect
vaccination attitudes, and then pilot the system with participants at a pediatric
clinic.
Research Objective 1 (RO1 ): Generate a comprehensive vaccine
knowledgebase using natural language processing (NLP) tools.
Hypothesis: Patient-level vaccine information can be faithfully represented in an
ontological knowledgebase. I will gather a corpus of patient-level vaccine and
HPV information documentation in electronic format, and will utilize an natural
language information extraction tool to extract predicates. I will evaluate the
accuracy and meaningfulness of the results of the extraction. Based on the results
of the evaluation, the information will be encoded using Protègè to produce an
ontological knowledgebase. After publishing the ontology, we will evaluate it
based on a semiotic framework for ontology evaluation [8].
Research Objective 2 (RO2 ): Design a speech-enabled conversa-
tional agent for patients to interface with the vaccine KB.
Hypothesis: An ontology-driven speech interface can accurately produce answers
for patient questions. I will design a question-answering query engine to allow pa-
tients to ask questions about the HPV vaccine and related information through
thin-client tablet application. The query engine will automatically translate nat-
ural language questions to SPARQL queries, which is then executed to a remote
triple-store hosting the vaccine knowledgebase. In addition, the agent will model
some cues and best practices for oral communication of health information for
better patient knowledge acquisition.
Research Objective 3 (RO3 ): Demonstrate feasibility of interactive
conversational agent for enhancing patients’ vaccine knowledge.
Hypothesis: Users of the interactive intervention will experience greater education
gain, trust in the intervention, and a stronger intention to vaccinate with HPV
vaccine than the control group. I will coordinate with our collaborators at the
Texas Children’s Hospital to launch this pilot study with patient participants
in one of their network clinics. The Texas Children’s Pediatric (TCP) clinics is
the most prominent network of pediatric clinics serving the Houston area [29].
I will implement a randomized two-group post-test research design where the
experimental group (participants utilizing the interactive agent) and a control
group (participants with vaccine pamphlets) will be compared.
4
4 Evaluation plan
Each research objective is associated with an evaluation process(es) in order to
measure outcomes of the research efforts.
RO1 : To faithfully represent the information collected from the corpus through
information extraction, this aim will need to evaluate the accuracy of the extrac-
tion results and the final ontological knowledgebase. Specifically, we will evaluate
the extraction results based on contextual accuracy and correctness of the triple
decomposition from sentences through the information extraction process. In
agreement, two evaluators will assess the output based on contextual accuracy
and minimally decomposed triples.
After serializing the vaccine information from the various resources, we will
use Burton-Jones, et al.’s semiotic metric framework [8] to evaluate the onto-
logical knowledgebase, with an automated, web-based Java application I have
developed [4]. Health professional experts will evaluate the accuracy of the in-
formation to derive one of the framework’s scores.
RO2 : The first part of the evaluation will assess the effectiveness of natural
language queries uttered by the consumer to be transformed into a SPARQL
query. For this segment of the evaluation, we will rely on system-level evalua-
tion and convert the the natural language queries to string data structures, since
speech recognition APIs and toolkits may not be accurate and distort the results.
I will calculate precision and recall based on [23] calculations for ontology-based
question-answering systems. Despite possible performance variances with dif-
ferent datasets, I may evaluate the accuracy of engine with unrelated subject
ontologies to assess comparable outcomes found in literature.
Also, for each question, there will be an intended response to determine
answer sufficiency (“good enough” answer). Three investigators will compare
the response generated from the question-answering engine to assess the quality
of the answer with the question, and they will compare the results and rate
the results based on a 5-point Likert scale. Inter-rater agreement scores will be
examined using Cohen’s Kappa.
RO3 : After completion of the pilot, we will collect the data and preform sev-
eral analyses from the surveys of the participants. Aside from demographic data
surveys, each participants will complete a survey to measure intention based
on the health belief model [31], a survey to measure trust with the agent [20],
and a validated HPV-related knowledge test [30]. To assess knowledge gain, we
will calculate a t-Test comparing mean knowledge test scores between partic-
ipants who used the conversational agent (AU ) and participants who did not
(N AU ). Similarly, we will also observe overall intention to vaccinate based on
the mean average of the two groups’ vaccination intention surveys and employ
a t-Test calculation to verify significant differences. To discover the users’ trust
with the conversational agent, we will utilize the overall midpoint of agent user
participants and denote the number of users whose trust scores exceeded the
midpoint mean of the trust scores, similar to [7]. With regression analysis, I
want to investigate whether the intention to vaccinate is affected by trust in
agent (independent variable) and/or knowledge gain (independent variable).
5
5 Preliminary results
Last year we developed a proof of concept prototype to determine technical
feasibility and future improvements [6]. Some of these areas include expanding
the domain ontology, enhancing the speech interface mobile application for un-
constrained queries and user personalization, and investigating the development
of application-based ontologies. I recently worked on an initial meta-level on-
tology framework called VISO (Vaccine Information Statement Ontology) that
is sourced from federally-mandated vaccine documentation by the Centers for
Disease and Control [4]. Using a semiotic-inspired ontology quality evaluation
metric, VISO was revealed to have strong pragmatic qualities, but needed addi-
tional improvement with syntactic aspects.
Additionally, the vaccine ontology work has been extended with initial devel-
opment of an HPV vaccine version initiated last year [46], and we are experiment-
ing with open information extraction (“ClausIE” [12]) for ontology development
for patient-centric HPV cancer knowledge [5], which we plan on utilizing to ex-
tend the HPV vaccine ontology, to enrich the knowledge space with patient-level
cancer information. Domain-expert collaborators had advised that our vaccine
ontology needs to include anecdotal stories, certain multimedia content, vac-
cine myths, and personalization, all to compliment the patient-level factoids and
improve the information quality.
6 Reflections
My work is in the early stages, and we foresee a two-year projection to complete
the three research objectives. The literature in utilizing dialogue or conversa-
tional agents for patient-centric health care is available but relatively scarce,
especially ontology-based implementations. However, there has yet to be re-
search that involves the use of dialogue agents that can affect health consumers’
attitudes for vaccination with the combination of knowledge and utilizing best
practices in patient-provider communication and verbal health literacy. As the
adage goes, this may either be a “blessing” or a “curse” as there is no guarantee
that ontology-driven interactive agents can affect vaccine attitudes. In collab-
oration with health communication experts, this multi-disciplinary study will
apply research in patient-provider communication that can be ported to tech-
nological solutions, and I will be working with experts in public health to help
tailor this intervention to show some impact that can help further the research
in ontology-based conversational agents in patient-centered health care.
Acknowledgments I thank my supervisor, Cui Tao, PhD, and my committee
for their guidance and feedback. Support was provided by a fellowship funded
by the Cancer Prevention Research Institute of Texas (CPRIT) Training Grant
#RP160015. Special thanks to the Immunization Project of the Texas Children’s
Hospital and the CPRIT research faculty for their collaboration.
6
References
1. Agarwal, N., Chaudhari, A., Hansberry, D.R., Tomei, K.L., Prestigiacomo, C.J.: A
comparative analysis of neurosurgical online education materials to assess patient
comprehension. Journal of Clinical Neuroscience 20(10), 1357–1361 (2013)
2. Ahken, S., Fleming, N., Dumont, T., Black, A.: HPV awareness in higher-risk
young women: The need for a targeted HPV catch-up vaccination program. J
Obstet Gynaecol Can 37(2), 122–128 (2015)
3. Allen, J., Ferguson, G., Blaylock, N., Byron, D., Chambers, N., Dzikovska, M.,
Galescu, L., Swift, M.: Chester: towards a personal medication advisor. Journal of
biomedical informatics 39(5), 500–513 (2006)
4. Amith, M., Gong, Y., Cunningham, R., Boom, J., Tao, C.: Devel-
oping VISO: Vaccine Information Statement Ontology for patient ed-
ucation. Journal of Biomedical Semantics 6(1), 23 (May 2015),
http://www.jbiomedsem.com/content/6/1/23/abstract
5. Amith, M., Song, H.Y., Zhang, Y., Tao, C.: Lightweight predication extraction for
patient-level cancer information and ontology development (In preparation, 2016)
6. Amith, M., Tao, C.: A prototype mobile virtual assistant for semantic-based vaccine
information retrieval. In: Zheng, X., Zeng, D., Chen, H., Zhang, Y., Xing, C., Neill,
D. (eds.) Smart Health, Lecture Notes in Computer Science, vol. 8549, pp. 200–
205. Springer International Publishing (Jan 2014), http://dx.doi.org/10.1007/978-
3-319-08416-9 21
7. Bickmore, T.W., Pfeifer, L.M., Byron, D., Forsythe, S., Henault, L.E., Jack, B.W.,
Silliman, R., Paasche-Orlow, M.K.: Usability of conversational agents by patients
with inadequate health literacy: evidence from two clinical trials. Journal of health
communication 15 Suppl 2, 197 (2010)
8. Burton-Jones, A., Storey, V.C., Sugumaran, V., Ahluwalia, P.: A semiotic metrics
suite for assessing the quality of ontologies. Data & Knowledge Engineering 55(1),
84–102 (Oct 2005)
9. Centers for Disease Control and Prevention: HPV vaccine resources for health
professions. http://www.cdc.gov/vaccines/who/teens/for-hcp/hpv-resources.html
(2014)
10. Centers for Disease Control and Prevention (CDC) and others: CDC fact sheet:
incidence, prevalence, and cost of sexually transmitted infections in the united
states. CDC (2013)
11. Centers for Disease Control and Prevention (CDC) and others: Human papillo-
mavirus vaccination coverage among adolescent girls, 2007-2012, and postlicensure
vaccine safety monitoring, 2006-2013-united states. MMWR. Morbidity and mor-
tality weekly report 62(29), 591 (2013)
12. Del Corro, L., Gemulla, R.: ClausIE: clause-based open information extraction.
In: Proceedings of the 22nd international conference on World Wide Web. pp.
355–366. International World Wide Web Conferences Steering Committee (2013),
http://dl.acm.org/citation.cfm?id=2488420
13. Doak, C.C., Doak, L.G., Root, J.H.: Teaching patients with low literacy skills. AJN
The American Journal of Nursing 96(12), 16M (1996)
14. Esposito, S., Bosis, S., Pelucchi, C., Begliatti, E., Rognoni, A., Bellasio, M., Tel, F.,
Consolo, S., Principi, N.: Pediatrician knowledge and attitudes regarding human
papillomavirus disease and its prevention. Vaccine 25(35), 6437–6446 (2007)
15. Evans, G., Bostrom, A.: The evolution of vaccine risk communication in the united
states: 1982–2002. National Institute of Allergy and Infectious Diseases, ed. Jordan
Report 20th anniversary: Accelerated Development of Vaccines pp. 58–71 (2002)
7
16. Fu, L.Y., Bonhomme, L.A., Cooper, S.C., Joseph, J.G., Zimet,
G.D.: Educational interventions to increase HPV vaccination accep-
tance: A systematic review. Vaccine 32(17), 1901 – 1920 (2014),
http://www.sciencedirect.com/science/article/pii/S0264410X14001546
17. Goff, S.L., Mazor, K.M., Gagne, S.J., Corey, K.C., Blake, D.R.: Vaccine counseling:
A content analysis of patient–physician discussions regarding human papilloma
virus vaccine. Vaccine 29(43), 7343–7349 (2011)
18. Gottlieb, S.L., Low, N., Newman, L.M., Bolan, G., Kamb, M., Broutet, N.: To-
ward global prevention of sexually transmitted infections (STIs): the need for STI
vaccines. Vaccine 32(14), 1527–1535 (2014)
19. Hesse, B.W., Nelson, D.E., Kreps, G.L., Croyle, R.T., Arora, N.K., Rimer, B.K.,
Viswanath, K.: Trust and sources of health information: the impact of the internet
and its implications for health care providers: findings from the first health infor-
mation national trends survey. Archives of internal medicine 165(22), 2618–2624
(2005)
20. Horvath, A.O., Greenberg, L.S.: Development and validation of the working al-
liance inventory. Journal of counseling psychology 36(2), 223 (1989)
21. Humiston, S.G., Albertin, C., Schaffer, S., Rand, C., Shone, L.P., Stokley, S., Szi-
lagyi, P.G.: Health care provider attitudes and practices regarding adolescent im-
munizations: a qualitative study. Patient education and counseling 75(1), 121–127
(2009)
22. Katz, M.L., Krieger, J.L., Roberto, A.J.: Human papillomavirus (HPV): college
male’s knowledge, perceived risk, sources of information, vaccine barriers and com-
munication. Journal of men’s health 8(3), 175–184 (2011)
23. Kaufmann, E.: Talking to the semantic web: natural language query interfaces for
casual end-users. Ph.D. thesis, University (2009)
24. Keselman, A., Smith, C.A.: A classification of errors in lay comprehension of med-
ical documents. Journal of biomedical informatics 45(6), 1151–1163 (2012)
25. Kessels, S.J., Marshall, H.S., Watson, M., Braunack-Mayer, A.J., Reuzel, R.,
Tooher, R.L.: Factors associated with hpv vaccine uptake in teenage girls: a sys-
tematic review. Vaccine 30(24), 3546–3556 (2012)
26. Kirsch, I.S., et al.: Adult Literacy in America: A First Look at the Results of the
National Adult Literacy Survey. ERIC (1993)
27. Kutner, M.A., Greenberg, E., Baer, J.: National Assessment of Adult Literacy
(NAAL): a first look at the literacy of America’s adults in the 21st century. Na-
tional Center for Education Statistics, US Department of Education, Institute of
Education Sciences (2005)
28. Lieu, T.A., Glauber, J.H., Fuentes-Afflick, E., Lo, B.: Effects of vaccine informa-
tion pamphlets on parents’ attitudes. Archives of pediatrics & adolescent medicine
148(9), 921–925 (1994)
29. Lipizzi, E., Savas, L., Polivka, K., Coan, S., Shegog, R., Healy, C.M., Spinner,
S., Miller, C., Fernandez, M., Vernon, S.: Increasing HPV vaccination in a net-
work of pediatric clinics in Houston, TX: A baseline assessment. In: 2015 CPRIT
Conference. No. 463, Cancer Prevention Research Institute of Texas (November
2015)
30. Marlow, L.A., Zimet, G.D., McCaffery, K.J., Ostini, R., Waller, J.: Knowledge of
human papillomavirus (HPV) and HPV vaccination: An international comparison.
Vaccine 31(5), 763–769 (Jan 2013)
31. McRee, A.L., Brewer, N.T., Reiter, P.L., Gottlieb, S.L., Smith, J.S.: The carolina
HPV immunization attitudes and beliefs scale (CHIAS): scale development and
8
associations with intentions to vaccinate. Sexually transmitted diseases 37(4), 234–
239 (2010)
32. Migneault, J.P., Farzanfar, R., Wright, J.A., Friedman, R.H.: How to write health
dialog for a talking computer. Journal of biomedical informatics 39(5), 468–481
(2006)
33. Publishing, O.: OECD Skills Outlook 2013: First Results from the Survey of Adult
Skills. OECD Publishing (2013)
34. Rambout, L., Tashkandi, M., Hopkins, L., Tricco, A.C.: Self-reported barriers and
facilitators to preventive human papillomavirus vaccination among adolescent girls
and young women: a systematic review. Preventive medicine 58, 22–32 (2014)
35. Roter, D., Ellington, L., Erby, L.H., Larson, S., Dudley, W.: The genetic counseling
video project (GCVP): models of practice. In: American Journal of Medical Ge-
netics Part C: Seminars in Medical Genetics. vol. 142, pp. 209–220. Wiley Online
Library (2006)
36. Roter, D.L.: Oral literacy demand of health care communication: challenges and
solutions. Nursing outlook 59(2), 79–84 (2011)
37. Roter, D.L., Erby, L., Larson, S., Ellington, L.: Oral literacy demand of prenatal
genetic counseling dialogue: Predictors of learning. Patient education and counsel-
ing 75(3), 392–397 (2009)
38. Roter, D.L., Erby, L.H., Larson, S., Ellington, L.: Assessing oral literacy demand
in genetic counseling dialogue: preliminary test of a conceptual framework. Social
science & medicine 65(7), 1442–1457 (2007)
39. Rubin, D.L., Parmer, J., Freimuth, V., Kaley, T., Okundaye, M.: Associations
between older adults’ spoken interactive health literacy and selected health care
and health communication outcomes. Journal of health communication 16(sup3),
191–204 (2011)
40. Schonlau, M., Martin, L., Haas, A., Derose, K.P., Rudd, R.: Patients’ literacy skills:
more than just reading ability. Journal of health communication 16(10), 1046–1054
(2011)
41. St-Amour, M., Guay, M., Perron, L., Clément, P., Baron, G., Petit, G., Lemaire,
J.: Are vaccination information leaflets useful for vaccinators and parents? Vaccine
24(14), 2491–2496 (2006)
42. Street, R.L.: Information-giving in medical consultations: the influence of patients’
communicative styles and personal characteristics. Social science & medicine 32(5),
541–548 (1991)
43. US Department of Health and Human Services and Office of Disease Prevention
and Health Promotion and others: Healthy People 2020 (2012)
44. Vadaparampil, S.T., Kahn, J.A., Salmon, D., Lee, J.H., Quinn, G.P., Roetzheim,
R., Bruder, K., Malo, T.L., Proveaux, T., Zhao, X., et al.: Missed clinical oppor-
tunities: provider recommendations for HPV vaccination for 11–12 year old girls
are limited. Vaccine 29(47), 8634–8641 (2011)
45. Waitzkin, H.: Information giving in medical care. Journal of health and social
Behavior pp. 81–101 (1985)
46. Wang, D., Cunningham, R.M., Boom, J., Amith, M., Tao, C.: Towards a HPV Vac-
cine Knowledgebase For Patient Education Content. Studies in Health Technology
and Informatics (2016)
47. Zimet, G.D., Rosberger, Z., Fisher, W.A., Perez, S., Stupiansky, N.W.: Beliefs, be-
haviors and HPV vaccine: correcting the myths and the misinformation. Preventive
medicine 57(5), 414–418 (2013)