Ontology-based dialogue systems for improved patient HPV vaccine knowledge and perception

Abstract

In Sir Tim Berners-Lee's seminal article that introduce his vision of the semantic web, one of the use-cases described was a healthrelated example where health consumers utilized intelligent hand-held devices that aggregated and exchanged health data from the semantic web. Presently, the majority of health consumers and patients rely on personal technology and the web to nd information and to make personal health decisions. This proposal aims to contribute towards that usecase, speci cally in the \hot-bed" issue of human papillomavirus (HPV) vaccine. The HPV vaccine targets young adults and teens to protect against life-threatening cancers, yet a segment of the public has reservations against the vaccine. I propose an interactive dialogue agent that harness patient-level vaccine information encoded in an ontology that can be \talked to" with a natural language interface using utterances. I aim to pilot this technology in a clinic to assess if patient knowledge about HPV and the vaccine is increased, and if their attitude toward the vaccine is modi ed as a result of using the interactive agent.

Keywords

• ontology
• natural language processing
• dialogue system
• conversational agents
• vaccine
• ontology learning
• question-answering

-

At 79 million reported cases, human papillomavirus (HPV) infection is one of the most common sexually transmitted infection, and per year, according to the Centers for Disease and Control, 14 million more infections will be reported [ 10 ]. For many of the life threatening cancers, like cervical cancer, the HPV virus (Type 16 and 18) is attributed to HPV infection. The human papillomavirus (HPV) vaccine is noted to be 100% e ective against the HPV virus [ 9 ], and yet the United States' population uptake for HPV vaccine is at a low 28% [ 11 ], short of the 80% uptake goal [ 43 ]. Irrespective of the e cacy of the HPV vaccine, it has yet to be fully embraced by the public, patients, and even healthcare providers [ 18 ]. Most HPV information interventions are paper-based in the form of brochures, yers, etc. [ 16 ], yet, patients ignore, misunderstands, or the content is not e ective [ 28, 41, 26, 27, 33, 1 ]. There is strong evidence that patientprovider vaccine counseling has a positive in uence in a ecting knowledge and vaccination rates [ 33, 34, 24 ].

Research Question: I postulate, will an ontology-driven health conversational agent improve the vaccine literacy of patients and/or have an in uence on patients' intention to vaccinate? This research will theorize that an interactive agent, verbally communicating HPV vaccine knowledge, will result in the likelihood in users opting for the HPV vaccine, improve their vaccine literacy, and improve patient satisfaction and the patient-provider relationship. Using an ontological knowledge-base, we can aggregate disparate and complex health information using OWL/RDF, and also contribute to validated, comprehensive HPV vaccine knowledge for linked open data. Because ontological knowledge-base will be semantically enriched and readable by machines, this will allow users to utilize natural language utterance to query information without any background expertise in complex query languages. This will provide a natural language interface where patients can easily interact with a knowledge-base and get precise information, and perhaps, foster further applicable research for consumer usability and interaction with the semantic web. 2

Related Work

Several studies reveal that oral communication in health literacy is as important as written communication when it comes to delivering health information for patients [ 36, 39, 40 ]. Patients are also likely to trust face-to-face interaction to acquire health information [ 19 ]. Personalizing health information that is esoteric or abstract for patients with low health literacy bene ts their comprehension and understanding [ 37 ]. A study from genetic counseling research on patient communication discovered that patients appear to learn better with interactivity during a counseling session [ 37 ]. Patients who ask questions are also likely to attain better health information from their provider [ 42, 45 ]. Also, patients favored dialogue with health provider that is uncomplicated and responses that swiftly addresses their concern [ 35, 13 ].

As it stands, there are providers who could educate patients about the HPV vaccine. Yet this is an added burden to the provider. One of them is being a good communicator, especially to patients who have low health literacy [ 17, 38, 15 ]. Another burden is taking the role of being health educator [ 47 ], even in the di cult situation of talking about sexual-related matters with both the parent and the teenage child [ 14, 21 ]. More importantly, there is not enough time to discuss and counsel the patient about the HPV vaccine [ 17, 44 ]. This proposal submits the possibility of delegating and automating the communication and education task to an interactive kiosk or tablet application that would be available for patients, similar to a \Siri"-like experience. The possible bene t other than cost e ectiveness [ 3 ] is facilitating the interactive dialogue that could approximate a real conversation [ 32 ], mimicking the personal interaction that patients and health consumers desire [ 22, 2 ]. Also using machine intelligence, the system can strategize how and when information is provided [ 32 ]. The research have noted that face-to-face counseling between patient and provider has an in uence in accepting vaccination for the patient or the patient's parent [ 25, 34 ]. Specifically, the verbal mode of the health dialogue system can o er opportunities to enhance interactivity between patient and provider, such as using machine intelligence for decision making and coordination of content delivery, utilizing interpersonal cues to imitate human conversation and improve communication e orts with non-experts [ 32 ]. 3

Approach & Hypotheses

I intend to design and develop a interactive dialogue system speci cally for patient question-answering for patient-level vaccine information and to a ect vaccination attitudes, and then pilot the system with participants at a pediatric clinic.

Research Objective 1 (RO1): Generate a comprehensive vaccine knowledgebase using natural language processing (NLP) tools. Hypothesis: Patient-level vaccine information can be faithfully represented in an ontological knowledgebase. I will gather a corpus of patient-level vaccine and HPV information documentation in electronic format, and will utilize an natural language information extraction tool to extract predicates. I will evaluate the accuracy and meaningfulness of the results of the extraction. Based on the results of the evaluation, the information will be encoded using Protege to produce an ontological knowledgebase. After publishing the ontology, we will evaluate it based on a semiotic framework for ontology evaluation [ 8 ].

Research Objective 2 (RO2): Design a speech-enabled conversational agent for patients to interface with the vaccine KB. Hypothesis: An ontology-driven speech interface can accurately produce answers for patient questions. I will design a question-answering query engine to allow patients to ask questions about the HPV vaccine and related information through thin-client tablet application. The query engine will automatically translate natural language questions to SPARQL queries, which is then executed to a remote triple-store hosting the vaccine knowledgebase. In addition, the agent will model some cues and best practices for oral communication of health information for better patient knowledge acquisition.

Research Objective 3 (RO3): Demonstrate feasibility of interactive conversational agent for enhancing patients' vaccine knowledge. Hypothesis: Users of the interactive intervention will experience greater education gain, trust in the intervention, and a stronger intention to vaccinate with HPV vaccine than the control group. I will coordinate with our collaborators at the Texas Children's Hospital to launch this pilot study with patient participants in one of their network clinics. The Texas Children's Pediatric (TCP) clinics is the most prominent network of pediatric clinics serving the Houston area [ 29 ]. I will implement a randomized two-group post-test research design where the experimental group (participants utilizing the interactive agent) and a control group (participants with vaccine pamphlets) will be compared.

Evaluation plan

Each research objective is associated with an evaluation process(es) in order to measure outcomes of the research e orts.

RO1: To faithfully represent the information collected from the corpus through information extraction, this aim will need to evaluate the accuracy of the extraction results and the nal ontological knowledgebase. Speci cally, we will evaluate the extraction results based on contextual accuracy and correctness of the triple decomposition from sentences through the information extraction process. In agreement, two evaluators will assess the output based on contextual accuracy and minimally decomposed triples.

After serializing the vaccine information from the various resources, we will use Burton-Jones, et al.'s semiotic metric framework [ 8 ] to evaluate the ontological knowledgebase, with an automated, web-based Java application I have developed [ 4 ]. Health professional experts will evaluate the accuracy of the information to derive one of the framework's scores.

RO2: The rst part of the evaluation will assess the e ectiveness of natural language queries uttered by the consumer to be transformed into a SPARQL query. For this segment of the evaluation, we will rely on system-level evaluation and convert the the natural language queries to string data structures, since speech recognition APIs and toolkits may not be accurate and distort the results. I will calculate precision and recall based on [ 23 ] calculations for ontology-based question-answering systems. Despite possible performance variances with different datasets, I may evaluate the accuracy of engine with unrelated subject ontologies to assess comparable outcomes found in literature.

Also, for each question, there will be an intended response to determine answer su ciency (\good enough" answer). Three investigators will compare the response generated from the question-answering engine to assess the quality of the answer with the question, and they will compare the results and rate the results based on a 5-point Likert scale. Inter-rater agreement scores will be examined using Cohen's Kappa.

RO3: After completion of the pilot, we will collect the data and preform several analyses from the surveys of the participants. Aside from demographic data surveys, each participants will complete a survey to measure intention based on the health belief model [ 31 ], a survey to measure trust with the agent [ 20 ], and a validated HPV-related knowledge test [ 30 ]. To assess knowledge gain, we will calculate a t-Test comparing mean knowledge test scores between participants who used the conversational agent (AU ) and participants who did not (N AU ). Similarly, we will also observe overall intention to vaccinate based on the mean average of the two groups' vaccination intention surveys and employ a t-Test calculation to verify signi cant di erences. To discover the users' trust with the conversational agent, we will utilize the overall midpoint of agent user participants and denote the number of users whose trust scores exceeded the midpoint mean of the trust scores, similar to [ 7 ]. With regression analysis, I want to investigate whether the intention to vaccinate is a ected by trust in agent (independent variable) and/or knowledge gain (independent variable).

Preliminary results

Last year we developed a proof of concept prototype to determine technical feasibility and future improvements [ 6 ]. Some of these areas include expanding the domain ontology, enhancing the speech interface mobile application for unconstrained queries and user personalization, and investigating the development of application-based ontologies. I recently worked on an initial meta-level ontology framework called VISO (Vaccine Information Statement Ontology) that is sourced from federally-mandated vaccine documentation by the Centers for Disease and Control [ 4 ]. Using a semiotic-inspired ontology quality evaluation metric, VISO was revealed to have strong pragmatic qualities, but needed additional improvement with syntactic aspects.

Additionally, the vaccine ontology work has been extended with initial development of an HPV vaccine version initiated last year [ 46 ], and we are experimenting with open information extraction (\ClausIE" [ 12 ]) for ontology development for patient-centric HPV cancer knowledge [ 5 ], which we plan on utilizing to extend the HPV vaccine ontology, to enrich the knowledge space with patient-level cancer information. Domain-expert collaborators had advised that our vaccine ontology needs to include anecdotal stories, certain multimedia content, vaccine myths, and personalization, all to compliment the patient-level factoids and improve the information quality. 6

Re ections

My work is in the early stages, and we foresee a two-year projection to complete the three research objectives. The literature in utilizing dialogue or conversational agents for patient-centric health care is available but relatively scarce, especially ontology-based implementations. However, there has yet to be research that involves the use of dialogue agents that can a ect health consumers' attitudes for vaccination with the combination of knowledge and utilizing best practices in patient-provider communication and verbal health literacy. As the adage goes, this may either be a \blessing" or a \curse" as there is no guarantee that ontology-driven interactive agents can a ect vaccine attitudes. In collaboration with health communication experts, this multi-disciplinary study will apply research in patient-provider communication that can be ported to technological solutions, and I will be working with experts in public health to help tailor this intervention to show some impact that can help further the research in ontology-based conversational agents in patient-centered health care. Acknowledgments I thank my supervisor, Cui Tao, PhD, and my committee for their guidance and feedback. Support was provided by a fellowship funded by the Cancer Prevention Research Institute of Texas (CPRIT) Training Grant #RP160015. Special thanks to the Immunization Project of the Texas Children's Hospital and the CPRIT research faculty for their collaboration.

References

1. Agarwal , N. , Chaudhari , A. , Hansberry , D.R. , Tomei , K.L. , Prestigiacomo , C.J.: A comparative analysis of neurosurgical online education materials to assess patient comprehension . Journal of Clinical Neuroscience 20 ( 10 ), 1357 { 1361 ( 2013 )

2. Ahken , S. , Fleming , N. , Dumont , T. , Black , A. : HPV awareness in higher-risk young women: The need for a targeted HPV catch-up vaccination program . J Obstet Gynaecol Can 37 ( 2 ), 122 { 128 ( 2015 )

3. Allen , J. , Ferguson , G. , Blaylock , N. , Byron , D. , Chambers , N. , Dzikovska , M. , Galescu , L. , Swift , M. : Chester: towards a personal medication advisor . Journal of biomedical informatics 39(5) , 500 { 513 ( 2006 )

4. Amith , M. , Gong , Y. , Cunningham , R. , Boom , J. , Tao , C. : Developing VISO : Vaccine Information Statement Ontology for patient education . Journal of Biomedical Semantics 6 ( 1 ), 23 (May 2015 ), http://www.jbiomedsem.com/content/6/1/23/abstract

5. Amith , M. , Song , H.Y. , Zhang , Y. , Tao , C. : Lightweight predication extraction for patient-level cancer information and ontology development (In preparation, 2016 )

6. Amith , M. , Tao , C. : A prototype mobile virtual assistant for semantic-based vaccine information retrieval . In: Zheng, X. , Zeng , D. , Chen , H. , Zhang, Y. , Xing , C. , Neill , D . (eds.) Smart Health, Lecture Notes in Computer Science , vol. 8549 , pp. 200 { 205 . Springer International Publishing (Jan 2014 ), http://dx.doi. org/10.1007/978- 3-319-08416-9 21

7. Bickmore , T.W. , Pfeifer , L.M. , Byron , D. , Forsythe , S. , Henault , L.E. , Jack , B.W. , Silliman , R. , Paasche-Orlow , M.K. : Usability of conversational agents by patients with inadequate health literacy: evidence from two clinical trials . Journal of health communication 15 Suppl 2 , 197 ( 2010 )

8. Burton-Jones , A. , Storey , V.C. , Sugumaran , V. , Ahluwalia , P.: A semiotic metrics suite for assessing the quality of ontologies . Data & Knowledge Engineering 55 ( 1 ), 84 {102 (Oct 2005 )

9. Centers for Disease Control and Prevention: HPV vaccine resources for health professions . http://www.cdc.gov/vaccines/who/teens/for-hcp/hpv-resources. html ( 2014 )

10. Centers for Disease Control and Prevention (CDC) and others: CDC fact sheet: incidence, prevalence, and cost of sexually transmitted infections in the united states . CDC ( 2013 )

11. Centers for Disease Control and Prevention (CDC) and others: Human papillomavirus vaccination coverage among adolescent girls, 2007 - 2012 , and postlicensure vaccine safety monitoring, 2006-2013-united states . MMWR. Morbidity and mortality weekly report 62(29) , 591 ( 2013 )

12. Del Corro , L. , Gemulla , R.: ClausIE: clause-based open information extraction . In: Proceedings of the 22nd international conference on World Wide Web . pp. 355 { 366 . International World Wide Web Conferences Steering Committee ( 2013 ), http://dl.acm.org/citation.cfm?id= 2488420

13. Doak , C.C. , Doak , L.G. , Root , J.H. : Teaching patients with low literacy skills . AJN The American Journal of Nursing 96 ( 12 ), 16M ( 1996 )

14. Esposito , S. , Bosis , S. , Pelucchi , C. , Begliatti , E. , Rognoni , A. , Bellasio , M. , Tel , F. , Consolo , S. , Principi , N.: Pediatrician knowledge and attitudes regarding human papillomavirus disease and its prevention . Vaccine 25 ( 35 ), 6437 { 6446 ( 2007 )

15. Evans , G. , Bostrom , A. : The evolution of vaccine risk communication in the united states: 1982{2002 . National Institute of Allergy and Infectious Diseases, ed. Jordan Report 20th anniversary: Accelerated Development of Vaccines pp. 58 { 71 ( 2002 )

16. Fu , L.Y. , Bonhomme , L.A. , Cooper , S.C. , Joseph , J.G. , Zimet , G.D.: Educational interventions to increase HPV vaccination acceptance: A systematic review . Vaccine 32 ( 17 ), 1901 { 1920 ( 2014 ), http://www.sciencedirect.com/science/article/pii/S0264410X14001546

17. Go , S.L. , Mazor , K.M. , Gagne , S.J. , Corey , K.C. , Blake , D.R. : Vaccine counseling: A content analysis of patient{physician discussions regarding human papilloma virus vaccine . Vaccine 29 ( 43 ), 7343 { 7349 ( 2011 )

18. Gottlieb , S.L. , Low , N. , Newman , L.M. , Bolan , G. , Kamb , M. , Broutet , N.: Toward global prevention of sexually transmitted infections (STIs): the need for STI vaccines . Vaccine 32 ( 14 ), 1527 { 1535 ( 2014 )

19. Hesse , B.W. , Nelson , D.E. , Kreps , G.L. , Croyle , R.T., Arora , N.K. , Rimer , B.K. , Viswanath , K. : Trust and sources of health information: the impact of the internet and its implications for health care providers: ndings from the rst health information national trends survey . Archives of internal medicine 165(22) , 2618 { 2624 ( 2005 )

20. Horvath , A.O. , Greenberg , L.S. : Development and validation of the working alliance inventory . Journal of counseling psychology 36(2) , 223 ( 1989 )

21. Humiston , S.G. , Albertin , C. , Scha er , S., Rand , C. , Shone , L.P. , Stokley , S. , Szilagyi , P.G. : Health care provider attitudes and practices regarding adolescent immunizations: a qualitative study . Patient education and counseling 75(1) , 121 { 127 ( 2009 )

22. Katz , M.L. , Krieger , J.L. , Roberto , A.J. : Human papillomavirus (HPV): college male's knowledge, perceived risk, sources of information, vaccine barriers and communication . Journal of men's health 8(3) , 175 { 184 ( 2011 )

23. Kaufmann , E.: Talking to the semantic web: natural language query interfaces for casual end-users . Ph.D. thesis , University ( 2009 )

24. Keselman , A. , Smith , C.A. : A classi cation of errors in lay comprehension of medical documents . Journal of biomedical informatics 45(6) , 1151 { 1163 ( 2012 )

25. Kessels , S.J. , Marshall , H.S. , Watson , M. , Braunack-Mayer , A.J. , Reuzel , R. , Tooher , R.L. : Factors associated with hpv vaccine uptake in teenage girls: a systematic review . Vaccine 30 ( 24 ), 3546 { 3556 ( 2012 )

26. Kirsch , I.S. , et al.: Adult Literacy in America: A First Look at the Results of the National Adult Literacy Survey . ERIC ( 1993 )

27. Kutner , M.A. , Greenberg , E. , Baer , J.: National Assessment of Adult Literacy (NAAL): a rst look at the literacy of America's adults in the 21st century. National Center for Education Statistics, US Department of Education, Institute of Education Sciences ( 2005 )

28. Lieu , T.A. , Glauber , J.H. , Fuentes-A ick , E., Lo , B. : E ects of vaccine information pamphlets on parents' attitudes . Archives of pediatrics & adolescent medicine 148(9) , 921 { 925 ( 1994 )

29. Lipizzi , E. , Savas , L. , Polivka , K. , Coan , S. , Shegog , R. , Healy , C.M. , Spinner , S. , Miller , C. , Fernandez , M. , Vernon , S. : Increasing HPV vaccination in a network of pediatric clinics in Houston, TX: A baseline assessment . In: 2015 CPRIT Conference. No. 463, Cancer Prevention Research Institute of Texas (November 2015 )

30. Marlow , L.A. , Zimet , G.D. , McCa ery , K.J., Ostini , R. , Waller , J. : Knowledge of human papillomavirus (HPV) and HPV vaccination: An international comparison . Vaccine 31 ( 5 ), 763 {769 (Jan 2013 )

31. McRee , A.L. , Brewer , N.T. , Reiter , P.L. , Gottlieb , S.L. , Smith , J.S.: The carolina HPV immunization attitudes and beliefs scale (CHIAS): scale development and associations with intentions to vaccinate . Sexually transmitted diseases 37(4) , 234 { 239 ( 2010 )

32. Migneault , J.P. , Farzanfar , R. , Wright , J.A. , Friedman , R.H. : How to write health dialog for a talking computer . Journal of biomedical informatics 39(5) , 468 { 481 ( 2006 )

33. Publishing , O. : OECD Skills Outlook 2013 : First Results from the Survey of Adult Skills . OECD Publishing ( 2013 )

34. Rambout , L. , Tashkandi , M. , Hopkins , L. , Tricco , A.C. : Self-reported barriers and facilitators to preventive human papillomavirus vaccination among adolescent girls and young women: a systematic review . Preventive medicine 58 , 22 { 32 ( 2014 )

35. Roter , D. , Ellington , L. , Erby , L.H. , Larson , S. , Dudley , W.: The genetic counseling video project (GCVP): models of practice . In: American Journal of Medical Genetics Part C: Seminars in Medical Genetics . vol. 142 , pp. 209 { 220 . Wiley Online Library ( 2006 )

36. Roter , D.L. : Oral literacy demand of health care communication: challenges and solutions . Nursing outlook 59(2) , 79 { 84 ( 2011 )

37. Roter , D.L. , Erby , L. , Larson , S. , Ellington , L. : Oral literacy demand of prenatal genetic counseling dialogue: Predictors of learning . Patient education and counseling 75(3) , 392 { 397 ( 2009 )

38. Roter , D.L. , Erby , L.H. , Larson , S. , Ellington , L. : Assessing oral literacy demand in genetic counseling dialogue: preliminary test of a conceptual framework . Social science & medicine 65(7) , 1442 { 1457 ( 2007 )

39. Rubin , D.L. , Parmer , J. , Freimuth , V. , Kaley , T. , Okundaye , M. : Associations between older adults' spoken interactive health literacy and selected health care and health communication outcomes . Journal of health communication 16(sup3) , 191 { 204 ( 2011 )

40. Schonlau , M. , Martin , L. , Haas , A. , Derose , K.P. , Rudd , R.: Patients' literacy skills: more than just reading ability . Journal of health communication 16(10) , 1046 { 1054 ( 2011 )

41. St-Amour , M. , Guay , M. , Perron , L. , Clement , P. , Baron , G. , Petit , G. , Lemaire , J.: Are vaccination information lea ets useful for vaccinators and parents? Vaccine 24 ( 14 ), 2491 { 2496 ( 2006 )

42. Street , R.L. : Information-giving in medical consultations: the in uence of patients' communicative styles and personal characteristics . Social science & medicine 32(5) , 541 { 548 ( 1991 )

43. US Department of Health and Human Services and O ce of Disease Prevention and Health Promotion and others: Healthy People 2020 ( 2012 )

44. Vadaparampil , S.T. , Kahn , J.A. , Salmon , D. , Lee , J.H. , Quinn , G.P. , Roetzheim , R. , Bruder , K. , Malo , T.L. , Proveaux , T. , Zhao , X. , et al.: Missed clinical opportunities: provider recommendations for HPV vaccination for 11{12 year old girls are limited . Vaccine 29 ( 47 ), 8634 { 8641 ( 2011 )

45. Waitzkin , H.: Information giving in medical care . Journal of health and social Behavior pp. 81 { 101 ( 1985 )

46. Wang , D. , Cunningham , R.M. , Boom , J. , Amith , M. , Tao , C. : Towards a HPV Vaccine Knowledgebase For Patient Education Content . Studies in Health Technology and Informatics ( 2016 )

47. Zimet , G.D. , Rosberger , Z. , Fisher, W.A. , Perez , S. , Stupiansky , N.W. : Beliefs, behaviors and HPV vaccine: correcting the myths and the misinformation . Preventive medicine 57(5) , 414 { 418 ( 2013 )