Detecting and representing contradictions and disagreements in medical guidelines Wlodek Zadrozny University of North Carolina at Charlotte Abstract we could formally reason about individual guidelines and guidelines corpora. I will describe some of our work on text mining and build- ing representations of contradictory information in medical This is a difficult problem, and even with injection of sub- guidelines. The talk will span from discussing specific archi- stantial resources, it is not clear it can be solved any time tectures we have been using to some very abstract formal rep- soon. However, we believe there are some technical prereq- resentations, and discuss many gaps that would need to be uisites that need to satisfied before we can start tackling this addressed before we can build reasoning systems to support problem: humans in medical decision making in this space. This pre- • We need to establish semantic repositories of guidelines sentation is largely based on joint work with my student Hos- and possibly relevant background material. This at the sein Hematialam and Dr. Luciana Garbayo from U. Central minimum is a specialized search engine enabling field Florida. search, and enabling adding additional automated annota- tions to the guidelines documents (Elastic Search or Solr 1 Introduction cold be a starting point). • A collection of document processing tools capable of M.Catillon (Catillon 2017) estimates that converting treatment guidelines documents to semi- structured formats amenable to deeper semantic process- “In August 2017, PubMed included about 27 million ing. In our view there is big gap here. citations, 500,000 clinical trials, 2 million reviews, 70,000 • A collection of linguistic tools capable of finding med- systematic reviews and/or meta-analyses, and 20,000 prac- ically related terms and relations (here we have GATE, tice guidelines. The rate of information growth is exploding: UIMA, Metamap, etc.). from 10 new clinical trials per day in 1975, to 55 in 1995, • A collection of tools to build discourse model repre- and 95 in 2015.” senting guidelines documents(the discourse processing field seems to be moving in the similar direction (IWCS This publication (Catillon 2017) also contains detailed 2019)). numbers about estimating the needs for systematic reviews • Tools to detect and reason with contradictory information of certain medical conditions, discussions of quality and (this we address now). quantity, etc. My point here is that medical guidelines is a large and im- portant part of healthcare. It is also widely noted that differ- 3 Tools to reason with contradictory ent accredited medical societies disagree about the treatment treatment guidelines information guidelines. Recent controversies about hypertension guide- We have done some preliminary work on detecting and rea- lines is just one of many examples (Hughes 2018). soning with contradictory information in the context of med- Except for estimated number of guideline documents be- ical guidelines. Thus in (Hematialam and Zadrozny 2017) ing in tens of thousands, we do not know how often two we introduce machine learning built language models al- treatment guidelines contradict each other, what happens if lowing us to find condition-action expressions in medical there are multiple conditions present at the same time (co- guidelines, and therefore potentially identify different ac- morbidities). We don’t have any numerical estimates of con- tions recommended for the same condition. In (Zadrozny, tradictions and disagreements, and we do not know how se- Hematialam, and Garbayo 2017), using a simple example rious they are. of mammography screening recommendations we showed that a combination of information retrieval, NLP, and text 2 The need to reason about guidelines mining tools allows us, in this simple case, to very reli- We believe patients outcomes will be improved, overtreat- ably pinpoint potentially contradictory recommendations. In ment will be reduced, and possibly better processes for cre- (Zadrozny and Garbayo 2018), we created a general model ation of treatment guidelines can be established, if only for reasoning about the some types of disagreements of- ten occurring in medical guidelines (frequency of checkups, sistencies are mine. The collaborators were not consulted on dosages, etc). The architecture of this model is shown in the the final version of this abstract figure below (reproduced from (Zadrozny, Hematialam, and Garbayo 2017)). Figure 1: The architecture used to evaluate extraction of contradictions in medical guidelines. 4 Towards better guidelines We are postulating that better tools will give us better pro- cesses for establishing treatment guidelines. As (Garbayo 2014) shows, experts opinions depend on the epistemic stances etc. I believe after discussions with L. Garbayo that we should be able to quantify and measure differences be- tween epistemic stances of different medical organization and analyze them interactively by playing with graphs such as these, showing the strength of semantic similarities be- tween different guideline documents via connections and thickness of lines, for example (shown below for illustration only). 5 Summary Better tools will give us better guidelines. Serious problems remain, but progress has been made, and one promising path was sketched above. I acknowledge discussions with H. Hematialam, L. Gar- bayo, X.Niu, and others. However, all the faults and incon- References Catillon, M. 2017. Medical knowledge synthesis: A brief overview. Garbayo, L. 2014. Epistemic considerations on expert disagreement, normative justification, and inconsistency re- garding multi-criteria decision making. In Constraint pro- gramming and decision making. Springer, Cham. 35–45. Hematialam, H., and Zadrozny, W. 2017. Iden- tifying condition-action statements in medical guide- lines using domain-independent features. arXiv preprint arXiv:1706.04206. Hughes, S. 2018. Us hypertension guide- lines: Cutting through the controversy. https://www.medscape.com/viewarticle/897118. IWCS. 2019. Iwcs-2019 shared task: Drs parsing. https://competitions.codalab.org/competitions/20220. Zadrozny, W., and Garbayo, L. 2018. A sheaf model of con- tradictions and disagreements. preliminary report and dis- cussion. arXiv preprint arXiv:1801.09036. Zadrozny, W.; Hematialam, H.; and Garbayo, L. 2017. To- wards semantic modeling of contradictions and disagree- ments: A case study of medical guidelines. arXiv preprint arXiv:1708.00850.