Detecting and representing contradictions and disagreements in medical guidelines

Abstract

I will describe some of our work on text mining and building representations of contradictory information in medical guidelines. The talk will span from discussing specific architectures we have been using to some very abstract formal representations, and discuss many gaps that would need to be addressed before we can build reasoning systems to support humans in medical decision making in this space. This presentation is largely based on joint work with my student Hossein Hematialam and Dr. Luciana Garbayo from U. Central Florida.

-

M.Catillon (Catillon 2017) estimates that

“In August 2017, PubMed included about 27 million citations, 500,000 clinical trials, 2 million reviews, 70,000 systematic reviews and/or meta-analyses, and 20,000 practice guidelines. The rate of information growth is exploding: from 10 new clinical trials per day in 1975, to 55 in 1995, and 95 in 2015.”

This publication (Catillon 2017) also contains detailed numbers about estimating the needs for systematic reviews of certain medical conditions, discussions of quality and quantity, etc.

My point here is that medical guidelines is a large and important part of healthcare. It is also widely noted that different accredited medical societies disagree about the treatment guidelines. Recent controversies about hypertension guidelines is just one of many examples (Hughes 2018) .

Except for estimated number of guideline documents being in tens of thousands, we do not know how often two treatment guidelines contradict each other, what happens if there are multiple conditions present at the same time (comorbidities). We don’t have any numerical estimates of contradictions and disagreements, and we do not know how serious they are.

The need to reason about guidelines

We believe patients outcomes will be improved, overtreatment will be reduced, and possibly better processes for creation of treatment guidelines can be established, if only we could formally reason about individual guidelines and guidelines corpora.

This is a difficult problem, and even with injection of substantial resources, it is not clear it can be solved any time soon. However, we believe there are some technical prerequisites that need to satisfied before we can start tackling this problem:

We need to establish semantic repositories of guidelines and possibly relevant background material. This at the minimum is a specialized search engine enabling field search, and enabling adding additional automated annotations to the guidelines documents (Elastic Search or Solr cold be a starting point).

A collection of document processing tools capable of converting treatment guidelines documents to semistructured formats amenable to deeper semantic processing. In our view there is big gap here.

A collection of linguistic tools capable of finding medically related terms and relations (here we have GATE, UIMA, Metamap, etc.).

A collection of tools to build discourse model representing guidelines documents(the discourse processing field seems to be moving in the similar direction (IWCS 2019) ).

Tools to detect and reason with contradictory information (this we address now).

3

Tools to reason with contradictory treatment guidelines information

We have done some preliminary work on detecting and reasoning with contradictory information in the context of medical guidelines. Thus in (Hematialam and Zadrozny 2017) we introduce machine learning built language models allowing us to find condition-action expressions in medical guidelines, and therefore potentially identify different actions recommended for the same condition. In (Zadrozny, Hematialam, and Garbayo 2017) , using a simple example of mammography screening recommendations we showed that a combination of information retrieval, NLP, and text mining tools allows us, in this simple case, to very reliably pinpoint potentially contradictory recommendations. In (Zadrozny and Garbayo 2018) , we created a general model for reasoning about the some types of disagreements often occurring in medical guidelines (frequency of checkups, dosages, etc). The architecture of this model is shown in the figure below (reproduced from (Zadrozny, Hematialam, and Garbayo 2017) ). sistencies are mine. The collaborators were not consulted on the final version of this abstract We are postulating that better tools will give us better processes for establishing treatment guidelines. As (Garbayo 2014) shows, experts opinions depend on the epistemic stances etc. I believe after discussions with L. Garbayo that we should be able to quantify and measure differences between epistemic stances of different medical organization and analyze them interactively by playing with graphs such as these, showing the strength of semantic similarities between different guideline documents via connections and thickness of lines, for example (shown below for illustration only).

5

Summary

Better tools will give us better guidelines. Serious problems remain, but progress has been made, and one promising path was sketched above.

I acknowledge discussions with H. Hematialam, L. Garbayo, X.Niu, and others. However, all the faults and incon

References

Catillon , M. 2017 . Medical knowledge synthesis: A brief overview .

Garbayo , L. 2014 . Epistemic considerations on expert disagreement, normative justification, and inconsistency regarding multi-criteria decision making. In Constraint programming and decision making . Springer, Cham. 35 - 45 .

Hematialam , H. , and Zadrozny , W. 2017 . Identifying condition-action statements in medical guidelines using domain-independent features . arXiv preprint arXiv:1706 . 04206 .

Hughes , S. 2018 . Us hypertension guidelines: Cutting through the controversy .

https://www.medscape.com/viewarticle/897118.

IWCS. 2019 . Iwcs-2019 shared task: Drs parsing .

https://competitions.codalab.org/competitions/20220.

Zadrozny , W. , and Garbayo , L. 2018 . A sheaf model of contradictions and disagreements . preliminary report and discussion . arXiv preprint arXiv: 1801 .09036.

Zadrozny , W. ; Hematialam, H.; and Garbayo , L. 2017 . Towards semantic modeling of contradictions and disagreements: A case study of medical guidelines . arXiv preprint arXiv:1708 . 00850 .