We present a comparative analysis of two sets of Referent Tracking Tuples (RTT), which each author of this paper crafted independently from the other and which are about the same portion of reality that one could assume to be described faithfully through registered diagnoses in the problem list of an electronic healthcare record system. The analysis thereby focused on (1) the choice of particulars that each of the authors deemed necessary and sufficient for an accurate description, (2) what these particulars are instances of, (3) how they relate to each other, and (4) the motivations of each author for the choices made. It was found that despite the large variety in RTTs crafted, there was wide, though not total, agreement about the appropriateness of the choices made. Disagreements arose from various issues such as potential lack of orthogonality in the OBO Foundry and in some cases on what types the classes in the ontologies represent. The authors' main source of disagreement was due to different interpretations of the literature on Information Content Entities (ICEs).
When during a clinical encounter a provider establishes a diagnosis for a patient under his care, he typically enters one or more diagnostic codes into that patient’s electronic healthcare record (EHR). When the same patient is later seen by a distinct provider, for instance for a second opinion or for a reason not related to the first encounter, this second provider will also enter one or more diagnostic codes. Patients’ records tend to accumulate many of these diagnostic assertions, specifically when the providers are working in the same EHR, or when such information is transferred from one to the other. They are also accumulated when records from various systems are merged into data warehouses equipped with master patient index facilities. There is a large variety amongst EHR systems and data warehouse interfaces in how they display such diagnostic information, a small hypothetical example being shown in Table 1.
A problem with information provided in this way, is that
it is not possible to construct a completely accurate view on
what is (and has been) the case in reality
ID 2 1234 3 1234 274.9: Gout, unspecified 715.97: Osteoarthrosis, unspecified whether generalized or localized, ankle and foot Table 1: Two diagnoses provided on the same day, about the same patient, entered by two distinct EHR users.
Date entered 9-1-2014 9-1-2014
Entered by J. Doe S. Thump persons that entered the diagnoses also made the diagnoses, how the dates when diagnoses were entered relate to the dates when the diagnoses were actually made, and so forth.
Referent Tracking (RT) is a methodology to avoid,
resolve, and document these sorts of ambiguities in EHRs
The purpose of the work reported here was to assess to what extent the authors of this paper—two experts in RT— would be able to develop independently from one another a collection of RTTs that describe the same POR in a semantically-interoperable way. The analysis presented is the first step in this endeavor and focuses on (1) the choice of particulars deemed necessary and sufficient for an accurate description of the selected POR, (2) what these particulars are instances of, (3) how they relate to each other, and (4) the motivations of each author for the choices made. 2
The POR selected for the experiment was the one
ambiguously described in Table 1. Since the goal of the exercise
was not to identify nor, when possible, resolve ambiguities,
it was further specified that the diagnoses were about the
same disorder, in the sense as formulated in the foundations
for the Ontology of General Medical Science (OGMS)
Table 2 lists the particulars and what they are instances of as originally—thus prior to comparison of the proposed representations—argued for by the author hereafter referred to as ‘X’. Table 3 does so for author Y. Each row represents part of an RTT asserting that the particular denoted in the ‘IUI’column is an instance of the universal denoted by the representational unit (RU) in the ‘Class’-column, drawn from the ontology named in the ‘Ontology’-column. The description relates the particulars informally to the scenario analyzed. The column labeled ‘Ind.’ contains the IUIs of the Information Content Entities (ICE) of which the RTTs themselves are concretizations. The columns ‘Y’ and ‘X’ contain scores reflecting how Y, resp. X, after discussion considered the RTT appropriate, ‘0’ meaning ‘not at all’, ‘1’ ‘ok, but’, and ‘2’ ‘absolutely’. Table 4 and 5 list for X and Y respectively the RTTs involving non-instantiation relationships. An IUI or Index in bold indicates that the corresponding POR is referred to by both authors. Author X listed 21 particulars involving 23 instantiations; Y did so for 28 particulars involving 1 instantiation each, not counting in both cases as particulars the temporal regions related to the timeindexing required for certain RTTs. 39 distinct particulars were identified, 10 of them by both authors. For only 2 (RTTs T2 and T3) did both authors select the same instantiating universal while for 2 others (T4 and T5) universals were selected from distinct ontologies, but with a close, nevertheless debatable, match. For the remaining 6, the universals chosen stand in is-a relations.
X drew 9 classes from 5 realism-based ontologies – the OGMS, the Ontology of Medically Related Social Entities (OMRSE), the Foundational Model of Anatomy (FMA), the Disease Ontology (DO) and the Ontology of Biomedical Investigations (OBI)—and identified the need for three more classes—‘denotator’, ‘EHR’ and ‘dataset record’, for which no realism-based ontology was found. Y used 9 classes drawn from 4 realism-based ontologies, 2 of which (FMA and OGMS) were also used by X, and 2 distinct ones: the Basic Formal Ontology (BFO) and the Information Artifact Ontology (IAO). He also identified the need for 2 classes currently without an ontological home: ‘patient identifier’ and ‘ICD-9-CM code and label’, as well as 2 classes (Gout and Osteoarthrosis, R49 and R51 in Table 5) for which he did not identify any particular as being required for an accurate description of the scenario.
53 particular-to-particular relationships in total were represented: 22 alone by X, 27 alone by Y and 4 (R14, R21, R22 and R23 in Table 4 and 5) by both authors, be it nevertheless through distinct, yet synonymous formulations. Y listed also two RTTs, each one expressing aboutness between a particular and a universal (R49 and R51, Table 5). Ind. IUI Description Ontology Class T24 P22 the ICE which is concretized in IAO ICE the spreadsheet you might be looking at T25 P23 the portion of chalk on the BFO Material 2 2 blackboard which make up what entity we call 'that spreadsheet' T26 P24 the pattern of chalk lines, spac- BFO Quality 2 2 es, characters, etc., in that portion of chalk T27 P1 the material entity whose ID is BFO Material 1 1 ‘1234’ in the spreadsheet entity T28 P15 the patient identifier which is Patient identi- 2 2 concretized in each first cell of fier the 2nd and 3rd row of the concretization of P22 T29 P25 the portion of chalk making up BFO Material 2 2 the text string ‘1234’ in the first entity cell of the 2nd row T30 P26 the quality in P25 which makes BFO Quality 2 2
P25 a concretization bearer T31 P27 portion of chalk making up the BFO Material 2 2 string ‘1234’ in the 1st cell of entity the 3rd row of the spreadsheet T32 P28 the quality in P27 which makes BFO Quality 2 2
P27 a concretization bearer T4 P4 the diagnosis which is concre- OGMS Diagnosis 2 2 tized in the first two cells of the 2nd row of the concretization of
P22 in front of your eyes T33 P29 the quality through which P4 is BFO Quality 2 2 concretized T5 P5 the diagnosis concretized in the OGMS Diagnosis 2 2 first two cells of the 3rd row of the concretization of P22 in front of your eyes T34 P30 the quality through which P5 is BFO Quality 2 2 concretized T2 P2 the person whose name is ‘J. FMA Human being 1 1
Doe’ in the spreadsheet T3 P3 the person whose name is ‘S. FMA Human being 1 1
Thump’ in the spreadsheet T35 P31 the clinical picture about P1 OGMS Clinical pic- 2 2 available to P2 and P3 ture T36 P32 part of the life of P1 which is OGMS Bodily pro- 1 1 described in P31 cess T37 P9 the interpretive process which OGMS Bodily pro- 1 2 resulted in P4 cess T38 P10 the interpretive process which OGMS Bodily pro- 2 2 resulted in P5 cess T39 P33 the disease in P1 OGMS Disease 2 2 T40 P16 the ICE concretized in the 2nd Icd-9-cm code cell of the 2nd row and label T41 P34 the quality through which P16 is BFO Quality 2 2 concretized T42 P18 the ICE concretized in the 2nd Icd-9-cm code 2 2 cell of the 3rd row and label T43 P35 the quality through which P18 is BFO Quality 2 2 concretized T44 P36 the process of, as we say ‘enter- BFO Process 2 2 ing’ diagnosis 1 in the EHR' T45 P37 the quality of some part of some BFO Quality 2 2 hard disk which concretizes d1 T46 P38 the process of, as we say ‘enter- BFO Process 2 2 ing’ diagnosis 2 in the EHR’ T47 P39 the quality of some part of some BFO Quality 2 2 hard disk which concretizes diagnosis 2 Table 3: Particulars and what they are instances of from the perspective of author ‘Y’.
Ind. : RTT in abstract syntax without time-component Y X R1 : P1 RO:bearer of P13 2 2 R2 : P1 RO:has part P6 2 2 R3 : P1 RO:has part P21 2 2 R4 : P10 RO:realizes P12 2 2 R5 : T7 corresponds with P16 2 2 R6 : T8 corresponds with P18 2 2 R7 : P15 RO:part of P7 1 1 R8 : P15 RO:part of P8 1 1 R9 : P15 IAO:denotes P1 0 1 R10 : P16 RO:part of P7 1 1 R11 : P16 IAO:denotes P4 1 1 R12 : P17 RO:part of P7 1 1 R13 : P17 IAO:denotes P2 0 1 R14 : P2 RO:agent of P9 2 2 R15 : P2 RO:bearer of P11 2 2 R16 : P18 RO:part of P8 1 1 R17 : P18 IAO:denotes P5 0 1 R18 : P19 IAO:denotes P3 0 1 R19 : P21 RO:has part P6 1 1 R20 : P3 RO:bearer of P12 2 2 R21 : P3 RO:agent of P10 2 2 R22 : P4 OBI:is specified output of P9 2 2 R23 : P5 OBI:is specified output of P10 2 2 R24 : P7 IAO:is about P6 2 2 R25 : P8 IAO:is about P6 2 2 R26 : P9 RO:realizes P11 2 2 Table 4: particular to particular relationships listed by author X
X indicated from which ontologies the relationships were drawn. Y used relations from the BFO 2.0 Draft Specifications, or under discussion in the context of the IAO. 4
Despite the large variation in RTTs crafted for what at first sight looks like a simple POR, there was after discussion wide, though not total agreement, about the appropriateness of the choices made (agreement is indicated by the same scores appearing in the X and Y columns of Tables 2-5). ‘Appropriateness’ is here to be measured in terms of what an optimal collection of RTTs for the POR under scrutiny would be since one could argue that the ground truth for what is expressed in EHR entries is largely unknown. The ‘ground truth’ is thus much broader than just what the patient had (this being part of the non-assertional part of the POR): it includes what the clinicians stated about what the patient had (these statements being part of the assertional part of the POR). If what the patient precisely had cannot be inferred from what was stated, it would be wrong to construct a collection of RTTs that states that the patient has such or such a specific type of disorder. To represent the non-assertional part of a POR that a collection of assertions is about, one has to resort to these assertions and to what has already been established to be the case through other means. The optimal collection of RTTs would be the one which satisfies the following criteria: (1) it consists of RTTs which describe the non-assertional part of the POR only to the extent to which there is enough evidence for what those RTTs themselves assert to be true (e.g. there is sufficient evidence that the patients are human beings, there is not sufficient evidence that the diagnoses are correct) and (2) it consists of other RTTs which describe the assertional part in relation to the RTTs referenced under (1).
We note here that the level of disagreement in the representations of X and Y do not invalidate the RT method, but rather reflect the need for uniform conventions on which ontologies and relations to use, as well as problems in the ontological theories, their implementations, and documentation that were available to represent the scenario. We return to this issue as we discuss the major sources of disagreement. Indeed, as will become clear, this work shows that the RT method is a stringent test of ontologies.
Although both authors agreed on the necessary existence of the patient (P1) and the two clinicians (P2, P3) for the analyzed scenario to be faithful to reality, they each selected distinct universals to assert instantiation. X represented P1 as a human with a patient role. Y represented P1 as a material entity without assigning a patient role, his choice of material entity being motivated by the fact that P1 has been a material entity all the time through its existence, but not a human (e.g., it was a zygote at a time prior to being human). This difference in representation is related to the temporal indexing that RT requires for continuants, an element not further discussed in this paper. But given the two authors’ temporal indexing, both authors agree that each other’s instantiations were correct.
Both authors disagreed though about how to interpret the representational units for the universal Human being from the selected ontologies. Y used ‘human being’ as synonym for the FMA’s ‘human body’ class, although FMA does not list synonyms. Y argued against X’s ‘Homo sapiens’ taken from OBI based on its linking to other ontologies in Ontobee, which altogether seem to confuse ‘Homo sapiens’ as an instance of the universal ‘species’ with those instances of organism that belong to – but are not instances of – the species ‘Homo sapiens’. X counters that despite the use of species names, ‘Homo sapiens’ and similar classes in OBI all descend from a class called ‘organism’. Also, the ‘Homo sapiens’ class in OBI has synonyms ‘Human being’ and ‘human’. It would be an enormous task indeed to find nontaxonomic names for every type of organism in the world and refactor ontologies based on the NCBI Taxonomy on this basis. The problem here is the lack of face value of terms selected as class names in the respective ontologies.
Both authors agreed on the existence of a disorder and a disease resulting from it in the patient, as well as two diagnoses and the two distinct processes that generated each. Both authors also agree that none of these entities should be confused or conflated: nothing at the same time can be an instance of two or more of the following: disease, disorder, diagnosis, and diagnostic process.
A problem is that the Disease Ontology selected by X, confuses not only disorders and disease, but also disease courses. For instance ‘physical disorder’ in DO is a subtype of ‘disease’, in direct contradiction to OGMS. X agrees that DO is flawed, however it is the only ontology of disease that at least purports to strive for compliance with realist principles, and represents an improvement over flawed medical terminologies such as SNOMED-CT and NCI Thesaurus. If perfection were a requirement to use an ontology, we could make no progress. Nevertheless, the persistent, glaring flaws of DO from the perspective of OGMS give serious pause on using it accurately and precisely.
Both authors offered a different perspective on what parts of Table 1 actually constitute a diagnosis. They agreed that any such table—whether presented as a problem list on a video monitor or tablet as the scenario worked with by X, or as a spreadsheet drawn with chalk on a blackboard as envisioned by Y—is built out of continuants that are concretizations of instances of ICE reflecting a diagnosis. But whereas X identified the mere concretization of the ICD-code and label to be denoting the diagnosis, Y argued that also the concretization of the patient identifier is part of that which concretizes the diagnosis (a requirement for the diagnosis to stand in an ‘is about’ relation to the patient). This, and a large amount of other differences in representation, were due to distinct interpretations of the literature on the nittygritty of how to deal with ICE and concretizations thereof, how instances of ICE relate to other instances of ICE, and what exactly the relata are of relationships such as aboutness and denotation. Whereas, for instance, X committed to ICE being parts of other ICE, Y commits only to parthood of the independent continuants in which inhere the qualities that concretize the corresponding ICE, without making that clear in the representation however. Another key issue with ICE is that Y represented the qualities concretizing the ICE as being about something, whereas X followed the IAO where the ICE itself denotes or is-about something. After further analysis both authors agreed that Y’s representation is better and that it advances the theory of ICE in IAO.
Although it was a priori agreed upon that the patient in the
scenario would have only one disorder, an ambiguity that
was left open was whether both diagnoses were actually
correct: so Table 1 could be interpreted in distinct ways: (1)
both diagnoses are correct from a medical perspective and
describe distinct aspects of the same disease, or (2) at least
one diagnosis is wrong. Also, because RT tuples contain
provenance as to whom is the source of the statement
contained therein – note that Ex.1 above is a simplified
representation not containing the provenance information – X
interpreted both (1) the RT tuple that instantiated the disease
as gout (by Doe) and (2) the RT tuple that instantiated it as
osteoarthritis (by Thump) as being faithful representations
of what Thump and Doe believed at the time they
formulated their diagnoses. X did not believe himself to be
recognizing both diagnoses as straightforwardly accurate and
therefore resorted in his representation to a mechanism
offered in RT to craft RTTs about RTTs that are later found to
have been based on a misunderstanding of the reality at the
time they were crafted
Although both authors resorted to OGMS for a large part
of their RTTs, differences in representation were observed
because of the source material consulted: X used the OGMS
OWL artifact as basis, whereas Y used the definitions and
descriptions in the paper that led to the development of
OGMS
In representing a common scenario in healthcare which EHR data are about, the two authors agreed on key entities including the patient, doctors, diagnoses, and the processes by which the doctors generated the diagnosis. Although they agreed in general about the types instantiated by the particulars in the scenario, and how the particulars are related to each other, they chose different representational units and relations from different ontologies due to various issues such as potential lack of orthogonality in the OBO Foundry and in some cases disagreement on what types the classes in the ontologies represent. These distinctions exist, not because the authors entertained distinct competing conceptualizations, but because they expressed matters differently.
Differences in the choice of ontologies constitute a risk: distinct ontologies may represent reality from distinct perspectives and despite being veridical might not be derivable from each other because the axioms required to do so would be missing, for the simple reason that such axioms would fall outside the purpose of the specific ontologies. This would lead the representations by each of the authors not to be semantically interoperable unless additional ontology bridging axioms would be crafted. The authors’ main source of disagreement was due to different interpretations of the literature on ICEs, which ultimately led to a planned reformulation of the theory of ICE and reference. Although this study is limited by the participation of only 2 subjects and the analysis of one report, it highlights the fact that the RT method and the clarity and precision it requires in representing reality is a powerful tool in identifying areas of needed improvement in existing, realism-based ontologies.
This work was supported by award UL1 TR000064 from the National Center for Advancing Translational Sciences. This paper is solely the responsibility of the authors and does not necessarily represent the official views of the NIH.