We present a formalization of indicators of diagnostic performance (sensitivity, specificity, positive predictive value and negative predic-tive value) in the context of a realist ontology. We dissociate the indica-tors of diagnostic performance from their estimations and argue that the former should be represented in a first place in biomedical ontolo-gies. Our formalization does not require to introduce any possible, non-‐actual entities -‐ like the result a person would get if a medical test would be performed on her -‐ and is therefore acceptable in an ontology built in a realist spirit. We formalize an indicator of diagnostic perfor-mance as a data item that is about a disposition borne by a group; the diagnostic value of this indicator is given by the objective probability value assigned to this disposition.
1.1
Biomedical ontologies aim at providing the most exhaustive and rigorous representation of reality as described by biomedical sciences. A large part of medical reasoning concerns diagnosis and is essentially probabilistic. It would be an asset for biomedical ontologies to be able to support such a probabilistic reasoning.
However, such positive and negative predictive values are typically not available in the scientific literature. Instead, they are generally computed from other probabilistic values, namely: the prevalence value of M in g, generally defined as the proportion of people who have the disease M in g, and hereafter abbreviated Prev(g,M); the sensitivity value of the test A for M in g, generally defined as the proportion of people who would get a positive result by A among those who have the disease M in g (that is, the proportion of true positives among diseased), hereafter abbreviated Se(g,A,M); and the specificity value of A for M, generally defined as the proportion of people who would get a negative result by A among those who do not have the disease M in g (that is, the proportion of true negatives among non-diseased), hereafter abbreviated Sp(g,A,M). As a matter of fact, these values are related through the following Bayesian equations: PPV(g, A, M) = NPV(g, A, M) =
Prev(g, M) Se(g, A, M) Prev(g, M) Se(g, A, M) + (1- Prev(g, M)) (1- Sp(g, A, M))
(1- Prev(g,M)) Sp(g,A, M)
Prev(g,M) (1- Se(g,A, M)) + (1- Prev(g,M)) Sp(g,A, M)
In the wake of
In the remainder of the articles, sensitivity, specificity, PPV and NPV will be called “indicators of diagnostic performance” and abbreviated “IDPs”. 1.2
To the extent that they aim at representing biomedical
knowledge and enabling medical reasoning, biomedical
ontologies should provide a formalization of IDPs as well as
the prevalence. This article will propose such a
formalization in the context of the OBO Foundry
The question of how probabilistic notions can be
represented in ontologies has been tackled from different
perspectives in the past. For example, da
Sensitivity and specificity have been recently introduced
in the Ontology of Biological and Clinical Statistics
Sensitivity value2, as we said, is generally defined as the proportion of people who would get a positive result by A among those who have the disease M. But note here the conditional structure: what is referred to is the proportion of true positives among diseased if A was performed on them. In practical situations, however, the sensitivity value will be estimated by performing the test on a sample of the population only – not the entire population g. This will lead to two difficulties. First, it will be necessary to differentiate clearly the IDPs’ values from their estimations, and to determine which of those should be represented in a first place in an ontology – part 2 will be devoted to this issue. Second, possible-but-non-actual situations cannot be straightforwardly defined in a realist ontology like BFO; this problem will be explained and solved in part 3, by considering that an IDP is a data item about a disposition borne by an instance of group of individuals, whose probability value will be identified to the diagnostic value of the IDP. This will provide a formal characterization of IDPs. 2 2.1
Numerical estimations of IDPs face two limits
For example,
Note that the estimations of the values of the prevalence, sensitivity, specificity, PPV and NPV depend on both the sample and the gold standard; however, the real values of the prevalence, sensitivity, specificity, PPV and NPV, as defined above, depend neither on the sample, nor on the gold standard. 2.2
This being clarified, one can ask which entities should be preferably represented in an ontology: the IDPs’ values, or their estimations?
For sure, we have no direct access to such IDPs’ values; but this does not imply that they should not be represented in an ontology. To clarify why, consider an analogy: the measure of the ambient temperature by reading the height of a mercury column in a thermometer. Suppose that at a given time, this height is aligned with the sign “20 °C” written on the thermometer. In such a case, an ontology curator would be in a first place interested in formalizing the fact that the ambient temperature is 20°C, rather than in formalizing the fact that the mercury column in the thermometer is at the same height as the sign “20°C”.
In a similar fashion, imagine that 65% of people are
tested positive for a gold standard of M in a sample s of a
population g. The ontology should then formalize in a first place
the fact that 65% of the people in g have M, rather than the
3 Even if the gold standard consists in the naked-eye observation of a
macroscopic disorder associated exclusively with this disease, this can still
theoretically lead to a diagnostic error: any empirical evidence is
defeasible.
fact that 65% of the people in s have a positive result to this
gold standard. This estimation of this prevalence value may
be false (it is indeed very likely to be false, strictly
speaking), but future estimations will lead to its being corrected to
bring it closer to the real value. As a matter of fact, realist
ontologies are built according to a fallibilist methodology
That being said, it is possible to represent in an ontology
the measurement process of a temperature involving the
height of a mercury column in a thermometer. Similarly,
one could represent the different estimation processes of the
IDPs, and the results to which they led. Such processes are
biomedical investigations, and should therefore be
formalized in an ontology like OBI
This clarification being made, we can now consider the second difficulty mentioned at the end of part 1, namely the formalization of possible-but-non-actual situations in BFO. 3
Sensitivity value has been interpreted as the proportion of people who would get a positive result to A among M’s bearers in g. This definition thus involves the condition of performing the test A on the members of g. As we said, such a condition is never realized, because the test is performed (at best) on a sample of the population, not on the whole population g: the performance of test A on g’s members is a possible (leaving aside practical difficulties), non-actual condition. Interpreting specificity, PPV, and NPV along the former lines would also imply such possible, non-actual conditions.
However, BFO is built according to the realist
methodology, which implies that all the instances it recognizes should
be actual entities
We will explain first how the proportion of a subgroup in a group can be formalized as a probability value assigned to a disposition; this will help explaining later how the proportion of a subgroup in a group undergoing a possible, nonactual condition can be formalized along similar lines.
Dispositions are entities that can exist without being
manifested; an example of disposition is the fragility of a
glass, which can exist even when the glass does not break.
We will use
We will now apply this model to the situation at hand.
Consider the prevalence Prev(g,M), which was defined
above as the proportion of bearers of M in the actual
population g. We can define the disposition dg,M, borne by the
group g, that a person randomly drawn in g has M. More
specifically, let’s write Tg the process “randomly drawing a
person in g”, and Rg,M the process “drawing by Tg someone
who has M”: the triggers of dg,M are instances of Tg and its
realizations are instances of Rg,M. Following the lines of
The randomization strategy may not be necessary to formalize a prevalence, which characterizes a proportion in an actual group, and thus could be formalized as such in an ontology based on BFO. But this strategy can also be applied to proportions of people in groups subject to a possible, non-actual condition – and thus, be relevant to formalize sensitivity and other IDPs. As a matter of fact, the sensitivity value Se(g,A,M) was defined as the proportion of people who would get a positive result to A among M’s bearers in g. This value can be “randomized” as follows. We can define dSe,g,A,M as the disposition to draw someone randomly who is tested positive by A, among the individuals of g who have M. More specifically, let’s define the process TSe,g,A,M as the “performance of test A on the individuals in g, and random draw of an individual among those who have the disease M”5; and the process RSe,g,A,M as the “drawing by TSe,g,A,M of someone who got a positive result to A”. The triggers of dSe,g,A,M are instances of TSe,g,A,M, and its realizations are instances of RSe,g,A,M . One can then define the sensitivity value Se(g,A,M) as the objective probability assigned to this disposition dSe,g,A,M,: indeed, if there are e.g. 15% of the diseased people in g who would get a positive result by A, then the probability of randomly drawing someone who would get a positive test result by A among diseased people in g is equal to 15%.
Specificity value can be defined along similar lines, as probabilities assigned to actual dispositions borne by the group g noted dSp,g,A,M (and similarly for the PPV and NPV). Although dSe,g,A,M and dSp,g,A,M are both dispositions inhering in g, they have different triggers and different realizations; the process TSp,g,A,M is the “performance of test A on the individuals in g, and random draw of an individual among those who do not have the disease M” and the process RSp,g,A,M is the “drawing by TSp,g,A,M of someone who got a negative result to A”. 3.2
Let us now consider how to formalize these probability values in ontologies. First, a group g will be considered as any collection of humans (for more on collections, see Jansen & Schultz, 2010). dSe,g,A,M is a disposition individual inhering in the group g; and a probability value can be assigned to this disposition using a datatype property has_probability_value. Sensitivity of A for M in g will be denoted Seg,A,M, and following OBCS, it will be defined as a data item. Thanks to our analysis above, we can now answer our original question, and state what this sensitivity is about: Seg,A,M is_about dSe,g,A,M. We can also introduce a relation has_diagnostic_value that relates a sensitivity to its value. 5 In general, we cannot determine in practice with certainty which individuals of g have M, and which do not; but the practical impossibility to realize this trigger does not preclude to define this entity.
In our framework, the (diagnostic) value of a sensitivity Seg,A,M is the probability value assigned to the disposition dSe,g,A,M; this can be formalized by writing that if s is a sensitivity, then:
s has_diagnostic_value p ⇔ ∃ d ∧ d is_a Disposition ∧ s is_about d ∧ d has_probability_value p
As dSe,g,A,M is an individual, it cannot be related directly to
the universals A and M. However, it can be related indirectly
to them, by the following formalization. First, dSe,g,A,M can
be seen as an instance of a disposition universal symbolized
as DSe,A,M, which has as trigger the processus universal
TSe,A,M: “performance of test A on the members of a group,
and random draw of a person among those who have the
disease M”; and as realization the process universal
RSe,A,M defined as “drawing by TSe,A,M of someone who got a
positive result to A”. We can then introduce two new
relations sensitivity_disposition_of_test and
sensitivity_disposition_for_disease (abreviated as se_of_test and
se_for_disease) such that DSe,A,M se_of_test A and
DSe,A,M se_for_disease M. These two relations are introduced
for pragmatic reasons of facility of use: on a foundational
level, DSe,A,M and M (resp. A) could be related through a
complex array of relations and entities that involve the
relation has_trigger between DSe,A,M and TSe,A,M, as well as a
sequence of relations between TSe,A,M and M (resp. A). Such
an analysis would raise theoretical issues though, as
instances of DSe,A,M can exist even if no instance of M or A do exist.
We would therefore face here issues similar to the ones
addressed by Röhl &
Finally, we introduce a class Sensitivity that can be characterized as a subclass of Data item, which is related to a disposition through the above-mentioned relations: s instance_of Sensitivity ⇒ s instance_of Data item ∧ ∃ d instance_of Disposition ∧ ∃ a instance_of Test ∧ ∃ m instance_of Disease ∧ s is_about d ∧ d se_of_test a ∧ d se_for_disease m
We can also introduce SeA,M, the class of sensitivities of test A for disease M (in whatever group), which can be formalized as a subclass of Sensitivity related to M and A through the following relations:
s instance_of SeA,M ⇒ s instance_of Sensitivity ∧ ∃ d instance_of Disposition ∧ ∃ a instance_of A ∧ ∃ m instance_of M ∧ s is_about d ∧ d se_of_test a ∧ d se_for_disease m
Figure 1 summarizes this formalization of sensitivity (with universals in boxes, instances in diamonds, and the numerical value assigned by datatype properties in a circle). Specificity, PPV and NPV can be formalized along similar lines, as data items about dispositions related to tests and diseases through relations that could be labeled sp_of_test, sp_of_disease, ppv_of_test, etc.
We have thus provided a practically tractable formalization of IDPs in a realist ontology, which clearly dissociates IDPs from their estimations (which are relative to a sample and a gold standard). It also solves the difficulty of considering possible, non-actual conditions in a realist ontology based on BFO.
Note that IDPs raise also other theoretical issues. For example, one may want to aggregate two sensitivity values Se(g,A,M) and Se(g’,A,M) assigned to two different groups g and g’ in order to reach a finer assessment of the sensitivity in a larger group; how to do this is a question for the meta-analyst though, not the ontologist, who is first and foremost concerned with representational issues.
This model could then be extended in three directions. A first step would consist in formalizing the estimations of the IDPs, and their relations to a given sample and gold standard. Second, the relations se_of_test and se_for_disease could be reduced to basic relations and entities already accepted in the OBO Foundry. Third, it could be used by ontology-based diagnostic systems that would compute positive predictive values or negative predictive values from the prevalence, sensitivity and specificity values; more generally, it could be articulated with medical Bayesian networks.
As it takes into account the dependence of IDPs upon the group of people considered, it has the potential to contribute to the development of precision medicine (Mirnezami, Nicholson & Darzi, 2012), an emerging approach that takes into consideration patients characteristics and dispositions, including individual variability in genes, to offer more personalized preventive, diagnostic and therapeutic strategies.
We would like to thank the audience at several seminars, as well as four anonymous reviewers, for their helpful comments. Adrien Barton thanks the Japanese Society for Promotion of Science for financial support. Figure 1 Sensitivity of a test A for a disease M in a group g with probability value 0.75