=Paper=
{{Paper
|id=Vol-1327/14
|storemode=property
|title=An Ontological Representation and Analysis of Patient-reported and Clinical Outcomes for Multiple Sclerosis
|pdfUrl=https://ceur-ws.org/Vol-1327/icbo2014_paper_44.pdf
|volume=Vol-1327
|dblpUrl=https://dblp.org/rec/conf/icbo/JensenCRTWRD14
}}
==An Ontological Representation and Analysis of Patient-reported and Clinical Outcomes for Multiple Sclerosis==
https://ceur-ws.org/Vol-1327/icbo2014_paper_44.pdf
ICBO 2014 Proceedings
An ontological representation and analysis of patient-
reported and clinical outcomes for multiple sclerosis
Mark Jensen*, Alexander P. Cox, Patrick L. Ray, Barbara E. Teter, Bianca-Weinstock Guttman,
Alan Ruttenberg, Alexander D. Diehl
The State University of New York at Buffalo
Buffalo, NY, USA
*
mpjensen@buffalo.edu
Abstract— We have developed the Multiple Sclerosis Patient visual and cognitive deficiencies. A hallmark of MS is a
Data Ontology (MSPD) to represent data from the patient data heterogeneous disease course characterized by varying patterns
registry of the New York State Multiple Sclerosis Consortium of exacerbations in neurological impairment. Disability in MS
(NYSMSC). MSPD is an application ontology that provides a set is assessed using the Kurtzke Expanded Disability Status Scale
of classes for the annotation of both clinical measures and patient (EDSS) [3]. In recent years a variety of new treatments have
reported outcome data obtained from the enrollment forms used improved outcomes for many MS patients, yet the disease is
by the NYSMSC. To do so, we have adopted the paradigm considered incurable and progressive in its course.
established for representing assays in the Ontology for
Biomedical Investigations. Our goal is to compare patient
reported outcomes, such as self-reported disability and quality of B. New York State Multiple Sclerosis Consortium
life perceptions, to objective outcome measures in clinical The New York State Multiple Sclerosis Consortium
practice, with reference to diagnoses and treatment modalities. (NYSMSC) is an alliance of treatment centers organized to
We have begun an ontology-driven retrospective analysis of the prospectively assess clinical attributes of MS patients [4]. The
patient records in the NYSMSC registry using an ontology term NYSMSC patient registry includes data from more than 15 MS
enrichment method in order to spot significant patterns in centers across New York State and is the largest clinical-based
patient-reported and clinical outcomes in subsets of patients in cohort of MS patients in the United States with over 10,000
the NYSMSC patient registry as compared to the NYSMSC registrants and 17,000 follow-up visits. It uses standardized
patient population as a whole. data collection forms addressing demographic and clinical
information, with an annual follow-up providing routine
Keywords—multiple sclerosis; neurological disease ontology;
tracking of disease progression. The LIFEware system is used
patient reported outcomes; OBI
to record patients’ perceptions of their physical and
psychosocial impairment as a way of capturing patient reported
I. INTRODUCTION data related to quality of life and wellbeing [5]. Clinical
We have developed the Multiple Sclerosis Patient Data information collected includes: disease status, number of
Ontology (MSPD) * to represent data from the patient data exacerbations, current therapies, EDSS scores, and imaging
registry for the New York State Multiple Sclerosis Consortium data. The data have been used for studies on the evolution of
(NYSMSC). MSPD is an application ontology that provides a benign MS and of correlations between fatigue and depression
set of classes for the annotation of both clinical measures and in patients with MS [6, 7].
patient reported outcome data obtained from the enrollment
forms used by the NYSMSC. Our goal is to compare patient C. Patient Reported Outcomes
reported outcomes, such as self-reported disability and quality A patient reported outcome (PRO) is generally considered
of life perceptions, to objective outcome measures in clinical to be an assessment of any aspect of a patient's health status
practice, with reference to diagnoses and treatment modalities. that comes directly from the patient without interpretation by a
clinician [8]. PROs are a valuable tool in assessing patients’
A. Multiple Sclerosis perceptions about their health and wellbeing, along with other
clinical metrics, such as efficacy of treatment, disease
Multiple sclerosis (MS) is an autoimmune demyelinating
progression, etc. [9]. Instruments for obtaining PRO provide a
disease of the central nervous system (CNS) affecting over 2
means for measuring treatment benefits by capturing
million people worldwide [1]. MS pathology results in the
information about patients’ perceptions of both their current
formation of sclerotic plaques that appear in multiple regions
disability and overall health. In order to better understand the
over time throughout the CNS and are associated with a wide
relationship between patient reported outcomes and clinical
range of neurological symptoms [2]. MS presents clinically
measurements, along with treatments for multiple sclerosis, it
through varied neurological impairments such as loss of motor
is important to see how practitioners’ assessments track with
control and balance, weakness, sensory disturbances, and
patients’ perceptions of their wellbeing [10].
*
https://neurological-disease-ontology.googlecode.com/svn/trunk/MSPD.owl
52
ICBO 2014 Proceedings
D. Ontology for Biomedical Investigations is the evaluant, by physically examining it or its proxies” [11].
MSPD extends the Ontology for Biomedical Investigations All assays specify an output, an information content entity,
(OBI), which is an integrated ontology for the description of which is about the evaluant. In our case, the evaluant is a MS
biological and clinical investigations [11]. OBI is a domain patient that is also an enrollee in the NYSMSC. More
ontology that provides a set of terms and relations to support precisely, the evaluant is a Homo sapien that bears an enrollee
precise annotation and querying of the kinds of data generated role. For simplicity, we defined a constructed class labeled
in biomedical investigations. It represents the design, types of ‘NYSMSC enrollee’ that is equivalent to: ‘Homo sapien’ and
analyses and assays performed, and specifications, resulting in is bearer of some ‘NYSMSC enrollee role’. The role is
classes such as ‘assay’, ‘plan specification’, and ‘measurement realized during the enrollment process
datum’. Two hierarchical distinctions emerged in generalizing the
types of assays present in the enrollment process for the
II. METHODS
NYSMSC data registry. One relates to distinguishing between
MSPD is an OWL2 ontology built using Protégé 4 and is who does the evaluating, either clinician or patient. We created
being developed according to OBO Foundry principles [12]. two upper level classes: ‘clinician reported assay’ and ‘patient
MSPD directly imports all of OBI, and along with it the Basic reported assay’. A ‘patient reported assay’ is “an OBI assay
Formal Ontology [13]. We import select classes from such wherein a patient produces information about themselves as the
ontologies as the Gene Ontology (GO) and Functional Model evaluant” and a ‘clinician reported assay’ is “an OBI assay
of Anatomy (FMA) via OntoFox according to MIREOT wherein a clinician produces information about a patient as the
standards [14, 15]. MSPD is a corollary project to the evaluant”. We further distinguish assays by what the
Neurological Disease Ontology (ND) [16]. information they produce is about. Assays are distinguished by
producing data about functional impairments, such as
De-identified patient data from the NYSMSC patient
perceived limitation with a limb, and information about
registry were annotated with a reasoned version of MSPD.
affective judgments, such as being bothered by depression or
Data was handled in a HIPAA-compliant fashion per our IRB
pessimistic thoughts, as well as assays that produce externally
approval. Ontology terms were assigned to patients’ ratings of
verifiable facts such as date of birth and marital status.
their perceived disabilities and current affective state based
upon thresholds used to determine whether responses to The second hierarchy involves parthood distinctions based
particular Lifeware questions merited annotation. For the on the structure and composition of the enrollment forms. The
results presented herein, the thresholds were set to annotate ‘NYSMSC enrollment form assay’ represents the overall
fairly stringently, in most cases at the second highest score (on encompassing process of completing all portions of the
scales of 1-4, 1-5, or 1-7), such that only stronger statements of enrollment form. It has two subparts, ‘NYSMSC clinician
disability or negative affective ratings resulted in annotation. reported enrollment assay and ‘NYSMSC patient reported
Following annotation, subsets of patients were compared enrollment assay’. These in turn have multiple subparts that
according to gender to the population of patients as a whole correspond to the numbered questions on the form, such as
with determination of p-values based on the hypergeometric ‘timed ambulation assay’ and ‘limitation assay’.
distribution in a way similar to that developed for term
enrichment analysis for the GO [17]. The hypergeometric Fig. 1 illustrates some of this structure in the ontology. As a
distribution was performed utilizing code taken from result of making these two general distinctions amongst assay
http://www.perlmonks.org/bare/?node_id=856875. Perl scripts types, MSPD contains an asserted subclass hierarchy of general
were written to perform both the annotation and term types of assays defined by what is being assayed, which are
enrichment portions of the analysis using MSPD. connected through parthood relations to assays that represent
the enrollment forms themselves. This way of building MSPD
gives us a clear separation between types of assays based
III. ONTOLOGY STRUCTURE
who’s producing the information and what that information is
A variety of processes are parts of the NYSMSC about, versus the assay’s place in the structure of the
enrollment process. One subprocess involves the clinician enrollment form, and subsequently, the enrollment process
preforming a comprehensive neurological examination, itself. Not only do we believe this to be more ontologically
elements of which can be seen as assays of the patient’s precise, but it allows for more robust reasoning capability.
neurological functioning. Another part of the enrollment
process includes the patient evaluating aspects of their own The definition for OBI assay requires the specified output
neurological and motor functioning, rating physical limitations, be about a material entity. In this case that entity is a patient.
and perceived disease progression. Along with these measures, But, when considering how to relate the information each assay
patients are asked to indicate overall life satisfaction and to produces to what aspect of the patient is being assessed, we
what extent they are bothered by certain moods and feelings. needed to specify more than existence of the patient. A patient
These self-assessments qualify as PRO and correlate to bears certain qualities and functions, such as his or her visual
standard quality of life metrics. To represent theses aspects of or cerebellar function, which ultimately are the entities of
the enrollment process we utilized the paradigm established by interest in these assays. These functions are realized during the
OBI for representing assays. assay process (when successful) as the patient is being
evaluated. It is these realizations (functionings) that can be
An OBI ‘assay’ is defined as “a planned process with the observed, measured, quantified in some cases, and used in
objective to produce information about the material entity that making judgments about impairment. Thus, to relate the datum
53
ICBO 2014 Proceedings
that each assay produces to the aspect of a patient’s functioning 1<606&SDWLHQWUHSRUWHG SDWLHQWUHSRUWHG FOLQLFDQUHSRUWHG 1<606&FOLQLFLDQUHSRUWHG
being evaluated, we utilized the following guideline for HQUROOPHQWDVVD\ DVVD\ DVVD\ HQUROOPHQWDVVD\
creating instance-level relations in MSPD assay classes: LVSDUWRI LVD LVD LVSDUWRI
OLPLWDWLRQ MXGJHPHQWDERXW UDWLQJRI ('66IXQFWLRQDO
DVVD\ IXQFWLRQDVVD\ IXQFWLRQDVVD\ VFRUHDVVD\
‘OBI assay’ has specified output some LVD LVD
LVSDUWRI
(‘OBI measurement datum’ and (is about some 1<606&
HQUROOHHUROH
LVSDUWRI
(‘NYSMSC enrollee’ and YLVLRQ
OLPLWDWLRQ
LVEHDUHURI
VHQVRU\
IXQFWLRQ
is bearer of some ‘BFO function’))) DVVD\ DVVD\
KDVRXWSXW KDVRXWSXW
1<606&
‘OBI assay’ realizes some (‘BFO function’ PHDVXUHPHQW LVDERXW HQUROOHH LVDERXW PHDVXUHPHQW
and inheres in some ‘NYSMSC enrollee’) GDWDLWHP GDWDLWHP
LVEHDUHURI LVEHDUHURI
UHDOL]HV
LVVXSSRUWHG
LVVXSSRUWHG
For example, the ‘vision limitation assay’ refers to the part E\GDWD YLVXDO
IXQFWLRQ
SURSULRFHSWLYH
IXQFWLRQ
E\GDWD
of the enrollment process wherein an enrollee is asked to rate LVDERXW LPSDLUHGSURSLRFHSWLYH
IXQFWLRQLQJFRQFOXVLRQ
how limited their vision is on a scale of 1 (“no limitation”) to 7 OLPLWHGYLVLRQ
FRQFOXVLRQ
UHDOL]HV UHDOL]HV
LVDERXW
(“severe”). We take the output of the assay to represent a SURSULRFHSWLYH LQIHUUHG
YLVXDOSHUFHSWLRQ
judgment the enrollee makes about their visual functioning. We LQIHUUHG
LVD
IXQFWLRQLQJ LVD
assert that every output datum from this assay is about the LVD LVD
enrollee who is the bearer of an instance of a ‘visual function’. FRQFOXVLRQIURP LVD VHQVRU\
SURFHVV
LVD FRQFOXVLRQIURP
FOLQLFLDQUHSRUW
SDWLHQWUHSRUW
Likewise, the clinical reported component of the visual LPSDLUHGVHQVRU\
score in the EDSS assay, the ‘EDSS visual function assay’ IXQFWLRQLQJFRQFOXVLRQ
produces a datum about the enrollee and the enrollee’s visual
functioning. But, it is important to connect the functions, which
Fig. 1. A subset of classes and relations in MSPD. Except
are borne by the enrollee, to instances of their realizations in
for “is a” all relations are between instances of classes.
that particular assay since the clinician is measuring these
realizations. We import the GO classes for various neurological
and sensory processes, thus enriching the ontological assay that evaluates a patient’s limb functioning produces data
representation by connecting these assays to the apparatus that that can be used to support various conclusions about limitation
GO provides for annotation to genes and molecular functions. with that particular limb. The assay itself and the datum it
produces are neutral with respect to whether limitation actually
‘MSPD EDSS visual function assay’ realizes some exists. But, a clinician or researcher can interpret the result and
(‘MSPD visual function’ and inheres in some then decide if it supports such a conclusion.
‘NYSMSC enrollee’)
IV. DATA ANALYSIS
Fig. 1 illustrates some of the relations in MSPD. We performed an analysis of 9331 patient records from the
A final structural component of the ontology relates to data NYSMSC data registry, selecting subsets of patients based on
analysis tasks. To give our clinical collaborators the ability to gender, 6916 female and 2389 male. All data points for
select and annotate subsets of patients based on varying and patients in each subgroup were annotated with the ‘conclusion
unique criteria, we developed slasses that extended ‘OBI based on data’ subclasses corresponding to the assays for both
conclusion based on data’, defined as “an information content PRO and certain clinical measures, such as EDSS scores. We
entity that is inferred from data” [11]. Such conclusions are determined which data were annotated by setting a unique
linked to their data by the OBI relation ‘is supported by data’. threshold for each particular assay output. To eliminate patient
Through this, the ontology enables user-specific instance level records where minimal or no disability was present, we set the
assertions about how certain data items support particular threshold high. Term enrichment was performed on each
conclusions about disability status, quality of life metrics, and annotated subset using the hypergeometric distribution method
so on. For example, a conclusion that a patient has limited established for the GO and used successfully for term
function in their right lower limb may be inferred based upon a enrichment studies based on disease ontologies [17,18].
score of 2 or higher (essentially any indication of limitation) in As Table 1 shows, terms related to patient reported
the assay wherein limitation in that limb is evaluated. limitations in limbs were associated with highly significant p-
Alternately, only the highest score of 4 (maximal limitation) values for over- or under-enrichment in the results for the two
could warrant such a conclusion in a different context. cohorts. Interestingly ontology terms annotated to the male
cohort were significantly over-enriched while those for the
‘MSPD limitation in right lower limb conclusion’ female cohort were significantly under-enriched in all terms in
is supported by data some the ontology that related to perceived limitations in limbs. This
(‘OBI data item’ and is specified output of finding suggests that the male MS population experience or
some ‘MSPD right lower limb limitations assay’) report limitations in limbs at a higher rate than female MS
patients do, or alternately, that female patients under report
An advantage of this design is that different instances of such limitations.
these conclusions can be created using the same datum. An
54
ICBO 2014 Proceedings
TABLE 1A: Male Cohort The preliminary results presented herein are fairly striking,
P-value label and we will work with our clinical collaborators to develop
1.02E-13 over-represented limitation with limb conclusion
interesting questions to answer via annotation of the NYSMSC
6.32E-13 over-represented limited lower limb function conclusion
1.11E-10 over-represented limitation in right limb conclusion patient data and application of the term enrichment
2.71E-10 over-represented limitation in right lower limb conclusion methodology. We recognize that there are a myriad of ways to
5.58E-10 over-represented limited left limb function conclusion select and group subsets of patient records. We are particularly
1.09E-09 over-represented limited left lower limb function conclusion interested in comparing patient cohorts treated with particular
1.81E-07 over-represented limitation in upper limb conclusion
2.28E-06 over-represented limited left upper limb function conclusion
drugs versus cohorts treated in other ways. Through this work
2.67E-03 over-represented impaired pyramidal tract functioning conclusion
we hope to gain insight into the efficacy of particular treatment
! regimens as measured via both patient reported and clinical
TABLE 1B: Female Cohort outcomes.
P-value label
5.44E-14 under-represented limitation with limb conclusion ACKNOWLEDGMENTS
4.34E-13 under-represented limited lower limb function conclusion
4.58E-11 under-represented limitation in right limb conclusion This work was supported by a National Multiple Society
1.49E-10 under-represented limitation in right lower limb conclusion Pilot Project Grant, PP1970, and by the State University of
5.02E-10 under-represented limited left limb function conclusion New York at Buffalo.
9.01E-10 under-represented limited left lower limb function conclusion
3.15E-07 under-represented limitation in upper limb conclusion REFERENCES
4.44E-06 under-represented limited left upper limb function conclusion
2.25E-03 under-represented impaired pyramidal tract functioning conclusion
! [1] NMSS. National Multiple Sclerosis Society. 2013; Available from:
Table 1. The first 8 lines in each table show p-values for terms related to http://www.nationalmssociety.org.
patient reported limitations in limbs. The last line shows the p-value for [2] Polman, C.H., et al., Diagnostic criteria for multiple sclerosis: 2010
the clinical measure associated with limb limitation: the pyramidal revisions to the McDonald criteria. Ann Neurol, 2011. 69(2)
function score component of the EDSS. [3] Kurtzke, J.F., Rating neurologic impairment in multiple sclerosis: an
expanded disability status scale (EDSS). Neurology, 1983. 33(11).
Also of interest, the conclusion based on data from the [4] Jacobs, L.D., et al., A profile of multiple sclerosis: the New York State
clinician reported component that most closely relates to limb Multiple Sclerosis Consortium. Mult Scler, 1999. 5(5): p. 369-76.
functioning, the pyramidal tract functional score of the EDSS [5] Baker, J.G., et al., A brief outpatient functional assessment measure:
validity using Rasch measures. Am J Phys Med Rehabil, 1997. 76(1).
assay, showed only relatively minimal significance between
[6] Krupp, L.B., et al. Longitudinal correlates of fatigue in a sample of 2753
each gender-based subgroup and the general population. This persons with multiple sclerosis. in Neurology. 2005. Lippincott
suggests that while male patients reported more limitation in Williams & Wilkins, 530 Walnut St, Philadelphia PA 19106, USA.
limb functioning than female ones, the objective clinical [7] Zivadinov, R., et al. Evolution of benign multiple sclerosis in the New
measure of disability in MS patients did not correspond to York state multiple sclerosis consortium according to different
these reports as strongly. That is to say, terms were only classification criteria. in Multiple Sclerosis Journal. 2006. Sage
minimally enriched for the clinician reported measures, Publications Ltd, 1 Olivers Yard, 55 City Road, London, England.
whereas they were highly enriched for the patient reported [8] Deshpande, P.R., et al., Patient-reported outcomes: A new era in clinical
measures that related to limb functioning. research. Perspect Clin Res, 2011. 2(4): p. 137-44.
[9] Health, U.S.D.o., et al., Guidance for industry: patient-reported outcome
measures: use in medical product development to support labeling
V. CONCLUSIONS AND FUTURE WORK claims: draft guidance. Health Qual Life Outcomes, 2006. 4: p. 79.
Thus far we have developed an ontology for representing [10] Benedict, R.H., et al., Predicting quality of life in multiple sclerosis:
aspects of the enrollment process in the NYSMSC patient data accounting for physical disability, fatigue, cognition, mood disorder,
personality, and behavior change. J Neurol Sci, 2005. 231(1-2): p. 29-34
registry. Specifically, we have classes for representing assay
[11] Brinkman, R.R., et al., Modeling biomedical experimental processes
processes and the data they produce. These assays are with OBI. J Biomed Semantics, 2010. 1 Suppl 1: p. S7.
evaluations of both patient reported and clinical measures of
[12] Smith, B., et al., The OBO Foundry: coordinated evolution of ontologies
quality of life, disease status, neurological impairment, and to support biomedical data integration. Nat Biotechnol, 2007. 25(11).
functional limitations. Classes were also developed to represent [13] Grenon, P. and B. Smith, SNAP and SPAN: Towards dynamic spatial
different types of conclusions that could be made using the ontology. Spatial cognition and computation, 2004. 4(1): p. 69-104.
available data so that thresholds could be individually [14] Xiang, Z., et al., OntoFox: web-based support for ontology reuse. BMC
established and easily changed for analysis purposes. The data Res Notes, 2010. 3: p. 175
produced from the assays are used to support conclusions [15] Courtot, M., et al., MIREOT: The minimum information to reference an
indicating patient-perceived or clinician-measured impairment. external ontology term. Applied Ontology, 2011. 6(1): p. 23-33..
[16] Jensen, M., et al., The neurological disease ontology. J Biomed
We used the ontology to perform a term enrichment Semantics, 2013. 4(1): p. 42.
analysis of subsets of patient records obtained from the data [17] Boyle, E.I., et al., GO::TermFinder--open source software for accessing
registry. We discovered that ontology terms annotated to male Gene Ontology information and finding significantly enriched Gene
patients in the NYSMSC registry are highly over represented Ontology terms associated with a list of genes. Bioinformatics, 2004.
for terms related to limitation in limbs versus terms annotated 20(18): p. 3710-5.
to female patients, even though the corresponding clinical [18] Shah, N. H., et al. Chapter 9: Analyses using disease ontologies. PLoS
measures were only marginally over-represented. Comput Biol, 2012. 8(12), e1002827. doi: 10.1371/journal.pcbi.1002827
55