=Paper=
{{Paper
|id=Vol-1114/Session3_Clark
|storemode=property
|title=Ontology-based Temporal Analysis for Medical Device Adverse Event - A Use Case Study on Late Stent Thrombosis
|pdfUrl=https://ceur-ws.org/Vol-1114/Session3_Clark.pdf
|volume=Vol-1114
|dblpUrl=https://dblp.org/rec/conf/swat4ls/ClarkSQCT13
}}
==Ontology-based Temporal Analysis for Medical Device Adverse Event - A Use Case Study on Late Stent Thrombosis==
https://ceur-ws.org/Vol-1114/Session3_Clark.pdf
Ontology-based temporal analysis for medical device adverse
event— a use case study on Late Stent Thrombosis
Kim Clark1, Deepak Sharma2, Rui Qin2, Guoqian Jiang2, Christopher G. Chute2, Cui
Tao2,3*
1
Boston Scientific Corporation, Maple Grove, MN
2
Division of Biomedical Statistics and Informatics, Mayo Clinic, Rochester, MN
3
School of Biomedical Informatics, University of Texas Health Science Center at Houston
Houston, TX
* Corresponding author email: cui.tao@uth.tmc.edu
Abstract. In this paper, we show how we have applied the Clinical Narrative Temporal
Relation Ontology (CNTRO) and its associated temporal reasoning system (the
CNTRO Timeline Library) for automatically identifying, ordering, and calculating
the duration of temporal events within adverse event report narratives. The Objective
of this research is to evaluate the feasibility of the CNTRO Timeline Library using a
real clinical use case application (late stent thrombosis adverse events). Narratives
from late stent thrombosis adverse events documented within the Food and Drug
Administration’s (FDA) Manufacturing and User Facility Device Experience
(MAUDE) database were used as a test case. 238 annotated narratives were
evaluated using the CNTRO Timeline Library. The CNTRO Timeline Library had a
95.38% accuracy in correctly ordering events within the narratives. The duration
function of the CNTRO Timeline Library was also evaluated and found to have 80%
accuracy in correctly determining the duration of an event across 41 narratives, and
76.6% accuracy in determining the duration between two given events across 77
narratives. Within this paper is an example of how the durations calculated by the
CNTRO Timeline Library can be used to examine therapeutic guidelines. Complaint
narratives were separated into two groups based on a long (greater than 6 months) or
short (6 months or less) duration of antiplatelet therapy administration. The duration
of antiplatelet administration was then compared to the duration between stent
implantation and occurrence of late stent thrombosis. The goal of this analysis was to
show how the CNTRO ontology and is associated Timeline Library could be used to
examine recommendations for length of drug administration. In this use case, the
result supports guidance for use of longer antiplatelet therapy. This example validates
the CNTRO System’s ability to confirm known temporal trends.
1 Introduction
The Food and Drug Administration (FDA) requires notification of all medical
device adverse events that are associated with malfunction, serious injury, or death
[1]. Events leading up to the device failure are compiled and reported in a narrative
text, and made publically available through the MAUDE (Manufacturer and User
Facility Device Experience) database [4,5]. Temporal patterns may exist (potentially
including similar sequences of events, similar durations of or between events, or
similar time/date stamps of event occurrences), but are often buried with the text of
the narratives. Analysts at the Center for Devices and Radiological Health (CDRH)
read the event histories of each report looking for trends within narratives of similar
adverse event failure modes [6]. With 80,000 to 120,000 device-related adverse
events reported annually to the FDA [7], this method for monitoring adverse events is
time consuming, expensive, and the potential exists for a missed pattern observation.
An automated temporal analysis of adverse event narratives would lead to faster
identification of patterns, earlier prediction of a future failure, and could be used to
drive improvements into the next generation of medical devices.
Within this paper, we propose the use of the Clinical Narrative Temporal Relation
Ontology (CNTRO) [2], with its associated temporal reasoning framework (The
CNTRO Timeline Library) [3,8] to facilitate an efficient and semi-automated
temporal analysis of medical device adverse events obtained from the FDA’s
MAUDE database [4,5]. Previously, we have shown how CNTRO can be combined
with LifeFlow [9], software developed by the University of Maryland that is capable
of visualizing event sequences, to see patterns in the order of events within several
narratives [10]. We have also shown CNTRO’s ability to correctly answer temporal-
related questions regarding events that have occurred within a narrative [11]. The
goal of this present article is to illustrate how the CNTRO system (refers to the
ontology and its associated Timeline Library) can be used to analyze temporal
properties of events that are documented across multiple narratives of adverse events.
Many previous efforts have been made for time information and temporal relation
modeling in computer-based systems. Furia et al. surveyed these approaches and
carefully compared their adoptions in different applications [12]. The authors of this
survey then concluded that the usage of formal time models should be domain
specific since the time expression, abstraction, and reasoning can be varied across
different applications. This claim has been agreed by other researchers, especially for
the complex and unique clinical data [13,14]. A few modeling approaches have been
applied to represent the temporal information in clinical data. Many of them focus on
a specific task such as temporal reasoning on discharge summaries [15], temporal
extraction on eligibility criteria [16], or for time delayed mutual information over
populations [17]. Ontologies such as Time ontology [18] and the SWRL Temporal
ontology [19] can formally model temporal information in general and connect with
semantic reasoners for inferring new temporal relations based on the semantics
defined in the ontologies. These ontologies, however, only focus on structured data
with absolute time information and therefore cannot precisely capture the temporal
information expressed in human language [20]. In clinical narratives, many temporal
features are expressed in relative (e.g., next Friday) or ambiguous (e.g. early last
week) ways. Ignoring this data will forgo a lot of valuable information that could be
otherwise leveraged in clinical research. Models such as the HL7 time specification
[21] and the TimeML model [22] offer a way to represent temporal information
originally from semi-structured or unstructured narratives. These approaches,
however, do not provide formal semantic definition capacities for domain knowledge
as ontologies do. In clinical narratives, much temporal information is not explicitly
expressed, but rather needs to be inferred before the data can be further analyzed.
Without a reasoning component, it is difficult to resolve a relatively complete patient
history for profound clinical studies [23]. Therefore, we believe that the CNTRO
system is necessary since it provides a formal ontology in OWL with well-defined
semantics for the time domain and enables semantic-web [24] based temporal
reasoning.
In this article, we use the Late Stent Thrombosis (LST) use case to demonstrate
how to apply the CNTRO system on temporal relation reasoning and temporal
analysis. Although the exact mechanism or mechanisms of LST are not known, it has
been observed to occur less frequently when dual antiplatelet therapy has been
administered over a period of time [25,26]. Current guidelines recommend the
administration of dual antiplatelet therapy for 3 to 6 months following drug-eluting
stent implantation, unless the patient is not at high risk for bleeding, in which case
therapy is recommended for 12 months [2]. We used the CNTRO System to evaluate
events over the timeline and query both the duration in which antiplatelet therapy was
administered for in each adverse event narrative and the duration between the initial
stent implantation and the occurrence of late stent thrombosis. This work provides an
example of how the CNTRO system is able to understand temporal information from
multiple narrative files can be used to identify and confirm temporal trends.
A computer program cannot create a timeline of events and answer time-related
questions by querying information directly from a narrative without semantic
annotation and inference. Human experts can understand temporal relationships
through the use of words such as “before”, “after”, “during”, “following”, etc, and
appreciates that 1 year, 12 months, and 365 days are approximately equivalent even
though differences in granularity are used. To allow for a “machine-understandable”
data representation and exchange of temporal information automatically, the CNTRO
System uses a Semantic-Web [24] based framework to apply relationships between
events within natural language narratives through the use of the RDF (Resource
Description Framework) triple representation [2]. An RDF triple consists of a
subject, an object, and a predicate, which indicates the relationship between the
subject and the object6.
Consider the following example. “60 days after stent implantation, antiplatelet
therapy was discontinued in preparation for a splenectomy surgery.” In this example,
stent implantation is identified as the subject, antiplatelet therapy discontinuation is
identified as the object, and “after” is identified as the predicate. CNTRO is able to
recognize the granularity of 60 days as equivalent to two months. A temporal
relationship is created between stent implantation and discontinuation of antiplatelet
therapy using a temporal offset of two months.
The computer program now “understands” that stent implantation occurred first,
and discontinuation of antiplatelet therapy occurred second. It also “understands” that
the time delay between these two events was two months. Additionally, there is an
inference that because atniplatelet therapy was stopped, it had to have started at some
point. Unless explicitly stated, it is inferred that therapy began on the day of stent
implantation. The CNTRO framework then creates a timeline for events and provides
a programmatic query interface to access the timeline information. This makes it easy
for the time-related information to be queried in an automated manner. In our
particular example, we could ask questions such as: Which event occurred first? How
long after stent implantation was antiplatelet therapy administration discontinued?
2 Methods
2.1 The CNTRO System
CNTRO [2] is an OWL ontology designed to model temporal relations among
clinical events. Figure 1 shows the ontology overview. It models clinical events,
temporal entities (such as time instants, time intervals, repeated time periods, and
durations), time granularity (such as minute, hour, day, month, year), temporal
relationships, and time uncertainties in the semantic web notation. In order to
facilitate users to annotate events and time-related information using CNTRO
semantics, we have also implemented a Protégé plug-in, called Semantator [27]. The
annotated information can be stored as an OWL/RDF file or in an RDF triple store.
The annotated data can then be run through the CNTRO Timeline Library to infer
temporal information that is not explicitly expressed in the original documents [3].
The CNTRO Timeline Library contains a rule-based normalizer which automatically
converts different temporal expressions into standard formats. It also leverages the
semantic definitions in the ontology (e.g., OWL DL axioms, property transitivity and
inversions) to support temporal relation inference. In addition, the Timeline Library
contains a set of Java functions for answering a list of time-related questions such as
when a particular event happened, chronological sequence of events, durations of
events, durations between events, and temporal relations between events.
Figure 1: CNTRO Overview
2.2 Late Stent Thrombosis Adverse Event Identification
Medical device adverse event narratives resulting in late stent thrombosis were
obtained from the MAUDE database for the years 2004 through 2010. 2004 was
selected because this was the first full year following the initial drug-eluting stent
(Cypher) launch within the United States [4]. Adverse events were filtered by device
manufacturer and brand name as there are a limited number of drug-eluting stent
devices commercially available within the United States.
The FDA provides some sortable failure modes within the MAUDE database;
however, late stent thrombosis is not included as a failure mode. This is a weakness
within the MAUDE database, but not the focus of the paper. To find the adverse
events that resulted in late stent thrombosis, a text search for “thrombosis” and “LST”
was executed. Approximately 65% of the returned narratives were further filtered out
due to thrombosis not being confirmed, thrombosis occurred less than 30 days
following stent implantation (indicating thrombosis was acute and not late), and / or
the narrative did not specify the duration or define the duration as “late”. The
resulting narratives documenting occurrence of late stent thrombosis were then
annotated.
Figure 2: Semantic Annotation Using Semantator
2.3 Adverse Event Narrative Annotation
We created a domain ontology including common events occurring after stent
implantation with specific normalized event types. The specialized cases of temporal
events were imported into CNTRO for temporal relationship modeling. The
following events were included: initial stent implantation, follow up stent
implantation(s), start and stop time points of antiplatelet therapy administration,
unrelated surgeries occurring after stent implantation, late stent thrombosis,
myocardial infarction, admission to the emergency room, and patient death. The
duration between initial stent implantation and occurrence of thrombosis is used as
the output of the survival analysis performed within this paper. The start and stop
points of antiplatelet therapy are required to determine the duration of therapy, which
is used as a factor in the survival analysis. Follow-up stent implantations, unrelated
surgeries (a common reason to stop antiplatelet therapy early), myocardial infarction,
admission to the emergency room, and patient death were included within the
annotations to verify the Event Order and Inferred Relationship functions of the
CNTRO Timeline Library. Events such as guide wire insertion, which may have been
documented in the adverse event report, are required for all stenting procedures;
therefore annotation of these events would not be beneficial and this was not
performed. Life-saving events following the detection of thrombosis were not
annotated either, as this use case investigates understanding why late stent thrombosis
occurs and not the potential to survive the adverse event.
The LST adverse event files we identified were annotated using Semantator to
identify the above events and their temporal features. Each adverse event file resulted
in an OWL file which embedded the annotated results. Figure 2 shows a screenshot of
the Semantator annotation environment. The top left panel shows the narrative being
annotated, the top right panel shows the domain ontology with CNTRO imported, and
the bottom three panels show some of the annotation results. As we can see, each
event of interest or temporal expression can be annotated as OWL individuals with
respect to one or more ontology classes. Individuals of different classes are displayed
in different colors. Relations between these individuals can also be created using
Semantator.
2.4 CNTRO Timeline Evaluation
For each annotated LST narrative, the CNTRO timeline library creates a matrix
that visually shows the temporal relationships between the events, which is a simple
way to track, view, document, and evaluate the accuracy of CNTRO system timeline
computations. Each annotated event is included within the matrix. The matrix
indicates which events occur at the same time, and then orders the remaining events
on a timeline as applicable.
The annotations of the Late Stent Thrombosis Adverse Event Narratives were
reviewed using these matrices and compared against the manually-annotated gold
standard results. The gold standard annotations were evaluated by at least two human
experts. The conflicts of the annotation were resolved after discussions among the
human experts.
2.5 CNTRO Duration Evaluation
Durations can be computed for an individual event, between two events, or
between an event and a timestamp. CNTRO first determines if ‘start’ and ‘end’ time
information exists for an event to calculate the duration. If one of these pieces of
information is missing, the program then computes it by either using a duration
annotation, “Antiplatelet therapy was administered for two months” (the antiplatelet
therapy event is defined here with a duration of 2 months) or uses a temporal relation
to another event with a relative time stamp, “Antiplatelet therapy was started in May
2006. In July 2006, the patient underwent prostrate surgery. Antiplatelet therapy
was stopped the day before surgery.” In this second example the occurrence of
antiplatelet therapy starting and stopping each have a time stamp, and CNTRO infers
that antiplatelet therapy was administered for 2 months based on the duration between
the start and end times. In some cases, the duration of a pair of events cannot be
calculated directly (the two events are not directly connected through the RDF graph),
but need to go through one or more intermediate events. In this case, the above two
functions need to be called iteratively until the duration of the two events are
calculated.
The adverse event narratives for late stent thrombosis could describe durations in
days, months, and/or years. Month was the most frequent granularity used in the
complaint data, followed by years, and then days. To be able to compare data from
different narratives, the duration granularity was normalized to ‘Month’ for this use
case as this was the most frequently used granularity, and estimating durations
reported in years by number of days would likely increase the noise within the data.
To normalize durations reported in days, the duration was divided by 30 and rounded.
To normalize durations reported in years, the durations were multiplied by 12.
Durations can also be calculated from start and end time stamps of a particular event
or time stamps of two events. In some narratives, timestamps were reported with a
granularity of ‘Year.’ (In example – “Stent implantation occurred in 2006 and
thrombosis occurred in 2008.”) The potential range of duration between these events
could be anywhere between 12 months (if the stent implantation occurred in
December 2006 and the thrombosis occurred in January 2008) and 24 months (if the
stent implantation occurred in January 2006 and the thrombosis occurred in December
2008). The average potential duration was used in this type of narration, (in the above
example, 18 months). The durations calculated by CNTRO were compared to manual
calculations to determine accuracy.
2.6 Application of CNTRO Temporal Analysis
To provide an example of how the CNTRO system can potentially be used to
evaluate temporal properties within narrative data, survival analysis was performed
using the narratives that specified both a duration of antiplatelet therapy and time
from stent implantation to late stent thrombosis (or in which a duration could be
inferred) to examine therapeutic guidelines for antiplatelet administration duration.
Note that as this data come from the FDA MAUDE Database, all records within the
example ended up with an event of late stent thrombosis. Data of patients who have
not had a late stent thrombosis occurrence are not easily accessible; therefore this
example is purely illustrative of the CNTRO system’s capability. Similarly, because
the data used within this analysis comes from adverse event files indicating
thrombosis occurred, no patient data requires censoring.
Late Stent Thrombosis adverse event files were divided into two different groups
based on how long antiplatelet therapy was administered in patients following
implantation of a drug-eluting stent. Using current antiplatelet therapy
recommendations, any adverse event narrative specifying that antiplatelet medication
was administered for less than 6 months was segregated into the Shorter Duration of
Antiplatelet Therapy group. Any adverse event narrative indicating that antiplatelet
medication was administered for 6 or more months was segregated into the Longer
Duration of Antiplatelet Therapy group. Adverse event narratives that did not provide
information specifying how long antiplatelet therapy was prescribed were excluded
from the analysis.
3 Result
3.1 CNTRO Timeline and Duration Evaluation
There were 238 late stent thrombosis adverse event narratives contained enough
event information such that a timeline could be created within CNTRO for system
evaluation. For each report, we compared the CNTRO system inferred timeline with
the gold standard result. The system was able to provide the correct results for all but
8 files. This resulted in an overall CNTRO timeline accuracy of 95.38%.
There were 41 late stent thrombosis adverse event narratives that included
information such that the duration of antiplatelet therapy was known. The CNTRO
automatic reasoning system had an 80% accuracy in inferring / calculating this
duration of an event. There were 77 late stent thrombosis adverse event narratives
included information such that the duration between stent implantation and late stent
thrombosis was known. The CNTRO automatic reasoning system had a 76.3 %
accuracy in inferring / calculating this duration between events. An analysis of the
errors and discussion of possible enhancements to the CNTRO system is included
within the Discussion section.
3.2 Late Stent Thrombosis Adverse Event Temporal Pattern Analysis
Within this paper, the CNTRO system was used to confirm what has been
previously identified as temporal patterns within the late stent thrombosis adverse
event in a semi-automated manner, which is more efficient than through manual
observation. The common event pattern within late stent thrombosis adverse events
(stent implantation, administration of antiplatelet therapy, discontinuation of
antiplatelet therapy, late stent thrombosis) was shown by CNTRO system through
timeline identification of events. This result shows that the CNTRO system has the
potential to be applied across multiple adverse event failure modes to identify new
trends that have previously not been observed.
3.3 Late Stent Thrombosis Survival Analysis
There were 36 late stent thrombosis adverse events that included both the duration
between drug-eluting stent implantation and occurrence of late stent thrombosis, and a
duration of antiplatelet therapy. These 36 reports were used to execute a survival
analysis. Although this represents only a limited subset of late stent thrombosis
events, the data can still be used for illustration purposes of CNTRO’s temporal
analysis capabilities. Late Stent Thrombosis adverse event files were divided into two
different groups based on how long antiplatelet therapy was administered in patients
with an implanted drug-eluting stent. Per the current guidelines, following drug-
eluting stent implantation, patients should take antiplatelet therapy for 3 to 6 months,
unless the patient is not at high risk for bleeding, in which case therapy is
recommended for 12 months [28]. In our case, we considered 2 different groups:
shorter Antiplatelet Therapy duration group (antiplatelet medication was administered
for less than 6 months) and longer Antiplatelet Therapy duration group (antiplatelet
medication was administered for 6 or more months). Adverse event narratives that
did not provide information specifying how long antiplatelet therapy was prescribed
were excluded from the analysis. 14 adverse events reported that antiplatelet therapy
was administered for 6 months or less following initial stent implantation. 22 adverse
events reported that antiplatelet therapy was administered greater than 6 months.
Survival analysis with Kaplan-Meier curve and log-rank test was performed in
Minitab. The median time to LST is 27.3 months for longer antiplatelet therapy group
and 14.6 months for shorter antiplatelet therapy group, respectively. The p-value of
log-rank test is 0.029, which indicates a significant association between duration of
antiplatelet therapy and time to LST. Figure 3 displays that late stent thrombosis
occurs later in patients who continued to take antiplatelet therapy longer than 6
months. Although this is a retrospective observational study of a subset of LST cases
only, the finding is consistent and supports guidance for use of longer antiplatelet
therapy [2]. This example validates the CNTRO System’s ability to confirm known
temporal trends.
Figure 3: Survival analysis of shorter duration of antiplatelet therapy (group
1) and longer duration of antiplatelet therapy (group 2) in late stent thrombosis
adverse events
4 Discussion
The CNTRO system was able to order the event sequences for 95.3% of the
narratives. A few cases failed due to different interpretations of time intervals.
Computing the order of two events is difficult when using ‘start’ or ‘finish’ temporal
relations when both the start and end times are missing from the annotation. For
example, a narrative might specify that antiplatelet therapy began at the time of stent
implantation, and specify that it occurred for a period of 2 months. The temporal
relation of the event1 (antiplatelet therapy) and event2 (stent implantation) depends
on whether we compare by them the start time or the end time of the events. By
considering start time, the two events start at the same time (event1 starts event2).
The system cannot infer the relationship by the end time since the duration of “stent
implantation” was not specified. Given the assumption that the stent implantation
procedure cannot last for 2 months, we can infer that event1 ends after event2. This
kind of background knowledge needs to be further specified in the domain ontology
so that the CNTRO system can infer the correct order. For example, “patient death”
has to be the last event, which occurs in the patient-care timeline. This kind of
inherited order needs to be incorporated in the domain ontology so that the sequence
of events can be inferred.
For duration inference, there are three major reasons the program failed to return the
correct results. (1) Annotation ambiguities: some narratives contain duration
information in an ambiguous way such as in range (e.g., 2-3 month), or in different
levels of granularity (e.g., “two month and 3 days”) that the program cannot
automatically process; (2) Long series of events: sometimes the duration calculation
involves a long series of events. The program fails when there is more than one
intermediate event between the start and the end events and the series of events
involve combination of different temporal relations and/or absence of any temporal
information of from an intermediate event; (3) Temporal relation granularity: an
annotator can specify the level of granularity over a temporal relation. For example,
we can specify that the granularity of “event1 before event2” is “day”. This means
that the temporal relation was compared on the granularity of day, which implies that
although event1 was before event2, but they happened on the same day. This
assumption was not programmed in the CNTRO system yet. This caused a few errors
when calculating the duration. For example, we further know that event3 happened
183 days after event2. Without the assumption that event1 and event2 happened on
the same day, the system cannot infer the duration between event1 and event3.
5 Conclusion
Although the CNTRO system can provide relatively good results for our use case,
there are still limitations in the system. First, the evaluation results work well with the
MAUDE reports because these reports are relatively short and simple compared to
other clinical narratives such as clinical notes. More system evaluation of to be done
for complex EHR data. Second, since the purpose of this study is to evaluate
CNTRO’s representation and reasoning capacities, the reports were annotated
manually. The current manual annotation method is not practical for long-term use,
and an automatic annotation process is currently under implementation. Third, many
ambiguities and uncertainties were resolved during the annotation process. We are
working on incorporating uncertainty reasoning in the CNTRO system. Nevertheless,
this study provides promising results and valuable analysis for us to continue develop
the CNTRO system.
ACKNOWLEDGMENTS This research is partially supported by the National
Science Foundation under Grant #0937060 to the Computing Research Association
for the CIFellows Project, and the ONC Strategic Health IT Advanced Research
(SHARP) award under Grant #90TR0002-01. We thank Ms. Donna Ihrke for her help
on annotating the files.
References
1. Medical Device Reporting. 21 CFR part 803 (2010).
2. Tao C, Wei W, Solbrig H, Savova G, Chute C. CNTRO: a semantic web ontology
for temporal relation inferencing in clinical narratives. AMIA Annu Symp Proc
2010;787-791.
3. Tao C, Solbrig H, Sharma D, Wei W, Savova G, Chute C. Time-oriented question
answering from clinical narratives using semantic-web techniques. ISWC 2010,
Part II. LNCS 2010;6497:241-256.
4. United States Food and Drug Administration (US FDA). Manufacturer and user
facility device experience (MAUDE Database, 2011). Available on the World
Wide Web:
http://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfmaude/search.cfm.
5. Feldman M, Petersen A, Karliner L, Tice J. Who is responsible for evaluating the
safety and effectiveness of medical devices? The role of independent technology
assessment. J Gen Intern Med 2007;23 (Suppl 1):57-63.
6. Levinson D, Inspector General, Department of Health and Human Services.
Adverse event reporting for medical devices. October 2009.
http://oighhs.gov/oei/reports/oei-01-08-00110.pdf.
7. Maisel W. Medical device regulation: an introduction for the practicing physician.
Ann Intern Med 2004;140:296-302.
8. CNTRO Timeline Library
http://informatics.mayo.edu/CNTRO/index.php/TimeLine_API_Library
9. Woongsuphasawat K, Gomez JAG, Plaisant C, Wang TD, Schneiderman B,
Taieb-Maiman M. LifeFlow: Visualizing an overview of event sequences. CHI
Annu Symp. 2011: 1747-1756.
10. Tao C, Wongsuphasawat K, Clark K, Plaisant C, Shneiderman B, Chute CG.
Towards event sequence representation, reasoning and visualization for EHR data.
In Proc. ACM SIGHIT International Health Informatics Symposium (IHI) 2012.
11. Clark K, Sharma D, Chute C, Tao C. Application of a temporal reasoning
framework tool in analysis of medical device adverse events, AMIA Annu Symp
Proc. 2011: 1366-1371.
12. Furia, CA, et al., Modeling Time in Computing: A Taxonomy and a Comparative
Survey. Acm Computing Surveys, 2010. 42(2).
13. Li, M. and J. Patrick, Extracting temporal information from electronic patient
records. AMIA Annu Symp Proc, 2012. 2012:542-51.
14. Sun W, Rumshisky A, Uzuner O AbstractTemporal reasoning over clinical text:
the state of the art. J Am Med Inform Assoc. 2013 May 15
15. Zhou, L., et al., A temporal constraint structure for extracting temporal
information from clinical narrative. Journal of Biomedical Informatics, 2006.
39(4):424-439.
16. Luo, Z., el at., Extracting temporal constraints from clinical research eligibility
criteria using conditional random fields. AMIA Annu Symp Proc, 2011:843-52.
17. Albers, DJ and G Hripcsak, Using time-delayed mutual information to discover
and interpret temporal correlation structure in complex populations. Chaos, 2012.
22(1):013111.
18. Time Ontology in OWL Available from: http://www.w3.org/TR/owl-time/.
19. The SWRL Temporal Ontology. Available from: http://protege.cim3.net/cgi-
bin/wiki.pl?SWRLTemporalOntology.
20. Tao, C. et al., CNTRO: A Semantic Web Ontology for Temporal Relation
Inferencing in Clinical Narratives. AMIA Annu Symp Proc, 2010. 2010:787-91.
21. HL7 Time Specification. 2012; Available from: http://www.h17.org/.
22. Boguraev, B., et al., TimeBank evolution as a community resource for TimeML
parsing. Language Resources and Evaluation, 2007. 41(1):91-115.
23. Zhou, L. and G. Hripcsak, Temporal reasoning with medical data—a review with
emphasis on medical natural language processing. J Biomed Inform, 2007.
40(2):183-202.
24. Palmer S. The sematic web: an introduction. (September 2001)
http://infomesh.net/2001/swintro/.
25. Harrington R, Ohman E. The enigma of drug-eluting stents: hope, hype, humility,
and advancing patient care. J Am Coll Cardiol 2007;297:2023-2030.
26. McFadden E, Stabile E, Regar E, Cheneau E, Ong A, Kinnaird T, Suddath W,
Weissman N, Torguson R, Kent K, Pichard A, Staler L, Waksman R, Serruys P.
Late thrombosis in drug-eluting coronary stents after discontinuation of
antiplatelet therapy. Lancet 2004;364:1519-1521.
27. Semantator (Feb 2012). Available on the World Wide Web:
http://informatics.mayo.edu/CNTRO/index.php/Semantator.
28. Grines C, Bonow R, Casey D Jr, Gardner T, Lockhart P, Moliterno D, O’Gara P,
Whitlow P. Prevention of premature discontinuation of dual antiplatelet therapy in
patients with coronary artery stents: a science advisory from the American heart
association, American college of cardiology, society for cardiovascular
angiography and interventions, American college of surgeons, and American
dental association, with representation from the American college of physicians.
Circulation 2007;115:813-818.