<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta />
    <article-meta>
      <title-group>
        <article-title>Making clinical trials available at the point of care - connecting Clinical trials to Electronic Health Records using SNOMED CT and HL7 InfoButton standards</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>y Kol</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>ong Wong</string-name>
          <xref ref-type="aff" rid="aff1">1</xref>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          <institution>Termlex Limited</institution>
          ,
          <addr-line>Spaces, The Porter Building, Slough, SL1 1FQ</addr-line>
          ,
          <country country="UK">United Kingdom</country>
        </aff>
        <aff id="aff1">
          <label>1</label>
          <institution>University College London Hospitals</institution>
          ,
          <addr-line>250 Euston Road, NW1 2PJ</addr-line>
          ,
          <country country="UK">United Kingdom</country>
        </aff>
      </contrib-group>
      <abstract>
        <p>Making clinical trials discoverable at the point of care (patient encounter) is one of the holy grails of connecting clinical research with clinical practice [1, 2]. Semantic interoperability standards designed for hospital systems do not interface well with clinical trials, which are predominantly unstructured/free text. In this paper, we describe our experiences of using SNOMED CT and HL7 InfoButton standards to make clinical trials from a trial registry accessible to clinicians within an Electronic Health Record (EHR) system in University College Hospitals, London. In particular we discuss the use of HL7 InfoButton standard [15] as a standardised interface for a clinical trials repository, which we believe is a first of its kind in the UK. We discuss some of the barriers to making clinical trials more accessible in EHR systems, including considerations for using standards and associated challenges &amp; opportunities.</p>
      </abstract>
      <kwd-group>
        <kwd>Clinical trials</kwd>
        <kwd>SNOMED CT</kwd>
        <kwd>HL7 InfoButton</kwd>
        <kwd>Trial eligibility</kwd>
        <kwd>Keytrials</kwd>
      </kwd-group>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>-</title>
      <p>1.1</p>
    </sec>
    <sec id="sec-2">
      <title>Introduction</title>
      <sec id="sec-2-1">
        <title>Connecting clinical research to clinical practice</title>
        <p>
          There is extensive literature that highlights how despite clinical research and trials
being vital to advances in clinical medicine [
          <xref ref-type="bibr" rid="ref1 ref2">1, 2</xref>
          ], multiple challenges exist in patient
recruitment [
          <xref ref-type="bibr" rid="ref3">3</xref>
          ], physician participation [
          <xref ref-type="bibr" rid="ref3 ref4">3, 4</xref>
          ] and identification of patient eligibility
[
          <xref ref-type="bibr" rid="ref4">4</xref>
          ]. One of the key challenges in both patient recruitment and physician participation
is the ability to expose existing local clinical study information (e.g. eligibility,
recruitment status) to providers and patients [
          <xref ref-type="bibr" rid="ref5">5</xref>
          ]. While external trial registries like
ClinicalTrials.gov [
          <xref ref-type="bibr" rid="ref6">6</xref>
          ] and UK Clinical Trials Gateway [
          <xref ref-type="bibr" rid="ref7">7</xref>
          ] exist, site-specific information
in these registries are often not kept updated with on-going studies. At times coverage
of on-going trials in external registries can be less than 50% [
          <xref ref-type="bibr" rid="ref8">8</xref>
          ]. In other cases,
information in external registries might not be kept up to date with changes to the study.
        </p>
        <p>
          In this paper (written as application notes), we describe our experience of
creating Keytrials, a clinical trials discovery platform, designed to make local clinical
Copyright © 2019 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).
trials accessible to physicians and patients. While Keytrials makes existing clinical
trials (either local or imported from external registries) available to users via a web
(REST1) API2, our objective was to integrate trial-matching (on-demand) into the
Electronic Health Record (EHR) system. There have been past attempts at creating
electronic solutions and novel specifications for making local registries accessible to
external consumers [
          <xref ref-type="bibr" rid="ref10 ref5 ref9">5, 9, 10</xref>
          ] including EHR systems. However, we based our
integration between the EHR system and Keytrials on existing healthcare standards like
SNOMED CT [
          <xref ref-type="bibr" rid="ref11">11</xref>
          ] and HL7, that are already in use in Electronic Health Record
(EHR) systems within our setting and also internationally.
2
        </p>
      </sec>
    </sec>
    <sec id="sec-3">
      <title>Keytrials Platform</title>
      <p>Keytrials is an open source clinical trials discovery platform, designed to make it
easier for clinicians and patients to find trials that are open, with a goal to increase
trial recruitment and improve visibility of clinical trial activity at University College
London Hospital (UCLH), UK. Keytrials is built using modern Web 2.0 and Java
enterprise technologies. There is a clean separation of its backend layer from the user
interface and backend layers using REST APIs as shown in Figure 1. This makes it
easy for other 3rd party applications and other apps to plugin into the RESTful service
layer.
1 REST – Representational State Transfer
2 API – Application Programming Interface
3 R&amp;D – Research and Development
4 However, since morbidity and mortality information in hospital systems has traditionally been
2 AcPoId–edApupsilnicgatIiConDPfroorgsratamtumtoinryg rIenpteorrftainceg to the World Health Organisation (WHO), aspects
of the clinically relevant information (e.g. diagnosis, age, gender, interventions, etc.) tend to</p>
      <p>For the purposes of this paper, three aspects of Keytrials are of interest –
R&amp;D3 Environment Integration, Terminology Integration and HL7 InfoButton
Integration. Together, these three functionalities allow local (or remote) trials to be
accessible for trial-matching within the EHR, at the point of care.
2.1</p>
      <sec id="sec-3-1">
        <title>R&amp;D Environment Integration</title>
        <p>This functionality allows Keytrials to import existing trials from a clinical trials
registry. Within UCLH, existing trials are held in a local trial management system (Edge),
which acts as the primary source of trials. However, Keytrials also allows existing
trials to be imported from remote registries like ClinicalTrials.gov.
2.2</p>
      </sec>
      <sec id="sec-3-2">
        <title>Terminology Integration</title>
        <p>This functionality allows Keytrials to access a centralized `terminology server` that
provides search (lookup) functionality for healthcare terminologies like SNOMED
CT. Keytrials uses both the terminological content (e.g. concept ids, descriptions,
etc.) and the semantic relationships within SNOMED CT. For example, when users
can search for disease conditions they can search for matches using the preferred
terms (small cell lung cancer) or synonyms (oat cell carcinoma of lung). Both return
the exact trials, since the terminology server resolves them to the same SNOMED CT
concept. Keytrials also uses the underlying semantics of SNOMED CT as part of
returning matches. For example, if a user searches for `Plasma Cell Neoplasm`, it will
also bring back `Multiple Myeloma` even though there is no textual match between
Plasma Cell Neoplasm and Multiple Myeloma. It does this because in SNOMED CT,
Multiple myeloma is defined as a type of Plasma Cell Neoplasm - which makes
results more intuitive to our clinician users. A longer discussion of how SNOMED CT
as a standard is implemented in our workflow is discussed in section 3.4.
2.3</p>
      </sec>
      <sec id="sec-3-3">
        <title>ULCH EHR System</title>
        <p>
          UCL Hospitals (UCLH) have recently implemented Epic as their EHR system across
all clinical specialties. As part of this roll out, UCLH decided to adopt SNOMED CT
as the reference terminology for their EHR, in keeping with the national requirements
in the UK. However, instead of natively using SNOMED CT to populate their
diagnosis, UCLH procured a 3rd party content provider that provides an interface
terminology system for clinicians to use. This is mapped to ICD 10 [
          <xref ref-type="bibr" rid="ref12">12</xref>
          ] and SNOMED CT.
However, Epic does not currently support the transmission of SNOMED CT concept
ID via the Infobutton interface. Instead it can only provide the ICD 10 code. So when
Keytrials interfaces with Epic, it receives ICD codes instead of SNOMED CT codes.
Keytrials then uses the `terminology server` to translate these ICD codes into their
SNOMED CT equivalents as needed.
3 R&amp;D – Research and Development
        </p>
      </sec>
    </sec>
    <sec id="sec-4">
      <title>Standards based Integration with EHR</title>
      <p>3.1</p>
      <sec id="sec-4-1">
        <title>SNOMED CT Annotation of Trials</title>
        <p>
          Trials that have been imported into Keytrials have both structured (defined) data
elements (e.g. status, open date, closing date, etc.) and unstructured elements (e.g.
eligibility criteria, description/summary of trial, etc.). In order to match suitable trials with
existing patient details (e.g. age, disease conditions, gender), it is often the eligibility
criteria of a trial that are of most relevance. However, most of this information is
provided as `free-text` in trial, which is not coded to any `standardised` medical
vocabulary/terminology. As described above, the EHR itself is coded in either ICD or
SNOMED CT – leading to situation where trial-matching will require the clinical
trials to also be `coded` using the same coding system. As part of the project, we use
Bio-YODIE [
          <xref ref-type="bibr" rid="ref13">13</xref>
          ], a `Natural Language Processing` (NLP) engine to annotate clinical
trials with their corresponding disease conditions. The results of this NLP process are
clinical trials with associated disease conditions coded in SNOMED CT. These
`annotated trials` are then stored in Keytrials, making them available for subsequent
queries.
3.2
        </p>
      </sec>
      <sec id="sec-4-2">
        <title>HL7 InfoButton interface with EHR</title>
        <p>
          Context-dependent `infobuttons` have been proposed &amp; used for displaying
contextually relevant knowledge resources within EHRs [
          <xref ref-type="bibr" rid="ref14">14</xref>
          ]. This approach for integrating
online knowledge resources with EHRs has been standardized by HL7 as the
InfoButton standard [
          <xref ref-type="bibr" rid="ref15">15</xref>
          ]. The InfoButton standard allows systems (e.g. EHR systems) to
request information from `knowledge resources` using a standardised `reference
model` which can be expressed as a series of URL (Uniform Resource Locator) query
parameters and values. These requests can then be sent to the `knowledge resource`
using Hyper Text Transfer Protocol (HTTP) technologies. A limited subset of these
InfoButton standardised URL parameters are shown in table 1 below.
The patient’s age as a value and a unit
The action the user is performing in a clinical
information system when a knowledge
request is triggered (e.g., order entry,
laboratory results review, problem list review)
        </p>
        <p>Code systems
ICD, SNOMED-CT
HL7 administrative
gender
Not Applicable
HL7 Act Code</p>
        <p>
          HL7 Infobutton has been used to varying degrees of success in EHR systems
for clinical decision support, medication alerts and for allowing access to online
references [
          <xref ref-type="bibr" rid="ref17">17</xref>
          ]. It has more recently also been used to integrate genomic resources within
EHRs to mixed success [
          <xref ref-type="bibr" rid="ref18">18</xref>
          ]. However, since its inclusion in the `meaningful use`
certification in the US [
          <xref ref-type="bibr" rid="ref19">19</xref>
          ], major EHR vendors support its use out of the box. Within
our project, Epic the EHR system in use in UCLH supports InfoButton based
requests, making it quite attractive as a way for accessing trial information held in
Keytrials. This in effect, turns Keytrials into a knowledge resource for clinical trials and
allows us to use InfoButton URL queries to access trials appropriate for a patient.
3.3
        </p>
      </sec>
      <sec id="sec-4-3">
        <title>InfoButton Queries for Clinical Trials in Keytrials</title>
        <p>Using the URL query parameters specified in the standard, it is possible to create a
InfoButton request to a knowledge resource as below:
The above request specifies that the value of `age` as `78`. A slightly more realistic
query being sent to a test server for Keytrials would look like:
This translates to a query for all matching trials suitable for a patient of age 78 years
and an ICD-10 diagnosis of `Lung Cancer` (C34).</p>
        <p>The workflow within ULCH, is set up such that a when a clinician is with a
patient, she/he can right
click on a patient’s
diagnosis/disease
condition to display an
option for retrieving
matching clinical trials.</p>
        <p>This creates an
`InfoButton` query that is
sent to the `InfoButton
API` in Keytrials. As
shown in Figure 2,
Keytrials then translates
this query into its
internal representation and
creates a list of
matching trials. In our
project, we chose to
conOnce we find an equivalent SNOMED CT concept for an ICD-10 code, we perform a
`semantic expansion` based on the meaning of this SNOMED CT concept. For
example, when the query is for `T-cell Lymphoma`, we know that in most cases the user is
expecting trials for all types of `T-cell Lymphomas`. Our terminology server
calculates this `semantic expansion` (transitive closure) on the fly and returns all transitive
sub-types (descendants) for that concept. We refer to this `semantic expansion during
search` as `semantic search`. This `semantic search` based on SNOMED CT has the
added benefit of picking up concepts that would otherwise have been missed by
`textbased` search alone. For example, in Figure 3, we are able to include trials for
`Lennert’s Lymphoma` as part of `T-cell Lymphoma` trials, since in SNOMED CT it is
declared as a sub-type of `T-cell Lymphoma`. Any `text-based` search for `T-cell
Lymphomas` would have likely missed `Lennert’s Lymphoma` as it does not have the
token `T-cell` in it.</p>
      </sec>
    </sec>
    <sec id="sec-5">
      <title>Discussion</title>
      <p>Since the ability to support queries is based on the HL7 InfoButton and SNOMED CT
standards, we believe our approach should be adoptable by other investigators. We
believe that within the UK we are the first project to adopt InfoButton and SNOMED
CT standards for accessing clinical trials from an EHR system. This approach
however was not without issues given how clinical trials and clinical medicine do not often
support the same standards. We share some of our experiences in this section. These
challenges can be separated into trials related and EHR related issues.
4.1</p>
      <sec id="sec-5-1">
        <title>Issues with Clinical Trials data</title>
        <p>
          We have previously mentioned how existing large registries of trials have issues in
staying up to date with trials that are on-going and open for recruitment. This
continues to be a problem even in smaller registries. In our project, we were forced to build
a batch import integration between the local trial management system and Keytrials.
This batch import is currently run weekly to ensure that Keytrials is kept in sync with
the updates to local trial registry. We however recognise that creating integrations for
multiple local trial registry systems will be expensive as every system will likely have
its own internal representation. Standards based interchange format would help
simplify this task. While CDISC-ODM [
          <xref ref-type="bibr" rid="ref20">20</xref>
          ] exists, it is tied to the operational workflow
of running clinical trials as opposed to specifying the data standards for trials. In the
future term, we believe that HL7 FHIR might evolve to become a standardised
representation for clinical trials [
          <xref ref-type="bibr" rid="ref21 ref22">21, 22</xref>
          ]. However, this current specification of a
`ResearchStudy` is still in early stages of development [
          <xref ref-type="bibr" rid="ref23">23</xref>
          ].
        </p>
        <p>
          A further issue with making clinical trials accessible to EHR systems is the
inability to explicitly specify eligibility criteria (inclusion, exclusion criteria, disease
conditions, interventions, etc.) as structured/coded entities. While FHIR seems to
allow this level of specification in the future, a vast number of existing studies are
free-text based, limiting the ability to automatically match trials to coded diagnosis,
age or other information in EHR systems. This limitation can be overcome using NLP
as adopted within our project and other initiatives [
          <xref ref-type="bibr" rid="ref24 ref25 ref26">24 - 27</xref>
          ]. However, this approach
of post-processing and annotating trials could be avoided if clinical trials registries
could facilitate the coding of eligibility criteria at the time of trial registration.
4.2
        </p>
      </sec>
      <sec id="sec-5-2">
        <title>Issues with EHR data</title>
        <p>Similar to the state of clinical trials ecosystem, the landscape in EHRs is still riddled
with large amounts of un-coded and unstructured free-text information4. While having
4 However, since morbidity and mortality information in hospital systems has traditionally been
coded using ICD for statutory reporting to the World Health Organisation (WHO), aspects
of the clinically relevant information (e.g. diagnosis, age, gender, interventions, etc.) tend to
be coded more commonly.
ICD-10 used for coding diagnosis provides a slightly better starting point for
integrating EHRs with clinical trials, often the level of granularity required by researchers
and physicians interested in research is not provided by ICD as it was primarily
designed for statistical reporting.</p>
        <p>SNOMED CT is starting to see adoption across the globe and in the UK, but
within our project we note that Epic did not support SNOMED CT natively. This
meant that when we had to integrate our EHR (coded using ICD-10) with clinical
trials (annotated using SNOMED CT via NLP), we were forced to use ICD codes as
part of the `mainSearchCriteria` attribute in InfoButton to send diagnosis codes to
Keytrials. This required a workaround within Keytrials, where all ICD codes passed
via InfoButton were then processed by the `terminology server` to convert them into
corresponding SNOMED CT codes. As knowledgeable readers will note, going from
ICD-10 to SNOMED CT will often result in a `lossy` transform, as SNOMED CT is
often more granular/specific than ICD. This `lossy transform` and incorrect use of the
semantics of SNOMED CT while perhaps not immediately relevant for trial-matching
is likely to become more important when automated trial-matching becomes more
prevalent. We believe that with greater adoption of SNOMED CT, we will likely see
native use of this standard in EHR systems in the future so these `lossy` transforms
can be avoided.</p>
        <p>While not immediately part of the EHR issues, we also noted within our
project that the use of SNOMED CT presented interesting challenges. For example, in
SNOMED CT searching for `adenocarcinoma` might present two exact matches –
one of them being a `morphological abnormality` and the other being a `disorder`
making it confusing for users as to which match to select. This can easily be
addressed by ensuring that only relevant SNOMED CT hierarchies are included by
default during search – in this case only including `clinical findings` hierarchy from
SNOMED CT. However, it should also be noted that even within `clinical finding`
hierarchy, exactly named matches could sometimes appear. For example, searching
for `fatigue` might return a `symptom` and a `disorder`, both of which are part of the
`clinical finding` hierarchy. Needless to say, like all clinical information systems
using a terminology, a degree of clinical assurance is required to improve usability.</p>
        <p>However, on the whole using a combination of SNOMED CT and InfoButton
has provided a degree of assurance and flexibility within our project. We believe that
as clinical trials registries and EHR systems continue to mature, standards based
integration will continue to become more prevalent and a lot more plug-n-play.
5</p>
      </sec>
    </sec>
    <sec id="sec-6">
      <title>Conclusions</title>
      <p>In this paper we shared our experience of using existing healthcare standards
SNOMED CT and HL7 InfoButton to make data in a clinical trials accessible to EHR
systems. While InfoButton has been used with mixed results in other domains, it has
not been previously been used to access clinical trials in the UK. One major challenge
in making clinical trials discoverable and connecting them to EHRs is the lack of
standardisation of trial eligibility criteria – with most being just un-coded, free-text
content. This is a barrier for matching patients to eligible trials, even if relevant
information (coded diagnosis, age, gender, etc.) is already available within the patient
record in the EHR system. We used an NLP approach to annotate eligibility criteria
(e.g. disease conditions) in SNOMED CT, thereby allowing us to use a fuller range of
InfoButton query parameters to match trials to patients directly from the EHR system.</p>
      <p>Since our approach is based on international standards, we believe it could
serve as a means of creating reusable integrations between clinical trial registries with
EHR systems. However, the lack of standardisation of clinical trials might mean that
significant effort is required to integrate a clinical trials registry needs to a HL7
Infobutton compliant EHR system. We note that a standardised specification of clinical
trials could make this integration less onerous. However, existing standards for
clinical trials do not yet specify this level of detail (CDISC-ODM) and others are not yet
sufficiently mature to meet this need (HL7 FHIR). A similar, albeit slightly different
problem exists within EHR systems where relevant information is coded but in
ICD10, which does not always provide the level of detail required for clinical research.
However the increasing adoption of SNOMED CT in this space will likely solve that
issue, even if SNOMED CT itself comes with its own set of challenges. We hope that
as standards for clinical trials and EHRs mature and become more widely adopted, it
will be possible to make clinical trials discoverable at the point of care in EHR
systems using a plug-n-play model.</p>
      <p>Acknowledgements The authors would like to acknowledge that UCLH BRC
(Biomedical Research Centre) and CRIU (Clinical Research Informatics Unit) funded
development of the Keytrials platform.</p>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          1.
          <string-name>
            <surname>Campbell</surname>
            <given-names>EG</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Weissman</surname>
            <given-names>JS</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Moy</surname>
            <given-names>E</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Blumenthal</surname>
            <given-names>D.</given-names>
          </string-name>
          <article-title>Status of clinical research in academic health centers: views from the research leadership</article-title>
          .
          <source>JAMA</source>
          .
          <year>2001</year>
          ;
          <volume>286</volume>
          (
          <issue>7</issue>
          ):
          <fpage>800</fpage>
          -
          <lpage>6</lpage>
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          2.
          <string-name>
            <surname>Rindfleisch</surname>
            <given-names>TC</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Brutlag</surname>
            <given-names>DL</given-names>
          </string-name>
          .
          <article-title>Directions for clinical research and genomic research into the next decade: implications for informatics</article-title>
          .
          <source>J Am Med Inform Assoc</source>
          .
          <year>1998</year>
          ;
          <volume>5</volume>
          (
          <issue>5</issue>
          ):
          <fpage>404</fpage>
          -
          <lpage>11</lpage>
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          3.
          <string-name>
            <surname>Siminoff</surname>
            <given-names>LA</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Zhang</surname>
            <given-names>A</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Colabianchi</surname>
            <given-names>N</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Sturm</surname>
            <given-names>CM</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Shen</surname>
            <given-names>Q</given-names>
          </string-name>
          .
          <article-title>Factors that predict the referral of breast cancer patients onto clinical trials by their surgeons and medical oncologists</article-title>
          .
          <source>J Clin Oncol</source>
          .
          <year>2000</year>
          ;
          <volume>18</volume>
          (
          <issue>6</issue>
          ):
          <fpage>1203</fpage>
          -
          <lpage>11</lpage>
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          4.
          <string-name>
            <surname>Embi</surname>
            <given-names>PJ</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Jain</surname>
            <given-names>A</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Clark</surname>
            <given-names>J</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Harris</surname>
            <given-names>CM</given-names>
          </string-name>
          .
          <article-title>Development of an electronic health record-based Clinical Trial Alert system to enhance recruitment at the point of care</article-title>
          .
          <source>AMIA Annu Symp Proc</source>
          .
          <year>2005</year>
          ;
          <year>2005</year>
          :
          <fpage>231</fpage>
          -
          <lpage>235</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          5.
          <string-name>
            <surname>Stahl</surname>
            <given-names>DC</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Evans</surname>
            <given-names>RM</given-names>
          </string-name>
          Jr,
          <string-name>
            <surname>Afrin</surname>
            <given-names>LB</given-names>
          </string-name>
          ,
          <string-name>
            <surname>DeTeresa</surname>
            <given-names>RM</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Ko</surname>
            <given-names>D</given-names>
          </string-name>
          , Mitchell K.
          <article-title>Web services-based access to local clinical trial databases: a standards initiative of the Association of American Cancer Institutes</article-title>
          .
          <source>AMIA Annu Symp Proc</source>
          .
          <year>2003</year>
          ;
          <year>2003</year>
          :
          <fpage>624</fpage>
          -
          <lpage>628</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          6.
          <string-name>
            <given-names>Clinical</given-names>
            <surname>Trials Gov Site</surname>
          </string-name>
          , https://clinicaltrials.gov last accessed
          <year>2019</year>
          /09/21
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          7.
          <string-name>
            <given-names>UK</given-names>
            <surname>Clinical Trials</surname>
          </string-name>
          <string-name>
            <surname>Gateway</surname>
          </string-name>
          , https://bepartofresearch.nihr.ac.uk/ last accessed 2019/09/21
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          8.
          <string-name>
            <surname>Manheimer</surname>
            <given-names>E</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Anderson</surname>
            <given-names>D.</given-names>
          </string-name>
          <article-title>Survey of public information about ongoing clinical trials funded by industry evaluation of completeness and accessibility</article-title>
          .
          <source>BMJ 2002 Sep</source>
          <volume>7</volume>
          ;
          <fpage>325</fpage>
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          9.
          <string-name>
            <surname>Embi</surname>
            <given-names>PJ</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Jain</surname>
            <given-names>A</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Clark</surname>
            <given-names>J</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Harris</surname>
            <given-names>CM</given-names>
          </string-name>
          .
          <article-title>Development of an electronic health record-based Clinical Trial Alert system to enhance recruitment at the point of care</article-title>
          .
          <source>AMIA Annu Symp Proc</source>
          .
          <year>2005</year>
          ;
          <year>2005</year>
          :
          <fpage>231</fpage>
          -
          <lpage>235</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          10.
          <string-name>
            <surname>Huang</surname>
            ,
            <given-names>Z.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Ten</surname>
            <given-names>Teije</given-names>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            , &amp;
            <surname>Van Harmelen</surname>
          </string-name>
          ,
          <string-name>
            <surname>F.</surname>
          </string-name>
          (
          <year>2013</year>
          ).
          <article-title>SemanticCT: a semanticallyenabled system for clinical trials</article-title>
          .
          <source>In Process Support and Knowledge Representation in Health Care</source>
          (pp.
          <fpage>11</fpage>
          -
          <lpage>25</lpage>
          ). Springer, Cham.
        </mixed-citation>
      </ref>
      <ref id="ref11">
        <mixed-citation>
          11.
          <string-name>
            <surname>Stearns</surname>
            ,
            <given-names>Michael Q.</given-names>
          </string-name>
          , et al.
          <article-title>"SNOMED clinical terms: overview of the development process and project status</article-title>
          .
          <source>" Proceedings of the AMIA Symposium. American Medical Informatics Association</source>
          ,
          <year>2001</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref12">
        <mixed-citation>
          12.
          <string-name>
            <surname>The</surname>
            <given-names>ICD</given-names>
          </string-name>
          -
          <article-title>10 classification of mental and behavioural disorders: clinical descriptions and diagnostic guidelines</article-title>
          . Geneva: World Health Organization,
          <year>1992</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref13">
        <mixed-citation>
          13.
          <string-name>
            <surname>Gorrell</surname>
            , Genevieve,
            <given-names>Xingyi</given-names>
          </string-name>
          <string-name>
            <surname>Song</surname>
            , and
            <given-names>Angus</given-names>
          </string-name>
          <string-name>
            <surname>Roberts</surname>
          </string-name>
          .
          <article-title>"Bio-YODIE: A Named Entity Linking System for Biomedical Text." arXiv preprint arXiv:</article-title>
          <year>1811</year>
          .
          <volume>04860</volume>
          (
          <year>2018</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref14">
        <mixed-citation>
          14.
          <string-name>
            <surname>Cimino</surname>
            <given-names>JJ</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Elhanan</surname>
            <given-names>G</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Zeng</surname>
            <given-names>Q.</given-names>
          </string-name>
          <article-title>Supporting infobuttons with terminological knowledge</article-title>
          .
          <source>In: Proc AMIA annu fall symp; 1997</source>
          . p.
          <fpage>528</fpage>
          -
          <lpage>32</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref15">
        <mixed-citation>
          15.
          <article-title>Context-aware knowledge retrieval (infobutton) product brief</article-title>
          . HL7 International Wiki Site. https://wiki.hl7.org/index.php?title=Product_Infobutton, accessed
          <year>2019</year>
          /09/21.
        </mixed-citation>
      </ref>
      <ref id="ref16">
        <mixed-citation>
          16.
          <string-name>
            <surname>Retrieval</surname>
          </string-name>
          , HL7
          <string-name>
            <surname>Context-Aware Knowledge</surname>
          </string-name>
          .
          <article-title>"Knowledge Request URL-Based Implementation." Health Level Seven Int'l specification</article-title>
          ,
          <source>Jan</source>
          (
          <year>2010</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref17">
        <mixed-citation>
          17.
          <string-name>
            <surname>Del Fiol</surname>
          </string-name>
          ,
          <string-name>
            <surname>Guilherme</surname>
          </string-name>
          , et al.
          <article-title>"Implementations of the HL7 Context-Aware Knowledge Retrieval (“Infobutton”) Standard: challenges, strengths, limitations, and uptake</article-title>
          .
          <source>" Journal of biomedical informatics 45.4</source>
          (
          <year>2012</year>
          ):
          <fpage>726</fpage>
          -
          <lpage>735</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref18">
        <mixed-citation>
          18.
          <string-name>
            <surname>Heale</surname>
          </string-name>
          ,
          <string-name>
            <surname>Bret</surname>
            <given-names>SE</given-names>
          </string-name>
          , et al.
          <article-title>"Integrating genomic resources with electronic health records using the HL7 Infobutton standard</article-title>
          .
          <source>" Applied clinical informatics 7</source>
          .03 (
          <year>2016</year>
          ):
          <fpage>817</fpage>
          -
          <lpage>831</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref19">
        <mixed-citation>
          19. US Department of Health and
          <string-name>
            <given-names>Human</given-names>
            <surname>Services</surname>
          </string-name>
          .
          <article-title>"2015 Edition Health Information Technology (Health IT) Certification Criteria, 2015 Edition Base Electronic Health Record (EHR) Definition,</article-title>
          and
          <source>ONC Health IT Certification Program Modifications."</source>
          (
          <year>2015</year>
          ):
          <fpage>1</fpage>
          -
          <lpage>159</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref20">
        <mixed-citation>
          20.
          <string-name>
            <surname>Kuchinke</surname>
          </string-name>
          ,
          <string-name>
            <surname>Wolfgang</surname>
          </string-name>
          , et al.
          <article-title>"CDISC standard-based electronic archiving of clinical trials."</article-title>
          <source>Methods of information in medicine48.05</source>
          (
          <year>2009</year>
          ):
          <fpage>408</fpage>
          -
          <lpage>413</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref21">
        <mixed-citation>
          21.
          <string-name>
            <surname>Bender</surname>
            , Duane, and
            <given-names>Kamran</given-names>
          </string-name>
          <string-name>
            <surname>Sartipi</surname>
          </string-name>
          .
          <article-title>"HL7 FHIR: An Agile and RESTful approach to healthcare information exchange." Proceedings of the 26th IEEE international symposium on computer-based medical systems</article-title>
          . IEEE,
          <year>2013</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref22">
        <mixed-citation>
          22.
          <string-name>
            <surname>Leroux</surname>
            , Hugo, Alejandro Metke-Jimenez, and
            <given-names>Michael J.</given-names>
          </string-name>
          <string-name>
            <surname>Lawley</surname>
          </string-name>
          .
          <article-title>"Towards achieving semantic interoperability of clinical study data with FHIR."</article-title>
          <source>Journal of biomedical semantics 8</source>
          .1 (
          <year>2017</year>
          ):
          <fpage>41</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref23">
        <mixed-citation>
          23. HL7 FHIR Research Study Specification, https://www.hl7.org/fhir/researchstudy.html, accessed
          <year>2019</year>
          /09/21
        </mixed-citation>
      </ref>
      <ref id="ref24">
        <mixed-citation>
          24.
          <string-name>
            <surname>Wu</surname>
          </string-name>
          ,
          <string-name>
            <surname>Menghua</surname>
          </string-name>
          , et al.
          <article-title>"Characteristics of drug combination therapy in oncology by analyzing clinical trial data on ClinicalTrials</article-title>
          .
          <source>gov." Pacific Symposium on Biocomputing CoChairs</source>
          .
          <year>2014</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref25">
        <mixed-citation>
          25.
          <string-name>
            <surname>Pfiffner</surname>
          </string-name>
          ,
          <string-name>
            <surname>Pascal</surname>
            <given-names>B.</given-names>
          </string-name>
          , et al.
          <article-title>"ClinicalTrials. gov as a data source for semi-automated point-ofcare trial eligibility screening." PloS one 9</article-title>
          .10 (
          <year>2014</year>
          ):
          <fpage>e111055</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref26">
        <mixed-citation>
          26.
          <string-name>
            <surname>Deleger</surname>
          </string-name>
          ,
          <string-name>
            <surname>Louise</surname>
          </string-name>
          , et al.
          <article-title>"Building gold standard corpora for medical natural language processing tasks." AMIA Annual Symposium Proceedings</article-title>
          . Vol.
          <year>2012</year>
          . American Medical Informatics Association,
          <year>2012</year>
          .
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>