<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta />
    <article-meta>
      <title-group>
        <article-title>Towards a Common Semantic Representation of Informed Consent for Biobank Specimens</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Frank J. Manion</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Yongqun He</string-name>
          <xref ref-type="aff" rid="aff3">3</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Elizabeth Eisenhauer</string-name>
          <xref ref-type="aff" rid="aff2">2</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Yu Lin</string-name>
          <xref ref-type="aff" rid="aff3">3</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Alla Karnovsky</string-name>
          <xref ref-type="aff" rid="aff1">1</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Marcelline R. Harris</string-name>
          <xref ref-type="aff" rid="aff2">2</xref>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          <institution>Comprehensive Cancer Center, University of Michigan Medical School</institution>
          ,
          <addr-line>Ann Arbor, MI 48109</addr-line>
          ,
          <country country="US">USA</country>
        </aff>
        <aff id="aff1">
          <label>1</label>
          <institution>Department of Computational Medicine and Bioinformatics, University of Michigan Medical School</institution>
          ,
          <addr-line>Ann Arbor, MI 48109</addr-line>
          ,
          <country country="US">USA</country>
        </aff>
        <aff id="aff2">
          <label>2</label>
          <institution>Division of Systems Leadership and Effectiveness Science, University of Michigan School of Nursing</institution>
          ,
          <addr-line>Ann Arbor, MI 48109</addr-line>
          ,
          <country country="US">USA</country>
        </aff>
        <aff id="aff3">
          <label>3</label>
          <institution>Unit for Laboratory Animal Medicine, Department of Microbiology and Immunology, and Center for Computational Medicine and Bioinformatics, University of Michigan Medical School</institution>
          ,
          <addr-line>Ann Arbor, MI 48109</addr-line>
          ,
          <country country="US">USA</country>
        </aff>
      </contrib-group>
      <abstract>
        <p>- Biospecimen-based research is rapidly growing in the post genomic era, and includes the need to retrieve specimens from distributed biobanks of various size and complexity in a fashion that ethically preserves the expressed wishes of specimen donors as represented by the informed consent process and its artifacts. This paper briefly describes existing work along these lines, presents some challenges unique to biobanks, and presents our own work on an ontology of informed consent.</p>
      </abstract>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>-</title>
      <p>Keywords— BioBank, Informed consent; ontology; ICO; OBO
Foundry, Basic Formal Ontology (BFO), OBI ontology</p>
      <p>I.</p>
      <p>INTRODUCTION</p>
      <p>
        Research in the post-genomic era requires access to high
quality biospeciments, often annotated with or linked to
clinical data. Many groups at varying levels of institutional
complexity, ranging from small scale individual laboratories to
distributed international collaboratives, have established and
operate biorepositories (also refered to by various names such
as biobanks, biolibraries, and even collections). Often, there
are needs to share data and specimens among multiple
biobanks [
        <xref ref-type="bibr" rid="ref1 ref2 ref3">1-3</xref>
        ]. The act of requesting specimens from a
biorespository may demand a complex series of transactions,
each of which in turn may convey a series of rights, obligations,
and permissions for access to specimens and data. Despite over
a decade of experience incorporating biospecimens in the
research process, formal models that describe the use of
biorepositories in human research are a relatively recent
development. Without a common formal model of consent and
the associated permissions on collection and distribution of
specimens and data, integration of data across the translational
spectrum, or from multiple banks and institutions will remain a
difficult, manually intensive problem.
      </p>
      <p>In this paper we briefly review current efforts toward such
models, describe our own work toward a formal model for
informed consent, and describe what we consider challenges
and opportunities for supporting biorespository-based research
with ontologies. A simple example that provides a motivation
for this effort follows.</p>
      <p>II.</p>
      <p>EXAMPLE OF THE CHALLENGE</p>
      <p>Clinical or translational research often involves the
extraction and usage of biospecimen from humans. Different
biospecimens may be stored and processed differently, and
* Corresponding author.
may be collected under different models of consent. A typical
scenario might read something like this:</p>
      <p>“For  my  study,  I  want to use samples from my
organizations’s  biobank, collected under a blanket biobank
informed consent form. I discover that I will need more
samples, so I contact  another  organization’s  biobank  to 
determine if they hold relevant and available specimens.
That  organization’s  samples  were  collected  under  a  tiered 
biobank informed consent form. While some samples are
shared with me, I still need more samples to address the
requirements of my study. I then collect additional samples
using a consent form specific to my study.”</p>
      <p>In this example there are three informed consents forms to
account for – a blanket consent, a tiered consent, and the
investigator’s single study consent. In an effort to support the
expressed wishes of the donors, informed consent documents
impose a series of legal and ethical restrictions, obligations,
and permissions to biobank operators and research teams using
the specimens and data collected in these banks. Often these
rights, obligations, and permissions accrue from multiple
sources of authority and are represented in multiple legal
documents. Consequently, the biobanking domain presents a
series of modeling challenges, including:</p>
      <p>The operational model of the biobank. A biobank can be a
single, dedicated resource that provides samples to single or
closely allied groups of studies using a common consent
model. It might be a virtual or distributed biorepository using
precoordinated consent models. Another organization structure
might be that of a shared biobank facility containing multiple
sets of tissues from multiple projects and attempting to
maximize use of these tissue resources by making them
available to requestors.</p>
      <p>The consent model used for the biobank. This can be opt-in
or opt-out. In the case of an opt-in consent model, a tiered
consent may be used to present the participant or volunteer
with choices of the type of data the participant may want
shared, and for what types of research or other constraints.</p>
      <p>
        The protocol model the bank operates under. Typically a
biobank serving more than one project would operate under
one or more Institutional Review Board (IRB)-approved
collection protocols and Health Insurance Portability and
Accountability Act (HIPAA) authorizations. Researchers
subsequently requesting specimens and data would operate
under separate IRB-approved protocols, and depending on this
protocol, separate consent and HIPAA authorization may be
required for use of a previously banked specimen. Such a
model is sometimes called a two-protocol model [
        <xref ref-type="bibr" rid="ref4">4</xref>
        ].
      </p>
      <p>Rights, obligations, and permissions accrue from multiple
sources and must be consistent across time. Properly modeling
the decisions typically made by human review boards and
regulatory personnel considering sample and data distribution
for research requires modeling not just the consent documents,
but the protocols, data use agreements, and possibly other
information artifacts used in both depositing samples into a
biobank, and withdrawing them for subsequent research.</p>
      <p>In a research oriented university such as the University of
Michigan, thousands of informed consent forms have been
generated, and there are over 100 biobanks in the Medical
School alone. Queries supporting appropriate use of banked
biospecimens and data must be linked to the signed informed
consent agremments with the biospecimen donor.</p>
      <p>III.</p>
      <p>EXISTING EFFORTS</p>
      <p>
        Several current efforts are evident, focused on modeling
aspects of the biobanking domain. At least two BFO-aligned
ontologies relate to biobanking. The Ontologized Minimum
Information About BIobank data Sharing (OMIABIS)
expresses data concepts in an ontology of biobank
administration [
        <xref ref-type="bibr" rid="ref5">5</xref>
        ]. OMIABIS is based on work by Norlin and
colleagues [
        <xref ref-type="bibr" rid="ref6">6</xref>
        ] to develop a minimum data set for eight
countries participating in the EU Biobanking and Biomolecular
Resources Research Infrastructure project. Limitations of this
effort are that it is intended to serve only as a description of a
biobank contents, and does not describe collection critera,
consenting, and protocol provenance of individual specimens.
A group at the University of Pennsylvania is developing an
ontology for the representation of biobanks, although the work
is in early stages [
        <xref ref-type="bibr" rid="ref7">7</xref>
        ]. Similarly, we are aware that a group at
Duke University is working on a collaborative effort to develop
a normative set of data elements and terms to recommend as
best practice to the International Society for Biological and
Environmental Repositories (ISBER), although this work is not
yet published [
        <xref ref-type="bibr" rid="ref8 ref9">8, 9</xref>
        ].
      </p>
      <p>
        There are also non-BFO aligned ontologies in related areas,
including a Permission Ontology used for development and
evaluation of software tools for reasoning about consent
permission, published by a group at the University of
California San Diego (UCSD) [
        <xref ref-type="bibr" rid="ref10">10</xref>
        ]. Related work to build a
Research Permission Management System was done at the
Medical University of South Carolina (MUSC) to support a
statewide research network [
        <xref ref-type="bibr" rid="ref11">11</xref>
        ]. A search of the term
“consent”  in  the  NCBO  biportal  identified  the  notion  of 
informed consent at the class level in 19 different systems
(http://bioportal.bioontology.org/search).
      </p>
      <p>Our efforts to develop a BFO-aligned informed consent
ontology (ICO) emphasizes the broad domain of informed
consent. Although motivated by a biobanking use case, initial
development reported here is not restricted to that domain.</p>
    </sec>
    <sec id="sec-2">
      <title>THE INFORMED CONSENT ONTOLOGY (ICO)</title>
      <p>
        Development of ICO, a BFO-based ontology represented in
the Web Ontology Language (OWL2) [
        <xref ref-type="bibr" rid="ref12">12</xref>
        ], follows OBO
Foundry principles of openness and collaboration. ICO is
aligned with the BFO [
        <xref ref-type="bibr" rid="ref13">13</xref>
        ], making it possible to align and
integrate with other BFO-based ontologies. The initial release
of the ontology focuses on modeling informed consent
documents. As for Aug. 14, 2014, ICO contains 471 terms
including 137 ICO-specific terms and other terms imported
from other BFO-aligned ontologies. Detailed ICO statistics can
be found on the Ontobee ICO web page:
http://www.ontobee.org/ontostat.php?ontology=ICO. ICO is
released under an open Creative Commons 3.0 License.
      </p>
      <p>
        The ontology was developed using a combination of
topdown and bottom-up approaches. Protégé-OWL 4.2 was used
for the ontology authoring and editing. To build the OBI-based
framework of ICO we manually identified informed consent
concepts from existing OBO Foundry library ontologies. These
were imported to ICO using Ontodog [
        <xref ref-type="bibr" rid="ref14">14</xref>
        ] and OntoFox [
        <xref ref-type="bibr" rid="ref15">15</xref>
        ]
which allowed for recursive inclusion of all defined axioms
and related terms. The results were then manually reviewed for
final approval before inclusion in the ICO framework.
      </p>
      <p>
        Bottom up construction proceeded by manually identifying
and extracting a list of candidate terms from two informed
consent templates used at the University of Michigan (one
from the Medical School Institutional Review Board, another
from the Health Sciences and Behavioral Sciences Institutional
Review Board). We also identified terms from a consent form
used for the University of Michigan Medical School
biorepository, and from World Health Organization (WHO)
informed consent templates. The candidate terms identified
from these templates were then enriched with metadata
including definitions, concept identifiers, preferred terms,
synonyms, and URIs extracted from three ontology
repositories: the National  Library  of  Medicine’s  Unified
Medical Language System (UMLS®) Metathesaurus [
        <xref ref-type="bibr" rid="ref16">16</xref>
        ]; the
National Center for Biomedical Ontology (NCBO) BioPortal
[
        <xref ref-type="bibr" rid="ref17">17</xref>
        ]; and Ontobee [
        <xref ref-type="bibr" rid="ref18">18</xref>
        ]. When textual definitions were not
provided, other sources such as clinical research glossaries or
the current literature were used. These enriched candidate
terms were manually mapped to several pre-identified
resources containing terms and definitions developed and
vetted by the United States regulatory community. This process
yielded candidate preferred terms contining definitions
accepted as robust and well defined by that community.
Resources used in this step included the National Cancer
Institute Thesaurus (NCIt), the Biomedical Research Integrated
Domain Group (BRIDG) [
        <xref ref-type="bibr" rid="ref19">19</xref>
        ], the Ontology of Clinical
Research (OCRe) [
        <xref ref-type="bibr" rid="ref20">20</xref>
        ], the Consumer Health Vocabulary
(CHV) and the University of California San Diego permission
ontology [
        <xref ref-type="bibr" rid="ref10">10</xref>
        ].
      </p>
      <p>The pool of enriched candidate terms was organized into
categories of like terms according to their definitions. For
example, the category  ‘authorization’  included the terms
‘authorization for medical records release’,  ‘ authorization
documentation’  or  ‘ authorization’. Enriched candidate terms
grouped by categories formed to-be-included terms in ICO.
The final set of categories (or modeling units) was then
mapped to branches of BFO. For example, terms categorized
under  ‘authorization’  were considered to be subclasses of
BFO:process. Informed consent workflows in a typical clinical
research study were modeled as three processes: (i)
preinformed consent processes, (ii) obtaining informed consent
processes, and (iii) processes after signing informed consent
documents. Relations between entities involved in the above
processes were defined. Finally, all terms and relations were
aligned with BFO.</p>
      <p>DISCUSSION AND CONCLUSIONS</p>
      <p>Modeling informed consent is a necessary but not sufficient
part of the modeling needed to support responsible use of
biospecimens and data in research. Biospecimen and data
release is complex, and informed consent plays a major role in
the regulatory and scientific governance used by
biorepositories to release specimens and data. In follow on
work we plan to examine the specific area of specimen and
data release involving the longitudinal agreements of rights,
permissions, and obligations. Other work is needed in the
complex areas of protocol representation, data use agreements
and material transfer agreements.</p>
      <p>Limitations of our preliminary work will inform further
development efforts toward a robust Informed Consent
Ontology. First, the ICO is admittedly preliminary work and is
currently focused on informed consent documents and
processes. More work is needed to validate the coverage and
completeness in the domain. Concepts from the US Common
Rule and the EU Prior Informed Consent legislation need to be
included. Our current models of informed consent processes
likely lack the richness and complexity of real-life informed
consent processes, and they need validation with research study
teams from a variety of domain areas. Aspects of rights,
obligations, permissions, and ethics must be modeled and used
to extend the ontology. Finally, axioms must be developed and
competency validation of the ICO must be conducted using a
series of still to be defined use case derived competency
questions.</p>
      <p>We have described our work on ICO, a preliminary
ontology of informed consent that provides general
classification of content contained in general informed consent
documents. It requires expansion, revisions and collaboration
to build a robust model, and to move toward a representation of
the complex area of biobank data sharing and specimen
release. We hope to collaborate with the broader community in
this effort.</p>
    </sec>
    <sec id="sec-3">
      <title>ACKNOWLEDGMENT</title>
      <p>This research was supported by a University of Michigan
interdisciplinary research award (MCubed) and by the National
Center for Advancing Translational Sciences of the National
Institutes of Health under Award Number 2UL1TR000433-06.
The content is solely the responsibility of the authors and does
not necessarily represent the official views of the National
Institutes of Health.</p>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          [1]
          <string-name>
            <given-names>A.</given-names>
            <surname>Cambon-Thomsen</surname>
          </string-name>
          ,
          <string-name>
            <given-names>E.</given-names>
            <surname>Rial-Sebbag</surname>
          </string-name>
          , and
          <string-name>
            <given-names>B. M.</given-names>
            <surname>Knoppers</surname>
          </string-name>
          ,
          <article-title>"Trends in ethical and legal frameworks for the use of human biobanks,"</article-title>
          <source>Eur Respir J</source>
          , vol.
          <volume>30</volume>
          , pp.
          <fpage>373</fpage>
          -
          <lpage>82</lpage>
          ,
          <year>Aug 2007</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          [2]
          <string-name>
            <given-names>R. E.</given-names>
            <surname>Hewitt</surname>
          </string-name>
          ,
          <article-title>"Biobanking: the foundation of personalized medicine,"</article-title>
          <source>Curr Opin Oncol</source>
          , vol.
          <volume>23</volume>
          , pp.
          <fpage>112</fpage>
          -
          <lpage>9</lpage>
          ,
          <year>Jan 2011</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          [3]
          <string-name>
            <given-names>G. E.</given-names>
            <surname>Henderson</surname>
          </string-name>
          ,
          <string-name>
            <given-names>R. J.</given-names>
            <surname>Cadigan</surname>
          </string-name>
          ,
          <string-name>
            <given-names>T. P.</given-names>
            <surname>Edwards</surname>
          </string-name>
          ,
          <string-name>
            <surname>I. Conlon</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A. G.</given-names>
            <surname>Nelson</surname>
          </string-name>
          ,
          <string-name>
            <surname>J. P.</surname>
          </string-name>
           Evans, A. M. Davis, C. Zimmer, and B. J. Weiner, “Characterizing 
          <article-title>biobank  organizations  in  the  U.S.:  results  from  a  national  survey,”  Genome Medicine</article-title>
          , vol.
          <volume>5</volume>
          , no.
          <issue>1</issue>
          , p.
          <fpage>3</fpage>
          ,
          <string-name>
            <surname>Jan</surname>
          </string-name>
          .
          <year>2013</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          [4]
          <string-name>
            <given-names>F. J.</given-names>
            <surname>Manion</surname>
          </string-name>
          ,
          <string-name>
            <given-names>R. J.</given-names>
            <surname>Robbins</surname>
          </string-name>
          ,
          <string-name>
            <given-names>W. A.</given-names>
            <surname>Weems</surname>
          </string-name>
          , and
          <string-name>
            <given-names>R. S.</given-names>
            <surname>Crowley</surname>
          </string-name>
          ,
          <article-title>"Security and privacy requirements for a multi-institutional cancer research data grid: an interview-based study,"</article-title>
          <source>BMC Med Inform Decis Mak</source>
          , vol.
          <volume>9</volume>
          , p.
          <fpage>31</fpage>
          ,
          <year>2009</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          [5]
          <string-name>
            <given-names>M.</given-names>
            <surname>Brochhausen</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M. N.</given-names>
            <surname>Fransson</surname>
          </string-name>
          ,
          <string-name>
            <given-names>N. V.</given-names>
            <surname>Kanaskar</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Eriksson</surname>
          </string-name>
          ,
          <string-name>
            <given-names>R.</given-names>
            <surname>Merino-Martinez</surname>
          </string-name>
          ,
          <string-name>
            <given-names>R. A.</given-names>
            <surname>Hall</surname>
          </string-name>
          ,
          <string-name>
            <given-names>L.</given-names>
            <surname>Norlin</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S.</given-names>
            <surname>Kjellqvist</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Hortlund</surname>
          </string-name>
          ,
          <string-name>
            <given-names>U.</given-names>
            <surname>Topaloglu</surname>
          </string-name>
          ,
          <string-name>
            <given-names>W. R.</given-names>
            <surname>Hogan</surname>
          </string-name>
          , and
          <string-name>
            <given-names>J.-E.</given-names>
            <surname>Litton</surname>
          </string-name>
          ,
          <article-title>"Developing a semantically rich ontology for the biobank-administration domain," J Biomed Semantics</article-title>
          , vol.
          <volume>4</volume>
          , p.
          <fpage>23</fpage>
          ,
          <year>2013</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          [6]
          <string-name>
            <surname>Norlin</surname>
            ,
            <given-names>M. N.</given-names>
          </string-name>
          <string-name>
            <surname>Fransson</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          <string-name>
            <surname>Eriksson</surname>
            ,
            <given-names>R.</given-names>
          </string-name>
          <string-name>
            <surname>Merino-Martinez</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          <string-name>
            <surname>Anderberg</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          <string-name>
            <surname>Kurtovic</surname>
          </string-name>
          , and J.-E.  Litton,  “A  Minimum  Data  Set  for  Sharing  Biobank  Samples,  Information,  and  Data:  MIABIS,”
          <article-title>  Biopreservation and Biobanking</article-title>
          , vol.
          <volume>10</volume>
          , no.
          <issue>4</issue>
          , pp.
          <fpage>343</fpage>
          -
          <lpage>348</lpage>
          , Aug.
          <year>2012</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          [7]
          <string-name>
            <given-names>J.</given-names>
            <surname>Zheng</surname>
          </string-name>
          , Personal Communication to Yongqun He,
          <year>2014</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          [8]
          <string-name>
            <given-names>H.</given-names>
            <surname>Ellis</surname>
          </string-name>
          , Personal Communication to Frank J.
          <string-name>
            <surname>Manion</surname>
          </string-name>
          ,
          <year>2014</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          [9]
          <string-name>
            <surname>ISBER.</surname>
          </string-name>
          (
          <year>2014</year>
          ). ISBER - International Society for Biological and
          <string-name>
            <given-names>Environmental</given-names>
            <surname>Repositories</surname>
          </string-name>
          . Available: http://www.isber.org/
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          [10]
          <string-name>
            <given-names>A.</given-names>
            <surname>Grando</surname>
          </string-name>
          and
          <string-name>
            <given-names>R.</given-names>
            <surname>Schwab</surname>
          </string-name>
          ,
          <article-title>"Building and evaluating an ontology-based tool for reasoning about consent permission,"</article-title>
          <source>AMIA Annu Symp Proc</source>
          , vol.
          <year>2013</year>
          , pp.
          <fpage>514</fpage>
          -
          <lpage>23</lpage>
          ,
          <year>2013</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref11">
        <mixed-citation>
          [11]
          <string-name>
            <given-names>J. S.</given-names>
            <surname>Obeid</surname>
          </string-name>
          ,
          <string-name>
            <given-names>K.</given-names>
            <surname>Gerken</surname>
          </string-name>
          ,
          <string-name>
            <given-names>K. C.</given-names>
            <surname>Madathil</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D.</given-names>
            <surname>Rugg</surname>
          </string-name>
          ,
          <string-name>
            <given-names>C. E.</given-names>
            <surname>Alstad</surname>
          </string-name>
          ,
          <string-name>
            <given-names>K.</given-names>
            <surname>Fryar</surname>
          </string-name>
          ,
          <string-name>
            <given-names>R.</given-names>
            <surname>Alexander</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A. K.</given-names>
            <surname>Gramopadhye</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J.</given-names>
            <surname>Moskowitz</surname>
          </string-name>
          ,
          <string-name>
            <given-names>and I. C.</given-names>
            <surname>Sanderson</surname>
          </string-name>
          , “
          <article-title>Development  of  an  Electronic  Research  Permissions  Management  System to Enhance Informed Consents</article-title>
          and Capture Research Authorizations  Data,” 
          <source>AMIA Summits Transl Sci Proc</source>
          , vol.
          <year>2013</year>
          , pp.
          <fpage>189</fpage>
          -
          <lpage>193</lpage>
          ,
          <year>18 2013</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref12">
        <mixed-citation>
          [12]
          <fpage>W3C</fpage>
          ,
          <article-title>"OWL 2 Web Ontology Language document overview," pp</article-title>
          .
          <fpage>http</fpage>
          ://www.w3.org/TR/2009/REC-owl2
          <string-name>
            <surname>-</surname>
          </string-name>
          overview-20091027
          <source>/. Accessed on March 1</source>
          ,
          <year>2014</year>
          ,
          <year>2009</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref13">
        <mixed-citation>
          [13]
          <string-name>
            <given-names>P.</given-names>
            <surname>Grenon</surname>
          </string-name>
          and
          <string-name>
            <given-names>B.</given-names>
            <surname>Smith</surname>
          </string-name>
          ,
          <article-title>"SNAP and SPAN: Towards dynamic spatial ontology,"</article-title>
          <source>Spatial Cognition and Computation</source>
          , vol.
          <volume>4</volume>
          , pp.
          <fpage>69</fpage>
          -
          <lpage>103</lpage>
          ,
          <year>2004</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref14">
        <mixed-citation>
          [14] J. Zheng, Z. Xiang, C. J. Stoeckert, and Y. He, “Ontodog:
          <article-title> a web -based ontology community view generation tool</article-title>
          ,”  Bioinformatics, vol.
          <volume>30</volume>
          , no.
          <issue>9</issue>
          , pp.
          <fpage>1340</fpage>
          -
          <lpage>1342</lpage>
          , May
          <year>2014</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref15">
        <mixed-citation>
          [15]
          <string-name>
            <given-names>Z.</given-names>
            <surname>Xiang</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Courtot</surname>
          </string-name>
          ,
          <string-name>
            <given-names>R. R.</given-names>
            <surname>Brinkman</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Ruttenberg</surname>
          </string-name>
          , and
          <string-name>
            <given-names>Y.</given-names>
            <surname>He</surname>
          </string-name>
          , “
          <article-title>OntoFox: web -based support for ontology reuse</article-title>
          ,”
          <source>  BMC Res Notes</source>
          , vol.
          <volume>3</volume>
          , p.
          <fpage>175</fpage>
          ,
          <string-name>
            <surname>Jun</surname>
          </string-name>
          .
          <year>2010</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref16">
        <mixed-citation>
          [16]
          <string-name>
            <surname>NLM</surname>
          </string-name>
          ,
          <article-title>"U.S. National Library of Medicine (NLM). Unified Medical Language System (UMLS®) Metathesaurus [version 2013AA][Internet]. Bethesda (MD): National Library of Medicine,"</article-title>
          <year>2013</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref17">
        <mixed-citation>
          [17]
          <string-name>
            <given-names>P. L.</given-names>
            <surname>Whetzel</surname>
          </string-name>
          ,
          <string-name>
            <given-names>N. F.</given-names>
            <surname>Noy</surname>
          </string-name>
          ,
          <string-name>
            <given-names>N. H.</given-names>
            <surname>Shah</surname>
          </string-name>
          ,
          <string-name>
            <given-names>P. R.</given-names>
            <surname>Alexander</surname>
          </string-name>
          ,
          <string-name>
            <given-names>C.</given-names>
            <surname>Nyulas</surname>
          </string-name>
          , T. Tudorache,  and  M.  A.  Musen,  “BioPortal:
          <article-title>  enhanced  functionality  via  new Web services from the National Center for Biomedical Ontology to access and use ontologies in software applications</article-title>
          ,” 
          <source>Nucleic Acids Res</source>
          , vol.
          <volume>39</volume>
          , no.
          <issue>Web Server issue</issue>
          , pp.
          <fpage>W541</fpage>
          -
          <lpage>W545</lpage>
          , Jul.
          <year>2011</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref18">
        <mixed-citation>
          [18]
          <string-name>
            <surname>Z.</surname>
          </string-name>
            Xiang, C. Mungall,  A.  Ruttenberg,  and  Y.  He,  “Ontobee:
          <article-title>  A  Linked  Data  Server  and  Browser  for  Ontology  Terms</article-title>
          .,”  in
          <source>  Proceedings of the 2nd International Conference on Biomedical Ontologies (ICBO)</source>
          , Buffalo,
          <string-name>
            <surname>NY</surname>
          </string-name>
          , USA,
          <year>2011</year>
          , pp.
          <fpage>279</fpage>
          -
          <lpage>281</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref19">
        <mixed-citation>
          [19]
          <string-name>
            <surname>D.</surname>
          </string-name>
            B.  Fridsma,  J.  Evans,  and  C.  N.  Mead,  “
          <source>The  BRIDG  Project:  A  Technical  Report,”  Journal of the American Medical Informatics Association</source>
          , vol.
          <volume>15</volume>
          , no.
          <issue>2</issue>
          , p.
          <fpage>130</fpage>
          ,
          <year>2008</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref20">
        <mixed-citation>
          [20]
          <string-name>
            <given-names>I.</given-names>
            <surname>Sim</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S. W.</given-names>
            <surname>Tu</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S.</given-names>
            <surname>Carini</surname>
          </string-name>
          ,
          <string-name>
            <given-names>H. P.</given-names>
            <surname>Lehmann</surname>
          </string-name>
          ,
          <string-name>
            <given-names>B. H.</given-names>
            <surname>Pollock</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Peleg</surname>
          </string-name>
          ,
          <article-title>and</article-title>
          K.  M.  Wittkowski,  “The  Ontology  of  Clinical  Research  (OCRe):
          <article-title>  An  informatics  foundation  for  the  science</article-title>
            of  clinical  research,” 
          <source>Journal of Biomedical Informatics, Nov 13</source>
          <year>2013</year>
          .
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>