<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta />
    <article-meta>
      <title-group>
        <article-title>ICDO: Ontological representation of the International Classification of Diseases (ICD) and its application in English and Chinese healthy data standardization</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Ling Wan</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
          <xref ref-type="aff" rid="aff1">1</xref>
          <xref ref-type="aff" rid="aff2">2</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Edison Ong</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
          <xref ref-type="aff" rid="aff2">2</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Yongqun He</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
          <xref ref-type="aff" rid="aff2">2</xref>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          <institution>National Standard V.</institution>
          <addr-line>1.0</addr-line>
        </aff>
        <aff id="aff1">
          <label>1</label>
          <institution>OntoWise</institution>
          ,
          <addr-line>Nanjing, Jiangsu</addr-line>
          ,
          <country country="CN">China</country>
        </aff>
        <aff id="aff2">
          <label>2</label>
          <institution>University of Michigan Medical School</institution>
          ,
          <addr-line>Ann Arbor, MI 48109</addr-line>
          ,
          <country country="US">USA</country>
        </aff>
      </contrib-group>
      <abstract>
        <p>The ICD-9/10/11 are released by the WHO and used worldwide to support applications including health insurance classification. However, different countries develop their own modifications of the ICD system and these versions are often incompatible. In addition, the semantic relations among ICD disease terms are unclear, and how these terms are related to other entities such as anatomic entities are not defined. To address these issues, we developed an ICD ontology (ICDO) to logically represent ICD terms and their relations with anatomic entities, qualities, etc. Different from other disease ontologies, all ICD diseases are defined disease processes in ICDO. The current ICDO focuses on English and Chinese representation. As a use case, we used ICDO to integrate ICD related data from 33 regions in Jiangsu province in China. Our strategy was able to identify and standardize local ICD versions in these regions.</p>
      </abstract>
      <kwd-group>
        <kwd>ICD</kwd>
        <kwd>ontology</kwd>
        <kwd>disease standardization</kwd>
      </kwd-group>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>Introduction</title>
      <p>
        The International Classification of Diseases (ICD), maintained
by the World Health Organization (WHO), is the international
standard for reporting diseases and health conditions. It is the
diagnostic classification standard for all clinical and research
purposes. ICD defines diseases, disorders, injuries and other
related health conditions in the biomedical and clinical domains
in a comprehensive and hierarchical fashion. The ICD has been
continuously revised and published in a series of editions to
reflect advances in health and medical science over time (
        <xref ref-type="bibr" rid="ref1 ref2">1,2</xref>
        ).
ICD is the foundation for the identification of health trends and
statistics in a global setting.
      </p>
      <p>
        Many countries have adopted the ICD standard and developed
their own modified versions. For example, there are the USA
version of ICD-10-CM (
        <xref ref-type="bibr" rid="ref3">3</xref>
        ) and Germany version of
ICD-10GM (
        <xref ref-type="bibr" rid="ref4">4</xref>
        ). In China, there are different formats including National
Standard V.1.1, GB/T14396-2016 and National Clinical
Version 1.1 (Table 1). The availability of so many versions makes
it difficult to standardize health records in China. This study
focuses on the GB/T14396-2016, which is the ICD10 Chinese
version authorized by the national administrative. Recently
WHO released the ICD-11, which is the latest version of ICD
and China reported to adapt the ICD11 version in 2019.
      </p>
      <sec id="sec-1-1">
        <title>ICD10 versions in China</title>
        <sec id="sec-1-1-1">
          <title>National StandardV.1.1</title>
          <p>GB/T 14396-2016</p>
        </sec>
        <sec id="sec-1-1-2">
          <title>National Clinical Version1.1</title>
        </sec>
        <sec id="sec-1-1-3">
          <title>Beijing Clinical versionICD-10 V6.01 National Standard V.1.0</title>
        </sec>
        <sec id="sec-1-1-4">
          <title>ICD10（2011modification）</title>
        </sec>
        <sec id="sec-1-1-5">
          <title>National ICD10 V1.3</title>
        </sec>
        <sec id="sec-1-1-6">
          <title>Shanghai ICD-10(2013 updated)</title>
        </sec>
        <sec id="sec-1-1-7">
          <title>National RC020-1CD-10 Diagnostic code Beijing version RC020-1CD-10 Diagnostic code Beijing ICD10 V5.0</title>
          <p>Guangdong ICD-10 ( 2017）</p>
        </sec>
        <sec id="sec-1-1-8">
          <title>National clinical ICD10 V.1.0</title>
          <p>Ji’nan city,Shandong province ICD10 (For Health</p>
          <p>information data sharing and exchange )</p>
          <p>Note: This source of this table comes from a survey by
OAMAHA:http://www.sohu.com/a/302897591_324186,
2019</p>
          <p>03-21, which is translated by author.</p>
          <p>
            The ICD is used as the controlled terminology of diseases in the
medical information platform in most healthcare
administrations. There are many application systems that exist
in hospitals, such as: HIS (health information systems) (
            <xref ref-type="bibr" rid="ref5">5</xref>
            ), LIS
(laboratory information system) (
            <xref ref-type="bibr" rid="ref6">6</xref>
            ), PACS (A picture archiving
and communication system), the EMR (Electronic medical
records). These data can be integrated by the ICD framework.
          </p>
          <p>
            Copyright © 2019 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).
On the other hand, both ICD codes and Diagnosis-related
Groups (DRGs) are a major method for medical insurance
control and the DRGs is dependent on the correctness of ICD
(
            <xref ref-type="bibr" rid="ref7">7</xref>
            ). Due to its important role in many medical and clinical
fields, a large amout of mapping effort is required to ensure
interoperability among different ICD versions.
          </p>
          <p>
            The semantic mapping among databases generated under two
different coding systems (e.g., ICD10 and ICD11) is very
difficult and generally requires manual intervention. The National
Institutes of Health (NIH) refer such difficulty to the
phenomenon of ‘data wrangling’ encompassing activities that make data
more usable by changing their forms but not their meanings (
            <xref ref-type="bibr" rid="ref8">8</xref>
            ).
          </p>
          <p>Although great efforts have been made on this area, the obstacle
still exists. The ICD terminology is composed of a code/value
pair. Each ICD standard code corresponds to a unique disease
name as a value. However, in reality, there are often multiple
synonyms expressed for one disease in the natural language.</p>
          <p>For example, the ICD 11 code AA0Z has value of Infectious
diseases of external ear, unspecified; the GB/T14396-2016
code H60.001 has value of 外耳疖 (external ear furuncle); the
ICD 10 code H60.5 has value of acute otitis externa,
noninfective. Due to the existence of polysemy in natural language
(especially in Chinese), the code-value mapping often encounters
ambiguity after using Extraction-Transfer-Load (ETL) tool for
data integration, and results in improper matching. Particularly
in China, these problems are mainly due to the different local
ICD versions with private extensions to certain ICD terms.</p>
          <p>These modifications are made according to the internal clinical
needs coming from different medical units. The large
discrepancy among different versions might cause many problems,
such as the appearance of the large amount of data with
different values but the same code, or the same value with different
codes. This also affects the accuracy of ICD-based DRG
grouping, the accuracy of Medicare payments as well as the statistics
accuracy of death causes.</p>
          <p>
            In addition to the ICD, there are many disease discription
models being developed and used. Hadzic et al. classify disease
into four dimensions: (i) generic disease types; (ii) phenotypes
that are mainly based on observations to describe the various
symptoms of the disease; (iii) etiology that is a strictly scientific
basis of pathogenic factors, mainly including two categories
genetic factors and environmental factors; (iv) treatment that is
a possible effective measures against a particular disease (
            <xref ref-type="bibr" rid="ref9">9</xref>
            ).
          </p>
          <p>
            These four dimensions together can describe the overall
knowledge of a disease field. Fang et al. learned from the
classification of SNOMED (
            <xref ref-type="bibr" rid="ref10 ref11">10,11</xref>
            ) and ICD to improve and
make a new disease description model. On the basis of the axis,
the general disease discription model of Hadzic was improved,
and two basic characteristics of complications and detection
methods were added, and the symptoms, signs, staging, sex,
age, acute and chronic and onset time were classified as clinical
manifestations (
            <xref ref-type="bibr" rid="ref12">12</xref>
            ).
          </p>
          <p>
            Ontology is likely the best approach to solve the issue of
semantic mapping among different databases and terminology
systems. A formal biomedical ontology is a set of computer and
human-interpretable terms that represent entities and relations
among the entities in a biomedical domain. Ontologies have
emerged to be critical to biomedical and clinical data
standardization, management, integration, and analysis. Two
different databases or terminologies may be formed based on
different organizational principles and are unlikely or difficult
to form an agreement about what each piece of information
refers to and how they can be aligned. The inability to achieving
interoperability can severely compromise the goals of
information integration and aggregation. Such issue is difficult
to solve internally or among the two databases (
            <xref ref-type="bibr" rid="ref8">8</xref>
            ). However,
the usage of community-based and consensus-based ontologies
provides a feasible way to solve the term mapping and
information integratoin issues.
          </p>
          <p>
            Many disease-related ontologies exist, including Human
Disease Ontology (DOID) (
            <xref ref-type="bibr" rid="ref13 ref14">13,14</xref>
            ), Monarch Disease Ontology
(MONDO) (
            <xref ref-type="bibr" rid="ref15">15</xref>
            ), and the Ontology of General Medical Science
(OGMS) (
            <xref ref-type="bibr" rid="ref16">16</xref>
            ). In DOID and MONDO, diseases are treated as
disposition, which is a realizable entity that bears in some
material entity and can be realized in a life process (
            <xref ref-type="bibr" rid="ref8">8</xref>
            ). However,
in the setting of ICD usage, diseases have already occurred and
are not disposition per se. OGMS includes two high level terms:
disease and ‘disease course’, where disease is asserted as a
disposition and ‘disease course’ as a process.
          </p>
          <p>
            To find a semantic mapping method between different ICD
versions, here we report the development of an ICD ontology
(ICDO) to address the issues of database interoperability and
data integration as listed above. Given that ICD is mainly
applicable to statistical analysis and disease grouping for
healthcare insurance, we present in this paper our disease
design pattern that combines the advantages of the above
disease description models. Our disease design pattern in ICDO
is based on the understanding that the disease in ICD is a human
pathological process that realizes disease disposition. Such
process is composed of a group of entities, which has reversible
decomposition. These entity are ‘anatomical structure’,
‘pathological anatomical entity’, ‘etiology’, ‘quality’ and
‘syndrome’. Therefore, all the ICD terms are defined as
subclasses of the ICDO “disease process” class, which is then
defined as a subclass of the imported OGMS term ‘pathological
bodily process’ (
            <xref ref-type="bibr" rid="ref16">16</xref>
            ). In this manuscript, we detail our ICDO
developmental strategy and provide a comprehensive use case
to illustrate the usage of the ICDO.
          </p>
        </sec>
      </sec>
    </sec>
    <sec id="sec-2">
      <title>Methods</title>
      <sec id="sec-2-1">
        <title>General ICDO development strategy</title>
        <p>
          Our ICDO development closely followed the WHO ICD 10/11
classification and principles. The ICDO development used the
eXtensive Ontology Development (XOD) strategy (
          <xref ref-type="bibr" rid="ref17">17</xref>
          ), which
emphasizes the reuse and alignment of ontology terms and
semantic relations, ontology design patterns, and community
effort. Specifically, we aligned the ICDO terms with Basic
Formal Ontology (BFO) and BFO-compatible ontologies (
          <xref ref-type="bibr" rid="ref8">8</xref>
          ).
Ontofox (
          <xref ref-type="bibr" rid="ref18">18</xref>
          ) was used to extact terms from existing ontologies
that were then imported and reused in ICDO.
        </p>
        <p>We focused our first stage ICDO development on the specific
area of external ear diseases as a proof-of-concept. This early
stage ICDO prototype includes all diseases related to external
ear part in: (i) ICD11 under the class, “Disease of the ear and
mastoid process”, coded from AA00 to AA6Z, (ii) ICD10 under
the “external ear diseases”, and (iii) GB/T 14396-2016.
The Protégé OWL editor (http://protege.stanford.edu) was used
to visualize ICDO, add new ICDO terms, edit imported terms
and merge imported ontologies. ICDO-specific terms were
generated using new ICDO identifiers with the prefix “ICDO_”
followed by 7-digit auto incremented numbers. The Hermit
reasoner was used for consistency checking and reasoning
(http://hermit-reasoner.com/).</p>
      </sec>
      <sec id="sec-2-2">
        <title>ICDO format, source code, and deposition</title>
        <p>ICDO is expressed using the W3C standard Web On-tology
Language (OWL2) (http://www.w3.org/TR/owl-guide/). The
current ICDO source code is openly available at GitHub:
http://github.com/icdo/ICDO.</p>
        <p>
          The ICDO ontology is deposited in the NCBO BioPortal
website: https://bioportal.bioontology.org/ontologies/ICDO,
as well as the ontology repository website Ontobee (
          <xref ref-type="bibr" rid="ref19">19</xref>
          ):
http://www.ontobee.org/ontology/ICDO.
        </p>
      </sec>
      <sec id="sec-2-3">
        <title>Application of ICDO for mapping and standardizing different versions of disease classifications</title>
        <p>The health systems in several regions of the Jiangsu Province
in China used different modified versions of ICD10. To
standardize the coding systems from these regional health
information platforms, we identified three regional ICD10
modification coding systems, and used ICDO to model and
standardize these coding systems.</p>
      </sec>
      <sec id="sec-2-4">
        <title>ICDO query and analysis</title>
        <p>Description Logic (DL) query was used to query the knowledge
built in ICDO. The DL query function in the Protégé-OWL
editor was used for the implementation.</p>
      </sec>
    </sec>
    <sec id="sec-3">
      <title>Results</title>
      <sec id="sec-3-1">
        <title>General disease definition of disease development strategy</title>
        <p>
          First we performed a survey of how the term “disease” is
defined in different ontologies and dictionaries (Table 1). It is
clear that the nature of disease is defined differntly. In four
ontologies including DOID, OGMS (
          <xref ref-type="bibr" rid="ref16">16</xref>
          ), MONDO, and EFO
(Experimental Factor Ontology) (
          <xref ref-type="bibr" rid="ref20 ref21">20,21</xref>
          ), disease are all defined
as a disposition. In the Semanticscience Integrated Ontology
(SIO) (
          <xref ref-type="bibr" rid="ref22">22</xref>
          ), disease is defined as an outward manifestatoin of
one or more disorders. Disease has also been defined as a
disorder by itself or pattern of abnormality (Table 1).
        </p>
        <p>In OGMS, there are two disease-related terms ‘disease course’,
and ‘pathological bodily process’. The term ‘disease course is
defined as “The totality of all processes through which a given
disease instance is realized”. However, it is unclear what the
“all processes” in the definition means. It is possibly that some
of the processes are not directly related to disease. The OGMS
term ‘pathological bodily process’ is defined as “A bodily
process that is clinically abnormal”. The diseases listed in
ICDO have already happened, and are not an upcoming event.
Given that the ICD is used primarily for post-disease recording
and insurance filing purposes, we think that the disease in ICD
is primarily meant to be a type of pathological bodily process;
therefore, the disease in ICD can be better regarded as a
“disease process” under OGMS ‘pathological bodily process’.
In ICDO, based on the nature of ICD and its applications, we
focus on the representation of disease processes instead.
Therefore, the term ‘disease process’ becomes our major term,
which is defined in ICDO as follows:</p>
        <p>Disease process =def. a pathological bodily process
that occurs in a specific anatomic location, realizes a
disease disposition, has abnormal bodily phenotype,
and results in a pathological anotomic entity.</p>
        <p>Therefore, all the specific diseases in ICDO are all defined as
disease processes, which are different from other disease
description frameworks. As a result, ICDO represents all
disease names from ICD11, ICD10, GB/T14396 as disease
processes, often abbreviaetd with the suffix “DP” in ICDO term
labels.</p>
        <p>In this study, ICDO is mainly used to support data
standardization among different ICD versions. ICDO aims to
standardize clinical data from international multi-center and
also data generated under different ICD local and modified
versions in China. To support the general interoperability goal,
we have included ICD10 and ICD11 terms in both English and
Chinese languages in the ICDO.</p>
      </sec>
      <sec id="sec-3-2">
        <title>ICDO top level design and structure</title>
        <p>
          ICD 10/11 has different classification and principles in top level
design and therefore we closely have followed OBO to develop
ICDO top level hierarchy. Extending from the formal definition
and classificaiton of this ICDO “disease process” term, we
generated an upper level ICDO hierarchical structure (Fig. 1).
ICDO reused many terms from existing ontologies such as the
BFO (
          <xref ref-type="bibr" rid="ref8">8</xref>
          ), OGMS (
          <xref ref-type="bibr" rid="ref16">16</xref>
          ), UBERON (
          <xref ref-type="bibr" rid="ref24">24</xref>
          ), PATO (Phenotype And
Trait Ontology, an ontology of phenotypic qualities (properties,
attributes or characteristics),
https://github.com/pato-ontology/pato/). The top-level terms were aligned with BFO.
Fig 1. ICDO top level hierarchical structure.
        </p>
      </sec>
      <sec id="sec-3-3">
        <title>ICDO general design pattern for diseases</title>
        <p>In ICDO, a disease process was composed of four major
elements: etiology entity, quality, anatomical stucture and
pathological anatomical entity. The disease pattern of ICDO
was shown in Fig. 2:</p>
        <p>Fig 2. ICDO disease process pattern
We represented different diease processes following the
disease process pattern (Fig. 2). For example, the ICD term
‘granuloma of external ear canal’ is defined in ICDO as
“granuloma of external ear canal DP”, which is a granuloma
disease process that “occurs in” some “external ear canal” and
“has disease output” some “granuloma”. The “necrotic
external ear otitis” is an otitis disease process “occurs in” some
“external ear” and “has quality” some “necrotic”. Note that
the ‘has quality’ is indeed a shortcut relation where the quality
is not the quality of the process per se. Instead, the quality is the
quality of the anatomical entity of the patient.</p>
        <p>ICDO also has different design strategy for diseases compared
to DOID and MONDO. In general, DOID and MONDO do not
discomposed a disease term into different entity components
such as etiology entity, quality and anatomical structure. We
paid a lot of attention to each of these issues and developed our
specific strategeis. In addition, due to the nature of ICD usage
in clinical disease classificaiton and insurance filing, we have
designed many special design patterns for the ICDO generation.</p>
        <p>Some of the special design patterns, together with the
approaches proposed in ICDO, are described in next session.</p>
        <p>Fig. 2. Disease process modeling in ICDO. In this example, the term ‘obstructive keratosis of external ear DP’ is represented
using the design pattern, including disease quality, occurs in location, and disease material output.</p>
      </sec>
      <sec id="sec-3-4">
        <title>ICDO strategy to represent special ICD10 disease classes</title>
        <p>Besides general disease classifications, ICD includes many
special terms such as “classified elsewhere”, “other specified”
and “unspecified”. ICDO has implemented special strategies to
handle the mentioned special terms.</p>
        <p>The first special terms, “classified elsewhere”, were treated as
obsoleted terms in ICDO. We believe that the disease
classification must be clear and consistent among various disease
categories. The definition of “classified elsewhere” is confusing
because there is no obvious and proper disease category for
“elsewhere”. To ensure the classification integrity, a disease
term can be classified under multiple disease categories based
on varies definitions and applications, but it should not be
classified as an undefined category, “elsewhere”. To balance the
mapping process among various ICD versions and proper
handling of the undefined category, we added all the ‘disease
classified elsewhere’ terms in ICDO but made them as obsoleted
terms in the ontology.</p>
        <p>There are also many ICD terms labeled as “other specified”.
Logically speaking, all ICD terms should be classified into
specific classes and there should not exist any ‘other’ class. We
consider this type of “other specified” terms class as logical
error and put all the terms under this class into their parent class.
In other words, we generated an ICDO term “other specified”
and put it under the obsolete to support mapping among to
existing ICD versions. To ensure the continuity of the various
versions of the ICD in the conversion adaptation process, in the
data adaptation process, this obsolete class term may still
participate in the operation to ensure the accuracy of data mapping.
Many ICD terms are labeled as “unspecified”. For their
corresponding classes, we have been able to determine the parent
classes. However, due to the limitations of current definitions
and the lack of knowledge, these “unspecified” terms have not
given any specific description as of now, and can be mapped to
their parent terms in ICDO.</p>
        <p>The ICD10/11 has the Extension Codes used to support clinical
treatment, such as the organ laterality in different ICD versions
and the “special anatomy” in ICD11. The laterality commonly
found in various ICD versions includes “left”, “right”,
“bilateral”, “unilateral, unspecified”, “unspecified laterality”.
Additionally, ICD11 introduced a term “special anatomy”,
which includes anatomical synonyms and possible anatomical
structure of disease. We built the laterality as the quality of the
disease in ICDO, and adopted the synonym of the specific
anatomy as a synonym label in annotation in anatomical
structure if UBERON have not included the synonym. Other
extension codes exist such as distribution and regional. For
example, abscess of right external ear DP can be defined to have
axiom assertion of:
“abscess of external ear DP” and (“occurs in
anatomical side” some “right side of anatomical
entity”)
Such design properly handles the issue of anatomic literality.
With the focus of disease process, ICDO also has a natural
advantage of defining different disease stages, or the beginning,
middle, and end of a disease process. Such a process aspect
supports real life disease representation.</p>
      </sec>
      <sec id="sec-3-5">
        <title>ICDO mapping process and use case application</title>
        <p>The development of ICDO starts from designing the disease
pattern first, then decomposing the terms into different
components of the semantic equivalence terms. Then we used
the Ontobee annotator (http://www.ontobee.org/annotate) to
discompose these disease terms into different components and
tag the components according to our disease design pattern
(Fig. 2). Finally, we established relationships between the
components of the disease terms by creating objective
properties and annotations following the disease desgin pattern.</p>
        <p>This entire process involves the decomposition of a disease
name and then establish the logical relations among the
components. (Fig. 3).</p>
        <p>In order to illustrate the process, we extracted terms from three
local ICD10 versions genereated by two districts and one city
(Pukou district, Liuhe district and Jiangyin city) in Jiangsu
province, China, and performed the mapping. The China
administrative departments from different regions often publish
and adopt their own ICD versions. Therefore, the
harmonization of the ICD in China is a complex and difficult
issue due to varies local versions and custom modifications. As
shown in Table 1, there are 14 different ICD local and modified
versions in China. Among them, GB/T-14396-2016 is a
Chinese version of ICD10 modification required by the Chinese
government since February 2017. Some local versions listed in
Table 1 are used for clinical purposes and the others used for
adminstrative statistics purposes. For example, in Jiangsu
province in China, we identified more than 30 regional datasets
but they used different locally modifed ICD versions. Even
though the Jiangsu province has the most advanced health
informatics system among all the provinces in China, many
incorrect code-value pairs existed in these local coding systems
and need to be mapped to the GB/T-14396-2016 coding system.</p>
        <p>(i) Same code but different values: For example, in the local
Liuhe district coding system, the two Chinese disease term “坏
死 性 外 耳 炎 ” and “ 恶 性 外 耳 炎 ” (English translation:
“necrotic external ear otitis” and “malignant otitis externa”
respectively) have the same code H60.200. However, the code
H60.200 corresponds to the “malignant otitis externa by ICD10
or GB/T14396-2016, and the term “necrotic external ear otitis”
should be coded as H60.900 (“otitis, externa”).</p>
        <p>(ii) Same values but different codes: For exmaple , in the
local Jiangyin city coding system, the Chinese disease term “后
天性外耳畸形” (English translation: “acquired deformity of
ear externa”) has two codes: H61.303 and H61.101. However,
according to ICD10 or GB/T14396-2016, the correct code of
this disease name should be H61.101. The code H61.303 even
does not exist in ICD10 or GB/T14396-2016.</p>
        <p>These two types of errors shown above widely exist in the local
Chinese ICD modified versions. For example, even for the
same external ear branch, there are 36 errors in Pukou district,
4 errors in Liuhe disctrict and 14 errors in Jiangyin city local
coding systems. Note that there are only 22 code-value pairs in
ICD10 and 41 code-value pairs in GB/T14396-2016 for the
external ear-related diseases. Considering the total number of
over 20,000 terms in ICD10 and 14 different local ICD10
versions in China (Table 2), it is a huge effort to manually
correct these code-value pair errors. the pair errors have become
a major issue to support data integration and systematic
statistical data analysis in China.</p>
        <p>In this study, we developed an ICDO-based semantic disease
name mapping algorithm (ISDNA), as shown in Fig. 4, with the
aim to solve the disease name mapping issue as illustrated in
the use cases of Jiangsu Province, China. First, the ISDNA
algorithm first accepted some disease names in a specific
langugae (e.g., English, Chinese) as input. The input names
were then decomposed via natual language process (NLP) to
different components using Ontobee Annotator. For example, a
disease name could be broken down into the anatomic entity
term as the location of the disease process, quality of the patient
or the anatomic bodily entity of the patient, and abnormal
pathological entity as the output of the disease. These identified
components were then mapped to their corresponding ontology
terms and IDs. Based on the axioms defined in ICDO, we can
use ontology reasoners to automatically infer the ICDO codes
that the input disease name belongs to (Fig. 5). Our ISDNA
algorithm is able to identify exact matches that perfectly
mapped or semantically inferred the parent terms of the
matched disease names from the Pukou district, Liuhe district
and Jiangyin city in Jiangsu province, China to the
corresponding ICDO terms. Next we will provide two examples
to illustrate the features and performance of the ISDNA
algorithm.
Fig. 5. provides the first example of how we can infer a specific
name to a perfectly matched ICDO term. Specifically, in this
use case, “cellulitis of external ear” (Chinese name: “外耳蜂窝
织 炎 ”) is an input disease term. It was first split into two
components: “cellulitis anatomic entity” and “external ear”.</p>
        <p>Given the nature of these two terms, the following two axioms
could be assigned:
‘occurs in’ some ‘external ear’
‘has disease output’ some ‘cellulitis anatomic entity’
Based on these two axioms, the ICDO reasoner was able to infer
“cellulitis of external ear” as an exact match to the ICDO
“cellulitis of external ear DP” (Chinese name “外耳蜂窝织
炎”) term with the GB/T14396 code H60.100, ICD10 code
H60.1, and ICD11code AA01. Then we can select one of the
code from them according to our needs.</p>
        <p>Fig. 6. provides another example that has the input disease
name “Pinna defect after burn” (Chinese name: “烧伤后耳廓
缺损”), a term from the Pukou district local coding system in
our use case. Similarly, our algorithm started with spliting the
long disease name to three components: “pinna”, “defect”, and
“after burn”, which could then be mapped to their
corresponding ontology terms in ICDO. Note that the term
“after burn” is an defined as an synonym of the “acquired after
burn”, which is a subclass of the quality “acquired”. The term
“Pinna defect after burn” is not included in either ICD or ICDO.</p>
        <p>But for demonstrative purpose, the term “Pinna defect after
burn” was introducted (Fig. 6) to simulate the situation that
there is no perfect matching. After direct mapping, there was no
exact match for this disease name. However, after running the
Hermit reasoner available in the Protégé OWL editor, we were
able to infer this disease name to be subclass of the ICDO
“acquired deformity of pinna DP” term (Chinese name: “获得
性耳廓畸形”) with ICD11 code AA41. Given that there was
no exact match for “Pinna defect after burn”, the ICDO
“acquired deformity of pinna DP” term was defined as the
preferred semantically matched ICDO term for the input
disease name.</p>
        <p>Fig.5. ISDNA inferred terms exact mapping include different codes and languages come from different codeing systems.The
candidate term “cellulitis of external ear”will be discomposed into components as “cellulitis DP” and “external ear” after NLP
first.Then they are mapped to respect terms in ICDO according to the dimensions disgned by disease pattern as “cellulitis DP”
and “external ear”.Finally inferred to “cellulitis of external ear DP” by axioms in ICDO .User can selects different ICD code by
application requirement.</p>
        <p>Note that in the above two examples, the NLP process was
preformed manually. In the future, we plan to develop an
automatic NLP process to achieve the same NLP results, which
is not within the scope of current study.</p>
      </sec>
      <sec id="sec-3-6">
        <title>ICDO query and analysis</title>
        <p>In this case, we demonstrate how to use the Description Logic
(DL) query in the Protege-OWL editor to identify from ICDO
specific diseases that occurs in the external ear canal, or called
external acoustic meatus. Basically, this DL query identified
those diseases that meet this axiom requirement:
“occurs in” some “external acoustic meatus”
The “external acoustic meatus” is the formal anantomical
structure term in UBERON and has synonam “external ear
canal” in ICDO.</p>
        <p>As shown in Fig. 7, a total of 14 diseases were identified,
including 4 testing terms in Chinese. This example shows that
ICDO is able to serve as a platform and knowledge system for
computational programs like DL query to perform semantic
query and analysis.</p>
        <p>Fig. 6. ISDNA mapping of a candidate term to ICD standard term and code. In this example, the term ‘Pinna defect after burn’
(Chinese name: “烧伤后耳廓缺损”) was used as the input. The term was decomposed into three components “after burn”,
“pinna”, and “defect”, where were then mapped to ‘acquired from burn’, pinna, and defect in ICDO, respectively. These three
ICDO terms provide the quality, location of the disease process, and the output of the disease process term. Based on the axiom
definition, our ontology reasoner was able to match this name to ‘acquired deformity of pinna DP’ (AA41 in ICD11). Note that
this term is not an exact match.</p>
        <p>Fig. 7. DL query of ICDO looking for all disease “occurs in external acoustic meatus”. This query was performed using DL
query in Protege-OWL editor 5.2 (http://protege.stanford.edu/).</p>
      </sec>
    </sec>
    <sec id="sec-4">
      <title>Discussion</title>
      <p>In this manuscript, we presented our development of the ICDO
ontology with the aim to standardize ICD disease records and
support health record integration and analysis. We also proposed
and tested a semantic analysis based on ICDO using the function
of reasoner. It realized the interpretation of terms at the semantic
level by reasoner between entities by axioms. ICDO improves the
mapping accuracy, supports exact and semantically preferred
mapping. and provides a useful application in terms of the
standardization of heterogeneous data between different ICD
versions.</p>
      <p>Our use case focused on the different ICD10 local versions used
in some local adminstrative healthcare information plateform in
Jiangsu province China. Not every disease has a clear physical
product "disease output" when entering clinical observation. For
example, inflammation is an immune with multiple symptoms
and are sometimes difficult to fully express in natural language.
However, the physical entity of this inflammation is clearly
present as specific anatomy structure in ontology. In ICDO we
defined the output of inflammation process with “inflammatory
anatomical entity” and asserted axiom in the form of “physical
pathological object” “occurs in” some “anatomical structure”
“caused by” “inflammation process”.</p>
      <p>In clinical practice, our disease pattern can cover most disease
types. Particularly there is a class of diseases which does not have
pathological abnormalities in specific anatomical structures but
have systemic symptoms. We designed a special dimension
“syndrome” in ICDO disease pattern for this class of diseases. In
the current stage of ICDO with the focus of external ear disease,
there is no syndrome included. However, this situation will be
carefully examined and appropriately handled in the future when
we extend the ICDO to fullly cover all disease in ICD10 or ICD
11.</p>
      <p>While many ICD terms can be clearly defined as disease
processes, there may be many concerns in terms of using disease
process for other scenarios. For example, the disease process may
not be able to represent a disease output such as the size or mass
of a tumor like external ear canal tumor. However, in this case,
we can semantically define a disease process like ‘external ear
canal tumor disease process’ that ‘has output’ of some tumor that
‘has size’ or ‘has mass’ of some specific values. Using this
strategy, we can semantically link the disease process to the
physical tumor (or other anatomic entity) and its qualities like size
or mass. Another concern is that a disease process may be
diagnosed before the result of the process becomes manifested.
Note that ICD is typically used to represent the health outputs
rather than unidentified or undiagnosed health issues. The fact
that a disease is not diagnosed does not mean that the disease
process does not occur. If the disease is not diagnosed, we may
not be able to use the disease process term; however, it may not
be necessary to use according to the ICD guidelines.</p>
    </sec>
    <sec id="sec-5">
      <title>Conclusions</title>
      <p>Ontology is clearly a very good tool for solving the problem of
semantic mapping between different ICD versions. ICDO will
improve the usability and interoperability among various ICD
systems. ICDO can also be used for data standardization and analysis
of international multi-center clinical trials between different
languages in different counties, data normalization processing before
DGRs grouping, data normalization and in hospital internal
information systems, and data standardization for regional health
information platform. The disease design pattern in ICDO can
provide effective contributions to the medical data mining and
retrospective researches.</p>
    </sec>
    <sec id="sec-6">
      <title>Acknowledgements</title>
      <p>We appreciate the discussion and editorial revision by Ms. Meng
Liu.</p>
    </sec>
    <sec id="sec-7">
      <title>Address for correspondence</title>
      <p>LW and YH are co-corresponding authors. Their emails addresses
are wanlingeric@qq.com and yongqunh@med.umich.edu.</p>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          1.
          <string-name>
            <surname>Percy</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Holten</surname>
            ,
            <given-names>V.v.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Muir</surname>
            ,
            <given-names>C.S.</given-names>
          </string-name>
          and
          <string-name>
            <surname>Organization</surname>
            ,
            <given-names>W.H.</given-names>
          </string-name>
          (
          <year>1990</year>
          )
          <article-title>International classification of diseases for oncology.</article-title>
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          2.
          <string-name>
            <surname>Trott</surname>
            ,
            <given-names>P.</given-names>
          </string-name>
          (
          <year>1977</year>
          )
          <article-title>International classification of diseases for oncology</article-title>
          .
          <source>Journal of clinical pathology</source>
          ,
          <volume>30</volume>
          ,
          <fpage>782</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          3.
          <string-name>
            <surname>Cao</surname>
            ,
            <given-names>L.</given-names>
          </string-name>
          and
          <string-name>
            <surname>Morley</surname>
            ,
            <given-names>J.E.</given-names>
          </string-name>
          (
          <year>2016</year>
          )
          <article-title>Sarcopenia is recognized as an independent condition by an international classification of disease, tenth revision, clinical modification (ICD-10-CM) code</article-title>
          .
          <source>Journal of the American Medical Directors Association</source>
          ,
          <volume>17</volume>
          ,
          <fpage>675</fpage>
          -
          <lpage>677</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          4.
          <string-name>
            <surname>Dilling</surname>
            ,
            <given-names>H.</given-names>
          </string-name>
          <article-title>and</article-title>
          <string-name>
            <surname>Freyberger</surname>
            ,
            <given-names>H.J.</given-names>
          </string-name>
          (
          <year>2012</year>
          )
          <article-title>Taschenführer zur ICD-10-Klassifikation psychischer Störungen</article-title>
          . Bern (Huber).
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          5.
          <string-name>
            <surname>Haux</surname>
            ,
            <given-names>R.</given-names>
          </string-name>
          (
          <year>2006</year>
          )
          <article-title>Health information systems-past, present, future</article-title>
          .
          <source>International journal of medical informatics</source>
          ,
          <volume>75</volume>
          ,
          <fpage>268</fpage>
          -
          <lpage>281</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          6.
          <string-name>
            <surname>Vermeer</surname>
            ,
            <given-names>H.J.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Thomassen</surname>
          </string-name>
          , E. and
          <string-name>
            <surname>de Jonge</surname>
            ,
            <given-names>N.</given-names>
          </string-name>
          (
          <year>2005</year>
          )
          <article-title>Automated processing of serum indices used for interference detection by the laboratory information system</article-title>
          .
          <source>Clinical chemistry</source>
          ,
          <volume>51</volume>
          ,
          <fpage>244</fpage>
          -
          <lpage>247</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          7.
          <string-name>
            <surname>Aiello</surname>
            ,
            <given-names>F.A.</given-names>
          </string-name>
          and
          <string-name>
            <surname>Roddy</surname>
            ,
            <given-names>S.P.</given-names>
          </string-name>
          (
          <year>2017</year>
          )
          <article-title>Inpatient coding and the diagnosis-related group</article-title>
          .
          <source>Journal of vascular surgery</source>
          ,
          <volume>66</volume>
          ,
          <fpage>1621</fpage>
          -
          <lpage>1623</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          8.
          <string-name>
            <surname>Arp</surname>
            ,
            <given-names>R.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Smith</surname>
            ,
            <given-names>B.</given-names>
          </string-name>
          and
          <string-name>
            <surname>Spear</surname>
            ,
            <given-names>A.D.</given-names>
          </string-name>
          (
          <year>2015</year>
          )
          <article-title>Building ontologies with basic formal ontology</article-title>
          . Mit Press.
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          9.
          <string-name>
            <surname>Hadzic</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          and
          <string-name>
            <surname>Chang</surname>
            ,
            <given-names>E.</given-names>
          </string-name>
          (
          <year>2005</year>
          ),
          <source>Proceedings of the 38th Annual Hawaii International Conference on System Sciences. IEEE</source>
          , pp.
          <fpage>143a</fpage>
          -
          <lpage>143a</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          10.
          <string-name>
            <surname>Spackman</surname>
            ,
            <given-names>K.A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Campbell</surname>
            ,
            <given-names>K.E.</given-names>
          </string-name>
          and
          <string-name>
            <surname>Côté</surname>
            ,
            <given-names>R.A.</given-names>
          </string-name>
          (
          <year>1997</year>
          ),
          <article-title>Proceedings of the AMIA annual fall symposium</article-title>
          .
          <source>American Medical Informatics Association</source>
          , pp.
          <fpage>640</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref11">
        <mixed-citation>
          11.
          <string-name>
            <surname>Stearns</surname>
            ,
            <given-names>M.Q.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Price</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Spackman</surname>
            ,
            <given-names>K.A.</given-names>
          </string-name>
          and
          <string-name>
            <surname>Wang</surname>
            ,
            <given-names>A.Y.</given-names>
          </string-name>
          (
          <year>2001</year>
          ),
          <source>Proceedings of the AMIA Symposium. American Medical Informatics Association</source>
          , pp.
          <fpage>662</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref12">
        <mixed-citation>
          12.
          <string-name>
            <surname>lin</surname>
            ,
            <given-names>F.A.H.W.J.W.F.</given-names>
          </string-name>
          (
          <year>2009</year>
          )
          <article-title>Method Research of Constructing Clinical Disease Domain Ontology (In Chinese)</article-title>
          .
          <source>Journal of intellgence</source>
          , Vol.
          <volume>28</volume>
          .
        </mixed-citation>
      </ref>
      <ref id="ref13">
        <mixed-citation>
          13.
          <string-name>
            <surname>Kibbe</surname>
            ,
            <given-names>W.A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Arze</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Felix</surname>
            ,
            <given-names>V.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Mitraka</surname>
            ,
            <given-names>E.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Bolton</surname>
            ,
            <given-names>E.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Fu</surname>
            ,
            <given-names>G.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Mungall</surname>
            ,
            <given-names>C.J.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Binder</surname>
            ,
            <given-names>J.X.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Malone</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          and
          <string-name>
            <surname>Vasant</surname>
            ,
            <given-names>D.</given-names>
          </string-name>
          (
          <year>2014</year>
          )
          <article-title>Disease Ontology 2015 update: an expanded and updated database of human diseases for linking biomedical knowledge through disease data</article-title>
          .
          <source>Nucleic acids research</source>
          ,
          <volume>43</volume>
          ,
          <fpage>D1071</fpage>
          -
          <lpage>D1078</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref14">
        <mixed-citation>
          14.
          <string-name>
            <surname>Schriml</surname>
            ,
            <given-names>L.M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Arze</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Nadendla</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Chang</surname>
            ,
            <given-names>Y.-W.W.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Mazaitis</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Felix</surname>
            ,
            <given-names>V.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Feng</surname>
            ,
            <given-names>G.</given-names>
          </string-name>
          and
          <string-name>
            <surname>Kibbe</surname>
            ,
            <given-names>W.A.</given-names>
          </string-name>
          (
          <year>2011</year>
          )
          <article-title>Disease Ontology: a backbone for disease semantic integration</article-title>
          .
          <source>Nucleic acids research</source>
          ,
          <volume>40</volume>
          ,
          <fpage>D940</fpage>
          -
          <lpage>D946</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref15">
        <mixed-citation>
          15.
          <string-name>
            <surname>Mungall</surname>
            ,
            <given-names>C.J.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>McMurry</surname>
            ,
            <given-names>J.A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Köhler</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Balhoff</surname>
            ,
            <given-names>J.P.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Borromeo</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Brush</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Carbon</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Conlin</surname>
            ,
            <given-names>T.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Dunn</surname>
            ,
            <given-names>N.</given-names>
          </string-name>
          and
          <string-name>
            <surname>Engelstad</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          et al.(
          <year>2017</year>
          )
          <article-title>The Monarch Initiative: an integrative data and analytic platform connecting phenotypes to genotypes across species</article-title>
          .
          <source>Nucleic acids research</source>
          ,
          <volume>45</volume>
          ,
          <fpage>D712</fpage>
          -
          <lpage>D722</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref16">
        <mixed-citation>
          16.
          <string-name>
            <surname>Ceusters</surname>
            ,
            <given-names>W.</given-names>
          </string-name>
          and
          <string-name>
            <surname>Smith</surname>
            ,
            <given-names>B.</given-names>
          </string-name>
          (
          <year>2015</year>
          ), MIE, Vol.
          <volume>210</volume>
          , pp.
          <fpage>155</fpage>
          -
          <lpage>159</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref17">
        <mixed-citation>
          17.
          <string-name>
            <surname>He</surname>
            ,
            <given-names>Y.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Xiang</surname>
            ,
            <given-names>Z.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Zheng</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Lin</surname>
            ,
            <given-names>Y.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Overton</surname>
            ,
            <given-names>J.A.</given-names>
          </string-name>
          and
          <string-name>
            <surname>Ong</surname>
            ,
            <given-names>E.</given-names>
          </string-name>
          (
          <year>2018</year>
          )
          <article-title>The eXtensible ontology development (XOD) principles and tool implementation to support ontology interoperability</article-title>
          .
          <source>Journal of biomedical semantics, 9</source>
          ,
          <fpage>3</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref18">
        <mixed-citation>
          18.
          <string-name>
            <surname>Xiang</surname>
            ,
            <given-names>Z.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Courtot</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Brinkman</surname>
            ,
            <given-names>R.R.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Ruttenberg</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          and
          <string-name>
            <surname>He</surname>
            ,
            <given-names>Y.</given-names>
          </string-name>
          (
          <year>2010</year>
          )
          <article-title>OntoFox: web-based support for ontology reuse</article-title>
          .
          <source>BMC research notes</source>
          ,
          <volume>3</volume>
          ,
          <fpage>175</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref19">
        <mixed-citation>
          19.
          <string-name>
            <surname>Ong</surname>
            ,
            <given-names>E.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Xiang</surname>
            ,
            <given-names>Z.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Zhao</surname>
            ,
            <given-names>B.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Liu</surname>
            ,
            <given-names>Y.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Lin</surname>
            ,
            <given-names>Y.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Zheng</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Mungall</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Courtot</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Ruttenberg</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          and
          <string-name>
            <surname>He</surname>
            ,
            <given-names>Y.</given-names>
          </string-name>
          (
          <year>2017</year>
          )
          <article-title>Ontobee: A linked ontology data server to support ontology term dereferencing, linkage, query and integration</article-title>
          .
          <source>Nucleic acids research</source>
          ,
          <volume>45</volume>
          ,
          <fpage>D347</fpage>
          -
          <lpage>d352</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref20">
        <mixed-citation>
          20.
          <string-name>
            <surname>Malone</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Holloway</surname>
            ,
            <given-names>E.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Adamusiak</surname>
            ,
            <given-names>T.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Kapushesky</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Zheng</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Kolesnikov</surname>
            ,
            <given-names>N.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Zhukova</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Brazma</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          and
          <string-name>
            <surname>Parkinson</surname>
            ,
            <given-names>H.</given-names>
          </string-name>
          (
          <year>2010</year>
          )
          <article-title>Modeling sample variables with an Experimental Factor Ontology</article-title>
          . Bioinformatics,
          <volume>26</volume>
          ,
          <fpage>1112</fpage>
          -
          <lpage>1118</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref21">
        <mixed-citation>
          21.
          <string-name>
            <surname>Malone</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Rayner</surname>
            ,
            <given-names>T.F.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Zheng Bradley</surname>
            ,
            <given-names>X.</given-names>
          </string-name>
          and
          <string-name>
            <surname>Parkinson</surname>
            ,
            <given-names>H.</given-names>
          </string-name>
          (
          <year>2008</year>
          ),
          <source>Proceedings of the Eleventh Annual Bioontologies Meeting</source>
          . Toronto, Canada.
        </mixed-citation>
      </ref>
      <ref id="ref22">
        <mixed-citation>
          22.
          <string-name>
            <surname>Dumontier</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Baker</surname>
            ,
            <given-names>C.J.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Baran</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Callahan</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Chepelev</surname>
            ,
            <given-names>L.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Cruz-Toledo</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Del Rio</surname>
            ,
            <given-names>N.R.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Duck</surname>
            ,
            <given-names>G.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Furlong</surname>
            ,
            <given-names>L.I.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Keath</surname>
            ,
            <given-names>N.</given-names>
          </string-name>
          et al. (
          <year>2014</year>
          )
          <article-title>The Semanticscience Integrated Ontology (SIO) for biomedical research and knowledge discovery</article-title>
          .
          <source>Journal of Biomedical Semantics</source>
          ,
          <volume>5</volume>
          ,
          <fpage>14</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref23">
        <mixed-citation>
          23.
          <string-name>
            <surname>Mattingly</surname>
            ,
            <given-names>C.J.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>McKone</surname>
            ,
            <given-names>T.E.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Callahan</surname>
            ,
            <given-names>M.A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Blake</surname>
            ,
            <given-names>J.A.</given-names>
          </string-name>
          and
          <string-name>
            <surname>Hubal</surname>
            ,
            <given-names>E.A.C.</given-names>
          </string-name>
          (
          <year>2012</year>
          ). ACS Publications.
        </mixed-citation>
      </ref>
      <ref id="ref24">
        <mixed-citation>
          24.
          <string-name>
            <surname>Mungall</surname>
            ,
            <given-names>C.J.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Torniai</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Gkoutos</surname>
            ,
            <given-names>G.V.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Lewis</surname>
            ,
            <given-names>S.E.</given-names>
          </string-name>
          and
          <string-name>
            <surname>Haendel</surname>
            ,
            <given-names>M.A.</given-names>
          </string-name>
          (
          <year>2012</year>
          )
          <article-title>Uberon, an integrative multi-species anatomy ontology</article-title>
          .
          <source>Genome biology</source>
          ,
          <volume>13</volume>
          ,
          <fpage>R5</fpage>
          .
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>