<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta />
    <article-meta>
      <title-group>
        <article-title>Identifying anatomical concepts associated with ICD10 diseases</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Fleur Mougin</string-name>
          <email>fleur.mougin@isped.u-bordeaux2</email>
          <xref ref-type="aff" rid="aff1">1</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Olivier Bodenreider</string-name>
          <email>olivier@nlm.nih.gov</email>
          <xref ref-type="aff" rid="aff2">2</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>et Anita Burgun</string-name>
          <email>Anita.Burgun@univ-rennes1.fr</email>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          <institution>INSERM U936, EA3888, School of Medicine, University of Rennes</institution>
          <addr-line>1, IFR 140</addr-line>
          ,
          <country country="FR">France</country>
        </aff>
        <aff id="aff1">
          <label>1</label>
          <institution>LESIM, INSERM U897, ISPED, University Victor Segalen Bordeaux</institution>
          <addr-line>2</addr-line>
          ,
          <country country="FR">France</country>
        </aff>
        <aff id="aff2">
          <label>2</label>
          <institution>National Library of Medicine</institution>
          ,
          <addr-line>Bethesda, Maryland</addr-line>
          ,
          <country country="US">USA</country>
        </aff>
      </contrib-group>
      <abstract>
        <p>Unlike recent biomedical terminologies, the International Classification of Diseases (ICD) does not state any explicit associations between a given disease and the corresponding anatomical structure(s). As a consequence, clinical repositories coded with ICD cannot be searched by anatomical structure. The objective of this work is to find associations between diseases from ICD10 and anatomical structures. Toward this end, we investigated three approaches (symbolic, lexical, and statistical) which exploit various features of the Unified Medical Language System (UMLS). We evaluated these approaches according to i) the consistency of resulting anatomical concepts with the high-level anatomical concept(s) identified for the chapter in which the disease is listed; and ii) the validity of resulting anatomical concepts assessed manually. We show that the symbolic approach is both the most productive and the most accurate approach.</p>
      </abstract>
      <kwd-group>
        <kwd>Biomedical terminology</kwd>
        <kwd>Anatomy</kwd>
        <kwd>Unified Medical Language System</kwd>
        <kwd>International Classification of Diseases</kwd>
      </kwd-group>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>-</title>
      <p>
        Biomedical terminologies developed recently often use formalisms based on
description logics for their representation
        <xref ref-type="bibr" rid="ref12">(Schulz &amp; Hahn, 2005)</xref>
        . A formal definition
of the concepts is provided along various dimensions through axioms relating
concepts to other concepts across subdomains. For examples, in SNOMED CT
        <xref ref-type="bibr" rid="ref11">(Rector &amp; Brandt, 2008)</xref>
        and the NCI Thesaurus
        <xref ref-type="bibr" rid="ref6">(Hartel, de Coronado, Dionne,
Fragoso, &amp; Golbeck, 2005)</xref>
        , diseases are defined in relation to anatomical structures.
Because it is represented explicitly, this information can be exploited in clinical
applications where the association between diseases and anatomical structures matters
(e.g., to identify all patients treated for a disease with manifestation in the knee).
      </p>
      <p>In contrast, although terminologies such as the International Classification of
Diseases (ICD) organize diseases in hierarchies based for the most part on body
systems, no explicit association between a given disease and the corresponding
anatomical structure(s) is stated in ICD. Therefore, clinical data coded with ICD
cannot be precisely searched by anatomical structure.</p>
      <p>The objective of this work is to discover relations between ICD10 diseases and
anatomical structures. Three approaches (symbolic, lexical and statistical) are
investigated.</p>
      <p>
        Terminological resources have been used for extracting or inferring relations
among concepts. The compositional nature of terms has been exploited for identifying
hierarchical and associative relations (e.g.,
        <xref ref-type="bibr" rid="ref10 ref2 ref3 ref9">(Bodenreider &amp; McCray, 2003; Campbell,
Tuttle, &amp; Spackman, 1998; Ogren, Cohen, Acquaah-Mensah, Eberlein, &amp; Hunter,
2004)</xref>
        ). Analogously,
        <xref ref-type="bibr" rid="ref4">(Chen, Hripcsak, Xu, Markatou, &amp; Friedman, 2008)</xref>
        have used
co-occurrence information for the acquisition of disease-drug relations from clinical
text. The specific contribution of this paper is to compare the performance of three
approaches for the extraction of one particular type of associative relations
(diseaseanatomy).
2
      </p>
    </sec>
    <sec id="sec-2">
      <title>Resources</title>
      <p>2.1</p>
    </sec>
    <sec id="sec-3">
      <title>International Classification of Diseases</title>
      <p>
        The International Classification of Diseases is an international standard diagnostic
classification for epidemiology, health management, and clinical use
        <xref ref-type="bibr" rid="ref5">(Gersenovic,
1995)</xref>
        . The 10th revision (ICD10)
        <xref ref-type="bibr" rid="ref7">(International Classification of Diseases, manual of
the International Statistical Classification of diseases, injuries and causes of death:
10th revision, 1993)</xref>
        is the latest version of the ICD and is organized in 21 chapters
including “Certain infectious and parasitic diseases” and “Diseases of the nervous
system”. Twelve chapters group diseases with respect to body systems, namely,
chapters II, III, IV and VI to XIV. ICD10 comprises more than 12,000 disease codes.
2.2
      </p>
    </sec>
    <sec id="sec-4">
      <title>Unified Medical Language System</title>
      <p>
        The Unified Medical Language System® (UMLS®)
        <xref ref-type="bibr" rid="ref8">(Lindberg, Humphreys, &amp;
McCray, 1993)</xref>
        includes two sources of semantic information: the Metathesaurus® and
the Semantic Network. The UMLS Metathesaurus is assembled by integrating close to
150 source vocabularies, including the ICD10. It contains more than 1.8 million
concepts and nearly 44 million relations among these concepts. There are more than
18 million relations explicitly defined in the Metathesaurus. The Semantic Network is
a much smaller network of 135 semantic types organized in a tree structure
        <xref ref-type="bibr" rid="ref2 ref9">(McCray,
2003)</xref>
        . The semantic types have been aggregated into fifteen coarser semantic groups
        <xref ref-type="bibr" rid="ref2 ref9">(Bodenreider &amp; McCray, 2003)</xref>
        , which represent subdomains of biomedicine (e.g.,
Anatomy, Disorders). Each Metathesaurus concept has a unique identifier and is
assigned at least one semantic type. Version 2008AB of the UMLS is used in this
study.
3
      </p>
    </sec>
    <sec id="sec-5">
      <title>Methods</title>
      <p>Three distinct approaches are investigated in this study (symbolic, lexical, and
statistical), exploiting various features of the UMLS. The symbolic approach uses the
relations defined between ICD10 concepts and anatomical concepts in the source
vocabularies included in the UMLS. The lexical approach attempts to identify
anatomical concepts in the names of ICD10 concepts. Finally, the statistical approach
relies on co-occurrence information. This investigation is restricted to the diseases
from the 12 chapters of ICD10 organized around a given body system (Table 1). In
this section, we present the three approaches, as well as the evaluation of the
consistency and validity of the results.
3.1</p>
    </sec>
    <sec id="sec-6">
      <title>Symbolic approach (approach A)</title>
      <p>In biomedical terminologies, the domain and range of defined relationships is
constrained to specific semantic types as part of their definition. In contrast, the range
of undefined relationships is unrestricted and must be constrained by users. Because of
this difference, two strategies were investigated in the symbolic approach.
3.1.1</p>
      <sec id="sec-6-1">
        <title>Defined relationships (strategy A1)</title>
        <p>Starting from a given disease concept, we search for target concepts connected to
the disease concept by defined relationships whose range is anatomy, including
disease_has_associated_anatomic_site and finding_site_of.
3.1.2</p>
      </sec>
      <sec id="sec-6-2">
        <title>All relationships (strategy A2)</title>
        <p>When all the relationships of a given disease source concept are exploited, we
constrain the range of these relationships to anatomy by requiring that target concepts
be associated with the semantic group (SG) Anatomy, i.e. categorized by a semantic
type (ST) belonging to the SG Anatomy.</p>
        <p>As illustrated in Figure 1, within each strategy, relations to anatomical concepts are
first searched directly from the disease concept (case a). Failing to find any, we start
the search again from concepts in mapping relation to the source disease concept (case
b). In practice, we allow the traversal of relations whose name contains the pattern
“mapp” (e.g., mapped_from and mapped_to) before searching for related anatomical
concepts. If none is found at this stage, we start the search for anatomical concepts
again, but from the parent concept(s) of the source disease concept (case c).
3.2</p>
      </sec>
    </sec>
    <sec id="sec-7">
      <title>Lexical approach (approach B)</title>
      <p>
        This approach attempts to identify anatomical structures in the name of ICD10
concepts. It uses MetaMap, a linguistically-motivated named entity recognition
program specially developed for biomedical entities, which extracts Metathesaurus
concepts from text
        <xref ref-type="bibr" rid="ref1">(Aronson, 2001)</xref>
        . Different options are proposed to configure
MetaMap, in particular a restriction to extract only concepts categorized by a given set
of STs. We first recover the names of each ICD10 concept from the Metathesaurus,
i.e. the preferred term of the concept and all its synonyms. This list is then processed
by MetaMap, with restriction to STs from the SG Anatomy. For example, from the
term “pulmonary edema”, MetaMap identifies the Metathesaurus concepts Edema
(C0013604) and Pulmonary (C0024109), the latter of which belongs to the SG
Anatomy.
      </p>
      <p>In order to assess the consistency of the anatomical concepts identified for a given
disease, we determine whether this anatomical structure is related to the body system
corresponding to the chapter in which this disease is listed. One of the authors (OB)
associated each chapter (or subdivision thereof) with one or two high-level anatomical
concepts (e.g., Cardiovascular system (C0007226) for the chapter “Diseases of the
circulatory system”). We computed the list of all descendants of each high-level
anatomical concept. The association between a disease and an anatomical concept is
deemed consistent if the anatomical concept is found among the descendants of the
high-level anatomical concept(s) identified for the chapter in which the disease is
listed (Figure 2).</p>
      <p>ICD10
chapter
D3</p>
      <p>D4</p>
      <p>High-level
anatomical</p>
      <p>concept
A1</p>
      <p>A2
D1</p>
      <p>D2</p>
      <p>A3</p>
      <p>A4
4</p>
    </sec>
    <sec id="sec-8">
      <title>Results</title>
      <p>4,391 ICD10 concepts amenable to our method were selected. The first three
columns of Table 1 show the repartition of ICD10 concepts by chapter (or subdivision
thereof). Detailed results for each approach are displayed Table 2.
4.1</p>
    </sec>
    <sec id="sec-9">
      <title>Symbolic approach (approach A)</title>
      <p>Overall, 4,342 ICD10 concepts were associated with at least one anatomical
concept. The resulting anatomical concepts are not systematically the same with both
strategies.</p>
      <p>For example, the ICD10 concept Idiopathic gout (C0149896) was associated with
distinct anatomical concepts according to the strategy. As strategy A1 could not find
any anatomical concept for Idiopathic gout through a defined relation, it first selected
“mapped” concepts: Gout (C0018099) and Gouty nephropathy (C0391820),
themselves related to Joints (C0022417), Connective and Soft Tissue (C1516798), and
Kidney (C0022646) through defined relations from SNOMED CT. Conversely,
strategy A2 selected one unique concept, Articular system (C0149896), through an
undefined relationship (other related) with Idiopathic gout.</p>
      <p>All ICD10 concepts successfully associated with anatomical concepts through the
first strategy are included in those found through the second strategy. For 82 ICD10
concepts, only the second strategy was able to find related anatomical concepts. An
example is the concept Lymphoedema, not elsewhere classified (C0494630) for which
only the second strategy proposed Lymph nodes (C0024204).</p>
      <p>Finally, it is worth noting that the two alternatives proposed when no anatomical
concept could be found directly through the ICD10 concept itself (cases b and c)
contributed to improve the results. For example, Embolism and thrombosis of iliac
artery (C0155755) was associated with the anatomical concepts Structure of iliac
artery (C0020887) and Arteries (C0003842) through its mapping relation to the
concepts Thrombosis of iliac artery (C0235518) and Embolism and thrombosis of
other specified artery (C0155754). On the other hand, the ICD10 concept
Nonrheumatic aortic valve disorders (C0003502) was associated with the anatomical
concepts Heart Valves (C0018826), Heart (C0018787), and Aortic valve structure
(C0003501) through its parent concepts Heart valve disease (C0018824), Other heart
disease C0178273, and Aortic valvular disorders (C1260873).
4.2</p>
    </sec>
    <sec id="sec-10">
      <title>Lexical approach (approach B)</title>
      <p>Overall, 2,987 ICD10 concepts exhibit at least one anatomical concept in their
name. An example is the ICD10 concept Biliary Fistula (C0005417) associated with
the two anatomical concepts Bile (Bile fluid) (C0005388) and Bile Duct (Bile duct
structure) (C0005400) through the synonymous term Fistula of bile duct from the
source vocabulary SNOMED CT.
4.3</p>
    </sec>
    <sec id="sec-11">
      <title>Statistical approach (approach C)</title>
      <p>Overall, only 783 ICD10 concepts co-occur with at least one anatomical concept.
An example is Ainhum (C0001860) which is a painful constriction of the base of the
fifth toe. This ICD10 concept co-occurs with the three following anatomical concepts:
Fingers (C0016129), Hallux structure (C0018534), and Toes (C0040357).
4.4</p>
    </sec>
    <sec id="sec-12">
      <title>Comparing the three approaches</title>
      <p>As shown in Figure 3, the symbolic approach was able to find anatomical concepts
for all but 49 of the 4,391 diseases from ICD10 (98.9%). The lexical approach found
anatomical concepts for 2,987 disease concepts, including 23 of the 49 concepts for
which the symbolic approach had failed. For example, the ICD10 concept Hereditary
nephropathy, not elsewhere classified, minor glomerular abnormality (C0868873) has
been associated with the anatomical concept Glomerular (Kidney Glomerulus)
(C0022663) only through the lexical approach. No new anatomical concepts could be
found through the statistical approach. Overall, only 26 ICD10 concepts could not be
associated with any anatomical concepts through any of the approaches, including
Defects of catalase and peroxidase (C0494349) and Other immunodeficiencies
(C0494262).
anatomical concepts associated were consistent and for 466 diseases, none of the
concepts associated was consistent.</p>
      <p>As an illustration, the concept Central pontine myelinolysis (C0206083) is listed in
the chapter “Diseases of the nervous system”. Therefore, the associated anatomical
concepts were expected to be found among the descendants of the corresponding
highlevel anatomical concept, Nervous system structure (C0027763) (last column of Table
1). The consistency results are as follows:
• Strategy A1: Two anatomical concepts were selected, namely Pontine
structure (C0032639) and Neuraxis (C0927232), both consistent with
Nervous system structure;</p>
      <p>Strategy A2: In addition to the two concepts obtained with strategy A1,
In Blood (C0005768) was also selected but is inconsistent with Nervous
system structure;</p>
      <p>Approach B: Only the
(C0032639) is found (consistent);</p>
      <p>Approach C: 26 anatomical concepts co-occur with Central pontine
myelinolysis, of which 19 are consistent with Nervous system structure,
including Basal Ganglia (C0004781) and Cerebellum (C0007765).
Incompatible concepts include Tongue (C0040408) and Structure of
pituitary fossa (C0036609).</p>
      <p>anatomical concept Pontine structure
4.6</p>
    </sec>
    <sec id="sec-13">
      <title>Assessing validity</title>
      <p>were valid and for 2 diseases, none of the concepts associated were valid. Finally, no
anatomical structures was identified for 3 diseases.</p>
      <p>Overall, the best results are obtained through the symbolic approach. Not only is
this approach more productive (i.e., it associates more ICD10 concepts with
anatomical structures), but it also exhibits superior consistency and validity (more
concepts have a percentage of 100%). As expected, strategy A1 (based on defined
relations) has better precision than strategy A2 (using all relations), but its recall is
slightly lower. The lexical approach could not bring much improvement over the
results of the symbolic approach. The statistical approach was disappointing as it did
not find any anatomical structures not already found by the two other approaches.</p>
      <p>The two evaluations in terms of consistency and validity sometimes lead to the
same conclusions. In some cases, however, the evaluations yield different results.
Indeed, the concept Hereditary haemorrhagic telangiectasia (C0039445) was
associated with four anatomical concepts through strategy A2: i) Cardiovascular
system (C0007226) and Vascular System (C0489903) which are consistent and valid;
ii) In Blood (C0005768) which is inconsistent and invalid; iii) Microscopic skin
vascular structure (C1283407) which is consistent but invalid. Surprisingly,
consistency evaluation was sometimes less permissive than validity evaluation. For
instance, Acute erythraemia and erythroleukaemia (C0001317) was associated,
among other anatomical concepts, with Bone Marrow (C0005953) which was rightly
deemed valid and was wrongly inconsistent.
5.2</p>
    </sec>
    <sec id="sec-14">
      <title>Limitations</title>
      <p>Of the 12,320 ICD10 codes, only 4,265 (35%) are amenable to the kind of
processing investigated in this study. For the UMLS concepts corresponding to the
remaining ICD10 codes, no relation to anatomical concepts was represented in any
vocabulary, and no anatomical entity could be identified in the names of these
concepts. In practice, the methods proposed here could, at best, be used to
complement other methods or expert knowledge.</p>
      <p>The evaluation does not include every ICD10 chapter but it must be noted that
chapters like “Mental and behavioural disorders” and “Certain infectious and parasitic
diseases” are evaluated indirectly through the dagger-asterisk representation in ICD10.
Indeed, some diseases are listed twice; in the chapter corresponding to their etiology
(dagger) and in the chapter corresponding to their manifestation (asterisk). For
example, the disease Alzheimer's disease (G30+) is listed primarily in chapter VI
“Diseases of the nervous system”, while its manifestation Dementia in Alzheimer's
disease (F00*) is found in chapter V “Mental and behavioural disorders”. Thus, a
large number of diseases from chapters not investigated in this study have been
considered indirectly through other chapters.</p>
      <p>The recall of the lexical and statistical approaches is quite low. While it could
likely be increased by exploiting mapping and parent relations as we did in the lexical
approach, we believe that this extension would adversely affect the precision of our
results.</p>
      <p>Finally, depending on the application, the level of granularity of the anatomical
structures associated with ICD10 diseases may or may not be appropriate. For
example, as mentioned earlier, Idiopathic gout is appropriately associated with joints
through the anatomical concepts Joints and Articular system. However, terminological
knowledge from the UMLS fails to capture the fact that, in most cases of gout, joint
inflammation affects the toes. Moreover, Idiopathic gout is also associated with
Kidney, while kidney manifestations are much less frequent.
5.3</p>
    </sec>
    <sec id="sec-15">
      <title>Practical implications</title>
      <p>The coding system used by French pathologists supports the retrieval of specific
anatomical structures from the code itself. However, ICD10 lacks this feature. Cancer
registries drawing from repositories coded with ICD10 could be queried precisely with
respect to anatomical structures if the relations between ICD10 codes and anatomical
structures were asserted as we suggest here.
6</p>
    </sec>
    <sec id="sec-16">
      <title>Acknowledgements</title>
      <p>This research was supported in part by the Intramural Research Program of the
National Institutes of Health (NIH), National Library of Medicine (NLM).
ICD10 Chapter / Subchapter label
Malignant neoplasms of lip, oral cavity and pharynx
Malignant neoplasms of digestive organs
Malignant neoplasms of respiratory and intrathoracic organs
Malignant neoplasms of bone and articular cartilage
Melanoma and other malignant neoplasms of skin
Malignant neoplasms of mesothelial and soft tissue
Malignant neoplasm of breast
Malignant neoplasms of female genital organs
Malignant neoplasms of male genital organs
Malignant neoplasms of urinary tract
Mal. neoplasms of eye, brain and other parts of central nervous system
Malignant neoplasms of thyroid and other endocrine glands
Malignant neoplasms of lymphoid, haematopoietic and related tissue
Diseases of blood and blood-forming organs...</p>
      <p>Endocrine, nutritional and metabolic diseases
Diseases of the nervous system
Diseases of the eye and adnexa
Diseases of the ear and mastoid process
Diseases of the circulatory system
Diseases of the respiratory system
Diseases of the digestive system
Diseases of the skin and subcutaneous tissue
Diseases of the musculoskeletal system and connective tissue
Diseases of the genitourinary system</p>
      <p>Reference anatomical concepts
Oral cavity, Pharyngeal structure
Gastrointestinal system
Respiratory System, Intrathoracic organ
Skeletal system
Skin
Mesothelium, Soft tissue
Breast
Female genitalia
Male Genital Organs
Urinary system
Eye, Neuraxis
Endocrine system
Hematopoietic System, Lymphoid organ structure
Blood, Hematopoietic System
Endocrine system
Nervous system structure
Eyes and eye appendages
Ears and mastoid cells
Cardiovascular system
Respiratory System
Gastrointestinal system
Integumentary system
Musculoskeletal System
Genitourinary system</p>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          <string-name>
            <surname>ARONSON</surname>
            ,
            <given-names>A. R.</given-names>
          </string-name>
          (
          <year>2001</year>
          ).
          <article-title>Effective mapping of biomedical text to the UMLS Metathesaurus: the MetaMap program</article-title>
          .
          <source>Proc AMIA Symp</source>
          ,
          <volume>17</volume>
          -
          <fpage>21</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          <string-name>
            <surname>BODENREIDER</surname>
            ,
            <given-names>O.</given-names>
          </string-name>
          , &amp;
          <string-name>
            <surname>MCCRAY</surname>
            ,
            <given-names>A. T.</given-names>
          </string-name>
          (
          <year>2003</year>
          ).
          <article-title>Exploring semantic groups through visual approaches</article-title>
          .
          <source>J Biomed Inform</source>
          ,
          <volume>36</volume>
          (
          <issue>6</issue>
          ),
          <fpage>414</fpage>
          -
          <lpage>432</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          <string-name>
            <surname>CAMPBELL</surname>
            ,
            <given-names>K. E.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>TUTTLE</surname>
            ,
            <given-names>M. S.</given-names>
          </string-name>
          , &amp;
          <string-name>
            <surname>SPACKMAN</surname>
            ,
            <given-names>K. A.</given-names>
          </string-name>
          (
          <year>1998</year>
          ).
          <article-title>A "lexically-suggested logical closure" metric for medical terminology maturity</article-title>
          .
          <source>Proc AMIA Symp</source>
          ,
          <volume>785</volume>
          -
          <fpage>789</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          <string-name>
            <surname>CHEN</surname>
            ,
            <given-names>E. S.</given-names>
          </string-name>
          , HRIPCSAK, G., XU,
          <string-name>
            <surname>H.</surname>
          </string-name>
          , MARKATOU,
          <string-name>
            <given-names>M.</given-names>
            , &amp;
            <surname>FRIEDMAN</surname>
          </string-name>
          ,
          <string-name>
            <surname>C.</surname>
          </string-name>
          (
          <year>2008</year>
          ).
          <article-title>Automated acquisition of disease drug knowledge from biomedical and clinical documents: an initial study</article-title>
          .
          <source>J Am Med Inform Assoc</source>
          ,
          <volume>15</volume>
          (
          <issue>1</issue>
          ),
          <fpage>87</fpage>
          -
          <lpage>98</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          <string-name>
            <surname>GERSENOVIC</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          (
          <year>1995</year>
          ).
          <article-title>The ICD family of classifications</article-title>
          .
          <source>Methods Inf Med</source>
          ,
          <volume>34</volume>
          (
          <issue>1-2</issue>
          ),
          <fpage>172</fpage>
          -
          <lpage>175</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          <string-name>
            <surname>HARTEL</surname>
            ,
            <given-names>F. W.</given-names>
          </string-name>
          , DE CORONADO,
          <string-name>
            <surname>S.</surname>
          </string-name>
          ,
          <string-name>
            <surname>DIONNE</surname>
          </string-name>
          ,
          <string-name>
            <surname>R.</surname>
          </string-name>
          , FRAGOSO,
          <string-name>
            <given-names>G.</given-names>
            , &amp;
            <surname>GOLBECK</surname>
          </string-name>
          ,
          <string-name>
            <surname>J.</surname>
          </string-name>
          (
          <year>2005</year>
          ).
          <article-title>Modeling a description logic vocabulary for cancer research</article-title>
          .
          <source>J Biomed Inform</source>
          ,
          <volume>38</volume>
          (
          <issue>2</issue>
          ),
          <fpage>114</fpage>
          -
          <lpage>129</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          <article-title>International Classification of Diseases, manual of the International Statistical Classification of diseases, injuries and causes of death: 10th revision (</article-title>
          <year>1993</year>
          ). WHO. http://www.who.int/classifications/apps/icd/icd10online.
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          <string-name>
            <surname>LINDBERG</surname>
            ,
            <given-names>D. A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>HUMPHREYS</surname>
            ,
            <given-names>B. L.</given-names>
          </string-name>
          , &amp;
          <string-name>
            <surname>MCCRAY</surname>
            ,
            <given-names>A. T.</given-names>
          </string-name>
          (
          <year>1993</year>
          ).
          <article-title>The Unified Medical Language System</article-title>
          .
          <source>Methods Inf Med</source>
          ,
          <volume>32</volume>
          (
          <issue>4</issue>
          ),
          <fpage>281</fpage>
          -
          <lpage>291</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          <string-name>
            <surname>MCCRAY</surname>
            ,
            <given-names>A. T.</given-names>
          </string-name>
          (
          <year>2003</year>
          ).
          <article-title>An upper-level ontology for the biomedical domain</article-title>
          .
          <source>Comp Funct Genomics</source>
          ,
          <volume>4</volume>
          (
          <issue>1</issue>
          ),
          <fpage>80</fpage>
          -
          <lpage>84</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          <string-name>
            <surname>OGREN</surname>
            ,
            <given-names>P. V.</given-names>
          </string-name>
          , COHEN,
          <string-name>
            <given-names>K. B.</given-names>
            ,
            <surname>ACQUAAH-MENSAH</surname>
          </string-name>
          ,
          <string-name>
            <given-names>G. K.</given-names>
            ,
            <surname>EBERLEIN</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J.</given-names>
            , &amp;
            <surname>HUNTER</surname>
          </string-name>
          ,
          <string-name>
            <surname>L.</surname>
          </string-name>
          (
          <year>2004</year>
          ).
          <article-title>The compositional structure of Gene Ontology terms</article-title>
          .
          <source>Pac Symp Biocomput</source>
          ,
          <fpage>214</fpage>
          -
          <lpage>225</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref11">
        <mixed-citation>
          <string-name>
            <surname>RECTOR</surname>
            ,
            <given-names>A. L.</given-names>
          </string-name>
          , &amp;
          <string-name>
            <surname>BRANDT</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          (
          <year>2008</year>
          ).
          <article-title>Why do it the hard way? The case for an expressive description logic for SNOMED</article-title>
          .
          <source>J Am Med Inform Assoc</source>
          ,
          <volume>15</volume>
          (
          <issue>6</issue>
          ),
          <fpage>744</fpage>
          -
          <lpage>751</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref12">
        <mixed-citation>
          <string-name>
            <surname>SCHULZ</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          , &amp;
          <string-name>
            <surname>HAHN</surname>
            ,
            <given-names>U.</given-names>
          </string-name>
          (
          <year>2005</year>
          ).
          <article-title>Part-whole representation and reasoning in formal biomedical ontologies</article-title>
          .
          <source>Artif Intell Med</source>
          ,
          <volume>34</volume>
          (
          <issue>3</issue>
          ),
          <fpage>179</fpage>
          -
          <lpage>200</lpage>
          .
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>