<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta />
    <article-meta>
      <title-group>
        <article-title>Visualizing the “Big Picture” of Change in NCIt's Biological Processes</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Yehoshua Perl</string-name>
          <email>perl@njit.edu</email>
          <xref ref-type="aff" rid="aff0">0</xref>
          <xref ref-type="aff" rid="aff1">1</xref>
          <xref ref-type="aff" rid="aff2">2</xref>
          <xref ref-type="aff" rid="aff3">3</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Christopher Ochs</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
          <xref ref-type="aff" rid="aff1">1</xref>
          <xref ref-type="aff" rid="aff2">2</xref>
          <xref ref-type="aff" rid="aff3">3</xref>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          <institution>A. National Cancer Insitute thesaurus</institution>
          ,
          <addr-line>NCIt</addr-line>
        </aff>
        <aff id="aff1">
          <label>1</label>
          <institution>Department of Computer Science New Jersey Institute of Technology Newark</institution>
          ,
          <addr-line>NJ</addr-line>
          ,
          <country country="US">USA</country>
        </aff>
        <aff id="aff2">
          <label>2</label>
          <institution>National Cancer Institute (NCI) National Institutes of Health Rockville</institution>
          ,
          <addr-line>MD</addr-line>
          ,
          <country country="US">USA</country>
        </aff>
        <aff id="aff3">
          <label>3</label>
          <institution>Sherri de Coronado</institution>
          ,
          <addr-line>Nicole Thomas</addr-line>
        </aff>
      </contrib-group>
      <abstract>
        <p>- The National Cancer Institute thesaurus (NCIt) is a large and complex ontology. NCIt is frequently updated; a new release is made available approximately every month. Tracking structural changes in NCIt is important for the editors of its content. In this paper we describe a methodology and tool using diff partial-area taxonomies to visually summarize structural changes between two NCIt releases. Diff partial-area taxonomies provide a comprehensible view of the overall impact of the changes. This methodology is illustrated using the Biological Process hierarchy. Specifically, we illustrate how diff partial-area taxonomies reflect change that occurred due to major restructuring of this hierarchy between September 2004 and December 2004. During this time the hierarchy nearly doubled in size and a large portion of the classes were extensively modified. Several kinds of change patterns are identified and discussed.</p>
      </abstract>
      <kwd-group>
        <kwd>ontology change</kwd>
        <kwd>ontology visualization</kwd>
        <kwd>ontology big picture</kwd>
        <kwd>NCIt</kwd>
        <kwd>abstraction network</kwd>
      </kwd-group>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>I. INTRODUCTION</title>
      <p>
        Large ontologies, such as the National Cancer Institute
thesaurus (NCIt) [
        <xref ref-type="bibr" rid="ref1">1</xref>
        ], change frequently and significantly over
their lifetimes. With each new release NCIt’s content
undergoes many modifications. New classes are added to
expand the ontology’s domain coverage and existing classes
are remodeled to address user requests, incremental
maintenance and improvement, errors, and inconsistencies.
      </p>
      <p>
        Our current research framework [
        <xref ref-type="bibr" rid="ref2">2</xref>
        ] is focused on
developing semi-automatic ontology quality assurance
methodologies. Tracking how an ontology’s content changed
due to modifications resulting from quality assurance efforts,
and other common ontology maintenance tasks, is important
for assessing the overall impact of the changes. By tracking the
structural differences between various versions of NCIt’s
content one can identify the types of changes that are being
applied by NCIt editors, and review how the changes affect the
overall structure of the ontology. When a significant change is
applied the overall impact of the change should be reviewed
for undesired consequences.
      </p>
    </sec>
    <sec id="sec-2">
      <title>Methods for detecting ontology change have been</title>
      <p>
        extensively studied. Various methodologies for computing
ontology diffs, which provide a detailed report of individual
changes, have been developed. For example, Noy et al. [
        <xref ref-type="bibr" rid="ref3">3</xref>
        ]
describes Promptdiff and Kremen et al. [
        <xref ref-type="bibr" rid="ref4">4</xref>
        ] describes
      </p>
      <p>
        NCIt is a large ontology composed of over 114,000 classes
(i.e., concepts) and tens of thousands of restrictions (i.e., roles
or relationships). A new version of NCIt is released, in OWL
[
        <xref ref-type="bibr" rid="ref6">6</xref>
        ] format, roughly once per month. NCIt’s content is
separated into 20 hierarchies, covering topics such as diseases
(e.g., cancers, the focus of the ontology), genetics, anatomy,
and biological processes.
      </p>
    </sec>
    <sec id="sec-3">
      <title>Various studies have investigated different aspects of NCIt. De Coronado et al. [7] review the quality assurance lifecycle of NCIt, describing how quality assurance is incorporated into the editing process.</title>
      <p>Fig 1. (a) An excerpt of 15 classes from NCIt’s Biological process hierarchy (Sept 2004). Classes are represented as labeled boxes and upward directed arrows
indicate subclass relationships. Labeled, dashed bubbles indicate that a set of classes have restriction with the given set of object properties. For example, Cellular
Process has a restriction Has associated location with a range of Cell. This restriction is inherited or refined by its descendants. Cell Cycle Stage introduces a
restriction with the object property Is part of process. (b) The same excerpt after various changes occurred to the content (Dec 2004). (c) The diff partial-area
taxonomy derived from these two releases. Diff areas are shown as boxes that are colored and organized into levels according to their number of object property
types. For example, the six classes in the Has associated location bubble in (b) are replaced by the green box labeled Has associated location. Newly introduced
areas are shown with a green outline, removed areas with a red outline, and modified areas with a yellow outline. Diff partial-areas are shown as boxes in their
respective diff areas. Introduced partial-areas have a green background, removed a red background, modified a yellow background, and unmodified a white
background (see Cell Cycle (1) in Fig 2). Each diff partial-area is labeled with the name of its root class, the number of classes in the diff partial-area in the to
release (in parenthesis), and a numeric summary of the changes that occurred. For example, the modified partial-area Cellular Process (6) is shown in the {Has
associated location} diff area, it contains 6 classes, and between the Sept 2004 and Dec 2004 releases one new class that was added to the ontology (Cell Death
Process) is now summarized by this diff partial-area (“+1 New”) and another class (Cell Cycle Stage) has moved to modified diff partial-area from its removed
partial-area in the {Has associated location, Is part of process} removed area (“+1 Moved” on the Cellular Process (6) modified partial-area). The class DNA
Sequence Alteration, in the same removed area, lost both of its restrictions and moved to the Ø modified area. This is reflected as a removed partial-area DNA
Sequence Alteration (0) and as part of the “+4 Moved” description in the Biological Process (7) modified partial-area.</p>
    </sec>
    <sec id="sec-4">
      <title>In Min et al. [8] we performed a quality assurance review</title>
      <p>
        of the Biological Process hierarchy’s content, finding a
significant number of inconsistencies in its modeling.
Gonçalves et al. [
        <xref ref-type="bibr" rid="ref9">9</xref>
        ] analyzed axiom changes across 88 releases
of NCIt and described a methodology for creating an NCIt diff.
      </p>
      <sec id="sec-4-1">
        <title>B. Diff Partial-area Taxonomies</title>
        <p>
          We define an abstraction network as a compact summary
of an ontology’s content and structure. An abstraction network
is composed of nodes that summarize sets of “similar” classes.
Nodes are organized into a hierarchy based on the underlying
subsumption hierarchy. For a review of abstraction networks
see Halper et al. [
          <xref ref-type="bibr" rid="ref10">10</xref>
          ]. Abstraction networks have been shown
to support ontology quality assurance in ontologies such as
NCIt, SNOMED CT, the Gene Ontology, and the Ontology of
Clinical Research (OCRe), among others [
          <xref ref-type="bibr" rid="ref10 ref2">2, 10</xref>
          ].
        </p>
        <p>
          In Ochs et al. [
          <xref ref-type="bibr" rid="ref5">5</xref>
          ] we introduced diff abstraction networks
to summarize (and visualize) the structural changes that occur
between two ontology releases. A diff abstraction network
called a diff partial-area taxonomy, which summaries changes
to sets of structurally and semantically similar classes, was
introduced to summarize the differences between two releases
of an ontology (e.g., the eagle-i Research Resource Ontology
(ERO) [
          <xref ref-type="bibr" rid="ref11">11</xref>
          ]). We will now describe the process of creating diff
partial-area taxonomies for NCIt. Due to space limitations we
provide an abridged explanation of the derivation. The full
derivation methodology, which is applicable to any OWL
ontology, is described in detail in Ochs et al. [
          <xref ref-type="bibr" rid="ref5">5</xref>
          ].
        </p>
      </sec>
    </sec>
    <sec id="sec-5">
      <title>Given two versions of NCIt, named from and to, a diff</title>
      <p>partial-area summarizes and visualizes changes in the
introduction and inheritance of restrictions on a hierarchy’s
classes. In OWL ontologies, such as NCIt, an owl:Restriction
axiom consists of a property, a constraint (e.g.,
someValuesFrom), and range class(es). For example, the class
Cellular Process has a restriction with the Has associated
location object property and a range of Cell. Similarly, the
class Mutation has a restriction with the object property has
initiator chemical or drug and a range of Mutagen. For diff
partial-area taxonomy derivation we consider the object
property in each restriction (e.g., Has associated location).</p>
    </sec>
    <sec id="sec-6">
      <title>In Fig 1a we show an excerpt of the Biological Process</title>
      <p>hierarchy from Sept 2004. In Fig 2b we show the excerpt after
some modifications were made in Dec 2004. The most
significant changes were in the removal of restrictions from
several of the classes (e.g., Mutation).</p>
    </sec>
    <sec id="sec-7">
      <title>In a diff partial-area taxonomy a diff area captures changes</title>
      <p>to sets of classes that explicitly, or through inheritance, have
class restrictions that use the same types of object properties.
For example, in Fig 1a, the classes Cell Cycle Stage and DNA
Sequence Alteration have restrictions that use the Has
associated location and Is part of process object properties. In
a later release, shown in Fig1b, Cell Cycle Stage no longer has
an Is part of process restriction and DNA Sequence Alteration
now has no restrictions. A diff area is named after the set of
object properties used in the class restrictions (e.g., {Has
associated location, Is part of process}). Based on the state of
such sets of classes we define four kinds of diff areas.</p>
      <p>An introduced area is a diff area with a set of classes that
have restrictions with a specific set of object properties in to
but no such class exists in from (e.g., {Has initiator chemical
or drug} in Fig 1c). A removed area indicates that there
existed a set of classes that had restrictions with a specific set
of object properties in from but no such class exists in to (e.g.,
{Is part of process} in Fig 1c). A modified area indicates that
there exists a set of classes that have restrictions with a
specific set of object properties in both from and to but the set
of classes is not the same (e.g., {Has associated location} in
Fig 1c). An unmodified area indicates that there exists the
same set of classes that have restrictions with a specific set of
object properties in both from and to.</p>
    </sec>
    <sec id="sec-8">
      <title>A class can be a member of up to two diff areas in the</title>
      <p>same diff partial-area taxonomy (e.g., Mutation in Fig 1c).
This occurs when the set of object properties used in
restrictions on the class changes (e.g., Has associated location
and Is part of process were removed from Mutation, as
captured by the removed area and the introduced area).</p>
    </sec>
    <sec id="sec-9">
      <title>A root is a class that has a set of restrictions different from</title>
      <p>its superclass(es). The set of root classes may differ in from
and to. Each root class represents an introduction point for a
set of object properties used in restrictions. Thus, root classes
are associated with their diff area. For example, in Fig 1a Cell
Cycle Stage introduced a restriction with the Is part of process
object property, which its superclass, Cell Division Process,
does not have. Thus, Cell Cycle Stage is a root class in the
{Has associated location, Is part of process} diff area. There
may be multiple root classes in a diff area (e.g., Cell Cycle
Stage and DNA Sequence Alteration in {Has associated
location, Is part of process}. Descendants of a root class in the
same diff area have restrictions with the same object
properties. For example, Cell Viability Process and Cell
Division Process, children of Cellular Process, inherit the Has
associated location restriction from their parent.</p>
    </sec>
    <sec id="sec-10">
      <title>Typically, the changes that occur at a root class will affect</title>
      <p>all of the classes that are the root’s descendants in its diff area.
To summarize changes to these subhierarchies in each diff
area we introduce diff partial-areas, which summarize
changes to the subhierarchies of classes in each diff area.</p>
    </sec>
    <sec id="sec-11">
      <title>An introduced partial-area consists of a class that is a root</title>
      <p>class in to but not a root class in from, and all of its
descendant classes that are in the same diff area in to (e.g.,
Mutation in Fig 1). A removed partial-area consists of a class
that is a root class in from but not in to and all of its
descendant classes in the same diff area in from (e.g.,
Subcellular Process in Fig 1). A modified partial-area
consists of a class that is a root class in both from and to and
all of its descendants in to. However, the set of descendants in
the diff area in to is not the same as the set of descendants in
the diff area in from (e.g., Cellular Process in Fig 1). An
unmodified partial-area consists of a root class in both from
and to and all of its descendants in the diff area. The set of
descendants in the diff area are the same in from and to.
# Diff
Partial
-areas
65</p>
    </sec>
    <sec id="sec-12">
      <title>These four kinds of diff partial-areas capture a structurally and semantically uniform set of classes that underwent the same changes. Thus, diff partial-areas are the natural building blocks for reflecting the “big picture” changes in an ontology.</title>
    </sec>
    <sec id="sec-13">
      <title>It is common to see a pair of introduced/removed partial</title>
      <p>areas in a diff partial-area taxonomy. For example, in Fig 1c,
there is a Mutation (3) introduced partial-area and a Mutation
(0) removed partial-area. This occurs when the set of object
properties used in restrictions on a root class changes between
from and to and the class is a root in both versions.</p>
    </sec>
    <sec id="sec-14">
      <title>III. METHODS</title>
    </sec>
    <sec id="sec-15">
      <title>Diff partial-area taxonomies provide a visual summary of</title>
      <p>how the structure of the Biological Process hierarchy changes
between two releases. Instead of displaying hundreds of
individual, axiom-level changes (e.g., a restriction was
removed from Mutation, a subclass was added to Cellular
Process, etc.), diff partial-area taxonomies visually capture
changes to sets of similar classes (i.e., diff partial-areas),
reducing the amount of knowledge a user has to view and
providing the “big picture” of the changes.</p>
    </sec>
    <sec id="sec-16">
      <title>Our diff partial-area taxonomy analysis of NCIt’s</title>
      <p>Biological Process hierarchy was conducted in two phases. In
the first phase we found that the hierarchy only underwent
significant changes in the Nov 2004 and Dec 2004 releases
(see Results). In the second phase we reviewed a diff
partialarea taxonomy created using Sept 2004 as from and Dec 2004
as to. To browse diff partial-area taxonomies we have
developed a software tool (described below). When an
interesting change is identified via the diff partial-area
taxonomy tool, a user can browse the explicit and implicit
causes of each change.
Fig 2. (a) An excerpt from the Sept 2004 – Dec 2004 Biological Process diff partial-area taxonomy. Each diff area is labeled with its total number/kind of diff
partial-area (many of which have been hidden). (b) An example of how changes to the set of classes summarized by a diff partial-area are displayed. (c) An
example of how a user can obtain information on why diff partial-areas were introduced or removed.</p>
    </sec>
    <sec id="sec-17">
      <title>In Table 1 we categorize different kinds of change patterns in the Biological Process diff partial-area taxonomy. In Table 2 we provide examples for each. Fig 2 shows an excerpt from the diff partial-area taxonomy derived using from and to.</title>
    </sec>
    <sec id="sec-18">
      <title>The modified partial-area Biological Process (538),</title>
      <p>located in the modified area Ø, captures two major changes.
First, it shows that 367 new classes were added to the
hierarchy and these classes were modeled without restrictions.
Additionally, 131 classes moved to Biological Process (538)
from other diff partial-areas, indicating that their restrictions
have been (explicitly or implicitly) removed in Dec 2004.</p>
      <sec id="sec-18-1">
        <title>A. Diff Partial-area Taxonomy Software Tool</title>
      </sec>
    </sec>
    <sec id="sec-19">
      <title>We have developed a software system, called BLUOWL,</title>
      <p>
        for deriving abstraction networks and diff abstraction
networks. In Ochs et al. [
        <xref ref-type="bibr" rid="ref5">5</xref>
        ] we described a prototype software
module for deriving, visualizing, and browsing diff
partialarea taxonomies. This software system is called the Diff
Module for BLUOWL. To support our analysis of NCIt we
have greatly expanded the functionality of this tool. The most
significant addition was a system for detailing why a diff
partial-area is added, removed, or modified. This allows an
editor to view the (explicit and implicit) changes that affected
the set of classes in each diff partial-area. This functionality
was used to provide the details described in Tables 1 and 2.
      </p>
      <p>With this software tool a user can select two versions of an
ontology, and a subhierarchy within the ontology, and the tool
will create a diff partial-area taxonomy. The tool enables a
user to search and navigate diff partial-area taxonomies
quickly. The visual display in the tool is based on the visual
scheme used in Fig 1c and Fig 2. Each element is selectable
and upon selection the tool provides information about what
changed in a selected diff partial-area taxonomy element.</p>
      <p>For each diff partial-area the modifications to its set of
classes are identified when the user selects the diff partial-area
in the tool’s display (an excerpt of information is shown in Fig
2b). If a diff partial-area is introduced or removed, the
structural changes that affected the root class (and all of the
descendants) are explicitly identified (e.g., addition/removal
of restrictions at the root or at an ancestor of the root). Fig 2c
provides an example of this information for the Mutation (12)
introduced partial-area in {Has initiator chemical or drug}.
Has initiator process and Has initiator chemical or drug restrictions removed by an
NCIt editor. Has result biological process restriction removed from Pathogenesis,
an ancestor, and no longer inherited. Same changes for Leukemogenesis, which was
in the removed partial-area Leukemogenesis (0) [-1] and is now in Oncogenesis (7).
Has associated location restriction removed by an NCIt editor. Five classes retained
this restriction and are in an introduced partial-area Telomere Maintenance (5).
Is part of process restriction removed from Mutation by an NCIt editor. Is part of
process restriction removed from Subcellular Process and Has associated location
restriction removed from DNA Sequence Alteration, ancestors of Mutation. Only two
classes (Mutation, Silent Mutation) are in both. Most classes from the removed
partial-area are now in introduced partial-area Gene Mutation (11), in {Has
associated location, Has initiator chemical or drug}. The Mutation (12) introduced
partial-area summarizes 10 classes from subtypes of Mutation that lost the same (and
additional) restrictions (e.g., Germline mutation).</p>
      <p>New class was modeled using a Is part of process restriction and its superclass has
no restrictions.</p>
      <p>Subhierarchy of Mutation classes that retained the Has associated location
restriction.</p>
      <p>Subhierarchy of classes from the Subcellular Process (0) [-87] removed partial-area
that retained the Is part of process restriction. Three new classes were also added to
the ontology within this subhierarchy (e.g., Farnysylation).</p>
      <p>No classes were removed from the Biological Process hierarchy in this time period.
Has initiator chemical or drug restriction removed by an NCIt editor. All classes are
now in Biological Process (538) in Ø.</p>
      <p>Is part of process restriction removed from Subcellular Process by an NCIt editor. A
subhierarchy of 56 classes moved to Biological Process (538) in Ø. The other classes
are in introduced partial-areas (e.g., a subhierarchy of 15 classes is in
PostTranslational Modification (18) in {Is part of process}).</p>
    </sec>
    <sec id="sec-20">
      <title>IV. RESULTS</title>
    </sec>
    <sec id="sec-21">
      <title>Between the Sept 2004 and Dec 2004 versions of the</title>
      <p>Biological Process hierarchy, several hundred changes were
applied to its content. The hierarchy has not undergone any
significant remodeling since that time period. For example,
between 2013 and 2016 the only change that occurred was the
addition of 32 new classes and the removal of two classes. The
significant changes that were applied between Sept 2004 and
Dec 2004 shaped the hierarchy as it still exists today. For
example, the addition of 419 (70%) new classes greatly
increased the hierarchy’s size (596 in Sept 2004).</p>
    </sec>
    <sec id="sec-22">
      <title>Other significant changes also occurred. A total of 128</title>
      <p>restrictions were removed from 115 classes and 22 restrictions
were added to 21 classes. These explicit changes, made by an
NCIt editor, implicitly affected restrictions at 54 other classes.
Table 3 shows the number of classes which had a certain
number of restrictions in the Sept 2004 and Dec 2004 releases.</p>
      <p>This extent of change is reflected by the large number of
added and removed partial-areas in the diff partial-area
taxonomy. In total, there are 118 introduced partial-areas, 105
removed partial-areas, 24 modified partial-areas, and 71
unmodified partial-areas. Table 4 lists the levels where these
diff partial-areas are located, showing that there are a large
number of removed partial-areas at higher indexed levels.</p>
    </sec>
    <sec id="sec-23">
      <title>As we mentioned in Results, the changes to the Biological</title>
      <p>Process hierarchy included a 70% increase in size and a
parallel decrease in restriction density. The remodeling
coincided with the addition of two new kinds of object
properties, Has mechanism of action and Has physiological
effect, both of which have the Chemical or Drug hierarchy as
a domain and Biological Process as a range. According to
NCIt’s editors (SdC, NT) this addition influenced the
remodeling of the Biological Process hierarchy.</p>
    </sec>
    <sec id="sec-24">
      <title>The modeling policy of NCIt does not include the addition</title>
      <p>of all possible restrictions for a class. Rather, restrictions are
typically introduced only when there is a relevant use case for
an NCIt user. This policy may explain why 367 (87.6%) of the
new classes have no restrictions, and thus, are in Biological
Process (538). The percentage of classes with no restrictions
in Dec 2004 is very high (Table 3). In an on-going study we
are reviewing the classes in Ø for missing restrictions (i.e.,
restrictions that should be added to classes to make them
modeled consistently with other classes in the Biological
process subhierarchy). A high percentage of missing
restrictions were found.</p>
      <p>The “big picture” of change reflected in the diff
partialarea taxonomy is specifically illustrated by larger introduced
or removed partial-areas. For example the Subcellular Process
(0) removed partial-area, which contained 87 classes, and the
emergence of Post-Translation Modification (18), a
subhierarchy under Subcellular Process that retained the Is
part of process restriction with a range of Post-Translational
Regulation. Some significant changes are detected via
modified partial-areas (e.g., the 367 new and 131 moved
classes in Biological Process (538), which illustrates only 53
classes had no restrictions in Sept 2004). Another example is
Cellular Process (53), with 26 new and seven moved classes.</p>
    </sec>
    <sec id="sec-25">
      <title>The removal of many restrictions is illustrated by</title>
      <p>removing two restrictions from 17 DNA Damage classes (as
summarized by the DNA Damage (0) removed partial-area),
leaving only the Has associated location and Has result
biomedical process restrictions. Similarly, the class DNA
Maintenance lost the Has associated location restriction and
moved from {Has associated location, Is part of process} to
{Is part of process} (as captured by the pair of DNA
Maintenance diff partial-areas). This example illustrates an
important benefit of the diff partial-area taxonomy in exposing
the impact of remodeling. For consistency, DNA Maintenance
should have kept the Has associated location restriction.
Whatever was the cause for the removal in 2004, this
restriction seems to be missing and it should be reintroduced.
It is typical that a remodeling project causes undesired
consequences. The diff partial-area taxonomy can help expose
them so they can be corrected.</p>
    </sec>
    <sec id="sec-26">
      <title>In this study we reviewed changes that occurred over ten</title>
      <p>years ago. However, those changes shaped the Biological
Process into its current form. We utilized a diff partial-area
taxonomy created using the Sept 2004 release as from and the
Dec 2004 release as to. This captured the end result of all the
changes. However, there were two releases in Nov 2004 that
contained intermediate changes. For example, in Nov 2004,
there were 278 classes that were descendants of Subcellular
Process and were modeled with an Is part of process
restriction. This is captured by a Subcellular Process (279) [+
192 New] modified partial-area in the diff partial-area
taxonomy created using Nov 2004. In the Dec 2004 release
the Is part of process restriction was eventually removed from
Subcellular Process and many of its descendants.</p>
      <p>We note that the diff partial-area taxonomy methodology
is not limited to NCIt’s Biological Process hierarchy; it is
applicable to any ontology. In future work we will apply the
approach on an NCIt subhierarchy that underwent recent
changes. With recent changes editors are aware of the
intended effects of each change (for Biological Process this
information was no longer available). This study will allow us
to determine how accurately the elements of a diff partial-area
taxonomy capture the intended effects of remodeling and if
diff partial-area taxonomies can be used during the ontology
design and development process, rather than as just a
mechanism for reflecting on change.</p>
      <p>
        A future component of integrating diff partial-area
taxonomies into the modeling process is a plugin for Protégé
[
        <xref ref-type="bibr" rid="ref12">12</xref>
        ] that will display a “live” diff partial-area taxonomy that
identify and summarizes change to the ontology as they are
being made. We will also improve how change information is
presented in the tool. For example, to determine where the
classes in a removed partial-area are summarized in the to
release, arrows would connect a selected removed partial-area
and the diff partial-areas that now contain its classes.
      </p>
    </sec>
    <sec id="sec-27">
      <title>VI. CONCLUSIONS</title>
    </sec>
    <sec id="sec-28">
      <title>In this paper we utilized diff partial-area taxonomies to</title>
      <p>review the structural changes resulting from a major
remodeling of NCIt’s Biological Process hierarchy. Using a
tool for deriving diff partial-area taxonomies we were able to
identify significant amounts of change in the structure of this
hierarchy. Issues related to the changes were discussed.</p>
    </sec>
    <sec id="sec-29">
      <title>ACKNOWLEDGMENTS</title>
    </sec>
    <sec id="sec-30">
      <title>Research reported in this publication was supported by the</title>
      <p>National Cancer Institute of the National Institutes of Health
under Award Number R01CA190779. The content is solely
the responsibility of the authors and does not necessarily
represent the views of the National Institutes of Health.</p>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          [1]
          <string-name>
            <surname>Fragoso</surname>
            , G., de Coronado,
            <given-names>S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Haber</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          , et al.
          <article-title>Overview and utilization of the NCI thesaurus</article-title>
          .
          <source>Comp Funct Genomics</source>
          .
          <year>2004</year>
          ;
          <volume>5</volume>
          (
          <issue>8</issue>
          ):
          <fpage>648</fpage>
          -
          <lpage>54</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          [2]
          <string-name>
            <surname>Ochs</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>He</surname>
            ,
            <given-names>Z.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Zheng</surname>
            ,
            <given-names>L.</given-names>
          </string-name>
          , et al.
          <article-title>Utilizing a structural meta-ontology for family-based quality assurance of the BioPortal ontologies</article-title>
          .
          <source>J Biomed Inform</source>
          .
          <year>2016</year>
          :In press.
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          [3]
          <string-name>
            <surname>Noy</surname>
            ,
            <given-names>N. F.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Musen</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          <article-title>Promptdiff: A fixed-point algorithm for comparing ontology versions</article-title>
          .
          <source>AAAI/IAAI</source>
          <year>2002</year>
          .
          <year>2002</year>
          :
          <fpage>744</fpage>
          -
          <lpage>50</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          [4]
          <string-name>
            <surname>Kremen</surname>
            ,
            <given-names>P.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Smid</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Kouba</surname>
            ,
            <given-names>Z.</given-names>
          </string-name>
          <article-title>OWLDiff: A Practical Tool for Comparison and Merge of OWL ontologies</article-title>
          .
          <source>22nd International Workshop on Database and Expert Systems Applications</source>
          .
          <year>2011</year>
          :
          <fpage>229</fpage>
          -
          <lpage>33</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          [5]
          <string-name>
            <surname>Ochs</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Perl</surname>
            ,
            <given-names>Y.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Geller</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          , et al.
          <article-title>Summarizing and Visualizing Structural Changes during the Evolution of Biomedical Ontologies Using a Diff Abstraction Network</article-title>
          .
          <source>Journal Of Biomedical Informatics</source>
          .
          <year>2015</year>
          ;
          <volume>56</volume>
          :
          <fpage>127</fpage>
          -
          <lpage>44</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          [6]
          <string-name>
            <surname>Motik</surname>
            ,
            <given-names>B.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Patel-Schneider</surname>
            ,
            <given-names>P. F.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Parsia</surname>
            ,
            <given-names>B.</given-names>
          </string-name>
          <article-title>OWL 2 Web Ontology Language Structural Specification and Functional Style Syntax</article-title>
          .
          <fpage>W3C</fpage>
          --
          <source>World Wide Web Consortium</source>
          ,
          <year>2009</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          [7]
          <string-name>
            <surname>de Coronado</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Wright</surname>
            ,
            <given-names>L. W.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Fragoso</surname>
            ,
            <given-names>G.</given-names>
          </string-name>
          , et al.
          <article-title>The NCI Thesaurus quality assurance life cycle</article-title>
          .
          <source>J Biomed Inform</source>
          .
          <year>2009</year>
          ;
          <volume>42</volume>
          (
          <issue>3</issue>
          ):
          <fpage>530</fpage>
          -
          <lpage>9</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          [8]
          <string-name>
            <surname>Min</surname>
            ,
            <given-names>H.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Perl</surname>
            ,
            <given-names>Y.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Chen</surname>
            ,
            <given-names>Y.</given-names>
          </string-name>
          , et al.
          <article-title>Auditing as part of the terminology design life cycle</article-title>
          .
          <source>J Am Med Inform Assoc</source>
          .
          <year>2006</year>
          ;
          <volume>13</volume>
          (
          <issue>6</issue>
          ):
          <fpage>676</fpage>
          -
          <lpage>90</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          [9]
          <string-name>
            <surname>Gonçalves</surname>
            ,
            <given-names>R. S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Parsia</surname>
            ,
            <given-names>B.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Sattler</surname>
            ,
            <given-names>U.</given-names>
          </string-name>
          <article-title>Analysing the evolution of the NCI thesaurus</article-title>
          .
          <source>Computer-Based Medical Systems (CBMS)</source>
          ,
          <year>2011</year>
          24th International Symposium on.
          <source>2011</source>
          (1
          <issue>-6</issue>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          [10]
          <string-name>
            <surname>Halper</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Gu</surname>
            ,
            <given-names>H.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Perl</surname>
            ,
            <given-names>Y.</given-names>
          </string-name>
          , et al.
          <article-title>Abstraction Networks for Terminologies: Supporting Management of “Big Knowledge”</article-title>
          .
          <source>Artificial intelligence in medicine</source>
          .
          <year>2015</year>
          ;
          <volume>64</volume>
          (
          <issue>1</issue>
          ):
          <fpage>1</fpage>
          -
          <lpage>16</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref11">
        <mixed-citation>
          [11]
          <string-name>
            <surname>Torniai</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Essaid</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Lowe</surname>
            ,
            <given-names>B.</given-names>
          </string-name>
          , et al.
          <article-title>Finding common ground: integrating the eagle-i and VIVO ontologies</article-title>
          .
          <source>ICBO</source>
          <year>2013</year>
          .
          <year>2013</year>
          :
          <fpage>46</fpage>
          -
          <lpage>9</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref12">
        <mixed-citation>
          [12]
          <string-name>
            <surname>Musen</surname>
            ,
            <given-names>M. A.</given-names>
          </string-name>
          <article-title>The protégé project: a look back and a look forward</article-title>
          .
          <source>AI Matters</source>
          .
          <year>2015</year>
          ;
          <volume>1</volume>
          (
          <issue>4</issue>
          ):
          <fpage>4</fpage>
          -
          <lpage>12</lpage>
          .
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>