<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta />
    <article-meta>
      <title-group>
        <article-title>Towards an Effective Methodology for Rapidly Developing Component-Based Domain Ontologies</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Dave Kolas</string-name>
          <email>dkolas@bbn.com</email>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Troy Self</string-name>
          <email>tself@bbn.com</email>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          <institution>BBN Technologies</institution>
          ,
          <addr-line>1300 N. 17th Street, STE 400, Arlington, VA 22209</addr-line>
          ,
          <country country="US">USA</country>
        </aff>
      </contrib-group>
      <abstract>
        <p>-As the Intelligence Community migrates from a paradigm of using disjoint, data and application stovepipes to a paradigm of shared knowledge and networkeded component services, the cost of developing appropriate domain ontologies becomes a concern. In this paper, we present a methodology for rapidly developing composite domain ontologies by linking and reusing existing ontologies. We will compare composite domain ontologies to componentbased software engineering, present a metric for measuring the compositeness of an ontology, and describe a composite domain ontology developed for real world use.</p>
      </abstract>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>I. INTRODUCTION</title>
      <p>
        While a significant amount of research effort has
been devoted to creating ontologies[
        <xref ref-type="bibr" rid="ref1">1</xref>
        ], very little
has been focused on the high-level engineering of
ontologies. Current research details interesting ways
to extract knowledge from subject matter experts[
        <xref ref-type="bibr" rid="ref2">2</xref>
        ],
ways to align and interoperate between existing
ontologies[
        <xref ref-type="bibr" rid="ref3">3</xref>
        ], and ways to extract information from
traditional relational databases into ontologies[
        <xref ref-type="bibr" rid="ref4">4</xref>
        ].
Current software products provide tools for building
ontologies, and thus make the process less painful,
but fail to explain how one should know what to
build.
      </p>
      <p>Many individuals and institutions in the Semantic
Web research area, particularly those who started
in knowledge representation work long before the
”Semantic Web” idea existed, think of the ultimate
solution of knowledge representation in the form
of one grand ontology that can cumulatively reason
about anything. The relationships in this ontology
will be objectively true in all cases, resulting in
an ontology that can be used for any purpose at
any time and in any context. This position fails to
account for the ”Web” portion of the Semantic Web;
many knowledge representations will be developed
by many individuals for many purposes, and they
may not always be exactly compatible. While the
one true ontology may one day exist, until that time,
a different type of engineering is required.</p>
      <p>
        It is our goal in this document to describe
a methodology for engineering ontologies in the
real world, where ontologies are expected to
interoperate. We will focus on designing ontologies
that will be used for a purpose, particularly in
concert with computer application software. The
recommendations in this document are derived from
research done on the GARCON-F program as well
as the collective experience of our research group
in building ontology-based applications. We hope to
espouse a set of ontology engineering principles that
closely parallel those of component-based software
design[
        <xref ref-type="bibr" rid="ref5">5</xref>
        ]. From an abstract engineering point of
view, ontologies and software components have
many common characteristics:
• Both ontologies and software components will
likely be used in a multitude of application
situations, not all of which are anticipated by
the designer.
• Both ontologies and software components will
likely evolve after their initial creation, due to
changing requirements, additional use cases,
etc.
• Unlike hardware, software components and
ontologies can be easily replicated once
developed.
• Reusing either ontologies or software
components has the potential to save a considerable
amount of engineering resources.
• Care must be taken to make both ontologies
and software components specific enough to
be useful, but general enough to be reusable.
      </p>
      <p>
        It is this set of common characteristics that
lead us to approach ontology engineering in
much the same way as component-based software
engineering[
        <xref ref-type="bibr" rid="ref6">6</xref>
        ]. When viewed abstractly, ontology
design and software design are merely two parts of
the larger task of designing an application: building
an intelligent data representation, and building
components that can act on that representation. Thus
we will define a composite domain ontology as an
ontology built from logical ontology components,
much the same way as a large piece of software is
built from software components.
      </p>
      <p>
        For the remainder of this document, when we
refer to an ontology, we will generally mean an
OWL Web Ontology Language[
        <xref ref-type="bibr" rid="ref7">7</xref>
        ] ontology. The
knowledge representation language, OWL, has
several qualities that make it particularly well suited
for building the types of purposeful ontologies we
have discussed. First, it has a natural ability to link
ontologies together. This is critical to achieving the
goals of reuse. Second, it takes a delicate balance
between expressivity and computational tractability,
yielding ontologies that are practical. Third, the
large and growing number of existing tools makes
interaction with software components significantly
easier. That said, we hope that much of what is
described here is applicable to engineering other
knowledge representations, particularly those with
similar properties to OWL.
      </p>
    </sec>
    <sec id="sec-2">
      <title>II. ONTOLOGY DESIGN GOALS</title>
      <p>In this section, we will describe the goals of
designing a composite ontology. The subsections
will ideally both propose general guidelines by
which ontologies should be designed and justify
these guidelines. To discuss the following
guidelines, a working definition of compositeness will
be required. In this section, we present an intuitive,
loose definition of compositeness; a more formal,
measurable definition of compositeness will be
presented later.</p>
      <p>The compositeness of an ontology is defined as
the degree to which its content is made up primarily
of other ontologies, and the degree to which these
ontologies are made up of other ontologies, etc.</p>
      <p>With that definition established, we examine why
and when composite ontologies should be created.</p>
      <sec id="sec-2-1">
        <title>A. Time Savings</title>
        <p>The most immediate benefit achieved by building
composite ontologies is elimination of much of the
time required to design and build them. Extracting
knowledge from subject matter experts can be a
difficult and expensive process, and encoding that
knowledge into a structured form can be quite
difficult as well. By building an ontology from the
building blocks of other ontologies, it is possible to
significantly reduce the amount of time required to
build the ontology.</p>
        <p>Just as in software, the time savings gained by
reusing another ontology is not totally cost-free;
incorporating another ontology subjects the new
ontology’s designer to the design decisions of the
component ontologies. However, as with software,
the time spent on tuning the incorporation of an
ontology into a larger ontology is most often vastly
less than the time spent developing a new ontology
from scratch.</p>
      </sec>
      <sec id="sec-2-2">
        <title>B. Interaction with Software</title>
        <p>As the number and scope of Semantic Web
applications increases, so too will increase the
symbiosis between these applications and their related
ontologies. For instance, in the GARCON-F
program, the client UI runs within ESRI’s ArcMap
software, providing the capacity to display semantic
annotations in a geospatial context. This software
expects specific geospatial and temporal ontologies:
GeoRSS for geometries and OWL-Time for
temporal information. Any additional ontological
information is processed dynamically by the software.</p>
        <p>Consider the case of an ontology designer
creating a new ontology to be used with this system.
It is likely that whatever domain he or she is
attempting to represent requires geospatial and
temporal information; otherwise, the software would
not be applicable. The ontology designer can choose
between creating their own definitions for space
and time, or merely adopting GeoRSS and
OWLTime. The former is possible by adding ontology
translations to the system, but the latter makes
the client software instantly able to process data
from the new ontology with no software changes
required.</p>
        <p>Many applications that deal with Semantic Web
data function this way. A few ontologies are treated
specially, being used for a particular purpose. Other
ontologies provide extra information or linkage to
other applications. In this paradigm, reusing
ontology components provides significantly greater tool
interoperability.</p>
      </sec>
      <sec id="sec-2-3">
        <title>C. Reasoning and Translation</title>
        <p>Even after an ontology has been created,
additional layers of reasoning are often added. These
might come in the form of additional SWRL rules,
translations, or linkages between ontologies. For
instance, there may be existing translations from
GeoRSS to another spatial representation. These
translations then inherently work on any ontology
that incorporates GeoRSS for spatial representation.
This is very much like the automatic interoperability
of software components described above. In a sense,
this type of reuse driven by composite ontologies
multiplies the effectiveness of software reuse with
composite ontologies. If there are three tools that
use three different ontologies for one particular
purpose, and there are existing mappings between
these ontologies, then a designer of a new ontology
can choose between any of the three and get the
automatic benefit of the translations and the software.</p>
      </sec>
      <sec id="sec-2-4">
        <title>D. Scope and Applicability</title>
        <p>It is not our intention to suggest that all
ontologies should be highly composite. We generally
expect the compositeness of an ontology to vary
in accordance with the simplicity or abstractness
of the task for which the ontology is designed.
The more abstract an associated task is, the higher
the level of appropriate compositeness. Consider
the GeoRSS ontology mentioned previously. This
ontology has a very particular, focused purpose. The
goal for the GeoRSS ontology is to attach point and
polygon spatial data to other ontological entities.
Because this purpose is narrow in scope and widely
applicable, it makes sense for the GeoRSS ontology
to be simple and not composite.</p>
        <p>On the other hand, consider an ontology for air
defense, as developed earlier in the GARCON-F
program. The goal for an air defense ontology might
be to annotate rich data about air defense scenarios,
to reason about these annotations, and to generate
reports about them. This ontology must be large
in scope but only narrowly applicable. This is the
scenario in which it makes sense for an ontology to
be very highly composite.</p>
        <p>Obviously there is a large amount of space
between these two extremes, and finding the
appropriate level of compositeness for an ontology is part of
the process of creating it. We will attempt to give
guidance on this as well in the following sections.</p>
      </sec>
    </sec>
    <sec id="sec-3">
      <title>III. MEASURING COMPOSITENESS IN ONTOLOGIES</title>
      <sec id="sec-3-1">
        <title>A. Relationship to Software Reuse Metrics</title>
        <p>
          Software metrics are effectively used to
alleviate quality and performance concerns during
development[
          <xref ref-type="bibr" rid="ref8">8</xref>
          ]. We believe that analogue metrics
can be developed to measure quality, reuse, and
performance in ontologies. These metrics could
eventually be applied to measure these
characteristics in existing ontologies as well as to guide the
planning and development of new ontologies.
        </p>
        <p>
          In software, reuse and compositeness is measured
by quantifying abstract constructs of the software
development language, such as lines of code and
the number of references between components[
          <xref ref-type="bibr" rid="ref9">9</xref>
          ].
While we envision an eventual set of metrics for
measuring quality and reuse of ontologies, we
begin in this paper by defining a single metric for
measuring the compositeness of an ontology.
        </p>
        <p>The goal of a compositeness metric for an
ontology is to have a quick estimate of the ontology’s
modularity. The more composite the ontology is, the
more likely it is that one could repurpose significant
parts of the ontology and thus minimize further
development.</p>
      </sec>
      <sec id="sec-3-2">
        <title>B. Definitions</title>
        <p>• The compositeness of a given ontology X is
the function C(X).
• The size of a given ontology X is the function</p>
        <p>S(X).
• An ontology X importing an ontology Y is
defined by the relation I(X, Y ).
• The set of all ontologies imported by X is</p>
        <p>I(X).</p>
      </sec>
      <sec id="sec-3-3">
        <title>C. Desirable Properties of a Compositeness Metric</title>
        <p>If an ontology imports another ontology, its
compositeness is greater than the imported ontology.</p>
        <p>I(A, B) → C(A) &gt; C(B)</p>
        <p>If two ontologies import the same ontology, and
one of the importing ontologies is smaller than the
other, that ontology has higher compositeness.</p>
        <p>I(A, B) ∧ I(C, B) ∧ S(A) &lt; S(C)</p>
        <p>→ C(A) &gt; C(C)</p>
        <p>If two ontologies that are otherwise identical
import different ontologies, the one that imports the
more composite ontology will have higher
compositeness.</p>
        <p>I(A, B) ∧ I(A , C) ∧ C(B) &gt; C(C)</p>
        <p>→ C(A) &gt; C(A )</p>
        <p>If one ontology that is otherwise identical to
a second ontology imports an ontology that the
second does not, that ontology will have higher
compositeness.
(I(A, B) ∧ I(A , B) ∧ I(A , C)
∧ ∀X : (I(A, X) ∧ X = C))</p>
        <p>→ C(A ) &gt; C(A)</p>
      </sec>
      <sec id="sec-3-4">
        <title>D. A Compositeness Metric</title>
        <p>To meet the desired properties, we define a
compositeness metric as illustrated in Equation 1.</p>
        <p>C(X) =
y∈I(X)</p>
        <p>S(y)
C(y) + y∈I(X)</p>
        <p>S(X)
(1)</p>
      </sec>
    </sec>
    <sec id="sec-4">
      <title>IV. BUILDING COMPOSITE ONTOLOGIES</title>
      <p>This section will define a process for creating
effective composite ontologies. By following this
process, ontology engineers should create an
ontology with maximum portability and reusability.</p>
      <sec id="sec-4-1">
        <title>A. Start With the Application Domain</title>
        <p>The most important aspect of creating a useful
ontology is starting with a particular application in
mind. By starting with an application, the
knowledge engineer immediately creates a scope that the
ontology will need to fulfill, and thus prevents the
ontology from slowly expanding to represent
unnecessary concepts and relationships. This process
is analogous to software requirements gathering. In
order to build an effective ontology, the knowledge
engineer must answer the following questions:
• What questions/queries should the user or
software client be able to ask of the ontology?
• What should the instance data look like?
• What types of inference will the ontology need
to provide?</p>
        <p>If a concept or relationship is not part of any of
these three sets, then it need not be included in the
ontology.</p>
      </sec>
      <sec id="sec-4-2">
        <title>B. Divide the Goals</title>
        <p>Once the overall scope of the ontology has been
established, the next step in the process is to divide
the scoped relationships and concepts into logical
subcomponents. These components could be both
aspects of the data (geolocation, temporal
information, provenance information) or dividable subparts
of the overall domain (vehicles, ground systems, etc
for air defense). When attempting to partition the
ontology, the following questions should be asked:
• Could this partition be successfully reused
without the other partitions?
• Would making this partition allow a piece of
software to work without understanding the
other partitions?
• Are there known existing ontologies that fulfill
part of the overall scope?
• Can some part of the overall ontology be
viewed as a specialization of another part?
By answering these questions, the knowledge
engineer should be able to tentatively create partitions
of the ontology, and create a dependency graph
between the parts.</p>
      </sec>
      <sec id="sec-4-3">
        <title>C. Identify Reusable Ontologies</title>
        <p>Once partitions have been identified, all attempts
should be made to fill them in with suitable existing
ontologies. The applicability of a given ontology
component should be evaluated by how well it fits
the partition, how widely it is used, and the quality
of its construction.</p>
        <p>It is entirely possible that an ontology that
does not perfectly fit a partition may be desirable
nonetheless, especially if it is already in wide
use. The ontology component breakdown may be
revisited during this step to accommodate existing
ontologies with slightly larger or smaller scope.</p>
      </sec>
      <sec id="sec-4-4">
        <title>D. Develop New Component Ontologies</title>
        <p>Once existing ontologies have been worked into
the overall ontology, the remaining pieces must
be created. Other work addresses this part of the
process; here we only advise careful attention to
the scope of the ontology created in the first step.
If a particular concept will never be directly inserted
as data, queried for, or inferred over, it should not
be included.</p>
      </sec>
      <sec id="sec-4-5">
        <title>E. Link Component Ontologies Together</title>
        <p>The final step is linking the ontology
components together. This is accomplished by importing
ontology components from the other components,
mirroring the dependency graph created. As in
software, care should be taken to avoid circular
dependencies. While a circular dependency will not
prevent compilation as in software, it will
significantly reduce reusability of any of the components
involved.</p>
        <p>When appropriate, ontology components should
link directly into the components they inherit from
via subclass, subproperty, or restrictions. This
provides the cohesion between parts necessary for
effective use of the overall ontology. Occasionally,
if multiple components are fulfilled by existing
ontologies, glue components will be required to
contain these relationships between parts. It is
preferable to create a new linkage component rather
than directly changing an existing ontology.</p>
      </sec>
    </sec>
    <sec id="sec-5">
      <title>V. EXAMPLE: BUILDING A RAID MISSION PLANNING ONTOLOGY</title>
      <sec id="sec-5-1">
        <title>A. Geospatial Semantic Annotation Tool</title>
        <p>
          The Geospatial Semantic Annotation Tool
(gSAT) is a platform for capturing imagery
annotations using ontologies[
          <xref ref-type="bibr" rid="ref10">10</xref>
          ]. Previously, gSAT
had been used for annotating imagery intelligence
in the Air Defense domain. The purpose of this
exercise was to rapidly develop a new domain
ontology that could be used within gSAT without
changing any of the software. The new domain was
Raid Mission Planning as defined by the United
States Marine Corps for Counter-Insurgency (CoIn)
operations. A raid mission is a military mission
where forces quickly advanced on a chosen target
and then leave. Examples of a raid can include
attacking a known enemy location to eliminate its
threat capability or evacuation procedures, such as
removing non-essential staff from an embassy.
        </p>
      </sec>
      <sec id="sec-5-2">
        <title>B. Raid Ontology</title>
        <p>The concepts and relationships defined for this
ontology are based on discussions with United</p>
        <p>States Marine Corp imagery analysts. The raid
ontology needed to include concepts and relationships
necessary to represent an observation of a physical
feature at some geospatial location at a particular
time. The representation for an observation
consisting of a what, where, and when were reused
from gSAT. The concepts to be annotated included
routes, landing zones, assault support vehicles,
enemy facility types, and dangerous locations, such as
potential IED areas and potential sniper positions.</p>
        <p>Figure 1 shows the various ontology components
in the Raid Mission ontology and their
dependencies. Each circle represents an ontology and
includes its compositeness score according to the
metric defined in Equation 1. The ontologies that
show C(X ) = 0 do not import any other ontologies.</p>
        <p>As the diagram shows, the Raid ontology is the
most abstract, and directly or indirectly imports
all of the other ontologies. The most foundational
ontologies are at the bottom, and are imported into
the Raid ontology along two different paths. Most
of the components in the ontology are reused.</p>
        <p>Reusing the ontologies as shown above allows the
gSAT annotation system to create spatiotemporal
semantic annotations in the Raid domain without
changing any aspect of the software. This is because
the software only needs to directly understand the
parts of the ontology that were reused: the
foundational, temporal, and spatial portions. Thus in this
case the reuse of ontologies has led to 100 percent
reuse of the software.</p>
      </sec>
    </sec>
    <sec id="sec-6">
      <title>VI. FURTHER RESEARCH</title>
      <p>
        This document describes the initial research into
a formal methodology for developing composite
domain ontologies. The compositeness metric defined
in this document requires further testing against
a larger reference set of ontologies to validate
its correctness. Further exploration into other
metrics of ontology reuse is necessary. It is
important to encorporate metrics that consider the
ontology’s internal complexity, cohesion, and other
measures[
        <xref ref-type="bibr" rid="ref11">11</xref>
        ]. Since component-based software has
multiple metrics for measuring quality, it is
expected that component-based ontologies should also
have multiple metrics. The compositeness metric
described here only considers the size of ontologies
and the single relationship of imports between
them. Future metrics must consider the amount of
linking and semantic complexity between linked
ontologies.
      </p>
    </sec>
    <sec id="sec-7">
      <title>VII. CONCLUSION</title>
      <p>In this document we have likened the process
of engineering ontologies to component-based
software engineering. We have demonstrated the
benefits of designing ontologies this way, and defined a
process for creating such ontologies. We have also
defined a metric for determining how composite a
particular ontology is. Our hope is that this analysis
will help others create more effective ontologies in
the future.</p>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          [1]
          <string-name>
            <given-names>D.</given-names>
            <surname>Bianchini</surname>
          </string-name>
          , V. De Antonellis, and
          <string-name>
            <given-names>M.</given-names>
            <surname>Melchiori</surname>
          </string-name>
          , “
          <article-title>Domain ontologies for knowledge sharing and service composition in virtual districts</article-title>
          ,
          <source>” in Database and Expert Systems Applications</source>
          ,
          <year>2003</year>
          . Proceedings. 14th International Workshop on, Sept.
          <year>2003</year>
          , pp.
          <fpage>589</fpage>
          -
          <lpage>594</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          [2]
          <string-name>
            <given-names>X.</given-names>
            <surname>Wang</surname>
          </string-name>
          ,
          <string-name>
            <given-names>X.</given-names>
            <surname>Wang</surname>
          </string-name>
          , and
          <string-name>
            <given-names>F.</given-names>
            <surname>Wang</surname>
          </string-name>
          , “
          <article-title>How to use class axioms to model ontology effectively in owl,” in Knowledge Acquisition</article-title>
          and Modeling Workshop,
          <year>2008</year>
          . KAM Workshop
          <year>2008</year>
          . IEEE International Symposium on,
          <source>Dec</source>
          .
          <year>2008</year>
          , pp.
          <fpage>601</fpage>
          -
          <lpage>604</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          [3]
          <string-name>
            <given-names>J.</given-names>
            <surname>Sampson</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Lanzenberger</surname>
          </string-name>
          , and
          <string-name>
            <given-names>C.</given-names>
            <surname>Veres</surname>
          </string-name>
          , “
          <article-title>Facilitating interoperability in semantic web applications using ontologies,” in Complex</article-title>
          ,
          <source>Intelligent and Software Intensive Systems</source>
          ,
          <year>2008</year>
          . CISIS 2008. International Conference on,
          <year>March 2008</year>
          , pp.
          <fpage>233</fpage>
          -
          <lpage>239</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          [4]
          <string-name>
            <given-names>Z.</given-names>
            <surname>Qu</surname>
          </string-name>
          and
          <string-name>
            <given-names>S.</given-names>
            <surname>Tang</surname>
          </string-name>
          , “
          <article-title>Research on transforming relational database into enriched ontology,” in Advanced Computer Theory</article-title>
          and Engineering,
          <year>2008</year>
          . ICACTE '08. International Conference on,
          <source>Dec</source>
          .
          <year>2008</year>
          , pp.
          <fpage>749</fpage>
          -
          <lpage>753</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          [5]
          <string-name>
            <given-names>L.</given-names>
            <surname>Etzkorn</surname>
          </string-name>
          and
          <string-name>
            <given-names>H.</given-names>
            <surname>Delugach</surname>
          </string-name>
          , “
          <article-title>Towards a semantic metrics suite for object-oriented design,”</article-title>
          <source>in Technology of ObjectOriented Languages and Systems</source>
          ,
          <year>2000</year>
          . TOOLS 34.
          <string-name>
            <surname>Proceedings</surname>
          </string-name>
          . 34th International Conference on,
          <year>2000</year>
          , pp.
          <fpage>71</fpage>
          -
          <lpage>80</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          [6]
          <string-name>
            <given-names>A.</given-names>
            <surname>Farooq</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Shah</surname>
          </string-name>
          , and
          <string-name>
            <given-names>K.</given-names>
            <surname>Asif</surname>
          </string-name>
          , “
          <article-title>Design of ontology in semantic web engineering process,” in High Capacity Optical Networks</article-title>
          and
          <string-name>
            <given-names>Enabling</given-names>
            <surname>Technologies</surname>
          </string-name>
          ,
          <year>2007</year>
          . HONET 2007. International Symposium on,
          <source>Nov</source>
          .
          <year>2007</year>
          , pp.
          <fpage>1</fpage>
          -
          <lpage>6</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          [7]
          <string-name>
            <given-names>M.</given-names>
            <surname>Dean</surname>
          </string-name>
          and
          <string-name>
            <surname>G</surname>
          </string-name>
          . Schrieber, Eds.,
          <source>OWL Web Ontology Language Reference. W3C Recommendation</source>
          ,
          <year>February 2004</year>
          , http://www.w3.org/TR/2004/REC-owl-ref-
          <volume>20040210</volume>
          /.
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          [8]
          <string-name>
            <given-names>S.</given-names>
            <surname>Sedigh-Ali</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Ghafoor</surname>
          </string-name>
          , and R. Paul, “
          <article-title>Metrics and models for cost and quality of component-based software,” in Object-Oriented Real-Time Distributed Computing</article-title>
          ,
          <year>2003</year>
          . Sixth IEEE International Symposium on, May
          <year>2003</year>
          , pp.
          <fpage>149</fpage>
          -
          <lpage>155</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          [9]
          <string-name>
            <given-names>W.</given-names>
            <surname>Frakes</surname>
          </string-name>
          and
          <string-name>
            <given-names>C.</given-names>
            <surname>Terry</surname>
          </string-name>
          , “
          <article-title>Software reuse: metrics and models</article-title>
          ,
          <source>” ACM Comput. Surv.</source>
          , vol.
          <volume>28</volume>
          , no.
          <issue>2</issue>
          , pp.
          <fpage>415</fpage>
          -
          <lpage>435</lpage>
          ,
          <year>1996</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          [10]
          <string-name>
            <given-names>T.</given-names>
            <surname>Self</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D.</given-names>
            <surname>Kolas</surname>
          </string-name>
          , and
          <string-name>
            <given-names>M.</given-names>
            <surname>Dean</surname>
          </string-name>
          , “
          <article-title>Ontology-driven imagery analysis</article-title>
          ,
          <source>” in Proceedings of the Second International Ontology for the Intelligence Community Conference OIC2007</source>
          ,
          <year>2007</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref11">
        <mixed-citation>
          [11]
          <string-name>
            <given-names>Y.</given-names>
            <surname>Ma</surname>
          </string-name>
          , X. Ma, S. Liu, and
          <string-name>
            <given-names>B.</given-names>
            <surname>Jin</surname>
          </string-name>
          , “
          <article-title>A proposal for stable semantic metrics based on evolving ontologies</article-title>
          ,
          <source>” in Artificial Intelligence</source>
          ,
          <year>2009</year>
          . JCAI '09. International Joint Conference on,
          <source>April</source>
          <year>2009</year>
          , pp.
          <fpage>136</fpage>
          -
          <lpage>139</lpage>
          .
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>