<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta />
    <article-meta>
      <title-group>
        <article-title>SABiO: Systematic Approach for Building Ontologies</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Ricardo de Almeida Falbo</string-name>
          <email>falbo@inf.ufes.br</email>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          <institution>Ontology and Conceptual Modeling Research Group (NEMO), Federal University of Espírito Santo</institution>
          ,
          <addr-line>Vitória</addr-line>
          ,
          <country country="BR">Brazil</country>
        </aff>
      </contrib-group>
      <abstract>
        <p>This paper presents the new version of SABiO - a Systematic Approach for Building Ontologies. SABiO focus on the development of domain ontologies, and also propose support processes. SABiO distinguishes between reference and operational ontologies, providing activities that apply to the development of both types of domain ontologies.</p>
      </abstract>
      <kwd-group>
        <kwd>ontology engineering</kwd>
        <kwd>domain ontology</kwd>
        <kwd>ontology development</kwd>
      </kwd-group>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>Introduction</title>
      <p>
        Nowadays, the Ontology Engineering community recognizes the great help Software
Engineering can provide for improving practices in the ontology engineering process.
Like any complex software development activity, building quality ontologies requires
appropriated methods and tools. In 1997, we defined SABiO, a Systematic Approach
for Building Ontologies, whose first version was published in [
        <xref ref-type="bibr" rid="ref1">1</xref>
        ]. Since then, SABiO
has been used for building several domain ontologies, such as ontologies for the
software process domain [
        <xref ref-type="bibr" rid="ref2">2</xref>
        ], and cardiology domain [
        <xref ref-type="bibr" rid="ref3">3</xref>
        ]. In [
        <xref ref-type="bibr" rid="ref4">4</xref>
        ], we discussed strong
points and weakness of the method, presenting some lessons learned and
improvement opportunities to evolve it. Minor changes, such as using a UML profile as
modeling language, were done in SABiO at that time.
      </p>
      <p>
        SABiO was originally conceived for supporting the development of domain
reference ontologies. By domain reference ontology we mean a domain ontology that is
built with the goal of making the best possible description of the domain. It is a
solution-independent specification (conceptual model) with the aim of making a clear and
precise description of domain entities for the purposes of communication, learning
and problem-solving. Once users have already agreed on a common
conceptualization, operational versions (machine-readable ontologies) of a reference ontology can
be implemented. Contrary to reference ontologies, operational ontologies are designed
with the focus on guaranteeing desirable computational properties [
        <xref ref-type="bibr" rid="ref5">5</xref>
        ].
      </p>
      <p>
        As pointed by several works, such as [
        <xref ref-type="bibr" rid="ref5 ref6">5, 6</xref>
        ], ideally domain ontologies should be
developed grounded in foundational (top-level) ontologies. Concepts and relations in
a domain ontology must be previously analyzed in the light of a foundational
ontology. The idea behind ontological analysis is to provide a sound foundation for
modeling concepts, if assumed that such concepts are aimed at representing reality [
        <xref ref-type="bibr" rid="ref7">7</xref>
        ].
Among the benefits of ontological analysis, we can cite [
        <xref ref-type="bibr" rid="ref8 ref9">8, 9</xref>
        ]: (i) the rigorous
definition of models, in terms of real-world semantics; (ii) the identification of problems in
the definition, interpretation or usage of concepts; and (iii) recommendations for
model formality improvements. We ourselves experienced these benefits when
ontologically analyzing the software process ontology [
        <xref ref-type="bibr" rid="ref2">2</xref>
        ], when several problems in the
ontology were detected and the entire ontology re-engineered [
        <xref ref-type="bibr" rid="ref10">10</xref>
        ].
      </p>
      <p>In order to incorporate these experiences, and the important distinction between
reference and operational ontologies, we evolved SABiO. Moreover, we take the
opportunity for incorporating to SABiO best practices commonly adopted in Software
Engineering and Ontology Engineering. The main differences between versions 1.0
and 2.0 of SABiO are: (i) SABiO 2.0 extends the original development process to
consider the design, implementation and test of operational ontologies; (ii) SABiO 2.0
considers new support processes for knowledge acquisition, reuse, and configuration
management; (iii) SABiO 2.0 recognizes the importance of the use of foundational
ontologies in the development of domain ontologies, and proposes applying
ontological analysis during ontology capture; (v) Regarding ontology evaluation, SABiO 2.0
addresses both ontology verification and validation.</p>
      <p>This paper presents the new version of SABiO, and it is organized as follows. In
Section 2, we present an overview of SABiO 2.0. Section 3 presents SABiO's
development process, describing each one of its activities, as well as the roles involved in
their accomplishment, artifacts required and produced, and some techniques and
guidelines for performing them. Section 4 presents SABiO's support processes.
Section 5 discusses related works. Finally, Section 6 presents our final considerations.
2</p>
    </sec>
    <sec id="sec-2">
      <title>An Overview of SABiO</title>
      <p>Figure 1 presents an overview of SABiO 2.0. As this figure shows, SABiO
development process comprises five main phases: (i) Purpose identification and requirements
elicitation; (ii) ontology capture and formalization; (iii) design; (iv) implementation;
and (v) test. Support processes are performed in parallel to the development process.
Although Figure 1 suggests a somewhat sequential workflow, SABiO does not
prescribe any specific life cycle model. Thus, life cycle models such as Incremental and
Spiral can be adopted. We recommend incremental and iterative development.</p>
      <p>
        SABiO focuses on the development of domain ontologies. Two types of domain
ontologies can be developed: reference and operational ontologies. As
aforementioned, a reference ontology is a special kind of conceptual model. An operational
ontology, in turn, is a machine-readable implementation version of the ontology,
designed with the focus on guaranteeing some desirable computational properties [
        <xref ref-type="bibr" rid="ref5">5</xref>
        ].
Thus, before implementing an operational ontology, a design phase should be
accomplished taking technological non-functional requirements and the ontology
implementation environment into account. If someone is interested in building a reference
ontology, then she shall accomplish only the first three activities of the development
process. If someone wants to build an operational ontology, then she has to perform
the entire development process.
      </p>
      <sec id="sec-2-1">
        <title>Development Process</title>
      </sec>
      <sec id="sec-2-2">
        <title>Design</title>
      </sec>
      <sec id="sec-2-3">
        <title>Implementation</title>
        <sec id="sec-2-3-1">
          <title>Operational Ontology</title>
        </sec>
      </sec>
      <sec id="sec-2-4">
        <title>Testing</title>
        <p>cquA Know
is le
i
t
ino ged</p>
        <p>D
o
c
u
m
e
n
t
a
t
i
o
n</p>
        <p>C
o
n
f
i
g
u
r
a
t
i
o
n
M
a
n
a
g
e
m
e
n
t</p>
        <p>E
v
a
l
u
a
t
i
o
n</p>
        <p>R
e
u
s
e
SABiO development process comprises five main phases. Each phase is composed of
activities, which are to be performed by workers playing certain roles. The main roles
considered in SABiO are: (i) domain expert, who is a specialist in the ontology
domain, and provides the knowledge that is to be modeled and implemented in the
domain ontology; (ii) ontology user, representing someone who intends to use the
ontology for a given purpose; (iii) ontology engineer, who is responsible for the reference
ontology, and thus is responsible for the initial phases of the ontology development
process; (iv) ontology designer, who is responsible for the design of an operational
ontology; (v) ontology programmer, who is responsible for implementing an
operational ontology; (vi) ontology tester, who is responsible for testing an operational
ontology. It is worthwhile to point out that a given worker can play several roles in a
given project. For instance, generally the roles of ontology designer and ontology
programmer are played by the same worker, since both roles require great knowledge
regarding the ontology implementation environment.</p>
        <p>Ontology Purpose Identification and Requirements Elicitation</p>
        <p>The first phase in SABiO's development process is Purpose Identification and
Requirements Elicitation. As Figure 2 shows, this phase comprises four activities that
occur in an iterative way, all of them involving the participation of the ontology
engineer, domain experts and potential ontology users.</p>
        <p>Ontology User
Ontology Engineer</p>
        <sec id="sec-2-4-1">
          <title>Ontology</title>
        </sec>
        <sec id="sec-2-4-2">
          <title>Modularization</title>
        </sec>
        <sec id="sec-2-4-3">
          <title>Requirements</title>
        </sec>
        <sec id="sec-2-4-4">
          <title>Elicitation</title>
        </sec>
        <sec id="sec-2-4-5">
          <title>Competency</title>
        </sec>
        <sec id="sec-2-4-6">
          <title>Questions</title>
        </sec>
        <sec id="sec-2-4-7">
          <title>Identification</title>
          <p>
            Initially, we need to identify the ontology purpose and its intended uses. Once
defined the ontology purpose and intended uses, we should elicit its requirements.
Ontology requirements, like software requirements, can be divided into functional and
non-functional requirements. Functional requirements refer to the knowledge
(content) to be represented by the ontology, and can be stated as competency questions,
i.e. the questions that the ontology should be able to answer [
            <xref ref-type="bibr" rid="ref11">11</xref>
            ]. By establishing the
competency questions, we reach an effective way to determine what is relevant to the
ontology and what is not, i.e. we define its scope. Moreover, we provide a way for its
evaluation. On the other hand, non-functional requirements refer to the characteristics,
qualities and general aspects not related to the ontology content [
            <xref ref-type="bibr" rid="ref12">12</xref>
            ]. Non-functional
requirements can be divided into: (i) ontology quality attributes, which refer to
characteristics that the ontology, as a software product, should present, such as reasoning
performance and availability (for the case of operational ontologies), usability (e.g.,
understandability), maintainability (e.g., extensibility); (ii) project requirements,
which are requirements derived from the ontology project, such as process
requirements (e.g., adherence to process models and document templates), implementation
requirements (e.g., whether the operational ontology must be implemented in a given
machine-readable language), delivery requirements (e.g., time to market),
consensusrelated requirements (e.g., who must agree with the ontology); (iii) intended
usesrelated requirements, which take the intended uses for the ontology into account, such
as knowledge sources-related requirements (e.g., whether the terminology to be used
in the ontology must be taken from standards), interoperability requirements (e.g.,
whether the ontology must be grounded in a specific foundational ontology in order to
easy its integration with other ontologies already existing).
          </p>
          <p>
            Functional requirements should be written in the form of competency questions
(CQs). CQs can be identified using different strategies, and written in different
granularity levels. Some strategies for identifying CQs are [
            <xref ref-type="bibr" rid="ref12">12</xref>
            ]: (i) Top-down: the ontology
engineer starts with complex questions that are decomposed in simpler ones; (ii)
Bottom-up: the ontology engineer starts with simple questions that are composed to
create complex ones; (iii) Middle out: the ontology engineer starts just writing down
important questions that are composed or decomposed later on to form abstract and
simple questions, respectively. Whatever strategy used, in the end, we need to achieve
a set of CQs in both levels. Simple CQs are important for deriving test cases; complex
and more abstract CQs are important to guide ontology modularization.
          </p>
          <p>
            If the domain of interest is complex, we need to modularize the ontology. Ontology
modularization consists in identifying modules (or sub-ontologies) that can be
considered separately while they are interlinked with other modules [
            <xref ref-type="bibr" rid="ref12">12</xref>
            ]. The benefits of
modularizing ontologies include [
            <xref ref-type="bibr" rid="ref12">12</xref>
            ]: (i) to facilitate the development and
maintenance of the ontology by dividing it in loosely coupled, self-contained modules; (ii) to
facilitate the reuse of parts of the ontology; (iii) to improve performance by enabling
distributed processing. However, there is no universal way to modularize an ontology
and the choice of a particular approach should be guided by the ontology
requirements [
            <xref ref-type="bibr" rid="ref12">12</xref>
            ]. SABiO suggests decomposing the ontology into sub-ontologies.
Techniques and criteria for ontology partitioning, such as the ones proposed in [
            <xref ref-type="bibr" rid="ref13">13</xref>
            ], apply.
Regarding the criteria, we suggest considering at least the following ones:
independence, cohesion, and size. UML package diagrams should be developed for graphically
showing the sub-ontologies and the dependencies between them. If a sub-ontology
so2 refers to a concept defined in another sub-ontology so1, then so2 depends on so1.
The four rules for partitioning ontologies proposed in [
            <xref ref-type="bibr" rid="ref12">12</xref>
            ] are also useful here.
          </p>
          <p>The four activities in Figure 2 must be performed in an iterative way. Based on the
purpose initially established for the ontology and the intended uses elicited with
potential users of the ontology, requirements are elicited and competency questions are
outlined. The ontology engineer may then identify sub-ontologies and allocate the
competency questions to the sub-ontologies identified. The growing understanding of
the domain can lead to a better understanding of the purpose of the ontology and the
identification of new intended uses, starting a new cycle.</p>
          <p>Ontology Capture and Formalization</p>
          <p>The main goal of this phase is to capture the domain conceptualization based on
the competency questions. As discussed in the Introduction of this paper, SABiO
suggests that concepts and relations in a reference domain ontology are analyzed in
the light of a foundational ontology. Ontological analysis can be applied in several
scenarios: portions of the ontology that are developed from scratch should be
analyzed, as well as non-ontological and ontological resources to be reused. A
foundational ontology must be selected for performing this task.</p>
          <p>
            The relevant concepts and relations should be identified and organized. A graphical
model is a key instrument for supporting communication, meaning negotiation and
consensus establishment with domain experts. For building reference domain
ontologies, highly-expressive languages should be used to create strongly axiomatized
ontologies that approximate as well as possible to the ideal ontology of the domain. The
focus on these languages should be on representation adequacy, since the resulting
specifications are intended to be used by humans [
            <xref ref-type="bibr" rid="ref5">5</xref>
            ]. An example of an ontology
representation language that is suitable for reference ontologies is OntoUML, a UML
class diagram profile that incorporates important foundational distinctions made by
the Unified Foundational Ontology (UFO) [
            <xref ref-type="bibr" rid="ref14">14</xref>
            ].
          </p>
          <p>
            As Figure 3 shows, this phase begins with a conceptual modeling activity. During
conceptual modeling, initially the ontology engineer should identify the main
concepts and relations in the domain. Concepts should be properly organized in
taxonomies. Since SABiO suggests the use of OntoUML, the ontology engineer should
classify each concept according to the types defined in OntoUML (kind, subkind, phase,
role, category, rolemixin etc.). Moreover, OntoUML constraints for relating these
types should be respected. Ontological patterns and modeling rules inherent to
OntoUML, such as the ones for modeling subkinds, phases, roles [
            <xref ref-type="bibr" rid="ref15">15</xref>
            ] and rolemixins
[
            <xref ref-type="bibr" rid="ref14">14</xref>
            ], should be applied in order to achieve consistent conceptual models.
          </p>
        </sec>
        <sec id="sec-2-4-8">
          <title>Ontology Capture and</title>
        </sec>
        <sec id="sec-2-4-9">
          <title>Formalization</title>
        </sec>
        <sec id="sec-2-4-10">
          <title>Reference Ontology</title>
        </sec>
        <sec id="sec-2-4-11">
          <title>Formal Axioms</title>
        </sec>
        <sec id="sec-2-4-12">
          <title>Definition</title>
        </sec>
        <sec id="sec-2-4-13">
          <title>Dictionary of</title>
        </sec>
        <sec id="sec-2-4-14">
          <title>Terms</title>
        </sec>
        <sec id="sec-2-4-15">
          <title>Definition</title>
        </sec>
        <sec id="sec-2-4-16">
          <title>Conceptual</title>
        </sec>
        <sec id="sec-2-4-17">
          <title>Modeling</title>
        </sec>
        <sec id="sec-2-4-18">
          <title>Informal</title>
        </sec>
        <sec id="sec-2-4-19">
          <title>Axioms</title>
        </sec>
        <sec id="sec-2-4-20">
          <title>Definition</title>
          <p>Ontology capture is strongly supported by the knowledge acquisition process.
Knowledge can be elicited from domain experts, as well as from sources of
consolidated knowledge, such as books, international standards, and reference models.
Concepts, relations and properties reused from non-ontological resources should be
ontologically analyzed in the light of the selected foundational ontology; fragments of
reused ontologies that are not grounded in the same foundational ontology too.</p>
          <p>Concepts and relations are the basis of an ontology, but they can be not enough to
capture the domain conceptualization. Constraints must also be taken into account.
Thus, axioms specifying constraints and inferences should be specified. Initially, we
do not need to write down formal axioms, rather the axioms should be written in
natural language, simply reflecting inferences and constraints on the universe of discourse.</p>
          <p>Ontology capture should be guided by the competency questions. The ontology
elements (concepts, relations, properties and axioms) in the ontology must be necessary
and sufficient to answer the competency questions. If the ontology elements are not
enough for this purpose, then additional concepts, relations, properties or axioms must
be added. In this sense, the ontology capture is an iterative process, strongly linked
with the evaluation process.</p>
          <p>During formalization, the informal axioms should be written in a formal language.
In this language, in contrast to the natural language, we have signs that are
unambiguous and formulations that are exact and, therefore, clarity and correctness can be more
easily evaluated. On the other hand, formal axioms are not able to substitute their
descriptions in natural language; rather, they are to be used to support these
descriptions. In fact, each representation plays a specific role.</p>
          <p>It is worthwhile to point out that in SABiO, formalization regards solely to writing
formal axioms from the informal ones. Thus, formalization should not be confused
with ontology implementation, when the entire ontology is codified in a
machinereadable language. In order to allow describing complex constraints and inference
rules, highly-expressive languages, such as such as first order logics and OCL, should
be used.</p>
          <p>Finally, many times, during the ontology capture and formalization phase, new
competency questions for the ontology arise. Thus, iteration with the previous phase
(Purpose Identification and Requirements Elicitation) is very common.
Ontology Design</p>
          <p>
            Once a reference ontology is produced, many times we want to get an operational
version to be used by computer applications. In order to achieve this operational
version, we need to design and implement it in a particular machine-readable ontology
language (e.g. OWL). In the design phase, the conceptual specification of the
reference ontology should be transformed into a design specification by taking into
account a number of issues ranging from architectural issues and technological
nonfunctional requirements, to target a particular implementation environment. The same
reference ontology can be used to produce a number of different designs [
            <xref ref-type="bibr" rid="ref5">5</xref>
            ].
          </p>
          <p>
            Figure 4 shows the activities that comprise the ontology design phase. First, the
ontology engineer must work together with the ontology designer to complement the list
of technological non-functional requirements for the operational ontology, and to
define the environment on which it will be implemented. Differently from reference
ontologies, operational ontologies are not focused on representation adequacy, but are
designed with the focus on guaranteeing desirable computational properties. The
design phase, thus, aims at bridging the gap between the conceptual modeling of
reference ontologies and the coding of them in terms of an operational ontology language
[
            <xref ref-type="bibr" rid="ref5">5</xref>
            ]. Issues that should be addressed in the design phase include: determining how to
deal with the differences in expressivity of the languages that are used in each of these
phases, and how to produce lightweight specifications that maximize specific
technological non-functional requirements, such as reasoning performance.
          </p>
          <p>Ontology Engineer
Ontology Designer</p>
        </sec>
        <sec id="sec-2-4-21">
          <title>Reference Ontology</title>
        </sec>
        <sec id="sec-2-4-22">
          <title>Design</title>
        </sec>
        <sec id="sec-2-4-23">
          <title>Ontology Design</title>
        </sec>
        <sec id="sec-2-4-24">
          <title>Specification</title>
        </sec>
        <sec id="sec-2-4-25">
          <title>Technical Non-Functional</title>
        </sec>
        <sec id="sec-2-4-26">
          <title>Requirements Elicitation</title>
        </sec>
        <sec id="sec-2-4-27">
          <title>Implementation</title>
        </sec>
        <sec id="sec-2-4-28">
          <title>Environment Definition</title>
        </sec>
        <sec id="sec-2-4-29">
          <title>Architectural Design</title>
        </sec>
        <sec id="sec-2-4-30">
          <title>Detailed Design</title>
          <p>Once defined the implementation environment, the ontology designer must revisit
the ontology modularization defined in the beginning of the project. Now, she has to
take the technological non-functional requirements and the characteristics of the
implementation environment into account to define the ontology architecture.</p>
          <p>
            Finally, during detailed design, the ontology designer has to address the problems
related to the lower expressivity of the operational languages when compared to the
models expressed in OntoUML and the formal axioms (outputs of the ontology
capture and formalization phase). Generally, heavyweight ontologies must give rise to
lightweight ones. If the implementation environment includes OWL and SWRL, the
transformation from OntoUML to these languages proposed in [
            <xref ref-type="bibr" rid="ref16">16</xref>
            ] can be applied. In
this case, we recommend using the Ontology Lightweight Editor (OLED) [
            <xref ref-type="bibr" rid="ref16">16</xref>
            ].
Ontology Implementation
          </p>
          <p>The implementation phase regards implementing the ontology in the chosen
operational language.</p>
          <p>Ontology Testing</p>
          <p>In SABiO, ontology test refers to dynamic verification and validation of the
behavior of the operational ontology on a finite set of test cases, against the expected
behavior regarding the competency questions. In this sense, SABiO's testing phase is
competency questions-driven, and considers mainly black-box testing, although white-box
testing can also be applied. A test case comprises an implementation of a competency
question as a query in the chosen implementation environment (the specification to be
tested), plus instantiation data from the fragment of the ontology being tested (input),
and the expected result based on the considered instantiation (expected output). As
Figure 5 shows, ontology testing in SABiO comprises three main activities.</p>
        </sec>
        <sec id="sec-2-4-31">
          <title>Reference Ontology</title>
        </sec>
        <sec id="sec-2-4-32">
          <title>Operational</title>
        </sec>
        <sec id="sec-2-4-33">
          <title>Ontology</title>
        </sec>
        <sec id="sec-2-4-34">
          <title>Ontology Design</title>
        </sec>
        <sec id="sec-2-4-35">
          <title>Specification</title>
          <p>Ontology Tester</p>
        </sec>
        <sec id="sec-2-4-36">
          <title>Ontology Testing</title>
          <p>Initially, test cases are run in the context of a sub-ontology. As sub-ontologies are
integrated, ontology integration testing is performed. In this activity, the same test
cases are re-run, but now considering the sub-ontologies already integrated. Finally,
ontology testing is performed. In this activity, the test cases are run again in the
context of the full ontology. During ontology testing, new test cases can be defined for
testing technological non-functional requirements. Thus, ontology testing may
include, among others, recovery and stress testing for web ontologies, and performance
testing for checking inference performance. Validation testing can also be performed
by using the operational ontology in actual software applications, according to the
intended uses originally proposed to the ontology. Validation testing should be
performed by ontology users.</p>
        </sec>
      </sec>
    </sec>
    <sec id="sec-3">
      <title>SABiO's Support Processes</title>
      <p>As shown in Figure 1, SABiO considers five main support processes: knowledge
acquisition, documentation, configuration management, evaluation and reuse. These
processes span the whole development process (or considerable parts of it).
Knowledge Acquisition Process</p>
      <p>Knowledge acquisition occurs mainly in the initial phases of the ontology
development process. Conventional methods and techniques for knowledge acquisition and
for requirements elicitation applies, mainly those devoted to collaborative knowledge
acquisition, such as brainstorming.</p>
      <p>
        Domain experts are the main source for knowledge acquisition. Without the
involvement of them, the ontology project can be impaired. Other important sources of
knowledge are consolidated bibliographic material, such as classical books,
international standards, glossaries, lexicons, classification schemes, and reference models.
The methodological guidelines for reusing non-ontological resources described in
[
        <xref ref-type="bibr" rid="ref12">12</xref>
        ] can be applied for selecting the most suitable resources to be used. The use of the
knowledge elicited from these sources necessarily involves reengineering. In this
context, ontological analysis based on a foundational ontology, as previously
discussed, is an important technique to be applied. For an example of such case, see [
        <xref ref-type="bibr" rid="ref17">17</xref>
        ].
Reuse Process
      </p>
      <p>Along the development process, there are many opportunities for reusing
conceptualizations already established for the domain in hands. Typically there are for main
sources for reuse: existing domain ontologies, core ontologies, foundational
ontologies, and ontology patterns.</p>
      <p>
        Existing ontologies for the domain can be reused, totally or partially. Especially in
the case of partially reusing existing domain ontology, techniques of ontology merge,
mapping and eventually reengineering apply [
        <xref ref-type="bibr" rid="ref12">12</xref>
        ]. Reuse of core ontologies
(ontologies that provide a precise definition of structural knowledge in a specific field that
spans across different application domains in this field [
        <xref ref-type="bibr" rid="ref18">18</xref>
        ]) are made mainly by
means of specialization. Concepts and relations of the core ontology are extended by
means of subtype relations in order to capture the more specific conceptualization
regarding the domain in hands. New concepts, relations, properties and axioms can
then be introduced in the domain ontology. Foundational ontologies can also be
reused. In this case, reuse can be done by means of specializations (as in the case of
core ontologies), but also by analogy. In reuse by analogy, foundational concepts and
relations are not explicitly extended in the domain ontology, but implicitly used for
deriving the structure of a portion of the domain ontology. In this sense, reuse by
analogy is strongly related to the structuring of the concepts and relations in a domain
ontology. A third way of reusing foundational ontologies is by using its foundations
to analyze fragments of the domain ontology (ontological analysis). The use of
OntoUML as modeling language for developing reference domain ontologies, and
OntoClean [
        <xref ref-type="bibr" rid="ref19">19</xref>
        ] are examples of ways of performing ontological analysis.
      </p>
      <p>
        Reuse can also be achieved by means of ontology patterns (OPs). In [
        <xref ref-type="bibr" rid="ref20">20</xref>
        ], Falbo et
al. propose an OP classification that is closely related to patterns in Software
Engineering, including four main types of OPs: conceptual OPs, architectural OPs, design
OPs, and programming OPs (idioms). Ontology Conceptual Patterns are fragments of
either foundational ontologies (Foundational OPs) or domain reference ontologies
(Domain-related OPs). Foundational OPs are reusable fragments of foundational
ontologies, while Domain-related OPs are reusable fragments extracted from reference
domain ontologies. These patterns are to be used during the ontology conceptual
modeling activity. Ontology Architectural Patterns are patterns that describe how to
arrange an ontology (generally a large one) in terms of sub-ontologies or ontology
modules, as well as patterns that deal with the modular architecture of an ontology
network. These patterns can be used both during the purpose identification and
requirements elicitation phase (ontology modularization activity), and in the ontology
design phase (ontology architectural design activity). Since modularity is recognized
as an important quality characteristic of good ontologies, we advocate for their use
since the first stages of ontology development, for splitting the ontology into smaller
parts, allowing tackling the problems one at a time. When applied in the architectural
design activity, the purpose is to reorganize the ontology modules for addressing
technological aspects, in special by taking non-functional requirements into account.
Ontology Design Patterns (ODPs) are to be used during the ontology detailed design
activity. There are two main types of ODPs [
        <xref ref-type="bibr" rid="ref20">20</xref>
        ]: logical and reasoning ODPs.
Reasoning ODPs addresses specific design problems related to improving reasoning with
ontologies (and qualities related to reasoning, such as computational tractability,
decidability and reasoning performance). Logical ODPs, in turn, regards problems
related to the expressivity of the formalism to be used in ontology implementation. They
help to solve design problems that appear when the primitives of the implementation
language do not directly support certain logical constructs. Logical ODPs are
extremely important for ontology design, since most languages for coding operational
ontologies are not focused on representation adequacy. We should highlight that
many patterns that address reasoning and logical problems are, in fact, Ontology
Idioms (or Ontology Programming Patterns), since they describe how to solve problems
related to reasoning or to the expressivity of a specific language (e.g., OWL).
Ontology idioms can be reused both during ontology detail design and ontology
implementation activities [
        <xref ref-type="bibr" rid="ref20">20</xref>
        ]. Finally, during ontology test, test cases can be reused.
Documentation Process
      </p>
      <p>Results from the ontology development process must be documented. Moreover,
results from some support processes, such as evaluation, must also be documented.
So, documentation is a process that has to occur in parallel with the others.</p>
      <p>In order to ensure uniformity in the ontology projects, it is useful to define the
basic set of documents to be produced in all projects. Templates for the main
documents must also be defined. SABiO suggests three main documents to be produced as
the documentation of an ontology project. As a result of the first two phases of the
development process, a Reference Ontology Specification should be produced. The
Operational Ontology Design Specification documents the aspects related to the
design phase, including: information related to the implementation environment,
technical non-functional requirements, ontology architecture, and main design decisions.
For documenting the operational ontology (i.e., the source code implementing the
ontology), SABiO suggests that the organization defines naming conventions and
rules for commenting the ontology code. Finally, for documenting the ontology test
phase, a test document shall be produced, including test cases and test results.</p>
      <p>Due to the highly collaborative nature of ontology projects, we suggest the use of
wikis for documentation, especially for documenting reference ontologies.
Configuration Management Process</p>
      <p>The main documents proposed by SABiO, as well as the source code of the
operational ontologies must have their configuration managed. Thus, once approved, they
must be submitted to the Configuration Management, where they will be controlled at
least concerning changes, versions, and delivery.</p>
      <p>Evaluation Process</p>
      <p>Although shown as an activity of the ontology development process, ontology
testing is, in fact, an evaluation activity. Ontology testing consists of the dynamic
evaluation (i.e. running code) of the behavior of an operational ontology on a finite set of
test cases, against the expected behavior. On the other hand, several other static
evaluation activities (those not involving running the code) must be performed during the
ontology development process. Those activities are performed by means of technical
reviews. In the context of ontology engineering, the purpose of a technical review is
to evaluate an intermediary work product of the ontology development process to
determine its suitability for its intended use. The results should confirm (or not) that
the work product meets the specifications, and adheres to standards.</p>
      <p>SABiO's evaluation process comprises two main perspectives:
 Ontology Verification: aims to ensure that the ontology is being built
correctly, in the sense that the output artifacts of an activity meet the specifications
imposed on them in previous activities.
 Ontology Validation: aims to ensure that the right ontology is being built,
that is, the ontology fulfills its specific intended purpose.</p>
      <p>Concerning ontology verification, the focus is on two main aspects: (i) Are the
ontology quality criteria (competency, clarity, coherence, consistency, minimal
ontological commitment, etc.) being met? (ii) Are the established standards (e.g., document
templates) and processes being correctly applied? Regarding the quality criteria, one
stands out: competency. As aforementioned, SABiO ontology testing phase is driven
by the competency questions. However, this is not enough. During the ontology
capture and formalization phase, the reference ontology should also be verified whether it
meets the requirements. This can be done by means of expert judgment. A table
indicating which ontology elements (concepts, relations, and axioms) are able to answer
each competency question should be built. The purpose of this table goes further
verification. It can also be used as a traceability tool, supporting change management.</p>
      <p>Concerning ontology validation, the participation of domain experts and ontology
users is essential. Ontology users have to evaluate whether the ontology is adequate
for their intended uses. For validating the reference ontology with domain experts, the
use of a graphical notation is very important, since generally they are not able to read
formal specifications. Besides expert judgment, another relatively easy way to
validate a reference ontology is by means of instantiation. The reference ontology should
be able to represent real world situations. Thus, an instantiation table should be
produced from real world situations.</p>
      <p>
        There are many evaluation techniques proposed in the literature. Several of them
can also be applied in a complementary way to the ones mentioned above. See a good
list of them in [
        <xref ref-type="bibr" rid="ref12 ref21">12, 21</xref>
        ].
5
      </p>
    </sec>
    <sec id="sec-4">
      <title>Related Work</title>
      <p>
        There are many ontology methods proposed in the literature. However, as pointed by
Corcho et al. [
        <xref ref-type="bibr" rid="ref22">22</xref>
        ], none of the existing approaches is fully mature if compared to
software engineering methodologies. Although we clearly have advanced in the last
years, as shown by the findings of the survey performed by Simperl et al. in 2009
[
        <xref ref-type="bibr" rid="ref23">23</xref>
        ], there are still room for improvements, and Software Engineering plays an
important role in this scenario.
      </p>
      <p>
        Among the various existing methods, we selected the NeOn Methodology for
Ontology Engineering [
        <xref ref-type="bibr" rid="ref12">12</xref>
        ] to compare to SABiO, since it is the result of the largest
project devoted to Ontology Engineering already performed, involving 14 partners and
during 4 years. The NeOn Methodology focuses on the engineering of ontology
networks, and does not prescribe a rigid workflow, but instead it suggests a variety of
pathways for developing ontologies. Like SABiO, the NeOn Methodology includes
several processes strongly linked to the development process, organized in 9
scenarios. Some processes, namely Ontology Development, Evaluation (Verification and
Validation), Reuse, Documentation, Configuration Management and Knowledge
Acquisition, are common to both methods. The NeOn Methodology considers also other
aspects not explicitly consider in SABiO, such as Ontology Localization,
Management, Alignment, and Consensus Reaching Process. In this sense, SABiO can be
enriched by applying several ideas from the NeOn Methodology. Aspects related to
collaborative development, addressed by methods such as DILIGENT [
        <xref ref-type="bibr" rid="ref24">24</xref>
        ], are not
adequately addressed in SABiO, and thus practices proposed by this (and other more
collaborative) methods should be introduced in SABiO.
      </p>
      <p>The striking features of SABiO when compared to the NeOn Methodology (and
also other ontology engineering methods) are: (i) The recognition that both reference
domain ontologies and operational ontologies are useful in themselves. SABiO
supports the development of both types of domain ontologies. This distinction leads to
the perception of the importance of a design phase (as largely recognized in Software
Engineering) in the ontology development process. Moreover, pattern-oriented reuse
in SABiO is also guided by this distinction. Other ontology engineering methods do
not prescribe a design phase in the sense used in Software Engineering. (ii) SABiO
recognizes the importance of the use of foundational ontologies in the development of
domain ontologies, and proposes the use of OntoUML during ontology capture, as
well as ontological analysis techniques. These two main features of SABiO have
impact in activities of other processes. For instance, Knowledge Acquisition and Reuse
Processes are strongly influenced by them. Thus, there are differences in the
approaches proposed by SABiO and the NeOn Methodology regarding these processes.
6</p>
    </sec>
    <sec id="sec-5">
      <title>Final Considerations</title>
      <p>
        This paper presented the new version of SABiO, an ontology engineering method.
This version was firstly used for developing ROoST (Reference Ontology on
Software Testing) [
        <xref ref-type="bibr" rid="ref25">25</xref>
        ], and an operational ontology derived from it. In this project, in
general, SABiO 2.0 worked well, especially the reuse process, by the reuse of
domain-related patterns organized as an ontology pattern language. It is important to
highlight also the great support provided by the Ontology Lightweight Editor (OLED)
[
        <xref ref-type="bibr" rid="ref16">16</xref>
        ] for automatically generating an OWL ontology (operational ontology) from the
OntoUML models.
      </p>
      <p>We know that we need more feedback in order to improve the current practices of
SABiO. Since SABiO is now being applied in the development of other domain
ontologies, we intend to collect new feedback from these projects, in order to better
evaluate the current version of the method.</p>
      <p>Acknowledgments. This research is funded by the Brazilian Research Funding
Agency CNPq (Process Number 485368/2013-7).</p>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          1.
          <string-name>
            <surname>Falbo</surname>
            ,
            <given-names>R.A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Menezes</surname>
            ,
            <given-names>C.S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Rocha</surname>
            ,
            <given-names>A.R.C.</given-names>
          </string-name>
          <article-title>A Systematic Approach for Building Ontologies</article-title>
          .
          <source>Proceedings of the 6th Ibero-American Conference on Artificial Intelligence</source>
          , Lisbon, Portugal, Lecture Notes in Computer Science, vol.
          <volume>1484</volume>
          ,
          <year>1998</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          2.
          <string-name>
            <surname>Falbo</surname>
            ,
            <given-names>R.A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Bertollo</surname>
            ,
            <given-names>G.</given-names>
          </string-name>
          ,
          <article-title>A software process ontology as a common vocabulary about software processes</article-title>
          .
          <source>International Journal of Business Process Integration and Management (IJBPIM)</source>
          ,
          <source>v. 4</source>
          , p.
          <fpage>239</fpage>
          -
          <lpage>250</lpage>
          ,
          <year>2009</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          3.
          <string-name>
            <surname>Oliveira</surname>
            ,
            <given-names>K.M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Zlot</surname>
            ,
            <given-names>F.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Rocha</surname>
            ,
            <given-names>A.R.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Travassos</surname>
            ,
            <given-names>G.H.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Galotta</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Menezes</surname>
            ,
            <given-names>C.S.</given-names>
          </string-name>
          ,
          <article-title>Domain-oriented software development environment</article-title>
          ,
          <source>Journal of Systems and Software</source>
          , v.
          <volume>72</volume>
          , issue 2, pp.
          <fpage>145</fpage>
          -
          <lpage>161</lpage>
          ,
          <year>July 2004</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          4.
          <string-name>
            <surname>Falbo</surname>
            ,
            <given-names>R. A.</given-names>
          </string-name>
          ,
          <article-title>Experiences in Using a Method for Building Domain Ontologies</article-title>
          , International Workshop on Ontology in Action,
          <source>Proceedings of the 16th International Conference on Software Engineering and Knowledge Engineering</source>
          ,
          <year>2004</year>
          , pg 474-
          <fpage>477</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          5.
          <string-name>
            <surname>Guizzardi</surname>
            ,
            <given-names>G.</given-names>
          </string-name>
          ,
          <article-title>On Ontology, ontologies, Conceptualizations, Modeling Languages and (Meta)Models</article-title>
          , In O.
          <string-name>
            <surname>Vasilecas</surname>
          </string-name>
          , et al. (
          <year>Org</year>
          .).
          <source>Frontiers in Artificial Intelligence and Applications</source>
          , Databases and Information Systems IV. IOS Press, Amsterdam,
          <year>2007</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          6.
          <string-name>
            <surname>Fielding</surname>
            ,
            <given-names>J.M.</given-names>
          </string-name>
          ;
          <string-name>
            <surname>Simon</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          ;
          <string-name>
            <surname>Ceusters</surname>
            ,
            <given-names>W.</given-names>
          </string-name>
          ;
          <string-name>
            <surname>Smith</surname>
            ,
            <given-names>B.</given-names>
          </string-name>
          ,
          <article-title>Ontological Theory for Ontology Engineering</article-title>
          .
          <source>In: Proceedings of 9th International Conference on the Principles of Knowledge Representation and Reasoning</source>
          , Whistler, Canada,
          <year>2004</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          7.
          <string-name>
            <surname>Fettke</surname>
            ,
            <given-names>P.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Loos</surname>
            ,
            <given-names>P.</given-names>
          </string-name>
          ,
          <article-title>Ontological Evaluation of Reference Models Using the Bunge-WandWeber Model</article-title>
          .
          <source>Proceedings of the 2003 Americas Conference on Information Systems</source>
          , Florida, USA,
          <year>2003</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          8.
          <string-name>
            <surname>Shanks</surname>
            ,
            <given-names>G.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Tansley</surname>
            ,
            <given-names>E.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Weber</surname>
            ,
            <given-names>R.</given-names>
          </string-name>
          ,
          <article-title>Using Ontology to Validate Conceptual Models</article-title>
          .
          <source>Communications of the ACM</source>
          , v.
          <volume>46</volume>
          , issue 10,
          <year>2003</year>
          , pp.
          <fpage>85</fpage>
          -
          <lpage>89</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          9.
          <string-name>
            <surname>Smith</surname>
            ,
            <given-names>B.</given-names>
          </string-name>
          ,
          <article-title>Against Idiosyncrasy in Ontology Development</article-title>
          .
          <source>In Proceedings of the 4th International Conference on Formal Ontology and Information Systems</source>
          , Baltimore,
          <year>2006</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          10.
          <string-name>
            <surname>Bringuente</surname>
            ,
            <given-names>A.C.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Falbo</surname>
            ,
            <given-names>R.A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Guizzardi</surname>
            ,
            <given-names>G.</given-names>
          </string-name>
          ,
          <article-title>Using a Foundational Ontology for Reengineering a Software Process Ontology</article-title>
          .
          <source>Journal of Information and Data Management, v. 2</source>
          , pp.
          <fpage>511</fpage>
          -
          <lpage>526</lpage>
          ,
          <year>2011</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref11">
        <mixed-citation>
          11.
          <string-name>
            <surname>Grüninger</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Fox</surname>
            ,
            <given-names>M.S.</given-names>
          </string-name>
          ,
          <article-title>Methodology for the Design and Evaluation of Ontologies</article-title>
          .
          <source>Workshop on Basic Ontological Issues in Knowledge Sharing</source>
          ,
          <year>1995</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref12">
        <mixed-citation>
          12.
          <string-name>
            <surname>Suarez-Figueroa</surname>
            ,
            <given-names>M. C.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Gomez-Perez</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Motta</surname>
            ,
            <given-names>E.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Gangemi</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          , Ontology Engineering in a Networked World. Springer, Berlin,
          <year>2012</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref13">
        <mixed-citation>
          13.
          <string-name>
            <surname>d'Aquin</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Schlicht</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Stuckenschmidt</surname>
            ,
            <given-names>H.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Sabou</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <article-title>Criteria and Evaluation for Ontology Modularization Techniques</article-title>
          ,
          <source>In: Modular Ontologies: Concepts</source>
          ,
          <source>Theories and Techniques for Knowledge Modularization</source>
          , Springer-Verlag,
          <year>2009</year>
          . pp.
          <fpage>67</fpage>
          -
          <lpage>89</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref14">
        <mixed-citation>
          14.
          <string-name>
            <surname>Guizzardi</surname>
            ,
            <given-names>G.</given-names>
          </string-name>
          ,
          <article-title>Ontological Foundations for Structural Conceptual Models</article-title>
          , Universal Press,
          <year>2005</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref15">
        <mixed-citation>
          15.
          <string-name>
            <surname>Guizzardi</surname>
            ,
            <given-names>G.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Graças</surname>
            ,
            <given-names>A.P.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Guizzardi</surname>
            ,
            <given-names>R.S.S.</given-names>
          </string-name>
          ,
          <string-name>
            <given-names>Design</given-names>
            <surname>Patterns</surname>
          </string-name>
          and
          <article-title>Inductive Modeling Rules to Support the Construction of Ontologically Well-Founded Conceptual Models in OntoUML, 3rd</article-title>
          <source>International Workshop on Ontology-Driven Information Systems</source>
          ,
          <year>2011</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref16">
        <mixed-citation>
          16.
          <string-name>
            <surname>Barcelos</surname>
            ,
            <given-names>P.P.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Santos</surname>
            ,
            <given-names>V.A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Silva</surname>
            ,
            <given-names>F.B.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Monteiro</surname>
            ,
            <given-names>M.E.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Garcia</surname>
            ,
            <given-names>A.S.,</given-names>
          </string-name>
          <article-title>An Automated Transformation from OntoUML to OWL and SWRL</article-title>
          .
          <source>In: Proceedings of the 6th Seminar on Ontology Research in Brazil (ONTOBRAS</source>
          <year>2013</year>
          ), Belo Horizonte, Brazil,
          <year>2013</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref17">
        <mixed-citation>
          17.
          <string-name>
            <surname>Ruy</surname>
            ,
            <given-names>F.B.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Falbo</surname>
            ,
            <given-names>R. A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Barcellos</surname>
            ,
            <given-names>M.P.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Guizzardi</surname>
            ,
            <given-names>G.</given-names>
          </string-name>
          ,
          <source>An Ontological Analysis of the ISO/IEC 24744 Metamodel. 8th International Conference on Formal Ontology in Information Systems</source>
          (FOIS'
          <year>2014</year>
          ), Rio de Janeiro, Brazil,
          <year>2014</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref18">
        <mixed-citation>
          18.
          <string-name>
            <surname>Scherp</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Saathoff</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Franz</surname>
            ,
            <given-names>T.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Staab</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          :
          <article-title>Designing core ontologies</article-title>
          .
          <source>Applied Ontology</source>
          , vol.
          <volume>6</volume>
          , n. 3,
          <issue>2011</issue>
          , pp.
          <fpage>177</fpage>
          -
          <lpage>221</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref19">
        <mixed-citation>
          19.
          <string-name>
            <surname>Guarino</surname>
            ,
            <given-names>N.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Welty</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          ,
          <article-title>Evaluating Ontological Decisions with OntoClean</article-title>
          .
          <source>Communications of the ACM</source>
          .
          <volume>45</volume>
          (
          <issue>2</issue>
          ),
          <year>2002</year>
          , pp.
          <fpage>61</fpage>
          -
          <lpage>65</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref20">
        <mixed-citation>
          20.
          <string-name>
            <surname>Falbo</surname>
            ,
            <given-names>R. A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Guizzardi</surname>
            ,
            <given-names>G.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Gangemi</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Presutti</surname>
            ,
            <given-names>V.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Ontology</surname>
            <given-names>Patterns</given-names>
          </string-name>
          :
          <article-title>Clarifying Concepts and Terminology</article-title>
          .
          <source>Proceedings of the 4th Workshop on Ontology Patterns (WOP2013)</source>
          , Sydney, Australia,
          <year>2013</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref21">
        <mixed-citation>
          21.
          <string-name>
            <surname>Obrst</surname>
            ,
            <given-names>L.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Ashpole</surname>
            ,
            <given-names>B.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Ceusters</surname>
            ,
            <given-names>W.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Mani</surname>
            ,
            <given-names>I.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Ray</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Smith</surname>
            ,
            <given-names>B.</given-names>
          </string-name>
          ,
          <article-title>The Evaluation of Ontologies - Toward Improved Semantic Interoperability</article-title>
          , In: Semantic Web:
          <article-title>Revolutionizing Knowledge Discovery in the Life Sciences</article-title>
          , Baker,
          <string-name>
            <given-names>C.J.O.</given-names>
            ,
            <surname>Cheung</surname>
          </string-name>
          ,
          <string-name>
            <surname>K-H</surname>
          </string-name>
          . (eds), Springer,
          <year>2007</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref22">
        <mixed-citation>
          22.
          <string-name>
            <surname>Corcho</surname>
            ,
            <given-names>O.</given-names>
          </string-name>
          ;
          <string-name>
            <surname>Fernández-López</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ;
          <string-name>
            <surname>Gómez-Pérez</surname>
          </string-name>
          ,
          <article-title>A. Methodologies, tools and languages for building ontologies. Where is their meeting point? Data &amp; knowledge engineering 46 (1</article-title>
          ),
          <year>2003</year>
          , pp.
          <fpage>41</fpage>
          -
          <lpage>64</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref23">
        <mixed-citation>
          23.
          <string-name>
            <surname>Simperl</surname>
            ,
            <given-names>E.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Mochol</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Burger</surname>
            ,
            <given-names>T.</given-names>
          </string-name>
          ,
          <article-title>Achieving Maturity: the State of Practice in Ontology Engineering in 2009</article-title>
          ,
          <source>International Journal of Computer Science and Applications</source>
          , Vol.
          <volume>7</volume>
          , No.
          <volume>1</volume>
          ,
          <issue>2010</issue>
          , pp.
          <fpage>45</fpage>
          -
          <lpage>65</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref24">
        <mixed-citation>
          24.
          <string-name>
            <surname>Pinto</surname>
            ,
            <given-names>H. S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Tempich</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Staab</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          ,
          <article-title>Ontology engineering and evolution in a distributed world using DILIGENT</article-title>
          .
          <source>In: Handbook on Ontologies. 2nd edition</source>
          , Springer,
          <year>2009</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref25">
        <mixed-citation>
          25.
          <string-name>
            <surname>Souza</surname>
            ,
            <given-names>E. F.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Falbo</surname>
            ,
            <given-names>R. A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Vijaykumar</surname>
            ,
            <given-names>N. L.</given-names>
          </string-name>
          <article-title>Using Ontology Patterns for Building a Reference Software Testing Ontology</article-title>
          . In: 8th International Workshop on Vocabularies,
          <article-title>Ontologies and Rules for the Enterprise</article-title>
          and Beyond, Vancouver, Canada,
          <year>2013</year>
          .
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>