<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta />
    <article-meta>
      <title-group>
        <article-title>Ontology Integration Approaches: A Systematic Mapping</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Jordana S. Salamon</string-name>
          <email>jssalamon@inf.ufes.br</email>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Cássio C. Reginato</string-name>
          <email>cassio.reginato@inf.ufes.br</email>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Monalessa P. Barcellos</string-name>
          <email>monalessa@inf.ufes.br</email>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          <institution>Ontology &amp; Conceptual Modeling Research Group (NEMO), Computer Science Department, Federal University of Espírito Santo- Vitória - ES -</institution>
          <country country="BR">Brazil</country>
        </aff>
      </contrib-group>
      <abstract>
        <p>Ontology integration is an important topic of interest in Ontology Engineering. Integrating ontologies is a complex process that involves finding suitable ontologies, interpreting them and integrating them into a new integrated ontology. This paper presents a systematic mapping that investigated ontology integration approaches and provides a panorama of this topic. The results revealed a limited use of semantic relationships to map ontologies and a lack of concern with the integration scope, the goals the integrated ontology must achieve, search and selection of the ontologies to be integrated, and integration-based development processes.</p>
      </abstract>
      <kwd-group>
        <kwd>Ontology Integration</kwd>
        <kwd>Systematic Mapping</kwd>
        <kwd>Mapping Study</kwd>
      </kwd-group>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>1. Introduction</title>
      <p>
        Given the importance of ontology integration, we carried out a mapping study to
investigate ontology integration approaches recorded in the literature. A mapping study
is a secondary study designed to give an overview of a research area through classification
and counting contributions in relation to the categories of that classification. It makes a
broad study in a topic of a specific theme and aims to identify available evidence about
that topic
        <xref ref-type="bibr" rid="ref13">(Kitchenham and Charters, 2007)</xref>
        . Moreover, the panorama provided by a
mapping study allows identifying issues in the researched topic that should be addressed
in future research.
      </p>
      <p>
        In this study we are particularly interested in ontology integration approaches
addressing the conceptual level. By “ontology integration approaches” we mean
approaches (i.e., methods, techniques, processes) that integrate (by merging,
consolidating, and, when necessary, modifying) two or more ontologies with the purpose
of building a new integrated ontology
        <xref ref-type="bibr" rid="ref20">(Pinto and Martins, 2001)</xref>
        . In this sense, approaches
that are limited to address ontology mapping, without producing a new integrated
ontology, are out of the study scope. By “conceptual level” we mean that we are interested
in approaches concerned with the meanings behind the concepts to perform the
integration. No matter if they are exclusively conceptual or associated to operational
concerns. Hence, approaches limited to propose operational solutions (i.e., focused on
computational aspects) are out of the study scope.
      </p>
      <p>This paper presents the mapping study and its main results. It is organized as
follows: Section 2 provides the background for the paper, talking briefly about ontology
integration; Section 3 presents the research protocol used in the study; Section 4 presents
the obtained results; Section 5 discusses the findings that emerge from the results; Section
6 addresses the study limitations; and, finally, Section 7 presents our final considerations.</p>
    </sec>
    <sec id="sec-2">
      <title>2. Ontology Integration</title>
      <p>
        In the context of computer and information sciences, an ontology is an artifact that
describes a certain reality with some purpose. As any artefact, ontologies have a lifecycle.
They are designed, implemented, evaluated, modified, reused, etc.
        <xref ref-type="bibr" rid="ref7">(Gangemi and Presutti,
2009)</xref>
        . Ontology development is a complex task and even specialists face difficulties.
Ontology reuse is a practice that can help in this matter, since existent ontologies can be
reused to build new ones
        <xref ref-type="bibr" rid="ref9">(Guizzardi et al., 2013)</xref>
        . Despite that, ontology engineers still
face problems to select the most suitable ontologies and integrate them
        <xref ref-type="bibr" rid="ref17">(Park et al., 2011)</xref>
        .
      </p>
      <p>
        In the literature there are several definitions for ontology integration. In short,
ontology integration can be defined as the process of integrating two or more source
ontologies to build a new (integrated) ontology
        <xref ref-type="bibr" rid="ref22">(Vergara et al., 2003)</xref>
        . During the
integration process, it may be necessary to refine the source ontologies before integrating
them. In addition, new concepts and relationships can be added to the integrated ontology,
so that its requirements can be met. Given the strong relation between merging and
integration, and the possible confusion this may cause, we explain them bellow.
      </p>
      <p>
        According to
        <xref ref-type="bibr" rid="ref19">Pinto et al. (1999)</xref>
        , there are three different situations involving
ontology integration: (i) development of a new ontology reusing other ontologies; (ii)
merging of different ontologies that deal with the same subject, resulting in a single
ontology unifying them; (iii) integration of ontologies into applications. In the first case,
a new ontology has to be developed and there are available ontologies that meet the new
ontology requirements; then these ontologies are reused to build a new one. In the second
case, the new ontology joins ideas, concepts, distinctions, axioms, etc. (i.e., knowledge)
of ontologies in the same domain. When ontologies are merged, a new ontology is created
and it unifies concepts, terminologies, definitions, constraints, etc. In the third case,
different ontologies are introduced into one application to specify or implement a
knowledge-based system.
      </p>
      <p>In both, integration and merging, there are, on one hand, source ontologies and,
on the other, the resulting ontology, obtained by reusing the source ontologies. In the
integration process, after the integration it is possible to identify in the resulting ontology
regions from the source ontologies. The domains of the source ontologies are different
from the domain of the resulting ontology, although there may be a relation between them.
The concepts from the source ontologies can be reused in the integrated ontology as they
are, they can be adapted (or modified), specialized or added to new concepts. In merging,
the goal is to build a more comprehensive ontology about a subject, gathering, in a
coherent way, knowledge from other ontologies in the same subject. Thus, the subject of
the merged ontologies and the resulting ontology is the same. After merging, it may be
difficult to identify regions from the source ontologies in the resulting ontology.</p>
    </sec>
    <sec id="sec-3">
      <title>3. The Research Protocol</title>
      <p>
        The study was performed following the approach defined in
        <xref ref-type="bibr" rid="ref13">(Kitchenham and Charters,
2007)</xref>
        , which involves: planning, when the research protocol is defined; conducting, when
the protocol is executed and data are extracted, analyzed, and recorded; and reporting,
when the results are recorded and made available to potential interested parties. In this
section we present the main parts of the research protocol used in the study.
      </p>
      <p>The study goal was to investigate ontology integration approaches addressing the
conceptual level. For achieving this goal, we defined thirteen research questions (RQ)
that are shown in Table 1.</p>
      <p>The search string adopted in the study is composed of terms related to ontology
integration and approach. The following search string was used: (("ontology
interoperability") OR ("ontology integration") OR ("ontology merging")) AND
((“approach”) OR (“method”) OR (“framework”) OR ("strategy") OR ("process")). As
discussed, we are interested in approaches that integrate (by merging, consolidating, and,
when necessary, modifying) two or more ontologies with the purpose of building a new
ontology. Therefore, in the search string we included merging among the terms related to
integration. For establishing the string, we performed some tests using different terms,
logical connectors, and combinations among them. More restrictive strings excluded
some important publications identified during the informal literature review that preceded
the study. These publications were used as control publications, meaning that the search
string should be able to retrieve them. We decided to use a comprehensive string that
provided better results in terms of number and relevance of the selected publications, even
thought it had selected many publications eliminated in subsequent steps.</p>
      <p>The search was performed in the following six sources: IEEE Xplore, ACM
Digital Library, Scopus, Science Direct, Engineering Village and Web of Science.</p>
      <p>Publications selection was performed in four steps. In Preliminary Selection and
Cataloging (S1), the search string was applied in the search mechanism of each digital
library (we limited the search scope to title, abstract and keywords metadata fields). After
that, in Duplications Removal (S2), publications indexed in more than one digital library
were identified and duplications were removed. In Selection of Relevant Publications –
1st filter (S3), the abstracts of the selected publications were analyzed considering the
following inclusion (IC) and exclusion (EC) criteria: (IC1) the publication presents an
approach for ontology integration addressing the conceptual level; (EC1) the publication
is not written in English; (EC2) the publication does not have an abstract; (EC3) the
publication is a copy or an older version of an already selected publication; (EC4) the
publication is a secondary study, a tertiary study, a summary or an editorial; (EC5) the
publication was published as an abstract. In Selection of Relevant Publications – 2nd filter
(S4), the full text of the publications selected in S3 were read and analyzed considering
the cited inclusion and exclusion criteria plus (EC6) The full text of the publication is not
available. Publication selection was performed by the first and second authors. For each
publication, an identifier was defined and the following information was recorded: title,
authors, year, reference and source. Publication selection was reviewed by the third
author, who performed the publication selection procedure and reviewed the all the results
obtained by the other authors in each step. Discordances were discussed and resolved in
meetings.</p>
      <sec id="sec-3-1">
        <title>Rationale</title>
        <p>Identify ontology integration approaches
addressing the conceptual level recorded in the
literature
Provide an overview about when and where the
publications have been published as a way of
analyzing the maturity of the research topic.
Moreover, verify the distribution of publications
per time and per publication vehicle.</p>
        <p>
          Identify the type of research using the
classification defined in
          <xref ref-type="bibr" rid="ref23">(Wieringa et al., 2006)</xref>
          . A
panorama about the research types can indicate the
maturity level of the research topic.
        </p>
        <p>Identify the main characteristics of the ontology
integration approaches and verify if there is a
predominance of some of them.</p>
        <p>Investigate if there has been concern with defining
systematic processes for ontology integration and
which steps have been proposed.</p>
        <p>
          Identify the type of approach according to the
classification defined in
          <xref ref-type="bibr" rid="ref20">(Pinto and Martins, 2001)</xref>
          .
Investigate whether integration approaches have
been proposed in the context of broader ontology
engineering methods and how the approach relates
to the method.
        </p>
        <sec id="sec-3-1-1">
          <title>Investigate whether goals have been used to support ontology integration. Investigate whether competence questions have been used to support ontology integration.</title>
        </sec>
        <sec id="sec-3-1-2">
          <title>Investigate whether the integration approaches provide mechanisms to support the selection of the ontologies to be integrated.</title>
        </sec>
        <sec id="sec-3-1-3">
          <title>Investigate whether ontology integration</title>
          <p>approaches addressing the conceptual level have
also addressed the operational level.</p>
          <p>Investigate whether computational assistance have
been used to support the ontology integration
approaches.</p>
          <p>Identify the types of semantic relationships
between concepts (e.g., equivalence,
specialization, generalization, etc.) have been
addressed in the ontology integration approaches.</p>
          <p>After selecting the publications, data were extracted and recorded. Data
extraction and recording consisted of extracting data from the publications for each
research question and recording them in a form designed as a spreadsheet. Data extraction
was performed by the first author and reviewed by the third author. Once data were
validated, data interpretation and analysis were carried out. Quantitative data were
tabulated and used in graphs and statistical analysis. Qualitative analysis was performed
considering the findings, their relation to the research questions and the systematic
mapping purpose. Data interpretation and analysis were performed by the first author and
reviewed by the third author. Discordances were discussed and resolved in meetings.</p>
        </sec>
      </sec>
    </sec>
    <sec id="sec-4">
      <title>4. Data Synthesis</title>
      <p>The systematic mapping considered studies published until November of 2017. In the first
step (S1) 1816 publications were obtained (229 from IEEE Xplore, 616 from Scopus, 49
from ACM, 57 from Science Direct, 512 from Engineering Village and 353 from Web of
Science). In the second step (S2) duplications were removed, remaining 591 publications.
In the third step (S3), 232 publications were selected (a reduction of about 61%). In the
fourth step (S4) 15 publications were selected (a reduction of about 93%). The expressive
reduction in S4 was due to the fact that most analyzed approaches addressed only the
operational level. Next, we present the main results for each research questions (RQ).
RQ01. Which approaches of ontology integration addressing the conceptual level have
been presented in the literature? - Table 2 presents a summary of the ontology integration
approaches found in the study.</p>
      <p>RQ02. When and in which type of vehicle (journal/conference/workshop) have the studies
been published? - The publications selected were published between 2001 and 2017, as
shown in Figure 1. Regarding the publication vehicle, four studies (27%) were published
in journals, ten studies (66%) in conferences and one (7%) in workshops.</p>
      <p>Figure 1 - Year and Publication Vehicle</p>
      <p>
        Workshop
Conference
Journal
RQ03.Which type of research has been done?- According to the classification presented
in
        <xref ref-type="bibr" rid="ref23">(Wieringa et al., 2006)</xref>
        and considering that a publication can be classified in more
than one type, all the analyzed publications were classified as Proposal of Solution. Eight
of them ([P03], [P04], [P07], [P08], [P09], [P11], [P12], [P15]) (53%) are also Validation
Research, because they use proof of concept, experiment, prototype or something similar
to evaluate the proposal. No publication was classified as Evaluation Research, which
means that none of the proposals have been applied in a real environment.
RQ04. What is the basic principle of the ontology integration approach? - The main
principles identified were: similarity calculation, semantic mappings and integration
operations. The former is about using heuristics to calculate proximity between names,
structures and concepts in order to find equivalences. The second is used to find semantic
relationships between concepts and indicate the proximity level between them in a
qualitative form. The last refers to operations involving terminology, definition and
documentation of concepts to make the integrated ontology consistent. Five approaches
(33%) ([P06], [P10], [P11], [P13], [P15]) use similarity calculation, seven (47%) ([P04],
[P05], [P07], [P08], [P09], [P12], [P14]) use semantic mappings and two ([P01], [P03])
(13%) apply integration operations. [P02] is the only approach combining different
principles. It uses similarity calculation and semantic mappings.
      </p>
      <p>Table 2 Integration approaches.</p>
      <sec id="sec-4-1">
        <title>Reference Description</title>
        <p>
          Ma(Prtiinntso, a2n0d01) aPwrrieothpnooostthepserream-nseeotlnhetcootdleosdgt.oy binutieldgroantitoonlopgrieoscewshsetnhathtecaonntboelougsieeds tion bceominbteingaratitoend
          <xref ref-type="bibr" rid="ref16">(Miyoung et al., Ontology merging approach that considers vertical and horizontal
2006)</xref>
          integration and uses WordNet.
(ÖBhlogmreqnv,i2s0t0an8)d dmMeaevntehulooadlplmypedronetpvoeolsofepdaendtoaontnhdteotlhaoeugtoyotmhferorotmiavuettohsmeuapitpincltiaeelgrlsryaddtioeovmnealooinfp.etdIwt.ocoontshiedresr,s otnhee
(Ge2u0m08et) al., ioAnnittmeogslroagttiyooinng.teenIgterariatnitoecnlunadenewds gtechnoreenercaetpisottesnposr:felnaseteewrdvsiceteroviocsneetrcovoliocnegcsieepstfsro.dmeveolonptomloengty,
        </p>
        <p>
          Geographic ontology merging method consisting of three phases:
(Châabane et al., determine the n-ary correspondences between the ontologies; identify
2009) mappings between the concepts of the candidate ontologies; rule-based
merging to produce a global ontology.
          <xref ref-type="bibr" rid="ref5">(Chen et al., Framework based on web services for knowledge integration from
2009)</xref>
          ontology integration.
        </p>
        <p>
          Approach in which several ontologies can be merged into one using
          <xref ref-type="bibr" rid="ref10">(Heer et al., 2009)</xref>
          semantic correspondences. It is based on the assumption that all ontologies
use a common top-level ontology.
          <xref ref-type="bibr" rid="ref11">(Hu and Wang, Presents merging heuristics/algorithms based on category theory for
2010)</xref>
          geology ontologies.
(Juá2re0z11e)t al., aoMnnedtothlpoorgdoidethusacttehsaretacmedievoeetmstahaiensmeotenortgfoilsnoogguyprcaeuranominfeytotienlrog.gikensoawnldedagmeefrrgoimngthpearasmouertceer
          <xref ref-type="bibr" rid="ref14">(Leung et al., Ontology development method that integrates methods of reuse and a
2011)</xref>
          system to support integration.
        </p>
        <p>
          <xref ref-type="bibr" rid="ref15">(Lv, 2011)</xref>
          Ampeapsrouraecshtocionmtebgirnaitnegonitnotleoggriaetsi.on heuristics/algorithms and similarity
(Pet2ro0v12e)t al., Ibhnaetsueerlidlsigtieconsnt/aslgydositrreietmchtmetdhs.atascuypcpliocrtsgmraeprhgingmoodfealsnataonmdicatlhorenetoloingtieegs.raIttioins
          <xref ref-type="bibr" rid="ref2">(Bova et al., Approach to integrate ontologies to support data interoperability and
2015)</xref>
          knowledge representation in intelligent information systems.
(Cue2n0c1a7e)t al., iIMnttaednrecaaoglensmneewcntitethdAdpiopnmltiecagairtnaiotoinonsnt)oolowngtitoiehlsion.gytnheetwOoErkM,wAhi(cOhnistocloomgypofsoerd oEfneeirgghyt
          <xref ref-type="bibr" rid="ref2">(Bova et al., Method that considers three processes and semantic mappings to integrate
2015)</xref>
          two ontologies.
        </p>
        <p>RQ05. Which are the steps of the ontology integration approach? Are they explicitly
defined? - Only the approach addressed in [P02] does not present steps explicitly defined.
Table 3 presents the steps of each approach.</p>
        <p>RQ06. Does the approach refer to ontology integration, ontology merging or both? - Five
approaches (33%) ([P01], [P06], [P11], [P13], [P15]) refer to ontology integration, while
nine (60%) P02], [P03], [P04], [P05], [P07], [P08], [P09], [P12], [P14]) refer to ontology
merging. The approach reported in [P10] refer to both, integration and merging.</p>
        <p>ID Title
[P01] Identify integration possibility, Identify modules, Identify ontological commitments and
assumptions, Identify knowledge to be represented, Identify candidate ontologies, Obtain
candidate ontologies, Study candidate ontologies, Select candidate ontologies, Apply
integration operations, Analyze the resulting ontology
[P03] Add top level concepts to both ontologies, Add intermediate concepts, Add more specific
concepts, Include attributes and relationships
[P04] Find ontology concepts and service descriptions to be integrated, Create new relationships
with the descriptions and related concepts
[P05] Find correspondences, Mapping, Merging
[P06] Receive ontologies to be integrated, Perform similarity calculation of the ontology concepts,</p>
        <p>Merge the ontologies top-level concepts, Merge the subsequent concepts
[P07] Ontology alignment, Ontology merging
[P08] Establish the semantic relations between two geological ontologies, Merge synonyms in the
overlapping set as a new concept, Add other concepts from source ontologies that do not
belong to the semantic overlapping set, Add semantic relation (hyponym) in the semantic
overlapping set, Add new semantic relations to the resulting ontology
[P09] Source ontologies evaluation, Merging parameters definition, Equivalence mapping, Mapping
filtering, Ontology merging using filtering results.
[P10] Candidate ontologies identification, Concepts evaluation, Source ontologies identification and
categorization, Knowledge modules modification, Connection points identification, Basic
ontology building, Knowledge modules integration
[P11] Identify alignment between entities, Find ontologies portions that overlap and integrate the
ontologies, Perform the pruning of the integrated ontology through redundancy detection,
Check the integrated ontology consistence
[P12] Mapp the two input ontologies, Merge input ontologies in a super ontology
[P13] Ontologies comparison, Concepts integration, Result checking, Interpretation, Ontology
matching
[P14] Ontology structure definition, Ontology selection for reuse, Addition of new information to the
ontology, Ontology integration
[P15] Semantic similarity calculation, Merging of concepts, Knowledge model building based on
network, Model decomposition in blocks, Blocks rebuilding using semantic mappings between
the concepts, Integrated ontology generation
RQ07. Is the ontology integration approach related to some ontology engineering
method? If it is the case, which is the method and how does the approach relate to it?
12 publications (80%) propose approaches addressing ontology integration in isolation
(i.e., given two or more pre-selected ontologies, how to integrate them). Only three
publications (20%) propose ontology integration approaches as part of a broader ontology
engineering processes. In [P03], the integration approach refers to the ontology merging
phase of SEMCO, an ontology engineering method that involves ontology development,
ontology evaluation and ontology merging. In [P10], the integration approach is
performed in the context of the analysis, design and implementation phases of the Method
for Integration-Oriented Ontology Development, which includes preparation, analysis,
design, implementation and maintenance. In [P14], the approach is performed in the
context of the first, second and fourth phases of an ontology development method that
includes requirements definition, ontology selection for reuse, implementation, ontology
integration and evaluation.</p>
        <p>RQ08. Is the approach guided by goals? - None of the approaches uses goals to support
ontology integration.</p>
        <p>RQ09. Does ontology integration use competence questions? - Only one approach ([P10])
uses competence questions to support ontology integration.</p>
        <p>RQ10. Does the ontology integration approach use pre-selected ontologies, or does it aid
in the selection of the ontologies to be integrated? - 12 approaches (80%) consider
preselected ontologies for integration, that is, they assume that there are two or more selected
ontologies that need to be integrated and focus on more specific issues of the integration
problem (e.g., how to integrate concepts). Only three approaches (20%) ([P01], [P10],
[P14]) define ontology selection as a step of the ontology integration approach.
RQ11. Does the ontology integration approach address only the conceptual level or also
the operational level? - Only the approach presented in [P01] does not address the
operational level. All others (93%) address both levels.</p>
        <p>RQ12. Is the ontology integration approach automatic semiautomatic or non-automatic?
- Six approaches (40%) ([P08], [P09], [P11], [P12], [P13], [P15]) provide automatic
solutions, four (27%) propose semiautomatic solutions ([P02], [P07], [P10], [P14]) and
three (20%) do not provide any computational assistance ([P01], [P03], [P05]). In ([P04],
[P06] (13%) it was not possible to conclude if there is or is not computational assistance.
RQ13. Which types of semantic relationships between concepts are addressed in the
approach? - Table 4 presents the identified semantic relationships and the approaches
that address each of them. In the table, semantically equivalent relationships are listed
separated by "/".</p>
      </sec>
    </sec>
    <sec id="sec-5">
      <title>5. Discussion</title>
      <p>Publications
[P02], [P04], [P05], [P07], [P08], [P09], [P11], [P12],
[P13], [P15]
[P02], [P06], [P08], [P11], [P13], [P14]
[P06], [P07], [P08], [P13], [P14]
[P04], [P08], [P12]
[P02], [P06], [P07], [P13],
[P02], [P07],
[P08],
[P05]
[P01], [P03], [P10]
By analysing the publications distribution over the years (RQ02), it is possible to notice
that although there has been research about the topic since 2001, it has not been regular.
Moreover, considering the publication vehicles (RQ02) (there is a predominance of
conferences instead of journals) and the types of research (RQ03), we can conclude that
the topic has been explored but it is not mature yet. The lack of studies classified as
Evaluation Research indicates that the approaches have not reached the practice.</p>
      <p>With respect to the basic principle of the approaches (RQ04), the majority of them
concern finding similarities or semantic relationships between concepts. These results
show that integration approaches have been concerned with establishing relations
between the ontologies concepts, which is very important to integration.</p>
      <p>There have been concern with defining systematic processes for ontology
integration (RQ05). Some steps are cited in several approaches, such as identification of
mappings between concepts and merging/integration of concepts. Since these are core
activities to integrate ontologies, it is not surprising that they are in most of the
approaches. Besides these activities, integration approaches related to broader ontology
engineering methods also include activities related to ontologies search and selection.
Nevertheless, only three approaches ([1], [11], [14]) address ontology evaluation, even
though this is an important activity in of ontology development. Only two approaches ([1]
and [10]) address ontology modularization, an important aspect to manage complexity of
ontologies. In the view of the above, we can notice that most of the integration approaches
have been focused only on integration activities. In fact, most approaches are not
concerned with the entire ontology development process (RQ07). As a consequence,
these approaches ignore important steps in ontology development such as requirements
elicitation, modularization, testing and evaluation.</p>
      <p>There is a predominance of merging approaches (RQ06). This is probably due to
two main reasons: (i) approaches focusing on more specific integration problems (e.g.,
how to integrate similar concepts) can deal with them by merging ontologies (i.e., there
is no need to add new concepts to the integrated ontology); and (ii) merging can be seen
as part of the integration process, i.e., ontology integration involves merging, since after
merging ontologies, new concepts can be added to the resulting ontology.</p>
      <p>Integration approaches have not been concerned with defining the goals the
integrated ontology should achieve (RQ08). Most of the analyzed approaches were
defined to be used in a specific situation (e.g., given two pre-selected ontologies, how to
integrate them). Thus, a big picture of the goals the integrated ontology should achieve is
not explored. Goals can be used to express the design rationale behind an ontology and
can be helpful in the search for the ontologies to be integrated. Since most of the
approaches (80%) use pre-selected ontologies (RQ10), they have not addressed issues
that could help ontology search and selection. Moreover, since most approaches do not
include steps to identify the integration goals, it is natural that they also are not concerned
with defining the integration scope by means of competency questions (RQ09), which
has to be made prior to carry out the integration itself.</p>
      <p>Only one of the analyzed approaches does not address the operational level
(RQ11). Most of the analyzed approaches were defined to deal with a specific situation,
often related to computational applications. Thus, it is not surprising that most of the
approaches address not only the conceptual but also the operational level. This can also
be influenced by the fact that most of the ontologies available on the Web and used to
form the Semantic Web are available only in their operational form. With the growth of
the Semantic Web, there is also a growing need to integrate these ontologies. This also
can explain the predominance of approaches with automatic or semiautomatic solutions
(RQ12). As ontology integration can be a hard process when the ontologies to be
integrated are large or heavy, the approaches have automated the process in order to
request less intervention from the users.</p>
      <p>Finally, the approaches have addressed few types of semantic relations (RQ13).
Some approaches use only the equivalence relation, which limits the integration
approach. Addressing only the equivalence relation leads to miss important mappings
when the overlap of definitions and properties from different concepts is not complete.
One of the reasons for some approaches to consider only the equivalence relation is their
focus on operational solutions. Implementing heuristics to find equivalence mappings is
easier than heuristics to find other types of semantic relations. However, ideally, the
integration approaches should deal with a diversity of semantic relations, to enable proper
identification of mappings between the ontologies to be integrated.</p>
    </sec>
    <sec id="sec-6">
      <title>6. Limitations of the Study</title>
      <p>
        Usually, when conducting secondary studies, researchers need to make a lot of decisions
and exercise a lot of judgement. The decisions taken by researchers and the judgments
influence the outcome both in terms of which publications are selected and what the
researchers conclude from their secondary studies
        <xref ref-type="bibr" rid="ref24">(Wohlin et al., 2012)</xref>
        . Thus, as in any
study, the study presented in this paper has some limitations.
      </p>
      <p>Some of the challenges researchers may face during a systematic mapping are: (i)
how to select a comprehensive and relevant source of publications; (ii) how to
consistently apply the inclusion/exclusion criteria; (iii) how to classify and interpret data.
In this study, we experienced these challenges and we take some actions aiming at
minimizing the influence on the results.</p>
      <p>With respect to (i), the study considered six digital libraries as source of
publications. They were selected based on other secondary studies recorded in the
literature, as well as on other secondary studies carried out in the research group in which
this work was carried out (NEMO). While this set of digital libraries represents a
comprehensive source of publications, the exclusion of other sources and the fact that we
did not perform snowballing may have left some valuable publications out of the analysis.</p>
      <p>As for (ii), publications selection and data extraction were initially performed by
two of the authors and some subjectivity may have been incorporated, especially
regarding the level addressed by the approach (conceptual or operational). Information
contained in the publications could lead to a misunderstanding about this aspect. In order
to reduce this subjectivity, the third author reviewed publications selection and data
extraction and, in case of discordance or possible bias, discussions were held until a
consensus was reached. With regard to the selection made from the search string,
terminological problems may have led to not select some publications. In order to
minimize this possibility, simulations were performed with variations of the string until
obtaining the one that was used.</p>
      <p>
        With respect to (iii), a classification scheme was defined for each research
question. Some categories were based on classifications previously proposed in the
literature (for example, for type of research we used the classification defined in
        <xref ref-type="bibr" rid="ref23">(Wieringa et al., 2006)</xref>
        ). Other categories were established during data extraction, based
on data provided by the analyzed publications (e.g., types of semantic relations and
categories of the approaches basic principle). Determining the categories and how the
publications fit into them involves a lot of judgment. Data extraction and classification
were performed by the first author and reviewed by the third. Even so, it is possible that
other researchers would obtain different results.
      </p>
    </sec>
    <sec id="sec-7">
      <title>7. Final Considerations</title>
      <p>This paper presented a systematic mapping that investigated ontology integration
approaches addressing the conceptual level. 591 publications were analyzed and 15
approaches were identified. Before performing the systematic mapping, we investigated
the literature searching for secondary studies on ontology integration. Since no secondary
study was found on the research topic, we decided to perform the systematic mapping.</p>
      <p>The mapping results provide an overview of the research related to the
investigated topic. In short, integration approaches, even when addressing the conceptual
level, have also addressed (and, many times, focused on) the operational level. Most of
the approaches have been proposed to address specific situations or specific integration
problems. There has been concern with defining a systematic process to guide integration.
However, in most cases, the integration process is not embedded in a broader ontology
development process and considers the integration of pre-selected ontologies, without
addressing search and selection of ontologies. Goals and competence questions have not
been used to support ontology integration. Finally, approaches have considered few types
of semantic relations, which tends to limit the mapping and integration of ontologies.</p>
      <p>These results point to some gaps in the context of integrating ontologies at
conceptual level: (i) lack of concern with defining the scope for the integrated ontology
prior to the integration process; (ii) lack of concern with the goals that the integrated
ontology must achieve; (iii) lack of concern with search and selection of ontologies to be
integrated; (iv) limited use of semantic relations to map ontologies; and (v) lack of
integration-based development processes that address the development of an integrated
ontology considering the main phases of ontology engineering. These gaps provide a
roadmap of issues to be explored in future researches.</p>
      <p>As future work, we plan to investigate ontology integration approaches addressing
only the operational level and verify if the concerns at this level are different than the
ones we identified in this study. Moreover, considering the gaps we perceived from this
study, we have proposed a goal-oriented systematic approach to develop ontologies based
on integration and a framework to support our approach.</p>
    </sec>
    <sec id="sec-8">
      <title>Acknowledgment</title>
      <p>This research is funded by the Brazilian Research Funding Agency CNPq
(461777/20142 and 407235/2017-5), CAPES (23038.028816/2016-41), and FAPES (69382549/2014).</p>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          <string-name>
            <surname>Blomqvist</surname>
            ,
            <given-names>E.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Öhgren</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          ,
          <year>2008</year>
          .
          <article-title>Constructing an enterprise ontology for an automotive supplier</article-title>
          .
          <source>Eng. Appl. Artif. Intell</source>
          .
          <volume>21</volume>
          ,
          <fpage>386</fpage>
          -
          <lpage>397</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          <string-name>
            <surname>Bova</surname>
            ,
            <given-names>V. V.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Kureichik</surname>
            ,
            <given-names>V. V.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Lezhebokov</surname>
            ,
            <given-names>A.A.</given-names>
          </string-name>
          ,
          <year>2015</year>
          .
          <article-title>Integration of ontologies in scope of model and conceptual semantics: Modified approach</article-title>
          ,
          <source>in: 9th Int. Conf. on Application of Information and Communication Technologies</source>
          , AICT., pp.
          <fpage>156</fpage>
          -
          <lpage>160</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          <string-name>
            <surname>Caldarola</surname>
            ,
            <given-names>E.G.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Picariello</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Rinaldi</surname>
            ,
            <given-names>A.M.</given-names>
          </string-name>
          ,
          <year>2015</year>
          .
          <article-title>An approach to ontology integration for ontology reuse in knowledge based digital ecosystems, in: Management of Computational and Collective intElligence in Digital EcoSystems</article-title>
          . ACM, pp.
          <fpage>1</fpage>
          -
          <lpage>8</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          <string-name>
            <surname>Châabane</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Jaziri</surname>
            ,
            <given-names>W.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Gargouri</surname>
            ,
            <given-names>F.</given-names>
          </string-name>
          ,
          <year>2009</year>
          .
          <article-title>A proposal for a geographic ontology merging methodology</article-title>
          ,
          <source>in: Proc. of the 2009 Int. Conf. on the Curr. Trends in Inf. Tech., CTIT 2009</source>
          . pp.
          <fpage>202</fpage>
          -
          <lpage>207</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          <string-name>
            <surname>Chen</surname>
            ,
            <given-names>Y.-M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Chen</surname>
            ,
            <given-names>Y.-J.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Wen</surname>
          </string-name>
          , C.-C.,
          <string-name>
            <surname>Chu</surname>
          </string-name>
          , H.-C.,
          <year>2009</year>
          .
          <article-title>Ontology-Based Knowledge Integration for Distributed Product Knowledge Service</article-title>
          , in: World Congress on Engineering ans Computer Science, WCECS,
          <string-name>
            <surname>vols</surname>
            <given-names>I</given-names>
          </string-name>
          and II, pp.
          <fpage>1197</fpage>
          -
          <lpage>1202</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          <string-name>
            <surname>Cuenca</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Larrinaga</surname>
            ,
            <given-names>F.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Curry</surname>
            ,
            <given-names>E.</given-names>
          </string-name>
          ,
          <year>2017</year>
          .
          <article-title>A unified semantic ontology for energy management applications</article-title>
          ,
          <source>in: CEUR Workshop Proceedings</source>
          . pp.
          <fpage>86</fpage>
          -
          <lpage>97</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          <string-name>
            <surname>Gangemi</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Presutti</surname>
            ,
            <given-names>V.</given-names>
          </string-name>
          ,
          <year>2009</year>
          .
          <article-title>Ontology design patterns</article-title>
          , in: Handbook on Ontologies. Springer, pp.
          <fpage>221</fpage>
          -
          <lpage>243</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          <string-name>
            <surname>Geum</surname>
            ,
            <given-names>Y.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Suh</surname>
            ,
            <given-names>Y.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Park</surname>
            ,
            <given-names>Y.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Oh</surname>
            ,
            <given-names>H.</given-names>
          </string-name>
          ,
          <year>2008</year>
          .
          <article-title>Generating new service concepts based on ontology integration</article-title>
          ,
          <source>in: Proc. of the 4th IEEE Int. Conf. Conference on Management of Innovation and Technology, ICMIT</source>
          . pp.
          <fpage>878</fpage>
          -
          <lpage>883</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          <string-name>
            <surname>Guizzardi</surname>
            ,
            <given-names>G.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Wagner</surname>
            ,
            <given-names>G.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Falbo</surname>
            ,
            <given-names>R.A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Guizzardi</surname>
            ,
            <given-names>R.S.S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Almeida</surname>
            ,
            <given-names>J.P.A.</given-names>
          </string-name>
          ,
          <year>2013</year>
          .
          <article-title>Towards ontological foundations for the conceptual modeling of events</article-title>
          , in LNCS. pp.
          <fpage>327</fpage>
          -
          <lpage>341</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          <string-name>
            <surname>Heer</surname>
            ,
            <given-names>T.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Retkowitz</surname>
            ,
            <given-names>D.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Kraft</surname>
            ,
            <given-names>B.</given-names>
          </string-name>
          ,
          <year>2009</year>
          .
          <article-title>Tool support for the integration of light-weight ontologies</article-title>
          .
          <source>Lect. Notes Bus. Inf. Process</source>
          .
          <volume>19</volume>
          ,
          <fpage>175</fpage>
          -
          <lpage>187</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref11">
        <mixed-citation>
          <string-name>
            <surname>Hu</surname>
            ,
            <given-names>L.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Wang</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          ,
          <year>2010</year>
          .
          <article-title>Geo-ontology integration based on category theory</article-title>
          , in: 2010
          <source>International Conference on Computer Design and Applications</source>
          ,
          <string-name>
            <surname>ICCDA</surname>
          </string-name>
          <year>2010</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref12">
        <mixed-citation>
          <string-name>
            <surname>Juárez</surname>
            ,
            <given-names>S.P.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Esquivel</surname>
            ,
            <given-names>H.E.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Rebollar</surname>
            ,
            <given-names>A.M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Suárez-Figueroa</surname>
            ,
            <given-names>M.C.</given-names>
          </string-name>
          ,
          <year>2011</year>
          .
          <article-title>CreaDO - A methodology to create domain ontologies using parameter-based ontology merging techniques</article-title>
          ,
          <source>in: Proc. 10th Mexican Int. Conf. on Artifical Intelligence and Applications</source>
          , MICAI - Proceedings of Special Session. pp.
          <fpage>23</fpage>
          -
          <lpage>28</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref13">
        <mixed-citation>
          <string-name>
            <surname>Kitchenham</surname>
            ,
            <given-names>B.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Charters</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          ,
          <year>2007</year>
          .
          <article-title>Guidelines for performing Systematic Literature Reviews in Software Engineering</article-title>
          .
          <source>Engineering</source>
          <volume>2</volume>
          ,
          <fpage>1051</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref14">
        <mixed-citation>
          <string-name>
            <surname>Leung</surname>
            ,
            <given-names>N.K.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Lau</surname>
            ,
            <given-names>S.K.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Fan</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Tsang</surname>
            ,
            <given-names>N.</given-names>
          </string-name>
          ,
          <year>2011</year>
          .
          <article-title>An integration-oriented ontology development methodology to reuse existing ontologies in an ontology development process</article-title>
          .
          <source>Proc. 13th Int. Confrence Inf. Integr. Web-Based Appl. Serv</source>
          .
          <volume>174</volume>
          -
          <fpage>181</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref15">
        <mixed-citation>
          <string-name>
            <surname>Lv</surname>
            ,
            <given-names>Y.</given-names>
          </string-name>
          ,
          <year>2011</year>
          .
          <article-title>An approach to ontologies integration</article-title>
          ,
          <source>in: Proc. 8th Int. Conf. n Fuzzy Systems and Knowledge Discovery</source>
          ,
          <string-name>
            <surname>FSKD</surname>
          </string-name>
          <year>2011</year>
          . pp.
          <fpage>1262</fpage>
          -
          <lpage>1266</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref16">
        <mixed-citation>
          <string-name>
            <surname>Miyoung</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Kim</surname>
            ,
            <given-names>H.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Kim</surname>
            ,
            <given-names>P.</given-names>
          </string-name>
          ,
          <year>2006</year>
          .
          <article-title>A New Method for Ontology Merging based on Concept using WordNet</article-title>
          .
          <source>8th Int. Conf. Adv. Commun. Technol</source>
          .
          <year>2006</year>
          . ICACT
          <year>2006</year>
          .
          <volume>1573</volume>
          -
          <fpage>1576</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref17">
        <mixed-citation>
          <string-name>
            <surname>Park</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Oh</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Ahn</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          ,
          <year>2011</year>
          .
          <article-title>Ontology selection ranking model for knowledge reuse</article-title>
          .
          <source>Expert System Applications</source>
          .
          <volume>38</volume>
          ,
          <fpage>5133</fpage>
          -
          <lpage>5144</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref18">
        <mixed-citation>
          <string-name>
            <surname>Petrov</surname>
            ,
            <given-names>P.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Krachunov</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Todorovska</surname>
            ,
            <given-names>E.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Vassilev</surname>
            ,
            <given-names>D.</given-names>
          </string-name>
          ,
          <year>2012</year>
          .
          <article-title>An intelligent system approach for integrating anatomical ontologies</article-title>
          .
          <source>Biotechnol. Equip</source>
          .
          <volume>26</volume>
          ,
          <fpage>3173</fpage>
          -
          <lpage>3181</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref19">
        <mixed-citation>
          <string-name>
            <surname>Pinto</surname>
            ,
            <given-names>H.S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Gómez-Pérez</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Martins</surname>
            ,
            <given-names>J.P.</given-names>
          </string-name>
          ,
          <year>1999</year>
          . Some Issues on Ontology Integration.
          <source>Praxis (Bern</source>
          .
          <year>1994</year>
          ).
          <volume>18</volume>
          ,
          <fpage>1</fpage>
          -
          <lpage>12</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref20">
        <mixed-citation>
          <string-name>
            <surname>Pinto</surname>
            ,
            <given-names>H.S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Martins</surname>
            ,
            <given-names>J.P.</given-names>
          </string-name>
          ,
          <year>2001</year>
          .
          <article-title>A methodology for ontology integration</article-title>
          .
          <source>Proc. Int. Conf. on Knowledge Capture KCAP</source>
          <year>2001</year>
          ,
          <volume>131</volume>
          .
        </mixed-citation>
      </ref>
      <ref id="ref21">
        <mixed-citation>
          <string-name>
            <given-names>Poveda</given-names>
            <surname>Villalon</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            ,
            <surname>Suárez-Figueroa</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.C.</given-names>
            ,
            <surname>Gómez-Pérez</surname>
          </string-name>
          ,
          <string-name>
            <surname>A.</surname>
          </string-name>
          ,
          <year>2010</year>
          .
          <article-title>Reusing ontology design patterns in a context ontology network</article-title>
          .
          <source>in CEUR Workshop Proceedings</source>
          ,
          <year>2010</year>
          , vol.
          <volume>671</volume>
          , pp.
          <fpage>35</fpage>
          -
          <lpage>49</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref22">
        <mixed-citation>
          <string-name>
            <surname>Vergara</surname>
            ,
            <given-names>J.E.L.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Villagrá</surname>
            ,
            <given-names>V.A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Berrocal</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Asensiot</surname>
            ,
            <given-names>J.I.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Pignaton</surname>
            ,
            <given-names>R.</given-names>
          </string-name>
          ,
          <year>2003</year>
          . Semantic Management:
          <article-title>Application of Ontologies for the Integration of Management Information Models</article-title>
          , in: Integ. Net. Manag. VIII.
          <string-name>
            <surname>Springer</surname>
            <given-names>US</given-names>
          </string-name>
          , Boston, MA, pp.
          <fpage>131</fpage>
          -
          <lpage>134</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref23">
        <mixed-citation>
          <string-name>
            <surname>Wieringa</surname>
            ,
            <given-names>R.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Maiden</surname>
            ,
            <given-names>N.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Mead</surname>
            ,
            <given-names>N.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Rolland</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          ,
          <year>2006</year>
          .
          <article-title>Requirements engineering paper classification and evaluation criteria: a proposal and a discussion</article-title>
          .
          <source>Requirements Engineering</source>
          .
          <volume>11</volume>
          ,
          <fpage>102</fpage>
          -
          <lpage>107</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref24">
        <mixed-citation>
          <string-name>
            <surname>Wohlin</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Runeson</surname>
            ,
            <given-names>P.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Höst</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Ohlsson</surname>
            ,
            <given-names>M.C.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Regnell</surname>
            ,
            <given-names>B.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Wesslén</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          ,
          <year>2012</year>
          . Experimentation in software engineering, Experimentation in Software Engineering. Berlin Heidelberg, Germany: Springer-Verlag.
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>