<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta />
    <article-meta>
      <title-group>
        <article-title>Using quantitative aspects of alignment generation for argumentation on mappings</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Antoine Isaac</string-name>
          <xref ref-type="aff" rid="aff1">1</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>C´assia Trojahn</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Shenghui Wang</string-name>
          <xref ref-type="aff" rid="aff1">1</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Paulo Quaresma</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          <institution>University of E</institution>
        </aff>
        <aff id="aff1">
          <label>1</label>
          <institution>Vrije Universiteit, Department of Computer Science</institution>
          ,
          <addr-line>Amsterdam</addr-line>
          ,
          <country country="NL">Netherlands</country>
        </aff>
      </contrib-group>
      <abstract>
        <p>State-of-the art mappers articulate several techniques using different sources of knowledge in an unified process. An important issue of ontology mapping is to find ways of choosing among many techniques and their variations, and then combining their results. For this, an innovative and promising option is to use frameworks dealing with arguments for or against correspondences. In this paper, we re-use an argumentation framework that considers the confidence levels of mapping arguments. We also propose new frameworks that use voting as a way to cope with various degrees of consensus among arguments. We compare these frameworks by evaluating their application to a range of individual mappers, in the context of a real-world library case.</p>
      </abstract>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>Introduction</title>
      <p>An important problem for ontology alignment is to find ways of choosing among
the many tools and techniques available and their variations, and then
combining their results. This is almost infeasible by purely manual efforts, and fixed
heuristics for combining a pre-selected set of mappers will not fit a situation
where more and more matching tools and options can be applied to an even
greater variety of cases.</p>
      <p>
        A first range of methods relies on (partial) evaluation of the results given by
different techniques so as to recommend the best performing ones for the case at
hand [
        <xref ref-type="bibr" rid="ref1 ref2">1, 2</xref>
        ]. Others anticipate such results by comparing the characteristics of the
considered alignment case with “profiles” of matchers, as determined by previous
evaluation [
        <xref ref-type="bibr" rid="ref3">3</xref>
        ]. However, these methods result in applying the same treatment to
all the mappings obtained by a same method; they do not allow for considering
each mapping. In the context of peer-to-peer systems, a more flexible approach
has been proposed [
        <xref ref-type="bibr" rid="ref4">4</xref>
        ] that explores the way peers agree on a set of mappings, by
evaluating the translations resulted from the application of each mapping when
one peer queries for information provided by another.
      </p>
      <p>
        A promising option is to use argumentation frameworks where arguments
in favour or against mappings between concepts are declaratively represented
and processed [
        <xref ref-type="bibr" rid="ref5 ref6">5, 6</xref>
        ]. Here, a set of mappers, representing different alignment
approaches, generate a set of arguments that support the mappings. According
to the definition of attacking relations, an argument for a mapping generated
by one mapper can be supported or attacked by other arguments from other
mappers. Based on the framework instantiation (using specific attacking relation
and preference order), it is possible to compute globally acceptable mappings.
      </p>
      <p>These argumentation frameworks consider however the arguments based on
their intention only. An argument against a concept mapping can successfully
attack all the arguments in favour of it, even if there are dozens of these. In
this paper, we investigate quantitative aspects of alignment generation among
a set of arguing mappers. We focus especially on investigating and comparing
the value, for the argumentation process, of alignment generation: (1) confidence
level: can we use the confidence level of the mappings to solve argumentation
conflicts? ; (2) consensus among mappers: can we use the agreement between
mappers to measure the validity of the mappings in question?</p>
      <p>
        In this paper, we re-use an argumentation framework that considers the
confidence levels of mapping arguments [
        <xref ref-type="bibr" rid="ref5">5</xref>
        ]. We also propose new frameworks that
use voting as a way to cope with various degrees of support for arguments. We
compare these frameworks by evaluating their application to a range of
state-ofthe-art individual mappers, in the context of a real-world library case.
2
      </p>
    </sec>
    <sec id="sec-2">
      <title>Argumentation Frameworks</title>
      <p>The framework we have re-used and extended to deal with consensus, S-VAF,
is based on Value-based Argumentation, itself based on Dung’s classical system.
In this section we present these three frameworks, as well as our new proposals.
2.1</p>
      <p>
        Classical argumentation framework
Dung, observing that the core notion of argumentation lies in the opposition
between arguments and counter-arguments, defines an argumentation framework
(AF) as follows:
Def. [
        <xref ref-type="bibr" rid="ref7">7</xref>
        ] An Argumentation Framework is a pair AF = (AR, attacks), AR is a
set of arguments and attacks is a binary relation on AR.
      </p>
      <p>
        attacks(a, b) means that the argument a attacks the argument b. A set of
arguments S attacks an argument b if b is attacked by an argument in S. The
key question about the framework is whether a given argument a ∈ AR should
be accepted or not. Dung proposes that an argument should be accepted only if
every attack on it is rebutted by an accepted argument. This notion then leads
to the definition of acceptability (for an argument), admissibility (for a set of
arguments) and preferred extension:
Def. [
        <xref ref-type="bibr" rid="ref7">7</xref>
        ] An argument a ∈ AR is acceptable with respect to set arguments S,
noted acceptable(a, S), if ∀x ∈ AR (attacks(x, a) −→ ∃y ∈ S, attacks(y, x))
Def. [
        <xref ref-type="bibr" rid="ref7">7</xref>
        ] A set S of arguments is conflict-free if ¬ ∃x, y ∈ S, attacks(x, y). A
conflict-free set of arguments S is admissible if ∀x ∈ S, acceptable(x, S). A
set of arguments S is a preferred extension if it is a maximal (with respect
to set inclusion) admissible set of AR.
      </p>
      <p>A preferred extension represents a consistent position within AF , which
defends itself against all attacks and cannot be extended without raising conflicts.
2.2</p>
      <p>
        Value-based argumentation framework
In Dung’s framework, all arguments have equal strength, and attacks always
succeed, except if the attacking argument is otherwise defeated. However, as noted
in [
        <xref ref-type="bibr" rid="ref8">8</xref>
        ], in many domains, including ontology alignment, arguments may provide
reasons which may be more or less persuasive. Moreover, their persuasiveness
may vary according to their audience. Bench-Capon has extended the notion of
AF so as to associate arguments with the social values they advance:
Def. [
        <xref ref-type="bibr" rid="ref9">9</xref>
        ] A Value-based Argumentation Framework (VAF) is a 5-tuple VAF =
(AR, attacks, V, val, P ) where (AR, attacks) is an argumentation framework,
V is a nonempty set of values, val is a function which maps elements of AR
to elements of V and P is a set of possible audiences.
      </p>
      <p>
        Practically, in [
        <xref ref-type="bibr" rid="ref6">6</xref>
        ], the role of value is played by the types of ontology match
that ground the arguments, covering general categories of matching approaches:
semantic, structural, terminological and extensional. We argue further — and
will use later — that any kind of matching ground identified during a mapping
process or any specific matching tools may give rise to a value. The only
limitations are (i) a value can be identified and shared by a source of mapping
arguments and the audience considering this information (ii) audiences can give
preferences to the values. An extension to this framework, required for deploying
argumentation processes, indeed allows to represent how audiences with different
interests can grant preferences to specific values:
Def. [
        <xref ref-type="bibr" rid="ref9">9</xref>
        ] An Audience-specific Value-based Argumentation Framework (AVAF)
is a 5-tuple VAF p = (AR, attacks, V, val, valprefaud) where AR, attacks, V
and val are as for a VAF, aud is an audience and valprefaud is a preference
relation (transitive, irreflexive and asymmetric), valprefaud ⊆ V × V .
valprefaud(v1, v2) means that audience aud prefers v1 over v2. Attacks are
then deemed successful based on the preference ordering on the arguments’
values. This leads to re-defining the notions seen previously:
Def. [
        <xref ref-type="bibr" rid="ref9">9</xref>
        ] An argument a ∈ AR defeats an argument b ∈ AR for audience aud,
noted def eatsaud(a, b), if and only if both attacks(a, b) and
not valprefaud(val(b), val(a)). An argument a ∈ AR is acceptable to
audience aud with respect to a set of arguments S, noted acceptableaud(a, S), if
∀x ∈ AR, def eatsaud(x, a) −→ ∃y ∈ S, def eatsaud(y, x).
      </p>
      <p>
        Def. [
        <xref ref-type="bibr" rid="ref9">9</xref>
        ] A set S of arguments is conflict-free for audience aud if
∀x, y ∈ S, ¬attacks(x, y) ∨ valprefaud(val(y), val(x)). A conflict-free set of
arguments S for aud is admissible for aud if ∀x ∈ S, acceptableaud(x, S). A
set of arguments S in the VAF is a preferred extension for audience aud if
it is a maximal admissible set (with respect to set inclusion) for aud.
      </p>
      <p>
        In order to determine preferred extensions with respect to a value ordering
promoted by distinct audiences, objective and subjective acceptance are defined:
Def. [
        <xref ref-type="bibr" rid="ref6 ref9">9, 6</xref>
        ] An argument a ∈ AR is subjectively acceptable if and only if a appears
in the preferred extension for some specific audiences. An argument a ∈ AR
is objectively acceptable if and only if a appears in the preferred extension
for every specific audience.
2.3
      </p>
      <p>
        Strength-based Argumentation Framework
Value-based argumentation acknowledges the importance of preferences when
considering arguments. However, in the specific context of ontology alignment,
an objection can still be raised about the lack of complete mechanisms for
handling persuasiveness. Indeed, off-the-shelf matching tools very often provide a
mapping with a measure that reflects the strength of the similarity between the
two entities, or a more general confidence they have in the mapping – almost
always it is provided without any detail allowing to distinguish between the two.
These measures – we will use strength in the following – are usually derived
from similarity assessments made during the alignment process, e.g. from edit
distance measure between labels, or overlap measure between instance sets, as
in [
        <xref ref-type="bibr" rid="ref10">10</xref>
        ]. They are therefore often based on objective grounds.
      </p>
      <p>However, there is no objective theory nor even informal guidelines for
determining such strengths. Using them to compare results from different mappers is
therefore questionable especially because of potential scale mismatches. For
example, a same strength of 0.8 may not correspond to the same level of confidence
for two different mapper.</p>
      <p>
        It is one of our goals to investigate whether considering strengths gives better
results or not.3 To this end, we adapt a formulation introduced in [
        <xref ref-type="bibr" rid="ref11 ref5">11, 5</xref>
        ] to
consider the strength granted to mappings for determining attacks’ success:
Def. A Strength and value-based Argumentation Framework (S-VAF) is a
6tuple (AR, attacks, V, val, P, str) where (AR, attacks, V, val, P ) is a
valuebased argumentation framework, and str is a function which maps elements
of AR to real values from the interval [
        <xref ref-type="bibr" rid="ref1">0, 1</xref>
        ], representing the strength of the
argument. An audience-specific S-VAF is an S-VAF where the generic set
of audiences is replaced by the definition of a specific valprefaud preference
relation over V.
      </p>
      <p>Def. In an audience-specific S-VAF, an argument a ∈ AR defeats an argument
b ∈ AR for audience aud if and only if attacks(a, b) ∧ ( str(a) &gt; str(b) ∨
(str(a) = str(b) ∧ valprefaud(val(a), val(b))) )</p>
      <p>
        In other words, for a given audience, an attack succeeds if the strength of the
attacking argument is greater than the strength of the attacked one; or, if both
arguments have equal strength, the attacked argument is not preferred over the
attacking argument by the concerned audience. Similarly to what is done for
VAFs, an argument is acceptable for a given audience w.r.t a set of arguments
if every argument defeating it is defeated by other members of the set. A set of
arguments is conflict-free if no two members can defeat each other. Such a set
is admissible for an audience if all its members are acceptable for this audience
w.r.t itself. A set of arguments is a preferred extension for an audience if it is a
maximal admissible set for this audience.
3 Note that as opposed to what is done [
        <xref ref-type="bibr" rid="ref11 ref5">11, 5</xref>
        ] this paper aims at experimenting with
mappers that were developed prior to the experiment, and hence more likely to
present strength mismatches.
2.4
      </p>
      <p>
        Argumentation Frameworks with voting
The previously described frameworks capture the possible conflicts between
mappers, and find a way to solve them. However, they still fail at rendering the fact
that sources of mappings often agree on their results, and that this agreement
can be meaningful. Some large-scale experiments involving several alignment
tools – as the OAEI 2006 Food track campaign [
        <xref ref-type="bibr" rid="ref12">12</xref>
        ] – have indeed shown that
the more often a mapping is agreed on, the more chances for it to be valid.
      </p>
      <p>In the following, we adapt the S-VAF presented above to consider the level
of consensus between the sources of the mappings, by introducing voting into
the definition of successful attacks. We first describe the notion of support which
enables arguments to be counted as defenders or co-attackers during an attack:
Def. A Support-aware Framework (Sup-VAF) is a 7-tuple (AR, attacks,
supports,V,val,P,str) where (AR, attacks, V, val, P, str) is a S-VAF, and supports
and attacks are disjoint (reflexive) binary relations over AR.</p>
      <p>The voting is used to determine whether an attack is successful or not. Our
first proposal opts for a simple voting scheme, where the number of supporters
decides for success — as done in the plurality voting system.</p>
      <p>Def. In a Simple plurality voting Sup-VAF an argument a ∈ AR def eatsaud an
argument b ∈ AR for audience aud if and only if
attacks(a, b) ∧ ( |{x|supports(x, a)}| &gt; |{y|supports(y, b)}| ∨
(|{x|supports(x, a)}| = |{y|supports(y, b)}| ∧ valprefaud(val(a), val(b))) ).</p>
      <p>This voting mechanism is based on simple counting. In fact, as we have
seen previously, mappers sometimes return mappings together with a confidence
value. There are voting mechanisms which address this confidence information.
The first and most elementary one would be to sum up the strengths of
supporting arguments. However, as for the S-VAF, this would rely on the assumption
that the strengths assigned by different mappers are similarly scaled, which as
we have seen is debatable in practice.</p>
      <p>
        One possible option is to consider rankings derived from those confidence
levels. First, we rank arguments on a value basis. For a given value v ∈ V , we
define a function rankv : AR −→ N that enables to order all the arguments
according to their strength. Practically we choose to count, for each arguments,
the ones that have a lower confidence level: rankv(a) = |{x ∈ AR|val(x) =
v ∧ str(x) &lt; str(a)}|. Notice that this “ranking” reflects a partial order, as it
allows for ties (for mappings with a same strength). It however avoids turning to
random ordering decisions, and allows for seamless ranking of arguments derived
from mappings that were not given any strength, by just considering that these
arguments have an infinitely low strength. Based on this ranking, it is possible
to define a voting process inspired by the Borda count method, which is one
the reference methods for aggregating ranked choices – for each argument, we
average the ranks given to it by the audiences which support it: [
        <xref ref-type="bibr" rid="ref13">13</xref>
        ]:
Def. In a Borda count Sup-VAF an argument a ∈ AR def eatsaud an argument
b ∈ AR for audience aud if and only if
      </p>
      <p>bordaCount(arg) =
attacks(a, b) ∧ ( bordaCount(a) &gt; bordaCount(b) ∨
( bordaCount(a) = bordaCount(b) ∧ valprefaud(val(a), val(b)) ) ),
where P{x|supports(x,arg)} rankval(x)(x)
.
|{x|supports(x, arg)}|
3
3.1</p>
    </sec>
    <sec id="sec-3">
      <title>Experiments</title>
      <p>Experiment case
Our testbed reproduces the Library Track of the 2007 OAEI campaign.4 The
National Library of the Netherlands maintains two book collections, each
annotated with one thesaurus – GTT (35K concepts) and Brinkman (5K). These
thesauri have to be aligned with links that correspond to classical thesaurus
relations (broadMatch, narrowMatch, relatedMatch) or to semantic equivalence
(exactMatch). It is important to mention that among the 2.4 Million of books
in the two collections, 250K are actually dually annotated by both thesauri.
3.2</p>
      <p>
        Mappers used
To carry out our experiments, we have selected the results of six mappers, which
we believe to be a realistic sample of the available technology. The first three
are state-of-the-art mappers developed by the community (OAEI participants),
while the others result from our previous work. They exhibit a balance between
generic methods – e.g., string edit distance – and strategies that are arguably
more appropriate to the case at hand – e.g., using Dutch lexical knowledge.
OAEI participants. The first group of mappers we used are the participants of
the OAEI Library Track: Falcon [
        <xref ref-type="bibr" rid="ref14">14</xref>
        ], DSSim [
        <xref ref-type="bibr" rid="ref15">15</xref>
        ] and Silas [
        <xref ref-type="bibr" rid="ref16">16</xref>
        ]. These tools
are hybrid, as they use several alignment techniques in an integrated process. For
instance, Falcon considers the similarity of both lexical and structural
information of concepts, while Silas combines lexical techniques with applying
instancebased similarity measures on books descriptions accessed from a library service.
Note that, as generic matchers, they mainly return equivalence (exactMatch)
mappings, except Silas, which provides a significant number of related matches.
“Homegrown” mappers. We also re-used mappers developed for previous
experiments. First, an edit-distance lexical mapper applies string
similiarity to (tokenized) labels, resulting in various exact equivalent, broader,
narrower and related weighted matches. Second, a Dutch SKOS lexical mapper
outputs weighted equivalent and broader mappings, based on Dutch
morphological knowledge, exploiting the different type of labels of concepts as
represented in SKOS. Third, an extensional mapper exploits the simple
cooccurrence of concepts in KB book annotations [
        <xref ref-type="bibr" rid="ref10">10</xref>
        ] to produce weighted
equivalence links. For more details, see http://www.few.vu.nl/∼aisaac/om2008/
mappers-om08.pdf.
4 http://oaei.ontologymatching.org/2007/library
We set our evaluation in a scenario where mappings are used to translate book
annotations from one thesaurus to the other [
        <xref ref-type="bibr" rid="ref17">17</xref>
        ]. One mapping – it is of course
possible to restrict the mappings by selecting only one kind of relation, for
instance exactMatch – is considered as a translation rule, which translates one
GTT concept into its corresponding Brinkman concept. All mappings which
involve the same GTT concept are aggregated into a single rule.
      </p>
      <p>To carry out our evaluation, we use the 250K dually annotated books we have
mentioned as a golden standard. For one such book, if one of its GTT annotation
concept has a translation rule, we consider this book can be fired. Each of its GTT
annotation concepts is then translated into its Brinkman correspondence(s). The
original Brinkman annotation is taken as a gold standard, which is used to
measure the quality of the generated mappings.</p>
      <p>We measure how many translated concepts are correct (precision), how many
real Brinkman annotation concepts are missed (recall), and a Jaccard overlap as
combined measure of these two:</p>
      <p>Pa =</p>
      <sec id="sec-3-1">
        <title>P #correct</title>
        <p>|Bt|
#books f ired
,</p>
        <p>Ra =</p>
      </sec>
      <sec id="sec-3-2">
        <title>P #correct</title>
        <p>|Bo|
#all books
,</p>
        <p>Ja =</p>
      </sec>
      <sec id="sec-3-3">
        <title>P #correct</title>
        <p>|Bo∪Bt|
#all books
where #correct is the number of translated Brinkman concepts actually used,
Bo and Bt are the original and translated Brinkman annotation, respectively.
3.4</p>
        <p>
          Argumentation settings
Characterisation of mapping arguments and attacking relation. All the
mappers we used return correspondences in the form of m = (e1, e2, s, r), where e1
and e2 are entities from the two ontologies, s a confidence level, and r a
mapping relation — exactMatch, broadMatch, narrowMatch or relatedMatch.
Following [
          <xref ref-type="bibr" rid="ref5 ref6">6, 5</xref>
          ], arguments were created from these correspondences, as 6-tuples
arg = (e1, e2, s, r, v, h) where v denotes a value or type of mapping argument
(here, the tool which created the mapping) and h a support token (+ or −,
depending on whether the argument supports the correspondence or not). An
attack relationship holds between two arguments if these involve the same pair
of concepts but exhibit opposite support tokens.
        </p>
        <p>
          Generating negative arguments. Our problem is to define the arguments which
are against a given correspondence. The results of most of the
state-of-theart tools must be interpreted as supporting correspondences; except in some
formal approaches, there is no “negative mapping”. [
          <xref ref-type="bibr" rid="ref6">6</xref>
          ] solves this by examining
the features of the concepts, such as their label or position in the ontologies’
structural network, and use OWL semantics to find whether agents argue for or
against a correspondence. In practice, this complex process amounts to re-define
a mapping step, as the strategy and material used are very similar to the ones
exploited by the individual mappers. Here, we propose to experiment with two
simpler strategies which do not require to investigate the alignment space again.
        </p>
        <p>Negative arguments as failure (NAF). This basic strategy relies on the
assumption that mappers return complete results. For every possible pair of
concepts and mapping relation, we check whether a mapper outputs it. If not, this
correspondence is considered to be at risk, and a negative argument is
generated, with an arbitrary strength of 1. This assumption, at first sight quite bold,
is nevertheless supported by the observation that most mappers try to provide
as many mappings as possible, the amount of (equivalent) mapping pairs being
comparable to the size of the smallest ontology aligned.</p>
        <p>Negative arguments based on relation disjointness (NARD). The second
strategy assumes that two different thesaurus-inspired mapping relations
(broadMatch, narrowMatch or relatedMatch) cannot hold between a same pair of
concepts – a usual consistency check for thesauri – and that such a relation cannot
hold between two equivalent concepts. An argument is thus considered to attack
another if they link the same two concepts with different mapping relations.
Frameworks tested. For our evaluation, we experimented with the following
selection of framework and attack strategy settings:</p>
        <p>Baseline. This consists of a single aggregation – union – of mappers’ results
into a single set of mappings.</p>
        <p>F1 (Strength-based, attacks based on relation disjointness). This setting
corresponds to the S-VAF described in Section 2.3 with the NARD attack strategy.
Two versions are explored: (F1cont) adopting the confidence values produced
by the mapper as the strength of the generated arguments; (F1disc) applying a
threshold (0.5) on the original confidence values to produce arguments with a
discrete strength — 0 if the confidence level is below 0.5, 1 otherwise.</p>
        <p>F2 (Strength-based, attacks based on absent correspondences). This setting
corresponds to an S-VAF with the NAF attack strategy. The same two
alternatives as for the previous framework are explored (F2cont and F2disc).</p>
        <p>F3 (Plurality voting-based, attacks based on absent correspondences). This
setting combines the Sup-VAF framework of Section 2.4 with the NAF strategy.</p>
        <p>F4 (Borda count-based, attacks based on absent correspondences). This is
the Borda count Sup-VAF framework of Section 2.4, applying the NAF strategy.
Mapper configuration. For all settings, three groupings are considered: (1) the
three OAEI participants; (2) our three Homegrown matchers; (3) All matchers.
Preference ordering. For all settings, we create an audience for each mapper
involved. We define a complete preference order by defining a default order that
is adapted, for each audience, by lifting itself to first position: for OAEI, the
default order is Falcon&gt;Silas&gt;DSSim, but for the Silas audience the order defined
is Silas&gt;Falcon&gt;DSSim. The default for Homegrown is Co-occurrence&gt;SKOS
lexical&gt;Edit-distance. For All, it is Falcon&gt;Co-occurrence&gt;SKOS
lexical&gt;Editdistance&gt;Silas&gt; DSSim. This order, even though inspired by observing
respective mappers’ general performances, remains rather arbitrary. Crucially, it is also
fixed: we did not aim at analyzing the influence of this factor in our experiment.
3.5</p>
        <p>Results and discussion
Tables 1 and 2 show the results we obtained – w.r.t. evaluation measures and
amount of obtained annotation translation rules – both for individual matchers
and their combinations. For brevity, we show the results of evaluation only when
using all types of mappings in order to produce rules. We also performed
evaluation using only the exactMatch ones, but that did not bring significant changes,
both for absolute and relative performances of matchers and frameworks.</p>
        <p>One can first observe the great difference between F1 and F2 – F1 filtering
out only a few mappings compared to the baseline. The NARD strategy actually
does not result in the generation of many counter-arguments, causing final
results similar to those of the union of matchers. This is especially true for OAEI
matchers, which output almost only exactMatch mappings – Silas outputs
relatedMatch links, but these seem to relate concepts not involved in exactMatch
links, even considering Falcon and DSSim. Results vary more for the Homegrown
and All combinations, as these include many mappings with different relations,
as well as with different strengths, implying more (successful) attacks. Making
strengths discrete seems to have muscled up some counter-arguments, leading to
slightly stricter (but less efficient!) selection.</p>
        <p>F2 is much more selective. When a counter-argument with strength 1 is
generated for one matcher, it is likely to defeat the positive arguments issued by
matchers with lesser preference. For a given audience, a selective matcher causes
the removal, from the subjectively acceptable mappings, of many results from
all matchers below him. When each audience privileges the arguments produced
by the matcher it represents, this amounts to filter out from the objectively
acceptable mappings all those beyond the intersection of mappings with strength
1. This of course implies an expected great increase in precision and a decrease
in recall, compared to the union of results. This also makes the practical
interest of NAF with such a strength and preference configuration quite low. And
it suggests further experiments, with different preference order patterns and
default strengths for counter-arguments. For the OAEI combination (as well as for
All, which includes it), the intersection is very small (caused by DSSim missing
a lot of good mappings) which causes recall to be dramatically low. For the
Homegrown configuration, which combines much less stringent mappers, the
intersection is larger, explaining an evolution for precision and recall which is more
beneficial. Note that there is almost no difference between the continuous and
discrete settings for OAEI and All configurations. For these, the OAEI mappings
almost entirely dictate the intersection, and most of them already have a strength
of 1 – out of Falcon’s 3,697 mappings, only 20 have a strength lower than 1. For
the Homegrown configuration the effect is opposite to the one obtained for F1: a
number of mappings are now “saved”, as their strength being discretized up to
the one of counter-arguments. However, even if saved mappings are numerous,
their consequence on evaluation results is not striking, arguably because of their
involving infrequent concepts in the collection. These observations lead to the
conclusion that anticipating the effect of making strengths discrete is difficult,
without more precise knowledge on the content of alignments.</p>
        <p>For OAEI, the severe selection caused by NAF is partly compensated in F4
because of our ranking strategy. Falcon outputs a smaller number of precise
results, all of them with a strength of 1. All the good mappings are therefore
not attackable: if DSSim produces an attack on one Falcon correspondence, the
rank of the attacker is very likely to be lower than the rank of the attacked.</p>
        <p>The results for homegrown mappers hint at F3 being the only one able to
compensate for attacks on correct correspondences, if enough mappers vote for
them. This is certainly true for the OAEI combination, where framework 3 has
produced the best precision. This is due the fact that using such framework, it is
possible to retrieve significant part of the intersection sets of all mappings,
considering the selection of the mappings based on supporters. For example, if both
Falcon and DSSim have a positive argument in favour a mapping, independently
of the strength of a possible negative argument against the mapping from Silas,
the mapping is acceptable. But yet this is not always done at the cost of recall.
Even if F3 had worse recall than Silas, it obtains more resulting mappings than
F2 with the same continuous setting.5</p>
        <p>The same applies for the “homegrown” combination. F3 has a slightly lower
recall than F2 with continuous strengths, but, again, better precision and Jaccard
average than the baseline results, and by an even greater margin. Even when
individual mappers return large sets of overlapping mappings, argumentation
with voting appears to be more promising than simple union. The results for
the last All combination however hint that this positive effect may disappear
5 Note that our evaluation strategy computes precision on the basis of books for which
alignment allows to compute new annotations; it is therefore possible to have a
greater set of mappings with a better general precision.
when the number of combined mappers gets bigger, and their precision lower.
When too many lax mappers are involved, it is possible that wrong mappings
find enough supporters to remain undefeated – the combined influence of DSSim
and the un-filtered co-occurrence matcher may be instrumental here.
4</p>
      </sec>
    </sec>
    <sec id="sec-4">
      <title>Related work and conclusion</title>
      <p>
        Many methods, such as in [
        <xref ref-type="bibr" rid="ref1 ref2 ref3">1–3</xref>
        ], articulate mappings on a source basis: all
mappings from a given source are selected (or weighted, in a weighted sum
aggregation system) at once. This can be compared to the preference relation over
mapping sources that we use. However, our framework is more precise, since it
considers every mapping individually. In this respect, the alignment
argumentation frameworks of [
        <xref ref-type="bibr" rid="ref5 ref6 ref8 ref9">9, 5, 6, 8</xref>
        ], which we re-use and extend, relate to the efforts
focusing on the logical soundness of alignments. As an example, [
        <xref ref-type="bibr" rid="ref18 ref19">18, 19</xref>
        ]
investigates how to detect individual mappings which cause inconsistencies, considering
both aligned ontologies and proposed alignments. However, these approaches,
similarly to the way argumentation is done in [
        <xref ref-type="bibr" rid="ref6">6</xref>
        ], require full-fledged formal
ontologies, which will lack in many applications.
      </p>
      <p>Instead, we have experimented with counter-argument generation techniques
which can be applied to a wider range of cases. Our proposal to consider the
strength of mapping arguments – and the consensus about them – assumes that
quantitative aspects of alignment can help to compensate for the lack of formal
knowledge, in contexts such as our library case.</p>
      <p>However, our results are somehow inconclusive wrt. our initial research
questions on the benefits of using strengths and consensus in argumentation. In some
cases performances are comparable to those of best individual matchers. This is
a significant outcome, when the best performing matcher is not known in
advance. Still, no framework manages to outperform baseline merging for every
configuration. Worse, results point at complex phenomena that may be
inherent to combining alignments resulting from very different strategies – confidence
assignments, filtering of results. . . Further investigation is therefore necessary.</p>
      <p>
        First, we will complete our experiments by considering negative arguments
based on relation disjointness for the frameworks 3 and 4 and comparing our
results with using the basic VAF framework. Beyond, the problem of negative
argument generation needs more attention. In our type of application scenarios,
we cannot turn to formalized reasoning as done in [
        <xref ref-type="bibr" rid="ref6">6</xref>
        ]. It would be still interesting
to investigate techniques that take into account more semantic constraints than
done in our current strategies, using for instance detection of mapping cycles, or
equivalence mappings that relates one concept to two distinct ones. We might
benefit here from the constraints specified in the latest SKOS developments [
        <xref ref-type="bibr" rid="ref20">20</xref>
        ].
      </p>
      <p>
        Relevance feedback, as used in [
        <xref ref-type="bibr" rid="ref1 ref2 ref3 ref4">4, 1–3</xref>
        ], is also absent in our argumentation
system, in which only abstract arguments are considered. A possible option
could be to combine both approaches, and raise counter-arguments based on the
evaluation – either directly by assessing a correspondence, or in an end-to-end
way by studying its effects on the application at hand.
      </p>
      <p>Acknowledgements Authors are supported by the EU Programme Alban
for High Level Scholarships for Latin America, the EU eContentPlus project
TELplus and the Dutch NWO programme CATCH (STITCH project).</p>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          1.
          <string-name>
            <surname>Tan</surname>
            ,
            <given-names>H.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Lambrix</surname>
            ,
            <given-names>P.:</given-names>
          </string-name>
          <article-title>A method for recommending ontology alignment strategies</article-title>
          .
          <source>In: 6th Intl. Semantic Web Conference (ISWC</source>
          <year>2007</year>
          ), Busan, Korea (
          <year>2007</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          2.
          <string-name>
            <surname>Ehrig</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Staab</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Sure</surname>
            ,
            <given-names>Y.</given-names>
          </string-name>
          :
          <article-title>Bootstrapping ontology alignment methods with apfel</article-title>
          .
          <source>In: 4th Intl. Semantic Web Conference (ISWC</source>
          <year>2005</year>
          ), Galway, Ireland (
          <year>2005</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          3.
          <string-name>
            <surname>Mochol</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Jentzsch</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Euzenat</surname>
          </string-name>
          , J.:
          <article-title>Applying an analytic method for matching approach selection</article-title>
          . In: Ontology Matching Workshop,
          <string-name>
            <surname>ISWC</surname>
          </string-name>
          <year>2006</year>
          .
          <article-title>(</article-title>
          <year>2006</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          4.
          <string-name>
            <surname>Aberer</surname>
            ,
            <given-names>K.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Cudr´</surname>
            e-Mauroux,
            <given-names>P.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Hauswirth</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          :
          <article-title>Start making sense: The chatty web approach for global semantic agreements</article-title>
          .
          <source>J. Web Semantics</source>
          <volume>1</volume>
          (
          <issue>1</issue>
          ) (
          <year>2003</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          5. dos Santos, C.T.,
          <string-name>
            <surname>Moraes</surname>
            ,
            <given-names>M.C.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Quaresma</surname>
            ,
            <given-names>P.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Vieira</surname>
            ,
            <given-names>R.:</given-names>
          </string-name>
          <article-title>A cooperative approach for composite ontology mapping</article-title>
          .
          <source>Journal of Data Semantics</source>
          <volume>10</volume>
          (
          <year>2008</year>
          )
          <fpage>237</fpage>
          -
          <lpage>263</lpage>
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          6.
          <string-name>
            <surname>Laera</surname>
            ,
            <given-names>L.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Blacoe</surname>
            ,
            <given-names>I.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Tamma</surname>
            ,
            <given-names>V.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Payne</surname>
            ,
            <given-names>T.R.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Euzenat</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Bench-Capon</surname>
            ,
            <given-names>T.</given-names>
          </string-name>
          :
          <article-title>Argumentation over ontology correspondences in mas</article-title>
          .
          <source>In: 6th Intl. Conference on Autonomous Agents and Multi-Agent Systems</source>
          . (
          <year>2007</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          7.
          <string-name>
            <surname>Dung</surname>
            ,
            <given-names>P.M.</given-names>
          </string-name>
          :
          <article-title>On the acceptability of arguments and its fundamental role in nonmonotonic reasoning, logic programming and n-person games</article-title>
          .
          <source>AI</source>
          <volume>77</volume>
          (
          <year>1995</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          8.
          <string-name>
            <surname>Laera</surname>
            ,
            <given-names>L.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Tamma</surname>
            ,
            <given-names>V.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Payne</surname>
            ,
            <given-names>T.R.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Euzenat</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Bench-Capon</surname>
            ,
            <given-names>T.</given-names>
          </string-name>
          :
          <article-title>Reaching agreement over ontology alignments</article-title>
          .
          <source>In: ISWC</source>
          <year>2006</year>
          .
          <article-title>(</article-title>
          <year>2006</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          9.
          <string-name>
            <surname>Bench-Capon</surname>
            ,
            <given-names>T.</given-names>
          </string-name>
          :
          <article-title>Persuasion in practical argument using value-based argumentation frameworks</article-title>
          .
          <source>Journal of Logic and Computation</source>
          <volume>13</volume>
          (
          <year>2003</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          10.
          <string-name>
            <surname>Isaac</surname>
          </string-name>
          , A.,
          <string-name>
            <surname>van der Meij</surname>
          </string-name>
          , L.,
          <string-name>
            <surname>Schlobach</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Wang</surname>
            ,
            <given-names>S.:</given-names>
          </string-name>
          <article-title>An empirical study of instancebased ontology matching</article-title>
          .
          <source>In: ISWC</source>
          <year>2007</year>
          , Busan, Korea (
          <year>2007</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref11">
        <mixed-citation>
          11. dos Santos, C.T.,
          <string-name>
            <surname>Quaresma</surname>
            ,
            <given-names>P.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Vieira</surname>
            ,
            <given-names>R.:</given-names>
          </string-name>
          <article-title>An extended value-based argumentation framework for ontology mapping with confidence degrees</article-title>
          .
          <source>In: Argumentation in Multi-Agent Systems, 4th Intl. Workshop</source>
          , Honolulu,
          <string-name>
            <surname>HI</surname>
          </string-name>
          , USA (
          <year>2007</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref12">
        <mixed-citation>
          12.
          <string-name>
            <surname>Euzenat</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Mochol</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Shvaiko</surname>
            ,
            <given-names>P.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Stuckenschmidt</surname>
            ,
            <given-names>H.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Svab</surname>
            ,
            <given-names>O.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Svatek</surname>
          </string-name>
          , V.,
          <string-name>
            <surname>van Hage</surname>
            ,
            <given-names>W.R.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Yatskevich</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          :
          <article-title>Results of the ontology alignment evaluation initiative 2006</article-title>
          . In: Ontology Matching Workshop,
          <string-name>
            <surname>ISWC</surname>
          </string-name>
          <year>2006</year>
          .
          <article-title>(</article-title>
          <year>2006</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref13">
        <mixed-citation>
          13. de Borda,
          <string-name>
            <surname>J.C.:</surname>
          </string-name>
          <article-title>M´emoire sur les elections au scrutin</article-title>
          . Histoire de l'
          <source>Acadmie Royale des Sciences (1781)</source>
        </mixed-citation>
      </ref>
      <ref id="ref14">
        <mixed-citation>
          14.
          <string-name>
            <surname>Hu</surname>
            ,
            <given-names>W.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Zhao</surname>
            ,
            <given-names>Y.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Li</surname>
            ,
            <given-names>D.</given-names>
          </string-name>
          , Cheng, G.,
          <string-name>
            <surname>Wu</surname>
            ,
            <given-names>H.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Qu</surname>
            ,
            <given-names>Y.</given-names>
          </string-name>
          :
          <article-title>Falcon-AO: results for oaei 2007</article-title>
          . In: Ontology Matching Workshop,
          <string-name>
            <surname>ISWC</surname>
          </string-name>
          <year>2007</year>
          .
          <article-title>(</article-title>
          <year>2007</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref15">
        <mixed-citation>
          15.
          <string-name>
            <surname>Nagy</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Vargas-Vera</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Motta</surname>
          </string-name>
          , E.:
          <article-title>DSSim - managing uncertainty on the semantic web</article-title>
          .
          <source>In: Ontology Matching Workshop</source>
          ,
          <string-name>
            <surname>ISWC</surname>
          </string-name>
          <year>2007</year>
          .
          <article-title>(</article-title>
          <year>2007</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref16">
        <mixed-citation>
          16.
          <string-name>
            <surname>Ossewaarde</surname>
          </string-name>
          , R.:
          <article-title>Simple library thesaurus alignment with SILAS</article-title>
          . In: Second Intl. Workshop on Ontology Matching,
          <string-name>
            <surname>ISWC</surname>
          </string-name>
          <year>2007</year>
          .
          <article-title>(</article-title>
          <year>2007</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref17">
        <mixed-citation>
          17.
          <string-name>
            <surname>Isaac</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Matthezing</surname>
          </string-name>
          , H., van der Meij, L.,
          <string-name>
            <surname>Schlobach</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Wang</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Zinn</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          :
          <article-title>Putting ontology alignment in context: usage scenarios, deployment and evaluation in a library case</article-title>
          .
          <source>In: ESWC</source>
          <year>2008</year>
          , Tenerife, Spain (
          <year>2008</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref18">
        <mixed-citation>
          18.
          <string-name>
            <surname>Stuckenschmidt</surname>
          </string-name>
          , H., van
          <string-name>
            <surname>Harmelen</surname>
            ,
            <given-names>F.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Serafini</surname>
            ,
            <given-names>L.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Bouquet</surname>
            ,
            <given-names>P.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Giunchiglia</surname>
            ,
            <given-names>F.</given-names>
          </string-name>
          :
          <article-title>Using c-owl for the alignment and merging of medical ontologies</article-title>
          .
          <source>In: Formal Biomedical Knowledge Representation Workshop</source>
          , KR 2004, Whistler, Canada (
          <year>2004</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref19">
        <mixed-citation>
          19.
          <string-name>
            <surname>Meilicke</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Stuckenschmidt</surname>
            ,
            <given-names>H.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Tamilin</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          :
          <article-title>Applying an analytic method for matching approach selection</article-title>
          .
          <source>In: Ontology Matching Workshop</source>
          . (
          <year>2006</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref20">
        <mixed-citation>
          20.
          <string-name>
            <surname>Miles</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Bechhofer</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          :
          <article-title>Skos reference</article-title>
          .
          <source>Technical report, W3C (January 25</source>
          <year>2008</year>
          )
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>