1 Presentation of the system 1.1 State, purpose, general statement AgreementMakerLight (AML) is an automated ontology matching framework derived from the AgreementMaker system [2,4]. It was developed with the main goal of tackling very large ontology matching problems such as those in the life science domain, which AgreementMaker cannot handle efficiently.

The key design principles of AML were efficiency and simplicity, although flexibility and extensibility-which are key features of AgreementMaker-were also high on the list [5]. Additionally, AML drew upon the knowledge accumulated in Agreement-Maker by reusing, adapting, and building upon many of its components. Finally, one of the main paradigms of AML is the use of external resources as background knowledge in ontology matching.

AML is primarily focused on lexically rich ontologies in general and on life sciences ontologies in particular, although it can be adapted to many other ontology matching tasks, thanks to its flexible and extensible framework. However, due to its short development time (eight months), it does not include components for instance matching or translation yet, and thus cannot handle all ontology matching tasks.

The AML workflow for the OAEI 2013 can be divided into six steps, as shown in Fig. 1: ontology loading, baseline matching and profiling, background knowledge matching (optional), extension matching and selection, property matching (conditional), and repair (optional). Ontology Loading In the ontology loading step, AML reads and processes each of the input ontologies and stores the information necessary for the subsequent steps in its own data structures. First, AML reads the localName, labels and synonym properties of all classes, normalizes them, and enters them into the Lexicon [5] of that ontology. Then, it derives new synonyms for each name in the Lexicon by removing leading and trailing stop words [8], and by removing name sections within parenthesis. After class names, AML reads the class-subclass relationships and the disjoint clauses and stores them in the Relation-shipMap [5]. Finally, AML reads the name, type, domain, and range of each property and stores them in the PropertyList. Note that AML currently does not store or use comments, definitions, or instances.

In the baseline matching and profiling step, AML employs an efficient weighted string-equivalence algorithm, the Lexical Matcher [5], to obtain a baseline class alignment between the input ontologies. Then, AML profiles the matching problem by assessing the size (i.e., number of classes) of the input ontologies, the cardinality of the baseline alignment, and the property/class ratio. Regarding size, AML divides matching problems into three size categories (small, medium or large), which will affect decisions and thresholds during the background knowledge matching and the extension matching and selection steps. Regarding cardinality, AML also considers three categories (near-one, medium and high), which will determine how selection is performed during the extension matching and selection step. As for the property/class ratio, it determines whether AML will match properties during the property matching step.

Background Knowledge Matching For the OAEI 2013, AML employs three sources of background knowledge: Uberon [6], UMLS [1] and WordNet [10]. When using background knowledge, AML tests how well each source fits the matching problem by comparing the coverage of its alignment with the coverage of the baseline alignment. The Uberon Matcher uses the Uberon ontology (in OWL) and a table of pre-processed Uberon cross-references (in a text file). Each input ontology is matched both against the Uberon ontology using the Lexical Matcher and directly against the cross-reference table, and AML determines which form of matching is best (giving priority to the crossreferences, since they are more reliable). When Uberon is a good fit for the matching problem, it is selected as the only source of background knowledge and is used to extend the Lexicons of the input ontologies [8]. When it is a reasonable fit, its alignment is merged with the baseline alignment.

The UMLS Matcher uses a pre-processed version of the MRCONSO table from the UMLS Metathesaurus (in a text file). Each input ontology is matched against the whole UMLS table, then AML decides whether to use a single UMLS source (by comparing the coverage of all sources) or the whole table. When UMLS is a good fit for the matching problem, its alignment is used exclusively, and the extension matching and selection step is skipped. Otherwise, if it is a reasonable fit, its alignment is merged with the baseline alignment.

The WordNet Matcher queries the WordNet database for synonyms of each name in the Lexicons of the input ontologies, using the Jaws API CITATION. These synonyms are used to create temporary extended Lexicons, which are matched with the Lexical Matcher. Because WordNet is prone to induce errors, AML uses it only to extend the baseline alignment, meaning that it matches only previously unmatched classes.

The extension matching and selection step comprises two matching sub-steps that alternate with two selection sub-steps. First, AML employs a word-based similarity algorithm, the Word Matcher [5], to extend the current alignment globally, followed by a selection algorithm to reduce the alignment to the desired cardinality. Then AML employs the Parametric String Matcher [5], which implements the Isub string similarity metric [11], to extend the resulting alignment locally (i.e., by matching the children, parents and siblings of already matched class pairs). This is followed by a final selection sub-step. When the matching problem is profiled as 'large', the Word Matcher is skipped because it is too memory intensive to be used globally, and its local use is subsumed by that of the Parametric String Matcher [3]. In the interactive matching track, AML employs an interactive selection algorithm, which asks the user for feedback about mappings in case of conflict or below a given similarity threshold, until a given number of negative answers is reached.

Property Matching In the property matching step, AML matches the ontology properties. AML compares the properties' types, domains and ranges, looking for mappings in the class alignment when the domains/ranges are classes. Then, if the properties have attributes in common, AML measures the word-based similarity between their names (as per the Word Matcher [5]), employing also WordNet when background knowledge is turned on.

Repair In the repair step, AML employs a heuristic repair algorithm [9] to ensure that the final alignment is coherent with regard to disjoint clauses. The repair algorithm was used by default in all OAEI tracks, except for the Large Biomedical Ontologies track where we ran AML both with and without repair.

The AML system and the alignments it produced for the OAEI 2013 are available at the SOMER project page (http://somer.fc.ul.pt/).

AML had a very high precision (100%) but a fairly low recall (40%) in the Benchmark track, returning empty alignments in several of the tests. This is a consequence of AML's simple framework, which is exclusively based on element-level matching and does not handle instances. Nevertheless, it was interesting to note that AML had the highest F-measure/time ratio, which attests to its efficiency.

The AML Anatomy results are shown in Table 1. AML ran in this track both with and without background knowledge (AML-BK and AML respectively). In the case of this track, AML-BK selects Uberon exclusively as the source of background knowledge, and uses it for Lexicon extension. Thus, the only difference between AML and AML-BK is that the latter has Lexicons enriched with Uberon synonyms. The results of AML-BK were very good, with a fairly high precision, and the highest recall, F-measure and recall+ in this year's evaluation. However, the AML results without background knowledge were also good, ranking fourth overall in F-measure, and second if we exclude the systems using Uberon. In fact, we believe that the AML results are near-optimal for a strategy based solely on element-level matching, and that background knowledge is required to obtain substantial improvements. The impact and quality of the Uberon cross-references is clear when we note that AML-BK gained 10% recall over AML without any loss in precision. Finally, it is also noteworthy that AML was one of only two systems to produce coherent alignments.

The AML Conference results with reference alignment 1 are shown in Table 2 (the results with reference alignment 2 are slightly worst for all systems, but do not affect their ranking). AML ran in this track with and without background knowledge, with AML-BK using WordNet as the only source of background knowledge (to match both classes and properties).

The results of both AML-BK and AML were good, having the highest precision of this year's evaluation and ranking second and tied for third in terms of F-measure, respectively. An important part of the success of AML in this task was the property matching algorithm, which found 9 and 11 property mappings with 100% precision, with and without background knowledge respectively.

As we expected, the performance of AML in the Multifarm track was poor, with Fmeasures of only 4% and 3% when comparing different ontologies and the same ontologies respectively. Participation in this track was beyond our scope, as AML does not handle translations or employ structural-level matching, which are essential for success in this track.

The results of AML in the Library track were reasonable, as it ranked 4th in terms of F-measure (with 73%) and had the second highest recall of this year's OAEI (87.7%). Nevertheless, there is clearly room for improvement regarding precision, which was significantly lower than that of other top systems (62.5%) likely due to the fact that AML does not take the language of labels into account. Indeed, the results of AML were very similar to the MatcherAllLabels benchmark.

The AML Interactive Matching results are shown in Table 3. AML ran with the same configurations used in the Conference track, except that in this track the selection algorithm employed is interactive, rather than automatic. The results show that AML's interactive selection algorithm was effective, gaining both precision and recall in comparison with the conference results. Nevertheless, this algorithm is far from optimized, and it should be possible to reduce the number of user interactions without sacrificing F-measure.

The AML Large Biomedical Ontologies results are shown in Table 4. AML ran in this track with six different configurations: without background knowledge (AML); with background knowledge (AML-BK); with specialized background knowledge (AML-SBK); and in all three cases with (-R) and without repair. AML-BK selects Uberon in all six tasks of this track (although never for Lexicon extension) and selects WordNet only in the SNOMED-NCI small task. AML-SBK is given access to UMLS, and selects it exclusively for all six tasks. Our goals in testing all these configurations were: to assess the impact of using domain background knowledge both unrelated (Uberon) and directly related (UMLS) to the reference alignments; to assess the effect of using repair on the quality of the results; and to contribute to improve the quality of the reference alignments. The results of AML-SBK were very good, with a marked advantage over all other systems in this year's evaluation. This is unsurprising given that AML-SBK derived its alignments from UMLS using an automatic strategy that is likely analogous to that used to build the reference alignments in the first place. This evidently gives AML-SBK an advantage over systems that do not use UMLS. Note, however, that the strategy employed by AML is a general-purpose strategy for reusing preexisting mappings and cross-references, which is used for both UMLS and Uberon. The only issue is that the reference alignments were also automatically derived from UMLS, which makes the evaluation of AML-SBK positively biased. The results of AML-BK were also good, ranking second overall in recall and F-measure if we exclude the systems that used UMLS. However, in this case the evaluation of AML-BK is negatively biased by the reference alignments. The reason for this is that AML-BK uses Uberon, and many of the mappings derived from Uberon are not present in UMLS despite being correct. This is particularly evident in the FMA-NCI matching problem with whole ontologies, where the contribution of Uberon (based on crossreferences which are manually curated) was approximately neutral, decreasing the precision as substantially as it increased the recall (in relation to AML). Perhaps extending the reference alignments by compiling mappings from multiple reliable data sources such as Uberon could enable a fairer evaluation of the systems competing in this track, and make the tasks less trivial for systems using background knowledge. The use of repair led to clearly more coherent alignments, as all AML configurations with repair obtained very low degrees of unsatisfiability. However, in terms of quality of the results, the use of repair led to a minor increase in F-measure in some cases, but a substantial decrease in others, and thus had a negative effect overall. This is tied to yet another bias in the reference alignments, caused by the fact that they were automatically repaired [7]. Employing a repair strategy that differs from that used to build the reference alignments can be more penalizing than not doing any repair at all, since for each different decision a repair algorithm makes, it will remove a "correct" mapping and keep an "incorrect" one, whereas without repair we would only have the latter. The problem is that such decisions are essentially arbitrary regarding correctness.

3 General comments

On the whole, the results of AML (without background knowledge) were interesting, and show that, for many ontology matching tasks, a lightweight approach based solely on element-level matching can compete with more complex approaches. It is worth highlighting that AML was among the quickest systems in all tracks, and thus had a consistently high F-measure/time ratio in all tracks except for Multifarm. However, the results in the Multifarm track, and to a lesser degree those in the Benchmark track, remind us that AML is still a system in development. The results of AML-BK (and SBK) show that using suitable background knowledge is critical in specialized domains such as the biomedical, but can be advantageous even for more typical matching problems (such as those in the Conference track).

Implementing efficient and effective structural-level matching algorithms will be critical to improve the performance of AML overall. Language handling and translation will also be important to expand the scope of AML, and allow it to tackle tasks such as those in the Multifarm track. Finally, the inclusion of more sources of background knowledge will undoubtedly contribute to improve the performance of AML in tasks beyond the biomedical domain.

The participation of AML in the OAEI 2013 was a success overall, with very good results in the Anatomy, Conference, Interactive Matching and Biomedical Ontologies tracks, and reasonable results in the Library track. These results validate the background knowledge paradigm of AML, and demonstrate the effectiveness of a lightweight ontology matching strategy based solely on element-level matching. Nevertheless, it is also clear from the results that AML is not a complete ontology matching system yet, and that it can benefit from the addition of new tools to its base strategy. Regarding its namesake, AML was able to build upon the success AgreementMaker had in the Anatomy track in previous OAEI competitions, and was able to transpose this success to the Large Biomedical Ontologies track.