Biomedical vocabularies have specific characteristics that make their lexical alignment challenging. We have built a string-based vocabulary alignment tool, AnAGram, dedicated to efficiently compare terms in the biomedical domain, and evaluate this tool's results against an algorithm based on Jaro-Winkler's edit-distance. AnAGram is modular, enabling us to evaluate the precision and recall of different normalization procedures. Globally, our normalization and replacement strategy improves the F-measure score from the edit-distance experiment by more than 100%. Most of this increase can be explained by targeted transformations of the strings with the use of a dictionary of adjective/noun correspondences yielding useful results. However, we found that the classic Porter stemming algorithm needs to be adapted to the biomedical domain to give good quality results in this area. 1 http://river-valley.tv/elsevier-merged-medical-taxonomy-emmet-from-smart-content-to-smartcollection/
Elsevier has a number of online tools in the biomedical domain. Improving their interoperability involves aligning the vocabularies these tools are built on. The vocabulary alignment tool needs to be generic enough to work with any of our vocabularies, but each alignment requires specific conditions to be optimal, due to vocabularies’ specific lexical idiosyncrasies.
We have designed a modular, step-wise alignment tool: AnAGram. Its
normalization procedures are based on previous research[
We compare the precision and recall of AnAGram against an implementation of
Jaro-Winkler’s edit-distance method (JW)[
In Section 2 we present our use-case: aligning Dorland’s to Elsevier’s Merged Medical Taxonomy (EMMeT)1. Section 3 describes related work in vocabulary alignment in the biomedical domain. Section 4 and 5 present AnAGram and evaluate against Jaro-Winkler’s edit-distance. Section 6 presents future work and conclusions.
Elsevier’s Merged Medical Taxonomy (EMMeT) is used in “Smart Content” applications2; it contains more than 1 million biomedical concepts and their hierarchical, linguistic and semantic relationships. We aim at expanding EMMeT with definitions from the authoritative biomedical dictionary Dorland’s3 by aligning them.
Cheatham and Hitzler[
Chua and Kim’s[
AnAGram was built for use on a local system5, and is tuned to performance by using hash-table lookup to find matches. Currently, no partial matching is possible. The matching steps are built in a modular way: one can select the set of desired steps. The source taxonomy is processed using these steps and the target taxonomy is processed sequentially: the alignment stops at the first match. Modules are ordered to increasing distance between original and transformed string, simulating a confidence value. Exact matching: corresponds to JW edit-distance 1.
Normalization: special characters are removed or transformed (Sjögren’s syndrome to Sjogren’s syndrome; punctuation marks to space), string is lower cased. Stop word removal: tokenization by splitting on spaces, removal of stop words, using a list that was fine-tuned over several rounds of indexing with EMMeT. 2 http://info.clinicalkey.com/docs/Smart_Content.pdf 3 http://www.dorlands.com/ 4 http://wordnet.princeton.edu/ 5 Dell™ Precision™ T7500, 2x Intel® Xeon® CPU E5620 2.4 GHz processors, 64 GB RAM.
Software: Windows 7 Professional 64 bit, Service Pack 1; Perl v5.16.3 Re-ordering: tokens are sorted alphabetically, enabling matches for inverted terms. Substitution: sequences of tokens are replaced with the corresponding value from our dictionary, applying a longest string matching principle.
Stemming: using the Porter stemming algorithm[
We align EMMeT version 3.2 (13/12/13) (1,027,717 preferred labels) to Dorland’s 32nd edition (115,248 entries). We evaluate AnAGram as a whole against JW, with a 0.92 threshold (established experimentally). The JW implementation can work only with preferred labels.
To evaluate the recall of AnAGram vs the JW implementation, we use a manual gold set of 115 mappings created by domain experts (Table 1). AnAGram gives better recall and better precision than the JW method.
Correct mapping Incorrect mapping Recall (%) Precision(%) F-measure Jaro-Winkler 46 8 43% 85% 0.57 AnAGram 80 3 71% 96% 0.82
Table 1 - Results of AnAGram vs. Jaro-Winkler on Dorland’s Gold Set pairs
We evaluate a random sample of 25 non-exact alignments from each module to get a better insight on AnAGram’s normalization process. The results are either: Correct, Related (useful but not exactly correct), or Incorrect (Table 2 and Figure 1). AnAGram gives more correct results but JW is useful for finding related matches. 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0%
Correct
Related
Incorrect
Preferred labels Jaro-Winkler AnAGram non-exact Normalised No stop words
C 16 6 Some modules are based on the result of a previous transformation, so the later the module comes in the chain, the more complicated matches it faces. ization does very well (100% correct results). Removal of stop words causes some errors and related matches: single-letter stop words can be meaningful, like A for hepatitis A. Word order rearranging ranks second: it does not often change the meaning of the term. Substitution performs reasonably well; most of the non-correct results are related matches. Stemming gives the poorest results with false positives due to nouns/verbs stemmed to the same root, such as cilitated/ciliate. The substituted and stemmed matches have a result similar to the stemmed results. Still, even the worst results from any AnAGram module are better than the overall results of the non-exact matches from the JW algorithm. One reason for this is that the JW does not stop the alignment at the best match, but delivers everything that satisfies the threshold of 0.92.
Not all modules account for an equal portion of the non-exact results. The normalization module delivers around 70% of matches, stemming accounts for 15 to 20% and the other modules account for 2% to 4% of the matches each.
Results are good compared to OAEI large biomedical vocabularies alignment’s
results for string-based tools[
References