Biomedical ontologies such as the 11th revision of the International Classification of Diseases and others are increasingly produced with the help of collaborative ontology engineering platforms that facilitate cooperation and coordination among a large number of users and contributors. While collaborative approaches to engineering biomedical ontologies can be expected to yield a number of advantages, such as increased participation and coverage, they come with a number of novel challenges and risks. For example, they might suffer from low participation, lack of coordination, lack of control or other related problems that are neither well understood nor addressed by the current state of research. In this paper, we aim to tackle some of these problems by exploring techniques for recommending concepts to experts on collaborative ontology engineering platforms. In detail, this paper will (i) discuss different recommendation techniques from the literature (ii) map and apply these categories to the domain of collaboratively engineered biomedical ontologies and (iii) present prototypical implementations of selected recommendation techniques as a proof-of-concept.
In the field of biomedical research, an increasing number of
ontologies are created collaboratively by a large group of people.
Examples include biomedical ontologies such as the Gene
Ontology (GO), the Ontology of Biomedical Investigations (OBI), the
National Cancer Institute Thesaurus (NCI) or the 11th revision of
the International Classification of Diseases (ICD-11). While
collaborative approaches to engineering biomedical ontologies can be
expected to yield a number of advantages, such as increased
participation and coverage, higher acceptance or improved quality, they
come with a number of novel challenges and risks. For example,
recent research on collaborative authoring environments indicates
that the quality of collaboratively constructed products depends on
the number of active participants, the ability to direct qualified
participants to relevant content, amongst other factors
The main focus of this paper is to explore recommender techniques for collaborative ontology engineering platforms in the biomedical domain. The paper is structured as follows: In Section 2 we will discuss related work on collaborative engineering of biomedical ontologies as well as existing work on recommender techniques. In Section 3 we will map different recommender techniques to the biomedical ontology engineering domain. In Section 4 we will provide a short introduction to an exemplary collaborative ontology engineering project from the biomedical domain: the ICD-11 project. In Section 5 we will present results from three proof-of-concept recommender implementations. In section 6, we will conclude by discussing our approaches and point to future work.
The overall contributions of this paper are a high level mapping of recommender techniques to collaborative ontology engineering platforms in the biomedical domain, and a proof-of-concept in the form of implementations in the context of the ICD-11 project. 2 2.1
In the context of collaboration platforms, many factors are known
to influence the motivation and activity of individuals. For example,
we know that transparent and well defined goals can affect groups
and their performance. Making contributors aware of the utility of
their contributions represents another important factor
An increasing number of biomedical ontologies, such as the
Gene Ontology, the National Cancer Institute Thesaurus, or the
ICD-11, are created using collaborative ontology engineering
platforms. Requirements for collaborative ontology engineering
platforms have been discussed by, for example,
The main objective of recommender technology is to provide
personalized suggestions that help an individual, or a group of
individuals, to find objects or items of interest
In the literature, a distinction between three basic
recommendation strategies can be identified: item/content based, collaborative
filtering, and knowledge-based recommender techniques
In the context of collaborative authoring systems, such as
Wikipedia, recommender systems can be useful not only to help finding
items of interest, but also to increase participation
In the following, we aim to explore how the three identified recommender strategies map onto collaborative ontology engineering platforms in the biomedical domain. 3.1
The intuition behind content based recommender techniques is to find and identify similar items or concepts by calculating and comparing similarity between content-related features of each concept.
Content in an ontology can be defined as features of concepts, textual properties such as titles and descriptions, notes and discussions or ratings. In the context of collaboratively engineered biomedical ontologies, these properties can be term names, textual definitions of the concepts, clinical descriptions such as related/affected body parts, synonyms, signs and symptoms, investigation findings such as lab activities or measures needed to diagnose a disease or even treatment plans.
Similarity for content based recommender systems is usually calculated on a common set of features or properties that all items or concepts share, using similarity or correlation measures such as Pearson correlation, cosine similarity or the Jaccard coefficient. Other textual similarity measures that could be used include the Levenshtein distance, or simple overlap of textual properties of concepts. Depending on the environment, different similarity measures can yield different results when presented with the same input.
Potentials & Limitations for content based recommender systems are closely tied to the properties and content of the concepts in the ontology. An advantage of content-based recommendations is that they can be generated even in the absence of social usage data (i.e. they do not suffer from the ramp-up problem). A lack of rich textual content however might impair the overall usefulness of content-based recommendations. 3.2
The intuition behind collaborative filtering is to find concepts or
items based on similar user behavior. This is accomplished by
identifying behavioral patterns or usage patterns of users, and by
grouping them according to their similarity
Usage patterns are patterns that define the interest of a user for items or concepts. They either can be explicitly entered information such as ratings or implicit measures deducted from the amount of previously viewed, bought, or changed items by a single user. In the context of collaboratively engineered biomedical ontologies, usage patterns could be defined by grouping different behavior of users on a collaborative platform such as adding, editing or even moving or deleting a concept, property or individual. Notes can be used as an indication for interest as well as viewing patterns or viewing times.
Similarity for collaborative filtering is usually calculated by
identifying users with common interests which can be done by
calculating the similarity between their usage patterns. In collaborative
systems, interest is often modeled by explicit item rankings
entered by the users. Since this kind of information is typically not
available in biomedical contexts (users do not rate their favorite
concepts), other features, such as the number of times a concept has
been viewed or changed or even other properties assigned to users
and concepts, can be used. To calculate similarity, a series of
different similarity measures, including Pearson correlation or cosine
similarity
Potentials & Limitations for collaborative filters are closely tied to the extent that usage data is available. Collaborative filtering approaches are particularly prone to the early phases of collaborative ontology engineering projects, where little data about user interactions is available. However, once sufficient data is collected, collaborative filtering approaches can recommend concepts that are not necessarily related content-wise, but through other usage pattern based characteristic. 3.3
The intuition behind knowledge based recommender systems is to find similar concepts based on specific domain knowledge. They represent a sub-class of content based recommender systems and differ from them by using domain knowledge to create rules to determine the best item or concept to recommend, instead of simple properties.
Domain knowledge is specific knowledge extracted from the environment of the system or the system itself. The biggest challenge in creating knowledge based recommendations is identifying viable domain knowledge, that will produce good results when used to calculate similarity. Recommendations can be produced by traversing along the edges in an ontology to identify related sub, super or sibling concepts. In addition, linkages between ontologies can be exploited to generate knowledge based recommendations as well.
Similarity for knowledge based recommender techniques can be calculated using properties that could either be actively collected, by querying users for input, or implicitly by analyzing previous behavior of a user. Potentials & Limitations for knowledge based recommender systems are mostly related to the problem of distinguishing between basic content and domain knowledge. In addition, not every domain knowledge property will provide an equal basis for good recommendations. An advantage of knowledge based recommender techniques is that, at least in some way, they are less dependent on the quantity of content and contributions. 4
In the following, we will briefly introduce the ICD-11 project. We
will use the project later as an example to illustrate the adoption
of recommender techniques for collaborative ontology engineering
platforms in the biomedical domain. The International Classification
of Diseases is a taxonomy maintained by the World Health
Organization and is updated to a new revision around every decade. It is
used worldwide for monitoring health related expenses, to inform
policy makings, and to collect disease statistics. ICD-10 and all
other predecessors of the ICD-11 were created by selected
international experts; the production process was closed to the public. For
ICD-11, the WHO decided on a more open, collaborative approach.
This new approach allows experts all over the world to contribute to
ICD-11, using a web based collaboration platform called the ICD-11
Collaborative Authoring Tool (iCAT, as depicted in Figure 1)
Next, we will present a number of proof-of-concept implementations of recommender techniques aiming to demonstrate how recommenders could be applied to collaborative ontology engineering platforms in the biomedical domain. 5
To study the general feasibility of recommending concepts to experts, we implemented three selected recommendation techniques for the ICD-11 project as a proof-of-concept. In the following we will illustrate the implications of different recommender techniques through examples using actual interaction data obtained from selected ICD-11 users. We will refer to these users as LB, AR and RC from here on. 5.1
We used real data excerpts, extracted from the ICD-11 and its log of changes, to demonstrate how content based recommender systems can be applied to collaborative ontology engineering platforms in the biomedical domain.
For all users U and the set of all concepts C, we extracted their previously changed concepts Cu C, together with all words from the title and the words included in the definition of a concept c 2 Cu.
Before doing so, we performed three additional tasks: (i) stop word (e.g.: is, as, and, so etc.) removal, (ii) stemming, a mechanism used in natural language processing to reduce words to their stem and (iii) data cleaning, i.e. we have removed special characters from the textual properties of the concepts. For this example, we used the stop word list available from the Natural Language Toolkit.
For similarity calculations, we used cosine similarity, as there is
evidence that it provides good results for existing collaborative
environments
~ WLB = fdisease : 906; skin : 125; contact : 33; acute : 97g W~AR = fdisease : 386; skin : 34; contact : 0; acute : 39g W~RC = fdisease : 272; skin : 841; contact : 399; acute : 65g V~LZ1 = fdisease : 1; skin : 1; contact : 0; acute : 0g V~L56 = fdisease : 0; skin : 1; contact : 0; acute : 1g V~Z20 = fdisease : 1; skin : 0; contact : 1; acute : 0g ~
Wu represents all words and their respective number of appearances in the title and definition of all d 2 Cu and V~c collects all words and word counts per concept c 2 C. Table 1 shows excerpts of W~ and V~ depicting word lists for the users LB, AR and RC as well as excerpts of the concepts LZ1 (“LZ1 Impairment of normal functioning resulting from skin disease”), L56 (“Other acute skin changes due to ultraviolet radiation”) and Z20 (“Contact with and exposure to communicable diseases”), after stemming and stop word removal.
Cosine similarity was calculated for every pair cos( W~u; V~c) where u 2 U and c 2 C and stored in the user-concept similarity matrix MU;C (see Table 2).
In Table 2, recommendations for LB can be generated from MU;C by suggesting those concepts that have the highest similarity values. In our example, this approach would recommend LZ1 and Z20. 5.2
To demonstrate collaborative filtering-based recommendations, we used log data from iCAT to calculate similarity between all user u 2 U . In this approach, two users are similar if they have modified similar concepts. The concepts c 2 C that have been changed by u from November 2009 to 30th August 2011, are denoted as sets Cu (as seen in Table 3). For the users LB, AR and RC these concepts are H40:1 (“Primary open-angle glaucoma”), BP N CS (“Benign proliferations, neoplasms and cysts of the skin”), XII (“Diseases of the skin”) and DBS (“De Barsy syndrome”).
CLB = fH40.1, BPNCS, XIIg CAR = fDBSg
CRC = fBPNCS, XII, DBSg Table 3. The set Cu (left) represents an excerpt of the set of all concepts (right) modified by u from November 2009 to 30th August 2011 that was used for calculating similarity between users.
Based on this matrix, we used the Jaccard coefficient to calculate similarity between users based on the set of all concepts cu for all u 2 U resulting in a user-user similarity matrix MU;U (see Table 4) as Mi;j = J (ui; uj ).
LB AR RC LB 1 0 0,5 AR 0 1 0,33
RC 0,5 0,33 1 Table 4. The user-user similarity matrix MU;U lists the Jaccard coefficient similarity values between all pairs of users.
We introduced an arbitrary threshold minSimilarity set at 0:0001 which excludes user pairs with very little similarity. Based on that modified table, we start to count the number of changes done by uj on every single concept c 2 Cuj and store the results in the matrix NU;C (see Table 5).
H40:1 BP N CS XII DBS LB 2 1 12 0 AR 0 0 0 20
RC 0 12 45 14 Table 5. NU;C , the user-concept change count matrix lists the number of changes done by every user u 2 U to every concept c 2 C.
The values for the user concept similarity matrix OU;C are calculated as depicted in Equation 1. (1)
The final results are illustrated in Table 6. Collaborative filtering recommends concepts for a user based on the concepts that similar users found interesting. In our example, the user LB and RC have a high similarity rating of 0; 5, and user RC has contributed to DBS many times. Hence, the concept DBS is recommended (similarity of 21) to LB.
H40:1 BP N CS XII DBS LB - - - 21 AR 0 15,96 59,85
RC 3 - - Table 6. User-concept similarity matrix OU;C holds the similarity results according to Equation 1 for the collaborative filtering approach. The higher the similarity the likelier it is, that the user is interested in that concept. 5.3
Our final proof-of-concept implementation will assume that users are most likely interested in concepts that are ontologically related to the ones they have already shown interest in. The more a concept is interlinked or referred to by previously changed concepts in the ontology, the more related it is to the interests of that specific individual. An excerpt of the ICD-11 ontology, represented as directed graph using the is-a relationships, is depicted in Figure 2.
We assumed that user LB has only changed the concept B57:1 (“Acute Chagas disease without heart involvement”). To explore related concepts, we followed links in the ontology until either a predefined depth level was reached or enough highly interlinked concepts were discovered. Table 7 depicts all values for expansions of concepts B57 (“Chagas’ disease”), B57:2 (“Chagas disease (chronic) with heart involvement”), B57:3 (“Chagas disease (chronic) with digestive system involvement”), SC1 (“Selected Cause is Remainder of certain infectious and parasitic diseases in the Condensed and Selected Infant and child mortality lists”), SC2 (“Selected Cause is Trypanosomiasis”) and M ortality (“Tabulation list for mortality”).
Next we traversed along all paths, as shown in Figure 2 and
Table 7, from all previously edited concepts, counting the number of
encounters of each concept. The higher the number of encounters,
the more weight it will receive in a ranking of concepts for user u.
34
28
24
23
19
18
The final recommendations for RC (see Table 8) produced by
our different proof-of-concept recommendation algorithms differ
significantly with regard to level of detail, scope and sub-domain.
Recently or currently browsed concepts could be included in the
similarity calculations to increase the scope of the generated
recommendations. While we believe that concept recommender systems
could represent a useful instrument to augment user experience
on collaborative ontology engineering platforms in the biomedical
domain, we have not performed evaluations of our implementations
yet. Understanding what kinds of recommender techniques are
useful in what kind of contexts, including a more sophisticated analysis
of user-behavior similar to