Ontologies such as the SNOMED Clinical Terms (SNOMED CT), and the Medical Subject Headings (MeSH) play a major role in life sciences. Modeling formally the concepts and the roles in this domain is a crucial process to allow for the integration of biomedical knowledge across applications.
In this direction we propose a novel methodology to learn formal definitions for biomedical concepts from unstructured text. We evaluate experimentally the suggested methodology in learning formal definitions of SNOMED CT concepts, using their text definitions from MeSH. The evaluation is focused on the learning of three roles which are among the most populated roles in SNOMED CT: Associated Morphology, Finding Site and Causative Agent. Results show that our methodology may provide an Accuracy of up to 75%. For the representation of the instances three main approaches are suggested, namely, Bag of Words, word n-grams and character n-grams.
The biomedical domain is characterized by an exponential growth in the produced data volumes, primarily scientific published articles, knowledge and databases, nucleotide sequences and protein structures. To handle such amounts of data and information, the notion of organizing the biomedical knowledge using ontologies has been the focus point of many initiatives and activities in the biomedical domain [6]. The basic motivation is that since ontologies represent a conceptualization of how things are organized in reality in the underlying domain, this formal representation may provide an actual language for the community, with which they can talk about entities and concepts, and exchange data in the same representation. Moreover, sharing the same conceptualization of entities in the biomedical domain allows researchers to communicate new facts and knowledge referring to the same concepts that may be found with different labels across several different data sources.
More formally, an ontology is a set of logical axioms which model the reality of the domain. With the advent of description logics (DL) [2] and OWL's description logic flavor OWL DL [www.w3.org/TR/owl-guide/], the task of designing and implementing formally ontologies has become easier, as the ontology engineers may express the ontology concepts and their relations without losing computational completeness, and in parallel retain decidability of reasoning systems. In practice DL has become the leading formalism for representing ontologies, a trend which nowadays is also supported by many popular ontology editors such as Protégé [protege.stanford.edu/] and OBO-Edit[oboedit.org/]. Notably, many large biomedical ontologies have adopted this formalism, such as GALEN [17], which was also the first biomedical ontology to be developed in DL and the NCI Thesaurus [ncit.nci.nih.gov/].
In particular, SNOMED CT [www.ihtsdo.org/snomed-ct/] has adopted the lightweight description logic EL + +, which allows for tractable reasoning. For several years now, research on how other biomedical ontologies may be translated in DL has been conducted [9,21,12]. Strickingly, the application of formal ontologies in the biomedical domain has produced interesting results, e.g., the works of Rubin et al. [19] and King et al. [13] to name a few. In the former work, the authors used the Foundational Model of Anatomy (FMA) ontology to develop a methodology through which they can automate reasoning about penetrating injuries. In the latter work the authors presented Adam, a laboratory robot that can perform independent experiments to test hypotheses and interpret findings without human guidance.
It is, thus, evident that coherent formalization of biomedical ontologies has valuable applications in the biomedical domain. In this work we present a novel methodology to learn formal definitions of biomedical concepts from their textual definitions, which can be considered as a first step towards the automated process of creating formal biomedical ontologies. We approach the problem from three different perspectives: (i) learning the Bag of Words (BOW) representation that participate in the expression of each role3 within the textual definitions of concepts, (ii) learning the word n-grams that participate in the expression of each role, and, (iii) learning the character n-grams that participate in the expression of each role. The first approach is a standard representation methodology in text mining, while the main difference between the other two approaches is that the former considers the order of words in the definitions and the fact that the words may form composite terms, i.e., word n-grams, while the latter considers the order of characters, i.e., character n-grams. Finally, we merge the three representations into one, by combining their features, in an effort to get the best of all worlds.
The rest of the paper is organized as follows. Section 2 refers to related work pertaining to the generation of formal definitions from unstructured text. Section 3 introduces formally the problem that we are addressing. Section 4 introduces the methodologies used, and describes how the textual definitions we are using for our analysis were obtained and annotated. Section 5 presents the results of our experimental evaluation and discusses the findings, and Section 6 concludes and provides pointers to possible extensions, applications and future work.
As argued above, formal ontologies are useful, but their creation is a labor intensive task. Hence, it is desirable to automate aspects of it. Towards the direction of processing automatically text descriptions from biomedical ontologies, there exists much related work (e.g., [3,5]); however, most of these approaches assume the existence of an ontology to be enriched, while our method is mostly related to approaches that can create an ontology from scratch. Under this scope, in the following we refer to approaches that aim to generate axioms from unstructured text.
Several approaches use a deep syntactic analysis of natural language definitions [24], and others apply lingustic patterns [20]. Fuzzy logic components have been also developed towards the automated generation of logical axioms [16], whilst semantic analysis and word sense disambiguation have also contributed [1] towards the completion of the task of generating formal descriptions of entities and relations. Finally, there exist also running systems which are designed to generate ontologies from text, an example being Text-To-Onto [15], which, however, in their majority do not take into account axioms and instances.
In the biomedical domain little work exists concerning the automated axiom generation and learning of formal representations [14]. More specifically, regarding MeSH, to the best of our knowledge there is only one work that attempts its representation in OWL, but in an indirect way that makes use of CISMeF (Catalogue and Index of French-speaking Medical Sites) which encapsulates the French version of MeSH [21]. Other works that target the expansion of MeSH, attempt to automatically suggest its expansion with synonyms and provide alternative definitions for the concepts, but not in a formalized way [25].
A non-exclusive list of other recent examples of works that attempt to model formally concepts in the biomedical domain are the works by: Boelling et al. [7], who attempt to model biochemical processes; Chepelev and Dumontier [8], who define the Chemical Entity Semantic Specification (CHESS) for the representation of polyatomic chemical entities, their substructures, bonds, atoms, and reactions using Semantic Web technologies; Stenzhorn et al. [23], who attempt to map clinical documentation to the formal representation of SNOMED CT; Jupp et al. [11], who introduce Populous, a tool that may populate ontologies from the analysis of spreadsheets; and, finally, the work by Hastings et al. [10], who use OWL and description graphs to represent classes of chemical entities, such as molecules, ions and groups.
Evidently, there is significant research work towards representing the biomedical knowledge using a formal representation, yet there is a gap regarding the generation of formal definitions from unstructured biomedical text. Our work addresses this open challenge by presenting a novel methodology for extracting formal definitions from unstructured text. We argue that the proposed methodology may aid the time-consuming and demanding process of ontology generation, evolution and maintenance, by constituting the first step to transit successfully from unstructured text to the extraction of formal biomedical concept definitions.
3 The Problem of Learning Formal Definitions 3.1 Formal Definitions in SNOMED CT SNOMED CT ( [22]) is a medical ontology describing concepts such as anatomical structures, disorders, organisms, and it is becoming adopted by a growing number of countries worldwide as a reference vocabulary in clinical research [18]. Its underlying structure is based on formal logics, more specifically on the lightweight Description Logic EL + +. Description Logics (DLs) can express a rich network of different types of relationships between concepts. For example, using the SNOMED CT vocabulary one can express that the concept Baritosis is caused by Barium dust by writing:
The formal semantics of SNOMED CT are a key advantage, however, they come at a cost. Adding new concepts to a formal ontology is a tedious, costly and error-prone process, that needs to be performed manually by specially trained knowledge engineers. The suggested methodology can provide assistance in this process by automatically extracting the relations between concepts from text. The approach is based on the assumption that the set of roles (relations) remains relatively stable while the set of concepts constantly increases. To facilitate the addition of new concept description, we formulate the following problem: for a given input sentence in natural language that is annotated with two SNOMED CT concepts decide whether the sentence describes a role between the two concepts and which role precisely. More formally, we can express this problem as a classification problem. Let C be the class label based on which the training of the classifiers takes place. C can be the label of any role R contained in SNOMED CT. Each example (instance) is a sentence, denoted with I, which is annotated with SNOMED CT concepts, and for which a set of features X has been computed, which are explained in Section 4. Thus, I = [X 1 , ..., X N ]. If I is a sentence which describes a role R i between two SNOMED CT concepts, where R i ∈ R, then I is a positive example for this role, and, hence C = R i in this case. Therefore, the problem of role extraction from unstructured text can be seen as a multi-class classification problem, following the aforementioned representation. In the following section we explain in detail the methodology of building a data set for learning three SNOMED CT roles, which are among the most widely populated in SNOMED CT.
The methodology comprises three steps: (i) creating a data set with labelled instances from which the roles can be learned based on a set of features, (ii) representing formally the instances, for which we explore three different representations, and, (iii) using a machine learning methodology to train a model in the produced data set following the suggested instance representation that may recognize any of the labelled roles in an unseen input sentence. For the purposes of our evaluation this latter step is conducted on the created data set, using 10−fold cross validation to measure the performance of the tested methods. Steps (i) and (ii) are explained respectively in Sections 4.1 and 4.2 that follow.
In order to obtain high quality sentences that describe the role between two SNOMED CT concepts we need to obtain sentences that primarily contain both concepts, and in turn, describe a relation between the two. For that purpose, we chose to use MeSH definitions that contain SNOMED CT concepts, since MeSH definitions are produced manually by medical experts, and, thus, constitute precise, scientifically valid, and high quality sentences.
The first step for the aforementioned transition is to obtain a mapping between MeSH and SNOMED CT concepts. Such a mapping exists via the Unified Medical Language System (UMLS). UMLS defines a Concept Unique Identifier (CUI) for each of the UMLS concepts. Each CUI may be associated with one or more concepts from external libraries or thesauri, such as MeSH and SNOMED CT. Analyzing this association, we extracted the CUIs that are associated with both a MeSH and a SNOMED CT concept, which is interpreted as a mapping between the two concepts. Using the latest UMLS version (2012AB), we obtained in this manner a total of 21, 461 mappings.
Next, we used the produced mappings to create a high quality data set for learning roles between SNOMED CT concepts. For the purposes of our data set creation we focused into three widely populated roles in SNOMED CT, namely Associated Morphology (AM), Causative Agent (CA) and Finding Site (FS). To explain in detail the process of the data set creation we define the following notation. Let R i be a SNOMED CT role, where R i ∈ AM, CA, FS. Let A and B be two SNOMED CT concepts that populate R i such that: A ∃R i .B. From all of the MeSH definitions we retained only those which define any A involved in R i and we further filtered with the definitions that contain B. After filtering there were 424 MeSH definitions remaining. For the purposes of filtering we identified the definitions that contain B through annotating with SNOMED CT concepts. The two annotators were used sequentially to provide a broader coverage of annotations; hence, we considered the union of the provided annotations from the two tools. Following the aforedescribed steps, the produced dataset contains 424 instances in total for the three roles (AM, CA and FS). Its details are summarized in Table 1.
Using the aforedescribed dataset, we can now proceed with providing a description of how the features can be generated, with which the instances may be represented for the learning process. For the feature engineering, we use three approaches: (i) Bag of Words, (ii) Word n-grams, and, (iii) Character n-grams. The three approaches are described next, and are summarized with an example in Table 2. The example is drawn from the formal description presented in Equation 1. In all three approaches, the annotated sentences are split in a way such that the words that occur between A and B, may be isolated and processed. The basic assumption behind this alignment lies in the hypothesis that each role R i has a characteristic way of being expressed in natural language text, which may be captured by the analysis of the words that occur between concepts A and B. All three representations have a default feature weight equal to the value of 1 if they occur in this text, or 0 otherwise. We also expand these representations to their weighted versions, i.e., instead of boolean representation of the features, a real value is used.
Bag of Words (BOW) Representation: The representation of text following the Bag of Words model has been used traditionally both in the fields of information retrieval and text mining [4]. According to this representation, each distinct term constitutes a dimension of the collection. More formally, in our case let T be the text between A and B in an annotated sentence (instance). T is naturally a series of ordered words, e.g.,
The BoW representation of this instance will be the unordered set of all unique words w i ∈ T . Thus, the feature space according to BoW comprises the union of all unique terms appearing in all text definitions T . Each instance can then be represented as a set of features X i : w i . In its simple (unweighed) version, as a value of each feature we use 0 or 1 (boolean representation), depending on whether X i = w i occurs in T (1) or not (0).
Word n-grams Representation: Given T and an assigned value to a parameter n, we can expand the BoW representation in order to represent each instance with all the possible word n-grams occurring in T . For the extraction of the word n-grams we are using a sliding window of search in the ordered words of T . Note also that this representation includes at least all features of the BoW representation; in fact, if n = 1, the word 1-gram representation is reduced to the BoW representation. Regarding the weight of each feature, in the simple (unweighed) version, we use a boolean representation, as previously.
Character n-grams Representation In an analogy to representing instances at a word n-gram level, we can also represent instances at the character n-gram level. Given again T and a value for the parameter n, we now examine T as an ordered series of characters, instead of words. For the extraction of the character n-grams, as in the case of word ngrams, we are using a sliding window of search in the ordered characters of T , and we do not exclude space characters. Again the weight of each feature in the simple (unweighed) version, follows a boolean representation, as previously.
Weighted Feature Representations: In all of the three aforedescribed feature representations of instances, we have assumed a boolean representation for the feature values. Ideally, we would like to have a real value v i for each feature X i acting as a weight that would discriminate their flat contribution of the boolean representation. For this purpose, we utilize the dataset and define a global weight for each feature, which can be computed always on the part of the dataset kept for training. A local weight is not a realistic option, as the MeSH definition sentences are usually short, significantly shorter than text passages or documents. Hence, for each feature X i of any of the aforedescribed representations, we define a weight: v i = P (X i ), where P (X i ) is the probability of occurrence of feature X i in the training corpus. However, since the training corpus contains instances from several roles R i (class labels), it is important to discriminate the probabilities of the features' occurrences per role (class). Hence, to create the weighted representations of instances following any of the aforementioned schemes, we create for each feature X i , k features, where k is the number of all roles (labels): X i1 ...X ik , with the real value of each feature being: v im = P X i |R m , and m ∈ [1..k]. Thus, the weight of each feature is the probability of its occurrence in the respective role, and each instance in the weighted version may be now represented with k * X features.
Combined Feature Representations: A final consideration is the representation of the instances using the union of all the features that were described in each case. This potentiality can show whether the synergy of word n-grams and character n-grams may provide better predicting power for the extraction of roles from unstructured text. Naturally, this combined representation can be utilized both for the weighted and the unweighed versions of the aforedescribed instance representations.
The experimental evaluation was conducted on the dataset that was created as explained in Section 4.1 and summarized in Table 1. As machine learning methodologies we compared four different state of the art supervised algorithms, namely: (1) Logistic Regression, (2) Support Vector Machines, (3) Multinomial Naive Bayes, and, (4) Random Forests4 . For the evaluation we apply 10−fold cross validation and for performance measuring we report on the overall accuracy, precision, recall and F-Measure per role (AM, CA, and FS), and macro-averaged precision, recall and F-Measure over all roles.
The results are reported in Tables 3 and 4 for the unweighed and weighted versions of the instance representations respectively. Analyzing the results from the perspective of how difficult it is to learn a model than can recognize each of the roles, the reported numbers examining both tables suggest that the easiest role to identify is CA, with an F-Measure that can reach up to 82.9%, the second easiest is FS, with an F-Measure that can reach up to 80.3%, and the hardest role is AM, with an F-Measure that can reach up to 65.8%. A second finding when examining the results from the point of view of the underlying feature representation is that the character n-grams tend to report better results than the rest representations. More precisely, the character 3-grams report the top performance in terms of accuracy and macro-averaged F-Measure (75.71% and 74.91% respectively). In addition, the combined representation does not seem to boost the performance of the character n-grams, which is probably due to the fact that the word n-grams cannot perform individually equally well as the character n-grams. However, further ensemble or combination, or feature selection methodologies will have to be explored in the future in order to assess whether there are feature subsets that can boost the overall performance with the concurrent reduction of the feature space complexity. Considering the tested classifiers, though the performance differences in absolute numbers are subtle, SVM tends to produce the better accuracy and F-Measure in the majority of the cases. Furthermore, in some cases Naive Bayes tends to select the majority class (cf., Table 4), which means that further training examples would be needed in these representations for the specific setup.
Finally, with regards to the contribution of the weighted representations the reported numbers in Table 4 suggest that this is minor, and, in fact there are several cases where the weighting of the features drops the performance, compared to the numbers shown in Table 3. It seems that the selected global weighting is not enough to differentiate the predictive power of the features. Perhaps, a local weighting of the features would be able to make a fine-grained differentiation, taking into account the way TF-IDF works for the term features in typical text mining tasks. However, such an application is not feasible in our case since the instances are short and the representation is extremely sparse.
In this paper we introduced a methodology for learning formal definitions from unstructured text, formulating the problem from the point of view of learning roles between concepts. From this perspective the results are encouraging and we showed that for three widely populated SNOMED CT roles, namely Associated Morphology, Causative Agent and Finding Site, the task can be achieved with accuracy reaching up to 75%. On the other hand side, the suggested methodology has limitations and the experimental results showed that there is definitely room for improvement regarding the underlying representations. In this direction, the next step is to analyze the cases that the method fails, and bring into the surface the reasons, as well as the nature of additional features that might be used to correct these cases. In the same direction, no syntactic or semantic information was taken into account for the feature engineering, which we plan to integrate in future work. In addition, the text pre-processing for the preparation of the dataset can be enriched with more elaborate steps such as: better alignment of sentences which contain roles inside concept names, or nested roles. Finally, another direction that we will look into in our future work is the extraction of roles from MeSH definitions in comparison to employing the existing SNOMED CT roles for learning formal definitions, motivated by the different nature of MeSH and SNOMED CT.