Many developers of biomedical knowledge bases typically validate and update formalized knowledge based on reviews of full-text scientific articles, but finding text relevant to domain concepts can be tedious and prone to errors. Prior methods have automated this process by matching term-based patterns within a single sentence. In our work developing a knowledge base of autism phenotypes, specified using Semantic Web standards, we are interested in finding multi-sentence sections of text that contains complex phenotype definitions. In this paper, we present a text-mining method that incorporates both ontology- and rule-based semantics to determine which section is relevant. We evaluated our method in undertaking text extraction for the set of full-text articles used to create the knowledge base. We show that our method has higher precision and recall than a term-based approach in identifying definitions that contain complex patterns and occur across sentence boundaries.
Keywords: Information Extraction, Text Analysis, Semantic Ontology, Rules Base, OWL, SWRL Web,
Biomedical knowledge resources, such as terminologies and ontologies, are important
for community-based annotation and sharing of data. Creating and maintaining these
resources is challenging given the rapid growth of scientific knowledge. Generally,
scientists, annotators and developers try to keep up by using search engines that find
publications relevant to given concepts in the knowledge resource. However, users
still need to review the publications and find sections within the documents that relate
to the concept being searched. One solution to this challenge is to automatically
identify the relevant parts of a full-text document. Prior methods, such as Textpresso
[
Our work is motivated by the needs of developers of an ontology of autism
phenotypes [
The Savant Skills Factor was based on … current and ever scores of four ADI-R items: visuospatial ability, memory skill, musical ability, and computational ability. Item scores were summed and divided by total number of items to generate a score between 0 and 1. … Participants were then divided into two groups: Savant-positive and Savant-negative … .
The autism ontology uses the Web Ontology Language (OWL) [
Finding text relevant to a search term is undertaken by some web search engines,
which provides a few lines of site description or snippet for a search result to indicate
the relevance of a web page to the search query. Google, for example, uses the
description provided by meta tags, references to the web pages, Open Directory [
Another related work in this field is question-answering systems, which return a
part of a text from a corpus as the answer to a specified question. These techniques
rank the snippets from the relevant documents by criteria such as: containing expected
types of named entities, the percentage of overlap with question terms, containing
lexical patterns, and using information from lexicon dictionaries [
In our work, we find the most relevant parts of science publications to domain concepts using existing OWL ontologies and SWRL rules. As noted, both provide formal definitions of domain concepts and their relationships to other concepts.
As the first step, we need a formal representation of domain concepts. In this work,
we use vector space modeling, a common method in the web search engines for
indexing web pages [
Besides the classes and properties that are mentioned in the rule, we use ontology hierarchies to extract more related concepts and incorporate them in the concept presentation. We consider the parents and grandparents of the main concept and its related concepts extracted from the corresponding rule as potential related concepts that can strengthen our concept vector modeling. However, the relevance of these concepts from the ontology hierarchy decreases by their distance from the main concept in the hierarchy graph. Therefore, we weight these related terms in the vector presentation less than the main class and the related concepts explicitly mentioned in the rule that defines the concepts. As a heuristic choice to capture these differences, we count the frequencies of the parent classes or properties as half of the actual frequencies, and the frequencies of grandparent classes or properties as one-quarter of the actual frequencies.
After we model the concept, we go through a publication to find the most relevant
parts of the text for a particular concept. As the first step, we look at the vector
representation of the concept and found all the terms associated with that concept as
the concept terms. Concept terms are the terms that have weights greater than zero in
the concept vector presentation. We then go through the publication and mark all the
occurrence of the concept terms in the text. We cover occurrences of different forms
of a concept terms by applying, Porter stemming algorithm, a common stemming
method for English terms [
Given the occurrences of concept terms in a publication, we treat them as indicators of relevant parts of the text and use single linkage hierarchal clustering to find the candidates for the most relevant parts of the publication. The average sentence length in our corpus is 20 words. In the single linkage clustering we use 30 words as a heuristic threshold and in every step we merge the closest clusters that are separated by less than 30 words. Thus, we ensure that a continuous section of text without any concept term is limited to a few sentences and the whole cluster is continuously correlated to the concept. We consider these clusters as the candidates for the most relevant parts of the text.
In this work, our goal is to quantify the relevance between concepts and pieces of text.
Therefore, we need a mathematical modeling of texts. We use vector space modeling
again to provide a common basis for comparison. Vector space modeling for
documents’ text is based on term frequencies. To model a part of a text as a vector,
we first remove the stop words, the most common English words that are not
informative about the context. We use a common list of stop words in English [
After we present both text words and domain concepts as vectors, we need to compute the correlations between them in order to find the most relevant parts of a publication for a concept. To do that, we use cosine similarity as the measure of correlation between texts and concepts. The cosine similarity for two vectors is the cosine of the angle between them. Similarity values range from 0 for orthogonal vectors to 1 for parallel vectors.
In this work, we applied our method on the autism phenotype ontology and the papers used to derive those concepts as mentioned in Section 1. We examined only the top five most relevant parts of the publication for each concept and had an autism ontology expert review these text sections to determine the efficacy and accuracy of whether each section was related or not to the definition of the concept. To investigate the significance of using ontological hierarchies and rule bases, we compared our method to a baseline, which is a term-only method. The baseline method is a variation of our method that only uses the terms in the semantic concept-modeling step. That is, our baseline approach does not include concepts from the ontology or rules that are related to the term. To eliminate bias in the assessment of the performance of the two approaches, the expert was blind to which method produced the extracted text.
The autism ontology contains 1726 classes and properties, and it includes 156
SWRL rules that correspond to 145 phenotype definitions. The ontology and rules
were based on a review of 26 publications that had been undertaken by one of the
authors (AKD) and other domain experts in autism [
In our evaluation strategy, we knew that every concept had been defined in the corresponding publication. For further investigation of the relevance strength in our results, we asked the reviewer to identify which of the five most relevant parts of the publications for a concept contained a clear definition. We used this to calculate the recall for each method, which is the percentage of concepts that their definitions were found. Table 2 shows the recall of the concept- and ontology-based methods in finding the definitions of the concepts in the corresponding publication text.
In this paper, we present a novel method to find parts of text in scientific publications that relate to definitions of biomedical concepts. In comparison to methods that do term matching to find individual sentences that contain a single concept or pairwise sets of concepts, our ontology-based approach addresses the challenge of finding a concept definition that occurs across multiple sentences or that is semantically similar to predefined concepts. Our approach was particularly driven by the need to identify text related to complex domain concepts like autism phenotypes, in which use different terms and terminologies refer to similar concepts. Our evaluation shows that ontology hierarchies and rules have a large impact on identifying the relevant parts of the text. This is because of the informative nature of ontological hierarchies and the inter-relationship of concepts maintained in rule bases.
As future work, we are planning to improve upon our method by using the text’s syntactic structures through constituent and dependency parsing methods. The syntactic and dependency information can be used in the text modeling to improve the concept relevance detection. Also, we will consider further addition of name entity recognition methods, which can extract the information about the biomedical concepts outside of the ontologies in texts. We are planning to use this information to develop a richer presentation of text and find relationships between the publication text and the queried biomedical concept.
Acknowledgments. The authors would like to acknowledge Martin O’Connor and Siddharth Taduri for their comments on the approach. This research was supported in part by grant R01 MH87756 from the National Institutes of Health.