Background: Current search engines are keyword-based. Semantic technologies promise a next generation of semantic search engines, which will be able to answer questions. Current approaches either apply natural language processing to unstructured text or they assume the existence of structured statements over which they can reason. Results: Here, we introduce a third approach, GoWeb, which combines classical keyword-based Web search with text-mining and ontologies to navigate large results sets and facilitate question answering. We evaluate GoWeb on three benchmarks of questions on genes and functions, on symptoms and diseases, and on proteins and diseases. The first benchmark is based on the BioCreAtivE 1 Task 2 and links 457 gene names with 1352 functions. GoWeb finds 58% of the functional GeneOntology annotations. The second benchmark is based on 26 case reports and links symptoms with diseases. GoWeb achieves 77% success rate improving an existing approach by nearly 20%. The third benchmark is based on 28 questions in the TREC genomics challenge and links proteins to diseases. GoWeb achieves a success rate of 79%. Conclusion: GoWeb's combination of classical Web search with textmining and ontologies is a first step towards answering questions in the biomedical domain. GoWeb is online at: www.gopubmed.org/goweb
With the tremendous growth of the World Wide Web, search engines became key
tools to find documents. Search engines retrieve documents for a user’s keywords
from a large index and rank them by various criteria. While such keyword-based
search is fast and powerful to retrieve single documents, it is far from the vision
of answering a user’s questions by “understanding” the user’s query and answers
in the documents as put forward already in the late 1960s [
Consider e.g. a biomedical researcher, who might ask questions such as the following: Which model organisms are used to study the Fgf8 protein? Which processes are osteoclasts involved in? What are common histone modifications? Which diseases are associated with wnt signaling? Which functions does Rag C have? Which disease can be linked to fever, anterior mediastinal mass, and central necrosis? What is the role of PrnP in mad cow disease?
The Web holds answers to these questions, but classical keyword-based search is not suitable to answer them, since the keywords are required to appear literally in text. However, documents do contain statements such as e.g. “wnt signalling is linked to cancer” or “we studied fgf8 expression in Zebrafish development”. If there is background knowledge that cancer is a disease and that zebrafish is a model organism, then the above questions can be answered.
The use of such knowledge is at the heart of the semantic web, which
promotes the use of formal statements and reasoning to deliver advanced services
not available on the Web now [
Besides the expressiveness of OWL, mark up for vocabularies and meta-data
emerged such as Simple Knowledge Organisation Systems (SKOS) [
All of the above standards serve the need to formally represent knowledge and facilitate reasoning over this knowledge. They require explicit statements of knowledge. As a consequence, the amount of such structured data is still small in comparison to the unstructured data. Thus, to support semantic search there are essentially two approaches: Those, searching structured documents and reasoning over them and those, searching unstructured documents and extracting knowledge and reasoning over it. The knowledge extraction step of the latter uses combinations of natural language processing, information retrieval, text-mining, and ontologies for the knowledge extraction.
Table 1 summarises a number of semantic search engines, which work on
structured and unstructured documents. The former comprise Swoogle [
As mentioned, the above systems are limited by the availability of structured
documents, a problem addressed by approaches such as the semantic media wiki
[
Hakia, START [
Wikipedia is a valuable resource to answer questions and hence some engines
are specifically applied to it. PowerSet applies e.g. natural language processing
to Wiki in a similar manner to Hakia. QuAliM [
The above tools are intended to be general and as a result they generally do not cover the biomedical domain well. Searching for example for a protein such as Fgf8, PowerSet and Hakia do not offer an answer for model organisms. They offer information on the protein, but are not able to find zebrafish as a model organism.
Engines such as askMedline, EAGLi [
Finally, besides all of the automated approaches, Google, Yahoo! and Microsoft use humans to answer questions in their services Google Answers, Yahoo! Answers and MSN Live Search QnA. 3 www.cuil.com 4 clusty.com 5 www.carrot-search.com
Closely related to semantic search, is semantic hyperlinking as implemented
in the Conceptual Open Hypermedia Service (COHSE). COHSE annotates a
given web page with concepts and offers services based on the identified concepts
([
None of the above systems combines the simplicity of keyword search on the vast amounts of Web documents with the use of biomedical background knowledge to filter large keyword results with biomedical ontologies. Here, we address this by introducing the GoWeb search engine. GoWeb issues queries to Yahoo and indexes the snippets semantically with ontology terms. These are then offered to filter results by concepts. In order to demonstrate the power of this approach in question answering, we evaluate GoWeb on three benchmarks with questions on gene/function, symptom/disease, and protein/disease relationships and compare it to existing solutions. 2
GoWeb is an internet search engine based on ontological background
knowledge. It helps to filter potentially long lists of search results according to the
categories provided by the GeneOntology (GO) [
To facilitate the easy usage of GoWeb, the website is structured in three panels. Please consider figure 1, a screen shot of the GoWeb web page. The left panel contains the background knowledge and other retrieved meta-data. For quicker navigation the panel is organized in the four categories what, who, where, when (4w panel). The top-right panel holds the query field (search panel). In the third panel below the input field, the actual search results are presented (document panel).
The semantic filters of GoWeb are presented in the 4w panel. The what category contains the result tree for the ontological background knowledge. It uses a tree representation for relations between the ontology concepts of background knowledge. In this way a concept in the tree represents the concept itself and its children. Thus, allowing the user to select relevant concepts with one click.
The who panel contains filters related to persons, companies or institutions. In the where panel there are filters related to locations, such as cities, countries and similar. The when category assists with time related filters, e.g. a date. 2.1
Algorithm The search is executed by a traditional keyword based search service. We use Yahoo! Search BOSS service6. The result of a submitted search is a list of textual summaries for web documents, called snippets. Next, GoWeb uses entity 6 developer.yahoo.com/search/boss/ w F recognition techniques to map concepts from the background knowledge to the snippets.
The algorithm for the identification of ontological concepts in text is based
on the GoPubMed algorithms [
The identified entities of each result and found keywords are the basis for a co-occurrence based semantic filtering mechanism of GoWeb. The filter uses the part-of and is-a relationships from GO and the tree structure of MeSH. These relations are used to induce the relevant search result for each concept from the background knowledge. The induction result for all search results for a query is also used to select important concepts. These top concepts are selected for the entire background knowledge and for each sub category. 2.2
Architecture The workflow for GoWeb can be described as follows. The user submits a query through the search form on the GoWeb website to the server. The server preprocesses the query and sends a search request to the search service. The search service returns the first results. The first results are then annotated, highlighted (concepts and keywords), rendered and sent to the user.
The user can now already browse the first results. Once the first results are processed, the server starts fetching the remaining results. This is done for up to 1000 results. Then all fetched results are annotated. To reduce the response time, the fetching and annotation of results is done in parallel. The annotation information is then used to induce a tree representation and top concepts of the ontological background knowledge for the submitted query (result tree). Then this information is rendered and sent to the user-interface using AJAX technologies like JSON to reduce the required time and bandwidth. An overview is available in Figure 2.
If the user selects a concept in the result tree by clicking it, a request is made to update the presented documents. This includes a filtering step of the result set and a re-ranking step. For an illustration see also Figure 3. The new ranking is based on the found concepts, keywords and the original ranking.
Once the user has decided to open a web page, GoWeb offers to highlight the page with the keywords and concepts from the background knowledge. This is done with a proxy-based solution. The server checks if this page is annotatable, e.g. the content is HTML based. Then the GoWeb-server fetches the site and analyzes the content, adds the annotations and sends the result to the user. If the content is not processable by the proxy there is an automatic forward to the original content.
Fig. 2. General workflow for GoWeb showing the main components and the interactions between the external services.
The goal of GoWeb is to use ontologies and text-mining in semantic web search to answer questions. Here, we give some examples and evaluate the question answering capabilities of GoWeb on three benchmarks. In the introduction we raised questions such as the following: Which model organisms are used to study the Fgf8 protein? Which processes are osteoclasts involved in? What are common histone modifications? Which diseases are associated with wnt signaling?
An answer to these questions can be found using GoWeb. For example Fgf8 is studied in Mice, Zebrafish; osteoclasts are involved in bone resorption; common histone modifications are Methylation and Acetylation and the wnt signaling pathway is associated with neoplasms like breast cancer, tumors or leukemia. The answers where directly obtained with GoWeb using simple keyword searches and the induced background knowledge. For example the answer to the first question can be found in the following way: first submit the query Fgf8 the answer is directly shown as listed concepts in the organism’s part of the background knowledge (see also Figure 1). To retrieve the corresponding search results click on the organism. This simple strategy can be generalised to support semi-automatic question answering. Next, we will demonstrate this using three independent benchmarks. 3.1
Genes and Functions
The first benchmark is based on the association of Genes and their functions.
The BioCreAtIvE 1 (Task 2)[
The test set for GoWeb now contains all GO annotations and genes from the competition, which were labeled as identified with high confidence in the results. This yields a list of 457 gene names with a total of 1352 GO concepts. For example for “Rag C” there are 10 annotations: cytoplasm, small GTPase mediated signal transduction, RNA splicing, transcription, GDP binding, protein heterodimerization activity, small monomeric GTPase activity, heterotrimeric G-protein complex, protein binding and nucleus.
For a test run GoWeb was given a gene name as query. Then it is checked if the induced ontology tree contained the concepts corresponding to the expected functional annotations for the gene. For all 457 submitted names the search returned documents and the GoWeb system could identify GO concepts from these snippets. The results show that for 58.1% (785 of 1352) of the benchmark concepts are contained in the tree (recall). 3.2
Symptoms and Diseases
The second benchmark demonstrates the capabilities of GoWeb concerning the
association of symptoms and diseases as carried out by general practitioners and
medical researchers. It is based on the study by Tang et al. [
In the experimental setup for GoWeb the same keywords as in the original
paper, where used. Each diagnosis has been mapped to the corresponding MeSH
concept, if possible. Then in the experiment a query was given to the GoWeb
system and the resulting induced background knowledge tree was evaluated.
As an additional comparison for GoWeb we also applied this benchmark and
experimental setup to the GoPubMed system [
GoWeb can provide the correct answer in 20 out of 26 (77%) cases. In 10
of these cases, the answer term is found directly in the top categories of the
Diseases subtree of MeSH (see Table 4). The cases 8, 10 and 18 are not marked as
successful, although they mention the searched concepts. But they all find only
one article, the article[
For example for the case study number 28 GoWeb finds 126 articles for the query “ANCA haematuria haemoptysis”. Under diseases one can find the MeSH concept “Churg-Strauss Syndrome”. A click on the concepts in the tree retrieves three snippets containing the concept. The resulting snippets are: – Laboratory imposed restrictions on ANCA testing – 63 (5): 594 – Annals of the Rheumatic Diseases The laboratory has performed ANCA testing only when the request form indicated . . . haematuria (requests from the renal/transplant unit), ChurgStrauss syndrome, . . .
ard.bmj.com/cgi/content/extract/63/5/594 – include haemoptysis (13% of patients), cystic bone lesions (4-20% of . . . Some 70-75% of patients with Churg-Strauss syndrome have ANCA . . . www.hospitaldoctor.ie/hospital doctor/pdfs/HOS DOC MARCH APRIL 05.pdf – Churg-Strauss Syndrome - Patient UK
Pulmonary: asthma, pneumonitis and haemoptysis . . . patients are perinuclearANCA (p-ANCA) positive (antimyeloperoxidase antibodies) . . . www.patient.co.uk/showdoc/40024815/ The GoWeb system performs better than GoPubMed because the underlying search engine has a larger repository of documents. Additionally, it can also index the full text, if it is available on the web. The MEDLINE search for all PubMed based search engines, like GoPubMed, is only based on abstracts. This corresponds with the fact that the MEDLINE search returns often none or only one article abstract. 3.3
Proteins, Diseases and Evidences
Linking proteins and disease is a key task for many molecular biomedical
researchers. The third benchmark for GoWeb is based on the questions from the
TREC Genomics Track 2006[
For GoWeb the questions were transformed into keywords. The complete listing of questions and keywords is available in Table 5. A question was marked as successfully handled, if there was a snippet that contained a valid answer. The second aspect addressed with this benchmark was to show the capabilities of the filtering feature. Filtering by background knowledge helps to reduce large results to a smaller set of relevant documents. It was marked as applied, if the answers, where found by using the filtering feature.
The answers for the first four questions (160–164) are shown in Table 6. They
also demonstrate what kind of textual evidence GoWeb can provide as answers.
The answer to question 160 (What is the role of PrnP in mad cow disease), for
instance, was found by submitting the keywords and selecting the MeSH concept
“Encephalopathy, Bovine Spongiform” (mad cow disease is a synonymous label
for the concept) as semantic filter. The selected answer was now in the first
part of the remaining relevant results. The given number 378 corresponds to the
original position. This demonstrates that without the filter this answer would not
have been found normally [
There are two main reasons for GoWeb to not be able to find a answer for all question in the benchmark. The first is that the question is too complex. The answer is too long to be formulated in a sentence or snippet. For example the question 171 contains actually two questions. The second reason is that the question domain is not sufficiently modelled in the background knowledge. For question 178, for instance, skin biology has no corresponding concept and is too general to be mentioned directly in text. 4
The three used benchmarks provide a basis for the evaluation of GoWeb. They demonstrate the power of the idea but also its limitations. The starting point is the usage of snippets. This is already a limitation in terms of completeness. Snippets can be seen as an abstraction. They try to summarize facts related to the keywords. A snippet might be too short to contain very complex facts, some information will be lost. But important information is more likely to be in the snippet, because important facts are often near in the original text. Thus it is more likely to be also contained in the snippet. The co-occurrence is used as approximation for relation extraction. One advantage of using this simpler approach is the reduced computational complexity. A proper NLP based approach would need more computational time. With GoWeb the complete annotation for 1000 results can be done on-the-fly and still provides response time of a few seconds.
If more information than the snippet is required, it is necessary to fetch the web pages and analyze them. This could be done on runtime for the result set or pre-calculated during a crawl of the internet. Both options have major drawbacks. Fetching and analyzing of web pages on-the-fly is not feasible with the requirement of a short response time. The crawling of the internet is possible, but requires a significant amount of resources in terms of hardware and bandwidth to keep the index up-to-date. This is demonstrated by the popular search engines. Each of them uses several data and computing centers. Although the search requests from the user are the main load, keeping the index up-to-date is an important aspect. One advantage of a separate crawl is the chance to build a semantically enhanced index. Such an enhanced index offers the option to include concepts directly into the search and not as post-processing step like GoWeb.
To include all information from a web page will increase the recall. But it would also increase the problem of false positives from matching errors or irrelevant parts. The false positives would also unnecessarily increase the size of an index. With the option to pre-process the information, e.g. with topic recognition or disambiguation algorithms, this can be compensated. For a specialized system with a limited number of documents and known document structure, a semantic index might be a better solution than GoWeb.
The application of text mining for concept identification is important for finding the relevant snippets in the search results. A simple keyword can not easily replace the additional information from the background knowledge. This includes synonymous labels and related concepts. For example for heart diseases in MeSH there are over 570 related labels.
The types of questions handled best by GoWeb have to be transformable
into keywords and concepts. The answer provided by GoWeb will be either an
inferred concept or a sentence/short text extract in the snippet. These options
reduce the types of questions which can be answered by GoWeb. For example in
a question classification by Tomuro and Lytinen [
In comparison to existing systems which are mainly focusing on searching OWL and RDF content (e.g. Swoogle, SWSE) GoWeb covers a broader area. Current RDF search engines cover millions of RDF statements, whereas the internet search engines cover billions of websites. Unfortunately, most of the information in websites is unstructured text. GoWeb tries to bridge the semantic gap with the limited amount of available semantic annotations by employing text-mining for extraction of ontology concepts from text. In a nutshell, GoWeb exploits that keywords and ontology terms co-occurring in snippets are often facts.
Traditional search engines like Google have the coverage but they miss the explicit usage of ontological background knowledge. They only present a long list of results. This works very well for simple retrieval of documents, but has limits for more complex task, e.g. answering questions. Here the semantic filtering with concepts as in GoWeb helps to reduce the result list to relevant answers. If a snippet does not contain the relevant terms, it is likely to be not relevant.
For other internet search based systems like Hakia or PowerSet the advantage of GoWeb is its additional background knowledge for the biomedical domain. Only GoWeb combines the usage of the GeneOntology (GO), the Medical Subject Heading (MeSH) and protein identification. A clustering of text labels like Clusty or Carrot can not replace the structural knowledge of an ontology. In comparison to PubMed-based systems GoWeb can index the additional resources of full text articles on the web.
The search interface of GoWeb provides with the what -where-who-when categories a simple way to browse the results. Next to the actual search results GoWeb offers also additional information like definitions of concepts or Wikipedia links.
Together with the filtering mechanism to reduce the result set from 1000 possible results to a small number of relevant entries GoWeb offers powerful tool for semantic search in the biomedical domain. Overall, the paper shows that co-occurrences of keywords and ontology terms in lists of snippets are often approximating facts, which aid to answer questions. The simplicity of the approach ensures scalability and speed, still achieving success rates of up to 80%. Acknowledgments We kindly acknowledge the help and discussions from our colleagues. In particular we would like to thank A. Doms and T. W¨achter. For the helpful disccussions about the implementation we acknowledge the GoPubMed team from Transinsight, especially M. Alvers, L. Barrio-Alvers, J. M¨onnich, C. Plake, and M. Zschunke. This work was supported by the European Union for the Sealife Project, Project Reference IST-2006-027269. 5 Acute ”Aortic regurgitation” depression Tree: Endocarditis, Bacterial 7 (1000) abscess 6 oesophageal cancer hiccup nausea vomiting Tree: Adenocarcinoma AND 2 (1000) Intestinal Obstruction 7 hypertension ”adrenal mass” Top categories: Cushing Syn- 41 (1000) drome 8 ”hip lesion” child no, bmj article 258 9 HRCT centrilobular nodule ”acute respira- Finds the case studies this 15 tory failure” analysis relies on 10 fever bilateral ”thigh pain” weakness no, bmj article 500 11 fever ”anterior mediastinal mass” central Top categories: Lymphoma 66 (323) necrosis 12 multiple ”spinal tumors” ”skin tumors” Top categories: Neurofibro- 21 (240) matoses 14 ”ulcerative colitis” ”blurred vision” fever Tree: Vascultits 2 (1000) 15 nephrotic syndrome ”Bence Jones” ventric- Top categories: Amyloidosis 20 (247) ular failure 16 hypertension papilledema headache ”renal Tree: Pheochromocytoma 1 (31) mass” 17 ”sickle cell” pulmonary infiltrates ”back Top5 snippet is ACS 1000 pain” 18 fibroma astrocytoma tumor leiomyoma sco- no, bmj article 1 (47) liosis 19 pulmonary infiltrates ”cns lesion” OR no 87 ”Central nervous system lesion” 22 CLL encephalitis 25 ”portal vein thrombosis” cancer 26 ”cardiac arrest” exercise young Tree: West Nile Fever 3 (1000) Tree: Phlebitis 9 (1000) top categories: Cardiomy- 22 (1000) opathy, Hypertrophic 27 ataxia confusion insomnia death Tree: CJD 17 (1000) 28 ANCA haematuria haemoptysis Top categories: Churg- 3 (126) Strauss Syndrome 29 myopathy neoplasia dysphagia rash perior- Top categories: Dermato- 4 (32) bital swelling myositis 30 ”renal transplant” fever cat lymphadenopa- Top categories: Cat-Scratch 13 (322) thy Disease 31 ”buttock rash” ”renal failure” edema no 120 33 polyps telangiectasia epistaxis anemia Top categories: Telangiecta- 33 (1000) sia, Hereditary Hemorrhagic 34 ”bullous skin” ”respiratory failure” carba- Top categories: Epidermal 4 (25) mazepine Necrolysis, Toxic 36 seizure confusion dysphasia lesions no 1000 37 cardiac arrest sleep Tree: Brugada Syndrome 3 (1000) e N y in R h
t ce ov P a
p n B
p rm .. s/
c m .s
m , r
o p o n o lo p
a S C h e p n e i s a tayh ised e e s u rm ca o s l n a z
n l
n . o se i
t a a
r D g β m im A yz e , n h z E e l p li I
/ E r is , s re ir
a
d n m i i
b
p M 3 .
M 1 e
C c
n d B e an U r
y om d ll e a en
n g . c r z s In fe co al In I a
D n : 1 . 0 6 1
n ip io r t c a s c n fi ra it
e 2 a d 1
i s a ts
. .. .. ]8
9 l e , n c 7 t s e .
s e ... isea icm
d t fi e e r s d e n k o k e
n ffi e o d e a w n nd i
n