This paper describes practical experiences and lessons learned in the project “A Semantic Web for Pathology”, a project using domain ontologies and ontology-driven natural language processing to support a content-based retrieval of text and image-based medical information. In trying to develop the target application ontology we investigated the potential of reusing the huge amount of domain knowledge already available in ontology-like form in the medical domain as an input for the domain conceptualization. According to our experiences we confirm previous findings in the knowledge acquisition literature and in recent surveys of the state of the art in the ontology engineering area: 1). building ontology-based applications is still a tedious process mainly because of the lack of mature tools and methods, which can handle the requirements of real-world applications; and 2). using existing ontologies in new application contexts is currently related to considerable efforts, which might outweigh the benefits of the reuse process. We use the insights gained during this project to derive a set of guidelines for developing Semantic Web retrieval applications in similar domains.
Parallel with the continuous dissemination of ontologies and the emergence of methods and tools which assist the ontology developers in creating, managing and extending them, ontology engineering is currently evolving from a pure research topic to real-world applications. This state of the art is emphasized by the wide range of European and international projects with major industry involvement and by the increasing interest of small and medium size enterprizes asking for consultancy in this domain. However, due to the well-known difficulties associated with ontology engineering activities, building and deploying ontologies at a large scale, beyond the boundaries of the academic community, has to be supported not only by elaborated technologies and tools, but also by extended case studies and comprehensive guidelines, which aid the engineering team in practical situations in their attempt to develop ontology-based applications. This need is currently addressed in several joined initiatives, such as the W3C Semantic Web Best Practices and Deployment Working Group1 or the
European research projects OntoWeb and KnowledgeWeb.2
In this paper we aim at modestly contributing to these efforts by reporting our practical experiences and lessons learned in building an ontology-based application in the medical domain. The project we present uses domain ontologies and ontology-driven natural language processing to support a content-based retrieval of medical information in textual and image form. In trying to develop the target application ontology we investigated two engineering alternatives: 1). the reuse of the huge amount of domain knowledge already available in ontologylike form in the medical domain, followed by 2). the usage of domain-centered text documents as an input for the domain conceptualization. The second engineering experiment was triggered by the poor fitness of use of the reuse-derived medical ontology in the application context (as described below). Our experiences confirm previous findings in the knowledge acquisition literature and in recent surveys of the state of the art in the ontology engineering area: 1). building ontology-based applications is still a tedious process because of the lack of proved and tested support tools and methods; and 2). reusing existing ontologies in new application contexts is currently related to considerable efforts which might outreach the costs of a new implementation.
The setting of our project is often considered a typical use scenario for the
deployment of Semantic Web technologies, both w.r.t the application type (i.e.
information retrieval) and the application domain (i.e. medicine)[
The rest of the paper is organized as follows: Section 2 gives a brief overview
of the application scenario. Section 3 describes the generation of the ontology
used by the retrieval system and summarizes lessons learned during this project
phase. We conclude with extending existing, domain-independent guidelines for
building Semantic Web retrieval applications[
The aim of the project “A Semantic Web for Pathology” was the realization of a
content-based retrieval system for the domain of lung pathology.3 The application
2 http://ontoweb.org, http://knowledgeweb.semanticweb.org/index.html.
3 This work has been partially supported by the EU NoE “KnowledgeWeb”
(FP6507482). The project “A Semantic Web for Pathology” is funded by the DFG
(German Research Foundation) and is a cooperation between our working group, the
Charit´e Hospital in Berlin, Germany and the Department of Appl. Computer
Linguistics at the University of Potsdam, Germany.
uses an explicit semantic representation of the text documents (i.e. pathology
reports), which are linked to the semantically annotated digital images they
describe[
The most important design criterion to obtain a high user acceptance was
the seamless integration of the system into the workflow of a pathologist. This
minimal invasive usage is the result of a careful design of the user interface,
query language and multi-modal content presentation methods. The system
offers the ability to write a medical report while examining the images with the
conventional microscope or related software, thus restoring the current missing
link between text and image-based data.4 A new patient report is annotated
with ontology concepts by a text processing component[
The system components and their main features are described in detail in
[
Pathology”.
and extend the domain ontology. In this paper we focus solely on the insights
gained during the knowledge acquisition phase; practical experiences in using
existing tools such as ontology editors, reasoners or APIs are described in [
In the last decades a wide rage of methodologies for building ontologies have
been proposed [
6 In the second, corpus-driven ontology engineering experiment the ontology was generated from a “training” set of approximately 400 domain-specific documents, while the remaining 350 documents were used to evaluate the results. ontologies, was not included to the final application because of its poor fitness of use and user acceptance.
In the following we give some details about the generation of the ontologies
and conclude with lessons learned during this attempt. Due to space
considerations we focus on reporting on our experiences in the reuse-oriented setting,
while the second one is described in [
Due to the difficulties and costs involved in building an ontology from scratch,
methodologies often recommend to rely on available domain-related ontologies
in order to simplify the domain analysis and the conceptualization phases.
However, most of the methodologies deal with this topic only in passing, and there
is no common practice for reusing current knowledge sources for generating new
ontologies.7 For example, Uschold and King [
Likewise other engineering disciplines reuse implies finding, evaluating,
customizing and integrating existing (reusable) components to a target system [
Therefore, in order to differentiate among the concepts within the remaining 10 libraries, pathology experts selected 4 central concepts in lung anatomy (i.e. “lung”, “pleura”, “trachea” and “bronchia”) and extracted similar or related concepts from the libraries. They considered the list of all distinct concepts related through a relation of any kind10 to the 4 initial concepts. The result was a set of approximately 1000 concepts describing the anatomy of the lung and lung diseases and served as initial input for the application ontology.
The linguistic analysis of the patient report corpus evidenced the contentrelated limitations of UMLS w.r.t. the concrete vocabulary of former. These concepts are really used by pathologists when putting down their observations and will therefore also occur as search parameters when using the retrieval system. We modeled additional, pathology-specific concepts (approximately 200), such as the components and typical content of a medical report, and integrate them in the available ontology library. Besides content-related adaptation needs, the analysis of the generated ontology outlined the absence of concept names in the German language: due to the predominance of English in denominating UMLS concepts and the predominance of German terms in the pathology report archive we had to translate the English terms in order to enable an automatic, ontologydriven annotation of the documents.
After identifying the relevant knowledge sources and the list of concepts
which can be used as input for our application, we translated the UMLS data
model to the OWL model and transformed the relevant data from one format to
another[
Reusing medical ontologies for the generation of the target application ontology proved to be a tedious, time-consuming and error-prone process. The success of reusing ontologies was influenced to a large extent by two factors: the costeffective evaluation and customization of the source ontologies11.
Typically ontology reuse starts with the identification and evaluation of knowledge sources potentially useful for the application domain, which differ both in the covered content, and in the formalization (thesauri, XML-Schemes and DTDs, ER models, UML diagrams, textual descriptions etc.). An automatic integration of the source ontologies means not only the translation of the representation languages to a common format, but eventually also the matching of the resulting schemes. Due to scalability and heterogeneity issues both of 10 The UMLS Metathesarus contains 7 important relations between concepts: parent, child, sibling, narrower, broader, related-other, source-synonymy. 11 These tasks required more than 50% of the total engineering time. these steps could not be performed efficiently using current techniques in our project setting. However our experiences showed that using simple techniques which rely on the smallest common denominator of the employed knowledge sources—usually their vocabulary— significantly increased the efficiency of the reuse process. This finding was confirmed by an analysis of the resulted ontology w.r.t. its dependencies to the source ontologies: the target ontology reuses their vocabulary to a large extent—properties and axioms are not supported explicitly in many knowledge sources and their integration to the target ontology was non-trivial for both humans and tools.
UMLS contains several problematic modeling decisions which have been
often described in research projects aiming to integrate it in knowledge-based
applications[
Another important issue was the usage of the ontology by the community of
domain experts, which reported serious acceptance problems w.r.t. the
UMLSbased ontology: domain experts seemed to have difficulties in trusting the content
of the ontology and in methodically extending it for a more detailed
representation of pathology-specific knowledge. This was the main motivation for
alternatively building a second application ontology on the basis of the domain corpus
of patient records provided by our health-care partner[
The engineering process relied on the same engineering methodology as the
first experiment, while XML-based medical reports were employed as an input
for the conceptualization phase. 12 The main advantages of the latter
experiment compared to the UMLS-based one were the significant cost savings and
the increased fitness of use of the generated ontology w.r.t. the semantic
annotation task. From a resource point of view, building the first ontology involved
four times as many resources than the second approach (5 person-months for
the UMLS-based ontology with 1200 concepts vs. 1.25 person-months for the
“text-close” ontology of a similar size). The evaluation of the suitability of the
two ontologies to semantically annotating medical documents (described in [
According to our findings in the mentioned setting we conclude this paper with a
(incomplete) list of guidelines for building Semantic Web retrieval applications in
12 Refer to [
Description 1 2 3 4 5
Domain analysis Ontology reuse Ontology management Updates Ontology learning
Specify the tasks the ontology will be involved in. They have consequences on the content and on the representation of the target ontology. Different tasks imply different relevance criteria for selecting potential reusable resources during knowledge acquisition and require adequate evaluation criteria: -- Semantic annotation task · concepts should be denominated in natural language · the natural language used in the ontology should be the same as the one used
by the users and in the documents to be annotated · concepts should be denominated using naming conventions and in a
linguistically predictable form · modelling decisions should be recorded during the conceptualization phase in
order to simplify the ontology-driven annotation -- Information retrieval task · the ontology should be formal to enable automatic reasoning · concepts should be denominated in natural language to enable an ontology
based query formulation · the ontology should provide a rich semantic representation of the domain to
improve and refine the retrieval algorithm Despite of the large number of very comprehensive medical ontologies, reusing them is related to significant costs, which might outweigh the costs of a new implementation: -- Knowledge resources which will be reused to create the target ontology eventually necessitate considerable modifications in order to fulfil the requirements listed in (1): · concepts are denominated in an ad-hoc manner even within the same ontology · the semantics of the concepts is sometimes encoded in their names · most of the medical ontologies are stored in proprietary forms, there are no
translation tools · most of the ontologies are modelled in an ambiguous way -- Existing medical ontologies have a considerable size, but a relatively simple structure. Adapt your reuse methodology to their particularities: · a complete evaluation of their application relevance is extremely tedious, if
not impossible. · the same domain is covered to a similar extent by several ontologies. There are no fundamental differences among them w.r.t. their suitability in the Semantic Web context. Eliminating candidate ontologies which are definitely not relevant is sometimes more feasible than an attempt to a complete evaluation. · even when an ontology is assigned a high relevance score, its usage in the application setting might depend on the availability of tools which are able to handle it and on the user acceptance. · matching and merging ontologies with overlapping domains imposes serious scalability and performance problems to available tools in this area.
Nevertheless, using simple algorithms (e.g. linguistic matchers) considerably increases the efficiency of this activity. · the merging results are to be evaluated by human experts. Due to the size of the ontologies, the merging methodology should foresee a flexible and transparent involvement of the users during the process in order to reduce the complexity of the merging evaluation. · reasoning over these models requires inference engines which are able to
manage their dimensions The size of the target ontology requires powerful storage mechanisms with adequate reasoning support (e.g. for automatically checking inconsistencies) Elaborate a detailed evaluation framework to control ontology evolution. The maintenance of large size ontologies requires additional effort for documenting modelling decisions.
Medicine is a dynamic domain, most of the ontologies change within relatively short time. Updating the target ontology under these circumstances can be very tedious, especially if the source ontologies were not directly integrated to the new application. Identify update needs and elaborate a detailed update strategy.
The success of ontology learning approaches depends on the quality of the document corpus (domain-focused documents are expected to perform better).
Data noise (telegraphic writing style, the intensive usage of non-standard abbreviations etc.) is common to medical texts such as medical findings. The ontology learning algorithm should be able to deal with these particularities.
The knowledge acquisition process should be performed incrementally, because
of the complexity of the domain to be modelled.
the domain of medicine. As a starting point we used a set of domain-independent
guidelines emerged in the European project OntoWeb, which focus less on
technical aspects, but mainly on “issues that relate to the business environment
that affects the deployment, integration and acceptance of the ontology-based
application”[