In the life sciences data must be described unambiguously. We apply this principle in our multi-modal bio-imaging database in which images are stored together with comprehensive metadata. We use ontology terms to describe the semantic content of images. Ontologies are obtained from dedicated ontology repositories in the life sciences. For our users, the process of image annotation with ontology terms was proven to be a challenging task. Therefore, we have made improvements on both usability and speed of annotation. We developed search facilities across our ontology collection and implemented a new graphical ontology viewer. This tool allows for both querying and visualizing ontology terms by means of a 2Dgraph representation. Our viewer provides a means to collect ontology terms and at the same time familiarizes users with ontologies and their structure. In making these tools available we succeeded in our goals to reduce time and effort for accurate image annotation.
INTRODUCTION
The Cyttron Scientific Image Database for Exchange
(CSIDx) is a multi-modal imaging database for images
produced in the life sciences [
CSIDx is built to support a wide range of imaging
modalities and techniques and it is the backbone database
of the Cyttron project [
In CSIDx, we propose that metadata as the information that
describes an image is essential to support exchange and
linking as well as analysis of images [
In order to assure a comprehensive annotation that also
represents the actual image acquisition conditions, CSIDx
maintains metadata about the 'who', the 'what' and the 'how'
of an imaging experiment [
In this paper, we describe our efforts and tools to support
and facilitate image annotation with ontology terms. In
particular, we describe our ontology viewer, a graphical
tool developed to assist image annotation based on
ontologies. CSIDx currently incorporates 37 life-science
related ontologies, the majority of which are retrieved from
the Open Biomedical Ontologies (OBO) Foundry [
CHALLENGES OF USING ONTOLOGIES IN IMAGE
ANNOTATION
In an earlier prototype of CSIDx, users could annotate
images by selecting ontology terms from our ontology tree
browser. This application depicts the hierarchical relation
in the ontologies as a tree view; this view only displays the
subClassOf relation. With an interactive tree view (cf.
Figure 1), users can navigate by collapsing and expanding
terms in the hierarchy and can select terms to be assigned
to the image under annotation. The ontology tree browser
parses ontologies in the OWL-format by means of the Jena
framework [
FORM OF A SOLUTION To address the challenges of using ontology terms for annotation, we examined the difficulties faced by our users and the limitations of the hierarchical visualization of the ontologies. Testing with our prototype provided useful knowledge on how users interact with ontologies. Through participatory evaluations, we learned that users need to learn the ontology content, to build a mental model of ontology structure and to extract information from ontologies. The complexity of the annotation task increases due to the lack of search facilities, the overwhelming amount of terms and the lack of experience with and understanding of ontologies. Hence, we implement a solution that aims to improve the annotation process both in terms of usability and in terms of ontology comprehension. In particular, we provide search facilities on the ontology terms corpus and implement the ontology viewer, a graphical tool used for both querying and visualizing ontology terms. In combination with these facilities and in order to reduce the annotation effort, the concept of MyTerms was also introduced in the workflow of the annotation process. A⊆B A⊆∃ P.B A⊆B∩C ∀x [B(x) → A(x)] ∃x∃y [A(x)∧B(y)∧P(x,y)] ∀x [A(x) → B(x)∧C(x)] A⊆ B∪C
B⊆A, C⊆A
Querying Ontologies Using a Database Back End
From the user perspective, quick concept (keyword-like)
searches across the ontologies are essential in order to
complete an extensive image annotation with ontology
terms. Keywords and textual descriptions of images as
conceived by the image owner need to be mapped to
existing ontology terms. Such a procedure is not easily
accomplished without any search facilities especially when
the user is not familiar with the content of the ontologies.
Ontology querying mechanisms can involve the use of
dedicated RDF query languages such as SPARQL [13].
More elaborate forms of querying, like reasoning can be
accomplished with reasoners such as Pellet [11] and
KAON2 [
By designing and populating the ontology database, which currently consists of 565,600 terms and 825,724 relations, we are able to support querying facilities across the ontologies. Users of CSIDx can search for a corresponding term by keywords using either the ontology viewer (cf. next section) or a simplified web search form.
MyTerms: A User Specific Collection of Terms for Annotation To reduce the effort required for identifying annotation terms in the ontology collection, we have introduced the concept of MyTerms in the workflow of the annotation process. MyTerms is a collection of user specific ontology terms that are saved under a user's profile and can be reused across annotations. Prior to an actual image annotation, users can browse the ontology collection with the querying tools available looking for terms that are relevant to their study or field of research. During an image annotation, users assign terms to images by selecting terms from their own relevant subset (MyTerms) instead of the complete corpus of terms available. This process is an attempt to minimize the effort of searching for terms (search once, use in all subsequent annotations) and to reduce the overwhelming amount of ontology terms to a subset that is both meaningful to the user and easier to browse and use. The MyTerm concept can be further elaborated to match the structure of our system. In CSIDx, users are organized in groups that correspond to their actual research institute or group and this organization is often used throughout the interface as a mechanism for exchanging shared resources, such as images or microscopes. Therefore, we also provide the possibility of sharing identified terms among group members, who are likely to work on a similar topic. In the case of group shared ontology terms, the time and effort spent by a group member to locate and identify useful terms across the ontologies profits all members of the research group. On the whole, MyTerms assure that the admiringly time-consuming process of mapping metadata to existing ontology terms does not need to be unnecessarily repeated.
THE ONTOLOGY VIEWER The ontology viewer provides a graphical interface for querying ontology terms and a means to visualize the ontology structure. We believe that a graphical representation can assist our users in building a mental model next to building a collection of terms. In practice, it is a tool to assist building a MyTerms list and an attempt to demystify ontologies to our users by making the relations among ontology terms obvious. The application is developed in Java and deployed as a WebStart application. It can be accessed via the CSIDx web interface or used as a standalone application for registered users.
The ontology viewer (cf. Figure 2) consists of two major panels: a query form and a 2D viewer. In the query form, users search for ontology terms within an ontology by providing one or more keywords and by specifying the level of detail for the search. Queries can be performed on the label, synonym or definition of terms and keywords can be combined in an 'AND' or 'OR' query. Users can choose from the list of results to either visualize particular terms or directly add terms to their MyTerms list.
In the 2D viewer, the ontology structure is represented as a graph in which terms are graph nodes and relations are graph edges. Selected ontology terms, as collected from a query, are used to produce a sub-graph of the ontology graph. This sub-graph provides the local context for the selected nodes which are highlighted green to distinguish from their connected terms.
A short description with information on any given term can
be obtained by mouse over the corresponding graph node.
Regular graphical manipulations are supported on the
ontology graph which can be zoomed, paned, rotated and
sheared. In this manner, user can adjust the view to better
understand the displayed relations. The ontology viewer
also provides different graph layouts to support a more
suitable or preferred arrangement in space, especially in the
case of complex sub-graphs. The supported layouts are
given in Table 2. To improve clarity of the presentation,
both the text labels of either nodes or relations and nodes
other than the selected nodes can be toggled on or off. The
graph drawing and manipulation is implemented by means
of the Java Universal Network/Graph Framework (JUNG)
[
DISCUSSION AND CONCLUSIONS Overall, the CSIDx ontology viewer provides an informative graphical interface to the collection of ontologies in CSIDx. As most ontologies are derived from the OBO matrix, this viewer is also an alternative graphical entry point to exploring the most popular and acknowledged ontologies in the domain of the life sciences. Importantly, for most CSIDx users, this interface is their first impression on biological ontologies and a first step towards familiarizing themselves with the concept and content of ontologies. Compared also to the web based querying facility that lacks the graph display, users have reported that the connected terms often help clarify ambiguities: when the label of a term can be explained differently depending on the context or when the description of a term is insufficient, the connected terms are often conclusive on the exact meaning of the term. As a result, users feel more confident that they have selected the proper term for precise annotation. The graph representation also seems to assist users in rethinking the way they translate their desired annotation to ontology terms. By exploring the ontology, users often conclude on more terms than they initially queried for and they often express the desire to automatically add to their user term list (MyTerms) the whole graph structure as displayed in the 2D viewer. Overall, the ontology viewer contributes towards a more complete and accurate annotation based on ontology terms.
As they familiarize themselves with the ontology structure, users demonstrate the wish to further interact with the ontology. Often, they request to expand the displayed nodes, a requirement that equals with interactively traversing the complete ontology. While the ontology viewer was basically aimed to provide some context for the queried terms rather than a complete overview of an ontology, we are interested to explore if the graph representation can be useful as a querying tool in itself. While the contributions of a graphical interface for ontology exploration are encouraging, the overall performance remains an issue. Querying an ontology is satisfactory fast but displaying the graph structure has significant memory requirements and may halt for large ontologies. Also, while mapping familiar keywords to ontology terms, many users reported a difficulty in specifying which ontology to query in. In the current prototype, querying for a keyword in the whole collection of ontologies is not supported and needs further attention. Our results conclusions: can be represented by the following 1. The Ontology viewer provides an intuitive interface for (novice) users; the options are self explanatory and the user is assisted in understanding ontology concepts while at the same time ontologies are queried and terms selected. The mapping to the graphs is very helpful to that respect. 2. The MyTerms list provides a good simplification to the otherwise “oversized” ontologies. Users can now use ontology concepts with ease in their image annotations. 3. To assure visibility in the interface of the ontology viewer a fast response to queries is required which can be provided through a transformation of the ontology structure to an RDBMS.
FUTURE DIRECTIONS The work presented in this paper is the result of a participatory design trajectory; in the design phase we aimed to learn how we could bring the concept of annotation of images with ontologies across the users of the CSIDx database. The design process also included requirement generation by users. We accomplished this design phase with an artifact that is a fully working prototype, rather than proposing a final application. Now that we have gained sufficient information on how novice users can work with ontologies, we can make next steps towards observatory evaluations in which different annotation strategies can be tested. Further user evaluations by surveys will render sufficient data for statistical analysis on ontology interaction.
Annotations are the basic components of the semantic structure in CSIDx. Furthermore, the relations included in the ontologies provide additional material to be explored. Initially, we wish to investigate the direct relations as maintained in the RDBMS. Still, mechanisms to profit from the OWL-DL expressiveness can be expanded based on the existing annotation with ontology concepts.
for Java,