Morphbank, an on-line collection of museum-quality biological images, is an NSF funded project designed to facilitate the on-line collaboration of biologists from around the world [3]. Our primary focus is to aid in the collection and management of images that are useful in phylogenetic research. Morphbank users are actively collaborating on the creation of information that represents the associations among images and related biodiversity data objects. This paper describes the Morphbank annotation tool and data models and gives examples of how users create structured information in the system. Schematized annotation provides biologists with a flexible framework to create semantically-rich annotations using their own data models.
The discovery, identification, and documentation of biological entities are time consuming and tedious tasks. The subtle differences between similar species may be so minute as to require the collaboration of several experts to identify. Each taxonomic group has many experts who can assist in the identification of specific organisms. However, with the increase in the number of new organisms that have been discovered and a decrease in number of senior specialists, identification and curation of data have become more difficult. Often, it involved the need for scientists to travel to the location of the specimens or for specimens to be sent to the scientists for first hand examination. This is still standard practice among most biologists today. Morphbank contains information about organisms. Each image in the system is associated with one or more specimens. Each specimen is a representation of information about an organism. Specimens are in turn associated with localities, contributors, taxonomic concepts, and a variety of annotations.
The design and development of the Morphbank system identified several challenges in discovering and creating information about images and their related objects. Finding images and specimens associated with a specific species and genus, Finding and recording information about that image and its related objects, and The discovery and recording of ad-hoc associations among the various objects.
Discovering and recording ad-hoc data is the most problematic. It is particularly difficult to find ways that users can record associations among objects.
As long as data is well formatted and constrained to the database
schema then finding and retrieving it is simple. However, as
we’ve discovered, there is no practical limit to the amount of
information a scientist may wish to store with a particular
specimen. Most of the knowledge is contained in the memory of
these scientists or in hand written notebooks. Although it is
recognized that manual annotation is expensive and time
consuming it is nevertheless still essential in documenting
collaborative knowledge in biological systems [
Morphbank is an open Web repository of images serving the
biological research community. It is currently being used to
document specimens in natural history collections, to voucher
DNA sequence data, and to share research results in disciplines
such as taxonomy, morphometrics, comparative anatomy, and
phylogenetics. Morphbank can serve as a virtual reference
collection of named organisms or a resource for comparative
morphological study; new use cases are continuously added [
Each object in the Morphbank system is uniquely identified and
includes a set of standard fields that assist us in cataloging the
location and type of each object, the identification of the user who
added the object, the date and time of creation, an optional
description of the object, and the last time the object was
modified. These attributes allow anyone accessing Morphbank
sufficient information to find and catalog data and associate
related objects. Each object is externally identified by a Life
Science Identifier (LSID) [
Since each Morphbank object is uniquely identified, any object can be the target of a stored reference. A single column within a Morphbank table holding a foreign key may refer to several an object of any type. Thus a collection object can be heterogeneous. For instance, an annotation object may define an association among images, specimens, locations, users, or even other annotations.
This flexibility allows for the creation of complex collections of objects that can be shared with other users of the Morphbank system. Although there are a series of predefined relationships in Morphbank, the use of unique identifiers allows users to define an unrestricted set of complex relationships of objects within the confines of the system.
The users of the Morphbank database system have identified
several requirements for image and object annotation to be used
by authorized users of the system. These requirements are
consistent with the Specifications For Image Annotation On The
Semantic Web as described W3C in their draft document [
There has been considerable effort put into the development of
general purpose Web-based annotation tool sets over the past
several years. In their paper on Web annotations, Venu
Vasudevan and Mark Palmer [
The problem of biodiversity annotation is that biologists have increased the number of specimens they can gather but have not increased their ability to catalog, identify, and study them. Collaborations still include the exchange of physical specimens and the manual annotations of the images using indexed cards and paper documents. At the functional level, many users have developed their own specific but proprietary solution to this problem. Through the use of Morphbank and a Web based annotation tool, we can solve most if not all of these problems.
A variety of annotation technologies allow users to add value to images by creating associations between those images, text and other digital objects. Morphbank takes this one step further by making the associations into first class objects that can themselves be annotated and associated with other objects. Morphbank also allows associations to take on specific semantic characteristics that constrain their meaning and thereby improve searching and understanding.
Image annotation is available in a variety of image management Web sites. The simplest annotations are found in systems that support attaching tags to images and other media. Flickr.com and YouTube.com, e.g., allow users to add text attributes (tags) to images and use those tags to support searching. FotoTagger.com, among others, goes a step further and allows the tags to be attached to specific locations on images.
Blogging is another form of image annotation in which text passages are linked to images, Web pages and other digital objects. A blog entry creates an associate between its own text and the linked objects.
Annotea.org supports the creation of RDF attributes for image tags. These attributes can be used to provide search inference capabilities for users of image repositories.
Another annotation strategy involves the development of
laboratory notebooks such as those under development at the
United States Department of Energy, National Collaboratories
under the guidance of Dr. Jim Myers [
An annotation is an assertion that a collection of objects are related in a particular way. For annotation and search purposes, the Morphbank object annotation tool provides a minimum set of tools common to all annotation requirements. The tool uses the terminology of the Darwin Core [1] biodiversity ontology initiative. We strove to keep the tool-set as simple and as straight forward as possible and to provide specializations that make it easy for particular types of annotations to be created.
Flexibility is particularly important because all annotations must be made using only a Web browser. The template for the tool defines several functional areas required for basic biodiversity annotation and specimen determination.
Using the ability to store complex metadata with annotations gives allows us to define associative semantic relationships with ad-hoc data and other Morphbank data. The data model that supports annotation is intended to be extended to incorporate additional types as needed by users. The categories of annotations in the current system are as follows: General: There are instances where users desire to make some ad-hoc comments concerning a collection of images, specimens or other objects. The requirement for this type of annotation was made to allow maximum flexibility for including comments, measurements, and other related data to be stored and associated with the collection of objects. A very useful example of a general annotation is a simple collection of objects, much like a shopping cart, that can be stored, organized, and labeled for later use. Image: As a phylogenetic database, images are vitally important to the users of the system. Therefore, many of the annotation types described in this section will apply specifically to images. The types of image annotations are listed as:
Spot location on an image associated with the annotation. The user will identify a specific spot on the image to associate with a label, title, and paragraph description.
Circle associated with an area on the image.a The user will place a circle encapsulating an area to associate with a label, title, and paragraph description.
Rectangle associated with an area on image. The user will place a rectangle encapsulating an area to associate with a label, title, and paragraph description. Taxon Determination: Used for discussion concerning the species or other taxonomic determination of a specimen. Users will select a specimen and by using the associated images, make a recommendation as to the specific genus and species determination. Taxon determinations are extremely important to the research activities of the primary users.
Phylogenetic Character and State: This type of annotation will be used to organize physical features (called ―characters‖) of organisms into objects of interest to research users. Phylogenetic characters and possible values (states) of those characters are associated with specific images, with species, and with collections of species. In this type of annotation, the user will associate an image or specimen in the database with phylogenetic characters and states.
Relationship: Morphbank comes standard with predefined data relationships. Relationship annotations allow the user to define additional relationships associating Morphbank objects with each other. User will select any two Morphbank objects (image, specimen, view, location, publication, user, group, etc) and then describe the relationship among the two.
Specimen image annotation captures people’s knowledge of species such as new observations, and disagreements with previous annotations. Image annotation enables semantic image retrieval and maintains a record of user comments concerning the data. Furthermore, a collection of featured annotations provides a way to assign species to a specimen. Image annotation associates textual information to the specific region of an image to enable semantic querying.
Two technologies are frequently used: Text-based approach and field-based approach. The former simply add keywords to the whole image using natural language. However, keyword-based retrieval returns irrelevant documents (i.e., low accuracy of retrieval). A field-based method describes and retrieves an item using one or more field-value pairs, thus improves the retrieval precision. Figure 2 shows an image annotation of the field-based approach. This annotations asserts that a particular portion of an image (of a wasp leg) is a femur. However, both text-based and field-based approaches store the information in a plain text format. It is known that querying the plain text is inefficient. Furthermore, storing annotation information using only plain text is not suitable to satisfy the higher level requirements for the system. Meaning and ontology must be associated with the data. The heterogeneous data models from different biologists and the diversity of association types require frequent update and evolving data structures.
The annotation of Fig. 3 asserts that the wasp whose leg is shown has a particular feature, which is called ―femur swollen medially‖. Such features are used by experts to categorize specimens into taxonomic units (genus, species, etc.) and, after analysis, to develop evolutionary models.
Morphbank is using annotation and association technology to collect information that is directly used in scientific research. Each of the Morphbank objects related to the annotation of Figure 3—the image, the annotations, the related specimen, etc.—are represented as first-class objects with globally-unique identity. Thus the objects can be stored in collections, included in other annotations, and referenced in external sites. Mass annotations are possible as well. Figure 4 shows an interface that allows a user to annotate each of a group of objects. In this case, the user is preparing to comment on the species identification, also called the determination of several botanical specimens. This annotation interface has been developed to enable a specific activity to be performed by experts on plant morphology. Creating the determination annotation sheet began with interviews with domain experts and the evaluation of typical manual records. Figure 6 shows a detail of the herbarium sheet of Figure 5 that contains the information cards that are attached to the sheet. Two cards are attached. The lower card is the primary information about the specimen including who collected it, when and where. The lower card also shows the species determination that was recorded when the specimen was collected.
The Morphbank research team has been working closely with a group of botanists at the Department of Biological Sciences at Florida State University to use the annotation tool for the curation of specimens from the Robert K. Godfrey Herbarium at Florida State University. Figure 5 shows some of the Morphbank information for a typical herbarium sheet. The upper card shows a determination annotation that was added to the specimen in 1983. J. Farmer of the University of North Carolina agreed that the determination was correct.
In pencil, between the two cards is second annotation. D. D. Ward in 1983 also agreed on the correctness of the determination. The Morphbank annotation tool is intended to allow the online collection and dissemination of information like that shown in
A major benefit of the Web tools is its support for distributed collaboration. Before the sheets were The annotation interface shown in Fig. 4 can be used to agree with the recorded determination of the set of specimens, or to disagree and select a different taxon. In this way the annotation represents a qualitative evaluation of the recorded information. Fig 4 shows that 19 annotations already record agreement (A) with the determination.
The results so far are very promising. Fifteen taxonomists were asked to use Morphbank images of specimens from the Robert K. Godfrey Herbarium at Florida State University to make digital determination annotations for 50 specimens each. The scientists found the online tools to be an excellent replacement for the manual task. They were particularly pleased to be able to see the results online and to be able to see the effects of this online collaboration.
An additional study of the feasibility of making determinations from images in lieu of physical specimens was conducted by bringing some of these experts to Florida. The study is ongoing. We hope to be able to establish that digital representations of these specimens are more than adequate replacements for the real objects.
We have described an existing need in the biological community to store and retrieve complex information on specimen and related images. In creating a Web site that stores the elements common to all entities in the Tree of Life, we have made biodiversity research more effective.
Our work in developing a tool that allows users to annotate images via the Web using only the essential elements has proven successful. The non-intrusive method permits biologists to mark images without altering the original image, and share this annotations with others in an easy and open format. Our hope is that the work performed under this NSF grant by the Morphbank project will provide the Tree-of-Life initiative with a stable digital image database and annotation tool set that can be used by biologists around the world.
[1]
L. Alexander, A. Runyan, and V. Anderson. Taxonomic data working group, Darwin Core 2. TDWG.org integration. In Workshop on Knowledge Markup and Semantic Annotation, KCAP03, 2003.