In this paper, we describe a tool for rich annotation of images. This tool allows creation of graphical annotations to identify and describe features in images. It also provides the capability to then catalog the annotation layer in metadata repositories. Application of the annotation tool to describe satellite images in the areas of meteorology and vegetation will be presented.
The amount of image-based content has exploded on the web. However, there is a lack of effective tools to describe, enhance, catalog, search and retrieve images. This leads to poor use of image-based resources in general and for teaching in particular. The issues are: a) Limited metadata to describe and find high quality and "content rich" images b) No methods for teachers and students to extend or customize an image, c) No tools to interact with, inquire about, experiment with and discuss content in images, and d) No "visual" methods to collaborate on or discuss an image.
In a typical image, the image as a whole is usually not interesting; the features contained in the image are of primary interest. Features inside an image are the true resources that need to be described, cataloged, searched and retrieved. We adopt this approach.
Rich annotation (RA) is a combination of a) graphical drawing or marking to identify the location and geometry of a feature in an image and b) attributes or properties to describe the annotated feature. RA is created in a transparent layer placed on top of the image using Scalable Vector Graphics (SVG, a World Wide Web Consortium's standard for 2-D graphics on the web). The transparent layer is called an annotation layer, and it can contain several annotated features. We have developed Rich Annotation Service for Images (RASI), a web-based tool to create rich annotations of images. Rich annotation is a fusion of a graphical marker (to identify a feature) and a structured object (to describe the feature) that contains attribute names and attribute values, this is the reason for calling it a "rich" annotation.
RASI is a web-based application for creating rich annotations of images, it may be accessed at http://www.indent.org/rasi.htm. The drawing capabilities in the browser are programmed in Scalable Vector Graphics (SVG, a World Wide Web Consortium standard for 2-D graphics on the web). See Figure 1 for a screen shot of RASI.
RASI contains an annotation canvas, an annotation toolbar and section to manage properties. In the annotation toolbar, a palette of annotation markers is available to draw freeform or place icons in the annotation canvas. When a feature is annotated in an image using an annotation marker, a rich annotation is created. This rich annotation object contains the geometry of the annotation marker and a set of properties. See Figure 2 for relationship between the various entities in RASI.
As an example consider an Infrared (IR) image of a storm with features like cold front, warm front, occluded front and low pressure point. These features are annotated with appropriate annotation markers. Cold front, warm front and occluded front feature have a property called "precipitation," and low pressure point has properties: surface pressure, surface temperature, precipitation, and cloud top temperature. The annotation markers used for the features are: cold front is drawn using the blue line with triangles, warm front with the red line with semicircle, and low pressure point is an icon with letter L in the center.
The RASI tool is customizable to other domains-user with appropriate authorization can create a new problem domain, define new features for the domain, and create annotation marker and annotation properties for the new features.
The process of creating annotations is the following, user: picks an annotation tool based on the feature they want to annotate; creates an annotation to identify the feature; chooses the created annotation and assigns values to the properties associated with the feature; add more properties if required; create more features on the same annotation layer; saves the annotation layer; creates metadata for the annotation layer and submits to DLESE or NASA-GCMD.
The three components of its technical architecture are (see Figure 3): i) User-Agent, which includes HTML, Javascript, SVG and XML files that are delivered to the browser or to a metadata repository; ii) Middle-tier, which is ASPIRE (http://www.indent.org/aspire.htm, a Java-based Commercial Off The Shelf product of Innovative Decision Technologies, inc.) and is used to process SVG-XML data from browser and store into database, extract relational data from database and transform into SVG-XML, and metadata repository specific XML formats; iii) Database. The middle-tier generates the SVG, HTML and XML files based on templates that contain Aspire specific replacement, loop and if tags. RASI also contains a search facility that allows users to search for images based on advanced queries like:
RASI has been built for creating rich annotations for images. Several projects have been created in areas of meteorology, vegetation and others to allow users to capture knowledge in a graphical manner on the image. In addition, this knowledge is used to generate metadata for images
We thank NASA for partially funding the development of RASI; NASA-GCMD and DLESE for cataloging the metadata records from RASI.