Visual representations are crucial for communicating biomedical knowledge, particularly in the domain of anatomy. Methods for integrating standardized visual representations into knowledge representation schemes are largely unexplored, leaving users dependent upon textual descriptions or ad-hoc visuals. In this work we introduce SVG-based graphics libraries as a tool for standardizing visual representations of biomedical knowledge. We first present design requirements for graphics libraries and a scheme for implementing web-accessible libraries. Finally, we introduce a prototype application for managing and sharing a graphics library. This work uses a library for orofacial anatomy as a test case, but is applicable to any domain of biomedical knowledge that can benefit from visual representation.
Ontology, terminology, knowledge representation A shared understanding of anatomy is crucial for progress in both clinical research and basic research with model organisms. Terminologies and ontologies play important roles in standardizing the representation of anatomy, but text-based representations of anatomy may obscure differences in how people understand and apply anatomical terms (1). Without a mechanism for rapidly establishing a shared understanding of anatomy, the efficiency and precision of communication among researchers is impeded. The precision with which terms are used is particularly important in community databases and applications that integrate data from multiple labs—because once data is tagged with an anatomical term, any differences in how terms are used across research groups are obscured. Therefore, there is a need for visual representations of anatomy that can be used as standards which are referenced by text-based representations of anatomical knowledge.
The purposes of this work are (1) to introduce graphic libraries as a tool for knowledge representation, and (2) to describe implementation of a prototype web application for managing and providing access to a graphic library of orofacial anatomy.
Laboratory images and illustrated graphics have a long history in
the communication of anatomical knowledge. Several online
anatomy atlases have paired laboratory images with terms from
ontologies. The Zebrafish Atlas (2) provides a labeled
histological section for each term, and the e-Mouse Atlas (
Graphics libraries have been effectively employed for public communication purposes. Two examples of graphic libraries are shown in Figure 1. Well-designed graphics libraries benefit both the individuals using the library (by providing readymade graphic assets) and the viewers (who need to learn the meaning of the graphics only once).
Figure 1– Examples of graphics libraries. (Top) International transportation symbols created in the 1970s through a partnership between the American Institute of Graphic Arts and the U.S. Department of Transportation. (Bottom) Map and recreation symbols from the U.S. National Park Service.
But unlike the stylized, iconic graphics that help us navigate through public spaces, representation of anatomical structures requires a different style—one that preserves spatial properties of anatomical structures. This challenge of constructing standardized graphics for anatomy is rarely undertaken. Anatomical graphics are commonly designed to accompany anatomical atlases and textbooks, but seldom as a library of visual assets where individual graphics are intended to be reproduced within another context.
Anatomy and malformations of the human mouth and face
Adult human anatomy with a focus on organs and regions of the body
Development of the human head and face
Palate Complete
bilateral cleft palate Bone Stage 12 (early) Stage 12 (late)
Stage 13 (late)
Stage 14 Stage 15 Stage 16 Stage 17 Stage 18
Four characteristics will facilitate use of graphics libraries as standards in knowledge representation and allow the graphics to be customized for use in biomedical informatics applications: • SVG: Scalable Vector Graphics (SVG) is an XML-based standard developed by the World Wide Web Consortium (W3C) (9). Vector graphics are simple to create, have small file sizes, and are scalable to any size. In addition, the SVG format allows application developers to use CSS to control the appearance of elements in web applications, as well as incorporate interactivity through JavaScript or CSS. • Composable: By designing graphics as compositions of graphic elements, the flexibility of a library is maximized while minimizing the number of elements to be produced. See Figure 3 for an example of composing graphics from a common set of lines. • Extensible: Just as additions are made to terminologies and ontologies based on the needs of users, graphic libraries will need additional graphics. One way to address this need is for the author of a library to add additional graphics. But because SVGs can be created and manipulated using readily available software tools and only minimal training, researchers and application developers can customize 155 lines 59 phenotype graphics canonical anatomy midline clefts incomplete bilateral clefts malformations of lower lip left unilateral clefts right unilateral clefts complete bilateral clefts existing graphics (for example, adjusting the anchor points of graphics to create new morphologies) or develop entirely new graphics for their specific applications. • Identifiable: In order to serve as standards for knowledge representation, the graphics should have unique identifiers. This will allow authors of terminologies and ontologies to map to the graphics that serve as visual representations of terms and classes.
To implement the design requirements for graphic libraries in a standardized and web-assessable format, we have developed the approach shown in Figure 4. The three types of visual components of a graphics library (Figure 4, part A) are: • Graphical element: An SVG element of type <path>, <line>, <ellipse>, <polygon>, <polyline>, or <rect> • Graphical unit: Two or more graphic elements or graphical units • Graphic: A complete “ready-to-use” representation composed of two or more graphical elements or graphical units.
A B
Graphical
unit composed of
Class in ontology Term in terminology
Graphical element Graphic
mapped to Graphic library
DOI Figure 4– Scheme for implementing graphic libraries. (A) Graphics are composed of graphical units and graphical elements. Authors of terminologies and ontologies map to graphics. (B) A graphic library is identified by a DOI. Each graphic, graphical unit, and graphical element is identified by an IRI. Tags to aid search and retrieval within the web interface may be applied to graphics, graphical units, and graphical elements. Classes are applied to graphical elements to enable styling with CSS.
Four types of identifiers and tags are used with this approach (Figure 4, part B): • DOIs: A graphics library will be identified by a DOI. Minor updates will be tracked with version numbers. Major updates will be assigned a new DOI. • IRIs: Each graphic, graphical unit, and graphical element will receive a unique IRI. An example of an IRI for a graphic is “http://endlessforms.info/graphics/ofac0207”. The “ofac” prefix of identifier indicates it belongs to the Orofacial Graphics Library. • Tags: Text tags can be applied to graphics, graphical units, and graphical elements to aid in retrieval through the web interface. We anticipate tags will be useful for classifying into categories such as type of tissue (bone or mucosa of the palate) or anatomical regions of a graphic (lips, nose). • Classes: Classes are applied to the SVG elements to control appearance with CSS. Graphical elements will be supplied with default styling, but this can be altered by web application developers (Figure 6).
Graphical unit
Graphical element
Figures 7 and 8 demonstrate how the use of anatomical graphics can clarify the meaning of anatomical concepts represented by classes of ontologies. Authors of terminologies and ontologies can create mapping between our graphics and their terms or classes in several ways: as a property of the class or term, by creating a mapping ontology, or within a CSV file or relational database.
Groove of philtrum proper
fma305940 Vermilion proper of philtrum fma305953 Vermilion of philtrum fma305942 Vermilion border region of philtrum fma313899 Right lateral crest of philtrum proper fma281713
Left lateral crest of philtrum proper fma281715 all relations are regional parts
Upper lip fma59817 Labial part of mouth fma59815
Philtrum fma59819
Philtrum proper
fma281717 Right side of upper lip fma224400
Right side of upper lip
proper fma281630 Left side of upper lip fma224402
Left side of upper lip
proper fma281632 Lower lip fma59818
Exporting the graphics and composition information Graphics were exported from Adobe Illustrator CS5 as an SVG file. During the export, Illustrator automatically assigned a descriptive identifier using the name given to each element (such as “nostrils_canonical”) based the label of each element. CSV files were used to define the parent/child relationships between a given graphic and its constituent graphical units and graphical elements based on the descriptive identifiers. Additional CSV files contained information about (a) CSS classes for graphical elements and (b) mappings to FMA IDs for selected graphics. Preparing the database and SVGs Each graphical element (path, line, circle, or ellipse) was extracted from the SVG file and assigned a library identifier B
C A
D E
F consisting of a prefix (“ofac”, an abbreviation for “orofacial”) followed by four digits. Additional identifiers were created for each graphical unit and graphic following this convention. Data were processed and loaded into a SQLite3 database. We used Ruby on Rails to implement the prototype web server running on a Ubuntu Linux 18.04 server.
Functionality When the web server receives a request for a graphic, the database is queried to determine which units and elements comprise that graphic. Elements are then assembled into an SVG.
The library identifier is used to create an IRI as the primary identifier for each graphic (such as “https://endlessforms.info/ofac/ofac0100”).
Our prototype application is available at https://endlessforms.info. Screen captures are shown in Figure 9.
Figure 9– Interface of the web application displaying the Orofacial Graphics Library. (A) Table of the graphics displaying the ID, label, and thumbnail are displayed for each graphic. (B and C) Examples of pages for graphics. Each graphical unit and element (path) of the composition is listed. (D) Table of the graphical units. (E) Example of a graphical unit. (F) Example of a graphical element.
The prototype functions as both as website for browsing the graphics and an API for retrieving the graphics. Users can browse the library as HTML pages using a conventional web browser that supports SVG-embedded graphics. SVG paths of the graphics serve as links to the webpages describing the paths. Developers can embed the SVG graphics into applications by referencing the graphics IRIs with a “.svg” extension, such as “https://endlessforms.info/ofac/ofac0207.svg”.
Because we assign IRIs to graphics, graphical units, and graphical elements, any part of the library can be referenced and retrieved using a standard identifier.
We are actively improving the web application prototype and expanding the graphics libraries. Our source code and files for the work described in this paper are available at https://gitlab.com/infodesign.science/graphics_libraries.
As introduced in this work, SVG-based graphic libraries offer a way for authors of ontologies and terminologies to better document the meaning of the classes and terms they create. By standardizing visual representations for domains such as anatomy, different terminologies and ontologies can map to a common visual reference.
Our libraries use simple 2D SVG graphics. 3D graphics are another popular method for depicting anatomy, and offer the advantage of greater realism. However, 2D SVGs offer a number of advantages: they are much simpler to create and edit, only relevant features are presented, file sizes are small, and they can serve as interactive components in application interfaces.
Future developments to our prototype will improve the production pipeline for entering graphics into the database, add capabilities for tag-based querying, and improve the design of the user interface. (Note: URLs and IRIs will change as we transition from the proof-of-concept described here into a more fully developed tool with stable IRIs. Updates to the application can be found at endlessforms.info.)
This work demonstrates an approach to developing and distributing standardized graphic libraries to aid in knowledge representation of physical and spatial entities. By augmenting text-based representations with visuals, authors of ontologies and terminologies can improve the the clarity and efficiency of communication with users.
We thank José (Onard) Mejino, Todd Detwiler, and James Brinkley of the Structural Informatics Group at the University of Washington for supporting M.C. during her doctoral and postdoctoral studies when this work was initiated. Additional thanks to Tim Cox and Carrie Heike for advising on phenotypes depicted within the Orofacial Graphics Library.
1.