In recent years, researchers have argued that using ontologies to represent and drive knowledge infrastructure of software projects provides separation of the declarative and procedural knowledge and enables easier evolution of the declarative knowledge. We have validated these conjectures in the context of BioPortal, a repository of biomedical ontologies, which was developed in our group. We are using the BioPortal Metadata Ontology to represent details about all the ontologies in the repository, including internal system information and the information that we collect from the community such as mappings between classes in different ontologies, ontology reviews, and so on. To the best of our knowledge, BioPortal is the first large-scale application that uses ontologies to represent essentially all of its internal infrastructure. The BioPortal Metadata Ontology extends several other ontologies for representing metadata, such as the Ontology Metadata Vocabulary and the Prote´ge´ Changes and Annotations Ontology. In this paper, we show that it is feasible to describe the structure of the data that drives an application using ontologies rather than database schemas, which are used traditionally to store the infrastructure data. We also show that such approach provides critical advantages in terms of flexibility and adaptability of the tool itself. We demonstrate the extensibility of the approach by enabling representation of views on ontologies and their corresponding metadata in the same framework.
database schema. In this paper, we describe an approach to application development that pushes this envelope to use ontologies and ontology instances to represent essentially all data that the application requires in a single flexible framework, from declarative high-level descriptions of the data as in the examples above to internal system data.
For large-scale distributed architectures today, the development stack includes
several technologies wrapped around a SQL database schema, such as persistence
managers (e.g., Hibernate), a web server, and so on. When the database schema changes,
these changes often need to be propagated through the development stack, thus making
such changes expensive in distributed web-based applications. We have encountered
this problem in developing BioPortal1—a community-based repository of biomedical
ontologies, containing 170 ontologies with more than one million classes among them
at the time of this writing. Users can submit their ontologies to BioPortal; search across
all ontologies; browse the ontologies, their different versions, and the associated
descriptions and provenance information; describe their ontology-related projects and link
the descriptions to the ontologies; leave comments on classes and on ontologies; create
mappings between concepts in one ontology and concepts in another ontology [
The BioPortal application is heavily knowledge-driven: most of what the users see when browsing BioPortal (in addition to the ontologies themselves), is some rendering of information that would traditionally be in a database. This internal information that drives the application includes the metadata about ontologies in the repository, such as ontology domain, authors, and other provenance information, as well as information on which property to use for preferred name and synonyms in each ontology, information on where the ontology itself resides in the system (e.g., the specific database table), when it was uploaded, the name of the administrator of the ontology in BioPortal, and so on. Some of this information (such as provenance) is intrinsic to the ontology artifact and is relevant outside of BioPortal; some information is internal system information.
Because the BioPortal application is novel in many of its aspects, our internal infrastructure continues to evolve constantly, as we understand better user requirements, learn what works and what does not, get new collaborators that would like to extend BioPortal in a certain way. With the knowledge infrastructure constantly in flux, we found that describing and representing the structure of the knowledge as a relational database schema did not provide the flexibility and quick adaptability that our users required. Making changes was cumbersome and put a bottleneck in the development of the software code. It also made it much harder for anyone to adapt the BioPortal code for their own purposes as the developers had to be familiar with the entire development stack (including Prote´ge´, Java, Spring, Hibernate, and Ruby-on-Rails).
Thus, we decided to “eat our own dog food:” we developed an ontology to describe this infrastructure and represented the application data itself as ontology instances. Thus the whole BioPortal application is driven by ontologies and ontology instances. Note that while BioPortal is a repository of ontologies, the infrastructure that we describe is not specific to the artifacts represented in a repository. It will work for a repository of any other artifacts, not necessarily ontologies. To the best of our knowledge, BioPortal is the first large-scale application of this approach.
This paper makes the following contributions: 1 http://bioportal.bioontology.org – We developed an ontology to represent the infrastructure and run-time data of a large community-based ontology repository. – We implemented the infrastructure of BioPortal using an ontology to represent most of the data required to drive the application.2 – We validated the extensibility of the approach by adding functionality to support flexibly representation of ontology views. 2
The BioPortal ontology repository is an active ontology repository with a large user
community that contributes its content and uses its web services in their applications.
In addition to more than 170 ontologies, it currently contains multiple versions of these
ontologies, submitted by their authors and almost one million mappings between
concepts in the ontologies. There are descriptions of ontology-based projects, and notes
and discussions on classes and ontologies. The BioPortal Resource index provides
ontology-based access to several biomedical data sets available online (e.g. entries in
GEO, ClinicalTrials.gov). All BioPortal functionality is supported by a rich metadata
infrastructure, which includes the following types of metadata:
– ontology metadata describing the ontologies and their provenance and includes
ontology name, domain, description and keywords, authors, license information,
versions, references, and metrics such as the number of classes and properties;
– mappings between concepts, and metadata associated with mappings, such as how
the mapping was created, whether it was created manually or computed
automatically by a particular algorithm (and which one) and context for the mapping [
The BioPortal application dictates the following functional and architectural requirements for the metadata support: Efficient and scalable support of BioPortal main functions: Any metadata infrastructure must support fast access to metadata, flexible querying of specific metadata items and their combination, and be scalable. We envision that the number of users, notes, projects, and mappings will grow significantly in the coming months. 2 At the time of this writing, some data, such as mappings and user information, is still in database tables from our initial implementation of BioPortal.
Support for ontology versioning: Users can upload successive versions of their
ontologies and explore any ontology version. There must be services that always resolve to the
latest version of an ontology, with each ontology having a “virtual” location that always
redirects to the latest version. Metadata referring to an ontology or its components (e.g.,
reviews, notes, mappings) must be attached to a specific version of an ontology.
Flexible evolution of the metadata schema: One of the key requirements for metadata
support is its ability to adapt easily to new requirements and types of metadata. The
types of metadata that an ontology repository requires is still an active area of research.
Thus, the structure of the metadata and the specific properties change frequently. These
changes to the schema describing the metadata must be easy to implement and roll out.
Customizability of the metadata schema: Some groups install their own versions of
BioPortal software to support either a broader scope than just biomedicine (e.g., the
Open Ontology Repository sandbox3) or to maintain a repository open only to a
specific set of users (e.g., the Marine Metadata Initiative4). Developers that maintain these
BioPortal installations usually customize the code to satisfy the local requirements. For
example, the fields that describe an ontology and its provenance are different for
different communities. Definitions of mappings and the associated metadata differ as well.
The representation of metadata schema must make it easy for these developers to
custom tailor what gets represented and what gets presented in the user interface.
Reuse of existing technologies and ontologies: Wherever possible, we would like to use
existing technologies and standards for representing metadata. For example, the
Ontology Metadata Vocabulary (OMV) [
In our initial implementation of BioPortal we used a database schema to describe our metadata, with column names corresponding to metadata fields. This approach is fairly traditional for many large-scale implementations and supports the first two requirements in our list—efficient and scalable handling of metadata and support for ontology versioning. However, in our experience, this approach did not fare so well on other requirements.
Any time we needed to add a new metadata field, we had to change not only the database schema, but also the rest of the application stack (e.g., Hibernate) to reflect the 3 http://oor-01.cim3.net/home/release 4 http://mmisw.org/or/ change. These changes were time-consuming and cumbersome. Yet, the more we were developing BioPortal, the more features we were adding, the more often we needed to adjust the metadata representation. For instance, in addition to describing the ontologies themselves, we needed to add ontology views, to support ontology reviews along several different dimensions, and to represent a large set of ontology metrics—all of these requirements crystalized after the start of the development.
With the metadata schema encoded as a database schema, any customization of new BioPortal installation requires changes to the schema as well. And, as we mentioned earlier, this process is cumbersome and error-prone.
Reusing and sharing the metadata that we collect also was not straightforward: it is hard to find two applications that use the same database schema. Thus, in order to transform the metadata from our internal representation to the representation that another repository (e.g., Oyster) uses or to represent comments in a format that a tool such as Prote´ge´ would understand, we must write a script to export the data.
While none of these challenges are insurmountable, we decided to apply a completely different approach to representing metadata infrastructure in BioPortal to address these requirements. As we show in the remaining sections, this approach satisfied our requirements and proved to be extensible enough to support new requirements. 4
Figure 1 shows the architecture of BioPortal. It is a traditional service-oriented layered architecture, with the front end (Ruby-on-Rails) accessing the backend information through RESTful services.5 There are services to access ontology information (e.g., get information about a specific ontology, upload a new version, get a diff between two versions), concept-level services (e.g., get class definition), hierarchy services (e.g., get all subclasses of a class), search services (e.g., search for a term across all ontologies), and other services. The business logic tier implements these services in Java, using the Spring framework. This layer is the one that contains the logic to translate the internal metadata representation into responses to service request (e.g., a service may request a list of all versions for a specific ontology). Before we transitioned to the ontology-based approach, the database schema of the underlying relational database (mySQL) was reflected directly in the implementation of this layer and its metadata functions. In our current approach, the metadata structure is accessed through the Prote´ge´ ontology API. This API, in turn, does use a database to store the ontology and the instances. However, the schema of the database that the Prote´ge´ uses does not depend on the ontology itself. It is a single table that stores both the ontology and the instance information.6
The types of metadata and their properties are describe in the BioPortal Metadata Ontology.7 The metadata values are Prote´ge´ instances and property values (see Figure 2 for an example). We use the Prote´ge´ API to access the ontology and instances. 5 http://bioontology.org/wiki/index.php/BioPortal_REST_services 6 http://protege.cim3.net/cgi-bin/wiki.pl?JdbcDatabaseBackend 7 http://bioportal.bioontology.org/virtual/1148 NCBO BioPortal
Ruby on Rails
JSF .NET
PHP
FLEX RESTful Services (HTTP)
Acegi Security Framework Spring Framework
Protégé Hibernate ORM The BioPortal Metadata Ontology is an OWL ontology that imports a number of other ontologies (Figure 3) and includes classes to describe an ontology itself, its versions, information about the ontology, creators of an ontology, user-contributed content, such as notes, reviews, and mappings. It also contains the system information that is relevant for maintaining and representing the ontology in BioPortal, such as which users administer the ontology in BioPortal, where the ontology itself is located in the BioPortal system, internal ontology id and version ids, and so on. The instances of classes in this ontology represent the actual metadata for the BioPortal content. The BioPortal Metadata Ontologyis an OWL-Lite ontology, specifically, RDF Schema constructs, plus owl:import, thus supports in a scalable manner any reasoning that BioPortal requires (for example for transitivity in getting superclasses or subclasses of a class).
The BioPortal Metadata Ontology imports several ontologies that deal with the types of metadata that BioPortal supports: – The Ontology Metadata Vocabulary (OMV) describes most of the metadata for ontologies themselves (e.g., domain, author, version, ontology language, etc.) – The Prote´ge´ Changes and Annotations Ontology (CHAO) provides definitions for generic annotations and ontology components that they annotate.
rdf:type rdf:type rdf:type rdf:type FMA, v.2.0 reviewOn hasVersion Review_1209
Instances
FMA, v.1.0 hasVersion FMA – The Prote´ge´ Mapping Ontology provides vocabulary for describing one-to-one mappings between concepts and corresponding metadata.
The OMV provides the vocabulary for describing a specific ontology version. An instance of the class OMV:Ontology describes a single version of an ontology. This class contains properties describing pertinent information about the ontology. The BioPortal Metadata Ontology extends this class to add properties that are specific to BioPortal as well as some missing properties that should have been in OMV.8 These properties include system information such as the internal id, the user who submitted the ontology, the internal status (e.g., scheduled for parsing, loaded, error), and associated reviews.
We use the instances of the Prote´ge´ CHAO ontology to represent comments that BioPortal users contribute to the ontologies. Each comment is represented as an annotation attached to a specific class (in a specific ontology version) or to another annotation (if it is a response to a comment). Users can use the comments, for example, to carry out discussions about modeling decisions, make suggestions for changes, ask questions. The same mechanism exists in Collaborative Prote´ge´, a version of the Prote´ge´ ontology editor that supports collaborative ontology editing. Because BioPortal and Prote´ge´ share the same structure for representing user comments and discussions, one can potentially 8 We collaborate with OMV developers to include these properties in future versions of OMV FMA
GO
NCIT Represents an ontology version
OMV
Class
OMV:Ontology versions
views owl:imports
Class VirtualOntology
BioPortal OWL ontologies Represents comments, reviews
References a specific
version of a class or property
References an ontology verison Protégé Changes ontology
changes:annotates
Class changes:Annotation
Class changes:Ontology
Class changes:Ontology_Component subclasOf
owl:imports BioPortal Metadata Ontology
Class Review
Class User
Represents a mapping
Represents mapping metadata Mappings ontology
Class mappings:One_to_one_mapping
Class mappings:Mapping_Metadata owl:imports
Class Project open a BioPortal ontology in Prote´ge´ and see the comments contributed by BioPortal users. We are currently working on the tighter integration of the two tools.
The BioPortal Metadata Ontology adds classes that are specific to BioPortal functionalities and that are not described in the imported ontologies. These classes represent ontology projects and reviews on ontologies and dimensions that users can use to evaluate ontologies.
The BioPortal Metadata Ontology introduces some convenience classes that abstract the information already present in other classes. This approach poses minor maintenance challenges, which we mitigate by changing instances only programmatically. However, this approach greatly facilitates access to the information. Consider the following example: In OMV, the description of an ontology version (instance of OMV:Ontology) points to a previous version of the same ontology. Thus, in order to present a table with the information about all versions of an ontology (as in Figure 4), we must first find the latest version of the ontology, and then traverse the instances to collect all versions. Thus, we introduce the notion of VirtualOntology which is an object representing the collection of all versions of the ontology. This object has the minimal information that is shared among all versions, such as the ontology name (e.g., Gene Ontology); the “virtual ontology id”—the global id that, when used to access an ontology, always resolves to the latest version; the information on who administers the ontology and the list of versions (instances of OMV:Ontology).
We have validated the feasibility of our approach by implementing it as an infrastructure for BioPortal.9 The previous version of BioPortal used a database schema to reflect the metadata schema and posed exactly the flexibility and customizability challenges that we described in Section 3.
First, we replaced the databases in the storage layer with a Prote´ge´ ontology, implemented in its own one-table ontology-independent schema (Figure 1). Second, we replaced the metadata implementation in the business-logic layer with the appropriate Prote´ge´ API calls. Third, we transferred the metadata from the old database to instances in the metadata ontology. Our goal was to maintain the same API at the service layer so 9 Note to reviewers: At the time of the paper submission, the main BioPortal server at http:// bioportal.bioontology.org runs using the database-based metadata representation. We use the new infrastructure in our development server that is not yet accessible to the public. We expect to transition the new implementation to our production server before the end of the Summer 2009. that the user interface does not need to be modified and other applications that already use our REST service API can continue to use it.
Since we access the metadata ontology through the Prote´ge´ API —which was successfully used in other projects to access in a scalable manner ontologies having more than 8000 classes and 5 million instances 10— we predict with confidence that the new representation of the metadata about ontologies will scale well. The version of BioPortal currently running on our development server uses this infrastructure successfully, thus validating the approach. We plan to release this implementation to production at end of August 2009.
At the time of this writing, the transition of metadata to ontology-based approach is not complete. Currently, ontology details, ontology versions, and ontology views (see Section 7) are represented as ontology instances. We are in the process of transitioning the rest of the metadata. 7
As we discussed earlier, one of our main motivations to moving to the ontology-based approach was greater flexibility and adaptability of the metadata. We validated these properties by implementing support for ontology views in BioPortal, which did not previously exist. In our implementation we store only materialized views computed and submitted by the users, together with the metadata describing how the view was generated (the language, engine, etc.)
In this context, a view is any subset of an ontology that is itself an ontology. A view
can be created manually or automatically by a view-generation tool. For instance, our
collaborators have several views of the Foundational Model of Anatomy (FMA) [
Requirements for Representing Views Discussions with our collaborators led to the following set of requirements on view representation: – Each view is itself an ontology and can have metadata, be explorable, have reviews, statistics, and so on.
– A view is defined on a specific version of an ontology. 10 http://protegewiki.stanford.edu/index.php/Scalability_and_ Tuning – There is a notion of a “virtual view” (cf. “virtual ontology”) such as a view of Liver-related concepts in FMA created for a particular purpose.
Each virtual view will have at least one version, but can have several.
Each version of the view also has its own metadata (inherited from ontology metadata, but with additional fields). – We must be able to represent views on views (with the same requirements). – We must be able to represent views that use more than one ontology.
Note that these requirements suggest a fairly complex structure for view representation, with many cross-references with different components of ontology metadata (ontologies and ontology versions). 7.2
Representing Views in BioPortal When representing views in BioPortal, we treated them essentially in the same way as regular ontologies, thus getting the browsing, annotation, and other features “for free.” We extended the BioPortal Metadata Ontology to represent features that are specific to describing views (Figure 5). A class VirtualView is a subclass of VirtualOntology and points to the list of versions of the view. Each version of a view is an instance of the class OntologyView, which is a subclass of OMV:Ontology. Thus, it inherits all the properties that describe an ontology version (e.g., description, domain, author) and adds its own. The view-specific properties include the following: – The property viewDefinition is the textual representation of the view definition. This definition can be a query that was used to create the view, a set of traversal directive (as in Prompt), or any other way that specifies how the view was extracted. – The property viewDefinitionLanguage defines the language that was used to define the view (e.g., the query language, such as SPARQL, for a view that was generated by a query). The range of the property is the class ViewDefinitionLanguage, whose instances will have all the pertinent attributes of the language (name, creator, url) as well as the specific version that was used for the view. – The property viewGenerationEngine contains the engine that was used to compute the view. The range of the property is the class ViewGenerationEngine, whose instances will have all the pertinent attribute of the engine (name, creator, url) as well as the specific version that was used to generate the view. 7.3
Providing Support for Views in BioPortal In this ontology-based infrastructure for representing metadata, in order to handle views to BioPortal and satisfying all the requirements that we have outlined earlier, we had to do the following. First, we extended the BioPortal Metadata Ontology as we described in Section 7.2 and Figure 5. After this step, we already had all the structure to represent the views. Second, we implemented the new view-specific REST services (e.g., returning all views for an ontology). This implementation uses the Prote´ge´ API to access the view metadata. As we expected, in this implementation we indeed needed to
Class VirtualOntology hasVersion isVersionOf
FMA Instance of
VirtualOntology virtualViewOf
isVersionOf NeuroFMA Instance of VirtualView isVersionOf isVersionOf isVersionOf FMA, v.1.0 Instance of OMV:Ontology isViewOn
FMA, v.2.0 Instance of OMV:Ontology isViewOn NeuroFMA, v.1.0
Instance of
OntologyView focus exclusively on the view-specific logic and structures and did not need to change anything else (e.g., the application infrastructure or the architecture). Some of the functional requirements for handling the views crystallized while we were already working on the implementation, and the only changes that were necessary were the changes to the ontology itself. We did not need to change any database schemas or to add any additional structures (e.g., Hibernate stubs). The work that was required was limited to implementing the logic for the services directly.
In comparison, adding views to the old (Hibernate-based) infrastructure, would have involved some additional changes like: (a) creating new database tables (for views and virtual views), (b) creating a number of foreign keys in order to represent superclasssubclass relationships and relationships expressed by object properties, (c) creating integrity constraints to enforce data integrity along those relations, and (d) generate Hibernate classes (so called “entity beans”) from the database tables.
Figure 6 shows the extended ontology-metadata page that includes the information about the views. 8
The new implementation of BioPortal infrastructure validates the use of ontologies and ontology instances to represent all the metadata as well as system data for a repository of this kind. To the best of our knowledge, BioPortal is the first large-scale web-based application that uses ontologies to represent its internal data. Our experience has shown that using an ontology to describe the knowledge infrastructure of an application does indeed provide the flexibility and adaptability that our application required. Our observations and discussions with developers show that it was easier and far more efficient to implement view support that satisfied the requirements that we outlined in Section 7.1 using this approach than using a database schema to describe the views and to link them to ontologies. It also enabled us to reuse much of the infrastructure that we have already developed for ontology metadata, as we could treat views as special cases of ontologies. As our requirements for view representation continue to evolve, we know that our changes will be easy to implement as we will need only to evolve the ontology representation and the corresponding service implementations.
It is important to note that some of the design choices in the BioPortal Metadata Ontology are driven purely by application and implementation considerations. Thus, there are parts of this ontology that are specific to the context of BioPortal implementation. We also made some of the ontology-design choices not because they were “ontologically correct” in an abstract sense but because they simplified access to information (cf. VirtualOntology class to collect information about all versions of an ontology). Thus, we used the ontology not only as a conceptualization of our domain (metadata representation) but also to represent the physical properties and location of the data.
There are a number of systems that use ontologies to represent some of their
metadata. These include Oyster [
Another class of applications that uses ontologies actively are semantic desktops
(e.g., [
In our work, we have extended other metadata ontologies. In the future, we plan to integrate other standards. Specifically, we plan to replace the Prote´ge´ Mapping ontology that we used here for expediency with an ontology that extends SKOS with mappingspecific metadata.
Our current effort leaves some questions and concerns, however. The main concern is scalability. While Prote´ge´ is quite scalable and has been used with ontologies that have hundreds of thousands of classes and instances, we do not know how it will behave with millions of instances describing metadata. As we noted in Section 6, we have not yet moved mappings to this infrastructure. At the same time, we have just uploaded one million new mappings to BioPortal. We are yet to test whether our current Prote´ge´-based infrastructure will be sufficiently scalable for this number of mappings. If we learn that it is not, it will be the limitation of the Prote´ge´ implementation itself as modern triplestore implementations easily handle this amount of data. If scalability turns out to be an issue, we will transition to a triplestore to store the instances.
We would like to emphasize that our solution is not limited to the biomedical domain. It so happens that our repository is a repository of biomedical ontologies. However, there is nothing in the BioPortal Metadata Ontology itself or in our use of it that is specific to biomedicine (except perhaps, the list of possible ontology categories). We will install the new infrastructure in other local BioPortal installations, such as the OOR sandbox that we mentioned earlier (and that accepts ontologies in any domain).
Finally, the BioPortal software is open-source. The software is domain-independent and can be used for an ontology repository in any domain or for a domain-independent one. The BioPortal Metadata Ontology is available in BioPortal and can be accessed through the BioPortal user interface or its web services.11 We have created a snapshot of the instances of the BioPortal Metadata Ontology for our development version of BioPortal. This ontology and instances can be accessed directly through the WebProte´ge´ server at http://bmir-protege-dev1.stanford.edu/webprotege/. 12
This work was supported by the National Center for Biomedical Ontology, under roadmapinitiative grant U54 HG004028 from the National Institutes of Health and by NIH grant HL087706. We are grateful to Jim Brinkeley, Todd Detwiller, Onard Mejino, and other members of the Structural Informatics Group at Univesity of Washington for helping us identify requirements for views in BioPortal. 11 http://bioportal.bioontology.org 12 Please use the Firefox browser to access the server. Select “BioPortal Metadata Ontology” and then go to the “Individuals” tab to view the instances.