<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta />
    <article-meta>
      <title-group>
        <article-title>Promotion of Ontological Comprehension: Exposing Terms and Metadata with Web 2.0</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Andrew Gibson</string-name>
          <email>andrew.p.gibson@manchester.ac.uk</email>
          <xref ref-type="aff" rid="aff0">0</xref>
          <xref ref-type="aff" rid="aff1">1</xref>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          <institution>General Terms Design, Human Factors</institution>
          ,
          <addr-line>Standardization, Languages</addr-line>
        </aff>
        <aff id="aff1">
          <label>1</label>
          <institution>University of Manchester School of Computer Science</institution>
          ,
          <addr-line>Kilburn Building, Oxford Road, Manchester</addr-line>
          ,
          <country country="UK">UK</country>
        </aff>
      </contrib-group>
      <pub-date>
        <year>2007</year>
      </pub-date>
      <abstract>
        <p>Knowledge artifacts that have been labeled as ontologies have many different qualities and intended outcomes. This is particularly true of bio-ontologies where high demand has led to a rapid growth in the number of these artifacts. Good communication between the human agents involved in the life cycle of ontologies is essential for the ontologist to encode the right knowledge in the ontology. Not only this, but it should be encoded such that subsequent retrieval of the knowledge from the ontology by any agent can be clear and precise. The ontologist can encode ontological statements, for interpretation by a computer agent, or meta-ontological statements, for interpretation by human agents. We consider how the current communication between agents and ontologies produces drawbacks that add to the considerable overheads associated with ontology development. We describe the processes of communication between human agents and ontologies as Ontology Comprehension. We then suggest how these processes could be augmented, particularly with the use of Web 2.0 ideas. By exposing and enhancing the social interactions involved in ontology comprehension, development overheads are potentially reduced and the prospect of ontology sharing and reuse is improved.</p>
      </abstract>
      <kwd-group>
        <kwd>eol&gt;Ontologies</kwd>
        <kwd>Semantic Comprehension</kwd>
      </kwd-group>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>Categories and Subject Descriptors</title>
      <p>I.2.4 [Artificial Intelligence]: Knowledge Representation
Formalisms and Methods – representations, representation
languages.</p>
      <p>Web,</p>
      <p>Web
2.0,</p>
      <p>OWL,</p>
      <sec id="sec-1-1">
        <title>Ontology</title>
      </sec>
    </sec>
    <sec id="sec-2">
      <title>1. INTRODUCTION</title>
      <p>The technologies of the Semantic Web [6] have been centrally
conceived, specified and designed with recommendations by the</p>
      <p>W3C1. This next generation Web promises to transform the
information Web into a machine computable utopia for
semantically described data and information. Despite the
development of the technologies, there is, however, only little
evidence of the materialization of the Semantic Web (or Webs).
Simple RDFS vocabularies such as Friend of a Friend have
provided small views on the potential of the Semantic Web [9].
Rich ontological views supported by reasoning have appeared in
applications [27, 30, 31], but less so in the Web itself, and when
they do, they often represent unconnected niche pockets of
interest.</p>
      <p>In contrast, Web 2.0 is in the here and now, in use by large
interconnected user communities, and is ever growing as more
people adopt and contribute to various community efforts. To try
and specify Web 2.0 would almost be a contradiction in terms,
and restricting its users with strong recommendations would be
seen as an attempt to unnecessarily limit the creativity of those
who have something new to try. Taxonomies give way to
folksonomies, letting the user mark-up things lightly on the Web
rather than specify a typed URI. The technologies of Web 2.0
were not specified; they evolved out of clear and present needs of
users to connect with one another. The principles of Web 2.0
grow out of a mixture of hindsight and insight to current practice,
and revolve around online community building, quick and easy
linking, unlimited customization in the hands of the masses. In
this article we use ‘Web 2.0’ to refer to these principles rather
than any specific technology.</p>
      <p>It has not gone unnoticed however that the artifacts, such as
vocabularies and ontologies, that will support the Semantic Web
need populating [25, 26], and for this to happen, both the
technology and the nature of ontology building need to be
accessible to the masses. Similarly in the computer science view
of knowledge artifacts such as ontologies inherently have this
community aspect—they are shared conceptualizations that aim to
enable both human and computational interoperation of diverse
resources at a semantic level.</p>
      <p>The simplicity and robustness of HTML fuelled the growth of the
current Web, but the highly-specified nature of the technologies
in the Semantic Web recommendations suggests that the semantic
side of the development, delivered through ontologies, will be
driven mostly by experts. In this way, it is key that somehow this
barrier of complexity is lowered through creating an easier user
1 http://www.w3.org/2001/sw/
experience, and that the motivators that are driving Web 2.0 are
harnessed to promote uptake of Semantic Web ideas.</p>
      <p>In this paper we consider the social and communication
dependent aspects of the ontology development life cycle, and
identify problems encountered by people with specific roles of
interaction. From this, we suggest that a clear, layered separation
is made between statements in ontologies that are logical and
those that are linguistic, supporting annotations on the ontology.
In doing so, the annotations can be exposed to the collaborative
aspects of Web 2.0, promoting light discussion at the level of
natural language about the meanings of terms, whilst leaving the
heavier encoding of knowledge into OWL as a task for
ontologists.</p>
    </sec>
    <sec id="sec-3">
      <title>2. ONTOLOGIES AND DEVELOPMENT</title>
      <p>The central premise of the Semantic Web is enabling
computational processing of Web resources through knowledge
artifacts. The W3C have provided the Resource Description
Framework (RDF) and the Web Ontology Language (OWL)
recommendations. The latter, particularly in its OWL-DL variant,
is offered as a means of building robust property based
descriptions with a logical underpinning that can be used to
provide vocabulary for describing Web content, but also support
reasoning across Web content [20]. Such ontologies are to be the
semantic backbone for linking resources in the Semantic Web.
Additionally, these ontologies are to represent knowledge of
domains, and have the virtues of being sharable and reusable. As
yet, it is difficult to find an ontology that could be said to have
been designed to fit the criteria for enabling a Semantic Web by
being domain general and rich in content. One prominent example
of an ontology approaching these criteria is the Foundational
Model of Anatomy (FMA) [12, 23]. The FMA could be said to be
more of a true domain ontology (or reference ontology) than any
other in bio-medicine. However, even the FMA has barriers to the
Semantic Web goals of sharing and reuse because of its large size,
perhaps because it was developed in Frames and later converted
to OWL.</p>
      <p>In computer science, what are called ontologies covers a broad
range of knowledge artifacts. Glossaries, vocabularies, thesauri,
informal and formal ontologies (both in language and ontological
discrimination) are all used at various points in the Semantic
Web. Different levels of expressiveness (sometimes called
formality) come from the purpose and demands of the ontology
being developed [28]. These demands can be considered with
increasing levels of expressiveness from very “light-weight” term
lists, thesauri, dictionaries or hierarchies up to “heavy-weight”
with very expressive constraints [10, 25]. OWL-DL offers a
formal language and can be used to build rich, logical
representations of descriptions of what exists; it can also be used,
in various forms, to develop other forms of knowledge artifact
while still retaining strict language semantics in the
representation, but weakening the ontological distinctions made in</p>
      <p>Building OWL-DL logic based ontologies is a difficult process
[21] and reaching a community consensus is hard, especially in
complex domains such as biology, where knowledge for making
ontological distinctions can be incomplete. These issues need to
be addressed if ontologies are to play their role in the Semantic
Web. Here, we are mostly interested in the aspect of reaching a
community consensus. Focus is often placed on the aspect of
collaborative ontology building, that is, a group of people
working directly with one ontology. We do not aim to discuss this
type of system, as we see such systems as expert systems for
logic-savvy ontologists rather than currently being suitable for
“the masses”. Much more work needs to be done on enabling true
collaboration in logic based ontologies. Instead, we currently
envisage a core of expertise for logic encoding supported by
people conceptualizing and gathering linguistic material. We
acknowledge that there is a wealth of methodologies that address
certain aspects of the ontology development lifecycle [10, 29] and
evaluation [8, 24], good reviews of these fields can be found in
the references. For the purposes of this paper, we wish to focus on
the social interactions during these processes rather than the
processes themselves.</p>
    </sec>
    <sec id="sec-4">
      <title>3. ONTOLOGY COMPREHENSION</title>
      <p>We learn from the field of software engineering that effective
reuse of elements of object oriented frameworks is reliant on
many levels of understanding from the point of view of the
programmer [4, 15]. In software engineering, improving these
levels of understanding is known as “software comprehension”,
and we extend the principles to ontology development. We
outline ontology comprehension as the interaction between human
agents and the knowledge expressed in an ontology.</p>
      <p>Development mode. Ontology development requires
that there is efficient interaction between experts that
represent the knowledge of the domain in the scope of
the ontology (domain experts) and the ontologist that is
responsible for the construction and continued
maintenance of the ontology. Here we assume a model
where, for a specific ontology development exercise,
there is a limited cohort of domain experts that are
involved with an ontologist.</p>
      <p>Inspection mode. Ontology inspection is a light
evaluative process that an agent will go through the
ontology to quickly assess whether or not that ontology
is of good quality and whether what it contains is
suitable for some specific needs of the inspector.</p>
      <p>What follows is an outline of task models that highlight how
currently, the interactions of agents involved with ontologies
leads to discrepancies in ontology comprehension.</p>
    </sec>
    <sec id="sec-5">
      <title>3.1 Task Model 1: Ontology Development</title>
      <p>We consider early ontology development as a process that begins
with the lightest possible knowledge structure, essentially a term
list, and subsequently moves up through levels of complexity and
expressiveness of the types discussed in [10]. This happens
socially as well as in the ontology as all those involved in the
development become more familiar with scope. At the beginning
of the ontology development life cycle, the ontologist (assuming
they have no domain knowledge) will usually rely on the domain
expert to provide a core set of terms from the domain of interest
as a starting point. The initial scope of the ontology, rather than
being rigidly defined, is often roughly determined from the initial
term list and this will get refined as things move on. At this early
stage it is necessary for the domain expert to be able to quickly
assess if the terms are appropriate. As things are, the easiest way
to do this is for the domain expert to be able to access the
ontology for themselves and browse the hierarchy of terms, whilst
checking and adding in textual annotations for the terms, as well
as any comments about the specific or contextual use of any of
the terms.</p>
      <p>The ontologist will be using one of the commonly available
ontology development tools such as Protégé-OWL2, Swoop3 and
OBO-Edit4. All of these tools are centered on the user interacting
with a class hierarchy view, which the ontologist will be building
from the terms given to them by the domain expert. At this stage,
the domain expert will primarily be concerned with having the
correct term-definition pairs represented in the proto-ontology.
Decisions regarding the class hierarchy signal the beginning of a
slightly more complex level of expressivity, as the ontologist will
be making assertions between classes about subsumption
relationships [14]. This is especially true of OWL ontologies, and
such decisions do not necessarily need to be considered for
simpler controlled structured vocabularies in which hierarchical
relationships “broader than” and “narrower than” are possible.
The ontologist may also start to guide the domain expert in how
to transfer knowledge regarding some of the more fundamental
object properties such as part-hood.</p>
      <p>At some point, the domain experts need to let the ontologists start
to make even more expressive assertions in the ontology that they
may not necessarily understand the implications of for
themselves. This signals the next stage of ontology development,
in which the balance shifts so that the ontologist starts to refine
the assertions in the ontology. Instead of being instructed and
guided by the domain expert, the ontologist now needs to ask
careful questions of the domain expert. The aim of these questions
should be to extract the intrinsic meaning of the terms that the
domain expert has provided so that the ontologist can encode
these meanings into the ontology using more and more expressive
restrictions and axioms. Significantly, unless the domain expert
has had training in understanding the meanings of logical
assertions of ontologies, they will still primarily rely on the
lexical annotations and definitions when evaluating the ontology.
Once the content of the ontology has begun to stabilize (i.e. there
are fewer major revisions in the content of the ontology being
made) it will be made available to a wider audience. This can
signal a whole new critical process of revision for the ontology. In
the next section we will consider what sort of interactions may
occur between different agents and ontologies when they are first
encountered.</p>
      <p>Eventually, the increase in the content of the ontology, both
lexical and logical, should start to level off as the content and the
intended scope, at which time further structural modifications
may be made, such as modularization, which could happen once
2 http://protege.stanford.edu/
3 http://code.google.com/p/swoop/
4 http://www.oboedit.org/
the micro-organization of the knowledge in a domain has become
clear. A publicly available and relatively stable ontology has a
new set of requirements, for which the topics of ontology
evolution and change management address [19]. Change
management of ontologies has been considered in a technological
sense for some time, and it should be clear that changes to a
publicly available ontology need to be transparent. However,
there is a growing trend for including extra hierarchical structures
into the ontology that represent deprecated classes (e.g. [30]). The
need to do this is obvious; it is less so how to do it neatly and
ontologically. Versioning etc. are all parts of the ontology
lifecycle that have no really, consistent support.</p>
      <p>The following discrepancies in ontology comprehension should
be clear from this section.</p>
      <p>1.</p>
      <sec id="sec-5-1">
        <title>Discrepancies in Early Development</title>
      </sec>
      <sec id="sec-5-2">
        <title>The most convenient means of constructing, looking at and sharing the early term list is, unusually, from within an ontology file, which implies some hierarchical structure.</title>
      </sec>
      <sec id="sec-5-3">
        <title>Early revisions of the ontology are experimental for the ontologist, yet are still subject to inspection and lexical evaluation from the domain expert.</title>
      </sec>
      <sec id="sec-5-4">
        <title>Domain experts, having looked directly at</title>
        <p>revisions of the ontology file, may be resistant
to subsequent major changes in structure and
terminology by the ontologist as knowledge is
disambiguated.
2.</p>
      </sec>
      <sec id="sec-5-5">
        <title>Emerging Discrepancies</title>
        <p>3.</p>
        <p>Communicative Discrepancies
a.
b.
c.
a.
a.
b.</p>
        <p>c.</p>
      </sec>
    </sec>
    <sec id="sec-6">
      <title>3.2 Task Model 2: Ontology Inspection</title>
      <p>Ontologies are complex entities. If any ontology is going to get
used by someone other than the person or group that implemented
it, there has to be a way in which it can be decided whether or not
it is an appropriate ontology for the task in hand [18]. Currently,
this inspection process is difficult because of the paucity of
ontologies available, and the fact that many have been designed
for a specific purpose. Also, the discrepancies listed in 3.1 result
in a general lack of information that can aid effective inspection
and overall ontology comprehension.</p>
      <sec id="sec-6-1">
        <title>Inclusion of information regarding deprecated classes into the class hierarchy of the ontology.</title>
      </sec>
      <sec id="sec-6-2">
        <title>Discussions between the domain experts about terminology that are potentially crucial for ontology comprehension are lost or are completely separated from the ontology itself.</title>
      </sec>
      <sec id="sec-6-3">
        <title>Discussions between the domain experts and the ontologists about disambiguation of terms are lost or are completely separated from the ontology itself.</title>
      </sec>
      <sec id="sec-6-4">
        <title>Potential for misinterpretation of logical</title>
        <p>aspects of the ontology by the domain experts
through exposure to the logical component.</p>
        <p>It is hard not to liken an ontology inspection process to some sort
of evaluation. What we describe here is fairly close to ontology
selection [24], except that ontology inspection is more of a
browsing process, driven by what access there is to comparative
information between several ontologies. Selection has much better
defined initial parameters for the desired outcome, and can give a
more targeted outcome. We do not wish to label this inspection as
an evaluation however, as we do not make the assumption that the
inspector will be following any pre-determined criteria, and if
they are, that they are rational criteria.</p>
        <p>The ontology inspection process is short lived, and for many
people’s goals, the choice of beginning a new ontology that they
know will satisfy their criteria is more favorable than editing an
existing one. However, such inspections can quickly be deemed
fruitless when the term searched for turns out not to be defined by
logical statements in an ontology. This is a common occurrence,
as such ‘classes’ can be placeholders for future development or
intrinsically defined terms where no logical definition was
thought necessary. Ontologies can be intensely developed in one
particular area where immediate goals are important, yet there is
no way to effectively discover this other than through thorough
browsing. For the goals of the Semantic Web, it is imperative that
such information required to carry out this inspection process be
made as clear as possible for the inspector, such that we do not
see immense reproduction of individual effort and no clear
“shared conceptualizations”.</p>
        <p>The domain knowledge these ontologies describe can require a
considerable amount of understanding for anyone trying to
inspect them. There are several ways in which this can be the
case.</p>
        <p>1. The domain knowledge encoded may be outside the
experience of the inspector, or in a different context to
what was expected. The inspector may not be able to
tell if the knowledge represented is valid because it is
not within their expertise, and will need to seek help
and advice from a domain expert.
2. The knowledge may be appropriate, but encoded with
axioms and restrictions that the inspector may not be
able to accurately interpret as real world meaning, such
that they have to find the advice of an ontologist.
3. The ontology may have been written for a specific
purpose. The inspector may not be able to tell whether
this is the case, and could therefore assume that the first
or second scenario above is true, unless it is possible to
seek advice from the original authors or find a resource
containing this information.</p>
        <p>The three scenarios above are serious issues for the future of
ontologies in the Semantic Web. Most ontologies are developed
as part of projects, and projects are usually pragmatic in terms of
their goals. Hence, people build these ontologies as application
ontologies that serve the immediate needs of the project. There is
no perceivable immediate benefit for a project to develop a more
general domain ontology in tandem with an application ontology,
and so it does not happen. Consequently the Semantic Web goals
of sharing and reuse become much harder, as people will tend to
assess these application specific ontologies as too specific for a
new purpose, as they see that they will need to invest effort in its
re-engineering. Another danger here is that with so many
application ontologies being developed, that inspectors always
start to assume that unusual features of ontologies are the result of
the needs of an application, and dismiss the ontology as
potentially unusable. What is really needed is for the inspector to
be sure what sort of artifact they are looking at by having easy
access to certain parameters.</p>
        <p>In the Semantic Web vision, the first course of action for an
ontologist would be to verify the existence or non-existence of a
domain ontology with close or overlapping scope to the ontology
they are to develop. This process will be laborious if it relies on
the current practice of downloading ontologies and browsing them
to see if they are at all reusable. In response to this, technologies
such as Swoogle [16] and AKTiveRank [2] are starting to provide
access to online ontologies through page ranking and other
analytical methods to establish potential target ontologies.
However, these technologies have been criticized for ignoring the
meaning of concepts and also relations [24]. Furthermore, we note
that the results returning from these searches are whole OWL
files, free and independent of contextual information. For
example, a Swoogle search for “Protein” has in its top hits an
ontology used in an educational tutorial (that in this case is
evident from its URL), which is by no means intended to be a
shared or reusable resource, but none the less is discovered and
accessible.</p>
        <p>Those inspecting ontologies can find themselves in an isolated
situation where Web searches and personal inspection of an
ontology or its documentation are the only means to ontology
comprehension. It has already been recognized that the Web has
enormous potential for social organization and engagement. In
ontology comprehension, for example, it offers the means of
asking those who know. It also, as Wikipedia has shown, offers
the means by which elements that aid ontology comprehension
can be developed. Having concluded the need for ontology
comprehension, we now explore what is necessary for such a
facility.</p>
        <p>The following discrepancies in ontology comprehension should
be clear from this section:
1.</p>
        <p>Discovery Level Discrepancies
a.
b.
c.
a.
b.
c.
a.</p>
      </sec>
      <sec id="sec-6-5">
        <title>Targeted discovery based on search for terms rather than meanings</title>
      </sec>
      <sec id="sec-6-6">
        <title>Ontologies are discoverable independently of</title>
        <p>statements of purpose, scope etc.</p>
      </sec>
      <sec id="sec-6-7">
        <title>Searches may discover anything from tutorial OWL files, programmatic OWL fragments, application ontologies, outdated or unmaintained ontologies etc.</title>
      </sec>
      <sec id="sec-6-8">
        <title>Statements of scope, purpose, expressivity etc are often missing altogether, or require extra searches to discover them.</title>
      </sec>
      <sec id="sec-6-9">
        <title>Discussions that have affected ontology development are not recorded overall</title>
      </sec>
      <sec id="sec-6-10">
        <title>Minimal opportunities to interact with the development team</title>
      </sec>
      <sec id="sec-6-11">
        <title>Feeding in from Section 3.1, ontologies need exploring in the development environment to assess appropriateness of terms. 2.</title>
      </sec>
      <sec id="sec-6-12">
        <title>Ontology Level Discrepancies 3.</title>
      </sec>
      <sec id="sec-6-13">
        <title>Term Level Discrepancies b.</title>
      </sec>
      <sec id="sec-6-14">
        <title>No indication without exploration of the level of effort put into different areas of an ontology.</title>
      </sec>
    </sec>
    <sec id="sec-7">
      <title>4. DESIDERATA FOR SEMANTIC</title>
    </sec>
    <sec id="sec-8">
      <title>ONTOLOGY COMPREHENSION</title>
      <p>Section 3 highlights the social and communicative discrepancies
that prevent an effective amount of ontology comprehension that
is required for the uptake of the Semantic Web goals of ontology
sharing and reuse. This section cross-analyses these discrepancies
to produce some desiderata that can be considered for future
systems. Whilst all types of data in and about an ontology may be
considered ‘ontological’, we specify ‘Meta-Ontological Data’,
‘Ontological Metadata’ and ‘Logical Statements’ as clearly
identifiable parts. For information contained within ontology files
that is only for human interpretation of the encoded semantic
content, we use the idea of Meta-Ontological data. For data
specific to an individual ontology that is necessary for interpret
and inspection across the whole structure and history of
development, we use the idea of Ontology Metadata. The ‘logical
statements’ in an ontology constitute the remainder of the content.</p>
    </sec>
    <sec id="sec-9">
      <title>4.1 Separating the Ontological from the</title>
    </sec>
    <sec id="sec-10">
      <title>Meta-Ontological</title>
      <p>Ontologies come with a considerable amount of meta-ontological
information (or should do so) which is used by the human to
assess and see the intended use of that ontology. Much of this
meta-ontological information is linguistically orientated. These
meta-ontological extensions to the ontology itself are meaningless
strings to the computer, and in this respect are unnecessary in so
far as the computational goals of the Semantic Web are
concerned. We know that this meta-ontological information is
necessary, but we also see that it is not convenient to access; it
lacks the human resources that often make the most of such
material, as in Section 3.2 where a lack of a single access point
means that secondary information needs to be sought out
manually.</p>
      <p>In reality, we have a chance to design and build support for the
meta-ontological in the light of current experience. OWL has
virtually no support, apart from some ad hoc solutions, for
carrying meta-ontological knowledge. We would advocate such a
separation of the ontological from meta-ontological and this is
where a Web 2.0 approach could help.</p>
      <p>Our current scenario places too much reliance on assessment
through simple linguistic inspection of, for instance, terms. These
are labels for concepts and a simple assumption of lexical
matching implying conceptual matching is dangerous. For
example, in biology, it might seem safe to assume that hepatocyte
and liver cells are the same thing. In fact, cells in the liver include
hepatocyte cells, but also include adipocyte cells. Hepatocytes
make the liver the liver, but there are other cells too.</p>
      <p>Ontologies are only intuitively discoverable through the
identification and inspection of the appropriate individual terms.
Even the construction of linguistic definitions can leave
ambiguous meanings for those inspecting an ontology, with no
real way to find out how those definitions were converged upon.
Even with logical definitions, we still rely upon natural language
labels. The aim of languages such as OWL-DL is, however, to
minimize potential ambiguities through logical descriptions.
Overall, there should be a synergy between logical and linguistic
definitions.</p>
      <p>Non-ontologist domain experts will attach intrinsic meaning to
terms by drawing on their internal knowledge and the context in
which a term is used. It is possible to restrict the intrinsic meaning
of a term using the consensus of a domain, so long as it is stated
in the context of the purpose of the controlled vocabulary.
Interpretation of meaning in these controlled vocabularies still
requires a human agent and the knowledge is logically
inaccessible to a computer agent.</p>
      <p>Thus, we define the inline linguistic portions of ontology files as
meta-ontological data. These include anything that a human agent
would use for the translation of specific complex logical
statements into meaning (including links to other meanings) but
are also intrinsically meaningless to the computer. Primarily,
these are:
knowledge in a way not possible in file-oriented development.
The ontologists have a way to interact with the domain experts as
a community to perform tasks such as the disambiguation of terms
before they have been encoded in an ontology, reducing the
chance that major revisions of ontological structure will be
required. As this resource is shared and linkable, project and
domain contexts for terms can be established. These contexts can
be used by both the ontologists and the domain experts to traverse
the gap into discussions that involve other groups, and discover
overlapping scopes more intuitively. Additionally, these resources
would provide ideal testing grounds for lexical research (e.g. [7])
that should lead to future improvement on methodologies for
these workspaces.
•
•</p>
      <sec id="sec-10-1">
        <title>The specific string by which the logical meaning is labeled, usually considered as the real meaning. E.g. (from celltype ontology) ‘subsidiary cell’</title>
        <p>Any number of labels that refer to the same
meaning.</p>
        <p>o E.g. (cont’d) ‘accessory cell’
• Definitions
o Short, concise description of the meaning,</p>
        <p>including links to other terms.
o E.g. (cont’d) ‘An epidermal cell associated
with a stoma and at least morphologically
distinguishable from the epidermal cells
composing the groundmass of the tissue’
• Annotations (examples of)
o Longer, more verbose descriptions.
o Examples of how the term is used.
o Explanations of contextual use for the term.
o Links to term provenance.</p>
        <p>o E.g. (cont’d) DBXREF - TAIR:0000296
Achieving the separation of this meta-ontological layer allows for
the consideration of how to manage this mostly linguistic
information. This separation is our major desideratum and from
this flows the means by which Web 2.0 can provide a platform to
expose meta-ontological information and harness and extend the
range of group activities.</p>
      </sec>
    </sec>
    <sec id="sec-11">
      <title>4.2 Promoting Social Interaction – A Meta</title>
    </sec>
    <sec id="sec-12">
      <title>Ontological Workspace</title>
      <p>Explicit logic based ontologies for the Semantic Web are going to
need to capture implicit knowledge with axioms and restrictions.
Yet, unless the experts with the knowledge all manage to learn
how to interpret complex logical statements, there needs to be a
workspace in which implicit knowledge can be discussed and
defined lexically within expert groups. In other words, terms and
term linked information can essentially exist independently of the
formal environment of ontologies. This implicit knowledge can be
used by ontologists as a resource. With such a resource,
development of early stage ontologies will not require the
construction of formal hierarchies until a critical amount of
implicit knowledge has been collected in these more lightweight
resources. Also, multiple hierarchies for different purposes could
be constructed from the same resource, reusing the collected</p>
      <p>Generating discussion of implicit meaning may sound a little like
cutting the domain expert out of the ontology construction
process. It should in fact considerably reduce the overhead of
ontology development by shifting the discussions based around
intrinsic meanings of terms and which terms are the most
appropriate to use away from the attention of the ontologists. It is
important not to make the division too wide, as there is a risk that
bias could creep in from the ontologists as the domain experts
would be unable to assess the implications of certain restrictions
and axioms. In terms of feedback to the domain expert from the
ontologist, we recognize that there is a need for some sort of
consistent translation methodology that can generate accurate
textual definitions from logical statements, but we consider this
outside of the scope of this article.</p>
      <p>It should be clear that such discussion workspaces would be well
suited for Web 2.0 style systems. These workspaces should
promote the creation of lexicons in which a group of experts can
start to add in and inspect lexical information. In this implicit
view, it is the terms that are the focus of discussion, not the
ontological interpretation, which are two different goals that
sometimes get confused during ontology development. Within the
workspace, the terms can be discussed, and annotated with textual
definitions, comments about usage, links to synonymous terms,
requests for clarification etc. Helium was, for instance, discovered
in 1894. Of course it was the category of Helium that was
discovered, not the instances of the helium atoms (which
presumably have existed much before 1894). This is an example
of meta-class statements that are part of the ontology. They are
class level statements, but those that are well suited to this
linguistic, community style of interaction.</p>
      <p>The purpose of targeting Web 2.0 as a base for this
metaontological data is not to completely remove this type of
information from the view of the ontology, we merely seek to
relocate it so that the incredibly social nature of the definition of
knowledge can be coupled with an environment that is equally
socially driven (see Figure 2). Modularization of ontologies is
seen to be one of the keys to making ontologies viable for the
Semantic Web vision, and as such, import mechanisms exist that
support the combination of different sources. Lexicons developed
by groups could be given URIs, as could all of the terms
described in them. Knowledge held in WordNet [17] style lexical
resources could be linked using online URIs in a similar way to
imported online ontologies taking advantage of well established
methods for dealing with words and their meanings at the lexical
level.</p>
    </sec>
    <sec id="sec-13">
      <title>4.3 Promoting Ontology Sharing and Reuse:</title>
    </sec>
    <sec id="sec-14">
      <title>An Ontological Metadata Workspace</title>
      <p>The production of ontologies that can be effectively shared and
reused is a major step towards achieving the goals of the Semantic
Web. There are significant barriers to these goals in our current
model of ontological comprehension. We have highlighted how
ontologists and domain experts alike need to inspect ontologies to
assess whether they are appropriate for their needs. Currently, the
information that would be necessary to effectively conduct this
investigation is hard to find, and does not always come in the
same format.</p>
      <p>An ontological metadata workspace would provide access to
whole-ontology level information for ontologies necessary to
carry out light evaluative processes. A collaborative Web 2.0
approach to ontologies would see ‘ontology profiles’ that include
clear statements about the purpose and scope of the ontology and
information regarding its status. Ontologies would clearly be
labeled as domain and application ontologies to help evaluation,
and subsequently, when application ontologies are derived from
domain ontologies, this can be marked up and become visible
such that ontology level provenance, a history of where
everything in an ontology originated from and how it changed
over time, can start to be built up.</p>
      <p>Introducing a strong community aspect would encourage those
developing ontologies to start using tagging, thereby linking up
their ontologies to particular domains and projects. Domain
ontology construction could be promoted by using ranking
systems where inspectors can rate how useful the ontology was in
terms of what was expected, assuming that more general
ontological models will fit the requirements of more people.
OWL has been made popular for use as an ontology language
because of the publicity of the Semantic Web, accessibility of
tools for creating OWL ontologies and the fact that it is useful
beyond the scope of the Semantic Web. OWL has been used for a
lot of purposes, and searching for ontologies based on the content
of their files seems like it may be unsustainable as the number of
files grows faster than the number of useful ontologies.
Efficient inspection of ontologies can be limited by a large size of
ontologies. The current tendency is to build larger ontologies, as
the tool support and methodologies for modularization have been
slow off the mark until recently. As we learn more about the
implications and methods of modularization [22], ontologies can
become more manageable, reducing the amount of evaluation cost
per ontology. This of course will require better indexing, along
with information about how each ontology has been
used/imported, perhaps leading to a ‘shopping cart’ model for
highly modular ontology construction.</p>
      <p>Perhaps one of the most motivating factors for achieving this
desire for more effective inspection is the aspect of learning. Once
it becomes easy to empirically see what constitutes a good and
useful ontology, then these features get propagated and discussed.
As has been noted in [1], the viral spread of understanding how to
write HTML was in part because existing HTML could be
inspected and copied. Also, the effect of newly written HTML
was instantly verifiable in a Web browser. It is harder to have this
sort of verification with ontologies, and there are a lot of
conflicting styles of ontology development with no consensus of
what is ‘right’, If the Ontological Metadata Workspace were to be
realized, then a hub of comparable, commented and marked-up
ontologies could develop in a much quicker and consistent
fashion than the solitary efforts that are currently the norm.</p>
    </sec>
    <sec id="sec-15">
      <title>5. BIO-ONTOLOGIES: EXPERIENCES</title>
    </sec>
    <sec id="sec-16">
      <title>AND PERSPECTIVES</title>
      <p>While our discussion in this article is most pertinent to the notion
of the Semantic Web as a whole, it originates from the discipline
of bioinformatics. Biologists were early adopters of the Web as a
means of disseminating data and the tools for their analysis. These
data and tools are developed in a highly autonomous manner and
consequently they are beset by both syntactic and semantic
heterogeneities. Bioinformaticians have seen ontologies as a
means to create common understandings for human and
computers about the meaning of data in their distributed resources
in a life science Semantic Web [13]. The DNA sequences of
different organisms, for example have a common representation,
but this is not so for the functional knowledge associated with
those sequences. So, the sequences can be interpreted by humans
and computers, but not what is known about those sequences.
Consequently, biologists have created ontologies to describe, for
instance, the functional attributes of DNA and proteins [3, 11].
Bioinformatics has, therefore, much Web accessible data
described by ontologies. The W3C have recognized a nascent
Semantic Web in this domain in the development of the Health
Care and Life Sciences SIG5. It is a significant feature of the
move towards ontologies in this sector that it is biologists who
build these tools, with some guidance from ontologists. Whilst
this community has not made great use of OWL, but its own
representation, OBO6, it still provides a good representation of
Semantic Web activities.</p>
      <p>The OBO ontologies have significant standing in biological
communities, and it is perhaps the community building aspect that
fuels this standing, as it includes:
•
•
•
•
•</p>
      <sec id="sec-16-1">
        <title>A large number of centrally available OBO ontologies7</title>
        <p>The OBO-Edit OBO ontology development tool that is
specifically designed by a working group of users.</p>
        <p>A committee, the OBO-Foundry8, that has been set up
and has produced a set of principles for new OBO
ontologies to aspire to, including the promise of textual
definitions for all terms and good documentation for all
ontologies.</p>
        <p>The OBO file format, for which the primary goals
include human readability and ease of parsing together
with a syntax that makes them exportable as OWL.</p>
        <p>Pages on the SourceForge9 open source software
development site, which includes the potential for
project information, forums, downloads and issue
tracking by which suggestions for new terms and
modifications can be submitted.</p>
        <p>Contributors to OBO are starting to pull together as a virtual
community by pooling its resources on the Web. The Gene
Ontology [3] saw a phenomenal growth in the number of terms it
contained through user interaction alone that is well documented
[5], such was the demand for the resource to represent so many
researchers. Since then, the trend has continues as more and more
biological domains aim to be represented by OBO.</p>
        <p>The caveat for the relative success of OBO has probably been
similar to that of Web 2.0 over Semantic Web (so far). Formality
and methodology have temporarily made way for ease of use and
ease of interaction. Interestingly, the majority of the OBO
ontologies clearly state that they are “structured controlled
vocabularies”, which require nothing like the expressive power of
OWL, and little in the way of knowledge engineering because the
statements linking things do not require it. This is not for any
other reason than nothing more complex than this is required,
OBO ontologies are used for marking biological data so that they
can be linked if they are annotated in the same way. Primarily,
these ontologies contain a hierarchy of terms denoting ‘is_a’
relationships. Less often but still common are ‘part_of’
relationships, and occasionally other properties key to biology</p>
      </sec>
      <sec id="sec-16-2">
        <title>5 http://www.w3.org/2001/sw/hcls/</title>
        <p>6 http://www.geneontology.org/GO.format.obo-1_2.shtml</p>
      </sec>
      <sec id="sec-16-3">
        <title>7 http://obo.sourceforge.net/</title>
      </sec>
      <sec id="sec-16-4">
        <title>8 http://obofoundry.org/</title>
      </sec>
      <sec id="sec-16-5">
        <title>9 http://sourceforge.net/</title>
        <p>such as ‘develops_from’. Despite having the full expressivity of
OWL available in the OBO 1.2 syntax, there is little evidence to
suggest that the developers in this community either see the need
or have the will to take on this level of expressivity in their
knowledge.</p>
        <p>Perhaps then, this community can be a model for the future of
ontology development on the Web. Quick and easy development
of terms by engaging the user, employing Web 2.0 design
principles to forge more coordinated communities for
development of Semantic Web technologies. Web 2.0 has the
capability to expose all of the ‘light’ lexical issues and some basic
assertions of linking meaning to terms. ‘Heavier’ more expressive
assertions in OWL are in the domain of the ontologist, who can be
informed by the interactions they can have with domain experts
and other ontologists through Web 2.0 communities.</p>
      </sec>
    </sec>
    <sec id="sec-17">
      <title>6. DISCUSSION</title>
      <p>We propose the construction of ontology specific resources, using
the Web as a platform, which specifically deals with the
management of lexical meta-ontological aspects of ontology
development together with the management of ontology metadata.
The applications of Web2.0 are geared towards harnessing these
types of community interaction, which is precisely the sort of
interaction that is not supported in the current model of ontology
development. Dealing with meta-ontological data in
downloadable ontology files and disparate descriptions of
ontology metadata on development sites is prohibitive to a more
universal appreciation of ontology design and implementation.
A centralized resource for sharing OWL resources would act as a
hub for community learning, sharing and reusing of ontology
resources, bringing together ontology users and builders in a way
that is currently not possible. Designing ontologies by consensus
in such workspaces would encourage best practice and speed up
the uptake of the more complicated Semantic Web technologies,
starting with OWL and the knowledge that is to be contained
within. At the same time the system would provide a measure of
control, ensuring that the dangers of misinterpreting the powerful
semantics of OWL by untrained eyes are avoided. Having the
community built lexical resources is the beginning of an
opportunity to link up ontologists with a more specific system that
can refer to the online lexical corpus.</p>
      <p>The widespread realization of the Semantic Web will depend on
the production of ontologies that can be effectively shared and
reused, but in order to achieve this, the overheads of ontology
development and ontology comprehension have to be
considerably reduced. The OBO community/consortium has
effectively demonstrated the advantages of lowering these
overheads by engaging a community of domain experts in
ontology development. OBO ontologies, however, are for human
interpretation, so the true Semantic Web vision of human and
computational understanding is not addressed. At the same time,
highly expressive OWL-DL ontologies, for both computer and
human interpretation are being produced, but largely in isolation.
We propose a solution here which would bridge the gap between
these approaches and effectively enable the same type of domain
expert community engagement for formal ontologies.</p>
    </sec>
    <sec id="sec-18">
      <title>7. ACKNOWLEDGMENTS</title>
      <p>Funding for this work was through BBSRC grant BBS/B/17156.
8. REFERENCES</p>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          <string-name>
            <surname>Alani</surname>
            ,
            <given-names>H.</given-names>
          </string-name>
          ,
          <article-title>Position paper: ontology construction from online ontologies</article-title>
          .
          <source>in Proceedings of the 15th International Conference on World Wide Web (Edinburgh</source>
          , Scotland,
          <year>2006</year>
          ), ACM Press, New York, NY,
          <fpage>491</fpage>
          -
          <lpage>495</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          <string-name>
            <surname>Alani</surname>
            ,
            <given-names>H.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Brewster</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          and
          <string-name>
            <surname>Shadbolt</surname>
            ,
            <given-names>N.</given-names>
          </string-name>
          ,
          <article-title>Ranking Ontologies with AKTiveRank</article-title>
          . in International Semantic Web Conference, (Athens, GA, USA,
          <year>2006</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          <string-name>
            <surname>Ashburner</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Ball</surname>
            ,
            <given-names>C.A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Blake</surname>
            ,
            <given-names>J.A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Botstein</surname>
            ,
            <given-names>D.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Butler</surname>
            ,
            <given-names>H.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Cherry</surname>
            ,
            <given-names>J.M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Davis</surname>
            ,
            <given-names>A.P.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Dolinski</surname>
            ,
            <given-names>K.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Dwight</surname>
            ,
            <given-names>S.S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Eppig</surname>
            ,
            <given-names>J.T.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Harris</surname>
            ,
            <given-names>M.A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Hill</surname>
            ,
            <given-names>D.P.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Issel-Tarver</surname>
            ,
            <given-names>L.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Kasarskis</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Lewis</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Matese</surname>
            ,
            <given-names>J.C.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Richardson</surname>
            ,
            <given-names>J.E.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Ringwald</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Rubin</surname>
            ,
            <given-names>G.M.</given-names>
          </string-name>
          and
          <string-name>
            <surname>Sherlock</surname>
            ,
            <given-names>G.</given-names>
          </string-name>
          <article-title>Gene Ontology: tool for the unification of biology</article-title>
          .
          <source>Nat Genet</source>
          ,
          <volume>25</volume>
          (
          <issue>1</issue>
          ).
          <fpage>25</fpage>
          -
          <lpage>29</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          <string-name>
            <surname>Austin</surname>
            ,
            <given-names>M.A.</given-names>
          </string-name>
          , III and Samadzadeh,
          <string-name>
            <surname>M.H.</surname>
          </string-name>
          ,
          <article-title>Software comprehension/maintenance: an introductory course</article-title>
          . in, (
          <year>2005</year>
          ),
          <fpage>414</fpage>
          -
          <lpage>419</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          <article-title>A short study on the success of the Gene Ontology</article-title>
          .
          <source>Web Semantics: Science, Services and Agents on the World Wide Web</source>
          ,
          <volume>1</volume>
          (
          <issue>2</issue>
          ).
          <fpage>235</fpage>
          -
          <lpage>240</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          <string-name>
            <surname>Berners-Lee</surname>
            ,
            <given-names>T.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Hendler</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          and
          <string-name>
            <surname>Lassila</surname>
            ,
            <given-names>O.</given-names>
          </string-name>
          <article-title>The Semantic Web</article-title>
          .
          <source>Scientific American</source>
          ,
          <volume>284</volume>
          .
          <fpage>34</fpage>
          -
          <lpage>43</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          <article-title>Assessing the consistency of a biomedical terminology through lexical knowledge</article-title>
          .
          <source>International Journal of Medical Informatics</source>
          ,
          <volume>67</volume>
          (
          <issue>1</issue>
          -
          <fpage>3</fpage>
          ).
          <fpage>85</fpage>
          -
          <lpage>95</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          <string-name>
            <surname>Brank</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Grobelnik</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          and
          <string-name>
            <surname>Mladenic</surname>
            ,
            <given-names>D.</given-names>
          </string-name>
          ,
          <article-title>A survey of ontological evaluation techniques</article-title>
          .
          <source>in Conference on Data Mining and Data Warehouses</source>
          , (Ljubljana, Slovenia,
          <year>2005</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          <string-name>
            <surname>Brickley</surname>
            ,
            <given-names>D.</given-names>
          </string-name>
          and
          <string-name>
            <surname>Miller</surname>
            ,
            <given-names>L.</given-names>
          </string-name>
          <article-title>FOAF vocabulary specification</article-title>
          ,
          <year>2005</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          <article-title>Methodologies, tools and languages for building ontologies</article-title>
          .
          <source>Where is their meeting point? Data &amp; Knowledge Engineering</source>
          ,
          <volume>46</volume>
          (
          <issue>1</issue>
          ).
          <fpage>41</fpage>
          -
          <lpage>64</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref11">
        <mixed-citation>
          <string-name>
            <surname>Eilbeck</surname>
            ,
            <given-names>K.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Lewis</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Mungall</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Yandell</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Stein</surname>
            ,
            <given-names>L.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Durbin</surname>
            ,
            <given-names>R.</given-names>
          </string-name>
          and
          <string-name>
            <surname>Ashburner</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          <article-title>The Sequence Ontology: a tool for the unification of genome annotations</article-title>
          .
          <source>Genome Biology</source>
          ,
          <volume>6</volume>
          (
          <issue>5</issue>
          ). R44.
        </mixed-citation>
      </ref>
      <ref id="ref12">
        <mixed-citation>
          <string-name>
            <surname>Golbreich</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Zhang</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          and
          <string-name>
            <surname>Bodenreider</surname>
            ,
            <given-names>O.</given-names>
          </string-name>
          <article-title>The foundational model of anatomy in OWL: Experience and perspectives</article-title>
          .
          <source>Web Semantics: Science, Services and Agents on the World Wide Web</source>
          ,
          <volume>4</volume>
          (
          <issue>3</issue>
          ).
          <fpage>181</fpage>
          -
          <lpage>195</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref13">
        <mixed-citation>
          <string-name>
            <surname>Good</surname>
            ,
            <given-names>B.M.</given-names>
          </string-name>
          and
          <string-name>
            <surname>Wilkinson</surname>
          </string-name>
          , M.D.
          <source>The Life Sciences Semantic Web is Full of Creeps! Brief Bioinform</source>
          ,
          <volume>7</volume>
          (
          <issue>3</issue>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref14">
        <mixed-citation>
          <string-name>
            <surname>Guarino</surname>
            ,
            <given-names>N.</given-names>
          </string-name>
          and
          <string-name>
            <surname>Christopher</surname>
            ,
            <given-names>W.</given-names>
          </string-name>
          <article-title>Evaluating ontological decisions with OntoClean</article-title>
          .
          <source>Commun. ACM</source>
          ,
          <volume>45</volume>
          (
          <issue>2</issue>
          ).
          <fpage>61</fpage>
          -
          <lpage>65</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref15">
        <mixed-citation>
          <string-name>
            <surname>Kirk</surname>
            ,
            <given-names>D.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Roper</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          and
          <string-name>
            <surname>Wood</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <article-title>Identifying and addressing problems in framework reuse</article-title>
          . in, (
          <year>2005</year>
          ),
          <fpage>77</fpage>
          -
          <lpage>86</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref16">
        <mixed-citation>
          <string-name>
            <surname>Li</surname>
            ,
            <given-names>D.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Tim</surname>
            ,
            <given-names>F.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Anupam</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Rong</surname>
            ,
            <given-names>P.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Cost</surname>
            ,
            <given-names>R.S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Yun</surname>
            ,
            <given-names>P.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Pavan</surname>
            ,
            <given-names>R.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Vishal</surname>
            ,
            <given-names>D.</given-names>
          </string-name>
          and
          <string-name>
            <surname>Joel</surname>
          </string-name>
          ,
          <source>S. Swoogle: a search [17] [18] [19] [20] [21] [22] [23] [24] [25] [26] [27] [28] [29] [30]</source>
          [31]
          <article-title>and metadata engine for the semantic web Proceedings of the thirteenth ACM international conference on Information and knowledge management</article-title>
          , ACM Press, Washington,
          <string-name>
            <surname>D.C.</surname>
          </string-name>
          , USA,
          <year>2004</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref17">
        <mixed-citation>
          <source>Communicationd of the ACM</source>
          ,
          <volume>38</volume>
          (
          <issue>11</issue>
          ).
          <fpage>39</fpage>
          -
          <lpage>41</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref18">
        <mixed-citation>
          <string-name>
            <surname>Noy</surname>
            ,
            <given-names>N.F.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Guha</surname>
            ,
            <given-names>R.</given-names>
          </string-name>
          and
          <string-name>
            <surname>Musen</surname>
            ,
            <given-names>M.A.</given-names>
          </string-name>
          ,
          <article-title>User Rating of ontologies: Who will rate the raters? in AAAI 2005 Spring Symposium on Knowledge Collection from Volunteer Contributors, (Stamford</article-title>
          , CA, USA,
          <year>2005</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref19">
        <mixed-citation>
          <string-name>
            <surname>Noy</surname>
            ,
            <given-names>N.F.</given-names>
          </string-name>
          and
          <string-name>
            <surname>Klein</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          <article-title>Ontology Evolution: Not the Same as Schema Evolution. Knowledge and Information Systems</article-title>
          ,
          <source>V6 (4)</source>
          .
          <fpage>428</fpage>
          -
          <lpage>440</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref20">
        <mixed-citation>
          <string-name>
            <surname>Knowledge-Based</surname>
            <given-names>Systems</given-names>
          </string-name>
          ,
          <volume>19</volume>
          (
          <issue>7</issue>
          ).
          <fpage>489</fpage>
          -
          <lpage>497</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref21">
        <mixed-citation>
          <string-name>
            <given-names>OWL</given-names>
            <surname>Pizzas</surname>
          </string-name>
          <article-title>: Practical Experience of Teaching OWLDL: Common Errors</article-title>
          &amp;
          <string-name>
            <surname>Common Patterns</surname>
          </string-name>
          ,
          <year>2004</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref22">
        <mixed-citation>
          <string-name>
            <surname>Rector</surname>
            ,
            <given-names>A.L.</given-names>
          </string-name>
          <article-title>Modularisation of domain ontologies implemented in description logics and related formalisms including OWL Proceedings of the 2nd international conference on Knowledge capture</article-title>
          , ACM Press, Sanibel Island, FL, USA,
          <year>2003</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref23">
        <mixed-citation>
          <string-name>
            <surname>Rosse</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          and
          <string-name>
            <surname>Mejino</surname>
            ,
            <given-names>J.L.V.</given-names>
          </string-name>
          <article-title>A reference ontology for biomedical informatics: the Foundational Model of Anatomy</article-title>
          .
          <source>Journal of Biomedical Informatics</source>
          ,
          <volume>36</volume>
          (
          <issue>6</issue>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref24">
        <mixed-citation>
          <string-name>
            <surname>Sabou</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Lopez</surname>
            ,
            <given-names>V.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Motta</surname>
            ,
            <given-names>E.</given-names>
          </string-name>
          and
          <string-name>
            <surname>Uren</surname>
            ,
            <given-names>V.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Ontology</surname>
            <given-names>Selection</given-names>
          </string-name>
          :
          <article-title>Ontology Evaluation on the Real Semantic Web</article-title>
          . in
          <string-name>
            <surname>WWW2006</surname>
          </string-name>
          , (Edinburgh, UK,
          <year>2006</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref25">
        <mixed-citation>
          <string-name>
            <surname>in</surname>
            <given-names>Semantics</given-names>
          </string-name>
          , (Vienna, Austria,
          <year>2005</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref26">
        <mixed-citation>
          <string-name>
            <surname>Shadbolt</surname>
            ,
            <given-names>N.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Hall</surname>
            ,
            <given-names>W.</given-names>
          </string-name>
          and
          <string-name>
            <surname>Berners-Lee</surname>
            ,
            <given-names>T.</given-names>
          </string-name>
          <article-title>The Semantic Web revisited</article-title>
          .
          <source>IEEE intelligent systems</source>
          ,
          <volume>21</volume>
          (
          <issue>3</issue>
          ).
          <fpage>96</fpage>
          -
          <lpage>101</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref27">
        <mixed-citation>
          <string-name>
            <surname>Stevens</surname>
            ,
            <given-names>R.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Baker</surname>
            ,
            <given-names>P.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Bechhofer</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Ng</surname>
            ,
            <given-names>G.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Jacoby</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Paton</surname>
            ,
            <given-names>N.W.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Goble</surname>
            ,
            <given-names>C.A.</given-names>
          </string-name>
          and
          <string-name>
            <surname>Brass</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          <article-title>TAMBIS: Transparent Access to Multiple Bioinformatics Information Sources</article-title>
          . Bioinformatics,
          <volume>16</volume>
          (
          <issue>2</issue>
          ).
          <fpage>184</fpage>
          -
          <lpage>186</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref28">
        <mixed-citation>
          <string-name>
            <surname>Uschold</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          <article-title>Knowledge level modelling: concepts and terminology</article-title>
          .
          <source>The Knowledge Engineering Review</source>
          ,
          <volume>13</volume>
          (
          <issue>1</issue>
          ).
          <fpage>5</fpage>
          -
          <lpage>29</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref29">
        <mixed-citation>
          <string-name>
            <surname>Vrandecic</surname>
            ,
            <given-names>D.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Pinto</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Tempich</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          and
          <string-name>
            <surname>Sure</surname>
            ,
            <given-names>Y.</given-names>
          </string-name>
          <article-title>The DILIGENT knowledge processes</article-title>
          .
          <source>Journal of Knowledge Management</source>
          ,
          <volume>9</volume>
          (
          <issue>5</issue>
          ).
          <fpage>85</fpage>
          -
          <lpage>96</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref30">
        <mixed-citation>
          <string-name>
            <surname>Whetzel</surname>
            ,
            <given-names>P.L.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Parkinson</surname>
            ,
            <given-names>H.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Causton</surname>
            ,
            <given-names>H.C.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Fan</surname>
            ,
            <given-names>L.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Fostel</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Fragoso</surname>
            ,
            <given-names>G.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Game</surname>
            ,
            <given-names>L.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Heiskanen</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Morrison</surname>
            ,
            <given-names>N.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Rocca-Serra</surname>
            ,
            <given-names>P.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Sansone</surname>
            ,
            <given-names>S.-A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Taylor</surname>
          </string-name>
          , C.,
          <string-name>
            <surname>White</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          and
          <string-name>
            <surname>Stoeckert</surname>
            ,
            <given-names>C.J.</given-names>
          </string-name>
          ,
          <source>Jr</source>
          . The MGED Ontology:
          <article-title>a resource for semantics-based description of microarray experiments</article-title>
          .
          <source>Bioinformatics</source>
          ,
          <volume>22</volume>
          (
          <issue>7</issue>
          ).
          <fpage>866</fpage>
          -
          <lpage>873</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref31">
        <mixed-citation>
          <string-name>
            <surname>Wolstencroft</surname>
            ,
            <given-names>K.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Lord</surname>
            ,
            <given-names>P.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Tabernero</surname>
            ,
            <given-names>L.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Brass</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          and
          <string-name>
            <surname>Stevens</surname>
            ,
            <given-names>R.</given-names>
          </string-name>
          <article-title>Protein classification using ontology classification</article-title>
          .
          <source>Bioinformatics</source>
          ,
          <volume>22</volume>
          (
          <issue>14</issue>
          ).
          <fpage>e530</fpage>
          -
          <lpage>538</lpage>
          .
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>