<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta />
    <article-meta>
      <title-group>
        <article-title>Towards Modelling the Intended Purpose of Ontologies: A Case Study in Geography</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Ronald Denaux</string-name>
          <email>r.denaux@leeds.ac.uk</email>
          <xref ref-type="aff" rid="aff1">1</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Anthony G. Cohn</string-name>
          <xref ref-type="aff" rid="aff1">1</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Vania Dimitrova</string-name>
          <xref ref-type="aff" rid="aff1">1</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Glen Hart</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          <institution>Ordnance Survey Research</institution>
          ,
          <addr-line>Romsey Rd, Southampton, SO16 4GU</addr-line>
          ,
          <country country="UK">UK</country>
        </aff>
        <aff id="aff1">
          <label>1</label>
          <institution>School of Computing, University of Leeds</institution>
          ,
          <addr-line>Woodhouse Lane, Leeds, LS2 9JT</addr-line>
          ,
          <country country="UK">UK</country>
        </aff>
      </contrib-group>
      <abstract>
        <p>As the Semantic Web technologies gain popularity, more organisations are providing semantic resources related to the geographical domain but are doing this based on ontologies that represent their own conceptualisations and needs. As a result: nding, understanding and evaluating relevant semantic resources is still very time consuming, cognitively demanding and often require a deep understanding of the domain and the used semantic technologies. To help address this problem, we are investigating how ontology purposes can be captured and used to facilitate ontology engineering. The paper presents an exploratory study which examines whether it is possible to identify generic categories to model the purpose of geography-related ontologies. As an output, we present an initial categorisation of ontology purposes and discuss how these categories can help us understand and reuse ontologies.</p>
      </abstract>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>Introduction</title>
      <p>
        As the Semantic Web technologies become more mainstream, more people and
organisations are creating and publishing conceptualisations (using OWL) and
descriptions of their data (using RDF). In the spirit of the World Wide Web, the
technologies allow for each user to publish his own data and conceptualisations,
while making it easy to interlink these semantic resources with those of other
people. However, a big di erence between the WWW and the SW is that in the
WWW it is relatively easy to understand websites made by other people if you
share the same natural language. In the SW the tasks of nding, understanding
and evaluating relevant semantic resources are still very challenging. These tasks
are currently time consuming, cognitively demanding and often require a deep
understanding of the represented domain and the semantic technologies used.
Much research is nding ways to alleviate these problems by providing
semantic search [
        <xref ref-type="bibr" rid="ref1 ref2">1, 2</xref>
        ] and making it easier to understand ontologies by, for example,
identifying the main concepts [
        <xref ref-type="bibr" rid="ref3">3</xref>
        ], providing vocabularies to annotate them [
        <xref ref-type="bibr" rid="ref4">4</xref>
        ]
and studying their usage [
        <xref ref-type="bibr" rid="ref5">5</xref>
        ]. Our aim is to contribute to these research e orts
by studying the stated purpose of ontologies.
      </p>
      <p>
        This paper describes our rst investigations in the area of ontology purposes,
based on the geography domain. This domain presents a good use case for our
investigation as the problem of nding and correctly re-using suitable
semantic resources has already been reported [
        <xref ref-type="bibr" rid="ref6">6</xref>
        ]. These problems are partly caused
by a a common practice where di erent organisations are producing their own
geographic resources based on di erent conceptualisations and needs in reponse
to (i) a need for interlinking geographic data with many di erent domains (e.g.
agriculture, environment); (ii) the availability of large amounts of geographic
data and (iii) a relative ease of data production as most people have at least
some basic geographic knowledge3.
      </p>
      <p>
        The motivation for this work is to provide a high-level understanding of
semantic resources by using the stated purpose. This is in line with other bene ts
of purpose descriptions pointed out by prominent ontology construction
methodologies[
        <xref ref-type="bibr" rid="ref7 ref8">7, 8</xref>
        ], which regard the de nition of the ontology purpose as a fundamental
steps in the ontology development. The purpose helps to set the scope of the
ontology and to evaluate whether it is t for purpose. Furthermore, agreeing
on a common ontology purpose improves collaborative ontology development
by allowing ontology developers to refer to the stated ontology purpose to
resolve modeling issues [
        <xref ref-type="bibr" rid="ref9">9</xref>
        ]. Thus, having a way to describe and formally represent
ontology purposes could enable tool support for ontology construction.
      </p>
      <p>
        Despite the possible advantages, we are not aware of a suitable way to
formally de ne the purpose of ontologies. Where a purpose is de ned it is done
so as free text and rarely distributed with the ontology. Competency questions
(CQs) [
        <xref ref-type="bibr" rid="ref8">8</xref>
        ] |questions that the ontology is required to answer| have been
proposed to capture ontology requirements and tool support to formulate [
        <xref ref-type="bibr" rid="ref10">10</xref>
        ] them
is available. However, CQs (i) can only refer to entities that are de ned by the
ontology itself, while purpose descriptions often need to refer to entities outside
of the scope of the ontology in order to put the ontology into a wider context;
(ii) are not commonly distributed with the ontology; (iii) can only be formalised
as queries at a low granularity level, so the overall purpose of the ontology may
not be clear based on the total set of CQs (especially for large ontologies) and
(iv) cannot be organised in a hierarchy of purposes.
      </p>
      <p>This paper presents an exploratory study which examines whether it is
possible to identify generic categories to model the intended purpose of a corpus of
ontologies that use geographical concepts. To nd an answer to this question, we
obtained a corpus of semantic resources as described in Section 2. In Section 3,
we describe how the corpus was analysed in order to obtain a categorisation of
the ontology purposes. We discuss some ndings of our study in Section 4 and
nish with a description of our future work in Section 5.
2</p>
    </sec>
    <sec id="sec-2">
      <title>Corpus Collection</title>
      <p>To obtain a corpus of ontologies, we considered a common ontology engineering
scenario where a domain expert with limited knowledge engineering experience
is looking for a concept to reuse in a new ontology. In particular, we decided to
3 This is demonstrated by, for example, the OpenStreetMap project. http://www.</p>
      <p>openstreetmap.org/
use the concept River as this is a common concept which will ensure that we
nd a large number of ontologies that we can add to our corpus. Furthermore,
River is a central concept in the Ordnance Survey Hydrology Ontology4, which
we can use as a baseline ontology.
2.1</p>
      <sec id="sec-2-1">
        <title>Finding Semantic Resources to Create a Corpus</title>
        <p>
          In order to nd semantic resources based on our seed concept River, we
followed recent research that advocates using semantic web search engines to nd
ontologies published on the web instead of minting new concepts [
          <xref ref-type="bibr" rid="ref11">11</xref>
          ]. As a
result to our query for semantic resources based on River, Sindice [
          <xref ref-type="bibr" rid="ref2">2</xref>
          ] reports over
204 103 results, Watson [
          <xref ref-type="bibr" rid="ref11">11</xref>
          ] reports over 1500, while Swoogle [
          <xref ref-type="bibr" rid="ref1">1</xref>
          ] nds 302
results5. We also use used Google to nd semantic resources by restricting our
search to letype .owl (572 results).
        </p>
        <p>The large number of results for our query required further ltering until
we obtained ontologies de ning the concept River. Sindice found the biggest
number of semantic resources, but most of the results were RDF documents
describing instances from large datasets such as DBPedia and Geonames. For
example, http://dbpedia.org/resource/Parramatta_River and http://dbpedia.
org/resource/Cry_Me_a_River. Finding River de nitions (not instances) with
sindice is di cult because there is no ltering interface at sindice.com to
navigate through the search results.</p>
        <p>The results from Watson contained ontologies in a variety of formats (e.g.
RDF(S), OWL, DAML+OIL). Watson does not allow ltering or sorting of
search results either, so we browsed through the ontologies using the
ontology URIs and metadata such as the size of the ontology in Kilobytes, number
of classes, properties and individuals, type of ontology (e.g. RDF, OWL) and
OWL-DL expressivity (e.g. SHOIN ). This information aided in discovering
duplicates (the same ontology is hosted on di erent servers) or results which are
not useful such as foaf user pro les, describing locations(e.g. Fall River) and
interests (e.g. River Dancing).</p>
        <p>Swoogle's results are similar to Watson's with the added advantage that
Swoogle provides options for ranking the results based on ontoRank (their own
ranking system), date or triple.</p>
        <p>Google's results included several ontologies that contained a comment with
string River but did not de ne a River concept. Also, some les with extension
owl were not OWL les. Google results only provided text snippets and the
ontology URI, but do not include OWL-speci c metadata.
2.2</p>
      </sec>
      <sec id="sec-2-2">
        <title>Finding Ontologies with a Purpose Description</title>
        <p>Our study required that the ontologies in the corpus should have a purpose
description. Initially, we searched for ontologies that de ne a purpose annotation.
4 http://www.ordnancesurvey.co.uk/oswebsite/ontology/Hydrology/v2.0/</p>
        <p>Hydrology.owl
5 data gathered in June 2009
However, the number of ontologies that contained such an annotation was very
limited, so we decided to also collect generic descriptions about the ontology. In
the absence of a standard way of describing ontologies, we used three additional
sources of purpose descriptions; which we describe below.</p>
        <p>First, we searched the website hosting the ontology for an ontology purpose
description. In many cases, the website provided a description of the ontologies.
This description sometimes included an explaination on why the ontology had
been created which we used as a purpose description (e.g. the GWSG ontology).</p>
        <p>We also searched the serving website for a project purpose description. In
some cases, when the website served multiple ontologies, it did not provide a
description for each separate ontology (e.g. NASA SWEET). In these cases,
we looked for a more generic description of the website as a whole or of the
project that produced the ontology. If this generic description included a purpose
description|and it was clear that the ontology was created as a means to achieve
that purpose|, we included the description for the ontology.</p>
        <p>
          Finally, we searched for publications that explain a project in more detail.
We conducted this step when the website did not contain a suitable purpose
description, but was part of a project for which publications could be found. We
then searched the publications for descriptions of the ontology and how it was
used in the project. For example, the purpose description of the German Geo
KB, was gathered from (i) a description of the ontology in [
          <xref ref-type="bibr" rid="ref12">12</xref>
          ]; (ii) a description
on why the ontology was used [
          <xref ref-type="bibr" rid="ref12">12</xref>
          ] and (iii) a description of the project in [
          <xref ref-type="bibr" rid="ref13">13</xref>
          ].
2.3
        </p>
      </sec>
      <sec id="sec-2-3">
        <title>Corpus Description</title>
        <p>The corpus collected consists of 9 OWL ontologies with a free text description
of their intended purposes and the provenance of the purpose description (see
Table 1). The ontologies in the corpus range from small knowledge bases such
as German Geo KB to relatively large ontologies like NASA SWEET.</p>
        <p>
          The purpose descriptions in the corpus came from a variety of places, as
described above. In the best of cases, the authors provided a succinct description
of the ontology purpose (e.g. Ordnance Survey Hydrology Ontology in Table 2.
In the worst cases, the purpose of the ontology has to be inferred from documents
describing the project that created the ontology (e.g. [
          <xref ref-type="bibr" rid="ref14">14</xref>
          ] describes the BOEMIE
project objectives and includes sentences that hint at the purpose of the GIO
ontology (see Table 2).
3
        </p>
      </sec>
    </sec>
    <sec id="sec-3">
      <title>Purpose Analysis, Conceptualisation and Formalisation</title>
      <p>The presence of ontology (and project) descriptions that did not describe the
ontology purpose explicitly posed a problem: modeling these implicit purpose
descriptions was a subjective task that required an interpretation of the
documentation based on assumptions. In order to minimise this e ect and to keep track of
hydroseek 16 0 0 0 15 0 0
navigation
Cs = Class Count OPs = Object Property Count DP = Data Properties Count
Inds = Individual Count SubC = Subclass Axiom Count EqAx = Equivalence Axiom Count
DisAx = Disjoint Axiom Count Anns = Entity Annotation Count Ver. = Ontology Version
15
Description Source Purpose phrases Code
Purpose: To describe in an unambiguous manner Ontology Describe the inland hy- OS1
the inland hydrology feature classes surveyed by Annotation drology feature classes
surOrdnance Survey with the intention of improving of Ordnance veyed by Ordnance Survey
the use of the surveyed data by our customers Survey in an unambiguous
manand enabling semi-automatic processing of these Hydrology ner.
data. Ontology Enable semi-automatic OS3
processing of the data
surveyed by Ordnance
Survey.</p>
      <p>
        Driven by domain-speci c multimedia ontologies, BOEMIE Provide domain-speci c GIO1
BOEMIE information extraction systems will be project background knowledge
able to identify high-level semantic features in descrip- To use in information ex- GIO2
image, video, audio and text and fuse these tion [
        <xref ref-type="bibr" rid="ref14">14</xref>
        ] traction tasks in
multifeatures for optimal extraction. [...] The rationale media
behind this approach is that multimedia To enable identi cation of GIO3
information extraction can bene t in many ways high-level semantic feature
from the background knowledge provided by in image, video, audio and
ontologies [...] text
the assumptions made during the analysis of the ontology purpose descriptions,
we used a Grounded theory-based approach [
        <xref ref-type="bibr" rid="ref15">15</xref>
        ] for analysing the corpus.
Figure 1 shows an overview of the steps taken during the analysis to arrive at a
conceptualisation.
      </p>
      <p>As a rst step towards conceptualistion,
we identify and extract purpose phrases
based on the ontology descriptions in
the corpus. These are simple statements
that describe a single goal that the
ontology should help to achieve. To
compose the purpose phrases, sections of free
text descriptions have to be rephrased
(we try to stay as close to the
original text as possible to minimise the
introducing assumptions). As an
example, Table 2 shows some purpose phrases
based on purpose descriptions. The
table also shows that each purpose phrase
is linked to a unique code that links the
purpose phrase to its originating
ontology and free text description. Due to
space limitations we cannot show all the
purpose phrases6.</p>
      <p>
        The next step in the analysis is to Fig. 1. Overview of steps to obtain an
make sure that the purpose phrases have initial ontology of purposes for
geoa common structure that represents an graphic ontologies based on free text
ontology purpose. This structure has purpose descriptions
been inspired by existing work
describing goals in the medical context [
        <xref ref-type="bibr" rid="ref16">16</xref>
        ].
Although there is no standard terminology for describing goals, [
        <xref ref-type="bibr" rid="ref16">16</xref>
        ] states that
most goals can be decomposed into a tuple consisting of a task and a focus. This
tuple can be extended with optional components such as restrictions, a situation
description, a policy de nition (e.g. whether goal is obligatory) and a rationale
(i.e. clari cation of the goal). For our purpose phrases (i) the situation
description is always the same: the creation of an ontology; (ii) no policy de nition is
necessary and (iii) the rationale is the purpose description source. So we can
decompose our purpose phrases (e.g. OS3 in Table 2) into a task (e.g. Enable), a
focus (e.g. semi-automatic processing of data) and zero or more restrictions(e.g.
the data has been surveyed by Ordnance Survey ).
      </p>
      <p>The task is the verb phrase at the beginning of each purpose phrase. For
example, the task for GIO1 is Provide and the task for OS3 is enable (see Table 2).
De ning a single best focus and a set of restrictions is not always easy. For
6 At the moment of writing we have identi ed 63 purpose phrases. A
full table can be found at http://spreadsheets.google.com/pub?key=
tkeezO29qTGOVZ1Rd41JbTw&amp;gid=2
purpose phrase OS3, the focus can be semi-automatic processing of data (plus
some restrictions), or it can be processing of data (where the fact that this
processing is semi-automatic is a restriction)7.</p>
      <p>We found generic concepts to use in our categorisation by making the focus of
each purpose phrase as abstract as possible and expressing as many restrictions
as possible. The restrictions of the purpose phrases de ne specialisations of the
generic concepts in the categorisation. After re ning these generic categories we
arrived at 7 types of purposes:
1. Domain De ning: these purposes occur when there is a need to
conceptualise a domain. They usually restrict the scope of the ontology and may
also impose restrictions on how concepts are represented. An example of this
is gwsg7:Find an ontological consistent description of the qualities of a data
set of observations.
2. Ontology process related: these purposes state that the ontology should
in uence (enable, improve, etc.) ontology processes such as the use, re-use,
merging and alignment of ontologies. An example: seres3:Enable
comparison between conceptualisations.
3. Data process related: the ontology in uences a data process: for example
the creation, editing, navigation, annotation and publishing of data. These
type of purposes also can impose restrictions on the scope of the domain,
but di er from Domain De ning and Ontology process related purposes in
that the focus of the purpose lies on the data that can be represented by the
ontology instead of the conceptualisation itself. An example is EnvO3:support
the annotation of the environment of any organism or biological sample.
4. Investigative: when the ontology is created to study a system (e.g. GermanGeo1
Use as a knowledge base to study the ORAKEL system ), the Ontology
Engineering process itself (e.g. gwsg5:Investigate how to convert the German
classi cation schema for ecological assessment of watercourse structure into
a DOLCE-aligned domain ontology ) or speci c ways to formulate and encode
knowledge (e.g. gwsg4:Investigate the possible combinations of basic qualities
into complex qualities ).
5. Collaboration enhancing: when the ontology should enhance social
processes. As social processes may include ontology and data related processes,
this category can overlap those two categories. For example: Pont9:Enabling
collaborative discussion of ontologies by practitioners.
6. External application: are purposes where an ontology is used by an
external application to perform a task. E.g. GIO2:Optimise information extraction
tasks in multi-media.
7. Analogous: these are purposes that refer to a similar ontology in a di erent
domain that is being emulated. E.g. EnvO1:Provide similar bene ts as the
Gene Ontology.
7 All the purpose phrase decompositions can be found at http://spreadsheets.</p>
      <p>google.com/pub?key=tkeezO29qTGOVZ1Rd41JbTw&amp;gid=3
Formalisation A lightweight ontology has been created based on the shown
conceptualisation. This ontology is currently only suitable for annotating
purpose descriptions. In the future we would like to produce an ontology that enables
authors to easily formalise their purpose for an ontology and enables services such
as purpose comparison and classi cation.
3.1</p>
      <sec id="sec-3-1">
        <title>Application</title>
        <p>Currently, we are planning two extensions to this study where we investigate
whether we can apply the model of ontology purposes to compare the ontologies
in the corpus. In the rst study extension we will study whether we can use the
purpose categories to determine the similarity between two ontology purposes.
This similarity can then be used to compare the ontologies. For example, our
corpus analysis shows that the NASA SWEET ontology states its purpose in
terms of ontology engineering, community bene ts and domain description while
ignoring data processes such as data creation, publishing or annotations. The
Hydroseek navigation ontology has an opposite approach where the purpose
focuses on data search, mapping and navigation, while not much is said about
ontology engineering.</p>
        <p>The second study extension will investigate whether we can nd a
correlation between ontology metrics and ontology purpose. The intuition is that the
stated purpose of the ontology should have an impact on the design decisions
made when building the ontology, so there might be a relation between ontology
characteristics and ontology purpose. For example, an ontology that has as a
Data process related purpose (e.g. German Geo KB), may be more likely to
provide instances of the modelled concepts than an ontology that does not specify
this type of purpose. Another example is the Hydroseek Navigation ontology,
that aims to make navigation tasks through data easier. This purpose may be
re ected in its small number of concepts and shallow taxonomic depth.
4</p>
      </sec>
    </sec>
    <sec id="sec-4">
      <title>Discussion</title>
      <p>The categorisation identi ed in our study suggests that it is possible to identify
generic categories to model the purpose of geographical ontologies. In fact, the
presented categorisation does not rely on geography speci c terminology, which
suggest that the model could be applicable to ontologies outside of the geographic
domain. Indeed, the corpus already contains ontologies that are not strictly
geographical such as E-response(emergencies).</p>
      <p>The dependence on the geographic domain is only apparent when we look
at the set of restrictions, where we nd geographic domain restriction (e.g
restriction for wow5 to geographic objects valuable in tourism ) and data restrictions
that are strongly related to the geographical domain (e.g wow4:Allow automatic
creation of multilingual hiking path descriptions ).</p>
      <p>An unexpected result of this study is that geography-speci c external
processes were not found: ontologies were built to aid information extraction in
multi-media and to study semantic reference systems, but none of the geographic
ontologies mentions speci c usages in geography related tasks. The OS
Hydrology ontology has a generic data-process (use of the surveyed data by Ordnance
Survey's customers ). NASA SWEET does not mention how the data described
by the ontology will be used and only refers to the conceptualisation in terms of
ontology engineering tasks such as evolution and alignment.</p>
      <p>
        A clari cation for the small number of speci c geography-related processes
may be the relatively small corpus of ontologies and the di culty of nding
good purpose descriptions of the ontologies. Another clari cation may follow
the argument by C.M. Keet [
        <xref ref-type="bibr" rid="ref17">17</xref>
        ] who says that many ontology authors do not
de ne a particular purpose for an ontology because their ultimate aim is to build
an ontology that is application independent|even when the ontology is initially
built for a particular purpose. This may be the case in the geographic domain,
when ontologies encode well-established classi cation schemas that already are
used by many applications (e.g. OS Hydrology ontology), so that ennumerating
the speci c use cases is not desireable.
      </p>
      <p>While the presented study suggests that it is possible to nd generic
categories of ontology purposes, the approach that we used has several limitations:
(i) the introduction of subjective interpretation of purpose descriptions; (ii) only
a subset of the geographical ontologies was used as we restricted ourselves to
ontologies that de ned a single concept (i.e. we missed ontologies that de ne
geographical features, but do not de ne the River concept); (iii) nding
appropriate purpose descriptions for ontologies was time consuming; so that, even when
a suitable ontology is found, no purpose description can be found by searching
the (semantic) web (e.g. the Mooney ontology of geographic data8).
5</p>
    </sec>
    <sec id="sec-5">
      <title>Future Work</title>
      <p>To address the limitations of the current study, we hope to be able to
contact some of the authors of these ontologies to (i) verify the ontology purpose,
(ii) get feedback on their personal goals when constructing the ontology, (iii)
validate and improve the vocabulary to describe ontology purposes and (iv)
investigate whether providing the vocabulary helps to make ontology purposes
explicit, which were previously implicit and not published.</p>
      <p>Further future work include (i) investigating whether there are links between
stated ontology purposes and features of the ontology such as number of classes
and ontology expressivity; (ii) designing and investigating the e ect of tool
support based on ontology purposes; (iii) researching ways to represent the ontology
purpose and the contributor's goals to be able to model con icts of interests to
support multi-perspective ontology development; (iv) investigate the link
between ontology purpose and usage; speci cally, whether we can design a shared
vocabulary to describe both purpose and usage of an ontology and whether we
can use this vocabulary to evaluate the tness-for-purpose of an ontology.
8 http://www.ifi.uzh.ch/ddis/fileadmin/ont/nli/geography.owl</p>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          1.
          <string-name>
            <surname>Ding</surname>
            ,
            <given-names>L.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Finin</surname>
            ,
            <given-names>T.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Joshi</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Pan</surname>
            ,
            <given-names>R.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Cost</surname>
            ,
            <given-names>R.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Peng</surname>
            ,
            <given-names>Y.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Reddivari</surname>
            ,
            <given-names>P.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Doshi</surname>
            ,
            <given-names>V.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Sachs</surname>
          </string-name>
          , J.:
          <article-title>Swoogle: A search and metadata engine for the semantic web</article-title>
          .
          <source>In: Proceedings of the 13th International Conference on Information and Knowledge Management</source>
          ,
          <string-name>
            <surname>ACM</surname>
          </string-name>
          (
          <year>2004</year>
          )
          <volume>652</volume>
          {
          <fpage>659</fpage>
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          2.
          <string-name>
            <surname>Oren</surname>
            ,
            <given-names>E.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Delbru</surname>
            ,
            <given-names>R.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Catasta</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Cyganiak</surname>
            ,
            <given-names>R.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Stenzhorn</surname>
            ,
            <given-names>H.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Tummarello</surname>
          </string-name>
          , G.:
          <article-title>Sindice. com: A document-oriented lookup index for open linked data</article-title>
          .
          <source>International Journal of Metadata, Semantics and Ontologies</source>
          <volume>3</volume>
          (
          <issue>1</issue>
          ) (
          <year>2008</year>
          )
          <volume>37</volume>
          {
          <fpage>52</fpage>
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          3.
          <string-name>
            <surname>Peroni</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Motta</surname>
          </string-name>
          , E.,
          <string-name>
            <surname>d'Aquin</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          :
          <article-title>Identifying key concepts in an ontology, through the integration of cognitive principles with statistical and topological measures</article-title>
          .
          <source>In: Proceedings of the 3rd Asian Semantic Web Conference</source>
          . (
          <year>2008</year>
          )
          <volume>242</volume>
          {
          <fpage>256</fpage>
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          4.
          <string-name>
            <surname>Hartmann</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Sure</surname>
            ,
            <given-names>Y.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Haase</surname>
            ,
            <given-names>P.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Palma</surname>
            ,
            <given-names>R.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Suarez-Figueroa</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          :
          <article-title>Omv{ontology metadata vocabulary</article-title>
          .
          <source>In: Workshop on Ontology Patterns for the Semantic Web at ISWC2005</source>
          . (
          <year>2005</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          5.
          <string-name>
            <surname>Alberts</surname>
            ,
            <given-names>R.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Calvanese</surname>
            ,
            <given-names>D.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Giacomo</surname>
            ,
            <given-names>G.D.</given-names>
          </string-name>
          , et al.:
          <article-title>Analysis of test results on usage scenarios</article-title>
          .
          <source>Deliverable D27</source>
          ,
          <fpage>FP6</fpage>
          -7603
          <string-name>
            <surname>Thinking ONtologiES (TONES)</surname>
          </string-name>
          (
          <year>2008</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          6.
          <string-name>
            <surname>Goodwin</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Dolbear</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Hart</surname>
          </string-name>
          , G.:
          <article-title>Geographical linked data: The administrative geography of great britain on the semantic web</article-title>
          .
          <source>Transactions in GIS 12(s1)</source>
          (
          <year>2008</year>
          )
          <volume>19</volume>
          {
          <fpage>30</fpage>
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          7.
          <string-name>
            <surname>Fernandez-Lopez</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Gomez-Perez</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Juristo</surname>
          </string-name>
          , N.:
          <article-title>Methontology: from ontological art towards ontological engineering</article-title>
          .
          <source>In: Proceedings of the AAAI97 Spring Symposium Series on Ontological Engineering</source>
          . (
          <year>1997</year>
          )
          <volume>33</volume>
          {
          <fpage>40</fpage>
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          8.
          <string-name>
            <surname>Uschold</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>King</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          :
          <article-title>Towards a methodology for building ontologies</article-title>
          .
          <source>In: IJCAI95 Workshop on Basic Ontological Issues in Knowledge Sharing</source>
          , Montreal (
          <year>1995</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          9.
          <string-name>
            <surname>Dellschaft</surname>
            ,
            <given-names>K.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Engelbrecht</surname>
            ,
            <given-names>H.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Barreto</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Rutenbeck</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Staab</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          : Cicero:
          <article-title>Tracking design rationale in collaborative ontology engineering</article-title>
          .
          <source>In: Proceedings of the 5th European Semantic Web Conference</source>
          , Springer (
          <year>2008</year>
          )
          <volume>782</volume>
          {
          <fpage>787</fpage>
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          10.
          <string-name>
            <surname>Sure</surname>
            ,
            <given-names>Y.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Erdmann</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Angele</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Staab</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Studer</surname>
            ,
            <given-names>R.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Wenke</surname>
            ,
            <given-names>D.</given-names>
          </string-name>
          : Ontoedit:
          <article-title>Collaborative ontology development for the semantic web</article-title>
          .
          <source>In: Proceedings of the International Semantic Web Conference</source>
          , Springer (
          <year>2002</year>
          )
          <volume>221</volume>
          {
          <fpage>235</fpage>
        </mixed-citation>
      </ref>
      <ref id="ref11">
        <mixed-citation>
          11.
          <string-name>
            <surname>d'Aquin</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Motta</surname>
            ,
            <given-names>E.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Dzbor</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Gridinoc</surname>
            ,
            <given-names>L.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Heath</surname>
            ,
            <given-names>T.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Sabou</surname>
            ,
            <given-names>M.:</given-names>
          </string-name>
          <article-title>Collaborative semantic authoring</article-title>
          .
          <source>IEEE Intelligent Systems</source>
          <volume>23</volume>
          (
          <issue>3</issue>
          ) (
          <year>2008</year>
          )
          <volume>80</volume>
          {
          <fpage>83</fpage>
        </mixed-citation>
      </ref>
      <ref id="ref12">
        <mixed-citation>
          12.
          <string-name>
            <surname>Cimiano</surname>
            ,
            <given-names>P.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Haase</surname>
            ,
            <given-names>P.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Heizmann</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Mantel</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Studer</surname>
          </string-name>
          , R.:
          <article-title>Towards portable natural language interfaces to knowledge bases { the case of the orakel system</article-title>
          .
          <source>Data &amp; Knowledge Engineering</source>
          <volume>65</volume>
          (
          <issue>2</issue>
          ) (
          <year>2008</year>
          )
          <volume>325</volume>
          {
          <fpage>354</fpage>
        </mixed-citation>
      </ref>
      <ref id="ref13">
        <mixed-citation>
          13.
          <string-name>
            <surname>Cimiano</surname>
            ,
            <given-names>P.:</given-names>
          </string-name>
          <article-title>The orakel natural language interface (</article-title>
          <year>2009</year>
          ) http://www.aifb. uni-karlsruhe.de/WBS/pci/home/doku.php?id=orakel accessed on 29/07/
          <year>2009</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref14">
        <mixed-citation>
          14.
          <string-name>
            <surname>Spyropoulos</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Paliouras</surname>
            ,
            <given-names>G.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Tsakonas</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          :
          <article-title>Project description and website</article-title>
          .
          <source>Deliverable D1</source>
          .1,
          <string-name>
            <given-names>Bootstrapping</given-names>
            <surname>Ontology</surname>
          </string-name>
          <article-title>Evolution with Multimedia Information Extraction (</article-title>
          <year>2006</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref15">
        <mixed-citation>
          15.
          <string-name>
            <surname>Strauss</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Corbin</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Lynch</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          :
          <article-title>Basics of qualitative research: Grounded theory procedures and techniques</article-title>
          . Sage Newbury Park, CA (
          <year>1991</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref16">
        <mixed-citation>
          16.
          <string-name>
            <surname>Fox</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Alabassi</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Patkar</surname>
            ,
            <given-names>V.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Rose</surname>
            ,
            <given-names>T.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Black</surname>
            ,
            <given-names>E.</given-names>
          </string-name>
          :
          <article-title>An ontological approach to modelling tasks and goals</article-title>
          .
          <source>Computers in Biology and Medicine</source>
          <volume>36</volume>
          (
          <issue>7-8</issue>
          ) (
          <year>2006</year>
          )
          <volume>837</volume>
          {
          <fpage>856</fpage>
        </mixed-citation>
      </ref>
      <ref id="ref17">
        <mixed-citation>
          17.
          <string-name>
            <surname>Keet</surname>
            ,
            <given-names>C.M.</given-names>
          </string-name>
          :
          <article-title>Ontology design parameters for aligning agri-informatics with the semantic web</article-title>
          .
          <source>In: Proceedings of 3rd International Conference on Metadata and Semantic</source>
          , Milan, Italy (
          <year>2009</year>
          ) to be published.
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>