<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta />
    <article-meta>
      <title-group>
        <article-title>Ontology Maturing: a Collaborative Web 2.0 Approach to Ontology Engineering</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Simone Braun</string-name>
          <email>braun@fzi.de</email>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Andreas Schmidt</string-name>
          <email>aschmidt@fzi.de</email>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Andreas Walter</string-name>
          <email>awalter@fzi.de</email>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          <institution>FZI Research Center for, Information Technologies</institution>
          ,
          <addr-line>Karlsruhe</addr-line>
          ,
          <country country="DE">Germany</country>
        </aff>
      </contrib-group>
      <pub-date>
        <year>2007</year>
      </pub-date>
      <fpage>8</fpage>
      <lpage>12</lpage>
      <abstract>
        <p>Most of the current methodologies for building ontologies rely on specialized knowledge engineers. This is in contrast to real-world settings, where the need for maintenance of domain specific ontologies emerges in the daily work of users. But in order to allow for participatory ontology engineering, we need to have a more realistic conceptual model of how ontologies develop in the real world. We introduce the ontology maturing processes which is based on the insight that ontology engineering is a collaborative informal learning process and for which we analyze characteristic evolution steps and triggers that have users engage in ontology engineering within their everyday work processes. This model integrates tagging and folksonomies with formal ontologies and shows maturing pathways between them. As implementations of this model, we present two case studies and the corresponding tools. The first is about image-based ontology engineering (introducing so-called imagenotions), the second about ontology-enabled social bookmarking (SOBOLEO). Both of them are inspired by lightweight Web 2.0 approaches and allow for realtime collaboration.</p>
      </abstract>
      <kwd-group>
        <kwd>Ontology engineering</kwd>
        <kwd>ontology maturing</kwd>
        <kwd>Web 2</kwd>
        <kwd>0</kwd>
        <kwd>tagging</kwd>
        <kwd>social software</kwd>
        <kwd>work integration</kwd>
        <kwd>collaboration</kwd>
      </kwd-group>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>Categories and Subject Descriptors</title>
      <p>H.1.m [Miscellaneous]; H.5.3 [Group and Organization
Interfaces]; I.2.4 [Knowledge Representation
Formalisms and Methods]: Semantic networks; K.4.3
[Organizational Impacts]: Computer-supported collaborative
work</p>
    </sec>
    <sec id="sec-2">
      <title>1. INTRODUCTION</title>
      <p>Within state-of-the-art semantic approaches, ontologies
have emerged as the key to enable more advanced
technological support for end users and their work processes, which
particularly applies to knowledge work. However, current
research and development concentrates more on what we
can do as soon as we have ontologies—rather than having a
closer look at the processes of creating and especially
maintaining such domain-specific ontologies. In real-world
settings these issues are crucial to fulfill the users’ needs and
currently insufficiently dealt with.</p>
      <p>It is usually acknowledged that ontologies are shared
understandings of a particular domain that have to be
constructed within social processes among the stakeholders.</p>
      <p>However, current methods and tools do not empower the
users to actually carry out these negotiation processes
embedded in their work.</p>
      <p>
        This is mainly due to lopsided and partially naive
perspectives on ontology engineering. One perspective requires
ontology engineering experts who moderate the ontology
creation processes and on whom users will depend indefinitely.
In the other perspective, it is assumed that users become
ontology modeling experts right away. Presupposing an
ontology engineering expert, we have to face two problems:
First, involving ontology engineering specialists is expensive
and second, ontologies produced by modeling experts
instead of domain experts can contain errors caused by the
insufficient domain knowledge and experience of knowledge
engineers [
        <xref ref-type="bibr" rid="ref2">2</xref>
        ]. Leaving the complete, challenging and
complex task of ontology modeling to the domain experts on the
other hand is usually not an option either! Often users are
not motivated to invest the effort because they are concerned
with their work processes and regard ontology modeling in
its traditional form as an overhead. The main reason
therefore is the fact that the time lag between the emergence of
concepts and their inclusion in ontologies is far too big for
ontologies to be useful [
        <xref ref-type="bibr" rid="ref14">14</xref>
        ]: concepts are already becoming
obsolete by the time they are entering the ontology.
      </p>
      <p>Indeed, users are almost constantly constructing and
negotiating shared meaning in collaboration with others by
augmenting and evolving a community vocabulary. The
main challenge is then how to leverage this implicit and
informal ontology building for the explicit formal models
needed for semantic approaches. What we actually need in
order to cope with this challenge are two things: First, on
the conceptual level: a more realistic and work-integrated
view of how ontologies actually are or can be created and
second, on the technical level: tools supporting interweaving
working and ontology engineering activities and the
associated social negotiation process.</p>
      <p>This paper is organized as follows: Section 2 discusses
related work. In section 3, we present our notion of ontology
maturing as a conceptual model and analyze the
motivation for users, to participate on such collaborative maturing
processes. We present two case studies in section 4 and 5—
they show how this conceptual model in combination with
lightweight and collaborative Web 2.0 tools enables and
fosters the maturing process. Section 6 concludes.</p>
    </sec>
    <sec id="sec-3">
      <title>RELATED WORK</title>
      <p>
        There are several proposed methodologies for the process
of ontology engineering that require ontology engineers as
moderators. These methodologies define how to support
the ontology lifecycle from development, via evaluation and
maintenance, to further evolution. An overview is given in
[
        <xref ref-type="bibr" rid="ref7">7</xref>
        ] and [
        <xref ref-type="bibr" rid="ref10">10</xref>
        ]. They do not allow for the work-embedded and
collaborative engineering of ontologies
      </p>
      <p>
        A more “human-centered approach” is taken by [
        <xref ref-type="bibr" rid="ref17">17</xref>
        ] with
the Human-Centered Ontology Engineering Methodology
(HCOME). Kotis et al. [
        <xref ref-type="bibr" rid="ref17">17</xref>
        ] view ontology development as
a dynamic process and focus particularly on ontology
evolution. They assume a decentralized engineering model where
everyone first formalizes her own ontology and shares it in
a further step within the community. There, the
individual ontologies are merged or further developed. However,
findings in [
        <xref ref-type="bibr" rid="ref1">1</xref>
        ] (based on action theory) suggest that
collaboration plays a more important role before we have
formalized (individual) ontologies. So we think that the HCOME
methodology can benefit from incorporating the notion of
different maturity levels. This methodology does not
support our goal of embedding ontology engineering of
ontologies in work processes.
      </p>
      <p>
        Editing tools like Prot´eg´e [
        <xref ref-type="bibr" rid="ref22">22</xref>
        ] or KAON OIModeler [
        <xref ref-type="bibr" rid="ref19">19</xref>
        ]
are commonly used for ontology building ([
        <xref ref-type="bibr" rid="ref21">21</xref>
        ] and [
        <xref ref-type="bibr" rid="ref6">6</xref>
        ]
discuss them in detail). Most existing ontology editors are
standalone desktop applications. In addition, they
consider ontology construction as an isolated and detached task
where ontology engineering experts explicitly sit together
with knowledge workers to model the ontologies. They do
not provide a collaborative environment or support
collaboration only in a restricted way (except for KAON). Like the
above mentioned methodologies, these tools are not geared
towards knowledge workers and their work processes.
      </p>
      <p>
        Wiki systems consider the aspects of collaboration and
can support the early phases of ontology construction.
Semantic wiki systems1 try to extend the traditional wikis with
semantic web technologies (e.g. Semantic MediaWiki [
        <xref ref-type="bibr" rid="ref28">28</xref>
        ],
OntoWiki [
        <xref ref-type="bibr" rid="ref15">15</xref>
        ] or IkeWiki [
        <xref ref-type="bibr" rid="ref23">23</xref>
        ]). These systems help users
in creating definitions, e.g. beginning with informal texts.
Because of discussion pages and versioning for each article
they are suitable for complex coordination and
consolidation processes, but they are usually more time-consuming
and lack work process integration.
      </p>
      <p>
        Collaborative tagging systems form a third application
group that is getting more and more popular on the web.
Their lightweight approach allows users to easily assign
keywords to various contents. The social bookmarking systems
BibSonomy ([
        <xref ref-type="bibr" rid="ref16">16</xref>
        ]) and Del.icio.us [
        <xref ref-type="bibr" rid="ref5">5</xref>
        ] further allow grouping
of tags underneath a super-tag. However, they do not define
a clear concept structure and this functionality is very
restricted. For instance these applications do not display the
tree structure formed by these relations and the structure
on top of tags is not shared.
      </p>
      <p>
        There were first attempts to analyze occurrence patterns
and in particular the usage of tags within collaborative
tag1See website of the SemanticWiki Interest Group for
updated list of existing semantic wiki systems [
        <xref ref-type="bibr" rid="ref12">12</xref>
        ]
ging systems using the example of Del.icio.us [
        <xref ref-type="bibr" rid="ref5">5</xref>
        ] and Flickr
[
        <xref ref-type="bibr" rid="ref8">8</xref>
        ]. Golder &amp; Huberman [
        <xref ref-type="bibr" rid="ref9">9</xref>
        ] observed that tags used for
an individual resource stabilize over time. Marlow et al.
[
        <xref ref-type="bibr" rid="ref20">20</xref>
        ] showed that the tag vocabularies of socially connected
users have a bigger overlap than those of randomly selected
users. Both might be traced back to issues of imitation and
an upcoming shared knowledge. Sen et al. further
investigated the community impact on personal vocabulary and
tagging behavior [
        <xref ref-type="bibr" rid="ref25">25</xref>
        ]. They explored different forms of
selecting tags used in the community. They observed that
users apply (“borrow”) an already used tag more often in
case tags of community members compared to when no tags
are displayed. Our model in the next chapter considers these
observations as a change from ideas to consolidation in
communities.
3.
      </p>
    </sec>
    <sec id="sec-4">
      <title>THE ONTOLOGY MATURING PROCESS</title>
      <p>In the following, we will present the steps of our model
and the motivation and triggers for user to participate on
the ontology maturing progress.
3.1</p>
    </sec>
    <sec id="sec-5">
      <title>Observations about ontologies</title>
      <p>At first, we will collect some important observations about
ontologies, based on our experiences, that help in developing
a more appropriate model for building ontologies:
• Ontology building as a learning process. Key is
the insight that building ontologies is not just about
eliciting knowledge and formalizing it according to a
particular formalism. Rather this construction process
itself is a learning process in which the involved
individuals deepen their understanding of the real world
and of an (appropriate) vocabulary to describe it. This
is especially true if we consider emerging ideas: it is
simply not possible to integrate these into an ontology
straight away.
• Formality and complexity of use as barrier.
Related to that learning process, we should also consider
the lessons learnt from the success of Web 2.0
approaches. Web 2.0 approaches empower the individual
to take part in community activities by lowering the
barriers: informal, lightweight, easy-to-use, and
easyto-understand. Tagging as an organizing paradigm has
replaced folder hierarchies. As a conclusion, we should
acknowledge that the degree of formality poses a
barrier. And that the compelling simplicity of Web 2.0
applications is part of their success model.
• Continuous evolution in work processes.
Ontology building is usually not supposed to be a one-time
activity of an expert committee, but rather a
sustainable process of continuous evolution. But if we
cannot assume that everything is on the same formality
level, we need to acknowledge that in a living
ontology there are most likely concepts not clearly defined
yet. They underlie a process of continuous evolution
where ideas and understanding emerge implicitly in
daily work and mature only gradually through the
interaction with others to explicit formal and shared
conceptualizations.
new concept ideas
tags</p>
      <p>common
terminology</p>
      <p>formal
lightweight
ontology
heavy-weight
ontology</p>
      <p>
        These observations are similar to those made in [
        <xref ref-type="bibr" rid="ref24">24</xref>
        ] about
how new ideas develop in the context of knowledge
management and e-learning to become reusable training
material. This development process was described with the
metaphor of maturing and structured into five phases as
the so-called knowledge maturing process. This process is
viewed as a macro model for interconnected individual
learning processes. Based on this process model, we have come
up with a similar phase model that identifies characteristical
maturing transitions in collaboratively developing a shared
ontology (see Figure 1):
1. Emergence of ideas. In this initial transition, new
concept ideas are introduced which are rather ad hoc
and not well-defined. They are personal “utterances”
which are informally communicated and technically
typically represented by tags. For instance, while
annotating or seeking for resources, we recognize that
the tag we want to use does not exist or is misspelled.
Accordingly, we introduce a new tag or correct the
existing one without further reflecting.
2. Consolidation in Communities. Through reuse
and adaption of concept symbols (tags) of others, a
shared vocabulary emerges within a community. When
comparing currently envisioned tags with previously
used ones or with tags from other people assigned to
the same resource, we discover similarities and
differences that allow for creating concepts from tags. For
instance, we realize that we can improve our search by
using a synonymous term. We establish a link between
these terms in our understanding and thus can merge
synonyms into concepts. Moreover, preferred labels
also often develop in the same way [
        <xref ref-type="bibr" rid="ref9">9</xref>
        ]. It crystallizes
a common terminology still without formal semantics.
3. Formalization. Within the third phase, concepts are
organized into relations - both taxonomical
(hierarchical) and ad hoc relations. For instance, we want to
receive recommendations from the system and we
become aware of different abstraction levels (e.g. broader
and narrower) that we need to have in order to find
e.g. spaghetti and bigoli (thicker spaghetti) as noodles
and vice versa. The results are lightweight ontologies
that rely primarily on inferencing based on subconcept
relations.
4. Axiomatization. The last phase of our model
captures more domain semantics by adding background
knowledge for improving inferencing processes, e.g. for
query answering. This step requires a high level of
competence in logical formalism so that this can
usually only be carried out by domain experts.
3.3
      </p>
    </sec>
    <sec id="sec-6">
      <title>Motivation and Triggers for Maturing Activities</title>
      <p>This model as such only describes characteristic degrees
of formality of ontology elements from a user-driven point of
view. But it does not explain yet, how this integrates with
actual work processes.</p>
      <p>When and how do users engage in maturing activities?
What triggers the maturing of concepts and other ontology
elements? And why do we believe that users are motivated
to participate in this collaborative maturing effort?</p>
      <p>
        Starting point for our analysis is the observation that
ontologies are mainly used and deliver their main benefit for
information seeking or distribution activities. This includes
users wanting to:
• retrieve appropriate content
• promote their content so that it is found by others
• enhance their work performance (getting answers
instead of searching)
• be empowered by being able to state their opinions
This coincides with motivations for tagging resources in
[
        <xref ref-type="bibr" rid="ref20">20</xref>
        ]: Future retrieval, Contribution and Sharing, Attract
Attention, Play and Competition, Self Presentation, and
Opinion Expression.
      </p>
      <p>The embedding of ontology maturing into users work
processes also requires to have a look at the micro level of
concrete triggers initiating these activities. These triggers are
important to consider because they are the technical
integration points where we need to make the transition between
working and ontology maturing possible.</p>
      <p>
        If we consider the retrieval of appropriate content, we
can identify following typical triggers which are closely
connected to semantic shortcomings of tagging systems (cf. [
        <xref ref-type="bibr" rid="ref9">9</xref>
        ],
[
        <xref ref-type="bibr" rid="ref13">13</xref>
        ]):
• (Mis-)Spelling. The most obvious problem is that
tags were simply misspelled or that there are
typographical differences, for instance because of occurring
plurals, abbreviations or compound words. Even
inexperienced users tend to correct such issues if possible.
• Synonymy. Documents are not found with the first
keyword, but later under a synonym. Similar is the
issue for the author of a resource trying to promote
its usage: she has to use many synonyms in order to
ensure that other users will find it later on. This
typically leads users to abstract from keywords to concepts
with several synonyms (possibly also in multiple
languages).
• Multilingualism. Tags are always in one language.
      </p>
      <p>
        This requires that content owners, especially in
Europe with many different languages, have to describe
resources with many tags in different languages.
• Homonymy. A tag can have different meanings, so
called homonyms. This leads to search results with low
precision, as all resources that are relevant to these
different meanings are annotated with the same tag.
Fixing this issue is more demanding as it requires to
differentiate existing tags into different concepts. Its
effects can only be realized if resources are reindexed,
which often is not manageable.
• Missing concepts. During annotation activities, users
discover that a topic they want to describe is not yet
covered. This leads to additions to the ontology.
• Mismatch of abstraction level. A typical search
problem in complex domains is that search terms are
specified either too broad or too narrow. This
problem, also known as the “basic level phenomenon” [
        <xref ref-type="bibr" rid="ref27">27</xref>
        ],
can be traced back to different intentions and
expertise levels of the users. For instance, one user tags a
resource on the basic level with “spaghetti”, another
with “noodles” and a third differentiates “spaghetti”
from “bigoli” (thicker spaghetti) and “vermicelli”
(thinner spaghetti). Thus, later success in finding leads to
the adding of hierarchical relationships.
• Missing guidance to related information.
Especially the orientation phase of information seeking
processes (like [
        <xref ref-type="bibr" rid="ref18">18</xref>
        ] described) requires guidance to find
related fields. As soon as implicit links are discovered,
they can be added to the ontology. An increasing
number of links (at the latest) leads to defining new
properties.
3.4
      </p>
    </sec>
    <sec id="sec-7">
      <title>Tool support</title>
      <p>In the following two sections, we present two approaches
that support the collaborative maturing process of
ontologies based on our introduced model and the relevant triggers.
Both provide tool support in work integrated scenarios.</p>
      <p>The first case study is rooted in the domain of semantic
image retrieval. Here, the focus lies on the consolidation
of image descriptions in communities with the help of
visualized concepts and relations. The case stuy introduces a
new method called imagenotions. This method allows the
collaborative maturing of unstructured tags to commonly
accepted concepts.</p>
      <p>The second approach also provides consolidation
capabilities, but in particular concentrates on how to support
the formalization based on consolidated concepts. The key
idea here is to couple social bookmarking approaches with
a lightweight ontology editor. Its application domain is
informal learning and knowledge management support in
interdisciplinary application-oriented research.</p>
    </sec>
    <sec id="sec-8">
      <title>4. IMAGINATION</title>
      <p>The IMAGINATION EU project2 provides image-based
navigation for digital cultural and scientific resources. Users
can click on parts of an image to find other interesting
images to a given context. A click on an image part
automatically generates a semantic query. This feature requires
semantic metadata. In IMAGINATION, semantic metadata
are automatically generated by combining text-mining,
object recognition and object identification algorithms
exploiting domain ontologies. The metadata can be validated,
corrected and extended manually if needed.
4.1</p>
    </sec>
    <sec id="sec-9">
      <title>Requirements</title>
      <p>
        Based on our experience in implementing commercial
image search systems3, current metadata about the content
of images is largely based on the unstructured and
nonsemantic tagging paradigm (even before the popularity of
Web 2.0)—often called also keyword or label and
embedded into IPTC attributes [
        <xref ref-type="bibr" rid="ref4">4</xref>
        ]. This tag-based annotation of
images has many of the shortcomings mentioned as triggers
for entering ontology engineering activities in the previous
section. Previous attempts to solve the problem with the
help of thesauri have largely failed because thesauri were
not accepted by the users and content providers due to their
incompleteness and/or complexity.
      </p>
      <p>These experiences can be directly fed into requirements
for the use of ontologies in the area of image search and
navigation: (1) the ontology (and therefore semantic
annotations using it) must be comprehensible for the users and
(2) an ontology must cover the image repository completely.
The challenge is that image repositories can change rapidly
and consequently the ontology must be adapted regularly.
2http://www.imagination-project.org
3See e.g. www.fotomarktplatz.de
4.2</p>
    </sec>
    <sec id="sec-10">
      <title>Ontology Development Methodology Based on Imagenotions</title>
      <p>Based on the ontology maturing process, we can develop
a solution for these problems. In terms of the previously
introduced ontology maturing process we concentrate on the
first three steps from the emergence of ideas up to
formalization. What distinguishes our methodology from the usual
ontology development methodologies is the strong emphasis
on collaborative ontology development in the
consolidationin-communities phase. This is motivated by the success of
collaborative tagging in Web 2.0 projects.</p>
      <p>The basis of our ontology formalism is a concept we call
imagenotions. An imagenotion graphically represents a
semantic notion through an image, or a set of images. In
addition, similarly to many existing ontology formalism, it
is also possible to associate tags with an imagenotion in
different languages (such as English or German). For each
language, one of these tags is selected as the main label of
the imagenotion. The other tags are termed synonyms.</p>
      <p>Instead of tags, images are annotated with imagenotions.
It is easy to see the advantage: all the shortcomings of
tagging approaches are solved using imagenotions—it is easy
to find images using search terms in any language. In other
words, we provide semantic search instead of full-text search.</p>
      <p>In the terminology of classical ontologies, imagenotions
are usually instances, but they may also correspond to
concepts or relations. There are two major advantages of using
imagenotions over the classical ontology constructs:
1. The distinctions between concepts, instances and
relations are hard to understand for most users. In our
mind, notions play the role of an instance, a concept or
a relation, depending on the actual context. This fact
is acknowledged by many ontology formalisms that
allow metamodeling. Using imagenotions, users do not
need to understand this somewhat artificial separation
of notions.
2. Because imagenotions are associated with images, they
are meaningful internationally as an image has the
same meaning in different languages.</p>
      <p>The goal of our methodology is to guide the process of
creating an ontology of imagenotions. The main steps of
this methodology is based on the ontology maturing process
model:
1. Emergence of Ideas. In this step, new imagenotions are
created. Already this step can become collaborative,
as users can jointly collect the tags describing
imagenotions, and select the most representative images for an
imagenotion. Collaborative editing is especially
useful in a multi-lingual environment where it cannot be
expected that any individual user speaks all required
languages.
2. Consolidation in Communities. Because it is so easy to
create new imagenotions, it cannot be avoided that for
the same semantic notion initially many imagenotions
are created (synonyms, also in different languages) or
that an imagenotion represents more than one
semantic notion (homonyms). In this step, these problems
should be solved by merging synonymous
imagenotions, and by splitting imagenotions representing more
than one notion.
3. Formalization. In this step, taxonomical (“is-a”) and
ad-hoc relations are specified among imagenotions.</p>
      <p>After step 2, the quality of image search already increases
significantly, as the problems with synonyms and homonyms
do not appear anymore. Moreover, it is easy to see that all
annotated images automatically benefit from the maturing
imagenotions. E.g. adding a new tag to an imagenotions
automatically allows users to find all images annotated with
that imagenotion using the new tag. In addition, the
outcome of step 3 also allows requests for related images based
on the current context.</p>
      <p>Imagenotions are useful to collaboratively build an
ontology supporting manual annotation and semantic search.
However, to fulfill the requirement of IMAGINATION for
automatic annotation, a classical formal ontology is needed
that can be exploited by text-mining and object
identification algorithms. This last axiomatization step is not yet
directly supported by our methodology, it is subject of future
research. Nevertheless, it is easy to see that a conversion of
an imagenotion ontology to a standard ontology formalism
(e.g. OWL) is possible. The only missing information is
whether an imagenotion should be modelled as a concept,
instance or a relation in the target ontology formalism.
4.3</p>
    </sec>
    <sec id="sec-11">
      <title>Tool Support</title>
      <p>Currently we implement a web-based tool that allows the
creation of new imagenotions and the editing of available
imagenotions. This tool supports all three steps of our
methodology. It can be easily invoked during semantic search or
when uploading new images into an image repository: e.g.
it is fully integrated into the user’s workflow.</p>
      <p>We now demonstrate some functionality of the tool in
terms of the steps of our development methodology.
4.3.1</p>
      <sec id="sec-11-1">
        <title>Step 1: Emergence of Ideas</title>
        <p>Figure 2 shows an example for the emergence of ideas.
Let us assume that a content owner has new images about
elephants. The imagenotion “elephant” was so far not
available. Therefore, she creates a new imagenotion, adds an
image or part of an image that shows elephants and starts
describing the new imagenotion with more details. She uses
English as spoken language. As synonyms, she enters
“elephantidae” and “tusker”. Instead of tagging the new images
that show elephants with these words, she can use the new
imagenotion—she just pulls this imagenotion over the new
images via drag and drop.
The front end of the image search system also allows users
to add missing synonyms to already defined imagenotions in
order to achieve better search results. The label and
synonyms of imagenotions are editable in different languages.</p>
        <p>All images described with imagenotions automatically
benefit from all added information. In other words, the
community improves the quality of the image descriptions by
improving the quality of the corresponding imagenotions.
4.3.2</p>
      </sec>
      <sec id="sec-11-2">
        <title>Step 2: Consolidation in Communities</title>
        <p>Consolidation solves the problem of homonyms and
different languages. For instance, the word “tusker” is a homonym
because (among others) it is also the brand of a Kenyan beer
brewery (see Fig. 3). The first user noticing this homonym
can solve the problem: she splits the imagenotion tusker in
two new imagenotions—one with an image of an elephant
for the meaning “elephant”, and another with the label of
the Kenyan brewery for the meaning of the “Kenyan beer
brand”.</p>
        <p>In the case when someone detects two imagenotions that
have the same meaning (possibly in different languages), she
can merge them together.
4.3.3</p>
      </sec>
      <sec id="sec-11-3">
        <title>Step 3: Formalization</title>
        <p>Figure 4 shows an example for formalization—the creation
of hierarchies. In this example two new imagenotions for two
sorts of elephants are created: the “African elephant” and
the “Asian elephant” imagenotions. The trigger for creating
hierarchies may be the description of images in more details.</p>
        <p>As elephants are mammals, one could also add broader
imagenotions (the “is-a” relation): the user searches for the
imagenotion “mammal” and adds it by drag and drop to the
imagenotion of elephants to state that “every elephant is a
mammal”. It is also possible to define other kinds of
relations. For instance, in Figure 5 somone defined the relations
“has baby” and used this relation between the imagenotions
“Angelina Jolie” and her baby “Shiloh” as well as her son
“Maddox Chivan”.
4.4</p>
      </sec>
    </sec>
    <sec id="sec-12">
      <title>Application for Semantic Search</title>
      <p>In addition to improving search results, imagenotions also
allow a new principle of visual search refinement based on
formal knowledge. Users start by typing some search terms.
Our system proposes imagenotions containing these search
terms as label or synonym. Furthermore, our system also
proposes images and search refinements based on the current
context. These proposals are visualized with imagenotions
and determined using the relations from the available
imagenotion ontology. Refining of a search request is possible
by just clicking on the desired imagenotion.</p>
      <p>Figure 6 shows such an example. The current search
context is “images that contain the actors Angelina Jolie and
Brad Pitt” and the user already selected a matching image.
The system proposes two related groups of images: one for
the film “Mr. and Mrs. Smith” and one for their common
baby. These proposals are visualized using the
corresponding image notions.</p>
    </sec>
    <sec id="sec-13">
      <title>5. IM WISSENSNETZ – IN THE KNOWL</title>
    </sec>
    <sec id="sec-14">
      <title>EDGE WEB</title>
      <p>Our second use case is taken from the German research
project “Im Wissensnetz – Vernetzte Informationsprozesse
in Forschungsverbu¨nden”4 (“In the Knowledge Web –
Networked Information Processes in Research Associations”),
which aims to support efficient interdisciplinary
knowledgeadded processes within e-Science. Research is likely to be
the most knowledge-intensive environment. An empirical
analysis of existing (cooperation-)processes, information and
knowledge exchanges, and instruments for the preservation
of knowledge accomplished in the application domain “rapid
prototyping” revealed that scientific work is characterized by
high variability, dynamics and unpredictability as well as by
high significance of social interactions and communication.
5.1</p>
    </sec>
    <sec id="sec-15">
      <title>Requirements</title>
      <p>Linking people with individual expert knowledge and
contents from various disciplines like plastics, ceramics, and
mechanical engineering is one of the most important challenges.
For instance, one major problem is searching and retrieving
adequate contemporary resources. This process is very
tedious. The users have to access many various data sources
with different interfaces, but also the internet with common
search engines like Google.</p>
      <p>In the area of plastics and their market these high
dynamics are particularly obvious. New materials or new forms
of existing ones frequently enter the market; brand names
and manufacturers are permanently changing and hardly
trackable—attributes of a chemical substance retrievable
using its brand name today, are very hard to find once it’s sold
under a different label. There is also no general up to date
database which list manufacturer and brand names of
currently available forms of plastics. Thus, the users rely on
search engines like Google in order to find the wanted
material of which only the old brand or chemical name is known.
However, common search engines provide many irrelevant
results because of their missing focus on the domain. For
instance, when looking for “nylon” you receive lots of results
for stockings.</p>
      <p>At this point, using annotation and retrieval tools can
help; e.g. when a colleague already found the new brand
name of a product and tagged it with the old one you are
looking for. With such tools being further semantically
enriched with background knowledge and domain ontologies, it
is possible to find out the search context and thus to extend
or refine the search in order to reduce irrelevant results and
to guide the user. However, this also requires that (1) the
users collaboratively build up and maintain a shared
understanding and terminology, (2) these activities are embedded
into their information seeking activities and (3) this
understanding is expressed formally enough to enable
ontologybased query refinement.
5.2</p>
    </sec>
    <sec id="sec-16">
      <title>Ontology Development Methodology Based on Semantic Social Bookmarking</title>
      <p>In order to overcome these problems, we provide a
solution that combines collaborative collecting and sharing of
web resources (bookmarks) with collaborative development
of a shared vocabulary. This vocabulary is used to organize
the bookmarks. That means, collected bookmarks can be
annotated with concepts from the vocabulary (see Figure
7).</p>
      <p>If users find a web resource, e.g. the manufacturer’s
website for a specific plastic or an article about a new
material, they can pick it up into the shared bookmark collection
and annotate it with concepts from the vocabulary, e.g. the
specific plastic. If a needed concept does not exist in the
shared vocabulary (e.g. in the case of a new material) or
is not suitable (e.g. when the brand name changed), the
users can modify an existing concept or add arbitrary tags.
These new tags are automatically collected as “prototypical
concepts” within the vocabulary. The users can consolidate
them later by abstracting into concepts and placing them
within the ontology. In this way, we allow that new
concept ideas and tags are gathered seamlessly when they are
occurring and that users can define concepts in a free and
informal manner without the usual modeling overhead.</p>
      <p>
        For structuring the concepts we concentrate on taxonomic
relations because they are easy understandable for
non-modelling experts. Going beyond common tagging systems,
where the users can only bundle concepts underneath
another, the users can organize their concepts within our
approach in a shared structure according to the SKOS Core
Vocabulary [
        <xref ref-type="bibr" rid="ref3">3</xref>
        ] in which the concepts are connected by “broader”,
“narrower” and “related” links. The users can further
specify one preferred label and a number of alternative labels or
synonyms (e.g. former brand names) as well as a textual
description for each concept.
      </p>
      <p>The users within the community share and maintain one
taxonomy and one collection of bookmarks collaboratively.
Everyone has the right for editing and modifications
following the wiki paradigm of self-regulation.</p>
      <p>In this way, we can support in particular the second and
third transition phase of the ontology maturing process.
5.3</p>
    </sec>
    <sec id="sec-17">
      <title>Tool support with SOBOLEO</title>
      <p>We are developing SOBOLEO (Social Bookmarking and
Lightweight Engineering of Ontologies) in order to satisfy
these requirements. SOBOLEO’s goal is to support
knowledge workers working together in one domain in developing a
shared vocabulary and a shared collection of relevant web
resources with a lightweight ontology editor and an
ontologyenabled social bookmarking system.</p>
      <p>
        SOBOLEO is based on AJAX technology using the Google
Web Toolkit [
        <xref ref-type="bibr" rid="ref11">11</xref>
        ] and works in most current browsers—thus
does not require any local installation. It consists of four
application parts: an editor for the modification of the shared
taxonomy, a tool for the annotation of web resources, a
semantic search engine for the annotated web resources and
a taxonomy browser for navigating the taxonomy and the
bookmark collection.
      </p>
      <p>• Editing. The editor interface (see Figure 8), built
up tripartite, displays a tree view of the taxonomy on
the left hand side of the screen. It shows the
concepts with their preferred labels and their narrower
and broader relations. Informal, not yet consolidated
tags are collected in the special branch of
“prototypical concepts”. When a concept is selected in the tree
view, its details are displayed in the center part of
the screen. Here, the users can edit the preferred and
alternative labels, the description, and the narrower,
broader and related relations between concepts, which
is further supported through auto completion of
entities in the taxonomy. The right hand side of the screen
provides a chat panel and allows having a conversation
with other people editing the same ontology at the
same time. The chat panel is also used for displaying
changes made to the taxonomy. The system
automatically generates a chat message that details who did
which change. Changes done to the ontology by one
user are visible almost instantaneously on all machines
without requiring any intervention by the users.
Updates of other users are also immediately reflected in
the center screen part showing the details of the
currently selected resource—particularly important if two
users are editing one concept at the same time.
• Annotation. The annotation interface opens a pop
up (see Figure 7) in which users can enter title and url
of the web resource they want to add to the collection.
Users can further specify concepts from the taxonomy,
which is also supported by auto completion, or they
can add new tags to annotate the resource. If the
collection already contains the resource, the existing
annotations are displayed and can also be edited.
• Search. The semantic based search engine allows for
searching and retrieving resources within the shared
bookmark collection (see Figure 9). Users can enter
concept labels from the taxonomy or arbitrary search
terms. The engine presents all resources either
annotated with the identified concepts and their narrower
ones or containing the input terms within the page
content. The interface lists the resources with their
title linking to the original page, with annotated
concepts, a short excerpt of the page content highlighting
the search terms, and the exact url. It further
provides query refinement and relaxation proposals. Via
each result’s edit link, users can modify or remove the
annotations of a web resource.
• Browsing. With the browsing interface users can
navigate through the taxonomy and associated
bookmark documents (see Figure 10). Starting from the
root concepts, the users can click through the
taxonomy concepts. On top, the users see the currently
selected concept with its preferred and alternative
labels and its description. Additionally, all its broader,
narrower and related concepts are displayed as links
for further navigation. Underneath the concept
details there is a list of all resources which are annotated
with the currently selected concept or with one of its</p>
      <p>narrower concepts. These resources are further ranked
by their date they were collected, thus the newest
resources appear upmost.</p>
    </sec>
    <sec id="sec-18">
      <title>CONCLUSIONS</title>
      <p>We will only ever achieve sustainable ontology-based
systems by embedding the task of building and maintaining
ontologies into everyday work processes, enabling domain
experts to do it without the help of knowledge engineers and
by making it truly collaborative. We also have to
acknowledge that ontologies cannot be formalized from scratch, but
rather continuously evolve in a maturing process from
informal tags to formal taxonomy hierarchies for which the
ontology maturing process was presented.</p>
      <p>Therefore, our model for ontology maturing offers four
different steps. It allows for the emergence of ideas from
each individual and the consolidation in communities for a
common terminology. Then, in the third step of our model,
relations help in creating formal lightweight ontologies.
Finally, the fourth step could allow axiomatization. Such a
maturing view on ontology engineering can overcome the
problem of conceptual dynamics (e.g. the problem of the
time lag between emergence of topics and their inclusion in
an ontology).</p>
      <p>In order to support such a maturing process, we presented
two lightweight, easy-to-use and work embedded tools that
allow the collaborative maturing of ontologies. Both of them
lower the barriers to ontology editing for non-knowledge
formulation experts. The project IMAGINATION will use the
idea of image based ontolgy maturing with imagenotions
for the collaborative creation of required domain ontologies.
SOBOLEO allows social bookmarking with ontologies in the
project Im Wissensnetz.</p>
      <p>
        Our next work steps are evaluations and refinements of
our model for the ontology maturing processes in these
projects. We are mainly interested in evaluating which
further functionalities are needed and wanted by the users, e.g.
whether and when it is necessary to introduce big numbers
of relations between the available concepts. Likewise, for
the sake of simplicity, we are not considering aspects of
ontology versioning at the moment (here, we want to refer to
the works of [
        <xref ref-type="bibr" rid="ref26">26</xref>
        ]). Our main evaluation target is to analyze
if the shared ontologies, developed based on our tools and
models, converge to a stable and common accepted
understanding for the available concepts over time. If not, our
models require further support for agreement finding (e.g.
like wiki) or personal ontologies.
7.
      </p>
    </sec>
    <sec id="sec-19">
      <title>ACKNOWLEDGMENTS</title>
      <p>This work was co-funded by the European Commission
within the project IMAGINATION, by the German Federal
Ministry for Education and Research within the project Im
Wissensnetz and by German Federal Ministry of Economic
and Technology within the project KSIunderground.
8.</p>
    </sec>
    <sec id="sec-20">
      <title>ADDITIONAL AUTHORS</title>
      <p>Additional authors: Gabor Nagypal (disy
InformationsSysteme, Karlsruhe, Germany, email: nagypal@disy.net)
and Valentin Zacharias (FZI Research Center for
Information Technologies, email: zach@fzi.de).</p>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          [1]
          <string-name>
            <given-names>H.</given-names>
            <surname>Allert</surname>
          </string-name>
          ,
          <string-name>
            <given-names>H.</given-names>
            <surname>Markannen</surname>
          </string-name>
          , and
          <string-name>
            <given-names>C.</given-names>
            <surname>Richter</surname>
          </string-name>
          .
          <article-title>Rethinking the Use of Ontologies in Learning</article-title>
          . In M. Memmel and D. Burgos, editors,
          <source>Proceedings of the 2nd International Workshop on Learner-Oriented Knowledge Management and KM-Oriented Learning (LOKMOL 06)</source>
          ,
          <source>in conjunction with the First European Conference on Technology-Enhanced Learning (ECTEL 06)</source>
          , pages
          <fpage>115</fpage>
          -
          <lpage>125</lpage>
          ,
          <year>October 2006</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          [2]
          <string-name>
            <given-names>K.</given-names>
            <surname>Barker</surname>
          </string-name>
          ,
          <string-name>
            <given-names>V. K.</given-names>
            <surname>Chaudhri</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S. Y.</given-names>
            <surname>Chaw</surname>
          </string-name>
          ,
          <string-name>
            <given-names>P.</given-names>
            <surname>Clark</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J.</given-names>
            <surname>Fan</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D.</given-names>
            <surname>Israel</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S.</given-names>
            <surname>Mishra</surname>
          </string-name>
          ,
          <string-name>
            <given-names>B. W.</given-names>
            <surname>Porter</surname>
          </string-name>
          ,
          <string-name>
            <given-names>P.</given-names>
            <surname>Romero</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D.</given-names>
            <surname>Tecuci</surname>
          </string-name>
          , and
          <string-name>
            <given-names>P. Z.</given-names>
            <surname>Yeh</surname>
          </string-name>
          .
          <article-title>A question-answering system for AP Chemistry: Assessing KR&amp;R technologies</article-title>
          .
          <source>In Proceedings of the Ninth International Conference on Principles of Knowledge Representation and Reasoning</source>
          , pages
          <fpage>488</fpage>
          -
          <lpage>497</lpage>
          ,
          <year>2004</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          [3]
          <string-name>
            <given-names>D.</given-names>
            <surname>Brickley</surname>
          </string-name>
          and
          <string-name>
            <given-names>A.</given-names>
            <surname>Miles. SKOS Core Vocabulary</surname>
          </string-name>
          <article-title>Specification. W3C working draft</article-title>
          ,
          <source>W3C</source>
          ,
          <year>November 2005</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          [4]
          <string-name>
            <given-names>I. P. T.</given-names>
            <surname>Council</surname>
          </string-name>
          .
          <source>The IPTC Standard</source>
          ,
          <year>2007</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          <source>[5] Del.icio.us. Del.icio.us. http://del.icio.us/</source>
          ,
          <year>2007</year>
          . (accessed
          <year>2007</year>
          -
          <volume>01</volume>
          -25).
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          [6]
          <string-name>
            <given-names>M.</given-names>
            <surname>Denny</surname>
          </string-name>
          . Ontology Tools Survey, Revisited.
          <source>Technical report</source>
          , XML.com,
          <year>July 2004</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          [7]
          <string-name>
            <given-names>M.</given-names>
            <surname>Fern</surname>
          </string-name>
          <article-title>´andez-L´opez and A. G´omez-P´erez. A survey on methodologies for developing, mainaining, integrating, evaluating and reengineering ontologies</article-title>
          .
          <source>Deliverable 1</source>
          .4,
          <string-name>
            <given-names>EU</given-names>
            <surname>IST Project</surname>
          </string-name>
          <string-name>
            <surname>IST</surname>
          </string-name>
          -2000
          <source>-29243 OntoWeb</source>
          ,
          <year>2002</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          [8]
          <string-name>
            <surname>Flickr</surname>
          </string-name>
          . Welcome to Flickr - Photo
          <string-name>
            <surname>Sharing</surname>
          </string-name>
          . http://www.flickr.com/,
          <year>2007</year>
          . (accessed
          <year>2007</year>
          -
          <volume>01</volume>
          -25).
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          [9]
          <string-name>
            <given-names>S.</given-names>
            <surname>Golder</surname>
          </string-name>
          and
          <string-name>
            <given-names>B. A.</given-names>
            <surname>Huberman</surname>
          </string-name>
          .
          <article-title>The structure of collaborative tagging systems</article-title>
          .
          <source>Journal of Information Sciences</source>
          ,
          <volume>32</volume>
          (
          <issue>2</issue>
          ):
          <fpage>198</fpage>
          -
          <lpage>208</lpage>
          ,
          <year>2006</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          [10]
          <string-name>
            <surname>A. G</surname>
          </string-name>
          <article-title>´omez-P´erez, M. Fern´andez-L´opez, and</article-title>
          <string-name>
            <given-names>O.</given-names>
            <surname>Corcho</surname>
          </string-name>
          .
          <article-title>Ontological Engineering with examples from the areas of Knowledge Management, e-Commerce and the Semantic Web</article-title>
          .
          <source>Advanced Information and Knowledge Processing. Springer, 1st edition</source>
          ,
          <year>2004</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref11">
        <mixed-citation>
          [11]
          <string-name>
            <surname>Google</surname>
            <given-names>Web</given-names>
          </string-name>
          <string-name>
            <surname>Toolkit. Google Web Toolkit - Build</surname>
            <given-names>AJAX</given-names>
          </string-name>
          <article-title>apps in the Java language</article-title>
          . http://code.google.com/webtoolkit/,
          <year>2007</year>
          . (accessed
          <year>2007</year>
          -
          <volume>01</volume>
          -25).
        </mixed-citation>
      </ref>
      <ref id="ref12">
        <mixed-citation>
          [12]
          <string-name>
            <given-names>S. W. I.</given-names>
            <surname>Group</surname>
          </string-name>
          . Semantic Wiki State Of The Art,
          <year>2007</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref13">
        <mixed-citation>
          [13]
          <string-name>
            <given-names>M.</given-names>
            <surname>Guy</surname>
          </string-name>
          and
          <string-name>
            <given-names>E.</given-names>
            <surname>Tonkin</surname>
          </string-name>
          . Folksonomies: Tidying up tags?
          <string-name>
            <surname>D-Lib</surname>
            <given-names>Magazine</given-names>
          </string-name>
          ,
          <volume>12</volume>
          (
          <issue>1</issue>
          ),
          <year>January 2006</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref14">
        <mixed-citation>
          [14]
          <string-name>
            <given-names>M.</given-names>
            <surname>Hepp</surname>
          </string-name>
          .
          <article-title>Possible ontologies: How reality constraints building relevant ontologies</article-title>
          .
          <source>IEEE Internet Computing</source>
          ,
          <volume>11</volume>
          (
          <issue>1</issue>
          ):
          <fpage>90</fpage>
          -
          <lpage>96</lpage>
          , January/
          <year>February 2007</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref15">
        <mixed-citation>
          [15]
          <string-name>
            <given-names>M.</given-names>
            <surname>Hepp</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D.</given-names>
            <surname>Bachlechner</surname>
          </string-name>
          , and
          <string-name>
            <given-names>K.</given-names>
            <surname>Siorpaes.</surname>
          </string-name>
          <article-title>OntoWiki: community-driven ontology engineering and ontology usage based on Wikis</article-title>
          .
          <source>In WikiSym '06: Proceedings of the 2006 international symposium on Wikis</source>
          , pages
          <fpage>143</fpage>
          -
          <lpage>144</lpage>
          , New York, NY, USA,
          <year>2006</year>
          . ACM Press.
        </mixed-citation>
      </ref>
      <ref id="ref16">
        <mixed-citation>
          [16]
          <string-name>
            <given-names>A.</given-names>
            <surname>Hotho</surname>
          </string-name>
          ,
          <string-name>
            <given-names>R.</given-names>
            <surname>Jschke</surname>
          </string-name>
          ,
          <string-name>
            <given-names>C.</given-names>
            <surname>Schmitz</surname>
          </string-name>
          , and
          <string-name>
            <surname>G. Stumme.</surname>
          </string-name>
          <article-title>BibSonomy: A Social Bookmark and Publication Sharing System</article-title>
          . In A. de Moor, S. Polovina, and H. Delugach, editors,
          <source>Proceedings of the Conceptual Structures Tool Interoperability Workshop at the 14th International Conference on Conceptual Structures</source>
          , Aalborg, Denmark,
          <year>July 2006</year>
          . Aalborg University Press.
        </mixed-citation>
      </ref>
      <ref id="ref17">
        <mixed-citation>
          [17]
          <string-name>
            <given-names>K.</given-names>
            <surname>Kotis</surname>
          </string-name>
          ,
          <string-name>
            <given-names>G. A.</given-names>
            <surname>Vouros</surname>
          </string-name>
          , and
          <string-name>
            <given-names>J. P.</given-names>
            <surname>Alonso. HCOME</surname>
          </string-name>
          :
          <article-title>A Tool-Supported Methodology for Engineering Living Ontologies</article-title>
          . In C. Bussler,
          <string-name>
            <given-names>V.</given-names>
            <surname>Tannen</surname>
          </string-name>
          , and I. Fundulaki, editors,
          <source>Semantic Web and Databases</source>
          . Second International Workshop - SWDB
          <year>2004</year>
          , volume
          <volume>3372</volume>
          <source>of LNCS</source>
          , pages
          <fpage>155</fpage>
          -
          <lpage>166</lpage>
          , Berlin Heidelberg, Germany,
          <year>August 2004</year>
          . Springer-Verlag.
        </mixed-citation>
      </ref>
      <ref id="ref18">
        <mixed-citation>
          [18]
          <string-name>
            <given-names>C. C.</given-names>
            <surname>Kuhlthau</surname>
          </string-name>
          .
          <article-title>Seeking Meaning: A Process Approach to Library and Information Services</article-title>
          .
          <source>Libraries Unlimited, Westport, CT, 2nd edition edition</source>
          ,
          <year>2004</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref19">
        <mixed-citation>
          [19]
          <string-name>
            <given-names>A.</given-names>
            <surname>Maedche</surname>
          </string-name>
          ,
          <string-name>
            <given-names>B.</given-names>
            <surname>Motik</surname>
          </string-name>
          , and
          <string-name>
            <given-names>L.</given-names>
            <surname>Stojanovic</surname>
          </string-name>
          .
          <article-title>Managing multiple and distributed ontologies on the Semantic Web</article-title>
          .
          <source>The VLDB Journal</source>
          ,
          <volume>12</volume>
          (
          <issue>4</issue>
          ):
          <fpage>286</fpage>
          -
          <lpage>302</lpage>
          ,
          <year>November 2003</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref20">
        <mixed-citation>
          [20]
          <string-name>
            <given-names>C.</given-names>
            <surname>Marlow</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Naaman</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D.</given-names>
            <surname>Boyd</surname>
          </string-name>
          , and
          <string-name>
            <given-names>M.</given-names>
            <surname>Davis</surname>
          </string-name>
          . Position Paper, Tagging, Taxonomy, Flickr, Article, ToRead. In Collaborative Web Tagging Workshop at WWW2006, May
          <year>2006</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref21">
        <mixed-citation>
          [21]
          <string-name>
            <given-names>A. G.</given-names>
            <surname>Perez</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J.</given-names>
            <surname>Angele</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M. F.</given-names>
            <surname>Lopez</surname>
          </string-name>
          ,
          <string-name>
            <given-names>V.</given-names>
            <surname>Christophides</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Stutt</surname>
          </string-name>
          , and
          <string-name>
            <given-names>Y.</given-names>
            <surname>Sure</surname>
          </string-name>
          .
          <article-title>A survey on ontology tools</article-title>
          .
          <source>Deliverable 1</source>
          .3,
          <string-name>
            <given-names>EU</given-names>
            <surname>IST Project</surname>
          </string-name>
          <string-name>
            <surname>IST</surname>
          </string-name>
          -2000
          <source>-29243 OntoWeb</source>
          ,
          <year>2002</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref22">
        <mixed-citation>
          [22]
          <string-name>
            <surname>Prot</surname>
          </string-name>
          <article-title>´eg´e. The Prot´eg´e Ontology Editor and Knowledge Acquisition System</article-title>
          . http://protege.stanford.edu/,
          <year>2007</year>
          . (accessed
          <year>2007</year>
          -
          <volume>01</volume>
          -25).
        </mixed-citation>
      </ref>
      <ref id="ref23">
        <mixed-citation>
          [23]
          <string-name>
            <given-names>S.</given-names>
            <surname>Schaffert</surname>
          </string-name>
          .
          <article-title>IkeWiki: A Semantic Wiki for Collaborative Knowledge Management</article-title>
          .
          <source>In 1st International Workshop on Semantic Technologies in Collaborative Applications (STICA'06)</source>
          , Manchester,
          <string-name>
            <surname>UK</surname>
          </string-name>
          ,
          <year>June 2006</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref24">
        <mixed-citation>
          [24]
          <string-name>
            <given-names>A.</given-names>
            <surname>Schmidt</surname>
          </string-name>
          .
          <article-title>Knowledge Maturing and the Continuity of Context as a Unifying Concept for Knowledge Management and E-Learning</article-title>
          .
          <source>In Proceedings of I-KNOW '05</source>
          ,
          <string-name>
            <surname>Special</surname>
            <given-names>Track</given-names>
          </string-name>
          <source>on Integrating Working and Learning</source>
          ,
          <source>July/August</source>
          <year>2005</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref25">
        <mixed-citation>
          [25]
          <string-name>
            <given-names>S.</given-names>
            <surname>Sen</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S. K.</given-names>
            <surname>Lam</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A. M.</given-names>
            <surname>Rashid</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D.</given-names>
            <surname>Cosley</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D.</given-names>
            <surname>Frankowski</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J.</given-names>
            <surname>Osterhouse</surname>
          </string-name>
          ,
          <string-name>
            <given-names>F. M.</given-names>
            <surname>Harper</surname>
          </string-name>
          , and
          <string-name>
            <given-names>J.</given-names>
            <surname>Riedl</surname>
          </string-name>
          . tagging, communities, vocabulary, evolution.
          <source>In CSCW '06: Proceedings of the 2006 20th anniversary conference on Computer supported cooperative work</source>
          , pages
          <fpage>181</fpage>
          -
          <lpage>190</lpage>
          , New York, NY, USA,
          <year>2006</year>
          . ACM Press.
        </mixed-citation>
      </ref>
      <ref id="ref26">
        <mixed-citation>
          [26]
          <string-name>
            <given-names>L.</given-names>
            <surname>Stojanovic</surname>
          </string-name>
          .
          <article-title>Methods and Tools for Ontology Evolution</article-title>
          .
          <source>PhD thesis</source>
          , University of Karlsruhe (TH), Germany,
          <year>2004</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref27">
        <mixed-citation>
          [27]
          <string-name>
            <given-names>J. W.</given-names>
            <surname>Tanaka</surname>
          </string-name>
          and
          <string-name>
            <given-names>M.</given-names>
            <surname>Taylor</surname>
          </string-name>
          .
          <article-title>Object categories and expertise: Is the basic level in the eye of the beholder?</article-title>
          <source>Cognitve Psychology</source>
          ,
          <volume>23</volume>
          (
          <issue>3</issue>
          ):
          <fpage>457</fpage>
          -
          <lpage>482</lpage>
          ,
          <year>July 1991</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref28">
        <mixed-citation>
          [28]
          <string-name>
            <surname>M.</surname>
          </string-name>
          <article-title>V¨olkel</article-title>
          , M. Kr¨otzsch, D. Vrandecic,
          <string-name>
            <given-names>H.</given-names>
            <surname>Haller</surname>
          </string-name>
          , and
          <string-name>
            <given-names>R.</given-names>
            <surname>Studer</surname>
          </string-name>
          . Semantic Wikipedia.
          <source>In Proceedings of the 15th international conference on World Wide Web (WWW'06)</source>
          , pages
          <fpage>585</fpage>
          -
          <lpage>594</lpage>
          . ACM Press,
          <year>2006</year>
          .
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>