=Paper=
{{Paper
|id=None
|storemode=property
|title=Semantifying OpenStreetMap
|pdfUrl=https://ceur-ws.org/Vol-901/paper4.pdf
|volume=Vol-901
|dblpUrl=https://dblp.org/rec/conf/semweb/BaglatziKK12
}}
==Semantifying OpenStreetMap ==
https://ceur-ws.org/Vol-901/paper4.pdf
Semantifying OpenStreetMap
Alkyoni Baglatzi, Margarita Kokla, Marinos Kavouras
School of Rural and Surveying Engineering, National Technical University of Athens
H. Polytechniou Str. 9, 15780 Zografos Campus, Greece
alkyoni.baglatzi@gmail.com, (mkokla, mkav)@survey.ntua.gr
Abstract. OpenStreetMap is one of the best examples of Volunteered
Geographic Information. Its success relies on the ease of use and the
freedom it provides. Users are supposed to geolocate their Points Of
Interest and annotate them with a tag. There is no certain vocabulary
or ontology of the tags that users have to commit to. The whole tagging
process is done in a bottom-up manner in which the community on a
wiki basis discusses this issue. Allowing users to use tags freely increases
the usability of OpenStreetMap but at the same time causes semantic
interoperability problems. What is needed, is a way to structure the
tags while satisfying the freedom criterion. As a solution, we suggest the
alignment of the tags to well structured top level ontologies. A middle
layer approach for bridging the gap between the bottom-up tags of the
users and the top level Descriptive Ontology for Linguistic and Cognitive
Engineering (DOLCE) is proposed. The idea of “games with a purpose”
is utilized to assists non-expert users in aligning their tags to DOLCE.
Keywords: VGI, OpenStreetMap, tagging, semantics, alignment, games with
a purpose, top level ontology
1 Introduction
User generated content is an important emerging research area. Engaging the
crowd in performing new tasks brings about new opportunities and new chal-
lenges. In the geographic domain, the term Volunteered Geographic Information
(VGI) was coined by Goodchild [12] for describing the collaborative mapping
activities of users and contribution of geographic data. OpenStreetMap which
was initiated at University College London (UCL) in July 2004 by Steve Coast,
is one of the most pervasive and representative examples of VGI [13].
OpenStreetMap offers an open and easy to use platform that enables contrib-
utors to upload geographic information collected from mobile devices or aerial
images. The data model is simple and consists of nodes, ways and relations.
Each mapped entity is accompanied with a tag. There is no formal ontology
or vocabulary of predefined tags that have to be adopted by the users, because
as argued by Steve Coast, the founder of OpenStreetMap: “no individual could
design such an ontology that would be all-encompassing, and even if they could
start no two individuals would agree on it[9]”.
Tags that facilitate the annotation of Points of Interest (POIs) in Open-
StreetMap come in key-value pairs 1 for instance, amenity=bar, natural=beach,
landuse= forest. There is no standardized way on how users shall annotate
their POIs neither on the naming level nor on which entities shall be tagged
under a certain name.
On a wiki 2 and mailing list 3 basis, the community exchanges opinions about
the tags proposing new tags or tags that should be abolished. The most common
tags in use, can be looked up in Taginfo 4 . The tags (or Map Features as found
in the wiki) are listed in some kind of loose hierarchy.
Although the freedom and openness provided eases the tagging procedure, it
causes semantic interoperability problems. User generated content is heteroge-
neous which leads to ambiguity, redundancy and inconsistency of the tags. As a
result, findability of the correct tag for annotating a POI as well as information
searching and retrieval is ineffective, an issue that has already been described
for instance in [4].
What is needed, is the combination of this loose hierarchy with well struc-
tured, organized, formalized top level ontologies and specifically the alignment
of users’ tags to concepts of a top level ontology. Top level ontologies can be seen
as a structured collection of semantic primitives or meta level concepts that are
used to further define domain concepts [14]. By aligning the domain concepts
to the top level ontologies, the meaning of these concepts gets grounded in the
semantic primitives. This universal view of top level ontologies is also a reason
why they are more suitable than domain ontologies for the alignment process.
The meta level concepts act as reference points in relation to which, the domain
concepts are defined.
OpenStreetMap is open to non-expert users of geographic data and thus, the
tagging attitude is rather intuitive than based on scientific methodologies and
knowledge. This calls for an alignment to a top level ontology, which underly-
ing design principles are prescribed by common sense. As has been argued by
its creators, the Descriptive Ontology for Linguistic and Cognitive Engineering
(DOLCE) [21] “has a clear cognitive bias, in the sense that it aims at capturing
the ontological categories underlying natural language and human commonsense
”([10] p.2). That is the reason why in the present work the DOLCE ontology has
been chosen. Specifically, the extension of DOLCE, DOLCE Ultralite 5 (DUL)
was regarded as more suitable because it replaces the complicated endurant, per-
durant division with object and event. Nevertheless, also other ontologies that
satisfy this criterion could have been used instead.
Aligning the tags to the top level ontology in a top-down approach with the
aid of knowledge engineers would have been accurate but time consuming. Con-
cerning maintenance, the dynamical nature of OpenStreetMap with the option
1
We use true type fonts to refer to tags
2
http://wiki.openstreetmap.org/wiki/Map_Features, last accessed 27.07.2012
3
http://lists.openstreetmap.org/listinfo, last accessed 27.07.2012
4
http://taginfo.openstreetmap.org/tags, last accessed 27.07.2012
5
http://www.loa-cnr.it/ontologies/DUL.owl, last accessed 05.09.2012
of new tags being introduced or old removed, would demand the repetition of
the alignment procedure very often which is resource inefficient.
Our research question is “How to find a user friendly way to align the tags
to top level ontologies?”. We aim at defining a methodology that will enable the
bottom-up alignment of tags to top level ontologies.
For that, we propose the use of the well established idea of “games with
a purpose” [34] applied to OpenStreetMap. Specifically, we choose “question
games”, a certain type of “games with a purpose”, for assisting users in aligning
their tags to the concepts of the top level ontology. We use DOLCE Ultralite,
an extension of DOLCE to align the tags to.
The contribution of this work is twofold: (a) it provides an analysis of the
semantic inconsistencies that emerge from the current state of the tagging pro-
cess in OpenStreetMap and (b) it proposes a way to combine top-down and
bottom-up approaches by preserving the advantages of both that is, the freedom
and easiness of the first and the structure, organization of the latter. For that,
a “question game” is designed. The main objective is to reuse existing methods
in order figure out a methodology for overcoming the semantic inconsistencies
of OpenStreetMap.
The remainder of the paper is organized as follows. Section 2 provides infor-
mation on the related work. Section 3 investigates some semantic inconsistencies
that were found in the OpenStreetMap tags. Section 4 describes the proposed
methodology for aligning the tags to the top level ontology and Section 5 con-
cludes the paper and discusses possible future directions.
2 Related Work
With the increase of user involvement in the web and the rising amount of user
generated content, tagging was introduced for annotating purposes. Flickr 6 ,
del.icio.us 7 , citeulike 8 and youtube 9 are just few examples where users added
tags to describe their content.
The tagging behaviour has been examined in multifarious ways with several
methods in order to understand the commonsense ground of user generated con-
tent. Thomas Vander Wal introduced the term folksonomy for describing this
collaborative tagging [33]. Clustering methods have been used to investigate in
a bottom-up manner kinds of tags people use in their annotations.
Ontology learning has substantially benefited from these kind of studies as
they unfolded the way people perceive and use different tags. As a result, on-
tologies can be designed more efficiently.
The need for combining well structured ontologies with hierarchically loose
folksonomies has already been acknowledged. Especially the problem of the miss-
ing relations between tags in folksonomies has been addressed in [1]. With the
6
http://www.flickr.com/, last accessed 27.07.2012
7
http://delicious.com/, last accessed 27.07.2012
8
http://www.citeulike.org/, last accessed 27.07.2012
9
http://www.youtube.com/, last accessed 27.07.2012
aid of ontologies, different kinds of relations between tags like subsumption re-
lations, disjointness relations, generic relations, sibling relations and instance of
relations were found and added.
Data mining techniques and ontologies are combined in [29] to make the
semantics of the tags explicit. Tags are preprocessed, clustered and related to
concepts in different ontologies. Swoogle 10 is used as a search engine for finding
appropriate ontologies.
In [19] knowledge from folksonomies is extracted with data mining techniques
and related to upper ontologies. After a preprocessing step, tags are related
to WordNet 11 and enriched with its relations. The methodology is applied to
datasets from flickr and citeulike.
Aligning folksonomies to domain ontologies is utilized in [24] for annotating
blog posts. The main goal is to limit tags’ ambiguity and variation. In the first
step, users are free to choose tags for annotating the blog posts. In the second
step, ontology concepts are shown to them and in an interactive, semi-automatic
manner, users are asked to relate their tags to the concepts from these ontolo-
gies. Tags have to be to explicitly matched to concepts from the ontologies.
WordNet is used in [17] to order tags in a hierarchical way. A tool has been
built that makes the navigation in tag spaces more comprehensive for users. As
a result, browsing and retrieving related tags is made easier and more efficient.
Concerning OpenStreetMap, important research has been conducted in analy-
sing the tagging behaviour of users i.e. which the most edited entities are and
how they change over time [22,23]. This provides evidence about the importance
of certain entities and the different ways they are perceived by the users broad-
ening the research agenda of user generated geospatial content.
The power of enriching OpenStreetMap with other sources has been demon-
strated in the LinkedGeoData project [2]. Instances of OpenStreetMap are pub-
lished according to the linked data principles and linked to DBpedia12 .
The problem of missing relatedness between geographic entities is addressed
in [3]. OpenStreetMap “spatially rich but semantically poor vector dataset” and
DBpedia “spatially poor but semantically rich ontology”, are combined providing
the user with information about a geographic entity. An important contribution
is the consideration of map scale in relating concepts to geographic entities.
OSMonto13 has been developed as an OWL ontology of OpenStreetMap tags
[7]. Keys are translated into classes and values into subclasses. The design deci-
sion was to be as close as possible to the tagging process enabling querying of
the OpenStreetMap database. That is why the tags were adopted as represented
in the tag wiki and no conceptual conflicts or ontological mismatches were con-
fronted. OSMonto is used in the DO-ROAM 14 project which aims at expanding
10
http://swoogle.umbc.edu/ last accessed 27.07.2012
11
http://wordnet.princeton.edu/ last accessed 27.07.2012
12
http://wiki.dbpedia.org/About last accessed 27.07.2012
13
https://raw.github.com/doroam/planning-do-roam/master/Ontology/tags.
owl, last accessed 27.07.2012
14
http://planning.do-roam.org/, last accessed 27.07.2012
the search capabilities not only to the POIs but also to the activities that can
be performed at a certain location [6].
Scheider et al. [26] argue for the need of more functional or affordance ori-
ented representation of OpenStreetMap tags. They underline the fact that the
current tagging practise in OpenStreetMap is not efficient enough for repre-
senting the identity of the POIs. As they mention, tagging a cafeteria which is
also open at night serving alcohol with amenity=cafe may exclude the alcohol-
serving functionality. They suggest that tags should be grounded in affordances.
Accompanying the tags with richer descriptions on the affordances of the POIs,
may harden the annotation process but will make the querying process more
efficient.
3 Semantic Inconsistencies of OpenStreetMap Tagging
The freedom and easiness of assigning tags to POIs is a hallmark of the suc-
cess of OpenStreetMap. But, on the semantics of the tags, it leads to several
interoperability problems. As this tag collection is a result of a bottom-up user
generated effort, it lacks some proper semantic structure. Especially the absence
of relations like the hypernymic relation which describe the is-a relation between
a concept and its genus and the meronymic relation that is the part-of relation
both acting “as the cement that links up concepts into knowledge structures”[15]
makes the annotation and searching process cumbersome.
In contrast to geographic ontologies, vocabularies, taxonomies created by do-
main experts, the key-value pairs are organized in a loose way. Although there
is some type of clustering or thematic grouping of the tags in the wiki, which
differentiates it from traditional folksonomies where hierarchical information is
missing, conceptual inconsistencies still exist. This section aims at providing
some examples of the semantic inconsistencies that arise from the tagging strat-
egy of OpenStreetMap.
To start with, there is no common criterion according to which the tags are
organized. As a result all keys are in the same hierarchical level. That is, all pri-
mary features as listed in the wiki i.e. amenity, aeroway, historic, landuse,
manmade, craft, sport, tourism, power, shop etc. are treated equally, which
results in conceptual vagueness and inconsistency. For instance, the nature of
office or shop is manmade. By not relating these tags to the tag manmade with
the class-subclass relation, important inheritance information gets lost. Same
applies to i.e. shop, office, building which could be subclasses of amenity.
On a more sophisticated level, a disadvantage of the flat structure of the tags
is the fact that no deeper associations between geographic entities can be estab-
lished. For instance, in the current tagging procedure there is no way of explicitly
stating that within landuse=commercial, geographic entities like shop=bakery,
office=architect are located.
Redundancies of tags provide us with evidence about the different conceptu-
alizations of certain POIs that users have. For instance, hotel, hospital, school are
tagged as tourism=hotel, building=hotel, amenity=hospital, building=ho-
spital, amenity=school, building=school. Users assign the same value (be it
hotel, hospital or school), to two different keys namely tourism and amenity.
Another finding is that tags related to activities are used to describe POIs
i.e. sport=climbing, sport=basketball, leisure=dance, leisure=fishing.
While describing the activity that can be performed at a certain POI, they are
used to annotate the POI itself. This may be confusing in terms of information
search as it perplexes the geographic entity with its function.
The primary tag amenity also creates confusion since it is used to represent
a wide variety of heterogeneous features (e.g., schools, parking lots, bus stations,
banks, hospitals, nightclubs, etc.). Although different subcategories of amenities
are defined in the wiki (such as sustenance, education, transportation, financial,
healthcare, etc.) to further classify different types of amenities, their possible
values (such as school, bar, embassy, etc.) directly refer to the general tag
amenity, e.g., amenity=school, amenity=bar, amenity=embassy.
The above cases are some examples of the semantic problems caused by a
bottom-up, intuitive approach. Furthermore, although the proposed tags have
resulted from consensus, they do not necessarily represent a common and wide
conceptualization of geographic concepts. For this reason, although such ap-
proaches have stimulated considerable interest and resulted in the collection of
huge volumes of geospatial data, they are accompanied by problems, such as the
creation of arbitrary attributes or attribute values, multiple tags for the same
geographic features, disagreement on the name of features, etc. [22]. The present
research aims at the design of a “game with a purpose” to align the loose hierar-
chy of OpenStreet Map tags with a well structured top level ontology to provide
meaningful and cognitively important associations between tags.
4 Question Game for Aligning OpenStreetMap Tags to
DOLCE Top Level Ontology
4.1 Introduction to the DOLCE Ultralite Top Level Ontology
DOLCE [21] is a foundational or top level ontology developed within the WON-
DERWEB project15 . It is an ontology of particulars and comes in different ver-
sions DOLCE Lite-Plus 16 and DOLCE+ DnS Ultralite 17 (or DOLCE Ultralite).
In the present work, the DOLCE Ultralite was chosen because its categories
and organization is simpler and more intuitive than the other versions as ar-
gued by its authors. The top concept entity is categorized into abstract, event,
information entity, object and quality. For the alignment, the class object and es-
pecially its subclasses physical and social object are of high interest. For further
information on the ontology we point the reader to the related links.
15
http://wonderweb.semanticweb.org/, last accessed 27.07.2012
16
http://www.loa-cnr.it/ontologies/DLP397.owl, last accessed 05.09.2012
17
http://www.loa-cnr.it/ontologies/DUL.owl, last accessed 05.09.2012
4.2 Games with a Purpose
“Games with a purpose” were introduced by Luis von Ahn [34] as a way to
make use of the human computation for solving complicated tasks. As he ar-
gues, machine capabilities are limited in contrast to human reasoning capacities.
As a result, there is a need for human involvement. The core idea is that this
involvement be easy and motivating for users but at the same time efficient.
Two examples described by Luis von Ahn are the ESP game [35] where users
are labelling images in a simple web based game and the Peekaboom 18 game
for adding location information to the images. Further examples are the Listen
Game for the annotation of music [32] and the Phrase Detectives 19 game for
the annotation of text [5].
A detailed collection of different “games with a purpose” can be found in [31].
Also health sciences benefit from “games with a purpose”. For instance, the game
Foldit was developed for engaging the crowd in protein unfolding [8,11].
In the context of the semantic web, “games with a purpose” are used for
building ontologies such as the OntoGame [27]. Similarly to the annotation of
images game, wikipedia articles are shown to users who have to evaluate the
content of the text and summarize the content on a common way. A wider range
of “games with a purpose” for the semantic web can be found in [28].
For ontology alignment, SpotTheLink was designed as a continuation of the
OntoGame framework [30]. In this game, users are shown concepts and pictures
from DBpedia and then have to agree on one concept from the PROTON 20
ontology.
“Question games” (also seen as “question driven games”), is a type of “games
with a purpose” rooted back to the 20q 21 game where the computer tries to
guess the concept that a player has in mind based on his/her answers to certain
questions. This rational has been successfully used in [16] for ontology engineer-
ing purposes and specifically for knowledge acquisition. A knowledge engineer
was supposed to ask a domain expert up to 20 questions in order to obtain
important concepts and relations that had to be formalized in the ontology. A
similar technique has been used for aligning concepts to DOLCE in [18].
4.3 Question Game for OpenStreetMap
The current work, proposes the use of an interactive “question game” for aligning
OpenStreetMap tags to the DOLCE Ultralite ontology. The main prerequisite is
to hide the complexity of ontology from the users while designing a smart way
to align the tags to it. It would have been of little benefit to directly confront
users with concepts from the ontology like information object, designed artifact
etc. and ask for the direct alignment of their tags to them.
18
www.peekaboom.org, last accessed 27.07.2012
19
http://www.phrasedetectives.org, last accessed 27.07.2012
20
http://proton.semanticweb.org/, last accessed 27.07.2012
21
20q.net/, last accessed 27.07.2012
The “question game” plays this mediation role between the tags of the non-
expert and the well structured and formalized ontology. In such a way, users’
freedom of choosing tags is preserved. Anchoring the tags in the top level ontol-
ogy is catalytic for knowledge sharing and tag reconciliation and disambiguation.
The “question game” is part of the annotation process and its strategy is as
follows. Users who want to annotate a certain geographic feature, after deciding
which tag they prefer, have to answer some questions in a simple user interface.
These questions are simple and rather intuitive in order to be easily understood
and quickly answered. Each ontology concept is represented by one question.
Examples of these questions can be seen in Table 1.
1. Is it a (physical) object like a river or a stadium?
2. Is it a material, like sand or mud?
3. Is it a boundary of an area like an electoral division?
4. Does it imply some kind of action like swimming or dancing?
5. Can you observe and measure it?
6. Does it have a location?
7. ...
Table 1. Sample questions for the alignment process
The questions refer to the values of the tags. By answering a question, the
corresponding key of the tag is aligned as a class to the DOLCE Ultralite concept
and the value of the tag as a subclass to the corresponding key. For the first
prototype only boolean answers (yes and no) will be required. When a positive
answer is given, the tag is aligned to the DOLCE Ultralite concept related to
the question. Negative answers, trigger new questions until a positive answer is
given. Expected results of the alignment can be seen in Fig. 1.
As can be seen, the keys tourism, building, amenity, highway and leisure
are aligned to the class Designed Artifact. That is, they are grouped according to
their common criterion. As a result, their scattered listing in the OSM Features
wiki is organized facilitating easier findability of the tags and reference of their
meaning.
The purpose of the game is to directly align the tags to concepts from the
ontology. In this early stage, there is no interaction between users in order to
agree on a common tag which is a common practise in games with a purpose.
Moreover, the “question game” is not used in order to find out whether tags are
used correctly or how consistent they represent the POIs.
For motivating users, well known techniques such as ratings for top users
i.e. as seen in [25] or the geo-wiki project 22 are applied. This commonly used
technique is documented in [20] as the “glory or recognition motivator”.
Concerning the evaluation of the game possible options are usability test
measuring the level of ease and fun of the game. Analysis of the aligned tags can
22
http://www.geo-wiki.org/login.php?menu=home, last accessed 27.07.2012
Fig. 1. Tags aligned to DOLCE+ DnS Ultralite
show agreement or disagreement between users i.e. for the same tag what kind
of answers users provide and to which alignment it leads. Results can then be
evaluated with the aid of domain experts.
By introducing the “question game” for the alignment process, a solution is
found that allows users to keep the freedom of choice for their tags while at the
same time it enables the anchoring of them in the top level ontology.
5 Conclusion and Future Work
In this paper we have shown a way to bridge the gap between top-down onto-
logical and bottom-up crowdsourcing practices. OpenStreetMap is chosen as a
representative, widely used example of VGI. Although the freedom of assigning
tags to OpenStreetMap is very appealing and encouraging for users to contribute
their geodata, it humbles information search and retrieval. The community tries
to stabilize a common agreement on tags and an appropriate way that they be
used; however a proper order is missing. As a result, implicit knowledge (i.e.
inherent characteristics between classes and subclasses) cannot be unfolded.
The need for ordering the OpenStreetMap tags and constraining their mean-
ing, was fulfilled with the alignment of the tags to the DOLCE Ultralite top
level ontology. DOLCE Ultralite has a cognitive and linguistic orientation and
was therefore preferable to other top level ontologies. A bottom-up method is
used for the alignment process preserving the open and dynamic character of
OpenStreetMap.
With the aid of a “question game”, users are guided to align their tags to
DOLCE concepts. Keys and values of OpenStreetMap are translated into classes
and subclasses respectively and then anchored. Users are neither confronted with
the concepts of DOLCE Ultralite nor the DOLCE Ultralite hierarchy so that the
simple and common tagging procedure is maintained.
Opportunities for future work comprise the use of sophisticated reasoning
mechanisms to derive the implicit knowledge from the alignment result. Espe-
cially the analysis of inconsistencies between tags would provide evidence about
which geographic entities are perceived and used differently among the users.
For instance, if the alignment results show that the same tag is anchored in dif-
ferent DOLCE Ultralite concepts it can be inferred that the geographic entity it
describes, is conceptualized in heterogeneous ways by each user.
With additional analysis of these findings, new knowledge could be derived.
On a higher generalization level, this would be a way to derive a better un-
derstanding of how users conceptualize geographic entities providing insights to
spatial cognition.
Given the fact that OpenStreetMap is a multilingual project one research
question to be further investigated, would be whether tags in different languages
are aligned to the same DOLCE Ultralite concept or not. Such an analysis could
assist in investigating if there are differences in the conceptualization of the same
POI in different cultures.
Acknowledgments
The research leading to these results has received funding from the European
Union Seventh Framework Programme - Marie Curie Actions, Initial Train-
ing Network GEOCROWD under grant agreement n FP7-PEOPLE-2010-ITN-
264994.
References
1. S. Angeletou, M. Sabou, L. Specia, and E. Motta. Bridging the gap between
folksonomies and the semantic web: An experience report. In Workshop: Bridging
the Gap between Semantic Web and Web 2.0, European Semantic Web Conference,
page 93, 2007.
2. S. Auer, J. Lehmann, and S. Hellmann. LinkedGeoData - adding a spatial dimen-
sion to the web of data. In Proc. of 8th International Semantic Web Conference
(ISWC), 2009.
3. A. Ballatore and M. Bertolotto. Semantically enriching vgi in support of implicit
feedback analysis. In Proceedings of the 10th international conference on Web
and wireless geographical information systems, W2GIS’11, pages 78–93, Berlin,
Heidelberg, 2011. Springer-Verlag.
4. G. Begelman, P. Keller, and F. Smadja. Automated tag clustering: Improving
search and exploration in the tag space. In Collaborative Web Tagging Workshop
at WWW2006, Edinburgh, Scotland, pages 22–26, 2006.
5. J. Chamberlain, M. Poesio, and U. Kruschwitz. Phrase detectives: A web-based
collaborative annotation game. In Proceedings of the International Conference on
Semantic Systems (I-Semantics 08), Graz, 2008.
6. M. Codescu, G. Horsinka, O. Kutz, T. Mossakowski, and R. Rau. Do-roam:
activity-oriented search and navigation with openstreetmap. GeoSpatial Semantics,
pages 88–107, 2011.
7. M. Codescu, G. Horsinka, O. Kutz, T. Mossakowski, and R. Rau. Osmonto-an
ontology of openstreetmap tags. State of the map Europe (SOTM-EU) 2011, 2011.
8. S. Cooper, F. Khatib, A. Treuille, J. Barbero, J. Lee, M. Beenen, A. Leaver-Fay,
D. Baker, Z. Popovic, et al. Predicting protein structures with a multiplayer online
game. Nature, 466(7307):756–760, 2010.
9. S. Cosat. Openstreetmap-the best map. http://opengeodata.org/
openstreetmap-the-best-map, 2010. Last accessed: 27.07.2012.
10. A. Gangemi, N. Guarino, A. Masolo, C.and Oltramari, and L. Schneider. Sweeten-
ing ontologies with dolce. In Proceedings of the 13th International Conference on
Knowledge Engineering and Knowledge Management. Ontologies and the Semantic
Web, EKAW ’02, pages 166–181, London, UK, UK, 2002. Springer-Verlag.
11. B.M. Good and A.I. Su. Games with a scientific purpose. Genome Biology,
12(12):135, 2011.
12. M.F. Goodchild. Citizens as sensors: the world of volunteered geography. Geo-
Journal, 69(4):211–221, 2007.
13. M. Haklay and P. Weber. Openstreetmap: User-generated street maps. Pervasive
Computing, IEEE, 7(4):12–18, 2008.
14. M. Kavouras and M. Kokla. Theories of Geographic Concepts: Ontological Ap-
proaches to Semantic Integration. Taylor & Francis, 2007.
15. C.S.G. Khoo and J.C. Na. Semantic relations in information science. Annual
Review of Information Science and Technology, 40:157, 2006.
16. E. Klien. Deliverable D3.1 Ontologies in the swing application requirement spec-
ification. Technical report, FP6-26514 Project: Semantic Web-Service Interoper-
ability for Geospatial Decision Making, March 2007.
17. D. Laniado, D. Eynard, and M. Colombetti. Using wordnet to turn a folksonomy
into a hierarchy of concepts. In Semantic Web Application and Perspectives -
Fourth Italian Semantic Web Workshop, Bari Italy, pages 192–201, December 2007.
18. K. Latif, E. Weippl, and A. M. Tjoa. Question driven semantics interpretation for
collaborative knowledge engineering and ontology reuse. In IRI, pages 170–176.
IEEE Systems, Man, and Cybernetics Society, 2007.
19. H. Lin and J. Davis. Computational and crowdsourcing methods for extracting
ontological structure from folksonomy. In Proceedings of the 7th international
conference on The Semantic Web: research and Applications - Volume Part II,
ESWC’10, pages 472–477, Berlin, Heidelberg, 2010. Springer-Verlag.
20. T.W. Malone, R. Laubacher, and C. Dellarocas. Harnessing crowds: Mapping the
genome of collective intelligence. Research Paper No. 4732-09, MIT, Sloan School
of Management, Massachusetts Institute of Technology, Cambridge, MA, USA,
February 2009.
21. C. Masolo, S. Borgo, A. Gangemi, N. Guarino, and A. Oltramari. WonderWeb
deliverable D18 ontology library (final). Technical report, IST Project 2001-33052
WonderWeb: Ontology Infrastructure for the Semantic Web, 2003.
22. P. Mooney and P. Corcoran. Annotating spatial features in open-streetmap. Pro-
ceedings of the 19th annual gis research uk (gisruk), Portsmouth, England, 2011.
23. P. Mooney and P. Corcoran. Characteristics of heavily edited objects in open-
streetmap. Future Internet, 4(1):285–305, 2012.
24. A. Passant. Using ontologies to strengthen folksonomies and enrich information
retrieval in weblogs. In International Conference on Weblogs and Social Media,
Boulder, Colorado. ICWSM, March 2007.
25. N. Prestopnik and K. Crowston. Exploring collective intelligence games with design
science: A citizen science design case. In ACM Group Conference, Sanibel Island,
FL, October 2012. ACM.
26. S. Scheider, C. Kessler, J. Ortmann, A. Devaraju, J. Trame, T. Kauppinen, and
W. Kuhn. Semantic referencing of geosensor data and volunteered geographic
information. In Naveen Ashish and Amit P. Sheth, editors, Geospatial Semantics
and the Semantic Web, volume 12 of Semantic Web And Beyond Computing for
Human Experience, pages 27–59. Springer, 2011.
27. K. Siorpaes and M. Hepp. Ontogame: towards overcoming the incentive bottleneck
in ontology building. In On the Move to Meaningful Internet Systems 2007: OTM
2007 Workshops, pages 1222–1232. Springer, 2007.
28. K. Siorpaes and M. Hepp. Games with a purpose for the semantic web. 23:50–60,
2008.
29. L. Specia and E. Motta. Integrating folksonomies with the semantic web. In Proc.
of the European Semantic Web Conference (ESWC2007), volume 4519 of LNCS,
pages 624–639, Berlin Heidelberg, Germany, July 2007. Springer-Verlag.
30. S. Thaler, E. Simperl, and K. Siorpaes. Spotthelink: A game for ontology align-
ment. In Ronald Maier, editor, Proceedings of the 6th Conference for Professional
Knowledge Management WM, volume 182 of LNI, pages 246–253. GI, 2011.
31. S. Thaler, K. Siorpaes, E. Simperl, and C. Hofer. A survey on games for knowledge
acquisition. Technical report, Tech. Rep. STI TR 2011-05-01, Semantic Technology
Institute, 2011.
32. D. Turnbull, R. Liu, L. Barrington, and G. Lanckriet. A Game-Based Approach for
Collecting Semantic Annotations of Music. In Proceedings of the 8th International
Conference on Music Information Retrieval, Vienna, Austria, September 2007.
33. T. Vander Wal. Folksonomy coinage and definition. http://vanderwal.net/
folksonomy.html, 2007. Last accessed: 27.07.2012.
34. L. von Ahn. Games with a Purpose. Computer, 39(6):92–94, June 2006.
35. L. von Ahn and L. Dabbish. Labeling images with a computer game. In Proceedings
of the SIGCHI conference on Human factors in computing systems, pages 319–326.
ACM, 2004.