Semantifying OpenStreetMap Alkyoni Baglatzi, Margarita Kokla, Marinos Kavouras School of Rural and Surveying Engineering, National Technical University of Athens H. Polytechniou Str. 9, 15780 Zografos Campus, Greece alkyoni.baglatzi@gmail.com, (mkokla, mkav)@survey.ntua.gr Abstract. OpenStreetMap is one of the best examples of Volunteered Geographic Information. Its success relies on the ease of use and the freedom it provides. Users are supposed to geolocate their Points Of Interest and annotate them with a tag. There is no certain vocabulary or ontology of the tags that users have to commit to. The whole tagging process is done in a bottom-up manner in which the community on a wiki basis discusses this issue. Allowing users to use tags freely increases the usability of OpenStreetMap but at the same time causes semantic interoperability problems. What is needed, is a way to structure the tags while satisfying the freedom criterion. As a solution, we suggest the alignment of the tags to well structured top level ontologies. A middle layer approach for bridging the gap between the bottom-up tags of the users and the top level Descriptive Ontology for Linguistic and Cognitive Engineering (DOLCE) is proposed. The idea of “games with a purpose” is utilized to assists non-expert users in aligning their tags to DOLCE. Keywords: VGI, OpenStreetMap, tagging, semantics, alignment, games with a purpose, top level ontology 1 Introduction User generated content is an important emerging research area. Engaging the crowd in performing new tasks brings about new opportunities and new chal- lenges. In the geographic domain, the term Volunteered Geographic Information (VGI) was coined by Goodchild [12] for describing the collaborative mapping activities of users and contribution of geographic data. OpenStreetMap which was initiated at University College London (UCL) in July 2004 by Steve Coast, is one of the most pervasive and representative examples of VGI [13]. OpenStreetMap offers an open and easy to use platform that enables contrib- utors to upload geographic information collected from mobile devices or aerial images. The data model is simple and consists of nodes, ways and relations. Each mapped entity is accompanied with a tag. There is no formal ontology or vocabulary of predefined tags that have to be adopted by the users, because as argued by Steve Coast, the founder of OpenStreetMap: “no individual could design such an ontology that would be all-encompassing, and even if they could start no two individuals would agree on it[9]”. Tags that facilitate the annotation of Points of Interest (POIs) in Open- StreetMap come in key-value pairs 1 for instance, amenity=bar, natural=beach, landuse= forest. There is no standardized way on how users shall annotate their POIs neither on the naming level nor on which entities shall be tagged under a certain name. On a wiki 2 and mailing list 3 basis, the community exchanges opinions about the tags proposing new tags or tags that should be abolished. The most common tags in use, can be looked up in Taginfo 4 . The tags (or Map Features as found in the wiki) are listed in some kind of loose hierarchy. Although the freedom and openness provided eases the tagging procedure, it causes semantic interoperability problems. User generated content is heteroge- neous which leads to ambiguity, redundancy and inconsistency of the tags. As a result, findability of the correct tag for annotating a POI as well as information searching and retrieval is ineffective, an issue that has already been described for instance in [4]. What is needed, is the combination of this loose hierarchy with well struc- tured, organized, formalized top level ontologies and specifically the alignment of users’ tags to concepts of a top level ontology. Top level ontologies can be seen as a structured collection of semantic primitives or meta level concepts that are used to further define domain concepts [14]. By aligning the domain concepts to the top level ontologies, the meaning of these concepts gets grounded in the semantic primitives. This universal view of top level ontologies is also a reason why they are more suitable than domain ontologies for the alignment process. The meta level concepts act as reference points in relation to which, the domain concepts are defined. OpenStreetMap is open to non-expert users of geographic data and thus, the tagging attitude is rather intuitive than based on scientific methodologies and knowledge. This calls for an alignment to a top level ontology, which underly- ing design principles are prescribed by common sense. As has been argued by its creators, the Descriptive Ontology for Linguistic and Cognitive Engineering (DOLCE) [21] “has a clear cognitive bias, in the sense that it aims at capturing the ontological categories underlying natural language and human commonsense ”([10] p.2). That is the reason why in the present work the DOLCE ontology has been chosen. Specifically, the extension of DOLCE, DOLCE Ultralite 5 (DUL) was regarded as more suitable because it replaces the complicated endurant, per- durant division with object and event. Nevertheless, also other ontologies that satisfy this criterion could have been used instead. Aligning the tags to the top level ontology in a top-down approach with the aid of knowledge engineers would have been accurate but time consuming. Con- cerning maintenance, the dynamical nature of OpenStreetMap with the option 1 We use true type fonts to refer to tags 2 http://wiki.openstreetmap.org/wiki/Map_Features, last accessed 27.07.2012 3 http://lists.openstreetmap.org/listinfo, last accessed 27.07.2012 4 http://taginfo.openstreetmap.org/tags, last accessed 27.07.2012 5 http://www.loa-cnr.it/ontologies/DUL.owl, last accessed 05.09.2012 of new tags being introduced or old removed, would demand the repetition of the alignment procedure very often which is resource inefficient. Our research question is “How to find a user friendly way to align the tags to top level ontologies?”. We aim at defining a methodology that will enable the bottom-up alignment of tags to top level ontologies. For that, we propose the use of the well established idea of “games with a purpose” [34] applied to OpenStreetMap. Specifically, we choose “question games”, a certain type of “games with a purpose”, for assisting users in aligning their tags to the concepts of the top level ontology. We use DOLCE Ultralite, an extension of DOLCE to align the tags to. The contribution of this work is twofold: (a) it provides an analysis of the semantic inconsistencies that emerge from the current state of the tagging pro- cess in OpenStreetMap and (b) it proposes a way to combine top-down and bottom-up approaches by preserving the advantages of both that is, the freedom and easiness of the first and the structure, organization of the latter. For that, a “question game” is designed. The main objective is to reuse existing methods in order figure out a methodology for overcoming the semantic inconsistencies of OpenStreetMap. The remainder of the paper is organized as follows. Section 2 provides infor- mation on the related work. Section 3 investigates some semantic inconsistencies that were found in the OpenStreetMap tags. Section 4 describes the proposed methodology for aligning the tags to the top level ontology and Section 5 con- cludes the paper and discusses possible future directions. 2 Related Work With the increase of user involvement in the web and the rising amount of user generated content, tagging was introduced for annotating purposes. Flickr 6 , del.icio.us 7 , citeulike 8 and youtube 9 are just few examples where users added tags to describe their content. The tagging behaviour has been examined in multifarious ways with several methods in order to understand the commonsense ground of user generated con- tent. Thomas Vander Wal introduced the term folksonomy for describing this collaborative tagging [33]. Clustering methods have been used to investigate in a bottom-up manner kinds of tags people use in their annotations. Ontology learning has substantially benefited from these kind of studies as they unfolded the way people perceive and use different tags. As a result, on- tologies can be designed more efficiently. The need for combining well structured ontologies with hierarchically loose folksonomies has already been acknowledged. Especially the problem of the miss- ing relations between tags in folksonomies has been addressed in [1]. With the 6 http://www.flickr.com/, last accessed 27.07.2012 7 http://delicious.com/, last accessed 27.07.2012 8 http://www.citeulike.org/, last accessed 27.07.2012 9 http://www.youtube.com/, last accessed 27.07.2012 aid of ontologies, different kinds of relations between tags like subsumption re- lations, disjointness relations, generic relations, sibling relations and instance of relations were found and added. Data mining techniques and ontologies are combined in [29] to make the semantics of the tags explicit. Tags are preprocessed, clustered and related to concepts in different ontologies. Swoogle 10 is used as a search engine for finding appropriate ontologies. In [19] knowledge from folksonomies is extracted with data mining techniques and related to upper ontologies. After a preprocessing step, tags are related to WordNet 11 and enriched with its relations. The methodology is applied to datasets from flickr and citeulike. Aligning folksonomies to domain ontologies is utilized in [24] for annotating blog posts. The main goal is to limit tags’ ambiguity and variation. In the first step, users are free to choose tags for annotating the blog posts. In the second step, ontology concepts are shown to them and in an interactive, semi-automatic manner, users are asked to relate their tags to the concepts from these ontolo- gies. Tags have to be to explicitly matched to concepts from the ontologies. WordNet is used in [17] to order tags in a hierarchical way. A tool has been built that makes the navigation in tag spaces more comprehensive for users. As a result, browsing and retrieving related tags is made easier and more efficient. Concerning OpenStreetMap, important research has been conducted in analy- sing the tagging behaviour of users i.e. which the most edited entities are and how they change over time [22,23]. This provides evidence about the importance of certain entities and the different ways they are perceived by the users broad- ening the research agenda of user generated geospatial content. The power of enriching OpenStreetMap with other sources has been demon- strated in the LinkedGeoData project [2]. Instances of OpenStreetMap are pub- lished according to the linked data principles and linked to DBpedia12 . The problem of missing relatedness between geographic entities is addressed in [3]. OpenStreetMap “spatially rich but semantically poor vector dataset” and DBpedia “spatially poor but semantically rich ontology”, are combined providing the user with information about a geographic entity. An important contribution is the consideration of map scale in relating concepts to geographic entities. OSMonto13 has been developed as an OWL ontology of OpenStreetMap tags [7]. Keys are translated into classes and values into subclasses. The design deci- sion was to be as close as possible to the tagging process enabling querying of the OpenStreetMap database. That is why the tags were adopted as represented in the tag wiki and no conceptual conflicts or ontological mismatches were con- fronted. OSMonto is used in the DO-ROAM 14 project which aims at expanding 10 http://swoogle.umbc.edu/ last accessed 27.07.2012 11 http://wordnet.princeton.edu/ last accessed 27.07.2012 12 http://wiki.dbpedia.org/About last accessed 27.07.2012 13 https://raw.github.com/doroam/planning-do-roam/master/Ontology/tags. owl, last accessed 27.07.2012 14 http://planning.do-roam.org/, last accessed 27.07.2012 the search capabilities not only to the POIs but also to the activities that can be performed at a certain location [6]. Scheider et al. [26] argue for the need of more functional or affordance ori- ented representation of OpenStreetMap tags. They underline the fact that the current tagging practise in OpenStreetMap is not efficient enough for repre- senting the identity of the POIs. As they mention, tagging a cafeteria which is also open at night serving alcohol with amenity=cafe may exclude the alcohol- serving functionality. They suggest that tags should be grounded in affordances. Accompanying the tags with richer descriptions on the affordances of the POIs, may harden the annotation process but will make the querying process more efficient. 3 Semantic Inconsistencies of OpenStreetMap Tagging The freedom and easiness of assigning tags to POIs is a hallmark of the suc- cess of OpenStreetMap. But, on the semantics of the tags, it leads to several interoperability problems. As this tag collection is a result of a bottom-up user generated effort, it lacks some proper semantic structure. Especially the absence of relations like the hypernymic relation which describe the is-a relation between a concept and its genus and the meronymic relation that is the part-of relation both acting “as the cement that links up concepts into knowledge structures”[15] makes the annotation and searching process cumbersome. In contrast to geographic ontologies, vocabularies, taxonomies created by do- main experts, the key-value pairs are organized in a loose way. Although there is some type of clustering or thematic grouping of the tags in the wiki, which differentiates it from traditional folksonomies where hierarchical information is missing, conceptual inconsistencies still exist. This section aims at providing some examples of the semantic inconsistencies that arise from the tagging strat- egy of OpenStreetMap. To start with, there is no common criterion according to which the tags are organized. As a result all keys are in the same hierarchical level. That is, all pri- mary features as listed in the wiki i.e. amenity, aeroway, historic, landuse, manmade, craft, sport, tourism, power, shop etc. are treated equally, which results in conceptual vagueness and inconsistency. For instance, the nature of office or shop is manmade. By not relating these tags to the tag manmade with the class-subclass relation, important inheritance information gets lost. Same applies to i.e. shop, office, building which could be subclasses of amenity. On a more sophisticated level, a disadvantage of the flat structure of the tags is the fact that no deeper associations between geographic entities can be estab- lished. For instance, in the current tagging procedure there is no way of explicitly stating that within landuse=commercial, geographic entities like shop=bakery, office=architect are located. Redundancies of tags provide us with evidence about the different conceptu- alizations of certain POIs that users have. For instance, hotel, hospital, school are tagged as tourism=hotel, building=hotel, amenity=hospital, building=ho- spital, amenity=school, building=school. Users assign the same value (be it hotel, hospital or school), to two different keys namely tourism and amenity. Another finding is that tags related to activities are used to describe POIs i.e. sport=climbing, sport=basketball, leisure=dance, leisure=fishing. While describing the activity that can be performed at a certain POI, they are used to annotate the POI itself. This may be confusing in terms of information search as it perplexes the geographic entity with its function. The primary tag amenity also creates confusion since it is used to represent a wide variety of heterogeneous features (e.g., schools, parking lots, bus stations, banks, hospitals, nightclubs, etc.). Although different subcategories of amenities are defined in the wiki (such as sustenance, education, transportation, financial, healthcare, etc.) to further classify different types of amenities, their possible values (such as school, bar, embassy, etc.) directly refer to the general tag amenity, e.g., amenity=school, amenity=bar, amenity=embassy. The above cases are some examples of the semantic problems caused by a bottom-up, intuitive approach. Furthermore, although the proposed tags have resulted from consensus, they do not necessarily represent a common and wide conceptualization of geographic concepts. For this reason, although such ap- proaches have stimulated considerable interest and resulted in the collection of huge volumes of geospatial data, they are accompanied by problems, such as the creation of arbitrary attributes or attribute values, multiple tags for the same geographic features, disagreement on the name of features, etc. [22]. The present research aims at the design of a “game with a purpose” to align the loose hierar- chy of OpenStreet Map tags with a well structured top level ontology to provide meaningful and cognitively important associations between tags. 4 Question Game for Aligning OpenStreetMap Tags to DOLCE Top Level Ontology 4.1 Introduction to the DOLCE Ultralite Top Level Ontology DOLCE [21] is a foundational or top level ontology developed within the WON- DERWEB project15 . It is an ontology of particulars and comes in different ver- sions DOLCE Lite-Plus 16 and DOLCE+ DnS Ultralite 17 (or DOLCE Ultralite). In the present work, the DOLCE Ultralite was chosen because its categories and organization is simpler and more intuitive than the other versions as ar- gued by its authors. The top concept entity is categorized into abstract, event, information entity, object and quality. For the alignment, the class object and es- pecially its subclasses physical and social object are of high interest. For further information on the ontology we point the reader to the related links. 15 http://wonderweb.semanticweb.org/, last accessed 27.07.2012 16 http://www.loa-cnr.it/ontologies/DLP397.owl, last accessed 05.09.2012 17 http://www.loa-cnr.it/ontologies/DUL.owl, last accessed 05.09.2012 4.2 Games with a Purpose “Games with a purpose” were introduced by Luis von Ahn [34] as a way to make use of the human computation for solving complicated tasks. As he ar- gues, machine capabilities are limited in contrast to human reasoning capacities. As a result, there is a need for human involvement. The core idea is that this involvement be easy and motivating for users but at the same time efficient. Two examples described by Luis von Ahn are the ESP game [35] where users are labelling images in a simple web based game and the Peekaboom 18 game for adding location information to the images. Further examples are the Listen Game for the annotation of music [32] and the Phrase Detectives 19 game for the annotation of text [5]. A detailed collection of different “games with a purpose” can be found in [31]. Also health sciences benefit from “games with a purpose”. For instance, the game Foldit was developed for engaging the crowd in protein unfolding [8,11]. In the context of the semantic web, “games with a purpose” are used for building ontologies such as the OntoGame [27]. Similarly to the annotation of images game, wikipedia articles are shown to users who have to evaluate the content of the text and summarize the content on a common way. A wider range of “games with a purpose” for the semantic web can be found in [28]. For ontology alignment, SpotTheLink was designed as a continuation of the OntoGame framework [30]. In this game, users are shown concepts and pictures from DBpedia and then have to agree on one concept from the PROTON 20 ontology. “Question games” (also seen as “question driven games”), is a type of “games with a purpose” rooted back to the 20q 21 game where the computer tries to guess the concept that a player has in mind based on his/her answers to certain questions. This rational has been successfully used in [16] for ontology engineer- ing purposes and specifically for knowledge acquisition. A knowledge engineer was supposed to ask a domain expert up to 20 questions in order to obtain important concepts and relations that had to be formalized in the ontology. A similar technique has been used for aligning concepts to DOLCE in [18]. 4.3 Question Game for OpenStreetMap The current work, proposes the use of an interactive “question game” for aligning OpenStreetMap tags to the DOLCE Ultralite ontology. The main prerequisite is to hide the complexity of ontology from the users while designing a smart way to align the tags to it. It would have been of little benefit to directly confront users with concepts from the ontology like information object, designed artifact etc. and ask for the direct alignment of their tags to them. 18 www.peekaboom.org, last accessed 27.07.2012 19 http://www.phrasedetectives.org, last accessed 27.07.2012 20 http://proton.semanticweb.org/, last accessed 27.07.2012 21 20q.net/, last accessed 27.07.2012 The “question game” plays this mediation role between the tags of the non- expert and the well structured and formalized ontology. In such a way, users’ freedom of choosing tags is preserved. Anchoring the tags in the top level ontol- ogy is catalytic for knowledge sharing and tag reconciliation and disambiguation. The “question game” is part of the annotation process and its strategy is as follows. Users who want to annotate a certain geographic feature, after deciding which tag they prefer, have to answer some questions in a simple user interface. These questions are simple and rather intuitive in order to be easily understood and quickly answered. Each ontology concept is represented by one question. Examples of these questions can be seen in Table 1. 1. Is it a (physical) object like a river or a stadium? 2. Is it a material, like sand or mud? 3. Is it a boundary of an area like an electoral division? 4. Does it imply some kind of action like swimming or dancing? 5. Can you observe and measure it? 6. Does it have a location? 7. ... Table 1. Sample questions for the alignment process The questions refer to the values of the tags. By answering a question, the corresponding key of the tag is aligned as a class to the DOLCE Ultralite concept and the value of the tag as a subclass to the corresponding key. For the first prototype only boolean answers (yes and no) will be required. When a positive answer is given, the tag is aligned to the DOLCE Ultralite concept related to the question. Negative answers, trigger new questions until a positive answer is given. Expected results of the alignment can be seen in Fig. 1. As can be seen, the keys tourism, building, amenity, highway and leisure are aligned to the class Designed Artifact. That is, they are grouped according to their common criterion. As a result, their scattered listing in the OSM Features wiki is organized facilitating easier findability of the tags and reference of their meaning. The purpose of the game is to directly align the tags to concepts from the ontology. In this early stage, there is no interaction between users in order to agree on a common tag which is a common practise in games with a purpose. Moreover, the “question game” is not used in order to find out whether tags are used correctly or how consistent they represent the POIs. For motivating users, well known techniques such as ratings for top users i.e. as seen in [25] or the geo-wiki project 22 are applied. This commonly used technique is documented in [20] as the “glory or recognition motivator”. Concerning the evaluation of the game possible options are usability test measuring the level of ease and fun of the game. Analysis of the aligned tags can 22 http://www.geo-wiki.org/login.php?menu=home, last accessed 27.07.2012 Fig. 1. Tags aligned to DOLCE+ DnS Ultralite show agreement or disagreement between users i.e. for the same tag what kind of answers users provide and to which alignment it leads. Results can then be evaluated with the aid of domain experts. By introducing the “question game” for the alignment process, a solution is found that allows users to keep the freedom of choice for their tags while at the same time it enables the anchoring of them in the top level ontology. 5 Conclusion and Future Work In this paper we have shown a way to bridge the gap between top-down onto- logical and bottom-up crowdsourcing practices. OpenStreetMap is chosen as a representative, widely used example of VGI. Although the freedom of assigning tags to OpenStreetMap is very appealing and encouraging for users to contribute their geodata, it humbles information search and retrieval. The community tries to stabilize a common agreement on tags and an appropriate way that they be used; however a proper order is missing. As a result, implicit knowledge (i.e. inherent characteristics between classes and subclasses) cannot be unfolded. The need for ordering the OpenStreetMap tags and constraining their mean- ing, was fulfilled with the alignment of the tags to the DOLCE Ultralite top level ontology. DOLCE Ultralite has a cognitive and linguistic orientation and was therefore preferable to other top level ontologies. A bottom-up method is used for the alignment process preserving the open and dynamic character of OpenStreetMap. With the aid of a “question game”, users are guided to align their tags to DOLCE concepts. Keys and values of OpenStreetMap are translated into classes and subclasses respectively and then anchored. Users are neither confronted with the concepts of DOLCE Ultralite nor the DOLCE Ultralite hierarchy so that the simple and common tagging procedure is maintained. Opportunities for future work comprise the use of sophisticated reasoning mechanisms to derive the implicit knowledge from the alignment result. Espe- cially the analysis of inconsistencies between tags would provide evidence about which geographic entities are perceived and used differently among the users. For instance, if the alignment results show that the same tag is anchored in dif- ferent DOLCE Ultralite concepts it can be inferred that the geographic entity it describes, is conceptualized in heterogeneous ways by each user. With additional analysis of these findings, new knowledge could be derived. On a higher generalization level, this would be a way to derive a better un- derstanding of how users conceptualize geographic entities providing insights to spatial cognition. Given the fact that OpenStreetMap is a multilingual project one research question to be further investigated, would be whether tags in different languages are aligned to the same DOLCE Ultralite concept or not. Such an analysis could assist in investigating if there are differences in the conceptualization of the same POI in different cultures. Acknowledgments The research leading to these results has received funding from the European Union Seventh Framework Programme - Marie Curie Actions, Initial Train- ing Network GEOCROWD under grant agreement n FP7-PEOPLE-2010-ITN- 264994. References 1. S. Angeletou, M. Sabou, L. Specia, and E. Motta. Bridging the gap between folksonomies and the semantic web: An experience report. In Workshop: Bridging the Gap between Semantic Web and Web 2.0, European Semantic Web Conference, page 93, 2007. 2. S. Auer, J. Lehmann, and S. Hellmann. LinkedGeoData - adding a spatial dimen- sion to the web of data. In Proc. of 8th International Semantic Web Conference (ISWC), 2009. 3. A. Ballatore and M. Bertolotto. Semantically enriching vgi in support of implicit feedback analysis. In Proceedings of the 10th international conference on Web and wireless geographical information systems, W2GIS’11, pages 78–93, Berlin, Heidelberg, 2011. Springer-Verlag. 4. G. Begelman, P. Keller, and F. Smadja. Automated tag clustering: Improving search and exploration in the tag space. In Collaborative Web Tagging Workshop at WWW2006, Edinburgh, Scotland, pages 22–26, 2006. 5. J. Chamberlain, M. Poesio, and U. Kruschwitz. Phrase detectives: A web-based collaborative annotation game. In Proceedings of the International Conference on Semantic Systems (I-Semantics 08), Graz, 2008. 6. M. Codescu, G. Horsinka, O. Kutz, T. Mossakowski, and R. Rau. Do-roam: activity-oriented search and navigation with openstreetmap. GeoSpatial Semantics, pages 88–107, 2011. 7. M. Codescu, G. Horsinka, O. Kutz, T. Mossakowski, and R. Rau. Osmonto-an ontology of openstreetmap tags. State of the map Europe (SOTM-EU) 2011, 2011. 8. S. Cooper, F. Khatib, A. Treuille, J. Barbero, J. Lee, M. Beenen, A. Leaver-Fay, D. Baker, Z. Popovic, et al. Predicting protein structures with a multiplayer online game. Nature, 466(7307):756–760, 2010. 9. S. Cosat. Openstreetmap-the best map. http://opengeodata.org/ openstreetmap-the-best-map, 2010. Last accessed: 27.07.2012. 10. A. Gangemi, N. Guarino, A. Masolo, C.and Oltramari, and L. Schneider. Sweeten- ing ontologies with dolce. In Proceedings of the 13th International Conference on Knowledge Engineering and Knowledge Management. Ontologies and the Semantic Web, EKAW ’02, pages 166–181, London, UK, UK, 2002. Springer-Verlag. 11. B.M. Good and A.I. Su. Games with a scientific purpose. Genome Biology, 12(12):135, 2011. 12. M.F. Goodchild. Citizens as sensors: the world of volunteered geography. Geo- Journal, 69(4):211–221, 2007. 13. M. Haklay and P. Weber. Openstreetmap: User-generated street maps. Pervasive Computing, IEEE, 7(4):12–18, 2008. 14. M. Kavouras and M. Kokla. Theories of Geographic Concepts: Ontological Ap- proaches to Semantic Integration. Taylor & Francis, 2007. 15. C.S.G. Khoo and J.C. Na. Semantic relations in information science. Annual Review of Information Science and Technology, 40:157, 2006. 16. E. Klien. Deliverable D3.1 Ontologies in the swing application requirement spec- ification. Technical report, FP6-26514 Project: Semantic Web-Service Interoper- ability for Geospatial Decision Making, March 2007. 17. D. Laniado, D. Eynard, and M. Colombetti. Using wordnet to turn a folksonomy into a hierarchy of concepts. In Semantic Web Application and Perspectives - Fourth Italian Semantic Web Workshop, Bari Italy, pages 192–201, December 2007. 18. K. Latif, E. Weippl, and A. M. Tjoa. Question driven semantics interpretation for collaborative knowledge engineering and ontology reuse. In IRI, pages 170–176. IEEE Systems, Man, and Cybernetics Society, 2007. 19. H. Lin and J. Davis. Computational and crowdsourcing methods for extracting ontological structure from folksonomy. In Proceedings of the 7th international conference on The Semantic Web: research and Applications - Volume Part II, ESWC’10, pages 472–477, Berlin, Heidelberg, 2010. Springer-Verlag. 20. T.W. Malone, R. Laubacher, and C. Dellarocas. Harnessing crowds: Mapping the genome of collective intelligence. Research Paper No. 4732-09, MIT, Sloan School of Management, Massachusetts Institute of Technology, Cambridge, MA, USA, February 2009. 21. C. Masolo, S. Borgo, A. Gangemi, N. Guarino, and A. Oltramari. WonderWeb deliverable D18 ontology library (final). Technical report, IST Project 2001-33052 WonderWeb: Ontology Infrastructure for the Semantic Web, 2003. 22. P. Mooney and P. Corcoran. Annotating spatial features in open-streetmap. Pro- ceedings of the 19th annual gis research uk (gisruk), Portsmouth, England, 2011. 23. P. Mooney and P. Corcoran. Characteristics of heavily edited objects in open- streetmap. Future Internet, 4(1):285–305, 2012. 24. A. Passant. Using ontologies to strengthen folksonomies and enrich information retrieval in weblogs. In International Conference on Weblogs and Social Media, Boulder, Colorado. ICWSM, March 2007. 25. N. Prestopnik and K. Crowston. Exploring collective intelligence games with design science: A citizen science design case. In ACM Group Conference, Sanibel Island, FL, October 2012. ACM. 26. S. Scheider, C. Kessler, J. Ortmann, A. Devaraju, J. Trame, T. Kauppinen, and W. Kuhn. Semantic referencing of geosensor data and volunteered geographic information. In Naveen Ashish and Amit P. Sheth, editors, Geospatial Semantics and the Semantic Web, volume 12 of Semantic Web And Beyond Computing for Human Experience, pages 27–59. Springer, 2011. 27. K. Siorpaes and M. Hepp. Ontogame: towards overcoming the incentive bottleneck in ontology building. In On the Move to Meaningful Internet Systems 2007: OTM 2007 Workshops, pages 1222–1232. Springer, 2007. 28. K. Siorpaes and M. Hepp. Games with a purpose for the semantic web. 23:50–60, 2008. 29. L. Specia and E. Motta. Integrating folksonomies with the semantic web. In Proc. of the European Semantic Web Conference (ESWC2007), volume 4519 of LNCS, pages 624–639, Berlin Heidelberg, Germany, July 2007. Springer-Verlag. 30. S. Thaler, E. Simperl, and K. Siorpaes. Spotthelink: A game for ontology align- ment. In Ronald Maier, editor, Proceedings of the 6th Conference for Professional Knowledge Management WM, volume 182 of LNI, pages 246–253. GI, 2011. 31. S. Thaler, K. Siorpaes, E. Simperl, and C. Hofer. A survey on games for knowledge acquisition. Technical report, Tech. Rep. STI TR 2011-05-01, Semantic Technology Institute, 2011. 32. D. Turnbull, R. Liu, L. Barrington, and G. Lanckriet. A Game-Based Approach for Collecting Semantic Annotations of Music. In Proceedings of the 8th International Conference on Music Information Retrieval, Vienna, Austria, September 2007. 33. T. Vander Wal. Folksonomy coinage and definition. http://vanderwal.net/ folksonomy.html, 2007. Last accessed: 27.07.2012. 34. L. von Ahn. Games with a Purpose. Computer, 39(6):92–94, June 2006. 35. L. von Ahn and L. Dabbish. Labeling images with a computer game. In Proceedings of the SIGCHI conference on Human factors in computing systems, pages 319–326. ACM, 2004.