A pre-requisite for the Semantic Web to become a reality is the availability of ontologies [1] and meta-data. In many cases, it might be necessary to align between different ontologies in order to ensure interoperability. The research in the area of semantic content authoring has brought up an inventory of mature techniques and tools for semantic content creation. However, there is a severe lack of semantic data available on the Web: one can only find few well-maintained ontologies, respective alignments and very little semantic annotation. For instance, a search on Watson1 or Swoogle2 for a tourism ontology does not deliver a proper tourism ontology even though travel and tourism ontologies have been created in many academic projects in the last couple years. Furthermore, one can observe very little involvement of Web users in the process of semantic content creation. However, this involvement is urgently needed: there are tasks that are trivial for a human user but still difficult for a computer [2, 3]. Conceptual modeling and semantic annotation are tasks that depend on human intelligence: even though approaches for automating these activities exist, the problem has not been solved completely yet and human input is required at some stage. Therefore, we are now confronted with the situation that even though the technology is available, there is very little semantic content which can be traced back to only little user involvement. We believe that this is caused by missing incentive structures: the effort of building ontologies currently outweighs the benefit.
This is in sharp contrast to the Web 2.0 movement, which has proper incentive structures in place [4-6]. In my thesis, I investigate intrinsic motivations of users for contributing to Web 2.0 applications and propose to define possible incentive models for the Semantic Web. More precisely, I propose to masquerade core tasks of weaving the Semantic Web behind on-line, multi-player game scenarios, in order to create proper incentives for humans to contribute. Doing so, I adopt the findings from the already famous “games with a purpose” by von Ahn [2], who has shown that presenting a useful task, which requires human intelligence, in the form of an on-line game can motivate a large amount of people to work heavily on this task, and this for free.
The contribution of my thesis is (1) an overview of incentives for users to contribute to Web 2.0 applications, (2) a survey on serious games and games with a purpose, (3) a conceptual framework that aims at (a) defining incentives (more precisely, intrinsic motivations) for the Semantic Web and (b) describing how to hide semantic content creation and maintenance tasks behind online games. Furthermore, I will provide (3) a proof-of-concept implementation with four cool games scenarios that will be available to the general public. Finally, I will (4) evaluate the fun factor of the games and (5) analyze the output of the games checking the correctness and the usefulness of the resulting data. OntoGame is an approach to the massive generation of lightweight knowledge structures that can serve as a starting point for further axiomatization, as training sets for semiautomatic approaches, and that can be useful for machine learning techniques.
Several “games with a purpose” have been described by Luis von Ahn and colleagues; they also coined the term “human computation”: The ESP game [7] aims at labeling images on the Web - two players, who do not know each other, have to come up with identical tags describing an image. Peekaboom [8] works similar and has the objective of locating objects within images. Verbosity [9] is a game for collecting common sense facts. Phetch [10] is a computer game that collects explanatory descriptions of images in order to improve accessibility of the Web for the visually impaired. Law, von Ahn, and colleagues [11] came up with a game called Tagatune for music and sound annotation based on tags. However, their current prototypes remain mostly at the level of lexical resources only, i.e. terms and tags and are not directly connected with Semantic Web research.
Liebermann and colleagues describe the game Common Consensus [12], which aims at collecting human goals in order to recognize goals from user actions and conclude a sequence of actions from these goals.
Another approach to collecting common sense knowledge is the FACTory Game3 published by Cycorp4: FACTory is a single-player online game that randomly chooses facts from the Cyc knowledge base [13] and presents them to the players. The player has to say whether the statement is true, false, doesn’t make sense, or whether the user does not know. The answers are scored depending on accordance with the majority of answers.
A different type of games are so called passively multiplayer online games5, a term coined by Justin Hall. The idea of the PMOGs6 is to create avatars and game moves in multiplayer online games from user behavior on the Web. In other words, PMOGs translate e-mail content, chat logs, pictures, etc. into hunting parties, teams, puzzles, and so on.
In my PhD thesis, I propose to hide relevant tasks of semantic content authoring behind online games. In this section, I outline the challenges of my approach and the design
principles. Finally, I describe four cool OntoGame scenarios and explain how they address tasks in the Semantic Web lifecycle.
The design of games for building the Semantic Web involves several challenges: (1) Conceptual model of the games: it is crucial to make sure that the games are interesting and deliver useful output at the same time. This involves not only nice user interface design but also methods to keep up interest. One example for this would be revealing information about the partner (gender, age, nationality, etc.). (2) Input data: For most game scenarios, a large corpus of knowledge, such as Wikipedia or YouTube, is required. (3) Deriving formal semantics: from the games, formal semantics must be extracted, i.e. exports in common languages such as OWL. (4) Cheating: ways to avoid cheating must be described and implemented. (5) Re-use and analysis of generated data: in order to increase the amount of diverging data gathered as well as further deepening the degree of detail of the data, one must find algorithms and mechanisms to re-use gained data. (6) Typical Mistakes: from first experiments, it is obvious that there are some cases where users tend to make mistakes, i.e. classifying something as a sub-class of a concept that is not correct. One has to find ways to avoid the impact of these “false friends”.
Fun and the game challenge are the predominant user experience. The actual tasks are very well hidden such that their serious and useful nature does not decrease the “fun factor”. Additionally, the games should comprise an intellectual challenge being fun and interesting at the same time.
In our games, I adopt the “Wisdom of crowds” [14] paradigm. Groups only perform well under certain conditions: the group must be diverse, geographically dispersed, and members must be unable to influence each other. The settings of our games fulfill these requirements in order to tap the “wisdom of crowds”.
The assumptions about the intelligence of groups are only true given mass participation. Our games aim at the massive generation of semantic content, and thus mass user participation.
In order to evaluate the set of abstract game scenarios, four games were implemented7 that address the whole Semantic Web lifecycle (Fig. 1): certain tasks involved in ontology construction, alignment, and semantic annotation can be hidden behind online games. In a 7 OntoPronto is released, OntoTube and SpotTheLink are close to release, OntoBay implementation is starting now. All four scenarios can be expected by summer 2008 latest. nutshell, the OntoGame8 series includes the following games: OntoPronto is a game for annotating Wikipedia and for creating a huge general interest ontology (the English Wikipedia currently contains more than 2 Million articles). SpotTheLink aims at aligning the product and service classifications eCl@ss and UNSPSC, respectively their OWL counterparts eClassOWL and unspscOWL. OntoTube (Fig. 2) produces annotations for YouTube videos. A fourth upcoming scenario is called OntoBay and is a game for annotating eBay auctions by expressing the type of goods being offered using eClassOWL.9
The objective of the evaluation is twofold: (1) to evaluate whether our first prototype creates an entertaining gaming experience and (2) whether the consensual conceptual choices of players in the games are correct. My hypothesis is that the majority of the players’ decisions are ontologically correct and players will enjoy the games. We plan to release the four prototypes to the general public on several game platforms and make many users play the games. I will then analyze the resulting data: absolute number of games, absolute number of resources (Wikipedia articles, YouTube videos, etc.), ratio of singleplayer games, time invested by users, figures about the degree of consensus, and most importantly, the quality of conceptual salutation, i.e. mistakes that were made. For this purpose I will take representative samples and will ask experts to judge the correctness of the data. Furthermore, I will conduct surveys among players evaluating the fun factor, similar to the survey described in [15].
Preliminary evidence [15] indicates that this hypothesis is correct: OntoPronto, the first game of the OntoGame series, was released to the general public in Dec. 2007. Within the first two days, more than 200 players registered and played the game. The results of the analysis are promising: players make few mistakes and manage to find consensus in the majority of cases.
Designers of semantic applications should start to think about incentives for users to invest time in those applications: in my thesis, I will provide helpful guidelines for adopting those
from Web 2.0 to Semantic Web. More precisely, the thesis will focus on games, implementing the motivation fun and competition. I believe that the games described in my PhD thesis have the potential to generate a huge amount of lightweight knowledge structures that are useful in several aspects: (1) use of the resulting data with very little or no changes as lightweight ontologies and annotations, (2) use of the resulting knowledge structures as a basis for domain ontologies for further axiomatization, (3) use as training data for semi-automatic approaches, and (4) for machine learning.
So far, OntoPronto has been released to the general public. OntoTube and SpotTheLink are currently being tested. All four scenarios are expected to be online and broadly published by summer 2008. So far, my work was published in [15-17]. 11. 12. 13. 14. 15. 16. 17.