Web search engines provide a rich feature set to users that allows e cient satisfaction of information needs. Nevertheless, recent studies show that Internet users do not know how to use Web search engines e ectively for satisfying information needs. The overall level of Web search literacy leaves a lot to be desired and most users tend to overestimate their abilities in the domain of Web search. In this paper, we introduce a gami cation approach with the aim of promoting search literacy as well as the current state of our prototype application. We present plans for future work to answer whether gami cation is a viable means to improve Web search literacy. Our goals include nding indicators to di erentiate between low and high literacy users and running long-term user studies to investigate the sustainability of search literacy improvements.
Search literacy denotes the ability to locate and access desired information with e ciency and e ectiveness. It is, therefore, a subset of the much broader concept information literacy which also encompasses evaluation, reuse of information, and information synthesis. Instead of putting Web search on a level with information retrieval, we look upon it as a lifelong learning process that we aim to support in order to enable users orienting themselves in modern societies. Users employ Web search engines not only for answering trivial information needs but also trust in the machines and their own abilities when it comes to serious decisions, e.g., health related issues or nancial concerns.
In this paper, we draw attention to the problems arising from de ciencies in aforementioned Web search literacy and introduce a potential means that aims at increasing Web search literacy beyond traditional training methods like courses or tutorials. Only few approaches exist in this regard. One notable exception being A Google a Day 1 which features a fact nding search quiz. Users are encouraged to employ advanced features of the search engine to solve the tasks. While the system includes a scoring system and rewards fast task completion, it lacks key gami cation elements like levels, achievement badges, or leaderboards. Users solve tasks themselves and can not compete with other users.
We developed a gami cation framework that features di erent types of tasks (e.g., search and educational) to give users a deeper understanding of the functioning of Web search engines. At the same time, our users are to learn and develop skills that should help them in mastering their daily search tasks e ciently. Furthermore, we plan to use the presented system as a basis for long-term studies. The goal of these studies will be 1) to identify key factors that make a user actually Web search literate, 2) to measure whether Web search literacy was improved and by how much, and 3) to evaluate how sustainable these e ects are.
The remainder of this paper is structured as follows: First, we take a closer look at related work regarding search literacy and gami cation in Section 2. The gami ed application is presented in Section 3, covering game modes and later focusing on the employed game design elements. In Section 4, we draw a conclusion and provide an outlook for future work. 2 2.1
A recent study by Stark et al. [
In his recent talk at GamifIR'15 [
According to Fuhr [
Linguistic Functions A crucial problem in information retrieval is the language itself since every natural language is both vague and ambiguous. To deal with that problem and to represent a user's information needs as best as possible, Web search engines apply linguistic functions such as word normalization, lemmatization, and phrase identi cation to search queries and take composites and synonyms into account.
Query Language A specially designed language allows users to express complex information needs and leads to more speci c results since the latter are restricted to a limited set. Search operators (e.g., Boolean operators, number ranges, facets, elds, and URL predicates) and search options (e.g., for restricting the time, place, language, and document type of result items) are commonly employed search features.
Ranking One search query leads to a set of results where the ranking is of
utmost importance. Hochstotter et al. [
Strategies and Tactics The main goal of Web search engine users is to
satisfy their current information need. Complex information needs require a
series of search queries. Strategies are plans for performing a complex search
whereas tactics denote single operations to advance searches. Bates [
In the book by Zichermann et al. [
In [
Following the insights gained by literature review, we developed an application for improving search skills which appears to the user in the form of a game. We introduce the notion of game mode which emphasizes the playful character of the framework and summarizes a set of tasks of a speci c nature. In total, we developed three game modes each of which aiming at a di erent aspect of Web search literacy: Quiz, Search Hunt, and Query Tuning.
Quiz The quiz mode features single and multiple choice question answering tasks (see Fig. 1). They allow for a deeper understanding of the functioning of search engines. The main goal of this game mode is to familiarize a user with advanced search engine functionality in a series of tasks. Furthermore, the quiz acts as a means to measure a priori search knowledge.
Search Hunt This mode comprises typical fact nding tasks (see Fig. 2). Users are asked to complete a task by issuing queries to one of the world's leading search engines which is directly included in the game by a proxy solution in order to nd the solution being sought. Search hunt primarily trains users to formulate precise search queries, to identify relevant results, and to nd the desired content within the document. Furthermore, it promotes the ability to judge accuracy of results. We exploit the search engine's rich feature set to provide a complete interface that contains all commonly employed search functions to our users. The interface allows us to train users on how to use speci c features and can have more or less importance depending on the task. In addition, tasks will be designed in a way to familiarize users with commonly neglected search engine features and search strategies. Query Tuning is comprised of precision oriented tasks (see Fig. 3). Users are again required to interact with the search engine but this time to produce a result set that contains a speci c site at a top position. Along with the target site and a summary of the content comes a small set of search terms that are not allowed to be used in the query to avoid trivial solutions (e.g., querying for the URL of the site). Users formulate and reformulate queries until either the given site is ranked at the top position or the search performance can not be improved further. Hence, required skills for formulating precise queries are enhanced within a step-by-step re ning process. The main goal of this mode is to form the understanding of ranking and to get a feeling how small changes in the query can yield to major di erences in the result set. 3.2
The core game mechanics of the application consist of points, levels, badges, and leaderboards. Points are received for (partial) successful completion of tasks. The amount is determined by the degree of correctness, the current user level as well as the time needed to complete a task. Points act as the main performance indicator in the application. Levels are used to de ne the user's current state and represent a task's complexity. The next level up can be reached by exceeding the corresponding point threshold. Badges are special rewards that are acquired either for reaching a certain state or for various actions. They may come as a surprise and with varying frequency and act as a motivator to explore the application. Furthermore, badges can be used to \show o " individual user skills to other users via a pro le page (see Figure 4).
Leaderboards are overviews of the top performing players in each game mode and are represented as ordered lists with a points score beside each name to allow simple comparisons and to engage users in competition. Again, these boards act as a motivational means for continuing as well as an instrument along with levels to indicate that users have more or less status or achievement in the game. Besides the core mechanics, the application features di erent sound e ects to guide users and to introduce events, e.g., the beginning/completion of tasks and the receipt of awards. A comprehensive logging system collects user data in the background. The log data gives an insight into a user's behavior and thus can be used to create user pro les that reveal strategies and techniques being used to solve tasks. 4
In this paper, we presented a gami cation framework for Web search. The system in its current state features key gami cation elements like points, badges, and leaderboards. Furthermore, three di erent game modes, i.e., types of tasks, are included. The system was tested regarding usability in a small user study (N = 15) with great success. The main goal of the system is to improve Web search literacy among general Web search users. We believe that this will allow for more time e cient and e ective search sessions, which will lead to a higher task completion rate. To accomplish this goal, we will address various challenges that we are still facing with our prototype: 1. The actual search tasks and quizzes need to be tailored for the goal of improving literacy. Therefore, tasks will be created that are challenging for an average search engine user. Ideally, tasks should promote speci c learning goals, e.g., search strategies or search engine features. 2. After a larger collection of tasks has been created, we will run a long-term study with a larger user base. This will not only allow us to tune game balance. The results will also act as a ground truth for future experiments. Observing many users completing the same tasks will allow us to \pool" solution attempts and to generate an ideal solution for every task. Individual users will then be scored by the closeness of their solution to the ideal one. 3. In the end, we will isolate key factors that make a user Web search literate and nd means to speci cally promote these skills in a gami ed environment. The secondary goal of the long-term study will be to test the sustainability of literacy improvements. Therefore, we will invite participants to reuse the application with new tasks of similar complexity after a speci c time period and compare the outcomes of their endeavors.