Introduction

Interaction with semantic structures requires good understanding on how concepts are or can be related. A clear mental model of relations and their contributions in a knowledge representation can be of help when using applications in which semantic structure is central. One such application is the Cyttron Scientific Image Database for exchange (CSIDx), a repository of semantically related images from the life-sciences. The connections across the repository are of relevance and interest for the Cyttron database users and a good model and understanding of structural knowledge may facilitate working with the database.

This research investigates alternative ways to communicate semantic structure that are both understandable and enjoyable and that actively engage users into experiencing the absence or presence of connections. To that end, we propose gameplay as the means to interact with and internalize structure. Specifically, we attempt to devise game concepts that map semantic structure to actual game mechanics. As a starting point, we examine if existing games can be appropriated for our purposes and if an analogy with a known game can be beneficial.

Related Work

Communicating structure efficiently is a challenge commonly addressed in the field of information visualization. Structure as in a set of entities and their relations is normally abstracted to a graph and then visualized as a node-link diagram by means of graph drawing algorithms (see [1] for a survey). While the node-link diagram is not the only possible visualization for a graph [2], it is certainly the most prominent and a great amount of research is dedicated in improving graph layouts in terms of both efficiency and user satisfaction. On the other hand, interactivity has allowed for interactive graph visualizations that enable exploration and direct manipulation [3]. Our work, however, investigates interactive "views" that depart from the node-link metaphor and attempts to materialize connections rather that to visualize them.

In the last years, the interest of the academic community in videogames is exploding. Not only games and videogames are an independent field of study but also different disciplines have applied videogames in different contents and context. Serious games [4], i.e. videogames for purposes other than mere entertainment, have found applications in a variety of fields and industries such as education, marketing, training and social awareness [5]. Going one step further in exploiting player's engagement, von Ahn [6] introduced the concept of games with a purpose [7], i.e. games that harvest human effort during play, and proposed human computation as a substantial alternative in traditionally challenging areas of computation. Overall, games can be useful and applicable to a variety of (seemingly unrelated to videogames) tasks. Moreover, we believe that there are useful structural analogies between games and graphs1 which may justify the use of games for graph conceptualization.

Implementation

The Cyttron database as a graph

The Cyttron Scientific Image Database (for) Exchange (CSIDx) is a multimodal image database for multi-resolution microscopy data from the life sciences. It is the backbone database of Cyttron, a consortium of universities and companies focusing on the integration of bio-imaging techniques. In CSIDx, we attempt integration of multimodal image data on semantic content, rather than on pixel data, by assuring comprehensive and accurate image annotation by the expert [8], [9]. Next to minimizing ambiguity, the annotations impose structure across the repository and enable connecting images across biological concepts.

This work utilizes the existing annotations as the foundation for emergent structure across the repository. In particular, we use the assigned annotations to construct a (undirected) graph of the database. In this graph, images are considered as nodes and nodes share an edge when the corresponding images share an annotation term. Multiple annotations are equally reduced to a single connection but one can easily devise different heuristics on which annotations should contribute to an edge. Definitely, the current version of the database graph is a minimal representation that can be further elaborated e.g. by assigning weights to the edges or by calculating more elaborate similarity measures among the images. However, in all cases, the graph is the formal representation of the underlying structure. Yet, in communicating this structure, the actual graph imagery is to be hidden.

Onto-Frogger

Onto-Frogger is a game concept that borrows game elements from the arcade classic Frogger (Konami Industry Co. Ltd, 1981) [10] while, at the same time, adjusting the game logic to include and illustrate images and their annotations. The objective of the game is to arrive at a target image located on the opposite bank of a river by collecting coins while crossing lanes of floating image tiles. Jumping across lanes equals traversing a graph path as each subsequent lane depends on the preceding tile. Moreover, upon landing on an image, the player collects all of its associated annotations. But from the many annotations assigned to the selected image, only the ones in common with the target are rewarded as coins. Reaching the opposite bank without coins fails to complete the game.

Like Frogger, the player's task is to cross over and reach the opposite bank. Unlike Frogger, reaching the opposite bank is not merely a task of mechanic skill and timing but also a task of collecting enough information (in the form of coins) to access the image target. Coin terms indicate a semantic "nearness" to the target image and it is the presence of coins that qualifies the player to win. In other words, to try to reach the opposite side is to try to get semantically closer to the image target.

Discussion

Our research investigates if and how games can be applicable for communicating semantic structure: Onto-Frogger is an initial design and prototype addressing this topic. However, before we can comment on the applicability of games on conceptualizing structure, we need to examine if the current design fulfills its function. A first user evaluation of Onto-Frogger showed that, while the analogy with Frogger works well, the added constraint of coins is not self-explanatory. In fact, the evaluation identified several purely interface related issues that obscured the significance of coin terms. At the moment, we are evaluating an updated prototype to examine if interface improvements can bring the coin terms in focus or if the concept of coins severely contradicts with the attempted analogy to Frogger. Once the side effects of the interface are minimized, we will need a framework to further evaluate Onto-Frogger both as a game and as a structure representation.

The development of Onto-Frogger revealed that the underlying graph structure is imperative when designing a game, especially when attempting an analogy to a known game concept. Frogger's original gameplay was initially reformulated as pathfinding but the characteristic of our dataset dictated additional rules for Onto-Frogger. While the resulting gameplay can be interesting on its own sake, one can naturally question if an analogy is still useful or if it actually hinders the new game. Another approach would be to apply different heuristics for the graph generation rather than adjusting the gameplay: The resulting graphs may have different features while still deriving out of the same repository and still relating images on their semantic content. To what extent, then, should we revise our graph to fit the original game concept and when is it necessary to produce a new or adjusted game? We will continue researching both directions by examining both different datasets and different game concepts.