Onto-Frogger: Playing with Semantic Structure

                            Amalia Kallergi, Fons J. Verbeek

        Imaging & BioInformatics, Leiden Instistute of Advanced Computer Science.
                           Niels Bohrweg 1, 2333 CA Leiden
                       {akallerg, fverbeek}@liacs.nl



       Abstract. In this paper we discuss the use of games to interact with datasets
       with explicit semantic structure. We focus on image repositories enhanced with
       annotations and user metadata such as the Cyttron database, a repository of
       multimodal multi-resolution microscopy data from the life sciences. To be
       unambiguous in metadata, Cyttron images are comprehensively annotated with
       ontology terms obtained from domain specific ontologies. But annotations also
       impose structure across the repository and enable connecting images across
       biological concepts. Such a networked structure, if justly conceptualized, can be
       insightful for the biology researcher.
       Our research focuses on communicating the emergent structure in ways that are
       both understandable and enjoyable. We investigate interactive views that
       employ metaphors other than the node-link diagram and propose gameplay as
       the means to internalize a semantic structure. Onto-Frogger is our first
       prototype to examine this exact idea.

       Keywords: Gaming, semantic structure, image repository



1      Introduction

   Interaction with semantic structures requires good understanding on how concepts
are or can be related. A clear mental model of relations and their contributions in a
knowledge representation can be of help when using applications in which semantic
structure is central. One such application is the Cyttron Scientific Image Database for
exchange (CSIDx), a repository of semantically related images from the life-sciences.
The connections across the repository are of relevance and interest for the Cyttron
database users and a good model and understanding of structural knowledge may
facilitate working with the database.
   This research investigates alternative ways to communicate semantic structure that
are both understandable and enjoyable and that actively engage users into
experiencing the absence or presence of connections. To that end, we propose
gameplay as the means to interact with and internalize structure. Specifically, we
attempt to devise game concepts that map semantic structure to actual game
mechanics. As a starting point, we examine if existing games can be appropriated for
our purposes and if an analogy with a known game can be beneficial.
2        Related Work

   Communicating structure efficiently is a challenge commonly addressed in the
field of information visualization. Structure as in a set of entities and their relations is
normally abstracted to a graph and then visualized as a node-link diagram by means
of graph drawing algorithms (see [1] for a survey). While the node-link diagram is not
the only possible visualization for a graph [2], it is certainly the most prominent and a
great amount of research is dedicated in improving graph layouts in terms of both
efficiency and user satisfaction. On the other hand, interactivity has allowed for
interactive graph visualizations that enable exploration and direct manipulation [3].
Our work, however, investigates interactive “views” that depart from the node-link
metaphor and attempts to materialize connections rather that to visualize them.
   In the last years, the interest of the academic community in videogames is
exploding. Not only games and videogames are an independent field of study but also
different disciplines have applied videogames in different contents and context.
Serious games [4], i.e. videogames for purposes other than mere entertainment, have
found applications in a variety of fields and industries such as education, marketing,
training and social awareness [5]. Going one step further in exploiting player’s
engagement, von Ahn [6] introduced the concept of games with a purpose [7], i.e.
games that harvest human effort during play, and proposed human computation as a
substantial alternative in traditionally challenging areas of computation. Overall,
games can be useful and applicable to a variety of (seemingly unrelated to
videogames) tasks. Moreover, we believe that there are useful structural analogies
between games and graphs1 which may justify the use of games for graph
conceptualization.


3        Implementation

3.1      The Cyttron database as a graph

   The Cyttron Scientific Image Database (for) Exchange (CSIDx) is a multimodal
image database for multi-resolution microscopy data from the life sciences. It is the
backbone database of Cyttron, a consortium of universities and companies focusing
on the integration of bio-imaging techniques. In CSIDx, we attempt integration of
multimodal image data on semantic content, rather than on pixel data, by assuring
comprehensive and accurate image annotation by the expert [8],[9]. Next to
minimizing ambiguity, the annotations impose structure across the repository and
enable connecting images across biological concepts.
   This work utilizes the existing annotations as the foundation for emergent structure
across the repository. In particular, we use the assigned annotations to construct a
(undirected) graph of the database. In this graph, images are considered as nodes and

1 For the rest of this paper, the term ‘graph’ refers to the formal abstraction of structure and not

    the node-link diagram representation.
nodes share an edge when the corresponding images share an annotation term.
Multiple annotations are equally reduced to a single connection but one can easily
devise different heuristics on which annotations should contribute to an edge.
Definitely, the current version of the database graph is a minimal representation that
can be further elaborated e.g. by assigning weights to the edges or by calculating
more elaborate similarity measures among the images. However, in all cases, the
graph is the formal representation of the underlying structure. Yet, in communicating
this structure, the actual graph imagery is to be hidden.

3.2    Onto-Frogger

    Onto-Frogger is a game concept that borrows game elements from the arcade
classic Frogger (Konami Industry Co. Ltd, 1981) [10] while, at the same time,
adjusting the game logic to include and illustrate images and their annotations. The
objective of the game is to arrive at a target image located on the opposite bank of a
river by collecting coins while crossing lanes of floating image tiles. Jumping across
lanes equals traversing a graph path as each subsequent lane depends on the preceding
tile. Moreover, upon landing on an image, the player collects all of its associated
annotations. But from the many annotations assigned to the selected image, only the
ones in common with the target are rewarded as coins. Reaching the opposite bank
without coins fails to complete the game.
    Like Frogger, the player’s task is to cross over and reach the opposite bank. Unlike
Frogger, reaching the opposite bank is not merely a task of mechanic skill and timing
but also a task of collecting enough information (in the form of coins) to access the
image target. Coin terms indicate a semantic “nearness” to the target image and it is
the presence of coins that qualifies the player to win. In other words, to try to reach
the opposite side is to try to get semantically closer to the image target.


4      Discussion

   Our research investigates if and how games can be applicable for communicating
semantic structure: Onto-Frogger is an initial design and prototype addressing this
topic. However, before we can comment on the applicability of games on
conceptualizing structure, we need to examine if the current design fulfills its
function. A first user evaluation of Onto-Frogger showed that, while the analogy with
Frogger works well, the added constraint of coins is not self-explanatory. In fact, the
evaluation identified several purely interface related issues that obscured the
significance of coin terms. At the moment, we are evaluating an updated prototype to
examine if interface improvements can bring the coin terms in focus or if the concept
of coins severely contradicts with the attempted analogy to Frogger. Once the side
effects of the interface are minimized, we will need a framework to further evaluate
Onto-Frogger both as a game and as a structure representation.
   The development of Onto-Frogger revealed that the underlying graph structure is
imperative when designing a game, especially when attempting an analogy to a
known game concept. Frogger’s original gameplay was initially reformulated as
pathfinding but the characteristic of our dataset dictated additional rules for Onto-
Frogger. While the resulting gameplay can be interesting on its own sake, one can
naturally question if an analogy is still useful or if it actually hinders the new game.
Another approach would be to apply different heuristics for the graph generation
rather than adjusting the gameplay: The resulting graphs may have different features
while still deriving out of the same repository and still relating images on their
semantic content. To what extent, then, should we revise our graph to fit the original
game concept and when is it necessary to produce a new or adjusted game? We will
continue researching both directions by examining both different datasets and
different game concepts.

Acknowledgments. This work is supported by the Cyttron consortium (BSIK grant
#03036)


References
1. Battista, G.D., Eades, P., Tamassia, R., Tollis, I.G.: Algorithms for drawing graphs: an
    annotated bibliography. Computational Geometry 4(5), 235--282 (October 1994)
2. Schulz, H.J., Schumann, H.: Visualizing graphs - a generalized view. In: IV '06: Proceedings
    of the conference on Information Visualization. pp. 166--173. IEEE Computer Society
    (2006)
3. Herman, I., Melancon, G., Marshall, M.S.: Graph visualization and navigation in information
    visualization: A survey. Visualization and Computer Graphics, IEEE Transactions on 6(1),
    24--43 (2000)
4. Susi, T., Johannesson, M., Backlund, P.: Serious games: An overview. Tech. rep., GLS
    University of Wisconsin-Madison (February 2007)
5. Sawyer, B., Smith, P.: Serious games taxonomy.
    http://www.seriousgames.org/presentations/serious-games-taxonomy-2008_web.pdf
    (February 2008)
6. von Ahn, L., Dabbish, L.: Labeling images with a computer game. In: CHI '04: Proceedings
    of the SIGCHI conference on Human factors in computing systems, pp. 319--326. ACM
    (2004)
7. von Ahn, L.: Games with a purpose. Computer 39(6), 92--94 (2006)
8. Kallergi, A., Bei, Y., Kok., P., Dijkstra, J., Abrahams, J.P., Verbeek, F.J.: Cyttron: A
    virtualized microscope supporting image integration and knowledge discovery. In:
    Backendorf, C., Noteborn, M., Tavassoli, M. (eds.) Proteins Killing Tumour Cells, pp. 291--
    315. Cell Death and Disease Series, ResearchSignPost, Kerala India (2009)
9. Kallergi, A., Bei, Y., Verbeek, F.J: The ontology viewer: Facilitating image annotation with
    ontology terms in the CSIDx imaging database. In: Workshop on Visual Interfaces to the
    Social and the Semantic Web (VISSW2009) (February 2009)
10. Frogger. http://en.wikipedia.org/wiki/Frogger