=Paper=
{{Paper
|id=Vol-1628/Future_paper2
|storemode=property
|title=Multitouch-based Annotation Boards for Face-to-Face Discussions
|pdfUrl=https://ceur-ws.org/Vol-1628/Future_paper2.pdf
|volume=Vol-1628
|authors=Jessica Rubart
|dblpUrl=https://dblp.org/rec/conf/ht/Rubart16
}}
==Multitouch-based Annotation Boards for Face-to-Face Discussions==
https://ceur-ws.org/Vol-1628/Future_paper2.pdf
Multitouch-based Annotation Boards for Face-to-Face
Discussions
Jessica Rubart
OWL University of Applied Sciences
An der Wilhelmshöhe 44
37671 Höxter, Germany
jessica.rubart@hs-owl.de
ABSTRACT
This paper presents a hybrid approach to annotations for face-to- This paper presents a hybrid approach to annotations, which
face settings. Firstly, multitouch-based cooperative interaction for focuses on face-to-face cooperation and
large displays is used to support group discussions with many
different structures. Secondly, augmented notes are used to combines multitouch-based cooperative interaction for
integrate paper-based notes in electronic structures. large advanced interactive displays with
CCS Concepts Augmented notes that integrate paper-based notes in
• Human-centered computing ➝ Human computer interaction electronic structures.
(HCI) ➝ Interaction paradigms; • Human-centered computing With advanced interactive displays we mean display hardware,
➝ Human computer interaction (HCI) ➝ Interaction devices; which supports multitouch technology and delivers good
• Human-centered computing ➝ Collaborative and social responsiveness and scalability. The number of supported
computing ➝ Computer supported cooperative work; • Applied touchpoints as well as the quality of its responsiveness and
computing ➝ Document management and text processing ➝ scalability shall be based on the number of users who can
Document preparation ➝ Annotation; physically interact with the display at the same time.
In the following, this paper presents related work and describes
Keywords the proposed approach in the context of different application
Multitouch; augmented notes; annotation; discussion; cooperative scenarios.
interaction
2. RELATED WORK
1. INTRODUCTION Touch and tabletop technology has been proposed for the
Annotations are a very useful means to support knowledge application areas education, e.g. [4], computer games, e.g. [8], and
building processes. For example, scribbling, adding comments art design, e.g. [11]. Compared to traditional user interfaces
and references, or highlighting media are very useful annotation richer user experience is pointed out. Another trend is to use
mechanisms supporting discussions. Many annotation approaches surface technologies for analysis work [2]. Visualization
apply the paper-and-pen paradigm to the digital world, which can techniques and exploring huge amounts of data are in the focus.
result in some shortcomings [6]. For example, multiple
annotation systems are used for different document formats or The annotation systems used in schools are often limited. Just one
media, such as paper-based documents, PDF files, or documents user at one point in time can interact with the device. In recent
of a word processing system. Another example for such years large displays providing multitouch functionality have
shortcomings is missing functionality to capture relations between become affordable so that more solutions providing simultaneous
documents to reflect lateral reading. A third example is very interaction should become available.
limited sharing capabilities of annotations among a group of users,
In the Hypertext community, tools for making notes, structuring
in particular with respect to paper-based documents. A repository
and sharing ideas have been worked on in several directions. In
for different document formats and annotations on them is
NoteCards [5], for example, notes are represented as cards
proposed, which gets replicated to cooperating users. In this way,
holding text and images, and being interconnected with typed
diverse users could get access to different annotations in specific
links. Fileboxes are a kind of composite notes useful to organize
documents [6].
large collections of notecards. In gIBIS [3], for instance,
Ted Nelson often argues that due to the simulation of paper in the hypertext networks based on the Issue-Based Information System
electronic world, structure is limited to hierarchy [10]. He usually (IBIS) approach can be created in a group. IBIS is an
argues for more sophisticated structures in the electronic world, argumentation-based approach focusing on problem solving.
such as ZigZag® [9]. Cooperative hypermedia approaches, such as in [13], focus on
collaboration support – both asynchronously as well as
synchronously with fine-grained notifications of other users’
interactions. In the context of hypertext narrative there is
Workshop “The Future of Computer Annotation” at the ACM Tinderbox [1], for example. Tinderbox is a tool for making,
conference on Hypertext and Social Media, July 10, 2016, Halifax, analyzing, and sharing notes focusing on spatial hypertext,
Canada informal semantics, and web collage. All these tools use the
Copyright is held by the owner/author(s). strength of hypertext to make information more structured and
meaningful. The Digital Desk [14] is early work to integrate
paper on a physical desk with the virtual world.
In this article, we combine a multitouch-based cooperative
annotation board with paper-based notes.
3. MULTITOUCH-BASED ANNOTATION
BOARDS
A multitouch-based annotation board, proposed in this article,
supports the annotation of different document formats and media
either individually or in a group. For the latter, we are currently
focusing on face-to-face settings. We are using widespread
gestures, known from smartphones and tablets, and apply them to
the simultaneous annotation and discussion of different media,
such as text, images, or video, on large displays. Gestures support
moving, resizing, and rotating an artifact, and can also be
executed by multiple users, e. g. for enlarging a document. Users Figure 2: Individual Multitouch-based Learning
can show documents to each other, annotate them cooperatively, Figure 3 shows a group of students annotating and discussing
and store the discussion results in group-related knowledge learning material on a multitouch table.
repositories. Onscreen keyboards are provided to edit text
simultaneously.
In addition, this approach utilizes augmented notes [12] by
integrating a camera in the environment. Figure 1 shows a camera
setup, which we are currently using above a multitouch table in
order to take pictures from the display triggered by a specific
gesture. In this way, a paper-based document can be put on the
table and automatically integrated in the electronic hypermedia.
Figure 3: Cooperative Multitouch-based Learning
Figure 4 shows an augmented note on a multitouch table. By a
specific gesture the camera is triggered and an electronic version
of the note is integrated in the learning process.
Figure 1: Camera Setup for Multitouch Table
The camera is connected to the wireless LAN and runs the
Android operating system. Through a specifically developed
mobile app we can remotely control the camera. Currently, we
are experimenting with optical character recognition to identify
text. If the handwritten text cannot be recognized, we integrate
the electronic version of the document as an image.
We have developed multitouch-based solutions for supporting Figure 4: Augmented Note
different usage scenarios. Two of them are described in the
following. 3.2 Industry 4.0
In the context of the German manufacturing industry new
3.1 Interactive and Cooperative Learning developments towards “Industry 4.0” are carried out [7]. We are
In our university, we are using several multitouch-based currently discussing innovative control panels for the
applications to get more attention from students in exercises as manufacturing industry, which visualize data mining results from
well as to support self-learning. Students can individually analyzing production data. The data analysis focuses on
navigate through learning material or cooperatively in groups identifying causes for quality problems. The visualization and
using larger multitouch-enabled displays. They can use intuitive interaction focuses on multitouch-based annotation boards. We
gestures and annotation functionality, such as scribbling or discuss the usage of multitouch screens in order to support
highlighting. In this way, learning material can be discussed problem solving in a group. Incident management is handled
effectively. Annotated artifacts can be saved and linked in the similar to case management. Information and knowledge about
learning process. Figure 2 shows an individual student using and the current situation is visualized and can be interacted with and
annotating learning material on a mid-size multitouch-enabled explored further cooperatively.
display.
Figure 5 shows a current prototype for this application scenario. the 1st international conference on Tangible and embedded
Reports are visualized as interactive documents, which can be interaction, 87-90.
annotated cooperatively to support discussions. [5] Halasz, F.G. 2001: Reflections on NoteCards: seven issues
for the next generation of hypermedia systems. In: ACM
Journal on Computer Documentation (New York, NY, USA:
ACM Press) 25 (3), pp. 71–87. doi:10.1145/507317.507321,
reprint.
[6] Hoff, C. and Rothkugel, S. 2008: Shortcomings in Computer-
based Annotation Systems. In: Proceedings of E-Learn:
World Conference on E-Learning in Corporate, Government,
Healthcare, and Higher Education, pp. 3715-3720, ISBN
978-1-880094-66-2.
[7] Kagermann, H., Wahlster, W., and Helbig, J., Eds. 2013:
“Securing the Future of German Manufacturing Industry:
Recommendations for Implementing the Strategic Initiative
Figure 5: Prototype for "Industrie 4.0" INDUSTRIE 4.0,” Final Report of the Industrie 4.0 Working
Group. Forschungsunion im Stifterverband für die Deutsche
Wirtschaft e.V., Berlin.
4. CONCLUSIONS AND FUTURE WORK
[8] Magerkurth, C., Memisoglu, M., Engelke, T., and Streitz, N.
This paper has proposed a hybrid approach to annotations for
2004: Towards the next generation of tabletop gaming
face-to-face settings. Multitouch-based cooperative interaction
experiences. In Proceedings of Graphics Interface, 73-80.
for large displays is combined with augmented notes. In this way,
both paradigms – rich hypermedia-based structures as well as [9] Nelson, T.H. 2001: Tech Briefung: ZigZag. In: Proceedings
paper-based notes and annotations – can be used and integrated in of Hypertext 2001, p. 261-262, ACM Press.
the knowledge media. The feedback from our students was very [10] Nelson, T.H. 2003: Structure, Tradition and Possibility
positive. In particular, the cooperative multitouch-based learning (Keynote). In: Proceedings of Hypertext 2003, p. 1, ACM
has been considered useful to support the knowledge building Press.
process. In our future work, we will do more evaluations and
continue on the “Industrie 4.0” scenario. [11] Patten, J., Recht, B., and Ishii, H. 2006: Interaction
techniques for musical performance with tabletop tangible
5. ACKNOWLEDGMENTS interfaces. In Proceedings of the int. conference on Advances
Thanks to Benjamin Lietzau for implementing different in computer entertainment technology, Article No. 27.
cooperative multitouch-based prototype solutions. [12] Rubart, J. 2015: Augmented Notes: Supporting Paper-based
Notes on Multitouch Tabletops. In: SIGWEB Newsletter,
6. REFERENCES Summer 2015, ACM Press.
[1] Bernstein, M. 2003: Collage, Composites, Construction. In:
Proceedings of Hypertext’03, ACM Press. [13] Rubart; J., Haake, J.M., Titze, D.A., and Wang, W. 2001:
Organizing Shared Enterprise Workspaces Using
[2] Brown, J., Wilson, J., Gossage, S., Hack, C., and Biddle, R. Component-Based Cooperative Hypermedia. In: Proceedings
2013: Surface Computing and Collaborative Analysis Work. of Hypertext’01, ACM Press.
Morgan & Claypool Publishers.
[14] Wellner, P. Interacting with Paper on the Digital Desk 1993.
[3] Conklin, J., and Begemann, M.L. gIBIS 1988: A Hypertext In: Communications of the ACM, 36 (7), ACM Press, pp. 86-
Tool for Exploratory Policy Discussion. In: Proceedings of 96.
Computer Supported Cooperative Work (CSCW), ACM
Press, pp. 140-152.
[4] Döring, T., and Beckhaus, S. 2007: The card box at hand:
exploring the potentials of a paper-based tangible interface
for education and research in art history. In Proceedings of