=Paper=
{{Paper
|id=None
|storemode=property
|title=Designing Tabletop-Based Systems for User Modelling of Collaboration
|pdfUrl=https://ceur-ws.org/Vol-743/ASTC2011_Paper6.pdf
|volume=Vol-743
}}
==Designing Tabletop-Based Systems for User Modelling of Collaboration==
https://ceur-ws.org/Vol-743/ASTC2011_Paper6.pdf
Designing Tabletop-Based Systems for User Modelling
of Collaboration
Roberto Martinez, Christopher Ackad, Judy Kay and Kalina Yacef
School of Information Technologies, University of Sydney, NSW 2006, Australia
{roberto, cack7320, judy, kalina}@it.usyd.edu.au
Abstract. Tabletops offer a new form of interaction and create new possibilities for
small groups of people to collaborate and discuss tasks aided by the shared use of
digital materials and tools. The collaborative affordances of tabletops make them
suitable for many uses in public spaces as well as in more restricted environments
such as workplaces and learning settings. This creates new opportunities for
improving collaboration, particularly by capturing data that can be used to model the
nature of the interactions and to present this model to the users in a form that will
facilitate improved collaboration. It is timely to establish principles for designing
tabletop-based systems in a manner that can facilitate such modelling. These
principles should support effective use of data mining tools to create group
collaboration models. In this paper, we outline theoretical design principles based on
a careful analysis of the nature of tabletop datasets and collaboration.
Keywords: interactive tabletops, group modelling, collaborative learning, collocated
collaboration, machine learning
1 Introduction
Interactive tabletops offer a new medium for supporting collocated collaboration.
They provide a shared environment for small groups of people to work together,
making use of digital materials based on collaborative activities. A well recognised
role of tabletops is that they offer new means of interaction with special affordances
for small groups. A less recognised possibility is to exploit the user's digital footprints
as people make use of the tabletop. These footprints, along with the verbal
communication and contextual information related to the users, have the potential to
provide new opportunities to build models of their collaborative processes.
Collaboration is critical in a range of areas, from the workplace to learning spaces.
However, learning to collaborate effectively is difficult, partly because it involves a
long term development of skills. In addition, new collaboration contexts, working
with different people and complex tasks, require finer grained monitoring of the
collaboration and learning how to make it effective [1]. This means that collaborators
need to be able to monitor the effectiveness of the group as a whole and their
individual performance as part of the team. One approach that has proven effective in
covering such a need is to promote social translucence, an external representation that
mirrors objective measures of the group work to help them to be aware on their
collaborative process and monitor whether their actions match what they intended [2].
48 R. Martinez, et al.
The idea of reflecting overview information back to collaborators by exploiting the
huge quantity of data generated by their interactions is not new. Research on user
modelling has emphasised the potential of using machine learning techniques to
monitor learners’ collaborative processes and build adaptive tools that can intercede
to make such learning process more productive [3]. Even though the development of
collocated collaboration skills is very important in the classroom and beyond, most of
the research work on adaptation in collaboration has focused on the use of e-learning
tools (e.g. chat, forums, IM, email). However, e-learning and face to face
environments are not two separate domains. Nowadays, students are immersed in both
experiences: virtual and real worlds. They interact via email or chat, but also have
moments in which they have to work face to face. The benefits that tabletops offer to
this vision lie in the provision of support during the instants when students have to
create understanding in real world settings.
Our work aims to create new tabletop tools to exploit the activity logs and feed
them into user models in order to provide adapted support, so that the collaborative
process can be more effective and the individuals in the group can each learn to
improve their own performance. Currently, there are many tabletop interfaces but it is
timely to establish principled approaches to design the key features that should define
these learning systems. These range from the design of the tabletop setting to specific
user interface features. We propose a top-down approach in which the design of these
principles mandates what data should be captured and how it should be exploited to
build a model of the group’s interactions. Figure 1 shows the elements of our
approach. This starts with choosing adequate theories of small group collaboration
since they indicate the key elements of effective collaboration and learning. These
theories should define the ideal goals and drive the design of the collaborative setting.
For example, if we choose to measure symmetry of knowledge based on the definition
given by Dillenbourg [4], the system should be designed to capture elements that can
give insights on each learner’s understanding about a given topic. However, even
when these theories establish the ideal aims, the technology tradeoffs between the
scope of what is possible to capture and the associated cost bounds the system design.
The rest of the process consists of exploiting the electronic footprints that can be
captured as people interact at a tabletop and transform them into a useful data source
for these goals. To do this, we consider three elements: capture of useful data; mining
the data to transform it into a set of models of collaboration; and interfaces that make
use of these models to offer adapted feedback to the group. In this paper, we focus on
outlining generic principles for capturing and mining data in tabletop-based learning
systems. Further exploration on specific user interface design elements and ways to
access to the user models is mandatory, but the details are not important at this stage.
Collaboration theories Technology affordances
ideal goals feasibility, cost
Data capture Content of the data Hardware / software
Data mining Format of the data Data mining techniques
Adapted feedback Access to the user models Visualisations / adapted
actions
Fig. 1. Top-down approach for designing tabletop-based systems based on the dataset
requirements, grounding on theories of collaboration and the affordances of technology.
Designing Tabletop-Based Systems for User Modelling of Collaboration 49
2 Principles for Capturing Data
Special attention should be given to the architecture of the tabletop-based setting to
make the collection of data useful and successful. Next, we outline the key principles
of tabletop-based settings design capturing data effectively considering both the
learning theories and technology affordances.
Capture speech/video information. The analysis of peer communication is very
important for analysing the collaborative processes and it should be instrumented in
tabletop settings. The data that is useful to capture depends on the collaborative
learning theories underpinning the system. It can include just the presence of voice to
measure the participation of learners [5] or more detailed information like tone,
volume or, as most learning theories state is crucial, the speech content [4]. Current
solutions to record verbal interactions in collocated settings range from the use of
individual wearable audio recorders to the use of directional microphone arrays.
Detection of affective states in learners may also be considered by exploiting video
and sensors information [6].
Identify users (authorship of actions). As the collaborative setting becomes more
sophisticated, and the provision of certain types of adaptation are required, identifying
users’ actions becomes mandatory for updating the model of the group [7]. Current
solutions for identifying the authorship of each touch on the tabletop include high-
priced hardware devices such as the DiamondTouch1 or encumbering learners by
attaching gadgets to their hands (e.g. gloves or pens). There are also software based
solutions that constraint the design of the collaborative task, such as the assignment of
roles, resource ownership, personal territories, fixed production lines or individual
lenses [5].
Interconnection with other devices. In collaborative environments learners can
make use of multiple devices. Interconnected devices provide added speciality and
flexibility for specific tasks that come up during a collaborative session [8]. The
interconnection of all these as sources of information, can potentiate the use of
tabletops as a shared device in which all group members can work at the same time
contributing each to the group task. An example of this is using a digital whiteboard
to brainstorm ideas to afterwards store the results on a personal device or share them
on the tabletop to its revision.
Integration with services. Tabletop applications can also be integrated within a
larger scale system that can give continued support to the learning process of the
students. Current online e-learning and project management tools support
asynchronous collaboration in the form of wikis, chat and forums. Using tabletops as
an added interface to these pre-existent online collaboration tools can extend the
collaboration facilities provided by these services and compensate the lack of face to
face collaboration of the e-learning environments [9].
1
MERL- Diamond Touch.: http://www.merl.com/projects/DiamondTouch/
50 R. Martinez, et al.
3 Principles for Formatting and Mining Tabletop Data
Once the datasets are collected from the tabletop and before starting to use data
mining tools, the data has to be transformed into a suitable format for data mining
techniques. In this section, we propose a number of principles to ease the formatting
of the data according to the data mining requirements and the theoretical goals.
Define the logging granularity. The lowest level in which the tangible actions on
the tabletops can be recorded corresponds to logging the coordinates of each touch
point on the tabletop. Analyses of learners territoriality can be conducted using this
raw data. However, higher-level data logs, such as activity dependent information
(e.g. move object, press a button, delete an element), should be logged to get
meaningful insights on the strategies followed by groups. Besides, it could be
required to set up even higher-levels of abstraction by giving meaning to sets of basic
actions based on heuristics specifically created for the task. For example, basic
actions, such as dragging objects, inserting text or resizing images, in conjunction can
be related with higher level group strategies like brainstorming, agreement, or
formalisation of a solution.
Add user and contextual data. The user model of a group working at the tabletop
can be enriched by the incorporation of learner information that is normally beyond
the boundaries of the system, such as personal details or outcomes reached in related
academic activities [10] (e.g. the familiarity between group members, parts of each
learner model or the marks of previous assignments). Additional data can also be
generated by other systems related to the tabletop application (e.g. vertical displays,
smart-phones, laptops) [11] or if the tabletop is used after other technologies [12]. A
possible solution to ease the formatting of the data is to adhere to a common user
modelling framework which can give support to multiple services.
Define the focus of attention. The raw tabletop log data can contain detailed
contextual information about each action that users perform and it is normally
formatted as a very long sequence of events. It is very important to define the focus of
attention of the user modelling to capture and format the adequate contextual data to
fulfil the learning goals. Researchers on collaborative learning or the learners’
facilitators can specify this focus of attention. It can be directed to specific users, the
spatial position of resources, types of users or the disposition of learners around the
tabletop. For example, if the analysis is focused on the resources present at the
tabletop the dataset should identify and keep track of such resources along with the
stream of events.
Define the format of the data according to the data mining technique. Finally,
the data need to be extracted in the required format of the data mining technique to be
used. This is important because different algorithms need might require specific
contextual information. For example, sequential pattern mining algorithms need data
formatted as a detailed sequence of elements. Other techniques might require the
historical status of the objects at the tabletop to measure the progress of the group.
Designing Tabletop-Based Systems for User Modelling of Collaboration 51
4 Conclusion
Tabletops are an emerging form of interactive device for small group collaboration, in
educational and other settings. In order to design adaptive applications in collocated
settings where horizontal tabletops are present, it is crucial to establish the design
principles required by user modelling and machine learning techniques –two core
scaffoldings to offer such adaptation. We discussed a number of elements that should
be addressed by the architecture of collaborative tabletop systems. We look forward
to explore the possibilities of tabletops as supporters of learning and hope this
position paper can initiate a discussion regarding the technology and social issues that
must be addressed towards the provision of adapted support through tabletops.
Acknowledgements. This research was carried out as part of the activities of, and
funded by, the Smart Services Cooperative Research Centre (CRC). We thank Dr.
Anthony Collins for his suggestions.
References
1. Dillenbourg, P., Self, J.A.: Designing Human-Computer Collaborative Learning. In:
O’Malley, C. (ed.) COOP’96, vol. 91. Lancaster University (1996)
2. Erickson, T., Kellogg, W.A.: Social translucence: an approach to designing systems that
support social processes. ACM Trans. Comput.-Hum. Interact. 7, 59-83 (2000)
3. Magnisalis, I., Demetriadis, S., Karakostas, A.: Adaptive and Intelligent Systems for
Collaborative Learning Support: A Review of the Field. IEEE Trans. Learn. Technol. 4, 5-
20 (2011)
4. Dillenbourg, P.: What do you mean by 'collaborative learning'? Collaborative Learning:
Cognitive and Computational Approaches. Advances in Learning and Instruction Series.,
pp. 1-19. Elsevier Science (1998)
5. Martinez, R., Kay, J., Yacef, K.: Visualisations for longitudinal participation, contribution
and progress of a collaborative task at the tabletop. CSCL 2010, pp. 2-11 (2011)
6. Calvo, R.A., D'Mello, S.: Affect detection: An interdisciplinary review of models, methods,
and their applications. IEEE Transactions on Affective Computing 18-37 (2010)
7. Martín, E., Haya, P.A.: Towards Adapting Group Activities in Multitouch Tabletops.
International Conference on User Modeling, Adaptation, and Personalization Hawaii, USA
(2010)
8. Katherine, E.: MultiSpace: Enabling Electronic Document Micro-mobility in Table-
Centric, Multi-Device Environments. In: Chia, S., Kathy, R., Clifton, F. (eds.) Workshop
on Horizontal Interactive Human-Computer Systems pp. 27-34 (2006)
9. Paramythis, A., Mühlbacher, J.R.: Towards New Approaches in Adaptive Support for
Collaborative e-Learning. International Conference on Computers and Advanced
Technology in Education, (2008)
10. Mostow, J.: Some Useful Design Tactics for Mining ITS Data. ITS2004 Workshop on
Analyzing Student- Tutor Interaction Logs to Improve Educational Outcomes, (2004)
11. Wigdor, D., Jiang, H., Forlines, C., Borkin, M., Shen, C.: WeSpace: the design
development and deployment of a walk-up and share multi-surface visual collaboration
system. Human factors in computing systems, pp. 1237-1246. ACM (2009)
12. Martinez, R., Kay, J., Yacef, K.: Collaborative concept mapping at the tabletop. In ACM
International Conference on Interactive Tabletops and Surfaces, pp. 207-210 (2010)