This work presents a multimodal dataset of 17 workgroup sessions in a collaborative learning activity. Workgroups were conformed of two or three students using a tabletop application in a co-located space. The dataset includes time-synchronized audio, video and tabletop system's logs. Some challenges were identi ed during the collection of the data, such as audio participation identi cation, and user traces identi cation. Future work should explore how to overcome the aforementioned di culties.
Computer Science (CS) students are required to develop teamwork abilities to be
successful in their professional life. Nevertheless, the state of the art presented
in [
Multimodal Learning Analytics (MMLA), which has been explored in recent
years [
One particular approach that has been used to capture traces of participants
in co-located workgroup activities is the use of multitouch tabletops through
tabletop systems (TS). Several studies have included these artifacts. Nonetheless,
most studies only involved the use of one modality (user's touch) for analysis [
The main contribution of this paper is to provide a multimodal dataset of
collaborative learning activities in co-located spaces using a TS proposed by
[
This paper is structured as follows: section 2 shows how the collaborative sessions were organized. Data collection setting is described in section 3. Next, our data set is explained in section 4. In section 5, challenges and future of the data collection process are presented. 2
Participants were 47 undergraduate students from CS, they were enrolled in an introductory Database Systems course. Students conformed groups of two or three members, based on their own a nity. During each session, groups were asked to collaboratively solve a database design problem and generate an entityrelationship (ER) model. They used the tabletop system to draw the design solution. Each student could interact with the system using their hands and tablets. The tablets were used to read the task description and create objects (entities and attributes). Students used their hands to move or delete objects over the tabletop and create relations between them. Each collaborative session had a duration of 20-30 minutes approximately. 3
To capture student's collaboration process, several input devices recorded student activities on the tabletop. Those devices includes: Video-camera and a Kinect v2. Figure 1 depicts the setting of the system. Our dataset includes: Audio, Video and tabletop system's logs. In the following paragraphs it is explained how the capture of each information source was carried out. 3.1
To record the audio, the microphone array of a Kinect v2 was located in front of the students. An application was developed using the Kinect SDK in order to estimate which student is talking at any time, based on the angle of the audio source, assuming that students never change their position around the tabletop. It is worth to say that this application does not recognize multiple participants talking at the same time. As a result, a CSV le with student's speaking intervention information was generated, along with a 4-channel wav audio le. 3.2
Each group session was recorded with a Lucy 360 camera, and later processed and transformed into a wide screen video. The camera was situated in front of the students. 3.3
The tabletop application recorded every action the students made in a log le, which was a plain text le. Every action was saved on a line of text, which indicated the student and the name of the action performed, along with the date and time of when the action was performed. 3.4
Every di erent source of capture was manually initiated by a member of the research team. However, since there was a di erence in seconds between every input device, all of the les had to be manually synchronized later. The CSV audio les contain a time-stamp in case it is needed for synchronization in the future.
An additional Excel le was created to join the les from the audio capture
and the log le. A new column was added to display the relative time, which
started on 0 when the audio recording began.
All the collected data is public available from http://www.cti.espol.edu.ec/
tabletopDataset.html prior to sign a collaborator agreement. The dataset
consists of 17 collaborative sessions. For each group the following les were saved:
{ A 360 video le in mp4 format, recorded at 15 FPS, with a resolution of
1152x320.
{ A 4-channel wav format audio le.
{ An audio log le in a CSV format with the following columns: angle; con
dence; timestamp; student id, at a rate of 16FPS (average). The angle value
is within the range of -50 and 50 . The con dence value is a number between
0 (lower con dence) and 1 (higher con dence). The timestamp is recorded
for further synchronization if needed. The student id is assigned as 0, 1 or 2
(see Fig.1).
{ A tabletop system's log le in a CSV format. Each line contains an action
performed by a student. The actions recorded can be one of the following:
create, delete or move entity; create, delete or move relation; join notes (to
create entities); and split entity into notes. The timestamp of the action
was also saved, as well as the student who performed the action. The
identi cation of which student created an entity comes from the tablet. The
identity of actions performed over tabletop objects were estimated using an
implementation of the approach proposed by [
Collecting and analyzing data from multiple sources comes along with some challenges described below: { The identi cation of student's speaking participation. Since we are using a single audio stream to identify student's oral interventions it's di cult to determine when two or more students speak at the same time. { Identi cation of actions' ownership on the tabletop. The application that was used for user's identi cation has an accuracy of about 90%. { Synchronization of the multiple inputs. Given that the audio and video recording started at di erent time, they had to be manually synchronized after the recording session.
Further research implies more reliable methods for tracking both the student's speaking participation and their interactions on the tabletop. Also, a centralized sever could be the solution for automatically synchronize all captured data. Learning analytics applications on this dataset may explore awareness and re ection of collaborative learning process through on time feedback; modeling groupwork behavior for predicting failure or success of collaborative tasks; automatic evaluation of group performance and monitoring student's collaborative learning skills over time.