Presentations Preserved as Interactive Multi-video Objects

Abstract

We rst give an overview of a system which allows capturing a lecture to generate, as a result, a multi-video multimedia learning object composed of synchronized videos, audio, images and context information. We then discuss how a group of students interacted with a learning object captured from a problem solving lecture: a similar approach can be used by instructors to re ect about their performance during their lectures.

Keywords

• General Terms Documentation
• Measurement
• Veri cation

Introduction

Increasingly universities record lectures and make them available on the web, exploiting the fact that the classroom can be viewed as a rich multimedia environment where audiovisual information is combined with annotating activities to produce complex multimedia objects [ 1 ]. Although in some cases the web lecture may be a single video stream (e.g. Kan Academy and TED talks), more elaborate viewing alternatives are available (e.g. opencast [ 2 ] and openEya [ 3 ]).

Once captured lectures have been made available, being able to analyse how the users watch them { and learn from them { is a challenging task, as illustrated by Brooks and colleagues [ 2 ]. In such a scenario, extracting semantics from the captured information is a must [ 7 ]. We built a system prototype that allows recording a lecture: audio and video streams from the instructor, slides, writings on whiteboards, as well as contextual information { the aim is to automatically generate an interactive multimedia object [ 5, 6 ]. Given the several sources of information, students must be given a broad range of interaction alternatives when reviewing the lecture: our system generates multi-video objects in a standard for interactive multimedia, so that students have several interaction alternatives at the same time that can use a standard HTML5 browser. The actual student interactions are also captured so they can be analysed. Next, we brie y introduce our system, and outline results from analysing the student interactions with the resulting interactive multi-video object.

From capture to interactive multi-video

We have instrumented a classroom with cameras (Figure 1(a)), and built a prototype system whose modules (Figure 1(b)) capture several information streams from a lecture and generate an interactive multimedia object (NCL1), which can then be played-back in a player which runs on standard HTML5 browsers (Figure 1(c)). 1Nested Context Language - http://ncl.org.br/en (a) (b) (c) The player (Figure 1(c)) is designed so that the multi-video object corresponding to the lecture may be reconstituted and explored in dimensions not achievable in the classroom. The student may be able, for example, to obtain multiple synchronized audiovisual content that includes the slide presentation (1), the whiteboard content (2), video streams with focus on the slide (3) or the lecturer's full body (4), or the lecturer's web browsing, among others. Moreover, the student may choose at any time what content is more appropriated to be exhibited in full screen. The student may also be able to perform semantic browsing using points of interest like slides transitions and the position of lecturer in the classroom. Moreover, facilities can be provided for users to annotate the captured lecture while watching it, as advocated by the Watch-and-Comment paradigm [ 4 ].

One lecture, 12 modules

Using the capture-tool prototype, one instructor captured one lecture without students in the classroom: students had access to the multi-video object to prepare to their nal exam.

The lecture was a problem solving session for a Computer Organization course in which an instructor solved a total of 15 exercises. These exercises were related to each other and usually a subsequent exercise used some results from the previous one. The exercises also become more di cult as the presentation progressed.

The lecture was divided into 12 modules, totalling 1 hour and 18 minutes. Module 1 presented 3 exercises, module 5 contained 2 exercises, and all the other modules presented one exercise each.

Eighteen students watched the lecture for at least 4 minutes: the average playback time was 59 minutes, with standard deviation of 39 min. The average number of interactions was 118.6, also with a large deviation (99.6).

Checking out student interactions

Given that the multimedia object has more than one video stream and that students can choose which stream they want as the main stream (presented in the large window in the player), the information of which stream is the most selected as the main stream at each moment can be useful for the instructors to re ect about their performance during the lecture.

We present next how students interacted with the several video components that make up the multi-video object of modules 1 and 4. A detailed discussion of the students interaction with all modules is available elsewhere [ 8 ]. Figure 2(a) and Figure 2(b) summarize which streams were most selected as the main stream, respectively, for module 1 and module 4. Each line represents how many times a stream was watched in a speci c moment: the blue line corresponds to the slides as presented in the instructors notebook (Figure 1(c-1)); the red line corresponds to the conventional whiteboard (Figure 1(c-2)); the green line corresponds to the electronic whiteboard which presented slides which could be annotated by the instructor (Figure 1(c-3)); and the purple corresponds to the camera giving an overview of the classroom (Figure 1(c-4)).

As shown in Figure 2(a), students watched more, as the main stream, the slides and the whiteboard. The three regions with higher values for the red line correspond to the moments in which the instructor solved the three exercises writing on the conventional whiteboard. Accordingly, the higher values for the blue line correspond to the slides with the speci cation of the exercises, and precede properly the higher values of the red line. A similar behavior is shown in Figure 2(b): the di erence is that this module discussed a single exercise.

Final Remarks

Our plans for future work include capturing more contextual information during the presentation toward providing novel navigation facilities, and the development of visualization tools for the instructors to analyse the students multi-video object interaction.

Acknowledgements

We thank the courses instructor and the students, the WAVe13 organizers for the opportunity to present our work, and the workshop participants for their inspiring presentations.2

References

[1] Abowd , G. , Pimentel , M. d. G. C. , Kerimbaev , B. , Ishiguro , Y. , and Guzdial , M. Anchoring discussions in lecture: an approach to collaboratively extending classroom digital media . In Proc. CSCL ' 99 ( 1999 ).

[2] Brooks , C. , Thompson , C. , and Greer , J. Visualizing lecture capture usage: A learning analytics case study . In Proc. WAVe ' 2013 ( 2013 ).

[3] Canessa , E. , Fonda , C. , Tenze , L. , and Zennaro , M. Apps for synchronized photo-audio recordings to support students . In Proc. WAVe ' 2013 ( 2013 ).

[4] Cattelan , R. G. , Teixeira , C. , Goularte , R. , and Pimentel , M. D. G. C. Watch-and-comment as a paradigm toward ubiquitous interactive video editing . ACM TOMCCAP 4 , 4 (Nov. 2008 ), 28 :1{ 28 : 24 .

[5] Hurst, W., Maass , G. , Muller, R., and Ottmann , T. The "authoring on the y" system for automatic presentation recording . In CHI'01 Extended Abstracts ( 2001 ), 5 { 6 .

[6] Pimentel , M. , Abowd , G. D. , and Ishiguro , Y. Linking by interacting: a paradigm for authoring hypertext . In Proc. HYPERTEXT ' 00 ( 2000 ), 39 { 48 .

[7] Ronchetti , M. Videolectures ingredients that can make analytics e ective . In Proc. WAVe ' 2013 ( 2013 ).

[8] Viel , C. , Melo , E., da Graca Pimentel, M. , and Teixeira , C. How are they watching me: learning from student interactions with multimedia objects captured from classroom presentations . In Proc. ICEIS ' 13 ( 2013 ).