Introduction

The advent of video in online and blended learning started at the turn of the century and became more and more popular as lecture recording, how-to-videos and screen casts could be easily produced and distributed. Since 2012 video obtained a wide echo in Massive Open Online Courses (MOOCs). Because videos are mainly used in online distance learning teachers can not observe the user behavior, resource usage, and learning activities in a directly manner. Instead methods from Learning Analytics, Educational Data Mining, and Video Usage Mining [MBD06] are required to track and analyze the user activities. This paper offers a systematic literature review of the state of research in the field that could be summarized as video analytics. Since this is a work in progress paper, the review focuses only on three research question (RQ) concerning data gathering, measurements and visualizations, rather than providing a complete overview on video analytics:

•

RQ1: What data needs to be captured form video players and learning environments in order to perform analytics?

• RQ2: What measures can be derived from the captured data?

• RQ3: What data representation are suitable to support visual analytics.

Methodology

The literature review was conducted in a four-step process: i) search in the selected academic databases by using the proposed search terms; ii) selecting relevant articles from the title and abstract of the search results; iii) identify further articles that were referenced in the selected articles; vi) paper review by following the guidelines of [Ch16].For the review we selected seven academic databases for articles related to technology enhanced learning: ACM Digital Library, IEEE Xplore, SpringerLink, Science Direct, Taylor & Fransis, dblp, and Wiley. Additionally, we queried Google Scholar, Research Gate, and Mendeley in order to embrace potentially relevant "gray literature" such as technical reports or position papers.

To perform the search we used combinations of two sets of search terms: i) video, audiovisual media, electure, lecture recording, and ii) analytics, data, user behavior, usage, mining, watch*, click, log. Overall 93 publications were gathered from search and the recognized references. After getting an overview of the field 44 relevant articles could be identified for deeper analysis. Most of these articles were retrieved from the ACM Digital Library. The publication dates range from 1994 to 2017, whereas 14 were published in the year 2014. 14 articles covered analytics about MOOCs. The same amount of articles described studies in a university setting. The remaining papers covered technological or methodological aspects as well as experiments that were not directly related with educational technology.

Results

Data gathering (RQ1)

Analytics about video-based learning mainly gather data form log files. In terms of videobased learning environments dedicated logs from the video player are required to capture the entire user interactions. Currently there is a lack in standardization of log formats and data structures. Only a minority of platform providers published their log format (e.g. edX data API), while existing drafts (e.g. the videoprofiles for the xAPI) have not been recognized by the community yet.

Watching: On of the core question in video analytics is the way of approximating the user's time spent on watching a video. Whereas modern web application make use of accurate Javascript timeupdate events, some systems still lack the possibility to gather fine-grained second-by-second data. However, captured playback activity does not imply user engagement. Therefore, playback measures need to be compared with clickstream data in order to ensure minimal engagement indicators. Furthermore, the time that a user needs to watch a particular part of the video also depends on the playback speed [Li15b,GKR14].

As a consequence the length of a video depends on the users' watching habits.Table 1 summarizes different approaches for approximating playback durations by amplifying varying granularities.

Although imprecise measurements may limit statistical inferences, privacy concerns should urged as an important argument. [HGM14,Br11] outlined how the principle of data economy could be applied by factoring playback traces as well as click events into binary data.

Video: Learning videos can not be considered as a homogeneous type of media. Today we find various technical representations, formats, and styles. A few researchers focused on

Measurements (RQ2)

Measurements are built upon the captured data that was described in the previous subsection.

Regarding video-based learning measurements can be classified in three categories: i) video watching behavior, ii) video interactions, and iii) other user input considered as learning results.

Video watching behavior: Analyzing the users' behavior when watching the video can provide insight regarding in-video drop out rates [Li15a, Ki14c, BLGS17] and most frequent watched segments. [KE16b] even demonstrated on how to derive playback events from the timeupdate events. Video interactions: Gathering clickstream data is essential for analyzing, modeling and predicting video interactions. Typically the frequency (total and per segment) and the duration of clickstream events is used (see Tab. 4) to perform various analyses. The majority of researchers focus on in-video interactions, rather than inter-video interactions (see [GR14,HGM14,Br11]). The latter consider browsing behavior between multiple videos in a course or database. The analysis of clickstream data has different purposes. Basically, the [Dí15] seek to [Dí15] events are used to identify access patterns across courses [HGM14], week days or hours of the day [Br11,Se14]. Considering in-video interactions [Ki14c] identified and analyzed peaks of both frequently watched scenes and the used playback controls during that scenes. So fare the properties of the peaks (width, height, area) have been analyzed and related to possible explanations (e.g. visual transitions or returning to missed content) [Ki14c]. [Ki14a] is looking forward to classify peaks automatically considering visual transitions, speech properties and topic transitions derived from the video transcripts. [SJD15] found significant lower pause and seek back rates when the teacher gaze augmented to the video.

[Si14] determined clickstream profiles of students in-order to predict engagement states as well as in-video and course drop outs. [MBD06,dBT08,Ch17,CdBB17], and [Li15b] found different in-video viewing and interaction patterns. [Li15a] demonstrated how the perceived difficulty of the video content correlates with some of the determined video interaction patterns.

Learning results: Learning results in a broader sense include student contributions such as answers to quizzes, forum or wiki entries as well as annotations. These contributions may either being entered during the video playback or separate from the video.

[Li15b] distinguished strong from weak students by comparing correct answers in relation to the number of attempts made to pass an assignments. The interaction patterns of strong students significantly differed from the students considered as weak. [MD13] found correlations between quiz scores and watched portions of lecture recordings, not least because the correct answers were told in the corresponding parts of the video. [KE16b] found significant correlations between the quiz attempts as well as results and the watched video segments. [GMD14] applied quantitative text analysis to evaluate large amounts of video annotations. Measuring word counts related to linguistic and psychometric processes aims to reduce the time for reading and scoring submissions.

Visualizations (RQ3)

Data charts are essential for visual analytics tasks. Effective visualization are primarily determined by the selected dimensions, rather than the type of chart. Going into detail about the various forms of data visualization would go beyond the scope of this article, but should be considered as a future research direction. The same is true for learning dashboards. According to [Sc16] a "learning dashboard is a single display that aggregates different indicators about learner(s), learning process(es) and/or learning context(s) into one or multiple visualizations." The latest review articles on learning dashboards did not go into detail about visualizations or dashboards representing video-based learning activities [Sc16,Ve14]. Particular dashboards for MOOC instructors or students as reported by [Fr16,Vi17,KE16a] stay on the surface by explaining selected data charts instead of providing a complete overview. Some insights could be gained from edX. However, the advances in visual analytics in terms of data visualization like rewatching graphs [BTG13], forward-backward diagrams [Se14], or interaction peaks [Ki14c] have not been transfered to dashboards yet. Learner-centered social navigation aids along the player timeline are known for many years now [MKB10, Ki14c, WL15, Ch16], but have not spread beyond research prototypes. Potential data representations for visual analytics tasks could be identified in the works of [GKR14, Li15a, HGM14, BET99, LW10, Br13, De14, Co14].

Conclusions

This literature review presented the foundations, current state and potentials of video analytics. However, the review should be enlarged upon effective visualizations for video learning dashboards. Furthermore, the analysis methods that were stated in the literature are worth to be compared considering the available data. The method set ranges from statistics over sequence mining to natural language processing.