The use of online videos is a common practice amongst education professionals and the interactive features found in these videos are constantly evolving. In a previous research paper we presented a roadmap on how open source technologies and open internet resources can be used to build a video based learning environment that supports learning analytics. This paper describes how an environment for teaching image editing techniques is build using similar principles. The video based environment is used in educational settings and a dataset of learner activity behaviors is obtained. Cluster analysis is then used for identifying groups of students with similar viewing and activity behaviors and an attempt is made to interpret the results.
Nowadays, educational video distribution over the Internet is a widespread practice. Together with the increase in the use of educational videos there is also an increase in the features that accompany these videos. Interactivity in educational videos is a relatively new trend with the level and types of interactivity to be in constant evolution. This is evident from the appearance of new tools and online platforms for creating and hosting interactive videos. Examples of such platforms are Zaption1, EdPuzzle2, EduCanon3, Vialogues4 and Raptmedia5. These platforms support a di erent set of features (e.g., in-video quizzes, time based discussions, branching videos etc.) and aim at making the video learning process a more interactive experience. However, most of these applications either do not come for free (e.g., Raptmedia) or o er only a limited set of features for free (e.g., Zaption, Educanon). Moreover, these applications are not open source meaning that they are not open to further development or customization by independent developers.
In a previous research paper [
More speci cally, in Section 2 we describe the details of a video based learning environment for learning `image editing techniques' together with the educational settings in which the environment was used. In Section 3 we describe a module for storing learner viewing behaviours and actions. In Section 4 cluster analysis is carried out in order to identify groups of students with similar behaviour. The paper concludes in Section 5. 2
In [
In this section we describe the features of an environment that is build using similar principles. This environment combines an instructional video together with a web based image editing tool, for teaching image editing techniques. The web based tool that is chosen is Pixlr7, a tool that is free to use and which has similar features to Photoshop. Pixlr also provides an API that enables web developers to use the application on their own site.
For a rst semester course called `Image and Video Editing Principles' an instructional video is produced in order to teach students the basics of image editing using Pixlr. Students are required to perform a login procedure in order to enter a platform with video lessons. One of the video lessons is the lesson
on `Image Editing using Pixlr'. Next to the video link there is also a link to a page containing detailed instructions. Students are strongly advised to visit the instructions webpage rst before clicking on the video link. When the video link is clicked learners are directed to an environment that contains the video together with other features as shown in Figure 1. The video occupies the top of the page and under the video there is an area with a table of contents (on the left) with links to di erent topic sections. By sections we mean the logical segmentation of video content into segments that cover a particular subtopic. The video lesson on image editing is divided into seven logical sections and the starting points of these sections are stored in the database. The rst section is a general introduction to Pixlr and the six sections that follow present di erent image editing techniques. The techniques are presented by descriptive exercise implementations. We also de ned one extra dummy section (or marker point) in the end in order to provide a link (in the table of contents) to the end of the video.
On the right side of the webpage (under the video) there is an area where exercise descriptions appear together with links that learners have to follow to complete the exercises. Each exercise (six in total) is associated with a di erent section and appears only during that section. Students are prompted to follow the link in order to complete a similar exercise as the one described in the related section. When the link is followed the Pixlr environment opens at a di erent browser window (or tab) together with the related image les that are needed in order to complete the exercise. For the autumn semester of 2015-2016, the exercises were part of an assignment that counted for the 15% of the nal mark.
Although Pixlr could also be opened as separate iframe in the same webpage and next to the video lesson, we chose not to follow this option since we wanted to give enough browser space to the video and the application and have menu actions related to the Pixlr software be clearly visible. After completing the exercise, the learner can save it by pressing the `Save' menu item in Pixlr. The image le of the completed exercise is saved on the learning environment web server and not on the students computer (or the students account on the Pixlr web server). This is achieved by using the Pixlr API. We chose this option since students would have to deliver the same exercises and we wanted to make sure that cheating is avoided and that students use their accounts to view the videos and complete the exercises. Learners are able to view the stored les whenever they want by pressing on a relevant link on the environment interface. Students that used this environment to complete the assignment were told in the instructions that images above 1000 width or 1000 pixel height would not be stored on the server and that was done in order to avoid delays and storage problems. An image with a relevant informative message was saved in case the student attempted to save an image that exceeded these dimensions. 3
Analysis of video viewing data is a relatively recent trend and there is a small
but increasing body of on-going research that focuses on the topic (e.g., [
The `timeupdate' is an event that res few times in every second and it can be used to track the state of the video. This event can also be used to build more events. For example, if at some instance the current video time is higher than the video time recorded a second before by two seconds or more, then we can safely assume that a jump took place (in our setting we use the threshold of 5 seconds to record a jump). A backward or forward jump can take place if the learner pauses the video at a certain point and resumes at another or when he/she moves the play head (or slide bar) to a new position in the video timeline. The events that are created using the `timeupdate' event are the following: a) section enter, b) time section enter, and, c) jump event. `Section enter' event is called when the play head enters one of the educator de ned sections. As already mentioned, sections de ned by the educator re ect di erent conceptual topics. Sections can also play the role of marker points in the video. When a marker point is reached, or when the video play head lands in a section after a jump, an entry is stored in the database together with the current date and time. Another way of splitting the video is through equal time intervals (rather than di erent conceptual topics). In this case the markers are inserted in equal time intervals. The `time section enter', event is triggered when the playhead reaches one of these intervals either during sequential video execution or after a jump. The time interval is set by the educator and stored in a general parameters database table. For the speci c video the interval was two minutes meaning that the 36 minute video was split into 18 intervals. Finally, the jump events are called when a jump takes place and two entries are stored in the database (with two di erent codes). The rst entry indicates the video time point of where the playhead was before the jump, and the other entry the video time point after the jump.
The section enter events together with the events pause, play and jump can give us a good estimate of the video portions viewed by learners. One advantage of logging viewing activity into a database is that many things can be quickly calculated by using the SQL query language (e.g., number of jumps and logical sections visited).
Actions that are related to the Pixlr activities are also stored in a separate database table. The additional actions that are stored in the database are the following: a) visiting the instructions page, b) following one of the exercise related links (six actions for the six di erent exercises), c) saving an image le (again six `save' actions for the six di erent activities), d) viewing the image les stored on the server, and, e) clicking on a section link of the table of contents in order to be directed to a particular section. 4
In this section, we present some metrics related to learners' activity that were used to carry out cluster analysis with Weka 8. The method of clustering is used in order to reveal di erent groups of students with similar viewing and activity behaviours and then an attempt is made to interpret the results.
More speci cally, the following metrics are obtained from the dataset: a) number of video visits, b) distinct section visits (logical sections de ned by the
educator), c) time section visits, this indicator is proportional to the time spend on active video viewing, d) distinct time sections visited, the video duration is 36 minutes and it is split into 18 equal (2 minute) time intervals, e) jumps, the number of forward and backward jumps that took place during video viewing, f) backward jumps, g) forward jumps, h) pauses, the number of times that the learner pressed the pause button, i) volume change, the number of time the learner attempted to change the volume, j) exercise link clicks, the number of times that an exercise link was clicked by a learner, k) save clicks, the number of times that a learner saved an image le, l) view les clicks, the number of times that the learner viewed the `saved' image les, and, m) read instruction clicks, the number of times that the learner visited the instructions page. These metrics are then used for carrying out cluster analysis. The analysis is carried out using the K-means algorithm. To determine the optimum number of clusters we used the `within sum of squared error SSE' in order to assess the clustering results. For each observation, the error is the distance to the nearest cluster. To get SSE, the errors are squared and summed up. To obtain the optimum K, we start from one cluster and continue adding clusters until diminishing returns are achieved, meaning no signi cant reduction in within SSE. Of course another important factor in accepting the resulting clusters is that we should be able to interpret them. The results that were obtained after following this process with Weka are shown in Figure 2.
Cluster 0 comprises of students that have a large number of video views, large number of jumps and time section visits. These students visited all logical sections of the video and also all the time sections (18 in total). However, the troubling fact is that these students also exhibit a large number of exercise visits and a large number of visits to the instructions page. A session where video viewing and exercises are performed without trouble is expected to be in line with the following scenario: in the beginning, the learner visits the instruction page, and, after carefully reading the instructions, he/she proceeds to view the related video. After (or while) viewing the second section of the video that is associated with exercise 1, the learner presses on the exercise link and performs the rst exercise in the Pixlr environment. After completing the exercise the learner saves the image le and views the le by pressing the `view- les' link. In the same manner the learner views the subsequent sections and performs the rest of the exercises. Ideally, the learner would click the exercise link only once in order to perform the exercise. The high number of clicks to the exercise link and the high number of le views as well as the high number of visits to the instruction page are a sign that the learner is disorientated in the environment.
Cluster 3 consists of learners that have a lower score on the mentioned indicators. However, still the number of exercise link clicks in the particular cluster is signi cantly higher than the number of exercises (6 exercises). Thus these learners also faced some problems. By looking in the dataset at a number of sessions from learners in this cluster we observed that a lot of these sessions were `troubled' at the beginning (e.g., multiple clicks to the exercise links, visits to the instruction page, high number of jumps) but normalized after a while.
Cluster 1 is a less `troubled' cluster. This cluster is characterized by low number of exercise visits. The exercise visits in this cluster are close to the number of exercises. There are also less visits to the instruction page and also fewer jumps. Moreover, there are fewer distinct time section visits and this means that a number of students in this cluster skipped some time segments of the video probably because it was clear to them how they should proceed.
Finally Cluster 2 consists of learners that did not nish the assignment since the number of exercise and `save' clicks are less than the number of the assignment exercises.
During the completion of the assignment a number of students contacted the educator in order to express problems that they were facing. The students encountered two types of problems mainly because they did not understand the instructions. One problem was that students did not understand that in order to perform the 6 exercises they would have to follow 6 di erent links that appeared in the exercise area during the related video sections. Another problem was that some of them tried to store images with dimensions larger than the de ned limit. An image with a relevant message was saved in this case but not all students understood what this message was about. Out of the 10 students that expressed their problems by email 2 of them were members of cluster 0, 7 were members of cluster 3 and 1 from cluster 1.
A survey was also handed out to the students via Google forms in order for students to evaluate the environment. 61 students took part in the survey out of the 90 that attempted the assignment. One of the survey questions asked students whether they faced any of the two problems mentioned above (or any other problem). Out of the 22 students who stated that they had at least one of the two problems described above, 16 were found to be members of cluster 3, 1 in cluster 0 and 4 in cluster 1.
Thus, we observe that the clustering results are in line with the questionnaire and the emails. However, the data suggests that more students faced problems than those who stated so in the survey. To summarize, the clustering scheme revealed clusters of students with di erences in the scores of the various indicators. In these educational settings the higher scores in these indicators were associated with problems that students faced during the assignment completion. Performance issues are not discussed in this study since once the students got acquainted with the environment features they completed the activities with success. 5
The purpose of this paper was to present a video based learning environment that supports learning analytics for teaching `image editing techniques'. After using this environment in educational settings as part of an assignment we obtained a dataset of viewing & activity behaviours. Indicators from this dataset were used in a clustering scheme to obtain groups of learners with similar characteristics. The clustering scheme helped us distinguish between learners that seem to have completed the assignment without any problems and those who encountered problems. This clustering scheme could again be used in the future as a method to assess the environment after having made all the necessary amendments in order to overcome the problems that were caused mainly due to misunderstanding of the instructions.