Utilizing Gaze Data in Learning:

Abstract

Although a lot of attention has been dedicated towards improvement of the modeling of learners' knowledge within learning systems, recommendation, or personalization, there is less attention on improvement of the learning content itself and providing support to learning content creators. In addition, the complexity of learning systems requires utilization of novel sources of implicit feedback, such as gaze data in order to model learners' interactions in their entirety. In this poster paper, we present a framework for collection of gaze data and its utilization in the learning systems environment. We focus on the analysis of reading patterns for the detection of problematic parts of text and present results of a preliminary evaluation in a web-based learning system ALEF.

Keywords

• learning system
• personalization
• implicit feedback
• eye tracking
• reading patterns
• learning content comprehension
• ALEF

-

and read, which parts are easy to understand and which need clarification or rewriting. This can be also achieved by the analysis of gaze data; the existing body of work found useful to focus on distinguishing between at least two forms: reading and scanning (or skimming) the text [ 6, 3 ]. Less focus is given on detecting which parts of the text are hard to comprehend for the learners [ 2 ]. This would be especially useful to the learning content creators, because it has potential to help them to improve the learning texts and in turn, improve learning experience of the learners.

Although the eye tracking hardware is becoming more and more affordable, the challenges associated with the utilization of the gaze data remain, namely: ─ Limited support of the dynamic web content, which is now prevalent in the modern web-based learning systems, but complicates analysis of the learners’ fixations on the areas of interests. The existing studies often limit their analysis only to the static content which does not require scrolling and does not change with time. ─ Limited support of reading patterns detection which focuses mainly on the reading vs. scanning, but does not identify the reading comprehensibility problems; identification of these problems is crucial for improvement of the learning content. 2

Framework for Collection and Utilization of Gaze Data

In order to address the presented challenges we have developed a framework for collection of the gaze data extending the infrastructure described in [ 8 ]. It consists of Gaze Monitor, a desktop client that communicates with the eye tracker and sends the gaze data to the server as well as to the web browser where the gaze data are processed in real-time by a reusable JavaScript component. The component provides: ─ Enrichment of the gaze data by the DOM element of the web page that corresponds to the gaze coordinates; the element is identified by a unique XPath string which enables to unambiguously determine whether the gaze coordinates fall within any of the defined areas of interest (AOIs). The fact that the user looks at a certain web page element triggers a gaze event that can be processed in the real-time. ─ A communication with the Gaze API that enables to retrieve the AOIs defined for a web page (or application), as well as pre-defined gaze events that should be triggered if the preconditions are met, such as when the user does not fixate on the specified area of interest over a given period of time. It also enables the users to retrieve the gaze-based statistics for the areas of interest, e.g. the number of fixations, o dwell time, and most importantly, the identified reading patterns for the AOIs. The main advantage of our framework is that by providing a reusable JavaScript component, it can easily extend any web-based application (learning system) so that it can benefit from the eye tracking. For example, based on a more precise estimation of a student’s active learning time by gaze analysis, we can improve the estimation of the knowledge level related to the particular concepts presented in the learning object. In addition, since the areas of interest are defined as elements of the web page, they are robust to changes of their position or size; we can easily aggregate the gaze data from different web pages of the same web-based system (e.g. navigation menu or widgets that are usually present in all the pages of the system). 3

Learning Content Comprehension Detection

Using our framework, we have extended our web-based learning system ALEF [ 4 ] with the gaze data collection. We propose a method of reading patterns detection based on [ 6 ]. Our modification of the original method in [ 6 ] allows to detect not only reading vs. scanning patterns, but also reading comprehension problems.

The comprehension problems present a subset of the reading pattern identified by the original method. We label a sequence of fixations (classified by the Tobii I-VT fixation filter [ 1 ]) as problematic to comprehend if the number of revisits (recurrent saccades) of the given area is above the threshold value. We also consider the average length of the fixations which is higher in case of a comprehensibility problem.

We are interested not only in identifying the individual patterns on the per-user basis which can be used for improvement of the user modeling within the learning system, but also in providing an aggregate value of area comprehensibility. It serves as a feedback to the learning content creator, suggesting which parts of text need rewriting or rephrasing, thus leading to the improvement of the quality of the learning materials. The proposed visualization of the text comprehensibility can be seen in Fig. 1.

Each paragraph (area of interest) has its font colored with a shade of gray based on the normalized comprehensibility value from the <0; 1> interval, where 0 means no problems with comprehensibility and 1 means very problematic to comprehend. Thus, the most problematic areas of text immediately stand out from the remaining parts.

In order to determine the optimal parameters of the proposed algorithm, we started collection of a reading dataset using Tobii TX300 eye tracker. Up to now we have gathered the data from four participants. Each of them read four texts with different tasks that included reading the text thoroughly, scanning and skimming the text to find the required information, omitting fragments of the text or simulating comprehensibility problems by re-reading parts of the text. Each of the four texts was divided into multiple areas of interest where each paragraph of text corresponded to one area. Additionally, during the post-processing, the recordings were segmented and annotated with one of the following labels: reading, skimming, scanning, and re-reading. As a result, each fixation is assigned one of these labels together with the information whether it falls into any of the areas of interest. We continue to gather a larger dataset. For this purpose, we plan to utilize our UX Lab that is equipped with 20 Tobii X2-60 eye trackers which allow the parallel collection of the gaze data.

Acknowledgement. This work was partially supported by grants No. VG 1/0646/15, No. KEGA 009STU-4/2014, and it was created with the support of the Research and Development Operational Programme for the project “University Science Park of STU Bratislava”, ITMS 26240220084, co-funded by the European Regional Development Fund.

References

1. Olsen , A. : The Tobii I-VT Fixation Filter: Algorithm description . ( 2012 ). Available online at http://www.tobii.com/eye-tracking-research/global/library/white -papers/the-tobii-i-vt-fix ation-filter/ . Cited 6th April 2015 .

2. Biedert , R. , Dengel , A. , Elshamy , M. , Buscher , G.: Towards Robust Gaze-Based Objective Quality Measures for Text . In: ETRA '12: Proc. of the Symposium on Eye Tracking Research and Applications , pp. 201 - 204 . ACM, New York ( 2012 )

3. Biedert , R. , Hees , J. , Dengel , A. , Buscher , G.: A Robust Realtime Reading-Skimming Classifier . In: ETRA '12: Proc. of the Symposium on Eye Tracking Research and Applications , pp. 123 - 130 . ACM, New York ( 2012 )

4. Bieliková , M. et al. ALEF: From Application to Platform for Adaptive Collaborative Learning . In: Manouselis, N. et al. (eds.): Recommender Systems for Technology Enhanced Learning , pp. 195 - 225 , Springer ( 2014 )

5. Busjahn , T. et al.: Eye Tracking in Computing Education . In: Proc. of the 10th Annual Conf. on Int. Computing Education Research , pp. 3 - 10 . ACM, New York ( 2014 )

6. Campbell , C.S. , Maglio , P.P.: A Robust Algorithm for Reading Detection . In: PUI '01: Proc. of the Workshop on Perceptive User Interfaces , pp. 1 - 7 . ACM, New York ( 2001 )

7. Kardan , S. , Conati , C. : Comparing and Combining Eye Gaze and Interface Actions for Determining User Learning with an Interactive Simulation . In: UMAP '13: Proc. of the 21th Int. Conf. on User Modeling , Adaptation, and Personalization, LNCS 7899 , pp. 215 - 227 . Springer ( 2013 )

8. Móro , R. , Daráž , J. , Bieliková , M. : Visualization of Gaze Tracking Data for UX Testing on the Web . In: Hypertext '14 Extended Proc. of the 25th ACM Hypertext and Social Media Conference , vol. 1210 . CEUR-WS ( 2014 )

9. Navrat , P. , Tvarozek , J.: Online Programming Exercises for Summative Assessment in University courses . In: CompSysTech '14: Proc. of the 15th Int. Conf. on Computer Systems and Technologies , pp. 341 - 348 . ACM, New York ( 2014 )