Research on learning analytics and educational data mining has been published since the rst conference on Educational Data Mining (EDM) in 2008 and gained momentum through the establishment of the Learning Analytics and Knowledge (LAK) conference in 2011. This paper addresses the LAK Data Challenge from the perspective of visual analytics of topic dynamics in the LAK Dataset between 2008 and 2012. The data set was processed using probabilistic, dynamic topic mining algorithms. To enable exploration and visual analysis of the resulting topic model by LAK researchers and stakeholders we developed and deployed DVITA, a web-based browsing tool for dynamic topic models. In this paper we explore answers to the questions about past, present, and future of LAK posed in the Data Challenge based on a topic model of all papers in the LAK Dataset. We also brie y describe how users can explore the LAK topic model on their own using D-VITA.
Copyright 2013 by the authors questions posed in the LAK Data Challenge into the user's hands. D-VITA is a web-based tool that o ers topic-based views on the LAK Dataset using a pointand-click metaphor and simple visualizations.
The LAK Dataset underlying the analyses presented in this
paper includes the EDM conference proceedings 2008{2012
(239 papers), the LAK conference proceedings 2011{2012
(66 papers), and the papers of the 2012 Special Issue on
Learning Analytics in the Educational Technology and
Society journal (10 papers; herafter referred to as ETS). The
RDF representation of the LAK Dataset was processed by a
script that extracted for each paper the identi er, venue 2
fLAK, EDM, ETSg, year of publication, title, authors,
abstract, full text, and hyperlink to the full RDF description
on data.linkededucation.org. The distribution of the 315
papers over time and venues is given in Figure 1.
In the next preprocessing step the paper records were cleaned
by removing stopwords and by applying stemming methods
on the included word sets. For word stemming we used the
Porter Stemming technique [
From a text mining perspective the LAK Dataset represents
a text corpus in which a set of words is used in a set of
papers. To identify what is relevant to LAK research, we used
the dynamic topic modeling approach described in [
Dynamic topic mining requires the analyst to pre-set the number of topics. Based on previous experiments with varying numbers of topics in paper collections in well-de ned subject areas, we decided to run the analysis of the LAK Dataset with a set of 20 topics. This number, while somewhat arbitrary, shall provide for su cient discriminatory power for both the distribution of topics over papers and the distribution of words over topics. With fewer topics, terms like `learning', for instance, are more likely to be present with relatively high relevance in many topics, while a larger preset would increase the number of topics exposed in each paper. Both situations would impede reasonable interpretation and visualization of the results.
A word of explanation regarding the labels used to refer to topics in this paper: mathematically each topic is a distribution over words. In a dynamic topic model this distribution changes over time, i.e. a speci c word may rise or fall in relevance for a topic. In the rest of the paper we will therefore label each topic with an ordered tuple representing those words with the highest mean relevance for this topic over time. In topic modeling literature we found that four words is a good number to form a topic label. For instance, for topic \students model parameters skill" the most relevant word on average is student followed by model, parameters, and skill. For illustration, based on the word distribution for this topic in 2008 only, the label would be \model student skill learning". Often, such word tuples are rephrased as more expressive labels; for instance \student modeling" could be appropriate in our example.
The obtained topic model including 20 topics was analyzed
to see whether the topics have su cient discriminatory power.
To this end, we used the ten most important words for
each topic and the corresponding probability distributions to
compute a dissimilarity measure of the distributions by
using the Jensen-Shannon divergence measure [
In this section we explore three questions about the LAK Dataset, intending to shed some light on the past and present topics of learning analytics research, along with a cautious glimpse into the future.
Question 1: What have been the most relevant topics overall in the LAK data set? This question addresses the LAK Data Challenge aspects of roots and current state of learning analytics. Figure 3 shows an overview chart of the 20 topics identi ed in the LAK Dataset. The horizontal axis re ects the rank of mean relevance of each topic and the vertical axis re ects the rank of stability3 over the ve time slices in the dataset. The size of each bubble re ects the relevance of the topic in 2012, the most recent period. We make several observations:
The most relevant topics most prominently feature the
terms students/learners, model, and data. This aligns
well with SoLAR's de nition of learning analytics as
\the measurement, collection, analysis and reporting
of data about learners and their contexts, for purposes
of understanding and optimizing learning and the
environments in which it occurs,"[
The topic with the highest mean relevance is \student model parameters skill" (A); this topic also has the highest variance in relevance.
In the top-right quadrant we nd topic \model data features prediction" (B) which has a strong relevance in 2012, high mean relevance rank over all years and a high stability. As such, it can be considered as one of the core topics in the LAK Dataset. In 2012 the distribution of words in this topic would advocate the label \prediction model data students", i.e. prediction is currently most relevant for this topic.
Topic \network community discussion analysis" (R) is also worth looking at. While it is relatively irrelevant and volatile, it is among the relevant topics in 2012 (cf. the bubble size). The topic evolution chart in Figure 4 reveals that this topic accumulated most of its relevance in 2011, the year of the rst LAK conference. Also, 8 of the 10 papers with the strongest focus on this topic in 2011 were published in the LAK conference (see bottom portion of Figure 4) although EDM published 2.5 times the number of papers in that 3Stability was computed by inverting the variance of the topic's relevance over time year. This topic, in 2012 represented by the word order \network community social user", therefore appears to be a genuine LAK topic which was previously rather irrelevant for the EDM conference.
Question 2: What changes in topic dynamics did the first LAK conference in 2011 bring about? This question aims to reveal whether and how the LAK community relates to the EDM community in terms of topics covered by their papers. To explore this we look (a) at at the overall distribution of topics over time and (b) at the relative change of topic relevance between 2010 and 2011.
The evolution of the overall distribution of topics is
illustrated in the ThemeRiver in Figure 5. In a ThemeRiver [
To see which topics rose in relevance between 2010 and 2011 we lter for topics and zoom into the transition between 2010 and 2011 as illustrated in Figure 6. Those three topics that have their absolute highest relevance in 2011 are marked with an up-pointing triangle with a solid-black outline. These are \model students data probability", \network community discussion analysis", and \problem students model types", indicating an increased focus on student modeling as well as community and network analysis through the rst LAK conference in 2011.
Question 3: What topics rose the most in 2012, the most recent time slice in the data set? This question looks into what the dynamic topic model of the LAK Dataset suggests as rising topics over the next year(s). We try to answer this by identifying those ve topics that had the highest rise in relevance between 2011 and 2012. The topic labels represent the word distribution in 2012, and the number in parentheses indicates the absolute gain in relevance: 6. CONCLUSION 1. students data courses system (+.054) In a nutshell, we discovered the following: Regarding the 2. students interaction participants analysis (+.036) past, we found that LAK and EDM do have a substantial 3. learning analytics social learners (+.035) shared topic foundation including themes like student mod4. students actions learning state (+.025) eling, data classi cation, and clustering. We also found that 5. data user learning dataset (+.013) the EDM conference series had some turbulence in topical focus between 2008 and 2010, the time window when only EDM papers are present in the dataset.
The Document and Word Evolution Panel shows for the selected topic an ordered list of the most relevant papers in the \Relevant Documents" tab. The icons next to each document allow showing the topic pie for the document and its content, respectively. The \Similar Docs" icon will bring up the Document Browser with a list of similar documents. Under the \Word Evolution" tab the user will nd a ThemeRiver illustrating the evolution of the distribution of words in the selected topic over time.
D-VITA also o ers a Document Browser to perform keywordbased search, explore the topic distribution of documents, and navigate documents based on similarity.
In sum these ve topics have accumulated a share of 42% of the topic distribution by 2012, starting from 11% in 2008 (cf.
Figure 7). These developments indicate a strong increase in focus on the students' activities and actions in courses as well as social and interaction analytics.
Except for Figures 1 and 3 all gures were produced using D-VITA, a web-based visual analytics tool we developed and deployed for visual analytics of dynamic topic models. The tool allows users to visually interact with the output of the dynamic topic mining algorithms on the LAK Dataset. The application window shown in Figure 8 has three panels: The Topics Panel shows the list of topics obtained by the dynamic topic modeling algorithm; topics can be sorted by rising, falling and mean relevance, as well as variance of relevance. The topics can be ltered using keywords; in the screen shot the keyword \visual" is used as a lter. The topic list thus only includes topics whose set of relevant words includes this word stem. Topics checked by the user will be visualized in the ThemeRiver in the Topic Evolution Panel. The Topic Evolution Panel shows a ThemeRiver of evolution of relevance of the topics selected in the Topics Panel. Data points for each topic and time slice, respectively, can be clicked, which will trigger the display of detailed information on the clicked topic at the selected time slice in the Document and Word Evolution Panel.
Regarding the present we found that the LAK Dataset exposes a strong emphasis on learner modeling, data modeling, analysis and prediction. The rst LAK conference in 2011 also brought some considerable shifts in topic focus; e.g. LAK 2011 has visibly strengthened network and social analysis aspects on top of EDM topics.
Regarding the near future we found that the shifts in the topics' proportions in 2012 appear rather moderate, thus indicating a phase of convergence of LAK research topics. Projecting recent topic shifts into the future, we can expect increased emphasis on social and interaction aspects and a sustained, strong role of students as research subjects.
This work was supported by the European Commission through the the support action TEL-Map (FP7-257822) and the integrated project Layers (FP7-318209).
Figure 8: Application window showing ThemeRiver and document list (rotated image)