Students' interaction data from learning environments has an inherent temporal dimension, with successive events being related through the \next event" relationship. Exploratory learning environments (ELEs), in particular, can generate very large volumes of such data, making their interpretation a challenging task. Using two mathematical microworlds as exemplars, we illustrate how modelling students' eventbased interaction data as a graph can open up new querying and analysis opportunities. We demonstrate the possibilities that graph-based modelling can provide for querying and analysing the data, enabling investigation of student-system interactions and leading to the improvement of future versions of the ELEs under investigation.
In this paper, we consider how modelling students'
eventbased interaction data as a graph makes possible
graphbased queries and analyses that can provide insights into
the ways that students are using the a ordances of the
system and the e ects of system interventions on students'
behaviour. Our case studies are two intelligent ELEs: the
MiGen system, that aims to foster 11-14 year old students'
learning of algebraic generalisation [
Teacher Assistance tools can subscribe to receive real-time
information relating to occurrences of indicators for each
student, and can present aspects of this information
visually to the teacher [
In this paper we explore how graph-based representation of
event-based interaction data arising from ELEs such as
MiGen and FractionsLab can aid in the querying and analysis
of such data, with the aim of exploring both the behaviours
of the students in undertaking the exploratory learning tasks
set and the e ectiveness of the intelligent support being
provided by the system to the students. Data relating to
learning environments has often been modelled as a graph
in previous work, for example in [
Previous work that is close to ours is the work on
interaction networks and hint generation [
Figure 1 illustrates our Graph Data Model for ELE interaction data. We see two classes of nodes: Event | representing indicator occurrences; and EventType | representing di erent indicator types. The instances of the Event class are occurrences of indicators that are detected or generated by the system as each student undertakes a task. We see that instances of Event have several attributes: dateTime: the date and time of the indicator occurrence; userID: the student it relates to; sessionID: the class session that the student was participating in at the time; taskID: the taskID that the student was working on; and constrID: the construction that the student was working on1. 1The model in Fig. 1 focusses on the interaction data. The full data relating to ELEs such as eXpresser and FractionsLab would also include classes relating to users, tasks, sessions and constructions; and attributes describing instances of these classes, such as a user's name and year-group, a task's name and description, a construction's content and description, and a session's description and duration.
Event dateTime taskID constrID userID sessionID next There is a relationship `next' linking an instance of Event to the next Event that occurs for the same user, task and session. There is a relationship `occurrenceOf' linking each instance of Event to an instance of the EventType class.
The instances of the EventType class include: startTask,
endTask, numberCreated, numberUnlocked,
unlockedNumberChanged, buildingBlockMade, correctModelRuleCreated,
incorrectModelRuleCreated, interventionGenerated,
interventionShown, in the case of eXpresser (see [
We see that instances of the EventType class have several attributes, including: eventID: a unique numerical identi er for each type of indicator; eventStatus: this may be -1, 0 or 1, respectively stating that an occurrence of this type of indicator shows that the student is making negative, neutral or positive progress towards achieving the task goals; an additional status 2 is used for indicators relating to system interventions; eventCat: the category into which this indicator type falls; for example, startTask and endTask are taskrelated indicators; interventionGenerated and interventionShown are system-related ones; numberCreated, numberUnlocked, unlockedNumberChanged are numberrelated; and fractionCreated, fractionChanged, fractionReleased are fraction-related.
Figure 2 shows a fragment of MiGen interaction data conforming to this graph data model. Speci cally, it relates to the interactions of user 5 as he/she is working on task 2 during session 9. The user makes three constructions during this task (with constrIDs 1, 2 and 3). The start and end of the task are delimited by an occurrence of the startTask and endTask indicator type, respectively | events 23041 and 33154. We see that the two events following 23041 relate to an intervention being generated and being shown to the student (this is likely to be because the student was inactive for over a minute after starting the task); following which, the student creates a number | event 24115.
There are additional attributes relating to events, not shown here for simplicity, capturing values relating to the student's next next 23041 dateTime: 20150331091524 taskID:2 constrID:1 userID:5 sessionID:9 23921 dateTime: 20150331091637 taskID:2 constrID:1 userID:5 sessionID:9 23923 dateTime: 20150331091638 taskID:2 constrID:1 userID:5 sessionID:9 ... next next 344712 dateTime: 20150215091741 taskID:56 constrID:4 userID:5 sessionID:1 344758 dateTime: 20150215091828 taskID:56 constrID:4 userID:5 sessionID:1 344759 dateTime: 20150215091828 taskID:56 constrID:4 userID:5 sessionID:1 constructions and information relating to the system's interventions. For example, for event 24115, the value of the number created, say 5; for event 23921, the feedback strategy used by the system to generate this intervention, say strategy 8; and for event 23923, the content of the message displayed to the user, say \How many green tiles do you need to make your pattern?" and whether this is a high-level interruption by the system or a low-level interruption that the student can choose to view or not. Such information can be captured through additional edges outgoing from an event instance to a literal-valued node: 24115 valu!e 5, 23921 strateg!y 8, 23932 messag!e \How many green tiles do you need to make your pattern?", 23932 lev!el \high". Since graph data models are semi-structured (and graph data therefore does not need to strictly conform to a single schema), this kind of heterogeneity in the data is readily accommodated.
Figure 3 similarly shows a fragment of FractionsLab interaction data, relating to the interactions of user 5 working on task 56 during session 1. The user makes one construction during this task. We see events relating to the student changing and `releasing' a fraction. Following which the system displays a message (in this case, it was a highinterruption message of encouragement \Great! Well Done").
We see from Figures 2 and 3 that the sub-graph induced by edges labelled `next' consists of a set of paths, one path for each task undertaken by a speci c user in a speci c session.
The entire graph is a DAG (directed acyclic graph): there are no cycles induced by the edges labelled `next' since each links an earlier indicator occurrence to a later one; while the instances of EventType and other literal-valued nodes can have only incoming edges. The entire graph is also a bipartite graph, with the two parts comprising (i) the instances of Event, and (ii) the instances of EventType and the literal-valued nodes.
As a nal observation, we note that Figures 1 { 3 adopt
a \property graph" notation (e.g. as used in the Neo4J
graph database, neo4j.com) in which nodes may have
attributes. In a \classical" graph data model, each attribute
of a node would be represented by an edge and its value by
a literal-valued node. So, for example, the information that
the taskID of event 23041 is 2 would be represented by an
edge 23041 taskI!D 2. The query examples in the next section
assume this \classical" graph representation.
3. GRAPH QUERIES AND ANALYSES
Because the sub-graph induced by edges labelled `next'
consists of a set of paths, the data readily lends itself to
exploration using conjunctive regular path (CRP) queries [
(X1; R1; Y1); : : : ; (Xn; Rn; Yn) where each Xi and Yi is a variable or a constant, each Zi is a variable that appears also in the right hand side of Q, and each Ri is a regular expression over the set of edge labels.
In this context, a regular expression, R, has the following syntax:
R := j a j j (R1:R2) j (R1jR2) j R j R+ where denotes the empty string, a denotes an edge label, denotes the disjunction of all edge labels, and the operators have their usual meaning. The answer to a CRP query on a graph G is obtained by nding for each 1 i n a binary relation ri over the scheme (Xi; Yi), where there is a tuple (x; y) in ri if and only if there is a path from x to y in G such that: x = Xi if Xi is a constant; y = Yi if Yi is a constant; and the concatenation of the edge labels in the path satis es the regular expression Ri. The answer is then given by forming the natural join of the binary relations r1; : : : ; rn and nally projecting on Z1; : : : ; Zm.
To illustrate, the following CRP query returns pairs of events x, y such that x is an intervention message shown to the user by the system and y indicates that the user's next action { in eXpresser { was to create a number (note, variables in queries are distinguished by an initial question mark): (?X,?Y) <- (?X,occurrenceOf,interventionShown), (?X,next,?Y), (?Y,occurrenceOf,numberCreated) The result would contain pairs such as (23923,24115) from Figure 2, demonstrating that there are indeed situations where an intervention message displayed by the MiGen system leads directly to the creation of a number by the student.
The following query returns pairs of events x, y such that that x is an intervention message shown to the user by the system and y is the user's next action; the type of y is also returned, through the variable ?Z: (?X,?Y,?Z) <- (?X,occurrenceOf,interventionShown), (?X,next,?Y), (?Y,occurrenceOf,?Z) The result would contain triples such as (23923,24115,numberCreated) from Figure 2 and (344760,344761,clickButton) from Figure 3, allowing researchers to see what types of events directly follow the display of an intervention message. This would allow the con rmation or contradiction of researchers' expectations regarding the immediate e ect of intervention messages on students' behaviours.
Focussing for the rest of this section on the data in Figure 2, the following query returns pairs of events x, y such that x is any type of event and y indicates that the user's next action was to unlock a number; the type of x is also returned, through the variable ?Z: (?X,?Y,?Z) <- (?X,occurrenceOf,?Z), (?X,next,?Y), (?Y,occurrenceOf,numberUnlocked) The result would allow researchers to see what types of events immediately precede the unlocking of a number (i.e. the creation of a variable). This would allow con rmation of researchers' expectations about the design of the MiGen system's intelligent support in guiding students towards generalising their models by changing a xed number to an `unlocked' one.
The following query returns pairs of events x, y such that that x is an intervention generated by the system and y is any subsequent event linked to x through a path comprising one or more `next' edges; the type of y is also returned, through the variable ?Z: (?X,?Y,?Z) <- (?X,occurrenceOf,interventionGenerated), (?X,constrID,?C), (?X,next+,?Y), (?Y,constrlID,?C), (?Y,occurrenceOf,?Z) The result would contain triples such as (23921, 23923, interventionShown), (23921, 24115, numberCreated), (23921, 24136, numberUnlocked), (23921, 24189, unlockedNumberChanged), relating to construction 1 made by user 5 during session 9 for task 2 (two more events | 24136 and 24189 | relating to construction 1 have been assumed here, in addition to 23923 amd 24115 shown in Figure 2, for illustrative purposes). The results would not contain (23921,33154,endTask), since event 33154 relates to construction 3.
To show more clearly the answers to the previous query in
the form of possible event paths, we can use extended regular
path (ERP) queries [
The use of the regular expressions next and next+ in the
previous queries matches precisely one edge labelled `next',
or any number of such edges (greater than or equal to 1),
respectively. However, for ner control and ranking of query
answers, it is possible to use approximate answering of CRP
and ERP queries (see [
For example, using the techniques described in [
We can modify the query to retain only pairs x, y that relate to the same construction: (?X,?P,?Y,?Z) <(?X,occurrenceOf,interventionGenerated), (?X,constrID,?C), APPROX(?X,next:?P,?Y), (?Y,constrID,?C), (?Y,occurrenceOf,?Z) would return rst exact answers, such as (23921, [next], 23923, interventionShown). The regular expression next in the conjunct APPROX(?X,next:?P,?Y) would then be automatically approximated to next.next, leading to answers such as (23921, [next, 23923, next], 24115, numberCreated) at an edit distance of 1 from the original query. Following this, the regular expression next.next would be automatically approximated to next.next.next, leading to answers such as (23921, [next, 23923, next, 24115, next], 24136, numberUnlocked) at distance 2. This incremental return of paths of increasing length can continue for as long as the user wishes, and allows researchers to examine increasingly longer-term effects of intervention messages on students' behaviours. It would also be possible for users to specify from the outset a minimum and maximum edit distance to be used in approximating and evaluating the query, for example to request paths encompassing between 2 and 4 edges labelled `next'.
Queries based on evaluating regular expressions over a
graphbased representation of interaction data, such as those above,
can aid in the exploration of students' behaviours as they are
undertaking tasks using ELEs and the e ectiveness of the
intelligent support being provided by the ELE. The query
processing techniques employed are based on incremental
query evaluation algorithms which run in polynomial time
with respect to the size of the database graph and the size
of the query and which return answers in order of increasing
edit distance [
The rst of these has rather `linear' data, similar to the interaction data discussed here, while the second has `bushier' connectivity. Query performance is generally better for the former than the latter, and the paper discusses several possible approaches towards query optimisation.
In addition to evaluating queries over the interaction data,
by representing the data in the form of a graph it is possible
to apply graph structure analyses such as the following:
path nding and clustering: this would be useful for
determining patterns of interest across a whole dataset,
or focussing on particular students, tasks or sessions
c.f. [
1002 3006
1010 applied across a whole dataset, or focussing on particular students, tasks or sessions; nodes that have a high probability of being visited on a randomly chosen shortest path between two randomly chosen nodes have high betweenness centrality; determining this measure for pairs of event type nodes (ignoring the directionality of the `occurrenceOf' edges) would identify event types that play key mediating roles between other event types.
We have already undertaken some ad hoc analyses of interaction data arising from classroom sessions using ELEs.
For example, Figure 4 shows the normalised incoming transitions for a 1-hour classroom session involving 22 students using MiGen (in the diagram, s denotes the `startTask' and e the `endTask' event types). Event types with an adjacent circle show transitions where this type of event occurs repeatedly in succession. The thickness of each arrow or circle indicates the value of the transition probability: the thicker the line, the higher the probability. Red (light grey) is used for probabilities < 0:2 and black for probabilities
0:2. We can observe a black arrow 3007 ! 1005,
indicating transitions from events of type 3007 (detection by the
system that the student has made an implausible building
block for this task) to events of type 1005 (modi cation of a
rule by the student). Such an observation raises a
hypothesis for more detailed analysis or further student
observation, namely: \does the construction of an incorrect building
block lead students to self-correct their rules?". Developing
a better understanding of such complex interaction can lead
to improvement of the system. For this particular example,
we designed a new prompt that suggests to students to rst
consider the building block against the given task before
proceeding unnecessarily in correcting their rules. More
examples of such ad hoc analyses are given in [
We have explored the possibilities that evaluating regular path queries over this graph-based representation might provide for exploring the behaviours of students as they are working in the ELE and the e ectiveness of the intelligent support that it provides to them. We have also identi ed additional graph algorithms that may yield further insights about learners, tasks and signi cant indicators.
Planned worked includes transformation and uploading of
the interaction data sets gathered during trials and full
classroom sessions of the two systems into an industrial-strength
graph database such as Neo4J, following the graph model
presented in Section 2; followed by the design,
implementation and evaluation of meaningful queries, analyses and
visualisations over the graph data, building on the work
presented in Section 3. Equipped with an appropriate user
interface, educational researchers, designers or even
teachers with less technical expertise could in this way explore
the data from their perspective. This has the potential to
lead to an improved understanding of interaction in this
context and to feed back to the design of the ELEs. We
see this approach very much in the spirit of \polyglot
persistence" (i.e. using di erent data storage methods to
address di erent data manipulation problems), and hence
being used in conjunction with other EDM resources such as
DataShop [
This work has been funded by the ESRC/EPSRC MiGen project, the EU FP7 projects iTalk2Learn (#318051) and M C Squared (#610467). We thank all the members of these projects for their help and insights.