=Paper=
{{Paper
|id=Vol-2384/paper04
|storemode=property
|title=Scaffolded, Scrutable Open Learner Model (SOLM) as a Foundation for Personalised e-Textbooks
|pdfUrl=https://ceur-ws.org/Vol-2384/paper04.pdf
|volume=Vol-2384
|authors=Judy Kay,Bob Kummerfeld
|dblpUrl=https://dblp.org/rec/conf/aied/KayK19
}}
==Scaffolded, Scrutable Open Learner Model (SOLM) as a Foundation for Personalised e-Textbooks==
https://ceur-ws.org/Vol-2384/paper04.pdf
Scaffolded, Scrutable Open Learner Model
(SOLM) as a foundation for personalised
e-textbooks.
Judy Kay1[0000−0001−6728−2768] and Bob Kummerfeld1[0000−0002−6046−6393]
University of Sydney, Sydney NSW, Australia,
Judy.Kay@sydney.edu.au
Bob.Kummerfeld@sydney.edu.au
http://sydney.edu.au/
Abstract. While e-textbooks have become increasingly widespread, they
have made little use of personalisation. This paper presents an archi-
tecture for making a learner’s own lifelong learner model as the foun-
dation for a highly flexible form of personalisation for learners using
e-textbooks. The architecture has the following core components: an ar-
bitrary e-texbook platform that minimally provides log data about the
learner; a personal user model ; an open learner model (OLM) interface
that provides personalised scaffolding to support reflection on learning
progress and support for the learner to scrutinise their model. We present
a worked example of the approach, explaining design decisions in terms of
a substantial body of previous work on open learner models and support
for meta-cognitive processes of self-monitoring, self-reflection and plan-
ning. Our key contribution is a new approach to personalised e-textbooks
based on the learner’s own long term learner model.
Keywords: e-textbook · personalisation · personal learner model · OLM.
1 Introduction
Learners in tertiary education increasingly have ubiquitous access to smart-
phones and many also own portable larger screen devices that cross the tablet-
laptop continuum as well as desktops. This makes it feasible to make use of
electronic textbooks since they can be available to the students. This creates op-
portunities for creating textbooks that are central to the learning in a university
subject. It also is timely to create e-textbooks that are far more than simply an
electronic form of a conventional textbook using tools such as Runestone [13, 14]
which make it feasible for a teacher to create or customise their own e-textbook
that includes embedded interactive learning activities such as multiple-choice
questions, short answer questions and programming testing. These e-textbooks
also make it easy for a teacher to reuse some of the many high quality materials
openly available on the web, simply by linking to them.
This paper presents our architecture for a new class of such textbooks that
makes use of the rich data from learners interacting with the book. The key
2 J. Kay et al.
innovation is the creation of an independent Open Learner Model (OLM) [10,
6]. This is a representation of the learner’s current progress through the Knowl-
edge Components (KC) of the course. This enables a learner to see a high level
view of the teacher’s curriculum design, with an indication of their progress on
each KC. We call this a Scaffolded, Scrutable Open Learner Model (SOLM).
The scrutability means that we designed it, from its foundations, to ensure that
a learner can scrutinise all the key aspects of the model. From a learner’s per-
spective, this means that the scrutiny interface enables the learner to see: the
details of the evidence used to infer whether they know a concept or not; the de-
scription of the resolver which interprets the evidence; and a control mechanism
that enables the learner to select different resolvers, for example to set their own
standard of performance on self-text quizzes to determine whether a particular
Knowledge Component is treated as known or not [12].
The idea of an OLM has a long history, as reviewed in [4] and the idea of
making this available to students to guide their navigation was used even in
early intelligent tutoring systems, such as ELM-ART [3]. The first use of an
independent OLM and of scrutability also have a long history [10]. Similarly,
there has been long recognition of the importance of OLMs to support learners
in taking responsibility for their own learning because it supports their key meta-
cognitive processes of self-monitoring, reflection and planning their learning [11,
5]. Our approach also scaffolds the OLM to help the learner in thee challenging
meta-cognitive tasks. This is important for two reasons. First, we want to ensure
practical scrutability, meaning that the learner actually can scrutinise their OLM
successfully to answer core questions about it. Secondly, there is ample evidence
that learners benefit from scaffolding to help them reflect and plan [2, 15].
A final key element of our work is the commitment to a deeply human-centred
approach to giving learners control over algorithmic systems. There has recently
been broad recognition of the importance Fair, Accountable, Transparent ma-
chine learning (FAT*ml)1 . We consider that learning contexts make these issues
even more pressing since we want to create e-textbooks that are deeply based on
ensuring learners can access and harness their own learning data and can both
scrutinise and control its use in an OLM that shows them their learning progress
based on data from an e-textbook.
2 SOLM Architecture
Figure 1 shows the architecture of our system. The core idea is that we use
an available e-texbook platform and extract data from each learner’s activity
to create the evidence in their personal learner model. The arrow from the e-
textbook to the learner model indicates a program that processes the learner
data, transforming it and adding evidence to the model. The teacher curriculum
model at the right represents the teacher’s intended progress for a plausibly ideal
student [7], one who does activities as scheduled by the teacher and performs well,
1
http://www.fatml.org/,https://fatconference.org/)
Title Suppressed Due to Excessive Length 3
but not necessarily perfectly, on the e-textbook activities. The architecture could
also extract whole class data to add comparative information of the student’s
progress compared with the class [8]. At the bottom of the figure is the OLM
interface. This will enable the learner to navigate their learner model to scrutinise
any part of the model and to select the resolver that interprets the evidence about
learning. The learner model will also represent the learner’s use of the OLM and
this will be used to personalise scaffolding [9, 15].
Personis
● e-text model
● learner knowledge
e-text platform Teacher
components
(eg Runestone) curriculum model
● preferences
● goals
OLM + scaffold
Fig. 1. SOLM Architecture
In order to test our ideas for textbooks of the future we chose the Rune-
stone 2 platform. Runestone is a web based system for constructing interactive
textbooks. It allows content to be created in the “restructuredText” markup
language and has extensive facilities for inline interactive content such as mul-
tiple choice quizzes, fill in the blanks questions, Parson’s problems and more. It
also allows program code to be written and executed directly in the textbook
interface.
The Runestone server is written in Python with the user interface imple-
mented in HTML/Javascript/CSS. The server stores all user data in a database
with extensive logging of user actions. Log entries are created when a user loads
a page, uses an interactive feature such as multiple choice question or Parson’s
problem, edits or runs some program code, and others.
For our e-textbook prototype we plan to modify the Runestone system to
include API calls on the Personis server to add information to the user model.
2
http://runestoneinteractive.org/
4 J. Kay et al.
We prefer to make minimal changes to the e-textbook platform with a view to
instrumenting other e-text platforms in the future. The minimum requirement
is that the platform be capable of logging activity to a remote server with a
simple http request. The sort of activity to be logged includes: fetching a page
of the book, answering multiple choice questions, attempting Parson’s problems
(rearranging lines of text to get the answer).
We do not plan to customise or personalise the e-text itself. Instead the
logged activity will be used to drive a scaffolding interface that will guide the
user through the e-text.
The architecture diagram illustrates this. The e-text platform sends activity
information to our user modelling system(Personis) via a thin layer (shown in
grey) that maps e-text elements onto the user model ontology. The user model
contains rules that fire when incoming log information cause a significant change
in the resolved value of model components and send information to the Open
Learner Model and scaffolding interface.
3 Worked example and design rationale
We briefly outline plans for creating an e-textbook for a subject on Human-in-
the-loop Data Analytics. This starts with the definition of the learner model
hierarchy of contexts and components. The top level should be small enough for
learners to readily see their high level progress. For example, our top level has:
– Ethics, privacy, anonymisation, provenance, security and access control
– Human aspects of raw data
– Data collection
– Data cleaning and interpretation
– Processing, machine learning
– Exploration and interpretation interfaces
– Reporting
Within these topics are finer grained components. Ideally, the e-textbook would
have interfaces to support easy linking of the content to knowledge components
[1] but we will start by simply post-processing of the log data to create evidence
in the model. The OLM will also have links back to the relevant parts of the
e-textbook, based on the evidence in the model as in MOOClm [7].
4 Discussion and conclusions
Our proposed approach has the following core concepts:
Core role of the Open Learner Model : This offers a light-weight way to provide
personalised information about learning progress and a foundation for planning
learning. It avoid any need to alter the actual e-textbook and should be able to
integrate learning data from additional sources such as an LMS that has data
about results of graded assessments.
Title Suppressed Due to Excessive Length 5
Need for scaffolding: Since the OLM is intended to play an important role in
supporting learner reflection, progress monitoring and planning learning, we see
the need for personalised support to help the learner see and understand their
model.
Our architecture is based on a vision for each student to have a personal
long term scrutable user model. This can provide a unified interface to support
their meta-cognitive processes of reflection, self-monitoring and planning their
learning in a subject. This is based on the teacher using an e-textbook platform
to create a coherent set of resources that include teacher-authored material, links
to online materials used as readings as well as self-test and assessment materials.
References
1. Apted, T., Kay, J., Lum, A.: Supporting metadata creation with an ontology built
from an extensible dictionary. In: International Conference on Adaptive Hyperme-
dia and Adaptive Web-Based Systems. pp. 4–13. Springer (2004)
2. Azevedo, R., Hadwin, A.F.: Scaffolding self-regulated learning and metacognition–
implications for the design of computer-based scaffolds. Instructional Science 33(5),
367–379 (2005)
3. Brusilovsky, P., Schwarz, E., Weber, G.: Elm-art: An intelligent tutoring system
on world wide web. In: International conference on intelligent tutoring systems.
pp. 261–269. Springer (1996)
4. Bull, S., Kay, J.: Student models that invite the learner in: The smili:() open learner
modelling framework. International Journal of Artificial Intelligence in Education
17(2), 89–120 (2007)
5. Bull, S., Kay, J.: Open learner models as drivers for metacognitive processes.
In: Azevedo, R., Aleven, V. (eds.) International Handbook of Metacognition and
Learning Technologies, pp. 349–365. Springer (2013)
6. Bull, S., Kay, J.: SMILI: a Framework for Interfaces to Learning Data in Open
Learner Models, Learning Analytics and Related Fields. I. J. Artificial Intelligence
in Education 26(1), 293–331 (2016). https://doi.org/10.1007/s40593-015-0090-8,
https://doi.org/10.1007/s40593-015-0090-8
7. Cook, R., Kay, J., Kummerfeld, B.: Mooclm: User modelling for moocs. In: User
Modeling, Adaptation and Personalization - 23rd International Conference, UMAP
2015, Dublin, Ireland, June 29 - July 3, 2015. Proceedings. pp. 80–91 (2015)
8. Guerra-Hollstein, J., Barria-Pineda, J., Schunn, C.D., Bull, S., Brusilovsky, P.:
Fine-grained open learner models: complexity versus support. In: Proceedings of
the 25th Conference on User Modeling, Adaptation and Personalization. pp. 41–49.
ACM (2017)
9. Hsiao, I.H., Sosnovsky, S., Brusilovsky, P.: Guiding students to the right questions:
adaptive navigation support in an e-learning system for java programming. Journal
of Computer Assisted Learning 26(4), 270–283 (2010)
10. Kay, J.: The um toolkit for cooperative user modelling. User Modeling and User-
Adapted Interaction 4(3), 149–196 (1994)
11. Kay, J.: Learner know thyself: Student models to give learner control and respon-
sibility. In: Proceedings of International Conference on Computers in Education.
pp. 17–24 (1997)
6 J. Kay et al.
12. Kay, J., Kummerfeld, B.: Creating personalized systems that people can scruti-
nize and control: Drivers, principles and experience. ACM Trans. Interact. In-
tell. Syst. 2(4), 24:1–24:42 (Jan 2013). https://doi.org/10.1145/2395123.2395129,
http://doi.acm.org/10.1145/2395123.2395129
13. Miller, B., Ranum, D.: Runestone interactive: tools for creating interactive course
materials. In: Proceedings of the first ACM conference on Learning@ scale confer-
ence. pp. 213–214. ACM (2014)
14. Miller, B., Resnick, P., Ericson, B.: Using and customizing open-
source runestone ebooks for computer science classes (abstract only).
In: Proceedings of the 2017 ACM SIGCSE Technical Symposium
on Computer Science Education. pp. 741–741. SIGCSE ’17, ACM,
New York, NY, USA (2017). https://doi.org/10.1145/3017680.3017814,
http://doi.acm.org/10.1145/3017680.3017814
15. Tang, L.M., Kay, J.: Scaffolding for an olm for long-term physical ac-
tivity goals. In: Proceedings of the 26th Conference on User Model-
ing, Adaptation and Personalization. pp. 147–156. UMAP ’18, ACM,
New York, NY, USA (2018). https://doi.org/10.1145/3209219.3209220,
http://doi.acm.org/10.1145/3209219.3209220