=Paper=
{{Paper
|id=None
|storemode=property
|title=Epistemic Beliefs and Open Learner Models
|pdfUrl=https://ceur-ws.org/Vol-732/paper6.pdf
|volume=Vol-732
}}
==Epistemic Beliefs and Open Learner Models==
https://ceur-ws.org/Vol-732/paper6.pdf
Epistemic Beliefs and Open Learner Models
Matthew D. Johnson1, Peter Reimann2, Susan Bull1, Nobuko Fujita3
1
Electronic, Electrical and Computer Engineering, University of Birmingham, UK
2
MTO Psychologische Forschung und Beratung, Germany
3
Center for Applied ICT, Copenhagen Business School, Denmark
{mdj384, s.bull}@bham.ac.uk, p.reimann@mto.de, nf.caict@cbs.dk
Abstract. Information on epistemic beliefs has the potential to enrich feedback
to students and teachers, through open learner modelling (OLM). We outline
existing OLM applications with epistemic content and ways in which more ho-
listic aspects of learning and development (epistemic beliefs, values, identity
cognition etc.) may extend OLM for 21st Century learning. We propose three
steps to extend epistemic network analysis to provide an OLM which contains
epistemic beliefs.
Keywords: Open Learner Modelling. Epistemic Beliefs.
1 Introduction
In adaptive learning environments (ALEs), a learner model (LM) represents student
knowledge, allowing for personalisation and adaptation towards learners and their
current needs [1]. Traditionally the information in the LM may be read only by the
ALE (i.e. it is closed to the learner it represents.) An open learner model (OLM) al-
lows learner access to the LM’s content in a meaningful way, for example through a
visual representation of its content, during interaction. OLMs may be presented in a
variety of forms from skill meters and coloured nodes (Fig 1, a,b), through to concept
maps (Fig 1, d), animations and domain specific representations (Fig 1, c) [2]. Learner
responsibility, awareness and independence in learning are benefits of viewing an
OLM, and metacognitive activities such as self-assessment, planning and reflection
may be promoted [2,3,4].
Traditionally the content of the LM may range from core competencies (knowl-
edge, misconceptions etc. [5]) to measures of affect (e.g. motivation [3]). Information
less commonly modelled includes epistemic activities such as argument construction
[6]. Such activities are highly relevant to learning and may potentially enrich the
learning-based feedback OLMs provide. In this paper we look at the suitability of
including epistemic beliefs in OLMs, and consider methods to capture and display
epistemic information in the OLM context and conclude with a method for doing this.
Fig. 1. Example methods for presenting OLM information. The top two rows show existing
ways of presenting the information through: (a) knowledge level skill meters, metaphors col-
oured nodes and text etc.; (b) group model information; (c) detailed or domain specific presen-
tations; (d) concept maps. The bottom row shows potential ways further epistemic information
may be presented using existing means: (e) epistemic network analysis (adapted from [11]); (f)
overviews of 21st Century skill mastery.
2 Epistemic Beliefs in the Context of Open Learner Models
Learning is not restricted to mastering concepts, procedures or specific skills, but
includes the “ability to think, act and interact with others in productive ways to solve
complex tasks” [7]. Epistemic beliefs help address this point and can be considered as
those an individual holds about the nature of knowledge, and ‘knowing information’.
Epistemic beliefs focus on the systematic linking of knowledge and the justification
of understanding using evidence or prior understanding [8].
In the case of the epistemic activity of argument construction this can be summa-
rised formally by Toulmin’s [9] argumentation structure. OLMs such as xOLM [3]
use this structure to formally reason about student understanding, although these be-
liefs belong to the system rather than being the epistemic beliefs of the student. The
OLM presentation is highly visual, presents links between evidence and makes pre-
dictions about student ability. Warrants that justify claims are important to establish
their legitimacy. OLMs such as STyLE-OLM [5] allow joint construction of the LM,
requiring learners to justify beliefs, and in doing so the OLM may foster reflective
thinking in students, through negotiation. Again, representations are highly visual and
links are forged between beliefs. Relationships modelled are epistemic in nature and
show the systematic linking of low-level information. This is similar in concept map-
ping. In Flexi-OLM [4] learners constructed their own OLM presentations by linking
concepts. Relationships specified included formal relationships, information related to
planning learning, and information to aid future knowledge acquisition. In such activi-
ties, labels on links between information are important for developing epistemological
understanding. These examples show that existing OLMs have the potential to capture
and model epistemic information. Future challenges include scaling up the domain
content, increasing the epistemic information quality, visualising greater volumes of
epistemic beliefs, and inferring epistemic beliefs not explicitly stated by the learner.
Stoeger [10] highlights the potential for creating a LM composed of epistemic be-
liefs and proposes a non-computer based learner model, constructed from fixed ques-
tionnaire items (e.g. “The manner in which I learn [maths] will never change”). Using
the model, composed of three principal facets (epistemic inclination, epistemic acqui-
sition and knowledge characteristics), he demonstrates epistemic beliefs are effective
predictors of ability and more accurate than IQ tests. Extending OLMs to include
these facets provides a starting point for more formally modelling epistemic beliefs.
In the context of 21st Century learning, epistemic beliefs may not be solely about
knowledge and its acquisition, but of wider social/cultural experience, values and
practices [7] – potentially, a community of practice. For example, epistemic games
encourage learners to use concepts/ procedures in activities to learn what it is to think
and act as a member of a specific profession (e.g., journalist) [7]. The professional
competence is modelled as an epistemic frame containing inter-related skills, knowl-
edge, identity conceptions, values and epistemological beliefs [11]. The class-
room/home setting constitutes itself a community of practice, albeit a ‘non-
professionally based’ one; arguably OLMs could also model student identity concep-
tions, values and epistemic beliefs – by being extended to encompass rich, contextual
‘epistemic frame’ inferences. Epistemic network analysis (ENA) [11] is used to
model and display epistemic frame information, and the extent to which students
internalise its attributes. Using graphs and network diagrams the model is visualised
(in ways similar to Fig 1e), and changes over time may be discerned. OLMs could be
extended to include ENA-type information, and so give feedback to students and
teachers in real time regarding how they resemble epistemic frames of experienced
members of the community of practice. The LM incorporates process and product
data, and the OLM reveals this information to the student. This makes explicit, nor-
mally tacit metacognitive processes that experts participate in to engage students in
21st century learning. In alignment with the initial aims of OLMs, this may help with
short and long term planning, in addition to being a source to promote reflection. This
extends the benefits of OLMs to permit the acquisition of 21st Century skills (e.g.
collaboration [6], leadership [7], critical thinking [11]), through inspection of episte-
mologically based information. These skills, often labelled “soft skills”, are recog-
nised as key to innovation. They make the process of knowledge building accessible
to all, permitting the development of new ideas that are of value to others in a com-
munity. Students can build knowledge relative to their current level of understanding
through acculturation to the wider social/cultural practices of a community and ex-
perience more meaningful learning.
Thus: existing OLMs have the potential to encompass epistemic information within
current technologies; techniques exist as a starting point for modelling learners’ epis-
temic beliefs; and through considering the wider context provided by an epistemic
frame, OLMs can be extended to support the acquisition of 21st Century skills.
3 Realising Epistemic Open Learner Model Content
Epistemic information may typically originate from two sources: process data (people
interaction, whether between learners, a teacher or a non-learner) and product data
(tangible outcomes/artefacts, e.g. from learning activities) [7]. For OLMs, epistemic
beliefs could explicitly result from educational activities (e.g. writing reflective prose
[11], concept mapping [4] or negotiation [5]), or OLMs may be extended to encom-
pass other, more informal, sources (e.g. teacher observation, chat logs etc.). It is im-
portant to acknowledge that epistemic information may be latent and affected by other
measurable attributes [7] as a result of learners systematically linking information
[11], and that strong relations exist between epistemological facets. Information is
often modelled to a course level of granularity for precisely this reason [11].
OLM based epistemic information, originating in the classroom or home, has prac-
tical benefits for students, teachers and parents alike. Independent from technology,
teachers and parents can use the information to scaffold students’ learning and further
21st Century skills that mainstream classroom technologies do not necessarily pro-
mote. Teachers and parents may draw on their own personal experience to support
students in developing aspects of these skills, using the epistemic OLM information.
Furthermore, teachers’ feedback to students often draws on in-depth knowledge of
local circumstances and general context [13]. Epistemic OLM information could
specify the context in which to interpret core abilities (knowledge, misconceptions
etc.) and visualise pedagogically-based information. This may allow the prediction of
future behaviour, its evolution over time and inter-student relationships or belief net-
works that exist; these are useful tools for designing educational interaction.
Our research project makes advances in three steps to realise the vision that OLMs
for 21st Century learning may include more holistic aspects of learning and develop-
ment. These include epistemic beliefs, values, and identity cognitions, in addition to
'classic' KSAs (knowledge, skills and aptitudes).
The first step is towards automating ENA [11], which has so far been realised by
employing human coders. The construction of the basic data table, required to analyse
inter-frame development, will be automated by graph-theoretical means: the list of
time slices together with the frequencies of frame elements. In our approach we will
apply natural language processing techniques to extract concepts, relationships and
sentiments, and will use machine learning techniques to build classifiers (e.g. Support
Vector Machines). These techniques rely heavily on corpus data that is obtained when
human ‘raters’ perform content analysis in order to code student-produced materials
(mainly students’ writing documents) in terms of the epistemic frame dimensions.
This content analysis (valuable in its own right to answer research questions) will then
yield the data required to train the automatic classification method.
In the second step we will automate the analysis of the adjacency matrices that can
be built from the basic data table. This will require respective software to be written
and/or to reuse existing implementations of graph algorithms, such as the ones used
for social network analysis.
Our third step will visualise the information contained in adjacency matrices as
well as plotting graph theoretical parameters (such as centrality, density, etc.) over
time. In addition to writing the respective software, this requires research into the
graphical interface provided to the users. As stated above, existing OLM presentations
containing epistemic information are highly visual (links between information, col-
oured nodes etc.) and network diagrams/graphs are appropriate methods for represent-
ing epistemic information, with an emphasis on clarity. This is our starting point for
visualisation, and the OLM may be extended to include more emerging techniques
(e.g. tag clouds, timelines or sparklines [12]), which also allow collaborative activity
content to be represented.
In summary: appropriate information sources exist to build on the capabilities of
OLMs highlighted in Section 2, and extend epistemic information in alignment with
current student and teacher practices; secondly, we have highlighted that epistemic
OLM information may have the potential to support human-based scaffolding of
students’ 21st Century skill development; finally, we have outlined three steps towards
an OLM for 21st Century learning that can include more holistic aspects of learning
development.
4 Summary
Existing OLMs include activities that are appropriate to formally model epistemic
beliefs. By capturing aspects of process (people interaction) and product (learning
artefact) data, epistemic OLM information may be modelled and presented in a highly
visual manner. This has the potential to extend the state-of-the art to allow OLMs to
support students, teachers and parents in their respective roles, and to further students’
development of epistemic practices and 21st Century skills (e.g. collaboration, critical
thinking). In our work, we will extend approaches developed in former OLM research
to model and visualise epistemic activities and beliefs, and we will extend ENA in
three steps so that it can be applied to the assessment of school-relevant 21st century
learning.
Acknowledgments. This project is supported by the European Community (EC)
under the Information Society Technologies (IST) priority of the 7th Framework Pro-
gramme for R&D under contract number 258114 NEXT-TELL. This document does
not represent the opinion of the EC, and the EC is not responsible for any use that
might be made of its content.
References
1. Holt, P., Dubs, S., Jones, M., Greer, J.: The State of Student Modelling, Student Models. In
J. Greer and G. McCalla (eds.) The Key to Individualized Educational Systems, Springer-
Verlag, New York. (1994)
2. Bull, S., Kay, J.: Open Learner Models. In: in R. Nkambou, J. Bordeau & R. Miziguchi
(eds), Advances in Intelligent Tutoring Systems. 318-338, Springer-Verlag Berlin.
(2010)Morales, R.,
3. Van Labeke, N., Brna, P., Chan, M.: Open Learner Modelling as the Keystone of the Next
Generation of Adaptive Learning Environments. Intelligent User Interfaces: Information
Science Reference, IGI Global. (2008)
4. Mabbott, A., Bull, S.: Comparing Student-Constructed Open Learner Model Presentations to
the Domain, Artificial Intelligence in Education, IOS Press, Amsterdam (2007).
5. Dimitrova, V.: STyLE-OLM: Interactive Open Learner Modelling. International Journal of
Artificial Intelligence in Education 12. Pp 35-78. (2003)
6. Aditomo, A., Reimann, P.: Learning from virtual interaction: A review of research on online
synchronous groups. CSCL (2007)
7. Rupp, A., Gushta, M., Mislevy, R., Shaffer, D.: Evidence-centred Design of Epistemic
Games: Measurement Principles for Complex Learning Environments. Journal of Technol-
ogy, Learning and Assessment. Vol. 8, No. 4, pp. 3-47 (2010)
8. Porath, M., Lupart, J., Katz, J., Ngara, C., Richardson, P.: Gifted learners’ epistemological
beliefs. Talent Development and Excellence, 1, pp. 57-66. (2009).
9. Toulmin, S.: The uses of argument. MA: Cambridge University Press (1958)
10.Stoeger, H.: First steps towards an epistemic learner model. High Ability Studies, 17(1),
(2006)
11.Shaffer, D., Hatfield, D., Svarovsky, G., Nash, P., Nulty, A., Bagley, E., Frank, K., Rupp,
A., Mislevy, R.: Epistemic Network Analysis: A Prototype for 21st Century Assessment of
Learning. International Journal of Learning and Media. Vol. 1, No 2, pp 1-21 (2009)
12.Harrer, A., Zeini, S., Kahrimanis, G., Avouris, N., Marcos, J., Martinex-Monez, A., Meier,
A., Rummel, N., Spada, H.: Towards a flexible model of computer-based analysis and visu-
alisation of collaborative learning activities. Proceedings of the Intl. Society of the Learning
Sciences Computer-supported Collaborative Learning Conference. New Brunswick (2007).
13.Mislevy, R.: Issues of Structure and Issues of Scale in Assessment From a
Situative/Sociocultural Perspective (No. 668). Los Angeles: National Center for Research on
Evaluation, Standard and Student Testing (CRESST), (2006).