=Paper=
{{Paper
|id=Vol-381/paper-6
|storemode=property
|title=Sharing Knowledge and Promoting Reflection through the Learner Model
|pdfUrl=https://ceur-ws.org/Vol-381/paper05.pdf
|volume=Vol-381
}}
==Sharing Knowledge and Promoting Reflection through the Learner Model==
https://ceur-ws.org/Vol-381/paper05.pdf
1
Sharing knowledge and promoting reflection through the
learner model
Kyparisia A. Papanikolaou1 and Maria Grigoriadou2
1
Department of Education, School of Pedagogical and Technological Education, Greece
2
Department of Informatics & Telecommunications, University of Athens,
Panepistimiopolis, GR 15784, Athens, Greece
spap@di.uoa.gr, gregor@di.uoa.gr
Abstract. In this paper, we discuss how externalising learners’ interaction be-
haviour may support learners’ explorations in an adaptive educational hyper-
media environment that provides activity-oriented content. In particular, we
collect raw data from learners’ interaction, model the state of interaction using
a set of indicators and contextual information, and visualize this information
alongside with comparative information coming from the instructor or col-
leagues. This way we provide learners with a mirror of their behaviour and
relative measures such as instructor’s proposals or peers’ behaviour, aiming (a)
to promote learners’ reflection on their learning and support them self-
diagnose the efficacy of their interaction; (b) to help learners to plan their
learning; (c) to facilitate collaboration because learners can improve under-
standing of themselves and each other, and select appropriate partners; (d) to
support tutors in providing personalised guidance and instruction and evaluate
the available educational content.
1 Introduction
INSPIRE [1] is an adaptive educational hypermedia environment that allows learn-
ers to freely explore the available content offering them individual advices. The con-
tent consists of a variety of modules for learners, ranging from expository examples
to open problems that promote learners to explore the underlying concepts. Several
activities embed microworlds developed with MicroworldsPro (LCSI:
http://www.microworlds.com/) or involve tools available on the Internet such as
simulations, aiming to increase interactivity and enhance learner control. Activities
are usually based on a specific scenario that promotes observation, exploration, and
hypothesis testing. Currently the data available from learners’ interaction with the
microworlds are their answers and explanations to particular questions embedded in
the activity-scenario.
INSPIRE supports learners to improve the effectiveness of their explorations,
mainly at content level, providing adaptive support based on learners’ individual
characteristics, i.e. structuring the content around specific learning goals augmented
with visual queues that inform learners about the content that they are ready to study
based on their knowledge level (adaptive navigation support technique), or providing
individualized versions of the educational material pages with alternative sequencing
of the modules involved based on learners’ learning style (adaptive presentation sup-
2
port technique). Learners are free to follow or not these advices on how and what to
study. Another type of support that we elaborate on is modelling the learners' interac-
tion with the system and visualizing this information to the learner and tutor in a
meaningful way through the learner model. Opening the learner model to learners and
using a variety of strategies to support interaction with the learner model provide
learners with opportunities for reflection [2, 3, 4].
Especially, in an educational hypermedia environment such as INSPIRE, learners
make explicit decisions repeatedly during interaction usually resulting in complex
interaction protocols. These protocols refer to the series of events which occur during
hypermedia usage with corresponding time stamps [5]. However, collecting learner
actions is the first step for re-constructing a view of learners’ activity able to promote
learners’ reflection on their explorations. Additionally, heterogeneous data included
in interaction protocols must be carefully handled in order to yield meaningful infor-
mation and build a thorough view of learners’ activity. To this end, contextual infor-
mation about the learner, the content, the available tools, the adaptive guidance of-
fered is necessary.
Our proposal for designing support for learners’ explorations combines and
expands ideas coming form the areas of open learner modelling, interaction
analysis [5,6,7] and computer supported collaborative learning [8]. In particular, we
collect raw data from learners’ interaction, model the state of interaction using a set
of indicators and contextual information, and visualize this information alongside
with comparative information coming from the instructor or colleagues. This way we
aim to provide learners with a mirror of their behaviour and relative measures such as
instructor’s proposals or peers’ behaviour, to support learners self-diagnose the effi-
cacy of their interaction. In this context, challenging research goals are modelling
learners’ behaviour and the ‘context’ that affects learners’ actions, and visualizing
this information in a meaningful way for both learners and tutors. In particular we
aim to design an open learner model that supports (a) learners observe and self-reflect
on their behaviour - i.e. think about consequences and implications of their own ac-
tions -, and change it if necessary -i.e. consider the consequences and efficacy
of their actions-, (b) the system in putting an interpretation on learners' actions, (c)
tutors in acquiring a comprehensive image of learners' work useful to assess learners'
performance, interests and needs, and evaluate the content.
2 Modelling the content
INSPIRE provides learners with structured content which is comprised of units, such
as learning goals, concepts and educational material modules that can be reused by
learners of different profiles. The notion of learning goals is used in order to build a
hypermedia structure that provides learners with an overview of how all the relevant
information fits together. In particular, each goal is associated with a conceptual
structure that includes all the necessary domain concepts and their relationships –
outcomes, prerequisites, related concepts. Each outcome concept is accompanied by
educational material pages that consist of a variety of content modules of different
interactivity levels, usually focus on learners’ misunderstandings/false beliefs, and
3
support specific levels of performance. For example, a page for the ‘loop construct’
concept (Learning goal: ‘How to use loop constructs’) may focus on the condition
terminating the loop or the infinite loop, topics quite difficult for novice program-
mers. The design of the educational material is activity-oriented aiming to promote
learners to use tools, generate and test hypothesis in a real context, solve open prob-
lems exploring alternative options. To this end, microworlds have been developed,
and several tools have been located on the Internet.
Different types of content modules have been developed such as (i) modules that
visualise specific internal processes along with appropriate explanations aiming to
stimulate learners observe important parameters that affect the evolution of the proc-
ess, (ii) modules that simulate a guided exploratory environment and usually incorpo-
rate a microworld, promoting learners to explore specific issues following a scenario
(see Fig. 1), (iii) modules that pose open problems for investigation.
Goal
Concept
Educat. ‘Activity’
material
Pages
Link to ‘Theoretical issues’
Navigation Link to an ‘Example’
area Content
Link to an ‘Exercise’ area
Fig.1: Educational material page of the ‘Use’ level of performance appears at the Content area
of INSPIRE including multiple types of modules: an exploratory activity that embeds a mi-
croworld, theoretical tips, an expository example, an experimental activity.
Content modules are combined in educational material pages of different performance
levels (‘Remember’ level: focusing on understanding, ‘Use’ level: focusing on the
use of the underlying concepts, ‘Find’ level: focusing on generating new generali-
ties). For instance, an educational material page of the ‘Use’ level that aims to gradu-
ally introduce the loop construct to learners includes the following modules: (a) an
example that visualises the instruction flow in the loop construct in a real program
and explain the main parameters involved and the evolution of the process through
the execution of the program, (b) a guided exploratory environment in which learners
are expected to investigate specific parameters of the loop construct following a spe-
4
cific scenario, such as the role of the counter in the evolution and termination of the
loop construct, and (c) an open problem involving the loop construct.
Each educational material page is currently described by a set of metadata con-
sisted of three types of descriptors based on the ARIADNE Educational metadata
recommendation (see Table 1): (a) General: groups the general information that de-
scribes the learning object such as document title, document language, etc., (b) Se-
mantics: groups elements that describe the semantic classification of the learning
object, (c) Pedagogical: groups elements that describe the pedagogic and educational
characteristics of the learning object, (d) Technical: groups elements that describe the
technical requirements and characteristics of the learning object.
Table 1: A sample of the metadata information of the educational material page {Edxx}. The
page is entitled “The main parameters of the loop construct” and belongs to the outcome con-
cept “Loop construct” of the learning goal “How to define a loop construct”.
General Information Semantics of the re- Pedagogical attributes
source
Identifier: Edxx Discipline: End User Type: 'Learner'
Title: “The main parameters 'Computer Programming' Doc. Format: 'Text' & ‘mi-
of the loop construct” Sub-discipline: croworlds’
Authors: 'K. Papanikolaou, 'Programming constructs' Usage Remarks: ‘get Mi-
K.Maragos, K.Glezou' Main Concept: croworldsPro plugin’
Date: '19/01/2006' ‘Loop construct’ Didactical Context:
Publisher: 'Dept. of Informat- Learning Goal: “How to 'High School’
ics & Telecommu-nications, define a loop construct” Interactivity level: ‘High’
Univ. of Athens' Main Concept Synonyms: Difficulty Level: 'Use'
Sources: 'Computer Pro- Other Concepts: Semantic Density: 'High'
gramming’ textbook Pedagogical Duration: 20
We currently work on a typology of the content (at module and page level), available
tools, and tasks involved aiming to extent the above metadescription and support the
production of more interpretative views of learners’ interaction. An interesting direc-
tion is also to extend descriptive metadata with ‘usage information’ representing
information about how the learner interacted with the content, including observed
metrics such as study time, number of learner hits, submissions, along with patterns
of access, explorations etc [9]. This type of information may come from individual
learners by recording their experiences through the interaction and inspecting these
interaction instances for meaningful patterns of success or failure for learners with
particular profiles.
3 Enabling shared decision making through the learner model
While learners working with INSPIRE, the system maintains information about learn-
ers’ interaction, selections, and submissions. This information is shared with learners
through their learner model. Especially, the learner model of INSPIRE has been ex-
tended to provide learners with appropriate tools and information, allowing them to
intervene to the adaptive behaviour of the system, see and contribute to their profile,
5
and acquire an image of their interaction behaviour. In particular, the learner model of
INSPIRE is divided in 4 sub-areas, whilst learners are currently allowed to update the
first three areas: (a) the ‘Learning Style’ area which shows learners’ current learning
style category used for system adaptation as well as the whole pattern of learners’
learning style (i.e. learners’ preference on all the different styles is shown), and pro-
vides them the opportunity to manually change it or resubmit the learning style (ques-
tionnaire of Honey and Mumford); (b) the ‘Adaptive Navigation Mode’ area which
allows learners to select the type of adaptive navigation technique among Hiding
(hides non-suggested content), Disabling (disables non-suggested content), and Vis-
ual Commenting (graphically augments links to non-suggested content) since there
are pros and cons for all the three techniques depending on the learners’ knowledge
level and level of expertise in using computers; (c) ‘Knowledge Level’ area shows
learners’ knowledge level on the domain concepts and information about their per-
formance, objectives they have attained; (d) ‘Interaction Analysis’ area providing a
mirror of learners’ interaction compared to a model suggested by the tutor.
Below we focus on the externalisation of interaction analysis which is a main chal-
lenge in opening the learner model of INSPIRE [10] to learners and tutors. INSPIRE
gathers data from learner’s interaction with the system and visualises it augmented
with contextual information, in order to support learners gather evidence to evaluate
the efficacy of their moves. Key issues in this process are the selection of the appro-
priate data (learners’ actions and contextual information) and the production of inter-
pretative views along with a meaningful way for conveying them to the learner.
Selection of appropriate data. A set of indicators from learners' interaction with the
content and tools of the educational environment have been selected that represent the
state of interaction. In particular, we use navigational indicators such as number of
hits, frequency of visits, temporal indicators such as time spent on different types of
resources and assessment - cases of long intervals of learners’ work are marked -, and
performance indicators such as attempts on assessment questions, performance on
multiple types of questions, indicators of learner’s interaction. Indicators are recorded
at three levels of observation grain, coarse, intermediate, fine, in order to provide a
comprehensive view of learners’ activity (see below for a detailed description). The
above information is provided along with contextual information about the content
that the learner encounters during the interaction such as type, semantic density, and
the tools they use such as the learner model, note keeping, adaptation controls.
Producing interpretative views of learners’ activity. Interpretative views of learners’
activity may support the investigation of purposeful chunks of actions in learners’
interaction protocols taking into account that the key to finding meaningful patterns is
the purpose for which the patterns are sought. Such views aim to be used as reflec-
tion-support mechanisms by learners and evaluation tools by tutors. A first step to-
wards this direction is to combine the indicators of learners’ interaction with contex-
tual information, and design appropriate visualizations. The indicators are illustrated
along with the currently available semantic information of the content such as type
and semantic density. Semantic density is proposed by the tutor but we intend to
alternatively evaluate it based on peers’ interaction e.g. reflect mean time spent on
specific resources by selected peers. For example, the time that the learner has spent
on specific resources, is presented aside the semantic density of the resources as pro-
6
posed by the tutor filling the corresponding line– when this time exceeds the tutor’s
proposal it turns to red (see Fig.2, area (a)).
A critical issue in representing the interaction indicators is the definition of the ap-
propriate observation grain, which relates to the precision of the events considered as
units in the analysis of the interaction protocols [5]. Three different levels of observa-
tion grain have been considered for learners’ interaction, ranging from global activity
patterns (coarse grain) when studying a goal, to specific aspects of the interaction at
an intermediate grain, that relate to specific events of interest useful when testing
specific hypotheses about the cognitive processes at work, and at a fine grain where
all the observable actions are taken into account and the analysis focuses on meaning-
ful patterns.
PartI
(a)
PartII
PartIII
Fig.2: Mirroring learner’s interaction at the coarse observation grain. The screenshot illustrates
learners’ interaction with the content of a learning goal during a time span of 27 minutes. The
data available is divided in three subareas reflecting learners’ activity on: (i) the content of the
domain concepts (Part I), (ii) the different types of pages and keeping notes (Part II), (iii) the
Learner model and adaptation controls (Part III).
In more detail, at the coarse level, information about learners’ global activity is
captured as a means to evaluate learners’ involvement in a learning goal. For in-
stance, information provided includes the total time they spent on the goal (at particu-
lar sessions or total study time) as well as learners’ activity with the content of the
domain concepts including for each concept, the time spent along with the semantic
density of the resource, visits along with total number hits on the content, level of
performance on different types of questions and the way this was evaluated (auto-
matically by the system or learner defined) (see Fig.2, Part I). Moreover, learners’
activity with the relevant educational material pages of different types is represented
(see Fig.2, Part II), as well as the use of tools like the learner model, note keeping and
adaptation controls (see Fig.2, Part II & III).
Information at the coarse observation grain may support learners plan their work,
manage their time, organize materials and resources, and schedule the procedures
7
necessary to complete a task. Moreover, tutors may acquire an image for the learn-
ers’ global activity and level of performance on the domain concepts.
The information at the intermediate grain permits a more detailed observation of
learners’ actions that reflect learners’ work with different types of resources –related
to learner’s style preferences - and the impact on their performance. For instance,
information provided reflects (a) learners’ global activity (see Fig.3, Part I) with the
educational material pages of a concept (time spent along with the semantic density of
the resource, visits along with total number hits on the content), and the content mod-
ules of different types involved (for each type of module, time spent is presented
along with the semantic density of the module), (b) learners’ activity with all the
different types of content such as educational material pages (see Fig.3, in Part II
pages of multiple types), and knowledge modules (see Fig.3, in Part III modules of
multiple types) including time spent, visits, information about learners’ performance.
Domain Concept
PartI
PartII
PartIII
(Learner A) (Learner B)
Fig.3: Mirroring learner’s interaction at the intermediate observation grain. These screenshots
illustrate interaction of two learners (Learners A and B) with the educational content of a par-
ticular domain concept. The data available is divided in three subareas reflecting learners’
activity on: (i) the particular educational material pages of the concept and on keeping notes
(Part I), (b) the different types of pages (Part II), (c) the different types of content modules
included in the pages (Part III).
For instance, information about the time a learner spent on specific resources com-
bined with the semantic density of the resources and the learners’ knowledge level
could provide a means to evaluate learners’ progress as well as the adequacy of the
content for particular learners. In Fig. 3, Learners A and B have spent almost the
same time on the domain concept, 14 and 11 minutes respectively. During this ses-
sion, Learner A concentrated on pages of the ‘Remember’ level working with exam-
ples (time spent exceeds the proposed one) but without submitting the relevant as-
sessment questions, whilst Learner B concentrated on ‘Use’ pages (although s/he also
visited the ‘Remember’ pages) working mostly with examples and answered success-
fully questions of the ‘Remember’ and ‘Use’ levels of performance. Both learners
8
seem to prefer working with ‘examples’, although their progress and the type of
pages they seem to prefer differ. However, more information about the particular
resources, tasks involved and learners’ submissions is necessary for a deeper view to
their activity.
Information at the intermediate observation grain reflects learners’ current activ-
ity with the content of a concept, their progress, as well as their preferences on spe-
cific types of resources. This information may support learners plan their work and
cultivate their style awareness. Moreover, sharing this information with peers may
support learners seeking for help. It may also support tutors acquire an image for the
learners’ global activity, progress and needs as well as for the adequacy of the re-
sources offered to learners with particular profiles.
Fig.4: Mirroring learner’s interaction with the content modules of a ‘Use’ page at a fine obser-
vation grain. In this screenshot, the learner’s navigation pattern at a particular educational
material page is illustrated. Note that ‘x’ illustrates long intervals.
The information at the fine grain regards learner’s activity on particular tasks al-
lowing the investigation of purposeful chunks of actions, the identification of repeti-
tive patterns of learners’ behaviour, and may provide a deeper view on the way learn-
ers use the resources and available tools. Content and tasks metadescriptions as dis-
cussed in Section 2 will provide a framework for interpreting learners’ actual use and
submissions. Valuable information at this grain may come also from learners’ interac-
tion with embedded microworlds – a direction that we intend to investigate. In any
case, the information at this grain allows the investigation of the evolution of learn-
ers’ activity. To this end, a record of learners’ interaction with the content over time
is necessary including a sequence of interaction instances for subsequent time peri-
ods. This sequence forms a “learning trail” through the content for a learner, and this
trail may reveal learners’ preferences, strategies and interesting patterns of success
and failure. Furthermore, by comparing learners’ trails, we may result in interaction
patterns for learners with particular profiles.
For instance, information about the resources or tools that a learner uses when un-
dertaking specific tasks combined with self-explanations and evaluations submitted
9
by the learners may provide a view of learner strategies and/or a valuable resource for
evaluating the content. Thus, a possible interpretation of the pattern of Fig. 4 depict-
ing the sequencing of modules that a learner adopted, is that the focus of the interac-
tion is on the ‘Activity’ since the learner revisits the ‘Activity’ module and in the
meantime s/he frequently visits the modules ‘Example’ and ‘Theory’ in order to get
information and complete the activity. Contextual information about the tasks that
learners undertake through the activity and modules used, may provide a deeper in-
sight in learners’ goals and the purpose of the interaction. Moreover, involving learn-
ers in the interpretation of their interaction patterns is necessary for minimising arbi-
trariness in the identification of meaningful patterns.
Information at the fine observation grain may promote learners’ awareness on
their learning and reflection on the efficiency of their learning strategies. Sharing this
information with colleagues may give them new ideas and encourages deeper thought
about the implications of their own ideas and strategies. Moreover, this information
may support tutors evaluate the difficulties that a learner faces when working with
specific resources as well as the quality and adequacy of the content.
3 Discussion and future plans
Intelligent and Adaptive Educational Systems usually integrate adaptive and adapt-
able components that are based on shared decision making between the learner and
the system. Sharing knowledge that the system maintains through the interaction
promotes transparency in communication with the learner and involves learners in
decision making cultivating meta-cognitive skills. In this paper we discussed the open
learner model of INSPIRE as a means for sharing system internal knowledge about
learners and their interaction behaviour, with learners and tutors. Opening the learner
model and specifically visualizing the interaction aims to provide a meaningful mirror
promoting learners reflect on their activity considering efficacy of their actions to
their objectives. Interaction patterns if related to learners’ profiles may also support
content evaluation, as well as adaptation of tasks, tools or study advices to learners’
individual characteristics. Moreover, it may support social interaction providing a
basis for learners to share their experiences or for group formation purposes.
Especially in Exploratory Learning Environments which encourage the learner to
create their own solutions to problems, the provision of a meaningful mirror of their
activity may support self-reflection, and knowledge sharing. However, there are an
enormous number of patterns that can be found when inspecting actual learner behav-
iour. As key issues that should be taken into account in producing interpretative
views of meaningful patterns useful to learners and their peers are the purpose for
which the patterns are sought, and contextual information relating for instance to the
learner (profile and personal view), the content, the available tools. Purpose and ap-
propriate contextual information about the learning environment, place their own
particular constraints on what patterns are meaningful, how to look for these patterns,
and how to use what these patterns reveal in order to achieve the purpose. However,
the interpretative views of learners’ interaction produced by system designers and
their expressive power of learners’ cognitive processes is important to be evaluated
10
by the learners themselves. Learners’ personal views on their interaction patterns or
of their peers will prove what actually these patterns reveal.
Currently the evaluation of the learner model of INSPIRE
(http://hermes.di.uoa.gr/inspire3) is in progress. In particular we investigate the ex-
pressiveness of the indicators and contextual information selected and the visualisa-
tions used. Preliminary results show that learners want to have access to their model
and to information maintained by the system, but most of them do not feel safe to
intervene to the information provided. They need support in order to interpret the
contents of their model and be able to creatively use them. They suggest that the com-
bination of temporal and performance indicators may support them in changing their
studying behaviour, whilst navigational indicators increase their awareness of the way
they use different types of resources. We investigate what the interaction patterns of
themselves or their peers reveal to them and how they might use them. We also intend
to further work on building interpretative views of the fine level of observation grain
for learners’ interaction based on specific purposes and learners’ profiles and on the
way these may augment the learner model of individual or groups of learners.
4 References
1. Papanikolaou, K.A., Grigoriadou, M., Kornilakis, H., Magoulas, G.D. (2003). Personaliz-
ing the interaction in a Web-based educational hypermedia system: the case of INSPIRE.
User-Modeling and User-Adapted Interaction 13 (3) 213-267.
2. Bull, S. & Kay, J. (2007). Student Models that Invite the Learner In: The SMILI Open
Learner Modelling Framework, International Journal of AI in Education, 17(2), 89-120.
3. Morales, R., Pain, H., and Conlon, T. (2001). Effects of Inspecting Learner Models on
Learners’ Abilities, Proceedings of AIED’01. IOS Press, 434-445.
4. Dimitrova, V., Self, J. & Brna, P. (2001). Applying Interactive Open Learner Models to
Learning Technical Terminology, in M. Bauer, P.J. Gmytrasiewicz & J. Vassileva (eds),
User Modeling 2001: 8th International Conference, Springer-Verlag, 148-157.
5. Rouet, J., Passerault, J. M. (1999). Analyzing learning hypermedia interaction: An over-
view of on line methods. Journal of Instructional Science, 21 (3).
6. Zapata-Rivera, J.D. and Greer, J. (2003). Analyzing Learner Reflection in the Learning
Game. Proc. of AIED’03 workshop on Learner Modelling for Reflection, 288-298.
7. Avouris, N., Komis, V., Fiotakis, G., Margaritis, M. and Voyiatzaki. E. (2005) Logging of
fingertip actions is not enough for analysis of learning activities. Proc. Workshop Usage
Analysis in learning systems, AIED 2005, Amsterdam, July 2005.
8. Jerman, P., Soller, A. and Muhlenbrock, M. (2001). From Mirroring to Guiding: A Re-
view of State of the Art Technology for Supporting Collaborative Learning, in P. Dillen-
bourg, A. Eurelings, and K. Hakkarainen (eds.), Proc. of EuroCSCL, 324-331.
9. Brooks, C. and G. McCalla (2006). Towards flexible learning object metadata. Int. Jour-
nal of Continuing Engineering Education and Life-Long Learning 16(1/2) 50-63.
10. Papanikolaou, K. and Grigoriadou, M. (2005). Modelling and Externalising Learners’
Interaction Behaviour. In: J.Kay, A.Lum, D.Zapata (Eds.): Proceedings of the LeMoRe05
workshop in the context of AI in Education (AIED2005), Amsterdam, Netherlands, 52-61.