=Paper=
{{Paper
|id=Vol-495/paper-2
|storemode=property
|title=The Potential of Open Learner Models in Adaptive Virtual Learning Environments
|pdfUrl=https://ceur-ws.org/Vol-495/paper2.pdf
|volume=Vol-495
|dblpUrl=https://dblp.org/rec/conf/aied/VelezFBH09
}}
==The Potential of Open Learner Models in Adaptive Virtual Learning Environments==
https://ceur-ws.org/Vol-495/paper2.pdf
11
The Potential for Open Learner Models in
Adaptive Virtual Learning Environments
Jeimy VÉLEZa,b, Ramon FABREGAT b, Susan BULL c and David HUEVA b
a
University Pontificia Bolivariana Montería, Colombia
b
University of Girona, Spain
c
Electronic, Electrical and Computer Engineering, University of Birmingham, UK
Abstract. This paper presents the initial design of an Open Learner Model for an
Adaptive Virtual Learning Environment (SAVEMA, The Spanish acronyms of
Adaptive Educational Virtual System with Open Model), with the aim of helping
learners to reflect on their knowledge, and to support their self-directed use of a
Virtual Learning Environment.
Keywords. Adaptive virtual learning environments, open learner models
Introduction
In the context of virtual and blended education, several functionalities have been
proposed for Learning Management Systems (LMS): intelligent LMS (iLMS) based on
standards in aLFanet [1]; integration of the LMS Moodle (Modular Object-Oriented
Dynamic Learning Environment) and the adaptive hypermedia system (AHS) APeLS
(Adaptive Personalized eLearning Service) [2]; recommending service integrated into
the OpenACS/dotLRN framework via Web services in ALPE, EU4ALL,
ADAPTAPlan [3]; an adaptive virtual learning environment based on an Integral user
model [4]. Each of these works add adaptive characteristics to an existing LMS through
learner models. However, in these approaches the learner does not have access to their
learner model.
Learning management systems or courseware management systems offer a wide
variety of functionalities, such as integrating instructional material, e-mail, chat
sessions, online discussions, forums, assignments, etc. Recently some environments
have been extended to support standards and specifications in E-learning [5, 6].
Although these characteristics make this kind of system more versatile, and extensions
give them the potential for adaptive characteristics, even the most advanced LMS
systems tend to be used similarly to more traditional computer-assisted instruction
support.
On the other hand, many educational research projects have built systems which
may have lost some of the versatility, but gained characteristics such as: adaptive
behavior [7; 8; 9]; support for collaborative learning [10; 11] and promoting reflection
[12; 13; 14], encouraging learner independence and responsibility [15], improving
accuracy of the learner model [13; 14]; helping learners to plan and/or monitor their
learning [13; 14] and affording learners greater control over their learning [16] through
an Open Learner Model (OLM), among others. Although their use is generally more
restricted than LMS (for example, to a specific domain, or in specific research studies),
12
these approaches have shown some positive results, specifically system which have
open the learner model to the students in the educational area.
In [17] Adaptive Virtual Learning Environment named SAVE (the Spanish
acronyms of Adaptive Educational Virtual System) has been proposed. SAVE use the
LMS dotLRN which, besides be open source, has been suggested as useful for
reusability, accessibility [18] and usability [19]. The dotLRN platform was extended
with adaptive characteristics based on the competence levels of each learner. To carry
out the adaptive behavior, a unit of learning (UoL) has been designed with the IMS
learning design specification [IMS-LD]. During the design phase, the instructor defines
some variables which are used to set the competence level of the student. The
competence level is inferred by a multi-agent system (MAS) based on the questions
answered by the learner in tests with IMS questions and test interoperability [IMS-QTI].
The adaptive behavior is then obtained through the different paths previously defined
in the UoL. (Further details can be found in [4]).
To improve SAVE an Open Learner Model is proposed, this new proposed system
is named SAVEMA (the Spanish acronyms of Adaptive Educational Virtual with Open
Model). This paper focuses on the potential for opening the learner model in AVLEs
and the design of an OLM in SAVE.
The paper is organized as follow: In section 1 the Open Learner Model (OLM) and
relations with Adaptive Virtual Environment (AVLE) are introduced. In section 2
details about the learner model and adaptive characteristics of the Unit of Learning
(UoL)/course are presented. In section 3 initial design of SAVEMA is proposed.
Finally, the summary is presented.
1. The Potential for Using an OLM in an Adaptive VLE
Open Learner Models (OLM) are learner models that can be accessed by the user, in
full or in part, and have been used for a variety of purposes, e.g. improving accuracy of
the learner model; promoting learner reflection; helping learners to plan and/or monitor
their learning; and affording learners greater control over their learning [20].
At this stage of our work we focus on promoting learner reflection on their
competence level, as an important element to facilitate meta-cognitive behavior, in
accordance with suggestions that students who engage at a meta-cognitive level tend to
achieve significantly higher learning results [21]. In [22] reflection is defined as “a
generic term for those intellectual and affective activities in which individuals engage
to explore their experiences in order to lead to a new understanding and appreciation”.
There is evidence to suggest that effectiveness in the learning process could be
enhanced when a student reflects about their own knowledge [22; 23; 24]. Along the
same lines, it has been argued that OLMs have the potential to foster reflection and
meta-cognitive skills, as the system provides the user with a representation of their
understanding of a subject as a starting point [15]. Learning gains have indeed been
demonstrated in some instances, using a simple OLM presentation [25; 26].
As adaptive capabilities are added to a traditional VLE, learner model is available
to open to the user. The considerations and characteristics of this OLM are presented
below.
13
2. Learner Model and Adaptive Courses in SAVEMA
In this section general details about the learner model and adaptive characteristics of
the course are presented. The open learner model design in SAVEMA is presented in
the next section.
2.1. Learner Model
The learner model of the VLE is presented in accordance with the three layers
identified for the analysis of user models by Brusilovsky and Millan in [27]: what is
being modeled (nature), how this information is represented (structure) and how
different kinds of models are maintained (user modeling approaches).
The information represented in our learner model relates to competences; although
there are similarities with knowledge representation, the differences can be found in the
conception and the implications that these have for the learning process. The
competences are structured in a taxonomy (e.g. for a career, high school program),
defined with the IMS Reusable Definition of Competency or Educational Objective
[28] specification, and implemented as shown in figure 1.
As an overlay approach has been used [28], the implementation takes into account
the UoL structure used to build the domain model. The learner model is maintained
through a multi-agent system which builds and updates the learner model overlaying
the domain model with the competence level obtained by the student after answering
questions in the respective UoL [4].
Figure 1. Structure of the learner and domain model.
In figure 1, each division shows the specification used and the relations defined
between them. The structure of the domain model is based on components identified in
a competence proposed by Tobón in [29]. Tobón proposes a model which considers
tree elements in a competence: problems that the competence address to solve,
description of the competence which summarizes the main idea of the competence and
14
their context and criteria for evaluate if a competence is achieved or not. Besides,
Bloom taxonomy [30] has been used to classify development criteria in a competence
[31] (see table 1), and also for question classification. In table 1, competence levels are
defined: novice, intermediate and expert.
Table 1. Level of competence based on Bloom’s taxonomy
BLOOM
DESCRIPTION (COGNITIVE DOMAIN) LEVEL
OBJECTIVE
Remembers a fact without a real understanding of the
Knowledge
meaning Novice
Understanding Gets the meaning of the material
Can use the learned material in new and specific
Application
situations
Intermediate
Analysis Can divide a complex problem into different parts
Synthesis Can join different parts in order to create new entities
Evaluation Can judge values of a subject with a specific propose Expert
2.2. Adaptive Course
The adaptive course was designed for students in the Universidad Pontificia
Bolivariana in Colombia as a part of an introductory computing course for informatics
students. The course includes the topic Object Oriented Programming (OOP), which
has been used as the main topic for the design of the virtual course. Many of the
resources used for course generation were taken from SHABOO [32], and other
resources were provided by the course instructor. The course includes three parts:
Introduction, Objects and Class, and was built using the authoring tool “Reload
Learning Design Editor” and IMS-LD [33] specification.
The designer defines the level and number of competence(s) that could be
achieved in a course by a learner. In the Unit of Learning/course used two competences
were defined. The first competence could be achieved until novice level and the second
one could be achieved until intermediate level. Rules for adaptive behavior in
accordance with the competence level of each student were defined in the IMS-LD.
These rules take into account the values of each variable for carrying out the adaptation.
However, these variables need to be updated during run time. On completion of the
design phase, the UoL was uploaded to the dotLRN VLE and a run was created with
the package ims-ld for the Unit of Learning/course available for the students. Because
the variables in the package ims-ld in dotLRN need to be updated manually, we have
integrated it with a multi-agent system which performs this task (additional detail can
be found in [4]).
The Unid of Learning was loaded in the dotLRN platform that runs on a server in
UPB [34]. On this platform a class named Object Oriented Programming was created;
in which students have different services such as forums, chat, space to store and share
files, calendar, news, questionnaires, units of learning, among others. In the link to
units of learning, students can choose the UoL available to them.
The course was available for one month, and two tests were administered, in the
middle and at the end of the course for evaluate the student competence in the course.
15
The questions were designed mostly in SHABOO [32] and characterized with the IMS-
QTI [35] specification in the package Assessment in dotLRN. Each question is
identified with an id that allows the competence, the level and the performance
criterion that it assesses, to be tracked. A low average was obtained by the students in
the two tests. (The averages for the two tests are based on a scale from 0 to 5, and were
3.49 and 2.77 respectively.) We therefore aim to increase user engagement as has
previously been found to occur with the introduction of a simple OLM [36], in an
AVLE context.
3. SAVEMA in the SMILI� OLM Framework
The SMILI� OLM framework [20] is designed to help researchers to focus on the
main considerations for opening a learner model. These considerations have been used
in this section to present the description of our OLM. The framework include an overall
view in the OLM design which help to the designer focus on the main considerations
for open a learner model. The framework take into account the purpose, what is
modeled, how is the model presented and who controls de access to the model.
Furthermore some additional aspects are considered in each one of these considerations,
(see tables 2 to 5).
Table 2. Purpose of Open Learner Modelling.
Purpose
Navigation
access, control,
Collab /
Right of
Competition
Assessment
Plan /
Reflection
Accuracy
Monitor
trust
SAVEMA XX
In the table No. 2 the upper part shows the general issues. The lower part shows
the goals of openness of the learner model: XX for central goals; X for lesser goals and
x for minor concerns. There are many purpose for open a learner model, however in
this work the reflection purpose have been chosen as a way of promote meta-cognitive
state that encourage the autonomy and responsibility in the learning process.
Table 3. WHAT is modeled?
Properties Reflection
Elements Purpose
1. Extent of model accesible Complete
16
Partial X
2. Match underlying Competence level X
representation Knowledge
Difficulties
Misconceptions
3. Access to uncertainty Learning issues
Preferences
Other
Other users' LM
4. Role of time Previous X
Current X
Future
5. Access to sources of input Complete
Partial X
System X
Self
Peer
Instructor
Other
6. Access to model effect on Complete
personalization Partial
In the table 3 what is modeled is summary. The main aspects considered are that
the open learner model shows the competence level which take into account the current
and previously level achieved. Only the system has access to the sources input and the
learner model is showed in a partial way because there is some additional information
in the learner model that at this time is not opened.
Table 4. HOW is the model presented?
Properties Reflection
Elements Purpose
7. Presentation Textual (i.e...)
Graphical (i.e...) X (level, skill meter and
colours)
Overview
Targeted/all Details X
All Details
8. Access method Inspectable X
Editable
Addition
Student persuade
System encourage
Negotiated
9. Flexibility of access Complete
Partial
In the table 4 the way as the model is presented is described. The learner model is
presented in a skill meter way with some colors that help to identify levels and
competences. Not all details are available at this design in the leaner model. There are
different methods for do that presentation of the model. The inspectable method has
17
been chosen in the presentation of the learner model in SAVEMA, this means that the
student can view their learner model.
Table 5. WHO controls access to the model?
Properties Reflection
Elements Purpose
10. Access initiative comes System
from User X
Peer
Instructor
Other
11. Control over Complete
accessibility (to others) Partial
System X
Peer
Instructor
Other
Finally in the table 5 details about who control the access to the models is given. In
the design of this OLM the user access are defined but the student cannot decide what
is available to see.
In figure 2 the OLM is presented. On the left side, the competence levels novice,
intermediate and expert are shown using the colors yellow, blue and green, respectively.
Others visual effects are added to facilitate their differentiations. On the right hand side,
a skill meter is used for each competence at a specific level. Skill meters were chosen
as they are one of the most common forms of simple OLM adopted in systems [e.g. 25;
37; 38], and have enjoyed high levels of use in real (voluntary) use settings to support
university courses [36; 39].
As we have said, in the Unid of Learning/course design a competence could have
until tree levels: Novice, Intemediate and Expert. The number of levels depends of the
scope that the designer consider achievable. Because the designer of the course used
has considered only two levels, figure 2 does not show any criterion in the expert level.
The first competence has two criteria and the second one has one criterion. The skill
meter shows how much the learner has achieved in a specific level for a specific
competence.
18
Figure 2. Open Learner Model representation in SAVEMA.
4. Summary
SAVEMA was created with the purpose of achieving reflection in the context of an
adaptive VLE. The OLM represents competence level: novice, intermediate and expert,
which have been defined based on Bloom’s taxonomy [30]. The presentation of the
learner model is done through the use of levels, skill meters and colors, and the method
of access to the learner model is ‘inspectable’ – i.e. the learner can view their learner
model, but may not directly contribute information about their knowledge. Although
some preliminary studies have been done for validate SAVEMA this paper focus on the
design of OLM. Future work will deploy new designs of OLM and also other studies to
investigate the extent to which a OLM may facilitate use of a VLE; and investigate
whether students might also benefit from an OLM using other features described in the
SMILI� Framework, in the AVLE context.
References
[1] Santos, O.C., Boticario, J.G. and Barrera, C. (2004). A Machine Learning Multi-Agent Architecture to
provide adaptation in a standards-based Learning Management System. WSEAS Transactions on
Information Science and Applications Vol. 1 468 – 473
[2] Tiarnaigh, M. (2005) An integration of Moodle (Modular Object-Oriented Dynamic Learning
Environment) with an AHS (Adaptive Hypermedia System). Final Year Project.
[3] Santos, O.C., Raffenne, E., Granado, J., and Boticario, J.G. (2008). Dynamic support in
OpenACS/dotLRN: Technological infrastructure for providing dynamic recommendations for all in
open and standard-bsed LMS. OpenACS and .LRN conference. International Conference and
Workshops on Community based environments, Guatemala.
19
[4] Vélez, J., Fabregat, R., Nassiff, S., Fernandez, A., and Petro, J. (2008) User Integral Model in Adaptive
Virtual Learning Environment. World Conference on E-Learning in Corporate, Government,
Healthcare, & Higher Education, E-Learn.
[5] dotLRN. http://dotlrn.org/
[6] Moodle. http://moodle.org/
[7] Henze, N. (2000). PhD Thesis, Adaptive Hyperbooks: Adaptation for Project-Based Learning Resources.
[8] De Bra, P., and Stash, N. (2002). AHA! Adaptive Hypermedia for All. SANE Conference, Maastricht.
[9] Cristea, A. Floes, D. Stash, N. and De Bra, P. (2003). MOT meets AHA!, PEG’03, St. Petersburg, Russia.
[10] Olguin, C.J., Delgado, A.L., and Ricarte, I.L. (2000). An Agent Infrastructure to Set Collaborative
Environments. Educational Technology & Society, 3(3), 65-73
[11] Alfonseca, E., Carro, R., Martín, E., Ortigosa, A., Paredes, P. (2006). The impact of learning styles on
student for collaborative learning: a case study. Computer Science and Humanities, Social Sciences and
Law Issue Volume 16, Numbers 3-4
[12] Kay, J., Li, L., and Fekete, A. (2007). Learner reflection in student self-assessment. In Proceedings of
ACE 2007, 9th Australasian Computing Education Conference, pages 89-95. Australian Computer
Society.
[13] Bull, S., and Pain, H. (1995). Did I say what I think I said, and do you agree with me? Inspecting and
Questioning the Student Model. In J. Greer (Ed.) Proceedings of the World Conference on Artificial
Intelligence in Education (pp. 501-508). Charlottesville, VA, USA: Association for the Advancement of
Computing in Education.
[14] Bull, S., and McKay, M. (2004). An Open Learner Model for Children and Teachers: Inspecting
Knowledge Level of Individuals and Peers. In J.C. Lester, R.M. Vicari & F. Paraguaçu (Eds.)
Intelligent Tutoring Systems: 7th International Conference, ITS 2004 (pp. 646-655). Berlin Heidelberg:
Springer-Verlag.
[15] Bull, S. and Kay, J. (2008). Metacognition and Open Learner Models, in I. Roll & V. Aleven (eds),
Proceedings of Workshop on Metacognition and Self-Regulated Learning in Educational Technologies,
International Conference on Intelligent Tutoring Systems, 7-20.
[16] Czarkowski, M., and Kay, J. (2000). Bringing scrutability to adaptive hypertext teaching. In G. authier,
C. Frasson & K. VanLehn (Eds.) ITS '00: Proceedings of the 5th International Conference on Intelligent
Tutoring Systems (pp. 423-433). London: Springer-Verlag.
[17] Vélez, J. and Fabregat, R.: SAVE: Sistema Adaptativo Virtual de Educación. Submitted to Conferencia
de la Asociación Española para la Inteligencia Artificial (CAEPIA), Sevilla, España. 9-13. November
2009
[18] Santos, O.C. Boticario, J.G. Raffenne, E. and Pastor, R. (2007). Why using dotLRN? UNED use cases.
1st International Conference on FLOSS: Free/Libre/Open Source Systems.
[19] Martin, L., Roldán Martinez, D., Revilla, O., Aguilar, M.J., Santos, O.C., and Boticario, J. (2008).
Usability in e-Learning Platforms: heuristics comparison between Moodle, Sakai and dotLRN.
OpenACS and .LRN conference 2008. International Conference and Workshops on Community based
environments, Guatemala.
[20] Bull, S. and Kay, J. (2007). Student Models that Invite the Learner In: The SMILI� Open Learner
Modelling Framework, International Journal of Artificial Intelligence in Education 17(2), 89-120.
[21] White, B.Y., and Shimoda, T. A, and Frederiksen, J.R. (1999). Enabling Students to Construct Theories
of Collaborative Inquiry and Reflective Learning: Computer Support for Metacognitive Development.
Int. Journal on AI in Education, 10 151-182.
[22] Boud, D., et al. (1985). Promoting reflection in learning: A model. Reflection: Turning Experience into
Learning. D. Boud, R. Keogh and D. Walker. London, Kogan Page: 18-40.
[23] Schön, D. A. (1983). The reflective practitioner. New York, Basic Books.
[24] Schön, D. A. (1987). Educating the reflective practitioner. San Francisco, Jossey-Bass Publishers.
[25] Mitrovic, A. and Martin, B. (2007). Evaluating the Effect of Open Student Models on Self- Assessment,
International Journal of Artificial Intelligence in Education 17(2), 121–144.
[26] Shahrour, G. & Bull, S. (in press). Interaction Preferences and Learning in an Inspectable Learner
Model for Language, Artificial Intelligence in Education 2009, IOS Press, Amsterdam.
[27] Brusilovsky, P., and Millan, E. (2007). User Models for Adaptive Hypermedia
and Adaptive Educational Systems. The Adaptive Web Book. Springer Berlin / Heidelberg. Vol. 4321,
3-53
[28] IMS Reusable Definition of Competency. Version1.0. Final Specification. 2002.
[29] Tobon, S. (2008). La Formación Basada en Competencias en la Educación Superior. Bogota: cife
[30] Bloom, B.S. (1956). Taxonomy of Educational Objectives. New York: David Mckay.
[31] Baldiris, S., Santos, O. C., Barrera, C., Boticario J. G., Velez, J., and Fabregat, R. (2008). Integration of
educational specifications and standards to support adaptive learning scenarios in ADAPTAPlan.
20
International Journal of Computer Science and Applications (IJCSA). Special Issue on New Trends
on AI techniques for Educational Technologies. Vol 5, 1
[32] Llamosa, R. Guarín, I. Moreno, G. and Baldiris, S. (2003). Sistema Hipermedia Adaptativo para la
Enseñanza de los Conceptos Básicos de la Programación Orientada a Objetos. Universidad Industrial de
Santander, Coordinador CIDLIS.
[33] IMS Learning Design. Version 1.0 Final Specification, 2003.
[34] UPB Server with dotLRN installed. http://64.76.48.166:8999/
[35] IMS Question and Test Interoperability. Version 1.2.1 Final Specification, 2003.
[36] Bull, S., Mabbott, A., Gardner, P., Jackson, T., Lancaster, M., Quigley, S. and Childs, P.A. (2008).
Supporting Interaction Preferences and Recognition of Misconceptions with Independent Open Learner
Models, Adaptive Hypermedia 2008, Springer-Verlag, Berlin Heidelberg.
[37] Papanikolaou, K.A., Grigoriadou, M., Kornilakis, H., and Magoulas, G.D. (2003). Personalizing the
Interaction in a Web-Based Educational Hypermedia System: The Case of INSPIRE. User-Modeling
and User-Adapted Interaction, 13(3), 213-267.
[38] Weber, G., and Brusilovsky, P. (2001). ELM-ART: An adaptive versatile system for Web-based
instruction. International Journal of Artificial Intelligence in Education. 12(4), 351-384.
[39] Bull, S., Quigley, S. & Mabbott, A. (2006). Computer-Based Formative Assessment to Promote
Reflection and Learner Autonomy, Engineering Education: Journal of the Higher Education Academy
Engineering Subject Centre 1(1), 8-18.