=Paper= {{Paper |id=Vol-2188/Paper2 |storemode=property |title=Supporting Competence-based Learning in Blended Learning Environments |pdfUrl=https://ceur-ws.org/Vol-2188/Paper2.pdf |volume=Vol-2188 |authors=Mikel Villamañe,Ainhoa Álvarez,Mikel Larrañaga,Jessica Caballero,Oscar Hernández-Rivas |dblpUrl=https://dblp.org/rec/conf/lasi-spain/VillamaneALCH18 }} ==Supporting Competence-based Learning in Blended Learning Environments== https://ceur-ws.org/Vol-2188/Paper2.pdf

Supporting Competence-based Learning in Blended
Learning environments

Mikel Villamañe1[0000-0002-4450-1056], Ainhoa Álvarez2[0000-0003-0735-5958],
Mikel Larrañaga2[0000-0001-9727-1197], Jessica Caballero1 and Oscar Hernández-Rivas1

1
Escuela de Ingeniería de Bilbao (UPV/EHU), Bilbao, Spain
{mikel.v, jcaballero009, ohernandez018}@ehu.eus
2
Escuela de Ingeniería de Vitoria-Gasteiz (UPV/EHU), Vitoria-Gasteiz, Spain
{ainhoa.alvarez, mikel.larranaga}@ehu.eus

Abstract. Assessment is a very relevant part of any instructional process. Its re-
sults, when adequately provided, can improve students’ learning. Assessment re-
sults could also be used to guide teachers in order to align their course design
with the students’ results. However, the analysis of the assessment results and the
extraction of useful conclusions for both teachers and students is not an easy task
and computerized tools can be very helpful to provide automation of these pro-
cesses. Moreover, the information regarding assessment of students can come
from many sources and in the case of blended learning environments, the number
of sources increases. In this paper, a system aimed at supporting Competence-
based Learning in blended learning environments (COBLE) is presented. The
paper presents the system’s architecture and its main elements. It is specially de-
scribed how information from different sources can be gathered together into the
system in order to update its models and analyze the assessment data as a whole.

Keywords: Competence-based Learning, Blended Learning, Visual Learning
analytics.

1 Introduction

Assessment is a key element of the instructional process [1], not only for the marking
of students but also for the enhancement of teaching and learning processes. Therefore,
it is very important to align the learning design of the courses with the results of stu-
dents’ assessment [1]. For that, it is required to guide teachers to make decisions over
their course design. This can be provided through, among others, the use of visualiza-
tion techniques, an important part of learning analytics [2]. The information provided
can also be helpful for students in order to improve their learning process. Those tech-
niques can also help students to self-reflect about their learning process [2, 3] and even
incite them to take remediation actions to improve their performance [4, 5].
Current educational approaches in Higher education are focusing on the one hand on
blended learning [6], which implies a thoughtful integration of face-to-face and online

Copyright © 2018 for this paper by its authors. Copying permitted for private and academic purposes
Learning Analytics Summer Institute Spain – LASI Spain 2018

approaches and technologies [7]. On the other hand, they also center on the mastery of
skills or competences [8].
In this kind of educational environment, the information regarding assessment in a
course can come from many sources and it should be integrated in order to analyze it
and provide adequate feedback to users [9]. To help users in this process, this paper
presents a system that aims to support users (students and lecturers) in competence-
based blended learning environments through the use of Visual Learning Analytic tech-
niques called COBLE (Competence-Based Learning Environment).
This paper describes the main components of COBLE and how its models can be
updated with information from other learning environments or tools. This paper first
presents the architecture of COBLE and then it describes its domain and learner models.
After that, the paper centers on the aspects related to the data extraction from systems
the students have interacted with, on the loading of that information into COBLE and
on how to update the learner model using that information. Next, some examples of
visualizations provided by COBLE are presented. The paper finishes with some con-
clusions and future work.

2 COBLE

COBLE, like any learning support system, represents the domain and learner models
(see Fig. 1) which structure is detailed in the following section. It has also two main
modules (see Fig. 1) that interact with the information stored regarding the domain and
the learners. These two modules interact with those models in order to support compe-
tence-based learning environments.

……

DATA M AN AGEM EN T M ODU LE

DOMAIN MODEL LEARNER MODEL

V I SUALI Z AT I ON M ODU LE

Fig. 1. Architecture of COBLE
Learning Analytics Summer Institute Spain – LASI Spain 2018

In a blended learning environment, students can interact with different tools and their
marks can come from many different sources. For example, the students can use online
learning tools such as the Moodle Learning Management System (LMS) or e-assess-
ment systems such as AdESMuS [10] and some of the students’ marks can be stored in
them. However, these students can also have part of their marks collected out of these
systems by the lecturer either in a spreadsheet or they can also be directly stored in
COBLE.
COBLE is able to integrate all this information in its learner and domain models as
it will be described in the following section. In order to do that, COBLE incorporates a
Data Management module.
Finally, a Visualization module is in charge of applying visual learning analytic tech-
niques in order to provide visualizations for both students and lecturers. The particular
aspects of the provided visualizations are described in the following sections.

3 Domain and Learner Models

The domain model of COBLE represents the course competences taking as a basis the
Competence-based knowledge space theory (CbKST) [11]. Therefore, they are for-
mally structured using prerequisite relationships (see Fig. 2).
The competences are also related to the course topics, resources and to a set of as-
sessable items or exercises that will allow to determine the mastery level on each com-
petence. The system also includes the possibility of using evaluation rubrics for the
assessment of the assessable items.

Fig. 2. Domain model structure

The learner model is divided into the following three main parts.
• Historical data: represents information related to the different work sessions carried
out by the student (duration, exercises the student has been involved with, made
mistakes and so on).
• Knowledge model: stores information related to the student knowledge level on each
course topic.
• Competence model: This model includes the competence mastery level achieved by
the student for each competence considered.
Learning Analytics Summer Institute Spain – LASI Spain 2018

4 Data Extraction, Loading and Model Updating

The Data Management module allows lecturers to interact with the domain and learner
models in order to load or update the information stored in them. The module has an
interface that allows lecturers to directly define or update both the domain and the
learner model. COBLE also allows lecturers to import the information related to the
interactions of the students with the domain extracted from an external system.
In order to do that, the user can interact with the interface to introduce the infor-
mation individually but it can also load information provided in a text file with comma-
separated values.

4.1 Data Extraction
Several systems allow the extraction of information related to the interactions and per-
formance of the students related to different resources. For example, Moodle allows
lecturers to export information related to grades obtained by the students in the course’s
assignments and quizzes among other items. Fig. 3 shows the lecturer selection of items
to be exported from Moodle’s gradebook (on the left) and the generated text file with
comma-separated values loaded in a spreadsheet (on the right).

Fig. 3. Data extraction from Moodle

Even when lecturers do not use any management tool to evaluate the student tasks, they
usually store this information in some document, frequently in a spreadsheet.
In order to be able to load into COBLE the extracted data, it must be stored in a
comma-separated values text file, which is one of the available formats in Moodle and
spreadsheets among other platforms. The only compulsory field in the file is the stu-
dent’s e-mail because it is used as the student’s identifier in COBLE. This makes it
Learning Analytics Summer Institute Spain – LASI Spain 2018

easier to import and integrate data from different systems as it does not compel the
student to have the same user name in the different systems being used.
Currently, COBLE supports the loading of comma-separated values text files where
each line represents the information of a particular student. This format was established
because it is the same structure in which Moodle data can be exported and because it
also matches the structure of the spreadsheets used by many teachers. However, it can
be easily extended to incorporate other file structures.

4.2 Loading
Once the information to integrate in the system has been extracted, the Data Manage-
ment module is in charge of incorporating the information into the domain and learner
models.
As it can be seen in Fig. 4, the first thing the user must do is selecting the text file
(Fig. 4, a) that contains the information to be loaded into COBLE.
When the file has been selected, the user is provided with a preview of the infor-
mation of the file (Fig. 4, b) to assure that the information and its structure are correct.

IMPORTACIÓN DE DATOS
a
e

d
c

Fig. 4. Learner model data loading

The file can have more information than the one the lecturer wants to load into the
system. For example, when exporting from the Moodle’s gradebook, the whole
Learning Analytics Summer Institute Spain – LASI Spain 2018

information about the student is incorporated, including the system’s internal identifi-
cation for the student or the institution’s identifier. Thus, the lecturer has to select from
all the fields that appear in the file, which ones should be incorporated into COBLE
(Fig. 4, c). The user must also identify which is the field that contains the student’s e-
mail account as that will be the identifier in COBLE (Fig. 4, d).
Sometimes lecturers might have defined completely in COBLE the tasks whose
marks are going to be loaded, but sometimes it will not be the case. In this last case,
COBLE warns the user about the tasks that are not defined in the domain model and
allows the updating of the domain model defining the tasks and their relationships in
the system (Fig. 5).

Fig. 5. Domain data incorporation

4.3 Model Updating
The information related to the assessment of the different tasks, is stored in the histor-
ical data part of the learner model. Taking into account this information, COBLE must
update the knowledge and competence mastery levels of the learner model.
Literature reports different means to update knowledge and competence models [12,
13] In particular, Bayesian Knowledge Tracing (BKT), a type of Hidden Markov
Model, has successfully been used for updating the knowledge model [14, 15].
BKT represents the changing knowledge state during skill acquisition based on the
results of the student opportunities to apply the skill. In this model, the student
knowledge is represented by a set of variables, one per topic, that describe the proba-
bility that the topic is either mastered by the student or not. This model considers the
student’s performance on an activity related to a topic to update the probability of the
topic being mastered. The model does not take into account the possibility of the student
forgetting a topic, but it considers that the student might succeed by chance or fail even
when mastering the topic. The model entails four parameters:

• p(Lo) or p-init, which represents the probability of the student knowing the topic
beforehand.
• p(T) or p-transit, which represents the probability of the student having learnt the
topic.
Learning Analytics Summer Institute Spain – LASI Spain 2018

• p(S) or p-slip, the probability the student makes a mistake when working on the
topic.
• p(G) or p-guess, the probability that the student has a lucky guess on an activity
related to an unknown topic.
These parameters allow updating the students’ mastery for each topic. The initial prob-
ability of a student u mastering topic k is set to the p-init parameter for that topic (1).
𝑝(𝐿$ )'& = 𝑝(𝐿) )' (1)
After that, computing the conditional probability depends on whether or not the student
succeeded on the task related to that topic.
If the student correctly solved the task, the conditional probability is computed using
(2). In this equation, the probability of knowing the topic before the assignment and not
slipping is divided by the probability of correctly solving the activity (which also con-
siders succeeding by chance).
1(2345 )7 7
6 8$+1(9) :
𝑝(𝐿*+$ |𝑜𝑏𝑠 = 𝑜𝑘)'& = 1(2 7 7 7 7 (2)
345 )6 8$+1(9) :;($+1(2345 )6 )81(<) :

If the student does not pass the task, the conditional probability considers the probabil-
ity of knowing the topic before the assignment and slipping divided by the probability
of failing (3).
1(2345 )7 7
6 81(9) :
𝑝(𝐿*+$ |𝑜𝑏𝑠 = 𝑘𝑜)'& = 1(2 7 7 7 7 (3)
345 )6 81(9) :;($+1(2345 )6 )8$+1(<) :

The conditional probability obtained using (2) or (3), depending on whether the student
succeeded or not, is used in (4) to update the probability of the student knowledge on
the topic. Equation (4) considers the probability of knowing the topic before the activity
plus the probability of not knowing it and having learnt the topic.

𝑝(𝐿* )'& = 𝑝(𝐿* − 1|𝑜𝑏𝑠)'& + (1 − 𝑝(𝐿* |𝑜𝑏𝑠)'& )𝑝(𝑇) (4)
The described model is used in COBLE to update the students’ knowledge level and
the competence model in the Learner Model.

5 Visualizations

The main objective of COBLE is to support the main user types in competence-based
learning environments. For that, it currently incorporates a Visualization module that
provides different visualizations for students and lecturers. The visualizations provided
by this module are intended to promote students’ self-reflection and to help lecturers to
adapt their course plan.
COBLE provides diverse simple visualization of the knowledge level of the students,
but it also provides richer visualizations of the competence acquisition level. This in-
formation is always provided in context, providing what is called an Open Social
Learning Analytics Summer Institute Spain – LASI Spain 2018

Student Model [16]. This social dimension has proven to increase student engagement
and learning effectiveness [17].
Next, some of the visualizations COBLE provides are described.
For example, lecturers can consult the mastery level of a particular student on each
competence contextualized as a bullet in a boxplot chart that represents the mastery
level of the whole class.
Lecturers can also analyze the acquisition level of different competences for a group
of students. For example, the system generates a violin-plot chart where the distribution
of the group’s students having each level of mastery for each of the competences is
shown (see Fig. 6).

COMPETENCES ACQUISITION

Competences’ acquisition level for the group

Fig. 6. Group mastery level for each competence

Lecturers have also the possibility to see the information related to a particular compe-
tence, observing the evolution of the mastery level on that competence. For example,
Fig. 7 shows the evolution of a student in a competence and compares it to the class or
group the student belongs to.
Learning Analytics Summer Institute Spain – LASI Spain 2018

COMPETENCE ACQUISITION

Competence Competence 1 From Week 1 To Week 30

Fig. 7. Mastery level evolution on a competence

COBLE also provides some customizable visualizations for lecturers, which can help
them detecting interesting information in order to organize their lectures. For example,
a lecturer interested in determining which competence to reinforce in the following lec-
ture, could be interested in detecting competences with lower mastery levels than ex-
pected. Fig. 8 shows the visualization that summarizes the percentage of students that
have mastery levels above the threshold established by the lecturer (0.8 in this case) for
each competence.

COMPETENCES ACQUISITION

Show competences’ acquisition percentages according to the threshold: 0.8

Competence 3 9% 91%

Competence 2 96% 4%

Competence 1 62% 38%

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
YES NO

Fig. 8. Customizable interface to obtain group information
Learning Analytics Summer Institute Spain – LASI Spain 2018

6 Conclusions and Future Work

This paper has presented the proposal of the system COBLE, which has been designed
to support competence-based learning. The system capabilities have been improved
providing a module that allows integrating domain and student information coming
from different systems.
The import characteristics have been evaluated, importing student interactions with
Moodle for several courses, and also from files created by lecturers to store information
related to face to face interactions with the students.
Future work will be devoted to two different aspects. On the one hand, it will be
analyzed the possibility of importing the whole domain data from files extracted from
other systems. On the other hand, we plan to design and implement a new module that
will use recommendation techniques in order to provide customizable automatic alerts
and recommendations both for students and lecturers.

Acknowledgments

This work is supported by the Basque Government (IT980-16), the UPV/EHU (EHUA
16/22) and the Office of the Vice-Chancellor for Innovation, Social Engagement and
Cultural action of the UPV/EHU through the SAE-HELAZ (HBT-Adituak 2018-19/6).

References

1. Guàrdia, L., Crisp, G., Alsina, I.: Trends and Challenges of E-Assessment to Enhance Student
Learning in Higher Education. Innov. Pract. High. Educ. Assess. Meas. 36–56 (2017).
2. Kay, J., Bull, S.: New Opportunities with Open Learner Models and Visual Learning Analyt-
ics. Presented at the Artificial Intelligence in Education (AIED 2015) , Cham (2015).
3. Klerkx, J., Verbert, K., Duval, E.: Enhancing Learning with Visualization Techniques. In:
Spector, J.M., Merrill, M.D., Elen, J., and Bishop, M.J. (eds.) Handbook of Research on Ed-
ucational Communications and Technology. pp. 791–807. Springer New York (2014).
4. Bull, S., D. Johnson, M., Masci, D., Biel, C.: Integrating and Visualising Diagnostic Infor-
mation for the Benefit of Learning. In: Measuring and Visualizing Learning in the Infor-
mation-Rich Classroom. Taylor and Francis (2015).
5. Pardo, A., Dawson, S.: Learning Analytics: How can Data be used to Improve Learning Prac-
tice. In: Reimann, P., Bull, S., Kickmeier-Rust, M., Vatrapu, R., and Wasson, B. (eds.) Meas-
uring and visualizing learning in the information-rich classroom. pp. 41–55. Routledge
(2016).
6. Vaughan, N.: Student Engagement and Blended Learning: Making the Assessment Connec-
tion. Educ. Sci. 4, 247–264 (2014).
7. Garrison, D.R., Vaughan, N.D.: Blended Learning in Higher Education: Framework, Princi-
ples, and Guidelines. John Wiley and Sons, San Francisco (2007).
Learning Analytics Summer Institute Spain – LASI Spain 2018

8. Henri, M., Johnson, M.D., Nepal, B.: A Review of Competency-Based Learning: Tools, As-
sessments, and Recommendations: A Review of Competency-Based Learning. J. Eng. Educ.
106, 607–638 (2017).
9. Miller, H.G., Mork, P.: From Data to Decisions: A Value Chain for Big Data. IT Prof. 15,
57–59 (2013).
10. Villamañe, M.: Análisis y mejora de los marcos actuales de desarrollo y evaluación de los
Trabajos Fin de Grado mediante el uso de las TIC, (2017).
11. Mohammed Khalidi Idrissi, Meriem Hnida, Samir Bennani: Competency-Based Assessment:
From Conceptual Model to Operational Tool. In: Elena Cano and Georgeta Ion (eds.) Inno-
vative Practices for Higher Education Assessment and Measurement (2017).
12. Falmagne, J.-C., Albert, D., Doble, C., Eppstein, D., Hu, X. eds: Knowledge Spaces. Springer
Berlin Heidelberg, Berlin, Heidelberg (2013).
13. Woolf, B.P.: Building Intelligent Interactive Tutors: Student-centered Strategies for Revolu-
tioniizing E-learning. Morgan Kauffmann Publishers, Inc. (2008).
14. Corbett, A.T., Anderson, J.R.: Knowledge tracing: Modeling the acquisition of procedural
knowledge. User Model. User-Adapt. Interact. 4, 253–278 (1995).
15. Yudelson, M.V., Koedinger, K.R., Gordon, G.J.: Individualized Bayesian Knowledge Trac-
ing Models. In: Artificial Intelligence in Education. pp. 171–180. Springer, Berlin, Heidel-
berg (2013).
16. Brusilovsky, P., Somyürek, S., Guerra, J., Hosseini, R., Zadorozhny, V.: The Value of Social:
Comparing Open Student Modeling and Open Social Student Modeling. Presented at the In-
ternational Conference on User Modeling, Adaptation, and Personalization (UMAP 2015) ,
Dublin June 29 (2015).
17. Brusilovsky, P., Somyürek, S., Guerra, J., Hosseini, R., Zadorozhny, V., Durlach, P.J.: Open
Social Student Modeling for Personalized Learning. IEEE Trans. Emerg. Top. Comput. 4,
450–461 (2016).