=Paper=
{{Paper
|id=Vol-1388/PALE2015-paper3
|storemode=property
|title=Personalising e-Learning Systems: Lessons Learned from a Vocational Education Case Study
|pdfUrl=https://ceur-ws.org/Vol-1388/PALE2015-paper3.pdf
|volume=Vol-1388
|dblpUrl=https://dblp.org/rec/conf/um/TangY15
}}
==Personalising e-Learning Systems: Lessons Learned from a Vocational Education Case Study==
https://ceur-ws.org/Vol-1388/PALE2015-paper3.pdf
Personalising e-Learning Systems: Lessons
learned from a vocational education case study
Lie Ming Tang and Kalina Yacef
Computer Human Adapted Interation Reserach Group
School of Information Technology
University Of Sydney
http://chai.it.usyd.edu.au
ltan8012@uni.sydney.edu.au
Abstract. Vocational education refers to the training of specific skills
or trades. It is often done part time or in personal time over a lengthy
period (months to years). As such, it requires persistence, self motivation
and self regulatory skills including goal setting, planning and time man-
agement. A growing body of evidence suggests that these self-regulating
skills are a key determinant in learning performance and can be im-
proved with support. We report in this paper our experience with a
leading vocational education provider in Australia who is transitioning
from classroom-based training to a pilot e-learning system. We present
the key lessons learned and the prototype interface we designed to im-
prove user self-regulation in planning and time management.
Keywords: Vocational Education, Personalization, Self-Regulated Learn-
ing, e-learning
1 Introduction
Vocational education, which refers to the training of specific skills or trades, dif-
fers from academic education as the focus lies in skills and knowledge for specific
industries or job roles and often requires proof of practical competency to com-
plete. However, similarly to academic learning, self-regulated learning [11] and
the ability to direct one’s own learning is seen as an essential part of success in
vocational education [5, 9]. Important self-regulation skills include setting appro-
priate proximal goals [8], resource management (e.g., allocation of resource, time
management, managing ones learning environment), self-monitoring and adjust-
ing behaviour based on performance. There is a growing body of evidence that
suggests personalised and computer aided support for self-regulated learning im-
proves student engagement and performance in e-learning systems [2]. However,
such studies has generally focused on academic learning. Self-regulation skills
are very relevant in vocational education and help in making students aware of
their own role in education and developing their learning are key components
[5].
It is important to understand the key challenges for e-learning systems aimed
at vocational education and how their personalisation and scaffolding features
can support self-regulation. We performed a study with an Australian leading
vocational education provider who transitioning from a classroom focused train-
ing model to an online and self-managed training model using a pilot e-learning
system. We analysed student usage patterns over a 6 month period to under-
stand how this e-learning system is used and interviewed trainers and trainees to
get qualitative feedback on how students managed their learning in the system.
In this paper, we present the key lessons learned from our study and a pro-
totype with user interface designed to demonstrate the key features for future
e-learning systems to personalise and support for self-regulation of learning.
2 Vocational Education & Training
Vocational education and training represents a critical sector of education where
skills for a particular industry, trade and career are created. Over 11 percent of
the Australian population between 15 and 64 undertake vocational education
and training and the age spread is wide [7]. Moreover, the percentage of adults
with professional education as the highest qualification is also very high (be-
tween 5 and 34 percent [1]). Learning topics are broad and include industry
accreditation and certification for fields in health care, accounting, engineering,
law and information technology and many others. Vocational education is usu-
ally competence based: where training and assessments are assessed on whether
a participant is skilled and competent in a particular job or trade rather than
measuring theoretical knowledge alone. This may involve collecting evidence and
reports over many weeks or months as part of the assessment. For example, flight
training requires that trainee pilots log the number of hours of actual and sim-
ulated flying. Similarly to academic learning, vocational education can span a
lengthy period of time from months to years. Both require students to maintain
self-motivation and persistence. However, over 88 percent of vocational learners
are part time in Australia. This suggests a higher competing presence of other
priorities for most vocational learners compared to academic students, with the
unavoidable challenges in attention focusing, organisation and time management
skills.
3 Case Study: e-learning system
We worked with a leading vocational education provider in Australia who has
operated a nationally accredited certification program for their employees for
over 10 years. Recently, they started transitioning from a classroom training
model to a pilot e-learning training program requiring self directed learning and
assessments. A key motivating factor is motivate and encourage students to
regulate their own learning and to reduce the contacts needed with trainers.
The pilot e-learning system is used for accessing online learning materials,
perform assessments and uploading evidence for practical experience and com-
petence. In the pilot program, participants first read or view online learning ma-
terial, gain practical experience and reinforce their learning and then physically
attend a classroom learning workshop. They then complete online assessments
via the e-learning system. While the online learning material is not mandatory,
they contain the knowledge needed to pass the online assessments. The workshop
offers an opportunity for trainers to reinforce the online learning material, pro-
vide discussion and simulated practical experience to prepare learners for their
formal online assessment.
We performed a 6 month study to analyse student usage patterns based on
data from the e-learning system for over 600 trainees. We also interviewed 3
participants and 3 trainers to gain an qualitative view of the challenges with
self-regulation and performance.
3.1 Planning & Time Management
Planning and time management is a significant challenge for many students.
A commonly cited problem by students, including the trainees in our study,
is the lack of time. However, interviews with trainers indicate that this is not
the case because trainees are allocated time or are getting paid for their time
spent on learning. Rather, the key issues cited by trainers are attention focusing,
planning, and time management rather than time constraints. In many cases,
work priorities conflict with the planned learning times and students do not
adjust their planning or they forget.
While the majority of trainees complete their learning on their own, trainers
needed to organise separate workshops specifically for certain groups of trainees
to concentrate and complete their online learning away from their workplace
which can be busy and not conducive for learning. Trainers found that the
trainees’ lack of self-regulation skills in time management, planning, prioriti-
sation, and remembering to perform tasks were key challenges. The difficulty
level of the learning material was rarely an issue in this program.
3.2 Environment management
According to the social cognitive theory, the social and work environment are
a key determinant of behaviour [3]. As part of the program, most trainees are
paired with a coach who helps and supports their learning progress. Feedback
from both trainers and trainees were very positive in terms of the support pro-
vided. Trainer feedback suggests that when coaching support is not very strong,
the trainee is less motivated and requires more trainer engagement. Trainees
also highlighted that pairing with a study partner provided mutual support and
improved their motivation and persistence.
Feedback from both trainer and trainee suggests that those who managed
their social and learning environment well had little trouble completing the
course. For instance, one successful trainee, who managed his environment by
performing his study during his day off, had a study partner in the program.
He also ensured that he completed the planned task on schedule through either
performing them on time or adjusting his plan. This trainee was able to complete
the learning tasks well ahead of schedule.
4 Our Approach
Our hypothesis is that trainees who are struggling in the vocational program can
be supported through scaffolding or computer aided support to improve their
planning and time management. We built a prototype system, augmenting the
existing e-learning platform, with user interface elements designed to promote
self regulation. The prototype system lets trainees set time schedules for their
learning objectives, monitor their performance and adjust them when necessary.
It is also available via mobile application and sends them reminders of upcoming
tasks. The prototype system also allows trainers to monitor the performance of
their trainees and identify those who need personalised attention.
4.1 Planning & Time Management
To address the feedback above, we made planning and time management a core
skill for the prototype system to support and scaffold. There have been very few
user interfaces designed to scaffold time management and planning. A previous
approaches used Zimmerman’s cyclic model of self-regulated learning as the basis
to detect and model the learner states [10]. This approach used a calendar-like
interface where users define their learning schedules with recommendations and
help support. However, we found that the trainees of our vocational program can
benefit from an initial engagement to setup a simple schedule and keeping track
of their learning task and maintaining their plans. We targeted the more general
skill of scheduling of tasks and following through with that plan through moni-
toring and reminders. In our interface, when users first login to the application,
they are first presented with a wizard where they are prompted to set a plan or
schedule for when they expect to complete a task, as shown in figure 1. The user
can also add this task to their Google calendar and activate an email or SMS
notification when their planned task is due. The wizard does this for the first
task only and users can access the wizard later if needed. After the wizard exits,
they can set schedule for tasks via the plan button (see figure 2). The objective
of this wizard is to scaffold users taking control and managing their plan and
time.
4.2 Self Monitoring
As part of the interface, the user is allowed to monitor his/her progress in the
program. They are also reminded about their upcoming planned tasks, what they
have completed and what they still have pending, see figure 3. This allows them
to monitor their progress, reflect on their planning and scheduling. In addition,
users can monitor their progress compared to their peers for each of the learning
objectives in their program, see figure 4. Studies have shown that behaviour can
be modified through comparing one’s own performance against peers [6], [4].
Fig. 1. Wizard to help the user get started with planning their learning schedule. Note:
a schedule is referred to as a ”goal” in the interface
Fig. 2. Add learning schedule to Google calendar and set reminder.
Fig. 3. Learning progress of each learning objective: 1) planned and started but not
completed, 2) started but no plan, 3) completed
Fig. 4. Monitor learning progress versus other students.
4.3 Monitoring Trainees
We have also provided trainers with the ability to monitor the progress of each
of their participants and the status of each e-learning modules i.e., not started,
started or completed. This allow trainers to see which students are lagging behind
their peers and require personalised attention. See figure 5. Trainers can also see
which students have not accessed their e-learning materials so they can send
them reminders to maximise the workshop outcomes.
Fig. 5. Trainers see student progress and activity. Further details can be obtained by
clicking on the interested status bar drill down (highlighted by red circle).
5 Conclusion & Future Work
We found a key challenge for vocational education students is time and envi-
ronment management. We designed a prototype system to support students in
becoming better planners and time managers. We believe such goal setting and
time management interface designs can also be integrated into other e-learning
systems where the learning profile is similar to vocational education (e.g., self
redirected professional learning, part time academic studies). Peer and trainer
engagement and support appears to be important to trainees and trainers and
future systems should investigate how scaffolding can be applied. This can poten-
tially reduce withdrawals, increase engagement and motivation for the trainees.
6 Acknowledgement
This work was funded by Smart Services Cooperative Research Centre.
References
1. OECD 2014. Skills beyond School: Synthesis Report, OECD Reviews of Vocational
Education and Trainning. 2014.
2. Roger Azevedo and Allyson F. Hadwin. Scaffolding Self-regulated Learning and
Metacognition Implications for the Design of Computer-based Scaffolds. Instruc-
tional Science, 33(5-6):367–379, November 2005.
3. Albert Bandura. Human agency in social cognitive theory. The American psychol-
ogist, 44:1175–1184, 1989.
4. Albert Bandura. Social cognitive theory of self-regulation, 1991.
5. Elly De Bruijn and Yvonne Leeman. Authentic and self-directed learning in vo-
cational education: Challenges to vocational educators. Teaching and Teacher
Education, 27(4):694–702, 2011.
6. Sten Govaerts, Katrien Verbert, Erik Duval, and Abelardo Pardo. The student
activity meter for awareness and self-reflection. In Proceedings of the 2012 ACM
annual conference extended abstracts on Human Factors in Computing Systems
Extended Abstracts - CHI EA ’12, page 869, New York, New York, USA, May
2012. ACM Press.
7. Kathrin Hoeckel, Simon Field, Troy R Justesen, and Moonhee Kim. Learning
for Jobs. OECD Reviews of Vocational Education and Training. Education And
Training, (November):220, 2009.
8. Gary P Latham and Edwin a Locke. Self-regulation through goal setting. Or-
ganizational Behavior and Human Decision Processes, 50(2):212–247, December
1991.
9. Paul R. Pintrich and Elisabeth V. de Groot. Motivational and self-regulated learn-
ing components of classroom academic performance. Journal of Educational Psy-
chology, 82(1):33–40, 1990.
10. KP Shih and HC Chen. The Development and Implementation of Scaffolding-
Based Self-Regulated Learning System for e/m-Learning. Educational Technology
& Society, 13:80–93, 2010.
11. Barry Zimmerman. Becoming Learner: Self-Regulated Overview. Theory into
Practice, 41(2), 2002.