<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta />
    <article-meta>
      <title-group>
        <article-title>Personalising e-Learning Systems: Lessons learned from a vocational education case study</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Lie Ming Tang</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Kalina Yacef</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          <institution>Computer Human Adapted Interation Reserach Group School of Information Technology University Of Sydney</institution>
        </aff>
      </contrib-group>
      <abstract>
        <p>Vocational education refers to the training of speci c skills or trades. It is often done part time or in personal time over a lengthy period (months to years). As such, it requires persistence, self motivation and self regulatory skills including goal setting, planning and time management. A growing body of evidence suggests that these self-regulating skills are a key determinant in learning performance and can be improved with support. We report in this paper our experience with a leading vocational education provider in Australia who is transitioning from classroom-based training to a pilot e-learning system. We present the key lessons learned and the prototype interface we designed to improve user self-regulation in planning and time management.</p>
      </abstract>
      <kwd-group>
        <kwd>Vocational Education</kwd>
        <kwd>Personalization</kwd>
        <kwd>Self-Regulated Learning</kwd>
        <kwd>e-learning</kwd>
      </kwd-group>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>-</title>
      <p>
        Vocational education, which refers to the training of speci c skills or trades,
differs from academic education as the focus lies in skills and knowledge for speci c
industries or job roles and often requires proof of practical competency to
complete. However, similarly to academic learning, self-regulated learning [
        <xref ref-type="bibr" rid="ref11">11</xref>
        ] and
the ability to direct one's own learning is seen as an essential part of success in
vocational education [
        <xref ref-type="bibr" rid="ref5 ref9">5, 9</xref>
        ]. Important self-regulation skills include setting
appropriate proximal goals [
        <xref ref-type="bibr" rid="ref8">8</xref>
        ], resource management (e.g., allocation of resource, time
management, managing ones learning environment), self-monitoring and
adjusting behaviour based on performance. There is a growing body of evidence that
suggests personalised and computer aided support for self-regulated learning
improves student engagement and performance in e-learning systems [
        <xref ref-type="bibr" rid="ref2">2</xref>
        ]. However,
such studies has generally focused on academic learning. Self-regulation skills
are very relevant in vocational education and help in making students aware of
their own role in education and developing their learning are key components
[
        <xref ref-type="bibr" rid="ref5">5</xref>
        ].
      </p>
      <p>It is important to understand the key challenges for e-learning systems aimed
at vocational education and how their personalisation and sca olding features
can support self-regulation. We performed a study with an Australian leading
vocational education provider who transitioning from a classroom focused
training model to an online and self-managed training model using a pilot e-learning
system. We analysed student usage patterns over a 6 month period to
understand how this e-learning system is used and interviewed trainers and trainees to
get qualitative feedback on how students managed their learning in the system.</p>
      <p>In this paper, we present the key lessons learned from our study and a
prototype with user interface designed to demonstrate the key features for future
e-learning systems to personalise and support for self-regulation of learning.
2</p>
    </sec>
    <sec id="sec-2">
      <title>Vocational Education &amp; Training</title>
      <p>
        Vocational education and training represents a critical sector of education where
skills for a particular industry, trade and career are created. Over 11 percent of
the Australian population between 15 and 64 undertake vocational education
and training and the age spread is wide [
        <xref ref-type="bibr" rid="ref7">7</xref>
        ]. Moreover, the percentage of adults
with professional education as the highest quali cation is also very high
(between 5 and 34 percent [
        <xref ref-type="bibr" rid="ref1">1</xref>
        ]). Learning topics are broad and include industry
accreditation and certi cation for elds in health care, accounting, engineering,
law and information technology and many others. Vocational education is
usually competence based: where training and assessments are assessed on whether
a participant is skilled and competent in a particular job or trade rather than
measuring theoretical knowledge alone. This may involve collecting evidence and
reports over many weeks or months as part of the assessment. For example, ight
training requires that trainee pilots log the number of hours of actual and
simulated ying. Similarly to academic learning, vocational education can span a
lengthy period of time from months to years. Both require students to maintain
self-motivation and persistence. However, over 88 percent of vocational learners
are part time in Australia. This suggests a higher competing presence of other
priorities for most vocational learners compared to academic students, with the
unavoidable challenges in attention focusing, organisation and time management
skills.
3
      </p>
    </sec>
    <sec id="sec-3">
      <title>Case Study: e-learning system</title>
      <p>We worked with a leading vocational education provider in Australia who has
operated a nationally accredited certi cation program for their employees for
over 10 years. Recently, they started transitioning from a classroom training
model to a pilot e-learning training program requiring self directed learning and
assessments. A key motivating factor is motivate and encourage students to
regulate their own learning and to reduce the contacts needed with trainers.</p>
      <p>The pilot e-learning system is used for accessing online learning materials,
perform assessments and uploading evidence for practical experience and
competence. In the pilot program, participants rst read or view online learning
material, gain practical experience and reinforce their learning and then physically
attend a classroom learning workshop. They then complete online assessments
via the e-learning system. While the online learning material is not mandatory,
they contain the knowledge needed to pass the online assessments. The workshop
o ers an opportunity for trainers to reinforce the online learning material,
provide discussion and simulated practical experience to prepare learners for their
formal online assessment.</p>
      <p>We performed a 6 month study to analyse student usage patterns based on
data from the e-learning system for over 600 trainees. We also interviewed 3
participants and 3 trainers to gain an qualitative view of the challenges with
self-regulation and performance.
3.1</p>
      <sec id="sec-3-1">
        <title>Planning &amp; Time Management</title>
        <p>Planning and time management is a signi cant challenge for many students.
A commonly cited problem by students, including the trainees in our study,
is the lack of time. However, interviews with trainers indicate that this is not
the case because trainees are allocated time or are getting paid for their time
spent on learning. Rather, the key issues cited by trainers are attention focusing,
planning, and time management rather than time constraints. In many cases,
work priorities con ict with the planned learning times and students do not
adjust their planning or they forget.</p>
        <p>While the majority of trainees complete their learning on their own, trainers
needed to organise separate workshops speci cally for certain groups of trainees
to concentrate and complete their online learning away from their workplace
which can be busy and not conducive for learning. Trainers found that the
trainees' lack of self-regulation skills in time management, planning,
prioritisation, and remembering to perform tasks were key challenges. The di culty
level of the learning material was rarely an issue in this program.
3.2</p>
      </sec>
      <sec id="sec-3-2">
        <title>Environment management</title>
        <p>
          According to the social cognitive theory, the social and work environment are
a key determinant of behaviour [
          <xref ref-type="bibr" rid="ref3">3</xref>
          ]. As part of the program, most trainees are
paired with a coach who helps and supports their learning progress. Feedback
from both trainers and trainees were very positive in terms of the support
provided. Trainer feedback suggests that when coaching support is not very strong,
the trainee is less motivated and requires more trainer engagement. Trainees
also highlighted that pairing with a study partner provided mutual support and
improved their motivation and persistence.
        </p>
        <p>Feedback from both trainer and trainee suggests that those who managed
their social and learning environment well had little trouble completing the
course. For instance, one successful trainee, who managed his environment by
performing his study during his day o , had a study partner in the program.
He also ensured that he completed the planned task on schedule through either
performing them on time or adjusting his plan. This trainee was able to complete
the learning tasks well ahead of schedule.</p>
      </sec>
    </sec>
    <sec id="sec-4">
      <title>Our Approach</title>
      <p>Our hypothesis is that trainees who are struggling in the vocational program can
be supported through sca olding or computer aided support to improve their
planning and time management. We built a prototype system, augmenting the
existing e-learning platform, with user interface elements designed to promote
self regulation. The prototype system lets trainees set time schedules for their
learning objectives, monitor their performance and adjust them when necessary.
It is also available via mobile application and sends them reminders of upcoming
tasks. The prototype system also allows trainers to monitor the performance of
their trainees and identify those who need personalised attention.
4.1</p>
      <sec id="sec-4-1">
        <title>Planning &amp; Time Management</title>
        <p>
          To address the feedback above, we made planning and time management a core
skill for the prototype system to support and sca old. There have been very few
user interfaces designed to sca old time management and planning. A previous
approaches used Zimmerman's cyclic model of self-regulated learning as the basis
to detect and model the learner states [
          <xref ref-type="bibr" rid="ref10">10</xref>
          ]. This approach used a calendar-like
interface where users de ne their learning schedules with recommendations and
help support. However, we found that the trainees of our vocational program can
bene t from an initial engagement to setup a simple schedule and keeping track
of their learning task and maintaining their plans. We targeted the more general
skill of scheduling of tasks and following through with that plan through
monitoring and reminders. In our interface, when users rst login to the application,
they are rst presented with a wizard where they are prompted to set a plan or
schedule for when they expect to complete a task, as shown in gure 1. The user
can also add this task to their Google calendar and activate an email or SMS
noti cation when their planned task is due. The wizard does this for the rst
task only and users can access the wizard later if needed. After the wizard exits,
they can set schedule for tasks via the plan button (see gure 2). The objective
of this wizard is to sca old users taking control and managing their plan and
time.
4.2
        </p>
      </sec>
      <sec id="sec-4-2">
        <title>Self Monitoring</title>
        <p>
          As part of the interface, the user is allowed to monitor his/her progress in the
program. They are also reminded about their upcoming planned tasks, what they
have completed and what they still have pending, see gure 3. This allows them
to monitor their progress, re ect on their planning and scheduling. In addition,
users can monitor their progress compared to their peers for each of the learning
objectives in their program, see gure 4. Studies have shown that behaviour can
be modi ed through comparing one's own performance against peers [
          <xref ref-type="bibr" rid="ref6">6</xref>
          ], [
          <xref ref-type="bibr" rid="ref4">4</xref>
          ].
Fig. 1. Wizard to help the user get started with planning their learning schedule. Note:
a schedule is referred to as a "goal" in the interface
We have also provided trainers with the ability to monitor the progress of each
of their participants and the status of each e-learning modules i.e., not started,
started or completed. This allow trainers to see which students are lagging behind
their peers and require personalised attention. See gure 5. Trainers can also see
which students have not accessed their e-learning materials so they can send
them reminders to maximise the workshop outcomes.
        </p>
      </sec>
    </sec>
    <sec id="sec-5">
      <title>Conclusion &amp; Future Work</title>
      <p>We found a key challenge for vocational education students is time and
environment management. We designed a prototype system to support students in
becoming better planners and time managers. We believe such goal setting and
time management interface designs can also be integrated into other e-learning
systems where the learning pro le is similar to vocational education (e.g., self
redirected professional learning, part time academic studies). Peer and trainer
engagement and support appears to be important to trainees and trainers and
future systems should investigate how sca olding can be applied. This can
potentially reduce withdrawals, increase engagement and motivation for the trainees.
6</p>
    </sec>
    <sec id="sec-6">
      <title>Acknowledgement</title>
      <p>This work was funded by Smart Services Cooperative Research Centre.</p>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          1.
          <string-name>
            <surname>OECD</surname>
          </string-name>
          <year>2014</year>
          .
          <article-title>Skills beyond School: Synthesis Report</article-title>
          ,
          <source>OECD Reviews of Vocational Education and Trainning</source>
          .
          <year>2014</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          2.
          <string-name>
            <given-names>Roger</given-names>
            <surname>Azevedo and Allyson F. Hadwin</surname>
          </string-name>
          .
          <article-title>Sca olding Self-regulated Learning and Metacognition Implications for the Design of Computer-based Sca olds</article-title>
          .
          <source>Instructional Science</source>
          ,
          <volume>33</volume>
          (
          <issue>5-6</issue>
          ):
          <volume>367</volume>
          {
          <fpage>379</fpage>
          ,
          <string-name>
            <surname>November</surname>
          </string-name>
          <year>2005</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          3.
          <string-name>
            <given-names>Albert</given-names>
            <surname>Bandura</surname>
          </string-name>
          .
          <article-title>Human agency in social cognitive theory</article-title>
          .
          <source>The American psychologist</source>
          ,
          <volume>44</volume>
          :
          <fpage>1175</fpage>
          {
          <fpage>1184</fpage>
          ,
          <year>1989</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          4.
          <string-name>
            <given-names>Albert</given-names>
            <surname>Bandura</surname>
          </string-name>
          .
          <article-title>Social cognitive theory of self-regulation</article-title>
          ,
          <year>1991</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          5. Elly De Bruijn and
          <string-name>
            <given-names>Yvonne</given-names>
            <surname>Leeman</surname>
          </string-name>
          .
          <article-title>Authentic and self-directed learning in vocational education: Challenges to vocational educators</article-title>
          .
          <source>Teaching and Teacher Education</source>
          ,
          <volume>27</volume>
          (
          <issue>4</issue>
          ):
          <volume>694</volume>
          {
          <fpage>702</fpage>
          ,
          <year>2011</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          6.
          <string-name>
            <given-names>Sten</given-names>
            <surname>Govaerts</surname>
          </string-name>
          , Katrien Verbert, Erik Duval, and
          <string-name>
            <given-names>Abelardo</given-names>
            <surname>Pardo</surname>
          </string-name>
          .
          <article-title>The student activity meter for awareness and self-re ection</article-title>
          .
          <source>In Proceedings of the 2012 ACM annual conference extended abstracts on Human Factors in Computing Systems Extended Abstracts - CHI EA '12</source>
          ,
          <string-name>
            <surname>page</surname>
            <given-names>869</given-names>
          </string-name>
          , New York, New York, USA, May
          <year>2012</year>
          . ACM Press.
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          7.
          <string-name>
            <given-names>Kathrin</given-names>
            <surname>Hoeckel</surname>
          </string-name>
          , Simon Field,
          <string-name>
            <surname>Troy R Justesen</surname>
            , and
            <given-names>Moonhee</given-names>
          </string-name>
          <string-name>
            <surname>Kim</surname>
          </string-name>
          .
          <article-title>Learning for Jobs</article-title>
          .
          <source>OECD Reviews of Vocational Education and Training. Education And Training</source>
          , (November):
          <fpage>220</fpage>
          ,
          <year>2009</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          8.
          <string-name>
            <surname>Gary</surname>
            <given-names>P Latham</given-names>
          </string-name>
          and
          <article-title>Edwin a Locke. Self-regulation through goal setting</article-title>
          .
          <source>Organizational Behavior and Human Decision Processes</source>
          ,
          <volume>50</volume>
          (
          <issue>2</issue>
          ):
          <volume>212</volume>
          {
          <fpage>247</fpage>
          ,
          <year>December 1991</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          9.
          <string-name>
            <surname>Paul</surname>
            <given-names>R.</given-names>
          </string-name>
          <string-name>
            <surname>Pintrich</surname>
          </string-name>
          and Elisabeth V. de Groot.
          <article-title>Motivational and self-regulated learning components of classroom academic performance</article-title>
          .
          <source>Journal of Educational Psychology</source>
          ,
          <volume>82</volume>
          (
          <issue>1</issue>
          ):
          <volume>33</volume>
          {
          <fpage>40</fpage>
          ,
          <year>1990</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          10.
          <article-title>KP Shih and HC Chen. The Development and Implementation of Sca oldingBased Self-Regulated Learning System for e/m-Learning</article-title>
          .
          <source>Educational Technology &amp; Society</source>
          ,
          <volume>13</volume>
          :
          <fpage>80</fpage>
          {
          <fpage>93</fpage>
          ,
          <year>2010</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref11">
        <mixed-citation>
          11.
          <string-name>
            <given-names>Barry</given-names>
            <surname>Zimmerman</surname>
          </string-name>
          . Becoming Learner:
          <article-title>Self-Regulated Overview</article-title>
          .
          <source>Theory into Practice</source>
          ,
          <volume>41</volume>
          (
          <issue>2</issue>
          ),
          <year>2002</year>
          .
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>