<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta />
    <article-meta>
      <title-group>
        <article-title>Engaging Students with Extra Tasks: Motivational Design and Student-Generated Improvements</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Ilya Musabirov</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Joseph Jay Williams</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          <institution>University - University of Toronto</institution>
          ,
          <addr-line>Toronto, Ontario</addr-line>
          ,
          <country country="CA">Canada</country>
        </aff>
      </contrib-group>
      <abstract>
        <p>Online courses have become increasingly popular in recent years, providing students with the flexibility to learn at their own pace and from any location. Optional activities, such as practice problems or additional readings, are commonly included in these courses to provide students with additional opportunities for learning. However, previous research has shown that students often elect to skip these optional activities, despite their potential benefits for improving learning outcomes. Providing students with motivational text prompts as they solve optional practice problems as the potential to engage students and encourage them to continue to pursue more activities. In this work, we leveraged messages that rely on motivational theories, such as explaining how solving additional problems will benefit student learning. We also asked students to provide their own motivational reasons to pursue additional and optional problems, which can then be used as motivational prompts for future courses.</p>
      </abstract>
      <kwd-group>
        <kwd>1 Learnersourcing</kwd>
        <kwd>Motivation</kwd>
        <kwd>Nudges</kwd>
      </kwd-group>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>1. Introduction</title>
      <p>Engaging students with optional additional activities is one of the crucial success factors in choice-rich
learning environments. Thus, if we assume these activities have a potentially positive effect on learning,
engaging students with them becomes a significant underlying factor for them to have an effect. If a
student does not choose to perform the activity, they are missing valuable opportunities. While the
benefit of having students participate in activities is supported, they can not always be made mandatory.
For instance, one might include optional activities in their online course to allow students a choice in
what material they use to practice. Additionally, optional activities allow students who might have more
time or motivation to pursue additional practice opportunities.</p>
      <p>Interventions based on motivational theories [9] can come to the rescue, encouraging students to
pursue these optional activities. We propose applying the ideas behind educational psychological
interventions to create just-in-time interventions that are triggered straight at the decision points with a
focus on stimulating students’ engagement with extra activities.</p>
      <p>In this paper we briefly describe our approach to design of extra task nudges, reflect on
learnersourced student improvement ideas for them, outline open questions for students’ engagement
with learnersourcing, and discuss implementation efforts.</p>
    </sec>
    <sec id="sec-2">
      <title>2. Designing Extra Tasks Nudges</title>
      <p>
        To stimulate student engagement with extra activities, we created a factorial mechanism generating
nudges in text prompt format (Fig. 1), emphasizing the potential benefits of engaging with the activity
from the standpoint of different motivational theories and behavioral science techniques [
        <xref ref-type="bibr" rid="ref4 ref5">4, 5, 10</xref>
        ].
These types of prompts encourage students to take the extra time and participate in these different
optional activities. Leveraging the motivational theory, they can include the benefit of students
participating in these activities, citing how it can help their learning.
3. Students’ Ideas of Better Messages
We followed up with an end-of-term survey, asking students to reflect on nudges’ utility and propose
better messages to get them to attempt another problem. Fig. 2 shows keywords from top frequency
skip-grams (two-word sequences with 0–3 words in between) of student message ideas. Using that as a
high-level map to navigate the text corpus by looking at keywords in context [
        <xref ref-type="bibr" rid="ref2">2</xref>
        ], we can outline the
most frequent student ideas
      </p>
      <p>These ideas are connecting motivation to:
• providing extra credit or bonus marks to stimulate optional activities
• the benefits of better understanding course concepts and content
• the benefits provided by regular practice
• a chance to be ready for similar problems in other assignments and exams</p>
      <p>While the last three ideas rely on the same motivational factors as the original nudges design, students
helped generate new ways of representing them, referring to enactment mechanisms and outcomes they
see as important. In the next iteration of our study, we plan to test these ideas on the next student cohort
together with the effect of framing them as student advice.</p>
    </sec>
    <sec id="sec-3">
      <title>4. Relevant Questions for Learnersourcing</title>
      <p>During the research phase, motivational or instructional intervention activities are often tested without
direct competitors for students’ attention in the intervention group. However, during production
deployments in online and blended environments, these activities, e.g. learnersourcing activities, extra
problems, and reflection exercises, often compete with what the instructors feel is "the core content,"
maladaptive behaviors [8], and other direct and indirect competitors.</p>
      <p>This underlines the importance of finding ways to engage and sustain students’ engagement with
learnersourcing and other optional activities. Many interventions based on traditional theories of student
motivation [9] were tested in the traditional intervention format, where part of the potential effect relies
on students’ monopolized focus on the intervention during a relatively long period.</p>
      <p>
        Meanwhile, decisions to engage in extra activities are often situationally motivated [
        <xref ref-type="bibr" rid="ref5">5</xref>
        ] and do not
provide an opportunity to use full-scale interventions. The format we propose borrows from rich
behavioral science research on nudges and allows us to create just-in-time micro-interventions
integrated into the fabric of the everyday learning process.
      </p>
      <p>
        Thus, we follow the recent turn to situational motivation theories [
        <xref ref-type="bibr" rid="ref5">5,9</xref>
        ], but inherit the need to answer
the open questions both from this turn and from research on nudges, namely:
• What specific constructs we nudge for [
        <xref ref-type="bibr" rid="ref5">5</xref>
        ]?
• How to take into account different susceptibility of students to nudges [
        <xref ref-type="bibr" rid="ref3">3</xref>
        ]?
• Do we need to and how to provide personalized ways of students’ engagement and adaptivity
[
        <xref ref-type="bibr" rid="ref1 ref7">1, 7</xref>
        ]?
• What are the limits, cumulative and long-term effects [
        <xref ref-type="bibr" rid="ref5">5</xref>
        ] of repeating micro interventions or
combinations of traditional interventions and supporting nudges?
      </p>
    </sec>
    <sec id="sec-4">
      <title>5. Towards Implementation Efforts</title>
      <p>The motivational text prompts we served to students in this work were in-the-moment, as they went
about solving problems in their online courseware. Many digital courseware platforms can make use of
these messages, such as adding in a purely text-based “questions” to students as they complete practice
problems or even a quiz, encouraging them to keep going and putting forth their best effort. Outside of
these in-the-moment interventions, motivational prompts can be leveraged before or after the problem
solving process.</p>
      <p>For instance, before student embark on a set of problems, such as a summative quiz at the end of a
unit, they could be prompted to read such a motivational message before beginning. This extra
motivation before engaging with the activity could increase their engagement with it, potentially leading
to better outcomes. When the activity is optional, such as a learnersourcing one that has students
generated a multiple-choice question, motivational prompts before the activity can potentially increase
student participation. These tasks are typically quite daunting and potentially time consuming for the
students, thus a motivational prompt prior to engaging with it could give the student the extra push
needed to elect to participate in it.</p>
      <p>Further tailoring of these motivational messages to the students can also increase the potential effect
they have in encouraging the students to participate in optional activities [8]. Using an adaptive
algorithm, such as multi-armed bandits, could enable an online course to begin with multiple
motivational prompts that the course authors believe will be effective. Then as students go about
working on the course and receiving these different prompts, the algorithm could identify which
motivational messages seem to work the best for certain students, based on factors such as their prior
knowledge level measured by previous assessments in the course. Depending on these student factors,
a different motivational message might then be shown to a student to encourage their participation with
optional activities.</p>
    </sec>
    <sec id="sec-5">
      <title>6. Conclusion</title>
      <p>In conclusion, the use of text-prompts to engage students in optional activities that rely on different
motivational theories and behavioral science techniques can be effective at encouraging students
participation. As students work through optional problems in an online course, a simple motivational
prompt that encourages them to continue and states how this practice benefits their learning can lead to
a student completing more of these optional activities. In regards to learnersourcing, students can be
asked to contribute their own motivational reasons for participating in optional activities, which can
then be leveraged for future prompts. We found that when we polled students for these ideas, many of
their contributions related back to our original message intent, which relies on highlighting the learning
benefit for solving additional practice problems. Future work can look to leverage adaptive algorithms
that allow for more variety in the motivational prompts that can be served to students and potentially
identify which motivational messages might work best for certain students.</p>
    </sec>
    <sec id="sec-6">
      <title>7. Acknowledgements</title>
      <p>We want to use this opportunity to say thanks to Andrew Petersen, Anna Rafferty, Michael Liut, Mohi
Reza and all people from the Intelligent Adaptive Interventions group for their support and relevant
advice. Ilya also thanks Denis Bulygin and Vsevolod Suschevskiy.</p>
    </sec>
    <sec id="sec-7">
      <title>8. References</title>
      <p>[8] Jaemarie Solyst, Trisha Thakur, Madhurima Dutta, Yuya Asano, Andrew Petersen, and Joseph Jay
Williams. 2021. Procrastination and Gaming in an Online Homework System of an Inverted CS1.
In Proceedings of the 52nd ACM Technical Symposium on Computer Science Education (SIGCSE
’21). Association for Computing Machinery, New York, NY, USA, 789–795.
https://doi.org/10.1145/3408877.3432440
[9] Allan Wigfield, Katherine Muenks, and Jacquelynne S. Eccles. 2021. Achievement Motivation:
What We Know and Where We Are Going. Annual Review of Developmental Psychology 3, 1
(Dec. 2021), 87–111. https://doi.org/10.1146/annurevdevpsych-050720-103500
[10] David S. Yeager and Carol S. Dweck. 2020. What can be learned from growth mindset
controversies? American psychologist 75, 9 (2020), 1269. Publisher: American Psychological
Association. T</p>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          [1]
          <string-name>
            <given-names>Yuya</given-names>
            <surname>Asano</surname>
          </string-name>
          , Madhurima Dutta, Trisha Thakur, Jaemarie Solyst, Stephanie Cristea, Helena Jovic,
          <string-name>
            <given-names>Andrew</given-names>
            <surname>Petersen</surname>
          </string-name>
          , and Joseph Jay Williams.
          <year>2021</year>
          .
          <article-title>Exploring Additional Personalized Support While Attempting Exercise Problems in Online Learning Platforms</article-title>
          .
          <source>In Proceedings of the Eighth ACM Conference on Learning @ Scale (L@S '21)</source>
          .
          <article-title>Association for Computing Machinery</article-title>
          , New York, NY, USA,
          <fpage>235</fpage>
          -
          <lpage>238</lpage>
          . https://doi.org/10.1145/3430895.3460145P.
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          [2]
          <string-name>
            <given-names>Kenneth</given-names>
            <surname>Benoit</surname>
          </string-name>
          , Kohei Watanabe,
          <string-name>
            <given-names>Haiyan</given-names>
            <surname>Wang</surname>
          </string-name>
          , Paul Nulty, Adam Obeng, Stefan Müller, and
          <string-name>
            <given-names>Akitaka</given-names>
            <surname>Matsuo</surname>
          </string-name>
          .
          <year>2018</year>
          .
          <article-title>quanteda: An R package for the quantitative analysis of textual data</article-title>
          .
          <source>Journal of Open Source Software</source>
          <volume>3</volume>
          ,
          <issue>30</issue>
          (Oct.
          <year>2018</year>
          ),
          <volume>774</volume>
          . https://doi.org/10.21105/joss.00774
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          [3]
          <string-name>
            <surname>Denise de Ridder</surname>
          </string-name>
          , Floor Kroese, and Laurens van Gestel.
          <year>2022</year>
          .
          <article-title>Nudgeability: Mapping Conditions of Susceptibility to Nudge Influence</article-title>
          .
          <source>Perspectives on Psychological Science 17, 2 (March</source>
          <year>2022</year>
          ),
          <fpage>346</fpage>
          -
          <lpage>359</lpage>
          . https://doi.org/10.1177/1745691621995183
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          [4]
          <string-name>
            <given-names>Angela</given-names>
            <surname>Lee</surname>
          </string-name>
          <string-name>
            <surname>Duckworth</surname>
          </string-name>
          ,
          <string-name>
            <given-names>Heidi</given-names>
            <surname>Grant</surname>
          </string-name>
          , Benjamin Loew, Gabriele Oettingen, and
          <string-name>
            <surname>Peter M. Gollwitzer</surname>
          </string-name>
          .
          <year>2011</year>
          .
          <article-title>Self-regulation strategies improve self-discipline in adolescents: Benefits of mental contrasting and implementation intentions</article-title>
          .
          <source>Educational Psychology</source>
          <volume>31</volume>
          ,
          <issue>1</issue>
          (
          <year>2011</year>
          ),
          <fpage>17</fpage>
          -
          <lpage>26</lpage>
          . Publisher: Taylor &amp; Francis.
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          [5]
          <string-name>
            <surname>Jacquelynne</surname>
            <given-names>S.</given-names>
          </string-name>
          <string-name>
            <surname>Eccles</surname>
            and
            <given-names>Allan</given-names>
          </string-name>
          <string-name>
            <surname>Wigfield</surname>
          </string-name>
          .
          <year>2020</year>
          .
          <article-title>From expectancy-value theory to situated expectancy-value theory: A developmental, social cognitive, and sociocultural perspective on motivation</article-title>
          .
          <source>Contemporary Educational Psychology</source>
          <volume>61</volume>
          (
          <year>April 2020</year>
          ),
          <volume>101859</volume>
          . https://doi.org/10.1016/j.cedpsych.
          <year>2020</year>
          .101859
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          [6]
          <string-name>
            <given-names>Samuli</given-names>
            <surname>Reijula</surname>
          </string-name>
          and
          <string-name>
            <given-names>Ralph</given-names>
            <surname>Hertwig</surname>
          </string-name>
          .
          <year>2022</year>
          .
          <article-title>Self-nudging and the citizen choice architect</article-title>
          .
          <source>Behavioural Public Policy</source>
          <volume>6</volume>
          ,
          <issue>1</issue>
          (Jan.
          <year>2022</year>
          ),
          <fpage>119</fpage>
          -
          <lpage>149</lpage>
          . https://doi.org/10. 1017/bpp.
          <year>2020</year>
          .5
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          [7]
          <string-name>
            <given-names>Mohi</given-names>
            <surname>Reza</surname>
          </string-name>
          , Juho Kim, Ananya Bhattacharjee,
          <string-name>
            <given-names>Anna N.</given-names>
            <surname>Rafferty</surname>
          </string-name>
          , and Joseph Jay Williams.
          <year>2021</year>
          .
          <article-title>The MOOClet Framework: Unifying Experimentation, Dynamic Improvement, and Personalization in Online Courses</article-title>
          .
          <source>In Proceedings of the Eighth ACM Conference on Learning@ Scale</source>
          .
          <fpage>15</fpage>
          -
          <lpage>26</lpage>
          .
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>