<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta>
      <journal-title-group>
        <journal-title>The History of Technological Anxiety and the Future
of Economic Growth: Is This Time Different?" Journal of Economic Perspectives</journal-title>
      </journal-title-group>
    </journal-meta>
    <article-meta>
      <title-group>
        <article-title>Serious Games Education for Working Life Needs - a Pilot Study</article-title>
      </title-group>
      <contrib-group>
        <aff id="aff0">
          <label>0</label>
          <institution>Andrew Sirkka, Sari Merilampi Satakunta University of Applied Sciences</institution>
          ,
          <country country="FI">Finland</country>
        </aff>
        <aff id="aff1">
          <label>1</label>
          <institution>Pauliina Tuomi Tampere University of Technology</institution>
          ,
          <country country="FI">Finland</country>
        </aff>
        <aff id="aff2">
          <label>2</label>
          <institution>Teija Lehto Tampere University of Applied Sciences</institution>
          ,
          <country country="FI">Finland</country>
        </aff>
        <aff id="aff3">
          <label>3</label>
          <institution>Teija Ravelin, Heli Tolonen Kajaani University of Applied Sciences</institution>
          ,
          <country country="FI">Finland</country>
        </aff>
      </contrib-group>
      <pub-date>
        <year>2017</year>
      </pub-date>
      <volume>29</volume>
      <issue>3</issue>
      <fpage>9</fpage>
      <lpage>10</lpage>
      <abstract>
        <p>Serious games possess huge market opportunities and offer possible solutions for various real-life challenges in different fields of businesses. Although the potential of serious games is well recognized, the educational and training possibilities in the field are scarce. Through cooperation and multiform learning solutions, the lack of education particularly around serious games could be filled in to answer the requirements of the future job descriptions. This paper presents results of student feedback (n=27) around a pilot study of a serious games course that bases on the survey of actual working life needs (company interviews) and was designed to bring closer the substance experts and the game development experts.</p>
      </abstract>
      <kwd-group>
        <kwd>Gamification</kwd>
        <kwd>Learning</kwd>
        <kwd>Serious Game</kwd>
      </kwd-group>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>1. Background</title>
      <p>
        Digitalization is one of the most promising ways to increase productivity in the society
        <xref ref-type="bibr" rid="ref2">(e.g.
Brynjolfsson &amp; McAfee 2011; Mokyr et al. 2015)</xref>
        . In addition, digitalization has been perceived as a
reformator of the economy, also substantially affecting the transformation of working life. The
educational system is an essential nexus of digitalization process since digitalization has influenced
almost all work tasks and it creates completely new jobs requiring competence to utilise computers and
other digital devices
        <xref ref-type="bibr" rid="ref13 ref6 ref9">(e.g. Frey &amp; Osborne 2013; Niemi et al. 2014; Michaels et al. 2014; Bessen 2015)</xref>
        .
Media and game industry is an example of a recently widely recognised area. Based on the report of
Finnish game education executed by University of Jyväskylä (2014), digital game industry in Finland
has grown tremendously and is becoming one of the largest sectors of cultural industries worldwide.
        <xref ref-type="bibr" rid="ref10 ref13 ref7">(Hiltunen, Latva &amp; Kaleva 2013; Mononen, Neittaanmäki &amp; Vähäkainu 2014.)</xref>
        This growth naturally
requires development in education in this field. Only in recent years, learning institutions have started
to invest in education that prepares for the future positions.
        <xref ref-type="bibr" rid="ref10 ref13 ref14">(Mononen, Neittaanmäki &amp; Vähäkainu
2014, 4; Pohjola 2014.)</xref>
        Digital games, including simulations and virtual worlds, have the potential in
teaching providing with interactive, engaging and immersive activities and tools
        <xref ref-type="bibr" rid="ref15 ref4 ref4 ref8">(Gee 2008; Shaffer
2005; Smith 2007: Kiili et al. 2015; Clark et al. 2015)</xref>
        . Learning has been somewhat gamified.
Studying of gamification is popular topic nowadays among the academic research (Hamari et al.,
2014). Gamification can be seen as an umbrella term for the use of game design elements in
nongaming contexts to improve user experience and user engagement
        <xref ref-type="bibr" rid="ref5">(Deterding et al., 2011)</xref>
        . During the
recent years, gamification of both online and more traditional courses has been researched and benefits
of using gamified solutions as a part of teaching have been suggested
        <xref ref-type="bibr" rid="ref3">(Camilleri, Busuttil, &amp;
Montebello, 2011; Muntean, 2011)</xref>
        . Gamification is generally theorised to affect the process of learning
by altering the behaviour of the student, affecting the relationship between the instructional content and
the learning outcomes (Landers, 2014). Research on gamification in a learning context often measures
student perspectives on a gamified module using such a metric as: enjoyment, engagement and impact
of learning. (Tao et al., 2012). For this reason, some gamification elements were applied in the pilot
course to be discussed later: a bird watcher’s table and division of the course content into levels.
The interest in digital games designed for learning is not restricted only to formal educational practice.
Serious games possess huge market opportunities and solutions for learning already deployed in
domains as diverse as military, commerce, health and informal learning. Benefits of gamified solutions
have been acknowledged also in healthcare education (e.g., Koivisto, Haavisto, Niemi, Katajisto, &amp;
Multisilta, 2016). For example, serious games in the health sector is a growing domain. This is an area
where challenges may occur when the topic, content and practises in serious games development differ
a lot from the very gaming business. That is why, it is of crucial importance to have the knowledge of
the area of expertise serious game operates on in order to generate appropriate and user-friendly serious
games for therapeutic use
        <xref ref-type="bibr" rid="ref1 ref16">(Bellotti et al. 2010; Sirkka &amp; Koivisto 2015)</xref>
        .
      </p>
      <p>
        Although the potential of serious games is well recognised, the educational and training possibilities
are scarce. The actual need for cooperation is also acknowledged in the report where all the Finnish
game education was studied and listed. It is evident that there are lots of possibilities to study and
research games in different educational levels.
        <xref ref-type="bibr" rid="ref10 ref13">(Mononen, Neittaanmäki &amp; Vähäkainu 2014, 7-24.)</xref>
        Since the educational operators have all their own areas of emphasis in their game-related curricula, it
would be beneficial to generate new patterns and models for cross-sectoral collaboration between
various educational levels, industries and organisations in order to create and tailor-make new
educational opportunities on serious game technology that could make the most of that time and place
independence that digitalisation offers. Through cooperation and multiform learning solutions, the lack
of education around serious games in particular could be filled in order to answer the requirements of
the future job descriptions. It is evident that both the actual development process of the game and
mastering the content serious game is developed for are equally important. This paper presents results
around the study of a serious games course that bases on the survey of actual working life needs and
was designed to bring closer the substance experts and the game development experts.
      </p>
    </sec>
    <sec id="sec-2">
      <title>2. Survey on Working Life Needs</title>
      <p>Serious Games Platform for Business and Education (SeGaBu) is an EU funded cooperation project
targeted on generating virtual online studies (30 ECTS) on serious games based on identified working
life needs. SeGaBu actors interviewed seven (n=7) of game developer companies and two other
largescale companies (n=2) in Kajaani and Oulu regions in Finland interested in implementing gamified
solutions into their businesses. The interviews were semi-structured, and the questions addressed to
company representatives were based on possible educational solutions implemented as a part of the
course. Interviews took approximately one hour each.</p>
      <p>Several of the requirements named in the interview data introduced a category of game technology.
Another category essential in serious games studies is business related topics such as budgeting, sales
and marketing, and service design. The game companies highlighted the understanding of the nature of
game development as utmost important even over mastering the technology since game development is
always a multidisciplinary teamwork. Finding a mutual language in the team is important in order to
understand each other. A serious game development team should consist of actual game developers,
but also of substance experts. The course “Basics of Serious Games” (“Hyötypelien perusteet”) was
constructed on the results of game company interviews incorporating both expertise in substance and
game development. A main goal of this course was to share the required knowledge base and thus
eliminate misunderstandings and unnecessary delays due to miscommunication.</p>
    </sec>
    <sec id="sec-3">
      <title>3. Piloting Online Course Basics of Serious Games</title>
      <p>The pilot online course “Basics of Serious Games” was implemented on the DIGMA learning platform
(http://digma.fi) administered by Tampere University of Applied Sciences (TAMK). DIGMA is a
Moodle-based learning management system (LMS) designed specially to serve open education and
cross-institutional studies and projects. The course content was created by Kajaani University of
Applied Sciences (KAMK), and the pedagogical design and gamification implemented by KAMK and
TAMK.</p>
      <p>The course was realised in November 2016 to January 2017. The aim of the course was to offer online
studies regardless of time and space. The course was targeted to persons interested in expanding their
expertise in serious games. The course objectives were (based on the interview results):
1) understanding of the concept of serious games and also the role and significance of serious
games in the field of the actor’s own industry;
2) getting familiar with the game development process, and acquiring the skills to act in the
product owner's role in a serious games development project;
3) understanding the technology perspective, acquiring a variety of innovation methods and
learning to apply a chosen method for generating and developing game ideas; and
4) learning to present own game ideas, including any relevant market research information.
The learning material consisted of videos, PowerPoint presentations, web pages, games and scientific
and popular articles. The studies were mainly carried out as independent studies with some extent of
collaboration with peer students. The assignments consisted of group discussions, playing games, a
competence test, a game evaluation and an assignment where the students accomplished brainstorming
of their own serious games.</p>
      <p>To start the course, students were assigned to describe their experience in games and motivation to join
the course. In the end, the students were asked for an optional feedback of the overall course by filling
in a semi-structured e-questionnaire. The feedback was analysed by conventional contents analysis, and
the main findings are presented in this paper.</p>
      <p>Unfortunately, at the time when the pilot course “Basics of Serious Games” first started, none of the
add-ins supporting gamification had been installed on the LMS yet. However, there were roundabouts
and other ways to gamify courses. “Basics of Serious Games” made use of levels. The higher levels
stayed closed for the students, until they had passed the assignments required on the lower levels. The
levels were created by enabling the standard Moodle course completion tracking, and then adjusting
the conditions for opening the next course section (“level”). At the moment, Level Up! and Level
Availability blocks (Level Up 2017) are installed on the DIGMA platform for controlling the students’
access from level to level.</p>
      <p>
        On the pilot course, the Moodle Progress bar tool was deployed to create a Bird watcher’s table. The
idea was to scatter little bluebird icons around the course material to be searched. From pedagogical
view, the bird watching was installed to encourage students to browse through all the course material in
the course area. Currently the Stash and Stash Availability blocks
        <xref ref-type="bibr" rid="ref17">(Stash 2017)</xref>
        are installed on the
DIGMA platform mainly to serve the same purpose: picking up objects into a repository inside the
course. The Moodle Grid layout was applied in order to create a slightly game-like visual look and feel
to the course (see Fig. 1).
Several different gamification add-ins have been installed by now on our learning platform. The Game
activity module (Moodle Game 2017) builds on Moodle’s standard questions, quizzes and glossaries
and can be applied to create interactive games, such as Hangman, Crossword, Cryptex, Snakes and
ladders, Hidden picture, Millionaire and Sudoku. Students play these games independently. In order to
meet the learning context and objectives of the course, the games need to be designed in a
pedagogically meaningful way.
      </p>
    </sec>
    <sec id="sec-4">
      <title>4. Student Feedback of the Course Implementation</title>
      <p>The student feedback was collected and analysed in order to get the insight of students’ opinions on
this type of online multidisciplinary course. Total number of 27 (N=27) answered the online
questionnaire regarding their motivation to join the course but only 12 (n=12; 44%) gave requested
feedback at the end of the course by the time of data collection. The main reasons named of joining this
course were: enhancement of one’s personal skills in gamification (n=15; 56%), general interest in
gamification (n=9; 33%), and increased gamification skills required at work (n=3; 11%).
Most of the respondents learned about the course by email (n=19; 70%), informed by teachers of some
other course (n=2; 7%), by a friend (n=1; 4%), or accidentally by surfing in the net while searching any
online courses (n=1; 4%). Four (n=4; 15%) did not indicate the source of information regarding the
course attendance.</p>
      <p>The course contents and objectives were clearly presented (n=12; 44%) in the beginning of the course,
and they were estimated to match the implementation excellently (n=6; 22%) or well (n=6; 22%). The
course was assessed to work as an online course excellently (n=9; 33%) or well (n=3; 11%) by the
students, and the assignments supported excellently (n=4; 15%), well (n=7; 26%) or satisfactorily (n=1;
4%) the set course objectives. Total amount of 15 students did not respond to these questions (n=15;
56%).</p>
      <p>The students assessed their own input in attainment of the course objectives good (n=6; 22%) or very
good (n=3; 11%), and satisfactory (n=2; 7%) or poor (n=1; 4%). Seven of the participants reported
having worked more than 70 hours to accomplish the course (n=7; 26%), while the rest (n=5; 19%)
used less than 50 hours for the studies of this 5-credit entity (total of 135 study hours). Reported study
time was used to get oriented into teaching material (n=6; 22%), accomplishing the assignments (n=6;
22%), commenting other students’ assignments (n=2; 7%) or developing one’s own game concept
(n=1;4%). Four (n=4; 15%) of the respondents assessed the course too easy one as a 5credit course.
Total amount of 15 students did not respond to these questions at all (n=15; 56%). (Table 1.)
The workload was generally experienced reasonable compared with the course guidelines, or the
guidelines being even overestimated. Some of the students described their starting level skills to be
quite high which made performing the course easy and quick. On the other hand, the study hours also
depended on the students’ motivation. Some of the answers indicate very thorough reading and
studying of the provided materials, which naturally takes more time.</p>
      <p>As to the identified learning outcomes, two main categories emerged: 1) meanings and application
possibilities of serious gaming and gamification, and serious games terminology; and 2) understanding
of the processes in serious games development. This result indicates the experienced main learning
outcomes being in line with the set course goals. More in details, the main learning achievements
attained in the course were an overview of requirements and processes in generating games (n=8;
30%), to realize the potential in serious games (n=6; 22%), to get an overview of existing serious
games (n=3; 11%), or attaining other personal learning objectives (n=9; 33%). One student did not
reply (n=1; 4%). As most rewarding parts in the course were named course materials (n=3; 19%),
assignments (n=3; 19%), activating approach in the course (n=3; 19%), interaction with other students
(n=3; 19%), achieving own learning objectives (n=3; 19%) and positive changes in attitudes (n=1; 5%).
The course gained overall grade as very good (n=5; 19%), good (n=5; 19%) or satisfactory (n=2; 7%).
15 students did not assess the course at all (n=15; 56%). (Table 2.)
better instructions in</p>
      <p>required forms
timetabling the assignments</p>
      <p>better
technical issues in exams
more material in Finnish
more entertainment game</p>
      <p>elements
audibility issues
chat for peer communication
more contact teaching
The suggestions concerning course improvement (n=9) in the future were mainly technical details
rather than anything to do with the course content: more material in Finnish, clearer instructions in
filling the required forms, better timetabling of assignments, chat for peer communication, and removal
of technical problems still existed in audibility and exams. One of the students (n=1) expected more
contents of entertainment games, too, even if the course was particularly on serious games.</p>
    </sec>
    <sec id="sec-5">
      <title>5. Discussion</title>
      <p>This paper presented results of student feedback (N=27) around a pilot study of a serious games course
that bases on the survey of actual working life needs (company interviews). It was designed to bring
together the substance experts and the game development experts. The main areas categorized from
the interview data concerning the education of serious games development were: category of game
technology, business related topics, and substance expertise. These factors were implemented as parts
of the course “Basics of Serious Games”. Main objectives of the course were: 1) understanding of the
concept of serious games and the role and significance of serious games in the field of the actor’s own
industry, 2) getting familiar with the game development process, and acquiring the skills to act in the
product owner's role in a serious games development project, 3) understanding the technology
perspective, acquiring a variety of innovation methods and learning to apply a chosen method for
generating and developing game ideas and 4) learning to present own game ideas, including any
relevant market research information. At the end of the course, students were asked for an optional
feedback of the overall course by filling a semi-structured e-questionnaire. The feedback was analysed
by conventional contents analysis. The two main learning outcomes were: 1) meanings and application
possibilities of serious gaming and gamification, and serious games terminology; and 2) understanding
of the processes in serious games development.</p>
      <p>Since giving feedback of the course was optional, the responses varied somewhat depending on the
questions. However, the data is indicative for the future development of serious games education
regarding both content and implementation. Only one of the answers related to learning outcomes was
dealing the evaluation and judgement of serious games. Further emphasis may be needed in the studies
to discuss what factors make the game a serious game and how the utility of the game is measured and
analysed. Games in general provide a lot of data which could be used in many ways. The application to
this data could also be a topic for further studies. Lack of communication with course members was
mentioned in the course feedback. In general, the challenge in online courses is to enable discussion,
collaboration and teaming. One of our developmental goals is to use new, gamified, teaching methods
and technical tools to enhance the communication and active role of the students. Gamification of the
course platform is one of the approaches to be focused on in our future studies.</p>
      <p>Especially the gamification elements, such as “Bird watch”, ranking tables, levels, collecting treasures
or coins in a Stash, adapting gamified interface on the standard LMS test tools must be further studied
in our future pilot courses. A few of gamification elements were already implemented in this course but
the research will continue during the other courses running on the platform. The objective is to learn
how students perceive and adapt gamified approach in their online learning context. This study,
however, as a pilot study offers orientation and reflects the actual needs of the game industry, working
life, and student experience. The results well indicate that within our modestly gamified environment,
the experienced learning outcomes were mainly in line with the course goals.</p>
    </sec>
    <sec id="sec-6">
      <title>Acknowledgements</title>
      <p>This paper discusses development processes and experiences of a piloting course on gamification
education organised by SeGaBu (Serious Games Platform for Business and Education) project,
implemented in 2015 –2017 and funded by the European Social Fund (ESF) in collaboration with
another ESF funded project HYPE - Multidisciplinary serious game education launched out in January
2017.</p>
      <p>Smith, R. (2007). Game impact theory: the forces that are driving the adoption of game technologies
within multiple established industries. Games and Society Yearbook.
block
for</p>
      <sec id="sec-6-1">
        <title>Moodle.</title>
      </sec>
      <sec id="sec-6-2">
        <title>Moodle plugins directory.</title>
      </sec>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          <string-name>
            <surname>Bellotti</surname>
            ,
            <given-names>F.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Berta</surname>
            ,
            <given-names>R.</given-names>
          </string-name>
          &amp;
          <string-name>
            <surname>De Gloria</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          (
          <year>2010</year>
          ).
          <article-title>Designing effective serious games: opportunities and challenges for research</article-title>
          .
          <source>International Journal of Emerging Technologies in Learning (iJET)</source>
          ,
          <volume>5</volume>
          (
          <issue>SI3</issue>
          ),
          <fpage>22</fpage>
          -
          <lpage>35</lpage>
          . http://dx.doi.org/10.3991/ijet.v5s3.1500 Bessen,
          <string-name>
            <surname>J.</surname>
          </string-name>
          (
          <year>2015</year>
          ).
          <article-title>Learning by Doing: The Real Connection between Innovation, Wages, and</article-title>
          <string-name>
            <surname>Wealth</surname>
          </string-name>
          , New Haven: Yale University Press.
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          <string-name>
            <surname>Brynjolfsson</surname>
            ,
            <given-names>E.</given-names>
          </string-name>
          &amp;
          <string-name>
            <surname>McAfee</surname>
            <given-names>A.</given-names>
          </string-name>
          (
          <year>2011</year>
          ).
          <article-title>Race against the Machine: How the Digital Revolution Is Accelerating Innovation, Driving Productivity, and Irreversibly Transforming Employment and the Economy</article-title>
          . Digital Frontier Press.
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          <string-name>
            <surname>Camilleri</surname>
            ,
            <given-names>V.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Busuttil</surname>
            ,
            <given-names>L.</given-names>
          </string-name>
          &amp;
          <string-name>
            <surname>Montebello</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          (
          <year>2011</year>
          ).
          <article-title>Social interactive learning in multiplayer games</article-title>
          .
          <source>In Serious games and edutainment applications</source>
          , Springer-Verlag, London, England, pp.
          <fpage>481</fpage>
          -
          <lpage>501</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          <string-name>
            <surname>Clark</surname>
            ,
            <given-names>D. B.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Tanner-Smith</surname>
            ,
            <given-names>E. E.</given-names>
          </string-name>
          &amp;
          <string-name>
            <surname>Killingsworth</surname>
            ,
            <given-names>S. S.</given-names>
          </string-name>
          (
          <year>2015</year>
          ). Digital Games, Design, and Learning.
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          <string-name>
            <surname>Deterding</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Sicart</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Nacke</surname>
            ,
            <given-names>L.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>O'Hara</surname>
            ,
            <given-names>K.</given-names>
          </string-name>
          , &amp;
          <string-name>
            <surname>Dixon</surname>
            ,
            <given-names>D.</given-names>
          </string-name>
          (
          <year>2011</year>
          ).
          <article-title>Gamification: using game-design elements in non-gaming contexts</article-title>
          .
          <source>In CHI'11 Ext. Abst. on Human Factors in Comp. Syst</source>
          . (
          <volume>2425</volume>
          -
          <fpage>2428</fpage>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          <string-name>
            <surname>Frey</surname>
            ,
            <given-names>C. B.</given-names>
          </string-name>
          &amp;
          <string-name>
            <surname>Osborne</surname>
            ,
            <given-names>M. A.</given-names>
          </string-name>
          (
          <year>2013</year>
          ).
          <article-title>”The Future of Employment: How Susceptible Are Jobs to Computerization”, unpublished article</article-title>
          , http://www.oxfordmartin.ox.ac.uk/downloads/academic/The_Future_of_Employment. pdf
          <string-name>
            <surname>Gee</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          (
          <year>2008</year>
          ).
          <article-title>Getting over the slump: innovation strategies to promote children's learning, Joan Ganz Cooney Center</article-title>
          .
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          <string-name>
            <surname>Hiltunen</surname>
            ,
            <given-names>K.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Latva</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          &amp;
          <string-name>
            <surname>Kaleva</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          (
          <year>2013</year>
          ). Peliteollisuus - kehityspolku,
          <source>Katsaus</source>
          <volume>303</volume>
          /
          <year>2013</year>
          , Tekes.
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          <string-name>
            <surname>Kiili</surname>
            ,
            <given-names>K.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Devlin</surname>
            ,
            <given-names>K.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Perttula</surname>
            <given-names>A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Tuomi</surname>
            ,
            <given-names>P.</given-names>
          </string-name>
          &amp;
          <string-name>
            <surname>Lindstedt</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          (
          <year>2015</year>
          ).
          <article-title>Using video games to combine learning and assessment in mathematics education</article-title>
          .
          <source>International Journal of Serious Games</source>
          <volume>2</volume>
          /4.
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          <string-name>
            <surname>Michaels</surname>
            ,
            <given-names>G.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Natraj</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          &amp;
          <string-name>
            <surname>Van Reenen</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          (
          <year>2014</year>
          ). ”
          <string-name>
            <surname>Has ICT Polarized Skill</surname>
          </string-name>
          <article-title>Demand? Evidence from Eleven Countries over Twenty- five Years”</article-title>
          ,
          <source>The Review of Economics and Statistics</source>
          <volume>96</volume>
          ,
          <fpage>60</fpage>
          -
          <lpage>77</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          <string-name>
            <surname>Mononen</surname>
            ,
            <given-names>L.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Neittaanmäki</surname>
            ,
            <given-names>P.</given-names>
          </string-name>
          &amp;
          <string-name>
            <surname>Vähäkainu</surname>
            ,
            <given-names>P.</given-names>
          </string-name>
          (
          <year>2014</year>
          ).
          <source>Suomen pelialan koulutuksen kartoitus</source>
          <year>2014</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref11">
        <mixed-citation>
          <source>Informaatioteknologian tiedekunnan julkaisuja No</source>
          .
          <volume>19</volume>
          /
          <year>2014</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref12">
        <mixed-citation>
          https://www.jyu.fi/it/tutkimus/192014_Suomenpelialankoulutuksenkartoitus_NETTI.pdf Muntean,
          <string-name>
            <surname>C.I..</surname>
          </string-name>
          (
          <year>2011</year>
          )
          <article-title>Raising engagement in e-learning through gamification</article-title>
          .
          <source>The 6th International Conference on Virtual Learning</source>
          , Romania, pp.
          <fpage>323</fpage>
          -
          <lpage>329</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref13">
        <mixed-citation>
          <string-name>
            <surname>Niemi</surname>
            ,
            <given-names>H.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Multisilta</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Lipponen</surname>
            ,
            <given-names>L.</given-names>
          </string-name>
          &amp;
          <string-name>
            <surname>Vivitsou</surname>
          </string-name>
          , M. (eds.) (
          <year>2014</year>
          ).
          <article-title>Finnish innovations and technologies in schools: A guide towards new ecosystems of learning</article-title>
          (p.
          <fpage>175</fpage>
          ). Rotterdam: Sense Publishers.
        </mixed-citation>
      </ref>
      <ref id="ref14">
        <mixed-citation>
          <string-name>
            <surname>Pohjola</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          (
          <year>2014</year>
          ).
          <article-title>Suomi uuteen nousuun: ICT ja digitalisaatio tuottavuuden ja talouskasvun lähteinä</article-title>
          ,
          <source>Finnish Technology Industries.</source>
        </mixed-citation>
      </ref>
      <ref id="ref15">
        <mixed-citation>
          <string-name>
            <surname>Shaffer</surname>
            ,
            <given-names>D.W.</given-names>
          </string-name>
          (
          <year>2005</year>
          ).
          <article-title>Epistemic games</article-title>
          .
          <source>Innovate: Journal of Online education</source>
          ,
          <volume>1</volume>
          (
          <issue>6</issue>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref16">
        <mixed-citation>
          <string-name>
            <given-names>Sirkka A.</given-names>
            &amp;
            <surname>Koivisto</surname>
          </string-name>
          <string-name>
            <surname>A.</surname>
          </string-name>
          (
          <year>2015</year>
          ).
          <article-title>Arts and technology play well together</article-title>
          .
          <source>In Sirkka A. (ed.) Art</source>
          ,
          <article-title>Games and Sensors Harnessed to Enhance Well-being</article-title>
          . Satakunta University of Applied Sciences,
          <string-name>
            <surname>Series</surname>
            <given-names>B</given-names>
          </string-name>
          , Reports 3/
          <year>2015</year>
          , pp.
          <fpage>10</fpage>
          -
          <lpage>16</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref17">
        <mixed-citation>
          <string-name>
            <surname>Stash</surname>
          </string-name>
          (
          <year>2017</year>
          ). The Stash https://moodle.org/plugins/block_stash
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>