<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta />
    <article-meta>
      <title-group>
        <article-title>Primary School STEM education through co-creative methodologies</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Styliani Siouli</string-name>
          <email>sioulisa@nured.auth.gr</email>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Ioanna Dratsiou</string-name>
          <email>idratsiou@nured.auth.gr</email>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Panagiotis E. Antoniou</string-name>
          <xref ref-type="aff" rid="aff1">1</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Panagiotis D. Bamidis</string-name>
          <email>bamidis@auth.gr</email>
          <xref ref-type="aff" rid="aff1">1</xref>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          <institution>Education Science - Learning Technologies post-graduate program, Faculty of Education Aristotle University of Thessaloniki (AUTH) Thessaloniki</institution>
          ,
          <country country="GR">Greece</country>
        </aff>
        <aff id="aff1">
          <label>1</label>
          <institution>Lab of Medical Physics, Faculty of Medicine, School of Health Sciences Aristotle University of Thessaloniki (AUTH) Thessaloniki</institution>
          ,
          <country country="GR">Greece</country>
        </aff>
      </contrib-group>
      <abstract>
        <p>Technology Enhanced Learning (TEL) in Science Technology Engineering Mathematics (STEM) Education is a well-established method for engaging learners with difficult and counterintuitive concepts such those in space and astronomy. Co-creative, participatory methods are easily applicable in TEL for STEM due to a multitude of existing platforms and services for creative technology based education. This work describes three co-creative endeavors in astronomy and solar system education for children of primary education. The cases comprised of an exploratory game teaching about the surface of the planet Mars, an interactive Augmented Reality (AR) application exploring the surface of the Moon and a team endeavor for learning through creating a general astronomy quiz. In all cases, participants were excited and engaged with the subject matter and the technology a fact that led to successful educational episodes. Beyond that the core outcome of this work was a first identification of a common co-creative workflow of introduction-toolset provision- implicit creative education -publishing content engagement in all three cases. This co-creative workflow identifies very closely with the core theoretical tenets of constructivist learning theory. In that context this work is the first step towards formal identification of the co-creative workflows with pure educational methodologies.</p>
      </abstract>
      <kwd-group>
        <kwd>STEM education</kwd>
        <kwd>Technology-Enhanced learning</kwd>
        <kwd>Programming</kwd>
        <kwd>Co-creation</kwd>
        <kwd>Augmented Reality</kwd>
        <kwd>Primary Education</kwd>
      </kwd-group>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>Introduction</title>
      <sec id="sec-1-1">
        <title>Technology-Enhanced Learning (TEL) in Science Technology</title>
      </sec>
      <sec id="sec-1-2">
        <title>Engineering Mathematics (STEM)</title>
        <p>
          The endeavor of supporting students in conceiving heavily the gist of STEM
disciplines is strengthened by the integration of technology [
          <xref ref-type="bibr" rid="ref12">12</xref>
          ]. Considered essential
to promoting innovation, productivity and overall economic growth, STEM education
is considered as crucial factor in fueling the current and future STEM workforce [
          <xref ref-type="bibr" rid="ref9">9</xref>
          ].
STEM education provides students with appropriate skills that correspond to the
current development of digital technologies [
          <xref ref-type="bibr" rid="ref22">22</xref>
          ], and supports them in developing
new competencies which are considered crucial to their effective adaptation to our
digital world [
          <xref ref-type="bibr" rid="ref17">17</xref>
          ]. ‘Technological literacy’ and ‘ICT literacy’ are terms that widely
describe “the interest, attitude and ability of individuals to appropriately use digital
technology and communication tools to access, manage, integrate and evaluate
information, construct new knowledge, and communicate with others in order to
participate effectively in society” [
          <xref ref-type="bibr" rid="ref30">30</xref>
          ].
        </p>
        <p>
          Such technological integration into school education is supposed to be an essential
means capable of reinforcing learning and raising student achievement in STEM
disciplines [
          <xref ref-type="bibr" rid="ref8">8</xref>
          ]. Thus, learning that is supported and enhanced by technology, known
as Technology-Enhanced Learning (TEL), could be an effective pathway to follow
aiming students’ smooth approach and acclimatization with STEM education. The
appropriate use and integration of technology could set the ground a more
learnercentered environment by allowing students to take on more responsibilities in their
learning process [
          <xref ref-type="bibr" rid="ref28">28</xref>
          ]. TEL in science develops a learning environment that could help
students to make their thinking visible, encourage student collaboration, promoted
autonomous lifelong learning and make science concepts more accessible [
          <xref ref-type="bibr" rid="ref21">21</xref>
          ].
Because of this, TEL environments have emerged as the subject of study for many
researchers with an interest in science education and technology [
          <xref ref-type="bibr" rid="ref31">31</xref>
          ].
1.2
        </p>
      </sec>
      <sec id="sec-1-3">
        <title>Computational Thinking through programming activities</title>
        <p>
          Technology has literally brought important and considerable changes in every aspect
of our lives, including education. These days, a significant part of the discussion is
about how students can end up skilled designers and makers of computer artifacts
[
          <xref ref-type="bibr" rid="ref29">29</xref>
          ].
        </p>
        <p>
          Computational thinking has been raised as an essential educational approach that has
received increased consideration in the last years and focuses on bridging the gap
between students and computer programming across the years of schooling [
          <xref ref-type="bibr" rid="ref18">18</xref>
          ]. That
kind of pedagogical practices encourage teachers to attempt activities in the classroom
that advance learning and creating, with computer programming proposed as a
learning technology that can enable the improvement of capabilities and
competencies, for example, critical thinking and problem-solving [
          <xref ref-type="bibr" rid="ref14">14</xref>
          ]. Besides, there
have been conducted several attempts aiming introducing programming to children,
though not all necessarily within the classroom, while some have created after school
clubs to introduce children to computing [
          <xref ref-type="bibr" rid="ref16">16</xref>
          ].
Computer programming requires students to engage in problem-solving process that
form the defining core of computational thinking [
          <xref ref-type="bibr" rid="ref25">25</xref>
          ]. However, due to the
complexity of some such tasks students are possible to be confronted with difficulties.
To alleviate frustration in these cases programming is taught through a game-based
approach, so that the whole process becomes an enjoyable experience [
          <xref ref-type="bibr" rid="ref26">26</xref>
          ] accessible
at an early age [
          <xref ref-type="bibr" rid="ref20">20</xref>
          ]. Computational thinking utilizes programming tasks successfully
because these tasks incorporate a multi-dimensional and iterative process. This
process runs in a number of phases including: forming and dealing with problems in a
manner that using computational tools appears to be the most effective practice;
organizing and analyzing data; represent data via models and simulations; adopting
algorithmic thinking in order to automate solutions; assessing solutions; and applying
the problem-solving practice to other contexts too [
          <xref ref-type="bibr" rid="ref25">25</xref>
          ]. Apart from domain-specific
skills, developing a strong background in computational thinking competencies
support students in enhancing their problem solving skills and their ability to think
critically [
          <xref ref-type="bibr" rid="ref37">37</xref>
          ], as in meantime students develop their synthetic and analytical thinking,
foster their skills in designing and solving algorithms, and affects in a positive way
their creativity and imagination [
          <xref ref-type="bibr" rid="ref24">24</xref>
          ].
1.3
        </p>
      </sec>
      <sec id="sec-1-4">
        <title>Co-creation as participatory method between students and instructors</title>
        <p>
          During the past decades, the focus on the management of experience, student
participation and value co-creation in education has increased [
          <xref ref-type="bibr" rid="ref7">7</xref>
          ]. In the last years
there has been an increased interest in the numerous benefits of engaging students as
partners, co-inquires, who produce and co-create their own learning experience [
          <xref ref-type="bibr" rid="ref6">6</xref>
          ].
As students have unique views about teaching and learning, they need to be given the
opportunity to share their views on the education processes and to actively participate
in the training process, to achieve improved results in learning, developing thinking
skills and creativity; to obtain rich understanding of concepts, and to create
knowledge and new educational processes [
          <xref ref-type="bibr" rid="ref35">35</xref>
          ].
        </p>
        <p>
          Co-production of learning and teaching advocates a more prominent democratization
of the educational process and recent work on it challenges the traditional norms in
learning. When educators and pupils work cooperatively to make parts of educational
module as well as pedagogical approaches, then co-production of learning and
teaching take place [
          <xref ref-type="bibr" rid="ref5">5</xref>
          ].
        </p>
        <p>
          As Biggs emphasizes [
          <xref ref-type="bibr" rid="ref3">3</xref>
          ], knowledge is constructed through the learner’s activities
and interactions among lecturers, students, and peers [
          <xref ref-type="bibr" rid="ref36">36</xref>
          ]. Constructivist learning
reflects the principle that students discover their own truths through acting [
          <xref ref-type="bibr" rid="ref10">10</xref>
          ] while
the constructionist perspective encourages students’ knowledge through developing
games and objects [
          <xref ref-type="bibr" rid="ref32">32</xref>
          ].
        </p>
        <p>
          Related to the above there is a rapidly developing research literature on ‘‘student
voice’’ activities, which encourage and advance a different and transformative scope
in educational practices [
          <xref ref-type="bibr" rid="ref34">34</xref>
          ]. In school settings, the ‘‘student voice’’ (SV) approach
includes a wide level of aspects: considering and evaluating students’ views express
regarding their learning experiences; communicating students’ views to
decisionmakers; equating students to equal partners in teaching and learning processes; and
encouraging them to be more active in shaping or changing their learning [
          <xref ref-type="bibr" rid="ref13">13</xref>
          ].
Another term that incorporates co-creation is participatory pedagogy. Participatory
pedagogy aiming for student centered learning and enabling learners to be part of
creating both content and structure [
          <xref ref-type="bibr" rid="ref1">1</xref>
          ]. Moreover, opportunities to practice work‐
group skills were provided to children (aged 7 to 12) by playing games and
gamification [
          <xref ref-type="bibr" rid="ref21">21</xref>
          ]. Designing games enhance the sense of classroom community,
which encourages students to be active members and cooperate with others [
          <xref ref-type="bibr" rid="ref2">2</xref>
          ] and to
share possible ideas and different ways of doing things in the game‐making
environment [
          <xref ref-type="bibr" rid="ref33">33</xref>
          ].
        </p>
        <p>
          Concerning the roles that students can embody in co-creating they are: (1) consultant,
sharing and discussing valuable perspectives on learning and teaching, (2)
coresearcher, cooperating in a meaningful way on teaching and learning research with
staff, (3) pedagogical co-designer, being responsible for designing learning, teaching
and evaluation and (4) representative, sharing student voices in a range of university
settings [
          <xref ref-type="bibr" rid="ref11">11</xref>
          ].
1.4
        </p>
      </sec>
      <sec id="sec-1-5">
        <title>Aim and scope of this work.</title>
        <p>This paper describes the methodology and outcomes of implementing co-creation
methodology in primary education. Specifically it reports on the co-creation
endeavors of primary school learners who, through programming tasks, became
versed in computational thinking, while co-creating with their teachers useful digital
content for solar system and astronomy education.
2</p>
      </sec>
    </sec>
    <sec id="sec-2">
      <title>Co-creation support Methods</title>
      <p>The activities reported in this work were conducted at the Experimental Primary
School of the University of Thessaloniki in the city of Thessaloniki, in northern
Greece. For the first time and for the school year 2017-2018 an Astronomy club
“Space Explorers” was offered for 14 children from Year 5 to Year 6 aged between
11 and 12 years old, every Friday from 1:45 p.m. to 3:15 p.m. The aim of the “Space
Explorers Club” was to teach students life skills through the intelligent application of
astronomy and space science concepts. Students that enrolled in this club learnt about
the mysteries of the universe. As the children and teacher who offered the space club
at the school were the first users of this new type of club and learning activity, their
role as designers and developers of the environment and the pedagogy was
significant.</p>
      <p>The “Space explorers” participants attended the club in the computer lab once per
week. Integration of technology into classroom instruction was particularly
emphasized. There was a SMART Board connected to the teacher’s computer and a
digital projector in order to show the computer image. Students and teacher could
control computer applications directly from the SMART Board display. The computer
lab at the Experimental Primary School is equipped with 12 individual workstations, 6
desktops and 6 portable laptops. Each computer is equipped with Windows 10, Office
2016. Wireless internet access is available through the lab. Students of Space Club
also were encouraged to “bring their own device” like smartphones and iPads to use
in the class for educational purposes.
3
3.1</p>
    </sec>
    <sec id="sec-3">
      <title>Results; Collaborative game-making</title>
      <sec id="sec-3-1">
        <title>Case I</title>
        <p>While the fourteen students were playing a “Mars rover game” on NASA kid’s Club
site, they were motivated by their teacher to form seven pairs and come up with an
idea of constructing a similar, simple game in order to reformulate their
understandings, express their personal ideas, practice their work-group skills and
learn programming basics. Building a game requires a higher level of thinking and the
ability to solve problems proactively. Firstly, students were encouraged to decide on
the type of game they would like to build taking as inspiration the NASA game they
played. Students were urged to ask questions and provide help to others, if needed.
Developing a collaborative game is a complex task which requires strong ICT skills.
Primary school students generally have no previous programming experience and a
low ICT skill level. For this reason “Scratch” (http://scratch.mit.edu) developed by
the research group at the MIT Media Lab was introduced to students as visual
programming language designed for young learners. Scratch is a free and open
source, which gives students the opportunity to understand how to code using block
coding instead of typing commands. The students were familiar with “Scratch”
because they have been using it since the third grade and they had already developed
a basic competency programming. They were also encouraged to use the digital
environment in order to imagine, discuss, create, play and share the game with their
friends and classmates on the web. In order to facilitate cooperation, negotiation and
collaboration the teacher gave different areas of responsibility to each learner of the
team (problem finding, problem solving, generate ideas, concept design,
implementation, animation, sound, visuals etc.).</p>
        <p>
          Students made mistakes while programming and learnt from their mistakes by
debugging their coding project. There were seven games created, each by one of the
pairs formed during the class. The games varied in complexity (timer, scoring system)
and all used keyboard and mouse control to play the game. All games consisted of a
Mars rover (sprite) that landed on Mars and had the same backdrop theme (Mars
surface area). The teams coded the rover to start moving each time a key is pressed
and to turn in the direction they wanted by pressing other keys. By reordering the
programming blocks, they created command blocks that made their rover turn in the
direction of their wish and by adjusting the numerical content of these blocks, they
made their rover move slowly or fast. They also used conditions. When the Mars
rover was moving around, if it touched an object (water, gold, Martians etc.) then it
caused an event to happen (change score, bounce, disappear). Throughout game
development, students took turns testing out other students’ game functionality and
design. All games ended with a sound playing and a message stating “WE ARE THE
WINNERS!” A characteristic screenshot from the development environment is
presented in Fig. 1. Engagement during classroom activities was high on inspiring,
designing and making a game and provided the stimulus to learn more.
The next co-creative activity was implementing augmented reality in designing and
producing AR-animation system to represent what students have learnt about the
moon, its surface and the first man who stepped on the moon. In order to achieve this,
Blippbuilder by Blippar [
          <xref ref-type="bibr" rid="ref4">4</xref>
          ] was used in building the augmented contents. Blippar
allows applications to be developed for both Android and iOS operating systems.
Blippar for Education platform includes a suite of three products tailored for teachers:
an Educator Dashboard, an Educator Community, and a special Educator version of
BlippBuilder. Initially, students learned and discussed about moon surface, the
craters, the marks on its surface and the dusty footsteps of astronauts as depicted by
historical photographs [
          <xref ref-type="bibr" rid="ref27">27</xref>
          ]. Students were randomly divided in 4 groups of 3 students
and 1 pair. Each group collected information about a certain learning topic (erosion
and atmosphere, tectonics and craters, volcanism and craters, the unseen side of the
moon and the first step on moon), to share, discuss and verify to accurate. Students
were encouraged to build an augmented tour on moon using a drag and drop of
“Blippbuilder” portraying the gathered information regarding their learning topic.
Before starting all students of the club educated on the Blippbuilder AR Creator for
education. They watched video tutorials and discussed and analyzed the workflow
with their teacher. Interaction and dialogue in the classroom among students and with
teacher was the key to overcome difficulties and develop representational
competence. Students then created an account with Blippar, downloaded the Blippar
app to their mobile devices (smart phones, tablets and iPads) in order to be able to
iterate on content creation with Blippbuilder. They decided the on the location of the
markers (the real-word 2D image) that they were going to use for their topic, created
their graphics built in Photoshop and uploaded them into Blippbuilder. YouTube
videos and mp3 sounds were also added through drag and drop interface. Once they
finished building their blipps they tested in on their mobile devices and they set it live
(Fig. 2).
At the end of the school year the 14 students of the “Space Explorers Club” were
motivated by their teacher to create a formative evaluation Astronomy-quiz for the
other students of the school, who did not participate in the Astronomy Club, in order
to test their peers’ knowledge. Involving games in the teaching process is a method to
open up knowledge transfer and to increase students’ interest. Space Explorers were
engaged in content in a fun way. For this purpose, the teacher introduced a
gamebased learning platform, Kahoot! [
          <xref ref-type="bibr" rid="ref19">19</xref>
          ]. “Kahoot!” is a student response framework
that engages students through game-like pre-made or improvised quizzes, discussions
and reviews. It is a free and simple tool to learn and utilize. Initially, the 14th Club
students divided in two groups of seven created two accounts on Kahoot! and then
they logged into their account created two quizzes for evaluation of astronomy
knowledge named “Astronomy Quiz 1” and “Astronomy Quiz 2”. The students also
uploaded pictures and music to each question of the quiz by using the “drag and drop
option” in order to look aesthetically better. For each question, the students as creators
included up to four answers for the other students to choose from. Time and score
value for each question were set at 10 seconds and 1,000 points respectively. During
the creation of the quizzes the students acted as a team supporting one another and
they were moving all around the class. The students of the club created 14 questions
for each quiz working collaboratively.
        </p>
        <p>After creating the quiz the teacher received the PIN code for both quizzes and
launched the quiz for the 25 evaluated students (Year 5 and Year 6). These did not
participate in the “Space Explorers Club”, were randomly divided into groups of three
or two and used their own mobile devices (cell phones, tablets and iPads) in order to
compete on which group knows best on Astronomy. They signed in using the web
address (https://Kahoot.it) to access this eLearning platform and the selected team
mode to use on device per team. They entered the game pin displayed on the SMART
Board and the game started with active participation in the classroom.
The students of the school were excited about playing both games that were created
by the Space Explorers. Fun and joy surrounded the games strengthening group
dynamics. It must be noted that for reasons of confidence building the “Space
Explorers” group facilitated along with the teacher the quiz session answering
clarifying questions and helping with the technical nuances of the system. After
quizzing, the percentage of right answers and time scores were seen in the SMART
Board display. The real-time feedback further engaged students and gained their
attention. Indeed, students remarked that they enjoyed this assignment because they
were creatively using technology within a learning environment.
4</p>
      </sec>
    </sec>
    <sec id="sec-4">
      <title>Discussion</title>
      <p>This work presented 3 cases of co-creations in primary education. The cases by
themselves are not impressive for study on the methods and scope of co-creation and
the content created was adequate for use by other participants in the educational scope
of this endeavor. The key outcome from this work is, however, the use of co-creative
methodologies for purely educational aims. The 3 cases had a common structure that
is at the core of co-creation. Informing the user/student about what is the “product”
that is to be built. 2) Providing her/him with a toolset that facilitates this creative
process and removes highly technical burdens. 3) Offering the creative freedom to the
participant to express himself and produce ownership of the created “product” so that
4) the participant/user engages with the “product”. In all of the presented cases this
workflow was followed as an educational method. First the students were informed
about the educational topic. Then the facilitating teacher provided an easy to use
technical solution for exploring the topic creatively, without resorting to heavy
programming solutions. After that, the little learners were left to their own devices to
develop content as they saw fit, while, at the same time, implicitly being educated on
the topic. Finally, the content that was developed was “published” either as material
for others, or for the learners themselves to engage and anchor, through it, the
implicitly transferred knowledge. That way the learners absorbed the content and
concluded the educational process.</p>
      <p>
        This direct overlapping of the co-creative workflow with the educational process is
also formalized in the tenets of constructivist learning. The idea of constructivist
learning includes four components. These are: 1) Students construct meaning on their
own, 2) new learning is built on previous information and knowledge, 3) learning is
united with social association and interaction and 4) meaningful learning develops
through authentic tasks [
        <xref ref-type="bibr" rid="ref10">10</xref>
        ]. Additionally, by using technology from the
constructivist perspective learners can 1) access, select and interpret information, 2)
review and adjust their work to enhance the quality, 3) share or exchange information
with others and present data, 4) assess their work and enhance efficiency, 5) be
innovative and take risks and 6) gain certainty, boost confidence and independence
[
        <xref ref-type="bibr" rid="ref15">15</xref>
        ].
      </p>
      <p>Given this theoretical context, this work is the first step towards exploring the
interplay between co-creative methods and constructivist educational paradigms.
Moreover, further analysis and research must be conducted to validate or not the
interaction and learning experiences of students in STEM. For this purpose, in order
to examine the degree to which these variables (ex. excitement, engagement, learning)
are affected in co-creation situations, a multidimensional co-creation STEM
experience scale should be created and be combined with in depth interviews that may
provide future insight into the students experiences and learning.</p>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          1.
          <string-name>
            <surname>Andersen</surname>
            ,
            <given-names>R.</given-names>
          </string-name>
          , &amp;
          <string-name>
            <surname>Marisa</surname>
            <given-names>P.</given-names>
          </string-name>
          :
          <article-title>Participatory pedagogy in an open educational course: challenges and opportunities</article-title>
          .
          <source>Distance Education</source>
          <volume>35</volume>
          (
          <issue>2</issue>
          ),
          <fpage>234</fpage>
          -
          <lpage>249</lpage>
          (
          <year>2014</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          2.
          <string-name>
            <surname>Baytak</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          &amp;
          <string-name>
            <surname>Land</surname>
            ,
            <given-names>S. M.:</given-names>
          </string-name>
          <article-title>A case study of educational game design by kids and for kids</article-title>
          .
          <source>Procedia Social and Behavioral Sciences</source>
          ,
          <volume>2</volume>
          ,
          <fpage>5242</fpage>
          ‐
          <lpage>5246</lpage>
          (
          <year>2010</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          3.
          <string-name>
            <surname>Biggs</surname>
            ,
            <given-names>J. B.</given-names>
          </string-name>
          :
          <article-title>Teaching for Quality Learning</article-title>
          at University. Society for Research into Higher Education and Open University, Open University Press, London (
          <year>1999</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>4. https://web.blippar.com/</mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          5.
          <string-name>
            <surname>Bovill</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Cook-Sather</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Felten</surname>
            ,
            <given-names>P.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Millard</surname>
            ,
            <given-names>L.</given-names>
          </string-name>
          &amp;
          <string-name>
            <surname>Moore-Cherry</surname>
          </string-name>
          , N.:
          <article-title>Addressing potential challenges in co-creating learning and teaching: overcoming resistance, navigating institutional norms and ensuring inclusivity in student- staff partnerships</article-title>
          .
          <source>High Education</source>
          <volume>71</volume>
          ,
          <fpage>195</fpage>
          -
          <lpage>208</lpage>
          (
          <year>2015</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          6.
          <string-name>
            <surname>Bovill</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Cook-Sather</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          , &amp;
          <string-name>
            <surname>Felten</surname>
            ,
            <given-names>P.</given-names>
          </string-name>
          :
          <article-title>Students as co-creators of teaching approaches, course design and curricula: Implications for academic developers</article-title>
          .
          <source>International Journal for Academic Development</source>
          <volume>16</volume>
          (
          <issue>2</issue>
          ),
          <fpage>133</fpage>
          -
          <lpage>145</lpage>
          (
          <year>2011</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          7.
          <string-name>
            <surname>Bowden</surname>
          </string-name>
          , J. and
          <string-name>
            <surname>D'Alessandro</surname>
            ,
            <given-names>S.:</given-names>
          </string-name>
          <article-title>Co-creating value in higher education: the role of interactive classroom response technologies</article-title>
          .
          <source>Asian Social Science</source>
          <volume>7</volume>
          (
          <issue>11</issue>
          ),
          <fpage>35</fpage>
          -
          <lpage>49</lpage>
          (
          <year>2011</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          8.
          <string-name>
            <surname>Brophy</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Klein</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Portsmore</surname>
            ,
            <given-names>M</given-names>
          </string-name>
          , &amp;
          <string-name>
            <surname>Rogers</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          :
          <article-title>Advancing engineering education in P‐12 classrooms</article-title>
          .
          <source>Journal of Engineering Education</source>
          <volume>97</volume>
          (
          <issue>3</issue>
          ),
          <fpage>369</fpage>
          -
          <lpage>38</lpage>
          (
          <year>2008</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          9.
          <string-name>
            <surname>Caprile</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Palmen</surname>
            ,
            <given-names>R.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Sanz</surname>
          </string-name>
          , &amp;
          <string-name>
            <surname>Dente</surname>
            ,
            <given-names>G:</given-names>
          </string-name>
          <article-title>Encouraging STEM studies for the labor market (Directorate-General for Internal Policies: European Parliament)</article-title>
          . Brussels, Belgium: European Union. Retrieved from http://www.europarl.europa.eu/RegData/etudes/STUD/
          <year>2015</year>
          /542199/IPOL_STU%
          <volume>282015</volume>
          %29542199_EN.pdf,
          <source>last accessed</source>
          <year>2018</year>
          /06/18.
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          10.
          <string-name>
            <surname>Cooperstein</surname>
            ,
            <given-names>S. E.</given-names>
          </string-name>
          , &amp; E. Kocevar-Weidinger:
          <article-title>Beyond Active Learning: A Constructivist Approach to Learning</article-title>
          .
          <source>Reference Services Review</source>
          <volume>32</volume>
          (
          <issue>2</issue>
          ),
          <fpage>141</fpage>
          -
          <lpage>148</lpage>
          (
          <year>2004</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref11">
        <mixed-citation>
          11.
          <string-name>
            <surname>Dunne</surname>
            ,
            <given-names>E.</given-names>
          </string-name>
          , &amp;
          <string-name>
            <surname>Zandstra</surname>
            ,
            <given-names>R</given-names>
          </string-name>
          :
          <article-title>Students as change agents: new ways of engaging with learning and teaching in higher education</article-title>
          .
          <source>ESCalate Higher</source>
          Education Academy Subject Centre for Education/University of Exeter, Bristol. http://escalate.ac.uk/8064. last accessed
          <year>2018</year>
          /06/20/.
        </mixed-citation>
      </ref>
      <ref id="ref12">
        <mixed-citation>
          12.
          <string-name>
            <surname>English</surname>
          </string-name>
          . D.L.:
          <article-title>Advancing Elementary and Middle School STEM Education</article-title>
          .
          <source>International Journal of Science and Math Education</source>
          <volume>15</volume>
          (
          <issue>1</issue>
          ),
          <fpage>5</fpage>
          -
          <lpage>24</lpage>
          (
          <year>2017</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref13">
        <mixed-citation>
          13.
          <string-name>
            <surname>Faux</surname>
            ,
            <given-names>F.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>McFarlane</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Roche</surname>
            ,
            <given-names>N.</given-names>
          </string-name>
          , &amp;
          <string-name>
            <surname>Facer</surname>
            ,
            <given-names>K.</given-names>
          </string-name>
          :
          <article-title>Research publications: Listening to the learner</article-title>
          . Future Lab Publications,
          <string-name>
            <surname>Bristol</surname>
          </string-name>
          (
          <year>2006</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref14">
        <mixed-citation>
          14.
          <string-name>
            <surname>Fessakis</surname>
            ,
            <given-names>G.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Gouli</surname>
            ,
            <given-names>E.</given-names>
          </string-name>
          , &amp;
          <string-name>
            <surname>Mavroudi</surname>
            ,
            <given-names>E.</given-names>
          </string-name>
          :
          <article-title>Problem solving by 5-6 years old kindergarten children in a computer programming environment: A case study</article-title>
          .
          <source>Computers &amp; Education</source>
          <volume>63</volume>
          ,
          <fpage>87</fpage>
          -
          <lpage>97</lpage>
          (
          <year>2013</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref15">
        <mixed-citation>
          15.
          <string-name>
            <surname>Ghasemi</surname>
            ,
            <given-names>B.</given-names>
          </string-name>
          &amp;
          <string-name>
            <surname>Hashemi</surname>
            ,
            <given-names>M.:</given-names>
          </string-name>
          <article-title>ICT: Newwave in English language learning/teaching</article-title>
          .
          <source>Procedia Social and Behavioral Sciences</source>
          <volume>15</volume>
          ,
          <fpage>3098</fpage>
          -
          <lpage>3102</lpage>
          (
          <year>2011</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref16">
        <mixed-citation>
          16.
          <string-name>
            <surname>Gibson</surname>
            ,
            <given-names>J.P.:</given-names>
          </string-name>
          <article-title>A noughts and crosses Java applet to teach programming to primary school children</article-title>
          , Computer Science Press, Inc.,
          <string-name>
            <surname>Kilkenny</surname>
            <given-names>City</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Ireland</surname>
          </string-name>
          (
          <year>2003</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref17">
        <mixed-citation>
          17.
          <string-name>
            <surname>Gut</surname>
            ,
            <given-names>D. M.</given-names>
          </string-name>
          :
          <article-title>Integrating 21st century skills into the curriculum</article-title>
          . In G. Wan, &amp;
          <string-name>
            <surname>D. M. Gut</surname>
          </string-name>
          (Eds.),
          <source>Bringing schools into the 21st Century</source>
          . Springer, Dordrecht (
          <year>2011</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref18">
        <mixed-citation>
          18.
          <string-name>
            <surname>Israel</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Pearson</surname>
            ,
            <given-names>J. N.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Tapia</surname>
            ,
            <given-names>T.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Wherfel</surname>
            ,
            <given-names>Q. M.</given-names>
          </string-name>
          , &amp;
          <string-name>
            <surname>Reese</surname>
            ,
            <given-names>G.</given-names>
          </string-name>
          :
          <article-title>Supporting all learners in school-wide computational thinking: A cross-case qualitative analysis</article-title>
          .
          <source>Computers &amp; Education</source>
          <volume>82</volume>
          ,
          <fpage>263</fpage>
          -
          <lpage>279</lpage>
          (
          <year>2015</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref19">
        <mixed-citation>19. https://create.kahoot.it/</mixed-citation>
      </ref>
      <ref id="ref20">
        <mixed-citation>
          20.
          <string-name>
            <surname>Kalelioğlu</surname>
            ,
            <given-names>F.</given-names>
          </string-name>
          :
          <article-title>A new way of teaching programming skills to</article-title>
          K-12
          <source>students: Code.org. Computers in Human Behavior</source>
          <volume>52</volume>
          ,
          <fpage>200</fpage>
          -
          <lpage>210</lpage>
          (
          <year>2015</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref21">
        <mixed-citation>
          21.
          <string-name>
            <surname>Kangas</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          :
          <article-title>Creative and playful learning: Learning through game co‐creation and games in playful learning environment</article-title>
          .
          <source>Thinking Skills and Creativity</source>
          ,
          <volume>5</volume>
          ,
          <fpage>1</fpage>
          ‐
          <lpage>15</lpage>
          (
          <year>2010</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref22">
        <mixed-citation>
          22.
          <string-name>
            <surname>Kong</surname>
            ,
            <given-names>S. C.</given-names>
          </string-name>
          :
          <article-title>Developing information literacy and critical thinking skills through domain knowledge learning in digital classrooms: An experience of practicing flipped classroom strategy</article-title>
          .
          <source>Computers &amp; Education</source>
          <volume>78</volume>
          ,
          <fpage>160</fpage>
          -
          <lpage>173</lpage>
          (
          <year>2014</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref23">
        <mixed-citation>
          23.
          <string-name>
            <surname>Linn</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          :
          <string-name>
            <surname>Computers</surname>
          </string-name>
          , Teachers, Peers: Science Learning Partners. Lawrence Erlbaum Associates Publishers, New Jersey (
          <year>2000</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref24">
        <mixed-citation>
          24.
          <string-name>
            <surname>Liu</surname>
            , C. C., Cheng,
            <given-names>Y. B.</given-names>
          </string-name>
          , &amp;
          <string-name>
            <surname>Huang</surname>
            ,
            <given-names>C. W.:</given-names>
          </string-name>
          <article-title>The effect of simulation games on the learning of computational problem solving</article-title>
          .
          <source>Computers &amp; Education</source>
          <volume>57</volume>
          (
          <issue>3</issue>
          ),
          <fpage>1907</fpage>
          -
          <lpage>1918</lpage>
          (
          <year>2011</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref25">
        <mixed-citation>
          25.
          <string-name>
            <surname>Macdonald</surname>
            ,
            <given-names>C. V.</given-names>
          </string-name>
          :
          <article-title>STEM Education: A review of the contribution of the disciplines of science, technology, engineering and mathematics</article-title>
          .
          <source>Science Education International</source>
          <volume>27</volume>
          (
          <issue>4</issue>
          ),
          <fpage>530</fpage>
          -
          <lpage>569</lpage>
          (
          <year>2016</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref26">
        <mixed-citation>
          26.
          <string-name>
            <surname>Marginson</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Tytler</surname>
            ,
            <given-names>R.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Freeman</surname>
            ,
            <given-names>B.</given-names>
          </string-name>
          &amp;
          <string-name>
            <surname>Roberts</surname>
            ,
            <given-names>K.</given-names>
          </string-name>
          : STEM:
          <article-title>Country comparisons</article-title>
          .
          <source>Australian Council of Learned Academies</source>
          , Melbourne, Australia (
          <year>2013</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref27">
        <mixed-citation>27. https://spaceplace.nasa.gov/craters/</mixed-citation>
      </ref>
      <ref id="ref28">
        <mixed-citation>
          28.
          <string-name>
            <surname>Nisan-Nelson</surname>
            ,
            <given-names>P.</given-names>
          </string-name>
          :
          <article-title>Technology Integration: A Case For Professional Development</article-title>
          .
          <source>Journal of Technology and Teacher Education</source>
          <volume>9</volume>
          (
          <issue>1</issue>
          ),
          <fpage>83</fpage>
          -
          <lpage>103</lpage>
          (
          <year>2001</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref29">
        <mixed-citation>
          29.
          <article-title>Organisation for Economic Co-operation and Development-OECD. Students, computers and learning: Making the connection</article-title>
          .
          <source>PISA</source>
          , OECD Publishing, Paris (
          <year>2015</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref30">
        <mixed-citation>
          30.
          <string-name>
            <surname>Osborne</surname>
            , J., Collins,
            <given-names>S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Ratcliffe</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Millar</surname>
            ,
            <given-names>R.</given-names>
          </string-name>
          , &amp;
          <string-name>
            <surname>Duschl</surname>
            ,
            <given-names>R.</given-names>
          </string-name>
          :
          <article-title>What" ideas-aboutscience" should be taught in school science? A Delphi study of the expert community</article-title>
          .
          <source>Journal of research in science teaching 40(7)</source>
          ,
          <fpage>692</fpage>
          -
          <lpage>720</lpage>
          (
          <year>2003</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref31">
        <mixed-citation>
          31.
          <string-name>
            <surname>Osler</surname>
            ,
            <given-names>E.J.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Hollowell</surname>
            ,
            <given-names>G.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Palmer</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          :
          <article-title>Technology engineering science instruction in the information age: Integrating instructional technology in K12 education. I-managers</article-title>
          <source>Journal on School Educational Technology</source>
          <volume>4</volume>
          (
          <issue>1</issue>
          ),
          <fpage>12</fpage>
          -
          <lpage>17</lpage>
          (
          <year>2008</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref32">
        <mixed-citation>
          32.
          <string-name>
            <surname>Papert</surname>
            ,
            <given-names>S. Situating</given-names>
          </string-name>
          <string-name>
            <surname>Constructionism</surname>
          </string-name>
          . In I. Harel &amp; S. Papert (Eds.) Constructionism Norwood, NJ, Ablex Publishing (
          <year>1991</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref33">
        <mixed-citation>
          33.
          <string-name>
            <surname>Robertson</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          and
          <string-name>
            <surname>Nicholson</surname>
            ,
            <given-names>K.</given-names>
          </string-name>
          :
          <article-title>Adventure Author: a learning environment to support creative design</article-title>
          .
          <source>In Proceedings of the 6th international conference on Interaction design and children (IDC '07)</source>
          ,
          <fpage>37</fpage>
          -
          <lpage>44</lpage>
          . New York, NY: ACM (
          <year>2007</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref34">
        <mixed-citation>
          34.
          <string-name>
            <surname>Seale</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Gibson</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Haynes</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          , &amp;
          <string-name>
            <surname>Potter</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          :
          <article-title>Power and resistance: Reflections on the rhetoric and reality of using participatory methods to promote student voice and engagement in higher education</article-title>
          .
          <source>Journal of Further and Higher Education</source>
          <volume>39</volume>
          (
          <issue>4</issue>
          ),
          <fpage>534</fpage>
          -
          <lpage>552</lpage>
          (
          <year>2015</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref35">
        <mixed-citation>
          35.
          <string-name>
            <surname>Siau</surname>
            ,
            <given-names>K.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Fui-hoon nah</surname>
          </string-name>
          , F.,
          <string-name>
            <surname>Mennecke</surname>
            ,
            <given-names>E.B.</given-names>
          </string-name>
          and
          <string-name>
            <surname>Schiller</surname>
            ,
            <given-names>S.Z.:</given-names>
          </string-name>
          <article-title>Co-creation and collaboration in a virtual world: a 3d visualization design project in second life</article-title>
          .
          <source>Journal of Database Management</source>
          <volume>21</volume>
          (
          <issue>4</issue>
          ),
          <fpage>1</fpage>
          -
          <lpage>13</lpage>
          (
          <year>2010</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref36">
        <mixed-citation>
          36.
          <string-name>
            <surname>Tlhoaele</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Suhre</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          , &amp;
          <string-name>
            <surname>Hofman</surname>
            ,
            <given-names>H.</given-names>
          </string-name>
          :
          <article-title>Using technology-enhanced, cooperative, groupproject learning for student comprehension and academic performance</article-title>
          .
          <source>European Journal of Engineering Education</source>
          <volume>41</volume>
          (
          <issue>3</issue>
          ),
          <fpage>263</fpage>
          -
          <lpage>278</lpage>
          (
          <year>2015</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref37">
        <mixed-citation>
          37.
          <string-name>
            <surname>Wing</surname>
            ,
            <given-names>J. M.:</given-names>
          </string-name>
          <article-title>Computational thinking</article-title>
          .
          <source>Communications of the ACM</source>
          <volume>49</volume>
          (
          <issue>3</issue>
          ),
          <fpage>33</fpage>
          -
          <lpage>35</lpage>
          (
          <year>2006</year>
          ).
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>