<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta />
    <article-meta>
      <title-group>
        <article-title>Enhancing Python Programming Education through Dynamic Code Execution and Assessment Methods</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Nurtugan Azatbekuly</string-name>
          <email>nurtugang17@gmail.com</email>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Bazargul Matkerim</string-name>
          <email>bazargul.matkerim@gmail.com</email>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          <institution>Al-Farabi Kazakh National University</institution>
          ,
          <addr-line>71 Al-Farabi Avenue, Almaty, 050040</addr-line>
          ,
          <country country="KZ">Kazakhstan</country>
        </aff>
      </contrib-group>
      <abstract>
        <p>This study introduces a web-based educational platform with a novel approach to Python programming skill development for students. It emphasizes task-based learning, real-time code checking, and dynamic assessments, all of which are aimed at enhancing problem-solving abilities and coding practices. The primary objective of this platform is to advance education, particularly in the computer science field. It seeks to inspire a deeper understanding of Python programming, creative problem-solving, and nurture proficient programmers through hands-on learning experiences. The platform employs dynamic code execution, a global namespace, and regular expressions for real-time code checking. The platform's unique features such as real-time feedback, dynamic assessments, and visualizations contribute to a more effective and engaging learning experience. It fosters students' problem-solving skills, adherence to coding best practices, and an overall improvement in Python programming. This innovative educational tool offers a comprehensive approach to Python programming skill development, enhances students' problem-solving abilities, and encourages adherence to best coding practices. It aims to inspire a deeper understanding of Python programming, creative problem solving, and nurture proficient programmers, contributing to the advancement of education in the field of computer science. This study provides a detailed account of the platform's architecture, pedagogical foundations, and results of usage cases, showcasing its potential as an innovative educational resource.</p>
      </abstract>
      <kwd-group>
        <kwd>1 Programming education</kwd>
        <kwd>Python</kwd>
      </kwd-group>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>1. Introduction</title>
      <p>
        Programming education is a dynamic field that constantly seeks innovative approaches to
empower students with the skills and knowledge required to excel in software development. In
this era of rapid technological advancement, Python has emerged as a cornerstone in computer
science and software engineering programs owing to its simplicity and versatility. Awareness of
the key design principles of educational applications is crucial. In [
        <xref ref-type="bibr" rid="ref1">1</xref>
        ] and [
        <xref ref-type="bibr" rid="ref2">2</xref>
        ], it was observed that
educational applications should be engaged and stimulated in the context of the educational
process. Additionally, they should provide opportunities for practice and feedback. In the
development of the work done in these articles [
        <xref ref-type="bibr" rid="ref3 ref4 ref5">3-5</xref>
        ], peculiar systems were used for effective
programming training. It is also worth noting this article [
        <xref ref-type="bibr" rid="ref6">6</xref>
        ], in which the system used a visual
programming language that allowed students to solve programming problems by dragging blocks
of code. To facilitate an engaging and effective learning environment, we introduced a novel
webbased platform that transcends traditional instructional methods and offers a unique, interactive
approach to Python programming education.
      </p>
      <p>The appeal of Python lies in its readability, ease of use, and wide-ranging applicability.
However, mastering language and its associated concepts necessitates a pedagogical framework
that challenges students while providing invaluable feedback to foster their growth. Our platform
is designed to address this educational imperative by affording students the opportunity to tackle
Python programming tasks in a manner that not only builds their coding acumen but also instills
a deeper understanding of the nuances of the language.
0009-0007-5843-8995 (N. Azatbekuly); 0000-0002-5336-687X (B. Matkerim)
© 2023 Copyright for this paper by its authors.</p>
      <p>Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).</p>
      <p>This study elucidates the architectural underpinnings of our innovative platform, highlighting
the pivotal role played by its dynamic code-checking mechanisms. While traditional pedagogical
methods rely on static assignments, our platform generates randomized variables for each task,
thus offering a distinctive, ever-evolving experience for each student. This approach encourages
creative problem-solving and critical thinking, as students grapple with the multifaceted
challenges presented by each task.</p>
      <p>Central to the platform's educational ethos is its use of dynamic code assessment techniques,
such as the restriction of specific keywords, enforcement of essential programming constructs,
and real-time code validation. These mechanisms ensure not only code correctness, but also
adherence to best coding practices, thereby promoting the cultivation of proficient and
responsible programmers.</p>
      <p>
        To further enhance the learning experience, the platform provides students with insightful
visualizations, including charts and performance metrics, enabling them to systematically
monitor their progress. Assessing user progress in learning is crucial to enable the identification
of strengths and weaknesses and to guide appropriately oriented learning in educational
applications, as highlighted by a referenced paper [
        <xref ref-type="bibr" rid="ref7">7</xref>
        ]. These visual aids offer a comprehensive
view of their task-solving proficiency, empower them to gain insight into their performance by
difficulty level and theme and motivate a continuous journey of improvement. In educational
studies of these works [
        <xref ref-type="bibr" rid="ref10 ref8 ref9">8-10</xref>
        ], it was proven that the greatest effect was achieved for applications
focused on specific learning skills. The conclusions of [
        <xref ref-type="bibr" rid="ref11">11</xref>
        ] suggest that the possibility of using a
flexible software development process for teaching computer programming should be
considered.
      </p>
      <p>This study seeks to elucidate the novel approach adopted in our educational platform and its
implications for Python programming education. We aim to contribute to the evolution of
programming education by harnessing technology to inspire students, foster a deeper
understanding of Python, encourage creative problem-solving, and ultimately shape the next
generation of proficient programmers. The subsequent sections delve into the platform's
architecture, pedagogical foundations, and empirical evidence of its effectiveness, thus providing
a comprehensive account of this pioneering educational tool.</p>
    </sec>
    <sec id="sec-2">
      <title>2. Methods and solutions</title>
      <p>In our ongoing commitment to advancing Python programming education, we designed and
developed an adaptive web application that represents a pioneering addition to the educational
technology landscape. This innovative web application leverages Python’s inherent capabilities
to offer a dynamic and interactive platform for users to engage with programming concepts in a
hands-on and immersive manner. At its core, our web application harnesses the power of
Python's built-in functions to facilitate a dynamic code execution. This feature is central to our
mission of creating a learning environment in which users can not only grasp the theoretical
aspects of programming but also apply this knowledge in practical scenarios. By executing the
code in real-time, our platform enables learners to experiment, troubleshoot, and refine their
coding skills, thereby bridging the gap between theory and practice. To gauge the effectiveness of
our web application in enhancing user learning, we incorporated various assessment tools and
diagrams to evaluate user progress and performance. These diagrams provide visual
representations of learning trajectories, allowing educators and learners to track the evolution of
their coding proficiency over time. This data-driven approach offers valuable insights into the
impact of our platform on learning improvements.</p>
      <p>The section concentrates on analyzing the problem generation system, testing it against
different regular expressions, and comparing it with the correct outcome. Thus, it is crucial to
present a database structure in the context of web app development.</p>
      <sec id="sec-2-1">
        <title>2.1 Task preparation</title>
        <p>In this section, we present a detailed exploration of the task variable generation process in the
context of a web application designed to facilitate educational endeavors. Systematic generation
of random data plays a pivotal role in enhancing the educational experience offered by platforms.
The dynamic nature of this process ensures that with each visit to the task page, users encounter
a unique and tailored set of variables, specifically engineered to correspond to the characteristics
and demands of individual tasks.</p>
        <p>The linchpin of this process resides within the implemented function for generating variables,
which was meticulously crafted within the Python file in the Django project. This function
operates as the crux of the data-generation mechanism, orchestrating a series of conditional
expressions informed by validated task IDs. This sophisticated approach ensures the creation of
data that is distinct and attuned to the intricacies of each task. The versatility of this data
generation method is manifested through a range of techniques, including random number
generation within customizable bounds. For selected tasks, the configurability of these bounds
allows precise data tailoring.</p>
        <p>Moreover, the data-generation mechanism encompasses the acquisition of words, sentences,
and lists. To achieve this, the function orchestrates a harmonious interplay of methods, such as
parsing data from designated websites to generate words, harnessing the capabilities of the
essential_generator library to formulate sentences, and orchestrating complex algorithms to
synthesize random lists. The composite result is a comprehensive dataset that exhibits diversity
and intricacy.</p>
        <p>Integral to the operation of the platform is the creation and management of a dictionary that
encapsulates variables and their corresponding values, constituting a vital prerequisite for the
functioning of the `eval()` function. This dictionary structure was meticulously constructed with
the names of variables meticulously extracted from the database entries associated with each
task. Within the `eval()` function, these variable names assume the role of keys within the
dictionary, while the corresponding values are drawn from the data previously generated using
the diverse array of data generation techniques. This systematic organization is pivotal, as the
`eval()` function's `globals` parameter exclusively accommodates a dictionary format, thus
necessitating this structured arrangement for the successful evaluation of user-submitted code.</p>
        <p>In summary, the random variable generation process is a multifaceted and meticulously
orchestrated component of the platform, contributing to its ability to dynamically tailor tasks and
provide users with a unique and enriching educational experience. This process underscores the
platform's commitment to fostering creative problem-solving and critical thinking by presenting
users with an ever-evolving and diverse set of variables, thus facilitating an environment in which
learning thrives through active engagement.</p>
      </sec>
      <sec id="sec-2-2">
        <title>2.2 Dynamic code execution and Runtime Environment</title>
        <p>In the subsequent stages of our operational framework, we delve into the intricate process of
code execution. Specifically, we examine the execution of two distinct types of code: code
retrieved from the database, which is considered "correct" code, and user-submitted code. This
phase is integral to our platform's educational objectives as it allows for a detailed comparison to
determine the correctness of the user's code in relation to the predetermined standards
embodied by the "correct" code.</p>
        <p>The execution of "correct" code, sourced from our database, is a multifaceted procedure. This
code undergoes a series of manipulations and evaluations that are pivotal for assessing
conformity to task requirements. The process begins with the importation of essential application
models, with the "TAnswer" table taking the center stage. This importation lays the groundwork
for subsequent variable declarations and manipulation.</p>
        <p>Variable declarations are a crucial aspect of this process and are managed through the
utilization of the “globals(). Update the ()” method. This method integrates the variables stored
within a specialized dictionary known as "vardict," meticulously constructed to encompass
variables and their associated values. Careful orchestration of these variables is central to the
execution of the code.</p>
        <p>Moreover, the code specific to each task is retrieved from the database through the "TAnswer"
table. This code serves as a benchmark for the system, representing the standard of correctness
against which user-generated code must be measured. The retrieval process was facilitated
seamlessly, and the imported code was scrutinized to ensure compliance with predefined
standards and task-specific requirements.</p>
        <p>Concurrently, user-submitted code undergoes a different form of execution. Unlike the
"correct" code, which undergoes a series of meticulous manipulations, user-generated code is
executed within the confines of the “eval()” method. This approach ensures efficient execution of
user-submitted code and allows for a streamlined comparison against the benchmark "correct"
code.</p>
        <p>Furthermore, our system incorporates an additional layer of scrutiny using regular
expressions (regex) for keyword validation. Certain tasks require the inclusion or exclusion of
specific keywords to determine the correctness of the code. This facet of our platform facilitates
a granular examination of the user code to assess its adherence to predefined keyword
requirements by adding an additional layer of precision to the assessment process.</p>
        <p>In summary, the execution and comparison of codes within our platform is a multifaceted
process that is meticulously designed to ensure educational integrity. This approach allows for a
systematic evaluation of user-generated code against predefined benchmarks, offering users a
constructive and educational experience, while maintaining the highest standards of quality and
correctness.</p>
      </sec>
    </sec>
    <sec id="sec-3">
      <title>3. Development results</title>
      <p>An interactive web application was developed on Django to teach programming with its own
unique implementation of the Python code interpreter and with its own flexible task validation
system. The creation of an interactive web application meticulously developed on the Django
framework marks a significant milestone in the realm of programming education. This innovative
platform boasts its own distinctive implementation of the Python code interpreter, setting it apart
as a pioneering tool for learners and educators. In addition, the application features a highly
adaptable and flexible task validation system, adding a layer of customization that caters to the
individualized learning needs of users. This web application is a testament to our commitment to
redefine the boundaries of programming education. Its unique Python code interpreter offers
users a dynamic and immersive learning experience where they can experiment, iterate, and
explore coding concepts in real time. The flexibility of the task validation system empowers
educators to craft tasks that align with their pedagogical objectives while ensuring a rigorous and
standards-based approach for assessing user progress.</p>
      <p>In this pivotal screenshot, we unveiled a programming task that showcases the ingenuity of
our web application. The task was generated using a dynamic array of random variables, each
carefully configured to provide a unique and diverse set of challenges for learners.</p>
      <p>The structure of this programming task is a testament to the versatility and adaptability of the
proposed platform. The random variables injected into the task prompted users to apply their
coding skills to a real-world scenario, mirroring the unpredictability of coding in professional
settings.</p>
      <p>This task is a testament to our commitment to offer learners a broad and challenging
educational experience. This finding reinforces the importance of practicality and adaptability in
programming education. The ability to generate dynamic tasks with random variables ensures
that users are continually engaged and equipped to address the ever-evolving landscape of coding
challenges. This screenshot underscores the dynamic nature of our web application and its
capacity to prepare learners for real-world programming complexities.</p>
      <p>In Figure 5, we offer a glimpse of the critical aspect of our web application's assessment
process. The system meticulously checks the task answers to ensure that they meet specific
keyword requirements.</p>
      <p>In this instance, the assessment mechanism identified that not all the required keywords have
been used in the submitted answer. This meticulous examination goes beyond the surface and
dives deeply into the completeness of the user's solution.</p>
      <p>It is a testament to our commitment to maintaining educational standards and ensuring that
learners are not only provided with accurate feedback but also guided towards a comprehensive
understanding of coding concepts. The screenshot in Figure 5 underscores the precision and
depth of our assessment mechanisms, allowing educators and learners to fine-tune their
approach to tasks and achieve a more complete grasp of programming principles.</p>
      <p>One of the main components of our dynamic graphs is their division into programming topics.
This allows students to easily assess their skills in each specific area and focus their efforts on
improving their weaknesses. For example, students can see how they are produced when
working with arrays compared to when working with conditional expressions.</p>
      <p>One of the key components of our methodology for improving programming learning is
dynamic graphs, which provide students with the opportunity to observe and analyze their
progress in solving programming problems. These graphs are divided according to various
criteria, such as topics and difficulty levels, and serve as valuable tools to stimulate active learning
and increase student motivation.</p>
      <p>These dynamic graphs not only provide a visual representation of student progress but also
allow students to make informed decisions about where they need to improve their learning
efforts. They also motivated students by providing visual confirmation that they really improved
their programming skills. Thus, our methods of visualizing student progress, divided by topic and
complexity, play an important role in enriching students' educational experiences and contribute
to more effective programming learning.</p>
      <p>The analysis of graphs and diagrams presented in this study sheds light on the importance of
an open and systematic view of the results of solved tasks in the field of programming training.
These visual aids enrich our understanding of the learning processes and confirm the
effectiveness of the developed methods and tools.</p>
      <p>The graphs and diagrams presented in this paper reflect the dynamics of learning and
students’ level of success in solving various tasks. They are an integral part of educational
analytics and provide valuable insight into the learning process.</p>
      <p>Visual representations of data allow researchers and teachers to track changes in student
skills over time. Graphs allow us to observe how students improve their skills, depending on the
complexity of the tasks and topics. These data make it possible to optimize the educational
process and identify successful teaching methods.</p>
      <p>In addition, graphs and diagrams emphasize the importance of an open view of the results.
They provide indisputable evidence on how learning changes students’ level of knowledge and
skills. This visual data also serves as a feedback tool, allowing students to evaluate their progress
and learn from mistakes.</p>
      <p>Thus, the graphs and diagrams in this paper not only represent statistical data but also open
an important window into the educational process. They give shape to abstract concepts and
allow informed decisions to be made based on observed facts. These visual representations have
become reliable companions in the field of programming education, enriching the experience of
students and researchers.</p>
    </sec>
    <sec id="sec-4">
      <title>4. Use cases and specific outcomes</title>
      <p>To evaluate the effectiveness of our web application platform for learning Python, a survey
was conducted among groups of participants. The group of participants is mainly represented by
students of the first and second year of bachelor's degree who have little experience in
programming but want to hone their practical programming skills. The purpose of the survey was
to evaluate key characteristics, such as simplicity, content quality and visualization, using a
percentage scale that allows participants to rate each dimension on a scale from 0 to 100. 90
people with different levels of programming experience took part in the survey. Participants were
asked to evaluate these aspects based on their experience with the web application for 1-2 days.
The key characteristics assessed during the survey included the next:
• Simplicity of the user interface. The user interface was evaluated for its simplicity,
focusing on ease of navigation and the intuitiveness of design to ensure a user-friendly
experience. The simplicity of the interface and its convenience allows the student to focus on
learning and solving programming problems.
• The quality of content. The quality of practical tasks that were presented in the
educational web application is evaluated, how much they reveal programming concepts in the
opinion of students and help to develop in the study of programming.
• Interactivity. Interactivity in a web application is very important, there are functions for
searching for tasks, saving to bookmarks, logging in to the system. The quality of all this is
evaluated in this criterion.
• Visualization. Visualization is important to facilitate learning by making abstract concepts
more tangible. The visually attractive interface not only creates a pleasant learning
environment but also helps to concentrate on solving programming tasks.
• Feedback and assessment. The mechanisms of the web application, such as sending the
written code for verification, offering the correct answer in case of a large number of attempts
spent, the comments section where students can ask questions is evaluated according to this
criterion.
• Quantity of content. The quantity of content is pivotal enough in providing a rich and
dynamic learning experience, ensuring that the educational web application remains robust,
adaptable, and capable of meeting the diverse needs of its user base.
• Performance of the platform. Participants provided feedback on the platform's overall
performance, including its speed, responsiveness, and reliability in delivering content and
interactive elements. The indicator of this criterion shows the effectiveness of the applied
methods of code generation and dynamic code execution.
• Convenience of progress checking. The effectiveness of progress tracking and reporting
features will be examined to ensure users can easily monitor their advancement, fostering
motivation and a sense of accomplishment.</p>
      <p>The survey results were analyzed to identify trends and patterns. Visual representations in
the form of a diagram were used to present quantitative results. In addition, with the help of open
questions, high-quality feedback was collected, allowing to receive detailed information from the
participants. The average values of the survey results are shown in Figure 9.</p>
      <p>Specific use cases were obtained from the survey data, demonstrating cases where the
platform was different, or areas in which improvements were proposed. During the survey,
training results were obtained, shedding light on how users perceive the strengths and
weaknesses of the platform. Correlations and patterns in the data were studied in connection
with the overall goals of the educational platform. The students highly appreciated the simplicity,
interactivity, speed and checking of progress, which confirms the effectiveness of the proposed
methodology for developing a web application.</p>
      <p>Survey results were reviewed, which provided valuable information for future improvements.
Among them, it is worth noting the relatively low result in the amount of content. This
information is very important for improving the developed platform of the educational web
application, bringing user ratings in line with the goals of the educational initiative.</p>
    </sec>
    <sec id="sec-5">
      <title>5. Conclusions</title>
      <p>In this paper, we embarked on a journey to address a prevalent challenge in programming
education: the need for an interactive and effective learning tool that bridges the gap between
theory and practice. Our research introduces a solution in the form of an innovative web
application, a digital ecosystem that not only addresses the shortcomings of existing educational
apps, but also augments the learning process for programming enthusiasts.</p>
      <p>As demonstrated in this study, our methodology for task correctness checking offers a robust
mechanism for assessing the quality and accuracy of user-generated code. By implementing
regular expressions (regex) to validate keywords, we ensure that the user solutions adhere to
predefined requirements. The integration of these methods provides a comprehensive evaluation
system that is instrumental in maintaining the educational standards of our web applications.</p>
      <p>The survey results provide a comprehensive view of the user experience, providing specific
information about the strengths of the platform, which confirms the effectiveness of the proposed
methodology and design of the web application. Also giving information about areas for
development.</p>
      <p>As we bring this paper closer, we emphasize the significance of our work in reshaping the
landscape of educational technology. Our web application is a testament to the fusion of
technology and pedagogy and offers a dynamic, interactive, and adaptive learning environment
for coding enthusiasts. It is our hope that this contribution will not only bridge the gap in
programming education but also inspire further innovation in the field of educational apps,
ultimately empowering learners on their journey to programming excellence.</p>
    </sec>
    <sec id="sec-6">
      <title>6. References</title>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          [1]
          <string-name>
            <given-names>J.</given-names>
            <surname>Zosh</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S.</given-names>
            <surname>Lytle</surname>
          </string-name>
          ,
          <string-name>
            <given-names>R.</given-names>
            <surname>Golinkoff</surname>
          </string-name>
          , and
          <string-name>
            <given-names>K.</given-names>
            <surname>Hirsh-Pasek</surname>
          </string-name>
          ,
          <article-title>“Putting the Education Back in Educational Apps: How Content and Context Interact to Promote Learning</article-title>
          ,”
          <year>2017</year>
          , pp.
          <fpage>259</fpage>
          -
          <lpage>282</lpage>
          . doi:
          <volume>10</volume>
          .1007/978-3-
          <fpage>319</fpage>
          -45102-2_
          <fpage>17</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          [2]
          <string-name>
            <given-names>C.</given-names>
            <surname>Fernández</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M. A.</given-names>
            <surname>Vicente</surname>
          </string-name>
          ,
          <string-name>
            <surname>M. M. Galotto</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          <string-name>
            <surname>Martinez-Rach</surname>
          </string-name>
          ,
          <article-title>and</article-title>
          <string-name>
            <given-names>A.</given-names>
            <surname>Pomares</surname>
          </string-name>
          , “
          <article-title>Improving student engagement on programming using app development with Android devices</article-title>
          ,
          <source>” Comput. Appl. Eng. Educ.</source>
          , vol.
          <volume>25</volume>
          , no.
          <issue>5</issue>
          , pp.
          <fpage>659</fpage>
          -
          <lpage>668</lpage>
          ,
          <year>2017</year>
          , doi: 10.1002/cae.21827.
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          [3]
          <string-name>
            <given-names>D.</given-names>
            <surname>Zhao</surname>
          </string-name>
          ,
          <string-name>
            <given-names>C. H.</given-names>
            <surname>Muntean</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A. E.</given-names>
            <surname>Chis</surname>
          </string-name>
          , G. Rozinaj, and G.
          <string-name>
            <surname>-M. Muntean</surname>
          </string-name>
          , “
          <article-title>Game-Based Learning: Enhancing Student Experience, Knowledge Gain, and Usability in Higher Education Programming Courses,”</article-title>
          <source>IEEE Trans. Educ.</source>
          , vol.
          <volume>65</volume>
          , no.
          <issue>4</issue>
          , pp.
          <fpage>502</fpage>
          -
          <lpage>513</lpage>
          , Nov.
          <year>2022</year>
          , doi: 10.1109/TE.
          <year>2021</year>
          .
          <volume>3136914</volume>
          .
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          [4]
          <string-name>
            <given-names>Z.</given-names>
            <surname>Zhan</surname>
          </string-name>
          ,
          <string-name>
            <given-names>L.</given-names>
            <surname>He</surname>
          </string-name>
          ,
          <string-name>
            <given-names>Y.</given-names>
            <surname>Tong</surname>
          </string-name>
          ,
          <string-name>
            <given-names>X.</given-names>
            <surname>Liang</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S.</given-names>
            <surname>Guo</surname>
          </string-name>
          , and
          <string-name>
            <given-names>X.</given-names>
            <surname>Lan</surname>
          </string-name>
          , “
          <article-title>The effectiveness of gamification in programming education: Evidence from a meta-analysis,”</article-title>
          <string-name>
            <surname>Comput. Educ. Artif. Intell.</surname>
          </string-name>
          , vol.
          <volume>3</volume>
          , p.
          <fpage>100096</fpage>
          ,
          <string-name>
            <surname>Jan</surname>
          </string-name>
          .
          <year>2022</year>
          , doi: 10.1016/j.caeai.
          <year>2022</year>
          .
          <volume>100096</volume>
          .
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          [5]
          <string-name>
            <given-names>S.</given-names>
            <surname>Azmi</surname>
          </string-name>
          ,
          <string-name>
            <given-names>N. A.</given-names>
            <surname>Iahad</surname>
          </string-name>
          , and
          <string-name>
            <given-names>N.</given-names>
            <surname>Ahmad</surname>
          </string-name>
          , “
          <article-title>Attracting students' engagement in programming courses with gamification,” in 2016 IEEE Conference on e-Learning, e-Management and eServices (IC3e</article-title>
          ), Oct.
          <year>2016</year>
          , pp.
          <fpage>112</fpage>
          -
          <lpage>115</lpage>
          . doi:
          <volume>10</volume>
          .1109/IC3e.
          <year>2016</year>
          .
          <volume>8009050</volume>
          .
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          [6]
          <string-name>
            <given-names>O.</given-names>
            <surname>Chernikova</surname>
          </string-name>
          ,
          <string-name>
            <given-names>N.</given-names>
            <surname>Heitzmann</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Stadler</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D.</given-names>
            <surname>Holzberger</surname>
          </string-name>
          ,
          <string-name>
            <given-names>T.</given-names>
            <surname>Seidel</surname>
          </string-name>
          , and
          <string-name>
            <given-names>F.</given-names>
            <surname>Fischer</surname>
          </string-name>
          , “
          <article-title>SimulationBased Learning in Higher Education: A Meta-</article-title>
          <string-name>
            <surname>Analysis</surname>
          </string-name>
          ,
          <source>” Rev. Educ. Res.</source>
          , vol.
          <volume>90</volume>
          , no.
          <issue>4</issue>
          , pp.
          <fpage>499</fpage>
          -
          <lpage>541</lpage>
          , Aug.
          <year>2020</year>
          , doi: 10.3102/0034654320933544.
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          [7]
          <string-name>
            <given-names>A.</given-names>
            <surname>Nguyen</surname>
          </string-name>
          ,
          <string-name>
            <given-names>T.</given-names>
            <surname>Tuunanen</surname>
          </string-name>
          ,
          <string-name>
            <given-names>L.</given-names>
            <surname>Gardner</surname>
          </string-name>
          , and
          <string-name>
            <given-names>D.</given-names>
            <surname>Sheridan</surname>
          </string-name>
          , “
          <article-title>Design principles for learning analytics information systems in higher education,”</article-title>
          <string-name>
            <given-names>Eur. J.</given-names>
            <surname>Inf</surname>
          </string-name>
          . Syst., vol.
          <volume>30</volume>
          , no.
          <issue>5</issue>
          , pp.
          <fpage>541</fpage>
          -
          <lpage>568</lpage>
          , Sep.
          <year>2021</year>
          , doi: 10.1080/0960085X.
          <year>2020</year>
          .
          <volume>1816144</volume>
          .
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          [8]
          <string-name>
            <given-names>R.</given-names>
            <surname>Suzuki</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J.</given-names>
            <surname>Kato</surname>
          </string-name>
          , and
          <string-name>
            <given-names>K.</given-names>
            <surname>Yatani</surname>
          </string-name>
          , “
          <article-title>ClassCode: An Interactive Teaching and Learning Environment for Programming Education in Classrooms</article-title>
          .” arXiv, Jan.
          <volume>22</volume>
          ,
          <year>2020</year>
          . doi:
          <volume>10</volume>
          .48550/arXiv.
          <year>2001</year>
          .
          <volume>08194</volume>
          .
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          [9]
          <string-name>
            <given-names>J. C.</given-names>
            <surname>Farah</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S.</given-names>
            <surname>Ingram</surname>
          </string-name>
          , and
          <string-name>
            <given-names>D.</given-names>
            <surname>Gillet</surname>
          </string-name>
          ,
          <article-title>Supporting Developers in Creating Web Apps for Education via an App Development Framework</article-title>
          .
          <year>2022</year>
          . doi:
          <volume>10</volume>
          .4995/HEAd22.
          <year>2022</year>
          .
          <volume>14587</volume>
          .
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          [10]
          <string-name>
            <given-names>S.</given-names>
            <surname>Doi</surname>
          </string-name>
          and Y. Imai, “
          <article-title>Development of Web Application to Support Program Learning of Python and Ruby with Error Accumulation and Analysis Facility,”</article-title>
          <source>Fourth Int. Conf. Electron. Softw. Sci. ICESS2018</source>
          , Jan.
          <year>2018</year>
          , Accessed: Oct.
          <volume>29</volume>
          ,
          <year>2023</year>
          . [Online]. Available: https://www.academia.edu/38133220/Development_of_Web_Application_to_Support_
          <article-title>Pro gram_Learning_of_Python_and_Ruby_with_Error_Accumulation_and_Analysis_Facility</article-title>
        </mixed-citation>
      </ref>
      <ref id="ref11">
        <mixed-citation>
          [11]
          <string-name>
            <given-names>B.</given-names>
            <surname>Isong</surname>
          </string-name>
          , “
          <article-title>A Methodology for Teaching Computer Programming: first year students' perspective,”</article-title>
          <string-name>
            <given-names>Int. J.</given-names>
            <surname>Mod</surname>
          </string-name>
          .
          <source>Educ. Comput. Sci.</source>
          , vol.
          <volume>6</volume>
          , pp.
          <fpage>15</fpage>
          -
          <lpage>21</lpage>
          , Sep.
          <year>2014</year>
          , doi: 10.5815/ijmecs.
          <year>2014</year>
          .
          <volume>09</volume>
          .03.
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>