<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta>
      <journal-title-group>
        <journal-title>ORCID:</journal-title>
      </journal-title-group>
    </journal-meta>
    <article-meta>
      <title-group>
        <article-title>Improvements of Existing Autograding System for Interactive Learning Functional Programming in Java</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Nikola Dimitrijević</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Nemanja Zdravkovic</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Milena Bogdanović</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          <institution>Faculty of Information Technolgy, Belgrade Metropolitan University</institution>
          ,
          <addr-line>Tadeuša Košćuška 63, 11000 Belgrade</addr-line>
          ,
          <country country="RS">Serbia</country>
        </aff>
      </contrib-group>
      <pub-date>
        <year>2023</year>
      </pub-date>
      <volume>000</volume>
      <fpage>0</fpage>
      <lpage>0002</lpage>
      <abstract>
        <p>In Computer Science, functional programming is a programming paradigm which uses functions to describe the logic of what the program has to achieve. These are mathematical function-like constructs (e.g., lambda functions) that are evaluated in expression contexts. Google Trends currently ranks functional programming as more popular than object-oriented programming. Also, functional programming is very important in existing popular platforms like Hadoop and Apache Spark. Many students may ask what are the benefits and why should they use functional programming in Java. From its beginnings, best practices in Java have encouraged object-oriented programming, which is an extension of procedural programming type of imperative programming. Since version 8 Java introduced Java developers to functional programming with lambda expressions, method references and predefined functional interfaces. This Java release effectively notified developers that it's no longer sufficient to think about Java programming only from the imperative, object-oriented perspective. The first purpose of this paper is to present the benefits of adopting a functional paradigm to students which worked with the older versions of Java (6, 7 or earlier) and to the ones working with legacy code or using newer versions but without switching to functional paradigm. Secondly, we present some improvements to existing interactive system for online learning of functional programming in programming language Java, supporting automatic grading of given assessments. These improvements are aimed at students who already have basic knowledge of Java programming languages, and want to switch to functional programming. Finally, we analyzed our solution by comparing its content to multiple commercially available solutions and point out the advantages and disadvantages of each.</p>
      </abstract>
      <kwd-group>
        <kwd>1 Functional programming</kwd>
        <kwd>eLearning</kwd>
        <kwd>autograding</kwd>
        <kwd>Big Data</kwd>
      </kwd-group>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>1. Introduction to Functional Programming</title>
      <p>Functional programming, a paradigm at the heart of computer science, has seen a resurgence in
popularity and practical application in recent years. Unlike imperative programming, which focuses on
how a program operates (through statements and instructions), functional programming emphasizes
what the program should accomplish, using pure functions and immutable data.</p>
      <p>
        At its core, functional programming is about building software by composing pure functions. A pure
function is one that, given the same input, will always return the same output and does not cause any
observable side effects. This concept is not only central to functional programming but also to the
mathematical notion of a function, making the paradigm highly expressive and predictable [
        <xref ref-type="bibr" rid="ref1">1</xref>
        ].
      </p>
      <p>
        One of the key principles of functional programming is immutability. Immutability in this context
refers to the idea that once a data structure is created, it cannot be altered. This principle reduces
complexity and improves the predictability of code. Side-effect-free functions, another cornerstone of
functional programming, contribute to the reliability and maintainability of software, as they do not
alter the state outside their scope [
        <xref ref-type="bibr" rid="ref2">2</xref>
        ].
      </p>
      <p>
        In recent years, functional programming has gained traction in the software development industry,
particularly in the development of large-scale, concurrent applications. Languages like Haskell and
Scala, and features in Java 8 such as lambda expressions, have brought functional programming
concepts to the forefront of software design [
        <xref ref-type="bibr" rid="ref3">3</xref>
        ].
      </p>
      <p>
        Functional programming's emphasis on immutability and stateless functions makes it particularly
well-suited for concurrent and distributed systems. In these systems, managing state and dealing with
mutable data can lead to complex issues. Functional programming alleviates these challenges, making
it easier to write safe and efficient concurrent code [
        <xref ref-type="bibr" rid="ref4">4</xref>
        ]. The influence of functional programming
extends to big data processing and cloud computing. Frameworks like Apache Spark and Hadoop
MapReduce, which are fundamental in the processing of large datasets, employ functional concepts
such as higher-order functions and lazy evaluation to efficiently process data across distributed systems
[
        <xref ref-type="bibr" rid="ref5 ref6">5, 6</xref>
        ]. Functional programming offers a different approach to software development, focusing on pure
functions and immutable data. Its principles are increasingly being integrated into modern software
practices, influencing the development of concurrent, distributed, and big data applications. As the
industry continues to evolve, the relevance and application of functional programming are likely to
expand further.
      </p>
    </sec>
    <sec id="sec-2">
      <title>2. Functional Programming and Big Data</title>
      <p>
        In the rapidly evolving landscape of software development and data processing, functional
programming has emerged as a key paradigm, not just in theory but also in practical applications. This
paradigm shift is particularly evident in the realm of big data and distributed computing, where
functional programming principles have become integral to the design and operation of major
platforms. A prime example of this trend is the significant role functional programming plays in existing
popular platforms like Hadoop and Apache Spark. These platforms, which are at the forefront of
handling large-scale data processing, leverage functional programming concepts to enhance efficiency,
scalability, and fault tolerance. This integration of functional programming into such platforms is not
incidental but stems from several core advantages that this programming style offers in dealing with
complex data processing tasks [
        <xref ref-type="bibr" rid="ref5 ref6">5, 6</xref>
        ].
2.1.
      </p>
    </sec>
    <sec id="sec-3">
      <title>Immutability and Parallelism</title>
      <p>
        Functional programming promotes the use of immutable data, which is crucial for efficient parallel
execution of operations. In the context of distributed systems like Hadoop and Apache Spark,
immutability allows data operations to be executed in parallel without the risk of state conflicts or the
need for complex locking mechanisms. This is particularly important in big data processing, where
parallelism plays a key role in achieving high performance [
        <xref ref-type="bibr" rid="ref6">6</xref>
        ].
2.2.
      </p>
    </sec>
    <sec id="sec-4">
      <title>MapReduce Paradigm</title>
      <p>
        Hadoop utilizes the MapReduce paradigm, deeply rooted in functional programming. MapReduce
consists of map and reduce functions, which are concepts taken from functional programming. The map
function processes input data and transforms it into intermediate results, while the reduce function
aggregates these results into the final output. This model is naturally suited to the functional style of
programming, where functions are first-class citizens and shared state is avoided [
        <xref ref-type="bibr" rid="ref7">7</xref>
        ].
2.3.
      </p>
    </sec>
    <sec id="sec-5">
      <title>Apache Spark and Functional API</title>
      <p>
        Apache Spark, often used for big data processing, provides an API that is strongly inspired by
functional programming, especially in its Scala interface. Spark uses concepts like RDD (Resilient
Distributed Dataset) and Datasets, which enable functional operations such as map, filter, reduce, and
aggregate on distributed data sets. These operations allow developers to write code that is both
expressive and efficient for parallel execution [
        <xref ref-type="bibr" rid="ref5">5</xref>
        ].
2.4.
      </p>
    </sec>
    <sec id="sec-6">
      <title>Scalability and Resilience</title>
      <p>
        The functional approach in Spark and Hadoop contributes to better scalability and resilience of the
systems. Side-effect-free functions are easier to test and understand, which is crucial in complex
distributed systems. Also, the functional approach facilitates the re-execution of operations in case of
failures, important for system resilience [
        <xref ref-type="bibr" rid="ref8">8</xref>
        ].
      </p>
      <p>Functional programming plays a key role in the design and implementation of modern distributed
data processing systems, such as Hadoop and Apache Spark. The principles of functional programming,
such as immutability, problem decomposition into functions, and avoidance of shared state, are essential
for the efficiency, scalability, and resilience of these systems.</p>
    </sec>
    <sec id="sec-7">
      <title>3. Curriculum for Functional Programming in Java 8 and Beyond</title>
      <p>
        This curriculum is meticulously crafted to guide learners through the intricacies of functional
programming in Java, particularly emphasizing the transformative features introduced in Java 8 and
further refined in subsequent versions. As Goetz [
        <xref ref-type="bibr" rid="ref9">9</xref>
        ] insightfully points out, the advent of Java 8 marked
a paradigm shift in Java programming, steering it towards the functional programming approach1.
Additionally, Schildt [
        <xref ref-type="bibr" rid="ref10">10</xref>
        ] provides a comprehensive beginner's guide to Java, which is an invaluable
resource for understanding the basics and nuances of Java programming. Warburton [
        <xref ref-type="bibr" rid="ref11">11</xref>
        ] further
complements this by offering a focused exploration of lambda expressions and functional programming
in Java 8, making it accessible to a broader audience.
      </p>
      <p>This curriculum is ideally suited for individuals who have a foundational understanding of Java and
are keen to explore the realm of functional programming, which has become increasingly relevant in
modern software development. In this journey, learners will delve into the essence of functional
programming as conceptualized in Java, starting from the basic principles and gradually progressing to
more advanced features and concepts. The curriculum not only covers the theoretical aspects but also
provides practical insights and applications, ensuring a holistic understanding of functional
programming in the context of Java's ongoing evolution.</p>
      <sec id="sec-7-1">
        <title>Lambda Expressions and Functional Interfaces</title>
      </sec>
      <sec id="sec-7-2">
        <title>Module 3</title>
      </sec>
      <sec id="sec-7-3">
        <title>Streams API</title>
      </sec>
      <sec id="sec-7-4">
        <title>Module 4</title>
      </sec>
      <sec id="sec-7-5">
        <title>Collectors and Data Processing</title>
      </sec>
      <sec id="sec-7-6">
        <title>Module 5</title>
      </sec>
      <sec id="sec-7-7">
        <title>Optional and New Date/Time API</title>
      </sec>
      <sec id="sec-7-8">
        <title>Module 6</title>
      </sec>
      <sec id="sec-7-9">
        <title>Parallel Data Processing and Concurrency</title>
      </sec>
      <sec id="sec-7-10">
        <title>Enhancements</title>
      </sec>
      <sec id="sec-7-11">
        <title>Module 7</title>
      </sec>
      <sec id="sec-7-12">
        <title>Advanced Topics and New Features in Later</title>
      </sec>
      <sec id="sec-7-13">
        <title>Java Versions</title>
      </sec>
      <sec id="sec-7-14">
        <title>Module 8</title>
      </sec>
      <sec id="sec-7-15">
        <title>Project and Practical Applications</title>
      </sec>
    </sec>
    <sec id="sec-8">
      <title>4. Autograder and functional programming</title>
    </sec>
    <sec id="sec-9">
      <title>4.1. Overview of autograders</title>
      <p>
        Autograders are software tools that assist educators, including professors and teaching assistants, by
automating the grading process of student assignments. This not only reduces the workload of manually
evaluating each submission but also eliminates potential biases from the grading process. This
technology is particularly beneficial for students in Computer Science (CS) and Electrical Engineering
(EE), where assignments predominantly involve coding tasks. The concept of autograders in these fields
has a history spanning over fifty years [
        <xref ref-type="bibr" rid="ref12 ref13">12, 13</xref>
        ].
      </p>
      <p>
        Initially, autograders were basic, capable of assessing only straightforward programming tasks and
primarily supporting procedural programming. These early systems typically provided binary feedback,
labeling student work as either "correct" or "incorrect" based on a rigid set of criteria. The latest
generation of autograders has developed alongside advancements in high-speed internet and web
technologies [
        <xref ref-type="bibr" rid="ref14">14-19</xref>
        ]. These systems often feature a web-based application, allowing students to code
directly in a browser without needing a local interpreter, compiler, or integrated development
environment (IDE). This approach also lessens the hardware demands on students' personal or campus
computers. Modern autograders are more versatile, supporting multiple programming languages and
paradigms, including object-oriented and functional programming, unlike their predecessors, which
were mostly confined to procedural programming.
      </p>
      <p>Despite the availability of various autograder tools, as noted in the literature [17], many are
customdesigned for specific languages and requirements. Besides tools created for higher education
institutions, there are also specialized autograders in Massive Open Online Courses (MOOCs) and
commercial platforms, often tailored for particular courses or programming language basics.
Autograders face two primary challenges: technical and pedagogical [20]. Technical challenges involve
ensuring security against potentially harmful code in assignments and integrating with the Learning
Management Systems (LMS) of educational institutions. Pedagogically, the inconsistency in grading
systems is a concern. While some autograders still rely on the basic correct/incorrect feedback model,
more advanced systems use a detailed step-by-step test case evaluation. However, there is no
standardized model or widely accepted grading recommendation for these systems.
4.2.</p>
    </sec>
    <sec id="sec-10">
      <title>Proposed model</title>
      <p>We have proposed a system that was primarily developed for learning basic and object-oriented
programming, and in this paper, it has been expanded to include support for functional programming.
The system we propose is built upon an existing autograder system, originally developed for teaching
fundamental programming concepts and object-oriented programming [15-18]. As depicted in Figure 1
on the following page, this system includes:
•
•
•
•
•</p>
      <p>Initial Assessment: Students begin by taking a self-assessment on data structures to establish a
starting point. This could be a brief quiz or a set of multiple-choice questions..</p>
      <p>Learning Module: Students are introduced to advanced data structure topics through lessons
provided in written, video, and animated formats.</p>
      <p>Assessment type 1: Students are tasked with a relatively easy to moderately challenging exercise.
They write code in a browser-based text editor, where a Java compiler operates server-side. The
exercise includes some pre-written code.</p>
      <p>Assessment type 2: More challenging exercises are presented, requiring students to write code from
scratch in a text editor without any pre-written code.</p>
      <p>Progress Tracking: After completing the assessments for each topic, the autograder system notifies
students of their success in the current topic. Progression to subsequent topics is contingent on
successful completion of previous ones. At the course's conclusion, students retake the initial
assessment (or a similar one) to gauge their learning progress.</p>
      <p>After each of the topics’ assessments, an autograder system informs the student if they have passed
the current topic successfully, and can only continue with the next topic if the previous is passed. At
the end of the whole course, the student completes the same (or similar) baseline test to self-assess their
progress.</p>
    </sec>
    <sec id="sec-11">
      <title>Comparison of code using imperative and functional approaches</title>
      <p>Within the framework of our developed autograder system, a variety of tasks are designed to enhance
the learning experience of students. One such task is as follows:</p>
      <p>The system presents students with a specific challenge: Convert the code shown in Figure 2 into
code that effectively utilizes functional programming, leveraging the Java Stream API, and employing
the methods filter() and collect(). Figure 2 serves as a practical illustration, demonstrating how a
particular programming task can be approached in two distinct ways in Java: the traditional imperative
approach and the modern functional approach. This task, specifically, involves filtering a list of
numbers, retaining only those that are greater than 10. The imperative method is shown as a starting
point, and students are tasked with transforming this into a more streamlined, functional version using
Java 8 features.</p>
      <p>This exercise not only reinforces their understanding of functional programming concepts but also
allows them to directly compare and contrast different programming paradigms within Java, thereby
deepening their comprehension and skill in the language. This task is representative of the kind of
practical, hands-on challenges that are integral to our autograder system, designed to provide students
with real-world programming scenarios that enhance their coding skills and theoretical knowledge."
In the functional approach, we use the Java Stream API. The stream() method converts the list into a
stream, the filter() method applies a lambda expression that specifies the filtering condition, and the
collect() method gathers the results and returns them in a list. Figure 3 shows how the same task can be
solved using a functional approach with Java 8.</p>
    </sec>
    <sec id="sec-12">
      <title>5. Conclusion</title>
      <p>In this paper, we have explored the characteristics of functional programming and proposed a
comprehensive curriculum for functional programming in Java 8 and its subsequent versions. We have
also presented significant enhancements to an existing interactive system for the online learning of
functional programming in Java, emphasizing support for automatic grading of assessments. These
improvements are meticulously designed for students who have a foundational understanding of Java
programming and are keen to transition to functional programming paradigms. The introduction of an
automatic grading system has streamlined the learning process, offering immediate and personalized
feedback, which is vital for effective learning. The updated curriculum, encompassing comprehensive
modules on key functional programming concepts in Java, such as lambda expressions and the Stream
API, ensures that learners are thoroughly prepared to comprehend and implement these advanced
programming techniques.</p>
      <p>Moreover, the integration of an interactive coding environment within the system fosters hands-on
learning and experimentation, which is crucial for a deep understanding of functional programming
nuances. The system's provision of personalized learning paths addresses the varied needs of students,
accommodating different learning styles and paces.</p>
      <p>In conclusion, the enhancements made to the interactive system mark a significant progression in
the realm of programming education. By aligning the system with the latest trends in Java programming
and concentrating on the practical application of functional programming concepts, we have developed
a robust educational platform. This platform not only imparts knowledge but also readies students for
the evolving demands of the software development industry. Our approach effectively bridges the gap
between traditional Java programming and the contemporary functional programming paradigm,
equipping learners with the necessary skills and knowledge to thrive in today's technological landscape.</p>
    </sec>
    <sec id="sec-13">
      <title>6. Acknowledgements</title>
      <p>This paper was supported by the Ministry of Education, Science and Technological Development,
Republic of Serbia (Project III44006).</p>
    </sec>
    <sec id="sec-14">
      <title>7. References</title>
      <p>[15]J. C. Caiza and J. M. del Alamo, “Programming assignments automatic grading: review
of tools and implementations,” In Proc. of the 7th international technology, education and
development conference (INTED2013), pp. 5691, 2013.
[16]G. Haldeman, M. Babeş-Vroman, A. Tjang, and T. D. Nguyen. “CSF: Formative
Feedback in Autograding,” ACM Transactions on Computing Education, vol. 21, no. 3,
2021.
[17]S. Krusche and A. Seitz, “ArTEMiS: An automatic assessment management system for
interactive learning,” in Proc. of the 49th ACM Technical Symposium on Computer
Science Education, pp. 284-289, 2018.
[18]Fangohr, H., O'Brien, N., Prabhakar, A. and Kashyap, A., 2015. “Teaching Python
programming with automatic assessment and feedback provision,” arXiv preprint
arXiv:1509.03556.
[19]D. Radošević, T. Orehovački, and A. Lovrenčić, “Verificator: educational tool for
learning programming,” Informatics in Education, vol. 8, no. 2, pp. 261-280, 2009.
[20]G. E. Forsythe, and N. Wirth, “Automatic grading programs,” Communications of the
ACM, vol. 8, no. 5, pp. 275-278, 1965.</p>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          [1]
          <string-name>
            <given-names>P.</given-names>
            <surname>Hudak</surname>
          </string-name>
          , Conception, evolution, and
          <article-title>application of functional programming languages</article-title>
          ,
          <source>ACM Computing Surveys</source>
          ,
          <year>1989</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          [2]
          <string-name>
            <given-names>M.</given-names>
            <surname>Odersky</surname>
          </string-name>
          ,
          <string-name>
            <given-names>L.</given-names>
            <surname>Spoon</surname>
          </string-name>
          , &amp; B.
          <string-name>
            <surname>Venners</surname>
          </string-name>
          , Programming in Scala, Artima,
          <year>2010</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          [3]
          <string-name>
            <surname>P. Van Roy</surname>
          </string-name>
          , &amp; S. Haridi, Concepts,
          <source>Techniques, and Models of Computer Programming</source>
          , MIT Press,
          <year>2004</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          [4]
          <string-name>
            <given-names>P.</given-names>
            <surname>Haller</surname>
          </string-name>
          , &amp;
          <string-name>
            <surname>M. Odersky</surname>
          </string-name>
          , Scala Actors:
          <article-title>Unifying thread-based and event-based programming</article-title>
          ,
          <source>Theoretical Computer Science</source>
          ,
          <year>2009</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          [5]
          <string-name>
            <surname>Zaharia</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          , et al. (
          <year>2012</year>
          ).
          <article-title>Resilient Distributed Datasets: A Fault-Tolerant Abstraction for In-Memory Cluster Computing</article-title>
          .
          <source>Proceedings of the 9th USENIX Conference on Networked Systems Design and Implementation</source>
          .
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          [6]
          <string-name>
            <given-names>J.</given-names>
            <surname>Dean</surname>
          </string-name>
          , &amp; S. Ghemawat,
          <source>MapReduce: Simplified Data Processing on Large Clusters, Communications of the ACM</source>
          ,
          <year>2008</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          [7]
          <string-name>
            <given-names>R.</given-names>
            <surname>Lämmel</surname>
          </string-name>
          ,
          <string-name>
            <surname>Google's MapReduce Programming Model - Revisited</surname>
          </string-name>
          ,
          <source>Science of Computer Programming</source>
          ,
          <year>2008</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          [8]
          <string-name>
            <surname>Armbrust</surname>
          </string-name>
          ,
          <string-name>
            <surname>Michael</surname>
          </string-name>
          , et al.
          <article-title>Spark sql: Relational data processing in spark</article-title>
          .
          <source>In: Proceedings of the 2015 ACM SIGMOD international conference on management of data</source>
          .
          <year>2015</year>
          . p.
          <fpage>1383</fpage>
          -
          <lpage>1394</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          [9]
          <string-name>
            <given-names>B.</given-names>
            <surname>Goetz</surname>
          </string-name>
          ,
          <article-title>Java 8: Functional features and libraries</article-title>
          , Oracle,
          <year>2014</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          [10]
          <string-name>
            <given-names>H.</given-names>
            <surname>Schildt</surname>
          </string-name>
          ,
          <article-title>Java: A Beginner's Guide, Seventh Edition</article-title>
          ,
          <string-name>
            <surname>McGraw-Hill</surname>
            <given-names>Education</given-names>
          </string-name>
          ,
          <year>2017</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref11">
        <mixed-citation>
          [11]
          <string-name>
            <given-names>R.</given-names>
            <surname>Warburton</surname>
          </string-name>
          ,
          <article-title>Java 8 Lambdas: Functional Programming for the Masses</article-title>
          ,
          <string-name>
            <surname>O'Reilly Media</surname>
          </string-name>
          ,
          <year>2014</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref12">
        <mixed-citation>
          [12]
          <string-name>
            <given-names>G. E.</given-names>
            <surname>Forsythe</surname>
          </string-name>
          , and
          <string-name>
            <given-names>N.</given-names>
            <surname>Wirth</surname>
          </string-name>
          ,
          <article-title>Automatic grading programs</article-title>
          ,
          <source>Communications of the ACM</source>
          , vol.
          <volume>8</volume>
          , no.
          <issue>5</issue>
          , pp.
          <fpage>275</fpage>
          -
          <lpage>278</lpage>
          ,
          <year>1965</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref13">
        <mixed-citation>
          [13]
          <string-name>
            <given-names>H.</given-names>
            <surname>Aldriye</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Alkhalaf</surname>
          </string-name>
          , and
          <string-name>
            <given-names>M.</given-names>
            <surname>Alkhalaf</surname>
          </string-name>
          , “
          <article-title>Automated grading systems for programming assignments: A literature review</article-title>
          ,”
          <source>International Journal of Advanced Computer Science and Applications</source>
          , vol.
          <volume>10</volume>
          , no.
          <issue>3</issue>
          , pp.
          <fpage>215</fpage>
          -
          <lpage>221</lpage>
          ,
          <year>2019</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref14">
        <mixed-citation>
          [14]
          <string-name>
            <given-names>C.</given-names>
            <surname>Douce</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D.</given-names>
            <surname>Livingstone</surname>
          </string-name>
          , and
          <string-name>
            <given-names>J</given-names>
            ,
            <surname>Orwell</surname>
          </string-name>
          , “
          <article-title>Automatic test-based assessment of programming: A review</article-title>
          ,
          <source>” Journal on Educational Resources in Computing (JERIC)</source>
          , vol.
          <volume>5</volume>
          , no. 3 pp.
          <fpage>4</fpage>
          -
          <lpage>es</lpage>
          ,
          <year>2005</year>
          .
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>