The choice of programming language in an introductory programming course (CS0) is crucial for understanding of basic programming principles. From our previous research, C programming language seems to be the most appropriate for these courses. In this paper we propose 8 problem sets (PS), the one of whom focused on Arduino boards, and we implement them to the curriculum of an CS0 course in C which is yearly attended by 500 students. We have collected PS results from four semesters of the course and we observed the impact of PS quantity on students' motivation and overall results. From our observation students mostly start to work since the last two weeks before the deadline, regardless of the solution evaluation length. A surprising result was the fact that with a higher number of PSs in a semester the average rate of submissions and final results was lowered. At the same time, we confirm the positive impact of game programming in CS0 courses that motivate students to work from the early beginning of testing and it is 27% earlier than in the case of classical PSs.
Computer science and software engineering have become very popular at
universities around the world due to the constant lack of IT engineers in the market.
Universities respond to this by expanding demanded study fields, and thanks
to that massive open online courses (MOOC) has grown considerably [
Universities exploit mentioned situation and try to attract students to less
popular study fields through IT departments, so they often accept all students
without admission procedure. In this case, it is necessary to classify the most
prospective students for considered study field. If the course is attended by a
student who has earlier programming experience, he often underestimate the
role of CS0 courses relying on his previous experience. This student brings his
own writing code habits and thinking, which, according to our experience with
teaching, are often negative for such students. Many authors in their papers [
As we can see, the task of introductory courses is very crucial for the future
development of the student. According to Allain [
In this paper we provide some partial results for the future hypothesis verification we formulated. Since we need basic information about students’ motivation in CS0 course to compare with other courses at the university, we formulate the following research questions focused on the CS0 course Programming : RQ1: Do students work more continuously if they have more time to develop a solution for particular problem set? RQ2: Does the assignment quantity affect the number of student submissions rate and final results? RQ3: Are students more motivated when programming games or motivation is similar to classical problem sets?
The following sections we describe current recommendations and research in this area as well as the design and partial results of the performed experiment. 2
For a research in university courses, one must first look at the pedagogical point of view, therefore in what form and in what structure information should be provided to the student. That is the reason we divided this section into pedagogical methods and experiments in university courses. 2.1
Pecinovský in his book OOP - Learn Object Oriented Thinking &
Programming [
Pair programming is a technique where two students collaborate on the same
project. In [
Hickey [
Use of Games First approach was addressed by Leutenegger and
Edgington [
Alvado et. al. [
Matzko and Davis [
In 2014, focusing on the problem sets, we noticed the obsolescence of a CS0 course at our university. Starting in 2013, we started using automated tests to evaluate assignments in the course. Despite our attempt to motivate students to better and more successful results by having multiple evaluations of their assignment, we have observed the same problem as during the manual evaluation - students submitted their solutions at the last minute. At the same time, only 2 assignments were included in the course, so the course offered students more theoretical and less practical experience.
Another problem was the poor connection between the lecture, the labs and the problem sets. The students had often necessary knowledge to complete the assignment since the last week before the deadline, so they were unable to work continuously. At the same time, the disadvantage was that the labs were mainly used to consult the students’ solutions, therefore, the labs potential was not sufficiently exploited. 4
Based on the problems of the original course curriculum and according to the recommendations mentioned in section 2, we decided to change the CS0 curriculum at our university in 2015. During making big changes in the course we also considered the suitability of C language for CS0 courses. 4.1
Although the Programming course was taught in the C language from the
beginning of its existance, we registered the increased interest of other universities in
other languages, so we were also thinking about changing the programming
language. We therefore conducted an analysis [
We found out that 70% of 37 top universities in the USA use Python in CS0 courses. This has gradually changed over the last decade and we consider it as a result of simpler understanding. With increasing popularity of object-oriented programming (OOP), student’s comprehension is higher thanks to simpler realworld objects mapping to the source code. We can say that increasing abstraction simplifies program comprehension. The choice of language for universities is likely affected by trying to provide programming in a language that is very popular and perspective for the student’s future.
On the other hand, we see a big gap in understanding how low-level programs
work. The low-level language, such as C, which is the 2nd most popular in CS0
courses, includes topics such as structures, pointers, static and dynamic memory,
stack overflow, etc., which have a very positive impact on the understanding of
programming contexts. Based on our previous analysis in [
Inspired by different universities in the world or interesting programming competitions (e.g., ACM ICPC1)), we have decided to split the problem sets into 2 categories: games and classical problems to solve. We wanted to evenly spread problem sets in the course for these two categories in order to observe the impact of the type of assignment and its number in the course on the submission rate and the final score of the student.
The goal of the new PSs is a better distribution in the course, so that students are motivated to work continuously during the semester. The design of several smaller tasks within the semester indirectly influenced the final curriculum of the course, so each problem set could be adapted to the lecture content. We also designed moderated scenarios for the labs to improve students’s cooperation with the teacher by solving an issue together.
For the 4 semesters of the course in the period 2015-2018t following PSs were proposed: K An 2048 game alternative. We use the character version of the game so students can not use existing solutions. The student must create the whole game, including screen rendering and hall of fame. Goal: arrays, structures, game loop.
Top Secret Classical ciphers assignment. The task is to encrypt or decrypt a string using a defined sequence of simple ciphers, such as playfair and our own ciphers (using bit masks, chain variations). Goal: arrays, strings, bit operations, pointers.
Adventure A text adventure game or interactive fiction game. The task of the student is to create the whole game, i.e. rooms, items, player and other extentions. Goal: bigger project maintenance, unions, dynamic memory, bit masks, regular expressions.
Tiktak & Triangulation Tiktak is a simple game (not considered as game in this experiment) with listing of numbers count requested by the user. If the actual printed number is divisible by 3 is typed "tik", if by 2 "tak", if both "tik tak". In triangulation the student has to calculate the correct coordinates of the enemy ship against the cannon on the land. Goal: working with expressions, using standard libraries.
Tesco Simulation of a store self-service cash register. The student has to deal with the emerging inaccuracies of type double, for example when issuing coins. Goal: representation of numbers, return values.
Hangman A game of guessing a random word by player. If the word is not guessed into a defined number of attempts the game ends. Goal: lists, arrays, game loop.
Problems to Solve It contains 4 complex algorithmic tasks. Goal: algorithmization, stdin, stdout.
For all PSs also test suites have been created to evaluate students’ solutions. In 2017 and 2018, we used Arduino board as one of the assignments, where student had to implement the game Mastermind. In this way, students are in touch with the electronics programming, which can increase their motivation for programming Internet of Thinks (IoT) in the future. The results of PS Mastermind are not included in this paper, as we focus exclusively on the results from the automated tests. The full text definitions of all PSs can be found at the official course website2 in slovak. 5
With the implementation of new problem sets from section 4.2 into the course, a testing environment called Arena has been developed. This environment automatically downloads student solutions from the Gitlab version control system (VCS) and tests them according to the defined structure. To assess the impact of the type of assignment and the number of assignments within the semester on the students’ motivation and results, we use the biggest course at our university (approximately 500 students/semester), so it provides a diverse sample for us.
From 2015 we have 4 semester results of total 2090 students. For each year, we selected different PSs combination and always with similar game/classic problem set ratio. At the same time, we have distributed different types of PSs during semester evenly so the results tracking at the beginning/end of the semester has no negative effect on the results of particular PS type. As usual, at the end of the semester students are working on assignments from different courses at once, so if all the PS games are at the end of the semester, it could negatively affect the results.
We have always adapted the number of assignments to the speed and focus of lectures. This way we tried to preserve following presentation of the curriculum: 1. lecture, 2. labs, 3. problem set. We also tried to sort PSs in a way to create better interconnections of new knowledge. Use of proposed PSs in individual years and their order during the semester can be seen in the table 1.
As can be seen, some PSs were used only in 2015. These assignments were less complex, so they were moved to another course and in later years we used more complex tasks. 6
In total, we collected 64,825 submissions of students. For PS, which was repeated in different years, there was also a different testing period for the solution. Students could submit the solution for evaluation at any time, but the tests were run every 3 hours and only if the student made a change to the solution. General statistics of all years are shown in Table 2.
To answer the RQ1 we have tracked the submission rate of solutions over time. In the image 1 we can see that even with a larger time for submission students start working since the last 2 weeks before the deadline and rarely students have used the possibility of an earlier submission. An interesting finding is that the highest peak of submissions is 1 day before deadline, while in general the last week is the most productive for students, both of submission rate and the increase of final score.
Result 1: Student’s ongoing work is not affected by the amount of time to submit the solution, students work mostly last 2 weeks before the deadline.
To compare the submission rate and results (RQ2) in different years we calculated for each problem set the average submission rate per year/PS/student (figure 2). As we can see, the submission rate in years with a higher number of PSs is lowered, but this result is ambiguous. When analyzing the submissions of particular PSs individually, we found the negative impact of using the PSs from the previous semesters of the course on these results. Despite the changes to the definition of the assignment, the number of submissions has been reduced in most cases for unchanged tasks, probably due to the plagiarism of solutions from previous years. When observing the tasks used for the first time the students worked more actively, because the student was not influenced by the correct solution (from the previous year’s course). With a higher number of PS in the semester the average submission rate decreases and the overall results in score are worse. However, the worse results do not represent the student’s learning experience, because with a greater number of issues the student meets, it can help him understand the context better.
Result 2: The more problem sets a student has to develop in the course, the average submission rate is lowered and the final score is worsened.
A good indicator of ongoing work of students we consider the number of submission per day. If the student is sufficiently motivated it is assumed that he or her works on the solution earlier. We found out that students are more active when programming games (RQ3) rather than programming classical PSs (see figure 3). When we analyzed the results of individual PSs, games were the fastest growing in the beginning of the chart, indicating an increased student motivation. Subsequently, the relatively similar number ( 500 submissions/day) of attempts to evaluate the assignment was counted. When observing the graphs of individual classical PS we saw a gradual increase in interest and the peak was reached just before the deadline, which means low student motivation. The average score on game programming was on average 3.19% higher over the overall evaluation of the award, but we consider this difference as not significant.
Result 3: Students are more motivated developing games and they work from the beginning of the PS publication. While programming a classical PS is the motivation initially lower, reaching a similar level in 27% later of total evaluation time. 7
Introductory programming courses play an important role in the way of how students think and develop the source code, both from a pedagogical point of view and previous empirical studies. The actual impact on students for following programming courses after CS0 has not been quantified in such way. In this paper, we answer partial questions about students’ motivation and continuously work in the introductory programming courses, with 4-year observation of students in the biggest course at our university. From our observation, students start to work since the last 2 weeks before the deadline, regardless of the length of the solution evaluation. For authors, the surprising result was the fact that with a higher number of PSs in the semester the average rate of submissions and final results was lower. At the same time, we confirm the positive impact of game programming in CS0 courses that motivate students to work from the early beginning of testing and it is 27% earlier than in the case of classical PSs.
In our future work, we would like to include additional assignments in the course in order to achieve more relevant results, as we noticed the negative impact of PS reusing. We also would like to take into account the results of the Arduino PS and evaluate the impact of CS0 courses on the results in other programming courses. Thanks to students’ solutions availability at VCS, we can take a closer look at programming styles and habits, to improve students’ source code comprehension.
The authors would like to thank all the participants in the research, especially students and teachers of course Programming of Technical university of Košice. This work was supported by project KEGA No. 053TUKE-4/2019: Learning Software Engineering via Continues Challenges and Competitions.