How large impact does one ECTS credit have on the student motivation and eofrt? In this paper, we compared the results from learning environment data and post-course questionnaires between three diefrent years of the same introductory programming course to gain insight on the perceived workload from a student's point of view. While the teaching material and required assignments for successfully completing the course stayed mostly the same apart from minor scheduling tweaks, the reward for completing the course was raised from 5 ECTS credits to 6 ECTS during the observation period. According to our statistical analysis, the diference in student perception of the course workload in relation to the reward was insignificant: Even though the reward was higher in the later years and passing requirements were mostly the same, the students' assessment of the workload and their course activity did not change or did not lead to beetr results. This indicates that the sixth credit may have been lost to the credit inafltion caused by the revised curricula, and that the one extra credit does not increase the overall motivation towards the course to a large degree. This also implies, that from the viewpoint of providing more educational content, ofering several small c ourses might be more eficient than ofering few large course modules .
• Social and professional topics → Professional topics; Computing education; Student assessment Curriculum, course workload, grade inflation Copyright © 2018 for the individual papers by the papers' authors. Copying permitted for private and academic purposes. This volume is published and copyrighted by its editors.
The 2018 Workshop on PhD Software Engineering Education: Challenges, Trends, and Programs, September 17th, 2018, St. Petersburg, Russia ACM Reference format:
With increased student volumes, the teaching methods have
to adjust towards self-oriented approaches, and the curricula in
general adapts to the needs and realities of the available teaching
resources [
In this paper, we study the eefct of grade and study point inafltion against the actual workload of the students, with three mostly identical courses. The first one is a five credit course and second is a six credit course extending the five credit course. Additionally, we present the data from the same six credit course arranged the following year with some changes in place, to put the comparison of the two diferent ECTS rewards in beter context. eTh research questions are 1) how to measure the incentive of ECTS reward to student eofrt, and 2) how a revised reward afects students ’ perception of course workload. These questions are important since many European universities use the ECTS system as a common quantifier for measuring the volume of studies, and many times students plan their own study schedules based on factors, which include course sizes. Curricula sizing is therefore an important part of software engineering education on all levels, from undergraduate to graduate and doctoral programs.
To answer these questions, this work presents a case study where the students’ results, the efort requ ired to complete coursework, and perceived course workload from an introductory programming course are analyzed. eTh study was conducted by comparing student data from two iterations of the ifrst programming course (CS1): The first group of respondents took the course in 2015 and were asked to evaluate their own workload when they received 5 ECTS credits for completing the course. The same course was arranged in 2016, where the passing requirements and course material stayed the same with 2 more weeks to accomplish everything, this time awarding 6 ECTS credit points We also compare the diferences of the 2015 and 2016 course versions with the latest iteration of the course in 2017.
We accumulated data from three sources: eTh virtual learning
environment (VLE) (see [
hTis study is also a continuance work to our previous studies
into the student motivation and activities during the
programming courses. In our prior work, we have studied for
example collaborative learning [
hT e rest of the article is structured as follows: Section 2 discusses related existing studies, and the research process is explained in the Section 3. Section 4 has the results, which are discussed in the Section 5. Section 6 summarizes the paper with the conclusions.
It is an established fact that in software engineering in
general, and especially in education, motivation is the key for
achieving progress and enhanced results (for example [
In a study by Forte and Guzdial [
In purely motivational aspects, the diferent technical
solutions such as robots or game-like design for added
motivational push has been studied. For example McGill [
Games as the motivation tactics was studied by Jiau et al.
[
hTe intrinsic student motivation is an important aspect of the
course outcomes, but the other aspect of motivation is the
incentive-based motivation [
hTe course Introduction to Programming was used as a test
case in our experimental setup. The course spans the fall
semester, consisting of 12 to 14 weeks of lectures, programming
exercises, a 50 hour programming project and midterm exams or
a separate final exam. eTh original course was designed to
minimize the amount of so-called hygiene problems (see [
Essential learning goals also remained the same throughout the comparable years. In 2017, the weekly assignments and programming project were developed further to beetr tfi the ECTS sizing, and therefore we were unable to use the data from the latest iteration of the course for statistical analysis. However, we can use the descriptive indicators from 2017 in comparison to the last two years to establish context beyond examining the dif erence between single years.
Overall, on both 2015 and 2016 implementations the final grade was based on the separate grades from the exam, exercises and the project work. The courses organized in 2015 and 2016 were identical in the expected minimum ef ort, and the efort needed to receive the best possible grade. Additionally, even though the amount of students on the course rose from previous in 2016, the student body was still very homogenous, as all the students for whom the course is mandatory came from degree programs in technology and engineering.
hT e two student populations from the courses were compared
against each other to test whether the groups were statistically
similar. This was tested with chi -squared test [
In this section the diefrent results and metrics collected from the activity logs are presented. First of all, the Mann-Whitney U test was applied to assess whether the two courses (2015 and 2016) had significant diefrences between the reported time distribution and sessions activity based on the VLE data logs. After applying sanitation measurements of rejecting students with less than 30 hours of recorded activities, and compensating the 2016 group for the online exams which weren’t available for the 2015 course, the H0 hypothesis of “The distribution of total hours is the same across categories of course year” was retained, despite the averages and medians being higher. Median time used in 2016 was 58.78 hrs, and 52.25 hrs in 2015; the distributions in the two groups did not difer significantly (Mann-Whitney U=50591.5, n1=454, n2=287, P > 0.05). In comparison, the median online time in 2017 was 50.12 hrs.
Similarly, the student activities were tracked based on their weekly submited e xercise assignments during the course. This data indicates, that the activity paetrns are almost identical, with most of the students working similarly on both courses during the first half, and dropping of at the laetr part of the course, with the larger 2016 course having a small increase in activity during the weeks 10-14. This data is summarized in the Table 3..
Finally, the student activity and the perceived workload was assessed with the student feedback. The student feedback survey covered topics such as the perceived workload and dificulty of the course, grading of the diferent course components and also open feedback on how the course could be improved. Overall, the results indicate that the sixth credit given on the 2016 course did not afect the student performance, workload or motivation to a large degree. eTh results are summarized in Table 4.
Overall, the student feedback did not diefr to a large degree between the courses, although the trend was that the 5 ECTS course was considered beter by the student feedback; the appropriateness and overall grades for the 2015 implementation were both over 4 (in scale 1-5, 5 best), whereas in the 2016 course they were half a grade worse. Similarly, the amount of positive feedback declined in the 6 ECTS course, although this can be explained also with the technical problems concerning the online exams and the learning environment. On the assessment of the amount of efort, the self -assessed perceived workload actually declined somewhat, but the diference between the implementations (from 3.1 to 2.9) is not very meaningful. In contrast, the workload was perceived as much higher the in 2017. eTh amount of student feedback about the workload and required eofrt increased between 2015 and 2016, and the 2017 course collected significantly more feedback about the amount of work than either of the previous years.
Obviously there are limitations to the collected data, and elements presented in the results. For example, the survey instrument changed between the years to prompt more
2017 (6 ECTS) 477 (91% of all) 436 (83 % of all) 400 (76 % of all) 316 (60 % of all) 329 (63 % of all) 256 (49 % of all) 385 (73 % of all) 280 (53 % of all) 83 (16 % of all) 136 (26 % of all) 122 (23 % of all) 185 (35 % of all) 90 (17 % of all) 31 (6 % of all) descriptive feedback, and the wording of the question has changed over time. However, in all the feedback surveys questions used the Likert scale of 1 to 5, and asked the students to evaluate how the actual course workload compares to the ECTS sizing of the course. The average grade from these questions was not very diferent between the years (3.1 in 2015, 2.9 in 2016). Additionally, as the statistical analysis shows, the answers are not statistically significantly diefrent from each other.
In all years the course workload and insuficient credits are some of the most highlighted themes students gave feedback about, especially if ignoring technical details such as bugs in the learning environment or exercises. It should be noted though, that in 2017 the survey included a separate open-ended question about the perceived course workload, which may explain the high number of negative workload related feedback in that year. Additionally, even though our VLE system has an automatic 30 minute inactivity logout feature for the sessions left open, a handful of students recorded very unrealistic hundreds of hours of online activity. These clear outliers were sanitized, and due this issue the median values were applied in the overall analysis.
hTe student body for whom the course is mandatory is very heterogenic, as it covers almost all undergraduate engineering students in the university. Additionally, most students take the course during their first year of studies. This is a limitation in the sense that the freshmen have few other courses to compare the workload with, and may for that reason have dificulty estimating the real hours they have had to invest.
hTe usage statistics from the VLE platform provide some insight to the actual working hours of students. eTh statistical analysis indicated that the time usage was similar between the years, even if the 2016 statistics exhibit about 7 hours more median online working time, most likely caused by the online exam, which was added to the 2016 course. Regardless of the reason, from a teacher’s perspective the same learning goals were achieved using the same amount of time or even slightly more. In 2017 the average and median online working time was again similar to 2015, this time in a course seting which had been altered for the specific reason of adding more content for the new ECTS sizing. These numbers suggest at least partial credit inafltion, as the students are passing the same course with the same learning goals but geting a higher reward for their eforts. In addition, since the relative amount of negative feedback on the workload increased for the 2016 course, and also in 2017, it can be said that the incentive of extra credit did not provide much of a diference in our case.
In general, it seems that the incentive-based motivation of
one additional study credit probably does not translate into
actual work efort of 27 hours. As based on our observations, the
student workloads do not diefr to a la rge degree between the
ifve and six credit eofrt. hT e grade averages actually fell by one
grade when the course was revised to become a larger module; it
seems that the students were content with geting a worse
passing grade, instead of puting more e fort into the course. This
observation is in line with the observations on the college-level
student time usage and efort reported in [21]. Even if we did not
observe the ofline work hours, the student performance and
course outcomes does not imply that there would have been a
meaningful diference, especially since the populations and their
performance results were statistically comparable. In the grand
picture, this also implies that the students receive 20 percent
larger reward for their efort, since for the Master’s degree, the
students are required to take in average 50 completed 6 ECTS
modules instead of 60 completed 5 ECTS modules. As based on
the workload estimations from our case study, it could be argued
that the approach with 60 modules with 5 credits provides beter
learning outcomes, and the one ECTS course credit diference in
the default module content scaling imposes the risk of the
diference of ten modules in the Master ’s degree curricula. This
translates to the issue that on the long run, almost one year
worth of studies could be lost to the credit inafltion, similar to
the grade [
Considering the entire curricula, the results here are an interesting observation on that the larger, topic-spanning courses are not as eficient as smaller topic -oriented courses, especially if the course can be successfully completed with a subset of information not covering the entire content. One suggestion on why this happens is that since the students can optimize their efort during the course, they can simply select to submit works where the assignments are relatively easier to compensate on the added topics which tend to be more dificult. In our case, the only more active weeks on the 2016 course were during the weeks 10 to 12, where there were 10 to 15 percent more activity. However, it is worthwhile to observe that geting 25 percent of the mandatory assignments completed is possible within the first 5 weeks of the course. If the students are willing to accept worse final grades, like our students based on the median grades did on the 2016 course, they are not actually required to do extra work on the later part of the course.
In this paper we have presented the results of our study into the incentive-based motivation in the student participation activity and studied the efect on a course implementation, where the only major diferences were the two additional weeks of lectures, and one additional study credit.
Based on our observations, the answer to the research questions 1) how to measure the incentive of ECTS reward to student efort, and 2) how a revised reward afects students ’ perception of course workload can be summarized as follows: one credit diference does not translate to the student motivation or workload in a meaningful manner. Th ere were no indications that the one-point diference, or the two extra weeks of the course had a meaningful impact since the only actual diference was 7-hour increase in the median, which could also be explained with minor changes in the course arrangements. Even though the 7-hour increase in online working time between 2015 and 2016 could indicate that the students were in fact working more and the rest of their active work was completed ofline (which we could not measure), we could see from the 2017 course data that the average and median working times came down again. Additionally, it is worth noting that if the students’ working time was on the rise, the additional reward in credits would still be lost in inafltion, as the course ’s key learning goals stayed the same throughout the years.
Similarly, as reported elsewhere [
In our observed cases, the sixth ECTS was lost to the credit inafltion, meaning that the e xtra credit was awarded with no additional learning activities required from the student, and no extra learning objectives achieved. On a scale of a degree program, this inafltion -caused small diference would mean that the student with an average of tyfi 6 ECST modules would be a full semester worth of knowledge behind the student, who did sixty 5 ECST modules, while technically receiving the same degree from the same program.
Obviously there are also risks in this study; the amount of independent self-study was not measured, and the activity logs were inaccurate to the degree that some sanitation had to be done. However, the statistical analyses indicated that the results between the courses were identical, as were the worktime and activity estimations. Even though there might have been issues with the data collection tools, the issues were the same for both datasets.
As for future work in this topic, we have established that the efect of credit inafltion exists and that one study credit is not very efic ient motivator for students to put more eofrt into their work. Therefore it would be interesting to study this efect further, for example from the viewpoint of