=Paper=
{{Paper
|id=Vol-3643/paper16
|storemode=property
|title=What motivates university students to engage in learning computer
skills?
|pdfUrl=https://ceur-ws.org/Vol-3643/paper16.pdf
|volume=Vol-3643
|authors=Andrea Micheletti,Nicola Orio,Daniel Zilio
|dblpUrl=https://dblp.org/rec/conf/ircdl/MichelettiOZ24
}}
==What motivates university students to engage in learning computer
skills?==
https://ceur-ws.org/Vol-3643/paper16.pdf
What motivates university students to engage in
learning computer skills?
Andrea Micheletti, Nicola Orio and Daniel Zilio
University of Padua, Department of Cultural Heritage: Archaeology and History of Art, Cinema and Music.
Abstract
In this paper, we present an approach to engage students in learning basic computer skills, in particular
on text formatting. We developed a suite of computer tools that manage the direct interaction with
students by assigning, correcting, and informing the students. The suite, called ACK, exploits a number
of gamification techniques that are covertly inserted in the process. We present a survey of student
reception on the effectiveness of these techniques.
Keywords
gamification, learning tools, user tests, office automation,
1. Introduction
ACK (Automatically Checking Know-How) is a suite of computer procedures designed to provide
a large number of humanities students (history, foreign languages, philosophy, archaeology) with
the computer skills needed to process written texts using a video terminal (office automation).
Once acquired, these skills will be helpful to students during their university studies for writing
essays and reports, preparing a final dissertation, and after graduation in the workplace. The
procedure consists of a series of increasingly difficult exercises assigned daily by a specially
developed automatic system. Students are expected to carry out the activities individually
at home with their smartphones/tablets/PCs or at one of the locations made available by the
University (computer classrooms, libraries, media centers). A software suite specially developed
by the authors automatically corrects the activities to give immediate feedback to students and
allow them to correct eventual errors. Among the goals of ACK, two important ones include
making students aware of the existence of a range of open-source word processing software –
LibreOffice and OpenOffice in particular – that can be a viable alternative to proprietary software,
as well as capturing the attention of the students themselves by engaging them in a targeted
activity through gamification techniques to increase involvement and motivation.
Gamification can be defined as the application of game dynamics and mechanics in non-
game scenarios [1]. Since its emergence, it has established itself as a methodology with almost
unlimited interdisciplinary applications [2]. In education, gamification can, for example, help
generate an engaging environment that, when combined with effective teaching techniques, can
20th IRCDL (The Conference on Information and Research science Connecting to Digital and Library science), February
22–-23, 2024, Bressanone-Brixen, Italy
Envelope-Open andrea.micheletti@unipd.it (A. Micheletti); nicola.orio@unipd.it (N. Orio); daniel.zilio@unipd.it (D. Zilio)
Orcid 0009-0006-9466-3607 (A. Micheletti); 0000-0002-0665-000X (N. Orio); 0000-0001-7107-8858 (D. Zilio)
© 2023 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).
CEUR
Workshop
Proceedings
http://ceur-ws.org
ISSN 1613-0073
CEUR Workshop Proceedings (CEUR-WS.org)
CEUR
ceur-ws.org
Workshop ISSN 1613-0073
Proceedings
enhance the learning experience [3, 4]. In the wake of these theories, several studies have shown
empirically that gamification can be used in education to motivate students to learn and have fun
while performing the activities assigned to them (learning by doing) [5, 6, 7]. Several findings
highlight the potential of applying gamification with traditional and innovative methodologies
in educational settings to improve students’ overall learning experience, motivating, engaging,
and pushing them to develop desired behaviors [8]. As such, gamification helps to improve
students’ learning and academic performance by simultaneously promoting positive behavioral
and psychological changes that vary depending on the context and characteristics of the students
under consideration [9, 10, 11].
The effectiveness of gamification in education and learning has been widely researched.
However, its effectiveness in teaching computer science in an academic setting has not been
fully explored [12]. Therefore, in this study, we aim to investigate how gamification techniques
are useful in promoting student engagement in learning basic computer science skills. In
particular, the impact of ACK will be assessed. This procedure is a combination of established
technologies (e-mail communication, word processing) with a new software suite developed in
Java and PHP for interaction management and outcome assessment. ACK has been evaluated in
terms of users’ responses to a number of gamification techniques that have been applied to the
whole procedure. This kind of “covert” gamification takes the form of a series of automated tools
that promote interactions among students, giving them control over learning progress, providing
immediate feedback to their activities and involving them in the adjustment of difficulty in
the proposed steps. In particular, we are interested in measuring the impact of gamification
on student learning outcomes in terms of performance on exams, quizzes and specific tasks
while measuring the spillover effect of various gamification approaches (such as player control,
pathway to mastery, immediate feedback, etc.). Therefore, the objective of this research is to
investigate the impact of gamification techniques in rendering the learning of basic computer
skills more engaging and effective.
2. Gamification Framework
One of the goals of gamification is to transform boring and repetitive activities into something
more fun and engaging, as games usually are. Points, badges, and leaderboards are part of
the process, but fun and engagement require the design of a coherent game experience [13].
According to Werbach and Hunter [1], gameplay resembles a branching path where choices
follow one another. To put oneself in the shoes of a game designer, it is necessary to consider
problems like games and the people involved like players. The latter will be the center of the
game and the universe created must revolve around them. For players to experience that they are
in control they must have the ability to make choices (player control) by operating independently
within, however, a set path (player journey) that unfolds within certain constraints and provides
a quick response to players’ actions (immediate feedback). Player control is a kind of “mind map”
followed by the player and revealed to them as they progress. The player’s path must be linear
in order to have a beginning, a middle, and an end. There are three stages that characterize
the player’s evolutionary process: (a) Onboarding involves “catapulting” the player into the
game so as to engage the player as quickly as possible. (b) Scaffolding incorporates a training
phase where the game is easier at first and becomes progressively more difficult as the player
progresses through the game. The progressiveness with which the difficulty is modulated
is designed not to halt the player’s progress within their path at the beginning and not to
bore them at a later stage when their skills are enhanced by experience. (c) The pathway to
mastery is the third level and coincides with the attainment of mastery i.e., the highest degree
of experience of the game and its dynamics. Thus, the creation of an immersive experience
requires that the player is not a passive spectator but the protagonist of a truly immersive
experience: this is where one of the most powerful emotional components of the experience
comes in, namely the fun part of the experience, which ought to be an indispensable element
in all the moments of the game [1].
The system-game can be regarded as a vast loop with repetitive and recursive structures
branching off in different directions. In particular, there are two different types of loops to
pay attention to: engagement loop and progression loop. The former operates at the micro
level, impacting individual actions, while the latter operates at the macro level, impacting
broad structures of the game. We will focus on engagement loops because they are more
relevant for the purposes of this research. Among the various engagement loops we have
identified: (a) Motivation: the game gives the user ”something to do,” some reason to feel
motivated, perform an action, overcome a challenge etc. If the motivation is strong it leads to
action. If the motivation is weak the loop is broken and another motivating factor emerges.
(b) Action: the user initiates the action for the purpose of being rewarded and obtaining
feedback. (c) Immediate feedback: the understands the effect of his/her actions through a
clear, direct and immediate feedback. This, in turn, becomes a motivating factor.
3. A Test on Computer Skills
In 2001, the Italian Government, with the introduction of Ministerial Decree no. 270, modified the
organization of all undergraduate courses: at least 3 ECTS of “Computer Skills” are a mandatory
part of the syllabus of any degree program, including the ones in humanities. Some degree
programs have chosen to introduce courses in Foundations of Computer Science, assigning
6 − 9 ECTS, but many have remained with the minimum number of CFU/ECTS required by law.
Thus, in the School of Human and Social Sciences and Cultural Heritage of the University of
Padova, there are around 1200 students that need to certify 3 ECTS in computer skills every year.
For these students, the most relevant computer skills involve the ability to properly manipulate
text in digital form. Since the introduction of the DM270 regulations, study courses targeted by
the project have provided 3 ECTS of Computer Skills, which have been assessed using a test
based on a set of closed-ended questions inspired by the test for the European Computer Driving
Licence (now International Computer Driving Licence). Until June 2018, about 11800 tests have
been administered, assigning the ECTS to about 6700 students. Data analysis has shown an
average pass rate of 60%, with the distribution of grades distributed in a roughly Gaussian shape
with an average slightly above proficiency. This distribution can be partially attributed to the
method of passing the test: since a mark of 18/30 is sufficient to pass, strong motivation toward
learning content is needed to prepare and achieve 30/30. This is compounded by an average
30% absentee rate at the tests, a sign of low interest in the subject and in the overall structure of
the evaluation.
These data suggest a change in approach is needed. A new software suite – called ACK
(Automatically Checking Know-How) – has been developed by the authors to manage the
interaction with students and the evaluation of their activities. The new approach aims at
increasing students’ involvement and motivation. An initial pilot was launched in the 2017-2018
academic year with about 800 students enrolled. The success rate on the final test, which
consists of doing again – this time in presence – five-six of the activities already done during
the training, was 98% with an average absentee rate of 4%. After the initial experimental phase,
the ACK suite of computer programs has been made available to all students of the School
of Human and Social Sciences and Cultural Heritage of the University of Padova, using the
following structure of the interaction.
ACK provides a daily assignment of an activity via email to each enrolled student. The
automated message contains instructions for a layout activity, the contest in which the particular
layout activity can be applied in a formal text, and an attached file on which to apply the layout.
The file is automatically generated to be different for each student to limit the possibility of
plagiarism. After receiving the email, the student can do the activity at any time during the day
with his/her computer/tablet/smartphone, and send back the edited file to an auto responder. The
responder analyzes the file, also checking for plagiarism, and sends the result to the student. If
the activity is correct, ACK sends an acknowledgment of the positive accomplishment, together
with a tidbit of information about computer science trivia; the student will receive a new activity
the next morning. Otherwise, ACK sends a notification of an incorrect layout and the student
can send a new file, possibly requesting help from teachers and colleagues through a dedicated
forum on Moodle.
Each student can thus decide the pace at which to proceed with the activities, depending on
his/her schedule (some do the activity as early as a few minutes after receiving it, others a few
days later); the software thus allows for student-controlled progression, while still ensuring
that each activity is corrected in real time. The activities are organized in ascending order of
difficulty, so that an increase in skills is matched by an increase in effort. The interaction with
ACK is paired with interaction through Moodle, the e-learning platform provided by the School.
In particular, there are two forums: one on general issues (e.g., dates of exams, software to
be used, curiosities about ACK) and another on individual activities, with a thread for each
activity where students can ask for help by sending the file that ACK considered incorrect. The
teacher replies within a working day to all questions, but students are encouraged to reply and
help one another. Moreover, students can reach a personalized web page through a link in
Moodle where they find the list of completed activities, the usage of all proposed formatting,
the percentage of completed activities, and the expected time to finish the mandatory activities
both by following the planned frequency of one activity a day or according to their actual
pace. There are 60 mandatory activities. Since ACK sends a new activity every day apart from
Sundays, the course can be completed in ten weeks, which is the average duration of in-person
courses at the University of Padova. The figure below shows the flow of information between
ACK and the student for each individual activity.
ACK is based on spaced repetitions, because it sends variation of the same layout at different
times, usually with increasing difficulty. For instance, there are eleven activities on page
formatting, starting from the simple insertion of page break to the complex use of three columns
Figure 1: Flow of information between the system and the student
with different margins. There are, on average, four/five days between each variant of an activity.
Students should then recall and repeat the required steps at different times during the course,
consolidating their skills. To encourage regular pace, ACK sends a “reminder” email with
the current activity when students are inactive for a number of days (reminders are sent at
increasing time intervals, until ACK stops sending reminder after a month of inactivity). For
instances, for the activities about list there are five different degrees of complexity: (a) create a
bulleted list of at least three items, (b) create a numbered list, in Arabic numerals, of at least
three items (c) create a numbered list, in Roman numerals, of at least three items (d) create a
bulleted list of at least three items using an image (such as a blue star) as a graphic symbol to
indicate the items (e) create a numbered list, in arabic numerals, starting with the number 7 (f)
create a two-tiered list, the main one in Arabic numerals and the secondary one in letters.
4. Gamification in ACK and Its Effects
ACK implements several gamification techniques internally to increase student engagement. The
use of gamification is not made explicit, because the procedure might not been taken seriously
(both by students and colleagues). This section will illustrate the relevance and effectiveness of
each method, measured through a survey conducted on 873 students by administering a set of
questions with answers on a Likert scale with seven levels, from totally disagree to totally agree.
Questions in the survey were relevant to some gamification techniques exploited in ACK:
• Player control and Immediate feedback: The survey investigated students’ opinion
about a daily assignment. A large portion of the participants, 311 out of 873, reported
that they quite agreed with the frequency of assignments while the majority, 321 out of
873, completely agreed. A total of 88% of students were positive about a daily assignment.
Moreover, the survey investigated students’ opinion about the possibility to undertake
the assignment at any time. Results were similar, with 321 students that quite agreed
and 390 students that completely agreed, and a total of 92% of students positive about the
immediate feedback at any time of the day.
– Onboarding: In order to join the enrollment list, students have to pass an “All
Aboard” (‘‘Sali a Bordo” quiz with a minimum score of 18/20, whose content con-
cerns the functioning of ACK procedure). A student can attempt the test several
times until the required grade is achieved; usually, three attempts are sufficient, and
to date, all students have passed the test. According to the survey, most students
considered the onboarding quiz a practical way to understand the nature of ACK
and the correct way to enroll, as shown in Figure 2.
Figure 2: Results of student agreement on the effectiveness of the “All Aboard” quiz as a
way to understand the different steps of the ACK procedure.
– Scaffolding: The first activities are very easy, making failure an unlikely option
even for beginners. Moreover, similar activities are presented a number of times,
with increasing difficulty, allowing for a cyclical consolidation of certain concepts.
This approach runs the risk of becoming boring for the students if they perceive they
are doing a repetitive task. Yet, the activities are more and more demanding, trying
to find a balance between the acquired competences and the required effort. This
keeps engagement in ACK high at all times, promoting a steady pace in performing
the assigned tasks on the one hand and maintaining control of progression in the
hands of the student on the other. Progression of tasks at increasing difficulty levels
also appeals to the Achiever User, the choice to try additional activities appeals to
the Explorer, while Socializers can enjoy completing ambitious and complex goals
through collaboration. According to the survey, students appreciated the fact that
new activities were based on previous ones, as shown in Figure 3.
Figure 3: Results of student agreement on the usefulness of repeating some activities for
improve memorization of procedures.
– Pathway to mastery: Due to how activities are sequenced, students feel pro-
gressively more competent as they perform increasingly complex activities. As
previously mentioned, the first few activities are not particularly difficult, but the
complexity gradually increases as activities progress, and students’ skills increase in
turn. Data from the experimental phase show that the time to do individual activities
remains virtually constant, even as they become progressively more difficult.
• Meaningful choices: ACK provides an explanation of the practical application (e.g., in
the work environment) of each skill acquired. This information is reported in the mail
that proposes the activity and in the personalized web page where each students finds
his/her own progression. The survey highlighted that students appreciated receiving an
explanation of how activities have practical applications, as shown in Figure 3. Moreover,
the survey investigated whether students thought that the skill will be useful in the future;
86% of the students gave a positive answer to this question.
• Collaborative problem-solving: ACK provides two forums: one is dedicated to ques-
tions and discussions about the general operation of ACK; the other is devoted to issues
encountered with individual activities. Within these communication channels, anyone
can answer the question contained in a post, particularly students who have already
solved the problem. The forums are considered an effective tool by 90% of the students,
with 395 out of 873 strongly agreeing with the statement.
• Fun: ACK activities were not expected to be fun, since it is rather unusual that people
enjoy text formatting. Yet students who pass an activity receive, as a sort of “prize”,
a computer science tidbit related to the scientific and technological evolution of the
discipline. The survey investigated the reception of this marginal aspect of the ACK
Figure 4: Results of student agreement on the usefulness of receiving an explanation of how the
activities provide useful skills.
procedure, showing that 79% of the participants agreed with the statement that tidbits
were interesting. Moreover, the final statement of the survey was even more direct, asking
if doing the activities, although compulsory, was enjoyable. A somehow surprisingly
majority of students, 69% of the participants, agreed with the statement even if only 135
out of 837 strongly agreed to the statement while 207 and 257 participants somewhat
agreed respectively.
5. Conclusions
ACK has been designed and developed to manage, with a minum amount of human effort, large
numbers of students in a basic course on computer skills. Typically, more than 1000 students
participate every year to the online course. One of the main issues of this kind of courses is to
maintain high motivation in students, even without a direct interaction with teachers. To this
end, we explored the application of a number of gamification techniques that, while granting
the students’ freedom of when and where performing the activities, provide an immediate
feedback to their actions and creates a constant motivation by: suggesting a routine in the
performance of the activities, reacting promptly at students activities, describing the usefulness
of the requested tasks, giving some pieces of free (thus useless) yet pertinent information. A
survey has been conducted on students’ reception of many of the aspects of the ACK procedure.
Results are really encouraging. The large majority of participants, on a seven-level Likert scale,
were positive to all the gamification elements. As a concluding remark, when asked if they
would have appreciated the introduction of gamification elements in the ACK procedure –
described as badges, leaderboards, quests – the majority of participants was neutral (289 out of
873) while the others were almost equally distributed between pro and against the proposal
(respectively 351 and 233) showing that explicit gamification techniques might not be a good
choice in this context. ACK is developed as part of a doctoral project, we plan to make the source
available for sharing with the scientific community after the PhD thesis has been defended.
References
[1] K. Werbach, D. Hunter, W. Dixon, For the win: How game thinking can revolutionize your
business, volume 1, Wharton digital press Philadelphia, 2012.
[2] L. E. Nacke, S. Deterding, The maturing of gamification research, 2017.
[3] S. de Freitas, D. Gibson, V. Alvarez, L. Irving, K. Star, S. Charleer, K. Verbert, How to use
gamified dashboards and learning analytics for providing immediate student feedback
and performance tracking in higher education, in: Proceedings of the 26th International
Conference on World Wide Web Companion, WWW ’17 Companion, International World
Wide Web Conferences Steering Committee, Republic and Canton of Geneva, CHE, 2017, p.
429–434. URL: https://doi.org/10.1145/3041021.3054175. doi:10.1145/3041021.3054175 .
[4] H. Yin, What motivates chinese undergraduates to engage in learning? insights from
a psychological approach to student engagement research, Higher Education 76 (2018)
827–847. URL: http://www.jstor.org/stable/45116783.
[5] K. Seaborn, D. I. Fels, Gamification in theory and action: A survey, International Journal of
Human-Computer Studies 74 (2015) 14–31. URL: https://www.sciencedirect.com/science/
article/pii/S1071581914001256. doi:https://doi.org/10.1016/j.ijhcs.2014.09.006 .
[6] Y. Attali, M. Arieli-Attali, Gamification in assessment: Do points affect test performance?,
Computers & Education 83 (2015) 57–63. doi:10.1016/j.compedu.2014.12.012 .
[7] M. D. Hanus, J. Fox, Assessing the effects of gamification in the classroom: A longitudi-
nal study on intrinsic motivation, social comparison, satisfaction, effort, and academic
performance, Comput. Educ. 80 (2015) 152–161.
[8] P. T. Palomino, A. M. Toda, W. Oliveira, A. I. Cristea, S. Isotani, Narrative for gamification
in education: why should you care?, in: 2019 IEEE 19th International Conference on
Advanced Learning Technologies (ICALT), volume 2161, IEEE, 2019, pp. 97–99.
[9] S. Kim, K. Song, B. Lockee, J. Burton, What is gamification in learning and education?, in:
Gamification in learning and education, Springer, 2018, pp. 25–38.
[10] T. Sitzmann, A meta-analytic examination of the instructional effectiveness of computer-
based simulation games, Personnel psychology 64 (2011) 489–528.
[11] M. D. Hanus, J. Fox, Assessing the effects of gamification in the classroom: A longitudi-
nal study on intrinsic motivation, social comparison, satisfaction, effort, and academic
performance, Computers & education 80 (2015) 152–161.
[12] A. Ahmad, F. Zeshan, M. S. Khan, R. Marriam, A. Ali, A. Samreen, The impact of gamifica-
tion on learning outcomes of computer science majors, ACM Trans. Comput. Educ. 20
(2020). URL: https://doi.org/10.1145/3383456. doi:10.1145/3383456 .
[13] P. Roganti, Gamification Semplice: Game design applicato in contesti non ludici, Paolo
Roganti, 2014.