=Paper=
{{Paper
|id=Vol-2354/w3paper4
|storemode=property
|title=Tower of Questions: Gamified Testing to Engage Students in Peer Evaluation
|pdfUrl=https://ceur-ws.org/Vol-2354/w3paper4.pdf
|volume=Vol-2354
|authors=Nafisul Kiron,Julita Vassileva
|dblpUrl=https://dblp.org/rec/conf/its/KironV18
}}
==Tower of Questions: Gamified Testing to Engage Students in Peer Evaluation==
https://ceur-ws.org/Vol-2354/w3paper4.pdf
Tower of Questions: Gamified Testing to Engage
Students in Peer Evaluation
Nafisul Kiron, Julita Vassileva
Computer Science Department
University of Saskatchewan, Canada
ni.kiron@usask.ca, julita.vassileva@usask.ca
Abstract. In recent years, the use of gamification in various software application
areas is commonly used with success. Gamification is a technique of using game
rules, designs and mechanics in non-game applications. Educational testing is an
area that can benefit from this technique. It can help motivate, engage and en-
courage learners to participate in problem-solving and testing. In this research in
progress, we propose a gamified peer-testing system called “The Tower of Ques-
tions”, in the form of a web-based tower defense game. Tower defense games are
a subgenre of strategy games commonly found in computer, mobile, and console-
based platforms. Our game is a question and answer game that the students will
play with each other. Towers will be created and given to the students each time
they ask questions. The students will then attack other students’ towers by an-
swering those questions. This will continue until all the towers have either been
defended or conquered. We believe this testing system will engage students in
testing each other constructively and challengingly.
Keywords: Gamification, Peer-evaluation, Game-based testing
1 Introduction
The active participation of students in designing class test questions can make a more
engaging learning environment [1]. Students who can keep pace with what the instruc-
tor is teaching in the class have a better understanding of the course materials and do
better in the tests. Similarly, students, who can envision the questions the instructor
may ask in the tests have a better knowledge about the key lessons of the course. Both
types of students possess the ability to process their course studies thoroughly and come
up with good class test questions. Course instructors sometimes engage students in
making test questions and distributing these to the class to test each other [2]. The idea
is to let the students submit the questions they find challenging. This creates scope for
discussions and further learning of the course materials among the students and acts as
a revision technique.
Having students actively participate in class activities can benefit the entire class.
One way of achieving this is by gamifying the activities. Gamification means applying
the elements of games, for example, rules, aesthetics, rewards etc. to non-game appli-
113
cations [3]. Gamification can make activities interesting by allowing students to com-
pete, giving them status, achievements, self-expression, rewards etc. [4][5]. The course
instructor can design the activities by adding game mechanics to the tasks by using a
goal-oriented system. The gameplay experience of the players can be enhanced by
providing them with long and short-term goals [6]. That is, by making the activities
playful and rewarding, while all the points received from those activities will accumu-
late to something greater like securing a position in the leaderboard.
By using gamification in testing, students can be motivated to participate in a com-
petitive learning environment. Letting students construct their own questions from the
information they receive from the course gives them the freedom in designing questions
from their own conceptual structures [1]. With the rules and rewards offered by the
system, the students will be able to make plans on how they would want their games to
end [6]. Having students test each other playfully will make them focus on the game-
goals instead of thinking it as a class test. This will also create scope for discussions,
learning the details, and preparation for future tests.
2 Literature Review
A number of studies on gamification in education have shown that it can engage people
in various activities by imposing game rules, game aesthetics, rewards or a combination
of all of these [5]–[9].There are several successful educational and scientific services
and applications utilizing gamification, for example, Khan Academy, treehouse, foldit,
galaxy zoo etc. Most of the research done so far emphasized on engagement, data col-
lection, behavioral outcomes and performance improvements by using game elements
[6][9][10].
Yu et al. [1] introduced a web-based question posing system called QPPA (Question-
Posing and Peer Assessment). This system allows students to construct, assess, review
and practice answering questions [1]. There is a ranking list that shows the statistics of
the students’ performance. In another study, Yu [2] used multiple peer-assessment
mode to increase question generation by students. The interaction between question
authors and peers has been facilitated by a web-based system by allowing them to ex-
plain and negotiate each other’s’ feedbacks. Both studies focused on student-engage-
ment and participation in generating questions.
In their study of a gamified assessment system, Kocadere and Çağlar [7] used the
game dynamics, mechanics and components defined by Werbach and Hunter [11]. In
their study, they found that 9 out of 11 participants preferred gamified assessment. They
used game components such as question unlocking, points, leaderboard etc. in their
assessment system to make it enjoyable and motivational. Attali et al. [12] studied the
effect of points as a means of feedback in a gamified assessment. They performed the
studies on adult and middle school participants. In their assessment, they considered
the accuracy and speed of answering questions for awarding points for solving mathe-
matical problems. They found that rewarding points might influence the efforts of the
participants.
114
There are mixed results from studies regarding the effect of gamification on intrinsic
motivation [12]. One research found that using game-like features to reward students
for their performance might not be very effective in the long run and the effect of chang-
ing the incentives in the short run was inconclusive [12]. In another study where stu-
dents found the peer-assessment system favorable, it was found that the sources of mo-
tivation might have come from a mixture of multiple factors like a sense of achieve-
ment, security, altruism, “challenging one’s own and other’s existing knowledge” etc.
[1]. Kocadere and Çağlar [7] discussed both positive and negative aspects of gamified
assessment grouped by themes (enjoyment, flow, motivation, learning, low anxiety,
leaderboard and content unlocking). These studies were mostly focused on engagement
for learning and assessment. Depending on the use of gamification and implementation
of game-like motivational affordances the outcomes will vary from study to study [10].
Therefore, the use of gamification in testing for engaging students in a tower defense
type peer-testing game is a solution we think is worth exploring.
3 Proposed System
3.1 Tower Defense Games
Our proposed system uses some game dynamics and mechanics from tower defense
games, which are a subgenre of strategy games [13][14]. There are many variations and
versions of this genre, but the basic rule is the same. In a tower defense game, players
defend their towers from enemy attacks. Enemies attack the tower to conquer it. In our
game, the questions asked by the players will create virtual towers, and by answering
the questions other players will attack it. At the beginning of the game, players ask
questions to create towers, for each question asked one tower will be created. In regular
tower defense games, the tower has a health-bar that shows how many attacks it can
receive before breaking down. In our case, the tower can be conquered by attacking it
with the correct answer. During the gameplay, the players get gems for creating new
towers and by conquering other player’s towers. The leaderboard will be based on the
number of gems the players earned throughout the gameplay. There is a time limit for
attacking the towers after which those towers will be considered safe and cannot be
attacked. However, the closer the deadline is the more damage the towers will take and
the amount of reward gems will be increased accordingly.
3.2 Design and Method
“The Tower of Questions” will be a web-based game. The game mechanics are similar
to that of a real tower defense game. The players and enemies are students from the
class. The players use the game to post questions based on their course topics. Each
student can ask multiple questions from the available question types. The question can
be true-false, MCQ (Multiple choice questions) or in short answer form. Each question
115
posted in the game will act like a tower. The other students in the class will try to attack
that tower by answering it.
At the beginning of the game, the course instructor will set the number of gems
available for the game. Players can earn gems by asking questions and by answering
other players’ questions. Throughout the entire gameplay, the number of gems earned
by the players will not exceed the amount of gems set by the instructor at the beginning
of the game. Each question asked by the players will deduct a fixed amount of gems
from the main reserve. The main reserve is the place where all the un-earned gems are
stored. If there are no gems left in the reserve, no further questions can be asked by the
players. However, the instructor may increase or decrease the amount of total gems in
the reserve and let the game proceed or end.
Once a student asks a question, a tower will be created virtually in the game for that
question. The player who asked the question is the Lord of that tower. Then the tower
will be made visible to other players. That tower will then be available for a fixed period
of time to other players to attack it by answering the question. Each player can attack
each tower of other players only once. During the time of the attack, other players can-
not attack it. The attack consists by the attacker submitting an answer to the question,
the answer is shown to the Lord of that tower for review. When reviewing the answer,
the Lord will mark it correct, incorrect or partially correct. This concludes the attack.
Until the Lord has marked the answer, no other players can attempt an attack on the
tower by answering it. After the Lord has marked an answer fully or partially correct,
that question and answer will be made publicly visible and cannot be attacked again. If
it is correct, the attacker will receive a portion of the gems awarded for the creation of
the tower, otherwise, the Lord keeps the gems. Each player can give partial marks up
to three times, after which they have to award full marks. That means that only 3 dif-
ferent attackers can give partially correct answers, after that every next attempt would
be marked as either “right” or “wrong”. If the answer was wrong, the tower will be
open for attacks again. However, if the question was not successfully answered within
the fixed period, the Lord will have to answer it and then it will be made visible to
public and the tower will be considered safe from all attacks. In this case, the Lord
keeps the gems earned by creating the tower.
The players will continue to add new towers by asking questions and attack other
player’s towers by answering until all the questions have been answered, all the avail-
able gems have been earned or all the question deadlines have been reached. After that,
the course instructor monitors the status of the game and the current leaderboard. Dur-
ing the entire gameplay, the game is moderated by assistants assigned by the course
instructor. Players can report low quality or spam questions and unfairness in marking
during the gameplay. The moderators will keep a watch on reported issues and keep the
gameplay stable. For example, they review the leaderboard and especially the top
achievers – were their questions well formulated? Did they actually have an answer?
After a human review, the final leaderboard is posted. Fig. 1 illustrates a flowchart of
the processes.
An example walkthrough of the game is as follows:
116
• A player called Lord-X posts 15 unique questions of several types in the game, thus
15 towers are created. For each question, Lord-X receives 10 gems. So, for the 15
questions, Lord-X earns 150 gems.
• The players can post as many questions as they want while there are enough gems
available in the reserve. For example, there are 500 gems left in the reserve and Lord-
X asked 15 questions in the game. So, Lord-X will earn 150 gems and the remaining
gems in the reserve will be 350. Similarly, player Lord-Y and Lord-Z ask 15 and 19
questions respectively. Therefore, they earn 150 and 190 gems respectively and the
number of remaining gems in the reserve becomes 10.
• Now that the players have some towers, they start attacking each other. Lord-X suc-
cessfully attacks Lord-Y’s tower by answering a true-false question. So, he receives
6 out of the 10 gems from that tower. The remaining 4 gems are for Lord-Y to keep
for his contribution in building that tower. The distribution of gems for true-false
and MCQ type questions are the same.
Fig. 1. Flowchart of the game. The variables A, B, X and D are set by the course instructor.
• Then, Lord-Y attacks Lord-Z’s tower with a short answer. Lord-Z found the answer
to be partially correct. So, he marks it partially correct and publishes the Lord's and
attacker’s answer in the system. Lord-Y receives 4 and Lord-Z keeps the remaining
117
6 gems for that tower. The system allows the players to partially award gems to the
attacker up to the number of times allowed by the teacher. For example, the system
allows each player to award partial gems up to 3 times. So, the player can award
partial gems for 3 towers that have partially correct answers. After that, they will
have to give full marks.
• The players keep on attacking each other’s tower and in the end, only a few towers
are left. The reserve still has 10 gems from the original 500. Lord-X, Lord-Y, and
Lord-Z each hold 146, 164 and 180 gems respectively. Lord-Y decides to ask another
question, and this uses up the remaining 10 gems from the reserve. Lord-Y’s new
score is 174 gems. Lord-Z tries to post another question but fails because there are
no gems left in the reserve.
• With what is left, they attack each other for the last stand and eventually must stop.
Each of them defends a few towers completely because no one has answered those
questions. So, the Lord’s answer for those questions is published publicly. Since
these towers are untouched, the Lords of these towers keeps the full 10 gems for each
untouched tower.
• Finally, before the course instructor publishes the final leaderboard a final check for
all reported question and answers are reviewed by the moderators. The moderators
are actively monitoring the game throughout the entire session. They investigate sit-
uations which the players report as unfair.
4 Conclusion and Future Work
In this paper, we propose a gamified web-based game to motivate students to participate
in a question and answer posing system, masked as a tower defense game. The game
will allow students to test each other using the asynchronous web-based system. By
rewarding gems for submitting test questions and defending the towers, we are expect-
ing students to post high-quality questions to better defend their position in the game.
The game supports learning since is requires students to think of good questions about
the material themselves, which is an important aspect of active learning; it does in a
playful context, and allows students to test each other in a game that, we believe, will
motivate them to learn the material better and perform well in real exams.
Our future plan is to test the system with students at the University of Saskatchewan.
We will evaluate the interactions within the system by counting the number of banked
gems among the students and the number of questions and answers. We will evaluate
separately the quality of the questions generated and the good questions will become
part of the test-item bank for the class (a useful byproduct of the game). Finally, a post
gameplay survey will be presented to the students to learn their level of satisfaction in
using the system. We will measure the student engagement through their participation
and satisfaction. We will also measure student achievement through the scores in the
game, counting both the scores earned by creating questions and by answering them.
We will attempt to correlate these scores with those obtained at mid-term and final
exams and we expect to find positive significant positive correlations.
118
References
[1] F. Y. Yu, Y. H. Liu, and T. W. Chan, “A web-based learning system for
question-posing and peer assessment,” Innov. Educ. Teach. Int., vol. 42, no. 4,
pp. 337–348, 2005.
[2] F. Y. Yu, “Multiple peer-assessment modes to augment online student
question-generation processes,” Comput. Educ., vol. 56, no. 2, pp. 484–494,
2011.
[3] S. Deterding, D. Dixon, R. Khaled, and L. Nacke, “From game design elements
to gamefulness,” Proc. 15th Int. Acad. MindTrek Conf. Envisioning Futur.
Media Environ. - MindTrek ’11, no. September, p. 9, 2011.
[4] Bunchball, “Gamification 101 : An Introduction to the Use of Game Dynamics
to Influence Behavior,” no. October, 2010.
[5] C. C. I. Muntean, “Raising engagement in e-learning through gamification,”
6th Int. Conf. Virtual Learn. ICVL 2011, no. 1, pp. 323–329, 2011.
[6] F. F. Nah, V. R. Telaprolu, S. Rallapalli, and P. R. Venkata, “Gamification of
Education Using Computer Games,” vol. 8018, no. July 2013, 2013.
[7] S. A. Kocadere and Ş. Çağlar, “The design and implementation of a gamified
assessment,” J. E-Learning Knowl. Soc., vol. 11, no. 3, pp. 85–99, 2015.
[8] B. B. Morrison and B. DiSalvo, “Khan academy gamifies computer science,”
Proc. 45th ACM Tech. Symp. Comput. Sci. Educ. - SIGCSE ’14, pp. 39–44,
2014.
[9] P. Łupkowski and P. Wietrzycka, “Gamification for Question Processing
Research – the QuestGen Game,” vol. 1, no. 7.
[10] J. Hamari, J. Koivisto, and H. Sarsa, “Does gamification work? - A literature
review of empirical studies on gamification,” Proc. Annu. Hawaii Int. Conf.
Syst. Sci., pp. 3025–3034, 2014.
[11] K. Werbach and D. Hunter, For the Win: How Game Thinking Can
Revolutionize Your Business. Wharton Digital Press, 2012.
[12] Y. Attali and M. Arieli-Attali, “Gamification in assessment: Do points affect
test performance?,” Comput. Educ., vol. 83, pp. 57–63, 2015.
[13] “Tower Defense games on Kongregate.” [Online]. Available:
https://www.kongregate.com/tower-defense-games. [Accessed: 21-Mar-2018].
[14] P. Avery, J. Togelius, E. Alistar, and R. P. Van Leeuwen, “Computational
intelligence and tower defence games,” 2011 IEEE Congr. Evol. Comput. CEC
2011, pp. 1084–1091, 2011.
119