=Paper=
{{Paper
|id=Vol-3410/short1
|storemode=property
|title=Coverage of Course Topics in Learnersourced SQL Exercises
|pdfUrl=https://ceur-ws.org/Vol-3410/short1.pdf
|volume=Vol-3410
|authors=Nea Pirttinen,Arto Hellas,Juho Leinonen
|dblpUrl=https://dblp.org/rec/conf/lats/PirttinenH022
}}
==Coverage of Course Topics in Learnersourced SQL Exercises==
https://ceur-ws.org/Vol-3410/short1.pdf
Coverage of Course Topics in Learnersourced SQL
Exercises
Nea Pirttinen1 , Arto Hellas2 and Juho Leinonen2
1
University of Helsinki, Helsinki, Finland
2
Aalto University, Espoo, Finland
Abstract
Learnersourcing is a common task in modern computing classrooms, where it is used, for example, for
the creation of educational resources such as multiple-choice questions and programming exercises.
One less studied type of learnersourced artefact is SQL exercises. In this work, we explore how well
different SQL topics are covered by learnersourced SQL exercises. Covering most course topics would
allow students to practice the full content of the course by completing learnersourced exercises. Our
results suggest that learnersourcing can be used to create a large pool of SQL exercises that cover most
of the topics of the course.
Keywords
learnersourcing, crowdsourcing, SQL, databases, topic coverage
1. Introduction
Traditionally, learnersourcing has been used, for example, for creating multiple-choice questions
(MCQs) [1, 2, 3, 4]. In computing classrooms, learnersourcing is increasingly used for the
creation of programming exercises [5, 6]. In our recent prior work, we introduced a system for
crowdsourcing SQL exercises [7], which to the best of our knowledge is the first such system.
One concrete goal of learnersourcing systems is to create a vast repository of exercises that
could complement or even replace instructor-created exercises. An important question related to
this is whether student-created exercises cover all the course topics, which would be necessary
to allow students to practice all learning objectives of the course. Prior work has studied the
coverage of course topics of learnersourced MCQs and found that student-created exercises did
cover all course topics [8].
In this work-in-progress paper, we report preliminary results related to how well learner-
sourced SQL exercises cover different SQL concepts. Our goal is to examine whether – by using
mostly learnersourced exercises – students will get to practice the concepts they are expected
to learn during the course.
The first annual workshop on Learnersourcing: Student-generated Content @ Scale, June 01, 2022, NYC, NY
Envelope-Open nea.pirttinen@helsinki.fi (N. Pirttinen); arto.hellas@aalto.fi (A. Hellas); juho.2.leinonen@aalto.fi (J. Leinonen)
Orcid 0000-0001-5249-5162 (N. Pirttinen); 0000-0001-6502-209X (A. Hellas); 0000-0001-6829-9449 (J. Leinonen)
© 2022 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).
CEUR
Workshop
Proceedings
http://ceur-ws.org
ISSN 1613-0073
CEUR Workshop Proceedings (CEUR-WS.org)
2. Tool and Context
2.1. SQL Trainer
SQL Trainer is a learnersourcing system for practicing SQL queries [7]. When using the system,
a student is presented with a list of topics in the order of appearance in the course material.
Students can choose to either create exercises or practice a specific topic. If the student chooses
to create an exercise for a particular topic, the system will ask which database the student wishes
to use (selection is made from a pre-defined list of databases created by the course instructor).
The student is then required to give a name for the exercise, an exercise description, and a
sample solution for that exercise. Once the exercise is completed, it is added to the pool of
exercises for that topic. If the student chooses to practice a topic, they will be given a randomly
selected exercise that they have not completed yet from the pool of created exercises.
A more detailed description of SQL Trainer and its usage, as well as the description of the
teacher view, is presented in [7].
2.2. Context and Research Questions
For the present study, we use data collected from three introductory database courses offered
by the University of Helsinki, where SQL trainer was used to support SQL practice. In the
courses, SQL trainer had 11 topics (briefly outlined in Table 1). Approximately 10% of the course
grade was based on completing at least four exercises per course topic and creating at least a
single exercise per course topic. Students were free to complete and create more exercises than
required for the points, however.
Our research questions are as follows:
RQ1. Which topics do students create exercises for?
RQ2. How well do the exercises created by students cover the course topics?
To answer research question 1 and 2, we partially replicate earlier work by Denny et al.
[8] and Purchase et al. [9] by studying to what extent the exercises that students complete
and create cover the course topics. In particular, we study how many exercises are created by
students per course topic, and how the used SQL concepts match the topics.
3. Results and Discussion
3.1. Created Exercises
In total, 1569 students entered the system. Out of these, a total of 1187 created at least one
exercise of their own. In total, students created 11247 exercises for 11 different instructor-defined
topics. The number of created exercises for each topic can be found in Table 2 (topic number
references listed in Table 1).
From Table 2, we can see that there is a clear downwards trend in the number of created
exercises per topic. One possible reason for this is that the latter topics come later in the course
and thus fewer students actively participate at that point in the course. Another potential
Table 1
Course topics. Arranged in the order the topics appear in the cource material.
# Topic
1 Select from a table
2 Filter and order data from a single table
3 Select from multiple tables
4 Select from even more tables
5 Other types of joins
6 Adding and removing tables
7 Adding data to a database
8 Using update and delete
9 Using functions
10 Using group by and functions
11 Functions, group by, having, order
Table 2
The number of exercises created per topic, 11247 in total. Arranged in the order the topics appear in the
course material.
Topic Exercises created
1 1301 (11.6 %)
2 1205 (10.7 %)
3 1113 (9.9 %)
4 1048 (9.3 %)
5 666 (5.9 %)
6 1063 (9.5 %)
7 1047 (9.3 %)
8 1037 (9.2 %)
9 1028 (9.1 %)
10 948 (8.4 %)
11 879 (7.8 %)
explanation for the trend is that the latter topics are related to somewhat more difficult topics
and thus students might not be as inclined to create exercises for those. One clear outlier is
topic 5, which was related to “other types of joins”. Why this topic elicited fewer learnersourced
exercises requires further research.
3.2. Topic Coverage
Figure 1 presents the most common SQL concepts for each topic (topic number references
listed in Table 1). Only concepts that were relatively common (occurred in over 10% of created
exercises in at least a single topic) are included. In the vast majority of cases, students are able
to identify relevant concepts for the topic, and utilize them when creating an exercise.
Interestingly, based on the SQL concepts used in topic 5, students seem to majorly prefer
creating exercises related to left joins compared to right or inner joins (which occurred rarely
enough that they were not included in the figure). Similarly, for topic 6, we can see that students
are more inclined to create exercises related to creating tables than dropping tables.
Figure 1: The number of SQL concepts used per topic. Color legend: concept used in over 80 % of the
created exercises in dark blue, over 40 % light blue, over 10 % yellow, under 10 % gray, and 0 occurrences
white.
3.3. Future Work
Altogether, these preliminary results support the use of learnersourcing for creating a large
exercise pool with a good coverage of course topics. This provides the opportunity for students
to practice a wide variety of SQL concepts using the learnersourced exercises. In our future
work, we are interested in extending this research to similarly examine how much students
practice the different concepts using the system. In addition, we are keen to study whether the
exercises created by students cover varying levels of difficulty and if the exercises are effective.
Lastly, we are exploring whether there are differences between demographic groups in to what
extent they participate in learnersourcing activities.
Acknowledgments
We are grateful for the doctoral research grant awarded by Jenny and Antti Wihuri Foundation
to the first author.
References
[1] P. Denny, J. Hamer, A. Luxton-Reilly, H. Purchase, Peerwise: students sharing their multiple
choice questions, in: Proceedings of the fourth international workshop on computing
education research, 2008, pp. 51–58.
[2] S. Moore, H. A. Nguyen, J. Stamper, Examining the effects of student participation and
performance on the quality of learnersourcing multiple-choice questions, in: Proceedings
of the Eighth ACM Conference on Learning@ Scale, 2021, pp. 209–220.
[3] A. Singh, C. Brooks, Y. Lin, W. Li, What’s in it for the learners? evidence from a randomized
field experiment on learnersourcing questions in a mooc, in: Proceedings of the Eighth
ACM Conference on Learning@ Scale, 2021, pp. 221–233.
[4] H. Khosravi, K. Kitto, J. J. Williams, Ripple: A crowdsourced adaptive platform for recom-
mendation of learning activities., Journal of Learning Analytics 6 (2019) 91–105.
[5] N. Pirttinen, V. Kangas, I. Nikkarinen, H. Nygren, J. Leinonen, A. Hellas, Crowdsourc-
ing programming assignments with crowdsorcerer, in: Proceedings of the 23rd Annual
ACM Conference on Innovation and Technology in Computer Science Education, 2018, pp.
326–331.
[6] P. Denny, A. Luxton-Reilly, E. Tempero, J. Hendrickx, Codewrite: supporting student-driven
practice of java, in: Proceedings of the 42nd ACM technical symposium on Computer
science education, 2011, pp. 471–476.
[7] J. Leinonen, N. Pirttinen, A. Hellas, Crowdsourcing content creation for sql practice, in:
Proceedings of the 2020 ACM Conference on Innovation and Technology in Computer
Science Education, 2020, pp. 349–355.
[8] P. Denny, A. Luxton-Reilly, J. Hamer, H. Purchase, Coverage of course topics in a student
generated mcq repository, ACM SIGCSE Bulletin 41 (2009) 11–15.
[9] H. Purchase, J. Hamer, P. Denny, A. Luxton-Reilly, The quality of a peerwise mcq repository,
in: Proceedings of the Twelfth Australasian Conference on Computing Education - Volume
103, ACE ’10, Australian Computer Society, Inc., AUS, 2010, p. 137–146.