-In face-to-face teaching, students work in pairs on programming exercises and present their solutions to tutors. This setting fosters social skills. Students benefit from immediate feedback loops and personalized explanations. However, with an increasing number of students, it becomes challenging to scale this approach to very large courses due to the logistic and organizational effort. In this paper, we first report on significant challenges that we identified while conducting face-toface teaching in a software development course with more than 600 students and 50 tutors. Second, we introduce a preliminary socio-technical framework for face-to-face teaching to facilitate logistical aspects, monitor the students exercise progress, and improve the students' learning experience. Index Terms-face-to-face exercises, software development education, learning analysis
Our Applied Software Technology group at the University of Hamburg hosts the introductory programming course Software Development 1 (SE1). SE1 consists of two parts: a weekly 90-minute lecture and a weekly 3-hour face-to-face exercise session. The course is compulsory for all computer science majors. Students with other majors are also eligible to register for SE1 including mathematics, pedagogy, physics, or psychology. SE1 is an entry-level course in programming and requires no prior knowledge. We host SE1 annually in winter terms.
The exercises are structured thematically in accordance with
the lecture. At first, we introduce new concepts with examples
in the lecture. In the subsequent practice week, the students
work in pairs on an exercise sheet in the laboratory rooms as
shown in Figure 1. The programming exercises are inspired
by Barnes and Ko¨ lling [
During this session, tutors (teaching assistants and student tutors) are present for two main purposes. On the one hand, they support the students in case of questions or uncertainties and on the other hand they approve the solutions of the students. After the students completed a subtask, they have to present their solutions to a tutor. The tutor either accepts or declines the solution. A tutor accepts the solution if it (1) solves the task, (2) the students present and explain their solution with their own words, and (3) answer subsequent Fig. 1. Laboratory room with pair programming students (yellow frame) and student tutors (red frame). The tutors check the solutions of the students. Fig. 2. Part of a student’s evaluation sheet: student details (top), attendance table (top right), and one table per exercise sheet, each with four subtasks. questions. Tutors check off each passed subtask by signing the student’s evaluation sheet, which is shown in the tables in Figure 2. The tutor declines the solution if the presentation or the solution is insufficient. In this case, the students have to improve the shortcomings and present their corrected solution. Students have to have all exercises accepted to pass SE1. Course instructors prepare the teaching material, monitor the students’ progress, and manage the logistic aspects. The course evaluation showed that students appreciate this format.
Face-to-face exercises encourage the students to reproduce
their just learned knowledge by explaining it with their own
words to a tutor. Thereby, tutors identify and correct
misunderstandings early and provide immediate feedback with
personalized explanations. Students have to program in pairs
as previous research has shown an improved code quality and
better student grades [
In recent years, the number of students has risen steadily and has now reached more than 600, which are mostly first semester students. Applying this approach to large software development classes is challenging because it does not scale easily. We summarize the main challenges that we identified in Section II. In Section III, we introduce our preliminary sociotechnical framework to facilitate the logistical workload, to enable learning analysis for course instructors, and to keep the face-to-face component in the exercise sessions.
In recent years, the number of students in our SE1 course has steadily increased (~300 students in 2007-09, ~400 students in 2010-12, ~500 students in 2012 and ~600 students in 2016). Additionally, computer science is becoming more and more important across various disciplines. In particular, SE1 is also eligible for non-computer scientists. We expect that the number of students will increase to maybe more than 800 students in the next 2-3 years. A higher number of students will make it challenging to keep conducting faceto-face exercises in SE1 as it requires more laboratory rooms and more tutors.
The logistics of the SE1 exercises comprise various parts about tutor management, teaching material, and student communication. For the preparation of the SE1 exercises, we recruit student tutors in addition to teaching assistants. The number of required tutors scales directly with the number of students. We contact students via different channels including other courses, mailing lists, or face-to-face. The students apply via an online form for a tutor position. We mainly select students based on their tutoring experience, recommendations by previous tutors, and prior grades.
Students register for one of eight different time slots during the week for the face-to-face exercises. The preferences of the students differ each semester. The course instructors try to assign a sufficient number of tutors to each exercise session, depending on the availability of tutors and the number of registered students. Furthermore, a teaching assistant is assigned for each exercise session for supervision and answering questions of the tutors.
Most of the exercise sessions require multiple rooms, for instance, one of our sessions with more than 120 students spreads across seven rooms. Tutors change rooms to check the occupancy in other rooms. Based on our experiences, we found that a ratio of one tutor to seven students is sufficient. For tutoring, we usually require ~40 student tutors and ten teaching assistants. Most of the tutors apply for more than one exercise session.
The programming experience among the tutors is heterogeneous. We observed that experienced tutors have higher demands on the solutions of the students. However, it is challenging to maintain consistent acceptance criteria across all tutors for the evaluation. Therefore, we set up a two-hour introductory meeting with all tutors before the semester starts. Additionally, we provide a briefing session each week for the tutors to collect feedback about the previous exercise week, walk through the upcoming exercise sheet, and discuss the acceptance criteria.
The general aim is to maintain knowledge about the logistics independently of an individual course instructor so that different employees can take over course management activities at any time. This is particularly important as the logistics contain fine-grained tasks with crucial deadlines and the staff is typically rotating after two years. Therefore, we explicitly document single steps and their deadlines for each part. Always two course instructors take responsibility for the SE1 exercises to ensure reliability. Furthermore, the fluctuation among the tutors is high with ~60% new tutors each semester. Experienced tutors are beneficial to support new tutors and teaching assistants in many aspects of the practice. To mitigate the knowledge loss, we require student tutors to follow along with other tutors during the semester for three hours in total to enable a knowledge transfer between experienced and new tutors.
The occupancy of each exercise session is inconsistent as students visit other time slots when they could not finish the exercises in time. Also, tutors are regularly absent due to illness. Due to these unexpected factors, single exercise sessions exceed a proportional ratio between tutors and students. This leads to long waiting times for students until a tutor is available for their requests. In this case, tutors request tutoring support on demand from teaching assistants to overcome a temporary shortcoming.
As each of the student tutors have their course schedule, we provide two redundant briefing sessions per week. The student tutors have to attend one of these sessions to prepare for the upcoming exercise sheet. Two teaching assistants carry out two separate briefing sessions for the student tutors so that they can substitute each other. The teaching assistants prepare the briefing session and discuss the current version of the exercise sheet.
To pass the SE1 exercises, students have to pass the requirements on each exercise sheet. During the semester B. Tutors App the number of predecessors in the queue and the number of available tutors, the framework indicates an estimated waiting time for the students until the next tutor will be available. the tutors continuously monitor how many students currently comply with the conditions to pass the SE1 exercises. As the tutors check off the approvals on the evaluation sheet, it is currently time-consuming to analyze each sheet manually and to continuously monitor the students’ progress.
Moreover, course instructors collect feedback on the SE1 exercise part via three different channels: (1) from the tutors in the briefing sessions on their impression about how the students cope with the exercises, (2) from students via an anonymous online survey, (3) from students via an anonymous evaluation of the whole course at the end of each semester.
Course instructors aggregate the feedback and formulate concrete change requests to improve both the process as well as each weekly exercise sheet. The complexity of these changes varies from typos to replacing subtasks or source code projects.
Tutors log into the framework via their smartphone. They have access to an extended version of the exercise sheet with additional remarks, sample solutions, and sample questions to ask students during the evaluation. During the exercise session, they process the queue of student requests. An available tutor queries the next request from the student queue and visits their workplace, which might be in a different room. The tutor has access to the evaluation sheets of the students and sees comments by tutors on previous evaluations. The tutor evaluates the solution and rates different categories as, for instance, presentation, code quality, and speech share. Thereby, E. Participants with Different Prior Knowledge we keep the operational face-to-face teaching by the tutors
The students in SE1 are from diverse studies with different as a social component in our framework. Following tutors prior knowledge. Because the participants increasingly come utilize previous evaluation data and take previous comments from subjects outside of the computer science department and into account. For instance, if a tutor notices that one of the outside of the MIN faculty, we have to adapt to different prior students shows less participation in the presentations, the tutor knowledge levels. Therefore, in the lecture and especially in could directly address questions to that student. the exercises, we must be able to act in a target-group-oriented In our course, the attendance of the weekly exercise sessions and adaptive manner to reach all students. Consequently, it is mandatory. The framework automatically flags students as is a challenge to design the course for students without any “attended”, if a tutor checks a solution to a subtask. Otherwise, previous experience and at the same time motivate advanced students have to enqueue to contact a tutor for requesting programmers. In an ideal case, students could learn from each attendance manually. other. We observed that restrained students hesitate to ask tutors
We still explore and experiment with different prior knowl- when they get stuck on an exercise. Our framework monitors edge combinations and other mechanisms to motivate experi- the progress of each student pair in the background and enenced programmers and educate beginners. We observed two queues a request for slow progressing students automatically. different patterns when students, having an unequal level of Thereby, tutors intervene and offer support early when students prior knowledge work together. Some experienced students fall behind. allow their partners to contribute to the solution and they actively explain concepts to their partners. Unfortunately, C. Course Instructors Dashboard the other pattern is that experienced programmers finish the exercises quickly alone, explain the solution to their partner, and finally get the solution checked by a tutor and leave early.
We highlight this as a negative example in the lecture and ask the tutors to spot this pattern.
In this section, we introduce our preliminary socio-technical face-to-face teaching framework that partly solves the challenges we described in the previous section. We describe the framework from the perspective of the three roles: students, tutors, and course instructors.
Students access the framework via a web interface. They access the current state of their evaluation sheet, exercise sheets, lecture slides, and further references. At the beginning of each exercise session, each student pair logs into the system and enters their seat and pair programming partner. During the exercise session, students enqueue requests for the tutors, for instance, an evaluation of a subtask or a question. Based on Course instructors access the framework via a web interface. For the tutor recruitment, they set up the questions for the application form, which is an integrated part of the framework. Course instructors monitor both the application process as well as the distribution of students in the exercise sessions during the registration period. Applicants fill out the form with their personal information, additional references, information about previous grades, as well as their availability and how many exercise sessions they plan to supervise. The framework automatically indicates unbalanced exercise sessions and notifies the course instructors. Course instructors constantly monitor the ratios between tutors and students for each time slot and cap the participation limits for students to register for the exercise sessions.
Based on the data, collected in the exercise sessions, course instructors acquire an insight into the current status of the SE1 exercise part. The dashboard for course instructors offers an insight into the following metrics: • Ratio of students compliant with conditions. The students in SE1 require to fulfill the conditions of the SE1 exercises. Our framework shows the proportion of students who are currently compliant with the SE1 exercise conditions, how many are behind, and how many dropped out. • Attendance. Course instructors get an overview of when students attend the exercise sessions. How many students have to catch up with exercise sheets when they were not able to complete the exercise sheet in one exercise session? We further use this information to fine-tune the balancing of tutors among the time slots. • Complexity of exercises. As tutors check each subtask of an exercise sheet individually, course instructors get an insight into the time it takes students to complete each subtask. Based on this metric, course instructors scope and fine-tune each subtask so that solving a complete exercise sheet is manageable in a three hour exercise session. • Pair programming partner matching. Course instructors get an overview about the pair programming partner combinations. Which pair programming partner combinations are common? What is their level of priorknowledge? Course instructors use these insights to adjust future pair programming partner matching. • Quality assurance for tutors. Each evaluation of a student solution is different. It is crucial that tutors have uniform requirements for approving a subtask. Course instructors are interested in questions such as: How many evaluations does each tutor perform? How much time does a tutor spend on checking each subtask? What is the accept and reject ratio for each tutor? Course instructors can discuss significant evaluation differences in the tutor briefing sessions.
Other researchers discussed teaching approaches for
teaching software development face-to-face to benefit from
immediate feedback loops. Kothiyal et al. [
Face-to-face exercises entail a huge logistic effort, which covers diverse aspects including tutor management, improving teaching material, and student communication. We report on major challenges that we identified in hosting the exercises of SE1, a beginner level software development course with more than 600 students with different levels of prior knowledge from different majors. We conduct face-to-face exercises assisted by 50 tutors who assist and evaluate the students and identified challenges including, sustaining hidden knowledge in case of alternating teaching staff, reacting adaptively when tutors are overwhelmed by a high number of students, monitoring the workload of students, and dealing with heterogeneous participants.
We suggest a preliminary socio-technical teaching framework which keeps the face-to-face tutoring as an essential social part. Tutors enter the evaluation of the students’ solutions in an app. This data enables course instructors to gain insight into the practical exercise sessions to further improve the learning experience for students in subsequent SE1 courses. So far, we implemented a proof of concept, but in the future, the complete framework has to become robust, secure and tested extensively to be deployed as it processes sensitive student data.