This paper presents a segment of the Erasmus+ ILEDA project that focuses on the analysis of student satisfaction and overall success of implemented project-based learning (PBL) activities in teaching and learning during PBL piloting in a programming course. Goals of this paper are to: (i) analyze student satisfaction with the implemented PBL methodology, and (ii) identify good practices and challenges encountered during the implementation. The implemented methodology was in line with the established ILEDA project guidelines, encouraging students to collaborate in teams to address real-world problems using programming knowledge. The research methodology adopted a combination of qualitative and quantitative methods, analyzing student individual and group reporting for qualitative insights and student surveys for quantitative results. This paper contributes valuable insights into the practical application of PBL in higher education, shedding light on student satisfaction, successful implementation strategies, and challenges faced during the process.
During the COVID-19 crisis two main modes of delivery have distinguished themselves in higher
education, full online learning and blended (hybrid) learning modes. Both require innovative
tools to support teaching and learning, in ofering flexible learning pathways, which include
a mix of digital solutions for diferent pedagogies, approaches and technological platforms.
Student retention in online and blended courses is critical, as shown in diferent studies about
e-learning and that has become especially evident during the COVID-19 pandemic [
Active learning methodologies have emerged as powerful strategies in education, transforming traditional pedagogical approaches by actively engaging students in the learning process [4]. Whereas traditional learning methods rely on lectures and rote memorization, active learning encourages students to participate in activities that promote problem-solving, critical thinking, and meaningful interaction with course content [5]. This paradigm shift places learners at the center of their educational experience, fostering a dynamic and collaborative environment.
In active learning, techniques such as group discussions, case studies, problem-solving exercises, and hands-on experiments, are employed to stimulate student involvement and deepen their understanding of complex concepts. Project-based learning (PBL), as one of the most used active learning methodologies is switching the classroom dynamics by immersing students in real-world, hands-on experiences [6, 7]. In contrast to traditional methods, PBL shifts the focus from passive reception of information to active engagement through student engagement in projects. This approach allows students to explore, inquire, and solve authentic problems, fostering a deep understanding of content, simultaneously improving critical skills such as collaboration, communication, and problem-solving. By presenting students with challenging, open-ended tasks that mirror the complexities of the professional world, PBL not only cultivates subject-specific knowledge but also nurtures a sense of curiosity and self-directed learning [8, 9].
This paper is organized as follows. Section 2 focuses on the application of PBL and lesson design used when preparing the course. The methodology used is presented in Section 3, and the results, both quantitative and qualitative, are presented in Section 4 with appropriate discussion. Finally, concluding remarks are given in Section 5.
Within the Erasmus+ ILEDA project detailed guidelines were developed on how to implement active learning methodologies in online and blended courses. The goal of these guidelines was to inform teachers how to create a more motivating environment for learning, especially in online learning environments and utilize learning analytics to follow student progress during the semester. This section presents project-based learning methodology that was developed within the ILEDA project and implemented in pilot courses. This methodology covers lesson design, project planning, lesson flow, and in-course activities.
In the planning phase following activities were taken: 1. Planning learning outcomes for each lesson; 2. Planning learning outcomes that should be achieved with the project; 3. Planning the project placement in the lesson (it should be decided whether the project is lesson based, or if it covers more lesson); 4. Planning how feedback will be collected (i.e. rough drafts, journal reflections, requirements, brainstorming plans, entrance and exit tests, checklist, observations, participating in discussion, report submission, code submission); 5. Creating project assessment plan (developing a comprehensive assessment plan, incorporating teacher-graded assessments, peer assessments, or a combination of both); 6. Creating project calendar and timeline (creating detailed project timeline with specifying date and deadlines, lesson plans, teacher and student activities, and availability of assessment resources); 7. Defining an open-ended driving question (this driving question will be an overarching theme guiding the project, stimulating critical thinking and problem-solving skills).
For each lesson where project-based learning was implemented lesson flow was carefully planned using following steps: 1. Each lesson’s Introduction starts with clearly stating the learning outcomes of the lesson and gives a real-world problem that is chosen for the lesson topic and assigned project, in order to motivate students to be more engaged in the topic of the lesson. 2. Lesson’s content should be designed in such way to cover diversity of content formats such as text, videos, code examples, etc., while keeping in mind that content should be in line with lesson’s learning outcomes. 3. Additional teaching materials that will contribute to students achieving learning outcome should be a part of the lesson teaching material and can include diferent examples and solved problems. 4. Projects are introduced to students through previously defined project assignment containing clear assignment requirements, rules, and deadlines, 5. System for progress reporting includes collecting data from learner and is implemented in the Learning Management System (LMS) through diferent phases of the project, and should collect data through brainstorming, observations, and final reports, 6. Teacher should record assessments for each progress report (here either teacher and/or peer assessment grading can be used), 7. Within each lesson students should be assessed about acquired knowledge, for instance, through self-assessments or graded tests, 8. Students are graded based on their assessments, homework assignments and project assignment, and additionally students are given a grade for the course participation, which evaluates their engagement and contribution of work throughout the semester.
When launching and implementing project-based learning following guiding principles were implemented when planning activities in class: 1. Project initiation should be initiated with a launch event with group discussions; 2. Form groups of 4-5 students or allow individual student work, catering to the nature and requirements of the project; 3. Teacher facilitated discussions guiding students on what they need to know to solve the driving question; 4. Implement ongoing student feedback such as: a) Each student shares what they are struggling with. Students discuss possible solution; b) Teacher generates the questions about the projects. Students are grouped in groups of 3-4. Students take turns in presenting their project to other students in the group (or part of the project). They discuss how to improve the project.
In this paper analysis of students’ satisfaction with PBL approach is performed as well as overall success of PBL teaching. PBL was piloted in the programming course CS100 – Introduction to programming-Python. This is a mandatory course for all first-year students at the Faculty of Information technology at BMU. CS100 was attended by 159 students including traditional and online students, and piloting was performed only for traditional students. Course has 15 lessons and lessons from 7 to 12 were chosen for PBL implementation. Methodology followed established ILEDA project guidelines, i.e., students were encouraged to work in teams and solve real-world problems through a project using programming knowledge. Preparation before PBL implementation included planning and lesson flow. After introducing a general PBL topic – Pollution mitigation, 24 PBL groups with 4-5 team members were formed. Groups worked through 6 weeks on the project split into 6 smaller projects. In the last week of course each group presented their project and gained results. For purposes of detailed analysis of PBL approach during piloting, data were continuously collected using Learning Activity Management System (LAMS) and using student surveys. Every week during PBL implementation students were uploading their individual and group reporting through LAMS. At the end of the course, students were asked to fill out the survey and answer 16 single choice questions and 3 free form questions. Students’ answers on one choice questions were scaled according to Likert scale: 5 - I agree, 4 I partially agree, 3 - I neither agree nor disagree, 2 - I partially disagree and 1 - I disagree.
Research methodology combined qualitative and quantitative methods. Students’ break points-individual and group reporting and answers on free form questions were analyzed for gaining qualitative results and students’ answers on single choice questions were analyzed for gaining quantitative results.
For better understanding of student impressions during PBL implementation, student individual reports for each lesson are categorized in four categories: 1. Reporting on individual contribution without stating problems in working with a team or anything else; 2. Stating stronger contribution of individual members; 3. Did not report on anything, stating how they will organize their work or reporting on group efort; 4. Reporting problem with the course (material).
Also, selection of responses for the open-ended questions are classified in three categories: 1. Positive impressions; 2. Negative impressions; 3. Improvement suggestions.
This kind of categorization can help in getting a better insight about a student’s satisfactory level and for possible improvements of PBL approach.
Student single choice answers were analyzed using descriptive statistics - mean, standard deviation and median. Also, for better understanding of significant relationships between collected answers, survey responses were analyzed using correlation matrix with Pearson’s coeficients that included statistical significance. In order to analyze the overall success of each PBL group, final grades of all students who participated in the piloting were examined.
Figure 1 shows an example of PBL lesson plans for weeks 7 and 8.
From Table 1 we can conclude that students were generally satisfied with organization of the course, learning materials, communication with the teachers and gained practical knowledge. Also, students were partially satisfied with the PBL approach. They were partially satisfied with PBL tasks, some of them think that tasks could be more interesting and more practical. They were partially satisfied with equal engagement of all members in the group. These answers were very helpful for understanding some drawbacks of the PBL approach when it is implemented for first-year students, which can be quite challenging. PBL approach can be a more suitable approach for the higher year students, and needs more careful project/task design and team coordination for the first-year students. PBL learning can be very encouraging and motivating when PBL tasks are well designed and adjusted to a specific subject and student level of knowledge.
Figure 2 shows the correlation coeficient matrix for survey answers, which mostly emphasizes a positive correlation between questions. The degree of correlation was classified as small (0.10 − 0.29), moderate (0.30 − 0.49), and high (0.50 − 1). We have found that the strongest correlation exists in course organization, as is reflected in communication with the professors (Q1 and Q5, < 0.001). However, this strong correlation is not present with teaching assistants (TAs), suggesting that the TA’s were not accustomed to incorporating PBL in their tasks. Furthermore, it should be noted that only the teachers and one TA had any PBL training prior to the course piloting, which may reflect the results.
The overall satisfaction with the inclusion of PBL was positive, as stated by a number of points. Firstly, the PBL-based teaching contents were interesting and allowed students to master the material easily (Q9 and Q19, < 0.001), and stimulated their creativity (Q9 and Q15, < 0.001). The students agreed that the PBL tasks were well designed and had no trouble using the LMS to upload their assignments (Q10 and Q11, and Q11 and Q16, both with < 0.001).
No correlation was found regarding the interplay of PBL with a course aimed to cover basic concepts of procedural programming (Q8 and Q10), suggesting that this approach should work better with a course starting from at least 2nd year of studies. A low and negative correlation of −0.01 from Q2 and Q16 suggests that students who did not find the course itself interesting also were not satisfied with the PBL reporting. For students who had prior programming knowledge, they did not agree that the course material (Q7 and Q3), nor PBL (Q7 and Q9) was applicable in practice. From this we can infer that students with prior knowledge were not motivated enough.
Each week students submitted their individual and group progress reports, in which they reported on their progress and potential problems they had. Individual student reports were analyzed for each lesson where project-based learning was implemented. The goal of this analysis was to identify dynamics of team collaboration and potential problems in their learning. Student reports, based on their content, were classified in several categories. Each category had similar features addressing potential problems students had. From reading all of the responses in all five lessons, following categories were identified: • Reporting on individual contribution without stating problems in working with team or anything else; • Stating stronger contribution of individual members; • Did not report on anything, stating how they will organize their work or reporting on group efort; • Reporting problem with the course (assignment).
The categorization of student answers for each lesson, as far as the percentage of answers who were placed in each category is presented in Figure 4.
A significant number of students reported on their individual contributions without indicating any problems in team collaboration. It is worth noting that some of these responses were superficially stated making it hard to really identify the level of contribution, making these individual reports potentially needed classification as lower-level contributors to the team. From certain answers it could be inferred that students did not contribute evenly, which was concluded from their either low-level tasks they stated that they performed (for instance, students reported their task was to search for information on the Internet, without having any programming activities that week). Also, students stated in individual reports that they intentionally did less work that week, as they agreed to have uneven distribution of work every week, and that the following week they will redistribute the project tasks among each other according to their dificulty level.
As the student progress reports were tracked from week to week, each week the distribution of work within groups showed variations. Several teams decided to adopt approach of completing the task assignments individually and later on combine them for the group submission. On the other hand, some groups reported that they had dificulty in understanding the assignment and needed more clarification. Most of these issues were clarified and solved throughout the semester by course instructors, however, this still points out the necessity to conduct careful planning and define clear and comprehensive instructions for PBL assignments.
Future implementation of PBL should consider implementing progress tracking by allowing the progress reports to be cross-referenced with other individuals from the same PBL group, as the manual tracking is time consuming and ineficient. This will contribute to analyzing individual team contributions and team work dynamics.
Furthermore, additional individual contributions to the team and collaborative eforts can be also analyzed through the implementation of the group peer assessments. In this evaluation students were allowed to submit individual progress reports in free form, however, it may give more insights if the reporting is guided with predefined questions that will dictate type of the information they will submit such as the level of communication, individual contribution, lack of work from some group members, and list of potential questions, etc.
Besides the progress reports, this paper uses another source of data for qualitative analysis, and this includes open ended questions from the survey. Student open-ended answers were analyzed and classified in categories (Table 2). Method that was used for the answer categorization is the method proposed by Miles and Huberman [10]. Categories that were chosen for categorization of students’ answers are: (i) improvement suggestions, (ii) positive impressions, (iii) and negative impressions. Positive impressions of students mainly revolved on their impressions that the course and the instructors were well organized. Students stated that they found the course interesting, well-organized, and excellently structured. Noteworthy are students’ positive impressions regarding their improved teamwork skills, creativity and gained knowledge. On the other hand, negative impressions revolved around challenges in team dynamics, including uneven distribution of responsibilities and dissatisfaction with team members. While some students found certain assignments confusing, other had concerns about the course being too challenging for beginners.
While there was a certain focus on student answers in positive and negative impressions, improvement suggestions covered a broad range of topics, from needing a clearer assignments instruction for students and better organization of PBL in the course, to needing the clearer connections between course assignments and lesson content. Some students proposed reducing the number of team members, incorporating more individual exercises, and introducing lfexibility in assignment requirements.
Analyzed data implies that students value the gained knowledge and practical skills, however they have addressed several challenges they witnessed during the PBL implementation, particularly regarding teamwork distribution and the clarity of PBL assignments. This pinpoints the need for the future improvements of PBL implementation in the analyzed programming course including refining assignments, implementing clearer communication channels, and potentially reconsidering the level of dificulty for beginners.
In this paper, we have presented the implementation of PBL active learning methodology incorporated in an undergraduate introductory programming course. The intent of this paper was to do an exploratory study of student satisfaction, conducted on a sample of students taking a single course. We have analyzed how satisfied the students were after completing the course with newly implemented PBL methodology, as well as their overall perception of the contribution of PBL in their learning.
The major advantages of implementing PBL can be summarized as follows. Firstly, students are given concrete problems to tackle on their own, reflecting examples during classes and exercises. Secondly, as their PBL assignments were team-based and with a deadline, their collaboration was emphasized, reflecting real-world situations with employees working within a team. Finally, with real-time progress tracking through weekly submission deadlines, the teaching staf was able to detect students and/or teams at risk of falling behind.
For future work, we intend to extend the use of PBL to courses in which problems are more tangible, and which can be solved using prior knowledge and competencies gained throughout the studies.
Research presented in this paper is supported by the project “Improving online and blended learning with educational data analytics” ILEDA - ref. number 2021-1-BG01-KA220-HED000031121, co-funded by the Erasmus+ Programme of the European Union. The paper is co-supported by the Ministry of Science, Technological Development and Innovations of the Republic of Serbia ref. no. 451-03-47/2023-01/200029. Information technology Trends for a global and Interdisciplinary research community, IGI Global, 2021, pp. 1–18. [2] A. Aristovnik, D. Keržič, D. Ravšelj, N. Tomaževič, L. Umek, Impacts of the covid-19 pandemic on life of higher education students: A global perspective, Sustainability 12 (2020) 8438. [3] M. Á. Conde, A. Georgiev, S. López-Pernas, J. Jovic, I. Crespo-Martínez, M. Raspopovic Milic, M. Saqr, K. Pancheva, Definition of a learning analytics ecosystem for the ileda project piloting, in: International Conference on Human-Computer Interaction, Springer, 2023, pp. 444–453. [4] R. M. Neves, R. M. Lima, D. Mesquita, Teacher competences for active learning in engineering education, Sustainability 13 (2021) 9231. [5] S. Hartikainen, H. Rintala, L. Pylväs, P. Nokelainen, The concept of active learning and the measurement of learning outcomes: A review of research in engineering higher education, Education Sciences 9 (2019) 276. [6] P. C. Blumenfeld, E. Soloway, R. W. Marx, J. S. Krajcik, M. Guzdial, A. Palincsar, Motivating project-based learning: Sustaining the doing, supporting the learning, Educational psychologist 26 (1991) 369–398. [7] D. Kokotsaki, V. Menzies, A. Wiggins, Project-based learning: A review of the literature,
Improving schools 19 (2016) 267–277. [8] J. Thomas, A review of research on project-based learning, Autodesk Foundation (2000). [9] M. J. Lage, G. J. Platt, M. Treglia, Inverting the classroom: A gateway to creating an inclusive learning environment, The journal of economic education 31 (2000) 30–43. [10] M. B. Miles, A. M. Huberman, Qualitative data analysis: An expanded sourcebook, sage, 1994.