=Paper=
{{Paper
|id=Vol-3618/scme_paper_3
|storemode=property
|title=Teaching conceptual modeling leveraging formative assessments and adaptive release paths
|pdfUrl=https://ceur-ws.org/Vol-3618/scme_paper_3.pdf
|volume=Vol-3618
|authors=Pavani Vemuri,Stephan Poelmans,Estefania Serral,Monique Snoeck,Robert Andrei Buchmann,Ana-Maria Ghiran
|dblpUrl=https://dblp.org/rec/conf/er/VemuriPSS23
}}
==Teaching conceptual modeling leveraging formative assessments and adaptive release paths==
https://ceur-ws.org/Vol-3618/scme_paper_3.pdf
Teaching conceptual modeling leveraging formative
assessments and adaptive release paths
Pavani Vemuri1,∗ , Stephan Poelmans1 , Estefania Serral1 and Monique Snoeck1
1
KU Leuven, 3000 Leuven, Belgium
Abstract
Teaching and learning conceptual modeling have been reported to be difficult tasks. In the past few
decades, there have been a myriad of studies and resources on how to simplify teaching, different teaching
methods, and frameworks proposed for conceptual modeling education. Nevertheless, there exist gaps
in helping educators construct facile study modules to help activate students toward achieving their
learning goals. In order to make the choices on pedagogy, method, and assessment for an educator easier,
we present three courses containing a business process management teaching module at different levels
of mastery (introductory, familiar, and advanced) in blended contexts at a higher education institute.
We examine and deliberate on several aspects of the course design. Through correlation analysis, this
study explores whether (1) there are any discernible effects of the design on outcomes by analyzing the
association between formative and summative scores; (2) the course design employed in all three study
modules possesses the necessary flexibility to accommodate various levels of mastery that are intended to
be achieved; and (3) the utilization of adaptive release of learning material effectively stimulates student
engagement and participation.
Keywords
Conceptual modeling education, formative assessment, formative feedback, learning report
1. Introduction - Importance of teaching conceptual modeling
Conceptual modeling (CM) plays a vital role in modern businesses, specifically in software
development lifecycles and in process (re)design and (re)engineering. Especially when using
poor quality models in the early stages of design, harmful effects are caused by the use and
application of these models [1] which can prove to be very expensive in later stages of design.
In describing several systems the field and role of CM have expanded dramatically including
the modeling of workflows, data flows, workforces, processes etc, utilizing the most commonly
used techniques and methods such as Unified Modeling Language (UML) or specific data models
like ERD [2]. In this study, we focus on process modeling (PM), which belongs to the realm
of CM, particularly in the area of information systems design, where it deals with dynamic
organizational processes that concern diverse audiences (domain experts, managers, employees,
ER2023: Companion Proceedings of the 42nd International Conference on Conceptual Modeling: ER Forum, 7th SCME,
Project Exhibitions, Posters and Demos, and Doctoral Consortium, November 06-09, 2023, Lisbon, Portugal
∗
Corresponding author.
Envelope-Open pavani.vemurii@kuleuven.be (P. Vemuri); stephan.poelmans@kuleuven.be (S. Poelmans);
estefania.serralasensio@kuleuven.be (E. Serral); monique.snoeck@kuleuven.be (M. Snoeck)
Orcid 0000-0001-8668-2011 (P. Vemuri); 0000-0002-4208-9723 (S. Poelmans); 0000-0001-7579-910X (E. Serral);
0000-0002-3824-3214 (M. Snoeck)
© 2023 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).
CEUR
Workshop
Proceedings
http://ceur-ws.org
ISSN 1613-0073
CEUR Workshop Proceedings (CEUR-WS.org)
CEUR
ceur-ws.org
Workshop ISSN 1613-0073
Proceedings
ICT staff, etc) and purposes in the clarification, redesign or automation of existing processes.
Novice designers and business analysts will be placed at an advantage in organizations if they
have developed strong modeling skill sets during their university education. However, teaching
requirements formalization through CM comes with its own set of challenges[3]. Acquiring
proficiency in PM is essential for effectively managing or automating business processes,
but learning PM poses challenges due to its inherent complexity, requiring specific cognitive
schemata and practical experience [4]. Enhancing the training of process modelers contributes
to the acquisition of necessary skills for PM, potentially addressing the prevalent problem of
subpar process model quality within organizations [5].
There have been several publications researching CM education and on the usefulness of
feedback in teaching CM in Higher education [6, 7, 8, 4]. But thus far, no standardized approach
has been established for teaching CM or incorporating feedback into the teaching process. While
there are myriad of educational materials available such as MOOCs, textbooks, and traditional
classroom courses for/on teaching CM as listed in [9], there is no one-size-fits-all approach.
Neither is there an understanding of how to support CM learning [10]. When offering/designing
a course with PM, a teaching team needs to address (1) at what level (basic introduction to
advanced knowledge/skill); (2) how (choose the blended scale, tools used, pedagogy); and (3)
how much (should be influenced by learning goals and assessments) to teach. The teaching
of PM occupies a prominent position on research agendas within the PM discipline [11, 12].
Nevertheless, in PM there are very few works connecting teaching PM with pedagogy or
instructional design. In the presentation of this work, we try to provide direction to educators
of PM in order to simplify these choices and disentangle the complexities of teaching PM using
Business Process Modeling Notation (BPMN). To this end, we present a PM module in three
courses at different knowledge/skill levels. While designing the courses discussed in this work, a
few major decision points were kept in mind to align them pedagogically, with existing research
and resources based on CM education.
2. Course design
2.1. Audience
The BPMN module was presented to business program cohorts enrolled in the academic year
2022-2023 at three different academic levels: Bridging program (preparatory for an Academic
Master for Non-Academic Bachelor), Academic Bachelor , and Academic Master. The cohort
following the Bridging program completed a Non-Academic Bachelor. The Academic Bachelor
cohort is in their third year of Bachelor in Business Engineering. The Masters’ cohort is
following a 1-year MBA. The courses are ICT Management (ICTM) in the Bridging program,
Business Information Systems (BIS) in the Bachelor program, and Business Process Modelling
and Information Systems Design (BPM & ISD) in the Masters program. All cohorts are considered
novice process modelers with none or very little modeling experience.
2.2. Learning objectives & evaluation
The details on learning aims, ECTS, and summative evaluation of the three courses ICTM1 ,
BIS2 and BPM&ISD3 can be found in the links below in the footnotes. The three courses have
distinct levels of learning goal complexities which we encapsulate pedagogically using Bloom’s
Taxonomy[13]. [6] applied the revised Bloom’s Taxonomy of educational objectives[13] for
domain modeling, identifying and defining 6 cognitive process levels and 4 knowledge levels
with examples in the context of learning CM. In designing the module, we enforce these
scaffolding levels and apply them to PM. We classified the contents starting from the evaluation
(summative assessment:SA), the learning objectives, learning items, formative assessments
(FAs) of the BPMN module of each course into different Bloom categories as defined in [6].
For example, the SA for the ICTM course only contains Multiple Choice questions (MCQs)
which classifies it into the Understand, Analyse, and Evaluate levels. For the BIS and BPM&ISD
courses, students have to model a given textual description using BPMN, which classifies these
into the Create level. Additionally, BPM&ISD students have to solve a real case study. Table 1
lists the course characteristics and how the Bloom’s taxonomy levels map to course components
and assessments. After following the courses ICTM, BIS, and BPM&ISD respectively, students
are expected to have an introductory, moderate, and expert level of familiarity using BPMN.
2.3. Learning modules (content) & mode of delivery
At first, the complete BPMN material was designed with some fixed elements (lessons and
practice exercise sessions) partitioned into BPMN Basics and Advanced subsections. In lieu of
the university’s response to post-pandemic circumstances and culture, where a reduced number
of students attended in-person classes but took the exam, most of the learning material was
required to be online. Lessons were planned as videos with each video having a central focus
on specific BPMN concepts. Interspersed between the 4 BPMN Basics and 7 BPMN advanced
videos, the FAs (6 quizzes; 2-8 exercises) were planned to help the student with self-evaluation
and self-regulation. These learning items were also offered through an online component, thus
giving autonomy to the learners to pace themselves and to work on it at home in case they
cannot come to class.
In-person sessions were set up for solving the exercises. During these in-person sessions, after
a quick review of the lectures, a few exercises are solved step-by-step to introduce how a textual
description can be interpreted and a process diagram be realized. Then, students are given
time to solve exercises by themselves and the instructor and teaching assistant are available to
answer any individual difficulties. A few possible solutions, common difficulties, and errors are
discussed for each of the exercises at the end. This is done to facilitate discussion and improve
on the quality of models. Exercises discussed in the exercise sessions for the three module levels
were chosen carefully giving particular attention to the difficulty level of the exercises. In ICTM,
more basic exercises were seen, and quiz MCQs were discussed as to how models had to be read
and comprehended. In the BIS course, a few quiz items were discussed with more emphasis on
1
ICTM: https://onderwijsaanbod.kuleuven.be//2022/syllabi/e/HSA17AE.htm
2
BIS: https://onderwijsaanbod.kuleuven.be//2022/syllabi/e/HBN63BE.htm
3
BPM&ISD: https://onderwijsaanbod.kuleuven.be//2022/syllabi/e/HMH28FE.htm
Table 1
Course and assessment characteristics
Course Characteristics ICTM BIS BPM & ISD
Academic Level Program Bridging Bachelors Masters
Semester Sem 2 Sem 1 Sem 2
OPO ID HSA17a HBN63 B HMH28F
Number of Students 75 37 44
ECTS Total course 3 6 6
ECTS BPMN Module 1.3 2 2.3
No. of teaching hours 8 hours 12 hours 14 hours
BPMN Tool used Signavio Signavio Visual Paradigm
No. of FA quizzes 6 6 6
No. of FA Exercises 2 4 8
Extra exercises 8 11 15
Summative Assessment (SA):
Possible SA points 20 20 20
SA Type Written exam (20) Written exam (20) Group Case (10) +
Written exam (10)
Questions types on SA MCQs Draw models Draw models
(BPMN only)
No. of points 6 (out of 20) 4 (out of 20) 5 (out of 10) Written
on SA (BPMN only) 5 (out of 10) Case
Bloom levels:
SA questions Understand, Create Create
Analyse, Evaluate
Teaching module Understand, Create Create
Analyse, Evaluate
Formative quizzes Understand, Understand, Understand,
Analyse, Evaluate Analyse, Evaluate Analyse, Evaluate
Formative Exercises Create Create Create
Exercise Sessions Create Create Create
modeling and understanding common errors for both basic and advanced BPMN. In the masters
level course BPM&ISD, the exercise sessions were intense with a strong emphasis on advanced
exercises. From table 1, one can notice that the total number of FA exercises varies per course
according to the expertise level of PM needed to be mastered in that course. The higher the
expertise, the FA exercises offered are more advanced. This flexibility in the learning material
is also reflected in the total number of extra exercises (quantity) and difficulty levels offered
during the in-person exercise sessions.
2.4. Formative assessments, feedback, and adaptive release
Effective learning is described as the process of appraising knowledge, understanding, and skills
[14]. In higher education, assessments are used mainly for measuring effectiveness. Beyond
evaluation, assessments can also influence several factors and purposes like feedback, motivation,
self-regulation, helping promote learning [15] and pushing the learners towards achieving their
learning goals. In the study modules at all three levels, to help with self-regulation, and for
the students to benefit from a structured study to progress towards the learning goals of the
BPMN module, FAs have been introduced in the courses as quizzes and modeling assignments
(exercises). Aligning with the SA and learning goals of the three levels of BPMN modules, the
number of FA exercises is reduced in ICTM (only 2 basic ones are offered) and BIS (4 offered- 2
basic, 2 advanced) as compared to 8 FA exercises (2 basic and 6 advanced) in BPM&ISD.
The Hattie and Timperley feedback model [16] advocates that feedback should be able to
answer simple questions (the what and how) which direct the student in progressing towards the
learning goals and what steps should be taken to reach them. Incorporating feedback into the
teaching and learning processes and its influence on the learning outcomes in different learning
contexts (traditional, blended, or online) have been well-established in the extant literature
[17, 14]. With the increased use of technologies in education, feedback is also being automated
by building it into the learning management systems (LMS) so that real-time, personalized
feedback is available to learners [17]. In the case of the 6 FA quizzes, the feedback is directly
built into the LMS quiz tool and video feedback via adaptive release for each FA exercise. In
line with the Bloom’s taxonomy levels on SA and the learning objectives of the ICTM course,
4 videos (solution to FA exercises) with feedback on common errors are available to students
(without adaptive release elements) to learn how to discern if a model is correct or not.
Specifically, in teaching how to make conceptual models and giving feedback to novice
modelers, [8] suggests that knowing and learning about the kinds of modeling errors that are
most likely produced, helps novice modelers in developing conceptual models of higher quality.
Bogdanova [9] takes it a step further by classifying errors into an ontology and aligning it
pedagogically with Bloom’s taxonomy and providing personalized feedback reports using a
semi-automatic tool. Following this prescription, offering personalized, individual feedback
for every modeling exercise is not always feasible for instructors in Higher education when
student cohort sizes are large. Offering group, online, or just-in-time teaching feedback sessions
have also been extensively proven to be great alternatives. To this end, during the in-person
exercise sessions, individual and group feedback with an emphasis on common errors is provided.
Similarly, for the FA exercises, feedback on the common mistakes and errors in the solutions has
been offered in feedback videos which become available via adaptive release when a formative
modeling assignment has been submitted. The adaptive release of learning material was chosen
in order to motivate students to solve the FA exercises before consulting the solutions directly.
3. Methodology
3.1. Data and context
The log data of the BPMN modules in the three courses was extracted from the institutional
LMS and the breakdown of summative scores for the BPMN questions on the exam was received
from the course instructors. The courses ran in the academic year 2022-2023 and were offered in
programs at the Faculty of Economics and Business at KU Leuven. Since the BPMN module was
only a part of larger courses, for the summative scores we consider only the BPMN questions on
the final exam. The percentage of students completing the FAs per course is shown in Table 2.
Table 2
Percentages of students accessing FAs: Quizzes (FA Qz) and Exercises (FA Ex)
FAs ICTM BIS BPM & ISD
FA Qz_1 80.0 62.16 59.09
FA Qz_2 82.67 54.05 54.55
FA Qz_3 73.33 51.35 52.27
FA Qz_4 74.67 48.65 56.82
FA Qz_5 62.67 37.84 52.27
FA Qz_6 56.0 29.73 50.0
FA Ex_1 13.33 37.84 70.45
FA Ex_2 13.33 37.84 52.27
FA Ex_3 - 29.73 45.45
FA Ex_4 - 27.03 43.18
FA Ex_5 - - 38.64
FA Ex_6 - - 34.09
FA Ex_7 - - 29.55
FA Ex_8 - - 38.64
3.2. Correlation study
Many previous correlation studies searching for associations between FAs and SAs as listed
in reviews [18, 19]. Usually, these studies present regression controlling for variables like
gender, previous academic performance, and attendance [20]; reflect on whether frequency of
FA attempts [21] influences summative scores; or by statistical test of the Pearson’s correlation
coefficient[22]. In our dataset, we do not collect any personal data due to strict GDPR guidelines
and to a large extent since such data as gender or prior achievement do not influence the teacher
and cannot be influenced by the teacher. Attending a class in-person is also left to the student
as a matter of choice or as a situational choice post-pandemic and hence this data was not
available to collect as well. In the BPM&ISD course, since the scores on the case study can be
influenced by group effects and not individual performance, we only include the written exam
scores and do not include the score for the Case study in the correlation comparisons. Pearson’s
correlation coefficient is used to indicate the relationship between the scores on the quizzes, the
number of quiz attempts, the exercise attempts, the number of views on the feedback video,
versus the summative scores on the BPMN questions.
4. Results and discussion
The results of the correlation study are presented in Figure 1. The figure shows the correlations
between the number of attempts, scores, and feedback video views associated with the FAs
(quizzes and exercises) versus the SA score. The top four rows correspond to the average values
of exercise attempts, quiz scores, quiz attempts, and (FA exercise) feedback video views. The
pattern in the bars indicates whether the adaptive release of the material was implemented
or not. In places where there are no bars, those FA exercises and/or feedback videos were
not offered in that particular course. A correlation of zero is clearly indicated. The color bar
indicates the p-value giving the significance of Pearson’s tests; with darker color representing a
higher p-value. When p<0.05 the correlation scores are significant.
The following are the key observations from the correlation study: (1) The ICTM and BIS
courses have the highest correlation between FA quiz scores with the SA score. (2) For the BPM
& ISD course, the highest correlation is for the number of FA exercise attempts. (3) The feedback
video views available through adaptive release do not correlate to SA scores in the ICTM and
BIS courses but are highly correlated (p<=0.05), especially for the advanced FA exercises in the
BPM&ISD course. (4) In the ICTM course, the feedback solution videos for advanced exercises
without adaptive release also have a high correlation with exam scores. In Table 2 we can
see that in the ICTM course, very few students have attempted the two FA exercises. This is
reflected in the insignificant and low correlation score for the FA exercises in Figure 1. Moreover,
the high correlation between the FA quizzes and the SA scores could be a reflection of the
course design as the ICTM SA contains MCQs (Bloom levels - Understand, Analyse, Evaluate)
and the FA Quizzes are on the same Bloom levels. In BPM&ISD, while more than 50 percent of
the students attempt all the quizzes (see Table 2), the correlation is not significant either for the
number of attempts or scores of FA quizzes. The correlation between FA (advanced) exercise
attempts and SA scores is high. This is expected (similar in ICTM) as the SA is composed of
modeling questions (Bloom level - Create) and FA exercises are on the same Bloom level as well.
In Table 2, across courses there is a tendency of students to not access later FAs. This could
be due to a drop in motivation towards the end of the chapter or giving up on FAs which are
more complex or simply due to content fatigue. This can be further investigated by surveying
students and collecting their perspectives and preferences.
In the BIS course, the pattern of Bloom-level correlation between FA and SA seemingly breaks
down. The FA quiz scores and attempts show a higher correlation with SA scores as opposed to
the FA exercise attempts. The SA of BIS course is at Bloom level - Create, whereas the high
correlation occurs with the FA Quizzes which are at lower Bloom levels. It is possible that
Bachelor level students are more focused on or have preferences for quizzes as opposed to
Exercises as learning items or for self-regulation. A second possibility is this choice could be
SA dependent - the SA of the complete course also included MCQs, so this cohort could have
anticipated MCQs for the BPMN questions on the SA. The differences in correlations (Figure 1)
and the percentage of students accessing formative tests (Table 2) could also be an effect of the
course design.
At the introductory or moderate knowledge/skill levels, only a few FA (basic) exercises (2
in ICTM and 4 in BIS) are offered. Perhaps offering a higher number of FA exercises (8 in
BPM&ISD) and a higher number of extra exercises during the exercise sessions (see Table 1)
would have shifted the students’ perspective towards solving more FA exercises. Moreover,
notice that the views on the feedback videos for advanced FA exercises (Ex 5 onward) whether
they are offered via adaptive release (in BPM&ISD) or not (in ICTM) show a positive correlation
with the SA scores. These observations can help instructors improve the next run of these
courses: (1) very simple exercises (specifically Ex1 and 2) can be dropped as FAs; (2) FA quizzes
can be dropped from BPM&ISD, (3) offer the feedback videos for advanced FA exercises in
the BIS course (similar to that of ICTM) without adaptive release; (4) adaptive release is more
relevant in BPM&ISD and hence this restriction on feedback videos can be dropped in other
Figure 1: Correlation between the formative and summative assessments
courses.
There are several limitations to this study. While most of the learning material was online
and the activity was collected through log data, in-person interaction, and attendance could
not be captured. These could also affect SA scores. For any correlation study it is important
to remember that it is not causation, so these results will have to be considered carefully. The
flexible course design is heavily dependent on both the teacher’s execution of the material
provided and the difficulty of the FA quizzes and exercises used in these courses. The FAs need
to be carefully chosen with the right alignment of learning goals. Also, this flexibility in course
design has been tested on only one module of three different courses. Other than BPMN, this
flexible course design could possibly be expanded to other CM modules. In the future, this
should be extended to complete courses of different mastery levels. Also, a higher sample size
will allow for more in-depth analysis leveraging machine learning techniques. Further analysis
is needed to dig deeper into the FA exercises that were used in the course. The feedback given
in videos explaining different solutions and the most common errors can be improved/updated
by correcting the BPMN diagrams submitted by the students.
5. Conclusion
A PM teaching module containing a fixed (lessons and exercise sessions) and a flexible part
(number of FA and extra exercises) was designed to pedagogically align the learning goals,
FAs, SAs and learning items with Blooms Taxonomy levels. This module was incorporated
into courses with three different expertise levels to be attained by cohorts following different
programs at a university. This exploratory work is aimed at bridging the gap that exists in
connecting PM modeling and instructional design. A correlation study was conducted to check
whether the different components of the course design had any effects on the summative scores.
It can be seen from the results that both formative quizzes and exercises can be effective learning
tools to teach PM. Specifically, in two of the three courses, we notice that when the Bloom level
on an FA matches that of the SA, the correlation is higher. The BIS cohort breaks this pattern
which could be possibly explained by a cohort’s preferences in learning material, or by teacher
effects. Further research is needed to ascertain if these trends can be observed across cohorts,
courses, or disciplines. From this study, we realize an incremental course design aligned with
Bloom’s taxonomy formative tests (exercises and quizzes) paired with feedback that makes for a
flexible solution for teaching modeling at different levels of expertise. Useful insights are gained
on aligning FAs paired with feedback for educators helping them align their teaching resources
and assessments better. In future research, we intend to focus on improving the feedback, by
analyzing frequent errors in student models, as well as considering student perceptions and
preferences in learning material.
Acknowledgments
Thanks to Bert Coenen, ICT, KU Leuven for providing the data for this study.
References
[1] H. A. Reijers, J. Mendling, J. Recker, Business process quality management, in: Handbook
on Business Process Management 1, Springer, 2010, pp. 167–185.
[2] R. Buchmann, A.-M. Ghiran, V. Döller, D. Karagiannis, Conceptual modelling in education:
a position paper, in: R. Matuleviius, R. A. Buchmann, V. epa, M. Kirikova, K. Sandkuhl,
M. Pańkowska (Eds.), 4th Workshop on Managed Complexity, 2019.
[3] G. Sedrakyan, M. Snoeck, Effects of simulation on novices’ understanding of the concept
of inheritance in conceptual modeling, in: M. A. Jeusfeld, K. Karlapalem (Eds.), Advances
in CM, Springer International Publishing, Cham, 2015, pp. 327–336.
[4] G. Sedrakyan, M. Snoeck, S. Poelmans, Assessing the effectiveness of feedback enabled
simulation in teaching cm, Computers & Education 78 (2014) 367–382.
[5] J. Claes, I. Vanderfeesten, F. Gailly, P. Grefen, G. Poels, The structured process modeling
method (spmm) what is the best way for me to construct a process model?, Decision
Support Systems 100 (2017) 57–76.
[6] D. Bogdanova, M. Snoeck, Domain modelling in bloom: Deciphering how we teach it, in:
G. Poels, F. Gailly, E. Serral Asensio, M. Snoeck (Eds.), The Practice of Enterprise Modeling,
Springer International Publishing, Cham, 2017.
[7] D. Bogdanova, M. Snoeck, Use of personalized feedback reports in a blended conceptual
modelling course, IEEE/ACM, 2019.
[8] N. Bolloju, F. Leung, Assisting novice analysts in developing quality conceptual models
with uml, Communications of the ACM 49 (2006) 108–112.
[9] D. Bogdanova, M. Snoeck, Let’s automate! making use of a learning ontology for conceptual
data modelling, volume 2469, CEUR workshop proceedings, 2019.
[10] C. Verbruggen, M. Snoeck, Exploratory study on students’ understanding of multi-
perspective modelling, Springer, 2022.
[11] J. F. George, S. Paul, R. De’, E. Karahanna, S. Sarker, G. Oestreicher-Singer (Eds.), Building
‘holistic’ Business process modelling skills for IS graduates, Association for Information
Systems, 2020.
[12] B. Rosenthal, Kristina; Ternes, S. Strecker, Learning conceptual modeling: Structuring
overview, research themes and paths for future research, in: Proceedings of the 27th ECIS,
Stockholm Uppsala, Sweden, 2019.
[13] D. R. Krathwohl, A revision of bloom’s taxonomy, Theory Into Practice 41 (2002).
[14] J. Gikandi, D. Morrow, N. Davisa, Online formative assessment in higher education: A
review of the literature, Computers & Education 57 (2011) 2333–2351.
[15] P. Marriott, A. Lau, The use of on-line summative assessment in an undergraduate financial
accounting course, Journal of Accounting Education 26 (2008) 73–90.
[16] J. Hattie, H. Timperley, The power of feedback, Review of Ed. research 77 (2007).
[17] G. Deeva, D. Bogdanova, E. Serral, M. Snoeck, J. De Weerdt, A review of automated
feedback systems for learners: Classification framework, challenges and opportunities,
Computers and education 162 (2021).
[18] S. M. Ismail, D. R. Rahul, I. Patra, E. Rezvani, Formative vs. summative assessment: impacts
on academic motivation, attitude toward learning, test anxiety, and self-regulation skill,
Language Testing in Asia 12 (2022) 1–23.
[19] F. M. Rizo, An empirical study on the impact of formative assessment: A literature review,
Revista electrónica de investigación educativa 14 (2012).
[20] K. Daniel, Assessing the benefits of online formative assessments on student performance,
Journal of learning development in higher education (2023).
[21] J. A. McNulty, B. R. Espiritu, A. E. Hoyt, D. C. Ensminger, A. J. Chandrasekhar, Associations
between formative practice quizzes and summative examination outcomes in a medical
anatomy course, Anatomical sciences education 8 (2015) 37–44.
[22] N. K. Mitra, A. Barua, Effect of online formative on summative performance in integrated
musculoskeletal system module, BMC medical education. 15 (2015).