An essential element of higher education is the assessment of student work, either formative for improving learning or summative for looking back at what has been achieved. However, applying assessments as part of larger assignments is prone to some phenomena such as students not being aware of the quality of their work during the assignment or assessing at non-suitable moments in time, resulting in unnecessary low grades. In this work we discuss dichotomy-thinking as possible reason and how Hybrid Pedagogy as design guideline can help with nding appropriate solutions. Besides discussing this approach in general we also provide concrete examples of how it was applied for the design of assessment strategies in a course on software engineering.
Assessments, both formative and summative, form an essential element in higher education: they provide insight in the outcomes of student learning, o er opportunities for feedback and check whether learning goals have been met. A common course design comprises various assessment types such as written exams or obligatory tests, and often there are one or more larger and longer-running assignments as well. These assignments are usually assessed after the nal version has been handed in. Additionally, there are some moments when feedback is given or even an intermediate assessment, based on the student's current work status.
While this is an established approach, we still can observe some phenomena which potentially have a negative impact on student performance and hence hinder learning, even though in various degrees. With respect to timing and quality issues, the following speci c examples of such phenomena can be recognized: { Timing Phenomenon: Snapshot Assessments
In many longer-running assignments such as projects, case studies or research trajectories, there are some xed assessment moments: usually at the end of a course and somewhere midterm or after regular periods in time, e.g. every two weeks. These midterm assessments often result in lower grades because they are a snapshot in time, looking at work in progress. Many teachers use this approach for showing the students that they need to work harder and deliver better quality, hereby hoping that this motivates the students. But even though it might become obvious to students where their shortcomings are, the grade is given and usually can't be improved. This is unnecessarily frustrating, especially if the students actually are able to deliver much better quality, just not at this snapshot moment. It seems that in that case the assessment is somehow disconnected from the desired learning outcomes. { Timing Phenomenon: Feedback Timing
In many cases, students get feedback some time after they have handed in some work. Usually students have already continued working on the project or focus on other assignments, which makes it more di cult for them to relate the feedback to their work as it is not present in their heads anymore. However, there are also occasions where feedback is given directly, e.g. as part of an assessment session or during a work group. These moments are of more value as the feedback is easier to directly relate to the work, but these moments are usually also more rare as they require more time of the teacher. They furthermore often do not t into the planning of both students and teachers. { Quality Phenomenon: Quality Unawareness
Students often seem unsure about the quality of their nal work or have a rough feeling about the quality but are unable to predict the grade. Even if they apply self-assessment they are not sure if the teacher will come to the same result. Therefore the nal grades often come as surprise, either being lower or higher than expected which might result in decreasing con dence in a fair grading system. { Quality Phenomenon: Little Value of Feedback
Feedback which is given by teachers as part of assessments is not always experienced as valuable by students. It is often for looking back and not experienced as relevant for future work. If the feedback focus is on the quality of the work (which it should), then it is often experienced as not directly relevant for getting a higher grade. So its value is limited in a grade-centered educational system as it is present in most higher educational institutions.
There likely are many reasons for these phenomena. We assume that one
reason is the lack of awareness of alternatives. As a consequence, educational
designers of such assignment and assessment strategies tend to rely on well-known
standard solutions. A potential cause for the lack of alternative educational
design solutions might be a proneness to thinking in dichotomies, which was also
discussed for various other educational domains (see e.g. [
We believe that consciously intermingling these dichotomies|the core of the concept of Hybrid Pedagogy|might help with opening the space for new solutions which positively in uence the aforementioned phenomena and therefore can help with improving the design of educational strategies. In this work we discuss how this hybrid approach was applied for the design of assessment strategies which potentially have a positive impact on student performance and in consequence improve learning.
In the next section, we will shortly introduce dichotomies in education and describe the concrete dichotomies which have impact on the aforementioned phenomena. This is followed by an introduction to Hybrid Pedagogy and the description of concrete examples of how hybridity was used for nding alternative solutions which potentially in uence the phenomena as described above. The paper concludes with a summary and an outlook on future work. 2
A dichotomy can generally be de ned as two things that are on the outer ends
of a speci c dimension and often contrasting or opposing each other. Such
dichotomies can be encountered in many aspects of education. In mathematics
education, Sierpinska discussed the dichotomy of practical versus theoretical
thinking and the issues related to this [
Following these examples, we can identify some speci c dichotomies which might have impact on the above mentioned phenomena. We will discuss these also with respect to the timing and quality aspects.
{ Timing Dichotomy: Planning for Organization versus Planning for Learning
Fixed assessment moments are often dictated by organizational planning
issues and not intended as milestones, related to quality aspects. They are
dictated by the availability of the assessor, a certain moment based on the
total duration of the assignment (such as after each third or at the half of
the assignment duration), academic holiday planning and other similar
issues. This seems understandable as most educational institutions work with
xed time structures such as two semesters per year, courses with a
delimited duration of multiple weeks to few months or other timely restrictions.
It also happens that students take courses in parallel and that, with the
goal of making studying more easy, assessment moments are distributed in
a way so that students do not get overworked. The contrast would be to
adjust assessment to student's learning, meaning that assessment happens
when students have achieved a learning goal or created a product that
fullls some pre-de ned quality criteria. However, there are only few examples
related to this planning for learning-end of the dimension, one of them being
Programmatic Assessment [
With synchronous feedback, which is usually given in working groups or
face-2-face sessions, feedback given is more relevant because it arrives at a
teachable moment. Students are still engaged in working on the product, they
are still thinking about the task domain [
Asynchronous feedback o ers the advantage that the teacher has more time to deeply assess the product and to provide more detailed feedback. Planning is not a big issue, it usually does not matter much if the feedback arrives a bit earlier or later. However, when feedback arrives students likely already continued working on the product or even moved on to the next assignments or learning tasks. In both cases the feedback is arriving when they are not engaged in working on the product anymore and therefore of less value. The spare use of synchronous feedback, mainly because of planning issues, and the disadvantages of the more often applied asynchronous feedback mode are contributing to the phenomenon Feedback Timing. { Quality Dichotomy: Teacher versus Student Grade Determination Responsibility Some dichotomies are not directly recognizable as such. When thinking about the responsibility of determining the grades for students, most teachers would not dare to argue that this responsibility lies anywhere else than by themselves. Likely reasons are that they fear loss of control on the quality of the work or grade in ation due to student's over-assessment. This means in consequence that the students never will be fully aware of what quality level they have achieved with their product and in consequence what grade they could expect for it. Self-assessment does provide some help here, but is often applied independently from actual grading3 and therefore only partially helpful.
There are however examples where teachers have students indeed grade their
own work. If done well, most authors report various positive e ects of
selfgrading such as quicker and more detailed feedback for students, deeper
3 For example, the widely used learning management system Blackboard has two
distinct modules: one for assignments with grading and one for self-assessments.
understanding of the topic, and greater awareness of own strengths, progress,
and gaps [
Not involving students in the grading process is contributing to Quality Unawareness { Quality Dichotomy: Formative versus Summative Assessment As teachers we often either give feedback only|intended for supporting learning and improvement|or we provide a grade with some justi cation, usually after some work has been nished. These relate to the assessment functions of being either formative or summative.
Both are valuable but also have some shortcomings: even though formative assessment helps the students to know where they stand, they are dependent on the teacher to provide them with this information. This feedback is also often experienced as todo-list by the students, potentially resulting in the e ect that elements of their work where no speci c feedback is given on are seen as good enough. Furthermore, the feedback which is valued most by students is which parts already are good enough for getting a su cient grade. This kind of feedback does not trigger a growth mindset as they likely won't do more work on parts which are already of su cient quality. It keeps the students reactive.
Summative assessment on the other hand is mainly for looking back. Its relevance for the students is often limited, as the work has been nished and the students usually already moved on to the next assignment or course. Only intrinsically motivated students see the value of such feedback as being relevant for future work as well.
The distinction between formative and summative assessments contributes to student's experience of Little Value of Feedback.
The above described dichotomies are four examples which we assume to have impact on the phenomena. In the next section we will discuss how these dichotomies can be addressed by using hybridity as explicit design guideline. 3
As also described by Heames and Service, applying kaleidoscope thinking|using
another viewpoint when something seems di cult from a certain point of view|
is a good start for developing solutions [
Rorabaugh and Stommel describe hybridity as follows:
"[...] hybridity suggests hesitation at a threshold. Hybridity is not an attempt to neatly bridge the gap, but extends the moment of hesitation and thereby confuses easy categorization. And, as we allow two things to rub against each other, two things that might not otherwise touch, we invite them to interact, allowing synthesis (and even perforation) along their boundaries. As the digital and analog|the physical and virtual| commingle, we must let go of the containers for learning to which we've grown accustomed. We must open to random acts of pedagogy|to connections that are, like the web, associative and lively but sometimes violent and deformed. In this, hybridity is not always safe, moving incessantly (and dangerously) toward something new|something as yet undetermined." [15, unpaginated]
Hybrid Pedagogy is not a new concept and there are a growing number of
examples of hybrid practices in education [
{ Solution: Self-Grading
In the example in Figure 1, the unused solution space suggests to share the responsibility of determining the grades with the students. This could be done by sharing the responsibility with them (in various degrees) or even by completely moving it to the students (as applied in self-grading).
The semester we designed was for the second year of a part-time study Computer Science. The students were a diverse group with many already doing professional work in the eld. Part of this semester was a long-running case study with various aspects of software engineering such as requirements elicitation or software design. As we wanted to integrate academic and workplace learning and wanted to increase the value of the study for them, we also applied the hybrid practice of Bring Your Own Assignment4. This means that the students could decide themselves about the content of the case study as long as they were able to ful ll the assessment criteria described in a set of rubrics.
However, assessing these work products would have cost the teachers much more time due to the potential variety of used techniques and application domains. Besides that we also wanted to help students being more aware of the quality of their products. The solution was to put the initial responsibility for grading to the students themselves and moving the responsibility of the teacher to determining if the result of this self-grading was correct. Whenever students think they achieved a certain quality level in a product, based on a self-assessment using the provided rubrics, they were encouraged to apply self-grading and hand in a grading request.
Applying this self-grading helped the students to become more aware of the quality of their work. This e ect was increased by adding the following practice of grade motivation. { Solution: Grade Motivation
To make sure that this self-grading is done appropriately, we added the
requirement that not only a grade (based on rubrics) had to be requested,
but that also a su cient motivation had to be added in order to show that
the quality of the work is in accordance with the rubric quality level and
associated grade. This motivation had to be not just a repetition of the
rubric descriptions, but a thorough underpinning of the achieved quality
level. Figure 2 shows an example of a complete grading request including (1)
for whom it was, (2) for which assignment (the case study) and rubric, (4)
the requested grade according to the quality description in the rubric, (4)
the motivation for the grade, (5) the reference to the actual work product,
and (6) a link to the grading queue tool (see Grading Queue solution below).
4 Bring Your Own Assignment [
The rst idea during designing the semester was to de ne some of xed assessment moments. But when looking at it using the hybridity viewpoint, the option came up that assessments can take place whenever students think they've achieved some certain quality levels for (parts of) their work. This idea was also triggered by the usage of learning outcomes and the fact that some of the students already created some products, e.g. at their professional work, which could serve as evidence that they already achieved the learning outcomes. So we interchanged the snapshot moments with student-de ned milestones. In consequence, grading is not limited to only looking back at some nished work, but was also applied to partial work results whenever these reached some pre-speci ed quality levels as described in the rubrics. The grades are combined with feedback which is still relevant to the work as it can be used for improvement until the nal deadline. This way, grades and the associated feedback are used for looking back and forward, making them a more valuable combination. { Solution: Grading Queue
While synchronous direct feedback is valuable, it is often hard to realize in su cient quantity. The idea was to make asynchronuous feedback as synchronuous as possible, hereby combining the advantages of both. In the example in Figure 4, this means that feedback does not have to be either synchronous or asynchronous, but that it also could be asynchronous in such a timely manner that it feels more synchronous (and also has the bene ts of synchronous feedback).
The resulting practice is a Grading Queue5. After students performed a
Self-Grading, including a Grade Motivation, they had to add an issue to
the queue (in our case a Kanban board), hereby letting the teachers know
that they had performed a self-grading and are now waiting for feedback
(see Figure 5 for an example). The teachers then picked the longest waiting
requests, examined them and returned them with corresponding feedback.
The e ect was that most grading requests were handled not longer than 1 or
2 days after they were handed in. This way the feedback that came with the
handling was given close to the moment of nishing that part of the work. In
consequence this feedback was given during learning and students could still
act on it, some of the characteristics of e ective feedback [
Another important aspect for adding value to the feedback is to have students
react on it. The above described solutions allowed such reactions. Additional
elements of the applied assessment strategy comprised these practices:
Act On Feedback [
Repair It Yourself [
In this work we described how we applied Hybrid Pedagogy as guideline for addressing some remaining phenomena of standard assessment strategies. First experiences show that these practices indeed help with addressing the phenomena. However, further research is needed to more thoroughly evaluate their e ectiveness.
We believe that explicitly using hybridity as guideline during educational design can help to widen the solution space and to identify potential alternative practices that help to address existing challenges in educational strategies. Exploring this approach will also be part of future work. Further research is needed to determine if and to what extent the approach of using Hybrid Pedagogy as design guideline is also applicable in other educational domains.