While in traditional learning scenarios, formative assessment relies solely on the judgment of teachers, technology enhanced learning provides means for automatically identifying room for e ective learning improvement. Given a digital learning environment and the support of learning analytics, it is possible to infer input for formative assessments. Teachers might be assisted by systems to adapt their teaching practice and to individually assess students. In this paper, we present evidence of this matter by reporting the experience of a teacher during a 13 week language course and by analyzing the students' content production in an online environment during the same period .
Formative assessment is an adaptive approach used by teachers during the learning process. Based on formal and informal assessments, teachers adapt proposed learning activities in order to improve students achievement. Typically, the assessments consist of qualitative and implicit feedback of the students’ performance and engagement, rather than exams’ results.
Black and Wiliam provide a much broader definition of formative assessment: ‘all
those activities undertaken by teachers, and/or by students, which provide information
to be used as feedback to modify the teaching and learning activities in which they
are engaged’ [
Similarly, Cowie and Bell define formative assessment as: ‘the process used by
teachers and students to recognize and respond to student learning in order to enhance
that learning, during the learning‘ [
Formative assessment enables students to restructure their understanding/skills and
build more powerful ideas and capabilities [
With the catch up of Web 2.0 technologies, online learners have gained more
influence in the whole learning process. Online e-learning systems provide learners with the
ability to interact, collaborate, and socialize, thus a ecting directly their self-directed
learning, as well as teachers and classmates actions. Consequently, assessments must
be adapted in order to promote e ective learning strategies [
In [
First, it is necessary to identify which aspects can be explored to infer learners’ development in a given system. Second, it is essential that teacher’s actions are tailored to equivalent environmental setups. One simple example: given an online forum where learners can post questions, teachers’ feedback is better appreciated if it comes contextualized, at the same place, using the same tool.
In this light, this paper provides two main contributions. First, we describe the inclass experience of a language teacher who has reported to e ectively apply formative assessments during an intensive Greek language course using Web 2.0 technologies. Second, we report a post-course analysis where we demonstrate that it is possible to infer useful data for formative assessment from simple user data on an online learning environment. 2
This study explores the application of formative assessment in a wiki environment that was used in a 650-hour Greek language course conducted at a Greek-speaking public university in the Republic of Cyprus. The course design aimed to combine online and face-to-face contact modes, in other words blended learning. The class met face-to-face every day for five hours for two semesters (total 26 weeks) in the academic year 20092010. The course was particularly designed to meet the needs of university students who planned to study nursing. In the first semester, the language and content were drawn from students’ experiences and some key learning areas such as nursing. In the second semester, the language and content were drawn exclusively from nursing. The course and the materials were tailored to meet the academic and professional needs of the nursing students. 2.1
The participants of the intensive course were four students (two female and two male) from Kenya, who came to Cyprus, for five years, on full scholarships. The students’ age ranged from 19-23 years. Their mother tongue was Swahili and they were very fluent in English (B2-C2 on the CEFR 1;). They did not have any knowledge of Greek. Their computer skills were in general at basic level. Therefore, a brief training session on basic computer functions and on the use of wikis was provided.
The instructor was a female, with two years experience in teaching Greek as a second language (L2). The instructor was both participant and observer and her role in the online environment provided her with access to the widest possible range of data. 2.2
A number of tasks were designed on the wiki during the course, including an online reflective journal, an online glossary and an online newspaper. Wikispaces2, one of the most widely used wiki software programs, was employed to create the course wiki and the students’ individual wikis. It was chosen because of its simple, user-friendly interface that allows page layout to be easily changed. Wikispaces is currently available in many languages, including Greek, which enabled students to develop their site in the target language.
The Greek09-CUT (a wiki created by the teacher) constituted an online interactive environment where learners could find useful course information, material for further study on a specific theme, and announcements, at anytime from anywhere. To incorporate wiki technology into the course, the instructor developed the course wiki and designed various tasks and activities. Through a series of worksheets and videos which were shown at the beginning of the course (week 1), the students learned about wikis.
Furthermore, they were provided with hands-on experience on how to browse the course wiki, and familiarise themselves with its use. The students were invited to become members of the course wiki and were thus allowed to edit material. The instructor provided continuous help, both in class, in the form of a step-by-step hand-out, and online by means of wiki notes to guide them to use Wikispaces. Five links were prepared by the instructor on the course wiki: – (a) Participants Homepage o ered the students the possibility to present themselves in Greek to the potential visitors of the wiki. – (b) Reflective Diary provided the students with an online space for adding their weekly reflections. – (c) Our Glossary hosted various sub-links, one for each of the thematic areas covered in class, where students collaborated to gather the basic words and develop an online database of new vocabulary of each thematic area. – (d) Our Newspaper included various sub-links of the topics a newspaper covers.
During the development of the newspaper, the students investigated the di erent elements typical of online newspapers and made their suggestions as to what could be useful and relevant in the construction of an online newspaper. The students continued to add material in these links until the completion of the foundation course. – e) Instructors’ reflections provided the instructor with the opportunity to give her point of view on the course development. 1 http://www.coe.int/t/dg4/linguistic/Cadre1_en.asp 2 http://www.wikispaces.com/
As previously mentioned, the students had access to an online wiki learning environment, where they maintained their Reflective diary. As explained to the students, a Reflective Diary is a way of thinking in a critical and analytical way about their work in progress, and generally about their lives in Cyprus. Basically, they were supposed to write about what happened and what they learnt over the weeks, on a weekly basis.
There were no restrictions on which language (English or Greek) they should use, and there was no evaluation or grading of the Reflective Diary - it was a self motivated learning activity. The students had access to each others’ pages and to the instructor’s reflections in order to enable the social pressure and to increase the students’ motivation to contribute.
The teacher provided assessment and suggested remedial actions during the course, both in class and online on the instructors’ Reflective Diary. Thanks to the students’ reflections, the instructor became aware of the students’ di culties and was thus able to take the necessary actions to smooth their learning and ease the cultural di culties they reported:
Things got harder and I realised that what I was doing was not enough. The verbs were pilling up in di erent forms and I had to change to be harder. Learning a new language is not easy but it also depends on how one takes it (S4, Week 6 Reflective Diary)
Moreover, as the students started writing their reflections in Greek, the instructor could informally assess their language progress and take actions to help them overcome their learning di culties. The students’ complaints on excessive cognitive load over the course were taken into consideration whilst implementing the course. Additionally, erroneous sentences in the students’ reflective diaries were isolated by the instructor and were returned to the students in class for correction, in order to enable them to discuss their mistakes and work together to correct them.
The students also shared good practices for studying and learning Greek on the wiki, which derived from their experience but also their emotional state of mind during the course:
I believe that being close to the instructor is a quality that should be practised by every student so as to fully understand. What you expect is what comes to your way and I always begin my day by saying that I believe that something wonderful is going to happen to me and it will be fulfilled (S4, Week 2 Reflective Diary)
Finally, the instructor’s Reflective Diary provided a platform for hosting impersonal comments and feedback on students’ overall progress. Having observed S1 and S2 disengagement from the wiki activities in the first weeks, the instructor posted the following on her reflective diary (week 7):
Set your own daily programme and follow it! Being a student should include both fun and study. In order to enjoy both, you should make a schedule and state clearly when you will be studying and when you will be relaxing. This is the only way to follow your obligations and enjoy your studies as well! Don’t expect a test to study! 4
We collected data from the four students’ contributions on the Reflective Diary during the first 13 weeks of the course. The data consist of quantitative information about each student’s contribution.
Table 1 exposes the overall averages of each student during the 13 weeks. We can see a dominance of students S3 and S4, who contributed much more than the other students. They were more engaged in the diary, producing longer entries and iteratively editing and correcting mistakes (more revisions). In addition to that, Table 2 quantitatively reports the level of engagement of each student by considering a single page (Participants Homepage) that was maintained during the course. Here, the di erences are not so evident as in the previous table.
Since the figures in the two tables hold a strong correlation ( >0.94 Pearson correlation coe cient), we only plotted the length values discriminated by week (Figure 1). The chart shows that students S1 and S2 did not work on the Reflective Diary for long periods. More specifically, they had six and five weeks of inactivity, respectively. In such cases, burst detection algorithms (in this case used to detect drops in activities) could automatically alert teachers of the loss of interest of some students.
As reported in Section 3, without any learning analytics tool, the course instructor was able to identify disengagement. Also, the instructor properly approached the students by using the same tools (Wiki) and without being intrusive. However, manually identifying disengagement is not scalable. In a group of four students, it is a rather simple task, but with a group of over 20 students it would become a real burden to instructors.
There are several possible methods to automatically identify student’s disengagement from a task, and, as we have seen in section 4, straightforward analyses are already able to provide awareness to teachers regarding students disengagement. 6
In this paper, we presented formative assessment from a teacher’s perspective during an intensive Greek language course. Additionally, we exposed that based on very minimal data, is possible to infer students’ engagement during the run of the course, and with the right analytic tools alert teachers to take actions and adapt lessons. According to the teacher’s report, in the case presented in this paper, online students’ engagement directly reflected students’ learning performance. Thus, it highlights the importance of approaching students individually in order to address their needs and to take actions before they completely abaddon their tasks. 7
We would like to thank the students who took the Greek language course and kindly agreed to let us evaluate and publish their contributions and outcomes.