Technology-based assessment has its origin in the early nineties where paper-pencil test was replaced more and more by assessment using technologies. Learning often occurs collaboratively in learner networks, formal learning is combined with informal learning, and learners increasingly use open educational offers such as MOOCS or publicly available Open Education Resources, personalised learning environments adapted to their needs and preferences, or they use specific workplace oriented learning services. Technology-based assessment are part of many learning environments with a large variance in item types and feedback formats. Formative assessment is a means to continuously generate feedback and to increase learners’ awareness about their learning progress as well as to support them in moving forward towards achieving the targeted learning goals. Formative Assessment has already demonstrated its positive impact on learning in general, and it can help learners to become self-regulated learners.
Formative assessments aim for a continuous assessment cycles with appropriate feedback. This challenges researchers and engineers to develop formative assessment solutions that scale on the hand (e.g., large amount of items needed) and which are aligned with the underlying competency (construct) models on the other hand. In addition, new technologies (e.g., augmented reality, wearables, etc.) are used in learning environments that provide new opportunities but also challenges to embed assessment into learning processes.
From a visionary perspective, technology-enhanced formative assessment will be integrated seamless in diverse technical learning environment, where it offers personalized learning experiences in different contexts, for everybody and for many relevant skills of the 21st Century. 2
On these bases, the TEFA workshop (Technology-Enhanced Formative Assessment, http://www.ccead.puc-rio.br/tefa2016/) focused on formative assessment as a support for learning in today’s innovative technology-enhanced learning (TEL) environments, and as a component for extending existing or new TEL approaches. The goal was to develop the notion of technology-enhanced formative assessment further combining expertise from pedagogy, educational measurement, cognitive science, and information technology.
The event was initiated by the special interest group on Technology Enhanced Assessment (SIG TEA) and the European Association of Technology-Enhanced Learning (EA-TEL: http://ea-tel.eu); it was held in September 2016 in conjunction with the 11th European Conference on Technology Enhanced Learning 2016 in Lyon, France.
The workshop provided an opportunity to bring together experts from different fields: technology-based assessment, educational measurement, IT&TEL, pedagogy, teacher education, educational psychology, etc., willing to share their experience about their work in the field of technology-based learning and to provide insights about how formative assessment could enhance motivation and learning in TEL environments.
These proceedings collect contributions of the best papers presented at the workshop as well as invited papers providing an interdisciplinary forum in which scientists and practitioners exchange new ideas and applications on technology-enhanced formative assessment.
The following contributions represent different approaches to innovate in formative assessment. They cover different phase of assessment (e.g. learning gap detection, rubrics design, data analysis, feedback processes etc.)
• • • •
How are established forms of formative assessment changing with the inclusion of technology and what new forms of formative assessment become possible? How is this reflected in new approaches to educational measurement? How can we build, maintain, communicate and negotiate learner models so that they maximize the value of formative assessment for learning? What is required to empower the users (students, peers, teachers, parents, employers, workers) to make best use of TEFA in order to guide learning? How can integrated and continuous formative assessment be supported by an ICT infrastructure? •
How can information technology help to innovate the ways in which TEFA methods are developed? 3
The following presentation led to a fruitful discussion during the workshop: • Buchem (Distributed Assessment with Open Badges of 21st Century Skills) presented Open Badges as an alternative form of assessment. She promotes Open Badges to help to recognise skills and other achievements from multiple sources. Badges offer a verifiable representation of a competency, skill etc. and can be shared across OBI-compatible portals. A case study with migrant academics demonstrates how distributed assessment works. • Fernandes Medeiros et al. (Towards a Personalized Placement Assessment Method to Support Students' Knowledge Gaps Identification) explained how teachers can be supported technically to identify learning gaps of their students. The adaptive assessment techniques make use of graphs and is based on the structure of the content taught. • Kiy et al. (Facilitating Portfolio-driven Learning in a Personal Learning Environment) elaborated on how e-portfolios can be integrated into personalized learning environments. Portfolio pages in form of Blogs are submitted by the learner to receive feedback from peers. In a second step the lecturer assesses the portfolio pages as well as the peer feedback to give a final grade. Results of first user tests with the platform were very positive. • Rusman et al. (Viewbrics: Formative Assessment of Complex Skills with Video-Enhanced Rubrics in Dutch Secondary Education) address the limited capacity of mental model building of text-based rubrics by offering videobased rubrics to learners. The design of a quasi-experiment was proposed at the early stage of this project.
In addition, two papers were invited to the proceedings of TEFA 2016: • •
Finamore et al. (A Case Study of a Probability and Statistics Course: Improving Students’ Performance with Formative Assessment) presented a case study of a Probability and Statistics course using a formative assessment approach as an attempt to improve students’ approval rates, identify students who need support and enhance the teaching and learning process. The proposed formative assessment was responsible for improving students’ performance and approval rates in approximately 11% of the cases. Morais et al. (Formative Assessment supported by Analytics Techniques: a case study on teacher and students perceptions) investigated which metrics can be used to better predict student’s dropout in a higher education context. Analysis techniques such as the K-Means clustering algorithm were used to cluster student interactions logged by the LearnWeb platform and to investigate the perception about the formative assessment process as well as to get a better understanding of students’ interpretation of feedback.
Program Committee • • • • • • • • • • • • • • • • • • •
Bernardo Pereira Nunes – PUC-Rio / UNIRIO Bill Warburton – University of Southampton Cristian Cechinel – UFPel – Federal University of Pelotas Davide Taibi – Italian National Research Council Desirée Joosten-Ten Brinke – OUNL Elena Caldirola – Università degli Studi di Pavia Elisabetta Parodi – Lattanzio Learning S.p.A.
Ellen Rusman – Open University of the Netherlands Eric Ras – Luxembourg Institute of Science and Technology Gilda Helena Bernardino de Campos – CCEAD PUC-Rio Giovanni Fulantelli – National Research Council of Italy Ivana Marenzi – L3S Research Center Marco Kalz – Welten-Institute – Research Center for Learning, Teaching and Technology Maria Bortoluzzi – University of Udine, Italy Maria Cinque – Università degli Studi di Udine / Fondazione Rui Maria Perifanou – University of Athens Patricia Jaques – UNISINOS Sean Siqueira – UNIRIO
Tania Di Mascio – University of L’Aquila
We, as guest editors and the SIG TEA, are grateful to all the authors who provided their valuable contributions, as well as to the reviewers who helped us in the review process, the participants of the workshop for the fruitful discussions, and finally the organiser of the ECTEL 2016 conference. We hope that readers enjoy reading this multi-facet collection dedicated to formative assessment. Further, we thank the organisation committee and in particular the workshop chair for providing us with the necessary rooms and equipment to conduct this event.
Eric Ras received his Ph.D. in computer science from the University of Kaiserslautern, Germany. He has worked for more than 15 years as a scientist and project manager on different public and industrial projects in the domain of technology-enhanced learning and knowledge management. Today, he leads the group Embedded Assessment at the Luxembourg Institute of Science and Technology, Luxembourg. His team manages several international projects in the field of technology-based assessment (TBA) and learning. With regard to technology-based assessment, his particular interest is to assess 21st Century skills with smart enabling technologies (e.g., tangible user interfaces or dialogue-based systems). Further, he researches how to automatize the assessment lifecycle, for example, by using knowledge models and Linked Data to generate test items for formative assessment. He has authored more than 100 scientific publications. He is PC member of different conferences in the domain of technology-enhanced learning and assessment and has organised many workshops and symposiums on the topic of assessment. He is chair of the Special Interest Group TEA (Technology Enhanced Assessment) of the European Association of Technology Enhanced Learning (EATEL). Next year, he will chair the 2017 International Technology Enhanced Assessment Conference (TEA2017).
Ivana Marenzi, PhD, is senior researcher at the L3S Research Center in Hannover, Germany. She obtained her PhD on the interplay between Content and Language Integrated Learning and Web 2.0. Her main area of research in Technology Enhanced Learning (TEL) includes the support of collaborative and lifelong learning. As educational technologist her main interest is dealing with issues related to the adoption of new technologies in education. She is referee for international journals (e.g. Elsevier LeaInD, IJTEL) and conferences, and has collaborated in organizing conferences and workshops in the field of TEL (e.g. ICWL 2016, EbTEL, TEFA 2013, 2016).
Bernardo Pereira Nunes holds a Ph.D. and master in computer science, postgraduate in pedagogical mediation for distance education and a computer engineering degrees from Pontifical Catholic University of Rio de Janeiro (PUC–Rio). Currently, Bernardo is an Adjunct Professor of the Department of Informatics at PUCRio, Collaborator Professor in the Computer Science Department at Federal University of the State of Rio de Janeiro (UNIRIO), senior system analyst of the Central Coordination for Distance Learning (CCEAD) of PUC–Rio and Associate Editor of the Brazilian Journal on Computers in Education. Bernardo has participated and led many academic and industry-oriented projects that have been awarded by Brazilian and International institutions. He has also has published about 50 scientific publications in peer-reviewed journals, and conferences, and worked as a research assistant at the L3S Research Center of Leibniz Universitä t Hannover in Germany and as a research visitor at the High Performance Computing Lab in the CNR Italy in Linked Data, TEL, Semantic Web, e-Learning, Data Mining and Web Archiving E.U. projects.