Infusing reflective practice in eLearning courses - Can widgets help?

Abstract

This paper explores the potential of widgets for reflection, viz. instructional widgets that would be designed to prompt and support students' reflection in eLearning courses. The investigation builds on a literature review, an inventory of 35 reflection techniques and a survey amongst eLearning course owners. The outcomes demonstrate that a large part of the reflection techniques are acknowledged by teachers as being of relevance for their courses. Yet, practical application in distance education is quite rare. Results of the survey are used to ascertain possible contribution of software widgets to the implementation of some of these reflection techniques.

Keywords

• eLearning
• Widgets
• reflection
• Open Content
• meta-cognition

1 Introduction

This article is positioned at the cross-section of a new Internet technology and a new pedagogical trend by combining a new breed of software applications called widgets with the call for more reflection in learning.

As for the first trend, the term "widget" refers to a miniature Web application performing a single task and displaying a very clear and appropriate graphical style [ 1 ]. It provides a single interaction point for the visualization and direct manipulation of a given kind of data [ 2 ]. Typical examples would be widgets that show today’s weather forecast, upcoming birthdays or information stocks, designed for the desktop, the Web or the mobile. Personal learning environments are already taking advantage of widgets. More broadly, the technical approach based on widgets has become available to eLearning in general whilst it is not yet clear how they can best be used within a formal context of instruction. This paper opens a line of inquiry about "widgets for reflection", namely widgets designed to prompt and support clear, small and single reflection-related tasks occurring prior, during or after a learning sequence.

With regard to the second trend, reflection is an active process of witnessing one’s own learning experience and evaluating it on different aspects. Reflective practice (and akin notions like "learning to learn" and "meta-cognitive development") is a highly influential factor of learning [ 3, 4, 5, 6 ], enhancing both the domain-specific knowledge and the knowledge about the self-as-a-learner. However, current instruction shows a shortage of training for this generic skill [ 5, 7, 8 ]. Quantity, quality and persistence of reflection might substantially go up if the offer of tools for reflection could be enriched. In this respect, the option of "widgets for reflection" would nicely align with the call for more reflection in learning. This paper presents a survey amongst teachers on techniques for reflection. It uses the outcomes to identify which of these techniques are feasible candidates to an implementation as software widgets. It discusses the potential of these specialized widgets in transforming a learning environment so that it can become supportive for meta-learning training.

2 The survey: Reflective Practice in eLearning courses

The survey was carried out amongst 22 owners of an eLearning course who partook in the Open Educational Resources project (OpenER) of the Open University of the Netherlands [ 9 ]. The OpenER-project makes available a variety of higher education eLearning content free of charge. Like similar initiatives over the world (MIT OpenCourseWare, MERLOT, OPENLEARN, etc.), it targets an expansion of the higher education learning opportunities. The choice of OpenER courses for the survey has three reasons. First, the Open University of the Netherlands has defined a program aimed at enhancing its offer of open educational resources. Second, the course owners are experienced developers of eLearning content. Third, the research was conducted in the context of the i-Coper project which is dedicated to open educational resources. Course owners were asked to fill in an electronic questionnaire in regard to opportunities for reflection displayed in their courses. The survey is presented below. 2.1

Objectives of the survey

The objective of the survey was threefold: - to investigate the state of affairs of reflection amplifiers in OpenER courses. Reflection amplifiers are defined as structured opportunities for students to examine and evaluate aspects of their learning experience; - to give insight about the relevance of reflection in the eyes of instructors; - to ground the discussion about harnessing widget technology to instructional objectives into data coming from practitioners. 2.2

Method and presentation

In March 2009, 22 owners of an Open Educational Resource course of the Open University of the Netherlands answered to an online questionnaire which presented the description of 35 existing techniques meant to amplify reflection, as reviewed and categorized by Verpoorten, Westera & Specht [ 10 ]. For each reflection amplifier, respondents were asked to tick one of the following options: I do not understand this technique / This technique is not relevant for my course / This technique would be relevant for my course but is not implemented / This technique is implemented in my course. This type of investigation was chosen in order to find out the extent to which reflective approaches to learning are implemented in existing courses, and which concrete techniques are considered as relevant by faculty. The research was exclusively based on the answers given by participants. No reality check was done in the courses. The small size of the sample, the subjective dimension of the answers and the fact that the survey was the entry point to a larger investigation concerned with meta-learning practice in formal education account for the presentation of results in plain numbers and not in more sophisticated measures and correlations. 2.3

Results

Overall, 13 course owners (cfr. Fig. 1) out of 22 completed the questionnaire. In view of the 35 reflection amplifiers that were presented, this means that the study collected 455 (13 x 35) practitioners' qualifications over reflection amplifiers. Below, the aggregated outcomes are summarised.

Understanding of the reflection amplifiers. The reflection amplifiers seem to be

well understood. Only 23 out of 455 answers "I do not understand this technique" were collected. Amplifiers the least understood, that is amplifiers for which the option "I do not understand this technique" has been the most often ticked, are Formative assessment (4/13), Structure for regulative support (3/13), On-demand assessment (3/13), Confidence-Based Learning (3/13).

Relevance of specific reflection amplifiers. Respondents, 73 times out of 455, claim

that a specific amplifier would be relevant for their course but is not implemented. Amplifiers with the most potential are: Help seeking behaviour guide (4/13), Graphical presentation of contents (4/13), Students set the test (4/13), Indicators of understanding (4/13).

Overall relevance of reflection amplifiers. When grouping the answer categories

"This technique would be relevant for my course but is not implemented" and "This technique is implemented in my course" versus "This technique is not relevant for my course", it gives 157 claims of relevance versus 275 claims of non-relevance. (The 23 "I do not understand this technique" are not taken into account). So, a large part of the reflection techniques are recognised as being of relevance for the eLearning courses. Existing practice. According to respondents, 82 out of 455 reflection amplifiers are implemented in the courses. Highest occurrences: Making pedagogical rationale transparent (9), Metacognitive modelling (8), Self-explanations (6), Practice of evocation (4), Justify your choice (4), Graphical presentation of contents (4), Room for choice (4). Used amplifiers are unevenly spread in the courses (cfr. Fig. 1).

We analysed data to find patterns of aggregation of reflection amplifiers but no significant one could be identified, not even at the level of one-one combinations.

Focus on the exploitation of tracked data. Several reflection amplifiers are based on the mirroring of personal tracked data. The study reveals that 7 out of 13 course owners don’t know whether the eLearning platform on which they developed the course provides any tracking facility. Overall, 3 respondents state that they use tracked data as a teacher. Only 1 respondent says that the tracked data is used by the students. When asked whether they would give their students access to their learning traces as a reflection amplifier, 4 teachers out of 13 answer positively (cfr. Fig. 2). The questionnaire survey was carried out to investigate the practitioners' views about reflection in eLearning courses. From this empirical output, four observations are derived regarding possible contribution of widget technology to the facilitation of a reflective approach to learning. These observations should be seen as working hypotheses that ought to be further refined and tested. 3.1

Widgets for reflection can capitalize on interest for reflective practice

Supported by literature about the relevance of reflection skills, widgets for reflection may also find supporters amongst teachers. According to the present study, course owners express a fairly high rate of relevance regarding techniques for fostering reflection (157 out of 432). This may even be an underestimation because of the specific context of open educational resources. Some of the respondents used the open comments section in the questionnaire to express practical disclaimers for not using reflection amplifiers: three respondents stressed that they had limited time and resources to devote to the design of the course. They also state that the request of the Open Educational Resources project was just striving for digital content in order to have a few courses available as soon as possible. Despite these adverse conditions, 36% of the reflection amplifiers submitted to the whole group are ticked as relevant. 3.2

Feasible candidates to “widgetisation” can be found

Teachers gave their opinion about 35 reflective techniques. Most of these techniques are too complex to be used as widgets, according to the definition given in the introduction1. Amongst others, this probably holds for "On-demand assessment", "Portfolios", "Students set the test", "Help seeking behaviour guide", etc. However, "widgetisation" seems possible for the following techniques. - Growing progress visualization tool: the widget would offer visual displays (e.g. progress sliders, understanding meters) enabling learners to determine their progress (actions and mastery) towards the learning goals. 3 respondents out of 13 consider this feature as relevant for their course; - Comparison with yardstick: the widget would specialise in comparing certain aspects of the learning process (time spent, exercises completed, estimation of knowledge, own performance, etc.) with some yardstick (teacher, peer, expert, classroom average, oneself in similar circumstances, compliance ratio, etc.). 7 respondents out of 13 consider this feature as relevant for their course; - Indicators of understanding: the widget would prompt learners to qualify their understanding of the course with simple indicators like "lost/foggy/got it" or some similar labels. 7 respondents out of 13 consider this feature as relevant for their course; - Judgement of learning: the widget would allow students to report the progress they believe they made in the learning domain or objectives as a consequence of doing the course. 7 respondents out of 13 consider this feature relevant for their course; 1 It is not excluded that changes in the technology may make a different definition appropriate (such an evolution of definition can be observed, for instance, with learning objects) or that advanced widget-like techniques could address these complex techniques or complement them (it could, for example, be the case with smart indicators [ 11, 12 ]). - Self-efficacy judgments: the widget would engage students in self-assessments of their perceived level of knowledge or ability for a task. 7 respondents out of 13 consider this feature as relevant for their course; - Mirroring of personal tracked data: the widget would allow a visualisation by the learners of different interactions they had with the course. 3 respondents out of 13 consider this feature as relevant for their course.

Once developed and embedded in the courses, such specialized widgets would represent self-contained learning activities. Each of them could be formalised as a triplet [13, p. 3] of: (a) at least one tool. Example: "I use the widget "Understanding indicators"; (b) an action. Example: "With the widget, I rate my understanding of this content"; (c) an outcome. Example: "Thanks to this widget, and through the clear, small and single action it allows, I train my meta-learning skill for selfassessment". 3.3

A pick-and-mix, progressive and cheap approach to reflective practice

No single outstanding reflection amplifier emerges from faculty's answers and no preferred combination either. It means that teachers can just pick out one or several techniques as relevant (cfr. Fig. 1). The modular approach conveyed by widget technology, and more broadly by Web 2.0, seems quite suitable to cater for these variations. An individual teacher could select and aggregate (or not) widgets for reflection according to the courses, the student's needs or the level of reflection to be pursued. In such a pick-and-mix approach, inclusion of opportunities for reflection, within the framework of regular instruction [ 8, 14 ], might be tailored and progressive. Moreover, such a widget-driven evolution of already existing courses is more conceivable and cheaper than introducing new courses. Also, when applying these tiny and not much disruptive appliances, users (teachers, learners) are shielded against the need of dropping the existing VLE and having to get acquainted with a completely new system. Furthermore, long-lasting results cannot be expected from one or even a few reflection exercises. Many of them must take place during the academic year, over several years and throughout different courses. The agile nature of widgets for reflection might ease this deployment and concur to the acquisition of reflective habits. The survey also suggests that research on widgets has a value on its own and does not need to be tightly coupled with research into Personal Learning Environments. eLearning courses can be relevant containers for widgets. 3.4

Widgets and tracking and tracing data can be mutually supportive

The survey delivers ambiguous answers regarding contemplation of personal tracked data as a lever for student's reflection. On the one hand, to have students pondering upon their interaction footprints is granted some potential by practitioners (cf. Fig. 2). On the other hand, 7/13 course owners don’t know whether their eLearning platform provides any tracking facility. They do not use the traces themselves and do not know whether students do. Several studies indicate that teachers [ 15, 16 ], students [ 17 ] and learners [ 18 ] can reap meta-learning benefits from the observation of their traces. Making this data available through specialised tracking and tracing widgets is likely to boost the extent of this practice. It implies to establish links between tracking and tracing facilities and specialised tracking and tracing widgets. From an application point of view, such widgets would remain single objects but their semantics, visual appearance, dependencies and overall development can become very complex and demanding, as already observed in an early article on the topic [19, p.3].

4 Conclusion

Looking at reflection as a desirable educational goal induces the quest for instruments that are likely to foster it. This article considered the possibility of harnessing the agility of widgets to the training of thinking skills, within the framework of subject matter instruction. Based on literature, an inventory of 35 reflection techniques and a small-scale survey amongst teachers, it was argued that the development of widgets for reflection is a promising means to the infusion of certain types of reflective practice in eLearning courses. Due to its specific features – agility, interoperability, self-contained activities, aggregation power – widgets technology seems especially appropriate: - to increase opportunities for instant and focused reflection within a particular learning task; - to support an extended training of auto-cognitive skills (awareness during study, self-assessment, presence-to-learning) by embedding widgets for reflection within a variety of courses and systems; - to provide teachers with ready-to-use reflective tools likely to be seamlessly activated according to the configuration they find the most pedagogically relevant; - to facilitate cognitive regulation of personal learning by providing coordinated access to a variety of personal tracked data.

These promises should now be transformed into proper examples and associated hypotheses which, in turn, should be tested through additional research. In this respect, the authors are currently creating an eLearning course prototype enriched with concrete examples of widgets for reflection. This course prototype is meant to provide a convenient context for research on conditions of use, impact and possible drawbacks and benefits of these pedagogically and technologically innovative learning tools. At this initial stage, widgets for reflection will be developed for the Moodle platform as "Moodle modules". Technical feasibility, possibilities of truly smooth integration, acceptability and familiarity conditions will be studied. Through experimental setups, it is planned to explore effects of various displays, groupings, sequencing, and coordination of reflective tools on different dimensions of the instructional design and learning processes. If the experiments carried out in this particular environment deliver evidence of instructional benefits that buttress the interest expressed by teachers, issues related to the interoperability of the useful widgets in various learning environments will be addressed.

References

1. Wilson, S.: Wookie Widget Developer's Guide. (2008) Retrieved 23-06 , 2009 , from http://getwookie.org/Widgets_files/widget_dev_guide.pdf

2. Gui Widget , n. d. ( 2009 ). Retrieved 27-06 , 2009 , from http://fr.wikipedia.org/wiki/Wikip%C3% A9dia:Conventions_bibliographiques

3. Wang , M. , Haertel , G. , & Walberg , H.: What Influences Learning? A Content Analysis of Review Literature . Journal of Educational Research , 84 , 30 - 43 ( 1990 )

4. Marzano , R. J.: A theory-based meta-analysis of research on instruction . Office of Educational Research and Improvement , Department of Education, Mid-continent Regional Educational Laboratory, Aurora, CO ( 1998 )

5. Watkins , C. : Learning about Learning Enhances Performance . Institute of Education, University of London, London ( 2001 )

6. Schön , D. : Le praticien réflexif. À la recherche du savoir caché dans l'agir professionnel . Editions Logiques , Montréal ( 1994 )

7. Claxton , G.: Expanding the Capacity to Learn: A new end for education? Keynote presented at the British Educational Research Association Annual Conference, University of Warwick ( 2006 ).

8. Csapó , B. : Improving thinking through the content of teaching . In H. Hamers, J. van Luit & B. Csapó (eds.) Teaching and learning thinking skills , pp. 37 - 62 . Swets and Zeitlinger, Lisse ( 1999 )

9. Schuwer , R.: Zicht op OpenER Resultaten en effecten van een experiment met Open Educational Resources . Open University Netherlands ( 2008 )

10. Verpoorten , D. , Westera , W. , & Specht , M.: A classification framework of techniques for amplifying student's reflection in eLearning courses ( 2009 ) (submitted)

11. Glahn , C. , Specht , M. , Koper , R.: Smart indicators on learning interactions . In: E. Duval, R. Klamma , M. Wolpers (eds.) Creating new learning experiences on a global scale . LNCS , vol. 4753 , pp. 56 -- 70 . Springer, Berlin, Heidelberg ( 2007 )

12. Diagne , F. : Instrumentation de la supervision de l' apprentissage par la réutilisation d'indicateurs: Modèles et Architecture, dissertation . Université Joseph Fourier, Grenoble ( 2009 )

13. Moedritscher , F. , & Wild , F. : Why not Empowering Knowledge Workers and Lifelong Learners to Develop their own Environments? Paper presented at the I-Know 09 , Graz , Austria ( 2009 )

14. Resnick , L. , & Klopfer , L . (Eds.). Toward the Thinking Curriculum: Current Cognitive Research . Alexandria, VA: ASCD. ( 1989 ).

15. Scheuer , O. , Zinn , K. : How did the e-learning session go? - The Student Inspector . In 13th International Conference on Artificial Intelligence in Education (AIED'07) , Los Angeles ( 2007 )

16. Mazza , R. , Dimitrova , V. : Visualizing student tracking data to support instructors in webbased distance education . In: 13th International World Wide Web Conference (WWW 2004 ) - Educational Track . New York ( 2004 )

17. Johnson , M. , Sherlock , D. : Personal Transparency and self-analytic tools for online Habits . In: TENCompetence Workshop Stimulating Personal Development and Knowledge Sharing , Sofia, Bulgaria ( 2008 )

18. Glahn , C. : Contextual support of social engagement and reflection on the Web, dissertation . Open University of the Netherlands, Heerlen ( 2009 )

19. Swick , R. , Ackerman., M.: The X Toolkit: More Bricks for Building User Interfaces, or Widgets for Hire . In: Proceedings of the Usenix Winter 88 Conference , pp. 221 - 228 ( 1988 )