Supporting the Development of 21st Century Skills: Student Facilitation of Meetings and Data for Teachers

Abstract

This paper proposes providing teachers with real-time accurate and pedagogically-relevant information to assist students in the development of 21st Century skills, across subject areas, using a variety of technologies and data sources. We suggest that, while allowing students to practice skills such as meeting facilitation, recording activities both directly and indirectly (student and peer reporting) will likely be a useful step in supporting students in their acquisition of such skills, while helping teachers guide development in their students through visualisations of their students' competencies.

Keywords

• facilitating meetings
• student competencies
• teacher support

1 Introduction

With the development of technology, it is necessary to consider how to support students in their acquisition of 21st Century skills that will remain important in their future; and to help teachers in their understanding or monitoring of students' competencies, to enable them to better support their students' development. As an example to span many subject areas, and incorporating 21st Century skills, we use the case of student meetings. In principle these may take place face-to-face or online (e.g. through web conferences or virtual worlds), and may or may not be supported by specific meeting-facilitation technology (e.g. [ 1 ]).

To support student skill development, teachers need access to data about their competencies related to meeting planning, facilitation and outcomes. We provide this in the Next-TELL project [ 2 ] through student self- and peer-evaluations, and some simple automated methods. This information forms part of the ‘learner model’, which is provided visually to users, as proposed to be useful for teaching analytics [ 3 ].

We consider this to be a contribution to the challenges of ‘classroom orchestration’ [ 4 ]. Because it is a challenging task for a teacher to micro-manage group work (with or without ICT) in a classroom with many students, software such as LAMS (http://www.lamsfoundation.org/index.htm) has been developed to mitigate teachers’ cognitive load. In our approach, we aim to go beyond activity tracing (e.g. as implemented with LAMS) by providing the teacher with visual information on students’ on-going learning on the competency level. Our approach also makes intentionally more use of the students themselves as a resource for managing collaboration: by putting a few students in the role of meeting facilitators, the teacher has fewer activity management tasks to deal with and can concentrate on the overall classroom process, rather than individual groups. By the same token, students are provided with authentic opportunities to practice preparing and running on-line and face-to-face meetings, a competency that is valuable both inside and outside schools. 2

Meeting Facilitation: A 21st Century Competency Next-TELL considers three areas with reference to student-led meetings: planning meetings; facilitating meetings; and documentation and communication of outcomes. Each contains competencies represented in the learner model.

The meeting process begins when an individual determines that a meeting is required. One approach is a short yes/no ‘Should You Meet?’ checklist that can be used for the identification of whether a meeting is necessary. For example, from Francisco’s 8 item checklist: “Can you state the purpose of your meeting?”; “Do you have the information you need to meet productively?” [ 5 ]. Seibold further emphasises the importance of identifying the specific purpose(s) of the meeting and delineating a range of goals, to create the basic structure of the meeting [ 6 ].

A crucial step is to decide on the group composition for the meeting, to ensure that all those affected, are represented [ 6 ]. The tasks involved – including allocation of roles and responsibilities – can be seen to form their own competency. In addition, all members should be briefed on the points through an agenda, allowing individual feedback prior to the meeting [ 6 ]. The importance of group participation before the meeting starts has also been highlighted, with a series of steps to improve preparations before a meeting [ 7 ]. This includes introducing complex issues at one meeting and deferring discussion and questions for the next meeting; and the importance of supplying written materials to participants in sufficient time prior to the meeting. Participants should then read the material in advance, and perhaps have discussions beforehand. Updating the agenda may result before a meeting [ 8 ].

It is suggested that an agenda should comprise three basic points: (i) the topics to be discussed; (ii) the desired outcomes for each topic; (iii) processes needed to achieve the desired outcomes [ 9 ]. A variety of activities may be used in a meeting which can be teamed with relevant level of involvement and time estimates in order to specify the processes needed to achieve the desired outcomes. For clarity, desired outcomes should be split into: the overall goal for the topic (what final result should be achieved to complete the topic); and the meeting goal (what narrowly defined, specific objective should be achieved for the topic at an upcoming meeting). These goals do not necessarily have to be written in the agenda, but should be explicitly stated during the course of the meeting [ 9 ].

Once the composition of the meeting has been decided, all appropriate group roles should be delineated, responsibilities assigned and authority delegated where necessary [ 6 ]. It is suggested that at least two key roles of chair and secretary are necessary in a meeting scenario [ 10 ].

To conduct a meeting in a meaningful way, it is necessary to balance creative and critical thinking to productively support discussion and decisions [ 5 ]. A facilitator might use different types of intervention strategy if problems develop during a meeting, for example: interpretation (shifting focus to the process, describing, inviting discussion); direct action (interrupting meeting flow e.g. preventing interruption, encouraging an individual) [ 11 ].

A meeting facilitator may effectively set the frame by describing: the task, the outcome, the process, the rationale for the process, and the expected amount of time required [ 9 ]. Data resulting from group discussion or brainstorming may need to be sorted using pre-defined criteria or creating categories, and the list may need to be reduced through prioritising items.

A meeting exit survey may be used to evaluate a meeting, for example: "How well did we use the time allotted?", "How well thought-out were our decisions?" [ 5 ]. Exit survey questions can also address the skills of individual participants, such as "How effective was the facilitator?" Indeed, questions similar to these have been used in consultancy, in relation to evaluating the cost-effectiveness of meetings, such as: “The meeting leader…runs meetings effectively”, “…listens carefully and actively”, “…creates an environment where people are comfortable disagreeing” [ 12 ].

While a meeting must be documented by capturing minutes and noting actions [ 5 ], this need for a record is often overlooked by students and, indeed, some textbooks do not adequately cover all types of minutes in different settings [ 13 ]. Thus, it is particularly important for teachers to ensure students’ awareness of this requirement.

Different technologies can provide different levels of system support to meeting facilitation, e.g. from complete automation with no facilitator function, to simply providing support for recording and reporting information [ 8 ]. As Next-TELL is designed for use in a range of subjects and settings, we concentrate on generic tasks. This makes it particularly difficult to rely on detailed automation, hence the focus on self- and peer-evaluation of meetings, in addition to simple automated measures, in our approach to providing competency data in easy-to-use visual form, for teachers.

3 Information for Teachers

As stated in the Introduction, visualisations of students’ competencies in the form of their learner model, are made available to teachers. Such ‘open learner models’ (OLM) may be visualised in a variety of ways (see [ 14 ]), and our current work is building on the already common use of skill meters and smilies for younger users in OLMs [ 2 ], and developing state-of-the-art visualisations in the form of zoomable treemaps and word clouds. The data in the learner model, as described above, comes from some simple automated methods and student self- and peer-evaluations.

In order to provide teachers with information on students’ competency development in an automated manner, we provide students with a tool to plan their meeting facilitation, and compare the meeting as planned with the meeting as conducted. Planning a meeting consists of sequencing a number of group activities, which are represented as meetlets. Meetlets combine the description of a series of steps (e.g. the steps necessary to have a group perform a brainstorming activity) with a specification of the tools and artifacts with which to conduct the activity. For instance, for a brainstorming activity this could be a (collaboratively edited) Google Spreadsheet document. Meetlets also contain information about how to evaluate the success of the activity; e.g. in a brainstorming meetlet, this could be the number of ideas generated. This is used to update the competency model of the facilitator and/or group members.

Figure 1 depicts how information in the meetlet structure is used to drive a specific meeting activity and to appraise/assess an activity. For the case considered here, the meeting is conducted online. The descriptions of the sub-steps of an activity (for instance, for a brainstorming activity this may include eliciting and combining individual ideas) are interpreted by an Activity Stepper that guides the team members through these steps, and then rules describing how the resulting artefacts (e.g., individual and collective idea lists) are to be appraised, are applied to the artefacts. This artifact appraisal information is then used to update the learner model.

We are considering simple appraisal rules that build on information directly available in the artefacts. For example, for brainstorming,the number of individual ideas, collective ideas, and the ratio between them, can be used to formulate appraisal rules. More advanced rules could calculate the semantic overlap between ideas generated individually and those proposed as a group solution, but are at present not implemented. Our focus is currently on how to represent knowledge that is typically formulated by teachers in rubrics in a way that can (in principle) be interpreted by machines.

Manual input to the learner model may come from peers and the students themselves, provided as follows, on a scale of 1 (strongly disagree) to 5 (strongly agree), using the example of self-assessment by the meeting facilitator: • I created an agenda • I correctly allocated roles and responsibilities in the agenda • I distributed the agenda in time for other participants to feed back before meeting • Before the meeting I updated the agenda based on comments, as necessary • I set the frame before each task (introduced task; clear outcome; process; rationale; duration) • • • • •

I organised the meeting well I made decisions after full analysis of all factors I communicated the next steps and action plan effectively in the meeting I assigned all tasks and actions to the right people, with clear follow-up deadlines I reviewed the next steps and action plan at the end of the meeting

The automatically calculated and student-provided appraisal data is brought together in a (currently simple) quantitative learner model, which in turn is available to the teacher (and usually also to students). Figure 2 shows excerpts for some of the meeting competencies (upper – skill meters; lower left – word cloud (under development); lower right – the reverse of lower left (i.e. competencies not yet demonstrated) (under development). These displays can be used on-the-spot by teachers to provide feedback to students, and to offer feedforward (guidance) as to which meeting facilitation competencies to develop further. (While these representations are simple, for immediate teacher reaction, more detailed information including evidence is also possible [ 15 ].) The granularity of display can be determined by the teacher (e.g. “creating an agenda” could be further split into Kaner’s three points: topics to be discussed; desired outcomes for each topic; processes necessary to achieve the outcomes [ 9 ]. Teachers can use the visualisation(s) that best fit their purpose or preferences at the time. For example, for a quick, on-the-spot decision about where a group needs help, “setting the frame” clearly stands out as needing improvement on the lower right of Figure 2. In contrast, the lower left of Figure 2 indicates that a meeting is probably already well-planned, and perhaps the next phase should now be considered. The skill meters (top of Figure 2) provide a more organised, quick visual context. They can also display competencies from different areas (here Communication in English for Norwegian speakers) - one way to practise English is through discussion, in meetings. While open learner models have somewhat of a tradition in intelligent tutoring systems, they may also meet the need for providing teachers with accurate and pedagogically relevant information in real-time, when built on top of other educational software. 21st Century “skills” is a particularly interesting case, because these skills do not fit into just one teaching area (e.g. maths, science, language), but are ‘horizontal’ in nature. Hence, opportunities to learn and practise the skills required to develop the competencies should be provided across classrooms, teachers, and school years. The combination of an OLM with an infrastructure for recording learning-related activities both directly (e.g. with Google Doc APIs) and indirectly (e.g. through student and peer reporting) may prove a practical step to support students in learning and practicing horizontal skills, and to support teachers in monitoring and guiding such learning.

Acknowledgement

This project is supported by the European Community (EC) under the Information Society Technologies (IST) priority of the 7th Framework Programme for R&D under contract number 258114 NEXT-TELL. This document does not represent the opinion of the EC, and the EC is not responsible for any use that might be made of its content.

References

1. Vivacqua , A.S. , Carreira Marques , L. , Ferreira , M.S. & de Souza , J.M. ( 2011 ). Computational Indicators to Assist Meeting Facilitation, Group Decision & Negotiation 20 ( 5 ), 667 - 684 .

2. Reimann , P. , Bull , S. , Halb , W. & Johnson , M. ( 2011 ). Design of a Computer-Assisted Assessment System for Classroom Formative Assessment, CAF11 , IEEE.

3. Vatrapu , R. , Teplovs , C. , Fujita , N. & Bull , S. ( 2011 ). Towards Visual Analytics for Teachers' Dynamic Diagnostic Pedagogical Decision-Making , Proceedings of Learning Analytics and Knowledge , ACM , New York, 93 - 98 .

4. Dillenbourg , P. , & Fischer , F. ( 2007 ). Basics of Computer-Supported Collaborative Learning , Zeitschrift für Berufs- und Wirtschaftspaedagogik 21 , 111 - 130 .

5. Francisco, J. ( 2007 ). How to Create and Facilitate Meetings that Matter , The Information Management Journal 41 ( 6 ), 54 - 58 .

6. Seibold , D. ( 1979 ). Making Meetings More Successful: Plans, Formats and Proceedures for Group Problem-Solving , J. of Business Communication 16 ( 4 ), 3 - 20 .

7. Weston , H. ( 2009 ). Successful Board Meetings: Participation Begins Before the Meeting Starts , The Corporate Government Advisor 17 ( 2 ), 17 - 21 .

8. Macaulay , L. & Alabdulkarim , A. ( 2005 ). Facilitation of E-Meetings: State-of-the- Art Review , Proc. of E-Technology, E-Commerce and E-Service , IEEE, 728 - 735 .

9. Kaner , S. ( 2007 ). Facilitator's Guide to Participatory Decision-Making, Jossey-Bass, CA , USA.

10. Jonker , C.M. , Schut , M.C. , Treur , J. & Yolum , P. ( 2007 ). Analysis of Meeting Protocols by Formalisation , Simulation and Verification, Computational and Mathematical Organization Theory 13 ( 3 ), 283 - 314 .

11. Viller , S. ( 1991 ). The Group Facilitator: A CSCW Perspective , in L. Bannon, M. Robinson & K. Schmidt (eds), Second European Conference on Computer-Supported Cooperative Work , Amsterdam, 81 - 95 .

12. Rogelberg , S.G. , Rhoades Shanock , L. & Scott , C.W. ( 2012 ). Wasted Time and Money in Meetings: Increasing Return on Investment , Small Group Research 43 ( 2 ), 236 - 245 .

13. Wolfe , J. ( 2006 ). Meeting Minutes as a Rhetorical Genre: Discrepancies Between Professional Writing Textbooks and Workplace Practice , IEEE Transactions on Professional Communication 49 ( 4 ), 354 - 364 .

14. Bull , S. & Kay , J. ( 2010 ). Open Learner Models, in R. Nkambou, J. Bordeau & R. Miziguchi (eds), Advances in Intelligent Tutoring Systems , Springer, 318 - 338 .

15. Bull , S. & Vatrapu , R. (in press). Negotiated Learner Models for Today , ICCE 2012 .