=Paper=
{{Paper
|id=Vol-1411/paper-05
|storemode=property
|title=The Orchestration Matrix: A Tool to Design Heterogeneous Classroom Ecosystems
|pdfUrl=https://ceur-ws.org/Vol-1411/paper-05.pdf
|volume=Vol-1411
}}
==The Orchestration Matrix: A Tool to Design Heterogeneous Classroom Ecosystems==
https://ceur-ws.org/Vol-1411/paper-05.pdf
The Orchestration Matrix:
A Tool to Design Heterogeneous Classroom Ecosystems
Luis P. Prieto and Pierre Dillenbourg
CHILI Lab, École Polytechnique Fédérale de Lausanne (EPFL), Switzerland
{luis.prieto, pierre.dillenbourg}@epfl.ch
Abstract: Modern classrooms are complex socio-technical spaces where multiple
heterogeneous legacy tools and novel technologies coexist. The design and the study of such
technological ecosystems (or of single technologies that have to integrate into one such
ecosystem) is still under-researched, and we lack established conceptual tools to understand
how a learning scenario is orchestrated across such heterogeneous sets of resources, and the
role and affordances of each technology in it. In this contribution, we present the orchestration
matrix, a simple table-like form that can help technology designers and researchers to analyze
an existing technological classroom ecosystem from the point of view of teacher orchestration
of a learning scenario, or propose new ecosystems that are (hopefully) more effective.
Keywords: orchestration, classroom technology, classroom as ecosystem, technology design
Introduction
Classrooms have always been complex, both in terms of social dynamics and processes. They are public spaces
with their own history, highly immediate, where multiple activities take place simultaneously (Doyle, 2006).
And now that technologies are permeating also these spaces, classrooms represent also a complex mix of digital
and legacy technologies: pen and paper, laptops, traditional blackboards and tablets and interactive whiteboards
and specialized and general-purpose pieces of software running on each of those devices… the list, and its
heterogeneity, is almost endless.
As educational technology researchers, we not only strive to find new ways of using technology to
learn. We also need to take into account which other technologies already exist there, which may fulfill a similar
role, or a complementary one, in a sort of ecosystem where the different new and legacy technologies compete
(or collaborate) for teacher and student usage (Zhao & Frank, 2003). However, there is no standard, structured
way for researchers (or technology designers) to think about this classroom ecosystem, and how teaching and
learning within it can be facilitated or hampered.
In this contribution we present a simple, table-like form to do precisely this kind of analysis, which we
call the “orchestration matrix”. We have already used it during the design of our own research efforts into
classroom technologies, and we hope it can be useful in the context of the ‘orchestrated collaborative classroom’
workshop, to understand current and future classroom ecosystems, and to ideate new technologies (or new
ecosystems of technologies) for them.
The next section reviews briefly the notion of classroom as a technological ecosystem, and other
ecosystemic approaches in education and human-computer interaction (HCI). Then, we present the orchestration
matrix itself, and illustrate its usage with a real example from our own research. We end the contribution with a
few conclusions and projections about future work in this line of research.
Learning Ecosystems vs. Orchestration Ecosystems
The metaphor of socio-technical spaces (such as the classroom) ‘ecosystems’1 has been present in the fields of
HCI and education for a long time. Before the turn of the 21st century, Nardi & O’Day (1999) speak of
‘information ecologies’ as “system of people, practices, technologies, and values in a local environment”, and
we can even trace the idea further back, to find ecosystemic approaches to interaction design of complex
industrial settings, such as nuclear plants (Vicente & Rasmussen, 1992), or even further back, when Gibson's
1
Some authors use the word ‘ecologies’ in an equivalent fashion, albeit ‘ecosystem’ seems to be the most
formally correct, when talking about complex systems of living things and the relationships among them. We
will use ‘ecosystem’ throughout the paper, except in those cases in which we directly quote other authors.
Orchestrated Collaborative Classroom Workshop 2015, June 7, 2015, Gothenburg, Sweden.
Copyright held by the author(s).
(1979) notion of ‘affordances’ became widely adopted in HCI. This interest in looking at “technological
artifacts as ‘species’, whose survival is determined by a dynamically unfolding interaction with other species in
their shared natural environment”, and the practices that emerge in this interplay of multiple actors and pieces of
nearby technology, is what Kaptelinin & Bannon (2012) denominate ‘the ecological turn in interaction design’.
In the field of software engineering, (Bosch, 2009) pioneered the concept of “software ecosystems”, as the
evolution of software product lines into platforms open to third parties (e.g., Android and its application
marketplace), thus supporting associated human activity (social and commercial) ecosystems.
This interest in ecosystems as a metaphor can also be found in the field of educational research, where
we can find “ecological approaches to classroom management” (Doyle, 2006), or “learning ecologies” (Brown,
2000) when referring to how the web can change (informal) learning.
Closer to our topic as educational technology researchers and designers, Zhao & Frank (2003) also use
this metaphor to talk about technology adoption in schools: basically, within the school ecosystems, computer
uses are living species, with teachers acting as ‘keystone species’, and technological/pedagogical innovations
representing the invasion of an exotic species. Luckin (2008) expands on the student side of the equation by
proposing a “learner-centric ecology of resources”, as the availability of technologies for capturing data, or
publishing/creating content increases – which can be exploited for learning purposes if designed adequately.
From the teacher perspective, the notion of the classroom as a complex ecosystem has been retaken
recently by Dillenbourg et al.'s (2011) notion of ‘orchestration’, as the teacher manages the multiple learning
processes at multiple social planes and using this ecosystem of resources. They also propose that classroom
technologies should be designed, not only taking into account how to best encourage the individual/group
learning processes, but also this class-wide teacher orchestration process (what they call the ‘third circle of
usability’). Unfortunately, as of today there is no widespread model of what the orchestration process entails for
the teacher, or what sub-processes may lie within this complex, multi-layered activity.
Hence, we can see that it may be useful, when designing technologies for the classroom, to consider not
only our proposed innovation, but also the different resources that teachers and students have at their disposal.
Also, it may be useful to consider the role that each of these elements can have in the different (teaching and
learning) practices that will take place within this ecosystem: will they be supported by the different
technological (or legacy) elements? Will they be hampered by them? Will they collaborate or compete in
providing this support?
The Orchestration Matrix
In order to understand the role of the different technological (or legacy) resources of the classroom in the
teaching practice, we created the ‘orchestration matrix’, whose basic principle is to map such resources and the
orchestration activities that occur in the classroom. Although no clear model of orchestration activities exists,
drawing from existing literature on orchestration and classroom management in general (Dillenbourg, 2013;
Doyle, 2006; Prieto, Holenko-Dlab, Abdulwahed, Gutiérrez, & Balid, 2011), we could propose that, at least, the
following processes take place (more or less simultaneously) during the run-time orchestration of a lesson: the
management of the content and learning objectives (e.g., remembering what concepts and curriculum topics to
cover today, and delivering the corresponding explanations, etc.), the awareness of the ongoing learning
processes (including the modelling of students’ understanding of the contents), the logistics of the classroom
(handing out worksheets, logging into a system, moving chairs around for groupwork, etc.), and the self-
regulation and adaptation of the activities (deciding to cut an exercise short due to lack of time, or improvise an
explanation to address an unexpected misconception).
Table 1: Structure of the orchestration matrix
Teacher orchestration processes
Technologies Content Modelling Classroom Adaptation &
management logistics Regulation
Tech 1 (How Tech 1
supports,
hampers or
affects content
management, if
applicable)
Tech 2
Tech 3
…
Then, by creating a table with the different classroom technological resources (including legacy,
hardware and software), we obtain the orchestration matrix (see table 1). Within this table, in each cell we write
how each element affects the corresponding orchestration process (e.g., if students are using laptops, the
awareness of students’ progress may only be obtained if the teacher goes behind each student, not from the front
of the class; if the teacher is using a web application to have students do individual exercises, the logistics of
distributing the task will require the URL to be delivered somehow to each student). When filling in this table, it
is important to consider the concrete learning activity taking place; thus, we can have a different orchestration
matrix for each pedagogical scenario taking place in a concrete classroom ecosystem. The order of the columns
presented in Table 1 has no specific significance, and can be modified.
Illustrative case: Designing Tessellated Battleship
To illustrate this approach, we briefly present how we used the ‘orchestration matrix’ when designing a recent
classroom experiment in a secondary school classroom, in which a new technology was going to be used. We
were considering the usage of multiple tabletop computers in the classroom (along with the existing classroom
ecosystem that included a projector, a teacher computer, etc.), as part of the evaluation of a new educational
game (called ‘Tessellated Battleship’). Tessellated Battleship is a review activity about geometry, coordinate
systems and directed numbers (the activity was designed for 11-12 year old students), all topics within their
mandatory curriculum. This experiment was framed within a larger project in which we explore the effects of
paper (and especially, augmented paper technologies) in classroom orchestration (1).
Figure 1 shows the actual orchestration matrix used, with the different technologies in the ecosystem
(including the tabletops we were introducing, but also existing elements such as the wall display usually present
in the classroom) represented as rows on the left side. Thinking about the learning scenario we had in mind
(basically, a collaborative game in which different groups of students try to stop the advance of a virus infection
using lasers with different geometrical forms, in a similar fashion to the classic battleship game), we filled in the
role that we envisioned for each technology (e.g., the wall display would help in the classroom logistics by
displaying the current phase and the overall state of the game; orchestration cards were to be used to facilitate
disciplinary measures like freezing the game, etc.).
Figure 1. Example of orchestration matrix during the ‘Tessellated Battleship’ classroom experiment design
In this orchestration matrix example we can also see its usage as we were considering several variants
in the design of the classroom ecosystem, and in the orchestration of the scenario (represented by the arrows
between different cells). For instance, in the right side of the figure, we can see how we were considering two
different ways of aiding the teacher in the regulation/adaptation of the lesson: either by showing prompts at
different times on the wall display, or by providing teacher with a ‘cheatsheet’ of prompts and adaptations to be
performed at different times during the lesson.
It is worth noting that, in this example, the designer was operating alone, considering especially how
the different research questions (e.g., what is the role of augmented paper elements on the orchestration load of
the teacher) could be investigated by modifying different elements in the ecosystem. Later on, the different
alternatives that emerged while brainstorming with the orchestration matrix were discussed with the teacher,
helping shape which ecosystem variations were feasible within her particular classroom.
Conclusions and the future
We have presented here a new way of organizing information about the technology in a classroom ecosystem,
aimed at helping researchers and designers better understand their use in classrooms, from the point of view of
how each of the present pieces of technology affect the orchestration processes. As presented here, the focus is
on classroom orchestration (often performed by the teacher) in run-time. However, this table form could be
extended in the future to cover other orchestration processes (such as the activity preparation beforehand or the
post-hoc evaluation of the learning process). It could also be expanded to include the student view of things
(e.g., how the different technologies affect the learning processes of students), for example using Bloom's
(1956) or Krathwohl's (2002) taxonomy of learning activities. Also, this classroom-centric design tool could
also be expanded to take into account the learning that happens across different contexts (e.g., in a field trip, in a
museum, etc.): these different ecosystems could be designed using a separate table, or by expanding the table to
include non-classroom technologies and moments. All these additions, and its use and refinement in real other
research efforts, can help in having a more global understanding of what will happen in the classroom
ecosystem when all these processes interplay.
Endnotes
(1) http://chili.epfl.ch/miocti
References
Bloom, B. S. (1956). Taxonomy of educational objectives: The classification of educational goals (1st ed.).
Bosch, J. (2009). From software product lines to software ecosystems, 111–119. Retrieved from
http://dl.acm.org/citation.cfm?id=1753235.1753251
Brown, J. S. (2000). Growing up digital: How the web changes work, education, and the ways people learn.
Change: The Magazine of Higher Learning, 32(2), 11–20.
Dillenbourg, P. (2013). Design for classroom orchestration. Computers & Education, 69, 485–492.
doi:10.1016/j.compedu.2013.04.013
Dillenbourg, P., Zufferey, G., Alavi, H., Jermann, P., Do-Lenh, S., Bonnard, Q., … Kaplan, F. (2011).
Classroom orchestration: The third circle of usability. In Proceedings of the International Conference of
Computer-Supported Collaborative Learning (CSCL2011).
Doyle, W. (2006). Ecological approaches to classroom management. In C. M. Evertson & C. S. Weinstein
(Eds.), Handbook of classroom management: Research, practice, and contemporary issues (pp. 97–125).
Lawrence Erlbaum Associates Publishers.
Gibson, J. J. (1979). The ecological approach to visual perception.
Kaptelinin, V., & Bannon, L. J. (2012). Interaction design beyond the product: Creating technology-enhanced
activity spaces. Human–Computer Interaction. Retrieved from
http://www.tandfonline.com/doi/abs/10.1080/07370024.2011.646930#.VRQo6nXd-90
Krathwohl, D. R. (2002). A revision of Bloom’s taxonomy: an overview. Theory Into Practice, 41(4), 212–218.
doi:10.1207/s15430421tip4104_2
Luckin, R. (2008). The learner centric ecology of resources: a framework for using technology to scaffold
learning. Computers & Education, 50, 449–462.
Nardi, B. A., & O’Day, V. L. (1999). Information ecologies: using technology with heart. Retrieved from
http://dl.acm.org/citation.cfm?id=299579
Prieto, L. P., Holenko-Dlab, M., Abdulwahed, M., Gutiérrez, I., & Balid, W. (2011). Orchestrating Technology
Enhanced Learning: a literature review and a conceptual framework. International Journal of Technology-
Enhanced Learning (IJTEL), 3(6), 583–598.
Vicente, K. J., & Rasmussen, J. (1992). Ecological interface design: theoretical foundations. IEEE Transactions
on Systems, Man, and Cybernetics, 22(4), 589–606. doi:10.1109/21.156574
Zhao, Y., & Frank, K. A. (2003). Factors affecting technology uses in schools: An ecological perspective.
Acknowledgments
This research was supported by a Marie Curie Fellowship within the 7th European Community Framework
Programme (MIOCTI, FP7-PEOPLE-2012-IEF project no. 327384).