=Paper=
{{Paper
|id=Vol-1411/paper-03
|storemode=property
|title=Classroom Technology Design Guidelines: A Focused Survey
|pdfUrl=https://ceur-ws.org/Vol-1411/paper-03.pdf
|volume=Vol-1411
}}
==Classroom Technology Design Guidelines: A Focused Survey==
https://ceur-ws.org/Vol-1411/paper-03.pdf
Classroom Technology Design Guidelines: A Focused Survey
Luis P. Prieto
CHILI Lab, École Polytechnique Fédérale de Lausanne (EPFL), Switzerland
luis.prieto@epfl.ch
Abstract: Although digital technologies have been permeating our classrooms for the last
three decades, educational technology designers have begun considering the specific
challenges of such complex multi-user environments only recently. This contribution
describes the result of a limited-scope review of works proposing technology design
guidelines that address the challenges of the (physical) classroom and its orchestration by
teachers. The resulting synthesis of guidelines is expected to help the workshop participants in
finding ways to address the challenges of concrete classroom ecologies, while respecting the
challenging restrictions of authentic settings.
Keywords: classroom technologies, design guidelines, survey, review, orchestration
Introduction and motivation
The fact that a classroom is a very challenging, multi-user social environment has been known for a long time in
educational research. Classrooms are inherently time-restricted, multi-activity public spaces (Doyle, 2006). With
the advent of digital technologies, a new layer of complexity has been added to the challenges of managing a
classroom. In educational technologies, researchers have reckoned the difficulties of managing technology-
enhanced educational environments, especially in formal education, under the label of ‘orchestration’
(Roschelle, Dimitriadis, & Hoppe, 2013).
With this recognition, CSCL and other educational technology researchers/designers have begun
studying more closely how their proposed innovations are used in the context of an authentic educational
setting, trying to understand how the specific restrictions of a real classroom (in terms of time, curriculum,
activity or social constraints) affect such usage. In this line of work, (Dillenbourg et al., 2011) speak of
orchestration of learning as “usability at the classroom level”, highlighting that educational technologies should
not only be usable for individuals and enable small-group activities; they also should be manageable under the
tight restrictions of learning (and managing the learning) in a classroom with a large group of students.
Educational technology design guidelines based on these ideas of “classroom usability”, going beyond
existing individual and small-group usability advice have started to appear, distilling the results of researchers’
empirical evaluations of technology in authentic settings. However, their number is still small and there exists
no single scientific forum where this emergent flavor of usability/interaction is discussed. This contribution tries
to provide an initial seed for such discussions, synthesizing the works that currently offer explicit technology
design guidelines, specific for classroom environments. Such synthesis may provide a strong base to help
researchers (especially, the present workshop participants) in designing novel solutions for concrete classroom
ecologies where the heterogeneous technologies are becoming an increasingly critical problem.
Survey methodology
Our survey followed coarsely Kitchenham’s classic systematic review guidelines: a focused systematic review
of main technological literature databases was complemented with the author’s existing expert knowledge of the
field, which already included a database with several research works addressing the challenges of designing for
the classroom. The systematic part of the review was performed by querying the three main literature databases
specializing on technical and socio-technical literature (ScienceDirect, ACM Digital Library and IEEExplore),
using the following query: (classroom AND technology AND guidelines). This search string was applied only
over the metadata of the entries (e.g., title, abstract and keywords). Such query returned a total of 104 results (68
for IEEExplore, 23 for ScienceDirect and 13 for ACM Digital Library). These results were then inspected and
filtered according to the following criteria: they have to provide explicit technology design guidelines (as
opposed to research or pedagogical guidelines), and do so from a technology design perspective (as opposed to a
teacher or insititutional perspective). Also, the guidelines should be applicable beyond a single system or kind of
learning activity, and considerable author or paper overlaps should be resolved by looking at the most complete
Orchestrated Collaborative Classroom Workshop 2015, June 7, 2015, Gothenburg, Sweden.
Copyright held by the author(s).
source. After such filtering, only four sources were left. To these four resulting sources, the author added several
previously-known works that propose design guidelines for technology design in the classroom (i.e., applying
the same filtering criteria as in the systematic review selection). Table 1 presents the resulting set of 11 works to
be surveyed and analyzed more deeply.
The resulting sources were carefully read, and their guidelines were clustered by similarity, as well as
prioritized (so that clusters appearing in more separate sources are given more importance). The results of this
analysis are presented in the next section.
Table 1: Summary of literature sources surveyed
Review part Reference Journal/Conference Specific area/subject
Systematic (Cramer & Hayes, 2013) International Conference on Economy in primary
search Interaction Design and Children education
(Nussbaum & Infante, 2013) International Conference on Educational technology
Advanced Learning Technologies
(Stanton et al., 2001) SIGCHI Conference on Human Tangible interfaces,
Factors in Computing Systems storytelling
(Ting, 2013) Computers & Education Mobile learning
Previously- (Balaam, 2013) Computers & Education Educational technology
known (Cuendet, Bonnard, Do-Lenh, & Computers & Education Augmented reality-based
database Dillenbourg, 2013) activities
(Dillenbourg, 2013) Computers & Education Educational technology
(Dillenbourg & Jermann, 2010) New Science of Learning (book) Educational technology
(Kharrufa, Martinez-Maldonado, ACM International Conference on Tabletop-based activities
Kay, & Olivier, 2013) Interactive Tabletops and Surfaces
(Kreitmayer, Rogers, Laney, & ACM Conference on Pervasive Collaborative learning,
Peake, 2013) and Ubiquitous Computing Tablet-based activities
(Sharples, 2013) Computers & Education Educational technology
Results: The guidelines
Among the literature sources surveyed (see above), up to 58 design guidelines for classroom technologies were
found. Despite this large number, many of the design guidelines were very similar to each other, or could
otherwise be clustered in closely-related themes. These (overlapping) clusters define the following technology
design guidelines for the classroom (in parentheses, number of guideline instances appearing in each cluster):
1. Provide awareness and visibility mechanisms (10): Probably the single most often mentioned design
guideline for classroom technologies was the need to have awareness mechanisms, especially for the
teacher to keep track of what is going on in the classroom, but also for mutual awareness among
students in the form of public displays (Cuendet et al., 2013; Dillenbourg & Jermann, 2010;
Dillenbourg, 2013; Kharrufa et al., 2013; Nussbaum & Infante, 2013). This includes making visible
student states that may otherwise be invisible or hard to infer by the teacher, such as the phase/activity
progress, or even student emotions (Balaam, 2013). Two other themes are closely related:
a. Record and show classroom history (4): Since a classroom is a social space with its own
history across multiple sessions and activities, it makes sense to record and eventually
visualize data about process followed by students, for run-time action or post-hoc reflection
(Kharrufa et al., 2013; Kreitmayer et al., 2013; Ting, 2013).
b. Take into account and record data for (summative) assessment (4): Most of the above
awareness mechanisms are intended for the general awareness of the teacher, to enable
effective formative assessment during the lessons. However, mandatory summative
assessments (often, individual) are still needed in most formal education. Thus, taking those
assessments into account when technologies gather data, and facilitating individual
accountability can greatly aid in the integration of a technology in everyday classroom
practice (Cramer & Hayes, 2013; Dillenbourg & Jermann, 2010; Kharrufa et al., 2013;
Nussbaum & Infante, 2013).
2. Do not break the classroom flow (9): Many authors highlight the immediacy of actions in a physical
classroom, and the tight schedules teachers and students often experience. Thus, classroom
technologies should allow for quick operation (e.g., no complex login/initiation mechanisms) and
changes, so as to make use of small “time pockets” and avoid unnecessary distractions (Cramer &
Hayes, 2013; Nussbaum & Infante, 2013). Related to this classroom flow, many authors mention that
classroom technologies should enable smooth transitions from individual to group and classroom
activities, both at the level of digital and physical workflow, and managing students’ attention between
autonomous and group/class work (Cuendet et al., 2013; Dillenbourg & Jermann, 2010; Dillenbourg,
2013; Kharrufa et al., 2013; Kreitmayer et al., 2013). Another aspect of this integration into the
classroom flow is the fact that technologies should be open and compatible with existing legacy
technologies and props already in the classroom ecosystem (Stanton et al., 2001).
3. Keep (easy) control in the hands of the teacher (6): Many of the design guidelines reviewed make
emphasis on the teacher-centeredness of classroom interactions, and the need to provide centralized
means for the teacher to control the flow of the classroom easily (Cuendet et al., 2013; Dillenbourg &
Jermann, 2010; Dillenbourg, 2013; Kharrufa et al., 2013). Other authors, like (Sharples, 2013) actually
propose an opposing approach, suggesting that in some cases the teacher should share this orchestration
load with students and/or the technological systems in place.
4. Allow for flexible adaptation of the technology-enhanced activities (4): Very related to the issue of
teacher control of classroom flow is the fact that extraneous events (or the classroom awareness
mechanisms mentioned above) may sometimes make advisable to modify flexibly the original lesson
plans of the teacher, either in terms of activities, timing, etc. (Cuendet et al., 2013; Dillenbourg &
Jermann, 2010; Dillenbourg, 2013).
5. Take into account the physicality of the classroom (6): Another common guideline made for classroom
technologies is to consider the physical layout of the classroom and the physical properties of the
classroom objects and technologies (Dillenbourg & Jermann, 2010; Dillenbourg, 2013; Kharrufa et al.,
2013; Stanton et al., 2001; Ting, 2013). This can have an impact in the visibility (physical actions are
more visible/glanceable) or the flow of the classroom (reifying the classroom workflow in physical
actions/objects) as mentioned above, but also on finer details of the interaction, such as having devices
or actions that better map to certain aspects of the subject content.
6. Design to support (small-group) sharing and collaboration (5): Especially when considering particular
pedagogies such as collaborative learning, the ability to support group work is also a very important
aspect, such as the creation, modification or resuming of student groupings (Cramer & Hayes, 2013;
Kharrufa et al., 2013; Nussbaum & Infante, 2013). Along this dimension we can also find
recommendations of how to design interfaces to encourage (equal) participation by all students and the
creation of an atmosphere of productive collaboration.
7. Provide activity structures and guidance (6): Another classic educational technology advice is to
provide systems that somehow provide a structure for the learning activities (Dillenbourg & Jermann,
2010; Kharrufa et al., 2013; Kreitmayer et al., 2013; Nussbaum & Infante, 2013). In particular, most
authors advocate the use of simple linear sequences of activities/phases (as they are easy to understand
and to explain rapidly), and to keep the individual/group activity progress as synchronized as possible
(e.g., with transitions centralized/controlled by the teacher), to keep the sense of togetherness across the
classroom. Aside from these classic instructional strategies, there are also recommendations more
artistic in nature, for example the advice of (Dillenbourg & Jermann, 2010) about using ‘drama’ –
occasional high-emotion states to engage students and carry them along the learning scenario.
8. Consider the curriculum constraints and relevance (3): As one of the main restrictions of any formal
education setting is the adherence to a mandatory curriculum, some of the surveyed works highlight
that the proposed activities and technologies should be relevant to this curriculum (either by design or
through teacher customization), and that the amounts of time dedicated to the different learning
objectives should be proportional to their relevance in the curriculum (Dillenbourg & Jermann, 2010;
Nussbaum & Infante, 2013).
9. Map technologies and data to the subject content (2): When multi-device, multi-technology setups are
considered, the gathered data (see Awareness above) and the different interaction modes they enabled
should be mapped as best as possible to different aspects of the subject content (Stanton et al., 2001;
Ting, 2013).
10. Balance student needs and discipline (1): The fact that new technologies enable further personalization
of the learning processes for each student should be balanced against the need for the classroom to keep
an order and discipline (Nussbaum & Infante, 2013).
11. Be minimalistic! (4): A kind of meta-principle, which runs in a way counter to all of the previous
guidelines (which propose functionalities to incorporate in classroom technologies), is the fact that the
classroom technology should offer only “just enough” information and functionality for the teacher or
students (Cuendet et al., 2013; Dillenbourg & Jermann, 2010; Dillenbourg, 2013). The rationale in this
case is that each new option or information item present in the technology adds to the cognitive load of
the actors, whose cognitive resources are already stressed by the multi-task nature of the classroom.
Conclusions and implications
As we can see, the guidelines extracted from this focused survey of the literature is not clearly separable, with
overlaps and strong relationships among clusters (e.g., physicality and awareness), but also oppositions/tensions
among them (activity structure vs. flexibility, no-login ease of use vs. need for individual assessment,
minimalism vs. everything else). This focused review shows several methodological limitations (mainly, the
restrictiveness of its search keywords, and the requirement for explicit technology guidelines). We hope this
review can serve as a seed to be expanded in later, deeper reviews of this area (e.g., adding terms like
“orchestration” or “lessons learned”, or extracting implicit guidelines present in other classroom technology
studies, or commonalities in the reification of these guidelines). We believe the critical discussion and
application of these guidelines can be crucial to the development of this emergent field within learning
technologies and human-computer interaction, which can eventually spawn its own research sub-community and
venues for scientific discussion (of which the present workshop is an early example).
References
Balaam, M. (2013). A part practical and part conceptual response to orchestration. Computers & Education, 69,
517–519. doi:10.1016/j.compedu.2013.04.009
Cramer, M., & Hayes, G. R. (2013). The digital economy. In Proceedings of the 12th International Conference
on Interaction Design and Children - IDC ’13 (pp. 431–434). New York, New York, USA: ACM Press.
doi:10.1145/2485760.2485832
Cuendet, S., Bonnard, Q., Do-Lenh, S., & Dillenbourg, P. (2013). Designing augmented reality for the
classroom. Computers & Education, 68(0), 557–569. doi:10.1016/j.compedu.2013.02.015
Dillenbourg, P. (2013). Design for classroom orchestration. Computers & Education, 69, 485–492.
doi:10.1016/j.compedu.2013.04.013
Dillenbourg, P., & Jermann, P. (2010). Technology for classroom orchestration. In M. S. Khine & S. I. M.
(Eds.), New Science of Learning: Cognition, Computers and Collaboration in Education. Springer.
Dillenbourg, P., Zufferey, G., Alavi, H., Jermann, P., Do-Lenh, S., Bonnard, Q., … Kaplan, F. (2011).
Classroom orchestration: The third circle of usability. In Proceedings of the International Conference of
Computer-Supported Collaborative Learning (CSCL2011).
Doyle, W. (2006). Ecological approaches to classroom management. In C. M. Evertson & C. S. Weinstein
(Eds.), Handbook of classroom management: Research, practice, and contemporary issues (pp. 97–
125). Lawrence Erlbaum Associates Publishers.
Kharrufa, A., Martinez-Maldonado, R., Kay, J., & Olivier, P. (2013). Extending tabletop application design to
the classroom. In Proceedings of the 2013 ACM international conference on Interactive tabletops and
surfaces - ITS ’13 (pp. 115–124). New York, New York, USA: ACM Press.
doi:10.1145/2512349.2512816
Kreitmayer, S., Rogers, Y., Laney, R., & Peake, S. (2013). UniPad. In Proceedings of the 2013 ACM
international joint conference on Pervasive and ubiquitous computing - UbiComp ’13 (p. 801). New
York, New York, USA: ACM Press. doi:10.1145/2493432.2493506
Nussbaum, M., & Infante, C. (2013). Guidelines for educational software design that consider the interests and
needs of teachers and students. In 2013 IEEE 13th International Conference on Advanced Learning
Technologies (pp. 243–247). IEEE. doi:10.1109/ICALT.2013.75
Roschelle, J., Dimitriadis, Y., & Hoppe, U. (2013). Classroom orchestration: Synthesis. Computers &
Education, 69, 523–526. doi:http://dx.doi.org/10.1016/j.compedu.2013.04.010
Sharples, M. (2013). Shared orchestration within and beyond the classroom. Computers & Education, 69, 504–
506. doi:10.1016/j.compedu.2013.04.014
Stanton, D., Pridmore, T., Bayon, V., Neale, H., Ghali, A., Benford, S., … Wilson, J. (2001). Classroom
collaboration in the design of tangible interfaces for storytelling. In Proceedings of the SIGCHI
conference on Human factors in computing systems - CHI ’01 (pp. 482–489). New York, New York,
USA: ACM Press. doi:10.1145/365024.365322
Ting, Y.-L. (2013). Using mobile technologies to create interwoven learning interactions: An intuitive design
and its evaluation. Computers & Education, 60(1), 1–13. doi:10.1016/j.compedu.2012.07.004
Acknowledgments
This research was supported by a Marie Curie Fellowship within the 7th European Community Framework
Programme (MIOCTI, FP7-PEOPLE-2012-IEF project no. 327384).