Studying the Impact of Orchestrating Intelligent Technologies in Hybrid Learning Spaces for Teacher Agency

Abstract

The spread of intelligent technologies allows teachers to orchestrate learning scenarios expected to adapt to the needs of both, teachers and learners. However, these intelligent technologies can exert a high level of control over learning activities (e.g., recommending learning resources), raising questions about the role of the teacher. This new role that teachers are acquiring when orchestrating intelligent technologies should be studied in terms of teacher's agency, as a means of limiting potential ethical risks, misalignments, and empowering teachers. This thesis addresses the issues associated to teacher agency when designing (co-design of the technologies or of the learning design) and managing (when the technology is deployed) intelligent technologies in hybrid settings.

1. Motivation

The implementation of intelligent technologies in education has the potential to help teachers in providing appropriate support to learners (e.g., feedback, recommendations, third party resources) considering their behaviors, performances, and/or real-time locations [ 1 ]. Furthermore, the widespread use of Information and Communication Technologies (ICT) entails the potential to break traditional dichotomies (formal/informal, physical/virtual, teacher/student roles) and transform the notion of traditional learning spaces into hybrid learning spaces, especially in higher education contexts [ 2 ]. The deployment of systems that autonomously enable adaptive support in context-aware learning environments demonstrates a high potential for education, while it represents a sociotechnical challenge for practitioners and researchers [ 3 ].

The post-pandemic reality is revealing the potential of tools that provide flexible a personalized opportunities to learn course-related contents; nonetheless, it points at the need for the need to solve pedagogical challenges (e.g., management of learning designs, controlling participation) [ 4 ][ 5 ]. Such tools may support more effective and efficient teaching, since teachers might be able to focus on complex tasks, while intelligent and adaptive technologies handle tasks with a lower level of complexity [ 6 ]. However, concerns regarding the shared control of the learning designs may arise when implementing systems with the capacity to interact with the students without being constantly supervised by the teacher [ 7 ]. This concern is tightly related with one of the requirements of the European Commission for designing and deploying trustworthy artificial intelligence (AI) systems: human agency and oversight [ 8 ]; which reinforces the relevance of approaching teacher agency in educational scenarios in which intelligent technologies are to be deployed.

Agency has been framed from the Social-Cognitive Theory (SCT) as the active role that human agents take to influence and affect their thoughts, actions, and environments. For this theory, agentic outcomes produced by human agents are shaped by external social factors (e.g., culture, values) and by internal cognitive factors (e.g., self-regulatory mechanisms) [ 9 ].

Alternative perspectives have built the argument that human agency is achieved and enacted by subjects situated actions embedded in social systems [ 10 ][ 11 ].

The implications for teacher agency derived from this sociological-rooted theory were captured by [ 12 ] in the ecological model, depicted in figure 1. In this model, teacher agency seen as something to be achieved in the present within contexts-of-actions in which culture, social-structures and resources either contribute or mine to achieve agency. Teacher Agency is affected or informed by past experiences (e.g., professional background), but it is future oriented (in terms of goals and/or desires). The ecological model of teacher agency stresses the importance that teachers’ goals, contextual opportunities and constraints have for shaping their professional practice. In the realm of Technology-Enhanced Learning (TEL), it is essential to examine whether teachers’ backgrounds and goals are aligned with the affordances that intelligent technologies bring to the learning designs. Besides, implementing intelligent technologies to support learning in hybrid learning settings could be even more challenging for teachers as they must balance benefits and trade-offs of technology at different levels (e.g., individual-social, physical-virtual).

The complex process of (1) understanding the potential benefits and risks that a particular technology entangles at a given context, (2) designing a learning scenario supported by that technology and (3) managing the learning experience, is referred to as orchestration [ 13 ][ 14 ]. Management of learning designs is complemented by other processes such adaptation or awareness in which intelligent technologies could provide support for teachers to make the adequate adjustments of the previously created learning design. Typically, achieving an alignment among the intended learning outcomes at different levels is characteristic of a well-orchestrated learning experience [ 15 ]. Therefore, orchestrating TEL holds implications for teacher agency. Successful integration of technologies aligned with teachers’ values enables them to achieve teaching goals, contributing significantly to agency. In the case of intelligent technologies, notably AI, research is essential to address the emergent challenges impacting teacher agency, such as the establishment of new boundaries within teacher-student dynamics [ 16 ].

TEL research that addresses the impact that smart technologies that use Learning Analytics (LA), such as recommender systems or AI-based tools, acknowledge the relevance of the teachers’ role for successfully integrating the innovations. Nevertheless, explicitly approaching the phenomena from the perspective of teacher agency could strengthen the research-practice relation [ 17 ][ 18 ]. For example, teacher agency was indicated as one of the requirements of co-design [ 19 ].

In the field of LA, recent studies are starting to consider the notion of teacher agency; however, the conducted research could benefit from a better grounding of the concept of teacher agency [ 20 ][ 21 ].

While readiness, knowledge, and skills concerning teacher agency have been explored in contexts where technology played a crucial role to enable remote teaching (e.g., during emerging situations like the Covid-19 pandemic) [ 5 ][ 22 ], and teacher agency has also been approached within smart learning environments supporting hybrid teaching and learning [ 23 ], a notable gap persist in the literature. To the best of our knowledge, there is a dearth of studies specifically investigating teacher agency through the lens of orchestration-related actions of intelligent technologies in hybrid settings. Furthermore, it is noteworthy that cited studies do not use wellgrounded or consolidated frameworks of teacher agency to reveal context understanding.

Whereas the importance of considering teachers as co-designers of innovations in the field of TEL has been acknowledged [ 24 ], there is still a need to considered teacher agency for empowering their figure when designing and deploying intelligent technologies in hybrid spaces [ 25 ]. In education, empowerment refers to the process in which teachers develop the competence to assume control of their professional development and they use their knowledge and skills to improve situations [ 26 ]. Therefore, empowerment is needed for teachers to achieve agency. Nevertheless, the potential effects that the deployment of intelligent technologies could have on teacher agency are still unknown (e.g., adaptive systems enabling personalization) [ 5 ]. Thus, the research question (RQ) that motivates this thesis is: How is teacher agency influenced when orchestrating intelligent technologies in hybrid learning scenarios? Two sub-questions arise from the previous RQ, as there are two different focuses for teacher agency (RQ1): What are the implications for teacher agency when co-designing a hybrid learning scenario to be supported by intelligent technologies? and (RQ2): How do intelligent technologies support or mine teacher agency in the enactment of hybrid learning scenarios?

Figure 2 displays the two-fold focus in which studying two orchestration processes separately, design and management (together with awareness and adaptation) may contribute to a better understanding of the phenomena while identifying opportunities for agency achievement.

Design

Time

Management

Awareness

Adaptation

Teacher Agency

Figure 3 displays the thesis overview, including the foregoing contextual elements, the RQ, the partial targeted objectives, and the projected contributions.

The first objective (O1) is to understand teacher agency in the field of TEL. The second objective (O2), which is the main objective, is to identify the main threats and opportunities for teachers to achieve agency when orchestrating intelligent technologies in hybrid settings. O2 has been separated in two objectives. O2a to detect casuistries that emerge when generating. learning designs (or even when codesigning or fine-tuning the tools to be deployed).

Subsequently, O2b aims at unveiling issues that are likely to emerge for teachers at learning time (e.g., the Capacity that teachers have to change the course of actions related with their professional practice. Co-design and deployment of tools that are aligned with their goals, values, and beliefs with the aim to foster students' learning

How is teacher agency influenced when orchestrating intelligent technologies in hybrid learning scenarios? C1: A set of guidelines for teachers to negotiate/navigate the balance of control-agency among the human stakeholders involved). The third objective (O3) is to evaluate how teachers are included in-the-loop when intelligent innovations are being designed and deployed. This dissertation aims at understanding and analyzing how teachers are included by researchers and developers in the different iterations and how that affects system’s fine-tuning [ 20 ].

Ultimately, this thesis aims at raising awareness for teachers themselves to detect opportunities for agency achievement and self-empowerment when orchestrating intelligent technologies in hybrid settings. Moreover, this thesis also aims at providing guidance for the TEL community on how to help teachers achieve agency in the aforementioned contexts.

The rest of this paper presents the methodology to be followed for this doctoral dissertation, a brief overview of the previous work done in the thesis, and some conclusions drawn from this proposal.

2. Methodology

The methodology chosen to tackle the abovementioned objectives is Multiple Case Study, which intents at examining deeply a collection of different activities (e.g., co-design workshops, real-time orchestration of learning designs) and stakeholders (e.g., teachers, developers, researchers). Although each case entails its own particularities, the focus is on the shared phenomenon displayed in those cases [ 27 ]. For this thesis project the phenomenon to be studied across the cases are the casuistries (e.g., alignment between teacher’s goals and system functioning, unexpected problems) that may shape and impact (positive or negative) on teacher agency.

The strongest argument for selecting Multiple-case study is that the findings of an already conducted pilot case study (that is to be included in the collection of cases of the Multiple-case report) point at the need of conducting other cases to extend the comprehension of the reality and contrast those findings. Moreover, evidence from multiple cases is likely to be more compelling, resulting in a more robust study [ 28 ].

The methodological design aims at identifying similarities and differences across-cases. Then, selecting cases comes along with defining and justifying data collection protocol: topics, informant, techniques, and evaluation happenings. Each individual case study may constitute a crucial entity with may inform the design of other cases. As figure 4 shows, the multiple-case design should be open enough to allow modifications on-the-fly as emerging findings are likely to impact in case selection and protocols. At this point, the author has conducted a pilot case study in which a novel system was tested in a real setting [ 29 ]. The pilot case study aimed at understanding the opportunities and threats for teacher agency that implied deploying a Smart Learning Environment, SCARLETT (Smart ContextAware Recommendation of Learning Extensions in ubiquitous seTtings) [ 30 ]. SCARLETT provides adaptive support to learners, based on their contexts, enabling the connection among formal and non-formal learning experiences by means of a web-based application. As the SLE was not initially developed following a human-centered approach, the teacher had togain understanding on it by means of co-designing the learning design with the developer. The learning design serves a the pivot for SCARLETT functioning as it reflects roles, resources linked to learning goals, and point at activity sites. The pilot case study provided insights on the willingness of the teacher about sharing control of the learning situation with the SLE (e.g., autonomous reactions non-supervised by the teacher) in real-life contexts. Moreover, the study showed that a highly facilitated collaboration when creating the learning design could help the teacher to achieve agency partially [ 23 ]. The learning design deployed by the teacher blurred some boundaries between formal and informal learning settings and with regard to physical/virtual spaces. Figure 5 represents the events that were studied in the pilot case study. The event 1 (the co-design process of the learning design), which was the main focus of the study due to the amount of happenings and gathered data. The event 3 (evaluation) also provided a high amount of data based on teacher, developer, and students reflections. The event 2 (the enactment of the learning design) could not lead to develop conclusions due to contextual complexities. In consequence, learning design maganement must be studied within this project to address the RQ.

In upcoming cases, the aim is to comprehend how to include teachers in-the-loop when developing, fine tuning, and evaluating different intelligent solutions that support the enactment of learning designs with different particularities of hybridity. Currently, we are seeking for research projects that are developing or studying intelligent technologies that would align with this dissertation’s RQ or variations suitable for different contexts, such as: How would the technology provide opportunities for teachers to collect and analyze student data to inform their decision-making and support personalized learning? and/or How will teachers be able to customize the parameters, algorithms, rules used by the technology to align with their pedagogical preferences to better achieve agency?

The selection of new cases is a critical step due to

temporal constraints associated with this dissertation. These limitations preclude the consideration of conducting a longitudinal study with an ethnographic approach. Such an approach could have provided a more in-depth understanding of how agency is achieved or not within the research contexts, aligning better with the ecological perspective of agency [ 10 ][ 11 ]. Thus, adapting the ecological perspective, which has been used to framed questions and analyze gathered data of the pilot study, stands as another challenge apart from the selection of cases.

4. Conclusions

This doctoral dissertation aims at helping teachers to achieve agency when orchestrating (focus on co-design and deployment) intelligent technologies in hybrid settings. Intelligent technologies bring particular affordances to educational contexts that shape the opportunities for teachers (inside and outside the classroom), playing a substantial role in transforming teacher agency [ 6 ]. TEL researchers, developers, and teachers (among other stakeholders) should address the pedagogical challenges that will arise in this paradigm. Consequently, this project is targeting teachers as the ultimate cohort of stakeholders that may benefit from this research. Nevertheless, TEL researchers and developers are targeted stakeholders that could benefit by better addressing teacher agency. Even though, the core focus of the research has been identified and a pilot case study has been conducted, the project is still open for changes regarding methodological strategies, further cases of study, and especially evaluation strategies.

Acknowledgements

This research is partially funded by the European Regional Development Fund and the National Research Agency of the Spanish Ministry of Science and Innovation, under project grant PID2020-112584RBC32. Víctor Alonso-Prieto has received funding from the call for UVa 2021 pre-doctoral contracts, cofinanced by Banco Santander. The author would like to express gratitude to his supervisors, Yannis Dimitriadis and Sara L. Villagrá-Sobrino, as well as the rest of the GSIC-EMIC research group, with special emphasis on the support provided by Alejandra Martínez-Monés and Paraskevi Topali.

References

[1] G. J. Hwang , "Definition, framework and research issues of smart learning environments-a contextaware ubiquitous learning perspective," Smart Learning Environments , vol. 1 , no. 1 , pp. 1 - 14 , 2014 . doi: 10 .1186/s40561-014-0004-5

[2] C. Hilli , R. T. Nørgård , and J. H. Aaen , "Designing Hybrid Learning Spaces in Higher Education," vol. 14 , no. 27 , pp. 66 - 82 , 2019 . doi: 10 .1145/3489449.3490013

[3] Kinshuk , N. S. Chen , I. L. Cheng, S. W. Chew, "Evolution Is not enough: Revolutionizing Current Learning Environments to Smart Learning Environments," International Journal of Artificial Intelligence in Education , vol. 26 , no. 2 , pp. 561 - 581 , 2016 . DOI: 10 .1007/s40593-016- 0108-x.P. S. Abril, R. Plant , The patent holder's dilemma: Buy, sell, or troll? , Communications of the ACM 50 ( 2007 ) 36 - 44 . doi: 10 .1145/1188913.1188915.

[4] A. Bashir , S. Bashir , K. Rana , P. Lambert , and A. B. Vernallis , "Post-COVID-19 Adaptations; the Shifts Towards Online Learning, Hybrid Course Delivery and the Implications for Biosciences Courses in the Higher Education Setting," Frontiers in Education , 2021 . doi: 10 .3389/feduc. 2021 .711619

[5] M. Ehren , R. Madrid, S. Romiti, P. Armstrong , P. Fisher, and D. McWhorter , "Teaching in the COVID-19 era: Understanding the opportunities and barriers for teacher agency," Perspectives in Education , vol. 39 , no. 1 , pp. 61 - 76 , 2021 . doi: 10 .18820/2519593X/pie.v39. i1 . 5

[6] R. L. Jørnø , B. L. Andersen , and P. Gundersen , "The imaginary of personalization in relation to platforms and teacher agency in Denmark," Nordic Journal of Studies in Educational Policy , vol. 8 , no. 1 , pp. 20 - 29 , 2022 . doi: 10 .1080/20020317. 2021 .2022073

[7] I. Molenaar , "Personalisation of learning: Towards hybrid human-AI learning technologies," OECD digital education outlook , pp. 57 - 77 , 2021 .

[8] HLEG-AI , "Ethics guidelines for trustworthy AI," European Commision , 2019 .

[9] A. Bandura , "Social cognitive theory: An agentic perspective," Annual review of psychology , vol. 52 , no. 1 , pp. 1 - 26 , 2001 doi: 10.1146/annurev. psych.52.1.1

[10] M. Emirbayer and A. Mische , "What is agency?," American journal of sociology , vol. 103 , no. 4 , pp. 962 - 1023 , 1998 . doi: 10 .1086/231294

[11] G. Biesta and M. Tedder , "Agency and learning in the lifecourse: Towards an ecological perspective," Studies in the Education of Adults , vol. 39 , no. 2 , pp. 132 - 149 , 2007 . doi: 10 .1080/02660830. 2007 .11661545

[12] M. Priestley , G. Biesta, and S. Robinson , "Teachers as agents of change: Teacher agency and emerging models of curriculum," Reinventing the curriculum: New trends in curriculum policy and practice , pp. 187 - 206 . Doi: 10 .5040/9781472553195.ch- 010

[13] P. Dillenbourg , "Design for classroom orchestration," Computers & Education , vol. 69 , pp. 485 - 492 , 2013 . doi: 10 .1016/j.compedu. 2013 . 04 .013

[14] L. P. Prieto , Y. Dimitriadis , J. I. Asensio-Pérez , and C. K. Looi , "Orchestration in learning technology research: evaluation of a conceptual framework," in System Sciences (HICSS) , pp. 4597 - 4605 , IEEE, 2015 . doi: 10 .3402/rlt.v23. 28019

[15] L. P. Prieto , M. Holenko Dlab , I. Gutiérrez, M. Abdulwahed , and W. Balid , "Orchestrating technology enhanced learning: a literature review and a conceptual framework," International Journal of Technology Enhanced Learning , vol. 3 , no. 6 , pp. 583 - 598 , 2011 . doi: 10 .1504/IJTEL. 2011 .045449

[16] K. Seo , J. Tang , I. Roll , et al., "The impact of artificial intelligence on learner-instructor interaction in online learning," Int J Educ Technol High Educ , vol. 18 , no. 54 , 2021 . doi: 10 .1186/s41239-021-00292-9

[17] M. Deschênes , "Recommender systems to support learners' Agency in a Learning Context: a systematic review," International Journal of Educational Technology in Higher Education , vol. 17 , no. 1 , 2020 . doi 10.1186/s41239-020-00219- w

[18] M. Saar , M. J. Rodríguez-Triana , and L. P. Prieto Santos , "Towards data-informed teaching practice: A model for integrating analytics with teacher inquiry," Journal of Learning Analytics , vol. 9 , no. 3 , pp. 88 - 103 , 2022 . doi: 10 .18608/jla. 2022 . 7505 .

[19] J. Roschelle , W. R. Penuel , and N. Shechtman , "Codesign of innovations with teachers: Definition and dynamics," in ICLS 2006 - International Conference of the Learning Sciences, Proceedings , vol. 2 , pp. 606 - 612 , 2006 .

[20] J. Ahn , H. Nguyen , and F. Campos , "From Visible to Understandable: Designing for Teacher Agency in Education Data Visualizations," Contemporary Issues in Technology & Teacher Education , vol. 21 , no. 1 , pp. 1 , 2021 .

[21] M. J. Rodríguez-Triana , L. P. Prieto , A. MartínezMonés, J. I. Asensio-Pérez , and Y. Dimitriadis , "The teacher in the loop: Customizing multimodal learning analytics for blended learning," in ACM International Conference Proceeding Series, pp. 417 - 426 , 2018 . doi: 10 .1145/3170358.3170364.

[22] D. M. Hathaway , G. B. Gudmundsdottir , and M. Korona , "Teachers' online preparedness in times of crises: trends from Norway and US," Educ Inf Technol , 2023 . doi: 10 .1007/s10639-023-11733- 5

[23] A. Carruana Martín , C. Alario-Hoyos , and C. Delgado Kloos , "A Study of Student and Teacher Challenges in Smart Synchronous Hybrid Learning Environments," *Sustainability* , vol. 15 , no. 15 , p. 11694 , Jul . 2023 , doi: 10.3390/su151511694.

[24] P. A. Kirschner , "Do we need teachers as designers of technology enhanced learning?," Instr Sci , vol. 43 , pp. 309 - 322 , 2015 doi: https://doi.org/10.1007/s11251-015-9346-9

[25] M. Pischetola , J. K. Møller , and L. Malmborg , "Enhancing teacher collaboration in higher education: the potential of activity-oriented design for professional development," Educ Inf Technol , vol. 28 , pp. 7571 - 7600 , 2023 . doi: https://doi.org/10.1007/s10639-022-11490-x

[26] P. M. Short , "Defining teacher empowerment," Education , vol. 114 , no. 4 , pp. 488 +, 1994 .

[27] R. E. Stake , Multiple Case Study Analysis , Guilford Press, 2006

[28] R. E. Herriott and W. A. Firestone , "Multisite qualitative policy research: Optimizing description and generalizability," Educational Researcher , vol. 12 , pp. 14 - 19 , 198. doi: 10 .1177/0950422219875083

[29] V. Alonso-Prieto , Y. Dimitriadis , S. L. Villagrá- Sobrino , A. Martínez-Monés , P. Topali , and A. Ortega-Arranz , "Exploring Teacher Agency within a Learning Analytics Informed CoDesigned Scenario," in Proceedings of the Fourth International Workshop on Human-Centred Learning Analytics (HCLA) : Towards trustworthy learning analytics , in conjunction with LAK 2023 : 13th International Learning Analytics and Knowledge Conference, Arlington, TX , USA, March 13 -17, 2023 . https://sites.google.com/view/hcla23/home

[30] S. Serrano-Iglesias , E. Gómez-Sánchez , M. L. BoteLorenzo , J. I. Asensio-Pérez , A. Ruiz-Calleja , and G. Vega-Gorgojo , "Demonstration of SCARLETT: A Smart Learning Environment to Support Learners Across Formal and Informal Contexts," in *Technology-Enhanced Learning for a Free, Safe, and Sustainable World*, T. de Laet, R. Klemke , C. Alario-Hoyos , I. Hilliger , & A. OrtegaArranz, Eds., Springer International Publishing, 2021 , pp. 404 - 408 .