=Paper=
{{Paper
|id=Vol-3738/paper3
|storemode=property
|title=Learning Analytics Driven ARC-Tutoring for Individual Study Success (short paper)
|pdfUrl=https://ceur-ws.org/Vol-3738/paper3.pdf
|volume=Vol-3738
|authors=Ummay Ubaida Shegupta
|dblpUrl=https://dblp.org/rec/conf/lasispain/Shegupta23
}}
==Learning Analytics Driven ARC-Tutoring for Individual Study Success (short paper)==
https://ceur-ws.org/Vol-3738/paper3.pdf
Learning Analytics Driven ARC-Tutoring for Individual Study
Success
Ummay Ubaida Shegupta
Chemnitz University of Technology, Chemnitz, Germany
Abstract
Students have to face challenges in applying scientific research skills during their internship and
thesis writing at the universities. For this purpose, they receive some static web information and
in the best cases holistic mentoring support from supervisors. However, they often require
additional assistance in getting suggestions, immediate responses to errors, scaffolding, and
reminders of their own learning goals. In this doctoral study, the concept of ARC tutoring guided
by learning analytics has been realized as a proposition to address the aforementioned need for
assistance in study success in higher education. It advocates leveraging learning experience data
by employing learning analytics to develop the assessment, recommendation, and conversational
agent (ARC) integrated tutoring workbench featuring distinct learner and tutor perspectives.
This will enable the student to gain access to performance metrics and semi-automated
individualized tutoring support, while tutors can observe group and individual performance,
facilitating required interventions.
Keywords
learning analytics, tutoring, assessment, recommendation, tutoring agent, study success1
1. Introduction
In higher education, study success is the graduation from the degree program at the
institution level whereas, it is assumed as completion to the attained mastery of the specific
learning objectives of the individual student [5]. This acquired mastery leads to individual
learning success. One of the mastery goals at universities around the world is to equip
students with research competencies. From the natural and social sciences to engineering
and humanities, students are expected to have the ability to engage in scientific inquiry,
analyze data, and draw informed conclusions. As a result, the study success of the students
depends on the successful completion of their research internship and/or thesis where they
need to demonstrate their skills in conducting and reporting scientific research. Different
research methodology seminars are offered before they commence their final assignment
of their degree program at the undergraduate and postgraduate levels. Despite this, most
of the students require individual learning support during their internship report and thesis
LASI Europe 2024 DC: Doctoral Consortium of the Learning Analytics Summer Institute Europe 2024, May 29-31
2024, Jerez de la Frontera, Spain
ummay-ubaida.shegupta@informatik.tu-chemnitz.de (U.U.Shegupta)
0000-0003-1092-9510 (U.U.Shegupta)
© 2024 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).
CEUR
ceur-ws.org
Workshop ISSN 1613-0073
Proceedings
writing. They are supported by some static web information from the universities on how
to conduct and report their research. Moreover, they get mentoring support from their
supervisors who may have time and work constraints to holistically support their students.
This leads to students' need for more individual learning support during their application
phase of scientific research methodology. This need has been addressed as the research
problem in this doctoral research. Consequently, this empirical research aims to provide
individual learning support to higher education students in conducting and reporting
scientific research.
2. Theoretical overview of Tutoring support and Learning analytics (LA)
Mentoring has been recognized as the pedagogical approach to support students'
perseverance in academic success. While the broader concept of mentoring nurtures the
psychosocial and career growth among the students, the derived concept of tutoring
following the didactical strategies of mentoring can narrow down the focus on students’
domain-specific learning needs. The concept of the zone of proximal development (ZPD)
from Vygotsky’s social learning theory is the fundamental basis of the proposed concept of
tutoring. This indicates the state of the students having the ability to master the knowledge
with the needed guidance or support from the tutor to reach the status of mastering it
independently. This guidance or gradual release of support is embedded in the instructional
design theory of Scaffolding which emerged in the context of ZPD. Accompanied by self-
regulated learning and reflective learning strategies, technology-enhanced tutoring can
independently support individual students with minimal human-tutor involvement from
time to time.
Due to the technology integration and learning data generation in the tutoring
environment, learning analytics (LA) is essential to gain actionable insight. In this era of
Web 2.0 technology, LA is a potent tool for anticipating and improving student success by
utilizing actionable insight from the generated data during the learning process. This
practice entails the systematic measurement, collection, analysis, and reporting of data
about learners and their contexts [6]. The objective is to offer meaningful feedback and
scaffolding support when necessary for individual study success [5]. While statistical
analysis serves as a common thread unifying all types of LA, the purposes behind these
analyses dictate four primary categories. Descriptive LA involves the exploration and
summarization of historical patterns of behaviors and performance in online learning
environments. Diagnostic LA seeks to pinpoint the root causes of problems or challenges
encountered in the learning process, fostering a deeper understanding of areas that may
require intervention or improvement. Predictive LA leverages various methods and
technologies to model and foresee future learner outcomes, enabling proactive measures to
enhance overall educational effectiveness. Lastly, Prescriptive LA involves the generation
of recommendations and decision-making based on computational findings derived from
algorithmic models, offering actionable insights to guide and optimize the learning
experience.
The Five Steps Learning Analytics (LA) Model, introduced by Campbell and Oblinger in
2007, was designed to address academic analytics and improve student retention. Notably
employed in the Signals Project at Purdue University, the model involves capturing data,
reporting data patterns, predicting outcomes using statistical regression, implementing
interventions to improve the learning process, and refining the model based on the obtained
results, serving as a baseline for empirical research and application in education. The
Learning Analytics Cycle, introduced by Doug Clow [1], integrates learning theories and the
five-step Learning Analytics model by Campbell and Oblinger [2]. The cycle encompasses
four linked stages: beginning with learners in various learning environments, followed by
generating and capturing diverse data, processing the data to develop metrics and analytics,
and concluding with interventions based on the developed metrics to impact learners and
improve learning practices. Clow emphasizes that effective learning analytics projects
involve closing the loop with interventions based on the generated metrics. The Reference
Model of Learning Analytics is structured around four dimensions—"What?," "Who?,"
"Why?," and "How?"—with the goal of identifying challenges and research opportunities in
Learning Analytics (LA). It addresses data sources, stakeholders, objectives, and methods,
emphasizing tailored approaches for effective interventions, ethical considerations, and
stakeholder expectations.
2.1. Virtual and Intelligent Tutoring Systems
The state of the art Intelligent and AI-independent virtual tutoring approaches are
articulated via synchronous and asynchronous formats in online learning platforms as well
as in learning management systems (LMS). The scope of analysis has been delimited to
blended, asynchronous, and Intelligent tutoring approaches. These transformative
approaches to learning leverage the strengths of technology to create a more inclusive,
interactive, and effective learning environment for students of all backgrounds and levels
[6]. Employing a wide array of artificial intelligence (AI) techniques including natural
language processing (NLP), machine learning (ML), and expert systems, Intelligent Tutoring
Systems (ITS) go beyond conventional tutoring systems. They exhibit a remarkable ability
to discern and adapt to diverse learning styles, pacing preferences, and individual
inclinations [9]. Through this adaptive approach, ITS offers targeted instruction along with
invaluable feedback and guidance, ensuring a finely tuned and enriching learning journey
for every student. ITS encompasses four fundamental components, each playing a crucial
role in the learning process. At the core lies the domain model, which encapsulates the
breadth of knowledge and skills required to proficiently navigate a specific subject area. The
second module is the student model which intricately maps out the unique knowledge,
aptitudes, and capabilities of each student. This personalized profile serves as the compass
guiding instructional adjustments and feedback delivery, ensuring a tailored learning
experience [7]. The pedagogical model stands as the strategic backbone of the system,
orchestrating the methodologies and approaches employed to effectively impart
knowledge. It encompasses a spectrum of techniques, from elucidative worked examples to
insightful hints and informative feedback loops. This dynamic array of instructional
strategies adapts in real-time, aligning with the student's progress and learning pace. The
fourth module is the user interface that serves as the gateway for students to engage with
the system. Beyond its functional role, it creates an immersive learning environment,
facilitating seamless interaction with the material. Additionally, it acts as a conduit for the
timely delivery of constructive feedback and personalized instructions, further enhancing
the learning journey [7]. This personalized approach can integrate formative assessments,
learning recommendations, and interactive support via tutoring agents [7] and significantly
contributes to enhanced learning outcomes by providing immediate feedback. This timely
guidance empowers students to swiftly recognize and rectify any misconceptions or errors
in their comprehension, fostering a deeper grasp of the subject matter. Furthermore, these
systems equip educators with invaluable insights into their students' learning trajectories
and specific needs. Equipped with this information, human tutors are better positioned to
customize their teaching methods, ensuring a more effective and targeted educational
experience for each student [9].
While ITSs hold immense promise, they do come with their share of challenges. These
systems operate within the confines of predetermined rules and responses, lacking the
capability to dynamically adjust to the unique needs and preferences of individual learners.
While proficient in providing foundational support, they fall short in delivering
personalized, tailored learning experiences that cater to the diverse requirements of each
user [3]. There is a critical requirement for the development of highly effective pedagogical
models and instructional strategies capable of not only proficiently imparting knowledge
but also seamlessly adapting to the unique learning needs of individual students. These
challenges collectively underscore the complexity and depth of considerations involved in
the development and utilization of virtual and intelligent tutoring systems [7]. These have
been shown as impactful approaches for supporting study success either in combination or
in segregation [9].
2.2. Research Question
According to Ifenthaler [4], the comprehensive approach of LA is geared towards near real-
time modeling, prediction, and optimization of learning processes, as well as the
environments in which learning occurs. The learning environment includes a range of
technologies, from learning management systems (LMS) to more sophisticated automated
systems like automated feedback-based online assessment systems, and conversational
agents like chatbots. The term does not inherently imply the sole use of advanced adaptive
AI technologies. This enables the formal face-to-face higher education system to use the
advantages of LA by employing the data from LMS.
Overall, the problem is that students require learning support while writing their
research reports for seminars, internships, and thesis. The proposed solution is to provide
them with individual tutoring support with digital formative assessments, recommendation
system, and tutoring agent. The activation of students’ learning initiation, the processing of
their learning status and progression, and acknowledging them are done utilizing learning
analytics. Accordingly, the research questions are,
RQ1: How to design and develop ARC-Tutoring environment by integrating formative
assessments, recommendation system, and conversational agent?
RQ2: How to integrate learning analytics functionalities (by integrating the tasks of
descriptive, diagnostic, and predictive LA) to guide the proposed “ARC-Tutoring”
environment?
RQ 3: What is the perception of the primary stakeholders on LA-guided ARC-tutoring
support concerning study success?
3.1: What is the perception of the students on the ARC-Tutoring environment?
3.2: What is the perception of the teachers on the ARC-Tutoring environment?
Henceforth, the research objective of this empirical research is to design and develop a
prototype of an ARC-tutoring workbench with dual perspectives (Student and Tutor) by
incorporating descriptive, diagnostic, and predictive LA to facilitate individual learning
success.
3. Concept of ARC-Tutoring model
Technology-enhanced tutoring can actively utilize LA within a specific learning context to
discern tutoring needs among students, whether acknowledged or unrecognized by the
students themselves. In realizing the research objective, this thesis introduces the Model of
ARC-Tutoring facilitated with LA. LA is the main driving force in this model because, at the
same time, it is the input and main source of representation of students’ learning
intervention, optimization, and progression [8].
A conceptual framework is designed from the inspiration of Campbell and Oblinger's
five-step LA model [2], with an emphasis on Clow's coherent cyclic process [1]. In the
mentioned ARC-Tutoring model, A stands for formative assessments, R stands for
recommendation and reminders, and C stands for Conversational agents (Chatbot).
Formative assessments are used for learning to identify the learning gaps and with feedback
to optimize it. Recommendations are the suggestions and advice that the students seek to
act for their learning progression. Reminders are repeated cues for time management to
cope with the recommendations and learning goals. Last but not least the conversational
agent is the Tutoring avatar that initiates reflection through its’ question-based interaction,
providing self-regulatory level feedback on the student's response and supporting by
delivering organizational information.
Capture data from the Refine
tutoring environment
Reporting Descriptive
LA
Diagnostic LA Predictive LA
No Is Yes
Tutoring
needed?
Stop ARC-Tutoring Actions
Figure 1: Flow diagram of LA Process in ARC-Tutoring
The phases encompass specifying the learning context, capturing diverse data from the
tutoring environment and securely storing them in linked tables according to data sources,
reporting on descriptive LA through tables and charts, employing interactive visualizations
for diagnosis, predicting and identifying tutoring needs using reported and captured data
through regression analysis, acting through ARC-tutoring support, refining the learning
process based on tutoring input, and connecting to the initial phase for subsequent LA
iterations shown in figure 1. The central actionable tutoring phase, informed by
investigations from prior phases, implements interventions like synchronous and self-study
support, formative e-assessment facilities, and consultations facilitated by recommender
systems and virtual tutoring agents. Incorporating these tutoring strategies, the motivation
of this doctoral thesis is to develop a tutoring environment embedded with LA visualization.
4. Status of implementation and plan of evaluation
The final phase of this doctoral research is currently underway, focusing on the
implementation of the technological components necessary to realize the learning analytics
(LA) tasks integrated with the ARC-Tutoring model in a testbed-specific scenario. This
phase is crucial for assessing the practical application and effectiveness of the proposed
learning environment.
The plan of empirical research for this implementation employs a convergent parallel
mixed method design, as outlined by Creswell and Plano Clark [10]. This approach involves
the simultaneous collection and analysis of both qualitative and quantitative data to provide
a comprehensive understanding of this learning environment for writing scientific reports
for academic purposes.
Data collection will include qualitative data from five expert interviews and quantitative
data from an online survey administered to targeted students working with scientific
research seminars, internships, and thesis. The instruments for data collection consist of a
questionnaire featuring Likert scale items to measure technical usability, learning support,
and overall satisfaction, along with a semi-structured interview protocol to guide expert
interviews. This mixed-method approach is planned to conduct the acceptance and
applicability investigation of the ARC-Tutoring model, capturing diverse perspectives and
detailed insights into its usability and learning support following the principles of SRL.
5. Conclusion
This doctoral study systematically addressed the prevalent challenges faced by higher
education students in applying scientific research skills during their internships and thesis
writing. Recognizing the inadequacies of static web information and often limited tutoring
support due to supervisors' constraints, this research introduced the ARC tutoring model,
guided by learning analytics, as a robust intervention to enhance study success. The ARC
tutoring model integrates assessment, recommendation, and conversational agent (ARC)
features, providing students with access to their own performance metrics and
individualized tutoring support while enabling tutors to monitor group and individual
performance for timely and targeted interventions.
Study success in higher education is a multifaceted construct. At the institutional level, it
is typically defined by graduation rates, whereas at the individual level, it is characterized
by the mastery of specific learning objectives, particularly in scientific research
competencies. Mastery in scientific inquiry, data analysis, and informed conclusion-drawing
is critical across disciplines, from natural and social sciences to engineering and humanities.
The successful completion of research internships and theses is essential for demonstrating
these competencies. Despite the availability of research methodology seminars, many
students require additional individualized learning support during the practical application
phase of scientific research.
The final phase of this research involves integrating the technological components to
realize the learning analytics tasks within the ARC-Tutoring model in a testbed-specific
scenario. The evaluation plan includes expert interviews and an online student perception
survey to assess the developed tutoring environment’s technical usability, learning support,
and overall satisfaction. This comprehensive approach aims to substantiate the efficacy of
the ARC tutoring model in enhancing individualized learning support and overall study
success in higher education.
References
[1] D. Clow, The learning analytics cycle: closing the loop effectively, in: Proceedings of the
2nd International Conference on Learning Analytics and Knowledge, LAK ’12, ACM
Press, New York, NY, (2012), pp. 134-138. doi:10.1145/2330601.2330636.
[2] J. P. Campbell, P. B. DeBlois, D. G. Oblinger, Academic analytics: A new tool for a new
era, EDUCAUSE Review 42 (4) (2007) 40.
[3] M. A. Hasan, N. F. M. Noor, S. S. B. A. Rahman, M. M. Rahman, The transition from
intelligent to affective tutoring system: a review and open issues, IEEE Access 8 (2020)
204612-204638. doi:10.1109/ACCESS.2020.3035963.
[4] D. Ifenthaler, Learning analytics, in: J. M. Spector (Ed.), The SAGE Encyclopedia of
Educational Technology, Vol. 2, Sage, Thousand Oaks, CA, (2015), pp. 447-451.
[5] D. Ifenthaler, J. Y. K. Yau, Utilising learning analytics to support study success in higher
education: a systematic review, Educational Technology Research and Development 68
(2020) 1961-1990. doi:10.1007/s11423-020-09788-z.
[6] P. Long, G. Siemens, What is learning analytics, in: Proceedings of the 1st International
Conference on Learning Analytics and Knowledge, LAK ’11, ACM Press, New York, NY,
2011.
[7] E. Mousavinasab, N. Zarifsanaiey, S. R. Niakan Kalhori, M. Rakhshan, L. Keikha, M. Ghazi
Saeedi, Intelligent tutoring systems: a systematic review of characteristics,
applications, and evaluation methods, Interactive Learning Environments 29 (1)
(2021) 142-163. doi:10.1080/10494820.2018.1558257.
[8] B. A. Motz, Y. Bergner, C. A. Brooks, A. Gladden, G. Gray, C. Lang, ... J. D. Quick, A LAK of
Direction Misalignment Between the Goals of Learning Analytics and its Research
Scholarship, 2023.
[9] N. Singh, V. K. Gunjan, A. K. Mishra, R. K. Mishra, N. Nawaz, SeisTutor: a custom-tailored
intelligent tutoring system and sustainable education, Sustainability 14 (7) (2022)
4167. doi:10.3390/su14074167.
[10] Creswell, J. W., V. L. Plano Clark, Designing and Conducting Mixed Methods Research,
2nd ed., Sage, Thousand Oaks, CA, 2011.