Recent advances in artificial intelligence have created new opportunities for nursing education, yet empirical understanding of how nursing students engage with AI-enhanced learning environments remains limited. This study proposes to examine student engagement patterns within Sherpath AI, an advanced generative AI chat tool powered by trusted Elsevier content. Using Chi and Wylie's (2014) Interactive, Constructive, Active, Passive (ICAP) framework as its theoretical foundation, this research will analyze how different modes of student engagement correlate with learning outcomes. Through a mixed-methods approach combining learning analytics with qualitative analysis, the study will examine chat logs, system interaction data, and learning outcomes across three phases: development of analytics framework, data collection and primary analysis, and pattern analysis. Expected outcomes include understanding the distribution of ICAP engagement modes, their relationship to learning performance, and identification of effective engagement patterns. This research will contribute to both learning analytics and nursing education by providing a theory-driven framework for analyzing AI-supported learning interactions and developing evidence-based guidelines for implementation.
Recent advances in artificial intelligence have spurred cautious optimism in nursing education
delivery and student learning processes [
This proposal outlines a plan to examine logs of student interactions with Sherpath AI (SPAI), an
advanced generative AI chat tool designed for nursing education. Through application of the
Interactive, Constructive, Active, Passive (ICAP) theoretical framework [
Learning analytics presents opportunities for understanding student engagement patterns in digital
learning environments, with recent research demonstrating how meaningful insights can be derived
even from small datasets [
The Interactive, Constructive, Active, Passive (ICAP) framework provides a structured
theoretical foundation for examining these engagement patterns, offering well-defined categories
for classifying different modes of cognitive engagement and their relationship to learning outcomes
[
This framework's emphasis on observable learning behaviors makes it particularly suitable for analyzing student interactions with AI-supported learning tools, where engagement patterns can be systematically tracked and analyzed. As such, the driving questions of this research agenda are, ‘How do nursing students' interactions with Sherpath AI align with the ICAP framework's modes of cognitive engagement? (RQ1) What relationships exist between ICAP-classified engagement patterns and measurable learning outcomes? (RQ2) and, How can we support nursing students in developing more effective AI-learning interactions? (RQ3).
This investigation employs the ICAP framework [
Sherpath AI combines evidence-based nursing content with generative AI capabilities, enabling: (a) Real-time student support through natural language dialogue, (b) Practice question generation and feedback, (c) Clinical reasoning development through case discussions, and (d) Self-directed learning pathways. Students primarily interact with the system through: (a) Content queries and clarification requests, (b) Practice question generation, (c) Clinical case discussions and care planning and (d) Concept exploration and synthesis. The system generates detailed interaction logs including timestamped chat messages, activity completion records, system feature usage patterns, session duration and frequency data.
This study will employ a mixed-methods approach combining learning analytics with qualitative analysis of student-SPAI interactions. The research design will unfold in three distinct phases, each building upon the previous one to develop a comprehensive understanding of student engagement patterns and their relationship to learning outcomes (see Table 2).
Phase 1 will focus on developing the analytics framework. This initial phase encompasses three key components: First, the development of the ICAP classification framework, including creating classification schemes for each mode, developing natural language processing (NLP) algorithms for engagement pattern identification, and establishing classification rules. Second, a validation process incorporating expert review of classification criteria, pilot testing with sample interaction data, and refinement of algorithms. Third, reliability assessment through inter-rater reliability testing, cross-validation of automated classification, and establishment of quality metrics.
Phase 2 will involve data collection and primary analysis. This phase begins with systematic collection of SPAI interaction logs, including chat dialogue transcripts, activity completion patterns, system feature usage data, and session metrics. Course performance data, including examination scores (e.g., HESI Specialty Exam) will also be collected. The ICAP classification framework will be implemented through automated categorization of engagement patterns, with manual validation and refinement of classification criteria. Preliminary analysis will include sequential pattern mining of engagement sequences, analysis of transition patterns between ICAP modes, and initial correlation analysis with learning outcomes.
Phase 3 will focus on integration and pattern analysis. This phase involves comprehensive analysis of engagement patterns across ICAP modes, investigating relationships between engagement patterns and learning outcomes, and examining temporal aspects of engagement. Pattern validation includes expert review, validation against learning outcomes, and cross-validation across student cohorts. The phase concludes with model development, including creation of predictive models and development of guidelines for optimal engagement.
Phase 1: Analytics Phase 2: Data Collection Framework Development and Primary Analysis Development of Data collection and classification framework initial analysis and validation
scheme creation, NLP algorithm development, Classification rule establishment
In summary, the study will address the three primary research questions through specific analytical approaches: RQ1 examines how nursing students' interactions with SPAI align with ICAP modes through NLP analysis of chat dialogues, pattern recognition in interaction sequences, and automated classification of engagement modes. Data sources will include chat transcripts, system logs, and session metrics, producing outputs of ICAP mode distribution and engagement patterns. RQ2 investigates relationships between engagement patterns and learning outcomes through correlation analysis and predictive modeling. This includes examining engagement patterns versus course performance, mode transitions versus learning progression, and developing regression models of engagement factors. Data sources combine ICAP classification results with course performance metrics. RQ3 explores how to support effective AI-learning interactions through pattern analysis and qualitative examination. Methods include identifying successful engagement sequences, comparing high versus low performing students, and analyzing dialogue quality. This analysis draws from classified ICAP patterns, performance data, and dialogue metrics.
Primary outcomes of this study focus on ICAP mode distribution (frequency, progression patterns, duration, transitions) and learning performance (course examinations such as HESI Specialty Exams). Secondary outcomes examine engagement quality metrics including dialogue depth, content quality, and self-regulation indicators. These outcomes will contribute to several broader areas of impact in nursing education and AI-supported learning.
First, understanding the relationship between engagement patterns and learning outcomes will inform the design of more effective AI-supported learning environments in nursing education. Second, the methodological framework developed through this study will provide a structured approach for analyzing student-AI interactions in healthcare education more broadly. Finally, insights from this research will contribute to our understanding of how to effectively integrate AI tools into nursing education in ways that support both knowledge acquisition and the development of critical thinking skills. This understanding is particularly crucial as nursing education continues to evolve with technological advancement, requiring evidence-based approaches to technology integration that maintain pedagogical effectiveness.
Several important limitations and potential sources of bias must be acknowledged in this study. First, the analysis of student engagement through digital trace data may not capture the full complexity of learning interactions. While the ICAP framework provides a structured approach for classifying observable behaviors, students' internal cognitive processes and motivations may not be fully reflected in their digital interactions. Technical limitations include the constraints of NLP in accurately classifying engagement modes, particularly for complex or ambiguous interactions. The automated classification system may require ongoing refinement and validation to ensure reliable categorization of student engagement patterns.
Potential biases include: (a) Selection bias: Students' comfort level with technology may influence their engagement patterns; (b) Measurement bias: The focus on quantifiable interactions may undervalue qualitative aspects of learning; and (c) Temporal bias: Student engagement patterns may vary across the academic term. Additionally, the study's focus on a single AI system (Sherpath AI) within one educational context may limit generalizability. While findings may inform understanding of AI-supported learning broadly, specific patterns may not transfer directly to other educational contexts or AI platforms. To address these limitations, the study incorporates multiple validation approaches and acknowledges these constraints in interpreting results and forming recommendations.
This research can advance both learning analytics and nursing education in several critical
ways. For learning analytics, it can demonstrate how educational theory (ICAP) can be systematically
integrated into analytics frameworks, addressing a persistent gap in the field [
Generative AI tools have not been used by the author to prepare the submission.