The emergence of intelligent textbooks (iTextbooks) has broadened the landscape of education, yet poses new challenges in meeting the needs of diverse learners. In response to these challenges, we developed Curio, a personalized educational recommendation system for help-seeking embedded within iTextbooks. By using video transcripts and text extracted from images in iTextbooks as an index for a search engine, Curio provides targeted and context-specific content to aid comprehension. Integrated within the iTextbooks itself, this on-demand tool ofers instant clarification on complex STEM concepts, making learning more adaptable to individual needs. We have been working with Open Learning Initiative (OLI) to refine and test Curio's potential, and envisage its application across various iTextbook platforms. This paper discusses Curio's contribution to the educational recommendation algorithms ifeld.
The transition from traditional textbooks to intelligent textbooks (iTextbooks) has democratized
access to education on an unprecedented scale [
Curio seeks to address these challenges by recommending learning content within the learning platform and showing the results in the context of the video player. This approach simultaneously reduces the friction of leaving the platform while ensuring the quality of search results. Moreover, the tool can leverage the context of the question and the learner model to filter on-demand recommendations to individual learners’ current needs, making learning experiences more personalized. By lowering barriers to help-seeking, Curio aims to not only improve domain-specific learning outcomes but also increase learners’ self-eficacy.
In interactive learning environments (ILEs), help-seeking is a pivotal strategy that promotes the
development of independent skills and abilities [
Help-seeking involves a series of processes, such as becoming aware of the need for help,
deciding to seek help, identifying potential helpers, using strategies to elicit help, and evaluating
the help-seeking episode [
Existing help-seeking mechanisms in digital learning environments, such as discussion
forums, have low levels of engagement [
While search engines have been valuable tools in educational contexts, there is a pressing need
to curtail the cognitive overload that learners experience from non-educational snippets in
search engine result pages [
Furthermore, research is sparse on the optimization of extraction and ranking algorithms in search engines based on learning needs. No current algorithm is interested in determining the learner knowledge level from the request or the elimination of non-educational content from search engines.
Educational recommendation systems have received substantial attention over the years, with
research investigating various aspects such as machine learning-based recommendation systems
for e-learning [
Educational recommendation systems that adapt to students’ existing knowledge of specific
domain concepts have demonstrated eficacy in delivering personalized scafolding within
iTextbooks [
Learner modeling is a key aspect of adaptive learning systems, encapsulating the systems’
assumptions about a learner’s unique attributes related to educational processes [
For instance, the Skills Map learner model of OLI platform applies BKT to classify learners’ knowledge into states of ’learned’ or ’not learned’[21]. This method delineates probabilities for learners’ responses premised on their comprehension of a concept[22].
Curio functions as an extensible tool capable of integrating learner models within iTextbook platforms to further personalize content recommendations for help-seeking. For example, we intend to incorporate the Skill Map model from the OLI platform to enhance the relevance of video recommendations based on a learner’s prior knowledge. If a learner has successfully mastered a skill, Curio’s recommendation algorithm will prioritize videos that do not pertain to this skill. Conversely, if the learner is in the process of understanding a skill, the system will prioritize videos relevant to that skill, thereby assisting the learner in grasping the concept.
The Curio user interface is a comprehensive solution designed to enhance the e-learning experience by being implemented as an overlay atop any video player or text within various learning platforms. It has been structured to facilitate easy access to learning resources, promote interactivity, and provide tailored explanations for students’ misconceptions.
Curio’s In-Widget Recommendation feature serves as a powerful tool that provides learners with a range of educational resources directly within the platform. The feature operates by analyzing the content currently being viewed by the learner and generates relevant educational materials to supplement their learning. These results, organized according to their relevance to the current learning context, are directly presented to the learners within the video player (see Figure 1) or text interface, without necessitating a disruptive shift to a new page or platform.
Curio takes advantage of large language models (LLM) to supplement the recommendation results, specifically integrating GPT-4 to provide text-based instructions. This feature targets students’ misconceptions, delivering tailored guidance to clarify their understanding and address gaps in their knowledge. We crafted a example result and explicly providde regulations to the GPT-4 endpoint, to ensure the quality of generated summaries. The main prompt body we used in the query is
What’s the definition of $text related to $learningObj in a course $courseName? By implementing the few-shot prompting technique and supplying the context of the learning material the learner is seeking help with as variables in the GPT-4 prompt, Curio is equipped to deliver context-specific, real-time explanations for any term, formula, and code snippets within iTextbooks.
The Collection Feature in Curio is designed to facilitate personalized learning by allowing learners to earmark specific resources for future reference. This feature is particularly advantageous for learners who discover relevant materials that they wish to revisit later. This ability to curate learning resources based on individual needs further enhances the personalization capabilities of Curio.
User Reverse Proxy (Nginx)
Frontend (React.js) API Server (Koa.js)
DB (MongoDB)
Course Texts (Elasticsearch) Curio Preprocessor
Analyzer (PyTesseract) Static Files & Course Videos
Formatter (Python) Course Subtitles
Course Videos
In summary, the Curio user interface serves as an embedded help-seeking tool within learning platforms. It provides on-demand, uninterrupted access to tailored educational resources, thus ofering personalized scafolding for learners according to their unique needs.
The primary objective of Curio’s system implementation is to address three key technical challenges that may hinder users in their quest to comprehend learning contents on the platform. The first challenge is how to efectively index all teaching materials available on the platform. The second challenge is how to guide users in conducting efective searches with appropriate keywords. Lastly, the third challenge is how to ensure that when users query the platform, they do not receive excessive false positives.
To tackle the first challenge, we employed Tesseract v5.0 [ 23], an open-source LSTM-based OCR solution. This solution enables us to scan the video frame every few seconds and extract all text, source code, and formulae displayed on the screen. We store this information, along with subtitles and timecodes, in Elasticsearch [24], which facilitates full-text searching.
Additionally, we integrated the Tesseract engine into our video player with the same configurations, which enables students to initiate searches by using a selection tool on the interface and encourages them to search with the same vocabulary set stored in the Elasticsearch server.
Lastly, to mitigate the risk of false positives, we apply a weight calculated based on the learner’s current proficiency on prerequisites, ensuring that users receive relevant learning materials. Currently, we are testing the platform with a simpler version of algorithm ( indicates whether a returned video share the same learning object with current video or not): newScore = (1 · (1 − ) + 2 · ) · originalScore While advanced learner models have not been integrated, Curio was intentionally designed as an extensible approach capable of accommodating such enhancements. This design allows for further improvements to the search results by incorporating advanced learner models if desired.
Curio aims to accelerate learning science research by empowering researchers to experiment with diferent educational recommendation algorithms and collect valuable help-seeking data that is traditionally dificult to measure. Research on optimization of extraction and ranking algorithms based on learning needs is relatively rare, and existing algorithms do not aim to determine the learner’s knowledge level from the request. Curio can address this issue by enabling researchers and developers to experiment with diferent educational recommendation algorithms on both established platforms and research prototypes, facilitating data collection. Researchers can import platform- or lesson-specific video metadata and learner models into Curio’s Elasticsearch service and try out diferent weights to optimize recommendation strategies for specific research questions.
In addition to accelerating research in educational recommendation algorithms, Curio can help researchers and educators gain insights into learner’s help-seeking behavior. One of the major challenges in measuring the help-seeking behavior of iTextbook learners is that it often happens on external platforms like Google search and YouTube. Curio ofers a solution by recommending learning content within the iTextbook, enabling researchers to gain insights into learners’ help-seeking behaviors and providing guidelines for optimal educational content recommendation strategies. Additionally, the help-seeking data collected by Curio can serve as a supplement to measure iTextbook learners’ self-regulation, agency, and curiosity, which are nowadays measured by self-report questionnaires that raise concerns about susceptibility to bias.
The data collected by Curio will be openly published and uploaded to DataShop, an opensourced repository for sharing and analyzing data on the interactions between students and educational software. Furthermore, researchers who utilize Curio for research can also collect and share the help-seeking data they collect with our data pipeline and infrastructure. learner modeling techniques, User Modeling and User-Adapted Interaction 27 (2017) 313–350. [19] K. Seta, Y. Taniguchi, M. Ikeda, Learner modeling to capture meta-cognitive activities through presentation design, the Journal of Information and Systems in Education 14 (2015) 1–12. [20] S. Bulathwela, M. Pérez-Ortiz, E. Yilmaz, J. Shawe-Taylor, Semantic truelearn: Using semantic knowledge graphs in recommendation systems, arXiv preprint arXiv:2112.04368 (2021). [21] N. Bier, S. Lip, R. Strader, C. Thille, D. Zimmaro, An approach to knowledge component/skill modeling in online courses, Open Learning (2014) 1–14. [22] M. V. Yudelson, K. R. Koedinger, G. J. Gordon, Individualized bayesian knowledge tracing models, in: Artificial Intelligence in Education: 16th International Conference, AIED 2013, Memphis, TN, USA, July 9-13, 2013. Proceedings 16, Springer, 2013, pp. 171–180. [23] GitHub - tesseract-ocr/tesseract: Tesseract Open Source OCR Engine (main repository) — github.com, https://github.com/tesseract-ocr/tesseract, 2023. [Accessed 20-May-2023]. [24] Elastic, Elastic/elasticsearch: Free and open, distributed, restful search engine, 2023. URL: https://github.com/elastic/elasticsearch, [Accessed 20-May-2023].