Active learning has been shown to provide bene ts over traditional didactic approaches to teaching and learning. These bene ts are particularly important in elds in which students must master large amounts of information and e ectively operationalize it, as in medicine and dentistry. While online learning platforms have the potential to provide students with active learning without taxing scarce faculty resources, a recognized challenge in producing such systems is the engineering of the domain knowledge needed for engaging interaction. In this paper we address this problem by developing an open-source chatbot-based tutoring system trained on dental textbooks in the area of endodontics, one of the most challenging areas of dentistry. Dental TutorBot is built using Rasa for modular training purposes. It asks short questions from students and evaluates their answers. If the student cannot answer a question, the system provides a hint, rather than immediately giving the student the answer. In this way, it coaches the student to nd the answer and thus helps them to understand the connections between concepts while creating a more intellectually stimulating learning experience.
Active learning has been shown to provide bene ts over traditional didactic
approaches to teaching and learning, including increased learner engagement,
material retention, critical thinking, and problem-solving ability [
A healthcare profession that could bene t positively from such an implementation is that of Dentistry. With the precaution yet necessity of social distancing when dealing with possibly asymptomatic patients, a dentist's job { which is reliant on applicable learning { can be signi cantly impacted. In comparison to other elds of education, medical areas still lag behind due to the reluctance of learning appropriately and e ciently by virtual education. It has hence become important for aspiring dentistry students to be able to learn the theoretical and practical aspects of their expected job by digital means just as well as traditional ones during such times.
To address this concern, this research aims to develop an open-source chatbotbased tutoring system trained on dental textbooks to teach and test dental students. It is trained on standard Natural Language Processing (NLP) datasets along with academic textbooks pertaining to endodontics, one of the most challenging areas of dentistry. The Dental TutorBot is built using Rasa for modular training purposes (rasa.com). It asks short questions from students and evaluates their answers. If the student cannot answer a question, the system provides a hint, rather than immediately giving the student the answer. In this way, it coaches the students to link their understanding to the answer, thus helping them gauge the connections between concepts better while establishing a more intellectually stimulating exchange.
Prior work has explored development of intelligent tutoring systems for
medical problem-based learning as one approach to active learning. The work of
Suebnukarn et al. [
Chan et al. [
Bocklisch et al. [
The IBM Watson Tutor [
In this section we describe data acquisition from dental text books, as well as the methods used to build the Dental TutorBot system. 3.1
This work uses the text corpus from the work of Yin et al. [
Our overall objective is to create a tutoring system in which the TutorBot asks the students questions based on the topic and di culty level they select, checks whether the answers they provide are a) correct, b) semi-correct or c) incorrect, and provides solution hints to coach them to nd the right answer. The speci c objectives achieved in pursuit of this are described below.
Question Answer Generation: First, such a TutorBot must be di erent { in implementation and deployment { than usual chatbots since the user responds to the bot's query rather than vice versa. By this, the user in question { that is the student utilizing the system { would respond to the bot's questioning in a continuous manner. This speci cation has primary importance in the setting up of the research problem, as the approach varies substantially: we must implement the training and learning phases of each sub-solution with respect to the bot's responses, rather than the user's as the dialogue at their end would be limited to providing correct or incorrect responses to questions, topic selection, the usual salutation/goodbye, and the start/end period commands to signal the initiate/stop of the tutoring session. However, at the bot's end, we must check if relevant questions and their correct answers are being generated and selected given the user's topic selection. After this validation, the dialogue ow would initiate in a loop, until the generated QA pairs for the selected topic or di culty have been exhausted (in which the tutor would ask to select a new topic or to adjust the di culty level, or end the session altogether) or the user has speci cally asked to stop the session. While within this loop, the underlying bot system keeps checking the similarity of the student input with the answers stored in a database housing pre-generated QA pairs from dental textbooks. A certain level of matching would see the response deemed correct, and if this match fails to meet the set threshold { say 75% { clues will be provided in order to aid recalling or at least attempting to guess the correct answer.
This process of QA generation is carried out by the T5 model through the algorithmic approach of answer aware generation, in which the model is presented with user-selected topic sets of all exhaustive passages extracted from the dental textbooks - for which it has been trained - where each potential sentence is considered an answer to a question. The system generates a question for these 'answers' by considering the passage context. Ingesting the textual data in the form of spans, questions are mapped to each answer outputted by way of the Encoder-Decoder framework and then compared with the generated answer pair of the same model validated on SQuAD dataset, which is used as baseline to check the capability of the questions being generated due to its academically accepted nature. The encoder captures the passage sentences as answers based on each sentence span, while the decoder then breaks down these very sentences into possible pairs of possible question and potential answer. Higher similarity to the ingested passage text amongst these pairs implies that a correct QA pair was generated, and is thus appended to a comma-separated values (csv) le with the relevant metadata read from the dental texts. This approach ensures that relevant questions are generated instead of non-addressing ones. Hint Generation: The sample questions show a potential set of questions similar to what are asked in physical courses. To generate clues for the student interacting with the TutorBot in case of incorrect responses, we address the \hint" as a semantic chunk in an input passage that will be included (or rephrased) in the target question and the accompanied answer. Based on this de nition, we perform syntactic parsing and chunking on input text, and identify those chunks which are the most relevant to the target question as the clue set. By leveraging two sub-systems for the task (with and without the Word Embeddings) - Model A for Lexical Simpli cation using WordNet, BabelNet and Rapid Automatic Keyword Extraction (RAKE) for the processes of Text Summarization, Expansion, Keyword Similarity and Rephrasing - along with Model B, which uses a GPT-2-small model trained on Semantic Clues learned from SQuAD using above pipeline and tested with the dental text. GPT-2 was used to exploit its generative ability to produce coherent text from minimal inputs. From our initial test-run, the word embedding module coupled with Model B performed the best in giving syntactically learned hints. A few examples of hints from these models are presented in Table 1. However, due to the lack of exhaustive hint-related data that could be sourced once the system is tested by its targeted audience as in uenced through the literature review conducted, the hints still remain quite similar to the actual answer. With more data consumed from its intended employment, the models are expected to comparatively have a better chance to learn and provide more generic and answer-excluded hints.
StudentTeacher interaction
Sub-modules of the system, integrated with
Rasa into one unit Intents Stories Domain.yml
Initial Bot
Training Fig. 1. Overview of TutorBot Framework showing the QA module (for question-answer pair generation), the Hint module (for hint generation) and the Topic module (for topic identi cation).
Rasa Framework: The modules described above are incorporated into the Rasa bot using the system architecture shown in Fig. 1. The integrated subsystems are able to e ectively handle multiple modes of interaction between the student and the TutorBot, based on the user intent and accompanying action trigger. Student answers are matched against model-generated answers using cosine similarly. If the similarity between answer and student response is less than 75% a hint is generated to help the student answer the question. For bot interaction, we use Rasa which is easily integrated with di erent platforms and supports multiple languages that are available for such NLP exploitation. The students directly interact with the TutorBot in this manner as shown in Fig. 2, which illustrates how di erent parts of the interaction with the student are implemented on Rasa. After the initial interaction where the tutoring session is initiated, the inherent question-answer ow and hint generation start concurrently as well.
While conducting this study, the primary step was to deploy an essentially working mechanism and then enhance it by extending on the same order of sub-problems. We were able to successfully generate appropriate and extensive Question-Answer pairs, label topics for these pairs by using their associated occurrence in the dental textbooks, enable hints with regards to the user's response and conduct matching to determine correctness of answers. Most importantly, we were able to bring these sub-modules into one customized API and link the Rasa chatbot to it as well for endowing the dialogue ow with instructional content. From this preliminary study, we intend to investigate several ways to further augment the user{bot experience with the roles of a student{teacher relationship.
As seen in Table 1, the resultants of the QA and Hint Module show considerable improvement in the hint generation alone once the dataset's extracted word embeddings are utilized in the train-test phase of the module. With the local information well-captured by the embedding (Model B), the hints are likely to become more relevant and clue-like to the student rather than what a simple rewording of the keywords would ever appear to be (Model A). Our future work includes analyzing how certain parameters { when ne-tuned { could a ect the response of the system. For example, the threshold for similarity matching could be learned by the TutorBot by carrying out trend analysis for each individual question in a tutoring session. Similarly, we aim to include multiple and more profound hints in order to ease the student's experience and provide maximal learning capability. Once the system has been well-used by the stakeholders of the research problem { the students and the dental experts { users can provide rating tags for questions after each session in order for the bot to adapt and provide questions based on the gaps in the student's knowledge, apart from the initial topic selection. This will provide an evaluation metric to gauge the di culty level of associated topics and their related questions, while capturing the number of attempts the student takes in answering them accordingly. Field testing and evaluation by dentists along with medical students is a necessity even if rating generation was not an immediate concern, as they can provide better elements of critique. The problem upon conclusion aims to create testing and feedback procedures of the implementation in collaboration with the respective dental experts, health-care professionals and students to maximize the in uence and outreach of this research.
Up to this point we have evaluated the system as part of the iterative development process, receiving feedback from endodontists at each iteration. We are currently working on hosting the custom API as a mobile application in order to evaluate the system's e ectiveness in teaching students. Once deployed, we will be able to gauge the student and bot responses and to ne-tune the system. In addition, we plan to expand to other question types such as multiple-choice questions and descriptive questions.