An intelligent textbook is a traditional textbook enhanced with a knowledge graph (KG) making it a source of enhanced learning and instruction. The nodes of a KG are key terms in a textbook and the edges are the relationships between the terms. Relationship selection is the process of selecting the most appropriate relationship type between two di erent terms to be incorporated into a KG. We demonstrate a tool that allows learners to select relationships between terms embedded in a textbook sentence. We created this tool as a key component of a scalable infrastructure for KG construction through crowdsourcing of relationships between automatically extracted terms from a textbook. This task has the potential to be exibly adapted to di erent textbooks and content domains. It is also suitable for encouraging relational processing and, we believe that it has instructional value. Therefore, our future work is focused on the pedagogical evaluation of the relationship selection task with students reading from a textbook.
Intelligent Textbooks (ITBs) using Arti cial Intelligence (AI) and knowledge
graphs (KG) allow students to dynamically interact with the textbook content,
increasing their ability to understand concepts, raising engagement, and thereby,
improving academic performance. Initial trials of ITBs that utilize KGs have
been found to improve student grade outcomes by a full letter grade over the
control group that was using a conventional textbook [
However, the process of constructing KGs that power such ITBs are time
consuming and resource intensive. For example, the Inquire Biology ITB that was
created by author Chaudhri for the popular introductory textbook, Campbell's
Biology [
We thank Abhay Agarwal for deploying the task on AWS and making signi cant additions to the code documentation.
Copyright © 2021 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).
To create a scalable AI-crowdsourcing hybrid infrastructure for KG
construction, we focused on three elements of the overall process. First, to capture the
critical terms from textbook content, we used an adapted version of the BERT
language deep learning model [
Relationship selection refers to the process of selecting the relationships between key terms in a textbook. To illustrate a relationship, consider the sentence eukaryotic cells contain a nucleus. In this example, the key terms are \eukaryotic cells" and \nucleus", and the relationship linking these two terms is \contains". Failure to identify important relationships a ects the quality of a KG and ultimately limits the performance of the ITB.
The example relationships presented in Fig. 1 can be encoded
computationally with tuples of the form (entity, relationship, entity), which is the standard
structure used in knowledge graphs. The relationships needed for ITBs include
taxonomy-based relationships (e.g., \prokaryotic cells" and \eukaryotic cells" are
both subset of the class of \cells"), meronymic relationships (e.g., "nucleus" is
inside "eukaryotic cell"), event structure relationships (e.g., "Anaphase" is a sub
step of "mitosis") and causal relationships (e.g., "di usion" enables "respiratory
gas exchange") [
There has been signi cant interest in the ML community in automatically
identifying such relationships from natural language text. However, automatic
extraction of relationships from text requires massive amounts of training data
and rarely yields the high accuracy needed for an ITB. Therefore, our strategy
was to develop a crowdsourcing task that could not only serve the purpose of
creating the necessary relationship data, but also provide pedagogical bene ts to
student participants who are actively learning the material. To this end, we
leveraged the popular educational task of concept mapping [
We describe the key steps in the development of RST: identifying relationship vocabulary, identifying terms, and developing a crowdsourcing tool.
Our rst step in this process was to choose an appropriate set of relationships
also known as relationship vocabulary. Our relationship vocabulary is based on
an upper ontology called Component Library or CLIB [
We used automated term extraction to identify all the terms in the textbook. As the precision and recall of the automated method is not perfect, the biologists on the team validated the terms. We then parsed the textbook section into individual sentences and automatically identi ed all term pairs that existed in each sentence. A sentence that contained N terms would have n2 possible pairings, with each pairing considered to be a single task. After generating all possible tasks, we presented them to crowd workers using the tool that we describe next. 2.3
We developed a tool to guide the user in choosing the correct relationship between a pair of terms in the context of a sentence. The intended user of this tool is a student who does not have any formal training in knowledge engineering. During the development of the tool, we iteratively validated our designs through rapid prototyping with such users. The user is rst asked to read a section from the textbook and then to undergo a short training on the relationships. We designed the training using simple common-sense examples that new users would nd easy to understand. For example, we explain the is inside relationship using a visual in which a cat is shown hiding inside a box (Fig. 2). We developed similar illustrations for all the di erent relationships supported by the tool. After the training, the user completes a series of tasks through an interactive dialog to identify relationships between various textbook terms. All possible relationships between a pair of concepts can be extremely large. Some simple insights make our task tractable, namely: most term Fig. 2. Illustration of the pairs are not related (i.e., the nal graph is sparse), inside relationship. the terms that are connected are also likely to co-occur closely in the text, and that we can group the relationships into families so the user rst chooses a relationship family before choosing the actual relationship (Fig. 3).
Fig. 3. Respondents on the Relationship Selection Task rst select the correct family of relationship, followed by the actual relationship.
As a concrete example, consider this two-step selection in the dialog shown in Figure 3 where the user is asked to relate the terms \cytoplasm" and \nucleus". The user rst chooses the appropriate relationship family for the terms, including taxonomic, spatial, and component-based relationships. The user further can select that the terms have no relationship between them, that they are unsure of the relationship, or that they would like to de ne a new relationship to relate the terms. In this example, the correct relationship family is a spatial relationship and clicking on this option takes them to a second set of options to specify which spatial family relationship is correct. In this dialog, they have an option to ip the order of the terms to ensure that the chosen relationship applies in the correct direction. Once they ip the order of terms, they can correctly indicate that the nucleus is inside cytoplasm.
Using the RST, we successfully crowdsourced relationships between terms in sections of college level biology and psychology textbooks respectively, and are currently investigating its pedagogical e cacy.