When students interact with an eTextbook, it typically logs their interactions while engaged in activities like watching a visualization, attempting to solve an exercise, or refreshing the page. These event logs allow instructors and researchers to evaluate students' engagement level and approaches to using the artifacts. We predict that the way students use the book and the artifacts a ects their performance on the exercises, their learning gains, and their performance in other aspects of the course. In this paper, we describe a data set gathered from a complete semester course on Formal Languages. This includes all student interactions with the Formal Languages eTextbook. The book contains a set of auto-graded exercises and visualizations about Formal Languages course contents in the form of slideshows.
Formal Languages course is a theory course that contains a
number of proofs and on-paper assignments. Formal
Languages courses face a few challenges. They are often
presented as fairly abstract and highly mathematical. This has
the bene t of making students practice useful skills like proof
writing, but might make it less appealing to students more
used to the hands-on style of the typical CS programming
course. A typical FLA class presents several models of
computing (deterministic and non-deterministic nite state
machines, regular expressions, push-down automata,
contextfree languages, Turing machines), with many proofs about
Copyright ©2021 for this paper by its authors. Use permitted under
Creative Commons License Attribution 4.0 International (CC BY 4.0)
their relationships and limitations. There are many
algorithms associated with each model that students must learn
to apply. Many instructors have their students use
simulators to support this process, such as the state-of-the-art
simulator Java Formal Languages and Automata Package
(JFLAP) [
To increase student understanding and interaction with the
course materials we implemented an eTextbook using the
OpenDSA system. The OpenDSA project [
JFLAP is used extensively in FLA courses to help students
visualize and observe the behavior of models and associated
algorithms [
These drawbacks inspired us to develop an open-access,
webbased version of JFLAP with enhanced support for
autograded exercises. We have largely re-implemented JFLAP
functionality within the OpenDSA framework. We refer
to it as OpenFLAP. OpenFLAP is implemented using the
JSAV library. OpenFLAP also allows us to create exercises,
auto-assess them, and report the result to an LMS through
OpenDSA's standard framework [
OpenFLAP allows us to create two types of exercises.
Auto-Graded exercises Auto-graded exercises ask students to build di erent models, i.e., Deterministic Finite Automata (DFA) or writing a Context-Free grammar. These exercises are similar to programming exercises. To test students' solutions, instructors can assign some test cases. That can be used to test the correctness of students model/grammar.
Pro ciency exercises Pro ciency exercises ask students to apply an algorithm to a given model like convert an NFA to a DFA. OpenFLAP allows instructors to create pro ciency exercises where students need to apply algorithm steps on a given model. OpenFLAP checks the correctness of every student step and shows a message to the student to prompt them to retry the incorrect step before moving forward to the next steps
When students work with our eTextbook, the OpenDSA system collects data about students interactions with the book components. The book contains several slideshows, exercises, khan-academy exercises, and traditional text with some images. Students need to answer all exercises to earn credit and they can freely skip looking at the slideshows or read the text. Our Formal Languages eTextbook includes: Prose and images. Traditional text about the algorithms and proofs for di erent Formal Languages models. We added some images that can help to understand the text.
Slideshows A series of slides is often used to describe a topic to students. Slideshows include four buttons that allow students to navigate in the slide show. These buttons are a) next slide, b) previous slide, c) rst slide, and d) last slide. Figure 1 shows an example for NFA to DFA slideshow. Auto-graded exercises and Pro ciency exercises. A large number of exercises are available, related to building various example machines. Exercises are included that require students to build Deterministic and NonDeterministic Finite Automata, Push Down Automata, or Turning Machines. Some exercises are about writing Grammars for a given language, or converting a model to another model. All exercises require students to score 100% correctness to get the credit for the exercise. Students can repeat the exercise as necessary to achieve credit. Figures 2 and 3 shows examples for Auto-graded exercises and pro ciency exercises. Multiple Choice, T/F, Fill-in-the-blank. OpenDSA includes many questions in standard simple question formats, implemented using the Khan Academy Exercises Framework. Figure 4 shows an example for a Khan Academy exercise.
Every primitive user interaction (button clicks, page loads, window focus and blur events) is captured and stored in the database. Table 1 lists speci c events from the data set along with their meaning.
The data comes in the form of a CSV le with 262205 rows, where each row is an event that is made by a student. Each event row includes the interaction ID, user ID, event name, event description, event time, browser name, Operating System name, Device used, and chapter id.
event Window-load Window-focus jsav-forward jsav-exercise-reset jsav-exercise-grade jsav-matrix-click jsav-node-click submit-deleteButton submit-edgeButton window-unload description loaded a book module
Window focus Slide show forward button
Exercise reset button
Request exercise grade Click in cell for grammar production
Select a graph node
Deleted a graph node Button click: enter add-an-edge state
Closed the module
The data set can be found at https://pslcdatashop.web. cmu.edu/DatasetInfo?datasetId=3427.
This work is supported by the National Science Foundation under grants DUE-1139861, DUE-1431667 and IIS-1258471. The Egyptian Ministry of Higher Education funded Mostafa Mohammed during his PhD. We are grateful to the many, many students who have worked on OpenDSA, OpenFLAP, and the FLA eTextbook over the years.