This research focuses on determining whether a student's GRIT impacts their behavior within an intelligent tutoring system, towards developing better student models and feature sets that can help a tutor predict student behavior and determining whether computer tutors might foster improvements in students' grit, perseverance and recovery from failure. We use rare Association Rule Mining to explore how students' grit may be associated with students' behaviors within MathSpring, an intelligent tutoring system, as a first step.
Studies have shown that grit is more predictive of life’s outcomes compared to the
“Big Five” personality model, which is a group of broad personality dimensions (e.g.
conscientiousness, extraversion, agreeableness, and neuroticism [
Our research focuses on how a student’s grit and perseverance might impact behavioral patterns in a tutoring system, towards understanding how digital tutors might foster gritty-like behaviors, and in turn, grit assessments.
We move research on grit forward as a tool to refine student models in intelligent tutoring systems, by answering the following questions:
Grit has typically been assessed using Duckworth’s instrument of the Grit Scale [
Our testbed is MathSpring, an intelligent tutoring system (ITS) that personalizes
problems by assessing students’ knowledge as well as effort and affect as they engage
in mathematics practice online [
Seventh grade students from two school districts participated in a research study. After combining the two datasets, there were 456 rows of Grit survey responses representing thirty-eight students. Sixty-eight students used MathSpring, producing 3,012 rows of data, each representing a student-math problem interaction. Variables were discretized into Booleans, indicating high/low or true/false. We created the negation of each variable (e.g., for GUESS, we also created a counterpart NoGUESS variable with the opposite truth value) to be considered also. Along with Guess, other variables included Hi/Low Grit, is/is not Solved, Hi/Low Mistakes, Hi/Low Hints, Yes/No Finished, Not/Likely Read (the problem).
We used Association Rule Mining to discover rules, a non-parametric method for exploratory data analysis, which finds associations that occur more frequently than expected from random sampling. The four critical parameters and minimum thresholds used are the following: Support 0.05, Confidence 0.84, Lift 1.15, Conviction 1.75. Last, we subjected the most important rules to a Chi-Square statistical test, those with solely “High Grit” or “Low Grit” as a consequent or antecedent. 3
The mean Grit Score for the N=38 students in the sample was M=3.07, SD=0.51,
Median=3, Range= [
Due to a low support threshold of 0.05, thousands of rules were created. Only a selected subset of rules was chosen for interpretation, mainly those rules with a single consequent or antecedent, and those which met thresholds and had highest values for the metrics of confidence, conviction and lift.
A notable finding was that no rules with LowGrit as a consequent appeared at all according to our criteria specified in the parameter thresholds. This made us realize that, due to the much lower number of math problems seen by Low Grit students, the confidence for any rule with LowGrit=1 as a consequent would be at chance level at 0.288 (as opposed to 0.5). We realized how the confidence metric is not very reliable in this case due to the imbalanced dataset. On the other hand, the metric that balances the rarity of the premises of a rule and their confidence is the ‘conviction’ parameter. We thus set conviction as our first priority for selection of rules.
Table 5 shows the rules that had the highest conviction, confidence and lift. These rules also are the most complete rules (as generally subsequent rules that met the parameter thresholds had similar premises, but combined subsets of the propositions). Rule A is the rule with highest confidence, conviction and lift, and states that if a student made a high amount of mistakes in a math problem, and asked for many hints as a way to help them solve the problem, then it means the student has a high level of Grit. This joint condition happened in 19% of the total student-problem interactions examined. The significance of the effect for each rule was verified with a Chi-Square test by computing cross-tabulations between the premise being true/false vs. High/Low Grit (p<0.0001 for rules 1, 2, and 3).
On the other hand, no rules were found that met the thresholds of confidence, lift and conviction for LowGrit as a consequent. Still, we show the rule that has the best outcome for those metrics. The implication LowMistakes ^ isSolved → Low Grit has a confidence level of 0.45, which is low, however, it is higher than chance as stated earlier (chance level for any LowGrit row is 0.288). The rule suggests that if a student solves problems by making a low number of mistakes, then the student is NOT gritty.
Table 6 summarizes the found rules with Low/High Grit as a premise. This time, it was easier to find rules with LowGrit as an antecedent that met the thresholds of confidence, lift and conviction but not for HiGrit. Rule C is the main rule found for Low Grit as an antecedent (other similar rules are variations of this same effect), suggesting that if a student has low grit, then they will likely ask for few hints in a problem.
The rule that contains HiGrit as an antecedent is Rule D. While Rule D does not
meet the lift and conviction thresholds we had set, it does meet the confidence
threshold, and is the rule found with the highest values of confidence and conviction. This
rule captures that if a student is gritty, then the student will not quick-guess the correct
answer to a problem. Remember that guessing implies that a student entered many
answers incorrectly and did not ask for help/hints, until they manage to solve it correctly
(the multiple-choice format in most questions in MathSpring probably favors this type
of disengagement behavior in general). We consider that students who guess are
avoiding help when they should instead be asking for it, as they are answering incorrectly, as
stated in previous research [
It was harder to find Association Rules that associate students with low grit with behaviors (there are not as many systematic behavior patterns that could be associated to students of low grit). Still, the few rules found suggest that when a student has low levels of grit, they will seek for a low amount of hints. Conversely, the behavior that a student is NOT gritty is that he/she makes a low number of mistakes and eventually solves the problems correctly. Given the agency that MathSpring allows (more than most other learning environments) this does not necessarily mean that low-grit students tend to solve problems correctly (otherwise solve-on-first would have been part of the rules found). Students who skip problems or give-up will receive easier problems in an adaptive tutor. Also, students could choose material that is easier, or already mastered, to guarantee higher levels of success. Further analyses could help discern if this is the case, by analyzing the level of difficulty of the problems students received. Grit is a construct that will predetermine students to have different kinds of self-regulatory behaviors while learning in interactive learning environments.
This material is based upon work supported by the National Science Foundation under NSF award #1551594 INT: Collaborative Research: Detecting, Predicting and Remediating Student Affect and Grit Using Computer Vision. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.