The Impact of Cognitive and Non-Cognitive Text-Based Factors on Solving Mathematics Story Problems Candace Walkington Virginia Clinton Steven Ritter Southern Methodist University University of Wisconsin - Madison Carnegie Learning, Inc. 3011 University Blvd. Ste. 345 1025 West Johnson Street 437 Grant Street, Suite 918 Dallas, TX, 75205 Madison, WI 53706 Pittsburgh, PA 15219 1-214-768-3072 1-608-890-4259 1-888-851-7094 x122 cwalkington@smu.edu vclinton@wisc.edu sritter@carnegielearning.com Mitchell Nathan Stephen E. Fancsali University of Wisconsin - Madison Carnegie Learning, Inc. 1025 West Johnson Street 437 Grant Street, Suite 918 Madison, WI 53706 Pittsburgh, PA 15219 1-608-262-0831 1-888-851-7094 x219 mnathan@wisc.edu sfancsali@carnegielearning.com ABSTRACT domain model and student model [1]. It is through the Intelligent tutoring systems (ITSs) that personalize construction of the student model and its contribution to the instruction to individual learner background and tutoring model that ITSs can enact personalization where preferences have emerged in K-16 classroom settings all they adapt to the needs and backgrounds of individual over the world. In mathematics instruction, ITSs may be learners. Here we explore cognitive and non-cognitive especially important for tracking mathematical skill factors related to how students react to and understand the development over time. However, recent research has text of mathematics story problems. We argue that these pointed to the importance of text-based measures when non-mathematical factors may be an important element to solving mathematics word problems, suggesting that in consider for an ITS in secondary mathematics. In order to accurately model the student it is important to particular, we provide evidence suggesting that both the understand how they respond to text characteristics. We students’ reading level (a cognitive factor) and the students’ investigate the impact of text-based factors (readability and interests, preferences, and motivational outlooks (non- problem topic) on the solving of mathematics story cognitive factors) have the potential to influence how they problems using a corpus of N = 3394 students working respond to text-based mathematics problems situated in through an ITS for algebra, Cognitive Tutor Algebra. We “real world” contexts. leverage recent advances in computerized text-mining to Cognitive Tutor Algebra (CTA; [2]) is a prominent automate fine-grained text analyses of many different word mathematics ITS used in many schools across the United problems. We find that several elements of the text of States. CTA uses model-tracing approaches to relate mathematics word problems matter for performance – student actions to the domain model and provides including the concreteness of the problem’s topic, the individualized error feedback. CTA also uses knowledge- length and conciseness of the story’s text, and the words tracing approaches to track students’ learning from one and phrases used. problem to the next, using this information to identify the students’ strengths and weakness in terms of production Keywords rules (i.e., knowledge components or skills). The software Intelligent tutoring system, readability, mathematics, word then uses this analysis to individualize the selection of problems, personalization problem tasks. However, missing from this tutoring model is a consideration of other non-mathematical characteristics 1. INTRODUCTION of the story problem texts – including the reading difficulty Since the 1980s, Intelligent Tutoring Systems (ITSs) have of the text respective to students’ reading ability and risen as an important instructional tool to support student preferences, and the real-world topic of the text respective learning in classrooms, especially in middle and high to students’ interests and preferences. school. ITSs typically consist of at least three components: For example, a learner presented with a mathematics (1) the domain model of the appropriate steps needed to word problem that is difficult to read – with high-level correctly solve each problem, (2) the student model, which vocabulary, complex sentence structure, etc. - may lack the captures the evolution of an individual student’s cognitive reading ability to appropriately comprehend that problem. states as they relate to the domain model, and (3) the This cognitive element of the problem’s difficulty is not tutoring model which selects tutor actions based on the typically monitored by ITSs for mathematics learning. In addition, such a problem may inhibit the students’ because of their relatedness to learner prior knowledge. In motivation – a non-cognitive factor. In particular, even if related work [5], they also identified the prevalence of the learner is technically able to read the problem, they may issues with verbal interpretation of mathematics story be intimidated by the problem text, and request a hint problems, finding that even high school students struggle to instead of putting forth the effort of understanding the text understand difficult vocabulary words and construct an of the problem. ITSs also do not typically monitor the accurate propositional textbase and situation model from a learner motivation for reading and understanding text-based story problem’s text. problems. 2.2 Non-Cognitive Factors Another non-mathematical element of the text of An important precursor to students’ motivation is their mathematics story problems is the real world topic – level of interest – defined as the state of engaging and the whether the story is about working at a part-time job or predisposition to re-engage with particular topics, ideas, or harvesting a field of grain. The way in which students react activities [6]. Two types of interest have been described in to the topic of the story problem is also based on both the literature. First, situational interest is an immediate, cognitive and non-cognitive factors. Students may be temporary state of heightened attention and affective unfamiliar with elements of the context that are important engagement that stems from elements of a learning for fully comprehending the problem – for example, in a environment that are surprising, salient, evocative, banking context, they may not know what “break even” challenging, personally relevant, etc. Situational interest means. In this way, they may lack the prior knowledge can be triggered in response to a stimuli within a learning needed to interpret the story. Similarly, different real world environment, and then may or may not become maintained topics may differ in the motivation they elicit from students over time [6]. A second type of interest is individual – students may experience greater motivation when solving interest – learners’ enduring predispositions to engage with a problem about a familiar, interesting context than about a certain activities or topics over time. context they find boring or unfamiliar. Elements of a story problem’s text have the potential to We next provide a theoretical framework that provides both trigger and maintain situational interest. In particular, an explanation of how students comprehend story problems story problems that are accessible, easy to read, and and how cognitive and non-cognitive factors may interact situated within the topics and contexts that a particular as they solve story problems. learner finds relevant and interesting may trigger and maintain interest. In the other hand, difficult reading 2. THEORETICAL FRAMEWORK passages disconnected from a learner’s experiences and interests may not trigger interest and may cause 2.1 Cognitive Factors disengagement if interest has previously been triggered. Nathan and colleagues [3] proposed a model of mathematics story problem solving where students navigate 2.3 Research Purpose three levels of representation as they comprehend and solve If text-based measures like readability and problem story texts: (1) a textbase containing the propositional topic matter for student performance, these might be statements made in the story problem, (2) a situation important elements to add to future systems for model, a qualitative representation of the actions and events personalized learning in mathematics. For example, an ITS in the story, and (3) a problem model, containing the formal might present weak readers with problems with simplified mathematical equations, variables, and operands. Because verbal language as these learners are initially mastering a mathematics word problems are stated in verbal language new mathematical skill. As the student gains expertise with (rather than mathematics notation), we hypothesize that the the mathematics by mastering skills, additional levels of reading difficulty and topic of the problem matters for the verbal difficulty could be layered on by the ITS. Similarly, construction of the situation model and its successful learners that lack motivation may be presented with story coordination with the problem model. problems that are less intimidating to read and situated Various aspects of the reading difficulty, including within their interests, with this support faded out over time. readability measures, may be important in situation model By neglecting to model this aspect of the user’s experience construction. Readability measures often include the kinds in the ITS, the system may be generating inferences about of words used, the length of the story, and the structure of learner knowledge states that are inaccurate. the sentences. These elements of the text’s structure may make it more difficult to comprehend, especially for 3. LITERATURE REVIEW students with weaker reading skills. 3.1 The Impact of Reading Difficulty on Another aspect of reading difficulty is the topic of the problem – whether it is about, for example, farming or Solving Mathematics Story Problems banking. Walkington and colleagues [4] proposed that story Recent research has found that reading ability is especially contexts that are related to topics that are familiar and important as students solve mathematics word problems accessible to students are easier for them to solve because [7]. Studies examining the association of reading difficulty these contexts can facilitate situation model construction of mathematics word problems and U.S. student performance on large-scale assessments has found that evidence points to the importance of considering the real problems that use words with multiple meanings, complex world topic of mathematics story problems and its verbs, and mathematics vocabulary words are more relationship to students’ interests and experiences. difficult [8]; the effect is especially pronounced for students However, more research is needed to determine which who speak English as a second language [9]. A small study topics may be more or less likely to trigger and maintain of students working in CTA found that extraneous text that students’ interest. provided a real world context for the problem, as well as references to concrete people, places, and things, were 3.3 Research Questions associated with less concentration and more confusion in In the present study, we investigate the relationship the tutor [10]. However, a similar study found that the between readability and topic measures and student extraneous text was also associated with fewer performance on mathematics story problems. We examine unproductive “gaming the system” behaviors in the tutor these issues within an ITS for Algebra I, Cognitive Tutor [11]. Converging evidence suggests text characteristics Algebra (CTA), that tracks student hint requests in addition relating to reading difficulty are important when solving to whether they get problems correct or incorrect. We mathematics word problems, but studies are needed that investigate two research questions: (1) How are readability address which elements of reading difficulty are most and topic measures associated with correct answers and important. hint requests when students label independent and dependent quantities in stories in CTA? (2) How are 3.2 The Impact of Problem Topic on Solving readability and topic measures associated with correct answers and hint requests when students write algebraic Mathematics Story Problems expressions from stories in CTA? Answers to these The topic of mathematics story problems also has an questions could inform the design of future ITSs for important relationship to students’ prior knowledge and personalized instruction. motivation. A study of high school students solving either standard story problems or story problems personalized to 4. METHOD topics they were interested in (e.g., sports, video games, Data from N = 3394 students with active CTA accounts social networking) within one unit of CTA found that were collected from 9 high schools and 1 middle school personalized stories were associated with higher that were diverse in terms of their socio-economic, racial, performance. This performance gain was present in two and achievement background (Table 1). Data were tasks – labeling independent and dependent quantities collected for students solving 151 distinct word problems given in algebra story problems, and writing algebraic accross the first 8 units of CTA; later units were not expressions from the story scenarios [12]. It was included because many students did not advance beyond hypothesized that during these two tasks, students are these units. We collapsed for all analyses (i.e., treat as working closely with the problem text, constructing their identical) problems containing an identical story but using situation model and coordinating it with a problem model. slightly different numbers. On average, each problem had This study also found that students receiving problems in been solved by 742 students (SD = 495). Each problem the context of their out-of-school interests were less likely included a story scenario that outlined one or more linear to game the system – to exploit regularities in hints and functions within a real world situation (Figure 1). The feedback provided by CTA in order to avoid productive student was asked to complete steps in which they learning behaviors. Further, students who received identified the independent and dependent quantities in the personalization had stronger performance in future units story, wrote a linear algebraic expression for the story, and where the problems were no longer personalized. solved their expression for different x and y values; we In a recent follow-up study [13], story problems in four consider only the first two skills. units of CTA were personalized to topics students were CTA log data from students in the selected schools interested in, and students solving personalized problems were uploaded to DataShop (pslcdatashop.web.cmu.edu), were compared to a control group solving normal an online repository of detailed student interaction data. problems. Results showed that personalized problems both These logs contained information on whether the student triggered students’ situational interest and enhanced got each problem correct, incorrect, or requested a hint on students’ individual interest for learning algebra. their first attempt; because requesting a hint is a distinct Personalization was associated with greater learning gains outcome, correct and incorrect are not completely repetitive than a control condition only when the personalization was measures. Thus, for each problem, we compiled the matched to deep features of the students’ interest area. This percentage of students who had gotten the problem correct was contrasted with personalization that was only matched on the first attempt, incorrect, or requested a hint. This surface features of the learners’ interests – i.e., percentage was our dependent measure in three distinct modifications to the problems that simply involved regression models.We analyzed the text of the introduction inserting familiar pop-culture words rather than considering to each story problem (i.e., the initial text that gives the how learners might actually use relationships between linear rate of change and intercept; see Figure 1) with the quantities in their everyday activities. Thus converging Coh-Metrix and LIWC text-mining programs. Coh-Metrix [14] measures a large number of aspects of text readability, including the amount semantic overlap between sentences, the number of verbs, use of concrete versus abstract words, the average sentence length, and others. Table 1. Demographic characteristics of schools in study ID Math State School School Prof % Prof % Enrollment Type 1 88% 70% 797 Middle 2 81% 47% 1,482 High 3 95% 84% 2,163 High 4 55% 46% 708 High 5 27% NA 1,875 High 6 68% 59% 986 High 7 2% 31% 602 High 8 76% 84% 1,333 High 9 19% 39% 397 High 10 68% 79% 800 High ID White Black Hispanic F/R Lunch 1 72% 7% 15% 21% 2 90% 4% 2% 4% 3 84% 10% 3% 6% 4 99% 1% 1% 41% 5 20% 4% 72% 77% 6 9% 2% 88% 41% 7 1% 99% 1% 82% 8 36% 60% 2% 48% 9 100% 0% 0% 45% 10 38% 51% 11% 62% Because some of our story introductions had only one sentence, measures that pre-supposed multiple sentences Figure 1. Screenshot of algebra story problem in CTA with were ommitted. LIWC [15] was used to determine the topic answer key superimposed of the story problems – this program counts how many words in the story fall into various word categories, including social processes (family, friends, people), For each category in Coh-Metrix and LIWC, the affective processes (positive emotions and negative correlation was computed between the list of each emotions), biological processes (body, health, ingestion), problem‘s score on that category, and the percentage of cognitive processes (insight, causation, discrepancy, students who got each problem correct, incorrect, or tentativeness, certainty, inhibition, requested a hint. Correlations that were significantly inclusive/exclusiveness), perceptual processes (see, hear, different from 0 were tested for inclusion as fixed effects in feel), relativity processes (motion, space, time), and regression models predicting the performance measures personal concerns (work, achievement, leisure, home, (hints, corrects, incorrects). These models included random money, religion). If a story contained any words that fell effects that described various aspects of the problem’s into one of these topic categories, that story was coded as a mathematical structure, including the unit and section it 1 for that category; otherwise it was coded as a 0. came from in CTA, and the numbers it used. Models were initially fit using the lmer() command in R including all potential fixed and random effects. Then we used the step() command in R to perform backwards elimination on fixed and random effects, leaving a model with only the effects abstract physics quantities – like distance and speed – may that significantly improved the fit of the model. These have been more difficult for students than using quantities analyses were carried out separately for a dataset that relating to specific concrete objects (e.g., accumulating included only instances of students labeling independent cards, toys, or money). Finally, inhibition words were and dependent quantities, and a dataset that included only associated with more hint requests. Inhibition words were instances of students writing algebraic expressions. often included in story problems that discussed safety issues or saving money. Students may have persieved these 5. RESULTS less concrete, finance- or safety-oriented contexts as less accessible, making them more likely to request a hint rather 5.1 Labeling Independent and Dependent than attempt to write the labels. These problems often Variables involved money as the dependent variable, but the label for Regression results showing the relationship between this variable may have been complex because the actor in performance measures (% incorrect, hint, and correct) and the story might have already saved or spent some money readability and topic measures for labeling quantities in when the story started. Thus a label of simply money may story problems are provided in Table 2. Table 2 shows that not be appropriate, and the student would have to enter a problems that use adverbial phrases (DRAP) were label that captured that it was total money or net money associated with fewer incorrect answers. Adverbial phrases saved or spent. are phrases that add on to verbs, answering the questions where, when, or how? In the present data set, adverbial 5.2 Writing the Algebraic Expression phrases mostly answered when the action occured, and Regression results showing the relationship between often included words like currently, already, next, first, performance measures and readability and topic measures every day/week, and not yet. However, some of these for writing the expression are shown in Table 3. We again adverbs also answered the how question, relying see that inhibition words – often associated with financial information about quantities that might be useful to cue contexts – are more difficult for students – they are students to the constraints of the problem – examples of associated with more incorrect answers, more hint requests, words used in this manner included only, completely, and and fewer correct answers. The conceptual difficulty of this evenly. These words may have given important details topic area might become especially important as students about how the quantities involved in the story were move from formulating their situation model to changing as the action in the story proceeded. coordinating their situation model with a problem model. Table 2. Regression tables relating performance Table 3. Regression tables relating performance measures on labeling quantities to readability/topic measures on writing expressions to readability/topic categories categories Estimate Std. Err t value Pr(>|t|) Estimate Std. Err t value Pr(>|t|) % Incorrect % Incorrect (Intercept) 0.182 0.032 5.63 0.00018 *** (Intercept) 0.195 0.060 3.26 0.00167 ** DRAP -0.0008 0.0003 -2.34 0.02104 * WRDPOLc 0.0494 0.013 3.91 0.00014 *** motion 0.036 0.0137 2.65 0.00899 ** inhibition 0.086 0.034 2.52 0.01286 * % Hint % Hint (Intercept) 0.045 0.014 3.21 0.01407 * (Intercept) 0.055 0.014 3.95 0.00050 *** One sentence (ref.) inhibition 0.023 0.008 2.83 0.00543 ** Two sentences -0.045 0.016 -2.82 0.00548 ** % Correct Three Sentences -0.057 0.017 -3.48 0.00067 *** (Intercept) 0.784 0.044 17.87 0.00000 *** 4 + Sentences -0.033 0.019 -1.77 0.07868 motion -0.042 0.0182 -2.29 0.02370 * RDL2 0.002 0.001 3.51 0.00061 *** family 0.030 0.015 2.05 0.04282 * Stories that involve motion words (e.g., go, move, ran, inhibition 0.052 0.011 4.74 0.00001 *** arrive, come, enter, threw) are associated with more incorrect answers and fewer correct answers. These stories motion 0.025 0.009 2.77 0.00637 ** often incldued contexts where people were walking, biking, % Correct hot-air-balooning, driving, or actively constructing (Intercept) 0.334 0.17478 1.91 0.05778 something. In terms of the quantities used, there was often a rate of change (e.g., per hour, per minute, a day) that LDTTRc 0.428 0.169 2.53 0.01242 * involved this motion, and students had to identify the two WRDPOLc -0.041 0.01469 -2.78 0.00609 ** quantities that made up this rate of change. Using more inhibition -0.128 0.03909 -3.28 0.00132 ** measure (RDL2) was associated with greater hint-seeking when writing expressions. This measure is calculated through measures of word frequency (with words that occur Another factor that stands out in the regression results more frequently in the English language yielding higher is word polysemy (WRDPOLc) – or the number of different scores), sentence syntax similarity (with sentences that meanings that a word has (for example, in English, mine have similar grammatical structures yielding higher scores), can be something you own or an explosive device). The and word overlap (with words that share semantic meaning results show that stories that contain words with more yielding higher scores; [16]). Given that a higher second potential meanings are associated with more incorrect language readability score is typically associated with answers and fewer correct answers. Polysemous words greater ease in comprehending the text [17], it is suprising have been found to make mathematics word problems more that stories that score higher on this measure would be difficult to interpret accross other studies [8-9]. associated with students seeking more hints. The Results also showed that higher type-token ratios explanation of this finding may be similar to that for our (LDTTRc) are associated with more correct answers. As finding with type-token ratio; story problems that use type-token ratio increases, more unqiue words are being similar words and sentence structures often use a lot of used in the story problem, and fewer words are being reptition as a way to present complex ideas. Stories that are repeated. These results suggest that students have an easier simple and concise may be easier for students to solve. time writing the expression in a story that is relatively concise with little reptition of ideas. While it makes sense 6. DISCUSSION that this type of story may be more amenable to translation Results indicate that readability and topic measures have into mathematics notation, this result contrasts with important associations with students‘ performance when research in text comprehension in reading tasks [14] which solving mathematics word problems in an ITS. In generally finds that repitition and lower type-token ratios particular, it was more difficult for students to name the facilitate reading comprehension. However, the story independent and dependent quanitities in problems relating problems with high levels of word repetition frequently to motion (physics) and inhibition (saving and safety), discuss complex topics of which students may lack while adverbial cues facilitated this skill. When writing familiarity, including operating capital, business inventory, algebraic expressions, we again see that motion and and wholesale prices. In this way, a high type-token ratio inhibition topics are difficult, but also find other important may be indicative of a complex topic rather than increased readability measures that matter. Words with multiple readability in these story problems. meanings make story problems more difficult, which Students‘ tendency to seek hints when writing the corresponds to previous findings in both mathematics and algebraic expression is associated with a number of reading education. different readability factors. First, we see an effect for the However, mathematics stories that use concise length of the story text; students are more likely to seek language with little repitition, which in terms of their hints for one sentence story problems, compared to readability level makes them technically less readable, are problems that have two or more sentences. Having only one actually easier for students to solve. Thus measures of single sentence in a story problem might not be enough to readability that stem from research on reading ground or fully describe a linear rate of change as it arises comprehension may need to be considered differently when in a real-world situation, and these overly-sparse stories working with mathematics problems. Results also suggest might consequently inhibit performance. that while a story problem that includes only a single In addition to greater difficulty of inhibition words, sentence is concise, it might present difficulty for students stories with family words and motion words were by not providing necessary context and information for associated with greater hint-seeking. Only 13 of the them to feel they can respond without needing a hint. problems involved family words, and these were often Overall, our results suggest that mathematics story complex scenarios where multiple actors (e.g., a main problems that have story texts that are more accessible to character and his brother) were each contributing to the students have several characteristics: (1) they are concise algebraic rate of change in their own way (e.g., with little repetition, but not a single sentence only, (2) they saving/earning/splitting money together). Motion words use only a single actor performing actions, (3) they use often involved physics contexts (e.g., traveling in a car or simple words with clear meanings, (4) they avoid more plane) in which students had to track distance, rate, and abstract physics or financial contexts, instead focusing on time. This suggests that keeping track of multiple familiar contexts involving accumulation or loss of individuals engaging in mathematical actions and solving concrete physical objects, and (5) they make use of problems with physical distances and rates may be adverbial cues. Story problems with these characteristics significant difficulty factors when writing expressions. may allow students to more easily construct a situation Finally, the regression results showed that scoring model from a propositional textbase. They may promote higher on Coh-Metrix’s second language readability situation-model construction by both increasing students‘ ability to comprehend the semantics of the problem, and by situation-based reasoning with algebraic representation. increasing students‘ interest in working on the problem. Journal of Mathematical Behavior, 31(2), 174-195. [6] Hidi, S., & Renninger, K. 2006. The four-phase model of 7. CONCLUSION interest development. Educational Psychologist, 41(2), 111- Future adaptive ITSs will be designed to model student 127. characteristics at an extremely fine-grained level, as [7] Vilenius-­‐Tuohimaa, P. M., Aunola, K., Nurmi, J. E. 2008. technology for personalized learning continues to advance. The association between mathematical word problems and Here we argue that an important element of these future reading comprehension. Educational Psychology, 28(4), 409- adaptive systems will be a consideration of the non- 426. mathematical text-based characteristics of the problem [8] Shaftel, J., Belton-Kocher, E., Glasnapp, D., Poggio, J. 2006. tasks they present to students. Making inferences about The impact of language characteristics in mathematics test students‘ current level of mathematical knowledge or items on the performance of English language learners and motivation without considering these characteristics may students with disabilities. Educational Assessment, 11(2), 105-126. lead to misspecifications. [9] Wolf, M. K., Leon, S. 2009. An investigation of the language Readability and topic measures may be an important demands in content assessments for English language consideration for ITSs to model in a variety of domains, learners. Educational Assessment, 14(3-4), 139-159. including when considering tasks from history, social [10] Doddannara, L. S., Gowda, S. M., Baker, R. S., Gowda, S. studies, and science. Future research should focus on the M., De Carvalho, A. M. 2011. Exploring the relationships readability and topic measures that are most important for between design, students’ affective states, and disengaged students of different age groups in different subject behaviors within an ITS. In: Proceedings of the 16th domains, and narrow down which characteristics are most International Conference on Artificial Intelligence and critical to include in student and domain models as we Education, pp. 31-40. build future ITSs. In current work, we are analyzing the [11] Baker, R. S., de Carvalho, A. M. J. A., Raspat, J., Aleven, V., mathematics problems on the National Assessment of Corbett, A. T., Koedinger, K. R. 2009. Educational software Educational Progress (NAEP) and Trends in International features that encourage and discourage “gaming the system.” Mathematics and Science Study (TIMSS) to examine how In: Proceedings of the 14th International Conference on readability and topic measures impact the performance of Artificial Intelligence in Education, pp. 475-482. 4th and 8th graders in the United States, and how these [12] Walkington, C. 2013. Using learning technologies to factors interact with cognitive and non-cognitive student personalize instruction to student interests: The impact of background characteristics. relevant contexts on performance and learning outcomes. Journal of Educational Psychology, 105(4), 932-945. 8. REFERENCES [13] Bernacki, M. & Walkington, C. 2014. The Impact of a [1] Padayachee, I. 2002. Intelligent tutoring systems: Personalization Intervention for Mathematics on Learning Architecture and characteristics. University of Natal, Durban, and Non-Cognitive Factors. Submitted to the 2014 Information Systems & Technology, School of Accounting International Conference of Educational Data Mining, & Finance. London. [2] Ritter, S., Anderson, J. R., Koedinger, K. R., Corbett, A. [14] Graesser, A. C., McNamara, D. S., Louwerse, M. M., Cai, Z.: 2007. Cognitive Tutor: Applied research in mathematics Coh-Metrix: Analysis of text on cohesion and language. education. Psychonomic Bulletin & Review, 14(2), 249-255. Behavior Research Methods, Instruments, & Computers, [3] Nathan, M. J., Kintsch, W., Young, E.: A theory of algebra- 36(2), 193-202 (2004) word-problem comprehension and its implications for the [15] Pennebaker, J. W., Chung, C. K., Ireland, M., Gonzales, A., design of learning environments. Cognition and Instruction, Booth, R. J.: The development and psychometric properties 9(4), 329-389 (1992) of LIWC2007. Austin, TX, LIWC. Net. (2007) [4] Walkington, C., Petrosino, A., Sherman, M. 2013. [16] Crossley, S., Allen, D., McNamara, D. 2011. Text readability Supporting algebraic reasoning through personalized story and intuitive simplification: A comparison of readability scenarios: How situational understanding mediates formulas. Reading in a Foreign Language, 23(1), 84-101. performance and strategies. Mathematical Thinking and [17] Crossley, S. A., Greenfield, J., McNamara, D. S. 2008. Learning, 15(2), 89-120. Assessing text readability using cognitively based [5] Walkington, C., Sherman, M., & Petrosino, A. 2012. indices. TESOL Quarterly, 42(3), 475-493. ‘Playing the game’ of story problems: Coordinating