A theoretical link between reasoning and mathematical ability has been supported by some recent empirical evidence. We argue that some of this evidence is indirect and measure selection may have influenced this relationship. We report one study in which mathematical ability was measured using a fluency and a calculation measure (Wood-cock Johnson-III, 2001) and reasoning ability was measured using the extended cognitive reflection test (Toplack, West & Stanovich, 2014) and a belief bias conditional reasoning task. Results from 68 undergraduate students suggest that mathematical ability is predicted by performance on the cognitive reflection test but not conditional reasoning performance. We discuss the implications of these findings for research on the link between mathematical ability and reasoning skills.
Research has established that a variety of general cognitive
skills are necessary for mathematical success, such as
working memory, inhibitory control and shifting skills
Recent research on reasoning and mathematical ability
suggests a close relationship between these two skill sets.
However, careful examination of some of the evidence
suggests that the measures of reasoning ability are in fact
frequently tapping into domain specific mathematical
reasoning skills. For example,
An additional factor that should also be acknowledged is
the impact of task selection when measuring mathematical
ability. Numerous studies attempting to establish a link
between reasoning and mathematical ability have not used
gold-standard measures of mathematical ability, and instead
have used self-report questionnaires, brief experimental
maths tasks or school achievement measures
Handley, Capon, Beveridge, Dennis and Evans (2004) established a relationship between teacher-reported numeracy levels, a standardized mathematics measure and reasoning skills in a small sub-sample (N=32) of 10 - 11 year old children. They used relational and conditional reasoning problems in which they varied the believability of the conclusions. Several indexes relating to logic and believability of the problems were highly correlated with numeracy skills. However, again it is unclear whether reasoning performance is related to specific numeracy skills.
Vamvakoussi, van Dooren, and Verschaffel (2012) present evidence of intuitive reasoning in processing mathematical problems which are incongruent with natural number rules. The authors argue that this points to a distinction between system 1 and system 2 processing within mathematical reasoning. However, it is unclear from this research whether the skills demonstrated by participants in these studies are domain specific or whether they draw on domain general reasoning processes.
A recent study by
Some evidence suggests that deductive and mathematical
processing draw on distinct neural substrates which implies
that the two skills may be drawing on different cognitive
systems
One measure that has recently attracted a lot of attention,
both within the reasoning community but also as a measure
of reasoning abilities in relation to mathematical abilities is
the Cognitive reflection test – CRT
However, the CRT has been criticized, as all of the
problems also have a large mathematical component to
them. For example,
conditional reasoning task. Participants presented with a conditional premise, such as: task is the are typically If the weather gets warmer then more people will go to the beach. followed by a minor premise such as: The weather gets warmer.
Participants are then either asked to make an inference or to select an inference from a set of options such as:
More people go to the beach More people do not go to the beach
More people may or may not go to the beach Four different inferences can be made from a conditional by either affirming or denying either the first part (the antecedent) or the second part of the conditional (the consequent) in the minor premise that follows the conditional. Affirming the first part of the conditional, as in the example above, results in the modus ponens inference of ‘more people go to the beach’, which, according to propositional logic, is a valid inference. Similarly, denying the consequent (e.g. not more people go to the beach) results in the equally valid modus tollens inference (e.g. the weather does not get warmer). The other two inferences, denial of the antecedent (e.g., the weather does not get warmer) and affirmation of the consequent (e.g., more people go to the beach) are not considered valid inferences, hence, one should conclude for both inferences that the proposed conclusion may or may not occur.
The valid modus ponens inference is fairly easy, with
endorsement rates ranging between 89% and 100%. There is
considerable variability in endorsement rates of the valid
modus tollens and the invalid inferences (denial of the
antecedent and affirmation of the consequent), particularly
when the content of the materials is arbitrary
Manipulating the believability of the initial conditional
has an impact on the rate at which participants endorse all of
the four inferences
If fast food is taxed then childhood obesity will increase.
Fast food is taxed A participant has to deal with the conflict between what is logically correct (childhood obesity increases) and what is believable (childhood obesity does not increase). In this sense, this task is similar to the cognitive reflection task as a person must overcome their intuitive response and engage in analytical thinking in order to arrive at the correct response. The advantage of this task over the cognitive reflection task is that it does not require any mathematical processing, but like the cognitive reflection task it requires the suppression of an intuitively appealing response. We felt it was important to retain this aspect of the reasoning measure in order to assess whether it is this ability to detect a conflict between an intuitive and a correct response which is driving the relationship between reasoning abilities and mathematical abilities.
Being sensitive to conflict in stimuli could be potentially
useful for mathematical achievement.
The present study thus seeks to examine the relationship between mathematical and reasoning ability by employing gold-standard measures of mathematical ability and a reasoning task which does not rely on mathematical ability. However, we also include the CRT in order to examine its relationship with the mathematical measures and reasoning task we employ.
In order to improve validity, the study will utilize standardized measures of mathematical achievement, the Wood-cock Johnson-III Math fluency test and the Woodcock Johnson-III calculation test (2001). The Math Fluency measure requires participants to complete as many simple arithmetic problems as possible in a set period of time, in contrast the un-timed calculation test requires more complex mathematical reasoning skills. We would anticipate a stronger association between performance on the general reasoning tasks and the calculation test, than with the fluency test due to the overlap in reasoning content between the two measures.
We therefore predict that mathematical ability as measured by the calculation task in particular will be related to performance on the cognitive reflection task and conditional reasoning task and that after taking mathematical fluency into consideration, the reasoning tasks will contribute to the prediction of mathematical ability.
Participants Seventy-four University of Leicester Psychology students participated in this study in return for course credit. However, six participants were removed prior to analysis as they had failed to follow the instructions for the Woodcock Johnson Mathematical Fluency task (they completed the calculations column by column rather than row by row, thus affecting the difficulty of the problems solved). The remaining 68 students had a mean age of 19.57 years (range 18-46 years) and there were 13 male and 55 female participants. All participants had a minimum grade C or equivalent GCSE in mathematics.
Materials and Design The study had a correlational design and all participants completed the same four measures. There were two measures of mathematical ability the Woodcock Johnson-III Math fluency test and the Wood-cock Johnson-III calculation test (2001). The fluency test is a measure of people’s ability to solve simple arithmetic calculations (e.g., 4 - 2) within a 3 minute time period. It includes 160 simple addition, subtraction and multiplication problems. The calculation test is a 45-item test used to assess more complex mathematical ability. The questions start with basic maths calculations and get progressively harder, reaching A-level (final year of secondary school) standard. Participants in this study started with item 14 on the calculation task due to the age of the sample.
Reasoning abilities were assessed using the extended
Cognitive Reflection Test
The second reasoning measure was a belief bias
conditional reasoning task, for which the materials were
provided by
Each problem was presented on a different page of the 32 page booklet. Half of the conditionals had believable content and the other half had unbelievable content, that is participants were presented with conditionals with 32 distinct contents. Within each set of believable and unbelievable conditionals participants were presented with four modus ponens, four modus tollens, four denial of the antecedent and four affirmation of the consequent inferences.1 The 32 conditional inferences were presented in different random orders to each participant.
Procedure Participants were tested in groups of up to 12 people. Participants were advised to work in exam conditions, meaning in silence, independently, and without 1 Due to an administrative error four participants did not see the full set of 32 conditionals, two participants only received 3 instead of 4 inferences of one type and two participants were not given one set of inferences, e.g. they did not receive the four believable affirmation of the consequent inferences. the use of calculators. The four tasks were completed in four different orders. Participants either started with the fluency measure or with the CRT and after completing the first two tasks either completed the calculation task or the conditional reasoning task. The fluency task was timed and participants had to complete as many items as possible in three minutes. All other tasks were completed at the participants’ own pace. When completing the calculation task, as per standardized administration rules, the experimenters advised the participants to move on to the next task (or finish if this was their last task) when the experimenters noted that they had 6 consecutive incorrect answers. The test battery took approximately 40 – 45 minutes in total to complete, with a small minority of participants completing it in 30 minutes or 60 minutes.
This booklet contains 32 statements. The statements and associated tasks are about how people think in their daily lives and are not tests of intelligence. Please read each statement carefully and decide which conclusion follows. Please answer the questions in the order in which they are presented and do not try to change your answers once you have written them.
This booklet contains six items that vary in difficulty. Answer as many as you can.
For the two maths tasks, participants received the standardised verbal instructions.
Table 1 displays the mean scores on the four measures the participants completed. Participants scored a mean of 103.53 out of a possible 160 on the fluency measure and a mean of 15.26 out of a possible 32 on the calculation task. Performance on the cognitive reflection task was fairly poor with a mean number of correct responses of 1.18 out of 6 and as can be seen from Table 1 the majority of incorrect responses were the intuitive ones (mean 3.35).
For the conditional reasoning task we calculated a score of the number of logically consistent inferences (i.e., we counted the number of endorsements of the modus pones and modus tollens inferences, and the number of times participants did not endorse the affirmation of the consequent and denial of the antecedent inferences) and converted these scores into proportions due to the fact that a small number of participants had not received the same number of conditional inferences (see footnote 1). This score gave us a logic index which we could use for our analyses. However, we also present the responses to the conditional reasoning task in the customary way of endorsement rates for each inference in Figure 1. A 2x4 ANOVA on the endorsement rates confirmed that the believability manipulation of the conditional reasoning task had been successful. There was a main effect of believability; F(1,65) = 26, p < .001, n2 = .286 indicating that the participants endorsed the believable inferences more than the unbelievable inferences. There was also a main effect of inference, F(3,195) = 7.35, p < .001, n2 =.102, with participants endorsing the modus ponens inference more than any of the other inferences.
As can be seen from Table 2, the fluency measure only correlated with the calculation measure, r = .49, N = 68, p < .001. The calculation measure was correlated most strongly with the number of correct responses on the CRT, r = .36, N = 68, p =.003. However, the proportion correct responses on the conditional reasoning task was also moderately correlated with the calculation task, r = .24, N = 68, p = .047. But an examination of the correlations between the calculation task and the different conditional inferences revealed no significant correlations; rs< .12, ps > .16. The CRT and the conditionals task were also moderately correlated, r = .271, N = 68, p =.026. Interestingly, this correlation between CRT and the conditionals task is driven by a correlation between correct inferences on the believable conditionals, r = .257, N = 68, p = .034 and not by a correlation between unbelievable inferences and CRT, r = .154, p = .21. Examination of the correlations between the CRT and the different conditional inferences suggested that there were no (for modus ponens inferences) or negative (for modus tollens inferences) correlations between CRT performance and the valid inferences and there were positive correlations between CRT and the invalid inferences (affirmation of the consequent and denial of the antecedent). Hence we examined whether participants who made the correct inference for the invalid inferences (i.e. chose the option that the conclusion may or may not be drawn) performed better on the CRT. Indeed there was a positive correlation between the CRT and the correct inferences for the invalid inferences, r = .377, N= 68 p =.002.
A hierarchical regression with fluency entered at step 1 and CRT correct and proportion correct on the conditionals task to predict calculation scores indicates that fluency accounted for 24% of the variance in calculation scores (F(471.332, 22.938) = 20.55, p < .001) and that the addition of the two reasoning measures added .089 to R2, thus accounting for 33% of the variance in calculation scores (the R2 change was significant, F = 4.209, p = .019). Inspection of the beta weights for the final model revealed that fluency was the best predictor; beta = .428 (t=4.085, p < .001) and CRT correct also added significantly to the prediction; beta = .246 (t =2.28, p = .026), but performance on the conditional reasoning task was not a significant predictor.
The key finding from this study is that while conditional reasoning performance is correlated with mathematical ability as measured by the Wood-cock Johnson-III calculation test (2001), it does not predict performance on this task when mathematical fluency is taken into account. The cognitive reflection task however does predict performance on the calculation task. Hence, this study provides further support for the idea that the relationship between the CRT and mathematical achievement is driven by the mathematical component of the task.
We acknowledge a potential limitation of our study is that that mathematical education levels were not recorded for our participants, although we do know that all our participants had at least a grade C or equivalent in Mathematics at GCSE level. Furthermore, the measures of mathematical ability included in this study are sensitive standardized measures and so we are confident that these measures give us a fair reflection of the participants’ mathematical abilities.
In terms of the relationship between mathematical ability and reasoning ability, we argue that future studies examining this relationship must take into consideration whether they are investigating a relationship between domain specific (mathematical) reasoning skills or more domain general reasoning skills. We acknowledge that our domain general reasoning measure relied heavily on verbal abilities and so we propose that future studies should also include a non-verbal reasoning measure in order to assess whether abstract reasoning skills are linked to mathematical calculation skills. It is thus still unclear whether reasoning per se is essential for mathematical achievement or whether these are independent cognitive skills as suggested by Kroger et al (2008) who reported that different brain areas are linked to mathematical and reasoning processes. An alternative hypothesis worth pursing is whether mathematical and reasoning abilities are linked due to their relationship with more domain-general characteristics of intelligence.
Another interesting finding from this study is the fact the CRT was only correlated with good performance on the believable conditionals and not with performance on the unbelievable conditionals. Arguably, good performance on the CRT requires a person to recognize a conflict when considering the intuitive answer which results in a more considered response. Similarly, good performance on the unbelievable conditionals requires a person to recognize the conflict between believability and logic before they can recognize the correct inference. The fact that this aspect of the conditionals task was not related to CRT performance suggests that further investigation into what the CRT is measuring is also warranted.
We thank Matt Roser for supplying the materials for the conditional reasoning task and Priscilla Baafi, Sach Dhesi and Mesel Teklebrhan for their assistance in data collection. (1) If John can drink one barrel of water in 6 days, and Mary can drink one barrel of water in 12 days, how long would it take them to drink one barrel of water together? _____ days [correct answer = 4 days; intuitive answer = 9] (2) Jerry received both the 15th highest and the 15th lowest mark in the class. How many students are in the class? ______ students [correct answer = 29 students; intuitive answer = 30] (3) A man buys a pig for $60, sells it for $70, buys it back for $80, and sells it finally for $90. How much has he made? _____ dollars [correct answer = $20; intuitive answer = $10]