Two-Valued Logic is Not Sufficient to Model Human Reasoning, but Three-Valued Logic is: A Formal Analysis

Abstract

There is an ongoing debate in the psychology of reasoning whether and how logic can be used to describe the human inference process. Many psychological findings indicate that humans deviate from classical logic inferences. Some researchers have proposed to use ternary logics instead to model human reasoning processes. In this article we re-analyze the famous Wason Selection Task that has been researched in more than 100 publications and can be regarded as one of the most important reasoning experiment in the psychology of reasoning. It investigates how participants check if a given conditional statement holds. Most cognitive modeling approaches have focused on explaining the general response pattern. Instead, we focus on the pattern generated by each participant. In particular, we conduct a meta-analysis to identify these patterns. Thereafter, we analyze these patterns. If there is a two-valued model of human reasoning processes, then there must be two-valued truth-tables that can generate the patterns. Finally, we show by a search through the space of all two-valued truth tables that there are patterns that cannot be explained by two-valued logics. However, these patterns can be explained, when extending the representation to three-valued logics.

-

Conditional reasoning, i.e. reasoning about “if”-statements, is common in everyday life. Statements with “if” can express rules (deontic reasoning), assumptions and hypotheses (hypothetical reasoning), and reasoning about impossible scenarios (counterfactual reasoning) among many others. Experimental findings have widely demonstrated that many humans deviate from understanding “if” as the material implication taught in classical logic [ 16 ]. One of the first approaches to analyze the human inference processes has been proposed by Wason in the so-called Wason Selection Task [ 21 ]. Consider the four cards shown in Figure 1 and consider the following task:

On each card there is a number on one side and a letter on the other side.

Because the cards are laying on a table you see only one side. As you can see,

two of the cards show letters “D” and “K” and two other cards show numbers “3” and “7”. The participant’s task is to select cards that need to be turned over in order to test the truth of the statement “If the letter side shows a “D” then the other side shows a “3”. How many cards at most and which have to be turned in order to show that the rule does or does not hold?

Answers from the participants about the number of cards to be turned range from 0 to 4. A meta-study by [ 16 ] analyzed how many of the cards are turned (see Table 1). – Most participants (89%) would turn the card “D” because “given the rule p ! q and observing p on the one side, q must be on the other side for the rule to hold”. This is the apparently well-known modus ponens (MP) and is classical logically correct. – 62% of the participants want to flip the card “3” because “given p ! q and observing q, p must be on the other side”. This is known as affirmation of the consequence (AC) and is classical logically incorrect. – 25% of the participants want to flip the card “7” because “given p ! q and observing q, p must be on the other side”. This is known as modus tollens (MT) and is classical logically correct. – 15% of the participants want to flip the card “K” because “given p ! q and observing p, q must be on the other side”. This is known as denial of the antecedent (DA) and is classical logically incorrect.

Possible explanations for the behavior of the participants vary from heuristics and information gain [ 17 ] to a bi-conditional interpretation of the conditional statement [ 15 ] of the logically na¨ıve reasoners.3 The classical inference rules are summarized in Table 2.

One explanation for the behavioral findings was that the domain is too abstract and the possible lack of background knowledge was why participants made this mistake. For this reason Johnson-Laird and colleagues [ 11 ] examined the influence of background knowledge and were able to show that a similar problem was solved correctly. Cosmides and Tooby [ 3 ] showed in an experiment that in a “social” or content formulation of the Wason Selection Task more than 70% of the participants gave the correct answer (turning the MP and MT-cards and none of the other cards). An explanation is that humans are better in detecting deviations from social rules. Consider the following problem: You are a police officer and have to check if guests in a restaurant adhere to the following rule “If someone drinks alcohol, then this person must be over 21.” There are 4 guests sitting in the restaurant at this time (that are represented on cards as above). On the first card there is a person drinking beer, on the second one there is a person drinking Coke, the third card represents a person with 22 years and the fourth card a person 17 years old. Which card(s) only do you need to turn to check if the rule holds?

It is easy to see that this formulation is isomorphic to the previous abstract version as follows: drinks alcohol corresponds to the card “D”, 22 years to “3”, Coke to “K”, and age 17 to “7”. The experimental findings of Cosmides and Tooby [ 3 ] indicate that despite an isomorphic formulation of the Wason Selection Task, humans perform in the social/content based case considerably better than in the abstract case. This shows that humans are not simply applying rules regardless of the domain. Hence, human reasoning is context dependent and a semantic approach seems to be a more appropriate cognitive modeling option. 3 This term means that participants in such experiments do not have any training in logic.

Modeling approaches using a ternary logic like the Łukasiewicz logic [ 14 ] in [ 5 ] can explain the differences between the abstract and social case version of the task, where the conditional is understood as a biconditional and in addition for the abstract case abduction is applied to explain the cards. Hence, ternary logic is sufficient. But can we explain the results of the Wason Selection Task by a two-valued logic as well?

Any cognitive modeling approaches so far which we are aware of (see e.g., [ 16 ]) has focused on the aggregated data for each card, i.e., the number of participants that could pick a card (see the Table 1 above). Such an aggregated analysis may lead to a wrong interpretation and erroneous modeling, it disguises important factors in the data. Hence, in the following we will re-analyze the data in the line of the following questions: 1. Data/Empirical Analysis: Which response patterns are chosen by each participant? Are there empirical differences between the abstract and social version of the Wason Selection Task? 2. Modeling: Can these patterns be modeled by two-valued logics, rule-based, or probabilistic approach? Can three-valued logics model these patterns? 3. Processing/System-based analysis: In which order are the answers given and can they be explained? 2

Empirical Analysis: The Response Patterns

To gain a broader data base on the existing experimental literature we searched pubmed4 and google scholar5 with the keywords: ”Wason Selection Task” and ”Experiment”. We identified 43 articles that reported at least six of the sixteen possible answer patterns (see Table 3). We classified the answer patterns according to “social” based experiments (like the reported social version above) and an abstract version (cp. the abstract version above in Fig. 1). Additionally, we ranked the given answer patterns according to their frequency (see Table 3). Analyzing the table we find the following: – In the social and in the abstract version, only few participants chose the pattern All.

This pattern is chosen by less than 5% of the cases in the abstract case and less than 1% in the social case. – The classical logical correct response MP+MT is highest ranked in the social version of the Wason Selection Task, although not even half of the participants have chosen this pattern (about 45%). – Patterns in the first three ranks are the same for both tasks, only their order is different. – The matching hypothesis [ 6 ], i.e., the pattern MP+AC appears similarly often in the abstract and the social version, and only in about 22-23% of the cases. – The pattern MP + MT + AC was used significantly often in several studies (e.g., cp Table 4). This pattern has been chosen by as many as 25% [ 22 ], 19% [ 1 ], 12% [ 7 ] of the participants in several experiments. We will see later that this answer pattern is not replicable by two-valued logics. 4 http://www.pubmed.de 5 http://www.scholar.google.de MP MP+AC MP+MT AC All MT MT+DA AC+DA DA None MP+DA MP+MT+AC MT+AC MP+AC+DA MT+AC+DA MP+MT+DA

Ss 2047 1808 1022 455 448 383 332 267 266 195 167 163 99 59 50 35 % 26 23 13 6 5 5.9 4.4 3.5 3.5 2.6 2.2 2.2 1.3 3

Modeling Approaches

In the previous section we reported and ranked answer patterns in the literature (for a complete overview over each single study please refer to Table 4). In this section we investigate if these patterns can be explained by different modeling approaches, e.g., by two-valued logics. 3.1

Two-valued Logics Given that L(R) is a set of propositional formulas on a set of propositional variables R, a two-valued valuation of a formula is a function v : L(R) ! {0, 1}, where 0 means false and 1 means true. The valuation function for the material implication is as follows: v(p ! q) = ⇢ 0 if v(p) = 1, v(q) = 0

1 otherwise As we are in particular interested in reasoning with conditionals, we will only consider valuations of the form p !i q, where the index i denotes the pattern that follows from the valuation of !i and possibly differs from the valuation of the material implication. The material implication as defined above corresponds to the chosen truth table for !MP+MT in Table 5, from which the pattern MP+MT follows.

Note that we need to distinguish between skeptical and credulous reasoning here. A pattern follows skeptically, if it holds under all possible valuations of a given implication. In contrast, a pattern follows credulously if it holds under at least one valuation of a given implication. For instance, consider again the Wason Selection Task: Given that the implication D ! 3 needs to be verified and considering card D, the majority of participants knows that the other side of the card could be either 3 or not 3 and turn the card. It is not enough to assume that there exists one valuation for which the implication is valid (in this case where 3 is on the other side), but all possible valuations for the given propositional variable need to be taken into account.

From this point of view, it is quite natural to assume that participants reason skeptically, i.e. they require that a given pattern follows from an implication only if that pattern follows for any valuation with respect to this implication.

Additionally, we require that patterns indicate different inference processes. For instance, if participants chose a certain pattern, e.g. MP+AC, they did reason differently than the participants who chose only MP or only AC. Consequently, the representation for each of the sixteen patterns needs to be unique, i.e., excluding all other patterns. For instance a model representation that corresponds to MP+AC, cannot correspond at the same time to exclusively MP or AC, as in the first case AC and in the second case MP holds. Note that accordingly, both, the patterns MP and MT separately should follow from a different valuation of the binary operator.

In total, there can be 24 valuations for a binary operator. In order to define a unique truth table for each pattern, we conducted a complete search of all possible valuations. We did so by systematically assigning the value 0 or 1 as outcome to the truth table. Depending on which of the conditions is satisfied, the algorithm assigns each truth table to a pattern.

According to Table 2, the valuation functions for MP, MT, AC and DA, respectively, are defined as follows: v(p !MP q) = v(p !MT q) = v(p !AC q) = v(p !DA q) = 8 < 8 < 8 < 8 < 0 if v(p) = 1 and v(q) = 0 1 if v(p) = v(q) = 1 : 0/1 otherwise 0 if v(p) = 1 and v(q) = 0 1 if v(p) = v(q) = 0 : 0/1 otherwise 0 if v(q) = 1 and v(p) = 0 1 if v(p) = v(q) = 1 : 0/1 otherwise 0 if v(q) = 0 and v(p) = 1 1 if v(p) = v(q) = 0 : 0/1 otherwise Lemma 1. The inference rules defined in Table 5 have the following correspondences: 1. !MT+AC holds if and only if !MP+DA holds. 2. !MT+AC implies !DA. 3. !MP+DA implies !AC. 4. !MT+AC+DA holds if and only if !MP+DA+AC holds. 5. !MP+MT+AC holds if and only if !MP+MT+DA holds.

Proof. (1- 3.) and (5.) follow immediately from the truth tables in Table 5. (4.) follows from (1- 3.).

As already mentioned above we require that each pattern follows skeptically from a uniquely determined truth table, i.e. the particular patterns follows from all valuations of a given implication and it needs to be the only pattern that follows from this implication.

Accordingly, Lemma 1.(2-3.) gives us the explanation why we had to choose for the highlighted values in the truth tables of !MP, !MT, !AC and !DA in Table 5: Consider, for instance !AC, where the values in the first and the third row don’t matter according to the valuation function. If instead of 0 and 1, we would have chosen 1 and 0 for the second and fourth row, respectively, the truth table would have been identical to the truth table of !All. The chosen values in the truth tables for !MP, !MT, and !DA can be explained analogously. Corollary 1 follows immediately from Lemma 1: Corollary 1. Under two-valued logics, there are no unique inference rules which represent the patterns MT+AC, MP+DA, MT+AC+DA, MP+DA+AC, MP+MT+AC and MP+MT+DA.

Hence there are six patterns out of the sixteen that cannot be represented by any possible interpretation of the conditional in two-valued logic. Table 4, however, shows that more than just a few participants in several experiments decided to turn cards, that cannot be explained by guesses (or pure random behavior). Since so many studies show that such patterns are significant, it seems that the model-based approach based on two-valued logics is not sufficiently explaining human reasoning patterns.

Conditionals as license for inferences. Another option could be that conditionals are “licenses for inferences” [ 20 ], i.e., a conditional needs to be understood as p ^ ¬ab ! q i.e., humans do understand a conditional not as p ! q but that if p and nothing abnormal is known then q. This is a typical interpretation used in non-monotonic approaches [ 20 ].

Searching through the space of all possible two-valued valuations and connectives between p and ¬ab (like ^ , _ , and so forth) shows that this leads only to the same ten patterns as above and not to all sixteen patterns. So an additional degree of freedom by an “abnormality predicate” does not increase the number of possible answer patterns. Additional inference rules. The psychological literature reports four inference rules: MP, MT, DA and AC. For two variables, sixteen Boolean functions are possible. It seems unlikely, but there could be other interpretations of the underlying operator. For instance, the implication could be interpreted as well as an exclusive or (xor) (please refer to Table 6). In the following we restrict ourselves to those rules that infer something about the missing fact (or its negation), e.g., for a rule like p ! q and a given p then there can be two rules, the classical MP-rule and its negated version MP’.

This leads to sixteen combinations of 8 rules. By considering the relations MP’, MT’, AC’ and DA’, we notice that there are no truth tables which would satisfy a pattern which includes some prime-relations and some non-prime relations (e.g., MP, MT’, AC, DA), because there is a clash due to mapping in the prime condition of the truth table the case p and q to 0, as depicted in Table 7. We only get two patterns by the prime versions, namely: All and None. Rule-based Approach Could the missing patterns be reproduced by a pure rule-based approach? Some researchers in Cognitive Science and Psychology like Rips [ 19 ] and O’Brian [ 2 ] proposed that humans apply such inference rules. They argued that people use more often the MP instead of the MT because there is a proof necessary to show that MT holds. Such a proof makes the MT reasoning scheme more difficult than a simple application of MP. Errors in the reasoning process can be traced back to a misunderstanding of the conditional or the application of a wrong rule. However, even if we assume that people might apply a rule-based approach (instead of a model-based approach) the Corollary 1 above already demonstrates the limitations of any such rulebased approach and that if people apply such a rule-based approach there is no way that they can generate one of the six missing patterns.

Probabilistic Approach Following an approach by Oaksford and Chater [ 16 ] people might assign probabilities, i.e., instead of interpreting p ! q in the classical sense, reasoners might understand this as the conditional probability q given p, i.e., P (q | p).

We calculated the possible probabilistic results by iterating the values 0.1, 0.3, 0.5, 0.7 and 0.9 for a and b for the proposed Independence Model (in Table 8) of [ 16 ]. The model accepts a certain conditional probability only if it is above a given threshold. We iterate the value of the threshold from 0.1 to 0.9, and we noticed that it needs to be around 0.1-0.2, otherwise we do not get most of the patterns. In this approach, apart from the 6 patterns missing in binary logic, we also do not get the pattern All, a pattern that is on rank 4 in a content-based version. Hence, this approach cannot reproduce the patterns. Another finding is that the analysis reveals that distribution does not fit the distribution of participant’s answers. As none of the previous approaches can explain the missing six possible patterns in human reasoning, we will investigate if there are ternary logics, that can generate these patterns. To this end we assign the values 0, u, and 1 to the variables p and q, where u means unknown. Such an extension results in 29 different valuations. Since the Wason Selection Task restricts the option to turn or not turn (see Fig. 1), we map the valuations in the three-valued case to the set {0,1} with 1 “turn” and 0 “not turn”. This extension to three-valued logics shows that it is possible to find a uniquely determined truth table from which the patterns MP+MT+AC and MP+MT+DA follow skeptically (cp. Table 10). The highlighted values show where we have more freedom of the interpretations than under two valued logics: The values in light gray show that by mapping {0, u} to 1, !MP+MT+AC does not imply !DA. Similarly, the highlighted values in dark gray show that by mapping {u, 1} to 1, !MP+MT+DA does not imply !AC.

All six missing patterns can be uniquely represented under some three-valued logic valuations and therefore Lemma 1 can not be extended for three-valued logics. A further analysis shows, as expected, that there are at least two possible truth tables that satisfy each pattern in the three-value case (see Table 9 for an overview). The different answer patterns hinge mostly on an interpretation of the u ! 1 or 0 ! u (cp. Table 10). As

MP MP+MT MP+AC MP+MT+AC MP+AC+DA MP+MT+DA MT+AC+DA All Two-valued 1 Three-valued 34 1 10 1 10 we have now several truth tables that can reproduce the human answer patterns, the question is, how do the truth tables differ? We analyzed the intersection of the truth tables and see again, as expected, that it depends mostly on the interpretation of the third-value u in the conditional.

There is an ongoing discussion on possible interpretations of the implication under three-valued logics (e.g. Łukasiewicz logic [ 14 ], Kripke-Kleene logic [ 13 ]). So far there is no psychological research on how participants may in general interpret such assertions, leaving this point open.

Here we have shown that the answer patterns humans produce can be captured by three-valued logics. True cognitive modeling aims not only in reproducing the answer patterns, but additionally the processes that lead to the results. The data from the literature analysis above does not, however, indicate how humans process the presented information, i.e., if they read the cards from left to right, which card they select first, and how fast they answer to each problem. 4

Conclusion

The Wason-Selection-Task is ‘the’ fundamental research and modeling problem in the psychology of conditional reasoning. We have shown that: (i) instead of analyzing aggregated values single response patterns provide the “real” inference process, (ii) human reasoners generate patterns that cannot be reproduced by classical logical approaches, (iii) some answer patterns have implications for other answer patterns, and (iv) threevalued logics can explain the answer results.

The different answer patterns generated by human reasoners demonstrate a great variety in the inference process. All of the patterns can be explained by different threevalued valuations of the implication operator. Why did so many participants falsely chose patterns with the “3” instead of the logical correct “7”? Is it only a wrong matching of the conditional, i.e., do the participants simply chose to turn the wrong card because they misunderstood the conditional? If so our method provides possible interpretations of the conditional. But our results go one step further: by using a third-value everything seems to hinge on how humans may interpret the truth-value u in a conditional. This is for some answer patterns the only way to differentiate between them (e.g., Table 10). Future work will investigate the interpretations of human reasoners on evaluating problems that have been assigned the truth-value unknown.

The long ongoing debate about using two-valued logics (or restrictions of it) as a modeling framework can be rejected. From this perspective three-valued logics seem to be a more appropriate approach modeling human reasoning.

References

1. Beattie , J. , Baron , J.: Confirmation and matching biases in hypothesis testing . The Quarterly Journal of Experimental Psychology 40 ( 2 ), 269 - 297 ( 1988 )

2. Braine , M.D.S. , O 'Brien , D.P. : Mental logic . Erlbaum, Mahwah, NJ ( 1998 )

3. Cosmides , L. , Tooby , J.: Cognitive adaptations for social exchange . In: Barkow, J.H. , Cosmides , L. , Tooby , J . (eds.) The Adapted Mind: Evolutionary Psychology and the Generation of Culture , pp. 163 - 228 ( 1993 )

4. Cox , J.R. , Griggs , R.A. : The effects of experience on performance in wason's selection task . Memory & Cognition 10 ( 5 ), 496 - 502 , http://dx.doi.org/10.3758/ BF03197653

5. Dietz , E.A. , Ho¨lldobler, S. , Ragni , M. : A Computational Logic Approach to the Abstract and the Social Case of the Selection Task . In: Morgenstern, L. , Davis , E. , Williams , M.A . (eds.) 11th International Symposium on Logical Formalizations of Commonsense Resaoning ( 2013 )

6. Evans , J.S.B. , Stanovich , K.E. : Dual-process theories of higher cognition advancing the debate . Perspectives on psychological science 8 ( 3 ), 223 - 241 ( 2013 )

7. Feeney , A. , Handley , S.J.: The suppression of q card selections: Evidence for deductive inference in wason's selection task . The Quarterly Journal of Experimental Psychology: Section A 53 ( 4 ), 1224 - 1242 ( 2000 )

8. Fiddick , L. , Cosmides , L. , Tooby , J.: No interpretation without representation: the role of domain-specific representations and inferences in the wason selection task . Cognition 77 ( 1 ), 1 - 79

9. Griggs , R.A. : Memory cueing and instructional effects on wason's selection task . Current Psychology 3 ( 4 ), 3 - 10 , http://dx.doi.org/10.1007/BF02686552

10. Johnson-Laird , P.N. , Wason , P.C. : A theoretical analysis of insight into a reasoning task . Cognitive Psychology 1 ( 2 ), 134 - 148 ( 1970 )

11. Johnson-Laird , P. , Legrenzi , P. , Legrenzi , M. : Reasoning and a sense of reality . British Journal of Psychology 63 , 395 - 400 ( 1972 ), http://books.google.de/books?id= FFGjYgEACAAJ

12. Kirby , K.N. : Probabilities and utilities of fictional outcomes in wason's four-card selection task . Cognition 51 ( 1 ), 1 - 28 ( 1994 )

13. Kleene , S.C. : Introduction to Metamathematics. North-Holland, Amsterdam ( 1952 )

14. Łukasiewicz , J.: O logice tro´jwartos´ciowej . Ruch Filozoficzny 5 , 169 - 171 ( 1920 ), english translation: On three-valued logic . In: Łukasiewicz J. and Borkowski L. (ed.). ( 1990 ). Selected Works , Amsterdam: North Holland, pp. 87 - 88 .

15. Manktelow , K. : Reasoning and thinking . Psychology Press ( 1999 )

16. Oaksford , M. , Chater , N.: A Rational Analysis of the Selection Task as Optimal Data Selection . Psychological Review 101 ( 4 ), 608 - 631 ( 1994 )

17. Oaksford , M. , Chater , N.: Bayesian rationality: The probabilistic approach to human reasoning . Oxford University Press, Oxford ( 2007 )

18. Pollard , P. : The effect of thematic content on the 'wason selection task' . Current Psychological Research 1 ( 1 ), 21 - 29 , http://dx.doi.org/10.1007/BF02684422

19. Rips , L.J.: The psychology of proof: Deductive reasoning in human thinking . The MIT Press, Cambridge, MA ( 1994 )

20. Stenning , K. , Van Lambalgen , M. : Human reasoning and cognitive science . MIT Press ( 2008 )

21. Wason , P. : Reasoning about a rule . Quarterly Journal of Experimental Psychology 20 ( 3 ), 273 - 281 ( 1968 )

22. Yachanin , S.A. : Facilitation in wason's selection task: Content and instructions . Current Psychology 5 ( 1 ), 20 - 29 , http://dx.doi.org/10.1007/BF02686593

23. Yachanin , S.A. , Tweney , R.D.: The effect of thematic content on cognitive strategies in the four-card selection task . Bulletin of the Psychonomic Society 19 ( 2 ), 87 - 90 ( 2013 ), http: //dx.doi.org/10.3758/BF03330048