In this paper I present the double disjunction task as introduced by Johnson-Laird. This experiment is meant to show how mental model theory explains the discrepancy between logical competence and logical performance of individuals in deductive reasoning. I review the results of the task and identify three problems in the way the task is designed, that all fall under a lack of coordination between the subject and the experimenter, and an insu cient representation of the semantic/pragmatic interface. I then propose a reformulation of the task, that makes explicit the underlying semantic reasoning and emphasizes the di erence of interpretation of the ddt between the experimenter and the subjects.
1.1 which are called negative, because of the incompatibility between (P1) and (P02), that gives rise to a contradiction. With subscript for inclusive or exclusive, and superscripts for positive and negative, one obtains four variants, ddti+, ddte+, ddti and ddte .
The task was aimed at enforcing some predictions by the mental model theory (see below) according to which the more models an agent has to represent the less she is able to make correct inferences from the premises. Inclusive disjunctions should be therefore more di cult to cope with than exclusive ones, as well as disjunctions involving a contradiction in the premises. For matters of clarity, here is the truth table for the inclusive (noted _) and exclusive (noted _) disjunctions: (P1) (P2) (P02)
P Q P _ Q P_Q T T T F T F T T F T T T
F F F F The mental model theory (mmt) has been developed in reaction against the idea of a ‘mental logic’ according to which individuals have logical rules in their mind and draw logical inferences in line with these rules. According to the mmt, people make logical inferences by following neither the rules of natural deduction nor those of truth tables, but by constructing mental models of the premises from which they derive a model of the conclusion. Deductive reasoning does not depend on syntactic rules but on a three-steps semantic process. First, semantic information is mentally construed as a representation of situations given by the premises. Second, an attempt to draw a conclusion from the mental models of the premises is made. Third, the validity of the conclusion is checked by ensuring that no model of the premises renders it false. For instance, a disjunction such as:
|
:|
When the following premise is added:
the model representing the club is discarded and the informational content of the new premise is added to the remaining model:
The conclusion ‘There is a spade’ can be drawn, since there is no model of the premises that makes this conclusion false.
Relying on the assumption that individuals are fundamentally rational but make errors in practice, Johnson-Laird identifies three principles for deductions made by ‘logically untutored individuals’: [They] eschew conclusions that contain less semantic information than premises. [...] Similarly, they seek conclusions that are more parsimonious than the premises. [...] They try to draw conclusions that make explicit some information only implicit in the premises. In short, to deduce is to maintain semantic information, to simplify, and to reach a new conclusion. (Johnson-Laird et al. 1992, p. 419)
These constraints on deductive reasoning are close to Grice’s maxims of
conversation
Grice stresses that conversations where the maxims apply are a special case of cooperative situations, and that these maxims are instantiations of a more general Cooperation Principle (cp). An experiment is, after all, a cooperative situation, and therefore the e ect of cp should be evaluated at the beginning of the task, because deductive reasoning of lay people is constrained by semantic principles as well as pragmatic principles.
The remainder of this paper is organized as follows. The next section firstly discusses the results of the ddt, secondly identifies the main issues at stake in the interpretation of the results and the way the task is formulated. Section 3 shows that most of these problems arise because of a lack of coordination between the subject and the experimenter in the interpretation of the task. Section 4 suggests some possible reformulations that could improve the base rate of originally expected answer, by eliciting both the adequate modal representation, and the selection of relevant information, through pragmatic principles. 2 2.1
Alice is in India and Cheryl is in Afghanistan, or Barbara is in Pakistan, or both (Ci+)
The valid conclusion for the negative disjunctions were predicted to be more difficult to make because they suppose the ability to first detect the contradiction in the premises. The percentages of valid conclusions made by the subjects for the four kind of disjunctions are displayed in Fig. 2.
Exclusive Inclusive
A rmative Negative 21% 8% 6% 2%
As predicted by the mmt, the percentage of success decreases when the number of models to build increases, coupled with a floor e ect: when a deduction needs three or more models, it is almost impossible for the subjects to make a correct inference, or even to decide whether a conclusion follows from the premises.
But while Johnson-Laird is interested in the link between one particular case of
error with one particular subset of the models of the premises
The two categories collapsed into the disjunctive responses were what
While mmt assumes that “naive individuals have a modicum of deductive
competence”
Changing the perspective preserves some insights from mmt (i.e. why ‘trivial’
conclusions are not even considered), but sheds light on other aspects left unexplained by
the theory. For instance, answers in disjunctive form are more common than in
conditional form
One may then identify three problematic categories in the formulation of the ddt. The first category of problems concerns the logical form of the premises. First, the conjunction of the two premises is implicit, but this conjunction must become explicit in order for the intended (correct) conclusion to be drawn. Second, it is not clear whether the disjunct ‘Barbara is in Pakistan’ should be at the same place in the two premises, or at the first place in the first premise and at the second place in the second premise. Indeed, it is not sure that not logically trained individuals always interpret the disjunction as commutative in the ddt. But at the same time, it seems that people want to preserve the syntactic form of the premises: answers in disjunctive form are more common than in conditional form. Although Johnson-Laird et al. (1992, p. 433) explained subjects the di erence between an inclusive and an exclusive disjunction, guaranteeing a common interpretation of “. . . or . . . or both” and “. . . or . . . but not both” (between participants and experimenter), they give little detail about this explanation, save for the fact that it did not rely on truth-tables or natural deduction rules.
The second category pertains to the linguistic context opened by the ddt. First, it seems that, in designing the task, Johnson-Laird and his colleagues wanted to avoid the problem of an abstract task by giving the subjects a thematic task. But it is unclear whether the di erence would negatively a ect the results. Indeed, the premises in the ddt are rather abstract since they present names and places out of the blue. No context or situation supports the story. Second, nothing in the task says that ‘Barbara’ in both premises is the same individual, for the reasons mentioned above.
The third category of problems relates to the way the ddt is framed in general, and particularly the way the question is asked: ‘what, if anything, follows from these premises?’ Either classical logic is assumed and an infinite number of valid conclusions follows from the premises, and then subjects have no time or no space to give the full answer; or the answer is intended to be the conclusion that preserves the most semantic information from the premises and ‘if anything’ can be dropped from the question.
In what follows I will show that these issues can be addressed if we reconsider the ddt as a coordination problem between subject and experimenter. I will focus on the second and third category of problems, the first one being answered in the new formulation of the task I propose next. 3 3.1
Experimenters take often for granted that only one possible interpretation of the
logical vocabulary is correct, and think that departure from this interpretation is an error.
But, as shown by
As noted by
Moreover, as put forward by relevance theory
Further discussion of the relevance theory explanation of the success and failure of logical reasoning is not necessary, since my argument needs no other agreement than retaining its central insight. The ddt is content- and context-dependent and it seems reasonable to think that subjects interpret it as such, in a way that intermingles semantic and pragmatic inferences in reasoning. 3.2
The way the question is asked in the ddt is disputable, for reasons that are admitted
by Johnson-Laird himself, but that he does not seem to factor in the formulation of the
question. Indeed, Johnson-Laird acknowledges that “infinitely many valid conclusions
follow from any premises”
If it is true that logically untrained individuals do not draw ‘trivial’ conclusions from a set of premises, there is no need to try to lure them in thinking that there might be 2 Sperber et al. (1995, p. 44): “Past discussions of subjects’ performance have tended to focus on the task as already interpreted (by the experimenter). [...] But interpreting the task is part and parcel of performing it, and obeys criteria of rationality in its own right. The study of ‘content e ects’ is the study of sound cognitive processes that are by no means out of place in subjects’ performance.” some. Besides, the formulation of the question must indicate that there is some relevant information that can be inferred from the premises. Indeed, the Gricean cp would ask for a speaker’s intentions to be recognized by the hearer in order for the speaker’s utterance to be correctly interpreted. Why not assuming such a process in the ddt? As previously mentioned, a psychological experiment involves a certain level of cooperation between subjects and the experimenter. Certainly cp governs inferences to intentions and goals of others, and reconstructing them in ddt inevitably involves speculation. But it is not impossible to reconstruct a possible route to Ci+ and Ce+ that takes cp more seriously than the original theory. 4 4.1
The theory predicts that subjects will: (a) compute a representation of three (two) mental scenarios, or ‘mental models’, for each inclusive (resp.: exclusive) premise; (b) combine the representations, eliminating those incompatible with the information; and: (c) ‘read o ’ some appropriate conclusion and answer the question.
In this general setting, the intended interpretation requires an abstraction step. In step (a), subjects are supposed to replace the premises with a variable X 2 fA; B; Cg where A; B; C stand for each of the individual in the disjunct, to which a value is assigned, equivalent to ‘I know/I do not know where is X’. Step (b), for each variant of the ddt, is tantamount to assigning a value to each variable, compatible with the constraints embodied in the models of the premises.
Comparatively, subjects should be expected to have greater di culties to solve the ‘negative’ tasks. Johnson-Laird et al. explain this di culty by the necessity of obtaining an additional (implicit) premise, which follows from subjects background knowledge:
Of course, if [Barbara] is in [Bangladesh], then [she] is not in [Pakistan].
According to the model theory, a rmative deductions should be easier than
such negative deductions because the latter call for the detection of the
inconsistency between the contrary constituents.
So far, this reconstruction of steps (a) and (b) of ddt agrees with Johnson-Laird et al. However, reconstruction of step (c) raises serious issues with both the design of the experiment, and the interpretation of its results. Johnson-Laird et al. discuss only the selection made for the ‘positive’ ddt. Their justification for this selection is the “support” mental models yield to these conclusions.
Indeed, treating A, B and C as propositions, and substituting known (asserted) and
unknown (excluded) locations with 1 and 0, respectively, one obtains distributions of
truth-values that satisfy (Ci+) and (Ce+). However, this explanation is at odds with the
claim that reasoning based on mental models does not proceed by building and reading
equivalent of truth tables, since “truth tables [. . . ] are too bulky to be mentally
manipulated”
Considering the representation Johnson-Laird et al. gives for the set of final mental models in ddti+ and ddte+, reproduced Fig. 3, (Ce+) is the most natural of the answers considered. If the mental construction is actually carried as hypothesized by JohnsonLaird et al., (Ce+) can be ‘read o ’ neglecting excluded locations, and using ‘or. . . but not both’ to express the existence of (two) alternative scenarios. This method does not rely on explicit substitution with truth-values, but has no straightforward equivalent in ddti+ that would read (Ci+) o .
Alice Barbara Cheryl Alice Barbara Cheryl [I] [A] [I] [P] [A] [P] [I] [P]
[I]
One could hypothesize that subjects approximate ddti+ by addressing it as if it were ddte+, and then simply switch the proviso from “but not both” to “or both.” However, this rationale for the choice of (Ci+), is not consistent with the data, in particular the relative di erence between the rate of success in ddte+ and ddti+. Johnson-Laird et al. may have selected (Ci+) by the above reasoning, or simply by judging (Ce+) intuitively natural enough, and, switching the proviso, assuming that (Ci+) was natural, too. However they give no hint has to what their selection is based on, and rest content with the relative frequencies, that validate the predictions of mmt.
In the absence of an analysis of the ‘reading o ’ method that outputs (Ci+), it seems di cult to argue whether the answer corresponds to some natural method. Likewise, the absence of discussion of answers expected to be given to ddti and ddte makes the assessment of the task more complicated than Johnson-Laird et al. seem to assume. The naturalness of (Ce+) depends essentially on the representation of the problem that neglects specific locations, and in which one checks only whether a location is known, or not. If one assumes with Johnson-Laird et al. that reasoners somehow automatically select representations that are as implicit as possible, the inescapable conclusion is that the underlying automatisms are not very good at selecting the level of abstraction that makes the task solvable.
Yet, abstraction in variants of the ddt may not be as simple as assumed by JohnsonLaird et al. Reading (or hearing) (P1), (P2) and (P02), in either their inclusive or exclusive variants, may elicit a very di erent representation, if the range of values is taken to be (v(P1)) (v(0P1)) (v(P2)) (v(0P01)) (v(P02)) the set of locations. Then, upon reading (hearing) (P1), a subject may consider that the set of relevant values is:3 (where i p corresponds to any other possible location than India and Pakistan). However, reading (P2) produces a new set of values for that premise, as well as an update of (v(??)), as follows:
The e ect is even more striking with (P02), where the number of seemingly relevant values (and therefore of their combinations) increases even more, with: v(P1) = fi; p; i pg v(0P1) = fa; i; p; ai pg v(P2) = fa; i; p; ai pg v(0P01) = fa; b; i; p; abi pg v(P02) = fa; b; i; p; abi pg
Given the complexity of the task, one will typically not complete step (a) before the time the answer has to be given. Again, when “unable to construct a set of models or [. . . ] to discern what holds over all of them” subject will respond that no valid conclusion follows. 5
DDT as it is designed in psychology settings, is not a bad test for reasoning skills in the sense that it would mistake the competence to test. The problem comes from the way experimenters interpret the task itself. The di erence in interpretation between experimenters and subjects is what leads to a misinterpretation of the results, and shows the role of relevance and computational complexity in the answers of the subjects. Reconsidering the ddt as a coordination situation brings to light the reasoning process of the subjects in drawing inferences from a double disjunction. This process, as shown in the sketch of the formal reformulation of the task above, is a sequential procedure which starts from a step-by-step construction of representations task, and outputs a global conclusion. Our reformulation emphasizes the discrepancy between the intended interpretation of the ddt by the experimenters, and the task as it may be understood by the subjects.
What remains to do in a further research is, from a theoretical perspective, building a
complete formal representation of the ddt displaying the intended interpretation by the
experimenter and the possible interpretation(s) by the subjects. This reformulation of
the task relies on the di erence made by
From an empirical perspective, we need to design an experiment based on the explanatory hypothesis argued for in this paper, especially concerning the level of abstraction required from the subjects in order to solve the task. To this e ect, it seems desirable to handle both sides of the problem by testing subjects’ performance in a fairly abstract task where the disjuncts are displayed as propositional variables such as A; B; C, and compare the results with a ddt where the premises are contextually fleshed out by a story to fulfill the need for a thematic task.