<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta />
    <article-meta>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Caspar Oesterheld</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Vincent Conitzer</string-name>
          <email>conitzerg@cs.duke.edu</email>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          <institution>Duke University, Department of Computer Science</institution>
        </aff>
      </contrib-group>
      <pub-date>
        <year>2020</year>
      </pub-date>
      <abstract>
        <p>Newcomb's problem has spawned a debate about which variant of expected utility maximization (if any) should guide rational choice. In this paper, we provide a new argument against what is probably the most popular variant: causal decision theory (CDT). In particular, we provide two scenarios in which CDT voluntarily loses money. In the first, an agent faces a single choice and following CDT's recommendation yields a loss of money in expectation. The second scenario extends the first to a diachronic Dutch book against CDT. Contact Author</p>
      </abstract>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>-</title>
      <p>In Newcomb’s problem [Nozick, 1969; Ahmed, 2014], a
“being” offers two boxes, A and B. Box A is transparent and
contains $1,000. Box B is opaque and may contain either
$1,000,000 or nothing. An agent is asked to choose between
receiving the contents of both boxes, or of box B only.
However, the being has put $1,000,000 in box B if and only if the
being predicted that the agent would choose box B only. The
being’s predictions are uncannily accurate. What should the
agent do?</p>
      <p>Causal decision theory (CDT) recommends that the agent
reason as follows: I cannot causally affect the content of the
boxes – whatever is in the boxes is already there. Thus, if
I choose both boxes, regardless of what is in box B, I will
end up with $1,000 more than if I choose one box. Hence, I
should choose both boxes.</p>
      <p>Evidential decision theory (EDT), on the other hand,
recommends that the agent reason as follows: if I choose
one box, then in all likelihood the being predicted that I
would choose one box, so I can expect to walk away with
$1,000,000. (Even if the being is wrong some small
percentage of the time, the expected value will remain at least close to
$1,000,000.) If I choose both, then I can expect to walk away
with (close to) $1,000. Hence, I should choose one box.</p>
      <p>While Newcomb’s problem itself is far-fetched, it has been
argued that the difference between CDT and EDT matters in
various game-theoretic settings. First, it has been pointed out
that Newcomb’s problem closely resembles playing a
Prisoner’s Dilemma against a similar opponent [Brams, 1975;
Lewis, 1979]. Whereas CDT recommends defecting even
in a Prisoner’s Dilemma against an exact copy, EDT
recommends cooperating when facing sufficiently similar
opponents. What degree of similarity is required and whether
EDT and CDT ever come apart in this way in practice has
been the subject of much discussion [Ahmed, 2014,
Section 4.6]. A common view is that CDT and EDT come
apart only under fairly specific circumstances that do not
include most human interactions. Still, Hofstadter [1983],
for example, argues that “superrational” humans should
cooperate with each other in a real-world one-shot Prisoner’s
Dilemma, reasoning in a way that resembles the reasoning
done under EDT. Economists have usually been somewhat
dismissive of such ideas, sometimes referring to them as
“(quasi-)magical thinking” when trying to explain observed
human behavior [Shafir and Tversky, 1992; Masel, 2007;
Daley and Sadowski, 2017]. Indeed, standard game-theoretic
solution concepts are closely related to ratificationism [Joyce
and Gibbard, 1998, Section 5], a variant of CDT which we
will revisit later in this paper (see Part 3 of Section 4).</p>
      <p>Second, even if the players of a game are dissimilar, it is
in the nature of strategic interactions that each player’s
behavior is predicted by the other players. Newcomb’s
problem itself (as well as the ADVERSARIAL OFFER discussed in
this paper) differs from strategic interactions as they are
usually considered in game theory in its asymmetry. However,
one might still expect that CDT and EDT offer different
perspectives on how rational agents should deal with the mutual
prediction inherent in strategic interactions [Gauthier, 1989,
Section XI].</p>
      <p>
        Third, CDT and EDT differ in their treatment of situations
with imperfect recall. Seminal discussions of such games are
due to Piccione and Rubinstein [
        <xref ref-type="bibr" rid="ref8">1997</xref>
        ] and Aumann et al.
[
        <xref ref-type="bibr" rid="ref8">1997</xref>
        ] [cf. Bostrom, 2010, for an overview from a
philosopher’s perspective]. While these problems originally were
not associated with Newcomb’s problem, the relevance of
different decision theories in this context has been pointed
out by Briggs [
        <xref ref-type="bibr" rid="ref12 ref29 ref4 ref9">2010</xref>
        ] [cf. Armstrong, 2011; Schwarz, 2015;
Conitzer, 2015].
      </p>
      <p>The importance of the differences in decision theory is
amplified if, instead of humans, we consider artificial agents.
After all, it is common that multiple copies of a software
system are deployed and other parties are often able to obtain
the source code of a system to analyze or predict its behavior.
As some software systems choose more autonomously, we
might expect their behavior will be (approximately)
describable by CDT or EDT (or yet some other theory). If either of
these theories has serious flaws, we might worry that if a
system implements the wrong theory, it will make unexpected,
suboptimal choices in some scenarios. Such scenarios might
arise naturally, e.g., as many copies of a system are deployed.
We might also worry about adversarial problems like the one
in this paper.</p>
      <p>One argument against CDT is that causal decision theorists
(tend to) walk away with less money than evidential decision
theorists, but this argument has not proved decisive in the
debate. For instance, one influential response has been that CDT
makes the best out of the situation – fixing whether the money
is in box B – which EDT does not [Joyce, 1999, Section 5.1].
It would be more convincing if there were Newcomb-like
scenarios in which a causal decision theorist volunteers to lose
money (in expectation or with certainty).1 Constructing such
a scenario from Newcomb’s problem is non-trivial. For
example, in Newcomb’s problem, a causal decision theorist may
realize that box B will be empty. Hence, he would be
unwilling to pay more than $1,000 for the opportunity to play the
game.</p>
      <p>In this paper, we provide Newcomb-like decision problems
in which the causal decision theorist voluntarily loses money
to another agent. We first give a single-decision scenario in
which this is true only in expectation (Section 2). We then
extend the scenario to create a diachronic Dutch book against
CDT – a two-step scenario in which the causal decision
theorist is sure to lose money (Section 3). Finally, we discuss the
implications of the existence of such scenarios (Section 4).</p>
      <p>1Walking away with the maximum possible (expected) payoff
under any circumstances is not a realistic desideratum for a decision
theory: any decision theory X has a lower expected payoff than some
other decision theory Y in a decision problem that rewards agents
simply for using decision theory Y [cf. Skalse, 2018, for a
harderto-defuse version of this point]. However, such a setup does not
allow one to devise a generic scenario in which an agent voluntarily
loses money, i.e. loses money in spite of having the option to walk
away losing nothing.</p>
      <p>Furthermore, scenarios with voluntary loss appear significantly
more problematic for pragmatic reasons. Regardless of what you
think is the right option in Newcomb’s problem, you might not view
Newcomb’s problem as relevant ground for decision-theoretical
argument because it is so unlikely that one would ever face
Newcomb’s problem in the real world. For instance, even if you thought
that one-boxing is rational (and two-boxing is not), you might stick
with CDT nonetheless because your real-world expected
opportunity costs from two-boxing in Newcomb’s problem are negligible.
[For some discussion of this deflationary argument, see, e.g.,
Gauthier, 1989, Section XI; Ahmed, 2014, Section 7.1.iv; Oesterheld,
2019, Section 1, and references therein.] However, if there is a
Newcomb-like problem in which the causal decision theorist
voluntarily loses money to some other agent, this generates a significant
incentive to place him in such a situation.</p>
    </sec>
    <sec id="sec-2">
      <title>Extracting a profit in expectation from causal decision theorists</title>
      <sec id="sec-2-1">
        <title>Consider the following scenario:</title>
        <p>ADVERSARIAL OFFER: Two boxes, B1 and B2,
are on offer. A (risk-neutral) buyer may purchase
one or none of the boxes but not both. Each of
the two boxes costs $1. Yesterday, the seller put
$3 in each box that she predicted the buyer not to
acquire. Both the seller and the buyer believe the
seller’s prediction to be accurate with probability
0:75. No randomization device is available to the
buyer (or at least no randomization device that is
not predictable to the seller).2</p>
        <p>If the buyer takes either box Bi, then the expected money
gained by the seller is
=
=
$1
$1
$0:25:</p>
        <p>P ($3 in Bi j buyer chooses Bi) $3
0:25 $3
Hence, the buyer suffers an expected loss of $0:25 (if he buys
a box). The best action for the buyer therefore appears to be
to not purchase either box. Indeed, this is the course of action
prescribed by EDT as well as other decision theories that
recommend one-boxing in Newcomb’s problem [e.g., those
proposed by Spohn, 2012; Poellinger, 2013; Soares and
Levinstein, 2017].</p>
        <p>In contrast, CDT prescribes that the buyer buy one of the
two boxes. Because the agent cannot causally affect
yesterday’s prediction, CDT prescribes to calculate the expected
utility of buying box Bi as</p>
        <p>P ($3 in box Bi) $3
$1;
(1)
where P ($3 in box Bi) is the buyer’s subjective probability
that the seller has put money in box Bi, prior to updating
this belief based on his own decision. For i = 1; 2, let pi be
the probability that the buyer assigns to the seller having
predicted him to buy Bi. Similarly, let p0 be the probability the
buyer assigns to the seller having predicted him to buy
nothing. These beliefs should satisfy p0 + p1 + p2 = 1. Because
p0 0, we have that (p0+p1)+(p0+p2) = 2p0+p1+p2 1.
Hence, it must be the case that p0 + p1 21 or p0 + p2 12
(or both). Because P ($3 in box Bi) = p0 + p3 i for i = 1; 2,
it is P ($3 in box Bi) 12 for at least one i 2 f1; 2g. Thus,
the expected utility in eq. 1 of at least one of the two possible
purchases is at least 21 $3 $1 = $0:50, which is positive.</p>
        <p>Any seller capable of predicting the causal decision
theorist sufficiently well will thus have an incentive to use this
scheme to exploit CDT agents. (It does not matter whether
the seller subscribes to CDT or EDT.) It should be noted that
even if the buyer uses CDT, his view of the deal matches the
seller’s as soon as the dollar is paid. That is, after
observing his action, he will realize that the box he bought is empty
2This decision problem resembles the widely discussed Death in
Damascus scenario [introduced to the decision theory literature by
Gibbard and Harper, 1981, Section 11] and even more closely the
Frustrater case proposed by Spencer and Wells [2017], though these
are not set up to result in an expected financial loss.
with probability 0:75 and thus worth less than a dollar. CDT
knows that it will regret its choice [see Joyce, 2012; Weirich,
1985 for discussions of the phenomenon of anticipated regret
a.k.a. decision instability in CDT].
3</p>
      </sec>
    </sec>
    <sec id="sec-3">
      <title>A diachronic Dutch book against causal decision theory</title>
      <p>ADVERSARIAL OFFER results in a loss in expectation for the
causal decision theorist. It is natural to ask whether it is
possible to set up the scenario so that the causal decision
theorist ends up with a sure loss; effectively, a Dutch book.
Arguably, Dutch books are more convincing than scenarios with
expected losses since the very meaning of “expectations” is
the subject of the debate about EDT and CDT. Of course, if
the seller could perfectly predict the buyer in ADVERSARIAL
OFFER (instead of being right only 75% of the time), then
ADVERSARIAL OFFER would become a Dutch book. But
can we construct a Dutch book without perfect prediction?</p>
      <p>We have already observed that in ADVERSARIAL OFFER
the causal decision theorist always regrets his decision after
observing its execution. This suggests the following simple
approach to constructing a Dutch book. After the box is sold,
the seller allows the buyer to reverse his decision for a small
fee (ending up without any box and having lost only the fee).
However, a CDT buyer may then anticipate eventually
undoing his choice and therefore not buy a box in the first place
[Ahmed, 2014, Section 3.2; though cf. Skyrms, 1993;
Rabinowicz, 2000].3 To get our Dutch book to work, we add
another choice before ADVERSARIAL OFFER.</p>
      <sec id="sec-3-1">
        <title>ADVERSARIAL OFFER WITH OPT-OUT: It is</title>
        <p>Monday. The buyer is scheduled to face the
ADVERSARIAL OFFER on Tuesday. He also knows
that the seller’s prediction was already made on
Sunday.</p>
        <p>As a courtesy to her customer, the seller approaches
the buyer on Monday. She offers to not offer the
boxes on Tuesday if the buyer pays her $0:20.</p>
        <p>Note that the seller does not attempt to predict whether the
buyer will pay to opt out. Also, we assume that the buyer
cannot, on Monday, commit himself to a course of action to
follow on Tuesday.</p>
        <p>It seems that a rational agent should never feel compelled
to accept the Monday offer. After all, doing so loses him
money with certainty, whereas simply refusing both offers
(on Monday and on Tuesday) guarantees that he loses no
money.</p>
        <p>CDT, however, recommends opting out on Monday, for the
following reasons. A CDT buyer knows on Monday that if
he does not opt out, he will buy a box on Tuesday (though
he may not yet know which one). Further, he believes that
whatever box he will take on Tuesday will contain $3 with
only 25% probability, thus implying an overall expected
payoff of 0:25 $3 $1 = $0:25. This is because, on Monday,
CDT treats the decision on Tuesday in the same way as it
3This, of course, requires that the reversal offer does not come as
a surprise. Throughout, we insist that the buyer knows all the rules
of the game.
treats any other random variable in the environment. So the
causal expected utility of not opting out is just what an
outside observer would expect the payoff of a CDT agent facing
ADVERSARIAL OFFER to be. Because this expected payoff
of $0:25 is less than the certain payoff of $0:20 that can
be obtained by opting out, CDT recommends opting out.</p>
        <p>In fact, for the argument in the previous paragraph to
succeed, it is only necessary that CDT is used on Tuesday; other
decision theories would also recommend accepting the
Monday offer, if they anticipate that the agent will use CDT on
Tuesday. For instance, if the agent followed EDT on Monday
and CDT on Tuesday (and is aware on Monday that he will
use CDT on Tuesday), then he would still accept the Monday
offer. Similarly, if the seller believes that the buyer will pick
one of the boxes on Tuesday, then she will hope that he
rejects the Monday offer. Thus, it seems that what creates the
opportunity for a Dutch book is the prospect of buying a box
on Tuesday (as CDT recommends), not the use of CDT on
Monday.
4</p>
      </sec>
    </sec>
    <sec id="sec-4">
      <title>Discussion</title>
      <p>We differentiate four types of responses to these scenarios
available to supporters of causal decision theory:
1. They could claim that these scenarios are irrelevant for
evaluating decision theories, in the sense that they are
impossible to set up or otherwise out of scope, and
therefore unpersuasive.
2. They could concede that these scenarios are relevant for
evaluating decision theories, but claim that CDT’s
recommendations in them are acceptable.
3. They could concede that our analysis obliges them to
give up on certain specific formulations of CDT, but try
to modify CDT to get these scenarios right while
maintaining some of its essence, in particular two-boxing and
the causal dominance principle.
4. They could concede that these scenarios show that the
very core of CDT (two-boxing and the causal dominance
principle) is implausible.</p>
      <p>We will discuss these options in turn.
1 Surely, if one could show that a CDT agent will or can
never face these scenarios – despite the seller having an
obvious incentive to set them up – that would be the most
convincing defense of CDT. In particular, a causal decision theorist
might claim that sufficiently accurate prediction of a CDT
agent is simply impossible.4 However, not much accuracy
is required, for the following reasons. The CDT agent will
take one of the two boxes. Even if the seller picks the box to
fill with money uniformly at random, she would therefore be
right half of the time. If she can do any better than that,
predicting correctly with probability 1=2+ , then she can extract
money from the CDT agent by putting (instead of $3) some
amount between $2=(1 2 ) and $2 in the box predicted not
4For a general discussion of such unpredictability claims in
defense of CDT, see Ahmed [2014, Chapter 8].
to be taken. Thus, the CDT agent needs to be completely
unpredictable in order to avoid being taken advantage of in these
examples.</p>
      <p>Most human beings are, generally speaking, at least
somewhat predictable in their actions even when such
predictability can be used against them. For example, in
rock-paperscissors – which structurally resembles the ADVERSARIAL
OFFER – most people follow exploitable patterns in what
moves they select [see, e.g., Farber, 2015, and references
therein].5 Consider such a somewhat predictable person
who aims to be a causal decision theorist. It seems that he
would indeed be vulnerable to the examples discussed
earlier. The only defense for the supporter of causal decision
theory would seem to then be that if so, the person in
question is not truly acting in the way that CDT describes. That
is, acting according to CDT also requires being unpredictable
to the seller, either by succeeding at out-thinking the seller
sufficiently often, or by acting sufficiently randomly.</p>
      <p>Is it reasonable to consider this a requirement of acting
according to CDT? CDT does not suggest any strict preference
for choosing randomly across options, as opposed to just
deterministically choosing one of the options that is best
according to the buyer’s beliefs. Hence, the unpredictability would
have to emerge from the buyer attempting to out-think the
seller. But it does not seem that this is always an attainable
goal. For example, imagine that the buyer is a deterministic
computer program whose source code is known to the seller.
Then regardless of how exactly the agent works, the seller
can predict the buyer’s behavior perfectly [cf. Soares and
Fallenstein, 2014, Section 2; Cavalcanti, 2010, Section 5]. We
would thus be forced to conclude that such a program cannot
possibly follow CDT, which to us is an unsatisfactory
conclusion. Plausibly any other physically realized agent that
chooses deterministically can at least in principle (if not with
current technology) be predicted by creating or emulating an
atom-by-atom copy of that agent [cf. Yudkowsky, 2010, pp.
85ff.].</p>
      <p>
        Even if the supporter of CDT acknowledges that these
scenarios are possible, he might nevertheless argue that they are
irrelevant, in the sense that the decision theory is not intended
to be used for such scenarios and hence nothing that one could
show about its performance in such a scenario is of
significance for evaluating the theory. “It is as if one evaluated a
car by testing how it performs underwater.” There is little we
can say about this response. Still, we expect it to be
unattractive to most decision theorists. After all, our scenarios (in
particular the ADVERSARIAL OFFER) resemble Newcomb’s
problem – the problem that has led to the development of
CDT in the first place. Further, if our scenarios were out of
CDT’s scope, then we (and presumably most other decision
theorists) would still be interested in identifying a decision
theory that does make good recommendations for predictable
agents (such as artificial intelligent agents whose behavior is
5There are multiple rock-paper-scissors bots available online
which attempt to predict their opponent’s future moves based on
past moves (using data from other players). As of July 2019, the
bot at http://www.essentially.net/rsp/ has reportedly played about 2
million rounds and won 57% more often than it lost.
determined by a computer program) facing a wide range of
scenarios including the ones given in this paper.
2 If our scenarios are within the scope of causal decision
theory, then the supporter of causal decision theory has to
contend with the fact that one can extract expected money
from, and even Dutch-book, CDT agents in them. But he
might question the significance of Dutch-book arguments and
other money extraction schemes, either in general or in this
particular context. For some general discussion of whether
(diachronic) Dutch books are conclusive decision-theoretic
arguments, see, e.g., Vineberg [
        <xref ref-type="bibr" rid="ref48">2016</xref>
        ] or Ha´jek [
        <xref ref-type="bibr" rid="ref23">2009</xref>
        ]. Note,
though, that some of the most influential arguments in favor
of expected utility maximization (EUM) – of which CDT is
a refinement – are Dutch books. Of course, one may use
different arguments to justify EUM. But it would seem odd to
follow Dutch-book arguments to EUM but no further.
      </p>
      <p>Instead of rehashing some of the more generic reasons for
and against the persuasiveness of Dutch books and loss of
money in expectation, we here discuss a response that is
specific to CDT and ADVERSARIAL OFFER WITH OPT-OUT.6
A causal decision theorist may argue that it is not
generally fair to expect any kind of coherence from CDT’s
recommendations when multiple decisions are to be made across
time, due to the different perspectives that the decision maker
adopts (and, arguably, has to adopt) at different points in time.
Consider Newcomb’s problem. Let t0 be the time at which
the predictor observes the agent (perhaps using fMRI or the
like) in order to make a prediction. Then, before t0, CDT
recommends committing – and if needed paying money to
commit – to one-boxing [cf. Barnes, 1997; Joyce, 1999, pp. 153f.;
Meacham, 2010]. After t0, CDT recommends two-boxing.
However, most decision theorists do not consider this to be a
compelling argument against CDT. The causal decision
theorist can easily justify the difference in the decision made by
the fact that, before t0, the commitment decision has a causal
effect on what is in the boxes, and after t0, it does not.</p>
      <p>It would be hypocritical for an evidential decision theorist
to disagree, since EDT is dynamically inconsistent in
analogous ways. For instance, consider a version of Newcomb’s
problem in which both boxes are transparent [Gibbard and
Harper, 1981, Section 10; also discussed by Gauthier, 1989;
Drescher, 2006, Section 6.2; Arntzenius, 2008, Section 7;
Meacham, 2010, Section 3.2.2]. Let t00 be the time at which
the EDT agent sees the content of both boxes. Then before t00,
EDT recommends committing – and if needed paying money
to commit – to one-boxing. After t00, EDT recommends
twoboxing.7 The evidential decision theorist can easily justify
this along similar lines: before t00, her commitment is
evidence about what is in the boxes, and after t00 it no longer
is.</p>
      <p>6For a discussion of similar arguments about other diachronic
Dutch books, see, e.g., Rabinowicz [2008].</p>
      <p>
        7 Parfit’s (1984) hitchhiker [Barnes, 1997], XOR Blackmail
[Soares and Levinstein, 2017, Section 2] and Yankees vs. Red Sox
[Arntzenius, 2008, pp. 22-23; Ahmed and Price, 2012] similarly
expose dynamic inconsistencies in EDT. Conitzer [
        <xref ref-type="bibr" rid="ref17 ref37">2015</xref>
        ] gives a
somewhat different type of scenario – based on the Sleeping Beauty
problem – in which EDT is dynamically inconsistent.
      </p>
      <p>Thus, at least some types of dynamic inconsistency do not
constitute strong arguments against a decision theory.
However, in our opinion, the dynamic inconsistency displayed
by CDT in the ADVERSARIAL OFFER WITH OPT-OUT is
much more problematic. For one, it leads to a Dutch book.
Often, the main argument that is given for why a
particular inconsistency is problematic is precisely that it allows
for a Dutch book. Conversely, defenses of dynamic
inconsistencies [Ahmed, 2014, Section 3.2, for an example in a
Newcomb-like scenario] often focus on arguing that they do
not allow for Dutch Books.</p>
      <p>Further, it seems that some of the reasons for (or defenses
of) dynamic inconsistency in the above decision problems do
not apply to CDT’s dynamic inconsistency in ADVERSARIAL
OFFER WITH OPT-OUT. For CDT in Newcomb’s problem,
there is a particular event at time t0 that splits the decision
perspectives: the loss of causal control at t0 over the content
of box B. Similarly, for EDT in the Newcomb’s problem with
transparent boxes, that event is the loss of evidential control
[cf. Almond, 2010, Section 4.5] at t00 over the content of box
B. It is thus easy to argue for defenders of the respective
theories that the perspectives from before and after t0 or t00 should
diverge [Ahmed and Price, 2012, pp. 22-23, Section 4]. In
sharp contrast, the ADVERSARIAL OFFER WITH OPT-OUT
lacks any such event between the decision points. The
difference in perspectives for CDT appears to be purely a result of
CDT viewing its current choice differently than it views past
and future decisions.</p>
      <p>All that being said, we agree that caution should be taken
when evaluating a decision theory based on scenarios with
multiple decisions across time. In general, more research on
what conclusions can be drawn from such scenarios is needed
[Steele and Stefa´nsson, 2016, Section 6]. Nevertheless, we do
not see any clear path by which such research would justify
CDT’s recommendations in the ADVERSARIAL OFFER WITH
OPT-OUT. In any case, even if one is at this point unwilling to
consider scenarios with multiple decision points at all for the
purpose of evaluating decision theories, one would still have
to contend with the simpler ADVERSARIAL OFFER scenario,
in which there is only one decision point.
3 If a straightforward interpretation of CDT cannot be
defended against our scenarios, one may look to modify it to
avoid expected or sure loss while preserving some of CDT’s
core tenets. In particular, in response to other alleged
counterexamples, some authors have tried to modify CDT while
maintaining the causal dominance [Joyce, 1999, Section 5.1]
a.k.a. sure thing [Gibbard and Harper, 1981, Section 7]
principle [though see Ahmed, 2012, for an argument against the
motivation behind some of these approaches]. For example,
one may turn to the concept of ratifiability. In Newcomb-like
scenarios such as those under discussion here, for any choice
a, we can consider the beliefs about what is in the boxes that
would result from knowing that one will choose a. Then, a
choice a is ratifiable if it is an optimal choice – as judged
by CDT – under those beliefs. For example, in Newcomb’s
problem only two-boxing is ratifiable, precisely because it is
causally dominant. For an overview of ratification and its
relation to CDT, see Weirich [2016, Section 3.6].
Unfortunately, this concept is of no help in the ADVERSARIAL
OFFER, because none of the three options (buying B1, buying
B2 or declining) is ratifiable. For instance, under the beliefs
that would result from knowing that you will take box Bi, it
would be better to buy the other box B3 i.</p>
      <p>The ratificationist may respond by claiming that
unpredictable randomization should always be possible. If that
were true, then the only ratifiable option would be to take
each box with probability 50%, thus gaining money in
expectation. But again, we would like to have a decision
theory that works in a broad variety of scenarios, including ones
where the agent expects to be somewhat predictable.
Furthermore, even if a true random number generator (TRNG) (e.g.,
one based on nuclear decay) is in fact available, this does not
settle the issue. For example, consider a variant of the
ADVERSARIAL OFFER in which the seller refrains from putting
money in any box if she predicts the buyer to make different
choices depending on the output of the TRNG. In this
ANTIRANDOMIZATION ADVERSARIAL OFFER, again no option
is ratifiable: under the beliefs that would result from
knowing that you will make different choices depending on the
TRNG’s output (and therefore choose a box with some
positive probability), you would rather not pick any box. To
circumvent this example, the ratificationist could argue that the
decision maker should be able to randomize in such a way
that whether he is randomizing is unpredictable. However, at
this point, one might just as well assert the impossibility (or
irrelevance) of Newcomb-type scenarios altogether, which we
have addressed in 1.</p>
      <p>A different strategy for modifying CDT to avoid the Dutch
book in the ADVERSARIAL OFFER WITH OPT-OUT is the
following. The Dutch book arises from a disagreement
between CDT on Monday and CDT on Tuesday (cf. the
discussion under 2). A tempting possibility is to modify CDT so that
it considers all decisions to be made at once. That is, such a
version of CDT – let us refer to it as policy-CDT – prescribes
that one decide on one’s general policy all at once.8 In the
ADVERSARIAL OFFER WITH OPT-OUT, there are four
possible policies: opt out, buy B1, buy B2, and buy nothing (where
the last three possibilities include declining the opt-out offer).
When considering these policies, buy nothing dominates opt
out. Hence, policy-CDT will decline the opt-out offer and
thereby avoid the Dutch book. (Note, however, that such a
modification of CDT will make no difference to the choices it
prescribes in ADVERSARIAL OFFER, which has only one
decision point. Hence, it will still lose money in expectation.)</p>
      <p>
        While this appears to be a promising approach, it is
nontrivial to flesh out, because on other examples it is less clear
what policy-CDT should prescribe. For illustration, consider
the following interpretation of policy-CDT: follow the
policy to which CDT would like to commit ex ante, where “ex
8 Policy-CDT resembles Fisher’s [nd] disposition-based decision
theory. Compare Meacham [
        <xref ref-type="bibr" rid="ref12 ref29 ref4 ref9">2010</xref>
        ] for a discussion of explicit
precommitment. Similarly, Gauthier [1989] has argued for evaluating
“plans” not decisions in Newcomb-like problems (without basing
this argument on any particular theory like CDT or EDT). A few
authors have also proposed policy versions of other, more EDT-like
decision theories [Drescher, 2006, Section 6.2; Yudkowsky and Soares,
2018, Section 4].
ante” refers to some point in time before the first decision of
the scenario. Now, let us consider a version of Newcomb’s
problem which is supplemented by another trivial and
unrelated decision – say, whether to eat a peppermint – that
takes place when the agent still has a causal influence over
the prediction. Then the ex-ante-commitment interpretation
of policy-CDT would recommend one-boxing. To the causal
decision theorist, this may be unacceptable, especially given
that adding the peppermint decision is such a minor
modification of Newcomb’s problem. Perhaps there is a way to
define policy-CDT that avoids such dependence on irrelevant
decisions while also prescribing two-boxing, but it is not
immediately obvious how to do so.
      </p>
      <p>
        Many other ways of modifying CDT are worth
considering. For instance, in the ADVERSARIAL OFFER, it may be
unrealistic for the buyer to form a single probability distribution
over box contents. Instead, he may consider multiple different
probability distributions, including one under which box B1
is probably empty and one under which box B2 is probably
empty. He could then evaluate each option pessimistically,
i.e., w.r.t. the probability distribution that is worst under that
option. Such a version of CDT would prescribe declining
to buy a box. At the same time, it would recommend
twoboxing in Newcomb’s problem and more generally obey the
causal dominance principle. For a discussion of this maxmin
criterion for choice under multiple probability distributions,
see, e.g., Gilboa and Schmeidler [1989] and in particular
game-theoretic interpretations such as that of Gru¨nwald and
Halpern [2011]. A more general discussion of how using sets
of probability distributions (while potentially decision rules
other than the maxmin criterion) is offered by Bradley [
        <xref ref-type="bibr" rid="ref10 ref2 ref26 ref44">2012</xref>
        ].
In our setting, B1 are B2 are, roughly, complementary bets in
the causalist’s beliefs. In all worlds in which Bi is empty,
B3 i is full. As discussed by Bradley, it has been argued that
a rational agent should accept one of a pair of complementary
bets. Indeed, expected utility maximization for a single
probability distribution satisfies this complementarity criterion –
to the causalist’s detriment in the Adversarial Offer. Bradley
[
        <xref ref-type="bibr" rid="ref10 ref2 ref26 ref44">2012</xref>
        ] argues that in general, an agent with imprecise
probabilities should not satisfy the complementarity criterion and
that this allows him to avoid Dutch books – though, of course,
he considers Dutch books of a very different type.
4 Finally, one may view at least one of the scenarios in
this paper as supporting a persuasive argument against the
very core of CDT. EDT is the obvious alternative.
However, depending on how problematic we find EDT’s
prescriptions in other cases – such as the Smoking lesion [Ahmed,
2014, Section 4.1–4.3] or cases of dynamic inconsistency like
Newomb’s problem with transparent boxes (and the problems
listed in footnote 7) – we may also look to various other
decision theories that have been proposed [Gauthier, 1989;
Spohn, 2012; Poellinger, 2013; Soares and Levinstein, 2017].
      </p>
    </sec>
    <sec id="sec-5">
      <title>Acknowledgements</title>
      <p>We thank Johannes Treutlein and Jesse Clifton for comments
and discussions.
Anthropic decision
[Fisher, nd] Justin C. Fisher. Disposition-based decision
theory. n.d.</p>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          <source>[Ahmed and Price</source>
          , 2012]
          <string-name>
            <given-names>Arif</given-names>
            <surname>Ahmed</surname>
          </string-name>
          and
          <string-name>
            <given-names>Huw</given-names>
            <surname>Price</surname>
          </string-name>
          .
          <article-title>Arntzenius on 'why ain'cha rich?'</article-title>
          . Erkenntnis,
          <volume>77</volume>
          (
          <issue>1</issue>
          ):
          <fpage>15</fpage>
          -
          <lpage>30</lpage>
          , 7
          <year>2012</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          <source>[Ahmed</source>
          , 2012]
          <string-name>
            <given-names>Arif</given-names>
            <surname>Ahmed</surname>
          </string-name>
          .
          <article-title>Push the button</article-title>
          .
          <source>Philosophy of Science</source>
          ,
          <volume>79</volume>
          (
          <issue>3</issue>
          ):
          <fpage>386</fpage>
          -
          <lpage>395</lpage>
          , 7
          <year>2012</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          <source>[Ahmed</source>
          , 2014]
          <string-name>
            <given-names>Arif</given-names>
            <surname>Ahmed</surname>
          </string-name>
          .
          <article-title>Evidence, Decision and Causality</article-title>
          . Cambridge University Press,
          <year>2014</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          <source>[Almond</source>
          , 2010]
          <string-name>
            <given-names>Paul</given-names>
            <surname>Almond</surname>
          </string-name>
          .
          <article-title>On causation and correlation part 1: Evidential decision theory is correct,</article-title>
          <year>9 2010</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          <source>[Armstrong</source>
          , 2011]
          <string-name>
            <given-names>Stuart</given-names>
            <surname>Armstrong</surname>
          </string-name>
          . theory,
          <year>Nov 2011</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          <source>[Arntzenius</source>
          , 2008]
          <string-name>
            <given-names>Frank</given-names>
            <surname>Arntzenius</surname>
          </string-name>
          .
          <article-title>No regrets, or: Edith piaf revamps decision theory</article-title>
          .
          <source>Erkenntnis</source>
          ,
          <volume>68</volume>
          (
          <issue>2</issue>
          ):
          <fpage>277</fpage>
          -
          <lpage>297</lpage>
          ,
          <year>2008</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          [Aumann et al.,
          <year>1997</year>
          ]
          <string-name>
            <given-names>Robert J.</given-names>
            <surname>Aumann</surname>
          </string-name>
          , Sergiu Hart, and
          <string-name>
            <given-names>Motty</given-names>
            <surname>Perry</surname>
          </string-name>
          .
          <article-title>The absent-minded driver</article-title>
          .
          <source>Games and Economic Behavior</source>
          ,
          <volume>20</volume>
          :
          <fpage>102</fpage>
          -
          <lpage>116</lpage>
          ,
          <year>1997</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          <source>[Barnes</source>
          ,
          <year>1997</year>
          ]
          <string-name>
            <given-names>R. Eric</given-names>
            <surname>Barnes</surname>
          </string-name>
          .
          <article-title>Rationality, dispositions, and the newcomb paradox</article-title>
          .
          <source>Philosophical Studies: An International Journal for Philosophy in the AnalyticTradition</source>
          ,
          <volume>88</volume>
          (
          <issue>1</issue>
          ):
          <fpage>1</fpage>
          -
          <lpage>28</lpage>
          ,
          <year>10 1997</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          <source>[Bostrom</source>
          , 2010]
          <string-name>
            <given-names>Nick</given-names>
            <surname>Bostrom</surname>
          </string-name>
          .
          <source>Anthropic Bias: Observation Selection Effects in Science and Philosophy. Studies in Philosophy. Routledge</source>
          ,
          <year>2010</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          <source>[Bradley</source>
          , 2012]
          <string-name>
            <given-names>Seamus</given-names>
            <surname>Bradley</surname>
          </string-name>
          .
          <article-title>Dutch book arguments and imprecise probabilities</article-title>
          . In D. Dieks,
          <string-name>
            <given-names>W.</given-names>
            <surname>Gonzalez</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S.</given-names>
            <surname>Hartmann</surname>
          </string-name>
          , M. Sto¨ltzner, and M. Weber, editors, Probabilities, Laws, and Structures, volume
          <volume>3</volume>
          <source>of The Philosophy of Science in a European Perspective</source>
          , pages
          <fpage>3</fpage>
          -
          <lpage>17</lpage>
          . Springer, Dordrecht,
          <year>2012</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref11">
        <mixed-citation>
          <source>[Brams</source>
          , 1975]
          <string-name>
            <given-names>Steven J.</given-names>
            <surname>Brams</surname>
          </string-name>
          .
          <article-title>Newcomb's problem and prisoners' dilemma</article-title>
          .
          <source>The Journal of Conflict Resolution</source>
          ,
          <volume>19</volume>
          (
          <issue>4</issue>
          ):
          <fpage>596</fpage>
          -
          <lpage>612</lpage>
          ,
          <year>12 1975</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref12">
        <mixed-citation>
          <source>[Briggs</source>
          , 2010]
          <string-name>
            <given-names>Rachael</given-names>
            <surname>Briggs</surname>
          </string-name>
          .
          <article-title>Putting a value on beauty</article-title>
          . volume
          <volume>3</volume>
          of Oxford Studies in Epistemology, pages
          <fpage>3</fpage>
          -
          <lpage>34</lpage>
          . Oxford University Press,
          <year>2010</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref13">
        <mixed-citation>
          <source>[Cavalcanti</source>
          , 2010] Eric G.
          <article-title>Cavalcanti. Causation, decision theory, and bell's theorem: A quantum analogue of the newcomb problem</article-title>
          .
          <source>The British Journal for the Philosophy of Science</source>
          ,
          <volume>61</volume>
          (
          <issue>3</issue>
          ):
          <fpage>569</fpage>
          -
          <lpage>597</lpage>
          , 9
          <year>2010</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref14">
        <mixed-citation>
          <source>[Conitzer</source>
          , 2015]
          <string-name>
            <given-names>Vincent</given-names>
            <surname>Conitzer</surname>
          </string-name>
          .
          <article-title>A dutch book against sleeping beauties who are evidential decision theorists</article-title>
          .
          <source>Synthese</source>
          ,
          <volume>192</volume>
          (
          <issue>9</issue>
          ):
          <fpage>2887</fpage>
          -
          <lpage>2899</lpage>
          ,
          <year>10 2015</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref15">
        <mixed-citation>
          <source>[Daley and Sadowski</source>
          , 2017]
          <string-name>
            <given-names>Brendan</given-names>
            <surname>Daley</surname>
          </string-name>
          and
          <string-name>
            <given-names>Philipp</given-names>
            <surname>Sadowski</surname>
          </string-name>
          .
          <article-title>Magical thinking: A representation result</article-title>
          .
          <source>Theoretical Economics</source>
          ,
          <volume>12</volume>
          :
          <fpage>909</fpage>
          -
          <lpage>956</lpage>
          ,
          <year>2017</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref16">
        <mixed-citation>
          <source>[Drescher</source>
          , 2006] Gary
          <string-name>
            <given-names>L.</given-names>
            <surname>Drescher</surname>
          </string-name>
          .
          <article-title>Good and Real - Demystifying Paradoxes from Physics to Ethics</article-title>
          . MIT Press,
          <year>2006</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref17">
        <mixed-citation>
          <source>[Farber</source>
          , 2015]
          <string-name>
            <given-names>Neil</given-names>
            <surname>Farber</surname>
          </string-name>
          .
          <article-title>The surprising psychology of rock-paper-</article-title>
          <string-name>
            <surname>scissors</surname>
          </string-name>
          ,
          <year>4 2015</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref18">
        <mixed-citation>
          [Gauthier, 1989] David Gauthier.
          <article-title>In the neighbourhood of the newcomb-predictor (reflections on rationality)</article-title>
          .
          <source>In Proceedings of the Aristotelian Society</source>
          , New Series,
          <fpage>1988</fpage>
          -
          <lpage>1989</lpage>
          , volume
          <volume>89</volume>
          , pages
          <fpage>179</fpage>
          -
          <lpage>194</lpage>
          .
          <year>1989</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref19">
        <mixed-citation>
          <source>[Gibbard and Harper</source>
          , 1981]
          <string-name>
            <given-names>Allan</given-names>
            <surname>Gibbard</surname>
          </string-name>
          and William L. Harper.
          <article-title>Counterfactuals and two kinds of expected utility</article-title>
          . In William L. Harper, Robert Stalnaker, and Glenn Pearce, editors,
          <source>Ifs. Conditionals</source>
          , Belief, Decision,
          <source>Chance and Time</source>
          , volume
          <volume>15</volume>
          of The University of Western Ontario Series in Philosophy of Science. A Series of Books in Philosophy of Science, Methodology, Epistemology, Logic,
          <source>History of Science, and Related Fields</source>
          , pages
          <fpage>153</fpage>
          -
          <lpage>190</lpage>
          . Springer,
          <year>1981</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref20">
        <mixed-citation>
          <source>[Gilboa and Schmeidler</source>
          , 1989]
          <string-name>
            <given-names>Itzhak</given-names>
            <surname>Gilboa</surname>
          </string-name>
          and
          <string-name>
            <given-names>David</given-names>
            <surname>Schmeidler</surname>
          </string-name>
          .
          <article-title>Maxmin expected utility with non-unique prior</article-title>
          .
          <source>Journal of Mathematical Economics</source>
          ,
          <volume>18</volume>
          :
          <fpage>141</fpage>
          -
          <lpage>153</lpage>
          ,
          <year>1989</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref21">
        <mixed-citation>
          <source>[Gru¨nwald and Halpern</source>
          , 2011]
          <string-name>
            <given-names>Peter</given-names>
            <surname>D. Gru</surname>
          </string-name>
          <article-title>¨nwald</article-title>
          and
          <string-name>
            <surname>Joseph</surname>
            <given-names>Y.</given-names>
          </string-name>
          <string-name>
            <surname>Halpern</surname>
          </string-name>
          .
          <article-title>Making decisions using sets of probabilities: Updating, time consistency, and calibration</article-title>
          .
          <source>Journal of Artificial Intelligence Research</source>
          ,
          <volume>42</volume>
          ,
          <year>2011</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref22">
        <mixed-citation>
          <source>[Hofstadter</source>
          , 1983]
          <string-name>
            <given-names>Douglas</given-names>
            <surname>Hofstadter</surname>
          </string-name>
          .
          <article-title>Dilemmas for superrational thinkers, leading up to a luring lottery</article-title>
          .
          <source>Scientific American</source>
          ,
          <volume>248</volume>
          (
          <issue>6</issue>
          ),
          <year>Jun 1983</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref23">
        <mixed-citation>
          [Ha´jek, 2009]
          <article-title>Alan Ha´jek. Dutch book arguments</article-title>
          .
          <source>In The Handbook of Rational and Social Choice, chapter 7</source>
          . Oxford University Press,
          <year>2009</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref24">
        <mixed-citation>
          <source>[Joyce and Gibbard</source>
          , 1998]
          <string-name>
            <surname>James</surname>
            <given-names>M.</given-names>
          </string-name>
          <string-name>
            <surname>Joyce</surname>
            and
            <given-names>Allan</given-names>
          </string-name>
          <string-name>
            <surname>Gibbard</surname>
          </string-name>
          .
          <article-title>Causal decision theory</article-title>
          .
          <source>In Handbook of Utility Theory</source>
          , Volume
          <volume>1</volume>
          : Principles., chapter
          <volume>13</volume>
          , pages
          <fpage>627</fpage>
          -
          <lpage>666</lpage>
          . Kluwer,
          <year>1998</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref25">
        <mixed-citation>
          <source>[Joyce</source>
          , 1999]
          <string-name>
            <surname>James</surname>
            <given-names>M.</given-names>
          </string-name>
          <string-name>
            <surname>Joyce</surname>
          </string-name>
          .
          <article-title>The Foundations of Causal Decision Theory. Cambridge Studies in Probability, Induction, and Decision Theory</article-title>
          . Cambridge University Press,
          <year>1999</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref26">
        <mixed-citation>
          <source>[Joyce</source>
          , 2012]
          <string-name>
            <surname>James</surname>
            <given-names>M.</given-names>
          </string-name>
          <string-name>
            <surname>Joyce</surname>
          </string-name>
          .
          <article-title>Regret and instability in causal decision theory</article-title>
          .
          <source>Synthese</source>
          ,
          <volume>187</volume>
          :
          <fpage>123</fpage>
          -
          <lpage>145</lpage>
          ,
          <year>2012</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref27">
        <mixed-citation>
          <source>[Lewis</source>
          ,
          <year>1979</year>
          ]
          <string-name>
            <given-names>David</given-names>
            <surname>Lewis</surname>
          </string-name>
          .
          <article-title>Prisoners' dilemma is a newcomb problem</article-title>
          .
          <source>Philosophy &amp; Public Affairs</source>
          ,
          <volume>8</volume>
          (
          <issue>3</issue>
          ):
          <fpage>235</fpage>
          -
          <lpage>240</lpage>
          ,
          <year>1979</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref28">
        <mixed-citation>
          <source>[Masel</source>
          , 2007]
          <string-name>
            <given-names>Joanna</given-names>
            <surname>Masel</surname>
          </string-name>
          .
          <article-title>A bayesian model of quasimagical thinking can explain observed cooperation in the public good game</article-title>
          .
          <source>Journal of Economic Behavior &amp; Organization</source>
          ,
          <volume>64</volume>
          (
          <issue>2</issue>
          ):
          <fpage>216</fpage>
          -
          <lpage>231</lpage>
          ,
          <year>10 2007</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref29">
        <mixed-citation>
          <source>[Meacham</source>
          , 2010]
          <string-name>
            <given-names>Christopher J. G.</given-names>
            <surname>Meacham</surname>
          </string-name>
          .
          <article-title>Binding and its consequences</article-title>
          .
          <source>Philosophical Studies</source>
          ,
          <volume>149</volume>
          (
          <issue>1</issue>
          ):
          <fpage>49</fpage>
          -
          <lpage>71</lpage>
          , 5
          <year>2010</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref30">
        <mixed-citation>
          <source>[Nozick</source>
          , 1969]
          <string-name>
            <given-names>Robert</given-names>
            <surname>Nozick</surname>
          </string-name>
          .
          <article-title>Newcomb's problem and two principles of choice</article-title>
          . In Nicholas Rescher et al., editor, Essays in Honor of Carl G. Hempel, pages
          <fpage>114</fpage>
          -
          <lpage>146</lpage>
          . Springer,
          <year>1969</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref31">
        <mixed-citation>
          <source>[Oesterheld</source>
          , 2019]
          <string-name>
            <given-names>Caspar</given-names>
            <surname>Oesterheld</surname>
          </string-name>
          .
          <article-title>Approval-directed agency and the decision theory of newcomb-like problems</article-title>
          . Synthese,
          <year>2019</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref32">
        <mixed-citation>
          <source>[Parfit</source>
          , 1984]
          <string-name>
            <given-names>Derek</given-names>
            <surname>Parfit</surname>
          </string-name>
          .
          <source>Reasons and Persons</source>
          . Oxford University Press,
          <year>1984</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref33">
        <mixed-citation>
          <source>[Piccione and Rubinstein</source>
          , 1997]
          <string-name>
            <given-names>Michele</given-names>
            <surname>Piccione</surname>
          </string-name>
          and
          <string-name>
            <given-names>Ariel</given-names>
            <surname>Rubinstein</surname>
          </string-name>
          .
          <article-title>On the interpretation of decision problems with imperfect recall</article-title>
          .
          <source>Games and Economic Behavior</source>
          ,
          <volume>20</volume>
          :
          <fpage>3</fpage>
          -
          <lpage>24</lpage>
          ,
          <year>1997</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref34">
        <mixed-citation>
          <source>[Poellinger</source>
          , 2013]
          <string-name>
            <given-names>Roland</given-names>
            <surname>Poellinger</surname>
          </string-name>
          .
          <article-title>Unboxing the concepts in newcomb's paradox: Causation, prediction, decision</article-title>
          .
          <year>2013</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref35">
        <mixed-citation>
          <source>[Rabinowicz</source>
          , 2000]
          <string-name>
            <given-names>Wlodek</given-names>
            <surname>Rabinowicz</surname>
          </string-name>
          .
          <article-title>Money pump with foresight</article-title>
          .
          <source>In Michael J</source>
          . Almeida, editor,
          <source>Imperceptible Harms and Benefits</source>
          , pages
          <fpage>123</fpage>
          -
          <lpage>154</lpage>
          . Springer,
          <year>2000</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref36">
        <mixed-citation>
          <source>[Rabinowicz</source>
          , 2008]
          <string-name>
            <given-names>Wlodek</given-names>
            <surname>Rabinowicz</surname>
          </string-name>
          .
          <article-title>Pragmatic Arguments for Rationality Constraints</article-title>
          , pages
          <fpage>139</fpage>
          -
          <lpage>163</lpage>
          . Reasoning, Rationality and Probability.
          <source>CSLI Publications</source>
          ,
          <year>2008</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref37">
        <mixed-citation>
          <source>[Schwarz</source>
          , 2015]
          <string-name>
            <given-names>Wolfgang</given-names>
            <surname>Schwarz</surname>
          </string-name>
          .
          <article-title>Lost memories and useless coins: revisiting the absentminded driver</article-title>
          .
          <source>Synthese</source>
          ,
          <volume>192</volume>
          :
          <fpage>3011</fpage>
          -
          <lpage>3036</lpage>
          ,
          <year>2015</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref38">
        <mixed-citation>
          <source>[Shafir and Tversky</source>
          , 1992]
          <string-name>
            <given-names>Eldar</given-names>
            <surname>Shafir</surname>
          </string-name>
          and
          <string-name>
            <given-names>Amos</given-names>
            <surname>Tversky</surname>
          </string-name>
          .
          <article-title>Thinking through uncertainty: Nonconsequential reasoning and choice</article-title>
          .
          <source>Cognitive Psychology</source>
          ,
          <volume>24</volume>
          (
          <issue>4</issue>
          ):
          <fpage>449</fpage>
          -
          <lpage>474</lpage>
          ,
          <year>1992</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref39">
        <mixed-citation>
          <source>[Skalse</source>
          , 2018]
          <string-name>
            <given-names>Joar</given-names>
            <surname>Skalse</surname>
          </string-name>
          .
          <article-title>A counterexample to perfect decision theories and a possible response</article-title>
          .
          <year>2018</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref40">
        <mixed-citation>
          <source>[Skyrms</source>
          , 1993]
          <string-name>
            <given-names>Brian</given-names>
            <surname>Skyrms</surname>
          </string-name>
          .
          <article-title>A mistake in dynamic coherence arguments?</article-title>
          <source>Philosophy of Science</source>
          ,
          <volume>60</volume>
          :
          <fpage>320</fpage>
          -
          <lpage>328</lpage>
          , 6
          <year>1993</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref41">
        <mixed-citation>
          <source>[Soares and Fallenstein</source>
          , 2014]
          <string-name>
            <given-names>Nate</given-names>
            <surname>Soares</surname>
          </string-name>
          and
          <string-name>
            <given-names>Benja</given-names>
            <surname>Fallenstein</surname>
          </string-name>
          .
          <article-title>Toward idealized decision theory</article-title>
          .
          <source>Technical Report 2014-7, Machine Intelligence Research Institute</source>
          ,
          <year>2014</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref42">
        <mixed-citation>
          <source>[Soares and Levinstein</source>
          , 2017]
          <string-name>
            <given-names>Nate</given-names>
            <surname>Soares</surname>
          </string-name>
          and
          <string-name>
            <given-names>Benjamin A.</given-names>
            <surname>Levinstein</surname>
          </string-name>
          .
          <article-title>Cheating death in damascus</article-title>
          .
          <source>In Formal Epistemology Workshop (FEW)</source>
          <year>2017</year>
          , University of Washington, Seattle, USA, 5
          <year>2017</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref43">
        <mixed-citation>
          <source>[Spencer and Wells</source>
          , 2017]
          <string-name>
            <given-names>Jack</given-names>
            <surname>Spencer</surname>
          </string-name>
          and
          <string-name>
            <given-names>Ian</given-names>
            <surname>Wells</surname>
          </string-name>
          .
          <source>Why take both boxes? Philosophy and Phenomenological Research</source>
          ,
          <year>2017</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref44">
        <mixed-citation>
          <source>[Spohn</source>
          , 2012]
          <string-name>
            <given-names>Wolfgang</given-names>
            <surname>Spohn</surname>
          </string-name>
          .
          <article-title>Reversing 30 years of discussion: why causal decision theorists should one-box</article-title>
          .
          <source>Synthese</source>
          ,
          <volume>187</volume>
          (
          <issue>1</issue>
          ):
          <fpage>95</fpage>
          -
          <lpage>122</lpage>
          ,
          <year>2012</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref45">
        <mixed-citation>
          [Steele and Stefa´nsson, 2016]
          <string-name>
            <given-names>Katie</given-names>
            <surname>Steele and H. Orri</surname>
          </string-name>
          <article-title>Stefa´nsson</article-title>
          .
          <source>Decision theory</source>
          .
          <year>2016</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref46">
        <mixed-citation>
          <source>[Vineberg</source>
          , 2016]
          <string-name>
            <given-names>Susan</given-names>
            <surname>Vineberg</surname>
          </string-name>
          .
          <article-title>Dutch book arguments</article-title>
          . In Edward N. Zalta, editor,
          <source>The Stanford Encyclopedia of Philosophy</source>
          . Metaphysics Research Lab, Stanford University, spring
          <year>2016</year>
          edition,
          <year>2016</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref47">
        <mixed-citation>
          <source>[Weirich</source>
          , 1985]
          <string-name>
            <given-names>Paul</given-names>
            <surname>Weirich</surname>
          </string-name>
          .
          <article-title>Decision instability</article-title>
          .
          <source>Australasian Journal of Philosophy</source>
          ,
          <volume>63</volume>
          (
          <issue>4</issue>
          ):
          <fpage>465</fpage>
          -
          <lpage>472</lpage>
          ,
          <year>1985</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref48">
        <mixed-citation>
          <source>[Weirich</source>
          , 2016]
          <string-name>
            <given-names>Paul</given-names>
            <surname>Weirich</surname>
          </string-name>
          .
          <article-title>Causal decision theory</article-title>
          .
          <source>In The Stanford Encyclopedia of Philosophy. Spring 2016 edition</source>
          ,
          <year>2016</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref49">
        <mixed-citation>
          <source>[Yudkowsky and Soares</source>
          , 2018]
          <string-name>
            <given-names>Eliezer</given-names>
            <surname>Yudkowsky</surname>
          </string-name>
          and
          <string-name>
            <given-names>Nate</given-names>
            <surname>Soares</surname>
          </string-name>
          .
          <article-title>Functional decision theory: A new theory of instrumental rationality,</article-title>
          <year>5 2018</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref50">
        <mixed-citation>
          <source>[Yudkowsky</source>
          , 2010]
          <string-name>
            <given-names>Eliezer</given-names>
            <surname>Yudkowsky</surname>
          </string-name>
          .
          <source>Timeless decision theory</source>
          ,
          <year>2010</year>
          .
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>