<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta>
      <journal-title-group>
        <journal-title>Corresponding author.
$ ecavenaghi@unibz.it (E. Cavenaghi); alessio.zanga@unimib.it (A. Zanga); fabio.stella@unimib.it (F. Stella);
Markus.Zanker@unibz.it (M. Zanker)</journal-title>
      </journal-title-group>
    </journal-meta>
    <article-meta>
      <title-group>
        <article-title>The Importance of Causality in Decision Making: A Perspective on Recommender Systems</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Emanuele Cavenaghi</string-name>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Alessio Zanga</string-name>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Fabio Stella</string-name>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Markus Zanker</string-name>
        </contrib>
      </contrib-group>
      <pub-date>
        <year>2024</year>
      </pub-date>
      <volume>000</volume>
      <fpage>0</fpage>
      <lpage>0002</lpage>
      <abstract>
        <p>Causality is receiving increasing attention from the artificial intelligence and machine learning communities. Similarly, growing attention to causality is currently going on in the Recommendation Systems (RSs) community, which has realised that RSs could greatly benefit from causality to transform accurate predictions into efective and explainable decisions. Indeed, the RS literature has repeatedly highlighted that, in real-world scenarios, recommendation algorithms sufer many types of biases since assumptions ensuring unbiasedness are likely not met. In this discussion paper, we formulate the RS problem in terms of causality, using potential outcomes and structural causal models, by giving formal definitions of the causal quantities to be estimated and a general causal graph to serve as a reference to foster future research and development.</p>
      </abstract>
      <kwd-group>
        <kwd>eol&gt;Causal Models</kwd>
        <kwd>Decision Making</kwd>
        <kwd>Recommender Systems</kwd>
      </kwd-group>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>1. Introduction</title>
      <p>
        Predicting and deciding are two fundamentally diferent tasks. As described by the Ladder of Causation
[
        <xref ref-type="bibr" rid="ref1">1</xref>
        ], a decision manipulates the system which can react to our decision, while a prediction does not
afect the system in any manner: the system is eventually afected only when we exploit the prediction
to make a decision. Overlooking this diference usually leads to biased predictions that, in turn, result
in wrong decisions. In this sense, the RSs community is facing several problems with biased estimates
[
        <xref ref-type="bibr" rid="ref2">2</xref>
        ] to assess the efect of recommendations based on predictions. Indeed, according to [
        <xref ref-type="bibr" rid="ref3 ref4">3, 4</xref>
        ], the
recommendation problem is usually framed as a prediction problem while, as pointed out in [
        <xref ref-type="bibr" rid="ref5 ref6">5, 6</xref>
        ], it
is indeed a decision-making problem, since we have to decide which item(s) to recommend to which
user(s).
      </p>
      <p>
        Furthermore, human beings are not interested in mere correlations but in understanding the actual
causes of the efects, manipulating the world to achieve the desired outcome. In fact, scientists are
familiar with the phrase: “Correlation is not causation”, that is, for example, “the rooster’s crow is highly
correlated with the sunrise; yet it does not cause the sunrise” [
        <xref ref-type="bibr" rid="ref1">1</xref>
        ]. Indeed, under general conditions,
machine learning approaches do not allow to state that  is the cause of  but only that they are
“correlated” or “associated” to each other.
      </p>
      <p>
        This is why causality becomes important: we need a way to translate cause-and-efect relations and
interventions on a system using a mathematical formulation. To this end, in [
        <xref ref-type="bibr" rid="ref6">6</xref>
        ], we proposed a causal
decision-making framework for RSs using Potential Outcomes (POs) [
        <xref ref-type="bibr" rid="ref7">7</xref>
        ] to define causal estimands
of interest and Causal Graphs (CGs) [
        <xref ref-type="bibr" rid="ref8">8</xref>
        ] to propose a general probabilistic graphical model for RSs to
encode the cause-and-efect relations among variables. Using this framework, we introduce the process,
illustrated in Figure 1, which allows to make decisions by combining data and expert knowledge.
      </p>
    </sec>
    <sec id="sec-2">
      <title>2. Making Decisions Through Causality</title>
      <sec id="sec-2-1">
        <title>2.1. Causal Discovery</title>
        <p>
          The first step is to have a CG that describes the data-generating process of the system under study. The
CG can be learned by combining observational data with experts’ knowledge through a process called
causal discovery, which is enabled by several causal discovery algorithms [
          <xref ref-type="bibr" rid="ref10 ref9">9, 10</xref>
          ]. While the CG must be
learned in each scenario, we proposed a reference CG for RSs [
          <xref ref-type="bibr" rid="ref6">6</xref>
          ] to guide the construction of a CG for
specific RSs problems as done in [
          <xref ref-type="bibr" rid="ref11 ref12">11, 12</xref>
          ].
        </p>
        <p>U</p>
        <p>I</p>
        <p>C</p>
        <p>The problem of recommending a single item with features I to a user U in context C is described by
the CG of Figure 2 where the node  represents the action of recommending an item whose domain
corresponds to the item set, i.e.,  ∈ (). For example, in the context of film recommendation,
() is the catalogue of the films and our recommendation  is one of the films in the catalogue. To
decide which item to recommend to the current user U in context C, we use a policy   based on user
and context features. Once we decide which item  to recommend, the corresponding item’s features I
are fixed and they mediate the efect of our recommendation  on the user feedback  through the
path  → I →  . For example, once we decide to recommend a film, its genre is fixed (  → ),
and the genre (likely) afects a user’s feedback (  →  ). It is worth noticing that not all the item’s
features have to influence the user’s feedback, i.e., some item’s features are not taken into account by
the users. On the other hand, part of the efect of the recommendation  on the user’s feedback  may
not be captured by the modelled features I and flows directly through the edge  →  . For example, if
we are not able to model the film’s popularity, since it is dificult to know or to model it, the efect of
the film’s popularity will flow through the edge  →  as this feature is not included in our model.</p>
      </sec>
      <sec id="sec-2-2">
        <title>2.2. Causal Estimand Identification</title>
        <p>
          To exploit the potential of causality, we should frame the quantity to estimate as a causal estimand that
encodes the notion of the causal efect of a variable (the cause) on another (the efect). Generally, we
can define it, using the POs framework and the do-operator [
          <xref ref-type="bibr" rid="ref13 ref14">13, 14</xref>
          ], as E[ |( = ), u, i, c]. This
encodes the value of the expected feedback  given by the user u in context c when we recommend
item  with features i. The diference that separates causal estimands from classical statistical estimands
is the presence of the so-called do-operator, denoted with ( = ), that defines the intervention of
ifxing the value of  to  for the whole population of users. In contrast, conditioning on  =  means
that  takes a value  naturally, which simply translates to focusing only on the sub-population where
X has been observed to be equal to . In a decision-making problem, such as RSs, we are interested in
estimating causal estimands since we actively decide which item(s) to recommend.
        </p>
        <p>
          However, expressions with the do-operator can only be estimated in controlled experiments where
the variables in the do-terms can be appropriately controlled. To estimate a causal estimand using only
observational data, it is necessary to remove the do-terms and obtain an equivalent expression. To this
end, the adjustment formula estimator [
          <xref ref-type="bibr" rid="ref14">14</xref>
          ] adopts a model-based approach to adjust for an adjustment
set Z and obtain a statistical estimand:
 ( = |( = )) = ∑︁  ( = | = , Z = z) (Z = z)
z
(1)
        </p>
        <p>
          To identify the variables that must be included in Z, we can query the CG by evaluating an
identification criterion, such as the backdoor criterion [
          <xref ref-type="bibr" rid="ref14">14</xref>
          ], frontdoor criterion [
          <xref ref-type="bibr" rid="ref8">8</xref>
          ] or do-calculus [
          <xref ref-type="bibr" rid="ref13">13</xref>
          ]. In particular,
if no identification is possible with do-calculus, the causal efect is guaranteed to be unidentifiable . Thus,
every estimate of the causal estimand will be biased.
        </p>
      </sec>
      <sec id="sec-2-3">
        <title>2.3. Estimation</title>
        <p>Once we have proved the identifiability of the causal estimand, i.e., once we have shown that the causal
estimand is equal to a statistical estimand, this can be estimated using classical statistical estimators. To
this end, any model that is compatible with the type of the outcome variable, e.g. linear regression for a
continuous outcome or neural networks for non-linear relations, is suitable for this estimation. Clearly,
the model should be chosen carefully for each problem by considering the available data characteristics
to avoid estimation errors.</p>
      </sec>
      <sec id="sec-2-4">
        <title>2.4. Making Decisions</title>
        <p>
          Finally, with the estimated causal efects, e.g., the efect of our recommendation on the user’s propensity
to click on the recommended item(s), we can decide which items to recommend. This could be done
in diferent ways: (i) greedy, (ii)  -greedy and (iii) more sophisticated policies. To this end, in recent
years, several works have exploited causality by linking it to Multi-Armed Bandit (MAB) [
          <xref ref-type="bibr" rid="ref15 ref16 ref17">15, 16, 17</xref>
          ] and
Reinforcement Learning (RL) [
          <xref ref-type="bibr" rid="ref18">18</xref>
          ]. For example, in [
          <xref ref-type="bibr" rid="ref19">19</xref>
          ], the authors define the notion of Possibly-Optimal
Minimal Intervention Set with the idea of determining the minimum set of variables on which a MAB
agent should intervene to understand all the possible arms that are worth intervening on. Moreover,
[
          <xref ref-type="bibr" rid="ref20">20</xref>
          ] extends the method by considering that some variables can not be manipulated. Using causality
with RL, [
          <xref ref-type="bibr" rid="ref21">21, 22</xref>
          ] approached the Dynamic Treatment Regimes problem with confounded observational
dataset.
        </p>
      </sec>
    </sec>
    <sec id="sec-3">
      <title>3. Conclusions</title>
      <p>In this paper, we proposed a causal view of the RS problem and highlighted the importance of framing
the recommendation problem in terms of causality. The causality framework can, in our view, be
considered as a single framework allowing researchers to wholistically define and address several
problems widely acknowledged in the RSs community to bridge the gaps in future works.</p>
      <p>However, we would like to stress that causality is not magic but ruthlessly honest and, diferently
from other approaches, it makes explicit assumptions, such as ignorability and unconfoundedness,
leaving us with the burden of judging whether they are likely to be satisfied for the addressed context.
Indeed, causality is not the sole ingredient to solve the RS problem while we are fully convinced that
exploiting the body of knowledge generated over more than 30 years of research in RSs and users’
behaviour remains fundamental.
[22] J. Zhang, E. Bareinboim, Near-optimal reinforcement learning in dynamic treatment regimes, in:
H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché-Buc, E. Fox, R. Garnett (Eds.), Advances in
Neural Information Processing Systems, volume 32, Curran Associates, Inc., Vancouver, Canada,
2019.</p>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          [1]
          <string-name>
            <given-names>J.</given-names>
            <surname>Pearl</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D.</given-names>
            <surname>Mackenzie</surname>
          </string-name>
          ,
          <article-title>The book of why: the new science of cause and efect, Basic books</article-title>
          , New York, United States,
          <year>2018</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          [2]
          <string-name>
            <given-names>J.</given-names>
            <surname>Chen</surname>
          </string-name>
          ,
          <string-name>
            <given-names>H.</given-names>
            <surname>Dong</surname>
          </string-name>
          ,
          <string-name>
            <given-names>X.</given-names>
            <surname>Wang</surname>
          </string-name>
          ,
          <string-name>
            <given-names>F.</given-names>
            <surname>Feng</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Wang</surname>
          </string-name>
          ,
          <string-name>
            <given-names>X.</given-names>
            <surname>He</surname>
          </string-name>
          ,
          <article-title>Bias and debias in recommender system: A survey and future directions</article-title>
          ,
          <source>ACM Trans. Inf. Syst</source>
          .
          <volume>41</volume>
          (
          <year>2023</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          [3]
          <string-name>
            <given-names>G.</given-names>
            <surname>Adomavicius</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Tuzhilin</surname>
          </string-name>
          ,
          <article-title>Toward the next generation of recommender systems: a survey of the state-of-the-art and possible extensions</article-title>
          ,
          <source>IEEE Transactions on Knowledge and Data Engineering</source>
          <volume>17</volume>
          (
          <year>2005</year>
          )
          <fpage>734</fpage>
          -
          <lpage>749</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          [4]
          <string-name>
            <given-names>F.</given-names>
            <surname>Ricci</surname>
          </string-name>
          ,
          <string-name>
            <given-names>L.</given-names>
            <surname>Rokach</surname>
          </string-name>
          ,
          <string-name>
            <given-names>B.</given-names>
            <surname>Shapira</surname>
          </string-name>
          ,
          <article-title>Introduction to recommender systems handbook</article-title>
          , in: Recommender systems handbook, Springer, Boston, MA,
          <year>2011</year>
          , pp.
          <fpage>1</fpage>
          -
          <lpage>35</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          [5]
          <string-name>
            <given-names>O.</given-names>
            <surname>Jeunen</surname>
          </string-name>
          ,
          <string-name>
            <given-names>B.</given-names>
            <surname>Goethals</surname>
          </string-name>
          ,
          <article-title>Pessimistic decision-making for recommender systems</article-title>
          ,
          <source>ACM Trans. Recomm. Syst</source>
          .
          <volume>1</volume>
          (
          <year>2022</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          [6]
          <string-name>
            <given-names>E.</given-names>
            <surname>Cavenaghi</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Zanga</surname>
          </string-name>
          ,
          <string-name>
            <given-names>F.</given-names>
            <surname>Stella</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Zanker</surname>
          </string-name>
          ,
          <article-title>Towards a causal decision-making framework for recommender systems</article-title>
          ,
          <source>ACM Trans. Recomm. Syst</source>
          .
          <volume>2</volume>
          (
          <year>2024</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          [7]
          <string-name>
            <given-names>D. B.</given-names>
            <surname>Rubin</surname>
          </string-name>
          ,
          <article-title>Estimating causal efects of treatments in randomized and nonrandomized studies</article-title>
          .,
          <source>Journal of educational Psychology</source>
          <volume>66</volume>
          (
          <year>1974</year>
          )
          <fpage>688</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          [8]
          <string-name>
            <given-names>J.</given-names>
            <surname>Pearl</surname>
          </string-name>
          , Causality, Cambridge university press, Cambridge, United Kingdom,
          <year>2009</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          [9]
          <string-name>
            <given-names>M. J.</given-names>
            <surname>Vowels</surname>
          </string-name>
          ,
          <string-name>
            <given-names>N. C.</given-names>
            <surname>Camgoz</surname>
          </string-name>
          ,
          <string-name>
            <given-names>R.</given-names>
            <surname>Bowden</surname>
          </string-name>
          ,
          <string-name>
            <surname>D'</surname>
          </string-name>
          <article-title>ya like dags? a survey on structure learning and causal discovery</article-title>
          ,
          <source>ACM Computing Surveys (CSUR)</source>
          (
          <year>2021</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          [10]
          <string-name>
            <given-names>A.</given-names>
            <surname>Zanga</surname>
          </string-name>
          ,
          <string-name>
            <given-names>E.</given-names>
            <surname>Ozkirimli</surname>
          </string-name>
          ,
          <string-name>
            <given-names>F.</given-names>
            <surname>Stella</surname>
          </string-name>
          ,
          <article-title>A survey on causal discovery: Theory and practice</article-title>
          ,
          <source>International Journal of Approximate Reasoning</source>
          <volume>151</volume>
          (
          <year>2022</year>
          )
          <fpage>101</fpage>
          -
          <lpage>129</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref11">
        <mixed-citation>
          [11]
          <string-name>
            <given-names>E.</given-names>
            <surname>Cavenaghi</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Zanga</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Rimoldi</surname>
          </string-name>
          ,
          <string-name>
            <given-names>P.</given-names>
            <surname>Minasi</surname>
          </string-name>
          ,
          <string-name>
            <given-names>F.</given-names>
            <surname>Stella</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Zanker</surname>
          </string-name>
          ,
          <article-title>Causal discovery in recommender systems: a case study in online hotel search 1 (</article-title>
          <year>2023</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref12">
        <mixed-citation>
          [12]
          <string-name>
            <given-names>E.</given-names>
            <surname>Cavenaghi</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Zanga</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Rimoldi</surname>
          </string-name>
          ,
          <string-name>
            <given-names>P.</given-names>
            <surname>Minasi</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Zanker</surname>
          </string-name>
          ,
          <string-name>
            <given-names>F.</given-names>
            <surname>Stella</surname>
          </string-name>
          ,
          <article-title>Analysis of relevant factors in online hotel recommendation through causal models 1 (</article-title>
          <year>2023</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref13">
        <mixed-citation>
          [13]
          <string-name>
            <given-names>E.</given-names>
            <surname>Bareinboim</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J. D.</given-names>
            <surname>Correa</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D.</given-names>
            <surname>Ibeling</surname>
          </string-name>
          , T. Icard,
          <article-title>On pearl's hierarchy and the foundations of causal inference, in: Probabilistic and Causal Inference: The Works of Judea Pearl, Association for Computing Machinery</article-title>
          , New York, NY, USA,
          <year>2022</year>
          , p.
          <fpage>507</fpage>
          -
          <lpage>556</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref14">
        <mixed-citation>
          [14]
          <string-name>
            <given-names>M.</given-names>
            <surname>Glymour</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J.</given-names>
            <surname>Pearl</surname>
          </string-name>
          ,
          <string-name>
            <given-names>N. P.</given-names>
            <surname>Jewell</surname>
          </string-name>
          , Causal inference in statistics: A primer, John Wiley &amp; Sons, Hoboken, United States,
          <year>2016</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref15">
        <mixed-citation>
          [15]
          <string-name>
            <given-names>F.</given-names>
            <surname>Lattimore</surname>
          </string-name>
          ,
          <string-name>
            <given-names>T.</given-names>
            <surname>Lattimore</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M. D.</given-names>
            <surname>Reid</surname>
          </string-name>
          ,
          <article-title>Causal bandits: Learning good interventions via causal inference</article-title>
          , in: D.
          <string-name>
            <surname>Lee</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          <string-name>
            <surname>Sugiyama</surname>
            ,
            <given-names>U.</given-names>
          </string-name>
          <string-name>
            <surname>Luxburg</surname>
            ,
            <given-names>I.Guyon</given-names>
          </string-name>
          , R.Garnett (Eds.),
          <source>Advances in Neural Information Processing Systems</source>
          , volume
          <volume>29</volume>
          ,
          <string-name>
            <surname>Curran</surname>
            <given-names>Associates</given-names>
          </string-name>
          , Inc.,
          <string-name>
            <surname>Barcelona</surname>
          </string-name>
          , Spain,
          <year>2016</year>
          , p.
          <fpage>9</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref16">
        <mixed-citation>
          [16]
          <string-name>
            <given-names>Y.</given-names>
            <surname>Lu</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Meisami</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Tewari</surname>
          </string-name>
          , W. Yan,
          <article-title>Regret analysis of bandit problems with causal background knowledge</article-title>
          , in: J.
          <string-name>
            <surname>Peters</surname>
          </string-name>
          , D. Sontag (Eds.),
          <source>Proceedings of the 36th Conference on Uncertainty in Artificial Intelligence (UAI)</source>
          , volume
          <volume>124</volume>
          <source>of Proceedings of Machine Learning Research</source>
          , PMLR,
          <string-name>
            <surname>Virtual</surname>
            <given-names>Event</given-names>
          </string-name>
          ,
          <year>2020</year>
          , pp.
          <fpage>141</fpage>
          -
          <lpage>150</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref17">
        <mixed-citation>
          [17]
          <string-name>
            <given-names>V.</given-names>
            <surname>Nair</surname>
          </string-name>
          ,
          <string-name>
            <given-names>V.</given-names>
            <surname>Patil</surname>
          </string-name>
          , G. Sinha,
          <article-title>Budgeted and non-budgeted causal bandits</article-title>
          , in: A.
          <string-name>
            <surname>Banerjee</surname>
            ,
            <given-names>K.</given-names>
          </string-name>
          Fukumizu (Eds.),
          <source>Proceedings of The 24th International Conference on Artificial Intelligence and Statistics</source>
          , volume
          <volume>130</volume>
          <source>of Proceedings of Machine Learning Research</source>
          , PMLR,
          <string-name>
            <surname>Virtual</surname>
            <given-names>Event</given-names>
          </string-name>
          ,
          <year>2021</year>
          , pp.
          <fpage>2017</fpage>
          -
          <lpage>2025</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref18">
        <mixed-citation>
          [18]
          <string-name>
            <given-names>Y.</given-names>
            <surname>Lu</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Meisami</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Tewari</surname>
          </string-name>
          ,
          <article-title>Causal markov decision processes: Learning good interventions eficiently</article-title>
          ,
          <source>arXiv preprint arXiv:2102.07663 1</source>
          (
          <year>2021</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref19">
        <mixed-citation>
          [19]
          <string-name>
            <given-names>S.</given-names>
            <surname>Lee</surname>
          </string-name>
          ,
          <string-name>
            <given-names>E.</given-names>
            <surname>Bareinboim</surname>
          </string-name>
          , Structural causal bandits: Where to intervene?, in: S. Bengio,
          <string-name>
            <given-names>H.</given-names>
            <surname>Wallach</surname>
          </string-name>
          ,
          <string-name>
            <given-names>H.</given-names>
            <surname>Larochelle</surname>
          </string-name>
          ,
          <string-name>
            <given-names>K.</given-names>
            <surname>Grauman</surname>
          </string-name>
          ,
          <string-name>
            <given-names>N.</given-names>
            <surname>Cesa-Bianchi</surname>
          </string-name>
          , R. Garnett (Eds.),
          <source>Advances in Neural Information Processing Systems</source>
          , volume
          <volume>31</volume>
          ,
          <string-name>
            <surname>Curran</surname>
            <given-names>Associates</given-names>
          </string-name>
          , Inc.,
          <string-name>
            <surname>Montréal</surname>
          </string-name>
          , Canada,
          <year>2018</year>
          , p.
          <fpage>11</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref20">
        <mixed-citation>
          [20]
          <string-name>
            <given-names>S.</given-names>
            <surname>Lee</surname>
          </string-name>
          ,
          <string-name>
            <given-names>E.</given-names>
            <surname>Bareinboim</surname>
          </string-name>
          ,
          <article-title>Structural causal bandits with non-manipulable variables</article-title>
          ,
          <source>Proceedings of the AAAI Conference on Artificial Intelligence</source>
          <volume>33</volume>
          (
          <year>2019</year>
          )
          <fpage>4164</fpage>
          -
          <lpage>4172</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref21">
        <mixed-citation>
          [21]
          <string-name>
            <surname>J. Zhang,</surname>
          </string-name>
          <article-title>Designing optimal dynamic treatment regimes: A causal reinforcement learning approach</article-title>
          , in: H.
          <string-name>
            <surname>D. III</surname>
          </string-name>
          , A. Singh (Eds.),
          <source>Proceedings of the 37th International Conference on Machine Learning</source>
          , volume
          <volume>119</volume>
          <source>of Proceedings of Machine Learning Research</source>
          , PMLR, Vienna, Austria,
          <year>2020</year>
          , pp.
          <fpage>11012</fpage>
          -
          <lpage>11022</lpage>
          .
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>