<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta />
    <article-meta>
      <title-group>
        <article-title>Careful Explanations: A Feminist Perspective on XAI</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Laura State</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
          <xref ref-type="aff" rid="aff2">2</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Miriam Fahimi</string-name>
          <xref ref-type="aff" rid="aff1">1</xref>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          <institution>Department of Computer Science, University of Pisa</institution>
          ,
          <addr-line>Largo B. Pontecorvo, 3, 56127 Pisa</addr-line>
          ,
          <country country="IT">Italy</country>
        </aff>
        <aff id="aff1">
          <label>1</label>
          <institution>Digital Age Research Center (D</institution>
        </aff>
        <aff id="aff2">
          <label>2</label>
          <institution>Scuola Normale Superiore</institution>
          ,
          <addr-line>Piazza dei Cavalieri, 7, 56126 Pisa</addr-line>
          ,
          <country country="IT">Italy</country>
        </aff>
      </contrib-group>
      <abstract>
        <p>Explainable artificial intelligence (XAI) is a rapidly growing research field that has received a lot of attention during the last few years. An important goal of the field is to use its methods to detect (social) bias and discrimination. Despite these positive intentions, aspects of XAI can be in conflict with feminist approaches and values. Therefore, our conceptual contribution brings forward both a careful assessment of current XAI methods, as well as visions for carefully doing XAI from a feminist perspective. We conclude with a discussion on the possibilities for caring XAI, and the challenges that might lie along the way.</p>
      </abstract>
      <kwd-group>
        <kwd>eol&gt;explainable AI</kwd>
        <kwd>feminism</kwd>
        <kwd>care work</kwd>
      </kwd-group>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>-</title>
      <p>relations and how (structural) discrimination emerges through them; 2) a special focus on the
relationship between gender identities, and the necessity to think beyond the gender binary;
3) acknowledging care and care work in our society, its (often intentional) invisibility, and
implications for the redestribution of this work.</p>
      <p>
        Following these key aspects of feminism, and central work that is already existing at the
intersection between feminism, feminist epistemology, care work, data and AI [
        <xref ref-type="bibr" rid="ref10 ref11 ref12 ref13 ref14 ref15 ref8 ref9">8, 9, 10, 11, 12,
13, 14, 15, 16</xref>
        ], we seek to critically assess and contribute towards a feminist perspective on XAI.
      </p>
      <p>To do this, our work is asking the following two research questions (RQ): ) How can we
assess current methods of XAI from a feminist viewpoint? and ) How do we imagine future
versions of XAI, and which changes to the current methodologies does this entail?</p>
      <p>We can dissect RQ 1) using a couple of follow-up questions: Which aspects of existing XAI
methods interfere with our working definition of feminism? How does existing XAI interact
with power relations: do they perpetuate, or foster to challenge them? Which aspects of existing
XAI methods can be considered feminist? Why? Regarding the RQ 2), we will ask questions
such as: how could a feminist explanation look like? Which technical and social conditions are
needed along the way? How do we imagine a feminist future towards caring XAI?</p>
      <p>Answering the posed questions is an experimental and explorative endeavor, set out to
forward a meaningful and critical contribution to the XAI community over techno-determinist
solutionism [17]. As we will give answers to these questions, the issues that are raised might
not be solved. Instead, they allow for further questioning and research.</p>
      <p>This work in progress is of interdisciplinary character, bringing together perspectives from
the social sciences, the computer sciences, and our shared interest in feminist theories, concepts
and methodologies. While it is important to understand the global dimension of XAI and AI,
we acknowledge our positionality as researchers that are based in Europe, and with European
institutions.</p>
      <p>Outlook A feminist perspective is an important and necessary addition to XAI and AI - it
brings a view to the table that is historically neglected or marginalized and ofers possibilities of
”studying up” instead of ”studying down” [18, 19]. This is especially important if we reason about
(social and historical) discrimination, and about how this discrimination can be counter-acted.</p>
      <p>If we do not integrate a feminist perspective, we will build a world of (X)AI that leaves some
people outside of it.</p>
      <p>Acknowledgments
This work has received funding from the European Union’s Horizon 2020 research and
innovation programme under Marie Sklodowska-Curie Actions (grant agreement number 860630) for
the project “NoBIAS - Artificial Intelligence without Bias” ( nobias-project.eu). This work reflects
only the authors’ views and the European Research Executive Agency (REA) is not responsible
for any use that may be made of the information it contains.
IEEE Transactions on Cognitive and Developmental Systems (2020) 1–1. doi:10.1109/TCDS.
2020.3044366, conference Name: IEEE Transactions on Cognitive and Developmental
Systems.
[16] A. Kasirzadeh, Reasons, Values, Stakeholders: A Philosophical Framework for
Explainable Artificial Intelligence, in: Proceedings of the 2021 ACM Conference on
Fairness, Accountability, and Transparency, FAccT ’21, Association for Computing
Machinery, New York, NY, USA, 2021, p. 14. URL: http://doi.org/10.1145/3442188.3445866.
doi:10.1145/3442188.3445866.
[17] E. Morozov, To Save Everything, Click Here: The Folly of Technological Solutionism,</p>
      <p>PublicAfairs, New York, 2013.
[18] L. Nader, Up the Anthropologist: Perspectives Gained From Studying Up, Technical Report,
1972. URL: https://eric.ed.gov/?id=ED065375, eRIC Number: ED065375.
[19] C. Barabas, C. Doyle, J. Rubinovitz, K. Dinakar, Studying up: reorienting the study of
algorithmic fairness around issues of power, in: Proceedings of the 2020 Conference
on Fairness, Accountability, and Transparency, FAT* ’20, Association for Computing
Machinery, New York, NY, USA, 2020, pp. 167–176. URL: https://dl.acm.org/doi/10.1145/
3351095.3372859. doi:10.1145/3351095.3372859.</p>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          [1]
          <string-name>
            <given-names>A. B.</given-names>
            <surname>Arrieta</surname>
          </string-name>
          ,
          <string-name>
            <given-names>N. D.</given-names>
            <surname>Rodríguez</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J. D.</given-names>
            <surname>Ser</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Bennetot</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S.</given-names>
            <surname>Tabik</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Barbado</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S.</given-names>
            <surname>García</surname>
          </string-name>
          , S. GilLopez, D. Molina,
          <string-name>
            <given-names>R.</given-names>
            <surname>Benjamins</surname>
          </string-name>
          ,
          <string-name>
            <given-names>R.</given-names>
            <surname>Chatila</surname>
          </string-name>
          ,
          <string-name>
            <given-names>F.</given-names>
            <surname>Herrera</surname>
          </string-name>
          ,
          <article-title>Explainable artificial intelligence (XAI): concepts, taxonomies, opportunities and challenges toward responsible AI, Inf</article-title>
          .
          <source>Fusion</source>
          <volume>58</volume>
          (
          <year>2020</year>
          )
          <fpage>82</fpage>
          -
          <lpage>115</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          [2]
          <string-name>
            <given-names>E.</given-names>
            <surname>Ntoutsi</surname>
          </string-name>
          ,
          <string-name>
            <given-names>P.</given-names>
            <surname>Fafalios</surname>
          </string-name>
          ,
          <string-name>
            <given-names>U.</given-names>
            <surname>Gadiraju</surname>
          </string-name>
          ,
          <string-name>
            <given-names>V.</given-names>
            <surname>Iosifidis</surname>
          </string-name>
          ,
          <string-name>
            <given-names>W.</given-names>
            <surname>Nejdl</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Vidal</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S.</given-names>
            <surname>Ruggieri</surname>
          </string-name>
          ,
          <string-name>
            <given-names>F.</given-names>
            <surname>Turini</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S.</given-names>
            <surname>Papadopoulos</surname>
          </string-name>
          ,
          <string-name>
            <given-names>E.</given-names>
            <surname>Krasanakis</surname>
          </string-name>
          , I. Kompatsiaris,
          <string-name>
            <given-names>K.</given-names>
            <surname>Kinder-Kurlanda</surname>
          </string-name>
          ,
          <string-name>
            <given-names>C.</given-names>
            <surname>Wagner</surname>
          </string-name>
          ,
          <string-name>
            <given-names>F.</given-names>
            <surname>Karimi</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Fernández</surname>
          </string-name>
          ,
          <string-name>
            <given-names>H.</given-names>
            <surname>Alani</surname>
          </string-name>
          ,
          <string-name>
            <given-names>B.</given-names>
            <surname>Berendt</surname>
          </string-name>
          ,
          <string-name>
            <given-names>T.</given-names>
            <surname>Kruegel</surname>
          </string-name>
          ,
          <string-name>
            <given-names>C.</given-names>
            <surname>Heinze</surname>
          </string-name>
          ,
          <string-name>
            <given-names>K.</given-names>
            <surname>Broelemann</surname>
          </string-name>
          , G. Kasneci,
          <string-name>
            <given-names>T.</given-names>
            <surname>Tiropanis</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S.</given-names>
            <surname>Staab</surname>
          </string-name>
          ,
          <article-title>Bias in data-driven artificial intelligence systems - an introductory survey</article-title>
          ,
          <source>Wiley Interdiscip. Rev. Data Min. Knowl. Discov</source>
          .
          <volume>10</volume>
          (
          <year>2020</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          [3]
          <string-name>
            <given-names>S.</given-names>
            <surname>Harding</surname>
          </string-name>
          , Whose Science?
          <article-title>Whose Knowledge?: Thinking from Women's Lives</article-title>
          , Cornell University Press,
          <year>2016</year>
          . URL: https://www.degruyter.com/document/doi/10.7591/ 9781501712951/html. doi:
          <volume>10</volume>
          .7591/9781501712951, publication Title: Whose Science? Whose Knowledge?
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          [4]
          <string-name>
            <surname>I. Lowrie</surname>
          </string-name>
          ,
          <string-name>
            <given-names>Algorithmic</given-names>
            <surname>Rationality</surname>
          </string-name>
          .
          <source>Epistemology and Eficiency in the Data Sciences, Big Data &amp; Society</source>
          <volume>4</volume>
          (
          <year>2017</year>
          )
          <article-title>2053951717700925</article-title>
          . URL: https://doi.org/10.1177/2053951717700925. doi:
          <volume>10</volume>
          .1177/2053951717700925, publisher: SAGE Publications Ltd.
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          [5]
          <string-name>
            <given-names>J.</given-names>
            <surname>Wajcman</surname>
          </string-name>
          , Feminism confronts Technology, Polity Press, Cambridge, UK,
          <year>1991</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          [6]
          <string-name>
            <given-names>D.</given-names>
            <surname>Haraway</surname>
          </string-name>
          ,
          <string-name>
            <given-names>Situated</given-names>
            <surname>Knowledges</surname>
          </string-name>
          .
          <article-title>The Science Question in Feminism and the Privilege of Partial Perspective</article-title>
          ,
          <source>Feminist Studies</source>
          <volume>14</volume>
          (
          <issue>3</issue>
          ) (
          <year>1988</year>
          )
          <fpage>575</fpage>
          -
          <lpage>599</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          [7]
          <string-name>
            <given-names>P.</given-names>
            <surname>Hill</surname>
          </string-name>
          <string-name>
            <surname>Collins</surname>
          </string-name>
          ,
          <article-title>Black feminist thought: knowledge, consciousness, and the politics of empowerment, Routledge classics</article-title>
          , 2nd ed. ed.,
          <string-name>
            <surname>Routledge</surname>
          </string-name>
          , New York,
          <year>2009</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          [8]
          <string-name>
            <given-names>T.</given-names>
            <surname>Gebru</surname>
          </string-name>
          ,
          <article-title>Oxford handbook on AI ethics book chapter on race and gender</article-title>
          , CoRR abs/
          <year>1908</year>
          .06165 (
          <year>2019</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          [9]
          <string-name>
            <given-names>L.</given-names>
            <surname>Hancox-Li</surname>
          </string-name>
          ,
          <string-name>
            <given-names>I. E.</given-names>
            <surname>Kumar</surname>
          </string-name>
          ,
          <article-title>Epistemic values in feature importance methods: Lessons from feminist epistemology</article-title>
          , in: M.
          <string-name>
            <surname>C. Elish</surname>
            ,
            <given-names>W.</given-names>
          </string-name>
          <string-name>
            <surname>Isaac</surname>
          </string-name>
          , R. S. Zemel (Eds.),
          <source>FAccT '21: 2021 ACM Conference on Fairness, Accountability, and Transparency</source>
          , Virtual Event / Toronto, Canada, March 3-
          <issue>10</issue>
          ,
          <year>2021</year>
          , ACM,
          <year>2021</year>
          , pp.
          <fpage>817</fpage>
          -
          <lpage>826</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          [10]
          <string-name>
            <given-names>C.</given-names>
            <surname>Perez</surname>
          </string-name>
          , Invisible Women:
          <article-title>Data Bias in a World Designed for Men</article-title>
          ,
          <source>Random House</source>
          ,
          <year>2019</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref11">
        <mixed-citation>
          [11]
          <string-name>
            <surname>C. D'Ignazio</surname>
            ,
            <given-names>L.</given-names>
          </string-name>
          <string-name>
            <surname>Klein</surname>
          </string-name>
          , Data Feminism, MIT Press,
          <year>2020</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref12">
        <mixed-citation>
          [12]
          <string-name>
            <given-names>C.</given-names>
            <surname>Bath</surname>
          </string-name>
          ,
          <article-title>Searching for methodology: Feminist technology design in computer science</article-title>
          , http://elib.suub.uni-bremen.de/ip/docs/00010427.pdf (
          <year>2009</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref13">
        <mixed-citation>
          [13]
          <string-name>
            <given-names>M.</given-names>
            <surname>Ruckenstein</surname>
          </string-name>
          ,
          <string-name>
            <given-names>L. L. M.</given-names>
            <surname>Turunen</surname>
          </string-name>
          ,
          <article-title>Re-Humanizing the Platform</article-title>
          .
          <source>Content Moderators and the Logic of Care, New Media &amp; Society</source>
          <volume>22</volume>
          (
          <year>2020</year>
          )
          <fpage>1026</fpage>
          -
          <lpage>1042</lpage>
          . URL: http://journals.sagepub. com/doi/10.1177/1461444819875990. doi:
          <volume>10</volume>
          .1177/1461444819875990.
        </mixed-citation>
      </ref>
      <ref id="ref14">
        <mixed-citation>
          [14]
          <string-name>
            <given-names>N.</given-names>
            <surname>Seaver</surname>
          </string-name>
          ,
          <string-name>
            <surname>CARE</surname>
          </string-name>
          AND SCALE: Decorrelative Ethics in Algorithmic Recommendation,
          <source>Cultural Anthropology</source>
          <volume>36</volume>
          (
          <year>2021</year>
          )
          <fpage>509</fpage>
          -
          <lpage>537</lpage>
          . URL: https://onlinelibrary.wiley.com/ doi/abs/10.14506/ca36.3.11. doi:
          <volume>10</volume>
          .14506/ca36.3.11, _eprint: https://onlinelibrary.wiley.com/doi/pdf/10.14506/ca36.3.11.
        </mixed-citation>
      </ref>
      <ref id="ref15">
        <mixed-citation>
          [15]
          <string-name>
            <surname>K. J. Rohlfing</surname>
            ,
            <given-names>P.</given-names>
          </string-name>
          <string-name>
            <surname>Cimiano</surname>
            , I. Scharlau,
            <given-names>T.</given-names>
          </string-name>
          <string-name>
            <surname>Matzner</surname>
            ,
            <given-names>H. M.</given-names>
          </string-name>
          <string-name>
            <surname>Buhl</surname>
            ,
            <given-names>H.</given-names>
          </string-name>
          <string-name>
            <surname>Buschmeier</surname>
            ,
            <given-names>E.</given-names>
          </string-name>
          <string-name>
            <surname>Esposito</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          <string-name>
            <surname>Grimminger</surname>
            ,
            <given-names>B.</given-names>
          </string-name>
          <string-name>
            <surname>Hammer</surname>
            ,
            <given-names>R.</given-names>
          </string-name>
          <string-name>
            <surname>Häb-Umbach</surname>
            ,
            <given-names>I.</given-names>
          </string-name>
          <string-name>
            <surname>Horwath</surname>
            ,
            <given-names>E.</given-names>
          </string-name>
          <string-name>
            <surname>Hüllermeier</surname>
            ,
            <given-names>F.</given-names>
          </string-name>
          <string-name>
            <surname>Kern</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          <string-name>
            <surname>Kopp</surname>
            ,
            <given-names>K.</given-names>
          </string-name>
          <string-name>
            <surname>Thommes</surname>
            ,
            <given-names>A.-C. N.</given-names>
          </string-name>
          <string-name>
            <surname>Ngomo</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          <string-name>
            <surname>Schulte</surname>
            ,
            <given-names>H.</given-names>
          </string-name>
          <string-name>
            <surname>Wachsmuth</surname>
            ,
            <given-names>P.</given-names>
          </string-name>
          <string-name>
            <surname>Wagner</surname>
            ,
            <given-names>B.</given-names>
          </string-name>
          <string-name>
            <surname>Wrede</surname>
          </string-name>
          ,
          <article-title>Explanation as a Social Practice</article-title>
          .
          <article-title>Toward a Conceptual Framework for the Social Design of AI Systems,</article-title>
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>