<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta />
    <article-meta>
      <title-group>
        <article-title>Responsible Configuration Using LLM-based Sustainability-Aware Explanations</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Sebastian Lubos</string-name>
          <email>slubos@ist.tugraz.at</email>
          <xref ref-type="aff" rid="aff1">1</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Alexander Felfernig</string-name>
          <xref ref-type="aff" rid="aff1">1</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Lothar Hotz</string-name>
          <email>lothar.hotz@uni-hamburg.de</email>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Thi Ngoc Trang Tran</string-name>
          <email>ttrang@ist.tugraz.at</email>
          <xref ref-type="aff" rid="aff1">1</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Seda Polat-Erdeniz</string-name>
          <xref ref-type="aff" rid="aff1">1</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Viet-Man Le</string-name>
          <xref ref-type="aff" rid="aff1">1</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Damian Garber</string-name>
          <email>dgarber@ist.tugraz.at</email>
          <xref ref-type="aff" rid="aff1">1</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Merfat El-Mansi</string-name>
          <email>merfat.el-mansi@student.tugraz.at</email>
          <xref ref-type="aff" rid="aff1">1</xref>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          <institution>Hamburger Informatik Technologie-Center e.V.</institution>
          ,
          <addr-line>Hamburg</addr-line>
          ,
          <country country="DE">Germany</country>
        </aff>
        <aff id="aff1">
          <label>1</label>
          <institution>Institute of Software Technology, Graz University of Technology</institution>
          ,
          <addr-line>Graz</addr-line>
          ,
          <country country="AT">Austria</country>
        </aff>
      </contrib-group>
      <abstract>
        <p>Configuration systems play an important role in achieving the sustainable development goals (SDGs) defined by the United Nations. As decision support systems, configurators help users to decide which components or features to include in or exclude from a configuration. An important task of configurators is the provision of explanations which help to achieve goals such as increasing configuration understandability, increasing a user's trust, and persuading users/customers to include specific configuration components. Our goal in this paper is to introduce the concept of „sustainability-aware explanations“ which can help to support the sustainable development goals. The type of explanations we propose in this context are somehow orthogonal to typical explanations used in industrial configuration environments. A major objective in this context is to follow a „less-is-more“ principle focusing on diferent aspects of the idea of „responsible configuration“ which refers to configuration techniques explicitly supporting the mentioned sustainability goals. We report the initial results of an evaluation that provide insights on potential impacts of the proposed explanations.</p>
      </abstract>
      <kwd-group>
        <kwd>eol&gt;Explanations</kwd>
        <kwd>Sustainability</kwd>
        <kwd>Green Configuration</kwd>
        <kwd>Responsible Configuration</kwd>
        <kwd>Configuration for Good</kwd>
        <kwd>Nudging</kwd>
        <kwd>Persuasion</kwd>
        <kwd>Knowledgebased Configuration</kwd>
      </kwd-group>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>1. Introduction</title>
      <p>
        The 17 sustainable development goals (SDGs) defined by
the United Nations (UN) provide a blueprint for peace and
prosperity on our planet.1 Examples of such goals are
good health and well-being (e.g., in terms of fostering the
consumption of healthy food), responsible consumption and
production (e.g., in terms of reduced energy consumption),
and sustainable cities and communities (e.g., in the context
of tourism, avoiding negative environmental impacts and
taking into account the local communities and cultural
heritage) [
        <xref ref-type="bibr" rid="ref1">1</xref>
        ].
      </p>
      <p>
        Knowledge-based configuration [
        <xref ref-type="bibr" rid="ref2 ref3 ref4 ref5">2, 3, 4, 5</xref>
        ] can be
regarded as a core-technology of mass customization
[
        <xref ref-type="bibr" rid="ref6">6</xref>
        ]. On the basis of configurators, users are enabled to
design a product in an individualized fashion that fits
their wishes and needs. In configuration settings, we
can observe an ever-increasing demand for taking into
account sustainability aspects [
        <xref ref-type="bibr" rid="ref7 ref8">7, 8</xref>
        ]. Following the basic
definition of „ configuration “ given by Sabin and Weigel [
        <xref ref-type="bibr" rid="ref3">3</xref>
        ],
i.e., „configuration is a special case of design activity where
the configured artifact is assembled from a fixed set of
welldefined component types and components are interacting in
predefined ways “, we define „ responsible configuration “ as
„configuration which takes into account the United Nation’s
sustainable development goals“.
      </p>
      <p>
        In knowledge-based systems, explanations can be applied
for diferent purposes [
        <xref ref-type="bibr" rid="ref9">9</xref>
        ]. First, so-called why explanations
[
        <xref ref-type="bibr" rid="ref10 ref11 ref12">10, 11, 12</xref>
        ] focus on the aspect of mentioning the most
relevant user requirements that lead to the determination of
a specific configuration. Furthermore, why not explanations
focus on supporting users in situations where no solution
can be identified [
        <xref ref-type="bibr" rid="ref13 ref14 ref15">13, 14, 15</xref>
        ]. From the application point
of view, explanations can be applied to achieve diferent
goals [
        <xref ref-type="bibr" rid="ref16">16</xref>
        ].2 Examples thereof are eficiency (reducing the
time that is needed to complete a configuration task),
persuasiveness (convincing users to change their component
selection behavior), transparency (making the inclusion or
exclusion of specific components transparent to the user),
trust (increasing a user’s confidence in the configuration
system), scrutability (making it possible for the user to
adapt the configurator behavior, e.g., in terms of the used
component inclusion/exclusion strategy), and satisfaction
(e.g., increasing the usability of a configuration system).
These goals must be regarded as examples – for related
details we refer to [
        <xref ref-type="bibr" rid="ref11 ref16 ref17 ref18">11, 16, 17, 18</xref>
        ].
      </p>
      <p>
        In this paper, we focus on the persuasion aspect of
explanations [
        <xref ref-type="bibr" rid="ref19">19</xref>
        ]. More precisely, we analyze possibilities
to formulate explanations in such a way that users are
nudged towards more sustainability-aware configuration
decisions. Following a „less-is-more“ principle, we show
how to formulate explanations following Cialdini’s six
principles of persuasion [
        <xref ref-type="bibr" rid="ref20">20</xref>
        ] (see Table 1).
      </p>
      <p>
        Sustainability-aware explanations have to focus on
argumentations including sustainability aspects. Our
formulation of such explanations is based on large
language model (LLM) prompts [
        <xref ref-type="bibr" rid="ref21">21</xref>
        ] which help to
associate sustainable development goals with the mentioned
persuasive principles. For example, in the context of
car configuration, explanations could refer to the positive
environmental aspects of purchasing smaller cars or on the
advantages of electric vehicles compared to gasoline-driven
ones.
      </p>
      <p>
        Positive impacts of such sustainability-aware
explanations can be, for example, higher-quality
configuration decisions, a lower amount of unneeded
components, and components with less negative
2The categorizations of [
        <xref ref-type="bibr" rid="ref11 ref16">11, 16</xref>
        ] have been developed in the context of
recommender systems but can also be applied in configuration contexts.
environmental impacts [
        <xref ref-type="bibr" rid="ref8">8</xref>
        ]. From a commercial point of
view, such explanations might appear – at least to some
extent – counterproductive due to potential consequences
in terms of decreasing turnovers. Thus,
sustainabilityaware explanations are often in contrast to explanations in
mainstream configuration environments which focus on
increasing sales rates in most of the cases.
      </p>
      <p>The contributions of this paper are the following. First,
we propose the concept of sustainability-aware explanations
for configurations. Second, we provide reference examples
of such explanations in the automotive domain. Third, we
present initial results of a corresponding evaluation.</p>
      <p>The remainder of this paper is organized as follows. In
Section 2, we provide diferent examples of LLM-generated
sustainability-aware explanations in the car configuration
domain. Thereafter, we discuss initial results of a related
evaluation (Section 3). In Section 4, we discuss threats to
validity. Finally, we conclude the paper with Section 5.</p>
    </sec>
    <sec id="sec-2">
      <title>2. Sustainability-Aware Explanations with LLMs</title>
      <p>In the following, we discuss scenarios where
sustainabilityaware explanations can have an impact on user decisions.
All scenarios are related to car configuration where users
receive explanations of current configurations. The major
goal of such explanations is to make users think about their
current configuration settings and to potentially adapt their
articulated preferences. Consequently, our explanations are
not in the line of why or why not explanations but focus
more on indicating potential alternatives to the current
configuration, i.e., a kind of why not choose something else
explanation. All example explanations in this paper have
been generated on the basis of the LLM ChatGPT 3.5.3</p>
      <sec id="sec-2-1">
        <title>Scenario 1: SUV vs. smaller car. The idea is to make</title>
        <p>persons (configurator users) who intend to purchase an
SUV more aware of sustainability aspects of smaller cars.
To support this, we have generated LLM-based explanations
using the following (example) LLM prompt: Assume the
following scenario: person A wants to purchase a car and is
interested in an SUV. Please provide persuasive explanations
against purchasing an SUV following the six persuasion
principles of Cialdini. The resulting explanations are
depicted in Table 2.
3https://chat.openai.com
Scenario 2: Long vs. standard range battery. The idea
is to make configurator users interested in purchasing a car
with a long-range battery aware of the sustainability aspects
of standard-range batteries. To support such explanations,
we have generated LLM-based explanations using the
following LLM prompt: person A wants to purchase an
electric car and is interested in a long-range battery. Please
provide persuasive arguments against purchasing a long range
battery following the six persuasion principles of Cialdini.
The corresponding LLM-generated sustainability-aware
explanations are depicted in Table 3.</p>
      </sec>
      <sec id="sec-2-2">
        <title>Scenario 3: Car not needed in city center.</title>
        <p>Configurator users should think about the advantages of
not having a car when living in the city center. We have
generated related LLM-based persuasive explanations
using the following LLM prompt: person A who lives
directly in the city center with various connections to public
transportation wants to purchase a car. Please provide
persuasive arguments against purchasing a car following
the six persuasion principles of Cialdini. The corresponding
sustainability-aware explanations are depicted in Table 4.</p>
      </sec>
      <sec id="sec-2-3">
        <title>Scenario 4: Less costly car due to financial situation.</title>
        <p>The idea is making configurator users with limited financial
resources intending to purchase an expensive car to
change their mind and purchase a less expensive car.
To support such explanations, we have generated
LLMbased explanations using the following (example) LLM
prompt formulation: person A with very limited financial
resources and a family with three children wants to purchase
an expensive car. Please provide persuasive arguments
against purchasing an expensive car following the six
persuasion principles of Cialdini. The related LLM-generated
explanations are depicted in Table 5.</p>
      </sec>
    </sec>
    <sec id="sec-3">
      <title>3. Evaluation</title>
      <p>Properties of LLM-based explanations. In Table 6,
we summarize the diferent argumentation lines generated
by the large language model (LLM). (1) In the context
of the persuasion dimension reciprocity, LLM-generated
explanations refer to the aspect of „giving something
back to the community“, for example, purchasing an
eco-friendly vehicle can be a way of giving back to the
environment. (2) Explanations related to the persuasion
dimension scarcity on the one hand refer to decreasing
incentives for sustainable equipment (e.g., cars), on the
other hand to limited resources (e.g., financial resources).
(3) In the context of the persuasion dimension authority,
diferent experts such as environmental experts are used as
representatives of authorities. (4) Explanations related to
the persuasion dimension commitment assume an existing
commitment of the current user, for example, already proved
in previous configuration sessions. (5) In the context of the
persuasion dimension liking, LLM-generated explanations
refer to a user’s family, friends, and neighbors (e.g., your
family will like your decision). (6) Explanations related to
the persuasion dimension social proof refer to trends of
peers, the wider community, and similar families.</p>
      <sec id="sec-3-1">
        <title>LLM-based impact estimates. Using the LLM prompt</title>
        <p>
          which of these explanations would have the highest persuasive
impact on a user? Please provide a ranking., we asked the
LLM also for a ranking of the impact of the generated
explanations following the idea of LLM self-evaluation
[
          <xref ref-type="bibr" rid="ref22">22</xref>
          ]. In most settings, the dimensions authority and
commitment &amp; consistency have been regarded as the two
most impactful persuasion dimensions (see Table 7). The
related LLM-based argument is that the latter appeals
directly to a person’s desire to act in accordance with their
past beliefs and statements and the former has a high impact
specifically in scenarios where the underlying topic involves
technical details. Finally, the dimension of social proof can
be impactful in situations where social trends and peer
behaviors impact decisions.
        </p>
        <p>Feedback of study participants. For a very first
evaluation beyond the LLM feedback, we conducted a small
user study in our research group with N=10 participants
(computer science students at our university). The
participants were asked to rank the diferent explanations
according to their potential persuasive impact. The results
are included in Table 7. Overall, there exists an overlap
between the explanations ranked highest by the LLM and
those selected by the study participants. In this context,
the authority principle has been regarded as relevant in all
example configuration scenarios. This initial result indicates
basic LLM capabilities to recommend persuasion strategies.</p>
      </sec>
    </sec>
    <sec id="sec-4">
      <title>4. Threats to Validity</title>
      <p>The explanation concepts presented in this paper are based
on one selected configuration scenario (car configuration).
These explanations are just high-level examples and many
further (also more detailed ones) can be envisioned for
car configuration (e.g., the sustainability aspects of a less
powerful car engine) and beyond. In our work, we did not
focus on a specific phase of a configuration process, i.e.,
the mentioned explanations could even be used before the
configuration process has been started (e.g., as explanations
in wish lists or product information). Gaining more
related insights is a major focus of our future research.</p>
      <p>
        The presented impact ranking of explanations has been
primarily discussed on the basis of an LLM-generated
ranking [
        <xref ref-type="bibr" rid="ref22">22</xref>
        ] including corresponding argumentations that
help to understand the proposed ranking. More detailed
studies with real users (and more detailed related preference
and context information) are planned within the scope of
future work also to better understand the limitations of
LLMs with regard to the recommendation of persuasion
strategies. Up to now, no LLM-related hallucination efects
could be observed, however, this is an important aspect to
be taken into account in future work. A recently mentioned
new persuasion principle (identification) [
        <xref ref-type="bibr" rid="ref23">23</xref>
        ] will be taken
into account in future studies. Finally, more detailed
LLM prompts better taking into account the context (and
preferences) of the current user are regarded as an important
topic of future work.
      </p>
    </sec>
    <sec id="sec-5">
      <title>5. Conclusions</title>
      <p>In this paper, we have introduced the concept of
sustainability-aware explanations of configurations. Using
the example of car configuration, we have explained
and exemplified this type of explanation. Following
a set of persuasion dimensions, we have analyzed the
LLM-generated explanations with regard to the used
argumentation lines and analyzed the impact of the
generated explanations on the user. In this context, LLMs
show to be applicable in terms of generating explanations
in a flexible fashion but also to recommend explanations in
specific configuration contexts. Our future work will include
detailed studies with real users with the goal to compare
LLM-based rankings with the perception of explanations
by real users. Further research will include an analysis
of the efects of combining explanations (e.g., integrating
authority-based with commitment-based explanations).</p>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          [1]
          <string-name>
            <given-names>A.</given-names>
            <surname>Felfernig</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Wundara</surname>
          </string-name>
          ,
          <string-name>
            <given-names>T.</given-names>
            <surname>Tran</surname>
          </string-name>
          , S. PolatErdeniz, S. Lubos,
          <string-name>
            <given-names>M. E.</given-names>
            <surname>Mansi</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D.</given-names>
            <surname>Garber</surname>
          </string-name>
          ,
          <string-name>
            <given-names>V.</given-names>
            <surname>Le</surname>
          </string-name>
          ,
          <article-title>Recommender systems for sustainability: overview and research issues</article-title>
          ,
          <source>Frontiers in Big Data</source>
          <volume>6</volume>
          (
          <year>2023</year>
          ). URL: https://www.frontiersin.org/articles/10. 3389/fdata.
          <year>2023</year>
          .
          <volume>1284511</volume>
          . doi:
          <volume>10</volume>
          .3389/fdata.
          <year>2023</year>
          .
          <volume>1284511</volume>
          .
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          [2]
          <string-name>
            <given-names>A.</given-names>
            <surname>Felfernig</surname>
          </string-name>
          ,
          <string-name>
            <given-names>L.</given-names>
            <surname>Hotz</surname>
          </string-name>
          ,
          <string-name>
            <given-names>C.</given-names>
            <surname>Bagley</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J.</given-names>
            <surname>Tiihonen</surname>
          </string-name>
          ,
          <article-title>Knowledge-based Configuration -</article-title>
          From Research to Business Cases, Morgan Kaufmann,
          <year>2014</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          [3]
          <string-name>
            <given-names>D.</given-names>
            <surname>Sabin</surname>
          </string-name>
          ,
          <string-name>
            <given-names>R.</given-names>
            <surname>Weigel</surname>
          </string-name>
          ,
          <article-title>Product configuration frameworks-a survey</article-title>
          ,
          <source>IEEE Intelligent Systems</source>
          <volume>13</volume>
          (
          <year>1998</year>
          )
          <fpage>42</fpage>
          -
          <lpage>49</lpage>
          . doi:
          <volume>10</volume>
          .1109/5254.708432.
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          [4]
          <string-name>
            <given-names>M.</given-names>
            <surname>Stumptner</surname>
          </string-name>
          ,
          <article-title>An Overview of Knowledge-based Configuration</article-title>
          ,
          <source>AI</source>
          Communications
          <volume>10</volume>
          (
          <year>1997</year>
          )
          <fpage>111</fpage>
          -
          <lpage>126</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          [5]
          <string-name>
            <given-names>A.</given-names>
            <surname>Felfernig</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Falkner</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D.</given-names>
            <surname>Benavides</surname>
          </string-name>
          , Feature Models:
          <string-name>
            <surname>AI-Driven</surname>
            <given-names>Design</given-names>
          </string-name>
          ,
          <article-title>Analysis and Applications</article-title>
          , SpringerBriefs in Computer Science, Springer, Cham,
          <year>2024</year>
          . doi:
          <volume>10</volume>
          .1007/978-3-
          <fpage>031</fpage>
          -61874-1.
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          [6]
          <string-name>
            <given-names>B. J.</given-names>
            <surname>Pine</surname>
          </string-name>
          ,
          <string-name>
            <given-names>B.</given-names>
            <surname>Victor</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Boynton</surname>
          </string-name>
          , Making mass customization work,
          <source>Harvard Business Review</source>
          <volume>71</volume>
          (
          <year>1993</year>
          )
          <fpage>108</fpage>
          -
          <lpage>119</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          [7]
          <string-name>
            <given-names>N.</given-names>
            <surname>Tchertchian</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D.</given-names>
            <surname>Millet</surname>
          </string-name>
          ,
          <article-title>Optimization approach for attractive and sustainable products</article-title>
          ,
          <source>Procedia CIRP 90</source>
          (
          <year>2020</year>
          )
          <fpage>350</fpage>
          -
          <lpage>354</lpage>
          . doi:
          <volume>10</volume>
          .1016/j.procir.
          <year>2020</year>
          .
          <volume>01</volume>
          . 103.
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          [8]
          <string-name>
            <given-names>R.</given-names>
            <surname>Wiezorek</surname>
          </string-name>
          ,
          <string-name>
            <given-names>N.</given-names>
            <surname>Christensen</surname>
          </string-name>
          ,
          <article-title>Integrating sustainability information in configurators</article-title>
          ,
          <source>in: Configuration Workshop (Conf WS)</source>
          ,
          <year>2021</year>
          , pp.
          <fpage>58</fpage>
          -
          <lpage>64</lpage>
          . URL: https://ceur-ws.
          <source>org/</source>
          Vol-
          <volume>2945</volume>
          /
          <fpage>52</fpage>
          -
          <string-name>
            <surname>RW-Conf</surname>
            <given-names>WS21</given-names>
          </string-name>
          _ paper_16.pdf .
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          [9]
          <string-name>
            <given-names>T. N. T.</given-names>
            <surname>Tran</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Felfernig</surname>
          </string-name>
          ,
          <string-name>
            <given-names>V. M.</given-names>
            <surname>Le</surname>
          </string-name>
          ,
          <string-name>
            <given-names>T. M. N.</given-names>
            <surname>Chau</surname>
          </string-name>
          , T. G.
          <article-title>Mai, User needs for explanations of recommendations: In-depth analyses of the role of item domain and personal characteristics</article-title>
          ,
          <source>in: 31st ACM Conference on User Modeling, Adaptation and Personalization</source>
          ,
          <source>UMAP '23</source>
          ,
          <string-name>
            <surname>ACM</surname>
          </string-name>
          , New York, NY, USA,
          <year>2023</year>
          , pp.
          <fpage>54</fpage>
          -
          <lpage>65</lpage>
          . URL: https://doi.org/10.1145/3565472.3592950. doi:
          <volume>10</volume>
          .1145/3565472.3592950.
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          [10]
          <string-name>
            <given-names>G.</given-names>
            <surname>Friedrich</surname>
          </string-name>
          ,
          <article-title>Elimination of spurious explanations</article-title>
          ,
          <source>in: 16th European Conference on Artificial Intelligence, ECAI'04</source>
          , IOS Press, NLD,
          <year>2004</year>
          , pp.
          <fpage>813</fpage>
          -
          <lpage>817</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref11">
        <mixed-citation>
          [11]
          <string-name>
            <given-names>G.</given-names>
            <surname>Friedrich</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Zanker</surname>
          </string-name>
          ,
          <article-title>A taxonomy for generating explanations in recommender systems</article-title>
          ,
          <source>AI</source>
          Magazine
          <volume>32</volume>
          (
          <year>2011</year>
          )
          <fpage>90</fpage>
          -
          <lpage>98</lpage>
          . doi:
          <volume>10</volume>
          .1609/aimag.v32i3.
          <fpage>2365</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref12">
        <mixed-citation>
          [12]
          <string-name>
            <given-names>S. P.</given-names>
            <surname>Erdeniz</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Schrempf</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D.</given-names>
            <surname>Kramer</surname>
          </string-name>
          ,
          <string-name>
            <given-names>P. P.</given-names>
            <surname>Rainer</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Felfernig</surname>
          </string-name>
          ,
          <string-name>
            <given-names>T.</given-names>
            <surname>Tran</surname>
          </string-name>
          ,
          <string-name>
            <given-names>T.</given-names>
            <surname>Burgstaller</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S.</given-names>
            <surname>Lubos</surname>
          </string-name>
          ,
          <article-title>Computational evaluation of model-agnostic explainable ai using local feature importance in healthcare</article-title>
          , in: J.
          <string-name>
            <surname>M. Juarez</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          <string-name>
            <surname>Marcos</surname>
            ,
            <given-names>G.</given-names>
          </string-name>
          <string-name>
            <surname>Stiglic</surname>
            ,
            <given-names>A</given-names>
          </string-name>
          . Tucker (Eds.),
          <source>Artificial Intelligence in Medicine</source>
          , Springer Nature Switzerland, Cham,
          <year>2023</year>
          , pp.
          <fpage>114</fpage>
          -
          <lpage>119</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref13">
        <mixed-citation>
          [13]
          <string-name>
            <given-names>A.</given-names>
            <surname>Felfernig</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Schubert</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S.</given-names>
            <surname>Reiterer</surname>
          </string-name>
          ,
          <article-title>Personalized diagnosis for over-constrained problems</article-title>
          , in: 23rd
          <source>International Joint Conference on Artificial Intelligence, IJCAI '13</source>
          , AAAI Press,
          <year>2013</year>
          , pp.
          <fpage>1990</fpage>
          -
          <lpage>1996</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref14">
        <mixed-citation>
          [14]
          <string-name>
            <surname>V.-M. Le</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          <string-name>
            <surname>Felfernig</surname>
            ,
            <given-names>T. N. T.</given-names>
          </string-name>
          <string-name>
            <surname>Tran</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          <string-name>
            <surname>Uta</surname>
          </string-name>
          , Informedqx:
          <article-title>Informed conflict detection for overconstrained problems</article-title>
          ,
          <source>AAAI Conference on Artificial Intelligence</source>
          <volume>38</volume>
          (
          <year>2024</year>
          )
          <fpage>10616</fpage>
          -
          <lpage>10623</lpage>
          . doi:
          <volume>10</volume>
          .1609/ aaai.v38i9.
          <fpage>28932</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref15">
        <mixed-citation>
          [15]
          <string-name>
            <surname>B. O'Sullivan</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          <string-name>
            <surname>Papadopoulos</surname>
            ,
            <given-names>B.</given-names>
          </string-name>
          <string-name>
            <surname>Faltings</surname>
            ,
            <given-names>P.</given-names>
          </string-name>
          <string-name>
            <surname>Pu</surname>
          </string-name>
          ,
          <article-title>Representative explanations for over-constrained problems</article-title>
          ,
          <source>in: 22nd National Conference on Artificial Intelligence -</source>
          Volume
          <volume>1</volume>
          , AAAI'
          <fpage>07</fpage>
          , AAAI Press,
          <year>2007</year>
          , pp.
          <fpage>323</fpage>
          -
          <lpage>328</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref16">
        <mixed-citation>
          [16]
          <string-name>
            <given-names>N.</given-names>
            <surname>Tintarev</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J.</given-names>
            <surname>Masthof</surname>
          </string-name>
          ,
          <article-title>Evaluating the efectiveness of explanations for recommender systems, User Modeling and User-Adapted Interaction 22 (</article-title>
          <year>2012</year>
          )
          <fpage>399</fpage>
          -
          <lpage>439</lpage>
          . doi:
          <volume>10</volume>
          .1007/s11257-011-9117-5.
        </mixed-citation>
      </ref>
      <ref id="ref17">
        <mixed-citation>
          [17]
          <string-name>
            <given-names>N.</given-names>
            <surname>Tintarev</surname>
          </string-name>
          , Explanations of recommendations,
          <source>in: ACM Conference on Recommender Systems</source>
          , RecSys '07,
          <string-name>
            <surname>Association</surname>
          </string-name>
          for Computing Machinery, New York, NY, USA,
          <year>2007</year>
          , pp.
          <fpage>203</fpage>
          -
          <lpage>206</lpage>
          . doi:
          <volume>10</volume>
          .1145/1297231. 1297275.
        </mixed-citation>
      </ref>
      <ref id="ref18">
        <mixed-citation>
          [18]
          <string-name>
            <given-names>N.</given-names>
            <surname>Tintarev</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J.</given-names>
            <surname>Masthof</surname>
          </string-name>
          ,
          <article-title>A survey of explanations in recommender systems</article-title>
          , in: ICDEW '07, IEEE Computer Society, USA,
          <year>2007</year>
          , pp.
          <fpage>801</fpage>
          -
          <lpage>810</lpage>
          . doi:
          <volume>10</volume>
          .1109/ICDEW.
          <year>2007</year>
          .
          <volume>4401070</volume>
          .
        </mixed-citation>
      </ref>
      <ref id="ref19">
        <mixed-citation>
          [19]
          <string-name>
            <given-names>A.</given-names>
            <surname>Felfernig</surname>
          </string-name>
          ,
          <string-name>
            <given-names>B.</given-names>
            <surname>Gula</surname>
          </string-name>
          , G. Leitner,
          <string-name>
            <given-names>M.</given-names>
            <surname>Maier</surname>
          </string-name>
          ,
          <string-name>
            <given-names>R.</given-names>
            <surname>Melcher</surname>
          </string-name>
          , E. Teppan,
          <article-title>Persuasion in knowledge-based recommendation</article-title>
          , in: H.
          <string-name>
            <surname>Oinas-Kukkonen</surname>
            ,
            <given-names>P.</given-names>
          </string-name>
          <string-name>
            <surname>Hasle</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          <string-name>
            <surname>Harjumaa</surname>
            ,
            <given-names>K.</given-names>
          </string-name>
          <string-name>
            <surname>Segerståhl</surname>
          </string-name>
          , P. Øhrstrøm (Eds.),
          <source>Persuasive Technology</source>
          , Springer, Berlin, Heidelberg,
          <year>2008</year>
          , pp.
          <fpage>71</fpage>
          -
          <lpage>82</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref20">
        <mixed-citation>
          [20]
          <string-name>
            <given-names>R.</given-names>
            <surname>Cialdini</surname>
          </string-name>
          , Influence: The Psychology of Persuasion, Quill, New York, NY, USA,
          <year>1993</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref21">
        <mixed-citation>
          [21]
          <string-name>
            <given-names>S.</given-names>
            <surname>Lubos</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Felfernig</surname>
          </string-name>
          ,
          <string-name>
            <given-names>T.</given-names>
            <surname>Tran</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D.</given-names>
            <surname>Garber</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Mansi</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S.</given-names>
            <surname>Erdeniz</surname>
          </string-name>
          ,
          <string-name>
            <given-names>V.</given-names>
            <surname>Le</surname>
          </string-name>
          ,
          <article-title>Leveraging llms for the quality assurance of software requirements</article-title>
          ,
          <source>in: Proceedings of the 32nd IEEE International Requirements Engineering 2024 Conference (RE@Next! Track)</source>
          , IEEE,
          <year>2024</year>
          , p. to appear.
        </mixed-citation>
      </ref>
      <ref id="ref22">
        <mixed-citation>
          [22]
          <string-name>
            <given-names>Y.</given-names>
            <surname>Chang</surname>
          </string-name>
          ,
          <string-name>
            <given-names>X.</given-names>
            <surname>Wang</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J.</given-names>
            <surname>Wang</surname>
          </string-name>
          ,
          <string-name>
            <given-names>Y.</given-names>
            <surname>Wu</surname>
          </string-name>
          ,
          <string-name>
            <given-names>L.</given-names>
            <surname>Yang</surname>
          </string-name>
          ,
          <string-name>
            <given-names>K.</given-names>
            <surname>Zhu</surname>
          </string-name>
          ,
          <string-name>
            <given-names>H.</given-names>
            <surname>Chen</surname>
          </string-name>
          ,
          <string-name>
            <given-names>X.</given-names>
            <surname>Yi</surname>
          </string-name>
          ,
          <string-name>
            <given-names>C.</given-names>
            <surname>Wang</surname>
          </string-name>
          ,
          <string-name>
            <given-names>Y.</given-names>
            <surname>Wang</surname>
          </string-name>
          ,
          <string-name>
            <given-names>W.</given-names>
            <surname>Ye</surname>
          </string-name>
          ,
          <string-name>
            <given-names>Y.</given-names>
            <surname>Zhang</surname>
          </string-name>
          ,
          <string-name>
            <given-names>Y.</given-names>
            <surname>Chang</surname>
          </string-name>
          ,
          <string-name>
            <given-names>P. S.</given-names>
            <surname>Yu</surname>
          </string-name>
          ,
          <string-name>
            <given-names>Q.</given-names>
            <surname>Yang</surname>
          </string-name>
          ,
          <string-name>
            <given-names>X.</given-names>
            <surname>Xie</surname>
          </string-name>
          ,
          <article-title>A survey on evaluation of large language models</article-title>
          ,
          <source>ACM Trans. Intell. Syst. Technol</source>
          .
          <volume>15</volume>
          (
          <year>2024</year>
          ). doi:
          <volume>10</volume>
          .1145/ 3641289.
        </mixed-citation>
      </ref>
      <ref id="ref23">
        <mixed-citation>
          [23]
          <string-name>
            <given-names>R.</given-names>
            <surname>Cialdini</surname>
          </string-name>
          , Influence, New and Expanded: The Psychology of Persuasion, HarperCollins,
          <year>2021</year>
          .
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>