<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta />
    <article-meta>
      <title-group>
        <article-title>A Multi-Dimensional Conceptualization Framework for Personalized Explanations in Recommender Systems</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Qurat Ul Ain</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Mohamed Amine Chatti</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Mouadh Guesmi</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Shoeb Joarder</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          <institution>Social Computing Group, University of Duisburg-Essen</institution>
          ,
          <addr-line>47048, Duisburg</addr-line>
          ,
          <country country="DE">Germany</country>
        </aff>
      </contrib-group>
      <abstract>
        <p>Recommender systems (RS) have become an integral component of our daily lives by helping decision making easier for us. The use of recommendations has, however, increased the demand for explanations that are convincing enough to help users trust the provided recommendations. The recommendations are desired by the users to be understandable as well as personalized to their individual needs and preferences. Research on personalized explainable recommendation has emerged only recently. To help researchers quickly familiarize with this promising research field and recognize future research directions, we present a multi-dimensional conceptualization framework for personalized explanations in RS, based on five dimensions: WHAT to personalize?, TO WHOM to personalize?, WHO does the personalization?, WHY do we personalize?, and HOW to personalize?. Furthermore, we use this framework to systematically analyze and compare studies on personalized explainable recommendation.</p>
      </abstract>
      <kwd-group>
        <kwd>eol&gt;Recommender systems</kwd>
        <kwd>Explainable recommendation</kwd>
        <kwd>Personalized explanation</kwd>
      </kwd-group>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>1. Introduction</title>
      <p>users’ ratings and likes or dislikes, as compared to the
items consumed by similar users.</p>
      <p>Over the past few years, with the increased usage of However, the majority of RS still act as a black-box
online services like social media, e-learning, and e- and users have no idea why and how items are being
commerce, recommender systems (RS) have become an recommended to them. Therefore, it is increasingly
integral part of our lives. These RS help in shaping the important to make RS more intelligible and investigate
decisions of users and helping them choose what they methods to explain them to end-users. Explaining the
want based on a number of relevant options presented reasoning behind a recommendation has become an
acto them, called recommendations. However, with the tive area of research in the last few years. Researchers
increased amount of available recommendations, there have argued that explanations in RS could be very
benis a chance of creating mistrust among users about the eficial [ 1, 2, 3]. To “explain” means “to make known,
presented information. A huge amount of available in- to make plain or understandable, to give the reason
formation creates ‘information overload’ which might for or cause of” [4]. An explanation seeks to answer
lead to users questioning the validity of the provided questions such as what, why, how, what if, why not,
content and might think of it as misinformation. and how to [5]. Providing the reasoning behind why</p>
      <p>
        One way to overcome this challenge is to provide per- an item is recommended to the user or how the
recsonalized recommendations to the users. The content ommendation process works, as an explanation, adds
of these recommendations is adapted to users’ inter- to the system’s transparency [6] and can benefit user
ests and only relevant items are recommended to them. experience and trust in the RS [1] .
The goal of personalized recommendations is to pre- Explainable RS have traditionally followed a
onedict items considered attractive and interesting by the size-fits-all model, whereby the same explanation is
user. This relevant item prediction is made by either provided to each user, without taking into
considera(1) content, i.e., items having similar content with the tion an individual user’s context, i.e., abilities, goals,
items already used by the user are recommended or (2) needs, or preferences. In the explainable
recommendapast behavior, i.e., items are recommended based on tion field, research regarding personalized explanation
has emerged only recently, showing that personal
charJHoeilnstinPkrio,cFeiendlianngds of the ACM IUI Workshops 2022, March 2022, acteristics have an impact on the perception of
expla$ qurat.ain@stud.uni-due.de (Q.U. Ain); nations, and that there is potential for the development
mohamed.chatti@uni-due.de (M.A. Chatti); of personalized explanations in RS [7, 8]. For example,
mouadh.guesmi@stud.uni-due.de (M. Guesmi); researchers have focused on investigating what specific
shoeb.joarder@uni-due.de (S. Joarder) characteristics may play a role in a user’s interaction
with an explainable RS [
        <xref ref-type="bibr" rid="ref6">9, 10</xref>
        ]. An analysis on existing
© 2022 Copyright © 2022 for this paper by its authors. Use permitted under
CPWrEooUrckReshdoinpgs IhStpN:/c1e6u1r3-w-0s.o7r3g CCreEatUivReCWommoornkssLhiceonpse APtrtroibcuteioend4i.0nIgntsern(aCtioEnUal R(C-C WBYS4..0o).rg)
explainable recommendation work focusing on person- with varying backgrounds and proposed an algorithm
alized explanation is vital to help researchers quickly that allows constructing personalized explanations that
familiarize with this promising topic, compare studies are optimal in an information-theoretic sense.
Assumin this field, and recognize future research directions. ing that, based on varying backgrounds like training,
To fill this critical research gap, we present a timely domain knowledge and demographic characteristics,
inconceptualization framework for personalized expla- dividuals have diferent understandings and hence
mennations in RS and provide an overview of the current tal models about the learning algorithm, Kuhl et al. [15]
state research in this emerging research area. To get investigated how personalized explanations of learning
at this, we gathered, analyzed, and connected existing algorithms afect employees’ compliance behavior and
concepts related to personalized explanations in the ar- task performance. On a conceptual level, Schneider and
tificial intelligence (AI), machine learning (ML), and RS Handali [16] proposed a conceptualization of
personaldomains. We then proposed a conceptualization frame- ized explanation in ML based on a framework covering
work that can be used to systematically categorize and desiderata of personalized explanations, dimensions
compare the literature on personalized explainable rec- that can be personalized, what and how information
ommendation. Based on this framework, we analyzed can be obtained from individuals and how this
inforstudies in this research domain. mation can be utilized to customize explanations.
      </p>
      <p>
        The paper is structured as follows. We first outline In the field of explainable recommendation, research
the background for this research (Section 2). We then regarding personalized explanation is emerging,
recpresent the details of the proposed conceptualization ognizing that it is increasingly important not only to
framework (Section 3) and use the framework to ana- explain recommendations to the user but also to
perlyze the literature on personalized explainable recom- sonalize these explanations [7, 8]. Studies showed that
mendation (Section 4). Finally, we summarize the work diferent users have diferent reactions to, and
expecand outline future research plans (Section 5). tations from explainable RS [17, 9] and that personal
characteristics play a major role in the perception of,
and interaction with these systems [
        <xref ref-type="bibr" rid="ref6">10, 18, 19</xref>
        ].
How2. Personalized Explanation ever, a comprehensive framework to categorize related
work on personalized explanation in the RS field is
lacking.
      </p>
      <p>In the field of explainable AI (XAI), Mohseni et al. [11]
argue that diferent user groups will have other goals
in mind while using such systems. For example, while
machine learning experts might prefer highly-detailed
visual explanations of deep models to help them
optimize and diagnose algorithms, lay-users do not expect
fully detailed explanations for every query from a
personalized agent. Instead, systems with lay users as
target groups aim to enhance the user experience with
the system through improving their understanding and
trust. In the same direction, Miller [12] argues that
providing the exact algorithm which generated the
specific decision is not necessarily the best explanation.</p>
      <p>Therefore, the literature on AI/ML in recent years has
emphasized the need for explanations that are tailored
to individuals, i.e., personalized explanations. For
example, Arya et al. [13] stressed that one explanation does
not fit all, as diferent AI stakeholders present diferent
requirements for explanations and may desire
diferent kinds of explanations (e.g., feature-based,
instancebased, language-based). The authors presented an AI
toolkit, which contains eight state-of-the-art
explainability algorithms that can explain an AI model in
different ways to a diverse set of users. Jung and Nardelli
[14] pointed out that XAI is challenging since
explanations must be tailored (personalized) to individual users</p>
    </sec>
    <sec id="sec-2">
      <title>3. A Framework For Personalized</title>
    </sec>
    <sec id="sec-3">
      <title>Explainable Recommendation</title>
      <p>
        To dive deeper into the understanding of key concepts
related to personalized explanation in RS and provide
a systematic categorization of the literature in this
area, we propose a multi-dimensional
conceptualization framework for personalized explanations in RS
(see Figure 1). To develop this framework, we gathered,
utilized, and adapted ideas, concepts, and methods
related to personalized explanations in the RS literature
and formulated a succinct and concise framework based
on five dimensions: WHAT to personalize?, TO WHOM
to personalize?, WHO does the personalization?, WHY
do we personalize?, and HOW to personalize?. Similar
to the conceptualization of personalized explanation
in ML presented in [16], we adopted and adapted the
framework presented by Fan and Poole [
        <xref ref-type="bibr" rid="ref2">20</xref>
        ], and
extended it from "what, to whom, and who personalizes?"
by adding two more dimensions, namely "why to
personalize?" which describes the goals of personalized
explanation and "how to personalize?" which describes
the methods for personalized explanations. Below, we tween the user profile and recommended item features.
discuss in detail the five dimensions of our proposed To personalize an explanation, its content must be
taiconceptualization framework for personalized explana- lored to diferent user profiles and should be adapted
tion in RS. according to the explanation context.
      </p>
      <sec id="sec-3-1">
        <title>3.1. WHAT to Personalize?</title>
        <sec id="sec-3-1-1">
          <title>3.1.2. Explanation Design Choices</title>
          <p>The "WHAT" dimension refers to the properties of an There is a large design space for explainable RS.
Reexplanation that can be adjusted to the user profile searchers presented diferent ways to design
explainto provide personalized explanations. We identified able RS, referred to as explanation design choices [17,
two main explanation properties that can be adapted 21]. Like the content of an explanation, these design
based on explainee (for whom explanations are pro- choices represent further characteristics of an
explavided) data, namely content and design choices of the nation that can be customized based on a user profile.
explanation. Based on the literature on explainable recommendation,
we identified five explanation design choices, namely
3.1.1. Explanation Content explanation style, explanation scope, explanation format,
level of detail, and intelligibility types.</p>
          <p>Content of an explanation refers to the information Explanation Style: The explanation style refers
presented in an explanation. This information repre- to the model or strategy used for generating
explanasents a description of details related to the recommen- tions [3]. In general, the explanation style is dependent
dation process. These include, for example, user/item on the recommendation approach used in the RS, e.g.,
attributes contributing to the recommendation, inner a content-based RS produces content-based
explanaworkings of background algorithms, information re- tions [2]. In case of complex RS (e.g., deep learning
lated to the user model used as input to the RS, simi- models), however, the explanation style for a given
larities between users and/or items, and connection be- explanation may not reflect the underlying algorithm
by which the recommendations are computed [2, 3]. image, a graph, or a chart (visual). Textual
explanaPersonalizing the explanation style means to present tions are usually in form of short or long sentences
explanation in diferent styles to diferent users based using verbal elements, i.e., words, phrases or natural
on their preferences. Explanation styles have been per- language describing the reasoning behind a
recommenceived diferently in diferent domains [ 22, 23]. In this dation. Visual explanations are usually in a graphical
paper, we build on the taxonomy of explanation styles format using visual elements to explain a generated
in RS used in [2, 9, 24]. recommendation. Personalizing the explanation
format means to present the explanation in the format
• User-based Explanations: Explains similarity with preferred by the user.</p>
          <p>other users having same tastes. For example: Level of detail: The level of detail refers to the
User A with whom you share similar tastes, likes amount of information exposed in an explanation that
item B. should be presented to a user [17, 11]. Users are not
• Item-based Explanations: Explains recommended always interested in all the information that is
proitems based on item (rating) similarity. For ex- duced in an explanation [12]. Diferent users demand
ample: People who like item A in your profile diferent levels of explanation information and
explaalso like item B. nations may cause negative efects if an explanation
is dificult to understand [ 26]. Thus, it is important
• Content-based Explanations: Explains similarity to provide explanations with enough details to allow
between item features. For example: Item A has users to build accurate mental models of how the RS
similar features as of item B purchased by you. operates without overwhelming them. For example,
in an explanation, providing the exact algorithm that
• Social-based Explanations: Explains similarity be- was used to generate the explanation may be a good
tween users who know each other. For example: choice for a Machine Learning (ML) expert but not for
Your friend User B likes item A. a lay user [12]. Furthermore, the demand for level of
• Knowledge-based Explanations: Explains the de- detail in an explanation also varies with the cognitive
scription of user’s needs or interests in the con- abilities of the user [26]. Only when the users have
text of a recommendation. For example: This des- enough time to process the information and enough
tination has higher average temperature, which ability to figure out the meaning of the information,
is better for sunbathing. a higher level of detail in the explanation will lead to
a better understanding. But as soon as the amount of
• Demographic-based Explanations: Explains the information is beyond the users’ comprehension, the
use of demographic data and its connection to explanation could lead to information overload and
the recommendations. For example: This movie bring confusion [17].
was recommended to you according to your age. Intelligibility Types: When user encounters a
complex system, she might demand diferent type of
ex</p>
          <p>Explanation Scope: The explanation scope refers planatory information based on the system and its
conto the part of RS that an explanation focuses on, i.e., text [11]. Lim and Dey [5] identified several queries
input, process or output [25]. Explanation focusing on (called intelligibility types) that a user may ask of a
input tries to explain the user model which is taken smart system. These include:
as an input by the RS. Explanation focusing on process
tries to explain the working and flow of the underlying • How Explanations: demonstrate how the
underalgorithm used to generate a recommendation. Expla- lying system behind a recommendation works.
nation focusing on output tries to explain why an item How explanations are useful when Users are
inwas recommended. Diferent users demand explana- terested in knowing how the system generates
tions with a diferent scope. Not every user is interested certain recommendations. How explanations
in knowing the details of the user model or the inter- aim to answer the question: "How (under what
nals of the underlying algorithm [11]. Personalizing conditions) does the system do Y?".
the explanation scope means to present explanations
that focus on the input, output or process, according • Why Explanations: demonstrate why a
recomto the user’s preferences and needs. mendation is made for a particular user. These</p>
          <p>Explanation Format: The explanation format refers explanations cover what was the input to the
to how an explanation is displayed to the user. It can system (user model) and what logic was used to
either be in form of a text description (textual) or an generate the recommendation. Why questions
are very common by the user hence it is an es- 3.2.1. Target users
sential intelligibility requirement. Some users
might expect a very informative answer from
this why explanation and a simple explanation
may not satisfy them [5]. Why explanations aim
to answer the question: "Why did the system do
X?".</p>
          <p>The explanations can be personalized for target users
at diferent levels of granularity. A target user for an
explanation can be an individual user, category of
individuals (e.g., experts, lay users), or group of individuals
(in case of group RS).
• Why Not Explanations: (also called Contrastive 3.2.2. User model attributes</p>
          <p>Explanations) help the users in understanding
why a specific item was not recommended to To personalize the explanations for a user or a group of
them. Lim et al. [27] argued that these explana- users requires to tailor explanation based on user
modtions are useful for high-risk circumstances and els. Schneider and Handali [16] summarize diferent
when user might ask for alternative possibilities user model attributes that can be used to customize an
from the RS. Why Not explanations aim to an- explanation. These include:
swer the question: "Why did the system not do
Y?".</p>
          <p>• Prior knowledge: What an explainee knows, e.g.,
knowledge about the RS domain, expertise
regarding the RS methods to be explained
• What Íf Explanations: deal with the manipulation
of inputs to the RS. These explanations illustrate
how the manipulation of inputs afect the output
of the RS, i.e., recommendations. These
explanations involve users’ interaction with the system
when they can change an input to the RS and
want to know what will happen as a consequence.</p>
          <p>What If explanations aim to answer the question:
"What If there is a change in conditions, what
would happen?".</p>
          <p>
            • Preferences/Interests: What an explainee likes and
prefers, e.g., preferred presentation format
(visual or textual), desired level of detail of the
explanation, the time or efort an explainee wants
to invest to understand the explanation. User
preference and interests are used
interchangeably in the literature on RS
• Decision information: What information is used
by an explainee to make decisions
• What Else Explanations: demonstrate diferent
examples of the similar inputs to the RS that can • Purpose: What the explanation is used for
produce similar outputs (recommendations). Lim
and Dey [5] demonstrated that although the de- Recent studies on explainable recommendation showed
mand for these explanations is low but these are that personal characteristics have an efect on the
perhelpful in critical situations when users expect ception of explanations and that it is important to take
that the RS is doing more than shown to them, personal characteristics into account when designing
to handle a critical situation. What else explana- explanations. Examples of personal characteristics that
tions aim to answer the question: "What else the can be used as further user model attributes
accordsystem is doing?". ing to which an explanation can be tailored include
the Big Five personality traits: openness,
conscien• How To Explanations: (also called Counterfactual tiousness, extraversion, agreeableness, and neuroticism
Explanations) are basically explanations about [28, 29, 30, 31, 32], need for cognition (NFC), [
            <xref ref-type="bibr" rid="ref6">33, 10, 9</xref>
            ],
what hypothetically needs to change for the de- ease of satisfaction, visualization familiarity, domain
sired outcome to happen. How To explanations experience [9], locus of control, need for cognition,
aim to answer the question: "How to make the visualisation literacy, visual working memory,
techsystem recommend X?". savviness [
            <xref ref-type="bibr" rid="ref6">10, 18</xref>
            ], and musical sophistication [19].
          </p>
        </sec>
      </sec>
      <sec id="sec-3-2">
        <title>3.2. TO WHOM to Personalize?</title>
        <sec id="sec-3-2-1">
          <title>3.2.3. User data collection</title>
          <p>The "TO WHOM" dimension focuses on to which user(s) User data collection indicates how data is collected to
to personalize. A crucial requirement in explainable generate a model of the user for whom an explanation
RS is to build detailed user models that can be used by is personalized, in our case the explainee. This could
the system to recommend items or to provide personal- be same as the data used to generate recommendations
ized explanations. These user models are based on data or diferent. There are two ways to get explainee data
collected from the user to generate diferent attributes. to generate user models:
• Implicit data collection: refers to getting explainee There are possible refinements to these goals. For
examdata to generate a user model, implicitly through ple, in [1] satisfaction is not considered as a single goal,
user’s past activity, browser search history, mouse but is split into ease to use, enjoyment, and usefulness.
clicks, social media information, system usage Other explanation goals in RS include user engagement
history, previously consumed items, etc. This re- resulting from more confidence and transparency in
quires techniques like preference elicitation and the recommendations [34], compliance with legal
reguknowledge extraction from raw data. lations e.g., European Union’s GDPR [7], education by
allowing users to learn from the system [8], debugging
• Explicit data collection: refers to getting explainee to be able to identify wrong or unexpected
recommendata to generate a user model, explicitly by ask- dations and take control to make corrections [8]. This
ing the user to write reviews and feedback to goal is closely related to scrutability [1]. The goal might
items, giving ratings, filling out questionnaires, also be seen as obtaining an answer to diferent
intelliinterviews and surveys, liking and disliking items gibility types of explanations, e.g., what, how and why
etc. questions [27, 12].</p>
          <p>
            We consider these goals as also important in
rela3.3. WHO does the Personalization? tion to personalized explanation and we use them as
possible candidates for the "WHY" dimension in our
The "WHO" dimension focuses on who does the person- conceptualization.
alization. The literature on personalized systems
distinguish between automatic personalization by the system
providing explanations (i.e., system-driven personalized 3.5. HOW to Personalize?
explanation) and manual personalization which is done The "HOW" dimension refers to the methods used to
on-demand by the explainee, actively setting the expla- generate personalized explanations. In general,
pernation parameters, e.g., choosing the level of detail to sonalized explanations can be created using a two-step
be shown in an explanation (i.e., user-driven personal- process, namely (1) adjusting explanation properties and
ized explanation) [
            <xref ref-type="bibr" rid="ref2">20, 17, 16</xref>
            ]. (2) applying adaptation rules.
          </p>
        </sec>
      </sec>
      <sec id="sec-3-3">
        <title>3.4. WHY to Personalize?</title>
        <sec id="sec-3-3-1">
          <title>3.5.1. Adjusting explanation properties</title>
          <p>The "WHY" dimension refers to the intended goals
of personalizing an explanation. Diferent users have
diferent goals when they interact with an explainable
RS. Thus, the explanation presented to a specific user
should be personalized in a way to achieve the specific
goal(s) intended by the user. Prior work on explainable
recommendation has presented diferent explanation
goals. For example, Tintarev and Masthof [2] identified
seven goals, as follows:
the extent to which an explanation describes all of the
underlying system [36, 37]. The intelligibility type can
be personalized by adjusting the query that a user can
ask from the RS (e.g. how, why, why not, what if, what
else, how to). Finally, the choice of the explanation
style, e.g., user-based, item-based, content-based, social
explanation can also be adjusted.</p>
          <p>We used our conceptualization framework to
systematically categorize and compare the literature on
personalized explainable recommendation. Our aim was not
to conduct a systematic literature review, but rather
to show how the framework can be applied to analyze
3.5.2. Applying adaptation rules studies in the this research area. To identify relevant
A system-driven personalized explanation requires to works, we focused on recent studies that explicitly
adadjust explanation properties by taking into account dressed personalized explanation in RS. The results of
users’ preferences and personal characteristics. Thereby, the categorization of these studies are summarized in
it is crucial to decide which adaptation to apply and Table 2.
then apply that adaptation [38]. This is a
straightforward task in case of personalizing the content as an 4.1. WHAT to Personalize?
wwTliittrsscaeebdbpmotsiieohhxxnyeeoheooiiwuqngopsptastrmnneohhhiuswtoalullgsilarpaaiisrlionelcgaenleinrua.nntdozndaserhiennHtlaaliowulgeobdlnttdftetdinoslheowoieisggeto,oofgowaoesrhdochMpemnnudlppooertqw,rerfeieihavfnaop(toudrndnsnh2oreieitrdrveeiigsnuth)nigrofelsuicicdit,imedpjrnetsspionhkcintiecsiserfehetogbtmneecnasodureorunniseh.ottesvsttisirsittylowdtn“eoiltaieaohued,iowsiertlnrroclavd.wefinfweaiuet,hsnt[etideyetsbeih1scihal,heuxgcspserh9denaeishpdieerc]wroaislaedsilspb?ehlfsiaetfd.peol”neou(esndpta.xeulAu3tshgnaeatepxwpTsstl)osygegctdrautpeloiepaasieiineeropladnlneonstaraexvcndpebingtgnonacaeoaaeaoevwhatdeaenlmtdupoifrulalixitoosthetdtfeirtaspcponeaormhuviekauthllcertnedeaelaspisahedieeostseanenerrnstexosen(dxraohaotst1cpirisxhcpatpsiginfa)iilsgptpplasweonasfaef,anlotrtoroennadrainiisetrrtirdtnncspaayuoafsdhhioeuutg.paptmftgesiiirotssTtllpgorcontriioiaueeiosheehenmencot-rrws.snrdodeeesssstt--- iittttt"rcegSdbppupmszhuaouLeoxitalleseebaaegacearaepptnrtdnyneiaoosrlsdshletltit,]atadiofmeezrreecit.fontexannnfsetyoxtoimmhnrodtahtvogtanpritatanenepiolye’toliienitwtsaateaosnrznpfxuefilnnelnetdoifdseempiarffhttoadzasredraelhn.isetera)matnrtt(iAboatdnimreycReotshoyfyehs.oqaxlni"SgbneaedontetnBupS,.mywht,iau,dsaolceaio"sreaiirttriokhaWnclnctenaecnegerraptenosnmaloaakokmmue.ybrHpettsft,oripeIr.sietinkofiAtndtafileoefsiTt,eheh.sgenechnTnaf"gthte[aeFrkgohmt"uceite[soeiuracnsuaensastdtmroynerhutrreasl9eiepnxlnetderesdma]r[asapest]xtt.aUrti,ceHwih)hrleee,alInlcnas2s,nnewesnilmaeeonoetk]tstnpslrrhdrhtvanlieplioievhdydsaeesteiosalaflceiieraeslaonlopnsseht[sltii,toafisinexggzgheemeuosfareep[needoxctesfsr1xddnliioopfexoeanle6coiterarnasnblnohx]etprtueasrtmhtyaaaeapoui]n’irpentliecltn"liptiorlneovieachcgz,eoev[ltrftraenoeeiee[v3onesualnrscdnaCsdi4iorcnsysdot,tetc(,]oodneeetiitituwaaeeonr.rhmlhulaotrtelpld’xangsodaeelieesr.-------,,
the explainee.</p>
          <p>All reviewed studies focused on personalizing the
content of the provided explanation. For example, Gedikli
et al. [40] presented and discussed the results of a user mation available in the Linked Open Data (LOD) cloud.
study where recommendation systems were provided Their user study results revealed that their strategy
outwith diferent types of explanation. The study revealed performed both a non-personalized explanation
basethat the content-based tag cloud explanations were line and a popularity-based one. McInerney et al. [34]
efective and well accepted by the majority of users. presented a method (Bart) that combines bandits and
Tintarev and Masthof [45] focused on personalized recommendation explanations. This method is able to
feature-based explanations that described item features, jointly learn which explanations each user responds to
tailored to the user’s interests. Musto et al. [42] pre- (personalized explanation), and learn the best content
sented a framework for generating personalized natural to recommend for each user (personalized
recommenlanguage explanations of the suggestions produced by a dation). The conducted experiments revealed that
pergraph-based recommendation model based on the infor- sonalizing explanations and recommendations provides
a significant increase in estimated user engagement. Lu textual explanation format by explaining the
reasonet al. [41] presented a multi-task learning framework ing behind an explanation in natural language. Only
that simultaneously learns to perform rating prediction few studies have used a visual explanation format. For
and generate personalized recommendation explana- example, Kouki et al. [9] have used Venn diagrams and
tion. They employed a matrix factorization model for static cluster dendrograms, Gedikli et al. [40] have used
rating prediction, and a sequence-to-sequence learning tag clouds, and Quijano-Sanchez et al. [44] have used
model for explanation generation by generating person- graphical representation of images to present
explaalized reviews for a given recommendation-user pair as nations. Finally, none of the studies have worked on
they consider user-generated reviews as explanations personalizing intelligibility types of explanations
deof the ratings given by users. Inspired by how people pending on user profile.
explain word-of-mouth recommendations, Chang et al. In summary, it can be observed that personalizing
[39] designed a process, combining crowd-sourcing the content of an explanation to each user’s data and
and computation, that generates personalized natural personality is dominant in the literature on explainable
language explanations. And, Chen et al. [43] provided recommendation. By contrast, personalized
explanapersonalized explanations visually by highlighting dif- tions that focus on tailoring a certain explanation
deferent parts of an item based on user preferences. sign choice, such as explanation scope, format, or level</p>
          <p>
            Considering the design choices (i.e., explanation style, of detail are under-explored and deserve more research
scope, format, level of detail, intelligibility types) which in the future.
could also be personalized based on user profiles, most
of the studies have kept design choices fixed in ex- 4.2. TO WHOM to Personalize?
planations. Only the work presented in [24] takes the
personalized explanation to the explanation style design Related to the "To WHOM" dimension, which identifies
choice level. The authors proposed a hybrid method of the target users of a personalized explanation (i.e.,
indipersonalized explanation of recommendations, which vidual user, category of individuals, or group of
individcombines basic explanation styles to provide the appro- uals), it has been observed that almost all the reviewed
priate type of personalized explanation to each user. studies have personalized for individual users. Only the
Based on this method, each user will be given an expla- study in [44] provided explanations targeting a group
nation adapted to what most impressed her (i.e., expla- of users. The authors argued that adding a social
comnation style which she prefers). Furthermore, only the ponent to explanations in group recommenders can
works in [
            <xref ref-type="bibr" rid="ref6">10</xref>
            ] and [17] reported personalizing the level enhance the impact that explanations have on users’
of detail in an explanation depending on how much de- likelihood to follow the recommendations and used
extail the user prefers to see in an explanation. Millecamp planations like: “Although we have detected that your
et al. [
            <xref ref-type="bibr" rid="ref6">10</xref>
            ] developed a music RS that not only allows preference for this item is not very high, your friends
users to choose whether or not to see the explanations X and Y really like it. Besides, we have detected that
by using a "Why?" button but also to select the level they usually don’t give in".
of detail by clicking on a "More/Hide" button. Guesmi In terms of user model attributes and user data
collecet al. [17] developed a transparent Recommendation tion, most of the studies have focused on user
preferand Interest Modeling Application (RIMA) that pro- ences or interests to personalize the explanations and
vides on-demand personalized explanations of both the collected data implicitly to generate user models. For
interest models and the recommendations with three example, in the study by Kouki et al. [9], a user model
diferent levels of of detail (i.e., basic, intermediate, ad- was created based on users’ music preferences, Chang
vanced). et al. [39] generated a user model based on users’
pref
          </p>
          <p>
            In all reviewed studies, there was no personalization erence of movies modeled from their activities with
related to explanation scope, explanation format, or in- the system. Data was also collected implicitly through
telligibility types. None of the studies has focused on users’ listening history to generate their music interests
varying these design choices depending on the user in [34], through users’ likes to generate their movie
profile. In terms of the explanation scope design choice, preferences [42], through users’ interactions, readings,
all reviewed studies have focused only on explaining brought and clicked books, to generate user models
the output of the RS (i.e., recommendation), none of based on preferences [24]. Furthermore, a user model
them has tried to explain the input of the RS (i.e., user was created by [
            <xref ref-type="bibr" rid="ref6">10</xref>
            ] based on users’ music preferences
model) or the process (i.e., algorithm used used to gen- generated implicitly based on listening history. Zhang
erate a recommendation). Concerning the explanation et al. [6] generated a user model based on user
prefformat design choice, most of the studies have used a erences collected implicitly through applying
phraselevel sentiment analysis on user reviews and opinions. eficiency [ 39, 40], confidence [
            <xref ref-type="bibr" rid="ref6">9, 10</xref>
            ], and user
engageFor visual explanations, Chen et al. [43] used users’ ment [34, 42]. Only the study in [25] aimed to
proattention and users’ visual preferences to generate user vide personalized explanations to achieve scrutability.
models, used to personalize visual explanations. Sim- Moreover, only two studies aimed at comparing
perilarly, Gedikli et al. [40] created a used model based sonalized and non-personalized variants of an
explaon user’s preferences of movies, however, users were nation. The study in [40] found that content-based tag
explicitly asked to provide overall rating for at least 15 cloud explanations were particularly helpful to increase
items from a collection of 1000 movies, to record their user satisfaction as well as the user-perceived level of
preferences. transparency thanks to its personalized variant.
How
          </p>
          <p>
            Only the work by Quijano-Sanchez et al. [44] gener- ever, they found that personalization was detrimental
ated a user model based on user preferences collected to efectiveness. Similarly, Tintarev and Masthof [45]
implicitly from users’ activities in Facebook as well investigated the impact of personalizing simple
featureas personal characteristics (e.g., cooperative, assertive) based explanations on efectiveness and satisfaction.
obtained explicitly through a personality evaluation They also reported that their personalization method
test to get users’ behaviors, and social information re- hindered efectiveness, but on the other hand increased
lated to friends and their preferences to generate per- the satisfaction with the explanations. More studies
sonalized explanations where each user will receive are needed to investigate the benefits of personalized
a diferent explanation of the group recommendation explanations compared to non-personalized variants
presented by the system. In general, it can be observed related to diferent goals. Furthermore, only two
studthat there is less focus on personal characteristics as ies investigated the efects of personal characteristics
a user model attribute that can be collected (explicitly on the perception of explanations in terms of
persuaor implicitly) and used to personalize the explanations. siveness [9] and confidence [
            <xref ref-type="bibr" rid="ref6">10</xref>
            ]. As diferent design
This represents an interesting future research direction. choices will be afected by the user type, more research
seems required to understand the interaction efects
4.3. WHO does the Personalization? of design choice and user type on the perception of
personalized explainable RS with regard to diferent
explanation goals.
          </p>
          <p>
            Related to the "WHO" dimension, we have observed
that only the works presented in [
            <xref ref-type="bibr" rid="ref6">10</xref>
            ] and [17] have
followed a user-driven personalized explanation approach 4.5. HOW to Personalize?
by providing on-demand explanations with varying
level of details. All the other works have focused on In terms of adjusting explanation properties, the
masystem-driven personalized explanation, mainly to au- jority of the reviewed studies only focused on
persontomatically adapt the content of the explanation, based alizing the explanations by adjusting the content as
on users’ preferences. This opens a new avenue of explanation property. In most cases, a personalized
exresearch in the field of explainable recommendation, planation is generated by highlighting the similarities
and researchers should try to fill in this gap. More re- between users (user-based explanation) [40] or items
search is needed to focus on user-driven personalized (item-based explanation) [41, 34, 42, 43]. Following
explanation in RS by having the users in the loop and a content-based explanation approach, the studies in
giving them control to steer the explanation process. [
            <xref ref-type="bibr" rid="ref6">39, 10, 6, 17</xref>
            ] generated personalized explanations by
Furthermore, there is a need to follow a system-driven highlighting feature similarities between items. And,
personalized explanation approach, that not only fo- the study in [44] used a social-based explanation
apcuses on adapting the content of an explanation, but proach to explain individual and group
recommendaalso the design choice. tions, by highlighting similar users in a social circle.
However, only a few studies have worked on
personal4.4. WHY to Personalize? izing the explanations by adjusting the design choice
as explanation property. Among these studies, Svrcek
The next dimension is "WHY" to personalize?" which et al. [24] worked on peronalizing explanation style,
refers to possible goals of providing an explanation. and Millecamp et al. [
            <xref ref-type="bibr" rid="ref6">10</xref>
            ] and Guesmi et al. [17] have
The most common goals evaluated in the reviewed personalized level of detail as explanation property,
studies are user satisfaction [
            <xref ref-type="bibr" rid="ref6">9, 39, 40, 34, 10, 44, 25, 45</xref>
            ], by varying only completeness (show/hide explanation)
transparency [40, 24, 42, 9, 25], persuasiveness [9, 40, and both soundness and completeness (basic,
interme42, 44], trust [39, 42, 24], efectiveness [ 39, 40, 42, 45], diate, advanced explanation), respectively.
Referring to applying adaptation rules, only few
studies have worked on proposing and/or applying adap- future work, we will leverage the proposed framework
tation rules to personalize an explanation. In order to to conduct a thorough systematic literature review to
assign the appropriate explanation style for the spe- gain more insights into the domain of personalized
cific user, Svrcek et al. [24] proposed and applied an explanations in recommender systems.
adaptation rule, following a test-then-train approach
that identifies if users prefer a certain explanation style,
based on a continuous monitoring of the user clicks References
on each item explained by diferent styles. As a
result, the users obtain more explanations generated by
their preferred style. Kouki et al. [9] and Millecamp
et al. [
            <xref ref-type="bibr" rid="ref6">10</xref>
            ] also proposed adaptation rules without
applying them. In [9], only the content of an
explanation is personalized to each user’s data and personality,
while the explanation styles are kept fixed. The
authors, however, investigated the efect of personality on
the perception of explanations and found that (1) calm
participants (low neuroticism) preferred
popularitybased explanations, while anxious participants (high
neuroticism) preferred item-based explanations.
Likewise, neurotic participants, showed a slight preference
for item-based explanations, (2) open participants were
persuaded by many explanations, while conscientious
participants preferred fewer. The work in [
            <xref ref-type="bibr" rid="ref6">10</xref>
            ] did not
follow a system-driven explanation approach. The
authors, however, investigated the efect of personal
characteristics on the perception of explanations and found
that participants with a low need for cognition (NFC)
were more confident about their playlist when
recommendations were explained, as opposed to those with
a high NFC. In general, there is a lack of research on
adaptation rules to tailor explanations in RS to users
with diferent preferences and personal characteristics.
          </p>
          <p>Thus, more user studies need to be conducted on the
same lines, to come up with concrete adaptation rules
that can be used to personalize system-driven
explanations.
[1] I. Nunes, D. Jannach, A systematic review and
taxonomy of explanations in decision support and
recommender systems, User Modeling and
User</p>
          <p>Adapted Interaction 27 (2017) 393–444.
[2] N. Tintarev, J. Masthof, Explaining
recommendations: Design and evaluation, in: Recommender
systems handbook, Springer, 2015, pp. 353–382.
[3] Y. Zhang, X. Chen, Explainable recommendation:</p>
          <p>A survey and new perspectives, arXiv preprint
arXiv:1804.11192 (2018).
[4] M.-W. O. Dictionary, Explain, 2021. URL: https://</p>
          <p>www.merriam-webster.com/dictionary/explain.
[5] B. Y. Lim, A. K. Dey, Assessing demand for
intelligibility in context-aware applications, in:
Proceedings of the 11th international conference on</p>
          <p>Ubiquitous computing, 2009, pp. 195–204.
[6] Y. Zhang, G. Lai, M. Zhang, Y. Zhang, Y. Liu, S. Ma,</p>
          <p>
            Explicit factor models for explainable
recommendation based on phrase-level sentiment analysis,
in: Proceedings of the 37th international ACM
SIGIR conference on Research &amp; development in
information retrieval, 2014, pp. 83–92.
[7] M. Naiseh, N. Jiang, J. Ma, R. Ali,
Personalising explainable recommendations: literature and
conceptualisation, in: World Conference on
Information Systems and Technologies, Springer, 2020,
pp. 518–533.
[8] D. Jannach, M. Jugovac, I. Nunes, Explanations
and user control in recommender systems, in:
Personalized Human-Computer Interaction, De
5. Conclusion Gruyter Oldenbourg, 2019, pp. 133–156.
[9] P. Kouki, J. Schafer, J. Pujara, J. O’Donovan,
In this paper, we presented a multi-dimensional concep- L. Getoor, Personalized explanations for hybrid
tualization framework for personalized explanations recommender systems, in: Proceedings of the
in recommender systems, based on five dimensions: 24th International Conference on Intelligent User
WHAT to personalize in an explanation, TO WHOM to Interfaces, 2019, pp. 379–390.
personalize, WHO does the personalization, WHY an [
            <xref ref-type="bibr" rid="ref6">10</xref>
            ] M. Millecamp, N. N. Htun, C. Conati, K. Verbert,
explanation should be personalized, and HOW to per- To explain or not to explain: the efects of personal
sonalize an explanation. Through this work we aimed characteristics when explaining music
recommento (1) provide researchers with a structured way to dations, in: Proceedings of the 24th International
organize current and future research on personalized Conference on Intelligent User Interfaces, 2019,
explainable recommendation, (2) provide an overview pp. 397–407.
of what has been done in the domain of personalized ex- [11] S. Mohseni, N. Zarei, E. D. Ragan, A
multidisplanations in RS so that more knowledge can be built on ciplinary survey and framework for design and
top of it, and (3) identify research gaps in this area. As evaluation of explainable ai systems, ACM
Transactions on Interactive Intelligent Systems (TiiS) chology 5 (2014) 373.
          </p>
          <p>11 (2021) 1–45. [23] R. Larasati, A. De Liddo, E. Motta, The efect
[12] T. Miller, Explanation in artificial intelligence: of explanation styles on user’s trust., in:
ExSSInsights from the social sciences, Artificial intelli- ATEC@ IUI, 2020.</p>
          <p>gence 267 (2019) 1–38. [24] M. Svrcek, M. Kompan, M. Bielikova, Towards
[13] V. Arya, R. K. Bellamy, P.-Y. Chen, A. Dhurand- understandable personalized recommendations:
har, M. Hind, S. C. Hofman, S. Houde, Q. V. Hybrid explanations, Computer Science and
InLiao, R. Luss, A. Mojsilović, et al., One expla- formation Systems 16 (2019) 179–203.
nation does not fit all: A toolkit and taxonomy [25] M. Guesmi, M. Chatti, Y. Sun, S. Zumor, F. Ji,
of ai explainability techniques, arXiv preprint A. Muslim, L. Vorgerd, S. Joarder, Open, scrutable
arXiv:1909.03012 (2019). and explainable interest models for transparent
[14] A. Jung, P. H. Nardelli, An information-theoretic recommendation, in: IUI Workshops, 2021.
approach to personalized explainable machine [26] R. Zhao, I. Benbasat, H. Cavusoglu, Do users
learning, IEEE Signal Processing Letters 27 (2020) always want to know more? investigating the
825–829. relationship between system transparency and
[15] N. Kuhl, J. Lobana, C. Meske, Do you comply user’s trust in advice-giving systems (2019).
with ai?–personalized explanations of learning [27] B. Y. Lim, Q. Yang, A. M. Abdul, D. Wang, Why
algorithms and their impact on employees’ com- these explanations? selecting intelligibility types
pliance behavior, arXiv preprint arXiv:2002.08777 for explanation goals., in: IUI Workshops, 2019.
(2020). [28] L. R. Goldberg, An alternative" description of
[16] J. Schneider, J. Handali, Personalized explanation personality": the big-five factor structure., Journal
in machine learning: A conceptualization, arXiv of personality and social psychology 59 (1990)
preprint arXiv:1901.00770 (2019). 1216.
[17] M. Guesmi, M. A. Chatti, L. Vorgerd, S. Joarder, [29] S. D. Gosling, P. J. Rentfrow, W. B. Swann Jr, A
S. Zumor, Y. Sun, F. Ji, A. Muslim, On-demand very brief measure of the big-five personality
dopersonalized explanation for transparent recom- mains, Journal of Research in personality 37 (2003)
mendation, in: Adjunct Proceedings of the 29th 504–528.</p>
          <p>ACM Conference on User Modeling, Adaptation [30] M. Tkalcic, L. Chen, Personality and
recomand Personalization, 2021, pp. 246–252. mender systems, in: Recommender systems
hand[18] M. Millecamp, N. N. Htun, C. Conati, K. Verbert, book, Springer, 2015, pp. 715–739.</p>
          <p>What’s in a user? towards personalising trans- [31] M. P. Graus, B. Ferwerda, Theory-grounded user
parency for music recommender interfaces, in: modeling for personalized hci, in: Personalized
Proceedings of the 28th ACM Conference on User human-computer interaction, De Gruyter
OldenModeling, Adaptation and Personalization, 2020, bourg, 2019, pp. 1–30.</p>
          <p>
            pp. 173–182. [32] S. T. Völkel, R. Schödel, D. Buschek, C. Stachl,
[19] M. Martijn, C. Conati, K. Verbert, “knowing me, Q. Au, B. Bischl, M. Bühner, H. Hussmann,
Opknowing you”: personalized explanations for a portunities and challenges of utilizing personality
music recommender system, User Modeling and traits for personalization in hci, in: Personalized
User-Adapted Interaction (2022) 1–38. Human-Computer Interaction, De Gruyter
Olden[
            <xref ref-type="bibr" rid="ref2">20</xref>
            ] H. Fan, M. S. Poole, What is personalization? bourg, 2019, pp. 31–64.
          </p>
          <p>perspectives on the design and implementation [33] S. Naveed, T. Donkers, J. Ziegler,
Argumentationof personalization in information systems, Jour- based explanations in recommender systems:
nal of Organizational Computing and Electronic Conceptual framework and empirical results, in:
Commerce 16 (2006) 179–202. Adjunct Publication of the 26th Conference on
[21] M. Guesmi, M. A. Chatti, L. Vorgerd, S. Joarder, User Modeling, Adaptation and Personalization,
Q. U. Ain, T. Ngo, S. Zumor, Y. Sun, F. Ji, A. Mus- 2018, pp. 293–298.
lim, Input or output: Efects of explanation focus [34] J. McInerney, B. Lacker, S. Hansen, K. Higley,
on the perception of explainable recommenda- H. Bouchard, A. Gruson, R. Mehrotra, Explore,
tion with varying level of details, in: IntRS’21: exploit, and explain: personalizing explainable
Joint Workshop on Interfaces and Human Deci- recommendations with bandits, in: Proceedings
sion Making for Recommender Systems, 2021. of the 12th ACM conference on recommender
sys[22] S. Wilkinson, Levels and kinds of explanation: tems, 2018, pp. 31–39.</p>
          <p>lessons from neuropsychiatry, Frontiers in psy- [35] T. Munzner, Visualization analysis and design,</p>
        </sec>
      </sec>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          CRC press,
          <year>2014</year>
          . [36]
          <string-name>
            <given-names>T.</given-names>
            <surname>Kulesza</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Burnett</surname>
          </string-name>
          , W.-K. Wong, S. Stumpf,
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          <source>of the 20th international conference on intelligent</source>
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          <source>user interfaces</source>
          ,
          <year>2015</year>
          , pp.
          <fpage>126</fpage>
          -
          <lpage>137</lpage>
          . [37]
          <string-name>
            <given-names>T.</given-names>
            <surname>Kulesza</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S.</given-names>
            <surname>Stumpf</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Burnett</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S.</given-names>
            <surname>Yang</surname>
          </string-name>
          ,
          <string-name>
            <surname>I. Kwan</surname>
          </string-name>
          ,
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          els,
          <source>in: 2013 IEEE Symposium on visual languages</source>
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          <article-title>and human centric computing</article-title>
          , IEEE,
          <year>2013</year>
          , pp.
          <fpage>3</fpage>
          -
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          10. [38]
          <string-name>
            <given-names>A.</given-names>
            <surname>Paramythis</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S.</given-names>
            <surname>Weibelzahl</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J.</given-names>
            <surname>Masthof</surname>
          </string-name>
          , Layered
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          <string-name>
            <surname>User-Adapted</surname>
            <given-names>Interaction</given-names>
          </string-name>
          20 (
          <year>2010</year>
          )
          <fpage>383</fpage>
          -
          <lpage>453</lpage>
          . [39]
          <string-name>
            <given-names>S.</given-names>
            <surname>Chang</surname>
          </string-name>
          ,
          <string-name>
            <given-names>F. M.</given-names>
            <surname>Harper</surname>
          </string-name>
          ,
          <string-name>
            <given-names>L. G.</given-names>
            <surname>Terveen</surname>
          </string-name>
          , Crowd-
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          <article-title>for recommendations</article-title>
          ,
          <source>in: Proceedings of the 10th</source>
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          <source>ACM Conference on Recommender Systems</source>
          ,
          <year>2016</year>
          ,
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          pp.
          <fpage>175</fpage>
          -
          <lpage>182</lpage>
          . [40]
          <string-name>
            <given-names>F.</given-names>
            <surname>Gedikli</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D.</given-names>
            <surname>Jannach</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Ge</surname>
          </string-name>
          ,
          <article-title>How should i ex-</article-title>
        </mixed-citation>
      </ref>
      <ref id="ref11">
        <mixed-citation>
          <string-name>
            <surname>of</surname>
          </string-name>
          Human-Computer Studies
          <volume>72</volume>
          (
          <year>2014</year>
          )
          <fpage>367</fpage>
          -
          <lpage>382</lpage>
          . [41]
          <string-name>
            <given-names>Y.</given-names>
            <surname>Lu</surname>
          </string-name>
          ,
          <string-name>
            <given-names>R.</given-names>
            <surname>Dong</surname>
          </string-name>
          ,
          <string-name>
            <given-names>B.</given-names>
            <surname>Smyth</surname>
          </string-name>
          ,
          <article-title>Why i like it: multi-task</article-title>
        </mixed-citation>
      </ref>
      <ref id="ref12">
        <mixed-citation>
          <source>in: Proceedings of the 12th ACM Conference on</source>
        </mixed-citation>
      </ref>
      <ref id="ref13">
        <mixed-citation>
          <string-name>
            <given-names>Recommender</given-names>
            <surname>Systems</surname>
          </string-name>
          ,
          <year>2018</year>
          , pp.
          <fpage>4</fpage>
          -
          <lpage>12</lpage>
          . [42]
          <string-name>
            <given-names>C.</given-names>
            <surname>Musto</surname>
          </string-name>
          ,
          <string-name>
            <given-names>F.</given-names>
            <surname>Narducci</surname>
          </string-name>
          ,
          <string-name>
            <given-names>P.</given-names>
            <surname>Lops</surname>
          </string-name>
          , M. De Gemmis,
        </mixed-citation>
      </ref>
      <ref id="ref14">
        <mixed-citation>
          <article-title>data cloud</article-title>
          ,
          <source>in: Proceedings of the 10th ACM</source>
        </mixed-citation>
      </ref>
      <ref id="ref15">
        <mixed-citation>
          <source>Conference on Recommender Systems</source>
          ,
          <year>2016</year>
          , pp.
        </mixed-citation>
      </ref>
      <ref id="ref16">
        <mixed-citation>
          151-
          <fpage>154</fpage>
          . [43]
          <string-name>
            <given-names>X.</given-names>
            <surname>Chen</surname>
          </string-name>
          ,
          <string-name>
            <given-names>H.</given-names>
            <surname>Chen</surname>
          </string-name>
          ,
          <string-name>
            <given-names>H.</given-names>
            <surname>Xu</surname>
          </string-name>
          ,
          <string-name>
            <given-names>Y.</given-names>
            <surname>Zhang</surname>
          </string-name>
          ,
          <string-name>
            <given-names>Y.</given-names>
            <surname>Cao</surname>
          </string-name>
          ,
          <string-name>
            <given-names>Z.</given-names>
            <surname>Qin</surname>
          </string-name>
          ,
        </mixed-citation>
      </ref>
      <ref id="ref17">
        <mixed-citation>
          recommendation,
          <source>in: Proceedings of the 42nd In-</source>
        </mixed-citation>
      </ref>
      <ref id="ref18">
        <mixed-citation>
          <source>and Development in Information Retrieval</source>
          ,
          <year>2019</year>
          ,
        </mixed-citation>
      </ref>
      <ref id="ref19">
        <mixed-citation>
          pp.
          <fpage>765</fpage>
          -
          <lpage>774</lpage>
          . [44]
          <string-name>
            <given-names>L.</given-names>
            <surname>Quijano-Sanchez</surname>
          </string-name>
          ,
          <string-name>
            <given-names>C.</given-names>
            <surname>Sauer</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J. A.</given-names>
            <surname>Recio-Garcia</surname>
          </string-name>
          ,
        </mixed-citation>
      </ref>
      <ref id="ref20">
        <mixed-citation>
          <source>Expert Systems with Applications</source>
          <volume>76</volume>
          (
          <year>2017</year>
          )
          <fpage>36</fpage>
          -
          <lpage>48</lpage>
          . [45]
          <string-name>
            <given-names>N.</given-names>
            <surname>Tintarev</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J.</given-names>
            <surname>Masthof</surname>
          </string-name>
          , Evaluating the efective-
        </mixed-citation>
      </ref>
      <ref id="ref21">
        <mixed-citation>
          <string-name>
            <given-names>User</given-names>
            <surname>Modeling</surname>
          </string-name>
          and
          <string-name>
            <surname>User-Adapted Interaction</surname>
          </string-name>
          22
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>