Even though Recommender Systems (RS) have been widely applied in various financial domains such as Robo-advisors (RA), these systems still operate as a black box with no or limited explanations. Even in cases, where explanations are provided, such systems are mostly designed from the developers' perspective where the user needs and perspective of explanations are not taken into account. In this work, we aim to address the challenges of designing eXplainable Robo-Advisors (XRA) - by adopting a user-centric methodology. For this purpose, we applied a mixed-method approach, in which we conducted three qualitative focus group discussions (FGD) and supplemented the results with a quantitative survey insight. More specifically, we made two major contributions: 1) We extended the existing explanation categories to contextualize it for the financial domain - by identifying the user's specific needs for explainability in the context of the financial domain, 2) We quantified the user preferences of specific explanations with regard to the financial domain and explainable RA - by evaluating the user's personal relevance (PRE) and perceived quality (PQE) of explanations.
With recent advances in artificial intelligence (AI) and machine learning (ML), a growing number
of complex decision-making tasks are delegated to software systems and applications. In this
context, recommender systems (RS) have been widely used in various financial services domains
(such as online banking, loan, stocks, asset allocation, and portfolio management) – to support
people in complex financial decisions e.g., financial investment or retirement planning [
Compared to other domains, such as movie or song recommendations, the financial domain
has several peculiarities: the finance world is complex where the risk involved in wrong decisions
is high and the average financial literacy of common users might be quite low [
In recent years, Robo-Advisors (RA) [
To deal with the black box situation, various authorities such as the European Commission
(EC) and finance regulators have demanded better transparency and explainability of such
system [
Currently, explanations of such financial RS have been commonly designed based on the
developer’s intuition for a "good explanation" [
To address the challenges of designing eXplainable Robo-Advisors (XRA) [
RQ1: What are the domain-specific user needs for explanations w.r.t. RA systems?
RQ2a: How important are the explanations for the users when interacting with RA
systems?
RQ2b: How the quality of explanations are perceived by the users when interacting with
RA systems?
These research questions aim to address the following: (1) the specification of explanation
taxonomies [
In this work, we adopted a mixed-method approach: To answer the "What" aspect of explanations, we first conducted three qualitative focus groups discussion to explore the domain-specific need for explanations by users (RQ1).
To answer the "How" aspect of explanations, in the second step, we applied a quantitative online survey to evaluate the personal importance (RQ2a) and the perceived quality (RQ2b) of explanations identified in the first step.
Overall, this work contributes in the following ways: • Contextualizing and extending the theoretically-driven general taxonomies of explanations with regard to the financial domain and XRA. This theoretical contribution of a user-centric taxonomy provides in-depth insights into the user’s understanding of what constitutes a good explanation and why it is needed. • Quantifying user preferences of explanations with regard to the financial domain and XRA. This practical contribution aims to help designers to identify and prioritize the user need and relevance of specific explanations in the context of designing explainable ifnancial RS.
In the following, we will discuss relevant related work. Next, we will describe the methodologies for both the qualitative study using focus group discussions and a quantitative online survey. Afterward, we will present the results and insights from both qualitative and quantitative studies. Finally, we will critically discuss the findings of both studies and conclude the work by providing an outlook for future research.
In the recommender systems (RS) literature, a number of explanation approaches for RS have
been proposed [
Diferent types and classifications of explanations have been proposed in the RS literature.
A work presented in [
In line with the type of information sources that generate explanations, a more recent work,
presented in [
RS for stock investments have existed since the 1990s. While early work focused on individual
stocks [
Robo Advisors (RA) have become the most prominent example of portfolio management
in the mass market [17]. As they currently work as black boxes, the call for explainable RAs
becomes a serious issue in society and as well as in academia [
To address the issue of the black box situation, in the last years technical studies have been
carried out [
Both works [
Ben et al. [
Schemmer et al. [19] implemented an AI advice system, which estimates the price of a property to prevent buying overvalued houses. Their system estimates the price as the output by various input features, such as the year of construction or living area. The system gives inputoutput explanations by showing the feature importance calculated by the LIME framework [20]. The researchers conducted two focus groups, where the quality feedback given indicates that explanations, in general, will be useful but for instance, the concept of feature importance was dificult to understand by users and some also found it confusing to get multiple explanations.
Deo et al. [
Overall, even though the application of RS can be found extensively in various financial
domains e.g., online banking, loan, insurance, real estate, stocks, and asset allocation, to the
best of our knowledge, providing explanations in financial RS is underexplored in current
explainable RS literature. This holds especially true for RAs, which have become popular in
recent years. There is not just a lack of quantitative studies evaluating XRA, but also qualitative
studies understanding users’ perceptions and the need for explanations in the financial domain.
Regarding this, our work complements the research presented in [
In this work, we used a mixed-method approach, supplementing the qualitative focus group discussions with a quantitative online survey. In the following, we will describe both methodologies in detail.
By its very nature, users’ needs and personal meanings are subjective and open-ended. For this reason, we used a qualitative research approach using Focus Group Discussions (FGD) [21]. As a qualitative research technique, FGD has the ability to explore topics in-depth and determine the constituting elements of the phenomenon (the "What") and their meaning (the "Why") from a user perspective [22]. Such an explorative approach is particularly suitable when there is a lack of predetermined hypotheses, theoretical concepts need to be contextualized regarding a specific domain, or diferent perspectives on the issue need to be explored.
We conducted three focus group discussions with the following stakeholders to consider various perspectives about designing and using an eXplainable Robo-Advisor: 1. Domain Experts – participants with background and/or expertise in the financial domain. 2. HCI Experts – participants with background and/or expertise in Human-Centered
Interaction and eXplainable AI (XAI). 3. Common Users – participants with no or limited background and/or expertise in the ifnancial domain and XAI.
We used the convenience sampling method [23] to recruit participants for the diferent focus groups. In total, we recruited 13 participants (Domain Experts: 5, HCI Experts: 4, Common Users: 4). All of them were well-educated (at least a university degree), but only the domain experts had long experience with the financial domain (see Table 1).
The domain experts and common users focus groups were conducted online using video conferencing software with cameras switched on. The whole session was video recorded with the consent of all participants. To conduct the sessions online for the two groups, for each group we organized a virtual whiteboard using Miro1. The HCI experts focus group was conducted in presence and audio of the entire session was recorded with the consent of all participants.
The procedure for all focus groups was similar, but we had to make some adaptations for the special needs of the online FGD. To conduct the focus group discussions, we followed the steps and procedures described below which are also shown in Figure 1.
1. Welcome: Participants were welcomed and were asked to introduce themselves in terms of their background knowledge as well as their expertise in the financial domain. 2. Introduction I: A short introduction on the purpose of the focus group discussion was provided as well as the expected outcomes of the study were explained to the participants. 3. Introduction II: In case of the Onsite-FGD, participants were given a brief explanation of how to use Post-Its to record their thoughts. In the case of the Online-FGD, participants were given a brief introduction to the Miro Board to collect thoughts and structure the discussion. 4. Walk through: Participants were given a walk-through to a replica of a real (German) Robo-Advisor bevestor2. Our RA replica was an interactive mock-up that was built using Axure3, with no real functionality but allowing participants to interact with it. 5. Evaluate and Discussion: Participants were then asked to explore and critically analyze the prototype. The following session was designed for brainstorming and reflects a discussion covering three steps: 2https://bevestor.de/ 3https://ancnjj.axshare.com • Brainstorming – in which participants were required to write down any questions they want the system to answer or explain to them, related to any aspect of the system that is unclear to them. • Presenting – in which participants were asked to present and explain the motivation behind each written question. • Clustering – in which participants were then asked to create clusters of similar questions and label the clusters at the end.
Each focus group was carried out by two researchers. One researcher took on the role of a moderator. Her task was to explain the individual steps and stimulate the discussion. Beyond this, she refrained from interfering. For instance, she took care not to participate in the discussion herself or to evaluate the opinions and views of participants. The second researcher took on the role of an observer by taking supplementary field notes in addition to the audio and video recordings.
All FGDs were audio recorded and transcribed using amberscript4. The analysis was done with the help of MAXQDA5 following the principles of the Thematic Analysis [24]. During analysis, each FGD data set was coded by two independent researchers. Subsequently, the two researchers discussed and analyzed each FGD jointly. The codes were consolidated and used to define themes, which capture and summarize the core issue of coherent and meaningful pattern [24]. Themes were discussed in joint interpretation workshops by the group of authors to gain a mutual understanding of the material.
Regarding the coding and defining themes, three principles have been discussed in the literature: First, the principle of emergent, inductive, data-driven, and button-up coding [25], where the researcher starts with a blank sheet and all codes resulted only from reading and interpreting the empirical data. Second, the principle of deductive, theory-driven, and top-down coding [25] starts with a pre-defined category system, where the aim of the analysis is mainly to validate and illustrate the category system based on the empirical data. Third, the principle of abductive, reflexive, and counter-current coding that integrates the elements of both approaches [24]. The approach seems to sensitize researchers in their empirical work but needs to be flexible enough to allow for new experiences.
In our research, we adopted this third, abductive coding principle, where the theoretical concepts outlined in Section 2.1 served as a sensitizing lens to analyze our data but allowed us to open to new experiences that could not be well-explained by existing explanation categories. As a result, our coding represents a dialog between the theory and the empirical data, by going through the data and themes several times to refine them.
The FGD is an established methodology in qualitative research for stimulating group discussion and exploring a topic from diferent perspectives [ 26]. However, the method is limited in terms 4https://www.amberscript.com/en/ 5https://www.maxqda.com/ of assigning specific statements to a person and quantitatively measuring personal perceptions and preferences at an individual level. Therefore, we conducted a follow-up online survey, to quantitatively evaluate the user perception of the RA replica in terms of its explainability and to quantify the user relevance for diferent types of explanations they want in the RA. To address both RQ2a and RQ2b, we asked our participants to evaluate both, the Personal Relevance of Explanations (PRE) as well as Perceived Quality of Explanations (PQE). Both dimensions have been evaluated in terms of the explanation categories identified in the FGD (see Section 4), i.e. Recommender Explanation, Domain-Specific Information , and Shared Understanding.
We adopted items from the User-Centric Evaluation framework [27] to get a comprehensive evaluation of the Recommender Explanation category. As this framework does not cover the aspects of Domain-Specific Information and Shared Understanding, for these categories we created items on our own, where we tried to use the wordings from the FGDs as close as possible. For evaluating the PRE, the questionnaire items were rated on a five-point scale from "Not at all important" to "Very important" (see Table 3 and Table 4). For evaluating PQE, all questionnaire items were rated on a five-point Likert response scale from "Strongly Disagree" to "Strongly Agree" (see Table 5).
To conduct the online survey, the following steps and procedures were followed: 1. Introduction: A short introduction on the purpose of the online survey and the procedure of the survey was provided to the participants. 2. Prototype Exploration: Participants were presented with the same RA replica that they saw before in the FGD. 3. Evaluation: Participants were then asked to explore and critically analyze the prototype and were asked to return to the questionnaire after exploring the prototype, to answer a series of questions.
Except for P09, all the other 12 participants from the FGDs completely filled out the survey.
We identified the following major findings from the focus group studies: 1) The users have multiple perceptions and understanding of explanations, 2) The empirical view of explanations is diferent from the theoretical view of explanations, and 3) Overall, there is a general structure of the empirical view about explanations for all three focus groups. These findings are reported in the following sections.
To address the RQ1, we followed the labels of clusters given by each focus group during the
Clustering phase. The labeled clusters were: "Explanation" (given by domain experts and
common users) and "Definition and Explanation of Terms" (given by HCI experts). In addition,
common users clustered several cards that explicitly used the term "Explaining", but the cluster
Recommender Explanations
16 7 6 3 IImmppaacctt ooff itnhpeuintpountroisnkpcolarstfsoifliicoagtieonneration
UOsuetrpuPtreEfxeprelannceatoiornIn[p8u]t12 6 4 2 SAyssstuemmp’tsiopnosrtmfoalidoegteongereanteioranteprtohceepssortfolio Procedural Explanation [
By analyzing the codes in detail, we found that the user’s definition of explanation is quite broad, but not arbitrary. From a user’s point of view, an explanation should help the user to understand and make sense of the system and the provided recommendations. From such a user perspective, we coded 107 total responses in the context of requesting an explanation from the system (see Table 2).
The further thematic analysis reveals that these responses can be grouped into three main categories: "Recommendation Explanation" (43 of 107 codes), "Domain-Specific Information" (54 of 107 codes), and "Shared Understanding" (10 of 107 codes). In the following, we discuss these categories in more detail.
Recommendation Explanation. In our analysis, we used the explanation taxonomy
presented in [
User Preference Explanation is used by us, to classify all responses where participants wanted an explanation that shows a link between the recommendation and their preferences or the answers given to the questions. For example, "What if I give a wrong answer?" (P07), "What was the impact of each answer I gave on the result?" (P02), "What would be the impact of answering it not honestly?" (P02).
Output Explanation is used by us to classify all responses where participants wanted an explanation about the recommended portfolio, but was not directly linked to the input in terms of their answered questions, such as knowing more about risk, expected revenues, or reasons for the specific portfolio composition. For example, "How high is the chance of having less than 200 % of my investment after 10 years?" (P03), "Why the system is not highlighting the aspects that were considered or the reason why it suggested certain shares or funds?" (P02), "Why am I not 100 % invested?" (P03).
Procedural Explanation is used by us to classify all responses where participants wanted an explanation about the internal logic and reasoning of the system to recommend a specific portfolio. For example, "What are the assumptions based on?" (P10), "Why are the specific assets selected for me?" (P04), "Where do the values (proportion) come from?" (P10), "Why should I use this system and not other systems like Trading View?" (P10).
Knowledge-Based Explanation is used by us, to classify all responses where participants wanted an explanation about the data used by the system to generate the portfolio or additional information about other users or situations to answer their questions. For example, "What is the standard duration for investment"? (P09), "What is the rate of success for these investment portfolios?" (P06), "What is the database used to derive the results?" (P08), "What is the average amount of trades of users within my peers?" (P03).
Only 46 of the 107 coded requests for explanations are classified in the category of
"Recommendation Explanation". This indicates that in most cases, explanations requested by participants
do not fall into the categories provided by the theoretical taxonomy of explanations [
Domain-Specific Information. According to [28], the information relates to a specific context that the user is not familiar with. For example, some domain-specific terminologies or concepts, provide knowledge about something comprising either facts or details about a subject, event, or situation, or provide knowledge that adds a value to a situation in a particular context to make it understandable.
We adopt this understanding to define the category of Domain-Specific Information , to classify all responses accordingly. For example, "What is the risk-return profile?" (P04), "What exactly is meant by the number of transactions?" (P12), "What does security assets mean?" (P09).
According to the FGD responses, this category was the most prominent one, as 51 of the 107 coded requests for explanations are classified into this category.
Shared Understanding. According to [29], Shared Understanding can be defined as elaborating the mutual knowledge, beliefs, or assumptions or it can be an elaboration of how the system interpreted the user’s goals and objectives. We adopt this understanding to define the category of Shared Understanding, to classify all responses where participants wanted an explanation from the system in order to know, if there is mutual knowledge or how the system interpreted their answers to the questions. For example, "What if I understand small loss diferently from you?" (P07), "How can I know, how you interpreted my risk assessment answers?" (P08), "What about ethical aspects?" (P02).
This category was the least prominent, as only 10 out of the 107 coded requests for explanations were classified in this category.
By comparing the focus groups (see Table 2), we further observed that HCI Experts group was the most responsive one (52 codes), followed by the Domain Experts group (37 codes), where the Common Users group was least responsive (18 codes).
In most cases, the responses from the three groups were related to all three aspects, namely Recommender Explanation, Domain-Specific Information , and Shared Understanding. The only diferences between the groups were that there were no responses related to Knowledge-based Explanation from the Common Users group and Outcome Explanation from the Domain Experts. Overall, the results depict the need for integrating all three aspects for users in the system design to have an explainable financial RS.
We identified the following major findings from the online survey: 1) The domain-specific aspects of explainability, i.e. Recommendation Explanation, Domain-Specific Information , and Shared Understanding, are equally relevant for participants without any significant diferences, 2) The domain-specific taxonomy is significantly more important for participants as compared to the domain-general taxonomy adopted from the literature, 3) Evaluation of the RA replica is not well-assessed by the participants w.r.t. its PQE. In the following, we describe these findings in detail.
To address our RQ2a, we identified the users’ importance for explanations in the context of the RA system in two steps: 1) Personal relevance of explanations w.r.t. the empirically-driven, domain-specific categories, and 2) Personal relevance of explanations w.r.t. theory-driven, domain-general categories.
We first computed the overall mean score of the combined three domain-specific categories and we saw that the personal importance of the combined categories for all participants is quite high on average (Avg. Mean = 4.08). This is also true for individual categories, especially regarding Recommendation Explanation (Mean = 4.33) and Domain-Specific Information (Mean = 4.0) (as shown in Table 3). The results are also in line with the focus group discussion from Section 4.2, where most of the responses were related to these two categories (see Table 2). Moreover, the category Shared Understanding (Mean = 3.91) was also rated relatively high, indicating the user importance to have explanations about how the system interpreted the user’s input.
To statistically check the inter-category diferences, we utilized an ANOVA ( = 0.05) on the mean scores of all participants. The result shows that there is no significant diference among the three domain-specific categories regarding personal importance ( (2, 33) = 1.21, = .310, 2 = .07). Despite the insignificant diference, the high mean scores for all three categories still indicate that these categories are important to participants.
We further analyzed the inter-group diferences. We observed that Recommendation Explanation was rated higher (Mean = 4.40) by the Domain Experts group. In the case of Domain-Specific Information (Mean = 4.55) and Shared Understanding (Mean = 4.33), the HCI Experts had higher ratings as compared to other groups.
We checked the observed inter-group diferences statistically by applying one-way MANOVA ( = 0.05) on aggregated categories. The result showed that the categories are not significantly rated diferently among all three groups ( (6, 14) = 0.69, = 0.66, 2 = 0.22). To further determine individual efects regarding each category, we ran univariate tests. The results shown in Table 3 indicate that there is no significant diference among the groups in terms of individual category i.e. Recommendation Explanation ( = 0.95), Domain-Specific Information ( = 0.23), and Shared Understanding ( = 0.56). As the results were not significant, therefore the observed inter-group diferences should not be overrated.
In addition to the empirically grounded categories, to obtain a comprehensive evaluation, we
also adopted the well-established, generally defined categories from the explanation taxonomy
presented in [
We found that the personal importance of the combined categories for all participants is between "moderately important" to "important" (Avg. Mean = 3.56). This is also true for individual categories, where the Input-Output Explanation seems to be more important for the participants (Mean = 3.89), followed by Outcome Explanation (Mean = 3.58), Procedural Explanation (Mean = 3.54), and Knowledge-based Explanation (Mean = 3.21). To statistically check the inter-category diferences, we utilized an ANOVA ( = 0.05) on the mean scores of all participants. The result shows that there is a significant diference among the four categories regarding personal importance ( (2, 45) = 5.44, = .008, 2 = .19). We further ran individual T-tests to check which of these categories are significantly diferent from others and found that Input-Output Explanation received a significantly higher rating ( = 0.01) than Knowledge-Based Explanation. We further found that Procedural Explanation had a significantly higher rating than Knowledge-Based Explanation ( = 0.009).
Moreover, we analyzed the inter-group diferences. We observed that HCI Experts have higher ratings for all four categories as compared to other groups i.e. Input-Output Explanation (Mean = 4.25), Outcome Explanation (Mean = 4.0), Procedural Explanation (Mean = 4.16), Note: 5-point Likert Scale coding (1 = "Not at all important", 3 = "Moderately Important", 5: "Very Important"). Significant diferences measured with ( = 0.05), are marked with *. Higher values (highlighted in bold) indicate better results. Explanation Category and Items For me, it is important to know ... and Knowledge-Based Explanation (Mean = 3.93). Another rating pattern that can be observed is that the Common Users have worse ratings for all categories except the Knowledge-Based Explanation, as compared to other groups.
To statistically check the inter-group diferences, we applied one-way MANOVA ( = 0.05) on aggregated categories. The result showed that all four categories were not rated significantly diferently among all three focus groups ( (8, 12) = 1.26, = 0.34, 2 = 0.45). To determine individual efects regarding each category, we ran univariate tests. The results, shown in Table 4, indicate that there was no significant diference among the groups in terms of individual category, i.e. Input-Output Explanation ( = 0.07), Outcome Explanation ( = 0.40), Procedural Explanation ( = 0.28), and Knowledge-Based Explanation ( = 0.27). As the results were not statistically significant, the observed inter-group diferences should not be overrated.
In our study, we combined two types of taxonomy: the domain-specific (as shown in Table 3) and the domain-general (as shown in Table 4).
With our analysis, we further wanted to identify if there is any inter-taxonomy diference
regarding personal importance. For this, we first computed the overall mean score of combined
categories for each taxonomy and found that on average the personal importance of the general
categories taken from theory [
To statistically check the inter-taxonomy, we conducted T-test and observed that intertaxonomy diference is significant ( (82) = 3.20, = .002, = .70). This significant intertaxonomy diference indicates that the empirically-grounded categories seem to be more relevant for the participants than the theoretically adopted ones, in the context of the financial domain.
The previous section showed that XRA is an important issue for the participants, especially regarding the categories identified in the FDGs. To further address our RQ2b, we asked the participants to evaluate the perceived quality of explanations provided by the RA replica w.r.t. the domain-specific categories i.e. Recommendation Explanation, Domain- Specific Information , and Shared Understanding using a five-point scale from "Strongly Disagree" to "Strongly Agree".
We first computed the overall mean score of the combined three domain-specific categories and saw that the perceived quality of the combined categories for all participants, is quite low (Avg. Mean = 2.16) which is equal to "Disagree". This is also true for individual categories (as shown in Table 5), where the Recommendation Explanation (Mean = 2.08) performed the worst, Domain Specific Information (Mean = 2.19), and Shared Understanding (Mean = 2.22) are rated higher, but still below 3 ("Disagree"). Overall, the results indicate that the explanations provided by the RA replica are not well-perceived by participants w.r.t. all categories.
To statistically check the inter-category diferences, we utilized ANOVA ( = 0.05) on the mean scores of all participants. The result shows that there is no significant diference among the categories regarding their perceived quality ( (2, 33) = .06, = .943, 2 < .01). This indicates that participants perceived the explanation quality for all categories as similarly worse.
We further analyzed the inter-group diferences. The results shown in Table 5 indicate that Common Users perceived the quality of explanations in all cases as higher than the other groups. To check the inter-group diferences statistically, we applied one-way MANOVA ( = 0.05) on aggregated categories. The result showed that all categories are not significantly rated diferently among the three focus groups ( (6, 14) = 2.09, = 0.11, 2 = 0.47). To determine individual efects regarding each category, we ran univariate tests. The results shown in Table 5 indicate that Common Users perceived the quality of explanations in all cases as higher than the other groups. However, only in the case of "Recommendation Explanation", the rating is significantly higher for Common Users as compared to other groups ( = .013).
To address the challenges of designing an eXplainable Robo-Advisor (XRA), we applied a mixedmethod approach to qualitatively explore the user’s need and understanding of explanations in ifnancial RS through FGD, which we then quantitatively verified and supplemented the findings with an online survey.
First, we addressed our RQ1, by conducting three qualitative FGD and identified a
usercentered taxonomy of domain-specific explanations. With this taxonomy, we demonstrated
that general explanation frameworks as presented in [
We further quantitatively verified the FGD results and addressed our RQ2a by evaluating
the personal relevance of explanations (PRE) in two phases: 1) PRE w.r.t. our domain-specific
categories, and 2) PRE w.r.t. the general categories presented in [
Our study further reveals that all general categories have received lower mean scores compared to the domain-specific categories. The further quantitative comparison reveals that this inter-taxonomy diference is significant. This means on average, the domain-specific categories have higher personal relevance compared to the domain-general categories. These enumerative depictions of the results thus verify the importance of domain-specific explanation needs in the context of financial RS to perceive the system as explainable.
We further addressed our RQ2b by evaluating the replica of an existing RA in terms of
the perceived quality of the explanations (PQE) (See Section 5.2). The participants rated the
explanation quality as rather low, which highlights the existing need from academia [
Overall, the mixed-method approach of our study provides novel insights. However, the approach has its limitations in terms of the small sample size used for both studies. Despite this limitation, the results still shed a positive light on taking the domain-specific user’s needs into account, to design the complex financial RS explainable from the user’s perspective. Future work will validate the quantitative findings on a large sample size and will further focus on providing the design implications from the user’s perspective to make the financial RS explainable for users. rule-base evidential reasoning using level 2 quotes, Expert Systems with Applications 39 (2012) 7150–7157. [17] F. Abraham, S. L. Schmukler, J. Tessada, Robo-advisors: Investing through machines,
World Bank Research and Policy Briefs (2019). [18] G. Babaei, P. Giudici, E. Rafinetti, Explainable artificial intelligence for crypto asset allocation, Finance Research Letters (2022) 102941. [19] M. Schemmer, P. Hemmer, N. Kühl, S. Schäfer, Designing resilient ai-based robo-advisors: A prototype for real estate appraisal, in: 17th International Conference on Design Science Research in Information Systems and Technology, 1st-3rd June 2022, St. Petersburg, FL, USA, 2022. [20] M. T. Ribeiro, S. Singh, C. Guestrin, " why should i trust you?" explaining the predictions of any classifier, in: Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, 2016, pp. 1135–1144. [21] D. L. Morgan, Focus groups as qualitative research, volume 16, Sage publications, 1996. [22] M. Bloor, Focus groups in social research, Sage, 2001. [23] A. S. Acharya, A. Prakash, P. Saxena, A. Nigam, Sampling: Why and how of it, Indian
Journal of Medical Specialties 4 (2013) 330–333. [24] V. Braun, V. Clarke, Thematic analysis., American Psychological Association, 2012. [25] J. Fereday, E. Muir-Cochrane, Demonstrating rigor using thematic analysis: A hybrid approach of inductive and deductive coding and theme development, International journal of qualitative methods 5 (2006) 80–92. [26] T. O. Nyumba, K. Wilson, C. J. Derrick, N. Mukherjee, The use of focus group discussion methodology: Insights from two decades of application in conservation, Methods in Ecology and evolution 9 (2018) 20–32. [27] P. Pu, L. Chen, R. Hu, A user-centric evaluation framework for recommender systems, in:
Proceedings of the fifth ACM conference on Recommender systems, 2011, pp. 157–164. [28] A. D. Madden, A definition of information, in: Aslib Proceedings, MCB UP Ltd, 2000. [29] E. A. C. Bittner, J. M. Leimeister, Why shared understanding matters–engineering a collaboration process for shared understanding to improve collaboration efectiveness in heterogeneous teams, in: 2013 46th Hawaii International Conference on System Sciences, IEEE, 2013, pp. 106–114. [30] J. Prümper, Software-evaluation based upon iso 9241 part 10, in: Vienna Conference on
Human Computer Interaction, Springer, 1993, pp. 255–265. [31] S. Naveed, An Interactive Hybrid Approach to Generate Explainable and Controllable
Recommendations, Ph.D. thesis, University of Duisburg-Essen, 2021. [32] S. Naveed, B. Loepp, J. Ziegler, On the use of feature-based collaborative explanations: An empirical comparison of explanation styles, in: Adjunct Publication of the 28th ACM Conference on User Modeling, Adaptation and Personalization, 2020, pp. 226–232. [33] S. Naveed, J. Ziegler, Featuristic: An interactive hybrid system for generating explainable recommendations–beyond system accuracy, system 18 (2020) 33. [34] M. Eiband, D. Buschek, A. Kremer, H. Hussmann, The impact of placebic explanations on trust in intelligent systems, in: Extended abstracts of the 2019 CHI conference on human factors in computing systems, 2019, pp. 1–6.