Transparency in Advice-Giving Systems: A Framework and a Research Model for Transparency Provision

Abstract

Advice-giving systems (AGSs) provide recommendations based on users' unique preferences or needs. Maximizing users' adoptions of AGSs is an effective way for ecommerce websites to attract users and increase profits. AGS transparency, defined as the extent to which information of a system's reasoning is provided and made available to users, has been proved to be effective in increasing users' adoptions of AGSs. While previous studies have identified providing explanations as an effective way of enhancing AGS transparency, most of them failed to further explore the optimal transparency provision strategy of AGSs. We argue that instead of setting a uniform rule of providing AGS transparency, we should develop optimal transparency provision strategies for different types of AGSs and users based on their unique features. In this paper, we first developed a framework of AGS transparency provision and identified six components of AGS transparency provision strategies. We then developed a research model of AGS transparency provision strategy with a set of propositions. We hope that based on this model, researchers could evaluate how to effect transparency for AGSs and users with different characteristics. Our work would contribute to the existing knowledge by exploring how AGS and user characteristics will influence the optimal strategy of providing AGS transparency. Our work would also contribute to the practice by offering design suggestions for AGS explanation interfaces.

-

CCS Concepts • Information systems➝Decision support systems • Human-centered computing➝HCI theory, concepts and models • Computing methodologies➝Cognitive science • Applied computing➝Online shopping Author Keywords Advice-giving systems; justifications; adoption.

transparency; explanations; IUI Workshops'19, March 20, 2019, Los Angeles, USA.

Copyright © 2019 for the individual papers by the papers' authors. Copying permitted for private and academic purposes. This volume is published and copyrighted by its editors.

INTRODUCTION Advice-giving systems (AGSs) are software systems that offer users with personalized recommendations or decision aids based on users’ unique preferences or needs (Xiao and Benbasat 2007; 2014) . Due to their effectiveness in reducing users’ information overload (Komiak and Benbasat, 2008) and facilitating users’ decision-making process (Wang and Benbasat, 2008) , maximizing users’ adoptions of AGSs is an effective way for e-commerce websites to attract users and increase profits (Komiak and Benbasat 2006) . System transparency, defined as the extent to which information of a system’s reasoning is provided and made available to users (Amalia, 2017; Cho et al., 2017; Hosseini et al., 2018; Leape et al. 2009; Yamazaki and Yoon, 2016; Zhu, 2002) , is considered as a key influential factor of users’ adoptions of AGSs and their acceptances of AGS outcomes (Cramer et al., 2008; Pu et al., 2011) . Previous studies have identified providing explanations as an effective way of enhancing AGS transparency (Bilgic and Mooney 2005; Gedikli et al. 2014; Herlocker 2000) and users’ adoptions of AGSs (Arnold et al. 2006; Gregor and Benbasat 1999; Hernando et al. 2013; Mao and Benbasat 2000; Pu and Chen, 2007; Wang and Benbasat 2007; Ye and Johnson 1995) . Despite the fruitful research findings in the IS literature, there is still a lack of attentions to the optimal transparency provision strategy of AGSs. In this paper, we define optimal AGS transparency provision strategy as a way of providing transparency which will maximize users’ adoptions of AGSs and their outcomes. We argue that there is not a global optimal transparency provision strategy for all kinds of AGSs and users. Rather, local optimal transparency provision strategies should be developed based on different characteristics of both AGSs and users.

The provision of AGSs transparency has a number of features, the combination of which can form different provision strategies. For example, when providing transparency, AGSs can reveal to users what they do (e.g. how AGSs generate advice) and why they do it (e.g. why AGSs collect certain user data). The information revealed by AGSs can be either long or short, complex or simple, with higher or lower accessibility, use professional or plain languages, be provided before or after a certain behavior is done by AGSs, etc. The above features may be more effective in different contexts. For instance, it has been shown that feedforward explanations (i.e. explanations provided before advice-generating process) are preferred by novice users, and feedback explanations (i.e. explanations provided after advice-generating process) are preferred by expert users (Arnold et al. 2006; Dhaliwal and Benbasat, 1996) . However, while some studies considered users’ characteristics when designing explanations, very few studies have focused on how AGS characteristics might influence AGSs’ transparency provision strategy. In recent years, due to the rapid development of advicegenerating technology, AGSs has transformed from traditional ones, which explicitly ask users to indicate their preferences or needs and generate advice accordingly, to more advanced ones, which employ AI-based techniques (e.g. collaborative filtering and content-based filtering) to generate advice based on implicitly collecting users’ personal data (e.g. their age, location, browsing behaviors, etc.). Due to the implicit user data collection and high complexity of advice-generating techniques, users may have higher privacy concerns and be more confused about how the advice is generated when they use AGSs with advanced features. Consequently, the existing theories and rules of transparency provision that have been developed and tested in the context of traditional AGSs might need to be modified or redeveloped to adapt the usage of more advanced AGSs. In this paper, we argue a necessity of developing a comprehensive model for AGS transparency provision. Such a model will allow us exploring ways of providing transparency in different types of AGSs and for different kinds of users, which can lead to the highest level of users’ adoptions of AGSs and their acceptances of AGS outcomes. Using this model, we would like to address the following research questions:

What are the components of AGS transparency provision? How should we explore the optimal way of providing AGS transparency considering both AGS characteristics and user characteristics? Our research would contribute to the existing knowledge by proposing how AGS and user characteristics will influence the optimal strategy of providing AGS transparency, and contribute to the practice by offering design suggestions on AGS explanation interfaces. The remainder of our paper is organized as follows: Section 2 defines transparency in AGSs and provides a literature review about AGS transparency. Section 3 develops a framework of AGS transparency provision. Section 4 develops a research model of AGS transparency provision strategy. Section 5 discusses the contributions, limitations and future research directions.

TRANSPARENCY IN ADVICE-GIVING SYSTMES In the domain of AGSs, transparency is defined as users’ understanding of systems’ inner logic, i.e. why a particular recommendation is recommended (Pu et al., 2011; Swearingen and Sinha, 2002; Tintarev and Masthoff, 2007) . In other domains such as e-government and health care, it is also defined as systems’ voluntary release of information (Amalia, 2017; Hosseini et al., 2018; Leape et al. 2009) or the visibility and accessibility of such information (Cho et al., 2017; Zhu, 2002) . This suggests two alternative ways of measuring system transparency, from users’ perspective and systems’ perspective respectively. In this paper, we define objective transparency as the extent to which AGSs release information regarding what they do and why they behave in a certain way, and subjective transparency as the extent to which users perceive that the information regarding what systems do and why they behave in a certain way is provided by AGSs and is visible/available/accessible to them (Cho et al., 2017; Zhu, 2002) .

Providing transparency is generally considered to be beneficial to users. Some studies argued that systems with high transparency would inform users about who can collect, access, and use their personal data, and give users the right to control the utilization of their own personal data (Hedbom, 2008). Some other studies proposed that highly transparent systems would articulate the systems’ goals (Zouave and Marquenie, 2017) , why data is collected from users (Hedbom, 2008), and the rationale for system outputs (Cramer et al., 2008; Diakopoulos and Koliska, 2017; Hedbom, 2008; Pu et al., 2011; Zouave and Marquenie, 2017) . In this way, such systems could increase the accountability of system algorithms (Ananny and Crawford, 2018; Diakopoulos and Koliska, 2017; Spagnuelo and Lenzini, 2016) , reduce users’ uncertainty (Diakopoulos and Koliska, 2017) and increase users’ confidence (Sinha and Swearingen, 2002) when interacting with systems, facilitate the elicitations of users’ personal information (Hosseini et al., 2018) , and enhance users’ trust in systems (Diakopoulos and Koliska, 2017; Tintarev and Masthoff, 2007) and adoptions of systems outcomes (Cramer et al., 2008; Pu et al., 2011) .

Despite the great benefits brought by providing transparency, some researchers indicated that misuses of system transparency might occur if transparency was not provided in a proper way. Some studies argued that excessively detailed information with great transparency might reduce the efficiency of systems in that such information required too much time to be processed by users (Tintarev and Masthoff, 2007) and might make users become distracted from the central, more important information (Ananny and Crawford, 2018). In addition, Hosseini et al. (2018) and Xu et al. (2018) also warned that if the information provided by systems was not understandable/interpretable/ actionable to users, users’ trust in systems might be reduced rather than being increased.

A FRAMEWORK FOR AGS TRANSPARENCY PROVISION Before exploring how AGS transparency should be provided to users, we first defined what is AGS transparency provision by developing a framework. Drawn from the framework of knowledge-based system explanations developed by Dhaliwal and Benbasat (1996), we included the characteristics of both the provided information and information-providing interfaces into our framework. Specifically, our AGS transparency provision framework has six components, namely transparency provision stage, transparency type, the content of the provided information, the timing of transparency provision, the type of transparency provision interface, and the format of the provided information. Each component has a number of different values. The framework of AGS transparency provision is shown in Figure 1.

Transparency Provision Stage Adapted from Xiao and Benbasat (2007; 2014), we divide the utilization of AGSs into input, process, and output stages. Input stage is the stage during which users’ preferences or needs are elicited; process stage is the stage during which advice is generated by systems based on the data collected in input stage, and output stage is the stage during which systems present the generated advice to users (Xiao and Benbasat, 2007; 2014) . AGS transparency can be provided to users in each of the three stages. In input stage, AGSs can reveal the process and rational of their data collections from users. In process stage, AGSs can disclose and justify their advice-generating process. In output stage, AGSs can explain to users why they think the advice is a good one for users.

Transparency Types In the field of AGSs, revealing explanations about how the system work is commonly considered to be one feasible way of providing system transparency to users (Tintarev and Masthoff, 2007) . In addition to this, some studies have also proposed another way of conveying the idea of how advice is generated without elucidating precisely the mechanism of AGSs (Lipton, 2016) , e.g. we recommended movie A to you because it is more suitable for you compared to 92% of the movies in our website. In this paper, we argue that AGSs can provide transparency to users through presenting two types of messages, namely explanations, i.e. what systems will do/are doing/have done, and justifications, i.e. information about why systems behave in a certain manner. Different from explanations, which offer users objective information regarding systems’ behaviors, justifications enable AGSs to show users the advantages of their advice-generating technique and outcomes, and can thus make users feel less uncertain and more confident to accept the generated advice. In our framework, we suggest that AGSs can provide two types of transparency, i.e. explanations and justifications, in input, process, and output stage respectively. The content and examples are shown in Table 11. 1 All the examples in this table are copied from Facebook’s instructions for their customized ads.

Stage

Content of Transparency Provision In addition to stage and transparency type, the content of the information provided by AGSs, which aims to enhance systems’ transparency, can also influence the effects of transparency provision. For example, the information provided by AGSs can be either long (e.g. one paragraph) or short (e.g. one sentence). The content of the information can be either complex (e.g. using profession language) or easy to understand (e.g. using plain language). The information regarding how systems work can be either very detailed (e.g. providing the formulas of calculating the similarity between users) or less detailed (e.g. briefly saying “users who are similar to you also bought…”).

Timing, Interface, and Information Format of Transparency Provision In addition to the above-mentioned factors, there are still some other factors that can be components of AGS transparency provision strategies, including timing of provision, interface design, and information format. Timing of provision refers to the fact that AGSs can choose to provide information to explain or justify their behaviors either before or after a specific behavior is performed by AGSs. The transparency provided before performing a specific behavior is called ex-ante transparency, while the transparency provided after a specific behavior is done is called ex-post transparency (Ananny and Crawford, 2018; Spagnuelo and Lenzini, 2016) . Interface design is also an influential factor of transparency provision. Dhaliwal and Benbasat (1996) indicated that there were two types of AGS transparency-providing interfaces – one employs active strategy where the information is automatically presented to users, and the other one employs passive strategy where users have to make explicit requests to access the provided information. Finally, the format of the provided information may also make a difference on AGS transparency provision. AGSs can reveal information about their inner logic in a variety of formats, e.g. text, histogram, graph, matching score, or even formula, etc. Users may have different perceptions when the same information is provided by AGSs in different formats (Herlocker et al., 2000) .

A RESEARCH MODEL OF AGS TRANSPARENCY PROVISION STRATEGY In order to explore the optimal way of providing AGS transparency, which can help maximize users’ adoptions of systems, we developed a research model for AGS transparency provision strategies (see Figure 2) for future studies to test. In this model, we take into consideration the characteristics of both AGSs and users. The model proposed in our research can be tested through conducting lab experiments and field studies. Through testing the propositions generated based on this model, we hope future research could find out the best ways of providing transparency in different types of AGSs and for different types of users. General Propositions In this paper, we define users’ perceived understanding of AGSs as the extent to which users perceive that they understand the meaning of the information provided by AGSs, which explains what AGSs do and justifies why they do it. We study users’ subjective perception of how much they understand the inner logic of AGSs rather than their true level of understanding how AGSs work because it has been proved that compared to users’ actual knowledge, users’ subjective perceptions of their understanding of a website are more influential on their intentions to reuse the website (Jiang and Benbasat, 2007) . Providing explanations regarding how AGSs work has been proved to be effective in improving users’ perceived understanding and acceptance of AGSs (Hengstler et al. 2016; Lakkaraju et al. 2016; Lehikoinen and Koistinen 2014; Lim et al. 2009; Pieters, 2011; Wang and Benbasat 2007; 2008; Zliobaite et al. 2012) . In addition, providing information regarding AGSs’ data collection in input stage can help users know better about what kind of their personal data is collected and why it is collected, and thus have an effect on their privacy concerns.

P1: The provision of AGS transparency will improve users’ perceived understanding of how AGSs work and why AGSs perform certain behaviors.

P2: The provision of AGS transparency will increase users’ privacy concerns.

In our model, we assume that the effect of transparency provision on users’ perceptions will be moderated by both AGS characteristics and user characteristics. When users have higher level of domain knowledge of AGSs, they will have the ability to process and comprehend more complex information provided by AGSs. In this case, providing information which is harder to process and understand will have a more positive influence on users with higher levels of domain knowledge compared to those who have lower levels of domain knowledge.

P3: User characteristics will moderate the influence of transparency provision on users’ perceived understanding of how AGSs work and why AGSs perform certain behaviors.

In addition, the effect of transparency provision may be different in different types of AGSs. For example, compared to AGSs with explicit user data collection, input stage transparency may have a stronger effect on users’ perceptions of systems in AGSs with implicit user data collection because users are less likely to know the details of how and why their data is collected when interacting with such AGSs, and may thus benefit more from the explanations and justifications provided by AGSs. P4: AGS characteristics will moderate the influence of transparency provision on users’ perceived understanding of how AGSs work and why AGSs perform certain behaviors.

P5: AGS characteristics will moderate the influence of transparency provision on users’ privacy concerns. Previous studies have proved that users’ perceived understanding of AGSs will positively influence their attitudes towards AGSs, intentions of adopting AGSs, and acceptances of AGS outcomes (Cramer et al., 2008; Pu et al., 2011; Wang and Benbasat, 2007; 2008) , while users’ privacy concerns will negatively influence them (Hengstler et al., 2016; Wang and Benbsat; 2008; Yan et al., 2016) . We also assume that whether or not users’ expectations of AGSs can be met will moderate the effect of users’ perceived understanding of AGSs on their attitudes towards AGSs, intentions of adopting AGSs, and acceptances of AGS outcomes. Users’ expectations of AGSs include both process and behavior expectations, e.g. “Yelp should ask me where do I live” or “Yelp should make recommendations based on the restaurants that I have been to”, and outcome expectations, e.g. “Yelp should recommend me some Chinese restaurants” (Wang and Benbasat, 2008) . A higher-level perceived understanding of AGSs can help users know better how AGSs actually work, and can thus confirm users’ perceptions of consistency/inconsistency between their expectations and the way AGSs work. Therefore, we assume that when there is a consistency, users’ perceived understanding of AGSs will positively influence their attitudes towards AGSs, intentions of adopting AGSs, and acceptances of AGS outcomes because they are clearer that their expectations are met by AGSs. However, when there is an inconsistency, a higher-level perceived understanding of AGSs will have negative influences in that users become more aware that the way AGSs work are different from their original expectations.

P6: The effect of users’ perceived understanding of AGSs on users’ attitudes towards AGSs, intentions of adopting AGSs, and acceptances of AGS outcomes will be moderated by the consistency between users’ expectations of AGSs and the way AGSs actually work. Specifically, if there is a consistency, users’ perceived understanding of AGSs will have positive influences. On the contrary, if there is an inconsistency, users’ perceived understanding of AGSs will have negative influences.

P7: Users’ privacy concerns will negatively influence their attitudes towards AGSs, intentions of adopting AGSs, and acceptances of AGS outcomes.

Transparency Provision Strategy Considering AGS Characteristics Input Stage Characteristics In the input stage, different AGSs have different ways of collecting user data. Some of them collect data through explicitly asking users to indicate their preferences or needs (e.g. a filter), while others collect data through implicitly tracking and recording users’ interactive behaviors (e.g. browsing behaviors, transaction records, locations, etc.). It is traditionally thought that providing explanations and justifications in input stage will positively influence users’ attitude towards AGSs (Wang and Benbasat, 2007) because this enables users to know more about data-collecting process. However, we propose in our model that we should also consider another “side effect” of doing so in AGSs with implicit user data collection – it makes users become more aware of the fact that their personal data is being collected by AGSs. In this case, providing explanations about what kind of data is collected and how it is collected may increase users’ privacy concerns when interacting with AGSs.

P8: For AGSs with explicit input data collection, both input stage explanations (P8a) and justifications (P8b) will have a positive influence on users’ perceived understanding of AGSs.

P9: For AGSs with implicit input data collection, both input stage explanations (P9a) and justifications (P9b) will have a positive influence on users’ perceived understanding of AGSs.

P10: For AGSs with implicit input data collection, input stage explanations will increase users’ privacy concerns. Process Stage Characteristic In process stage, different AGSs employ different techniques to generate advice. Some techniques have relatively easy inner logics (e.g. information retrieval), while some other techniques’ logics are more like black boxes and may be beyond non-professional users’ comprehensions (e.g. machine learning techniques). We assume that both process explanations and justifications of easily-understood advice-generating techniques will be comprehended by users, and can thus help users understand AGSs better. However, for complex advice-giving systems, while the process justifications can still be understood by users, the process explanations may not. We argue that the process explanations can help users understand AGSs only when users have enough time to process the explanations and enough ability to figure out the meaning of the explanations. Once the explanations are beyond a user’s comprehension, users may feel more confused because they may become aware that their original ideas of how AGSs work are not accurate, and realize that they actually know very little about the real way AGSs work. P11: For AGSs whose advice-generating techniques are easily understood by users, both process stage explanations (P11a) and justifications (P11b) will have positive effects on users’ perceived understanding of AGSs.

P12: For AGSs whose advice-generating techniques are not easily understood by users, the process stage explanations can positively influence users’ perceived understanding of AGSs only when users have the ability to process and comprehend the explanations. Once the provided explanations are beyond users’ comprehensions, it will start to negatively influence users’ perceived understanding of AGSs.

P13: For AGSs whose advice-generating techniques are not easily understood by users, process stage justifications will positively influence users’ perceived understanding of AGSs Transparency Provision Strategy Considering User Characteristics In addition to AGS characteristics, we also assume that users’ domain knowledge of AGSs will moderate the effect of the provision of AGS transparency on users’ perceived understanding of AGSs. According to the Elaboration Likelihood Model (Petty and Cacioppo, 1986) , users who have higher levels of domain knowledge will use a central route to process the provided information and thus focus more on explanations, which describe AGSs’ behaviors. In contrast, users who have lower levels of domain knowledge will be more likely to use peripheral route to process information and focus more on justifications, which justify AGSs’ behaviors through emphasizing the importance/advantages of doing so.

P14: For users with higher levels of domain knowledge, explanations will have stronger positive effect on their perceived understanding of AGSs (P14a), while justifications will have less positive effect on their perceived understanding of AGSs (P14b).

P15: For users with lower levels of domain knowledge, justifications will have stronger positive effect on their perceived understanding of AGSs (P15a), while explanations will have less positive effect on their perceived understanding of AGSs (P15b).

DISCUSSION Conclusions Providing information of AGSs’ reasoning has been proved to be effective in enhancing users’ adoptions of AGSs and acceptances of AGS outcomes. Despite fruitful research findings, little attention has been paid to the explorations of optimal transparency provision strategy for different types of AGSs and users. In this paper, we first defined transparency in the context of AGSs and summarized the existing research findings of AGS transparency. We then developed a framework of AGS transparency provision, identifying six components of AGS transparency provision strategies, i.e. transparency provision stage, transparency type, the content of the provided information, the timing of transparency provision, the type of transparency provision interface, and the format of the provided information. Finally, we developed a research model of AGS transparency provision strategy and proposed a set of propositions. Based on this model, we are expecting to find out the optimal way of providing transparency for AGSs and users with different characteristics.

Contributions Our research has both academic and practical significance. It could contribute to the existing literature by indicating that the strategies of providing transparency should not be the same across all types of AGSs and users, and exploring the optimal ways of providing transparency for different types of AGSs and users. Our research could also contribute to the practice by offering design suggestions on AGS explanation interfaces.

Limitations and Future Research Due to the limited time and cost, we have not yet conducted empirical studies to test the propositions in our research model. In addition, as an early-stage exploration of AGS transparency provision strategies, the model developed in this paper only considered the influence of a single factor (e.g. the complexity of advice-generation techniques, users’ domain knowledge, etc.) on the selection of AGS transparency provision strategy. However, despite the limitations, our work figured out the possible rules of providing transparency for different types of AGSs and users. Future research could be conducted based on our work through refining and expanding our model by including more factors and considering the combined effects of multiple factors, as well as testing this model by conducting empirical research.

ACKNOWLEDGMENTS The authors would like to thank the anonymous referees for their valuable advice and suggestions.

implementation: barriers to enhance information transparency and accountability." In SHS Web of Conferences, vol. 34, 02003.

Mike Ananny and Kate Crawford. 2018. Seeing without knowing: Limitations of the transparency ideal and its application to algorithmic accountability. New

Media & Society 20, 3: 973-989.

Vicky Arnold, Nicole Clark, Philip A. Collier, Stewart A. Leech, and Steve G. Sutton. 2006. The differential use and effect of knowledge-based system explanations in novice and expert judgment decisions. MIS

Quarterly: 79-97. 7. Izak Benbasat, and Weiquan Wang. 2005. Trust in and adoption of online recommendation agents. Journal of the association for information systems 6, 3: 4.

Mustafa Bilgic and Raymond J. Mooney. 2005.

Explaining recommendations: Satisfaction vs. promotion. In Beyond Personalization Workshop (IUI '05), Vol. 5, 153.

Bangho Cho, Sung Yul Ryoo, and Kyung Kyu Kim. 2017. Interorganizational dependence, information transparency in interorganizational information systems, and supply chain performance. European

Journal of Information Systems 26, 2: 185-205. argumentation. Information Systems Research 17, 3: 286-300.

Perceptions of a Lack‐of‐Group Bias and Transparency in the Performance Evaluation System Relate to Job Satisfaction. Human Resource Management 55, 6: 1059-1077.

References

1. Ibrahim M. Al-Jabri and Roztocki Narcyz . 2015 . Adoption of ERP systems: Does information transparency matter? . Telematics and Informatics 32 , 2 : 300 - 310 .

2. Sameh Al‐Natour , Izak Benbasat , and Ronald T. Cenfetelli . 2008 . The effects of process and outcome similarity on users' evaluations of decision aids . Decision Sciences 39 , 2 : 175 - 211 .

3. Sameh Al-Natour , Izak Benbasat , and Ron Cenfetelli . 2011 . The adoption of online shopping assistants: perceived similarity as an antecedent to evaluative beliefs . Journal of the Association for Information Systems 12 , 5: 347 .

4. Fitri Amalia . 2017 . Socio-technical analysis of Indonesian government e-procurement system

10. Henriette Cramer , Vanessa Evers, Satyan Ramlal, Maarten Van Someren, Lloyd Rutledge , Natalia Stash, Lora Aroyo, and Bob Wielinga . 2008 . The effects of transparency on trust in and acceptance of a contentbased art recommender . User Modeling and UserAdapted Interaction 18 , 5 : 455 .

11. James Davidson , Benjamin Liebald, Junning Liu, Palash Nandy, Taylor Van Vleet, Ullas Gargi , Sujoy Gupta et al. 2010 . The YouTube video recommendation system . In Proceedings of the fourth ACM conference on Recommender systems , 293 - 296 .

12. Fred D. Davis . 1989 . Perceived usefulness, perceived ease of use, and user acceptance of information technology . MIS quarterly: 319 - 340 .

13. Jasbir S. Dhaliwal and Izak Benbasat . 1996 . The use and effects of knowledge-based system explanations: theoretical foundations and a framework for empirical evaluation . Information systems research 7 , 3: 342 - 362 .

14. Nicholas Diakopoulos and Michael Koliska . 2017 . Algorithmic transparency in the news media . Digital Journalism 5 , 7 : 809 - 828 .

15. Gerhard Friedrich and Markus Zanker . 2011 . A taxonomy for generating explanations in recommender systems . AI Magazine 32 , 3 : 90 - 98 .

16. Fatih Gedikli , Dietmar Jannach, and Mouzhi Ge . 2014 . How should I explain? A comparison of different explanation types for recommender systems . International Journal of Human-Computer Studies 72 , 4 : 367 - 382 .

17. David Gefen, Izak Benbasat , and Paula Pavlou . 2008 . A research agenda for trust in online environments . Journal of Management Information Systems 24 , 4: 275 - 286 .

18. David Gefen, Elena Karahanna , and Detmar W. Straub . 2003 . Trust and TAM in online shopping: an integrated model . MIS quarterly 27 , 1: 51 - 90 .

19. Nelson Granados , Alok Gupta, and Robert J. Kauffman . 2010 . Research commentary-information transparency in business-to-consumer markets: concepts, framework, and research agenda . Information Systems Research 21 , 2 : 207 - 226 .

20. Shirley Gregor and Izak Benbasat . 1999 . Explanations from intelligent systems: Theoretical foundations and implications for practice . MIS quarterly: 497 - 530 .

21. Ido Guy , Naama Zwerdling, David Carmel, Inbal Ronen , Erel Uziel, Sivan Yogev, and Shila OfekKoifman. 2009 . Personalized recommendation of social software items based on social relations . In Proceedings of the third ACM conference on Recommender systems , 53 - 60 .

22. Hans Hedbom , Tobias Pulls, and Marit Hansen . 2011 . Transparency tools . In Privacy and Identity Management for Life , 135 - 143 .

23. Monika Hengstler , Ellen Enkel, and Selina Duelli . 2016 . Applied artificial intelligence and trust-The case of autonomous vehicles and medical assistance devices . Technological Forecasting and Social Change 105 : 105 - 120 .

24. Jonathan L. Herlocker , Joseph A. Konstan , and John Riedl . 2000 . Explaining collaborative filtering recommendations . In Proceedings of the 2000 ACM conference on Computer supported cooperative work , 241 - 250 .

25. Antonio Hernando, JesúS Bobadilla, Fernando Ortega, and Abraham GutiéRrez. 2013 . Trees for explaining recommendations made through collaborative filtering . Information Sciences 239 : 1 - 17 .

26. Mahmood Hosseini , Alimohammad Shahri, Keith Phalp, and Raian Ali . 2018 . Four reference models for transparency requirements in information systems . Requirements Engineering 23 , 2 : 251 - 275 .

27. Zhenhui Jiang and Izak Benbasat . 2007 . The effects of presentation formats and task complexity on online consumers' product understanding . MIS Quarterly: 475 - 500 .

28. Dongmin Kim and Izak Benbasat . 2006 . The effects of trust-assuring arguments on consumer trust in Internet stores: Application of Toulmin's model of

29. Dongmin Kim and Izak Benbasat . 2009 . Trust-assuring arguments in B2C e-commerce: impact of content, source, and price on trust . Journal of Management Information Systems 26 , 3: 175 - 206 .

30. Dongmin Kim and Izak Benbasat . 2010 . Designs for effective implementation of trust assurances in internet stores . Communications of the ACM 53 , 2 : 121 - 126 .

31. René F. Kizilcec . 2016 How much information?: Effects of transparency on trust in an algorithmic interface . In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems , 2390 - 2395 .

32. Sherrie Xiao Komiak and Izak Benbasat . 2004 . Understanding customer trust in agent-mediated electronic commerce, web-mediated electronic commerce, and traditional commerce . Information technology and management 5 , 1 - 2 : 181 - 207 .

33. Sherrie YX Komiak and Izak Benbasat . 2006 . The effects of personalization and familiarity on trust and adoption of recommendation agents . MIS quarterly: 941 - 960 .

34. Sherrie YX Komiak and Izak Benbasat . 2008 . A twoprocess view of trust and distrust building in recommendation agents: A process-tracing study . Journal of the Association for Information Systems 9 , 12 : 2 .

35. Joseph A. Konstan and John Riedl. 2012 . Recommender systems: from algorithms to user experience. User modeling and user- adapted interaction 22 , 1 - 2 : 101 - 123 .

36. Himabindu Lakkaraju , Stephen H. Bach , and Jure Leskovec . 2016 . Interpretable decision sets: A joint framework for description and prediction . In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining , 1675 - 1684 .

37. Lucian L. Leape . 1994 . Error in medicine . Jama 272 , 23 : 1851 - 1857 .

38. Juha Lehikoinen and Ville Koistinen . 2014 . In big data we trust? . Interactions 21 , 5 : 38 - 41 .

39. Brian Y. Lim , Anind K. Dey , and Daniel Avrahami . 2009 . Why and why not explanations improve the intelligibility of context-aware intelligent systems . In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems , 2119 - 2128 .

40. Zachary C. Lipton . 2016 . The mythos of model interpretability . arXiv preprint arXiv:1606 . 03490 .

41. Ji-Ye Mao and Izak Benbasat . 2000 . The use of explanations in knowledge-based systems: Cognitive perspectives and a process-tracing analysis . Journal of Management Information Systems 17 , 2: 153 - 179 .

42. Olivera Marjanovic and Dubravka Cecez-Kecmanovic . 2017 . Exploring the tension between transparency and datification effects of open government IS through the lens of Complex Adaptive Systems . The Journal of Strategic Information Systems 26 , 3: 210 - 232 .

43. David McSherry . 2005 . Explanation in recommender systems . Artificial Intelligence Review 24 , 2 : 179 - 197 .

44. Richard E. Petty and John T. Cacioppo. 1986 . The elaboration likelihood model of persuasion . In Communication and persuasion , 1 - 24 .

45. Wolter Pieters . 2011 . Explanation and trust: what to tell the user in security and AI? . Ethics and information technology 13 , 1: 53 - 64 .

46. Pearl Pu and Li Chen . 2007 . Trust-inspiring explanation interfaces for recommender systems . Knowledge-Based Systems 20 , 6: 542 - 556 .

47. Pearl Pu , Li Chen , and Rong Hu . 2011 . A user-centric evaluation framework for recommender systems . In Proceedings of the fifth ACM conference on Recommender systems , 157 - 164 .

48. Pearl Pu , Li Chen , and Rong Hu . 2012 . Evaluating recommender systems from the user's perspective: survey of the state of the art . User Modeling and UserAdapted Interaction 22 , 4 - 5 : 317 - 355 .

49. Marco Tulio Ribeiro, Sameer Singh , and Carlos Guestrin . 2016 . Why should i trust you?: Explaining the predictions of any classifier . In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining , 1135 - 1144 .

50. Andrew K. Schnackenberg and Edward C. Tomlinson . 2016 . Organizational transparency: A new perspective on managing trust in organization-stakeholder relationships . Journal of Management 42 , 7 : 1784 - 1810 .

51. Dayana Spagnuelo and Gabriele Lenzini . 2016 . Patient-centred transparency requirements for medical data sharing systems . In New Advances in Information Systems and Technologies , 1073 - 1083 .

52. Kirsten Swearingen and Rashmi Sinha . 2002 . Interaction design for recommender systems . In Designing Interactive Systems , vol. 6 , no. 12 , 312 - 334 .

53. Panagiotis Symeonidis , Alexandros Nanopoulos, and Yannis Manolopoulos . 2008 . Providing justifications in recommender systems . IEEE Transactions on Systems, Man, and Cybernetics-Part A: Systems and Humans 38 , 6 : 1262 - 1272 .

54. Panagiotis Symeonidis , Alexandros Nanopoulos, and Yannis Manolopoulos . 2009 . MoviExplain: a recommender system with explanations . RecSys 9 : 317 - 320 .

55. Nava Tintarev and Judith Masthoff . 2007 . A survey of explanations in recommender systems . In 2007 IEEE 23rd international conference on data engineering workshop , 801- 810 .

56. Nava Tintarev and Judith Masthoff . 2012 . Evaluating the effectiveness of explanations for recommender systems . User Modeling and User-Adapted Interaction 22 , 4 - 5 : 399 - 439 .

57. Weiquan Wang and Izak Benbasat . 2007 . Recommendation agents for electronic commerce: Effects of explanation facilities on trusting beliefs . Journal of Management Information Systems 23 , 4: 217 - 246 .

58. Weiquan Wang and Izak Benbasat . 2008 . Attributions of trust in decision support technologies: A study of recommendation agents for e-commerce . Journal of Management Information Systems 24 , 4: 249 - 273 .

59. Bo Xiao and Izak Benbasat . 2007 . E-commerce product recommendation agents: use, characteristics, and impact . MIS quarterly 31 , 1: 137 - 209 .

60. Bo Xiao and Izak Benbasat . 2014 . Research on the use, characteristics, and impact of e-commerce product recommendation agents: A review and update for 2007-2012 . In Handbook of Strategic e-Business Management, 403 - 431 .

61. Jingjun David Xu, Izak Benbasat , and Ronald T. Cenfetelli . 2014 . The Nature and Consequences of Trade-off Transparency in the Context of Recommendation Agents . MIS quarterly 38 , 2 .

62. David Jingjun Xu, Izak Benbasat , and Ronald T. Cenfetelli . 2017 . A Two-Stage Model of Generating Product Advice: Proposing and Testing the Complementarity Principle . Journal of Management Information Systems 34 , 3: 826 - 862 .

63. Yoshitaka Yamazaki and Jeewhan Yoon . 2016 . A Cross‐National Study of Fairness in Asia: How

64. Zheng Yan , Jun Liu, Robert H. Deng , and Francisco Herrera. 2016 . Trust management for multimedia big data .

65. L. Richard Ye and Paul E. Johnson . 1995 . The impact of explanation facilities on user acceptance of expert systems advice . MIS Quarterly: 157 - 172 .

66. Fahri Yetim . 2008 . A Framework for Organizing Justifications for Strategic Use in Adaptive Interaction Contexts . In ECIS , 815 - 825 .

67. Liying Zhou , Weiquan Wang, Jingjun David Xu, Tao Liu, and Jibao Gu . 2018 . Perceived information transparency in B2C e-commerce: An empirical investigation . Information & Management 55 , 7 : 912 - 927 .

68. Xujuan Zhou , Yue Xu, Yuefeng Li , Audun Josang , and Clive Cox . 2012 . The state-of-the-art in personalized recommender systems for social networking . Artificial Intelligence Review 37 , 2 : 119 - 132 .

69. Kevin Zhu . 2002 . Information transparency in electronic marketplaces: Why data transparency may hinder the adoption of B2B exchanges . Electronic markets 12 , 2: 92 - 99 .

70. Indre Zliobaite , Albert Bifet , Mohamed Gaber, Bogdan Gabrys, Joao Gama, Leandro Minku, and Katarzyna Musial . 2012 . Next challenges for adaptive learning systems . ACM SIGKDD Explorations Newsletter 14 , 1 : 48 - 55 .

71. Erik T. Zouave and Thomas Marquenie . 2017 . An Inconvenient Truth: Algorithmic Transparency & Accountability in Criminal Intelligence Profiling . In 2017 European Intelligence and Security Informatics Conference , 17 - 23 .