Explaining Podcast Recommendations To Users with Content Diversity Labels Bernd Huber1 , Yixue Wang2 , Jean Garcia-Gathright1 and Jenn Thom1 1 Spotify, USA 2 Northwestern University, USA Abstract Podcasts are available in a broad range of formats and content, and by a variety of producers. There are unique challenges with podcast recommendations that make it hard for users to distinguish between podcast episodes, and choose the right podcast to listen to. We see an opportunity to explain recommendations to users in order to help them make decisions about what to listen to. In this work, we study the characteristics of podcasts that make them different from each other. Based on a formative study with podcast experts, we find that host/guest information, format, length, and “vibe” are dimensions that constitute differences among podcasts. In a user study, we tested how highlighting such dimensions with explanatory labels impact users in a podcast selection task. Keywords Podcast, Recommendation Systems, Explainability 1. Introduction podcast ecosystem, current podcast streaming platforms, such as Apple Podcasts, Audible, Spotify and YouTube, Podcasts, portable and on-demand spoken-word audio neither fully support listener exploration of the breadth content available on a variety of streaming platforms, of available podcasts nor highlight the differentiating have emerged as a popular medium for information, en- dimensions among podcast shows and episodes. These tertainment, and advertising [1, 2]. Podcast listening is podcast streaming platforms aim to assist listeners in dis- on the rise – in 2006, 22% of the U.S. population older covering new podcasts through recommender systems. than 12 years were aware of podcasts; in 2021 it was over However, listeners still rely mostly on both offline and 78% [3]. More and more people listen to podcasts as they online word of mouth (e.g., friends, family, co-workers, explore a broader range of genres and use more stream- social media, discussion boards, etc.) and podcast cross- ing platforms [4]. In addition, there is a large amount of promotion, rather than podcast platform recommenda- podcast content available, with over two million podcast tions [9, 10]. shows and over 48 million podcast episodes available on There are also challenges for listeners to navigate pod- popular streaming platforms[5]. cast streaming platforms if they are interested in finding Because of their low barrier to entry for both pro- unique and appealing content. For instance, podcasts usu- ducers and listeners, podcasts encourage both amateur ally are released and organized in a series in the listening producers and mainstream media to provide a variety of interface, where new podcasts are pushed to subscribers content [6, 1], breaking traditional hierarchical gatekeep- through RSS feeds [11]. In addition, podcasts tend to be ing practices where only a professional can decide what long and dense (i.e., more than 30 minutes) [12], which to publish [7]. In addition, podcast listeners have indi- can further constrain listeners from exploring outside of cated a willingness to explore when listening to podcasts. their current listening habits. Due to the limited infor- 39% of podcast listeners listen to podcasts to learn some- mation shown in current podcast streaming interfaces, thing new [8]. Because of the wide variety of content listeners also need to actively choose when discovering from a large set of producers combined with listeners new podcasts [13]. who have indicated a willingness to explore, there is an This work introduces podcast recommendations as a opportunity to help listeners discover varied and diverse unique setting in which explainable recommendations podcast content effectively. are necessary. To define podcast dimensions that help Despite the large variety of podcasts offered by the podcast listeners differentiate content, we conducted a formative study with four experts in the design and eval- Joint Proceedings of the ACM IUI Workshops 2022, March 2022, uation of the podcast listening experience. Our formative Helsinki, Finland study revealed several dimensions to highlight when sup- $ bhb@spotify.com (B. Huber); yixue.wang@u.northwestern.edu porting listener exploration of different podcast content. (Y. Wang); jean@spotify.com (J. Garcia-Gathright); jennthom@spotify.com (J. Thom) We found diversity of creators (e.g., ethnicity, genders, © 2022 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). age groups, social status, etc.), differences of presenta- CEUR Workshop Proceedings http://ceur-ws.org ISSN 1613-0073 CEUR Workshop Proceedings (CEUR-WS.org) tion (interviews vs. storytelling), differences in “vibe” [19]. In this subsection, we discuss what user interfaces (light-hearted vs. more serious conversation), and differ- have been explored in the podcast ecosystem. ences in length, to be particularly relevant. Our findings Current podcast interfaces are typically sequential and suggest that listeners’ exploration of podcasts can be sup- organized by show-level [20], which could potentially ported by showing content labels, and that these labels limit users’ ability to explore beyond the shows they al- can increase users’ awareness of podcast diversity. ready listen to. Similar to other long-form and episodic media (e.g., movies, television series, documentaries, etc.), one challenge for podcasts is to summarize the most inter- 2. Related Work esting content from the long and dense content [21]. Due to the limitations of the audio format, which does not sup- 2.1. Differentiation in podcasts port skimming and browsing [22], it becomes a challenge Finding a podcast among the large variety of available for listeners to navigate among podcasts. One possible podcasts can be hard for users, since topical relevance is solution to ease the navigation of audio content is the use merely one aspect. As previous research suggests [14], of summaries. Previous research has experimented with podcasts vary along multiple dimensions: the specific different ways to understand how users navigate large host or guest in the episode, presentation styles, and pro- and dense content via summaries. For instance, one study duction quality, which all help a user determine whether proposes a way to summarize content via human and they are interested in listening to an episode or not. Mean- automatic methods [21]. Interface designs can also help while, episode metadata, including podcast episode titles, users view an entire recorded audio conversation and descriptions, along with the audio files [14, 2], create a allow zooming in and out to see the transcripts and listen large amount of data for podcast streaming platforms, re- to the audio at the same time [23]. Summaries can be also sulting in opportunities for designers to use this metadata combined with a hierarchical visualization mode to allow as a resource to help differentiate between the content. users to explore a large corpora through intuitive visual However, listeners still need to actively choose and differ- and textual methods [24]. Instead of proposing ways to entiate when discovering new podcasts because current automatically summarize podcast content as in previous podcast streaming interfaces show limited information research, our study aims at a user-centric exploration of about podcasts[13]. how users’ perceptions of a variety of podcast content is One approach to help differentiate between content is impacted by summaries. via diverse recommendation so that users can navigate Apart from summaries, content labels are another com- and identify relevant items faster in the exploratory stage mon way to provide snippets of information about con- [15]. Several dimensions of diversity are highlighted by tent and enable users to filter content easily. In prior previous studies [16], including diversity of entities (i.e., research, tags have been extensively studied as a form people, group, and organizations), topic diversity, view- of content labels. Social tags can be used as a key el- point diversity (e.g., different angles), and medium diver- ement in recommendation systems [25]; different tag sity (e.g., audio, video). Yet, these dimensions do not take selection algorithms as well as tag designs have been podcasts’ unique attributes into account. For instance, tested and suggested in movie recommendations [26]. some non-textual attributes unique to podcasts may con- Previous studies have also investigated the reasons be- tribute to a differentiated listening experience, such as hind users’ tagging behaviors in photos [27]. The use of energy, seriousness, vibe, novelty of the episode, dura- social tags can enhance navigation and search [28, 29]. tion, number of speakers, popularity, etc., as suggested Tags can also be combined with audio content for style in previous research [17, 2]. clustering in music [30]. Previous work in content tags Though many studies have explored specific dimen- has been applied in web pages, movies, photos, and mu- sions of how podcasts differ from each other, a holistic sic, but these studies do not fully understand how the view of how users perceive these differences is underex- tags impact the user navigation process as well as users’ plored. As a result, we propose that providing ways to actual selections. differentiate can help listeners make better choices. 3. Formative Interviews with 2.2. Interface elements for podcast Experts differentiation Users’ interactions with podcasts are influenced by the We interviewed professional podcast curators and one interfaces in which podcasts are presented [18], and many designer to understand how they differentiate between podcasters rely heavily on platforms to support discovery podcasts, and we sought to understand expert practices around supporting listeners’ exploration of various pod- casts. Previous studies have shown the power of experts in understanding users’ needs [31, 32]. Therefore their This low barrier also allows bringing “individualistic” expertise and knowledge about the breadth of podcast perspectives into public discourse, or the intimate self content can offer in-depth insights around users’ explo- [33], in contrast to the traditional standard. The inti- ration and navigation processes. macy from podcasters also makes podcast listening an We recruited four interviewees, including three pod- intimate and personal listening experience for listeners. cast curators and one designer, through snowball sam- I2 discussed that podcasts are “just for myself” and “a pling at a podcast streaming company: I1 (male, three self-experience,” instead of for a group of people. Podcast years of experience in podcast curation), I2 (female, two listening is also a passive listening experience for most years of experience in podcast curation), I3 (male, two listeners and listeners listen to it for mainly entertain- years of experience in podcast curation) and I4 (male, less ment purposes. As I3 mentioned, listeners “use podcasts than one year of experience in podcast playlist design and to fill that space” when they “are commuting, cleaning, four years experience in audio design). All interviewees or some activity that’s mundane.” were also long-time podcast listeners. Each interview Another unique aspect about podcasts is that listeners was conducted virtually via video meetings and lasted have control over what they listen to. Listeners them- from 46 to 72 minutes. selves choose what to listen to and decide “whether or Our semi-structured interviews began with self- not this podcast is something that is worth listening to, introduction questions to learn each interviewee’s profes- as opposed to the advertising world that tells you which sional background and personal background in podcast are the podcasts that are most worth listening to.” (I1) listening. We then asked interviewees to describe how I4 also discussed that discovering new podcasts is pull- they differentiate between podcasts, and their process focused, relying on users to make the decision, similar of evaluating diverse and varied content for listeners, if to the active pull strategy discussed by [13]. But the low applicable. Next, we prepared a list of various podcast barrier also means that there is “an endless amount of episodes on a topic (i.e., Science and Nature) for intervie- people talking at length about things that they know a wees. We asked the interviewees to analyze what they lot about.” Therefore, listeners need to actively “go after” thought would contribute to a varied and diverse podcast new podcasts to discover. listening experience and what they thought was missing from the list of podcast episodes. We also asked them 4.2. Highlight aspects from different to discuss current interface design elements, including search result lists, playlists, and recommendation grids, voices that might encourage listeners to explore more varied Our interview results emphasize that podcasts offer a podcasts. We also discussed interface design elements variety of different voices to listeners. As I3 defined it, that might prevent users from exploring different and podcast diversity is “about hearing from people who are diverse content. outside of your own normal circles” and “being exposed to different voices.” As mentioned before, the low bar- rier to become a podcaster enables listeners to explore 4. Results things from a different perspective, and the intimacy of podcasts also create a unique opportunity for listeners 4.1. Open format leads to varied content to personalize the listening experience from a different Our results highlight that podcasts are a unique medium angle. that offers diversity naturally due to its low barrier to Our interviewees spoke about how podcasts dif- entry. As I1 mentioned, “barrier to entry creates a lot fer from each other, including diversity of creators of diversity...there’s no requirement to becoming a pod- (hosts/guests) from different backgrounds, ethnicity, gen- caster.. as long as someone can hold a conversation on a ders, age groups, and social status (e.g. celebrities vs. non- subject for an X amount of time, [and] there’s not even celebrities as mentioned by I1), diversity of opinions and a structure around it.” The low or no barrier to entry viewpoints, diversity of presentation (“how these things also creates such a unique opportunity for many voices are being discussed” as mentioned by I2), variations of to be “elevated” and surfaced, which is not common in “vibe” (e.g., light-hearted vs. more serious conversation other mainstream media formats. Meanwhile, as pod- as discussed by I1), differences in popularity to “give casts become more and more popular, celebrities also use more opportunity to smaller podcasts” (I3), variations podcasts to voice their opinions. The mix of ordinary in formats and mixed media (e.g., conversational, story- people and powerful voices in podcasts creates a diversity telling, music, etc.), differences in topics (e.g., technology of voices in podcasts. Podcasts are “a great way to learn vs. meditation) (I4). Among all the dimensions, a wide about new topics or hear things from people you might variety of creators is most important, since it can also not have heard things from in your day-to-day life”(I3). naturally bring a diversity of viewpoints, as I2 argued. As suggested by our interview results, it is crucial to show a variety of “voices” when presenting podcasts to form that allows researchers to view and record partic- listeners. In our study, we define a list of dimensions ipants’ experiences during their interactions with the that highlight the differences between podcasts and then prototype. We recruited the participants from 20 to 35 we highlight those differentiating dimensions in the pro- years old living in the United States, who subscribe to totype. This can be achieved by current shelf-like de- a podcast streaming service, listen to podcasts multiple signs or achieved by creating different tags/labels for times per week, and explore new podcast shows at least each episodes once per week. We excluded participants who failed to test all three interfaces, as well as participants who tested our prototype twice. In total, we recruited 34 participants. 5. Ongoing Work: Study With We collected qualitative feedback from participants about Explanatory Labels what they liked and disliked about the interface, as well as measures for perceived diversity of the recommenda- As a result of our formative study, we built a web applica- tions. tion that presents podcast recommendations to users in the interface shown in 1. Participants were instructed to use our web application to select one episode they would 6. Discussion like to listen to from 10 episodes shown on the interface. We investigate how users can better differentiate among The episodes were from the same topic to avoid any top- the various kinds of podcast recommendations with la- ical effects. Users could not proceed to the next interface bels as a way to explain differences to users. In a forma- until they listened to the chosen episode for at least five tive study, we observed that diversity of creators (e.g., minutes. We followed a similar approach from [22] to ethnicity, gender, age groups, social status, etc.), differ- enable an organic exploration of podcasts. After partic- ences of presentation (interviews vs. storytelling), dif- ipants listened to the selected episode for five minutes, ferences in “vibe” (light-hearted vs. more serious con- they were directed to a survey to reflect on their podcast versation), and differences in length, to be particularly exploration process, including naming the episode they relevant dimensions in helping people differentiate be- chose to listen to, providing a summary of the episode, tween podcasts. Further, in a user study based on a web and the other questions mentioned in the next subsection. application that explains these dimensions through labels Once participants finished selections, they were asked to and summaries, we observed that the label explanations complete a questionnaire about their preferences on how help users better differentiate between varied podcast well the interface helped the user perceive the differences episodes when summaries are provided, and listeners between the content, usefulness, and informativeness. selected more podcasts in total when explanations were The prototype interface (see Figure 1) has seven key provided. These findings indicate that explanations for components for each episode: Episode Image, Episode differences in content may lead to more varied and di- Title, Publisher, Episode Description, Host/Guest Infor- verse selection of content. mation, Audio Preview, and Diversity Labels. We use the Our study suggests that listeners are able to better dif- term diversity to describe the differentiating dimensions ferentiate between podcast episodes when the differences of podcasts that we observed during the formative study. of the content are highlighted via labels and summaries We chose a list of labels to explain the differentiating and chose more distinct and varied content. We also find dimensions of each podcast. Based on host/guest infor- that users were more satisfied with their podcast explo- mation, we had “Diverse Voices” to highlight podcasters ration experience when provided explanatory labels. This from a wide variety of demographics. Based on the length finding is consistent with prior research suggesting that of each episode, we had “Deep Dive” for podcasts that are users are more satisfied when they are made aware of very long (close to or more than 60 minutes) and “Quick the available options [34] and are presented with diverse Listen” for podcasts that are short (less than 10 minutes). content [35, 36]. These findings are relevant for practi- Based on the vibe of each episode, we had “Serious Con- tioners who design and build recommender systems for versations” and “Light-hearted Conversations”. Based podcasts and other types of streaming content. If the on format, we had “Solo Podcast”, “Interview Podcast” aim of these systems is to encourage diversity in con- and “Storytelling Podcast.” To amplify up-and-coming sumption, providing recommendation explanations that podcasters, we had “New Voices” for podcasters who are help scaffold user awareness of the diversity of available new in podcast spaces. For our user study, we manually content can potentially help achieve that goal. labeled each episode with the applicable diversity labels We propose that explaining and highlighting the differ- and highlighted these label(s) in blue to captivate users’ entiating dimensions between podcasts is a first step to attention to the different dimensions between podcast understand how to recommend a diverse set of podcasts episodes while exploring. to listeners. Diversity can help reflect existing differences We recruited participants on usertesting.com, a plat- Figure 1: The data shown to users: (A) Episode Title, (B) Episode Art, (C) Episode Description from Publishers, (D) Diversity Labels Manually Selected and Highlighted, (E) Host/Guest Info by Social Media/Wikipedia/Personal Website, (F) Audio Preview that includes the first 30 seconds of the episode. Final Interface Design. in societies, to give equal access to any different points of house cause a stir, and listeners continue to view and actors, and to offer a wide range of choices for appreciate ads in brand new podcast listeners audiences[37]. From the individual perspective, listeners study, https://www.westwoodone.com/2021/05/11/ can be more satisfied with the options provided for them cumulus-media-and-signal-hill-insights-podcast\ with an awareness of all the options and choices they -download-spring-2021-report-podcast-listening-bolstered\ have [34]. If listeners only encounter similar content, -by-pandemic-subscription-services-and-clubhouse-cause\ then they may find themselves in an echo chamber [38]. -a-stir-and-listeners/, 2021. Accessed: 2021-9-25. Furthermore, from the content creator perspective, it is [5] 2021 global podcast statistics, demograph- essential for streaming platforms to encourage equitable ics & habits, https://podcasthosting.org/ opportunities for a diverse set of creators so listeners can podcast-statistics/, 2021. Accessed: 2021-9-25. encounter a broad spectrum of perspectives and back- [6] R. Berry, Willthe ipod killthe radio star?, Conver- grounds. A diverse exploration experience can then fa- gence 12 (2006) 143–162. cilitate change of ideas and dialogues between different [7] K. Thorson, C. Wells, How gatekeeping still matters: viewpoints and arguments so that users can have more Understanding media effects in an era of curated informed opinions and become less polarized [39, 34]. flows, Gatekeeping in transition (2015) 25–44. In future work, it will be an important next step to [8] N. Newman, R. Fletcher, A. Kalogeropoulos, R. K. understand how users learn and adopt their listening Nielsen, Reuters institute digital news report, habits through such explanatory labels, ultimately lead- Reuters Institute for the Study of Journalism (2019). ing to better exposure to various kinds of podcasts. Fur- [9] What everyone gets wrong about audio dis- thermore, we believe that longitudinal studies of user covery – but facebook, spotify might get exposure to explanations of podcast recommendations right, https://www.hypebot.com/hypebot/ and how it affects listening behavior will be an important 2021/05/what-everyone-gets-wrong-about-\ next step, e.g., through analyzing long term trends of pod- audio-discovery-but-facebook-spotify-might-get-right. cast discovery behavior, or the long term development html, 2021. Accessed: 2021-9-25. of user satisfaction. [10] People who listen to a lot of podcasts really are different, https://blog.mozilla.org/ux/2019/12/ people-who-listen-to-a-lot-of-podcasts-really-are-different/, References 2021. Accessed: 2021-9-25. [11] C. Drew, Edutaining audio: An exploration of ed- [1] S. McClung, K. Johnson, Examining the motives of ucation podcast design possibilities, Educational podcast users, Journal of Radio & Audio Media 17 Media International 54 (2017) 48–62. (2010) 82–95. [12] D. Misener, Podcast episodes got shorter in 2019, [2] R. Jones, H. Zamani, M. Schedl, C.-W. Chen, 2019. S. Reddy, A. Clifton, J. Karlgren, H. Hashemi, [13] M. Spinelli, L. Dann, Podcasting: The audio media A. Pappu, Z. Nazari, et al., Current challenges revolution, Bloomsbury Publishing USA, 2019. and future directions in podcast information access, [14] B. Carterette, R. Jones, G. F. Jones, M. Eskevich, arXiv preprint arXiv:2106.09227 (2021). S. Reddy, A. Clifton, Y. Yu, J. Karlgren, I. Sobo- [3] The infinite dial 2021, 2021. roff, Podcast metadata and content: Episode rel- [4] Spring 2021 report: Podcast listening bolstered evance and attractiveness in ad hoc search, in: by pandemic, subscription services and club- Proceedings of the 44th International ACM SIGIR Computing Machinery, New York, NY, USA, 2006, Conference on Research and Development in Infor- p. 124–131. URL: https://doi.org/10.1145/1111449. mation Retrieval, SIGIR ’21, Association for Com- 1111480. doi:10.1145/1111449.1111480. puting Machinery, New York, NY, USA, 2021, p. [25] I. Cantador, A. Bellogín, D. Vallet, Content-based 2247–2251. URL: https://doi.org/10.1145/3404835. recommendation in social tagging systems, in: Pro- 3463101. doi:10.1145/3404835.3463101. ceedings of the Fourth ACM Conference on Rec- [15] D. Bridge, J. P. Kelly, Ways of computing diverse ommender Systems, RecSys ’10, Association for collaborative recommendations, in: International Computing Machinery, New York, NY, USA, 2010, conference on adaptive hypermedia and adaptive p. 237–240. URL: https://doi.org/10.1145/1864708. web-based systems, Springer, 2006, pp. 41–50. 1864756. doi:10.1145/1864708.1864756. [16] F. Loecherbach, J. Moeller, D. Trilling, W. van At- [26] S. Sen, J. Vig, J. Riedl, Learning to recognize valu- teveldt, The unified framework of media diversity: able tags, in: Proceedings of the 14th Interna- A systematic literature review, Digital Journalism tional Conference on Intelligent User Interfaces, 8 (2020) 605–642. IUI ’09, Association for Computing Machinery, [17] L. Yang, Y. Wang, D. Dunne, M. Sobolev, M. Naaman, New York, NY, USA, 2009, p. 87–96. URL: https: D. Estrin, More than just words: Modeling non- //doi.org/10.1145/1502650.1502666. doi:10.1145/ textual characteristics of podcasts, in: Proceedings 1502650.1502666. of the Twelfth ACM International Conference on [27] M. Ames, M. Naaman, Why we tag: Motivations for Web Search and Data Mining, 2019, pp. 276–284. annotation in mobile and online media, in: Proceed- [18] J. Besser, M. Larson, K. Hofmann, Podcast search: ings of the SIGCHI Conference on Human Factors User goals and retrieval technologies, Online infor- in Computing Systems, CHI ’07, Association for mation review (2010). Computing Machinery, New York, NY, USA, 2007, [19] S. Resler, Do podcasts have a “discover- p. 971–980. URL: https://doi.org/10.1145/1240624. ability problem?”, https://jacobsmedia.com/ 1240772. doi:10.1145/1240624.1240772. do-podcasts-have-a-discoverability-problem-\ [28] A. Zubiaga, Enhancing navigation on wikipedia we-asked-leaders-in-the-space/, 2021. Accessed: with social tags, arXiv preprint arXiv:1202.5469 2021-07-26. (2012). [20] J. W. Morris, E. Patterson, Podcasting and its apps: [29] K. Bischoff, C. S. Firan, W. Nejdl, R. Paiu, Can Software, sound, and the interfaces of digital audio, all tags be used for search?, in: Proceedings of Journal of Radio & Audio Media 22 (2015) 220–230. the 17th ACM Conference on Information and doi:10.1080/19376529.2015.1083374. Knowledge Management, CIKM ’08, Association for [21] A. King, E. Zavesky, M. J. Gonzales, User pref- Computing Machinery, New York, NY, USA, 2008, erences for automated curation of snackable con- p. 193–202. URL: https://doi.org/10.1145/1458082. tent, in: 26th International Conference on In- 1458112. doi:10.1145/1458082.1458112. telligent User Interfaces, IUI ’21, Association for [30] D. Wang, T. Li, M. Ogihara, Are tags better than Computing Machinery, New York, NY, USA, 2021, audio features? the effect of joint use of tags and p. 270–274. URL: https://doi.org/10.1145/3397481. audio content features for artistic style clustering, 3450690. doi:10.1145/3397481.3450690. in: Proceedings of the 11th International Society [22] L. Yang, M. Sobolev, C. Tsangouri, D. Estrin, Under- for Music Information Retrieval Conference, ISMIR standing user interactions with podcast recommen- 2010, Proceedings of the 11th International Society dations delivered via voice, in: Proceedings of the for Music Information Retrieval Conference, ISMIR 12th ACM Conference on Recommender Systems, 2010, 2010, pp. 57–62. 11th International Society 2018, pp. 190–194. for Music Information Retrieval Conference, ISMIR [23] S. Basu, S. Gupta, M. Mahajan, P. Nguyen, J. C. 2010 ; Conference date: 09-08-2010 Through 13-08- Platt, Scalable summaries of spoken conversa- 2010. tions, in: Proceedings of the 13th International [31] N. Zhao, N. W. Kim, L. M. Herman, H. Pfister, R. W. Conference on Intelligent User Interfaces, IUI ’08, Lau, J. Echevarria, Z. Bylinskii, Iconate: Automatic Association for Computing Machinery, New York, compound icon generation and ideation, in: Pro- NY, USA, 2008, p. 267–275. URL: https://doi.org/ ceedings of the 2020 CHI Conference on Human 10.1145/1378773.1378809. doi:10.1145/1378773. Factors in Computing Systems, CHI ’20, Associa- 1378809. tion for Computing Machinery, New York, NY, USA, [24] G. Carenini, R. T. Ng, A. Pauls, Interactive multi- 2020, p. 1–13. URL: https://doi.org/10.1145/3313831. media summaries of evaluative text, in: Proceed- 3376618. doi:10.1145/3313831.3376618. ings of the 11th International Conference on In- [32] M. Chang, L. V. Guillain, H. Jung, V. M. Hare, J. Kim, telligent User Interfaces, IUI ’06, Association for M. Agrawala, Recipescape: An interactive tool for analyzing cooking instructions at scale, in: Proceed- ings of the 2018 CHI Conference on Human Factors in Computing Systems, CHI ’18, Association for Computing Machinery, New York, NY, USA, 2018, p. 1–12. URL: https://doi.org/10.1145/3173574.3174025. doi:10.1145/3173574.3174025. [33] M. Sienkiewicz, D. L. Jaramillo, Podcasting, the in- timate self, and the public sphere, 2019. [34] N. Helberger, K. Karppinen, L. D’acunto, Exposure diversity as a design principle for recommender systems, Information, Communication & Society 21 (2018) 191–207. [35] C.-N. Ziegler, S. M. McNee, J. A. Konstan, G. Lausen, Improving recommendation lists through topic di- versification, in: Proceedings of the 14th interna- tional conference on World Wide Web, 2005, pp. 22–32. [36] R. Hu, P. Pu, Helping users perceive recommen- dation diversity., in: DiveRS@ RecSys, 2011, pp. 43–50. [37] D. McQuail, Media performance: Mass communi- cation and the public interest, volume 144, Sage London, 1992. [38] T. T. Nguyen, P.-M. Hui, F. M. Harper, L. Terveen, J. A. Konstan, Exploring the filter bubble: the effect of using recommender systems on content diversity, in: Proceedings of the 23rd international conference on World wide web, 2014, pp. 677–686. [39] R. K. Garrett, N. J. Stroud, Partisan paths to expo- sure diversity: Differences in pro-and counteratti- tudinal news consumption, Journal of Communica- tion 64 (2014) 680–701.