=Paper=
{{Paper
|id=Vol-2554/paper1
|storemode=property
|title=Public Service Media, Diversity and Algorithmic Recommendation: Tensions between Editorial Principles and Algorithms in European PSM Organizations
|pdfUrl=https://ceur-ws.org/Vol-2554/paper_01.pdf
|volume=Vol-2554
|authors=Jannick Kirk Sørensen
|dblpUrl=https://dblp.org/rec/conf/recsys/Sorensen19
}}
==Public Service Media, Diversity and Algorithmic Recommendation: Tensions between Editorial Principles and Algorithms in European PSM Organizations==
https://ceur-ws.org/Vol-2554/paper_01.pdf
Public Service Media, Diversity and Algorithmic
Recommendation
Tensions between Editorial Principles and Algorithms in European PSM Organizations
Jannick Kirk Sørensen
Center for Communication, Media and Information Technologies (CMI), Aalborg University Copenhagen
js@cmi.aau.dk
ABSTRACT as they in return for public funding have obligations to pro-
Public Service Media (PSM) websites are an interesting case duce and promote content that e.g. strengthen local language
for the implementation of recommender systems for media and culture, help inclusion and societal cohesion, foster demo-
personalization, as the PSM organizations need to balance cratic processes, reflect diversity in opinions and worldviews.
the optimization of exposure with traditional but ill-defined In the age of broadcasting, priority was given to this type of
PSM policy goals such as fairness, viewpoint diversity and content by scheduling strategically before or after popular
transparency. Furthermore, the mathematical logic of rec- content. When content exposure is personalized via a recom-
ommender system needs to be adapted to the legacy broad- mender system, the question emerges: To which extend will
casting scheduling and publishing strategies and procedures. the algorithm and the editors / data curators recommend
Finally, as the PSM organizations step into new territories, content that reflects individual user interests if these do not
domestication and adaption of the recommender system tech- align with the content obligations? The question is relevant
nologies must take place while PSM organizations try to for more than the user experience. The funding and political
embrace the new knowledge and new professions associated legitimacy of public service media is closely connected with
with recommender systems. Based on 25 in-depth interviews the content obligations. If this content is not being exposed
conducted from December 2016 to April 2019, this paper to users, the legitimacy of the institution may be endan-
presents a cross-European analysis of the implementation of gered. Conversely, the programming and scheduling strategy
recommender systems in nine European public service me- of the PSB and PSM organizations have always been to offer
dia organizations from eight countries. The findings indicate popular content, such as music, sports and entertainment
that PSM organizations, although viewing personalisation to listeners, viewers and users. How will PSM organizations
as competitive necessity, approach recommendation systems approach this balancing?
with hesitation in order to maintain core PSM-values in the Based on 25 in-depth interviews with practitioners, like
online environment. Furthermore, although the collaborative project leaders, product owners, programmers, data scientists
filtering chosen by the PSM organizations indicate a user- and data curators the paper shows how they adopt PSM core
centered approach, curation systems on top of recommender values to algorithmic recommender systems. The paper is
systems re-install a broadcaster-centric approach. structured as follows: Below we introduce the special require-
ments to recommender systems that the PSM context poses.
CCS CONCEPTS After that we provide an overview of PSM interviewees’ ap-
General and reference Surveys and overviews; proaches to recommender system implementation. Finally,
Empirical studies; Information systems Collabo-
we point at a fundamental paradox related to the contextual
rationality of recommender systems.
rative filtering; Personalization.
KEYWORDS 2 BACKGROUND
Public service media is often linked to a set of ideals—the
Diversity, Filter bias, Editorial control, Public service media,
PSM remit [27]. UNESCO [43] lists as principles for public
Recommender systems, Curation systems
service broadcasting:
∙ Universality
1 INTRODUCTION
∙ Diversity
Recommender systems have been build to serve individual ∙ Independence
interests by selecting and filtering content, while public ser- ∙ Distinctiveness
vice broadcasting traditionally have aimed to reach the whole
population with content that editors think is beneficial for Public service media want to be agenda-setters with a mis-
citizens. The intention is to mediate a public sphere. Bringing sion of e.g. strengthening social cohesion, facilitating demo-
the two entities sounds as a contradiction in terms. Public cratic discussions, supporting national culture and language,
service media have a special position in the media landscape, ensuring freedom of speech, supporting cultural diversity [9].
Diversity and inclusion are today important values, e.g. in the
Copyright © 2019 for this paper by its authors. Use permitted under
Creative Commons License Attribution 4.0 International (CC BY 4.0).
BBC [2]. Nissen [31] argues that “influencing the listener’s
or viewer’s choices, and thus media consumption pattern is
INRA 2019, September 16–20, 2019, Copenhagen, Denmark Jannick Kirk Sørensen
the very reason why public media were established and why The case of recommender systems in the context of pub-
their existence has been upheld even in times of abundant lic service media has earlier been discussed at RecSys in a
media supply.” (p. 69) keynote by Berry from Canadian CBC [3], and by Fields,
Idealized, the principle for selecting and recommending Jones and Cowlishaw from the BBC [11]. Furthermore, Van
content to the audience is defined by a consensus about what den Bulck and Moe [45] compare public service media orga-
citizens should learn and know to participate in a society, cf. nizations’ different strategies in respect to personalization,
the classic BBC motto “enlighten, educate and entertain” [35]. with the Belgium VRT and the Norwegian NRK as case stud-
In reality, the production and exposure in the public service ies. Pöchhacker et al. [34] analyze the implementation of a
mass media has been and still is shaped by conflicting societal recommender system in a German context, discussed in [36].
interests, opportunistic possibilities and media politicians’ Sørensen [37] analyzes early attempts to personalize PSM
expectations. online services. Bodo [4], Kunert and Thurman [25], Thur-
Traditional algorithmic recommendation, i.e. collaborative man and Schifferes [41] analyze private and public media use
filtering (CF) [18, 19, 23], on the other side addresses the in- of personalization.
dividual user to optimize the exposure of content and, ideally,
protecting him / her from information overflow. Typically, 3 METHODOLOGY
no communicative intention is associated with the recommen- Using convenience sampling, a set of case were selected repre-
dation. Instead, key performance indicators (KPIs) are being senting both on-going projects and recommender systems in
measured. production. A sampling criteria is that the PSM organization
Will PSM organizations stick to the broadcaster-centric is a member of European Broadcasting Union (EBU) and
approach that prioritizes certain content or will they embrace located in Europe. Among the 54 EBU member organiza-
the customer-choice logic of personalization? A key to this tions located in Europe1 , we selected nine cases of develop-
question is how diversity is created in the recommendations: Is ment or implementation of recommender systems. Developers,
it defined by an editor defined to reflects different viewpoints data scientists and project leaders from the nine PSM or-
(source diversity) [29], or is calculated to reflect the global ganizations were interviewed via semi-structured in-depth
diversity of items available for recommendation, the diversity interviews.
within the recommended items (intra-list diversity) [8], or Interviewees were selected based on their direct involve-
does it take the point of departure in the users’ profile [46]? ment in the development or implementation of a recommender
Due to the special content obligations, diversity is a sensitive system. The strategic management level has thus not been
topic in public service media. included. The interview data was collected between December
In media politics, the diversity of media content providers, 2016 and April 2019. Additionally, developers, data scientists
ownership and media outlets is a measure of a well-functioning and project leaders from five other European PSM organiza-
democracy [17, 29]. In the area of news and media, a specific tions were interviewed: ARD (Germany), FranceTv (France),
concern has grown that algorithmic recommender systems MDR (Germany), Radio France (France), RAI (Italy), YLE
will lead to a loss of exposure diversity [29]. Subsequently, (Finland).
much discussion has revolved around filter bubbles [32] caused The interviews, 25 in total, were conducted as semi-structured
by collaborative filtering [5, 14, 30]. Suggestions for regulat- expert interviews based on a question guide [26]. The ques-
ing exposure diversity have been proposed by [6, 7, 16, 17] tion guide was adjusted to reflect both the development of
but also criticized [39]. Recently, measurements in a news the projects over time, as well as the insights gained during
context have however showed that algorithms are as good at the research. The interviews were audio-recorded (in total:
providing diversity as human editors [28] and that the prob- 820 minutes), transcribed and topic coded.
lem of filter bubbles may be overestimated [47]. Fletcher [13] Most PSM organisations implement content-based filtering
finds that news repertoires of users who find news via search (C-B) and collaborative filtering (CF). A number of PSM
engines have a more diverse and balanced news consumption, organisation also report to use a diversity module for rec-
while [12] reaches a more mixed conclusion. However, diver- ommendations. Finally, some PSM organisations use the
sity exposure concerns are still associated with algorithmic recommender system as part of a curation system, controlled
recommender systems [10, 15]. by editors and scheduling staff via business rules.
While diversity in itself is a problematic concept [22],
achieving diversity in algorithmic recommendations is a prob- 4 ANALYSIS
lem with importance for user experience and cross-selling of Reflecting our inductive and praxis-oriented approach, the
products [8, 24]. Sørensen and Schmidt [39] indicates that topics discussed by and with the interviewees can be grouped
there might exist a gap between how diversity is understood in the following categories.
and treated as a mathematical concept in recommender sys-
∙ Organizational Opportunities and Challenges
tems [8], and how media policy and editors construct the
∙ Diversity, Exposure and PSM Obligations
concept [29]. Where the former seldom takes the semantic
∙ Editorial control and curation vs. automation
context in consideration, the core parameter in the latter is
∙ KPIs - evaluation of exposure
contextually and socially defined view-point diversity [1].
1
https://www.ebu.ch/about/members accessed April 2, 2019
Public Service Media, Diversity and Algorithmic Recommendation INRA 2019, September 16–20, 2019, Copenhagen, Denmark
Table 1: Cases of PSM recommender systems
PSM organization Country Type of system Status Provider
BR Germany CF, C-B and Diversity module Running In-house (PEACH)
DR Denmark Curation system Implementing Vendor
EER Estonia C-B, stemming module Development In-house
NPO Netherlands C-B (CF in production) Running In-house
NRK Norway CF, C-B and Diversity module Running In-house
RTBF Belgium CF, C-B Running In-house (for sale)
RTVE Spain CF, C-F Developing External, custom built
SR Sweden CF, Curated playlists Running Partly In-house, partly vendor
ZDF Germany CF, C-B, Sequence and Curation system Running In-house
∙ Trust and Transparency main motive for implementing recommender systems. The ex-
∙ Build or buy? posure of the less used content is part of the PSM obligation
In the following lines we will summarize the interviewees’ of promoting diversity. But interviewees are also concerned
viewpoints on each of the topic areas, and discuss the per- of being accused for producing filter bubbles. Recommender
spectives. systems for PSM organizations must put a special emphasis
on diversity. In some cases, software modules have been build
4.1 Organizational Opportunities and to calculate diversity of the recommendations3 , but the idea
Challenges of regulating diversity via algorithms is generally received re-
luctantly. Most interviewees have an editorial understanding
A notion of strategic necessity was e.g. expressed Decem-
of diversity, arguing that the diversity is ensured by the global
ber 2016 by the EBU President Jean-Paul Philippot at a
set of items (videos) available in the system. In one case, a
Recommender Systems workshop for EBU members: “While
business rule ensures that the user, after having seen two
PSM still detain the largest audience in the EU, they should
pieces of content of the same genre is presented to two other
take these trends [increasing use of on-demand services] into
genres, e.g. two fiction videos are followed by documentary
account if they want to keep an advantage over Netflix and
and news videos. The solution is by the interviewee described
other competitors. This migration implies new tools such as
as “simplistic”, as the genre may not indicate diversity. The
efficient recommender systems, new skills, and new mindsets.”
2 issue of viewpoint diversity [1] was expressed as a problem
The objective here is the competitiveness of public service
by one interviewee, discussing whether a system is needed
media.
to make sure that a users watches different political parties’
Being able to offer personalization is seen as an important
video presentations prior to elections.
value proposition. Users expect personalization features, e.g.
Clear and official definitions of diversity are typically miss-
My page or Recommended to you to see the PSM web service
ing, leaving data curators on their own. Defining diversity has
or app as a competitive alternative to other media providers.
traditionally been an organizationally sensitive task, as diver-
Furthermore, the technical departments in the PSM organi-
sity is understood differently by media politicians, by PSM
zations want to build know-how on data-driven business, and
professionals and users. The absence of a formal definition
management wants to explore demand-driven production of
has traditionally enabled PSM editors to navigate between
content and position the organization in the media market.
conflicting ideas of diversity without direct confrontations.
However, the data analysts, programmers and data cu-
rators complain that it is difficult to explain recommender
4.3 Editorial control and Automation
systems to editors and journalists. E.g. the value of good
meta data is not always clear to the editorial staff. Many During the two years of interviewing, we have observed a
editorial assistants may simply copy-paste generic meta data shift in PSM staff understanding of recommender systems:
descriptions from one episode item to the next resulting in Initially skeptically approached as automated black boxes,
low-quality recommendations. recommender systems are now typically used as part of a
curation system. Furthermore, editors have gained control
4.2 Diversity, Exposure and PSM over the recommendations, e.g. by setting business rules for
certain slots on the page, by the tagging of content items or
Obligations
by giving priority to certain content in the keyword search
The improvement of user loyalty and content exposure, par- system. Many systems also have a manual override function-
ticular of long-tail content and under-performing content, is a ality.
2
RTBF - EBU Workshop on Algorithms and Soci-
3
ety 12-13/12 2016, accessed September 10, 2018 at the PEACH recommender system, produced in collaboration between
https://www.ebu.ch/contents/events/2016/12/big-data-initiative- BR (Germany), RTS (Switzerland) and EBU https://peach.ebu.io/
workshop-algorithms-and-society.html technical/tutorials/algorithms/diversified/ accessed October 1, 2018
INRA 2019, September 16–20, 2019, Copenhagen, Denmark Jannick Kirk Sørensen
A challenge for data curators is the manual configuration monitor the level of transparency of the PSM recommender
of business rules. Several interviewees report experiences systems, to protect users’ rights of transparency [20] and the
with the content-based filtering that recommended out-dated competitive balance between commercial media and publicly
sports- or news shows or inappropriate content: e.g. hard funded media.
debating on Islam after a satirical video on the same topic.
Maintaining the high level of editorial quality in the auto- 4.6 Build or buy?
mated recommendation is a challenging task. Most of the PSM organizations in our survey have either
The business rules, the curation and the placement of the build their in-house systems with help of open-source tools, or
algorithmic recommended content typically at the bottom commissioned external developers to develop a systems that
of the page or after video play-back imply that users only later can be transferred to the PSM organization. Only one
to a limited degree experience personalisation at the pages. PSM organization (DR in Denmark) has chosen a proprietary
There is only few opportunities to show users new, unknown solution offered by an external provider. DR argues that the
content. As novelty of the content often is a criteria for PSM organization neither is nor should be a technology
recommendation the pool of relevant content is reduced. organization; it should focus on content.
Furthermore, as content similar to the previous watched is As reasons for building own systems, interviewees pro-
given priority such as the next episode of a show / season the vide different reasons. For some, the in-house competence
window for diversity exposure is even smaller, interviewees building that enables fine-tuned configuration, control and
report. development is important. Other interviewees emphasize the
complex organizational (data-) structures and the special
4.4 KPIs - the evaluation of exposure requirements of PSM as motivation for custom-built systems.
With personalized recommender systems, PSM organizations Some interviewees explained that no usable external solu-
inherit a concept that has been invented for commercial tions were available at project start, thus stimulating them
purposes [33]. As PSM organization have other objectives to develop own systems. In the case of Estonia, the special
than e.g. improving click-through-rates, other KPIs must grammar of the Estonian language forced developers to build
be defined. However, defining PSM-specific KPIs, e.g. for their own stemming tools. Finally, ensuring the technical
the diversity of the recommendations, social cohesion or the independence of PSM organization in a future with IP-based
public service value is a challenging and controversial task. In- media consumption plays for many organizations a role for
terviewees thus report that they mainly use KPIs internally building their own systems [40].
in the development team to benchmark algorithm perfor- The lack of systems addressing specific PSM requirements
mance. The management levels of PSM organizations have (e.g. diversity) prompted two software projects to build
so far not shown much interest in KPIs from the recom- generic PSM recommender solutions: The PEACH project
mender systems; broadcast-related KPIs have a much bigger was initiated by Bayerische Rundfunk (BR) in Germany, and
managerial attention. PSM managements typically see the al- continued as a EBU project4 . In parallel, a system was devel-
gorithmic recommendation projects as experiments. In many oped by the Belgian PSM organization RTBF. In both cases,
organizations editors and journalists however have access to the interest from other PSM organizations to implement has
dashboards on online performance of the site and the content. however been limited.
5 DISCUSSION AND CONCLUSION
4.5 Trust and Transparency
PSM organizations are hesitant to personalize the content
Interviewees all stress that a PSM organization has a par-
exposure. Organizational acceptance of algorithmic recom-
ticular duty in explaining users how recommendations are
mendation is one barrier, the broadcast work-flow, lack of
being made, and how personal data is being treated. While
good meta-data and technical skills are other barriers. With
the latter, at least from a legal perspective is covered by
algorithmic recommendation legacy public service broadcast-
GDPR, the former is a bigger challenge. While content-based
ing organizations are stepping into unknown territory. On this
recommendations more easily can be explained to users, rec-
journey, the potential of loosing the core value proposition,
ommendations based on more complex data processing are
namely the agenda-setting, is felt as a danger. Movements
hard to explain in details to users. The problem of explaining
in the wilderness are thus careful, as PSM organizations are
recommendations is discussed in general terms e.g. by [42],
vulnerable to political and commercial accusations of being
but what specifically are PSM users’ expectations concern-
too commercial, too personal, too little distinctive or for lack
ing transparency? Interviewees agree that the PSM context
of different types of diversity could hurt the organizations
is particular sensitive as public service media stands out
severely.
as trusted institutions [38, 40]. Thus full transparency or
However, a more profound reason for the hesitation could
at least openness on recommendations is needed, however
be suggested, based in a paradox: The public service mis-
competitive interests may prevent PSM organisations from
sion is - as stated - to enlighten, inform and educate. The
sharing too many details about their recommender systems.
role of entertainment has either been the sugar-coating on
However, the special position of PSM organisations in the
4
media markets [21, 31, 44] may call for media regulation to https://peach.ebu.io accessed 2019-08-18
Public Service Media, Diversity and Algorithmic Recommendation INRA 2019, September 16–20, 2019, Copenhagen, Denmark
the pill of information, part of the provision of diversity or [17] Natali Helberger. 2015. Public Service Media - Merely Facilitating
the justification of the license fee. Based on the apparent or Actively Stimulating Diverse Media Choices? Public Service Me-
dia at the Crossroad. International Journal of Communication
rationality of the enlightenment, information and education 9 (2015), 17. http://ijoc.org/index.php/ijoc/article/view/2875
that PSM organizations are based on, PSM should warmly [18] Jonathan L Herlocker, Joseph A Konstan, Al Borchers, and John
Riedl. 1999. An algorithmic framework for performing collab-
embrace recommender systems. These systems should be per- orative filtering. In 22nd Annual International ACM SIGIR
fectly rational tools for efficient dissemination of knowledge. Conference on Research and Development in Information Re-
The hesitation we notice among the PSM organization could trieval, SIGIR 1999. Association for Computing Machinery, Inc,
230–237.
either be explained by the insensitivity of well-known recom- [19] Jonathan L. Herlocker, Joseph A. Konstan, and John Riedl. 2000.
mender systems to the type of rationality desired, or it could Explaining collaborative filtering recommendations. In Proceed-
point to the ambiguity and fragility of the enlightenment ings of the 2000 ACM conference on Computer supported co-
operative work - CSCW ’00. ACM Press, New York, New York,
that PSM organizations claim to promote. The absence of USA, 241–250. https://doi.org/10.1145/358916.358995
clear definitions of diversity and of the relation between the [20] High-Level Expert Group on Artificial Intelligence. 2019. Ethics
Guidelines for Trustworthy AI. Technical Report. European
personal and the societal could point to the latter. Not defin- Commission, Brussels. https://ec.europa.eu/newsroom/dae/
ing or operationalizing these concepts can however ironically document.cfm?doc{ }id=60419https://ec.europa.eu/digital-
be a strategically fortunate position for PSM organizations. single-market/en/news/ethics-guidelines-trustworthy-ai
[21] Karol Jakubowicz. 2007. Public Service Broadcasting in the 21st
Century. What Chance for a New Beginning? In From Public
REFERENCES Service Broadcasting to Public Service Media, Gregory Ferrell
[1] Christian Baden and Nina Springer. 2017. Conceptualizing view- Lowe and Jo Bardoel (Eds.). Nordicom, Göteborg Universitet,
point diversity in news discourse. Journalism: Theory, Practice Kungälv, 29–49.
& Criticism 18, 2 (feb 2017), 176–194. https://doi.org/10.1177/ [22] KENNETH JUNGE. 1994. Diversity of ideas about diversity
1464884915605028 measurement. Scandinavian Journal of Psychology 35, 1 (mar
[2] BBC. 2017. Diversity and Inclusion Strategy 2016-20. 1994), 16–26. https://doi.org/10.1111/j.1467-9450.1994.tb00929.
(2017). http://downloads.bbc.co.uk/diversity/pdf/diversity-and- x
inclusion-strategy-2016.pdf [23] Joseph A Konstan, Bradley N Miller, David Maltz, Jonathan L
[3] Christopher Berry. 2018. Recommending social cohesion. In Pro- Herlocker, Lee R Gordon, and John Riedl. 1997. GroupLens:
ceedings of the 12th ACM Conference on Recommender Sys- applying collaborative filtering to Usenet news. Commun. ACM
tems - RecSys ’18. ACM Press, New York, New York, USA, 3–3. 40, 3 (mar 1997), 77–87. https://doi.org/10.1145/245108.245126
https://doi.org/10.1145/3240323.3267110 [24] Matevž Kunaver and Tomaž Požrl. 2017. Diversity in recommender
[4] Balázs Bodo. 2018. Means, not an end (of the world) - the systems: A survey. Knowledge-Based Systems 123 (2017), 154–
customization of news personalization by European news media. 162. https://doi.org/10.1016/j.knosys.2017.02.009
(2018). https://ssrn.com/abstract=3141810 [25] Jessica Kunert and Neil Thurman. 2019. The Form of Con-
[5] Engin Bozdag and Jeroen van den Hoven. 2015. Breaking the tent Personalisation at Mainstream, Transatlantic News Out-
filter bubble: democracy and design. Ethics and Information lets: 20102016. Journalism Practice (2019), 1–22. https:
Technology 17, 4 (dec 2015), 249–265. https://doi.org/10.1007/ //doi.org/10.1080/17512786.2019.1567271
s10676-015-9380-y [26] Steinar Kvale. 2007. Doing Interviews. SAGE, Los Angeles,
[6] Mira Burri. 2016. Nudging as a Tool of Media Policy. In Nudging - London, New Delhi, Singapore, Washington DC.
Possibilities, Limitations and Applications in European Law and [27] Gregory Ferrell Lowe and Taisto Hujanen. 2003. Broadcasting
Economics. Springer International Publishing, Cham, 315–341. & Convergence: New Articulations of the Public Service Remit.
https://doi.org/10.1007/978-3-319-29562-6 16 RIPE@2003. Nordicom, Göteborg. 335 pages.
[7] Mira Burri and Natali Helberger. 2015. Public Service Media [28] Judith Möller, Damian Trilling, Natali Helberger, and Bram van
and Exposure Diversity: Introduction. International journal of Es. 2018. Do not blame it on the algorithm: an empirical assess-
communication 9 (2015), 1319–1323. ment of multiple recommender systems and their impact on con-
[8] Pablo Castells, Neil J. Hurley, and Saul Vargas. 2015. Novelty and tent diversity. Information, Communication & Society 21, 7 (jul
Diversity in Recommender Systems. In Recommender Systems 2018), 959–977. https://doi.org/10.1080/1369118X.2018.1444076
Handbook. Springer US, Boston, MA, 881–918. https://doi.org/ [29] Philip M. Napoli. 2011. Exposure Diversity Reconsidered. Journal
10.1007/978-1-4899-7637-6 26 of Information Policy 1 (2011), 246–259.
[9] EBU. 2007. EBU viewpoint: The UNESCO Convention on [30] Tien T. Nguyen, Pik-Mai Hui, F. Maxwell Harper, Loren Terveen,
Cultural diversity. http://www.ebu.ch/CMSimages/en/ and Joseph A. Konstan. 2014. Exploring the filter bubble. In
CulturalDiversity{ }03{ }07{ }E{ }tcm6-50247.pdf Proceedings of the 23rd international conference on World wide
[10] Sarah Eskens, Natali Helberger, and Judith Moeller. 2017. Chal- web - WWW ’14. ACM Press, New York, New York, USA, 677–
lenged by news personalisation: five perspectives on the right 686. https://doi.org/10.1145/2566486.2568012
to receive information. Journal of Media Law 9, 2 (jul 2017), [31] Christian S Nissen. 2006. No public service without both Public
259–284. https://doi.org/10.1080/17577632.2017.1387353 and Service - Content provision between the Scylla of populism
[11] Ben Fields, Rhianne Jones, and Tim Cowlishaw. 2018. The Case and the Charybdis of elitism. In Making a Difference: Public
for Public Service Recommender Algorithms. (2018), 22–24 pages. Service Broadcasting in the European Media Landscape, Chris-
https://piret.gitlab.io/fatrec2018/program/fatrec2018-fields.pdf tian S Nissen (Ed.). John Libbey Publishing, Eastleigh, Chapter 5,
[12] Seth Flaxman, Sharad Goel, and Justin M. Rao. 2016. Filter 65–82.
Bubbles, Echo Chambers, and Online News Consumption. Public [32] Eli Pariser. 2011. The filter bubble : what the Internet is hiding
Opinion Quarterly 80, S1 (2016), 298–320. https://doi.org/10. from you. Penguin Books.
1093/poq/nfw006 [33] Don Peppers and Martha Rogers. 1993. The One to One Future:
[13] Richard Fletcher and Rasmus Kleis Nielsen. 2018. Automated Building Relationships One Customer at a Time. Currency
Serendipity. Digital Journalism 6, 8 (sep 2018), 976–989. https: Doubleday, New York.
//doi.org/10.1080/21670811.2018.1502045 [34] Nikolaus Pöchhacker, Marcus Burkhardt, Andrea Geipel, and
[14] Mario Haim, Andreas Graefe, and Hans-Bernd Brosius. 2017. Jan-Hendrik Passoth. 2017. Interventionen in die Produktion
Burst of the Filter Bubble? Digital Journalism (jul 2017), 1–14. algorithmischer Öffentlichkeiten: Recommender Systeme als Her-
https://doi.org/10.1080/21670811.2017.1338145 ausforderung für öffentlich-rechtliche Sendeanstalten. kommu-
[15] B. Hallinan and T. Striphas. 2016. Recommended for you: The nikation@ gesellschaft 18 (2017), 25.
Netflix Prize and the production of algorithmic culture. New [35] John C W Reith. 1924. Broadcast Over Britain. Hodder and
Media & Society 18, 1 (jan 2016), 117–137. https://doi.org/10. Stoughton, London.
1177/1461444814538646 [36] Jan-Hinrik Schmidt, Jannick Kirk Sørensen, Stephan Dreyer,
[16] Natali Helberger. 2011. Diversity by design. Journal of Informa- and Uwe Hasebrink. 2018. Wie können Empfehlungssysteme zur
tion Policy 1 (2011), 441–469.
INRA 2019, September 16–20, 2019, Copenhagen, Denmark Jannick Kirk Sørensen
Vielfalt von Medieninhalten beitragen? Perspektiven für öffentlich- //doi.org/10.1007/978-1-4899-7637-6 10
rechtliche Rundfunkanstalten. Media Perspektiven 11 (2018), [43] UNESCO. 2001. Public Broadcasting Why? How? Technical
522–531. Report. UNESCO, Paris. 1–28 pages. http://unesdoc.unesco.org/
[37] J. K. Sørensen. 2013. PSB goes personal : The failure of images/0012/001240/124058eo.pdf
personalised PSB web pages. MedieKultur 29, 55 (2013), 43– [44] Hilde Van den Bulck. 2008. Can PSB Stake its Claim in a Media
71. http://ojs.statsbiblioteket.dk/index.php/mediekultur/article/ World of Digital Convergence?: The Case of the Flemish PSB
view/7993 Management Contract Renewal from an International Perspective.
[38] Jannick Kirk Sørensen and Sokol Kosta. 2019. Before and Convergence 14, 3 (2008), 335–349. https://doi.org/10.1177/
After GDPR: The Changes in Third Party Presence at Pub- 1354856508091085
lic and Private European Websites. In WWW ’19 Companion [45] Hilde Van den Bulck and Hallvard Moe. 2017. Public service
Proceedings of the The Web Conference 2019. ACM. https: media, universality and personalisation through algorithms: map-
//doi.org/10.1145/3308558.3313524 ping strategies and exploring dilemmas. Media, Culture & So-
[39] Jannick Kirk Sørensen and Jan-Hinrik Schmidt. 2016. An Algorith- ciety (oct 2017), 016344371773440. https://doi.org/10.1177/
mic Diversity Diet? Questioning Assumptions behind a Diversity 0163443717734407
Recommendation System for PSM. In RIPE@2016. Antwerp, [46] Michele Zanitti, Sokol Kosta, and Jannick Sørensen. 2018. A User-
Belgium. http://ripeat.org/library/2016 Centric Diversity by Design Recommender System for the Movie
[40] Jannick Kirk Sørensen and Hilde Van den Bulck. 2018. Public Application Domain. In Companion of the The Web Conference
service media online, advertising and the third-party user data 2018 on The Web Conference 2018 - WWW ’18. ACM Press,
business. Convergence: The International Journal of Research New York, New York, USA, 1381–1389. https://doi.org/10.1145/
into New Media Technologies (aug 2018). https://doi.org/10. 3184558.3191580
1177/1354856518790203 [47] Frederik J Zuiderveen Borgesius, Damian Trilling, Judith Moeller,
[41] Neil Thurman and Steve Schifferes. 2012. The Future of Personal- Balázs Bodó, Claes H de Vreese, and Natali Helberger. 2016.
ization at News Websites. Journalism Studies 13, 5-6 (oct 2012), Should we worry about filter bubbles? Internet Policy Review 5,
775–790. https://doi.org/10.1080/1461670X.2012.664341 1 (2016).
[42] Nava Tintarev and Judith Masthoff. 2015. Explaining Rec-
ommendations: Design and Evaluation. In Recommender Sys-
tems Handbook. Springer US, Boston, MA, 353–382. https: