<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta />
    <article-meta>
      <title-group>
        <article-title>Can Celebrities Burst Your Bubble??</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Tugrul</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>n Elm</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Kristin</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          <institution>EPFL</institution>
          ,
          <country country="CH">Switzerland</country>
        </aff>
      </contrib-group>
      <abstract>
        <p>Polarization is a growing, global problem. As such, many social media based solutions have been proposed to try to reduce it. In this study, we propose a new solution that recommends topics to celebrities to encourage them to join a polarized debate and increase exposure to contrarian content | bursting the lter bubble. Using a state-of-the art model that quanti es the degree of polarization, this paper makes a rst attempt to empirically answer the question: Can celebrities burst lter bubbles? We use a case study to analyze how people react when celebrities are involved in a controversial topic and conclude with a list possible research directions.</p>
      </abstract>
      <kwd-group>
        <kwd>lter bubble</kwd>
        <kwd>polarization</kwd>
        <kwd>twitter</kwd>
      </kwd-group>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>Introduction</title>
      <p>
        Polarization is a state in which the public is divided into groups with opposing
opinions on an issue [
        <xref ref-type="bibr" rid="ref4">4</xref>
        ]. Polarization is regarded as a threat to democracy and
is detrimental to healthy dialogue in a community. Echo chambers | the
phenomena in which individuals only hear the side of a debate they already agree
with | are a primary driver of polarization, as they are where extremist ideas
foster [
        <xref ref-type="bibr" rid="ref18">18</xref>
        ]. Social media platforms themselves hold some responsibility for the
formation of such echo chambers; the algorithms that determine the information
diet of users are believed to rank belief-reinforcing information higher as a result
of maximizing engagement, which in turn minimizes cognitive dissonance. The
term lter bubble, coined by Eli Pariser [
        <xref ref-type="bibr" rid="ref17">17</xref>
        ], describes this phenomenon by which
echo chambers are caused by the design of the system.
      </p>
      <p>
        Due to their potential detriment to democracy and society, others have
proposed methods to burst lter bubbles in order to reduce polarization. Many of
these solutions rely on action by the social media platforms, ignoring the fact
that social networks have created this problem and may not be incentivised to
act, including recommending users [
        <xref ref-type="bibr" rid="ref8">8</xref>
        ] (aided by intermediary topics [
        <xref ref-type="bibr" rid="ref10">10</xref>
        ]) or
content [
        <xref ref-type="bibr" rid="ref13">13</xref>
        ] with opposing opinions, and presenting the information in a di erent
way (i.e. showing the credibility of a source) [
        <xref ref-type="bibr" rid="ref21">21</xref>
        ]. Other work focuses on raising
awareness to users and therefore requires action on the part of the users who are
in a lter bubble. These include exposing users to contrarian news [
        <xref ref-type="bibr" rid="ref7">7</xref>
        ], raising
awareness of one's connections' and own biases [
        <xref ref-type="bibr" rid="ref6">6</xref>
        ], convincing some users in the
social network to reduce the overall polarization via education [
        <xref ref-type="bibr" rid="ref14">14</xref>
        ].
      </p>
      <p>
        We present a new recommendation scheme that requires neither buy-in from
the social network nor action on the part of those in the lter bubble. This
scheme bursts lter bubbles by recommending polarizing topics to in uential
users, i.e. celebrities. Prior work has shown that celebrities increase exposure
and in uence opinion on controversial subjects like vaccination [
        <xref ref-type="bibr" rid="ref2">2</xref>
        ], suggesting
that their involvement in a debate can reduce polarization by means of exposing
users to counter opinions. Other work has shown that users value connections
while selecting content [
        <xref ref-type="bibr" rid="ref15">15</xref>
        ], so messages conveyed by celebrities whom users are
connected to, are likely to be valued over content from non-connections. Finally,
this method leverages the fact the social media is a small-world network [
        <xref ref-type="bibr" rid="ref22">22</xref>
        ]
so people with counter opinions are connected to the same users who do not
explicitly posit their opinions. For example, LeBron James is both followed by
both liberals and conservatives and also is the only liberal source in most of his
conservative followers' pro les and therefore can burst those users' bubbles [
        <xref ref-type="bibr" rid="ref12">12</xref>
        ].
      </p>
      <p>We identi ed the following research questions related to this scheme and
answer them in the remainder of this paper:
RQ1 If a celebrity joins a debate on a controversial topic will exposure to the
contrarian content be increased and will polarization be reduced?
RQ2 How to select celebrities that would lead to such e ects?
RQ3 Will such exposure mitigate the extreme opinions and hence decrease
polarization?
RQ4 How would people react when a celebrity joins the debate? Will we observe
mitigation of thoughts or back re e ect?</p>
      <p>We de ne celebrity as \anyone popular and although not strictly impartial,
not politically polarized." To address RQ1 and RQ2, we empirically show that
the inclusion of popular and neutral nodes into a polarized graph decreases
the polarization of the graph, hence celebrity inclusion reduces polarization. To
address RQ3 and RQ4, we perform a qualitative analysis of users' reactions to
celebrities participating in a controversial topic.
2
2.1</p>
    </sec>
    <sec id="sec-2">
      <title>Empirical Results on E ect of Celebrity Involvement</title>
      <sec id="sec-2-1">
        <title>Theoretical Background</title>
        <p>
          First, we address RQ1 to determine the e ect that a celebrity joining a debate
will have on polarization. We use the quantifying controversy model [
          <xref ref-type="bibr" rid="ref9">9</xref>
          ] which
quanti es the controversy of a topic by computing how likely a user on one side
of a polarized debate is to be exposed to content disseminated by a popular user
on the opposing side. As such, this model serves as a proxy for polarization score
on a topic. We recap the model brie y before detailing our application of it.
Quantifying Controversy Model First, consider a social graph G(V; E) in which
vertices V are users who hold an opinion on a topic and edges E are the social
connections between them. G is partitioned into two disjoint sets of users, X
and Y , which possibly correspond to the two di erent sides of the discussion.
For each node in a set of randomly selected nodes, a random walk is started and
concludes when it reaches any high-degree user. Let PAB be the probability that
a random walk begins in partition A and ends in partition B. The \Random
Walk Controversy Score" (RWC) is the di erence of the probabilities that a
random walk begins and ends in the same partition (PXX and PY Y ) and that a
random walk begins and ends in the other partition (PXY and PY X ).
        </p>
        <p>RW C = PXX PY Y</p>
        <p>PXY PY X</p>
        <p>The resulting RW C score is inversely correlated with the likeliness of
exposure to popular content from the opposite side and implies polarization of the
debate on the topic.</p>
        <p>Since the users in the same partition are well connected due to the homophily
principle, we assume that in a polarized network content produced in the same
partition has the same stance. Conversely, content produced by users in di erent
partitions have di erent stances. Thus, in a polarized network, content that is
quickly reached by a user (via a random walk) is from the same stance as the
user. Hence, the user is trapped in an echo chamber. We leave a model for echo
chambers that works without this assumption about stances to future work.</p>
        <p>In the context of Twitter, the topic is modeled as tweets containing relevant
hashtags to a seed hashtag that de nes the topic. The social network, G, is built
by including users who authored these tweets. The links between the nodes of
these networks could be following, retweeting, or both. We use following
relationships as they are a better proxy to measure exposure to content from the
connected user. As such, G is directed and users receive incoming links from
their followers. The users who are recommended the topic will be added to the
G if they accept the recommendation. Links between users already in G and
a newly added user will be added to G if these users already follow the newly
added user. Our problem is then to identify such nodes to recommend the topic
so that their inclusion in the network will decrease RW C of G.</p>
        <p>
          One issue with this topic modeling approach is that it draws mostly from
politically interested users and hence exaggerates the polarization of a popular
topic that involves many hashtags and keywords [
          <xref ref-type="bibr" rid="ref19">19</xref>
          ]. However, this approach
is plausible when you consider the scenario in which a user clicks on a hashtag
about a controversial topic that is trending. We assume that the tweets from
users' connections are more likely to be ranked higher and hence the user will
be in a lter bubble when presented with tweets on that topic.
2.2
        </p>
      </sec>
      <sec id="sec-2-2">
        <title>Node Addition Problem</title>
        <p>In order to determine which celebrities to select (RQ2), we de ne the Node
Addition Problem as determining which nodes to add to the network in order
to maximize the reduction of the controversy of the topic. Consider a topic T
and a social graph G = (V; E) made up of users who participated in a debate
about T . Let the controversy score of G be RW C(G). Let the Potential Social
Graph G (V ; E ) be the union of G = (V; E) and all the vertices that are
connected to V but did not discuss T and the edges connecting them to V . The
node addition problem is to nd a set of k nodes V 0 not in G but in G to
add to G and obtain the Augmented Graph G = (V ; E ) which maximizes
RW C(G) - RW C(G ).</p>
        <p>
          We hypothesize that k nodes that maximize the decrease of RW C will be
those who have 1) high in-degree 2) edges distributed evenly between two
partitions. We leave mathematical proof for future work and only present empirical
results. To nd those k nodes, we use the Fagin algorithm [
          <xref ref-type="bibr" rid="ref5">5</xref>
          ] to rank nodes by
1) their in-degree and 2) their minimum ratio of connections' to one partition
over all connections. The rst is a proxy for popularity and the second a proxy
for neutrality. We compute RW C for each candidate and choose the k nodes
which yield the largest RW C decrease. We assume these k nodes will consist
of candidates which minimize RW C individually to avoid computing RW C for
every k combination of nodes, which is very costly.
2.3
        </p>
      </sec>
      <sec id="sec-2-3">
        <title>Experimental Results</title>
        <p>
          For the empirical results, we used the follower data of users who participated
in the debate about the topic #Russia March. This topic is studied in [
          <xref ref-type="bibr" rid="ref9">9</xref>
          ] and
is already found to be polarized. We rst created the follower graph G which
involves only the users who tweeted with #Russia March and relevant hashtags.
Then we collected all the followees of users in the G to create the augmented
graph. See Figure 1 for the two graphs.
        </p>
        <p>We used two baselines to evaluate our node selection process. First, the most
popular nodes to study the e ect neutrality and second \random nodes", which
are arti cially created nodes that have xed degree (50) and are connected to 25
randomly chosen nodes in each partition to study the e ect of popularity. Figure
2 shows the nal polarization score with respect to the number of nodes added.
Colors denote the method used to choose nodes. Although adding popular nodes
seems bene cial initially, they become ine ective after 20 nodes. As the results
indicate, most popular and neutral nodes reduce polarization, who happen to be
celebrities by our de nition.</p>
        <p>A possible side e ect of this method is that users will unfollow celebrities who
discuss controversial topics and join the debate. To simulate this, we randomly
break incoming links of the k nodes. As seen in Figure 3, the polarization is still
reduced unless the celebrities lost most of their followers, which is unrealistic.</p>
        <p>The empirical results show that polarization as measured by RW C reduces
when celebrities join a controversial debate. It is not clear, however, how much
reduction in RW C equates to real-life implications. In the next section, we
explore what happens in a real-life study of celebrities weighing in on a controversial
topic.</p>
        <p>
          The Case of 2019 I_stanbul Election Rerun Decision
The 2019 I_stanbul Election Rerun was a controversial decision by the Supreme
Electoral Council when the opposition's candidate won, overruling the AKP's
candidate by a slim margin despite high voter turnout. The decision was deemed
unfair by supporters of the opposition. Many celebrities started to tweet after
the opposition candidate, Ekrem I_mamoglu, gave a speech and said \Everyone
should speak, the celebrities should speak!" [
          <xref ref-type="bibr" rid="ref11">11</xref>
          ]. This serves as a suitable case
study as those celebrities' messages reached a very wide audience; many have
more than 100,000 followers.
        </p>
        <p>We address RQ4 and determine how users react to celebrities joining a debate
by analyzing immediate reactions to the celebrity tweets regarding this decision.
We also address RQ3 on this real-world data set and determine via a longitudinal
study if this celebrity intervention actually made a di erence in this case.</p>
        <p>
          We selected 81 celebrities from a list of Turkish celebrities [
          <xref ref-type="bibr" rid="ref1">1</xref>
          ] who tweeted
in favor of the opposition's candidate on the night of 6 May 2019. We collected
their tweets, retweets, and replies to their tweets for one week using Twitter's
Streaming API. By manual inspection, we found that 47 of them are in
cinematv business, 24 in music, and 10 in other elds. Judging from the tweets since
September 2019, 43 celebrities were already found to be posting frequently about
controversial but apolitical topics and 7 were found to be occasionally posting
about such topics. The remaining 31 only post personal and professional updates.
No celebrity showed explicit political a liation or criticism towards a party or
government, but 25 criticized recent government policies, 15 infrequently posted
content that could be interpreted as anti-government, and the remaining 41
appear politically neutral. This suggests that for most of the celebrities in our
dataset, the I_stanbul Election Rerun was the rst time they spoke out on a
political topic on Twitter.
        </p>
        <p>For those celebrities with more than one relevant tweet, we selected the tweet
that received the most replies for each celebrity in 6-7 May 2019, then annotated
each celebrity according to their stance according to that tweet. Note that 60 of
the celebrities showed explicit support to the opposition's candidate (30 used the
opposition slogan #HerSeyCokGuzelOlacak), while 8 celebrities only commented
on the unfairness of the decision of rerun. In addition, 11 celebrities called for
citizens to vote in the rerun, and 2 called for other celebrities to tweet.</p>
        <p>We randomly sampled 10 direct replies per celebrity tweet. We annotated
these replies according to 1) stance on the celebrity (positive, negative, neutral)
and 2) narratives they contain. Note that not all replies had a narrative. We
removed the celebrity tweets that were irrelevant or had less than 10 replies. We
annotated 679 tweets in total. We found that 434 tweets had a positive stance
towards the celebrity and their idea, 178 tweets had a negative stance (with 31
containing insults), and 60 tweets had a neutral stance. Our analysis indicates
the following narratives are prevalent in tweets, which have a non-positive stance
unless otherwise speci ed.
1. Counter argument: The celebrity is wrong as the opposition has committed
voter fraud and the decision was correct. (n = 29)
2. Ad hominem: The celebrity is wrong or does not deserve a voice on the matter
due to their past political actions, or their character is not harmonious with
their idea. (n = 26)
3. Self-interests: The celebrity is behaving this way not because of patriotism
but for self interest because they are not successful in their work. (n = 19)
4. Questioning authority: The celebrity does not have a right to speak because
they are not a real celebrity. (n = 16)
5. Whataboutism: The celebrity's patriotism is in question as they did not
react to soldiers killed by terrorist attacks or on the night of the July 15,
2016 coup. (n = 11)
6. Too late: Positive with the celebrity's opinion but blames them or celebrities
in general for acting too late. (n = 10)
7. Reactionary: The celebrities should not expose their political beliefs or
champion one political side or should be remembered with their art only. (n =
5)
8. Hopeless: The situation is hopeless and they will not win the rerun, although
the celebrity is championing hope. (negative: n = 3), (positive: n = 2)
9. Mitigate: Indicates a non-polarized a liation, but agrees with the celebrity
on that issue with positive stance. (n = 2)
10. Back re: Threatens the celebrity (n = 9), or indicates they will no longer
follow them (n = 3).</p>
        <p>Based on this analysis, we make the following observations:</p>
      </sec>
      <sec id="sec-2-4">
        <title>The celebrities' messages reached users with the opposite stance:</title>
        <p>The presence of negative replies from the opposite side shows that the goal of
bursting the lter bubbles has been achieved.</p>
      </sec>
      <sec id="sec-2-5">
        <title>The source of the message matters: if the celebrity is not having a</title>
        <p>successful career and is not respected, they have negative reactions implying
those elements. Even the supportive replies had sarcastic elements in some cases.
Their past political deeds also matter especially if they took a pro-government
stance before.</p>
      </sec>
      <sec id="sec-2-6">
        <title>The content of the message also matters: if the content is sarcastic or</title>
        <p>has some logical aw, the replies indicate this rather than agreement or
disagreement which causes distraction. The reactions which would lead to meaningful
discussions (although still rare) come when the celebrity's tweet contains an
argument.</p>
      </sec>
      <sec id="sec-2-7">
        <title>There is no evidence of correlation between political activity on</title>
      </sec>
      <sec id="sec-2-8">
        <title>Twitter and stances of replies a celebrity gets: We averaged the stances</title>
        <p>of replies each celebrity gets (1 for positive, 0 for neutral and -1 for negative.)
and ran a t-test for those who were political / commented on contemporary
issues on Twitter and those who do not. The stance of replies turned out to be
independent of both factors as the p-value was insigni cant.</p>
      </sec>
      <sec id="sec-2-9">
        <title>The negative replies mainly come from politically motivated users:</title>
        <p>We inspected 100 users who had a negative stance and replied to celebrities.
Three of these users had deleted their accounts and 15 were suspended. Among
the 82 remaining users, 55 were very polarized | their account seemed to be
opened only to share pro-AKP content, and they constantly spread fake news
about the opposition. This suggests that the replies should not be taken as a
genuine public reaction during analysis as they also likely part of coordinated
attacks. However, the narratives they contain are still important as they may
in uence genuine Twitter users.</p>
      </sec>
      <sec id="sec-2-10">
        <title>Both mitigation and back re e ect appear to be small: Inferring</title>
        <p>from the reactions, we had only two cases where an artist's presence made a
positive e ect. The back re e ect is also small; follower counts increased rather
than decreased, which may show that they do not go out of favor dramatically.
4</p>
      </sec>
    </sec>
    <sec id="sec-3">
      <title>Open Questions</title>
      <p>Celebrity acceptance: Would a celebrity accept the recommendation to join
the debate 1) by public request, 2) by platform request, 3) by a fellow celebrity,
or 4) not at all?</p>
      <p>Factors on user's reactions: Are user's reactions to celebrities joining a
political debate dependant on the side they join, on whether they try to mitigate
extreme opinions, or whether multiple celebrities observe the same behavior?</p>
      <p>Modi cation of the platform: What would be the e ects of a modi cation
to the platform so that 1) it recommends topic to users that would increase
exposure to the contrarian content and decrease polarization and 2) it recommends
content by such users to users with extreme views?</p>
      <sec id="sec-3-1">
        <title>Categorization of celebrity candidates: Not all popular accounts are</title>
        <p>suitable to recommend topics to comment on, i.e. corporate and media related
accounts may be less likely to take the recommendation for fear of backlash.</p>
      </sec>
      <sec id="sec-3-2">
        <title>When do celebrities weigh-in on a controversial topic? Do they join</title>
        <p>in early and help the topic spread or do they join later? Is it due to
peerpressure, self-interest, or neither? Such analysis would be useful in determining
if recommending topics to celebrities is realistic or helpful.</p>
        <p>Non-political users: If many celebrities are tweeting about political topics
due to this method, users who use Twitter for entertainment purposes and not
for political engagement may be negatively impacted or leave the platform.</p>
      </sec>
      <sec id="sec-3-3">
        <title>Revision of quantifying polarization: Current algorithms do not scale,</title>
        <p>do not take temporal signals into account, and do not take graph modularity
due to factors like language into account.</p>
        <p>
          RT = Endorsement? Quantifying polarization studies assume that social
connections like retweets and follows are endorsements without justi cation.
However, this assumption often falls apart in real-world applications. Many users
even explicitly state that retweets are not endorsements on their pro les. In
some cases, endorsement occurs without a like: videos involving #BLM (Black
Lives Matter) protests and police intervention on Facebook were not liked but
shared [
          <xref ref-type="bibr" rid="ref20">20</xref>
          ]. In Twitter terms, this would mean that newsworthy posts with a
negative sentiment are not liked but retweeted, which breaks this assumption.
A survey among 316 users revealed that only 68% of people endorse what they
retweet, and 73% of users agree with what they retweet [
          <xref ref-type="bibr" rid="ref16">16</xref>
          ]. Thus, coming up
with better connection models is needed.
        </p>
      </sec>
      <sec id="sec-3-4">
        <title>Revision of identi cation of a topic: Hashtags do not capture all the</title>
        <p>discussion on a topic and focus attention on already polarized users, thus creating
biased results. Therefore, the methodology to model a topic should be revised.</p>
        <p>Back ring e ect: Anti-polarization tools assume that views will be
moderated when a user is connected to users of opposite views by the implicit
assumption that views will be averaged, ignoring the possibly back re e ect.</p>
        <p>Universal Interest: There is an underlying assumption that an unbiased
user has a medium opinion. Most works do not consider that a user may have
no opinion on a topic.</p>
        <p>
          Lurkers Matter: The observations from Twitter analysis are based only
on the audience that actively reacts. However, Facebook users were found to
underestimate their audience by 27% [
          <xref ref-type="bibr" rid="ref3">3</xref>
          ]. We expect a more dramatic result on
Twitter since most pro les are public and timelines are created based on more
than simple follow relationships. Thus, future work is needed to verify these
results.
        </p>
      </sec>
      <sec id="sec-3-5">
        <title>Not all views should be moderated:. In fact it can be harmful in some</title>
        <p>cases to encourage users towards some position. For example, encouraging
normal users to read anti-vaccination content could be detrimental to public health.</p>
      </sec>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          1.
          <article-title>6 may s 2019 imamoglu lehine tweet atan unluler</article-title>
          . https://eksisozluk.com/6-mayis2019
          <string-name>
            <surname>-</surname>
          </string-name>
          imamoglu
          <article-title>-lehine-tweet-atan-unluler{6030169?a=nice</article-title>
          , accessed:
          <fpage>2019</fpage>
          -05-06
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          2.
          <string-name>
            <surname>Alatas</surname>
            ,
            <given-names>V.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Chandrasekhar</surname>
            ,
            <given-names>A.G.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Mobius</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Olken</surname>
            ,
            <given-names>B.A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Paladines</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          :
          <article-title>When celebrities speak: A nationwide twitter experiment promoting vaccination in indonesia</article-title>
          .
          <source>Tech. rep., National Bureau of Economic Research</source>
          (
          <year>2019</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          3.
          <string-name>
            <surname>Bernstein</surname>
            ,
            <given-names>M.S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Bakshy</surname>
            ,
            <given-names>E.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Burke</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Karrer</surname>
            ,
            <given-names>B.</given-names>
          </string-name>
          :
          <article-title>Quantifying the invisible audience in social networks</article-title>
          .
          <source>In: Proceedings of the SIGCHI conference on human factors in computing systems</source>
          . pp.
          <volume>21</volume>
          {
          <fpage>30</fpage>
          .
          <string-name>
            <surname>ACM</surname>
          </string-name>
          (
          <year>2013</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          4. DiMaggio,
          <string-name>
            <given-names>P.</given-names>
            ,
            <surname>Evans</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J.</given-names>
            ,
            <surname>Bryson</surname>
          </string-name>
          ,
          <string-name>
            <surname>B.</surname>
          </string-name>
          :
          <article-title>Have american's social attitudes become more polarized?</article-title>
          <source>American journal of Sociology</source>
          <volume>102</volume>
          (
          <issue>3</issue>
          ),
          <volume>690</volume>
          {
          <fpage>755</fpage>
          (
          <year>1996</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          5.
          <string-name>
            <surname>Fagin</surname>
            ,
            <given-names>R.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Lotem</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Naor</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          :
          <article-title>Optimal aggregation algorithms for middleware</article-title>
          .
          <source>Journal of computer and system sciences 66(4)</source>
          ,
          <volume>614</volume>
          {
          <fpage>656</fpage>
          (
          <year>2003</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          6.
          <string-name>
            <surname>Gao</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Do</surname>
            ,
            <given-names>H.J.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Fu</surname>
          </string-name>
          , W.T.:
          <article-title>Burst your bubble! an intelligent system for improving awareness of diverse social opinions</article-title>
          .
          <source>In: 23rd International Conference on Intelligent User Interfaces</source>
          . pp.
          <volume>371</volume>
          {
          <fpage>383</fpage>
          .
          <string-name>
            <surname>ACM</surname>
          </string-name>
          (
          <year>2018</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          7.
          <string-name>
            <surname>Garimella</surname>
            ,
            <given-names>K.</given-names>
          </string-name>
          , De Francisc iMorales,
          <string-name>
            <given-names>G.</given-names>
            ,
            <surname>Gionis</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            ,
            <surname>Mathioudakis</surname>
          </string-name>
          ,
          <string-name>
            <surname>M.</surname>
          </string-name>
          :
          <article-title>Mary, mary, quite contrary: Exposing twitter users to contrarian news</article-title>
          .
          <source>In: Proceedings of the 26th International Conference on World Wide Web Companion</source>
          . pp.
          <volume>201</volume>
          {
          <fpage>205</fpage>
          .
          <string-name>
            <surname>International World Wide Web Conferences Steering Committee</surname>
          </string-name>
          (
          <year>2017</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          8.
          <string-name>
            <surname>Garimella</surname>
            ,
            <given-names>K.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>De Francisci Morales</surname>
            ,
            <given-names>G.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Gionis</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Mathioudakis</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          :
          <article-title>Reducing controversy by connecting opposing views</article-title>
          .
          <source>In: Proceedings of the Tenth ACM International Conference on Web Search and Data Mining</source>
          . pp.
          <volume>81</volume>
          {
          <fpage>90</fpage>
          .
          <string-name>
            <surname>ACM</surname>
          </string-name>
          (
          <year>2017</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          9.
          <string-name>
            <surname>Garimella</surname>
            ,
            <given-names>K.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Morales</surname>
            ,
            <given-names>G.D.F.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Gionis</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Mathioudakis</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          :
          <article-title>Quantifying controversy on social media</article-title>
          .
          <source>ACM Transactions on Social Computing</source>
          <volume>1</volume>
          (
          <issue>1</issue>
          ),
          <volume>3</volume>
          (
          <year>2018</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          10.
          <string-name>
            <surname>Graells-Garrido</surname>
            ,
            <given-names>E.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Lalmas</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Quercia</surname>
            ,
            <given-names>D.</given-names>
          </string-name>
          :
          <article-title>People of opposing views can share common interests</article-title>
          .
          <source>In: Proceedings of the 23rd International Conference on World Wide Web</source>
          . pp.
          <volume>281</volume>
          {
          <issue>282</issue>
          (
          <year>2014</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref11">
        <mixed-citation>
          11.
          <string-name>
            <surname>Gumrukcu</surname>
            ,
            <given-names>T.</given-names>
          </string-name>
          :
          <article-title>Turkish artists stand by istanbul's ousted mayor</article-title>
          .
          <source>Reuters</source>
          (
          <year>2019</year>
          ), https://www.reuters.com/article/us-turkey
          <article-title>-election-celebrities/turkish-artistsstand-by-istanbuls-ousted-mayor-idUSKCN1SG1S7</article-title>
        </mixed-citation>
      </ref>
      <ref id="ref12">
        <mixed-citation>
          12.
          <string-name>
            <surname>Hayat</surname>
            ,
            <given-names>T.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Galily</surname>
            ,
            <given-names>Y.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Samuel-Azran</surname>
            ,
            <given-names>T.</given-names>
          </string-name>
          :
          <article-title>Can celebrity athletes burst the echo chamber bubble? the case of lebron james and lady gaga</article-title>
          .
          <source>International Review for the Sociology of Sport</source>
          p.
          <volume>1012690219855913</volume>
          (
          <year>2019</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref13">
        <mixed-citation>
          13.
          <string-name>
            <surname>Lex</surname>
            ,
            <given-names>E.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Wagner</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Kowald</surname>
            ,
            <given-names>D.</given-names>
          </string-name>
          :
          <article-title>Mitigating con rmation bias on twitter by recommending opposing views</article-title>
          . arXiv preprint arXiv:
          <year>1809</year>
          .
          <volume>03901</volume>
          (
          <year>2018</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref14">
        <mixed-citation>
          14.
          <string-name>
            <surname>Matakos</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Terzi</surname>
            ,
            <given-names>E.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Tsaparas</surname>
            ,
            <given-names>P.</given-names>
          </string-name>
          :
          <article-title>Measuring and moderating opinion polarization in social networks</article-title>
          .
          <source>Data Mining and Knowledge Discovery</source>
          <volume>31</volume>
          (
          <issue>5</issue>
          ),
          <volume>1480</volume>
          {
          <fpage>1505</fpage>
          (
          <year>2017</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref15">
        <mixed-citation>
          15.
          <string-name>
            <surname>Messing</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Westwood</surname>
            ,
            <given-names>S.J.:</given-names>
          </string-name>
          <article-title>Selective exposure in the age of social media</article-title>
          .
          <source>Communication Research</source>
          <volume>41</volume>
          ,
          <volume>1042</volume>
          {
          <fpage>1063</fpage>
          (
          <year>2014</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref16">
        <mixed-citation>
          16.
          <string-name>
            <surname>Metaxas</surname>
            ,
            <given-names>P.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Mustafaraj</surname>
            ,
            <given-names>E.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Wong</surname>
            ,
            <given-names>K.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Zeng</surname>
            ,
            <given-names>L.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>O'Keefe</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Finn</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          :
          <article-title>What do retweets indicate? results from user survey and meta-review of research</article-title>
          . In: Ninth
          <source>International AAAI Conference on Web and Social Media</source>
          (
          <year>2015</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref17">
        <mixed-citation>
          17.
          <string-name>
            <surname>Pariser</surname>
            ,
            <given-names>E.</given-names>
          </string-name>
          :
          <article-title>The lter bubble: What the Internet is hiding from you</article-title>
          .
          <source>Penguin UK</source>
          (
          <year>2011</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref18">
        <mixed-citation>
          18.
          <string-name>
            <surname>Sunstein</surname>
            ,
            <given-names>C.R.</given-names>
          </string-name>
          : Republic. com. Princeton university press (
          <year>2001</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref19">
        <mixed-citation>
          19.
          <string-name>
            <surname>Tufekci</surname>
            ,
            <given-names>Z.</given-names>
          </string-name>
          :
          <article-title>Big questions for social media big data: Representativeness, validity and other methodological pitfalls</article-title>
          . In: Eighth International AAAI Conference on Weblogs and Social
          <string-name>
            <surname>Media</surname>
          </string-name>
          (
          <year>2014</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref20">
        <mixed-citation>
          20.
          <string-name>
            <surname>Tufekci</surname>
            ,
            <given-names>Z.</given-names>
          </string-name>
          :
          <article-title>Twitter and tear gas: The power and fragility of networked protest</article-title>
          . Yale University Press (
          <year>2017</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref21">
        <mixed-citation>
          21.
          <string-name>
            <surname>Vydiswaran</surname>
            ,
            <given-names>V.V.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Zhai</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Roth</surname>
            ,
            <given-names>D.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Pirolli</surname>
            ,
            <given-names>P.</given-names>
          </string-name>
          :
          <article-title>Overcoming bias to learn about controversial topics</article-title>
          .
          <source>Journal of the Association for Information Science and Technology</source>
          <volume>66</volume>
          (
          <issue>8</issue>
          ),
          <volume>1655</volume>
          {
          <fpage>1672</fpage>
          (
          <year>2015</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref22">
        <mixed-citation>
          22.
          <string-name>
            <surname>Watts</surname>
            ,
            <given-names>D.J.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Strogatz</surname>
            ,
            <given-names>S.H.</given-names>
          </string-name>
          :
          <article-title>Collective dynamics of 'small-world' networks</article-title>
          .
          <source>nature</source>
          <volume>393</volume>
          (
          <issue>6684</issue>
          ),
          <volume>440</volume>
          (
          <year>1998</year>
          )
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>