How are users exposed to Wikipedia results, in return to their web searches? Where are such results positioned on the screen? In this study, we experimentally measure the ranking of Wikipedia pages on Google Italia.
As observed by a recent article of Nature News [
In this study, we measure the ranking of Wikipedia pages on Google Italia. To the best of our knowledge, it is the rst study of this kind for the Italian language. The procedure is as follows. For our web searches, we concentrate on Italian keywords (and set of keywords). To collect the most popular search keywords, they have been chosen from Google Trend,from the Google Display Planner5, and from the Italian trending words on Twitter. Google Trend gives the most searched terms in a year, as well as trending searches in the past 24 hours6 and trending searches in the recent past7. Google Display Planner is a ? Work partially funded by the Registro.it project MIB (My Information Bubble) 4 http://www.alexa.com/topsites (7th March, 2016) 5 https://support.google.com/adwords/answer/3056115?hl=en 6 https://www.google.com/trends/home/all/IT 7 https://www.google.com/trends/hottrends#pn=p27 tool providing a series of websites linked to speci c categories. It has also been used, to look for suggested keywords tied to particular categories, like, e.g., Sport and Vehicles. We have performed all the searches on Google Italia, from Italy. To avoid personalised results, we have used newly created browser instances and we have simulated users not logged into Google. The default settings for searches were the Google default settings. Among the obtained results, we considered only organic results and not sponsored one (i.e., Advertisements, Google News, and so on). In the following, we present the experiments and the results. 2
Experiment settings. Our reference date is April 7th, 2016. We have collected the top search terms on Google Italia from 2011 to 2015. Also, we have extracted the trending news for the reference date and the hot trends stories of the ten days prior to that date. As a whole, we obtained 1169 unique search terms. For a wider view, we have further gathered the top Italian trending keywords from Twitter (updated three times and within three hours, on April 7th, 2016), leading to 40 unique terms from Twitter.
For the experiments, we have extended AdFisher[
For each keyword (or keywords set), a new browser instance has been launched
and we have searched such keyword(s). The browser instance has been destroyed
after saving the query results. New browser instances have been used to avoid
the so called carry-over e ects [
Examples of keywords that we have searched for are \Elezioni presidenziali negli Stati Uniti d'America del 2016" \Credito Valtellinese", \Una lama di luce". The complete list of keywords (and keywords set) is at https://goo.gl/9KasJc. Results. Experiments were performed with more than 1,200 keywords, spanning 33 categories, with an average of 21 keywords per category. In order to evaluate the search results, we have considered only those keywords for which the corresponding Wikipedia link appeared in the rst page of the result list this corresponds to 708 keywords. Searching for those keywords, we found that Wikipedia pages in the result lists are ranked 1st 41.1% and 2nd 19.9%. Overall, they appear in the rst ve positions 78.8 times over 100. Figure 1 shows the occurrence of Wikipedia pages according to their position.
Searching keywords belonging to \Topic Emergenti" and \Mostre d'Arte" always yield Wikipedia links as the rst result. Searching keywords belonging to \Assicurazioni", \O erte di Lavoro" and \Economia e Finanza" yield Wikipedia links as rst result for less than 10% of such keywords, see Figure 2. 8 https://github.com/tadatitam/info-flow-experiments Amongst all, \Mostre d'Arte" is a top category: all the keywords belonging to that category always lead to results with a Wikipedia link located within the rst three Google Italia results. Examples of keywords are \Picasso - Milano", \Renoir - Pavia" and \Leonardo - Venaria". \Scienze" and \Benessere" are top categories too. More than 50% of searches of related keywords lead to Wikipedia results in the top three positions, see Figure 3. Instead, only 2% of keywords in the \News" category lead to results in the top three positions, while results for keywords related to \Politica", \Economia e Finanza", and \Hobbies" are ranked in the top three positions less than 10% of times (Figure 3). Figure 2 and Figure 3 show only the categories with the largest number of keywords.
Work in [
This preliminary study measures the position of Wikipedia links, resulting from searching set of keywords over Google Italia. The outcome provides evidence that Wikipedia is dominant on Google Italia search results ranking: more than 78% of the times, there is one Wikipedia page within the rst ve search results. A closer look shows that keywords related to \Mostre d'Arte" category always have an associated Wikipedia page in the top three results, while those related to \News" are less than 10%. We have focused on quantifying the phenomenon, without investigating the semantics motivation behind the di erence rankings for categories. We leave this study as a future work.