Wikidata is steadily becoming more central to Wikipedia, not just in maintaining interlanguage links, but in automated population of content within the articles themselves. It is not well understood, however, how widespread this transclusion of Wikidata content is within Wikipedia. This work presents a taxonomy of Wikidata transclusion from the perspective of its potential impact on readers and an associated indepth analysis of Wikidata transclusion within English Wikipedia. It nds that Wikidata transclusion that impacts the content of Wikipedia articles happens at a much lower rate (5%) than previous statistics had suggested (61%). Recommendations are made for how to adjust current tracking mechanisms of Wikidata transclusion to better support metrics and patrollers in their evaluation of Wikidata transclusion.
into the nature of Wikidata transclusion. Speci cally, the table serves to support the Mediawiki software in updating its cache of rendered Wikipedia articles when updates are made to transcluded Wikidata items. In many cases, however, the data is very coarse and gives little indication as to how the Wikidata content appears in the Wikipedia article. Anecdotally, Portuguese Wikipedia (ptwiki) has a featured list article3 that is generated purely from Wikidata content using a template4 and many languages make extensive use of Wikidata-powered infoboxes whereas, by consensus,5 transclusion of Wikidata into infoboxes and article content in English Wikipedia (enwiki) is quite restricted.
The goal of this research is to contextualize the coarse Wikidata transclusion numbers so as to better understand the diversity of ways in which Wikidata is being used within Wikipedia and how these di erent ways might impact Wikipedia readers. Informed by design decisions and community consensus, a taxonomy of Wikidata transclusion is developed and applied to English Wikipedia via a mixture of qualitative coding and quantitative analyses. Speci cally, this research makes the following contributions: { Taxonomy of transclusion: a framework is provided to categorize di erent types of Wikidata transclusion based upon their perceived potential impact on readers. { Analysis of transclusion on English Wikipedia: it is determined that the vast majority of the 61% of articles with evidence of Wikidata transclusion on English Wikipedia can be classi ed as low-importance and only 5% of articles on English Wikipedia have medium- or high-importance Wikidata transclusion. 2
Relatively little research has focused on how Wikidata is transcluded within
Wikipedia, with most of the focus of studies of Wikidata (outside of the large
body of work building tools with Wikidata) either being on Wikidata as an
isolated entity|e.g., detecting vandalism on Wikidata [
ADticos na ditadura militar brasileira 4 https://en.wikipedia.org/wiki/Template:Wikidata list 5 https://en.wikipedia.org/wiki/Wikipedia:Requests for comment/Wikidata Phase 2
Wikidata transclusion is tracked via the wbc entity usage database table6, the intention of which is to keep the cache of parsed Wikipedia articles up-to-date. It thus tracks which Wikidata items, and which aspects of these items, are transcluded in Wikipedia articles. It is based on what Wikidata content is requested when rendering a Wikipedia article, however, which is a superset of what Wikidata content actually is transcluded (and displayed) in an article.
There are a variety of mechanisms through which Wikidata transclusion happens within Wikipedia articles such as templates that depend on Lua modules or more speci c parser calls.7 While the end result should not depend on the method employed, the exact method used does determine how the transclusion is recorded in the wbc entity usage table. In particular, whereas the speci c parser calls often allow the table to record the speci c statements used within an article, the more general Lua modules often trigger a much broader usage aspect (\some statements") that obscures which particular properties are being transcluded.
Calculating which articles have some form of Wikidata transclusion through the wbc entity usage table, then, is a very coarse measure of what Wikidiata content is actually included in a Wikipedia article. Improving the insights that can be gleaned from this table would require substantial reengineering of how the table works. This research, however, does provide some insight into how to interpret the data in this table as well as o er an alternative, richer, though less automatic, framework for evaluating Wikidata transclusion within Wikipedia. 3 3.1
Not all forms of Wikidata transclusion have the same potential impact. In this
research, impact was de ned as how strongly it is believed a reader could be
a ected if the content that was transcluded was incorrect. There are many
alternative de nitions|e.g., impact on editors' backlog to maintain Wikipedia|that
would undoubtedly lead to di erent classi cations. This de nition was chosen
as it informs how important it is for patrollers to be aware of the transclusion
and track changes to the content [
Instances of Wikidata transclusion was classi ed as either high, medium, or
low importance|i.e. impact|depending on three di erent criteria:
{ Platform: does the transclusion show up on all platforms|i.e. desktop,
mobile web, and Android/iOS apps|or only in certain form factors?
Transclusion that shows up on more platforms has a higher potential for impact
because more readers can see it. English Wikipedia (like most other language
6 https://www.mediawiki.org/wiki/Wikibase/Schema/wbc entity usage
7 https://en.wikipedia.org/wiki/Wikipedia:Wikidata#Inserting Wikidata values
into Wikipedia articles
editions) is largely split between desktop and mobile web readers with only
a small percentage of pageviews coming from the apps.8
{ Salience: when transclusion appears, where on the page does it appear
and how obvious is it? Content that appears at the top of articles is more
likely to be interacted with by readers versus content that appears at the
end of articles [
All
Low
Low Low Low Low
Ib person Wikidata;
Ib company Medium IMDb title;
Commons category High Default without
shortdesc High Authority control;
Taxonbar High Birth date; Coord
(when hard-coded) Medium USGS gazetteer;
Cite Q 8 https://stats.wikimedia.org/#/en.wikipedia.org/reading/total-page-views/normalj barj2-yearjaccess desktop*mobile-app*mobile-webjmonthly 9 https://en.wikipedia.org/wiki/Wikipedia:Categorization#Wikipedia administrative categories
One hundred random articles on English Wikipedia were chosen and evaluated to determine what, if any, templates were being recorded as transcluding Wikidata content in the article and categorized into the transclusion categories (low-, medium-, high-importance) described above.10 An article was categorized with its highest level of transclusion|e.g., an article with a low-importance transclusion and high-importance transclusion would be coded as high-importance transclusion. Only a single researcher evaluated the data given the time-intensive nature of the coding. While there is in theory only a single correct answer for what templates are triggering Wikidata transclusion, in practice it can be di cult to trace back the calls made by templates to determine whether they might be causing the transclusion listed within the wbc entity usage table. The articles were evaluated in their current form on 31 July 2020. 3.3
The nal step was to verify the representativeness of the sample of articles coded and prototype an approach to continuing to track Wikidata transclusion without having to re-code a new sample of articles. Speci cally, for the templates that were identi ed as major sources of Wikidata transclusion, a Python script was developed that would detect usage of these templates in wikitext dumps11 and determine whether the template was actually transcluding content or just generating tracking categories based on a series of heuristics.12 This included two common metadata templates (Template:Authority control and Template:Taxonbar), all templates that generate external links from Wikidata13 as they follow a very similar pattern, and the templates for inserting latitude/longitude coordinates (Template:Coord) and birth date (Template:Birth date and age) as very common Wikidata templates. Redirects to these templates were also included. Infobox templates were not included in this script as there are approximately 100 with varying but low usage14 and each template has its own speci c parameters that control Wikidata transclusion. The 1 August 2020 wikitext dumps were used for this evaluation.15 10 The data from the wbc entity usage table can be easily viewed from within the article in the \Page information" tab under \Wikidata entities used in this page". 11 https://github.com/geohci/wikidata-transclusion 12 Alternative strategies such as selectively ablating templates from a page in a test environment to see how content and tracking changes would be excessively complex because most templates are written to use the Wikidata item of the article containing them and so cannot be evaluated outside of the context of the article they are in. 13 https://en.wikipedia.org/wiki/Category:External link templates using Wikidata 14 https://en.wikipedia.org/wiki/Category:Infobox templates using Wikidata 15 https://dumps.wikimedia.org/enwiki/20200801/enwiki-20200801-pages-articles.
xml.bz2
As a baseline, the wbc entity usage table indicates as of August 2020 that 61% of articles on English Wikipedia include some form of Wikidata transclusion.16 In line with this statistic, from the 100 article sample selected to be evaluated (full data in Table 2 in the Appendix), 59 had some form of Wikidata transclusion recorded in the wbc entity usage table.
For those 59 articles with some evidence of Wikidata transclusion, however, only 5 had medium- or high-importance Wikidata transclusion on further inspection. Speci cally, 2 articles had Wikidata-powered infobox templates, 3 articles had Wikidata-powered external link templates, 21 articles just had metadata templates, and 33 articles just had Wikidata tracking categories. Overlapping with these categories, 54 of the 100 articles did not override the Wikidata description for use in article previews in Search on the o cial iOS and Android apps.17 This form of transclusion is not tracked by the wbc entity usage table though.
The quantitative evaluation supported the validity of the transclusion rates seen in the qualitative coding. From all of English Wikipedia (6,125,693 articles), 3% of articles had external link templates that transcluded Wikidata (only 47.5% of the 3,132,833 instances found of external link templates that could transclude Wikidata did while the rest provided hard-coded parameters and thus only served to generate tracking categories), 29% of articles had metadata templates18, and 27% of articles were found to just have tracking categories generated via Wikidata templates. For instance, out of the 1,841,505 times in which the coordinates template (displays latitude-longitude coordinates in an article with a link to a map) appears in English Wikipedia, it only uses Wikidata to provide coordinates 2,730 times (0.1%) and the rest of the template instances have latitude-longitude coordinates hard-coded.19 5 5.1
This analysis indicates that the prior measure of Wikidata transclusion in English Wikipedia (61%) both greatly underestimates and overestimates Wikidata transclusion. Transclusion that a ects the content of articles (high- and mediumimportance) was determined to be 5% of articles, indicating that 61% is far too 16 https://wmdeanalytics.wm abs.org/WD percentUsageDashboard/ 17 https://www.mediawiki.org/wiki/API:Page info in search results 18 This actually tracks closely to the 21% statistic from the qualitative evaluation as that number is for articles that only had metadata templates but an additional 3% of articles were found to have metadata templates and higher-importance templates. 19 This is likely the result of English Wikidata determining that existing information should not be overwritten with Wikidata data: https://en.wikipedia.org/wiki/ Wikipedia:Requests for comment/Wikidata Phase 2 high of a measure for these types. On the other hand, usage of the Wikidata description in the mobile apps has not been previously tracked and so this 61% number does exclude some articles that are indeed using Wikidata descriptions within the context of the Wikipedia apps.
Metrics that track how well-incorporated Wikidata is into Wikipedia articles will need more nuanced approaches that capture these distinctions. Two xes would make a huge impact. First, if standard statements|e.g., date of birth| were tracked separately from identi ers|e.g., VIAF, GNIS IDs|this would separate out metadata templates. This analysis found that metadata templates account for much of Wikidata transclusion. They speak to Wikidata's value as a linked data repository but are a very di erent use-case than transclusion of content like date of birth or coordinates. Secondly, transclusion that merely generates tracking categories|e.g., Category:Date of birth not in Wikidata|could be tracked separately from transclusion that inserts content into the page.
Together, these two changes would greatly improve the ease with which the di erent importance-levels of Wikidata transclusion could be tracked without indepth qualitative coding wiki by wiki. This would both support more nuanced metrics of Wikidata transclusion|e.g., for tracking progress|but also support much more nuanced lters for patrollers of changes to Wikidata that might affect articles on their watchlist. Currently, patrollers of vandalism can choose to see or not see Wikidata transclusion,20 but do not have good tools for distinguishing from transclusion that might have a sizeable impact on an article and therefore warrant checking and changes to Wikidata items that would have no or only a very minimal impact on an article. As these analyses demonstrated, much of Wikidata transclusion could reasonably be considered \noise" from a patrolling perspective, especially because the metadata templates are tracked via wbc entity usage in such a way that any change to any part of the article's associated Wikidata item shows up in recent changes (even if it has no impact on the article whatsoever). 5.2
English Wikipedia was analyzed in this work as the most accessible Wikipedia for qualitative coding for the researcher. While there are both many wikis with more and with less evidence of transclusion, English Wikipedia does have some very strong policies with regards to Wikidata transclusion that are not necessarily found in other language editions. Further analyses of Wikidata transclusion in other languages would undoubtedly discover di erent distributions of usage and new types of templates that have not shown up on English Wikipedia.
A di erent approach to the taxonomy would also change the results. For
instance, this taxonomy does not take into account any measure of the importance
of the article that is transcluding the Wikidata content|e.g., number of views
the article gets [
Ib Anatomy 79 McAfee Peak 80 Sommatino Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y
Taxonbar Taxonbar
USGS gazetteer
Low Auth. Ctrl. Ib Italian High
Comune Taxonbar Low None None Low Low Low Low Low None None