Two of the newest features of Wikidata are support for lexicographic data (lexemes), and support for Shape Expressions (ShEx). We demonstrate the rst application of ShEx for validation of entity data for Wikidata lexemes. Validation of entity data in Wikidata against ShEx schemas allows editors to discover missing or incorrect information. It may also form a basis for discussion of the data models implicitly used in Wikidata. We present a use case and benchmark for ShEx and discuss its current limitations.
Since 2018, it has been possible to represent lexeme data with associated
information about forms and senses in Wikidata [
ShEx (Shape Expressions) is a concise, formal language for modeling and
validating RDF graphs [
Here we describe our initial experiences with using ShEx for validation of linguistic data on Wikidata, focusing on Danish Wikidata lexemes. This is a convenient method for a user to quickly gain an overview of the current status of the data with regard to a speci c schema, entirely using tools in the Wikidata ecosystem. Apart from discovering missing or incorrect data, we identify cases where ShEx pinpoints issues for discussion about a `data model' for the Danish lexicographic data, as well as cases where ShEx validation is di cult. 2
Numerous mechanisms and tools are available that allow Wikidata users to
validate entered lexeme data [
Scholia, a SPARQL-based web application and Wikidata frontend focusing
on scholarly data [
Ordia is a web application for Wikidata lexemes [
In addition to [
This is the rst application of ShEx validation to the lexeme namespace in Wikidata. Validation using ShEx provides more comprehensive validation than has previously been possible with Wikidata tools. Datatype restrictions and literal value restrictions can be expressed in ShEx and identi ed alongside property 5 https://www.wikidata.org/wiki/Wikidata:Lexicographical_data/Ideas_of_ queries. constraints for multiple properties at the same time. This o ers more complete information, by allowing users to test situations that previously could only be approached piece by piece according to the above rules. The constraint system is useful for communicating expectations and potential issues to users on a perproperty basis. Validating entity data against a schema describing a data shape can span multiple properties. 3
We wrote ShEx schemas for Danish lexemes with the identi ers E15 (Danish lexeme), E34 (Danish noun), E54 (lexeme), E56 (Danish verb), E62 (Danish pronoun) and E65 (Danish numeral) as well as a ShEx for Danish hyphenation E68.6
Many rules exist for Danish words and grammar, see, e.g., [
For the rst rule, related to DanNet identi ers, Wikidata has the ability to
explicitly state `no value' and ShEx can test for the presence of this `no value'
in the Wikidata property for DanNet (P6140) with \a [ wdno:P6140 ]". In
the cases where the DanNet identi er is present with a single value, we can
test its format with a regular expression. Current DanNet identi ers conform
to a format with 8 digits. The combined ShEx constraint for Wikidata DanNet
statements then reads a [ wdno:P6140 ] | ps:P6140 /^[
The second rule for two grammatical genders is represented by the following ShEx schema: wdt:P5185 [ wd:Q1305037 wd:Q1775461 ]. Danish does not reveal grammatical gender in plural, so plurale tantum (word with only plural forms) and its derived lexemes have no obvious gender and no gender is speci ed in Wikidata. The same lack of obvious gender occurs for a word such as a druk 6 The Wikidata page with editing capabilities are, e.g., https://www.wikidata.org/ wiki/EntitySchema:E15 while the URL for ShEx import is https://www.wikidata. org/wiki/Special:EntitySchemaText/E68 for E15 and for proper nouns. Either we need to de ne the gender or modify the ShEx schema to accommodate the exceptions. We also note that a few nouns have multiple genders, and these nouns also require exceptions.
For the third rule, we note that a Danish hyphenation rule can apply across most, if not all, word classes, thus we can write a general schema for hyphenation and use ShEx's IMPORT feature to include shapes of the schema as part of another schema. The essential part in the current Danish hyphenation schema (E68) is: \a [ wdno:P5279 ] | ps:P5279 /.*[aeiouy a] .*\u2027.*[aeiouy a]. */ ;". Here wdno:P5279 catches Wikidata's `no value' for forms without possible hyphenation. The regular expression uses the hyphenation point unicode character. It could be further re ned and extended to accommodate accented vowels and words with multiple possible hyphenation points.
The fourth rule, related to the gender of compounds, requires a more elaborate ShEx schema. Our current implementation enumerates over grammatical gender and over the number of compound parts as well as creates exceptions for the -fuld su x and words that are not compounds. The result is a verbose ShEx.
A ShEx schema is submitted to the ShEx2 Simple Online Validator7 with a list of Wikidata items to be tested, | most conveniently found by a SPARQL query to WDQS. The conformance reports produced in the ShEx validation process provide detailed feedback about which statements on which items have issues. Testing a subgraph for conformance to the ShEx schema, users can survey multiple items in a single validation session. This allows users to address issues across a set of items rather than reviewing them individually. By reading these conformance reports we discovered numerous issues. Most of the non-conformant items we discover are errors of omission, rather than errors of commission. In the former case, e.g., nouns missing grammatical gender, while in the latter case, e.g., nouns with a wrong gender. We corrected a number of the erroneous entries and added many new statements. 4
ShEx conformance reports can be used as a basis for concrete discussions among editors about approaches for improving the data. Communicating via a schema language allows editors to be explicit about their expectations and intentions.
We note that some of the rules may be contested by Wikidata users. For instance, there has been discussion on which character should indicate hyphenation and how Wikidata should indicate that a word form cannot be hyphenated.8
For people preparing to write schemas for use in the Wikidata context, we o er the following practical recommendations. While writing schemas is an investment of e ort, the ability to quickly gain an overview of a subset of the Wikidata graph is useful. It may be possible to reuse shapes from existing schemas, 7 https://tools.wmflabs.org/shex-simple/wikidata/packages/shex-webapp/ doc/shex-simple.html. 8 The hyphenation discussions have taken place on the so-called talk page of the property on Wikidata: https://www.wikidata.org/wiki/Property_talk:P5279. thus it may be helpful to explore existing schemas. Reading existing schemas is also a helpful way to get a sense of how others have expressed data models that could serve as examples.
The opportunity to provide a link to a schema in the ShEx namespace of Wikidata supports collaborative re nement of data models, such as those for Danish lexemes. Thus, if some of the rules discussed in this paper require renement, editors of Wikidata will be able to point to schemas, to query maps indicating relevant sub-graphs of Wikidata, and to individual lexeme items which may need to be revised.9