=Paper=
{{Paper
|id=Vol-3643/paper7
|storemode=property
|title=Evaluation of Expressing without asserting approaches in RDF. The case of
Conjectures
|pdfUrl=https://ceur-ws.org/Vol-3643/paper7.pdf
|volume=Vol-3643
|authors=Valentina Pasqual,Gerald Manzano,Eduart Uzeir,Francesca Tomasi,Fabio Vitali
|dblpUrl=https://dblp.org/rec/conf/ircdl/PasqualMUTV24
}}
==Evaluation of Expressing without asserting approaches in RDF. The case of
Conjectures==
https://ceur-ws.org/Vol-3643/paper7.pdf
Evaluation of Expressing without asserting
approaches in RDF. The case of Conjectures.
Valentina Pasqual1 , Gerald Manzano2 , Eduart Uzeir2 , Francesca Tomasi1 and
Fabio Vitali2
1
Digital Humanities Advanced Research Center, Department of Classica Philology and Italian studies, University of
Bologna
2
Department of Computer Science, University of Bologna
Abstract
In this paper, we evaluate the existing reification approaches for expressing without asserting (EWA)
statements in RDF along with their related contextual information and compare them with a new method,
called Conjectures. Conjectures express RDF statements with three states of knowledge: undisputed
claims, disputed claims, and settled claims. Conjectures extend the semantics of RDF Named graphs and
introduce a new syntactical form to represent both conjectural and asserted information. Our evaluation
tests were performed on a large sample of Wikidata entities about artworks interspersed with additional
dummy statements to simulate alternative or abandoned claims and enrich the set of non-asserted
claims. Our study evaluates metrics such as the total number of triples, loading time, dataset weight,
and in particular query execution time for many different and meaningful types of queries. Results
show that Conjectures is competitive with existing methods and outperforms other methods in terms of
efficiency when retrieving debated statements, thereby demonstrating its potential as an effective tool
for expressing nuanced RDF statements.
Keywords
RDF Reification, Efficiency assessment, Conjectures, Expressing Without Asserting,
1. Introduction
RDF [1, 2] is a powerful tool for expressing statements as absolute, asserted relationships between
entities. Yet, it falls short when it comes to representing nuances about such relationships,
e.g., to enrich them with contextual information, or to represent in full how they relate to
each other. While some statements may never need to be questioned (or maybe there is no
interest in questioning them), and therefore can be represented adequately with plain RDF
triples, frequently on the other hand scientific and critical discourse is filled with concurrent
opinions and interpretations. This knowledge cannot be simply expressed as plain RDF triples
but requires the specification of contextual information (e.g. who is the author of such claims),
and possibly of multiple and competing statements, which the community may settle or discard
over time.
Woodstock’21: Symposium on the irreproducible science, June 07–11, 2021, Woodstock, NY
Envelope-Open valentina.pasqual2@unibo.it (V. Pasqual); gerald.manzano@studio.unibo.it (G. Manzano);
eduart.uzeir@studio.unibo.it (E. Uzeir); francesca.tomasi@unibo.it (F. Tomasi); fabio.vitali@unibo.it (F. Vitali)
GLOBE https://valentinapasqual.github.io/ (V. Pasqual)
Orcid 0000-0001-5931-5187 (V. Pasqual); 0000-0002-6631-8607 (F. Tomasi); 0000-0002-7562-5203 (F. Vitali)
© 2021 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).
CEUR
Workshop
Proceedings
http://ceur-ws.org
ISSN 1613-0073
CEUR Workshop Proceedings (CEUR-WS.org)
CEUR
ceur-ws.org
Workshop ISSN 1613-0073
Proceedings
This is for instance the case of the hypotheses made by scholars about the many (still
undetermined) locations of the Mona Lisa painting when it was stolen in 1911 [3], or of the
attribution of the painting Salvator Mundi, possibly by Leonardo da Vinci and world-famous
for having been the most expensive painting ever sold at public auction. Its attribution is now
under discussion by many scholars [4]. Our guiding example in this paper, on the other hand,
is the debated attributions of the painting Napoleon crossing the Alps 1 , currently attributed to
Jacques-Louis David, but previously attributed to Jérôme-Martin Langlois and the workshop of
Jacques-Louis David.
RDF Reification2 methods [5] have been used to make statements about statements in RDF
[6] and even allowing the coexistence of multiple opinions in RDF. For instance, reification
enables to express sentences like ”JocondeLab3 reports that Napoleon crossing the alps author is
Jacques-Louis David”. Usually, reification methods typically involve the use of extra triples to
provide information about the original triple being reified. Listing 1 exemplifies the statement
above presented using traditional reification. In the example, a new entity is introduced as an
instance of the class rdf:Statement to introduce ”JocondeLab reports” and the original triple
(Napoleon crossing the alps author is Jacques-Louis David) is expressed by the means of three
additional triples whose predicates are strictly rdf:subject, rdf:predicate, rdf:object .
@ p r e f i x r d f : < h t t p : / / www. w3 . o r g / 1 9 9 9 / 0 2 / 2 2 − r d f − s y n t a x − ns #> .
: statement1 rdf : type rdf : Statement ;
r d f : s u b j e c t wd : Q19801150 ; # Napoleon C r o s s i n g t h e A l p s
r d f : p r e d i c a t e wd : P170 ; # c r e a t o r
r d f : o b j e c t wd : Q67215 . # J a c q u e s − L o u i s David
: s t a t e m e n t 1 wdt : P248 wd : Q29633776 . # s t a t e d i n J o c o n d e L a b
Listing 1: Representation of the claims ”XXX states that Napoleon Crossing the Alps” author is
Jacques-Louis David using standard reification
Accepted statements can be therefore characterised by a plain triple to assert the statement,
and the same triple under reification to provide additional information about the claim itself.
This would be the case of the settled attribution of Napoleon crossing the alps currently attributed
to Jacques-Louis David. Statements that represent alternative or historical information would
then be represented with just the claim in reified form and no assertion through a plain RDF
triple. This would be case of the earlier attributions of Napoleon crossing the alps to Jérôme-
Martin Langlois and the workshop of Jacques-Louis David. We call this approach Expressing
Without Asserting (EWA)[7] and we consider it as a powerful tool to express claims with different
degrees of validity and even critical debate.
In addition to analysing the expressivity and effectiveness of the many existing reification
approaches, their efficiency is also an open issue. To the best of our knowledge, no study
has been proposed on the evaluation of the efficiency of EWA mechanisms with ontology-
independent approaches. This paper evaluates existing reification methods for expressing
without asserting in RDF with the related contextual information and compares them with a
new approach, called Conjectures.
The paper is structured as follows: in section 2 we discuss of a number of syntactical
1
http://www.wikidata.org/entity/Q19801150
2
https://www.w3.org/wiki/RdfReification
3
http://www.wikidata.org/entity/Q29633776
Figure 1: Concurring attributions concerning Napoleon crossing the Alps in Wikidata
approaches to EWA and benchmarks adopted in the literature for their evaluation. In section
3 we briefly introduce and compare conjectures as an alternative method to represent EWA
statements in addition to existing proposals such as RDF reification, RDF star, and others. In
section 4 we discuss the dataset and the metrics used for our comparison of the various EWA
approaches, and in section 5 we analyze our findings. In section 6 we discuss the results before
drawing some conclusions in section 7.
2. State of the art
Many Reification methods have been implemented [8, 9, 10, 11, 12, 13] to represent statements
about RDF claims. Reification provides additional triples about such claims enabling querying
and reasoning mechanisms to integrate the following key attributes[12, 14, 15]: provenance:
enables the identification and representation of the source of a fact or statement (i.e. Who
claimed this fact? ), time: we can communicate the fact’s time-related information (i.e. Which
is the most up-to-date claim about this fact? ), location: reveals locational information about an
event (i.e. Which is the location in which this claim is applicable? ), certainty: indicate the level of
confidence that is attributed to a statement (i.e. how confident are we about a certain event? or is
a given statement true? ), versioning: it is helpful to keep track of RDF datasets’ updating history
(e.g. what data version I am using right now? )
The scientific community has suggested the following major reification strategies to be able
to encapsulate all this variety of contextual information about RDF statements.
Among existing Knowledge Graphs, Wikidata provides RDF data with its reification method[11],
and a long list of qualifiers about provenance, time, and place to represent complex and com-
peting information. Additionally, each claim validity is provided by a customised approach to
express without asserting [7] such claims (via ranking mechanism4 ) which truth state cannot
be given for granted. Consider for example the case of concurring attributions of Napoleon
crossing the Alps, as shown in figure 1, is represented in Wikidata with three different claims:
the currently accepted attribution to Jacques-Luois David (marked with a preferred rank and
therefore asserted), the former attribution to the his workshop (marked with a normal rank and
non-asserted) and the former attribution to Jérôme-Martin Langlois (marked with a deprecated
rank and therefore non-asserted). The two former attributions are therefore marked with
contextual information about the claim (e.g. reason for deprecated rank, former attribution) to
express additional information on the claim itself.
To be widely accepted, every data representation method must be both effective and efficient.
While a method’s effectiveness can be formally demonstrated, we must rely on empirical
observations to determine its efficiency. Every new method that is suggested typically includes
performance data as well. This is accomplished by carrying out the necessary tests to see how
the approach performs on performance indicators like the number of triples, query execution time,
query complexity, dataset storage consumption, support by existing tools and implementations and
other pertinent metrics. There is a vast number of academic papers that suggest this kind of
experimental setup to benchmark the aforementioned reification methods[16, 12, 6, 14, 17, 18, 19].
Additionally, all the benchmarks mentioned above rely on the same four components, namely
datasets, queries, triplestores and reification methods.
3. Conjectures
Conjectures is an approach intended to express RDF statements with three main states of
knowledge: undisputed claims, disputed claims or evolving situations, and settled claims.
Conjectures is a RDF 1.1 compliant characterization of Named graphs (the weak form), and an
extension of RDF Named graphs semantics to distinguish plain Named graphs from disputed
(conjectural) and settled (the strong form)5 .
Undisputed claims Non-disputed claims are expressed as asserted Named graphs (and
therefore introduced by the keyword GRAPH ). For example, the main subject of Napoleon Crossing
the Alps has been recognised as Napoleon itself without any doubt as shown in listing2.
Disputed claims Conjectures is a prototypical extension of the syntax of Trig, where the
keyword GRAPH is replaced with CONJECTURE in front of a graph whose contents is expressed but
not asserted and expressing those statement which is disputes or conveys evolving situations.
For example, Napoleon crossing the Alps former attribution to Jérôme-Martin Langlois can be
represented via a Conjecture as in listing 2. Naturally, graphs that are not marked as conjectures
maintain the same (locally-decided) semantics as before, and therefore they may or may not
contribute to the truth value of the entire dataset depending on such choice. A key aspect
is that Conjectures do not use reification, 𝑛-ary relationships or ad hoc classes For example,
4
https://www.Wikidata.org/wiki/Help:Ranking
5
A complete overview of Conjectures semantics is available at https://conjectures.altervista.org//CONJ_
semantics.pdf
the rdf:Statement class, as employed in standard reification and illustrated in Listing 1, and
therefore they are orthogonal to, and fully compatible with, most of the other approaches.
# The p a i n t i n g main s u b j e c t i s Napoleon
GRAPH s : c l a i m 1 {
wd : Q19801150 wdt : P921 wd : Q517
}
# t h e p a i n t i n g ( wd : Q19801150 ) c r e a t o r ( wdt : P170 ) was s a i d t o be Jérôme − M a r t i n L a n g l o i s ( wd : Q672158 )
CONJECTURE s : c l a i m 2 {
wd : Q19801150 wdt : P170 wd : Q672158
}
# t h e p a i n t i n g c r e a t o r ( wdt : P170 ) was s a i d t o be from t h e s c h o o l o f J a c q u e s − L o u i s David
CONJECTURE s : c l a i m 3 {
wd : Q19801150 wdt : P170 _ : b l a n k
}
# A f t e r d e b a t e , t h e p a i n t i n g ( wd : Q19801150 ) c r e a t o r ( wdt : P170 ) i s J a c q u e s − L o u i s David ( wd : Q83155 )
SETTLED s : c l a i m 4 {
wd : Q19801150 wdt : P170 wd : Q83155
}
Listing 2: Representation of the disputed claim about ”Napoleon crossing the Alps” creator with
Conjectures
Settled claims Settled claims (introduced by the keyword SETTLED ) record both the dispute,
as well as its subsequent resolution. This is specifically and intentionally different from a trivial
re-assertion of the disputed claims, in which we do not acknowledge or mention the dispute
at all (the case of GRAPH in 2). To handle settled disputes we introduce Settled Conjectures,
a third type of named graph that is at the same time conjectured and asserted. The collapse
graphs allow us to both represent the conjectural triples (inside the usual conjectural graph)
as well as the same triples but completely asserted (inside the collapse graph). In addition,
the 𝑐𝑜𝑛𝑗 ∶ 𝑠𝑒𝑡𝑡𝑙𝑒𝑠 relation connects the conjecture and its settlement, simplifying the task of
exploring the relationships between disputes and their settlements. The rationale behind Settled
Conjectures is two-fold: on the one hand, to stress the difference between claims that have
not been challenged and claims that emerged as winning among competing and incompatible
hypotheses and on the other to represent the dual nature of settled claims as both conjectures
and assertions. Consider for example ”Napoleon Crossing the Alps” current attribution to
Jacques-Louis David can be represented via a Settled conjecture as in listing2.
4. Testing EWA approaches
The methodology adopted to run the tests outlined in this study can be summarized as follows:
first, a series of datasets were generated, scaled, and converted in RDF to mimic a set of EWA
approaches (see Section 4.1). Next, the hardware and software employed to run our experiments
on the collected data has been setup (see Section 4.2). A set of metrics has been defined to
evaluate the efficiency of EWA approaches (see Section 4.3). Lastly, the outcomes of the tests
are presented in the next section (see Section 5).
4.1. Data Acquisition, scaling and conversion
The dataset on which these experiments have been run is composed as follows and has been
named D3:
• Art: A thematic set of claims about 300k artwork entities in Wikidata (i.e., painting,
manuscripts, books). This corresponds to about 10% of all artwork entities currently
present inside Wikidata.
• Random: after considerable deliberation, we concluded that adding some kind of entropy
to the dataset would make it more representative. This dataset contains the claims of
300k Wikidata random entities.
• Dummy: a selection of dummy statements regarding the artwork attributions (repre-
sented by the property wdt:P50 and wdt:P170 and including from 1 to 4 authors in each
claim and the source of the claim) and artworks locations (represented by the property
wdt:P276 , including 1 possible location, time constraints and source) has been created6 .
Those new statements contain dummy arbitrary information ranked as deprecated and
therefore non-asserted to represent alternative or historical claims to those contained in
Art dataset. This design choice was made to increase the number of conjectural statements
in the final dataset.
An excellent way to evaluate an algorithm’s performance is to observe how it responds to
variations in input size[6]. We started by downloading the whole subset of artwork entities,
related individuals (basically, attributed authors) and locations. This dataset, called D4, is
composed of about 3,5 million artwork entities and 188 thousand related entities (humans and
locations). We have not used this dataset for our comparison due to the excessive number of
timeouts in many of the queries and methods we used. Thus we scaled the dataset logarithmically
in three further sizes:
• Dataset D3 : D3 is obtained by extracting one tenth of the data in D4 (D3 = D4/10).
• Dataset D2 : D2 is obtained by extracting one tenth of the data in D3 (D2 = D3/10).
• Dataset D1 : D1 is obtained by extracting one tenth of the data in D2 (D1 = D2/10).
We then surveyed the state of the art regarding reification methods to express without
asserting and selected a set of methods for our analysis: Singleton properties [12], Named
graphs[10] (using Wikidata rankings to decide whether a triple is asserted or not), Wikidata
[11] and the recent RDF-star[13] approach. We converted Wikidata JSON files into the six
selected reification methods through automatic scripts. In table 1 we provide some data about
our datasets. At the end of this process, we obtained 18 new method-specific datasets. In other
words, for each dataset 𝐷𝑛, 𝑛 ∈ [1, 3], we constructed the following datasets:
Consider the case of Napoleon crossing the alps and its concurring attributions. In addition to
the statements present in Wikidata, the listings below present an additional dummy statement
reporting the historical attribution of the painting to Sophie Chéradame (Q60804575), a state-
ment claimed by the source ContrivedAttributionsInArtHistory-VP and never adopted by the
6
The choice of adding the dummy claims is that of non-asserted statements in the Wikidata dump was circa 1%,
a low figure for this experiment
name Serialization Reification EWA # RDF stmts in D3
Dn-Wikidata Turtle Wikidata yes 66,768,937
Dn-rdfStar Turtle RDF-star yes 29,779,850
Dn-conjStrong TriG Conjectures - strong form yes 29,058,944
Dn-nGraphs TriG Named graphs via ranking 28,896,268
Dn-conjWeak TriG Conjectures - weak form yes 29,199,650
Dn-Singleton Turtle Singleton properties yes 55,325,270
Table 1
Datasets created (𝑛 ∈ [1, 3])
scholars (and therefore marked with deprecated rank). An example of these concurring claims
is represented in the listing below with a different reification method: Wikidata statements
(listing 3), RDF-star (listing 4), Conjectures in strong form (listing 5), Named graphs with the
(listing 6), Conjectures in weak form (listing 7) and Singleton properties (listing 8)7 .
wd : Q19801150 wdt : P170 wd : Q83155 .
s : Q19801150 s : P170 wd : Q19801150 − s 1 ;
ps : P170 wd : Q83155 ;
wd : Q19801150 wdt : P170 wd : Q83155 .
w i k i b a s e : r a n k w i k i b a s e : NormalRank .
<< wd : Q19801150 wdt : P170 wd : Q83155 >>
w i k i b a s e : rank w i k i b a s e : P r e f e r r e d R a n k .
wd : Q19801150 s : P170 s : Q19801150 − s 2 ;
ps : P170 ” unknown v a l u e ” ;
<< wd : Q19801150 wdt : P170 ” unknown v a l u e ” >>
pq : P1774 wd : Q83155 ;
pq : P1774 wd : Q83155 ;
pq : P3831 wd : Q4233718 ;
pq : P3831 wd : Q4233718 ;
w i k i b a s e : rank w i k i b a s e : DeprecatedRank .
w i k i b a s e : r a n k w i k i b a s e : NormalRank .
wd : Q19801150 s : P170 s : Q19801150 − s 3 ;
<< wd : Q19801150 wdt : P170 wd : Q672158 >>
ps : P170 wd : Q672158 ;
w i k i b a s e : r a n k w i k i b a s e : NormalRank .
w i k i b a s e : r a n k w i k i b a s e : NormalRank .
<< wd : Q19801150 s : P170 wd : Q60804575 >>
wd : Q19801150 s : P170 s : Q19801150dummy ;
pq : P248 c o n j : C o n t r i e v e d A t t r i b u t i o n s I n A r t H i s t o r y −VP ;
ps : P170 wd : Q60804575 ;
w i k i b a s e : rank w i k i b a s e : DeprecatedRank .
pq : P248 c o n j : C o n t r i e v e d A t t r i b u t i o n s I n A r t H i s t o r y −VP ;
w i k i b a s e : rank w i k i b a s e : DeprecatedRank .
Listing 4: RDF-star
Listing 3: Wikidata statements
SETTLED s : Q19801150 − s 1 { GRAPH s : Q19801150 − s 1 {
wd : Q19801150 wdt : P170 wd : Q83155 . wd : Q19801150 wdt : P170 wd : Q83155 .
} }
s : Q19801150 − s 1 w i k i b a s e : r a n k w i k i b a s e : P r e f e r r e d R a n k . s : Q19801150 − s 1 w i k i b a s e : r a n k w i k i b a s e : P r e f e r r e d R a n k .
CONJ s : Q19801150 − s 2 { GRAPH s : Q19801150 − s 2 {
wd : Q19801150 wdt : P170 ” unknown v a l u e ” . wd : Q19801150 wdt : P170 ” unknown v a l u e ” .
} }
s : Q19801150 − s 2 pq : P1774 wd : Q83155 ; s : Q19801150 − s 2 pq : P1774 wd : Q83155 ;
pq : P 3831 wd : Q4233718 . pq : P3831 wd : Q4233718 .
w i k i b a s e : r a n k w i k i b a s e : NormalRank . w i k i b a s e : r a n k w i k i b a s e : NormalRank .
CONJ s : Q19801150 − s 3 { GRAPH s : Q19801150 − s 3 {
wd : Q19801150 wdt : P170 wd : Q672158 . wd : Q19801150 wdt : P170 wd : Q672158 .
} }
s : Q19801150 − s 3 w i k i b a s e : r a n k w i k i b a s e : NormalRank . s : Q19801150 − s 3 w i k i b a s e : r a n k w i k i b a s e : NormalRank .
CONJ s : Q19801150dummy { GRAPH s : Q19801150dummy {
wd : Q19801150 s : P170 wd : Q60804575 wd : Q19801150 s : P170 wd : Q60804575
} }
s : Q19801150dummy w i k i b a s e : r a n k w i k i b a s e : D e p r e c a t e d R a n k ; s : Q19801150dummy w i k i b a s e : r a n k w i k i b a s e : D e p r e c a t e d R a n k ;
pq : P248 c o n j : C o n t r i e v e d A t t r i b u t i o n s I n A r t H i s t o r y −VP ; pq : P248 c o n j : C o n t r i e v e d A t t r i b u t i o n s I n A r t H i s t o r y −VP ;
Listing 5: strong form of Conjectures Listing 6: Named graphs
7
The approach adopted for the data acquisition and conversion is documented at https://github.com/
conjectures-rdf/expressing-without-asserting-efficiency-datasets
GRAPH s : Q19801150 − s 1 {
wd : Q19801150 c o n j 0 1 : P170 wd : Q83155 .
c o n j 0 1 : P170 c o n j : i s A C o n j e c t u r a l F o r m O f wdt : P170 .
}
GRAPH s : c o l l a p s e O f Q 1 9 8 0 1 1 5 0 − s 1 {
wd : Q19801150 wdt : P170 wd : Q83155 . wd : Q19801150 wdt : P170 wd : Q83155 .
s : c o l l a p s e O f − s 1 c o n j : c o l l a p s e s s : Q19801150 − s 1 wd : Q19801150 sng : Q19801150 − s 1 wd : Q83155 .
} Q19801150 − s 1 sng : i s A S i n g l e t o n P r o p e r t y O f wdt : P170 .
s : Q19801150 − s 1 w i k i b a s e : r a n k w i k i b a s e : P r e f e r r e d R a n k . s : Q19801150 − s 1 w i k i b a s e : r a n k w i k i b a s e : P r e f e r r e d R a n k .
GRAPH s : Q19801150 − s 2 {
wd : Q19801150 c o n j 0 2 : P170 ” unknown v a l u e ” . wd : Q19801150 s : Q19801150 − s 2 ” unknown v a l u e ” .
c o n j 0 2 : P170 c o n j : i s A C o n j e c t u r a l F o r m O f wdt : P170 . s : Q19801150 − s 2 sng : i s A S i n g l e t o n P r o p e r t y O f wdt : P170 .
} s : Q19801150 − s 2 pq : P1774 wd : Q83155 ;
s : Q19801150 − s 2 pq : P1774 wd : Q83155 ; pq : P3831 wd : Q4233718 .
pq : P3831 wd : Q4233718 . w i k i b a s e : r a n k w i k i b a s e : NormalRank .
w i k i b a s e : r a n k w i k i b a s e : NormalRank .
wd : Q19801150 s : Q19801150 − s 3 wd : Q672158 .
GRAPH s : Q19801150 − s 3 { s : Q19801150 − s 3 c o n j : i s A C o n j e c t u r a l F o r m O f wdt : P170 .
wd : Q19801150 c o n j 0 3 : P170 wd : Q672158 . s : Q19801150 − s 3 w i k i b a s e : r a n k w i k i b a s e : NormalRank .
c o n j 0 3 : P170 c o n j : i s A C o n j e c t u r a l F o r m O f wdt : P170 .
} wd : Q19801150 s : Q19801150dummy wd : Q60804575 .
s : Q19801150 − s 3 w i k i b a s e : r a n k w i k i b a s e : NormalRank . s : Q19801150dummy c o n j : i s A C o n j e c t u r a l F o r m O f wdt : P170 .
s : Q19801150dummy w i k i b a s e : r a n k w i k i b a s e : D e p r e c a t e d R a n k ;
GRAPH s : Q19801150dummy { pq : P248 c o n j : C o n t r i e v e d A t t r i b u t i o n s I n A r t H i s t o r y −VP ;
wd : Q19801150 c o n j 0 4 : P170 wd : Q60804575 .
}
c o n j 0 3 : P170 c o n j : i s A C o n j e c t u r a l F o r m O f wdt : P170 . Listing 8: Singleton properties
s : Q19801150dummy w i k i b a s e : r a n k w i k i b a s e : D e p r e c a t e d R a n k ;
pq : P248 c o n j : C o n t r i e v e d A t t r i b u t i o n s I n A r t H i s t o r y −VP ;
Listing 7: weak form of Conjectures
4.2. Hardware and software configuration
Tests have been run on a computer with processor Intel Core i5-8259U CPU @ 2.30GHz 2.30
GHz, RAM 32,0 GB, Windows 10 pro 64 bits, 1T hard disk. The TriG and SparQL parsers
of our GraphDB engine were modified to parse Conjectures in strong form. Our GraphDB
configuration uses 28G Ram allocated to the application,89 , and 10G cache size. A repository
has been created for each dataset with inferences off, no rule set assigned, predicates list index
enabled and (when possible) contexts enabled. All other parameters are left in their default
values. Repositories are already running before their performance tests are executed.
4.3. Metrics
We decided to base the comparison of our reification methods on four major metrics. These
metrics are well-established when it comes to RDF quantitative analysis. The performance-
related features of the reification methods under consideration should all be covered by those
criteria, which should also give us a clear picture of the benefits and drawbacks of each method.
• Total number of triples in endpoint: This value is particularly interesting since it makes it
possible to assess the verbosity of each method.
• Loading time: Time consumed by each dataset to be uploaded in the SPARQL endpoint.
• Dataset weight in triplestore: The storage size of the dataset after it has been uploaded
and stored in the triplestore.
8
https://graphdb.ontotext.com/documentation/10.1/configuring-graphdb-memory.html
9
https://graphdb.ontotext.com/documentation/10.2/getting-started.html#:~:text=the%20aforementioned%20icon.
-,Configuring%20the%20JVM,Contents%2Fapp%2FGraphDB%20Desktop
• Query execution time: Response time on a selected set of queries. Each query is executed
automatically ten times. The average value is then computed.
4.3.1. Queries
Two sets of SPARQL queries (GQn, FQn) have been designed. While GQn queries do not
include any filter, FQn queries restrict the results only to paintings (Q3305213 ). Each query
set is composed of 6 queries assessing all possible statuses of claims validity. In particular, the
queries retrieve the following topics: valid claims (Q1), debated claims (Q2), debated claims
with their provenance/time (Q3), currently disputed claims (Q4), accepted claims after being
debated (Q5), undisputed claims (Q6). Considering that authors’ and locations’ attributions
provide a simple, yet effective use case to test RDF representation of EWA over our dataset,
GQn and FQn have been then customised on retrieving authorship attributions (GQn-P170
and FQn-P170) and artworks’ locations (GQn-P276 and FQn-P276) respectively by the use of
Wikidata properties P170 and P276 . All queries return the same data and are all correct in their
respective datasets. Each query set has been automatically run 10 times and the average times
have been calculated. Table 4.3.1 summarizes the nature of the queries. All actual queries are
available at https://github.com/conjectures-rdf/expressing-without-asserting-efficiency-tests
as well as the full set of results.
Query Predicate Data selected by query
GQ1 P170 All attributions of artworks that are currently considered valid
GQ1 P276 All locations of artworks that are currently considered valid
GQ2 P170 All attributions of artworks that have been debated
GQ2 P276 All past and debated locations of artworks
GQ3 P170 All attributions of artworks that have been debated, with provenance
GQ3 P276 All past and debated locations of artworks, with date of move
GQ4 P170 All currently debated attributions of artworks
GQ4 P276 All locations of artworks whose current location is uncertain
GQ5 P170 All settled attributions of artworks
GQ5 P276 All current locations of artworks that were moved
GQ6 P170 All attributions of artworks that were never debated
GQ6 P276 All locations of artworks that never moved
FQ1 P170 All attributions of paintings (Q3305213) that currently are considered valid
FQ1 P276 All locations of paintings (Q3305213) that are currently considered valid
FQ2 P170 All attributions of paintings (Q3305213) that have been debated
FQ2 P276 All past and debated locations of paintings (Q3305213)
FQ3 P170 All attributions of paintings (Q3305213) that have been debated, with provenance
FQ3 P276 All past and debated locations of paintings (Q3305213), with date of move
FQ4 P170 All currently debated attributions of paintings (Q3305213)
FQ4 P276 All locations of paintings (Q3305213) whose current location is uncertain
FQ5 P170 All settled attributions of paintings (Q3305213)
FQ5 P276 All current locations of paintings (Q3305213) that were moved
FQ6 P170 All attributions of paintings (Q3305213) that were never debated
FQ6 P276 All locations of paintings (Q3305213) that never moved
Table 2
Overview of query types (GQn and FQn) for artworks’ attributions and locations
Figure 2: Dataset measurements for the surveyed reification methods. In particular, number of triples
in endpoint (top-left), dataset loading time (top-right) and dataset weight in endpoint (bottom-left)
5. Test results
5.1. Number of triples in endpoint
All reification methods add additional triples to the already existing ones to represent the
necessary metadata (e.g. Singleton properties) or extend RDF 1.1 syntax (e.g. RDF-star). As
shown in table 2, Named graphs are the method which uses reification with the lower number
of triples, but with no explicit distinction between asserted and non-asserted graphs. While
other surveyed methods (in particular, RDF-star, Wikidata statements and Singleton properties)
use reification methods and assert each claim with an additional triple, Conjectures uses Named
graphs structure to express both statements and reification without adding additional triples
resulting in the method to express without asserting with the lowest addition of triples.
5.2. Loading Time
In the context of dataset D1 and D2, Conjectures in the strong form remain competitive with the
most efficient methods, notably RDF-star, and outperform Wikidata statements and Singleton
properties as shown in 2. However, the loading times increase in D3. This performance discrep-
ancy is attributed to the triplestore’s parser method for recognizing conjectural data. Specifically,
the process of checking each resource’s presence in a collection during loading contributes to
the observed delays. In essence, the loading time of the dataset increases proportionally with
the quantity of non-asserted triples (conjectures).
5.3. Dataset weights in triplestore
The Singleton method exhibits a storage size tenfold greater than alternative approaches, with
Conjectures in their weak form and Wikidata occupying intermediate positions. RDF-star,
Conjectures in their strong form, and Named graphs demonstrate similar sizes as shown in
figure 2.
5.4. Query Execution Time
The time response average for each dataset seems to increase linearly for each surveyed dataset
(D1, D2, D3). For this reason, figure 3 provides the snapshot of the execution time of queries
GQn and FQn on attributions and locations only on dataset D3.
As illustrated in figure3, the response times obtained from the execution of general queries
(GQn) on dataset D3 show that the strong form of Conjectures is less efficient than other methods
when retrieving asserted data, particularly in retrieving valid claims, currently disputed claims
and undisputed claims (queries GQ1, GQ4 and GQ6) for both creators and locations. However,
Conjectures perform better with disputed claims. In particular, Weak and Strong Conjectures
outperform other surveyed methods in the retrieval of debated statements with and without
provenance information (queries GQ2 and GQ3) and accepted claims after being debated (GQ5)
for both locations and creators.
Similar to what was observed in GQn, Conjectures are less efficient in retrieving valid claims
(FQ1). On the contrary, Conjectures strong form is the most efficient method regarding the
retrieval of debated claims with and without contextual information (queries FQ2 and FQ3)
and accepted claims after debate (FQ5). In the remaining queries, currently disputed claims
(FQ4) and statements that have never been subject to debate (FQ6), Conjectures still maintain
competitive times with the rest of the methods. Strong Conjectures, in particular, address the
significant increase in response times for weak-form queries FQn[3:8]. Essentially, a notable
improvement in the performance of the strong form has been detected, proving to be the most
efficient method in half of the selected queries and, in the remaining ones, a valid competitor.
6. Discussion
6.1. EWA expressivity
In query performance assessment, we registered some slightly different values for the number
of results of some queries, such differences are explained below highlighting some intrinsic
differences in the models. Typically, a SPARQL query can still retrieve claims that won a certain
debate by accessing concurring ones stored in the Knowledge Graph (KG). In cases where debates
are not recorded, and only accepted statements are reported, reification approaches fall short.
Reification methods do not distinguish between claims that have never been questioned and
Figure 3: Time responses for queries sets GQn and FQn on creators and locations run against D3 dataset
settled claims post-debate since both are recorded as asserted triples. At this point, an ontology-
dependent solution must be addressed to represent this differentiation such as Wikibase ranking.
For instance, the painting Portrait of Dona Isabel de Requesens (Q29651096) has been attributed
to Giulio Romano and marked as the settled claim, but no concurrent claim is reported. The
concept of SETTLED in Conjectures can also capture this nuanced distinction in an ontology-
independent fashion which is transversal to be used on whichever KG.
Another instance is when two claims express the same triple but with different qualifiers.
Consider a scenario where a historical painting X was initially attributed to author Y (marked
as an attribution) and afterwards this attribution has been considered settled by the community.
The historical attribution cannot be retrieved in a SPARQL query since its content is also asserted.
But, while Wikidata and Singleton provide unique IDs to distinguish such claims which can be
involved in the query to retrieve such claim, RDF-star associates all contextual triples with the
same quoted triple. This becomes more complex in multi-triple claims, where expressing them
as individual Wikidata statements is not feasible. For instance, when dealing with paintings
attributed to the collaboration of multiple individuals, expressing this complexity becomes
particularly interesting. Conjectures use Named graphs to group statements. which allows to
express and retrieve such complex statements with simple SPARQL queries. Other methods
would require to adopt other types of grouping methods (e.g. a n-ary relationship and/or a
blank node) with additional complexity and execution times.
6.2. EWA efficiency
Overall, we can immediately see several trends concerning the surveyed methods’ efficiency:
Singleton properties are systematically slower than the others, while Named graphs and Con-
jectures in weak form performs at an intermediate level about the fastest methods, Wikidata,
Conjectures in strong form and RDF-star. Strong form also outperforms RDF-star in many
queries where the specifics of debated attributions and past locations become meaningful.
Strong form is the quickest method for expressing debates (disputed claims, GQ2 and FQ2,
disputed claims with provenance GQ3 and FQ3, settled claims GQ5 and FQ5), with a small loss
in term of performance for what concern asserted claims (valid claims GQn and FQn, undisputed
claims GQ6 and FQ6) and currently disputed claims (GQ5 and FQ5).
Conjectures in strong form are also competitive for what concerns number of triples and
overall weight in the triplestore. It is competitive on loading time for, but loading times show
an interesting loss of performance for large datasets, that need to be investigated further.
7. Conclusions
This work evaluates the efficiency of EWA mechanisms by comparing several reification methods
(Wikidata, RDF-star, Named graphs, Singleton properties) and the novel Conjectures approach
(weak and strong form) on four major metrics (number of triples, loading time, dataset weight
and query execution time). Among the most efficient methods as RDF-star and Wikidata
statements, the strong form of Conjectures exhibits notable performance gains, particularly in
retrieving claims about debates (e.g., disputed claims with and without provenance information
and settled claims). In the future, we aim to optimize some aspects concerning the efficiency of
the strong form of Conjectures. In particular, we will prioritize the optimization of the loading
process, aiming to reduce the loading time and enhance its overall performance.
References
[1] J. J. C. Graham Klyne, Resource Description Framework (RDF): Concepts and Abstract
Syntax (2004). URL: https://www.w3.org/TR/2004/REC-rdf-concepts-20040210/.
[2] M. Giunti, G. Sergioli, G. Vivanet, S. Pinna, Representing n-ary relations in the semantic
web, Logic Journal of IGPL (2019). doi:10.1093/jigpal/jzz047 .
[3] A. Freundschuh, Crime stories in the historical urban landscape: narrating the theft of the
mona lisa, Urban History 33 (2006) 274–292. doi:10.1017/S0963926806003816 .
[4] B. Lewis, Salvator mundi: Going out on a limb, KUR - Kunst und Recht 25 (2023). URL:
https://doi.org/10.15542/KUR/2023/2/3. doi:10.15542/KUR/2023/2/3 .
[5] P. Hayes, J. Carroll, C. Welty, M. Uschold, B. Vatant, F. Manola, I. Herman, J. Lawrence,
Defining N-ary Relations on the Semantic Web (2006). URL: https://www.w3.org/TR/2006/
NOTE-swbp-n-aryRelations-20060412/.
[6] F. Orlandi, D. Graux, D. O’Sullivan, Benchmarking rdf metadata representations: Reifi-
cation, singleton property and rdf*, 2021 IEEE 15th International Conference on Seman-
tic Computing (ICSC) (2021) 233–240. URL: https://api.semanticscholar.org/CorpusID:
232151947.
[7] M. Daquino, V. Pasqual, F. Tomasi, F. Vitali, Expressing without asserting in the arts, in:
Proceedings of the Italian Research Conference on Digital Libraries. Padova, Italy, 2022.
[8] P. Hayes, Rdf semantics, w3c recommendation, http://www. w3. org/TR/rdf-mt/ (2004).
[9] N. Noy, A. Rector, P. Hayes, C. Welty, Defining n-ary relations on the semantic web, W3C
working group note 12 (2006).
[10] J. J. Carroll, C. Bizer, P. Hayes, P. Stickler, Named graphs, Journal of Web Semantics 3
(2005) 247–267.
[11] F. Erxleben, M. Günther, M. Krötzsch, J. Mendez, D. Vrandečić, Introducing Wikidata to
the Linked Data Web, in: P. Mika, T. Tudorache, A. Bernstein, C. Welty, C. Knoblock,
D. Vrandečić, P. Groth, N. Noy, K. Janowicz, C. Goble (Eds.), The Semantic Web – ISWC
2014, Springer International Publishing, Cham, 2014, pp. 50–65.
[12] V. Nguyen, O. Bodenreider, A. Sheth, Don’t like rdf reification? making statements
about statements using singleton property, in: Proceedings of the 23rd International
Conference on World Wide Web, WWW ’14, Association for Computing Machinery, New
York, NY, USA, 2014, p. 759–770. URL: https://doi.org/10.1145/2566486.2567973. doi:10.
1145/2566486.2567973 .
[13] O. Hartig, Foundations of rdf* and sparql*:(an alternative approach to statement-level
metadata in rdf), in: AMW 2017 11th Alberto Mendelzon International Workshop on
Foundations of Data Management and the Web, Montevideo, Uruguay, June 7-9, 2017.,
volume 1912, Juan Reutter, Divesh Srivastava, 2017.
[14] A.-C. Ngonga Ngomo, I. Fundulaki, A. Krithara, J. Frey, K. Müller, S. Hellmann, E. Rahm,
M.-E. Vidal, A.-C. Ngonga Ngomo, I. Fundulaki, A. Krithara, Evaluation of metadata
representations in rdf stores, Semant. Web 10 (2019) 205–229. URL: https://doi.org/10.3233/
SW-180307. doi:10.3233/SW- 180307 .
[15] V. Nguyen, O. Bodenreider, K. Thirunarayan, G. Fu, E. Bolton, N. Queralt-Rosinach, L. I.
Furlong, M. Dumontier, A. Sheth, On reasoning with rdf statements about statements
using singleton property triples (2015).
[16] D. Hernández, A. Hogan, M. Krötzsch, Reifying RDF: what works well with wikidata?, in:
T. Liebig, A. Fokoue (Eds.), Proceedings of the 11th International Workshop on Scalable
Semantic Web Knowledge Base Systems, volume 1457 of CEUR Workshop Proceedings,
CEUR-WS.org, 2015, pp. 32–47.
[17] A. Hogan, J. Umbrich, A. Harth, R. Cyganiak, A. Polleres, S. Decker, An empirical survey
of linked data conformance, J. Web Semant. 14 (2012) 14–44. doi:http://dx.doi.org/10.
2139/ssrn.3198962 .
[18] G. Demartini, I. Enchev, M. Wylot, J. Gapany, P. Cudré-Mauroux, BowlognaBench—Bench-
marking RDF Analytics, in: K. Aberer, E. Damiani, T. Dillon (Eds.), Data-Driven Process
Discovery and Analysis, Springer Berlin Heidelberg, Berlin, Heidelberg, 2012, pp. 82–102.
[19] Y. Theoharis, V. Christophides, G. Karvounarakis, Benchmarking Database Representations
of RDF/S Stores, in: Y. Gil, E. Motta, V. R. Benjamins, M. A. Musen (Eds.), The Semantic
Web – ISWC 2005, Springer Berlin Heidelberg, Berlin, Heidelberg, 2005, pp. 685–701.