=Paper=
{{Paper
|id=Vol-3749/SEMMES_2024_paper_2
|storemode=property
|title=Let the fallen Voussoirs of Notre-Dame de Paris Speak: Scientific Narration and 3D
Visualization of Virtual Reconstruction Hypotheses and Reasoning
|pdfUrl=https://ceur-ws.org/Vol-3749/SEMMES_2024_paper_2.pdf
|volume=Vol-3749
|authors=Anaïs Guillem,John Samuel,Gilles Gesquière,Livio De Luca,Violette Abergel
|dblpUrl=https://dblp.org/rec/conf/esws/Guillem0GLA24
}}
==Let the fallen Voussoirs of Notre-Dame de Paris Speak: Scientific Narration and 3D
Visualization of Virtual Reconstruction Hypotheses and Reasoning==
https://ceur-ws.org/Vol-3749/SEMMES_2024_paper_2.pdf
Let the fallen Voussoirs of Notre-Dame de Paris
Speak: Scientific Narration and 3D Visualization of
Virtual Reconstruction Hypotheses and Reasoning
Anaïs Guillem1,4,* , John Samuel2,* , Gilles Gesquière3 , Livio De Luca1 and
Violette Abergel1
1
UPR MAP CNRS 2002, Campus CNRS, 31 chemin Joseph Aiguier, 13009, Marseille, France
2
CPE Lyon, LIRIS, UMR-CNRS 5205, Université de Lyon, Lyon, France
3
LIRIS UMR-CNRS 5205, Université Lumiere Lyon 2¸Université de Lyon, Lyon, France
4
Ludwig Maximilian University of Munich (LMU), Geschwister-Scholl-Platz 1, Munich, Germany
Abstract
Virtual reconstruction should go beyond merely presenting images and 3D models by docu-
menting the scientific context and reasoning underlying the reconstruction process, rather than
just showcasing the final product. For instance, the collapsed transverse arch in the nave of
Notre-Dame de Paris serves as a case study to demonstrate an interdisciplinary reconstruction
workflow. By letting the voussoirs speak, we mean that the materiality of the voussoirs and
immateriality of digital surrogates are the support to make explicit the argumentation of the
reconstruction. The 3D visualization moves away from a static and finalized illustrative output
of the reconstruction study and toward an open and dynamic visualization of reconstruction
data. The data explicitly records both factual information on the physical and digital objects,
as well as the counterfactual propositions of the reasoning of reconstruction hypotheses. The
proposed experiment is twofold: (1) setting up of the 3D dataset of the arch reconstruction
with archaeological argumentation where the hypotheses are modeled as versions, and (2)
evaluation of the scientific narrative of reconstruction argumentation through both a custom
3D visualization and competency questions on the enriched 3D data combining hypotheses
and arguments. The humanistic question of reconstruction is the starting point for a nonlinear
scientific narrative composed of hypotheses and argument loops. We consider the conflicting
interpretations of the voussoirs and the knowledge encapsulated in relation to the spatial
configuration of the arch. The results of the queries and 3D visualization are interdependent:
they demonstrate that hypothetical reasoning facets of the arch reconstruction are embedded
in the spacetime volumes of the voussoirs.
Keywords
Virtual reconstruction, Notre-Dame de Paris, argumentation, scientific narrative, narrative
argument, 3D visualization, logic programming, serendipity, counterfactual reasoning, scientific
reasoning, scientific hypothesis
SEMMES’24: Semantic Methods for Events and Stories co-located with 20th Extended Semantic Web
Conference (ESWC2024), May 26-27, 2024, Hersonissos, Greece
*
Corresponding author.
$ anais.guillem@map.cnrs.fr (A. Guillem); john.samuel@cpe.fr (J. Samuel);
gilles.gesquiere@univ-lyon2.fr (G. Gesquière); violette.abergel@map.cnrs.fr (V. Abergel)
0000-0002-1473-7594 (A. Guillem); 0000-0001-8721-7007 (J. Samuel); 0000-0001-7088-1067
(G. Gesquière); 0000-0003-0656-3165 (L. D. Luca); 0000-0002-3688-3306 (V. Abergel)
© 2024 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0
International (CC BY 4.0).
CEUR
ceur-ws.org
Workshop ISSN 1613-0073
Proceedings
1. Introduction
Virtual reconstructions have the amazing power to bring a past to life. A virtual
reconstruction is a common exercise for the disciplines interested in cultural history to
represent some aspect of a past reality. However, the resulting reconstruction images and
3D visualizations conceal the underlying scientific process and hypothetical reasoning that
contributed to their production. In this article, we tackle the gap in the literature between
the knowledge that is produced in a reconstruction study and actual reconstruction data
that one can either query or visualize. In the context of Notre-Dame de Paris, the fallen
transverse arch in the nave provides a case study (Figure 1), where the challenge is to
determine the positioning of its voussoirs before its destruction. The 3D visualization
prototyping and querying bring to the foreground the hypotheses and argumentation,
where the challenge is to determine the positioning of its voussoirs before its destruction.
Given the fragmentary documentation available, there are several plausible solutions
to reconstruct this arch, some more valid than others. We then need to present the
reasoning behind these choices. Such a case study allows for testing a data workflow on
the reconstruction argumentation reasoning.
The question of virtual reconstruction can be defined as a scientific study that aims at
formulating an argumentation linking three elements: (1) the material objects (voussoirs,
arch), (2) the digital objects (pointclouds, digital photographs, datasets), and (3) expert
knowledge (archaeology, art history, architecture, computer science etc). These elements
are tied together with the hypothetical scientific reasoning about reconstruction question.
In this work, we posit that the 3D visualization and querying are constitutive research
tools for the virtual reconstruction research, rather than an end result medium for
dissemination. By letting the voussoirs speak, we mean that the materiality of the
voussoirs and immateriality of digital surrogates are the support to make explicit the
argumentation of the reconstruction. The virtual reconstruction research is rooted both
in the real (material and immaterial object) and the unreal (hypotheses formulation),
where the scientific inquiry about the past can be developed. Then the challenge is about
reasoning both about factual knowledge and counterfactual propositions: a reconstruction
formulates a hypothesis about something that might have existed based on the existing
presence of objects that are surviving witnesses of this lost past.
This contribution proposes a twofold experiment. Firstly, it sets up the 3D dataset of
Notre-Dame’s de Paris’ transverse arch reconstruction, including the data enrichment
with archaeological knowledge proposed by archaeologists and subject experts. These
requirements were detailed in previous studies [1, 2, 3, 4, 5, 6]. Far from being straightfor-
ward, this step introduces the innovative elements of linking counterfactual reasoning to
real things (physical or digital) and unreal things (competitive reconstruction hypotheses)
at the dataset level. The scientific narrative of the reconstruction study is tested using
Prolog programming, where the formal expressions of relations between hypotheses,
arguments, and objects are modeled. Secondly, after the transformation to Prolog facts
and rules of the initial reconstruction dataset, the evaluation of the scientific narrative of
reconstruction argumentation is tested through both the 3D visualization and query of
the 3D enriched data. The humanistic question of reconstruction is the starting point for
Figure 1: the Notre-Dame de Paris arch reconstruction study (T4-T9) aims at formulating knowledge
about a disappeared past (T1), the arch before its destruction (T2). T[N] corresponds to the time of
the reconstruction study activities.
the nonlinear scientific narrative composed of hypotheses and argument loops that we
want to query and visualize in 3D. Both 3D visualization prototyping and querying bring
to the foreground the hypotheses and argumentation of the reconstruction question.
Do the 3D visualization and querying have this ability to explicitly convey the recon-
struction study process and arguments? To answer positively, the visualization needs to
move away from a static and finalized illustrative output of the reconstruction study and
toward an open, dynamic visualization of reconstruction data. Following the principles
of open science, the proposed data practice (i.e., enrichment, transformation, inference
etc.) can serve as a lever for the scientific narrative and the argumentation. Beyond the
factual level, we consider the interpretation of results and the counterfactual knowledge
encapsulated in relation to its spatial configuration. The interpretation of results and 3D
visualization become interdependent and complementary. Whereas 3D visualization tools
typically represent 3 dimensional objects + 1 dimension for time, this experiment aims
to encompass counterfactual information (hypothetical reasoning of the reconstruction)
in relation to spatial (3D) and temporal information.
In section 2, we present the existing gap in documenting counterfactual reasoning of
virtual reconstruction. The scientific inquiry over reconstruction leads to formalizing
concurrent points of view about space, place, and memory, that bring the theoretical
concept of place making. Querying and visualizing reconstruction data opens the problem
of representing factual/counterfactual and spatio-temporal data. In section 3, we develop
the experiment’s methodology: the reconstruction hypotheses are considered as versions
or knowledge steps. Subsequently, in section 4, we detail the implementation the
counterfactual layer in the reconstruction data using Prolog. In sections 5 and 6, the
complementarity of the 3D visualization and querying for the understanding of the
reconstruction reasoning is demonstrated: in section 5, the resulting transformed data
is evaluated with the competency questions focusing on the counterfactual elements,
while in section 6, the 3D visualization shows how the hypothetical reasoning of the arch
reconstruction is embedded in the spacetime volumes of the voussoirs.
2. State of the Art
2.1. Reconstruction in Archaeology: the Problem of Documenting
Counterfactual Reasoning
In this part, we present the gap in the literature between the knowledge about the
past in a reconstruction study and the actual reconstruction data. The problem of
documenting virtual reconstruction has roots in philosophy of language, semantics, and
logic. For Clark [7], the reconstruction of the past in archaeology is a misnomer since
virtual reconstruction is rather the construction of a model for the study of the past as
“tools for understanding, not statements of reality”. This debate is exacerbated with the
development of virtual archaeology and the democratization of digital documentation
[8, 9, 10, 11]. A reconstruction study is “based on complex chains of reasoning grounded
in primary and secondary evidence that enable a historically probable whole to be
reconstructed from the partial remains left in the archaeological record” [9]. In a
previous work [4] we demonstrated that these complex chains of reasoning in virtual
reconstructions experience a phenomenon of compression both at a level of representation
and reconstruction. The reconstruction study builds on knowledge of real/“actual” things
(artifacts, remains etc.), as well as unreal or “intricate counterfactual blends” [12].
For memorizing the reconstruction reasoning, the problem can be formulated as follows:
the reconstruction study (or model) formulates implicitly a knowledge object that is
in-between real objects and counterfactual propositions. Making the voussoirs speak, we
aim at unfolding and explicitly formulating these intricate counterfactual blends. The
goal is the explanation of the proposition sets, arguments, and inference logic that are
used in the argumentation of the reconstruction [4]. From the perspective of data science,
Pearl [13] posits interpretation as how to connect the reality and the data, as a way to
encode causal assumptions. Transparency and testability are proposed as requirements for
data transformation: “Transparency enables analysts to discern whether the assumptions
encoded are plausible (on scientific grounds) or whether additional assumptions are
warranted. Testability permits us (whether analyst or machine) to determine whether
the assumptions encoded are compatible with the available data [...]” ([13], p.58). Our
proposed experiment enriches the reconstruction data and encodes the reconstruction
arguments as the causal assumptions used in the virtual reconstruction, applying the
transparency and testability criteria.
2.2. Formalizing the Concurrent Points of View about Space, Place, Memory,
and Placemaking
We explore the problem of the reconstruction of historical buildings and their restoration,
considering a series of competing interpretations. This leads us to look at space as meaning-
laden, that is a place. Nora introduced the notion of lieu de mémoire (place of memory)
[14], which can help in formalizing our understanding of the interrelations between space
and memory. Lieu de mémoire, as defined by Nora, is not just a physical location,
it embraces the “embodiment of memory in certain sites where a sense of historical
continuity persists” [14]. The lieux de mémoire exist because of the disappearance
of milieux de mémoire (cultural material environment). The virtual reconstruction of
the transverse arch is consecutive to its destruction. The voussoirs have survived and
became the material witnesses of the destroyed arch. Lieux de mémoire are the surviving,
remaining material elements of embedded memory. Memory, either social, collective or
individual, is affected by its relation to space and materiality: its meaning attachment
to a space creates a place. Stories and events play an important role in making a space
a place. For every place, different people have different associated memories [15, 16],
often providing a subjective version of the same place and the events that occurred.
Number of research works in records and documents oral, sensorial, and written histories
associated with places. Place encompasses materiality that ranges from the scale of
objects, buildings to urban fabric or landscapes. That explains why urban planners,
historians, architects, and anthropologists give a significant importance to the concept of
placemaking since memories give meaning to the surrounding materiality.
At the urban scale, memories may be available in several forms [17], media documents
forming a key form of recording. The documents narrating the historical events and their
chronology frequently serve as evidence for understanding the incremental changes that
happened to urban and architectural objects. Historical documents and archives have
been used with their 2D/3D representations [18, 19] to represent concurrent points of
view of urban evolution capable of narrating numerous possible scenarii of city changes
at different levels of detail. Numerous ontologies and documentation standards have
been proposed to represent (historical) episodes, events (CIDOC CRM as event-centric
ontology), stories, historical narratives [16], and argumentation using CIDOC CRMinf
[20] or other argumentation models [21].
With the models and ontologies in place, a methodological approach based on versioning
is used to store and visualize historical narratives about cultural heritage artifacts at
different scales. Extending the use of version control systems like Git, Samuel et al. [22]
propose to store multiple scenarios of urban evolution as city versions. Hypermedia stories
- graph-structured stories with navigation links - [23, 16] uses media along with these
narratives to better explain the modification of the urban fabric and cultural heritage
assets. At the architectural scale, the evolution of objects is apprehended through
diagrammatic visualizations called diachrogram to capture the changes and periods of
stability in building lives [24]. Metral et al. [25], Bruseker et al. [9] show the compatibility
and applicability of knowledge representation and ontology design approach respectively
to the urban and built heritage domain.
In this article, we want to go beyond the use case of representing and visualizing urban
evolution, as well as the linear [26, 27] or the graph-structured stories [28, 16] of a series
of events [29]. We propose the extension of some of the above works to represent the
hypotheses evolution during the reconstruction process. We build upon the studies on
historical changes to model the hypotheses associated with virtual reconstruction, where
the goal is to ultimately bring back a damaged object to its hypothetical lost state.
2.3. Visualization
For the visualization in heritage field, one observes a duality between semantic-oriented
approaches and 3D visualization-oriented approaches. On one hand, the approaches
centered on knowledge modeling mainly present information systems designed to manage
documentation activities, without directly exploiting the full wealth of heterogeneous
data mobilized in heritage contexts, in particular 3D data [30, 31, 32]. On the other
hand, approaches based on the exploitation of 3D digital data, focus mostly on their
description, omitting any reference to the complex relationships specific to the observation,
interpretation, or documentation processes [33, 34]. Some works, such as Soler et al.
[35], Apollonio et al. [36], clearly demonstrate the strong interest in bringing these two
approaches together, making it possible to spatialize and query heterogeneous information
in the context of restoration operations. However, these initiatives are generally built
around isolated 3D models, excluding the possibility of handling and confronting the
diversity of resources involved in heritage studies, like Notre-Dame scientific action.
From a software point of view, there are a number of very effective libraries dedicated
to the manipulation of 3D data, which can be used for reconstruction studies [37, 38, 39].
But these tools focus on visualization and interaction modalities, allowing only in the
best cases to display provenance metadata, and in most cases ignoring the full potential
of an upstream work on data curation and conceptual modeling. In [40], spatio-temporal
information of 3D data is characterized describing the transformations affecting buildings
over time and considering their changes (demolition, amplification, division, or change of
function). It is based on the addition of a semantic layer to the geometric restitution
through description graphs, enabling information on buildings to be enriched according
to existing documentary sources. While this approach is of considerable interest, it
remains ill-suited to virtual reconstruction data documenting the scientific context and
reasoning about the reconstruction process. The temporal dimension addressed in [40]
is linear, making it difficult to semantically and geometrically characterize competing
hypotheses and their underlying arguments. Samuel et al. [19] and Jaillot et al. [41]
propose knowledge versioning in 3D urban contexts. While this approach is interesting
in terms of data structuring, all the narrative potential remains to be explored.
3. Methodology
In this section, we consider the example of the arch of Notre-Dame de Paris that was
damaged during the 2019-fire. The voussoirs that together formed the arch fell on
the nave’s floor and were displaced. The scientific research and restoration activities
have divergent objectives and interpretations, while working on the same object. The
virtual reconstruction study aims at gathering knowledge about the destroyed arch and
understanding which original place or slot was for each authentic voussoir in the arch
before the fire [3, 5, 6]. However, this process is extremely difficult and several sessions
were held [1, 2], where historians discussed and proposed reconstruction hypotheses.
Figure 2 shows multiple rounds of discussion for placing a voussoir in specific slots.
Six overall hypotheses were formulated at different times for identifying the position of
voussoirs. It is important to note that a given voussoir may have changed their slots
during the different hypotheses based on additional collected evidence. The ultimate
goal is to possibly relocate the voussoirs and the arch to their hypothetical original
positions on time T1 (i.e., before the fire). Unlike [18] or [19] that look at the evolution
of historical artifacts or city objects (T1, T2, T3) as recorded in time, the reconstruction
uses a fiction as if we were looking backward in time. In fact, the past object of study
(T1) is a hypothetical model for knowledge, where the reconstruction hypotheses can
be seen as versions. At the time T1, there are several hypothetical versions that were
proposed with the end goal of creating a reconstructed arch that gets as close as the arch
before the fire (i.e. the hypothetical original version V0 at T1). It must be pointed out
that we may never arrive at the ultimate version of the reconstruction V?? (shown in
Figure 2 ) that closely approximates V0.
Figure 2: Diagram representing the different hypotheses of reconstruction, as versions of knowledge
about the arch.
In order to restore the arch to its hypothetical original state, it is critical to understand
that the arch has two sides, the northern and southern sides. A number of 122 voussoirs
together form the whole arch with 40 voussoirs still in place and 77 recovered voussoirs.
The (possible) slots for the fallen voussoirs have been identified and numbered, using
survey documentation prior to the fire. The slots on the south side are numbered with
even numbers, while those on the north are numbered with odd numbers, starting with
R0 for the keystone. Thus, it is easy to understand that R12 is the 6th slot on the north
side of the arch. The fallen voussoirs have also been identified and inventoried N1,. . . , etc.
The goal of reconstruction is to assign a slot 𝑅𝑛 (values of n are between 0 and 121) to
every voussoir 𝑁𝑘 (values of k are between 0 and 121). There are multiple collaborative
work sessions between researchers (archaeologists, architects, art historians, etc.) to
produce such possible reconstruction hypotheses. Each hypothesis has its associated
arguments.
In Figure 2, the arrows between the two versions may have different significance for the
reconstruction reasoning. A decision about a voussoir based on a particular argument
may remain valid across different versions. However, this also means that any initial
hypotheses, if proven false at a later date may lead to the falsification of some pairs of slot-
voussoirs from another hypothesis. Take, for example, the evolution of the argumentation
process related to cross-shaped marks on the joint sides of the voussoirs. These lapidary
marks constitute an archaeological predicate for the arch reconstruction. The marks were
believed to possibly serve as guide the masons to orient the positioning of the voussoir
during the construction. The mark could indicate either the downward or upward side. In
a first hypothesis the voussoirs were oriented with their lapidary mark facing downward.
This was later proved to be incorrect. This meant that we had to relook all the previous
use of these arguments and make corrections to the assigned facts: slot-voussoir pairs,
voussoirs, if necessary. This makes the problem particularly challenging since we need
to record this aspect for every voussoir and the assigned slot along with the associated
arguments in the different hypotheses. In the reconstruction data, we have a selection of
relevant information about the physical objects (voussoirs) and possible locations (slots)
in the state of the arch before destruction. The difficulty lies in the reasoning about the
characteristics about the real objects in relation to the argumentation hypothesis of the
reconstruction, i.e., counterfactual reasoning. The reconstruction data is an open-ended
hypothesis, as shown in Figure 2.
4. Implementation
After exploring the initial reconstruction dataset available in [3], the experiment proposes
to encode the reconstruction arguments as the causal assumptions used in the virtual
reconstruction. It was important to represent them in a format so that we could easily
perform analyses, querying and reasoning, applying the transparency and testability
criteria [13]. The first step consists of the representation of reconstruction hypotheses
and the associated metadata as RDF. The full list of information of voussoirs and slots
is available in textual format [1, 2, 3]. We represent essential information about the
proposed locations of the voussoirs along with the arguments for this hypothesis (Tables 1
and 2).
Prolog is a versatile declarative programming language that allows a set of facts
and rules (e.g., the details of a voussoir or the details of a hypothesis) and rules (e.g.,
linking hypotheses to voussoirs and proposed slots), which define relations [42]. Prolog
has been chosen, considering its declarative nature and as a useful language for quick
prototyping and short iterative tests on the data. It allows inference and reasoning,
data validation and verification. The dataset has been transformed as facts and rules
No. Predicates in Prolog Description
1. 𝑣𝑜𝑢𝑠𝑠𝑜𝑖𝑟(𝑁, 𝑊, 𝑋, 𝑌, 𝑍, 𝐹 ) information related to a voussoir: N, its identifier, W,
the width, F, the face where the crossmark is present
(up or down), the center of gravity given by X,Y,Z.
2. 𝑠𝑙𝑜𝑡(𝑅, 𝑋, 𝑌, 𝑍) location of the slot with R its identifier and X,Y,Z the
center of gravity (useful for visualization).
3. ℎ𝑦𝑝𝑜𝑡ℎ𝑒𝑠𝑖𝑠(𝐻, 𝑅, 𝑁, 𝐶) details of a hypothesisl H linking the voussoir N to the
slot R, with the comment C.
Table 1
Some key reconstruction predicates that were used for querying and inference.
in Prolog (Table 1) to explicitly express the argumentative relations between voussoirs,
slots, and hypotheses. We could then apply the testability criteria on the counterfactual
reasoning of the reconstruction to verify whether there are incoherent elements in the
data. Additional information is inferred about the reconstruction reasoning from the
existing data: for any voussoir, we know the sequence of hypotheses that a voussoir went
through. Instead of tracking events (temporal data) of the reconstruction work sessions,
we are more interested in modeling the hypotheses about past states (Hypothesis [N:1-6]
in Figure 2). To say it differently each voussoir can tell its own story in the reconstruction.
As in [19], another facet of the reconstruction reasoning is inferred: the transition (as
event) between two versions (as stabilized state of an hypothesis) are made explicit, e.g.,
when a voussoir was deleted, or replaced in a given slot or when a new voussoir was
replaced in a slot. A sequence of hypotheses and hypothesis transitions can be considered
as an overall story, i.e., one possible narration of the reconstruction. We move from the
known documented state to the implied events that lead to it.
5. Evaluation using competency questions and Prolog queries
The objective here is to go beyond commonly found linear temporal visualization, where a
temporal bar shows the state of an object at different time spans. Instead, archaeologists
and subject experts want to go at a more granular hypothesis versioning level. Their
focus ranges from individual objects (voussoirs) or the overall main structure (arch) or a
combination of both. It opens up parallel narratives about the arch, representing the
virtual reconstruction hypotheses with the argumentation about the authentic voussoirs
as archaeological artifacts. The virtual reconstruction process is not just about the
story before and after the reconstruction, but the narrative in-between. The study of
different possible arguments used in this process serve as lessons for future research.
To evaluate the experiment, a list of queries are formulated as competency questions
(Table 2). For example, the analysis (based on semantic, temporal, spatial elements)
helps in understanding the case of a voussoir staying at the same place across certain
hypotheses or all hypotheses. A subset of the research questions have been formalized as
competency questions and translated using Prolog queries. The Table 2 shows how to
No. Competency questions Query (Prolog)
1. What do we know about all the vous- 𝑣𝑜𝑢𝑠𝑠𝑜𝑖𝑟(𝑁, 𝑊, 𝑋, 𝑌, 𝑍, 𝐹 ).
soirs and their associated information
(N as the identifier, W as the width,
X as . . . )?
2. What do we know about all the recon- 𝑠𝑙𝑜𝑡(𝑅, 𝑊, 𝑋, 𝑌, 𝑍).
struction locations for the voussoirs, ie.
the slots and their related information
(R as the slot identifier, W as the slot
width, . . . )?
3. What are all the hypotheses? ℎ𝑦𝑝𝑜𝑡ℎ𝑒𝑠𝑖𝑠(𝐻, 𝑁, 𝑅, 𝐶).
ℎ𝑦𝑝𝑝𝑎𝑖𝑟(𝐻, 𝑁, 𝑅) : −ℎ𝑦𝑝𝑜𝑡ℎ𝑒𝑠𝑖𝑠(𝐻, 𝑅, 𝑁, _),
4. What are the pairs of voussoirs and
the associated hypotheses? 𝑣𝑜𝑢𝑠𝑠𝑜𝑖𝑟(𝑁, _, _, _, _, _), 𝑠𝑙𝑜𝑡(𝑅, _, _, _, _).
ℎ𝑦𝑝𝑝𝑎𝑖𝑟(𝐻, 𝑁, 𝑅, 𝑊 1, 𝑊 2) : −ℎ𝑦𝑝𝑜𝑡ℎ𝑒𝑠𝑖𝑠(𝐻, 𝑅, 𝑁, _),
5. What are the pairs of voussoirs and
the associated hypotheses and their 𝑣𝑜𝑢𝑠𝑠𝑜𝑖𝑟(𝑁, 𝑊 1, _, _, _, _), 𝑠𝑙𝑜𝑡(𝑅, 𝑊 2, _, _, _).
widths (for verification)
Table 2
Some competency questions to query the reconstruction data using Prolog fact and rules in regards to
the reconstruction question.
use the predicates formatted in Prolog to answer the reconstruction questions. Finally,
the validated data is then visualized in the 3D viewer for visualization and evaluation of
results, as shown in the next section.
6. Results: 3D visualization of the reconstruction narrative and
hypotheses
The reconstruction hypothesis is projected in the real existing space of Notre-Dame de
Paris cathedral. This is the reason why it is equally essential to verify the reconstruction
reasoning using 3D visualization and query-based data validation approaches. Once the
required responses were obtained using Prolog, these were then visualized in the custom
3D nd-Viewer [43], allowing the use of multi-faceted variables in their spatial context
(like semantic information including links to the current or past versions). The objective
is to obtain a virtual equivalent of these results and to visually verify them. The 3D
visualization is a complementary approach to querying. Whereas Prolog gives information
concerning the way that data and causal assumptions are encoded and described, the
3D visualization investigates the relation of digital place making as the relation between
space, time, semantics, and memory. It is one thing to note that the position of a
keystone is never questioned and remains stable between several hypotheses, and another
to see it in context and understand that this stability can be explained by the spatial or
geometric relations linking this keystone to its direct environment. The concurrent states
of the same voussoir in different hypotheses can be highlighted and compared, since the
reconstruction reasoning is revealed in the 3D viewer. Thanks to inferred data about
reconstruction reasoning (state of hypotheses + events of change), the 3D viewer can
animate the voussoirs in regards to the reconstruction reasoning as trajectory. Unlike 3D
visualization with temporal bars, where the different states can be simply visualized, the
goal here was also to obtain a smooth animation of transition of these hypotheses for one
voussoir. Several problems arise: the orientation of the voussoir when it changes the slots
and orientation of the trajectory. For example, in Figure 3, we fix the visualization of
the voussoir in one slot and show its possible trajectory across different hypotheses. We
can see that each point corresponds to the different hypotheses. The visualization also
shows the archaeological information for the voussoir for further validation by subject
experts. It is also possible to glide the same voussoir in different slots across hypotheses.
Figure 3: Visualization of the voussoir N1A trajectory through the different reconstruction hypotheses
(slots successively R63, R17, R58, R16) associated with its archaeological information.
7. Discussion
Virtual reconstruction of Notre-Dame’s arch is a complex question, several rounds of
hypothesis sessions have been conducted since 2020. The data does not represent a finished
process: the ongoing input of new data leads to new arguments that will themselves
enrich further the argumentation of future hypotheses, as shown in figure 2. Because it
is an interdisciplinary research, it ensues that communication is a crucial aspect. We
require different ways to communicate, document, and visualize the evolution of these
hypotheses about the reconstruction. Neither 3D visualization nor formalizing/querying
may not be enough separately. A data spreadsheet, obtained from inference tools, like
Prolog, lacks readability without visualization. 3D visualization, on the contrary, without
data formalization and inference, is missing the depth of reconstruction knowledge.
The combination - formalization, reasoning, 3D visualization - helps to demonstrate
the hypotheses and can also be used to explore new questions that were inaccessible
previously. For example, we can assess whether a high degree of variability between
different hypotheses can still exhibit some form of local stability (e.g., voussoirs whose
position have been much questioned but which are still stuck together as clusters).
The questions on semiotic visualization still remain open: the semiology about 2D
graphics is part of the general culture due to geography and architecture, while the 3D
graphics mostly represent a mimetic relation with the reality. The codes and symbols for
representation in 3D are not yet conveying interpretation and narration, which impacts
the cognitive engagement and interactions from the users. They are yet to be invented
and explored as spatial patterns.
8. Conclusion and Future Works
This article presented the study, representation, and visualization of the hypotheses
elaborated for a virtual reconstruction study, especially after a destruction event like in
Notre-Dame de Paris. A number of past research works on cultural heritage, storytelling,
and urban evolution have previously explored models, methodologies, or ontologies for
narrating linear or concurrent scenarii of evolution. In this article, we presented a different
case, where the goal is not to obtain the past evolution of objects, but to build knowledge
about objects in a hypothetical state. The granularity of argumentation is still to be
explored, i.e., the possibility to find the most granular arguments (or atomic arguments)
upon which further arguments can be built.
We proposed a quick prototyping and short iterative tests on the data for exploration.
Making use of the hypotheses’ data, both logic and visualization tools were tested. It
allowed us to track the different hypotheses, and rediscover the reconstruction problem
under a new light. The combination of formalization and 3D visualization changes the
paradigm for reconstruction scholarship. The need goes beyond data provenance and
reusability. It opens avenues for exploring data intensive collaborative environments.
The experiment nourished our initial intuition that serendipity is a key component for
data visualization. It allows the users to navigate and interact with the 3D objects and
discover some additional characteristics that are inferred via reasoning over the data. The
experiment shows that 3D visualization setup opens the possibility for spatial serendipity
as spatialized query of embedded memory in place.
Acknowledgments
This work was supported by: the projects TEATIME funded by the Mission pour les
Initiatives Transverses et Interdisciplinaires (MITI) of the Centre National de la Recherche
Scientifique (CNRS, France), the ERC Ndame_Heritage funded by ERC advanced Grant
2021, and the project REPERAGE funded by the Fondation des Sciences du Patrimoine
(France). The authors wish to acknowledge the help and collaborative support from: the
chief architects of historical monuments in charge, Philippe Villeneuve, Pascal Prunet and
Rémi Fromont, the Etablissement public chargé de la conservation et de la restauration
de la cathédrale Notre-Dame de Paris (RNDP), and the heritage conservators. The
authors thank the numerous scientific partners and collaborators from the research groups
working on Notre-Dame de Paris cathedral, with special recognition for the Digital Data,
the Stone working groups.
References
[1] C. Moulis, B. Phalip, E. Baillieul, Les claveaux de l’arc doubleau effondre de la nef
de Notre-Dame de Paris, rapport intermediaire, 2020.
[2] E. Baillieul, L. Leroux, D. Chaoui-Derieux, Reconstructing the fallen arch of notre-
dame. new insights into gothic vault-building, Journal of Cultural Heritage 65 (2024)
68–73. URL: https://www.sciencedirect.com/science/article/pii/S1296207423000638.
doi:https://doi.org/10.1016/j.culher.2023.05.010, Notre-Dame de Paris : a
multidisciplinary scientific site.
[3] A. Gros, A. Guillem, L. De Luca, E. Baillieul, B. Duvocelle, O. Malavergne, L. Leroux,
T. Zimmer, Faceting the post-disaster built heritage reconstruction process within
the digital twin framework for Notre-Dame de Paris, Scientific Reports 13 (2023)
5981. URL: https://www.nature.com/articles/s41598-023-32504-9. doi:10.1038/
s41598-023-32504-9.
[4] A. Guillem, A. Gros, V. Abergel, L. De Luca, Reconstruction beyond representation
in Notre-Dame de Paris, in: 2023 IMEKO TC-4 International Conference on
Metrology for Archaeology and Cultural Heritage, 2023.
[5] A. Guillem, A. Gros, La restitution hybride de l’arc doubleau effondré dans la nef
de la cathédrale Notre-Dame de Paris., in: Monumental, 2023.
[6] A. Guillem, A. Gros, L. Deluca, Faire parler les claveaux effondrés de la cathédrale
Notre-Dame de Paris, in: Humanistica 2023, Données et patrimoine, Association
francophone des humanités numériques, Genève, Switzerland, 2023. URL: https:
//hal.science/hal-04106101.
[7] J. T. Clark, The fallacy of reconstruction, Cyber-archaeology 2177 (2010) 63–73.
[8] D. Pletinckx, An epoch common infrastructure tool for interpretation management,
Paradata and Transparency in Virtual Heritage (2008).
[9] G. Bruseker, A. Guillem, N. Carboni, Semantically Documenting Virtual Re-
construction: Building a Path to Knowledge Provenance, ISPRS Annals of
the Photogrammetry, Remote Sensing and Spatial Information Sciences II-5/W3
(2015) 33–40. URL: https://isprs-annals.copernicus.org/articles/II-5-W3/33/2015/.
doi:10.5194/isprsannals-II-5-W3-33-2015.
[10] A. Guillem, R. Zarnic, G. Bruseker, Building an argumentation platform for 3D
reconstruction using CIDOC-CRM and Drupal, in: 2015 Digital Heritage, volume 2,
2015, pp. 383–386. URL: https://ieeexplore.ieee.org/abstract/document/7419529/.
doi:10.1109/DigitalHeritage.2015.7419529.
[11] E. Demetrescu, D. Ferdani, From Field Archaeology to Virtual Reconstruction: A
Five Steps Method Using the Extended Matrix, Applied Sciences 11 (2021) 5206.
URL: https://www.mdpi.com/2076-3417/11/11/5206. doi:10.3390/app11115206.
[12] G. Fauconnier, M. Turner, The way we think: Conceptual blending and the mind’s
hidden complexities, Basic books, 2008.
[13] J. Pearl, The seven tools of causal inference, with reflections on machine learning,
Communications of the ACM 62 (2019) 54–60. URL: https://dl.acm.org/doi/10.
1145/3241036. doi:10.1145/3241036.
[14] P. Nora, Between Memory and History: Les Lieux de Mémoire, Representations
(1989) 7–24. URL: https://www.jstor.org/stable/2928520. doi:10.2307/2928520.
[15] G. Styliaras, D. Koukopoulos, F. Lazarinis (Eds.), Handbook of Research on
Technologies and Cultural Heritage: Applications and Environments, IGI Global,
2011. URL: http://services.igi-global.com/resolvedoi/resolve.aspx?doi=10.4018/
978-1-60960-044-0. doi:10.4018/978-1-60960-044-0.
[16] P. Lisena, D. Schwabe, M. van Erp, I. Leemans, W. Tullett, R. Troncy, S. C. Ehrich,
T. Ehrhart, An ontology for creating hypermedia stories over knowledge graphs, in:
A. Bikakis, R. Ferrario, S. Jean, B. Markhoff, A. Mosca, M. N. Asmundo (Eds.),
Proceedings of the International Workshop on Semantic Web and Ontology Design for
Cultural Heritage co-located with the International Semantic Web Conference 2023
(ISWC 2023), Athens, Greece, November 7, 2023, volume 3540 of CEUR Workshop
Proceedings, CEUR-WS.org, 2023. URL: https://ceur-ws.org/Vol-3540/paper5.pdf.
[17] D. Ringas, E. Christopoulou, M. Stefanidakis, Urban memory in space and time,
in: Handbook of research on technologies and cultural heritage: applications and
environments, IGI Global, 2011, pp. 325–340.
[18] K. Chaturvedi, C. S. Smyth, G. Gesquière, T. Kutzner, T. H. Kolbe, Managing
Versions and History Within Semantic 3D City Models for the Next Generation of
CityGML, Springer International Publishing, Cham, 2017, pp. 191–206. URL: https:
//doi.org/10.1007/978-3-319-25691-7_11. doi:10.1007/978-3-319-25691-7_11.
[19] J. Samuel, S. Servigne, G. Gesquière, Representation of concurrent points of view of
urban changes for city models, Journal of Geographical Systems (2020). URL: http://
link.springer.com/10.1007/s10109-020-00319-1. doi:10.1007/s10109-020-00319-1.
[20] S. Stead, A New Approach to Interoperable Argumentation Documentation, in:
C. Gonzalez-Perez, P. Martin-Rodilla, M. Pereira-Fariña (Eds.), Discourse and Ar-
gumentation in Archaeology: Conceptual and Computational Approaches, Springer
International Publishing, Cham, 2023, pp. 29–36. URL: https://link.springer.com/
10.1007/978-3-031-37156-1_2. doi:10.1007/978-3-031-37156-1_2.
[21] K. Al-Khatib, A. de Waard, D. Freitag, I. Gurevych, Y. Hou, H. Scells, Towards
a Unified Model of Scholarly Argumentation (Dagstuhl Seminar 22432), Technical
Report, [object Object], 2023. URL: https://drops.dagstuhl.de/entities/document/
10.4230/DagRep.12.10.175. doi:10.4230/DAGREP.12.10.175.
[22] J. Samuel, S. Servigne, G. Gesquière, UrbanCo2Fab: Comprehension of concur-
rent viewpoints of urban fabric based on GIT, ISPRS Annals of the Photogram-
metry, Remote Sensing and Spatial Information Sciences IV-4/W6 (2018) 65–72.
URL: https://isprs-annals.copernicus.org/articles/IV-4-W6/65/2018/. doi:10.5194/
isprs-annals-IV-4-W6-65-2018.
[23] N. Baloian, G. Zurita, J. A. Pino, S. Peñafiel, W. Luther, Developing Hyper-stories
in the Context of Cultural Heritage Appreciation, in: H. Nakanishi, H. Egi, I.-A.
Chounta, H. Takada, S. Ichimura, U. Hoppe (Eds.), Collaboration Technologies
and Social Computing, volume 11677, Springer International Publishing, Cham,
2019, pp. 110–128. URL: http://link.springer.com/10.1007/978-3-030-28011-6_8.
doi:10.1007/978-3-030-28011-6_8.
[24] I. Dudek, J.-Y. Blaise, Diachrograms – a theoretical framework for the modelling
and analysis of a heritage artefact’s diachronic evolution., 2020. URL: https://shs.
hal.science/halshs-03070375.
[25] C. Metral, R. Billen, A.-F. Cutting-Decelle, M. Van Ruymbeke, Ontology-based
approaches for improving the interoperability between 3D urban models, Journal
of Information Technology in Construction 15 (2010). URL: https://orbi.uliege.be/
handle/2268/26716.
[26] A. F. Khan, A. Bellandi, G. Benotto, F. Frontini, E. Giovannetti, M. Reboul,
Leveraging a Narrative Ontology to Query a Literary Text (2016) 10 pages, 1548145
bytes. URL: https://drops.dagstuhl.de/entities/document/10.4230/OASIcs.CMN.
2016.10. doi:10.4230/OASICS.CMN.2016.10.
[27] J. Yeh, StoryTeller: An event-based story ontology composition system for bio-
graphical history, in: 2017 International Conference on Applied System Innovation
(ICASI), IEEE, Sapporo, Japan, 2017, pp. 1934–1937. URL: http://ieeexplore.ieee.
org/document/7988448/. doi:10.1109/ICASI.2017.7988448.
[28] P. Rissen, H. Lippell, M. Chadburn, T. Leitch, D. Brickley, M. Smethurst,
S. Cevey, Storyline Ontology, 2013. URL: https://www.bbc.co.uk/ontologies/
storyline-ontology/bbc.com/ontologies/storyline-ontology/.
[29] R. Piryani, N. Aussenac-Gilles, N. J. Hernandez, Comprehensive Survey on On-
tologies about Event, in: ESWC Workshops on Semantic Methods for Events and
Stories (SEMMES @ ESWC 2023), volume 3443, Pasquale Lisena and Ilaria Tiddi
and Simon Gottschalk and Luc Steels, ceur-ws.org, Hersonissos, Greece, 2023, pp.
1–15. URL: https://hal.science/hal-04283673.
[30] D. Calvanese, P. Liuzzo, A. Mosca, J. Remesal, M. Rezk, G. Rull, Ontology-
based data integration in EPNet: Production and distribution of food during the
Roman Empire, Engineering Applications of Artificial Intelligence 51 (2016) 212–
229. URL: https://linkinghub.elsevier.com/retrieve/pii/S0952197616000099. doi:10.
1016/j.engappai.2016.01.005.
[31] C. Meghini, R. Scopigno, J. Richards, H. Wright, G. Geser, S. Cuy, J. Fihn, B. Fanini,
H. Hollander, F. Niccolucci, A. Felicetti, P. Ronzino, F. Nurra, C. Papatheodorou,
D. Gavrilis, M. Theodoridou, M. Doerr, D. Tudhope, C. Binding, A. Vlachidis,
ARIADNE: A Research Infrastructure for Archaeology, Journal on Computing and
Cultural Heritage 10 (2017) 1–27. URL: https://dl.acm.org/doi/10.1145/3064527.
doi:10.1145/3064527.
[32] C. Niang, C. Marinica, B. Markhoff, E. Leboucher, O. Malavergne, L. Bouiller,
C. Darrieumerlou, F. Laissus, Supporting Semantic Interoperability in Conservation-
Restoration Domain: The PARCOURS Project, Journal on Computing and Cultural
Heritage 10 (2017) 1–20. URL: https://dl.acm.org/doi/10.1145/3097571. doi:10.
1145/3097571.
[33] C.-H. Yu, J. Hunter, Documenting and sharing comparative analyses of 3D digital
museum artifacts through semantic web annotations, Journal on Computing and
Cultural Heritage 6 (2013) 1–20. URL: https://dl.acm.org/doi/10.1145/2532630.
2532634. doi:10.1145/2532630.2532634.
[34] W. Shi, E. Kotoula, K. Akoglu, Y. Yang, H. Rushmeier, CHER-Ob: A Tool for Shared
Analysis in Cultural Heritage, Eurographics Workshop on Graphics and Cultural
Heritage (2016) 4 pages. URL: https://diglib.eg.org/handle/10.2312/gch20161404.
doi:10.2312/GCH.20161404.
[35] F. Soler, F. J. Melero, M. V. Luzón, A complete 3D information system for cultural
heritage documentation, Journal of Cultural Heritage 23 (2017) 49–57. URL: https:
//linkinghub.elsevier.com/retrieve/pii/S1296207416302576. doi:10.1016/j.culher.
2016.09.008.
[36] F. I. Apollonio, V. Basilissi, M. Callieri, M. Dellepiane, M. Gaiani, F. Ponchio,
F. Rizzo, A. R. Rubino, R. Scopigno, G. Sobra’, A 3D-centered information
system for the documentation of a complex restoration intervention, Journal of
Cultural Heritage 29 (2018) 89–99. URL: https://linkinghub.elsevier.com/retrieve/
pii/S1296207417301905. doi:10.1016/j.culher.2017.07.010.
[37] M. Potenziani, M. Callieri, M. Dellepiane, M. Corsini, F. Ponchio, R. Scopigno,
3DHOP: 3D Heritage Online Presenter, Computers & Graphics 52 (2015) 129–
141. URL: https://linkinghub.elsevier.com/retrieve/pii/S0097849315001041. doi:10.
1016/j.cag.2015.07.001.
[38] V. Picavet, M. Brédif, M. Konini, A. Devaux, iTowns, framework web pour la
donnée géographique 3D, Revue XYZ 147 (2016) 49–52. URL: https://hal.science/
hal-02551483.
[39] M. Schütz, S. Ohrhallinger, M. Wimmer, Fast Out-of-Core Octree Generation
for Massive Point Clouds, Computer Graphics Forum 39 (2020) 155–167. URL:
https://onlinelibrary.wiley.com/doi/10.1111/cgf.14134. doi:10.1111/cgf.14134.
[40] C. Stefani, L. De Luca, P. Véron, M. Florenzano, Time indeterminacy and spatio-
temporal building transformations: an approach for architectural heritage under-
standing, International Journal on Interactive Design and Manufacturing (IJI-
DeM) 4 (2010) 61–74. URL: http://link.springer.com/10.1007/s12008-009-0085-5.
doi:10.1007/s12008-009-0085-5.
[41] V. Jaillot, V. Rigolle, S. Servigne, J. Samuel, G. Gesquière, Integrating multimedia
documents and time-evolving 3D city models for web visualization and navigation,
Transactions in GIS 25 (2021) 1419–1438. URL: https://onlinelibrary.wiley.com/doi/
10.1111/tgis.12734. doi:10.1111/tgis.12734.
[42] A. Colmerauer, An introduction to Prolog III, Communications of the ACM 33
(1990) 69–90.
[43] V. Abergel, 3D Environment for Interactive Visualization for Notre-Dame de Paris’
Scientific Research, 2024. URL: https://nd.aioli.map.cnrs.fr/.