Virtual reconstruction should go beyond merely presenting images and 3D models by documenting the scientific context and reasoning underlying the reconstruction process, rather than just showcasing the final product. For instance, the collapsed transverse arch in the nave of Notre-Dame de Paris serves as a case study to demonstrate an interdisciplinary reconstruction workflow. By letting the voussoirs speak, we mean that the materiality of the voussoirs and immateriality of digital surrogates are the support to make explicit the argumentation of the reconstruction. The 3D visualization moves away from a static and finalized illustrative output of the reconstruction study and toward an open and dynamic visualization of reconstruction data. The data explicitly records both factual information on the physical and digital objects, as well as the counterfactual propositions of the reasoning of reconstruction hypotheses. The proposed experiment is twofold: (1) setting up of the 3D dataset of the arch reconstruction with archaeological argumentation where the hypotheses are modeled as versions, and (2) evaluation of the scientific narrative of reconstruction argumentation through both a custom 3D visualization and competency questions on the enriched 3D data combining hypotheses and arguments. The humanistic question of reconstruction is the starting point for a nonlinear scientific narrative composed of hypotheses and argument loops. We consider the conflicting interpretations of the voussoirs and the knowledge encapsulated in relation to the spatial configuration of the arch. The results of the queries and 3D visualization are interdependent: they demonstrate that hypothetical reasoning facets of the arch reconstruction are embedded in the spacetime volumes of the voussoirs.
Virtual reconstructions have the amazing power to bring a past to life. A virtual reconstruction is a common exercise for the disciplines interested in cultural history to represent some aspect of a past reality. However, the resulting reconstruction images and 3D visualizations conceal the underlying scientific process and hypothetical reasoning that contributed to their production. In this article, we tackle the gap in the literature between the knowledge that is produced in a reconstruction study and actual reconstruction data that one can either query or visualize. In the context of Notre-Dame de Paris, the fallen transverse arch in the nave provides a case study (Figure 1), where the challenge is to determine the positioning of its voussoirs before its destruction. The 3D visualization prototyping and querying bring to the foreground the hypotheses and argumentation, where the challenge is to determine the positioning of its voussoirs before its destruction. Given the fragmentary documentation available, there are several plausible solutions to reconstruct this arch, some more valid than others. We then need to present the reasoning behind these choices. Such a case study allows for testing a data workflow on the reconstruction argumentation reasoning.
The question of virtual reconstruction can be defined as a scientific study that aims at formulating an argumentation linking three elements: (1) the material objects (voussoirs, arch), (2) the digital objects (pointclouds, digital photographs, datasets), and (3) expert knowledge (archaeology, art history, architecture, computer science etc). These elements are tied together with the hypothetical scientific reasoning about reconstruction question. In this work, we posit that the 3D visualization and querying are constitutive research tools for the virtual reconstruction research, rather than an end result medium for dissemination. By letting the voussoirs speak, we mean that the materiality of the voussoirs and immateriality of digital surrogates are the support to make explicit the argumentation of the reconstruction. The virtual reconstruction research is rooted both in the real (material and immaterial object) and the unreal (hypotheses formulation), where the scientific inquiry about the past can be developed. Then the challenge is about reasoning both about factual knowledge and counterfactual propositions: a reconstruction formulates a hypothesis about something that might have existed based on the existing presence of objects that are surviving witnesses of this lost past.
This contribution proposes a twofold experiment. Firstly, it sets up the 3D dataset of
Notre-Dame’s de Paris’ transverse arch reconstruction, including the data enrichment
with archaeological knowledge proposed by archaeologists and subject experts. These
requirements were detailed in previous studies [
Do the 3D visualization and querying have this ability to explicitly convey the reconstruction study process and arguments? To answer positively, the visualization needs to move away from a static and finalized illustrative output of the reconstruction study and toward an open, dynamic visualization of reconstruction data. Following the principles of open science, the proposed data practice (i.e., enrichment, transformation, inference etc.) can serve as a lever for the scientific narrative and the argumentation. Beyond the factual level, we consider the interpretation of results and the counterfactual knowledge encapsulated in relation to its spatial configuration. The interpretation of results and 3D visualization become interdependent and complementary. Whereas 3D visualization tools typically represent 3 dimensional objects + 1 dimension for time, this experiment aims to encompass counterfactual information (hypothetical reasoning of the reconstruction) in relation to spatial (3D) and temporal information.
In section 2, we present the existing gap in documenting counterfactual reasoning of virtual reconstruction. The scientific inquiry over reconstruction leads to formalizing concurrent points of view about space, place, and memory, that bring the theoretical concept of place making. Querying and visualizing reconstruction data opens the problem of representing factual/counterfactual and spatio-temporal data. In section 3, we develop the experiment’s methodology: the reconstruction hypotheses are considered as versions or knowledge steps. Subsequently, in section 4, we detail the implementation the counterfactual layer in the reconstruction data using Prolog. In sections 5 and 6, the complementarity of the 3D visualization and querying for the understanding of the reconstruction reasoning is demonstrated: in section 5, the resulting transformed data is evaluated with the competency questions focusing on the counterfactual elements, while in section 6, the 3D visualization shows how the hypothetical reasoning of the arch reconstruction is embedded in the spacetime volumes of the voussoirs.
In this part, we present the gap in the literature between the knowledge about the
past in a reconstruction study and the actual reconstruction data. The problem of
documenting virtual reconstruction has roots in philosophy of language, semantics, and
logic. For Clark [
For memorizing the reconstruction reasoning, the problem can be formulated as follows:
the reconstruction study (or model) formulates implicitly a knowledge object that is
in-between real objects and counterfactual propositions. Making the voussoirs speak, we
aim at unfolding and explicitly formulating these intricate counterfactual blends. The
goal is the explanation of the proposition sets, arguments, and inference logic that are
used in the argumentation of the reconstruction [
At the urban scale, memories may be available in several forms [17], media documents forming a key form of recording. The documents narrating the historical events and their chronology frequently serve as evidence for understanding the incremental changes that happened to urban and architectural objects. Historical documents and archives have been used with their 2D/3D representations [18, 19] to represent concurrent points of view of urban evolution capable of narrating numerous possible scenarii of city changes at diferent levels of detail. Numerous ontologies and documentation standards have been proposed to represent (historical) episodes, events (CIDOC CRM as event-centric ontology), stories, historical narratives [16], and argumentation using CIDOC CRMinf [20] or other argumentation models [21].
With the models and ontologies in place, a methodological approach based on versioning
is used to store and visualize historical narratives about cultural heritage artifacts at
diferent scales. Extending the use of version control systems like Git, Samuel et al. [22]
propose to store multiple scenarios of urban evolution as city versions. Hypermedia stories
- graph-structured stories with navigation links - [23, 16] uses media along with these
narratives to better explain the modification of the urban fabric and cultural heritage
assets. At the architectural scale, the evolution of objects is apprehended through
diagrammatic visualizations called diachrogram to capture the changes and periods of
stability in building lives [24]. Metral et al. [25], Bruseker et al. [
In this article, we want to go beyond the use case of representing and visualizing urban evolution, as well as the linear [26, 27] or the graph-structured stories [28, 16] of a series of events [29]. We propose the extension of some of the above works to represent the hypotheses evolution during the reconstruction process. We build upon the studies on historical changes to model the hypotheses associated with virtual reconstruction, where the goal is to ultimately bring back a damaged object to its hypothetical lost state.
For the visualization in heritage field, one observes a duality between semantic-oriented approaches and 3D visualization-oriented approaches. On one hand, the approaches centered on knowledge modeling mainly present information systems designed to manage documentation activities, without directly exploiting the full wealth of heterogeneous data mobilized in heritage contexts, in particular 3D data [30, 31, 32]. On the other hand, approaches based on the exploitation of 3D digital data, focus mostly on their description, omitting any reference to the complex relationships specific to the observation, interpretation, or documentation processes [33, 34]. Some works, such as Soler et al. [35], Apollonio et al. [36], clearly demonstrate the strong interest in bringing these two approaches together, making it possible to spatialize and query heterogeneous information in the context of restoration operations. However, these initiatives are generally built around isolated 3D models, excluding the possibility of handling and confronting the diversity of resources involved in heritage studies, like Notre-Dame scientific action.
From a software point of view, there are a number of very efective libraries dedicated to the manipulation of 3D data, which can be used for reconstruction studies [37, 38, 39]. But these tools focus on visualization and interaction modalities, allowing only in the best cases to display provenance metadata, and in most cases ignoring the full potential of an upstream work on data curation and conceptual modeling. In [40], spatio-temporal information of 3D data is characterized describing the transformations afecting buildings over time and considering their changes (demolition, amplification, division, or change of function). It is based on the addition of a semantic layer to the geometric restitution through description graphs, enabling information on buildings to be enriched according to existing documentary sources. While this approach is of considerable interest, it remains ill-suited to virtual reconstruction data documenting the scientific context and reasoning about the reconstruction process. The temporal dimension addressed in [40] is linear, making it dificult to semantically and geometrically characterize competing hypotheses and their underlying arguments. Samuel et al. [19] and Jaillot et al. [41] propose knowledge versioning in 3D urban contexts. While this approach is interesting in terms of data structuring, all the narrative potential remains to be explored.
In this section, we consider the example of the arch of Notre-Dame de Paris that was
damaged during the 2019-fire. The voussoirs that together formed the arch fell on
the nave’s floor and were displaced. The scientific research and restoration activities
have divergent objectives and interpretations, while working on the same object. The
virtual reconstruction study aims at gathering knowledge about the destroyed arch and
understanding which original place or slot was for each authentic voussoir in the arch
before the fire [
In order to restore the arch to its hypothetical original state, it is critical to understand that the arch has two sides, the northern and southern sides. A number of 122 voussoirs together form the whole arch with 40 voussoirs still in place and 77 recovered voussoirs. The (possible) slots for the fallen voussoirs have been identified and numbered, using survey documentation prior to the fire. The slots on the south side are numbered with even numbers, while those on the north are numbered with odd numbers, starting with R0 for the keystone. Thus, it is easy to understand that R12 is the 6th slot on the north side of the arch. The fallen voussoirs have also been identified and inventoried N1,. . . , etc. The goal of reconstruction is to assign a slot (values of n are between 0 and 121) to every voussoir (values of k are between 0 and 121). There are multiple collaborative work sessions between researchers (archaeologists, architects, art historians, etc.) to produce such possible reconstruction hypotheses. Each hypothesis has its associated arguments.
In Figure 2, the arrows between the two versions may have diferent significance for the reconstruction reasoning. A decision about a voussoir based on a particular argument may remain valid across diferent versions. However, this also means that any initial hypotheses, if proven false at a later date may lead to the falsification of some pairs of slotvoussoirs from another hypothesis. Take, for example, the evolution of the argumentation process related to cross-shaped marks on the joint sides of the voussoirs. These lapidary marks constitute an archaeological predicate for the arch reconstruction. The marks were believed to possibly serve as guide the masons to orient the positioning of the voussoir during the construction. The mark could indicate either the downward or upward side. In a first hypothesis the voussoirs were oriented with their lapidary mark facing downward. This was later proved to be incorrect. This meant that we had to relook all the previous use of these arguments and make corrections to the assigned facts: slot-voussoir pairs, voussoirs, if necessary. This makes the problem particularly challenging since we need to record this aspect for every voussoir and the assigned slot along with the associated arguments in the diferent hypotheses. In the reconstruction data, we have a selection of relevant information about the physical objects (voussoirs) and possible locations (slots) in the state of the arch before destruction. The dificulty lies in the reasoning about the characteristics about the real objects in relation to the argumentation hypothesis of the reconstruction, i.e., counterfactual reasoning. The reconstruction data is an open-ended hypothesis, as shown in Figure 2.
After exploring the initial reconstruction dataset available in [
Prolog is a versatile declarative programming language that allows a set of facts and rules (e.g., the details of a voussoir or the details of a hypothesis) and rules (e.g., linking hypotheses to voussoirs and proposed slots), which define relations [ 42]. Prolog has been chosen, considering its declarative nature and as a useful language for quick prototyping and short iterative tests on the data. It allows inference and reasoning, data validation and verification. The dataset has been transformed as facts and rules No. Predicates in Prolog 1. (, , , , ,
Description ) information related to a voussoir: N, its identifier, W, the width, F, the face where the crossmark is present (up or down), the center of gravity given by X,Y,Z. location of the slot with R its identifier and X,Y,Z the center of gravity (useful for visualization). details of a hypothesisl H linking the voussoir N to the slot R, with the comment C. in Prolog (Table 1) to explicitly express the argumentative relations between voussoirs, slots, and hypotheses. We could then apply the testability criteria on the counterfactual reasoning of the reconstruction to verify whether there are incoherent elements in the data. Additional information is inferred about the reconstruction reasoning from the existing data: for any voussoir, we know the sequence of hypotheses that a voussoir went through. Instead of tracking events (temporal data) of the reconstruction work sessions, we are more interested in modeling the hypotheses about past states (Hypothesis [N:1-6] in Figure 2). To say it diferently each voussoir can tell its own story in the reconstruction. As in [19], another facet of the reconstruction reasoning is inferred: the transition (as event) between two versions (as stabilized state of an hypothesis) are made explicit, e.g., when a voussoir was deleted, or replaced in a given slot or when a new voussoir was replaced in a slot. A sequence of hypotheses and hypothesis transitions can be considered as an overall story, i.e., one possible narration of the reconstruction. We move from the known documented state to the implied events that lead to it.
The objective here is to go beyond commonly found linear temporal visualization, where a temporal bar shows the state of an object at diferent time spans. Instead, archaeologists and subject experts want to go at a more granular hypothesis versioning level. Their focus ranges from individual objects (voussoirs) or the overall main structure (arch) or a combination of both. It opens up parallel narratives about the arch, representing the virtual reconstruction hypotheses with the argumentation about the authentic voussoirs as archaeological artifacts. The virtual reconstruction process is not just about the story before and after the reconstruction, but the narrative in-between. The study of diferent possible arguments used in this process serve as lessons for future research. To evaluate the experiment, a list of queries are formulated as competency questions (Table 2). For example, the analysis (based on semantic, temporal, spatial elements) helps in understanding the case of a voussoir staying at the same place across certain hypotheses or all hypotheses. A subset of the research questions have been formalized as competency questions and translated using Prolog queries. The Table 2 shows how to Query (Prolog) (, , , , ,
). (, , , ,
).
(, , , ). (, , ) : −ℎ (, _, _, _, _, _),
(, , , _), (, _, _, _, _). (, , , (, 1, 2) : −ℎ 1, _, _, _, _), (, (, , , _), 2, _, _, _).
No. Competency questions 1. What do we know about all the vous- soirs and their associated information (N as the identifier, W as the width,
X as . . . )? 2. What do we know about all the recon- struction locations for the voussoirs, ie. the slots and their related information (R as the slot identifier, W as the slot width, . . . )? 3. What are all the hypotheses?
What are the pairs of voussoirs and the associated hypotheses? What are the pairs of voussoirs and the associated hypotheses and their widths (for verification) ℎ ℎ ℎ use the predicates formatted in Prolog to answer the reconstruction questions. Finally, the validated data is then visualized in the 3D viewer for visualization and evaluation of results, as shown in the next section.
The reconstruction hypothesis is projected in the real existing space of Notre-Dame de Paris cathedral. This is the reason why it is equally essential to verify the reconstruction reasoning using 3D visualization and query-based data validation approaches. Once the required responses were obtained using Prolog, these were then visualized in the custom 3D nd-Viewer [43], allowing the use of multi-faceted variables in their spatial context (like semantic information including links to the current or past versions). The objective is to obtain a virtual equivalent of these results and to visually verify them. The 3D visualization is a complementary approach to querying. Whereas Prolog gives information concerning the way that data and causal assumptions are encoded and described, the 3D visualization investigates the relation of digital place making as the relation between space, time, semantics, and memory. It is one thing to note that the position of a keystone is never questioned and remains stable between several hypotheses, and another to see it in context and understand that this stability can be explained by the spatial or geometric relations linking this keystone to its direct environment. The concurrent states of the same voussoir in diferent hypotheses can be highlighted and compared, since the reconstruction reasoning is revealed in the 3D viewer. Thanks to inferred data about reconstruction reasoning (state of hypotheses + events of change), the 3D viewer can animate the voussoirs in regards to the reconstruction reasoning as trajectory. Unlike 3D visualization with temporal bars, where the diferent states can be simply visualized, the goal here was also to obtain a smooth animation of transition of these hypotheses for one voussoir. Several problems arise: the orientation of the voussoir when it changes the slots and orientation of the trajectory. For example, in Figure 3, we fix the visualization of the voussoir in one slot and show its possible trajectory across diferent hypotheses. We can see that each point corresponds to the diferent hypotheses. The visualization also shows the archaeological information for the voussoir for further validation by subject experts. It is also possible to glide the same voussoir in diferent slots across hypotheses.
Virtual reconstruction of Notre-Dame’s arch is a complex question, several rounds of hypothesis sessions have been conducted since 2020. The data does not represent a finished process: the ongoing input of new data leads to new arguments that will themselves enrich further the argumentation of future hypotheses, as shown in figure 2. Because it is an interdisciplinary research, it ensues that communication is a crucial aspect. We require diferent ways to communicate, document, and visualize the evolution of these hypotheses about the reconstruction. Neither 3D visualization nor formalizing/querying may not be enough separately. A data spreadsheet, obtained from inference tools, like Prolog, lacks readability without visualization. 3D visualization, on the contrary, without data formalization and inference, is missing the depth of reconstruction knowledge. The combination - formalization, reasoning, 3D visualization - helps to demonstrate the hypotheses and can also be used to explore new questions that were inaccessible previously. For example, we can assess whether a high degree of variability between diferent hypotheses can still exhibit some form of local stability (e.g., voussoirs whose position have been much questioned but which are still stuck together as clusters).
The questions on semiotic visualization still remain open: the semiology about 2D graphics is part of the general culture due to geography and architecture, while the 3D graphics mostly represent a mimetic relation with the reality. The codes and symbols for representation in 3D are not yet conveying interpretation and narration, which impacts the cognitive engagement and interactions from the users. They are yet to be invented and explored as spatial patterns.
This article presented the study, representation, and visualization of the hypotheses elaborated for a virtual reconstruction study, especially after a destruction event like in Notre-Dame de Paris. A number of past research works on cultural heritage, storytelling, and urban evolution have previously explored models, methodologies, or ontologies for narrating linear or concurrent scenarii of evolution. In this article, we presented a diferent case, where the goal is not to obtain the past evolution of objects, but to build knowledge about objects in a hypothetical state. The granularity of argumentation is still to be explored, i.e., the possibility to find the most granular arguments (or atomic arguments) upon which further arguments can be built.
We proposed a quick prototyping and short iterative tests on the data for exploration. Making use of the hypotheses’ data, both logic and visualization tools were tested. It allowed us to track the diferent hypotheses, and rediscover the reconstruction problem under a new light. The combination of formalization and 3D visualization changes the paradigm for reconstruction scholarship. The need goes beyond data provenance and reusability. It opens avenues for exploring data intensive collaborative environments. The experiment nourished our initial intuition that serendipity is a key component for data visualization. It allows the users to navigate and interact with the 3D objects and discover some additional characteristics that are inferred via reasoning over the data. The experiment shows that 3D visualization setup opens the possibility for spatial serendipity as spatialized query of embedded memory in place.
This work was supported by: the projects TEATIME funded by the Mission pour les Initiatives Transverses et Interdisciplinaires (MITI) of the Centre National de la Recherche Scientifique (CNRS, France), the ERC Ndame_Heritage funded by ERC advanced Grant 2021, and the project REPERAGE funded by the Fondation des Sciences du Patrimoine (France). The authors wish to acknowledge the help and collaborative support from: the chief architects of historical monuments in charge, Philippe Villeneuve, Pascal Prunet and Rémi Fromont, the Etablissement public chargé de la conservation et de la restauration de la cathédrale Notre-Dame de Paris (RNDP), and the heritage conservators. The authors thank the numerous scientific partners and collaborators from the research groups working on Notre-Dame de Paris cathedral, with special recognition for the Digital Data, the Stone working groups.