Cultural heritage has benefited for years from the availability of technology in the domain of digitalization; hence digital heritage emerged. Researchers in the cultural heritage domain have used tools and digital techniques as way to preserve historical and religious buildings so that they are everlasting in time. These are mostly viewed as autonomous attempts, rarely organized. One of the digital tools that arose from the ifeld of product life cycle management is the digital twin, which is defined as digital representation of physical product. There is an ongoing debate whether cultural heritage can be fully viewed in terms of digital twin and if the application of the digital twin concept can be sustainable in the management of the cultural heritage environment. This paper aims to address the role of the digital twin within the cultural heritage domain and if it can be used to recreate certain phenomena or environmental situation resulting in reducing deterioration over time. This is important since heritage sites and historical buildings must be preserved for future generations.
Heritage preservation is well-known as an activity that takes time and efort. A lot of work is
done on a daily basis to protect built heritage from deteriorating and to keep it as authentic
as possible. Over the previous decade, built heritage documentation, study, and preservation
have gotten more technologically advanced, resulting in a greater diversity and complexity of
information sources. Data acquisition (Drones, Electronic Distance Measurement (EDM), 3D
scanning, Global Positioning System (GPS), Aerial Terrestrial Photogrammetry, etc.), data
structuration (Computer Aided Design (CAD), Building Information Modeling (BIM), Geographic
Information System (GIS), etc.), and data dissemination (CAD, BIM, GIS, etc.) technologies have
aided in the creation of digital representations of built heritage, allowing for more efective
planning, predictive maintenance, and strategic management. In other words, heritage
digitization techniques have given not only a visual representation of heritage, but also a technological
VIPERC2022: 1st International Virtual Conference on Visual Pattern Extraction and Recognition for Cultural Heritage
Understanding, 12 September 2022
* Corresponding author.
solution for efective conservation management [
The rise of the Internet of Things (IoT) has enabled the introduction of new supporting tools
for building operation and maintenance. Digital twins are made possible by the emergence of IoT
sensors. Even though the phrase "digital twin" has no uniform definition (it is defined in a variety
of ways by industry and academia), it can be characterized as a virtual representation of what
has been generated [
This paper presents an attempt to examine the role of digital twins in the cultural heritage domain. The emphasis is on using digital twins to recreate specific phenomena or environmental situations in order to reduce deterioration over time. This is of immense importance because heritage sites and historical buildings must be preserved for future generations.
The remainder of this paper is organised as follows. The second section presents the digital twin concept and its enabling technologies. The usage of digital twins in the cultural heritage domain is presented in section 3, while benefits and problems of its implementation are discussed in the fourth section. The last section draws conclusions.
A digital twin is a connected, virtual counterpart of a physical product, asset, or system that
has both the elements and dynamics of the way the complex system runs and evolves over time.
Hence, the digital twin architecture consists of three main components: the physical entity, the
virtual model, and their connection [
• Do: smart algorithms select and implement the most appropriate solution. • Technologies for physical objects - Physical objects are essential components of the digital twin because they produce a large quantity of heterogeneous datasets from the real world. Sensing and measurement technologies (e.g. IoT sensing technologies, particle-sensing technologies, reverse engineering, laser measurement, image recognition measurement, etc.), are used to sense real-world data. • Technologies for data construction and management - A huge quantity of multi-source, diverse datasets are created from physical objects. The next steps include data collection, transmission, storage, processing, fusion, and visualization. For data identification and near-real time perception, IoT technologies, cameras, sensors, bar codes, quick response (QR) codes, and radio frequency identification (RFID) devices are commonly used. A vast amount of data can be correctly structured and utilized thanks to the development of big data storage frameworks like distributed file storage (DFS), HBase, MySQL, NoSQL database, and NewSQL database. Big data analytics are used to undertake data processing, with the goal of extracting and generating valuable and meaningful data from a considerable number of diverse sets of data that can be incomplete, unstructured, noisy, fuzzy, and random. Big data analytics can be broken down into three categories: analytic visualizations, data mining algorithms, and predictive analytic capabilities. Data visualization is a technique for presenting data analysis results in a simple, intuitive, and interactive way and it changes in response to the applications it is utilized for. Data visualization methods can be classified into geometry-based technologies, pixel-oriented technologies, iconbased technologies, layer-based technologies, image-based technologies, etc, according to the concept of their visualization. • Technologies for virtual modeling - Modeling is the process of converting a physical item into digital representations that computers can process, analyze, and manage. Modeling is likely the most important aspect of a digital twin. A complete digital twin model includes its geometry (shape, size, position, and assembly relationship), physical characteristics (tolerances, material properties, and assembly information), behavior (how the virtual model reacts to external stimuli), and its rules (associations and constraints that can be used to analyze, judge, evaluate, optimize, and predict object performance). To assess the accuracy of the virtual model, it is essential to use verification, validation, and accreditation (VVA) technology. • Services technologies – Digital twin services technologies are designed to achieve various goals in various applications. A service description is a precise declaration of a specific need. A primary purpose of a digital twin is to have near-real time visualization of digital twin services, which necessitates computer graphics processing technologies (e.g. computer graphics, 3D rendering, image processing, graphics engine, virtual-reality synchronization technologies, etc.). • Connection and data transmission technologies - For the digital twin to provide near-real time control and virtual-real state mapping, high-fidelity connection mechanisms are required. There are numerous connection protocols for data flow between the physical space and the digital twin, as well as within the cyber space among various software. Wire transmission and wireless transmission are the two current data transmission methods.
The adoption of 5G contributes to meeting high accuracy and low latency demands. • Environment coupling technologies - The virtual environment model, like the digital twin virtual model, includes geometry, physics, and behavioral data. Environment visualization is just as important as virtual model visualization. A multi-channel immersive stereoscopic display approach using wearable devices (e.g. head-mounted displays, tactile gloves) in combination with virtual reality (VR) and mixed reality (MR) would be appropriate for environment visualization. In other words, human-computer interaction technologies (e.g., VR, augmented reality (AR), MR) as well as human–robot interaction and collaboration should be incorporated.
Although each type of digital twin has unique properties, all digital twins share the following
characteristics [
The formation of a virtual representation of a physical object and interchange of quantitative
and qualitative data, historical data, environmental data, and near-real time data enable the
digital twin to accomplish the following activities [
• Controlling the physical twin in near-real time.
Considering that the wealth of humankind embedded in cultural heritage domain is large, we
are in constant need of promoting cultural heritage [
Digital twin is a promising technology used in cultural heritage preservation that allows for a complex interaction between actual physical objects (be it monuments, historical buildings, or archaeological sites) and their virtual representations. Preventive approaches when it comes to preservation of cultural heritage are preferred to corrective ones. Heritage Building Information Modelling (HBIM) models are one of the researched instances of using collaborative data management together with preservation projects. The research framework presented in [16] consists of integrating HBIM models in the digital twin environment with focus on supporting preservation of cultural heritage. The encompassing method recognizes the importance of HBIM model integration beyond the project stage, automatization of data analytics and simulation processes in the digital twin and consequently increased understanding of the efects preservation would have on cultural heritage sites and their patrons. The authors provide a complete value-based risk management plan process consisting of analysis, diagnosis, therapy, and control part presented in Charter—Principles for the Analysis, Conservation and Structural Restoration of Architectural Heritage [17].
Historical buildings are subject to environmental factors that could interfere with their preservation. In the past, there have been numerous cases of diferent kinds of proposals and analysis aiming at assessing micro-climatic performances of historical buildings and monuments [18, 19, 20]. In addition, there are strategies for energy eficiency improvements of historic buildings in Europe in accordance with UN initiatives for sustainable development. The authors in [21] address preservation of modern heritage (prior to 1970s) in Italy arguing that the built heritage of this time lacks a framework sensible in energy and seismic eficiency. Their case study encompassed all the buildings belonging to Alma Mater Studiorum University of Bologna with particular focus on the Institute of Mathematics of Bologna being built prior to the existence of energy eficiency regulations and seismic building codes. HBIM and Building Energy Model (BEM) model were developed in accordance with an open international standard – digital description of built environment. The authors concluded that the research they conducted presents the first step in advancing towards a digital twin of the building that could be used to improve energy eficiency via calibration, evaluation, and predictive techniques for energy consumption reduction.
The authors in [22] employ an integrated informative system together with digital twin technology aiming at maintenance and preservation of cultural heritage assets. They specifically focus on natural and human induced disasters and their efects on tangible cultural heritage, namely art objects (Mater Matuta and the Resting Satyr), housed in an archaeological museum in South Italy. The methodology proposed is in line with principles of safeguarding and conservation of historical buildings and it is interdisciplinary in nature. Their general framework consists of firstly surveying the art object in three aspects (historical, artistic, and architectural) providing the history of the art objects, properties of the materials used, and the present state of conservation; secondly acquiring two and three dimensional data of the whole art object and its relevant elements providing a high-precision laser-scanned 3D model and high-quality orthomosaics; and lastly providing a diagnosis of the art object state based on information gathered or performing additional surveying and data acquiring needed from other disciplines. The authors’ advantages of modeling art objects as a composition of three elements (container, content, and sensors) are twofold: first, the model is easily connected to a monitoring system, hence management and cataloging of the information can be supervised and improved, and second, the structural performance of the art object can be assessed either by observing all elements together or each component element separately. The authors conclude that use of a digital twin with an integrated informative system is plausible although more research is needed for the development of generalized practical applications. Nevertheless, they suggest that a fully operational digital twin is needed for protection and preservation of cultural heritage.
The authors in [23] employ digital twin technology aiming at inspecting structural systems of historic buildings, namely Milan’s Cathedral, as well as providing predictive analysis predictive analysis of the maintenance of the cathedral in terms of damage restoration processes. In addition, based on previous failures in parts of the structure, deeper insight into future structural behavior could be obtained. The authors developed a simulation model for digital twin application based on nonlinear finite element modelling. Geometric inputs assembled in hierarchical way (complete model, structural elements, and components) were used to create a complete model of the building structure. The authors use concepts from real geometry and structural geometry to understand the correspondence of geometry to structural components and identify positions for new sensor locations as well as continually update the existing model without compromising structural behavior. The results of the presently developed digital twin model show there is a possibility of using it for various preservation tasks while future development could include using virtual sensors for positions not reachable physically or taking into account efects of environment on the damages.
The authors in [24] propose the use of a digital twin for preservation of historical buildings recognizing that improving energy eficiency, while using machine learning models for energy consumption prediction on historical data, can yield sustainable building maintenance. In addition, proposed are various prediction tasks to be performed in a cloud-based digitalization framework, such as facade decay, visitor movement as well as anomaly detection. The proposed framework architecture consists of two main parts that are organized in several layers. The local part uses a perception layer for data collection and the collected information is communicated via edge devices and a transmission layer to the cloud part of the framework. In addition, storage, analysis, and application layer of the cloud part are responsible for storing data, evaluating data, and creating AI models as well as providing maintenance and energy consumption prediction by means of user applications. The authors have created digital twin room using Platform as a Service (PaaS) Azure digital twin in the chosen historical building, deployed sensors for data collection and are proposing to extract useful information from the data to continue researching energy eficiency optimization and smart maintenance.
The authors in [25] present ROCK (Regeneration and Optimization of Cultural heritage in creative and Knowledge cities), a platform supporting the use of digital tools in the cultural heritage domain intended to encompass various digital platforms and strengthen the connection between the physical and virtual domains. Its goals are not only applicable in cultural heritage preservation but also in collecting, managing and sharing of information that could use the combined eforts of all stakeholders, such as academic researchers, professionals, and in particular citizens in creating future collaborative platforms for tangible and intangible cultural heritage resources. The authors conclude that historic centers of several European cities were regenerated by means of citizen participation, hence digital twins were used for learning and contribution in the transformation of physical spaces and understanding the impacts of such transformations.
The authors in [26, 27] presented a live-guided VR tour of an underground oil-mill in the town of Gallipoli, Italy for a content that is not accessible for people with disabilities and inspired by the pandemic situation in 2019 that limited public access to enclosed places. The author developed a virtual visit using a web application with a remotely present real guide in a platform for multiple simultaneous users. The complex interior of the underground mill was virtualized using digital photogrammetry and a 3D model was created. The authors’ future aim is to create an advanced model for cultural heritage management and to develop a digital twin for optimization, increase in eficiency and predictive data analytics.
The authors in [28] present methodology for AR visualization and interpretation of two buildings: the Ballroom, and the St. Francis of Assisi Church, in Belo Horizonte, Brazil. The general framework employed for preservation of cultural heritage consists of data collection (historical and spatial), data processing for dense surface model creation and use of HBIM for digital twin modeling. The authors’ contribution is implementation of the heritage buildings in virtual world via AR application on a tablet or smart phone in addition to real life experience. They propose that future work will include knowledge-based information that could be used within previously developed HBIM for heritage preservation purposes.
Any system or process can benefit from digital twin technology because it reduces errors,
uncertainties, ineficiency, and costs. A digital twin is said to have the following advantages
[
In the context of digital twin benefits in cultural heritage, the following can be emphasized. The benefits of applying the digital twin concept in the cultural heritage domain are mainly for preservation purposes of cultural heritage buildings. Recently, there have been initiatives regarding energy eficiency of historical buildings within the sustainable development framework of modern cities. Predominantly, cultural heritage can benefit from digital twin technology in the maintenance domain together with predictive tasks of determining the influence of environmental conditions as well as natural events such as earthquakes on heritage sustainability. A digital twin improves the relationship between the digital model and the physical domain of heritage assets by merging digital replicas with near-real time operational data from on-site sensors using IoT infrastructure. The major benefit of a digital twin is its ability to automate some processes related to early detection of threat, risk assessments, solution identification, and impact assessment. Thus, in the cultural heritage domain, deploying digital twins allows decision-makers to remotely monitor and observe structure performance in near-real time and focus attention on the most critical parts of the system. This saves the time it takes to gather the information decision-makers need, improving the ability to predict when maintenance is needed and responding quickly to damaged systems. As a result, the rate of degradation and the negative consequences of failures are reduced, the resulting loss of relevance is avoided or at least lowered, and the need for intervention and the expense of conservation work are decreased. Digital twin technology in cultural heritage also allows for availability, accessibility and transparency in documentation and communication via intelligent and semantically enhanced 3D representation.
Although the initial costs of creating a digital twin of cultural heritage wealth is beyond comprehension small steps need to be undertaken in that direction. By allowing virtualization of maintenance tasks future costs can be marginally minimized and solutions to problems optimized.
On the other hand, there are issues involved with the creation of digital twins. These
challenges vary in terms of size, application domain, and complexity of a digital twin, but there
are a few that are constant [
IT infrastructure and a better understanding of data required for analytics. • Standards and regulations - Modeling is crucial for a digital twin to be used in practice.
Because there is no standard for how each model should be created, a consistent methodology is required from the early design phase through the simulation of a digital twin. In addition to standardized modeling, standardizing interfaces, protocols, and data are critical for efective third-party communication, product and human safety, data security, and integrity, among other things. • Design modelling - For future eficient usage of the digital twin concept, guaranteeing that domain use information is transmitted to each of the development and functional stages of a digital twin’s modeling is crucial.
In the domain of cultural heritage, the major challenges are presented as follows. One of the major issues is obtaining adequate communication and computing technologies that can facilitate, for example, preservation tasks. In particular, the process of data collection and data storage over a long period of time can be challenging as performance of these tasks is in direct correlation with the success of cultural heritage digital twin performance. In order to develop predictive tasks computing power is needed, especially if near-real time predictive tasks are required. Another challenge is regarding unification of protocols and standards used in the wide range of technologies and tools that are used and developed by diferent entities. Data and models used in cultural heritage should be standardized by using common standards, formats and protocols.
The expansion of digitalization has resulted in grouping together already existing technologies such as 3D modeling, prototyping, and simulation of the system and labeling it digital twin technology. Researchers in the pre-digital-twin cultural heritage domain have used tools and digital techniques as a way to preserve historical and religious buildings but it was not the most cost-efective and eficient process since it lacked integration. Digital twin technology emerges as inevitable progress of the virtual and physical worlds coupling and providing integrated solutions to monitoring, diagnostic, predictive and optimizing tasks. Even though digital twin technology is used successfully in many diferent fields, the cultural heritage domain has yet to experience its full impact. To provide sustainable management of a cultural heritage environment most stakeholders involved need information to support the decision making process. This is done through integration of HBIM model and management process data implementing forecasting performance management in cultural heritage. Still, there is a global under-investment in digital twin technology in cultural heritage and the question of whether the digital twin concept can be sustainable in the management of cultural heritage environment. Information 11 (2020). URL: https://www.mdpi.com/2078-2489/11/1/22. doi:10.3390/ info11010022. [11] D. Mijić, M. Miljanović, S. Ćurić, D. Jokanović, M. Milićević, M. Jokanović, Digitization of the museum materials of the “museum herzegovina trebinje”, Review of the National Center for Digitization 38 (2021) 28–39. [12] B. Ramic-Brkic, M. Cosovic, S. Rizvic, Cultural heritage digitalization in bih: State-of-theart review and future trends., in: VIPERC@ IRCDL, 2019, pp. 39–49. [13] M. Cosovic, R. Jankovic, B. Ramic-Brkic, Cultural Heritage Image Classification, Springer International Publishing, Cham, 2021, pp. 25–45. URL: https://doi.org/10.1007/ 978-3-030-66777-1_2. doi:10.1007/978-3-030-66777-1_2. [14] M. Ćosović, R. Janković, Cnn classification of the cultural heritage images, in: 2020 19th International Symposium INFOTEH-JAHORINA (INFOTEH), 2020, pp. 1–6. doi:10.1109/ INFOTEH48170.2020.9066300. [15] M. Cosovic, A. Amelio, E. Junuz, Classification methods in cultural heritage., VIPERC@
IRCDL 2320 (2019) 13–24. [16] P. Jouan, P. Hallot, Digital twin: Research framework to support preventive conservation policies, ISPRS Int. J. Geoinf. 9 (2020) 228. [17] ICOMOS, Charter—principles for the analysis, conservation and structural restoration of architectural heritage (2003). URL: https://www.icomos.org/charters/charters.pdf. [18] G. Fidan, Analysis of historic buildings in terms of their microclimatic and thermal comfort performances “example of konya slender minaret madrasah”, J. Arch. Eng. Technol. 03 (2014). [19] M. Maksimović, M. Ćosović, Preservation of cultural heritage sites using iot, in: 2019 18th International Symposium INFOTEH-JAHORINA (INFOTEH), 2019, pp. 1–4. doi:10.1109/ INFOTEH.2019.8717658. [20] M. Maksimović, M. Ćosović, Towards the implementation of iot system for preservation: the church of holy archangels michael and gabriel case study, in: 2022 21st International Symposium INFOTEH-JAHORINA (INFOTEH), 2022, pp. 1–6. doi:10.1109/ INFOTEH53737.2022.9751248. [21] A. Massafra, G. Predari, R. Gulli, Towards digital twin driven cultural heritage management: A hbim-based workflow for energy improvement of modern buildings, ISPRS - Int. Arch.
Photogramm. Remote Sens. Spat. Inf. Sci. XLVI-5/W1-2022 (2022) 149–157. [22] A. Marra, S. Gerbino, A. Greco, G. Fabbrocino, Combining integrated informative system and historical digital twin for maintenance and preservation of artistic assets, Sensors 21 (2021) 5956. URL: http://dx.doi.org/10.3390/s21175956. doi:10.3390/s21175956. [23] G. Angjeliu, D. Coronelli, G. Cardani, Development of the simulation model for digital twin applications in historical masonry buildings: The integration between numerical and experimental reality, Computers Structures 238 (2020) 106282. doi:10.1016/j. compstruc.2020.106282. [24] Z. Ni, P. Eriksson, Y. Liu, M. Karlsson, S. Gong, Improving energy eficiency while preserving historic buildings with digital twins and artificial intelligence, IOP Conference Series: Earth and Environmental Science 863 (2021) 012041. URL: https://doi.org/10.1088/ 1755-1315/863/1/012041. doi:10.1088/1755-1315/863/1/012041. [25] B. Turillazzi, G. Leoni, J. Gaspari, M. Massari, S. Boulanger, Cultural heritage and digital tools: the rock interoperable platform, International Journal of Environmental Impacts: Management, Mitigation and Recovery 4 (2021) 276–288. doi:10.2495/ EI-V4-N3-276-288. [26] F. Gabellone, A digital twin for distant visit of inaccessible contexts (2020). [27] F. Gabellone, Digital twin: a new perspective for cultural heritage management and fruition, ACTA IMEKO 11 (2022) 7. [28] E. Dezen-Kempter, D. L. Mezencio, E. D. M. Miranda, D. P. De Sá, U. Dias, Towards a digital twin for heritage interpretation-from hbim to ar visualization (2020). URL: http: //papers.cumincad.org/cgi-bin/works/paper/caadria2020_141. [29] O. Eleftheriou, C. Anagnostopoulos, Digital twins: A brief overview of applications, challenges and enabling technologies in the last decade [version 1; peer review: awaiting peer review], Digital Twin 2 (2022). doi:10.12688/digitaltwin.17581.1. [30] A. Fuller, Z. Fan, C. Day, C. Barlow, Digital twin: Enabling technologies, challenges and open research, IEEE Access 8 (2020) 108952–108971. doi:10.1109/ACCESS.2020. 2998358. [31] M. Jay, Defining infrastructure requirements for the creation of digital twins, master thesis, malardalen university sweden (2020). URL: https://www.diva-portal.org/smash/get/diva2: 1473114/FULLTEXT01.pdf.