=Paper=
{{Paper
|id=Vol-3081/01paper
|storemode=property
|title=Ontology based anamnesis and diagnosis of natural stone damage for retrofitting
|pdfUrl=https://ceur-ws.org/Vol-3081/01paper.pdf
|volume=Vol-3081
|authors=Al-Hakam Hamdan,Peter Katranuschkov,Raimar Scherer
|dblpUrl=https://dblp.org/rec/conf/ldac/HamdanKS21
}}
==Ontology based anamnesis and diagnosis of natural stone damage for retrofitting==
Proceedings of the 9th Linked Data in Architecture and Construction Workshop - LDAC2021 Ontology based anamnesis and diagnosis of natural stone damage for retrofitting Al-Hakam Hamdan, Peter Katranuschkov and Raimar J. Scherer 1 Institute of Construction Informatics, Technische Universität Dresden, Dresden, Germany al-hakam.hamdan@tu-dresden.de Abstract. During the inspection and assessment of existing buildings, a large portion of the recorded data is stored in documents or models that are not com- puter interpretable. Managing this data digitally is a complex process because the evaluation of the building components and the affecting structural damages in- volves a significant amount of manual tasks that can be error-prone and time consuming. Therefore, a knowledge-based approach has been developed, which utilizes web ontologies to store building and damage information in a semantic representation and processes them in an automated assessment via predefined rulesets. The concept is specified and applied for structures made of natural stone and is specifically tested and verified on the common case of a damaged façade. A newly developed software platform is presented, which provides functions for managing and evaluating a damage ontology as well as linking damage infor- mation with a geometry-based BIM model by utilizing an Information Container for linked Document Delivery (ICDD) according to ISO 21597-1 and adapting a general-purpose BIMification approach for retrofitting. Keywords: Natural stone, Retrofitting, Damage ontology, SHACL, BIMification 1 Introduction Building Information Modeling (BIM) is rapidly advancing as an efficient new approach to cooperative building design and construction. However, in the inspection and assessment of existing buildings, a large portion of the recorded data is stored in documents that are not computer interpretable. This is especially important in the con- text of European construction, which considers the retrofitting of the aging building stock as high priority issue, especially regarding energy efficiency and cultural heritage building preservation. It requires a modified BIM-based process, specifically suited to the stepwise development of retrofitting planning, in which an initial building model does not exist and the occurrence of damages and defects that often lead to deterioration of structural components has to be dealt with. We name this new model-based retrofit- ting process BIMification, which we have proposed as a structured method to obtain from an existing real building a BIM model and use that model in a clearly defined 3- stage design process of Anamnesis, Diagnosis and Therapy (Scherer & Katranuschkov, 2018. However, to apply that process successfully it is not sufficient to only generate a Copyright 2021 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0) 8 Proceedings of the 9th Linked Data in Architecture and Construction Workshop - LDAC2021 3D geometry model of the building. A knowledge-based approach built upon semantic web ontologies is additionally needed, aiming at the digital representation of detected damages and their automatic evaluation to determine appropriate maintenance measures, provide important information that is missing or incompletely presented in BIM and reason about that information to enable efficient process application. In this paper, we describe how the generic BIMification approach is applied for the anamnesis and diagnosis of natural stone damages, with specific practical emphasis on the damages on stone façades of historical buildings, which is a frequently occurring task with high societal relevance. In chapter 2 we outline the related work upon which the suggested new ontology-based approach is built. In chapter 3, the overall BIMifi- cation methodology and the developed ontology framework are presented and in chap- ter 4 its application within a newly developed software platform is discussed on the example of a specific test case. The paper concludes with a summary of the results achieved so far and outlines current and envisaged further development work. 2 Related Work The idea of a structured 3-stage design process has been initially suggested by van Balen, Verstrynge and others (2016) as a generic approach towards the structural reha- bilitation of cultural heritage buildings. Albeit not related to BIM use, it enables a deep understanding of the existing building and drafts a picture of the pros and cons of the investments for retrofitting at the earliest stages of the design process. Considerable results have been reached for the creation of a geometry-based digital representation of an existing building, which we denote as part of the geometrical BIMi- fication. Since the introduction of IFC as standardized exchange format for BIM models according to ISO 167391, multiple extensions for the IFC schema has been proposed that add entities and properties for representing the as-damaged state of deteriorated constructions (Artus & Koch, 2020; Hüthwohl et al., 2018). However, the integration of maintenance related data in the IFC model schema can lead to problems, such as the required bilateral extension of compatible IFC Tools or the difficulty of controlling and managing one consistent supermodel. For this reason, more modular approaches have been developed that store information about detected damages in separate ontologies typically formalized in the Web Ontology Language (OWL)2. Cacciotti et al. (2015) have developed a web ontology in the project MONDIS, which allows for modelling damages in cultural heritage buildings, together with the corresponding damage causa- tions. Simeone et al. (2019) have developed a platform for heritage representation, in which data from a geometry-based BIM model is mapped to an ontology for represent- ing restoration processes. To assess the state of the damaged structure and interpret properly collected facts about the deteriorated components for concluding about potential maintenance measures, artificial intelligence methods such as Machine Learning (ML) as well as 1 ISO 16739-1 (2018): Industry Foundation Classes (IFC) for data sharing in the construction and facility management industries – Part 1: Data schema, ISO, Geneva, 1474 p. 2 https://www.w3.org/TR/owl2-overview/ 9 Proceedings of the 9th Linked Data in Architecture and Construction Workshop - LDAC2021 knowledge-based approaches have been proposed. Current ML approaches usually ex- ploit convolutional neural networks to classify and consequently evaluate detected damages (Li et al., 2018). However, such approaches evaluate damages solely based on their characteristics, e.g., the width or height of a crack. Contextual information about the damaged component or building, such as the building material or age, is usually ignored. Taking a different AI approach, Lee et al. (2016) have developed an OWL ontology, which is used in a case-based reasoning process for evaluating construction defects. Thereby, a set of previously assessed defect cases is stored in an ontological database. Using SPARQL queries, case-based reasoning for similar defects can then be applied. Contrary to ML, which relies on correlations between training data and input data, knowledge approaches utilize rules that are based on logic and expert know-how. For example, Hu et al. (2019) successfully apply rules formalized in N3Logic to reason the causations of detected tunnel damages and provide decision making support. 3 Methodology 3.1 Workflow Based on the BIMification concept of Scherer & Katranuschkov (2018) the proposed retrofitting process can be structured in three major phases, namely (1) Anamnesis, (2) Diagnosis and (3) Therapy. The latter is not different from the typical BIM approach where multiple variants are examined and is therefore not further discussed here. This generic process can be easily adapted for the domain of natural stone damage. The overall workflow shown in Fig. 1 does thereby remain valid but sub-processes, inputs and outputs are adjusted to the specific characteristics of the application domain. BIMification Process BIM-based Retrofit Design Anamnesis Diagnosis Therapy BIM of the existing building The models from the Anamnesis BIM of the retrofitted and Ontology Framework extended by an ontology of building of the building topology proposed renovation measures and the stone damages Fig. 1. Overall workflow The Anamnesis phase is dedicated to the survey and collection of facts about the building and the affecting damages. In this phase a geometry-based BIM model of the existing building is created and linked via an ICDD with gradually constructed ontolo- gies providing semantic representations of the construction and the affecting damages. The process is divided into several sequential sub-processes (see Fig. 2). It starts with the creation of a pure geometric model based on the existing building by utilizing BIM authoring tools like Revit, which allow for an IFC export. In subsequent steps, the 10 Proceedings of the 9th Linked Data in Architecture and Construction Workshop - LDAC2021 resulting IFC model is enriched with additional information. The resulting ontological models described in detail in section 3.3 comprise a subset of the geometry data ob- tained from the BIM model, but also topological (Topological BIMification) and infra- structure information (Neighbourhood BIMifcation) as well as semantic data about the embedded building components (Element- and Subcomponent BIMification). Basic BIMification Advanced BIMification Geometrical Topological Neighbourhood Element Subcomponent BIMification BIMification BIMification BIMification BIMification BIM/IFC geometry Building Topology BOT extended with BOT further extended BOT ext. for stones (SCO) Ontology (BOT) infrastructure context with element and material Damage Ontology (DOT) (arch. style, climate etc.) description (DICBM) Fig. 2. Anamnesis phase The diagnosis phase is dedicated to the analysis and interpretation of the collected facts from the preceding anamnesis phase to obtain the necessary understanding of the build- ing’s damage state and propose possible retrofitting measures. In this phase the infor- mation of the ontologies that were linked in an ICDD to the BIM model in the previous steps as semantic enrichment are processed by a reasoning engine to infer additional information about the damages and their impact on the inspected building. Rules are applied to infer potential maintenance measures, which are defined via the Shapes Con- straint Language (SHACL)3. The process is structured in two sub-processes (see Fig. 3). In the first sub-process, the anamnesis results recorded in the ontology framework are assessed via SPARQL queries as described in section 4.1. On that basis, in the sec- ond sub-process variants for potential retrofitting measures as well as an ontology for representing potential renovation measures (SRMO) are generated. This ontology is used later in the Therapy phase to create and examine different retrofitting actions, and evaluate these in terms of resources, cost, and time. Extended BIMification Damage Variability BIMification BIMification Anamnesis Damage Proposed retrofitting measures and results assessment variants obtained via SHACL rules (SPARQL queries) to create retrofit ontology (SRMO) Fig. 3. Diagnosis phase 3 https://www.w3.org/TR/shacl/ 11 Proceedings of the 9th Linked Data in Architecture and Construction Workshop - LDAC2021 3.2 Multimodel environment The largest portion of the modelling data needed in a retrofitting project is geometry related information about the existing building. Two sets of such data have to be gen- erated in the geometric BIMification step: (1) the BIM data for the building, represented as an IFC data set comprising the building structure composed of storeys, spaces and building elements (slabs, walls, columns, windows, doors etc.), and (2) the BIM data for the natural stone subcomponents that have to be inspected and evaluated with regard to damages. The first of these data sets is comprised mostly of instances of standardized well-defined IFC object classes. In contrast, the second data set contains aggregated elements, which are usually defined through proxy classes and share the same geomet- rical space as the higher-level building element they belong to. For example, an IfcWall in the first data set, which represents a stone façade, is represented at the same time by many IfcProxy objects in the second data set that describes the aggregated individual stones. A decomposition of the building structure at that level of detail does not exist in the IFC schema and has to be provided by other means. In addition, various non- geometric modelling data about the building need to be considered as well such as the building age or climatic and environmental conditions. Because this information is not represented in a single model, we apply the Multimodel approach by utilizing the stand- ardised Information Container for linked Document Delivery (ICDD) of ISO 21597-14. Thereby, the separate models in the ICDD are provided as payload documents and are linked together via link ontologies that are called payload triples, providing the connec- tion semantics between the elements of the models. In the domain addressed herein, this semantics is broadly as follows: • Stone elements are linked to the corresponding building elements in the BIM/IFC model and later to semantic representations in the ontological damage model as well. • Data about the building are linked to the IfcBuilding object in the BIM/IFC model; • Material data are linked to the building elements in the BIM/IFC model and through that to the stone elements in the detailed stone models; however, if stones of different material are used that link is reversed and material is directly linked to the proxy elements representing the stones; • Stone damages are linked to the stone elements within the ontology framework, thereby indirectly linking them to the containing building element if necessary. Based on the BIM/IFC information a non-geometric ontological representation is generated, which provides a knowledgebase for further semantic reasoning. The links to the geometry data in BIM are thereby preserved and can be used whenever needed. 3.3 Ontology framework The ontology framework, which is used for creating a representation of deteriorated components made of natural stone, consists of 3 ontology domains each serving a 4 ISO 21597-1 (2020): Information container for linked document delivery - Exchange specifi- cation, Part 1: Container, ISO, Geneva, 41 p. 12 Proceedings of the 9th Linked Data in Architecture and Construction Workshop - LDAC2021 specific function. The representation of the natural stone components in their as-built state is achieved by applying an extension of the Building Topology Ontology (BOT) proposed by Rasmussen et al. (2017), which we call Stone Component Ontology (SCO) (Seeaed & Hamdan, 2019). It provides terminology for defining stone and gap repre- sentations and their topological relations with each other, specifically considering ad- jacencies and aggregations. In addition, the Digital Construction Building Material on- tology (DICBIM) proposed by Valluru et al. (2020) can be used for specifying material properties of the stone and gap representations. To model the as-damaged state of a building, damage representations that are related to deteriorated components must be further defined. For that purpose, the Natural Stone Damage Ontology (NSD)5 proposed in Seeaed & Hamdan (2019) is used. It is an ex- tension of the generic Damage Topology Ontology (DOT) developed by Hamdan et al. (2019). While DOT6 provides generic classes for defining damage individuals, NSD extends the DOT concept and adds specific classes and properties describing damages on natural stone structures, such as fractures or blistering. In this regard, NSD is based on the classified damage types of the ICOMOS glossary of the International Scientific Committee for Stone (ICOMOS, 2010). As already mentioned in section 3.1, an inte- gral part in the diagnosis stage is the inference of appropriate renovation measures. Thereby, new individuals that represent potential stone renovation measures are rea- soned by processing predefined SHACL rules. To characterize the newly inferred ren- ovation measures the Stone Renovation Measure Ontology (SRMO)7 has been devel- oped as extension of the Construction Tasks Ontology (CTO) 8 (Bonduel, 2021). In this regard, various measure methods are structured in a taxonomy as subclasses of the class srmo:StoneRepairTask, which is a subclass of cto:RepairTask and could be assigned to deteriorated stone components via the object property cto:isSubjectOf. Fig. 4. Main Classes required for representing a damaged stone component and its renovation 5 https://w3id.org/nsd# 6 https://w3id.org/dot# 7 https://w3id.org/srmo# 8 https://w3id.org/cto# 13 Proceedings of the 9th Linked Data in Architecture and Construction Workshop - LDAC2021 Fig. 4 provides an example demonstrating the links between components of the pro- posed ontology framework, which forms only the base knowledgebase for semantically representing a damaged stone component and its associated potential renovation measures. Therefore, an information extension through utilizing additional ontologies, e.g., for defining structural functions or energy parameters, is recommended. 4 Application on Test Case As proof of concept, the methodology described in the preceding chapter has been ap- plied for a deteriorated natural stone façade. Accordingly, the geometry is represented in two IFC models, the first defining the geometry of the whole building and its general building components, and the second defining the detailed stone geometry of the dete- riorated wall. The IFC Model that defines the stone geometry is linked to the corre- sponding IFC entity representing the whole stone façade through a linkset in an ICDD. The ICDD creation and management of the IFC models is processed by a software platform, which was developed in the frames of this research and is published on Github9. Thereby, the IFC management and processing features utilize the APSTEX IFC framework by Tauscher & Theiler (2018). Fig. 5 below shows an example of the performed test case. Fig. 5. IFC models for representing the natural façade and its stone components The developed software platform allows automated generation of an ontology from the IFC model by utilizing the BOT and SCO ontology definitions, similar to the IFC to Linked Building Data Converter (Bonduel et al., 2018). In the generation process all individuals that relate to a building element are linked with their IFC counterparts 9 https://github.com/Alhakam/bimsage 14 Proceedings of the 9th Linked Data in Architecture and Construction Workshop - LDAC2021 through an additional ICDD linkset. The generated ontology is then enriched with ad- ditional individuals that define damages, which affect the deteriorated stone compo- nents, by utilizing the DOT and NSD terminology. In the test case shown in Fig. 5, a total number of 1115 IFC entities were transformed into instances of sco:StoneComponent. Furthermore, 227 damage individuals were created and assigned to the ontological stone representations. 4.1 SPARQL-utilized Damage Assessment When assessing the damaged structure, several parameters about the detected dam- ages and the affected stone components must be considered, such as the damage type, the stone material, or the estimated impact of the damage on the construction. To re- trieve this information, SPARQL queries are appropriate. Two information sets are thereby of high importance for the assessment of the addressed specific application do- main, i.e. (1) information about the damaged area in relation to the overall façade area, and (2) information about the damage index for each detected damage type as suggested by Fitzner et al. (2002). The SPARQL query in Listing 1 is used for determining the damaged area in relation to the overall stone area of a façade. For that purpose, two sub queries are used. The first computes the damaged area and the second the area of all considered stones. The area values of each damage and component are asserted as literals of props:area, which is calculated by multiplying the length and width of the corresponding entities surface. The construction ontology generated from the BIM/IFC model determines the compo- nent areas in advance, while the damage areas are asserted in the diagnosis process. SELECT (?damageAreaSum / ?stoneAreaSum AS ?damageSpread) WHERE { #sub-query for damage area SELECT (SUM(?damageAreaValue) AS ?damageAreaSum) WHERE { ?damageArea rdf:type dot:DamageArea . ?damageArea dot:aggregatesDamageElement ?damage . ?damage rdf:type dot:Damage . ?damageArea props:area ?damageAreaValue . #sub-query for stone area SELECT (SUM(?stoneAreaValue) AS ?stoneAreaSum) WHERE { ?damageArea rdf:type dot:DamageArea . ?damageArea props:stoneArea ?stoneAreaValue . Listing 1. SPARQL query for calculating the damaged area in relation to the façade area Another important assessment information is the damage index, which is calculated according to the below formula from (Fitzner et al., 2002). Thereby, the damage index can be calculated through a linear or progressive function. Since SPARQL does not provide the functionality for calculating mathematical square root functions, only the required data parameters, namely the damaged area (props:area) and damage impact (nsd:damageImpact) are queried, and the formula is processed in a separate algorithm. 15 Proceedings of the 9th Linked Data in Architecture and Construction Workshop - LDAC2021 Fig. 6. Formulas for determining the linear and progressive damage index (Fitzner et al., 2002) Applying the SPARQL queries on the test case ontology delivered the results shown in Table 1. Table 1. Detected damage types on natural stone façade of the test case NSD-class Number of damages Damaged Area DIlin DIprog nsd:HairCrack 207 20,7 % 1,063 2,299 nsd:StarCrack 14 1,0 % 0,050 0,500 nsd:Fracture 3 0,6 % 0,030 0,387 nsd:Lichen 1 0,4 % 0,020 0,316 nsd:Plant 2 0,5 % 0,025 0,354 4.2 SHACL-based Reasoning Besides the general reasoning through OWL axioms that are implemented in the ontol- ogies, additional rules can be applied through separate rule models formalized in SHACL. This allows for a more modular use of certain rule sets, depending on domain specific needs, such as damage assessment based on structural viewpoints or construc- tion durability. Consequently, various rule sets for damage evaluation could be defined and applied to the proposed ontology framework. In this regard, an exemplary rule set has been developed and applied on the test case for the inference of potential renovation measures that are appropriate for certain damage types and properties. Since new individuals of the type srmo:StoneRepairTask are generated in the infer- ence process, the SHACL-based reasoning is processed in two stages. First, it is inferred if a stone component has a recommended renovation measure of a certain type. Thereby, the reasons for a required renovation are primarily dependent on the charac- teristics of damages that affect the component. Listing 2 shows an exemplary rule for inferring filling of cracks (srmo:CrackFilling) as potential renovation measure for stones that are affected by star cracks (nsd:StarCrack) and have a damage impact value between 2 and 3 as estimated during inspection. If the conditions are fulfilled a new statement is added, in which the sco:Stone instance is related to a new object through 16 Proceedings of the 9th Linked Data in Architecture and Construction Workshop - LDAC2021 an auxiliary object property like rm:needsCrackFilling, which is defined as subproperty of cto:isSubjectOf inside SHACL.The URI of the new individual is generated randomly via a SPARQL function. In the second reasoning step, the previously inferred individ- ual is further characterized (see Listing 3). Thereby, it is important that an appropriate subclass of srmo:StoneRepairTask is assigned. sco:Stone a rdfs:Class , sh:NodeShape ; sh:rule [ a sh:TripleRule ; sh:order 1 ; sh:subject sh:this ; sh:predicate cto:isSubjectOf; sh:object [rm:RandomURIGeneration] ; sh:condition [ sh:property [ sh:path dot:hasDamage ; sh:class nsd:StarCrack ; sh:qualifiedValueShape [ sh:path nsd:damageImpact ; sh:minCount 1 ; sh:minInclusive 2 ; sh:maxInclusive 3 ; ] ; sh:qualifiedMinCount 1 ; ] ; ] ] . Listing 2. SHACL Rule for inferring crack filling measure for deteriorated stones rm:UnclassifiedRenovationMeasure a sh:NodeShape ; sh:targetObjectsOf rm:needsCrackFilling ; sh:rule [ a sh:TripleRule ; sh:order 2 ; sh:subject sh:this ; sh:predicate rdf:type ; sh:object srmo:CrackFilling ; ] . Listing 3. SHACL Rule for classifying an inferred renovation measure individual In a next step, SHACL rules for inferring appropriate renovation measures were applied on the damage ontology. Thereby, for each instance of sco:StoneComponent that is affected by at least one damage instance, a corresponding set of renovation measures was assigned (see Table 2). Table 2. Inferred renovation measure recommendations in the test case SRMO-class Retrofitted Damage Types srmo:CrackFilling nsd:HairCrack ; nsd:StarCrack srmo:StaticConstructiveMeasure nsd:Fracture srmo:StoneCleaning nsd:Plant ; nsd:Lichen Some inferred renovation classes comprise of multiple sub-measure representations that could be applied. For instance, the class srmo:StoneCleaning has subclasses such 17 Proceedings of the 9th Linked Data in Architecture and Construction Workshop - LDAC2021 as srmo:MechanicalCleaning or srmo:ChemicalCleaning which in turn also have sev- eral subclasses. Consequently, all subclasses of an inferred renovation measure are rec- ommended as potential retrofitting solutions. Furthermore, it must be emphasized that the inferred measures are only recommendations. When deciding for an appropriate renovation measure, a cost and risk analysis need to be made, which is part of the Ther- apy phase of the retrofitting process and has not been performed in the current test case. Nonetheless, the definition of additional rules for this subsequent step is possible and hence a subject of further research. 5 Conclusion In this research, a methodology has been proposed for ontologically representing damaged structures made of natural stone in OWL and performing a logic-based eval- uation on them utilizing SPARQL-queries and SHACL shapes. In this regard, the ex- isting building and its aggregated components are represented geometrically through an IFC model. By utilizing the ICDD Multimodel, the IFC model is linked with an OWL ontology that functions as semantic model for storing information about the to- pology, the damage entities that are detected on each stone, as well as additional metadata. Currently, the damage assessment is focussed on the reasoning of recom- mended renovation measures. An inference of information through additional ontology extensions and rules, such as reasoning the consequential damages from the current damage state, or a detailed classification and risk analysis based on detected damage properties would be a conceivable enhancement of the presented approach and is sub- ject of further research. Furthermore, the ontology framework and evaluation queries and rulesets have been designed specifically for the assessment of natural stone struc- tures. In future research, this approach could be extended for other construction or dam- age types. Acknowledgments This research work was enabled by the support of the German Ministry for Education and Research (BMBF) in the frames of the BIM-SIS project (Grant no. 01IS18017/2018) and partially by the preceding EU FP7 program in the frames of the eeEmbedded project (Grant no. 609349/2013). References Artus, M., & Koch, C. (2020). Modeling geometry and semantics of physical damages using IFC. EG-ICE 2020 Workshop on Intelligent Computing in Engineering, Proceedings, 144–153. Bonduel, M. (2021). A Framework for a Linked Data-Based Heritage BIM. https://lirias.kuleuven.be/handle/123456789/674476 18 Proceedings of the 9th Linked Data in Architecture and Construction Workshop - LDAC2021 Bonduel, M., Oraskari, J., Pauwels, P., Vergauwen, M., & Klein, R. (2018). The IFC to Linked Building Data Converter - Current Status. 6th Linked Data in Architecture and Construction Workshop (LDAC). Fitzner, B., Heinrichs, K., La Bouchardiere, D., Bouchardiere, D. La, Galan, E., & Zezza, F. P. O. (2002). Damage index for stone monuments. 5th International Symposium on the Conservation of Monuments in the Mediterranean Basin. Hamdan, A.-H., Bonduel, M., & Scherer, R. J. (2019). An ontological model for the representation of damage to constructions. 7th Linked Data in Architecture and Construction Workshop. Hu, M., Liu, Y., Sugumaran, V., Liu, B., & Du, J. (2019). Automated structural defects diagnosis in underground transportation tunnels using semantic technologies. Automation in Construction, 107(August), 102929. https://doi.org/10.1016/j.autcon.2019.102929 Hüthwohl, P., Brilakis, I., Borrmann, A., & Sacks, R. (2018). Integrating RC Bridge Defect Information into BIM Models. Journal of Computing in Civil Engineering, 32(3). https://doi.org/10.1061/(ASCE)CP.1943-5487.0000744 Hüthwohl, P., Lu, R., & Brilakis, I. (2019). Multi-classifier for reinforced concrete bridge defects. Automation in Construction, 105(December 2018), 102824. https://doi.org/10.1016/j.autcon.2019.04.019 International Scientific Committee for Stone (ISCS). (2010). ICOMOS - Illustrated Glossary on Stone Deterioration Patterns. Lee, D. Y., Chi, H. lin, Wang, J., Wang, X., & Park, C. S. (2016). A linked data system framework for sharing construction defect information using ontologies and BIM environments. Automation in Construction, 68(May), 102–113. https://doi.org/10.1016/j.autcon.2016.05.003 Li, D., Cong, A., & Guo, S. (2018). Sewer damage detection from imbalanced CCTV inspection data using deep convolutional neural network with hierarchical softmax. Submited to: Automation in Construction, 101(June 2018), 199–208. https://doi.org/10.1016/j.autcon.2019.01.017 Rasmussen, M., Pauwels, P., Lefrançois, M., Schneider, G. F., Hviid, C., & Karlshøj, J. (2017). Recent changes in the Building Topology Ontology. https://hal- emse.ccsd.cnrs.fr/emse-01638305 Scherer, R. J., & Katranuschkov, P. (2018). BIMification: How to create and use BIM for retrofitting. Advanced Engineering Informatics, 38, 54–66. https://doi.org/10.1016/j.aei.2018.05.007 Seeaed, M. K., & Hamdan, A. (2019). BIMification of stone walls for maintenance management by utilizing Linked Data. 31st Forum Bauinformatik, November. Simeone, D., Cursi, S., & Acierno, M. (2019). BIM semantic-enrichment for built heritage representation. Automation in Construction, 97(November 2018), 122– 137. https://doi.org/10.1016/j.autcon.2018.11.004 Tauscher, E., & Theiler, M. (2018). apstex IFC Framework. www.apstex.com Valluru, P., Karlapudi, J., Menzel, K., & Shemeikka, J. (2020). A Semantic Data Model to Represent Building Material Data in AEC Collaborative Workflows. January 2021. https://doi.org/10.1007/978-3-030-62412-5 19