During the inspection and assessment of existing buildings, a large portion of the recorded data is stored in documents or models that are not computer interpretable. Managing this data digitally is a complex process because the evaluation of the building components and the affecting structural damages involves a significant amount of manual tasks that can be error-prone and time consuming. Therefore, a knowledge-based approach has been developed, which utilizes web ontologies to store building and damage information in a semantic representation and processes them in an automated assessment via predefined rulesets. The concept is specified and applied for structures made of natural stone and is specifically tested and verified on the common case of a damaged façade. A newly developed software platform is presented, which provides functions for managing and evaluating a damage ontology as well as linking damage information with a geometry-based BIM model by utilizing an Information Container for linked Document Delivery (ICDD) according to ISO 21597-1 and adapting a general-purpose BIMification approach for retrofitting.
Building Information Modeling (BIM) is rapidly advancing as an efficient new
approach to cooperative building design and construction. However, in the inspection
and assessment of existing buildings, a large portion of the recorded data is stored in
documents that are not computer interpretable. This is especially important in the
context of European construction, which considers the retrofitting of the aging building
stock as high priority issue, especially regarding energy efficiency and cultural heritage
building preservation. It requires a modified BIM-based process, specifically suited to
the stepwise development of retrofitting planning, in which an initial building model
does not exist and the occurrence of damages and defects that often lead to deterioration
of structural components has to be dealt with. We name this new model-based
retrofitting process BIMification, which we have proposed as a structured method to obtain
from an existing real building a BIM model and use that model in a clearly defined
3stage design process of Anamnesis, Diagnosis and Therapy (Sc
The idea of a structured 3-stage design process has been initially suggested by van Balen, Verstrynge and others (2016) as a generic approach towards the structural rehabilitation of cultural heritage buildings. Albeit not related to BIM use, it enables a deep understanding of the existing building and drafts a picture of the pros and cons of the investments for retrofitting at the earliest stages of the design process.
Considerable results have been reached for the creation of a geometry-based digital
representation of an existing building, which we denote as part of the geometrical
BIMification. Since the introduction of IFC as standardized exchange format for BIM models
according to ISO 167391, multiple extensions for the IFC schema has been proposed
that add entities and properties for representing the as-damaged state of deteriorated
constructions
To assess the state of the damaged structure and interpret properly collected facts
about the deteriorated components for concluding about potential maintenance
measures, artificial intelligence methods such as Machine Learning (ML) as well as
1 ISO 16739-1 (2018): Industry Foundation Classes (IFC) for data sharing in the construction and
facility management industries – Part 1: Data schema, ISO, Geneva, 1474 p.
2 https://www.w3.org/TR/owl2-overview/
knowledge-based approaches have been proposed. Current ML approaches usually
exploit convolutional neural networks to classify and consequently evaluate detected
damages
Based on the BIMification concept of Sc
BIMification Process
Anamnesis BIM of the existing building and Ontology Framework of the building topology and the stone damages
Diagnosis The models from the Anamnesis extended by an ontology of proposed renovation measures
The Anamnesis phase is dedicated to the survey and collection of facts about the building and the affecting damages. In this phase a geometry-based BIM model of the existing building is created and linked via an ICDD with gradually constructed ontologies providing semantic representations of the construction and the affecting damages. The process is divided into several sequential sub-processes (see Fig. 2). It starts with the creation of a pure geometric model based on the existing building by utilizing BIM authoring tools like Revit, which allow for an IFC export. In subsequent steps, the resulting IFC model is enriched with additional information. The resulting ontological models described in detail in section 3.3 comprise a subset of the geometry data obtained from the BIM model, but also topological (Topological BIMification) and infrastructure information (Neighbourhood BIMifcation) as well as semantic data about the embedded building components (Element- and Subcomponent BIMification).
Basic BIMification
Geometrical BIMification
Topological BIMification
Neighbourhood BIMification
Advanced BIMification
Element BIMification
Subcomponent BIMification BIM/IFC geometry
Building Topology Ontology (BOT)
BOT extended with infrastructure context (arch. style, climate etc.)
BOT further extended with element and material description (DICBM)
BOT ext. for stones (SCO)
Damage Ontology (DOT) The diagnosis phase is dedicated to the analysis and interpretation of the collected facts from the preceding anamnesis phase to obtain the necessary understanding of the building’s damage state and propose possible retrofitting measures. In this phase the information of the ontologies that were linked in an ICDD to the BIM model in the previous steps as semantic enrichment are processed by a reasoning engine to infer additional information about the damages and their impact on the inspected building. Rules are applied to infer potential maintenance measures, which are defined via the Shapes Constraint Language (SHACL)3. The process is structured in two sub-processes (see Fig. 3). In the first sub-process, the anamnesis results recorded in the ontology framework are assessed via SPARQL queries as described in section 4.1. On that basis, in the second sub-process variants for potential retrofitting measures as well as an ontology for representing potential renovation measures (SRMO) are generated. This ontology is used later in the Therapy phase to create and examine different retrofitting actions, and evaluate these in terms of resources, cost, and time. 3.2
The largest portion of the modelling data needed in a retrofitting project is geometry related information about the existing building. Two sets of such data have to be generated in the geometric BIMification step: (1) the BIM data for the building, represented as an IFC data set comprising the building structure composed of storeys, spaces and building elements (slabs, walls, columns, windows, doors etc.), and (2) the BIM data for the natural stone subcomponents that have to be inspected and evaluated with regard to damages. The first of these data sets is comprised mostly of instances of standardized well-defined IFC object classes. In contrast, the second data set contains aggregated elements, which are usually defined through proxy classes and share the same geometrical space as the higher-level building element they belong to. For example, an IfcWall in the first data set, which represents a stone façade, is represented at the same time by many IfcProxy objects in the second data set that describes the aggregated individual stones. A decomposition of the building structure at that level of detail does not exist in the IFC schema and has to be provided by other means. In addition, various nongeometric modelling data about the building need to be considered as well such as the building age or climatic and environmental conditions. Because this information is not represented in a single model, we apply the Multimodel approach by utilizing the standardised Information Container for linked Document Delivery (ICDD) of ISO 21597-14. Thereby, the separate models in the ICDD are provided as payload documents and are linked together via link ontologies that are called payload triples, providing the connection semantics between the elements of the models. In the domain addressed herein, this semantics is broadly as follows: • Stone elements are linked to the corresponding building elements in the BIM/IFC model and later to semantic representations in the ontological damage model as well. • Data about the building are linked to the IfcBuilding object in the BIM/IFC model; • Material data are linked to the building elements in the BIM/IFC model and through that to the stone elements in the detailed stone models; however, if stones of different material are used that link is reversed and material is directly linked to the proxy elements representing the stones; • Stone damages are linked to the stone elements within the ontology framework, thereby indirectly linking them to the containing building element if necessary.
Based on the BIM/IFC information a non-geometric ontological representation is generated, which provides a knowledgebase for further semantic reasoning. The links to the geometry data in BIM are thereby preserved and can be used whenever needed. 3.3
The ontology framework, which is used for creating a representation of deteriorated
components made of natural stone, consists of 3 ontology domains each serving a
specific function. The representation of the natural stone components in their as-built
state is achieved by applying an extension of the Building Topology Ontology (BOT)
proposed by
To model the as-damaged state of a building, damage representations that are related
to deteriorated components must be further defined. For that purpose, the Natural Stone
Damage Ontology (NSD)5 proposed in Seeaed &
As proof of concept, the methodology described in the preceding chapter has been
applied for a deteriorated natural stone façade. Accordingly, the geometry is represented
in two IFC models, the first defining the geometry of the whole building and its general
building components, and the second defining the detailed stone geometry of the
deteriorated wall. The IFC Model that defines the stone geometry is linked to the
corresponding IFC entity representing the whole stone façade through a linkset in an ICDD.
The ICDD creation and management of the IFC models is processed by a software
platform, which was developed in the frames of this research and is published on
Github9. Thereby, the IFC management and processing features utilize the APSTEX
IFC framework by Tausc
When assessing the damaged structure, several parameters about the detected
damages and the affected stone components must be considered, such as the damage type,
the stone material, or the estimated impact of the damage on the construction. To
retrieve this information, SPARQL queries are appropriate. Two information sets are
thereby of high importance for the assessment of the addressed specific application
domain, i.e. (1) information about the damaged area in relation to the overall façade area,
and (2) information about the damage index for each detected damage type as suggested
by
The SPARQL query in Listing 1 is used for determining the damaged area in relation to the overall stone area of a façade. For that purpose, two sub queries are used. The first computes the damaged area and the second the area of all considered stones. The area values of each damage and component are asserted as literals of props:area, which is calculated by multiplying the length and width of the corresponding entities surface. The construction ontology generated from the BIM/IFC model determines the component areas in advance, while the damage areas are asserted in the diagnosis process.
SELECT (?damageAreaSum / ?stoneAreaSum AS ?damageSpread) WHERE { #sub-query for damage area SELECT (SUM(?damageAreaValue) AS ?damageAreaSum) WHERE { ?damageArea rdf:type dot:DamageArea . ?damageArea dot:aggregatesDamageElement ?damage . ?damage rdf:type dot:Damage . ?damageArea props:area ?damageAreaValue . #sub-query for stone area SELECT (SUM(?stoneAreaValue) AS ?stoneAreaSum) WHERE { ?damageArea rdf:type dot:DamageArea .
?damageArea props:stoneArea ?stoneAreaValue .
Listing 1. SPARQL query for calculating the damaged area in relation to the façade area
Another important assessment information is the damage index, which is calculated
according to the below formula from
Applying the SPARQL queries on the test case ontology delivered the results shown in Table 1. Besides the general reasoning through OWL axioms that are implemented in the ontologies, additional rules can be applied through separate rule models formalized in SHACL. This allows for a more modular use of certain rule sets, depending on domain specific needs, such as damage assessment based on structural viewpoints or construction durability. Consequently, various rule sets for damage evaluation could be defined and applied to the proposed ontology framework. In this regard, an exemplary rule set has been developed and applied on the test case for the inference of potential renovation measures that are appropriate for certain damage types and properties.
Since new individuals of the type srmo:StoneRepairTask are generated in the inference process, the SHACL-based reasoning is processed in two stages. First, it is inferred if a stone component has a recommended renovation measure of a certain type. Thereby, the reasons for a required renovation are primarily dependent on the characteristics of damages that affect the component. Listing 2 shows an exemplary rule for inferring filling of cracks (srmo:CrackFilling) as potential renovation measure for stones that are affected by star cracks (nsd:StarCrack) and have a damage impact value between 2 and 3 as estimated during inspection. If the conditions are fulfilled a new statement is added, in which the sco:Stone instance is related to a new object through an auxiliary object property like rm:needsCrackFilling, which is defined as subproperty of cto:isSubjectOf inside SHACL.The URI of the new individual is generated randomly via a SPARQL function. In the second reasoning step, the previously inferred individual is further characterized (see Listing 3). Thereby, it is important that an appropriate subclass of srmo:StoneRepairTask is assigned.
sco:Stone a rdfs:Class , sh:NodeShape ; sh:rule [ a sh:TripleRule ; sh:order 1 ; sh:subject sh:this ; sh:predicate cto:isSubjectOf; sh:object [rm:RandomURIGeneration] ; sh:condition [ sh:property [ sh:path dot:hasDamage ; sh:class nsd:StarCrack ; sh:qualifiedValueShape [ sh:path nsd:damageImpact ; sh:minCount 1 ; sh:minInclusive 2 ; sh:maxInclusive 3 ; ] ; sh:qualifiedMinCount 1 ; ] ; ] ] .
Listing 2. SHACL Rule for inferring crack filling measure for deteriorated stones rm:UnclassifiedRenovationMeasure a sh:NodeShape ; sh:targetObjectsOf rm:needsCrackFilling ; sh:rule [ a sh:TripleRule ; sh:order 2 ; sh:subject sh:this ; sh:predicate rdf:type ; sh:object srmo:CrackFilling ; ] .
Listing 3. SHACL Rule for classifying an inferred renovation measure individual In a next step, SHACL rules for inferring appropriate renovation measures were applied on the damage ontology. Thereby, for each instance of sco:StoneComponent that is affected by at least one damage instance, a corresponding set of renovation measures was assigned (see Table 2). Some inferred renovation classes comprise of multiple sub-measure representations that could be applied. For instance, the class srmo:StoneCleaning has subclasses such as srmo:MechanicalCleaning or srmo:ChemicalCleaning which in turn also have several subclasses. Consequently, all subclasses of an inferred renovation measure are recommended as potential retrofitting solutions. Furthermore, it must be emphasized that the inferred measures are only recommendations. When deciding for an appropriate renovation measure, a cost and risk analysis need to be made, which is part of the Therapy phase of the retrofitting process and has not been performed in the current test case. Nonetheless, the definition of additional rules for this subsequent step is possible and hence a subject of further research. 5
In this research, a methodology has been proposed for ontologically representing damaged structures made of natural stone in OWL and performing a logic-based evaluation on them utilizing SPARQL-queries and SHACL shapes. In this regard, the existing building and its aggregated components are represented geometrically through an IFC model. By utilizing the ICDD Multimodel, the IFC model is linked with an OWL ontology that functions as semantic model for storing information about the topology, the damage entities that are detected on each stone, as well as additional metadata. Currently, the damage assessment is focussed on the reasoning of recommended renovation measures. An inference of information through additional ontology extensions and rules, such as reasoning the consequential damages from the current damage state, or a detailed classification and risk analysis based on detected damage properties would be a conceivable enhancement of the presented approach and is subject of further research. Furthermore, the ontology framework and evaluation queries and rulesets have been designed specifically for the assessment of natural stone structures. In future research, this approach could be extended for other construction or damage types.
This research work was enabled by the support of the German Ministry for Education and Research (BMBF) in the frames of the BIM-SIS project (Grant no. 01IS18017/2018) and partially by the preceding EU FP7 program in the frames of the eeEmbedded project (Grant no. 609349/2013).