In this paper, we explain through a case study how to develop a digital twin that can be used for safety analysis of missions in physical contexts. More specifically, we consider a scenario where firefighters are operating inside a building under fire, but communicating online with a mission control station. One of the main tasks of the mission control is to ensure that the firefighters always have enough oxygen to exit the building safely. To this end, a Digital Twin can be created that reflects the physical structure of the burning building, the location of the firefighters and the oxygen level in their breathing apparatus. The Digital Twin uses these models and a shortest path algorithm to estimate the oxygen required to exit the building, and alerts mission control and the respective firefighter to exit the building on time. The case study is used to illustrate key concepts for building Digital Twins in safety-critical contexts.
continuously monitors the physical system and triggers alarms when predetermined conditions are met, and enables the user to simulate various scenarios and predict the expected outcome, thereby facilitating decision making.
This paper focuses on the problem class of safety-critical missions regarding a dedicated use
case. Most of the diferent aspects of the use case have been considered in isolation. For example,
[
While DT have proven to be beneficiary [ 14, 15, 16], it is still a challenge to design and build up suitable DT. The goal of this paper is not to provide yet another solution for supporting ifrefighter scenarios but to identify key artefacts occurring in this and similar use cases. We identify their mathematical nature and discuss corresponding formal modelling and analysis techniques as well as supporting tools. We identify that in our use case, we have to deal with • building models • discrete mathematical objects and optimisation, and • physical processes, typically modelled as diferential equations.
We discuss formal representations from a computer science perspective and tools to be used for realising the case study. We implement our case study using state of the art tools (ROS and Python) to validate our findings.
In Section 2, a case study is provided, motivating a problem suitable for using a DT; it is analysed regarding its artefacts, and main categories of objects and processes are determined. Their formal models and supporting tools are discussed in Section 3.1 and their integration in Section 3.2. In Section 4, we have implemented our case study to gain first practical insights.
The research is part of the O5G-N-IoT project1, which aims to enhance security components with 5G technology.
Let us describe our case study in detail to derive the main kinds of artefacts informally before we identify their mathematical nature.
The case study presented in this paper deals with firefighters’ rescue from a burning building. We depict our scenario for buildings ranging from one to about five storeys and approximately 200 m2, ensuring that a single team of firefighters is suficient to deal with the fire. We assume that a fire is detected and reported to a central fire station, which sends out an appropriate response team. The team is usually divided into two functional groups: mission command and emergency personnel. The mission commander operates from the mission control centre, part of one of the fire vehicles outside the building. The emergency personnel can be a group of up to 40 firefighters who carry out diferent tasks inside the building (see Fig. 1). The main tasks we have in mind are rescuing people, checking for people to be rescued and containing the ifre. The firefighters operating in the building are supplied with oxygen by a self-contained breathing apparatus with a compressed air tank. The firefighters are in constant contact with the mission command centre via a permanent voice radio. Since this is the aim of our project, we also assume that 5G will be used for both voice radio and a data link to each firefighter. The data link is used, among other things, to receive vital signs data from each firefighter. We assume that their position and level of the compressed air tank are transmitted to the mission control centre.
The current state of the art is for the mission command to manually record the position of each firefighter and the corresponding air pressure levels regularly by radio and to record these values on a board in the mission control centre. As soon as a critical pressure value is detected for a firefighter, he or she is instructed to leave the building (again by radio). The incident commander can also draw up a plan to search the entire building for people. Last but not least, the incident commander continuously monitors the spread of the fire. The project aims to digitise and improve the first two tasks.
Let us revisit the scenario described above. We identify the following artefacts: • There is a mission commander in direct contact with the firefighters. • There is a constant data link sending vital data such as air pressure and position in the building. • There is a plan of the building used for coordination and mission planning. • There are physical processes taking place, such as the deflation of the air containers and the spread of the fire. • There are optimisation problems, such as mission planning to search the whole building. • There is a critical property that needs to be checked, i.e. the assessment of the remaining oxygen level in relation to the distance/time required to evacuate the building.
To support the tasks in question, we plan to model these (and similar) scenarios using a DT and apply simulation and optimisation methods based on the DT. To this end, we use the following simplifications: • We assume that our 5G network connection provides a reliable voice and data link. • We assume there is an existing reliable indoor location solution – see [17, 18] for an overview of current methods. • We assume that a suitable 3D plan of the building is available, although we discuss diferent options in the next section. • We consider “simple” physical processes, such as the emptying of the air reservoirs, but leave “complicated” physical phenomena to later studies. In particular, we assume a static digital plan to represent the building initially but ignore any changes to the building, for example, due to fire. • As a concrete task, we only consider the intelligent estimation of the remaining oxygen level with respect to the time needed to leave the building.
Clearly, these are highly simplifying assumptions. However, the current setup already shows important challenges and basic solutions.
We identify the need for • modelling the structure (e.g. floor plan) and semantics (e.g. accessible doors and stairs) of a building to represent locations of people and plan (feasible) routes • solving discrete optimisation problems, such as finding the shortest path to an exit and travelling salesman to search all rooms (although the latter is not discussed in this paper) • modelling physical processes, such as draining a compressed air supply
In the next section, we discuss diferent modelling and digital solution options for these artefacts, both mathematical and using standard formats and tools, and their interplay. In Section 4 we then show a concrete implementation together with a first evaluation.
In the previous paragraph, we identified building models, discrete optimisation problems, and physical processes as the main artefacts to be formalised and addressed. Let us discuss the corresponding possibilities in the following.
Building models There are a wide range of standards and variants for the digital representation of buildings, from human-readable drawings to machine-usable 3D graphics with semantic metadata.
City Geography Markup Language (CityGML)2 is an open standard that defines a conceptual model and exchange format for describing the geometry and appearance, topology (relationships, neighbourhoods) and semantics (meaning) of 3D city objects, facilitating the integration of urban geodata for applications in smart cities and digital city twins. It supports diferent levels of detail (LoD 0-3) so that objects become more detailed as the LOD increases to represent elements such as rooms, doors, corridors, stairs and even furniture. CityGML is based on standards from the Open Geospatial Consortium (OGC) and ISO 191XX.
Building Information Modelling (BIM) technology, as opposed to traditional CAD technology, can represent geometric and rich semantic information about building components and their relationships to support lifecycle data sharing. BIM is defined in ISO 29481-1:2016 as: “[the] use of a shared digital representation of a built object (including buildings, bridges, roads, process plants, etc.) to facilitate design, construction and operation processes to form a reliable basis for decisions.” 2https://www.ogc.org/standards/citygml
An important data exchange standard for BIM is the IFC (Industry Foundation Classes)3 standard. The IFC object-based data model contains geometric and rich semantic information about building components and is supported by most BIM software vendors. A body of research has focused on extracting and managing semantic information about building components in the form of IFC for various applications, including indoor path planning [3].
The Green Building XML Schema (gbXML)4 is an open schema developed to facilitate the transfer of building data stored in BIMs to engineering analysis tools. It is integrated into several computer-aided design (CAD) software packages, notably Autodesk. gbXML is a type of XML ifle with over 500 types of elements and attributes that can be used to describe all aspects of a building.
Discrete optimisation processes The optimisation problem described in the scenario is to ifnd the shortest path to any existing exit. This can be seen as a shortest path graph problem as the building model can be formed into a graph where each door is a node, and each direct connection between doors is a weighted edge. In the taxonomy provided in [19], the scenario is static, as the weights of the graph do not change over time.
Physical processes Physical processes are usually specified in terms of diferential equations, which can then be solved explicitly in simple cases. In (real) more complex cases, this is usually not possible in an acceptable amount of time, so approximation algorithms are used.
This can be addressed in three types of approaches: • Solving by hand or writing custom code adapted to the equation in question. • Using dedicated libraries in appropriate programming languages (e.g. scipy [20] in Python, dsolve in Matlab or Mathematica). • Use of languages specifically designed to describe physical processes (e.g. Modelica [ 21, 22, 23, 24]). This can be complemented by the use of the Functional Mock-up Interface (FMI)5 – a standard that defines a container and interfaces for exchanging dynamic simulation models from diferent modelling tools. It also specifies co-simulation Functional Mock-up 3https://www.buildingsmart.org/ 4https://www.gbxml.org/ 5https://fmi-standard.org/
Units (FMU), which contain the model and the simulation solver. In this way, simulation models with diferent time steps can be coupled.
In our setting, the digital twin needs to model physical processes to predict the depletion of compressed air over time, which may also depend on the type of activity, such as walking up or down stairs, whether additional equipment needs to be carried, and so on. Accurate modelling may require modular, multi-domain models of individual component models.
In the previous subsection, we have identified key artefact categories of safety-critical rescue missions in buildings. However, it is essential that the concrete formalisation and supporting tools can be integrated into a single digital twin. A typical approach is encapsulating all artefacts as functional mock-up units and using a manually written integrator as coordinator. This approach has been mechanised by providing a programming layer to encapsulate simulators compliant with the FMI standard into OO structures, integrate FMOs into the class, and type systems [25].
To obtain practical insights into developing Digital Twins, we have implemented the discussed scenario following the discussion in the previous sections. To this extent, we focus on constantly checking whether each firefighter has enough air pressure to continue operating. To this end, two predictions must be made: • How long will it take the person to reach the nearest exit? • How long will the air last?
The localisation and the building model are used to calculate the time it will take to exit the building.
Therefore, based on the building plan, a graph can be created where passageways are nodes like doors, and all passageways directly connected by rooms are connected by edges weighted by distance. On this graph, a shortest path analysis can be performed from a given starting point to all possible exits, with the minimum result describing the shortest exit path. This can then be converted (e.g. by assuming a constant speed) into an estimated running time.
The pressure curve can be used to estimate how long the remaining air pressure in the tank will last. This is described by a monotonically decreasing function, which can be approximated, for example, by a polynomial.
The two durations must be determined periodically – in our scenario, with a bufer of a few minutes, low frequencies such as 0.1 Hz are suficient since the emergency personnel cover a maximum of 10 s equivalent distance within 10 s and thus add a maximum of 20 s time compared to immediate detection.
After subtracting the bufer, an alarm is triggered as soon as the time needed to evacuate the building falls below the time that breathing air is safely available.
Our implementation of the DT to support rescue missions uses the Robot Operating System (ROS)6 as a way of determining routes in a building map and implements the air pressure forecast as a linear approximation. Currently, the implementation has the following limitations: • Only one floor is taken into account. • Building changes (e.g. caused by fire) are not considered, as the fire itself, and its ability to make pathways impassable are not considered. • Air consumption is approximated as constant, ignoring influences such as load, fitness and environmental characteristics such as temperature.
A plan suitable for navigation is created based on a 3D model of a house (see Fig. 2). A file in IFC format can be reformatted [26] into a ROS-usable image file in PGM format using a toolkit called ifcOpenShell7. If a scale is known, it can be added to a ROS-usable YAML file with metadata. The resulting plan (see Fig. 3) is used as input for the ROS navigation stack8, and a path to a fixed location (like the exits) can then be calculated by calling the GetPlan service 9. By calculating the length of the returned path and assuming a constant walking speed, the time required to leave the building via a selected exit can be estimated. In the case of multiple exits, the shortest path to an exit is used for further calculations.
A simple approximation of the remaining air pressure is obtained by assuming that the tank capacity is known and the respiration rate is constant; therefore, the remaining usable time of the gas container can be estimated. The elapsed time is then subtracted from this to obtain the remaining time.
In our prototype, both the time needed to leave the building and the time of remaining oxygen are calculated every ten seconds. A warning is issued if the diference falls below a safety margin of 300 seconds.
In this paper, we have considered a real-world case study. We examined the artefact categories of Digital Twins and explored how to model them from a mathematical and computer science point 6https://www.ros.org/ 7https://ifcopenshell.org/ 8https://wiki.ros.org/navigation 9https://docs.ros.org/en/api/nav_msgs/html/srv/GetPlan.html of view. Additionally, we briefly outlined tool support and identified integration techniques. We do not claim any completeness of the overviews, yet hope to contribute a valuable contribution when building digital twins. We have implemented our scenario to gain practical insights and understand the limits of current approaches and tool support.
For our implementation, we considered several simplifications but also learned that many additional simplifications are needed to make the problem easier to handle. Notably, it was assumed that a suitable 3D model of the building already existed, that there was a stable radio link throughout the building, that suficiently accurate indoor localisation was possible, and that the fire itself was disregarded entirely. While these simplifications have made the problem more tractable, it is important to note that, in reality, these assumptions may not hold (but may be overcome in future improvements/extensions of the approach). It is also important to note that no real-world experiments have yet been conducted, with only a simulation having been evaluated at this stage.
In future studies, conducting experiments in real-world environments would be beneficial to validate the proposed system. In addition, it seems useful to extend the Digital Twin to enable more comprehensive mission planning. The Digital Twin can simulate possible mission scenarios, assist in route planning for systematic searches, and estimate whether a task force can reach a specific location. This can lead to increased safety and eficiency in rescue operations.
Nevertheless, as our main finding, we learn that further tool support is essential to limit the burden of building up and employing digital twins. [3] Y.-H. Lin, Y.-S. Liu, G. Gao, X.-G. Han, C.-Y. Lai, M. Gu, The IFC-based path planning for 3D indoor spaces, Advanced Engineering Informatics 27 (2013) 189–205. [4] L. Liu, B. Li, S. Zlatanova, P. van Oosterom, Indoor navigation supported by the industry foundation classes (ifc): A survey, Automation in Construction 121 (2021) 103436. URL: https://www.sciencedirect.com/science/article/pii/S0926580520310165. doi:https://doi. org/10.1016/j.autcon.2020.103436. [5] W. Palacz, G. Ślusarczyk, B. Strug, E. Grabska, Indoor Robot Navigation Using Graph
Models Based on BIM/IFC, 2019, pp. 654–665. doi:10.1007/978-3-030-20915-5_58. [6] N. El-Sheimy, Y. Li, Indoor navigation: state of the art and future trends, Satellite
Navigation 2 (2021) 7. doi:10.1186/s43020-021-00041-3. [7] ISO 19650-1:2018, 2018. URL: https://www.iso.org/standard/68078.html. [8] ISO 16739-1:2018, 2018. URL: https://www.iso.org/standard/70303.html. [9] Stanford Artificial Intelligence Laboratory et al., Robotic operating system, 2018. URL: https://www.ros.org. [10] M. A. Gopee, S. A. Prieto, B. G. de Soto, IFC-based generation of semantic obstacle maps for autonomous robotic systems, volume 3 of Computing in Construction, University of Turin, Rhodes, Greece, 2022. URL: https://ec-3.org/publications/conference/paper/?id= EC32022_161. doi:10.35490/EC3.2022.161. [11] S. Kim, M. Peavy, P.-C. Huang, K. Kim, Development of bim-integrated construction robot task planning and simulation system, Automation in Construction 127 (2021) 103720. URL: https://www.sciencedirect.com/science/article/pii/S0926580521001710. doi:https: //doi.org/10.1016/j.autcon.2021.103720. [12] U. Rueppel, K. M. Stuebbe, BIM-based indoor-emergency-navigation-system for complex buildings, Tsinghua Science and Technology 13 (2008) 362–367. URL: http://ieeexplore. ieee.org/document/6073006/. doi:10.1016/S1007-0214(08)70175-5. [13] Davidson, Anne, Gales, John, BIM and fire safety engineering - overview of state of the art, International Journal of High-Rise Buildings 10 (2021) 251–263. [14] Y. Gao, D. Chang, C.-H. Chen, Z. Xu, Design of digital twin applications in automated storage yard scheduling, Advanced Engineering Informatics 51 (2022) 101477. URL: https: //www.sciencedirect.com/science/article/pii/S1474034621002275. doi:https://doi.org/ 10.1016/j.aei.2021.101477. [15] T. Jin, Z. Sun, L. Li, Q. Zhang, M. Zhu, Z. Zhang, G. Yuan, T. Chen, Y. Tian, X. Hou, C. Lee, Triboelectric nanogenerator sensors for soft robotics aiming at digital twin applications, Nature Communications 11 (2020) 5381. URL: https://doi.org/10.1038/s41467-020-19059-3. doi:10.1038/s41467-020-19059-3. [16] O. C. Madubuike, C. J. Anumba, R. Khallaf, A review of digital twin applications in construction, Journal of Information Technology in Construction 27 (2022) 145–172. URL: https://www.itcon.org/paper/2022/8. doi:10.36680/j.itcon.2022.008. [17] H. Obeidat, W. Shuaieb, O. Obeidat, R. Abd-Alhameed, A review of indoor localization techniques and wireless technologies, Wireless Personal Communications 119 (2021) 289–327.
URL: https://doi.org/10.1007/s11277-021-08209-5. doi:10.1007/s11277-021-08209-5. [18] A. Yassin, Y. Nasser, M. Awad, A. Al-Dubai, R. Liu, C. Yuen, R. Raulefs, E. Aboutanios, Recent advances in indoor localization: A survey on theoretical approaches and applications, IEEE Communications Surveys & Tutorials 19 (2017) 1327–1346. doi:10.1109/COMST.2016.