Thermal point-clouds are becoming increasingly relevant in times of climate change, and it is essential that efficient methods for calculating heat losses exist. Whilst heat losses can be calculated by means of simulations and / or on-site expertise, such methods can consume significant financial resources. With the rise of artificial intelligence methods and the availability of thermal imaging technologies, they can be utilized for the automation of such calculations. We propose a methodology for calculating heat losses based on thermal photogrammetry and imaging. By segmenting thermal point-clouds for buildings and removing noise from the result of the segmentation, the output is a point-cloud that is void of unnecessary data for heat loss calculations. This model is then converted to a mesh, and heat losses are calculated for each triangle of the mesh by mapping the area of each triangle to the surface temperature of it based on the closest RGB color from the thermal images, resulting in a direct map between triangle surface area and triangle surface temperature. Our results indicate that such a methodology can be used for more efficient heat loss calculations, as we have achieved a mean average error of 0.42 kW or 0.14 kW depending on whether the ground is considered during calculations or not, respectively. Further work could explore calculating heat losses for multiple buildings at a time, calculating heat losses during different seasons. Furthermore, different emissivity and thermal loss coefficients can be used, as using static values for these parameters limits the accuracy of the calculations.
Buildings account for more than 36 % of all CO2 emissions in the EU. For this reason, it is important to be able to evaluate the thermal losses of buildings accurately and efficiently. More thermally efficient buildings are not only more comfortable for their users, but also contribute less to greenhouse gas emissions. The detection and evaluation of thermal losses using traditional means can be expensive both in terms of time and finances. With the rise in popularity of various artificial intelligence algorithms, such tasks can be delegated at least in part to machine learning methods that can solve them efficiently.
Thermal anomalies, like cold bridges, can be detected by the use of thermal imaging [
For digitizing buildings, point-cloud models are often used [
Regarding incomplete point-cloud models, this challenge is especially noteworthy in the context
of creating digital twins of buildings and cities [
In regards to calculating thermal heat losses for buildings by utilizing thermal photogrammetry, there were no identified works in which such an approach was studied, although thermal bridges and certain façade elements have been identified based on their heat print in thermal photogrammetry models.
In our work, we propose a methodology for calculating thermal losses for buildings by using artificial intelligence methods and thermal photogrammetry. Our proposed methodology for evaluating thermal losses relies on both thermal images and point-clouds that are created by using thermal images.
As can be seen in Figure 1, thermal loss evaluation was performed by acquiring specific data from thermal images and thermal point-clouds – areas of different surfaces of the building and surface temperatures for each of the surfaces. To acquire the required surfaces, outlier points must be removed from the point-cloud, such that all points left in the point-cloud should be of the building. To acquire the temperature for each of the surfaces, the RGB color of each point in the point-cloud must be mapped to a surface temperature value. The results were evaluated against a white-box model, in which the thermal losses were calculated based on the dimensions and thermal characteristics of the materials present in the building envelope. Since such a dataset does not exist (that would include true heat loss values to evaluate against), it was decided to create a custom dataset for this task.
The thermal images were captured using a DJI Mavic 2 Enterprise drone and include thermal data encoded for each pixel of the images.
As it can be seen from the Figure 2, the temperature in the sample images varies from about 0 to 60 °C and certain surfaces of the building are visibly heated. Due to the challenge of creating geometrically accurate photogrammetry models, it was chosen to work using images that were taken in August. For decoding the temperature data, the DJI Thermal SDK was chosen due to its’ compatibility with the images taken with the drone. For decoding the temperature data using the SDK, an emissivity parameter must be supplied. Since the materials on the surfaces of the building in question are varied, it was chosen to use the default value of 0.95. In total, the dataset consists of 362 thermal images, in which, as can be seen from x, the temperature values are between -10.8 and 200 °C (see Table 1).
Since there are no publicly available point-clouds for this specific task, we have generated a thermal point-cloud from the previously mentioned images. The point-cloud was generated from a photogrammetry model that was created using Bentley Systems Context Capture (see Figure 3).
The software takes input images and uses photogrammetry algorithms to form a 3D
representation of the scene in question. As can be seen from the figure, there is a significant amount
of noise surrounding the building. Much of this noise is the ground-level, which must be cleared up
before pursuing building energy calculations. Additionally, there are automobiles that should not be
considered during the calculations. Regarding preparing the point-cloud for further use in building
energy calculations, several approaches were chosen and compared. The first approach was to use a
plane segmentation algorithm for detecting and removing irrelevant planes from the point-cloud,
whilst preserving planes that correspond to the building surfaces. This approach was realized via
RANSAC segmentation. RANSAC is an algorithm that was originally used for finding an optimal
line for a 2D dataset, but it can be adapted for finding an optimal plane in 3D space, as well. The
unsupervised machine learning algorithm works by detecting clusters of points by finding the
minimal sets that correspond to certain geometric primitives, like planes, roofs, etc. [
The results of building segmentation using RANSAC, KPConv and RandLA-Net algorithms were compared using a unified benchmark. The results of the algorithms were compared to the boundingboxes that represent the real surfaces of the building and were evaluated using different error metrics. Amongst the error metrics used, was ∈1
∈2 = ∑ min‖ − ‖2 +2∑ min‖ − ‖2 , ∈2 ∈1
2 , (1) (2) (3) (4) (5) = =
+ + 1 = =
2 × 2 × + + + + where TP – true positives, FP – false positives, FN – false negatives and S – point-cloud, used exclusively for comparing via Chamfer distance (see Equation 5).
The benchmark was performed using a Ryzen 3700U CPU and 16 GB of DDR4 memory. KPConv and RandLA-Net performed better than RANSAC, whilst KPConv performed best out of the three algorithms (see Table 2).
Although the building in the point-cloud was segmented, its segmentation result included a significant amount of noise, most notably – cars and other objects that have the potential to significantly skew the results of further calculations. To address this, it was chosen to statistically remove noise using K-nearest neighbors. Statistical outlier removal ensures that only points that have a certain amount of neighbors n within an average distance threshold of s remain as inliers. Statistical outlier removal was applied to the results of KPConv, which resulted in a significant number of outliers being detected (see Figure 4).
Regarding the numerical results of the outlier removal, the most significant increase in accuracy is according to the Chamfer distance (increase of 0.02), whilst there is also a small accuracy decrease according to the Recall metric (decrease of 0.01).
For retrieving the surface areas of the different surfaces of the building, the point-cloud was converted into a mesh using Poisson surface reconstruction. Poisson surface reconstruction is a robust algorithm for converting point-clouds to meshes by means of converting the task to a wellposed sparse Poisson problem, resulting in a noise-resilient algorithm [19]. Such a conversion enables analyzing the object as a collection of inter-connected triangles, each with their respective area and color. The average RGB color of the three triangle vertices is mapped by Euclidean distance to the closest temperature value from the thermal images. This results in a mapping between each triangle in the mesh and its respective surface temperature, as can be seen in Table 3. = × × ( − ); (6) where is the heat transfer coefficient, is the surface area, and are the indoor and outdoor temperatures.
Since we are not familiar with the indoor temperature of the building in question during the time that the thermal images were taken, different temperature values between 20 and 25 were considered. A value of 0.202 was used, as it is a common value for walls, which is the dominant type of surface in the building. Lastly, the results of the calculations were compared to a white-box model of the building in question, in which the calculation of was based on known measurements of the building and its’ structural materials that make up the envelope.
0.6 0.4 0.2 ,Q 0 kW -0.2 -0.4 -0.6 -0.8 -1 2020,52121,52222,52323,52424,525
Indoor air temperature, °C
As can be seen from Figure 5, the predicted values are close to the real values, especially when the ground layer (which is not visible to the drone at the time of capturing the thermal images) is not considered during calculation. The mean average error for the calculations is 0.42 kW when the ground layer is considered, and 0.14 kW when it is not.
The most common use case for thermal point-clouds when it comes to analyzing the state of buildings – identifying cold bridges and other thermal efficiency phenomena visually. When it comes to evaluating the thermal energy loss of a building, the thermal information of the surfaces of the building envelope that is captured in the model can be used to automate this process. Since a photogrammetry-based point-cloud contains a geometrically accurate representation of the building, this can be used together with thermal information to evaluate thermal losses of the building without the need for constructing white-box models.
Since there is no publicly available dataset that could be used for such a task, we have created our own dataset utilizing 362 thermal images and a point-cloud that was created based on a thermal photogrammetry model constructed from said images. Whilst creating the dataset, we faced difficulties in creating thermal photogrammetry for the heating season, which is why thermal images and models from August were utilized. Using our approach, the point-cloud of the building is prepared by segmenting it using a pre-trained KPConv model – a deep-learning based algorithm for point-cloud segmentation and is further cleaned up by statistical outlier removal. The segmented model is then converted into a mesh via the Poisson surface reconstruction algorithm and the surface areas and colors of each of the triangles of the mesh is calculated. The color of each triangle is mapped to a surface temperature via the closest color based on the Euler distance, which results in a mapping of surface temperature value and triangle area for each of the triangles in the mesh. Lastly, this mapping is used in the calculation of thermal losses based on Equation 6.
Our approach yielded results that have a mean average error of 0.42 kW or 0.14 kW depending on if the ground is considered, or not, respectively. The results resemble the true values that are based on white-box simulations of the building in question. Although the results cannot be directly compared to other studies due to the custom nature of the dataset, the methodology should apply to other works if the thermal point-cloud geometry and thermal of the images accuracy are high.
For further work, it would be suggested that heat losses would be calculated for different seasons, as in our work, they were calculated only during the summer season, in which thermal losses are minimal or sometimes even negative. It is also suggested that an additional step of segmenting the different elements of the building envelope and façade should be segmented, e.g. the roof, windows, etc. and that based on this information, different ε values would be set dynamically when calculating the surface temperature from the thermal images. [7] M. Liu, Y. Shao, R. Li, Y. Wang, X. Sun, J. Wang, Y. You, Method for extraction of airborne LiDAR point cloud buildings based on segmentation, PloS One. 15 (2020) e0232778–e0232778. https://doi.org/10.1371/journal.pone.0232778. [8] S. Lagüela, L. Díaz-Vilariño, J. Martínez, J. Armesto, Automatic thermographic and RGB texture of as-built BIM for energy rehabilitation purposes, Automation in Construction. 31 (2013) 230– 240. https://doi.org/10.1016/j.autcon.2012.12.013. [9] P. Kavaliauskas, J.B. Fernandez, K. McGuinness, A. Jurelionis, Automation of construction progress monitoring by integrating 3D point cloud data with an IFC-based BIM model, Buildings. 12 (2022) 1–19. https://doi.org/10.3390/buildings12101754. [10] E.K. Dey, F. Tarsha Kurdi, M. Awrangjeb, B. Stantic, Effective Selection of Variable Point Neighbourhood for Feature Point Extraction from Aerial Building Point Cloud Data, Remote Sensing (Basel, Switzerland). 13 (2021) 1520-. https://doi.org/10.3390/rs13081520. [11] M. Gil-Docampo, J.O. Sanz, I.C. Guerrero, M.F. Cabanas, UAS IR-Thermograms Processing and Photogrammetry of Thermal Images for the Inspection of Building Envelopes, Applied Sciences. 13 (2023) 3948-. https://doi.org/10.3390/app13063948. [12] J. Zhu, Y. Xu, Z. Ye, L. Hoegner, U. Stilla, Fusion of urban 3D point clouds with thermal attributes using MLS data and TIR image sequences, Infrared Physics & Technology. 113 (2021) 103622. https://doi.org/10.1016/j.infrared.2020.103622. [13] S. Ponti, I. Girola, M. Guglielmin, Thermal photogrammetry on a permafrost rock wall for the active layer monitoring, The Science of the Total Environment. 917 (2024) 170391–170391. https://doi.org/10.1016/j.scitotenv.2024.170391. [14] A. Dlesk, K. Vach, K. Pavelka, Photogrammetric Co-Processing of Thermal Infrared Images and RGB Images, Sensors (Basel). 22 (2022) 1655. https://doi.org/10.3390/s22041655. [15] Y. Hou, M. Chen, R. Volk, L. Soibelman, Investigation on performance of RGB point cloud and thermal information data fusion for 3D building thermal map modeling using aerial images under different experimental conditions, Journal of Building Engineering. 45 (2022) 103380. https://doi.org/10.1016/j.jobe.2021.103380. [16] H. Macher, T. Landes, P. Grussenmeyer, AUTOMATION OF THERMAL POINT CLOUDS ANALYSIS FOR THE EXTRACTION OF WINDOWS AND THERMAL BRIDGES OF BUILDING FACADES, Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci. XLIII-B2-2020 (2020) 287–292. https://doi.org/10.5194/isprs-archives-XLIII-B2-2020-287-2020. [17] M. Jarząbek-Rychard, D. Lin, H.-G. Maas, Supervised Detection of Façade Openings in 3D Point Clouds with Thermal Attributes, Remote Sensing (Basel, Switzerland). 12 (2020) 543-. https://doi.org/10.3390/rs12030543. [18] M.K. Biswanath, L. Hoegner, U. Stilla, Thermal Mapping from Point Clouds to 3D Building Model Facades, Remote Sensing (Basel, Switzerland). 15 (2023) 4830-. https://doi.org/10.3390/rs15194830. [19] M. Kazhdan, M. Bolitho, H. Hoppe, Poisson Surface Reconstruction, The Eurographics Association, 2006. https://doi.org/10.2312/SGP/SGP06/061-070.