Multi-Agent Monitoring System for Heat Loss Mapping of Multi-Story Buildings Ivan Burlachenko1[0000-0001-5088-6709], Iryna Zhuravska1[0000-0002-8102-9854], Oleksii Tohoiev1[0000-0003-3465-7767], Yehor Ukhan1[0000-0001-8745-2541] and Yevhen Tiutiunyk2[0000-0002-4359-975X] 1 Petro Mohyla Black Sea National University, Mykolaiv, Ukraine 2 LeadsMarket LLC, Woodland Hills, California, USA ivan.burlachenko2010@gmail.com, iryna.zhuravska@chmnu.edu.ua, oleksiitohoiev@gmail.com, yehor.ukhan@chmnu.edu.ua, borman.program@gmail.com Abstract. In this paper, the problem of developing a multi-agent method for de- tecting the places of heat energy leaks on the multi-story buildings using ma- chine learning is solved. Efficient data processing of scanning areas for the heat energy leak monitoring was achieved using the multi-agent monitoring system (MAMS) that can perform calculations in the cloud conditionally. Features of the monitoring system with the integration of an analytical model for presenting a heat loss map with an account of multiple autonomous separated UAV’s for temperature measurements were contained. The MAMS reliability of the syn- chronization model between simultaneous localization and mapping method and generated heat loss map based on temperature measurements was con- firmed. It has been experimentally proven that theoretical assumptions and ac- curacy for experimental usage during the multi-story building leaks analysis are sufficient. The recognition time of markers of the front of the building is in the range from 0 to 27 s. In this case, with the proposed model СNN, the CPU load during the execution of tasks did not exceed 26%. Keywords: heat loss mapping, heat leak detection, machine learning, multi- agent system, GPS, pyrometer, UAV, MAMS. 1 Introduction The general situation in the field of heating systems is that the main purpose of heat supply to consumers is dominated by the need for an efficient system. About 90% of all Ukrainian high-rise buildings require measures to improve the functioning of the heat supply systems. Of these, 60-70% of the houses were built in the years of indus- trial construction in typical series who are currently faced with the problem of heat loss[1]. Heat loss at home is the amount of heat generated by a house on the street per unit of time. They are measured in watts (watts). Heat loss is affected by temperature differences inside and outside the house. This dependence is directly proportional - the larger the temperature difference, the higher the heat loss[2]. Also, heat loss de- Copyright © 2019 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). pends on the design of the house. How strongly the external walls or windows impede the generation of heat characterizes the resistance to heat transfer. Between the re- sistance to heat transfer of building envelopes and heat losses there is an inversely proportional relationship - with increasing thermal resistance, heat losses decrease [3]. The Quick U-Building (QUB) method is a dynamic method developed to estimate the heat loss coefficient of a building in one night without occupancy[4]. Feasibility measurements and comparisons with various references have been done in earlier studies whatever numerically, experimentally in an ideal case, or experimentally in real cases [5,6]. This article presents a review of various perturbation methods devel- oped to assess building thermal performance, details of theoretical understanding of the QUB method, and gathers experimental results obtained in many different config- urations[7]. The heat loss coefficients estimated with the QUB method are in good agreement with experimental references and are reproducible. This demonstrates that the QUB method has a real potential to estimate the heat loss coefficient of a building in a short duration and with a reasonable accuracy [8]. The thermal imager is a modern device that analyzes the air circulation in the room, helps to identify structural defects and provide the customer with visual inspec- tion results [9]. The device emits infrared light and picks up the electromagnetic reaction of the surfaces of the studied object. By measuring the intensity of such radiation, the ther- mal imager can calculate the maximum temperature of a surface and determine the place of heat leakage[10]. The device is able to analyze the input data and display a graph of temperature dif- ferences, as well as calculate the optimal performance for the object [11]. The thermal imager on its screen creates a thermogram - this is a spectrozonal picture of the circu- lation of warm and cold air in a room. The color scheme in the picture varies from saturated red to blue or blue. The main problem in measuring heat loss is about a thermal imager is used for measurement, this is a high price for the device, data transfer complexity, averaging of readings along the edges of the measurement zones. In turn, the need for a high density of measurement points for the accuracy of the result is added to the pyrome- ter. Also a common problem will be low mobility, which is completely dependent on human capabilities. After processing the data, there is a problem with an error when averaging data during the collection, and then when calculating the heat loss of the region and reducing it to a heat map [12]. 2 Heat loss measurement hardware To implement idea for automated heat loss measurements list of equipment was ana- lyzed. The testo 805i (see Fig. 1, a), for example, is a professional measuring infra- red (IR) thermometer from the Testo Smart Probes series, for use with smartphones/tablets with either Android or Apple operating systems. It is, however, worth drawing attention that you need to download and install free Testo Smart Probes App on your device before using the Testo 805i Infrared Thermometer. Fig. 1. Testo 805i Infrared Thermometer (a) and Crosse MA10006-BLA Wireless Weather Station with Gateway (b) A La Crosse MA10006-BLA smart weather station with Mobile Alerts Weather Gateway MA10000 and Wireless Wifi Thermo-Hygro Transmitter TX29DTH-IT+ options can also be used as an instrument for detecting the heat leakage areas in multi- story buildings and industrial facilities. And furthermore, the data obtained with the help of the weather station can be used to develop the Heat Leakage Detecting app. Besides build-in weather station features, such as 12-hour forecast, outdoor/indoor temperature and humidity sensors) the La Crosse MA10006-BLA is able to share weather data (indoor/outdoor humidity and temperature, wind speed, etc) via the In- ternet, as well. The weather data will further be available on any smartphone with necessary app installed. Fig. 2. Quadrocopter DJI Matrice 210 with thermal Zenmuse XT and video camera on board (a) and quadrocopter DJI Phantom 4 with ТХ29DTH-IT on board (b) Moreover, it should be mentioned that up to 50 Mobile-Alerts sensors at the same time can be connected to the weather station due to the build-in Gateway MA10000 functionality (see Fig. 1, b). Thus, with the help of any drone being equipped with heat sensors, it would be possible not only collecting walls temperature data necessary for heat mapping, but also receiving inside and outside temperature data for further comparison and subsequently more accurate detection of the heat leak rate. The DJI with thermal imaging (see Fig. 2, a) or with the previously mentioned temperature sensor (see Fig. 2, b) can be used as transport means for the heat measur- ing equipment. An external sensor transmits the information to the weather station with the help of an IT+ technology (Instant Transmission technology) at 868 mega- hertz. IT+ technology advantages: 1. High Level System Security ; 2. The transmission distance is increased to 100 meters; 3. More economical (Cost-effective); 4. High-quality sensors; Functional scheme, allow transmission distance is increased to 100m take sensors data on IT + on station, across cloud service Mobile-Alerts via ethernet – on mobile. So device must include a smartphone, with OS Android above 3.2. functional diagram system we can see on Fig. 3. Fig. 3. Functional diagram for the heat loss measurement unit of MAMS. At the current, the weather station can be upgraded to analyze the data to prevent the fungus formation. In order to protect the walls of houses from damage, such as mold, fungus, fluctuations in temperature, the comprehensive approach is required, to be outlined in the next report. This upgrade can also help to prevent an occurrence of microcracks between floor panels and in the seams between walls. 3 Multi-agent monitoring system To solve technical problems, a multi-agent monitoring system for the efficient control of the trajectories of many UAVs was proposed. The functional diagram of MAMS for scanning heat losses was presented (See Fig. 4). The DSHLS set describes an array of UAVs that perform HLS heat loss scanning. Each HLSj scanning path includes an HLA scanning area. Processing of the scan area by each DSiHLS UAV is implemented and based on a neural network, which is capable of detecting markers of the scanning area of building windows using the Deep CNN architecture. Positioning accuracy is ensured according to the SLAM algorithm. Fig. 4. The heat loss scanning process using MAMS. TR The SLAM AMAMS ( i ) is a path mapping system for the AMAMS (i ) agent and given as a set of HLS trajectories. A section of the trajectory TRj is considered correct if, in the implementation of the SLAM algorithm, the region of the surrounding space RG(TRj) was defined. Displayed equations are described the model of the MAMS logic to control the UAVs set:  P SLAM AMAMS i    TR j DTC ( RG(TR j ))  HLS ; TR  j 0   DCNN HLA W  (1)  AMAMS ( i )  MD ACC ( HLA) MDL ( HLA)  VTHRБ ; k 0   ITPAMAMS ( i )  TSNS  TUAV FLC  CTR ST  CTR M  TWS  TETH  VLDT ;   HLS AMAMS ( i )  K e  VIS HLS DCNN  (W x , W yDCNN , TMP ITP ( x, y, c), t ). The next condition for the correct operation of the model is adequate recognition of markers within the HLA. The AMAMS (i ) agent entity that operating based on one or several UAVs must ensure the recognition of all W markers in the HLA scanning area with floating MDACC recognition accuracy at the MDL recognition threshold. The total data processing time ITPAMAMS (i ) depends on the data transmission time from the tem- perature sensor, the processing time of the sensor signals by the system, the compu- ST ting resources of which are occupied by the CTR stabilization commands and the M recognition of window markers by the CTR neural network. The dynamic dependence of the HLS AMAMS (i ) visualization map of the heat loss map taking into account the noise Ke has been determined. Fig. 5. Software architecture for MAMS included algorithms for scanning optimization. The architecture of the MAMS software implementation with a multithreaded object- oriented model of managing functional agents was presented in Fig. 5. Abstractions of algorithms that optimize UAV positioning during scanning of the heat loss region were determined. This provides the flexibility to control processing in the MAMS- Mixer object based on the interpretation of the MAMSRulesInterpreter rules. Fig. 6. The result of combination DCNN and SLAM algorithms inside MAMS for heat loss mapping. Fig. 7. Plot boxes diagrams of markers recognition time (a), CPU load percentage (b) and de- scription for the proposed CNN model (c). Video quality and stabilization issues became the main reasons for the Deep CNN architecture to be applied to facades' windows recognition. If additional facade's markers are detected, UAVs will define more precisely position during the HLA scanning. Collected content about the façade’s markers for repeatable CNN real-time learning was used. 4 Conclusion The functional scheme of the mobile system for detecting heat leakage through the elements of construction of a residential building is developed. The developed mobile system connects up to 50 wireless sensors up to 100 meters away via the Mobile Alerts cloud server. External wireless sensors transmit infor- mation to the Smart Weather Station using IT + (Instant Transmission Technology) at 868 MHz. The device includes a smartphone with Android OS version not lower than 3.2. System testing was performed using a mobile phone Xiaomi Mi A2 6/128GB. It is suggested to use a helicopter to lift the sensors to the specified height UAV DJI Phan- tom 4. The software for determining the places of heat leakage of structural elements of buildings was developed. As can be seen in Fig. 7. the recognition time of markers of the front of the building is in the range from 0 to 27 s. In this case, with the pro- posed model СNN, the CPU load during the execution of tasks did not exceed 26%. References 1. Sonderegger, R., Modera, M., March, M.: Electric Co-heating: A Method for Evaluating Seasonal Heating Efficiencies and Heat Loss Rates in Dwellings, Lawrence Berkeley Na- tional Laboratory (1979). 2. Giraldo-Soto, C., Erkoreka, A., Mora, L., Uriarte, I., Del Portillo, L. A.: Monitoring Sys- tem Analysis for Evaluating a Building's Envelope Energy Performance through Estima- tion of Its Heat Loss Coefficient. Sensors, vol. 18(7):2360, Basel, Switzerland (2018). 3. Johnston, D., Miles-Shenton, D., Farmer D., Wingfield, J.: Whole House Heat Loss Test Method(Coheating ), Leeds Metropolian University, Leeds, UK (2013). 4. Maillet, D., André, S., Batsale, J.C., Degiovanni, A., Moyne, C.: Thermal Quadrupoles: Solving the Heat Equation through Integral Transforms John Wiley (2000). 5. Zhuravska, I., Musiyenko, M., Tohoiev, O.: Development the heat leak detection method for hidden thermal objects by means the information-measuring computer system. CEUR Workshop Proceedings: Proceedings of the 2nd International Workshop on Computer Modeling and Intelligent Systems, vol. 2353, pp. 350–364, Zaporizhzhia, Ukraine (2019). 6. Wang, L., Wang, Z., Yang, R.: Intelligent Multiagent Control System for Energy and Comfort Management in Smart and Sustainable Buildings. IEEE Transactions on Smart Grid, vol. 3, pp. 605-617 (2012). 7. Yang, J., Ning, C., Deb, C., Zhang, F., Cheong, D., Lee, S.E., Sekhar, C., Tham, K.W. k- Shape clustering algorithm for building energy usage patterns analysis and forecasting model accuracy improvement. Energy and Buildings, vol. 146, pp. 27–37 (2017). 8. Brajterman, O., Jamieson, M., Verstraeten, Y., Arbon, J.: EPBD Compliance Study-Final Report. Energy Performance of Buildings Directive (EPBD), European Union, Luxem- bourg (2018). 9. Firth, S.K., Lomas, K.J., Wright, A.J.: Targeting household energy-efficiency measures us- ing sensitivity analysis. Building Research and Information, vol. 38(1), pp. 24–41(2010). 10. Garmston, H., Fox, M., Pan, W., De Wilde, P.: Multi-storey building retrofit with a focus on the façade selection process: a UK commercial office case study. Management, pp. 81– 90 (2013). 11. Möller, B., Nielsen, S.: High resolution heat atlases for demand and supply mapping. In- ternational Journal of Sustainable Energy Planning and Management, vol. 1, pp. 41-58 (2014). 12. ISO. Energy Performance of Buildings. Energy Needs for Heating and Cooling, Internal Temperatures Sensible and Latent Heat Loads–Part 1: Calculation Procedures. AENOR; Genoa, Italy: ISO Standard 52016-1 (2017).