A threat prediction method based on the intellectual analysis of historical data in complex distributed systems (CDS) is proposed. The relevance of the chosen research topic in terms of considering the flood as a physical process of raising the water level, which is measured at stationary and automatic hydrological posts, is substantiated. Based on this, a mathematical formulation of the problem is formulated, within the framework of which an artificial neural network based on the freely distributed TensorFlow software library is implemented. The analysis of the effectiveness of the implemented artificial neural network was carried out, according to which the average deviation of the predicted water level when forecasting for one day at a stationary hydrological post was 3.323%. For further research on forecasting water levels, an algorithm is proposed for evaluating historical data at automatic posts, which will allow using these data to predict water levels according to the proposed method and at automatic posts. Thus, the neural network allows predicting the flood situation with acceptable accuracy, which allows special services to take measures to counter this threat.
The ever-more rapid development of digital technologies allows us to take a fresh look at the interaction of various components of the real world: physical, biological, social, etc. Moreover, digital technologies themselves become the most important component of this world, and these technologies allow people to influence the interaction of all components among themselves and especially to people. This complex interconnected set of objects of various complexity and physical nature can be considered as a complex distributed system (CDS), which is characterized by a significant remoteness of the component components from each other and a quick change in their characteristics over time.
The components of such CDS themselves are complex distributed systems and have (or potentially can have), including negative impact on each other, that is, they constitute or pose a threat to each other. For example, such a biological (natural) component of the CDS as a river network when a natural (spring or rain flood) or artificial (dam destruction) high water rise occurs on it can have a negative impact on such technical components of the CDS as power lines, piping systems, etc., or on the social components of the CDS – places of residence and recreation of people. Today, one of the urgent tasks is the development of various, as a rule, highly computerized, means and methods of countering various threats based on forecasting the development of processes that form the basis of these threats.
This article discusses the task of countering one of the types of threats - flood, based on the physical process of raising the water level in water bodies, and capable of causing significant material damage to individual components of the CDS in the territory of the subject of the Russian Federation (for example, the Republic of Bashkortostan). To counter this threat, complex technical systems with an increasingly significant digital component are created and intensively developed, which allow monitoring and predicting water levels in water bodies. These systems are based on the integration of modern technical means of obtaining the information necessary for monitoring (measuring the level and temperature of water and air, water flow rate, etc.) and modern information technologies for processing this information for forecasting (analysis of large, including poorly structured data; distributed databases data; the Internet of things (or the Internet of everything); artificial intelligence, etc.) and can be classified as cyberphysical systems.
Traditionally, the water level is measured at stationary hydrological observation posts (gauging stations) of the regional department of hydrometeorology and environmental monitoring (Bashhydromet) and planning of measures to counter the flood threat is carried out on the basis of the forecast of the water level according to these data (for these posts). Recently, water bodies have additionally begun to install automatic water level monitoring posts equipped with video cameras for early detection and recording of a sharp rise in the water level, which is dangerous for the components of the CDS. It would be logical to use the values of water levels obtained from automatic posts as additional data for forecasting water levels in water bodies, however, there are no historical data needed for forecasting.
There are various approaches to solving the problem of predicting water levels in
water bodies, based on the analysis of hydrogeological and hydrodynamic parameters
of the state of water bodies [
Intelligent Data Analysis for Forecasting Threats in Complex Distributed Systems 3 2
The use of artificial intelligence technologies to predict the levels of water rise during the spring flood One of the main parameters of the possible negative impact of the flood situation on various components of the emergency control system is h – the water level in water bodies, measured daily at n stationary hydrological posts by Bashhydromet employees. We introduce the notation: ℎ is the value of the water level measured at the k-th hydrological post on the i-th date of the j-th year. Here = ⃗1⃗⃗,⃗⃗⃗ , where n is the number of hydrological posts involved in the calculations, j is the number of the year, i is the specific measurement date. At the stage of making a short-term forecast of water levels, the forecasting task is to calculate the value of the level of water rise on the next i+1 day, that is, ℎ +1, for any = ⃗1⃗⃗,⃗⃗⃗ on a specific current i-day of measure ment. years.
To solve this problem, it is proposed to use the results of previous measurements of the level of water rise ℎ
at all stationary hydrological posts located in the territory under consideration in the same climatic and hydrological conditions for all previous
In each specific territory (including the Republic of Bashkortostan), various water bodies can be in different conditions that affect the nature of the development of the flood situation. For example, in one part of the territory the river flows in the mountains, and in another part – along the plain. Also, on the same territory, different rivers can belong to different basins with different hydrological and climatic characteristics. Therefore, to predict the water level at a particular gauging station, it is logical to use data on water levels at those hydrological posts that are in the same conditions with it and the number of hydrological posts participating in the forecasting can be less than n. Determining the uniformity of conditions in which hydrological posts are located is an independent scientific task, and in this article, for simplicity of presentation of the forecasting method, but without violating the generality of reasoning, we assume that data from n hydrological posts are used.
A similar remark must be made with respect to the possible values of the index i denoting the date of measurement. In fact, the value of the date itself for forecasting is not significant, since in different years at different posts the rise in the water level and the return of the water level to the normal value occur on different days. The serial number of the measurement since the start of the flood is significant. The number of measurements, which corresponds to the duration of the flood situation, for each fixed pair of values of the indices k and j is also different, therefore we introduce its notation . In the framework of the comments made, we denote the entire set of previously measured water level values as 1 = {ℎ }, = ⃗1⃗⃗,⃗⃗⃗ ; = ⃗1⃗⃗⃗,⃗⃗⃗⃗ ; = ⃗1⃗⃗,⃗⃗⃗⃗⃗⃗ , (1) where m is the number of years of observations. But in the current (m+1) th year, by the time the flood starts and during the flood, measurements are also taken and can be used to predict the values of the water level on each specific day on the next ( 0 + 1) th day, therefore into consideration another set
2 = {ℎ +1, }, = ⃗1⃗⃗,⃗⃗⃗ ; = ⃗1⃗⃗,⃗⃗⃗⃗0.
In this case, of course, it is important to choose the day the flood starts, from which the forecasting starts (that is, to which day i=1 corresponds). This article is not considered in this article, since it affects the specific values of , the number and list of elements of the sets H1 and H2, which in turn affects the time for calculating forecast values, but does not affect the method and forecasting algorithm itself.
It is also necessary to introduce an additional notation for the predicted value of the water level – hp, since in the future, to evaluate the effectiveness of forecasting methods, it will be necessary to use the notation h and hp together.
Based on the introduced notation, predicting the water level in the current m+1 year at a fixed hydrological station 0, 1 ≤ 0 ≤ at some fixed point in time 0 will be described one day in advance by a certain function of the set of previously performed measurements: ℎ 0
+1, 0+1 = ( 1, 2).
There are many approaches and methods for constructing this function, using all or
part of the H1 and H2 data, including various statistical [
Currently, the so-called artificial intelligence methods and, in particular, the
methods of constructing artificial neural networks (ANNs) are widely used to solve various
problems [
At the first stage, which is carried out before the development of the flood situation, the parameters are selected and ANNs are trained based on existing values from the set H1. The result of the training are the values of the weight coefficients of the synapses, which are subsequently used for forecasting. At the second stage, the forecast values of ℎ +1, 0 are calculated daily for all n observation posts ( = ⃗1⃗⃗,⃗⃗⃗ ) using a trained ANN.
Since ANNs only process data that varies in the range [
Normalization
H1
Synapse weights
ANN parameters
ANN training
Synapse weights Trained
ANN
Intelligent Data Analysis for Forecasting Threats in Complex Distributed Systems 5 1 st stage DB
H1 2nd stage
Current values
H2
Normalization
H 2 hp Denorma
hp lization
Flood control measures
Subsequently, normalized values of ℎ⃗ are fed to the input of the ANN both at the training stage and at the forecasting stage, as a result of which the result (predicted value) is also normalized. Therefore, before applying the predicted values to counter the threat of flood, they are denormalized (reduced to the usual values of the water level) in a ratio that is the opposite (4):
ℎ = ℎ⃗ ⋅ (ℎ − ℎ ) + ℎ . 3
Analysis of the effectiveness of the proposed approach
for predicting water levels at stationary hydrological
posts
To predict water levels, the authors developed the software module “Forecaster” [
Every day, since the beginning of monitoring the development of the flood situation (in 2020 it was April 18, that is, i=1), using the Forecaster program, the forecast for the next day was made for 3 hydrological posts (that is, n=3; this value is n taken to reduce the time of the experiment), that is, it was determined ℎ +1, +1 = ( 1, 2).
(5) (6)
The next day, the actual value of the water levels at the same stations ℎ +1, +1 was measured and the predicted value was compared with the actual value based on the value of the relative difference, which most often characterizes the forecast accuracy: = (ℎ +1, +1−ℎ +1, +1) , ℎ +1, +1
2 = , +1 =1 1
∑ , +1 , = 1 ∑
=1 . for all hydrological posts
= ⃗1⃗⃗,⃗⃗⃗ . Carrying out forecasting daily until May 17 (that
is, , +1 = 30), we obtain 30 forecasts for each post and actual values of water
levels for each of n posts, which makes it possible to determine the average forecast
accuracy for each k-th post:
and for all posts for the entire forecasting period in 2020:
In fig. 2 shows the actually measured and predicted water levels in 2020 at one of the
observation posts. The calculation of the accuracy of the forecast showed the value of
average accuracy for each post E1 = 2.900%; E2 = 3.511%; E3 = 3.560%, and the
average forecast accuracy for the entire flood period of 2020 is E = 3.323, which
corresponds to the forecasting accuracy by other methods [
Intelligent Data Analysis for Forecasting Threats in Complex Distributed Systems 7 4
Using data from automatic posts to predict the level of
water rise
In connection with the emergence of new technical capabilities for automatic
measurement of the state of the CDS (water level, temperature, wind speed and direction,
etc.), automatic stations for measuring and recording water level in water bodies have
recently been increasingly used [
Figure 3 shows the location of stationary hydrological posts (34 objects) and
automatic posts (38 objects) at the water bodies of the Republic of Bashkortostan [
Downstream hydrological post Krasnaya Gorka village (Ufa river) Krasnaya Gorka village (Ufa river) Andreevka village (Belaya river) Lyakhovo village (Urshak river) Sterlitamak city (Belaya river) Sterlitamak city (Belaya river) Meteli village (Ay river)
Khmelnitsky) Sterlitamak (Vodolazhenko St.) Bolsheustikinsky
station, n. pool (r. Ufa) Pavlovskaya hydroelectric station, n. pool (r. Ufa) Birsk city (Belaya river)
river) Novofedorovskoe village (Ashkadar river) the village of New Otradovka (p. Sterlya) Lakly village (Ay river)
Intelligent Data Analysis for Forecasting Threats in Complex Distributed Systems 9 Denote the total number of automatic posts located between two stationary posts by na. In our case, today na=7, but over time, automatic posts can be closed, moved to another place or liquidated, so na, generally speaking, is a variable. Since water level measurements at automatic posts (real and interpolated) were not included in the definition of the set H1 by relation (1), we denote ℎ as the interpolated estimated value of the water level at the location of the k-th automatic post, = ⃗1⃗⃗,⃗⃗⃗⃗⃗⃗ , in the i-th day of the j-th year, where = ⃗1⃗⃗⃗,⃗⃗⃗⃗ , = ⃗1⃗⃗⃗,⃗⃗⃗⃗⃗ , as before. Actual measurements at stationary posts adjacent to the selected kth automatic post are included in the set H1, but the order of the numbers of these posts in the set H1 and in table 1 do not coincide, therefore, to automate the calculations, we introduce additional notation. We denote by KD the set of numbers of stationary gauging stations located downstream, and by KU the set of numbers of stationary gauging stations located upstream of the corresponding automatic post so that and the automatic post with number k is located between two stationary posts with numbers kdk and kuk.
In the calculations (interpolation) of the ℎ value, the distance between the automatic and neighboring stationary gauging stations is involved, therefore we introduce the variable x, which denotes the distance (distance) of the corresponding post from the river mouth, and, generally speaking, the water level at any point of the river can be considered as a function from this distance: ℎ = ℎ( ). (11) For each k-th stationary post, this distance is a fixed number xk, = ⃗1⃗⃗,⃗⃗⃗ ; accordingly, for automatic posts this distance is denoted by xak, = ⃗1⃗⃗,⃗⃗⃗⃗⃗⃗ . By virtue of the notation introduced, the location of some automatic post between two stationary hydrological posts is described by the relation (10) (12)
< < , ℎ = ℎ ( ), = 1⃗⃗⃗,⃗⃗⃗ , ℎ = ℎ ( ), = ⃗1⃗⃗,⃗⃗⃗⃗⃗⃗ .
In the general case, this dependence has the same nature, defined by relation (11), but different notations are introduced to explain the interpolation algorithm for stationary and automatic posts.
Assuming that the change in the water level in the river between two points (the locations of gauging stations) occurs linearly (which is quite acceptable for small distances), the interpolated value of ℎ is on a straight line connecting the water level points at neighboring stationary posts and is calculated (in accordance with known rules Euclidean geometry, see Fig. 4) by the relation
ℎ = ℎ + (ℎ − ℎ ) ⋅ −− . (13) hkuk
ji ha kji hkdk ji h, m 0 xkuk xak xkdk x, m
Fig. 5 shows an example of interpolation of the values of the level of water rise for the automatic hydrological post “Red Key”, based on data from neighboring stationary hydrological posts – “p. Krasnaya Gorka (Ufa River)” and “Pavlovskaya Hydroelectric Power Station, n. pool (Ufa River) ". As an example, we use the historical data of water levels at these hydrological posts from 04.05.2009; water level at the post “s. Krasnaya Gorka (p. Ufa)” was 197 cm, at the post “Pavlovskaya hydroelectric power station, n. pool (p. Ufa) ” – 413cm. As a result of the interpolation for the “Red Key” automatic post, the water level values were obtained – 223 cm, which we will consider as historical data for this automatic post on 04.05.2009. The water level values thus obtained as a result of interpolation for each automatic post and for all the remaining dates will be considered historical data for this automatic post. The amount of these data is the same as the number of observations at neighboring stationary gauging stations. Now, the array of the estimated data can be used to predict water levels according to the method proposed in paragraph 2 using the ANN for all automatic posts included in table 1.
Intelligent Data Analysis for Forecasting Threats in Complex Distributed Systems 11 Solving complex socially and economically significant tasks for managing the CDS, which undoubtedly includes parrying such types of threats as floods, requires the integration of modern breakthrough technologies, such as data mining, distributed databases, the Internet of things and artificial intelligence, which are inextricable parts of a single process for obtaining and analyzing heterogeneous data on the state of the CDS and its individual components. The article proposes an approach that allows for the joint analysis and processing of data on water levels from automatic and stationary gauging stations to predict flood threats using artificial neural networks. The analysis of the results of applying the proposed approach for predicting the water level in the water bodies of the Republic of Bashkortostan during the 2020 flood showed its effectiveness, as it gives a fairly accurate forecast, and thereby allows the relevant services to quickly respond to this threat and take the necessary measures to counter it. 6
Acknowledgments The reported study was funded by RFBR, project number 20-08-00301.