A significant territory of our country is littered with explosive objects due to the war and the attack on Ukraine. Demining is a complex, dangerous and long-term process. Civilians and sappers are exposed to constant danger from these explosive devices.
The purpose of this study is to develop an information system for classifying explosive devices among other metal objects using detector signal analysis or GPR scans. The object of research is devices that allow detecting explosives buried in the ground. The purpose of this work is to develop an information system for the classification of explosive devices in the ground using detector signal analysis or ground-penetrating radar images. The object of research in this case is devices that allow detecting explosives buried in the ground. During the research, we must learn the principle of operation of these devices and find out what information they can provide about the object discovered in the ground. Metal detectors and GPR are often used for non-contact detection of mines or explosives. These devices differ in the principle of operation and sets of information provided about the found object. Metal detectors help detect various metals. They are of different types and functionality, but ground metal detectors are used for demining. They work on the basis of variable magnetic using a double coil, the windings of which are tuned to one frequency. The indicators are activated, which notify about the detection of a disturbance caused by eddy currents in the metal, when a metal object falls into the area of action of this coil [1,2].
The advantages of metal detectors include their relative ease of setup and lower cost compared to GPR. Also, they are versatile and convenient to use, which is why they are popular for demining tasks. However, they also have a number of disadvantages, namely:
• a dangerous explosive object that does not contain metal cannot be detected;
• difficult work on littered areas, because metal detectors react to almost all metals contained in the ground;
• they do not provide enough information about the object discovered in the ground, which makes it difficult to predict what exactly was discovered.
Another popular device for detecting dangerous objects in the ground is ground-penetrating radar. GPR makes it possible to obtain a soil scan without excavation or drilling. This allows the specialist to get an approximate view of all objects or rocks that are at a depth of several centimeters to several meters. GPR is one of the most advanced devices for soil scanning tasks and has various uses: detection of explosive devices, geological exploration before construction, for scientific research, in archeology, etc [3].
Devices that combine the properties of a metal detector and ground-penetrating radar are considered the most modern and perfect systems at the moment. The MDS-10 from the Minelab company is such a device, which is now actively used in many NATO armies and in Ukraine. It combines metal detector and ground-penetrating radar technologies to detect all metallic and nonmetallic targets, in various soils and climatic zones. It has continuous real-time display functionality with the ability to pause the GPR scan for more accurate identification of a potential target [4].
To develop an information system for the classification of dangerous objects, we will use the input data received from one of the listed devices: a metal detector, ground-penetrating radar or their combined hybrid.
The set research problem can be solved by machine learning algorithms [5][6][7][8]. The developed software will be optimized for use in Ukraine and forecasting accuracy will be improved compared to existing systems.
There are many different methods for demining nowadays. For example, manual demining can be used, or heavy specialized armored vehicles can be used to defuse explosive devices on site, or small robots or drones can be used.
For example, methods of applying machine learning methods for detecting mines using drones are considered in the paper [9]. Recent advances in drone-based remote sensing using lightweight multispectral and thermal infrared sensors allow rapid detection of landmine contamination over a large area and map surveys. In their publication, the researchers focused on describing the development and testing of an automated technique for the remote detection of anti-personnel mines and the identification of scattered anti-personnel mines during large-area surveys. A methodology was proposed for the detection of scattered plastic mines, the construction of which uses liquid explosives encapsulated in a polyethylene or plastic case. This makes it impossible to detect such explosives using a metal detector, but analyzing images of the earth's surface using artificial intelligence methods can help solve the problem. The study analyzes multispectral and thermal data sets collected by an automated drone survey system containing scattered PFM-1 landmines as test objects and presents the results of an attempt to automate landmine detection by relying on supervised learning algorithms using faster regional convolutional neural network (Faster R-CNN). An RGB visible light demonstration of Faster R-CNN showed a test accuracy of 99.3% for the partially barred test set and 71.5% test accuracy for the fully barred test set. In multiple test environments, the use of centimeterscale, precisely georeferenced datasets combined with Faster R-CNN enabled accurate automated detection of PFM-1 test landmines. This method can be calibrated for other types of cluster antipersonnel mines in future tests to aid humanitarian demining initiatives. As millions of remnants of PFM-1 and similar plastic mines are scattered in post-conflict regions in large quantities and over large areas, they pose a long-term threat to the lives of people in these areas.
In the article [10], the researchers described the relevance of this topic and suggested using convolutional neural networks to solve the task. This article proposes a technology for detecting buried explosive objects. The proposed solution uses a special type of CNN, known as an autoencoder, to analyze spatial data acquired by GPR. Experiments conducted with real data show that the proposed technique does not require special data preparation and preprocessing to achieve accuracy above 93% in complex data sets. This study correlates with ours because our work also uses this type of convolutional neural network, but we have improved the accuracy.
In the article [11], scientists proposed to use CNNs, which are applied based on images obtained from ground-penetrating radar images. The proposed algorithm is capable of recognizing whether the examined soil surface contains hidden explosive devices. Validation of the presented system is performed on real GPR data, although system training is performed relying on artificially generated data. The results show that 95% accuracy can be achieved. The purpose of the proposed system is to determine whether a picture of the soil thickness obtained with the help of GPR contains signs of the presence of dangerous devices there. The proposed method is able to work on small fragments of the image with high accuracy, which allows for sufficiently accurate localization of the target. Experimental results confirmed the idea that CNN can learn from artificially generated data. However, by adding some background GPR data to the training images, the detection accuracy can be greatly increased. However, the system does not necessarily need to be trained on images depicting specific real-world objects. This characteristic is most important for the scenario of detecting hidden explosive devices. Thanks to this approach, it is possible to detect mines that were not in the training dataset before. This work is generally highly competitive with a high accuracy rate of around 95%, but the intelligent system we developed should be more versatile when processing new data and for detecting mine types that were not encountered in the training datasets.
The authors of the article [12] describe the methods of analyzing the ground-penetrating radar signal for the classification of mines that may be in the ground. This study describes a technology for multipolarization-based mine detection: a ground-coupled GPR platform that provides adequate performance without compromising operational safety by incorporating a flexible ballistic shield to support potential explosions. Experimental results have shown that the protective shield can not only absorb the impact of fragments caused by the explosion, but also allows obtaining data with the accuracy necessary to create a correct three-dimensional reconstruction of the underground layer. The resulting GPR data is then processed using a CNN capable of detecting hidden objects with a high degree of accuracy. Compared to this study, our information system should be more versatile on new data and detecting explosive devices that were not present in the training data sets.
The article [13] describes the development of a new dual-modality metal detector, which integrates georadar spectroscopic metal detection using GPR. This paper presents a feature-level sensor fusion strategy based on three features derived from two sensors. This article shows how data from the two components can be combined together to enrich operator feedback. The algorithms presented in this work are aimed at automating the location of hidden objects. The described system is also able to collect information that can also be used for potential classification of such items.
The article [14] describes the development of algorithms for more accurate classification of small metals in mineralized soils. Detecting small metal objects buried in mineralized soil is a challenge for metal detectors. This paper describes a new, portable MIS-based system that can be used to detect buried metallic objects even in difficult soil conditions. Experimental results consisting of 1669 passes through hidden objects or empty ground are presented. Fourteen objects were buried at three different depths in three soil types, including unmineralized and mineralized soils. A new processing algorithm is proposed to demonstrate how spectroscopy can be used to detect metallic objects in mineralized soils based on MIS data. The algorithm is robust for all soil types, objects, and depths used in this experiment, achieving a true positive rate of over 99% with a false positive rate of less than 5% based on just one pass over the object. It was also shown that the algorithm does not need to be trained separately for each soil type.
The technical and physical part of the work of metal detectors for detecting explosive devices is presented in the study [15]. The combination of metal detector and GPR to improve the detection of anti-personnel mines is also investigated in this paper. A number of problems arise due to the close integration of these two types of detection. One problem is the proximity of metal GPR antennas to metal detector coils, which affects the performance of the metal detector due to eddy currents created in the structure of the GPR antenna. This article examines the impact of a traditional solid butterfly antenna design on a metal detector, as well as ways to reduce eddy currents.
In our research, we are developing a system that, based on a detector or GPR signal, will classify objects in the ground and predict with what probability an explosive device is located there. Of course, similar systems already exist, but there are few of them, as well as open access scientific publications on this topic. However, our system is adapted to Ukrainian needs as much as possible and is based on research [16], where a detailed analysis of the electromagnetic properties of soils in Donbas is carried out and the approaches that are most optimally used when demining such soils are described.
This study uses the dataset described in [10]. The researchers collected this data set and uploaded part of it to their GitHub repository [17]. This dataset contains GPR data. This allows for higher accuracy when training the network, compared to using data from metal detectors.
To create this dataset, 8 different objects were buried in a plot measuring 1 by 2 meters. There were: 3 real mines PROM1, DM11 and PMN58, a grenade, a metal sphere, an aluminum can, a plastic bottle and a plant root. As a result of the rain that fell a few days before the experiment, the sand was not completely dry, providing a propagation speed of 10 cm/ns and a resulting dielectric value of 9. The consequence of this is that the medium can not be considered strictly homogeneous, since the upper layers of sand were more dry compared to the lower layers. This helped to provide the experiment with conditions even closer to real ones.
Figure 1 shows a sample of images that the dataset contains and that will be processed by our neural network.
It can be seen from Figure 1 that GPR does not allow to obtain a clear picture of the soil section. For example, it is something similar to the work of a locator or ultrasound images of a person. That is why only highly qualified specialists who have learned to understand well what is visible on such images can work with GPR to detect dangerous explosive devices. However, the use of artificial intelligence tools can simplify this task for people or make it possible for robot sappers to work independently without the participation of this operator who would remotely control it in manual mode.
We have provided Figure 2. and Figure 3. below to see the difference in images between those that have mines or other unknown objects and those that only have uniform soil with nothing. The first of these two pictures has a uniform color without noticeable significant anomalies. This shows that it was made on a homogeneous surface of the earth, which does not contain any objects.
However, in the second figure, certain "waves" and disturbances are already visible, which indicate an obstacle in the soil and indicate its approximate location on the time line. From the appearance of these disturbances in the picture, it is possible to find out whether it is a depicted cavity, some object or a rock. The algorithm must record all such places where an explosive device can potentially be located. From Figure 1 and Figure 2 above, the difference is noticeable even to an inexperienced person, but often the situation is more complicated, since various rocks, stones, foreign objects that are not mines may be present in the ground. soils are also of different types and moisture content with different dielectric permeability, which affects the quality and efficiency of GPR. That is why the algorithm must be able to recognize dangerous objects under different conditions.
The input data for training the network are the scans shown above. They can be presented in twodimensional space or in three-dimensional space. This makes the data set more versatile and allows us to train different types of networks on its basis and compare their effectiveness. This means that based on this dataset, we can develop a network that will work with 2D or 3D space and show how objects are placed in the real world.
Table 1 shows the main physical characteristics of the data from the dataset. It is convenient to use neural networks to solve the given problem. This will make it possible to build an information system for automatic recognition of dangerous explosive devices. The selected dataset requires working with images of scanned soil areas, so in this case it will be appropriate to use convolutional neural networks. This type of network effectively copes with tasks of this kind. There are many well-known examples of applications and scientific studies that show high performance of CNNs for solving problems in the field of computer vision.
The peculiarity of the task set in the work is that completely different types of dangerous explosive devices can be encountered here. These can be mines made in the last century, and modern secret mines that were not used before. Such a complete dataset simply does not exist, so an important requirement for choosing an algorithm and designing a system is the ability of this network to maintain its effectiveness on new types of mines that were not even in the training dataset. Of course, it will not be possible to verify this without practical experiments, but researchers from one of the analyzed works [18] claim that they managed to develop a model that has similar properties. In their publication, they describe a type of network built on the basis of an autoencoder. Therefore, we also chose the autoencoder architecture.
We used an incomplete convolutional autoencoder [19], which is characterized by a hidden representation of information of a reduced dimension relative to the input data, to solve the task. This autoencoder translates the original data into a smaller value space and then decodes it. It is appropriate to choose this type of neural network, as it can be used to investigate anomalies [20,21], data that do not correspond to the expected results. This is necessary in order to effectively detect types of explosive devices that were not even present in the training data set. The proposed algorithm is actually trained on those data and parts of GPR scans where there are no mines or explosive devices. When an area with a land mine appears on the image, it is fixed as an anomaly, and passing data through different layers of the neural network should make this anomaly more clear and obvious, so that the algorithm does not make a mistake. Such a clever approach makes it possible to detect, through anomalies, even those types of explosive devices that were not included in the training dataset and thereby make the algorithm more universal and suitable for use on the territory of Ukraine.
Figure 4. gives a more visual understanding of the principles of CNN and the autoencoder. There is a schematic representation of the general view of the convolutional network. The autoencoder is built on the basis of CNN and is somewhat similar to it and also uses convolutional layers, but schematically the autoencoder can be divided into two parts: the encoder and the decoder, as shown in Figure 5.
With this approach, it is possible to train the autoencoder on relatively small volumes of data obtained from ground-penetrating radar, which will not contain a complete set of data on possible explosive devices. After training, this autoencoder processes the entire amount of analyzed data. Then the value of anomalies is calculated for each scan. If an anomaly, which could be a hidden mine, is detected on one of the scans, the value 1 is assigned to the special label Ĵ, but if the magnitude of the anomaly is insufficient or there is no anomaly, then the label Ĵ equal to zero is assigned. As a result, all scans begin to be marked with Ĵ=1 or Ĵ=0. (1) where 𝑤𝑤 𝑖𝑖 and ŵ 𝑖𝑖 are the input and output data. This loss function is the root-mean-square error between the input data 𝑤𝑤 𝑖𝑖 and the output data of the autoencoder ŵ 𝑖𝑖 . If these two measurements come as close as possible to each other in value and the loss function decreases, then we can say that the autoencoder has learned to reproduce the received information correctly. In practice, the value of this function is almost always positive, but it can be zero if the network prediction is 100% correct.
For each block 𝑤𝑤 𝑖𝑖 of input data W, the Euclidean distance 𝑒𝑒 𝑖𝑖 is calculated according to (2). 𝑒𝑒 𝑖𝑖 = �ℎ 𝑖𝑖 − ĥ 𝑖𝑖 � (2) where ℎ 𝑖𝑖 and ĥ 𝑖𝑖 are already encoded data blocks by transforming ℎ 𝑖𝑖 =Ɛ(𝑤𝑤 𝑖𝑖 ) and ĥ 𝑖𝑖 = Ɛ(ŵ 𝑖𝑖 ). Using decoding functions, this hidden data layer can be decoded in the following way: ŵ 𝑖𝑖 = Ɗ(ℎ 𝑖𝑖 ).
This Euclidean distance 𝑒𝑒 𝑖𝑖 between hidden data blocks can be used as an anomaly detector. Because data blocks containing hyperbola features form a hidden data block ĥ 𝑖𝑖 , which is very different from ℎ 𝑖𝑖 . It also works vice versa, if there are two hidden blocks of data that are similar, it means that the data being analyzed is similar to the samples on which the network is trained.
In the data aggregation step, the main goal is to combine all the obtained e_i values to detect a volumetric anomaly M of the same size as the data W. This allows to find out which data W should be considered anomalies. These anomalies are likely to be hidden mines. The volumetric anomaly M is constructed by overlapping and averaging the data. Volumetric means that the data is taken in threedimensional space.
Formula (3) can be used to determine the set of indexes i of blocks w_i containing the sample W(t, x, y).
Ƒ 𝑡𝑡,𝑥𝑥,𝑦𝑦 = {𝑖𝑖 | 𝑊𝑊(𝑡𝑡, 𝑥𝑥, 𝑦𝑦) ⊂ 𝑤𝑤 𝑖𝑖 } (3) where, the notation ⊂ is used to express that the sample W(t, x, y) is contained in the data block 𝑤𝑤 𝑖𝑖 . Then the anomaly M can be determined by (4).
𝑀𝑀(𝑡𝑡, 𝑥𝑥, 𝑦𝑦)
where �Ƒ 𝑡𝑡,𝑥𝑥,𝑦𝑦 � is the power of the set Ƒ 𝑡𝑡,𝑥𝑥,𝑦𝑦 or its cardinal number. After calculating the magnitude of the anomaly M, we can proceed to check whether this anomaly refers to a mine. Formula (5) can be used to detect anti-personnel mines on a scanned image of the ground from an M-based GPR.
where D is a global threshold value that is chosen during system setup based on a set of GPR scans. If we detect a large anomaly during the analysis of the scans, then this scan is marked as suspicious for the presence of explosive devices there.
The AUC metric was chosen to assess the accuracy of the trained model. Formula (6) can be used to calculate the rate of true-positive solutions (TPR). TPR = 𝑇𝑇𝑇𝑇 𝑇𝑇𝑇𝑇+𝐹𝐹𝐹𝐹 (6) where TP is the correct prediction of the positive prediction and FN is the incorrect prediction of the negative prediction.
The number of false positive forecasts (FPR) can be calculated using formula (7). FPR = 𝐹𝐹𝑇𝑇 𝐹𝐹𝑇𝑇+𝑇𝑇𝐹𝐹 (7) where FP is the incorrect prediction of the positive forecast and TN is the correct prediction of the negative forecast.
The ROC curve shows the dependence of TPR on FPR. Lowering the classification threshold allows more items to be predicted as a positive prediction, thus increasing the number of false positives and true positives. But to calculate the ROC curve, it would be necessary to calculate the logistic regression model many times with different classification threshold values. This approach to calculations is complex and inefficient, so the AUC metric is used, which is equal to the area under the ROC curve. This allows us to apply integral calculus and makes it convenient to use.
We should use the activation function to bring the output data to a certain range of values. We chose the tanh function, or another name, the hyperbolic tangent function, which is calculated by formula (8).
tanh(𝑥𝑥) = 2 1+𝑒𝑒 −2𝑥𝑥 − 1 (8) Formula ( 9) can be used to calculate the derivative function of the hyperbolic tangent.
𝑖𝑖´(𝑥𝑥) = 1 − 𝑖𝑖(𝑥𝑥) 2 (9) Summarizing all of the above, the algorithm of the designed intelligent system can be described in the following steps:
• Prepared data is fed to the network input.
• Five convolutional layers of the network were created with decreasing filter dimension and image size at each convolution step.
• The encoder layer was created with the smallest dimension and pass images through it.
• Five symmetric deconvolution layers were created with a gradual increase in the dimension of the filter and with the return of the image to the original dimension.
• The decoder was created as the output layer of the autoencoder using the tanh activation function.
During the test runs of the developed software, a number of experiments were conducted regarding the selection of different configurations of models, different number of layers and their parameters. We conducted experiments to understand under which model parameters the best values of the metrics will be obtained. Numerical experiments were carried out with different network input settings. As a result, two designed networks with the best obtained results were selected for a more detailed comparison. Figure 6 shows basic information about the two designed networks.
The above data show that both models have similar characteristics. For example, each model has several convolutional layers and several decoder layers transposed to them.
Figure 7. shows information about the training epochs of the network. The data was divided into normal and validation and the main metrics were calculated precisely on the validation data for better reliability of the results.
As a result, 32 epochs were completed during the training of the first model, after which the EarlyStopping function was called, so the loss reduction and improvement of the model did not occur for five epochs before that. The course of training of this network and changes in the value of losses on the validation data with each epoch can be visually seen in Figure 8.
We can analyze the graph above and see that the rapid decrease in losses occurred up to about 15 epochs, and after 20 it almost stopped and the improvement was apparently quite insignificant and almost not visually noticeable. However, the effect of retraining also did not occur, since a rapid increase in losses is not observed and the function decreases along its entire length. There are a few small fluctuations, where losses could increase for a short time, but this is acceptable, because the test was carried out on validation data.
A similar graph of the dependence of losses on the number of epochs is also constructed for the second model (see Figure 9).
We can visually see that the training of the second model happened a little faster than the first. A total of 27 epochs were completed, which means that after the 22 After training the models, we checked their performance on test data. After testing, we derived the value of the AUC metric, which describes the accuracy of the prediction. For the first model, the AUC value is equal to 97.83%, and for the second trained model, AUC = 96.32%. Therefore, the configuration of building the first model is more successful for solving the given problem using this dataset. It can be concluded that the first model with five convolutional layers and five decoder layers should be used.
A large area of our country was mined and required a long and expensive demining process even after the 2014 invasion, but in 2022 the scale of the problem has greatly increased. It is necessary to implement the automation of the demining process and develop new or improve existing technologies to save human lives. The system for predicting or classifying mines in the ground with the help of machine learning can be used as a smart assistant built into metal detectors or GPR, or for sapper robots. This allows us to conclude that such systems will remain relevant in the coming years.
The autoencoder algorithm is selected. This algorithm is suitable for fast and efficient processing of input images and is flexible to detect explosive devices that might not have entered the training dataset. Therefore, this algorithm is optimal for training the system that will be used in Ukraine, where there may be many unknown types of mines. The paper provides a mathematical description of the algorithm, and describes the principle of calculating anomalies on GPR scans, which may be mines or explosive devices. But for maximum improvement and the most effective detection of dangerous objects, it will be worth retraining the model on a dataset collected using Ukrainian soils with their unique dielectric permeability. Unfortunately, there is currently no such dataset available. Therefore, we chose the most relevant dataset from those that were publicly available on the Internet. Also, we analyzed the principle of operation of metal detectors and ground-penetrating radars. We found out the advantages and disadvantages of each of these devices for demining. We investigated which of these devices provides more accurate information about the object detected in the ground and, accordingly, data from which device is better to use for training the model. After the comparison, we concluded that GPR or hybrid devices that combine sensors of both a metal detector and GPR are best suited to solve the given problem. Such hybrid devices are currently among the most modern. But they appeared relatively recently, and we did not find any datasets on the Internet that use data from such devices. Therefore, we used the data obtained from a conventional GPR to build a classification system. Pictures taken with the help of GPR are usually sufficiently informative for the detection and classification of dangerous explosive objects.
We used Python and Google Collaboratory to develop the software using the selected autoencoder algorithm and dataset. After that, we compared the results of the models with two different configurations. We conducted numerical experiments and found that the first of the two models gives better performance results (AUC is equal to 97.83%) based on the compared metrics.