The presence of foreign inclusions in physical objects may affect their physical properties. Existing research employs different techniques, but all of them rely on data that can be obtained either by conducting a realworld experiment or employing numerical methods. Both options are quite time-consuming and resourceconsuming. This study aims to propose a more optimal way to approach estimating both geometric and conductivity parameters using a more resource-efficient numerical method called the indirect near boundary method (INBEM). Using INBEM to generate the dataset makes it possible to consider a wider variety of objects and inclusion parameters. As more data is available, deeper research into preprocessing techniques and neural network models is becoming feasible. Thus, several CNN architectures and preprocessing methods were evaluated. The proposed preprocessing method improved geometric parameters prediction by up to 4 times compared to just normalization. The best-performing model (CNN3FD 2nd-order) achieved a mean absolute error close to 0.01. Thus, combining INBEM with CNNs and proposed preprocessing techniques proves to be an efficient method of inclusion localization. It established ground for future research aimed at detecting several inclusions at once and fine-tuning INBEM to increase accuracy in real-world experiments.
Neural networks, inclusion recognition, stationary process, convolution neural network, machine learning, non-destructive testing, potential theory, computational experiment.1
The presence of inclusions in physical objects has an impact on their properties and behavior. Thus, detection and identification of them is an essential task to be researched. From a mathematical perspective, such problems belong to the class of inverse problems, particularly in the modeling of stationary processes, and are part of potential theory in inhomogeneous objects.
Most studies focus on detecting defects in objects of clearly defined size [1, 2]. Developing models capable of recognizing inclusions in objects of varying sizes or even different shapes is much less common. Most of the recent studies focus on defects localization in an object [1, 3, 4], rarely on defects or media characterization [6]. T researched as geometric parameters estimation.
conductivity parameter estimation is not as well
To detect and localize defects, architectures like CNN or its descendants are often employed. Despite following quite similar ideas, the researchers employ different combinations of feature extraction methods or modifications of state-of-the-art models [6, 7]. A combination of GAN methods [3, 8] with deep neural networks is enabled as an alternative way to improve performance.
Despite having all the powerful tools employed, there is a persistent problem of building quite a large dataset with a wide variety of features. Data obtained during real-world experiments is highly valuable, but it is costly in terms of money and time to obtain. Using numerical methods, such as FEM [1], partially resolves the issue. However, such techniques require quite a lot of computational resources.
This paper proposes employing an indirect near-boundary element method [9] (INBEM) to generate the dataset. A model to estimate both geometric and conductivity parameters is proposed. A combination of feature extraction methods is proposed and tested out as well. As described in advance, conductivity estimation highly relies on accurate geometric parameters estimation. This paper mainly focuses on estimating geometric parameters as a key to approach accurate conductivity estimation.
The object of the research is the recognition of the physical and geometric characteristics of a foreign inclusion.
The subject of the research is neural network methods and tools for recognizing the characteristics of an inclusion.
The purpose of the research is to analyze methods and tools for recognizing the physical and geometric characteristics of a foreign inclusion.
The research novelty is combining neural networks with an efficient numerical method (the indirect near boundary element method) for solving inverse problems of potential theory.
In paper [1] finite element simulation is employed to generate synthetic data. To augment data brightness, contrast, exposure and noise adjustments were employed. To preprocess data Sobel, Prewitt and Roberts operators were enabled. Transfer learning was considered to fine-tune state of art models.
The research [2] aims to recognize defects with different sizes and depth of placement. It employs both real-world experiments and FEM simulation to generate synthetic data. The defect was considered as holes with circles of different diameters and depths. To preprocess data, polynomials and Padé approximation were employed. The proposed method enables using LSTM architecture.
In paper [3] IRT-GAN architecture was proposed to detect defects. The architecture uses U-Net as a generator and PatchGAN with GlobalGAN as a discriminator to improve segmentation accuracy. To improve noise suppression, Spatial Group-wise Enhance Layer is employed. The dataset was obtained by numerical experiment. To improve model generalization data augmentation is employed by altering image rotation, scale and noise level.
The research [4] proposed a method that consists of data compression to reduce data size by edge detection. Defect detection is achieved by computing thermal contrasts, maximum amplitude and time constants. A change in heat flow direction is considered to improve precise defect localization.
In research [5], a real-world experiment was conducted. Thermal feature extraction is done by considering thermal amplitude, time constant and thermal constant. Defect edge highlighting is done by computing derivative and processing it using Canny edge detection, applied thresholding before. Thermal features were mapped into an RGB image to make defect visualization clearer.
The paper [6] considers such defects as flat-bottom holes and Teflon inserts in plexiglass, carbon fiber reinforcement polymer and steel. An experiment was conducted to build a dataset. Data augmentation was employed too. This paper proposes using PCT (Principal Component Thermography) to extract features. It uses YOLOv3 and Faster-RCNN to detect defects. U-Net and Res-U-Net were employed for semantic segmentation. Instance segmentation is done by Mask-RCNN and Center-Mask.
In paper [7], the proposed method uses Principal Component Analysis (PCA) to enhance defect features, deep convolutional neural networks with Region of Interest (ROI) to extract features and Faster-RCNN to detect objects. The dataset was built by conducting a real-world experiment. Both PCA and data augmentation were employed.
The paper [8] aims to address the limitations of PCT. It proposes using generative artificial neural networks (GANs) to enhance data. The proposed method employs using GAN before applying PCT. Specifically, it employs Spectral Normalized GAN and Convolutional Autoencoder. The real-world experiment was conducted to scan panel painting with artificially induced defects.
In paper [10], an adaptive wavelet neural network is employed to detect defects by using a twostep approach: it reconstructs the 3D point cloud, then it enhances the defect area. As a result, the proposed approach outperforms traditional Terrestrial Laser Scanners (TLS) based crack detection.
The paper [11], contrary to other papers, focused on crack detection based on photos of concrete objects. The dataset consists of 990 RGB images. It proposed an approach of using transfer learning. It estimated the performance of a few well-known models, such as ResNet18, GoogLeNet, MobileNetV2 and VGG16. The best performance was shown by ResNet18.
A method of continuous laser-based active thermography was proposed in the paper [12]. A realworld experiment was conducted. Except for the experimental tools, it proposes using thermal sequence processing by pixel-wise averaging. It states that Roberts filtering performed better than the others.
The research [13] aims to use infrared thermography to monitor delicate cultural heritage objects. The preprocessing is done by contrast enhancement and noise reduction. It proposes using Mask RCNN to detect defects.
In paper [14], improved Faster-RCNN is employed for crack detection. It uses VGG-16 as its backbone by employing transfer learning. The dataset was built by conducting a real-world experiment. It consists of 3125 infrared thermal images of steel plates.
In paper [15], machine learning algorithms, such as linear regression, multilayer perceptron and a few more, were compared to characterize internal defects in additive materials. The features considered by the models were conductivity coefficient, max heating temperature, front contrast, rear contrast and other physical parameters.
The paper [16] employs hybrid CNN in combination with a physically informed neural network (PINN) to reconstruct temperature distribution for 3D thermal tomography. The finding of the paper is that using standard CNNs fails to reconstruct a distribution under real-world noise. Besides that, a combination of CNN with PINN achieved the best result in processing noisy data. 3. Solving the direct problem of potential theory for a piecewisehomogeneous body Before developing an approach to solving the inverse problem, it is necessary to obtain a series of datasets corresponding to the solutions of the direct problem with varying geometric and physical characteristics of the inclusion. The mathematical models of steady-state processes, constructed for methods based on the use of natural and artificial potential fields (gravitational, magnetic, electric, thermal, filtration) to detect inhomogeneities within an object, consist of Laplace equations: (0 )( ( )( )) = ( )( ) = 0, ∈ , ⊂ 2, = 0,1, supplemented with prescribed boundary conditions:
(0)( ) = (
(0)( ) = (
(
(
(
The boundary integral equations (
To solve the boundary value problem (
(
The external near-boundary regions are discretized into near-boundary elements (NBEs) ( = 1, . . . , ) and families of points ( = 1, . . . , ) are introduced on them. For each unknown functions ( ) sources placed at the points .
Since for the Laplace operator (0 )
( ( )( )) the known fundamental solution (FS) ̆ ( )( , )
exists, the integral representations of the solutions to equations (
( ) are introduced, which describe the collective behavior of fictitious ∑
=1 ̆ ( , 1) = 0, x R2m , where ̆( )
( , ( )) = ∑ =1 ( )( , 0 ).
Having determined the unknowns and Cm from (
(
(
̆1( 1 , ̆ (
( , ̆ ( )) + ( , ̆ ( )), where is the set of observation points ∈ ∪ .
It is worth mentioning that the use of point families on NBEs made it possible, unlike in the BEM,
to avoid prior analytical extraction of singularities (principal values) when computing the normal
derivatives of the desired functions. Point families significantly simplify the problem-solving
algorithm, as they replace integration over NBEs with the summation of products of the fundamental
solution and the unknown (
( ∗ ) . The
The dimension of the generated data is 100 * 100 to make both the dataset and model more generic. The object heat data is put at the very beginning of the distribution array. The rest is filled up with zeros as the padding.
In addition to temperature data, each dataset element includes information about the coordinates of the lower-left corner of the inclusion, its height, width, and thermal conductivity coefficient. The data is stored in JSON files, each of which represents information from a computational experiment with specific variable parameters. The dataset consists of 4 subsets described in detail in Table 1. To increase the variance of geometric parameters, the variability of the conductivity parameter is quite poor. The common parameters of the subsets are: • • • • • thermal conductivity coefficient variability: 4.2, 5.9, 8.6, 12.9, 14.6; a number of fictitious point sources in each NBE: 5 rows, 5 columns, a total of 25; number of NBEs on the contact boundary: equal to the width/length of the inclusion; height of the NBE at the object's boundary: 1; height of the NBE at the contact boundary: 0.5.
X/Y coordinate
Number of NBEs
Total elements 0.6 to 8.1, step 0.5/ 0.6 to 8.1, step 0.5 0.6 to 8.1, step 0.5/ 0.6 to 3.1, step 0.5 0.6 to 3.1, step 0.5/ 0.6 to 8.1, step 0.5 0.6 to 6.1, step 0.5/ 0.6 to 2.1, step 0.5 10 per side, total 40 10 * 5, total 30 5 * 10, total 30 8 * 4, total 24 91,125 13,500 13,500 3,465 To extract informative features from the heat distribution data gradient approximation and binarization are employed. All of them are aimed at detecting the edges of the inclusion. As the input data may contain paddings the representable part is extracted based on the object dimensions provided. Derivative approximation is considered as a basic method [19, 20]. To approximate derivatives the forward difference, a combination of forward, central, and backward differences were considered. As the heat distribution data is present as a 2D array then its derivative is calculated in the following way: | | = √(
A combination of forward, central, and backward methods (CD) is taken to achieve a better
quality of approximation. The formulas for calculating central and backward differences are the
following:
( + △ ) − ( − △ ) (
≈ ,
2 △
( ) − ( − △ ) (
≈ .
△
The central difference method (
As the second derivative represents acceleration in terms of heat distribution, it allows identifying clear corners and edges of the inclusion [19]. Of course, noise presence may be considered as parts of false edges or contours. However, we are not considering noise robustness in the scope of this work and it is the point for future research. A visual representation of the preprocessed thermal data using the forward difference method is shown in Figure 5. The red-highlighted areas likely represent the contours of inclusions.
Visual results of using a combination of forward, central and backward differences to approximate the second derivative are given in Figure 6.
As shown in the previous figures, the contours are visually identifiable. However, there is a presence of peaks at the edges of the inclusion. Some parts of the contour are quite thick. The reason is partially lying in the lower accuracy of the INBEM on the corners. To resolve it, quite a fine-tuning is required to optimize accuracy at the corners; it is the point of the upcoming research.
Aiming to receive a strict contour of the inclusion, binarization is also employed. The threshold to choose between 1 and 0 is determined by taking the mean of the heat image distribution. Visualization is shown in Figure 7. 6. Method of estimating both geometric and conductivity parameters The model of processing heat distribution data is visualized in Figure 8. As the input the model takes distribution data. Considering the input dimension as 100 * 100, to support objects with different sizes and shapes the model also requires dimension data. It is necessary to extract the object heat data and do not consider paddings as it might lead to less accurate estimation. Then the features are extracted using already described techniques. The next step involves a neural network model to predict the geometric parameters of the inclusion. This data is required to proceed with extracting conductivity features from the distribution. A neural network estimates conductivity. As a result, both geometric parameters and conductivity are estimated.
As is clearly shown, estimating conductivity relies on precise geometric data estimation. The main goal of this paper is to validate feature extraction techniques first as a tool to improve the precision of the neural network. At the same time, straight-forward neural network models are enabled to achieve the next goals: do not overfit the model, as the large one might just remember the data instead of generalizing it; start building the model from scratch, increasing complexity only when it is required as a complex one will also require more computational resources.
Despite the paper aiming to improve geometric parameters estimation, the method of conductivity estimation is provided in advance. The conductivity features extraction method consists of the following steps: identify a horizontal segment that runs along the entire object precisely through the center of the inclusion; determine the gradient at four points before the contact boundary and at four points beyond the contact boundary. • • • •
The method reduces the data dimensionality to 1D and enables clearer extraction of a greater number of informative features. The neural model architecture for estimation inclusion conductivity is visualized in Fig. 9. We assume that the main conductivity is equal to 1 and do not accept it as an input parameter. In the future, object conductivity also must be considered. The proposed model includes three main stages: • • deep object and inclusion feature extraction from the already preprocessed data; concatenating features; •
conductivity parameters estimation based on concatenated features.
As of the early stage of the research, the inclusion detection problem is considered a simple regression task that predicts a bounding box for a single object. We neglect the fact that there might be no inclusion or more than one. This is the purpose of the upcoming research and will be resolved by enabling a bounding box or anchor-free methods to the model. For the research purpose CNNs [21] with several convolution layers from 1 to 3 were used. We call those models CNN1, CNN2, CNN3.
The CNN1 network consists of one convolutional layer followed by a pooling operation and two fully connected (dense) layers. The architecture is as follows: • • • • • • • •
Input Layer: the model accepts a single-channel input image of size 100×100.
Conv2D: 16 filters of size 3×3, a stride of 1 and padding of 1.
Activation: ReLU.
Max Pooling Layer.
Flattening Layer.
Fully Connected Layer.
Activation: ReLU.
Output Layer: 4 output values representing x, y, width, height.
The architecture of CNN2 is the following: • • • • • • • • • • • • • • • • • • • • • • • •
Input Layer: the model accepts a single-channel image input of dimensions 100×100. Conv2D: 16 filters of size 3×3, a stride of 1, and padding of 1.
Activation: ReLU.
Max Pooling Layer.
Conv2D: 32 filters of size 3×3, stride 1 and padding 1.
Activation: ReLU.
Max Pooling Layer.
Flattening Layer.
Fully Connected Layer.
Activation: ReLU.
Output Layer: 4 output values representing x, y, width, height.
The CNN3 architecture is the following:
Input Layer: The model accepts a single-channel image of dimensions 100×100 as input. Conv2D: 32 filters of size 3×3, with stride 1 and padding 1.
Activation: ReLU.
Max Pooling Layer.
Conv2D: 40 filters of size 3×3, stride 1, and padding 1.
Activation: ReLU.
Max Pooling Layer.
Conv2D: 48 filters of size 3×3, stride 1, and padding 1.
Activation: ReLU.
Flattening Layer.
Fully Connected Layer.
Activation: ReLU.
Output Layer: 4 output values representing x, y, width, height.
The derivatives of first and second order, in combination with binarization techniques, are employed in this experiment to preprocess data and extract representative features. The preprocessed data is considered as an input for the already described neural network models. The limit of epochs is set as 100. To regularize the model, such techniques as early stopping and restoring best weights are employed. Each of the neural network models is tested out without any preprocessing technique but normalization.
The following metrics are taken to measure the performance of the models: mean absolute error (MAE) and mean square error (MSE). Some of the most representative results are presented in Table 2. Analyzing them there are next findings. 1. CNN with just normalization showed the worst results. Basically, preprocessing improves metrics by 2-4 times considering the same neural network architecture. 2. The combination of forward, central and backward derivatives of both the first and second derivatives showed quite a close impact on the performance. In some specific cases, forward differences outperformed it by around 10%. It is clearly shown for CNN with 3 convolution layers. However, considering the metrics for both CNN with 1 and 2 convolution layers, the difference is quite inconsistent. It might be stated that using both preprocessing techniques at the same time makes sense for future research. Thus, it means that the neural network architecture might be modified to be multi-channel in terms of input. 3. As for the previous point, using second-order or first-order derivatives had an impact mostly visible for CNN with 3 layers. The increase in performance, generally, does not exceed 10%. 4. U threshold-setting method than using just the meaning of the image. The method is not noiseresistance. We are speaking about the data that, in real-world scenarios, will be obtained by an IR scanner. Thus, the noise as Gaussian or similar will be present and binarization might provide many false contours as a result.
The following improvements and future research aspects are listed below.
Considering the specific nature of the INBEM method, the accuracy of potential calculation at the edges of the inclusion might be low. The parameters of INBEM, considering the scope of the paper, were looked up empirically and presented in the paper. The execution time was the main goal. The analysis showed quite moderate accuracy in specific cases, mainly with
the large inclusion. Thus, fine-tuning as an optimization task might be successfully done by a genetic algorithm (GA) or different techniques.
Considering the improvements in accuracy by using all the derivatives-based methods, it makes sense to use them for future investigation. It might make sense to use multi-channel input to combine all the positive aspects of the methods. At the same time, the idea of building ensembles of the models is worth considering.
The future research should include evaluating the already mentioned preprocessing methods with noisy data. Denoising methods should be considered too.
As the future aims to process real-world data, which are quite noisy, the preferred architecture for the model will be relying on CNN with 3 or more layers, as it is not just more accurate but extracts low-order features.
The results presented in the paper proved that INBEM, in combination with data preprocessing methods and neural networks, can solve the task of recognizing inclusion parameters. The numerical results showed that using second-order derivative preprocessing improved the prediction accuracy of geometric parameters by up to four times. The CNN3+FD (2nd-order) model showed the best performance, achieving a mean absolute error of less than 0.01, while CNN3+Normalization showed a mean absolute error between 0.019 and 0.03. It showed the perspective of using this combination of methods for estimating specifically the geometric parameters. Using such methods looks promising in terms of future research, aiming to be able to recognize multiple inclusions in 2D and 3D dimensions. As is stated, using INBEM as a tool for numerical experiments makes it feasible to produce large datasets in a reasonable time. Thus, it allows exploring new techniques and models of inclusion detection, preserving a wide variety of parameters of inclusions and the object.
The proposed approach, however, has certain limitations. It demonstrated lower accuracy of INBEM in corner regions. Thus, fine-tuning and regularization techniques look promising as a solution and must be considered in advance. The neural network model was validated only for stationary (steady-state) heat conduction and was not tested for transient (non-stationary) processes. In addition, the method is not yet noise-robust and requires further improvement in this direction. Also, the method was validated only for one inclusion detection task.
Thus, future research will aim to overcome these limitations, to extend the method towards recognizing multiple inclusions, and to perform simulation and detection in 3D space. Another important direction will be improving the robustness of the method when applied to noisy and realworld data, as real experiments typically introduce significant measurement noise. In addition, adapting the method for transient (non-stationary) heat conduction processes will also be considered, as this would expand its practical applicability in various industrial and scientific domains.
The author(s) have not employed any Generative AI tools.