The relevance of the bilateral filtering algorithm for digital signal processing, namely, X-ray rocking curves, has been substantiated. The software for removing noise on digital X-ray curves with automatic determination of the bilateral filter parameters has been developed. The digital bilateral filter has been constructed as a combination of two Gaussian filters that perform signal processing in the intensity domain and in the angular domain. A mathematical model has been developed for calculating the noise level on the curve and the standard deviations of the bilateral filter in the intensity domain and in the angular domain, respectively. The high-frequency component of the signal has been extracted by a Laplace filter. The histogram of the high-frequency component has been calculated, and the acceptable interval for noise values on the high-frequency component has been established. The noise level based on the high-frequency component has been calculated taking into account the established acceptable interval. The software for automatic bilateral filtering of X-ray curves has been developed in the Python. The parameters of bilateral filtering have been set taking into account the results of processing 20 experimental X-ray curves of the train dataset. The bilateral filtering of 10 experimental X-ray curves of the test dataset has been performed. It has been shown that the bilateral filtering of X-ray curves allows not only to significantly reduce their noise level, but also to preserve the shape of the useful signal.
Experimental X-ray rocking curves are widely used to study the structure of various materials, in particular crystal ones [1]. X-ray curves are one-dimensional signals, the intensity of which depends on the angle of rotation of the studied sample or the X-ray detector. Analysis of the angular distribution of such curves allows us to obtain information about the structural perfection of the studied samples. However, the analysis of X-ray curves is complicated by the presence of significant noise levels in them, which is especially noticeable in areas of low intensity. Noise distorts the signal shape, which leads to a decrease in the accuracy of determining the parameters of the studied samples. Reducing the noise of X-ray curves and increasing their signal-to-noise ratio at the hardware level is difficult since there are limitations on the power of the primary X-ray beam. In addition, it is technically extremely difficult to reduce the level of noise that occurs in X-ray detectors. Even modern X-ray detectors generate a signal with a certain level of noise [2]. Therefore, the current task is to programmatically reduce the noise level on X-ray curves using digital filtering methods [3]. However, in the practical implementation of such signal filtering, distortions occur. For example, common methods of Gaussian [3], median [5] and wavelet [6] filtering lead to blurring of the peaks of the curves. Smoothing of the shape of X-ray curves is observed both during filtering in the spatial domain and during filtering in the frequency domain using the Fourier transforms [7-9]. Compared to Gaussian filtering, wavelet filters lead to less smoothing of peaks, but there is a need to choose the wavelet family and order, method of correcting the wavelet coefficients [10, 11].
According to the criterion of preserving the waveform, one of the most effective is the bilateral filtering algorithm, which processes the signal simultaneously in the spatial (angular) domain and in the intensity domain [12-14]. By taking into account the difference in intensity for different points of the curve, the bilateral filtering algorithm allows to significantly remove noise with minor distortions of the useful signal. However, a certain complexity of the application of bilateral filtering lies in the selection of filter parameters in the spatial domain and in the intensity domain. The selection of such parameters in manual mode is subjective and time-consuming. Therefore, in this work it has been proposed to determine the parameters of the bilateral filter automatically taking into account the noise level on the X-ray curve. To determine the noise level, the high-frequency component of the signal is calculated by the Laplace filter, and the histogram of the obtained highfrequency component is analyzed. Automatic bilateral filtering allows processing of series of curves obtained on a multi-crystal triaxial X-ray diffractometer with high accuracy and speed [2].
The analysis of X-ray rocking curves is described in the work [1], where the dependence of the shape of the curves on the defects of the studied metal alloys is analyzed. It is shown that based on the parameters of the shape of the curves, in particular, the half-width (the width of the peak at halfheight), it is possible to diagnose the mechanical and other properties of the studied materials. In the work [15], the defective structure of CdTe crystals is investigated by analyzing X-ray rocking curves. Such studies are important since the structural perfection of the crystals significantly affects the quality of the detectors made on their basis [16]. In the considered works [14-16], the experimental X-ray curves contained a certain level of noise, which limits the accuracy of their analysis.
Signal processing using bilateral filtering is described in the works [17-19]. The possibilities of bilateral filtering are considered in the work [17] using the example of medical image processing. As a result of such filtering, the signal-to-noise ratio on the studied images has been increased. However, bilateral filtering tools can be used to process not only images, but also one-dimensional X-ray curves. Bilateral filtering using the Fourier transform is described in the work [18]. The possibilities of bilateral filtering in image processing are also considered in the works [19, 20]. Analysis of the considered works [14, 17, 20] shows that bilateral filtering provides a reduction in the noise level with minor signal distortions. At the same time, there is a problem of choosing the parameters of the bilateral filter.
A promising direction of noise filtering on signals is the use of artificial neural networks (ANN). In the work [21], the possibilities of ANN in filtering noise on speech and biomedical signals are considered. Signal filtering is performed by various types of ANN, in particular, autoencoders [22], convolutional neural networks and recurrent neural networks [23]. Despite their effectiveness, ANN require special training before application. In addition, the output signals of ANN with a high probability may contain artifacts.
Thus, the analysis of the considered publications confirms the relevance of bilateral filtering and the need for software development for automatic bilateral filtering of X-ray curves.
Digital X-ray curves Ih(ω) describe the dependence of the intensity of the curve Ih on the rotation angle ω, with the values of Ih and ω given at Q points. Such curves are processed programmatically as arrays Ih = Ih(i) and ω = ω(i), where i = 0, ..., Q-1; Ih(i) is the intensity of the curve at point number i; ω(i) is the angular value of point number i. size Mw elements according to the formula
Bilateral filtering [14] consists in convolution of the initial curve Ih with the filter kernel wB of
= 3 ∙ ,
= (
ℎ = ℎ ∗ .
The kernel wB of the bilateral filter is described by the formula ( ) = exp −( − )
2 2 2 ∙ exp −( ℎ( ) − )
2 2 2 where m = 0,..., Mw-1; m is the kernel element number; Mw is the size of the filter kernel; σBS is the standard deviation (SD) of the bilateral filter kernel in the spatial (angular) domain; σBI is the standard deviation of the bilateral filter kernel in the intensity domain; mc is the element number for the center of the filter kernel; Ih(m) is the intensity of the curve point corresponding to the kernel element with number m; IC is the intensity of the curve point corresponding to the center of the kernel.
Thus, the bilateral filter is a combination of two Gaussian filters: one filter performs processing in the spatial domain (with SD σBS), and the other performs processing in the intensity domain (with SD σBI). The bilateral noise filtering method allows to preserve the clarity of the curve peaks, since this method uses spatially weighted averaging of the curve intensity. That is, in the peak region, the smoothing will be less and the peak shape will be preserved, and in the region with a smooth change in brightness, the noise smoothing will be stronger.
The size Mw of the filter kernel wB is calculated taking into account the 3σ rule for a onedimensional Gaussian distribution. Similarly, the SD σBI of the kernel in the intensity domain is calculated according to the 3σ rule:
−1 =0 ℎ ( ) =
ℎ( − + ) ∙ ( ), where IhB is the curve after filtering (of the same size as Ih); Mw is the size of the filter kernel; mc is the element number for the center of the filter kernel.
The operation of convolution of the signal Ih with the kernel wB is simply written as where σN is the noise level on the X-ray curve.
Since on the experimental X-ray curves are dominated the Gaussian noise [15, 24], as the noise level σN its standard deviation is used.
The high-frequency component IhNC is extracted by convolution of the initial curve Ih with the kernel of the high-frequency Laplace filter wL [24], which is described by the formula:
For the obtained high-frequency component IhNC, its histogram hN is constructed, defined by Qh for intervals (bins) (Figure 1).
In order to simplify the analysis of the histogram and to reduce its random deviations, the
convolution of the initial histogram hN with the kernel of the Gaussian filter (with SD σHG; for
example, σHG =1) is performed, as a result of which the smoothed histogram hNF is calculated. In the
resulting histogram, the central peak mainly corresponds to noise (minor deviations from the
average value), and the other (peripheral) parts of the histogram correspond to the high-frequency
component of the useful signal (significant deviations from the average value).
(
The central peak of the histogram is extracted using the threshold Th_hNF; the peak includes the histogram elements whose values exceed Th_hNF. The threshold Th_hNF value is calculated using the empirical formula:
ℎ_ℎ = ℎ _ ∙ ℎ, (
The left boundary Ih_nbL of the histogram peak is defined as the smallest IhNC value that exceeds the threshold Th_hNF. The right boundary Ih_nbR of the hNF histogram peak is defined as the largest IhNC value that exceeds the threshold Th_hNF. For the histogram peak (in the range from Ih_nbL to Ih_nbR), its SD σH0 is calculated.
Taking into account the convolution of the initial histogram hN with the Gaussian filter kernel (with SD σHG), the exact value of SD σH of the histogram peak is calculated by the formula:
After that, the Gaussian distribution hNG (with SD σH) is calculated, which describes the central
peak of the histogram (Figure 1). The noise level σN is calculated based on such values of the
highfrequency component IhNC that correspond mainly to noise (and not to sharp changes in the useful
signal) and are in the interval (ThIh_min, ThIh_max). The limits of the interval are calculated by
empirical formulas:
(
The value of the coefficient kσ = 2 is chosen from the condition that in the interval (ThIh_min, ThIh_max) the values of the histogram hNF are close to the normal distribution hNG (since Gaussian noise on the X-ray curve is analyzed). With a narrower interval (ThIh_min, ThIh_max), part of the noise values will not be taken into account, and with a wider interval, the values of the useful signal in IhNC will be taken into account.
The obtained value of the noise level σN is used to calculate the SD σBI of the bilateral filter kernel
in the intensity domain according to formula (
The value SD σBS of the bilateral filter kernel in the spatial (angular) domain is determined based on the given value of the coefficient kN of the noise level reduction as a result of filtering. The coefficient kN = σNC/ σN is calculated as the ratio of the noise SD σNC after filtering to the SD σN of = 20 − 2 . ℎ ℎ ℎ ℎ = − ∙ ,
= ∙ , the noise before filtering. If the noise level on the signal Ih is equal to σN, then after smoothing the signal with the kernel wG of the Gaussian filter with the SD σBS, the noise level σNC after filtering is equal to [24]: where Mw = (2[3∙σBS]+1) is the number of elements of the filter kernel.
From formula (10), the value of the coefficient kN is equal to:
= =
−1 =0
2( ),
−1 =0
2( ) .
According to formula (11), for SD σBS = 3 (the number of elements is used as the unit of measurement) there is the value kN = 0.307094; for σBS = 5 there is the value kN = 0.237978. The value of σBS is chosen so that the coefficient kN does not exceed the threshold value (according to the filtering conditions of a certain type of signal).
The software for bilateral filtering of X-ray curves is implemented in Python [25] based on the developed mathematical model (section 2). Bilateral filtering of X-ray curves Ih(ω) involves the following stages:
Reading the initial curve Ih(ω) from a text file.
Calculating the high-frequency component of the signal IhNC by convolving the initial curve
Ih with the Laplace filter kernel wL (
Calculating the histogram hN for the high-frequency component IhNC using the «histogram» Calculating the smoothed histogram hNF by convolving the initial histogram hN with the Gaussian filter kernel (with SD σHG) using the «gaussian_filter1d» function of the scipy (10) (11)
Using the developed program, bilateral filtering of 20 X-ray curves of the training dataset is performed. During the processing of such curves, processing parameters are determined that provide the required reduction in the noise level (according to the value of the coefficient kN from the formula (11)) while maintaining the clarity of the peaks. As a result, the following values of the program parameters are set:
library.
(
Calculation of the SD σBI of the kernel in the intensity domain through the SD of the noise
σN according to the formula (
Determination of the SD σBS of the kernel in the spatial (angular) domain, provided that the permissible value of the noise level reduction coefficient kN (11).
Calculation for each point of the X-ray curve (with number i) of the bilateral filter kernel wB
(
SD σHG = 1 (SD of the Gaussian filter kernel, which is used to smooth the histogram hN). kTh = 0.25 (threshold coefficient for histogram peak extraction, which is determined by the minimum of the mean square difference between the histogram hNF and the Gaussian distribution hNG in the interval from Ih_nbL to Ih_nbR).
SD σBS = 5 (SD σBS of the bilateral filter kernel in the spatial (angular) domain).
Let us consider an example of bilateral filtering of experimental X-ray curve Ih(ω) # 1 (Figure 2), which belongs to the training dataset of curves. The intensity of the curve is given in Q points. When visualizing such curves, it is especially effective to use a logarithmic scale for the intensity of the curve, since in this case not only peaks are well visualized, but also signal values for low intensities (the so-called 'tails' of the curves) (Figure 2a), which contain important information about the objects under study.
Next, based on the initial X-ray curve Ih(ω), the value of the high-frequency component IhNC(ω) is calculated (Figure 3). In order to improve the visualization of noise (to which small values of IhNC correspond), the graph shows only the values of IhNC that are within the permissible range (for example, from -100 to 100 in conventional units).
Based on the high-frequency component of the signal IhNC, its histogram hN is calculated, and by convolving the initial histogram hN with the kernel of the Gaussian filter (with SD σHG), a smoothed histogram hNF is calculated (Figure 4).
Taking into account the threshold Th_hNF, the left Ih_nbL and right Ih_nbR boundaries for the central peak of the histogram are calculated, and the SD σH of the histogram peak is also calculated. The resulting normal distribution hNG (with SD σH) quite accurately describes the peak of the histogram, which corresponds to Gaussian noise.
Taking into account the SD σH of the histogram peak, the values of the permissible interval (ThIh_min, ThIh_max) for the values of the high-frequency component IhNC are calculated (Figure 5). The noise level σN is calculated based on the values of the high-frequency component IhNC, which are in the permissible interval.
Taking into account the calculated noise SD σN, the SD σBI of the bilateral filter kernel in the intensity domain was calculated and value of the SD σBS = 5 was determined for the kernel in the spatial (angular) domain. The SD σBS of the bilateral filter kernel is chosen so that the noise level in areas with a smooth change in intensity (without peaks) is reduced by kN=0.24 times. After this, bilateral filtering of the initial curve Ih(ω) is performed. On the obtained curve IhB(ω) after bilateral filtering, a significant reduction in the noise level is observed while maintaining the clarity of all peaks (Figure 6). The reduction in the noise level on the IhB(ω) curve compared to the initial curve is particularly noticeable (Figure 7).
The filtered curve IhB(ω) contains mainly the useful signal, which simplifies and increases the accuracy of its subsequent analysis. For example, the angular intensity distribution is visually better perceived on the filtered curve IhB(ω) compared to the initial Ih(ω). Software processing of the curves IhB(ω) allows to accurately determine the coordinates of the peaks, their half-widths (width at half height), the slopes of the angular intensity distributions for local areas and other parameters that carry important diagnostic information about the structural perfection of the studied materials. a) b)
An example of bilateral filtering of the experimental X-ray curve Ih(ω) # 2 (Figure 8), which belongs to the test dataset of curves, shall be considered. Based on the initial X-ray curve Ih(ω), the value of the high-frequency component IhNC(ω) and its histogram hN are calculated. By convolving the initial histogram hN with the Gaussian filter kernel (with SD σHG), the smoothed histogram hNF is calculated (Figure 9). The left Ih_nbL and right Ih_nbR boundaries for the central peak of the histogram are calculated, and the SD σH of the histogram peak is also calculated. The obtained normal distribution hNG (with SD σH) describes the peak of the histogram quite accurately, which confirms the correctness of the selected values of SD σHG and the threshold coefficient kTh (established based on the analysis of the X-ray curves of the training sample).
Using the SD σH of the histogram peak, the values of the permissible interval (ThIh_min, ThIh_max) for the values of the high-frequency component IhNC are calculated (Figure 10). As can be seen in Figure 10, the IhNC values in the left and right parts are limited by the interval (ThIh_min, ThIh_max). This means that the interval (ThIh_min, ThIh_max) correctly limits the noise value on the IhNC graph, since it is in the left and right parts of the curve that the noise level is significant compared to the level of the useful signal. The noise level σN is calculated based on the IhNC values that are in the permissible interval.
Based on the obtained noise SD σN, the SD σBI of the bilateral filter kernel in the intensity domain was calculated, and value of the SD σBS =5 was determined for the kernel in the spatial (angular) domain. Bilateral filtering of the initial curve Ih(ω) is performed. On the obtained curve IhB(ω) after bilateral filtering, a significant smoothing of noise is observed while maintaining the clarity of the peaks, which is especially noticeable in comparison with the initial curve (Figure 11). The curve IhB(ω) after bilateral filtering contains mainly a useful signal, which allows for a more accurate analysis of its shape and the detection of patterns in the angular intensity distribution. In particular, on the filtered curve IhB(ω) a local maximum for the angle ω = 9.42° is clearly observed, which is barely noticeable on the initial curve. Accurate determination of angular coordinates for the peaks of X-ray curves allows, in particular, to determine with high accuracy lattice deformations and changes in interplanar distances for the crystals under study.
Bilateral filtering of 10 experimental X-ray curves of the test dataset is performed, and in all cases similar results are obtained: a significant reduction in their noise level while preserving the shape of the useful signal.
Software has been developed for noise removal on digital X-ray curves Ih(ω) using a bilateral filter whose parameters are determined automatically. The digital bilateral filter has been constructed as a combination of two Gaussian filters that perform signal processing in the intensity domain and in the spatial (angular) domain. A mathematical model has been developed for calculating the noise level σN on the initial X-ray curve and the parameters of the bilateral filter. The following filter parameters were used: the standard deviation σBI of the filter kernel in the intensity domain and standard deviation σBS in the angular domain. The high-frequency component IhNC of the X-ray curve has been extracted by a Laplace filter. The histogram hNF of the high-frequency component has been calculated, and permissible interval (ThIh_min, ThIh_max) for noise values on the high-frequency component has been set. The noise level σN has been calculated based on the high-frequency component IhNC, taking into account the established permissible interval.
The software for automatic bilateral filtering of X-ray curves has been developed in the Python language. The parameters of bilateral filtering have been set taking into account the results of processing 20 experimental X-ray curves of the training dataset. The value σHG =1 has been set for the SD of the Gaussian filter kernel, which is used to smooth the initial histogram. The value kTh = 0.25 has been set for the threshold coefficient of the histogram peak selection. The SD σBS of the bilateral filter kernel has been chosen so that the noise level in areas with a smooth change in intensity (without peaks) is reduced by kN=0.24 times.
Bilateral filtering of 10 experimental X-ray curves of the test dataset has been performed. It has been shown that bilateral filtering of X-ray curves allows not only to significantly reduce their noise level, but also to preserve the shape of the useful signal.
The author(s) have not employed any Generative AI tools.
[10] J. Isabona, A.L. Imoize, S. Ojo, Image Denoising based on Enhanced Wavelet Global
Thresholding Using Intelligent Signal Processing Algorithm, International Journal of Image,
Graphics and Signal Processing (IJIGSP) 15 (
Engineering 17 (2023) 77-85. doi: 10.54254/2755-2721/17/20230917.
[13] F. Spagnolo, P. Corsonello, F. Frustaci, S. Perrim, Approximate bilateral filters for real-time and
low-energy imaging applications on FPGAs, The Journal of Supercomputing 80 (2024) 15894–
15916. doi: 10.1007/s11227-024-06084-y.
[14] S. Balovsyak, M. Borcha, M. Gregus ml., Kh. Odaiska, N. Serpak, Automatic Processing of
Digital X-ray Medical Images by Bilateral Filtration Method, in: Proceedings of the 2nd
International Workshop on Intelligent Information Technologies and Systems of Information
Security (IntelITSIS 2021), March 24-26, 2021, ceur-ws.org, Khmelnytskyi, Ukraine, 2021, pp.
280-294.
[15] I.M. Fodchuk, A.R. Kuzmin, I.I. Hutsuliak, M.D. Borcha, V.O. Kotsyubynsky, Defect structure of
high-resistance CdTe:Cl single crystals and MoOx/CdTe:Cl/MoOx heterostructures according
to the data of high-resolution X-ray diffractometry, Semiconductor Physics, Quantum
Electronics and Optoelectronics 26(
Technique, Journal of Medical Systems 43/ 240 (2019) 1-12. doi: 10.1007/s10916-019-1370-x.
[18] Y. Sumiya, N. Fukushima, K. Sugimoto, S.-i. Kamata, Extending Compressive Bilateral Filtering
For Arbitrary Range Kernel, in: IEEE International Conference on Image Processing (ICIP), Abu
Dhabi, United Arab Emirates, 2020, pp. 1018-1022, doi: 10.1109/ICIP40778.2020.9191123.
[19] X.-Y. Jia, C.-L. DongYe, Seismic section image detail enhancement method based on bilateral
texture filtering and adaptive enhancement of texture details, Nonlin. Processes Geophys. 27
(2020) 253–260. doi: 10.5194/npg-27-253-2020.
[20] G. Liu, The Novel Bilateral Quadratic Interpolation Image Super-resolution Algorithm,
International Journal of Image, Graphics and Signal Processing (IJIGSP) 13 (