Data processing method for multimodal distribution parameters estimation

Abstract

The increase in the amount of data makes it necessary to develop new methods of their processing. In telecommunications and radio engineering, this trend is associated with the complication of signals for the transmission of messages and an increase in the measurement parameters of both the equipment itself and the processes of its operation. During the operation of information transmission systems, the task of evaluating the parameters of the received signals, which are usually afected by interference, is important. This paper considers the problems of synthesis and analysis of a data processing method for estimating the parameters of multimodal distributions. The problem of synthesis is considered on the example of the trimodal probability density function of the sample population, which includes chaotic impulse noise of positive and negative polarity. The problem of analysis is solved on the basis of statistical simulation.

Keywords

• eol>Data processing
• estimation
• method of moments
• method of quantiles
• synthesis and analysis

1. Introduction

The development of Industry 4.0 is accompanied by an increase in data volumes in all its systems [1, 2]. The capabilities of monitoring systems and computing systems make it easy to collect, store and process these data [3, 4]. Intelligent data processing technologies give the opportunity to implement the principles of data-driven decision-making [5], which significantly increases the eficiency of using equipment for its intended purpose.

Information and measurement systems with use of statistical data processing technologies solve problems of testing hypotheses, detection, estimation and measurement of distribution parameters, filtration and extrapolation, pattern recognition, and others [ 6]. To ensure the eficient functioning of measured data processing structures, it is advisable to have a priori information about the parameters that characterize the distribution () of the noise component [7]. If such information is missing, then it is necessary to have estimates of the parameters of the probability density function (PDF) () [8].

The analysis showed that it is quite dificult to obtain an analytical solution to the problem of synthesizing an algorithm for estimating PDF parameters within the framework of any of the known methods of estimation theory if the type of PDF is non-Gaussian [9, 10]. Therefore, this paper considers the problem of synthesizing a suboptimal method for estimating PDF parameters based on a combination of using the method of moments and the method of quantiles. 2. Literature review and problem statement During the operation of radioelectronic and telecommunication systems, control actions are formed to maintain the eficiency of using the equipment for its functional purpose [ 11, 12]. Control actions are formed based on the results of monitoring the condition of equipment, components of the operation system, electromagnetic environment, and others [13]. As a rule, information signals, parameters and data that characterize monitoring results are stochastic [14]. In radioelectronic and telecommunication systems, data can be associated with the trends of changes in defining parameters, reliability indicators, and information signals for transmitting messages [15, 16].

While measuring the defining parameters and reliability indicators, control and measuring equipment is used, which can be located close to or remote from the equipment [17, 18]. In this case, interference is possible, which is observed especially when monitoring the electromagnetic environment [19, 20]. Data transmission channels may also be subject to interference influence [21]. Interference distorts objective data about the state of radioelectronic and telecommunication systems [22, 23]. Data processing algorithms must be developed on the principles of adaptation and readiness to process data with noise [24, 25].

For adaptation, it is necessary to estimate the interference parameters, and for this we need appropriate estimation algorithms [26].

The literature presents a wide variety of methods for estimating distribution parameters. Among these methods are [6, 7, 27]: 1. Maximum likelihood method. 2. Method of moments. 3. Method of maximum posterior probability. 4. Method of quantiles. 5. Heuristic methods and others.

Let us consider the generalized statement of the problem of this paper. The block diagram of data processing includes a number of algorit→h−− − − − − − −

ms ℎ→−−(−/, ). Knowledge of signal and noise patterns is the key to high-quality and eficient data processing for decision-making. In this case, we can consider a generalized operator that generates eficiency estimates and is associated with the functioning of data processing algorithms →−− − − − − − − =→−− − Θ−− − (−−ℎ→−−(−/, )). ( 1 )

Evaluation and optimization of the eficiency is a complex task, so in this paper we will consider the problem of estimating the noise parameters for a given signal-interference situation. 3. Synthesis of data processing method for estimating the parameters of multimodal distributions While solving problems of statistical data processing, a following model is often used for describing samples of measurement information

() = () + (). where () is observable process; () is signal component, which reflects the objective process of changing the properties of the phenomenon under study; () is noise component, which is caused by errors in control and measuring equipment and the presence of interference in the measuring circuits.

Analysis of measurement data shows that the noise component may include chaotic pulsed noise of both positive and negative polarity relative to the nominal level. For this case, the noise component can be characterized by the PDF of the following form

() = (1 − 1 − 2) (1() = + = − = 0, (/+ = − = 0))+ + 1 (1() = +, (/+ = − = 0)) + 2 (1() = − , (/+ = − = 0)), ( 2 ) where (1() = + = − = 0, (/+ = − = 0)) is normal PDF of sample values in case when chaotic pulsed noise is absent; (1() = +, (/+ = − = 0)) is normal PDF of sample values in case when chaotic pulsed noise is occurred with positive amplitude + and average value of probability of pulsed noise presence 1; (1() = − , (/+ = − = 0)) is normal PDF of sample values in case when chaotic pulsed noise is occurred with negative amplitude − and average value of probability of pulsed noise presence 2; (/+ = − = 0) is standard deviation for noise component for those values of the sample for which chaotic pulsed noise is absent; 1() is expected value of noise component.

As can be seen, equation ( 2 ) is a weighted sum of three normal distributions. In this case, the noise component is a non-stationary random process, which in general depends on time. Therefore, to simplify mathematical calculations, we will make the assumption that the mixture of the noise component is stationary. Then the data sample is homogeneous, and its mathematical expectation and standard deviation do not depend on time. It should be noted that for the example under consideration, five parameters need to be estimated, namely (/+ = − = 0), +, − , 1, 2 . In addition, we will assume that between the parameters (/+ = − = 0), +, − the following relationship exists The procedure for synthesizing a method for estimating PDF parameters consists of two stages. The estimation algorithm is based on a fixed and known sample size . At the same time, we believe that when forming a training sample→−, there is no signal component in the measured process. Then the PDF of the mixture will coincide with the PDF of the noise ( 2 ). The estimation algorithm splits the original sample into two parts. In this case, the first part contains samples of the positive region, and the second contains negative values of samples.

|+| ⩾ 3 (), |− | ⩾ 3 ().

In accordance with this assumption, when comparing samples with a zero threshold, two samples are formed + and − . Let’s denote the sample + values as 1, and the sample − values as 2.

The sample size + corresponds to the situation when > 0; the sample size c − corresponds to the situation when ⩽ 0. In general, the equation + + − = is correct.

For the estimation method, two pairs of sampling thresholds are presented, namely 1+, 2+ and 1− , 2− . A possible view of the PDF () and the location of the thresholds 1+, 2+, 1− , 2− is shown schematically in figure 1.

The thresholds 1+, 2+ can be choosen using following conditions: ⎨⎧ () <1+1<+ <2+2 , ()), ⎩ 3 () < 2+ < +. ( 3 )

This means that the threshold 1+ should not exceed the samples of that part of the chaotic impulse noise, the mathematical expectation of which is zero, and the threshold 2+, in addition, should not exceed the samples of that part of the chaotic impulse noise, the mathematical expectation of which is 1() = 1() = +. Similar considerations for choosing thresholds 1− , 2− take place for samples ⩽ 0.

Sample population→s−1 an→d−2 have a probabilistic description that is diferent from that used for the noise term in equation ( 2 ).

In particular case, for training sampl→e−1 when 0 < < ∞ one-dimensional PDF of 1 has the following form ⎧ ⎪ ⎪ ⎪ ⎪ ⎪ ⎨ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎩ ⎪ (1/1+, +, (1)) = (1 − 1+) √2

1 +1+ √2 (1)

(− 1 = ∫︀0∞ (/1, 2, +, − , ()), where 1+ is the average probability of the appearance of chaotic pulse noise of positive polarity with an average amplitude + for the samples 1 that satisfy the condition 1 > 0; 1 is normalization factor, which takes into account the truncated nature of the PDF; (1) is standard deviation of sample 1 that coincides with standard deviation () of initial PDF ( 2 ).

The PDF ( 4 ) is a weighted sum of two distributions: 1) truncated normal and 2) normal, which corresponds to the chaotic pulse noise of positive polarity with the average amplitude +. Taking into account the form of the original PDF ( 2 ), the truncated normal distribution corresponds to the situation of the absence of chaotic pulse noise of positive polarity.

The number of unknown parameters of PDF ( 4 ) is equal to three, not counting the normalization factor 1, the estimate of which can be obtained using the quantile method in accordance with the following formulas: *1 = 1 ∑︁

; = ︂{ 1, > 0,

0, ⩽ 0; + = ∑︁ .

=1 =1

Taking into account the conditions for setting discretization thresholds ( 2 ), we will obtain a system of three equations for unknown parameters 1+, +, (1) using two estimation methods: the method of moments and the method of quantiles. In accordance with the quantile method, we equate the sample estimate ℎ*1(0 < 1 ⩽ 1+) of the probability of not exceeding the threshold 1+ to a theoretically determined value ℎ1(0 < 1 ⩽ 1+) , taking into account PDF ( 4 ).

The second equation of the system is obtained after equating the values of ℎ*2(0 < 1 ⩽ 2+) and ℎ2(0 < 1 ⩽ 2+). For the third equation of the system, we use the method of moments within the framework of the first initial moment of the random variable 1. The system of equations will take the following form: ⎧ ⎪ ⎪ ⎪ ⎪ ℎ*1+ = ∫︀ 1+ (1 − 1+) √2

0 ⎨ℎ*2+ = (1 − 1+) + ∫︀02+ √2 1 1+ (1) exp − (1) exp −

︁( ⎪⎪⎩ *1 (1/0 < 1 < ∞) = (1 − 1+) √︁ 2 ︁( (1) + 1++,

( 5 ) where Note that the first equation in the system ( 5 ) is obtained under the condition ∑︀=+1 1 ; ℎ*1+ = ∑︀=+1 / + ; ℎ*2+ = ∑︀=+1 // ;

+ / = // = ︂{ 1, 0 < ≤ 1+,

0, > 1+; ︂{ 1, 0 < ≤ 2+,

0, > 2+. ⎧ ⎨ ⎪ ⎪

2 ≤ 1, 1 = (1 − 1+) ∫︀01+ √2 1 (1) exp − 2 2(1) ︁( − 22 = ( + ) − ≤ ≤ . ( 6 ) The parameter in equation ( 6 ) is chosen in such a way that the condition ( = 0) = 0.5 is satisfied when = 0. In particular, for = 2, the parameter = 1/4, and for = 3, the parameter = 1/6.

In general, linear approximations of the probability integral in Laplace form can have diferent values of the parameter in equation ( 6 ). Therefore, we assume that in formula ( 5 ) when determining ℎ*1+ and ℎ*2+ we use the following approximation ︂{ Φ (1) = 1 (1 + 1) ,

Φ (1) = 2 (1 + 2) .

The solution of the system of equations ( 5 ) will yield the following expressions for determining the desired parameters of PDF ( 4 ) ; ( 7 ) where ⎪ ⎨ ⎧ * (1) = [︁− 2 ± (︀ 22 − 413)︀ 1/2]︁ (21)− 1 , ⎪

*1+ = 1 − ℎ*1+ * (1) (211+)− 1 ; ⎪⎪⎩ +* = [︁*1 (1) − (1 − *1) * (1)

√︁ 2 ]︁ (*1)− 1 , ︃( 1 = ℎ* 1 1 + 2 √︂ 2

)︃ − 22 ; 2 = 22211+ − ℎ*1+22+ − 2ℎ*2+11+; 3 = 211+2 [2+ − *1 (1)] . The presented equation are also valid for the 2 values, which correspond to the training dataset − .

According to the equation ( 7 ), it follows that there is uncertainty in choosing the sign in front of the square root in the formula for estimation of * (1). The conditions for choosing the sign before the square root were determined based on the results of statistical modeling of algorithms ( 7 ). It should be noted that when implementing the data processing algorithm, we obtain two estimates of the standard deviation of the noise component, characterized by the Gaussian PDF of its values: * (1) after processing the counts from the sample + and * (2) after processing the counts from the sample − . The final estimate of the standard deviation sigma(y) * () for the PDF ( 2 ) is obtained as the arithmetic mean of these estimates, i.e. * () = * (1) + * (2) .

2 ( 8 ) For the parameter estimation algorithm ( 7 ), it should be noted that figure 2 shows a scheme of additional processing of information regarding the values of parameters * (1), 1+, +*, obtained as a result of the data processing algorithm implementation. The data processing algorithm initially calculates estimates for (*+)(1), *(+)1+, (*+)+, when the "+" sign appears before the square root in ( 7 ), and estimates for (*− )(1), *(− )1+, (*− )+, when the " − " sign precedes the square root in ( 7 ).

Thus, when forming the desired estimates of the five parameters of PDF ( 2 ) (), − * , +*, *1, *2, the original training dataset is divided into two subsets: + and − . Then, based on data processing from the + subset, taking into account the information processing scheme (figure 2), we estimate a portion of the desired parameters +*, *1, as well as estimate * (1). Based on data processing from the − subset, according to the information processing scheme (figure 2), we estimate a portion of the desired parameters − * , *2, as well as estimate * (1). Then, considering eqution ( 8 ), we determine the final estimate of * (). The additional information processing scheme (figure 2) is constructed using known rules of algebraic logic. We assume that if the *(+)1+ > 0.1, (*+)(1) > 0, (*+)+ > 1, = (*+)+/ (*+)(1) > 2 are simultaneously fulfilled, this corresponds to the situation of forming the logical "one". Similarly, if the conditions *(− )1+ > 0.1, (*− )(1) > 0, (*− )+ > 1, = (*− )+/ (*− )(1) > 2 are simultaneously fulfilled, this also corresponds to the situation of forming the logical "one". 4. Analysis of data processing method for estimating the parameters of multimodal distributions The analysis of methods for estimating parameters is carried out on the basis of finding the statistical characteristics of the estimates. The most complete one is the probability density function of estimate. However, finding it causes significant dificulties when solving the analysis problem analytically.

In this research, the problem of analyzing the method for estimating five parameters of the PDF was solved on the basis of statistical simulation.

Statistical simulation was performed for 1000 iterations. During the simulation, the following values of the parameters of the estimation algorithm were selected: 1+ = 1; 2+ = 5; 1− = − 1; 2−

= − 5; 1 = 13 ; 1 = 1.5; 2 = 16 ; 2 = 3.

The simulation process was carried out in the following sequence: 1. Generating three independent normal random variables in accordance with PDF ( 2 ). 2. Formation of uniform random variable in the interval [0; 1] and comparing it with thresholds in accordance with the values 1 and 2. 3. Obtaining a dataset of the noise component. 4. Checking the conditions according to figure 2. 5. Dividing the dataset into two datasets. 6. Estimation of unknown PDF parameters according to formulas ( 7 ). 7. Finding the expected values and standard deviations of PDF parameter estimates. The results of statistical simulation are presented in table 1.

Table 1 contains six options of initial parameters of PDF ( 2 ). The numerical values of parameters are shown in table 2.

In general, the simulation results indicate the eficiency of the proposed method for estimating the parameters of the PDF of the noise component. The advantage feature is the low value of the estimates bias and the low level of the standard deviation. To increase the eficiency of estimation more accurate techniques of approximation can be used, for example considered in [28].

5. Conclusions

The paper is devoted to the problem of synthesis and analysis of the method for estimating the parameters of multimodal distribution. Such distributions are often used to describe non-Gausian noise. The paper considers the specific example of interference in the form of chaotic pulsed interference with positive and negative polarity values. Such noise case can be described by a ifve-parameter PDF according to Tukey’s model as a weighted sum of three normal distributions.

The synthesis of the method for estimating the five parameters of the PDF was carried out based on the use of the method of moments and the method of quantiles, which made it possible to obtain the system of equations containing the estimation parameters. The numerical solution of the equations was made possible by approximating the probability integral using the linear function.

The analysis of the method for estimating the five parameters of the PDF was carried out on the basis of statistical simulation. The simulation results have showed satisfactory estimation results.

References

[1] J. Duda , A. Gąsior , Industry 4 .0. A Glocal Perspective , Routledge, New York, NY, 2022 .

[2] J.-C. André , Industry 4 .0: Paradoxes and Conflicts, Wiley, Hoboken, NJ, 2019 .

[3] N. S. Kuzmenko , I. V. Ostroumov , Performance Analysis of Positioning System by Navigational Aids in Three Dimensional Space , in: 2018 IEEE First International Conference on System Analysis and Intelligent Computing (SAIC) , 2018 , pp. 1 - 4 . doi: 10 .1109/SAIC. 2018 . 8516790 .

[4] O. A. Sushchenko , Robust Control of Platforms with Instrumentation , in: 2019 IEEE 2nd Ukraine Conference on Electrical and Computer Engineering (UKRCON) , 2019 , pp. 518 - 521 . doi: 10 .1109/UKRCON. 2019 . 8879969 .

[5] J. Lu , Z. Yan , J. Han, G . Zhang, Data-Driven Decision-Making (D3M): Framework, Methodology, and Directions , IEEE Transactions on Emerging Topics in Computational Intelligence 3 ( 2019 ) 286 - 296 . doi: 10 .1109/TETCI. 2019 . 2915813 .

[6] J. T. McClave , T. Sincich , Statistics, 13 ed., Pearson , London, 2020 .

[7] A. Renyi , Probability Theory, Dover Publications, New York, NY, 2007 .

[8] I. V. Ostroumov , N. S. Kuzmenko , Accuracy estimation of alternative positioning in navigation , in: 2016 4th International Conference on Methods and Systems of Navigation and Motion Control (MSNMC) , 2016 , pp. 291 - 294 . doi: 10 .1109/MSNMC. 2016 . 7783164 .

[9] L. Feng , W. Pingbo , T. Suofu , C. Zhiming , Parameters Estimation for Colored Non-Gaussian Background in Signal Detection , in: 2010 Second International Conference on Computer Modeling and Simulation, volume 2 , 2010 , pp. 300 - 303 . doi: 10 .1109/ICCMS. 2010 . 319 .

[10] O. A. Sushchenko , Y. M. Bezkorovainyi , V. O. Golytsin , Modelling of Microelectromechanical Inertial Sensors, in: 2019 IEEE 15th International Conference on the Experience of Designing and Application of CAD Systems (CADSM) , 2019 , pp. 1 - 5 . doi: 10 .1109/CADSM. 2019 . 8779286 .

[11] A. Anand , M. Ram, System Reliability Management: Solutions and Techniques , CRC Press, Boca Raton, 2021 .

[12] M. Modarres , K. Groth , Reliability and Risk Analysis , CRC Press, Boca Raton, 2023 .

[13] I. Ostroumov , N. Kuzmenko , Risk Assessment of Mid-air Collision Based on Positioning Performance by Navigational Aids , in: 2020 IEEE 6th International Conference on Methods and Systems of Navigation and Motion Control (MSNMC) , 2020 , pp. 34 - 37 . doi: 10 .1109/ MSNMC50359. 2020 . 9255506 .

[14] O. Shmatko , V. Volosyuk , S. Zhyla , V. Pavlikov , N. Ruzhentsev , E. Tserne , A. Popov , I. Ostroumov, N. Kuzmenko , K. Dergachov , O. Sushchenko , Y. Averyanova , M. Zaliskyi , O. Solomentsev , O. Havrylenko , B. Kuznetsov , T. Nikitina, Synthesis of the optimal algorithm and structure of contactless optical device for estimating the parameters of statistically uneven surfaces , Radioelectronic and Computer Systems ( 2021 ) 199 - 213 . doi: 10 .32620/REKS. 2021 . 4 .16.

[15] O. Solomentsev , M. Zaliskyi , Correlated Failures Analysis in Navigation System , in: 2018 IEEE 5th International Conference on Methods and Systems of Navigation and Motion Control (MSNMC) , 2018 , pp. 41 - 44 . doi: 10 .1109/MSNMC. 2018 . 8576306 .

[16] D. J. Smith , Reliability, Maintainability and Risk. Practical Methods for Engineers , 1- ed., Elsevier , London, 2021 .

[17] M. Popela , O. Šimon , S. Vaněk , Universal Measuring Device for Antenna Parameter Testing and Radio Trafic Analysis , in: 2022 32nd International Conference Radioelektronika (RADIOELEKTRONIKA) , 2022 , pp. 01 - 05 . doi: 10 .1109/RADIOELEKTRONIKA54537. 2022 . 9764958 .

[18] O. V. Solomentsev , V. H. Melkumyan , M. Y. Zaliskyi , M. M. Asanov, UAV operation system designing , in: 2015 IEEE International Conference Actual Problems of Unmanned Aerial Vehicles Developments (APUAVD) , 2015 , pp. 95 - 98 . doi: 10 .1109/APUAVD. 2015 . 7346570 .

[19] H. W. Ott , Electromagnetic Compatibility Engineering, Wiley, New York, NY, 2009 .

[20] D. Ma , D.-l. Su, Research on data mining processing methods for electromagnetic environment monitoring results , in: 2010 Asia-Pacific International Symposium on Electromagnetic Compatibility , 2010 , pp. 1626 - 1629 . doi: 10 .1109/APEMC. 2010 . 5475487 .

[21] V. P. Kharchenko , N. S. Kuzmenko , I. V. Ostroumov , Identification of unmanned aerial vehicle flight situation , in: 2017 IEEE 4th International Conference Actual Problems of Unmanned Aerial Vehicles Developments (APUAVD) , 2017 , pp. 116 - 120 . doi: 10 .1109/ APUAVD. 2017 . 8308789 .

[22] H. W. Ott , Noise Reduction Techniques in Electronic Systems , Wiley, New York, NY, 1988 .

[23] R. S. Odarchenko , S. O. Gnatyuk , T. O. Zhmurko , O. P. Tkalich , Improved method of routing in UAV network , in: 2015 IEEE International Conference Actual Problems of Unmanned Aerial Vehicles Developments (APUAVD) , 2015 , pp. 294 - 297 . doi: 10 .1109/ APUAVD. 2015 . 7346624 .

[24] M. P. Deisenroth , A. A. Faisal , C. S. Ong , Mathematics for Machine Learning , Cambridge University Press, 2020 .

[25] I. Prokopenko , Nonparametric Change Point Detection Algorithms in the Monitoring Data , in: Z. Hu , S. Petoukhov , I. Dychka, M. He (Eds.), Advances in Computer Science for Engineering and Education IV , Springer International Publishing, Cham, 2021 , pp. 347 - 360 . doi: 10 .1007/978-3- 030 -80472-5_ 29 .

[26] M. K. Srivastava , A. H. Khan , N. Srivastava , Statistical Inference: Theory of Estimation , PHI, Delhi, 2014 .

[27] H. L. van Trees , Detection , Estimation, and Modulation Theory, Wiley, New York, NY, 2001 .

[28] J. Al-Azzeh , A. Mesleh , M. Zaliskyi , R. Odarchenko , V. Kuzmin , A Method of Accuracy Increment Using Segmented Regression , Algorithms 15 ( 2022 ) 378 . doi: 10 .3390/a15100378.