parabolic regression usage

Abstract

The paper is devoted to the analysis of different techniques of statistical data approximation for the best model building. Three approximation options (single and segmented cases) are investigated using a specific example. The paper presents the exact formula for the best segmented quadric regression. The connection of the segments was performed using Heaviside function. The problem of optimizing the abscissa of the segments connection point was solved to obtain the highest the accuracy of approximation. The advantages of using two-segmented regression are proved in terms of both approximation accuracy and prognostic properties based on the comparative statistical analysis. accuracy of approximation Approximation, segmented regression, least squares method, choosing the best model, The simple form compatible with a permissible error. Reasonable physical substantiation (derived from some law). The minimum value of maximal deviation. The coincidence with geometrical structure.

The minimum variance [2].

1. Introduction

The choice of the best mathematical becomes the urgent problem of scientific research [ 1 ]. The different assumptions can be applied individually or in some combination to perform the choice:

The minimum possible quantity of model coefficients for given margin of permissible error.

The various tools of the theory of the approximation theory are widely used to build mathematical models. Moreover, until the 80s of the twentieth century, Lagrange, Chebyshev and other high-order polynomials were often used. However, at present, when using the least squares method, polynomials above the third order are practically not used.

Recently, foreign researchers apply segmented regression approaches. This regression can contain linear or parabolic segments for data approximation in different ranges.

Regression analysis has become a modern research tool. The variety of methods have been synthesized to produce misleading results for empirical data samples [ 3 ].

The simplest model is linear (that contains only one independent variable) of following form

= 0 + 1 + , EMAIL: [e-mail] (O.

Solomentsev);

Zaliskyi);

2022 Copyright for this paper by its authors. where and are the dependent and independent variables, 0 and 1 are parameters which must be evaluated. Error of evaluation is represented by symbol (it tells about the absence of exact dependence between the dependent and independent variables) [ 3 ].

In practice, depending on the situation, parabolic, exponential and other types are also used for the construction of regression models. 2. Analysis of literature and problem statement

The analysis of literature [ 4 – 12 ] in the field of empirical data processing shows that for modern era of science and technology evolution, sufficient attention is paid to the problems of approximation. However, there are new tasks of optimizing the switching point abscissas between individual subsets of data when using segmented regressions [13 – 36]. These issues are not fully shown in the literature.

The calculation of the optimal values for location of the connection points will increase the approximation accuracy (for example, reduce the standard deviation) and generally improve the predictive properties [19].

This paper concentrates on the important scientific and practical task of empirical data approximation using segmented regression by the ordinary least squares (OLS) with the subsequent calculation of the optimal value of two segments (parabolas) connection point.

Consider the problem statement from a mathematical point of view. Assume that we observed twodimensional dataset ( , ). Let different functions for approximation ̂ = ( , ⃗ , ) exist, where ⃗ , is a vector of parameters for the approximation function, is a total number of approximation options. Standard deviation σ between real values and estimates ̂ can be calculated for each approximation function. In this case, selection of the best model can be carried out according to the following formula =

( ∈ ∀ : ( ( , ⃗ , )) ≤ ( ( , ⃗ , )). 3. Polysegmented parabolic regression usage

Let us consider an example of the initial data given in [ 1 ]. These data characterize the dependence of the percentage y of defective goods on the percentage x of silicon in steel. The origin data are given in Table 1. where 0, 1 and 2 are coefficients of parabola. 2 ∑ =1 2 ∑ =1 , where presented in Fig. 1.

y(x) x

Let us analyze the deviations and variances for this approximation option. The deviation values are given in Table 2.

The variance is 1.556.

2. The usage of cubic regression.

To build this model, a parabola of the third order as an approximating function was used:

( ) = 0 + 1 + 2 2 + 3 3, where 0, 1, 2 and 3 are coefficients of parabola. where ( ∑ =1 ∑ =1 2 ∑ =1 3 ∑ =1 ∑ =1 2 ∑ =1 3 ∑ =1 4 ∑ =1 2 ∑ =1 3 ∑ (∑ =1 ∑ =1 ∑ =1 2 ∑ =1 3 ∑ =1 ∑ =1 2 ∑ =1 3 ∑ =1 4 ∑ =1 2 ∑ ( ∑ =1 ∑ =1 2 ∑ =1 3 ∑ =1 ( ∑ =1 ∑ =1 ∑ =1 3 ∑ =1 ∑ =1 2 ∑ =1 ∑ =1 4 ∑ =1 2 ∑ =1 3 ∑ ( ∑ =1 ∑ =1 2 ∑ =1 ∑ =1 ∑ =1 2 ∑ =1 3 ∑ =1

After calculations, the equation of the following form for the data of Table 1 can be obtained:

( ) = 25.151− 302.823 + 1253 2 − 1487 3.

The same equation as the approximating one was chosen in [1]. A graphical representation of the initial data and their approximation using a third order parabola are shown in Fig. 2.

y(x) x

Let us analyze the deviations and variances for this approximation option. The deviation values are given in Table 3.

The variance is 0.601. The obtained result was a prerequisite for choosing a cubic function as approximating in [ 1 ], because variance is 2.5 times less than with quadratic approximation.

According to this article authors opinion, the choice of a cubic function is unsuccessful because this function has a maximum at an abscissa = 0.385, and then has a monotone decreasing character. This property of the function does not allow it to be used as a predictor.

Therefore, it is necessary to consider a more rational approximation.

3. The usage of two-segmented parabolic regression.

Alternative option to build mathematical model is usage of two branches of a second order parabola as an approximating function. The equation in this case has the form:

( ) = 0 + 1 + 2 2 + 3( − )2 ( − ), where is switching (connection) point of parabola segments, ( − ) is Heaviside function.

The formulas for unknown coefficients )

2 ∑ =1 )

2 ∑ =

= −1 , = ( 0 1 2 3 ), = (∑ =1

∑ =1

is a sample number of order statistic that corresponds to abscissa of switching point.

In case of segmented regression utilization, uncertainty arises. This uncertainty is associated with determining the rational position of the segments switching points. This uncertainty can be eliminated by optimizing the abscissa of the connection points.

To perform optimization for the investigated option of the origin data, the next technique can be used. In this case, the hypothesis is accepted that the true value of the optimal connection point is within a certain interval from which several (for example, five) discrete values are selected. If the optimum abscissa is out of the selected interval, its range must be expanded. For the selected five values of the switching points, the corresponding approximating functions can be computed using the ordinary least squares method.

For initial data of Table 1 the following mathematical models were obtained:

( ) = 34.554 − 614.629 + 3390 2 − 3068( − 0.08)2 ( − 0.08).

( ) = 29.653 − 432.501 + 1897 2 − 1600( − 0.1)2 ( − 0.1).

( ) = 27.347 − 354.488 + 1338 2 − 1074( − 0.12)2 ( − 0.12).

( ) = 25.769 − 306.487 + 1039 2 − 806.268( − 0.14)2 ( − 0.14).

( ) = 24.673 − 275.853 + 866.475 2 − 673.248( − 0.16)2 ( − 0.16).

A visual view of the initial data and their approximation using obtained options of regression are presented in Fig. 3. empirical data are given in Table 4. connection point [31]. The resulting parabola is:

For each approximation, standard deviations are calculated. The computation results for the given The second order parabola approximated the data from the Table 4 in order to find the best value of ( ) = 1.350 − 18.069 + 91.577 2 .

The optimum of calculated parabola corresponds to the best value of abscissa of the switching point

Standard Deviations for Different Abscissas of Switching Points

Abscissas of Switching Points

Standard Deviations 0.08 0.1 0.12 0.14 0.16

Then the final formula for optimal two-segmented parabolic regression can be obtained:

( ) = 24.673 − 275.853 + 866.475 2 − 673.248( − 0.0991)2 ( − 0.0991).

A visual view of the initial data and their approximation using obtained optimal two-segmented parabolic regression are given in Fig. 4.

y( x)

Let us analyze the deviations and variances for this approximation option. The deviation values are given in Table 5.

The variance is 0.195. The criterion for choosing the best approximation is the minimum total variance and the minimum maximum deviation.

As can be seen from the analysis of both the total variances and the maximum deviations, the proposed two-segment parabolic regression increases the accuracy of approximation according to the selected criteria by several times.

4. Conclusion

The paper discusses the tasks of building mathematical models for statistical data using the mathematical apparatus of segmented regression analysis and OLS method. A comparative analysis of three approximation options was performed: using the quadratic and cubic polynomials and the twosegmented parabolic regression.

The analysis showed that the accuracy of approximation using segmented regression increased in three times compared with cubic regression. At the same time, the predictable properties have also improved. 5. References Experience of Designing and Application of CAD Systems (CADSM), Polyana, Ukraine, February 26 – March 2, 2019, pp. 110–113. doi: 10.1109/CADSM.2019.8779279. [13] M. Zaliskyi, O. Solomentsev, Method of Sequential Estimation of Statistical Distribution Parameters in Control Systems Design, in: 3rd International Conference Methods and Systems of Navigation and Motion Control, Kyiv, Ukraine, 2014, pp. 135–138. doi: 10.1109/MSNMC.2014.6979752. [14] M. Zaliskyi, Yu. Petrova, M. Asanov, E. Bekirov, Statistical Data Processing During Wind Generators Operation, International Journal of Electrical and Electronic Engineering & Telecommunications 8(1) (2019) 33–38. doi: 10.18178/ijeetc.8.1.33-38. [15] I. V. Ostroumov, N. S. Kuzmenko, K. Marais, Optimal Pair of Navigational Aids Selection, 5th International Conference Methods and Systems of Navigation and Motion Control (MSNMC), Kyiv, Ukraine, 2018, pp. 32–35. doi:10.1109/MSNMC.2018.8576293. [16] A. Goncharenko, A Multi-optional Hybrid Functions Entropy as a Tool for transportation Means Repair Optimal Periodicity Determination, Aviation 22(2) (2018) 60–66. doi: 10.3846/aviation.2018.5930. [17] I. V. Ostroumov, N. S. Kuzmenko, Accuracy Assessment of Aircraft Positioning by Multiple Radio Navigational Aids, Telecommunications and Radio Engineering 77(8) (2018) 705–715. doi: 10.1615/TelecomRadEng.v77.i8.40. [18] O. Solomentsev, M. Zaliskyi, O. Shcherbyna, O. Kozhokhina, Sequential Procedure of Changepoint Analysis During Operational Data Processing, in: IEEE Microwave Theory and Techniques in Wireless Communications, Riga, Latvia, 2020, pp. 168–171. doi: 10.1109/MTTW51045.2020.9245068. [19] O. Solomentsev, M. Zaliskyi, T. Herasymenko, O. Kozhokhina, Yu. Petrova, Efficiency of Operational Data Processing for Radio Electronic Equipment, Aviation 23(3) (2019) 71–77. doi: 10.3846/aviation.2019.11849. [20] O. Solomentsev, M. Zaliskyi, T. Herasymenko, O. Kozhokhina, Yu.Petrova, Data Processing in Case of Radio Equipment Reliability Parameters Monitoring, Advances in Wireless and Optical Communications (2018) 219–222. doi: 10.1109/RTUWO.2018.8587882. [21] O. Solomentsev, M. Zaliskyi, T. Herasymenko, Yu. Petrova, Data Processing Method for Deterioration Detection During Radio Equipment Operation, in: Microwave Theory and Techniques in Wireless Communications, Riga, Latvia, 2019, pp. 1–4. doi: 10.1109/MTTW.2019.8897232. [22] I. V. Ostroumov, N. S. Kuzmenko, Compatibility Analysis of Multi Signal Processing in APNT with Current Navigation Infrastructure, Telecommunications and Radio Engineering 77(3) (2018) 211–223. doi:10.1615/TelecomRadEng.v77.i3.30. [23] A. V. Goncharenko, Optimal UAV Maintenance Periodicity Obtained on the Multi-optional Basis, in: 2017 IEEE 4th International Conference on Actual Problems of Unmanned Aerial Vehicles Developments (APUAVD), Kyiv, Ukraine, 2017, pp. 65–68. doi: 10.1109/APUAVD.2017.8308778. [24] V. P. Kharchenko, N. S. Kuzmenko, I. V. Ostroumov, Identification of Unmanned Aerial Vehicle Flight Situation, in: 2017 IEEE 4th International Conference on Actual Problems of Unmanned Aerial Vehicles Developments (APUAVD), Kyiv, Ukraine, 2017, pp. 116–120. doi: 10.1109/APUAVD.2017.8308789. [25] M. H. DeGroot, Optimal Statistical Decisions, New York, John Wiley & Sons, 1970, 489 p. [26] O. Solomentsev, M. Zaliskyi, O. Zuiev, Radioelectronic Equipment Availability Factor Models, in: Signal Processing Symposium 2013, Serock, Poland, 2013, pp. 1–4. doi: 10.1109/SPS.2013.6623616. [27] Z. Hu et al, Statistical Techniques for Detecting Cyberattacks on Computer Networks Based on an Analysis of Abnormal Traffic Behavior, International Journal of Computer Network and Information Security (IJCNIS) 12(6) (2020) 1–13. doi: 10.5815/ijcnis.2020.06.01. [28] V. Kozlovskyy, I. Parkhomey, R. Odarchenko, S. Gnatyuk and T. Zhmurko, Method for UAV trajectory parameters estimation using additional radar data, in: 2016 4th International Conference on Methods and Systems of Navigation and Motion Control (MSNMC), 2016, pp. 39-42, doi: 10.1109/MSNMC.2016.7783101. [29] R. Odarchenko, S. Gnatyuk, T. Zhmurko, O. Tkalich, Improved Method of Routing in UAV Network, in: IEEE 3rd International Conference on Actual Problems of Unmanned Aerial Vehicles

Developments (APUAVD), Kyiv, Ukraine, October 13-15, 2015, рр. 294–297.

[30] O. Solomentsev, M. Zaliskyi, O. Kozhokhina, T. Herasymenko, Efficiency of Data Processing for UAV Operation System, in: 4th International Conference Actual Problems of Unmanned Aerial Vehicles Developments, Kyiv, Ukraine, 2017, pp. 27–31. doi:10.1109/APUAVD.2017.8308769. [31] G.V. Reklaitis, A. Ravindran, K.M. Ragsdell, Engineering Optimization. Methods and

Applications, New York, John Wiley and Sons, 1983, 688 p.

[32] S. Gnatyuk, B. Akhmetov, V. Kozlovskyi, V. Kinzeryavyy, M. Aleksander, D. Prysiazhnyi, New Secure Block Cipher for Critical Applications: Design, Implementation, Speed and Security Analysis, Advances in Intelligent Systems and Computing 1126 (2020) 93–104. doi: 10.1007/9783-030-39162-1_9. [33] O. Ivashchuk et al, A Configuration Analysis of Ukrainian Flight Routes Network, in: IEEE 16th International Conference on the Experience of Designing and Application of CAD Systems, Lviv, Ukraine, 2021, pp. 6–10. doi: 10.1109/CADSM52681.2021.9385263. [34] S. Gnatyuk, Critical Aviation Information Systems Cybersecurity, Meeting Security Challenges Through Data Analytics and Decision Support, NATO Science for Peace and Security Series, D:

Information and Communication Security, 47 (3) (2016) 308–316.

[35] M. Kalimoldayev, S. Tynymbayev, S. Gnatyuk, M. Ibraimov, M. Magzom, The Device for Multiplying Polynomials Modulo an Irreducible Polynomial, News of the National Academy of Sciences of the Republic of Kazakhstan, Series of Geology and Technical Sciences 2(434) (2019) 199–205. [36] M. Zaliskyi, R. Odarchenko, S. Gnatyuk, Y. Petrova, A. Chaplits, Method of Traffic Monitoring for DDoS Attacks Detection in E-Health Systems and Networks, CEUR Workshop Proceedings, 2255 (2018), 193–204.

References

[1] A. K. Mitropolsky , The Technique of Statistical Computing, Nauka , 1971 .

[2] D. M. Himmelblau , Process Analysis by Statistical Methods , New York, John Wiley and Sons, 1970 .

[3] S. Weisberg , Applied Linear Regression, New York, John Wiley and Sons, 2005 , 310 p.

[4] Th. P. Ryan , Modern Regression Methods, Second Edition , New York, John Wiley & Sons, 1997 .

[5] F. Mills , Statistical Methods, Moscow, State Statistical Press, 1958 . [in Russian]

[6] M. Ezekiel , K. A. Fox , Method of Correlation and Regression Analysis . Linear and Curvilinear , New York, John Wiley and Sons, 1959 .

[7] G. A. F. Seber , C. J. Wild , Nonlinear Regression, New York, John Wiley and Sons, 2003 .

[8] S. Huet , A. Bouvier , M. - A. Poursat , E. Jolivet, Statistical Tools for Nonlinear Regression. A Practical Guide With S-PLUS and R Examples, Springer-Verlag, New York, 2004 .

[9] H. Hirose , Y. Soejima , K. Hirose , NNRMLR: A Combined Method of Nearest Neighbor Regression and Multiple Linear Regression , in: International Conference on Advanced Applied Informatics, Fukuoka, Japan, 2012 , pp. 351 - 356 . doi: 10 .1109/IIAI-AAI. 2012 . 76 .

[10] X. Feng , Y. Zhou , T. Hua , Y. Zou , J. Xiao , Contact Temperature Prediction of High Voltage Switchgear Based on Multiple Linear Regression Model , in: Conference of Chinese Association of Automation, Hefei, China, 2017 , pp. 277 - 280 . doi: 10 .1109/YAC. 2017 . 7967419 .

[11] R. Volianskyi , O. Sadovoi , N. Volianska , O. Sinkevych , Construction of Parallel Piecewise-linear Interval Models for Nonlinear Dynamical Objects , Advanced Computer Information Technologies ( 2019 ) 97 - 100 . doi: 10 .1109/ACITT. 2019 . 8779945 .

[12] V. Shutko , L. Tereshchenko , M. Shutko , Iu. Silantieva, O. Kolganova , Application of SplineFourier Transform For Radar Signal Processing , in: IEEE 15th International Conference on the