The paper concentrates on a comparative analysis of the use of various mathematical models, as well as a justification for choosing the best of them. To limit the types of possible models, the unacceptability of utilizing a linear approximating function was proved. Therefore, functions based on parabolas of the second degree and an exponential function were chosen as approximating functions, namely: general parabola, conditional parabola, conditional parabola with heteroscedasticity, conditional parabola built through cluster centers, exponential functions, and two-segmented linear-quadratic regression. To obtain specific equation, ordinary and weighted least squares methods were used. To account for heteroskedasticity, a new approach is considered for determining both a quantitative measure of heteroscedasticity in the analyzed data and calculating heteroscedasticity weighting coefficients for each empirical point. For two-segmented linear-quadratic regression, the abscissa of the switching point was optimized, which made it possible to obtain the best approximating function. A single analytical expression for the two-segmented linearquadratic regression was obtained using the Heaviside function. Two-segmented linear-quadratic regression was substantiated as the best mathematical model.
The mathematical models building is an important branch of scientific activity [1]. It allows to establish a functional dependence between two or more variables [2, 3], explains natural phenomena [4, 5], the functioning of technical devices [6, 7], social and economic processes [8, 9], and others.
To build mathematical models, scientists use theoretical and experimental research [10]. In the first case, the basis is the laws of physics, mathematics, and other fundamental sciences. In the second case, it is necessary to rely on the means of planning and conducting experiments, as well as methods of computer science and mathematical statistics [11, 12].
When mathematical models building, scientists try to find a compromise between the accuracy and simplicity of describing the phenomenon under study [13]. In general, the factors that should be taken into account when building mathematical models are shown in Figure 1.
The mathematical models building for data obtained through experimentation is widely studied today in various technical fields, in particular, in econometrics [14, 15], telecommunications [16, 17], radio engineering [18, 19], control theory [20, 21], cybersecurity [22, 23], maintenance theory [24, 25], artificial intelligence application [26, 27], and other.
The classical means of mathematical models building is the method of ordinary least squares (OLS). However, this method has a number of limitations, in particular, the assumption of a Gaussian distribution of errors and the constancy of the standard deviation [28]. Therefore, alternative methods of regression analysis currently exist [29]. These methods include: – – – – – least absolute deviation regression; lasso regression; ridge regression; weighted least squares (WLS); elastic regression, and others [30].
The literature also presents a large number of heuristic engineering approaches [31, 32]. One such approach is the median center method. To implement this method, the initial data are divided into groups (clusters), in each of which there are medians. Then a smooth curve is drawn through the obtained median centers by the OLS method. This method allows to take into account bias and to reduce the impact of outliers. At the same time, there are other methods of approximation, for example, minimization of the maximum absolute deviation, minimization of Mahalanobis distance, minimization of the range for cumulative curve of deviations, and others [33, 34]. The use of such methods and approaches is the basis for building adequate mathematical models. Choosing the best mathematical model is a very complex scientific and technical task, and it will become more complicated and sophisticated because new approaches and methods of approximation of empirical data are developed [35]. Different scientific publications generally consider various approaches to approximate empirical data but heteroscedasticity was not taken into account during the study.
Recently, the applied use of the theory of approximation during the mathematical models building is developing at an active pace, namely methods of accounting for heteroscedasticity and for the bias that arises as a result of not fulfilling the condition of constant variance of empirical data within a data band. The theoretical base for detecting heteroscedasticity is sufficiently described in [36, 37] but practical methods are still absent.
It should be noted that heteroscedasticity taking into account often improves the accuracy of approximation, so heteroscedasticity analysis should be considered a necessary tool for building reliable mathematical models. However, the detection and substantiation of heteroscedasticity is a complex scientific and practical task. The analysis of the literature and the practice of mathematical models building shows that at the present time insufficient attention is paid to the issues of heteroscedasticity calculation.
Therefore, this paper describes several approaches to solving the recent scientific and technical problem of data approximation in the conditions of heteroscedasticity. The aim of the paper is to present step-by-step methodology of mathematical models building in case of heteroscedasticity for specific numerical example.
When processing data on the strength of various materials, the tasks of mathematical models building and choosing the best ones arise. The analysis shows that these empirical data in most cases have heterogeneous variance, which leads to the need to account for heteroscedasticity. The use of the OLS as the approximation toll of empirical data leads to the significant errors of mathematical models since it does not take into account the real physical and natural conditions of data change.
Initial data on the dependence of the hardness of various types of wood on its average density (kg/m3) [38] are given in Table 1. The sample size of data is = 36.
During the preprocessing we need to check the possibility of approximating the initial data using a linear function. For this, we use the linearity test. Calculation of unknown coefficients of approximation was carried based on OLS. As a result, the equation of the following form was obtained
= −1159 + 57.478 . (
The initial data and their approximation using linear function and OLS are shown in Figure 2. Visual analysis of obtained dependence shows that the linear function drops significantly below zero, which does not correspond to the physical nature of the empirical data under study.
For a justified statistical conclusion, a cumulative curve of residuals was calculated, the graphic representation of which is presented in Figure 3. The range of the cumulative residual curve is 1685 with a standard deviation of 182.75. The ratio of the range of the cumulative residual curve to the standard deviation is 9.22, which with a confidence probability of 0.99 indicates that the data under study cannot be described by the linear function [39].
Let’s approximate the data by a general parabola of second degree using the OLS method. We get the equation of the form
= −115.888 + 9.373 + 0.5094 . (
For the resulting equation (
As we can see from the graph in Figure 4, the parabola passes almost through the origin of the coordinates, so it could be replaced by a conventional one that can be described using equation = + . In this case, we get the equation
= 4.275 + 0.561 . (
Let's consider three variants of taking into account the heteroscedasticity. 1. The method of sliding regressions.
Let us perform a sliding approximation of a group of data with a size of 12 points using a parabola of the second degree by the OLS method. The width of the sliding window was chosen from the following considerations: – – visual analysis of the initial data made it possible to distinguish clusters with a number of points from 6 to 9; it is desirable to use data from two adjacent clusters so that there is no sudden change in the structure of the approximating function.
Standard deviations and average values ̄ along the ordinate were found for each approximation variant. Two statistics of 25 points each were obtained. To construct the heteroscedasticity equation, this data was approximated by a linear function using the OLS method. As a result, the equation was obtained
̄ = −0.09864 + 0.1052 ̄ . (
The resulting equation can be used to calculate a system of weighting coefficients when approximating the initial data by a function of any type.
We can find the current weighting coefficients for each empirical point using the formula
̄
(
, ̄ where ̄ is the average value of standard deviations.
As a result, the equation of the general parabola was obtained, taking into account heteroscedasticity coefficients
= −102.01 + 8.471 + 0.522 . (
As we can see from the obtained equation (
To construct the best general parabola of the second degree, we will use the following scheme for
determining the weighting coefficients for each empirical point
̄
=
,
where is the heteroscedasticity parameter, ̄ is the average strength value for the entire sample,
is the current value calculated according to equation (
To obtain the best weighted parabola, it is necessary to find the optimal value of the heteroscedasticity parameter, which characterizes the structure of the initial empirical data. This will be done by calculating the weighted sum ! of squares deviations for seven variants of the parameter values. Such approach of optimization is suitable for numerical tasks [40]. The results of the calculations are shown in Table 2.
To find the optimal value of parameter "#$ based on these data, we will construct a parabola of the second degree using the OLS. As a result, we get the parabola of the form
! = 161.379 − 0.814 + 0.408 . (
опт = − 2−⋅00.8.41048 = 0.998. (
As a result, the optimal equation of the general parabola was obtained taking into account the optimal indicator of heteroscedasticity
= −1.273 + 0.978 + 0.507 . (
The optimal equation for the conditional parabola will have the following form:
= 3.765 + 0.571 . (
This algorithm can be described by the following sequence:
Determining the conditional parabola over the entire set of points. Equations of the
conditional parabola using the OLS method for the studied set of data have the form (
In this case, the following equation is obtained
= 2.623 + 0.596 .
Let’s perform an analysis of the logarithms of the initial data, which is shown in Figure 7.
(
We will perform a comparative analysis of the obtained mathematical models. The results of the comparison are shown in Figure 8.
A comparative analysis of the obtained equations and standard deviations is shown in Table 3.
The analysis shows that the cluster method is the worst both from the point of view of predictive properties and from the point of view of the value of the standard deviation. The general parabola has the negative free coefficient, which does not correspond to the physical nature of the phenomenon under study. In addition, this model does not take into account heteroscedasticity.
The lack of the general parabola in the form of the negative free coefficient is eliminated by using the conditional parabola. However, this parabola also does not take into account heteroscedasticity.
Of all the mathematical models with the use of parabolas, the best one is the conditional with taking into account heteroskedasticity. However, it has unsatisfactory predictive properties.
Approximation using the exponential function has an acceptable standard deviation but
unsatisfactory predictive properties (sharp growth). The use of the exponential model according to
formula (
The best mathematical model is the optimal two-segmented model consisting of segments with the conditional parabola and linear function. This is explained both by the minimum value of the standard deviation and by the best predictive properties.
The paper considers the issues of mathematical models building and choosing the best of them. Six variants of approximation were studied with a preliminary analysis of the data for linearity. As approximating functions, the following were used: general parabola, conditional parabola, conditional parabola with heteroscedasticity, conditional parabola built through cluster centers, exponential function, and two-segmented linear-quadratic regression.
Alternative variants of taking into account the heteroscedasticity are discussed in the paper. At the same time, a new approximation approach is proposed, which involves the calculation of the heteroscedasticity parameter for correct definition of the approximating function.
For two-segmented linear-quadratic regression, optimization of the abscissa of the switching point was performed to find the best option from the point of view of minimizing the standard deviation. The comparative analysis showed that the two-segmented linear-quadratic regression is the best among the considered mathematical models. This is explained by the minimal standard deviation and the best predictive properties.
This research is partially supported by the Ministry of Education and Science of Ukraine under the projects “Methods of building protected multilayer cellular networks 5G / 6G based on the use of artificial intelligence algorithms for monitoring critical infrastructure objects of country” (# 0124U000197) and is partially supported by EURIZON project # 871072 (Project EU #3035 EURIZON “Research and development of Ukrainian ground network of navigational aids for increasing the safety of civil aviation”).
The author(s) have not employed any Generative AI tools.
volume 992 of Lecture Notes in Networks and Systems, Springer, Cham, 2024, pp. 325–338. doi:
10.1007/978-3-031-60196-5_24.
[16] R. S. Odarchenko, S. O. Gnatyuk, T. O. Zhmurko, O. P. Tkalich, Improved method of routing in
UAV network, in: Proceedings of International Conference Actual Problems of Unmanned
Aerial Vehicles Developments (APUAVD), IEEE, Kyiv, Ukraine, 2015, pp. 294–297. doi:
10.1109/APUAVD.2015.7346624.
[17] O. Lavrynenko, R. Odarchenko, G. Konakhovych, A. Taranenko, D. Bakhtiiarov, T. Dyka,
Method of semantic coding of speech signals based on empirical wavelet transform, in:
Proceedings of IEEE 4th International Conference on Advanced Information and
Communication Technologies (AICT), IEEE, Lviv, Ukraine, 2021, pp. 18–22. doi:
10.1109/AICT52120.2021.9628985.
[18] I. Zhukov, B. Dolintse, S. Balakin, Enhancing data processing methods to improve UAV
positioning accuracy, International Journal of Image, Graphics and Signal Processing 16 (
Wiley, New York, 2007. [33] G. A. F. Seber, C. J. Wild, Nonlinear Regression, Wiley, New York, 2003. [34] S. Huet, A. Bouvier, M.-A. Poursat, E. Jolivet, Statistical Tools for Nonlinear Regression. A
Practical Guide With S-PLUS and R Examples, Springer-Verlag, New York, 2004. [35] Z. Poberezhna, Y. Petrova, K. Slimani, Information technologies in logistics processes of enterprises in the aviation industry, CEUR Workshop Proceedings 3732 (2024) 90–102. URL: https://ceur-ws.org/Vol-3732/paper07.pdf. [36] R. L. Kaufman, Heteroskedasticity in Regression, SAGE Publications, 2013. [37] M. Zaliskyi, et al., Heteroskedasticity analysis during operational data processing of radio electronic systems, in: S. Shukla, A. Unal, J. Varghese Kureethara, D.K. Mishra, D.S. Han (Eds.), Data science and security, volume 290 of Lecture Notes in Networks and Systems, Springer, Singapore, 2021, pp. 168–175. doi: 10.1007/978-981-16-4486-3_18. [38] A. Dlin, Factor Analysis in Production, Statistics, 1975. [39] V. N. Kuzmin, New statistical method for identification of nonlinearity of empirical data, in: Proceedings of Fifth International Conference on Computer Data Analysis and Modeling, Volume 1: A-M, 1998, pp. 159–164. [40] G. V. Reklaitis, A. Ravindran, K. M. Ragsdell, Engineering Optimization. Methods and
Applications, Wiley, New York, 1983.
[41] J. Al-Azzeh, A. Mesleh, M. Zaliskyi, R. Odarchenko, V. Kuzmin, A method of accuracy increment
using segmented regression, Algorithms 15(