The paper presents a new goodness-of- t test on the basis of the interval estimate of the probability distribution function. The comparative analysis is carried out for the proposed test and for the Kolmogorov test. The results of the numerical modeling show that the proposed test has greater performance than the Kolmogorov test in speci c cases. The analysis of correlation properties of the normalized empirical cumulative distribution function is carried out. As a result, we obtain the equation for calculating the signi cance level of the proposed goodness-of- t test.
Copyright ⃝c by the paper's authors. Copying permitted for private and academic purposes.
Papers [6, 7] present a new GoF test on the basis of the interval estimate of the cumulative distribution function (CDF). The proposed method uses a more accurate measure of the difference of the empirical and theoretical distribution functions compared to a single number.
In the current paper we examined the performance of the interval estimate test with a computational experiment. The results show that the proposed method is more efficient than the Kolmogorov test in case when the true distribution has greater variance then the estimated one. In the opposite case both methods show nearly equal efficiency. Correlation properties of the normalized empirical CDF were examined. It is shown that this function can be considered with a good accuracy as a Markov process. On this basis we construct an algorithm for computing the signi cance level of the proposed GoF test. Analytical solutions are in good agreement with the results of the numerical modeling. 2
The Interval Estimate Goodness-of-Fit Test Let F (x) = P ( x), 1 < x < 1 be the true CDF of the examined random variable , where P ( x) is the probability of x. We represent the measurements of by a sample x1; : : : ; xn of the size n. The empirical CDF of a sample (the point estimate of the function F ) is de ned as F^(x) = (xi x)=n, where (xi x) is the number of elements xi such as xi x. The random variable (xi x) has the binomial probability distribution: P ( = k) = Cnkpkqn k, k = 0; 1; : : : ; n, where Cnk is the number of k-combinations from a set of size n, p = P ( x) = F (x) and q = 1 p. Let M, D be the operators of mathematical expectation and dispersion, respectively. Thus, M = np, D = npq, MF^(x) = F (x) and DF^(x) = F (x)[1 F (x)]=n. If the probability p is constant and 0 < p < 1 then, according to the local Moivre-Laplace theorem, the binomial distribution converges to the normal distribution as n ! 1. Therefore, we may assume that for n ≫ 1 the value (x) = [F^(x)
F (x)] pn √F (x) [1
F (x)] is normally distributed with zero expected value and unit variance.
In the paper [6] the Moivre-Laplace asymptotic is used to construct an interval estimate of the function F (x) as a con dence interval [z1; z2] with a given con dence factor 1 (we also use the symbol to denote the signi cance level of the Kolmogorov test). The boundaries [z1; z2] of the con dence interval are de ned by the condition: P (j j ") = 1 , where " is the solution to the equation (") = 1 =2 and (") is the probability density function (PDF) of the normally distributed random variable with zero mathematical expectation and unit variance. The boundaries [z1; z2] are obtained as the solutions for the equation 2 = "2 for F (x) and represented as follows: z2;1 = 2(n + "2) 2F^n + "2 √ 4n"2(1
F^)F^ + "4
2(n + "2)
:
(
Testing a statistical hypothesis H0 that a function F0(x) is the true CDF of a random variable reduces to checking the condition F0(x) 2 [z1(x); z2(x)]. If the condition is true for each x 2 [xmin; xmax] then the hypothesis H0 is accepted, otherwise it is refused. xmin and xmax here represent the lowest and the highest values in the sample. The suggested procedure was implemented as an algorithm with 3 inputs: a dataset (x1; : : : ; xn), a signi cance level and an estimated CDF F0(x). The functions F^ and F0(x) are evaluated in N points with the step of ∆x over the argument x not grater than (xmax xmin)=5n. 3
The Computational Experiment Modeling the hypothesis H0 testing procedure involves the following steps. First, a sample (x1 : : : xn) of the random variable distributed as F (x) is generated. Then the empirical CDF Fb(x) is calculated and the hypothesis H0 is tested using the Kolmogorov test and the interval estimate test. This procedure was repeated for m independent samples of the variable and two values were obtained: the frequency of the event \H0 is accepted" according to the Kolmogorov test { 1(H0) and according to the interval estimate test { 2(H0). The frequency here means the ratio of the number of occurrences of H0 to the number of samples m.
The computational experiment was conducted for the following series of values of
the con dence factor : 0.0002, 0.0006, 0.001, 0.002, 0.006, 0.02, 0.06, 0.2, 0.4 and
for six hypotheses HL, H0, H1, H2, H3, H4, one of which is valid. The hypotheses
H0; : : : ; H4 correspond to the following values of the argument : 1, 1.5, 2, 2.5, 3 of a
family of distributions [2] with the PDF:
f (x) = c exp
[
( jxj ) ]
;
(
Two instances of true hypothesis were examined: HL { is distributed by the
Laplace law with zero expectation, variance equals to 1.65 and HG { has the normal
distribution (corresponds to = 2 for the distribution (
A GoF test performance can be evaluated by the following variable: ∆ = = ∑i4=0 j (H) (Hi)j =5, where H is the correct hypothesis. As we can see from the table 1-A, the maximum value of ∆ for the interval estimate test is equals to 0:728 with the signi cance level = 0:0006. For the Kolmogorov test the maximum is ∆ = 0:226 with = 0:4. Therefore, in this case interval estimate test provides more reliable results compared to the Kolmogorov test. In the table 1-B we need to exclude the frequency (H2) for calculating ∆ correctly, because the hypothesis H2 is equivalent to
A the true hypothesis HG. Here for the Kolmogorov test the maximum ∆ = 0:205 with = 0:4 and the interval estimate test shows the maximum ∆ = 0:247 with = 0:001. Thus, we may assume that if the true hypothesis is HG, both algorithms are nearly equal in performance. It should also be mentioned that the proposed method has better performance only on small sampling sizes, because with the increase of a sampling size the error of any conventional GoF test asymptotically converges to zero. 4
Covariance of the Empirical Distribution Function A random function of a real argument is called a random process in literature. According to this F^(x), (x) are random processes. To calculate the signi cance level 0 of the GoF test constructed on the basis of the interval estimate, we de ne the covariance of the empirical CDF:
K(x; x + u) = M [F^(x)
F (x)] [F^(x + u)
]
F (x + u) = = MF^(x)F^(x + u)
F (x)F (x + u):
If u
0, then we can assume
r(x; x + u) = MF^(x)F^(x + u) = MF^(x) {F^(x) + [F^(x + u)
F^(x)]} :
(
Let us show that for small arguments x and x + u, u > 0, the random variables under study can be assumed independent. Let the event A = f (xi x) = n1g, B = f (x < xi x + u) = n2g and C = f (xi > x + u) = n3g. The sample size is n = n1 + n2 + n3. We denote the probabilities p1 = P ( x), p2 = P (x < x + u) and p3 = P ( > x + u). The probability P (B) of the event B is described by the binomial distribution:
P (B) = Cnn2 p2n2 (1
p2)n n2 ; n2 = 0; 1; : : : ; n:
Similarly the conditional probability P (B=A) of the event B given that A has occurred:
P (B=A) = Cnn2 n1 p2n2 (1
p2)n n1 n2 ; n2 = 0; 1; : : : ; n
n1;
From the equations (
V =
P (B) P (B=A) = n!(n (n
n1 n2)!(n n2)! n1)! (1 p2)n1
Small values of x and x + u correspond to the conditions n1 << n, n2 << n, n1 + n2 <p< n and the factorial asymptotic can be approximated with a good accuracy by n! 2 nnne n. Thus, we can substitute the factorials in (8) with the approximate expressions and calculate ln V . In the expression obtained we expand the function ln(1 + y) in a series about the parameter y to order y2. Here we consider that the ratios n1=n, n2=n, (n1 + n2)=n and the value p2 are small parameters. After rather simple transformations we obtain ln V n1 (n2 n p2): The value n2=n converges in probability to p2 as n ! 1. Therefore, if n is nite and n1, n2 are small, the value ln V 0 compared to n. Hence, P (B) P (B=A) and the events A and B can be considered independent.
Assuming the variables independence in (
F ] ; u
0:
Considering that MF^(x) = DF^ + F 2(x), thus
Similarly for u < 0 we nd
r(x; x + u) = DF^(x) + F (x)F (x + u); u
0:
r(x; x + u) = DF^(x + u) + F (x + u)F (x); u < 0:
(
K(x; x + u) = { DF^(x); u 0;
DF^(x + u); u < 0: Hence, covariance function of the process (x) is equal to 8 √ > > > > > > (x; x + u) = M (x) (x + u) = <
F (x) [1
F (x + u) [1 >>>> √ F (x + u) [1 >:> F (x) [1
F (x)] F (x + u)] F (x + u)] F (x)] ; u
0; ; u < 0: Function simpli es if we substitute x for y = x + u=2. Therefore, for any u (y u 2 ; y + u ) = uuuv F (y ju2j ) [1 2 t F (y + ju2j ) [1
F (y F (y + ju2j )]: juj )] 2
F (y + juj )) = F (y) + f (y) juj + : : : 2 2 (13) (14) (15) (16) and substitute the function in (15) for (16): (y u 2
u ) ; y + 2 = √ 1 ajuj 1 + bjuj 1 + ajuj 1 bjuj (1 ajuj)(1 + bjuj) e (a b)juj; (17) where a = f (y)=2F (y), b = f (y)=2[1 F (y)].
Function (17) represents the covariance function of a random process only when a b > 0 i.e. for such values of y that F (y) < 0:5. This statement is in good agreement with the assumption of independence of the random variables (xi x) and (x < xi x + u) for relatively small x and u. It should be mentioned that we may obtain the result similar to the formulas (15, 17) for relatively large arguments y such as F (y) > 0:75.
To test the obtained relations function is calculated using the equation (15) for the normal distribution (F = FG) and for the Laplace distribution (F = FL). The results obtained can be approximated with a good accuracy by the exponential equation (17) for any values of the parameter y in the set fy : F (y) < 0:4g and juj < y.
Equations (15, 17) de ne the covariance function of the process (x) for relatively a small values of the parameter y. We simulate the estimates of the correlation function of the process (x) to obtain the complete representation of its correlation properties. Therefore, we examine two hypotheses: (x) is a second-order stationary process and (x) is a non-stationary process. For each sample x1; : : : ; xn we calculate the trajectory To improve the accuracy we nd the average covariance estimation B0(k) for m samples. Minimization with respect to the parameter :
B(k) =
N
i=1 1 N k
∑ i i+k; k = 0; 1; : : : ; N=2: N=2 ∑ [B0(k) k=0 e
k]2 ! min
of the process (x) for x 2 [ 5; 5] according to the equation (
Assuming that the process (x) is non-stationary we de ne its covariance estimate as:
B(i; k) =
l=1 1 ∑m i(l) i(+l)k; i = 1; : : : ; N m k; where 1(l); : : : ; N(l) is the array of values of the process (x) and l is the index of the corresponding sample. If (x) is a stationary process, then for the given k the function B(i; k) is a constant value within the error limits for all i = 1; : : : ; N k. To check this property we evaluate function B(i; k) for given value k∆x. This value equals to 0, 0.01, 0.1, 0.2 and 0.5 times the length of the interval [ 5; 5], which is the domain of (x). The process (x) has a zero mathematical expectation and a unit variance. Therefore, as it is expected, B(i; k) remains a constant for small values of k∆x (i.e. 0 and 0.01) and for the value of 0.5, which is relatively large. For k∆x = 0:1 we observe the maximum deviation of B(i; k) from a constant approximately by 0.1 near the middle value i = (N k)=2. Hence, as a rst approximation, the random process (x) can be considered a second-order stationary process with an exponential correlation function. 5
Signi cance Level of the Goodness-of-Fit Test If a random process (x) with a zero mathematical expectation and a unit variance is Markov, normal and stationary, its correlation function has the following form (u) = e juj. This result is known in literature as the Doob`s theorem. It is not difficult to derive, by analyzing the proof of that theorem, that if a random process (x) is normal, stationary and have a correlation function of the form (u) = e juj, then this process is Markov. Thus, the signi cance level evaluation problem for the proposed GoF test reduces to calculating the probability of hitting the boundaries of the interval [ "; "] (18) (19) (20) of the normal stationary Markov process (x). Let p0 be the probability of accepting the true hypothesis. Thus, p0 = P [ N ∩ (j ij i=1
] ") =
" ∫ "
" ∫ " : : : fN (y1; : : : ; yN )dy1; : : : ; dyN ; (21) where ∩is the set intersection operation, fN is the joint PDF of the normal random variables 1; : : : ; N .
Numerical evaluation of the equation (21) is not possible due to the high integral
multiplicity. If we consider all random variables 1; : : : ; N mutually independent, then
it follows from (21) that p0 = (
Considering the statistical correlation between the elements of the Markov chain 1; : : : ; N allows us to reduce the calculation error. For this purpose we examine the joint PDF of variables i and i+1: Random events j ij " ∩ j i+1j " for i = 1; 4; 7; : : : are statistically independent as (x) is a Markov process. Therefore, approximately p0 = QN=3. Using this formula for calculation reduces the error to the order of 4-5%. Thus, the signi cance level of the GoF test based on the interval estimate is de ned by the equation 0 = 1 QN=3. (22) (23)
The results obtained show that GoF test on the basis of the interval estimate is significantly more efficient then Kolmogorov test if the true distribution function has grater variance than the hypothetical one. Otherwise the advantage of the proposed test is insigni cant. It is shown that normalized empirical CDF is similar to a Markov process on the statistical properties. We yield an equation for approximate calculation of the signi cance level of the proposed GoF test.