In this work we trait the problem of mixtures of normal distributions and methods for estimating the number of components as well as the parameters in a mixture. Also, we present a practical method for the detection of normal finite mixture distribution and respective model validation. Finally, we apply the exposed procedure to a sample of old grape-vine castes.
B R&S\ULJKWEHD¶VXQPGOIYF RIU6XVWDLQEOH$JSGF(YP+,&7NK Q,06DOPSVL$WRXHG3UFJIK& QDG&RPXLFW7HKOJV Finite mixtures of distributions, and in particular mixtures of normal distributions, have been extensively used to model a wide variety of important practical situations, in which data can be considered from two or more populations mixed in varying populations. It is therefore evident interest in this subject attending the vast applications that have been developed by statisticians.
topics that have been successfully addressed in this area, which include among others the problem of identification of outliers, (Atkin & Tunnicliffe, 1980) (Wilson, 1980) or (Beckman &
models (Goodman, 1974), classification analysis (Symons, 1981) (Celeux, 1986) or (Bozdogan, 1992), investigating the robustness of certain statistics such as correlation coefficient sample studied (Srivastava & Lee 1984).
In our work there will be some introductory remarks in the context of finite mixtures
in order to create the enabling environment for a better understanding of this subject.
We will continue with a statistical approach to key issues in the context of mixtures
which will focus on the main methods of estimating parameters of a mixture and one
of the most used algorithms in the identification of a finite mixture of distributions,
i.e., the so-called EM algorithm
Estimating the number of components in a mixture The problem of statistical analysis of finite mixtures can be divided into the following phases: i) ii) iii) iv)
f(x, r) function is identifiable if and only if
k k*
å p j f (xq j ) = å pi* f (xq i* )Þ k = k* Ù (" j=1,...,k ,$i=1,...,k* : p j = pi* Ùq j =q i* )
j=1 j=1
(1)
That is to say that the mixture is identifiable if it admits only a single decomposition.
Methods for the estimation of mixture parameters
The problem of parameter estimation in mixtures, in the case of normal
distributions, is one of the oldest problems in the statistical literature. It was first
introduced by
Although at present the study of mixtures takes place in several areas by applying
other methods, such as the method of maximum likelihood, the original method, the
method of moments is still considered one of the best approximation methods in
separating mixtures of normal distributions. It is useful even in the generation of
initial estimates for the iterative resolution of maximum likelihood equations.
In order to make the analysis of mixtures of distributions a computational problem
more accessible, in the decades of the forties and fifties it was fostered the
development of a high number of graphical techniques. A first step consisted in the
detection of turning points of the curves
These graphical techniques are not only to give a first estimation of parameters, they can be quite useful in an initial examination of data, since they have the advantage of running without a prior knowledge of the number of components of the mixture. They can play a role as an indicator of the number of components, since this information is needed before applying any of the other methods described below.
moment (M r ) and their theoretical moment (mr ) , This method is used on obtaining and solving a system of equations, often of the nonlinear type. The equations are obtained from the empirical equality of every and
M r = 1 n
å (x j - x ) r , r = 1,...,m n j=1
r mr = ò (x - m ) f (x) dx, r = 1,...,m Where x represents the mean of a sample obtained from a population with probability density function f (x) ,
m is the mean value of random variable X , with the same probability function, m the number of moments needed to estimate all parameters.
Let for each value of r obtaining an equation: 1 n r
å (x j - x ) r = ò (x - m ) f (x) dx
n j=1
Note that with the currently available computational means this method becomes
very advantageous. The disadvantage lies in the fact that it is not applicable to
mixtures of distributions with a large number of components, as well as to the
multidimensional case.
3.2 Method of Quandt and Ramsey
This method proposed by
(2) (3) (4) (5) (6) The method minimizes the sum of squared deviations between the empirical moment generating function and the theoretical moment generating function 2 Eˆ(etx ) = 1 n
å etxj n j=1 j=1 S (t ) = å [Eˆ(etxj )- E(etxj )] n where k represents the number of t values in a selected interval (a,b) , with a < 0 and b > 0 . 3.3 Method of maximum likelihood Consider the sample values x1 ,..., xn , x j ÎÂm , j= 1,...,n , a mixture of k density functions and consider the log-likelihood function of this sample, represented by
n é n ù L(x q) = å lnêå pi f (x j ai )ú
j=1 ë j=1 û
where q = ( pi ,ai ), i = 1,...,k is the vector of unknown parameters.
By derivation of the function L(x q) in order to each of these parameters and
equating to zero each of the expressions obtained, we have the so-called likelihood
equations:
The equations obtained are sometimes impossible to solve analytically or very
difficult to resolve, so often resort to the use of iterative methods. However, not
always the most common processes are able to respond to the scale of problems. In
order to overcome these difficulties there is the EM algorithm of
In particular if we have a sample size n and a mixture of two univariate normal components, the log-likelihood function is given by:
n é L(x q ) = å lnê p j=1 êë
1 2p s 1 e -(xj -m1 )2 2s12 + (1 - p)
1 2p s 2 e -(xj -m2 )2 ù 2s 22 ú ú û (9)
distributions method for detection finite mixture of normal The method aims on the one hand prove the existence of mixtures of normal distributions and on the other hand achieving the adjustment of an appropriate model. This approach will first check the detailed curves detected in the histogram of simple frequencies, and then estimate the parameters of these curves and the model checking by applying the Kolmogorov-Smirnov test, adapted to a mixture of normal.
Consider the original data set x1 ,..., xn of the sample under study, and begin a first step by ordering them in ascending order. We then have the ordered sample z1 ,..., zn with z1 £ ... £ zn . Let n1 ,n2 populations derived from the and
n3 represent the sizes of subdecomposition of the initial population, n
2 with n1 + n2 + n3 = n .
Considering the hypothesis of existence of two normal distributions, denote 2 = p = 2n1 and 1 - p = 2n3 where p and 1 - p represent the proportions of n n each one of the curves found on the original curve.
These curves are normal, so they are symmetrical and therefore there is n1 + n 1 , or is 50% the size of the sample size n . Let m1 and s 1 be respectively the mean and standard deviation in the distribution curve 1 and and s 2 be respectively the mean and standard deviation in the distribution curve 2. Thus the probability density function f of weighing the two normal curves is m 2 defined by f (x p,m1 ,m 2 ,s 1 ,s 2 ) = pn(x m1 ,s 1 ) + (1 - p)n(x m 2 ,s 2 ) (10) 4.2 Estimation of parameters for the general model 4.2.1 Estimates of n1*
n1* The value of is the one that minimizes D(n' ) = x - (p'n*m1*,n' + (1 - p'n* )m 2*,n' ) , where x is the sample mean and D (n' )
with For each n' there is an estimate of the fraction correspondent to each subpopulation p'n* and 1 - p '* n and of the respective mean values m1*,n' and m 2*,n' .
Combining these estimates we obtain: and the minimization of: m '* = p'*m * n n 1,n' + (1- p'* )m *
n 2,n' D (n' ) = x - m '* n 4.2.2 Estimates of p , m1 and m
2 n
1 Once estimated
we use the correspondent estimates for the fractions and the mean values of sub-populations obtaining: ïm1*,n' = ï ï ï ím 2*,n' = ï ï î ï '* = 2n' ï pn n z n' + zn' +1
2 z n' + n 2 + z 2 ï ï ï ím 2* = ï ï ï p* = ï î zn1 + zn1+1
2 z n + n 1 2 2n
1 n + z 2 n1+ n2+1 the distribution mean value will be given by m = pm + (1- p)m . 1 2 4.2.3 Estimates of s 1 and s 2 (11) (12) (13) (14) To obtain the estimates for s 1 and s 2 we use: As we have seen the reason for equality n3* = - n1* and is justified by the fact that the tails of two distributions assume negligible values. So, since n2 = n1 + n3 , n 2 n 2 and n2 = it finally comes n3 = n 2 - n1 . The term (z j - m1* ) reflects the differences between each value of sample 1. s 12* and s 22* denote the estimated variances for populations 1 and 2.
We will use the Kolmogorov-Smirnov test to check the model. The statistic of this test is the maximum module of the difference between the empirical distribution ì0 ; x < z1 ï * ï i Fx í ï n ïî1 ; x > zn ; zi £ x £ zi+1 , i = 1,...,n (16) and the adjusted
p* N (x m1* ,s 1* )+ (1 - p* )N (x m 2* ,s 2* ).
One of the observed values is then equal to
ì i Maxí î n - (p* N (zi m1* ,s 1* )+ (1 - p* )N (x m 2* ,s 2* ))üý þ
; i = 1,...,n We will obtain the values of distributions N (z m l* ,s l* ) ; l = 1,2 , in zi points, i = 1,...,n . From the zi points, i = 1,...,n , we calculate ì ïui = ï í ïv = ï i î zi - m1* i = 1,...,n and at ui , i = 1,...,n .
We have
1 w N (w 0,1) = + ò 2 w 0 w e- y22
2p ïìN (zi m1* ,s 1* ) = N (ui 0,1) ; i = 1,...,n í ïîN (zi m 2* ,s 2* ) = N (vi 0,1) ; i = 1,...,n The problem is reduced to obtain the values of standardized normal at ui , dw = items. The above expression is justified since if w < 0 we have and if w > 0
w e- y22 N (w) = ò -¥
2p w e- y22 N (w) = ò
2p 0 dw = dw = After obtaining the values for normal distribution we calculate the values: F* ( z ) = j p* N( uj 0 1,) + (1 - p* )N(vj 0,1) (18) (19) (20) (21) And we obtain the statistic
ì j ü T = maxí - F * (z j ) ; j = 1,...,ný
î n þ This value should be compared with the value in normal distribution table and we concluded about the existence of a mixture of normal distributions if the T value is less than the tabulated. For the previous iterative procedure we decided to simplify the calculations, very time consuming, by designing an algorithm, and we used it in our practical application.
First, we will carry out a comprehensive statistical study of available data relating to populations of vine varieties.
We will discuss three types of caste, namely the Trincadeira Preta in years 1988, 1989 and 1990 in the region of Almeirim, Aragon 1987, 1988 and 1989 in the region of Reguengos and Touriga Nacional 1994, 1995 and 1996 in the region of Foz Coa. Regarding the number of observations and replicates of clones for analysis we noted that with regard to the caste Trincadeira Preta there is in each year a total of (271 observations x 5 repetitions) , in Touriga Nacional (197 observations x 5 repetitions) and in the Aragonês (153 observations x 4 repetitions). Note that each observation is respective to a different clone, repeating this clone 4 or 5 times. For a better layout follows the table 1: For each repetition, we proceeded to calculate the mean, variance, standard deviation, median, sum of sample values, 95%, first quartile, third quartile, range of the sample, inter-quartile range, skewness coefficient, coefficient of flattening and determination of maximum and minimum values of the sample.
Then we obtained the histograms for each of the repetitions, and we tested the normality through the Kolmogorov-Smirnov test. 5.1 Analysis of distributions of repeat genotypes 5.1.1 Testing the normality of distributions As the normal distribution in one of the most important ones, it is useful at this point proceed to test data normality. To this end we then base our conclusions on the results of a nonparametric test, as mentioned above, the application of the
With the help of statistical software and consulting the Table of Critical Values of the Kolmogorov-Smirnov test for one sample, then we obtained the values KS
We considered 1.63 for 1% and 1.36 for 5% significance level, respectively. In the analysis of the results, we note that the null hypothesis that the sample comes from a normal population, is rejected in five cases for the significance level of 5% and only in one case to significance level of 1%, which is not altogether surprising since we are facing a total of 42 cases.
If we consider a binomial distribution with n = 42 and x = 5 then it's expected that 5% of observations fall outside the standard of reference, in our case the normal distribution. 6 Considerations and remarks The vegetative reproduction seems to guarantee the homogeneity of genotypes. So in a given year and local the productions of the same genotype should be distributed normally. Not always this happens because when we studied the 42 repetitions of genotypes: Aragonês, Trincadeira Preta and Touriga Nacional four cases were found where the theoretical model did not fit significantly. In these four cases it was possible to fit the data a mixture of two normal distributions.
We consider important in future work using the techniques of ANOVA to estimate the variance components internal to the genotypes and between genotypes.