Changepoint problems are often encountered when series undergo abrupt changes or discontinuities. Detecting changepoints can signal useful actions towards sustainable developments. However the presence of changepoints have often been known to lead to failure of some regular assumptions. In theory much has not been done on which assumptions fail and to what extent will it affect the score functions of the likelihood asymptotic. In this work we concentrate on simulating the likelihood function using R to establish the failure of regular assumption due to the presence of changepoint. The failure of regular assumption is established using various score functions coded in R thereby making it possible to understand the statistical theory and the consequences of the failure of assumptions as a result changepoints.
likelihood asymptotic, regular assumption,
Changepoints are referred to as discontinuities that can lead to non-linearity even in complex functions (Chen and Gupta, 2000). The causes of changepoints include; changes in locations of observations, equipment, measurement methods, environmental effects, regulations, standards and so on. Generally we need to investigate the potential presence of possible changes in the data set indicating data quality problems that should be resolved prior to any subsequent analysis. This will therefore signal signs for timely protection and knowing this could be highly advantageous in planning for the future. However Yang.et.al.(2006) noted that changes do occur even in the best regulated systems. They indicated that discrepancies in records, occasional disagreement between documentation and data, abnormal data entry, changed units of measurement and other problems require adequate attention. Most times we need to detect the number of changepoints, or jumps, and their locations whereas it is noted in Mainly(2001) that it is much easier if the point of change is known. This case is referred to as intervention analysis. In
It is indicated in Obisesan.et.al(2013) that the data analysed were the physico-chemical properties of water samples obtained from two reservoirs in Oyo State Nigeria. The data were seen to contain some abrupt changes in behaviour. In the work various charts and diagrams were engaged in showing the positions and locations of changepoints and the likelihood function was written to show the single changepoint detection. However the theory on changepoint linking failure of assumption was not shown therefore this present work attempts to extend the likelihood theory to show the implications of failure of regular assumptions as a result of the presence of changepoint.
To study the inference of changepoint problems such as to understand its non-standard nature it is important to review some properties of likelihood functions. The likelihood function for a scalar parameter based on data as a collection of independence observations is defined to be ( | ) ( ) ∏ ( ) which is simply the joint density of the data, regarded as a function of the parameter (Rice, 2007). For convenience, we study the log-likelihood function ( | ) ( ) and write ( ) ( | ) ( ) ( ) (
) ∑ ( ) The maximum likelihood estimate of ̂ which is a value of that minimizes the log-likelihood function. If the likelihood function is a differentiable function of then ̂ will be the root ( ) ( )
at ̂
Assumption 1 : The log-likelihood is a twice differentiable function.
Assumption 2 : The second derivative ( )
at ̂ 3. THE SCORE FUNCTION: SIMULATION Under Assumption 1, the first derivative is usually called the score function: ( ) [∑ ( )] and is regarded as a function of for fixed X This function plays a central role in maximum likelihood theory. We can also define the observed information as ( ) ( ) ∑ ( ) ∑ ( ) which is a sum of defined as components. Also the Fisher information is ( ) { ( ) } ( ) and Which can be written ( ) {
∑ ( ) ∑ ( ( )} ) } { ( )
( ) refers to single observation information.
Now we show some characteristics of the score function when
data are assumed generated from ( ) so that (assumed true
value of is the parameter to be estimated. If we have an
independent and identically distributed sample of size n, the
loglikelihood is written as
( )
∑
( | )
( )
A careful illustration of the behavior of the score function is given
in Figure 1. This allows the sampling variation of score function
for different models (Normal, Poisson, Binomial and Cauchy) for
samples of size . Figure 1(a) shows 25 score function,
each based on independent and identically distributed sample of
size from N(
In all examples in Figure 1, the score varies around zero at the true parameter value. We can show this is generally the case. Recall from Section 3 that the score function is the first derivative of the log-likelihood function where we set ( ) ( ), then at the true value of [ ] The major assumption here is needed to justify interchanging the order of differentiation and integration and can be stated in
Assumption 3 : The range of integration does not depend on since ( ( say
Therefore using the stated assumptions we have ( ( )) as required. We have also find the variance of the score function} as ( ( )) ( ( )) ( ( ( ( )] ))) )) as seen above. We can rewrite ( ) as [
( ( (
)) Since the score function is a sum of n independent random variables, the last equation above shows that ( (
)) contradicts Assumption 4).
The discussion so far has dealt with the behavior of the score function at the true parameter value. We now consider its behavior at other values of . In general (we need to investigate a case that indicate the existence of changepoint) for = we find that there may be need for another assumption: Assumption 4: For the density ( ) on a set of non zero measure. (
) differs from unless for all x (which itself therefore we have # ............................... Poisson Score Functions: t0<- 4 x<- rpois(n,t0) theta<- seq(t0/2,t0*2,len=40) stheta<- -n + sum(x)/theta plot(theta,stheta,type='n',xlab=expression(theta),
ylab='Score',ylim=c(
( | )
( | ) *∑ ( | )
+ ) and ( | ) ( | )+ is finite for all . Therefore Therefore as
the ) tends to a deterministic function ( ) consider ) ) ( ) ( )
In this section the R code used in simulating the likelihood functions for the Normal and Poison distributions are stated as run from the prompt. The Binomial and Cauchy distributions follow similar way. After simulating from the distributions the likelihood functions are plotted to show the distribution of the parameter. It is clear from the code that the expected value of the score function moves around 0.
We now consider whether ̂ is a constant estimator of
( ) ( (
| Which can be rewritten as ̂ ̂ ) ( ) ( ) | ( )
( ) and so we can write ( ) ( ) )
| ( ) | ̂ then we have (nothing that ( ̂) ( ) ( ) ( ) ) ( ) Note that the numerator of Equation 6 approaches 0 as If we assume that the denominator is guaranteed nonzero, then Equation 6 implies that and therefore ̂ . This requires the following assumption which can be seen as a strengthened version of Assumption 2.
is non- zero in an interval containing . 6. Limiting Distribution of ̂ As well as demonstrated the consistency of the maximum likelihood estimator ̂, Equation 5 allows us to establish its distribution when n is large. Recall again that Moreover, ( ) is a sum of independent and identically distributed contributions. Hence from the central limit theorem we have asymptotically, ( ) | ] | ( ) [
| ( ( ) )and | | ( ) since it lies ̂ √
Now we discuss the basic test statistic used for testing hypothesis using the principles of likelihood functions. Suppose that l(.) is the log-likelihood established from the probability density f . then the consistency of ̂ implies that we can write ( ) ( ̂) ( ̂ ) ( ̂) ( ̂ ) ( )
Then representing the likelihood ratio statistic with ( ( ̂)
( )) gives and since (̂) ( ̂ ) ( ̂) ( ̂ ) ( ) ( )( ̂ by definition we can write that
( ) ( ) ) ( )
( ) ( )( ̂
) ( ) ( ) ( ) ( ) ( ) It is clear that the first part of Equation 8 is asymptotically the square of a standard normal random variable and it is therefore a distribution in addition, the last two ratios (( )) and (( )) tend to 1 using similar arguments to those applied in the previous subsection. In the same direction, we can obtain the distribution for a case when is vector (without proof) in that as above we write [ ( ̂) ( )] ( ̂ ) ( )( ̂ ) It is therefore noted that ( ) has an approximate chi-square distribution on p degree(s) of freedom for repeated sampling of data from the model. We can write ( )
In Obisesan et.al(2013), the development of changepoint detection was based on Hinkley(1970) work. Hinkley(1970) considered sequences of random variables and discussed the point at which the probability distribution changes using a normal distribution with changing mean. The asymptotic distribution of the maximum likelihood estimate discussed in this paper is particularly relevant to change-point. The author indicated the simplest model over a whole range of data as ( ) as usual where ( )is a mean function and refer to error terms. Hinkley (1970) computed the asymptotic distribution in the normal case when $\theta_0$ and $\theta_1$ are unknown. The asymptotic dsitribution is found to be the same when the mean levels are known. The two-mean model to be considered supposes that there exist a mean ( ) and mean ( ) and respectively. He also computed the asymptotic distribution of the likelihood estimate of the change-point ̂ (where and are known and is unknown) is obtained from a sample by simply maximizing the likelihood function of the form ( ) ∏ ( ) ∏ ( ) which can be written in form of log likelihood as ( ) ∑ ( ) ∑ ( ) ( ) Moreover, many cases arise when the mean levels are not known. The log-likelihood of the observed sequence ( ) is ( |
) {∑(
) ∑ ( ) } ( ) If we assume that estimators ∑ and ̂ respectively are
̂ is known therefore the maximum likelihood ∑ ̂ ∑ ( ) ∑ ( Particularly for convenience. Hinkley (1970) substituted as known so that Equation 10 becomes ( | ) ) ) ) }
( ) {∑( ∑ (
likelihood estimates of and back into the log-likelihood in Equation 11 and re-arranging the emerging sums of squares conditional on t Equation 11 was used to estimate changepoint of water pollution in Eleyele and Asejire reservoirs in Nigeria. This confirms the application of the likelihood theory of changepoint. More on the applications are discussed in Obisesan(2011, 2015).
In this work it has been shown that changepoint arises as a result of failure of some regular assumptions specifically in this case Assumptions 1 and Assumptions 4 may fail. This work has justified using simulation in the theory of likelihood function for the score functions to show the change in parameter allowing changepoint to occur. The work also justifies the application of changepoint detection as used in Obisesan et.al(2013). The use of R has therefore made it possible to show the failure of regular assumption.
Single changepoint detection has been discussed in the framework of the failure of regular assumptions that have not been commonly noticed. Likelihood function was used to merge the two-mean levels and various score functions were simulated using the successful statistical computing language R. The theoretical implications of failure of regular assumptions were discussed and the failed assumption identified using R . This work has therefore provided a basis for using computational statistics methods in solving a mathematical problem.