Introduction

Nowadays, object tracking is a widely investigated topic in engineering and computer vision [1,2]. Video surveillance, vehicle navigation, human-computer interaction, and robotics are examples of tracking applications [3,4,5,6,7,8,9,10,11]. In tracking, objects are localized in a current frame automatically by applying a detection engine [12,13,14,15]. A main difficulty in object tracking is that the observed scene is commonly degraded by additive noise, the presence of a cluttered background, geometric modifications such as pose changing and scaling, gesticulations, and nonuniform illumination. Additionally, eventual occlusions and real-time requirements are challenges that a modern tracking algorithm must solve.

Object tracking based on correlation-based methods are widely utilized as an attractive alternative to existing tracking algorithms [16,17,18]. Correlation filters have a good formal basis, and they can be easily implemented for real-time applications [19,20]. Recognition methods involving template matching are not useful in some cases, for instance, when articulation changes global features like the object outline. So, conventional correlation filters without training may yield a poor performance to recognize objects possessing incomplete information [21,22,23]. Adaptive approach to the filter design helps us to synthesize adaptive filters for object tracking [24,25].

In this work, we propose an algorithm for object tracking based on locally adaptive correlation filtering. The algorithm is able to carry out object tracking with a high accuracy in an video without offline training. The objects are selected at the beginning of the algorithm. Afterwards, a composite correlation filter optimized for distortion tolerant pattern recognition is designed to recognize the target in the next frame. The impulse responses of optimum correlation filters are used to synthesize composite filters for distortion invariant object tracking. Two techniques are used to improve the detection performance: adaptive procedure that achieves a prespecified performance for a typical scene background, and multiple composite filters (bank of composite filters) when numerous views are available for training. The filter is dynamically adapted to each frame using information of current and past scene observations. The paper is organized as follows. Section 2 recalls the optimum composite filter design. Section 3 describes the suggested algorithm for object tracking by locally adaptive correlation filtering. Computer simulation results obtained with the proposed algorithm are presented and compared with common algorithms in terms of detection efficiency and location accuracy in section 4. Finally, section 5 presents our conclusions.

Composite filter design using optimum correlation filters

We are interested in the design of a correlation filter which is able to recognize an object embedded into a disjoint background in the scene corrupted with additive noise. The designed filter should be also able to recognize geometrically distorted versions of the target. Let T = {t i (x, y); i = 1, . . . , N} be an image set containing geometrically distorted versions of the target to be recognized. The input scene is assumed to be composed by the target t(x, y) embedded into a disjoint background b(x, y) at unknown coordinates (τ x , τ y ), and the whole scene is corrupted with additive noise n(x, y), as follows:

f (x, y) = t(x − τ x , y − τ y ) + b(x, y)w(x − τ x , y − τ y ) + n(x, y), (1)

where w(x, y) is a binary function defined as zero inside the target area, and unity elsewhere. The optimum filter for detecting the target, in terms of the signal to noise ratio (SNR) and the minimum variance of measurements of location errors (LE), is the generalized matched filter (GMF) [26], whose frequency response is given by In (2), T (u, v) and W(u, v) are the Fourier transforms of t(x, y) and w(x, y), respectively; µ b is the mean value of the background b(x, y); P b (u, v) and P n (u, v) denote power spectral densities of b 0 (x, y) = b(x, y) − µ b and n(x, y), respectively. The symbol ⊗ denotes convolution.

H * (u, v) = T (u, v) + µ b W(u, v) P b (u, v) ⊗ |W(u, v)| 2 + P n (u, v). (2)

Let h i (x, y) be the impulse response of a GMF constructed for the ith available view of the target t i (x, y) in T . Let H = {h i (x, y); i = 1, . . . , N} be the set of all GMF impulse responses constructed for all training images t i (x, y). Additionally, let S = {s i (x, y); i = 1, . . . , M} be an image set containing M unwanted patterns to be rejected. We want to synthesize a filter capable to recognize all target views in T and to reject the false patterns in S , by combining the optimum filter templates contained in H, and by using only a single correlation operation. The required filter p(x, y), can be constructed as follows [26]:

p(x, y) = N ∑ i=1 α i h i (x, y) + N+M ∑ i=N+1 α i s i (x, y),(3)

where the coefficients {α i ; i = 1, . . . , N + M} are chosen to satisfy prespecified output values for each pattern in U = T ∪ S . Using vectormatrix notation, we denote by R a matrix with N + M columns, where each column is the vector version of each element of U. Let a = [α i ; i = 1, . . . , N + M] T be a vector of coefficients. Thus, (3) can be rewritten as

p = Ra.(4)

Let us denote by

u =           1, . . . , 1 Nones , 0, . . . , 0 Mzeros           T ,

the desired responses to the training patterns, and denote by Q the matrix whose columns are the elements of U. The response constraints can be expressed as

u = Q + p,(5)

where superscript + denotes complex conjugate. Substituting (4) into ( 5), we obtain

u = Q + Ra.

Thus, the solution for a, is

a = [Q + R] −1 u.(6)

Finally, substituting ( 8) into (4), the solution for the composite filter is given by

p = R[Q + R] −1 u.(7)

Note that the value of the correlation peak when using the filter given in Eq. 7, is expected to be close to unity for true-class objects, and close to zero for false-class objects.

The discrimination capability (DC) is a measure of the ability of the filter to distinguish a target from unwanted objects; it is defined by [26]

DC = 1 − |c b | 2 |c t | 2 ,

where c b is the value of the maximum correlation sidelobe in background area and c t is the value of the correlation peak generated by the target. A DC value close to unity indicates that the filter has a good capability to distinguish between the target and any false object. Negatives values of the DC indicate that the filter is unable to detect the target. Also, if the obtained DC is greater than a prespecified threshold (DC > DC th ), then the target is considered as detected and, otherwise, the target is rejected.

Object tracking with locally adaptive correlation filtering

In this section we describe the proposed algorithm for object tracking based on composite correlation filtering. The proposed algorithm is robust to pose changes and appearance modifications of objects, as well as to the presence of scene noise, illumination changes, and target occlusions.

The algorithm starts with an initialization step where the objects are selected. Next, an optimum correlation filter for reliable detection and location estimation of the target is designed. Afterwards, a composite locally adaptive correlation filter is synthesized. The proposed algorithm incorporates an automatic re-initialization mechanism that reestablishes the tracking if it fails. The block diagram of the proposed algorithm is depicted in Fig. 1. The detailed operation steps are explained below.

Step 1: For each object select a small target t i (x, y) from a captured scene frame f i (x, y) containing the object to be tracked.

Step 2: Synthesize an optimum correlation filter h i (x, y) with (2) for reliable detection and location estimation of the target t i (x, y) in the observed local frame l i (x, y).

Step 3: Synthesize a composite locally adaptive correlation filter p i (x, y) as follows. First, detect and locate the target by h i (x, y) filter from the observed local frame l i (x, y). If the obtained DC is greater than a prespecified threshold (DC > DC rec ), then the target is considered as successfully detected, t i (x, y) added into the set T and recursion should be stopped. Otherwise, the target s i (x, y) corresponding to a false peak added into the set S . Second, synthesize a composite filter p i (x, y) with the help of (7). Third, detect and locate the target by p i (x, y) filter from the observed local frame l i (x, y) recursively until the condition DC > DC rec is satisfied.

Step 4: Detect and locate the target in the observed local frame l i+1 (x, y) from a new scene frame f i+1 (x, y) by p i (x, y) filter. The coordinates of the observed local frame l i+1 (x, y) are provided by a prediction process that analyzes the motion kinematics of the target. If the obtained DC is greater than a prespecified threshold (DC > DC th ), then the target is considered as successfully detected and p i (x, y) filter added to the bank B of composite correlation filters. Otherwise, the target is lost in the observed local frame l i+1 (x, y) and we recursively used the filters from bank B until condition DC > DC con is satisfied.

The filter from bank B with condition DC > DC con is used to a new scene frame. If the target is lost in the observed local frame l i+1 (x, y) with help the filters from bank B, then the coordinates of the target is set coordinates of the past scene frame f i (x, y) and we proceed to a new scene frame f i+2 (x, y).

Begin

Computer simulation

In this section, computer simulation results obtained with the proposed algorithm for object tracking are presented and compared with common algorithms in terms of detection efficiency, tracking accuracy, and speed of processing.

In order to evaluate the performance of our tracker, we conduct experiments on 100 challenging image sequences from Object Tracking Benchmark (TB-100 database) [27]. These sequences cover most challenging situations in object tracking: Illumination Variation (IV), Scale Variation (SV), Occlusion (OCC), Deformation (DEF), Motion Blur (MB), Fast Motion (FM), In-Plane Rotation (IPR), Out-of-Plane Rotation (OPR), Out-of-View (OV), Background Clutters (BC), Low Resolution (LR).

For comparison, we run 3 state-of-the-art algorithms with the same initial position of the target. The first tracking algorithm (SURF) [28] is based on matching of local features and descriptors. The second tracking algorithm (STRUCK) predicts the target location change between frames on the basis of structured learning [29]. The third collaborative tracking algorithm (SCM) is combined a sparsity-based discriminative classifier and a sparsity-based generative model [30]. The work [27] performed largescale experiments to evaluate the performance of recent 33 object-tracking algorithms. Tracking algorithms STRUCK and SCM perform much better than the others.

For evaluating of detection efficiency we use an evaluation metric of the overlap score. Given a tracked bounding box r t and the ground-truth bounding extent r 0 of a target object, the overlap score is defined as

S = ∥r t ∩ r 0 ∥ ∥r t ∪ r 0 ∥ ,(8)

where ∩ and ∪ represent the intersection and union operators, respectively, and ∥ • ∥ denotes the number of pixels in a region. This average overlap score (AOS) can be used as the performance measure. In addition, the overlap scores can be used for determining whether an algorithm successfully tracks a target in a frame, by testing whether S is larger than a threshold of 0.5. Also we evaluate the tracking algorithms using the average center location error (ACLE) for all image sequences from database.

Table 1 shows the average overlap score (AOS), the average center location errors (ACLE) and the Average Processing Time (APT) on a scena for all the tracking algorithms with the overlap threshold of 0.5. The evaluation results show that our proposed algorithm is faster than the others and more accurate in terms of the average center location errors. When an object moves fastly on the FM subset, the proposed algorithm performs much better than the others. However, the proposed algorithm does not perform well in the subset (IV, OCC, OV) due to illumination variation, and partial occlusion of the target. On the other subsets, the Struck, SCM, and the proposed algorithms outperform other the state-of-the-art algorithms. Fig. 2 shows sample tracking results of the proposed algorithms where the target objects are marked with red rectangles and the actually tracked objects by the proposed algorithm are marked with green rectangles.

Conclusion

A tracking algorithm using locally adaptive correlation filtering is proposed. The algorithm is designed to track multiple objects with invariance to pose, partial occlusion, clutter, and illumination variations. The algorithm employs a prediction scheme and composite correlation filters. The filters are synthesized with the help of an iterative algorithm, which optimizes discrimination capability for each target. The filters are adapted online to targets changes using information of current and past scene frames. The evaluation results show that our proposed algorithm is faster than the others and more accurate in terms of the average center location errors. On the majority test sets the proposed algorithm performs much better than the state-of-the-art algorithms.