The automated tracking of subcellular structures in live microscopy image sequences is an actual problem in many biological research areas. A universal solution for this problem still does not exist due to a huge variety of data of different nature. In this work, we propose an algorithm for tracking actin filaments in 2D fluorescent image sequences. The filaments are moving in a random and abrupt manner frequently crossing each other. We used steerable filters based ridge detection followed by crossing filaments correction algorithm for filaments detection. The tracking was performed using a greedy nearest neighbor method. The quantitative evaluation of our approach was performed on several manually annotated image sequences using object tracking quality metric MOTA. It was shown that the proposed approach outperforms an existing approach in tracking accuracy. In addition, the proposed approach allows processing crossed filaments, unlike the existing methods.
Research on muscle functions is conducted for many years [
Most existing methods [
In [
In this work, we propose a new method for actin filament tracking. We use
steerable filters with Canny-like criteria [
Our filament tracking approach consists of three stages: filament detection, correction of crossed filaments, and data association.
(x) = arg max(f (x)
h(R x)); r (x) = f (x) h(R x); where f (x) is the initial image, h(x) is the filter, r is the filter response, is the feature orientation, and R is the rotation matrix.
To detect the feature in all orientations one needs to have a very fine angle grid resulting a number of convolutions with templates oriented at various angles. The steerable filters are used to simplify the computations as they can be rotated efficiently by taking a linear combination of a small number of filters.
In this work, the following filter was chosen [
= 13 q 43
,
=
We use steerable filters with Canny-like criteria [
For ridge detection we use steerable filters [
We can derive the equation to compute this filter at any orientation by applying Fourier transform to a rotated filter, combining like terms, and applying inverse Fourier (1) (2) (3) transform: h(R x) = cos2 + cos sin + sin2 ( | ( | gxx +
q1{(zx) (2 | gyy + q3{(zx) gxx) :
} gyy) +
} gxy
2 q2{(zx) gxy) + } (4)
Thus, plugging Eq. (4) into Eq. (1) we can analytically maximize the filter response by at every pixel of the image after computing only three convolutions with the filters q1(x), q2(x), and q3(x).
After getting filter response in every pixel of the image, we thin the found ridges by non-maximum suppression (we suppress all pixel values that have lower intensity than their neighbors along the angle). Then, we apply double thresholding and hysteresis to remove the noisy detections. The low threshold is computed using the Otsu method, while the high threshold is a parameter of the algorithm and is typically set in a way that the thresholded image has 90% of the brightest pixels from the image after nonmaximum suppression step.
As the result of the hysteresis we get the detected filaments. They can be seen in Fig. 3 as magenta lines. 2.2
In case the filaments cross, the detection approach described above causes splitting of one of the filaments or merging two neighboring filaments (see Fig. 3). In the third frame of Fig. 3 the filaments splitting problem is shown. In the second and the fifth frames of Fig. 3 one can see merged filaments.
A Method for Actin Filament Tracking in Fluorescent Microscopy Images 5 (a) The 11-th frame of the image sequence.
(b) The maximum projection of 5 frames (from 10-th to 15-th). (a) False merging filaments problem. Two fil- (b) False splitting filaments problem. On the aments that were merged during the detection left, the horizontal filament is split into two filstage are on the left. The result of the false aments. The result of false splitting correction merging correction algorithm where the fila- algorithm where two filaments are merged into ment is split into two is on the right. one is shown on the right.
To address this problem, we use the information from several consecutive frames (the number of frames is the parameter of the algorithm) by taking the maximum intensity over time in each pixel (maximum projection). An example of the maximum projection can be seen in Fig. 5.
The resulting maximum projection images represent the filament motion traces
within the specified period. To analyze these structures we use FIRE algorithm [
First, the maximum projection image is binarized. We consequentially apply the
steerable (see Section 2.1) and median (5 5) filters to the image. After that, the image
is thresholded by the global Otsu threshold. Then, the distance transform is applied
followed by Gaussian smoothing ( = 1:5) resulting in a smoothed distance transform E.
After that, the nucleation points are found as the local maxima in a square neighborhood
of the radius smax (which is set to 5 in our method). Then we form a set of branches
for every nucleation point. The branch begins in the nucleation point and follows the
directions given by local maxima of the distance transform E. The initial direction is
given by the local maxima on a border of a square neighborhood of the nucleation point.
The branch extension process ends when the branch meets the background or another
nucleation point. For the formal description of this approach we refer to [
After the filament traces are found each filament in every frame is assigned to its closest trace. Thus, one filament trace can have a lot of filaments associated with it. Then each trace can be described by frame numbers where it appeared ti and the number of filaments assigned to the trace in each frame nti : ft1 : nt1 ; t2 : nt2 ; ::: ; tk : ntk g ; (5) where k is the number of consecutive frames that are taken for maximum projection (we typically use k = 5).
For example, f11: 1, 12: 2, 13: 1, 14: 1, 15: 1g means that in the 11-th, 13-th, 14-th, and 15-th frames the trace was assigned to one filament, and in the 12-th frame it was assigned to two filaments.
Ideally, each trace in every frame must have exactly one filament that forms the trace. However, we can distinguish two kinds of errors: zero filaments associated with the trace in a frame (nti = 0), and more than one filaments associated with the trace in a frame (nti > 1). When we have zero filaments in a frame, it means that the filament which was supposed to be detected and assigned to this trace was not detected or merged with some other filament. We call this false merging problem (see Fig. 5a). In case we have more than one filament associated with the trace in one frame, it means that the filament was incorrectly split into two separate filaments (see Fig. 5b). The solution for both problems is described below. To simplify further descriptions, let us define m as the mean length of all filaments from the current trace where nti = 1. False Merging Filaments Correction. First, we find all traces with nti = 0 at least for one frame number. We restore lost filaments using the following procedure. If nti 1 = nti+1 = 1, we can construct filament along the trace with length m and centroid computed as mean of centroids of filaments in the neighboring frames ti 1 and ti+1. If the missing filament is in the first or last frame (nt1 = 0 or ntk = 0), we form the filament by cutting a piece of the trace of the length m from its start (or end). In case
A Method for Actin Filament Tracking in Fluorescent Microscopy Images 7 the miss of the filament was caused by merging with another filament, we have to correct the length of the latter one by removing the part corresponding to the reconstructed filament. Thus, we get two distinct filaments from one merged filament (see Fig. 5a).
frame number. For each trace, for every frame ti where nti > 1 we calculate the sum of filaments lengths in this frame lti . If the length lti is close to the mean length m (their difference is less than 20 pixels), we merge these filaments into a single one (see Fig. 5b).
Algorithm Summary. The described correction steps are used iteratively using the following algorithm: 1. Construct the set of filament traces from one aggregated image. 2. Assign all filaments to the traces. 3. Apply false splitting filaments correction to all traces. 4. Apply false merging filaments correction to all traces. 5. If any filament was split or merged, mark the corresponding traces as changed, and mark them unchanged overwise. 6. If all traces are unchanged, then the algorithm is finished. Otherwise, return to step 3. 2.3
For filament association we use the generalized nearest neighbor algorithm. The algorithm is applied iteratively from the first frame to the last frame.
Suppose that all the tracks are constructed till t-th frame f 1; : : : ; N g. We need to continue tracks from t frame to t + 1 frame. Let us describe the procedure for the track i and its last filament f .
First, we compute distances between the centroids of the filament f and all the filaments from the frame t + 1. Then, we filter out all the filaments which length differ more than 1:5 times from the length of the last filament in this track. We do not filter out the filaments shorter than 25 pixels, as the length of the filaments can vary up to 10 pixels between the frames due to the noise. After that, we filter out all filaments that are too far. We do not consider the filaments that are further than the search radius of 3r, where r is the mean of all filament displacements in the track i. Finally, for the filament f we choose the closest filament from the remaining candidates from the frame t + 1. If there is no such filament then the track is ended. The procedure is repeated greedily for all the tracks i. All filaments from frame t + 1 that were not assigned to any track are considered as the beginnings of the new tracks. 3 3.1
The data was kindly provided by the Institute of Immunology and Physiology, Ural Branch of the Russian Academy of Sciences. The image sequences consist of 30 frames with the size of 512 512. The time interval between the frames is 300 ms. For the algorithm evaluation, three sequences with 123 tracks were manually annotated. 3.2
We used the MOTA metric from the CLEAR MOT metrics [
The multiple object tracking accuracy (MOTA) is defined as
MOTA = 1
Pt mt P g t t
Pt fpt P g t t
Pt mmet
P g t t (6)
Here, mt is the number of misses, fpt is the number of false positives, mmet is the number of mismatch errors, gt is the number of ground truth objects in frame t.
Note, that MOTA can be negative (because there could be more false positives and misses than ground truth objects). 3.3
We calculated the values of MOTA metric on three manually annotated image sequences
for the FAST method [
In Table. 1 the results of MOTA are presented. Our method has higher MOTA values. It is important to note that the proposed approach is able to work with crossed filaments, while FAST method does not consider them at all, which partly affects such a difference in MOTA values.
In Fig. 6 the tracking results for the first 24 frames of one of the sequences are
visualized. To make images less cluttered the finished tracks are not shown.
In this paper, we have presented a method for tracking actin filaments in fluorescence
microscopy image sequences. The filament detection is based on ridge detection using
steerable filters with Canny-like criteria. In addition, the filament detection includes
crossed filaments correction algorithm based on filament trace analysis. The tracking
is based on the greedy nearest neighbor approach that takes into account the distance
between filaments and their lengths. We compared the results of the proposed method
with the existing approach [