The Usage of Optical Flow Algorithm to the Problem of Recovery Contour of the Left Ventricle of the Human Heart on the Ultrasound Image Data

Abstract

Cardiologist builds contour bounding area of the left ventricle (LV) in each frame of the ultrasound video sequence usually in a manual mode. The article describes results of application of semi-automatic algorithm using the optical ow. The visual and qualitative characteristic of contouring results of this algorithm are obtained. This algorithm is used to select the contour on ultrasound images.

Keywords

• Contouring
• left ventricle
• optical ow
• ultrasound images

Introduction

Contemporary research methods greatly simplify investigations in the medicine area. Diagnosis takes much less time and is more informative if an ultrasound image data are used for this purpose. The heart disease diagnosing on the basis of echographic imaging (echocardiography) is one of the challenges for cardiologists.

Analysis of ultrasonic image sequence allows one to investigate the heart muscle dynamics. The left ventricle (LV) of the heart is of the great interest for cardiologists, since most of various heart diseases and pathologies change, primarily, its state.

Existing devices of ultrasound heart research use tools that help doctors to calculate di erent indicators, which are necessary for the diagnosis of many diseases. To assess the state of LV, the cardiologist puts the contour bounding the region of the left ventricle on every frame of the ultrasonic images sequence, as a rule, in the manual mode.

Today, there are many di erent ultrasound scanners, each of which is equipped with any tools of LV contouring (Philips, Aloka Hitachi, Toshiba, Siemens, General Electric, and others.). However, analysis of commercial proposals of companies listed above showed that there are no devices for echocardiography with built-in software on the market, which could allow one to carry out the LV contouring in fully automatic mode. Thus, the problem of fully automatic algorithm delineation of LV is important.

Problems formulation

To solve the problem of delineation of LV on the ultrasound images, the usual methods of digital image processing do not t. The techniques such as segmentation, ltering, contrast enhancement, and morphological transformation do not give positive results of contour allocation [ 1, 4, 5, 7, 8 ].

Thus, to solve the problems of delineation, it was decided to investigate the method of optical ow [ 2, 3, 6 ]; and it is needed to write a program that will allow cardiologists to build a contour only on the rst frame of the video sequence. The contour should be build automatically starting form the second frame.

For example, in articles [ 9, 10 ] to track the areas on 3D echographic sequences, optical ow techniques, such as the "block ow" and "Real-time 3-dimensional echocardiographic" (RT3DE) were used. Since we apply the two-dimensional images, it has been already decided to use the Kanade-Lucas-Tomasi softwarebased algorithm. 3

Semi-automatic contour extraction algorithm on the video sequence Many current ow search algorithms based on the Lucas and Canada idea [11{ 14]: 1. Lucas-Kanade considered just the o set point, without distortion. 2. Tomassi-Kanade is a reformulation of the Lucas-Kanade. O set is calculated by an iterative solution of a built system of linear equations. 3. Shi-Tomassi-Kanade is an a ne distortion of the point neighborhood taken into account. 4. Jin-Favaro-Soatto is a modi cation of the Shi-Tomassi-Kanade considering a ne illumination changes of point neighborhood.

In this paper, the algorithm based on a Kanade-Lucas-Tomasi algorithm for tracking points on the rst frame was used. Video records of fourteen patients are used as input data. The method uses image pyramid. Each image of sequence excepting the rst is obtained as convolution of the previous image with the following lter: 1=4 1=2 1=4 1=4 1=2 1=4 T : (1)

The algorithm used the following parameters: 1. BlockSize [61x61] is the neighborhood around each point, which is monitored; 2. NumPyramidLevels, 5 is the fth level of the pyramid that can handle large displacements of points between frames.

The value of BlockSize de nes the area where the point is monitored. If this value is too small, some points may be lost or displaced causing the incorrect contouring. Example of the BlockSize parameter is shown in Fig.1.

The value of NumPyramidLevels parameter determines what size of the object displacement should be tracked. Without this option a dramatic valve shift causes the displacement of the tracking point. Example of NumPyramidLevels is shown in Fig.2.

It experimentally has been found, that contours constructed by this method is satisfactory only in the rst cycle of the heartbeat on the video sequence. Therefore, it was decided to allocate the rst cycle of the video sequences and process only it. The oscillogram, which is present on all videos, is used to determine the rst cycle (Fig. 3). R peaks were used to highlight one cycle of heartbeats.

Figure 4 shows the volume change of the left ventricle during the cardiac interval. where S\ is the intersection of square of area, limited by expert contour, and area, formed from classi ed pixels, Scont is the square of the area formed from the classi ed pixels; { recall

where Sexp is the area of the region bounded by the expert contour; { F-measure { the feature of the mass center motion of the left ventricle has been used as a quality criterion of the LV contour construction; next, for each patient the area of the respective ellipses was calculated that compared with the area of the left ventricle in diastole (the contour of the rst frame of the cardiac cycle)

K =

Sellipse ;

Sdiast cont where Sellipse is the area of the ellipse bounding CM LV, Sdiast cont is the area of the region contour in diastole.

Quantitative estimates of these parameters are presented in Table 1.

F 0.96 0.97 0.96 0.97 0.98 0.95 0.98 0.94 0.93 0.98 0.98 0.95 0.96 0.98 (5) 1 2 3 4 5 6 7 8 9 10 11 12 13 14

K 0.21 % 1.58 % 2.68 % 0.13 % 0.16 % 0.10 % 0.08 % 0.41 % 0.25 % 0.09 % 0.26 % 0.23 % 0.33 % 0.16 % 0.94 0.96 0.96 0.95 0.98 0.93 0.99 0.94 0.92 0.99 0.97 0.94 0.95 0.98 0.98 0.98 0.97 0.99 0.98 0.98 0.98 0.95 0.94 0.98 0.99 0.97 0.98 0.99 This paper studied the optic ow method as a semi-automatic contouring algorithm. The Kanade-Lucas-Tomasi algorithm is applied to tracking points of the contour on the video sequence. The parameters of the algorithm are analyzed and those were choosen, that allow more accurate delineation of the left ventricle.

The visual and qualitative characteristic of contouring results are obtained using this algorithm. The program was developed that allows cardiologists to use this algorithm and provide the assessments of the construction quality.

Based on the results of the contour construction, it is possible to conclude that this algorithm could preferally be used in the solution of the LV contouring problem.

References

1. Hans , C. , Mikhail , G. , Danilouchkine and Boudewijn, P. F. : Segmentation of Sparse and Arbitrarily Oriented Cardiac MRI Data Using a 3D-ASM (in Russian) . FIMH LNCS 3504 , 33 { 43 ( 2005 )

2. Veenman C. , Reinders M. and Backer E. : Resolving motion correspondence for densely moving points . Pattern Analysis Machine Intelligence . Vol. 23 , No. 1 , 54 { 72 ( 2001 )

3. Yilmaz A. , Javed O. and Shah M. : Object tracking: A survey . ACM Computing Surveys . Vol. 38 , No. 4 , ( 2006 )

4. Bobkova , A. O. , Porshnev , S. V. , Zuzin , V. V. and Bobkov , V. V. : Analysis results of delineation of the left ventricle on the echo graphic images of patients with pathologies by an automatic algorithm (in Russian) . Proceedings of the 24th International Crimean Conference Microwave equipment and telecommunication technology . 1073 { 1074 ( 2014 )

5. Bobkova , A. O. , Porshnev , S. V. , Zuzin , V. V. and Bobkov , V. V. : A study of methods for removing speckle noise on ultrasound images (in Russian) . Proceedings of the 23rd International Conference on Computer Graphics and Vision . 244 { 246 ( 2013 )

6. Konushin , A. : Point feature tracking (in Russian) . Computer graphics and multimedia. No. 5 ( 2003 )

7. Bobkova , A. O. , Porshnev , S. V. , Zuzin , V. V. and Bobkov , V. V. : A method of semi-automatic delineation of the left ventricle of the human heart on the echocardiographic images (in Russian) . Fundamental research. No. 8{1 , 44 { 48 ( 2013 )

8. Priorov , A. L. , Hrjashhev , V. V. and Sladkov , M. B.. : Improving the quality of ultrasound medical images (in Russian) . Medical equipment. No. 4 , 11 { 14 ( 2008 )

9. Linguraru , M. G. , Vasilyev , N. V. , Marx , G. .R. , Tworetzky W. , Del Nido PJ and Howe , R. D.: Fast block ow tracking of atrial septal defects in f4Dg echocardiography. Medical Image Analysis . No. 12 , 397 { 412 ( 2008 )

10. Veronesi F. , Corsi C. , Caiani , E. G. , Sarti A . and Lamberti C. : Tracking of left ventricular long axis from real-time three-dimensional echocardiography using optical ow techniques . IEEE Trans Inf Technol Biomed . No. 10 , 174 { 181 ( 2006 )

11. Lucas , Bruce D. and Takeo Kanade : An Iterative Image Registration Technique with an Application to Stereo Vision . Proceedings of the 7th International Joint Conference on Arti cial Intelligence . 674 { 679 ( 1981 )

12. Tomasi , Carlo and Takeo Kanade: Detection and Tracking of Point Features . Computer Science Department, Carnegie Mellon University. ( 1991 )

13. Shi , Jianbo and Carlo Tomasi: Good Features to Track . IEEE Conference on Computer Vision and Pattern Recognition . 593 { 600 ( 1994 )

14. Kalal , Zdenek, Krystian Mikolajczyk and Jiri Matas: Forward-Backward Error: Automatic Detection of Tracking Failures . Proceedings of the 20th International Conference on Pattern Recognition . 2756 { 2759 ( 2010 )

15. Zyuzin , V. V. , Bobkova , A. O. , Porshnev , S. V. , Mukhtarov , A. A. and Bobkov , V. V. : The Study of Applicability of the Decision Tree Method for Contouring of the Left Ventricle Area in Echographic Video Data . Proceedings of SPIE - The International Society for Optical Engineering . ( 2016 )