Detector of Interest Point within Region of Interest on NBI Endoscopy Images

Abstract

This paper presents a new method for detecting scale invariant interest point from region of interest on NBI endoscopy images. This work is related to design of decision support systems in the area of gastrointestinal endoscopy with image classication facility. The use of a computer-based system could support the doctor when making a diagnosis and help to avoid human subjectivity. Our method is based on computing a skeletons of gastric mucosa pit-patterns. As a result of applying the proposed approach to improve the accuracy of a selection of distinctive points, this allow a selection region of interest for endoscopy images in real time. These points are invariant to scale, rotation and translation as well as robust to illumination changes and limited changes of viewpoint.

Keywords

• image analysis
• endoscopy
• cancer diagnostic
• decision support
• point detector

-

Gastric cancer is the second most lethal cancer in the world. Cancer causes 20% of deaths in the European Region, being the second most important cause of death and morbidity in Europe after cardiovascular diseases with more than 3 million new cases and 1.7 million deaths each year. In many cases cancer can be avoided, and early detection substantially increases the chance of cure. Developments in the eld of medical equipment contribute to improvements in quality of diagnosis. Modern endoscopic technologies allow obtaining high-resolution images with distinguishable thin structure of neoplasms. Experts can use these endoscopic images experts to predict histologic structure of a tumor. Clinical decision making systems based on computer-aided image analysis are also actively used to improve diagnostics. One of the most common methods employed by the said systems is image classi cation on the base of selected visual features. Miyaki et al [ 8 ] investigated possibility of applying computer-based analysis to endoscopic images received using zoom-magni cation chromoscopy in order to allow distinction of benign tumors and early stomach cancer. Lee et al [ 4 ] presented preliminary results of analyzing stomach suspicious neoplasm images obtained with zoom-magni cation narrow band imaging (NBI) endoscopy. Authors used neural network to classify images of stomach mucous membrane. Coimbra et al [ 10, 11 ] presented application of classi cation methods for endoscopic images obtained using chromoscopy. The authors classi ed a ected mucosal areas using suggestions by Dinis-Ribeiro [ 1 ]. Tamaki et al [ 12 ] made a research in the area of classifying the endoscopic images obtained with zoom-magni cation narrow band imaging endoscopy. Authors used the NBI magni cation ndings classi cation scheme, presented in [ 3 ]. However, due to complexity of processed images existing solutions require manual selection of interest area. That, in term, does not allow implementation of a real-time decision support system In order to allow automated selection of local features within the interest area it is necessary to nd an optimal interest point detector. Such a detector should provide the greatest amount of points captured within an interest area, and satisfy the following criteria introduced in [ 13 ]: { repeatability; { distinctiveness/informativeness; { locality; { quantity; { accuracy; { e ciency.

The purpose of the work was to develop a decision support system for gastrointestinal endoscopy. It was suggested that this system should possess the following features: { self-training capabilities in order to allow diagnosis of various pathologies using images obtained with various endoscopic methods; { real time operation, in order to allow decision-making in course of inspection, and not after wards; { completely automated implementation work of analysis algorithms that does not require additional operator training.

The following article describes the approach taken for detection of local features within the interest areas of source images obtained with zoom-magni cation NBI endoscopy. This approach will allow implementation of image classi cation that does not require selection of interest areas. Application of a presented interest point detector will allow implementing the real-time operation mode for clinical decision support system. 2

Point detection

Analysis of endoscopic images is complicated by the presence of various artifacts that appear in course of image acquisition, including highlights (that appear because of an almost concentric arrangement of illuminator and camera, and wet mucous surface of stomach as well), oating scale, geometric deformations (associated with use of a wide-angle lens), and uneven brightness. The presence of artifacts requires image preprocessing, image-speci c selection of analysis parameters, and use of invariant methods.

Information relevant for the purpose of image classi cation is contained in several fragments of an analyzed image. Therefore, it is necessary to consider local features that belong to a given region of interest. Basing upon the results of textual analysis presented in [ 9 ] that allowed creating the list of key words describing key objects present on endoscopic images, and properties of the said objects, the authors speci ed properties of the key points that should be selected on endoscopic images, and determined properties of the objects that are important for decision-making. To highlight these points of interests, authors propose a detector that is invariant for the most distortions of an image, and uses the principle of constructing skeletons for gastric mucosa pit-patterns. The rst step of using this detector involves application of image convolution with the Gaussian kernel, and allocation of pit-pattern ridges using Di erence of Rotating Half Smoothing Filters [ 5 ]. The use of similar methods was proposed for extracting features of endoscopic images in [ 6, 9 ]. The pit-patterns skeleton was then built using fast fully parallel thinning algorithms [ 2 ], and nodes of this skeleton were considered to be interest points. The resulting points were invariant to uneven brightness, geometric deformation and rotation. Scale was selected using methods proposed by Mikolajczyk and Schmid [ 7 ]. In this method, a set of images with di erent resolution represented a so-called scale-space. Di erent resolution levels were obtained by varying the value of sigma parameter in course of image convolution with Gaussian kernel. The authors then calculated Laplacian response function values for the selected points of interest at di erent scale levels. Local maximum of the function was taken to represent a characteristic scale.

Experimental results

Image collection and annotation and collection software was designed and implemented. Endoscopy specialists were able to upload images, manually annotate and enter results of histology. For testing purposes, a dataset consisting of 204 images was collected. Figure 2 displays the expert-annotated image, and the set of points for this image selected using our detector. 5 http://pami.xmu.edu.cn/~wlzhao/lip-vireo.htm

The results demonstrate that the detector provides good accuracy of selecting points. High share of detecting informative points allows image classi cation without manual selection of interest regions, and visualization of expected region of interest as shown in gure 3.

Conclusions and perspectives

Existing solutions require manual selection of interest region, not real-time operation of decision support system. The authors, using the results of analyzing text conclusions, proposed a detector of key points to be used for automatic selection of visual features from the region of interest. The said detector provides the greatest amount of allocated key points from the region of interest. These points are invariant to uneven brightness, geometric deformation, rotation and scale. Image annotation and collection software was designed and implemented, allowing medical specialists to upload endoscopy images, manually annotate and input histology results. Further research goals include collection of training samples for all diagnoses with histological con rmation and application of classi cation methods for endoscopic images.

Acknowledgements.

The work was done within the framework of the project performed by SIAMS company, and supported by the Ministry of Education and Science of the Russian Federation (Grant agreement 14.576.21.0018 dated June 27, 2014). Project (applied research) unique ID RFMEFI57614X0018.

References

1. Dinis-Ribeiro , M. , da Costa-Pereira, A. , Lopes , C. , Lara-Santos , L. , Guilherme , M. , Moreira-Dias , L. , Lomba-Viana , H. , Ribeiro , A. , Santos , C. , Soares , J. , et al.: Magni cation chromoendoscopy for the diagnosis of gastric intestinal metaplasia and dysplasia . Gastrointestinal endoscopy 57(4) , 498 { 504 ( 2003 )

2. Guo , Z. , Hall , R.W.: Fast fully parallel thinning algorithms . CVGIP: Image Understanding 55 ( 3 ), 317 { 328 ( 1992 )

3. Kanao , H. , Tanaka , S. , Oka , S. , Hirata , M. , Yoshida , S. , Chayama , K. : Narrowband imaging magni cation predicts the histology and invasion depth of colorectal tumors . Gastrointestinal endoscopy 69(3) , 631 { 636 ( 2009 )

4. Lee , T.C. , Lin , Y.H. , Uedo , N. , Wang , H.P. , Chang , H.T. , Hung , C.W. : Computeraided diagnosis in endoscopy: A novel application toward automatic detection of abnormal lesions on magnifying narrow-band imaging endoscopy in the stomach . In: Engineering in Medicine and Biology Society (EMBC) , 2013 35th Annual International Conference of the IEEE. pp. 4430 { 4433 . IEEE ( 2013 )

5. Magnier , B. , Montesinos , P. , Diep , D. : Ridges and valleys detection in images using di erence of rotating half smoothing lters . In: Advanced Concepts for Intelligent Vision Systems . pp. 261 { 272 . Springer ( 2011 )

6. Majewski , P. , Jedruch , W. : Endoscopy images classi cation with kernel based learning algorithms . Innovations in applied arti cial intelligence pp. 167 { 180 ( 2005 )

7. Mikolajczyk , K. , Schmid , C. : Indexing based on scale invariant interest points . In: Computer Vision , 2001 . ICCV 2001 . Proceedings . Eighth IEEE International Conference on. vol. 1 , pp. 525 { 531 . IEEE ( 2001 )

8. Miyaki , R. , Yoshida , S. , Tanaka , S. , Kominami , Y. , Sanomura , Y. , Matsuo , T. , Oka , S. , Raytchev , B. , Tamaki , T. , Koide , T. , et al.: Quantitative identi cation of mucosal gastric cancer under magnifying endoscopy with exible spectral imaging color enhancement . Journal of gastroenterology and hepatology 28(5) , 841 { 847 ( 2013 )

9. Mizgulin , V.V. , Stepanov , D.M. , Kamentsev , S.A. , Kadushnikov , R.M. , Fedorov , E.D. , Buntseva , O.A. : Hybrid classi cation approach to decision support for endoscopy in gastrointestinal tract . In: Analysis of Images, Social Networks and Texts , pp. 218 { 223 . Springer ( 2015 )

10. Sousa , A. , Dinis-Ribeiro , M. , Areia , M. , Coimbra , M. : Identifying cancer regions in vital-stained magni cation endoscopy images using adapted color histograms . In: Image Processing (ICIP) , 2009 16th IEEE International Conference on. pp. 681 { 684 . IEEE ( 2009 )

11. Sousa , R. , Ribeiro , M.D. , Pimentel-Nunes , P. , Tavares Coimbra , M. : Impact of svm multiclass decomposition rules for recognition of cancer in gastroenterology images . In: Computer-Based Medical Systems (CBMS) , 2013 IEEE 26th International Symposium on. pp. 405 { 408 . IEEE ( 2013 )

12. Tamaki , T. , Yoshimuta , J. , Kawakami , M. , Raytchev , B. , Kaneda , K. , Yoshida , S. , Takemura , Y. , Onji , K. , Miyaki , R. , Tanaka , S. : Computer-aided colorectal tumor classi cation in nbi endoscopy using local features . Medical image analysis 17(1) , 78 { 100 ( 2013 )

13. Tuytelaars , T. , Mikolajczyk , K. : Local invariant feature detectors: a survey . Foundations and Trends R in Computer Graphics and Vision 3 ( 3 ), 177 { 280 ( 2008 )