=Paper=
{{Paper
|id=Vol-1710/paper33
|storemode=property
|title=Detector of Interest Point within Region of Interest on NBI Endoscopy Images
|pdfUrl=https://ceur-ws.org/Vol-1710/paper33.pdf
|volume=Vol-1710
|authors=Dmitry M. Stepanov,Vyacheslav V. Mizgulin,Vsevolod V. Kosulnikov,Radi M. Kadushnikov,Evgeny D. Fedorov,Olga A. Buntseva
|dblpUrl=https://dblp.org/rec/conf/aist/StepanovMKKFB16
}}
==Detector of Interest Point within Region of Interest on NBI Endoscopy Images==
https://ceur-ws.org/Vol-1710/paper33.pdf
Detector of Interest Point within Region of
Interest on NBI Endoscopy Images
Dmitry M. Stepanov1 , Vyacheslav V. Mizgulin2 , Vsevolod V. Kosulnikov1 ,
Radi M. Kadushnikov1 , Evgeny D. Fedorov3 , and Olga A. Buntseva4
1
SIAMS Ltd.
2
Ural Federal University named after the first President of Russia B. N. Yeltsin
3
Pirogov Russian National Research Medical University (RNRMU)
4
Lomonosov Moscow State University
Abstract. This paper presents a new method for detecting scale in-
variant interest point from region of interest on NBI endoscopy images.
This work is related to design of decision support systems in the area
of gastrointestinal endoscopy with image classication facility. The use of
a computer-based system could support the doctor when making a di-
agnosis and help to avoid human subjectivity. Our method is based on
computing a skeletons of gastric mucosa pit-patterns. As a result of ap-
plying the proposed approach to improve the accuracy of a selection of
distinctive points, this allow a selection region of interest for endoscopy
images in real time. These points are invariant to scale, rotation and
translation as well as robust to illumination changes and limited changes
of viewpoint.
Keywords: image analysis, endoscopy, cancer diagnostic, decision sup-
port, point detector
1 Introduction
Gastric cancer is the second most lethal cancer in the world. Cancer causes 20%
of deaths in the European Region, being the second most important cause of
death and morbidity in Europe after cardiovascular diseases with more than 3
million new cases and 1.7 million deaths each year. In many cases cancer can be
avoided, and early detection substantially increases the chance of cure. Develop-
ments in the field of medical equipment contribute to improvements in quality of
diagnosis. Modern endoscopic technologies allow obtaining high-resolution im-
ages with distinguishable thin structure of neoplasms. Experts can use these
endoscopic images experts to predict histologic structure of a tumor. Clinical
decision making systems based on computer-aided image analysis are also ac-
tively used to improve diagnostics. One of the most common methods employed
by the said systems is image classification on the base of selected visual features.
Miyaki et al [8] investigated possibility of applying computer-based analysis to
endoscopic images received using zoom-magnification chromoscopy in order to al-
low distinction of benign tumors and early stomach cancer. Lee et al [4] presented
preliminary results of analyzing stomach suspicious neoplasm images obtained
with zoom-magnification narrow band imaging (NBI) endoscopy. Authors used
neural network to classify images of stomach mucous membrane. Coimbra et
al [10, 11] presented application of classification methods for endoscopic images
obtained using chromoscopy. The authors classified affected mucosal areas using
suggestions by Dinis-Ribeiro [1]. Tamaki et al [12] made a research in the area
of classifying the endoscopic images obtained with zoom-magnification narrow
band imaging endoscopy. Authors used the NBI magnification findings classifi-
cation scheme, presented in [3]. However, due to complexity of processed images
existing solutions require manual selection of interest area. That, in term, does
not allow implementation of a real-time decision support system In order to
allow automated selection of local features within the interest area it is neces-
sary to find an optimal interest point detector. Such a detector should provide
the greatest amount of points captured within an interest area, and satisfy the
following criteria introduced in [13]:
– repeatability;
– distinctiveness/informativeness;
– locality;
– quantity;
– accuracy;
– efficiency.
The purpose of the work was to develop a decision support system for gas-
trointestinal endoscopy. It was suggested that this system should possess the
following features:
– self-training capabilities in order to allow diagnosis of various pathologies
using images obtained with various endoscopic methods;
– real time operation, in order to allow decision-making in course of inspection,
and not after wards;
– completely automated implementation work of analysis algorithms that does
not require additional operator training.
The following article describes the approach taken for detection of local fea-
tures within the interest areas of source images obtained with zoom-magnification
NBI endoscopy. This approach will allow implementation of image classification
that does not require selection of interest areas. Application of a presented in-
terest point detector will allow implementing the real-time operation mode for
clinical decision support system.
2 Point detection
Analysis of endoscopic images is complicated by the presence of various artifacts
that appear in course of image acquisition, including highlights (that appear
because of an almost concentric arrangement of illuminator and camera, and
wet mucous surface of stomach as well), floating scale, geometric deformations
(associated with use of a wide-angle lens), and uneven brightness. The presence
of artifacts requires image preprocessing, image-specific selection of analysis pa-
rameters, and use of invariant methods.
Information relevant for the purpose of image classification is contained in
several fragments of an analyzed image. Therefore, it is necessary to consider
local features that belong to a given region of interest. Basing upon the results
of textual analysis presented in [9] that allowed creating the list of key words
describing key objects present on endoscopic images, and properties of the said
objects, the authors specified properties of the key points that should be selected
on endoscopic images, and determined properties of the objects that are impor-
tant for decision-making. To highlight these points of interests, authors propose
a detector that is invariant for the most distortions of an image, and uses the
principle of constructing skeletons for gastric mucosa pit-patterns. The first step
of using this detector involves application of image convolution with the Gaus-
sian kernel, and allocation of pit-pattern ridges using Difference of Rotating Half
Smoothing Filters [5]. The use of similar methods was proposed for extracting
features of endoscopic images in [6, 9]. The pit-patterns skeleton was then built
using fast fully parallel thinning algorithms [2], and nodes of this skeleton were
considered to be interest points. The resulting points were invariant to uneven
brightness, geometric deformation and rotation. Scale was selected using meth-
ods proposed by Mikolajczyk and Schmid [7]. In this method, a set of images
with different resolution represented a so-called scale-space. Different resolution
levels were obtained by varying the value of sigma parameter in course of image
convolution with Gaussian kernel. The authors then calculated Laplacian re-
sponse function values for the selected points of interest at different scale levels.
Local maximum of the function was taken to represent a characteristic scale.
Fig. 1. Example selection skeleton
3 Experimental results
Image collection and annotation and collection software was designed and imple-
mented. Endoscopy specialists were able to upload images, manually annotate
and enter results of histology. For testing purposes, a dataset consisting of 204
images was collected. Figure 2 displays the expert-annotated image, and the set
of points for this image selected using our detector.
Fig. 2. The expert-annotated image, and the set of points for this image selected using
our detector
Results of comparing several popular interest point detectors are presented in
table 1. Detectors were implemented using the lip-vireo library5 . Table contains
average values of results for all images within the test set. Names of the detectors
are given in the first row of the table. The second row demonstrates percentage
of detected points. It can be observed that the highest amount of points was
detected by the DoG solution. The ratio of correct points to detected points
is given in row three, with The Skeleton Nodes detector providing the highest
value.
Table 1. Row 2: percentage of points for which a characteristic scale is detected. Row
3: percentage of points for which a correct scale is detected with respect to detected
points.
Difference of Laplacian of Harris- Hessian of Fast Skeleton
Gaussians (DoG) Gaussian (LoG) Laplacian Laplacian Hessian Nodes
detected 0,84% 0,10% 0,11% 0,10% 0,14% 0,15%
correct/
35,64% 35,58% 39,65% 35,89% 40,6% 45,88%
detected
5
http://pami.xmu.edu.cn/˜wlzhao/lip-vireo.htm
The results demonstrate that the detector provides good accuracy of selecting
points. High share of detecting informative points allows image classification
without manual selection of interest regions, and visualization of expected region
of interest as shown in figure 3.
Fig. 3. Example selection Region of Interest by interest points
4 Conclusions and perspectives
Existing solutions require manual selection of interest region, not real-time oper-
ation of decision support system. The authors, using the results of analyzing text
conclusions, proposed a detector of key points to be used for automatic selection
of visual features from the region of interest. The said detector provides the great-
est amount of allocated key points from the region of interest. These points are
invariant to uneven brightness, geometric deformation, rotation and scale. Im-
age annotation and collection software was designed and implemented, allowing
medical specialists to upload endoscopy images, manually annotate and input
histology results. Further research goals include collection of training samples
for all diagnoses with histological confirmation and application of classification
methods for endoscopic images.
Acknowledgements.
The work was done within the framework of the project performed by SIAMS
company, and supported by the Ministry of Education and Science of the Rus-
sian Federation (Grant agreement 14.576.21.0018 dated June 27, 2014). Project
(applied research) unique ID RFMEFI57614X0018.
References
1. Dinis-Ribeiro, M., da Costa-Pereira, A., Lopes, C., Lara-Santos, L., Guilherme,
M., Moreira-Dias, L., Lomba-Viana, H., Ribeiro, A., Santos, C., Soares, J., et al.:
Magnification chromoendoscopy for the diagnosis of gastric intestinal metaplasia
and dysplasia. Gastrointestinal endoscopy 57(4), 498–504 (2003)
2. Guo, Z., Hall, R.W.: Fast fully parallel thinning algorithms. CVGIP: Image Un-
derstanding 55(3), 317–328 (1992)
3. Kanao, H., Tanaka, S., Oka, S., Hirata, M., Yoshida, S., Chayama, K.: Narrow-
band imaging magnification predicts the histology and invasion depth of colorectal
tumors. Gastrointestinal endoscopy 69(3), 631–636 (2009)
4. Lee, T.C., Lin, Y.H., Uedo, N., Wang, H.P., Chang, H.T., Hung, C.W.: Computer-
aided diagnosis in endoscopy: A novel application toward automatic detection of
abnormal lesions on magnifying narrow-band imaging endoscopy in the stomach.
In: Engineering in Medicine and Biology Society (EMBC), 2013 35th Annual In-
ternational Conference of the IEEE. pp. 4430–4433. IEEE (2013)
5. Magnier, B., Montesinos, P., Diep, D.: Ridges and valleys detection in images using
difference of rotating half smoothing filters. In: Advanced Concepts for Intelligent
Vision Systems. pp. 261–272. Springer (2011)
6. Majewski, P., Jedruch, W.: Endoscopy images classification with kernel based
learning algorithms. Innovations in applied artificial intelligence pp. 167–180 (2005)
7. Mikolajczyk, K., Schmid, C.: Indexing based on scale invariant interest points.
In: Computer Vision, 2001. ICCV 2001. Proceedings. Eighth IEEE International
Conference on. vol. 1, pp. 525–531. IEEE (2001)
8. Miyaki, R., Yoshida, S., Tanaka, S., Kominami, Y., Sanomura, Y., Matsuo, T.,
Oka, S., Raytchev, B., Tamaki, T., Koide, T., et al.: Quantitative identification of
mucosal gastric cancer under magnifying endoscopy with flexible spectral imaging
color enhancement. Journal of gastroenterology and hepatology 28(5), 841–847
(2013)
9. Mizgulin, V.V., Stepanov, D.M., Kamentsev, S.A., Kadushnikov, R.M., Fedorov,
E.D., Buntseva, O.A.: Hybrid classification approach to decision support for en-
doscopy in gastrointestinal tract. In: Analysis of Images, Social Networks and
Texts, pp. 218–223. Springer (2015)
10. Sousa, A., Dinis-Ribeiro, M., Areia, M., Coimbra, M.: Identifying cancer regions
in vital-stained magnification endoscopy images using adapted color histograms.
In: Image Processing (ICIP), 2009 16th IEEE International Conference on. pp.
681–684. IEEE (2009)
11. Sousa, R., Ribeiro, M.D., Pimentel-Nunes, P., Tavares Coimbra, M.: Impact of
svm multiclass decomposition rules for recognition of cancer in gastroenterology
images. In: Computer-Based Medical Systems (CBMS), 2013 IEEE 26th Interna-
tional Symposium on. pp. 405–408. IEEE (2013)
12. Tamaki, T., Yoshimuta, J., Kawakami, M., Raytchev, B., Kaneda, K., Yoshida, S.,
Takemura, Y., Onji, K., Miyaki, R., Tanaka, S.: Computer-aided colorectal tumor
classification in nbi endoscopy using local features. Medical image analysis 17(1),
78–100 (2013)
13. Tuytelaars, T., Mikolajczyk, K.: Local invariant feature detectors: a survey. Foun-
dations and Trends R in Computer Graphics and Vision 3(3), 177–280 (2008)