=Paper=
{{Paper
|id=Vol-1353/paper_20
|storemode=property
|title=On Fuzzification of Color Spaces for Medical Decision Support in Video Capsule Endoscopy
|pdfUrl=https://ceur-ws.org/Vol-1353/paper_20.pdf
|volume=Vol-1353
|dblpUrl=https://dblp.org/rec/conf/maics/Prasath15
}}
==On Fuzzification of Color Spaces for Medical Decision Support in Video Capsule Endoscopy==
https://ceur-ws.org/Vol-1353/paper_20.pdf
On Fuzzification of Color Spaces for Medical Decision Support in Video Capsule
Endoscopy
V. B. Surya Prasath
Computational Imaging and Visualization Analysis Lab
Department of Computer Science
University of Missouri-Columbia
Columbia MO 65211 USA
Abstract
Advances in image and video processing algorithms and
the availability of computational resources has paved
the way for real-time medical decision support sys-
tems to be a reality. Video capsule endoscopy is a di-
rect imaging method for gastrointestinal regions and
produces large scale color video data. Fuzzification of
color spaces can improve contextual description based
tasks that are required in medical decision support. We
consider abnormalities detection in video capsule en-
doscopy using fuzzy sets and logic theory on different
colorspaces. Application in retrieval of bleeding detec-
tion and polyp vascularization are given as examples of
the methodology considered here and preliminary re-
sults indicate we obtain promising retrieval results.
Introduction
Figure 1: Each VCE exam involves a human operator (gas-
Video capsule endoscopy (VCE) is a revolutionary imag- troenterologist) looking at various abnormal frames and
ing technique which paved the way for unprecedented di- classifying the impressions using linguistic terms. Auto-
rect visualization of the gastrointestinal tract without much matic classification using a mapping of such linguistic terms
discomfort to patients. A typical colon VCE exam produces to colors using fuzzy logic can be used for various tasks such
around 8 hours of color (RGB) video data. For example, Pill- as image retrieval.
cam Colon capsule endoscope (Given Imaging, Yoqneam,
Israel) produces approximately 30, 000 frames per patient
and more than 1.6 million patients worldwide have used
capsule endoscopy over the past 10 years. Automatic algo- ample in HSI (Hue, Saturation, Intensity) space is provided
rithms can help augment computer aided diagnosis (CAD) in for obtaining a correspondence between colors and human
VCE medical decision support systems and can help reduce impressions. In this work, we adapt the ground-work done
the burden on gastroenterologists (Rey 2008; Niwa et al. in (Shamoi, Kawanaka, and Inoue 2014) to the VCE color
2008). For example, polyp detection (Figueiredo et al. 2010; space fuzzification for abnormality classifications. We pro-
2011), mucosa surface identification (Prasath, Figueiredo, vide an overview of different color spaces (RGB, CMYK,
and Figueiredo 2011; Prasath et al. 2012; Prasath and Del- HSV, La*b*) and their fuzzifications to organize all pos-
hibabu 2015b), contrast enhancement (Prasath and Del- sible human operator color perceptions of abnormalities in
hibabu 2015a). Nevertheless, designing automatic methods VCE images. Using operator defined impressions expressed
for automatically analyzing the VCE imagery via image in terms of linguistic terms we provide retrieval examples.
processing and computer vision techniques pose significant We note the overall framework is general in the sense that it
challenges as we are dealing with big data. can be expanded with domain knowledge for various related
Image processing involves uncertainty quantification tasks.
and fuzzy techniques are effective in handling various The rest of the paper is organized as follows. Next section
tasks (Kerre and Nachtegael(eds) 2000; Vlachos and Ser- introduces different color spaces useful in VCE imagery and
giadis 2006; Shamoi, Inoue, and Kawanaka 2014b; 2014a). fuzzification techniques for representing color perceptions
Recently, Shamoi et al (Shamoi, Kawanaka, and Inoue 2014) of human operators using fuzzy sets. Next, we provide some
used fuzzification of HSI color space for apparel coordina- example classification results on VCE images for bleeding
tion. In (Shamoi, Kawanaka, and Inoue 2014) a case ex- regions.
Figure 2: Fuzzy logic and mass assignment theory based
mapping of different colors and human (operator) impres-
sions of them. Image adapted from (Shamoi, Kawanaka, and
Inoue 2014).
Color Fuzzification for Abnormalities in video
Capsule Endoscopy
Fuzzification
Appearance of different abnormal regions such as polyps,
adenomas, and bleeding in VCE videos under different color
spaces provide different linguistic terms for description. (a) RGB (b) CMYK (c) HSV (d) La*b*
This can be utilized in the fuzzy logic framework advocated
in (Shamoi, Kawanaka, and Inoue 2014), see Figure 2. All Figure 3: Different color spaces can be utilized in identifying
three components of a medical decision system, namely, dif- abnormalities in VCE exams. In this example, a bleeding
ferent color spaces, impressions based on linguistic terms, region (from a Pillcam R Colon capsule image) is visualized
and mapping between them, are all interpretable using fuzzy in different color spaces and the proposal in this paper can
logic. In particular we consider an example of bleeding de- play a crucial rule in medical decision support system for
tection in VCE, see Figure 3. Note that various color spaces VCE. First row shows RGB, CMYK, HSV, La*b* images
can be utilized in the fuzzification framework and we con- and next rows their corresponding individual channels. For
sider standard color spaces such as RGB, CMYK, HSV and CMYK space, we only show magenta, yellow, key images
La*b*, we refer to (Wyszecki and Stiles 1982) for corre- and omit cyan as it did not provide any visual information in
sponding definitions and formulae. Each give a different per- this example.
spective of an abnormality, see Figure 3 for an example of
bleeding region in RGB, CMYK, HSV and La*b* spaces.
Advantages of color (spectral) information can be exploited bleeding regions respectively. Note that for different tasks
for different diagnostic decision purposes (Figueiredo et these context dependent color impressions are utilized as a
al. 2011; Prasath and Delhibabu 2015a). In this particu- query, for example light bleeding (see Figure 5(a)).
lar case of bleeding detection, gastroenterologists tend to
mark bleeding regions using linguistic terms such as dark Retrieval
red/medium red or pale red. For example, in RGB color Following (Shamoi, Kawanaka, and Inoue 2014) we utilized
space (see Figure 3(a)) the bleeding region is darker in green a taxonomy of color impressions adapted for VCE imagery
and blue channels and the overall appearance can be charac- based medical decision support systems. Here we describe
terized as dark red in the RGB spectrum. Thus, fuzzy logic it for bleeding regions and Table 1 provides the taxonomy
and mass assignment theory based mapping of different col- of color impressions in the RGB - Value case, and a similar
ors and human (operator) oriented impressions can be uti- table is generated for the polyp vascularization with RGB -
lized in making a medical decision support system. Density. Using these taxonomy we follow basic formulae in
Figure 4 shows RGB and HSV fuzzy sets which are fuzzy logic such as the intersection (minimum), union (max-
used to fuzzify different bleeding regions. In contrast to imum) of two membership functions,
the apparel coordination application considered in (Shamoi,
Kawanaka, and Inoue 2014), here we use only RGB color (µA ∩ µB)(x) = min{µA(x), µB(x)}
space and the Value (from HSV) fuzzy membership func-
tions. Hence, the {Red, Green, Blue} and {Dark, Medium, (µA ∪ µB)(x) = max{µA(x), µB(x)}
Pale} are the spectral and linguistic terms, respectively. We and α-cut
utilized a ground truth marked histogram from two expe-
fα = {x : µf (x) ≥ α}.
rienced gastroenterologists for various bleeding regions and
Figure 6 shows some examples with Red and three linguistic These basic fuzzy formulae are used to fuzzify color spaces
terms. Context dependent color impressions in the bleeding and interprets linguistic impressions of colors for composite
scenario are light, strong which indicate lighter or stronger cases. We used these formulae along with a map between
(a) RGB
(a) Input (b) Groundtruth
(b) HSV - Value
Figure 4: Example fuzzy sets for color spaces RGB and HSV (c) Bleeding
based on linguistic labels in bleeding regions classification
in VCE. Note for the HSV space we only show the Value
channel.
color impressions and color in RGB, Value in ranking simi-
lar images for bleeding region identification.
Similar interpretations are done in the polyp vasculariza-
tion and density, tortuosity are used as context dependent
impressions. Figure 5 provides example results in bleed-
ing and polyp vascularization using linguistic queries alone. (d) Input (e) Groundtruth
Figure 6 shows the corresponding ranking mechanism based
on linguistic impressions for the bleeding the vasculariza-
tion cases. As can be seen, histograms are utilized to iden-
tify the top three ranked nearest images matching the Dark
red interpretation and the retrieval results are accurate as per
gastroenterologist ground truth markings. All the retrieved
bleeding regions are from the jejunum area of the gastroin-
testinal tract and we only show top three results according to
(f) Polyp vasularization
color histogram matching, see Figure 6(a). A similar analy-
sis with RGB space and polyp vascularization density is un-
dertaken and the the query image (Figure 5(d)) is described Figure 5: Two example results in bleeding and polyp vas-
as Pale Red Dense and the ranking given in Figure 6(b) ranks cularization. (a): For a given input image of bleeding spot
the resultant images according to the density of vasculariza- which is described as (b) Dark Red the system retrieves sim-
tion. All the retrieved vascular regions contain dense ves- ilar images containing bleeding regions with (c) decreasing
sels and are malignant. We utilized 400 Pillcam R Colon ranks (matching). (d): For a given input polyp image which
capsule images for bleeding detection, these were obtained is described as (e) Dense tortuous the system retrieves sim-
from 5 different patients, and marked by two gastroenterol- ilar polyp images with (f) decreasing density. Note we only
ogists who provided ground truth regions along with bound- the top three retrieval results and the ground truth regions
aries separating bleeding from normal mucosa tissue. For are marked by human (operators) and used in training the
polyp vascularization based retrieval we used 100 images impressions and their mappings.
which are benchmarked against an automatic segmentation
method (Prasath, Pelapur, and Palaniappan 2014) for calcu-
lating the density of vascularization in polyps.
Table 1: Taxonomy of color impressions in the RGB case for
bleeding regions in VCE images.
Level Impressions Comment
I Red, Black Atomic
context independent colors
II Dark, Medium, Pale Atomic
context independent,
dependent colors
III Combinations of I & II Composite
context independent,
dependent colors
Conclusion
In this paper, we considered fuzzification of different color
spaces for medical decision support systems in gastroin- (a) Bleeding
testinal diagnosis using video capsule endoscopy. Follow-
ing (Shamoi, Kawanaka, and Inoue 2014) we utilized fuzzy
sets and logic, color space theory for VCE imagery inter-
pretation and used it for retrieval tasks. Our preliminary re-
sults in bleeding region detection and polyp vascularization
in various VCE images indicate promise for using fuzzifica-
tion techniques in a medical decision support system. Future
works include introducing shape (e.g. polyp appearance),
(b) Polyp vascularization
texture (e.g. pit patterns) features along with fuzzification
framework studied here for different VCE videos. Moreover,
increasing the number of experts (in our case study gastroen- Figure 6: Ranking based on linguistic terms and (a) color
terologists) and quantifying/enlarging the linguistic impres- histograms in the bleeding region identification (b) Density
sions is an important avenue to be explored. We also believe of vascularization in the polyps in decreasing order.
the framework considered here will help in identifying trash,
bubbles for uninformative frame classification.
Prasath, V. B. S., and Delhibabu, R. 2015a. Automatic con-
Acknowledgment trast enhancement for wireless capsule endoscopy videos
The author thanks the Gastroenterologists Dr. R. Shankar, with spectral optimal contrast-tone mapping. In Compu-
Dr. A. Sebastian from Vellore Christian Medical College tational Intelligence in Data Mining - Volume 1, 243–250.
Hospital, India for their help in interpreting VCE imagery Springer SIST (eds.: L. Jain, H. S. Behera, J. K. Mandal, D.
and Radhakrishnan Delhibabu (Kazan Federal University P. Mohapatra).
& Innopolis, Kazan, Russia) in helping with data collec- Prasath, V. B. S., and Delhibabu, R. 2015b. Automatic im-
tion/organization. This work was done while the author was age segmentation for video capsule endoscopy. In Compu-
visiting the Center for Scientific Computation and Mathe- tational Intelligence in Medical Informatics, SpringerBriefs
matical Modeling (CSCAMM) at the University of Mary- in Applied Sciences and Technology, 73–80. Springer CIMI
land, MD, USA. (eds.: N. B. Muppalaneni, V. K. Gunjan).
References Prasath, V. B. S.; Figueiredo, I. N.; Figueiredo, P. N.; and
Palaniappan, K. 2012. Mucosal region detection and 3D
Figueiredo, I. N.; Prasath, S.; Tsai, Y.-H. R.; and Figueiredo, reconstruction in wireless capsule endoscopy videos using
P. N. 2010. Automatic detection and segmentation of active contours. In 34th IEEE/EMBS International Confer-
colonic polyps in wireless capsule imagess. Technical Re- ence, 4014–4017.
port 10–37, University of Coimbra, Portugal.
Figueiredo, P. N.; Figueiredo, I. N.; Surya Prasath; and Tsai, Prasath, V. B. S.; Figueiredo, I. N.; and Figueiredo, P. N.
R. 2011. Automatic polyp detection in pillcam colon 2 cap- 2011. Colonic mucosa detection in wireless capsule endo-
sule images and videos: Preliminary feasibility report. Di- scopic images and videos. In Congress on Numerical Meth-
agnostic and Therapeutic Endoscopy 2011:7pp. Article ID ods in Engineering (CMNE 2011).
182435. Prasath, V. B. S.; Pelapur, R.; and Palaniappan, K. 2014.
Kerre, E., and Nachtegael(eds), M. 2000. Fuzzy techniques Multi-scale directional vesselness stamping based segmen-
in image processing. Heidelberg: Physica-Verlag. tation for polyps from wireless capsule endoscopy. In
Niwa, H.; Tajiri, H.; Nakajima, M.; and Yasuda, K. 2008. Figshare.
New Challenges in Gastrointestinal Endoscopy. Springer. Rey, J.-F. 2008. Future perspectives for esophageal and
colorectal capsule endoscopy: Dreams or reality? In New
Challenges in Gastrointestinal Endoscopy. Springer. 55–64.
Shamoi, P.; Inoue, A.; and Kawanaka, H. 2014a. Fuzzy color
space for apparel coordination. Open Journal of Information
Systems 1(2):20–28.
Shamoi, P.; Inoue, A.; and Kawanaka, H. 2014b. Percep-
tual color space: Motivations, methodology, applications. In
Joint 7th International Conference on Soft Computing and
Intelligent Systems (SCIS) and 15th International Sympo-
sium on Advanced Intelligent Systems (ISIS), 1354–1359.
Shamoi, P.; Kawanaka, H.; and Inoue, A. 2014. Fuzzifica-
tion of HSI color space and its use in apparel coordination.
In Proc. of 25th Modern Artificial Intelligence and Cognitive
Science Conference, 11–17. Spokane, WA, USA: CEUR-
WS.org. online CEUR-WS.org/Vol-1144/paper2.pdf.
Vlachos, I. K., and Sergiadis, G. D. 2006. A heuristic ap-
proach to intuitionistic fuzzification of color images. World
Scientific. chapter 108, 767–774.
Wyszecki, G., and Stiles, W. 1982. Color Science: Concepts
and Methods, Quantitative Data and Formulas. New York,
NY, USA: Wiley.