=Paper= {{Paper |id=Vol-1353/paper_20 |storemode=property |title=On Fuzzification of Color Spaces for Medical Decision Support in Video Capsule Endoscopy |pdfUrl=https://ceur-ws.org/Vol-1353/paper_20.pdf |volume=Vol-1353 |dblpUrl=https://dblp.org/rec/conf/maics/Prasath15 }} ==On Fuzzification of Color Spaces for Medical Decision Support in Video Capsule Endoscopy== https://ceur-ws.org/Vol-1353/paper_20.pdf
 On Fuzzification of Color Spaces for Medical Decision Support in Video Capsule
                                    Endoscopy
                                                        V. B. Surya Prasath
                                       Computational Imaging and Visualization Analysis Lab
                                                Department of Computer Science
                                                 University of Missouri-Columbia
                                                   Columbia MO 65211 USA


                           Abstract
     Advances in image and video processing algorithms and
     the availability of computational resources has paved
     the way for real-time medical decision support sys-
     tems to be a reality. Video capsule endoscopy is a di-
     rect imaging method for gastrointestinal regions and
     produces large scale color video data. Fuzzification of
     color spaces can improve contextual description based
     tasks that are required in medical decision support. We
     consider abnormalities detection in video capsule en-
     doscopy using fuzzy sets and logic theory on different
     colorspaces. Application in retrieval of bleeding detec-
     tion and polyp vascularization are given as examples of
     the methodology considered here and preliminary re-
     sults indicate we obtain promising retrieval results.


                       Introduction
                                                                    Figure 1: Each VCE exam involves a human operator (gas-
Video capsule endoscopy (VCE) is a revolutionary imag-              troenterologist) looking at various abnormal frames and
ing technique which paved the way for unprecedented di-             classifying the impressions using linguistic terms. Auto-
rect visualization of the gastrointestinal tract without much       matic classification using a mapping of such linguistic terms
discomfort to patients. A typical colon VCE exam produces           to colors using fuzzy logic can be used for various tasks such
around 8 hours of color (RGB) video data. For example, Pill-        as image retrieval.
cam Colon capsule endoscope (Given Imaging, Yoqneam,
Israel) produces approximately 30, 000 frames per patient
and more than 1.6 million patients worldwide have used
capsule endoscopy over the past 10 years. Automatic algo-           ample in HSI (Hue, Saturation, Intensity) space is provided
rithms can help augment computer aided diagnosis (CAD) in           for obtaining a correspondence between colors and human
VCE medical decision support systems and can help reduce            impressions. In this work, we adapt the ground-work done
the burden on gastroenterologists (Rey 2008; Niwa et al.            in (Shamoi, Kawanaka, and Inoue 2014) to the VCE color
2008). For example, polyp detection (Figueiredo et al. 2010;        space fuzzification for abnormality classifications. We pro-
2011), mucosa surface identification (Prasath, Figueiredo,          vide an overview of different color spaces (RGB, CMYK,
and Figueiredo 2011; Prasath et al. 2012; Prasath and Del-          HSV, La*b*) and their fuzzifications to organize all pos-
hibabu 2015b), contrast enhancement (Prasath and Del-               sible human operator color perceptions of abnormalities in
hibabu 2015a). Nevertheless, designing automatic methods            VCE images. Using operator defined impressions expressed
for automatically analyzing the VCE imagery via image               in terms of linguistic terms we provide retrieval examples.
processing and computer vision techniques pose significant          We note the overall framework is general in the sense that it
challenges as we are dealing with big data.                         can be expanded with domain knowledge for various related
   Image processing involves uncertainty quantification             tasks.
and fuzzy techniques are effective in handling various                 The rest of the paper is organized as follows. Next section
tasks (Kerre and Nachtegael(eds) 2000; Vlachos and Ser-             introduces different color spaces useful in VCE imagery and
giadis 2006; Shamoi, Inoue, and Kawanaka 2014b; 2014a).             fuzzification techniques for representing color perceptions
Recently, Shamoi et al (Shamoi, Kawanaka, and Inoue 2014)           of human operators using fuzzy sets. Next, we provide some
used fuzzification of HSI color space for apparel coordina-         example classification results on VCE images for bleeding
tion. In (Shamoi, Kawanaka, and Inoue 2014) a case ex-              regions.
Figure 2: Fuzzy logic and mass assignment theory based
mapping of different colors and human (operator) impres-
sions of them. Image adapted from (Shamoi, Kawanaka, and
Inoue 2014).


Color Fuzzification for Abnormalities in video
             Capsule Endoscopy
Fuzzification
Appearance of different abnormal regions such as polyps,
adenomas, and bleeding in VCE videos under different color
spaces provide different linguistic terms for description.          (a) RGB        (b) CMYK         (c) HSV        (d) La*b*
This can be utilized in the fuzzy logic framework advocated
in (Shamoi, Kawanaka, and Inoue 2014), see Figure 2. All         Figure 3: Different color spaces can be utilized in identifying
three components of a medical decision system, namely, dif-      abnormalities in VCE exams. In this example, a bleeding
ferent color spaces, impressions based on linguistic terms,      region (from a Pillcam R Colon capsule image) is visualized
and mapping between them, are all interpretable using fuzzy      in different color spaces and the proposal in this paper can
logic. In particular we consider an example of bleeding de-      play a crucial rule in medical decision support system for
tection in VCE, see Figure 3. Note that various color spaces     VCE. First row shows RGB, CMYK, HSV, La*b* images
can be utilized in the fuzzification framework and we con-       and next rows their corresponding individual channels. For
sider standard color spaces such as RGB, CMYK, HSV and           CMYK space, we only show magenta, yellow, key images
La*b*, we refer to (Wyszecki and Stiles 1982) for corre-         and omit cyan as it did not provide any visual information in
sponding definitions and formulae. Each give a different per-    this example.
spective of an abnormality, see Figure 3 for an example of
bleeding region in RGB, CMYK, HSV and La*b* spaces.
Advantages of color (spectral) information can be exploited      bleeding regions respectively. Note that for different tasks
for different diagnostic decision purposes (Figueiredo et        these context dependent color impressions are utilized as a
al. 2011; Prasath and Delhibabu 2015a). In this particu-         query, for example light bleeding (see Figure 5(a)).
lar case of bleeding detection, gastroenterologists tend to
mark bleeding regions using linguistic terms such as dark        Retrieval
red/medium red or pale red. For example, in RGB color            Following (Shamoi, Kawanaka, and Inoue 2014) we utilized
space (see Figure 3(a)) the bleeding region is darker in green   a taxonomy of color impressions adapted for VCE imagery
and blue channels and the overall appearance can be charac-      based medical decision support systems. Here we describe
terized as dark red in the RGB spectrum. Thus, fuzzy logic       it for bleeding regions and Table 1 provides the taxonomy
and mass assignment theory based mapping of different col-       of color impressions in the RGB - Value case, and a similar
ors and human (operator) oriented impressions can be uti-        table is generated for the polyp vascularization with RGB -
lized in making a medical decision support system.               Density. Using these taxonomy we follow basic formulae in
   Figure 4 shows RGB and HSV fuzzy sets which are               fuzzy logic such as the intersection (minimum), union (max-
used to fuzzify different bleeding regions. In contrast to       imum) of two membership functions,
the apparel coordination application considered in (Shamoi,
Kawanaka, and Inoue 2014), here we use only RGB color                        (µA ∩ µB)(x) = min{µA(x), µB(x)}
space and the Value (from HSV) fuzzy membership func-
tions. Hence, the {Red, Green, Blue} and {Dark, Medium,                      (µA ∪ µB)(x) = max{µA(x), µB(x)}
Pale} are the spectral and linguistic terms, respectively. We    and α-cut
utilized a ground truth marked histogram from two expe-
                                                                                   fα = {x : µf (x) ≥ α}.
rienced gastroenterologists for various bleeding regions and
Figure 6 shows some examples with Red and three linguistic       These basic fuzzy formulae are used to fuzzify color spaces
terms. Context dependent color impressions in the bleeding       and interprets linguistic impressions of colors for composite
scenario are light, strong which indicate lighter or stronger    cases. We used these formulae along with a map between
                           (a) RGB




                                                                              (a) Input                   (b) Groundtruth




                        (b) HSV - Value

Figure 4: Example fuzzy sets for color spaces RGB and HSV                                  (c) Bleeding
based on linguistic labels in bleeding regions classification
in VCE. Note for the HSV space we only show the Value
channel.



color impressions and color in RGB, Value in ranking simi-
lar images for bleeding region identification.
   Similar interpretations are done in the polyp vasculariza-
tion and density, tortuosity are used as context dependent
impressions. Figure 5 provides example results in bleed-
ing and polyp vascularization using linguistic queries alone.                 (d) Input                   (e) Groundtruth
Figure 6 shows the corresponding ranking mechanism based
on linguistic impressions for the bleeding the vasculariza-
tion cases. As can be seen, histograms are utilized to iden-
tify the top three ranked nearest images matching the Dark
red interpretation and the retrieval results are accurate as per
gastroenterologist ground truth markings. All the retrieved
bleeding regions are from the jejunum area of the gastroin-
testinal tract and we only show top three results according to
                                                                                      (f) Polyp vasularization
color histogram matching, see Figure 6(a). A similar analy-
sis with RGB space and polyp vascularization density is un-
dertaken and the the query image (Figure 5(d)) is described        Figure 5: Two example results in bleeding and polyp vas-
as Pale Red Dense and the ranking given in Figure 6(b) ranks       cularization. (a): For a given input image of bleeding spot
the resultant images according to the density of vasculariza-      which is described as (b) Dark Red the system retrieves sim-
tion. All the retrieved vascular regions contain dense ves-        ilar images containing bleeding regions with (c) decreasing
sels and are malignant. We utilized 400 Pillcam R Colon            ranks (matching). (d): For a given input polyp image which
capsule images for bleeding detection, these were obtained         is described as (e) Dense tortuous the system retrieves sim-
from 5 different patients, and marked by two gastroenterol-        ilar polyp images with (f) decreasing density. Note we only
ogists who provided ground truth regions along with bound-         the top three retrieval results and the ground truth regions
aries separating bleeding from normal mucosa tissue. For           are marked by human (operators) and used in training the
polyp vascularization based retrieval we used 100 images           impressions and their mappings.
which are benchmarked against an automatic segmentation
method (Prasath, Pelapur, and Palaniappan 2014) for calcu-
lating the density of vascularization in polyps.
Table 1: Taxonomy of color impressions in the RGB case for
bleeding regions in VCE images.
Level Impressions                Comment
I        Red, Black              Atomic
                                 context independent colors
II       Dark, Medium, Pale      Atomic
                                 context independent,
                                 dependent colors
III      Combinations of I & II Composite
                                 context independent,
                                 dependent colors


                       Conclusion
In this paper, we considered fuzzification of different color
spaces for medical decision support systems in gastroin-                                  (a) Bleeding
testinal diagnosis using video capsule endoscopy. Follow-
ing (Shamoi, Kawanaka, and Inoue 2014) we utilized fuzzy
sets and logic, color space theory for VCE imagery inter-
pretation and used it for retrieval tasks. Our preliminary re-
sults in bleeding region detection and polyp vascularization
in various VCE images indicate promise for using fuzzifica-
tion techniques in a medical decision support system. Future
works include introducing shape (e.g. polyp appearance),
                                                                                    (b) Polyp vascularization
texture (e.g. pit patterns) features along with fuzzification
framework studied here for different VCE videos. Moreover,
increasing the number of experts (in our case study gastroen-     Figure 6: Ranking based on linguistic terms and (a) color
terologists) and quantifying/enlarging the linguistic impres-     histograms in the bleeding region identification (b) Density
sions is an important avenue to be explored. We also believe      of vascularization in the polyps in decreasing order.
the framework considered here will help in identifying trash,
bubbles for uninformative frame classification.
                                                                  Prasath, V. B. S., and Delhibabu, R. 2015a. Automatic con-
                   Acknowledgment                                 trast enhancement for wireless capsule endoscopy videos
The author thanks the Gastroenterologists Dr. R. Shankar,         with spectral optimal contrast-tone mapping. In Compu-
Dr. A. Sebastian from Vellore Christian Medical College           tational Intelligence in Data Mining - Volume 1, 243–250.
Hospital, India for their help in interpreting VCE imagery        Springer SIST (eds.: L. Jain, H. S. Behera, J. K. Mandal, D.
and Radhakrishnan Delhibabu (Kazan Federal University             P. Mohapatra).
& Innopolis, Kazan, Russia) in helping with data collec-          Prasath, V. B. S., and Delhibabu, R. 2015b. Automatic im-
tion/organization. This work was done while the author was        age segmentation for video capsule endoscopy. In Compu-
visiting the Center for Scientific Computation and Mathe-         tational Intelligence in Medical Informatics, SpringerBriefs
matical Modeling (CSCAMM) at the University of Mary-              in Applied Sciences and Technology, 73–80. Springer CIMI
land, MD, USA.                                                    (eds.: N. B. Muppalaneni, V. K. Gunjan).
                        References                                Prasath, V. B. S.; Figueiredo, I. N.; Figueiredo, P. N.; and
                                                                  Palaniappan, K. 2012. Mucosal region detection and 3D
Figueiredo, I. N.; Prasath, S.; Tsai, Y.-H. R.; and Figueiredo,   reconstruction in wireless capsule endoscopy videos using
P. N. 2010. Automatic detection and segmentation of               active contours. In 34th IEEE/EMBS International Confer-
colonic polyps in wireless capsule imagess. Technical Re-         ence, 4014–4017.
port 10–37, University of Coimbra, Portugal.
Figueiredo, P. N.; Figueiredo, I. N.; Surya Prasath; and Tsai,    Prasath, V. B. S.; Figueiredo, I. N.; and Figueiredo, P. N.
R. 2011. Automatic polyp detection in pillcam colon 2 cap-        2011. Colonic mucosa detection in wireless capsule endo-
sule images and videos: Preliminary feasibility report. Di-       scopic images and videos. In Congress on Numerical Meth-
agnostic and Therapeutic Endoscopy 2011:7pp. Article ID           ods in Engineering (CMNE 2011).
182435.                                                           Prasath, V. B. S.; Pelapur, R.; and Palaniappan, K. 2014.
Kerre, E., and Nachtegael(eds), M. 2000. Fuzzy techniques         Multi-scale directional vesselness stamping based segmen-
in image processing. Heidelberg: Physica-Verlag.                  tation for polyps from wireless capsule endoscopy. In
Niwa, H.; Tajiri, H.; Nakajima, M.; and Yasuda, K. 2008.          Figshare.
New Challenges in Gastrointestinal Endoscopy. Springer.           Rey, J.-F. 2008. Future perspectives for esophageal and
colorectal capsule endoscopy: Dreams or reality? In New
Challenges in Gastrointestinal Endoscopy. Springer. 55–64.
Shamoi, P.; Inoue, A.; and Kawanaka, H. 2014a. Fuzzy color
space for apparel coordination. Open Journal of Information
Systems 1(2):20–28.
Shamoi, P.; Inoue, A.; and Kawanaka, H. 2014b. Percep-
tual color space: Motivations, methodology, applications. In
Joint 7th International Conference on Soft Computing and
Intelligent Systems (SCIS) and 15th International Sympo-
sium on Advanced Intelligent Systems (ISIS), 1354–1359.
Shamoi, P.; Kawanaka, H.; and Inoue, A. 2014. Fuzzifica-
tion of HSI color space and its use in apparel coordination.
In Proc. of 25th Modern Artificial Intelligence and Cognitive
Science Conference, 11–17. Spokane, WA, USA: CEUR-
WS.org. online CEUR-WS.org/Vol-1144/paper2.pdf.
Vlachos, I. K., and Sergiadis, G. D. 2006. A heuristic ap-
proach to intuitionistic fuzzification of color images. World
Scientific. chapter 108, 767–774.
Wyszecki, G., and Stiles, W. 1982. Color Science: Concepts
and Methods, Quantitative Data and Formulas. New York,
NY, USA: Wiley.