Fuzzification of HSI Color Space and its Use in Apparel Coordination Pakizar Shamoi Hiroharu Kawanaka Atsushi Inoue Department of Computer Science, Graduate School of Engineering, Department of Computer Science, Kazakh-British Technical University Mie University Eastern Washington University pakita.shamoi@gmail.com kawanaka@elec.mie-u.ac.jp inoueatsushij@gmail.com Abstract Human perception of colors constitutes an important part in color theory. The applications of color science are truly Motivations omnipresent, and what impression colors make on human plays vital role in them. In this paper we offer the novel It is a well-known fact that most search engines nowadays approach for color information representation and are based on indexing. For example, if we try to google processing using fuzzy sets and logic theory, which is „elegant dresses‟, we get the result set for what we were extremely useful in modeling human impressions. searching. If we open the provided links, it can be easily Specifically, we provide the fuzzification of HSI color space seen that the corresponding web pages contain the word and further use it for obtaining the correspondence between colors and human impressions. In addition, the methodology „elegant‟ in the header. But it is obvious that there are is applied in the implementation of a framework for the some really elegant dresses which are not indexed. apparel coordination based on a color scheme. It deserves Unfortunately, users will never see them in search results. attention, since there is always some uncertainty inherent in So, the disadvantage of giving the keywords is obvious. the description of apparels. Now let‟s provide another suitable example that can prove the validity of the proposed approach. Nowadays, Introduction “Taobao” website for online shopping is becoming extremely popular in Kazakhstan. After a talk with Taobao Nowadays, the meaning of color is becoming more and consultant, we identified two problems that most users more important almost in every industry (T. Lu and experience. The first problem is that very often users wish C.Chang, 2007). It is worth noting that the number of to find similar items at lower prices. The other one is that unique colors can reach up to 16 million. Needless to say, they want to be able to find some item given some picture most of these colors are perceptually close and cannot be from internet, without knowing the brand, for instance. differentiated by human eye which can differentiate only There is a plenty of sites, which provide the service of between 30 colors in cognitive space. finding goods on taobao based on the image that the user More and more related research efforts are oriented uploads. However, the functionality of these sites is towards application of fuzzy set theory to various tasks in limited. Specifically, they can only find goods with the image processing (L. Hildebrand and B. Reusch, 2000). absolutely same photo. That is why even if you download This can be explained by the fact that digital images are the photo from Taobao, the service won‟t find the mappings of natural scenes and, thus, they carry a corresponding item in case we change the image a bit – substantial amount of uncertainty, due to the imprecise add shading, brightness, cut some edges, flip it, etc. This nature of pixel values. Moreover, the human perception of happens because such services can find goods based on colors is in itself not precise. In most cases, human exactly the same images, not similar images. perception of colors contrasts with color theory science because these theories have always assumed perfect conditions. Overview of Color Representation Methods The paper presented herein tackles this problem by Color space is a method of color representation. There are presenting the new methodology for color information a number of color spaces popular today, but none of them processing using fuzzy set theory. can dominate the others for all kinds of images. Based on some color space we can develop Fuzzy color space, in order to systematically organize the set of all possible human color perceptions. Let‟s consider some of the most popular color spaces. Well-known RGB system represents additive color combinations (e.g. overlapping lights, display on LCD). It is convenient for color image display, but not for analysis, due to high correlation (N. Sugano, et al., 2009). So, if intensity changes, all r, g and b values change accordingly. As a result, chromatic information can be lost. Moreover, it is not a uniform scale, so it is difficult to calculate the Figure 1. Scheme of the methodology similarity between colors based on their distance in RGB space (K. Konstantinidis, et al., 2005; Y. Li-jie, et al., Taxonomy 2009). Another popular system is CMYK, which is based on As we know, in computer systems colors are represented subtractive color combinations (e.g. mixing dyes, inks, by various color spaces (RGB, CMYK, HSI). We chose pigments). Pigments display colors by the way of HSI for a number of reasons mentioned earlier. As for absorbing some wavelengths of lights and reflecting the impressions, they are expressed by linguistic terms (e.g. remaining ones. formal, black and white, pale blue, etc.). The Table 1 The HSI model is also a popular color model at present, below depicts the taxonomy, i.e. classification of color and it has good performance. In HSI model, colors are impressions. expressed using 3 attributes, namely, hue (e.g. red, orange, Table 1. Taxonomy of color impressions green), intensity (light vs dark) and saturation (intense vs dull). Level Impression Comment In most cases, the RGB model is often used to depict the color information of an image (Y. Li-jie, et al., 2009). However, recent researches in the field of image III Various combinations Composite, context- processing mostly make use of HSI space. The reason is of I and II dependent and context- that in HSI the specific color can be recognized regardless (e.g. Pale blue, independent colors of variations in saturation and intensity, since hue is elegant and formal, invariant to certain types of highlights, shading, and deep red etc.) shadows. So, it will be much easier to identify the colors II Elegant, formal, Pale, Atomic, that are perceptually close and combine them to form casual bright, context- homogeneous regions representing the objects in the deep dependent image. As a result, the image could become more and context- meaningful and easier for analysis. independent We adopt the HSI color space, because of its similarity colors with the way a human observes colors and the fact that the I Red, blue, black Atomic, context- intensity is separated from chrominance, so the chromatic independent colors information of the original image will be preserved. In fact, same color can create different impressions in Methodology different settings (apparel, interior coordination, medicine, etc.). So, we can claim that impressions are context- The main idea of the proposed methodology is to provide specific. Therefore, we need to emphasize that the the mapping of different colors, color combinations and proposed methodology aims to provide the correspondence human impressions of them (Fig.1). For achieving this, we between colors and certain impressions - atomic(red) and plan to use various tools, including color theory and color composite (pale red, formal and elegant) - expressed by harmony principles, fuzzy sets and logic, Mass Assignment linguistic terms in some context. In simpler words, the Theory, surveys, and histograms among others. methodology provides Context-based Image Retrieval (CBIR) based on color scheme. The context dependency can be easily handled by fuzzy logic. In case of composite color impressions, which are based on atomic ones with the help of various connectives, we need to employ basic formulas from fuzzy theory. Specifically, for the intersection (and) and union (or) we take the minimum and maximum of two memberships Finally, Intensity fuzzy variable is described by 5 respectively, to get the resultant membership value (Zadeh, linguistic terms, namely {"Dark", "Deep", "Medium", 1996; Zadeh, 2002): "Pale", "Light"}. Intensity values lie in the range X= [0;255], with the respective U = {0, 1, …, 254, 255}. For A  B( x)  min[ A( x), B( x)] the sake of simplicity, for all the fuzzy variables we employed either triangular or trapezoidal membership A( x)  B( x)  max[ A( x), B( x)] functions (Fig. 2), depending on the value range of a certain color property associated with the specific label (A. In addition, we use the following formula for the α-cut Younes, et al., 2005).. Particularly, for a wide range we (Alpha cut), which is a crisp set that includes all the used trapezoidal membership functions, and triangular members of the given fuzzy subset f whose values are not ones for all the other fuzzy sets that are not wide. less than α for 0< α ≤ 1 (Zadeh, 1965): f   {x :  f ( x )   } Alpha cuts and set operations are connected in the following way: ( A  B)  A  B , ( A  B)  A  B These formulas enable us to find the result of a query with a certain threshold (which is actually an Alpha cut) – α, containing or or and operations. We first find the α-cuts and then take the crisp or / and operation. If we analyze Table 1, it can be easily seen that the higher the abstraction level is, the fuzzier is the correspondence between linguistic labels(impressions) and colors. This has primary importance when the methodology is customized for a certain context. Color Space Fuzzication Based on perceptions, the colors can be modeled as fuzzy sets in HSI color space. As already mentioned, colors will be described by linguistic terms. Let's consider how the fuzzy encoding can be done on each of the parameters of HSI color space. Hue variable is specified by 8 linguistic labels, specifying various hues. Such division was done based on the subjective perception. In the future this can be done by experts in specific domain. Anyway, this division was done just for the purpose of demonstrating the fuzzy encoding process, after a more thorough analysis fuzzy sets can be tuned. The term set consists of 7 fuzzy sets - {"Red", "Orange", "Yellow", "Green", "Cyan", "Blue", "Violet", "Magenta"}. Hue values are cyclic and vary from 0 to 360. So, we define the Hue for the domain X = [0, 360], and the universal set is U = {0 ,1, 2, …..,359, 360}. Concerning the Saturation variable, it is represented by 3 fuzzy sets in our approach - {"Low", "Medium", "High"}. Saturation values vary from 0 to 1, from dull to intense, so the domain X = [0, 1], and the universal set is U = {0, 0.01,…,0.99,1}. Figure 2. Fuzzy sets for Hue, Saturation and Intensity Dominant Color Identification such atomic impressions and the corresponding colors One of the most important subtasks in our methodology is fitting to that impressions. This correspondence was identification of a dominant color(s) in the image. For that obtained by deep analysis of existing online shops with purpose, we employ a color histogram. As we know, color tagged images, fashion blogs, and some other related histograms reflect tonal distribution in digital images. They resources. can also be used to extract other various features of an The list can be further extended with impressions like image for similarity measure, classification, etc. (N. provocative, informal, etc. It is assumed that the proposed Sharma, et al., 2011; J. Han and K. Ma, 2002). system will provide the image retrieval functionality based For easy histogram purposes, we divide the colors into on the following queries: bins (J. Han and K. Ma, 2002), each of which contains 3 • Linguistic query, in which attributes are specified by fuzzy sets specifying certain fuzzy values of hue, words, for example, Retrieve popular elegant dresses. saturation and intensity. Since we have 8 sets for the hue, 3 • Exemplar query, which allows image input to a for the saturation and 5 for the intensity, we obtain 120 system. This is for the case when user wishes to retrieve color combinations (e.g. hue is red, saturation is medium, apparels that fit to some other apparel (inputted image), intensity is deep). But we can reduce it to 86 combinations by taking into account color harmony principles, among taking into account the following general observations: other, trivial ones. The relevance is ranked according to a • if (Saturation is low) then (Hue is irrelevant) color harmony measure computed from the dominant • if (Intensity is light) then (Hue and Saturation are color(s) in the images. irrelevant) • Combinational query, which has properties of both linguistic and exemplar queries. For instance, user aims to As we know, the Saturation measures the degree of find apparels of dark hues that are perfectly combined with mixing the hue with uniform white color. Therefore, low some other apparel in the uploaded image. saturation means that the color is a shade of gray. Furthermore, for each of the obtained combinations, we Note that in case of exemplar or combinational query, calculate the number of pixels. This serves as a primary the given RGB image is first converted into HSI color data for building the linguistic color histogram and space. Furthermore, based on the histogram, we identify identification of a dominant color(s). An illustration for the dominant color in the image and try to find apparels that is provided in Fig. 3. that fit to it. The harmony between a query image and database image is computed from the dominant color(s), using the table of color harmonies selections. For now, our Application prototype application provides the processing of just linguistic queries. In order to enliven our methodology we developed apparel Now let‟s look at some use cases of the system. For the coordination application, which is highly useful in initial testing, we used database containing 65 apparels, understanding the importance and practical application of the approach. Knowing the colors that are perceived as mostly dresses. Note that the default retrieval threshold is formal, for example, we can recommend formal apparels set to 0.5, but it can be tuned in the system. based on a dominant color in the image. It is even possible to recommend a whole “look”. So, the main idea of the application is to retrieve best matching images with relevant apparels corresponding to the complex query posed by user based on color scheme. Context-dependent Color Impressions Visual effects of specific color and color combinations cannot be underestimated. So, besides the atomic and composite context-independent impressions (e.g. light green, white and blue) the system provides the retrieval of images corresponding to context-dependent color impressions, that represent qualitative linguistic labels, e.g. informal, elegant, etc. Now the represented set is limited, but it can be easily extended in the future. Table 2 below presents some of Figure 3. Identification of dominant colors using color histogram Table 2. Map between some color impressions and colors Impression Colors (visual) Colors (in words) Elegant Black, jewel, emerald, sapphire as well as silver, bronze, and copper. Formal, Modest Dark and deep colors Casual Sweet and bright fascinating colors. Romantic Light to mid-tones of pink, purple, gray and blue colors. Mostly pastel colors Vintage Modest colors like gray variations, ill-saturated vinous color Fresh Bluish green, pure green, yellowish green, turquoise (of various intensities) Passion The dark and deep mid-tones of vinous and purple that generate a passionate feeling Example 1. Deep red or pale green dress. Results: Number of apparel types – 2 Number of dominant colors (apparel 1) – 1 Number of dominant colors (apparel 2) – 1 Color (apparel 1) – Color [Hue = red; Saturation = medium or high; Intensity=deep] Color (apparel 2) – Color [Hue = green; Saturation = medium or high; Intensity=pale] Example 2. Elegant dress Based on our knowledge (see Table 2), beige, milky, mocha, black, brown, dull blue, dull green can be considered as “elegant”. So, for example, for „milky‟ we Conclusion have: In the present article, we propose a fuzzy sets and logic Color1 (Milky) – Color [Hue = yellow; Saturation = guided novel technique for color processing. The proposed medium or high; Intensity=light] approach produces results which are highly relevant to the Results: content of the linguistic query corresponding to a human impression. In the prototype application we tried to provide the correspondence between linguistic labels and user‟s impression of a certain color or color combinations in a specific context – fashion. But impressions can greatly vary from context to context. Therefore, conducting Example 3. White and blue dress comparative evaluation of color perceptions considering different environments (even countries!) of use is very Number of dom. Colors - 2 important (O. Penatti, et al., 2012). For example, red can Color1 – Color [Hue = any; Saturation = any; Intensity = symbolize something exciting, sensual, romantic, feminine, light] good luck, signal of danger, etc. In apparel coordination, it Color2 – Color [Hue = blue; Saturation = medium or high; can mean something provocative. The problem of strong Intensity = medium] context dependency can be easily handled by fuzzy sets Results: and logic, e.g. by way of collecting the experts‟ opinions and building the corresponding fuzzy sets. We plan to address these issues in our future works. As we know, image processing rests upon analysis of color pixels and shapes. However, we did not focus on shape descriptors in our study, since almost all of them The main advantage of such systems is that they enable depend on segmentation, which is still hard and extremely retrieval of images based on their content and context, not application-dependent task. tags. As a result, there is no need in giving the tags or Finally, it needs to be highlighted that the proposed keywords specifying the image, with the exception of methodology is suitable for a number of domains. Namely, trivial ones (i.e. type of an apparel - dress, shirt, etc.). This it is also possible to use it in real-time medical decision will free administrators from spending time on describing support, interior design coordination, among others. the apparel. References Hildebrand, L., Reusch, B. Fuzzy color processing. In: Book Penatti, O., Valle, E., Torres, R. Comparative study of global Kerre E, Nachtegael M (eds) Fuzzy techniques in image color and texture descriptors for web image retrieval. J. Vis. processing, studies in fuziness and soft computing, vol 52. Commun. Image R. 23 (2012). Physica-Verlag,Heidelberg, 2000. Sharma, R., Rawat, P., Singh, J. Efficient CBIR using color Konstantinidis, K., Gasteratos, A., Andreadis, I. Image retrieval histogram processing. Signal Image Process. Int. J. (SIPIJ) 2 (1) based on fuzzy color histogram processing. Optics (2011). Communications 248 (2005). Sugano, N., Komatsuzaki, S., Ono, H., Chiba, Y. Fuzzy Set Li-jie, Y., De-sheng, L., Guan-ling, Z. Automatic Image Theoretical Analysis of Human Membership Values on the Color Segmentation Based on Human Color Perceptions. I.J. Image, . Triangle, Journal of Computers, VOL. 4, NO. 7, 2009. Graphics and Signal Processing, 1, 2009. Younes, A., Truck, A., Akdag, H. Color Image Proling Using Lu, T., Chang, C. Color image retrieval technique based on color Fuzzy Sets. Turk J Elec Engin, VOL.13, NO.3 2005. features and image bitmap. Inf. Process. Manag. 43 (2) (2007). Zadeh, L. Fuzzy sets. Information and Control, 8:338–353, 1965. Zadeh, L. Fuzzy Logic = Computing with Words. Life Fellow, IEEE Transactions on Fuzzy Systems, Vol.4,1996. Zadeh, L. From Computing with Numbers to Computing with Words – from Manipulation of Measurements to Manipulation of Perceptions. Int. J. Appl. Math. Comput. Sci., Vol. 12, No. 3, 307-324, 2002.