The study of colors has attracted Artificial Intelligence researchers for many years. Nevertheless, there are aspects of this issue that are still little analyzed. One of them is the investigation of fabric colors, which has some peculiarities, such as the necessity to handle textures, that are not found in other scenarios. In this paper, we present DLE4FC, a deep learning ensemble for identifying fabric colors. Specifically, we introduce the general basic model, which consists of a particular Convolutional Neural Network, define three versions of it and integrate them into an ensemble to get better results. Finally, we test our ensemble and compare it with other already known systems.
The study of colors has attracted, and is attracting, researchers from several areas [
Various methods have been proposed in the literature to recognize color components, patterns
and shades, as well as the layout of color yarns from images. To achieve their goals, these
methods use very diferent approaches, such as fuzzy C-Means [
In the past, deep learning has been used for color identification in various contexts, including vehicles and trafic lights [ 13, 14], flowers [ 15], stool medical images [16], facial images [17] etc. In all these cases they have shown better behavior than other machine learning approaches [13]. Nevertheless, these methods have been little used for color recognition in the context of textiles.
In this paper, we want to make a contribution in this setting and propose DLE4FC, a new deep-learning based approach for identifying colors in textiles. The architecture of DLE4FC is based on an ensemble of Convolutional Neural Networks (CNNs, for short) whose input belongs to the color diference domain. The latter is obtained by considering the diference between the input image and a set of reference color images. Using it allows for better capture of color variations, shades and patterns, and ultimately allows DLE4FC to learn features for color recognition in fabric images. In addition, adopting an ensemble strategy ensures greater robustness, and ultimately greater accuracy in the results obtained. DLE4FC focuses on a particular aspect of color recognition in the textile industry, namely color identification of fabric images. This aspect is important in this application context as it reduces ineficiency, wasted time and subjectivity. This ensures higher quality products capable of better meeting customer desires. In the paper, we also describe some of the experiments we conducted to evaluate the accuracy of DLE4FC compared to a variety of other color identification systems proposed in the past literature. The results obtained show that the peculiarities of DLE4FC enable it to achieve better results than pattern recognition and color identification frameworks proposed in the past.
The outline of this paper is as follows: Section 2 provides a technical presentation of DLE4FC. Section 3 presents the tests performed to evaluate its accuracy while also comparing the latter with the one of other systems proposed in the past. Finally, Section 4 draws conclusions and outlines some possible future developments.
In this section, we present the technical details of DLE4FC. In particular, in Subsection 2.1, we introduce the color diference domain underlying DLE4FC. In Subsection 2.2, we describe in detail the architecture and behavior of our framework.
A color space is a specific color organization based on a color model. Currently, the most widely
used color model is RGB (Red, Green, Blue). Although it is suficient for representing digital
images, it does not allow the extraction of handcrafted features, which play an important role
in the color classification task [
To the best of our knowledge, deep learning approaches for color identification proposed
in the past do not consider textural patterns (which are frequent in fabric images). In fact, the
latter can hamper the learning of the underlying neural network. This limitation stems at least
in part from the color representation used for the images, which does not provide suficient
information for color classification [
To construct our diference space, we used 12 reference colors, namely Orange, White, Blue, Cyan, Yellow, Magenta, Black, Red, Earth Brown, Green, Emerald Green, and Purple. The selection of these colors was not straightforward since there would be thousands of potential colors. To make this selection we consulted some managers from fabric companies, who provided us with two important insights. The first is that colors used in fabrics have a slightly diferent RGB representation from the pure color definition. For example, the pure black color corresponds to the hexadecimal code #000000. In contrast, in the context of textiles, there are several shades of black, the most widely used of which is that corresponding to the hexadecimal RGB code #1C1C1C. The second insight concerns the number of colors to be adopted. The experts interviewed told us that 12 colors are capable of handling most of the possible cases and represent a very good tradeof between the ability to represent reality, the complexity of the framework and the accuracy of its results. However, we would like to emphasize that our model can be easily scaled to include more colors, should that be necessary in this or other application contexts.
As a consequence of our way of proceeding, for each image we create 12 images, each representing its distance from one of the reference colors, expressed in the RGB model. To give an idea of this, in Figure 1 we show an example of the diferences between an input image and the 12 reference colors. Finally, we concatenate the resulting 12 images to give them as input to DLE4FC.
In the previous section, we have seen that, given a fabric image, when we switch to the color diference domain, we get 12 images, each represented in the RGB model. Since this model has 3 channels, it follows that each fabric image can be represented by 36 channels. To process such data we construct an extension of the traditional CNN which we call CNNΔ. It has an input layer that receives images of any width and height through 36 channels. After this layer, there is a convolutional layer with kernels of size 3 × 3 and stride 1, followed by a max pooling layer of size 2 × 2 and stride 1. After that there is a convolutional layer with the same kernel size as the previous one and, then, another pooling layer of size 2 × 2 and stride 1. The latter is followed by a dense layer with 128 units connected to a final dense layer with a softmax activation function. The output of CNNΔ is a vector of 12 elements each representing the classification probability of the corresponding reference color. In total, CNNΔ has 175,476 parameters.
Actually, as indicated by its name, DLE4FC consists of an ensemble of three CNNΔ models. This choice allows our system to improve its performance and generalization ability [20]. To tune the values of the hyperparameters of the three CNNΔ models composing DLE4FC, we performed a random search procedure. We chose this procedure because, compared with other existing hyperparameter optimization techniques, it can be easily combined with early stopping methods, which we adopted during the training phase for reducing the overfitting probability. The combination of these two techniques allows us to narrow the search space more eficiently.
For space limitations, we cannot describe in detail the whole implementation of this technique. The interested reader can refer to [21]. Here, we only say that, at the end of this procedure, the values of the hyperparameters that provided the best performance in terms of F1-score, Accuracy and Average epoch training time of the ensemble are those shown in Table 1.
Network CNNΔ 1 ADAM CNNΔ 2 ADAM CNNΔ 3 SGD
Having discussed the structure of DLE4FC, we can now describe its behavior. It is shown in Figure 2. As can be seen from this figure, first the input image is transformed into 12 corresponding images in the color diference domain. Then, these images are given as input to the three CNNΔ models, each of which returns a classification probability vector for the reference colors. In this way, three probability vectors are obtained. These are processed by a soft-voting function that first averages the classification probabilities, then identifies the highest values thus obtained and finally returns the corresponding class as output.
We tested DLE4FC on Fabric Dataset [22]. This contains 2000 color images concerning clothing and textiles. Each image was acquired using a photometric stereo sensor under 3 or 4 diferent illumination conditions. Therefore, the total number of actual images is 7,757. The images are in PNG format and their size is 400 × 400. They are not labeled with colors so we had to manually specify the ground truth for each of them. To achieve this goal, we showed each image to 11 diferent people and asked each of them to label it with one of the 12 reference colors specified in Section 2.1. Where diferent people assigned diferent colors to the same image, we assigned it the color specified by the majority of people. Finally, we discarded all images to which no color could be associated. At the end of this procedure we retained 4,591 images and discarded 3,166 ones. Afterwards, we performed an undersampling task on the most numerous classes, and an oversampling one on the least numerous classes, in order to balance them. At the end of these activities, we obtained a new dataset of 1,200 images, with 100 images per class. We call this new dataset.
We compared DLE4FC with several deep learning architectures proposed in the literature, i.e. ResNet [23], InceptionResNet [24], DenseNet [25], as well as Color Deep network [26], which is a reference network in the color recognition literature. Specifically, we selected two types of ResNet, namely ResNet50v2 and ResNet152v2, two types of DenseNet, namely DenseNet169v2 and DenseNet201v2, as well as InceptionResNetv2. We chose these networks because they have been shown to achieve the best experimental results. In addition, to test whether it was worthwhile to adopt an ensemble, we compared DLE4FC with a single CNNΔ. To all these networks we gave as input the 36 channels representing a single RGB image in the color diference domain. Since Color Deep could not receive 36 channels as input simultaneously, we gave it the 12 images corresponding to the diferences in cascade, and then aggregated the corresponding results. To avoid overfitting in color classification, we applied a data augmentation procedure to the images of . This procedure involves two steps, namely: (i) a random flip augmentation, which flips the image horizontally and vertically, and (ii) a random zoom augmentation, which randomly increases the image size by adding new pixels around or interpolating the values of pixels.
At this point, for each original fabric image, we identified the dominant color, i.e., the most prevalent color in the image area. For this purpose, we used the Modified Median Cut Quantization (MMCQ) algorithm [27]. In this way, we clustered the color space of the image and selected as the dominant color the one corresponding to the center of the largest cluster thus identified. Following this, we generated a new image containing only that color. We call − the corresponding dataset. At the end of all these activities, we had two datasets available for experiments, namely: (i) , which stores the original images, and (ii) − , which stores the same images but pre-processed to contain only the dominant color.
We provided the images of and − as input to the various models to be tested. For each image, we calculated the Euclidean distance between the corresponding color and each color in the palette of 12 colors that we had previously identified. We chose the color corresponding to the minimum distance as the image color.
To perform our tests, we divided each of the two datasets and − into two partitions comprising 80% and 20% of their images, respectively. We randomly chose the images to be included in each partition. Finally, we applied the 5-fold cross-validation technique to the first partition.
We trained each CNNΔ of DLE4FC from scratch and independently of the others. In the color recognition literature there are few trained learning models, and none of them were suitable for our color diference space. Therefore, we had to randomly initialize the weights of the three CNNΔ models. To limit overfitting, we used the early stopping technique. Specifically, we stopped the training task when there was no change in the (cross-)validation loss for three consecutive iterations. In Table 2, we report the parameter setting for the various learning models to be compared. The last three rows of this table refer to the three CNNΔ models composing DLE4FC. The hyperparameters of these models have been already shown in Table 1; we report them again in Table 2 for convenience.
Deep neural network
Optimizer Learning rate
Momentum
Table 3 (resp., 4) shows the values of Precision, Recall, F1-score and Accuracy obtained by the various learning models when they receive as input the images of (resp., − ). For Color Deep we considered both the average and the best results.
From the analysis of Table 3, we can observe that, as far as the dataset is concerned, the performance of DLE4FC is better than that of the other deep learning models for all metrics considered. Color Deep obtains very good performance values in its second configuration, although they are slightly lower than those of DLE4FC. Table 3 also reveals that the ensemble Color Deep SGD ResNet50v2, ResNet152v2 SGD DenseNet169v2, DenseNet201v2 SGD InceptionResNetV2 SGD CNNΔ ADAM CNNΔ 1 of DLE4FC CNNΔ 2 of DLE4FC CNNΔ 3 of DLE4FC
ADAM ADAM SGD DLE4FC CNNΔ ResNet50v2 ResNet152v2 InceptionResNetV2 DenseNet169v2 DenseNet201v2 Color Deep (avg.) Color Deep (max.) 0.80 0.78 0.74 0.67 0.73 0.72 0.69 0.73 0.78 DLE4FC CNNΔ ResNet50v2 ResNet152v2 InceptionResNetV2 DenseNet169v2 DenseNet201v2 Color Deep (avg.) Color Deep (max.) strategy achieves better results than using a single CNNΔ. In fact, the value of Precision (resp., Recall, F1-score, Accuracy) obtained by DLE4FC is 2.56% (resp., 5.26%, 3.89%, 5.26%) higher than the corresponding value obtained by a single CNNΔ.
Analyzing Table 4 and comparing it with Table 3, we can see that providing input images with only the dominant color to the models under consideration does not lead to better results. On the contrary, all models except DLE4FC show a marked decrease in performance values. The best results after those of DLE4FC are obtained by using only one CNNΔ. In this case, the Precision (resp., Recall, F1-score, Accuracy) is 11.39% (resp., 15.38%, 12.82%, 16.25%) worse than that obtained by DLE4FC. The other models show even lower performance results. Among them, again, the best is Color Deep in its second version. Even with the dataset − we can observe that the ensemble strategy gets better results than using only one CNNΔ. As previously pointed out, contrary to what might have been expected, using the images with only the dominant color returns worse results than using the original fabric images. This may be due to the flattening of some major color nuances, which increases the dificulty of recognizing the true color by the deep learning models under consideration.
In this paper, we have proposed DLE4FC, a new approach to identify colors in fabrics. This is a challenging issue since fabric images contain textural patterns that can make color classification dificult. DLE4FC is based on a new color space, called color diference space, which allows providing deep learning classifiers with much more information than the one that could be provided by the RGB color space. Because of this idea and the use of the ensemble strategy, DLE4FC manages to obtain equal or better performance than the main related approaches proposed in the past literature. DLE4FC is based on a particular CNN model, called CNNΔ; specifically, it is an ensemble of three CNN Δ models. In this paper, we also illustrated some experiments performed on the Fabric dataset through which we compared DLE4FC with other existing approaches.
In the future, we plan to enhance DLE4FC along several directions. First, we plan to increase the number of reference colors used during classification. Also, we plan to allow DLE4FC to assign multiple colors to a single fabric image. In addition, if we had access to a dataset much larger than Fabric, we could consider enriching DLE4FC with a vision transformer-based methodology. Indeed, the latter architecture is very promising but it needs the application of thousands of training images. Finally, the color diference space used by DLE4FC is currently built on the RGB color space. We could think of building this space on other color spaces, such as HSL (Hue, Saturation, Lightness), HSV (Hue, Saturation, Value) or CMYK (Cyan, Magenta, Yellow and Key Black). Indeed, we believe that they could provide additional information for DLE4FC to use in making the classification of fabric images. comparison of 2DCNN network architectures and boosting techniques for regression-based textile whiteness estimation, Simulation Modelling Practice and Theory 114 (2022) 102400. [11] L. Xie, Y. Shen, X. Chen, Study on Automatic Recognition of Fabric Color and Matching to Standard Color Chip by Computer Vision and Image Analysis Technology, Journal of Fiber Bioengineering and Informatics 9 (2016) 29–37. [12] Z. Zehra, M. Bashir, Color Fastness Grading System for Textile Industry Using CIEL*a*b Color Space, in: Proc. of the International Conference on Big Data and Smart City (ICBDSC’19), Yokohama, Japan, 2019, pp. 1–5. doi:10.1109/ICBDSC.2019.8645600. [13] Q. Zhang, L. Zhuo, J. Li, J. Zhang, H. Zhang, X. Li, Vehicle color recognition using Multiple-Layer Feature Representations of lightweight convolutional neural network, Signal Processing 147 (2018) 146–153. doi:https: //doi.org/10.1016/j.sigpro.2018.01.021. [14] H. Kim, J. Park, H. Jung, An Eficient Color Space for Deep-Learning Based Trafic Light Recognition, Journal of Advanced Transportation (2018) 2365414. [15] D. Apriyanti, L. Spreeuwers, P. Lucas, R. Veldhuis, Automated color detection in orchids using color labels and deep learning, PLoS ONE 16 (2021) e0259036. doi:10.1371/journal.pone.0259036. [16] Z. Yang, L. Leng, B. Kim, StoolNet for Color Classification of Stool Medical Images, Electronics 8 (2019) 1464.
doi:10.3390/electronics8121464. [17] A. Rame, A. Douillard, C. Ollion, CORE: Color Regression for Multiple Colors Fashion Garments, ArXiv preprint abs/2010.02849 (2020). [18] J. Zhang, R. Pan, E. Gao, D. Zhu, Automatic detection of layout of color yarns of yarn-dyed fabric. Part 1: Singlesystem-mélange color fabrics, Color Research & Application 40 (2015) 626–636. doi:10.1002/col.21927. [19] P. Chen, X. Bai, W. Liu, Vehicle color recognition on urban road by feature context, IEEE Transactions on
Intelligent Transportation Systems 15 (2014) 2340–2346. IEEE. [20] O. Sagi, L. Rokach, Ensemble learning: A survey, WIREs Data Mining and Knowledge Discovery 8 (2018) e1249. [21] A. Amelio, G. Bonifazi, E. Corradini, S. Di Saverio, M. Marchetti, D. Ursino, L. Virgili, Defining a deep neural network ensemble for identifying fabric colors, Applied Soft Computing 130 (2022) 109687. Elsevier. [22] C. Kampouris, S. Zafeiriou, A. Ghosh, S. Malassiotis, Fine-Grained Material Classification Using Micro-geometry and Reflectance, in: Proc. of the European Conference on Computer Vision (ECCV’16), Springer, Amsterdam, The Netherlands, 2016, pp. 778–792. [23] K. He, X. Zhang, S. Ren, J. Sun, Deep Residual Learning for Image Recognition, in: Proc. of the International Conference on Computer Vision and Pattern Recognition (CVPR’16), Las Vegas, NV, USA, 2016, pp. 770–778. doi:10.1109/CVPR.2016.90. [24] C. Szegedy, L. Wei, J. Yangqing, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, A. Rabinovich, Going deeper with convolutions, in: Proc. of the International Conference on Computer Vision and Pattern Recognition (CVPR’15), Boston, MA, USA, 2015, pp. 1–9. doi:10.1109/CVPR.2015.7298594. [25] G. Huang, Z. Liu, L. V. D. Maaten, K. Weinberger, Densely Connected Convolutional Networks, in: Proc. of the International Conference on Computer Vision and Pattern Recognition (CVPR’17), IEEE Computer Society, Honolulu, HI, USA, 2017, pp. 2261–2269. [26] M. Zhang, P. Wang, X. Zhang, Vehicle Color Recognition Using Deep Convolutional Neural Networks, in: Proc. of the International Conference on Artificial Intelligence and Computer Science (AICS’19), Wuhan, China, 2019, pp. 236––238. doi:10.1145/3349341.3349408, association for Computing Machinery. [27] D. Bloomberg, Color quantization using modified median cut, in: Leptonica, 2008, pp. 1–10.