Rapid non-contact estimation of fruit quality parameters is an essential factor for an efficient food processing pipeline. We propose a novel workflow for the contactless classification of strawberries based on their sugar content, using Hyperspectral Imaging (HSI) and One-Dimensional Convolutional Neural Network (1D - CNN). Sugar content is an important quality aspect of strawberries, hence classification based on sugar content gives more yield to the fruit producers. We used Visible and Near Infrared (VNIR) hyperspectral camera to acquire HSI data of 50 ripe strawberries and applied the proposed method to classify them. To verify the advantage of the proposed method, the results from 1DCNN are compared against other standard classification methods such as Spectral Angle Mapper (SAM), and Spectral Information Divergence (SID). The results show that the 1D-CNN outperformed other methods by achieving 96.6% classification accuracy.
Non-invasive measurement of different food attributes is of great interest. The sugar content is one of the characteristics that enhances the customer experience and influence the market value of the fruit [1]. Sugar content is one of the key factors that is useful for grading the fruits [2], measurement of this attribute with the application of imaging technologies eases the sorting process and offer several advantages like contactless and non-destructive measurement, the possibility for automation, and high accuracy within a certain limit. Hyperspectral imaging (HSI) is one of the most suitable imaging modalities that is proven to be useful for this purpose [3]. There has been many extensive studies reported on the usefulness of HSI for non-invasive measurement of several types of fruits such as apples, oranges, and kiwifruits [4][5][6]. Hyperspectral imaging systems capture both spatial and spectral information simultaneously, which enables HSI technology to make a more reliable classification compared to the traditional three channel imaging methods. In addition to this, HSI already proven to be one of the effective non-invasive methods to study material properties compared to the chemical analysis, which are generally invasive in nature. HSI finds applications in many different fields like remote sensing [7], cultural heritage [8][9], forensics [10], etc. Food science is an important field where hyperspectral imaging used extensively to monitor the quality, diseases, and stages of development in variety of food products. Strawberry is one of the favorite fruits for many people and the interest is reflected in the increased production in every year [11]. Motivated by this fact that lack of HSI and predictive sugar analysis for strawberries, we have studied and evaluated the strawberry’s classification based on sugar content using HSI. Customer interest in strawberry is mainly driven by the taste of the fruit and most of the times the sweetness of the fruit. Convolutional Neural Networks (CNNs) has been used in many computer vision and image classification applications, CNNs were designed to learn features from the training data and which can be used to classify the test samples. The most common CNNs are two-dimensional (2D-CNN), however in this work we proposed a one-dimensional CNN (1D-CNN) because the spectral signals are One-Dimensional (1D) signals with varying amplitude. 1D-CNNs can provide reliable solutions in many 1D signal-processing applications such as Electrocardiographic (ECG) signals [12], audio[13], and other 1D signals [14][15]. We can also find a few attempts to use 1D-CNN for spectral classification [16], this experiment used 1D-CNN to classify ink spectra. To verify the effectiveness of the proposed method, we used two well-known classification methods in HSI domain; they are Spectral Angle Mapper (SAM) and Spectral Information Divergence (SID).
Later part of this paper is organized as follows. Section 2 will present the details of the fruit samples used, hyperspectral acquisition of the fruits and processing of the HSI data. Followed by Section 3, which presents and discusses the results from this study and finally the conclusions in Section 4. 2 2.1
Strawberries were purchased from local markets in Norway and fifty of them were selected as candidates by avoiding the fruits having any defects, bruises, and infections. All the strawberries were belong to the same class and produced in the same environment. The fruits were kept in ideal storage conditions, and taken to the lab environment an hour before the hyperspectral image acquisition, in a controlled room temperature. The strawberries were washed to remove any contaminations and the water drops from the fruits were wiped away using a dry clean cloth, before measurement.
Hyperspectral acquisition system used for this experiment is shown in Fig.1, which is
the same setup used in this experiment [17] except for the samples used.
(2) p = { }j=1 (3) Finally, SID can be defined as Equation 4, where x and y are the normalized vectors generated from reference (r) and target (t) Where ( , ) = ( ∥ ) + ( ∥ ) ( ∥ ) = ∑ =1 log
( ∥ ) = ∑ =1
(4) (5) (6) 2.7
The major steps in the data processing pipeline are preprocessing, calculating normalized reflectance, and sample segmentation. The camera software performs the preprocessing of the data, which includes dark current reduction, sensor corrections, and radiometric calibration. After preprocessing, the HSI data were converted to normalized reflectance by utilizing the known reflectance of the reference target present in the scene. Then manual selection of the region of interest (ROI) will be done for each strawberry for segmentation to avoid saturation areas of the data. The saturation areas are part of the fruit possess abnormal spectra due to the glossiness of the strawberry. For faster processing of data, here we decided the ROI size as 5x5 pixels. 2.8
The proposed CNN architecture was implemented in Python using Keras [24] and a Python framework known as SHERPA [25] was used for parameter tuning. The number of filters, kernel size for convolution layer, the batch size for training, learning rate, number of hidden layers, and number of epochs of the proposed CNN model were tuned using SHERPA. All those parameters were initialized with random Gaussian distributions and optimized using Bayesian optimization for hyperparameters tuning [26]. 2.9
The entire strawberries were classified into two groups based on a threshold sugar value. The berries have sugar values greater than the threshold was considered as high sugar content and the berries having lower sugar values than the threshold were considered as low sugar content. The sugar values of the strawberries were varies between 6oBx and 12oBx, hence we used multiple threshold values starting from 7oBx to 10oBx with 0.5oBx increment. The usage of varying threshold caused imbalanced data sets and used random oversampling to compensate for the imbalanced data. The oversampled data were divided randomly into the train and test data, with train data, contains 80% of the total data, and the remaining were used for evaluation of 1D-CNN. To evaluate SAM and SID, the reference spectra were generated from the training data set by calculating the mean spectra[27]. The K-Fold technique with shuffle on and split count five was used to calculate the cross-validation result.
Accuracy was used as the parameter for comparing the classification capability of the proposed method against SAM and SID. Accuracy is defined as the ratio between truly predicted outcomes (true positives + true negatives) and the sum of all predictions. HSI of 50 strawberries were acquired and processed using the setup and processing methods described in Section 2.2 and 2,7. The average spectra for all strawberries obtained from their ROIs are presented in Fig.4, we can observe that they appear nearly similar in visible region and differ in near infrared (NIR) region. From the average spectra, it is difficult to classify them visually; hence, we required some reliable method for achieving this.
The proposed 1D-CNN is implemented and tuned for hyperparameters, the final architecture with fine-tuned values are shown in Fig.5 The number of hidden layers required was determined as three, the input and output data sizes for each block based on the final parameters are updated in the diagram. The details of the parameters and final tuned values are displayed in Table 1. The fine-tuned 1D-CNN is trained, tested, and compared the result against SID and SAM. Fig.6 shows the variation in the accuracy and loss against epochs, and it can be observed that accuracy and loss flattens in a few epochs. Table 2 provides the summary of accuracy results obtained after cross-validation for each threshold values. From these results, it is clear that the proposed method outperformed the traditional methods, the 1D-CNN possesses a high average accuracy score of 0.96 compared to 0.58 of SID and 0.6 of SAM. Also, it is not fare to compare this result against the previous studies related to sugar and spectra of strawberries because they were mainly focused on predicting sugar values rather than classification [28][29] To evaluate the effectiveness of the method, we varied the threshold sugar values and executed cross-validation for each threshold value. The average accuracy obtained from cross-validation is plotted in Fig.7, which showed that the threshold sugar value has a negligible effect on the accuracy of the classification in 1D-CNN method compared to SID and SAM. The performance of SAM and SID was low because of the nearly identical reference spectra, they need a reference spectrum to compare against the test spectra and the reference spectra were generated from the training spectra by calculating the average. The sample reference spectra were presented in Fig.8 and we can observe that the reference spectrum for low and high sugar values possess a nearly identical spectrum. Hence, these methods such as SAM and SID, which relies on geometry of the spectrum, fails to predict correctly in most cases. However, 1D-CNN that can extract features from every sample spectrum in the training set can learn effectively and make a precise prediction. 4
Hyperspectral acquisition of the strawberries were performed and created an HSI database of 50 strawberries. The proposed 1D-CNN method was implemented and tested on the strawberry’s HSI data set. In addition, validated the classification accuracy of the proposed method against SID and SAM and the proposed method produced a higher accuracy in classifying strawberries based on sugar levels. In future research, we aim to extend these results into wide verities and larger sample count in order to make an industrial application. Choi, K., Lee, K., Kim, G.: Nondestructive quality evaluation technology for fruits and vegetables using near-infrared spectroscopy. Int. Semin. Enhancing Export Compet. Asian Fruits. (2006).
Manolakis, D., Marden, D., Shaw, G. a: Hyperspectral Image Processing for Automatic Target Detection Applications. Lincoln Lab. J. (2003).
Pu, Y.Y., Feng, Y.Z., Sun, D.W.: Recent progress of hyperspectral imaging on quality and safety inspection of fruits and vegetables: A review. Compr. Rev. Food Sci. Food Saf. (2015). https://doi.org/10.1111/1541-4337.12123. Zhu, H., Chu, B., Fan, Y., Tao, X., Yin, W., He, Y.: Hyperspectral Imaging for Predicting the Internal Quality of Kiwifruits Based on Variable Selection Algorithms and Chemometric Models. Sci. Rep. (2017). https://doi.org/10.1038/s41598-017-08509-6.
Li, J., Huang, W., Tian, X., Wang, C., Fan, S., Zhao, C.: Fast detection and visualization of early decay in citrus using Vis-NIR hyperspectral imaging. Comput. Electron. Agric. (2016). https://doi.org/10.1016/j.compag.2016.07.016.
Adão, T., Hruška, J., Pádua, L., Bessa, J., Peres, E., Morais, R., Sousa, J.J.: Hyperspectral imaging: A review on UAV-based sensors, data processing and applications for agriculture and forestry. Remote Sens. (2017). https://doi.org/10.3390/rs9111110.
Deborah, H., George, S., Hardeberg, J.Y.: Spectral-divergence based pigment discrimination and mapping: A case study on The Scream (1893) by Edvard Munch. J. Am. Inst. Conserv. (2019). https://doi.org/10.1080/01971360.2018.1560756.
Melit Devassy, B., George, S., Hardeberg, J.Y.: Comparison of Ink Classification Capabilities of Classic Hyperspectral Similarity Features. Presented at the (2019). https://doi.org/10.1109/icdarw.2019.70137.
Melit Devassy, B., George, S.: Dimensionality reduction and visualisation of hyperspectral ink data using t-SNE. Forensic Sci. Int. (2020). https://doi.org/10.1016/j.forsciint.2020.110194.
Food and Agriculture Organization of the United Nations, http://www.fao.org/home/en/, last accessed 2020/06/10.
Donida Labati, R., Muñoz, E., Piuri, V., Sassi, R., Scotti, F.: Deep-ECG: Convolutional Neural Networks for ECG biometric recognition, (2018). https://doi.org/10.1016/j.patrec.2018.03.028.
Hershey, S., Chaudhuri, S., Ellis, D.P.W., Gemmeke, J.F., Jansen, A., Moore, R.C., Plakal, M., Platt, D., Saurous, R.A., Seybold, B., Slaney, M., Weiss, R.J., Wilson, K.: CNN architectures for large-scale audio classification. In: ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing Proceedings. pp. 131–135 (2017). https://doi.org/10.1109/ICASSP.2017.7952132.
Abdeljaber, O., Avci, O., Kiranyaz, S., Gabbouj, M., Inman, D.J.: Real-time vibration-based structural damage detection using one-dimensional convolutional neural networks. J. Sound Vib. 388, 154–170 (2017). https://doi.org/10.1016/j.jsv.2016.10.043. 18. 19. 21. 22. 23. 24. 25. 26. 27. 28. 29.