Infrared spectroscopy has proved to be a powerful tool for solving organic chemistry problems and finds a widening field in many industries. Infrared absorption and its relation to the molecular structure of organic material are discussed to give the essential background for detailed descriptions of techniques adopted in this work. Existing spectral analysis approaches rely on pre-processing and feature selection methods to remove signal artifacts based on prior experiences. This work introduces a data-driven deep learning approach and successfully applies it to predict organic powders' mixtures. In particular, in this work, we use a convolutional neural network to predict diferent composition percentages of mixed organic powders. We show that using specific pre-processing steps, such as Savitsky Golay smoothing and derivatives, can increase the accuracy of the results.
Since its discovery and application, Near-infrared (NIR) spectroscopy evolved from an addon unit to a standalone unit in many areas. Numerous applications of this methodology have been eminently successful and have become familiar to many chemometricians. Daily, spectroscopy is performing analyses impossible by any other method. More common analyses are completed in a few minutes, which previously required hours.
The underlying basis of applied infrared spectroscopy is that all organic substances possess selective absorption at specific frequencies in the infrared portion of the electromagnetic spectrum. Infrared spectra of biological organic materials are signals composed of peaks because of molecular vibrations of mostly O-H, CH, and N-H groups [15, 4] caused by their interaction with infrared light within the NIR wavelength region (800-2500 nm).
The plot of these transmission or absorption values versus frequency or wavelength units constitutes an infrared spectrum characteristic of the sample. It is used to describe the intrinsic factor of a sample. Applications of infrared spectroscopy can be divided into two general categories, qualitative and quantitative.
A mixture of materials can be analyzed quickly and accurately so long as the components present in the mix are known. From a study of the spectra of the known compounds, it is usually possible to find a frequency at which only one component possesses strong absorption and thus nfid its quantity in a mixture. This rapid method, combined suitably with deep learning methods, has shown an accuracy of 1 percent or better to a pair of organic mixtures [30].
Near-infrared spectroscopy is often used to assess the quality of rice adopting respectively PLS and a multi-linear regression (MLR) [34, 24]. Rapid methods, like NIR technology combined with multivariate analysis (PCA and partial least squares discriminant analysis (PLS-DA)), are used to detect fraud of cocoa powder [32].
Kernel-based methods, such as Support Vector Machines (SVM), improved the multivariate analysis [29]. Machine learning improved previous models in spectral profiles analysis [9, 8, 38], and in particular, Convolutional Neural Networks (CNN) for spectroscopy signal classification have reported promising results in the literature [30]. In their work[30] proposed a modified version of the 1D-CNN model proposed by [20], by tuning the hyper-parameters. They predicted the composition percentage of organic mixtures, dividing the dataset into three main groups for testing purposes. 1D-CNN combined with near-infrared spectra showed better results than traditional PLS models in classification and regression tasks [37, 21, 37]. In this work, we show a quantitative application of near-infrared spectroscopy with the use of Convolutional Neural Networks with the aid of signal filtering and pre-processing methods. 1.1
The filtering method introduced by Savitzky Golay has long been used in the absorption spectroscopy community for its ability to simultaneously smooth and diferentiate absorption spectra. In this investigation, the Savitzky Golay method is shown applicable to our ranges and improve the results.
The total number of mixtures of the datasets is fifteen , and each mixtures comes in diferent quantities of each component. While the number of the base materials is six. We extracted their near-infrared spectrum profiles. From our experimental setup, we delivered an answer to these research questions: – RQ1 Can we predict the composition percentage with high accuracy of mixtures of three materials? – RQ2 Can we improve the prediction results by adding extra features like the derivative to the original spectra?
This paper is structured as follows. Section 2 mainly presents the dataset and the 1D-CNN architecture. Section 3 shows the experiments’ setup and the results. In Section 4 the results are discussed and Section 5 presents the conclusions. 2
In this section, we present the data collection process and the methods adopted to predict the quantity of the materials in each mixture. The data collection comprises several steps: the sample preparation procedure of the six organic powders and the data acquisition that describes the spectral data’s mechanism. Furthermore, we describe the convolutional neural network architecture and its parameters, the Savitsky Golay filter, and the use of derivatives. 2.1
The sample preparation procedure is the same as described here [30]. Including the new mixtures of three materials, we added in this work, follow the same steps. However, the mixtures’ homogeneity is still not guaranteed due to the unique characteristics of the materials used, such as grain size and tendency to form lumps.
We made fifteen pairwise combinations using cocoa (Cocoa), ice sugar (IceSugar), baby milk powder (BabyMilk), potato starch (Potato), rice starch (Rice), and baking soda (NaHCO3). We added other pair mixtures to the first dataset totaling 69 samples. Their composition percentage is made up from set: P = {15, 25, 33, 35, 40, 45, 50, 65, 75, 85}. The composition percentage of a given mixture of two materials adds up to 100%, e.g., (A=15, B=85), where A and B are the mixed materials. Moreover, we added six other mixtures using three material in diferent compositions, and their percentages is made up from set P . Their composition adds up also to 100%.
E.g., (A=33, B=33, C=33) or (A=45, B=40, C=15), where A,B and C are the three mixed materials. In Table 1 and in Table 2 is shown the mixtures made and from which near-infrared spectra is acquired. 2.2
The data acquisition is the same as described here [30]. We used the same sensor to acquire the spectra of the triple material mixtures: the sensor captures two wavelength ranges: [1350 − 1650]nm and [1750 − 2150]nm. The total number of wavelength points captured is 702. Dataset Following the material preparation and the acquisition of the NIR spectra, we collected 506160 samples3. Each material has 702 features corresponding to the captured wavelengths, and for each composition percentage of a mixture, we have ∼ 7000 samples. The target variable of each sample is a percentage distribution over the six base materials describing the quantity of that material in the spectral sample. Given that each spectral sample represents only the mixture of two or three materials, only two or three elements in the target vector contain the value of the individual materials described in the spectral sample. At the same time, we set the remaining four base materials to 0. Whereas, for the mixtures containing only one material, we set the five remaining target variables to 0 and assigned the value 100% to the material represented by the pure material spectra.
We took the dataset of the spectra sample of the diferent organic materials [30] and added other mixture’s spectra as explained in Section 2.1, and made three major groups of spectra profiles : – Single Material Dataset (SMD) – Mixed Materials Dataset (MMD) – Triple Mix Materials Dataset (TMMD)
The performance of the 1D-CNN model is evaluated using each group separately. Then the results are compared with the use of derivatives and the Savitsky Golay lfiter, so each dataset is used with the following modes: – data as it is (standardized) – applying the Savitsky Golay lfiter and concatenating the first derivative – applying the Savitsky Golay filter and concatenating the first and second derivative 3 The dataset is available upon request.
As the baseline result, we used the dataset without any pre-processing other than standardizing. We compare the baseline against the pre-processing steps such as the application of Savitzky Golay filter and the concatenation of derivatives to the spectral data. 2.3
Convolutional Neural Network Convolutional Neural Networks (CNNs) are specific neural networks (NNs) used to process data with a known, grid-like structure [27]. (i.e., time-series data can be thought of as a 1D grid taking samples at regular time intervals, and image data, as 2D or 3D grid of pixels).
In this work, we adopted the 1D-CNN used here [30], which consisted of seven trainable layers - four convolutional layers and two fully connected layers. The ifnal model has a total of 713510 trainable parameters, the detailed architecture is summed up in Table 3. The input of the 1D-CNN is a one dimensional spectral vector containing values of the 702 wavelength points, and the target is also a one dimensional vector containing the percentage distributions of the six materials. Parameter Optimization The main objective of our experiments concerns the prediction of the percentage of material contained in a composite mixture. To this aim, we use the Kullback-Leibler (KL) divergence (Equation 1) as our loss function. The KL is a measure of divergence between two distributions defined as:
DKL(Q∥Z) =
X Q(x) log x∈X
Where Q and Z are two probability distributions, that in our case, correspond respectively to the true and the predicted distribution of percentages. Moreover, we applied Adam as an optimization algorithm [13], with a scheduled learning rate, starting from LR = 0.001, and exponentially decreasing it every epoch. We also use early stopping criteria by limiting epochs to 100. Finally, we used the Keras package as our machine learning framework for the entire work running on Asus VivoBook X580GD with an Intel(R) Core (TM) i7-8750H CPU. 3
This section specifies the datasets used and results obtained when predicting the quantity of the materials in each mixture. We used three diferent datasets, as mention in Section 2.2, to evaluate our model and the pre-processing approaches: the single material dataset (SMD), the mixed materials dataset (MMD), and the triple mix materials dataset (TMMD). We used each dataset with three diferent modalities: the standardized data, applying the Savitsky Savitsky Golay iflter, and concatenating the first derivative and then both the first and second derivatives.
Each dataset has been divided into train, validation, and test set. We first split each dataset into train and test sets. After this, we keep aside the test set and randomly choose 75% of the train set to be the actual Train set and the remaining 25% to be the validation set. While the validation set is 25% of the training set. The model is then iteratively trained and validated on these diferent sets.
We used the mean absolute error (MAE) to evaluate the models performance: MAE = 1 X n
ˆ |Y − Y | where n is the number of samples in the test set, Y is the vector containing the mixtures percentage and Yˆ is the vector of the predicted values. 3.1
The SMD is composed of six materials purely. For the baseline setup, SMD has ∼ 7000 samples for each material. Each instance contains 702 wavelength points. We Standardized the spectral data by removing the mean and scaling to unit variance. The standard score of a spectra xij is calculated as:
zij = (xij − uj )/sj where uj is the mean of the spectral data features and sj is the standard deviation of the spectral data features.
We have applied the Savitsky Golay filter to the standardized data, and
to each sample, we concatenated its first derivative obtaining a total of 1404
wavelength points as features. When also adding the second derivative, we reach
a total of 2106 features.
(
Figure 1 shows the result of the use of derivative information on the spectra compared to the baseline model. The mixed material dataset is composed of fifteen pairwise mixtures with diferent composition proportions that add up to 100% of the whole mix as described in Section 1.1. The MMD contains ∼ 450000 number of samples, and ∼ 7000 samples for each mixture quantity. The dataset has been divided into training validation and test set following the procedure mentioned in Section 3. We have more than 100k samples in the test set representing each mixture and its components quantity for this dataset. The baseline setup has 702 standardized features. In Figure 2 it is shown the results of the prediction of each material in composition, and it is compared against the use of pre-processing steps described above. 3.3
The TMMD contains mixtures composed of three diferent materials in diferent quantities. In Table 2 it is shown the mixtures combination, and their composition percentage is taken from set P described in Section 2. The TMMD is composed of ∼ 11000 number of samples, divided into train, validation and test set. The baseline setup has 702 standardized features. The Formula 2 is used to evaluate the results obtained from the model and pre-processing steps. In Figure 3, it is shown the result of the baseline setup in comparison with the pre-processing steps adopted. The results achieved from the SMD are assuring. With the baseline setup, we have been able to predict all the single materials with a very low error as seen from the blu line in Figure 1. The Savitsky Golay filter and the addition of derivative information to the spectra helped us gain even more accuracy when predicting.
Increasing the complexity of the dataset by mixing a pair of single materials with diferent quantities results in the increase of the error of the predictive model. We can see the massive diference of the MAE between the SMD and the MMD from Figure 1 and Figure 2. Nevertheless, adding the first derivative of the spectra to the spectra itself led to an overall 3% improvement, and while the 2nd derivative worsened by an average of 44%.
By increasing the number of materials in the mixtures from two to three increases the MAE, due to the intrinsic problem of mixing the materials and acquiring the near-infrared spectra. Nevertheless, the derivative information of the spectra lowered the MAE of the baseline to 47% and the second derivative to 30%. These results are promising since the model can extract the features of the specific composition percentages of mixtures.
We must take into account also the fact that the quantities of the materials are prepared by weight rather than volume; this means that we can have powders like BabyMilk that have a greater volume for a small amount. This characteristic can afect the spectral acquisition since the material with higher volumes tends to occupy most of the Petri dish, causing little signal for the other materials mixed with them.
Finally, the answers to our research questions are the following: – A1 - The prediction of the composition percentage of the three materials in a mixture is a very challenging task. Because it has inherent problems when making the mixture itself and acquiring the spectra. Since there is no guarantee of the homogeneity of the mix due to the diference of volume/weight ratio of each material causing dificulties when acquiring the near-infrared spectra. – A2 - Concatenating the derivative of each spectrum to the spectra itself adds even more information, and we saw that in each dataset, there is a vast diference compared to the baseline. There is a 37% decrease in the MAE when adding the first derivative and a 14% decrease when adding the 1st and second derivative. In this work, we evaluated the use of derivatives in the context of spectra preprocessing when predicting the composition percentage of organic material mixtures. The NIR spectra of organic materials hold intrinsic properties of the composition, including its quantity and derivatives, add another characteristic to the spectra, making it easy to analyze.
Combining the standardized NIR spectra of a material or mixtures with their relative derivatives can uncover further information and result in a classification or regression task, making the model robust.
We also uncovered the results of mixtures composed of three materials, and we saw a growing trend in the complexity of preparing such mixtures and their analysis. The increasing percentage of error also shows this compared to the SMD and MMD.
Additional work is needed to tackle the high number of attributes when concatenating the derivatives data to the original spectra. So, models that reduce the dimensionality of the single sample without losing too much information are good starting points to improve the overall performance of near-infrared spectra analysis. 26. Solange Sanahuja, Manuel Fed´ou, and Heiko Briesen. Classification of pufed snacks freshness based on crispiness-related mechanical and acoustical properties.
Journal of Food Engineering, 226:53–64, 2018.
27. H. Shin, H. R. Roth, M. Gao, L. Lu, Z. Xu, I. Nogues, J. Yao, D. Mollura, and
R. M. Summers. Deep convolutional neural networks for computer-aided detection:
Cnn architectures, dataset characteristics and transfer learning. IEEE Trans. on
Med. Imaging, 35(