-The article deals with the problem of choosing the neural network architecture in the problem of analyzing signal data obtained by shooting spectra from fluorescent sensors, which are based on the formation of exciplexes between the boron Dibenzoyl methanate fluorophore (DBMBF2) and aromatic compounds. Attention is paid to the problem of selecting the structural features and parameters of the network in the process of training and testing on available data.
This work is based on the usage of data obtained from the
sensor described in [
The problem of calculating the concentration of
hydrocarbons by using a known form of the spectrum can be
solved not only by using the least squares method. Other
method to solve similar tasks were presented in work [
However, the usage of neural networks is associated with a number of difficulties. In particular, we need to choose the network architecture that is appropriate for the task, as well as determine the values of hyperparameters that would allow us to solve the problem in the best way using the available data. With this approach, we will have to go through all the architecture and hyperparameters options, and choose a specific set that will give the best results of solving the problem among the presented options.
In the described case, we solve the problem of predicting the gas concentration based on the available values of the spectrum shape taken from two sensors. At the same time, the spectrum data taken from specific sensor does not depend on data of other sensor in any way and does not form any clear sequence. Each element of the source data is a onedimensional array of fluorescence intensities at different wavelengths, taken for a specific concentration of various hydrocarbons in the air.
Prediction problems are very often solved by using direct
propagation networks. The basis of networks of this type is a
multi-layer perceptron, which is widely used for data
processing in modeling, identifying various situations, and
predicting any events or values [
III. NEURAL NETWORK HYPERPARAMETERS SELECTION
Neural network hyperparameters are parameters that are used during network training, but do not change in the process. These include parameters such as: • Learning rate. • Neuron activation functions. • Optimization algorithm. • The batch size (Batch Size). • Number of neurons in hidden layers
The difficulty of choosing hyperparameters is that the choice must satisfy two conditions – solve the problem at the lowest values of prediction errors and provide sufficient generalizing ability of the network to avoid overfitting [6]. output data of training results and network predictions, was implemented.
There are various approaches for selecting the values of neural network hyperparameters [7,8]. The most popular method is called Grid Search, which could be described as searching for combinations of all the proposed values of hyperparameters in the network and selection of the best combination based on a certain metric, such as the deviation of predicted values from true values, in ppm in our case. It was decided to use this approach in this work.
Each of the sensor elements used in the experiment
provides data in form of an array of 2048 fluorescence
intensity values at different wavelengths. A detailed study of
the data allowed us to determine that out of 2048 channels
provided by each sensor, values of only 882 responded to
changes in concentration. Data from two sensors was used
simultaneously during network training. Thus, since
following above statements, the data used for network
training is a one-dimensional array of 1764 elements
representing fluorescence intensities captured by detectors
from two sensor elements consisting of different
chemosensory materials, characterized by surface
modification of the carrier matrix or the use of various
DBMBF2 derivatives[
The options of optimization algorithms was represented by such algorithms as Adam, Adagrad, Adadelta, SGD (Stochastic Gradient Descent).
The options of activation functions for hidden layers consisted of ReLU, LeakyReLU, Tanh (hyperbolic tangent), sigmoid, and linear function.
The learning speed was represented by values from 0.001 to 0.0001 with step of 0.0001.
Thus, all combinations of the parameters of neural
networks were trained and validated using cross-validation
on a samples of fluorescence intensity and concentration
values obtained from sensors during the experiment
described in [
As a result, the following parameters of the neural network prevailed: • Number of hidden layers: 3; • Number of neurons in hidden layers: 882, 882 and 220 in series; • Activation function of hidden layers of neurons: ReLU; • Learning rate: 0.0004; • Optimization algorithm: Adagrad.
IV. EVALUATION OF THE RESULTING NETWORK MODEL
To evaluate the obtained hyperparameters, a software tool, which was used to train a neural network with the proposed parameters and available data, as well as to obtain The software tool that was mentioned above consists of two components:
• A server-side program that directly trains the network using the capabilities of the supercomputer of the Samara national research University, by utilizing CUDA cores through Python libraries, TensorFlow and Keras to be exact; • A client application written in the Java using libraries such as Swing, for providing GUI, JSch to connect to server and control training process and Apache Commons to obtain data after training finishes. This application loads the Python program, and transfers data, used to train the network, to the server. It also was controlling the start of training and receiving output results obtained as a result of training the network and predicting gas concentration values on testing data-sets.
Screenshot of the operation of this application is shown in figure 1.
This application was used to test the previously obtained set of hyperparameters by training network on 1,376 examples of sensor measurements, with concentrations presented in fugures 2, 3 and 4.
Cross-validation by the K-Fold method, where k=10, was used to ensure validity of our results, which gave us the prediction results for benzene concentrations shown in figure 5.
The same set of hyperparameters was applied to training neural networks used for prediction of concentrations of toluene and p-xylene. The results of k-fold cross-validation for those neural networks is shown in figures 6 and 7.
As you can see from the picture, the predicted concentration values are very close to the real ones. The average error rate during validation was 8 ppm, when measuring benzene with evenly distributed concentrations equal to 30, 60, 100, 133, 171, 200, 240 and 300 ppm. This confirms that the multilayer perceptron with the architecture presented in this paper can be used in the task of analyzing spectral data using two or more sensors, which will allow us to obtain a sufficiently high accuracy of predicting gas concentrations.
Despite the fact that the obtained result of selecting hyperparameters in training already shows results close to real ones, it is possible to achieve even greater accuracy by selecting hyperparameters using evolutionary algorithms, such as the genetic algorithm, which has been growing in popularity in recent years when performing this task.
The research was supported by the Ministry of Science and Higher Education of the Russian Federation (Grant # 0777-2020-0017) and partially funded by RFBR, project number # 19-29-01135.