As robots become more pervasive in the service sector, control in dynamic environment has become an important element in optimising the deployment of mobile robots. A mobile robot should be knowledgeable not only of the barriers, but also of the surface on which the robot navigates to estimate slippage and adaptive control. We note that various terrains/surfaces have diferent characteristics, which can directly influence the handling, driving, eficiency, and stability of the robot vehicle. Knowledge of the terrain can provide valuable information for establishing efective and secure navigation strategies. We built a mobile robot prototype equipped by Inertial Measurement Unit (IMU) to obtain the terrain data and applied deep learning models to classify the terrain using the data. Three deep learning configurations have been proposed in this paper, i.e. long short-term memory (LSTM), 1D convolutional network (1D CNN), and convolutional neural network-long short-term memory network (CNN-LSTM). The deep learning architectures were trained and evaluated based on the data collected from ifve diferent surfaces. It is shown that the CNN-LSTM performs the best with an F1 score of 98.49%. The other two networks also generalize relatively well with the unseen vibration sequences with F1 scores of 97.47% and 95.98% for the 1D CNN and LSTM, respectively. Finally, we investigate the efect of varying input sequence to find the optimal length, so that we are able to obtain the highest accuracy and generalization of the deep learning networks.
Intelligent robotics have seen rapid advancement in their
scope of operations such as in military reconnaissance
in hostile environments [
The problems of adaptive control in mobile robots have
been constantly researched. The challenges presents
various opportunities for researchers to develop methods in
predicting the dynamic changes in the environment. In
which depend on the characteristics of the terrain.
speed of navigation and the parameters of route planning, results to recognize the surrounding environment of the
[
In [
Several sensing methodologies have been developed to Fourier transform (DFT). Experiments using a CNN for
tackle the problem of terrain classification. The method- vibrational wheel slip estimation in ground robotics was
ology is typically categorized into two main groups: vision- carried out in [
While vision-based approaches are useful because of measures six vibrational terrain signatures. The setup
their high accuracy, they are vulnerable to distortion is shown in Fig. 2. The form is very similar to the
concaused by lighting changes and other factors such as ventional indoor service robots such as robotic vaccuum
the realization of the surface’s physical properties (e.g. cleaner. The vibration characteristics are all dependent
material type and degree of hardness) [
is typically used for sequence prediction. In particular, LSTM solves the issues of the disappearing gradient present in the RNNs while allowing the long-term temporal dynamics of the series to be exploited. In contrast,
image classification task); however, it can be modified to classify the 1D vibrational problem. The dimensionality of the convolutional layers is reduced to match the model’s 1D input.
ing the temporal dependencies of the vibration sequence.
by the 1D CNN to extract the higher level features. This can be considered as the pre-processing step which allows the LSTM to interpret the features extracted at each block of the sequence. The concept is illustrated in Fig.
Layer LSTM (20 units) Dropout (25%) LSTM (70 units) Dropout (40%) Dense Dense Conv1D (80@6×1) Dropout (50%) Conv1D (128@6×1) Dropout(50%) Max pooling Flatten Dense Dense Conv1D (96@6×1) Conv1D (48@6×1) Dropout (30%) Max pooling Flatten LSTM (20 units) Dropout (20%) Dense Dense sorflow backend with the Keras API. A detailed overview of the three models was summarized in Table 1. The hyperband algorithm was used to select the hyperparameters allowing for the best balance between training time and accuracy. The learning rate, batch size, and number of epochs were set at 0.001, 64, and 30, respectively. Additionally, early stopping regularization was implemented to avoid overfitting during model training. Further, the CNN-LSTM Adam optimization algorithm based on the Stochastic gradient descent was used as the optimizer.
The implemented CNN-LSTM architecture is shown in Fig. 5. For both the LSTM and 1D CNN networks, the data length of a vibration training sample was a flat vector of 75 time steps. In a stacked LSTM network, the input sequence to the first LSTM layer returns a shape to retain its temporal representation during the convoof (timestep, unit) to be passed on to the next layer. The lution process. The time distributed layer expects a 3D output from the last LSTM layer returns only the unit. input and so the input sequence was reshaped from 75u For the 1D CNN, the input shape to the network is repre- time steps into 3 subsequences of 25 time steps. The consented as (timestep, features). In the case for the CNN- volutional layer used the ReLU activation and consisted LSTM network, a time distributed wrapper is first used of a 6 × 1 kernel that moves across in one dimension before the LSTM layers to allow the input vibration signal during the convolution operation.
The dropout layers were then added to tackle overfit- Fig. 7 illustrates the performance of the models on the ting issues by arbitrarily setting a fraction rate of input 5-class vibration test data set. In the worst case, about units to zero. The pooling layer was added to reduce the 7% (average) of the wood class was mistakenly classified spatial size of the output representation into half. Note as tiles across the three models. Overall, it can be seen that both the dropout and pooling layer allows for faster that the three models exhibited good performance and training time due to the reduced parameter size. The generalized well with the unseen data. Further, the CNNlfatten layer was used to transform the input from the LSTM architecture has the best performance with F1 previous layers as input to the LSTM layer where the score of 98.49% (average). The 1D CNN follows at the temporal characteristics of the vibration sequence were second place with F1 score of 97.49% (average). We note extracted. Lastly, the fully connected layers with the soft- that the slight improvement of the CNN-LSTM model max activation function was used to structure the outputs compared to the 1D CNN may suggest that the temporal of the previous layer for the final classification task. In characteristics of the LSTM is less important than the this experiment, the categorical cross-entropy loss func- feature generation capability of the CNN-LSTM. Table 4 tion was used to address the 5-class terrain classification summarizes the overall performance of the three models. problem.
Fig. 6. From the confusion matrix, we can calculate the F1 score, the precision , and the recall, . The F1 score, which is calculated using and , has been commonly used to analyze the performance of the models. We used macro-averaging technique to expand these benchmarks towards multi-class terrain classification. The equations for the precision, recall, and F1 score, respectively are given by = = 1 = , , + + 2 , + where shows the outcome where the model correctly classifies the positive class, is the outcome where the model incorrectly classifies the positive class, is the outcome where the model correctly classifies the negative class, and is the outcome where the model incorrectly classifies the negative class. (2) (3) (4)
One factor influencing the performance of the models is the sequence length of the input vibration. To further validate the performance of the three models, we analyzed the F1 score with varying segment lengths as shown in Fig. 8. It can be seen that a longer sequence length results in a better accuracy with the cost of performance saturation at a certain length. It can be shown that the average F1 score rises as the duration of the vibration series increases from 30 to 60 samples but decreases afterwards. This may be caused by the lack of the training data after the segmentation process of the given length. Therefore, we can consider an optimal sequence length of 75 (1.5 seconds) for obtaining high accuracy and generalization of the models. Furthermore, the proposed CNN-LSTM architecture slightly outperformed the CNN and LSTM models across the varying segment lengths.
IMU-based surface classification task. We have compared three candidates for classifying the IMU data, i.e. LSTM, 1D CNN, and a combination of CNN and LSTM. By comparing the results, CNN-LSTM provided the best results (F1 score of 98.49%). However, we can further observe that the 1D CNN presented favorable results although slightly lower than the CNN-LSTM. The results suggest that 1D CNN is able to map the classification better when compared to the LSTM on standalone basis. CNN and LSTM works on diferent principle in which the latter is based on the temporal dynamics of the data. On the other hand, 1D CNN is based on static convolution, similar to the 2D counterparts. This implies that there is a clear static pattern when the IMU data enabling well defined mapping to their respective classes.
The results, despite counter intuitive, may prompt furFigure 6: Confusion matrices of (a) LSTM, (b) CNN, and (c) ther research in the this direction. With the growing CNN-LSTM on the vibration test dataset. of edge computing and capacity of embedded system, enabling robots to recognize surface would enable further applications for indoor or industrial applications.