=Paper=
{{Paper
|id=None
|storemode=property
|title=Feature Extraction for Terrain Classification with Crawling Robots
|pdfUrl=https://ceur-ws.org/Vol-1422/179.pdf
|volume=Vol-1422
|dblpUrl=https://dblp.org/rec/conf/itat/MrvaF15
}}
==Feature Extraction for Terrain Classification with Crawling Robots==
https://ceur-ws.org/Vol-1422/179.pdf
J. Yaghob (Ed.): ITAT 2015 pp. 179–185
Charles University in Prague, Prague, 2015
Feature Extraction for Terrain Classification with Crawling Robots
Jakub Mrva and Jan Faigl
Czech Technical University in Prague, Technická 2, 166 27 Prague, Czech Republic
jakub.mrva|faiglj@fel.cvut.cz
Abstract: In this paper, we address the problem of terrain
classification using a technically blind hexapod walking
robot. The proposed approach is built on top of the ex-
isting method based on analysis of the feedback from the
robot’s actuators and the desired trajectory. The formed
method uses features for the Support Vector Machine clas-
sification method that assumes a regular time-invariant gait
to control the robot. However, such a gait does not allow
the robot to traverse rough terrains, and therefore, it is nec-
essary to consider adaptive motion gait to deal with small
obstacles, which is, unfortunately, not a regular gait with
some fixed predefined period. Therefore, we propose to al-
ter the features extraction process to utilize the terrain clas- Figure 1: Used hexapod walking robot for the terrain clas-
sification method also for an adaptive motion gait, which sification
enables the robot to traverse rough terrains. The proposed
method has been experimentally verified on several ter-
The existing method [7] uses a default robot motion gait
rains that are not traversable by a default regular gait. The
with regular and periodic phases of the leg movements,
achieved results not only confirmed the high accuracy of
and therefore, it is suitable only for flat terrains without
the terrain classification as the existing approach, but also
significant obstacles. Based on this method, we consider
expanded the area of operation of a hexapod walking robot
several terrains with obstacles or stairs and their classifica-
into more challenging terrains.
tion using an adaptive motion gait [8] that allows a smooth
transition while reducing the workload of the servos and
1 Introduction thus avoiding overheating. Such a gait does not preserve
the predefined trajectories of each leg as a default motion
Crawling robots can operate in a much greater scope in gait does. Hence, the existing method of feature extrac-
terms of terrain diversity than classical wheeled robots. tion proposed in [7] is not directly applicable because it
The control complexity is, however, much greater due to assumes a regular time-invariant gait with fixed trajecto-
the high number of degrees of freedom (DOF). One way ries. Therefore, in this paper, we propose a modification
to handle a high DOF is to generate a walking pattern—a of the feature extraction process of [7] to enable the terrain
gait [1]. A simple regular gait gives the robot predefined classification based only on the servo drive feedback also
trajectories for all legs, which are therefore alternating in in crawling rough terrain using the adaptive motion gait.
their support and transfer phases. Therefore, the method can be used in more challenging
In order to increase the robot’s perception of the terrains up to the structural limits of the hexapod walking
environment—for example to classify the terrain the robot robot.
is traversing—one can employ the robot with a variety The paper is organized as follows. A brief overview
of sensors. There can be found two complementary ap- of the adaptive motion gaits for rough terrains and ter-
proaches based on exteroceptive and interoceptive sensors. rain classification methods is provided in the next section.
In the case of exteroceptive sensing, we can utilize cam- A description of the considered robotic platform and def-
era [2, 3] or laser-based range measurements [4] for terrain inition of the problem is presented in Section 3. The uti-
classification. lized adaptive motion gait is briefly described in Section 4.
However, if the robot is technically blind and depen- The proposed feature extraction method is presented in
dent solely on interoceptive sensing, we can use force, Section 5 and experimental results in Section 6. The con-
torque [5, 6], or other tactile sensors to gather data about cluding remarks are in Section 7.
the interaction of the robot with the terrain. Moreover, we
can utilize the robot’s actuators themselves and develop a 2 Related Work
classifier based on the differences between the expected
and real trajectories of the robot servo drives [7] without Adaptive motion of a walking robot to traverse a rough ter-
the need of any additional sensor. rain has been addressed by many researchers and several
180 J. Mrva, J. Faigl
approaches can be found in literature. A complex control gait and thus it is applicable almost exclusively on flat ter-
architecture of quadruped walking robot to traverse chal- rains without obstacles.
lenging terrains has been presented in [9] using several In this paper, we proposed a combination of the terrain
sensors attached to the robot and a precise map created classification method [7] with the adaptive motion gait [8].
off-line. The off-line scanning can be avoided by using Both these approaches are based solely on interoceptive
an elevation map created from an on-board laser scanner sensing using active actuators and we propose a new fea-
and used further to alter the gait according to the terrain ture extraction procedure to overcome limitations of [7]
structure [10]. and enable on-line classification of rough terrains.
Another existing direction of the adaptive motion gaits
are based on approaches that do not utilize a terrain map.
They are based on a tactile information from force sen- 3 Problem Statement
sors [11] (or torque-based estimation of the force [12])
The main problem being addressed in this paper is to ex-
utilized to adapt the gait according to the terrain and to
tend the existing terrain classification approach [7] to the
ensure the leg reaches the foothold. A passive actuator
adaptive motion gait [8] and thus generalize terrain classi-
to measure the ground reaction force has been proposed
fication also for traversing rough terrains. A new feature
in [13] to substitute direct force or torque sensors, which
extraction method is needed to deal with rough terrains
is a suitable approach for the deployment of cheap robotic
because the adaptive motion gait does not preserve the re-
platforms. In [8], we proposed a similar approach that is
quired condition on the motion gait of [7], i.e., a time-
even more minimalist since it does not need additional ser-
invariant motion gait. In the proposed approach, we con-
vos and thus it is solely based on the robot’s actuators.
sider a relatively cheap and easy-to-use platform Phan-
The problem of terrain classification is widely investi-
tomX Hexapod Mark II with Dynamixel AX-12A actua-
gated also regarding on-board processing. A camera can
tors, see Fig. 1, which is further described in the next sec-
be used to estimate the terrain class based on extracted
tion. An overview of the terrain classification method [7]
features [3] that can be further used to select an energy
is provided in Section 3.2 to provide a background for the
efficient motion gait [2]. Authors of [14] used a laser
proposed approach.
range finder for distinguishing between twelve terrains and
achieved promising results; however, under specific labo-
ratory conditions only. 3.1 Hexapod Structure
Focusing on a structural point of view, an off-line scan
of the terrain from a precise external laser-scanning system The used hexapod platform has six legs each with three
was used in [15] to generate a database of terrain templates joints formed from the Dynamixel actuators. The schema
that are used for proper foothold planning. Authors of [4] of the leg and the description of its parts is depicted in
proposed an approach to avoid building a large database Fig. 2. All joints (θC , θF , and θT ) are controlled with a po-
of templates. Their idea is based on a creation of a set sition controller that provides every 33 ms the following
of several templates that define good and poor footholds information:
based on local concavity and sloppiness, which are useful • Desired position θ des ;
attributes for predicting slipperiness of the terrain. • Current position θ cur ;
Beside exteroceptive sensing, tactile sensors are used to • Error in position e = |θ des − θ cur |.
classify the terrain based on the direct measurements of
the robot interaction with the terrain. In [5], authors used Using the adaptive motion gait [8], the robot can tra-
features extracted from the measurements of force sensors verse small obstacles up to the limits of the robot structure.
placed at the tip of the leg that are combined with the mea-
surements of the motor current of the knee joint of a sin-
θC θT
gle vibrating robot leg detached from the body. A 6-DOF
ur
torque-force sensor was used in [16] (under the same lab-
m
Fe
oratory conditions as in [14]) for a discriminant analysis Coxa
between six types of terrain. However, all of these ap-
proaches are based on additional sensors, and therefore,
ia
θF
Tib
they increase a complexity of the robot.
A slightly different approach that utilizes only intero-
ceptive sensors built within the actuators was presented
in [7]. The actuators consist of position controllers that can
send both the desired and the current position of the servo. Figure 2: Schema of the leg consisting of three parts
The difference in these positions is then analyzed in time (links)—Coxa, Femur, and Tibia. The three joints (θC , θF ,
and frequency domains to extract a 660-dimensional fea- and θT ) are indexed according to the next respective link.
ture vector from the two front legs during each gait cycle. The joint θC is fixed to the body with a vertical rotation
This method is limited by using a periodic time-invariant axis while the other two joints have a horizontal axis.
Feature Extraction for Terrain Classification with Crawling Robots 181
We consider the robot is operating in an environment that Such 660-dimensional feature vectors for particular
satisfies the robot’s structural limits and there is not a large type of the terrain and several trials of traversing the ter-
obstacle that the robot cannot traverse. Hence, we are not rain are used to train a multi-class linear Support Vector
addressing obstacle avoidance and other high-level navi- Machine (SVM) classifier and authors of [7] report 95%
gation problem in this paper. Thus we are strictly focused accuracy in distinguishing between 3 terrain classes (con-
on the problem of terrain classification and its practical crete, grass, and rocks/mulch).
validation in real experiments. In our approach presented in this paper, we follow the
same idea of the terrain classifier based on the SVM, but
we propose a new feature extraction process to address the
3.2 Terrain Classification
absence of regularity in the adaptive motion gait for crawl-
The method of the terrain classification [7] has been pro- ing rough terrains.
posed for the same hexapod platform as we are using;
however, a regular time-invariant gait is utilized for robot
motion. The very general idea of the terrain classification 4 Adaptive Motion Gait
is based on the small errors in position control (e) of all
servo drives of the front legs (i.e., six servos) that are mea- The adaptive gait is originally based on a regular tripod
sured in the time domain at a non-uniform sample rate of gait in terms of predefined trajectories for each particular
approximately 20 Hz. leg. However, the trajectories can be changed and the gait-
In order to obtain a more dense data and to get a uniform cycle is divided into separate phases of the leg and body
sample rate for the FFT used in the feature extraction, the motion.
signal is interpolated using a cubic Hermite spline inter-
polation method that creates a continuous function with Choose next legs
a continuous first derivative. The interpolated function is STABLE STATE
from given order
then resampled at the frequency of 100 Hz. After that,
a feature vector for the classification is created from the Apply positions – Move legs up
sampled data and computed characteristics of the signal. – level body
Having the default regular gait, the data is windowed
Body Leveling
using a uniform window that contains the last three full
Leg motion
Transform all Move legs forward
gait cycles; so, each terrain-class prediction is based on the leg positions
past three gait cycles worth of data. Given the motivation
that different terrain surfaces induce a specific behavior in Compute Move legs
different sections of the gait cycle, the data are divided (R, ~t ) from down until
leg positions ground detected
into 16 equally wide segments within a gait cycle to form
a gait-phase domain. Respective segments from the last
three gait cycles are joined together and basic statistics of Figure 3: Diagram of a gait cycle. Firstly, the legs in the
all data samples that fall within are computed yielding in transfer phase move to find new footholds. Secondly, the
5 values (features) for each segment (i.e., minimum, max- body is leveled to adapt the new footholds. Finally, an-
imum, mean, median, and standard deviation). Repeated other legs are chosen for the next transfer phase. Orange
for each servo, we obtain a total of 480 gait-phase features, color highlights the motion of legs in the transfer phase
i.e., 2 (legs) × 3 (servos per leg) × 5 (features) × 16 (seg- only, while red color highlights the motion of all legs.
ments).
Additional 180 features are calculated in the frequency
domain. A Hamming window and discrete Fourier trans- The gait diagram is shown in Fig. 3, where it can be
form are applied on the same resampled position error sig- seen that legs in the transfer phase move through prede-
nal to obtain a frequency spectrum of 25 bins (0–12 Hz). fined checkpoints (up and forward) and begin approach-
All amplitude values of frequency bins are used alone, giv- ing another predefined checkpoint, which is situated far
ing another 25 features (for each servo), supplemented by below the current ground level (but still reachable). The
4 features obtained from the shape of the spectrum (i.e., ground sensing is done via observing the position error
centroid, standard deviation, skewness, and kurtosis) and e of the joint θF during lowering the leg with respect to
finalized with the energy of the spectrum. The overall a certain threshold value. Although all legs in the trans-
660-dimensional feature vector consists of: fer phase are moving simultaneously, each ground contact
stops only the particular corresponding leg.
• Statistics of each segment for each servo (5 × 16 × After the legs found their new footholds (the rest legs
6 = 480); stay motionless), a new body posture is found given the
• Bins of the frequency spectrum (25 × 6 = 150); feet positions in order to adapt to the terrain the robot is
• Shape of the frequency spectrum (4 × 6 = 24); traversing. The body motion itself is provided by moving
• Energy of the frequency spectrum (6). all legs according to a transformation of the feet positions.
182 J. Mrva, J. Faigl
In summary, the leg motion phase consists of 3 steps Due to the variances of the gait phases, which depends
(up, forward, and down) and is followed by a body leveling on the roughness of the terrain the robot is traversing, we
step. Given the tripod gait in which the legs are grouped cannot rely on a uniform partitioning of the gait phases
into two triplets that are alternating, we repeat the same into 16 segments for the feature extraction as in [7]. On the
steps for the other triplet to obtain a total of 8 discrete steps other hand, we can utilize the gait phases of the adaptive
per one full gait cycle. Notice, that as a consequence of the gait, as it is shown in the diagram in Fig. 3, and the data
adaptive-gait model, a leg is moving only in the transfer from one gait cycle can be therefore divided into 8 seg-
phase (3 steps) and in both body leveling steps (4th and ments according to the gait phases.
8th steps), where all legs are needed to move the body. Authors of [7] extended the feature vector by features
A more detail description can be found in [8]. extracted from a frequency analysis. However, such anal-
ysis requires a condition of periodicity that is not ful-
filled in the adaptive gait. Nevertheless, during a practi-
5 Feature Extraction cal experimenting, it has been observed that the absence
of frequency-based features did not prevent the classifier
The proposed feature extraction process is based on the to achieve accurate classification, which is shown in Sec-
terrain classification originally developed for a regular tion 6. Notice that the authors also did not consider fea-
gait [7], which has to be altered to deal with a different tures selection to reduce the 660-dimensional feature vec-
behavior of the adaptive gait [8]. The key difference be- tor; so, the frequency analysis may be expendable.
tween these two gaits is in their regularity and in the fact
that the regular gait is synchronized by a time signal and
a leg never stops moving, whereas the adaptive gait splits 6 Experimental Results
the leg and body motion according to the gait phases.
A leg trajectory during the phases of the gait is depicted Since the proposed method extends an existing approach
in Fig. 4. The regular gait is periodic and the robot is able by adding more rough terrains where the robot can op-
to traverse a flat terrain at the constant speed. Although erate, we focused the experimental evaluation of the pro-
the robot can pass very small obstacles at the cost of a posed method solely on those challenging terrains. Never-
high servo load, the robot is incapable to traverse a rough theless, we also used datasets from simple outdoor terrains
terrain using this regular default gait [8]. for completeness.
On the other hand, the adaptive motion gait utilizes a The proposed multi-class SVM classifier (with linear
tactile information to detect the ground-contact point and kernel) was trained for feature vectors collected from 7 dif-
thus it is able to decrease the servo load and adjust the ferent classes:
robot to the terrain. However, ground-contact points along
the vertical line (during moving the leg down) are not • Wooden stairs
known. Hence the time the leg spends in the ground- • Wooden blocks of different height
approaching phase is also not known. Moreover, the tra- • Office floor with small obstacles
jectory of the particular foot in the support phase is also in- • Office floor
fluenced by the contacts of the other legs with the grounds, • Asphalt
and therefore, the trajectory is not regular during crawling • Grass
rough terrain and it may vary significantly. These vari- • Dirt
ances have to be considered in the analysis of the servo
position signal in the feature extraction process to avoid
possible misinterpretation of the data.
Forward
Down
(a) Small obstacles (b) Wooden blocks (c) Wooden stairs
Up
fer ph
an
s a se Body leveling Figure 5: Terrains traversable by the adaptive gait.
Tr
Support phase The outdoor terrains (grass, dirt and asphalt) are very flat
and easily traversable by a default regular gait. However,
(a) Default gait (b) Adaptive gait the rough terrains shown in Fig. 5 are traversable only by
the used adaptive gait. Default gait is able to traverse only
small obstacles (cf. 5a) and this simple terrain type is used
Figure 4: Comparison of the leg trajectory using a regular to fill a gap between the flat terrains (outdoor and office
default gait and an adaptive gait. floor) and the rough terrains (blocks and stairs).
Feature Extraction for Terrain Classification with Crawling Robots 183
Terrain Dirt Asphalt Grass Office Obstacles Blocks Stairs
Dirt 62 0 0 0 0 0 0
Asphalt 1 79 0 0 0 0 0
Grass 0 0 75 0 0 0 0
Office 0 0 0 89 0 0 0
Obstacles 0 0 0 0 69 1 0
Blocks 0 0 0 0 0 21 0
Stairs 0 0 0 0 0 1 90
Table 1: Confusion matrix of 2-fold cross-validation with overall accuracy 99.4%
Each terrain class was trained from several trials with
3–7 minutes worth of data; the exact number of feature
vectors extracted from the data can be read from columns
of Table 1.
The evaluation strategy is based on the verification of
the distinguishability of the terrain using the features.
Then, we evaluate online detection of the terrain in a sep-
arate scenario, where particular types of the terrains are
altered and the robot is requested to traverse them and con-
tinuously detect the terrain. These two evaluation scenar-
ios are described in the following sections.
6.1 Distinguishability of the Terrains
Two-fold cross-validation with all datasets involved has
been used to validate whether the classifier is able to dis-
tinguish between the considered 7 terrain classes. As can Figure 6: Testing scenario consisting of small obstacles
be seen from the confusion matrix in Table 1, the overall (bottom right corner) on the office floor, followed by a pool
accuracy of 99.4% is very high even for only 2-fold cross- of wooden blocks and ending with climbing the stairs.
validation (with more folds, we can easily get 100%).
However, notice this test is based on using always data
each gait cycle and is computed from the last three gait
from the same single experiment in both training and test-
cycles worth of data. Therefore (with respect to the robot’s
ing partition, and therefore, the data are more likely refer-
length) there are long transition areas of the overlapping
ring to themselves than to a generalized model of partic-
terrains.
ular terrain class. In [7], authors achieved the same high
The transition between the office floor and the pool of
accuracy when evaluating on the same datasets that were
wooden blocks is mostly characterized as the stairs, which
used for training.
corresponds with the entry side of the pool with increasing
height of the blocks.
6.2 Terrain Classification The other transition between the blocks and the stairs
A more realistic practical scenario is based on evaluation is undefinable and can be predicted as either terrain class,
of the classification for traversing rough terrains in a sin- or a similar class (obstacles) based on the actual footholds
gle run, where undefined terrains at the overlap of the par- in the area during the experiment. We can also see that
ticular terrain types are provided. The scenario setup is there is some confusion between the dirt and the office
shown in Fig. 6 and it consists of a sequence of rough ter- floor terrain with obstacles which are both relatively flat
rains used for the learning. The robot starts from a de- and slippery for the robot.
fined position and crosses progressively few small obsta- Despite not analyzed, if another terrain class of stairs
cles, a pool of wooden blocks, and wooden stairs. This being traversed down was trained, it is highly probable that
scenario was repeated five times and the extracted fea- a transition from the blocks to the office floor would have
ture vectors were evaluated against the model previously been classified as this terrain (for the same reason as the
learned from a single-pass of the individual terrains. opposite transition mentioned above).
The predicted terrain labels from each of five runs are The main aspect of the challenging terrain traversing,
shown in Fig. 7. The class prediction is made once per which cannot be seen on the simple flat terrains, is the
184 J. Mrva, J. Faigl
Gait cycle: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32
Run #1
Run #2
Run #3
Run #4
Run #5
Legend: dirt obstacles blocks stairs grass
Figure 7: Successive predictions of terrain labels in the testing scenario.
occurrence of a foot slippage on the edge of an obstacle References
(stair) that yields in a sudden fall to a lower level and im-
pacts all legs. More slippages in a short time can lead
in a confusion in the prediction, as can be seen in Fig. 7 [1] Dudek, G., Jenkin, M.: Computational principles of mo-
where the transition between the blocks and the stairs was bile robotics. New York, NY, USA, Cambridge University
once predicted as a grass terrain. Press, 2000
The unequal length of all runs is purely dependent on [2] Zenker, S., Aksoy, E., Goldschmidt, D., Worgotter, F.,
Manoonpong, P.: Visual terrain classification for selecting
the event when the robot steps over the side edge of the
energy efficient gaits of a hexapod robot. In: IEEE/ASME
stairs and thus stops the experiment. This happens due
International Conference on Advanced Intelligent Mecha-
to the fact that the robot cannot steer and is strictly going tronics (AIM), 2013, 577–584
straight ahead.
[3] Filitchkin, P., Byl, K.: Feature-based terrain classification
Notice, it is not possible to show the ground truth for for littledog. In: IROS, 2012, 1387–1392
the predictions in Fig. 7 because the robot can spend dif- [4] Belter, D., Skrzypczyński, P.: Rough terrain mapping and
ferent number of gait cycles to get to the same point in classification for foothold selection in a walking robot.
the scenario in particular runs. Therefore, the only mea- Journal of Field Robotics 28(4) (2011), 497–528
sure we can get is to compare the results from Fig. 7 to the [5] Hoepflinger, M. A., Remy, C. D., Hutter, M., Spinello, L.,
overview of the testing scenario shown in Fig. 6. Siegwart, R.: Haptic terrain classification for legged robots.
In: ICRA, 2010, 2828–2833
[6] Schmidt, A., Walas, K.: The classification of the terrain by
a hexapod robot. In: Proceedings of the 8th International
7 Conclusion Conference on Computer Recognition Systems (CORES),
2013, 825–833
We proposed an alternative method to extract features [7] Best, G., Moghadam, P., Kottege, N., Kleeman, L.: Ter-
rain classification using a hexapod robot. In: Proceedings
from servo drives to classify terrains for a technically blind
of the Australasian Conference on Robotics and Automa-
robot traversing rough terrains. Although the proposed tion (ACRA), 2013
method simplifies the original feature extraction process,
[8] Mrva, J., Faigl, J.: Tactile sensing with servo drives feed-
the results indicate it is sufficient to distinguish evaluated back only for blind hexapod walking robot. In: Proceedings
terrain classes. Moreover, the results also indicate we can of the 10th International Workshop on Robot Motion and
employ the learned classifier in the on-line terrain classifi- Control (RoMoCo), 2015
cation in scenarios with rough terrains. [9] Kalakrishnan, M., Buchli, J., Pastor, P., Mistry, M., Schaal,
The classifier is based on the features of the robot mo- S.: Learning, planning, and control for quadruped locomo-
tion and interaction with the terrain. However, the features tion over challenging terrain. The International Journal of
of the terrain itself (e.g., slopiness, slipperiness, convex- Robotics Research 30 (2) (2011), 236–258
ity) are not analyzed directly—they may be hidden inside
[10] Belter, D.: Gait modification strategy for a six-legged robot
the SVM layer and could be addressed in the future work. walking on rough terrain. In: Proceedings of the 15th In-
ternational Conference on Climbing and Walking Robots,
Adaptive Mobile Robotics, World Scientific, A. Azad et al.
(Eds.), Singapore, 2012, 367–374
Acknowledgments
[11] Winkler, A., Havoutis, I., Bazeille, S., Ortiz, J., Focchi, M.,
Dillmann, R., Caldwell, D., Semini, C.: Path planning with
The presented work has been supported by the Czech force-based foothold adaptation and virtual model control
Science Foundation (GAČR) under research project for torque controlled quadruped robots. In: ICRA, 2014,
No. 15-09600Y. 6476–6482
Feature Extraction for Terrain Classification with Crawling Robots 185
[12] Walas, K., Belter, D.: Supporting locomotive functions of a
six-legged walking robot. International Journal of Applied
Mathematics and Computer Science 21 (2) (2011)
[13] Palmer, L., Palankar, M.: Blind hexapod walking over un-
even terrain using only local feedback. In: IEEE Interna-
tional Conference on Robotics and Biomimetics (ROBIO),
2011, 1603–1608
[14] Walas, K., Nowicki, M.: Terrain classification using laser
range finder. In: IROS, 2014, 5003–5009
[15] Kalakrishnan, M., Buchli, J., Pastor, P., Schaal, S.: Learn-
ing locomotion over rough terrain using terrain templates.
In: IROS, 2009, 167–172
[16] Walas, K.: Tactile sensing for ground classification. Journal
of Automation, Mobile Robotics & Intelligent Systems 7
(2) (2013), 18–23