Gait Genesis Through Emergent Ordering of RBF Neurons on Central Pattern Generator for Hexapod Walking Robot Jan Feber, Rudolf Szadkowski, and Jan Faigl Czech Technical University in Prague, Faculty of Electrical Engineering, Technicka 2, 166 27 Prague, Czech Republic, {feberja1|szadkrud|faiglj}@fel.cvut.cz, WWW home page: https://comrob.fel.cvut.cz Abstract: The neurally based gait controllers for multi- legged robots are designed to reproduce the plasticity ob- served in animal locomotion. In animals, gaits are regu- lated by Central Pattern Generator (CPG), a recurrent neu- ral network producing rhythmical signals prescribing each leg’s action timing, leading to coordinated motion of mul- tiple legs. The biomimetic CPG-RBF architecture, where leg motion timing is encoded by Radial Basis Function (RBF) neurons coupled with CPG, is used in recent gait controllers. However, the RBF neurons coupling is usually parameterized by the supervisor. Therefore, the RBF pa- rameters get outdated when the CPG signal’s wave-form Figure 1: On the left: schema of leg motion phase φi changes. We propose self-supervised dynamics for RBF relations. Each vertex represents the motion phase of parameters adapting to a given CPG and producing the the leg of the corresponding color on the corresponding required gait rhythm. The method orders the leg activ- anatomic position. The black oriented edges indicate re- ity with respect to inter-leg coordination rules and maps pulsive forces maintaining antiphase; the green edges indi- the activity onto CPG states. The proposed dynamics pro- cate keeping a specified distance from the other phase. The duce rhythmic control for three different hexapod gaits and colors and positioning correspond to the robot schema on adapts to the CPG parametric changes. the right. On the right: the robot schema with colored and labeled legs, corresponding to labeling in Figs. 2 and 3. The arrow indicates the direction of the robot’s movement. 1 Introduction The biomimetic approach of the gait control is adopted 6], where each leg movement corresponds to some CPG in multi-legged robotics to imitate robustness and adapt- state. The method of mapping the leg movements onto the ability observed in animal locomotion [1]. The locomo- CPG state is determined by the selected architecture of the tion is driven by a neural network that continually controls CPG-based controller. and adapts to the environment during movement. In the In this paper, we focus on the architecture where the context of the gait control, the essential part of the neu- CPG is coupled with Radial Basis Function (RBF) neu- ral network is a Central Pattern Generator (CPG) [2], re- rons, where each RBF neuron fires at a particular CPG currently connected neurons generating rhythmical signals state [7, 8]. The RBF neuron activity dependes on the dis- that drive the motion. The CPG is thus employed in many tance of the input point to the neuron’s parameter point, biomimetic multi-legged robot gait controllers [3]. i.e., the activity dependes on the radius around the fixed In CPG-based controllers, the CPG drives the repetitive parameter determining vicinity in which the input point is. gait motion. During a regular motion, the gait can be de- Hence, radial basis function neurons. scribed as a repeating sequence of leg movements, where In our method, the RBF neurons are parameterized by each movement is performed at a certain motion phase. the centers placed into the CPG’s state space such that The motion phase is a hidden state that can be inferred when the CPG state, representing the input, is near one of from sensory feedback [4], or, as in our case, tracked by the centers, the corresponding RBF peaks in the activity. the CPG. The RBF activations can then be used as motion phase en- The CPG signal is periodic, and thus the state of the coding, motion primitive trigger, or couple multiple CPGs. recurrent neural network representing the CPG creates a limit cycle, a closed trajectory in the state space. The CPG As far as we know, in all current CPG-RBF controllers, state then tracks the hidden motion phase of the gait [5, the RBF centers are set up by a supervisor. Such a prior parametrization assumes that the CPG properties remain Copyright ©2021 for this paper by its authors. Use permitted under unchanged during the locomotion. Due to the assumption Creative Commons License Attribution 4.0 International (CC BY 4.0). of static properties, the CPG cannot be optimized (e.g., by Righetti’s learning rule [9]) nor the entraining waveform Table 1: Phase offset of consecutive legs for corresponding can be changed, which poses a limitation to the adaptabil- gait pattern ity of the system. In this work, we propose a dynamic rule for RBF cen- gait pattern: tripod transition wave ters self-organization that generates gaits. The proposed phase offset ∆φ : π 2π/3 π/3 method decomposes the RBF centers organization into two tasks. First, the organization of leg movements in phase space that is consistent with Inter-leg Coordination Rules to contain at least one CPG per leg as each leg needs its (ICRs) [10] and given phase offset of consecutive legs’ ac- swing/stance timing. In multi CPG networks, the different tivity [11], providing phase relations within legs actions gaits are implemented by learning the correct connectivity (see Fig. 1). The second task is the mapping of the or- between CPGs, which might be difficult as the interaction ganized leg movements onto CPG states. Both tasks are between CPGs is generally non-linear. The networks of processed continually by the proposed dynamic rules and multiple CPGs can be avoided by generalizing the binary- organize the RBF centers along the CPG’s limit cycle, so phase into a multi-phase approach provided by CPG-RBF the resulting rhythm produces a corresponding gait. architecture. The method is implemented on a hexapod walking robot The CPG-RBF architecture has been recently proposed in the simulated environment, where we show that the pro- in [19], and it provides a straightforward representation posed solution generates multiple gaits consistent with the of the map between CPG states and motion phase using ICRs. We also demonstrate the adaptive capabilities dur- the RBF layer. The straightforward motion phase repre- ing change of CPG properties, where the proposed solu- sentation is utilized in [8], where the RBF output is used tion adapts to the changes. to learn the amplitude control with reinforcement learning mechanisms. The RBF neurons themselves can be trained 2 Related Work to adapt general CPG with the periodic Grossberg rule. It is presented in [7], where the learning rule is used to cou- CPG-based controllers can be found in wearable and ple two CPG layers specialized in sensory estimation and legged robotics. The controllers can consist of two sub- motor phase control. modules: amplitude control, providing the magnitude of In the context of gait generation, each of the three ap- actuation, and phase control, providing the actuation tim- proaches has a different way of parameterizing the gait. ing [12, 13]. The CPG is involved in the phase control, While there are already proposed methods for gait learn- where the CPG state represents the motion phase. One of ing for continuous and binary mapping approaches [14, the distinguishing features of the CPG-based controllers is 13, 12, 18], there is none for the CPG-RBF architecture. how the CPG state is mapped into the movement phase. The aforementioned CPG-RBF controllers have RBF cen- Three types of CPG-to-motion mapping can be distin- ters set by a supervisor, limiting the system to static CPG’s guished: continuous, binary-phase switch, and its gener- properties. Since we aim to increase the adaptability capa- alization multi-phase switch, characterized as follows. bilities of the CPG-RBF controllers, we propose a self- Continuous mapping reshapes the CPG signal into the organizing method for the RBF centers that generate the motion command with continuous function. In [14], where desired gait patterns for the hexapod walking robot. the CPG signal is empirically reshaped into joint angle command. The authors of [15] interpret the CPG output as a foot tip position that is transformed into joint angles by 3 Problem Statement inverse kinematics. In [16], the CPG output is directly fed as an input of a PD controller that transforms the CPG out- The gait phase controller provides the timing for each i- put into angles of leg joints. Continuous maps depend on th leg to coordinate the leg movement. We focus on the the wave-form of the CPG, which might limit the system movement patterns that are consistent with three inter-leg adaptation, as the wave-form changes non-linearly with coordination rules (ICRs) observed from hexapod insect changing CPG parameters. gaits [10]: (i) while a leg is lifted-off, suppress the lift-off In contrast, using the CPG as a binary switch makes of the consecutive leg; (ii) if the leg touches the ground, the system independent of the exact shape of the wave- initiate the lift-off of the consecutive leg; (iii) do not lift form and rather uses the CPG as a timing generator. The off the contralateral legs at the same time. The swing du- CPG can be used for switching between the stance and ration given by the phase offset ∆φ [11] then determines swing leg motion modes, where each mode has its con- the exact motion pattern (see Tab. 1) as it is depicted in trol rules [12, 17]. The switching approach can be com- Figs. 2 and 3, where the motion states of the feasible gaits bined with a continuous mapping approach as in [18], are visualized with the color labeling as in Fig. 1. where two different CPG output shapers are defined for For the CPG-RBF phase controller, we encode the coor- the stance and swing motion modes, respectively. Using dinated timing by coupling RBF neurons to a single CPG, the CPG as a switch between stance and swing leg mo- where each i-th RBF neuron drives the corresponding i-th tions is straightforward; however, it forces the architecture leg. Generally, the CPG state evolution, y (t) ∈ RD , can For a general CPG, the centers w i cannot be placed a pri- ory, as the shape of the limit cycle y 0 is not known. More- over, the limit cycle can change its shape dynamically with changing parametrization of CPG dynamics f (·) or differ- ent CPG input c(t). Thus the centers w i of RBF neurons need to be dynamically adjusted to drive the locomotion according to the coordination rules and given phase offset ∆φ . Figure 2: Visualization of legs’ activity during the repeat- 4 Method ing phase for the desired gait patterns. The color bar rep- resents the duration of the corresponding leg’s swing ac- We propose dynamic rules that form feasible gait patterns tion. Note that the ordering and relative distance (phase by organizing the RBF centers w i on the CPG’s limit cycle offset) of actions are important, not their particular posi- y 0 while respecting the ICRs and maintaining the given tion within the phase. The gaits have more valid ordering phase offset ∆φ . The task is decoupled into two subtasks: options, and the figure represents only one of the possible (i) order the lift-offs of each i-th leg into a sequence and orderings. (ii) map the sequence onto the CPG limit cycle. The i-th RBF center dynamics are given by the periodic Grossberg rule [7] wi = (yy(t) −w ẇ wi (t)) pi (t), (1) that pushes the center w i towards the states y (t) at which the target signal pi (t) ∈ [0, 1] is nonzero. In the previous work [7], the target signal is given by a supervisor. We introduce a new method for forming the target signal  pi (t) = ϕ φ̂ (t); φi (t) (2) where φi ∈ [0, 2π) is the phase of the i-th leg lift-off that Figure 3: Example visualization of motion phases φi (col- determines the sequence of lift-offs, and φ̂ ∈ [0, 2π) that orful dots) correctly ordered within the phase (black circle) maps the [0, 2π) phases onto the limit cycle y 0 . As the to produce the desired gait patterns. The interval [0, 2π) phase is within the circular space S1 , we define the circle representing phase is depicted as a unit circle using cos(φ ) metric as ||φ − φ 0 || = min(|φ − φ 0 |, 2π − |φ − φ 0 |), which for the horizontal axis and sin(φ ) for the vertical axis for gives the closest distance between two phases. For both φ ∈ [0, 2π). Note that the ordering and relative distance variables, φi and φ̂ , we present their dynamics in the fol- (phase offset) of actions are important, not their particu- lowing sections. lar position within the phase. The gaits have more valid ordering options and the figure represents only one of the 4.1 Organizing Phase of Legs Activity possible orderings. The motion phases for simultaneously activated legs in tripod gait are overlapping. The motion start phases φi of each i-th leg must be ordered within the interval [0, 2π), where the ordering has to be consistent with the ICRs, and phase offset ∆φ . Both con- be modeled by the differential equation ẏy = f (yy(t), c(t)), straints can be defined as distances between phases φi : (i) where the dot notation represents differential with respect front (hind) leg i is shifted by ∆φ with respect to ipsilateral to time, and the system f contains a limit cycle attrac- middle leg j, ||φi − φ j || = ∆φ ; (ii) contralateral legs i and tor, y 0 ⊂ RD : a looped trajectory to which all neighboring j are in anti-phase ||φi − φ j || = π, see Fig. 1. states converge. Thus, after the convergence, the CPG is We present the following dynamics to order randomly T -periodic, y (t) = y (t + T ), and the CPG generates a peri- initialized phases φi odic signal. The CPG state  is an input for  the RBF neuron 2 activation ϕ(yy;ww) = exp −ψ kyy −w wk2 , where the cen- 6 φ̇i (t) = ∑ αi j (∆i j + sign (φi − φ j ) ||φi − φ j ||) , (3) ter w ∈ RD and hyperparameter ψ determines the timing j and duration of activation, respectively. The RBF neu- ron peaks when the CPG state is close to the RBF center, where ∆i j ∈ [−π, π] parameterize the target offset between y (t) ≈ w (t), generating periodic peaks. For each i-th leg, φi and φ j , and αi j ∈ IR+ weighs the relation influence. The there is a corresponding RBF neuron with center w i , which relationships between contralateral legs are parameterized peaks trigger the lift-off. as ∆2,1 = ∆4,3 = ∆6,5 = −π and ∆1,2 = ∆3,4 = ∆5,6 = π. The relation between ipsilateral legs are ∆2,6 = ∆1,5 = ∆φ κ is empirically set to 0.02. The obtained CPG frequency and ∆4,6 = ∆3,5 = −∆φ . The above-defined relations have is used to estimate the CPG phase set αi j = 1 while the rest is turned off by α j0 i0 = 0. The variable φi serves as a parameter of the target signal (2), φ̂ (t) = 2π (1 − s(t)) . (10) the next input is the phase estimation. After the convergence of the CPG estimation φ̂ (t) and lift-off ordering φi , the variables pi (t) of (2) produce the 4.2 Mapping the Legs Activity onto the Limit Cycle target signal that orders the RBF centers of each i-th leg Using Phase Estimation to produce the gait pattern; which is demonstrated in the following section. The target signal (2) should have the same periodicity as the CPG; however, the periodicity of a general CPG nor its frequency cannot be obtained analytically. However, 5 Results the frequency is needed to modulate the phase of the target signal φ̂ (t), and thus the frequency must be learned. We The feasibility of the proposed method has been vali- propose to dynamically learn the CPG frequency by cou- dated by experimental deployment in several scenarios to pling the CPG with the pivot RBF neuron and measuring demonstrate the adaptability of the developed solution. We the RBF activity period to determine the CPG’s frequency. used Euler’s method to run the dynamic system consist- First, the randomly initialized pivot RBF center w sig ∈ ing of the proposed equations, running for 20000 itera- IRD must get close to the limit cycle. The pivot center tions with the step size 0.01. The correctness of the gener- w sig (t) is attracted to the CPG state y (t) if the CPG state is ated rhythm for different gaits is demonstrated using mod- within ε-neighborhood of the center ified signals, obtained from the RBF neuron activations   ϕ(yy,w wi ), to trigger the predefined swing movement, fol- 2 wsig ε −2 y (t) −w  wsig = 1 − y (t) −w ẇ wsig µ(t) (4) lowed by a predefined stance movement for the robot’s ( legs in CoppeliaSim1 simulator. The methods have been 1 y (t) −w wsig < ε implemented in Python 3 and a hexapod model Phan- µ(t) = . (5) 0 otherwise tomX MK-III has been used to run the simulations. The system’s adaptability has been evaluated for two different The neighborhood radius ε itself is dynamic, making the CPG models. ε-neighborhood expand when the CPG state is outside and The first CPG model is Matsuoka oscillator given by contract when the CPG state is closer than half the radius: 3 τ1 v̇1 = h(u1 ) − v1 , (11) wsig ε −1 − 0.5 γ(t)  ε̇ = clip y (t) −w (6) ( τ1 v̇2 = h(u2 ) − v2 , (12) wsig ε −1 < 0.5  σ1 clip y (t) −w τ2 u̇1 = −u1 − h(u2 )β1 − v1 β2 + 1, (13) γ(t) = 2 , (7) σ2 ε otherwise τ2 u̇2 = −u2 − h(u1 )β1 − v2 β2 + 1, (14) where clip(x) = min(1, max(0, x)) and hyperparameters h(x) := max(x, 0), (15) σ1 = 80, σ2 = 2 are set empirically. As the pivot center w sig (t) converges to the limit cycle, the pivot RBF activa- where the hyperparameters are set to τ1 = 0.5; τ2 = tion psig = ϕ(yy(t);wwsig ) produces T -periodic pulses. 0.25; β1 = β2 = 2.5, and the function h(x) represents the From the pivot RBF activation psig , we extract the fre- rectifier (i.e., ReLU function). For the Matsuoka oscillator quency of the CPG by adjusting descend of the variable the CPG’s state is y = (u1 , u2 , v1 , v2 ) ∈ IR4 . The second CPG is Van der Pol’s oscillator (VdP), given by ( (1 − s)ξ psig (t) ≈ 1 ṡ = , (8) −a(t) otherwise u̇ = v, (16) 2  where ξ is large enough to reset s to 1 when the pivot v̇ = ζ 1 − u v − u, (17) activation peaks (psig (t) ≈ 1), and a ∈ IR+ determines the where, ζ is a parameter indicating the strength of damping slope of the descend. If the slope of a has such a value and the CPG’s state is y = (u, v) ∈ IR2 . that descends from s(t1 ) = 1 to s(t1 + T ) = 0, then a is the CPG frequency. Thus slope a is adjusted as follows ( 5.1 Adaptability Experiments a(t− ) + κs(t− ) psig (t) ≈ 1 a(t+ ) := (9) The system’s adaptability to different limit cycle shapes is a(t− ) otherwise, demonstrated by learning the transition gait in four differ- ent scenarios. which increases the frequency if s(t1 + T ) > 0 and de- creases the frequency if s(t1 +T ) < 0. The hyperparameter 1 https://www.coppeliarobotics.com 1. Using unperturbed Matsuoka oscillator as the CPG model. 2. Using Matsuoka oscillator synchronized with four other coupled Matsuoka oscillators to demonstrate the method’s adaptability to small perturbations. 3. Using the VdP with ζ = 3 to demonstrate the usage on a different oscillator. 4. Using the VdP with the changed parameter ζ = 1 to show the adaptability to change of the oscillator pa- Figure 5: The progression of the learned frequency a for rameter. perturbed (green) and unperturbed (magenta) Matsuoka oscillator, and the VdP oscillator with the parameter ζ = 3 In scenario 2. the motion phases φi , their respective (yellow) and ζ = 1 (cyan). The initialization value of a is RBF centers w i , and the parameter a are initialized to −0.4 for the VdP and unperturbed Matsuoka oscillators. the transition gait values, which were previously success- For the perturbed Matsuoka oscillator, a is initialized as fully learned with the unperturbed CPG. The RBF neurons the final value of a of the unperturbed one (i.e., the value provide the rhythmical input, producing the signal psig (t) of the magenta line at the 20000-th iteration is the initial based on the position of the center w sig with the dynamic value of the green line). All the cases converge to a stable vicinity. value. The values s(t) dependent on a are visualized in Fig. 7. Figure 6: The progression of the learned frequency a for unperturbed Matsuoka oscillator for four different initial- Figure 4: Plot of w sig RBF center’s dynamics based on ization values of a. The plot shows that the frequency a its dynamic vicinity given by the dynamic radius ε. The correctly converges to the same value in all the cases. upper plots show the center’s movement (cyan path from green cross to red cross) in the CPG’s state space towards the CPG’s limit cycle (black closed shape). The lower successfully converges. The progression and convergence plots show the progression of the center’s respective ε. of s(t), estimating the CPG’s phase growth, and the modu- The unperturbed Matsuoka oscillator is shown on the left, lated learning signal psig (t) are presented in Fig. 7, where and the VdP is on the right. A darker shade of the cyan the fact that a converges can be seen in convergence to and black colors signalizes that more time is spent at the zero of local minimum values, marked by the orange line. corresponding place. As mentioned in Section 4, if learned correctly, the value of s(t) declines from one to zero during each period, i.e., The CPG’s phase is estimated as φ̂ (t) to map the cen- at the psig (t) signal pulse occurs, s(t) equals zero and it is ters w i ∈ IRD on the CPG’s limit cycle, by learning the reset to one. frequency a, (9), and pivot center w sig , (4). A showcase Based on the estimated phase, the centers are organized of w sig center’s attraction to the CPG’s limit cycle together around the CPGs’ limit cycles, as demonstrated by the with the progress of its dynamic vicinity radius ε, (6), is transition gait for Matsuoka and VdP oscillators shown in shown in Fig. 4 for both types of the oscillators with dif- Fig. 8, where centers are organized around the CPGs limit fering limit cycle shapes of a prior unknown shape. cycles of various a prior unknown shapes. The frequency a is learned based on the signal generated by the RBF neuron corresponding to the center w sig . The 5.2 Different Gait Patterns Experiment learning process of a is shown in Fig. 5. In Fig. 6, we pro- vide a learning process of a for Matsuoka oscillator with The ability to generate different gait patterns is demon- differently initialized a. In all the cases, the frequency a strated using Matsuoka neural oscillator. The motion phases φi ∈ [0, 2π) interact with each other according to the given phase offset ∆φ of two consecutive leg’s actions, and to ICRs, as shown in Fig. 9 for all three gait patterns. The correctness of the process can be observed by compar- Figure 7: Results of learning s(t) (blue), estimating the phase growth by the proposed relation (10). If learned Figure 9: On the left: the first 300 iterations of φi self- correctly, s(t) equals to zero at the moment of psig (t) sig- organizing motion phases, where the vertical axis repre- nal’s (gray line) pulse occurrence, which resets the value sents the value of the motion phase at each iteration given of s(t) to one. Hence, the minimum of s(t) in each period by the horizontal axis. On the right: the final positions at should equal to zero, if learned correctly. The plot shows the 20000-th iteration of the organized motion phases, de- that the values of the periods’ minimums (orange line) are picted as a unit circle using cos(φ ) for horizontal axis and correctly converging to zero. The plots correspond to Mat- sin(φ ) for vertical axis for φ ∈ [0, 2π) (compare with de- suoka unperturbed and perturbed oscillator and to the VdP sired patterns in Fig. 2). The plots correspond to the tripod, with ζ = 3 and ζ = 1 from top to down, respectively. The transition, and wave gaits from top to down, respectively. decline of s(t) depends on the value of the frequency a(t) Note that motion phases for simultaneously activated legs shown in Fig. 5. The signal psig (t) is modulated for im- in the tripod gait are overlapping. proved visibility. ing the results with schema,2 describing the gait patterns in Fig. 3. The process of ordering the motion phases within the phase is independent of the used CPG model. The successful mapping of the RBF centers’ for three different gait patterns with the use of Matsuoka oscillator is shown in Fig. 10. The signals produced via the centers’ RBF neurons producing gait pattern rhythm are visualized in Fig. 11. The RBF signals are used in the CoppeliaSim simulator to make the hexapod robot walking, as shown by snap- shots of one gait cycle for each of the given gait patterns in Fig. 12. The experiments demonstrated that the proposed mech- anism successfully produced rhythm for the desired gait patterns on both the CPG models with different dimen- Figure 8: The final state of the RBF centers (colorful dots) sionality and different shape of their limit cycles. placed onto the limit cycles (closed black curve) of unper- turbed (upper left) and perturbed (upper right) Matsuoka oscillator and the VdP with parameter ζ = 3 (lower left) 2 Note that important are the relative positions, i.e., ordering of the and ζ = 1 (lower right). motion phases with correct distance (phase offset) between them. Figure 12: The showcase of one gait cycle of each of the three gait patterns (tripod, transition, and wave) sim- ulated in CoppeliaSim simulator. The robot moves from left to right. The timestamps on the gray background are in format seconds:frames, where each second represents 25 frames. The time 00:00 marks the start of the gait cy- cle, beginning with the hind right leg’s swing. 5.3 Discussion Figure 10: The plot shows the progress and final state The current CPG-RBF controllers require setting the gait- of the RBF centers organizing along the limit cycle (black pattern-determining RBF neurons parameters by a super- closed shape) of Matsuoka oscillator to produce the de- visor, assuming that the CPG produces a signal of un- sired gait patterns. The centers’ movement (colorful paths changing wave-form and frequency. However, this as- from green crosses to red crosses) in the CPG’s state space sumption limits the architecture’s ability to adapt to chang- are shown on the left-hand side. A darker shade of the col- ing CPG parameters, enabling higher frequency (thus ors signalizes more time spend in that place. The centers’ faster movement) or adaptation after a change of the syn- final states (the colorful dots) are shown on the CPG’s limit chronizing signal. Our method enables the change of the cycle on the right-hand side. The gait patterns are tripod, CPG parameters, and therefore improves the adaptability transition, and wave gaits from top to down, respectively. to evolving conditions for the CPG-RBF architectures. The rhythm invoked by the centers’ respective RBF neu- The gait pattern rhythm is generated by correctly or- rons producing the gaits is presented in Fig. 11. Note that dered RBF centers w i ∈ IRD (see Figs. 8 and 10), corre- the centers for simultaneously activated legs in the tripod sponding to legs actions, around the CPG’s limit cycle. gait are overlapping. The centers w i produce signals via their respective RBF neurons if the CPG’s state is close enough to the corre- sponding center. The produced signal provides a rhythm for the required gait pattern if the centers’ ordering along the limit cycle respects ICRs and the given phase offset ∆φ of consecutive legs, determining the required gait pattern. As shown in Fig. 12, we successfully produced three de- sired gait patterns for the hexapod walking robot, tripod, transition, and wave gaits, described in Figs. 2 and 3. The model learns parameters which have real-wolrd meaning. The phases φi represent the respective leg’s swing phase start within the walking cycle. The placing Figure 11: The rhythm invoked by the RBF neurons cor- of the centers w i around the limit cycle represents the re- responding to the RBF centers from Fig. 10 at the end of spective leg’s swing phase start within the repeating walk- the learning process. The plots show one period of the pro- ing cycle. Slope a represents the frequency of the used duced tripod, transition, and wave gaits, from left to right, CPG. The pivot center w sig marks the start of the walking respectively, using unperturbed Matsuoka oscillator. Note cycle on the CPG limit cycle. Hence, the learning process that signals for simultaneously activated legs in tripod gait is explainable, which is an advantage in comparison with are overlapping. The snapshot of the signals’ use in the black-box approaches. simulation is shown in Fig. 12. The current method does not enable the generation of gait rhythm for different numbers of legs than six without further modifications. In our future work, we would like to [4] K. W. Wait and M. Goldfarb, “A biologically inspired ap- explore the ICRs possibilities in automatically generating proach to the coordination of hexapedal gait,” in IEEE gait patterns for any number of legs. Robots with differing International Conference on Robotics and Automation numbers of legs exist and malfunctions of the robot are (ICRA), 2007, pp. 275–280. also possible, requiring to learn to walk with damaged or [5] T. Yan, A. Parri, V. R. Garate, M. Cempini, R. Ronsse, and missing limbs while deployed on a mission. N. Vitiello, “An oscillator-based smooth real-time estimate of gait phase for wearable robotics,” Autonomous Robots, vol. 41, no. 3, pp. 759–774, 2017. 6 Conclusion [6] C. Maufroy, H. Kimura, and K. Takase, “Towards a gen- eral neural controller for quadrupedal locomotion,” Neural Networks, vol. 21, no. 4, pp. 667–681, 2008. In this work, we propose and test self-supervised dynam- [7] R. Szadkowski and J. Faigl, “Neurodynamic sensory-motor ics for organizing the RBF centers producing rhythm for phase binding for multi-legged walking robots,” in Inter- required gait patterns. The method improves CPG-RBF national Joint Conference on Neural Networks (IJCNN), controllers’ gait-generating adaptability towards a change 2020, pp. 1–8. of CPG properties. The method decouples the gait rhythm [8] M. Thor, T. Kulvicius, and P. Manoonpong, “Generic neu- generating problem into two tasks, the legs activity order- ral locomotion control framework for legged robots,” IEEE ing, and the CPG phase estimation, leading to mapping Transactions on Neural Networks and Learning Systems, the legs’ activity ordering onto CPG’s states. The ordering pp. 1–13, 2020. of legs’ activity within the phase is driven by biomimetic [9] L. Righetti, J. Buchli, and A. J. Ijspeert, “Dynamic heb- inter-leg coordination rules and given phase offset of con- bian learning in adaptive frequency oscillators,” Physica D: secutive legs’ activity, determining the required gait pat- Nonlinear Phenomena, vol. 216, no. 2, pp. 269–281, 2006. tern. The phase estimation is based on estimating the [10] V. Dürr, J. Schmitz, and H. Cruse, “Behaviour-based mod- phase growth (phase angular velocity) from the signal with elling of hexapod locomotion: linking biology and tech- the period equal to the CPG’s period. Combining the pro- nical application,” Arthropod Structure & Development, posed mechanisms enables mapping the RBF centers, cor- vol. 33, no. 3, pp. 237–250, 2004, arthropod Locomo- responding to ordered actions, onto CPG’s limit cycle. The tion Systems: from Biological Materials and Systems to phase controller produces the rhythm for three desired gait Robotics. patterns, tripod, transition, and wave gaits. [11] W. Chen, G. Ren, J. Zhang, and J. Wang, “Smooth transi- We demonstrate the correct functionality of the pro- tion between different gaits of a hexapod robot via a cen- posed method, including showcase from CoppeliaSim tral pattern generators algorithm,” Journal of Intelligent & Robotic Systems, vol. 67, no. 3, pp. 255–270, 2012. simulator, where the generated gait pattern rhythms are used to invoke the movement of the simulated hexapod [12] H. Chung, C. Hou, and S. Hsu, “A cpg-inspired controller for a hexapod robot with adaptive walking,” in CACS Inter- walking robot. The results are demonstrated for two dif- national Automatic Control Conference (CACS), 2014, pp. ferent CPG models, Matsuoka neural oscillator with/with- 117–121. out a rhythmical input from other coupled CPGs, and Van [13] H. Yu, W. Guo, J. Deng, M. Li, and H. Cai, “A cpg-based der Pol’s oscillator with two different parameter settings. locomotion control architecture for hexapod robot,” in In our future work, we aim to extend the model to gener- IEEE/RSJ International Conference on Intelligent Robots ate gait patterns for robots with differing numbers of legs. and Systems, 2013, pp. 5615–5621. [14] L. Xu, W. Liu, Z. Wang, and W. Xu, “Gait planning method Acknowledgments – This work has been supported by of a hexapod robot based on the central pattern generators: the Czech Science Foundation (GAČR) under research Simulation and experiment,” in IEEE International Con- project No. 21-33041J. ference on Robotics and Biomimetics (ROBIO), 2013, pp. 698–703. [15] G. Zhong, L. Chen, Z. Jiao, J. Li, and H. Deng, “Loco- References motion control and gait planning of a novel hexapod robot using biomimetic neurons,” IEEE Transactions on Control [1] A. Ayali, A. Borgmann, A. Büschges, E. Couzin-Fuchs, Systems Technology, vol. 26, no. 2, pp. 624–636, 2018. S. Daun-Gruhn, and P. Holmes, “The comparative investi- [16] W. Ouyang, H. Chi, J. Pang, W. Liang, and Q. Ren, “Adap- gation of the stick insect and cockroach models in the study tive locomotion control of a hexapod robot via bio-inspired of insect locomotion,” Current Opinion in Insect Science, learning,” Frontiers in Neurorobotics, vol. 15, p. 1, 2021. vol. 12, pp. 1–10, 2015. [17] Y. Fukuoka, H. Kimura, Y. Hada, and K. Takase, “Adaptive [2] A. J. Ijspeert, “Central pattern generators for locomotion dynamic walking of a quadruped robot ’tekken’ on irreg- control in animals and robots: A review,” Neural Networks, ular terrain using a neural system model,” in IEEE Inter- vol. 21, no. 4, pp. 642–653, 2008. national Conference on Robotics and Automation (ICRA), [3] J. Yu, M. Tan, J. Chen, and J. Zhang, “A survey on cpg- vol. 2, 2003, pp. 2037–2042. inspired control models and system implementation,” IEEE [18] L. Bai, H. Hu, X. Chen, Y. Sun, C. Ma, and Y. Zhong, Transactions on Neural Networks and Learning Systems, “Cpg-based gait generation of the curved-leg hexapod vol. 25, no. 3, pp. 441–456, 2014. robot with smooth gait transition,” Sensors, vol. 19, no. 17, 2019. [19] M. Pitchai, X. Xiong, M. Thor, P. Billeschou, P. L. Mailän- der, B. Leung, T. Kulvicius, and P. Manoonpong, “Cpg driven rbf network control with reinforcement learning for gait optimization of a dung beetle-like robot,” in Artificial Neural Networks and Machine Learning – ICANN 2019: Theoretical Neural Computation, I. V. Tetko, V. Kůrková, P. Karpov, and F. Theis, Eds. Cham: Springer Interna- tional Publishing, 2019, pp. 698–710.