In the article we present our idea for learning system made for a LEGO robot. We built a line following robot which is using four sensors to trace the line. Track trace logic is based on our original implementation written in Python language for LEGO Mindstorm and on a neural network implemented using Python as well. As a result of our work we see that using neural network might be an efective way of teaching a robot to follow a line, but not in case of LEGO EV3.
The fast pace of enrolling technology into our lives makes This article won’t be revolutionary technologically-wise,
it much easier and more available to implement robotics since there have been many attempts of implementing a
for house needs. Especially those are a great opportunity bit more complex algorithms into EV3 controlled LEGO
for young engineers to learn programming and develop robotics like fuzzy set rules [
There are many interesting applications in which neu- 3. Assumptions of the project
ral networks work with images as classifiers. In [
• a six-button interface • a black and white display
• USB port • a mini SD card reader • a speaker • four input ports and four output ports
The sensors are calibrated with a number from 0 to 100, where 0 is no light reflection (black) and 100 is the maximum value of reflected light. Due to the wear of the sensors, the data had to be normal- ized. The formula for
It includes four EV3 Color Sensors which tell the dif- the sensor 1 (see eq. 1) is similar for all sensors. For the ference between black and white and four EV3 Medium right side, a sign of the equation changes. Servo Motors which allow robot to drive. A side plan of the vehicle is shown at Fig. 2, top view at Fig. 3 and back view at Fig. 4. · (1. () − )
Our code for the machine is modular: we have started the code from a basic line follower with 4 engines (2 for each side) and 4 color detectors used as light sensors, also 2 for each side - their placement determines decisions the vehicle should make. We are going to present only right turns, as the left side related movements are just a mirror copy of the previously mentioned ones. (1) is maximum turn of the vehicle, sensor.reflection() is a value sent by the sensor and is a sum of maximum values of left-side sensors after calibration process (2). Analogue equation is for right-side sensors. = 1 + 3 (2)
Depending on the mode selected by the program, values of 1, 2, 3 and 4 are calculated or set to 0. Then SKR is a sum of all of them (eq. 3). = 1 + 2 + 3 + 4 (3)
A final engine power is determined by an equation below (eq. 4). This equation is for left first engine, the rest of equations are similar with the change of a sign for right engines.
1. = + (4) 5.2. Basis Depending on the values read from the sensors, the vehicle can choose one of 6 options: • Case1 (and-1)
If one of the internal sensors notices less than 50% of possible light reflection, external sensor is starting to be taken into account. If it notice the black line, the vehicle turns until the other internal sensor sees the line. If sensor 2 has seen the line, sensor 4 is activated and if it has seen the line, the vehicle turns right until the sensor 1 sees the black line. Analogous situation is for left turn.
has to go beyond a line and then turn toward sensors that have seen the line until an internal sensor on the other side see the line again. If sensors 1,2 and 4 have seen the line, the vehicle turns right until the sensor 1 sees the black line.
Analogous situation is for left turn. • Default
The assumption of the project was to create a neural network and train it so that the vehicle would run smoother than using the basic program. Our neural network is trained by backward propagation of errors which calculates the gradient of the error function and allow to change weights on nodes. The network (Fig. 10) has 4 neurons in an input layer, one for each of sensors and two hidden layers with 12 and 24 neurons. 21 neurons in an output layer represents percentages from 0 to 100%, every 5 percent. We create two neural networks, one for the left, and one for the right engines. A decision of neural network, a value of one of output
neurons is used in an equation below (eq. 5) to calculate percentage of maximum power of the engines which will change into appropriate turn. The equation is given for the left side. = · (5)
Where StronaL is an index of chosen output neuron multiplied by 5 to get a percentage value. mianownik depends on which neural network made the decision on the higher index.
It contains initial values of calibrated sensors, maximum power engine and maximum turn rate. It also initializes all motors and sensors of the vehicle (Fig. 11).
basing on sensor values (Fig. 12). It returnes table of results form output layout. The index of neuron witch returned the highest value is converted to percent of MAXMOC (Fig. 10).
However, before riding the route becomes possible, network have to be well trained. To do this, we used backpropagation algorithm. Training takes place outside the robot, using database created by basic program (Fig. 13). After training, weights are exported from simulation to robot.
A test route for creating a database of sensors values and for training the vehicle with neural network was made with white photographic paper and a black tape. The trace is shown at Fig. 15 below. First we needed a database with sensors values and turns calculated by the basic program. The vehicle collected data (shown at Fig. 16) which was sent to the program with neural network. Neural network has been correcting weights and learning to make appropriate decisions on the trace. The vehicle went through the trace twice and gave us the database containing almost 10000 records (Fig. 17). All of the records were normalized.
First our neural network was supposed to look diferent which is shown at Fig. 18. We wanted to create only one network with smaller hidden layers. Output layers was supposed to contain only two neurons, one for each wheel of the vehicle, which were supposed to return velocity of a motors. This method was not optimal and has been given inappropriate values. We change a concept to build a much larger neural network. The new concept is given much better results. After giving random values in nodes weights, before training the network, a sum of errors was enormous, but after training with backpropagation, errors were almost completely reduced. Below you can see the sum of errors of all neurons before and after training the network (Fig. 19 and Fig. 20). Neural network with backpropagation algorithm allowed to get really satisfying results, at least in the simulation. But calculations are too slow and the vehicle is not keeping up with making decisions. There are huge delays so that the vehicle runs mainly under the control of the basic
Despite the very promising simulations, because of the delays and hardware limitations we cannot say that our robot passed the route using artificial neural network. Program requires optimization and some changes. What is more, an EV3 system would need more RAM memory and better CPU to work properly.
Fortunately, not all of our work was in vain. During the experiments we have found out that neural network is doing much better with classifying using more neurons on output layout, than returning precise results.