=Paper=
{{Paper
|id=Vol-2845/Paper_4
|storemode=property
|title=Identification of Rolling Stock of Railways Based on Multi-Projection Image Processing Methods
|pdfUrl=https://ceur-ws.org/Vol-2845/Paper_4.pdf
|volume=Vol-2845
|authors=Stepan Bilan
|dblpUrl=https://dblp.org/rec/conf/iti2/Bilan20
}}
==Identification of Rolling Stock of Railways Based on Multi-Projection Image Processing Methods==
https://ceur-ws.org/Vol-2845/Paper_4.pdf
Identification of Rolling Stock of Railways Based on Multi-
Projection Image Processing Methods
Stepan Bilana,b
a. State University of Infrastructure and Technology, Kyrylivska, 9, Kyiv, 04071, Ukraine
b. Taras Shevchenko National University of Kyiv, Volodymyrska Street, 60, Kyiv, 01033, Ukraine
Abstract
The paper considers the method for identifying railway rolling stock based on the parallel
shift technology and Radon transformation using cellular automata. The main stages of the
method are considered, which consist of: fixing a moving object, extracting an identifier
image, removing noise, extracting characteristic features and comparing with a standard. To
fix the rolling stock, the selection of pixels is used, which have changed their properties on
adjacent frames, taking into account the sensitivity and speed of movement. The selected
pixels are used to select a rectangular window containing a license plate. To remove noise in
the selected image, the parallel shift technology and the Radon transform, implemented on
cellular automata with a hexagonal covering, are used. The method is implemented for
images subject to various distortions that affect the identification result.
Keywords 1
Image, identification, moving object, cellular automata, parallel shift technology, Radon
transform
1. Introduction
In modern intelligent systems, which are aimed at automating various processes in the transport
industry, there is a problem of identifying objects, both moving and stationary. Identification is a
partial case of object recognition. The identification process is based on the decision to classify the
image (image of the object) based on a comparison of its characteristic features (CF) with pre-known
CF reference images.
Identification is based on obtaining a model of an object or system based on the results obtained in
their experimental study. In modern conditions, it is possible to build a number of models for any
physical object. In this regard, an important task is to choose the optimal model that gives the best
result in identifying the object. The model is also selected according to pre-established criteria.
For effective identification of railway cars, a special numbering is used in the form of eight-digit
numbers, which are painted on the sides of the cars. Images of such license plates are read by special
optoelectronic sensors and then electronically are transmitted to a computer system for image
processing and recognition. In this case, the distances from the video sensors to the numbering license
plate can be located both at a fixed and at an arbitrary distance. The problem of automatic
identification is relevant, as it allows automating the structure of the tracking system for moving
objects of railway transport.
This paper solves the problem of automatic identification of rolling stock in real time based on the
use of new technologies based on the parallel shift technology, cellular automata and Radon
transformation [1,2], which allow high-precision identification of rolling stock on railway transport.
Also, the tasks of effective image preparation are solved by removing noise and searching for
distortions of the resulting images.
IT&I-2020 Information Technology and Interactions, December 02–03, 2020, KNU Taras Shevchenko, Kyiv, Ukraine
EMAIL:bstepan@ukr.net
ORCID: 0000-0002-2978-5556
©️ 2020 Copyright for this paper by its authors.
Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).
CEUR Workshop Proceedings (CEUR-WS.org)
33
2. Existing methods and means of solving the identification problem
To identify moving objects of railway transport, a large number of methods and means are used,
which are implemented on RFID technologies [3-5] and technologies based on transformation an
optical signal into an electrical [6-16].
Among modern systems for identification of rolling stock, the following can be distinguished:
«КАУ-В» (Ukraine);
«ARSCIS» (Russia);
«SL-Traffic» (Russia);
«CarFlow»( Russia);
SecurOS Transit (Ukraine)
To implement RFID technologies, the following are used: radio frequency tags, special readers,
inductive sensors and software. RFID - technology allows realizing high physical reliability and
contactless reading of carriage numbers at a distance of up to several meters at high speed and
depends little on the state of the environment. However, RFID technology has a number of
disadvantages, which are the effect of metal and conductive surfaces, the possibility of simultaneous
superposition of several transponders, the influence of electromagnetic fields, high cost, and also the
effect on human health.
The use of the optoelectronic method and means of control is a promising way to solve the
problem of informatization and control of transportation in railway transport. Optical - electronic
method has a number of advantage:
relatively low cost;
ease of operation, maintenance and upgrading;
flexibility of the system due to the ability to adapt algorithms and software for various objects
of implementation.
The principle of optical - electronic transformation of optical pictures is used in almost all
countries with a developed railway infrastructure and on its basis a lot of software and hardware
systems have already been created that have high performance. At the same time, in all existing
commercial projects, algorithms and software for processing images of numbers are practically not
disclosed. They are used mainly for accounting of goods, but they are little used on railway sections
in real conditions.
3. Identification methodology
As a rule, railway cars have identifiers in the form of a decimal number printed on the side surface.
In this case, the images of identifiers are located at certain levels and occupy a certain area in
advance. In the process of movement and the influence of weather conditions, the images of numbers
on the side surface of the carriage are distorted.
The implementation of methods for identifying license plate images involves preliminary
preparation of initial images to form a vector of characteristic features. The general structure of image
identification is considered in [1, 17], in which the system consists of a memory block, a block of
standards and a block of preliminary preparation. The system can operate in the following modes:
learning mode;
Identification mode;
Mode of combination of learning and identification.
The third mode assumes that learning is carried out in the identification mode if the input image
cannot match one of the standards in the memory of the standards. In this case, the unidentified image
is assigned an identifier by the operator if the number plate image is previously known. In this mode,
the generated vector of characteristic features is entered into the memory of standards along with the
identifier.
An algorithm consisting of four steps is used to identify moving objects of railway transport:
1. Selection of moving objects on the video image;
2. Selection of a field that outlines the number on a moving object in the input image;
34
3. Segmentation of numeric identifier characters;
4. Recognition of selected characters of the image of the identifier.
Selection of moving objects is carried out using several algorithms:
Algorithm for overlaying several adjacent frames of the video stream;
Algorithm for determining the difference between the states of pixels of two adjacent video
frames.
This work uses an algorithm for determining the difference between the states of two pixels [3].
The algorithm is a pixel-by-pixel comparison of two consecutive frames of the video stream. Pixels
that have changed their color code to a certain sensitivity threshold are considered to be those in
which motion is recorded. All others are recognized as belonging to the background area. This
method is the easiest to implement and is suitable for the case of a still video camera. A fixed video
camera is used to implement the method.
Also, the video camera is positioned so that the moving object is fixed against a solid background
to facilitate image processing. To solve this problem, the video sensor is installed perpendicular to the
movement of railway cars. In addition, the camcorder is located at short distances from the subject,
which improves the quality of the resulting images. If it is not possible to realize a solid background,
then rather complex methods of selecting moving objects are used, which take into account the
presence of other small moving objects. The selection of the image of the identifier sign is
implemented by selecting a rectangular area covering the numeric identifier. Algorithms are applied
that implement the selection of a small area covering the image of the identifier.
An example of selecting a moving object on Fig. 1 is shown.
Figure 1: An example of selecting a moving object (ball) based on the algorithm for determining the
difference between the states of two pixels
This shows the entire moving object (ball), which is completely allocated, since the camera is
located far from it. This shooting video uses uniform light background, which gives a clear image,
and efficient allocation of the moving object. However, the background can be non-uniform, which
leads to false selection of objects, since there can also be movement in the background of a moving
object, which is fixed by the selection system. To eliminate such situations, a simple approach is used,
which consists in the near position of the video camera. If the camera is placed closer to the object,
then the background is actually the moving object itself, and the movement is visible only for the
applied images of numerical signs (Fig. 2).
Here, a smaller area of the image is allocated and, accordingly, the algorithms for clearly
distinguishing the numbering plate image are simplified. However, different surfaces are used for
each moving object, which may not always give the desired result. At different sensitivity thresholds
and at different grayscale images used in calculations, different selection results are obtained (Fig. 3).
For the video sequence shown in Figure 2, 60% sensitivity and 40 grayscale were used.
35
Figure 2: An example of a highlighted moving number plate against the background of a carriage
Figure 3: Examples of dedicated driving numbering plate for different sensitivities and grayscale
In Fig. 3, the first number represents the sensitivity, and the second indicates the number of shades
(gradations) of gray used in the calculations. As can be seen from Fig. 3, the quality and size of the
selected image area is affected by the sensitivity. For a given image, the smallest area is defined as a
sensitivity of 70% - 80%.
As mentioned earlier, the size of the selected window with the identifier image is affected by the
number of selected pixels that the rectangular selection area covers. For the example shown in Figure
3 shows the dependence of the selected pixels on the sensitivity.
36
Figure 4: Graphical dependence of the change in the number of selected pixels on the used
sensitivity
The graph (Fig. 4) is presented for five video frames of the video sequence. Highest sensitivity
produces the least number of pixels.
The next step is to generate a binary image. To implement this step, an approach is used, which is
as follows.
The image field is divided into equal areas. This does not necessarily use the entire selected area of
the image. Pixel codes are analyzed in each selected field and the average value of each selected
image field is determined. By analyzing all the selected fields, the range of pixel codes that belong to
the background is determined. Pixels whose codes are included in the selected range take the same
code, which defines only one color. For example, white or black can be chosen, which is the opposite
of the pixel codes belonging to the numbering plate. If the background is black, then the image of the
numbering plate is formed in white and vice versa.
An example of binarization of the image of the selected area based on the average values areas on
Figure 5 is shown.
Figure 5: An example of binarization of the selected area image based on the average values areas
The area of the numbering plate is also determined by the frequency of the vertical lines of equal
height that describe each character. An area of the image with such frequent changes in color and
intensity is selected. If such a frequency of occurrence of vertical lines is high, then a rectangular area
is selected in which such vertical lines are present. Also, such an area can be determined by finding
the average value of the number codes while moving the scanning window of a given size. An area is
determined in which the average value of the color codes of pixels belonging to this window differs
by a large amount from the rest of the areas. The average value of such an area approaches the value
of the color code of the numbering plate characters.
The resulting image has many distortions that are associated with the connection of individual
symbols of the numbering plate image. There are also gaps in the symbols themselves. This situation
can lead to false identification. However, the geometric shapes of the symbols have significant
differences that allow them to be distinguished with large distortions.
For segmentation of numbering plate characters, a method is used, which consists in finding
vertical spaces in the selected area of the entire license plate. This is done using parallel shift
technology (PST) [1]. With the help of PST, the copy of the image of the selected area of the
37
numbering plate is shifted to the right and the function of areas of intersection (FAI) is determined.
The FAI form determines the locations of the smallest number of pixels along the vertical and the
time steps of the shift, at which there are sharp jumps down, after the upper jumps. An example of an
FAI numbering plate on Fig. 6 is shown. Each FAI downward change indicates a gap between the
symbols. Moreover, if there is a merging of characters, then this will not affect the shape of the curve.
Horse racing will still be present on the FAI curve.
The example (Fig. 6) shows that in images with distortions they give a sufficiently clear shape of
the curve, which can be used to determine the locations of the symbols.
Figure 6: Example of the numbering plate FAI
Noise removal on the selected numbering plate symbols is carried out after applying the Radon
transform, which is implemented on cellular automata (CA) with a hexagonal coating [2]. With the
help of a CA with a hexagonal coating, six Radon projections are formed. The shape of these
projections carries out the search for extra pixels present in the image and located at the edges of the
images of the symbols or the entire numbering plate. An example of such projections is shown in Fig.
7.
The CA is used to search for “salt and pepper” noises. According to the analysis of the
neighborhood, such pixels are removed. The CA selects the neighborhood for each cell and analyzes
each cell simultaneously at each time step.
In Fig. 7 clearly displays noise on projections 00, 900 and 1200. They are present near the main
array of pixels and are removed.
For highly distorted images, the Radon transform also gives projections that allow you to
determine the shape of the symbols. For the example shown in Fig. 5 the following symbols are
highlighted (Fig. 8). Each symbol is segmented according to FAI analysis by shifting a copy of this
image to the right.
The image of each symbol is analyzed using the Radon transform. The obtained projections are
compared with the projections of images of ideal symbols recorded in the base of standards. These
symbols and their Radon projections are represented as a sequence of numbers in the base of
38
standards. If the obtained projection codes correspond to the forms and projection codes of images of
ideal symbols and at the same time differ from other similar symbols in the form of symbols, then this
symbol is identified by the closest reference symbol and, at the user's discretion, can be written as an
additional standard of this symbol.
Figure 7: FAI example of one number plate character
Figure 8: Images of individual symbols of the numbering plate shown in Figure 5
For example, the sixth symbol of the numbering plate image is considered, which is a distorted
image of the seven symbol. This image is close in shape to the image of the digit one symbol. The
Radon transform was applied to the image of the ideal seven and the ideal digit one, and also Radon
projections were generated for the image of the sixth symbol (Fig. 9). All Radon projections were
obtained with a binarization threshold of 50%. Other binarization thresholds can also be used. At the
same time, the projection forms practically do not change.
Analysis of the obtained projections shows a significant similarity of the projections of the images
of the ideal and distorted sevens. A sharp difference in projection images for images of sevens and
digit one symbols is also clearly defined.
39
To obtain such projections, a cellular automaton with a hexagonal form of coverage was used,
which made it possible to obtain six Radon projections instead of the classical three Radon
projections. This circumstance makes it possible to significantly improve the accuracy of the analysis
of the symbol image.
Figure 9: Radon projections for the symbol of seven and for the symbol digit one images shown in
Fig. 8
For all symbols of the identifier image (Fig. 8), the Radon projections on Fig. 10 are shown.
Gaps in 00 and 1500 projections indicate horizontal gaps in the numbering plate image. These gaps
divide the projections into two large areas. This indicates that the gap is in the middle of the symbols.
Based on the obtained projections, discontinuities can be determined, if individual parts of the
projections are very small, then it can be argued that there is noise, as well as other symbol analyzes
can be carried out. At the same time, a small number of symbols are used and identification
algorithms are simplified thanks to a small database of standards.
40
To recognize the selected symbol, you can also apply the method of analyzing the extreme pixels
on the four sides of the symbol, as well as analyzing all six Radon projections and the FAI shape is
carried out. The result of this analysis is a decision on the shape of the symbol in the numbering plate.
Figure 10: Images of individual symbols of the numbering plate shown in Figure 8
Both the Radon projections and the FAI set are described by the corresponding numeric arrays.
Each Radon projection can be represented as a numerical sequence, which is efficiently processed and
stored in the memory of standards. The area intersection function is also represented as an array of
numbers that describe a geometric figure in time, and each array of numbers has its own sequence,
which is also easily processed and stored in the electronic memory of standards.
4. Conclusion
The paper presents a system for the identification of moving objects of railway transport, which
allows high-precision identification of moving objects. The use of parallel shift and Radon
transformation technology allows automatic identification of the driving train in real time. The
proposed method for identifying the image of an identifier and the experimental studies carried out
made it possible to formulate the main requirements and stages of effective preprocessing of
numbering plate images. Using the parallel shift technology, it is possible to select each symbol in the
numerical sequence of the identifier using simple methods. Radon transforms and PST allow you to
remove or account for the presence of various interferences, such as salt and pepper, breaks, and
intersymbol connections.
As a result of experimental studies, it was found that the identification accuracy on average
corresponds to 99.3% for a rigidly fixed camera at a carriage speed of 70 km / h, as well as when the
angle of deflection of the video camera is 150. For the case of using a conventional video camera from
a mobile phone, the accuracy reaches 96.7 % at a deflection angle of 15 0. In addition, the system does
41
not use a large number of standards, and also does not require large expenditures of time spent on
training and setting up the system (10 or more reference values are enough to display each symbol).
The software is implemented in such a way that the system can process 17 images per second. The
system is cost effective as it does not require the use of high definition video cameras.
In further works, the author plans to conduct research work in the direction of identifying moving
objects belonging to other areas of human activity with complex images of identifiers.
5. References
[1] S. Yuzhakov, S. Bilan. Identification System for Moving Objects Based on Parallel Shift
Technology. Handbook of Research on Intelligent Data Processing and Information Security
Systems. Edited by Bilan, S. M., & Al-Zoubi, S. I. Hershey, USA: IGI Global(2019): 374 – 387
[2] R. L. Motornyuk, S. Bilan. The Moving Object Detection and Research Effects of Noise on
Images Based on Cellular Automata With a Hexagonal Coating Form and Radon Transform.
Handbook of Research on Intelligent Data Processing and Information Security Systems.
Handbook of Research on Intelligent Data Processing and Information Security Systems. Edited
by Bilan, S. M., & Al-Zoubi, S. I. Hershey, USA: IGI Global (2019): 330 – 359.
[3] Manish Bhuptani, Shahram Moradpur. RFID for your business = RFID Field Guide: Deploying
Radio Frequency Identification Systems, Troitsky N . Moscow: "Alpina Publisher", (2007): 70-
290
[4] M. Klems. RFID: Transport und Logistik an der Schwelle eines neuen Zeitalters, (German
Edition), GBI-Genios Verlag (2005)
[5] Judith Symond, John Ayoade and David Parry. Auto-Identification and Ubiquitous Computing
Applications. IGI-global (2009): 350.
[6] Vasin N.N., Baranov A.M. Video signal processing for identifying objects at a railway crossing.
Computer Optics. (2005), Issue 28: 152-155
[7] Hiroaki Niitsiima, Tsutomu Maruyama Real-Time Detection of Moving Objects. FPL (2004),
LNCS 3203: 1155-1157
[8] Hiroaki Niitsuma, Tsutomu Maruyama Real-Time Generation Of Three-Dimensional Motion
Fields. FPL (2005): 179-184
[9] Ashit Talukder, Larry Maithies Real-time Detection of Moving Objects from Moving Vehicles
using Dense Stereo and Optical Flow, Intelligent Robots and Systems, (2004). vol.4, (IROS
2004): 3718 - 3725
[10] Sedat Doğan, Mahir Serhan, Temiz Sıtkı Külür Real Time Speed Estimation of Moving Vehicles
from Side View Images from an Uncalibrated Video Camera. Sensors 2010, 10: 4805-4824
[11] Chunrong Yuan, Hanspeter A. Mallot Real-Time Detection of Moving Obstacles from Mobile
Platforms, ICRA10 Workshop on Robotics and Intelligent Transportation System, (2010): 109-
113
[12] D. Liya and L. Jilin. Intelligent freight train ID recognition system. In IEEE International
Conference on Intelligent Transportation Systems, (2002): 417–422.
[13] W. Zhang, G. Zhou, and M. Jiang. “Convolutional neural network for freight train information
recognition”. In International Conference on Machine Learning and
Computing (ICMLC), (2017-02): 167–171
[14] Z. Liu, Z. Wang, and Y. Xing. “Wagon number recognition based on the YOLOv3
detector”. In IEEE International Conference on Computer and Communication Engineering
Technology, (2019): 159–163
[15] Rayson Laroca, Alessander Cidra Boslooper, and David Menotti. “Automatic Counting and
Identification of Train Wagons Based on Computer Vision and Deep Learning”
arXiv:2010.16307v1 [cs.CV] 30 Oct 2020. URL: https://arxiv.org/pdf/2010.16307.pdf
[16] X. Zou, Y. Fu, and X. Li, “Image feature recognition of railway truck based on machine
learning,” in IEEE Information Technology, Networking, Electronic and Automation Control
Conference, Mar 2019: 1549–1555.
[17] Stepan Bilan. Models and hardware implementation of methods of Pre-processing Images based
on the Cellular Automata, Advances in Image and Video Processing, Vol 2, No 5 (2014): 76-90
42