=Paper=
{{Paper
|id=Vol-3006/04_short_paper
|storemode=property
|title=Processing of data streams in the detection of retroreflective objects
|pdfUrl=https://ceur-ws.org/Vol-3006/04_short_paper.pdf
|volume=Vol-3006
|authors=Sergey M. Borzov,Oleg I. Potaturkin
}}
==Processing of data streams in the detection of retroreflective objects==
https://ceur-ws.org/Vol-3006/04_short_paper.pdf
Processing of data streams in the detection
of retroreflective objects
Sergey M. Borzov1 , Oleg I. Potaturkin1
1
Institute of Automation and Electrometry SB RAS, Novosibirsk, Russia
Abstract
This work is devoted to the study of methods for detecting retroreflective objects (RRO), including optical
and optoelectronic observation devices, based on the search for spatial anomalies in images formed
in pulse laser location systems. Algorithms and software for detecting RRO have been developed. At
the same time, special attention is paid to various methods of forming difference frames with periodic
illumination, including preliminary replacing each pixel of the background images with the maximum
value for the corresponding neighborhood. The effectiveness of the proposed methods for detecting
retroreflective objects in conditions of intense sunlight, despite the presence of specular and diffuse
reflecting surfaces in the field of view, is demonstrated.
Keywords
Remote detection of retroreflective objects, formation of difference images, search for spatial anomalies,
pulse laser location.
1. Introduction
Recently, much attention has been paid to a special class of optoelectronic laser location systems
designed to detect retroreflective objects (RRO), such as photo and video equipment, optical
and optoelectronic surveillance systems, various types of reflectors, etc. [1].
The operation of such systems is based on the use of the effect of retroreflection, which
occurs when highlighting objects of these types, as a result of which in the field of view in the
coordinates corresponding to the RRO, luminous points are formed against the background of
the underlying surface, observed only from a position close to the position of the emitter. To
increase the efficiency of the detection of RRO at long distances, the organization of synchronous
operation of the laser emitter used to illuminate the scene and the photodetector with a high-
speed shutter is carried out. Moreover, the illumination is carried out in short pulses, and the
photodetector shutter opens for a time close to their duration with a delay (relative to the
operation of the emitter) equal to the time of light propagation to the observed objects and back.
Due to this, the photodetector perceives the radiation reflected from the objects of interest, and
cuts off the light reflected from objects that are closer and further than the specified distance.
As a shutter in devices operating on the basis of the described method, as a rule, a high-speed
image intensifier is used. A detailed overview of such devices is provided in [2]. However,
the use of image intensifier leads to a complication of their design and a decrease in the
SDM-2021: All-Russian conference, August 24β27, 2021, Novosibirsk, Russia
" potaturkin@iae.nsk.su (O. I. Potaturkin)
Β© 2021 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).
CEUR
Workshop
Proceedings
http://ceur-ws.org
ISSN 1613-0073 CEUR Workshop Proceedings (CEUR-WS.org)
24
Sergey M. Borzov et al. CEUR Workshop Proceedings 24β31
spatial resolution of the generated images of the controlled scene. An alternative solution
is to implement the shutter function using a special algorithm for controlling a two-section
photodetector receiver with a lowercase transfer [3]. This approach is implemented in the
hardware and software complex [4], the purpose of which is to check the performance of CCD
arrays as part of active-pulse devices for detecting RRO without using an image intensifier tube
or other external high-speed shutter in its design.
To effectively solve the problem of detecting and recognizing RRO in real conditions, it is
necessary to equip laser pulse location systems with built-in functions for processing recorded
images, aimed at identifying fragments that potentially contain objects of interest. The pro-
cessing performed in automatic mode is based on the analysis of the spatial distribution of
the image brightness by a sliding window and the assessment of the presence of detectable
objects at various points of the observed scene. At the same time, this assessment is traditionally
performed by determining the correspondence between the analyzed fragments and the a priori
description of the detected objects [5]. However, as practice has shown, this approach is not
effective enough due to possible changes in illumination, low resolution of RRO images at a
considerable distance and unknown angles of the observed objects, the presence of significant
atmospheric distortions and a diverse dynamic background. Therefore, in this paper, to detect
RRO in the field of observation, it is proposed to use a method based on the formation of a
description of the background for each scene, followed by the search for fragments whose
parameters do not correspond to the received description. This approach in the literature is
called the search for anomalies [6], does not require the implementation of preliminary training
procedures and allows you to detect in the field of view of the observation system atypical
fragments in certain parameters on a non-uniform background.
The aim of this work is to develop and experimentally study the effectiveness of digital data
processing methods for pulse laser location systems designed for automatic detection of RRO by
accumulating signals with a pulse illumination frequency and intra- and inter-frame processing
of recorded images.
2. Software and hardware
An optoelectronic stand consisting of sensing channels and synchronous image recording was
used for experimental investigation of the RRO detection method [7]. The first of them provides
illumination of the observed scene by narrow-band pulsed laser radiation, and the second-the
formation of the resulting images. The control of the stand, as well as the visualization and
processing of the received data, is carried out on a personal computer using an application
specially developed for this purpose.
When the scene is illuminated, part of the radiation reflected from the RRO returns to the
light source, creating a light response (glare) of significantly greater intensity than in diffuse
reflection from other objects. The location can be made both by single-frequency pulses, and
their series with the accumulation of signals by the CCD receiver of the photodetector.
The stand allows you to detect with high confidence RRO with a small light aperture at a
significant range. As an example, Figure 1 shows an image of a scene containing a camera (the
lens focus is 50 mm, the aperture value is f/16) on a natural background (soil cover, tree-shrub
25
Sergey M. Borzov et al. CEUR Workshop Proceedings 24β31
Figure 1: Images of the RRO on a natural background, registered by the stand.
and herbaceous vegetation), recorded using an optoelectronic stand with the accumulation of a
signal from 100 illumination pulses.
It can be seen that the detection of RRO with such a small retroreflective index in this image
can be performed accurately due to a significant increase in the signal-to-background ratio,
even without the use of additional image processing. However, as practice has shown, with
intense sunlight and the presence of objects of artificial origin in the field of view, the task is
significantly more complicated.
To increase the efficiency of the detection of RRO under these conditions, it is proposed to
use periodic illumination of the scene, and at the stage of data analysis to perform inter-frame
processing [8] of the recorded images and search for spatial anomalies [9] by a series of 2π
frames of the sequence, where π is the number of frames with and without illumination.
Cross-frame processing is designed to form difference frames and is performed in two
ways. The first one is implemented by a simple pixel-by-pixel subtraction of the π frame with
illumination and the π frame without illumination [7].
πΌ(π₯, π¦; π) = πΌ1 (π₯, π¦; π) β πΌ0 (π₯, π¦; π). (1)
The second one is proposed to be carried out by pre-replacing each pixel of the image obtained
in the absence of laser illumination with the maximum value for its given neighborhood, i.e., by
local processing.
πΌ(π₯, π¦; π) = πΌ1 (π₯, π¦; π) β max [πΌ0 (π₯, π¦; π)] . (2)
π
In this case, the size of the neighborhood π must correspond to the estimated size of the
images of the detected objects.
At the final stage of inter-frame processing, it is advisable to perform averaging of a series of
difference images obtained in one way or another
1 βοΈ
πΌ(π₯, π¦; π) = ππΌ(π₯, π¦; π). (3)
π1
π=1
26
Sergey M. Borzov et al. CEUR Workshop Proceedings 24β31
The procedure for searching for spatial anomalies from the generated difference images
πΌ(π₯, π¦) includes:
β formation of the spatial distribution of the informative signal
(οΈ )οΈ
1 βοΈ 1 βοΈ
πΉ (π₯, π¦) = πΌ(π₯ + π₯1 , π¦ + π¦1 ) β πΌ(π₯ + π₯1 , π¦ + π¦1 ) Β· ππ (4)
ππ π πΞ©
Ξ©
where π and πππππ are the central and peripheral zones of the fragment, ππ and πΞ©
are the number of pixels in them, respectively, π₯1 , π¦1 β π, β¦;
β determination of local maxima in the resulting distribution πΉ (π₯, π¦) by processing with a
sliding window π
πΉ (π₯, π¦), if πΉ (π₯, π¦) > maxπ π (πΉ (π₯ + π₯1 , π¦ + π¦1 ))
{οΈ
π (π₯, π¦) = (5)
0, else
where π₯1 , π¦1 β π.
Next, in the array π (π₯, π¦), the mean π and the standard deviation π of non-zero values
are calculated, i.e., the parameters of local maxima typical for this scene are determined (a
description of the βbackgroundβ class is formed), and the anomalous elements of the array
π (π₯, π¦) are selected using threshold processing
1, if π (π₯, π¦) > π + ππ
{οΈ
π
(π₯, π¦) = (6)
0, else
where π is the threshold coefficient, which is usually 3 (it can be selected by the operator
depending on the observation conditions). The position of non-zero elements in the binary
array π
(π₯, π¦) corresponds to the coordinates of the detected objects in the original image
(π₯π , π¦π ).
3. Detection of RRO in difficult conditions
To confirm the prospects of the proposed method of data processing, experimental studies
were carried out on the detection of RRO in conditions of intense background radiation with
the use of periodic illumination by a series of pulses and the formation of difference images.
The measurements were carried out in the daytime under bright solar radiation. A number
of retroreflective and mirror-reflecting objects were located in the field of view at different
ranges. Among them are two RROs that are the target of detection: a television camera and a
road sign with a retroreflective coating, located on the roadside at a distance of 1800 m. At the
same distance, there were two objects that mirror the sunlight in the direction of the recording
system (car elements). In addition, there were several other bright objects in the foreground
and background of the sctne, including those with a reflective coating. Figure 2 shows images
πΌ1 (π₯, π¦) and πΌ0 (π₯, π¦) containing the specified objects registered in the presence and absence of
laser illumination.
27
Sergey M. Borzov et al. CEUR Workshop Proceedings 24β31
a b
Figure 2: Images of the road situation registered in the presence (a) and absence (b) of laser illumination.
It can be seen that in these images, the detected objects are difficult to distinguish without
performing special processing.
In the difference image (Figure 3, a), formed without accumulation according to expres-
sions (1), (3) by subtracting successive images recorded in the presence and absence of laser
illumination (π = 1), a large number of responses due to different objects are also observed.
The brightest of them are numbered from 1 to 6. Here, 1, 2 β are caused by retro-reflection
from the detected RROs, 3β6 β by mirror reflection of sunlight (3 β from the bumper of a car
moving in the opposite direction on the roadway, 4 β from the body of a car at short range, 5,
6 β from the catafoats of a car in the foreground. It should be noted that performing inter-frame
processing of the generated images even when using two frames leads to the suppression of
signals caused by the reflection of solar radiation from stationary objects. The contrast of the
detected RROs, as well as dynamic objects in the reflected solar radiation, increases significantly.
For clarity, Figure 3, b shows the maximum column values in the resulting difference image.
a b
Figure 3: The difference image (a) formed from two frames registered in the presence and absence of
laser illumination, and the maxima of signals in this image by columns (b).
28
Sergey M. Borzov et al. CEUR Workshop Proceedings 24β31
a b
Figure 4: A difference image accumulated over 20 frames, with the result of detection for pixel-by-pixel
processing (a) and the maxima of signals across columns (b).
Performing procedures (4)β(6) under these conditions leads to the selection of a large number
of false objects, in addition to the detected ones.
The difference image πΌ(π₯, π¦), formed according to (1), (3) from twenty frames (π = 10), the
maximum column signals in the resulting difference image and the result of subsequent object
detection are shown in Figure 4. Its analysis shows that the accumulation of a larger number of
frames leads to a further increase in the brightness of the signals corresponding to the detected
RROs (compared to the signals of static and dynamic objects in the reflected solar radiation).
However, individual relatively slow dynamic objects with high brightness, which shift by no
more than the size of the object during the recording of a series of images, still form quite
intense responses (for example, object 4).
To suppress them, it is proposed to use the formation of difference images with a preliminary
substitution of each pixel of the image obtained in the absence of laser illumination for the
maximum value in its given neighborhood π in accordance with (2), (3). Such a difference image
a b
Figure 5: A difference image accumulated over 20 frames, with the result of detection for local process-
ing (a) and maximums of the signal in columns (b).
29
Sergey M. Borzov et al. CEUR Workshop Proceedings 24β31
Table 1
Signals of objects with different methods of forming a difference image
Method of forming a difference image 1 2 3 4 5 6
I 0.11 0.23 0.63 1.00 0.23 0.43
II 0.26 0.55 0.09 1.00 0.13 0.30
III 0.60 1.00 0.07 0.12 0.08 0.13
formed from twenty frames, and the result of subsequent detection according to (4)β(6), are
shown in Figure 5, a. Figure 5, b, by analogy with the previous figures, shows the maximum
column signals in the resulting difference image.
In Table 1 given the relative signals of the objects in the resulting difference images for three
ways of their formation: I β in accordance with the expressions (1) and (3) when π = 1; II β
in accordance with the expressions (1) and (3) when π = 10; III β in accordance with the
expressions (2), (3) when π = 10.
The obtained data confirm that the accumulation of frames with the preliminary use of
the procedure for determining the maximum value for the neighborhood in the background
frame allows you to additionally (several times) increase the relative signal level of detected
objects (1), (2) and eliminate errors associated with the selection of bright objects (4) that have
a slight shift from frame to frame.
4. Conclusion
Algorithms are proposed and hardware and software tools are developed for complex intra- and
inter-frame processing of image sequences for the detection of retroreflective objects based on
the search for spatiotemporal anomalies using active pulse location by optoelectronic means.
Their efficiency is demonstrated for the selection of RRO in conditions of intense sunlight in
the presence of mirror and diffusely reflecting surfaces in the field of view, the signals of which
are transmitted through the use of the proposed methods of digital data processing. Thus, the
signals of static objects are suppressed by pixel-by-pixel inter-frame subtraction, and dynamic
objects are suppressed by the accumulation of difference images.
Of particular note is the method of forming difference frames with the preliminary replace-
ment of each pixel of the background images by the maximum value in its neighborhood. Its
use makes it possible to suppress the signals of relatively slow bright dynamic objects that shift
by no more than the size of the object during the recording of a series of images.
Acknowledgments
The work was carried out with the support of the Ministry of Science and Higher Education as
part of the implementation of the State Task No. 121022000116-0 in the Institute of Automation
and Electrometry of the Siberian Branch of the Russian Academy of Sciences.
30
Sergey M. Borzov et al. CEUR Workshop Proceedings 24β31
References
[1] Karasik V.E., Orlov V.M. Location laser vision systems. Moscow: Publishing House of the
Bauman Moscow State Technical University, 2013. 478 p. (In Russ.)
[2] Volkov V.G. Night vision devices for detecting glare elements // Specialnaya Tehnika. 2004.
No. 2. P. 2β9. (In Russ.)
[3] Golitsyn A.A, Seifi N.A. Active-pulse method of observation using a CCD photodetector
with a lowercase transfer // Izvestiya Visshih Uchebnih Zavedenii. Priborostroenie. 2017.
Vol. 60(11). P. 1040β1047. (In Russ.)
[4] Alantyev D.V., Golitsyn A.A., Golitsyn A.V., Seifi N.A. Stand for the study of the possibility
of using matrix photodetectors of the visible range as part of active-pulse observation
devices // Opticheskii Jurnal. 2018. Vol. 85(6). P. 53-57. (In Russ.)
[5] Roth P.M., Winter M. Survey of appearance-based methods for object recognition // Tech-
nical Report ICG-TR-01/08. Institute for Computer Graphics and Vision, Graz University
of Technology, Austria. January, 2008. 68 p.
[6] Chandola V., Banerjee A., Kumar V., Anomaly detection: A survey // ACM Computing
Surveys. 2009. Vol. 41(3). A. 15. 58 p.
[7] Alantiev D.V., Borzov S.M., Kozik V.I., Potaturkin O.I., Uzilov S.B., Yaminov K.R. Exper-
imental study of method of laser pulsed location for retroreflective objects detecting //
Optoelectronics, Instrumentation and Data Processing. 2021. Vol. 56(1). P. 103β111.
[8] Kirichuk V.S., Kosykh V.P., Popov S.A., Sinelβshchikov V.V. Suppression of a quasi-stationary
background in a sequence of images by means of interframe processing // Optoelectronics,
Instrumentation and Data Processing. 2014. Vol. 50(2). P. 109β117.
[9] Borzov S.M. Detection of dynamic objects on the basis of space-time anomalies in video se-
quences // Optoelectronics, Instrumentation and Data Processing. 2013. Vol. 49(1). P. 9β13.
31