=Paper=
{{Paper
|id=Vol-3006/05_short_paper
|storemode=property
|title=Visualization of the movements of natural objects based on remote sensing data
|pdfUrl=https://ceur-ws.org/Vol-3006/05_short_paper.pdf
|volume=Vol-3006
|authors=Aleksey A. Buchnev,Aleksandr V. Getling,Vladimir A. Krovotyntsev,Valery P. Pyatkin
}}
==Visualization of the movements of natural objects based on remote sensing data==
<pdf width="1500px">https://ceur-ws.org/Vol-3006/05_short_paper.pdf</pdf>
<pre>
Visualization of the movements of natural objects
based on remote sensing data
Aleksey A. Buchnev1 , Aleksandr V. Getling2 , Vladimir A. Krovotyntsev3 and
Valery P. Pyatkin1,✠
1
  Institute of Computational Mathematics and Mathematical Geophysics SB RAS, Novosibirsk, Russia
2
  Skobeltsyn Institute of Nuclear Physics, Lomonosov Moscow State University, Moscow, Russia
3
  Research Center “Planeta”, Moscow, Russia


                                         Abstract
                                         Methods of constructing vector fields of natural objects’ movements based on a series of consecutive
                                         satellite images are considered: cloud formations in the atmosphere based on a series of consecutive
                                         images obtained from geostationary satellites; water masses and ice fields based on a series of images
                                         from low-orbit satellites; using the example of the evolution of bipolar spots, the trajectories of trial
                                         corks in the Solar photosphere are constructed based on the data of sounders installed on heliophysical
                                         satellite observatories.

                                         Keywords
                                         Natural objects, cloud formation, standard, water masses, ice fields, Delaunay triangulation, trial corks.




1. Moving cloud formations
One of the urgent tasks of space monitoring is to determine the spatial movements of cloud
formations in the atmosphere from different-time images obtained from geostationary Earth
satellites. According to [1], one of the ways to determine the spatial movements of objects from
different-time satellite images is the method based on finding the maxima of the cross-correlation
coefficient. In [2], a similar approach is considered as a method of pattern recognition, known as
correlation matching. In both cases, correlation is used as a means of finding the equivalents of
the standard object represented as an image 𝑤(𝑥, 𝑦) by dimensions 𝐽 × 𝐾, on the image 𝑓 (𝑥, 𝑦)
by dimensions 𝑀 × 𝑁 ; it is assumed that 𝐽 ≤ 𝑀 and 𝐾 ≤ 𝑁 . Cross-correlation coefficient
                               ∑︀ ∑︀
                                     [𝑓 (𝑥 + 𝑠, 𝑦 + 𝑡) − 𝑓𝑚 (𝑥, 𝑦)] [𝑤(𝑠, 𝑡) − 𝑤𝑚 ]
                                𝑠 𝑡
                   𝛾(𝑥, 𝑦) =                                                        .           (1)
                                                 (𝐽𝐾 − 1)𝜎𝑤 𝜎𝑓

Here 𝑤𝑚 — the average value of pixels in the standard 𝑤, 𝑓𝑚 — the average value of the image
elements 𝑓 in the area covered by the reference. The denominator in (1) uses the product of the
standard deviation 𝜎𝑤 of the pixels of the standard 𝑤 by the standard deviation 𝜎𝑓 of the pixels
of the image 𝑓 in the area covered by the reference.

SDM-2021: All-Russian conference, August 24–27, 2021, Novosibirsk, Russia
   ✠
     Deceased author.
" baa@ooi.sscc.ru (A. A. Buchnev); a.getling@mail.ru (A. V. Getling); krv@planet.iitp.ru (V. A. Krovotyntsev)
                                       © 2021 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).
    CEUR
    Workshop
    Proceedings
                  http://ceur-ws.org
                  ISSN 1613-0073       CEUR Workshop Proceedings (CEUR-WS.org)



                                                                                                          32
Aleksey A. Buchnev et al. CEUR Workshop Proceedings                                         32–38


   In general, the size and orientation of the standard object found in the next image of a series
of images may have different values in the subsequent image. The expression (1) normalized
with respect to changes in amplitudes is not the same with respect to rotation or changes
in the size (scaling) of the standard. In [2] it is noted that “Normalization with respect to
dimensions is associated with spatial scaling, which in itself is associated with very time-
consuming calculations. Normalization with respect to rotation is an even more difficult task...”.
Sometimes, to solve the problem of matching (combining) images, taking into account the
scaling and rotation of the standard, less accurate criteria are used in comparison with (1), but
simpler from a computational point of view (for example, in [3] and a number of other works,
the sum of the absolute values of the differences of the corresponding image components is
used).
   According to [4], the search for the positions of the found standards (determination of offsets)
in the next image of the series can be implemented by one of three methods: determining the
maximum of the cross-correlation coefficient in the spatial domain, determining the maximum
of the cross-correlation coefficient in the frequency domain based on the fast Fourier transform,
and finding the minimum of the sum of the squares of distances. The mentioned source also
does not assume any standard transformations when searching for offsets, with the exception
of the transfer transformation.
   In the presented work, the standard displacements are determined on the basis of determining
the maximum of the cross-correlation coefficient in the spatial domain in accordance with the
formula (1). In this case, the standard can be transformed, consisting of scaling, rotation and
transfer. An efficient algorithm based on the scanning rows method has been developed. The
obtained results of computational experiments on images from the METEOSAT-8 spacecraft
indicate both the need to take into account the scaling and rotation of the standard, and the
acceptable time of the corresponding calculations. Level 1B data (10-bit pixels) is used.
   The solution of the problem consists of the following basic steps: search square standard
objects in the current image 𝑊 , based on achieving the maximum contrast or maximum entropy;
the center of the standard coincides with the center of the square; search for the positions of
the found standard in the subsequent image 𝐹 , based on the achievement of the maximum
value of the cross-correlation coefficient; plotting vector fields of spatial movements of objects
in accordance with the found positions.
   Search for standards. The search for standards is based on the methodology proposed by
EUMETSAT [4]. According to this methodology, two types of standards are provided: main
and secondary. The positions of the main standards coincide with the ends of the displacement
vectors of the standards from the previous image (there are no main standards for the first
image).
   The search for secondary standard of the Targ_Size size is performed in the grid nodes with
the Grid_Size size. The size of the square area centered at the grid nodes for searching for
standards is set by the Targ_Search parameter. The allowed minimum distance between the
standards is controlled by the Targ_Dist parameter. The optimal position for the standard within
the search area is the one where the maximum value of the control parameter Par — contrast or
entropy is reached. When searching for standards within a region of the Targ_Search size, local
averages and standard deviations calculated from the 3 × 3 neighborhood are used. Contrast is
defined as the difference between the maximum and minimum values of the local averages. In



                                                33
Aleksey A. Buchnev et al. CEUR Workshop Proceedings                                         32–38


addition to the listed parameters that determine the “physical” characteristics, when searching
for standards, the parameters that characterize the “variability” of the image inside the area
covered by the standard are used: Min_St_Dev — the minimum value of the local standard
deviation, Num_Gr_SD — the minimum number of pixels with a standard deviation greater than
Min_St_Dev.
   Determination of standard offsets. The offset is determined for each of the K_Targs found
benchmarks. The search for a new standard position is performed inside a square area of
the Search_Size size. The center of the search area coincides with the original position of the
standard. The new position of the standard is the position where the maximum value of the
cross-correlation coefficient Corr is reached. During the scan of the search area, the standard is
subjected to scaling and rotation transformations. The scale and angle of rotation of the standard
are determined by the following parameters: Scale_Min — the minimum value of the scale of
the standard; Scale_Max — the maximum value of the scale of the standard; Scale_Delta — the
increment of the scale of the standard; Angle_Beg — the initial angle of rotation of the standard
in degrees; Angle_End — the final angle of rotation of the standard in degrees; Angle_Delta — the
increment of the angle of rotation of the standard in degrees. Since, in general, the transformed
discrete grid of the reference does not coincide with the discrete grid of the output image, it
becomes necessary to interpolate the pixel values. We offer three ways to get the pixel values
of the transformed standard: rounding to the nearest integer; bilinear interpolation; bicubic
interpolation.




Figure 1: Frame vis006200603151130.              Figure 2: Frame vis006200603151130.




Figure 3: Vector field.




                                               34
Aleksey A. Buchnev et al. CEUR Workshop Proceedings                                        32–38


   Figures 1–3 illustrate a fragment of the obtained vector field of cloud formations moving
in the atmosphere. The process of moving the vortex is demonstrated. The vector field is
constructed from five consecutive images from the Meteosat-8 satellite in the optical range (the
time interval between the images is 30 minutes). Figure 1 shows the first frame of the image
sequence, Figure 2 — the last frame. Figure 3 shows the corresponding vector field.


2. Movement of water masses and ice fields
One of the important uses of satellite data is the monitoring the movement of water pollution
and ice fields in marine areas.
   The problem of constructing the fields of propagation of marine pollution from different-
time satellite data is closely related to the problem of determining the speed and direction of
vectors of spatial movements of water masses [5]. Satellite information of the visible, infrared,
or microwave (radar data) ranges is used as the initial data in the problem. The method of
constructing the fields of spatial movements of water masses based on identifiable changes in
some water objects (tracers) on successive satellite images transformed into a single cartographic
projection is used. As tracers in optical range images water bodies formed as a result of water
blooming (linear and vortex structures of phytoplankton and algae distribution) are used. For
infrared images, linear and vortex thermal structures are mainly used as tracers, while for radar
images, oil films and films of biogenic pollutants are used.
   A similar approach is used in monitoring the spatial movements of ice fields. Here, mosaics,
made up of radar satellite images transformed into the same cartographic basis, are mainly used
(images of the Arctic Ocean are most often used, because of their importance for the purposes
of meteorology and ships navigation).
   The process of entering the coordinates of tracer objects consists in specifying their position
on the current and next images and saving the entered coordinates in a file that will be used in
the final stage of processing and saved for further use.
   At the same time, satellite images are pre-made to “fix” the contours of the coastline using
reference points, and, thus, on the maps of the distribution of the fields of spatial movements of
natural objects, the stationary land is separated from the moving objects.
   The Delaunay triangulation is constructed based on the entered coordinates of the objects in
the current image. Each triangle of the triangulation corresponds to a triangle in the following
image. Such a set of pairs of triangles determined the set of piecewise affine transformations
of the plane. These transformations are applied to the nodes of the regular grid in the current
image, thereby forming the required displacement vectors.
   Simultaneously with the construction of the vector fields of spatial displacements of wa-
ter masses and ice fields, histograms of the velocities and directions of these vectors can be
constructed.


3. Evolution of bipolar spots in the Solar photosphere
Theoretically, the technologies for constructing displacement fields of natural objects presented
in paragraphs 1 and 2 can be applied to image sequences of any length. However, in the practice



                                               35
Aleksey A. Buchnev et al. CEUR Workshop Proceedings               32–38




Figure 4: Water mass transfer in the Sea of Azov 08–09.09.2006.




Figure 5: Bipolar group of spots.




                                                36
Aleksey A. Buchnev et al. CEUR Workshop Proceedings                                              32–38


of constructing operational hydrometeorological forecasts, for which these technologies were
developed, no more than 3–4 images are mainly used. Interactive versions of the corresponding
programs are used for this purpose.
   But in the processing of data obtained from heliophysical satellite platforms, there is a need
to process a series of consecutive images with a length of 100 or more frames. An example is the
image of a bipolar group of spots shown in Figure 5. We use here data from the Helioseismic and
Magnetic Imager of the Solar Dynamics Observatory (NASA, USA). The goal of processing from
the point of view of visualization is to build a field of horizontal velocities of the so-called “trial
corks”, which at the initial moment are located in the nodes of a regular rectangular grid. As
the process progresses, it is necessary to track the trajectory of each trial cork. We considered
a series of images of different duration from 2 to 4 hours with an interval of 135 s. between
frames [6].
   For this purpose, we used a console program [7], which includes the technology of claim 1
and a modified version of the technology of claim 2. The processing parameters are located in a
text file, which is a program parameter. For each pair of adjacent images, you must: 1) in the first
image, find the standards in accordance with claim 1; 2) find the positions of these standards in
the second image by correlation comparison (claim 1); 3) in the first image, build a Delaunay
triangulation based on the found positions of the standards; 4) from the obtained triangles of
triangulation and the corresponding triangles in the second image, build a set of piecewise
affine transformations; 5) for each trial cork, determine its new position by applying a suitable
affine transformation. The final stage of processing is the plotting of trial corks trajectories
(Fig. 6).




Figure 6: Velocity field of trial corks.




                                                  37
Aleksey A. Buchnev et al. CEUR Workshop Proceedings                                        32–38


Acknowledgments
The research was partially carried out within the framework of the state task of the ICM&MG
SB RAS 0251-2021-0003.


References
 [1] Jähne B. Digital image processing. Berlin: Springer, 2005.
 [2] Gonzalez R., Woods R. Digital image processing. Trans. from English. Edited by
     P.A. Chochia. Moscow: Technosphere, 2005.
 [3] Zabelin V.A., Pyatkin V.P. Identification of control points in aerospace images // In: Math-
     ematical and Technical Problems of Image Processing. CC SB of the USSR Academy of
     Sciences. Novosibirsk, 1980. P. 50–59.
 [4] MSG meteorological products extraction facility. Algorithm specification document // Doc.
     No. EUM/MSG/SPE/022. 2004. Is. 2.6.
 [5] Asmus V.V., Buchnev A.A., Krovotyntsev V.A., Pyatkin V.P. Software complex in the
     tasks of space monitoring of dangerous hydrometeorological phenomena // Problems of
     Informatics. 2018. No. 4(41). P. 34–49.
 [6] Getling A.V., Buchnev A.A. The origin and early evolution of a bipolar magnetic region in
     the solar photosphere // Astrophysical Journal. 2019. Vol. 871. No. 2. P. 224.
 [7] Getling A.V., Buchnev A.A. Some structural features of the convective-velocity field in the
     solar photosphere // Astronomical Reports. 2010. Vol. 54. P. 254.




                                               38

</pre>