One of important aspects of developing geographic information systems and remote sensing of the Earth is the problem of displaying the received data for visual analysis. Dynamic data are particularly difficult in this regard. In this case, not the current state may be of interest, but the change in data over time. It is very difficult to display such data and their change. At the same time, in some cases, it is the change of some spatial and vegetation indices that can give man a complete picture and allow to make the right decisions. This paper is devoted to the development of methods for displaying changes in the Earth remote sensing data based on automatically placed markers showing the direction and nature of the change. Experiments have shown that such markers make it possible to focus man's attention well on the dynamics of changes and improve the visual perception of such information. The paper proposes not only an original approach to markers, but also an algorithm for automatic arrangement of these markers on the map, taking into account both remote sensing data and the nature and dynamics of their changes over a certain period of time. This allows to build a fully automated system for preparing visual maps for people. The method of marker formation itself, which combines both color and symbolic information for a sharper image, may also be of interest. geographic information system (GIS), Earth remote sensing, displaying, dynamic data dated 21.08.2017 No. 452 "On approval of the data list included in the report on forest reproduction and afforestation, the form and procedure for submitting a report on forest reproduction and afforestation, as well as the requirements for the report on forest reproduction and afforestation in electronic form", it is necessary to submit reports on forest changes quarterly [1]. Information about forest reproduction is generated based on the data from the report. It is also necessary to compare the reality and reporting. Forest restoration is a long process that must be constantly monitored and, in the case of young vegetation death for any reason we should respond quickly to prevent the complete death of the entire planting.
According to paragraph 2 of the order of the Ministry of Natural Resources of the Russian Federation
2021 Copyright for this paper by its authors. received and processed data. The paper proposes an approach based on markers for displaying the dynamics of the average vegetation index of the region under consideration.
The analysis of the area in order to identify the features of developing vegetation or natural phenomena is a very complex and time-consuming task. Nowadays, methods of remote sensing of the Earth and images obtained from satellites or unmanned aerial vehicles are being used more and more widely for this purpose. This allows to track changes over long periods and respond faster to desirable or undesirable occurrences.
Satellite systems are one of image sources. At the moment, there is an extensive range of satellite
systems that provide images of the Earth [
The peculiarity of satellite images is that they are taken in different ranges. For example,
Sentinell2 provides images in 13 wave ranges that correspond to: B1 - ultra-blue; B2 - blue; B3 - green; B4
red; B5, B6, B7, B8, B8a - visible and near-infrared radiation; B9, B10, B11, B12 - short-wave infrared
radiation [
However, images in different ranges themselves do not always allow to conduct the necessary
analysis and draw the right conclusions. Therefore, in practice, not the images themselves are used, but
a combination of them, the so-called indices [
, + where NIR is the reflection in the near-infrared spectrum, RED is the reflection in the red spectrum.
A standardized continuous gradient scale is used to display NDVI index. Due to the reflection feature in NIR-RED regions of the spectrum, natural objects that are not associated with vegetation have a fixed NDVI value (which allows to use this parameter for their identification).
By calculating the index value at each point in space, we get some map of the indicator distribution
(for example, vegetation). At the same time, if it is necessary to analyze the dynamics of development,
then it is necessary to build and calculate such indices in different time periods and analyze them
together [
A few conclusions can be made from the given data. The images clearly show four areas. There is a forest plantation above (area No. 1), which vegetation does not change very much during the entire observation period (spring-summer). Just below is an agricultural field (area No. 2). In this area, it is clearly visible that there was almost no vegetation in spring (Figure 1a) and gradually appeared (Figure 1b and Figure 1c). In the second half of summer, the crop was mowed and the field was plowed, the vegetation again approached zero (Figure 1d).
Areas No. 3 and No. 4 are very similar in their composition in the initial period of time. However, it can be noted that the vegetation cover in area No. 3 practically does not develop during the entire growing season, which indicates the low fertility of this area.
Often, the analysis described above is quite a complex task that requires a lot of effort from the expert. Therefore, it is very important to develop ways of displaying dynamics that will help the expert in solving his problems. c d Figure 1: Dynamics of vegetation development on the example of NDVI index
As it was shown above, it is very important to add information about the dynamics of changes in the observed value to the map in some way. At the same time, we need to understand that the image itself is static, and it is not advisable to use several images (time periods), because this leads to a complication of visual analysis by man.
To solve the resulting contradiction, we suggest using some markers for displaying dynamics. These markers are presented as circles in which the change in the necessary value during the observation is displayed from the center to the periphery in the form of color rings (or a smooth transition). Such markers allow to evaluate visually the change in the situation at a certain point in space and see some trends.
However, practical studies have shown, that it is often possible to see the changes by means of the markers, but it is difficult to distinguish their nature (slowing down or acceleration, the speed of the change and trends). That is why it is proposed to supplement these markers with a symbolic representation of the change nature. The types of markers, algorithms for calculating their parameters and locations on the map are described below.
One of the key issues of displaying dynamics in this paper is the issue of drawing markers. The effectiveness of the entire proposed solution depends on their clarity [14].
Pie charts of small size with a radial display of the index for a certain time period are proposed to use as markers of dynamics. The general logic is as follows. In the very center of the circle, the color designation of the index at the initial time of observation is displayed. Next, there are concentric circles of colors corresponding to different time intervals (see Figure 2).
t0 t1 … tk
time moment tk
time moment t0
So, this display clearly shows the dynamics of the change. If the index value has not changed for a sufficiently long period of time, the marker will be filled in evenly. At the same time, frequent and large changes in the index will lead to gradient filling of the marker. And the more the color of the marker center differs from the edges, the more the index has changed during the observation process. And the color itself will show the value at each time, and the change from light to dark or from dark to light will show the direction of the change (the growth or fall of the value at this point).
However, the conducted research on the focus group showed that the markers themselves, when displayed in miniature on the map, very conditionally allow to see the dynamics and do not allow to evaluate the nature of changes at all, which is also a very important element in some cases. For example, it is difficult to judge from such a diagram whether the index change is accelerated or slowing down. To complete this information, it is proposed to add an arrow inside the marker, showing not only the direction of dynamics, but also the nature of such a change. Examples of resulting markers with symbolic arrows are shown in Figure 3.
a b
c d e f g
If there were no changes in the index at this point, it is proposed to display a horizontal arrow from left to right, symbolizing the value constancy (Figure 3a). At the same time, if there is a uniform increase or decrease in the indicator value, we offer to use an arrow with a positive or negative angle of inclination, respectively (Figure 3b and Figure 3c). Moreover, the angle of inclination corresponds to the growth. Thus, using the arrow, we can not only see the growth or fall, but also visually assess the significance of such a change.
Another display option is a curved arrow (Figure 3d-3g). In this case, the curve shows the nature of the change – whether it is uniform or accelerated. The exponential curve characterizes the accelerated growth of the index at a given point (Figure 3d), and the logarithmic one characterizes the slowing growth (Figure 3e).
One of the difficulties associated with the arrangement of markers is the choice of locations for them. As practice shows, markers should be placed automatically, otherwise they will not make much sense. The purpose of markers is to improve the visual perception of the map by man and allow to analyze the dynamics of changes in space. At the same time, if the location of the markers is chosen by man, then such a visual analysis will not allow to see the whole picture.
There are also difficulties in choosing the number of markers. Too few markers will not allow to analyze the entire situation, because it will display too rough assessment. At the same time, too many markers introduce excessive information noise and lead not to clarity, but to a more complex perception. Therefore, it is very important to choose the right places for placing markers and their number.
In the initial version, we analyzed the approach to the uniform arrangement of markers on the map using some predefined grid. However, this approach turned out to be ineffective, since it does not allow to see local changes in small spaces that do not fall into the grid nodes. There are almost always extended areas with a small change in the indicator on the map, and there are small areas with large dynamics. In addition, in some cases, this approach leads to false hypotheses if the grid nodes fall into some insignificant areas of space with uncharacteristic changes. Therefore, in our further research, we abandoned this approach in favor of segmentation algorithms. Let us consider the main idea of this approach.
Let us have an image that displays some state of the spatial index. For example, it can be a vegetation
index showing the state of the plant mass at each point in space. We use NDVI index [
However, the approach described above has one big disadvantage. Using segmentation for the image of the geographic information index, we do not take into account areas with different rates and patterns of data change. The current sample may have a large area with the same index value, but from the point of view of dynamics, different sections will have a different character. For example, segmentation of one-time period did not allow to distinguish a field with almost no growth dynamics 3 and gradually growing 4, shown in Figure 1. So, markers are designed to display such dynamics, which leads to a certain conflict of goals and methods for achieving them.
Another option is to use segmentation not for the original image, but for the image of gradients that take into account the index change, its rate at each point in space. To do this, several index images are selected for different time intervals with a predetermined time delta and a gradient is calculated (the index difference at each point). Then, the resulting image of the gradients is used for segmentation, not the original image. At the same time, after receiving the gradient map, it is necessary to normalize it.
This approach will allow to choose a place for markers for displaying dynamics more objectively, since the resulting areas will have a similar nature of changes. However, it also has a drawback. As practice has shown, in some cases, the nature of the change is almost identical across the entire map. For example, this behavior is typical for the water index during drought. Different areas of the map will have different index indicators, but the overall trend of dynamics will be the same.
Taking it into account, we propose to modify the segmentation approach and implement it not on the original index image or on the gradient map, but on the combined image, combining both the current sample and the information about the dynamics of changes at each point in space. To do this, it is proposed at the first stage to form a three-channel image (color), in which different time periods are substituted as each color channel (red, green and blue). In this case, an image is obtained that takes into account not only the current indicators, but also the dynamics of changes. It is this color image that is used for segmentation and segment search for dynamics markers. The result of such segmentation is shown in Figure 5.
In general, taking into account the above concepts and peculiarities, the resulting algorithm is shown in Figure 6.
START K = 0
K < 3 Calculation of NDVI index at a time tk:
NDVI(tk)
K ++
Colour image formation
<RGB>: R = NDVI(t0) G = NDVI(t1)
B = NDVI(t2) Image conversion RGB
=> GrayScale
Image segmentation Calculation of segment centers (position of dynamics markers)
END
To conduct an experimental test of the hypotheses mentioned above, we formed a set of maps of
different places in Russia, obtained from free sources. One of them is Copernicus system [
A focus group was formed to identify the effectiveness and clarity of the proposed methods for visualizing the dynamics of change. This group includes both people who are familiar with the specifics of vegetation indices and features of remote sensing and monitoring of the Earth, and people who are not great specialists in the applied field. This was done in order to assess the clarity of the proposed approach to displaying data dynamics and its dependence on the level and readiness of the decisionmaker.
Each person from the focus group had two series of experiments. In the first one, he was offered a set of maps of the vegetation index for different periods of time of the same area and asked about the degree of vegetation change, the search for areas with great and minor changes, the nature of the change. At the same time, the time of analyzing the images by a person was measured. Also, after evaluating a series of images, a person was asked to answer the question of how confident he is in his judgments on a scale from 0 to 10.
The second series of experiments provided a person with images with the dynamics markers described above. A special feature of the second series of experiments is that before it began, participants were provided with a legend describing the designations of the dynamics markers. At the same time, the questions asked were the same. The results of the experiment are shown in Table 1.
The studies conducted have shown that among absolutely all participants, the time for analyzing remote sensing images is reduced up to 70%. This is quite an important aspect and confirmation of the viability of the hypotheses put forward and the proposed approach. This was especially noticeable in photos with minor changes, the analysis of which required the most time for the participants. At the same time, confidence in their judgments increases by 20% on average.
Among the interesting conclusions of the experiment are that the degree of confidence in their judgments did not rise to the maximum level of 10 points either among specialists or among ordinary participants. This allows to conclude that this type of visualization needs to be developed further and the visual clarity of the input characters should be increased even more.
It should also be noted that the trust threshold among specialists was slightly lower than among the rest of the group. On the one hand, this is due to the fact that specialists are used to analyzing the dynamics of a series of images and this is a familiar tool and process for them. On the other hand, he emphasizes that the proposed visualization option cannot replace traditional methods, but is intended to improve and supplement them.
The paper proposes a method for displaying a map of the dynamics of changes in geographic information indices using circular markers. As the experimental test on the focus group has showed, such an approach can significantly reduce the time for analyzing remote sensing data and reduce the requirement for the qualification of specialists. Also, this approach increases the degree of the expert confidence in his judgments. At the same time, this approach is not intended to replace traditional methods of analyzing time periods, but to supplement them, making the analysis process simpler and more visual. 6. References Proceedings of the 30th International Conference on Computer Graphics and Machine Vision, Saint Petersburg, 2020. DOI: 10.51130/graphicon-2020-2-3-49. [14] A.O. Trubakov, A.A. Trubakova, Visualization of the Dynamics of Changes in Structured Data by the Example of Covid-19 Development, in: GraphiCon 2020 – Proceedings of the 30th International Conference on Computer Graphics and Machine Vision, Saint Petersburg, 2020. DOI: 10.51130/graphicon-2020-2-3-24.