The paper discusses the requirements for the system of remote sensing (hereinafter referred to as “System”) which is focused on the end user (EU), based on the concepts of object-oriented monitoring. The classification of simple queries generated with interactive tools in the context of the end user's subject area is presented. The research investigates the features of the System needed for maintaining task-setting interaction, such as the subject ontology, including knowledge about the object of interest, its static and dynamic properties, the hierarchy of vector layers that describe spatial objects, the updated image database. The authors indicate the resources necessary for solving specific tasks, including spatial data repository and specification of computational procedures capable of interpreting the query in terms of decision operators. Specifically, the paper considers the structure of task-setting interaction tools, which ensures the object space-time localization, finding its meaning in semantic space, performance specification parameters measured one time or in dynamics, requirements for the visualization of the results and the activity of the System in relation to the end user in the monitoring process. The solutions presented were tested on monitoring the condition of agricultural lands in the multi-task space monitoring system of the Institute of Space and Information Technology, Siberian Federal University, Krasnoyarsk.
At present there is an intrinsic gap between the existing interfaces of geographic information systems
and the information need of the end user, who, as a rule, operates within a closed professional
environment. New tools are needed to reconcile the mental model of the end user, on the one hand,
and the totality of the processed data, the algorithms of data assimilation, presentation and processing
on the other. They should provide the end user with an interactive environment where he could set
monitoring tasks in the usual terms of objects, attributes, classifiers, indicators, charts, graphs, etc. The
most intuitive in this sense is the object-oriented paradigm to analyze complex subject areas, which is
often used in practice to set the task of implementing software solutions that perform specific
functions [
The main variables discussed in setting the task of spatial object remote research are: the localization of the object of interest (context) and the informative characteristics to be determined.
The object is localized in the coordinate system of the earth’s surface, in time, and in semantic space.
Spatial localization can be carried out based on the use of a priori information about the study object (SO) position, represented as a pair of points defined in space by coordinates (x, y), which identify rectangular search area, or a polygonal vector object represented in one of the GIS- formats. In addition, the position of the object can be specified by the user directly, based on substrate maps, or by specifying the elements of the existing vector maps, for example, administrative or cadastral division, departmental affiliation, land use maps, etc. The position of an object can also be specified indirectly, based on its non-spatial properties, defined by the user and applying the knowledge of the System.
Localization in time can also be stated either explicitly, by specifying a time reference or interval, or indirectly, based on the characteristics of the object life cycle. Depending on the nature of the problem being solved, the following time localization options are possible: “now”, “exact date in the past”, “last measurement results”, “forecast” (for a future date), “monitoring” (a series of measurements with a specified interval).
The process of localization in semantic space, which, unlike the previous two methods of localization, depends essentially on the nature of the object and the subject area of research, is characterized by the greatest variability. In the first approximation, we assume that the specified process comes to choosing one or more classes from a predetermined set of domain classifiers. The examples presented in the article are given for the subject area of detecting agricultural land (AL) use with remote sensing.
Substantive characteristics to be determined can be divided into static, dynamic and structural. Depending on the nature of the problem being solved, it can be formulated as an object recognition task, a measurement task, or a prediction of the numerical value of one or more object attributes, a segmentation task (for heterogeneously structured objects). In the latter case, it is also possible to set the task of recognition or measurement for each of the selected segments. The recognition task requires the assignment of the observed object to a class defined in one of the domain classifiers. The statement of the measurement (forecasting) task requires the correlation of the measured property with the corresponding element of the attribute classifier.
Formally, the structure of interaction on setting the object monitoring task can be represented as a directed graph:
Gr (P,V ) , (1) where P is the set of vertices defining the nature of the action (operation) in the interaction, V is the set of directed edges defining the execution paths for operations. The set of the graph vertices consists of two subsets:
P S EU , (2) where S is a subset of the operational vertices of the system, EU is a subset of the operational vertices of the user.
The main stages of the task setting: 1) defining the context; 2) clarifying the goal; 3) developing the requirements for representation of findings.
The object is localized along three “axes” of interaction: in the coordinate system of the earth’s surface, in time and in semantic space, each of which is related to an interaction frame. The preferred direction of interaction is determined by the user. For example, when choosing a spatial axis, the user can specify the spatial coordinates of the desired point of the monitored object. If the object is recognized by the system, it notifies the user about the main characteristics of this object, thereby setting localization indirectly in the semantic space. In another possible scenario, the user can select one of the object classifiers and specify the required classifier vertex. When switching to a frame with a geospatial representation, the areas corresponding to the selected vertex at the current time are activated. When switching to the time frame, the user can select the observation interval. If an interval in the past is selected, corresponding changes should occur in the spatial frame, displaying the required period of the object observation.
In the general case, various localization results are possible, affecting the further course of the task solution process. Thus, the result of localization in space is the exact fixation of a single spatial object (AL), a set of objects (AL sets) united by common features, for example, spring wheat fields of “MAYAK" agricultural company, or indication of the region of interest where these objects can be located. When setting the time interval, we determine the need for: a) obtaining immediate information; b) obtaining retrospective information; d) monitoring; c) forecasting.
For an object (class of objects) defined during the previous stage of interaction, we determine the set
of characteristics to be evaluated. A set of characteristics available for selection depends on the model
for describing semantic characteristics used in the System. In our example, these are properties
associated with the corresponding nodes of the objects classifiers, such as “area”, “degree of
heterogeneity” [
When setting the forecasting problem in the subject area under consideration, as a rule, we assume the expected phytomass volumes, or the yield per unit area. At the considered stage of interaction, the appropriate forecast model is selected from those available in the System.
When setting the monitoring task, as well as the task of current observation, one or more characteristics to be monitored, as well as the method of organizing the monitoring process, are determined.
At this stage of interaction, the forms of findings representation on request are refined - numerical values, tables, elements of business graphics, thematic maps, and atlases. Since there is no complete formalization of tasks in practice, the resulting images of the study objects are also of interest to the user. They can be analyzed together with the information structured above.
When setting the monitoring task, particular interest consists in defining the probability of the System prompt response to the anomalous behaviours of the study object, detected during the analysis of timely remote sensing data. For example, when monitoring fallow lands, these are the excessive values of vegetation indices, indicating the presence of phytomass in the field and, indirectly, the use of inadequate tillage methods. The active method of monitoring provides for the possibility to notify the user about negative scenarios of agricultural lands dynamics immediately when such information is obtained by the System.
A subject knowledge base pre-formed in the System, which restricts the class of study objects defining
the area of study and information about their spatial localization, is a prerequisite for organizing an
interaction. The research relies on the ontology-based approach to describe and extract the semantics
of agriculture objects, developed in [
The other necessary components of the system are: a cartographic base and a set of vector layers
characterizing the spatial localization of objects and their attribute characteristics describing factual
information. To organize monitoring, the System should have access to space survey data that arrives
at the intervals determined by the characteristics of the object's life cycle. Accordingly, it is crucial to
maintain a constantly updated database of remote sensing, which has assimilation, preprocessing,
indexing, storage utilities and access to the specified data, as described in [
For each of the attributes calculated in the system, we should present a computational procedure to
determine the sequence of spatial data processing operators and the conditions for their use [
For active monitoring, as a supporting capability we need a formalized model of the object
development over time. Thus, the crops vegetation period includes an interconnected series of
phenophases changes, anthropogenic (farming operations) and environmental influences, which can be
interpreted as positive or negative in a specific context [
The main provisions of the concept of man-machine interaction for setting and solving different tasks of spatial objects remote monitoring were implemented for obtaining the crops condition information, taking into account their uneven growth over time. Within the structure of interaction tools described above, the task setting process can be decomposed into the following steps (Figure 1):
The stage allows introducing priori descriptions in the form of thematic sub-classifiers containing the
monitoring feature classes, as well as their unary and binary properties. For the task in question, the
structure of the classifier is formally as follows [
ci ID,{x, y},Ф, K, E,Tm, F, NS, NP,T , N , (3) where ci is the classifier of the AL objects, ci С , С is the global taxonomy classifier; ID – is object identifier; {x, y} – is the set of geographic coordinates of the object in a given projection; K – is the set of agricultural crops (property-object); Ф is the set of phenological phases throughout the growing season; E – is the sequence of planned farming operations (FO); Tm is the set of reference time intervals ei E , ei (ki , tmi ) and i Ф , i (ki , tmi , N i ) , where tmi is the time of measurement, and ki K ; N - is the set of reference intervals of vegetation index values, associated with phenological phases, allowing to interpret the current condition of vegetation; F is the set of the object conditions. The relation fi (N i ) is defined. In addition, the measured properties include NS, NP,T – area, perimeter and thickness, respectively.
Setting the position of an object in space is the part of interaction. Based on (1) and (2), its model, can be represented as a graph Gr1 (Figure 2). The object localization model includes the following specific steps:
The end user localizes an object (or objects, by multiple selection) on the basis of the “AL” object map ( EU11 ).
The system identifies an object on the map visually ( S 21 ).
The system displays an object operation panel ( S31 ).
The end user assigns the object to its agricultural company (AC) ( EU 21 ).
The Information System (IS) sets the ID j of the assigned object, fixes its coordinates {xi , yi} ( S41 ).
S11
The interaction of assigning a crop can be represented as a graph Gr2 (Figure 3), which includes the following steps:
The system displays a map of the “AL” objects assigned to the agricultural company ( S12 ). The end user localizes an object (or objects, by multiple selection) on the basis of the object map of the company's agricultural lands ( EU12 ).
The system identifies the object on the map visually ( S22 ).
The system displays an object operation panel ( S32 ).
The end user assigns the crop ki with the normative time points Tm for updating the state of the object, i.e. measurement frequency of specified characteristics to the selected object ( EU 2 ) on 3 the basis of a given classifier. Each object can have individual time intervals between the state changes, which are measured from the actual sowing date.
The system displays a map of "AL" objects with the given crops ( S62 ).
The interaction graph Gr3 depicts the process of setting the properties of an object to be monitored (Figure 4). The main steps of interaction are:
The system displays a map of "AL" objects assigned to the agricultural company ( S83 ). The end user localizes an object (or objects, by multiple selection) on the basis of the “AL” objects assigned to AC ( EU13 ).
The system identifies the object on the map visually ( S23 ).
The system displays an object operation panel ( S73 ).
The end user sets restrictions on the values of size NS ( EU 3 ) and form ( EU 3 ) parameters to be 4 5 calculated.
The end user confirms the request ( EU 63 ).
The task solution stage is considered as an information request which is impossible without prior training a multi-task monitoring system. The learning processes lie outside the framework of the considered interaction and consist in establishing the relation between subject classifiers and preprogrammed algorithm being program modules. The modular architecture implements the principle of reusing system components for solving various tasks, including those in parallel mode. At the physical level, this approach enables to represent a given operation for processing or analyzing data as a separate module, and a set of operations as a set of logically interrelated software modules running in a given sequence, with a given set of input parameters, at a specified time.
Thus, when choosing an object of a certain class, the System knows in advance which algorithm is required to be executed. If several algorithms are available for a class of objects, then we should provide steps how to choose the relevant one. At the same time, taking into account the fact that this interaction is user-focused, it is possible to use the natural language form of the search algorithm, which will in essence have the form of interaction “NL request - function”.
So, in the considered task of monitoring crops condition, taking into account their uneven growth over time, the object class “AL” is connected with an algorithm for localizing a spatial object with heterogeneous structure. The practical result of this task solution is a series of “AL” object maps displaying structural heterogeneity with the interpretation of conditions, some of which are presented in Figure 5. The results are visualized to the user as vector layers in the Monitoring System. On the one hand the number of the formed layers is limited by the number of time points of the object properties being actualized, on the other hand, by remote sensing data, which are available under favorable weather conditions.
The paper proposes a concept of designing an interaction environment for the end user of the remote sensing system for spatial objects detection and recognition within the paradigm of object-oriented monitoring. The following issues were considered in detail: 1) Initial data and constraints for the problem setting, which are grouped into the following categories: the representation of the object of interest in time, in the coordinate system of the Earth’s surface, in semantic space, and in the target characteristics. 2) Interaction dialogue frames needed for a System to interact with the end user, including knowledge base for the subject area of research, a set of vector layers characterizing the spatial localization of objects and their attribute characteristics, the remote sensing data stream, implemented in the form of regularly updated image database or web-services, as well as functionality for measuring the required properties of the object. 3) The process of task setting in terms of interaction steps, which help to define the necessary context; goal clarification; development of requirements for representation of findings.
The interactive tools were developed based on software prototyping within the project “Multi-task space monitoring system of the Institute of Space and Information Technology, Siberian Federal University, Krasnoyarsk”.
«This study was supported by the Russian Foundation for Basic Research (grant №: 18-47-242002 p_mk), the Government of the Krasnoyarsk Territory, the Krasnoyarsk Regional Science Foundation as a part of the research project titled “Development of technology for creating intelligent information systems of object-oriented monitoring of territories based on remote sensing data".