A distributed computing environment is being formed within the geoportal IDSTU SB RAS, which is mainly used for processing spatial data. Processing services that implement the WPS standard, and data storage services, a catalog of WPS services, and service execution systems and their compositions have been developed. In the system for executing services statistics data is collected on user-made service calls. The analysis of the obtained statistical data is carried out in order to identify the data transfer between services calls and build service compositions.
tion, programming languages are actively used to define service compositions. There
are a number of workflow systems that allow you to create service compositions and
have achieved significant results in various subject areas: Pegasus [
Within the framework of the geoportal IDSTU SB RAS, a distributed computing environment is being formed, which is mainly used for processing spatial data. Processing services that implement the WPS standard, and data storage services, a catalog of WPS services, and service execution systems have been developed. The interaction between processing services and data storage services is implemented, which allows simplifying the use of services by the user. In the system for executing services and their compositions, statistics of service calls is collected. The statistics data includes the name of the service and its address, values of input and output parameters, service execution time, successful execution, execution errors, etc. 3
We introduce the following notation necessary to describe the operation of the method.
s = <name, I, O> is the service, where «name» is the name of the service, I is the set of input parameters, O is the set of output parameters of the service. Below we will denote s.I and s.O parameters that belong to a particular service. A user service call has input parameter values specified by the user and output parameter values. t = <VI, VO> - service s call with values VI, VO of parameters s.I and s.O.
Log is the set of successful service calls made by users. The cause of the unsuccessful service call may be incorrect parameter values. Therefore, statistics are filtered and only successful calls are left. Based on the Log data, it is necessary to form a set of service compositions in the form of DAG.
To recognize a service composition, it is necessary to determine the presence of data transfer between two service calls. The determination of the relationship between two service calls can be made based on the analysis of parameter values. Parameters of service call can be divided into input and output. It is assumed that if the value of the output parameter matches the value of the input parameter, then there may be data communication between service calls. Parameter values can be string, numerical type, or resource URIs. Usually these are URLs to files, data services, etc. Each URI used to receive data uniquely identifies the transmitted data. Accordingly, using URI parameters, data transferring from one service call to another can be determined. Analysis of string and numerical parameters is more difficult, because of matching the values may be casual. Currently, string and numeric parameters are not considered.
Here is an algorithm for checking data transfer between two service calls. Input: ti, tj - two service calls Output: flag - connection between service calls and by what parameters function islinked flag <- False for each vm in ti.VO: for each vn in tj.VI:
If (typeof vm = file and typeof vn = file and vm = vn) then flag <- True vn.linkedwith <- vm end if end for end for return flag
This algorithm compares the output parameters of one service call with the input parameters of another service call. If the URL values of the parameters match, the connection between service calls is established. In addition, it is determined by what parameters they are connected. 5
To analyze the set of successful calls to Log services, the following algorithm is used, at the output of which we obtain a general directed acyclic graph DAGgen, where calls to ti services are connected by data. Additionally, we obtain the values of the parameters of services entered by the user.
Input: Log Output: list - semantic relationships between services for each ti in Log: for each tj in Log:
If islinked (ti, tj) then
add (ti, tj) into list end for end for
A user can perform a data-connected sequence of service calls that forms a connected subgraph Di (graph connectivity components) in DAGgen that is not connected to other subgraphs. Each such subgraph Di is a special case of the implementation of the composition of services. To search for Di subgraphs, the width search algorithm is used. Moreover, the complexity of the search is linear. It depends from the sum of the number of vertices and the number of edges of the graph DAGgen.
The user can often repeat the same sequence of service calls. This leads to the creation of several Di subgraphs in which calls of the same services are made, but the parameter values differ, i.e. creating isomorphic subgraphs Di, considering only services and data transfer. Checking for isomorphism of graphs is carried out by traversing oriented subgraphs.
As a result of the execution of the algorithm, we obtain several sets of isomorphic IDAGi subgraphs. The number of Dj subgraphs in IDAGi is an estimate of the frequency of use of this services composition. Next, filtering isomorphic IDAGi subgraphs is carried out, for which the number of vertices is more than one and there is at least one call to the service for publishing data. The resulting composition of services is shown in Fig. one. The proposed method, in contrast to existing approaches, uses statistics to build service compositions. To build a composition, the user only needs to sequentially perform the necessary services. The method automatically connects single service calls to each other, determines data dependencies. The application of the method is focused on the automation of frequently repeated user actions.
This work was supported in part by the Russian Federal Property Fund (grant 1807-00758-a, 17-57-44006-mong-a, 17-47-380007-r), the program of the Presidium of the Russian Academy of Sciences No. 2, the program “Information and telecommunication platform for digital monitoring of Lake Baikal, based on end-to-end technologies”, Shared Equipment Centers of ISDCT SB RAS. 16. Kwok, Y.-K. Static scheduling algorithms for allocating directed task graphs to multiprocessors / Y.-K. Kwok, I. Ahmad // ACM Computing Surveys. 1999. Vol. 31, № 4. P. 406 – 471. 17. Xie, G. A High-Performance DAG Task Scheduling Algorithm for Heterogeneous Networked Embedded Systems / G. Xie, R. Li, X. Xiao, Y. Chen // Proc. of IEEE 28th International Conference Advanced Information Networking and Applications. 2014. P. 1011– 1016.