In this paper, we discuss how data integration can improve the quality of oil pipelines monitoring systems. We outline approaches that have been developing for collecting data from different type of sensors and describe steps of designing a Program Apparatus Complex (PAC). Various aspects of this problem are included: a brief overview of related works and existing monitoring systems and their development prospects; data integrating approaches; requirement analysis for computerbased data integration.
Modern development and operation of the Russian oil pipeline network take place in subarctic environment that is characterized with sharp temperature fluctuations and unfavorable soils (wetlands, permafrost). It provokes internal and external corrosion, denting, buckling, object reorientation and deformation. In the course of time the number of exogenous processes increases. It makes necessary to install additional sensors and monitoring systems and formulates a plan for compensating activities in a form of diagnosis, repair and replacement of equipment.
At present there is а growing interest in data integrating platforms. Fig. 1 was
obtained with semantic search in the Exactus Patent system based on semantic analysis
of more than 64 thousand patents issued in the countries developing this problem
(Great Britain, USA, Canada, Russia, Israel, Germany, etc.). Articles also confine
attention to this aspect. The survey [
It includes accelerometers, underground acoustic sensors; optical, thermal and
hyperspectral video cameras or radar systems mounted on strategic areas. [
There are many types of the oil pipelines monitoring systems with unique feature and operational condition that have been studied (table 1). Linear dimensions, linear and angular movements, bending radius, oscillation amplitudes Tenso- and vibro- parameters, product\wall temperatures, distribution of protection voltage, metrological parameters, characteristics of pumpage product Vertical and horizontal ground vibration and movement Location and size of dangerous geological processes, humidity and density, strength, plasticity and soil deformation module, filtration factor, ice content, degree of heaving, salinity, porosity, heat, corrosive activity Ambient temperature, soil and permafrost temperature, temperature and level of groundwater, efficiency and effectiveness of the thermostabilizers, the depth of the seasonal freezing defrosting Depth of the oil seams, geoelectric conditions Metal Characteristics (wall thickness, steel mark, diameter, chemical composition, type and category of tubes), weld characteristics, insulation parameters, rheological characteristics Water specimen (temperature, component composition, acidity, current velocity, flow), air, pairs of oil products from reservoirs
Monitoring System Inline inspections, additional defectoscope control Remote data acquisition and control system (RDACS), Leak detection system (LDS), Corrosion protection system (CPS) Seismic effects monitoring system (SEMS) GIS, subsurface exploration, aerial survey, remote sensing, geodetic measurements System of geotechnical monitoring (SGM) GIS, geophysical surveys Electronic passport of pipes and pump stations’ equipment Environmental monitoring system
In the daily operation of oil pipelines, a lot of decisions have to be taken that
affects the volumes produced and the cost of production. These decisions are taken at
different levels and related to the choke/valve openings, compressors, pump settings,
etc. at every instance of time. These are the control elements of RDACS [
LDS, by itself, has no effect on the technical integrity of the pipeline as it is only
installed to make the operator aware of a leak in order for the operator to manage the
risk of pipeline failure [
The first important aspect related to the seismic response of civil and industrial
constructions is the proper measurement of the shaking level at the site of interest [
When a seismic event occurs, the SEMS determines the level of danger for a pipe and pump stations and gives warning or stop signals.
The SGM estimates the displacement of ground by inclinometric control points,
groundwater control points to calculate the stability of the landslide slopes, the points
of control of a pipe position to assess the effects of loads in the development of
dangerous geological processes [
The problem of data integration became more acute as far as enlisted monitoring systems work independently with sensors controlling only their technological parameters without knowing what is happening to the others. The solution of the problem is a human-operator who makes decisions. As a result, the quality of monitoring depends on experience and skills of operators. From the safe operation point of view, functional separation of controlled parameters is an artificial technique. There is a need to reduce its negative effects so that an information flow reflects a real change in pipeline maintenance. The trend of pipeline monitoring systems development wends the way of centralizing the top level to meet the challenge of complex, interrelated control of all sets of parameters, as well as to provide users with access to measurement data from any of the terminal devices of the corporate computer network.
Thus, a significant number of control nodes that are remote from each other for thousands of kilometers and equipped with heterogeneous sensors, the independence of contours over controlled parameters lead to an uncontrolled increase in the amount of initial information and need to develop methods for integration of distributed diverse information. 3
The detailed analysis of background information and technical reports let us to formulate a list of requirements for integration of data from different monitoring systems. We can group them into three broad types, namely: 1 - harmonizing, 2 - combining and 3 - merging. First group includes stages of interaction with sensors, estimation of reliability and quality of source data, analysis of data structures, restoring dimensions, description in terms of subject areas, formatting. Second group will consist in noise\trends detection and elimination; deduplicate and restore data; statistical processing and calculations with validation of the result. Last group contains merging at data and relation levels, deep learning of prepared data, modeling and forecasting. 4
Sensors installed in thousands locations along the route, produce values of the controlled parameters in real-time regime. They are recorded in local databases and archives of different structures and formats (Fig. 2). It laid the foundation for designing a computational suprasystem that contains web-services operating with every class of process, global database and archives storing data, and computational methods.
Each monitoring system that collects data is designed for the specific purpose of certain types of users, but same data may be useful for other types of users (economists, managers). Preliminary analysis of user profiles allows to develop specified methods for locating patterns in separate and intersecting sets of data and to interpret them correctly. For example, modeling thermal fields along the pipeline route and selecting subsequences from measurements, are typical for sustainable soil defrosting and allow early detection of anomalies and classification of the route segments.
Algorithms for detecting similar subsequences, use of search optimization techniques (indexing, dismissing unlike subsequences), various similarity measures, and dynamic transformation of the timescale that allows comparison of time series obtained at different speeds of data change. A discrete Fourier and wavelet transformations, suffix trees, and others are used to build the spatial index.
Data integration process can be described as follow [
As far as data integration process is layered, R can be detailed for processes of different levels. A flow of raw data is accepted for preprocessing and a partial aggregation is generated, which then adds the preprocessed data from another monitoring system. The result of iterations is a complex aggregation.
Thus, novelty of the proposed approach is include: ─ the service-oriented architecture, absorbing installed monitoring systems; ─ the methodology for implementation and remote deployment of a suprasystem; ─ verification of actual values of the controlled parameters; ─ identification wrong trends and noticeable variances in the controlled parameter values; ─ optimization of a number and frequency of the controlled parameters; ─ plans to implement compensating activities. 6
Pipeline operators face many threats to the integrity of pipelines. The proposed suprasystem is based on data flows and knowledge-retrieval techniques that make it possible to shape control effects by combining real-time data with already available information. At the stage of integrating data, received from multiple sensors and affixed to the available information (meteorological and geological data) we produce additional information, that is valued by ecologists, economists, geotechnicians and others. It is possible to analyze data with a variety of devices that recognize context of a corporate computer network. Using deep-learning algorithms that indicate patterns in data, it is possible to predict pipeline’s maintenance.