The requirements of modern legislation in the field of citizen security pose serious challenges to various organizations, including data storage. The anti-terrorist amendments adopted in 2016 (the socalled "Yarovaya law") obliged telecom operators to store traffic metadata for three years, and the traffic itself for six months. In addition, in June 2020, the Ministry of Digital Development, Communications and Mass Media of the Russian Federation proposed a bill, according to which the owners of technological communication networks are required to store traffic for three years [1].
There is also a legislative norm obligating to increase the capacity of traffic storages by 15% annually. Even though the government has postponed the introduction of this norm for 1 year, the problem of using and extending the physical resources of the storage is very acute.
The volume of Internet traffic over the past 4 years ranged from 32470.782391 PB to 61,226.217838 PB (Fig. 1) [2]. That is, in just four years, the volume of traffic has almost doubled. Consequently, the above norm on the annual increase in storage capacity is insufficient, while its implementation requires significant financial costs. In October 2020, only "Rostelecom" spent 7.8 billion rubles on data storage equipment. Other operators also purchase various storage systems (table 1) [3]. As can be seen from Table 1, telecom operators of the Russian Federation suffer serious financial costs for the purchase of equipment for storage systems. On the other hand, modern information technologies make it possible to manage the resources of data storage systems, use them efficiently and, therefore, avoid unnecessary costs.
The data storage system can manage the recording of the incoming data stream and, firstly, distribute it among different types of media, and secondly, monitor the state of the storage and make a forecast of capacity growth for its timely extension.
A research a study has been carried out in which a data storage system is considered as a storage management system that performs the following functions:
• Distribution of data files on various types of media, depending on the file size and storage time • Monitoring the storage state based on snapshots of each media state • Forecast of storage capacity extension. [4,5]. The storage management system diagram is shown in Figure 2. Obviously, for the implementation of such a storage management system, a soft-ware-hardware system is needed that performs the above functions.
It is proposed to include a programmable logic controller (PLC) in this system, which distributes files to media and software that monitors the state of the physical storage and builds a forecast for its extension (Figure 3).
The controller receives an incoming data stream (for example, internet traffic). The controller performs clustering of incoming traffic using Kohonen's neural networks and, in accordance with the resulting topological map, distributes data files to media in the physical data storage.
Physical storage can be organized depending on the information being recorded. In paper [6] there was considered a 3x3 matrix storage and assumed the distribution of files first by one of the storage levels, depending on the storage time, and then -the distribution among the level volumes depending on the file size.
This solution can be easily adapted to the needs of the owners of technological communication networks [7]. Since the storage time of data files, as well as metadata, in accordance with the existing legislation and the upcoming amendments is limited to three years, data files can be distributed across various types of media de-pending on the type of data (text, sound, video) and size. The structure of physical data storage is determined by the storage system administrator and can contain, for example, RAID arrays for text files, streamers for audio and video files. In addition, volumes inside a RAID array can have different operating systems with different sizes of the logical data block, which will avoid the "under-filling" of files during writing (Figures 4, 5).
where tlimmntime to reach limited media capacity;
tmaxmntime to reach maximum media capacity; f(t)incoming data function; Tpartition step equal to the unit of the minimum selected time scale. The forecasting task is to find the timeline point at which the limited capacity and the maximum capacity of each media are reached [5].
To solve this problem, it is necessary to predict the amount of incoming traffic in the storage system. The forecast can be made by various methods, while it is necessary to consider the peculiarities of the data stream entering the recording. Due to uneven user activity associated with weekends and working days, vacation periods, etc. the incoming data stream is heterogeneous and has a seasonal structure (Fig. 7)
In [6,9], a comparison was made between different forecasting methods: statistical forecasting using an autoregressive model and an integrated moving average (ARIMA) and machine learning methods. The results showed that the ARIMA model is the most suitable for short-term forecasts (Fig. 8), and for mid-term forecasts, machine learning methods (Fig. 9). To implement the forecast mechanism, an application was developed. This application helps automate the process of predicting the capacity extension of each cell of the storage matrix [10].
Thus, for the further implementation of the hardware-software system, it is necessary to develop a programmable logic controller that distributes files inside the physical data storage.
Programmable logic controllers are widely used in automatic control systems. The performance of modern controllers allows them to use the most efficient control algorithms, such as, for example, neural networks. Real Traffic Prediction
The norms of modern legislation oblige the owners of technological communication networks to store a large amount of data using their own data storage systems. This leads to serious costs, which, in the end, fall on the end user of communication services.
At the same time, modern technologies make it possible to create systems for managing physical data storage that can efficiently consume physical storage resources. Existing virtualization technologies make it possible to create structures containing various types of storage media and distribute the saved traffic files over them depending on certain characteristics of the files.
Since there is a need for regular scaling of the data storage, it is necessary to monitor its status and scale only those media whose capacity limits tend to be maximized. Predicting capacity extension allows for timely scaling.
Thus, dividing the total incoming data stream by media, predicting capacity consumption, and monitoring the state of physical data storage allow owners of technological communication networks to rationally use physical storage resources and avoid unnecessary costs when increasing storage.