In order to demonstrate the current importance of integrating a neural network with OLAP technology, the article highlights the main characteristics of OLAP systems that perform online analytical data processing. These systems, based on OLAP technology, are widely used both in government agencies and in private ones. The main characteristics, features and structure of OLAP systems are mentioned. The article emphasizes that OLAP is a data warehousing tool. OLAP allows analysts to explore and navigate a multidimensional structure of indicators called a data cube or OLAP cube. Indicators (measures) of OLAP cubes play an important role in the decision-making process. To solve some problems, these measures often need to be classified or clustered. Moreover, empty measures are common in OLAP cubes. Empty measures can present due to non-existing facts in data warehouse or due to empty cells which are unfilled in by mistake. The presence of empty measures negatively impacts strategic decision making. Unfortunately, OLAP itself is poorly adapted for forecasting empty measures of data cubes. Over the years, researchers and analysts have tried to improve the decision-making process in OLAP systems and add forecasting and other options to OLAP applications. Today, in the era of Industry 4.0, with the availability of big data, there is a need to apply new technologies to solve such problems. These technologies include neural networks. The article examines the problem of integrating OLAP and a neural network. In this regard, the article provides information about neural networks: information about their properties, types, as well as their capabilities. The article shows the possibility and advantages of integrating OLAP and neural network. It mentions that in the case of big data, the integration of OLAP and neural networks is very effective for solving problems of classification, clustering and prediction of empty measures of OLAP cubes. An architectural and technological model for integrating OLAP and neural networks is presented. It is noted what types of neural networks can be used to solve the problems of classification, clustering and forecasting specified in the model.
Artificial intelligence (AI) technology is a product that can provide people with the necessary
conveniences for everyday life [
The term “Artificial Intelligence” was first coined in history in 1956 by John McCarthy. The term refers to any system capable of performing creative functions and solving problems that typically require human intelligence, that is, which are traditionally performed by humans. In effect, AI imitates human intelligence in machines that are programmed to think and act like humans. AI plays a critical role in today’s world by enabling automation, improving decision making, increasing efficiency and productivity.
It opens up new opportunities for innovation and growth in a variety of industries, including healthcare, finance, manufacturing, transportation, e-commerce, education, goverment administration and more.
One area of AI is machine learning (ML). That is, the concept of AI is a broader concept
compared to the concept of ML. The idea of ML is that machines should be able to learn and adapt
through experience, making predictions based on statistical data collected by computers.
Another area of AI is neural networks (NN). Today, artificial NNs have firmly entered our lives and
are actively used where conventional algorithmic solutions are not effective or their use is
impossible.NNs are used to recognize hidden patterns in raw data, for clustering [
Recently, both government agencies and private ones have widely used OLAP systems based on
OLAP (Online Analytical Processing) technology [
The goal of this paper is to study the possibility of integrating OLAP and NN, to identify the benefits of this integration, since NN and OLAP are important tools for quickly and efficiently discovering valuable non-obvious information from a large collection of data, as well as to develop a model for integrating NN and OLAP.
The second section describes the main characteristics of OLAP systems. The third section provides information about NN: historical background, types of NN, their functions. The fourth section reviews related work. The fifth section is devoted to the problem of integrating NN and OLAP; an architectural and technological model for integrating OLAP and NN is presented. Finally, the sixth section presents the final conclusions of this article. 2.
An OLAP system is an information and analytical data processing system developed based on OLAP
technologies. The popularity of OLAP is explained by the fact that it is possible to solve many
problems with its help, namely: to implement operational processing of information, including
issuing information in various sections and dynamic report generation and its analysis based on the
data obtained, to perform monitoring and forecasting [
A DW is a source of processed information accumulated from already existing systems of
geographically distributed units. A DW is a domain-specific, non-volatile, integrated, time-varying
set of data for decision support [
The OLAP server is the core of the system, with the help of which multidimensional data structures are processed and communication between the DW and system users is ensured. Applications are used for user work. They formulate queries and visualize the responses received. OLAP applications are used to store DW analysis contexts in multidimensional data structures, i.e., in OLAP cubes. OLAP cubes enable analysts to explore information and report through interactive, easy-to-use dashboards. It is OLAP cubes that contain indicators (measures) used for analysis and management decision-making.
One of the important goals of OLAP is to make decisions based on historical data. Note that OLAP provides any analytical report within a few seconds due to its wide visualization functionality.
Recently, the volume of data has increased dramatically. Figure 1 shows the movement of data flows from lower to upper levels.
DM1 Fuzzy logic
DW
DM2
OLAP
DMn Blockchain Note that a Data Mart (DM) is a slice of a DW focused on one area of an organization’s activities. DM provides easier and faster access to data within a specific department of an organization because there is no need to waste time searching through a more complex DW.
In addition, the process of information accumulation is characterized by the fact that the data entering OLAP systems can be structured, poorly structured or unstructured. Even if we assume that this accumulated data comes in a structured form, the data comes from different sources, therefore, it can be presented in different formats. In this case, serious problems arise in the operation of OLAP. In this regard, the need to use new technologies and approaches in OLAP systems that transform classical OLAP into smart OLAP has become obvious.
Figure 2 presents a number of such technologies and approaches.
These include blockchain technology, NN, Data Mining, and fuzzy logic theory.
Some of these technologies have already been researched and applied in OLAP systems.
For example, article [
The article [
In the next chapter, we will look at different types of neural networks to obtain effective results when integrated with OLAP.
The basic principles of NN operation were described back in 1943 by Warren McCulloch and Walter
Pitts [
NNs, like biological ones, are a computing system with a huge number of parallel functioning simple processors with many connections. NNs have properties inherent to the human brain. This is learning from experience, generalization, and the extraction of significant data from a large amount of information.
NNs, like the human brain, have the ability to learn. In this case, learning refers to the process of
adjusting the network architecture and the weights of synaptic connections that influence the
coefficient signals in order to effectively solve the problem. Note that network architecture refers to
the structure of connections between neurons. Typically, a neural network is trained on a certain
sample [
Each NN includes a first layer of neurons called the input layer. This layer does not perform any transformations or calculations; it receives and distributes input signals to other neurons. This layer is the only one that is common to all NN types.
We will consider the main types of neural networks to determine which problems it is more
effective to use each of them for. The main types of NN are as follows [
Single layer neural network. It is a structure for the interaction of neurons, in which signals from the input layer are immediately sent to the output layer. The output layer converts the signal and immediately produces a response (Figure 3). The first layer (input) receives and distributes signals, and the required calculations are implemented in the second layer.
Input signals
Multilayer neural network. This NN, in addition to the output and input layers, has several hidden intermediate layers. The number of these layers depends on the complexity of the NN (Figure 4). Multilayer NNs has much more capabilities than single-layer NN, since each hidden intermediate layer is a separate stage at which information is processed and distributed.
NN can be classified not only for the number of layers, but also according to the direction of information distribution along synapses (connections) between neurons. These include:
Feed-forward neural network (unidirectional). In this structure (Figure 5.) the signal moves strictly in the direction from the input layer to the output layer. The signal does not move in the opposite direction and is, in principle, impossible.
Recurrent neural networks (with feedback) (RNN). Here the signal moves both forward and backward. As a result, the output result can be returned to the input. Types of RNN: • One to one • One to many • Many to one • Many to many
Self-organizing maps. These include self-organizing Kohonen maps. They are a powerful, selflearning clustering engine: the results are displayed in compact and easy-to-interpret two-dimensional maps. Thus, Kohonen maps combine two important data analysis paradigms - clustering and projection, that is, visualization of multidimensional data on a plane. The Kohonen map is used for exploratory data analysis. It is able to recognize clusters in data and also establish class proximity. The Kohonen network has only two layers: input and output, composed of radial neurons of an ordered structure. Note that the output layer is also called a topological map layer, or “screen”. The output layer neurons are located at the nodes of a two-dimensional grid with rectangular or hexagonal cells. The number of neurons p in the grid determines the degree of detail in the result of the algorithm, and, ultimately, the accuracy of the generalizing ability of the map depends on this (Figure 6).
The Kohonen card has the ability to predict client behavior. (Figure 6). If it is built a Kohonen map containing clusters for each group of clients according to their degree of loyalty, then with its help the expected behavior of the client can be predicted and applied appropriate marketing policies to them.
The winning neuron
The Kohonen map is also capable of detecting anomalies. It distinguishes clusters in the training
data and assigns all data to one cluster or another. If after this the map encounters a data set that is
unlike any of the known samples, then it will not be able to classify such a set and thereby reveal its
anomaly [
There are also other criteria for NN classification: depending on the types of neurons: homogeneous and hybrid; depending on the NN learning method: supervised learning, unsupervised learning, reinforcement learning;
according to the type of input information, NNs are: analogous (use information in the form of real numbers), binary (operate with information presented in binary form); figurative (operate with information presented in the form of images, signs, hieroglyphs, symbols);
according to the nature of synapse setup: with fixed connections (NN weight coefficients are selected immediately based on the conditions of the problem, with dW/dt=0, where W denotes NN weight coefficients); with dynamic connections (when the learning process is in progress in the settings of synaptic connections, that is, dW/dt≠0, where W denotes NN weight coefficients).
Note that NN uses various activation functions to normalize the input data. The most widespread among them are:
linear function f(x) = x. This function is the simplest of all, and is usually used only for testing the created NN;
sigmoid. This function is the most common activation function. The range of its values is from zero to one. This is a smooth monotonic function, shaped like the letter S, and has the following formula: •
= !!!!!!; hyperbolic tangent. Used to cover negative values as well. When their use is not intended, the hyperbolic tangent is not needed.
The need to add new capabilities to OLAP is due to the fact that when solving some tasks, problems arise due to the presence of big data entering the systems from different sources, as shown in Figure 1. Big data is a huge volume of heterogeneous and quickly arriving digital information that cannot be processed with traditional tools. The use of big data is important in various fields including healthcare, finance, education, government administration and many others.
On the other hand, it should be noted that today there is a need to solve various kinds of intellectual problems. Intelligent problems based on big data are most effectively solved using NNs. Using NN, big data can be analyzed efficiently. It is very important that the advantage of NN, such as the detection of hidden patterns that are invisible to humans, also is especially effective for big data. The integration of OLAP and NN provides these benefits. Thus, it can be argued that the integration of two technologies such as OLAP and NN is very useful and important, as it enriches each of them: OLAP is the ability to navigate the multidimensional structure of indicators, and NN is the ability to intelligently solve problems on large amounts of data.
In addition, when there is a large amount of data in intelligent systems using OLAP, NNs are effective for solving clustering and classification tasks. Note that clustering, unlike classification, does not have predefined categories into which all data should be grouped. In this case, the NN itself generates clusters based on common features of the data.
Clustering is one of the most important methods of data analysis. The article [
One of the classes of NNs primarily used to solve clustering tasks is the Kohonen neural network
[
In the decision-making process, you can encounter many fuzzy tasks. Therefore, the queries to the
DW that the analyst is trying to formulate may often contain uncertainties. Clustering applied to the
dimensions of an OLAP cube using NN produces term sets. This will make it possible to solve fuzzy
tasks as well [
To the clustering methods that were noted above (PAM, CLARA, CLARANS), also include the kmeans method. k-means is the most popular and simplest clustering method. Its main disadvantages are: you need to know the number of clusters in advance; very sensitive to the choice of initial cluster centers.
For data mining, the task of data classification plays an important role. Currently, a large number of different types of classifiers have been developed, including those built on machine learning. These include NN. Although the classification task for NNs is not the main one, their use has a number of advantages:
NNs are self-learning models, the operation of which requires almost no user intervention; NNs are universal approximators that allow you to approximate any continuous function with suitable accuracy;
NNs are nonlinear models. This allows you to effectively solve classification tasks even in the absence of linear separability of classes (Figure 7) [19].
Linearly separable classes
Linearly inseparable classes
By means of NN, forecasting problems that are of great importance in the production, economic and financial spheres are also solved. Forecasting in OLAP is important because when looking at the contents of a cube, it can often be sparse, meaning it is missing some measures, and may also be missing dimensions. This happens due to missing information or input errors. The absence of any measures and measurements is undesirable and can lead to incorrect analysis when making strategic decisions.
In OLAP systems, NN can be used in parallel with OLAP, i.e. OLAP cubes are created on historical DW data, and NN forecasting work is based on the same historical data [20]. The disadvantage of this approach is that there are no training data sets.
The approach proposed in [21] includes two stages. First, principal components are analyzed to reduce the dimensionality of the data cube and special training sets are created. Then, in the second stage, a new OLAP-oriented multi-layer perceptron network (MLP) architecture is proposed whereby training is implemented on each training set and predicted dimensions are generated.
In [22], the possibilities proposed in [21] are expanded. First, the authors introduce a generalized framework, i.e., Multi-perspectives Cube Exploration Framework (MCEF), for applying the classical data mining algorithm to OLAP cubes. Secondly, the authors refer to modular NNs that apply a neural approach to predicting multidimensional cubes (NAP-NN). Modular NNs are a collection of several different networks that operate independently and contribute to the result. Each NN has its own set of input data. These networks do not interact with each other during task execution. The main advantage of modular NN is that the huge computational process can be divided into several subprocesses. This reduces computational complexity and increases computational speed. But ultimately, the processing time will depend on the number of neurons and their participation in calculating the results. Note that NAP-NN includes a preprocessing step. In this step, principal component analysis (PCA) is performed to reduce the size of the OLAP cube of the proposed method. Modular neural networks work effectively in cases where several directions of the system are simultaneously processed [23]. Note that the article presents experimental results showing the effectiveness of NN.
As noted above, the integration of OLAP and NN makes it possible to intelligently solve tasks on big data in a multidimensional structure of indicators.
Figure 8 illustrates the architectural and technological model for integrating OLAP and NN.
OLAP-cube1 Data analysis
NN Classification
Clustering
Forecasting
ETL DW (DM) ……….
OLAP-cuben requests, reports, Data Mining
methods
NN
Decision making process
According to the figure, data from various sources, before getting into DW or DM, first goes through ETL (extract, transform, load) technology [24]. The ETL process is an integral part of the data consolidation phase. This is a set of methods and algorithms that transfer source data from various sources to the data warehouse. In addition, during the ETL process, data is cleared of duplication, contradictions, typos and brought into a common format, and compliance with approved requirements is also checked. Only after receiving reliable information further actions and the decision-making process begin.
OLAP cubes are built based on DW (or DM) data. The figure shows an example of integrating NN with an OLAP cube to perform classification and clustering tasks on OLAP cube data, as well as predicting empty measures.
The integration of NN and OLAP is that the NN environment is built into OLAP applications that operate on a multidimensional structure and a large volume of data cubes.
Note that at this stage, the execution of processes characteristic of traditional OLAP is also ensured, namely: analytical queries are implemented on data for their rapid viewing and analysis, reports are issued based on the OLAP cube data, which can be with either intermediate or final results. It is also possible to view the same data from different angles.
If an organization has several areas of activity, then it would be more rational to use modular NS. In this case, each NS module works with a separate DM. This will ensure efficiency, accuracy and high speed of the network.
Depending on the task set, the most appropriate NN is selected from the above types. For example,
• single-layer and multi-layer perceptrons are used for classification;
• single-layer and multi-layer perceptrons are used for clustering
• single-layer or multilayer perceptrons and Kohonen map are used for forecasting [
Finally the results obtained will serve to make management decisions. According to the figure, the decision making process is performed using NN. In this regard, it should be noted that until now the central point of decision-making has been people. Persons making strategic decisions in the field of planning and management usually deal with poorly formalized tasks. Therefore, they have to make decisions based on personal experience and intuition. Sometimes it is risky. AI decision making, including NN, as shown in the figure, enables businesses or companies to make faster, more accurate, and more consistent decisions. Moreover, it is of great importance that NNs are capable of learning to build big data collection models that can make accurate decisions in real time.
Currently, in the era of Industry 4.0, there is a dramatic increase in the flow of data. This creates great complications when solving some problems related, for example, to classification, data clustering, forecasting, pattern recognition etc. In some cases, solving these problems becomes impossible. AI can provide quick and effective solutions to such problems. AI refers to any system capable of performing creative functions and solving problems that would typically require human intelligence. AI contributes to development and innovation in various industries, such as healthcare, finance, manufacturing, transport, e-commerce, education, etc. NNs are one of the areas of AI NNs, being implemented into systems, can solve important tasks. Such systems include OLAP systems based on OLAP technology. Recently, these OLAP systems have been widely used both in government agencies and in private ones.
The article implemented the following tasks: • various types of NN are considered and it is noted which problems are solved most effectively using which type of NN; • analyzed the work related to the task of integrating OLAP and NN; • the possibility of integrating OLAP and NN was explored, the advantages of such integration were shown; • the developed architectural and technological model for integrating NN and OLAP is presented. According to this model, integration is carried out by embedding the NN framework into OLAP applications running on a multidimensional structure and a large volume of data cubes. • within the framework of the model, as an example, the three most important problems are given, the solution of which on big data is most effective using NN, these include classification, clustering and forecasting; • it is noted that decision making using NNs speeds up this process and produces more accurate results, which is especially effective for big data; • if an organization includes several areas of activity and therefore, there are several DMs, then it is recommended to use modular NN. • Further studies will develop methods for embedding NN environment into OLAP applications to integrate NN and OLAP. [18] Kumar K., Krishna R, Kumar De S. Fuzzy OLAP Cube for Qualitative Analysis / Proceedings of the 3rd International Conference on Intelligent Sensing and Information Processing (ICISIP), pp. 290-295, 2005, http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=1529464&tag=1. doi: 10.1109/ICISIP.2005.1529464 [19] V. Oreshko, Data classification using neural networks, 2021. Accessed 15.03.2024 https://loginom.ru/blog/neural-classification [20] Marcia Gulesian. Using Neural Networks and OLAP Tools to Make Business Decisions, 2008.
Accessed 11.02.2024. https://www.developer.com/database/using-neural-networks-and-olaptools-to-make-business-decisions/ [21] W. Abdelbaki, R. Messaoud, S. Yahia, Neural-Based Approach for Extending OLAP to Prediction, in: Proceedings of science conference Data Warehousing and Knowledge Discovery (DaWaK 2012), Springer-Verlag Berlin Heidelberg, pp. 117–129, 2012.
doi: 10.1007/978-3-642-32584-7_10 [22] W. Abdelbaki, S. Yahia, R. Messaoud, Modular Neural Networks for Extending OLAP to Prediction. Book Transactions on Large-Scale Data- and Knowledge-Centered Systems XXI, Springer Nature, pp. 73-93, 2015. doi: 10.1007/978-3-662-48429-2_4 [23] A. Shukla, R. Tiwari, R. Kala. Modular neural networks. Towards Hybrid and Adaptive Computing. Chapter, pp. 307-335. https://link.springer.com/chapter/10.1007/978-3-642-143441_14 [24] What is ETL? https://WWW.ibm.com/topics/etl , last accessed 2023/11/05 [25] Neural networks, perceptron. Accessed: 10.12.2023. https://neerc.ifmo.ru/wiki/index.php?title=Нейронные_сети,_перцептрон#cite_note-3 [in Russian]