This survey deals with up-to-date results in the field of hybrid algorithms development of GMDH-type Neural Networks (GMDH-NN) and other methods of Artificial Intelligence (AI) which are successfully used for solving complex economic problems. Such hybrid algorithms are now only in its early stage of active research. General characteristics and main weaknesses of GMDH-NN are firstly presented. The paper further gives brief information on some AI paradigms such as Swarm Intelligence and Evolutionary Computation. Then known hybridization cases of GMDH-NN and certain methods of these paradigms (Genetic Algorithms, Differential Evolution, Particle Swarm Optimization, and Genetic Programming) are considered eliminating a number of the networks shortcomings. In the future it is worth to study the most promising ways of GMDH-NN hybridization with other methods of AI in order to increase their efficiency and extend applications.
Methods of AI have been used successfully in real-world applications, for example, predicting, data mining, classification, pattern recognition, knowledge discovery, system identification, clustering, control etc. However, there are many problems of such complexity that no method of AI can solve them alone.
The current trend is in developing composition of AI methods to obtain synergic effect, or hybridization in other words, since no one method is superior to the others in all situations. In doing so, methods respectively aim to enhance strengths of the components of the hybrid system and eliminate weaknesses of individual components. Due to the possibilities of hybrid systems solve many real problems that are characterized by inaccuracy, unpredictability, uncertainty, high dimension and variable environment, they attract much attention. Thus, they can be used to solve problems in a non-standard and promising way.
- 559
The Group Method of Data Handling (GMDH) is a powerful method based on the principles of self-organization. Basic Multilayered Iterative Algorithm MIA GMDH is an original Polynomial Neural Network of perceptron type. It should be noted that research in the field of hybridization GMDH-NN with other methods of AI are now in its early stage. But the first results of these studies indicate already a strong potential, high efficiency, diversity of applications of hybrid models. For example, GMDH not suitable for modeling of multi-output systems, while hybrid one with Multi Objective Genetic Algorithm is able to solve this problem.
This survey is aimed to analyzing and structuring typical known approaches to hybridization of GMDH-NN and AI methods as powerful tool for solving complex problems of economic modeling as well as determining most expedient ways of developing research in this field. 2
The GMDH was developed by Ivakhnenko [
GMDH is ideal for complex, unstructured systems where the investigator is only interested in obtaining a high-order input-output relationship. Hence this can be treated as a good data mining tool where data is transformed into knowledge for decision making. The most pronounced feature of GMDH is that it can choose the really significant input variables among dozens of these, thus actually reducing the dimension of a solved problem.
There are two main problems in GMDH-NN modeling such as training parameters
and selection of an optimal topology, which have a great impact on their performance
and encourage searching more efficient methods for their solution. The mathematical
basis for the vast majority of training algorithms for NNs is to utilize gradient
information to adjust the connection weights between nodes in the network. Its use implies
drawbacks such as slow convergence (training) rates, neglected multiple extremum
points, infinitesimally small step sizes (e.g. learning rates), costly computation, no
guarantee that algorithm converges to an optimum point, entrapped in local minimum
points, necessarily imply a least-mean-squared-error criterion, non-differentiability of
many error function [
Selection of the optimal topology often has the following problems: the search space of possible topologies is infinitely large, complex, multimodal, and not necessarily differentiable; there is little reason to expect that Neural Network can find a uniformly best algorithm for selecting the weights in a feedforward artificial neural network. At present, neural network design relies heavily on human experts who have sufficient knowledge about the different aspects of the network and the problem domain; as the complexity of the problem domain increases, manual design becomes more difficult and unmanageable.
Anastasakis and Mort [
Evolutionary Computation [
Usually grouped under the term EC, the domains of Genetic Algorithm (GA), Genetic Programming (GP), Differential Evolution (DE), Evolutionary Strategy (ES), Learning Classifier Systems (LCS), Estimation of Distribution Algorithms (EDA) and most recent Paddy Field Algorithm (PFA) are founded. They all share a common conceptual base of simulating the evolution of individual structures and they differ in the way the problem is represented, processes of selection and the usage/implementation of reproduction operators. The processes depend on the perceived performance of the individual structures as defined by the problem.
Swarm Intelligence [
Swarm Intelligence can be described by considering five fundamental principles. Proximity Principle: the population should be able to carry out simple space and time computations. Quality Principle: the population should be able to respond to quality factors in the environment. Diverse Response Principle: the population should not commit its activity along excessively narrow channels. Stability Principle: the population should not change its mode of behavior every time the environment changes. Adaptability Principle: the population should be able to change its behavior mode when it is worth the computational price. The most popular Swarm Intelligence algorithms are Particle Swarm Optimization (PSO), Ant Colony Optimization (ACO), Artificial Bee Colony Optimization (ABC), Cuckoo Search (CS), Fiery Algorithm (FA), Intelligent Water Drops (IWD), Gravitational Search Algorithm (GSA) and Charged System search (CSS). 4
Hybrid with DE. In [
Hybrid with GP. In [
Hybrid with GA. In [
Hybrid with PSO. Such algorithms have been proposed in [
Group of Adaptive Model Evolution (GAME). P. Kordik [17] created the GAME as a hybrid GMDH-based self-organizing modeling system which uses neurons (units) with several possible types of transfer functions (linear, polynomial, sigmoid, harmonic perceptron network, etc.). The GAME is an original data mining method. It can generate models for classification, prediction, identification or regression purposes. It works with both continuous and discrete variables. The topology of GAME models adapts to the nature of a data set supplied. The GAME is highly resistant to irrelevant and redundant features, suitable for short and noisy data samples. The GAME engine further develops the MIA GMDH algorithm. A GAME model has more degrees of freedom (units with more inputs, interlayer connections, transfer functions etc.) than MIA GMDH models. GAME engine also use genetic search to optimize the topology of models and also the configuration and shapes of transfer functions within their units.
Drawbacks of the up-to-date hybrids. The hybrid structures developed by researchers abroad Ukraine are typically based on classical MIA GMDH algorithm. Those researchers generally do not take into account the fact that during last decades in Ukraine there was developed and widely used another types of GMDH algorithms, namely combinatorial COMBI [18] and relaxational-iterative RIA [19] ones. Moreover, recently it was shown that hybrids of these three types of GMDH algorithms generate new, more effective GMDH structures named generalized iterative algorithm GIA [20]. Accordingly, these new developments can be used to create more powerful hybrid structures with various types of Computation Intelligence paradigms.
Example 1. The cement industry is one of the most important and profitable industries in every developed country and great content of financial resources are investing in this sector yearly. In [21] GMDH-type neural network and genetic algorithm is developed for stock price prediction of cement sector in Iran. Genetic algorithm is arranged in a new approach to design the whole architecture of the GMDH-type-NNs. It provides the optimal number of neurons in each hidden layer and their connectivity configuration to find the optimal set of appropriate coefficients of quadratic expressions to model stock prices. The results are very encouraging and congruent with the experimental results.
Example 2. The prediction of stock price is an important task in investment and financial decision-making since stock prices/indices are inherently noisy and nonstationary. In [22] a GMDH type-neural network based on Genetic algorithm is used to predict stock price index of petrochemical industry in Iran. The results obtained by using GMDH type neural network are in excellent agreement with the experimental results and has high performance in stock price prediction.
Example 3. Forecasting of short-term water demand is useful in planning and management of water and wastewater facilities such as pump scheduling, control of reservoirs volume, pressure management and water conservation program. This helps network managers to decrease vulnerability of the system and consumers and to increase network reliability. In [23] two types of neural networks: Multi Layered FeedForward with backpropagation learning algorithm and GMDH with genetic learning algorithm were investigated to model the water demand forecast in Tehran city. The comparison reveals that the GMDH neural network with GA produces results that are close to the actual data.
Example 4. Wheat is the chief carbohydrate source which is taken to be human’s food too. Considering the irregular population augmentation in the world the provision of food stuffs is a predicament of intense significance. Prediction of wheat yield using different quantity microelements is an important issue as a key parameter in increase of wheat production. In [24] wheat yield in Iran is modeled and predicted using GMDH-type neural networks based on some experimental data. The aim of such modeling is to show how wheat yield change with the variation of levels of fertilizers (Zinc & Iron). In this way, GA are deployed in a new approach to design the whole architecture of the GMDH-type neural networks, i.e., the number of neurons in each hidden layer and their connectivity configuration, in combination with using regression method to find optimal set of appropriate coefficients of quadratic expressions for modeling and prediction of wheat yield.
Example 5. Accurate forecasting of electricity load is one of the most important issues in the electricity industry, economic and reliable power systems operation such as unit commitment, reducing spinning reserve, maintenance scheduling, etc. It is essential part of an efficient power system planning and operation. Short-term load forecasting (STLF) is a difficult task because the accuracy of forecasting is influenced by many unpredicted factors such as economic, temperature, etc. whose relationships are commonly complex, implicit and nonlinear. The modified locally weighted group method of data handling (M-LWGMDH) based on genetic algorithm is proposed in [25] to solve the STLF problem. The GA is used to design the whole architecture of M-LWGMDH network. 6
Hybridization of GMDH-NNs and methods of Artificial Intelligence is a new field of investigation that is actively developed by researchers around the world. As a main result of this paper, it may be pointed out that the most promising ways of research activities in the field of modeling structures hybridization are the followings: creating new hybrid structures based on other types of GMDH algorithms and methods of Artificial Intelligence which have not yet been used for hybridization; exploring which kinds of hybrids are more successful when solving a specific class of applied problems.