=Paper=
{{Paper
|id=Vol-2830/paper29
|storemode=property
|title=Cognitive Modeling of University Activity as a Poorly Structured System
|pdfUrl=https://ceur-ws.org/Vol-2830/paper29.pdf
|volume=Vol-2830
|authors=Andrey Mikryukov,Mikhail Mazurov,Dmitry Korneev,Vasilii Trembach,Davyd Rylenkov
}}
==Cognitive Modeling of University Activity as a Poorly Structured System==
Cognitive Modeling of University Activity as a Poorly Structured System Andrey Mikryukov 1[0000-0002-8206-677X], Mikhail Mazurov 1[0000-0001-9993-4687], Dmitry Korneev 1[0000-0001-7260-4768], Vasilii Trembach 2[0000-0001-7499-4368], Davyd Rylenkov 1[0000-0002-0198-4246] 1 Plekhanov Russian University of Economics, 36 Stremyanny lane, Moscow, 115998, Russia Mikrukov.aa@rea.ru, Mazurov37@mail.ru, Korneev.dg@rea.ru, Dave.Rylenkov@gmail.com 2 Moscow Aviation Institute (National Research University), 4, Volokolamskoe shosse, Moscow, 125993, Russia Trembach@yandex.ru Abstract. The relevance of the problem being solved is due to the need to develop scientifically grounded proposals to achieve the required values of the basic indicators of the university's activities in accordance with the international institutional rating QS to the required values necessary for the university to enter the TOP-500 universities by 2025. To solve this problem, an approach is proposed to the study of weakly structured systems, the class of which includes universities and their activities, based on scenario forecasting methods by building a fuzzy cognitive model in order to determine the necessary increments of target values. The proposed approach allows, under the given constraints, to find the most acceptable scenario for planning the increment of basic indicators to target values by identifying the latent factors influencing them and impulse influences (increments) on them, ensuring guaranteed achievement of the set goal. The results obtained make it possible to subsequently justify the annual costs to ensure an unconditional increase in the values of latent factors (private indicators) to guarantee the required values of the basic indicators by 2025. The novelty of the proposed approach lies in the use of correlations between latent factors, identified on the basis of factor analysis methods, with basic indicators in the construction of a fuzzy cognitive model, as well as the application of an iterative approach to solving the problem, which makes it possible to update every year according to the results of the next ranking of universities in the international institutional ranking a set of initial data, as well as train the developed cognitive model, taking into account the results of identifying latent factors and adjusting their correlations. The results obtained make it possible to form the most preferable scenario plan for the necessary stepwise increase in the values of basic indicators in the interval 2020 -2025 subject to resource constraints. Keywords: Cognitive model, scenario forecasting, baseline indicators, institutional ranking. Proceedings of the 10th International Scientific and Practical Conference named after A. I. Kitov "Information Technologies and Mathematical Methods in Economics and Management (IT&MM-2020)", October 15-16, 2020, Moscow, Russia © 2021 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). CEUR Workshop Proceedings (CEUR-WS.org) 1 Introduction The purpose of developing a cognitive model of scenario forecasting is to substantiate the list of necessary measures to ensure the increments of the values of the target indicators of the university's activities in accordance with the international institutional rating QS to the required values necessary for the university to enter the TOP-500 universities by 2025. To achieve the stated goal of the study, the application of methods for solving weakly structured tasks was substantiated based on the development of a scenario forecasting (planning) model using fuzzy cognitive maps, which made it possible to determine probable (possible) trends in the development of events by alternative options and the possible consequences of decisions made in order to choose the most preferable one. alternatives. The proposed approach allows, under the given constraints, to find the most acceptable scenario for planning the increment of the values of basic indicators and functional (university rating) to target values due to impulse influences on latent factors (assessing the feasibility of the necessary increments in the values of factors - causes), ensuring guaranteed achievement of the goal. It should be borne in mind that the value of the functional and the place in the QS rating from year to year are not one-to-one, since the number of universities participating in the QS rating changes annually towards their increase. It follows from this that the value of the functional corresponding to a certain place in the rating in the current year may not correspond to the same place in the rating in subsequent years. That is, the value of the functional must be calculated with a certain margin. In the course of the study, the following tasks were solved: a fuzzy cognitive model of scenario forecasting of measures to achieve the required values of the target performance indicators of the university in the international institutional ranking QS was developed, on the basis of the developed model, the calculation of the most preferable variant of the set of required intensities of influence on control variables (factors - causes) was carried out on the basis of the developed model. a given increment of the value of the target factor (functional). The scenario of planning measures for guaranteed achievement of the target value obtained on the basis of the cognitive model made it possible to substantiate the value of the necessary increments in the values of the identified latent factors that affect the baseline indicators. Based on the results obtained, in the future, it is possible to justify the necessary annual costs to ensure the guaranteed achievement of the required value of the rating functionality of the university by 2025. The results obtained made it possible to formulate a scenario plan for the necessary stepwise increase in the values of indicators, considering the factors influencing them. A feature of the developed approach is the identification of the most significant latent factors affecting the baseline indicators, as well as the assessment of the degree of their influence on the baseline indicators based on the methods of factor analysis. To ensure the adequacy of the cognitive model, an iterative approach was used, which makes it possible to build a learning cognitive model. The use of this approach provides for the correction of the cognitive model based on the results of assessing the activities of universities after the release of the annual report on the place of the university in the QS institutional ranking. The adjustment is carried out by clarifying the structure of the cognitive map and causal dependencies based on updating the sets of initial data, as well as the results of identifying latent factors using a factor model. The use of an iterative approach allows for an increase in the reliability and accuracy of obtaining a solution. Section 2 provides a literature review on the research topic, Section 3 presents the results of the development of a cognitive model, Section 4 presents the results of scenario forecasting of university performance indicators based on the developed cognitive model of scenario forecasting, Section 5 provides an interpretation of the results of a numerical experiment. 2 Literature Review A fairly large number of works by domestic and foreign scientists are devoted to the problem of cognitive modeling [1-14]. An important task when using cognitive models is to ensure their reliability, adequacy, and accuracy. In papers [1-4], an approach based on classical cognitive models is proposed. In work [5], problem solving using fuzzy cognitive models is considered. The work [7] solves the problem of assessing the stability of cognitive models, as well as methods for their verification. In works [8,11-14], the features of modeling nonlinear dynamic systems under uncertainty are considered. The analysis of the sources showed the expediency of using the class of fuzzy cognitive models and that in the presented formulation, the problem of cognitive modeling of university development scenarios was not considered. The choice of a fuzzy cognitive model is based on the following reasons. Classical cognitive maps do not always provide for the construction of an adequate and reliable mathematical model due to the high level of uncertainty in the interaction of the components of the research object. They are linear and do not fully consider the dynamic properties of real systems, which can be essentially nonlinear and nonstationary. The strength of connections between concepts of a fuzzy cognitive map is set using a fuzzy linguistic scale, which is an ordered set of linguistic meanings (terms) of estimates of the strength of connection. The use of fuzzy cognitive maps provides a convenient interpretation of causal relationships between concepts. All this taken together ensures the construction of a more adequate model of the object of research. 3 Solving Problem Methodology The cognitive modeling methodology is an independent research area that is actively developing. Cognitive maps are a convenient mathematical apparatus for describing and studying poorly structured systems (socio-economic, socio-technical, organizational, etc. systems), the distinctive features of which are functioning under conditions of uncertainty, characterized by a lack of information, processes occurring in them, the impossibility of analytical descriptions and construction of formal models that take into account the specifics of the systems under consideration. The analysis of the problem posed showed that it belongs to the class of semi- structured problems, which can be solved under the conditions of a limited amount of initial data and a number of uncertainties associated, among other things, with the absence of a linear relationship between the values of the functional and the place of the university in the QS ranking in the considered time interval. To solve this class of problems, the methodology of fuzzy cognitive modeling is used, designed for analysis and decision-making in poorly defined situations when it is not possible to analytically describe and build formal mathematical models that take into account the specifics of the studied poorly structured system and, in particular, the method of scenario forecasting (planning ) based on cognitive maps (cognition maps), which allows one to determine the probable (possible) trends in the development of events by alternative options and the possible consequences of decisions made in order to choose the most preferable alternative [14]. A cognitive model in the form of a fuzzy cognitive map is a subjective model of a weakly structured dynamically developing situation. As you know, a scenario is a certain relative, conditional assessment of the possible development of the object or situation under study, since it is always built within the framework of assumptions about future development conditions, which are often fundamentally unpredictable. In contrast to classical forecasting methods, in which the main attention is paid to assessing the most probable variant of system development, scenario forecasting proceeds from the idea of uncertainty and ambiguity of the trajectory of this development. A feature of scenario forecasting is that it allows you to simultaneously consider several options for the development of the situation, analyze opportunities and risks [12]. In the context of the problem under consideration, a scenario is understood as a dynamic sequence of possible events of change in the values of factors - causes and factors - consequences that affect the target indicators. Causal relationships between these events and decision points can change their course and the trajectory of movement in time for the entire system of indicators under consideration, and, therefore, choose the most preferred trajectory option. To solve this problem, an approach to scenario forecasting based on a fuzzy cognitive map is proposed, which includes the following stages [14]: - scenario generation and risk assessment, - scenario adjustment at discrete time intervals (in our case, annually), considering the achieved values of the place in the QS rating of the functional and indicators. At the end of the next time interval, a new scenario is built to achieve the next set goal (adjusted considering current calculations). At the first stage of the study, the identification and interpretation of the latent factors affecting the baseline indicators was carried out, and their significance was assessed using the methods of factor analysis, which is a class of multivariate statistical analysis procedures aimed at identifying the latent variables (factors) responsible for the presence of linear statistical relationships. (correlations) between the observed variables [15-18]. Factors are groups of certain variables that correlate with each other more than with the variables included in another factor. Thus, the meaningful meaning of the factors was revealed by examining the correlation matrix of the initial data. In order to identify the factors influencing the basic indicators of the university's activity, to identify the most significant factors for the subsequent implementation of the approach to solving the problem on the basis of cognitive modeling methods, their SWOT analysis was carried out [19-21]. As you know, the SWOT analysis method involves a deep analysis of the research object, provides the most objective assessment of it from the point of view of the strengths (positive) and weak (negative) sides of the external and internal environment, as well as opportunities and threats. The results of the SWOT analysis made it possible to build the problem field of the situation, on the basis of which the goals and objectives of cognitive modeling were formulated, as well as the structure of the cognitive map was determined, with the help of which the task of predicting the performance indicators of the university was solved. When constructing the problem field of the situation for the structuring of knowledge, an object-structural approach was used [9], according to which the analysis and presentation of knowledge is carried out in strategic, organizational, conceptual, functional, spatial, temporal, causal and economic aspects (strata). The results of the analysis made it possible to structure the knowledge of experts using the problem field of knowledge, to build and identify on the basis of the method of expert assessments a set of factors and the degree of their influence on the performance indicators of the university. Registration of SWOT- analysis data was carried out in tabular form, where the main elements were recorded according to the categories presented (Table 1). Table 1. SWOT analysis results. The analyzed factors (characteristics) The degree of The degree of embodiment of the factor importance of the (characteristics) factor Strong factor + (characteristics) Weak factor – 1. Availability of well-known scientific Strong factor + 0,6 schools and dissertation councils 2. The presence of close collaboration with Strong factor + 0,3 foreign universities and research organizations (the number of joint research projects 3. Availability of basic departments at Strong factor + 0,2 enterprises … …. ….. Factor (characteristic) N …. …. The result of the SWOT analysis was the identification and grouping of a set of factors that affect the performance of the university. Since the number of factors influencing the activities of the university is a significant value, it became necessary to highlight the most significant factors, taking into account the correlation relationships, including the factors of the second level that affect the factors of the first level. In order to identify the most significant factors, one of the most common methods of factor analysis was used - the method of principal components, which allows one to reduce (reduce) a large number of related correlated variables, since a large number of variables significantly complicates the analysis and interpretation of information [17]. Calculations based on the method of principal components were carried out in the environment of the analytical platform Deductor 5.3 using the "Factor analysis" module. The use of the apparatus of factor analysis made it possible to solve two main problems: 1) reducing the dimension of the number of variables used due to their explanation by a smaller number of factors; 2) grouping and structuring of the received data. The novelty of the proposed approach is determined by: - solving the problem of structuring knowledge using the object-structural approach when constructing the problem field of the situation, which provides a more effective grouping of latent factors and building an adequate architecture of the cognitive map; identifying the most significant latent factors affecting the baseline indicators, as well as assessing the degree of this influence based on the methods of factor analysis; - the implementation of a cognitive model based on a fuzzy cognitive map, which provides adaptation to the uncertainty of the initial data and the conditions of the problem being solved, which, in turn, made it possible to form a more adequate cognitive model and increase the reliability and accuracy of modeling results; - using an iterative approach to building a cognitive model that allows you to build a trained cognitive model; - the possibility of adjusting the fuzzy cognitive model based on the results of the annual assessment of universities' performance in the QS institutional ranking. The adjustment is carried out by clarifying the structure of the cognitive map and causal dependencies based on updating the sets of initial data, as well as the results of identifying latent factors using the factor model. 3 University Performance Indicators Scenario Forecasting Results Based on the Cognitive Model In contrast to classical cognitive maps, fuzzy cognitive maps (FCM) are represented as a fuzzy directed graph with feedback, the nodes of which are fuzzy sets. FCM combines the properties of fuzzy systems and neural networks [14]. As a result of the selection of the most significant factors affecting the baseline indicators, a cognitive map was built that reflects their interconnections, as well as the weight of these interconnections (Fig. 1). The weights of the relationships are determined based on the use of correlation analysis and expert methods. Fig. 1. Cognitive map of the scorecard based on the graph of the relationship of factors, baseline indicators and functionality (Legend: F - functional; R - rating, (F1-F24) - latent factors) The following designations are adopted in Figure 1: F is functional; R is university rating, AR is academic reputation; EP is reputation with the employer; RS/T is the ratio of the number of students to the number of teachers; CT - indicator of citation of teachers; IT is number of international teachers; IS is the number of international students; F1 is factor "Scientific schools and dissertation councils" (0.6); F2 is factor "Joint research projects" (0.3); F3 is Presence of basic departments (0.2); F4 is Number of publications in the Scopus database (0.6); F5 is In-demand areas of study (0.3); F6 is skill level of the teaching staff (0.2); F7 is Number of teachers (0.6); F8 is Students' Competence Level (0.5); F9 is teachers with language training (0.4); F10 is Dorm Places (0.2); F11 is Demand for graduates from employers (0.3); F12 is Areas for educational activities (0.3); F13 is The level of payment for teaching staff (0.4); F14 is Stimulating factors (0.2); F15 is Expansion of the CPD social package (0.3); F16 is Change in the structure of employment of the CPD (0.3); F17 is Proportion of teaching staff planning to pursue an international scientific career (0.2); F18 is Academic mobility of the teaching staff (0.3); F19 is Convergence of educational programs with foreign universities (0.4); F20, is Foreign Entry Company (0.3); F21 is Increase in the number of On-line MOOCs (0.3), F22 is Implementation of individual educational trajectories (0.4); F23 is Implementation of remote sensing technologies (0.3); F24 is The tightness of the relationship with the employer (0.4). The directed graph (digraph) of the FCM is defined using a set of sets [14, 22-28] FCM = <С, F, W>,……… ……..…..………… (1) where C = {Ci} is the set of graph vertices, called concepts, which are factors that are most significant from the point of view of studying the system under consideration; F = {Fk} - set of directed graph arcs - connections between concepts; W = {Wij} is the set of weights of arcs (links). The connections between concepts can be positive, “enhancing” the influence of the Ci concept on the Cj concept (Wij> 0), or negative, “weakening” the influence of the Ci concept on the Cj concept (Wij <0). Each concept is characterized by a term-set of a linguistic variable Ti = {T1i, T2i ,…, Tmji} (2) where mj is the number of typical states of the i-th concept. To describe each term Tki, a term is constructed - a set with a membership function . The connections between the typical states of each pair of concepts are set by fuzzy variables described by the corresponding fuzzy sets. The values of the weights (bond strength) Wij are set using a fuzzy linguistic scale, which is an ordered set of linguistic values (terms) of bond strength estimates, for example, of the form: LINK_POWER = {Does not affect; Weak; Average; Strong; Very strong}. Each of these values is associated with a certain numerical range belonging to the segment [0, 1] for positive links (Table 2), or the segment [–1, 0] for negative links. Table 2. Assessment of the strength of the connection between concepts. Linguistic value Term designation Numerical range Point estimate of bond strength + 0 Z 0 Does not affect (0; 0,15) VL 0,12 Very weak (0,15; 0,35) L 0,23 Weak (0,35; 0.60) M 0,47 Average (0,6; 0,85) H 0,72 Strong (0,85-1,0) VH 0,93 Very strong The strength of the connection between the concept Ci and the concept Cj is chosen by the expert by one of the linguistic values presented in the table, as well as by some "point" estimate of the strength of the connection - a number within this range (if there are several experts, then the weight Wij is averaged). In the general case, a weighted digraph with arbitrary values of the weights Wij –1, 1 is described by the dynamics of its state change in time. The state of the digraph (FCM) is determined by the set of states of its concepts Ci , (i = 1, 2, …, n), each of which is described by the state variable Xi (t), which takes values in the interval [0, 1]. State variables Xi (values of latent factors) are calculated by normalizing real ("physical") variables by the formula Xi = , (3) where Хi min and Хi max are the minimum and maximum values of the variable Хi, (i = 1, 2,…, n). The state of the FCM at an arbitrary moment of time is described by the equation of state of the following form [29]: The weighted digraph in Fig. 1 with arbitrarily set values of the weights Wij is described by the equations of state in the following form: where is some nonlinear “contracting” function that maps the values of the argument to the unit interval [0, 1]. This condition is satisfied, for example, by the sigmoid function (x) = (5) To calculate the state variables Xi(t), (i = 1, 2, …, n) using equations (4), it is necessary to set the initial conditions, i.e. vector X(0) = (X1(0), X2(0),…, Xn(0))T. An important stage in the FCM analysis is the analysis of its impulse stability [25], when for a given nonzero initial state Xi (0), for example, X1(0) = 1, X2(0) = …= Xn(0) = 0, the sequence of impulse values pi (t) = Xi(t)- Xi(t-1) is bounded at any time instant t = 1,2, ..., m for any of its vertices, and absolute stability, when for each vertex of the digraph (i = 1, 2 ,…, n) the sequence of values значений Xi(t), t = 1, 2, …m is bounded. In this case, the statement [30,31] is true: the weighted digraph, which is described by equations (4), (5), is absolutely stable, and there is a unique equilibrium (steady- state) solution of the indicated equations ("fixed point") X* in that and only in that case if where n is the number of FCM concepts. Verification of the developed FCM using formula (6) confirmed its stability. The results of assessing the influence of latent factors on the baseline and rating indicators led to the following conclusions. The least influence is exerted by the factors: F3, The presence of basic departments at enterprises; F14, Stimulating factors. In turn, the following factors have the greatest influence: F1, The presence of well-known scientific schools and dissertation councils; F2, Close collaboration with foreign universities and research organizations (number of joint research projects; F4, Number of publications in the Scopus database, WoS. Based on setting the values of the initial data using formula (3), setting the weights of the mutual influences of factors using expert estimates and the results of correlation-regression analysis, describing the behavior of the digraph using equations of state (4), (5) and indicating the values of the initial increments of factors, analysis of the dynamics of changes in factors and their influence on the development of the system of indicators as a whole. Verification of the developed cognitive model to check its adequacy was carried out by testing in the retrospective period 2014-2019. based on available statistics on measurable factors in the model. The general correctness of the model at this stage was confirmed by the closeness of the growth rates of factors calculated on the model to the actual growth rates. The control problem is considered solved if, under the conditions of the given constraints, an alternative option (planning scenario) is found to increment the values of the functional and indicators to the target values due to impulse influences (increments of factor values), which ensures the guaranteed achievement of the set goal [32-37]. The “weak” increments of the values of latent factors at the level of 10%, consistently set in the course of simulation modeling of controlled development of the system of indicators, made it possible to evaluate the sensitivity of the target indicator (rating functional) to control actions in these areas of regulation (Table 3). Table 3. Results of assessing the sensitivity of the target to changes in latent factors. Experiment Controlled Initial value Impulse of Change of rating number factor change functional 1 F1 0,59 0,01 0,09 2 F2 0,19 0,02 0,07 … … … … … 24 F24 0,32 0,01 0,03 Considering the preliminary results of the computational experiment in the course of scenario modeling, the required intensity of influence on the control factors in percentage terms was calculated for a given increment of the target factor. The problem was solved under the conditions of the given restrictions on the resources allocated to ensure the increment of latent factors. 4 Discussion. The Results of a Numerical Experiment Interpretation Considering the achievable values of latent factors in the interval 2020-2021. the scenario of their increment is selected, shown in Fig. 3. The greatest intensity of impact will be required for latent factors: Number of publications in the Scopus database, WoS, Joint research projects, Demand for areas of training, stimulating factors, the least for latent factors: Changes in the structure of teaching staff employment, Areas for educational activities, The share of teaching staff planning to build an international scientific career. The results of calculating the predicted values of growth indicators of basic indicators and functional in 2020, considering the chosen scenario, are presented in Fig 3. The obtained results of scenario forecasting show that the largest increase should be obtained by the PP indicator (reputation with the employer) - 54%, and the smallest - MP ( the ratio of the number of students to the number of teachers) - 16%. An increase in the values of basic indicators should lead to an increase in the value of the functional by 23%, i.e. achieving its planned value. Thus, the goal of the study has been achieved. Fig. 2. Calculated values of the control actions intensities on latent factors in the form of increments in%, necessary to achieve the target increase in baseline indicators and functionality in 2020 The results of calculating the predicted values of growth indicators of basic indicators and functional in 2020, considering the chosen scenario, are presented in Fig 3. The obtained results of scenario forecasting show that the largest increase should be obtained by the PP indicator (reputation with the employer) - 54%, and the smallest - MP ( the ratio of the number of students to the number of teachers) - 16%. An increase in the values of basic indicators should lead to an increase in the value of the functional by 23%, i.e. achieving its planned value. Thus, the goal of the study has been achieved. Scenario analysis of the development of the situation made it possible to choose the most preferable scenario, which, under the conditions of the given restrictions, ensures the achievement of the required planned value of the target indicator with minimal resource consumption for the increment of latent factors. The required intensity of influence on the control factors as a percentage for a given increment of the target factor were calculated during scenario modeling. The simulation results presented in Fig. 2, considering the attainable values of latent factors in the interval 2019-2020. correspond to the scenario of their increment, ensuring the achievement of the required target indicator (functional) with minimal resource costs for the increment of latent factors. Fig. 3. Forecasted values of basic indicators and functional growth in 2021. compared to 2020 (2020 -100%) As a result of the calculations performed, the following conclusions can be drawn: To guarantee the achievement of the functional value F = 26.89 by 2025, corresponding to the 435 (451) place in the QS rating, it is necessary to stepwise increase the values of indicators, taking into account the factors influencing them, identified on the basis of factor analysis methods, in the interval 2020 -2025. according to the following scheme. Based on the calculation performed using the cognitive modeling methodology, it is necessary to perform impulse influences (to provide an increment) on the set of factors identified as of the present in accordance with the results presented in Fig. 2. At the end of the control interval (after 1 year), after clarifying the actual values of the basic indicators and functional in the QS rating, assess the change in the influence and interaction of factors on the basic indicators based on factor analysis methods, identify additional latent factors, update the developed scenario forecasting model based on fuzzy cognitive maps taking into account additional latent factors, allowing to build a more adequate and reliable model of the cognitive map. Based on the updated model, perform calculations to justify a set of new impulse influences on the latent factors of the cognitive model for the period 2021-2022. 5 Conclusion During the research, an iterative learning cognitive model of scenario forecasting of activities to achieve the required values of the target performance indicators of the university in the QS international institutional ranking was developed. On the basis of the developed model, the calculation of the most preferable from the point of view of cost minimization of the scenario variant of the set of required intensities of influence on the control variables (latent factors) for a given planned increment of the value of the target factor was carried out. The novelty of the proposed approach is determined using factor analysis methods to determine the relationships between latent factors, which provide identification of the most significant latent factors for building an adequate cognitive model. The proposed approach allows training the developed model by adjusting the structure of the cognitive map and its parameters based on the results of updating the initial data for its construction. 5. Acknowledgments Work is performed with assistance of the Russian Federal Property Fund, the projects No. 18-07-00918, 19-07-01137, 20-07-00926. References 1. Maksimov V. I, Kachaev S. V. Technologies of the Information Society in Action: Application of Cognitive Methods in Business Management, http://www.rfbr.ru/default.asp?doc_id=5222. 2. Axelrod R. Decision structure: cognitive maps of political elites. - Princeton. University Press, 1976. 3. Kuznetsov O. P. Cognitive modeling of semi-structured situations // http://posp.raai.org/data/posp2005/Kuznetsov/kuznetsov.html. 4. Roberts FS Discrete mathematical models with applications to social, biological and environmental problems. - M .: Nauka, 1986 .-- 184 p. 13. Kosko B. Fuzzy cognitive maps. // International Journal of Human and Machine Research, (1986) 24, 65-75. 5. Carvalho J.P., Tom J.A.B. Rule-Based Fuzzy Cognitive Maps - Fuzzy Causal Relationships. // Computational Intelligence for Modeling, Control and Automation: Evolutionary Computing and Fuzzy Logic for Intelligent Control, Knowledge and Information Retrieval, edited by M. Mohammadyan, IOS Press, 1999. 6. Kulba VV, Mironov PB, Nazaretov VM Modern approaches to modeling complex socio- economic systems of socio-economic systems using sign digraphs // Automation and telemechanics - 1993. - №7. 7. Nikolay Karabutov, “Structural identifiability of nonlinear dynamical systems under uncertainty”, International Journal of Intelligent Systems and Applications (IJISA), Volume 12, No. 1, pp. 12-22, 2020. DOI: 10.5815 / ijisa.2020.01 .02. 8. Evgeny Bodyansky, Alexey K. Tishchenko, Anastasia O. Deineko, “An evolving neuro- fuzzy system with online learning / self-learning”, IJMECS, volume 7, No. 2, pp. 1-7, 2014. DOI: 10.5815 / ijmecs .2015.02.01. 9. V. Vijayalakshmi, K. Venkatachalapati, Comparison of Predicting Student Performance Using Machine Learning Algorithms, International Journal of Intelligent Systems and Applications (IJISA), Volume 11, No. 12, pp. 34-45, 2019. DOI: 10.5815 / ijisa .2019.12.04. 10. Gorelova G.V., Zakharova E.N., Rodchenko S.A. Study of semi-structured problems of socio-economic systems: a cognitive approach. - Rostov n / a: Publishing house of the Russian State University, 2006. – 332p. 11. Avdeeva ZK, Kovriga SV, Makarenko DI, Maksimov VI, Cognitive approach in management, Probl. Upr., 2007, issue 3, p. 2–8. 12. Bolotova L.S. Artificial intelligence systems: models and technologies based on knowledge: textbook / FSBEI RSUITP; FGAU GNII ITT "Informatics". - M: Finance and statistics, 2012-- 664 p. 13. Borisov V.V., Kruglov V.V., Fedulov A.S. Fuzzy moels and networks. - 2nd ed. stereotypes. - Hot line - Telecom, 2015. 284 p. 14. Goridko, N. P. Modern economic growth: theory and regression analysis: Monograph / N.P. Goridko, R.M. Nizhegorodtsev. - M: Infra - M, 2017-- 444p. 15. Draper, N. Applied regression analysis / N. Draper. - M: Williams I.D., 2019.- 912p. 16. Sokolov G.A. Introduction to regression analysis and planning of regression experiments in economics: Textbook / G.А. Sokolov, R.V. Sagiots. - M.: Infra-M, 2016, -352p. 17. Mikryukov A.A., Gasparian M.S., Karpov D.S. Development of proposals for promoting the university in the international institutional ranking QS based on statistical analysis methods. Statistics and Economics. 2020; 17 (1): 35-43. 18. Belyaev VI Marketing: fundamentals of theory and practice: textbook. / V.I.Belyaev. - 2nd ed., Stereotype. - Moscow: KNORUS, 2007.- 672 p. 19. Bogomolova EV SWOT analysis: theory and practice of application / EV Bogomolova // Economic analysis: theory and practice. - 2004. - No. 17 (32). - pp. 57-60. 20. Bogomolova EV Conceptual approach to the application of the SWOT-analysis methodology in trade organizations to assess the effectiveness of the organization's functioning / Е.А. V. Bogomolova // Audit and financial analysis. 21. Trakhtengerts E.A. Computer support for the formation of goals and strategies. - M.: SINTEG, 2005-224 p. 22. Kornoushenko E.K., Maksimov V.I. Situation control using the structural properties of the cognitive map // Tr. IPU RAS. - 2000. - T. XI. - p. 85-90. 23. Information and analytical system QS - analytics. https://analytics.qs.com/#/signin. 24. O.P. Kuznetsov, “Intellectualization of control decision support and creation of intelligent systems”, Probl. Upr., 2009, No. 3.1, p.4–72. 25. Saati T. Decision making. Method of analysis of hierarchies: per. from English M: Radio and communication, 1993.320 p. 26. V. V. Kulba Scenario analysis of the dynamics of behavior of socio-economic systems / V.V. Kulba, D.A. Kononov, S.S. Kovaleskiy et al. - Moscow: IPU RAN, 2002. - 122 p. 27. Maksimov VI Structural-target analysis of the development of socio-economic situations // Management problems. 2005. No. 3. 28. Gorelova G.V., Melnik E.V., Korovin Y.S. Cognitive analysis, synthesis, forecasting the development of large systems of intelligent RIUS "Artificial Intelligence" 3'2010. - p. 61- 72. 29. Roberts F.S. Discrete mathematical models with applications to social, biological and environmental problems / F.S. Roberts - M.: Nauka, 1986. 30. V. V. Kulba Methods for the formation of scenarios for the development of socio- economic systems / V. V. Kulba, D. A. Kononov, S. A. Kosyachenko, A. N. Shubin. Moscow: SINTEG, 2004.296 p. 31. Simon H. The Structure of Ill-structured Problems // Artificial Intelligence. - 1973. - Vol. 4. - P. 181-202. 32. Cognitive analysis and management of the development of situations: materials of the 1st international conference. Moscow: IPU, 2001.196 p. 33. Checkland P.B. Systems Thinking, Systems Practice. - New York: Wiley, 1981. 34. Axelrod R. The Structure of Decision: Cognitive Maps of Political Elites. - Princeton: University Press, 1976. 35. Swain J.J. Simulation Software Boldly Goes ..., // OR / MS Today. - 2007. - Vol. 36, No. 5. - P. 50-61. 36. Kosko B. Fuzzy cognitive maps // Man-Machine Studies. - 1986. Vol. 24. - P. 65-75 Eden C. Cognitive mapping // European Journal of Operational Research. - 1988. - N 36. - P. 1-13. 37. Kizim N.A., Khaustova V.E. Features of testing models based on cognitive maps for stability and reliability // Modern approaches to modeling complex socio - economic systems. - Kh .: FLP Aleksandrova K. M .; Publishing House INZHEK, 2011. - 280 p.