<?xml version="1.0" encoding="UTF-8"?>
<TEI xml:space="preserve" xmlns="http://www.tei-c.org/ns/1.0" 
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
xsi:schemaLocation="http://www.tei-c.org/ns/1.0 https://raw.githubusercontent.com/kermitt2/grobid/master/grobid-home/schemas/xsd/Grobid.xsd"
 xmlns:xlink="http://www.w3.org/1999/xlink">
	<teiHeader xml:lang="en">
		<fileDesc>
			<titleStmt>
				<title level="a" type="main">About methods of support for management decision-making under conditions of significant uncertainty</title>
			</titleStmt>
			<publicationStmt>
				<publisher/>
				<availability status="unknown"><licence/></availability>
			</publicationStmt>
			<sourceDesc>
				<biblStruct>
					<analytic>
						<author>
							<persName><forename type="first">V</forename><forename type="middle">I</forename><surname>Baluta</surname></persName>
							<email>vbaluta@yandex.ru</email>
							<affiliation key="aff0">
								<orgName type="institution">Keldysh Institute of Applied Mathematics</orgName>
								<address>
									<settlement>Moscow</settlement>
									<country key="RU">Russia</country>
								</address>
							</affiliation>
							<affiliation key="aff1">
								<orgName type="institution">Plekhanov Russian University of Economics</orgName>
								<address>
									<settlement>Moscow</settlement>
									<country key="RU">Russia</country>
								</address>
							</affiliation>
						</author>
						<author>
							<persName><forename type="first">V</forename><forename type="middle">P</forename><surname>Osipov</surname></persName>
							<email>osipov@keldysh.ru</email>
							<affiliation key="aff0">
								<orgName type="institution">Keldysh Institute of Applied Mathematics</orgName>
								<address>
									<settlement>Moscow</settlement>
									<country key="RU">Russia</country>
								</address>
							</affiliation>
						</author>
						<author>
							<persName><forename type="first">O</forename><forename type="middle">Y</forename><surname>Yakovenko</surname></persName>
							<affiliation key="aff2">
								<orgName type="department">Institute of applied modeling and forecasting</orgName>
								<address>
									<settlement>Moscow</settlement>
									<country key="RU">Russia</country>
								</address>
							</affiliation>
						</author>
						<title level="a" type="main">About methods of support for management decision-making under conditions of significant uncertainty</title>
					</analytic>
					<monogr>
						<imprint>
							<date/>
						</imprint>
					</monogr>
					<idno type="MD5">1D3E5250E9B36393EB287F35FEB77053</idno>
				</biblStruct>
			</sourceDesc>
		</fileDesc>
		<encodingDesc>
			<appInfo>
				<application version="0.7.2" ident="GROBID" when="2023-03-24T01:46+0000">
					<desc>GROBID - A machine learning software for extracting information from scholarly documents</desc>
					<ref target="https://github.com/kermitt2/grobid"/>
				</application>
			</appInfo>
		</encodingDesc>
		<profileDesc>
			<textClass>
				<keywords>
					<term>public administration</term>
					<term>decision support</term>
					<term>computer modeling</term>
					<term>expert judgement</term>
					<term>artificial intelligence</term>
					<term>group intelligence</term>
					<term>collective intelligence</term>
					<term>hybrid intelligence</term>
				</keywords>
			</textClass>
			<abstract>
<div xmlns="http://www.tei-c.org/ns/1.0"><p>Attempts to improve the quality of managerial decisions by introducing modern advances in information technology in various areas of public administration (socio-humanitarian, strategic, foreign policy, etc.)  do not give the desired effect, comparable to the effect of their implementation in the manufacturing sector. The solution to this problem requires qualitatively new approaches to the issues of information and analytical support for decision-making in conditions of significant uncertainty. This article highlights the difficulties of predictive management of social processes based on direct computer modeling of social systems, considers the disadvantages of expert decision support methods used in practice, and proposes new technologies for applying expert knowledge and competencies based on the use of computer modeling and research. Current approaches to working with experts can be described as methods of coordinating the opinions of a group of experts based on their personal views (models) on the issue under discussion. Our experience has shown that creating a common integrated model by a group of experts gives a much better result. While the traditional approach can be called "group" intelligence, the new approach is called "collective" intelligence. In addition, methods of decision support using artificial intelligence systems are currently being intensively developed. We propose to begin work on the creation of "hybrid" intelligence with the integration of these approaches to obtain a synergistic effect.</p></div>
			</abstract>
		</profileDesc>
	</teiHeader>
	<text xml:lang="en">
		<body>
<div xmlns="http://www.tei-c.org/ns/1.0"><head n="1.">Introduction</head><p>Many researchers note the fundamental problem of insufficient compliance of traditionally slow-evolving public administration systems with modern challenges of world development.</p><p>Large-scale institutional transformations of the world community in the course of scientific and technological progress have formed and continue to modify the new reality of human and society existence. An increase in the number of connections and relations into which each element of the social world order is somehow interwoven due to the processes of globalization and information and technological development increases the complexity of developing and making managerial decisions by orders of magnitude. The empirical knowledge previously accumulated in the field of social sciences and humanities in the new conditions is of little use, since it does not allow us to explain with sufficient degree of certainty the nature and tendencies of the changes that are taking place, derive their regularities and predict the further course of development of general society and its institutions, to predict the events and reactions of people, and therefore cannot serve as a support in the development of managerial decisions at the state level. Modern systems of government are forced to function in conditions of a significant level of uncertainty, which is due to the high dynamics and poor predictability of possible trajectories of changes in the current situation, affecting the dynamics of current priorities in the military-political, economic, social and humanitarian spheres.</p><p>Attempts to compensate for an insufficient understanding of the phenomena occurring in these areas with data volumes, as a rule, lead more to overloading of management systems than to improving the quality of decision-making.</p><p>In fact, modern civilization challenges can be interpreted as a consequence of the growing gap, on the one hand, between the demands of social development and, on the other hand, the capabilities of management systems to develop and implement adequate solutions (fig. <ref type="figure">1</ref>).</p></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head>Fig. 1. Generalized view of public process management problems</head></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head n="2.">The State of Research</head><p>As a subject of scientific research, the field of public administration appears as a system of tasks of developing managerial decisions and tasks of reproduction and development of management mechanisms in relation to certain areas and directions, for example, such as politics, economics, defense, production, international relations, society etc. Obviously, each of these areas has its own specificity, its own range of problematic issues and research methods, its own forms and methods of analyzing the situation, developing and implementing solutions. The key tools for analyzing, developing and supporting the implementation of solutions to date are various means of monitoring and visualizing data about the situation, including partially processed by relatively simple analytical methods.</p><p>In addition to fairly simple analytical processing tools for monitoring data and their visual presentation forms, recently in concepts and development plans for decision support systems, much attention has been paid to the development of mathematical modeling methods and various approaches to creating simulation models of complex controlled systems. Based on such models, it is supposed to conduct research and forecasting the behavior of the corresponding prototypes in various situations. As the analysis of literary sources shows, the level of development of modeling forms in various areas of public administration differs in their specificity and prevalence: if the economy has been using predominantly econometric and statistical models for a long time, and the military is persistently making attempts to introduce simulation, then in social the humanitarian sphere still focuses on scenario modeling, and, as a rule, at the verbal level.</p><p>Based on a relatively large number of publications of the corresponding orientation in foreign countries, we can conclude that at present, there are multiple attempts to comprehend the possibilities and limits of the application of traditional and new technologies of model research in order to increase management efficiency in conditions of significant uncertainty.</p><p>We can distinguish several among the most intensively developing areas modeling in the world. One of the main methods for analyzing the political situation remains models based on the application of the principles of equilibrium <ref type="bibr" target="#b0">[1]</ref><ref type="bibr" target="#b1">[2]</ref><ref type="bibr" target="#b2">[3]</ref>. Methods based on the agent approach are getting a great deal of development, including their variety -multi-agent modeling (as a rule, requiring significant computational resources, as well as significant costs for collecting and preparing data for modeling for practical purposes) <ref type="bibr" target="#b3">[4]</ref><ref type="bibr" target="#b4">[5]</ref><ref type="bibr" target="#b5">[6]</ref>. The method of system dynamics remains quite popular, which over the past decades has acquired many varieties, such as the use of technology of cognitive maps, fuzzy modeling, etc. <ref type="bibr" target="#b6">[7]</ref><ref type="bibr" target="#b7">[8]</ref><ref type="bibr" target="#b8">[9]</ref><ref type="bibr" target="#b9">[10]</ref><ref type="bibr" target="#b10">[11]</ref><ref type="bibr" target="#b11">[12]</ref><ref type="bibr" target="#b12">[13]</ref><ref type="bibr" target="#b13">[14]</ref><ref type="bibr" target="#b14">[15]</ref><ref type="bibr" target="#b15">[16]</ref><ref type="bibr" target="#b16">[17]</ref><ref type="bibr" target="#b17">[18]</ref><ref type="bibr" target="#b18">[19]</ref>. Unfortunately, we have not come across work with the idea of simultaneous research of one object using a scientifically based of different models, which does not allow us to evaluate their comparative effectiveness. Although some generalizing works can be mentioned that could form the basis of such an approach <ref type="bibr" target="#b19">[20]</ref><ref type="bibr" target="#b20">[21]</ref><ref type="bibr" target="#b21">[22]</ref><ref type="bibr" target="#b22">[23]</ref><ref type="bibr" target="#b23">[24]</ref><ref type="bibr" target="#b24">[25]</ref>.</p><p>Regarding the development of modeling methods in the field of applied social and humanitarian research in domestic science, several points of concentration of advanced scientific thought can be noted. So, at the Institute of Management Problems of the Russian Academy of Sciences, quite a lot of attention is paid to the development of general theoretical approaches to modeling various aspects of managing complex systems under the leadership of D. Novikov, methods for solving practical problems in the face of uncertainty -by V. Kulba's group, substantiation of the theoretical foundations of decision-making -by the group of Aleskerov F. et al. The approaches to a comprehensive review of socio-political, economic, demographic processes in their mutual influence are being developed by a team of researchers at the Plekhanov Russian University of Economics led by A. Kugaenko.</p></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head n="3.">Basic approaches</head><p>According to management theory, there are three basic methodologies for improving the efficiency of developing managerial decisions in relation to social and economic processes. These include full-scale experiments, expert assessments and mathematical modeling <ref type="bibr" target="#b25">[26]</ref>.</p><p>In the practice of public administration, targeted field experiments are usually used to test the effectiveness in local pilot areas of any regulatory innovations or proposed solutions in order to study the likely consequences of their implementation. Although it is believed that field experiments are most reliable, the cases of their application (unless, of course, the traditional "trial and error method" is referred to them) are extremely limited, in particular, they are unacceptable in crisis or urgent situations, and most cases and simply impossible.</p><p>An expert study of a problem situation is characterized by the fact that general information about the situation is limited to the expert's personal knowledge. Although expert knowledge has the important property of focusing on individual groups of alternatives, it is characterized by an extremely high level of subjectivity and limitedness simply because of the characteristics of human thinking. To increase confidence in expert conclusions, groups of experts are usually involved, but the factor of subjectivity is not eliminated in that case.</p><p>Model studies, as a rule, are associated with the formalization of the description of the control object and the current situation, the selection of criteria for the adequacy of modeling. A direct study of the situation on the model with an assessment of the possible consequences ends with an interpretation of the simulation results to redistribute the preference of alternatives.</p><p>As the capabilities of computer technology and the development of model research methods have grown, the field of applicability of the results of model studies of control objects and the environment has significantly expanded to solve the problems of managing complex systems such as the state, economy, armed forces, etc.</p><p>Thus, in practice, there is a competition between two fundamental approaches -an orientation toward the use of information systems and ever-increasing complexity of models or at attracting the experience and knowledge of experts. It is clear that this separation is somewhat contingent, since domain experts also participate in the creation of models, and any expert in solving the problem relies, inter alia, on the results of a model representation of the subject of study.</p></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head n="4.">Problems of expert approaches</head><p>One of the main difficulties in applying expert approaches is the impossibility of a full-scale vision of the analyzed picture in a holistic form by any of the experts with the degree of detail that is inherent in each individual specialist in the field of his competencies. This does not allow, within the framework of traditional approaches, a qualitative analysis of multi-parameter problems with the complex nature of the interconnections within the systems under consideration, which include most of the tasks of public administration. Moreover, it is obvious that the impossibility of a holistic vision of the problem, situation, goals and objectives inevitably leads to a decrease in the quality of management decisions.</p><p>Even when solving problems in a group, each of the experts uses their own ideas about the situation, on the basis of which they form their own conclusions about its development. To reduce the importance of the factor of specialization and subjectivity of experts, to date, many standard methods for rational construction of the processes of formation of expert conclusions and conclusions by expert groups have been proposed. All of them are, to one degree or another, based on methods of reconciling the resulting opinions by selecting the most preferred of them for some reason (acceptability, validity, authority, etc.). This approach allows us to reach consensus and increase the level of confidence in the results, however, despite the group nature of the discussion, in fact even the adjusted final conclusion is based on the limited vision of one expert with whom the others agree, which virtually eliminates the possibility of original solutions appearing.</p><p>In other words, the technology used in practice (fig. <ref type="figure">2</ref>), which we will arbitrarily call the technology of "group" intelligence, involves procedures for agreeing on particular conclusions that do not allow going beyond the scope of voiced proposals and gaining new knowledge.</p></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head>Fig. 2. Conditional presentation of an "agreed" group decision of experts</head></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head n="5.">Problems of modeling social systems</head><p>From a research point of view, society, society is a variety of large complex systems with many elements that are in a certain relationship and connections but with a large number of degrees of freedom. For this reason, various kinds of political, public, social, societal and another processes are characterized by a significant degree of subjective uncertainty due to the high complexity of the behavior of individual elements of the system, the complexity of the mechanisms for linking the behavior of the system and its elements and the objective limitation of the subject in perception and understanding of reality.</p><p>To build a detailed model of such a system, it is necessary to describe the behavior of almost each of the elements in all its relationships with others. Moreover, according to modern ideas about rational approaches to modeling, to obtain a qualitative model of the object of study, it is necessary to build a model of interaction of its components to the third structural level of the hierarchy of elements, which significantly increases the scale and complexity of the model.</p><p>Due to the increasing capabilities of computing tools, many of the approaches currently being developed are in line with such a mechanistic approach -in the direction of the greatest possible detail in the model reproduction of the system under consideration. As you know, the more complex the model, the more resources are needed for its creation and validation. Moreover, the more complex and accurate the model, the more difficult and costly it is to provide it with satisfactory input data, that is, in addition to the development itself, additional resources are also needed for collecting, validating, formalizing and entering large volumes of heterogeneous input data. When solving engineering, technical, and physical problems, where the behavior of modeling objects is objectively limited by the scope of well-studied physical laws, or in the production sphere, which is regulated by strict restrictions on resource balances and the rules of interaction of elements that reduce the degree of arbitrariness, such approaches are justified and give satisfactory results. But in humanitarian spheres, objective laws and restrictions that reduce the degrees of freedom of elements have not yet been established (if they exist for such systems at all with respect to the time and place scales in question), so the creation of complex models for describing social processes encounters insurmountable difficulties.</p></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head n="6.">The lack alternate to expert methods</head><p>If we go to the limit views, we can conclude that in the field of public administration for solving practical problems it is impossible to build large-scale detailed models or provide them with initial data with details satisfying the necessary and sufficient requirements to reduce the level of uncertainty in order to develop effective managerial of decisions.</p><p>However, one cannot deny the importance of modern methods of computer modeling for studying complex systems, testing hypotheses, identifying obscure relationships and dependencies, etc. In other words, if direct modeling of specific processes in society is objectively impossible, then the use of various forms of modeling the systems under study in this area can be effective, first of all, to build up expert experience and knowledge.</p><p>By the way, human evolution itself can serve as a confirmation of the higher efficiency of expert approaches in solving managerial problems in comparison with numerical modeling. There are facts about the unique abilities of individuals, for example, with absolute memory, able to perform ultrafast and voluminous calculations, etc. That is, the human brain is initially able to do what computers are doing more and more well. And physiologists believe that the potential of the human brain for certain types of information processing is almost unlimited. Nevertheless, evolution did not follow the path of improving the mechanisms of deep analysis and miscalculation of details, but the path of developing mechanisms for a "broad" assessment of situations for an operational response in difficult conditions with high uncertainty.</p><p>Strange as it may seem, despite the intensive development of multi-agent technologies, the prospects for the development of decision-making support systems based on computer technologies are also seen in the creation of "artificial intelligence", that is, imitating the intellectual functions of a person in terms of "wide" and operational analysis of reality and the development of rational management decisions in conditions of high uncertainty. From the above it follows that at present, the use of expert approaches in management systems is seen as the most rational direction for the development of decision support systems. The only question is how to most effectively use the progress in the development of modern computer technologies to increase the effectiveness of the application of these approaches and their further improvement.</p><p>It seems that now the society is in a situation where a transition to new, more effective forms of information support for management is needed, based on a combination of the use of expert knowledge with the possibilities of modeling situations, as well as on ways to "objectify" the subjective knowledge of experts . In our opinion, a transition from "group" intelligence to "collective" is needed, when in the process of analyzing the situation and making decisions, there is not a simple summation or averaging of the finished conclusions of individual experts, but a synergistic summation of the competencies of experts conducting a joint search solutions.</p></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head n="7.">The technology of "collective" intelligence</head><p>When performing several search and research works aimed at creating tools for predictive management of complex systems under significant uncertainty, a new technology was tested -the technology of "collective" intelligence, combining expert analysis and modeling of complex systems (fig. <ref type="figure">3</ref>).</p></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head>Fig. 3. Conditional representation of the "collective" intelligence of experts</head><p>The main contours of this technology were outlined in the process of considering problematic issues of modeling and studying complex systems, searching for rational (primarily in terms of mathematical optimality) solutions and predictive evaluation of their performance. The accompanying result of these works was a complex of various research and service software tools, some of which have found application in various areas of solving managerial problems, allowing to ensure their implementation at a high scientific and technical level. During the development, the foundations and the procedure for applying the technology were laid, its operability was verified, and its higher efficiency was confirmed in comparison with other expert methods for solving a certain range of tasks.</p><p>The general algorithm of work using the technology of "collective" intelligence involves the following set of interdependent procedures: analysis and classification of a problematic issue; determination of the required composition of professional and target competencies of the expert group; formation of a team of experts; teaching them the rules of technology; organization of coordinated interaction, including verification of psychological compatibility, distribution of roles, coordination of understanding of goals and perception of the subject of research; assignment of indicators and criteria of decision quality; the construction of a generalized cognitive model, as well as the selection of ready-made or the creation of a set of express models of various subsystems and subprocesses; study of the model for consistency and consistency; conducting iterative studies on model examples; adjustment and refinement of the model; conducting experiments; comparison, analysis and interpretation of results; preparation of an opinion. It is clear that, depending on the situation, some deviations from the regulations described here are possible in the implementation process.</p><p>As a result, a look at the constituent elements of the model and the forms of their relationship with respect to the competencies of each expert remains subjective, however, the complex model obtained as a result of collective work has the properties of an independent object alienated from personalities, on the basis of which it is possible to study the behavior of a controlled system for subsequent collective analysis of the resulting results.</p><p>To some extent, here we can draw an analogy with the method of "abstracting from the problem" within the framework of the so-called "Harvard School principal negotiations", the substantive essence of which is not a direct search for a compromise, but preliminary agreement on the principles of its search.</p></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head n="8.">Technology of "hybrid" intelligence</head><p>The technology of the "collective" intellect is considered as an intermediate stage in the development of decision support systems. It is expected that as the technology of "artificial" intelligence develops, the "collective" intelligence described here can evolve into some form of "hybrid" intelligence, forming a symbiosis with such properties that will ensure that the control system meets the new challenges of civilizational development. The possibility and pace of evolutionary development of management information support methods in the direction of "hybrid" forms will be substantially determined by the progress in the development of "artificial" intelligence technologies.</p><p>As noted above, it is advisable to use the modern capabilities of modeling complex systems to build up expert knowledge <ref type="bibr" target="#b26">[27,</ref><ref type="bibr" target="#b27">28]</ref>. One of the forms of using "artificial" intelligence can be the use of an electronic training ground with a model of the complex system under study deployed on it and constantly being improved. As an example, we can cite the results presented in <ref type="bibr" target="#b27">[28,</ref><ref type="bibr" target="#b28">29]</ref>, which relate to the methodology of training an intelligent agent at an "electronic training ground" with a layout of the environment that allows, during many experiments, to determine the mechanisms of formation of rational trajectories of movement to a target in the phase space of an agent's state from an arbitrary provisions.</p><p>After training at the training ground and introducing the function of assessing one's current position by an arbitrary number of features, one can use the rules developed at the "training ground" to make decisions in the current situation, including the situation understood with a certain degree of uncertainty. Considering that an "electronic polygon" is still some abstract model representation of the environment, it is advisable to combine both the construction of the model on polygon and the recommendations resulting from its application with expert assessment by specialists. Thus, with the help of artificial intelligence technologies, it is possible to expand the knowledge base on the basis of which alternatives are evaluated and recommendations are made on the most rational actions in the face of uncertainty. One of the possible versions of the model of the "hybrid" intellect based on the "collective" and "artificial" is presented in fig. <ref type="figure" target="#fig_0">4</ref>. </p></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head n="9.">Conclusion</head><p>It can be summarized that in order to increase the effectiveness of information and analytical support for government in the face of significant uncertainty, it is necessary to develop methodological foundations, principles of construction and architecture of a hybrid automated decision support system by synthesizing the collective intelligence of experts with artificial intelligence based on methods computer modeling of complex systems and solving optimization problems. The development of forms of expert management support should be focused on obtaining qualitatively new results through the integration of expert competencies, as opposed to the search for compromises that are oriented towards traditional approaches for coordinating expert opinions.</p><p>The originality of the proposed approach lies in the appropriate combination of social engineering methods with modern advances in the field of information technology. Implementation of the technology of "hybrid" intelligence will allow to obtain the results of an expert assessment of the situation and alternatives by synergistically integrating expert knowledge with the capabilities of "artificial" intelligence.</p></div><figure xmlns="http://www.tei-c.org/ns/1.0" xml:id="fig_0"><head>Fig 4 .</head><label>4</label><figDesc>Fig 4. A possible scheme of the symbiosis of technologies based on hybrid intelligence</figDesc><graphic coords="5,173.93,447.22,247.15,132.90" type="bitmap" /></figure>
		</body>
		<back>
			<div type="references">

				<listBibl>

<biblStruct xml:id="b0">
	<analytic>
		<title level="a" type="main">Biased Beliefs and Retrospective Voting: Why Democracies Choose Mediocre Policies</title>
		<author>
			<persName><forename type="first">I</forename><surname>Bischoff</surname></persName>
		</author>
		<author>
			<persName><forename type="first">L.-H</forename><surname>Siemers</surname></persName>
		</author>
	</analytic>
	<monogr>
		<title level="j">Public Choice</title>
		<imprint>
			<biblScope unit="volume">156</biblScope>
			<biblScope unit="page" from="163" to="180" />
			<date type="published" when="2011">2011</date>
		</imprint>
	</monogr>
</biblStruct>

<biblStruct xml:id="b1">
	<analytic>
		<title level="a" type="main">Research, Policy and Practice: Why Developing Countries Are Different</title>
		<author>
			<persName><forename type="first">J</forename><surname>Young</surname></persName>
		</author>
	</analytic>
	<monogr>
		<title level="j">Journal of International Development</title>
		<imprint>
			<biblScope unit="volume">17</biblScope>
			<biblScope unit="page" from="727" to="734" />
			<date type="published" when="2005">2005</date>
		</imprint>
	</monogr>
</biblStruct>

<biblStruct xml:id="b2">
	<monogr>
		<author>
			<persName><forename type="first">G</forename><forename type="middle">C</forename><surname>Rausser</surname></persName>
		</author>
		<author>
			<persName><forename type="first">J</forename><surname>Swinnen</surname></persName>
		</author>
		<author>
			<persName><forename type="first">P</forename><surname>Zusman</surname></persName>
		</author>
		<title level="m">Political Power and Economic Policy: Theory, Analysis, and Empirical Applications</title>
				<meeting><address><addrLine>Cambridge</addrLine></address></meeting>
		<imprint>
			<publisher>Cambridge University Press</publisher>
			<date type="published" when="2011">2011</date>
		</imprint>
	</monogr>
</biblStruct>

<biblStruct xml:id="b3">
	<monogr>
		<author>
			<persName><forename type="first">T</forename><forename type="middle">T</forename><surname>Allen</surname></persName>
		</author>
		<title level="m">Introduction to discrete event simulation and agent-based modeling: voting systems, health care, military, and manufacturing</title>
				<meeting><address><addrLine>London, UK</addrLine></address></meeting>
		<imprint>
			<publisher>Springer</publisher>
			<date type="published" when="2011">2011</date>
		</imprint>
	</monogr>
</biblStruct>

<biblStruct xml:id="b4">
	<analytic>
		<title level="a" type="main">Strategic voting and coordination problems in proportional systems: An experimental study</title>
		<author>
			<persName><forename type="first">A</forename><surname>Blais</surname></persName>
		</author>
		<author>
			<persName><forename type="first">C</forename><surname>Erisen</surname></persName>
		</author>
		<author>
			<persName><forename type="first">L</forename><surname>Rheault</surname></persName>
		</author>
	</analytic>
	<monogr>
		<title level="j">Political Research Quarterly</title>
		<imprint>
			<biblScope unit="volume">67</biblScope>
			<biblScope unit="issue">2</biblScope>
			<biblScope unit="page" from="386" to="397" />
			<date type="published" when="2014">2014</date>
		</imprint>
	</monogr>
</biblStruct>

<biblStruct xml:id="b5">
	<analytic>
		<title level="a" type="main">Cascade or echo chamber? A complex agentbased simulation of voter turnout</title>
		<author>
			<persName><forename type="first">E</forename><surname>Fieldhouse</surname></persName>
		</author>
		<author>
			<persName><forename type="first">L</forename><surname>Lessard-Phillips</surname></persName>
		</author>
		<author>
			<persName><forename type="first">B</forename><surname>Edmonds</surname></persName>
		</author>
	</analytic>
	<monogr>
		<title level="j">Party Politics</title>
		<imprint>
			<biblScope unit="volume">22</biblScope>
			<biblScope unit="issue">2</biblScope>
			<biblScope unit="page" from="241" to="256" />
			<date type="published" when="2016">2016</date>
		</imprint>
	</monogr>
</biblStruct>

<biblStruct xml:id="b6">
	<analytic>
		<title level="a" type="main">System Dynamics Modeling in International Relations</title>
		<author>
			<persName><forename type="first">A</forename><surname>Fisunoğlu</surname></persName>
		</author>
		<idno type="DOI">10.20991/allazimuth.476884</idno>
	</analytic>
	<monogr>
		<title level="j">All Azimuth: A Journal of Foreign Policy and Peace</title>
		<imprint>
			<biblScope unit="volume">8</biblScope>
			<biblScope unit="issue">2</biblScope>
			<biblScope unit="page" from="231" to="253" />
			<date type="published" when="2019">2019</date>
		</imprint>
	</monogr>
</biblStruct>

<biblStruct xml:id="b7">
	<analytic>
		<title level="a" type="main">Dynamics of Cultural Change: The Human Development Perspective</title>
		<author>
			<persName><forename type="first">Mark</forename><forename type="middle">A</forename><surname>Abdollahian</surname></persName>
		</author>
	</analytic>
	<monogr>
		<title level="j">International Studies Quarterly</title>
		<imprint>
			<biblScope unit="volume">56</biblScope>
			<biblScope unit="issue">4</biblScope>
			<biblScope unit="page" from="827" to="842" />
			<date type="published" when="2012">2012</date>
		</imprint>
	</monogr>
</biblStruct>

<biblStruct xml:id="b8">
	<analytic>
		<title level="a" type="main">A Bibliographical Study on the Academic Research of Peace and Conflict in Turkey</title>
		<author>
			<persName><forename type="first">Oner</forename><surname>Akgul</surname></persName>
		</author>
	</analytic>
	<monogr>
		<title level="m">Paper presented at the 7th Eurasian Peace Science Meeting</title>
				<meeting><address><addrLine>İstanbul</addrLine></address></meeting>
		<imprint>
			<date type="published" when="2018">2018</date>
		</imprint>
	</monogr>
</biblStruct>

<biblStruct xml:id="b9">
	<analytic>
		<title level="a" type="main">Systems Dynamics Model of Al-Qa&apos;ida and United States &apos;Competition</title>
		<author>
			<persName><forename type="first">Todd</forename><surname>Chamberlain</surname></persName>
		</author>
	</analytic>
	<monogr>
		<title level="j">Journal of Homeland Security and Emergency Management</title>
		<imprint>
			<biblScope unit="volume">4</biblScope>
			<biblScope unit="issue">3</biblScope>
			<biblScope unit="page" from="1" to="23" />
			<date type="published" when="2007">2007</date>
		</imprint>
	</monogr>
</biblStruct>

<biblStruct xml:id="b10">
	<monogr>
		<author>
			<persName><forename type="first">Yu-Che</forename><surname>Chen</surname></persName>
		</author>
		<author>
			<persName><forename type="first">Michael</forename><forename type="middle">J</forename><surname>Ahn</surname></persName>
		</author>
		<title level="m">Routledge Handbook on Information Technology in Government</title>
				<meeting><address><addrLine>New York</addrLine></address></meeting>
		<imprint>
			<publisher>Routledge</publisher>
			<date type="published" when="2017">2017</date>
		</imprint>
	</monogr>
</biblStruct>

<biblStruct xml:id="b11">
	<analytic>
		<title level="a" type="main">Turkish Terrorism Studies: A Preliminary Assessment</title>
		<author>
			<persName><forename type="first">Cenker</forename><surname>Demir</surname></persName>
		</author>
		<author>
			<persName><forename type="first">Engin</forename><surname>Korhan</surname></persName>
		</author>
		<author>
			<persName><surname>Avci</surname></persName>
		</author>
	</analytic>
	<monogr>
		<title level="j">All Azimuth</title>
		<imprint>
			<biblScope unit="volume">7</biblScope>
			<biblScope unit="issue">1</biblScope>
			<biblScope unit="page" from="21" to="44" />
			<date type="published" when="2018">2018</date>
		</imprint>
	</monogr>
</biblStruct>

<biblStruct xml:id="b12">
	<monogr>
		<title level="m" type="main">Industrial Dynamics</title>
		<author>
			<persName><forename type="first">Jay</forename><forename type="middle">W</forename><surname>Forrester</surname></persName>
		</author>
		<imprint>
			<date type="published" when="1961">1961</date>
			<publisher>The M.I.T. Press</publisher>
			<pubPlace>Cambridge</pubPlace>
		</imprint>
	</monogr>
</biblStruct>

<biblStruct xml:id="b13">
	<analytic>
		<title level="a" type="main">How Small System Dynamics Models Can Help the Public Policy Process</title>
		<author>
			<persName><forename type="first">Navid</forename><surname>Ghaffarzadegan</surname></persName>
		</author>
		<author>
			<persName><forename type="first">John</forename><surname>Lyneis</surname></persName>
		</author>
		<author>
			<persName><forename type="first">George</forename><forename type="middle">P</forename><surname>Richardson</surname></persName>
		</author>
	</analytic>
	<monogr>
		<title level="j">System Dynamics Review</title>
		<imprint>
			<biblScope unit="volume">27</biblScope>
			<biblScope unit="issue">1</biblScope>
			<biblScope unit="page" from="22" to="44" />
			<date type="published" when="2011">2011</date>
		</imprint>
	</monogr>
</biblStruct>

<biblStruct xml:id="b14">
	<analytic>
		<title level="a" type="main">A Tale of Two Cultures: Qualitative and Quantitative Research in the Social Sciences</title>
		<author>
			<persName><forename type="first">Gary</forename><surname>Goertz</surname></persName>
		</author>
		<author>
			<persName><forename type="first">James</forename><surname>Mahoney</surname></persName>
		</author>
	</analytic>
	<monogr>
		<title level="m">System Dynamics</title>
				<meeting><address><addrLine>Princeton</addrLine></address></meeting>
		<imprint>
			<publisher>Princeton University Press</publisher>
			<date type="published" when="2012">2012</date>
			<biblScope unit="page">253</biblScope>
		</imprint>
	</monogr>
</biblStruct>

<biblStruct xml:id="b15">
	<analytic>
		<title level="a" type="main">Navigating Towards Sustainable Development: A System Dynamics Approach</title>
		<author>
			<persName><forename type="first">Peder</forename><surname>Hjorth</surname></persName>
		</author>
		<author>
			<persName><forename type="first">Ali</forename><surname>Bagheri</surname></persName>
		</author>
	</analytic>
	<monogr>
		<title level="j">Futures</title>
		<imprint>
			<biblScope unit="volume">38</biblScope>
			<biblScope unit="issue">1</biblScope>
			<biblScope unit="page" from="74" to="92" />
			<date type="published" when="2006">2006</date>
		</imprint>
	</monogr>
</biblStruct>

<biblStruct xml:id="b16">
	<monogr>
		<author>
			<persName><forename type="first">Robert</forename><surname>Jervis</surname></persName>
		</author>
		<title level="m">System Effects: Complexity in Political and Social Life</title>
				<meeting><address><addrLine>Princeton</addrLine></address></meeting>
		<imprint>
			<publisher>Princeton University Press</publisher>
			<date type="published" when="1997">1997</date>
		</imprint>
	</monogr>
</biblStruct>

<biblStruct xml:id="b17">
	<monogr>
		<author>
			<persName><forename type="first">Craig</forename><forename type="middle">W</forename><surname>Kirkwood</surname></persName>
		</author>
		<title level="m">System Dynamics Methods: A Quick Introduction</title>
				<imprint>
			<date type="published" when="1998">1998</date>
		</imprint>
		<respStmt>
			<orgName>College of Business, Arizona State University</orgName>
		</respStmt>
	</monogr>
</biblStruct>

<biblStruct xml:id="b18">
	<analytic>
		<title level="a" type="main">When War Brings Peace: A Dynamic Model of the Rivalry Process</title>
		<author>
			<persName><forename type="first">David</forename><forename type="middle">S</forename><surname>Morey</surname></persName>
		</author>
	</analytic>
	<monogr>
		<title level="j">American Journal of Political Science</title>
		<imprint>
			<biblScope unit="volume">55</biblScope>
			<biblScope unit="issue">2</biblScope>
			<biblScope unit="page" from="263" to="275" />
			<date type="published" when="2011">2011</date>
		</imprint>
	</monogr>
</biblStruct>

<biblStruct xml:id="b19">
	<analytic>
		<title level="a" type="main">Computational modeling for what-is, what-might-be, and whatshould-be studies-and triangulation</title>
		<author>
			<persName><forename type="first">R</forename><forename type="middle">M</forename><surname>Burton</surname></persName>
		</author>
		<author>
			<persName><forename type="first">B</forename><surname>Obel</surname></persName>
		</author>
	</analytic>
	<monogr>
		<title level="j">Organization Science</title>
		<imprint>
			<biblScope unit="volume">22</biblScope>
			<biblScope unit="issue">5</biblScope>
			<biblScope unit="page" from="1195" to="1202" />
			<date type="published" when="2011">2011</date>
		</imprint>
	</monogr>
</biblStruct>

<biblStruct xml:id="b20">
	<analytic>
		<title level="a" type="main">Communication and culture: A complexity theory approach</title>
		<author>
			<persName><forename type="first">C.-Y</forename><surname>Chiu</surname></persName>
		</author>
		<author>
			<persName><forename type="first">L</forename><surname>Qiu</surname></persName>
		</author>
	</analytic>
	<monogr>
		<title level="j">Asian Journal of Social Psychology</title>
		<imprint>
			<biblScope unit="volume">17</biblScope>
			<biblScope unit="page" from="108" to="111" />
			<date type="published" when="2014">2014</date>
		</imprint>
	</monogr>
</biblStruct>

<biblStruct xml:id="b21">
	<monogr>
		<title level="m" type="main">Introduction to computational social science: Principles and applications</title>
		<author>
			<persName><forename type="first">C</forename><surname>Cioffi-Revilla</surname></persName>
		</author>
		<imprint>
			<date type="published" when="2017">2017</date>
			<publisher>Springer</publisher>
			<pubPlace>New York, NY</pubPlace>
		</imprint>
	</monogr>
	<note>2nd ed.</note>
</biblStruct>

<biblStruct xml:id="b22">
	<analytic>
		<title level="a" type="main">Big data, causal inference, and formal theory: Contradictory trends in political science?</title>
		<author>
			<persName><forename type="first">W</forename><forename type="middle">R</forename><surname>Clark</surname></persName>
		</author>
		<author>
			<persName><forename type="first">M</forename><surname>Golder</surname></persName>
		</author>
	</analytic>
	<monogr>
		<title level="j">PS: Political Science &amp; Politics</title>
		<imprint>
			<biblScope unit="volume">48</biblScope>
			<biblScope unit="issue">1</biblScope>
			<biblScope unit="page" from="65" to="70" />
			<date type="published" when="2015">2015</date>
		</imprint>
	</monogr>
</biblStruct>

<biblStruct xml:id="b23">
	<monogr>
		<author>
			<persName><forename type="first">T</forename><surname>Saaty</surname></persName>
		</author>
		<title level="m">Theory and Aplications of the Analytic Network Process: Decision Making with Benefits, Opportunities, Cost and Risk</title>
				<meeting><address><addrLine>Pittsburgh</addrLine></address></meeting>
		<imprint>
			<publisher>RWS Publications</publisher>
			<date type="published" when="2009">2009</date>
		</imprint>
	</monogr>
</biblStruct>

<biblStruct xml:id="b24">
	<monogr>
		<author>
			<persName><forename type="first">K</forename><forename type="middle">L</forename><surname>Casey</surname></persName>
		</author>
		<title level="m">Defining Political Capital: A Reconsideration of Bourdieu&apos;s Interconvertibility Theory // Paper University of Missouri</title>
				<meeting><address><addrLine>St. Louis</addrLine></address></meeting>
		<imprint>
			<date type="published" when="2007">2007</date>
		</imprint>
	</monogr>
</biblStruct>

<biblStruct xml:id="b25">
	<monogr>
		<title level="m" type="main">Decision Making Theory</title>
		<author>
			<persName><forename type="first">A</forename><forename type="middle">B</forename><surname>Petrovsky</surname></persName>
		</author>
		<imprint>
			<date type="published" when="2009">2009</date>
			<publisher>Publishing Center Academiya</publisher>
			<pubPlace>Moscow</pubPlace>
		</imprint>
	</monogr>
	<note>in Russian</note>
</biblStruct>

<biblStruct xml:id="b26">
	<analytic>
		<title level="a" type="main">The environment of modeling, forecasting and expertise as the intellectual core of support management complex systems</title>
		<author>
			<persName><forename type="first">V</forename><forename type="middle">I</forename><surname>Baluta</surname></persName>
		</author>
		<author>
			<persName><forename type="first">V</forename><forename type="middle">P</forename><surname>Osipov</surname></persName>
		</author>
		<author>
			<persName><forename type="first">O</forename><surname>Yakovenko</surname></persName>
		</author>
		<author>
			<persName><surname>Yu</surname></persName>
		</author>
	</analytic>
	<monogr>
		<title level="j">KIAM Preprint</title>
		<imprint>
			<biblScope unit="volume">82</biblScope>
			<date type="published" when="2015">2015</date>
		</imprint>
	</monogr>
</biblStruct>

<biblStruct xml:id="b27">
	<analytic>
		<title level="a" type="main">Comparative assessment of the NPP risk (on the example of Rostov and Kalinin NPP). Development of risk indicators atlas for Russian NPPs», GraphiCon 2019 Computer Graphics and Vision</title>
		<author>
			<persName><forename type="first">M</forename><forename type="middle">A</forename><surname>Berberova</surname></persName>
		</author>
		<author>
			<persName><forename type="first">K</forename><forename type="middle">I</forename><surname>Chernyavskii</surname></persName>
		</author>
		<idno type="DOI">10.30987/graphicon-2019-2-290-294</idno>
		<ptr target="http://ceur-ws.org/Vol-2485/paper67.pdf" />
	</analytic>
	<monogr>
		<title level="m">The 29 th International Conference on Computer Graphics and Vision. Conference Proceedings</title>
				<meeting><address><addrLine>Bryansk, Russia</addrLine></address></meeting>
		<imprint>
			<date type="published" when="2019-09-23">2019. September 23-26, 2019</date>
			<biblScope unit="volume">2485</biblScope>
			<biblScope unit="page" from="290" to="294" />
		</imprint>
	</monogr>
</biblStruct>

<biblStruct xml:id="b28">
	<analytic>
		<title level="a" type="main">Simulation of dynamics and adaptation in the space of conflict interaction of the intelligent agents</title>
		<author>
			<persName><forename type="first">V</forename><forename type="middle">I</forename><surname>Baluta</surname></persName>
		</author>
		<author>
			<persName><forename type="first">V</forename><forename type="middle">P</forename><surname>Osipov</surname></persName>
		</author>
		<author>
			<persName><forename type="first">B</forename><forename type="middle">N</forename><surname>Chetverushkin</surname></persName>
		</author>
		<author>
			<persName><forename type="first">O</forename><forename type="middle">Y</forename><surname>Yakovenko</surname></persName>
		</author>
		<ptr target="Javdet@gmail.com" />
	</analytic>
	<monogr>
		<title level="m">Institute of applied modeling and forecasting</title>
				<editor>
			<persName><forename type="first">@</forename><surname>Osipov</surname></persName>
		</editor>
		<editor>
			<persName><forename type="first">Oleg</forename><surname>Yakovenko</surname></persName>
		</editor>
		<editor>
			<persName><surname>Yu</surname></persName>
		</editor>
		<meeting><address><addrLine>Moscow</addrLine></address></meeting>
		<imprint>
			<date type="published" when="2019">2019</date>
		</imprint>
		<respStmt>
			<orgName>Candidate of Technical Sciences, senior researcher ; Keldysh Institute of applied mathematics of the Russian Academy of Sciences, Plekhanov Russian University of Economics ; Technical Sciences</orgName>
		</respStmt>
	</monogr>
	<note>KIAM Preprint № 111</note>
</biblStruct>

				</listBibl>
			</div>
		</back>
	</text>
</TEI>
