In 2021 the Assessment of the Future Security Environment of Ukraine (the forecasting period - till 2030) was elaborated and published for the first time. However, the methodology of strategic forecasting and planning in the national security and defense sphere still needs to be further developed and improved. That is why we propose to use the Method of Goal Dynamic Evaluation of Alternatives and its modifications to elaborate and process the Future Security Environment model, built by experts and analysts. According to our approach, prediction process should be structured into series of steps, and thoroughly consider the international experience of strategic geopolitical forecasting.
Forecasting of the future security environment (FSE) of Ukraine is one of the top-priority tasks of
analytical activity of governmental bodies responsible for conducting the Defense Review. It is based
on the elaboration of probable scenarios of military escalation within midterm and long-term
perspective [1]. The solution of this problem is the key to building of effective national resilience
system, providing the ability to quickly adapt to security environment changes and maintain sustainable
functioning through minimization of external and internal threats [
In 2021, for the first time, Ukrainian experts elaborated and published the Assessment of the Future
Security Environment of Ukraine for the period till 2030. According to their understanding, the security
environment should be considered as a combination of factors that may influence national interests and
pose a threat, which must be neutralized by national defense and security forces. Security environment
is comprised of states with their national interests (economic, political, military etc), natural
environment, specific physical and legal entities, political parties, national information space etc [[
At the same time, according to Ukrainian doctrines, one of the main tasks of national defense force
capabilities development is the introduction of a network-centric approach to implementation of modern
systems for management, obtaining and exchange of information, as well as building of a unified
information environment through automation of collection, processing and dissemination of
information [[
While considering the problem of security environment forecasting, we have to take into account foreign experience in this field. The United States of America, as one of the leading superpowers, has a very diversified network of government entities and nongovernment “think tanks” which develop different forecasts. In this context, analytic units of the US Intelligence Community are among the most powerful segments of the US defense and security sectors (which are responsible for strategic forecasting).
On many occasions, the US experts were trying to reconsider the problems of prediction. In 2008,
such a leading American “think tank” as the RAND Corporation (in its report “Assessing the Tradecraft
of Intelligence Analysis”) illustrated intelligence “failures” related to some unsuccessful attempts of
US Intelligence Community experts to predict historical events, that significantly influenced further
development of security environment [
Taking into account more recent developments, we may supplement this list with the incorrect forecasts concerning Russia’s armed aggression against Ukraine (2014, 2022), emergence of “Islamic State in Syria and Levant” (2014), the results of the referendum on the withdrawal of the United Kingdom from the European Union (2016) and COVID-2019 pandemic (2020).
That is why the US Intelligence Community is routinely raising the issue of improving the
procedures for forecasters. In this context it would be very interesting and fruitful to look into the
research conducted in 2011-2013 by Philip Tetlock, the Professor at the University of Pennsylvania,
and other scientists and analysts. The study was designed as a large-scale forecasting tournament under
request of the Intelligence Advanced Research Projects Activity (IARPA), the research and
development branch of the Office of the Director of National Intelligence (US). Some results of the
research were published in an article “The Psychology of Intelligence Analysis: Drivers of Prediction
Accuracy in World Politics” [[
The aim of the research was to empirically determine the factors, which could contribute to improvement of accuracy and credibility of the forecasts, developed by analysts. More than 750 American analysts from government and non-government institutions (particularly, those developing geopolitical forecasts and, thus, closely related to security environment forecasting tasks) took part in the experiment. For two years they were giving answers to probabilistic questions concerning some 200 different geopolitical events of different significance (close to issues of security environment).
As a result of the research, Philip Tetlock and his colleagues concluded that the accuracy of the forecasts depended on a set of individual and professional skills such as:
Dispositional Variables – inductive reasoning; cognitive control; numerical reasoning; openminded style of thinking; greater tolerance to ambiguity allowing to make correct choice in favor of certain interpretation of events; readiness to change paradigm accepting new information; capability to determine the level of uncertainty of forecaster’s judgments and/or probability of future events; political knowledge;
Situational Variables – the research indicated that professional environment matters. The most
beneficial environment is when analysts have an opportunity to cultivate their skills and receive
feedback on the subject of forecasting. The other key to success is the formation of problem-solving
teams or groups of forecasters united by the same goal. This leads to information and experience
sharing, diversity of knowledge, and “cognitive altruism”: forecasters in teams shared news articles,
argued about the evidences, and exchanged rationales using self-critical epistemic norms. Forecasters
who worked alone were less accurate [[
It is worth admitting that American scientists Daniel Kahneman and George Klein hold a similar
view. They think that professional environment provides analysts with possibility to develop their
predictive skills and better opportunities to practice by sharing of their experience, knowledge, ideas,
and judgments [[
Behavioral Variables – reflect the ability of a forecaster to self-develop and to question his own
analytical paradigm concerning conducted forecasting. In this context analysts “should spend more time
researching, discussing, and deliberating before making a forecast” [[
At the same time Philip Tetlock admitted that a key feature of forecaster’s activity is making
nonnumeric (verbal) forecasts “that are vague and hard to score for accuracy, so feedback is often
absent” [[
In order to solve this problem, we propose to use Ukrainian experience as well as best foreign practices in the sphere of forecasting. In this article we would like to describe:
(1) some results of the trainings of Ukrainian national security and defense analysts dealing with prediction of security environment;
(2) the use of the Method of Goal Dynamic Evaluation of Alternatives (MGDEA) for automation of strategic geopolitical forecasting. 3. Practical results of national security and defense analysts’ trainings
In 2016-2021 dozens of seminars (trainings) were held with the representatives of Ukrainian national security and defense sector, which led to some important conclusions. The most accurate and argued forecasts were built as a result of creative combination of Structured Analytic Techniques (SAT). The most efficient techniques included the following ones: Structured Brainstorming; Cognitive Mapping; Force Field Analysis, DIMEFIL/PMESII Analysis; SWOT Analysis; Alternative Futures Analysis etc.
So, it makes sense to break the forecasting process into the following phases: 1. pre-forecasting orientation; 2. preparation of formalized incoming information; 3. key problem (question) formulation; 4. key problem (question) decomposition; 5. determination of key trends in security environment; 6. build-up of security environment cognitive model; 7. generation of security environment scenarios; 8. evaluation of these scenarios in the context of their probability and level of threat to the national interests of the state. 4. Application of the method of goal dynamic evaluation of alternatives to automation of the proposed action algorithm
The method was developed for evaluation of decision alternatives based on the goal hierarchical
model [
Usage of these software tools enables us to: create a network environment for remote work of forecasters as members of object-oriented analytical groups; ensure administration and recording/backlogging of all actions of analysts involved in the forecasting process; automatically perform pairwise comparisons of importance and influence of various alternatives (hypotheses, scenarios, trends, decision options, measures, etc.), which is extremely effective when working with verbal (non-numerical) information; create cognitive models of the development of a security environment or a specific situation; determine (through a distributed expert assessment) the degree of probability of the security environment development scenarios and the level of danger to the national interests of the state.
MGDEA [
Unlike other existing methods, for example, multi-criteria ones [
The method envisions performing a set of procedures (involving experts), related to construction of the KB (knowledge base) of the subject domain with the determination of a number of numerical parameters regarding the degree of influence of goals, delays in the spreading of influences, the duration of project implementation, etc. When the construction of the KB (i.e. hierarchy of goals) is completed, the method allows us to calculate the ratings (estimates) of decision variants (actions, measures, projects) based on the KB. 4.1.
The essence of the method application and its modifications
It is appropriate to single out the following main stages in the course of applying the MGDEA. In addition, we will consider a number of concepts related to the operation of the method.
The construction process begins with the formulation of the main goal of the problem and possible options for its solution, which ultimately need to be evaluated. The main goal is subject to decomposition into simpler components – sub-goals that affect the main goal. In the future, these formulated goals are also subject to decomposition. Moreover, the list of goals that affect the achievement of the current goal, in addition to the newly formulated goals, may include those already present in the hierarchy (previously formulated during the decomposition of other goals). The decomposition process continues until the sets of goals that affect the formulated goals consist only of the evaluated decision options and the already formulated goals. That is, the decomposition process stops when there are no undisclosed goals left.
The hierarchy of goals, which is built by experts, is presented in the form of a directed graph, the vertices of which are marked with goal formulations. Every arc in the graph signifies the influence of achievement of one goal on achievement of another. Thanks to the described process of building a hierarchy of goals, the graph corresponding to it is one-way connected, since from any vertex of the graph there is a path to the vertex that denotes the main goal. Each vertex (goal) is matched with an indicator of the degree of achievement ∈ ℝ, ∈ 1. . , – the number of goals in the hierarchy. 0 1 , moreover, 1 when the i-th goal is fully achieved, and 0 process in its achievement. It should be noted that each influence (represented by an arc in the graph) can be both positive and negative for achieving one or another goal. The degree of influence of one goal on the achievement of another is expressed by the corresponding indicator – the partial coefficient of – when there is no influence (PCI). In MGDEA, the change in PCI over time is taken into account, therefore, the PCI of the i-th goal on the j-th at the moment of time t is determined by the expression: 0, if , otherwise ,
is an expert assessment of the delay of the influence of the i-th goal upon the j-th goal. For goals such as projects (variants of decisions), the delay in their influence on one or another goal is usually increased by the duration of the project, determined by the expert.
MGDEA provides for the possibility of determination of PCI by expert means. To increase the reliability of such expert assessments, as a rule, the method of pairwise comparisons is used. Regarding the comparison of the degrees of influence of a set of goals on some defined goal, it should be noted that within the described model, compatible and incompatible goals are distinguished, which jointly influence some defined goal. Goals are considered compatible, if the achievement of one of them does not exclude the necessity and possibility of achieving the other, and incompatible – in the opposite case.
Since any comparisons and corresponding comparative estimates are worth applying only among
sets of mutually compatible objects, there is a need to find such maximum-cardinality subsets of objects,
that each of them
would include only
mutually compatible objects. In [
The algorithm that implements the method is characterized by significantly lower computational
complexity compared to the method described in [
What is given: a set of objects A={ ai }, iI, where I={1,2,…,n} is a set of indices; P is the set of pairs of compatible objects, i.e. pairs <ai;aj>, [ai,ajA], for which ai * aj = "true" ('*' is the introduced binary operation of compatibility).
We should find: As A (s I), such that m[(amAs)( * am = "true")] k[(ak As)(akA)] mI equality holds: ak * am = "false".
ammAIm
The proposed method of finding subsets of As in the set A assumes the determination of these subsets in the process of successive exclusion of elements from the set of pairs of incompatible objects. The algorithm of the method is iterative and the number of iterations is equal to the number of elements in (1) the aforementioned set of pairs. Before starting the execution of the algorithm, we assume that all pairs of objects are compatible, while, naturally, the set of compatible objects coincides with the original set (As=A). Let us assume, that, as a result of some experiment, the pair <ak; al> is deemed incompatible, so this pair is excluded from the set of compatible pairs.
It was proved in [
In addition to lower computational complexity of the algorithm, the proposed method provides an opportunity to determine (select) compatible subgroups of goals while providing information on the pairwise compatibility of these goals. A useful feature of the method is the ability to redefine compatible subgroups of goals after editing information about the compatibility of pairs of goals, when adding or removing some goal from the hierarchy. It is especially useful for MGDEA in the expert construction of a hierarchy of objectives that the redefinition of compatible subgroups of objectives occurs without reusing or entering information about the compatibility of those pairs that remained unchanged. 4.1.3. Determining the partial coefficients of influence
As mentioned above, the determination of the PCIs, denoted in (1) by , ∈ – the coefficients of the influence of goals with index j on some goal with index i in the hierarchy, takes place within the already defined compatible subgroups. Such PCIs are normalized values and for each k-th group of compatible goals satisfy the condition: 1, (2) where number of compatible goals in the k-th group.
is the PCI of the j-th goal upon the i-th goal in the k-th group of compatible goals; K is the Since goals can have both positive and negative influence, which is reflected by the sign of the corresponding PCI, before performing influence comparisons, goals that have a negative influence are replaced by opposite values (i.e., their logical negations). As a result, when determining PCI, all goals j in the k-th subgroup of compatible goals have a positive effect on the achievement of the i-th goal.
To determine PCI, it is suggested to use the methods of group pairwise comparisons, which
sufficiently satisfy the requirements to reliability of obtained results [
The basis of MGDEA is a generalized procedure for determining the degree of achievement of an
arbitrary goal of the hierarchy at a given moment in time t. As stated in [
of solution variants –
The essence of the calculation with the help of MGDEA of the rating (relative estimate) of the alternative (decision variant) corresponding to the l-th goal of the hierarchy at some point in time t is reduced to the determination of the difference between the degrees of achievement of the main goal – under condition of full achievement of all goals that correspond to those defined for comparison 0. That is, the rating of one or another alternative is the difference between the degree of achievement of the main goal in the presence of the influence of this alternative on the main goal and without it.
In order to expand the application area of MGDEA, it was proposed to improve the method with the possibility of calculating the rating of alternatives not only in relation to the contribution to the achievement of the main goal of the hierarchy, but also in relation to any other chosen goal. This allows comparing the influence of alternative decision variants on intermediate goals in the general domain 1 , ∈ , . . and under condition 1 , ∈ \ , model.
The process of calculating
– the degree of achievement of the selected i-th goal at the moment of time t can be described as follows. The goal hierarchy graph is searched for goals that do not influence other goals of this hierarchy, i.e., a set of vertices that do not include any arc of the graph is selected. From this set of goals (they are usually alternatives / projects), the calculation of degrees of goal achievement begins. The initial values of the degrees of achievement of the goals from this set are assigned, as just noted, equal to 1 or 0, although intermediate values are also possible, in case it is necessary to consider the incomplete realization of a project at a given moment in time. It is advisable to take into account the expert estimation of the degree of project realization when analyzing the intermediate states of the model and in the case of partial resource allocation for (financing of) projects.
It should be noted that, in general, the graph may have vertices that do not have incoming arcs.
Although, according to the logic of model construction, this is unlikely, and such a scenario was not
considered in [
Subsequently, a set of goals is formed that can be achieved directly from the goals belonging to the set formed at the previous step. The formation process is carried out through inclusion of all goals (graph vertices) that are directly influenced by goals from the previous set (which include arcs coming from the corresponding vertices). This set may also include goals belonging to the previous set.
For each goal from the formed set, the degree of its achievement at the moment of time t is determined. In fact, in the process of determining the degrees of achievement of goals, there is progress along the graph of the hierarchy from the goals of the lower level to the goals of the upper levels and, finally, to the main goal of the problem. In the case of the presence of reverse connections in the graph (arcs coming out from the vertices of higher levels and incoming the vertices of lower levels), the iterative process of determining the degree of achievement of goals is terminated in the case when the absolute value of the difference between the calculated values of the degree of achievement of the selected goal at adjacent iterations (x) and (x+1) does not exceed the specified accuracy ε: .
4.1.6. Input parameters and reference time points
The accuracy of ε calculations, as well as the planning period, are entered as input parameters. Based on the specifics of the tasks solved with the help of this DSS, the minimum unit of measurement of time intervals is a day (twenty-four hours), that is, planning is carried out with an accuracy of one day. By default, a recommended planning period is entered into a special form as the time period over which the relative ratings of the selected projects are calculated. This value (in days) is calculated according to the graph of the hierarchy of goals by moving from the vertices of the lower level to the upper level, similar to the process of determining the degrees of achievement of goals, but during this process, the sum of the delays in the spreading of influences forms the value of the maximum duration of the time period, beyond which changes in the relative project ratings’ calculation results no longer occur.
Although the MGDEA allows you to calculate the relative ratings of projects at any point in time from the start of their realization, since the calculated values of the ratings change only in the so-called reference points of the time axis, it is suggested to determine these points a priory (in advance) and only once, and not determine them before each iteration. In contrast to the iterative method proposed in [89] for determining the next
moment of time for calculating the degrees of goal achievement: , , ∈ 1,2, … , 1 ,
is the value of the delays of the effects of the goals in the hierarchy, which contains n goals, it is currently proposed to pass from the goals of the lower level to the upper level with the calculation and addition to the list of all possible delays of the effects of the goals in the hierarchy. The passage is organized simultaneously with the process of determining the degree of achievement of the main goal of the hierarchy, and in the presence of feedback, it continues until condition (4) is fulfilled. In fact, the formation of the list of goal influence delays is performed together with the calculation of the abovementioned recommended planning period, the value of which corresponds to the maximum value among the calculated influence delays.
In addition to the calculated ratings of alternatives at reference time points, for analyzing the dynamics of changes in the ratios between these ratings, an additionally developed viewing mode, the so-called integral assessment, proves extremely useful. In this mode, the graphs display the integral index of project efficiency i(t):
, which in essence is the integral over time t of the rating (relative estimation) function of the project r(t).
In the integrated evaluation view, the intersection points of the graphs (curves) reflect the moments of time since the start of the projects, when the ratio between the projects is changed in the sense of their dominance compared to others. Thus, it can be concluded that in a certain time perspective, preference should be given to the project whose graph turned out to be higher on the chart at a certain period of time. (4) (5) (6) 5. Proposed technology application for the forecasting of future security environment
In the course of this scientific research, the possibility of forecasting future security environment was practically confirmed. The purpose of the forecasting is to determine the main factors affecting the security of the state, as well as, in the future, to determine directions and measures to increase security. The proposed technology makes it possible to structure a complex system, which is the general security situation in the country, in the constructed model through sequential decomposition of the system. After creating a model of the security situation, the mechanism embedded in the technology allows us to calculate priorities (through determining the degrees of achievement of the goals - components of the system), make forecasts, predict the development of the security situation.
According to the technology, at the initial stage, structuring is performed through group work on the
decomposition of the problem. It is performed by knowledge engineers, as well as expert analysts, who
are joining the knowledge engineers in the decomposition process if necessary. The system of
distributed collection and processing of expert information “Consensus-2” [
Figure 1: Screenshot with an image of the goal hierarchy graph in the “Consensus‐2” system
At a later stage, when the model of the future security environment is built completely, including
the determination of all key parameters, its usage is connected with the application of the “Solon-3”
DSS [
Figure 2: Screenshot with the image of the model and calculations in the “Solon‐3” DSS
The combination of international and domestic experience of information and analytical activities makes it possible to develop approaches to solving certain problems of forecasting the security environment through structuring of the forecasting process itself and the application of the method of goal dynamic evaluation of alternatives.
This work was supported by the ECHO project which has received funding from the European Union’s Horizon 2020 research and innovation programme under the grant agreement no 830943.
[1] On the approval of the Procedure for conducting a defense review by the Ministry of Defense of Ukraine (Resolution of the Cabinet of Ministers of Ukraine dated October 31, 2018 No. 941). URL: https://zakon.rada.gov.ua/laws/show/941-2018-%D0%BF#n10. (in Ukrainian)