Modern methods for developing problem situation scenarios using automated solutions on a knowledge transfer platform are considered. A two-stage process for creating models of possible developments is described, which includes event forecasting and scenario generation. The use of a knowledge transfer platform allows integrating and reliably storing data, analyzing the impact of key factors, and ensuring adaptability to changing conditions. The scientific novelty of the research stays in the development of new modeling methods, the use of artificial intelligence for data analysis, and the implementation of knowledge integration tools. The presented results are important for making strategic decisions in public administration, military affairs, and business under conditions of uncertainty.
In modern conditions, uncertainty is a key factor influencing decision-making. This creates the need to develop scenarios of possible events using automated solutions. The relevance of such approaches is especially growing in medium- and long-term forecasting, public administration, the military sector, and business.
Automating scenario generation processes using knowledge transfer platforms [1, 2] provides not only efficiency, but also adaptability to changing conditions. This allows analysts to focus on deep interpretation of data and identification of alternative possibilities of development of events. In addition, conditions are created for the application of collective work of expert analysts in the formation of models of subject domains, allowing the use of their knowledge more fully and, thereby, to increase the adequacy of the created models and the quality of recommendations based on them.
According to the requirements and methods of group decomposition [3] and expert evaluation with decision support [4, 5], a clear sequence of actions has been formed that allows the analytical group to systematically create realistic scenarios and automatically generate them in the required quantity to meet the customer's needs.
Methodology proposed in [6] is based on the decomposition of the problem to identify key areas of activity (using the technology implemented in the “Consensus-2” system [7]) and the subsequent ranking of influence factors using DSS (“Solon-3” [8]).
All stages are performed using automated solutions on a knowledge transfer platform, which ensures systematization and improvement of the quality of forecasts.
The main aspects of the scientific novelty of this study include the following: A new method for developing scenarios based on problem situation models that take into account cause-and-effect relationships between events has been developed. A toolkit has been implemented for analysts to obtain knowledge from various information sources. The use of artificial intelligence is proposed to automate analysis processes and increase the awareness of analysts.
The main purpose of scenario development is to create models of possible developments using automated solutions. These models are not predictions, but they take into account the influence of key factors, demonstrating different scenarios of the future.
It has been researched that for effective implementation it is advisable that the scenario generation process be based on: Using knowledge transfer platforms to integrate existing information about the problem situation and the knowledge of analysts. Using indicators that signal changing conditions or confirm certain events. Expert assessment of factors influencing the course of events.
This provides the creation of structured scenarios that cover a wide range of possible alternatives. In addition, it provides the opportunity to assess the likelihood of a particular scenario and give reasoned recommendations for timely response and effective influence on the course of events. Assessment and prioritization are carried out considering the conditions for maximizing the achievement of a certain pre-formulated main strategic goal.
The methodology is based on modeling a problem situation and predicting its development.
The main stages include the following: 1. Pre-forecast orientation 2. Preparation (formalization) of input data 3. Formulation of the main problem (problem situation) 4. Problem decomposition 5. Formulation of the main trends in the development of a problem situation 6. Creating a model of the external environment 7. Generating scenarios for the development of a problem situation 8. Assessing the likelihood of scenarios and their impact on the problem situation 9. Writing a predictive assessment 10. Setting the task of obtaining (clarifying) information that is missing in the model of the external environment.
Let's consider the stages of scenario development in more detail.
At this stage, you need to get answers to several questions related to the overall modeling and forecasting process. Who is the consumer of predictive assessment? What is the object of forecasting? For example: forecasting the development of a problem situation. What is the forecasting time horizon? For example: short-term / long-term forecast. What is the scale of the forecasting? For example: at the regional/state level.
The available collected data is processed by analysts for further use. For this purpose, they are systematized and structured according to a single form. Typically, analysts fill out electronically prepared form standardized according to template (uniform / formulary). It should be understood that the input data for the system are facts and conclusions from the original thematic and current documents.
3. Formulation of the main problem (goal / problem situation) carried out using the brainstorming method [9]: the question must have the following attributes: 1. object(s); 2. location (country, region, etc.); 3. the period of time for which the forecast is made.
The organizer-analyst (OA) sets a problem situation (problem) to which the scenarios of the events development are related. The OA formulates a short title of the problem, as well as a full text description of the problem situation.
An example of the OA web interface when performing this stage in the “Consensus-2” system is presented in Fig. 1.
The formulated main goal of the problem is subject to decomposition into simpler components – goals that influence (significantly) the main goal. These formed goals are subject to decomposition too. The list of goals that influence the achievement of the current goal, in addition to the newly formulated goals, may include those available in the hierarchy (previously formulated during the decomposition of other goals).
The decomposition process continues until the set of goals that influence the goals to be decomposed consists only of the decision variants being evaluated and the goals already decomposed. Accordingly, the question of completing the decomposition process is related to the fact that in which case should a goal be considered as a decision variant that is not subject to decomposition?
The answer to this question is strictly defined, namely, when building a model of a problem situation, during the decomposition, the goal as a decision variant should be verified if two conditions are met simultaneously: 1) it is possible to clearly define the time required to fully achieve the goal (the period of execution/implementation of this decision variant); 2) it is possible to clearly define the necessary (financial) resources to ensure the full achievement of this goal (implementation of this decision variant) within a certain period.
It is the responsibility of the OA to verify the goal as a decision variant and, thereby, decide on the need for further decomposition of this goal. It is the OA that is authorized to perform the decomposition of the goal or initiate a group expertise on this issue.
The decomposition process is the main stage of building a problem situation model. This process is associated with going through the goal hierarchy graph “from top to bottom” (from the main goal to the specified decision variants).
The result of the decomposition is a network-type graph – a hierarchical tree-like structure – an example of which in the “Solon-3” system is shown in Fig. 2.
After the top-down process, the next step is to supplement the decision variants formulated
during the decomposition with any decision variants that could potentially influence the problem.
The vertices of the graph (see Fig. 2) corresponding to the goals are represented as squares with
rounded corners, while those corresponding to the decision variants are depicted as regular squares.
This way, possible variants of solving the problem are generated, which finally need to be evaluated
with respect to their impact on the main goal. The generation of decision variants is possible using
methods such as morphological analysis [
Decision variants can be conveniently classified as: by objects (power centers); by domains using decomposition formulas of different levels: global, tactical and others; by scale: global, regional, local; by time: for example, chronology.
The OA performs problem decomposition – it specifies a list of components of the problem. The components can be selected from a list of existing components, or created as new ones with a description: Short wording (title); Full text or description of the component; Component characteristics – formula (quantitative, if it can be described numerically, or qualitative in the opposite case, and also, there is a possibility of expanding the set of characteristics).
To carry out the decomposition, the OA may involve other OAs for organization and analysts (A) for execution. The methodology provides for the possibility of remote work for all categories of users.
Carrying out collective decomposition can be divided into the following stages: The OA may appoint another (additional) OA to carry out this specific decomposition. The OA appoints and manages a group of A's competent in the given problem being decomposed and manages the progress of the group decomposition, personally participating in the group's work as an A. Each of the A's from the created group gains access and has the opportunity to participate in the group decomposition (formulates a list of the most important components in his opinion, adding existing components to the list and creating new ones if there are none). The OA, having the ability to monitor the work performed by the group of A's assigned, can suspend the process of formulating components and proceed to the next stage of group work on decomposition. The OA, having received all the impersonal (without indicating the authorship) formulations of the components that performed the A-s, groups them into groups of the same content, the OA can also add a number of its own formulations. For the preliminary automatic grouping, AI tools can be used, such as large linguistic models (LLM) or neural network tools for clustering formulations. Next, the OA moves the process to the next stage. A-s have the opportunity to express their opinion by voting for the choice of one of the formulations in each group of formulations with the same content, the one that best reflects the content of the component. It should be noted that each A has the opportunity not to participate in a specific separate vote, as well as the opportunity to indicate that none of the listed formulations should be included in the decomposition. The OA, having the ability to monitor the voting progress of the A-s, makes a decision to complete the voting stage and, in this case, the A’s votes are aggregated taking into account the relative competence of an A in the group on the issue of the subject of decomposition. In this case, one of the formulations in the group identical in content (by the maximum total weight of votes) can be selected as the name of one of the most important components and be included in the decomposition, or, in the case when the maximum total weight of votes corresponds to the non-selection of any of the formulations in the group identical in content, then none of the formulations from this group is selected as a component in this decomposition. OA determines the characteristics of components and relationships between components formed as a result of group decomposition (quantitative or qualitative assessment, resource required for achievement, implementation period, positive or negative influence, influence propagation time, etc.).
During the group expertise, in which A-s participate as experts, the areas of information analysis are determined by experts. The expert choice of specific areas depends on the purposes and tasks of the analysis, the expected results, the timeframe for conducting the research, etc. So, the group of As performs: Structuring and processing information to achieve the main goal according to predefined scenarios. Determining the levels of interaction between spheres and the weighting factors of their influence on a specific scenario.
At the same time, in the process of expertise, to determine the areas of information analysis, the following models can be used: Well-known analysis models (tools) or their combination. Models based on internal documents (instructions), in accordance with the tasks set or the requirements of the governing documents.
At the end of the decomposition stage of the main problem, after the top-down process and the addition of solution options, it is advisable to implement another stage of building a model of the problem situation. This is the stage of passing through the graph of the hierarchy of goals from the bottom up. This stage serves to ensure the completeness of the model, namely, the completeness of the relationships between components – influences between goals – specified in the model.
At this stage, all goals of the goal hierarchy are reviewed, starting from the lower-level goals (from the decision variants) and ending with the upper-level goal (the main goal), for the presence of important impacts on other goals along with the impact on the decomposed goal. Thus, the goal hierarchy graph is supplemented with inter-branch connections and the model becomes more adequate to the subject domain being modeled (to the real problem situation). Here, if they are available, feedback loops in the hierarchy are also set – the impacts of goals of lower levels on goals of higher levels.
5. Determining trends in the development of a problem situation 5.1 The formulation of the main trends is carried out by synthesizing the various components of the problem obtained as a result of decomposition. Here, the term "trend" should be understood as: a stable direction of influence on the problematic situation of such environmental factors as global warming, rising energy prices, development of alternative energy sources, development of new technologies, emergence of epidemics, etc.; a stable direction of development of relations between actors (centers of power), for example, relations between military-political blocs, countries, political forces, movements, etc.; a stable direction of change in the internal environment (center of power), which includes changes in the internal political, socio-economic, demographic situation, changes in legislation, the health of managers/leaders, etc.
To synthesize information elements obtained as a result of decomposition, it is advisable to apply the following methods: Stakeholder analysis method – allows to identify the strategic goals of the conflicting parties, to rank these objects in terms of influence (importance) and available resources. To assess influence, it is possible to use the coefficient of "cumulative state power" ( PwrIndx) for each country [11, 12]. force field analysis method (FFA) [13] – allows you to identify the main driving forces and restraining forces for achieving the relevant (main) goals, as well as to substantiate them. SWOT analysis method [14] – allows, through the prism of driving forces and restraining forces for achieving the relevant (main) goals, to identify their internal strengths and weaknesses, as well as external factors that pose both threats to their achievement of their goals and enable them to be achieved. brainstorming [9] – generating a large number of ideas and concepts to solve problems and explain events and phenomena. goal matrix [15] is a model for prioritizing, which allows you to highlight the most important tasks, eliminating everything unnecessary.
performed using the method of targeted dynamic evaluation of alternatives (MTDEA) [4, 5], which
includes the application of the method of pairwise comparisons [
As a result of the group expert assessment, the relative weights of the influences (relationships) are determined: the assigned analysts and the OA for each decomposition component can assess the importance of the component (its influence on the problem or the component being decomposed) on the specified scale, and when choosing the pairwise comparison method, they can assess the advantages of the components for each pair of components on a verbal scale.
An example of implementing individual pairwise comparison to determine the relative weights of objects (alternatives) in DSS "Solon-3" is shown in Fig. 3. During the comparison, the ratio between the weights is determined on a verbal scale with a corresponding graphic hint about the selected ratio.
Collective evaluation is carried out as follows: For each decomposition, the OA chooses a group expert evaluation method (this can be direct evaluation, evaluation on a point scale, pairwise comparisons, etc.). The OA creates a group by assigning A-s for expert evaluation. The OA has the opportunity to participate in the expert evaluation along with the assigned A-s. If the direct assessment method is chosen, the OA sets the scoring scale (maximum number of points) that will be used in the assessment. Assigned A's and OA's for each decomposition component can evaluate the importance of the component (its impact on the problem or the component being decomposed) on the specified point scale, and when choosing the pairwise comparison method, they can evaluate the advantages of the components for each pair of components on a verbal scale.
Expert assessments have been expanded to include the possibility of using different scales [19], taking into account the level of relative competence of the assessment participants [20], determining the consistency of estimates [21] and organization of feedback during assessment in order to increase the consistency of assessments when generalizing [22].
OA, after completing the impact assessment, determines the most important factors among those obtained as a result of the problem decomposition. For this, a numerical rating of the importance of factors should be calculated (the most influential factor is in the first place). And, since we have a graph decomposition model of the “tree” type (without cycles), then, following the example of the Analytic Hierarchy Process (AHP) method [23], a weighted convolution can be applied. In the future, for “network” type models, the possibility of using MTDOA is envisaged [4, 5].
To assess the impacts in the region and rank factors by influence, it is also possible, instead of conducting an examination, to use the indicator “aggregate power of the state” (PwrIndx) [24] for each country.
Further development of a model of the problem situation using the cognitive mapping (modeling) method, which allows reflecting external connections between actors (countries) and/or trends in the development of the problem situation, as well as assessing their mutual influence.
At this stage, a group of methods for constructing a "story" are involved. [25] and cognitive map [26] the behavior of the country as an actor, taking into account behavior at the local, regional 20 and global levels using retrospective analysis, analysis of connections [27] etc.
The scenario generation (SG) method used is designed to identify the most influential and, at the same time, unpredictable factors (the main driving forces and restraining forces), which allows for the identification of a number of likely alternative future options and the creation of a number of scenarios on which political courses, plans and strategies can be tested.
Solving the SG problem is proposed by following the following steps: OA, based on an ordered list of factors – the rating obtained at one of the previous stages, a pair of unpredictable factors is determined by conducting a collective assessment according to the criterion of “predictability”. This is done by determining the first two unpredictable factors from the rating list, starting with the most important. As a result of this assessment, the so-called “pseudo black swans” are determined – a pair of the most influential unpredictable factors. Based on each such pair of important and unpredictable factors, OA forms four alternatives to the predicted event (situation). Such alternative situations form the basis of the names of alternative scenarios, which are formed during the new group decomposition of the problem situation initiated by the OA. Analysts and possibly OA involved in decomposition usually formulate four alternative situations, each of which is formed on the basis of alternative consequences from the influence of a pair of unpredictable factors. It is possible to consider not two, but a larger number of important factors, as well as a larger number of alternative states of each unpredictable factor, then the number of alternative scenarios will be greater than four (6, 9, 12, etc.). OA manages the further group decomposition of each of the unpredictable factors (the isolation of alternative states in them). Decomposition involves formulating a list of events that are necessary, in the analyst's opinion, for the current state (which is being decomposed) to occur. Analysts participating in group decomposition formulate events, the occurrence of which leads to different paths of development of the problem situation (these events are indicators). OA estimates the time delays of the occurrence of interconnected events. That is, in the graph model, the arcs corresponding to the cause-and-effect relationships between events are assigned time delays determined by experts. OA defines relationships that can be positive and negative depending on whether an event contributes to or hinders the occurrence of a certain state (a certain event). Events identified as a result of decomposition (leaves of a graph-tree) that negatively affect (prevent) the occurrence of a certain alternative situation represent threats to the implementation of a certain scenario. The result is the final graph of interconnected events in the Consensus-2 system.
The final stage of solving the scenario generation problem: 1) A target is chosen that corresponds to the intentions of a certain "player" (this can be either one of the customer's opponents or the customer of the scenarios himself). 2) Establishing and determining the types (positive or negative) and level of impact of formulated possible consequences (results) of the occurrence of a certain predicted event on the achievement of the selected goal. 3) Decomposition of each of the predicted events (scenario names), each of which is led by certain sequences of situations (events). 4) Continuing sequential decomposition with branching to detail the hypothetical sequence of events.
In the sense described above, Scenarios are chains in a graph from the vertices of the lower level (indicators) to the vertices - possible consequences of the occurrence of a certain event (the level of the hierarchical structure of the tree-like graph, which is the next level after the root vertex). The 21 possibility of implementing a particular scenario is determined by the indicators of the occurrence of key events for the problem. The assessment of the probability of the occurrence of events (loading the edges of the graph with relative values) is carried out using group expert methods. Depending on the main goal, which was chosen in accordance with the intentions of a certain "player" and the results of assessing the probabilities of the implementation of scenarios, they are divided into categories, such as optimistic or pessimistic, realistic or unlikely in a certain period of time.
8. Assessing the probability of scenarios and their impact on the problem situation
Scenarios are assessed in terms of probability of implementation using:
- the Competing Hypotheses Analysis (CHA) method [28];
- methods using expert pairwise comparisons [
Procedure for assessing the probability of scenario implementation
The OA initiates a group assessment, as a result of which the probabilities of events occurring are estimated, and each connection is assigned the importance of the event occurring for the occurrence of a certain state (a certain event).
A separate scenario consists of a description, in some detail, of a hypothetical sequence of events that could likely lead to a predicted event.
The OA receives scenario options as chains of events leading to a specific alternative state (a specific scenario name).
The scenario description is detailed from the fields of the extended description of the formulations provided by analysts and OA, as well as taking into account the data of combining information processing methods.
Competing Hypotheses Analysis Method
Although the CHAM does not provide an exact numerical probability in the classical statistical sense, it allows us to rank scenarios according to their relative likelihood based on the available evidence. The main steps of the method are: 1. Identifying competing scenarios (hypothesis): It is necessary to clearly articulate all possible scenarios that we need to evaluate. Each scenario is considered as a separate hypothesis about the future. 2. Gathering and listing evidence (characteristic): All available information that can be used to confirm or refute any of these scenarios is gathered. This can be data, facts, observations, expert opinions, etc. Each piece of information is called "evidence" or " characteristic". 3. Evaluation matrix: This is the central part of the method. A matrix is created, with competing scenarios arranged horizontally and evidence arranged vertically. For each cell of the matrix, an assessment is made of the extent to which the given evidence agrees (confirms) or disagrees (refutes) with the given scenario. This assessment can be carried out, for example, on the following numerical scale: - +2 – very strongly confirms the scenario; - +1 – confirms the scenario; - 0 – not relevant or does not provide information; - -1 – refutes the scenario; - -2 – strongly refutes the scenario. 4. Identifying unique evidence: Particular attention is paid to evidence that strongly supports one scenario and strongly refutes another (or others). This is the "diagnostic" evidence that is key to distinguishing between scenarios. 5. Review and Discussion: After completing the OA matrix, the A (or team of analysts) reviews the results. The goal is to identify the scenario that has the most supporting evidence and the least refuting evidence. It is also important to check for evidence that is inconsistent with any scenario or consistent with all scenarios – this may indicate the need to revise the scenarios or seek additional evidence. 6. Determining the most plausible scenario: Based on the scores in the matrix and discussion, the scenario that is most plausible is determined, that is, the one that has the greatest amount of supporting evidence and the least amount of refuting evidence.
The probability of each scenario is estimated as a weighted convolution: the sum of the probabilities of the occurrence of events multiplied by the importance of the contribution of these events to the occurrence of a certain state (a certain event).
The sign of each term depends on whether this contribution is positive or negative. OA and analysts have at their disposal tools for accessing (sampling by various parameters) the database of information messages.
Counting the number of information messages allows us to draw conclusions about the importance of factors, the probability of events, etc.
To simplify the process of obtaining initial data for preparing final proposals and forecast estimates, and to increase their objectivity and accuracy, the analytical report within the framework of the most likely scenarios for the military-political leadership should be formalized according to certain criteria.
Thus, the task of defining indicators for tracking (in general, and according to decomposition formulas) and monitoring, detailing threats (5 levels) is specified.
Construction at the stage of determining trends of a priority series of key factors of the target, resources, driving forces and deterrent forces, opportunities and threats for generating scenarios in accordance with the identified threats to the national security of Ukraine.
At all stages of scenario generation, have the ability (interface) to annotate (argument) steps (goals, subgoals, areas, levels of influence, etc.) and the corresponding results of information processing.
In order to increase the efficiency and functionality of the method of targeted dynamic evaluation of alternatives [4, 5], the appropriate planning interval for each formed knowledge base (subject area model) was determined (calculated). During this process, it was proposed to form a set of reference points of the time scale [5] for further calculation of project ratings at these time points, which allows for a wider practical application of the method in modern decision support systems.
At this stage, the main results are formed in textual (descriptive) form. Typically, 4 possible scenarios of the development of the problem situation are described, based on the possible consequences caused by a pair of important and unpredictable factors ("pseudo black swans"). The description of each scenario includes: script name; the probability of its implementation (expert assessment on a verbal scale: “high”, “medium”, “low”); the degree of threat to the achievement of the global strategic goal on a verbal scale: “low”, “medium”, “high”; description (detailed description) of the scenario; assumptions (conditions for the implementation of a particular scenario); implementation indicators for a specific scenario are key events, the occurrence of which leads to different paths for the development of a problem situation.
10. Setting the task to obtain (clarify) information that is missing in the model of the external environment
We can distinguish the following two categories of information that are important for developing scenarios and their subsequent support: this is information that can supplement the created model and information about changes in the model as a result of obtaining new knowledge about the subject area.
The completeness of the knowledge used is one of the main criteria for the adequacy of the model [33], therefore, the model is analyzed for completeness, and, in addition, there may be a lack of knowledge when assessing the model parameters. When performing expert assessment, it is proposed to use scales of different detail [19], where the detail of the assessment scale corresponds to the competence (awareness) of the expert in the issue under consideration. During the assessment, the expert is invited to gradually increase the detail of the scale used and complete the assessment in a scale corresponding to the level of his/her awareness (competence) in the current issue. Thus, the lack of knowledge in a certain issue when assessing the model parameters may be indicated by the low detail of the scales in which the final expert assessments were obtained.
To maintain the adequacy of the model in the conditions of constant changes in subject areas, as well as, with the development of cognition, the constant acquisition of additional knowledge about the subject of modeling, it is necessary to constantly update the model. This update is carried out by periodically repeating the decomposition process used in modeling, involving new expert analysts in repeated group examinations, etc. Additional information is also necessary to update the relationships between the components of the system that are implemented and represented in the model.
Scenario development using automated solutions on knowledge transfer platforms is an important tool for strategic planning. It helps improve the quality of decision-making in areas with a high level of uncertainty, such as public administration, military affairs, and business.
This approach can become the basis for innovative management of future challenges.
Further research is planned to focus on improving the model of the subject area, namely, taking into account possible changes in the relative impacts of goals during the strategic planning interval. For cases of reliable prediction of such changes, this can significantly increase the adequacy of the developed model and the quality of strategic planning results.
The author(s) have not employed any Generative AI tools.
[1] V. V. Tsyganok, I. V. Borokhvostov, and P. D. Roik, "Problem-oriented knowledge transfer platform for decision making support in socio-technical systems", CEUR Workshop Proceedings: Selected Papers of the XVII International Scientific and Practical Conference on Information Technologies and Security (ITS 2017), Kyiv, Ukraine, 2017, vol. 2067, pp. 112–117. [2] V. Tsyganok, A. Astakhov, V. Minas, and M. Konovaliuk, "Knowledge Transfer Platform Toolkit for Strategic Planning", CEUR Workshop Proceedings: Selected Papers of the XXII International Scientific and Practical Conference on Information Technologies and Security (ITS-2022). CEUR Workshop Proceedings, 2022, vol. 3503, pp. 21–31. [3] V. G. Totsenko, Methods and systems for decision-making support. Algorithmic aspect, K.:
Naukova Dumka, 2002. [4] V. G. Totsenko, "One Approach to the Decision Making Support in R&D Planning. Part 2. The Method of Goal Dynamic Estimation of Alternatives", Journal of Automation and Information Sciences, vol. 33, no. 4, pp. 82–90, 2001. [5] V. V. Tsyganok, "Improvement of the method of target dynamic evaluation of alternatives and features of its application", Data Recording, Storage and Processing, vol. 15, no. 1, pp. 90–99, 2013. [6] V. V. Tsyganok and P. D. Roik, "Technology of scenario generation under conditions of uncertainty", Data Recording, Storage and Processing: Collection of scientific works based on the materials of the Annual Final Scientific Conference K: IPRI NAS of Ukraine, pp. 119–121, September 28, 2020. [7] "Computer program 'System of distributed collection and processing of expert information for decision-making support systems 'Consensus-2'", V. V. Tsyganok, P. D. Roik, O. V. Andriychuk, S. V. Kadenko, Copyright No. 75023, 27 Nov. 2017. [8] "Computer program 'Solon-3 Decision Support System' (Solon-3 DSS)", V. G. Totsenko, P. T.
Kachanov, V. V. Tsyganok, Copyright No. 8669, 31 Oct. 2003.
[9] A. Furnham, "The Brainstorming Myth", Business Strategy Review, vol. 11, no. 4, pp. 21–18,
2000, doi: 10.1111/1467-8616.00154.
[
Durham, NC: Duke University Press, 1938.
[14] R. W. Puyt, F. B. Lie, F. J. De Graaf, and C. P. M. Wilderom, "Origins of SWOT analysis", Academy
of Management Proceedings, vol. 2020, no. 1, p. 17416, 2020. doi: 10.5465/AMBPP.2020.132.
[15] S. R. Covey, The 7 Habits of Highly Effective People. New York, NY: Simon & Schuster, 2013.
[16] S. Kadenko, V. Tsyganok, Z. Szadoczki, and S. Bozoki, "An update on combinatorial method for
aggregation of expert judgments in AHP", Production, vol. 31, pp. 1–17, 2021. doi:
10.1590/01036513.20210045.
[17] S. Kadenko, V. Tsyganok, Z. Szádoczki, S. Bozóki, P. Juhász and O. Andriichuk "Improvement of
Pair‐wise Comparison Methods Based on Graph Theory Concepts", CEUR Workshop
Proceedings: Selected Papers of the XXI International Scientific and Practical Conference
"Information Technologies and Security" (ITS 2021), Kyiv, Ukraine, December 9, vol. 3241, pp.
46-55, 2021.
[