Experimental study of the model for calculating the quantitative criteria for assessing the security level of information and communication systems of the state critical infrastructure⋆ Sergiy Gnatyuk1,*,†, Viktoria Sydorenko1,†, Oleksii Yudin2,†, Andrii Paziuk1,† and Artem Polozhentsev1,† 1 National Aviation University, 1 Liubomyra Huzara ave., 03058 Kyiv, Ukraine 2 State Scientific and Research Institute of Cybersecurity Technologies and Information Protection, 03142 Kyiv, Ukraine Abstract In the context of rapid technological development and the introduction of Information Technology systems in all areas of life, including critical infrastructure management, today’s potential cyber-attacks can lead to very serious consequences. Therefore, the protection of such ICS has become critical in ensuring national security. Consequently, given the current requirements of national security and the need for a systemic approach to critical infrastructure protection, new approaches to ensuring the security of such infrastructure must be developed, which is now one of the most important challenges in the Ukrainian Defense sector. Therefore, there is an important and actual need to develop methods and models for the classification of the ICS as critical infrastructure to ensure the national security of the country. The paper develops a model for calculating the quantitative criterion for assessing the level of ICS security, which is based on the method of hierarchy analysis. The quantitative index of the security level is calculated by processing expert evaluations. It makes the procedure of expert selection easier, avoids the specifics of expert data processing, as well as to evaluates the ICS according to a limited number of statistical data. The model developed in the paper makes it possible to move from a qualitative assessment to a quantitative one, specifically, to move from an ordered series of alphanumeric combinations to a correlation of functional security profiles. Also, to verify the results and conduct experimental research, new software was developed, which is based on the model under study. Verification of the developed model was carried out based on the National Confidential Communication System. As part of future research, the authors will improve the developed model to apply it to other areas of critical infrastructure. Keywords information and communication system, critical infrastructure, critical infrastructure object, cybersecurity, security assessment criterion, functional security profile 1 1. Introduction The main open challenges to be met to achieve the above goal are the lack of unitary criteria and a specific procedure In the context of rapid technological development and the for attributing the ICS facilities to critical infrastructure; the introduction of Information Technology systems in all areas lack of unitary methods for assessing the level of protection of life, including critical infrastructure (Fig. 1) management, of critical infrastructure facilities of the ICS, etc. (Fig. 2). It today’s potential cyber-attacks can lead to very serious is important to note that according to the Law of Ukraine consequences [1]. “On the Fundamentals of Cybersecurity of Ukraine” [3], it is Therefore, the protection of such ICS has become necessary to create a list of critical information critical in ensuring national security. Consequently, given infrastructure facilities, for which it is necessary to develop the current requirements of national security and the need for criteria and methods of attributing such facilities to critical a systemic approach to critical infrastructure protection, new infrastructure. This is confirmed by the Decree of the approaches to ensuring the security of such infrastructure President of Ukraine [4], which provides that to ensure the must be developed, which is now one of the most important cyber security of critical infrastructure, it is necessary, challenges in the Ukrainian Defense sector [2]. especially, to determine the criteria for attributing information, communications, and the ICS to critical CSDP-2024: Cyber Security and Data Protection, June 30, 2024, Lviv, 0000-0003-4992-0564 (S. Gnatyuk); 0000-0002-5910-0837 Ukraine (V. Sydorenko); 0000-0002-5910-0837 (O. Yudin); 0000-0002-1622-1671 ∗ Corresponding author. (A. Paziuk); 0000-0003-0139-0752 (A. Polozhentsev) † These authors contributed equally. © 2024 Copyright for this paper by its authors. Use permitted under s.gnatyuk@nau.edu.ua (S. Gnatyuk); v.sydorenko@ukr.net Creative Commons License Attribution 4.0 International (CC BY 4.0). (V. Sydorenko); alex@ukrdeftech.com.ua (O. Yudin); inet.media.law@gmail.com (A. Paziuk); artem.polozhencev@gmail.com (A. Polozhentsev) CEUR Workshop ceur-ws.org ISSN 1613-0073 81 Proceedings information infrastructure. In addition, at the end of 2021, a organizational principles for the development and basic law in this area was adopted [5] (entered into force on implementation of the national system of critical June 15, 2022), providing the necessary legal and infrastructure protection. Figure 1: Critical infrastructure sectors by Guide for the Critical Infrastructure Community Austria. The Strategic Plan of the Austrian Critical Infrastructure Protection Program [6] defines the following global criteria: the number of citizens involved (health and social consequences); economic effect; environmental impact; psychological effect; political consequences; territorial extent; duration; lack of substitution options; interdependence of critical infrastructure sectors (destruction of one result in the destruction of others). Spain. The Law of the Kingdom of Spain on the establishment of measures to protect critical infrastructure [7] defines the following criteria for classifying an object as critical infrastructure: the number of citizens involved (deaths, injuries with serious injuries, and other serious health consequences); economic impact (economic losses and deterioration of products and services); environmental Figure 2: Up-to-date ICS interconnection impact; political impact (confidence in the public administration) and social impact (physical suffering, The mentioned regulations of Ukraine state the need to develop disruption of daily life). unified criteria and procedures for attributing the ICS Sweden. The Action Plan for the Protection of Critical infrastructure to the state’s critical infrastructure. It is Public Functions and Critical Infrastructure of the Kingdom of important to mention that the use of qualitative (rather than Sweden [8] defines critical facilities as those whose disruption quantitative) assessments is associated with the difficulty of results in the following: the number of citizens involved (about comparing them. Above all, such limitations are due to the 30 people killed or injured with severe injuries); the occurrence difficulty of selecting experts and the specifics of processing of economic effects or environmental impact (direct costs of expert data. As a consequence, there is an important scientific about 10 million euros); political consequences or social impact problem in determining the criteria for attributing the ICS to (citizens were killed, inability to influence the incident. critical information infrastructure. The Netherlands. The Dutch Ministry of Security and Justice Resilience Directive [9] divided infrastructure 2. Literature review criticality into two categories. Category A—Infrastructure disruptions would have the To determine the possible criteria for classifying an object following consequences: state financial loss of more than €50 as critical infrastructure, the analysis of the regulatory billion or a decline in revenue of about 5% in real terms; more documents of the European Union countries was performed. than 10,000 people would be killed, injured, or chronically ill; During the analysis of normative documents of Austria, more than 1 million people would be on the brink of survival or Spain, Sweden, the Netherlands, and Slovenia the following seriously mentally ill; at least two other critical infrastructure was found. sectors would begin to deteriorate. 82 Category B—Infrastructure disruptions would have the consequences described by a set of properties, following consequences: state financial losses of more than indicators, or factors. It is necessary to choose that €5 billion or a decline in revenues of about 1% in real terms; alternative, the result of which is the most more than 1,000 people would be killed, maimed, or preferred. By using the method, it is required to get chronically ill; more than 100,000 people would be at the a quantitative assessment of all possible outcomes, brink of survival or severely mentally injured. resulting from decision-making processes [2, 14]. The Republic of Slovenia. General and sectoral criteria for 2. The method of hierarchy analysis is a systematic defining the critical infrastructure of the national approach to complex decision-making problems. importance of the Republic of Slovenia [10] state that the Also implemented is a procedure to synthesize main criteria for defining critical infrastructure are: deaths priorities, that is calculated based on the expert’s of more than 50 people; health effects resulting in the decisions. The method makes it possible for the hospitalization of more than 100 people for a week; expert to determine a possible solution complications in the implementation of internal security of (alternative) for a problem, which would better the state; losses of more than 10 million euros per day; meet his comprehension of the problem and the inability to supply drinking water or food for a week for solution requirements. 100,000 people. 3. The method of the theory of fuzzy sets represents Summarizing the above and by [11] it can be concluded the formalization of the incoming values using a that the most common criteria for referring to critical vector of interval values (fuzzy interval), and each infrastructure are the following: the number of citizens interval is characterized by some level of involved (health and social consequences); economic effect uncertainty. The boundaries of potential values of (financial losses); environmental impact (pollution, parameters and their maximum values are destruction); political consequences or social impact specified based on the input data, the expert’s (citizens were killed, inability to influence the incident, experience, and intuition. reduced confidence in public administration, civil unrest, etc. Thus, the main parameter of any given method is the It is advised to evaluate the above criteria by qualitative membership function of an interval parameter [15]. There are and quantitative indicators. many advanced methods for the definition of the membership The analysis of existing decision-making methods was functions, for example, methods of pairwise comparisons, carried out in [2] to find the most adequate method for expert evaluations, linguistic terms based on statistical data, calculating the quantitative criteria for assessing the level of parametric, interval evaluations, and others [16]. ICS security. It was defined that decision-making methods The analysis carried out in this paper shows that the can be classified by the content and type of expert most effective methods are rule-based ones. Given the information that can be obtained [12–14]. In addition, the advantages and disadvantages of the above methods, to methods under study are decision-making methods under calculate the quantitative criterion for security assessment conditions of certainty as well as methods under conditions it was agreed to apply the method of hierarchy analysis. of uncertainty (fuzzy). According to [2], the following Also, in [2] the authors have proposed a calculation model methods are the most prospective in the opinion of the for the quantitative criteria for assessment of the ICS authors: security in the state’s critical infrastructure. In this context, Fig. 3 [21] demonstrates modern ICS security threats in 1. The method of the expected utility hypothesis different domains. determines that any possible action creates Figure 3: 5G security threats 83 However, this work provides only a theoretical justification block diagram of the above model for calculating the of the specified model without experimental research in a quantitative criteria for assessing the level of the ICS particular area of critical infrastructure. With this in mind, security based on the method of hierarchy analysis is the purpose of this work is to experimentally investigate the presented in Fig. 4 [2]. model for calculating quantitative criteria for assessing the The method of hierarchy analysis to determine the level of ICS security. correlation of alternatives (FSPB and FSPE) is carried out as follows: 3. Proposed model description The pairwise comparison matrices must be calculated for each criterion level (security criterion—level 1, security The model developed in the paper makes it possible to move service criterion—level 2, security service level criterion— from a qualitative assessment to a quantitative one, level 3): specifically, to move from an ordered series of alphanumeric combinations to a correlation of Functional Security Profiles 𝐴= 𝑎 (1) (FSP). The model inputs are the Basic FSP [18] (FSPB) and the Expert-approved FSP (FSPE). ND TPI 2.5-005-99, which where aij = wi/wj, wj is the value of the i criteria. th determines the FSP standard of the information being At the same time, aji = 1/aij and aii = 1, which means that the processed, contains the requirements for the protection matrix is positive and inversely symmetric. The following level of specific information against certain threats and Table 1 of the relative importance will be used to determine the known functional protection services to counteract these value. threats and ensure compliance with the requirements. A Figure 4: Block scheme of the model 84 Table 1 Relative importance scale of the criteria Verbal assessment of the expert aij Verbal assessment of the expert aij wi absolutely better than wj 9 wj insignificantly predominant wi 1/2 wi significantly better than wj 8 wj slightly predominant wi 1/3 wi much better than wj 7 wj predominant wi 1/4 wi better than wj 6 wj strongly predominant wi 1/5 wi strongly predominant wj 5 wj better than wi 1/6 wi predominant wj 4 wj much better than wi 1/7 wi slightly predominant wj 3 wj significantly better than wi 1/8 wi insignificantly predominant wj 2 wj absolutely better than wi 1/9 the criteria are equivalent 1 The comparison matrix for the security criteria is shown in Table 2. Table 2 The matrix for security criteria Confidentiality Integrity Availability Observability Confidentiality a11 a12 a13 a14 Integrity a21 a22 a23 a24 Availability a31 a32 a33 a34 Observability a41 a42 a43 a44 Matrices of pairwise comparisons are calculated for the n security criteria. Up to 4 matrices in total can be used. There Ai   aij , (4) i 1 are 22 matrices at most for the security level criteria. To calculate the set of eigenvectors of the matrix, the Ai  Ai aij , (5) geometric mean for each row of the matrix should be calculated: n max   Ai, (6) i 1 ai  n ai1  ai 2  ai 3  ain  n  j 1 aij , n (2) Calculation of the consistency index: where 𝑛 is a dimension of the matrix. To get the results normalized, the normalized priority max  m Jp  , (7) vector should be obtained: m 1 ai where m is the number of compared elements (matrix ai  n , size). (3) a j 1 j The index of consistency should be checked by calculating the coefficient of AC consistency according to It is necessary to check the consistency of local the formula: priorities. The largest eigenvalue of the matrix must be calculated: Jp Ac  , (8) Rc where Rc is the table value (Table 3). Table 3 Random consistency for matrices of order 2–9 Matrix size (n) 2 3 4 5 6 7 8 9 Random consistency (RC) 0 0.58 0.90 1.12 1.24 1.32 1.41 1.45 However, a comparison matrix must be revised and clarified vector of the higher-level criteria. The results are if the AC ≥ 0,10. summarized at the higher level. The global priority calculation by high-level criteria. n For each criterion of the lower level, the normalized Gi   ai bi , (9) priority vector is multiplied by the normalized priority i 1 where n is a number of the security level criteria. 85 Determination of the correlation of the alternatives (FSPB structural-functional method of formation of the FSP of the and FSPE). industry ICS. A global priority of confidentiality, integrity, availability, and observability must be calculated for each 4. Experiments and discussion FSP. The correlation of these global priorities, describing quantitative criteria, can be represented as an expression: In many countries of the world, the Information and Communications industry takes one of the first places on G FPZ B criticality after energy and transport [17, 19]. Given this, the VK AHP  , (10) experimental verification of the developed model was G FPZ E carried out on the example of the ICS of the National System of Confidential Communication (NSCC). To verify the where GFPZ B is the table value of the FSP for the model for calculating quantitative criteria, matrices of industry ICS, and GFPZ E is the FSP, which was obtained by pairwise comparisons for each level of criteria were constructed. For the security criteria (according to [18]) the the expert, using the structural-logical model and the comparison matrix is as follows in Table 4. Table 4 The matrix of comparisons for security criteria Confidentiality Integrity Availability Observability Confidentiality 1 a12 a13 a14 Integrity a21 1 a23 a24 Availability a31 a32 1 a34 Observability a41 a42 a43 1 For the security service criteria (according to [19]) the Table 6 comparison matrix will have the form presented in Tables The matrix of integrity criteria 4–7. The matrix of confidentiality criteria is presented in IE IT IA IR Table 4, where: CT is trusting confidentiality, СА is administrative confidentiality, CO is object reuse, CC is IT 1 a12 a13 a14 hidden channels analysis, and CE is confidentiality in the IA a21 1 a23 a24 exchange. IR a31 a32 1 a34 Table 5 IE a41 a42 a43 1 The matrix of confidentiality criteria Table 7 CT СА CO CC CE The matrix of availability criteria CT 1 a12 a13 a14 a15 AR AF AQ AD СА a21 1 a23 a24 a25 AR 1 a12 a13 a14 CO a31 a32 1 a34 a35 AF a21 1 a23 a24 CC a41 a42 a43 1 a45 AQ a31 a32 1 a34 CE a51 a52 a53 a54 1 AD a41 a42 a43 1 The matrix of integrity criteria is presented in Table 5, The matrix of observability criteria is presented in where: IT is trust integrity, IA is administrative integrity, IR Table 7, where: ON is registration, OI is identification and is recovery, IE is integrity in exchange. authentication, OC is reliable channel, OD is segregation of The matrix of availability criteria is presented in Table responsibilities, OP is the integrity of the Complex means of 6, where: AR is use of resources, AF is resistance to failures, protection, OT is self-testing, OE is identification during the AQ is quick replacement, AD is disaster recovery. exchange, OS is sender authentication, OR is recipient authentication. 86 Table 8 The matrix of observability criteria ON OI OC OD OP OT OE OS OR ON 1 a12 a13 a14 a15 a16 a17 a18 a19 OI a21 1 a23 a24 a25 a26 a27 a28 a29 OC a31 a32 1 a34 a35 a36 a37 a38 a39 OD a41 a42 a43 1 a45 a46 a47 a48 a49 OP a51 a52 a53 a54 1 a56 a57 a58 a59 OT a61 a62 a63 a64 a65 1 a67 a68 a69 OE a71 a72 a73 a74 a75 a76 1 a78 a79 OS a81 a82 a83 a84 a85 a86 a87 1 a89 OR a91 a92 a93 a94 a95 a96 a97 a98 1 For the matrices of security level criteria, as in our case, it is external analysis; ON-2 is protected log; ON-3 is danger necessary to make all of the possible 22 matrices for alarm; ON-4 is detailed registration; ON-5 is real-time criterion comparisons, according to Table 8, where ON-1 is analysis. Table 9 The matrix of security level criteria ON -1 ON -2 ON -3 ON -4 ON -5 ON -1 1 a12 a13 a14 a15 ON -2 a21 1 a23 a24 a25 ON -3 a31 a32 1 a34 a35 ON -4 a41 a42 a43 1 a45 ON -5 a51 a52 a53 a54 1 The scale given in Table 1 is used to fill in the matrices. The vector of priorities, calculated using (3) and the developed set of eigenvectors of a matrix is calculated using (2) and is software [14] was obtained. The consistency check of local calculated as the geometric mean for each matrix. The priorities was carried out by (4–7) also with the help of [14]. calculation is made using the specialized software At the same time, an error was made in the selection of developed by the authors [20]. As a result, a normalized priorities for accessibility services (Fig. 5). Figure 5: Possible error message Based on the results of the error analysis, the priorities of accessibility services were revised by the experts (Fig. 6). 87 Figure 6: Matrix of availability criteria The calculation of the global priority for the criteria of performed using (9). The result of the calculated ratio of confidentiality, integrity, availability, and observability is alternatives (FSPB and FSPE) is shown in Fig. 7. Figure 7: Result of the ratio of alternatives According to Fig. 7, the importance index of the describes the ratio of the FSPB to the FSPE, using qualitative confidentiality criteria, implemented in the NSCC, is indicators (security services) was developed. In follow-up significantly lower than the index, which is reasonable to studies, it is planned to use a model to calculate quantitative achieve. The ratio of global priorities, which characterize criteria for assessing ICS security in other critical infrastructure the quantitative security level, is calculated using (10). The industries (energy, transport, etc.) [22]. value of these criteria is: References 0, 417484 VK AHP   0, 716691 (11) [1] On the Main Principles of Ensuring Cyber Security of 0, 582516 Ukraine: officer. text, Kyiv: Bulletin of the Verkhovna Thus, the security level values of the main subsystems Rada of Ukraine, No. 45, Art. 403 (2017). of NSCC were obtained, using the developed model for [2] S. Gnatyuk, et al., The Model for Calculating the calculating the quantitative criteria for assessing the Quantitative Criteria for Assessing the Security Level security level of the ICS [22]. of Information and Telecommunication Systems, in: Intelligent Information Technologies and Systems of Information Security, vol. 3156 (2022) 390–399. 5. Conclusions [3] S. Gnatyuk, et al., Critical Aviation Information Therefore, a model for calculating quantitative criteria for Systems: Identification and Protection, Cases on Modern Computer Systems in Aviation (2019) 423– assessing the level of ICS security by processing expert 448. evaluations using the method of hierarchy analysis was [4] Decree of the President of Ukraine No. 96/2016 “On developed in the study. This made it possible to simplify the the decision of the National Security and Defense expert selection procedure, avoid the difficulties of expert Council of Ukraine dated January 27, 2016 “On the data processing, and carry out the ICS evaluation with a Cybersecurity Strategy of Ukraine”. limited amount of data. The developed model allows us to [5] About Critical Infrastructure: officer. text, Kyiv: move from a qualitative assessment in the form of an Bulletin of the Verkhovna Rada of Ukraine (2021). ordered series of alphanumeric combinations, denoting the [6] Masterplan Österreichisches Programm zum Schutz levels of realized services, to a quantitative assessment in Kritischer Infrastruktur (APCIP - Austrian Program the form of the correlation of the FSPB to the FSPE. Also, for Critical Infrastructure Protection). URL: https://www.bundeskanzleramt.gv.at/themen/sicherh the list of the NSCC components was obtained, using the eitspolitik/schutz-kritischer-infrastrukturen.html proposed model. There were identified 4 systems, 10 [7] Ley 8/2011, de 28 de Abril, Por La Que Se Establecen subsystems with Level 1, 34 subsystems with Level 2, and Medidas Para La Protección de Las infraestructuras 1036 constituent elements. In addition, the value of the Críticas, Boletín Oficial Del Estado 102 (2011). quantitative criteria of the security level was obtained, [8] Action Plan for the Protection of Vital Societal which is equal to. Functions & Critical Infrastruc- ture/Swedish Civil In addition, special software that implements the studied Contingencies Agency (MSB). Risk & Vulnerability model and allows to obtaining of a quantitative value that 88 Reduction Department. Natural Hazards & Critical [22] V. Sydorenko, et al., Experimental FMECA-based Infrastructure Section (2014). Assessing of the Critical Information Infrastructure [9] Ministerie van Veiligheid en Justitie. Directie Importance in Aviation, in: CEUR Workshop Weerbaarheidsverhoging (2015) URL: Proceedings, vol. 2732 (2020) 136–156. https://www.nctv.nl/actueel/nieuws/kabinet- versterkt-crisisbehe ersing.aspx?cp=126&cs=59950 [10] Osnovni in Sektorski Kriteriji Kritičnosti za Določanje Kritične Infrastrukture Državnega Pomena v Republiki Sloveniji (2012). URL: http://www.mo.gov.si/fileadmin/mo.gov.si/pageuploa ds/zki/SklepVlade- potrditev_osnovnih_in_sektorskih_kriterijev_kriticn osti2012.pdf [11] Law of Ukraine on Critical Infrastructure, 1882-IX, Kyiv: Bulletin of the Verkhovna Rada of Ukraine (2021) [12] T. Sarkar, et al., Mathematical Principles Related to Modern System Analysis, in Modern Characterization of Electromagnetic Systems and its Associated Metrology, IEEE (2021) 1–20. doi: 10.1002/9781119076230.ch1. [13] X. Guo, et al., Design and Implementation of Teaching Quality Assessment System based on Analytic Hierarchy Process Fuzzy Comprehensive Evaluation method, 8th International Conference on Orange Technology (2020) 1–3. doi: 10.1109/ICOT51877.2020.9468778. [14] O. Sandoval-Alfaro, R. Quintero-Meza, Application of Data Analytics Techniques for Decision Making in the Retrospective Stage of the Agile Scrum Methodology, Mexican International Conference on Computer Science (2021) 1–8. doi: 10.1109/ENC53357.2021.9534800. [15] Z. Hu, et al., A Multidimensional Extended Neo-Fuzzy Neuron for Facial Expression Recognition, Int. J. Intell. Syst. Appl. 9(9) (2017) 29–36. [16] Z. Ma, et al., An Improved Approach for Adversarial Decision Making Under Uncertainty Based on Simultaneous Game, Chinese Control and Decision Conference (CCDC) (2018) 2499–2503, doi: 10.1109/CCDC.2018.8407545. [17] S. Gnatyuk, V. Sydorenko, M. Aleksander, Unified Data Model for Defining State Critical Information Infrastructure in Civil Aviation, IEEE 9th International Conference on Dependable Systems, Services and Technologies (DESSERT) (2018) 37–42. [18] Normative Document of Technical Information Protection 2.5-004-99, Criteria for Assessing the Security of Information in Computer Systems Against Unauthorized Access, State Service of Special Communications and Information Protection of Ukraine (1999). [19] S. Gnatyuk, et al., Experimental Cybersecurity Level Determination in the Civil Aviation Critical Infrastructure, IEEE International Scientific-Practical Conference: Problems of Infocommunications Science and Technology (2020) 757–764. [20] Software for calculating the criticality factor of information and telecommunication systems, State Intellectual Property Service of Ukraine, Certificate of copyright registration for the work No. 9 (2018). [21] 5G Security Evaluation Process Investigation, Version 1 (2022). URL: https://www.cisa.gov/sites/default/files/publications/ 5G_Security_Evaluation_Process_Investigation_508c. pdf 89