Cybersecurity Indices: Review and Classification Mykola Khudyntsev1,2, Andrii Davydiuk3, Oleksiy Lebid4, Oleksandr Trofymchuck4, and Artem Zhylin3 1 International Cybersecurity University, 171, 6, Deputats’ka str., Kyiv, 03115, PO Box 10, Kyiv, 04050, Ukraine 2 G.E. Pukhov Institute for Modelling in Energy Engineering of the National Academy of Sciences of Ukraine, 15, General Naumov str., Kyiv, 03164, Ukraine 3 State Center of Cyber Defense of the State Service of Special Communication and Information Protection of Ukraine, 83B, Yu. Illienko str., Kyiv, 04119, Ukraine 4 Institute of Telecommunications and Global Information Space of the National Academy of Sciences of Ukraine, 13, Chokolivs’kyi boulevard, Kyiv, 03186, Ukraine Abstract The report is devoted to analyzing and classifying cybersecurity indices developed and implemented by leading global, international, and local organizations as of early 2021. It is proposed to include regular (periodic) information materials, which contain expert, analytical, statistical information on the state of cybersecurity and the level of protection of indexing subjects (rating), as well as on certain indicators regarding the harmful impact of implemented information security and cybersecurity threats, to cybersecurity indices. 65 existing cybersecurity indices and approaches for their formation are described. The definition of the terms necessary for the analysis of indexing (rating) in the field of information security and cybersecurity is offered. Keywords 1 Cybersecurity, information security, indices, indexes, ratings 1. Introduction Research on cybersecurity indicators is an extremely important and urgent task in the field of global security [1–3]. In Ukraine, this task is solved in the framework of building an organizational and technical model of cybersecurity and cyber protection for the national cybersecurity system [4]. The topics of the research are the current world reports, indices, and ratings in the field of information security and cybersecurity. The aim of the study is the describing, review analysis, and classification of cybersecurity indices (ratings) with the proposition of including all different regular (periodic) information materials, expert, analytical and statistical reports, and data. Systematically for mentioned well and little-known indices (rankings, ratings) in the field of information security and cybersecurity we proposed the classification by types and categories, analyzing the main methods of forming these indices to review and analyze the current state of cybersecurity, cybersecurity indices and ratings, approaches to their formation, as well as to determine the main terms required for indexing (rating) in the field of information security and cybersecurity. The paper schematizes information about known cybersecurity indices (ratings), proposes their classification by types and categories and analyzes the main methods of forming these indices (ratings). CPITS-II-2021: Cybersecurity Providing in Information and Telecommunication Systems, October 26, 2021, Kyiv, Ukraine EMAIL: mykola.khudyntsev@icu-ng.org (M. Khudyntsev); andrey19941904@gmail.com (A. Davydiuk); o.g.lebid@gmail.com (O. Lebid); itgis@nas.gov.ua (O. Trofymchuck); zhylinartem@gmail.com (A. Zhylin) ORCID: 0000-0002-9324-6901 (M. Khudyntsev); 0000-0003-1238-2598 (A. Davydiuk); 0000-0002-4003-8068 (O. Lebid); 0000-0003-3358- 6274 (O. Trofymchuck); 0000-0002-4959-612X (A. Zhylin) ©️ 2022 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). CEUR Workshop Proceedings (CEUR-WS.org) 117 2. Cybersecurity Indices: Rankings, Ratings 2.1. Cybersecurity Indices: Definitions, Types, Categories The subjects of indexing (rating), depending on the type of index (rating) are countries, industries (sectors) of the economy, corporations, and organizations (entities). The objects of indexing (rating) are the activities of these entities in the fields of information security and cybersecurity, the security status and level of protection of these entities from the relevant security threats, as well as certain indicators of security and safety. Cybersecurity indices in the sense of high-level indicators (or cybersecurity indexes in the sense of quantitative indicators) will include regular (periodic) information materials that contain expert, analytical, statistical information on the security status and level of protection of indexing entities (ranking), as well as some indicators regarding the harmful effects of the implemented threats to information security and cybersecurity. Typically, cybersecurity indices are compiled to assess the state of information security and cybersecurity, as well as the level of protection from threats. The list of cybersecurity indices includes some other indices that are indirectly related to cybersecurity, namely:  Artificial Intelligence Index  Index of Corporate Attractiveness  Digital Economy and Society Index  Index of ICT Development  Network Readiness Index  Nuclear Safety Index (in terms of cybersecurity) Cybersecurity ratings will include cybersecurity indices, which are characterized by lists of indexing (rating) subjects with the definition of the order place of the subject in the list (rating). Cybersecurity indices are proposed to be divided by:  types (Global, International, Corporate)  categories (Reports, Expert, Network, Data sets, Financial (Exchange), Combined)  access methods or other indicators (platform, questionnaires, libraries, applications, automatic or automated, regulatory, technical, marketing) Some indices contain features of different types and categories at the same time, so the proposed classification of cybersecurity indices is conditional. Global cybersecurity indices include cybersecurity indices, which relate to the assessment (indexing, ranking) of countries on the activities of state institutions. Global cybersecurity indices include, for example, the Digital Economy and Society Index (DESI), the Global Cybersecurity Index (GCI), the National Cybersecurity Index (NCSI), the National Cyberpower Index (NCPI). The developers of global cybersecurity indices are usually global and international organizations, such as the European Commission (EC), the International Telecommunication Union (ITU), or well-known research centers, such as the Academy of Electronic Government (EGA), the Robert and Rene Belfer Center for Science and International Relations (BCH). International cybersecurity indices include cybersecurity indices that relate to sectors of the economy, corporations, and organizations in different jurisdictions. Also, the international cybersecurity indices include an integrated assessment of countries on individual indicators of cybersecurity, relating to industries (sectors) of the economy, corporations, and organizations of certain jurisdictions, including when indexing (ranking) is carried out for a limited number of countries. International cybersecurity indices include, for example, the Cyber Threat Index (CTI), the Cyber Exposure Index (CEI). International cybersecurity indices are usually developed by rating agencies or other expert organizations that specialize in information technology and cybersecurity, such as Imperva Inc. (IMP), Cyber Intelligence House (CIH). Corporate cybersecurity indices include cybersecurity indices that apply to corporations and organizations. Corporative cybersecurity indices include, for example, the BitSight Security Ratings Platform (BSSR), the Cyber Risk Index (CRI), the Cyber Attacks Timeline Master Indexes (CATMI). Corporate cybersecurity indices are typically generated by manufacturers of technology products and 118 solutions or information technology expert organizations, such as BitSight Technology Company (BST), Trend Micro Inc. (TMI), HACKMAGEDDON (HMG). The cybersecurity indices of the reports category include regular (periodic) materials, which, as a rule, do not contain statistical information and ratings of indexing subjects (ratings) and relate to issues of analysis and assessment of threats, risks, events, incidents, negative consequences, and other specialized issues in the areas of information security and cybersecurity, not directly related to the subjects of indexing (rating), evaluation of their activities and security. The cybersecurity indices of this category include, for example, Microsoft Security Intelligence Report (MSIR) from Microsoft (MSF), Command Control Cybersecurity Index (CCCI) from Command Control (Event) - regular professional conference (CCE), Cybersecurity Capacity Maturity Model Review Reports (CMMRR) from Oxford University Global Center for Cybersecurity Capabilities (GCSCC). Expert cybersecurity indices include cybersecurity indices, which are formed and/or confirmed by surveys and/or conclusions of specially involved or self-involved experts. Expert cybersecurity indices include, for example, global cybersecurity indices, such as the Global Cybersecurity Index (GCI), the National Cybersecurity Index (NCSI), the National Cyberpower Index (NCPI), the Digital Economy and Society Index (DESI), the Network Readiness Index (NRI). It should be noted that some methods of expert evaluation are used to form the most of cybersecurity indices, but at the same time for the most of indices, the methods of examination and formation of groups of experts have significant differences and features. Cybersecurity indices of the network category include cybersecurity indices, which are formed, as a rule, for individual organizations through a multilevel analysis of control information contained in data packets transmitted in packet data networks. Organizations, which usually specialize exclusively in such activities, take part in the collection, processing, and analysis of management information. Global cybersecurity indices developers include BitSight Technology (BST), SecurityScorecard (SSC), UpGuard Inc. (UGI) with, respectively, BitSight Security Ratings Platform (BSSR), SecurityScorecard Ratings Platform (SSR), UpGuard Ratings (UGR). At a local level (of an individual organization or a particular sector of the economy) in the collection, processing, and analysis of management information are involved (if available) industry centers for monitoring and responding to information security incidents (Security Operation Centers). But, as a rule, the SOCs functionality does not provide for indexing (rating) of entities that use SOC services. Cybersecurity indices of the data sets category include cybersecurity indices, which are formed, as a rule, by means of automatic visualization of data (other than the data used in the formation of cybersecurity indices of the network category). If data are directly related to financial (exchange) information, such cybersecurity indices are classified as financial (exchange). The cybersecurity indices of the data sets category include, for example, the Index of Cybersecurity (ICS) of the New York University Tandon School of Engineering (TSE), the Cyber Threat Index (CTI) of Imperva Inc. (IMP), the IBM X-Force Threat Intelligence Index (XFTII) of IBM Inc. (IBM). Cybersecurity indices in the financial (exchange) category include, for example, the VP Cyber Index (BVPCI) of Bessemer Venture Partners (BVP), the Foxberry Tematica Research Cybersecurity & Data Privacy USD PR Index (FXCI) of Foxberry Ltd (FXB), the Indxx Pure Cyber Index (IPCI) from Indxx (IND). The cybersecurity indices of the combined category include cybersecurity indices (indicators, subindexes, domains) to which several other categories can be assigned simultaneously. The cybersecurity indices of the combined category include the Cyber Risk Index (CRI) of Trend Micro Inc. (TMI) and the Local Cyber Security Index (LCSI) of the International Cybersecurity University (ICU). 2.2. Methodology of Forming Cybersecurity Indices: Ratings The subject or object of indexing (rating) is determined separately for each cybersecurity index. In a broad sense, by indexing (rating) we mean any method of evaluation. The objects of indexing (rating) include:  activity of indexing subjects (entities) in the field of information security and cybersecurity  state of information security or cybersecurity of indexing subjects (entities)  the level of protection of indexing subjects (entities) from threats (cyberthreats) 119  certain indicators of information security and cybersecurity, as well as protection against relevant threats  some indicators regarding the harmful effects of implemented threats Indicators of indexing (rating) are the certain parameters of the subject of indexing (rating), which are used to describe and evaluate the index. Different cybersecurity indexes contain from 1 to 50 indicators that belong to the subject or object of indexing (rating) and can be evaluated separately (as a sub-index) or as part of the main index. The list of indicators of indexing (rating) is determined separately for each index, which is the basis of the methodology of formation of the corresponding index or rating. Methods of indexing (rating):  examination - definition and expert assessment of indicators of indexing (rating)  parameterization - quantitative assessment of indicators of indexing (rating)  indexing - a method of evaluation, the result of which is a dimensionless numerical indicator (index, score)  rating - a method of evaluation, the result of which is a natural numerical indicator (number in the rating) The methodology of index (rating) formation is based on the methods of expert evaluations, index method of mathematical statistics or mathematical rating theory [5]-[9]. The problem of choosing and/or optimizing the methodology for determining and forming indices (ratings) will be considered separately outside the scope of this study. Methods of index (ratings) formation, as a rule, consist in the use of expert evaluation methods (Delphi method). Determining the weight of individual indicators in the composition of indexes and procedures for calculating index values has significant quantitative differences for different indices (ratings). The formation of indices (ratings), usually takes place according to the following procedure:  creation of an expert group, analysis of the subject of research or improvement of the procedure for forming the previous version of the index (rating)  definition of the objects, subjects, and objectives of indexing (rating)  data collection  development or improvement of methods of index (rating) formation  preparation and publication of the report 3. Global Cybersecurity Indices Data Table 1 Cybersecurity Indices: Key Indicators Index Name Abbr. Categories 1st Edition Qual. Country Africa ACR International 2016 4 SAR Cybersecurity (Regional), Report Reports Artificial AIIR Global, 2021 1 USA Intelligence Reports Index Report Automate A3SRP Global, 2016 ထ USA Third-Party Network Security Rating Platform Black Kite BKCRRP Global, 2016 ထ USA Cyber Risk Network Ratings Platform BitSight BSSR Global, 2011 ထ USA Security Network 120 Ratings Platform BVP Cyber BVPCI Corporate, 2011 ထ USA Index Financial (Exchange) Corporate CAI International, 2018 1 USA Attractiveness Expert Index Cyber Attacks CATMI International, 2011 10 ITA Timeline Data sets Master Indexes Command CCCI Corporate, 2018 2 DEU Control Reports Cybersecurity Index Chubb Cyber CCI Corporate, 2009 2w USA Index Data sets Cyber Exposure CEI International, 2018 9 SGP Index Corporate, Network Cyber CGI Corporate, 2016 ထ LXB Governance Data sets Indices Cyber Green CGI Global, 2017 ထ USA Index Network Cyber Hygiene CHRI Corporate 2018 2 USA Risk Index (Individuals), Expert Cybersecurity CIGBR Global, 2021 1 UKR Indices Global Reports Brief Report Cyber Maturity CMAPR International 2014 4 AUS Asia Pacific (Regional), Region Reports Cybersecurity CMMRR Global, 2015 > 99 GBR Capacity Reports Maturity Model Review Reports Cyber Norms CNI International, 2007 ထ USA Index Data sets RSA CPI International, 2015 2 USA Cybersecurity Reports Poverty Index Cyber Policy CPP Global, 2008 (USA) ထ CHE Portal Reports Cyber CRI International, 2015 2 USA Readiness Reports Index Cyber Risk CRI International, 2018 3 USA Index Data sets Cyber Risk CRI Corporate, 2018 3 USA Index Combined 121 Cyber Risk CRLEI International, 2020 1 USA Literacy & Reports Education Index MSCI ACWI IMI CSI Corporate, 2017 4 USA Cyber Security Financial Index (Exchange) Dell CSI Corporate, 2020 2 USA Cybersecurity Reports Index Cyber Security CSPI Corporate, 2016 ထ DEU Performance- Financial Index (Exchange) Cyber Threat CTI International, 2001 1m USA Index Data sets Digital DESI International, 2014 1y EU Economy and Expert, Data Society Index sets European ECSI International, 2017 n/a NLD Cyber Safe Data sets Index Evolve Cyber ECSIF Corporate, 2017 ထ CAN Security Index Financial Fund (Exchange) FICO Cyber Risk FCRS Corporate, 2001 ထ USA Score Financial (Exchange) Foxberry FXCI Corporate, 2013 6m GBR Tematica Financial Research (Exchange) Cybersecurity & Data Privacy USD PR Index Global GCARC International, 2016 2 USA Cybersecurity Reports Assurance Report Card Global GCI Global, Expert 2014 4 UN Cybersecurity Index Global Cyber GCSI International, n/a (1974 - > 99 USA Strategies Reports 1st doc.) Index Global Threat GTIR International, 2017 4 JPN Intelligence Reports Report ISE Cyber HXR Corporate, 2010 ထ USA Security Index Financial (Exchange) International ICBI Corporate, 2017 21 GBR Cyber Reports Benchmarks Index 122 Index of ICS Corporate, 2011 1m USA Cybersecurity Data sets ICT IDI Global, Expert 2010 8 UN Development Index (ITU) iEdge-FactSet IECSI International, 2017 6m SNG Global Cyber Financial Security Index (Exchange) Indxx Pure IPCI Corporate, 2015 1y USA Cyber Index Financial (Exchange) Kaspersky KCI International, 2016 n/a RUS Cybersecurity Expert Index S&P Kensho KCSI Corporate, 2013 ထ USA Cyber Security Financial Index (Exchange) S&P Kensho KSECUREP Corporate, 2013 ထ USA Future Security Financial Index (Exchange) Local Cyber LCSI Corporate, 2021 1 UKR Security Index Combined Microsoft MSIR Global, 2006 6m USA Security Reports Intelligence Report National Cyber NCPI International, 2020 1 GBR Power Index Expert National NCSI Global, Expert 2018 ထ EST CyberSecurity Index Nasdaq CTA NQCYBR International, 2015 ထ USA Cybersecurity Financial Index (Exchange) Network NRI Global, Expert 2000 3 USA Readiness (2019) Index Nuclear NSI Global, Expert 2012 2y USA Security Index Prime Cyber PCDI Corporate, 2017 3m USA Defense Index Financial (Exchange) PCS Global PCS GCS Corporate, ~ 2017 n/a USA Cyber Suite Expert Prevalent PVTM Global, 2019 ထ USAGBR Vendor Threat Network CAN Monitor ReportLinker RL CIR International, <2016 > 99 FRA CyberSecurity Reports Industry Reports RiskRecon RRCR Global, 2019 ထ USA Cybersecurity Network Ratings 123 State of Cyber SCR International, 2018 1y 50 Resilience Reports countries Solactive SGCSI Corporate, 2017 ထ DEU Global Cyber Financial Security Index (Exchange) Security SSR Global, 2017 ထ USA Scorecard Network Ratings Platform Travelers Risk TRIC Corporate, n/a 1y USA Index/ Cyber Expert UpGuard UGR Global, ~2012 ထ USA Ratings Network Platform Unisys Security USI International, 2007 n/a USA Index Expert IBM X-Force XFTII International, 2011 1y USA Threat Data sets Intelligence Index ထ - great number of issues, y - per year, m - per month, w - per week Table 2 Cybersecurity Indices Publishers (Developers) Index Publisher Index Publisher ACR Africa Immersion FXCI Foxberry Ltd Research Centre for Innovation and Training Facilities AIIR Stanford Institute for GCARC Tenable Network Human-Centered Security Artificial Intelligence A3SRP Panorays GCI International Telecommunication Union BKCRRP Black Kite Inc. GCSI Center for Strategic and International Studies BSSR BitSight Technology LTD GTIR NTT Security LTD BVPCI Bessemer Venture HXR Nasdaq Group Inc. Partners CAI Institute of Electrical ICBI Neustar, Inc. and Electronics Engineers CATMI HACKMAGEDDON ICS Tandon School of Engineering (New York University) CCCI Command Control IDI International (Event) Telecommunication Union CCI Chubb Group Holdings IECSI Singapore Exchange Inc. LTD CEI Cyber Intelligence IPCI Indxx House 124 CGI Cyberhedge EUROPE KCI Kaspersky Lab S.a.r.l CGI CyberGreen Institute KCSI S&P Dow Jones Indices LLC CHRI Wakefield Research KSECUREP S&P Dow Jones Indices LLC CIGBR International MSIR Microsoft LCSI Cybersecurity University CMAPR Australian Strategic NCPI Belfer Centre (Harvard Policy Institute University) CMMRR Global Cyber NCSI e-Governance Security Capacity Centre Academy (University of Oxford) CNI Carnegie Endowment NQCYBR Nasdaq Group Inc. for International Peace CPI EMC Corporation NRI World Economic Forum (Portulans Institute) CPP United Nations Institute NSI Nuclear Threat for Disarmament Initiative Research CRI Potomac Institute for PCDI ETF Ventures LLC Policy Studies CRI NordVPN.com & PCS GCS Verisk Analytics, Inc. Tefincom S.A. CRI Trend Micro Inc.& PVTM Prevalent, Inc. Ponemon Institute CRLEI Oliver Wyman Forum RL CIR ReportLinker.com CSI Morgan Stanley Capital RRCR RiskRecon Co. International Inc. CSI Dell Secure Works SCR Accenture (Counter Threat Unit) CSPI Vontobel Holding AG SGCSI Soloactive AG CTI Imperva Inc. SSR SecurityScorecard Co. DESI European Commission TRIC Travelers Indemnity Co. ECSI VPNoverview UGR UpGuard Inc. ECSIF Evolve ETFs USI Unisys Company FCRS FICO XFTII IBM Inc. 4. Main Results and Conclusion The development of high-level indicators to describe the state of information security and cybersecurity of individual organizations, sectors (industries) of the economy, critical (including digital) infrastructures, states, regions, and the world as a whole is gradually becoming the main task of public and global security. A nonlinear increase in the rate of formation of the global hierarchy of safety indicators can be observed. The problem of defining a system of safety indicators, even at a special level, remains unsolved and an extremely urgent task. For the first time in the frame of a single study 65 existing global, international, and corporate cybersecurity indices and approaches to their formation are described and analyzed. The definition of 125 the terms necessary for the analysis of indexing (rating) in the field of information security and cybersecurity is offered. The materials of the report could be used for making a national contribution to the global cybersecurity reports of the world and international organizations in the field of information and communication technologies, telecommunications, and cybersecurity (ITU, FIRST, GFCE). 5. Acknowledgements The research within the project "Development of a Methodology for the Formation of Cybersecurity Indices and Implementation of the Original Integrated Cybersecurity Index (national, regional, sectoral, entity level)" is organized by the International University of Cybersecurity and supported by the Administration of the State Service of Special Communication and Information Protection of Ukraine, Ministry of Internal Affairs of Ukraine, Ministry of Energy of Ukraine. The authors are grateful for supporting the research and for discussions in the field of cybersecurity with colleagues from the Office of the National Security and Defense Council of Ukraine, the State Service of Special Communications and Information Protection and the Institute of Telecommunications and Global Information Space. 6. References [1] J. Lewis, G. Neuneck, “The Cyber Index International Security Trends and Realities”, UNIDIR United Nations Institute for Disarmament Research Geneva, Switzerland, 2013, no.3, URL: https://www.files.ethz.ch/isn/165142/the-cyber-index- international-security-trends-and-realities- en-463.pdf [2] Global Cybersecurity Index 2018, v.4, ISBN 978-92-61-28201-1, International Telecommunication Union, CH-1211, Geneva, Switzerland, 2019, 90 p., URL: https://www.itu.int/en/ITU-D/Cybersecurity/Pages/global-cybersecurity-index.aspx [3] S. Gnatyuk and oth.., “Method of Cybersecurity Level Determining for the Critical Information Infrastructure of the State”, [COAPSN-2020: International Workshop on Control, Optimisation and Analytical Processing of Social Networks], 2020, URL: http://ceur-ws.org/Vol- 2616/paper28.pdf [4] R. Boyarchuk, M. Khudyntsev, O. Lebid, O. Trofymchuk, “Organizational and Technical Model of National Cybersecurity and Cyber Protection”, [CPITS’2021, Workshop on Cybersecurity Providing in Information and Telecommunication Systems CEUR Workshop Proceedings ISSN 1613-0073, January 28, 2021, NURE, Kyiv, Ukraine], URL: http://ceur-ws.org/Vol- 2746/frontmatter.pdf [5] I. Gormley, S. Frühwirth-Schnatter, “Mixtures of Experts` Models”, arXiv:1806.08200v1 [stat.ME], 21 June, 2018, Preprint, 38 p., URL: https://www.researchgate.net/profile/ Sylvia_Fruehwirth-Schnatter/publication/325922320_Mixtures_ofdEfelin -Experts-Models.pdf [6] P. Lutereau and oth., “General Criteria: Group Rating Methodology”, Standard&Poor’s Financial Services LLC, 2013, 51 p., URL: https://www.maalot.co.il/Publications/MT20180219160103.pdf [7] L. Davidson, X. Ling, M. Sargis, T. Strauts, “Morningstar Quantitative Rating for funds”, 19 March, 2018, 37 p., Morningstar, URL: https://s21.q4cdn.com/198919461/files/doc_downloads/ 2019/11/Morningstar-Quantitative-Rating-for-funds-Methodology-v14.pdf [8] A. Solodov, Mathematical principles of building rating systems, Economics, Statistics, and Informatics, 2016, №1, p.75-82 (in Russ.), URL: https://cyberleninka.ru/article/n/ matematicheskie- printsipy-postroeniya-reytingovyh-sistem [9] M. Lyndina, A. Orlov, Mathematical theory of ratings. Nauchnyy zhurnal KubGAU [Scientific Journal of KubSAU], 2015, no. 114, p. 1-26 (in Russ.). URL: http://sj.kubsau.ru/2015/10/pdf/01.pdf 126