The report is devoted to analyzing and classifying cybersecurity indices developed and implemented by leading global, international, and local organizations as of early 2021. It is proposed to include regular (periodic) information materials, which contain expert, analytical, statistical information on the state of cybersecurity and the level of protection of indexing subjects (rating), as well as on certain indicators regarding the harmful impact of implemented information security and cybersecurity threats, to cybersecurity indices. 65 existing cybersecurity indices and approaches for their formation are described. The definition of the terms necessary for the analysis of indexing (rating) in the field of information security and cybersecurity is offered. Cybersecurity, information security, indices, indexes, ratings Research on cybersecurity indicators is an extremely important and urgent task in the field of global security [1-3]. In Ukraine, this task is solved in the framework of building an organizational and technical model of cybersecurity and cyber protection for the national cybersecurity system [4]. The topics of the research are the current world reports, indices, and ratings in the field of information security and cybersecurity. The aim of the study is the describing, review analysis, and classification of cybersecurity indices (ratings) with the proposition of including all different regular (periodic) information materials, expert, analytical and statistical reports, and data. Systematically for mentioned well and little-known indices (rankings, ratings) in the field of information security and cybersecurity we proposed the classification by types and categories, analyzing the main methods of forming these indices to review and analyze the current state of cybersecurity, cybersecurity indices and ratings, approaches to their formation, as well as to determine the main terms required for indexing (rating) in the field of information security and cybersecurity. The paper schematizes information about known cybersecurity indices (ratings), proposes their classification by types and categories and analyzes the main methods of forming these indices (ratings). ORCID: 0000-0002-9324-6901 (M. Khudyntsev); 0000-0003-1238-2598 (A. Davydiuk); 0000-0002-4003-8068 (O. Lebid); 0000-0003-33586274 (O. Trofymchuck); 0000-0002-4959-612X (A. Zhylin) CEUR Workshop Proceedings (CEUR-WS.org)
2022 Copyright for this paper by its authors.
Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).
The subjects of indexing (rating), depending on the type of index (rating) are countries, industries (sectors) of the economy, corporations, and organizations (entities).
The objects of indexing (rating) are the activities of these entities in the fields of information security and cybersecurity, the security status and level of protection of these entities from the relevant security threats, as well as certain indicators of security and safety.
Cybersecurity indices in the sense of high-level indicators (or cybersecurity indexes in the sense of quantitative indicators) will include regular (periodic) information materials that contain expert, analytical, statistical information on the security status and level of protection of indexing entities (ranking), as well as some indicators regarding the harmful effects of the implemented threats to information security and cybersecurity. Typically, cybersecurity indices are compiled to assess the state of information security and cybersecurity, as well as the level of protection from threats.
The list of cybersecurity indices includes some other indices that are indirectly related to cybersecurity, namely: Artificial Intelligence Index Index of Corporate Attractiveness Digital Economy and Society Index Index of ICT Development Network Readiness Index Nuclear Safety Index (in terms of cybersecurity)
Cybersecurity ratings will include cybersecurity indices, which are characterized by lists of indexing (rating) subjects with the definition of the order place of the subject in the list (rating).
Cybersecurity indices are proposed to be divided by: types (Global, International, Corporate) categories (Reports, Expert, Network, Data sets, Financial (Exchange), Combined) access methods or other indicators (platform, questionnaires, libraries, applications, automatic or automated, regulatory, technical, marketing)
Some indices contain features of different types and categories at the same time, so the proposed classification of cybersecurity indices is conditional.
Global cybersecurity indices include cybersecurity indices, which relate to the assessment (indexing, ranking) of countries on the activities of state institutions. Global cybersecurity indices include, for example, the Digital Economy and Society Index (DESI), the Global Cybersecurity Index (GCI), the National Cybersecurity Index (NCSI), the National Cyberpower Index (NCPI). The developers of global cybersecurity indices are usually global and international organizations, such as the European Commission (EC), the International Telecommunication Union (ITU), or well-known research centers, such as the Academy of Electronic Government (EGA), the Robert and Rene Belfer Center for Science and International Relations (BCH).
International cybersecurity indices include cybersecurity indices that relate to sectors of the economy, corporations, and organizations in different jurisdictions. Also, the international cybersecurity indices include an integrated assessment of countries on individual indicators of cybersecurity, relating to industries (sectors) of the economy, corporations, and organizations of certain jurisdictions, including when indexing (ranking) is carried out for a limited number of countries. International cybersecurity indices include, for example, the Cyber Threat Index (CTI), the Cyber Exposure Index (CEI). International cybersecurity indices are usually developed by rating agencies or other expert organizations that specialize in information technology and cybersecurity, such as Imperva Inc. (IMP), Cyber Intelligence House (CIH).
Corporate cybersecurity indices include cybersecurity indices that apply to corporations and organizations. Corporative cybersecurity indices include, for example, the BitSight Security Ratings Platform (BSSR), the Cyber Risk Index (CRI), the Cyber Attacks Timeline Master Indexes (CATMI). Corporate cybersecurity indices are typically generated by manufacturers of technology products and solutions or information technology expert organizations, such as BitSight Technology Company (BST), Trend Micro Inc. (TMI), HACKMAGEDDON (HMG).
The cybersecurity indices of the reports category include regular (periodic) materials, which, as a rule, do not contain statistical information and ratings of indexing subjects (ratings) and relate to issues of analysis and assessment of threats, risks, events, incidents, negative consequences, and other specialized issues in the areas of information security and cybersecurity, not directly related to the subjects of indexing (rating), evaluation of their activities and security. The cybersecurity indices of this category include, for example, Microsoft Security Intelligence Report (MSIR) from Microsoft (MSF), Command Control Cybersecurity Index (CCCI) from Command Control (Event) - regular professional conference (CCE), Cybersecurity Capacity Maturity Model Review Reports (CMMRR) from Oxford University Global Center for Cybersecurity Capabilities (GCSCC).
Expert cybersecurity indices include cybersecurity indices, which are formed and/or confirmed by surveys and/or conclusions of specially involved or self-involved experts. Expert cybersecurity indices include, for example, global cybersecurity indices, such as the Global Cybersecurity Index (GCI), the National Cybersecurity Index (NCSI), the National Cyberpower Index (NCPI), the Digital Economy and Society Index (DESI), the Network Readiness Index (NRI). It should be noted that some methods of expert evaluation are used to form the most of cybersecurity indices, but at the same time for the most of indices, the methods of examination and formation of groups of experts have significant differences and features.
Cybersecurity indices of the network category include cybersecurity indices, which are formed, as a rule, for individual organizations through a multilevel analysis of control information contained in data packets transmitted in packet data networks. Organizations, which usually specialize exclusively in such activities, take part in the collection, processing, and analysis of management information. Global cybersecurity indices developers include BitSight Technology (BST), SecurityScorecard (SSC), UpGuard Inc. (UGI) with, respectively, BitSight Security Ratings Platform (BSSR), SecurityScorecard Ratings Platform (SSR), UpGuard Ratings (UGR). At a local level (of an individual organization or a particular sector of the economy) in the collection, processing, and analysis of management information are involved (if available) industry centers for monitoring and responding to information security incidents (Security Operation Centers). But, as a rule, the SOCs functionality does not provide for indexing (rating) of entities that use SOC services.
Cybersecurity indices of the data sets category include cybersecurity indices, which are formed, as a rule, by means of automatic visualization of data (other than the data used in the formation of cybersecurity indices of the network category). If data are directly related to financial (exchange) information, such cybersecurity indices are classified as financial (exchange). The cybersecurity indices of the data sets category include, for example, the Index of Cybersecurity (ICS) of the New York University Tandon School of Engineering (TSE), the Cyber Threat Index (CTI) of Imperva Inc. (IMP), the IBM X-Force Threat Intelligence Index (XFTII) of IBM Inc. (IBM). Cybersecurity indices in the financial (exchange) category include, for example, the VP Cyber Index (BVPCI) of Bessemer Venture Partners (BVP), the Foxberry Tematica Research Cybersecurity & Data Privacy USD PR Index (FXCI) of Foxberry Ltd (FXB), the Indxx Pure Cyber Index (IPCI) from Indxx (IND).
The cybersecurity indices of the combined category include cybersecurity indices (indicators, subindexes, domains) to which several other categories can be assigned simultaneously. The cybersecurity indices of the combined category include the Cyber Risk Index (CRI) of Trend Micro Inc. (TMI) and the Local Cyber Security Index (LCSI) of the International Cybersecurity University (ICU). 2.2.
Methodology of Forming Cybersecurity Indices: Ratings
The subject or object of indexing (rating) is determined separately for each cybersecurity index. In a broad sense, by indexing (rating) we mean any method of evaluation.
The objects of indexing (rating) include: activity of indexing subjects (entities) in the field of information security and cybersecurity state of information security or cybersecurity of indexing subjects (entities) the level of protection of indexing subjects (entities) from threats (cyberthreats) certain indicators of information security and cybersecurity, as well as protection against relevant threats some indicators regarding the harmful effects of implemented threats
Indicators of indexing (rating) are the certain parameters of the subject of indexing (rating), which are used to describe and evaluate the index. Different cybersecurity indexes contain from 1 to 50 indicators that belong to the subject or object of indexing (rating) and can be evaluated separately (as a sub-index) or as part of the main index. The list of indicators of indexing (rating) is determined separately for each index, which is the basis of the methodology of formation of the corresponding index or rating.
Methods of indexing (rating): examination - definition and expert assessment of indicators of indexing (rating) parameterization - quantitative assessment of indicators of indexing (rating) indexing - a method of evaluation, the result of which is a dimensionless numerical indicator (index, score) rating - a method of evaluation, the result of which is a natural numerical indicator (number in the rating)
The methodology of index (rating) formation is based on the methods of expert evaluations, index
method of mathematical statistics or mathematical rating theory [
Methods of index (ratings) formation, as a rule, consist in the use of expert evaluation methods (Delphi method). Determining the weight of individual indicators in the composition of indexes and procedures for calculating index values has significant quantitative differences for different indices (ratings).
The formation of indices (ratings), usually takes place according to the following procedure: creation of an expert group, analysis of the subject of research or improvement of the procedure for forming the previous version of the index (rating) definition of the objects, subjects, and objectives of indexing (rating) data collection development or improvement of methods of index (rating) formation preparation and publication of the report 3. Global Cybersecurity Indices Data 1st Edition 2016 2021 2016 2016 2011
Qual.
4 1 ထ ထ ထ
Country
SAR USA USA USA USA
CAI CATMI CCCI CCI CEI CGI CGI CHRI CIGBR CMAPR CMMRR
CNI CPI CPP CRI CRI CRI
Corporate, Financial (Exchange) International,
Expert International,
Data sets Corporate,
Reports Corporate,
Data sets International,
Corporate, Network Corporate, Data sets Global, Network
Corporate (Individuals),
Expert Global,
Reports International (Regional),
Reports Global,
Reports International,
Data sets International,
Reports Global,
Reports International,
Reports International,
Data sets Corporate, Combined 2008 (USA) 1 10 2 2w 9 ထ ထ 2 1 4 ထ 2 ထ 2 3 3 USA ITA DEU USA SGP LXB USA USA UKR AUS GBR USA USA CHE USA USA USA
CSI CSI CSPI CTI DESI ECSI ECSIF FCRS
FXCI GCARC
GCI GCSI GTIR HXR ICBI Corporate, Financial (Exchange) Corporate,
Reports Corporate, Financial (Exchange) International,
Data sets International, Expert, Data
sets International,
Data sets Corporate, Financial (Exchange) Corporate, Financial (Exchange) Corporate, Financial (Exchange) International,
Reports Global, Expert International,
Reports International,
Reports Corporate, Financial (Exchange) Corporate,
Reports 1m 1y n/a ထ ထ 6m 2 4 4 ထ 21 USA USA DEU USA EU NLD CAN USA GBR USA UN USA JPN USA GBR
Kaspersky Cybersecurity
Index
S&P Kensho Cyber Security
Index
S&P Kensho Future Security
Index
Local Cyber Security Index
Microsoft
Security Intelligence
Report National Cyber
Power Index
National CyberSecurity
Index Nasdaq CTA Cybersecurity
Index Network Readiness
Index
Nuclear Security Index Prime Cyber Defense Index IDI IECSI IPCI KCI KCSI LCSI MSIR NCPI NCSI NRI NSI
PCDI NQCYBR PCS GCS PVTM RL CIR RRCR
Corporate,
Data sets Global, Expert International,
Financial (Exchange) Corporate, Financial (Exchange) International,
Expert Corporate, Financial (Exchange) Corporate, Financial (Exchange) Corporate, Combined
Global,
Reports International,
Expert Global, Expert International,
Financial (Exchange) Global, Expert Global, Expert
Corporate, Financial (Exchange)
Corporate, Expert Global,
Network International,
Reports 2000 (2019) 2012 2017 ~ 2017 2019 <2016 2019 1m 8 6m 1y n/a ထ ထ 1 6m 1 ထ ထ 3 2y 3m n/a ထ ထ UN SNG USA RUS USA USA UKR USA GBR EST USA USA USA USA USA FRA
USA USAGBR
CAN ထ - great number of issues, y - per year, m - per month, w - per week SGCSI SSR TRIC UGR USI XFTII
International,
Reports Corporate, Financial (Exchange)
Global, Network Corporate,
Expert Global,
Network International,
Expert International,
Data sets BKCRRP
BSSR BVPCI
CAI CATMI CCCI CCI CEI
Black Kite Inc.
BitSight Technology LTD
Bessemer Venture
Partners Institute of Electrical and Electronics
Engineers HACKMAGEDDON Command Control
(Event) Chubb Group Holdings
Inc.
Cyber Intelligence
House Index
FXCI GCARC
GCI GCSI GTIR HXR ICBI ICS IDI IECSI IPCI USA USA USA USA
USA
Publisher
Foxberry Ltd Tenable Network
Security
International Telecommunication
Union Center for Strategic and International
Studies NTT Security LTD Nasdaq Group Inc.
Neustar, Inc.
Tandon School of Engineering (New York University)
International Telecommunication
Union Singapore Exchange
LTD Indxx CGI CHRI CIGBR
LCSI CMAPR CMMRR
CNI CPI CPP CRI CRI CRI CRLEI
CSI CSI CSPI CTI DESI ECSI ECSIF FCRS
Cyberhedge EUROPE
S.a.r.l CyberGreen Institute Wakefield Research
International Cybersecurity
University Australian Strategic
Policy Institute
Global Cyber Security Capacity Centre (University of Oxford)
Carnegie Endowment for International Peace
EMC Corporation United Nations Institute for Disarmament
Research Potomac Institute for
Policy Studies NordVPN.com &
Tefincom S.A.
Trend Micro Inc.&
Ponemon Institute Oliver Wyman Forum Morgan Stanley Capital
International Inc.
Dell Secure Works (Counter Threat Unit) Vontobel Holding AG
Imperva Inc.
European Commission
VPNoverview Evolve ETFs
FICO
KSECUREP KCSI MSIR NCPI
NCSI NQCYBR
NRI NSI
PCDI PCS GCS PVTM RL CIR RRCR SCR SGCSI SSR TRIC UGR USI XFTII S&P Dow Jones
Indices LLC S&P Dow Jones
Indices LLC
Microsoft Belfer Centre (Harvard
University) e-Governance
Academy Nasdaq Group Inc.
World Economic Forum (Portulans
Institute) Nuclear Threat
Initiative
ETF Ventures LLC Verisk Analytics, Inc.
Prevalent, Inc.
ReportLinker.com
RiskRecon Co.
Accenture
Soloactive AG SecurityScorecard Co.
Travelers Indemnity
Co.
UpGuard Inc.
Unisys Company
IBM Inc. 4. Main Results and Conclusion
The development of high-level indicators to describe the state of information security and cybersecurity of individual organizations, sectors (industries) of the economy, critical (including digital) infrastructures, states, regions, and the world as a whole is gradually becoming the main task of public and global security. A nonlinear increase in the rate of formation of the global hierarchy of safety indicators can be observed. The problem of defining a system of safety indicators, even at a special level, remains unsolved and an extremely urgent task.
For the first time in the frame of a single study 65 existing global, international, and corporate cybersecurity indices and approaches to their formation are described and analyzed. The definition of the terms necessary for the analysis of indexing (rating) in the field of information security and cybersecurity is offered.
The materials of the report could be used for making a national contribution to the global cybersecurity reports of the world and international organizations in the field of information and communication technologies, telecommunications, and cybersecurity (ITU, FIRST, GFCE). 5. Acknowledgements
The research within the project "Development of a Methodology for the Formation of Cybersecurity Indices and Implementation of the Original Integrated Cybersecurity Index (national, regional, sectoral, entity level)" is organized by the International University of Cybersecurity and supported by the Administration of the State Service of Special Communication and Information Protection of Ukraine, Ministry of Internal Affairs of Ukraine, Ministry of Energy of Ukraine.
The authors are grateful for supporting the research and for discussions in the field of cybersecurity with colleagues from the Office of the National Security and Defense Council of Ukraine, the State Service of Special Communications and Information Protection and the Institute of Telecommunications and Global Information Space.