Advancing Human Performance in Cybersecurity, ADVANCES Ginta Majore1,*,† , Linas Bukauskas2,† , Stefan Sütterlin3,† and Agṅe Brilingaiṫe2,‡ 1 Vidzeme University of Applied Sciences, Tērbatas str. 10, Valmiera, 4201, Latvia 2 Institute of Computer Science, Vilnius University, Didlaukio str. 47, Vilnius, 08303, Lithuania 3 Østfold University College, B.R.A. Veien 4, Halden 1757, Norway Abstract Cybersecurity as a domain is essential for all complex digitalized environments within the public and private sectors. It incorporates technical requirements, workforce skills, and human aspects for system development, deployment, and support for business continuity. The technological advancement of cyber attacks and social engineering solutions of the adversary raise the demand for a competent cybersecurity workforce. The upskilling process has to go hand in hand with abilities to work in complex environments and even under crises. Project Advancing Human Performance in Cybersecurity (ADVANCES) contributes to research regarding the role of human factors and limitations in cybersecurity. Domain-specific engineering enables the development of a comprehensive framework as an ecosystem for future workforce development. The three Baltic countries, Lithuania, Latvia, and Estonia, and their partners from Norway and Liechtenstein investigate human behavior in cybersecurity by combining research areas of computer science, psychology, and human genomics. The project aims to develop a comprehensive, science- based interdisciplinary framework to develop and assess generic and subject-related competences of the current and future cybersecurity workforce. The team is developing an environment that supports testing behavioral patterns, attitudes toward cyber hygiene, and specific technical skills. Educational components integrating behavior change are tested in the student environment, while multidisciplinary research requires inviting participants using public announcements. Statistical and data mining tools are used to interpret multilayered data and to find correlations among genetic, behavioral, and technical skills. Keywords Domain-Specific Engineering, Multidisciplinary Approach, Cybersecurity Training, Human Performance RPE@CAiSE’23: Research Projects Exhibition at the International Conference on Advanced Information Systems Engi- neering, June 12–16, 2023, Zaragoza, Spain * Corresponding author. Representative of the ADVANCES team and presenter at the CAiSE’23 event. † These authors contributed equally. ‡ The ADVANCES project leader contributed to the work equally. $ ginta.majore@va.lv (G. Majore); linas.bukauskas@mif.vu.lt (L. Bukauskas); stefan.sutterlin@hiof.no (S. Sütterlin); agne.brilingaite@mif.vu.lt (A. Brilingaiṫe)  0000-0002-9514-7229 (G. Majore); 0000-0002-9781-9690 (L. Bukauskas); 0000-0002-4337-1296 (S. Sütterlin); 0000-0001-9768-4258 (A. Brilingaiṫe) © 2023 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). CEUR Workshop Proceedings http://ceur-ws.org ISSN 1613-0073 CEUR Workshop Proceedings (CEUR-WS.org) The Advancing Human Performance in Cybersecurity, ADVANCES, benefits from nearly €1 million grant from Iceland, Liechtenstein and Norway through the EEA Grants under the Baltic Research Programme. The aim of the programme is to consolidate research potential of Baltic States, Iceland, Liechtenstein and Norway, strengthen regional cooperation in research relevant to the countries of the region, and fill the gap between the national research funding and the European Union Structural Assistance. The aim of the project is to advance the performance of cybersecurity specialists by personalizing the competence development path and risk assessment. Project contract with the Research Council of Lithuania (LMTLT) No is S-BMT-21-6 (LT08-2-LMT-K-01-051). 1. Project Information The project Advancing Human Performance in Cybersecurity (in short ADVANCES) [1] started on January 1𝑠𝑡 , 2021, and it will end on December 31𝑠𝑡 , 2023. Eight partners from five countries implement the project. Lithuania, Latvia, and Estonia are beneficiary countries, while higher education institutions from Norway and Liechtenstein represent donor states: 1. Vilnius University (Institute of Computer Science and Institute of Biomedical Sciences), Lithuania — project promoter 2. The General Jonas Žemaitis Military Academy of Lithuania, Lithuania 3. Riga Technical University (Institute of Information Technology), Latvia 4. Vidzeme University of Applied Sciences (Socio-Technical Systems Engineering Institute), Latvia 5. Tallinn University of Technology (School of Information Technology), Estonia 6. Norwegian University for Technology and Science (Department of Information Security and Communication Technology), Norway 7. Østfold University College (Faculty of Health, Welfare and Organisation), Norway 8. University of Liechtenstein (Institute of Information Systems), Liechtenstein The main project objective is to develop the domain-specific infrastructure that enables advancing the performance of the cybersecurity specialist considering possible improvements from three different perspectives: by regarding the human as a biological entity, by analyzing the behavioral patterns of the person, and by addressing the necessary technical knowledge and skills of the cybersecurity specialist. A team of more than 25 IT and cybersecurity specialists, educators, psychologists, and human geneticists joined to apply the multidisciplinary approach when searching for a solution to the project-defined problem. The project relies on the assumption that it is possible to map cyber competences required to investigate digital crime, defend infrastructure, or be resilient to cyber abuse and afterward to develop a rational competence improvement path for a CS specialist. When dealing with critical infrastructures or handling life mission-critical support systems, tools that assess human traits or inherent risks are nonexistent, or research components must be validated scientifically. The project’s expected outcomes consist of methodologies and tools, including specific software components to gather and analyze data, self-report tools to collect factual data on socio- behavioral patterns, risk assessment methods based on cooperative interdisciplinary data, and recommendations to ensure a personalized skill development path. The designed comprehensive framework for developing and assessing generic and subject-specific competences would serve as a tool for the international research and professional community to understand human capabilities and challenges regarding the phases of the cyber-kill-chain and to build the future cybersecurity workforce. The envisioned research results include a) identification of key performance indicators in individual/team level training/exercises to develop an evidence base for a comprehensive assessment of cyber competences, b) analysis and development of methods to assess and predict the performance of a human in individual tasks and collaborative decision-making environments in cyberspace, c) development of research-proven specific tools to advance the performance of a human in learning to cope with challenges during stressful situations that require technological knowledge, and d) prototype implementation and testing to illustrate the developed framework’s applicability to support a complex ecosystem for future workforce development. 2. State of the Art The project’s ambitious goal to develop a multi-discipline-based infrastructure requires ensuring the engineering processes of gathering the requirements, performing analysis of the domain from different perspectives, designing the architecture components, and implementing the prototype of Technology Readiness Level 3–4 to build a characteristic proof of the concept with possible validation in the lab setting. The project already has nine associated papers published or accepted for publication [2, 3, 4, 5, 6, 7, 8, 9, 10] in international journals and scientific conferences. 2.1. Project Results Most cyber incidents occur to human error. Therefore, risk assessment strategies should consider digital assets and challenges that lead to risks due to individual human characteristics under cer- tain conditions, e.g., in stressful situations during crises. The initial project’s paper [2] presented a theoretical ontology-based model as a basis for a human trait semantic network. The built proof-of-the-concept prototype combined artificial intelligence algorithms and psychological questionnaires to demonstrate existing human trait links to cyber hygiene. Another paper [3] presents a holistic architecture to assess human traits and explains the links between the natural human and digital-self using the impulsivity trait example. Also, we deconstructed the stress factor understandable in an everyday setting of the cybersecurity specialist to emphasize the need for personalized training to build resilience against stress as genetics influences reaction to the environment’s triggers [4]. Therefore, in a project, the competence model of the trainee (see Figure 1) considers the trainee’s performance (behavior and results) under certain conditions Context Conditions Profiling impacts support impact measured by Personal Indicators & Scenario Result Characteristic Reflection includes measured by impacts measured by performs carried out Trainee demonstrates Role Task Behaviour Performance encapsulates requires demonstrates Skills, Attitude Competence Indicators includes & Knowledge measured by Figure 1: General competence model [5] with an impact of personal characteristics during a particular scenario that requires one to play a professional role and apply related competences [5]. The ADVANCES intervention mapping methodology [5, 6] supports the designed competence model. The methodology consists of three building blocks—competence model, course design process, and training environment [6]. The multidimensional approach that combines soft and hard skills, behavior, and cognitive aspects requires redesigning training methods and scenarios to involve trainees and stimulate their interest in cybersecurity [7, 8, 10]. For example, we demonstrated that a penetration testing course for military cadets with no prior technical skills could increase their interest in a cybersecurity career [7] if it was designed using the experiential learning paradigm, thus, making an additional professional development path. The developed CyberEscape approach [8] is based on the hybrid training environment with physical elements and virtual infrastructure to simulate the Computer Security Incident Response Team (CSIRT) tasks. The execution results showed the approach’s value in increasing self-efficacy and engagement, stimulating critical thinking, and fostering collaboration and communication skills. The project research scope involves an educational environment and professional training, i.e., cyber defense exercises. Thus, the ontology was developed to overcome the knowledge management gap [9]. Finally, we introduce the multidimensional approach for a cyber defense exercise based on the event cycle, stakeholders’ goals, and necessary social, emotional, and cognitive aspects [10]. The approach ensures psychological safety, motivation, and other event ingredients to achieve training goals. 2.2. Intermediate Results Figure 2 shows the overall architecture of the project system. The user at the focus is a cyber- security specialist willing to understand the personal strongest side and learn about possible future risks. In identifying recommendations, field professionals are involved: psychologists for behavioral analysis, health medical professionals for health risk assessment, and cybersecurity professionals for cybersecurity, engineering, and IT competence indication and assessment. Recommendations Psychologist Health Cybersecurity expert expert 1. Risk assessment, behavioral evaluation Behavioral Health Gamified 2. Health risks and overall situation questions questions CTF 3. Subject-specific competence indicators User Figure 2: The general architecture of the ADVANCES ecosystem The cybersecurity specialist has to go through all three steps of scenarios in order to get some feedback recommendations and possible risks assessed. For the project team in order to be able to observe, experiment, and gather medical data of a subject, Vilnius University received approval from the Bioethics committee to perform a multidisciplinary experiment, including gathering and analyzing genetic data (No. 2022/4- 1417-895, 12/04/2022). All ethical principles are assured, and written consent is received as voluntarily expressed declarations. All participants can leave and stop interviews at any time. Gathered data are managed according to the data management plan approved by the Research Council of Lithuania. 3. Advanced Information Systems Engineering The team has built an advanced information system as a prototype to demonstrate a proof of concept of the complex system that relies on a multidisciplinary approach for the educational process of cybersecurity specialists. The prototype corresponds to Technology Readiness Level 4. The prototype as a domain-specific system contains several components. The main three components are the questionnaire subsystem of self-assessment tools, the health data gathering module that maps health and genome data, and an interactive exercise platform to check cybersecurity skills. Due to the sensitive genetic data, anonymization is ensured in the infrastructure. The system has a link between genetic data and other parts via specific identifiers, and additional precautions are taken to guarantee data privacy. All the digitally sequenced genetic data stays on the limited-access network. Thus, only predefined views and aggregates are imported into analytical modules. Immediate experiment results are delivered to the participant regarding behavior, health, and cybersecurity skill assessment to ensure participant attention and satisfaction. This requirement arose due to the experimentation location and game duration. The complex system is a web-based solution involving several air-gaped components switched on or included on a need basis. One of the components is a Capture The Flag (CTF) module reflecting typical cybersecurity training platforms and providing a platform for cyber skill assessment. CTF is a gamified, hint-based three-level knowledge and skill testing platform with dynamically opened branches of problems to be solved according to the participant’s level. The platform keeps track of user parameters such as the number of guesses, number of hints, time used to submit an answer, and the time taken to complete overall CTF. The control group solves cyber hygiene questions but also can try more complex challenges of the target group. The CTF implementation and execution reflect the competence model design presented in Figure 1. Another component includes several psychological questionnaires for self-assessment. The self-assessment tools applied in this project are scientifically validated and have been constructed under considerations of behavioral science standards, ensuring quality criteria such as construct validity, predictive validity, reliability, and objectivity. Both personality traits, cognitive styles, and the capacity for cognitive, emotional, and behavioral regulation have been assessed. The theoretical base of this selection of assessment tools draws from research on the impact of personality, situational context, and work stress on decision-making under pressure. Inspired by behavioral-cognitive models such as the critical path mode, our methodological approach leading to the self-assessment toolbox incorporates several risk factors that are potential markers for human failure, errors and other factors beyond momentarily apparent behavior. This also includes risk factors for lacking personal integrity and deviant behavior (e.g., insider threats), individual vulnerabilities to social engineering-based attack vectors, and performance-related traits such as conscientiousness. The established self-assessment toolbox assesses information that is predictive for decision- making, risk assessment/taking within IT networks, the choice of strategies within cyber operations and how to handle ambiguous threats under pressure. To capture the whole picture and consider changing situational contexts, we go beyond pure personality factors and tap into behavioral patterns resembling the interaction of personality and environmental factors. With this approach of quantifying individual traits that are stable across situations and over time, we do not only measure momentary risk factors or potentials, but actually also a proxy for the individual’s overall susceptibility. In sum, this comprehensive approach addresses the behavioral science part of the project aims and identifies statistically valid and relevant predictors of cognitive performance and behavioral control in cyber operations. While building on previous research of project partners on communication and decision- making in socio-technical systems and cyber operations in particular, we use essential predictors for human performance that find their neural substrates in prefrontal cortical functions respon- sible for executive planning and execution and the control of emotional impulses and cognitive regulation. Currently, the project team is in the experimentation phase with research participants. Pre- liminary correlation results of behavior, health, and cybersecurity skills with recommendations are expected at the end of June 2023. The project team expects to share with the research community results as scientific publications. References [1] EEA Grants, Advancing human performance in cybersecurity, 2021. URL: https://www.eeagrants.lt/en/programmes/projects/program/26/id/92/advancing_ human_performance_in_cybersecurity. [2] A. Jurevičieṅe, A. Brilingaiṫe, L. Bukauskas, Digital human in cybersecurity risk assessment, in: Augmented Cognition - 15th International Conference, AC, Held as Part of the 23rd HCI International Conference, HCII, Proceedings, volume 12776 of Lecture Notes in Computer Science, Springer, 2021, pp. 418–432. doi:10.1007/978-3-030-78114-9_29. [3] L. Ambrozaityṫe, A. Brilingaiṫe, L. Bukauskas, I. Domarkieṅe, T. Rančelis, Human char- acteristics and genomic factors as behavioural aspects for cybersecurity, in: Augmented Cognition - 15th International Conference, AC, Held as Part of the 23rd HCI Interna- tional Conference, HCII, Proceedings, volume 12776 of Lecture Notes in Computer Science, Springer, 2021, pp. 333–350. doi:10.1007/978-3-030-78114-9_23. [4] I. Domarkieṅe, L. Ambrozaityṫe, L. Bukauskas, T. Rančelis, S. Sütterlin, B. J. Knox, K. Maennel, O. Maennel, K. Parish, R. G. Lugo, A. Brilingaiṫe, Cybergenomics: Ap- plication of behavioral genetics in cybersecurity, Behavioral Sciences 11 (2021) p. 15. doi:10.3390/bs11110152. [5] R. Pirta-Dreimane, A. Brilingaiṫe, G. Majore, B. J. Knox, K. Lapin, K. Parish, S. Sütterlin, R. G. Lugo, Application of intervention mapping in cybersecurity education design, Frontiers in Education 7 (2022) p. 12. doi:10.3389/feduc.2022.998335. [6] R. Pirta-Dreimane, A. Brilingaiṫe, E. Roponena, K. Parish, Multi-dimensional cybersecurity education design: A case study, in: IEEE Intl. Conf. on Dependable, Autonomic and Secure Computing, Intl. Conf. on Pervasive Intelligence and Computing, Intl. Conf. on Cloud and Big Data Computing, Intl. Conf. on Cyber Science and Technology Congress, DASC/PiCom/CBDCom/CyberSciTech, IEEE, 2022, pp. 1–8. doi:10.1109/DASC/PiCom/ CBDCom/Cy55231.2022.9927931. [7] A. Melnikovas, R. G. Lugo, K. Maennel, A. Brilingaiṫe, S. Sütterlin, A. Juozapavičius, Teaching pentesting to social sciences students using experiential learning techniques to improve attitudes towards possible cybersecurity careers, in: Proc. of the 22nd European Conference on Cyber Warfare and Security, 2023, p. 10. To appear. [8] R. Pirta-Dreimane, A. Brilingaiṫe, E. Roponena, K. Parish, J. Grabis, R. G. Lugo, M. Bonders, CyberEscape approach to advancing hard and soft skills in cybersecurity education, in: Proc. of the 25th HCI International Conference, July 2023 (LNCS series), Springer, 2023, p. 19. URL: https://hdl.handle.net/11250/3051549, To appear. URL is provided to the accepted version. [9] G. Babayeva, K. Maennel, O. M. Maennel, Building an ontology for cyber defence exercises, in: IEEE European Symposium on Security and Privacy, EuroS&P, IEEE, 2022, pp. 423–432. doi:10.1109/EuroSPW55150.2022.00050. [10] K. Maennel, A. Brilingaiṫe, L. Bukauskas, A. Juozapavičius, B. J. Knox, R. G. Lugo, O. Maen- nel, G. Majore, S. Sütterlin, A multidimensional cyber defense exercise: Emphasis on emotional, social, and cognitive aspects, SAGE Open 13 (2023) p. 12. doi:10.1177/ 21582440231156367.