Introduction

With the advance of technology in different sectors, cybersecurity becomes crucial to protect against attacks and ensure digital safety [1]. As the complexity and frequency of cyber attacks rise, there is a need to employ different measures for identifying and countering emerging threats. Artificial Intelligence (AI) cybersecurity solutions [2,3,4,5] have proven to be a good ally in fighting cybercrime, which, alongside eXplainable AI (XAI) techniques, have immense power in reducing and predicting cyber threats [6,7]. Recent studies discussed the benefits of integrating AI components with an appropriate Intelligent User Interface (IUI), providing principles to apply when developing intelligent threat modelling tools, especially considering users' interaction guidelines and User Experience (UX) [8,9,10,11,12]. However, one of the main challenges that may invalidate the effectiveness of cybersecurity systems lies in human errors, often caused by users' cognitive biases [13,14]. This calls for defining strategies to detect and mitigate human irrational judgements (e.g., optimism bias) [14,15].

In this paper, we collect a set of shared problems and overcome strategies arising from the converging literature in cybersecurity considering Artificial Intelligence (AI), Intelligent User Interfaces (IUI), and human cognitive biases in decision-making. We start by outlining the emerging challenges of AI applications in cybersecurity systems and the necessity for explainability resolutions. Next, we list the most encountered cognitive biases and phenomena that undermine users' decision-making and lead to potential errors, accompanied by mitigation approaches.

Related Work

In this section, we briefly summarize the emerging gaps and needs in the literature encompassing the fields of Artificial Intelligence (AI), Intelligent User Interface (IUI), and human cognitive bias in the cybersecurity domain.

Artificial Intelligence in Cybersecurity

Cybersecurity is continuously changing with the development of new technologies and the emergence of new threats [1]. The integration of Artificial Intelligence (AI) has the potential to significantly enhance cybersecurity systems by enabling them to identify and counter novel and unknown threats [16,17]. Common approaches encompass AI tactics for identifying and surveilling malicious activities, detecting cyber threats, and safeguarding an organization's networks, which may include Expert Systems, Intelligent Agents, Deep Learning, and Reinforcement Learning [3,4,5,18,19]. Some practical examples of these techniques involve AI systems for Intrusion Detection [2,20], Botnet Attack Detection [21,22,23,24], Malware detection, analysis, and mitigation [25]. However, integrating complex and black-box AI systems undermines the transparency of these systems' decision-making processes. Adopting eXplainable AI (XAI) techniques becomes a starting point for providing insights into the rationale behind AI-driven decisions and enhancing overall transparency in the cybersecurity domain [6,7,26]. In addition to adopting Explainable AI (XAI) techniques, assessing AI confidence and robustness becomes crucial to avoid unintended behaviours of AI-based cybersecurity systems. Specifically, estimating measures like uncertainty and implausibility [27,28] allows practitioners to make more informed decisions and adapt cybersecurity measures in responding to new threats.

Intelligent User Interfaces in Cybersecurity

Integrating Artificial Intelligence (AI) solutions in the cybersecurity ecosystem profoundly influences user interface (UI) design due to its ability to enhance user experiences and enable personalized interactions. This process is commonly referred to as Intelligent User Interface (IUI) design, ultimately leading to user interfaces that are user-centric, engaging, and effective in meeting user needs and expectations [8]. Previous literature already proposed potential designs of intelligent user interfaces for defending against Malicious Bot Attacks [24] and preventing Phishing Attacks [29,30]. Instead, other research explored how users make decisions when engaging with these interfaces during phishing attacks [12], showing that the interface design was understandable and familiar to users. However, they posit a need for future research to determine the most effective malicious features to display and how to enhance users' interest and trust. Additionally, further research studied how cognitive bias affects users' decisions in a phishing detection scenario [11], revealing that the higher occurrence of hyperbolic discounting bias (i.e., choose immediate rewards over rewards that come later in the future) made it more easily identifiable by humans, reducing its effectiveness in deceiving participants. Conversely, the lower occurrence of authority bias (i.e., the tendency to be more influenced by the opinions and judgments of authority figures) proved more effective in phishing human participants. We will deepen the topic of cognitive biases in the next section.

Human Behavioral Decision-Making in Cybersecurity

A major challenge in the cybersecurity domain concerns the detection and mitigation of human cognitive biases, as they can influence decision-making, leading users to think irrationally in certain situations and make unreasonable judgments [13,31]. Recent research gathered which are the most common cognitive biases based on different scenarios: for example, Gutzwiller et al. [32] found phenomena like confirmation bias, anchoring bias, and take-the-best heuristic are the most common among red teamers attackers. Furthermore, the authors of [33] investigated the role of four cognitive biases (i.e., selective perception, exposure to limited alternatives, adjustment and anchoring, and illusion of control) in anticipating and responding to Distributed-Denial-of-Service (DDoS) attacks. They highlighted several practical implications for managers in dealing with the increasing threat of cyberattacks like raising awareness, developing clear step-by-step tested and documented defense procedures, and identifying organizational vulnerabilities. Majumdar et al. [34] carried out a systematic literature review collecting human-related components (e.g., confirmation bias, availability bias, and framing effect) and risky habits (e.g., sharing passwords, accidental insider threats, and lack of perseverance) that impact cybersecurity practices, also suggesting solutions to overcome them among which: security awareness training, phishing simulations, and incident response plan. Alnifie and Kim [15] studied another relevant bias called optimism bias, which can result in an inaccurate perception of risks, leading to subjective decisions that lack objectivity. To reduce this bias, they suggest that employees regularly follow instructions from security teams, adhere to cybersecurity policies, and recognize optimism bias at both individual and organizational levels.

A novel security paradigm that emerged from recent literature [14] is referred to as cognitive security, where the authors emphasize the vulnerabilities in human cognitive processes (e.g., perception, attention, memory, and mental operations) that can be exploited by cognitive attacks, affecting performance and decision-making. The authors present several cognitive and technical defense methods to deter the kill chain (i.e., the stages of a cyberattack) of cognitive attacks, such as real-time tracking of cognitive attacks, identification of abnormal patterns in human behaviors, introducing compensation mechanisms to mitigate the impact of cognitive attacks, or reducing cognitive load during security incidents.

Another significant trend investigating human vulnerabilities involves cybersecurity games [35,36], where participants make strategic decisions in a simulated environment and tackle real-world cyber threats to enhance their practical understanding of cybersecurity. Jalali et al. [37] developed a simulation game to assess decision-makers effectiveness in addressing two challenges in cybersecurity capability development: potential delays and uncertainties in predicting cyber incidents. They found that (i) decision-makers respond poorly to time delays in dynamic settings under uncertainty, and (ii) experienced managers did not perform better than inexperienced individuals in making proactive decisions about building cybersecurity capabilities. These results call for a strong need for training tools to underscore the drawbacks of a solely cognition-focused strategy and to grasp the impacts of feedback delays.

Discussion

The review of the literature work we discussed in Section 2 shows that the integration of AI is increasingly applied to defense mechanisms against cyber threats. From an interaction point of view, solutions that team together humans and AI-based agents are particularly relevant, especially for the tasks where AI-based solutions perform better than humans [1,17]. This suggests that the synergistic teaming of humans and AI is a promising way to address cyber threats' dynamic and complex nature. However, the effectiveness of AI solutions in cybersecurity still poses several key challenges. One prominent issue is the shortage of skilled cybersecurity professionals [1,16,17]. This scarcity is a barrier even to the spread of AI-based solutions, which require professionals at least to set them up. The lack of professionals also sets a challenge to create and promote adequate educational programmes [3], which require expert human trainers again. However, building such programs is crucial for widespread awareness about cyber threats and fostering a culture of cybersecurity consciousness among organizations and individuals.

Another critical consideration is the design of computing platforms resilient to AI-based adversarial threats [19]. Rather than treating security measures as an afterthought, there is a growing recognition of embedding resilience into computing infrastructure from the outset. This proactive approach reflects a shared responsibility among stakeholders to mitigate cybersecurity risks effectively.

Transparency and interpretability are fundamental principles in deploying AI-driven cybersecurity solutions [16]. Biased data and decision-making processes pose significant challenges, as the opaque nature of AI models complicates understanding their logic and outcomes. It is often difficult for an administrator to understand the AI system logic in the event of a security breach. AI systems sometimes provide inaccurate findings in the form of false positives, which mislead security experts, jeopardizing the entire system's integrity [17]. Assimilating eXplainable AI solutions [6,7] along with factors such as AI robustness and uncertainty [27,28] is essential for maintaining trust and confidence in cybersecurity systems. Additionally, it is worth mentioning that XAI techniques can face security attacks, which emphasizes the need to carry out experimental studies of the impacts of various attacks on XAI methodologies, together with a balance between the security and usability of XAI-integrated cybersecurity systems [26].

Moreover, cognitive biases inherent in human decision-making introduce additional complexities to cybersecurity strategies [32]. The presence of biases, such as the take-the-best heuristic, confirmation bias, optimism bias, and anchoring bias [34], along with other phenomena like framing effects, sunk cost, irrational escalation, and the illusion of control [33], poses challenges for measurement. Future research should focus on inducing and assessing the exhibition of biased behaviour, moving away from over-reliance on observational assessment. Additionally, researchers should develop experimental designs and measures specifically designed to elicit particular biases.

In particular, optimism bias [15] refers to the tendency of individuals, regardless of their capacity, to perceive risks inappropriately. They often believe they are not vulnerable or exhibit overconfidence in the effectiveness of security measures: essentially, they think, "I/we won't be a target." To address this bias, researchers can explore longitudinal studies that track the development and evolution of optimism bias over time. Additionally, evaluating the effectiveness of different interventions, such as training programs, awareness campaigns, and educational initiatives, can help mitigate this bias. Furthermore, considering AI approaches may provide valuable insights into managing optimism bias.

Unique conditions significantly impact which biases are possible to study. For instance, it would be tough to investigate illusory correlation in a context lacking relevant data for correlation. Similarly, studying sunk cost [32] would be challenging if no resources were utilized. Some biases remain understudied, including the availability heuristic, default effect, and informationpooling bias. Additionally, social engineering techniques exploit cognitive biases to manipulate user behaviour [14], highlighting the importance of user training and awareness programs. Cultivating critical thinking skills and promoting a culture of cybersecurity consciousness is essential for defending against cognitive attacks and enhancing overall cybersecurity resilience.

Conclusion and Future Work

This paper summarised the shared needs and shortcomings of Artificial Intelligence (AI) solutions and human decision-making biases in cybersecurity. We discussed common points that emerge from the literature and provide potential directions against cybersecurity threats, which we outline as follows. The first significant aspect regards the importance of human-AI collaboration, urging the promotion of suitable professional programs to address the shortage of skilled cybersecurity experts. The second point highlights the necessity for transparency and explainability in cybersecurity AI solutions, revealing the necessity of planning new procedures to defend against AI-based cybersecurity attacks. Finally, the last trait calls for training programs to detect and measure cognitive biases, along with experimental settings that stimulate these biases, intending to elicit users' awareness and foster the growth of the cybersecurity culture.

Follow-up studies need to consider these aspects for a better AI-based cybersecurity systems administration, ensuring effective cyber-threat detection and mitigation by appropriately addressing human cognitive biases.