Supporting the Design of Phishing Education, Training and Awareness interventions: an LLM-based approach

Abstract

Phishing remains one of the most effective cyber threats in our digital world, affecting millions of organizations. Phishing education, training, and awareness programs are used to address employees' lack of knowledge about phishing attacks. However, despite being very expensive, these interventions are not always effective, mainly due to the lack of customization of training materials based on the employees' needs and profiles. In fact, creating customized training content for each employee and each context would require a huge effort from security practitioners and educators thus increasing costs even more. The proposal we present in this paper is to use Large Language Models to automate some steps in the design process of training content, which is tailored to the specific user profile.

Keywords

• eol>phishing education
• large language models
• warnings
• training
• simulated campaigns 1

1. Introduction

Phishing is currently one of the most significant cyber-threats, causing substantial losses for companies each year on a global scale [25]. Despite the technological solutions that exist to mitigate phishing attacks [ 2, 29 ], criminals are able to succeed due to the exploitation of vulnerabilities that originate from various human factors [16]. Among the primary human factors that can increase a user’s susceptibility to phishing, Lack of Knowledge and Lack of Resources are of particular importance. The former refers to users missing specific knowledge and experience to correctly deal with phishing attacks [18], while the latter refers to the lack of educative resources that can effectively help users recognize phishing attacks [16]. Despite the users being often considered the “weakest link” in the cybersecurity of an organization [ 41 ], improving their awareness level can lead to making the employees one of the most valuable defensive assets of an organization [ 36 ].

Consequently, companies invest considerable resources [ 7 ] in increasing employee awareness and educating them with Phishing Education, Training, and Awareness (PETA) programs [ 40 ]. However, despite the general agreement among researchers on the usefulness of anti-phishing training [26], its effectiveness can vary considerably [ 36 ]. This may be attributed to human factors such as age, gender, technical expertise, and personal traits [26]. To address the ineffectiveness of phishing training and education due to the employees’ individual differences, a viable solution would be to provide them with customized training material [ 17, 31, 50 ]. Furthermore, training material should be highly engaging and interesting for employees [17, 50], while also being easy and fast to consume. This is because users generally can dedicate little time to security aspects during their work hours [ 35 ]. Improving the relevance and quality of training material can indeed result in more effective phishing training programs [ 9 ]. This is likely to have the additional benefit of reducing the likelihood of users circumventing the educational process (e.g., by ignoring the training material altogether, or attempting to cheat in assessment quizzes).

However, the design and implementation of PETA material is not a trivial process and requires significant effort and human resources [ 3 ]. Typically, simulated phishing campaigns are employed to administer embedded training material [ 9, 33, 36, 51 ]. Although training campaigns are one of the most commonly used approaches, they are very expensive to conduct, and may even result in an increase in phishing susceptibility of some employees [ 36 ] or in a reduction in their click rate on legitimate links, potentially affecting their productivity [ 42 ]. In light of these problems, we deem it necessary to find a solution to address the lack of effective and affordable PETA resources.

This study presents ongoing research aimed at supporting the lightweight creation of effective PETA resources. The effectiveness of the resources will be achieved by tailoring each resource to the specific user, considering their profile created using ad hoc vulnerability assessment techniques (e.g. questionnaire). In this way, each user will be exposed to short, targeted, and relevant resources that they are more likely to accept than the traditional long and generic alternatives used today. The creation of these resources will also be facilitated by the use of LLMs, which, taking into account the user profile and the type of resource to which the user should be exposed (e.g., podcast, document, alert, etc.), will produce PETA resources tailored to the users, covering only their weakness, without exposing the user to aspects in which they are already confident.

This work establishes a first basis for a significant contribution to the broader Italian national project DAMOCLES (Detection And Mitigation Of Cyber attacks that expLoit human vulnerabilitiES), which aims to develop a framework for the Italian Public Administration to assess human factors in cyber incidents and mitigate their impact through security awareness and customized user training. This latter point can indeed be addressed by using technologies like LLMs to support the creation of training material tailored to the individual weaknesses assessed.

2. Related work

2.1. Phishing Education, Training, and Awareness (PETA)

The term “PETA” (Phishing Education, Training, and Awareness) was recently brought up by Sarker et al. [ 40 ] to refer to Security Education, Training, and Awareness (SETA) interventions [23] in the domain of phishing. Katsikas et al. [28] define awareness, training, and education as distinct concepts that together accomplish learning, starting with awareness and culminating in education. However, these terms are often used interchangeably in the literature [23]. Therefore, PETA is a broad concept that encompasses any type of intervention designed to enhance users’ awareness and skills, including formal learning (courses, seminars, etc.), simulated phishing campaigns, quizzes, serious games, and anti-phishing warnings.

Simulated phishing campaigns aim to mimic real-world phishing attacks. They are typically part of a dedicated security awareness training program. These campaigns generate simulated phishing emails that closely resemble actual phishing attempts. Employees receive these emails to test their vigilance and response. Generally, embedded training is employed in conjunction with simulated phishing campaigns to present employees with landing pages that include educational material immediately after they click on a fake phishing link.

Anti-phishing warnings can constitute a valid intervention for increasing phishing awareness of users. These tools are employed to alert users about potential threats by blocking access to malicious websites [ 1 ] or emails [ 8 ] and are typically found in browsers (e.g., Google Chrome) or in email clients (e.g., Gmail).

2.2. Limitations of current PETA interventions

A review of the literature conducted by Sarker et al. [ 40 ] reveals a number of issues with current PETA material. These include challenges in designing, implementing, and evaluating PETA interventions.

One prominent challenge is the lack of customization of training content [46]. This can lead to employees often being disengaged and uninterested in the training material [ 5, 50 ]. Additionally, poorly customized training content can result in being repetitive [ 9 ], culturally biased [17], or time-consuming [ 9, 34 ] for employees. A noteworthy example is AntiPhishing Phil [43], a serious game for phishing education in which the user is asked to examine URLs to determine if they are associated with malicious or legitimate websites. One of the primary issues with the game was that a significant number of the presented websites were associated with American companies and thus unfamiliar to users outside of the United States. This resulted in some participants in the original study experiencing difficulty in determining if some URLs were legitimate or not. It is similarly important for training material to include different attack scenarios, in order to improve the users’ ability to detect a wider spectrum of phishing attacks [ 32, 50 ]. Therefore, training material that addresses only, e.g., how to spot phishing URLs will not adequately teach users to defend against more advanced techniques such as spear phishing or persuasion cues [ 10 ].

Another critical issue is related to the warnings employed for the protection of users from phishing attacks. These warnings are generally ill-positioned [ 9, 34, 52 ], passive (i.e., do not block the user interaction flow) [19], and lead to users becoming habituated [ 1, 4, 30 ]. Moreover, the content of warnings usually lacks explanations [ 3, 6 ], which can result in users not trusting the system and being less motivated to adopt safe behaviors [ 6, 8, 48 ].

Finally, PETA programs tend to be highly expensive in terms of both economic and human resources [ 7, 46 ]. Furthermore, the deployment of embedded training requires a significant amount of manual human effort for the production of fake phishing emails, their evaluation, the management of related tickets, the setting up of firewall rules, and so forth [ 3, 7 ]. This can result in the training material, such as anti-phishing recommendations, being outdated or incomplete [ 38 ]; it is instead crucial to include recent cyber-attacks and detailed information about how attackers operate and the types of tactics they use [45].

2.3. LLMs and education

Large Language Models (LLM) are gaining traction in the field of education because of their ability to provide tailored feedback and suggestions, saving teachers time and effort in creating personalized materials and tailored feedback [27]. There are some attempts in the literature to use LLMs in fields as diverse as physics education [53] and medical education [ 39 ]. Common use cases involve support to educators in assessing and grading written tests, providing feedback to students, and generating educational content [54].

The use of LLMs in education is not without challenges [49]. There are many valid criticisms of these tools; one of the main problems is that they generate responses by predicting the most likely next word without any grasping of the semantical level; their stochastic nature makes the models possibly “hallucinate”, producing seemingly confident responses that are not factual, in part due to incomplete or biased training data.

Nonetheless, LLMs are undeniably performant also on human tasks [24]. For example, medical students use these tools to explain complicated medical concepts in simple terms, generate self-study questions, and create preliminary diagnoses and possible treatment plans [ 40 ].

Therefore, although the issue of hallucinations remains challenging to address, the proposal presented in this paper may still prove viable. Moreover, hallucinations can be limited by designing prompts that follow the established guidelines and best practices [14, 44]. For example, approaches like “chain-of-thought” have proved to help LLMs produce more grounded outputs [13].

The current limitations and future prospects of LLMs in education will constitute a valuable source of discussion during the workshop.

3. An LLM-based approach to mitigate challenges in PETA

To address some of the key challenges in producing quality PETA material, i.e., high costs, lack of customization, and ineffective warnings, we propose an approach that leverages LLMs to help security practitioners and educators automate some tasks in the design process.

Automating the creation of training materials has already been indicated as being potentially beneficial to IT security teams across several dimensions, including reducing deployment and maintenance efforts, and the amount of human hours required, ultimately reducing costs for organizations [ 40 ]. Automation can also facilitate the delivery of tailored, recurrent, and relevant training interventions, reducing the costs associated with manually customizing training content [ 32 ].

3.1. Addressing the Lack of customization in PETA content

3.1.1. Customized Simulated Phishing Campaigns

In order to create customized training content, it is first necessary to take into account the different psychological and demographic factors of the employees. Recently, in the context of the DAMOCLES Italian project, we proposed an approach to systematically conduct and assess the individual vulnerabilities of employees in an organization [20]. This approach has the goal of investigating, in the context of a simulated phishing campaign, the interaction between attack techniques (persuasion principles [ 10 ] and emotional triggers) and user personality traits, to determine which characteristics of phishing emails maximize the effectiveness of the attacks for specific employees. Therefore, once the employee’s profile has been gathered in terms of the Big 5 model [ 37 ] (e.g., collected by administering the NEO Five-Factor-Inventory-3 [ 11 ]), tailored phishing emails can be crafted and delivered to test their susceptibility under challenging conditions. Obviously, the difficulty of the email is an important factor to consider in a phishing campaign: for example, the level of challenge could start low, by presenting users with phishing emails that are easy to detect and testing the user with very difficult emails towards the end of the campaign [ 12 ].

The role of the employee within the organization also plays a critical factor in the design of tailored phishing emails. For example, it would be an obvious red flag for a CEO to receive a phishing email sent by another “CEO” of their own company; therefore, such an attack would certainly be recognizable, even if the most effective social engineering techniques are used, based on his or her profile. Another factor that could be considered is the set of websites that employees visit most commonly to generate phishing URLs that resemble domains that are relevant to them. These could be collected either automatically by analyzing which websites the employee most frequently visits during their work hours, or by asking him or her to provide them spontaneously through a questionnaire. Moreover, the URLs on the organization’s internal Domain Name System (DNS) server can be used to include domains that resemble the legitimate ones used by the company [26]. Finally, the employee’s demographic information, such as name and gender, is a valuable source of information for creating spear-phishing emails that are more relevant, e.g., that do not contain generic greetings and address them by name.

LLMs can be used to automate the process of writing convincing emails which can include different topics and/or different persuasion principles (e.g., see [21, 22]). It is worth noting that commercially available LLMs like ChatGPT cannot be directly used to generate phishing emails, as this is not considered an ethical activity even for white-hat purposes. Therefore, less ethical tools such as Worm-GPT might be considered for generating phishing emails.

Once we have all the information about a specific employee, we can generate a phishing email by filling out the following prompt and feeding it to the LLM: “Pretend to be a security practitioner at [organization name] who is planning a simulated phishing campaign. You must create a fake phishing email in HTML format that is tailored to an employee of the organization. The email must be addressed to [employee name] (use [employee pronouns]), who is [employee role] in the organization. The email must be about [topic]; use Cialdini’s persuasion principle of [persuasion principle] and include sentences that leverage [emotional trigger]. You must use fake [organization nationality] real names for the sender. Create a phishing link URL to include in the email that mimics one of the following legitimate links: [URLs list]; for example, a fake URL for the website ‘paypal.com’ could be ‘https://www.paypal-refund-claim.com’”.

The persuasion principle refers to Cialdini’s theory of persuasion [ 10 ] and can be one of the following: authority, scarcity, reciprocation, social proof, liking, or consistency. The emotional trigger can refer to one among the most leveraged emotions used in phishing attacks: curiosity, fear, greed, anger, joy, confusion, or empathy. The choice of which persuasion principle and emotional trigger to use is dictated by the employee’s personality traits. The topic can vary to generate different emails covering different plausible scenarios such as “request of password reset”, “account blocked”, “free giveaway”, “payment request”, etc. It is worth noting that in this example the prompt is considering exactly one persuasion principle and one emotional trigger at once, for simplicity; however, more complex attacks may also include a combination of two or more persuasion principles (e.g., authority and scarcity) and/or emotional triggers to create more effective phishing emails. An example of the usage of this prompt is presented in Appendix A.

To investigate the output of the model, it is possible to ask the LLM to explain how the persuasion principles and emotional triggers are addressed by individual sentences or words in the generated text; we can also ask the LLM to report the legitimate URL that was mimicked with the phishing URL. To do this, we can extend the previous prompt with the following text: “For each persuasion principle and emotional trigger used in the email, produce an explanation that points out the pieces of text used to employ them.

Finally, report the legitimate URL that is being mimicked in the email.

Output format: [EMAIL] -----[EXPLANATION]”.

3.1.2. Customized Embedded Training

In addition to customizing the emails in the simulated phishing campaign, the educational content for conducting the embedding training must be generated and personalized taking into account both the type of techniques used in the phishing attacks and the information about the employee [26]. For example, the employee who falls victim to a simulated phishing email must be presented with a customized landing page that: • • •

Debriefs them about the simulated attack, addressing them by their name. Explains the social engineering techniques used in the email (e.g., authority principle and urgency) and some tips on how to avoid them.

Reports the fake phishing URL they clicked on and presents it next to the legitimate one, highlighting the phishing cues that the victim should have noticed, such as the top-level domain being misplaced, or URL spoofing.

The educational level for generating tailored training material must also be considered in the generation of the educational content: for example, users who are more familiar with technical aspects of IT security might benefit from explanations that include technical jargon and/or more details; therefore, the explanation could, e.g., include terms such as “domain”, “homograph attack”, etc., which might be more relevant to them. This allows us to generate explanations that are more appropriate to who receives the explanation [47].

A possible prompt to generate such embedded training content would be: “Pretend to be a cybersecurity educator who teaches employees how to recognize an email of phishing. [organization] organized a simulated phishing campaign to test its employees' susceptibility to phishing. Specifically, [employee name] clicked on a phishing link in one of the fake emails and was redirected to a landing page containing training information. The email used [social engineering principles] principles; moreover, the phishing link was [phishing

URL], which mimicked the legit link [mimicked URL].

Create a short explanation webpage that:

1) debriefs them about the simulated attack

2) explains the techniques that were used and some tips on how to avoid them

3) reports both URLs (phishing and mimicked), highlighting the URL spoofing techniques that were used.

Consider that the employee is a [employee role] and is [expertise level] of cybersecurity, and

tailor the explanation accordingly.“

3.2. LLMs for improving anti-phishing warnings

In order to constitute a valuable phishing awareness resource for users, warning dialogs must include content that is relevant and educational to the user. Determining a priori whether the warning content is helpful to the user is not an easy task, as it is heavily affected by the user’s knowledge of phishing, security, and IT. Moreover, employees usually have limited time to dedicate to reading warnings, as cybersecurity is usually a secondary task for them [ 35 ].

Therefore, if we want explanations in warning messages to be considered, they must be designed to be readable, understandable, and alerting [15]. In addition, warning messages should vary so that users do not easily become habituated to seeing the same warning under different risk circumstances [ 4 ]. Since generating high-quality warning messages is an onerous task, it is simply not feasible to manually create diverse content that also adapts to each user’s knowledge level.

A possible solution to this problem is to use LLMs to automatically generate warning dialogs that include explanations that (i) are dynamically generated (thus helping to avoid habituation), (ii) address the specific phishing threat (thus potentially improving decisionmaking), and (iii) adapt to the user’s knowledge (thus being relevant and understandable).

The explanation in a warning dialog should follow the best practices established in the literature of warning design and have a standard structure such as those proposed in [15]. Specifically, it should be formed by three parts: 1) a description of the phishing feature that is being explained, 2) an explanation of the hazard of phishing, and 3) potential consequences of a successful attack. Hereafter, we propose a possible prompt to generate such comprehensive warning dialogs: “Construct a brief explanation message (max 50 words) directed to [employee expertise level] that will follow this structure:

1. description of the most relevant phishing feature

2. explanation of the hazard

3. consequences of a successful phishing attack

For example, a message that explains that a URL in the email (PHISHING_URL) is imitating

another legitimate one (SAFE_URL), would be: ‘The target URL (PHISHING_URL) is an imitation of the original one, (SAFE_URL). This site might be intended to take you to a different place. You might be disclosing private information.’”.

4. Conclusions and future work

While phishing remains a critical problem, user education, training, and awareness can mitigate the success of these attacks and improve an organization’s susceptibility. In fact, addressing human vulnerabilities can make employees an essential line of defense against phishing attacks [ 41 ]. Since PETA interventions are often very expensive for organizations, automation can help reduce the burden and produce high-quality training materials more easily. In this paper, we have proposed LLMs as a tool to reduce manual effort and produce highly customizable training material that can be tailored to user needs.

LLMs are a relatively new technology that, despite their impressive performance on various human tasks [24], still have clear limitations, such as suffering from hallucinations, and thus need to be carefully supervised by human experts. However, we envision a future in which human-AI collaboration is fundamental to support complex tasks that traditionally belong to humans. Therefore, this approach may still be valuable for security practitioners and educators to produce effective PETA interventions much more efficiently.

In future work, we plan to implement the approach presented in this work and to include it as a possible mitigation strategy against phishing attacks for public administration within the DAMOCLES Italian project. The effectiveness of the proposed approach will be evaluated by iteratively assessing an organization’s susceptibility to phishing over time. Specifically, the effectiveness of an LLM-powered simulated phishing campaign will be evaluated by measuring the employees’ click rate at time zero (e.g., if it is too low, the phishing emails are probably too easy to detect); on the other hand, the effectiveness of the educational content will be measured by monitoring the click-rate over time for the exposed users. Finally, it will be of paramount importance to collect the feedback of employees who are exposed to LLM-generated training content both during the design phase and once the system is deployed.

Acknowledgements

This work has been supported by the Italian Ministry of University and Research (MUR)and by the European Union - NextGenerationEU, under grant PRIN 2022 PNRR "DAMOCLES: Detection And Mitigation Of Cyber attacks that exploit human vuLnerabilitiES" (Grant P2022FXP5B).

This work is partially supported by the co-funding of the European Union - Next Generation EU: NRRP Initiative, Mission 4, Component 2, Investment 1.3 - Partnerships extended to universities, research centres, companies and research D.D. MUR n. 341 del 5.03.2022 - Next Generation EU (PE0000014 – “Security and Rights In the CyberSpace – SERICS” - CUP: H93C22000620001).

The research of Francesco Greco is funded by a PhD fellowship within the framework of the Italian “D.M. n. 352, April 9, 2022”- under the National Recovery and Resilience Plan, Mission 4, Component 2, Investment 3.3 - PhD Project “Investigating XAI techniques to help user defend from phishing attacks”, co-supported by “Auriga S.p.A.” (CUP H91I22000410007). [13] DAIR.AI. Chain-of-Thought Prompting - Prompt Engineering Guide, Url: https://www.promptingguide.ai/techniques/cot Last Access 7 May 2024. [14] DAIR.AI. Prompt Engineering Guide, Url: https://www.promptingguide.ai/ Last Access 24 Jan. 2024. [15] Desolda, G., Aneke, J., Ardito, C., Lanzilotti, R. and Costabile, M.F. 2023. Explanations in warning dialogs to help users defend against phishing attacks. In International

Journal of Human-Computer Studies, 20.

[16] Desolda, G., Ferro, L.S., Marrella, A., Catarci, T. and Costabile, M.F. 2021. Human Factors in Phishing Attacks: A Systematic Literature Review. In ACM Computing Survey ACM, 35. [17] Dixon, M., Arachchilage, N.A.G. and Nicholson, J., Engaging Users with Educational

Games: The Case of Phishing. Conference on Human Factors in Computing Systems.

Glasgow, Scotland Uk, 2019, pp. 1-6. 10.1145/3290607.3313026 [18] Dupont, G., The Dirty Dozen Errors in Maintenance. Proceedings of the 11th Meeting on

Human Factors In Aviation Maintenance and Inspection.1997. [19] Egelman, S., Cranor, L.F. and Hong, J., You've Been Warned: An Empirical Study of the Effectiveness of Web Browser Phishing Warnings. SIGCHI Conference on Human Factors in Computing Systems. 2008, pp. 1065–1074 10.1145/1357054.1357219 [20] Greco, F., Buono, P., Desiato, D., Desolda, G., Lanzilotti, R. and Ragone, G., Unlocking the Potential of Simulated Phishing Campaigns: Measuring the Impact of Interaction among Different Human Factors. DAMOCLES: Detection And Mitigarion of Cyber attacks that exploit human vuLnerabilitiES workshop, co-located with AVI '24.

Arenzano (Genoa), Italy, 2024, pp. 10. [21] Greco, F., Desolda, G., Esposito, A. and Carelli, A., David versus Goliath: Can Machine Learning Detect LLM-Generated Text? A Case Study in the Detection of Phishing

Emails. The Italian Conference on CyberSecurity. Salerno, Italy, 2024, pp.

[22] Hazell, J. Spear Phishing With Large Language Models, 2023. url: https://arxiv.org/abs/2305.06972 Last Access. [23] Hu, S., Hsu, C. and Zhou, Z., Security Education, Training, and Awareness Programs: Literature Review, JCIS 62 (2022) pp. 752-764. doi:10.1080/08874417.2021.1913671. [24] HuggingFace. Open LLM Leaderboard, 2024. Url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard Last Access 13 Feb. 2024. [25] IBM 2023. Security X-Force Threat Intelligence Index. [26] Jampen, D., Gür, G., Sutter, T. and Tellenbach, B., Don’t click: towards an effective antiphishing training. A comparative literature review, Hum. Cent. Comput. Inf. Sci. 10 (2020) pp. 33. doi:10.1186/s13673-020-00237-7. [27] Kasneci, E., Sessler, K., Küchemann, S., Bannert, M., Dementieva, D., Fischer, F., Gasser, U., Groh, G., Günnemann, S., Hüllermeier, E., Krusche, S., Kutyniok, G., Michaeli, T., Nerdel, C., Pfeffer, J., Poquet, O., Sailer, M., Schmidt, A., Seidel, T., Stadler, M., Weller, J., Kuhn, J. and Kasneci, G., ChatGPT for good? On opportunities and challenges of large language models for education, ChatGPT for good? On opportunities and challenges of large language models for education 103 (2023) pp. 102274. doi:10.1016/j.lindif.2023.102274. [28] Katsikas, S.K., Health care management and information systems security: awareness, training or education?, Int. J. Med. Inform. 60 (2000) pp. 129-135. doi:10.1016/S1386-5056(00)00112-X. [43] Sheng, S., Magnien, B., Kumaraguru, P., Acquisti, A., Cranor, L.F., Hong, J. and Nunge, E., Anti-Phishing Phil: the design and evaluation of a game that teaches people not to fall for phish. Symposium on Usable privacy and security. Pittsburgh, Pennsylvania, USA, 2007, pp. 88–99. 10.1145/1280680.1280692 [44] Shieh, J. Best practices for prompt engineering with OpenAI API OpenAI. [45] TerranovaSecurity. Phishing Benchmark Global Report, 2021. Url: https://www.terranovasecurity.com/resources/guides/gone-phishing-report2021 Last Access 21 Apr 2024. [46] Tessian 2022. Phishing Awareness Training: How Effective is Security Training? In

Advanced Email Threats.

[47] Viganò, L. and Magazzeni, D., Explainable Security. IJCAI/ECAI 2018 Workshop on

Explainable Artificial Intelligence. 2018, pp. 7. arXiv:1807.04178

[48] Vilone, G. and Longo, L., Notions of explainability and evaluation approaches for explainable artificial intelligence, Inf Fusion 76 (2021) pp. 89-106. doi:10.1016/j.inffus.2021.05.009. [49] Wang, S., Xu, T., Li, H., Zhang, C., Liang, J., Tang, J., Yu, P.S. and Wen, Q. Large Language

Models for Education: A Survey and Outlook, 2024. url:

https://arxiv.org/abs/2403.18105 Last Access. [50] Wen, Z.A., Lin, Z., Chen, R. and Andersen, E., What.Hack: Engaging Anti-Phishing Training Through a Role-playing Phishing Simulation Game. Conference on Human Factors in Computing Systems. Glasgow, Scotland Uk, 2019, pp. 1-12. 10.1145/3290605.3300338 [51] Wright, R.T., Jensen, M.L., Thatcher, J.B., Dinger, M. and Marett, K., Research Note: Influence Techniques in Phishing Attacks: An Examination of Vulnerability and Resistance, Inf. Syst. 25 (2014) pp. 385-400. [52] Wu, M., Miller, R.C. and Garfinkel, S.L., Do Security Toolbars Actually Prevent Phishing

Attacks? SIGCHI Conference on Human Factors in Computing Systems. 2006, pp. 601–

610. 10.1145/1124772.1124863 [53] Wulff, P., Physics language and language use in physics - What do we know and how AI might enhance language-related research and instruction, Eur. J. Phys. 45 (2024) pp. 023001. doi:10.1088/1361-6404/ad0f9c. [54] Yan, L., Sha, L., Zhao, L., Li, Y., Martinez-Maldonado, R., Chen, G., Li, X., Jin, Y. and Gašević, D., Practical and ethical challenges of large language models in education: A systematic scoping review, BJET 55 (2024) pp. 90-112. doi:10.1111/bjet.13370.

A. Appendix: An example of an email created using an LLM

The email was generated in Italian since ChatGPT recognized that the University of Bari is an Italian institution. We asked subsequently to translate it to English to present it to our English-speaking readers. In the following there is the translated email:

Subject: Urgent: Reset Your Account Password

Dear Dr. Francesco Greco,

I hope this email finds you well. I am Martina Rossi from the Department of Computer

Technology at the University of Bari.

We are conducting an important maintenance operation on the university's computer systems, and we need your prompt assistance. As part of this process, we have identified a potential security risk related to your login credentials.

To ensure the utmost security of your data and the university's systems, we kindly ask you to proceed with resetting your account password.

Please click on the following link to complete the password reset process: [https://www.uniba-idp-cineca-passwordreset.com]

Please note that this operation is essential to ensure the security of your personal data and the university's data. We kindly request you to act promptly.

Thank you for your cooperation and understanding in this matter.

Best regards,

Martina Rossi

Department of Computer Technology

University of Bari

Note: This email has been sent automatically. Please do not reply directly.

References

[1] Akhawe , D. and Felt , A.P. , Alice in warningland: a large-scale field study of browser security warning effectiveness . USENIX conference on Security. Washington, D.C., 2013 , pp. 257 - 272 .

[2] Almomani , A. , Gupta , B.B. , Atawneh , S. , Meulenberg , A. and Almomani , E. , A Survey of Phishing Email Filtering Techniques , IEEE Commun. Surv. Tutor . 15 ( 2013 ) pp. 2070 - 2090 . doi: 10 .1109/SURV. 2013 . 030713 .00020.

[3] Althobaiti , K. , Meng , N. and Vaniea , K. , I Don 't Need an Expert! Making URL Phishing Features Human Comprehensible . Conference on Human Factors in Computing Systems. Yokohama, Japan , 2021 , pp. 1 - 17 . 10 .1145/3411764.3445574

[4] Anderson , B.B. , Kirwan , C.B. , Jenkins , J.L. , Eargle , D. , Howard , S. and Vance , A. , How Polymorphic Warnings Reduce Habituation in the Brain: Insights from an fMRI Study . ACM Conference on Human Factors in Computing Systems . Seoul, Republic of Korea, 2015 , pp. 2883 - 2892 . 10 .1145/2702123.2702322

[5] Arachchilage , N.A.G. , Love , S. and Beznosov , K. , Phishing threat avoidance behaviour: An empirical investigation , Comput. Hum. Behav . 60 ( 2016 ) pp. 185 - 197 . doi: 10 .1016/j.chb. 2016 . 02 .065.

[6] Bravo-Lillo , C. , Cranor , L.F. , Downs , J. , Komanduri , S. and Sleeper , M. , Improving Computer Security Dialogs. International Conference on Human-Computer Interaction. Berlin, Heidelberg, 2011 , pp. 18 - 35 .

[7] Brunken , L. , Buckmann , A. , Hielscher , J. and Sasse , M.A. , "To Do This Properly, You Need More Resources": The Hidden Costs of Introducing Simulated Phishing Campaigns . 32nd USENIX Security Symposium 2023 , pp. 4105 - 4122 .

[8] Buono , P. , Desolda , G. , Greco , F. and Piccinno , A. , Let warnings interrupt the interaction and explain: designing and evaluating phishing email warnings . CHI Conference on Human Factors in Computing Systems. Hamburg Germany , 2023 , pp. 1 - 6 . 10 .1145/3469886

[9] Caputo , D.D. , Pfleeger , S.L. , Freeman , D.J. and Johnson, M.E., Going Spear Phishing: Exploring Embedded Training and Awareness, S&P 12 ( 2014 ) pp. 28 - 38 . doi: 10 .1109/MSP. 2013 . 106 .

[10] Cialdini , R.B. , Influence : The Psychology of Persuasion. revised ed, Harper Collins. 2009 .

[11] Costa , P.T. and McCrae , R.R. , Four ways five factors are basic , Pers. Individ. Dif . 13 ( 1992 ) pp. 653 - 665 . doi: 10 .1016/ 0191 - 8869 ( 92 ) 90236 - I .

[12] CybSafe . The ultimate people-centric guide to simulated phishing, 2023 . Url: https://www.cybsafe.com/value/simulated-phishing / Last Access 21 Apr 2024 .

[29] Khonji , M. , Iraqi , Y. and Jones , A. , Phishing Detection : A Literature Survey , IEEE Commun. Surv. Tutor . 15 ( 2013 ) pp. 2091 - 2121 . doi: 10 .1109/SURV. 2013 . 032213 .00009.

[30] Kim , S. and Wogalter , M.S. , Habituation , Dishabituation, and Recovery Effects in Visual Warnings, Habituation, Dishabituation, and Recovery Effects in Visual Warnings 53 ( 2009 ) pp. 1612 - 1616 . doi: 10 .1177/154193120905302015.

[31] Kirlappos , I. and Sasse , M.A. , Security Education against Phishing: A Modest Proposal for a Major Rethink , S&P 10 ( 2012 ) pp. 24 - 32 . doi: 10 .1109/MSP. 2011 . 179 .

[32] KnowB4 . Whitepaper: Building an Effective and Comprehensive Security Awareness Program Url: https://info.knowbe4.com/wp-building-effective-comprehensive-sat Last Access 21 Apr 2024 .

[33] Kumaraguru , P. , Cranshaw , J. , Acquisti , A. , Cranor , L. , Hong , J. , Blair , M.A. and Pham , T. , School of phish: a real-world evaluation of anti-phishing training . Symposium on Usable Privacy and Security . Mountain View, California, USA, 2009 , pp. 1 - 12 . 10 .1145/1572532.1572536

[34] Kumaraguru , P. , Rhee , Y. , Acquisti , A. , Cranor , L.F. , Hong , J. and Nunge , E. , Protecting people from phishing: the design and evaluation of an embedded training email system . SIGCHI Conference on Human Factors in Computing Systems . San Jose, California, USA, 2007 , pp. 905 - 914 . 10 .1145/1240624.1240760

[35] Kumaraguru , P. , Sheng , S. , Acquisti , A. , Cranor , L.F. and Hong , J. , Teaching Johnny not to fall for phish , Trans. Internet Technol . 10 ( 2010 ) pp. 1 - 31 . doi: 10 .1145/1754393.1754396.

[36] Lain , D. , Kostiainen , K. and Čapkun , S. , Phishing in Organizations: Findings from a LargeScale and Long-Term Study , in: Proceedings of the 2022 IEEE Symposium on Security and Privacy , SP , Year, pp. 842 - 859 . doi: 10 .1109/SP46214. 2022 . 9833766 .

[37] McCrae , R.R. and Costa, P.T.J., The five-factor theory of personality . Handbook of personality: Theory and research , The Guildford Press, New York, NY, US. 2008 .

[38] Mossano , M. , Vaniea , K. , Aldag , L. , Düzgün , R. , Mayer , P. and Volkamer , M. , Analysis of publicly available anti-phishing webpages: contradicting information, lack of concrete advice and very narrow attack vector , in: Proceedings of the IEEE European Symposium on Security and Privacy Workshops , EuroS&PW '20 , IEEE, Year , pp. 130 - 139 . doi: 10 .1109/EuroSPW51379. 2020 . 00026 .

[39] Pandey , M. and Mishra , V. , Large Language Models in Medical Education and Quality Concerns, JQHE 6 ( 2023 ) pp. 3 . doi: 10 .23880/jqhe-16000319.

[40] Sarker , O. , Jayatilaka , A. , Haggag , S. , Liu, C. and Babar , M.A. , A Multi-vocal Literature Review on challenges and critical success factors of phishing education, training and awareness , JSS 208 ( 2024 ) pp. 111899 . doi:https://doi.org/10.1016/j.jss. 2023 . 111899 .

[41] Sasse , M.A. , Brostoff , S. and Weirich , D. , Transforming the 'Weakest Link' - a Human/Computer Interaction Approach to Usable and Effective Security, Transforming the 'Weakest Link' - a Human/Computer Interaction Approach to Usable and Effective Security 19 ( 2001 ) pp. 122 - 131 . doi: 10 .1023/A: 1011902718709 .

[42] Sheng , S. , Holbrook , M. , Kumaraguru , P. , Cranor , L.F. and Downs , J. , Who falls for phish? a demographic analysis of phishing susceptibility and effectiveness of interventions . SIGCHI Conference on Human Factors in Computing Systems. Atlanta , Georgia, USA, 2010 , pp. 373 - 382 . 10 .1145/1753326.1753383