Artificial intelligence tools in the ongoing fight against bullying and cyberbullying: a multidisciplinary approach Giulia Orrù1,* , Vincenzo Gattulli2 , Guido Colaiacovo4 , Stefano Marrone3 , Giovanni Puglisi1 , Lucia Sarcinella2 , Grazia Terrone5 , Donatella Curtotti4 , Donato Impedovo2 , Gian Luca Marcialis1 and Carlo Sansone3 1 University of Cagliari, piazza d’Armi, 09123, Cagliari, Italy 2 University of Bari, Via Edorado Orabona 4, 70121, Bari, Italy 3 University of Naples Federico II, Via Claudio 21, 80125 Naples, Italy 4 University of Foggia, Via Antonio Gramsci 89, 71122 Foggia, Italy 5 Tor Vergata University, Via Columbia 1, 00133 Roma, Italy Abstract The problem of bullying and cyberbullying is growing and is a phenomenon that negatively influences our society: for this reason, it requires advanced solutions aimed at prevention. Starting from a previous project called BullyBuster which paved the way for the application of artificial intelligence (AI) in this area, exploiting interdisciplinary skills to develop algorithms capable of identifying and mitigating bullying behaviours, in this paper, we present the follow-up, called BullyBuster 2 (BB2). BB2 aims to broaden the spectrum of intervention to include adult populations, reflecting the universal nature of bullying across all age demographics. Furthermore, it updates and improves the original project’s methodologies with new psychological insights and broader AI applications, ensuring a more inclusive anti-bullying strategy. Furthermore, BB2 aims to form a large research consortium to set a new standard for interdisciplinary collaborations in preventive and response strategies against such social challenges. Keywords bullying, detection, cyberbullying, artificial intelligence, interdisciplinarity 1. Introduction bullying in both physical and virtual contexts and for this reason it has been included in the Global Top 100 list The worrying rise in bullying and cyberbullying in re- of AI projects addressing the 17 United Nations Strate- cent years has brought attention to the urgent need for gic Development Goals by the International Research efficient prevention measures and solutions [1]. In fact, Center for Artificial Intelligence under the auspices of such violent behaviours not only compromise individual UNESCO1 . feelings of security and dignity, but they also have signif- This paper introduces “BullyBuster 2 – the ongo- icant psychological effects that may reverberate across ing fight against bullying and cyberbullying with the society. Bullying has changed throughout time and is help of artificial intelligence for the human wellbeing” now prevalent in a variety of adult demographics and project (BB2, hereinafter), the follow-up to the Bully- settings, which has made it clear that creative and flexible Buster project. Funded by the European Union - NextGen- solutions are required. erationEU and within the PRIN 2022 PNRR, it again in- In this context, the “BullyBuster - A framework for volves four multidisciplinary research groups belonging bullying and cyberbullying action detection by computer to four universities in Southern Italy (University of Bari vision and artificial intelligence methods and algorithms” Aldo Moro, University of Cagliari, University of Foggia, project [2](BB1 in the following) was proposed: funded University of Naples Federico II) with the aim of combin- under the tender relating to Projects of Relevant National ing artificial intelligence, technology, law and psychology Interest (PRIN) in 2017 and recently concluded, it marked skills to develop a comprehensive framework. a pioneering effort by interdisciplinary teams from four BB2’s goal is to advance bullying and cyberbullying universities in Southern Italy to tackle this threat through detection capabilities by incorporating advanced models cutting-edge artificial intelligence tools [3]. This initia- that take into account a broader range of victim profiles tive sought to combine psychological insights with tech- and refining its detection algorithms to accommodate nological advances to create tools capable of identifying new forms of interpersonal aggression and IT misconduct. BB2 also aims to broaden the objectives of the previous Ital-IA 2024: 4th National Conference on Artificial Intelligence, orga- nized by CINI, May 29-30, 2024, Naples, Italy * Corresponding author. 1 https://ircai.org/top100/entry/bullybuster-a-framework-for- $ giulia.orru@unica.it (G. Orrù) bullying-and-cyberbullying-action-detection-by-computer-vision- © 2024 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). and-artificial-intelligence-methods-and-algorithms/ CEUR ceur-ws.org Workshop ISSN 1613-0073 Proceedings 2.1. Legal, juridical and psychological aspects Bullying and cyberbullying activities, characterised by aggressive and arrogant attitudes, repeated over time and perpetrated to the harm of children, adolescents, and even adults, have become a critical emergency [4]. Despite legislative efforts [5], research into protecting individuals and their well-being prior to the occurrence Figure 1: BullyBuster2 project logo. of an incident through the prevention of bullying and cy- berbullying behaviours is still in its early stages. In fact, intervening in the preventative phase serves to protect BB1 to include adults, reflecting the broader spectrum the subject’s psychophysical integrity and avert victimi- of bullying victims. This initiative is driven by recent sation. To ensure the individual’s safety, law enforcement social challenges and legislative demands, recognizing activities focused at punishing the perpetrator after the that bullying is a pervasive threat not limited to young damaging event has happened are no longer sufficient. people but a general public health issue. However, it is critical to respond before a potentially A further objective of the project is the creation of an detrimental or dangerous occurrence affects the victims. interdisciplinary reference laboratory to combat the phe- The methodological approach that BullyBuster adopts nomenon of bullying and cyberbullying. This laboratory, is based on the creation of technological tools to com- in which expert technologists, jurists, psychologists, so- bat the phenomenon in an integrated approach, both ciologists and economists will participate, will contribute teleologically oriented towards the protection of psycho- to greater dissemination and expansion of the research logical profiles based on the well-being of the subject and community in the field, including stakeholders such as attentive to legal aspects, with a view to prevention. institutions and companies and subjects potentially in- terested in the engineering of anti-bullying tools. 2.2. Multimedia contents artifacts The term "deepfake" refers to any way of creating fake 2. The BullyBuster framework multimedia material representing one or more individ- uals [6]. The modification may specifically target the In this section, we explore the key elements of Bully- individual’s identity, speech, or expression. Although Buster’s architectural framework (Fig. 2), which includes this technology is of considerable appeal to some sectors the tools created in BB1 as well as the enhancements of society, such as the film and video game industries, the planned for BB2. The architecture of the system is struc- harmful applications are vast and particularly concerning. tured into five primary modules: (1) a manipulated mul- In fact, deepfakes may position a person in non-real hu- timedia content detection module, to combat the spread miliating or compromising situations, such as scenes of of malicious and fake multimedia content, as deepfakes; sex or violence, or force her/him to say something she/he (2) a physical violence detection module, to monitor and never said. Deepfake can thus be used to insult, stalk, identify potential bullying incidents in the physical do- propagate revenge pornography, and abuse or cyberbully main; (3) a stress detection module, able to assesses the someone. In BB1, we exploited the complementarity of emotional state of individuals by analyzing the timing several individual video deepfake detectors with appro- and rhythm of keystrokes and other behavioral biomet- priate fusion rules to increase the generalization ability rics; (4) a verbal abuse detection module, focused on of modern deepfake detection systems [7]. The goal of the analysis of textual communications to detect verbal BB2 is to extend the deepfake video detection capabili- signs of bullying or cyberbullying; (5) a internet addic- ties and add voice cloning detection to make multimedia tion detection module, to detect psychological malaise by content analysis complete. automatic behavioral analyses. When combined, these components provide a complete system that is suited to address the complex problem of bullying in both physi- 2.3. Anomaly detection of events cal and digital scenarios. In order to address the critical The battle against bullying may be greatly aided by hav- issues of privacy and data protection, this solutions have ing a visual understanding of human behaviour through been examined from a legal and juridical perspective. video analysis. Behavioural "indicators," whether in the Below, we describe the individual modules of the Bully- form of victimisation or bullying, might include physi- Buster framework. cal aggression, seclusion, or other physical patterns like encircling. In BB1, systems have been created that can Figure 2: The BullyBuster 2 project framework. use one or more surveillance cameras to monitor groups [10]. The goal was to develop a system that automati- of people who are not individually identifiable [8]. The cally identified aggressive behaviour in texts, especially data is then processed to report "anomalous" events like in Italian-language comments. To do this, the language violent or panicked episodes based on behavioural mod- patterns used in aggressive comments were analyzed, els that have been suitably codified. In BB2, the crowd noting the frequent use of vulgar and negative language, analytics model will specialize in bullying contexts by insults, and offensive words. In addition, many of these generating synthetic data: obtaining contextualised train- comments began with "no/not," which is used to contra- ing data proved to be a challenge during the creation of dict or deny a statement. Based on these observations, the crowd analyzer in BB1, made worse by the start of several features were developed for feature engineering, the COVID-19 epidemic, which made it impossible to including the number of negative words, the number of mimic actual bullying scenarios. In addition, a substan- "no/not," use of capitalisation, positive/negative comment tial amount of video is needed for deep-learning models. weight calculated with WordNet and SentiWordNet, use In BB2, we aim to generate a sufficient amount of data of the second person, presence of threats and bullying and to include the typical behaviours of bullies and vic- terms [10]. These features were used to create several tims, highlighted in the psychological analysis, through dictionaries of terms and a dataset of aggressive and non- the synthetic creation of this data. aggressive comments. Several shallow learning models were tested to evaluate the effectiveness of these features 2.4. Verbale abuse models in detecting verbal aggression [10]. In the context of the BB2 project, this phenomenon Cyberbullying is a widespread phenomenon involving can also be analysed by considering different age groups, the use of digital technologies to perpetrate aggressive especially the preponderance of adults. Adults may re- behavior, threats, or harassment toward other individu- ceive significant criticism, fueling a cycle of online neg- als. This bullying occurs primarily through online plat- ativity and conflict that can also be reflected in real life. forms, such as social media, instant messaging, and fo- BB2 aims to address this problem by implementing and rums. It can take many forms, including name-calling, improving existing text models, with particular atten- defamation, social exclusion, and disclosure of private tion to the users’ ages. While BB1 primarily addressed information. Cyberbullying can have serious emotional, adolescents, BB2 includes an analysis of adult language, psychological, and social consequences on victims and recognizing that verbal aggression and communication poses a significant challenge for parents, educators, and dynamics are not limited exclusively to the younger gen- mental health professionals in ensuring a safe and re- eration. BB2’s approach is based on analyzing textual spectful online environment for all users. One of the dynamics found in social media comments, examining most predominant vectors is textual language [9]. how different users interact and communicate. This ap- In the BB1 project, UNIBA focused on identifying ver- proach allows for the identification of common linguistic bal aggression, a crucial aspect of cyberbullying that pri- and behavioral patterns. The categories used are simi- marily involves texts and comments on online platforms larly taken by assigning the label of cyberaggression or 2.6. Digital devices addiction non-cyberaggression. Through this in-depth analysis, Examining data from mobile devices has emerged as a BB2 aims to provide users, social platforms, and mental prominent approach for understanding individuals’ be- health professionals with a better understanding of how havior in today’s digital era [17]. However, the abun- verbal aggression occurs online and the factors that in- dance of features on these devices, such as web browsing, fluence it. This can enable the development of targeted online gaming, digital photography, GPS navigation, and interventions to prevent and address cyberbullying, im- various social applications, can easily engage users’ at- prove online safety, and promote a healthier and more tention, potentially causing significant distractions from respectful virtual environment for all users. real-world activities. Unfortunately, excessive engage- The text also fits within the framework of the Behav- ment with these features can lead to various health is- ioral Biometrics approach, in which touch-related fea- sues, particularly psychological challenges, such as lack tures, called Touch Dynamics, are extracted along with of self-control, withdrawal symptoms, sleep disruptions, textual output. This could be a starting point for future social isolation, depression, and difficulties in maintain- work [11]. ing focus. Additionally, users may display symptoms of irritability, restlessness, stress, and mood fluctuations. In 2.5. Stress detection response to these concerns, the goal of BB2 is to introduce Biometrics involves the utilization of body measurements an innovative module aimed at assessing individuals from and statistical analysis to extract and quantify human a psychological standpoint, based on their interactions characteristics [12]. Initially utilized for user authentica- with different features of their mobile devices. tion and identification purposes [13], , this technology is now increasingly utilized across various domains, includ- 3. The interdisciplinary laboratory ing entertainment and personalized user experiences [14]. Within the field of biometrics, there exist two distinct cat- Based on the results of the completed BB1 project and the egories of approaches: i) Physiological biometrics, which ongoing BB2 project, an interlaboratory group named involves the direct measurement of physical attributes "BullyBuster" is being formed with the goal of attracting such as facial features, fingerprints, iris patterns, reti- membership and engagement from individuals who are nal structures, and vocal characteristics; ii) Behavioral particularly interested in solving the issue of bullying and biometrics, which focuses on capturing specific human cyberbullying. The interdisciplinary laboratory brings to- behaviors such as handwriting, typing, or speaking. gether specialists from various fields to develop strategies Among various forms of behavioral biometrics, for analyzing bullying and cyberbullying behaviours, ex- keystroke dynamics has emerged as a highly efficient tending beyond just school-aged individuals. This team and economical technique, easily deployable using read- includes: (1) experts in information security, computer ily available hardware. In recent years, it has seen in- vision, and artificial intelligence, to create and propose creased utilization for user authentication, analyzing the innovative models and methods to be implemented in habitual rhythm patterns exhibited while typing on both appropriate demonstration products; (2) psychologists, physical and virtual keyboards [15]. to understand the behaviour patterns of individual in- In the context of combating cyberbullying, we employ volved in bullying events; (3) sociologists, to contribute keystroke dynamics for user emotion recognition. The by examining the effects of bullying and aggressive be- aim is to exploit its potential as a cost-effective and widely haviours in social media and other environments; (4) ju- accessible method for emotion recognition, requiring ridical and legal professionals, to explore new legislation only a standard keyboard as hardware [16]. Furthermore, that can address (cyber)bullying, enabling both preventa- since a keystroke recorder can be implemented as hard- tive and punitive technological solutions; (5) economists ware or software, with the latter being inconspicuous, and healthcare professionals, to assess the financial and individuals using the keyboard must be made aware of health impacts of (cyber)bullying, whether stemming the monitoring process. In BB2, keystroke dynamics will from the attacks themselves or from the origins of such be examined in real-world settings, considering both key- behaviours. The laboratory aims to create a genuine sup- board inputs and tapping habits to characterize a subject port system and reference services for preventing and based on their mobile device usage behavior. Addition- fighting bullying. ally, this analysis can incorporate touch dynamics on mobile devices. 4. Conclusions Bullying and cyberbullying are significant societal issues that have an adverse impact on people and communities. URL: https://www.mdpi.com/2078-2489/14/8/430. doi:10.3390/info14080430. [3] G. Orrù, A. Galli, V. Gattulli, M. Gravina, S. Mar- rone, M. Micheletto, A. Procaccino, W. Nocerino, G. Terrone, D. Curtotti, et al., Leveraging artifi- cial intelligence to fight (cyber) bullying for hu- man well-being: The bullybuster project, in: CEUR WORKSHOP PROCEEDINGS, volume 3486, CEUR- WS Team, Redaktion Sun SITE, 2023, pp. 189–194. [4] M. Wiertsema, C. Vrijen, R. van der Ploeg, M. Sentse, T. Kretschmer, Bullying perpetration and social Figure 3: The BullyBuster inter-laboratory group aims at status in the peer group: A meta-analysis, Journal gathering adhesions and participation of those who are par- of Adolescence 95 (2023) 34–55. doi:10.1002/jad. ticularly interested in addressing the issue of bullying and 12109. cyberbullying according to an interdisciplinary declination. [5] M. O. Mantovani, Profili penali del cyberbullismo: la l. 71 del 2017, Indice penale (2018) 475. [6] M. S. Rana, M. N. Nobi, B. Murali, A. H. Sung, Deep- fake detection: A systematic literature review, IEEE The BullyBuster (BB) framework, developed during two access 10 (2022) 25494–25513. projects funded under the tender relating to Projects of [7] S. Concas, S. M. La Cava, G. Orrù, C. Cuccu, J. Gao, Relevant National Interest (PRIN), one completed and X. Feng, G. L. Marcialis, F. Roli, Analysis of score- the other ongoing, combines advanced computer vision, level fusion rules for deepfake detection, Applied artificial intelligence technology, legal aspects and psy- Sciences 12 (2022). doi:10.3390/app12157365. chology concepts with a multidisciplinary approach to [8] G. Orrù, D. Ghiani, M. Pintor, G. L. Marcialis, F. Roli, provide a holistic response to these prevalent problems. Detecting anomalies from video-sequences: a novel In this paper, we present the development objectives of descriptor, in: 2020 25th International Conference the BullyBuster framework, which include the improve- on Pattern Recognition (ICPR), IEEE, 2021, pp. 4642– ment of tools for the detection of bullying and cyberbully- 4649. ing, the extension of its application to include a target of [9] S. Joksimovic, R. S. Baker, J. Ocumpaugh, J. M. L. adults and the creation of an interdisciplinary reference Andres, I. Tot, E. Y. Wang, S. Dawson, Automated laboratory. identification of verbally abusive behaviors in on- line discussions, in: Proceedings of the third work- Acknowledgments shop on abusive language online, 2019, pp. 36–45. [10] V. Gattulli, D. Impedovo, G. Pirlo, L. Sarcinella, This work is supported by the European Union – Next Cyber aggression and cyberbullying identification Generation EU under the PRIN 2022 PNRR project “Bully- on social networks (2022) 644–651. doi:10.5220/ Buster 2 – the ongoing fight against bullying and cyber- 0010877600003122. bullying with the help of artificial intelligence for the [11] V. Gattulli, D. Impedovo, G. Pirlo, F. Volpe, Touch human wellbeing” (CUP: P2022K39K8). events and human activities for continuous authen- tication via smartphone, Scientific Reports 13 (2023) 10515. References [12] P. S. Teh, A. B. J. Teoh, S. Yue, et al., A survey of keystroke dynamics biometrics, The Scientific [1] P. K. Smith, J. Mahdavi, M. Carvalho, S. Fisher, World Journal 2013 (2013). S. Russell, N. Tippett, Cyberbullying: its na- [13] A. Jain, L. Hong, S. Pankanti, Biometric identifica- ture and impact in secondary school pupils, tion, Communications of the ACM 43 (2000) 90–98. Journal of Child Psychology and Psychiatry [14] R. L. Mandryk, L. E. Nacke, Biometrics in gaming 49 (2008) 376–385. doi:https://doi.org/10. and entertainment technologies, in: Biometrics in a 1111/j.1469-7610.2007.01846.x. Data Driven World, Chapman and Hall/CRC, 2016, [2] G. Orrù, A. Galli, V. Gattulli, M. Gravina, pp. 215–248. M. Micheletto, S. Marrone, W. Nocerino, A. Pro- [15] M. Karnan, M. Akila, N. Krishnaraj, Biometric per- caccino, G. Terrone, D. Curtotti, D. Impedovo, G. L. sonal authentication using keystroke dynamics: A Marcialis, C. Sansone, Development of technolo- review, Applied soft computing 11 (2011) 1565– gies for the detection of (cyber)bullying actions: 1573. The bullybuster project, Information 14 (2023). [16] S. Marrone, C. Sansone, Identifying users’ emo- tional states through keystroke dynamics, in: Pro- ceedings of the 3rd International Conference on Deep Learning Theory and Applications - Volume 1: DeLTA„ INSTICC, SciTePress, 2022, pp. 207–214. doi:10.5220/0011367300003277. [17] F. S. Rahayu, L. E. Nugroho, R. Ferdiana, D. B. Se- tyohadi, Research trend on the use of it in digital addiction: An investigation using a systematic lit- erature review, Future Internet 12 (2020) 174.