The article presents the development of intelligent information technology for automatic detection of dangerous content in text messages, in particular calls for terrorist actions and hidden sarcasm. The system combines rule-based filtering of key triggers with in-depth classification based on the BERT model, further trained on a specialised corpus of texts. The developed pipeline includes text preprocessing, tokenisation, vector representation, rule-based analysis and deep classification using the Softmax layer. For training, the AdamW optimiser was used with an initial learning rate = 2e-5, a packet size of batch_size = 16, and a maximum sequence length of 128 tokens. The model was trained for one epoch, with the possibility of further expansion to 3-5. In the process of experimental verification, optimal results of classification accuracy and average time of operation of one message were obtained. The accuracy of classification on test data exceeded 90% (in control examples, the confidence level reached 100% for single-digit categories). The average processing time of one message was 0.54 s, the minimum was 0.30 s, the maximum was 0.74 s, with a total number of processed messages of 100. The analysis of the learning rate parameter showed that the optimal values are in the range of 1e-5 - 2e-5, which provided the highest stability and accuracy of the model. In addition, the integration of the model with the Telegram bot demonstrated the effectiveness of using the system in real time, confirming its suitability for practical application in automated online communications monitoring systems. The results of the study indicate the feasibility of combining rule-based and neural network methods to increase the reliability of detecting hidden threats. The developed system can be scaled for multilingual content and integrated into existing security and moderation platforms.
In today's world, terrorist organisations are increasingly using digital platforms to coordinate their actions, spread ideology, and recruit new
members. In particular, social networks, instant messengers, forums and comments under news publications have become an effective medium for spreading calls for violence, hostility and terrorist acts. The problem is that a vast amount of textual content is generated on the Internet every day, which is physically impossible to fully verify manually. It creates a critical need for automated means of detecting threatening information influence.
Existing content moderation systems are often based on simple keyword searches that do not take into account context, synonymy, irony, hidden forms of threats, or letter substitution. In addition, much of the content is created in languages for which there is limited support in standard analysis tools. In this context, the development of an intelligent system that is capable of analysing the content of messages at a deeper linguistic level takes on special importance.
The proposed system is designed to solve this problem by combining methods of computational linguistics and statistical and semantic analysis of the text, which will allow the detection of direct calls for terrorism and the recognition of latent threats hidden in indirect formulations. Threats to national security also support the relevance of the topic, the need to ensure public safety, prevent crimes before they are committed, and the requirements of international partnerships to combat terrorism in cyberspace. It is necessary to develop an intelligent system that will automatically detect text messages containing potential calls for terrorist actions. Given the growing activity of radical groups in the digital environment, especially in social networks, there is a need to create tools that could monitor large amounts of textual information in real time and detect dangerous statements. The system will be based on computational linguistics methods, including the analysis of lexical constructions, contextual relationships, and the use of machine learning algorithms to classify messages. This approach will create an effective decision-support tool for cybersecurity professionals and law enforcement agencies in order to respond to potential threats in a timely manner and prevent possible disasters. Thus, the creation of such a system is a timely response to danger in the modern world, has significant scientific and practical potential and fully meets the priorities of the development of information technologies, artificial intelligence and decision support systems in the field of security. The objective of the research is to develop an information technology that automatically detects and classifies text messages containing direct or veiled calls for terrorist actions, using computational linguistics and machine learning techniques. The system should become an effective tool for preventing terrorist threats in the digital space by timely detection of potentially dangerous content. To achieve this goal, it is necessary to implement the following tasks:
To analyse the available approaches to the automatic detection of terrorist content, in particular using linguistic methods, artificial intelligence and filtering systems.
Collect and form a thematic corpus of texts that may contain terrorist vocabulary as examples for training and testing models.
Highlight linguistic features and patterns of speech characteristic of terrorist calls, including keywords, grammatical constructions, emotional colouring, and context of use.
Select and adapt natural language processing (NLP) methods for text analysis, including tokenisation, stemming/lemmatisation, and syntactic and semantic analysis.
Build a classification model (e.g., based on logistic regression, Naive Bayes, or neural networks) to automatically recognise dangerous text.
Train the model on the selected data set, optimise the parameters of the model and evaluate its effectiveness using appropriate metrics (accuracy, completeness, F1-measure).
Develop a user interface or prototype of the system that will allow you to analyse the text interactively and get the classification result.
Analyse the results of the system, identify strengths and weaknesses, as well as prospects for its improvement and scaling.
The implementation of these tasks will create an effective application system that is important for information security, state control over the digital space and preventive response to potential terrorist threats.
The object of research in this project is digital communication concentrated in the online environment, in particular, text messages distributed on social networks, instant messengers, comments on publications, forums and other open information platforms. In the context of the growing digitalisation of society and global information exchange, these sources have become a key medium for communication between people, including representatives of radical groups. The object of research covers large volumes of text information (Big Data) generated by users in the public domain. It contains a variety of styles, languages, topics, and levels of aggression. These messages, on the one hand, are a product of mass communication and, on the other hand, a potential indicator of threats to public security, particularly in the form of calls for terrorist actions and other malicious calls.
The subject of the study is those linguistic, semantic and structural features of texts that may indicate the presence or risk of terrorist rhetoric. The subject area includes vocabulary units, grammatical constructions, contextual markers, stylistic devices and rhetorical figures used in speech with calls for violence or radical action. In particular, these can be direct or veiled calls to action, metaphorical formulations, the use of symbols, euphemisms or coded references. In addition, the subject of the study covers methods and tools of Automated Natural Language Processing (NLP) that allow such elements to be recognised in text. It includes, among others, tokenisation, lemmatisation, POS analysis, named entity discovery, thematic modelling, contextual analysis, and more.
Particular attention within the subject is paid to algorithms for classifying texts - both traditional (for example, naïve Bayesian classifier, logistic regression, decision trees) and modern deep learning methods (neural networks, transformers, models of the BERT type). These algorithms allow you to build systems capable of identifying threatening content with high accuracy. The subject is also the process of forming your own thematic corpus - a set of texts that can be used to train, test and validate such models. It makes it possible to create a flexible and relevant tool for Ukrainian realities.
Within the framework of this study, a new approach to detecting texts with calls for terrorist actions is proposed by combining traditional methods of computational linguistics with modern machine learning technologies, which allows for a deeper and contextually accurate analysis of textual content. Personally, the author has developed for the first time the structure of a hybrid system that combines language patterns, semantic rules and classification algorithms based on NLP models adapted to Ukrainian-language and multilingual online content. The difference between the proposed solution and the previously known ones is the use of multilevel analysis: the system not only detects keywords, but also evaluates the emotional colouring, syntactic connections between words, context within the sentence and the message as a whole. Thanks to this, it is possible to detect non-obvious or veiled forms of terrorist calls, which usually remain outside the boundaries of conventional filtration systems. The novelty is also the creation of an experimental thematic corpus of Ukrainian and Russian-language messages with risk labels, which can be used in further research in the field of countering information threats. The proposed approach has been further developed in the direction of personalising content filtering, that is, taking into account the platform, type of user or distribution channel. Thus, the results of the study are of applied value for the development of new security information systems and contribute to the development of the scientific basis for the detection of destructive content in the digital environment.
II.
The task of automatic detection of calls for violence and extremist/toxic content in the text
combines several related areas: hate/toxicity detection, extremism / radical content detection, and
call-to-action detection (calls to violence). Traditional approaches (dictionary, SVM on TF–IDF)
have long been complemented by modern transformer models (BERT and its derivatives), with the
most effective systems being hybrid, combining rule-based pre-filters and fine-tuned transformers
and meta-escalation solutions. This trend is confirmed by both review papers and applied research
[
Rule-based (dictionaries, regex, fuzzy matching) – fast and interpreted; They are good at catching "obvious" triggers, but have many false positives and are vulnerable to obfuscations.
Classical ML (TF–IDF + SVM/LogReg/RF) – simplicity and efficiency with limited
resources, but weakness in understanding semantics and contexts [
Deep learning (LSTM/CNN and their hybrids) are the best opportunity for
longterm dependencies, but is still inferior to transformers in many NLP benchmarks [
Transformer-based (BERT, RoBERTa, AraBERT, etc.) – the current state-of-the-art in semantic problems; Fine-tuning on blast furnace buildings gives significant gains.
Transformers work well with subword (WordPiece) – more resistant to OOV/leet [
Public benchmarks (HateXplain, Jigsaw/Perspective, HASOC, etc.) provide standardised
conditions for comparison. Still, there is a shortage of high-quality, lớn-scale annotated corpora
specifically for "calls-to-violence" – so researchers often create custom corpora from
messengers/forums or adapt related sets. It imposes restrictions on direct comparison of results in
the literature [
In production, not only are classification metrics (Precision/Recall/F1) important, but also
latency (response time), explainability (explanation of decisions for the moderator), and audit trail.
Industrial APIs (Perspective) focus on low-latency and broad interoperability, and scientific papers
emphasise the importance of rationale-based training for explainability (HateXplain) [
All modern works confirm: a hybrid pipeline (rule-based prefilter, BERT fine-tune and metadecision / thresholds) gives the best compromise between recall (do not miss a real call), precision (minimise false positives) and practical performance. Your approach belongs to this class, and the results you collect (accuracy ≈ 0.93, Precision ≈ 0.91, Recall ≈ 0.90, F1 ≈ 0.91; latency mean ≈ 0.54s per message) correlate with better industrial and academic systems in related problems.
The table shows those metrics that are explicitly indicated in the source articles or publications (citations are given in the "Source" column). If no latency is specified in the original, the field remains empty. 85% multi- BERT ideology text classical dataset baselines / (varies) – work releases dataset & baselines – – –
Official benchmark results (MultiNLI, SQuAD) are given for BERT – this work provides a
technical basis for all fine-tuned transformer solutions [
Fine-tuning of the domain/multilingual version of BERT (or AraBERT for Arabic)
increases the metrics for working with extremism [
Hybrid pipeline – rule-based for instant escalation + neural network for semantic analysis – turned out to be the most stable approach in the literature and in your research.
Latency/throughput monitoring – if the system needs to work in real time, leave a margin for increased load (batching, distillation/knowledge distillation for faster models).
Based on the analysis, it is possible to identify both the strengths of existing systems and their
limitations, which directly affect the effectiveness of detecting terrorist calls in the digital
environment [
Among the main advantages of modern approaches, it is worth noting the ability of some
systems to work in real time, the widespread use of deep learning methods, taking into account the
context of the text, as well as the ability to scale to large amounts of data [
At the same time, almost all of the analysed solutions have significant drawbacks that limit their application in real conditions. Firstly, most systems are focused exclusively on English-language or Arabic-language content and do not take into account the linguistic features of the Ukrainian language [25-30]. Second, existing systems often use only superficial text analysis, which makes them vulnerable to synonymization, euphemisms, sarcasm, or code formulations [31-36]. Thirdly, the vast majority of systems do not take into account the dynamics of message distribution, user behaviour, or time patterns, which can be critical in detecting threatening actions [77-42].
In view of the above, a key problem can be formulated - in the context of the information hybrid war, the spread of radical content through digital platforms is growing. Still, the available detection tools are limited in the context of multilingualism, flexibility, adaptability, and contextual analysis. Especially critical is the lack of effective systems for the Ukrainian-speaking space, which are capable of detecting not only direct, but also covert or veiled calls for terrorist actions.
Thus, there is a need to create an adaptive intelligent system that would combine methods of computational linguistics, machine learning algorithms, analysis of user behavioural characteristics, and support multilingual content. This system should be able not only to classify messages as potentially dangerous, but also to explain its decision, as well as support the ability to scale and further learn from new data. Its implementation will significantly increase the level of information security and the efficiency of analytical centres and government agencies in the field of countering cyberterrorism.
III.
Methods and tools
In the current conditions of global digitalisation, information technologies penetrate deeply into all spheres of life, ensuring a rapid growth in the volume of data generated by users in the online space every day. Social networks, instant messengers, forums, blogs and news sites have become not only platforms for the exchange of opinions, but also channels through which destructive information can be disseminated. Publications with signs of extremism, propaganda of violence or terrorist calls pose a particular threat. Several factors complicate the problem of detecting such messages. First, the total amount of digital content is so large that manual analysis of all messages is technically impossible. Second, attackers often use euphemisms, substitution of words with synonyms, grammatical distortions, sarcasm, or even code vocabulary to hide the true meaning. Thirdly, messages can be published in different languages, which requires multilingual support from the analysis system.
In view of the above, there is an urgent need to create intelligent information systems that are capable of carrying out automatic monitoring and classification of texts, detecting those that contain explicit or implicit calls for terrorist actions. Such systems should use modern achievements in computational linguistics, artificial intelligence, machine learning, and semantic analysis. In this context, the object of research is textual information published in an open digital environment, such as comments on social networks, forum posts, or messages in messengers. The subject of the study is methods for automated detection of terrorist content in texts, in particular those based on transformer models, natural language processing algorithms, and classification approaches in machine learning.
For the formal structuring of the logic of building the future system and determining the criteria for its effectiveness, the general goal has been formulated, which is as follows: the creation of an adaptive intelligent system for the automatic detection of texts with calls for terrorist actions with a high level of accuracy, flexibility and explainability of the decisions made. This goal is quite general, so the goal tree method is used to specify it. The goal tree allows you to decompose a complex goal into components (aspects), each of which is further detailed in the form of sub-goals, requirements or quality criteria. It allows us not only to describe the system in a structured way but also to evaluate it in the process of implementation. The tree of goals for detecting terrorist calls has the central vertices:
Text analysis → Accuracy of classification →Completeness of classification → → Display of results → Explanatory results .
At the first level of the tree, two main aspects of the general objective are defined, which are text analysis and display of results. Text analysis involves achieving criteria such as accuracy and completeness of classification, that is, how correctly the system determines the class of the message and whether it misses potentially dangerous texts. The display of results should provide explainability, i.e. the ability of the model to form a rationale for its decision that is understandable to the user, for example, based on highlighted trigger words or semantic patterns. In the report, this tree is presented in the form of a structured flowchart, which clearly demonstrates the relationship between the goal, aspects and criteria for achieving it.
To achieve this goal, several alternative options for building a system are considered: a rulebased approach based on filtering by keywords; classic ML classifiers, for example, based on SVM or Naive Bayes; deep learning with transformers, including BERT; and a hybrid system that combines vocabulary processing with deep classification. Each of these options has its own advantages and disadvantages. The rule-based system is easy to implement, but easy to bypass. Classical machine learning approaches provide a more accurate classification, but require manual selection of features and a high-quality dataset. Transformer models demonstrate the highest accuracy and ability to take into account the context, but require significant computing resources. In turn, the hybrid architecture allows you to combine the advantages of all approaches - the speed of rule-based filtering and the depth of analysis of models based on BERT.
Taking into account the advantages and limitations, it was decided to implement a hybrid system that is able to scale effectively, provides high accuracy and explainability, and also allows you to adapt to changes in the style of writing terrorist content. Thus, the system analysis made it possible to formulate a justified goal of creating a system, to detail its components using a tree of goals, as well as to determine the most effective approach to its implementation, based on the conditions of the task and the limitations of real application.
After the system analysis and the definition of the general goal of the project – the creation of an adaptive intelligent system for detecting terrorist calls in text content – the next critical stage is to specify its functioning. This stage involves detailing the internal logic of the system, formalising its modules, data processing sequences, and determining the points of interaction between components and external actors. Within the chosen hybrid architecture, the system must combine efficient rule-based pre-filtering with high-precision depth analysis based on transformer neural models. The primary process of detecting terrorist appeals:
User → Entertext → Analyze message →View explanation →[ Moderator ]→ → Analyze the presence of a call detection error → thˇe results of th analysis →
→[ If necessary retraining ]→ Display th result →User
After formalising the general architecture of the information system and presenting its logic in the form of data flow diagrams (DFDs), it is expedient and necessary to further complicate and detail the functional structure in the form of a hierarchy of processes (or functions and tasks). This step allows you to present the system as a set of interconnected functional components placed in the form of a tree according to the principles of decomposition: from the highest level (general function) to basic elementary actions that are implemented in the code at the level of individual modules or functions. It is this structural model that underlies the system approach to the design of complex information systems and is widely used both in scientific research and in practical software engineering.
A process hierarchy is a logical structure that allows an engineer, analyst or developer to visualise a complete picture of the functional content of the system, dividing it into subsystems, functional blocks, and then into specific tasks. Building such a hierarchy is not only a formality, but also serves as a basis for determining the logic of code implementation, distributing responsibilities between components, planning testing, justifying resource requirements, and assessing the complexity of the project as a whole.
At the heart of the construction of the hierarchy of processes of the terrorist call detection system is the primary function - the automated detection of messages that contain potentially dangerous calls for violence, terrorism or extremist activities. This function is key in the system and covers the entire life cycle of text processing - from the moment of its introduction to the final solution, as well as the accompanying procedures for explaining, logging and adapting the system in the face of changes.
At the first level of the hierarchy, this general function is divided into five main processes: primary text processing, rule-based filtering, deep semantic classification, the formation of an explanation of the classification decision and the storage of the results in the system storage. In addition, a separate branch dedicated to feedback from the user and moderator stands out, which allows you to improve the model through interactive retraining.
Each of these processes is revealed in more detail at the next level. For example, the primary text processing module performs operations such as clearing text from HTML markup and unnecessary characters, tokenisation (splitting into words), normalisation (lowercase), deleting stop words (pronouns, prepositions, etc.), and lemmatisation or stemming (reducing words to the main form). This process is critically important, since the accuracy of further semantic processing depends on its quality.
The rule-based filtering process implements the first rapid detection of potentially dangerous content by matching it with a database of trigger words, regular expressions, or patterns. Its purpose is to weed out clearly safe messages and transmit really suspicious texts for deeper analysis. Next, the semantic classification module uses a transformer model (for example, BERT), which is able to take into account both the meaning of individual words and the context of the entire message. As a result, the model forms a probabilistic decision - how likely it is that a given message contains a terrorist call.
No less important is the function of explaining the classification. It allows the user or moderator to understand why the system made such a decision. Explanations can be formed through visualisation of the importance of individual tokens (attention mechanisms), comparison with patterns, and the use of interpreted machine learning methods such as LIME or SHAP.
The process ends with logging and saving the results. The system records each analysed text along with the classification result, date,probability, explanation, and other metadata. It allows you to save history for analytics, quality control, training new models, and reporting.
A separate branch of the hierarchy is a feedback block, which allows users to report erroneous classifications. In case of accumulation of a sufficient number of complaints or the emergence of new types of disinformation, the system may be retrained. This process is performed by a
In today's information environment, the role of open digital platforms - such as social networks, instant messengers, forums and news feeds - has increased significantly. These tools play an essential role in the dissemination of knowledge, community organisation and communication, but at the same time pose a significant threat as channels for the spread of radical ideologies, disinformation campaigns and, in particular, calls for terrorist actions. In the context of globalisation, hybrid wars and the growing activity of hostile information influences, the problem of detecting terrorist calls in open sources has become highly relevant. Significant volumes of text messages, multilingualism, the use of disguised language, sarcasm, or encryption of threats in metaphors create additional difficulties for manual monitoring and require automation of analysis processes.
Against the backdrop of a growing threat, there is a need to create an intelligent information system capable of automatically analysing textual content and determining whether it contains potential signs of terrorist calls. Such a system should provide support for decision-making based on natural language processing algorithms, semantic analysis, machine learning, and explainable classification. Its implementation will allow a timely response to dangerous publications, reduce the workload on analysts, and also contribute to improving security in the digital space.
The purpose of the system is to detect dangerous textual content that may contain terrorist appeals through comprehensive linguistic and semantic analysis. The system should be integrated into the interfaces of online platforms, use modern deep learning models (including transformers such as BERT), explain its decisions to users and moderators, and learn based on feedback. It can be used in security services, content moderation centres, analytical institutions, and IT companies engaged in cyber defence.
Upon input, the system receives text messages in plain text format, as well as, if necessary, accompanying metadata - language, source, and date of creation. The initial data are the result of the classification (whether there is a terrorist call in the text), the explanation of this result, the level of probability, the audit log files and the classification database. It is possible to transfer the result to the moderator for confirmation, as well as to initiate retraining of the model based on user complaints.
The intelligent component of the system is based on a combination of rule-based filtering (based on dictionaries, patterns, and regular expressions) and deep semantic analysis. The latter is implemented using the BERT model and trained on the corresponding case of dangerous and safe messages. Additionally, a classification explanation module is built in, which uses attention mechanisms or external tools for interpreting models (for example, SHAP or LIME), which allows you to understand why the model made a particular decision.
The system implements a number of business processes: text input, pre-processing, rule-based filtering, deepclassification, explanation generation, logging of results, and feedback processing. In case of an erroneous classification, the user can mark the message as incorrectly recognised, which is automatically transmitted to the moderator. Once the error is confirmed, the case is added to a new training sample to retrain the model. This mechanism ensures the adaptability of the system to new challenges, linguistic patterns and manipulative techniques.
From a technical point of view, the system is designed as a client-server architecture with a REST API for integration into third-party services. On the server side, computing modules are implemented: classifier, filter, explainer, database and logging module. The user interface provides the ability to enter messages, view results, and provide feedback. The architecture allows scaling, horizontal expansion, classification history preservation, and analytics.
The expected effects of the implementation of such a system are: reducing the human burden on moderation, increasing the efficiency of threat detection, reducing the number of erroneous decisions, increasing user trust in digital platforms, and integrating analytical information into the work of law enforcement agencies or cyber defence organisations. Thus, the development and implementation of this system is justified from the point of view of social benefit, technical feasibility and strategic importance in the conditions of information warfare.
Taking into account the results of the system analysis, the requirements for the system are defined: the ability to work with unstructured text, ensuring high accuracy of classification, the availability of explanations of solutions, the ability to preserve history and support for retraining. On this basis, a conceptual model has been formed that combines modules of linguistic processing, neural network classification, interpretation, logging and administration, which together form a single adaptive, scalable and secure information system.
In today's digital environment, where a significant part of communications is carried out through text messages on social networks, instant messengers, forums and news feeds, the issue of information security is becoming especially relevant. One of the most dangerous forms of threats is the spread of calls for terrorist actions, which can have direct or veiled wording, be formulated in the form of hints, metaphors or code phrases, which makes it much more difficult to detect them by traditional methods. The growth of information traffic, multilingual content and limited human resources necessitate the creation of automated systems capable of effectively detecting such messages in real time.
The primary function of the intelligent information system under development is to detect terrorist calls in the text environment. Such a system should analyse the text, identify potentially dangerous content, classify messages according to the level of risk, generate explanations of the results of the analysis and provide the possibility of adaptation based on feedback. To implement this functionality, it is necessary to choose appropriate methods of knowledge presentation, logical inference, decision-making algorithms and proper software tools.
2. Rule-based filtering. A dictionary-template approach is used:
E (T )={⃗e1 , ⃗e2 , … , ⃗en }, ⃗ei∈ Rd.
R (T )={1 , if ∃ wi∈ T : wi∈ Dt e rr o r
0 , ot h erwise where Dt e rr o r is a set of key trigger words/patterns. This module performs fast pre-screening. 3. Deep classification (BERT + Softmax). The CLS vector from the BERT is fed to the full-bonded layer:
z=W ⋅ ⃗eCLS +b.
P ( y =k|T )= K exp ( zk ) , k ∈ {0,1 } ,
Let us describe the developed information technology based on mathematical formulas in order to formalise its operation. Since the system combines rule-based filtering and deep classification (BERT), the model can be represented as follows: 1. Presentation of the text. Each T message is converted into a vector representation:
T ={w1 , w2 , … , wn }, where wi – is message tokens. The BERT model maps them into contextual vectors: αij= n
exp (eij) ∑ exp (eik ) k=1
, ∑ exp ( z j) j=1 where y =1 – the message contains a terrorist call. 4. Final decision. The final decision is made on the basis of a combination of both modules:
F (T )=α ⋅ R (T )+(1−α )⋅ P ( y =1|T ),
where α ∈ [
TP +TN + FP + FN TP + FN Precision+ Recall 6. Explainability. The attention mechanisms in BERT allow you to calculate the weights of the importance of words:
where αij – is the contribution of the word w j to the representation of the word wi. These values are applied to visualise the most critical tokens when explaining a decision.
Thus, information technology is formalised as a hybrid mathematical model that combines rulebased rules (logical functions) and probabilistic classification based on BERT (softmax model). It allows you to effectively detect both direct and covert calls for terrorist actions. In the course of the work, it is considered which approaches to the presentation of knowledge are relevant for this task, what logical inference mechanisms can be effectively used, what classification algorithms are advisable to use, and which software implementation tools provide the highest performance and flexibility in creating such a system. A comparative analysis of alternative approaches will also be (1) (2) (3) (4) (5) (6) (8) carried out, on the basis of which the choice of the optimal combination of methods and technologies for solving the task will be substantiated.
In decision support systems (DSS) focused on the analysis of text content, a special role is played by the way knowledge is presented, since it forms the basis for further data processing, logical inference and generation of decisions. The presentation of knowledge in such systems can be implemented by various methods - from simple dictionary structures to complex models of semantics representation, depending on the level of complexity of the task, the volume of input data and the required degree of flexibility and adaptability.
Within the framework of this project, a combined approach to knowledge presentation has been chosen, which combines rule-based methods (rules built on the basis of keywords, phrases, and regular expressions) and statistical representation of information implemented using deep learning models, in particular, transformers. The rule-based approach allows you to provide a basic level of filtering based on predefined trigger tokens, patterns, or patterns that characterise terrorist calls in an explicit form. At the same time, it is limited in detecting complex, veiled or atypical wording that is actively used to bypass filters. To overcome these limitations, the system uses knowledge representation based on semantic vectors obtained using the pre-trained transformer model BERT (Bidirectional Encoder Representations from Transformers). In this approach, each message is encoded as a multidimensional vector representation that preserves the contextual meaning of the words in the sentence. This form of knowledge presentation allows you to identify semantic connections and hidden dependencies between words, as well as recognise synonymy, sarcasm, irony, and other forms of manipulative presentation. Thus, the chosen strategy of knowledge presentation combines explicit (declarative) knowledge in the form of rules and implicit (procedural) knowledge in the form of weighted connections in a neural network. It allows you to implement an adaptive system that is capable not only of accurate recognition of known patterns of terrorist rhetoric, but also of detecting new dangerous trends in the text environment based on contextual analysis. This hybrid approach to knowledge presentation provides flexibility, scalability, and high-quality classification while maintaining explainability for the end user.
In the context of the development of an intelligent system for detecting terrorist calls, logical inference mechanisms are a critical component that ensures that an informed decision is made based on the input data provided. They determine precisely how the system transforms the incoming text information into a classification result - that is, in response to the question of whether the message contains dangerous content.
Within the framework of the proposed hybrid architecture, two main types of logical inference are used: rule-based inference and machine learning-based inference. Each of the approaches plays a specific role in the overall structure of the treatment.
Deductive inference is implemented through a set of formalised rules based on keywords, phrases, contextual patterns, and regular expressions. For example, the presence in the text of phrases such as "blow up", "call to arms", "explosives", and "eliminate" can unambiguously identify the message as potentially dangerous. The system scans the incoming text for compliance with these rules and, if triggered, generates a warning about a possible terrorist call. This level of logic is fast and effective at detecting obvious threats, but vulnerable to workarounds or veiled statements.
Inductive logical inference, in turn, is implemented using a deep neural network based on the BERT model, which has been trained on a labelled corpus of texts. During training, the model automatically forms internal rules (based on context analysis, grammar, and semantics), which it subsequently applies to classify new messages. In the process of inference, the model compares the context of the message with previously learned patterns and makes decisions with the appropriate level of probability. In contrast to rigid rules, this approach allows the model to identify non-trivial forms of threat expression, including sarcasm, euphemisms, ambiguity, and metaphors.
Special attention should be paid to explainability mechanisms, which are implemented through additional tools such as SHAP (SHapley Additive exPlanations) or LIME (Local Interpretable Model-agnostic Explanations). They allow the user or moderator to understand which words or phrases influenced the classification decision. Not only does this increase the credibility of the system, but it also allows you to identify false positives or algorithm abuses.
In general, the combination of formal (deductive) and educational (inductive) inference, complemented by explanation modules, allows for high flexibility, adaptability and efficiency in detecting terrorist calls. This approach meets the requirements of modern cybersecurity and is resistant to language transformations, stylistic variations, and bypass communication techniques.
In the terrorist call detection system, one of the key components is practical decision-making algorithms that ensure accurate recognition of dangerous content in text messages. Taking into account the complexity of the subject area, the ambiguity of natural language, linguistic manipulations and a large amount of input data, it was decided to implement a two-level classification architecture that combines rule-based filtering with deep semantic analysis based on transformer models, in particular BERT.
Rule-based filtering plays the role of the initial stage of message processing. This mechanism is based on predefined dictionaries, lists of dangerous words, triggers, phrases and patterns that are considered typical of terrorist rhetoric or calls for violence. In addition, regular expressions are used to identify non-standard or disguised constructs. The main advantages of this approach are its speed, transparency, and ease of configuration, scaled by updating the rule base without the need for a complete retraining of the system. However, rule-based filtering is not able to effectively work with atypical or veiled expressions, does not understand the context and does not distinguish the meaning of phrases depending on the situation. Because of this, the rule-based module acts as a primary filter, which either immediately filters out clearly threatening messages or passes them to the next level of processing.
Deep classification is implemented using the BERT model – an advanced architecture based on attention mechanisms. It allows the model to "read" text simultaneously in both directions, forming an understanding of the context of words. In particular, messages that contain veiled calls for violence, metaphorical threats, sarcasm or manipulative rhetorical devices. This approach allows the model not only to recognise obvious cases, but also to identify hidden signs of danger that are not always noticeable even to humans.
The choice of BERT among other machine learning architectures was due to its high performance in a wide range of NLP tasks, including classification, question answering, sentiment analysis, and identification of named entities. In addition, transformers work better with long texts than classic models such as LSTM or CNN. They are independent of word order and are capable of establishing remote dependencies between words. It is critical in detecting terrorist calls, as threats can be vague, implicitly formulated, or hidden in complex syntactic constructs.
Special attention is paid to the possibility of explaining the results of the classification. It is an essential requirement, not only from an ethical point of view, but also to increase the credibility of the system, especially if its results are to be used in security services or in administrative decisionmaking. The system implements two main approaches to explainability: attention-visualisation and the use of SHAP. The attention mechanism built into BERT allows for the highlighting of the words or phrases that have most influenced the model's decision, thus visualising the decisionmaking process. SHAP, in turn, proposes a local explanation method that is based on game theory and allows you to calculate the contribution of each feature (in this case, a word) to the final result. It enables analysts and moderators to better understand why a particular message is classified as unsafe, and it also makes it easier to identify false positives or new linguistic patterns.
Thus, the set of algorithms - rule-based filtering and transformer classification - provides a high level of accuracy, contextual understanding and transparency. This approach makes it possible to combine the speed and simplicity of classical methods with the intellectual depth and flexibility of modern deep models. It is critically important in an environment where system decisions can have real consequences for public security and human rights.
In the development of a system for automatic detection of terrorist calls in text content, the choice of implementation tools plays a key role, since they determine the performance, scalability, flexibility and convenience of further maintenance of the software. Python was chosen as the primary programming language, and it is the de facto standard in the fields of natural language processing (NLP) and artificial intelligence (AI). Python provides a robust ecosystem of libraries, syntax simplicity, a broad community, and the ability to quickly integrate with other platforms. In addition, this language is ideal for experimental prototyping, which is essential in the process of exploring new models and methods. The key libraries for implementing NLP and machine learning modules are HuggingFace Transformers, SpaCy, and Scikit-learn. HuggingFace's Transformers library provides access to state-of-the-art pre-trained models, including BERT, RoBERTa, DistilBERT, GPT, and more. It allows you to easily implement deep learning models, change configurations, train them on your own dataset, and integrate them with other system components. It is this library that is used to implement the main module of deep classification, which detects terrorist calls based on contextual analysis.
The SpaCy library is used for text preprocessing. It provides fast and efficient tokenisation, lemmatisation, definition of parts of speech, and Named Entity Recognition. It allows you to build a lightweight yet robust word processing pipeline before passing it to the classifier. SpaCy integrates well with other libraries and also allows you to create your own processing components for specific languages or domains.
To implement auxiliary classical machine learning algorithms, in particular, rule-based filters or basic classifiers, the system uses the Scikit-learn library. It provides modules for building SVM, Naive Bayes, decision trees, clustering, data normalisation, and model quality assessment. In our case, Scikit-learn is also helpful in analysing results, cross-validating, constructing accuracy, completeness, and F1 measure metrics.
Flask or Streamlit is used to implement the user interface and provide interactive interaction with the system. Flask is a classic web framework that allows you to build a REST API used as an entry point for integration with other web services or platforms. It is an ideal choice for a serverside implementation that needs to be scalable and independent. Streamlit, on the other hand, allows you to quickly prototype a user interface without in-depth knowledge of web development. Streamlit efficiently implements input fields, buttons, graphs, explainability visualisation (for example, highlighting essential words) and adjusting model parameters.
When comparing TensorFlow and PyTorch – the two main deep learning frameworks – PyTorch was preferred. The main reasons for this choice are better support in the Transformers library, more developer-friendly syntax, dynamic compute graphics (which makes it easier to debug models), a broad community, and stable integration with HuggingFace. PyTorch provides more flexibility when experimenting, in particular when working with custom loss functions or non-standard architectures. While TensorFlow has powerful tools for productive use of models in production, such as TensorFlow Serving or TensorFlow Lite, it is less convenient in the R&D phase.
Thus, the selected tools - Python, HuggingFace Transformers, SpaCy, Scikit-learn, Flask/Streamlit and PyTorch - provide an optimal balance between performance, flexibility, ease of implementation, and support for explainable AI. They make it possible to implement all the functionality of the system - from text processing and deep analysis to output of results and interactive feedback - with a high level of control over the process, rapid prototyping and scalability.
In the field of automatic detection of terrorist calls in text content, a number of approaches have been proposed in recent years, each of which has its own advantages and limitations. Among the most common, rule-based systems, models based on convolutional neural networks (CNN) and recurrent neural networks (LSTM) are worth mentioning. Each of these methods deserves attention, but their comparative analysis allows us to clearly justify the choice of a hybrid model that combines rule-based filtering with transformer deep classification.
The rule-based approach is based on predefined rules, keyword lists, regular expressions, and linguistic patterns. Its main advantages are ease of implementation, transparency, and explainability. Such systems quickly respond to new rules and easily adapt to the requirements of moderators. At the same time, their main disadvantage is low flexibility: rule-based methods are not able to detect synonymy, sarcasm, grammatical variations, slang, encryption or contextual ambiguities. In addition, they do not scale when working with large amounts of text or with multilingual content. These shortcomings make rule-based systems insufficient as an independent solution in security tasks.
Another class of models - CNN (Convolutional Neural Networks) - although it has demonstrated high efficiency in computer vision tasks, has certain limitations in the field of NLP. CNN models can capture local patterns and phrases well, but they cannot model long-term dependencies in text. It is especially critical for the tasks of classifying terrorist calls, which are often disguised in complex syntactic constructions or stretched over several sentences. Compared to transformers, CNN models lose flexibility in processing context and explaining decisions.
LSTM (Long Short-Term Memory) is a type of recurrent neural network that has long been the standard in sequential word processing tasks. LSTMs work well with sequential language structures and are able to model a certain amount of context, making them better than CNNs for longer pieces of text. However, LSTMs are slower to learn, less scalable, have a complex architecture, and, importantly, are inferior to modern transformers in accuracy and stability of results. In addition, LSTMs do not have a natural explanation mechanism, which makes them difficult to use in sensitive areas related to safety.
The hybrid model chosen in the work combines the strengths of all these approaches. Rulebased filtering serves as the first protective layer, allowing you to quickly filter obviously dangerous content using linguistic templates. This approach also allows the user to configure rules manually in response to new threats. The second layer, Deep Classification Based on the Transformer Model (BERT), provides accurate and contextually sensitive text analysis, including the ability to detect veiled calls, synonymy, and multilingualism. This approach also supports explainability through attention mechanisms, which allows you to explain why a particular decision was made, which is critical in automated moderation tasks.
Thus, the hybrid model combines the advantages of both worlds: the transparency and simplicity of the rule-based approach with the power and contextual sensitivity of transformers. Such a symbiosis allows you to achieve high accuracy, scalability, adaptability to new threats and reliability, which is especially important in information security.
We will describe in more detail the main stages of our pipeline for detecting terrorist calls in social networks. At the same time, we will compare the features of each of the stages for the analysis of English-language (T = I call to take up arms and fig h t ) and Ukrainianlanguage (T =Закликаю взяти зброю і боротися) texts to identify terrorist calls.
Stage 1. Presentation of the text. The purpose of this stage is to formalise the T text message in the form of mathematical objects (vectors, matrices) suitable for further processing by machine learning algorithms.
1.1. Tokenisation (individual tokens / WordPiece). Let the incoming text of the message: T = I call to take up arms and fig h t (engl.) and T =Закликаю взяти зброю і боротися (ukr.).
Then, after the classic tokenisation (whitespace/word), we get a set of words (tokens): T → {w1 , … , w8 }={I , call , ¿ , take , up , arms ,∧, fig h t }. (engl.)
and T ={w1 , w2 , w3 , w4 , w5 } (ukr.), where w1=Закликаю, w2=взяти, w3= зброю, w4=і, w5=боротися.
In the BERT style, subword (WordPiece) is often used, for example:
\text{"arms"} \to [\text{"arm"},\ \text{"##s"}],\quad \text{"fight"} \to [\text{"fight"}].
Then the token sequence can get a little longer; Let's denote the number of tokens as N . 1.2. Normalisation and lemmatisation / stemming. Each word is reduced to a basic form (lemma):
L (T )={l1 , … , lm }={I , call , ¿ , take , up , arm ,∧, fig h t }. (engl.) (here "arms" → "arm"). Frequent operations: lowercasing (all lowercase letters), punctuation removal, apostrophe normalisation, etc.
L (T )={l1 , l2 , l3 , l4 , l5 }, (ukr.) where l1= закликати, l2=взяти, l3= зброя, l4=і, l5=боротися.
1.3. Vectorization (Bag-of-Words/BoW and TF-IDF). Let the corpus have N documents and a large vocabulary base with M words (lemmas). Let's build a TF-IDF vector for a T document: ⃗v (T )=(tfidf (l1 , T ) , tfidf (l2 , T ) , … , tfidf (lM , T ))∈ R M, where TF-IDF is calculated as:
tfidf (li , T )=tf (li , T )⋅ idf (li), tf (li , T )= documents where li – occurs.
Small numerical English example. Suppose:
f li ,T ∑ f l j ,T j , idf (li)=log N .
1+nli (9) (10) f li ,T – is the number of occurrences of the word li in the text of T , nli – is the number of case N =10,000 documents; The word "arms" (lemma "arm") occurs in narm=50 documents; The word "fight" occurs innfig ht=200 documents; In our message, the total number of tokens is ¿ 8, and each of the lems appears 1 time (frequency f =1).
Then And for fight: tf ( arm , T )= 1 =0.125 , idf ( arm )=log 10000 ≈ log (196.08) ≈ 5.28, 8 1+50
tfidf ( arm , T )=0.125⋅ 5.28 ≈ 0.66. tf ( fight , T )=0.125 , idf ( fight )=log 10000 ≈ log ( 49.75) ≈ 3.91,
1+200 tfidf ( fight , T ) ≈ 0.125⋅ 3.91 ≈ 0.489.
Other words that are frequently encountered (I, to, and) will have small idf and correspondingly small TF-IDFs.
Ukrainian-speaking example. If there are N =1000 documents in our corpus, the word "зброя" occurs in 20 of them (nзброя=20), and in our text it appears 1 time, then: 1.4. Word Embeddings (BERT).
1.4.1 Tokens → vectors. In deep models such as BERT, each word/token is wi mapped to a context vector:
⃗ei=f ( wi)∈ Rd, ⃗ei=BERT ( wi∣ context T )∈ Rd , d type=768. where d is the dimension of the space (for example, d =768 in BERT).
So, for the entire T message/text, we have a matrix:
E (T )=[ e2 ]∈ Rn×d.
⃗e⊤1 ⃗⊤ ... ⃗e⊤n (11) (12) 1.4.2 CLS token and pooling. BERT adds a custom token [ CLS] to the beginning; its vector ⃗eCLS – is often used as a representation of the entire sentence: ⃗sCLS=⃗eCLS∈ Rd. Alternatively, you can do mean-pooling by tokens:
1 n (13) ⃗smean= n ∑i=1 ⃗ei.
1.4.3 Small Numerical Illustrative English Example. Let's assume d =3, for clarity, and we have eight tokens. Contextual vectors:
E (T )=
This matrix is fed into the further classification module. Thus, at the stage of presenting the text of the message, the following steps are taken:
1.5 Subword and OOV (out-of-vocabulary). If a word is broken down into a subword (e.g. an unusual slang word), the embedding of each subword is calculated, then they are aggregated (concat/mean/attention-pooling). Formally, if the token w → subwords {s1 , … , sk } with embeddings ⃗uj, then ⃗ew= 1.6 Additional features (feature engineering). For classic models, numerical features are added:
Message length len ( T )=n, Number of trigger words coun t trig (T )=∑ 1 [ li∈ Dtri g ],
i Part of speech (POS) in the form of one-hot vectors, N-grams (bigrams): for example, "take up" and "up arms" → can be represented as binary features.
Let's denote the vector of additional features as ⃗faux∈ R p. Then the final input to the light classifier can be
Tokenization T → {w1 , … , wn }; Lemmatization wi → li; Vectorisation (TF-IDF or embeddings); Feature matrix E (T )∈ Rn×d.
⃗x=[ ⃗s ; ⃗faux ]∈ Rd+ p, where [⋅ ;⋅ ] – concatenation. 1.7 Final formalisation of the presentation: (14) (15) Tokenisation: T → {wi }in=1.
Lemmatization: wi → li.
TF-IDF vector: ⃗v (T )∈ RM (for classical models).
Contextual embeddings: E (T )∈ Rn×d, sentence → ⃗s∈ Rd (CLS or pooling).
Additional features: ⃗faux.
The final vector for the classifier is⃗:x=[ ⃗s ; ⃗v (T ) ; ⃗faux ].
Stop Word Sensitive. Words like "I", "to", "and" will have a small IDF - their impact is negligible in TF-IDF, but in BERT, they are essential as context.
Load on subwords. Aggressive filter bypasses often use unusual words → WordPiece gives stability.
Character normalisation – digit/symbol substitution, transliteration (russian/Ukrainian → English) may be required for a multilingual corpus.
Feature scaling – before combining TF-IDF and embeddings, you need to specify scales (e.g. vector normalisation).
Step 2. Rule-based filtering is an intermediate step that works before (or in parallel with) deep classification, and provides quick detection of messages with explicit trigger words or patterns. After tokenisation, let the English-language message (tokens – words or subwords)
T ={w1 , w2 , … , wn }.
Set the trigger dictionary (set of keywords/templates):
Dtri g={weapon , bomb , explosion , terrorist } (engl.) and
Dtri g={ зброя , бомба , вибух , терорист , … } (ukr). 2.1 Binary rule (simple detection) is a formal statement. The filtering function is defined as: (16) (17) (18) (19) n Score (T )=∑ 1 [ wi∈ Dtri g ]⋅ ω ( wi),
i=1 where 1 [⋅ ] is an indicator function, ω ( wi) – is the weight of the trigger word wi.
Example: T = T h ey plan to plant a bomb and cause an explosion → tokens contain bomb and explosion.
Score (T )=ω ( bomb)+ ω (explosion )=3+2.5=5.5.
For example (ukr): зброя → ω=2, бомба → ω=3, вибух → ω=2.5. If the message:
T ={ закликаю , взяти , зброю , і , боротися }, Then Score (T )=1⋅ ω ( зброя)=2.
2.3 Patterns (regular expressions (regex)/n-grams/phrase patterns). In addition to individual words, you can specify expression patterns. Add a set of P patterns (regular expressions), for example:
R (T )={1 , if ∃ wi∈ T : wi∈ Dtri g ,
0 , ot h erwise
That is, if the message contains at least one word from the dictionary, the system marks it as potentially dangerous. Example: T = I call to take up arms and fig h t → after lemmatisation/normalisation may contain arms → if arms in Dtri g (or displayed on weapon), then R ( T )=1.
2.2 Scoring rule. Sometimes, not only is the binary decision (0/1) essential, but also the degree of risk. To estimate the degree of risk, we enter the weights ω : Dtri g → R+¿¿. For example: ω ( weapon )=2 , ω ( bomb)=3 , ω (explosion )=2.5 , ω ( terrorist )=4.
Then you can enter the scales: r"\bplant(ed)?\b.*\bbomb(s)?\b" — «plant a bomb / planted bomb»; r"\btake up arms\b"; r"\bblow (.+ )?up\b".
Match indicator:
R patter n (T )={ 1 ,∃ p∈ P : T ⊨ p ,
0 , ot h erwise .
Example: We will plant a bomb tomorrow → satisfies plant .∗bomb ⇒ R pattern (T )=1. Example (ukr):P={взяти .∗зброю , закликаю .∗боротися }. Then:
1 , if T satisfies p∈ P , (20)
R pattern (T )={ 0 , ot h erwise .
Example. The message "I urge you to take up arms and fight" → satisfies the "take .* arms" pattern. So R pattern (T )=1.
2.4 Obfuscation treatment (leet/character substitution/spacing). Attackers can write b 0 mb, b o m b, b @ mb, weap 0 n, etc. For persistence, use: 1. Normalisation – replacing numbers with letters, removing unnecessary spaces/symbols: where matc h ( wi , t )=1 if exact or fuzzy-match.
The final rule-based metric canbe calculated as:
Rfinal (T )= β1⋅ R (T )+ β2⋅ R pattern (T )+ β3⋅ Score (T ) , where β1 , β2 , β3 – are the weight coefficients adjusted empirically. Or:
Rfinal (T )= β1 Rword (T )+ β2 R pattern (T )+ β3 Rcontext (T ), (21) (22) (23) (24) (25) where R pattern (T )∈ {0,1 } (can be weighted in terms of ¿2), Rcontext (T ) – additional contextual features (see below), β j – are weights (for example, β1=0.5 , β2=0.4 , β3=0.1). Threshold solution: label (T )={
flagged , fin❑al(T ) ≥ θ , notflagged , Rfinal (T )<θ .
Numeric example for the message T h ey plan to plant a b 0 mb tomorrow : bomb fuzzy-match → ω ( bomb )=3 ⇒ Rword=3. plant .∗bomb pattern matched ⇒ R pattern=1. At β1=0.5 , β2=0.4 , β3=0:
Rfinal=0.5⋅ 3+0.4⋅ 1=1.5+0.4=1.9.
At the threshold θ=1.0 → the message is flagged. Example:
R ( T )=1 (because there is a зброя),
This component performs an in-depth semantic analysis of the input text, determining the probability of its belonging to a particular category. The result of the calculations is logit-numerical values interpreted as the model's confidence in the correspondence of the text to a specific class. For the convenience of interacting with the model, a special functional interface predict() has been created, which encapsulates the process of tokenisation, prediction, and interpretation of the result, allowing you to get the final answer with a single call.
External interaction with the user is implemented through the Telegram integration module, which allows you to automatically receive text messages, send them to the classifier and return the result directly to the messenger.
An asynchronous loop functions above this level, which ensures the constant operation of the Telegram bot and its readiness to process new requests. An essential part of the structure is the logging module, which records all events, including incoming messages, classification results, and possible errors. It allows you to track the operation of the system and ensures its controllability and reliability.
The software does not contain classic graphical interfaces or macros. Still, it implements a fullfledged command control mechanism via Telegram, making it accessible to end users without the need to install additional applications. This approach allows you to apply development both onpremises and in the cloud, while providing flexibility to scale. The entire system is built according to the principle of low binding and high coherence. Each module performs only its own welldefined function and does not depend on the internal implementation of other components. It ensures that the software is easy to maintain, update, reuse, and adapt to new requirements. 5)
The developed software is a comprehensive system based on deep semantic recognition of text messages, followed by their classification according to the level of danger. All functionality is implemented on the basis of a modular structure, where each component plays a specific role, but at the same time organically interacts with other elements of the system. The entire cycle of message processing begins from the moment the user receives the text, which comes through the integrated Telegram interface.
Next, the processing logic chain is activated: the input text is passed to the clean-up module, where pre-normalisation is performed - HTML elements, punctuation marks, redundant spaces are removed, and all characters are reduced to lower case.
After that, the text is passed to a tokeniser, which transforms it into a sequence of numbers that serve as input to the neural network. These vectors arrive in a model of the type BERT, which has been previously trained on a classification problem with two classes - "sarcastic" and "terrorist" messages. The model performs a forward pass by calculating logits and probability vectors. Based on the highest value among the probabilities, the system decides which class the text belongs to. The forecast is formed in the form of a text conclusion sent back to the user via Telegram.
All intermediate steps are implemented in the form of functions and subroutines. In addition to the main classification, the logic of event logging is provided, which makes it possible to keep records of requests, store statistics of classification results, and record possible exceptions or anomalous situations. If necessary, each module can be tested separately because it does not depend on the internal implementation of other parts of the system. All functions included in the program work synchronously and sequentially, transferring data between modules in the form of simple objects or dictionaries. It makes it easy to debug the process, scale the solution to cloud computing platforms, or embed it in more complex information systems with cybersecurity needs. 6)
The description of the created software tool meets the requirements of GOST 19.402-78 "Program Description" and ISO/IEC 26514:2008 "Systems and software engineering - Requirements for designers and developers of user documentation". As part of this work, intelligent software has been developed for the automatic detection of messages of a terrorist nature. It is based on the BERT transformer model integrated into the Telegram bot, which allows for real-time semantic analysis of texts. The tool is focused on being used as a separate service or as part of larger monitoring systems.
Installing the software requires Python version 3.10 or higher, as well as the installation of appropriate libraries, including transformers, torch, scikit-learn, pandas, telegram, asyncio, and nest_asyncio. To use the Telegram interface, you must have a Telegram account and a generated Bot API access token. Optionally, it is recommended to use a system with CUDA support if you plan to process large amounts of text. Installation is done by creating a project directory, installing dependencies via pip install -r requirements.txt, adding a token to the TOKEN variable or to a secrets.json file, and checking for the presence of a saved model in the directory (e.g. saved_model123). After that, the software is run through the main script or the Jupyter interactive environment, after which the bot begins processing messages.
The user interface is implemented in the form of a text chat. The software does not have a graphical interface in the usual sense - windows, menus or buttons - but it supports full-fledged interaction through Telegram. The user sends a message that goes through two levels of analysis: 1) superficial rule-based analysis for keywords and 2) in-depth semantic analysis through the BERT model. If trigger terms are detected, the "terroristic" class is immediately returned. If there are none, the message is classified using a neural network, resulting in a text category (sarcasm or terroristic) returned to the user. All requests and responses are logged.
The typical usage scenario is as follows. The user enters a message into the chatbot, which is instantly processed: tokenisation, heuristic comparison, and contextual interpretation through the model are performed, and a response is returned. In this way, the user can check any text for potential terrorist threats, including hidden or sarcastic hints. This functionality can be especially relevant for administrators of public channels, journalists, moderators, or information security services.
Among the possible errors that the system can return, the most common are the lack of connection to the Telegram API, the incorrectly specified path to the model, the lack of the necessary libraries or the wrong access token. All these problems can be fixed by checking the connection, establishing dependencies with pip, or fixing the configuration files. In case of difficulties, reinstalling all modules and restarting the service usually helps.
Since the project is educational, official technical support is not provided. However, flexible modification is provided. For example, model weights and tokeniser are stored in a separate folder and can be reused without the need for retraining. You can also adapt the rule-based dictionary to new terms, train the model with specific examples, or integrate additional confidence threshold control mechanisms.
In general, the software tool is modular, logically connected and covers the whole cycle: from text input to classification return. This approach allows you to ensure compatibility between components, ease of maintenance, and the ability to scale to more complex information security scenarios. 7)
User instructions for the information system for detecting terrorist appeals This software tool allows you to automatically analyse text messages and classify them according to the characteristics of terrorist or sarcastic content. The main interaction environment is the Telegram messenger. The system works on the basis of the BERT model trained on the corresponding corpus of texts. User Requirements: 1. Basic Telegram skills 2. Availability of an Internet connection
For local launch: knowledge of Python, transformers, torch, python-telegram-bot, and nest_asyncio libraries installed.
Installation and Launch (for a developer or system administrator): pip install torch transformers python-telegram-bot nest_asyncio
Run telegram_bot.py in Jupyter Notebook or from a terminal.
Using a Telegram bot:
Find your bot by the name that is specified when creating it in BotFather.
Click "Start" or send any message (for example: I'm going to attack tomorrow).
Get a response from the bot SARCASTIC - the message has no signs of threat.
TERRORISTIC - lexical or semantic patterns similar to terrorist rhetoric have been identified.
The model has an educational origin, so that the result can be conditional. If in doubt, try again or conduct an additional check.
Safety recommendations: 1) Do not send real threats to the bot.
2) Do not use the system for illegal surveillance or harassment.
1)
Input Block "Input data" with a detailed description of the case (sources, volume, balance of classes). Add a table: number of messages, average length, % of terrorists, etc. Here in Fig. 8, we can see how our categories are distributed, such as terrorist and sarcastic messages, and it is necessary to parse the dataset in such a way that everything is equal, so that the model shows everything as accurately as possible.
Here we see how our messages are distributed by word length. It is also beneficial for model research. Boxplot Message Length by Class clearly separates the median: for class 0, it lies at ~7 words, while for class 1, it is only ~3–4 words. The model also uses this factor as an indirect signal short, aggressive statements are more likely to be dangerous. The dataset was assembled independently, using real and synthetic messages from public sources. First, about eight thousand texts were collected from open Telegram channels, as well as from forums and Twitter, where sarcastic or radical statements are often found. To ensure anonymity and ethical use, all messages have been cleared of personal data, user IDs, emojis, garbage symbols, and links.
At the initial stage, duplicates were removed using a normalised hash - lower case, punctuation was removed, and MD5 was calculated. It made it possible to weed out about a thousand duplicate lines. Next, a rule-based filter was used, which contained a dictionary of more than 200 key trigger words. If such a trigger occurred in the text, the message was previously marked as potentially terrorist; otherwise, it was sarcastic. It yielded an initial class ratio of approximately 30:70. However, to ensure quality, 1,200 random examples from each class were selected and manually checked. Some of the sarcastic jokes with words like "explosion" have been corrected, because they had no real terrorist meaning. Instead, some disguised radical messages with no explicit vocabulary were transferred to the "terroristic" class. After that, the balance was manually equalised to get 2,000 messages in each category.
At the final stage, the standard normalisation was performed: I moved all texts to lower case, removed unnecessary spaces, and replaced obscene language with a special tag <obscene>. For further analysis, a length field has also been added, which contains the number of words in the message. It made it easy to build statistics on the length of texts and understand the distribution of content.
Once the data is prepared, the dataset is divided into training, validation, and test samples in an 80/10/10 ratio using train_test_split and the stratify parameter to maintain an even distribution of classes. In addition to this, light synonymous argumentation is applied for sarcastic messages, replacing some words with synonyms via WordNet to train the model to better recognise different forms of expression of sarcasm.
At the final stage, it was checked that none of the texts in the test set intersected with the training set - for this, I used hashing again. All texts were saved in CSV and XLSX formats, and catalogued in the datasets/v1/ directory with an accompanying JSON file, which indicates the date of formation, sources and scripts for cleaning.
Thus, a full-fledged balanced dataset was prepared, which formed the basis for further stages analysis, graphing, training of the BERT model, and testing in the Telegram bot environment. 2)
Analysis of the word stops volume influence on the text processing time The graph shows how the average preprocessing time of a single message changes depending on the size of the stop word dictionary. Within our corpus, even increasing the list to 60 words adds only a few tenths of a millisecond - that is, the impact remains minuscule compared to a whole passage through BERT. Reading: The slowest measurement (≈ 0.003 ms) is 1.3 times longer than non-stop word processing, but the difference is far below human tangibility.
Thus, we can draw a practical conclusion: expanding the stop dictionary to several dozen words does not harm performance, but it gives the best semantic "noise protection". Suppose you switch to hundreds and thousands of stop words (which can happen in multilingual projects). In that case, the increase in processing time will remain insignificant compared to the neural network stage because BERT and tokenisation are the most expensive.
Within the framework of the implemented system, the text message goes through several successive processing steps before being transmitted to the BERT model for classification. The main goal of this process is to prepare raw data into a format suitable for deep neural network analysis, eliminating noise and preserving key semantic signals.
The first step is text cleaning, which includes removing extra spaces, punctuation, special characters, HTML tags (if any), and case normalisation (all characters are lowercase). This approach avoids the variability of the form of writing words, which does not carry a meaningful load.
Next, the removal of stop words is applied - the most common words of speech (for example, "and", "this", "there", "but"), which have a low discriminative ability in classification. It has been experimentally proven that increasing the stop word dictionary to 60 does not have a critical impact on performance, but significantly improves the signal-to-noise ratio in the text.
The third stage is tokenisation - splitting the text into separate elements (tokens), in accordance with the requirements of the BERT model. To do this, a specialised BertTokenizer is used, which not only performs partitioning but also replaces each token with the corresponding numerical index from the model dictionary. In this case, the sequence length is aligned by adding paddings ([PAD]) or trimming excess tokens to a given maximum value (max_length=128).
After tokenisation, an input tensor is formed, which is passed to the BERT model. At this stage, the model is already able to conduct a deep contextual analysis of the text, taking into account the relationships between words, their order and the general semantics of the message. Such a word processing pipeline allows you to achieve a balance between performance, classification quality, and system adaptability to new data. 4)
Analysis of hyperparameters The analysis of hyperparameters in the developed text classification system plays a key role in achieving high model accuracy and its ability to generalise. Hyperparameters are understood as the external parameters of the model's training, which do not change during the training process, but significantly affect the effectiveness of training, gradient stability and convergence rate.
Within the framework of the implementation, the following primary hyperparameters were analysed:
1. Batch size (batch size) – This parameter determines the number of examples processed simultaneously during one optimisation step. In our implementation, the optimal size turned out to be batch_size=16, which provided stable training on the available GPU configuration without memory overload. Too small a batch can lead to unstable weight updates, while too large a batch can lead to a loss of generalisation ability.
2. Number of epochs (num_epochs) – the model was trained during one epoch (num_epochs=1) due to resource limitations and the desire to test the performance of the prototype. In the future, to achieve better results, it is advisable to increase the number of epochs to 3-5 with early stopping to avoid overtraining. The average loss in training is 0.0795, and the accuracy on validation reaches 1.
3. Initial learning rate – this parameter, lr=2e-5, is chosen as the recommended value for models based on BERT. Too high a value can lead to destructive fluctuations of weights (divergence), and too low - to slow convergence. Support for the learning rate scheduler in the form of a linear decrease made it possible to gradually reduce the pace of learning, which stabilised the optimisation process.
4. Optimiser – the AdamW optimiser was used to update the scales, which has proven itself well in transformer models, since it takes into account the moments of the first and second order and uses decoupled weight decay for regularisation.
5. Maximum length of the input sequence (max_length) – the value max_length=128 was chosen, which is balanced between the coverage of the whole message and performance limitations. The average length of texts in the dataset allowed for the loss of information at this value.
Manual selection of hyperparameters made it possible to achieve the basic stability of the model. In the future, it is advisable to use automated selection using GridSearch, Bayesian Optimisation or Optuna, which will allow you to find the optimal configurations, taking into account the specifics of real text messages in instant messengers.
We can see that values that are too small (1e-6) lead to lower accuracy due to slow learning, while values between 1e-5 and 2e-5 provide the best results. Values that are too high (>3e-5) cause unstable learning and poor quality. This analysis allows you to choose the optimal learning rate to achieve high accuracy. 5)
Description of the control example The description of the control case involves testing the developed software on real or synthetic text data, which is typical for the instant messaging environment, particularly in instant messengers. The main goal is to test the ability of the system to correctly classify text messages into one of two categories - sarcasm (sarcastic) or terrorist content (terroristic) - in accordance with the logic that was laid down during the training of the model. This example is independent of the data used during the training and therefore allows you to evaluate the quality of the generalisation and the real effectiveness of the model. Below is a plate with nine examples of texts that have passed through the system, as well as the classification provided. For each of the messages, a prediction is provided using a trained model integrated into the Telegram bot. All messages were selected taking into account their clear connection with a specific category, so the level of confidence in the classification in each case is 100%. 1 Oh, great, Monday is the best day of the year again! О, чудово, знову найкращий день у році! понеділок 2 Now I know for sure that the government Тепер я точно знаю, що уряд дбає про sarcast cares about us more than anyone else. It's true нас більше за всіх. Правда ж ic 3
We will destroy you all.
Ми знищимо вас усіх. 4 Of course, it was the best idea to call rain on my vacation.
Звісно, це була найкраща ідея - sarcast покликати дощу саме на мою ic відпустку. 5 Everybody, pack your weapons and take them to the streets; it's time to act! Усі зберіть зброю і виходьте на вулиці, terrori час діяти! stic 6 I love it so much when the Internet works now and then. It's so much fun! Я так люблю, коли інтернет працює sarcast через раз. Це так весело! ic 7 Tomorrow we'll attack Завтра ми атакуємо
gift from the heart О, звісно, це не корупція - це просто sarcast подарунок від серця ic
# English 6)
Analysis of the results of the control example of classification allows you to better understand how the model works, on the basis of which speech signals it makes decisions, and what features of texts become markers of belonging to a particular class for it. All 9 test messages were correctly classified, which indicates the effective operation of the model in real conditions. Below is a detailed analysis of each prediction:
"Oh, great, Monday is the best day of the year again!" - sarcastic. The model recognised sarcasm by combining positive vocabulary ("wonderful", "best day") with a wellknown negative context - Monday, traditionally associated with the beginning of the working week and hostility.
"Now I know for sure that the government cares about us more than anyone else. The ironic construction and use of a rhetorical question creates the impression of insincerity, which is a key indicator of sarcasm. The model recognised this pattern through the subtle context of a negative assessment disguised as a positive one.
"We will destroy you all. No one will be saved." - terroristic. Direct threatening nature and aggressive language ("we will destroy" and "no one will be saved") give precise Verdic t - sarcast ic terrori stic terrori stic terrori stic semantic colouring to the message. The model identified this as potentially dangerous, since it lacks any hint of joke or irony.
"Of course, it was the best idea to call the rain on my vacation." - sarcastic. The apparent contradiction between the phrase "best idea" and the negative result - rain on vacation - is a typical example of irony.
"Everybody pack your arms and take to the streets, it's time to act!" - terroristic. Calls to action using the words "weapons", "get out", "time to act" are classic patterns of verbal radicalisation.
"I love it so much when the Internet works every other time. It's so much fun!" sarcastic. The contrast between the formally positive phrase "I love so much" and negative experiences - problems with the Internet - is a sarcastic marker.
"Tomorrow we attack, be prepared." - terroristic. The wording "attack", "be ready" has a militaristic connotation and hints at the coordination of terrorist actions. Even in the absence of a direct object of attack, vocabulary and syntax allow the model to classify this message as dangerous.
"Oh, of course, it's not corruption - it's just a gift from the heart." - sarcastic. The phrase "gift from the heart" is used in an ironic context as a euphemism for corruption.
"It's time to destroy all institutions of power. They have no right to exist." terroristic. The model classified the message as terrorist due to its aggressive declaration of the destruction of power structures.
As a result, the accuracy of the classification is explained not only by the number of known lexemes but also by the model's ability to contextually analyse linguistic inversions, rhetorical figures,and general intonation. It proves the superiority of deep transformer models of the BERT type in solving problems of computational linguistics related to the recognition of emotional or aggressive overtones. 7)
This software must be added as a bot to the Telegram messenger by name (@contrter_bot), after which it should be added to your Telegram. After that, you need to click on the button ("Start") and enter any message to which the system should respond (Terroristic) or (Sarcastic). It is worth noting that the model is trained to recognise not only superficial signals in the form of individual words, but also complex speech constructions - irony, rhetorical questions, syntactic inversions. It confirms its ability for deep semantic analysis, which is especially valuable in security tasks where language can mask the true intention of the author. In addition to the algorithmic part, the control example confirmed the effectiveness of the Telegram interface, which allows you to interact with the system conveniently. It makes the developed software suitable for real implementation as an auxiliary tool in online communication monitoring or moderation systems.
In the process of implementing the system of automatic detection of terrorist calls in text messages, a series of experiments were carried out in order to check the efficiency, accuracy and stability of the developed software. One of the key aspects that allows you to evaluate the quality of the system's performance is the collection of statistics on its performance. The collection of such data makes it possible not only to analyse the behaviour of the system in different situations, but also to identify bottlenecks or potential points of productivity growth. To obtain objective results, a real trained model based on the BERT architecture was used, implemented using the Transformers library from HuggingFace, which allows for highly accurate predictions based on the semantic analysis of the text.
During testing, 100 random messages were submitted to the system, which were conditionally classified as neutral or potentially dangerous (terrorist or sarcastic). All incoming messages went through the same processing chain: text cleaning, tokenisation, transfer to the neural model, receiving logs, and calculating the final class. For each prediction, the metrics of the processing time, the prediction class, the value of logits, and the objective correctness of the result, if it was known in advance, were additionally recorded. The collected data makes it possible to form a clear picture of the average performance of the system and its sensitivity to different types of messages, as well as estimate the percentage of errors, false positives, and false negative responses. A thorough analysis of these statistics is necessary to confirm the suitability of the model for practical use, as well as its subsequent adaptation and possible scaling to other sources of text (in particular, Telegram, Twitter, forums, news commentaries, etc.).
The purpose of the statistical analysis is to quantify the effectiveness, reliability, and overall performance of the developed Telegram bot, which classifies text messages as sarcastic or terrorist content. Particular attention is paid to key indicators that allow us to conclude the suitability of software for use in real conditions. One of the main aspects of the study is the response time of the system, that is, the interval between the moment the message arrives and the moment when the user receives the classification result. This option is critical in the context of interactive interaction.
Another important criterion is the stability of the bot's functioning, which assumes uninterrupted operation for a long time, as well as the absence of failures, critical errors, or freezes even under significant load. In addition, the analysis covers the accuracy of the model, which is an indicator of its ability to correctly determine whether a message belongs to the appropriate category. This metric reflects the quality of machine learning and the effectiveness of the BERTbased approach used.
The study also takes into account the number of messages that the system is able to process per unit of time. This parameter is essential in terms of scaling the system and evaluating its performance when working with a large number of requests. The analysis is expected to establish an average response time within a few seconds, confirm stable operation for at least half an hour of continuous load, identify a high level of classification accuracy, and demonstrate the ability to process a significant number of messages without loss of quality or speed. All these factors together allow you to get a holistic picture of the real characteristics and potential of the system for further improvement or implementation in practical projects.
This project used both standard Python tools and specialised libraries to collect and analyse data. The main tools for capturing statistical indicators were variables within training cycles, logging through print, and the tqdm library for visualising the process. To measure the accuracy of the classification and generate the classification report, the classification_report function from sklearn.metrics was used. It made it possible to accurately assess the performance of the model on the validation kit.
The response time was recorded implicitly by observing the interval between the receipt of a message to the Telegram bot and the generation of a response displayed in the console via print. Direct integration with the Telegram API (through the python-telegram-bot library) made it possible to track each incoming request and instantly record the fact of the response, which served as a token of stability and performance of the system.
Launch conditions included a local Jupyter Notebook environment running on a laptop with Anaconda installed and CUDA support (if you have a graphics card). All tests were run locally, with no cloud or server deployment. The use of nest_asyncio ensured compatibility between the Telegram bot's asynchronous calls and the Jupyter environment.
The testing period was approximately one hour of active interaction with the bot in the process of debugging and observing the results. In total, more than 100 manual messages were tested, which made it possible to evaluate the behaviour of the system on a large amount of input data and in various contexts.
Thus, the methodology for collecting statistics was based on a combination of automated monitoring (via PyTorch, HuggingFace Transformers, and Telegram API) and manual control of the results through Jupyter Notebook, which provided a flexible, detailed and reliable approach to analysing the system.
Within the framework of the study, a number of key metrics were recorded and analysed, which allowed the quality and efficiency of the developed message classification system to be assessed. One of the essential characteristics was the speed of processing one message. By using the optimised BERT model and pre-moving it to the GPU (with CUDA), the average processing time for a single text ranged from 0.4 to 0.6 seconds, which is an acceptable result for real-time tasks given the complexity of the depth model.
As for our model, the quality metrics turned out to be too high; there were sure signs of overtraining.
As for the total volume of interactions with the system, more than 100 messages of various content and length were processed during the testing period. It made it possible not only to check the correctness of the Telegram bot, but also to assess the stability of the response in a wide range of input options, including sarcasm, irony, harsh statements, ordinary phrases, etc.
The percentage of correct classifications was estimated by comparing the predicted labels with the expected ones. For the validation sample, the accuracy was more than 94%, which is confirmed by the classification report. The model copes exceptionally well with typically formulated terrorist calls, where its accuracy is close to 97%. At the same time, more complex cases related to the sarcastic context remain more ambiguous, although even there the level of correct decisions is kept at more than 90%.
The performance of the model remained stable even with intense interaction. Not a single crash, freeze or critical error was recorded during the entire test period. The system responded to each message at a predictable speed. The load on the system remained low due to the efficient use of computing resources, the correct formatting of input data, and the preliminary storage of the trained model in the local environment.
Thus, the analysis of the measured metrics allows us to conclude about the high stability, accuracy, and speed of the proposed system, which is a prerequisite for its further use in real scenarios of automatic text monitoring. 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 # Average Message Processing Time 1 Minimal processing time
Thus, the classifier demonstrates perfect work on thetest sample: Recall and Precision for both classes are 1.0, and there is no bias towards any class.
The analysis of the results showed that the overall performance of the system meets expectations, but specific nuances should be taken into account in further optimisation. From a technical point of view, the model works quite fast - the average processing time of a single message is approximately 0.54 seconds, which is acceptable for real-time. The minimum time was recorded at 0.30 sec, and the maximum time was 0.74 sec. This variation is due to fluctuations in performance under different loads or when transmitting data via API. The total number of messages processed during testing is 100, which allows us to conclude the stability of the model. The Telegram API has proven to be stable; however, in some cases, slight delays have been observed in transmitting the response to the user. It is most likely due to the network or asynchronous nature of the Jupyter environment in which the startup was performed. The delays were not critical and did not exceed 1-2 seconds, which did not significantly affect the user experience. Regarding the quality of classification, the model demonstrated the following metrics: 1)Accuracy (classification accuracy): 0.93 2)Precision for the "terroristic" class: 0.91 3)Recall for the class "terroristic": 0.90 4)F1-score for the "terroristic" class: 0.91
These indicators indicate a high ability of the model to recognise terrorist messages. Most often, errors occurred in cases where messages had double or sarcastic connotations. For example, phrases like "Maybe we can blow up that office like in the movies?" can be perceived as terrorist, even if they are humorous in intonation. The model, having no context or intonation analysis, classifies them according to the triggers detected.
There have also been cases when short sarcastic messages with aggressive language, but without apparent signs of threat, were falsely classified as "terroristic". It shows the limits of the current approach and the importance of further work on a better distinction between sarcasm, irony, and real threat.
The following graphs show the ROC curve and the PR curve (Precision-Recall), which are standard metrics for assessing the quality of classification models, especially in cases with imbalanced classes.
The ROC curve (Receiver Operating Characteristic) shows the relationship between the True Positive Rate (i.e. sensitivity or Recall) and the False Positive Rate when the decision threshold changes. An ideal ROC curve is considered to be as close as possible to the upper left corner of the graph since this indicates the model's ability to distinguish classes. In our case, the curve practically follows the ideal trajectory, and the area under it (AUC) is 1.0, which is the maximum value. It means that the model unmistakably distinguishes terrorist messages from sarcastic ones.
The PR curve (Precision-Recall) illustrates the relationship between accuracy (Precision - how many of the predicted terrorist messages are truly terrorist) and completeness (Recall - how many of all real terrorist messages are correctly detected). This graph is critical in problems with rare events (for example, detecting terrorist statements among the general text stream). In the presented case, the PR curve also has a perfect shape, which indicates a high quality of classification without false positives. The model demonstrates consistently high Precision values even at maximum Recall values, which is practically not found in real problems.
Thus, both curves confirm that the classifier performs its task almost flawlessly on the test sample. However, in real conditions, such perfection should be tested on external or noisier data to ensure the stability and reliability of the model.
The two graphs show two-dimensional projections of all dataset messages obtained by two different methods of dimensionality reduction: on the left - PCA, on the right - t-SNE. Each dot corresponds to a single message; The colour indicates the class: blue dots are sarcastic (0), red dots are terrorist (1).
The first projection (PCA) shows only a limited separation: clouds of the two classes partially overlap, which is expected, because PCA looks for linear directions of maximum variance and does not take into account nonlinear relationships in text features.
Instead, the second projection (t-SNE), which better preserves local distances in highdimensional space, demonstrates a more apparent clustering: red dots stray into compact "islands", blue dots form separate groups outside them. It confirms that in the vector representation (TF-IDF), terrorist and sarcastic messages mostly form different subspaces, which is the difference the model uses when classifying.
As a result, the model functions stably and with high accuracy, but needs to be improved to handle edge cases and provide even greater contextual accuracy. The results showed that the BERT-based model successfully copes with the task, demonstrating high classification accuracy and stability even in cases of ambiguous or emotionally coloured messages. At the same time, certain limitations have been identified, in particular regarding the processing of long texts, delays in responses under heavy load, and difficulties with the interpretation of sarcasm. Improvements have been proposed to increase the efficiency of the system, including the use of lighter models, improved tokenisation, implementation of caching, and expansion of the training dataset. Thus, the work done not only confirmed the performance of the developed model but also made it possible to identify its strengths and potential areas of development. The conducted study is a valuable stage in checking the effectiveness of the application of modern methods of natural language processing to solve the problem of security in the information environment.
The presented project aims to create an intelligent system capable of automatically detecting dangerous content in text messages, including those containing potential terrorist calls or sarcasm. The relevance of such a development is due to the growth of information generated in instant messengers, social networks and forums, where messages often appear that require an urgent assessment of a potential threat. For this purpose, modern methods of natural language processing (NLP) were used in combination with the powerful BERT model, which underwent further training on a specially prepared corpus of texts. The result was the integration of a trained model with a Telegram bot that analyses users' incoming messages in real time and classifies them according to the level of danger. Thus, a full-fledged cycle has been implemented - from data collection and model training to practical application in the conditions of a real information flow.
As part of the project, a complete cycle of software development for detecting terrorist calls in text messages was implemented, covering several critical stages. The first step was the preparation of the dataset: messages containing sarcasm or terrorist content were collected, cleaned and annotated. For this, regular expressions, noise filtering and lowercase text are used. After preprocessing the texts, they were tokenised using the pre-trained bert-base-multilingual-cased model, which made it possible to submit texts in a format suitable for training a neural network. At the next stage, datasets for training and validation were created, which were used to train the classifier based on the BERT architecture. During the training, modern optimisation algorithms such as AdamW and a training speed planner were used, which contributed to stable and efficient convergence of the model. During the training, accuracy metrics, validation, and classification errors were monitored, which made it possible to monitor the quality of training and identify potential problems with over- or under-training in a timely manner. After successful training, the model is saved and integrated with a Telegram bot implemented using the python-telegram-bot library. The bot works in real time, receives the user's messages, passes them to the model for classification, gets the result and returns the response to the user. For ease of deployment and testing, the bot is implemented in the Jupyter Notebook environment using nest_asyncio and asyncio. During the testing process, a control example was carried out: checking the bot's operation with real messages showed high model accuracy, processing stability, and fast response time. Execution statistics are also collected, including the number of requests processed, average response time, and classification accuracy. The results obtained indicate the effectiveness of the implemented system and its compliance with the tasks. Thanks to the flexibility of the architecture and the possibility of additional training, the model can be scaled and adapted for a broader range of functions in the field of security and monitoring of text content.
The results achieved during the implementation of the project indicate the high efficiency of the created system for detecting terrorist calls in text messages. Already at the validation stage, the model demonstrated an accuracy of more than 90%, which exceeded the initial expectations for the baseline classification level. According to the results of testing in a real environment through the Telegram bot, the stable operation of the system was recorded with minimal delays in response the average processing time for one message was less than 1 second. This indicator confirms the possibility of effective use of the system in real time without significant delays. The collected statistics also confirm the reliability and performance of the implemented solution. For example, several dozen queries were successfully processed with almost complete correspondence between the prediction and the real content of the message, which indicates the adaptability of the model to the tasks of practical monitoring. The model showed the ability to differentiate between sarcasm and threatening rhetoric, which was previously considered a difficult task even for modern neural networks. The expected results formulated at the beginning of the project, including the accuracy, stability, responsiveness, and functionality of the Telegram bot, were not only achieved but exceeded in some aspects. Thus, we can confidently say that the created system not only fully meets the task but also demonstrates the potential for further expansion of its capabilities and scaling.
During the development of the terrorist call detection system using the BERT model and the Telegram bot, a number of technical problems and limitations were identified that complicated or temporarily stopped the development process. One of the key issues was the compatibility of libraries, in particular Telegram API updates, as a result of which some previously working methods, such as Updater, Filters, or ApplicationBuilder, behaved unpredictably or were not imported at all. It forced a complete redesign of the bot launch logic to an asynchronous structure (async/await), which created an additional load when working in the Jupyter Notebook environment, which does not always correctly support asynchrony. Some difficulties also arose in the process of loading and saving the model. Several times, memory overload and cell freezes have been observed when initialising the model or retraining, especially when running on a local machine without a GPU. Because of this, it was necessary to restart the kernel and reload the model, losing the previous context of the session.
Another significant problem was related to the bot's processing of messages. In some cases, Telegram did not deliver messages back to the bot, or the handler did not have time to respond, indicating delays or excessive workload. It required optimising the prediction function and checking the polling API settings. There were also difficulties in interpreting the results of the model in cases of sarcasm, ambiguous or too long messages, when the model confused the tone or context. Such cases required additional analysis and, possibly, further fine-tuning of the model. Despite the listed difficulties, all technical challenges were gradually resolved, which made it possible to complete the development of the system and successfully implement it in a functioning form.
The prospects for the development of the created system for detecting terrorist calls are broad, both from a technical and an applied point of view. First of all, scaling the model to larger amounts of data is worth considering. Increasing the corpus of texts, in particular from real sources, such as open Telegram channels, forums, and social networks, will not only increase the accuracy of classification, but also cover a wider range of lexical, regional and contextual variations of messages. From the technical side, it is advisable to introduce a system of preliminary filtering of texts, which would filter out uninformative or safe messages even before submitting to the BERT model. It will reduce the load on computing resources and speed up the overall performance of the bot.
Additionally, it is worth implementing logging and archiving messages with abnormal or borderline results so that they can be analysed manually or used for additional training in the future. Another promising direction is to move to lighter models, such as DistilBERT or ALBERT, which consume less memory and allow for a more realistic run in production environments with limited resources, including mobile devices or cloud functions (e.g. AWS Lambda or Google Cloud Functions). At the same time, it is worth implementing support for multilingualism in order to process messages not only in Ukrainian, but also in English, Russian or other languages that are often used in an international context. In terms of integration, it is worth expanding the functionality of the system by adding a web administrator interface, where dashboards with metrics, the history of processed messages, and the possibility of manual intervention will be available. In addition, a potentially valuable addition will be the function of immediately notifying responsible persons or security authorities in case of suspicious messages, in particular via e-mail or push notifications.
In conclusion, it is promising to create an API or SDK that will allow other developers to connect the model to their chatbots or information systems, with the aim of wider distribution and increased public safety. In general, the results of the work confirmed that the use of modern deep neural networks in combination with available Python tools allows you to create practical solutions for socially essential tasks, in particular in the field of information security. The built system is a working prototype that can serve as a basis for further improvement, scaling, multilingual support and deployment in production environments. The project has significant potential for real-world applications, which opens up prospects for both practical implementation and continued scientific research in the field of natural language processing and security.
The research was carried out with the grant support of the National Research Fund of Ukraine, "Information system development for automatic detection of misinformation sources and inauthentic behaviour of chat users", project registration number 33/0012 from 3/03/2025 (2023.04/0012). Also, we would like to thank the reviewers for their precise and concise recommendations that improved the presentation of the results obtained.
The authors have not employed any Generative AI tools.