The DEFALAC project focuses on the automatic detection and generation of argumentative fallacies in Spanish using language models based on deep learning. Its main objective is to improve public debate and address the lack of resources in Spanish. This article presents the corpora generated during the execution of the project: FallacyES, composed of prototypical fallacies translated and revised from the Logic corpus, along with spontaneous fallacies extracted from comments on meneame.net; FallacyES-Political, containing approximately 2,000 fallacies extracted from 19 Spanish electoral debates over 30 years, classified into 16 types; and DebatES, based on the same transcripts as FallacyES-Political, enriched with additional information for automated analysis and manual exploration.
social contexts. Furthermore, the lack of studies in Spanish represents an opportunity to generate resources and methodologies specific to this language. The main objectives of the project are:
This work focuses on Objective 2, presenting the resources generated during the project’s execution, which have been published in various repositories. In the following sections, we present the main features of three resources: the FallacyES corpus, the FallacyES-Political corpus, and the DebatES corpus.
The FallacyES corpus is the first annotated fallacy corpus published in Spanish, central to the DEFALAC project. It contains two collections: prototypical fallacies, translated and reviewed, and spontaneous fallacies from meneame.net comments. Both include non-fallacious examples, crucial for training fallacy detectors. While prototypical covers 12 types, spontaneous examples focus on 8. Preliminary experiments highlighted the inherent complexity of detecting spontaneous fallacies.
The FallacyES-Political corpus comprises 1,965 fallacy instances extracted from 19 Spanish General Election debates spanning 30 years. Transcribed and annotated by experts, each entry provides the fallacy text, type, speaker, and crucial contextual information. This dataset classifies fallacies into 16 distinct types, including new categories like Appeal to Authority. Benchmark evaluations underscored the complexity of the classification task, highlighting benefits of fine-tuning domain-specific models for accuracy.
The DebatES corpus, still under development, will facilitate access and analysis of political information. It is based on transcriptions from the same 19 Spanish electoral debates as FallacyES-Political corpus, enriched with various data for automatic and manual analyses. Distributed in XML and HTML, annotations include linguistic metrics (e.g., lexical diversity) and political-domain insights. It will also integrate semantic and discourse information, such as thematic blocks and emotions.
The first annotated resource produced by the DEFALAC project was the FallacyES corpus 1 [
The types of fallacies included in the prototypical collection are as follows: • Hasty generalization: drawing a general conclusion based on one or few cases. • Ad hominem: attacking the person or entity making the argument instead of reasoning about the argument itself. • Ad populum: basing the truth (or falsehood) of an argument on its popularity. • False causality: establishing a causal relationship between two phenomena without providing evidence. • Circular reasoning: erroneous reasoning in which the premise and conclusion rely on each other for validation. • Appeal to emotions: attempting to manipulate the audience’s emotions to win the debate. • Red herring: introducing a new, unrelated topic to distract from the original debate. 1https://idus.us.es/items/ 170398a1-81b7-4e08-93cc-ccff0cabd506 • Invalid deduction: presenting a seemingly logical argument with formal errors (false analogy, denying the antecedent, afirming the consequent, etc.). • Credibility: basing the truth (or falsehood) of an argument on the opinion of an authority (argument from authority) or on traditionally accepted opinion (ad antiquitatem fallacy). • False dilemma: presenting two options as the only possible choices, when in fact more exist. • Straw man: reformulating the opponent’s argument in an exaggerated or caricatured way, then attacking that distorted version. • Intentional: any other type of fallacy not included in the above categories, in which the speaker’s intent to win the debate without sound reasoning is evident.
For the spontaneous fallacy corpus, examples were collected from comments on the website
meneame.net, a news aggregator where users discuss current events. To identify fallacies, following
the approach used by [
Not all accusations were substantiated, and some comments lacked enough context to clearly identify a fallacy. In total, more than 14,000 candidate comments were analyzed. The aim was to classify fallacies into the same 12 categories as the prototypical collection, but a significant number of examples were found in only 8 categories. The remaining 4 were excluded due to insuficient accusatory comments (fewer than 30) or lack of context needed for accurate classification.
In addition to the type and text, each instance in the resource also includes the headline and summary of the news article from which the comment was extracted. This context could be included as input for fallacy detection and classification models. Table 1 shows the total number of instances for each fallacy type.
A significant contribution of this corpus is the inclusion of non-fallacious examples with similar themes to the fallacies in both corpus sections, which is very useful for training and evaluating fallacy detectors.
In addition to the resource release, preliminary experiments on fallacy detection and classicfiation in
Spanish using the FallacyES corpus and the language models Flan-T5 and RoBERTa-BNE are detailed
in [
The debates were transcribed using WhisperX [
The corpus annotation process was divided into several phases. The first step involved creating a catalog of potential fallacy types, including definitions and examples based on existing literature. With this catalog, three annotators analyzed two full debates, identifying as many fallacies as possible. Subsequently, the detected cases were discussed, and 16 fallacy types were selected for the dataset. An initial version of the annotation guide was developed and refined through regular meetings.
The remaining 17 debates were independently annotated by two annotators per debate, aiming to maximize fallacy coverage. Cross-validation was then performed to assess annotation reliability, resulting in an average agreement rate of 47.67%.
Discrepancies were analyzed in meetings between annotators and project researchers, identifying three main causes: (1) overlapping definitions of some fallacies, which led to guide adjustments; (2) subjectivity in identifying certain fallacies, such as false cause; (3) the presence of multiple fallacies within a single text segment, which led to the use of multilabel annotation. Finally, the project researchers conducted a final curation, removing doubtful cases and ensuring the text segments were as concise as possible.
Table 2 shows the total number of instances per fallacy type.
To assess the dataset’s utility, a benchmark evaluation of state-of-the-art language models was conducted in a zero-shot classification setting. The results of the experiments underscored the complexity of the fallacy classification task and the limitations of current LLMs in understanding context-dependent reasoning. Additionally, the experiments demonstrated the benefits of fine-tuning a compact, domainspecific model instead of relying solely on general-purpose LLMs, achieving signicfiant improvements in classification accuracy with a more sustainable approach.
The analysis of political debates is a task that has attracted significant research interest, as these types of texts exhibit many features that make them particularly suitable for statistical and automated analysis. These include a clear argumentative structure, the repetition of key themes, the use of rhetorical and persuasive techniques, and a wide variety of approaches and styles.
The DebatES corpus is the final resource we plan to produce as part of the DEFALAC project. It is
still under development and therefore not yet publicly available. This corpus aims to serve as a tool
to facilitate access to and analysis of political information, particularly the messages that political
parties direct at potential voters in the weeks leading up to an election. The resource is based on
the textual transcriptions of the same collection of debates used in the FallacyES-Political corpus and
will include diverse types of information to enable automatic analyses, as well as tools for manual
exploration. Specifically, two versions will be distributed: one in XML format for automatic processing,
and another in HTML format (with charts, sections, hyperlinks, filters, etc.) to allow convenient
navigation throughout the resource. The corpus will include three types of annotations:
1. Descriptive information about the debates, following the style of [
Regarding linguistic information, the following metrics will be included, automatically computed
using the spaCy library [
To enrich the resource with semantic and discourse-related information, the texts are being processed
using the gemini-2.0-flash-exp language model [
In this work, we have presented the annotated resources developed within the DEFALAC project. The project focuses on the automatic detection and generation of argumentative fallacies using large-scale language models based on deep learning, with a particular emphasis on the Spanish language. The main objective of the project is to improve the quality of public debate through the development of tools that can identify and generate fallacies, addressing the current lack of Spanish-language resources for this task. Publishing such resources and making them available to the scientific community contributes to open science and enables other researchers to use them for training new models or evaluating text analysis systems.
We have presented three resources with diferent characteristics and objectives. The FallacyES corpus is composed of two collections: prototypical fallacies translated and reviewed from the Logic corpus, with manually created non-fallacious counterparts; and spontaneous fallacies, extracted from online comments on meneame.net, also with non-fallacious examples. The second corpus is FallacyESPolitical, which contains fallacy examples extracted from 19 Spanish electoral debates held over three decades, classified into 16 types of fallacies. The third resource, still under construction, is DebatES, based on the textual transcriptions of the same collection of debates used in the FallacyES-Political corpus. It will be enriched with various types of information to enable qualitative, quantitative, and automatic analyses.
This publication was produced by members of the ITALICA and TIC-134 research groups at the University of Seville. It was funded through the R&D project PID2021-123005, financed by the Ministry of Science, Innovation and Universities of the Government of Spain (MCIN) AEI/10.13039/501100011033/ and “FEDER A way of making Europe”.
During the preparation of this work, the authors used ChatGPT in order to: Grammar and spelling check. The authors reviewed and edited the content as needed and takes full responsibility for the publication’s content.