A Neuro-symbolic Approach to Argument Comparison in Structured Argumentation

Abstract

Defeasible Logic Programming (DeLP) is a structured argumentation formalism that uses a dialectical process to decide between contradictory conclusions. Such conclusions are supported by arguments, which are compared using a comparison criterion, to decide which one prevails in conflict situations. The definition of a formal comparison criterion is a central problem in structured argumentation, which is typically assumed to be provided by the user or knowledge engineer. In this work, we propose an integration between an argumentative approach to defeasible reasoning, such as DeLP, and machine learning models. Concretely, our goal is to train a neural network to learn a comparison criterion between arguments given a training set comprised of pairs of arguments labeled with which one prevails. We conducted several experiments, using a synthetic DeLP program generator, in order to assess the performance of a neural architecture under diferent kinds of DeLP programs. Our results show that under specific circumstances, a comparison criterion for arguments can be successfully learned by data-driven models.

Keywords

• eol>Structured Argumentation
• Defeasible Reasoning
• Defeasible Logic Programming
• Neuro-symbolic reasoning
• Neuro-symbolic learning

1. Introduction

The integration between machine learning and argumentation has received some attention in the past; though interesting results have been obtained, there are still many aspects to investigate; for a survey of works on this subject, see [ 1 ]. In this work, we focus on combining neural networks and an argumentative approach to defeasible reasoning. To explain why this integration is of our interest, we are going to briefly discuss some of the main characteristics of both parts.

Machine learning [ 2 ] is an exciting field that has matured into being capable of delivering important applications in the recent years after decades of slower progress. Today, much of the work in areas such as visual processing, language, and speech recognition is at least in part reliant on machine learning. In particular, neural networks [ 3 ], as a particular machine learning model, are capable of intrinsic massively parallel processing, which makes them suitable for several complex domains.

On the Knowledge Representation and Reasoning side, defeasible reasoning [ 4 ] is, intuitively, the process of tentatively inferring or deriving logical conclusions from available information— i.e., since the connection between premises and conclusions is a tentative one, then we can evaluate the rejection of our conclusions in the presence of information that contradicts them. In this work, we adopt an argumentative approach to defeasible reasoning, where queries are analyzed in a dialectical process that exhaustively considers arguments for and against specific answers in search of warrants, which refers to deciding how available information supports specific conclusions.

The combination of neural networks and argumentation is attractive because both fields can benefit from the capabilities of the other. On the one hand, neural networks can improve the performance of argumentative systems by allowing them to take advantage of available information (e.g., in real-world datasets) and to adapt to dynamic environments, with high volumes of data and where answers must be given quickly. On the other hand, argumentation can help neural networks deal better with inconsistent and incoherent information, can increase their explanatory power, and can also aid them in expressing specific domain knowledge.

In this work, our goal is to train a neural network to compare arguments and decide which one prevails in a conflict. For that purpose, we need the network to learn a specific component called a comparison criterion, which compares two arguments and decides which one prevails. This can benefit argumentative systems by allowing them to learn a new comparison criterion from examples, in a dynamic way and relevant to the particular situations that diferent programs aim to deal with. In general, it is not always simple to define a comparison criterion that fits the complexity of arguments in a given domain, so this task is often assumed to be carried out by a knowledge engineer. An approach like ours can therefore improve the performance of argumentative systems, for example in legal reasoning, where they could learn from datasets related to jurisprudence. While prior work on the integration of neural and symbolic AI architectures has introduced parameterized constants [ 5 ] and parameterized operators [ 6 ], we believe this is the first time a comparison criterion has been parameterized, thus advancing the eforts in the field.

Our two main hypotheses for this preliminary work are: 1. A neural network can learn a comparison criterion and decide, given two arguments, which one is preferred (if any), and 2. the complexity of the comparison criterion and the generated programs, and the number of examples afect the performance of the network.

This work is organized as follows. In Section 2 we review some basic concepts about the argumentation formalism that we use, called Defeasible Logic Programming (DeLP). In Section 3 we describe several aspects about the approach taken to arrive at an adequate neural encoding of arguments. Results obtained from the experimentation are shown in Section 4. In Section 5 we discuss related work, and finally, we summarize our main results and describe future lines of research in Section 6.

2. Background

We now recall some basic concepts on Defeasible Logic Programming, which from now on we refer to as DeLP for short; for the full details of the formalism, we refer the reader to [ 7 ].

2.1. Defeasible Logic Programming (DeLP)

DeLP [ 7 ] is a formalism that combines logic programming and defeasible argumentation. A DeLP program = (Π, Λ) is comprised of a set of facts and strict rules (Π), and a set of defeasible rules (Λ). In this work, we consider an extension of DeLP [ 8 ], called PreDeLP, that incorporates presumptions, which can be thought of as a kind of defeasible facts.

In the following, we briefly recall the most relevant concepts for PreDeLP from [ 8 ]. In the PreDeLP language, a literal is a ground atom or a negated ground atom. We represent strong negation with “∼ ” and say that and ∼ are complementary. A strict rule (resp., defeasible rule) has the form: ← 1, . . . , (resp., L≺− −

L1 , . . . , Ln ), where is the head of , and 1, . . . , , with ⩾ 1, is the body of . Facts are ground literals representing atomic information. Strict rules represent a strong relation between body and head . Both facts and strict rules constitute strict (sound) information.

A defeasible rule hea d≺− − body represents a weaker connection between body and head . It can be understood as expressing that “reasons to believe in the antecedent body provide reasons to believe in the consequent head ” [ 9 ], but it may be the case that body is true and head is not. Note that the symbols ← an≺ d− − denote meta-relations between a literal and a set of literals, and have no interaction with language symbols. As in Logic Programming, strict and defeasible rules are not conditionals nor implications, they are inference rules [ 7, 10 ].

A fact (resp., presumption) is a strict (resp., defeasible) rule with an empty body, denoted with (resp., L≺− − ). Intuitively, presumptions are defined as pieces of information that are tentatively taken to be true, usually in the absence of acceptable reasons to the contrary. Since they express reasons to believe in some information, they represent weaker assertions than facts. Definition 1 (PreDeLP). A PreDeLP program , denoted with (Ω, Θ, Δ, Φ), is a set of strict rules Ω, facts Θ, defeasible rules Δ, and presumptions Φ.

Next, we introduce the concept of annotated derivation, or just derivation, where we adopt the definition given in [ 8 ].

Definition 2. Let be a PreDeLP program and a literal. An annotated derivation of from , consists of a finite sequence of rules, facts, and possibly presumptions [1, . . . , ], where is 1. a fact , 2. a presumption , or 3. the head of the rule . Furthermore, if a rule is in the sequence, then its body 1, . . . , , is such that for all , with 1 ⩽ ⩽ , is a fact, a presumption or it appears as the head , for some rule with 1 ⩽ < .

A derivation is strict when neither presumptions nor defeasible rules occur in ; otherwise, is defeasible. A literal is strictly derived from , denoted ⊢ , if there exists a strict derivation for from ; on the other hand, is defeasibly derived from , denoted |∼ , if there exists a defeasible derivation for from and no strict derivation exists. A derivation for is minimal if no proper sub-derivation ′ of ( i.e., subsumes ′) is also a derivation of . Considering minimal derivations avoids the insertion of unnecessary elements that will weaken its ability to support the conclusion by possibly introducing needless points of conflict. Given a derivation for , there exists at least one minimal sub-derivation ′ for .

A PreDeLP program = (Ω, Θ, Δ, Φ) is contradictory if there exist derivations for two complementary literals. We denote with Π = (Ω, Θ) the strict part of , and assume that the sub-program Π is non-contradictory. Two literals 1 and 2 disagree w.r.t. if Π ∪ {1, 2} is contradictory.

Queries and Arguments. A query is issued to a program = (Ω, Θ, Δ, Φ) in the form of a ground literal . The dialectical process used in deciding if is warranted involves the construction and evaluation of arguments that either support or interfere with the query. Definition 3. Given a program = (Ω, Θ, Δ, Φ), let be a ground literal, and a derivation for obtained from . An argument for constructed from , denoted with ⟨, ⟩, is the set of facts, presumptions, and rules (strict and defeasible) used in the derivation .

Thus, we have that an argument ⟨, ⟩ is a minimal non-contradictory set of facts, presumptions, and rules, strict and defeasible, contained in .

Given a program , a literal is warranted if and only if there exists a non-defeated argument ⟨, ⟩ supporting . In order to establish whether ⟨, ⟩ is a non-defeated argument, defeaters for ⟨, ⟩ are considered, i.e., counterarguments that by some criterion are preferred to ⟨, ⟩. An argument ⟨, ⟩ is a counterargument for ⟨ℬ, ⟩ if and only if ⟨, ⟩ ∪ ⟨ℬ, ⟩ ∪ Π is contradictory. Given a preference criterion ≻ , an argument ⟨, ⟩ is called a proper defeater of an argument ⟨ℬ, ⟩ if it is preferred to ⟨ℬ, ⟩ according to ≻ , or is called a blocking defeater of ⟨ℬ, ⟩ if it is equally preferred or is incomparable with ⟨ℬ, ⟩ according to ≻ . In both cases, ⟨, ⟩ defeat ⟨ℬ, ⟩. Figure 1 shows an example where an argument ⟨, ⟩ is, according to an unspecified comparison criterion ≻ , a proper defeater of argument ⟨ℬ, ∼ ⟩.

The dialectical analysis that DeLP carries out to decide if an atom is warranted is outside the scope of this paper; we discuss the main points, and refer the reader to [ 7 ] for more details. Since there may be several defeaters for an argument, many acceptable argumentation lines could arise from one argument, leading to a tree structure. An argumentation line = [0, 1, ..., ] is a sequence of arguments in which every element of the sequence is a defeater of its predecessor (except for the rfist argument). A dialectical tree provides a structure for considering all possible acceptable argumentation lines that can be generated for deciding whether an argument is defeated (each path from the root to a leaf corresponds to a diferent acceptable argumentation line). This is called a dialectical tree because it represents an exhaustive dialectical analysis for the argument at its root.

Given a literal and an argument ⟨, ⟩ from a program , to decide whether is warranted, every node in the tree (⟨, ⟩) is recursively marked as (defeated) or (undefeated), obtaining a marked dialectical tree ′(⟨, ⟩) where: 1. All leaves in ′(⟨, ⟩) are marked as ; and 2. let be an inner node of ′(⟨, ⟩); then, is marked as if and only if every child of is marked as , i.e., all defeaters of are defeated.

Thus, node is marked as if and only if it has at least one child marked as , i.e., at least one defeater of is not defeated. Given an argument ⟨, ⟩ obtained from , if the root of ′(⟨, ⟩) is marked as , then ′(⟨, ⟩) warrants , and is warranted from .

Given a DeLP program , a query returns one of the following four possible answers: YES if is warranted from , NO if the complement of is warranted from , UNDECIDED if neither nor its complement are warranted from , or UNKNOWN if is not in the language of the program .

2.2. Comparison Criteria

As mentioned above, when two arguments attack each other, we must analyze which one prevails. Given an argument ⟨, ⟩ and a counter-argument ⟨ℬ, ⟩, a comparison criterion is used to determine if ⟨, ⟩ is preferred to ⟨ℬ, ⟩ and, therefore, defeats ⟨ℬ, ⟩. The definition of such a formal criterion is a central problem in structured argumentation, i.e., in those systems where the defeat relation must be computed from the structure of arguments. Although the comparison criterion used in DeLP is modular, the default one is Generalized Specificity [ 9, 11 ]. This criterion, intuitively, favors arguments with greater information content (classic specificity) or with less use of rules (a more direct derivation)—in other words, an argument is deemed better than another if it is more precise or more concise.

In the presence of presumptions, Generalized Specificity does not always have the intended results; for that reason, other preference criteria have been developed, such as Presumptionenabled Specificity [ 8 ]. According to this criterion, the accumulation of presumptive information in an argument weakens both the argument itself and its conclusion, meaning that given two presumptive arguments, if one uses a subset of the presumptions used by the other one, the former is considered to be a better argument. Figure 2 shows a simple example of the use of each criterion.

Towards other types of comparison criteria. The criteria discussed above are examples of domain-independent argument comparison criteria; however, in real life applications it may be necessary to introduce a criterion that depends on the particular domain and usage of information. In [ 7 ], an alternative general criterion is proposed assuming it is possible to elicit a preference order among the set of defeasible rules in the program, which is later lifted to a preference order among arguments that can be built from it.

In [ 12 ], an approach to handle multiple argument preference criteria is proposed in the setting of argumentation-based recommender systems using DeLP as the underlying knowledge representation and reasoning framework. The proposal allows to change the criterion that can be used in the comparison process, and even to use several criteria simultaneously that can be combined through adequate operators. Nevertheless, establishing which criteria should be used, and determining whether it needs to change over time, is not a trivial task. In the next section, we develop our proposal that seeks to learn domain (or application) specific criteria using neural networks.

3. Training a Neural Network to Compare Arguments

3.1. General Overview

Our methodology for training neural network models is comprised of three steps. First, we use a synthetic DeLP program generator [ 13 ] to produce a set of programs. Then, for each generated program, and using a DeLP solver [ 10, 14 ], we generate a list of pairs of arguments compared during the inference process indicating which one was chosen by the solver as a defeater. Finally, we train the neural network, having as input the pairs of arguments and a scalar as output, indicating which argument is preferred (or if neither is, which occurs in the

DeLP program

DeLP Solver

Neural

Network

Argument pairs Defeater

Dialectical tree

case of blocking defeaters).

Figure 3 illustrates our vision of how a neural network and the DeLP solver can be integrated, where the former is trained to learn a criterion and carry out the comparison between arguments, while the latter carries out the dialectical analysis.

3.2. Synthetic Program Generation and Experimental Settings

For the programs obtained from the DeLP generator, we consider two criteria: Generalized Specificity and Presumption-enabled Specificity . In addition, we explored two types of program settings: complexity (simple or complex programs) and the presence of blocking defeaters. The complexity of programs depends mainly on the following parameters: • MAX_BODYSIZE (sets the highest value in the average length of the defeasible derivation of any argument), • MIN_ARGSLEVEL (sets the number of dialectical trees), • RAMIFICATION (sets the number of argumentation lines), and • TREE_HEIGHT (sets the length of argumentation lines).

We suggest the reader to see [ 13 ] for more details on the synthetic DeLP program generating processes. The presence of blocking defeaters indicates whether or not they are considered when generating the pairs of arguments to be compared.

For each setting combination, three datasets were generated with diferent sizes: 200, 500, and 1000 programs. Table 1 shows the average number of argument pairs per program based on the program configuration described above.

3.3. Encoding Arguments

In order to be amenable to neural training methods, we need to develop a vector encoding of arguments obtained from DeLP programs, where the encoding is done in such a way that the connection between internal components is adequately represented.

Complexity

Simple Simple Complex Complex

Blocking × ✓ × ✓ #Argument pairs per program 9.63 18.80 57.07 243.52

The network input is constructed by encoding a pair of conflicting arguments into a single tensor, with the following conventions (illustrated in Figure 4): • Value “1” is used to represent defeasible rules, while “− 1” is used to represent strict rules. • Each atom is represented with an id consisting of an integer value strictly greater than 2.

The same integer will be used to represent the same atom in the context of a pair of conflicting arguments. The negation of an atom is represented by the negative counterpart of the number representing that atom. • Rules and atoms are separated using “2”, while arguments are separated using value“− 2”. • Arguments have a fixed length so if the representation of the argument is smaller than that fixed length, zeros are used to fill the unused space and the neural network ignores them. • For the output, we have three possible values: – 0 if the first argument is preferred over the second one, – 1 if the second argument is preferred over the first one, and – 2 if no argument is preferred over the other, i.e., the arguments are blocking defeaters.

As mentioned, we developed this encoding because it maintains the relationship between the inner components of the arguments, and it allows each argument to be represented as a tensor of the same length. Algorithms 1 and 2 describe the full processes of encoding a single argument and the construction of the input, respectively.

The following example shows how a pair of conflicting arguments are encoded and how the input is built from both encodings. Figure 4 provides a graphical illustration of the example. Example 1. Let ⟨, 8⟩ and ⟨ℬ, ∼ 8⟩ be two conflicting arguments.

In this case, literal 8 is represented with value 7, 0 with 8, and 9 with 9, while 8 is represented with − 7. The symbo l≺− “ − ” is represented with value 1, while − 1 is used after the value 8 to indicate that 0 is a fact. Rules are separated from facts and presumptions using the value 2, and both arguments are concatenated on the final input vector adding a value of − 2 in between. ⟨, 8⟩ Encoding of ⟨, 8⟩ : [(a 8≺− − a0 ), 0] : [7, 1, 8, 2, 8, − 1, 0, ..., 0] ⟨ℬ, ∼ 8⟩ : [(∼ a 8≺− − d9 ), (d 9≺− − true)]

Encoding of ⟨ℬ, ∼ 8⟩ : [− 7, 1, 9, 2, 9, 1, 0, ..., 0] 8 -1 Argument 1

Argument separator

Argument 2 7 1 8 2 encodedArgument ← [ ] for each element in Argument do encodedHead ← encodedArgument .append (encodedHead )

LiteralToInteger (element .head ) if element .isFact or element .isStrictRule then

encodedRuleType ← − 1 else

encodedRuleType ← 1 encodedArgument .append (encodedRuleType) for each literal in element .body do

encodedArgument .append (LiteralToInteger (literal )) if element is not Argument .last then

encodedArgument .append (2) return encodedArgument For the output, the value 0 represents that ⟨, 8⟩ defeats ⟨ℬ, ∼ 8⟩:

Input : [7, 1, 8, 2, 8, − 1, 0, ..., 0, − 2, − 7, 1, 9, 2, 9, 1, 0, ..., 0]

Output : 0

After presenting the neural encoding, in the next section we discuss the neural architecture proposed and used in our empirical evaluuation.

3.4. Neural Network Architecture

For this preliminary proposal, we used a simple neural network architecture with 3, 000 parameters (10 linear, fully connected hidden layers with 300 nodes each). The output of the neural Algorithm 2: EncodeNetworkInput

Input: Argument1 , Argument2 encodedArgument1 ←

EncodeArgument (Argument1 ) encodedArgument2 ←

FillToMaxLength(encodedArgument1 )

EncodeArgument (Argument2 ) input ← input .append (− 2) input .append (FillToMaxLength(encodedArgument2 )) returNn ienpuutral network training - Datasets Complexity e l p m i S x e l p m o C

Blocking defeaters Blocking defeaters s e Y o N s e Y o N

Presumptions Presumptions Presumptions Presumptions s e Y o N s e Y o N s e Y o N s e Y o N

Amount of programs Amount of programs Amount of programs Amount of programs Amount of programs Amount of programs Amount of programs Amount of programs network depends on the characteristics of the arguments being processed: In the next section, we discuss the details of our experimental evaluation using this setup.

4. Experimental Evaluation

Dataset. Diferent datasets were created combining all possible values of the following settings: with single or complex programs, with or without blocking defeaters, using Presumption-enabled 1

Specificity or Generalized Specificity, and the number of programs from which arguments were extracted. Figure 5 shows an overview of the diferent datasets considered.

Experimental Setup. For each dataset, a 3-fold cross-validation was done by training three networks with three diferent random partitions of train and test, respecting a proportion of 80% and 20% respectively.

Results. Figures 6 and 7 show the average Accuracy, Precision, Recall, and F1 scores of the three models along with the standard deviation of the F1 scores for both criteria.

Our results show that a simple neural network architecture with 3, 000 parameters can obtain close to perfect performance in learning two diferent comparison criteria for arguments of DeLP programs of varying complexity with only proper defeaters by analyzing examples of argument comparisons. If programs have also blocking defeaters, performance is still very good: models achieve high accuracy, precision, recall, and F1 score for datasets generated from simple programs and good accuracy score also for complex programs, though precision, recall, and F1 scores drop, indicating a bias towards the majority class, the blocking defeaters.

Larger datasets created with more programs perform slightly better. Although the improvement is in general very small, it was consistent in all our experiments. Complex programs have worse performance than simple programs only in the presence of blocking defeaters and/or presumptions. Finally, the large diference in performance between blocking and non-blocking datasets could be explained because of the diferent types of classification done while training with them, using binary vs. multi-class classification.

5. Related Work

In general terms, this work falls within the area known as Neuro Symbolic Artificial Intelligence (NeSy AI) [ 15, 16 ], a rich field that encompasses the combination of deep neural networks with symbolic logic for reasoning and learning. Currently, NeSy AI frameworks are capable of carrying out a variety of tasks such as incorporating prior knowledge into deep learning architectures, guiding the learning process with logical constraints, providing symbolic explainability, and using gradient-based approaches to learn logical statements.

More specifically, this paper focuses on Argumentation and Machine Learning, an area that aims to integrate the advantages of both fields to address their limitations (for a survey, see [ 1 ]). These two fields can greatly benefit from each other: argumentation can improve machine learning in several ways, e.g., by providing it with explainability [ 17, 18 ], and machine learning can aid argumentation by building arguments from real data, as in argument mining [ 19, 20 ]. The integration of argumentation and machine learning has been proven to be fruitful in diferent domains, like social network analysis [ 21 ], healthcare [ 22, 17 ], law [ 23, 22 ], and security [ 24, 25 ].

Finally, the following line of work is closely related to our own, though it is essentially diferent in spirit. In [ 26 ] an algorithm for translating an agent’s knowledge base, expressed as an ODeLP rule base, into a Perceptron-based neural network is introduced. In [ 27, 28, 29 ] the authors propose combining a set of criteria specified as a DeLP program with a Fuzzy ART neural network model for solving ambiguities in clustering problems, while [ 30 ] also proposes to combine a counter-propagation neural network with a DeLP program for filtering HTML

6. Conclusions and Future Work

DeLP and neural networks are powerful knowledge representation and problem-solving tools whose combination can be very fruitful. In this preliminary work, we have investigated the feasibility of training a neural network to learn a comparison criterion, obtaining promising results not only for proper defeaters but also when including blocking ones. Particularly, supporting our hypotheses we can say that: 1. Experimental results provide support for Hypothesis 2, since evidence shows that performance of the neural network is afected when the complexity of the program or the comparison criterion increases. 2. With respect of the number of examples being used, we can’t confirm that more examples lead to better performance since we obtained similar results on the experiments with only proper defeaters, and the experiments with blocking defeaters showed a decrease in performance with larger datasets.

As future work, we are interested in studying how other, possibly arbitrary and domaindependent comparison criteria can be learned from data. Towards this end, we are exploring how we can address the general lack of real-world datasets by deriving rules from text in natural language, for which we can take advantage of recent developments in Large Language Models (LLMs). Previous successes in applying DeLP to real-world programs like cyber attribution [ 24 ] support working in this direction, where instead of relying on knowledge engineering eforts we might develop general templates that can be instantiated en masse and then, after automatically deriving sets of arguments, these can be labeled based on available ground truth. It will be interesting to analyze whether the classical comparison criteria used in this work come up in such scenarios, or perhaps close variants or entirely new ones arise.

We are also interested in studying possible interactions between the dialectical analysis of arguments that DeLP carries out and neural computation architectures. Finally, it would be interesting to explore combining our models with diferent attention mechanisms.

ACKNOWLEDGMENTS

This work was supported by funds provided by Universidad Nacional del Sur (UNS) under grants PGI 24/N046 and PGI 24/ZN057), Agencia Nacional de Promoción Científica y Tecnológica under grants PICT-2018-0475 (PRH-2014-0007) and PICT-2020-SERIEA-01481, and CONICET under grants PIP 11220200101408CO and PIP 11220170100871CO. The authors also acknowledge support by the Spanish project PID2022-139835NB-C21 funded by MCIN/AEI/10.13039/501100011033, MCIN/AEI/ 10.13039/501100011033 and by “European Union NextGenerationEU/PRTR”.

The authors would like to thank Mario A. Leiva for providing the code for the DeLP program generator that he developed [ 13 ], and for his help during its deployment in this project.

References

[1] O. Cocarascu , F. Toni , Argumentation for machine learning: A survey. , in: Proc. COMMA , 2016 , pp. 219 - 230 .

[2] T. M. Mitchell , T. M. Mitchell, Machine learning , volume 1 , McGraw- hill New York, 1997 .

[3] K. Gurney , An introduction to neural networks , CRC press, 2018 .

[4] J. L. Pollock , Defeasible reasoning , Cognitive science 11 ( 1987 ) 481 - 518 .

[5] S. Badreddine , A. d. Garcez, L. Serafini , M. Spranger , Logic tensor networks , Artificial Intelligence 303 ( 2022 ) 103649 .

[6] R. Riegel , A. Gray , F. Luus , N. Khan , N. Makondo , I. Y. Akhalwaya , H. Qian , R. Fagin , F. Barahona , U. Sharma , et al., Logical neural networks , arXiv preprint arXiv: 2006 . 13155 ( 2020 ).

[7] A. J. García , G. R. Simari , Defeasible logic programming: An argumentative approach, Theory and practice of logic programming 4 ( 2004 ) 95 - 138 .

[8] M. V. Martinez , A. J. García , G. R. Simari , On the use of presumptions in structured defeasible reasoning , in: Proc. COMMA , IOS Press, 2012 , pp. 185 - 196 .

[9] G. R. Simari , R. P. Loui , A mathematical treatment of defeasible reasoning and its implementation , Artificial intelligence 53 ( 1992 ) 125 - 157 .

[10] A. J. García , G. R. Simari , Defeasible logic programming: DeLP-servers, contextual queries, and explanations for answers , Argument & Computation 5 ( 2014 ) 63 - 88 .

[11] F. Stolzenburg , A. J. García , C. I. Chesñevar , G. R. Simari , Computing Generalized Specificity, J. Appl. Non Class. Logics 13 ( 2003 ) 87 - 113 .

[12] J. C. Teze , S. Gottifredi , A. J. García , G. R. Simari , An approach to generalizing the handling of preferences in argumentation-based decision-making systems , Knowl. Based Syst . 189 ( 2020 ).

[13] M. A. Leiva , Practical Tools for Structured Probabilistic Argumentation with Applications to Cybersecurity (in Spanish) , Ph.D. thesis , Departament of Computer Science and Engineering, Universidad Nacional del Sur (UNS) , 2022 . URL: https://repositoriodigital.uns.edu. ar/handle/123456789/6314.

[14] M. A. Leiva , G. I. Simari , S. Gottifredi , A. J. Garcia , G. R. Simari , DAQAP: defeasible argumentation query answering platform , in: International Conference on Flexible Query Answering Systems , Springer, 2019 , pp. 126 - 138 .

[15] P. Shakarian , C. Baral , G. I. Simari , B. Xi , L. Pokala, Neuro Symbolic Reasoning and Learning , Springer, 2023 .

[16] P. Hitzler , M. K. Sarker , Neuro-symbolic artificial intelligence: The state of the art ( 2022 ).

[17] N. Prentzas , A. Nicolaides , E. Kyriacou , A. Kakas , C. Pattichis, Integrating machine learning with symbolic reasoning to build an explainable AI model for stroke prediction , in: Proc. BIBE , IEEE, 2019 , pp. 817 - 821 .

[18] A. Vassiliades , N. Bassiliades , T. Patkos, Argumentation and explainable artificial intelligence: a survey, The Knowledge Engineering Review 36 ( 2021 ) e5 .

[19] M. Lippi , P. Torroni , Argumentation mining: State of the art and emerging trends , ACM Transactions on Internet Technology (TOIT) 16 ( 2016 ) 1 - 25 .

[20] J. Lawrence , C. Reed , Argument mining: A survey , Computational Linguistics 45 ( 2020 ) 765 - 818 .

[21] K. Grosse , M. P. Gonzalez , C. I. Chesnevar , A. G. Maguitman , Integrating argumentation and sentiment analysis for mining opinions from Twitter , AI Comm . 28 ( 2015 ) 387 - 401 .

[22] M. Možina , J. Žabkar , I. Bratko, Argument based machine learning , Artificial Intelligence 171 ( 2007 ) 922 - 937 .

[23] M. Možina , J. Žabkar , T. Bench-Capon , I. Bratko, Argument based machine learning applied to law , Artificial Intelligence and Law 13 ( 2005 ) 53 - 73 .

[24] E. Nunes , P. Shakarian , G. I. Simari , A. Ruef , Argumentation models for cyber attribution , in: R. Kumar , J. Caverlee , H. Tong (Eds.), Proc. ASONAM 2016 , IEEE Computer Society, 2016 , pp. 837 - 844 .

[25] M. Bishop , C. Gates , K. Levitt, Augmenting machine learning with argumentation , in: Proceedings of the New Security Paradigms Workshop , 2018 , pp. 1 - 11 .

[26] S. A. Gómez , Modelling derivation in Defeasible Logic Programming with perceptron-based neural networks , in: Proc. CACIC , 2004 .

[27] S. A. Gómez , C. I. Chesñevar , Combining argumentation and clustering techniques in pattern classification problems , in: Proc. CACIC , 2003 .

[28] S. A. Gómez , C. I. Chesñevar , A hybrid approach to pattern classification using neural networks and defeasible argumentation ., in: FLAIRS conference, 2004 , pp. 393 - 398 .

[29] S. A. Gómez , C. I. Chesñevar , Integrating defeasible argumentation with fuzzy art neural networks for pattern classification , Journal of Computer Science & Technology , 2004 , vol. 4 , núm . 1, p. 45 - 51 ( 2004 ).

[30] S. A. Gómez , C. I. Chesñevar , Combining counterpropagation neural networks and defeasible logic programming for text classification , in: Proc. WICC , 2004 .