<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta>
      <journal-title-group>
        <journal-title>Italian Conference on Theoretical Computer Science, September</journal-title>
      </journal-title-group>
    </journal-meta>
    <article-meta>
      <title-group>
        <article-title>Recurrent Neural Networks for Guiding Proof Search in Propositional Logic</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Gianluca Amato</string-name>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Nicola Balestra</string-name>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Marco Maggesi</string-name>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Maurizio Parton</string-name>
        </contrib>
      </contrib-group>
      <pub-date>
        <year>2025</year>
      </pub-date>
      <volume>1</volume>
      <fpage>0</fpage>
      <lpage>12</lpage>
      <abstract>
        <p>DeepSAT is an ongoing research project investigating the use of deep reinforcement learning for automated theorem proving in propositional logic. In contrast to traditional SAT solvers, which focus on satisfiability checking, our aim is to construct formal proofs of validity within the sequent calculus framework. As a preliminary step toward this goal, we have focused on training recurrent neural networks to predict whether a given propositional formula is a tautology. These models will form the basis of the value function in the planned reinforcement learning architecture. Although in its early stages, DeepSAT lays the groundwork for a logicagnostic, explainable, and energy-eficient alternative to existing neural approaches based on large language models, which require substantial computational resources.</p>
      </abstract>
      <kwd-group>
        <kwd>eol&gt;Propositional logic</kwd>
        <kwd>Automated and interactive theorem proving</kwd>
        <kwd>Deep learning</kwd>
        <kwd>Recurrent neural networks</kwd>
      </kwd-group>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>1. Introduction</title>
      <p>
        Automated theorem proving is a longstanding challenge at the intersection of logic, artificial intelligence,
and symbolic reasoning. In propositional logic, the state of the art is dominated by SAT solvers [
        <xref ref-type="bibr" rid="ref1">1</xref>
        ],
which reduce the task of proving validity to checking Boolean satisfiability. These tools are remarkably
fast and efective.
      </p>
      <p>
        In contrast, neural approaches to reasoning have gained increasing attention for their ability to
learn inference patterns directly from data [
        <xref ref-type="bibr" rid="ref2 ref3">2, 3</xref>
        ]. These models do not follow explicit logical rules, but
instead learn internal representations that approximate logical behavior, ofering a complementary and
potentially more flexible approach.
      </p>
      <p>
        Our aim is to develop a system for automated proof search in propositional logic. This paper presents
a hybrid system that combines a neural model with a symbolic proof engine based on the sequent
calculus. As a first step, we train a Tree Long Short-Term Memory (Tree-LSTM) [
        <xref ref-type="bibr" rid="ref4">4</xref>
        ] classifier to predict
whether a given propositional formula is a tautology. This classifier is initially developed and evaluated
independently as a supervised model. In a second phase, it is embedded into a Monte Carlo Tree search
(MCTS) [
        <xref ref-type="bibr" rid="ref5">5</xref>
        ] agent, where it serves as a policy-value function to guide deductive proof search. Since the
ultimate goal of the search is to construct a valid proof, estimating the likelihood that a formula is a
tautology ofers a meaningful heuristic for both action selection and value estimation within the MCTS
framework.
      </p>
      <p>The architecture of the Tree-LSTM classifier leverages the hierarchical structure of logic syntax:
formulas are represented as trees, enabling this model to learn compositional embeddings aligned with
their syntactic structure. The symbolic component, in turn, applies inference rules over sequents to
construct formal derivations. We evaluate this model both as a standalone classifier and as a heuristic
guide within the MCTS agent.</p>
      <p>The Python code, the dataset used for the experiments and the trained models can all be found on
our git repository at the URL https://github.com/CLAI-UdA/deepsat-experiments/tree/ictcs2025.</p>
    </sec>
    <sec id="sec-2">
      <title>2. The DeepSAT Project</title>
      <p>DeepSAT is a broader research initiative in which the present work is situated. Its central idea is to
frame theorem proving as a sequential decision process, where each state corresponds to a collection
of sequents, and each action applies an inference rule to advance the proof. Neural networks provide
heuristics in the form of a policy — guiding the choice of inference steps — and a value function —
estimating the provability of a sequent. These components are integrated into a tree search procedure
such as Monte Carlo Tree Search.</p>
      <p>While we acknowledge that this approach is unlikely to compete with SAT solvers in terms of
raw performance, DeepSAT ofers several potential advantages in comparison to subsymbolic neural
approaches, particularly large-scale language models:
Logic-agnosticism. The architecture is not tied to classical propositional logic and can, in principle,
be adapted to non-classical logics such as intuitionistic or linear logic without major structural
changes.</p>
      <p>Extensibility. The system can be extended to support first-order logic, potentially serving as an
intelligent assistant in interactive proof development.</p>
      <p>Transparency and accessibility. Unlike solutions developed within large-scale industrial research
frameworks (e.g., by OpenAI or DeepMind), DeepSAT is designed around principles of openness
and explainability.</p>
      <p>These features are not intended to outperform traditional symbolic solvers, which remain the most
eficient and reliable tools for large-scale automated reasoning. Rather, DeepSAT is conceived as a
structured and interpretable alternative to purely neural approaches, particularly those based on large
language models. The goal is not to replace symbolic reasoning, but to complement it with learned
guidance that remains transparent, adaptable, and logically grounded.</p>
      <p>Specifically, the project embraces the following key principles:
Open-source. All software components, trained models, and data will be made publicly available.
Energy eficiency. The system is designed to operate with a fraction of the computational resources
typically required for training and inference in larger models.</p>
      <p>Explainability. Although human-readability of the produced proofs is not our primary goal, we aim
to design architectures whose internal functioning is more interpretable, owing to the need to
operate under constrained computational budgets. This will require novel strategies for the
representation and manipulation of formulas and sequents, tailored to the structure of logical
reasoning tasks.</p>
      <p>This paper represents a first step towards the broader goal of the DeepSAT project. We focus in
particular on the implementation of the value function.</p>
    </sec>
    <sec id="sec-3">
      <title>3. RNN for Predicting Truth Values of Formula</title>
      <p>We present a method for classifying propositional formulas as tautologies or non-tautologies using
a neural network architecture. This approach is based on a Tree-LSTM model, which is specifically
designed to operate on tree-structured data, making it well-suited for the recursive and compositional
nature of logical formulas.</p>
      <p>
        The central idea is to represent each formula as a binary syntax tree that captures its hierarchical
structure, and to process this tree using a recursive neural network that can learn to recognize various
logical patterns. The Tree-LSTM model, following the architecture proposed by Tai et al. [
        <xref ref-type="bibr" rid="ref4">4</xref>
        ],
computes vector representations for each node of the syntax tree in a bottom-up manner, combining the
information of the subformulas through a gated mechanism derived from standard LSTM cells [
        <xref ref-type="bibr" rid="ref6">6</xref>
        ].
      </p>
      <p>To train and evaluate the model, we construct a synthetic dataset of well-formed propositional
formulas labeled as tautologies or non-tautologies by means of exhaustive truth table evaluation. To
address class imbalance and ensure that well-known tautologies are well represented during training,
we perform targeted data augmentation using instantiated tautology schemata.</p>
      <sec id="sec-3-1">
        <title>3.1. Formula Representation and Dataset Generation</title>
        <p>In our framework, each formula is recursively defined in terms of atomic components (propositional
variables and logical constants) and logical connectives, according to the formal syntax of propositional
logic.</p>
        <p>Atomic formulas correspond to individual propositional variables, each assigned a unique numerical
identifier. Compound formulas are constructed recursively by applying logical connectives — such as
negation (¬) and binary connectives (∧, ∨, →) — to one or more subformulas.</p>
        <p>To train and evaluate the model, we constructed a synthetic dataset of normalized propositional
formulas, generated through a controlled recursive process, with bounded depth and a limited set of
propositional variables.</p>
        <p>The generation follows a recursive pattern: at each level, the formula may be the Boolean constant
false (⊥), a propositional variable (), or a compound formula.</p>
        <p>Once generated, a normalization procedure is applied to each formula, renaming propositional
variables in order of first appearance with sequential indices starting from 0. This step reduces superficial
syntactic redundancy, allowing the model to focus on logical structure rather than on arbitrary variable
names.</p>
        <p>Each formula is labeled either as a tautology or non-tautology by exhaustively evaluating all possible
truth assignments. A formula is labeled as a tautology if it is evaluated true under every possible
assignment of truth values. Given the limited number of propositional variables, this method is
computationally feasible. For future extensions involving a larger variable space, we plan to adopt SAT
solvers for more scalable and eficient labeling. The final dataset consists of 10, 000 distinct formulas,
each paired with its corresponding Boolean label.</p>
        <p>However, the initial distribution of tautologies was highly imbalanced: only 4.18% of the generated
formulas were tautologies. This imbalance stems from the random generation process itself: since
tautologies constitute only a small fraction of the space of well-formed formulas, they are rarely
produced by chance. Such imbalance risks biasing the model during training, potentially leading it to
ignore the minority class and underperform on tautological instances. Furthermore, an analysis of the
dataset revealed the absence of canonical tautologies such as the Law of the Excluded Middle and Law
of Non-Contradiction.</p>
        <p>To make the dataset more representative, we introduced a data augmentation phase based on the
generation of new examples from known tautological schemata. For this purpose, we defined a collection
of tautological schemata containing metavariables. These schemata include the following examples
from classical logic: Reflexivity (  → ), Law of the Excluded Middle, Law of Non-Contradiction, De
Morgan’s Law for Conjunction, De Morgan’s Law for Disjunction, De Morgan’s Law for Disjunction,
Distribution of Conjunction over Disjunction, Distribution of Disjunction over Conjunction.</p>
        <p>To generate concrete instances, each metavariable in the schemata is replaced with a randomly
generated formula, and the result is finally normalized. The generation process produced formulas with
structural properties consistent with those of randomly generated examples in the dataset. This ensures
structural consistency between instantiated and randomly generated formulas.</p>
        <p>In total 3, 000 new tautological formulas were generated and included as independent examples
in the dataset, increasing the proportion of tautologies to 26.29%. Even with the additions of these
formulas, the dataset was still unbalanced. To address this challenge, we adopted an asymmetric loss
function, described in Section 3.4.</p>
      </sec>
      <sec id="sec-3-2">
        <title>3.2. Preprocessing Data</title>
        <p>After dataset generation, each propositional formula is converted into a binary syntax tree that reflects
its compositional logic structure. In this representation, each node  corresponds to a subformula
 ∈ ℱ , where ℱ is the set of well-formed formulas. The children of each node represent the arguments
of the main connective in  . This tree structure is required by the Tree-LSTM model, which processes
formulas recursively from leaves to root.</p>
        <p>To enable numerical processing by the model, to each node  is associated an index () ∈ N, which
depends on the main connective of the subformula  it contains, via a tokenization function  :
• For each propositional letter , assign the token
• For each logical connective □ ∈ {¬, ∧, ∨, →}, assign a token
 () =  + 1,</p>
        <p>with  ≥ 0
 (□ ) =
⎧⎪100, if □ is ¬
⎪
⎪
⎪⎨101, if □ is ∧
⎪102, if □ is ∨
⎪
⎪
⎪⎩103, if □ is →
 (⊥) = 105
• For the Boolean constant ⊥, assign the token</p>
        <p>The index () is stored in the embedding_index field of the node , and subsequently passed to a
learnable embedding layer that maps symbolic identifiers to dense vector representations. Specifically,
the embedding layer is parameterized by a matrix E ∈ R ×  , where  denotes the vocabulary size (i.e.,
the number of distinct tokens) and  is the embedding dimension. Given an index  = (), the model
retrieves the corresponding embedding vector by selecting the -th row of E, denoted  = E[] ∈ R .
This vector  is then used as input to the subsequent Tree-LSMT cell at node .</p>
        <p>The embedding matrix E is composed of learnable parameters and participates in gradient-based
optimization. At each training step, only the rows corresponding to input symbols are updated.</p>
        <p>An example of an embedding-annotated tree is shown in Figure 1.</p>
        <p>[Implication] A0 ∧ (A0 → A1) → A1 (embedding_index=103)
[Conjunction] A0 ∧ (A0 → A1) (embedding_index=100)
[Letter] A0 (embedding_index=1)
[Implication] A0 → A1 (embedding_index=103)
[Letter] A0 (embedding_index=1)
[Letter] A1 (embedding_index=2)
[Letter] A1 (embedding_index=2)</p>
        <p>For model training, each data example consists of a binary syntax tree whose nodes are annotated
with embedding indices, encoding the logical structure of the corresponding formula. Each tree is
paired with a Boolean label indicating whether the formula is a tautology or not.</p>
        <p />
        <p>×
left child</p>
        <p>ℎ</p>
        <p>×
+

tanh

tanh
×</p>
        <p>ℎ


×</p>
        <p>ℎ
right child
 and ℎ vectors. These outputs are then passed to the Tree-LSTM unit corresponding to a higher-level (outer)
node in the tree structure. Elliptic pink nodes represent simple pointwise operations, while rectangular yellow
nodes represent perceptron layers.</p>
      </sec>
      <sec id="sec-3-3">
        <title>3.3. Tree-LSTM Architecture</title>
        <p>vector  ∈ R.</p>
        <p>Tree-LSTM networks generalize standard recurrent architectures to hierarchical structures such as
syntax tree. In our setting, each formula is represented as a binary tree, and the model computes hidden
representation for each node by recursively combining information from its children. This enables the
model to capture long-range dependencies and structural relationships that arise in propositional logic
formulas.</p>
        <p>In the binary Tree-LSTM variant, each internal node receives an input vector  ∈ R and two pairs
of hidden and cell states, (ℎ, ) and (ℎ, ), from its left and right children, respectively. At each
node in the formula tree, the Tree-LSTM performs a local computation — referred to as a Tree-LSTM
unit — which outputs a hidden state vector ℎ ∈ R and a memory cell vector  ∈ R, where  is the
hidden dimension. These two vectors encode the entire subtree rooted at that node and are recursively
propagated upward to the parent node.</p>
        <p>To compute these vectors, the Tree-LSTM unit maintains five internal components: an input gate
 ∈ R, two forget gates ,  ∈ R for the left and right children, an output gate  ∈ R, and an update
The Tree-LSTM transition equations, depicted in Figure 2, are the following:
ℎcat = [ℎ, ℎ]
 =  ( () +  ()ℎcat + ())
 =  ( () +  ()ℎcat + ())
 =  ( () +  ()ℎcat + ())
 =  ( () +  ()ℎcat + ())
 = tanh( () +  ()ℎcat + ())
 =  ⊗  +  ⊗  +  ⊗ 
ℎ =  ⊗ tanh()</p>
        <p>Here,  is the input embedding of the current node,  denotes the logistic sigmoid activation function,
and ⊗ denotes elementwise multiplication. The hidden state vector ℎ can be interpreted as a distributed
representation of the subformula rooted at the node, integrating structural and semantic information
from its subtree.</p>
        <p>Conceptually, the input gate  controls the incorporation of new information into the memory cell;
the forget gates ,  regulate the contribution of information from the left and right child children,
respectively; and the output gate  determines which parts of the memory cell are exposed to the
hidden state. As in standard LSTMs, the gating mechanism enables the model to learn multiscale
representations and retain relevant information over long-range structural contexts.</p>
        <p>In our implementation, leaf nodes (e.g., propositional letters or constants) are initialized with
zerovalued hidden and cell states. Unary nodes (e.g., negations) replicate the same child’s state as both left
and right inputs, in order to maintain a consistent binary interface.</p>
        <p>The output of the Tree-LSTM — that is, the hidden state ℎ at the root of the tree — is passed through
a small multi-layer perceptron (MLP) consisting of a fully connected layer with ReLU activation, and a
ifnal single-neuron output layer for binary classification.</p>
      </sec>
      <sec id="sec-3-4">
        <title>3.4. Loss Function</title>
        <p>
          To address class imbalance in the dataset, we adopt a variant of Focal Loss known as Asymmetric Focal
Loss [
          <xref ref-type="bibr" rid="ref7">7</xref>
          ]. This loss function assigns greater penalty to false negatives in the minority class (tautologies)
by introducing independent weighting and focusing parameters for each class. The loss is defined as:
ℒ(, ^) = −  pos(1 − ^) pos log(^) −  neg(1 − )^ neg log(1 − ^)
where  pos,  pos are the weight and focal parameter for the tautology class, respectively; and  neg,  neg
are the corresponding parameters for the non-tautology class, respectively. The criteria used to select
the values of  and  for both classes are detailed in the experimental Section 5.
        </p>
        <p>This formulation represents one possible instantiation within a broader class of methods designed
to address class imbalance during training. By dynamically modulating the contribution of individual
training examples based on their dificulty, it mitigates the dominance of the majority class and places
increased emphasis on correctly classifying underrepresented tautologies. In doing so, it aims to enhance
recall on the minority class without significantly compromising overall performance.</p>
      </sec>
    </sec>
    <sec id="sec-4">
      <title>4. RNN in an MCTS Agent</title>
      <p>The Tree-LSTM classifier, described in the previous section, provides a value estimate for propositional
formulas in the form of a predicted probability of being a tautology. We integrate this trained model
into a Monte Carlo Tree Search (MCTS) agent for automated proof search. The goal is to replace pure
pattern-based classification with an agent capable of actively constructing proofs within the sequent
calculus framework, guided by a learned probability distribution over inference rules and an estimated
value of provability for each sequent.</p>
      <sec id="sec-4-1">
        <title>4.1. Proof Search as Guided (Hyper-)Tree Exploration</title>
        <p>Proof construction is framed as search over a dynamically growing tree. Each node represents a sequent
to be proved, and inference rules decompose it into child sequents. Diferent inference rules may be
applied to the same sequent. Therefore, we have two diferent kinds of branch, which have to be dealt
with diferently in checking provability and in cost aggregation: all sequents coming from the same
inference rule must be proved in order to prove the original sequent, while the success of a single
inference rule is enough. Therefore, what we will call proof tree is actually an hyper-tree in which every
edge has a source node and a finite set of target nodes. Alternatively, we can view proof-trees as and-or
trees.</p>
      </sec>
      <sec id="sec-4-2">
        <title>4.2. MCTS Architecture</title>
        <p>
          We implement a variant of Monte Carlo Tree Search that combines the symbolic structure of sequent
calculus with a learned neural policy and value. The MCTS agent performs a series of simulations, each
involving four phases:
• Selection: Starting from the root sequent, the agent recursively selects child nodes based on
a polynomial Upper Confidence Bound for Tree (PUCT) [
          <xref ref-type="bibr" rid="ref8">8</xref>
          ] formula. At each step, it selects
the move whose subgoals maximize a logic-sensitive utility, computed from visit counts, value
estimates, and neural prior probabilities. The traversal stops when a leaf node is reached.
• Evaluation: Upon reaching a leaf, the agent evaluates whether the current sequent is an axiom. If
so, the corresponding node is marked as proved and assigned maximum value. Otherwise, the
neural network is queried to produce a value estimate for provability and a distribution over
possible inference rules. These outputs guide expansion and backpropagation.
• Expansion: If the sequent is not an axiom, all applicable inference rules are applied to generate
subgoals. These are inserted into both the search tree. Each move is associated with the neural
prior probability produced in the evaluation step.
• Backpropagation: The values estimate is then propagated recursively back through the path taken
during selection. Multiple children of the same move (hyper-edge) are aggregated by taking the
minimum value, while diferent moves are aggregated by taking their maximum value.
        </p>
        <p>This architecture implements a hybrid approach in which symbolic rule applications are guided by
probabilities. The neural model steers the exploration toward plausible proof paths, while the MCTS
framework ensures that all constructed proofs adhere to the formal semantics of the Sequent Salculus.
The next section presents the results of our experiments with this architecture.</p>
      </sec>
    </sec>
    <sec id="sec-5">
      <title>5. Experiments</title>
      <p>In this section we present the result of our experiments on both the Tree-LTSM model and the MCTS
agent. In particular, Sections 5.1 to 5.6 present the tuning process and the final results of the Tree-LTSM
model, while Section 5.7 deal with the MCTS agent, with either a random policy or a policy derived
from the Tree-LTSM model.</p>
      <p>All experiments were conducted on the synthetic dataset described in Section 3, generated using a
controlled recursive procedure with a maximum syntax tree depth of 5 and up to 7 distinct propositional
variables. This limit was chosen as a trade-of between the expressive richness of the generated formulas
and their interpretability, both for training analysis and for proof traceability. As discussed, the final
dataset includes 13,000 formulas labeled as either tautologies or non-tautologies, with the proportion of
tautologies equal to 26.29%. The dataset was randomly split into 80% for training and 20% for testing.</p>
      <p>All models were trained in an environment running on a virtualized Ubuntu 22.04.5 LTS server
equipped with an 8-core AMD EPYC processor, 31GB of RAM, and NVIDIA Tesla T4 GPU with 15GB of
memory. The system uses CUDA 12.6 and NVIDIA driver version 560.35.05.</p>
      <p>Training time for each Tree-LSTM variant was approximately 3 hours, while the GRU baseline —
trained with half the epochs — completed in approximately 1.5 hours.</p>
      <sec id="sec-5-1">
        <title>5.1. Baseline Model: Bidirectional GRU</title>
        <p>
          Before introducing the tree-based architecture, we first established a baseline for experiments, using
a sequential model based on stacked bidirectional GRU [
          <xref ref-type="bibr" rid="ref9">9</xref>
          ] layers. The model processes tokenized
formulas as sequences and aggregates bidirectional hidden states to perform binary classification.
        </p>
        <p>The architecture consists of an embedding layer of dimension 32, followed by two bidirectional GRU
layers: the first with 128 hidden units, and the second with 64 hidden units. The final hidden states
of the second GRU layer are concatenated and passed through a fully connected layer of size 32 with
ReLU activation, followed by an output layer with a single neuron for binary classification .</p>
        <p>
          The model was trained using the Adam optimizer [
          <xref ref-type="bibr" rid="ref10">10</xref>
          ] with a learning rate of 5 × 10− 4 and the
same Asymmetric Focal Loss employed in subsequent experiments. Specifically, we used  pos = 0.3,
 neg = 0.7,  pos = 3.0, and  neg = 1.5, based on a heuristic analysis of the class imbalance in the
training set. These values emphasize the penalty for false negatives in the minority class while avoiding
excessive gradient suppression in the majority class. These initial parameters were later refined through
a systematic grid search, as detailed in the following section.
        </p>
        <p>Although it lacks explicit access to the compositional tree structure of formulas, the GRU model
achieved promising results. Table 1 reports the metrics over 10 training epochs.</p>
        <sec id="sec-5-1-1">
          <title>Epoch Train Loss Train Acc. Test Loss Test Acc.</title>
          <p>These results indicate that even a sequential model can learn relevant regularities in propositional
formulas. However, we hypothesized that a model explicitly designed to exploit the tree structure of
formulas would be more efective at capturing logical composition and semantic abstractions. This
motivated the transition to the Tree-LSTM architecture.</p>
          <p>In the next section, we introduce a Tree-LSTM model that leverages the hierarchical syntax of logical
formulas to improve classification accuracy and generalization.</p>
        </sec>
      </sec>
      <sec id="sec-5-2">
        <title>5.2. Tree-LSTM: Hyperparameters Search</title>
        <p>To determine the best configuration for the Tree-LSTM classifier, we conducted a structured three-stage
grid search over a reduced subset of 1,000 formulas sampled from the full dataset, preserving the original
class distribution to ensure representativeness.</p>
      </sec>
      <sec id="sec-5-3">
        <title>5.3. Hidden Size and Fully Connected Layer Dimensions</title>
        <p>In the first stage, we varied the hidden state size of the Tree-LSTM unit and the dimension of the fully
connected layer. The embedding dimension was fixed at 32, and each trial was run for 5 epochs to
control computational cost.</p>
        <p>The parameters of the Asymmetric Focal Loss were initially set based on the same heuristic used for
the bidirectional GRU baseline:  pos = 0.3,  neg = 0.7,  pos = 3.0, and  neg = 1.5. We used the Adam
optimizer with a learning rate of 5 × 10− 4.</p>
        <p>The Tree-LSTM configuration with 128 hidden units and a fully connected layer of size 32 achieved
the best performance in this phase, reaching 92% test accuracy and a test loss of 0.0248.</p>
      </sec>
      <sec id="sec-5-4">
        <title>5.4. Asymmetric Focal Loss Parameters</title>
        <p>In the second stage, we fixed the architecture based on the results from the previous phase and varied
the class weights ( pos,  neg) and focusing parameters ( pos,  neg). The goal was to improve the model’s
ability to detect tautologies by refining the sensitivity of the loss function to class imbalance. The
best results were obtained with  pos = 0.25,  neg = 0.7,  pos = 3.0,  neg = 2.5. This configuration
achieved a test accuracy of 93% and a test loss of 0.0154.</p>
      </sec>
      <sec id="sec-5-5">
        <title>5.5. Learning Rate Optimization</title>
        <p>In the final stage, we optimized the learning rate, testing values from 1 × 10− 4 to 1 × 10− 3 using the
best model and loss parameters from the previous steps. The best performance was achieved with a
learning rate of 9 × 10− 4, which yielded a test accuracy of 93% and a test loss of 0.0112.</p>
        <p>This final configuration — combining architectural tuning, improved loss weighting, and learning
rate optimization — was adopted as the default setup for all experiments on the full dataset, discussed
in the next section.</p>
        <sec id="sec-5-5-1">
          <title>Sweep</title>
          <p>Hidden Size &amp; FC Layer (128, 32)
Focal Loss Params (  = 0.25,   = 3.0)
Learning Rate (9 × 10− 4)</p>
        </sec>
        <sec id="sec-5-5-2">
          <title>Test Accuracy</title>
        </sec>
        <sec id="sec-5-5-3">
          <title>Train Accuracy</title>
        </sec>
        <sec id="sec-5-5-4">
          <title>Test Loss Train Loss</title>
        </sec>
      </sec>
      <sec id="sec-5-6">
        <title>5.6. Final Training on the Full Dataset</title>
        <p>After selecting the optimal hyperparameters through grid search, we trained two Tree-LSTM variants
on the full dataset of 13,000 examples.</p>
        <sec id="sec-5-6-1">
          <title>5.6.1. Model without Dropout</title>
          <p>The first model corresponds to the final architecture determined in the previous sections and does not
include any dropout regularization. It was trained for 20 epochs and exhibited a stable and consistent
learning trajectory. The highest test accuracy was obtained at epoch 17, reaching 99.31%, with a
corresponding test loss of 0.0026. While training accuracy converged to 100% in the later epochs, test
accuracy remained consistently high across the training process. The test loss also stayed low and stable,
indicating that the model was able to generalize well despite the absence of explicit regularization.</p>
          <p>These results suggest that overfitting had minimal impact on overall performance.</p>
        </sec>
        <sec id="sec-5-6-2">
          <title>5.6.2. Model with Dropout</title>
          <p>To assess the impact of regularization, we trained a second variant that includes a dropout layer, with
dropout rate of 30%, after the first fully connected layer. The training configuration and number of
epochs were kept identical. This model also achieved high performance, with a peak test accuracy
of 99.08% at epoch 17. In the final epochs, test accuracy fluctuated between 98.35% and 99.08%, and
test loss varied in a narrow range between 0.0017 and 0.0066. Although these variations are not large
in absolute terms, they indicate a slightly less stable generalization behavior compared to the model
without dropout.</p>
          <p>Overall, both models performed strongly, but the version without dropout slightly outperformed the
regularized variant in terms of both peak accuracy and consistency. Based on these results, the model
without dropout was selected as the final version for evaluation and integration into the MCTS agent.</p>
        </sec>
        <sec id="sec-5-6-3">
          <title>5.6.3. Evaluation Metrics</title>
          <p>To better assess the classification performances of the final model, we computed a confusion matrix
and standard evaluation metrics focused on the tautology class. The confusion matrix obtained in the
test set is the following:
Based on this matrix, we derived the following metrics for the tautology class:</p>
          <p>Precision =</p>
          <p>Recall =
F1-score =</p>
          <p>TP 666</p>
          <p>=
TP + FP 666 + 4</p>
          <p>TP 666</p>
          <p>=
TP + FN 666 + 14
2 · Precision · Recall
Precision + Recall</p>
          <p>These results confirm that the model achieves strong performance not only in overall accuracy, but
also in terms of recall and precision on the tautology class — the more challenging and less frequent
class in the dataset.</p>
        </sec>
        <sec id="sec-5-6-4">
          <title>5.6.4. Qualitative Error Analysis</title>
          <p>Despite the model’s strong overall performance, a detailed inspection of the misclassifications reveals
certain limitations in generalization and abstraction capabilities. Table 5 reports the list of misclassified
formulas, grouped by type — false positives and false negatives — and accompanied by their predicted
confidence scores.</p>
          <p>These misclassifications reveal that the model struggles with recognizing tautologies that lack
syntactic regularity or that require a deeper level of semantic abstraction. Even structurally trivial
tautologies involving contradictions or logical identities may be missed if expressed in unusual forms.</p>
          <p>These residual errors illustrate the limitations of purely statistical generalization. While the model
performs well on structurally familiar inputs, it remains sensitive to syntactic variations — confirming
the absence of true deductive reasoning and the reliance on learned syntactic heuristics.</p>
        </sec>
      </sec>
      <sec id="sec-5-7">
        <title>5.7. Experiments with MCTS Agent</title>
        <p>The previous evaluation revealed that the Tree-LSTM classifier exhibits limitations in abstraction and
generalization — particularly when confronted with tautologies expressed in uncommon syntactic
forms.</p>
        <p>To test whether the learned value predictions could support structured inference, we embedded the
Tree-LSTM classifier within a symbolic agent based on Monte Carlo Tree Search (MCTS). This agent
navigates the proof space of the sequent calculus by applying inference rules to decompose sequents
into subgoals. The aim was to evaluate the model not in isolation, but as a guide within a search process
governed by logical constrains.</p>
        <p>Experiments were conducted on the subset of 680 tautological formulas extracted from the test set,
each encoded as a sequent with an empty set of premises and the formula as the conclusion. The agent’s
task was to find a proof for each tautology using a limited number of inference steps.</p>
        <p>Two diferent policy-value functions were tested:</p>
        <p>False Positives
Formula
(((0 → 1) ∨ ((¬2) → 3)) ∨ ((((4 ∨ 5) ∧ (6 → 7)) → (0 → (1 ∨ 2))) ∨ 3))
((0 ∨ 1) ∨ ((((2 → 3) ∧ 4) ∧ ((¬5) ∧ (6 → 7))) → (0 ∨ (1 ∧ (2 ∧ 3)))))
(((0 → 1) → (2 → 3)) ∨ (((4 ∧ (¬⊥)) → (¬(5 ∧ 6))) ∨ (¬0)))
((((0 → (1 ∨ 2)) ∨ (3 ∨ (4 ∧ 5))) ∨ 6) ∨ (((¬(¬7)) ∧ (0 ∧ (1 → 2))) ∨ 3))</p>
        <p>False Negatives</p>
        <p>• Uniform policy assigns equal prior to all applicable moves ad uses a constant value estimate of 0.5.
• Tree-LSTM model policy uses the Tree-LSTM classifier to estimate the probability of success for
each subgoal generated by legal inference moves, and assigns move priors accordingly.</p>
        <sec id="sec-5-7-1">
          <title>5.7.1. Uniform Policy Baseline</title>
          <p>We conducted a parameter sweep over two key settings: the number of MCTS simulations per step
playout ∈ {2, 3, 4, 5, 10} and the maximum number of proof steps allowed max_steps ∈ {3, 4, 5}.
The best performance achieved with the uniform policy was 63.24% accuracy at playout = 10 and
max_steps = 5, as shown in Table 6.</p>
        </sec>
        <sec id="sec-5-7-2">
          <title>5.7.2. Tree-LSTM Policy</title>
          <p>We then evaluated the MCTS agent guided by the Tree-LSTM classifier using the same configuration
values explored for the uniform policy. In this setup, the model assigns prior probabilities to inference
rules based on the predicted provability of the resulting subgoals.</p>
          <p>As shown in Table 6, the Tree-LSTM guided agent consistently outperformed the uniform baseline
across all tested configurations. With playout = 5 and max_steps = 5, the model-guided agent reached
a peak accuracy of 91.42%, compared to 56.76% for the uniform policy under the same conditions.
Strong performance was achieved even with minimal simulations per step, suggesting that the classifier
provides meaningful guidance that helps focus the search toward promising inference paths.</p>
        </sec>
      </sec>
    </sec>
    <sec id="sec-6">
      <title>6. Related Work</title>
      <p>
        Numerous studies have explored using machine-learning techniques to partially or fully assist in
constructing proofs for logical systems. A survey can be found in [
        <xref ref-type="bibr" rid="ref2">2</xref>
        ]. Here, we review the work most
closely related to ours in either aims or methodology:
      </p>
      <p>
        Tree-LSTM Policy
• Kusumoto et al. [
        <xref ref-type="bibr" rid="ref11">11</xref>
        ] is the work most similar to our. While both approaches are essentially
logicagnostic, we experiment with classical logic, while they focus on intuitionistic logic. Moreover,
they employ Graph Neural Networks instead of Tree-LSTMs and rely on a simple greedy search
rather than MCTS.
• Lample et al. [
        <xref ref-type="bibr" rid="ref12">12</xref>
        ] is, to our knowledge, the most advanced efort pursuing goals comparable
to those of the DeepSAT project. They, too, use MCTS adapted to hypertrees, but target richer
logical systems such as MetaMATH, Lean, and Equation. Their proof-space exploration is guided
by natural-language models built on the seq2seq paradigm [
        <xref ref-type="bibr" rid="ref13">13</xref>
        ]. Conversely, we want to avoid
natural-language processing to keep our models as lightweight as possible.
      </p>
    </sec>
    <sec id="sec-7">
      <title>7. Conclusion and Future Work</title>
      <p>In this paper, we have shown that it is possible to train a recurrent neural network to predict the validity
of propositional formulas. Furthermore, we have shown that a network trained in this manner can
efectively guide the exploration of a Monte Carlo Tree Search (MCTS) agent in the task of finding
proofs for sequents in propositional logic. This represents an initial step in the DeepSAT project, which
ultimately aims to develop a system capable of automatically proving theorems in type theory.</p>
      <p>One might wonder why formulas of the form 1 → 2 → . . . →  which frequently arise when
translating sequents into single formulas, are not well represented in the training data. This is a
consequence of the fact that, in this initial phase, the dataset was constructed independently of the proof
tasks. However, this limitation is expected to be naturally overcome in the next phase of the project:
by switching to reinforcement learning, the network will be trained directly on formulas encountered
during proof search episodes, which include precisely those implicative structures typical of sequents.
In this way, the training distribution will become progressively aligned with the actual inference tasks
the model is meant to solve.</p>
      <p>In the next phase of our project, we aim to address this and other limitations by transitioning from
supervised to reinforcement learning. The dataset of formulas currently used to train the neural network
in a supervised fashion will instead serve as a foundation for training, via reinforcement learning, a
neural network that computes the external policy of the MCTS agent. In parallel, we will continue
to train the network responsible for the external value estimate using supervised learning, with the
training data drawn from the formulas that appear in the intermediate nodes of the proof trees.</p>
      <p>
        Additionally, we plan to explore alternative neural network architectures, potentially incorporating
attention mechanisms as in [
        <xref ref-type="bibr" rid="ref14">14</xref>
        ] to improve the model’s ability to focus on relevant substructures
within complex formulas, or non-shared biases to improve generalization, as in [
        <xref ref-type="bibr" rid="ref15">15, 16</xref>
        ]. Also, the use
of variants of MCTS for and-or trees, such as [17, 18], may be explored. Finally, we plan to use static
analysis techniques [19] to formally verify the correctness of the code.
      </p>
    </sec>
    <sec id="sec-8">
      <title>Acknowledgments</title>
      <p>This work was partially supported by the PNRR project FAIR – Future AI Research (PE00000013), Spoke
9 - Green-aware AI, under the NRRP MUR program funded by the NextGenerationEU; Istituto Nazionale
di Alta Matematica – INdAM groups GNCS and GNSAGA.</p>
    </sec>
    <sec id="sec-9">
      <title>Declaration on Generative AI</title>
      <p>During the preparation of this work, the authors used X-GPT-4 to: Drafting content, Paraphrase and
reword, Improve writing style. After using these tools, the authors reviewed and edited the content as
needed and take full responsibility for the publication’s content.
IEEE Winter Conference on Applications of Computer Vision, 2024, p. 2798 – 2807. doi:10.1109/
WACV57701.2024.00279.
[16] C. Metta, M. Fantozzi, A. Papini, G. Amato, M. Bergamaschi, A. Fois, S. G. Galfrè, A. Marchetti,
M. Vegliò, M. Parton, F. Morandin, Increasing biases can be more eficient than increasing weights,
Advances in Data Analysis and Classification (2025). doi: 10.1007/s11634-025-00649-2.
[17] T. M. Moerland, J. Broekens, A. Plaat, C. M. Jonker, Monte carlo tree search for asymmetric trees,
2018. doi:10.48550/arXiv.1805.09218.
[18] E. Doe, M. H. M. Winands, D. J. N. J. Soemers, C. Browne, Combining Monte-Carlo tree search with
proof-number search, in: 2022 IEEE Conference on Games (CoG), IEEE Press, 2022, p. 206–212.
doi:10.1109/CoG51982.2022.9893635.
[19] G. Amato, M. Rubino, F. Scozzari, Inferring linear invariants with parallelotopes, Sci. Comput.</p>
      <p>Program. 148 (2017) 161–188. doi:10.1016/j.scico.2017.05.011.</p>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          [1]
          <string-name>
            <given-names>J. K.</given-names>
            <surname>Fichte</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D. L.</given-names>
            <surname>Berre</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Hecher</surname>
          </string-name>
          ,
          <string-name>
            <surname>S. Szeider,</surname>
          </string-name>
          <article-title>The silent (r)evolution of SAT, Commun</article-title>
          . ACM
          <volume>66</volume>
          (
          <year>2023</year>
          )
          <fpage>64</fpage>
          -
          <lpage>72</lpage>
          . doi:
          <volume>10</volume>
          .1145/3560469.
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          [2]
          <string-name>
            <given-names>Z.</given-names>
            <surname>Li</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J.</given-names>
            <surname>Sun</surname>
          </string-name>
          ,
          <string-name>
            <given-names>L.</given-names>
            <surname>Murphy</surname>
          </string-name>
          ,
          <string-name>
            <given-names>Q.</given-names>
            <surname>Su</surname>
          </string-name>
          ,
          <string-name>
            <given-names>Z.</given-names>
            <surname>Li</surname>
          </string-name>
          ,
          <string-name>
            <given-names>X.</given-names>
            <surname>Zhang</surname>
          </string-name>
          ,
          <string-name>
            <given-names>K.</given-names>
            <surname>Yang</surname>
          </string-name>
          ,
          <string-name>
            <given-names>X.</given-names>
            <surname>Si</surname>
          </string-name>
          ,
          <string-name>
            <surname>A</surname>
          </string-name>
          <article-title>Survey on Deep Learning for Theorem Proving</article-title>
          ,
          <year>2024</year>
          . URL: https://openreview.net/forum?id=zlw6AHwukB#
          <fpage>discussion</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          [3]
          <string-name>
            <given-names>M.</given-names>
            <surname>Crouse</surname>
          </string-name>
          ,
          <string-name>
            <given-names>I.</given-names>
            <surname>Abdelaziz</surname>
          </string-name>
          ,
          <string-name>
            <given-names>B.</given-names>
            <surname>Makni</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S.</given-names>
            <surname>Whitehead</surname>
          </string-name>
          ,
          <string-name>
            <given-names>C.</given-names>
            <surname>Cornelio</surname>
          </string-name>
          ,
          <string-name>
            <given-names>P.</given-names>
            <surname>Kapanipathi</surname>
          </string-name>
          ,
          <string-name>
            <given-names>K.</given-names>
            <surname>Srinivas</surname>
          </string-name>
          ,
          <string-name>
            <given-names>V.</given-names>
            <surname>Thost</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Witbrock</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Fokoue</surname>
          </string-name>
          ,
          <article-title>A Deep Reinforcement Learning Approach to First-Order Logic Theorem Proving</article-title>
          ,
          <source>Proceedings of the AAAI Conference on Artificial Intelligence</source>
          <volume>35</volume>
          (
          <year>2021</year>
          )
          <fpage>6279</fpage>
          -
          <lpage>6287</lpage>
          . doi:
          <volume>10</volume>
          .1609/aaai.v35i7.
          <fpage>16780</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          [4]
          <string-name>
            <given-names>K. S.</given-names>
            <surname>Tai</surname>
          </string-name>
          ,
          <string-name>
            <given-names>R.</given-names>
            <surname>Socher</surname>
          </string-name>
          ,
          <string-name>
            <given-names>C. D.</given-names>
            <surname>Manning</surname>
          </string-name>
          ,
          <article-title>Improved Semantic Representations From Tree-Structured Long Short-Term Memory Networks</article-title>
          , in: C.
          <string-name>
            <surname>Zong</surname>
          </string-name>
          , M. Strube (Eds.),
          <source>Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing</source>
          (Volume
          <volume>1</volume>
          :
          <string-name>
            <surname>Long</surname>
            <given-names>Papers)</given-names>
          </string-name>
          ,
          <source>Association for Computational Linguistics</source>
          , Beijing, China,
          <year>2015</year>
          , pp.
          <fpage>1556</fpage>
          -
          <lpage>1566</lpage>
          . doi:
          <volume>10</volume>
          .3115/v1/
          <fpage>P15</fpage>
          -1150.
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          [5]
          <string-name>
            <given-names>L.</given-names>
            <surname>Kocsis</surname>
          </string-name>
          ,
          <string-name>
            <given-names>C.</given-names>
            <surname>Szepesvári</surname>
          </string-name>
          ,
          <article-title>Bandit based Monte-Carlo planning</article-title>
          ,
          <source>in: Proceedings of the 17th European Conference on Machine Learning</source>
          ,
          <source>ECML'06</source>
          , Springer-Verlag, Berlin, Heidelberg,
          <year>2006</year>
          , p.
          <fpage>282</fpage>
          -
          <lpage>293</lpage>
          . doi:
          <volume>10</volume>
          .1007/11871842_
          <fpage>29</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          [6]
          <string-name>
            <given-names>S.</given-names>
            <surname>Hochreiter</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J.</given-names>
            <surname>Schmidhuber</surname>
          </string-name>
          ,
          <article-title>Long short-term memory</article-title>
          ,
          <source>Neural Computation</source>
          <volume>9</volume>
          (
          <year>1997</year>
          )
          <fpage>1735</fpage>
          -
          <lpage>1780</lpage>
          . doi:
          <volume>10</volume>
          .1162/neco.
          <year>1997</year>
          .
          <volume>9</volume>
          .8.1735.
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          [7]
          <string-name>
            <given-names>T.</given-names>
            <surname>Ridnik</surname>
          </string-name>
          ,
          <string-name>
            <given-names>E.</given-names>
            <surname>Ben-Baruch</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Noy</surname>
          </string-name>
          ,
          <string-name>
            <given-names>L.</given-names>
            <surname>Zelnik-Manor</surname>
          </string-name>
          ,
          <article-title>Asymmetric loss for multi-label classification</article-title>
          ,
          <year>2021</year>
          . doi:
          <volume>10</volume>
          .48550/arXiv.
          <year>2009</year>
          .
          <volume>14119</volume>
          .
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          [8]
          <string-name>
            <given-names>D.</given-names>
            <surname>Silver</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J.</given-names>
            <surname>Schrittwieser</surname>
          </string-name>
          ,
          <string-name>
            <given-names>K.</given-names>
            <surname>Simonyan</surname>
          </string-name>
          ,
          <string-name>
            <surname>I. Antonoglou</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Huang</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Guez</surname>
          </string-name>
          ,
          <string-name>
            <given-names>T.</given-names>
            <surname>Hubert</surname>
          </string-name>
          ,
          <string-name>
            <given-names>L.</given-names>
            <surname>Baker</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Lai</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Bolton</surname>
          </string-name>
          , et al.,
          <article-title>Mastering the game of Go without human knowledge</article-title>
          ,
          <source>Nature</source>
          <volume>550</volume>
          (
          <year>2017</year>
          )
          <fpage>354</fpage>
          -
          <lpage>359</lpage>
          . doi:
          <volume>10</volume>
          .1038/nature24270.
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          [9]
          <string-name>
            <given-names>M.</given-names>
            <surname>Schuster</surname>
          </string-name>
          ,
          <string-name>
            <given-names>K. K.</given-names>
            <surname>Paliwal</surname>
          </string-name>
          ,
          <article-title>Bidirectional recurrent neural networks</article-title>
          ,
          <source>IEEE Transactions on Signal Processing</source>
          <volume>45</volume>
          (
          <year>1997</year>
          )
          <fpage>2673</fpage>
          -
          <lpage>2681</lpage>
          . doi:
          <volume>10</volume>
          .1109/78.650093.
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          [10]
          <string-name>
            <given-names>D. P.</given-names>
            <surname>Kingma</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J.</given-names>
            <surname>Ba</surname>
          </string-name>
          ,
          <article-title>Adam: A method for stochastic optimization</article-title>
          ,
          <source>International Conference on Learning Representations (ICLR)</source>
          (
          <year>2015</year>
          ). URL: https://arxiv.org/abs/1412.6980. arXiv:
          <volume>1412</volume>
          .
          <fpage>6980</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref11">
        <mixed-citation>
          [11]
          <string-name>
            <given-names>M.</given-names>
            <surname>Kusumoto</surname>
          </string-name>
          ,
          <string-name>
            <given-names>K.</given-names>
            <surname>Yahata</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Sakai</surname>
          </string-name>
          ,
          <source>Automated Theorem Proving in Intuitionistic Propositional Logic by Deep Reinforcement Learning</source>
          ,
          <year>2018</year>
          . doi:
          <volume>10</volume>
          .48550/arXiv.
          <year>1811</year>
          .
          <volume>00796</volume>
          .
        </mixed-citation>
      </ref>
      <ref id="ref12">
        <mixed-citation>
          [12]
          <string-name>
            <given-names>G.</given-names>
            <surname>Lample</surname>
          </string-name>
          , M.
          <article-title>-</article-title>
          <string-name>
            <surname>A. Lachaux</surname>
            ,
            <given-names>T.</given-names>
          </string-name>
          <string-name>
            <surname>Lavril</surname>
            ,
            <given-names>X.</given-names>
          </string-name>
          <string-name>
            <surname>Martinet</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          <string-name>
            <surname>Hayat</surname>
            ,
            <given-names>G.</given-names>
          </string-name>
          <string-name>
            <surname>Ebner</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          <string-name>
            <surname>Rodriguez</surname>
          </string-name>
          , T. Lacroix,
          <article-title>Hypertree proof search for neural theorem proving</article-title>
          ,
          <source>in: Proceedings of the 36th International Conference on Neural Information Processing Systems</source>
          , NIPS '22, Curran Associates Inc.,
          <string-name>
            <surname>Red</surname>
            <given-names>Hook</given-names>
          </string-name>
          ,
          <string-name>
            <surname>NY</surname>
          </string-name>
          , USA,
          <year>2022</year>
          , pp.
          <fpage>26337</fpage>
          -
          <lpage>26349</lpage>
          . doi:
          <volume>10</volume>
          .5555/3600270.3602180.
        </mixed-citation>
      </ref>
      <ref id="ref13">
        <mixed-citation>
          [13]
          <string-name>
            <given-names>I.</given-names>
            <surname>Sutskever</surname>
          </string-name>
          ,
          <string-name>
            <given-names>O.</given-names>
            <surname>Vinyals</surname>
          </string-name>
          ,
          <string-name>
            <given-names>Q. V.</given-names>
            <surname>Le</surname>
          </string-name>
          ,
          <article-title>Sequence to sequence learning with neural networks</article-title>
          ,
          <source>in: Proceedings of the 28th International Conference on Neural Information Processing Systems - Volume 2, NIPS'14</source>
          , MIT Press, Cambridge, MA, USA,
          <year>2014</year>
          , p.
          <fpage>3104</fpage>
          -
          <lpage>3112</lpage>
          . doi:
          <volume>10</volume>
          .5555/2969033. 2969173.
        </mixed-citation>
      </ref>
      <ref id="ref14">
        <mixed-citation>
          [14]
          <string-name>
            <given-names>A.</given-names>
            <surname>Vaswani</surname>
          </string-name>
          ,
          <string-name>
            <given-names>N.</given-names>
            <surname>Shazeer</surname>
          </string-name>
          ,
          <string-name>
            <given-names>N.</given-names>
            <surname>Parmar</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J.</given-names>
            <surname>Uszkoreit</surname>
          </string-name>
          ,
          <string-name>
            <given-names>L.</given-names>
            <surname>Jones</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A. N.</given-names>
            <surname>Gomez</surname>
          </string-name>
          , Ł. Kaiser,
          <string-name>
            <surname>I. Polosukhin</surname>
          </string-name>
          ,
          <article-title>Attention is all you need</article-title>
          ,
          <source>in: Advances in Neural Information Processing Systems</source>
          ,
          <year>2017</year>
          , pp.
          <fpage>5998</fpage>
          -
          <lpage>6008</lpage>
          . doi:
          <volume>10</volume>
          .5555/3295222.3295349.
        </mixed-citation>
      </ref>
      <ref id="ref15">
        <mixed-citation>
          [15]
          <string-name>
            <given-names>C.</given-names>
            <surname>Metta</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Fantozzi</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Papini</surname>
          </string-name>
          , G. Amato,
          <string-name>
            <given-names>M.</given-names>
            <surname>Bergamaschi</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S. G.</given-names>
            <surname>Galfre</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Marchetti</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Veglio</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Parton</surname>
          </string-name>
          ,
          <string-name>
            <given-names>F.</given-names>
            <surname>Morandin</surname>
          </string-name>
          ,
          <article-title>Increasing biases can be more eficient than increasing weights</article-title>
          , in:
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>