Recent advances in Language Models (LMs) have failed to mask their shortcomings particularly in the domain of reasoning. This limitation impacts several tasks, most notably those involving ontology engineering. As part of a PhD research, we investigate the consequences of incorporating formal methods on the performance of Small Language Models (SLMs) on reasoning tasks. Specifically, we aim to orient our work toward using SLMs to bootstrap ontology construction and set up a series of preliminary experiments to determine the impact of expressing logical problems with diferent grammars on the performance of SLMs on a predefined reasoning task. Our findings show that it is possible to substitute Natural Language (NL) with a more compact logical language while maintaining a strong performance on reasoning tasks and hope to use these results to further refine the role of SLMs in ontology engineering.
CEUR
ceur-ws.org our methodology and formalizes the Common Logic Grammar Construction (CLGC) pipeline. Section 4 presents our extended experimental setup and reports our findings with discussions on our results. Finally, Section 5 provides a summary of our key findings and outlines the next steps in this PhD research as well as potential future research directions.
The focus of the first stage of this PhD is to combine formal knowledge representation with LM learning methods to improve LM performance on FOL reasoning tasks with the perspective of an application to ontology engineering. We report here the relevant research regarding state-of-the-art LM reasoning methods and logical reasoning applications.
Reasoning is a fundamental aspect of human intelligence and is a cognitive process that involves the use of evidence, arguments and logic to arrive at conclusions [13]. The impressive performance of LMs on Natural Language (NL) data has led to observations that these models may exhibit reasoning abilities when they are suficiently large [ 13]. However, reasoning also appears to be inherent in SLMs which are shown to be able to perform close to, and in some cases on par with, Large Language Models (LLMs). Approaches include extensive pre-training or guided techniques such as problem decomposition or the introduction of LLMs in SLM training processes for step correction as in the case of the SMART framework [14, 15, 16].
In an efort to determine reasoning abilities in LMs, researchers have explored novelties in techniques, benchmark datasets and evaluation metrics [13]. Reasoning techniques largely fall under two pillars: LM-based and Reinforcement Learning (RL) techniques [17]. LM-based reasoning techniques revolve around the methodologies spearheading model training, i.e. Supervised Fine-Tuning (SFT) on the one hand and Prompting and In-Context Learning on the other [13, 18]. Another emerging research direction is the exploration of hybrid or neuro-symbolic models which incorporate a form of symbolic knowledge into LMs to guide or enhance reasoning as seen in DomiKnowS and Logic [19, 20]. Diferent training techniques extending fine-tuning and prompting like COCONUT have resulted in LMs being used to tackle various reasoning tasks like mathematical, logical, causal, commonsense reasoning and coding [21]. None of these methods however consider the potential impact of the logical formalism used to prompt the LMs.
Logical reasoning is an interesting domain of application for LMs since it subjects them to diferent types of reasoning, namely deductive, inductive, abductive or analogical [13]. Reasoning problems are classified in the literature into 2 main groups: Logical Question Answering (LQA) and Logical Consistency (LC) [22]. LQA applications encompass tasks that require LMs to generate the correct answer within complex logical problems which require sophisticated reasoning given a collection of premises and constraints [22]. LC applications require LMs to generate verifiable answers to complex questions without violating consistency (e.g. the model should not generate answers that contain contradictions) [22]. As we will show, our first research questions, methods and experimental setup fall under LQA and rely on FOLIO, a human-annotated benchmark dataset for FOL reasoning [23].
LQA models can be grouped into 3 categories depending on the approach. Solver-aided models rely on LMs translating NL problems into FOL before passing them to a solver to compute the final answer. These models have shown an increased performance over traditional model training techniques on logical problem-solving [24, 25]. Prompt-based models rely on techniques like Chain-of-Thought (CoT) to break down complex problems using smaller reasoning steps (i.e. hops) and guide the model to the ifnal solution [ 26]. However, experiments on these models did not prove actual reasoning beyond two hops and could not determine that this type of reasoning scales with larger models [26, 27]. The last group of models adhere to the pre-training and fine-tuning approaches on logical questions to enhance their reasoning process. These models often include an additional neural layer that captures logical constraints and benefits from forward and backward passes to iteratively refine the model output in the constrained setting [28]. Our methodology targets the data representation of the input and evaluates models with diferent representations in both the prompting and fine-tuning settings. Research on input representation has already shown that the language in which a LM receives a problem has an impact on its success rate in solving the problem [29], but does not consider the variety of languages we evaluate in our work.
This section introduces our methodology in three steps: (
The Syllogistic Evaluation Framework (SEF) is a methodology for identifying and classifying diferent
types of syllogisms inspired by the work in [30]. The objective of SEF is to have an additional evaluation
criterion to trace and analyze performance on diferent types of logical reasoning: SEF allows
discrimination between diferent reasoning problems to explicit a model’s reasoning capabilities. We applied SEF
on the FOLIO dataset which is the focal data source for our experiments. FOLIO is a collection of stories,
with each story consisting of a set of premises = { 1, 2, ..., } and conclusions = {ℎ 1, ℎ2, ..., ℎ }
in NL and their corresponding FOL annotations of premises = { 1, 2, ..., } and conclusions
= {ℎ 1, ℎ 2, ..., ℎ }. Given each premises-conclusions pair, the goal is to determine the truth values
of the conclusions, i.e. “True”, “False” or “Uncertain” based on FOL reasoning. We determined 4
syllogistic categories to classify each premises-conclusions pair: Disjunctive, Hypothetical, Categorical and
Complex. The categories were identified based on the following logical criteria presented from highest
to lowest order of precedence: (
Examples of SEF classification are presented in Table 1. The classification statistics for FOLIO are presented in Table 2. The distribution shows an imbalance heavily favoring Disjunctive and Hypothetical kinds of reasoning, while Complex and Categorical examples are too sparsely represented to draw any conclusions on.
Diferent knowledge representation formalisms have been tested in the CLGC pipeline to measure the impact of the language on the SLM in solving a logical reasoning task. We present concrete transformations from FOL to CLGC languages in Table 3.
We implemented grammars from the Common Logic (CL) family suite as defined in [ 31, 32, 33]. In particular, we focused on the Common Logic Interchange Format (CLIF) and the Conceptual Graph Interchange Format (CGIF) languages for their ease of interoperability with FOL while retaining the advantages of formal logic representations. From a practical aspect, these languages can also be implemented due to the availability of their grammars1. We limited our implementation to the core Backus–Naur form (BNF) grammars for CLIF and CGIF and did not include the extended syntax for these languages for simplicity and compactness.
We also incorporated Tensor Function Logic (TFL), a formal reasoning language introduced in [34, 35, 36], and its extension, Tensor Function Logic Plus (TFL+). We limited the scope of TFL to the terms and their arithmetic signs and incorporated quantifier subscript representation as well as parentheses in TFL+ to work with grammars of varying complexity. 3.2.3. MINIFOL Additionally, we implemented a custom language called Miniature First-Order Logic (MINIFOL) which is directly derived from FOL by replacing ∀ and ∃ with “all” and “some” respectively and replacing operators with Boolean equivalents (e.g. “∧” with “&”). The advantage of MINIFOL is twofold: replace some FOL vocabulary (i.e. symbols) with vocabulary that is more familiar for SLMs (i.e. words) while benefiting from a rule set that enables an easy transformation from FOL to MINIFOL. ∀x ((Employee(x) ∧ all:x ((employee(x) & forall x ((employee(x) [@every *x [([(em- -+E1++S1- Schedule(x, meeting, schedule(x, meeting, and schedule(x, meet- ployee[(?x)] sched- +A1 ((+E0++S0)customers)) → Ap- customers)) :- ap- ing, customers)) im- ule[(?x meeting +A0) pearIn(x, company)) pearin(x, company)) plies appearin(x, com- customers)])] appany)) pearin[(?x company)])] ∀x ((Employee(x) ∧ all:x ((employee(x) & forall x ((employee(x) @every *x [([(em- -+E1++H1- HasLunch(x, com- haslunch(x, company)) and haslunch(x, com- ployee[(?x)] +S1 ((+E0++H0)pany)) → Schedule(x, :- schedule(x, meeting, pany)) implies sched- haslunch[(?x com- +S0) meeting, customers)) customers)) ule(x, meeting, cus- pany)])] schedule[(?x
tomers)) meeting customers)])] ∀x (Employee(x) → all:x (employee(x) :- forall x (employee(x) @every *x [(em- -+E1-+H1- -(+E0-(+H0(HasLunch(x, com- (haslunch(x, company) implies (haslunch(x, ployee[(?x)] +H1 +H0)) pany) ⊕ HasLunch(x, ĥaslunch(x, home))) company) xor [(haslunch[(?x comhome))) haslunch(x, home))) pany)] haslunch[(?x home)])])] ∀x ((Employee(x) ∧ all:x ((employee(x) & forall x ((employee(x) @every *x [([(em- -+E1++H1- HasLunch(x, home)) haslunch(x, home)) :- and haslunch(x, ployee[(?x)] +W1 ((+E0++H0)→ Work(x, home)) work(x, home)) home)) implies work(x, haslunch[(?x home)])] +W0) home)) work[(?x home)])] ∀x ((Employee(x) ∧ all:x ((employee(x) & forall x ((employee(x) @every *x [([(em- -+E1+-+I1- -((+E0+((¬In(x, homecountry))) ( in(x, homecountry))) and (not in(x, home- ployee[(?x)] [( in[(?x +W1 +I0))-+W0) → Work(x, home)) :- work(x, home)) country))) implies homecountry)])])]
work(x, home)) work[(?x home)])] ∀x (Manager(x) → all:x (manager(x) :- forall x (manager(x) @every *x [(man- - ¬Work(x, home)) work(x, home)) implies not work(x, ager[(?x)] work[(?x +M1–+W1 (+M0–+W0) home)) home)])] ¬(Manager(james) ⊕ (manager(james) ∧ not (manager(james) [(manager[(james)] -+M1-+A1 -(+M2(+j2)AppearIn(james, com- appearin(james, com- xor appearin(james, appearin[(james +A2) pany)) pany)) company)) company)])]] HasLunch(james, haslunch(james, com- haslunch(james, com- [haslunch[(james +H1 +H2 company) pany) pany) company)]]
Because this work investigates the impact of the formal representation of knowledge in the scope
of reasoning tasks, we devised a pipeline to transform one logical form to another with the goal of
generating diferent data representations from FOLIO to test as input to our SLMs. The CLGC pipeline
is presented in Figure 1 and consists of the following steps: (
Additionally, we introduced 2 new SFT configurations: Grammar Context-Passing (ICGP) and Tokenizer Re-Training. In ICGP, the BNF grammar of the input data language is passed as additional context to the classical SFT input (i.e. the logical premises, conclusions and label). In the Tokenizer Re-Training setup, the SLM tokenizer is retrained on the vocabulary of the grammar by passing all train, test and validation data in the specified language to the model. After re-training, the original size of the tokenizer is kept in some runs to incorporate the vocabulary of the grammar and suficiently resized empirically in others to eliminate as many tokens outside those making up the target vocabulary.
In this section, we describe our experimental setup and set out to answer the following research subquestions (SRQs): SRQ1: Which training method yields the best results for solving FOL problems with LMs? SRQ2: How do formal representations scale with models? SRQ3: Does having a more compact vocabulary Algorithm 1 Input-Output Grammar Transformation Algorithm 1: 2: 4: 5: 6: 7: 8: 9: 10: 11: 12: 13: 14: 15: 16: 17: 18: 19: 20: Require: ∶=
end if
then end for ⇐ ( if = else end if ⇐ ⇐ ( ) )
▷ loop through logical statements ▷ generate grammar parse tree from statement ▷ initialize output grammar parse tree for statement ▷ loop through each element of the input grammar parse tree
▷ map input element to corresponding output grammar ▷ there is a corresponding element in the grammar mapping
▷ add mapped element to output grammar parse tree ▷ there is no corresponding element in the grammar mapping , )
▷ check if final parse tree from statement is parsable , )
▷ update output grammar to parse mapped parse tree 21: end for boost model performance for formal languages?
We extend the experiments performed in [23] on the NL reasoning with FOL task by expanding the data representation of the premises and conclusions as well as the choice of models. For the data representation, we retain the NL and FOL languages in addition to the ones we derived from our pipeline in section 3 (i.e. CLIF, CGIF, TFL, TFL+ and MINIFOL). For our model selection, we restricted our choices to SLMs for their frugality and promising performances on the original task in [23] and because they avoid having a dependency to external and possibly expensive LLM services. We configure our models (M ), grammars (G) and learning methods (L) with the values: M = {Flan-T5-small, FlanT5-base, Flan-T5-large, GPT-2, Phi-3.5-mini-instruct, Gemma-2-2b-it}, G = {NL, FOL, CLIF, CGIF, TFL, TFL+, MINIFOL} and L = {SFT, Zero-Shot (ZS) Prompting, Few-Shot (FS) Prompting}.
Our experimental runs consisted of variations of the triple (M, G, L) to determine the truth value of a conclusion (i.e. “True”, “False” or “Uncertain”) based on FOL reasoning over a set of premises. We split the provided train portion of the original version of FOLIO3 into 80% train and 20% test and froze those splits along with the original validation set for all our experimental runs. Our final splits consisted of 800 training, 203 validation and 201 test rows. 3https://huggingface.co/datasets/yale-nlp/FOLIO
You are given a set of premises and a conclusion. The premises start after a <PREMISES> tag and conclusion starts after a <CONCLUSION> tag. Classify each conclusion as ”True”, ”False” or ”Uncertain” depending on its satisfiability of the premises. <PREMISES>...</PREMISES> <CONCLUSION>...</CONCLUSION> You are an expert in logic, and you need to output the conclusion for the following logic problem. You are given grammar rules for the language of the problem in <GRAMMAR> tags, and you should conform to these rules to understand the premises in the <PREMISES> tags and conclusion in the <CONCLUSION> tags. Classify each conclusion as ”True”, ”False” or ”Uncertain” depending on its satisfiability of the premises. Present your answer only in <output> tags. <GRAMMAR>...</GRAMMAR> <PREMISES>...</PREMISES> <CONCLUSION>...</CONCLUSION>
All experiments were run on Google Colab instances using L4 GPUs and, depending on availability, A100 High-RAM GPUs. In order to run some configurations in the absence of A100 GPUs (e.g. SFT with Flan-T5-large), we used Parameter-Eficient Fine-Tuning with Low-rank adaptation (PEFT-LoRA). PEFTLoRA freezes the pre-trained model weights to reduce the number of trainable parameters and has shown great eficacy for LM training [ 37]. In our setup, PEFT-LoRA is configured as follows: {r=16, lora_alpha=32, target_modules=[“q”, “v”], lora_dropout=0.05, bias=none”, task_type=“SEQ_2_SEQ_LM”}. In the SFT mode, all models were trained on 5 epochs without PEFT-LoRA and 10 epochs with PEFT-LoRA.
For the ZS and FS modes, we used 2 prompt templates. The first is a basic template explaining the task, the nature of the input and the expected output. The second is the basic template augmented with the BNF grammar rules of the language when applicable (i.e. for languages other than NL). We performed 8-shot prompting for all FS experiments. Table 4 shows an example of a prepared basic prompt and augmented prompt template for the ZS and FS experiments. No ablation study was performed on the CLGC pipeline as all components are needed to generate a correct grammar for our experiments.
We present the most prominent results from our experimental runs in Tables 5-13. Table 5 shows that formalizing the premises and conclusions in CLIF ties the performance of NL on Accuracy and ranks second-best on F1 score. The result shows that SLMs can perform well on first-order reasoning tasks with a more compact formalism than NL. Languages with more complex syntaxes like CGIF struggle to yield good performances, suggesting that it is more dificult for SLMs to reason over complex representations. It is also worth noting that the best performances occur with the smallest model, Flan-T5-small, which outperforms larger, fine-tuned models (e.g. Flan-T5-large), suggesting that added architectural complexity may hinder the learning process in this setting. The result is empirically supported by the diferent data formalisms in our runs.
Table 6 shows scaled results on bigger models. With the A100 GPUs, models like Flan-T5-base and Flan-T5-large can be trained in supervised fashion without PEFT-LoRA tuning. The results confirm our earlier findings that a compact language like CLIF can efectively model logical data and be competitive on a reasoning task by ranking second-best to NL, i.e. the current state of the art in [23].
Table 7 shows the impact of languages in the ZS setting. The results show that the performance of compact languages like CLIF and TFL+ are extendable to other models, as shown with Gemma-2-2b-it which ranks first in Precision with TFL+ and yields competitive results with CLIF. Not all grammars are created equally however, as shown by MINIFOL and TFL which for very small models like Flan-T5-small cripple the learning completely resulting in a flat performance of 0.
The results of Table 8 cement CLIF as the best language choice in the ZS setting. With and without BNF grammar prompting, CLIF outperforms all other languages. In addition, grammar prompting seems to improve SLM performance in the ZS case over first-order reasoning as empirically shown for all our languages. These results support similar findings in [ 38].
In Table 9 we present our results for the 8-shot setting. The performances are comparable to the ZS case in Table 7 and cement CLIF as a more compact contender to the verbose NL representation of logical input. A familiar pattern seen in the ZS case re-emerges as Flan-T5-small is unable to learn with TFL. This suggests a limitation to model learning with respect to size as even for very small grammars like TFL, very small models struggle to learn the formalism well enough to solve a reasoning task.
Table 10 showcases the efects of augmenting the FS prompt with the language BNF grammars. The ifrst takeaway is that augmenting the basic prompt template with the BNF grammar does not seem to impact the performance of a model in the FS case, unlike the ZS setting in Table 8. As for performance, the language choice does not seem to afect the model in the ZS setting, suggesting a model in that case might be less sensitive to the input representation. Finally, comparing these results to those in Table 8 shows that the same model performs better in the ZS case than the FS case. One explanation could be that the number of FS examples is not suficient for the chosen model on this particular task for learning. Another plausible hypothesis can be that the examples might be skewing the learning of the model by adding noise in the case of solving a reasoning task.
Table 11 shows the results of passing BNF grammars as context in addition to the standard input in the SFT setting. Empirical results confirm that including the BNF grammar in-context hinders learning and degrades model performance on the first-order reasoning task. Model size and complexity does not seem to impact performance as scaling models still yields better results without the inclusion of the grammar. Comparing results to those in Tables 5 and 6 also shows weaker performance with in-context information passing than traditional supervised training on inputs.
Table 12 shows the impact of re-training a model tokenizer to adapt to the specific vocabulary of a grammar and re-using the tokenizer in the SFT setting. The results show promise in the case of smaller models for more compact data representations. This is illustrated by the performance of Flan-T5-small on TFL+ with tokenizer re-training and vocabulary resizing which outperforms Flan-T5-small on TFL+ without tokenizer re-training and resizing. This configuration even outperforms Flan-T5-small and Flan-T5-large with PEFT-LoRA on NL from Table 5, both of which represent the baseline of SFT on lower resources (i.e. L4 GPUs) for our task. However, as shown when moving to bigger models like Flan-T5-base and Flan-T5-large on TFL+, this method is not scalable and breaks down when tokenizer re-training is done, resulting in worse performances than using the default tokenizer for the models. One possible explanation is that re-training the tokenizer for small models and suficiently compact data representations may make the models learn the formalism and solve the task more eficiently but also causes overfitting. This risk suggests that this method may not be adapted to take full advantage of the compactness of data representations as it sacrifices generalization and weakens overall performance on the reasoning task.
Finally, Table 13 uses SEF to showcase the predictive performances of our best configuration (i.e. Flan-T5-large in SFT setting) on the 3 best grammars (i.e. NL, CLIF and TFL+) on the FOLIO validation set. In all 3 cases, the model performs well on the Disjunctive and Hypothetical syllogisms which can be seen as an unsurprising result given the over-representation of these two types in FOLIO. Irrespective of the data representation, the model seems to also perform well on the Complex syllogisms, with TFL+ even yielding a slightly better performance on that type than NL and CLIF. Complex syllogisms typically contain 3 or more premises and polysyllogisms which might render them too ambiguous for NL and CLIF and may benefit more compact representations like TFL+. This may explain the slight increase in performance noted in the results. As for the Categorical syllogisms which are heavily under-represented, all grammars share equal performance with 2 hits and 2 misses on the prediction labels, but the data imbalance makes it dificult to interpret the results. Further investigation is needed on the quality of these predictions, especially in a comparative framework, to assess the capabilities of the model on diferent data representations. We provide an example of extended error analysis in Table 14. The analysis is conducted for the Categorical syllogistic type and shows that the variation in grammar generally results in the same model reasoning on these problems with the exception of TFL+ on a single occurence, whereby the model reasons ”False” as opposed to ”Uncertain” on NL and CLIF. A form of quantitative analysis may be needed to explore these results further and may be included in future work along with the analysis of the other syllogistic types which has not been included due to size restriction. The results enable us to answer our SRQs. For SRQ1, the best method shown empirically remains SFT as it provides the highest and most stable results among diferent representations. For SRQ2, our results show that controlled formal languages generally scale well with models while remaining consistent with performances. For SRQ3, our experiments show that tailoring the vocabulary size of the model tokenizer to that of the language results in erratic performances that do not scale well. Future work will include experiments to evaluate the efect of tokenizer post-retraining on the grammar token distribution for compact languages (e.g. TFL+) to understand the cases where re-training would help or fail.
In this paper, we presented the early stage of a PhD research tackling the formal representation of data in SLMs to possibly bootstrap the construction of ontologies with external knowledge. We focused here on the formalization of logical data with the goal of finding a compact representation that keeps SLMs performing competitively on reasoning tasks. Our experiments enable us to answer our RQ: logical data for first-order reasoning in NL can be challenged by more compact and formal representations. From our empirical results, CLIF emerged as a strong candidate due to the compactness of its vocabulary, its easy transformation to and from FOL and its competitive results especially in a SFT setting. The results were achieved on small, frugal language models that are all under 3 billion parameters, making them accessible for reasoning tasks over logical data. While our experiments do not point to an absolute best representation that overtakes NL in the formalization of logical data, they show that expressing inputs in compact formal languages comes very close for SLMs on reasoning tasks. This takeaway allows us to focus on the next goal of this PhD research which is to leverage SLMs and formal representations of ontological data to bootstrap ontology construction.
This PhD research will explore mixed-input representations (e.g. NL + CLIF) to assess whether combining expressiveness with the structure of a formal grammar improves reasoning. We will also explore injecting knowledge from a high-level ontology (e.g. DOLCE [39]) in a formal language to eficiently reuse relevant knowledge learned by the model in an ontology extension pipeline.
We identify several future research directions including the extension of the SEF-CLGC evaluation to datasets such as ProofWriter4, RuleTaker5, the Logical Entailment Dataset6 and SynLogic7. Another 4https://www.kaggle.com/datasets/mathurinache/proofwriter 5https://github.com/allenai/ruletaker 6https://github.com/google-deepmind/logical-entailment-dataset 7https://github.com/MiniMax-AI/SynLogic direction would be using LMs to extend the vocabulary of an ontology to handle new scenarios given Competency Questions (CQs) and the original ontology in practical use cases like HMAS8 using categorical syllogisms to explain the hierarchical connections between generated classes and properties.
This work is supported by 3IA Côte d’Azur (ANR-19-P3IA-0002), UCAJEDI (ANR-15-IDEX-01), Université Côte d’Azur’s Center for High-Performance Computing and Data ScienceTech Institute.
The author(s) have not employed any Generative AI tools. Uncertain
True
True True True
Uncertain
Uncertain True
True False
False
False
True False True False [16] Y. Kim, E. Yi, M. Kim, S.-Y. Yun, T. Kim, Guiding reasoning in small language models with llm assistance, Preprint arXiv:2504.09923 (2025). [17] M. Besta, J. Barth, E. Schreiber, A. Kubicek, A. Catarino, R. Gerstenberger, P. Nyczyk, P. If, Y. Li,
S. Houliston, et al., Reasoning language models: A blueprint, Preprint arXiv:2501.11223 (2025). [18] K. Kumar, T. Ashraf, O. Thawakar, R. M. Anwer, H. Cholakkal, M. Shah, M.-H. Yang, P. H. Torr, F. S.
Khan, S. Khan, Llm post-training: A deep dive into reasoning large language models, Preprint arXiv:2502.21321 (2025). [19] H. R. Faghihi, Q. Guo, A. Uszok, A. Nafar, E. Raisi, P. Kordjamshidi, Domiknows: A library for integration of symbolic domain knowledge in deep learning, Preprint arXiv:2108.12370 (2021). [20] P. Kesseli, P. O’Hearn, R. S. Cabral, Logic. py: Bridging the gap between llms and constraint solvers, Preprint arXiv:2502.15776 (2025). [21] S. Hao, S. Sukhbaatar, D. Su, X. Li, Z. Hu, J. Weston, Y. Tian, Training large language models to reason in a continuous latent space, 2024, URL https://arxiv. org/abs/2412.06769 (2022). [22] F. Cheng, H. Li, F. Liu, R. van Rooij, K. Zhang, Z. Lin, Empowering llms with logical reasoning: A comprehensive survey, Preprint arXiv:2502.15652 (2025). [23] S. Han, H. Schoelkopf, Y. Zhao, Z. Qi, M. Riddell, W. Zhou, J. Coady, D. Peng, Y. Qiao, L. Benson, et al., Folio: Natural language reasoning with first-order logic, in: Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, 2024, pp. 22017–22031. [24] Y.-F. Zhang, H. Zhang, L. E. Li, E. Xing, Evaluating step-by-step reasoning through symbolic verification, Preprint arXiv:2212.08686 (2022). [25] J. Xu, H. Fei, L. Pan, Q. Liu, M.-L. Lee, W. Hsu, Faithful logical reasoning via symbolic chain-ofthought, Preprint arXiv:2405.18357 (2024). [26] S. Yang, E. Gribovskaya, N. Kassner, M. Geva, S. Riedel, Do large language models latently perform multi-hop reasoning?, Preprint arXiv:2402.16837 (2024). [27] S. Yang, N. Kassner, E. Gribovskaya, S. Riedel, M. Geva, Do large language models perform latent multi-hop reasoning without exploiting shortcuts?, Preprint arXiv:2411.16679 (2024). [28] B. Ghosh, S. Hasan, N. A. Arafat, A. Khan, Logical consistency of large language models in fact-checking, Preprint arXiv:2412.16100 (2024). [29] Y. Peng, Z. Long, F. Dong, C. Li, S. Wu, K. Chen, Playing language game with llms leads to jailbreaking, Preprint arXiv:2411.12762 (2024). [30] Y. Wu, M. Han, Y. Zhu, L. Li, X. Zhang, R. Lai, X. Li, Y. Ren, Z. Dou, Z. Cao, Hence, socrates is mortal: A benchmark for natural language syllogistic reasoning, in: Findings of the Association for Computational Linguistics: ACL 2023, 2023, pp. 2347–2367. [31] J. F. Sowa, Conceptual graphs, Foundations of artificial intelligence 3 (2008) 213–237. [32] J. F. Sowa, Conceptual graphs as a universal knowledge representation, Computers & Mathematics with Applications 23 (1992) 75–93. [33] J. Sowa, Introduction to common logic, 2011. [34] F. Sommers, G. Englebretsen, An invitation to formal reasoning: the logic of terms, Routledge, 2017. [35] J. M. Castro-Manzano, L. I. Lozano-Cobos, P. O. Reyes-Cardenas, Programming with term logic,
BRAIN. Broad Research in Artificial Intelligence and Neuroscience 9 (2018) 22–36. [36] J. M. C. Manzano, An intermediate term functor logic, Argumentos - Revista de Filosofia 11 (2025) 17–31. [37] E. J. Hu, Y. Shen, P. Wallis, Z. Allen-Zhu, Y. Li, S. Wang, L. Wang, W. Chen, et al., Lora: Low-rank adaptation of large language models., ICLR 1 (2022) 3. [38] B. Wang, Z. Wang, X. Wang, Y. Cao, R. A Saurous, Y. Kim, Grammar prompting for domain-specific language generation with large language models, Advances in Neural Information Processing Systems 36 (2023) 65030–65055. [39] S. Borgo, R. Ferrario, A. Gangemi, N. Guarino, C. Masolo, D. Porello, E. M. Sanfilippo, L. Vieu, Dolce: A descriptive ontology for linguistic and cognitive engineering, Applied ontology 17 (2022) 45–69.