In this paper, we sketch a strategy to take advantage of LLM's capabilities with natural language processing while avoiding the risk of incorrect outputs. We aim to extract knowledge from LLMs into logic programs and queries and, therefore, add LLMs to the materials and tools used to teach logic programming. Large Language Models, LLM, are a compiled, sub-symbolic form of knowledge representation that can achieve human level performance in natural language processing[1]. The transformer architecture[2], that creates and processes LLMs, is an stochastic machine able to generate new, original, human-like outputs from natural language inputs (prompts), after having been trained with massive amounts of data[3]. This relatively new and increasingly popular brand of Artificial Intelligence Applications has been termed Generative AI[4]. Unfortunately, the stochastic machine is bound to produce some outputs that can not be logically justified from the inputs or the training data. These "original" outputs, sometimes called hallucinations, cannot be avoided[5] and can be regarded as serious errors in many domains of use.
This type of hybrid systems, combining Generative AI with a form of Logical AI, some times also
called neuro-symbolic systems, are being researched and tested intensively [
intervene the Transformer architecture with a representation closer connected with logical
formulae and a process that matches a reasoning system, as done by Thuy & Yamamoto [
In this paper, as a teaching exercise, we want to step back before moving onto the integration of an LLM with a reasoner, to revisit the basic structures and practical elements that underlie language models as they are understood in Generative AI. After, we will relate some experiments to decompile knowledge from an LLM into logic programs that provide explainable answers to users’ questions.
A compiler is a basic concept in Computer Science: “A compiler is a program that can read a
program in one language, the source language, and translate it into an equivalent program in
another language, the target language” [
program. When the target is a very basic code, like a binary representation, the process faces an enormous combinatorial complexity and it is very difficult to obtain the source. In some cases, however, with the use of inputs and corresponding outputs, and some other general information, like blue-prints or frameworks associated with the whole system, de-compilation is practically possible, as depicted in figure 2.
A neural network can be regarded as a compiled representation of knowledge, as will be illustrated below. It is normally produced by a training process that short-circuits inputs and outputs and computes the weights of the network that generalize the function that associates those inputs and outputs and, hopefully, variants of them. Once a neural network is trained, its weights and biases represent the learned knowledge in an executable form. In this operational sense, the net is like a compiled program: an executable form of the original source code. We do not see the ‘source code’ (the training data and learning process) directly, but we have the ‘executable’ (the trained model) that embodies the learned relationships. Trusting these relationships implies trusting there is a meaningful connection between the ‘source code’ and the ‘executable’.
language model trained on logic programs could be used to reproduce them by appropriate prompting.
We envisage the possibility depicted at the bottom of Figure 3. An LLM, which is a specialized form of neural network coupled with a processing device (e. g. a transformer). Having being trained to represent logic programs, the transformer could be prompted to reproduce correct sources by providing it with some samples. It would be de-compiling logic programs from the subsymbolic representation. In each case, we aim to extract a logic program whose inherent structure and rules represent meaningful, correct, and independently verifiable knowledge about the domain, irrespective of the neural network's specific input-output mappings.
networks and logic programs. It is no longer as simple as it will be presented below, because the
technology of the connectionist systems, as they are also called, has evolved. Transformers[
A communicative act is effective with a strength W if the act conveys information with a strength W1 and the act is easy to understand with a strength W2 and W = f(W1, W2).
A communicative act is more effective, the more it conveys information and the easier it is to understand.
to read or skip (From Kowalski [
Consider this set of examples that correspond, respectively, to Author, Thread, Length, Location, Action: known, new, long, home, skip unknown, new, short, work, read unknown, follow-up, long, work, skip known, new, short, home, read Write rules to decide when to read and when to skip. Write the rules as logical clauses like skip if known and new. Explain the rules.
and obtained this answer: <Start Content enhancement by Claude Sonnet 4>1
1Labeled as suggested here: https://ceur-ws.org/GenAI/Taxonomy.html Rules:
Explanation:
• All long emails are skipped (examples 1 and 3) • All short emails are read (examples 2 and 4)
instance: • Both known and unknown authors can be read or skipped • Both new threads and follow-ups can be read or skipped • Both home and work locations can be read or skipped
management strategy where you prioritize shorter emails (quick to process) over longer ones (time-consuming), regardless of who sent them, whether they're new conversations, or where you're reading them.
<End Content enhancement by Claude Sonnet 4>
This exercise illustrates how an LLM can take an input with a set of data and some instructions about how to produce an output and, not only carrying on the instructions regarding the data, but producing useful rules to model it. It even goes on to also hypothesize an explanation. At this point, one could test the rules to check on their validity independently from the LLM. We end up with a hybrid system that is very effective at parsing natural language and can be fully accounted for while reasoning.
To help us recover perspective, we have developed a very simple example of a language model
on one of software platform used for the large ones [
Input: 'a.' -> Predicted: '<eos>' (Expected: 'b') Input: 'a.b' -> Predicted: '.' (Expected: ':') Input: 'a.b:' -> Predicted: '-' (Expected: '-') Input: 'a.b:-' -> Predicted: 'a' (Expected: 'a') Input: 'a.b:-a' -> Predicted: '<eos>' (Expected: '.') Input: 'a.b:-a.' -> Predicted: '<eos>' (Expected: 'b') Input: 'a.b:-a.b' -> Predicted: '.' (Expected: '.') Input: 'b.c:-b.' -> Predicted: '<eos>' (Expected: 'c') Input: 'a.b.d:-a,b.' -> Predicted: 'd' (Expected: 'd')
This tiny model correctly predicts a correct continuation of 5 out of 9 cases (as marked in yellow and green highlighting). One of them (last line in red) could be (mis)interpreted as the correct application of forward reasoning. That output lists what comes from the model (the prediction) and what one can expect given the inputs. For example, in the last line, we expected d because of the input pattern: "a.b.d:-a,b.d.". And the model correctly predicts that d follows a.b.d:a,b. That prediction also coincides with the results of inferring d from a and b through the rule d:a,b., a coincidence that could have people believing that the model does modus ponens.
list the most common ways to ask the same question and then map the corresponding alternatives as new entries to a Prolog dictionary2 (.ibid). That dictionary is used by the control natural language Logical English, LE. We ended with a series of descriptions of relations in natural language, called templates in LE, that can be used to query a document processed by the regular LE engine in Prolog.
This is a set of rules, an scenario for the rules and a list of queries: and then included the content of a Logical English document describing an Australian Tax Law3, but without including templates. Then, we asked:.
Can you produce some interesting scenarios to test those rules and it produced a list of 6 scenarios to test the given rules, described in plain English but not in
Can you restate those scenarios using these relations:
and included all the templates that accompany that document. The answer is a partial but computable list of examples that could be added to the document in question, such as this one: <Start Content enhancement by DeepSeek V3> 2https://github.com/LogicalContracts/LogicalEnglish/blob/main/kb/4_affiliates_3.pl https://github.com/LogicalContracts/LogicalEnglish/blob/8ee7ff2740cb1e7231d115572a5a3dad7c673535/le_input.pl#L2478 (where the particular entries start) 3https://github.com/LogicalContracts/LogicalEnglish/blob/main/moreExamples/1_cgt_assets_and_exemptions_3.le Scenario 1: The Crypto Collector
could be added to a runnable document in Logical English, with the following content4 scenario Crypto is:
Alice is a taxpayer. asset bitcoin is a cryptocurrency. asset bitcoin belongs to Alice. asset bitcoin was acquired at 2020-01-01. asset bitcoin is used for the purchase of items for personal consumption. asset nft_rare belongs to Alice. asset nft_rare costed 15000 to acquire. asset nft_gift is a gift. asset nft_gift belongs to Alice.
asset nft_gift was gifted through a will to a deductible gift recipient beneficiary.
cover it all. Some adjustments and extra information are still required to make it work. But there
seems no need to use a particular syntax, like Prolog’s, and the generated outputs are very useful to
extend the codification of testing cases. It can be argued that the difficult part is to generate the
rules. Some preliminary results show that it is also feasible with few shots prompting [
We have made an argument for the use of LLM to generate logic programs. We have shown a few experiments in which proper prompting of an LLM produces fragments of logic programs and queries with immediate utility. The read/skip example produces a small logic program, a previous paper shows how we could generate the queries (in Prolog) from English and the last example is about producing facts and queries for scenarios that test the rules in a given document. This has led us to believe that we could exploit the extraordinary capacity of Transformers for natural language parsing, without having to rely on them for reasoning. By de-compiling logic programs from LLMs, we could scale the production of systems integrating generative AI + Logical AI, which could provide a flexible interaction with humans in natural language while offering well-verified 4https://github.com/LogicalContracts/LogicalEnglish/blob/main/moreExamples/cgt_assets.le capabilities for reasoning. We aim to extract knowledge from LLMs into logic programs and queries and, therefore, add LLMs to the materials and tools used to teach logic programming.
We envisage a road map to develop these ideas, bridging LLMs and symbolic logic together, and emphasizing the goal of extracting independently meaningful and correct logic. First, to establish a robust communication interface between LLMs and Prolog. Second, to define and implement methods to evaluate the semantic correctness of extracted logic, independent of the LLM's original output, which will probably bring us back to the discussion about the semantics of LP and will include developing an iterative agentic toolchain where the LLM proposes and refines Prolog rules based on feedback from the Prolog interpreter, and create benchmarks for evaluating this deep logic extraction. And, third, to explore the scalability of the approach and identify real-world applications for independently correct logic extraction. Some related research questions that we could discuss at the workshop are: What constitutes "independently meaningful and correct logic"? (A precise definition is key); What to do if the LLM "hallucinates" plausible-looking but incorrect logic? (The role of the Prolog interpreter as a strict validator could be crucial); What are the theoretical limits of extracting symbolic logic from sub-symbolic models?; What specific architectures for LLMs would best support logic extraction or LP decompilation? (e.g., models explicitly trained on symbolic reasoning tasks, or those with more transparent internal representations); and, How can we scale this beyond toy examples to real-world complexity? (Modularization, hierarchical reasoning, combining with other AI paradigms).
produce the example of guided-rule generation, in section 3.2. We also used DeepSeek V3 for
these tools/services, the author reviewed and edited the content as needed and takes full responsibility for the publication’s content. 5https://github.com/google/BIG-bench/graphs/contributors https://arxiv.org/abs/2206.04615
import torch import torch.nn as nn import torch.optim as optim from collections import defaultdict # Special tokens PAD_ID = 0 class PatternTransformer(nn.Module): def __init__(self, vocab_size, embed_size=16, nhead=1): super().__init__() self.embed = nn.Embedding(vocab_size, embed_size) self.attention = nn.MultiheadAttention(embed_size, nhead, batch_first=True) self.out = nn.Linear(embed_size, vocab_size) # Pre-process patterns def preprocess(text):
return [vocab[ch] for ch in text] + [EOS_ID] pattern_ids = [preprocess(p) for p in patterns] #print(pattern_ids) # Create training samples def create_samples(): samples = [] for seq in pattern_ids: for i in range(2, len(seq)): # Need at least 2 tokens for prediction input_seq = [BOS_ID] + seq[:i] target = seq[i] samples.append((torch.tensor(input_seq), torch.tensor(target))) return samples # Initialize model model = PatternTransformer(len(vocab)) opt = optim.Adam(model.parameters(), lr=0.01) criterion = nn.CrossEntropyLoss() # Training samples = create_samples() print(samples) for epoch in range(500): total_loss = 0 for input_seq, target in samples: opt.zero_grad() logits = model(input_seq.unsqueeze(0)) loss = criterion(logits[0,-1,:], target) loss.backward() opt.step() total_loss += loss.item() if epoch % 50 == 0:
print(f"Epoch {epoch}, Loss: {total_loss/len(samples):.4f}") # Testing test_cases = [ ("a.", "b"), ("a.b", ":"), ("a.b:", "-"), ("a.b:-", "a"), ("a.b:-a", "."), ("a.b:-a.", "b"), ("a.b:-a.b", "."), ("b.c:-b.", "c"), ("a.b.d:-a,b.", "d")