<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta />
    <article-meta>
      <article-id pub-id-type="doi">10.1186/s13054-023-04425-6</article-id>
      <title-group>
        <article-title>Using Large Language Models and Law-Based Rules for the Analysis of VAT Chain-Transaction Cases in Austrian Tax Law</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Marina Luketina</string-name>
          <xref ref-type="aff" rid="aff1">1</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Lukas Knogler</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Christoph G. Schuetz</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          <institution>Johannes Kepler University Linz</institution>
          ,
          <addr-line>Altenberger Str. 69, 4040 Linz</addr-line>
          ,
          <country country="AT">Austria</country>
        </aff>
        <aff id="aff1">
          <label>1</label>
          <institution>University of Applied Sciences Upper Austria</institution>
          ,
          <addr-line>Wehrgrabengasse 1, 4400 Steyr</addr-line>
          ,
          <country country="AT">Austria</country>
        </aff>
      </contrib-group>
      <pub-date>
        <year>2024</year>
      </pub-date>
      <volume>14644</volume>
      <issue>1</issue>
      <fpage>115</fpage>
      <lpage>131</lpage>
      <abstract>
        <p>In tax advisory practice, case descriptions are typically not structured in a machine-readable format, with clients describing their situation in natural language. Large language models excel at natural-language understanding. However, for legal reasoning, including tax law, the propensity of LLMs to hallucinate presents a considerable challenge. Rule-based systems, on the other hand, ofer verifiably correct reasoning given the correct input. Therefore, in this paper, we propose a hybrid approach to support tax advisors with analyzing tax cases, combining a rule-based system with large language models. We focus on the analysis of chain-transaction cases in valueadded tax (VAT) law, where the law states a clear set of rules for regular chain-transaction cases. We employ a large language model (LLM) for the construction of structured representations of natural-language VAT case descriptions and law-based rules for the identification of the movable supply, which determines tax liabilities. Human tax advisors can obtain a graphical visualization of the structured representation to verify the correctness of the LLM's output while the law-based rules return reliable decisions.</p>
      </abstract>
      <kwd-group>
        <kwd>eol&gt;Neuro-symbolic artificial intelligence</kwd>
        <kwd>Knowledge graphs</kwd>
        <kwd>Decision support systems</kwd>
        <kwd>Tax management</kwd>
        <kwd>Valueadded tax</kwd>
      </kwd-group>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>1. Introduction</title>
      <p>
        Tax advisors typically receive natural-language case descriptions from clients, which are not uniformly
structured. Before providing a legally founded analysis of the client’s situation, a tax advisor must first
make sense of the situation by structuring the case description. Ultimately, a legally-founded decision
is grounded in logic [
        <xref ref-type="bibr" rid="ref1">1</xref>
        ]. Thus, the tax advisor’s task consists of language understanding and logical
reasoning. In this paper, we focus on the chain-transaction cases in value-added-tax (VAT) law. In
cross-border chain transactions between multiple enterprises, VAT liabilities are determined according
to a set of rules. The movable supply in a chain transaction determines the tax liabilities.
      </p>
      <p>
        A large language model (LLM) is a type of machine-learning model with billions of parameters trained
on vast amounts of text data to predict the probability of the next word (or token) given a sequence
of previous words, with applications in many domains, e.g., in the legal field [
        <xref ref-type="bibr" rid="ref2 ref3 ref4 ref5 ref6">2, 3, 4, 5, 6</xref>
        ], including
applications in tax advisory [
        <xref ref-type="bibr" rid="ref10 ref7 ref8 ref9">7, 8, 9, 10</xref>
        ] and summarization of legal documents [
        <xref ref-type="bibr" rid="ref11 ref12 ref13">11, 12, 13</xref>
        ]. Nevertheless,
the well-known problem of hallucination limits the use of LLMs in sensitive areas such as the legal
ifeld, since the necessary reliable and comprehensible decisions cannot be ensured [
        <xref ref-type="bibr" rid="ref14 ref15 ref16 ref17">14, 15, 16, 17</xref>
        ].
LLMs show promising results in the extraction of knowledge from unstructured natural language
texts [18, 19]. In this regard, LLMs are becoming an increasingly popular tool for transforming natural
language texts into knowledge graphs [20, 18, 21, 22, 23], which serve as a framework for structuring
knowledge in a comprehensible format streamlined for certain applications [24]. In this paper, we
therefore investigate the capabilities of the integration of LLMs as knowledge extractors to build a
knowledge-graph representation of legal cases, which can be used as the basis for legal reasoning. To
analyze a VAT case, the natural-language description of the case is translated into a knowledge graph,
and rules can be executed to derive additional information. Tax advisors can obtain a visualization of
the extracted knowledge graph to easily verify the correct representation of a case.
      </p>
      <p>We employ the design-science research methodology by Wieringa [25]. Following Wieringa’s
template, we formulate the design problem as follows.</p>
      <p>• Increase the eficiency of tax consultants (problem context)
• by designing an LLM-based method (artifact)
• that (requirements)</p>
      <sec id="sec-1-1">
        <title>1. identifies movable and unmovable supplies in chain transactions</title>
        <p>2. by processing natural-language case descriptions
3. to conduct a logically founded and explainable analysis,
• enabling the allocation of correct tax liabilities under Austrian tax law (stakeholder goals).</p>
      </sec>
      <sec id="sec-1-2">
        <title>From this design problem, we derive the following research question.</title>
        <p>How can LLMs be used to extract information from natural-language text to form the basis for
logically founded and explainable decisions regarding chain transactions in Austrian tax law?
To answer the research question, we implemented the LLM-based method for analyzing VAT
chaintransaction cases illustrated in Fig. 1; the implementation is available in an online repository [26].
An LLM serves to construct a structured knowledge graph presentation from the natural-language
description of a VAT chain-transaction case. More specifically, the LLM outputs Cypher statements to
build the knowledge graph for storage in a Neo4j database. Law-based rules executed over the contents
of the knowledge graph then identify the movable supply, providing the basis for a decision in the VAT
chain-transaction case.</p>
        <p>We use diferent datasets for development of the prototype and evaluation of the performance (see
Fig. 2); data from the experimental evaluation is available in an online repository [27]. We use example
cases from Kollmann’s textbook on chain transactions [28] for prompt engineering. To evaluate
the performance of the developed prototype, we conducted experiments with example cases from
other textbooks [29, 30] as well as real-world cases and exam questions from a university course.
We measure the prototype’s performance in terms of the correctness of the obtained knowledge
graph representations from the natural-language descriptions of VAT chain-transaction cases and
the subsequent identification of the movable supply. The results of these experiments suggest good
performance of LLMs for knowledge graph construction, with an overall accuracy of 94.74 %. On
correctly constructed knowledge graphs, the law-based rules work as expected and never fail to
correctly identify the movable supply.</p>
        <p>The hybrid approach of using subsymbolic AI for language understanding and symbolic AI for actual
legal reasoning has clear advantages in terms of explainability and understandability for a human
tax advisor using the AI system for decision-making. A human tax advisor can manually review the
graphical representation of the chain transaction to verify the accuracy of the output, and law-based
rules provide reliable logical reasoning. A purely subsymbolic AI approach, e.g., using LLMs both for
language understanding and legal reasoning [31, 32], lacks the reliable logical reasoning; an LLM does
not really perform logical reasoning. Furthermore, a human tax advisor cannot easily review the LLM
“reasoning” output without conducting both an analysis of the case and the legal reasoning.</p>
        <p>The remainder of this paper is organized as follows. In Sect. 2, we provide a brief introduction to
the subject of chain transactions in VAT law. In Sect. 3, we present an implementation of the proposed
method. In Sect. 4, we describe the experimental evaluation of the implemented prototype of the
proposed method. In Sect. 5, we review related work. In Sect. 6, we conclude the paper with a summary
and an outlook on future work.</p>
      </sec>
    </sec>
    <sec id="sec-2">
      <title>2. Background: Chain Transactions in VAT Law</title>
      <p>VAT is a tax on consumption that is added to goods and services [33], which in Europe is harmonized
by EU regulation, implemented into national legislation by EU Member States. A chain transaction
is a series of at least two transactions for the same goods that are transported from the first supplier
to the last customer in a single transportation operation [29]; a chain transaction involves at least
three enterprises. To allocate the correct VAT liabilities in case of cross-border chain transactions, the
Austrian Value-Added Tax Act (“Umsatzsteuergesetz”) 1994 [34], like similar legal provisions in other
countries, defines step by step how to determine taxation of a regular chain transaction.</p>
      <p>The movable supply in a cross-border chain transaction determines the VAT liabilities, i.e., which
countries levy the tax. The movable supply is the delivery connected to the actual movement, i.e.,
transportation, of goods. In general, for most VAT chain-transaction cases, the responsibility for
transportation of the goods indicates the movable supply. The party responsible for transportation
can be the first party, the last party, or an intermediary in the chain transaction. If the first or the
last party in a VAT chain transaction has the responsibility for transportation, the movable supply is
the supply of the respective party. If the party responsible for transportation is an intermediary, the
indicated VAT identification number, i.e., “an individual identification number for companies that are
registered for VAT” [35], influences the decision regarding the movable supply. Depending on the VAT
identification number, the movable supply is either that of the intermediary or that of the party before
the intermediary. The responsibility for transportation and, consequently, the movable supply then
determines the allocation of tax liabilities.</p>
      <p>The following example, adapted from the textbook by Mayr and Weinzierl [29], describes VAT
chain-transaction case and the corresponding identification of the movable supply (see Fig. 3 for a
visualization).</p>
      <p>Case Description. The enterprise F from France orders goods from the enterprise OE2 in Salzburg,
Austria. The latter does not have the goods in stock and buys those goods from the enterprise OE1 in
Klagenfurt, Austria. OE1 has the goods transported directly from Austria to F in France on OE1’s own
account by a carrier. OE2 presents itself to OE1 with an Austrian VAT number.</p>
      <p>Solution. The first supplier in this example is OE1. The last customer is F and the intermediary is
OE2. According to the case description, OE1 is responsible for the transportation of the goods; a supplier
is not automatically responsible for transportation. Therefore, OE1’s supply is the movable supply (OE1
→ OE2). In the graph representation (Fig. 3), the movable supply relationship points in the opposite
direction of the corresponding order relationship.</p>
    </sec>
    <sec id="sec-3">
      <title>3. Implementation</title>
      <p>In this section, we describe the implementation of the proposed method, including the LLM-based
knowledge graph construction and the rule-based identification of the movable supply. The implementation
is available in an online repository [26].</p>
      <p>Listing 1: Condensed version of the prompt used for knowledge graph construction (translated from</p>
      <p>German)
I d e n t i t y : Tax c o n s u l t a n t f o r A u s t r i a n VAT law s p e c i a l i z e d on c h a i n t r a n s a c t i o n s
Response r u l e s : r e p r e s e n t a t i o n a s knowledge graph ; c a p t u r e t h e e n t i r e c h a i n
t r a n s a c t i o n ; no c h a n g e s o r a d d i t i o n s a l l o w e d
Detai led i n s t r u c t i o n s : i d e n t i f i c a t i o n o f e n t e r p r i s e s , t r a n s a c t i o n s , and t r a n s p o r t
r e s p o n s i b i l i t y ; i d e n t i f i c a t i o n o f a v a i l a b l e p r o p e r t y v a l u e s f o r Name , Country
and VATid ( i f VATid i s not s t a t e d , g e n e r a t e a VATid u s i n g t h e c o u n t r y o f t h e
r e s p e c t i v e e n t e r p r i s e and a c o n s e c u t i v e number ) ; d e f i n i t i o n o f a l l o w e d
v o c a b u l a r y , a s s i g n m e n t o f t r a n s p o r t r e s p o n s i b i l i t y and d i r e c t i o n o f r e l a t i o n s ;
d o u b l e c h e c k query s t a t e m e n t s w i t h your e x p l a n a t i o n
Structured output : I n c l u d e Cypher s t a t e m e n t s , a r g u m e n t a t i o n , and a summary .</p>
      <sec id="sec-3-1">
        <title>3.1. Knowledge Graph Construction from Case Description</title>
        <p>We use the Neo4j Aura graph database for knowledge graph management. To construct a knowledge
graph from natural-language case descriptions, an LLM was instructed to generate Cypher statements
based on the information contained in the textual description. We used GPT-4.1 as LLM due to its
ability to follow instructions and its long-context understanding [36]. The LLM was accessed via the
LangSmith framework published by LangChain, which provides comprehensive monitoring and prompt
engineering possibilities as well as data management for sample data [37]. Our model configuration of
GPT-4.1 was set to a temperature value of 0 and a top_p value of 1. This allowed the model to provide
deterministic results and reduce the risk of hallucination [38].</p>
        <p>We note that the general principle demonstrated in this paper is agnostic to the choice of underlying
database technology. The graph nature of VAT chain transactions makes a graph database such as
Neo4j Aura a convenient choice. An alternative to using Neo4j Aura as a database would be to use an
RDF database. Although the Resource Description Framework (RDF) [39] is also typically considered a
graph-based representation of data, RDF focuses more on expressing statements consisting of subject,
predicate, and object, rather than relationships between objects. When using an RDF database, SPARQL
statements [40] or Datalog statements [41] would then have to be generated analogously to the Cypher
statements presented in this paper, which would also be possible. An RDF database would have the
advantage of better support for ontology languages, e.g., OWL, and often provide the corresponding
built-in inference mechanisms for ontological reasoning, which would probably have been of limited
utility in the presented use case, though.</p>
        <p>Regarding prompt design, we adopted the principles mentioned in the GPT-4.1 prompting guide [42]
as well as best-practice approaches regarding query generation with ChatGPT from the experiments by
Meyer et al. [21]. We used a zero-shot approach, i.e., an approach without task-specific training, because
its positive performance has already been demonstrated in previous work by Sciannameo et al. [43]
on knowledge extraction using LLMs and work by Carta et al. [44] on knowledge graph construction
using LLMs. Furthermore, the use of a zero-shot approach ensures the ability of the application to
process various case settings [22], which was essential due to the diversity of VAT cases. The cases
used for prompt engineering were taken from course material on chain transactions by Kollmann [28],
a recognized VAT expert. A total of 20 cases were selected, which cover various real-world scenarios
and contain all the necessary information to solve and represent a valid chain transaction.</p>
        <p>Listing 1 shows a condensed version of a prompt, translated to English from the original German
version. The full prompt can be viewed in the online repository [26]. The prompt was developed
through an iterative process involving 35 experiments using the Kollmann cases, with the prompt
refined after each iteration to achieve an accuracy of 100 % on the Kollmann cases. The prompt is
structured as follows (see Listing 1). Initially, an identity is assigned to the model and a number of
high-level response rules are specified. Then, detailed instructions with example statements are declared
to define a specific behavior for recurring patterns in the text. In addition, we opted for a structured
output schema to ensure that the output contains the Cypher query to insert the knowledge graph into
the graph database, an explanation of how the query statements were derived, and a summary of the
overall case. Explanation and summary are important for a human tax advisor to be able to validate the
results of the extraction.</p>
        <p>The underlying schema of the knowledge graph was defined by a restricted vocabulary in the prompt.
The knowledge graph’s content is illustrated in Fig. 4. The knowledge graph consists of two types
of nodes, namely Enterprise nodes and Transport Responsibility nodes. Enterprise nodes have Name,
Country, and VATidproperties. Furthermore, Enterprise nodes in the extracted knowledge graph are
connected by two types of relationships, namely ORDER and GOODS MOVEMENT. The relationships are
directed and have a Product property. The Transport Responsibility node, which has a Product property,
is connected to an Enterprise node by the HAS relationship, which does not have any property. Beside
the case description as basis for the knowledge graph construction, a tax law specialist was engaged in
the development of the knowledge graph and validated it in accordance with tax law provisions and
juridical methods.</p>
      </sec>
      <sec id="sec-3-2">
        <title>3.2. Rule-Based Identification of Movable Supply</title>
        <p>The identification of the movable supply is performed using a combination of Cypher queries and
programmatic control structures (if–then–else statements) with logical expressions in Python. The
implementation includes a check whether the chain transaction is valid. The required information for
evaluating the rules is extracted using a set of Cypher queries, which are executed over the previously
constructed knowledge graph stored in the graph database. In particular, the queries retrieve the first
and last enterprise in the chain transaction, the country of dispatch of the goods, and the transport
responsibility as well as the numbers of goods traded, orders, enterprises, movement of goods, and
transport responsibilities. Furthermore, using the outputs from the previous queries, other queries
retrieved the first and last supply, the intermediary supplies, and the supply of the enterprise before the
intermediary.</p>
        <p>We derived the rules from §3(15) of the Austrian Value-Added Tax Act (Umsatzsteuergesetz 1994) [34],
which we therefore refer to as law-based rules. We chose this law-based process since in standard
chain-transaction cases the law ofers a clear step-by-step guide on how such a case must be solved.
The first set of rules uses the information from the queries to check if the chain transaction is valid or
not (Listing 2). The second set of rules, when applied on a valid chain transaction, uses information
about the first, last, and intermediary enterprises in the chain transaction to determine the movable
supply (Listing 3).</p>
        <p>Besides the textual representation of the movable supply, our implementation generates a visualization
of the chain transaction (see Fig. 3), using the Graphviz library [45], the output of which is also stored
in the knowledge graph. Thus, the implementation generates a machine-readable output on the one
hand and a visual representation comprehensible for humans on the other hand.</p>
        <sec id="sec-3-2-1">
          <title>Listing 2: Validation of Chain Transaction</title>
          <p>I F number o f ORDER r e l a t i o n s h i p s i s &gt;= 2 AND
number o f ENTERPRISE nodes &gt;= 3 AND
t h e GOODS MOVEMENT edge s t a r t s a t t h e f i r s t e n t e r p r i s e AND
t h e GOODS MOVEMENT edge ends a t t h e l a s t e n t e r p r i s e AND
a l l p r o d u c t p r o p e r t i e s o f e d g e s have t h e same v a l u e AND
t h e r e e x i s t s o n l y one GOODS MOVEMENT edge AND
t h e r e i s o n l y one T r a n s p o r t R e s p o n s i b i l i t y node
THEN
ELSE</p>
          <p>R e t u r n " V a l i d Chain T r a n s a c t i o n "
R e t u r n " I n v a l i d Chain T r a n s a c t i o n "</p>
        </sec>
        <sec id="sec-3-2-2">
          <title>Listing 3: Identification of Movable Supply Type</title>
          <p>I F f i r s t _ e n t e r p r i s e HAS T r a n s p o r t R e s p o n s i b i l i t y THEN
m o v a b l e _ s u p p l y : = " F i r s t S u p p l y "
ELSE I F l a s t _ e n t e r p r i s e HAS T r a n s p o r t r e s p o n s i b i l i t y THEN
m o v a b l e _ s u p p l y : = " L a s t S u p p l y "
ELSE I F i n t e r m e d i a r y HAS T r a n s p o r t r e s p o n s i b i l i t y THEN
I F c o u n t r y o f VATid o f i n t e r m e d i a r y =</p>
          <p>c o u n t r y o f f i r s t _ e n t e r p r i s e THEN
m o v a b l e _ s u p p l y : = " I n t e r m e d i a r y ’ s S u p p l y "
ELSE</p>
          <p>m o v a b l e _ s u p p l y : = " B e f o r e I n t e r m e d i a r y S u p p l y "</p>
        </sec>
      </sec>
    </sec>
    <sec id="sec-4">
      <title>4. Experimental Evaluation</title>
      <p>In this section, we describe the datasets and the setup for experiments with the implementation of the
proposed method. We then present and discuss the results of the experiments.</p>
      <sec id="sec-4-1">
        <title>4.1. Datasets and Experimental Setup</title>
        <p>To evaluate the proposed method and the prototype implementation, we selected a total of 167 VAT
chain-transaction cases from four diferent sources. Most cases were obtained from two
practicallyoriented textbooks, namely, 72 cases from a Mayr and Weinzierl [29] and 67 cases from Dupont [30].
Another 24 cases were selected from real-world cases, which were collected by a tax consultant. The
remaining four examples were taken from an exam on tax management at Johannes Kepler University
Linz. The dataset is available in an online repository [27].</p>
        <p>With regard to the length of the case descriptions from the diferent datasets, measured in terms
of token size, which we obtained from LangSmith, we noticed the following diferences (see Table
1). Relative to the “training set” by Kollmann, which we used for prompt engineering, the Mayr and
Weinzierl (M&amp;W) as well as the Dupont datasets were similar to the Kollmann dataset in terms of
mean token size. However, the higher standard deviation and maximum values for M&amp;W and Dupont
indicate that those dataset also contained some cases with longer descriptions. The longest M&amp;W
case description was 250 tokens longer, and the longest Dupont case was 177 tokens longer than the
mean of the Kollmann dataset. The M&amp;W and Dupont cases were more similar to real-life cases than
those in the Kollmann dataset. The real-world cases were more varied in terms of token size. In
general, the real-world cases were even longer than the M&amp;W and Dupont cases. The cases from the
tax management exam were similar to the real-world cases in terms of extensiveness and structure.</p>
        <p>In summary, the real-world cases and the exam cases had longer descriptions and either contained
more information which was not necessary to solve the case or the case was more extensive in general.
Consequently, they were more complex and dificult to understand, which made them harder to solve.
The M&amp;W, Dupont and the exam datasets already provided sample solutions containing the movable
supply, which was necessary for evaluating the performance of the implementation. The real-world cases
were solved by tax consultants, who manually identified the movable supply so that the performance of
the implementation could be evaluated.</p>
        <p>Each case was manually checked to ensure that the case was complete and not an exceptional case.
Incomplete and exceptional cases were excluded from further consideration. This restriction was
necessary because assumptions would have been necessary without complete information and this was
fundamentally limited by the configuration of the LLM. In addition, the use of law-based rules also
prevented the application from interpreting the law or making assumptions. A total of 15 cases from
M&amp;W had to be excluded because they were exceptional cases. Seven of these excluded cases involved
incomplete transportation of goods (e.g., due to a car accident), and six cases fell under an exemption
for electronic platform transactions, which are handled diferently to normal chain transactions. One
case involved a credulity assumption, which leads to diferent outcome than a regular case, and one case
contained contradictory information and was therefore undecidable. Moreover, two exceptional cases
were excluded from the Dupont dataset because they contained a shared transport responsibility, which
is also handled diferently than regular chain transactions. From the real-world cases, nine cases were
excluded. Of these excluded cases, four cases were excluded for containing more than one transaction,
which could not be represented in the current implementation, while three cases were excluded due
to missing or unclear information that cannot be resolved by a human and, therefore, also cannot be
resolved by the implementation. One real-world case was excluded due to the use of a consignment
warehouse, which required special handling, and one case was excluded due to shared transportation
responsibility. Eventually, we had to exclude also one exam case because it involved more than one
transaction. Ultimately, 57 cases of M&amp;W, 65 of Dupont, 15 real-world cases, and 3 exam cases were
suitable for the experimental consideration. The datasets are available in an online repository [27].</p>
        <p>Regarding real-world applicability, the exclusion of certain cases in the evaluation means the
following. The use of an AI-based decision support system for VAT cases is most useful for routine cases.
Exceptional and more complicated cases have to be reviewed from a human tax expert. Such cases
would not be solved by the implemented prototype.</p>
      </sec>
      <sec id="sec-4-2">
        <title>4.2. Results</title>
        <p>Table 2 summarizes the performance of the LLM for knowledge graph construction from the
naturallanguage case descriptions. For the Kollmann dataset, which was used for prompt engineering, the LLM
performs with 100 % accuracy, which is not surprising since we optimized the prompt based on these
cases. For our “test sets” M&amp;W and Dupont the accuracy for knowledge graph construction is 92.98 %
and 95.38 %, respectively. From the M&amp;W dataset, four cases were not represented correctly, even
though they included a valid chain transaction and the description included the required information.
From the Dupont dataset, three cases were not represented correctly. Focusing on the real-world
cases, the implementation performed at 93.33 % accuracy, with one incorrectly represented case. The
implementation was able to obtain correct representations for all three of the considered exam questions.</p>
        <p>Table 3 summarizes the performance of the rules in correctly identifying the movable supply in the
correctly represented VAT chain-transaction cases. Only those cases were considered where the LLM
produced a correct knowledge-graph representation. The results demonstrate the proper implementation
of the rules, which will provide the correct output when using correct knowledge-graph representations
as input.</p>
      </sec>
    </sec>
    <sec id="sec-5">
      <title>5. Related Work</title>
      <p>In the following, we review existing literature with respect to the contribution in this paper. We present
a novel LLM application in the field of tax law. The proposed approach aims to improve the reliability
and explainability of LLM applications by incorporating KGs and rule-based decision-making to mitigate
the problem of hallucination in sensitive domains and to provide explainable and reproducible results
from LLMs.</p>
      <sec id="sec-5-1">
        <title>5.1. LLM Applications in Tax Law</title>
        <p>
          Our work presents a novel LLM application in the legal domain, the proposed method allowing to
analyze VAT chain-transaction cases in Austrian tax law. Related LLM applications are designed for
various specific tasks, which can be categorized into applications to summarize legal documents [
          <xref ref-type="bibr" rid="ref11 ref12">12, 11</xref>
          ],
extract knowledge from legal documents [46], provide legal advice [
          <xref ref-type="bibr" rid="ref10 ref7 ref8 ref9">10, 7, 9, 8</xref>
          ], perform metamorphic
tax-software tests [
          <xref ref-type="bibr" rid="ref6">6</xref>
          ], or solve legal exams [47, 48]. The AMELIA system [
          <xref ref-type="bibr" rid="ref12">12</xref>
          ] and OntoVAT [46] focus
specifically on VAT law. AMELIA incorporates LLMs for argument mining on decisions on cases in
Italian VAT la. OntoVAT uses ontologies for knowledge extraction and LLMs for generating numeric
embeddings. Other related work [31, 32] employs LLMs to analyze VAT cases, investigating the potential
of LLMs to identify the place of delivery from textual descriptions of VAT textbook cases as well as a
general analysis of real-world VAT cases. Retrieval-augmented generation and fine-tuning are employed
to improve performance of LLMs. Crucially, unlike this paper, the related work employs LLM for both
natural language processing and legal reasoning. This approach difers from our approach: In our
case, the LLM output is a structured representation of the natural-language case description and the
law-based rules serve for actual legal reasoning.
        </p>
      </sec>
      <sec id="sec-5-2">
        <title>5.2. Using LLMs for Knowledge Graph Construction</title>
        <p>
          Our work provides insights into the possibilities of KG construction for solving complex tasks using
LLMs. The KG produced by the output of the LLM serves as the basis for actual legal decision-making.
The work of [21] already illustrated the possibilities of KG construction via simple prompt instructions.
Furthermore, [22] emphasized the use of zero-shot approaches to ensure applicability across various
scenarios. Related work demonstrated that zero-shot approaches can eficiently extract knowledge
graphs [44] and that they can outperform one-shot approaches for extraction tasks with GPT-4 in terms
of accurately extracted entities [23]. Other work [23] points out that LLMs do have limitations in KG
construction tasks, which can efectively be addressed by prompt engineering, though. Therefore, we
improved the prompt by providing the LLM with instructions on how to process recurring textual
patterns and formulations to increase the accuracy of the output. Nevertheless, hallucinations limit
the potential of LLMs for KG construction [23]. Moreover, hallucination is a frequently mentioned
problem that leads to random and untraceable results [
          <xref ref-type="bibr" rid="ref15 ref8">8, 15</xref>
          ]. In particular, LLM applications in the
legal field are severely restricted by hallucination, since comprehensible and explainable results are
required in this critical domain [
          <xref ref-type="bibr" rid="ref14 ref15">14, 15</xref>
          ]. Although there are techniques to detect hallucinations [
          <xref ref-type="bibr" rid="ref16">16</xref>
          ],
the corresponding results still need to be compared with related literature to ensure their factuality and
plausibility [
          <xref ref-type="bibr" rid="ref17">17</xref>
          ]. In our work, we aimed to mitigate hallucinations by removing randomization with a
temperature setting of 0, which was also proposed by Beutel et al. [38]. In addition, we used structured
knowledge to create a comprehensible and reliable basis for decision-making as well as to reduce the
risk of ambiguous and non-reproducible results.
        </p>
      </sec>
    </sec>
    <sec id="sec-6">
      <title>6. Summary and Future Work</title>
      <p>Tax advisors receive natural-language case descriptions from clients, which must be interpreted before
rendering a decision on tax liabilities. In this paper, we proposed a method and implementation for
using LLMs for structuring descriptions of VAT chain-transaction cases and apply rules derived from
VAT law to produce decisions compliant with the law. As opposed to purely LLM-based approaches,
the presented method is more reliable and explainable, the validity of the results can be easier checked
by human tax advisors and used as evidence for purposes of tax compliance and communication with
ifnancial authorities.</p>
      <p>We evaluated the approach using realistic use cases from textbooks and real-world practice.
Improvements could be made regarding the handling of natural-language descriptions that actually contain
more than one case. Future work will also investigate the applicability of the approach, where LLMs
serve to extract structured representations from natural-language case descriptions and legal-based
rules serve for actual decision-making, to other areas of tax law.</p>
    </sec>
    <sec id="sec-7">
      <title>Declaration on Generative AI</title>
      <p>During the preparation of this work, the authors used Writefull to check spelling and grammar, as well
as for suggestions for rewording and paraphrasing. After using this tool, the authors reviewed and
edited the content as needed and take full responsibility for the publication’s content.
arXiv:https://onlinelibrary.wiley.com/doi/pdf/10.1111/jels.12413.
[18] P. Bellan, M. Dragoni, C. Ghidini, Process Knowledge Extraction and Knowledge Graph
Construction Through Prompting: A Quantitative Analysis, in: Proceedings of the 39th ACM/SIGAPP
Symposium on Applied Computing, SAC ’24, Association for Computing Machinery, New York,
NY, USA, 2024, pp. 1634–1641. doi:10.1145/3605098.3635957.
[19] A. G. Regino, J. Cesar dos Reis, Generating e-commerce related knowledge graph from text: Open
challenges and early results using llms, 2024.
[20] D. N. Ribeiro, K. Forbus, Combining analogy with language models for knowledge extraction, in:
3rd Conference on Automated Knowledge Base Construction, 2021. doi:10.24432/C5KK5X.
[21] L.-P. Meyer, C. Stadler, J. Frey, N. Radtke, K. Junghanns, R. Meissner, G. Dziwis, K. Bulert, M. Martin,
LLM-assisted Knowledge Graph Engineering: Experiments with ChatGPT, in: C. Zinke-Wehlmann,
J. Friedrich (Eds.), First Working Conference on Artificial Intelligence Development for a Resilient
and Sustainable Tomorrow, Springer Fachmedien Wiesbaden, Wiesbaden, 2024, pp. 103–115.
doi:10.1007/978-3-658-43705-3_8, series Title: Informatik aktuell.
[22] Y. Lairgi, L. Moncla, R. Cazabet, K. Benabdeslem, P. Cléau, iText2KG: Incremental Knowledge
Graphs Construction Using Large Language Models, 2024. doi:10.48550/ARXIV.2409.03284,
version Number: 1.
[23] Y. Zhu, X. Wang, J. Chen, S. Qiao, Y. Ou, Y. Yao, S. Deng, H. Chen, N. Zhang, LLMs for knowledge
graph construction and reasoning: recent capabilities and future opportunities, World Wide Web
27 (2024) 58. doi:10.1007/s11280-024-01297-w.
[24] M. A. Haque, M. Kamal, R. George, K. D. Gupta, Utilizing structural metrics from knowledge
graphs to enhance the robustness quantification of large language models, International Journal
of Data Science and Analytics (2024). doi:10.1007/s41060-024-00643-5.
[25] R. J. Wieringa, Design Science Methodology for Information Systems and Software Engineering,</p>
      <p>Springer Berlin Heidelberg, Berlin, Heidelberg, 2014. doi:10.1007/978-3-662-43839-8.
[26] L. Knogler, VAT chain transactions solver - the application for the research paper: Using large
language models and knowledge graphs for deciding VAT chain-transaction cases in Austrian tax
law, 2025. doi:10.5281/zenodo.16112381.
[27] L. Knogler, Experimental results for the research paper: Using large language models and
knowledge graphs for deciding VAT chain-transaction cases in Austrian tax law, 2025. doi:10.5281/
zenodo.16112863.
[28] G. Kollmann, Reihen- und Dreiecksgeschäfte, Umsatzsteuer aktuell, 2024. URL: http://www.</p>
      <p>gerh-kollmann.at/Reihengeschaefte%20Kollmann%20August%202024.pdf.
[29] M. Mayr, C. Weinzierl, SWK-Spezial Reihen- und Dreiecksgeschäfte, SWK-Spezial, 3. edition 2023
ed., Linde Verlag Ges.m.b.H, Wien, 2023.
[30] F. Dupont, Umsatzsteuer: Reihen-und Dreiecksgeschäfte in Beispielen; Praxishandbuch; mit
praxisgerechten Erläuterungen, Grafiken und Beispielen, Kitzler, 2012.
[31] M. Luketina, C. G. Schuetz, T. Wageneder, An Experimental Evaluation of the Capability of Large
Language Models to Reason About Value-Added Tax Cases in Austrian Tax Law, Proceedings of
the 2024 Pre-ICIS SIGDSA Symposium (2024). URL: https://aisel.aisnet.org/sigdsa2024/15.
[32] A. Benkel, Using Large Language Models for Legal Decision Making in Austrian Value-Added Tax</p>
      <p>Law: an Experimental Investigation of Retrieval-Augmented Generation and Fine-Tuning, 2025.
[33] European Commission, VAT rules and rates: standard, special &amp; reduced rates, 2025. URL: https:
//europa.eu/youreurope/business/taxation/vat/vat-rules-rates/index_en.htm.
[34] Umsatzsteuergesetz 1994, 2023. URL: https://ris.bka.gv.at/GeltendeFassung.wxe?Abfrage=</p>
      <p>Bundesnormen&amp;Gesetzesnummer=10004873&amp;FassungVom=2025-06-05, accessed: 2025-06-04.
[35] Austrian Federal Ministry of Finance, Business Service Portal: VAT Identification
Number, 2025. https://www.usp.gv.at/en/themen/steuern-finanzen/umsatzsteuer-ueberblick/
weitere-informationen-zur-umsatzsteuer/umsaetze-mit-auslandsbezug/
umsatzsteuer-identifikationsnummer.html, accessed 2025-08-01.
[36] OpenAI, Introducing GPT-4.1 in the API, 2025. URL: https://openai.com/index/gpt-4-1/.
[37] LangSmith, Get started with LangSmith | LangSmith, 2025. URL: https://docs.smith.langchain.com/.</p>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          [1]
          <string-name>
            <given-names>H.</given-names>
            <surname>Kelsen</surname>
          </string-name>
          ,
          <article-title>Law and logic</article-title>
          ,
          <source>in: Essays in Legal and Moral Philosophy</source>
          , Springer,
          <year>1973</year>
          , pp.
          <fpage>228</fpage>
          -
          <lpage>253</lpage>
          . doi:
          <volume>10</volume>
          .1007/
          <fpage>978</fpage>
          -94-010-2653-6_
          <fpage>10</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          [2]
          <string-name>
            <given-names>J.</given-names>
            <surname>Shi</surname>
          </string-name>
          ,
          <string-name>
            <given-names>Q.</given-names>
            <surname>Guo</surname>
          </string-name>
          ,
          <string-name>
            <given-names>Y.</given-names>
            <surname>Liao</surname>
          </string-name>
          ,
          <string-name>
            <given-names>Y.</given-names>
            <surname>Wang</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S.</given-names>
            <surname>Chen</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S.</given-names>
            <surname>Liang</surname>
          </string-name>
          , Legal-LM:
          <article-title>Knowledge Graph Enhanced Large Language Models for Law Consulting</article-title>
          , in: D.
          <string-name>
            <surname>-S. Huang</surname>
            ,
            <given-names>Z.</given-names>
          </string-name>
          <string-name>
            <surname>Si</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          Zhang (Eds.),
          <source>Advanced Intelligent Computing Technology and Applications</source>
          , volume
          <volume>14878</volume>
          , Springer Nature Singapore, Singapore,
          <year>2024</year>
          , pp.
          <fpage>175</fpage>
          -
          <lpage>186</lpage>
          . doi:
          <volume>10</volume>
          .1007/
          <fpage>978</fpage>
          -981-97-5672-8_15, series Title: Lecture Notes in Computer Science.
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          [3]
          <string-name>
            <given-names>J.</given-names>
            <surname>Shi</surname>
          </string-name>
          ,
          <string-name>
            <given-names>Q.</given-names>
            <surname>Guo</surname>
          </string-name>
          ,
          <string-name>
            <given-names>Y.</given-names>
            <surname>Liao</surname>
          </string-name>
          , S. Liang,
          <article-title>LegalGPT: Legal Chain of Thought for the Legal Large Language Model Multi-agent Framework</article-title>
          , in: D.
          <string-name>
            <surname>-S. Huang</surname>
            ,
            <given-names>Z.</given-names>
          </string-name>
          <string-name>
            <surname>Si</surname>
          </string-name>
          , W. Chen (Eds.),
          <source>Advanced Intelligent Computing Technology and Applications</source>
          , volume
          <volume>14880</volume>
          , Springer Nature Singapore, Singapore,
          <year>2024</year>
          , pp.
          <fpage>25</fpage>
          -
          <lpage>37</lpage>
          . doi:
          <volume>10</volume>
          .1007/
          <fpage>978</fpage>
          -981-97-5678-
          <issue>0</issue>
          _3,
          <string-name>
            <surname>series</surname>
            <given-names>Title</given-names>
          </string-name>
          : Lecture Notes in Computer Science.
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          [4]
          <string-name>
            <given-names>D.</given-names>
            <surname>Shu</surname>
          </string-name>
          ,
          <string-name>
            <given-names>H.</given-names>
            <surname>Zhao</surname>
          </string-name>
          ,
          <string-name>
            <given-names>X.</given-names>
            <surname>Liu</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D.</given-names>
            <surname>Demeter</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Du</surname>
          </string-name>
          ,
          <string-name>
            <surname>Y. Zhang,</surname>
          </string-name>
          <article-title>LawLLM: Law Large Language Model for the US Legal System</article-title>
          ,
          <year>2024</year>
          . doi:
          <volume>10</volume>
          .1145/3627673.3680020, arXiv:
          <fpage>2407</fpage>
          .21065 [cs].
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          [5]
          <string-name>
            <given-names>S.</given-names>
            <surname>Ghosh</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D.</given-names>
            <surname>Verma</surname>
          </string-name>
          ,
          <string-name>
            <given-names>B.</given-names>
            <surname>Ganesan</surname>
          </string-name>
          ,
          <string-name>
            <given-names>P.</given-names>
            <surname>Bindal</surname>
          </string-name>
          ,
          <string-name>
            <given-names>V.</given-names>
            <surname>Kumar</surname>
          </string-name>
          ,
          <string-name>
            <given-names>V.</given-names>
            <surname>Bhatnagar</surname>
          </string-name>
          ,
          <article-title>Human centered ai for indian legal text analytics</article-title>
          ,
          <year>2024</year>
          . arXiv:
          <volume>2403</volume>
          .
          <fpage>10944</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          [6]
          <string-name>
            <given-names>D.</given-names>
            <surname>Srinivas</surname>
          </string-name>
          ,
          <string-name>
            <surname>R. Das</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          <string-name>
            <surname>Tizpaz-Niari</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          <string-name>
            <surname>Trivedi</surname>
            ,
            <given-names>M. L.</given-names>
          </string-name>
          <string-name>
            <surname>Pacheco</surname>
          </string-name>
          ,
          <article-title>On the potential and limitations of few-shot in-context learning to generate metamorphic specifications for tax preparation software</article-title>
          ,
          <year>2023</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          [7]
          <string-name>
            <given-names>M.</given-names>
            <surname>Kaoutar</surname>
          </string-name>
          ,
          <string-name>
            <given-names>B.</given-names>
            <surname>Chaima</surname>
          </string-name>
          ,
          <string-name>
            <given-names>B.</given-names>
            <surname>Omar</surname>
          </string-name>
          ,
          <string-name>
            <given-names>B.</given-names>
            <surname>Outmane</surname>
          </string-name>
          ,
          <article-title>Unlocking the Potential of Large Language Models in Legal Discourse: Challenges, Solutions, and Future Directions</article-title>
          , in: 2024
          <source>Sixth International Conference on Intelligent Computing in Data Sciences (ICDS)</source>
          ,
          <year>2024</year>
          . doi:
          <volume>10</volume>
          .1109/ICDS62089.
          <year>2024</year>
          .
          <volume>10756345</volume>
          .
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          [8]
          <string-name>
            <given-names>T.</given-names>
            <surname>Seabrooke</surname>
          </string-name>
          ,
          <string-name>
            <given-names>E.</given-names>
            <surname>Schneiders</surname>
          </string-name>
          ,
          <string-name>
            <given-names>L.</given-names>
            <surname>Dowthwaite</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J.</given-names>
            <surname>Krook</surname>
          </string-name>
          ,
          <string-name>
            <given-names>N.</given-names>
            <surname>Leesakul</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J.</given-names>
            <surname>Clos</surname>
          </string-name>
          ,
          <string-name>
            <given-names>H.</given-names>
            <surname>Maior</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J.</given-names>
            <surname>Fischer</surname>
          </string-name>
          ,
          <article-title>A Survey of Lay People's Willingness to Generate Legal Advice using Large Language Models (LLMs)</article-title>
          ,
          <source>in: Proceedings of the Second International Symposium on Trustworthy Autonomous Systems</source>
          ,
          <year>2024</year>
          . doi:
          <volume>10</volume>
          .1145/3686038.3686043.
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          [9]
          <string-name>
            <given-names>J.</given-names>
            <surname>Nay</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D.</given-names>
            <surname>Karamardian</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S.</given-names>
            <surname>Lawsky</surname>
          </string-name>
          ,
          <string-name>
            <given-names>W.</given-names>
            <surname>Tao</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Bhat</surname>
          </string-name>
          ,
          <string-name>
            <given-names>R.</given-names>
            <surname>Jain</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Lee</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J.</given-names>
            <surname>Choi</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J.</given-names>
            <surname>Kasai</surname>
          </string-name>
          ,
          <article-title>Large language models as tax attorneys: a case study in legal capabilities emergence</article-title>
          ,
          <source>Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences</source>
          <volume>382</volume>
          (
          <year>2024</year>
          ). doi:
          <volume>10</volume>
          .1098/rsta.
          <year>2023</year>
          .
          <volume>0159</volume>
          .
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          [10]
          <string-name>
            <given-names>J.</given-names>
            <surname>Presa</surname>
          </string-name>
          ,
          <string-name>
            <given-names>C.</given-names>
            <surname>Camilo Junior</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S.</given-names>
            <surname>Oliveira</surname>
          </string-name>
          ,
          <article-title>Evaluating Large Language Models for Tax Law Reasoning</article-title>
          ,
          <source>in: Intelligent Systems</source>
          , volume
          <volume>15412</volume>
          <source>of LNAI</source>
          ,
          <year>2025</year>
          , pp.
          <fpage>460</fpage>
          -
          <lpage>474</lpage>
          . doi:
          <volume>10</volume>
          .1007/ 978-3-
          <fpage>031</fpage>
          -79029-4_
          <fpage>32</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref11">
        <mixed-citation>
          [11]
          <string-name>
            <given-names>A.</given-names>
            <surname>Fidelangeli</surname>
          </string-name>
          ,
          <string-name>
            <given-names>F.</given-names>
            <surname>Galli</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Loreggia</surname>
          </string-name>
          , G. Pisano,
          <string-name>
            <given-names>R.</given-names>
            <surname>Rovatti</surname>
          </string-name>
          ,
          <string-name>
            <given-names>P.</given-names>
            <surname>Santin</surname>
          </string-name>
          ,
          <string-name>
            <surname>G.</surname>
          </string-name>
          <article-title>Sartor, The Summarization of Italian Tax-Law Decisions: The Case of the PRODIGIT Project</article-title>
          ,
          <source>IEEE Access 13</source>
          (
          <year>2025</year>
          )
          <fpage>38833</fpage>
          -
          <lpage>38855</lpage>
          . doi:
          <volume>10</volume>
          .1109/ACCESS.
          <year>2025</year>
          .
          <volume>3545419</volume>
          .
        </mixed-citation>
      </ref>
      <ref id="ref12">
        <mixed-citation>
          [12]
          <string-name>
            <given-names>G.</given-names>
            <surname>Grundler</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Galassi</surname>
          </string-name>
          ,
          <string-name>
            <given-names>P.</given-names>
            <surname>Santin</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Fidelangeli</surname>
          </string-name>
          ,
          <string-name>
            <given-names>F.</given-names>
            <surname>Galli</surname>
          </string-name>
          ,
          <string-name>
            <given-names>E.</given-names>
            <surname>Palmieri</surname>
          </string-name>
          ,
          <string-name>
            <given-names>F.</given-names>
            <surname>Lagioia</surname>
          </string-name>
          , G. Sartor, P. Torroni,
          <article-title>AMELIA - Argument Mining Evaluation on Legal documents in ItAlian: A CALAMITA Challenge</article-title>
          , in: CLiC-it
          <source>2024: Tenth Italian Conference on Computational Linguistics</source>
          ,
          <year>2024</year>
          . URL: https:// ceur-ws.
          <source>org/</source>
          Vol-
          <volume>3878</volume>
          /124_calamita_long.pdf.
        </mixed-citation>
      </ref>
      <ref id="ref13">
        <mixed-citation>
          [13]
          <string-name>
            <given-names>A.</given-names>
            <surname>Deroy</surname>
          </string-name>
          ,
          <string-name>
            <given-names>K.</given-names>
            <surname>Ghosh</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S.</given-names>
            <surname>Ghosh</surname>
          </string-name>
          ,
          <article-title>How Ready are Pre-trained Abstractive Models and LLMs for Legal Case Judgement Summarization?</article-title>
          ,
          <year>2023</year>
          . doi:
          <volume>10</volume>
          .48550/arXiv.2306.01248, arXiv:
          <fpage>2306</fpage>
          .01248 [cs].
        </mixed-citation>
      </ref>
      <ref id="ref14">
        <mixed-citation>
          [14]
          <string-name>
            <given-names>R.</given-names>
            <surname>El Hamdani</surname>
          </string-name>
          ,
          <string-name>
            <given-names>T.</given-names>
            <surname>Bonald</surname>
          </string-name>
          ,
          <string-name>
            <given-names>F. D.</given-names>
            <surname>Malliaros</surname>
          </string-name>
          ,
          <string-name>
            <given-names>N.</given-names>
            <surname>Holzenberger</surname>
          </string-name>
          ,
          <string-name>
            <given-names>F.</given-names>
            <surname>Suchanek</surname>
          </string-name>
          ,
          <article-title>The Factuality of Large Language Models in the Legal Domain</article-title>
          ,
          <source>in: Proceedings of the 33rd ACM International Conference on Information and Knowledge Management</source>
          ,
          <string-name>
            <given-names>ACM</given-names>
            ,
            <surname>Boise</surname>
          </string-name>
          <string-name>
            <surname>ID USA</surname>
          </string-name>
          ,
          <year>2024</year>
          , pp.
          <fpage>3741</fpage>
          -
          <lpage>3746</lpage>
          . doi:
          <volume>10</volume>
          .1145/3627673.3679961.
        </mixed-citation>
      </ref>
      <ref id="ref15">
        <mixed-citation>
          [15]
          <string-name>
            <given-names>M.</given-names>
            <surname>Dahl</surname>
          </string-name>
          ,
          <string-name>
            <given-names>V.</given-names>
            <surname>Magesh</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Suzgun</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D. E.</given-names>
            <surname>Ho</surname>
          </string-name>
          ,
          <article-title>Large legal fictions: Profiling legal hallucinations in large language models</article-title>
          ,
          <source>Journal of Legal Analysis</source>
          <volume>16</volume>
          (
          <year>2024</year>
          )
          <fpage>64</fpage>
          -
          <lpage>93</lpage>
          . doi:
          <volume>10</volume>
          .1093/jla/laae003.
        </mixed-citation>
      </ref>
      <ref id="ref16">
        <mixed-citation>
          [16]
          <string-name>
            <given-names>Y.</given-names>
            <surname>Chen</surname>
          </string-name>
          ,
          <string-name>
            <given-names>Q.</given-names>
            <surname>Fu</surname>
          </string-name>
          ,
          <string-name>
            <given-names>Y.</given-names>
            <surname>Yuan</surname>
          </string-name>
          ,
          <string-name>
            <given-names>Z.</given-names>
            <surname>Wen</surname>
          </string-name>
          ,
          <string-name>
            <given-names>G.</given-names>
            <surname>Fan</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D.</given-names>
            <surname>Liu</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D.</given-names>
            <surname>Zhang</surname>
          </string-name>
          ,
          <string-name>
            <given-names>Z.</given-names>
            <surname>Li</surname>
          </string-name>
          ,
          <string-name>
            <given-names>Y.</given-names>
            <surname>Xiao</surname>
          </string-name>
          , Hallucination Detection:
          <article-title>Robustly Discerning Reliable Answers in Large Language Models</article-title>
          ,
          <source>in: Proceedings of the 32nd ACM International Conference on Information and Knowledge Management</source>
          ,
          <string-name>
            <surname>ACM</surname>
          </string-name>
          , Birmingham United Kingdom,
          <year>2023</year>
          , pp.
          <fpage>245</fpage>
          -
          <lpage>255</lpage>
          . doi:
          <volume>10</volume>
          .1145/3583780.3614905.
        </mixed-citation>
      </ref>
      <ref id="ref17">
        <mixed-citation>
          [17]
          <string-name>
            <given-names>V.</given-names>
            <surname>Magesh</surname>
          </string-name>
          ,
          <string-name>
            <given-names>F.</given-names>
            <surname>Surani</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Dahl</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Suzgun</surname>
          </string-name>
          ,
          <string-name>
            <given-names>C. D.</given-names>
            <surname>Manning</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D. E.</given-names>
            <surname>Ho</surname>
          </string-name>
          ,
          <article-title>Hallucinationfree? assessing the reliability of leading ai legal research tools</article-title>
          ,
          <source>Journal of Empirical Legal Studies</source>
          <volume>22</volume>
          (
          <year>2025</year>
          )
          <fpage>216</fpage>
          -
          <lpage>242</lpage>
          . doi:https://doi.org/10.1111/jels.12413.
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>