<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta>
      <journal-title-group>
        <journal-title>June</journal-title>
      </journal-title-group>
    </journal-meta>
    <article-meta>
      <title-group>
        <article-title>Modelling Norm Types and their Inter-relationships in EU Directives</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Ilaria Angela Amantea</string-name>
          <email>amantea@di.unito.it</email>
          <xref ref-type="aff" rid="aff0">0</xref>
          <xref ref-type="aff" rid="aff1">1</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Luigi Di Caro</string-name>
          <email>dicaro@di.unito.it</email>
          <xref ref-type="aff" rid="aff0">0</xref>
          <xref ref-type="aff" rid="aff1">1</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Llio Humphreys</string-name>
          <email>humphrey@di.unito.it</email>
          <xref ref-type="aff" rid="aff0">0</xref>
          <xref ref-type="aff" rid="aff1">1</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Rohan Nanda</string-name>
          <email>nanda@di.unito.it</email>
          <xref ref-type="aff" rid="aff0">0</xref>
          <xref ref-type="aff" rid="aff1">1</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Emilio Sulis</string-name>
          <email>sulis@di.unito.it</email>
          <xref ref-type="aff" rid="aff0">0</xref>
          <xref ref-type="aff" rid="aff1">1</xref>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          <institution>Natural language processing</institution>
          ,
          <addr-line>Law, Machine learning, Recitals, European legislation, purposive interpretation, cosine similarity</addr-line>
        </aff>
        <aff id="aff1">
          <label>1</label>
          <institution>University of Torino Torino</institution>
          ,
          <country country="IT">Italy</country>
        </aff>
      </contrib-group>
      <pub-date>
        <year>2019</year>
      </pub-date>
      <volume>21</volume>
      <issue>2019</issue>
      <abstract>
        <p>EU directives are deliberately imprecise to allow Member States to fulfill the objectives in their own way. It follows that deontic norms in EU directives cannot be properly understood without consideration of the motivation behind those norms, as well as consideration of how other provisions, or even other laws, may afect the scope or efectiveness of the norm in question. This paper seeks to model norm types and their inter-relationships. We envisage that this preliminary analysis may help to develop an automated system to ifnd norms that are related in diferent ways in order to help legal professionals interpret laws for specific cases.</p>
      </abstract>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>INTRODUCTION</title>
      <p>
        Norms are not “legal flowers without stem or root” [
        <xref ref-type="bibr" rid="ref39">39</xref>
        ], page 27. A
normative provision almost always has to be read in the context
of other norms. Generally, the law has a holistic character (cf. [
        <xref ref-type="bibr" rid="ref43">43</xref>
        ]
and [
        <xref ref-type="bibr" rid="ref19">19</xref>
        ] among others) which emerges from a network of legal
documents. This means that at times the meaning of legal norms
emerges not from single parts of a normative provision but from a
wider legislative corpus. Indeed, it is arguable that the context is
even wider including parliamentary debates, legal common practice,
and doctrinal interpretation. As such, legal interpretation takes
much efort, and can be helped by automated eforts to find such
links.
      </p>
      <p>Less researched perhaps are important links between normative
provisions in the same piece of legislation. Our manual analysis of
a lengthy European Union (EU) directive revealed not only very
diferent kinds of norms, but also diferent kinds of relationships
between norms. While EU directives have particular
characteristics (outlined in the next section), the general point remains that
understanding of particular normative provisions can be greatly
enhanced by reading them in conjunction with those norms that
influence them in some way. Needless to say, not all normative
provisions are of such relevance, and our research goal is therefore
to help legal researchers find those provisions by semi-automated
means. It is hoped some of the insights in this paper about the
nature of norm types and relation between them will pave the way
to automated detection of diferent kinds of norms and relations.</p>
      <p>This research is concerned with normative provisions and recitals
in EU directives. While recitals lack the prescriptive status of
normative provisions proper (their main purpose being to provide the
wider context), they have been shown to yield some influence on
the interpretation of normative provisions. The research questions
of this paper are therefore:
(1) What kind of norms are present in normative provisions and
recitals?
(2) What kind of links are there between norms, be they
normative provisions or recitals?</p>
      <p>Section 2 describes the challenges of legal reasoning, European
law in the global context, and the nature of EU directives. Section
3 introduces our classification of norm types and section 4 our
classification of link types. Section 5 provides issues for discussion,
and section 6 outlines preliminary ideas for automated classification.
Section 7 describes related work, and section 8 ends the paper with
conclusions and future work.</p>
    </sec>
    <sec id="sec-2">
      <title>BACKGROUND 2 2.1</title>
    </sec>
    <sec id="sec-3">
      <title>The Nature of Legal Reasoning</title>
      <p>
        The problem of vagueness pertains to all human reasoning, not
just legal interpretation. As Sainsbury [
        <xref ref-type="bibr" rid="ref41">41</xref>
        ], page 324, explains:
‘Vagueness gives rise to borderline cases. Think, for example, of
a colour spectrum. There are clear cases of red and clear cases of
orange, and in between there are borderline cases: shades which
we don’t feel inclined to classify either as red or as orange.’
      </p>
      <p>
        Travis [
        <xref ref-type="bibr" rid="ref47">47</xref>
        ], pages 171-172, shows how context can influence the
meaning of even a simple question such as whether a leaf is green:
‘Suppose a Japanese maple leaf, turned brown, was painted green
for a decoration. In sorting leaves by colour, one might truly call
this one green. In describing leaves to help identify their species, it
might, for all the paint, be false to call it that. ’
      </p>
      <p>
        In the legal context the most famous example of such phenomena
is the vehicle in the park scenario outlined by Hart [
        <xref ref-type="bibr" rid="ref20">20</xref>
        ], page 607:
‘A legal rule forbids you to take a vehicle into a public park. Plainly
this forbids an automobile, but what about bicycles, roller skates,
toy automobiles? What about airplanes?...We may call the problems
which arise outside the hard core of penumbral instances “problems
of the penumbra”...If a penumbra of uncertainty must surround all
legal rules, then their application to specific cases in the penumbral
area cannot be a matter of logical deduction, and so deductive
reasoning, which for generations has been cherished as the very
perfection of human reasoning, cannot serve as a model for what
judges, or indeed what anyone, should do.’
      </p>
      <p>
        These problems are well-known and have been alluded to in the
AI &amp; Law community: ‘The law is normally represented in natural,
albeit technical, language: the language of statutes and cases. These
sources of law are not the law itself, but one possible representation
of the law. It is clear that these documents are not themselves the
law from the fact, that we must first interpret statutes and cases
to get at the law which they represent, and from the fact that
reasonable persons can disagree as to just what the law is, although
there is rarely disagreement as to what, words make up the statute
or case in question. It is the meaning of the statute or case which is
the law, not the text of the document itself.’ [
        <xref ref-type="bibr" rid="ref17">17</xref>
        ], page 2.
      </p>
      <p>
        The degree of interpretation required is dependent on the
nature of the legal document itself. Some technical legal instruments,
such as medical clinical guidelines, are very precise, such that it is
possible to identify the hierarchy of norms and model them with
defeasible logic [
        <xref ref-type="bibr" rid="ref37">37</xref>
        ]. We argue that a diferent approach is required
for legislation such as EU directives, which are more contextual and
abstract. One issue, as mentioned above, is that the determination
of whether a normative provision applies to a particular case is
rendered dificult when the norm is less precise. The other issue is
that the handling of conflicting legal principles is diferent to the
handling of normative rules. Rather than one principle winning
to the exclusion of the other, more often conflicting principles are
‘balanced’ to ensure that aspects of both principles are respected
in a way that is proportional and fair. The demarcation between
principles and rules is a point of contention, with some experts
arguing that it is a continuous spectrum [
        <xref ref-type="bibr" rid="ref14">14</xref>
        ].
2.2
      </p>
    </sec>
    <sec id="sec-4">
      <title>The Nature of European Law in the Global</title>
    </sec>
    <sec id="sec-5">
      <title>Context</title>
      <p>
        The phenomenon of globalisation has put into question the
interstate structure of the international community. The global
dimension of some phenomena (for example climate change,
underdevelopment, immigration, terrorism, nuclear proliferation, financial
transfers, information technology, human rights etc) have shown
the inadequacy of an international community founded upon the
independence and sovereignty of states. They require international
rules and institutions due to their transnational character.
Traditionally, international law governs relations between independent
states, The norms that bind states originate from their free will in
treaties or custom. The development of international law tends to
reduce state autonomy. In certain matters, such as human rights,
environmental protection, financial crime and terrorism,
international law binds national law. There has been a movement from
international commercial relations from direct inter-state
mechanisms (above all the General Agreement on Tarifs and Trade)
towards a system of international economic relations in the sphere
of specific international organisations bestowed with normative,
penal and judicial powers. For example, the evolution of GATT
into the World Trade Organisation, as well as other international
institutions such as the World Bank created to resolve controversies
in the world of investment. Unlike states, international
organisations are not given general competence, but are governed by the
principle of specialisation and limited powers to pursue common
interests attributed by the states. Not all organisations have a legal
personality. To this end, they must a) be given suficient autonomy,
also organisational, distinct from that of member states b) have a
well-defined mission with corresponding competence and status
within the international community. There are actors that do not
truly have an international legal personality, but are involved and
participate in such activities, including NGOs and multinational
companies. The first represents public interests of the universal
civil society, while the second represents productive interests of
the current economic-financial system [
        <xref ref-type="bibr" rid="ref10">10</xref>
        ].
      </p>
      <p>International law leaves great liberty to states in their choice
of implementation, being interested only in that the objectives are
achieved. This principle is not dissimilar to treaties of the European
Union. Notwithstanding much similarity to international law,
European Union law has certain characteristics that render it unique. In
particular: "[t]o exercise the Union’s competences, the institutions
shall adopt regulations, directives, decisions, recommendations and
opinions. A regulation shall have general application. It shall be
binding in its entirety and directly applicable in all Member States.
A directive shall be binding, as to the result to be achieved, upon
each Member State to which it is addressed, but shall leave to the
national authorities the choice of form and methods. A decision shall
be binding in its entirety. A decision which specifies those to whom
it is addressed shall be binding only on them. Recommendations
and opinions shall have no binding force"1.</p>
      <p>The limits of EU competences are governed by the principles
of conferral2: "the Union shall act only within the limits of the
competences conferred upon it by the Member States in the Treaties
to attain the objectives set out therein. Competences not conferred
upon the Union in the Treaties remain with the Member States"3.</p>
      <p>The use of Union competences is governed by the principles of:
• subsidiarity: "in areas which do not fall within its
exclusive competence, the Union shall act only if and in so far as
the objectives of the proposed action cannot be suficiently
achieved by the Member States [...] but can rather [...] be
better achieved at Union level"4. Moreover, "when the Treaties
confer on the Union a competence shared with the Member
States in a specific area, the Union and the Member States
may legislate and adopt legally binding acts in that area.
The Member States shall exercise their competence to the
extent that the Union has not exercised its competence. The
Member States shall again exercise their competence to the
extent that the Union has decided to cease exercising its
competence"5.
• proportionality: "the content and form of Union action shall
not exceed what is necessary to achieve the objectives of the
Treaties"6.</p>
      <p>The primary characteristic that distinguishes the European Union
from other international organisations is that member states have
relinquished some sovereign powers to the European Union.</p>
      <p>It follows other interesting phenomena. First, the European
Union has the competence to conclude agreements with third states
1Article 288 Treaty on the Functioning of the European Union (TFEU)
2Article 5.1 Treaty on European Union (TEU).
3Underline bot in the Article 4.1 and in the Article 5.2 of TEU.
4Articles 5.1 and 5.3 of TEU. See also Article 4.1 of TEU and Article 352 of TFEU
5Article 2.2 TFEU. We want to underline that the concept of subsidiarity also exist
in some Member States government, but with a meaning more similar to division of
competences e.g. see Article 118 of the Italian Constitution.
6Articles 5.1 and 5.4 of TEU.
or international organisations, where such competences have been
expressly given, or if they are acting within their limits.</p>
      <p>Secondly, according to the "principle of sincere cooperation, the
Union and the Member States shall [...] take any appropriate
measure, general or particular, to ensure fulfilment of the obligations
arising out of the Treaties or resulting from the acts of the
institutions of the Union [...] and refrain from any measure which could
jeopardise the attainment of the Union’s objectives"7. The
increasing trend towards harmonisation of EU law has put the European
Court of Justice as the supreme arbiter of European law, and
national judges are required to interpret their own laws in accordance
with European law, even when there is apparent conflict between
those laws8. The European Court can "review the legality of acts of
bodies, ofices or agencies. It shall for this purpose have jurisdiction
in actions brought by a Member State, the European Parliament,
the Council or the Commission on grounds of lack of competence,
infringement of an essential procedural requirement, infringement
of the Treaties or of any rule of law relating to their application, or
misuse of powers."9. Furthermore, another particular characteristic
of the EU is that "[a]ny natural or legal person may [...] institute
proceedings against an act addressed to that person or which is of
direct and individual concern to them", and more important "and
against a regulatory act which is of direct concern to them and does
not entail implementing measures"10. "If the action is well founded,
the Court of Justice of the European Union shall declare the act
concerned to be void"11. It means that the Court has the power
to "delete" some national law, and with it, delete also the future,
present, and even past efects that law, as if that law never existed.
2.3</p>
    </sec>
    <sec id="sec-6">
      <title>The Nature of European Directives</title>
      <p>
        Our focus in this paper is on European directives, which as
mentioned above, are prescriptive but suficiently general to allow
member states to articulate their own detailed norms and procedures as
they prefer in order to achieve the goal of the directive. Being by
nature goal-oriented, directives are particularly given to
principlebased (balance) rather than defeasible reasoning. Almost half of
the text of directives consists of recitals, which are intended to be
explanatory and do not have the same status as normative
provisions. There are diferent doctrinal positions [
        <xref ref-type="bibr" rid="ref24">24</xref>
        ] on the relationship
between recitals and normative provisions:
(1) recitals have no efect;
(2) recitals are dominant over normative provisions;
(3) recitals have an equal position in relation to normative
provisions;
(4) recitals encompass a subordinate position towards normative
provisions.
We are cognizant that the ECJ has assumed both positions 3 and
4 in its judicature in cases 24/6212 and C-162/9713 The proportion
of recitals in directives has increased over the years. Kierkegaard
alleges [
        <xref ref-type="bibr" rid="ref23">23</xref>
        ] that “recitals are used by the Member States to insert
normative provisions which they have failed to get into the text,
and by the Commission to dump normative provisions which they
do not want to prolong debate and disagreement on”. From all this
we can conclude that recitals cannot be ignored, but their influence
is uncertain.
3
      </p>
    </sec>
    <sec id="sec-7">
      <title>CLASSIFICATION OF NORM TYPES</title>
      <p>There are diferent kinds of norms with diferent functions (they
serve diferent purposes), and these diferent types are found in
both normative provisions and recitals. Our categorisation is purely
semantic and is intended to be generalizable to all directives. In
particular, we have found that the structural aspects (e.g. type of
modal verb used) are not a definitive indication. In this work, we
have found 5 diferent types of norms: objective, constitutive,
deontic, scope and meta-norms (procedural and contextual). All the
examples provided come from a lengthy directive which we have
analysed in some detail: Directive 2004/23/EC of the European
Parliament and of the Council of 31 March 2004 on setting standards
of quality and safety for the donation, procurement, testing,
processing, preservation, storage and distribution of human tissues
and cells.</p>
      <p>Outlines the purpose behind the directive as a whole,
or some parts of it, and the wider social and legal
context. Sometimes the objective is lofty and is a
general principle for the existence of the directive. Other
times the objective may be a more specific sub-goal.</p>
      <p>ARTICLE 1: This Directive lays down standards of
quality and safety for human tissues and cells
intended for human applications, in order to ensure
a high level of protection of human health.</p>
      <sec id="sec-7-1">
        <title>Objective</title>
        <sec id="sec-7-1-1">
          <title>Definition:</title>
        </sec>
        <sec id="sec-7-1-2">
          <title>Example 1:</title>
        </sec>
      </sec>
      <sec id="sec-7-2">
        <title>Constitutive</title>
        <sec id="sec-7-2-1">
          <title>Definition:</title>
          <p>
            Oficial definitions of directive-specific technical
concepts. Legislative constitutive norms are usually
general descriptions, but also not uncommon are
definitions by example, which allow extension by analogy,
as well as definitions that explicitly include or exclude
certain items from counting as the legal concept in
question.
12Position 3: Case 24/62, F.R.G. v. Comm’n of the Eur. Econ. Cmty., 1963 E.C.R.,
paragraph 18, retrieved at
http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=CELEX:61962J0024:EN:NOT
13Position 4: Case C-162/97, Nilsson et al, paragraph 54, 1998, E.C.R. I-07477, available
at
http://eurlex.europa.eu/LexUriServ/LexUriServ.do?uri=CELEX:61997J0162:EN:NOT
ARTICLE 3: For the purposes of this Directive: (a)
“cells” means individual human cells or a collection of
human cells when not bound by any form of
connective tissue; (b) [...]
Specifies types of behaviour to be expected or
permitted. Deontic norms have been further classified as
permission, obligation, prohibition etc (see [
            <xref ref-type="bibr" rid="ref21">21</xref>
            ] for an
elaborate study), but this level of detail is not required
for our purposes.
          </p>
          <p>ARTICLE 7.7: Member States shall, upon the request
of another Member State or the Commission, provide
information on the results of inspections and control
measures carried out in relation to the requirements
of this Directive.</p>
        </sec>
        <sec id="sec-7-2-2">
          <title>Example 6:</title>
        </sec>
        <sec id="sec-7-2-3">
          <title>Definition:</title>
        </sec>
        <sec id="sec-7-2-4">
          <title>Example 7:</title>
        </sec>
        <sec id="sec-7-2-5">
          <title>Example 2:</title>
        </sec>
      </sec>
      <sec id="sec-7-3">
        <title>Deontic</title>
        <p>RECITAL 11: This Directive does not cover research
using human tissues and cells, such as when used for
purposes other than application to the human body,
e.g. in vitro research or in animal models. Only those
cells and tissues that in clinical trials are applied to
the human body should comply with the quality and
safety standards laid down in this Directive.</p>
      </sec>
      <sec id="sec-7-4">
        <title>Meta-norms</title>
        <p>
          Meta-norms are norms about norms. Laws do not only pertain
to certain behaviours but also (1) the way in which the norms
are produced, (2) the way in which they are applied and (3) the
way in which upon violation a sanction is imposed. The distinction
between norms and meta-norms is that the first concerns behaviour,
the second concerns other norms or the production or application
of other norms [
          <xref ref-type="bibr" rid="ref15">15</xref>
          ]. We distinguish between two kinds of
metanorms:
Meta-norms: Procedural
Procedural refers to step-by-step processes for
implementing law e.g. get signatures, agreement from the
        </p>
        <p>Committee, further signatures.</p>
        <sec id="sec-7-4-1">
          <title>Example 5:</title>
          <p>ARTICLE 10.3: Member States and the Commission
shall establish a network linking the national tissue
establishment registers.</p>
          <p>While the above example is a procedure particular to the
subjectmatter of the directive, there are also other procedural norms that
occur in practically all directives, and are part of the law-making
process.</p>
          <p>ARTICLE 26.1: Member States shall send the
Commission, before 7 April 2009 and every three years
thereafter, a report on the activities undertaken in
relation to the provisions of this Directive,
including an account of the measures taken in relation to
inspection and control.</p>
        </sec>
      </sec>
      <sec id="sec-7-5">
        <title>Meta-norms: Contextual</title>
        <p>Contextualisation is about time, space, addressee and
hierarchy of norms.</p>
        <p>ARTICLE 32: This Directive shall enter into force on
the day of its publication in the Oficial Journal of the
European Union.
4</p>
      </sec>
    </sec>
    <sec id="sec-8">
      <title>CLASSIFICATION OF LINK TYPES</title>
      <p>Norms are related to one another in many diferent ways. Below
we provide definitions with examples from the above-mentioned
Directive 2004/23/EC, and are mainly focused on links between
normative provisions and recitals. Where not all parts of the recital
or (sub)article are connected, the underlined sections highlight the
related parts of the text.</p>
      <sec id="sec-8-1">
        <title>Conceptually Similar</title>
        <sec id="sec-8-1-1">
          <title>Definition:</title>
        </sec>
        <sec id="sec-8-1-2">
          <title>Example 8 (using same wording):</title>
          <p>There is content within the provisions that are about
the same subject-matter and may use similar or
different wording to say more or less the same thing.
ARTICLE 2.1: This Directive shall apply to the donation,
procurement, testing, processing, preservation, storage
and distribution of human tissues and cells intended
for human applications and of manufactured
products derived from human tissues and cells intended
for human applications. Where such manufactured
products are covered by other directives, this
Directive shall apply only to donation, procurement and
testing.
RECITAL 13: The donation, procurement, testing,
processing, preservation, storage and distribution of
human tissues and cells intended for human applications
should comply with high standards of quality and
safety in order to ensure a high level of health
protection in the Community. This Directive should
establish standards for each one of the steps in the human
tissues and cells application process.</p>
        </sec>
        <sec id="sec-8-1-3">
          <title>Example 9 (using diferent wording):</title>
          <p>ARTICLE 5.1: Member States shall ensure that tissue and
cell procurement and testing are carried out by persons
with appropriate training and experience and that they
take place in conditions accredited, designated,
authorised or licensed for that purpose by the competent
authority or authorities.</p>
          <p>RECITAL 27 : Personnel directly involved in the
donation, procurement, testing, processing, preservation,
storage and distribution of human tissues and cells
should be appropriately qualified and provided with
timely and relevant training. The provisions laid down
in this Directive as regards training should be
applicable without prejudice to existing Community
legislation on the recognition of professional qualifications.</p>
        </sec>
      </sec>
      <sec id="sec-8-2">
        <title>Constitutive</title>
        <sec id="sec-8-2-1">
          <title>Definition:</title>
        </sec>
        <sec id="sec-8-2-2">
          <title>Example 10:</title>
          <p>The constitutive link is where one provision provides
a definition of domain-specific terms contained in
another provision. Often, directives contain a glossary
of terms in one specific Article.</p>
          <p>ARTICLE 3: For the purposes of this Directive:
(a) ‘cells’ means individual human cells or a collection
of human cells when not bound by any form of
connective tissue;
(b) ‘tissue’ means all constituent parts of the human
body formed by cells;
(c) ‘donor’ means every human source, whether living
or deceased, of human cells or tissues;
(d) ‘donation’ means donating human tissues or cells
intended for human applications; [...]
(f) ‘procurement’ means a process by which tissue or
cells are made available; [...]
(p) ‘allogeneic use’ means cells or tissues removed
from one person and applied to another; [...]
RECITAL 16 Tissues and cells used for allogeneic
therapeutic purposes can be procured from both living
and deceased donors. In order to ensure that the health
status of a living donor is not afected by the donation,
a prior medical examination should be required. The
dignity of the deceased donor should be respected,
notably through the reconstruction of the donor’s
body, so that it is as similar as possible to its original
anatomical shape.</p>
          <p>In this case Article 3 contains definitions of many domain-specific
terms in Recital 16, so there is a Constitutive link between these
provisions.</p>
        </sec>
      </sec>
      <sec id="sec-8-3">
        <title>Motivation</title>
        <sec id="sec-8-3-1">
          <title>Definition:</title>
          <p>A link between a deontic norm and the motivation
behind it. The provisions have the same goal but
different levels of granularity. Sometimes the motivation
is lofty and is a general principle for the existence of
the directive, alternatively the motivation may be a
core value of human rights or a fundamental principle
of European Treaties. Other times the motivation may
be a more specific sub-goal.</p>
        </sec>
        <sec id="sec-8-3-2">
          <title>Example 11:</title>
          <p>ARTICLE 16.3: Tissue establishments shall take all
necessary measures to ensure that the quality system
includes at least the following documentation:
– standard operating procedures,
– guidelines,
– training and reference manuals,
– reporting forms,
– donor records,
– information on the final destination of tissues or
cells.</p>
          <p>RECITAL 1: The transplantation of human tissues and
cells is a strongly expanding field of medicine ofering
great opportunities for the treatment of as yet
incurable diseases. The quality and safety of these substances
should be ensured, particularly in order to prevent the
transmission of diseases.</p>
          <p>RECITAL 15: It is necessary to increase confidence
among the Member States in the quality and safety of
donated tissues and cells, in the health protection of
living donors and respect for deceased donors and in
the safety of the application process.</p>
        </sec>
      </sec>
      <sec id="sec-8-4">
        <title>Impact</title>
        <sec id="sec-8-4-1">
          <title>Definition:</title>
          <p>A provision may afect the scope or efectiveness of
another provision, or add additional requirements.
This may be because the norm requires some kind of
synchronisation with another procedure in another
norm, or because two norms have conflicting goals.</p>
          <p>The content may be granularity independent.</p>
        </sec>
        <sec id="sec-8-4-2">
          <title>Example 12 (conflicting goals):</title>
          <p>ARTICLE 8.1: Member States shall ensure that all
tissues and cells procured, processed, stored or distributed
on their territory can be traced from the donor to the
recipient and vice versa. This traceability shall also apply
to all relevant data relating to products and materials
coming into contact with these tissues and cells.</p>
          <p>RECITAL 23: All necessary measures need to be taken
in order to provide prospective donors of tissues and
cells with assurances regarding the confidentiality of any
health related information provided to the authorised
personnel, the results of tests on their donations, as
well as any future traceability of their donation.</p>
          <p>We can reason that traceability is a sub-goal of health protection,
and that confidential information is a sub-goal of respecting the
dignity of the individual, and that both these higher goals are
subgoals of maintaining the well-being of persons. However, there is a
potential risk of conflict between these sub-goals e.g. traceability
can be achieved without anonymity. As such, it is useful to link
such norms to any norms that impose restrictions or additional
requirements.</p>
        </sec>
        <sec id="sec-8-4-3">
          <title>Example 13 (enforcement):</title>
          <p>ARTICLE 6.3: The tissue establishment shall not
undertake any substantial changes to its activities without
the prior written approval of the competent authority
or authorities.</p>
          <p>RECITAL 26: Member States should organise inspections
and control measures, to be carried out by oficials
representing the competent authority, to ensure that
tissue establishments comply with the provisions of this
Directive. Member States should ensure that the
oficials involved in inspections and control measures are
appropriately qualified and receive adequate training.</p>
          <p>RECITAL 30: In order to increase the efective
implementation of the provisions adopted in accordance
with this Directive, it is appropriate to provide for
penalties to be applied by Member States.</p>
          <p>The impact of Recitals 26 and 30 on Article 6.3 is to render the norm
enforceable via monitoring and penalties.</p>
        </sec>
      </sec>
      <sec id="sec-8-5">
        <title>Indirect Internal</title>
        <sec id="sec-8-5-1">
          <title>Definition:</title>
          <p>An indirect internal link is a structural (not semantic)
link that exists purely because of an internal reference
to a (sub)article. An indirect structural link between
Recital X and (Sub)article Y depends on the existence
of a primary semantic link between Recital X and
another (sub)article that is cited by (Sub)article Y.</p>
        </sec>
        <sec id="sec-8-5-2">
          <title>Example 14:</title>
          <p>ARTICLE 11.3: The responsible person referred to in
Article 17 shall ensure that the competent authority or
authorities is or are notified of any serious adverse events
and reactions referred to in paragraph 1 and is or are
provided with a report analysing the cause and the
ensuing outcome.</p>
          <p>ARTICLE 11.1: Member States shall ensure that there
is a system in place to report, investigate, register and
transmit information about serious adverse events and
reactions which may influence the quality and safety of
tissues and cells and which may be attributed to the
procurement, testing, processing, storage and
distribution of tissues and cells, as well as any serious
adverse reaction observed during or after clinical
application which may be linked to the quality and
safety of tissues and cells.</p>
          <p>RECITAL 25: An accreditation system for tissue
establishments and a system for notification of adverse events and
reactions linked to the procurement, testing, processing,
preservation, storage and distribution of human tissues and
cells should be established in the Member States.</p>
          <p>In this example, Article 11.3 is indirectly structurally linked to
Recital 25 because Article 11.3 refers to Article 11.1, which is
Conceptually Similar to Recital 25.</p>
        </sec>
      </sec>
      <sec id="sec-8-6">
        <title>Via Other Law</title>
        <sec id="sec-8-6-1">
          <title>Definition:</title>
        </sec>
        <sec id="sec-8-6-2">
          <title>Example 15:</title>
          <p>When an article mentions another legal source.
Applies only if reading this source is required in order to
understand the provision or recital. This is a structural
link, and not a semantic link.</p>
          <p>ARTICLE 13.1 The procurement of human tissues or
cells shall be authorised only after all mandatory
consent or authorisation requirements in force in the
Member State concerned have been met.</p>
          <p>RECITAL 22: This Directive respects the
fundamental rights and observes the principles reflected in the
Charter of Fundamental Rights of the European Union
(1) and takes into account as appropriate the
Convention for the protection of human rights and dignity
of the human being with regard to the application of
biology and medicine: Convention on human rights
and biomedicine. Neither the Charter nor the
Convention makes express provision for harmonisation
or prevents Member States from introducing more
stringent requirements in their legislation.</p>
          <p>In the above example, the fundamental rights and dignity of the
human being mentioned in Recital 22 is the motivation behind the
requirement for consent for the procurement of human tissues and
cells in Article 13.1.</p>
        </sec>
      </sec>
      <sec id="sec-8-7">
        <title>Procedural</title>
        <sec id="sec-8-7-1">
          <title>Definition:</title>
          <p>Involving routine bureaucratic procedures of some
precision. The procedures referred to here are only
descriptions of what the Commission, other EU body,
or Member State, have undertaken to do to render the
directive efective. They are not norms in the deontic
sense. Procedural links do not pertain to norms about
what lower bodies within Member States are required
to do, which are still deontic norms, in the sense that
they impose an obligation.</p>
        </sec>
        <sec id="sec-8-7-2">
          <title>Example 16:</title>
          <p>ARTICLE 29.1: The Commission shall be assisted by a
Committee.</p>
          <p>RECITAL 34: The measures necessary for the
implementation of this Directive should be adopted in
accordance with Council Decision 1999/468/EC of 28 June
1999 laying down the procedures for the exercise of
implementing powers conferred on the Commission.
The procedural link exists to distinguish between deontic norms
and descriptions of procedures that render the law efective.
However, the linked sub-articles and recitals can have diferent levels of
granularity.</p>
        </sec>
      </sec>
      <sec id="sec-8-8">
        <title>Contextual</title>
        <sec id="sec-8-8-1">
          <title>Definition:</title>
          <p>Contextualising the applicability of all norms involved
in the directive in terms of time, jurisdiction, addressee
and position in the hierarchy of norms.</p>
          <p>The following are the contextual meta-norms that
should be linked to other articles and recitals of
Directive 2004/23/EC.</p>
        </sec>
        <sec id="sec-8-8-2">
          <title>Example 17:</title>
          <p>ARTICLE 32 (time): This Directive shall enter into
force on the day of its publication in the Oficial
Journal of the European Union.</p>
          <p>ARTICLE 33 (addressees): This Directive is addressed
to the Member States.</p>
          <p>RECITAL 31 (jurisdiction): Since the objective of this
Directive, namely to set high standards of quality and
safety for human tissues and cells throughout the
Community, cannot be suficiently achieved by the
Member States and can therefore, by reason of scale
and efects, be better achieved at Community level,
the Community may adopt measures in accordance
with the principle of subsidiarity as set out in Article
5 of the Treaty. In accordance with the principle of
proportionality, as set out in that Article, this
Directive does not go beyond what is necessary in order to
achieve that objective.</p>
        </sec>
      </sec>
      <sec id="sec-8-9">
        <title>Norm Group</title>
        <sec id="sec-8-9-1">
          <title>Definition:</title>
          <p>A link between norms that are connected due to
being part of the same general requirement. The links
between the norms may be conjunction, disjunction
or sequence. Such norms may be paragraphs of the
same article, or may occur in diferent provisions.</p>
        </sec>
        <sec id="sec-8-9-2">
          <title>Example 18:</title>
          <p>ARTICLE 20.1: Tissue establishments shall include in
their standard operating procedures all processes that
afect quality and safety and shall ensure that they
are carried out under controlled conditions. Tissue
establishments shall ensure that the equipment used,
the working environment and process design,
validation and control conditions are in compliance with
the requirements referred to in Article 28(h).
ARTICLE 20.3: Tissue establishments shall include
in their standard operating procedures special
provisions for the handling of tissues and cells to be
discarded, in order to prevent the contamination of other
tissues or cells, the processing environment or
personnel.</p>
          <p>ARTICLE 21.1: Tissue establishments shall ensure that
all procedures associated with the storage of tissues
and cells are documented in the standard operating
procedures and that the storage conditions comply
with the requirements referred to in Article 28(h).
5</p>
        </sec>
      </sec>
    </sec>
    <sec id="sec-9">
      <title>DISCUSSION</title>
      <p>
        Our classification of norm types and link types was data-driven and
based on line-by-line coding with an open mind, as espoused by
grounded theory[
        <xref ref-type="bibr" rid="ref11 ref12">11, 12</xref>
        ], followed by comparison and alignment
with theories in the literature. The analysis was carried out by
researchers with a background in law, computer science and legal
informatics. While the categories below were agreed by all to be
evident in the data, the attribution of individual provisions to a
particular category was more problematic. As such, our future work
will involve annotation of a corpus of directives by a larger group of
annotators in order to properly elicit the distribution of categories,
evaluate inter-annotation agreement and, where necessary, refine
the categories. In this preliminary work, we articulate below some
of the theoretical questions that require further analysis.
      </p>
      <p>In this work, we have used the term deontic norm to describe
what we originally called detailed technical norms and to
distinguish them from objectives. In some legal traditions, e.g. Italian,
detailed norms also may include norms that are not strictly deontic.
Further work is required to explore the correct classification of
these norms.</p>
      <p>The observant reader will have noticed that granularity is treated
in diferent ways for diferent classes. Relevant sections in
Conceptually Similar links are on the same level of granularity. On the
other hand, a Motivation link features diferent levels of
granularity, although the diference in granularity may be big or small.
For Impact, Procedural and Contextual, the level of granularity is
inconsequential. Is this diference justifiable? Should there be
subcategories beyond the classes identified in this work e.g. a ’general
principle’ versus ’immediate goal’ type of Motivation? Or does the
granularity issue indicate that there is something fundamentally
problematic about the classes as they are? More work is required
to explore this issue.
6</p>
    </sec>
    <sec id="sec-10">
      <title>INSIGHTS FOR AUTOMATED</title>
    </sec>
    <sec id="sec-11">
      <title>IDENTIFICATION</title>
      <p>
        The automated identification of norm types is a task which lends
itself to well-known supervised classification techniques, such as
SVM, Bayes, decision trees, and most recently neural networks [
        <xref ref-type="bibr" rid="ref26">26</xref>
        ].
      </p>
      <p>
        The automated classification of links types, on the other hand,
requires closer scrutiny. The input is two diferent texts and the
output is the relation between them. Much literature on relevance,
particularly in information retrieval, assume that relevance
correlates with similarity. However, our analysis shows that norms are
related in diferent ways. Our intuition is that diferent techniques
are required to identify those links, and we intend to experiment
with lexical-syntactic, knowledge-based and corpus-based
techniques. Lexical-syntactic techniques are based on the words in the
input data. They include edit distance [
        <xref ref-type="bibr" rid="ref8">8</xref>
        ], pattern-based [
        <xref ref-type="bibr" rid="ref5">5</xref>
        ],
Ngrams and Longest Common Subsequence [
        <xref ref-type="bibr" rid="ref25">25</xref>
        ]. Thesaurus-based
approaches [
        <xref ref-type="bibr" rid="ref31 ref38">31, 38</xref>
        ] rely on background knowledge. Corpus-based
techniques include word embeddings, such as Word2Vec [
        <xref ref-type="bibr" rid="ref32">32</xref>
        ] or
GloVe [
        <xref ref-type="bibr" rid="ref36">36</xref>
        ], and topic modeling [
        <xref ref-type="bibr" rid="ref30 ref50">30, 50</xref>
        ]. We envisage the
following techniques for each link type: word embeddings and topic
modelling for Conceptually Similar links (since the similar texts
need not necessarily contain the same wordings), pattern-based
approaches for Constitutive links, thesaurus-based approaches for
Motivation links (in order to capture information of diferent
speciifcity), topic modeling for Impact, pattern-based for Indirect Internal
links, pattern-based for Via Other Law links, pattern-based for
Procedural links in combination with a set of trigger words, n-grams
and longest common subsequence for Contextual links, and word
embeddings for Norm Group links.
      </p>
      <p>We believe that an eventual support system should take a greedy
approach to selecting a low threshold for relevance. A high recall,
low precision system means that users have to put up with the
presentation of irrelevant data, but this is still less work than manually
looking at all possibilities. On the other hand, a high precision, low
recall system, means that users may be confident in the validity
of the classification, but then have to go through all possibilities
to find others missed by the system. While a greedy approach is,
generally speaking, the most pragmatic approach to designing user
support systems, there is a danger of information overload
rendering the support system frustrating to use. The classic solution
to this problem is to rank the results. However, the issue of how
to rank the links is not simple for this work. Sections which are
identical in meaning do not ofer new information, and therefore
may not be the most valuable to a legal researcher. Then, there are
recitals providing general principles that motivate various norms.
Are these so generic that they can be safely ignored? Only
perhaps to those who are already familiar with the directive under
consideration. Otherwise, even general principles can on occasion
help disambiguate unclear provisions if one plausible interpretation
would clearly go against the purpose of the entire directive. Perhaps
the key to creating a user-friendly system is careful interface design
that enables users to select the information they want. For instance,
they could filter out certain classes of links. Or they could choose
whether to e.g. display Motivation links in order of Most Specific or
Most General, Conceptually Similar links in order of Most Similar
or Most Dissimilar etc.
7</p>
    </sec>
    <sec id="sec-12">
      <title>RELATED WORK</title>
      <p>
        On the classification of norms, the papers of Tiscornia and Turchi [
        <xref ref-type="bibr" rid="ref46">46</xref>
        ],
and de Maat [
        <xref ref-type="bibr" rid="ref13">13</xref>
        ] are notable for developing their classification
informed by legal philosophy, while being tested on Italian and
Dutch legislation respectively. In the first paper, provisions are
classified as definitions, attributing competence, constitutive,
interpretative, instituting, prescriptive, procedural, sanctioning, material
link (derogation or extension, amending link (abrogation,
substitution), temporal link (prorogation, suspension). The second paper
ifnds core rules, procedures for citizens, procedures for civil
servants, rule management and definitions in the body of law, with
introduction, conclusion and appendices completing the model of
legislative text. Although there are similarities in the categorisation,
there are also diferences due to the nature of the legislation being
modelled. EU law is particular for having less definite, more
goalbased and principle-based norms both in the recitals and provisions.
We would point out a couple of key diferences in the classification.
Tiscornia and Turchi [
        <xref ref-type="bibr" rid="ref46">46</xref>
        ] define constitutive norms more broadly
than us to include also power-conferring norms. De Maat [
        <xref ref-type="bibr" rid="ref13">13</xref>
        ]’s
categorisation includes procedural norms also for the addressees
of norms, whereas our procedural norms describe the procedures
undertaken by institutions of the EU to implement and maintain EU
legislation. Another important factor is in the motivation for
classifying the norms. The classification of Tiscornia and Turchi [
        <xref ref-type="bibr" rid="ref46">46</xref>
        ]
and De Maat [
        <xref ref-type="bibr" rid="ref13">13</xref>
        ] serve to model the content of norms. Our
classification of norms may help in determining the relatedness between
norms.
      </p>
      <p>
        The automated classification of norms have also been explored
by Biagioli et al [
        <xref ref-type="bibr" rid="ref16">16</xref>
        ] and Waltl et al [
        <xref ref-type="bibr" rid="ref51 ref53">51, 53</xref>
        ]. The first used
Support Vector Machines, while the second found similar performance
by local linear approximations and a manually crafted rule-based
approach.
      </p>
      <p>
        Important theoretical analysis on contextualisation has been
conducted in the development of annotation standards for
modelling legislation. The temporal dimension is modelled in Legal
Knowledge Interchange Format (LKIF) [
        <xref ref-type="bibr" rid="ref35">35</xref>
        ] by ascribing to each
norm blocks of information covering time of entry into force, time
of eficacy and time of application. Akoma Ntoso [
        <xref ref-type="bibr" rid="ref6">6</xref>
        ] has a
multilayered approach to modelling laws with the text layer continuing
the original legal text, the structure layer providing a
hierarchical representation of the parts present in the text layer, and the
metadata layer associating the first two layers with ontological
information to allow advanced reasoning using logic frameworks,
and allowing for multiple interpretations of norms by diferent
actors. Akoma Ntoso also incorporates the Functional Requirements
for Bibliographic Record (FRBR) model [
        <xref ref-type="bibr" rid="ref42">42</xref>
        ] to identify diferent
versions of the same work. The importance of contextualisation is
well-explained in [
        <xref ref-type="bibr" rid="ref4">4</xref>
        ], page 152: ’[P]rovisions, rules, applications of
rules, references to text, and references to physical entities. All of
these entities exist and change in time; their histories interact in
complicated ways...[A] rule has parameters which can vary over
time, such as its status (e.g., strict, defeasible, defeater), its validity
(e.g., repealed, annulled, suspended), and its jurisdiction (e.g., only
in EU, only in US). In addition, a rule has temporal aspects such as
internal constituency of the action, the time of assertion of the rule,
the eficacy, enforcement, and so on.’ As such LegalRuleML
annotates each rule with its defeasibility status, temporality, jurisdiction
and authorial tracking.
      </p>
      <p>
        On the classification of links between norms, we refer again to
Tiscornia and Turchi [
        <xref ref-type="bibr" rid="ref46">46</xref>
        ], who also define links between norms.
Their links are by logical implication - a norm of one class, such as
prohibition, will of necessity be linked to a norm of another class,
such as sanction. Our classification of link types are generally more
semantic and probabilistic.
      </p>
      <p>
        An important work on normative similarity is that of Lau [
        <xref ref-type="bibr" rid="ref29">29</xref>
        ].
Provisions are tagged with the stems of noun phrases, legislative
definitions or glossaries from reference books. The similarity analysis
core takes as an input the parsed regulations and associated
features, and produces a list of the most similar pairs of provisions. The
most similar research to this paper in terms of its subject-matter is
that of Humphreys et al [
        <xref ref-type="bibr" rid="ref22">22</xref>
        ], which explored the feasibility of
semiautomated mapping between normative provisions and recitals.
However, the focus of that paper was deontic norms and the links
explored were only Conceptually Similar. The work of Nanda et
al [
        <xref ref-type="bibr" rid="ref34">34</xref>
        ] is concerned with finding norms from national legislation
that implement norms from EU directives. This thorough research
investigates a variety of similarity algorithms suited for short text,
including a unifying text similarity measure (USM) which
incorporates methods for matching common words, common sequences of
words and approximate string matching. While informative for our
purposes, research on automated mechanisms for finding similar
norms is only part of our agenda, since we seek to extend the notion
of relatedness, and will require diferent approaches for diferent
types of links.
      </p>
      <p>
        On the nature of relations between norms, there is a wealth of
theoretical research on conflicts between norms [
        <xref ref-type="bibr" rid="ref2">2</xref>
        ] and defeasibility
(summarised in [
        <xref ref-type="bibr" rid="ref37">37</xref>
        ]). The main features of formal approaches to
defeasibility are:
• arguments that are satisfied with certain criteria;
• counter-arguments that serve to attack or undercut other
arguments;
• the non-monotonic nature of legal reasoning, where
conclusions can be revised with the addition of new information.
Such analyses can be used to build a formal legal reasoning expert
system such as that of Sergot et al [
        <xref ref-type="bibr" rid="ref44">44</xref>
        ] and Lam &amp; Governatori [
        <xref ref-type="bibr" rid="ref27">27</xref>
        ].
Some attention has been paid to contrary-to-duty obligations i.e. a
conditional obligation arising in response to a violation of another
obligation, particularly for compliance management [
        <xref ref-type="bibr" rid="ref18">18</xref>
        ]. The use of
automated systems in legal settings has gained credibility with the
rise of AI in general. Even robotic judges are becoming acceptable
for limited domains in Estonia14. However, there are certain kinds
of laws that are simply too abstract or dependent on other laws to
be modelled in this way with certainty.
      </p>
      <p>
        There is also a wealth of literature on the discovery and
classification of citations in legislation[
        <xref ref-type="bibr" rid="ref1 ref28 ref3 ref40 ref52">1, 3, 28, 40, 52</xref>
        ]. This work is chiefly
concerned with structural references between norms of the same
and diferent legislation, and we will look closely at the techniques
used with respect to Indirect Structural and Via Other Law links.
Interestingly, in addition to full-explicit references, semi-explicit
references, and implicit references, Waltl et al [
        <xref ref-type="bibr" rid="ref52">52</xref>
        ] also refer to
tacit references which are defined as follows: “[t]he connection
between the norms emerges due to systemic interpretation and
cannot not be determined by exclusively analyzing the norm text.”
Tacit references are not covered by their system.
      </p>
      <p>
        The case for support systems to aid legal support was made
by Opijnen and Santos [
        <xref ref-type="bibr" rid="ref48">48</xref>
        ], page 84, which states that ‘given the
importance of digital information for legal professionals - lawyers
easily spend up to fifteen hours per week on search, most of it in
electronic resources although the abandonment of paper does not
always seem to be a voluntary choice - the gap between LIR systems
and user needs is still big’. This is because ’retrieval engineering is
focused too exclusively on algorithmic relevance, but it has been
proven suficiently that without domain specific adaptations every
search engine will disappoint legal users’ (ibid, page 84). While there
are several projects that seek to find linked legal data from diferent
sources, such as EUCases [
        <xref ref-type="bibr" rid="ref9">9</xref>
        ] and the LATC project (Linked Open
Data Around-The-Clock)15, in this article we chiefly looked at the
relationship between norms within the same piece of legislation.
8
      </p>
    </sec>
    <sec id="sec-13">
      <title>CONCLUSIONS AND FUTURE WORK</title>
      <p>This paper sought to model norm types and their inter-relationships
in EU directives. Recitals and provisions were categorised and the
links between them were also categorised. We started from the
presumption of the hyper-textuality and high-level nature of EU
directives, which are deliberately imprecise to allow Member States
to fulfil the objectives in their own way. It follows that deontic
norms in EU directives cannot be properly understood without
consideration of the motivation behind those norms, as well as
consideration of how other provisions, or even other laws, may afect
the scope or efectiveness of the norm in question. Our analysis
includes the impact of meta-norms that render the norms efective
and determine when the provisions are valid and in which context
(time, space, jurisdiction).</p>
      <p>The nature of EU directives means that modelling the
interrelationship between norms with defeasible logic cannot be
undertaken with certainty. This does not mean AI &amp; Law cannot be
useful. Rather, we propose a diferent approach: to find automated
means to find norms that are related in diferent ways in order to
help legal professionals interpret laws for specific cases.</p>
      <p>
        Our future work will involve the creation of a gold standard to
help build a system to semi-automatically classify norms and their
inter-relationships. Legal corpora have been created for a variety of
diferent purposes including classifying norm types and extracting
norm elements ([
        <xref ref-type="bibr" rid="ref7">7</xref>
        ] and [
        <xref ref-type="bibr" rid="ref49">49</xref>
        ]), ontology learning (e.g. [
        <xref ref-type="bibr" rid="ref45">45</xref>
        ]) and
question and answering [
        <xref ref-type="bibr" rid="ref33">33</xref>
        ]. There is no specific legal corpora
on norm types beyond deontic norms and the linking of related
provisions as far as we know.
      </p>
      <p>Our preliminary analysis suggests that diferent algorithms are
required to identify diferent types of norms and their
interrelationships. Our future work will involve experimentation to
discover which algorithms are most appropriate for each case.
Moreover, we envisage that a semi-automated classification of norm
types and links may enable the end-user in a support system to
select which kind of norms and links (s)he wishes to view, further
reducing the problem of information overload.
14://www.wired.com/story/can-ai-be-fair-judge-court-estonia-thinks-so/
15http://latc-project.eu</p>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          [1]
          <string-name>
            <given-names>Morayo</given-names>
            <surname>Adedjouma</surname>
          </string-name>
          , Mehrdad Sabetzadeh, and Lionel C Briand.
          <year>2014</year>
          .
          <article-title>Automated detection and resolution of legal cross references: Approach and a study of luxembourg's legislation</article-title>
          .
          <source>In 2014 IEEE 22nd International Requirements Engineering Conference (RE)</source>
          . IEEE,
          <fpage>63</fpage>
          -
          <lpage>72</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          [2]
          <string-name>
            <surname>Carlos</surname>
            <given-names>E</given-names>
          </string-name>
          <string-name>
            <surname>Alchourrón</surname>
            and
            <given-names>David</given-names>
          </string-name>
          <string-name>
            <surname>Makinson</surname>
          </string-name>
          .
          <year>1981</year>
          .
          <article-title>Hierarchies of regulations and their logic</article-title>
          .
          <source>In New studies in deontic logic</source>
          . Springer,
          <fpage>125</fpage>
          -
          <lpage>148</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          [3]
          <string-name>
            <surname>Kevin</surname>
            <given-names>D</given-names>
          </string-name>
          <string-name>
            <surname>Ashley</surname>
            ,
            <given-names>Matthias</given-names>
          </string-name>
          <string-name>
            <surname>Grabmair</surname>
          </string-name>
          , and Rebecca Hwa. [n. d.].
          <article-title>Network Analysis of Manually-Encoded State Laws and Prospects for Automation</article-title>
          .
          <source>In Workshop Format</source>
          . 1.
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          [4]
          <string-name>
            <given-names>Tara</given-names>
            <surname>Athan</surname>
          </string-name>
          , Guido Governatori, Monica Palmirani, Adrian Paschke, and
          <string-name>
            <given-names>Adam Z.</given-names>
            <surname>Wyner</surname>
          </string-name>
          .
          <year>2015</year>
          .
          <article-title>LegalRuleML: Design Principles and Foundations</article-title>
          .
          <source>In Reasoning Web. Web Logic Rules - 11th International Summer School</source>
          <year>2015</year>
          , Berlin, Germany,
          <source>July 31 - August 4</source>
          ,
          <year>2015</year>
          ,
          <string-name>
            <given-names>Tutorial</given-names>
            <surname>Lectures</surname>
          </string-name>
          .
          <volume>151</volume>
          -
          <fpage>188</fpage>
          . https://doi.org/10.1007/ 978-3-
          <fpage>319</fpage>
          -21768-
          <issue>0</issue>
          _
          <fpage>6</fpage>
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          [5]
          <string-name>
            <given-names>Alain</given-names>
            <surname>Auger</surname>
          </string-name>
          and
          <string-name>
            <given-names>Caroline</given-names>
            <surname>Barrière</surname>
          </string-name>
          .
          <year>2008</year>
          .
          <article-title>Pattern-based approaches to semantic relation extraction: A state-of-the-art</article-title>
          .
          <source>Terminology</source>
          <volume>14</volume>
          ,
          <issue>1</issue>
          (
          <year>2008</year>
          ),
          <fpage>1</fpage>
          -
          <lpage>19</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          [6]
          <string-name>
            <given-names>Gioele</given-names>
            <surname>Barabucci</surname>
          </string-name>
          , Luca Cervone, Monica Palmirani, Silvio Peroni, and
          <string-name>
            <given-names>Fabio</given-names>
            <surname>Vitali</surname>
          </string-name>
          .
          <year>2009</year>
          .
          <article-title>Multi-layer markup and ontological structures in Akoma Ntoso</article-title>
          . In International Workshop on AI Approaches to the
          <source>Complexity of Legal Systems</source>
          . Springer,
          <fpage>133</fpage>
          -
          <lpage>149</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          [7]
          <string-name>
            <given-names>Roberto</given-names>
            <surname>Bartolini</surname>
          </string-name>
          , Alessandro Lenci, Simonetta Montemagni, Vito Pirrelli, and
          <string-name>
            <given-names>Claudia</given-names>
            <surname>Soria</surname>
          </string-name>
          .
          <year>2004</year>
          .
          <article-title>Automatic classification and analysis of provisions in italian legal texts: a case study</article-title>
          .
          <source>In OTM Confederated International Conferences" On the Move to Meaningful Internet Systems"</source>
          . Springer,
          <fpage>593</fpage>
          -
          <lpage>604</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          [8]
          <string-name>
            <given-names>Philip</given-names>
            <surname>Bille</surname>
          </string-name>
          .
          <year>2005</year>
          .
          <article-title>A survey on tree edit distance and related problems</article-title>
          .
          <source>Theoretical computer science 337</source>
          ,
          <fpage>1</fpage>
          -
          <lpage>3</lpage>
          (
          <year>2005</year>
          ),
          <fpage>217</fpage>
          -
          <lpage>239</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          [9]
          <string-name>
            <given-names>Guido</given-names>
            <surname>Boella</surname>
          </string-name>
          , Luigi Di Caro, Michele Graziadei, Loredana Cupi, Carlo Emilio Salaroglio, Llio Humphreys, Hristo Konstantinov, Kornel Marko, Livio Robaldo,
          <string-name>
            <given-names>Claudio</given-names>
            <surname>Rufini</surname>
          </string-name>
          , et al .
          <year>2015</year>
          .
          <article-title>Linking legal open data: breaking the accessibility and language barrier in european legislation and case law</article-title>
          .
          <source>In Proceedings of the 15th International Conference on Artificial Intelligence and Law</source>
          . ACM,
          <volume>171</volume>
          -
          <fpage>175</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          [10]
          <string-name>
            <given-names>Sergio</given-names>
            <surname>Maria</surname>
          </string-name>
          <string-name>
            <surname>Carbone</surname>
          </string-name>
          , Riccardo Luzzatto, Stefania Bariatti, Paola Ivaldi, Ilaria Queirolo, Francesco Munari, Luigi Fumagalli, Bruno Nascimbene, Lorenzo Schiano Di Pepe,
          <string-name>
            <given-names>Manlio</given-names>
            <surname>Frigo</surname>
          </string-name>
          , et al.
          <year>2016</year>
          .
          <article-title>Istituzioni di diritto internazionale</article-title>
          . G Giappichelli Editore.
        </mixed-citation>
      </ref>
      <ref id="ref11">
        <mixed-citation>
          [11]
          <string-name>
            <given-names>Kathy</given-names>
            <surname>Charmaz</surname>
          </string-name>
          .
          <year>1990</year>
          .
          <article-title>âĂŸDiscoveringâĂŹchronic illness: using grounded theory</article-title>
          .
          <source>Social science &amp; medicine 30</source>
          ,
          <issue>11</issue>
          (
          <year>1990</year>
          ),
          <fpage>1161</fpage>
          -
          <lpage>1172</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref12">
        <mixed-citation>
          [12]
          <string-name>
            <surname>Juliet</surname>
            <given-names>M Corbin</given-names>
          </string-name>
          and
          <string-name>
            <given-names>Anselm</given-names>
            <surname>Strauss</surname>
          </string-name>
          .
          <year>1990</year>
          .
          <article-title>Grounded theory research: Procedures, canons, and evaluative criteria</article-title>
          .
          <source>Qualitative sociology 13</source>
          ,
          <issue>1</issue>
          (
          <year>1990</year>
          ),
          <fpage>3</fpage>
          -
          <lpage>21</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref13">
        <mixed-citation>
          [13]
          <string-name>
            <surname>Emile De Maat and Radboud Winkels</surname>
          </string-name>
          .
          <year>2007</year>
          .
          <article-title>Categorisation of norms</article-title>
          .
          <source>FRONTIERS IN ARTIFICIAL INTELLIGENCE AND APPLICATIONS 165</source>
          (
          <year>2007</year>
          ),
          <fpage>79</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref14">
        <mixed-citation>
          [14]
          <string-name>
            <given-names>P</given-names>
            <surname>Caretti-U De Siervo</surname>
          </string-name>
          .
          <year>2002</year>
          .
          <article-title>Istituzioni di diritto pubblico</article-title>
          . Giappichelli, Torino, ultima ed.
          <source>(dal Cap. X alla fine)</source>
          (
          <year>2002</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref15">
        <mixed-citation>
          [15]
          <string-name>
            <given-names>Carla</given-names>
            <surname>Faralli</surname>
          </string-name>
          .
          <year>2016</year>
          .
          <article-title>Argomenti di Teoria del</article-title>
          <string-name>
            <given-names>Diritto. G Giappichelli</given-names>
            <surname>Editore.</surname>
          </string-name>
        </mixed-citation>
      </ref>
      <ref id="ref16">
        <mixed-citation>
          [16]
          <string-name>
            <given-names>Enrico</given-names>
            <surname>Francesconi</surname>
          </string-name>
          and
          <string-name>
            <given-names>Andrea</given-names>
            <surname>Passerini</surname>
          </string-name>
          .
          <year>2007</year>
          .
          <article-title>Automatic classification of provisions in legislative texts</article-title>
          .
          <source>Artificial Intelligence and Law</source>
          <volume>15</volume>
          ,
          <issue>1</issue>
          (
          <year>2007</year>
          ),
          <fpage>1</fpage>
          -
          <lpage>17</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref17">
        <mixed-citation>
          [17]
          <string-name>
            <surname>Thomas</surname>
            <given-names>F</given-names>
          </string-name>
          <string-name>
            <surname>Gordon</surname>
          </string-name>
          .
          <year>1986</year>
          .
          <article-title>The Role of Exceptions in Models of the Law</article-title>
          .
          <article-title>Formalisierung im Recht und Ansätze juristischer Expertensysteme (</article-title>
          <year>1986</year>
          ),
          <fpage>52</fpage>
          -
          <lpage>59</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref18">
        <mixed-citation>
          [18]
          <string-name>
            <given-names>Guido</given-names>
            <surname>Governatori</surname>
          </string-name>
          and
          <string-name>
            <given-names>Antonino</given-names>
            <surname>Rotolo</surname>
          </string-name>
          .
          <year>2010</year>
          .
          <article-title>A conceptually rich model of business process compliance</article-title>
          .
          <source>In Proceedings of the Seventh Asia-Pacific Conference on Conceptual Modelling-Volume</source>
          <volume>110</volume>
          . Australian Computer Society, Inc.,
          <fpage>3</fpage>
          -
          <lpage>12</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref19">
        <mixed-citation>
          [19]
          <string-name>
            <surname>Davide</surname>
            <given-names>Grossi</given-names>
          </string-name>
          , John-Jules Meyer, and Frank Dignum.
          <year>2008</year>
          .
          <article-title>The Many Faces of Counts-as: A Formal Analysis of Constitutive-rules</article-title>
          .
          <source>Journal of Applied Logic</source>
          <volume>6</volume>
          ,
          <issue>2</issue>
          (
          <year>2008</year>
          ),
          <fpage>192</fpage>
          -
          <lpage>217</lpage>
          . http://www.davidegrossi.name/Site/publications_files/ grossi07many.pdf
        </mixed-citation>
      </ref>
      <ref id="ref20">
        <mixed-citation>
          [20]
          <string-name>
            <given-names>Herbert</given-names>
            <surname>Lionel Adolphus Hart</surname>
          </string-name>
          .
          <year>1957</year>
          .
          <article-title>Positivism and the Separation of Law and</article-title>
          <string-name>
            <given-names>Morals. Harv. L.</given-names>
            <surname>Rev</surname>
          </string-name>
          .
          <volume>71</volume>
          (
          <year>1957</year>
          ),
          <fpage>593</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref21">
        <mixed-citation>
          [21]
          <string-name>
            <given-names>Risto</given-names>
            <surname>Hilpinen</surname>
          </string-name>
          and
          <string-name>
            <given-names>Paul</given-names>
            <surname>McNamara</surname>
          </string-name>
          .
          <year>2013</year>
          .
          <article-title>Deontic logic: A historical survey and introduction</article-title>
          .
          <source>Handbook of deontic logic and normative systems 1</source>
          (
          <year>2013</year>
          ),
          <fpage>3</fpage>
          -
          <lpage>136</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref22">
        <mixed-citation>
          [22]
          <string-name>
            <surname>Llio</surname>
            <given-names>Humphreys</given-names>
          </string-name>
          , Cristiana Santos, Luigi Di Caro, Guido Boella,
          <string-name>
            <surname>Leon Van Der Torre</surname>
            , and
            <given-names>Livio</given-names>
          </string-name>
          <string-name>
            <surname>Robaldo</surname>
          </string-name>
          .
          <year>2015</year>
          .
          <article-title>Mapping Recitals to Normative Provisions in EU Legislation to Assist Legal Interpretation.</article-title>
          .
          <source>In JURIX</source>
          .
          <volume>41</volume>
          -
          <fpage>49</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref23">
        <mixed-citation>
          [23]
          <string-name>
            <surname>Sylvia</surname>
            <given-names>Mercado</given-names>
          </string-name>
          <string-name>
            <surname>Kierkegaard</surname>
          </string-name>
          .
          <year>2006</year>
          .
          <article-title>Here comes the âĂŸcybernatorsâĂŹ</article-title>
          !
          <source>Computer Law &amp; Security Review</source>
          <volume>22</volume>
          ,
          <issue>5</issue>
          (
          <year>2006</year>
          ),
          <fpage>381</fpage>
          -
          <lpage>391</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref24">
        <mixed-citation>
          [24]
          <string-name>
            <given-names>T.</given-names>
            <surname>Klimas</surname>
          </string-name>
          and
          <string-name>
            <given-names>J.</given-names>
            <surname>Vaiciukaite</surname>
          </string-name>
          .
          <year>2008</year>
          .
          <article-title>The Law of Recitals in European Community Legislation</article-title>
          .
          <source>ILSA Journal of International &amp; Comparative Law</source>
          <volume>15</volume>
          ,
          <issue>6</issue>
          (7
          <year>2008</year>
          ),
          <fpage>61</fpage>
          -
          <lpage>93</lpage>
          . Available at: https://nsuworks.nova.edu/ilsajournal/vol15/iss1/6.
        </mixed-citation>
      </ref>
      <ref id="ref25">
        <mixed-citation>
          [25]
          <string-name>
            <given-names>Grzegorz</given-names>
            <surname>Kondrak</surname>
          </string-name>
          .
          <year>2005</year>
          .
          <article-title>N-gram similarity and distance</article-title>
          .
          <source>In International symposium on string processing and information retrieval</source>
          . Springer,
          <fpage>115</fpage>
          -
          <lpage>126</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref26">
        <mixed-citation>
          [26]
          <string-name>
            <surname>Kamran</surname>
            <given-names>Kowsari</given-names>
          </string-name>
          , Kiana Jafari Meimandi, Mojtaba Heidarysafa, Sanjana Mendu, Laura Barnes, and
          <string-name>
            <given-names>Donald</given-names>
            <surname>Brown</surname>
          </string-name>
          .
          <year>2019</year>
          .
          <article-title>Text Classification Algorithms: A Survey</article-title>
          .
          <source>Information</source>
          <volume>10</volume>
          ,
          <issue>4</issue>
          (
          <year>2019</year>
          ),
          <fpage>150</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref27">
        <mixed-citation>
          [27]
          <string-name>
            <surname>Ho-Pun Lam</surname>
            and
            <given-names>Guido</given-names>
          </string-name>
          <string-name>
            <surname>Governatori</surname>
          </string-name>
          .
          <year>2009</year>
          .
          <article-title>The making of SPINdle</article-title>
          . In International Workshop on Rules and
          <article-title>Rule Markup Languages for the Semantic Web</article-title>
          . Springer,
          <fpage>315</fpage>
          -
          <lpage>322</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref28">
        <mixed-citation>
          [28]
          <string-name>
            <surname>Jörg</surname>
            <given-names>LANDTHALER</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Ingo</surname>
            <given-names>GLASER</given-names>
          </string-name>
          , and Florian MATTHES.
          <year>2018</year>
          .
          <article-title>Towards Explainable Semantic Text Matching</article-title>
          .
          <source>In Legal Knowledge and Information Systems: JURIX</source>
          <year>2018</year>
          :
          <article-title>The Thirty-first Annual Conference</article-title>
          , Vol.
          <volume>313</volume>
          . IOS Press,
          <volume>200</volume>
          .
        </mixed-citation>
      </ref>
      <ref id="ref29">
        <mixed-citation>
          [29]
          <string-name>
            <surname>Gloria</surname>
            <given-names>T Lau</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Kincho H Law</surname>
            , and
            <given-names>Gio</given-names>
          </string-name>
          <string-name>
            <surname>Wiederhold</surname>
          </string-name>
          .
          <year>2003</year>
          .
          <article-title>Similarity analysis on government regulations</article-title>
          .
          <source>In Proceedings of the ninth ACM SIGKDD international conference on Knowledge discovery and data mining. ACM</source>
          ,
          <volume>711</volume>
          -
          <fpage>716</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref30">
        <mixed-citation>
          [30]
          <string-name>
            <surname>Jon</surname>
            <given-names>D</given-names>
          </string-name>
          <string-name>
            <surname>Mcaulife</surname>
            and
            <given-names>David M</given-names>
          </string-name>
          <string-name>
            <surname>Blei</surname>
          </string-name>
          .
          <year>2008</year>
          .
          <article-title>Supervised topic models</article-title>
          .
          <source>In Advances in neural information processing systems</source>
          .
          <volume>121</volume>
          -
          <fpage>128</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref31">
        <mixed-citation>
          [31]
          <string-name>
            <surname>Rada</surname>
            <given-names>Mihalcea</given-names>
          </string-name>
          , Courtney Corley,
          <string-name>
            <given-names>Carlo</given-names>
            <surname>Strapparava</surname>
          </string-name>
          , et al.
          <year>2006</year>
          .
          <article-title>Corpus-based and knowledge-based measures of text semantic similarity</article-title>
          .
          <source>In AAAI</source>
          , Vol.
          <volume>6</volume>
          .
          <fpage>775</fpage>
          -
          <lpage>780</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref32">
        <mixed-citation>
          [32]
          <string-name>
            <surname>Tomas</surname>
            <given-names>Mikolov</given-names>
          </string-name>
          , Ilya Sutskever, Kai Chen, Greg S Corrado, and
          <string-name>
            <given-names>Jef</given-names>
            <surname>Dean</surname>
          </string-name>
          .
          <year>2013</year>
          .
          <article-title>Distributed representations of words and phrases and their compositionality</article-title>
          .
          <source>In Advances in neural information processing systems</source>
          .
          <volume>3111</volume>
          -
          <fpage>3119</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref33">
        <mixed-citation>
          [33]
          <string-name>
            <surname>Alfredo</surname>
            <given-names>Monroy</given-names>
          </string-name>
          , Hiram Calvo, and
          <string-name>
            <given-names>Alexander</given-names>
            <surname>Gelbukh</surname>
          </string-name>
          .
          <year>2009</year>
          .
          <article-title>NLP for shallow question answering of legal documents using graphs</article-title>
          .
          <source>In International Conference on Intelligent Text Processing and Computational Linguistics</source>
          . Springer,
          <fpage>498</fpage>
          -
          <lpage>508</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref34">
        <mixed-citation>
          [34]
          <string-name>
            <surname>Rohan</surname>
            <given-names>Nanda</given-names>
          </string-name>
          , Luigi Di Caro, Guido Boella, Hristo Konstantinov, Tenyo Tyankov, Daniel Traykov, Hristo Hristov, Francesco Costamagna, Llio Humphreys, Livio Robaldo, and
          <string-name>
            <given-names>Michele</given-names>
            <surname>Romano</surname>
          </string-name>
          .
          <year>2017</year>
          .
          <article-title>A unifying similarity measure for automated identification of national implementations of european union directives</article-title>
          .
          <source>In Proceedings of the 16th edition of the International Conference on Articial Intelligence and Law</source>
          ,
          <string-name>
            <surname>ICAIL</surname>
          </string-name>
          <year>2017</year>
          , London, United Kingdom, June 12-16,
          <year>2017</year>
          .
          <fpage>149</fpage>
          -
          <lpage>158</lpage>
          . https://doi.org/10.1145/3086512.3086527
        </mixed-citation>
      </ref>
      <ref id="ref35">
        <mixed-citation>
          [35]
          <string-name>
            <surname>Monica</surname>
            <given-names>Palmirani</given-names>
          </string-name>
          , Guido Governatori, and
          <string-name>
            <given-names>Giuseppe</given-names>
            <surname>Contissa</surname>
          </string-name>
          .
          <year>2011</year>
          .
          <article-title>Modelling temporal legal rules</article-title>
          .
          <source>In Proceedings of the 13th International Conference on Artificial Intelligence and Law</source>
          . ACM,
          <volume>131</volume>
          -
          <fpage>135</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref36">
        <mixed-citation>
          [36]
          <string-name>
            <surname>Jefrey</surname>
            <given-names>Pennington</given-names>
          </string-name>
          , Richard Socher, and
          <string-name>
            <given-names>Christopher</given-names>
            <surname>Manning</surname>
          </string-name>
          .
          <year>2014</year>
          .
          <article-title>Glove: Global vectors for word representation</article-title>
          .
          <source>In Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP)</source>
          .
          <volume>1532</volume>
          -
          <fpage>1543</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref37">
        <mixed-citation>
          [37]
          <string-name>
            <given-names>Henry</given-names>
            <surname>Prakken</surname>
          </string-name>
          and
          <string-name>
            <given-names>Giovanni</given-names>
            <surname>Sartor</surname>
          </string-name>
          .
          <year>2004</year>
          .
          <article-title>The three faces of defeasibility in the law</article-title>
          .
          <source>Ratio Juris</source>
          <volume>17</volume>
          ,
          <issue>1</issue>
          (
          <year>2004</year>
          ),
          <fpage>118</fpage>
          -
          <lpage>139</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref38">
        <mixed-citation>
          [38]
          <string-name>
            <given-names>Ray</given-names>
            <surname>Richardson</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A</given-names>
            <surname>Smeaton</surname>
          </string-name>
          ,
          <string-name>
            <given-names>and John</given-names>
            <surname>Murphy</surname>
          </string-name>
          .
          <year>1994</year>
          .
          <article-title>Using WordNet as a knowledge base for measuring semantic similarity between words</article-title>
          .
        </mixed-citation>
      </ref>
      <ref id="ref39">
        <mixed-citation>
          [39]
          <string-name>
            <given-names>Rodolfo</given-names>
            <surname>Sacco</surname>
          </string-name>
          .
          <year>1991</year>
          .
          <article-title>Legal formants: a dynamic approach to comparative law (Installment I of II)</article-title>
          .
          <source>The American Journal of Comparative Law</source>
          <volume>39</volume>
          ,
          <issue>1</issue>
          (
          <year>1991</year>
          ),
          <fpage>1</fpage>
          -
          <lpage>34</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref40">
        <mixed-citation>
          [40]
          <string-name>
            <surname>Ali</surname>
            <given-names>Sadeghian</given-names>
          </string-name>
          ,
          <string-name>
            <given-names>Laksshman</given-names>
            <surname>Sundaram</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D</given-names>
            <surname>Wang</surname>
          </string-name>
          ,
          <string-name>
            <surname>W Hamilton</surname>
          </string-name>
          , Karl Branting, and
          <string-name>
            <given-names>Craig</given-names>
            <surname>Pfeifer</surname>
          </string-name>
          .
          <year>2016</year>
          .
          <article-title>Semantic edge labeling over legal citation graphs</article-title>
          .
          <source>In Proceedings of the workshop on legal text, document, and corpus analytics (LTDCA2016)</source>
          .
          <fpage>70</fpage>
          -
          <lpage>75</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref41">
        <mixed-citation>
          [41]
          <string-name>
            <given-names>Mark</given-names>
            <surname>Sainsbury</surname>
          </string-name>
          .
          <year>2000</year>
          .
          <article-title>Logical forms: An introduction to philosophical logic</article-title>
          . (
          <year>2000</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref42">
        <mixed-citation>
          [42]
          <string-name>
            <given-names>KG</given-names>
            <surname>Saur</surname>
          </string-name>
          .
          <year>1998</year>
          .
          <article-title>IFLA Study Group on the functional requirements for bibliographic records. Functional requirements for bibliographic records: final report</article-title>
          .
        </mixed-citation>
      </ref>
      <ref id="ref43">
        <mixed-citation>
          [43]
          <string-name>
            <given-names>J.R.</given-names>
            <surname>Searle</surname>
          </string-name>
          .
          <year>1995</year>
          .
          <article-title>The Construction of Social Reality</article-title>
          . London: Penguin.
        </mixed-citation>
      </ref>
      <ref id="ref44">
        <mixed-citation>
          [44]
          <string-name>
            <surname>Marek</surname>
            <given-names>J.</given-names>
          </string-name>
          <string-name>
            <surname>Sergot</surname>
            , Fariba Sadri, Robert A. Kowalski, Frank Kriwaczek,
            <given-names>Peter</given-names>
          </string-name>
          <string-name>
            <surname>Hammond</surname>
            , and
            <given-names>H Terese</given-names>
          </string-name>
          <string-name>
            <surname>Cory</surname>
          </string-name>
          .
          <year>1986</year>
          .
          <article-title>The British Nationality Act as a logic program</article-title>
          .
          <source>Commun. ACM 29</source>
          ,
          <issue>5</issue>
          (
          <year>1986</year>
          ),
          <fpage>370</fpage>
          -
          <lpage>386</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref45">
        <mixed-citation>
          [45]
          <string-name>
            <given-names>Peter</given-names>
            <surname>Spyns</surname>
          </string-name>
          and
          <string-name>
            <surname>Marie-Laure Reinberger</surname>
          </string-name>
          .
          <year>2005</year>
          .
          <article-title>Lexically evaluating ontology triples generated automatically from texts</article-title>
          .
          <source>In European Semantic Web Conference</source>
          . Springer,
          <fpage>563</fpage>
          -
          <lpage>577</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref46">
        <mixed-citation>
          [46]
          <string-name>
            <given-names>Daniela</given-names>
            <surname>Tiscornia</surname>
          </string-name>
          and
          <string-name>
            <given-names>Fabrizio</given-names>
            <surname>Turchi</surname>
          </string-name>
          .
          <year>1997</year>
          .
          <article-title>Formalization of legislative documents based on a functional model</article-title>
          .
          <source>In International Conference on Artificial Intelligence and Law: Proceedings of the 6 th international conference on Artificial intelligence and law</source>
          , Vol.
          <volume>30</volume>
          .
          <string-name>
            <surname>Citeseer</surname>
          </string-name>
          ,
          <volume>63</volume>
          -
          <fpage>71</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref47">
        <mixed-citation>
          [47]
          <string-name>
            <given-names>Charles</given-names>
            <surname>Travis</surname>
          </string-name>
          .
          <year>1994</year>
          .
          <article-title>On constraints of generality</article-title>
          .
          <source>In Proceedings of the Aristotelian Society</source>
          , Vol.
          <volume>94</volume>
          . JSTOR,
          <fpage>165</fpage>
          -
          <lpage>188</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref48">
        <mixed-citation>
          [48]
          <string-name>
            <surname>Marc Van Opijnen</surname>
          </string-name>
          and
          <string-name>
            <given-names>Cristiana</given-names>
            <surname>Santos</surname>
          </string-name>
          .
          <year>2017</year>
          .
          <article-title>On the concept of relevance in legal information retrieval</article-title>
          .
          <source>Artificial Intelligence and Law</source>
          <volume>25</volume>
          ,
          <issue>1</issue>
          (
          <year>2017</year>
          ),
          <fpage>65</fpage>
          -
          <lpage>87</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref49">
        <mixed-citation>
          [49]
          <string-name>
            <given-names>Giulia</given-names>
            <surname>Venturi</surname>
          </string-name>
          .
          <year>2011</year>
          .
          <article-title>Semantic annotation of Italian legal texts: a FrameNet-based approach</article-title>
          .
          <source>Constructions and Frames</source>
          <volume>3</volume>
          ,
          <issue>1</issue>
          (
          <year>2011</year>
          ),
          <fpage>46</fpage>
          -
          <lpage>79</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref50">
        <mixed-citation>
          [50]
          <string-name>
            <surname>Hanna</surname>
            <given-names>M</given-names>
          </string-name>
          <string-name>
            <surname>Wallach</surname>
          </string-name>
          .
          <year>2006</year>
          .
          <article-title>Topic modeling: beyond bag-of-words</article-title>
          .
          <source>In Proceedings of the 23rd international conference on Machine learning. ACM</source>
          ,
          <volume>977</volume>
          -
          <fpage>984</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref51">
        <mixed-citation>
          [51]
          <string-name>
            <surname>Bernhard</surname>
            <given-names>Waltl</given-names>
          </string-name>
          , Georg Bonczek, Elena Scepankova, and
          <string-name>
            <given-names>Florian</given-names>
            <surname>Matthes</surname>
          </string-name>
          .
          <year>2019</year>
          .
          <article-title>Semantic types of legal norms in German laws: classification and analysis using local linear explanations</article-title>
          .
          <source>Artificial Intelligence and Law</source>
          <volume>27</volume>
          ,
          <issue>1</issue>
          (
          <year>2019</year>
          ),
          <fpage>43</fpage>
          -
          <lpage>71</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref52">
        <mixed-citation>
          [52]
          <string-name>
            <surname>Bernhard</surname>
            <given-names>Waltl</given-names>
          </string-name>
          , Jörg Landthaler, and
          <string-name>
            <given-names>Florian</given-names>
            <surname>Matthes</surname>
          </string-name>
          .
          <year>2016</year>
          .
          <article-title>Diferentiation and Empirical Analysis of Reference Types in Legal Documents.</article-title>
          .
          <source>In JURIX</source>
          .
          <volume>211</volume>
          -
          <fpage>214</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref53">
        <mixed-citation>
          [53]
          <string-name>
            <surname>Bernhard</surname>
            <given-names>Waltl</given-names>
          </string-name>
          , Johannes Muhr, Ingo Glaser, Georg Bonczek, Elena Scepankova, and
          <string-name>
            <given-names>Florian</given-names>
            <surname>Matthes</surname>
          </string-name>
          .
          <year>2017</year>
          .
          <article-title>Classifying Legal Norms with Active Machine Learning.</article-title>
          .
          <source>In JURIX</source>
          .
          <volume>11</volume>
          -
          <fpage>20</fpage>
          .
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>