<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta>
      <journal-title-group>
        <journal-title>June</journal-title>
      </journal-title-group>
    </journal-meta>
    <article-meta>
      <title-group>
        <article-title>Utilising AI in the legal assistance sector</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Andrew Mowbray</string-name>
          <email>andrew@austlii.edu.au</email>
          <xref ref-type="aff" rid="aff1">1</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Philip Chung</string-name>
          <email>philip@austlii.edu.au</email>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Graham Greenleaf</string-name>
          <email>graham@austlii.edu.au</email>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          <institution>AustLII, University of New South Wales</institution>
          ,
          <addr-line>Sydney</addr-line>
          ,
          <country country="AU">Australia</country>
        </aff>
        <aff id="aff1">
          <label>1</label>
          <institution>AustLII, University of Technology Sydney</institution>
          ,
          <addr-line>Sydney</addr-line>
          ,
          <country country="AU">Australia</country>
        </aff>
      </contrib-group>
      <pub-date>
        <year>2019</year>
      </pub-date>
      <volume>17</volume>
      <issue>2019</issue>
      <abstract>
        <p>The use of artificial intelligence (AI) in law has again become of great interest to lawyers and government. Legal Information Institutes (LIIs) have played a significant role in the provision of legal information via the web. The concept of 'free access to law' is not static, and its principles now require a LII response to the renewed prominence of AI, possibly to include improving and expanding free access to legal advice. This paper proposes, and proposes to test, one approach that LIIs might take in the use of AI (specifically, 'decision support' or 'intelligent assistance' (IA) technologies), an approach that leverages the very large legal information assets that some LIIs have built over the past two decades. This approach focuses on how LIIs can assist providers of free legal advice (the 'legal assistance sector') to serve their clients. We consider the constraints that the requirement of 'free' imposes (on both the legal assistance sector and on LIIs), including on what types of free legal advice systems are sustainable, and what roles LIIs may realistically play in the development of such a 'commons of free legal advice'. We suggest guidelines for development of such systems. The AI-related services and tools that the Australasian Legal Information Institute (AustLII) is providing (the 'DataLex' platform) are outlined.</p>
      </abstract>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>CCS CONCEPTS</title>
      <p>• Information systems → Expert systems; Wikis; • Applied
computing → Law;</p>
    </sec>
    <sec id="sec-2">
      <title>INTRODUCTION</title>
      <p>
        The use of artificial i ntelligence ( AI) i n l aw, i ncluding i n r elation to
decision-support systems, has again become a matter of great
interest to both the legal profession and to government. The
previous wave of enthusiasm for, and investment in, ‘AI and law’
from the early 1980s to the mid-1990s was to a large extent
supplanted by the development of the World-Wide-Web and the
provision of legal information via the web. Legal Information
Institutes (LIIs) and the Free Access to Law Movement (FALM),1
played a very significant role in those developments [
        <xref ref-type="bibr" rid="ref1">1</xref>
        ]. What
roles might LIIs play in this new AI-oriented environment?
      </p>
      <p>
        The concept of ‘free access to law’ is not static, and has evolved
over the past quarter-century [
        <xref ref-type="bibr" rid="ref2">2</xref>
        ]. The principles of free access to
law now require a LII response to the renewed prominence of
AIrelated developments in law, which could include improving and
expanding free access to legal advice, as part of ‘free access to law’,
consistent with those of FALM’s Declaration of Free Access to Law
[
        <xref ref-type="bibr" rid="ref3">3</xref>
        ]. ‘Freeing the law’ is a continuous process.
      </p>
      <p>This paper proposes, and proposes to test, one approach that
LIIs might take in the use of AI (specifically, ‘decision support’ or
‘intelligent assistance’ (IA) technologies), an approach that
leverages the very large legal information assets that some LIIs
have built over the past two decades.2 This approach focuses on
how LIIs can assist providers of free legal advice (the ‘legal
assistance sector’) to serve their clients. We consider the
constraints that the requirement of ‘free’ imposes (on both the
legal assistance sector and on LIIs), including on what types of free
legal advice systems are sustainable, and what roles LIIs may
realistically play in the development of such a ‘commons of free
legal advice’. We suggest guidelines for development of such
systems.</p>
      <p>The Australasian Legal Information Institute (AustLII) is
providing AI-based services and tools (the ‘DataLex’ platform),
which are described, including how they implement these
guidelines. A decision support system on rental housing law is to
be developed to implement, test and evaluate the above approach,
with the DataLex platform being used by pro bono knowledge-base
(KB) developers from a large law firm to develop the application,
working in conjunction with a community legal centre that will
utilise it, and a university research centre to evaluate the project.
2</p>
    </sec>
    <sec id="sec-3">
      <title>DEGREES OF FREEDOM: TRAJECTORIES OF</title>
    </sec>
    <sec id="sec-4">
      <title>THE DIGITIZATION OF EXPERTISE</title>
      <p>The ‘Web 2.0’ context since about 2004 creates a very diferent
environment from the pre-1995 (pre-Internet, in popular usage)
context of the first wave of ‘AI and law’ This context makes it
more feasible to talk about the collaborative development of free
legal advice services based on AI. The reasons include the
significant roles that collaboration, in the form of FOSS (free and
open source software) and open content (exemplified by Creative
1FALM website &lt; http://www.falm.info/&gt; – FALM has over 60 members.
2Other LIIs which have built AI-based tools to complement their databases include
CanLII/Lexum (now merged).
Commons licensing and Wikipedia) have had on the development
of the Internet; the much greater sophistication of interfaces; and
the possibilities of interaction between AI-based tools and huge
amounts of free access legal content.</p>
      <p>
        We can distinguish three types of digitisation relevant to the
giving of professional legal advice: representation of information
used by experts; representation of expertise and its general
application; and application of expertise to individual situations.
These categories overlap in reality, but these distinctions enable us
to consider more precisely [
        <xref ref-type="bibr" rid="ref4">4</xref>
        ] how likely is it that each category
will become part of a ‘commons of legal expertise’.
      </p>
      <p>(I) Representing Expert Domain Information. ‘Raw’ (primary)
information used by experts is the most likely aspect of expertise
both to be digitised and to become part of the commons. Databases
of primary information essential to legal professionals (legislation,
treaties, court decisions etc) are already substantially digitised and
available online, and with increasing utility (eg smarter retrieval
systems, and smarter data structures). In many countries
substantial amounts are available as commons, at least for free
access and often as open content, usually via government sources.
In a few dozen countries such as Australia, free access ‘legal
information institutes’ (‘LIIs’) aggregate this data and add value to
it, making it a resource used by professionals and the general
public alike. Even though some primary information is only
available commercially, in less than 25 years since the start of
widespread availability of such data via the web, the increase in
free availability is extraordinary, and is tending toward a
comprehensive commons.</p>
      <p>(II) Representing Expertise in General Form. When professional
expertise is represented (or embodied or reified) this is usually in a
generalised form which may or may not be applicable to an
individual situation where expertise is needed, because of the
enormous variation of individual situations which may arise. It is
up to the reader (usually the correct term) to apply the expertise to
the individual situation. Legal professionals represented their
expertise in many ways prior to the Internet — in textbooks,
journal articles, encyclopedias, and in very significant, but more
mundane, forms such as citators and checklists (often as
supervisors of non-professionals). The economics of publishing
meant that such reification of expertise could rarely be provided as
a commons, and instead it usually became an economic asset of a
commercial publisher and an author.</p>
      <p>
        The Internet changes some but not all of these factors. Expertise
remains a very valuable asset which many professionals are
reluctant to embody in any form of commons. However, the last
quarter century has revealed revolutionary potential which is only
becoming apparent through the accretion of successes, including
free access repositories of current scholarship, archives of
published journals, changing academic funding requirements,
peer-reviewed free content, viral licensing; crowd-sourcing;
collaborative editing by closed professional groups; and automated
substitutions for expertise. A ‘closed wiki’ model, where content
may only be edited by professionals may be most suitable for law,
because of its emphasis on authority. Successful commons
examples developed by AustLII including multi-author guidebooks
[
        <xref ref-type="bibr" rid="ref5">5</xref>
        ], and automated citators performing to professional levels [
        <xref ref-type="bibr" rid="ref6">6</xref>
        ].
The results demonstrate that it is becoming viable for professionals
to control the representation of their own expertise, as a commons.
      </p>
      <p>
        (III) Applying Expertise to Individual Situations. It is the third
category, the application of expertise to individual situations (the
problems of individual clients) via programs, which is seen widely
as a major threat to the future of professionals and professions [
        <xref ref-type="bibr" rid="ref4">4</xref>
        ].
At present, the number of convincing examples and their
commercial viability do not make it inevitable that there will be
generalised dire results for professions. To understand the likely
implications, it is necessary to distinguish at least three types of
the programmatic applications of legal expertise: human expertise
embodied in knowledge-bases which interact with programs;
embedded knowledge in artifacts; and machine-generated
expertise. The first is most relevant to the provision of legal advice.
The question is whether, in those areas where legal expertise can
be efectively captured in knowledge-bases to be used in
decision-support systems, can they be developed as a commons, or
only as commercial products?
3
      </p>
    </sec>
    <sec id="sec-5">
      <title>AN ALTERNATIVE FUTURE: A COMMONS</title>
    </sec>
    <sec id="sec-6">
      <title>OF LEGAL EXPERTISE</title>
      <p>Although there is as yet no obvious tendency toward commons in
relation to the three categories of software-based application of
expertise to individual cases, we argue that this can be encouraged
to develop. Tools for knowledge engineering and for creating
machine-generated expertise are available as FOSS and are of high
quality, but the communities of users necessary to develop
applications (similar to the FOSS or Wikipedia communities) have
not yet developed. We argue that such collaborative alternative
could arise primarily from those organisations that seek to provide
free legal advice, and be driven largely by their needs, but could
expand to involve other participants in the legal profession.
3.1</p>
    </sec>
    <sec id="sec-7">
      <title>The providers and constraints of free legal advice</title>
      <p>
        There are many situations where, at least in a country like
Australia, our social expectation is that legal advice be provided
without cost to the public, whether as consumers, citizens or
(sometimes) litigants. The organisations most likely to be involved
in providing such free3 legal advice are quite diverse, and include
government legal aid providers, community legal centres,
government and community consumer advice centres, specialist
NGOs in law-related areas, government agencies giving advice
relevant to their functions, and ‘chamber magistrates’ in
courthouses. The legal profession, through state and regional Law
Societies and advice centres they provide, and through the
extensive pro bono schemes, also contributes. University law
schools, through their involvement in community legal centres
and internships in other organisations, are potential sources of
contributors who often have high computing skills. Bodies
assisting the legal profession as a whole to avoid liability problems,
such as some legal insurers, might also wish to participate.
3‘Free’ entails ‘free from surveillance’, a test which some commercial providers of
ostensibly ‘free’ services will fail: see [
        <xref ref-type="bibr" rid="ref2">2</xref>
        ].
A common factor for most of these providers of free legal advice
is that, if they choose to develop AI-related tools to assist their work,
they will usually have to do so within very constrained development
and maintenance budgets for software or applications. They are
not in a position to pass on such costs to clients, or to purchasers
of applications. Government or other grants for such developments
may provide up-front development costs (at least while the hype
cycle for AI is rising) but will rarely cover ongoing maintenance for
applications as the law changes, or technical issues arise. Bringing
in out-of-house consultants on specialised software problems, or
as ‘knowledge engineers’ in relation to particular legal domains, is
likely to be very expensive. It is therefore a reasonable assumption
that, at least in the medium to long term, providers of free legal
advice will have to work within significant financial constraints that
are more severe than those experienced by commercial providers.
      </p>
      <p>The implications of these constraints – limited institutional range
of providers, and limited financial resources – afect the types of
legal advisory systems that it is practical for this sector to develop
and support.
3.2</p>
    </sec>
    <sec id="sec-8">
      <title>Free legal advisory systems: Guidelines for sustainability</title>
      <p>
        We have previously set out and justified our views on what approach
to the use of AI tools is most likely to be of value to a free legal
advice service ([
        <xref ref-type="bibr" rid="ref7">7</xref>
        ], at 3.1-3.16). These guidelines are based on the
assumptions discussed above of the likely limited financial and
personnel resources of such a service, and on our own lengthy
experience with the DataLex project. They are implemented in the
DataLex platform discussed in the following section.
      </p>
      <p>First, the ‘AI and law’ systems that such a service could be
expected to find useful are those that justify their answers at least
in part in terms of the formal sources of law. These constraints will
mean that only some types of ‘AI and law’ tools are suitable to
their needs.</p>
      <p>Second, looked at from the user perspective, which could be
that of an employee of a free legal advice service, or perhaps one
of its clients, what counts as a useful level of legal expertise is
relative. A system may be valuable to a class of users even though
it has a relatively low point at which it admits that a problem is
beyond its expertise, and it may serve as a method of triage. In any
event, it is not realistic to try to build legal expert systems that
encapsulate all the knowledge necessary to answer user problems.
The more realistic aim is to build decision support systems, in
the use of which the program and the user in efect pool their
knowledge/expertise to resolve a problem. Expertise can and should
be represented and utilised by programs in many ways. This means
the knowledge-based system (the knowledge representation and
the program) should not be ‘closed’: it must be integrated with
text retrieval, hypertext and other tools which allow and assist the
user to obtain access to whatever source materials are necessary to
answer the parts of a problem dependent on the user’s expertise.
The result is an integrated decision-support system.</p>
      <p>Third, looked at from the developer perspective, the key
contextual factor is that user-organisations such as free legal
advice services, will probably need to both develop and maintain
their own knowledge-bases, as the only available domain experts.
The systems which non-technical legal domain experts are most
likely to be able to develop and maintain are those which represent
legal knowledge in a way which has a reasonably high level of
isomorphism (one-to-one correspondence) with the legal sources
on which it is based, where the representation is reasonably close
to natural language, and where it is not necessary to prescribe the
order(s) of the procedural steps necessary to reach a solution to a
problem, but only to declare what legal knowledge is available,
and leave it to the system to undertake the steps to apply that
knowledge.</p>
      <p>Fourth, correctly choosing the type of problem where ‘AI and law’
techniques are most likely to be appropriate is essential. Problem
areas based on legislation, or procedural steps, and where there is
complexity, will probably give the best results. Problems involving
multiple instances of one factor increase logical dificulty. If it is
administratively possible to have multiple organisations collaborate
to build and maintain a legal knowledge base, this may increase
sustainability.
3.3</p>
    </sec>
    <sec id="sec-9">
      <title>The likely roles of LIIs</title>
      <p>Fifth, we conclude that free access legal information institutes (LIIs)
are unlikely to be the builders of legal knowledge-bases in particular
legal domains, because they do not have the necessary in-house
expertise in legal subject domains. They have neither the client-base
that provides a continuing need for such expertise, nor the funds
to retain such expertise from outside (at least not on a continuing
basis, beyond an initial grant). As a result, LIIs are much more likely
to be the providers of tools by which such knowledge-bases are
built, the free access legal infrastructure within which they are built,
and education and support for those organisations that use their
tools and services to build and maintain subject-area applications.4
In light of that conclusion, we now move to the tools and services
that AustLII is building.
4</p>
    </sec>
    <sec id="sec-10">
      <title>AI IN A LII: AUSTLII’S DATALEX</title>
    </sec>
    <sec id="sec-11">
      <title>IMPLEMENTATION</title>
      <p>
        The Australasian Legal Information Institute (AustLII), through its
DataLex project ([
        <xref ref-type="bibr" rid="ref7">7</xref>
        ] at 2, [
        <xref ref-type="bibr" rid="ref8">8</xref>
        ]) is developing tools and infrastructure
so as to implement the above ‘sustainable legal advisory systems’
approach to AI and law in the context of a LII. This platform includes
ifve main elements, rectangles in the following diagram (Figure 1).
The features of each are then summarised.
4.1
      </p>
    </sec>
    <sec id="sec-12">
      <title>The DataLex inferencing software</title>
      <p>The DataLex inferencing software5 primarily carries out rule-based
reasoning. It has the following key features:
• Support for backward-chaining and forward-chaining
rulebased reasoning. Rules are expressed in a declarative form.
• Rule-based reasoning is supplemented by procedural code,
where procedural steps in reasoning are needed.
4Length constraints preclude a survey of what projects other LIIs are undertaking.
5The DataLex inferencing software was originally written by Andrew Mowbray, as
y-sh (‘y-shell’), with subsequent further layers by various authors including Simon
Cant and Philip Chung, to enable web-based operation.
• Rule based reasoning is also supplemented by example-based
(or ‘case-based’) reasoning,6 where needed.
• Rules of any degree of complexity may be written, using
propositional logic.
• A quasi-natural-language knowledge-base syntax (ie one
resembling English as far as is possible) is used to declare
rules (and examples).
• There is no separate coding of questions, explanations and
reports, because they are all generated automatically from
the declared rules, in dialogues generated ‘on the fly’ when
the system is in operation. This default operation can be
customised where special circumstances require.
• Isomorphic (one-to-one) relationships between the
knowledge-base and legislation is facilitated, and assists in
debugging and updating.
• The previous three elements allow easier development,
debugging and maintenance by domain experts (lawyers),
without involvement by software experts or ‘knowledge
engineers’.
• Collaborative development of larger applications across
distributed knowledge-bases is supported.</p>
      <p>
        An extract from the ElectKB knowledge-base [
        <xref ref-type="bibr" rid="ref10">10</xref>
        ] is shown in
Figure 2.
6PANNDA (Precedent Analysis by Nearest-Neighbour Discriminant Analysis); see [
        <xref ref-type="bibr" rid="ref9">9</xref>
        ]
for details about the FINDER (finders’ cases) application of PANNDA.
      </p>
    </sec>
    <sec id="sec-13">
      <title>The AustLII Communities environment – integrating AI with a LII</title>
      <p>The AustLII Communities environment is used to link
automatically both knowledge-bases under development, and
advisory systems when in operation, with all of the free access
legal materials provided by a LII. The hypertext links in the above
knowledge-base extract are inserted automatically, using AustLII’s
ifndacts software, into the knowledge-base as it is written and
saved. Further examples of links from applications in operation are
given below.
4.3</p>
    </sec>
    <sec id="sec-14">
      <title>The DataLex knowledge-base development tools</title>
      <p>
        The DataLex development tools [
        <xref ref-type="bibr" rid="ref11">11</xref>
        ] are situated within the AustLII
Communities infrastructure. They use a familiar wiki-like editing
interface for development and maintenance of knowledge-bases
(KBs). Development is within a closed wiki environment.
4.4
      </p>
    </sec>
    <sec id="sec-15">
      <title>The DataLex user interface</title>
      <p>The DataLex user interface uses the DataLex software and
knowledge-bases, the linkages provided by the Communities
environment, and user input, to provide legal advisory systems in
operation.</p>
      <p>From Figure 3, it can be seen that some of the features of the
interface include:
• Questions, Facts, Conclusions, and Reports are all generated
from the knowledge-base and user-provided facts, in
understandable form, and are available on screen at all
times.
• Facts can be deleted (‘Forget?’), and questions then
re-asked; Conclusions can be explained (‘How?’); and
reasons for Questions requested (‘Why?’), generated in the
same manner.
• The system also uses all information available to it, from the
knowledge-base and user-supplied facts, to suggest other
relevant Related Materials.</p>
      <p>As the consultation continues, conclusions are shown on the
right-hand side. Selection of a numbered conclusion results in a
‘How’ explanation of that conclusion being presented, as shown in
Figure 4.</p>
      <p>At the end of the consultation, a composite explanation of the
ifnal result, and of all the steps necessary for it to be reached,
is displayed and may be exported to word processing or other
programs for use.
4.5</p>
    </sec>
    <sec id="sec-16">
      <title>The LawCite citator and SINO search engine – updating and expanding advice</title>
      <p>
        SINO is the open source search engine, developed by AustLII [
        <xref ref-type="bibr" rid="ref12">12</xref>
        ],
used to operate AustLII and other LIIs. The LawCite citator [
        <xref ref-type="bibr" rid="ref6">6</xref>
        ] is an
automated international citator for case law and legal scholarship,
accessible to end-users free of any user charges. It is developed
and maintained by AustLII in conjunction with a consortium of
participating legal information institutes (LIIs). LawCite currently
contains index records of the citation histories of over 5.7 million
cases, law journal articles, law reform documents and treaties, going
back to the 1300s. It includes citation records in significant numbers
from court decisions in 75 countries. It is integrated fully into the
operations of AustLII and other LIIs that use it. The technical details
of LawCite are explained elsewhere [
        <xref ref-type="bibr" rid="ref13">13</xref>
        ].
      </p>
      <p>
        The significance of both LawCite and SINO within the DataLex
project is that they provide a means of (in efect) expanding the
scope of a knowledge-base by providing users with access to
knowledge which is not yet encoded within the knowledge-base.
Examples are as follows, from the ElectKB knowledge-base [
        <xref ref-type="bibr" rid="ref10">10</xref>
        ]
concerning disqualification for eligibility for election to the
Australian federal Parliament:
(a) Wherever the term ‘foreign power’ appears in a consultation
dialogue, it does so as a hypertext link which triggers a
search over AustLII for all occurrences of ‘foreign power’ in
the context of s 44 of the Australian Constitution (Figure 5).
      </p>
      <p>The user is able to note from the LawCite citation record
whether that case has been considered by other cases
subsequent to the knowledge-base being written, and to
check for any resulting changes to the law. No
knowledge-base can be updated as frequently as the law
might change,7 and this is particularly so when they are
subject to the constraints discussed in part 3. For example,
the LawCite record for this case alerts the user to recent
cases considering Sykes v Cleary, that may not yet be taken
account of in the knowledge-base (Figure 8).</p>
      <p>It should be clear from these examples that updating a legal
knowledge-base through links and searches requires access to the
case and legislation content of a whole legal system, updated
continuously. For providers of free legal advice, the most feasible
source of such information is a free access legal information
institute (LII).
5</p>
    </sec>
    <sec id="sec-17">
      <title>A COLLABORATIVE PROJECT FOR LEGAL</title>
    </sec>
    <sec id="sec-18">
      <title>ASSISTANCE LAWYERS AND THEIR</title>
    </sec>
    <sec id="sec-19">
      <title>CLIENTS</title>
      <p>
        The next steps of this project will be (once funding is secured) to
test the approach advocated in this paper will utilise AustLII’s
DataLex platform to demonstrate how pro bono legal assistance
programs can result in development and use of shared AI-based
legal resources (‘apps’), integrated fully with AustLII’s databases.
The application to be developed concerns NSW tenancy law
(particularly the Residential Tenancies Act 2010), an area of
7A knowledge-base maintained by a legislature is a partial exception (would not
include case-law changes). Automated programmatic updating is a formidable task,
and unlikely.
significant unmet legal need, as documented in the Redfern Legal
Centre (RLC) response ([
        <xref ref-type="bibr" rid="ref14">14</xref>
        ]) to [
        <xref ref-type="bibr" rid="ref15">15</xref>
        ]. There are four project
partners:
(1) AustLII is to provide (and further develop) the DataLex
platform, including the inferencing software, Communities
environment, and underlying legal databases on AustLII,
plus software developer skills and knowledge, and project
management.
(2) The Pro Bono &amp; Community Impact program of King and
Wood Mallesons (KWM)8 is to provide lawyers under its pro
bono program who will work with RLC to build a
knowledgebase (‘the application’) in the tenancy law area, using the
DataLex platform.
(3) Redfern Legal Centre (RLC)9 is to provide legal staf and
volunteers working in its tenancy advice practice, to utilise
the application in its advisory work. It will provide feedback
about both the application and the platform, so that they
can be improved iteratively, to KWM lawyers and to
AustLII developers. Once the application is tested by RLC, it
and AustLII will decide whether a version of it can also be
provided for direct use by the public, via AustLII and
through links from RLC’s website.
(4) The Australian Pro Bono Centre (APBC)10 at UNSW is to
provide an independent evaluation of the development of
the application, and the outcome of the trial of its use. Based
on this evaluation it will provide advice to AustLII to assist
AustLII to develop a methodology by which the DataLex
platform can be more widely applied in the pro bono field to
support free access legal advice to the community.
      </p>
      <p>The co-ordinating body for NSW Community Legal Centres
has also agreed to examine how it can both assist legal centres
sharing applications developed using this approach, and assist in
the identification of which bodies in the legal assistance sector
8https://www.kwm.com/en/au/about-us/corporate-responsibility/pro-bono
9https://rlc.org.au/
10https://www.probonocentre.org.au/
would be most likely to participate in the development of new
applications.
6</p>
    </sec>
    <sec id="sec-20">
      <title>CONCLUSIONS – WHEN IS AI FEASIBLE</title>
    </sec>
    <sec id="sec-21">
      <title>FOR THE LEGAL ASSISTANCE SECTOR?</title>
      <p>In this paper we have identified why providers of free legal advice
are likely to face significant constraints on the resources available
to them to develop and maintain AI-based legal advisory systems,
and the implications this has for the types of systems they are
most likely to use. We have proposed guidelines which will enable
development which is sustainable by the organisations likely to be
providing such advice, and which will contribute to an expanding
commons of legal expertise embodied in AI-based tools.</p>
      <p>We have set out the approach that AustLII, through its DataLex
platform, is taking to facilitate the development of such systems, and
how the DataLex approach allows implementation of the guidelines
for sustainable legal AI that we have proposed. We have outlined
a collaborative project to build, test and evaluate both the general
approach, and an application on rental housing law.</p>
    </sec>
    <sec id="sec-22">
      <title>REFERENCES</title>
      <p>All URLs were accessed 18 April 2019.</p>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          [1]
          <string-name>
            <given-names>Graham</given-names>
            <surname>Greenleaf</surname>
          </string-name>
          , Philip Chung, and
          <string-name>
            <given-names>Andrew</given-names>
            <surname>Mowbray</surname>
          </string-name>
          . Update:
          <article-title>Legal information institutes and the free access to law movement</article-title>
          .
          <source>Globalex</source>
          ,
          <year>February 2018</year>
          . http://www.nyulawglobal.org/globalex/Legal_Information_Institutes1. html.
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          [2]
          <string-name>
            <given-names>Graham</given-names>
            <surname>Greenleaf</surname>
          </string-name>
          ,
          <string-name>
            <given-names>Andrew</given-names>
            <surname>Mowbray</surname>
          </string-name>
          , and
          <string-name>
            <given-names>Philip</given-names>
            <surname>Chung</surname>
          </string-name>
          .
          <article-title>The meaning of 'free access to legal information': A twenty year evolution</article-title>
          .
          <source>Journal</source>
          of Open Access to Law
          <source>(JOAL)</source>
          ,
          <volume>1</volume>
          :
          <fpage>1</fpage>
          ,
          <year>2013</year>
          . https://papers.ssrn.com/sol3/papers.cfm?abstract_id=
          <fpage>2158868</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          <source>[3] FALM. Declaration on Free Access to Law</source>
          ,
          <fpage>2002</fpage>
          -. http://www.falm.info/ declaration/.
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          [4]
          <string-name>
            <given-names>Graham</given-names>
            <surname>Greenleaf</surname>
          </string-name>
          .
          <article-title>Review essay - technology and the professions: Utopian and dystopian futures</article-title>
          .
          <source>UNSW Law Journal</source>
          ,
          <volume>40</volume>
          (
          <issue>1</issue>
          ):
          <fpage>302</fpage>
          -
          <lpage>321</lpage>
          ,
          <year>2017</year>
          . https://ssrn.com/ abstract=2973244 or [2017]
          <article-title>UNSWLawJl 12</article-title>
          .
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          [5]
          <string-name>
            <given-names>Melinda</given-names>
            <surname>Schroeder</surname>
          </string-name>
          (ed) and
          <source>70 authors. Northern Territory Law Handbook, 26 July</source>
          <year>2016</year>
          . http://austlii.community/wiki/NTLawHbk/NTLawHandbook.
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          [6]
          <string-name>
            <given-names>Andrew</given-names>
            <surname>Mowbray. LawCite Citator</surname>
          </string-name>
          ,
          <fpage>2008</fpage>
          -. http://www.austlii.edu.au/lawcite/.
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          [7]
          <string-name>
            <given-names>Graham</given-names>
            <surname>Greenleaf</surname>
          </string-name>
          ,
          <string-name>
            <given-names>Andrew</given-names>
            <surname>Mowbray</surname>
          </string-name>
          , and
          <string-name>
            <given-names>Philip</given-names>
            <surname>Chung</surname>
          </string-name>
          .
          <article-title>Building sustainable free legal advisory systems: Experiences from the history of AI &amp; law</article-title>
          .
          <source>Computer Law &amp; Security Review</source>
          ,
          <volume>34</volume>
          :
          <fpage>314</fpage>
          -
          <lpage>326</lpage>
          ,
          <year>2018</year>
          . pre-publication version at [2017
          <source>] UNSWLRS 53.</source>
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          [8]
          <string-name>
            <given-names>Graham</given-names>
            <surname>Greenleaf</surname>
          </string-name>
          ,
          <string-name>
            <given-names>Andrew</given-names>
            <surname>Mowbray</surname>
          </string-name>
          , and
          <string-name>
            <given-names>Philip</given-names>
            <surname>Chung</surname>
          </string-name>
          .
          <source>The Datalex Project: History and Bibliography</source>
          ,
          <year>2018</year>
          . UNSW Law Research Paper No 18
          <article-title>-4</article-title>
          . https: //ssrn.com/abstract=3095897 or [2018
          <source>] UNSWLRS 4.</source>
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          [9]
          <string-name>
            <given-names>Alan</given-names>
            <surname>Tyree</surname>
          </string-name>
          , Graham Greenleaf, and
          <string-name>
            <given-names>Andrew</given-names>
            <surname>Mowbray</surname>
          </string-name>
          .
          <article-title>Generating legal arguments</article-title>
          .
          <source>Knowledge-Based Systems</source>
          ,
          <volume>2</volume>
          (
          <issue>1</issue>
          ):
          <fpage>46</fpage>
          -
          <lpage>51</lpage>
          ,
          <year>1989</year>
          . https://papers.ssrn. com/sol3/papers.cfm?abstract_id=
          <fpage>2988931</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          [10]
          <string-name>
            <given-names>Andrew</given-names>
            <surname>Mowbray. ElectKB</surname>
          </string-name>
          knowledge-base,
          <year>February 2019</year>
          . http://austlii. community/wiki/DataLex/ElectKB.
        </mixed-citation>
      </ref>
      <ref id="ref11">
        <mixed-citation>
          [11]
          <article-title>AustLII</article-title>
          . DataLex development tools,
          <fpage>2019</fpage>
          -. http://austlii.community/wiki/ DataLex.
        </mixed-citation>
      </ref>
      <ref id="ref12">
        <mixed-citation>
          [12]
          <string-name>
            <given-names>Andrew</given-names>
            <surname>Mowbray</surname>
          </string-name>
          .
          <article-title>SINO free text search engine, 1996-</article-title>
          . AustLII Open source software (Unix/C) http://www.austlii.edu.au/techlib/software/sino/.
        </mixed-citation>
      </ref>
      <ref id="ref13">
        <mixed-citation>
          [13]
          <string-name>
            <given-names>Andrew</given-names>
            <surname>Mowbray</surname>
          </string-name>
          , Philip Chung, and
          <string-name>
            <given-names>Graham</given-names>
            <surname>Greenleaf</surname>
          </string-name>
          .
          <article-title>A free access, automated law citator with international scope: The LawCite project</article-title>
          .
          <source>European Journal of Law and Technology (EJLT)</source>
          ,
          <volume>7</volume>
          (
          <issue>3</issue>
          ),
          <year>2016</year>
          . pre-print version at https://ssrn.com/ abstract=2768104 or [2016
          <source>] UNSWLRS 32.</source>
        </mixed-citation>
      </ref>
      <ref id="ref14">
        <mixed-citation>
          [14]
          <string-name>
            <surname>Redfern</surname>
            <given-names>Legal</given-names>
          </string-name>
          <string-name>
            <surname>Centre</surname>
          </string-name>
          .
          <source>Submission: Statutory Review of the Residential Tenancies Act</source>
          <year>2010</year>
          ,
          <issue>12</issue>
          <year>February 2016</year>
          . https://rlc.org.au/sites/default/files/attachments/ RLC%20submission
          <source>%20to%20RTA%20Review.pdf.</source>
        </mixed-citation>
      </ref>
      <ref id="ref15">
        <mixed-citation>
          [15]
          <article-title>Department of Fair Trading (NSW)</article-title>
          .
          <source>Discussion Paper: Statutory Review of the Residential Tenancies Act</source>
          <year>2010</year>
          ,
          <year>October 2015</year>
          . https: //www.fairtrading.nsw.gov.au/__data/assets/pdf_file/0010/398287/Residential_ tenancies_discussion_paper.pdf.
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>