=Paper=
{{Paper
|id=Vol-3435/short1
|storemode=property
|title=Narrative-Driven Case Elicitation
|pdfUrl=https://ceur-ws.org/Vol-3435/short1.pdf
|volume=Vol-3435
|authors=Karl Branting,Sarah McLeod
|dblpUrl=https://dblp.org/rec/conf/icail/Branting023
}}
==Narrative-Driven Case Elicitation==
https://ceur-ws.org/Vol-3435/short1.pdf
Narrative-Driven Case Elicitation
Karl Branting1,* , Sarah McLeod2
1
The MITRE Corporation, McLean, VA, USA
2
The MITRE Corporation, Seattle, WA, USA
Abstract
This paper proposes an approach for narrative-driven case elicitation that uses schemas induced from corpora of legal
case facts to distinguish relevant from irrelevant utterances and to identify facts that could distinguish between competing
hypotheses. This approach to narrative-driven case elicitation builds on recent research in narrative schema induction.
Keywords
case elicitation, narrative schema, law, computational linguistics, machine-learning, human-computer interface
1. Introduction broader architecture for induction and use of legal narra-
tives schemas. The next section provides a background
Law is full of stories, whether these are on the role of narrative understanding in providing legal
stories that are told in the courtroom as assistance, and recent research in narrative schema in-
lawyers try to weave together compelling duction is reviewed in Section 3. Section 4 presents an
and competing versions of an event, in the algorithm that uses case schemas for narrative-driven
legislative histories that subtend a body case elicitation. Section 5 describes an architecture that
of statutes, or in stories about the origins incorporates the narrative-driven case elicitation into
and acceptance of legal systems [1]. a framework that includes narrative schema induction.
Section 6 sets forth the text processing steps shared by
The facts of legal cases are more than mere collections both the schema induction and case elicitation compo-
of events. Instead, case facts are narratives that have nents. Six new corpora for narrative schema induction
settings, characters with goals and motives, and events and case elicitation are described in Section 7, and future
triggered by the characters’ actions. Just as not every set steps are proposed in Section 8.
of facts is a story, not every story is legally significant.
It is the role of a legal counsel to identify the goals that
the client hopes to achieve through a legal process and 2. The Role of Narrative in Law
then to elicit a coherent narrative that is relevant to one
or more possible legal remedies that could achieve those Law can be viewed as a framework of rules under which
goals. legal arguments often consist of alternative narratives
Lay (non-attorney) clients typically have little under- that lead to opposite consequences. Empirical evidence
standing of what facts are relevant to a possible legal has shown that jurors often decide cases based on which
remedy. Accordingly, attorneys must help clients express of two competing narratives imposes the highest degree
the facts that are relevant to possible legal remedies and of coherence on the evidence presented at trial [2] [3]. Un-
steer clients away from facts irrelevant to those remedies. surprisingly, the outcomes of trials often depend on the
The legal remedy that appears most likely to achieve relative story-telling ability of attorneys and witnesses
client’s goals may change during the interview, requir- [4] [5].
ing a reframing of the relevance of the facts that have In view of the importance of coherent legally relevant
been previously expressed and redirecting the elicitation narratives to success in litigation, narrative elicitation
toward the facts relevant to the new remedies. is widely recognized as a vital legal skill. One study of
This paper sets forth an approach to narrative-driven client interviews revealed the importance of permitting
case fact elicitation and situates that approach within a a client ample time to speak, during which the attorney
acknowledges the information received and expresses
interest but otherwise “refrains from interrupting” ex-
Workshop on Artificial Intelligence for Access to Justice (AI4AJ 2023),
cept to direct the narrative away from “precarious” legal
June 19, 2023, Braga, Portugal.
*
Corresponding author. grounds and toward “possible remedies” “until the client
" lbranting@mitre.org (K. Branting); smcleod@mitre.org has gone on for long enough to establish the problem at
(S. McLeod) hand” [6]. In general, attorneys try to elicit “the causal
0000-0002-9362-495X (K. Branting) and temporal connections that contribute to giving the
© 2023 Copyright for this paper by its authors. Use permitted under Creative Commons License
Attribution 4.0 International (CC BY 4.0). events contextual meanings . . . with the aim of defining
CEUR
Workshop
Proceedings
ceur-ws.org
ISSN 1613-0073
CEUR Workshop Proceedings (CEUR-WS.org)
‘Who has done what, how, when, why and where?’” [7]. a common actor.” The most common model for narrative
Clients’ narratives are often “redefined to be a legally rel- schemas are Markov chain models that assign a probabil-
evant narrative” by counsel, a process that can sometimes ity to an event based on cooccurring (e.g., prior) events.
diminish emotionally salient background information [8]. Such models can be used to detect distinguish expected
An “account of the situation the client faces in light of (high probability) events from unexpected (low proba-
the law” is sometimes term a case theory [9]. Case theo- bility) events and to predict the most likely events at a
ries “unite possible client narratives with possible legal given point in an event sequence.
theories” [9]. The performance of narrative schemas is typically mea-
In the context of protective order interviews, it has sured by a narrative cloze test, i.e., accuracy in predict-
been shown that the interviewer “acts to reshape, if not ing the next or a missing event [16]. Improving nar-
repair, the narratives of domestic violence victims, so rative cloze performances has been obtained by using
that they conform to the requirements of an affidavit stricter constraints on multi-argument consistency [17],
that must be submitted to a judge if a protective order is topic-specific training sets [18], and alternative language
to be issued” [10]. This is necessary because protective models, e.g., Hidden-Markov [19], Log-Bilinear [20], and
order applicants on their own often “tell their accounts Association Rule models [21].
of violence in a manner that differers quite markedly” The continuing progress in narrative schema induc-
from the “formulaic . . . structure and . . . thematic content” tion techniques suggests that this approach will be an in-
required for affidavits [10]. creasingly effective computational story model notwith-
In summary, the achievement of a client’s goals de- standing the limitation that narrative schemas are, as
pends on the degree to which the clients story can be mentioned above, only a partial representation of the ele-
presented, whether in a written petition or to a judge or ments of story. This is because the aspect of performance
jury, in a compelling, coherent, and legally relevant way. that narrative schema induction seeks to optimize, the
This depends in turn on the ability of the client’s (human cloze test, can play a central role in the model of legal
or automated) counsel to elicit case facts in a manner case elicitation, as described in the next section.
that has these characteristics.
4. Using Schemas for Case
3. Narrative Schema Induction Elicitation
A story has been described as “a character-based and
Section 2 described how effective case elicitation requires
descriptive telling of a character’s efforts, over time, to
helping a client articulate the events giving rise to a
overcome obstacles and achieve a goal” [5]. Early compu-
legal claim in a manner consistent with known legally
tational studies of narrative were motivated by efforts in
meaningful narratives, e.g., in the linear fashion required
cultural anthropology to formalize commonalities among
for protective order affidavit. Section 3 described how
folk stories [11]. Story grammars were an effort to char-
narrative schemas can be induced from training sets of
acterize narratives in a rigorous way [12]. However, the
stories and then used to distinguish expected, unexpected,
brittleness of story grammars eventually led to their al-
and missing events. This section describes an approach to
most complete abandonment [13] in favor of “scripts,”
narrative-driven case elicitation that uses the predictive
stereotypical sequences of events that create expectations
capability of narrative schemas derived from prior case
and fill in missing details to assist in story understanding
facts to guide interactions with a client.
[14].
A high-level view of a process for identifying the infor-
Unlike story grammars, scripts ordinarily lack the hi-
mation that is most relevant to a litigant’s legal goals, in
erarchical and recursive structure needed to account for
the sense of being the fact that would best discriminate
some important properties of stories (e.g., in a picaresque
among legally relevant narratives, is set forth in Algo-
novel or a protective order application there can be a
rithm 1. The key requirement for the narrative elicitation
variable number of episodes, and episodes can have sub-
process is a set of pairs, (SGs), where
episodes). Moreover, both story grammars and scripts
each schema is a model capable of evaluating the relative
initially were entirely manually constructed, which made
likelihood of a given event sequence, and each goal is a
them unscalable.
(possibly negated) legal remedy, e.g., a protective order, a
However, a series of research advances have made it
child custody order, a finding of employment discrimina-
increasingly feasible to induce “narrative schemas” (i.e.,
tion, etc. Each schema is induced from a corpus of related
“scripts” in Schank/Abelson terminology) from examples.
case facts for a given area of law using the techniques
The seminal work by Chambers and Jurafsky [15] defined
described in Section 3.1
“narrative chains” (later termed “narrative schemas” [16])
1
as “partially ordered set[s] of narrative events that share A companion paper to this work describes techniques developing
such schemas from representative legal case facts [22].
Figure 1: First two steps of an elicitation session.
In Algorithm 1, sgs is a library of SGs, the goal is the match). The hypothesis is the currently best matching
objective that the client hopes to achieve through a legal SG ∈ sgs. The function “getMissing(facts,hypothesis)”
process, the hypothesis is the best matching SG, and the returns the missing fact or event that, if added to the
facts consist of the events and relations elicited thus far facts, would most increase the probability of those facts.
from the client. Performance on this task is equivalent to performance
on the narrative cloze test [16], so as advances in narra-
Algorithm 1 Legal narrative elicitation tive schema induction improve performance on cloze, the
sgs ← {SG* } “getMissing” function should improve as well. The hy-
hypothesis ← ∅ pothesis is updated after each new fact has been elicited,
facts← ∅ reflecting the way that an attorney’s assessment of a case
goal ← ask(goal) may change as more facts are learned.
hypothesis ← BestMatch(facts,goal,sgs) The process of eliciting additional facts and refining
while match(facts,goal,hypothesis)< threshold do the hypothesis continues until there is no more progress
newFact ← ask(getMissing(facts,hypothesis) or the match between the hypothesis and the facts and
add(facts,newFact) goal exceeds a success threshold. Only at this point are
hypothesis ← BestMatch(facts,goal,sgs) the legal rules for the remedy applied to the case facts.
end while This corresponds to an attorney eliciting the full story
return facts, goal, hypothesis from a client before turning to an assessment of the via-
bility of a claim based on that story.
Figure 1 illustrates the initial steps of an elicitation
Algorithm 1 is only a high-level depiction of the actual
under process shown in Algorithm 1. The dialogue man-
complex process of case elicitation. The identification
ager starts by asking the client’s goal (Step 1). The client
of the goal or goals of a client can itself be a complex
replies by stating there was a threat and that the client
process requiring mixed-initiative dialogue techniques
wishes for help against the threat (Step 2). The dialogue
beyond the scope of this paper. However, but there is an
manager uses this goal to instantiate a new case with
extensive body of research on goal-directed dialog (e.g.,
a goal (Step 3) and the start of an event sequence. The
[23] [24]), and recent work has addressed the specific
matcher searches for the SG that best matches the goal
task of categorizing a legal aid clients’ problems [25].
and initial event sequence of the new case (i.e., the SG
The function “match(facts,goal,SG)” measures the de-
whose remedy matches the goal and whose schema max-
gree of match between the client’s goal and current facts
imizes the probability of the case’s fact sequence) and
and the SG. An appropriate baseline function for “match”
makes that SG the current hypothesis (Step 4).
is the probability of the facts under the schema, or 0.0
Figure 2 shows the predictor using the schema of the
if the goals don’t match (i.e., the schema is irrelevant if
current hypothesis to predict the most probable missing
the goals don’t match, regardless of how well the facts
Figure 2: Steps 5 and 6 of an elicitation session.
Figure 3: An early stage of a protective order case elicitation.
fact in the current case (Step 5) and the dialogue manager 5. RIM: An Architecture for
converting the fact into an appropriate discourse action
(Step 6).
Schema Induction and Use
A more detailed (although still simplified) visualization A system for narrative schema-based fact elicitation de-
of the initial stages in Algorithm 1 is shown in Figure 3. pends on two coordinated capabilities: acquiring narra-
The sequence of events starting with “meeting,” “inti- tive schemas from examples; and using those schemas to
mate relationship,” etc., represents the most probable guide interactions with litigants. Figure 4 sets forth an
event sequence under a typical protective order schema. architecture for providing these two capabilities. This ar-
The “case facts” represent a relational representation of chitecture is termed RIM, short for “Relevant, Irrelevant,
a client’s answer to the question, "How did you meet?" and Missing,” since the key functionality of the system is
Any practical implementation of Algorithm 1 must identifying these three categories of events.
ensure a common representation for the schemas and The left side of Figure 4 details an off-line mechanism
case facts; otherwise, the matching and prediction steps for inducing schemas from narrative corpora. The right
would not be possible. The next section describes an side of Figure 4 depicts the real time component, includ-
architecture to achieve this common representation. ing the process described in the previous section of using
these schemas to distinguish relevant from irrelevant
utterances and to identify facts that could distinguish
among legal schemas if confirmed or disconfirmed. The
Figure 4: The RIM architecture for narrative case elicitation.
Text Realizer generates questions to determine whether System Infrastructure) [26], a system that integrates the
missing events can be confirmed or disconfirmed. Ad- output of the Stanford Core NLP [27] constituency parser
ditional events elicited in this manner can distinguish and cTakes [28] into a temporal, causal, and intentional
among partially matching narratives or refine the match graph represented in Neo4j [29].
to the most similar narrative. Figure 5 shows a portion of the graph for the sentence
The real-time processing depicted on the right side of “During my employment, Respondent placed me on a
Figure 4 depends on the existence of a narrative schema leave of absence and required me to pass a medical exam
for each area of law for which facts are to be elicited. The in order to return to work” showing temporal and inten-
process of induction of schemata from event sequences, tional links between pairs of events.
depicted on the left side of Figure 4, is detailed in [22].
However, both the offline and real-time aspects of depend 6.2. Graph Linearization
on conversion of raw text into event sequences, as shown
as the second and third steps on both sides of Figure 4. The resulting graph representation for a collection of
one or more sentences is then linearized into an event
sequence with arguments and semantic roles standard-
6. Text to Event Sequence ized in the manner proposed in [17] to three alternatives:
Conversion agent, patient, and other complement. For example, in
the event sequence shown in Figure 6, the pronouns “I”
Offline narrative schema induction and online mixed- and “me” are normalized to “I”.
initiative case elicitation depend on a shared representa-
tion for event sequences. 6.3. Lemma Normalization
As discussed below in Section 7, corpora of legal narra-
6.1. Parsing tives are, in general, many orders of magnitude smaller
The first step in converting text to event sequences is to than the corpora used in previous narrative schema elici-
parse each sentence into individual events and, for each tation research, such as the Gigaword corpus. Such small
event, identify the entities that fill the semantic roles of corpora produce sparse transition matrices with little
that event. The next step is analyzing the relationships predictive value, e.g., most event pairs in a new (or held
among events by resolving coreferences and determining out) event sequence will have never been seen before,
the discourse relations among the events. Many alter- meaning that there is no frequency data on which to base
native approaches could be used to perform these two cloze predictions.
steps; we use ANAnSI (Advanced Narrative Analytics Several normalizations were therefore applied applied
Figure 5: A narrative fragment showing the temporal, causal, and intentional graph relationships extracted by ANAnSI.
Figure 6: A linearization of ANAnSI’s temporal, causal, and intentional graph.
to reduce vocabulary size to improve matching. The most of a form of “to be” that has as an argument the name of
important and general of these was lemma normaliza- an occupation with the event “be OCCUPATION.”3
tion, which consists of clustering events in semantic em- Lemma normalization shrinks the vocabulary size of
bedding space2 and replacing each event with the most the narrative, increasing transition matrix density and
central member of the cluster in which it occurs. For therefore reducing the likelihood that event cooccur-
example, Figure 7 show the results of complete-linkage rences will never have been observed in the training
hierarchical clustering of events in our EEOC corpus (de- corpus. This reduction in vocabulary size comes at the
scribed below) with a minimum cosine threshold of 0.75. cost of reducing the specificity of the event representa-
For example, both ‘harass’ and ‘threaten’ are replaced tion.
by ‘intimidate,’ and ‘ask,’ ‘hear,’ ‘know,’ and ‘let’ are all
replaced by ‘tell.’
A second normalization that was particularly useful in
3
the employment domains was to replace each occurrence We used the list of 1,156 occupations, from “accountant” to “zool-
ogist” set forth in https://github.com/johnlsheridan/occupations/
2
We used the spaCy large English model, https://spacy.io/models/en. blob/master/occupations.csv.
tion complaints filed in the Northern District of
Illinois in 2016. These texts are representative of
litigant-generated narrative texts.
2. Multi-LexSum Summaries of Civil Cases. These
three hundred sixty four summaries of civil rights
lawsuits were created for training and evaluating
legal case summarization [30]. The Multi-LexSum
text were included to typify procedural histories,
a type of narrative required for appeals that court
personnel have identified as being challenging
for pro se appellants.
3. WIPO cases. The “background facts” of 6,000 deci-
sions by World Intellectual Property Organization
in domain name disputes. These fact statements
were drafted by the panel deciding the case and
are therefore not representative of pro se text.
However, the similarity among these fact state-
ments suggests that they could be a benchmark
for narrative induction.
4. Board of Veteran Affairs decisions. The “Introduc-
tion” section of 1,680 Board of Veterans Appeals
(BVA) cases. As with the WIPO cases, these texts
are drafted by the judge writing the opinion and
are therefore not representative of pro se text but
Figure 7: Lemma normalization by clustering event types in potentially useful as a benchmark for narrative
semantic embedding space. induction.
5. SPOT-HO online housing questions. Two hun-
dred sixty three questions posed to the Suffolk
7. Corpora University Law School’s Legal Innovation and
Technology (LIT) Lab issue spotting service [31].
A key challenge for narrative schema-based case elicita- 6. SPOT-WO online employment questions. Two
tions is the difficulty of obtaining significant numbers hundred ninety five employment questions posed
of narrative texts representative of text produced by liti- to the SPOT site.
gants. In general, such text contains sensitive personal
information that precludes sharing in the form of pub- The size, type, and authors of each of the corpora are
lic corpora. Documents filed in legal or administrative summarized in Table 1.
bodies are typically public, so statements of facts in pe-
titions, complaints, and other filings can be a source of 8. Summary and Discussion
narrative texts. However, counsel for litigants often draft
the statements in facts of court filings, so the text of such
This paper has proposed an approach to narrative-driven
statements seldom contains language used by litigants case elicitation that builds on recent research in narrative
themselves except in the case of self-represented (pro se) schema induction. This approach uses schemas induced
litigants, i.e., those who have no attorney to draft their from corpora of legal case facts to distinguish relevant
statements in fact. The ideal corpus would consist of from irrelevant client utterances and to identify facts
statements of fact in pro se litigants’ filings, but such that could distinguish among competing hypotheses if
filings are difficult to obtain in bulk. confirmed or disconfirmed.
In this research, we obtained one small corpus of texts Only the offline portion of the RIM model has been
by pro se litigants together with five other data sets implemented in this project, as described in [22]. A work-
intended to reflect various characteristics of fact state- ing prototype of the narrative elicitation portion of RIM,
ments: which is the focus of this paper, would require assem-
bling a library of scheme-goal pairs for a given area of
1. EEOC complaints. The complaints were tran-
law from a suitable corpus, such as the BVA or WIPO
scribed from handwritten texts in the field titled
corpora described above. The matching and prediction
“The facts supporting the plaintiff’s claim of dis-
functions described in Algorithm 1 are basic capabili-
crimination,” in thirty employment discrimina-
ties of the schemas described in [22], and as mentioned
Corpus Size Text Type Author Type
EEOC 30 complaints pro se litigant
SPOT-WO 295 legal advice requests lay public
SPOT-HO 263 legal advice requests lay public
Multi-Lexum 364 procedural history federal judge
BVA 1,680 background facts administrative law judge
WIPO 6,000 background facts administrative law judge
Table 1
The size, type, and authors of each corpus of narrative texts.
above there is an extensive literature on the goal-directed (1992) 189–206. https://doi.org/10.1037/0022-3514.
mixed-initiative dialogue techniques needed for the dia- 62.2.18.
logue manager. Thus, there are no significant technical [4] C. D. Phillips, Reconstructing reality in the
obstacles to implementing a prototype. courtroom: Justice and judgment in american cul-
The narrative-driven case elicitation approach de- ture, American Political Science Review 77 (1983)
scribed here is quite unlike the dominant techniques 275–276. doi:10.2307/1956108.
for automated legal assistance, which are overwhelm- [5] K. D. Chestek, Competing stories: A case study of
ing organized around form-filling [32] or backchaining the role of narrative reasoning in judicial decisions,
through logical representations of legal rules [33] (see Legal Comm. & Rhetoric: JAWLD 9 (2012) 99.
generally [34]). The narrative-driven approach is in- [6] S. Kashimura, Hearing clients’ talk as lawyers’
tended to change the focus of client interviewing from work: the case of public legal consultation confer-
the structure of the target legal artifacts (completed pe- ence, in: M. L. Baudouin Dupret, T. Berard (Eds.),
titions and legal rules) to the life experiences that give Law at Work: Studies in Legal Ethnomethods, Ox-
rise to legal claims and to human interactions between ford University Press, Oxford, 2015, p. 87–113.
client and attorney. While the narrative-driven model [7] F. D. Donato, Fact-finding in contexts:
proposed here is a drastic simplification of the actual in- framing clients’ agentivity within ju-
terview process between clients and attorneys, we hope dicial and administrative procedures,
that it will be a first step toward more faithful model and file:///Users/lbranting/Downloads/DIDONATO_
is feasible to implement with current narrative schema position_paper_Rotterdam_22.9.15.pdf, last
induction and dialogue technology. accessed February 5, 2023, 2015.
[8] M. Livingston, A Sad Story in Not a Legal Defense:
Defining Legal Issues, Master’s thesis, University
Acknowledgments of Colorado at Boulder Dept. of Communication,
2014.
The authors express their gratitude to Charlotte Alexan-
[9] A. Shalleck, How clients and lawyers construct
der for permitting us to experiment with her collection
facts. the stories they tell each other and the stories
of EEOC complaints and to Charles Horowitz and Karine
that guide investigations into the world, in: Dig-
Megerdoomian for their assistance with ANAnSI. The
nifying and Undignified Narratives in and of (the)
MITRE Corporation is a not-for-profit company, char-
Law, Proceedings of the IVR World Congress 2019
tered in the public interest. This document is approved
WS Law and Narrative, 2020, pp. 119–144.
for Public Release; Distribution Unlimited. Case Num-
[10] S. L. Trinch, S. Berk-Seligson, Narrating in pro-
ber 23-1184. ©2023 The MITRE Corporation. All rights
tective order interviews: A source of interactional
reserved.
trouble, Language in Society 31 (2002) 383–418.
[11] A. J. Greimas, C. Porter, Elements of a narrative
References grammar, Diacritics 7 (1977) 23–40.
[12] D. E. Rumelhart, Notes on a schema for stories, in:
[1] A. Tait, L. Norris, Narrative and the origins of law, D. G. BOBROW, A. COLLINS (Eds.), Representation
Law and Humanities 5 (2011) 11–22. and Understanding, Morgan Kaufmann, San Diego,
[2] N. Pennington, R. Hastie, A cognitive theory of 1975, pp. 211–236.
juror decision making: The story model, Cardozo [13] S. ANDERSEN, B. SLATOR, Requiem for a theory:
L. Rev. 13 (1991) 519. the ‘ story grammar’ story, Journal of Experimental
[3] N. Pennington, R.Hastie, Explaining the evidence: & Theoretical Artificial Intelligence 2 (1990) 253–
Tests of the story model for juror decision making, 275.
Journal of Personality and Social Psychology 62
[14] R. C. Schank, R. Abelson, Scripts, Plans, Goals, S. Bethard, D. McClosky, The stanford corenlp nat-
and Understanding, Hillsdale, NJ: Earlbaum Assoc, ural language processing toolkit., in: ACL (System
1977. Demonstrations), The Association for Computer
[15] N. Chambers, D. Jurafsky, Unsupervised learning Linguistics, 2014, pp. 55–60.
of narrative event chains, in: Proceedings of ACL- [28] G. K. Savova, J. J. Masanz, P. V. Ogren, J. Zheng,
2007, Association for Computational Linguistics, S. Sohn, K. C. Kipper-Schuler, C. G. Chute, Mayo
Columbus, Ohio, 2008, pp. 789–797. clinical text analysis and knowledge extraction sys-
[16] N. Chambers, D. Jurafsky, Unsupervised learning tem (ctakes), JAMIA 17 (2010) 507–513.
of narrative schemas and their participants, in: Pro- [29] Neo4j, Neo4j -graph data platform, 2023. http://
ceedings ACL-2009, Association for Computational neo4j.com/, last accessed 7 March 2023.
Linguistics, USA, 2009, p. 602–610. [30] Z. Shen, K. Lo, L. J. Yu, N. Dahlberg, M. Schlanger,
[17] K. Pichotta, R. Mooney, Statistical script learning D. Downey, Multi-lexsum: Real-world summaries
with multi-argument events, in: Proceedings of the of civil rights lawsuits at multiple granularities,
14th Conference of the European Chapter of the ArXiv abs/2206.10883 (2022).
Association for Computational Linguistics, 2014, pp. [31] SPOT, The legal innovation & technology lab’s
220–229. spot api, https://spot.suffolklitlab.org, last accessed
[18] E. Rahimtoroghi, E. Hernandez, M. A. Walker, March 9, 2023, 2023.
Learning fine-grained knowledge about contingent [32] J. Frank, A2j author, legal aid organizations, and
relations between everyday events, arXiv preprint courts: Bridging the civil justice gap using docu-
arXiv:1708.09450 (2017). ment assembly, W. New Eng. L. Rev. 39 (2017) 251.
[19] J. W. Orr, P. Tadepalli, J. R. Doppa, X. Z. Fern, T. G. [33] M. J. Sergot, F. Sadri, R. A. Kowalski, F. Kriwaczek,
Dietterich, Learning scripts as hidden markov mod- P. Hammond, H. T. Cory, The British National-
els, CoRR abs/1809.03680 (2018). ity Act as a logic program, Commun. ACM 29
[20] R. Rudinger, P. Rastogi, F. Ferraro, B. Van Durme, (1986) 370–386. URL: http://doi.acm.org/10.1145/
Script induction as language modeling, in: Proceed- 5689.5920. doi:10.1145/5689.5920.
ings of the 2015 Conference on Empirical Methods [34] M. Lauritsen, Q. Steenhuis, Substantive legal soft-
in Natural Language Processing, 2015, pp. 1681– ware quality: A gathering storm?, in: Proceedings
1686. of the Seventeenth International Conference on Ar-
[21] A. Belyy, B. Van Durme, Script induction as asso- tificial Intelligence and Law, ICAIL ’19, ACM, New
ciation rule mining, in: Proceedings of the First York, NY, USA, 2019, pp. 52–62.
Joint Workshop on Narrative Understanding, Sto-
rylines, and Events, Association for Computational
Linguistics, Online, 2020, pp. 55–62.
[22] L. K. Branting, S. McLeod, K. Megerdoomian,
B. Park, Induction of narrative models for legal case
elicitation, in: Proceedings of the Sixth Workshop
on Automated Semantic Analysis in Law (ASAIL),
2023.
[23] B. Peng, M. Galley, P. He, C. Brockett, L. Liden,
E. Nouri, Z. Yu, B. Dolan, J. Gao, Godel: Large-
scale pre-training for goal-directed dialog, 2022.
arXiv:2206.11309.
[24] J. Lester, K. Branting, B. Mott, Conversational
agents, in: M. Singh (Ed.), Practical Handbook of
Internet Computing, CRC Press, 2004.
[25] M. Mistica, J. H. Lau, B. Merrifield, K. Fazio, T. Bald-
win, Semi-automatic triage of requests for free
legal assistance, in: Proceedings of the Natural Le-
gal Language Processing Workshop 2021, 2021, pp.
217–227.
[26] K. Megerdoomian, K. Branting, C. Horowitz,
A. Marsh, S. Petersen, E. Scott, Automated nar-
rative extraction from administrative records., in:
AIAS@ ICAIL, 2019, pp. 38–48.
[27] C. D. Manning, M. Surdeanu, J. Bauer, J. R. Finkel,