We present an end-to-end system for extracting deontic logic formulae from legal text using a generic semantic parsing module and task-specific graph grammars, and for performing automated reasoning on the extracted formulae. The pipeline enables automated compliance checking and is applied to text documents of the zoning map of the city of Vienna. All components are released as open-source software, the full pipeline is showcased in an online demo.
entirely rule-based, making our system an example of true explainable AI (XAI). Unlike in deep learning-based We present an end-to-end system for extracting deontic information extraction systems, extracted rules can be logic formulae from legal text using a generic semantic directly traced back to text patterns, making it straightparsing module and task-specific graph grammars, and forward to provide natural language explanations for for performing automated reasoning on the extracted decisions made based on them. This explainable nature formulae. An overview of the pipeline is shown in Fig. 1. also enables human-in-the-loop operation and provides Plain text regulations are processed by a pipeline of safeguards against biased decision-making. Our main domain-agnostic language processing tools, including a contributions are: system for building syntax-independent concept graphs that represent the meaning of each sentence. These • Specification of a formal representation of deontic graphs serve as the input for a task-specific rule extrac- statements including those of the construction tion module that maps them to deontic logic formulae, regulation domain for automated rule extraction which in turn are used in an automated reasoning system. and reasoning The proposed pipeline is applied to text documents of the zoning map of the city of Vienna1, an exciting corpus of • A preprocessed, structured corpus of sentences legal regulations whose highly structured nature renders extracted from the zoning plan of the City of Viit very well suited for formal approaches. In absence of a enna, a small subset of which is annotated manularge-scale annotated corpus we evaluate our approach ally with formal rule representations on a toy dataset of manually analyzed sentences that • A grammar-based system for explainable rule exwere selected to cover the most frequent attributes in traction from semantic graphs, evaluated on the the full dataset. Possible applications include automated annotated corpus compliance checking and question answering. The semantic parser and rule extractor components are both • System architecture and working prototype for an end-to-end system for rule extraction and automated reasoning from raw text
The paper is structured as follows. In Sec. 2 we review recent work on semantic parsing, automatic rule extraction, and automated reasoning for legal-tech applications, and present the dependencies of our pipeline: a task-independent semantic parser and a deontic logic prover. Sec. 3 presents the rule extraction method, Sec. 4 describes the architecture of the full pipeline. Sec. 5 PDF documents
Text segmentation
Sentences
Semantic parsing
Concept graphs
Rule extraction
Deontic rules
Reasoning provides preliminary evaluation of our rule extraction representations in NLP pipelines. Abstract Meaning Repmethod, Sec. 6 discusses next steps and draws some con- resentations (AMRs) [1] represent sentence meaning as clusions. All components of our system are available directed graphs of words, but do not provide a model as open-source software2. The example application is of word meaning and are highly English-specific, not showcased in an online demo3. intended as a language-independent framework of meaning representation. The task of parsing raw text to AMR graphs has recently attracted growing interest and is usu2. Related work ally performed using deep neural networks [2, 3] trained on annotated corpora, also called sembanks. Universal The related work for the two main components of the sys- Conceptual Cognitive Annotation (UCCA) [4] takes a tem is presented: (i) semantic parsing for extracting logic language-agnostic approach to meaning representation, expressions directly from legal text, and (ii) deontic logic modeling sentence meaning with directed acyclic graphs for performing automated reasoning on the extracted (DAGs) representing scenes evoked by predicates. Top formulae. UCCA parsers also rely on manually annotated corpora and neural networks [5, 6, 7, 8]. Since these frameworks 2.1. Semantic parsing do not provide a generic parsing algorithm, building representations for a new language and/or new domain would require the manual compilation of large annotated datasets that could be used to train end-to-end machine learning models. For the pipeline presented in this paper we choose the language-independent 4lang framework [9], for which a robust parsing method [10] with an opensource implementation4 [11] is also available.
The 4lang framework represents the meaning of both words and larger units like phrases and sentences as directed graphs of concepts. The representation is syntaxindependent, concepts do not have types such as part-ofspeech or even as predicate and argument. A key feature of 4lang graphs that enables uniform treatment of syntactically diferent constructions is the 0-relation (→− 0 ), a single representation for a range of closely related semanSemantic parsing is the task of automatically mapping natural language text to a formal representation of its meaning. Most contemporary architectures for solving information extraction tasks do not perform semantic parsing, instead relying on models which directly encode the correspondence between natural language text and some set of task-specific structures such as labels, sequences, attribute-value structures, etc. These models are primarily built using machine learning methods, whose performance is dependent on the quality and quantity of available training data and whose decisions are dificult to interpret and prone to bias. Rule-based models, on the other hand, require considerable expert efort to build and maintain and can still be dificult to adapt to changes in the task definition. The architecture we propose tackles the information extraction task in two tic relationships such as the ISA relationship (e.g. roo→f− 0 steps: the mapping of natural language text to task- and covering), attribution (e.g. sidewal→k− 0 paved), and preddomain-independent meaning representations (semantic ication (e.g. platform→− 0 extend). 4lang graphs can be parsing) followed by a task-specific information extrac- built from raw text automatically using a rule-based systion step that operates on these representations. In this tem that uses Universal Dependencies (UD) [12] as an section we give a brief overview of common approaches intermediate step. UD trees encode grammatical relations to semantic parsing and of the representation framework (dependencies) between pairs of words in a sentence — an used by our pipeline. example of such an analysis is shown in Fig. 2, described
Unlike other forms of automated linguistic annotation later. UD parsers are available for dozens of languages, such as part-of-speech tagging or syntactic parsing, se- in this pipeline we use the stanza package5 [13]. The mantic parsing is not in any way standardized in the transformation of UD trees into 4lang graphs is based natural language processing (NLP) community. Even on a small set of rules described in [10] and implemented those few frameworks that have recently attracted grow- by [11] as parsing and decoding of an Interpreted Reguing interest in the task are rarely used as intermediate lar Tree Grammar (IRTG) [14], a formalism that we also use in this work for implementing the rule extraction
3https://ir-group.ec.tuwien.ac.at/brise-extract
mechanism that maps 4lang semantic graphs to trees of
attributes (see Sec. 3.2). Most rules map a single UD edge
between two content words to 4lang edges connecting
the corresponding concepts (e.g., in the example in Fig. 2,
the relations amod and nsubj : pass are mapped to a
0-edge and a 2-edge, respectively).
minimal height of 10 metres. Due to this property the
reasoning process is non-monotone: While from the single
assumption obl(buildingHeightMin(
The specific reasoning engine used in this paper, the
theorem prover BRISEprover6, is an extension and modi- 3. Rule extraction
ifcation of the theorem prover deonticProver2.07
developed in [21] for reasoning with assumptions in dyadic de- The pipeline presented here takes as its input raw text
ontic logic. In this logical framework, propositional logic documents containing regulations of the zoning map of
is extended with dyadic deontic operators obl, for and per. the City of Vienna, builds representations of their
meanFormulae obl(, ), for(, ) and per(, ) are read ing using the 4lang system (see Sec. 2.1), uses the
reas “it is obligatory that given ”, “it is forbidden that sulting semantic graphs to extract the legal content of
given ” and “it is permitted that given ”, respec- regulations and makes them available to the prover (see
tively. As the first extension considered here we extend Sec. 2.2), which verifies whether some statement is
derivthe language with predicate symbols to capture proper- able given a set of assumptions. The full architecture is
ties, e.g., “building height at most 9 metres”, in atomic for- described in Sec. 4, we now present the novel rule
extracmulae, e.g., buildingHeightMax(9). Note that it would tion component and its interfaces to semantic parsing
be straightforward to include an additional argument rep- and automated reasoning.
resenting the subject, i.e., which building has a height of
at most 9 metres. Since in our case this is clear from the 3.1. Representation
context we simplify the representation by assuming the Since we do not want to commit to modeling
regusubject is always the same. The prover decides derivabil- lations in a particular formalism, and to facilitate the
ity from a set of factual and deontic assumptions, i.e., non- integration of diferent reasoning engines we first
condeontic and non-nested deontic formulae respectively. vert the legal content of the regulations into a generic
The reasoning engine supports the specificity principle representation. For this we assume that a deontic
regulain the form that more specific deontic assumptions over- tion, i.e., a regulation stating an obligation, prohibition
ride less specific conflicting ones. E.g., the assumption or permission, is comprised of the following parts:
obl(buildingHeightMax(9), facingStreet), stating
that buildings facing the street must have a maximal • Modality: This states whether the regulation is
height of 9 metres, overrides the less specific assump- an obligation, a prohibition or a permission;
tion obl(buildingHeightMin(
7See http://subsell.logic.at/bprover/deonticProver/version2.0/ • Conditions: The conditions of the regulation stat
ing when the regulation applies; Dachneigung case case det nmod case nummod nsubj:pass case det aux:pass nmod det
obl amod mark
NOUN ADP ADP DET NOUN ADP NUM NOUN AUX ADP DET NOUN DET ADJ NOUN Flachdächer bis zu einer Dachneigung von fünf Grad sind entsprechend dem Stand der technischen Wissenschaften PART VERB zu begrünen {"modality": "obligation", "attributes": [ {"type": "content", "name": "BegruenungDach", "value": null}, {"type": "condition", "name": "Dachart", "value": "Flachdach"}, {"type": "condition", "name": "DachneigungMax", "value": "5Grad"}}
details of the sentences found in the documents from the Here we modelled both flat roofs and glass roofs as roof city of Vienna zoning map while being flexible enough to types, while modelling the property of being a green pdeeromntiitctrloangsicla,tdioenfeainstiobldeidfereeonnttifcralmogeiwc o[r1k6s],liakregudmyaednicta- roof as an atomic propositional statement because the tion based approaches [27] or input output logic [28]. latter in the documents corresponds to a more complex Concretely, we represent this structure as a JSON object proposition.
Note that in contrast to, e.g., the approach in [17] at {”modality” : Modality, ”attributes” : List} this stage we do not commit to a particular modelling of exceptions by negation-as-failure or negated conditions. where the key “modality” takes one of the values “obli- This retains the flexibility of the general format necessary agnataiorrna”y,“cpornothaiibniitniogna”t,t“rpibeurmtesisosifotnh”e, afonldlowwhinegrefoLrims:t is for subsequent specification into a large number of different formalisms. There are of course some limitations {”name” : Name, ”value” : Value, ”type” : Type}. inherent in this representation, in particular we do not model negation. See Sec. 6 for a more detailed discussion.
and Type is one of “Content”, “Condition”, “ConditionException”, “ContentException”. We obtained the attribute names via collaboration with domain experts from the City of Vienna Baupolizei and verified that the structure is appropriate by manually annotating several hundred sentences from the documents of the zoning map.
modality that occur within a single sentence. A simple heuristic then matches these elements with each other in order to create the generic rule representations described in the previous section. The mapping we establish is between patterns in generic semantic graphs and formal rules. This two-step approach is an arbitrary simplification that makes implementation simpler and more flexi- ⇕ ble. We shall now describe the approach using some examples and point out some of its current limitations. The ifrst component of our rule extraction system is an IRTG OBL grammar mapping 4lang graphs to lists of strings representing attribute names (e.g. DachneigungMax ‘max- BegruenungDach imal roof pitch’), modalities (e.g. OBL ‘obligation’), as well as numbers and units of measurement that may be interpreted as attribute values (e.g. 5 and m). We shall Dachart v_Flachdach DachneigungMax refer to this grammar as fl_to_attr. The heuristics for matching values and modalities to attribute names, q_Grad v_5 bdeetswcreibenedmlautletirpilne stohliustsioecntsioifn,itciasninofnolrymdeidsaambboiugtutahtee fFlig_utroe_at3t:r graEmxammaprlefor mthaepspeinntgencimeipnleFmigeunrteed2. by the positions of patterns relative to each other. Hence we use a Tree Grammar as the output interpretation, which allows us to represent these strings as leaves of a tree that resembles the order in which they were recognized, corresponding to steps of composing the 4lang graph from subgraphs. An example of this mapping is presented in Fig. 3. Each IRTG rule encoding the correspondence Figure 4: Example rule of the fl_to_attr IRTG grammar. between a 4lang subgraph and an attribute or tree of The first interpretation line is wrapped for readability, the opattributes is a mapping between rule applications in an erations are explained in the text. s-graph algebra [29] and a tree algebra. S-graphs are graphs whose nodes may be marked by special labels called sources and the s-graph algebra’s core operation tation by heuristically matching modalities as well as pomerge creates new s-graphs by taking the union of its tential attribute values to attribute names. We illustrate arguments but merging nodes that have the same source. this process using an example with multiple attributes; We now illustrate this mechanism with a simple exam- consider the following sentence fragment, a subordinate ple, for a more detailed introduction to s-graph algebras clause of a longer regulation: bei einer Straßenbreite ab 10 and their application to semantic parsing the reader is m entlang der Fluchtlinien Gehsteige mit einer Breite von referred to [29]. The IRTG rule presented in Fig. 4 de- mindestens 2,0 m herzustellen sind. ‘in case of a street ifnes two binary operations, to be performed in parallel width of 10 m or more, sidewalks with a width of at least on corresponding pairs of s-graphs and trees. The op- 2.0 m are to be constructed along the alignment lines.’. eration of the 4lang interpretation merges two graphs The pipeline described so far will extract from this senalong their root nodes with the two nodes of the edge tence three attribute names (StrassenbreiteMin ‘minGebäudehöh→e− 0 maximal to create a single graph, while imum street width’, GehsteigbreiteMin ‘minimum the operation of the attr interpretation merges the two sidewalk width’, AnFluchtlinie ‘along the alignment attribute trees with each other and subsequently with a line’), two numbers (10, 2.0), two occurrences of the unit tree of two nodes (OBL, GebaeudeHoeheMax). S-graph of measurement m, and the modality OBL ‘obligation’ (the algebras use three types of operations: the merge opera- latter based on the word form herzustellen composed of tion merges two graphs on nodes with matching sources the verb herstellen ‘construct, produce’ and the infinitive while the rename and forget operations can be used to marker zu). These eight elements are organized in a tree change or delete sources. In the example rule in Fig. 4 structure according to the order in which they appeared the two argument graphs are renamed so that their root in the IRTG derivation of the corresponding semantic sources become src and tgt, and these sources need to graph, shown in Fig. 5. The tree of attributes is stored be deleted after the merge operation. in a custom data structure that in every node stores the
Once all relevant strings have been extracted from the length of the shortest path between any pair of attributes semantic graph, the next step is to build the rule represen- below that node. This means that by querying the root of the tree we can retrieve for any attribute a list of all other E -> a_gebaeudehoehe_maximal(E, E) [100] [fl] f_src(f_tgt(merge(r_src(?1), merge(r_tgt(?2), "(u<src> / Gebaeudehoehe :0 (v<tgt> / maximal))")))) [attr] *(*(?1, ?2), *("OBL", "GebaeudeHoeheMax"))
of correctly matching attributes to their values and for AnFluchtLinie distinguishing between the roles each attribute plays in a rule, allow us to keep the grammar simpler than it OBL would be if it was to directly generate structures like our GehsteigbreiteMin generic rule representations. The IRTG rules currently used (and exemplified in Fig. 4) simply represent oneto-one correspondences between 4lang subgraphs and strings, but the underlying Regular Tree Grammar only StrassenbreiteMin m 2.0 uses a single nonterminal symbol, i.e. rules are not sen
sitive to which other rules were used to construct their m 10 arguments. It would be quite straightforward to introduce non-terminal symbols representing attribute names, numbers, measurement units, and modalities, so that the IRTG itself would enforce the structure that is currently built in a postprocessing step. The tree in Figure 5 re.F.i.gbueireei5n:erESxatrmaßpelenborfeiatettraibbu1t0e mmatecnhtliannggfodrerthFeluscehntltienniecne sembles the order in which patterns corresponding to Gehsteige mit einer Breite von mindestens 2,0 m herzustellen each element were found in the semantic graph, which sind. ‘in case of a street width of 10 m or more, sidewalks in turn correspond to disjoint subgraphs of the semantic with a width of at least 2.0 m are to be constructed along the representation that are each connected to the concept alignment lines.’ herstellen and roughly correspond to fragments of the original sentence such as Gehsteig mit einer Breite von mindenstens 2,0 m herstellen ‘construct sidewalk with a width of at least 2.0 m’, bei einer Straßenbreite ab 10 m attributes ranked by their relative distance in the tree. We ‘with a road width of at least 10 m’, entlang der Fluchtlinifrst match all units of measurement to the nearest value ien ‘along the alignment lines’, etc. Here we limit our in the tree, allowing each value to be associated with at grammar to the task of understanding each of these patmost one unit of measurement. Next, all non-boolean at- terns independently because this proves suficient for our tributes are matched to the nearest value, using a greedy purposes of constructing formal rules from sentences of algorithm: all possible attribute-value pairs are sorted the zoning map of the City of Vienna, which tend to be by their relative distance in the tree, the pair with the in a one-to-one correspondence with rules of the general shortest path is stored as a match, its members are re- structure described in Sec. 3.1. I.e., we greatly simplify moved from the lists of attributes and values that are still our task by exploiting the fact that authors of this piece of to be matched, and this step is repeated until at least one legislation rarely express a rule in multiple sentences or of the lists becomes empty. For example, in Fig. 5 the incorporate several rules in a single sentence. A notable attribute StrassenbreiteMin ‘minimum road width’ is exception is when some conditions such as the ID or despaired with the value 10, since they are the closest of ignation of the areas that a rule refers to are not repeated any pairs of attribute and value in the tree (the attribute in every sentence within the same section. These condiAnFluchtLinie is excluded from this process because it tions are propagated by a simple inheritence mechanism is listed as a boolean attribute). In the second step the at- that assumes the values of such attributes to hold within tribute GehsteigbreiteMin ‘minimum sidewalk width’ a single section (see Sec. 4 for how section boundaries is matched with the only remaining value, 2.0. Finally, are detected). the type of each attribute must be detected, i.e. it must be determined whether an attribute is a condition of the rule, part of the content, or an exception to either one of 3.3. Specification to dyadic deontic logic these (contentException, conditionException, see The extracted rules in the generic format are then transSec. 3.1). Some attrbutes are explicitly listed as always be- lated into the language of dyadic deontic logic. We chose ing of type condition, e.g., Planzeichen and Widmung this particular framework because the sentences of the which refer to the ID and designation of an area. Next, the zoning map of Vienna exhibit a very clear deontic strucextracted modality elements OBL, FOR, EXC are matched ture, in contrast, e.g., to the largely definitional character to the nearest of the remaining attributes, which are of the British Nationality Act investigated in [15]. The in turn determined to be part of the content (in case translation is dependent on the modality in the following of FOR and OBL) or a conditionException (in case of way. Given a rule representation EXC). Finally, all remaining attributes are given the type {”modality” : Modality, ”attributes” : List} condition. let
Cnd := cnd1(cndV1) ∧ · · · ∧ cnd(cndV) The input to our pipeline consists of PDF documents CntEx := ctEx1(ctExV1) ∨ · · · ∨ ctEx(ctExV) downloaded from the public website of the City of ViCndEx := cdEx1(cdExV1) ∨ · · · ∨ cdExℓ(cdExVℓ) enna. Each PDF document contains regulations pertaining to one zoning area (Plangebiet), indicated by a fourwhere the cnd are all the attributes occurring with type digit ID. We discard the fraction of documents that are “condition” in List, and the cndV are their respective scanned images of printed documents and do not convalues, and similarly for ctEx for type “contentException” tain machine-readable text data (253/1431 = 17.7%) — and cdEx for type “conditionException”. For the content, we could include these in our experiments by running let further optical character recognition (OCR). PDF documents are
Cnt := cnt1(cntV1) ∧ · · · ∧ cnt(cntV) then converted to plain text using the pdftotext utilCntfor := cnt1(cntV1) ∨ · · · ∨ cnt(cntV) ity, part of the open-source Poppler library10. We use the − layout option of the tool to maintain page layout Where again the cnt are the attributes of type “content” in the output text file, this greatly simplifies the subseand the cntV their respective values. The translation of quent extraction of document structure. Next we use a rule representation with modality “obligation” then is: a small set of regular expressions to establish section obl(Cnt ∨ CntEx, Cnd) ∧ per(¬(Cnt ∨ CntEx), CndEx) boundaries and extract section numbers from the text.
Section numbering often makes use of several levels (e.g.
The translation of a rule with modality “prohibition” is: 1, 1.1, 1.1.1, etc.), but this is not consistent across docfor(Cntfor ∧ ¬CntEx, Cnd) ∧ per(Cntfor ∧ ¬CntEx, CndEx) uments, therefore we only consider top-level sections in subsequent steps that are sensitive to section boundFor the modality “permission” the translation is: aries. Besides the inheritance mechanism described in per(Cnt ∨ CntEx, Cnd) ∧ for(Cnt ∨ CntEx, CndEx) Section 3.2, this decision is crucial for sentence segmentation, the next step in our pipeline, for which we use Note that this translation commits to formalising, e.g., a customized version of the German sentence splitting condition exceptions to obligations or prohibitions as model of the stanza11[13] library. The output of the stanadditional permissions. Of course this is by no means dard model is postprocessed to undo sentence splits that the only possible translation: we could have chosen to have been made in error (e.g. those after periods followembed the condition exception explicitly in the condi- ing abbreviations characteristic of legal text) and also tion of the resulting formula. This choice is due to the those made after colons (:) that separate a predicate from fact that it facilitates the derivation of a general state- its object(s), such as in the text Für die mit BB4 bezeichment like “Flat roofs should be green roofs” from “Flat neten Grundflächen wird bestimmt: Die Errichtung von roofs should be green roofs unless they are glass roofs” Gebäuden mit einer maximalen Gebäudehöhe von 8 m ist in the particular logic used in the prover. In particular, zulässig. ‘For areas marked BB4 it is determined: conwhen checking whether a flat roof in general should be struction of buildings with a maximum building height a green roof we do not need to explicitly state that none of 8 m is allowed.’. The custom sentence segmentation of the condition exceptions are satisfied, in line with the step is followed by stanza’s default German pipeline standard approach in non-monotonic logic and default (de − gsd, stanza model version 1.1.0) for tokenization, reasoning [30]. part-of-speech (POS) tagging and universal dependency parsing.
that takes raw text documents as input, builds semantic The next step in our pipeline is to construct semantic graphs using the system described in Section 2.1, ex- graphs from each sentence. The rule extraction algorithm tracts rules using our method presented in Section 3.2, described in Section 3 assumes that all relevant informamaps them to deontic logic formulae as described in Sec- tion present in the input text is available in the semantic tion 3.3 and provides them as input to the prover (see graph that is the output of the generic semantic parsing Section 2.2). All components of our system are available pipeline described in Section 2.1. To ensure that this is the as open-source software8 under an MIT license and the case some minor modifications of the semantic parsing alend-to-end pipeline is showcased in an online demo9 gorithm were also necessary. First, we introduced a small integrating all of them. set of rules in the grammar mapping Universal Depen
dency representations to semantic graphs for common to the output in parallel to the construction of the semanwords expressing negation and modality. The lemmas tic graph, it is this additional information that allows us nicht and kein trigger the addition of the NEG element to to implement the matching heuristics described in Secthe 4lang graph, dürfen ‘may’ and zulässig ‘permitted’ tion 3.2. For parsing of 4lang graphs and generation of are mapped to PER, untersagen ‘prohibit‘ and unzulässig attribute trees with these IRTGs we use the open-source ‘not permitted’ to FOR, and müssen to OBL. Additionally, alto12 library, which also implements s-graph algebras the German construction consisting of the particle zu and tree algebras. The alto system also supports probafollowed by the infinitive form of a verb must also trigger bilistic parsing with weighted grammars, and we rely on the OBL element, since it can express modality without rule weights to ensure that rules which map subgraphs to any additional linguistic elements. This latter rule is im- attributes always take precedence over the ‘empty’ rules plemented by two mechanisms, one that looks for the that are only added to the grammar to ensure that the lemma zu with the universal part-of-speech tag (UPOS) full graph is derivable. In those few cases when more PART, the other for the language-specific part-of-speech than one such ‘content’ rule matches the same subgraph, tag (XPOS) VVIZU marking verbs that contain the par- precedence is given to rules that cover larger substructicle as an infix (e.g. herzustellen from herstellen ‘create, tures. The trees output by the IRTG parser serve as the produce’. While even the most rudimentary treatment input to the heuristic construction of rules described in of the semantics of German modal expressions would the previous section. Finally, rules are converted from the go beyond the simplicity of such a simple categorization generic (JSON) format to the language of dyadic deontic (and the scope of this work), in practice this small en- logic, as described in Section 3.3. hancement of the semantic representation of the input text was suficient to allow for the detection of modality 4.4. The prover by the rule extraction mechanism. Finally we also added an ad-hoc rule for detecting exceptions: the presence of the word sofern and soweit, both roughly equivalent to the English conjunction ‘provided’ and introducing a clause that limits the applicability of a previous statement, triggers the addition of an element EXC to the semantic graph which is then also available for processing by the rule extraction mechanism.
The final step in our pipeline consists of an exemplary reasoning mechanism to draw inferences from the extracted rules. This step is based on our adaption13 of the generic theorem prover deonticProver2.014 which implements backwards proof search in a sequent system for a dyadic deontic logic extended with rules for defeasibly reasoning from deontic assumptions [21]. Apart from specifying the prover to the language obtained from the examples we needed to further modify it in two ways. First, in order 4.3. Rule extraction to be able to handle attributes with numerical arguments, such as DachneigungMax for the maximal angle of the We now describe the implementation details of the two- roof, or with strings as argument, such as Dachart for step rule extraction method presented in Section 3.2. The the roof type, we extended the prover and the underlyoutput of the semantic parser, which serves as the input ing reasoning system to handle atomic propositions with to rule extraction, is a single directed graph for each in- arguments. In addition, we added ground sequents, i.e., put sentence, generated by an Interpreted Regular Tree structures which can be used as leaves in a derivation, corGrammar from Universal Dependency structures (see wreistphonndatinugratlonbuamsibceprrsopaservtiaelsueosf:mWeahseurree-mliskre aisttaribbuatseics Section 2.1 for details). For recognizing subgraphs and attribute for a measure such as Dachneigung, we conmapping them to attributes we also use an IRTG over sidered a triple consisting of the attributes msrGenau(), an algebra of s-graphs, this allows us to pipe the output msrMin() and msrMax(), expressing the facts that msr of the semantic parser directly into our rule extraction is exactly , at least , or at most , respectively. The grammar. For each 4lang graph we dynamically gener- relations between these three attributes are given by: ate a unique grammar. The static set of rules encoding • msrGenau() → msrMin() ∧ msrMax() the correspondence between generic semantic structures and task-specific attributes is extended with empty termi- • msrMin() → msrMin(), where ≤ nal rules for each concept in the input graph, this ensures • msrMax() → msrMax(), where ≤ that the entire graph can be constructed by a sequence of • msrMax() → ¬msrMin(), where < operations that is derivable by the underlying RTG and The ground sequents added to the prover then absorb bathus the object can be parsed by the IRTG. The output sic reasoning on these axioms, so that, e.g., the formulae interpretation of the IRTG is an algebra of trees, whose leaves are the individual strings that we use to construct ¬(DachneigungGenau() ∧ DachneigungGenau()) rules in a subsequent step. The trees resemble the order in which these strings (names and values of attributes, modal elements, units of measurement) have been added 12https://github.com/coli-saar/alto 13https://github.com/blellmann/BRISEprover 14http://subsell.logic.at/bprover/deonticProver/version2.0/ are derivable for ̸= , stating that the exact angle of check once in a preprocessing stage, store for every dea roof does not have two diferent values. ontic assumption the list of conflicting ones, and only
Second, and more significantly, to be more in line with check that none of the assumptions in this list is
appliother approaches in the area of deontic reasoning such cable and more specific during the actual computation.
as [31] as well as for eficiency reasons we modified the In our experiments this increased eficiency was
necesmechanism how the prover handles specificity reasoning sary for reasoning with a non-trivial number of deontic
washseunmreetahseondienognftriocmasdsuemonptticioanssumptions. To illustrate, assumptions. To compare the two reasoning methods
the user can switch between the original (“classic”) and
obl(DachneigungMax(5) ∧ BegruenungDach, modified (“modern”) versions on the web interface 15 for
Plangeb(7181)) (
Thus (disregarding prohibition and permission opera- 5. Evaluation
tors for the sake of exposition) to check whether (3) is
derivable we now check whether there is an assumption
obl(, ) such that
1. → ¬DachneigungGenau(
able
conflicting with obl(, ), i.e., there is no obl(, ) such that a) Plangeb(7181) ∧ Planzeichen(BB1) →
and → are derivable b) → ¬ is derivable Crucially, the “no-conflict” check in item (3b) above only needs to be performed between two assumptions and not between the formula to be proved and an assumption. This means that instead of checking for conflicts many times redundantly in the search for a derivation (as is done in deonticProver2.0) it sufices to perform this
The rule systems presented in Sec. 3.2 were developed based on a small annoted sample of sentences from the zoning plan of the City of Vienna. In order to establish a representative sample, we started by estimating the distribution of attributes in the entire corpus by manually labeling the sentences of 10 randomly selected documents with the attributes they mention (either as condition or content). This sample contains 344 mentions of attributes in 193 sentences (as well as 118 sentences without attribute mentions, mostly from the preambles). The number of unique attributes in the sample is 84, but 193 of the 344 instances (56%) come from the 16 most frequent attributes. We then chose 6 sentences from this sample that together contain mentions of 7 of these 16 attributes, including the 3 most frequent ones (GebaeudeHoeheMax, AbschlussDachMax, GebaeudeHoeheArt) that are alone responsible for 17% 15http://subsell.logic.at/bprover/briseprover/ of all attribute mentions in the larger sample. We anno- an appropriate translation. Second, our propositional tated these 6 sentences with the full representation of all language is currently rather restricted, since we do not rules stated by them and developed our rule extraction permit, e.g., disjunctions in the conditions or content system to achieve perfect performance on this toy corpus. of an obligation. Again, this could be addressed rather Both this fully annotated set and the larger sample of straightforwardly by extending the format of our repre10 documents annotated for attribute mentions only are sentation, possibly along the lines of JsonLogic17. We also released along with the software16. While our method of do not consider quantification or nested deontic operaselecting the sentences for the toy corpus ensures that tors. For the current application these features seemed the attribute extraction step of our method has high cov- not to be necessary. Most of these limitations are in line erage (recall above 51% with a precision above 93% on with other current approaches, e.g., [16, 28]. the sample of 10 documents and 344 attribute instances), The proof-of-concept application presented in this pathis cannot be considered as quantitative evaluation of per can serve as a blueprint for semantics-based solutions the full rule extraction pipeline. The limited amount to a wide range of information extraction tasks includof annotated data also does not permit any conclusions ing variants of entity recognition and relation extraction. about the efect of errors in syntactic parsing made by Such systems are generally more flexible, interpretable, the stanza model, but our assumption that this should not and less prone to bias than the large neural network modbecome a bottleneck for such standard text is reinforced els used for similar tasks. However, to make such systems by the fact that we did not observe any such errors in our a viable alternative for everyday NLP applications, novel sample. A larger-scale annotation of attribute mentions methods must be devised for the (semi-)automatic learnis currently in progress. ing of task-specific rule systems like the one manually built for this project. Concerning the automated reasoning part, we plan to consider specifications to diferent 6. Discussion frameworks in the future, including those of argumentation theory [27], I/O logic [28], and defeasible deontic In this article we have presented a system for extract- logic [16], and integrate existing provers for these foring formal rules from legal text using generic semantic malisms such as TOAST18, SPINdle19 or TurnipBox20. Adparsing and domain-specific pattern-matching, and con- ditionally, we plan to implement alternative translations verting them to deontic logic for use in an automated from the generic representation to the language of dyadic reasoning system. All components of the pipeline, includ- deontic logic, corresponding to diferent interpretations ing those contributed in this paper, are made available of the logical structure of deontic statements. Along the as open-source software under the MIT license, for un- lines of [35] this could be used to compare such diferrestricted use in future applications. Unlike machine ent interpretations. Finally, we would like to investigate learning based information extraction systems, our rule whether the part of our pipeline creating general rule extraction model is fully explainable and serves as an representations could be used in combination with the example for a specific application of semantic parsing NAI suite [17]. Our rule-based approach could be used as to domain-specific information extraction. While the se- a first step to automatically suggest a formalisation of a mantic representation and parsing algorithms used in given legal text, which then could be converted into the our pipeline are language-agnostic, they may require format used in the NAI suite and run through the quality adaptation to new languages and domains. Furthermore, assurance function provided there. The benefit would be for domains and text genres that more closely resemble that the legal experts do not need to actively formulate everyday language use, deep semantic analysis would the formalisation of a legal text, but only to potentially require lexical inference, a notoriously dificult task in adjust it based on the quality assurance checks. computational semantics [32, 33]. In our general rule representation we concentrated on deontic statements of a reasonably simple form. While this form seems to be Acknowledgments well adapted to the regulations provided in the texts for the zoning maps of Vienna, there are some obvious limi- We are grateful to the three anonymous reviewers for tations. First, since we always assume the presence of a their suggestions and for additional references. Work deontic modality (obligation, prohibition or permission), supported by BRISE-Vienna (UIA04-081), a European at the moment we cannot treat constitutive norms [34] Union Urban Innovative Actions project. such as “The area marked on the map with the label BB1 is designated a residential area”. This issue could be addressed by adding an additional modality “consitutiveNorm” to the general representation together with 17https://jsonlogic.com 18http://toast.arg-tech.org/ 19http://spindle.data61.csiro.au/spindle/ 20https://turnipbox.netlify.app