Knowledge-Based Translation of Natural Language into Symbolic Form Christodoulos Ioannou1 and Loizos Michael1,2 1 Open University of Cyprus 2 CYENS Center of Excellence christodoulos.ioannou@st.ouc.ac.cy, loizos@ouc.ac.cy Abstract Can this meta-level knowledge also be acquired by means analogous to how the object-level knowledge is acquired? We consider the scenario of machines that receive human advice in natural language to revise their In the case of interest to our work, the meta-level knowl- object-level knowledge for a domain of interest. edge represents a translation process (of object-level knowl- Although techniques exist to translate such natural edge) from natural language into symbolic form. One could language advice into a symbolic form that is appro- argue that question (Q1) is inconsequential in this case, since priate for machine reasoning, the translation pro- the meta-knowledge needed for translation purposes is pre- cess itself is typically pre-programmed and, thus, it determined and fixed, and comprises mostly generic rules ap- is not amenable to dynamic and gradual improve- plicable across advice givers. We agree only partly! Groups ment, nor can it be adjusted to the linguistic partic- of people in different parts of the world or different individu- ularities of the advice giver. We seek to examine als may use the same language differently. Therefore, being whether these limitations can be overcome through able to acquire meta-level knowledge by means analogous to the use of a knowledge-based translation process. how the object-level knowledge is acquired would allow the former knowledge to be treated as a policy of interpreting In this position paper we take a first step in this in- verbal commands. Individuals would be able to expand the vestigation by demonstrating how such meta-level policy to support interpreting additional sentence types, and translation knowledge can be engineered to support customize it to incorporate omitted but implied meanings or the interpretation of object-level advice. While, ad- different contexts when interpreting known sentence types. mittedly, our engineering of the meta-level knowl- Consequently, our sought goal is to design a process that edge amounts to pre-programming, it nonetheless will allow users to explain how to translate a sentence. For pushes towards the automated acquisition of this this reason, even though the translation process of natural lan- meta-level knowledge through advice-taking by guage is user-agnostic in its most part, the amendment of this demonstrating a key prerequisite: that it can be ex- process should be supported. A translation process providing pressed, and reasoned with, under the same syntax a dynamic and cognitively-light amendment procedure would and semantics as the object-level knowledge. also help decrease the needed amount of cognitively-heavy, batch-mode, offline pre-programming of meta-knowledge, 1 Introduction making an affirmative answer to question (Q1) important. An oft-quoted lesson from the early days of Artificial Intel- The line of research that we pursue, and the initial results of ligence is the importance of endowing machines with com- which we present in this position paper, seek to provide some monsense knowledge [McCarthy, 1989]. Equally oft-quoted kind of response to (Q1), in the particular setting of build- is the realization that such commonsense knowledge cannot ing a cognitive assistant able to help a human with a certain be directly pre-programmed into a machine, but can, more task [Leviathan and Matias, 2018]. Such a cognitive assis- realistically, be acquired through some form of learning or, tant cannot be pre-programmed, as it is expected to abide by more specifically, through some form of advice-taking from its user’s preferences when taking actions, which effectively a human [McCarthy, 1959]. But, most of the times, the re- necessitates the assistant to acquire its user-specific (object- quirement for pre-programming machines has not been com- level) knowledge through learning or advice-taking from the pletely side-stepped, but rather pushed up a level. Instead user [Bernard and Arnold, 2019]. For concreteness, in this of pre-programming the object-level (domain) knowledge of paper we focus on knowledge acquisition through a process machines, the majority of relevant AI research now effec- of advice-taking as originally proposed by McCarthy [1959] tively deals with the task of pre-programming machines at and further elaborated recently [Michael, 2017; MacLellan a meta-level (in the design of learning algorithms or archi- et al., 2018] towards the development of the human-machine tectures) that instructs machines how to acquire the object- interaction protocol of Machine Coaching [Michael, 2019]. level knowledge [Valiant, 2006; Michael and Valiant, 2008; According to Machine Coaching, then, a cognitive assis- Michael, 2016; Sap et al., 2019]. Hence, one may ask: (Q1) tant observes the state of the world and reasons using its cur- Copyright © 2021 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). 24 can interact with the CCA to help it improve its knowledge, so that future incoming call events can be handled more appro- priately (according to the user’s personal preferences). The interaction may have the form of verbal advice, handled by the Verbal Coaching Interface, or non-verbal commands han- dled by the Non-Verbal Coaching Interface (which includes applications that any standard smart phone has, like calen- dars, maps, etc.). The dialogue below demonstrates this: 1. Event: Incoming call from some caller. 2. User: Declines. “Decline calls when busy.” 3. CCA: “When do you consider yourself to be busy?” 4. User: Uses calendar and map applications and perhaps verbal commands to instruct that she considers herself busy when being in a meeting at work. Figure 1: Architecture for a Cognitive Call Assistant. 5. Event: (Later in the week. . . ) Incoming call from John while user is in a work meeting. rent object-level knowledge on how to act. A human coach 6. CCA: Declines the call. observes the assistant and can inquire as to the reasons behind 7. User: “Why did you decline this call?” the assistant’s choices. If the explanation is unconvincing or 8. CCA: “Because you are at work and at a meeting, so I otherwise insufficient or unacceptable to the human, then the conclude that you are busy.” latter can offer advice back to the assistant on how to improve its knowledge towards better future decision-making. Impor- 9. User: “Send SMS saying ‘Busy! Will call back later.’ to tantly, we focus on the case where the interaction happens in the caller when the call is important. If John is the caller natural language, and we are interested, more specifically, in then the call is important.” understanding how the natural language advice offered by the The example above is an instance of a Machine Coaching human to the assistant can be translated into some symbolic interaction used to improve the object-level (domain) knowl- form that can be used for reasoning by the assistant. edge of the CCA. The Verbal Coaching Interface is responsi- Translating natural language into symbolic form is, admit- ble for the translation of natural language into symbolic form. tedly, a well-studied problem, and numerous relevant parsing We consider this module’s policy to be implemented through tools are available [Kamath and Das, 2019]. To our knowl- meta-level knowledge expressed and reasoned with under the edge, systems built upon such parsing tools are effectively same syntax and semantics as the object-level knowledge, pre-programming the translation (meta-level) knowledge of and, thus, amenable to the same means of acquisition. the cognitive assistant on how to interpret the (object-level) We assume that the Verbal Coaching Interface ships with advice from a human. Our aim, then, is to examine whether an initial version of the meta-level (translation) knowledge. an alternative to pre-programming is feasible and effective. Compared to the CCA’s object-level (domain) knowledge, the Can this meta-level knowledge be learned? Or more specif- training of which is primarily the responsibility of the user, ically: (Q2) Can meta-level knowledge (of how to interpret the initial version of the meta-level knowledge is engineered object-level advice) be acquired through Machine Coaching? by experts to support some minimal level of translation. If Question (Q2) is our main long-term research question. We the end user is unsatisfied with the translation process (i.e., if take a first step towards answering this question by demon- the translated symbolic form of a natural language sentence is strating how the meta-level knowledge can be engineered to wrong or unexpected, or a natural language feature is unsup- achieve its goal of interpreting object-level advice. Although ported), the end user should be able to advise and revise the engineering the meta-level knowledge does, in fact, amount meta-level knowledge as done for the object-level knowledge. to pre-programming, it nonetheless pushes towards the auto- Apparently, the initial version of the meta-level knowledge mated acquisition of this meta-level knowledge through Ma- might not be rich enough to allow the user to explain and cor- chine Coaching by demonstrating the key prerequisite that it rect all errors or handle all unknown advice sentences. Since can be expressed, and reasoned with, under the same syntax we do not expect the user to be fluent in the symbolic form and semantics as the object-level knowledge, which itself is or to be a natural language expert, she must be able to ask an amenable to acquisition through Machine Coaching. expert for help, who will advise, in her stead, the translator module using logic-based communication and the Machine 2 A Machine Coaching Example Coaching protocol to improve the translation process and/or As our running example, we consider a Cognitive Call Assis- adjust it to her needs. We illustrate this type of interaction by tant (CCA) installed on a smart phone that takes actions upon extending the conversation of our running example. receiving incoming calls, with an architecture as in Figure 1. Suppose that the Verbal Coaching Interface fails to trans- Upon receiving an incoming call, the CCA uses its current late the last advice sentence from the user to the CCA during (object-level) knowledge to decide on how to act. The user their last interaction from above. The dialogue continues: 25 10. CCA: “I am unable to interpret the advice ‘If John is the As we have argued, this is a first step towards being able caller then the call is important’? Do you want to consult to acquire this knowledge gradually and dynamically, with- an expert to help improve the translation policy?” out the need of a pre-specified ontology, and in a manner that 11. User: “Yes, please!” allows the meta-level knowledge (and hence the translation process of the object-level advice) to be amenable to adjust- 12. CCA: “Please provide additional information that you ment to the linguistic particularities of each human. wish to pass on to the expert.” 13. User: “By this sentence I mean that I wish all calls com- 4 Architecture and Methodology ing from John to be labeled as important. This, in turn, The pipeline design of the CCA’s Verbal Coaching Interface will trigger a different treatment by the CCA.” responsible for the interpretation of a piece of advice given 14. Event: The advice that could not be interpreted, along by the user is shown in Figure 2. According to the pipeline, with the additional information provided by the user are a piece of advice given verbally by the user is parsed (in sent to an expert. The expert machine coaches the CCA Module 1) into a collection of predicates that describe the to improve its meta-level knowledge. dependencies in the sentence of the advice. These predicates act as input to the meta-level knowledge of the CCA, with 15. CCA: “Expert advice has been received and the transla- which the CCA reasons (in Module 2) to turn the piece of tion policy has been amended. Please continue!” advice into an object-level rule, thus interpreting the advice. Thus, using Machine Coaching (albeit communicating di- This object-level rule is then used to revise, through Machine rectly in the symbolic form of the meta-knowledge), the ex- Coaching, the CCA’s object-level knowledge, and it is this pert can dynamically and gradually incorporate additional object-level knowledge that is utilized by the CCA when de- features of natural language (e.g., word synonyms, sentence ciding how to handle future incoming calls. The details of the types, etc.) and user-specific adjustments (e.g., personalized Machine Coaching revision mechanism are inconsequential phrase disambiguation, local dialect, etc.) that might have not for this work, and can be found elsewhere [Michael, 2019]. been part of the initial version of the meta-level knowledge. In the remainder of this section we discuss in more detail the key components of the architecture described above. 3 Overview of Related Work 4.1 User Language for Advice-Giving Our primary focus is on translating natural language advice For the purposes of this work, we restrict the types of sen- or questions into symbolic form. A lot of work has been done tences in which object-level advice can be expressed to the in information and rule extraction from natural text, using on- ones below [Huddleston and Pullum, 2002]. The numbers in tologies or other scope-related information [Delannoy et al., brackets after each sentence type are references to the respec- 1993; Dragoni et al., 2016; Hassanpour et al., 2011]. Unlike tive sentence examples that are provided in Appendix A: that work, our methodology does not seek to find important words / phrases based on the scope of the application, and 1. Simple Declarative Sentences (SDS) no ontologies are used. Instead, we investigate the structure 1.1. Simple Declarative To-Be Sentences and relations within natural language sentences without tak- 1.1.1. with Adjective or Noun Predicate [1–3] ing the context into account, which has the advantage that 1.1.2. with Verb Predicate [4–5] our methodology is application-oblivious. Although not us- 1.2. Simple Declarative Verb Sentences ing ontologies or scope-related annotations might lead to vital information being unavailable when rules are extracted, this 1.2.1. with Object [6] can be gradually remedied through Machine Coaching. 1.2.2. without Object [7–9] Other work has also been done in symbolic analysis of nat- 2. Simple Imperative Sentences (SIS) [10] ural language text with the use of Controlled Natural Lan- 3. Conditional Sentences (zero conditional) guages (CNL) [Kuhn, 2014]. In our approach, although sim- Below SDC stands for Simple Declarative Clause ilar in some ways with CNL approaches [Kain and Tompits, and SIC stands for Simple Imperative Clause. 2019], we do not strictly define a controlled natural language, but instead we start by considering a basic subset of natural 3.1. Imperative Conditional Sentences (ICS) language that we can then gradually and iteratively expand. 3.1.1. If/When SDC, SIC [11–12] Our proposed architecture starts by extracting dependency 3.1.2. If SDC, then SIC [13] relations between words, along with Part of Speech (POS) 3.1.3. SIC if/when SDC [14] and Named Entity Recognition (NER) annotations [Grish- 3.1.4. all above with Implied Subject in SDC [15–16] man, 1997]. This information is subsequently used to iden- 3.2. Declarative Conditional Sentences (DCS) tify patterns in the semantic relations that hold in a sentence, 3.2.1. If/When SDC, SDC [17–19] which support the construction of a rule capturing what a user 3.2.2. If SDC, then SDC [20] wishes to communicate through that sentence. The key de- 3.2.3. SDC if/when SDC [21–22] parture of our work from past works [Akbik and Broß, 2009; Gerber and Ngomo, 2012; Bos, 2015] is that the translation 3.2.4. all above with Implied Subject in SDC [23] process itself is encoded as a collection of meta-rules, consti- For the purposes of simplicity, sentences with negation are tuting the meta-level knowledge of a cognitive assistant. not considered in this initial set of supported sentence types. 26 Figure 2: Implementation pipeline for a Verbal Coaching Interface. 4.2 Syntax of the Meta-Level Knowledge Each example comprises an input sentence and its type, Given an object-level advice sentence, we apply Natural Lan- the intuitively expected formal rule corresponding to that sen- guage Processing (NLP) tools to get dependency, part of tence, the dependency parse tree for that sentence, the appli- speech, and named entity recognition data (in the form of a set cable meta-rules that are triggered by that sentence type, the of meta-predicates) and apply dependency analysis to manu- inferences drawn from the meta-rules, and ultimately the gen- ally identify patterns across these sets, which we later use erated object-level rule. Within each dependency parse tree, to engineer meta-rules. We have defined and consider three the bounding boxes of the respective figures highlight the pat- types of meta-predicates for use in the body of meta-rules: terns that trigger the meta-rules. Each variable W and P in the meta-rules corresponds, respectively, to an arbitrary word < Dependency > ( in the sentence and the word’s position in that sentence. Parent Word, Parent Word Position, Child Word, Child Word Position) Example Sentence 1: A call is important. pos(< POS Tag >, Word, Word Position) Type: 1.1.1. Simple Declarative To-Be Sentence with Adjec- tive Predicate. ner(< NER Tag >, Word, Word Position) Expected Rule: call(X) implies important(X); where < Dependency >, < POS Tag >, and < NER Tag > Dependency Parse Tree: see Figure 3 (left). are placeholders for the dependency relation between two Applicable Meta-Rules: words, for the Part of Speech of a word, and for the Named root(root, 0, W1 , P1 ), cop(W1 , P1 , be, ), nsubj(W1 , P1 , W2 , ) Entity Recognition tag of a word, respectively, as identified implies ruleterms([[W1 ]], [[X1]], [[W2 ]], [[X1]]); by NLP. In addition to the above meta-predicates, a special Meta-Inferences: meta-predicate is used in the head of meta-rules to represent predicates and variables of the generated object-level rule: ruleterms([[important]], [[X1]], [[call]], [[X1]]) Generated Rule: ruleterms( Head Terms, Head Vars, call(X1) implies important(X1); Body Terms, Body Vars) where Head Terms and Body Terms are lists of lists that rep- Example Sentence 2: Send SMS when call is important. resent object-level predicates for the head and the body of the Type: 3.1.3. Imperative Conditional Sentence (SIC when generated rule, respectively. Each sub-list represents a single SDC). predicate, and each sub-list member represents a part of the Expected Rule: predicate (in case a predicate is a multi-word concatenation). call(X), important(X), SMS(Y) implies send(Y); Head Vars and Body Vars are also lists of lists and represent Dependency Parse Tree: see Figure 3 (right). the corresponding variables of the object-level predicates. Each advice sentence is assumed, in this work, to contain Applicable Meta-Rules: a single conditional, corresponding, thus, to a single object- advcl( , , W1 , P1 ), cop(W1 , P1 , be, ), nsubj(W1 , P1 , W2 , ) level rule. This single rule, however, might involve the ap- implies ruleterms([[]], [[]], [[W2 ], [W1 ]], [[X1], [X1]]); plication of multiple meta-rules on the given advice sentence, root(root, 0, W1 , P1 ), pos(verb, W1 , P1 ), dobj(W1 , P1 , W2 , ) with each meta-rule contributing part of the generated object- implies ruleterms([[W1 ]], [[X2]], [[W2 ]], [[X2]]); level rule; see, e.g., Example Sentence 2 in Section 4.3. Meta-Inferences: 4.3 Engineering the Meta-Level Rules ruleterms([[]], [[]], [[call], [important]], [[X1], [X1]]) ruleterms([[send]], [[X2]], [[SMS]], [[X2]]) For each of the sentence types considered in Section 4.1, we have engineered one or more associated meta-rules, which Generated Rule: are presented in Appendix B. Three meta-rules are discussed call(X1), important(X1), SMS(X2) below as part of two examples that demonstrate the key ideas. implies send(X2); 27 Figure 3: Dependency parse trees for Example Sentence 1 (left) and Example Sentence 2 (right). In addition to the example meta-rules that are presented • Simple sentences with omitted words and implied mean- above, there are versions of those meta-rules that cope with ings cannot be handled appropriately. the case of sentences with a named entity subject, since the • NLP sometimes labels incorrectly words with multiple resulting object-level rules need to be different in such cases. meanings, resulting in incorrect dependency trees. In terms of the methodology followed in engineering meta- rules, we note that this involved the manual analysis of multi- ple sentences of each of the considered types, and the identifi- 6 Conclusion and Future Work cation and clustering of dependency patterns. In a sense, then, We have presented our progress in developing a translation we have manually-induced these meta-rules from a training process from natural language into symbolic form that is, it- set of sentences, having in mind that the engineered meta- self, expressible in the same symbolic form. Even though rules are expected to apply and work well also with sentences the meta-level knowledge developed in the context of this pa- (of the same types) that were not part of the training set. per was engineered, our initial results provide some confi- dence that the meta-level knowledge itself could be acquired 5 Preliminary Empirical Evaluation through a process of Machine Coaching, as is the case with the object-level knowledge. This, in turn, suggests an affir- Towards empirically evaluating whether the engineered meta- mative answer to the two questions posed in this paper, pro- level knowledge adequately addresses the task of interpreting viding for a translation process that is amenable to dynamic user advice, we have implemented in Java the pipeline out- change in an elaboration tolerant manner [McCarthy, 1998]. lined in Section 4. Natural language is parsed using the Stan- Towards solidifying our initial work and findings, we are ford CoreNLP Library [Manning et al., 2014], the meta-level currently in the process of porting the meta-level knowledge knowledge is implemented, for quick prototyping purposes, from SWI-Prolog to Prudens, a first-order logic language and in SWI-Prolog, and the Java Prolog Library (JPL) is used framework (http://cognition.ouc.ac.cy/prudens/) with seman- to interface with the meta-level knowledge. The meta-level tics that supports directly the paradigm of Machine Coaching. knowledge can be invoked to translate advice from natural This will be followed by the undertaking of a quantitative em- language into symbolic form through a desktop application pirical study to evaluate the efficiency, effectiveness, and ex- or a web-service call (http://cognition.ouc.ac.cy/nestor/). plainability of a machine-coachable translation process. This position paper does not provide a full-fleshed quan- More specific goals to be pursued in future work include: titative empirical evaluation, which is part of our ongoing work. However, as part of a preliminary evaluation, we cre- • demonstrating the feasibility of machine coaching for the ated a custom data-set that includes at least one sentence of object-level task by utilizing engineered meta-rules; each type considered, along with its variants, and applied our • evaluating more thoroughly the ability of our engineered engineered meta-level knowledge on that data-set. The list of meta-level knowledge to cope with a varied collection of the generated object-level rules can be found in Appendix A. advice sentences of the types already supported; That outcome was evaluated by the authors qualitatively, and below we present some early findings from this evaluation: • examining how the sentence types that are supported can be extended (e.g., by supporting negation, logical connectors, • The generated object-level rules reasonably capture the implied meanings, pronoun disambiguation, textual entail- meaning of their associated natural language sentences. ment [Levesque et al., 2012; Korman et al., 2018]); • In some cases, slightly altering the syntax of a sentence • supporting other natural languages, and quantifying the ex- (e.g., removing the comma from the sentence “If John is tent to which meta-level knowledge developed for one nat- busy, send SMS”), alters the produced dependency tree. ural language can be transferred (even partially) to another; 28 • experimenting with the use of machine learning techniques [Grishman, 1997] Ralph Grishman. Information Extraction: for the acquisition of meta-level knowledge; Techniques and Challenges. In Proceedings of the Inter- • acquiring meta-level knowledge through a Verbal Coaching national Summer School on Information Extraction, pages Interface, analogously to the acquisition method of object- 10–27. Springer, 1997. level knowledge that is described in this paper. [Hassanpour et al., 2011] Saeed Hassanpour, Martin J. The last direction from above raises, in fact, certain inter- O’Connor, and Amar K. Das. A Framework for the esting theoretical questions: Is there a meta-language rich Automatic Extraction of Rules from Online Text. In enough to support its own verbal-coachability without the Proceedings of the International Workshop on Rules and need for the involvement of experts? Is there a way to decide Rule Markup Languages for the Semantic Web, pages or to prove this claim? Or is there a need for an infinite hierar- 266–280. Springer, 2011. chy of meta-languages needed to support the verbal-coaching [Huddleston and Pullum, 2002] Rodnry Huddleston and Ge- of knowledge at lower levels of the meta-language hierarchy? offrey K. Pullum. The Cambridge Grammar of the English Even without concrete answers to these theoretical ques- Language. Language. Cambridge: Cambridge University tions, central in our future work is to include a knowledge- Press, 2002. based translation process in a cognitive assistant that can be [Kain and Tompits, 2019] Tobias Kain and Hans Tompits. verbally-coached, and which will also support the machine- Uhura: An Authoring Tool for Specifying Answer-Set coaching (even if not the verbal-coaching) of the translation Programs Using Controlled Natural Language. In Pro- process itself, so that the latter can improve over time and ceedings of the 16th European Conference on Logics in adapt to the linguistic particularities of its current user. Artificial Intelligence, pages 559–575. Springer, 2019. [Kamath and Das, 2019] Aishwarya Kamath and Rajarshi Acknowledgements Das. A Survey on Semantic Parsing. In Proceedings of This work was supported by funding from the EU’s Horizon the 1st Conference on Automated Knowledge Base Con- 2020 Research and Innovation Programme under grant agree- struction, 2019. ments no. 739578 and no. 823783, and from the Government [Korman et al., 2018] Daniel Z. Korman, Eric Mack, Jacob of the Republic of Cyprus through the Deputy Ministry of Research, Innovation, and Digital Policy. Jett, and Allen H. Renear. Defining Textual Entailment. Journal of the Association for Information Science and Technology, 69(6):763–772, 2018. References [Kuhn, 2014] Tobias Kuhn. A Survey and Classification of [Akbik and Broß, 2009] Alan Akbik and Jügen Broß. Wan- Controlled Natural Languages. Computational Linguis- derlust: Extracting Semantic Relations from Natural Lan- tics, 40(1):121–170, 2014. guage Text using Dependency Grammar Patterns. In Pro- ceedings of the WWW Workshop on Semantic Search, vol- [Levesque et al., 2012] Hector Levesque, Ernest Davis, and ume 48, 2009. Leora Morgenstern. The Winograd Schema Challenge. In [Bernard and Arnold, 2019] Denys Bernard and Alexandre Proceedings of the 13th International Conference on the Arnold. Cognitive Interaction with Virtual Assistants: Principles of Knowledge Representation and Reasoning, From Philosophical Foundations to Illustrative Examples 2012. in Aeronautics. Computers in Industry, 107:33–49, 2019. [Leviathan and Matias, 2018] Yaniv Leviathan and Yossi [Bos, 2015] Johan Bos. Open-Domain Semantic Parsing Matias. Google Duplex: An AI System for Accomplish- with Boxer. In Proceedings of the 20th Nordic Conference ing Real-World Tasks Over the Phone. Technical report, of Computational Linguistics, pages 301–304, 2015. Google AI Blog, 2018. [Delannoy et al., 1993] Jean François Delannoy, C. Feng, [MacLellan et al., 2018] Christopher J. MacLellan, Erik Stan Matwin, and Stan Szpakowicz. Knowledge Extrac- Harpstead, Robert P. Marinier III, and Kenneth R. tion from Text: Machine Learning for Text-to-Rule Trans- Koedinger. A Framework for Natural Cognitive System lation. In Proceedings of the ECML Workshop on Machine Training Interactions. Advances in Cognitive Systems, Learning and Text Analysis, pages 7–13, 1993. 6:1–16, 2018. [Dragoni et al., 2016] Mauro Dragoni, Serena Villata, [Manning et al., 2014] Christopher D. Manning, Mihai Sur- Williams Rizzi, and Guido Governatori. Combining NLP deanu, John Bauer, Jenny Rose Finkel, Steven Bethard, Approaches for Rule Extraction from Legal Documents. and David McClosky. The Stanford CoreNLP Natural In Proceedings of the 1st Workshop on Mining and Language Processing Toolkit. In Proceedings of 52nd An- Reasoning with Legal Texts, 2016. nual Meeting of the Association for Computational Lin- guistics: System Demonstrations, pages 55–60, 2014. [Gerber and Ngomo, 2012] Daniel Gerber and Axel- Cyrille Ngonga Ngomo. Extracting Multilingual [McCarthy, 1959] John McCarthy. Programs with Common Natural-Language Patterns for RDF Predicates. In Sense. In Proceedings of the Teddington Conference on the Proceedings of the 18th International Conference on Mechanization of Thought Processes, pages 75–91, 1959. Knowledge Engineering and Knowledge Management, [McCarthy, 1989] John McCarthy. Artificial Intelligence, pages 87–96. Springer, 2012. Logic and Formalizing Common Sense. In Philosophical 29 Logic and Artificial Intelligence, pages 161–190. Springer, Sentence 4: John is working. 1989. Type: 1.1.2. Simple Declarative To-Be Sentences with Verb [McCarthy, 1998] John McCarthy. Elaboration Tolerance. In Predicate. Proceedings of the 4th International Symposium on Log- Generated Rule: ical Formalizations of Commonsense Reasoning, pages work(john); 198–216, 1998. [Michael and Valiant, 2008] Loizos Michael and Leslie G. Sentence 5: Students are studying. Valiant. A First Experimental Demonstration of Massive Knowledge Infusion. In Proceedings of the 11th Interna- Type: 1.1.2. Simple Declarative To-Be Sentences with Verb tional Conference on Principles of Knowledge Represen- Predicate. tation and Reasoning, pages 378–388. AAAI Press, 2008. Generated Rule: [Michael, 2016] Loizos Michael. Cognitive Reasoning and student(X1) implies study(X1); Learning Mechanisms. In Proceedings of the 4th Interna- tional Workshop on Artificial Intelligence and Cognition, Sentence 6: Birds have wings. volume 1895, pages 2–23. CEUR-WS.org, 2016. Type: 1.2.1. Simple Declarative Verb Sentences with Object. [Michael, 2017] Loizos Michael. The Advice Taker 2.0. In Generated Rule: Proceedings of the 13th International Symposium on Com- birds(X2), wing(X1) implies have(X2, X1); monsense Reasoning, volume 2052. CEUR-WS.org, 2017. [Michael, 2019] Loizos Michael. Machine Coaching. In Sentence 7: A man cries. Proceedings of the IJCAI Workshop on Explainable Artifi- cial Intelligence, pages 80–86, 2019. Type: 1.2.2. Simple Declarative Verb Sentences without Ob- ject. [Sap et al., 2019] Maarten Sap, Ronan Le Bras, Emily All- away, Chandra Bhagavatula, Nicholas Lourie, Hannah Generated Rule: Rashkin, Brendan Roof, Noah A Smith, and Yejin Choi. man(X1) implies cry(X1); Atomic: An Atlas of Machine Commonsense for If-Then Reasoning. In Proceedings of the 33rd AAAI Conference Sentence 8: John speaks. on Artificial Intelligence, volume 33, pages 3027–3035, 2019. Type: 1.2.2. Simple Declarative Verb Sentences without Ob- ject. [Valiant, 2006] Leslie G. Valiant. Knowledge Infusion. In Generated Rule: Proceedings of the 21st AAAI Conference on Artificial Intelligence, volume 6, pages 1546–1551. AAAI Press, speak(john); 2006. Sentence 9: Eagles fly high. Type: 1.2.2. Simple Declarative Verb Sentences without Ob- A Evaluation Data and Results ject (as part of a complex sentence). Generated Rule: Sentence 1: Students are smart. eagles(X1) implies fly(X1); Type: 1.1.1. Simple Declarative To-Be Sentences with Ad- jective or Noun Predicate. Sentence 10: Answer the phone. Generated Rule: Type: 2. Simple Imperative Sentences. student(X1) implies smart(X1); Generated Rule: phone(X1) implies answer(X1); Sentence 2: Eagle is a bird. Type: 1.1.1. Simple Declarative To-Be Sentences with Ad- Sentence 11: If John is busy, send SMS. jective or Noun Predicate. Type: 3.1.1. Imperative Conditional Sentences (If/When Generated Rule: SDC 1.1.1, SIC). bird(eagle); Generated Rule: busy(john), sms(X1) implies send(X1); Sentence 3: Gifts from my wife are precious. Type: 1.1.1. Simple Declarative To-Be Sentences with Ad- Sentence 12: If John is working, send SMS. jective or Noun Predicate (as part of a complex sentence). Type: 3.1.1. Imperative Conditional Sentences (If/When Generated Rule: SDC 1.1.2, SIC). gift(X1) implies precious(X1); Generated Rule: 30 work(john), sms(X1) implies send(X1); Sentence 21: Dad is happy when paint shines. Type: 3.2.3. Declarative Conditional Sentences (SDC 1.1.1 Sentence 13: If John has a phone, then send SMS. if/when SDC 1.2.2). Type: 3.1.2. Imperative Conditional Sentences (If SDC 1.2.1, Generated Rule: then SIC). paint(X3), shine(X3), dad(X1) implies happy(X1); Generated Rule: phone(X3), have(john, X3), sms(X1) implies send(X1); Sentence 22: Dad washes the car when the sun shines. Type: 3.2.3. Declarative Conditional Sentences (SDC 1.2.1 Sentence 14: Send SMS when John is busy. if/when SDC 1.2.2). Generated Rule: Type: 3.1.3. Imperative Conditional Sentences (SIC if/when SDC 1.1.1). sun(X3), shine(X3), car(X1), dad(X2) implies wash(X2, X1); Generated Rule: busy(john), sms(X1) implies send(X1); Sentence 23: When tired, head is aching. Type: 3.2.4. Declarative Conditional Sentences (If/When Sentence 15: Send SMS if busy. SDC with implied Subject, SDC 1.1.2). Type: 3.1.4. Imperative Conditional Sentences (SIC if/when Generated Rule: SDC with implied Subject). tired, head(X1) implies ache(X1); Generated Rule: busy, sms(X1) implies send(X1); B Engineered Meta-Level Knowledge Sentence 16: If tired, play the guitar. Type: 3.1.4. Imperative Conditional Sentences (If/When :- dynamic pos/3. :- dynamic ne/3. SDC with implied Subject, SIC). :- dynamic root/4. :- dynamic det/4. :- dynamic dobj/4. :- dynamic advmod/4. Generated Rule: :- dynamic advcl/4. :- dynamic punct/4. tired, guitar(X1) implies play(X1); :- dynamic nsubj/4. :- dynamic cop/4. :- dynamic mark/4. :- dynamic case/4. :- dynamic nmod/4. :- dynamic amod/4. Sentence 17: If John is clever, Anna is happy. :- dynamic nmod_poss/4. :- dynamic aux/4. Type: 3.2.1. Declarative Conditional Sentences (If/When :- dynamic ccomp/4. :- dynamic ner/3. SDC 1.1.1, SDC 1.1.1). %Utilities %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% Generated Rule: % addvarlist - Add indexed var list clever(john) implies happy(anna); addvarlist(0, _, []). addvarlist(N, OFFSET, [[VARNAME]|BASEVLL]) :- Sentence 18: When John is bad, Anna is crying. N > 0, Type: 3.2.1. Declarative Conditional Sentences (If/When VARINDEX is OFFSET + N, SDC 1.1.1, SDC 1.1.2). string_concat(‘X’, VARINDEX, VARNAME), PREVN is N - 1, Generated Rule: addvarlist(PREVN, OFFSET, BASEVLL). bad(john) implies cry(anna); % addvar - Add indexed var addvar(0, _, []). Sentence 19: When boss pays well, employees are happy. Type: 3.2.1. Declarative Conditional Sentences (If/When addvar(N, OFFSET, [VARNAME|BASEVL]) :- SDC 1.2.2 as part of a complex sentence, SDC 1.1.1). N > 0, VARINDEX is OFFSET + N, Generated Rule: string_concat(‘X’, VARINDEX, VARNAME), pay(boss), employee(X1) implies happy(X1); PREVN is N - 1, addvar(PREVN, OFFSET, BASEVL). Sentence 20: If John is rich, then Anna has money. % Meta-Level Knowledge Definition %%%%%%%%%%% Type: 3.2.2. Declarative Conditional Sentences (If SDC % Simple TO BE Clauses w. adj. or noun pred. 1.1.1, then SDC 1.2.1). sdc(_, W1, [[W1]], [[W2]], [[]], [[]]) :- cop(W1, P1, be, _), Generated Rule: nsubj(W1, P1, W2, P2), ner(_, W2, P2), !. rich(john), money(X1) implies have(anna, X1); 31 sdc(OFFSET, W1, [[W1]], HVL, [[W2]], BVL) :- sub_string(TAG, 0, _, _, ‘vb’ ), cop(W1, P1, be, _), nsubj(W1, P1, W2, _), dobj(W1, P1, W2, _), !. addvarlist(1, OFFSET, HVL), addvarlist(1, OFFSET, BVL). % Simple Imperative Clauses without subject sic([[W1]], [[‘X1’]], [[]], [[]]) :- % Simple TO BE Clauses with verb Predicate pos(TAG, W1, _), sdc(_, W1, [[W1]], [[W2]], [[]], [[]]) :- sub_string(TAG, 0, _, _, ‘vb’ ). aux(W1, P1, be, _), nsubj(W1, P1, W2, P2), ner(_, W2, P2), !. % Simple Imperative Sentences sis(HTL, HVL, BTL, BVL) :- sdc(OFFSET, W1, [[W1]], HVL, [[W2]], BVL) :- root(root, 0, W1, _), aux(W1, P1, be, _), nsubj(W1, P1, W2, _), sic([[W1]], HVL, BTL, BVL), addvarlist(1, OFFSET, HVL), append([], [[W1]], HTL). addvarlist(1, OFFSET, BVL), !. % Simple Declarative Sentences % Simple Transitive Verb Clauses sds(HTL, HVL, BTL, BVL) :- sdvc( _, W1, root(root, 0, W1, _), [[W1]], [[W2, W3]], [[]], [[]]) :- sdc(0, W1, HTL, HVL, BTL, BVL), !. pos(TAG, W1, P1), sub_string(TAG, 0, _, _, ‘vb’ ), sds(HTL, HVL, BTL, BVL) :- nsubj(W1, P1, W2, P2), ner(_, W2, P2), root(root, 0, W1, _), dobj(W1, P1, W3, P3), ner(_, W3, P3), !. sdvc(0, W1, HTL, HVL, BTL, BVL). sdvc( OFFSET, W1, % Simple IF conditional clause A [[W1]], [[W2|HVL]], [[W3]], BVL) :- scc([[]], [[]], BTL, BVL) :- pos(TAG, W1, P1), mark(W1, P1, ‘if’, _), advcl(_, _, W1, P1), sub_string(TAG, 0, _, _, ‘vb’), sdvc(2, W1, [[W1]], CHVL, CBTL, CBVL), nsubj(W1, P1, W2, P2), ner(_, W2, P2), append(CBTL, [[W1]], BTL), dobj(W1, P1, W3, _), addvar(1, OFFSET, HVL), append(CBVL, CHVL, BVL), !. addvarlist(1, OFFSET, BVL), !. % Simple WHEN conditional clause A sdvc( OFFSET, W1, scc([[]], [[]], BTL, BVL) :- [[W1]], [HVL], [[W2]], BVL) :- advmod(W1, P1, when, _), advcl(_, _, W1, P1), pos(TAG, W1, P1), sdvc(2, W1, [[W1]], CHVL, CBTL, CBVL), sub_string(TAG, 0, _, _, ‘vb’), append(CBTL, [[W1]], BTL), nsubj(W1, P1, W2, _), append(CBVL, CHVL, BVL), !. dobj(W1, P1, W3, P3), ner(_, W3, P3), addvar(1, OFFSET, PHVL), % Simple IF conditional clause B append(PHVL, [W3], HVL), scc([[]], [[]], BTL, BVL) :- addvarlist(1, OFFSET, BVL), !. mark(W1, P1, ‘if’, _), advcl(_, _, W1, P1), sdc(1, W1, [[W1]], CHVL, CBTL, CBVL), sdvc( OFFSET, W1, append(CBTL, [[W1]], BTL), [[W1]], [HVL], [[W2], [W3]], BVL) :- append(CBVL, CHVL, BVL), !. pos(TAG, W1, P1), sub_string(TAG, 0, _, _, ‘vb’ ), % Simple WHEN conditional clause B nsubj(W1, P1, W2, _), dobj(W1, P1, W3, _), scc([[]], [[]], BTL, BVL) :- addvar(2, OFFSET, HVL), advmod(W1, P1, when, _), advcl(_, _, W1, P1), addvarlist(2, OFFSET, BVL), !. sdc(1, W1, [[W1]], CHVL, CBTL, CBVL), append(CBTL, [[W1]], BTL), % Simple Verb Clauses append(CBVL, CHVL, BVL), !. sdvc(_, W1, [[W1]], [[W2]], [[]], [[]]) :- pos(TAG, W1, P1), % Simple IF cond. clause w. implied subject sub_string(TAG, 0, _, _, ‘vb’ ), scc([[]], [[]], [[W1]], [[]]) :- nsubj(W1, P1, W2, P2), ner(_, W2, P2), !. mark(W1, P1, ‘if’, _), advcl(_, _, W1, P1), !. sdvc(OFFSET, W1, [[W1]], HVL, [[W2]], BVL) :- % Simple WHEN cond. clause w. implied subject pos(TAG, W1, P1), scc([[]], [[]], [[W1]], [[]]) :- sub_string(TAG, 0, _, _, ‘vb’ ), advmod(W1, P1, when, _), advcl(_, _, W1, P1). nsubj(W1, P1, W2, _), addvarlist(1, OFFSET, HVL), ruleterms(HTL, HVL, BTL, BVL) :- addvarlist(1, OFFSET, BVL). scc(HTL, HVL, BTL, BVL). ruleterms(HTL, HVL, BTL, BVL) :- % Simple Imperative Clauses with subject sds(HTL, HVL, BTL, BVL), !. sic([[W1]], [[‘X1’]], [[W2]], [[‘X1’]]) :- ruleterms(HTL, HVL, BTL, BVL) :- pos(TAG, W1, P1), sis(HTL, HVL, BTL, BVL). 32