Formalization of Ukrainian-Language Content for Fuzzy Product in Court Anna Bakurova1 [0000-0001-6986-3769], Mariia Pasichnyk2 [0000-0002-5179-4272], Elina Tereschenko3 [0000-0001-6207-8071], Yurii Filei4 [0000-0001-5919-1129] National University «Zaporizhzhia Polytechnic», Zaporizhzhia, Ukraine 1abaka111060@gmail.com, 2mary.pasechnik@gmail.com, 3elina_vt@ukr.net, 4fileyyuriy@gmail.com Abstract. This paper is devoted to the development of a method of formalizing Ukrainian-language content for fuzzy output production systems. Three variants of variables, which described in a text document, were proposed. The applica- tion’s effectiveness of the proposed variants of variables initialization for pro- duction systems in the field of justice was carried out. It was made on the example of the processing of the verdicts texts in separate articles. The studies were con- ducted using the automated system Neuro-Adaptive Learning ANFIS in MATLAB. The best accuracy were provided by normalized initialization func- tions which used the positional calculus system with the least possible basis. Keywords: decision-making, court sentence, Ukrainian-language content, on- tology, fuzzy production system, the order ratio, graph. 1 Introduction There are different models of information technologies implementation in the justice field in the world practice [1]. The concepts of "e-justice", "e- proceedings", "e-court" or "online court" have been fixed by law. Ukraine is a member of the continental legal system. Among the countries with such legal model, the most developed is e-proceedings of Germany, where the filling of documents, processing and adjudication are done in electronic format [2]. The sentencing process formalizing is a discussion issue. The large volume of legal norms, documentation of cases, the pressure of respon- sibility for the human fate, the ambition of a just sentence creates a diversity of attitudes to this question [3]. Recognizing that modern advanced technologies will not replace the lawyer profession, it can be argued that they allow professionals to work effectively with large amounts of information that passes through the analytical system. The work [4] emphasizes the steady interest increase to the intellectual systems use in various fields, particular, in jurisprudence. Modern intellectual systems use knowledge bases that are formed according to the subject area. One of the main results of [4] is the development of data extraction method based on the ontological knowledge base of syntax analysis of Ukrainian-language text documents. Copyright © 2020 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). The study results on the construction of the product rules base and its optimization are presented, for example, in [5]. This study develops models and algorithms for opti- mizing the design of conjugated fuzzy knowledge bases using the criteria of “precision output - complexity” as a min-max clustering problem. In another work [6], a study was conducted on the comparative analysis of four hy- brid artificial intelligence models for the prediction of probable fires; each model con- sists of an adaptive neuro-fuzzy inference system (ANFIS) [7] combined with a specific metaheuristic optimization algorithm. In this paper, authors set out to develop a common knowledge base of sentencing in the combination of different circumstances to create a basis for determining similar sentences in similar in composition and circumstances of the crimes. Such knowledge base can be used as a reference or training base. 2 Related Works As you know, the production system System formally displays System : Input  Output , where in the most general sense, the input is given infor- mation Input that should affect the output result Output . In common case, the input information Input is verbal, qualitative, nonmetric, unstructured, or poorly structured. This information may contain a description of the phenomenon, object, process. It is necessary to distinguish such information about one or more attributes, features, prop- erties of these phenomena, objects or processes that are significant in the sense of in- fluencing the output result Output . The outputs Output contain information about possible alternative solutions, thus this information can be considered more structured. The research conducted in this work is a further development of the binary relation Listening : Court  Court Рrocess that is introduced into the basic ontology of the law system [8]. The preliminary results reported by the authors in [9, 10, 11] allowed us to determine the model and stages of building a knowledge base for sentencing, namely the presentation of knowledge in the form of a fuzzy inference product model. The development of the knowledge base goes through these four stages. At the first stage, the authors formed the structure of the ontology of the court deci- sion, which is outlined in [9]. The developed ontology of the sentence is a structure that clearly illustrates the relationship between the concepts that are the classes of input (offerder’s personality, judge's personality, circumstances burdening and mitigating the crime) needed to make a decision and the degree of punishment presented by a fine, restriction and imprisonment (real and conditional), public works (Fig.1). The ontology of a court sentence has become the basis for developing a model of a productive decision support system (DSS) in court. At the second stage, in [10], this article’s authors constructed a system of fuzzy in- ference to support court decisions. In such form, the construction of the ontology of the court decision allowed to develop a general model of DSS in a court that has the form: (Fine, Years, RF, Public Works, Condition) = F(Severity, Mitigation, Personality, Burden, Lawyer), (1) Fig. 1. Ontology of making a court decision where F is the corresponding output algorithm; Severity is characterizes the severity of the crime; Personality is characterizes the guilty person; Mitigation are mitigating cir- cumstances; Burden are circumstances that burden the punishment; Lawyer is the level of neutrality of a court’s decision and takes value with the term {loyal, neutral, strict}; Fine is the size of the fine; Years are the number of years of imprisonment; RF (Re- striction of freedom) is restriction of freedom; Public Works are Public Works. Condition determines the actual or conditional entry into action. General ideas were tested on the implementation of the DSS for a separate article of the Criminal Code of Ukraine in two variants, with Mamdani and Sugeno algorithms [10]. The experiment was conducted on sentences under Article 185 of the Criminal Code [12] of Ukraine on theft contained in the register of court decisions of Ukraine [13]. An interesting feature of production systems is that the Input information at the in- put of system is textual, and at the output there should be information containing a quantitative component. This feature gives rise to the need to initialize qualitative var- iables that are characterized by natural language. Therefore, in the third stage it is nec- essary to build a model of concepts developed by the ontology. This requires the crea- tion of algorithms for handling existing court cases. The theoretical substantiation of the stage is outlined in [11], which proposes a method of constructing a membership function for fuzzy sets of terms of a linguistic variable that allows to formalize verbal information for production systems. Next, in the fourth stage, it is necessary to build the production base and optimize it. By optimization we mean ensuring the achievement of conflicting goals, namely, im- proving the accuracy of output while reducing the complexity of the system. The auto- mated Neuro-Adaptive Learning ANFIS in MATLAB system was used to test possible approaches to performing the tasks of the fuzzy output production system optimization and optimization process [7]. In this work attention is focused on the last two stages and demonstration of possi- bilities of the formalization method of Ukrainian-language content for creation of a fuzzy production system on the basis of initialized input vectors. 3 Our Approach for Evaluating Variables Contained in Ukrainian-Language Content Further studies allowed to apply the results obtained in [11] to construct a method for initializing the input vectors of a fuzzy production system, which allows to formalize the information contained in textual form and to adjust the system itself. The method consists of four steps: modeling information of Ukrainian-language content in the form of digraphs; building order relations on the elements of this model; initialization of input vectors; fuzzy production system setup. In the setup of fuzzy production system, we will understand the definition of mem- bership functions for fuzzy sets of terms of linguistic input variable information, pro- duction rules, and accurate output estimation by the Sugeno fuzzy output method. Initialization variables must be defined for the vector of input variables. Assume that the variable  contains an estimate of certain text information Input that is put to the product system input. The first step of the method is the modeling of ontology concepts [9] in the form of a digraph H in Fig. 2 [14]. Assume that the variable  is characterized by n nonmetric features Pi , i  1, n forming the set P , P = n . Each feature Pi can take k i values p ij , j  1, k i . Each branch of the order H defines a specific feature Pi , i  1, n . The vertexes correspond- ing to the features Pi , i  1, n form the first tier. The second tier consists of the leaves of the order H , which are possible values p ij of the feature Pi . To determine the set of features P and the possible meanings of these features, a semantic analysis of textual information is required [14]. The second step of the method is to build the order of the plural order X . The domain of the variable definition  is determined by the Cartesian product of non- n metric features X  P1  P2  ... Pn , X   ki : i 1    X  xs xs  p1 j , p2 j ,..., pnj ,1  jsi  ki , i  1, n . s1 s2 sn  The problem is that the arbitrary positioning of the value xs on the abscissa does not allow to build an initialization function f ( x s ) with predefined properties. Therefore, it is necessary to define a bijective mapping f : X  R that meets two requirements: 1. the reproduction of a binary order  that corresponds to the natural ordering xs of a certain quality criterion, and 2. fulfillment of conditions of monotonicity of the initialization function f ( x s ) . The theoretical basis of the ordering of the set X is given in [11]. Fig. 2. Model of the concept of ontology (variable  ) in the form of a digraph H . The result of the method’s second step is an ordered set of feature values, which enables the ordered arrangement of the abscissa of the vectors set     X  x s x s  p1 j s1 , p 2 j s 2 ,..., p njsn ,1  j si  k i , i  1, n . The third step is to define the initialization functions for the variable  . To determine the identifier xs , we apply positional numeral system with base, in which the digit 10ni1 is determined by the sequence number of the vertex describing the feature Pi in the strict order column and the digit jsi - the sequence number of the value pij for the feature Pi , which was implemented in the vector xs . Let denote the si absence of a value for a particular sign as “0”. Place the vectors of the values P of the traits on the abscissa axis according to the defined order p,  . At the fourth step, to set up a fuzzy product system, it is needed to build the vectors  s1 s2 sn  of the input xs  p1 j , p2 j ,..., pnj ,1  jsi  ki , i  1, n , xs  X , and output data. Func- tional dependency views f ( xs ) should reproduce the intuitively expected evaluation of an object  , that is, fit the order  and provide general requirements for the monot- ony of the function. For large values ki , i  1, n and n , the number of vectors X is too large for the expert definition of the initialization function. Therefore, it is advisable to automate the initialization functions’ construction, which further will be used to train the fuzzy pro- duction system on the values of the input and output variables. The following types of formulas are offered to determine the initialization function f ( xs ) , where r , r  1,2 - parameters: n f ( xs )  1   2 *  jsi * base  i , 0  jsi  ki (2) i 1 One of definition variant is base  10 . It is also advisable to determine the basis of the calculation system base , where base  1mod(max ji ) . Another way to define the ini- tialization function as linear: f ( xs )  3  4 * num( xs ) (3) where num( xs ) is the sequence number of the vector xs in order  . 4 Experiments The authors aimed to conduct an experiment to investigate the effectiveness of applying formula (2) with different bases of the calculation system and formula (3) to build esti- mates of the input vectors of a fuzzy production system. To achieve this aim, the capabilities of the automated system Neuro-Adaptive Learn- ing ANFIS in MATLAB were used [7]. To train the fuzzy system with the neuroadap- tive methods’ help, matrix is formed from the input and output experiments’ data. In order to form a fuzzy output system, which works with neuro-adaptive methods and satisfies the output quality requirements, ANFIS training should be conducted on a sample data that fully reflects the features of the simulation data. According to Art. 65, 66, 67 of the Criminal Code of Ukraine [12], as in previous works [9, 10], the following methods were chosen by the method of direct logical de- duction by input linguistic variables: Severity - severity of the crime committed; Per- sonality - characteristic of the person of the offender which takes value from the term- set {negative, neutral, positive}; Mitigation - an assessment of the judge's ability to consider the amount of mitigating circumstances to the crime; Burden - assessment of the judge's ability of taking into account the number of burden circumstances; Lawyer characterizes the level of neutrality of the judge and takes the value from the term-set {loyal, neutral, strict}. We will assume that the judge is "neutral". The analysis of the verdicts in court cases showed that there was another element that influenced the court's decision. This is the plaintiff. Therefore, the model (1) pre- sented in [10] is supplemented by the linguistic variable Plaintiff, which takes the value from the term {adjusting to mitigation, neutral, not adjusting to mitigation}. The following general model of DSS is proposed in court: (Fine, Years, RF, Public Works, Condition) = F (Severity, Lawyer, Personality, Plain- tiff, Mitigation, Burden), (4) where F is the corresponding fuzzy inference algorithm. For the experiment were selected part 3 Art. 185 of the Criminal Code of Ukraine on theft and part 2 Art. 240 "Violation of the rules of protection or use of subsoil" [12]. The experiments were performed to construct a fuzzy inference system by the Sugeno algorithm. The mathematical model of DSS was adopted in accordance with the selected articles in court: (Years) = SUGENO(Personality, Plaintiff, Mitigation, Burden) (5) The models’ simplification with relation to the general (4) was made in view of the fact that the gravity of the crime is unchanged within the scope of a specific article of the Criminal Code. The automatic implementation of Sugeno output with Neuro-Adaptive Learning ANFIS in MATLAB allows to get the only one output that was chosen Years or Fine. 4.1 An Example of Production System Construction According to Part 385 of the Criminal Code of Ukraine The first step: modeling ontology concepts in the form of digraphs. For its realization it is necessary to select information concerning variables: Person- ality, Mitigation, Burden, Plaintiff, which are defined in (4). The collection of docu- ments consisted of 25 court sentences under part 385 of the Criminal Code of Ukraine. The documents were obtained from the Unified Register in accordance with the rules for using this electronic resource containing certain restrictions for access [13]. The statutory form of the court decision contains an introductory, motivating and resolutive part. The information about the offender is contained in the introductory part, in the explanatory part the court indicates what information about the person of the defendant is taken into account when determining his sentence, circumstances that mitigate or burden the sentence. The resolution section contains the court's decision on sanctions. The study used the methods of semantic analysis [15, 16]: the signal words in the HTML document highlighted the necessary paragraphs of the motivation part, which built the libraries, removed the stop words from the Unified Register of court sentences, stamming, calculation of term-document matrices with TF-IDF weight index use. The categories of terms and words corresponding to the terms of the ontologies con- cepts were distinguished: the person of the guilty; the plaintiff; the mitigating and bur- dening circumstances. Synonyms for each concept were expertly created from the cho- sen terms. Models are represented by digraphs in Fig. 3 and Fig. 4. For example, let’s look at many of the features for Personality in Ukrainian, in courts’ decision language: PP1ersonality  {працевлаштування, сімейний стан, проживання, реєстрація, харак- теристика, перебування на обліку, наявність зв‘язків, кримінальна відповідаль- ність, судимість}, P  9 . In the second step we will define the area of definition of variables Personality and Plaintiff. Therefore, the domain of the variable X definition is a Cartesian product of the fea- X  1728 tures X  P1  P2  ... P9 , whose power . The digraph in Fig. 3 reflects the expertly defined order of characteristics relative to the term "positive person". This makes it possible to arrange 1728 plural elements X . Fig. 3. Model of the Personality ontology concept In the third step, we calculate the normalized values of the estimates based on the for- mula (2) with base  10 , base  4 and on the formula (3) (Fig. 5 a)). Similar actions are performed for the Plaintiff model. 1 Ppla int iff  {відшкодування отримав, моральні збитки, просить пом‘якшення}, P  3. Therefore, the domain of the variable X definition is a Cartesian product of the fea- tures X  P1  P2  P3 , whose power X  8 . The digraph displays an expertly defined order of characteristics with respect to the term "Adjusting to mitigation". Fig. 4. Model of the Plaintiff ontology concept for Art.185 For variables Personality and Plaintiff initialization functions were built. In formulas (2) - (3) assume that  k , k  1,7 . Normalized initialization functions for variable Personality are presented in Fig. 5 a), for variable Plaintiff in Fig. 5 b). The fourth step, namely the definition of a linguistic variable based on the model created and the fuzzy production system setup, was carried out using the automated ANFIS in MATLAB system. The network learning method is a hybrid, which is a com- bination of the least-squares method and the inverse gradient reduction method. a) b) Fig. 5. Normalized initialization functions а) variable Personality , b) variable Plaintiff. In Fig. 6 shown the structure of the neuro-fuzzy system. The system consists of 4 layers. On the first layer put the numerical values of the input vector. The second layer is a phase layer in which the modules reproduce a fuzzy membership function to represent the input values in a fuzzy set. Each module at this level corresponds to a certain fuzzy set. The next layer defines the fuzzy rules. Each module at this level meets certain rules. Each node connects to those nodes of the first layer that form the preconditions of the corresponding rule. The modules implement operations "AND" with a minimum in the t-norm form [16]. The fourth layer implements an output device, one for each output. The input vectors are initialized by function (2) with the basis either base  10 , or base  4 for Personality and base  2 for Plaintiff, or (3) with normalization and without normalization. The experiment results showed that the best accuracy was ob- tained with the normalized initialization functions (2) base  4 for Personality and base  2 for Plaintiff. The biggest error is given by function (3). a) b) Fig. 6. A neuro-fuzzy system built on normalized data that is initialized by function (2) base  4 for Personality and base  2 for Plaintiff a) neural network, b) learning error. 4.2 An Example of Building Production System under Part 2 of Art. 240 of the Criminal Code of Ukraine The variable model Personality is the same as the previous example. The Plaintiff in Article 240 is considered the state. The state's mitigation of the sentence is provided by the Law of Ukraine on the Use of Amnesty in Ukraine [18]. Therefore, the model of the Plaintiff variable is represented in relation to the effect of this law, and the ordering is made in relation to the term "Adjusting to mitigation". Fig. 7. Plaintiff Ontology Concept Model for Art. 240 2 Ppla int if  { amnesty_law , P 2 pla int if  1 . Therefore, the variable definition area 2 Ppla intif 2 has power X plaintif  2 . The digraph in Fig. 7 reflects the expertly defined order of characteristics. In the third step, we calculate the normalized values of the estimates based on the formula (2) with base  10 or base  4 for Personality and base  2 for Plaintiff, or (3) with normalization and without normalization (Fig. 5 a)). Several experiments were performed for the selected sentence collection: for sen- tences containing fines; for sentences containing years of imprisonment and restriction of freedom, and for the general sample with pre-coding of sentences according to the severity of the sentence. Construction and training of fuzzy inference systems were performed. Experiments were conducted with variants of forming an input vector sim- ilar to Example 1. The conclusions were obtained similar to Example 1. 5 Discussions and Further Researches In practical application of the initialization functions (2) - (3), some certain features must be considered. The estimates (2) need normalization, to provide a global scale when not all attrib- utes of a qualitative variable  have their characteristics. The initialization functions are discrete, which makes it possible to obtain a finite set of evaluation values. For the adopted method, not all values of the abscissa axis are implemented in vec- tors xs . Entering an identifier corresponding to the number of possible values ki , i  1, n for each trait will eliminate the redundancy. With many features, there are limitations in the accuracy of the calculations. In this case, it would be appropriate to break the set of features into subsets, with their further implementation in separate product rules. In terms of further research, methods of distinguishing the terms of the concepts of the ontology of the court decision need to be developed. It is necessary to build suffi- ciently exact ‘fine’ algorithms for recognition of the semantics of court sentences, be- cause the vocabulary contains antonymic pairs, differing in the proportion "no", hom- onyms, the same keywords for different classes of information. 6 Conclusions and Acknowledgment Scientific novelty is determined by firstly developed common Sentence Knowledge Base based on the method of formalizing Ukrainian-language content for fuzzy output production systems. The structure of the information contained in the court sentence is based on a combination of different crime circumstances reflected in the court's DSS model and ways of formalizing it as input to the fuzzy output system. The development will allow the processing of large volumes of court sentences as training samples for fuzzy inference systems. The work was carried out as the part of the research work «Mathematical modeling of socio-economic processes and systems», the registration number DB05038, at the Department of System Analysis and Computational Mathematics of "Zaporizhzhia Polytechnic" National University. 7 References 1. Von, D., Zareczka, I., Suchenko, S. et al. Yevropejs`ki ta mizhnarodni standarty u sferi sudo- chynstva [European and international standards in the field of justice] Madoyan, Kyiv (2015) (In Ukr.). 2. Bryntsev, O. “Elektronnyi sud” v Ukraini. Dosvid ta perspektyvy [“E-court” in Ukraine. Ex- perience and perspectives] Pravo, Kharkiv (2016) (In Ukr.). 3. Tobota, Yu. Poniattia ta kryterii «spravedlyvoho sudovoho rozghliadu» u rishenniakh yevro- peiskoho sudu z prav liudyny [Conception and criteria of «fair legal process» in judgements of European Court of human rights]. Visnyk Kharkivskoho natsionalnoho universytetu imeni V. N. Karazina 1086, 65-68 (2013) (In Ukr.). 4. Lytvyn, V., Vysotska, V., Hrendus, M. Method of data expression from the ukrainian content based on the ontological approach. Radioelektronika, informatika, upravlinnya 30, 157 (2018). 5. Rakytianska, H. Optymizatsiia spoluchenykh nechitkykh baz znan na pravylakh i vidnoshen- niakh [Optimization of complied fuzzy knowledge based on rules and relations]. Informatsiini tekhnolohii ta komp`iuterna inzheneriia 1, 17-26 (2015) (In Ukr.). 6. Jaafari, A., Zennerb, E., Panahic, M., Shahabid, H. Hybrid artificial intelligence models based on a neuro-fuzzy system and metaheuristic optimization algorithms for spatial prediction of wildfire probability. Agricultural and Forest Meteorology 266-267, 198-207 (2019). 7. Neuro-Adaptive Learning and ANFIS, https://www.mathworks.com/help/fuzzy/neuro-adap- tive-learning-and-anfis.html#responsive_%20offcanvas. Last access 2020/02/28. 8. Valkman, Yu., Hala, E. Proektirovanie ontologii dlya pravovoy predmetnoy oblasti na osnove tekstovogo kontenta s ispolzovaniem nechyotkoy logiki [Designing ontology for a legal sub- ject area based on textual content using fuzzy logic]. Ontologiya proektirovaniya 2(12), 19- 39 (2014) (In Russian). 9. Bakurova, A., Tereschenko, E., Filei, Y., Pasichnyk, M., Ropalo, H. Modeling of decision making ontology. PROCEEDINGS OF THE 3RD INTERNATIONAL CONFERENCE ON COMPUTATIONAL LINGUISTICS AND INTELLIGENT SYSTEMS (COLINS-2019). Vol.1: Main Conference. http://ceur-ws.org/Vol-2362/paper18.pdf (2019). 10. Bakurova, A., Tereschenko, E., Filei, Y., Pasichnyk, M. Production model for administration of judicial decisions in the case of theft. EXPERIMENTAL ECONOMICS AND MACHINE LEARNING FOR PREDICTION OF EMERGENT ECONOMY DYNAMICS. Proceedings of the Selected Papers of the 8th International Conference on Monitoring, Modeling & Man- agement of Emergent Economy (M3E2-EEMLPEED 2019), vol. 2422, pp.284-296. Odessa, Ukraine. http://ceur-ws.org/Vol-2422 (2019). 11. Bakurova, A., Pasichnyk, M., Tereschenko, E. Fuzzy Modeling of Verbal Information for Production Systems. Innovative Technologies and Scientific Solutions for Industries 4(10), 5- 13. DOI: https://doi.org/10.30837/2522-9818.2019.10.005 (2019). 12. Kriminalniy kodeks Ukrayiny, (In Ukr.), http://zakon.rada.gov.ua/. Last access 2020/02/28. 13. Сourt decisions register of Ukraine, (In Ukr.), http://www.reyestr.court.gov.ua. Last access 2020/02/28. 14. Ore, O. Theory of graphs, American Mathematical Society Colloquium Publications vol. 38; 270 (1962). 15. Manning, Ch., Raghavan, P.,Schutze, H. Introduction to Information Retrieval, Cambridge University Press, UK (2008) Online: https://doi.org/10.1017/CBO9780511809071 Last ac- cess 2020/02/28. 16. Lande, D. Elementy kompiuternoi linhvistyky v pravovii informatytsi [Elements of computa- tional linguistics in legal informatics]. NDIIPR, Kyiv (2014) (In Ukr.). 17. Novak, V., Perfyleva, Y., Mochkrozh, Y. Matematicheskie principy nechetkoj logiki [Math- ematical Principles of Fuzzy Logic]. Fizmatlit, Sankt-Peterburg, (2006) (In Russian). 18. Amnesty Law of Ukraine, (In Ukr.), https://zakon.rada.gov.ua/laws/show/392/96- %D0%B2%D1%80. Last access 2020/02/28.