Multi-Criteria Knowledge-Based Recommender System for Decision Support in Complex Business Processes

Abstract

In this paper, we present a concept of a multi-criteria knowledgebased Recommender System (RS) designed to provide decision support in complex business process (BP) scenarios. The developed approach is based on the knowledge aspects of Stylistic Patterns, Business Sentiment and Decision-Making Logic extracted from the BP unstructured texts. This knowledge serves as an input for a multi-criteria RS algorithm. The output is prediction of the BP complexity, based on which the algorithm modifies the type and the way of decision support, ranging from full to minimal automation. We show how the algorithm can be applied in the real-life scenarios by the example of the IT ticketing case study. We also evaluate the BP complexity prediction quality using both quantitative (databased) and qualitative (interview-based) approach in the case study.

CCS CONCEPTS

• Information systems → Information systems applications; Decision Support Systems; Data Analytics With the considerable technology progress and enterprise digitization, the discussions around the timeworn term of complexity gain new power. Especially businesses and their IT departments report a dramatic increase in the process complexity [ 29 ]. In this context, a BP must have a certain level of complexity to correspond with the complexity of its environment. Thus, the complexity can be challenged and caused by both complex BP IT environment and constantly increasing information flow to be handled in the BP [ 26 ]. It is widely acknowledged that RS open wide opportunities for different domains and particularly businesses. Hereby, the main characteristic of RS e-business applications is an intensive use of the knowledge-based RS approaches, i.e. ontologies and semantic technologies. This can be explained by the fact that businesses demand a high degree of domain knowledge for adequate assistance in recommendations [ 14 ]. Hereby, the main RS challenges of robustness, 2

Related Work

Multi-criteria RS are based on the well-known Multi-Criteria Decision Making (MCDM) methods [ 1, 22 ]. The value of multicriteria recommendation approach in general and the MCDM methods in particular has been demonstrated long ago and in various application domains [ 15, 16, 25 ]. At present, one of the most popular categories has proven to be multi-criteria rating recommenders, which though suffer from a number of problems, e.g. constructing the best set of criteria [ 1 ]. In regards to knowledge-based recommenders, one differentiates two types: case-based and constraint-based. Constraint-based RS exploit the predefined knowledge bases with the explicit rules of delivering the recommendation and are considered to perform well, specifically in complex product domains [ 7 ]. Rule-based reasoning and rulebased expert systems have long been a focus of research on intelligent systems [ 4, 9 ]. Currently, they find another advantageous practical application as a part of constraint-based RS. Knowledge-based RS provide a major value in overcoming such limitations as lack of transparency, cold-start problem and data sparsity, which are common for content-based and collaborative filtering approaches. However, acquiring the necessary knowledge possessed by domain experts and converting it into formal, executable representations is a challenging task [ 8 ].

Thus, the contributions of the paper can be highlighted in the following: 1) construction of a set of criteria for a recommendation problem in the context of unstructured BP texts, which is an important topic for future research in multi-criteria RS [ 1 ] and 2) provision of a method to efficiently extract the necessary knowledge aspects and transform them into executable representations targeting the problem described above [ 8 ].

3 Concept of Multi-Criteria Knowledge-Based RS

A typical case study scenario from an ITIL-based CHM IT ticketing process considered during the research is the following: 1) a customer request (ticket) for a change in IT infrastructure products or services is sent per e-mail; 2) requested changes can be processed with various templates (pre-filled forms). Ideally, tickets addressing related problems are processed with the same template. However, key word search used at the case study department doesn’t yield relevant results. Thus, a new template is likely to be created both in case of a new type of request and when the template is not found; 3) based on the information documented in a ticketing system, the requested change is implemented. The goal of the RS concept is to address the problems described in 2), i.e. incorrect search results which imply inefficient work and time loss. While remaining an important starting point, key word search must be viewed as only one of several tools supporting the BP workers, especially in the context of key word search commonly known limitations [ 12 ]. On the user side, key word search is known for a constant need to reformulate the queries, no possibility to precisely specify the search intention and limited knowledge on or availability of the data to precisely express the search intention [ 2 ]. On the key word search technology side, most existing solutions focus on small datasets [ 30, 28 ] and efficiency instead of search quality [ 5, 28 ]. With the proposed RS, it is aimed to support a BP worker in finding the most successful way to process the request under given conditions, i.e. incoming ticket text. The RS modelled in BPMN [ 17 ] (see Figure 1) should, first, support the BP worker in ticket prioritization and, second, adapt the type and the way of recommendation based on the complexity level of the ticket text, i.e. perceived processing complexity ( ), identified with the help of multi-criteria knowledge aspects, i.e. Readability ( ), Perceived Anticipated Effort ( ) and Business Process Cognition ( ) (see Section 3.1). Hereby, it is important to note that the computation yields to the three levels of “low”, “medium” and “high”. This scale was selected for two reasons: 1) in order to simplify the method presentation and 2) it is a known scale of priority ratings especially for measuring intangible criteria in the context of decision-making [ 24 ]. Tickets with “low” can be described with clear rules and easily automated by one-to-one template recommendation. Tickets of “medium” are those where no exact rule set exists and there is a need of information acquisition and evaluation. Here, the RS can provide a partial processing support in a form of drop-down menu templates (multiple-choice recommendations). In case of “high”, the RS will offer a minimal assistance while listing the history of similar implemented tickets.

In a general IT ticket context, one can differentiate between three types of complexity: 1) ticket processing complexity a) perceived while reading the ticket and b) real complexity reported after the ticket is processed; 2) ticket implementation complexity related to the technical execution of the ticket related tasks. The scope of the proposed RS is targeted at 1a. At the moment of the IT ticket entry, the BP worker receives the textual description of the request characterized by the following parameters influencing the perception of request processing complexity: quality of the written text (comprehension of the request), urgency of the request and type of the requested activity. According to these factors, corresponding criteria and measures were selected in the scope of the present RS: quality of the written text measured by , urgency – by and type of the activity – by .

3.1 Conceptual Framework

In the context of the present research, we refer to the recommendation problem as an MCDM problem and use the conceptual notation by [ 22 ]. Accordingly, we specify the RS concepts for the present research as follows below.

Defining the object of decision. Object of decision is item that belongs to the set of all candidate items. In the case study of the research, the objects of the decision are classified into three categories based on the identified : 1) one-to-one ticket templates where is a number of the ticket template in the database; 2) drop-down menu templates where is the number of the drop-down menu suggestion; 3) similar tickets in the database history where is the number of the ticket record in the database. The elements of this set are specified as alternatives to which four types of decision problems (choice, sorting, ranking, and description) can be applied [ 22 ]. In the current research, we refer to sorting (classification of alternatives into a number of predefined three categories) and choice (selection of a more appropriate alternative). To sum up, O ∈ { , , }. Family of criteria. Performance fit of alternatives is analyzed upon a set of criteria. In the paper, fit of alternatives from the three categories mentioned above is evaluated upon a set of criteria for each incoming ticket text . As fairly stated by [ 22 ], the design of a consistent family of criteria for a given recommendation application has been largely ignored in the RS literature and constitutes an important problem for future research. Here, a family of three measurable quantitative-qualitative criteria = { 1 , 2 , 3 } is applied on the in order to predict and generate a recommendation (see also Section 3.2 for more details). The choice of the criteria and especially corresponding measures is justified by the textual nature of the input data. As the unstructured textual BP requests serve as the basis for recommendation, the technologies used for criteria extraction come from the domains of

Applied

Linguistics,

Stylistics,

Sentiment

Analysis, and Taxonomies. The approaches have been selected based on and therefore

are covering the three common levels of text understanding: objective (answering the who, what, where, when, etc. questions, e.g. taxonomies and ontologies), subjective (who has which opinion about what, e.g. Sentiment

Analysis) and metaknowledge (what can be extracted about the text apart from its contents, e.g. with Stylistics or Stylometry) [ 6 ].

Thus, the first criterion 1 is suggested to be Readability measured by Stylistic Patterns (SP) [ 19 ]. SP of ticket texts are considered to influence the BP worker’s perception of the contextual complexity of the ticket processing and express the quality of the written text affecting the understanding of the request (metaknowledge). In the present RS concept, SP are defined as a function of Syntactic Structure (SynS) and Wording Style (WS) for the different length values of the BP text . Hereby, SynS is a syntactic structure of text calculated as relative distributions σ of and unique , where are words organized as per part of speech (PoS) of nouns, verbs, adjectives, and adverbs. WS is the wording style of text bringing in relation rank-frequency and quantity-frequency of words [ 32 ] in approximated with coefficients and b in a form of ( + ) [ 27 ].

The second criterion 2 is suggested to be Perceived Anticipated Effort (

) measured by Business Sentiment (BS) representing emotional component of ticket complexity or also urgency of the request (subjective knowledge) [ 21 ]. BS is calculated based on the lexicon approach with the help of relative distributions of identified BS-loaded PoS of negative, positive and neutral valences σ( , , ), where , , are words with the corresponding valence of positive, neutral or negative.

The third criterion 3 is suggested to be Business Process Cognition (

) measured by semantic nature of activities in the ticket identified with Decision-Making Logic (DML) Taxonomy (objective knowledge) [ 20 ]. DML Taxonomy is built while extracting semantically loaded and calculating their relative distributions in σ( , , ). , , are DML elements (words) indicating routine, semi-cognitive and cognitive activities organized as PoS according to RTCC Framework (nouns ( ) as

Resources, verbs ( ) as Techniques, adjectives (

) as Capacities, adverbs (

) as Choices) into three classes of routine (r), semicognitive (sc) and cognitive (c).

Global preference (recommendation) model. The development of a global preference model provides a way to aggregate the values of each criterion = { 1, 2, 3} in order to express the preferences between the alternatives. In the paper, the most established approach of a value-focused model is pursued [ 1 ]. Marginal preferences upon each criterion are synthesized into a total value function, also known as utility function [ 13 ]. The utility-based formulation of the multi-criteria recommendation problem in the present research is formulated with the help of context dependent rule sets which determine the meaningfulness or the weight of each criterion in the specific context (see Section 3.2 and 4 for more details).

3.2 Case Study Application

Based on the qualitative interviews and literature reviews, the following assumptions are introduced: 1) ticket length is accepted as a parameter indicating . We discovered while performing the survey that case study BP workers usually receive short texts in case of simple, explicit and already familiar requests. To a certain extent, this fact is also supported by the theory of the least effort [ 31 ]. Based on the case study contextual specificity calculated with the help of statistical analysis, a threshold has to be set; 2) the distribution σ of PoS has a direct impact on contextual readability. Information in the tickets rich in unique nouns (BP Resources) and with low number of other PoS (for example, BP Techniques) is easy to perceive and systemize for a BP worker; 3) in case of word frequencies (Zipf's coefficient ), a threshold has to be set; 4) while implementing the approaches , and { 6, , the rule sets ∈ { 1, 2}, { 3, 4, 5 } and 7,

8 } have to be developed based on the specific statistical values of the case study in focus. First, we describe the extraction and interpretation of the knowledge aspects related to the three suggested criteria. After, we show how the extracted aspects and related criteria are used to feed the RS.

Readability ( 1). There are certain Stylistic Patterns (SP) embedded in the BP (ticket) texts influencing the worker’s perception of the contextual complexity of the task processing [ 19 ]. It is proposed to measure the SP with relative distributions of PoS and unique PoS (SynS) and Zipf’s word frequencies (WS). Input: incoming ticket with ( , , , ), accepted threshold for the ticket length and accepted threshold for coefficient in the corpus Output: exclusive qualitative values of 1 “telegraphic”, “effortless” and “involving effort” for all ∈ do if < and σ( )>0 and σ( , , )=0 and b=0 then 1=“telegraphic” if < and σ( , )>0 and σ( )>σ( , , ) and σ( )>σ(∃! ) and < then 1=“effortless” else 1=“involving effort” end The algorithm considers that: 1) depends on , short tickets being the simple ones; 2) the tickets containing only nouns are written in a very condensed telegraphic way, i.e. either BP worker already knows what needs to be done or the ticket is complex and this complexity will be captured with criteria 2 or 3 depending on their meaningfulness in the case study context; 3) ticket texts containing high relative number of BP Resources (nouns), which are also unique, are easy to understand. The WS ( ) indicates the information presentation flow, i.e. condensed versus disperse. Perceived Anticipated Effort ( 2 ). reflects the emotional component of the ticket contextual complexity perceived by the BP worker while reading the ticket text [ 21 ]. It is proposed to be measured by the specified Business Sentiment.

Input: incoming ticket in the corpus , manually created BS lexicon-computed valence values of , , , case study specific rule set ∈ { 1, 2} [ 21 ] Output: 2 exclusive qualitative values “low”, “medium”, “high” for all , , ∈ do , )= 1 then 2=“low”

, )= 2 then 2=“medium” if σ( ,

if σ( , else 2=“high” end The algorithm reproduces the computation of the emotional component of the BP contextual complexity expressed by urgency and task complexity.

Business Process Cognition ( 3 ). The algorithm presents the identification of semantic nature of activities in the ticket texts by means of DML Taxonomy. The knowledge aspects are extracted with the help of the mentioned RTCC Framework whereby nouns ( ) express Resources, verbs ( ) – Techniques, adjectives ( ) – Capacities, and adverbs ( ) – Choices. It is suggested to classify the BPs (tickets) into three categories of routine, semi-cognitive and cognitive based on the semantically implied complexity [ 20 ]. Input: incoming ticket in the corpus , manually created DML Taxonomy from with , , organized as PoS in RTTC Framework [ 20 ], case study specific rule set ∈ { 3, 4, 5} Output: 3 exclusive qualitative values “routine”, “semicognitive”, “cognitive” for all , , ∈ do if σ( , , ) = 3 then 3 =“cognitive” if σ( , , ) = 4 then 3 =“routine”

if σ( , , )= 5 then 3=“semi-cognitive” end The algorithm follows semantic tagging approach which classifies the activities described in tickets into three pre-defined categories. Multi-Criteria Recommendations. Computed criteria values and inferred are used to feed multi-criteria knowledge-based RS. Based on , the recommendation from ∈ { , , } alternatives should be offered to the BP worker.

Input: computed qualitative values for 1 ( ), 2( ), 3( ), the case study specific rule sets determining the meaningfulness or weight of each criterion in the case study context ∈ { 6, 7, 8} Output: and a recommendation for the BP worker for 1 ( ), 2( ), 3( ) do if 1, 2, 3= 6 then if 1, 2, 3= 7 then if 1, 2, 3= 8 then ={ , } = “low” and = = “medium” and = =“high” and end In the experimental session, we evaluated the knowledge aspects extraction according to = { 1, 2, 3} on the case study data set and calculated case study specific threshold parameters and rule sets which were iteratively adjusted based on the computed and its quantitative and qualitative evaluation. These values and an experimental set-up of the proposed RS on the example of a randomly selected ticket are presented in the section below. 4

Experiments and Evaluation

In the experimental and evaluation phase, we conducted quantitative (experiments) and qualitative (interviews) analyses as shown on the Figure 2 below. First (see point 1 on Figure 2), initial experiments were carried out in order to set up initial values of case study threshold parameters & rule sets. The computational analyses were conducted based on the pre-processed data set comprising CSV-formatted 28,157 text entries (tickets) in English language. The approaches of specified knowledge aspects extraction were executed on the data set subsequently. Inline and in the tables below, we present the final values for the threshold parameters and rule sets obtained after the evaluation rounds described in this section: 1) accepted threshold for the ticket length – 25 words ( ); 2) accepted threshold for coefficient – 3; 3) accepted rule set ∈ { 1, 2} for computation is presented in Table 1; 4) accepted rule set ∈ { 3, 4, 5} for computation is presented in Table 2; 5) accepted rule set ∈ { 6, 7, 8} of is presented in Table 3 (the values in each of the cell of the table represent possible alternatives).

In the evaluation phase (see point 2 and 3 on Figure 2), we conducted quantitative and qualitative analyses iteratively in order to fine tune the threshold parameters and rule sets from point 1 on Figure 2. While discussing the with the case study BP workers, it was discovered that there is no such a complexity definition as in the current case study context. However, another type of complexity (real complexity of the ticket processing mentioned in 1b, see Section 3.1.) can be measured based on the historical ticket data from the IT ticketing system. These data included configuration items, specifically affected applications (which is closely related to the number of tasks in the case study context), number of tasks, risk type of the ticket, and implementation type (online vs offline). Real complexity can be calculated on the ordinal scale yielding to the values of “low”, “medium” and “high”, those applied in the computation, and thus can be used for the evaluation of .

Consequently, quantitative analysis with a new data set from the same case study comprising 4,625 ticket text entries in English was performed to compute the of each of the ticket (see point 2 on Figure 2). To compute real complexity, we used mentioned historical data from the IT ticketing system. Following the rules provided by the case study BP workers, we calculated the real complexity for each of the ticket also classifying it into “low”, “medium” and “high”.

As shown in point 3 on Figure 2, we iteratively consulted with the case study BP workers and conducted qualitative evaluation of the RS in a form of the interviews. An overall conceptual framework was introduced to the team of 13 managers of the case study department responsible for the correct ticket processing. For this purpose, a semi-structured interview approach was developed with a planned set of questions regarding the feasibility and applicability of the computation and the development of recommendations based on the .

The qualitative evaluation was divided into three parts. First, we introduced the objectives, research motivations, theoretical and methodological background. Second, the RS concept, specifically the computation, was illustratively presented using a set of 60 randomly selected IT tickets containing 54% of correctly and 46% of incorrectly identified from the case study data set. The estimation of correctness was performed using the computed real complexity values. The case study BP workers were asked to critically evaluate the quality of the and real complexity, especially the rules and data applied for the computation of real complexity. Based on the discussions evolved with the BP workers, both real complexity and threshold parameters and rule sets were adjusted. All the presented inline and in tables below parameters and rule sets as well as evaluation numbers (see Table 5) are based on the obtained final values. Third, in order to assess the practical implications of the and RS, we conducted a short Q&A session using a so-called funnel model [ 23 ], i.e. we started with open questions and moved towards more specific ones regarding possible practical value of the RS. Hereby, not only providing “physical” recommendations in a form of templates or historical ticket data received a positive feedback but also the prioritization of an incoming ticket as a dashboard for correct time and workforce management in the team. In Table 4, we present the example of a manually selected ticket. According to the algorithm described in Section 3.2., the predicted is low and recommendation would be , i.e. one-to-one template from the database. : “telegraphic” ( ) installation, release, application, SAP XYZ

(∃! ) installation, release, application, SAP XYZ b 0 : “medium” low ( )

medium ( ) installation high ( )

: “routine”

installation, release, application index: “low” Recommendation : template BPC cog cog rout rout semi-cog semi-cog

PPC

In addition to the handcrafted rules for real complexity developed with the case study team and in order to be able to compare the evaluation results, we applied a technology based approach – the recursive partitioning classification and regression trees (CART) method [ 18 ] with complexity parameter cp=0.056 and measures of the error in classification xerror=0.39. For this purpose, we used the mentioned set of manually evaluated 60 IT tickets as a training sample and data set of 4,625 tickets as a test sample. Hereby, the distribution values show the qualitative characteristic of the data set on the total, i.e. what is the proportion of the BPs with low, medium and high complexity. Overall precision is the relative number of correctly identified PPC as compared to the whole number of identified real complexity. Recalls are calculated for each of the PPC values and represent a fraction of relevant values that have been retrieved over the total amount of relevant values. As it can be concluded from the table, the values from both approaches reveal similar evaluation results, the CART-based method showing slightly higher (0.5% increase) precision and better recalls in case of low (1.7% increase) and high (9.5% increase) values. 5

Limitations and Future Work

In this paper, we presented a multi-criteria knowledge-based RS approach, which exploits three core knowledge aspects of the BP textual descriptions to build a recommendation. The main contributions of this work are a construction of a set of criteria for a recommendation problem in the context of unstructured BP texts and provision of a method to efficiently extract the necessary knowledge aspects and transform them into actionable insights, representing a methodological guide for BP decision support. As shown in the experiments, the conceptual framework has proven to be a meaningful approach having obtained positive quantitative and qualitative evaluation results. The main limitations are related to: 1) testing of the approach in the real environment of the same case study, 2) applying of the framework on the case study from a different domain and 3) currently strong focus on the empirical handcrafted rules, i.e. absence of a “learning” component of the RS. As a part of future work, we will encode the algorithms to build a proof-of-concept of the suggested multi-criteria knowledge-based RS. Subsequently, the prototype will be evaluated on the case study data set and by the BP workers. In parallel, we will search for a case study from a different domain to test the framework.

References

[1] Gediminas Adomavicius , Nikos Manouselis, and YoungOk Kwon . 2015 . MultiCriteria Recommender Systems . In Recommender Systems Handbook, Francesco Ricci, Lior Rokach and Bracha Shapira (Eds.) , Springer, New York, 847 - 880 . Electronic ISBN: 978 -1- 4899 -7637-6.

[2] Zhifeng Bao , Yong Zeng, H.V. Jagadish , and Tok Wang Ling. 2015 . Exploratory Keyword Search with Interactive Input . In SIGMOD Proceedings. ACM, Melbourne , 871 - 876 .

[3] Allison J.B. Chaney , Brandon M. Stewart , and Barbara E. Engelhardt . 2018 . How Algorithmic Confounding in Recommendation Systems Increases Homogeneity and Decreases Utility . In RecSys '18 Proceedings of the 12th ACM Conference on Recommender Systems. ACM , Vancouver, 224 - 232 . DOI: 10 .1145/3240323.3240370.

[4] Robert Chi and Melody Kiang . 1991 . An Integrated Approach of Rule-based and Case-based Reasoning for Decision Support . ACM, 255 - 267 . DOI: 10 .1145/327164.327272.

[5] Joel Coffman and Alfred C. Weaver . 2010 . Framework for Evaluating Database Keyword Search Strategie . In CIKM Proceedings. Toronto , 729 - 738 . DOI: 10 .1145/1871437.1871531.

[6] Daelemans Walter . 2013 . Explanation in Computational Stylometry . In International Conference on Intelligent Text Processing and Computational Linguistics . Springer, Samos, 451 - 462 . DOI: 10 .1007/978-3- 642 -37256-8_ 37 .

[7] Alexander Felfernig and Robin Burke . 2008 . Constraint-based Recommender Systems: Technologies and Research Issues . In 10th International Conference on Electronic Commerce (ICEC) . ICEC, Innsbruck. DOI: 10.1145/1409540 .1409544.

[8] Alexander Felfernig , Gerhard Friedrich, Dietmar Jannach, and Markus Zanker . 2011 . Developing Constraint-based Recommenders . In Recommender Systems Handbook , Francesco Ricci, Lior Rokach, Bracha Shapira, Paul B. Kantor (Eds.) , Springer, New York, 187 - 215 . DOI: 10 .1007/978-0- 387 -85820-3.

[9] Crina Grosan and Ajith Abraham . 2011 . Rule-Based Expert Systems . Intelligent Systems Reference Library , Springer, 149 - 185 .

[10] Ferdaous Hdioud , Bouchra Frikh, and Brahim Ouhbi . 2013 . Multi-Criteria Recommender Systems based on Multi-Attribute Decision Making . In International Conference on Information Integration and Web-based Applications & Services. ACM , Vienna, 203 - 210 .

[11] ITIL® Service Transition . 2011 . TSO, London.

[12] Conrad J. Jacoby . 2012 . Understanding the Limitations of Keyword Search . White Paper, Equivio.

[13] Ralph L. Keeney . 1992 . Value-Focused Thinking. A Path to Creative DecisionMaking . Harvard University Press, Cambridge MA.

[14] Jie Lu , Dianshuang Wu, Mingsong Mao, Wei Wang , and Guangquan Zhang . 2015 . Recommender System Application Developments: A Survey. Decision Support Systems ( 2015 ), 12 - 32 . DOI: 10 .1016/j.dss. 2015 . 03 .008.

[15] Nikos Manouselis and Constantina Costopoulou . 2007 . Analysis and Classification of Multi-Criteria Recommender Systems . World Wide Web: Internet and Web Information Systems, Special Issue on Multi-channel Adaptive Information Systems on the World Wide Web , 1 - 27 .

[16] Hien Nguyen and Peter Haddawy . 1999 . DIVA: Applying Decision Theory to Collaborative Filtering . In AAAI Workshop on Recommender Systems. AAAI, Madison , 20 - 26 .

[17] OMG . 2013 . Business Process Model and Notation (BPMN) - Version 2.0.2 . Object Management Group, Needham.

[18] Vili Podgorelec , Peter Kokol , Bruno Stiglic, and Ivan Rozman . 2005 . Decision Trees: An Overview and Their Use in Medicine . Journal of Medical Systems 26 ( 5 ), 445 - 463 . DOI: 10 .1023/A: 1016409317640 .

[19] Nina Rizun , Aleksandra Revina and Vera G. Meister . 2019 . Discovery of Stylistic Patterns in Business Process Textual Descriptions: IT Ticket Case . In 33rd International Business Information Management Association Conference (IBIMA) . Web of Science , Granada.

[20] Nina Rizun , Aleksandra Revina and Vera G. Meister . 2019 . Method of DecisionMaking Logic Discovery in the Business Process Textual Data . In 22nd International Conference on Business Information Systems . Springer, Seville, 70 - 84 . DOI: 10 .1007/978-3- 030 -20485- 3 _ 6

[21] Nina Rizun , Aleksandra Revina and Vera G. Meister. 2019. Business Sentiment . Concept and Method for Perceived Anticipated Effort Identification . In 28th International Conference of Information Systems Development. ACM , Toulon.

[22] Bernard Roy . 1996 . Multicriteria Methodology Goes Decision Aiding . Springer US, New York. DOI: 10 .1007/978-1- 4757 -2500-1.

[23] Per Runeson and Martin Höst . 2009 . Guidelines for Conducting and Reporting Case Study Research in Software Engineering . Empirical Software Engineering 14 ( 2 ), 131 - 164 . DOI: 10 .1007/s10664-008-9102-8.

[24] Thomas L. Saaty , 2005 . The Analytic Hierarchy and Analytic Network Processes for the Measurement of Intangible Criteria and for Decision-Making. In Multiple Criteria Decision Analysis: State of the Art Surveys , Salvatore Greco (Ed.), Springer, New York, 345 - 405 . DOI: 10 .1007/b100605.

[25] Ralph Schäfer . 2001 . Rules for Using Multi-Attribute Utility Theory for Estimating a User's Interests . In ABIS Workshop Adaptivität und Benutzermodellierung in interaktiven Softwaresystemen. DFKI, Dortmund , 1 - 7 .

[26] Markus Schäfermeyer , Christoph Rosenkranz, and Roland Holten . 2012 . The Impact of Business Process Complexity on Business Process Standardization . Business & Information Systems Engineering , 261 - 270 . DOI: 10 .1007/s12599- 012-0224-6.

[27] Scorobey . 2017 . A Simple Program on Python for Hyperbolic Approximation of Statistical Data . In Russian. https://m.habr.com/ru/post/322954/.

[28] Yi Shan . 2016 . Semantic Keyword Search on Large-Scale Semi-Structured Data . Dissertation, Arizona State University, Arizona.

[29] Sam Wanekeya . 2018 . Can't Stand The Increasing Complexity Of IT Infrastructure , AI Is Here To Help. Headwaynews. https://www.headwaynews. org/cant-stand-the-increasing-complexity-of-itinfrastructure-ai-is-here-to-help/.

[30] Zhong Zeng , Zhifeng Bao, Tok Wang Ling, and Mong Li Lee . 2012 . iSearch: An Interpretation Based Framework for Keyword Search in Relational Databases . In Proceedings of the 3rd International Workshop on Keyword Search on Structured Data (KEYS 2012 ). ACM, Arizona, 3 - 10 .

[31] George Kingsley Zipf . 1949 . Human Behaviour and The Principle of Least Effort . Addison-Wesley, Reading.

[32] George Kingsley Zipf . 1932 . Selected Studies of the Principle of Relative Frequency in Language . Harvard University Press, Cambridge.