Multi-Criteria Knowledge-Based Recommender System for Decision Support in Complex Business Processes Aleksandra Revina† Nina Rizun Information and Communication Management Applied Informatics in Management Technical University of Berlin Gdansk University of Technology Berlin, Germany Gdansk, Poland revina@tu-berlin.de nina.rizun@pg.edu.pl ABSTRACT recommendation quality and its utility are still in discussion [3]. To address the mentioned challenges of growing BP complexity on the In this paper, we present a concept of a multi-criteria knowledge- one side and lack of recommendation quality in the knowledge- based Recommender System (RS) designed to provide decision based RS on the other, we suggest a concept of a multi-criteria support in complex business process (BP) scenarios. The developed knowledge-based RS that aims to predict the BP complexity based approach is based on the knowledge aspects of Stylistic Patterns, on the input in a form of unstructured BP textual request. Business Sentiment and Decision-Making Logic extracted from the Approaches from such subject areas as Applied Linguistics, BP unstructured texts. This knowledge serves as an input for a Stylistics, Sentiment Analysis, and Taxonomies are used to extract multi-criteria RS algorithm. The output is prediction of the BP relevant knowledge aspects out of the BP textual data. An IT complexity, based on which the algorithm modifies the type and the ticketing process from an ITIL-based Change Management (CHM) way of decision support, ranging from full to minimal automation. area [11] is taken as the case study of the research. We show how the algorithm can be applied in the real-life scenarios by the example of the IT ticketing case study. We also evaluate the BP complexity prediction quality using both quantitative (data- 2 Related Work based) and qualitative (interview-based) approach in the case study. Multi-criteria RS are based on the well-known Multi-Criteria Decision Making (MCDM) methods [1, 22]. The value of multi- CCS CONCEPTS criteria recommendation approach in general and the MCDM • Information systems → Information systems applications; methods in particular has been demonstrated long ago and in Decision Support Systems; Data Analytics various application domains [15, 16, 25]. At present, one of the most popular categories has proven to be multi-criteria rating KEYWORDS recommenders, which though suffer from a number of problems, e.g. constructing the best set of criteria [1]. In regards to Recommendation, Business Process Management, Complexity, knowledge-based recommenders, one differentiates two types: Stylistics, Linguistics, Sentiment, IT Tickets case-based and constraint-based. Constraint-based RS exploit the predefined knowledge bases with the explicit rules of delivering the 1 Introduction recommendation and are considered to perform well, specifically in complex product domains [7]. Rule-based reasoning and rule- With the considerable technology progress and enterprise based expert systems have long been a focus of research on digitization, the discussions around the timeworn term of intelligent systems [4, 9]. Currently, they find another complexity gain new power. Especially businesses and their IT advantageous practical application as a part of constraint-based RS. departments report a dramatic increase in the process complexity Knowledge-based RS provide a major value in overcoming such [29]. In this context, a BP must have a certain level of complexity limitations as lack of transparency, cold-start problem and data to correspond with the complexity of its environment. Thus, the sparsity, which are common for content-based and collaborative complexity can be challenged and caused by both complex BP IT filtering approaches. However, acquiring the necessary knowledge environment and constantly increasing information flow to be possessed by domain experts and converting it into formal, handled in the BP [26]. It is widely acknowledged that RS open executable representations is a challenging task [8]. wide opportunities for different domains and particularly Thus, the contributions of the paper can be highlighted in the businesses. Hereby, the main characteristic of RS e-business following: 1) construction of a set of criteria for a recommendation applications is an intensive use of the knowledge-based RS problem in the context of unstructured BP texts, which is an approaches, i.e. ontologies and semantic technologies. This can be important topic for future research in multi-criteria RS [1] and 2) explained by the fact that businesses demand a high degree of provision of a method to efficiently extract the necessary domain knowledge for adequate assistance in recommendations [14]. Hereby, the main RS challenges of robustness, ComplexRec 2019, 20 September 2019, Copenhagen, Denmark 2019. Copyright © 2019 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). ComplexRec’19, September 20, 2019, Copenhagen, Denmark Revina and Rizun knowledge aspects and transform them into executable system, the requested change is implemented. The goal of the RS representations targeting the problem described above [8]. concept is to address the problems described in 2), i.e. incorrect search results which imply inefficient work and time loss. While remaining an important starting point, key word search must be 3 Concept of Multi-Criteria Knowledge-Based RS viewed as only one of several tools supporting the BP workers, A typical case study scenario from an ITIL-based CHM IT ticketing especially in the context of key word search commonly known process considered during the research is the following: 1) a limitations [12]. On the user side, key word search is known for a customer request (ticket) for a change in IT infrastructure products constant need to reformulate the queries, no possibility to precisely or services is sent per e-mail; 2) requested changes can be processed specify the search intention and limited knowledge on or with various templates (pre-filled forms). Ideally, tickets availability of the data to precisely express the search intention [2]. addressing related problems are processed with the same template. On the key word search technology side, most existing solutions However, key word search used at the case study department focus on small datasets [30, 28] and efficiency instead of search doesn’t yield relevant results. Thus, a new template is likely to be quality [5, 28]. With the proposed RS, it is aimed to support a BP created both in case of a new type of request and when the template worker in finding the most successful way to process the request is not found; 3) based on the information documented in a ticketing under given conditions, i.e. incoming ticket text. Figure 1: BPMN Model of Multi-Criteria Knowledge-Based RS in CHM IT Ticket Processing The RS modelled in BPMN [17] (see Figure 1) should, first, perceived while reading the ticket and b) real complexity reported support the BP worker in ticket prioritization and, second, adapt the after the ticket is processed; 2) ticket implementation complexity type and the way of recommendation based on the complexity level related to the technical execution of the ticket related tasks. The of the ticket text, i.e. perceived processing complexity ( 𝑃𝑃𝐶 ), scope of the proposed RS is targeted at 1a. At the moment of the IT identified with the help of multi-criteria knowledge aspects, i.e. ticket entry, the BP worker receives the textual description of the Readability ( 𝑅𝐸 ), Perceived Anticipated Effort ( 𝑃𝐴𝐸 ) and request characterized by the following parameters influencing the Business Process Cognition (𝐵𝑃𝐶) (see Section 3.1). Hereby, it is perception of request processing complexity: quality of the written important to note that the 𝑃𝑃𝐶 computation yields to the three text (comprehension of the request), urgency of the request and levels of “low”, “medium” and “high”. This scale was selected for type of the requested activity. According to these factors, two reasons: 1) in order to simplify the method presentation and 2) corresponding criteria and measures were selected in the scope of it is a known scale of priority ratings especially for measuring the present RS: quality of the written text measured by 𝑅𝐸, urgency intangible criteria in the context of decision-making [24]. Tickets – by 𝑃𝐴𝐸 and type of the activity – by 𝐵𝑃𝐶. with 𝑃𝑃𝐶 “low” can be described with clear rules and easily automated by one-to-one template recommendation. Tickets of 𝑃𝑃𝐶 “medium” are those where no exact rule set exists and there is 3.1 Conceptual Framework a need of information acquisition and evaluation. Here, the RS can In the context of the present research, we refer to the provide a partial processing support in a form of drop-down menu recommendation problem as an MCDM problem and use the templates (multiple-choice recommendations). In case of 𝑃𝑃𝐶 conceptual notation by [22]. Accordingly, we specify the RS “high”, the RS will offer a minimal assistance while listing the concepts for the present research as follows below. history of similar implemented tickets. Defining the object of decision. Object of decision is item 𝑖 that In a general IT ticket context, one can differentiate between belongs to the set of all candidate items. In the case study of the three types of complexity: 1) ticket processing complexity a) Multi-Criteria Knowledge-Based Recommender System ComplexRec’19, September 20, 2019, Copenhagen, Denmark research, the objects of the decision 𝑂 are classified into three neutral valences σ(𝑥𝑝𝑜𝑠 , 𝑥𝑛𝑒𝑢𝑡 , 𝑥𝑛𝑒𝑔 ), where 𝑥𝑝𝑜𝑠 , 𝑥𝑛𝑒𝑢𝑡 , 𝑥𝑛𝑒𝑔 are categories based on the identified 𝑃𝑃𝐶 : 1) one-to-one ticket words with the corresponding valence of positive, neutral or templates 𝑀𝑒 where 𝑒 is a number of the ticket template in the negative. database; 2) drop-down menu templates 𝐿𝑓 where 𝑓 is the number The third criterion 𝑐3 is suggested to be Business Process of the drop-down menu suggestion; 3) similar tickets in the Cognition (𝐵𝑃𝐶) measured by semantic nature of activities in the database history 𝐻𝑔 where 𝑔 is the number of the ticket record in ticket identified with Decision-Making Logic (DML) Taxonomy the database. The elements of this set are specified as alternatives (objective knowledge) [20]. DML Taxonomy is built while to which four types of decision problems (choice, sorting, ranking, extracting semantically loaded 𝑥𝑃𝑜𝑆 and calculating their relative and description) can be applied [22]. In the current research, we distributions in 𝑇𝑔 σ(𝑥𝑟 , 𝑥𝑠𝑐 , 𝑥𝑐 ). 𝑥𝑟 , 𝑥𝑠𝑐 , 𝑥𝑐 are DML elements refer to sorting (classification of alternatives into a number of pre- (words) indicating routine, semi-cognitive and cognitive activities defined three categories) and choice (selection of a more organized as PoS according to RTCC Framework (nouns (𝑛) as appropriate alternative). To sum up, O ∈ {𝑀𝑒, 𝐿𝑓 , 𝐻𝑔 }. Resources, verbs (𝑣) as Techniques, adjectives (𝑎𝑑𝑗) as Capacities, Family of criteria. Performance fit of alternatives is analyzed upon adverbs (𝑎𝑑𝑣) as Choices) into three classes of routine (r), semi- a set of criteria. In the paper, fit of alternatives from the three cognitive (sc) and cognitive (c). categories mentioned above is evaluated upon a set of criteria for Global preference (recommendation) model. The development of a each incoming ticket text 𝑇𝑔 . As fairly stated by [22], the design of global preference model provides a way to aggregate the values of a consistent family of criteria for a given recommendation each criterion 𝐶 = {𝑐1 , 𝑐2 , 𝑐3 } in order to express the preferences application has been largely ignored in the RS literature and between the alternatives. In the paper, the most established constitutes an important problem for future research. Here, a family approach of a value-focused model is pursued [1]. Marginal of three measurable quantitative-qualitative criteria 𝐶 = {𝑐1 , 𝑐2 , preferences upon each criterion are synthesized into a total value 𝑐3 } is applied on the 𝑇𝑔 in order to predict and generate a function, also known as utility function [13]. The utility-based recommendation 𝑅 (see also Section 3.2 for more details). The formulation of the multi-criteria recommendation problem in the choice of the criteria and especially corresponding measures is present research is formulated with the help of context dependent justified by the textual nature of the input data. As the unstructured rule sets which determine the meaningfulness or the weight of each textual BP requests serve as the basis for recommendation, the criterion in the specific context (see Section 3.2 and 4 for more technologies used for criteria extraction come from the domains of details). Applied Linguistics, Stylistics, Sentiment Analysis, and Taxonomies. The approaches have been selected based on and 3.2 Case Study Application therefore are covering the three common levels of text Based on the qualitative interviews and literature reviews, the understanding: objective (answering the who, what, where, when, following assumptions are introduced: 1) ticket length 𝐿 is etc. questions, e.g. taxonomies and ontologies), subjective (who has accepted as a parameter indicating 𝑃𝑃𝐶. We discovered while which opinion about what, e.g. Sentiment Analysis) and performing the survey that case study BP workers usually receive metaknowledge (what can be extracted about the text apart from its short texts in case of simple, explicit and already familiar requests. contents, e.g. with Stylistics or Stylometry) [6]. To a certain extent, this fact is also supported by the theory of the Thus, the first criterion 𝑐1 is suggested to be Readability 𝑅𝐸 least effort [31]. Based on the case study contextual specificity measured by Stylistic Patterns (SP) [19]. SP of ticket texts are calculated with the help of statistical analysis, a threshold 𝑚 has to considered to influence the BP worker’s perception of the be set; 2) the distribution σ of PoS has a direct impact on contextual contextual complexity of the ticket processing and express the readability. Information in the tickets rich in unique nouns (BP quality of the written text affecting the understanding of the request Resources) and with low number of other PoS (for example, BP (metaknowledge). In the present RS concept, SP are defined as a Techniques) is easy to perceive and systemize for a BP worker; 3) function of Syntactic Structure (SynS) and Wording Style (WS) for in case of word frequencies (Zipf's coefficient 𝑏), a threshold 𝑞 has the different length values 𝐿 of the BP text 𝑇𝑔 . Hereby, SynS is a to be set; 4) while implementing the approaches 𝑃𝐴𝐸 , 𝐵𝑃𝐶 syntactic structure of text 𝑇𝑔 calculated as relative distributions σ and 𝑃𝑃𝐶 , the rule sets 𝑅𝑈 ∈ {𝑅𝑈1 , 𝑅𝑈2 }, {𝑅𝑈3 , 𝑅𝑈4 , 𝑅𝑈5 } and of 𝑥𝑃𝑜𝑆 and unique 𝑥𝑃𝑜𝑆 , where 𝑥𝑃𝑜𝑆 are words organized as per {𝑅𝑈6 , 𝑅𝑈7 , 𝑅𝑈8 } have to be developed based on the specific part of speech (PoS) of nouns, verbs, adjectives, and adverbs. WS statistical values of the case study in focus. First, we describe the is the wording style of 𝑇𝑔 text bringing in relation rank-frequency extraction and interpretation of the knowledge aspects related to the and quantity-frequency of words [32] in 𝑇𝑔 approximated with three suggested criteria. After, we show how the extracted aspects 𝑏 and related criteria are used to feed the RS. coefficients 𝑎 and b in a form of (𝑎 + ) [27]. 𝑥 Readability (𝑐1 ). There are certain Stylistic Patterns (SP) embedded The second criterion 𝑐2 is suggested to be Perceived in the BP (ticket) texts influencing the worker’s perception of the Anticipated Effort (𝑃𝐴𝐸) measured by Business Sentiment (BS) contextual complexity of the task processing [19]. It is proposed to representing emotional component of ticket complexity or also measure the SP with relative distributions of PoS and unique PoS urgency of the request (subjective knowledge) [21]. BS is (SynS) and Zipf’s word frequencies (WS). calculated based on the lexicon approach with the help of relative distributions of identified BS-loaded PoS of negative, positive and ComplexRec’19, September 20, 2019, Copenhagen, Denmark Revina and Rizun Input: incoming ticket 𝑇𝑔 with 𝑥𝑃𝑜𝑆 ( 𝑛 , 𝑣 , 𝑎𝑑𝑗 , 𝑎𝑑𝑣 ), accepted the BPs (tickets) into three categories of routine, semi-cognitive threshold 𝑚 for the ticket length 𝐿 and accepted threshold 𝑞 for and cognitive based on the semantically implied complexity [20]. coefficient 𝑏 in the corpus 𝐷 Input: incoming ticket 𝑇𝑔 in the corpus 𝐷, manually created DML Output: exclusive qualitative values of 𝑐1 “telegraphic”, Taxonomy from 𝐷 with 𝑥𝑟 , 𝑥𝑠𝑐 , 𝑥𝑐 organized as PoS in RTTC “effortless” and “involving effort” Framework [20], case study specific rule set 𝑅𝑈 ∈ for all 𝑥𝑃𝑜𝑆 ∈ 𝑇𝑔 do {𝑅𝑈3 , 𝑅𝑈4 , 𝑅𝑈5 } if 𝐿<𝑚 and σ(𝑛)>0 and σ(𝑣, 𝑎𝑑𝑗, 𝑎𝑑𝑣)=0 and b=0 then Output: 𝑐3 exclusive qualitative values “routine”, “semi- 𝑐1 =“telegraphic” cognitive”, “cognitive” if 𝐿< 𝑚 and σ(𝑛, 𝑣)>0 and σ(𝑛)>σ(𝑣, 𝑎𝑑𝑗, 𝑎𝑑𝑣) and for all 𝑥𝑟 , 𝑥𝑠𝑐 , 𝑥𝑐 ∈ 𝑇𝑔 do σ(𝑛)>σ(∃! 𝑛) and 𝑏< 𝑞 then 𝑐1 =“effortless” if σ(𝑥𝑟 , 𝑥𝑠𝑐 , 𝑥𝑐 ) = 𝑅𝑈3 then 𝑐3 =“cognitive” else 𝑐1 =“involving effort” if σ(𝑥𝑟 , 𝑥𝑠𝑐 , 𝑥𝑐 ) = 𝑅𝑈4 then 𝑐3 =“routine” end if σ(𝑥𝑟 , 𝑥𝑠𝑐 , 𝑥𝑐 )= 𝑅𝑈5 then 𝑐3 =“semi-cognitive” The algorithm considers that: 1) 𝑃𝑃𝐶 depends on 𝐿, short tickets end being the simple ones; 2) the tickets containing only nouns are The algorithm follows semantic tagging approach which classifies written in a very condensed telegraphic way, i.e. either BP worker the activities described in tickets into three pre-defined categories. already knows what needs to be done or the ticket is complex and Multi-Criteria Recommendations. Computed criteria values and this complexity will be captured with criteria 𝑐2 or 𝑐3 depending on inferred 𝑃𝑃𝐶 are used to feed multi-criteria knowledge-based RS. their meaningfulness in the case study context; 3) ticket texts Based on 𝑃𝑃𝐶 , the recommendation 𝑅 from ∈ {𝑀𝑒 , 𝐿𝑓 , 𝐻𝑔 } containing high relative number of BP Resources (nouns), which alternatives should be offered to the BP worker. are also unique, are easy to understand. The WS (𝑏) indicates the Input: computed qualitative values for 𝑐1 (𝑇𝑔 ), 𝑐2 (𝑇𝑔 ), 𝑐3 (𝑇𝑔 ), the information presentation flow, i.e. condensed versus disperse. case study specific rule sets determining the meaningfulness or Perceived Anticipated Effort ( 𝑐2 ). 𝑃𝐴𝐸 reflects the emotional weight of each criterion in the case study context 𝑅𝑈 ∈ component of the ticket contextual complexity perceived by the BP {𝑅𝑈6 , 𝑅𝑈7 , 𝑅𝑈8 } worker while reading the ticket text [21]. It is proposed to be Output: 𝑃𝑃𝐶 and a recommendation for the BP worker measured by the specified Business Sentiment. for 𝑐1 (𝑇𝑔 ), 𝑐2 (𝑇𝑔 ), 𝑐3 (𝑇𝑔 ) do Input: incoming ticket 𝑇𝑔 in the corpus 𝐷, manually created BS if 𝑐1 ,𝑐2 ,𝑐3 =𝑅𝑈6 then 𝑃𝑃𝐶 = “low” and 𝑅 = 𝑀𝑒 lexicon-computed valence values of 𝑥𝑝𝑜𝑠 , 𝑥𝑛𝑒𝑢𝑡 , 𝑥𝑛𝑒𝑔 , case study if 𝑐1 ,𝑐2 ,𝑐3 =𝑅𝑈7 then 𝑃𝑃𝐶 = “medium” and 𝑅 = 𝐿𝑓 specific rule set 𝑅𝑈 ∈ { 𝑅𝑈1 , 𝑅𝑈2 } [21] if 𝑐1 ,𝑐2 , 𝑐3 =𝑅𝑈8 then 𝑃𝑃𝐶=“high” and Output: 𝑐2 exclusive qualitative values “low”, “medium”, “high” 𝑅={𝐿𝑓 , 𝐻𝑔 } for all 𝑥𝑝𝑜𝑠 , 𝑥𝑛𝑒𝑢𝑡 , 𝑥𝑛𝑒𝑔 ∈ 𝑇𝑔 do end if σ(𝑥𝑝𝑜𝑠 , 𝑥𝑛𝑒𝑢𝑡 , 𝑥𝑛𝑒𝑔 )=𝑅𝑈1 then 𝑐2 =“low” In the experimental session, we evaluated the knowledge aspects if σ(𝑥𝑝𝑜𝑠 , 𝑥𝑛𝑒𝑢𝑡 , 𝑥𝑛𝑒𝑔 )=𝑅𝑈2 then 𝑐2 =“medium” extraction according to 𝐶 = {𝑐1 , 𝑐2 , 𝑐3 } on the case study data set else 𝑐2 =“high” and calculated case study specific threshold parameters and rule end sets which were iteratively adjusted based on the computed 𝑃𝑃𝐶 The algorithm reproduces the computation of the emotional and its quantitative and qualitative evaluation. These values and an component of the BP contextual complexity expressed by urgency experimental set-up of the proposed RS on the example of a and task complexity. randomly selected ticket are presented in the section below. Business Process Cognition ( 𝑐3 ). The algorithm presents the identification of semantic nature of activities in the ticket texts by means of DML Taxonomy. The knowledge aspects are extracted 4 Experiments and Evaluation with the help of the mentioned RTCC Framework whereby nouns In the experimental and evaluation phase, we conducted (𝑛) express Resources, verbs (𝑣) – Techniques, adjectives (𝑎𝑑𝑗) – quantitative (experiments) and qualitative (interviews) analyses as Capacities, and adverbs (𝑎𝑑𝑣) – Choices. It is suggested to classify shown on the Figure 2 below. Multi-Criteria Knowledge-Based Recommender System ComplexRec’19, September 20, 2019, Copenhagen, Denmark Figure 2: Experiments and Evaluation of 𝑷𝑷𝑪 First (see point 1 on Figure 2), initial experiments were carried out this purpose, a semi-structured interview approach was developed in order to set up initial values of case study threshold parameters with a planned set of questions regarding the feasibility and & rule sets. The computational analyses were conducted based on applicability of the 𝑃𝑃𝐶 computation and the development of the pre-processed data set comprising CSV-formatted 28,157 text recommendations based on the 𝑃𝑃𝐶. entries (tickets) in English language. The approaches of specified The qualitative evaluation was divided into three parts. First, knowledge aspects extraction were executed on the data set we introduced the objectives, research motivations, theoretical and subsequently. Inline and in the tables below, we present the final methodological background. Second, the RS concept, specifically values for the threshold parameters and rule sets obtained after the the 𝑃𝑃𝐶 computation, was illustratively presented using a set of 60 evaluation rounds described in this section: 1) accepted threshold randomly selected IT tickets containing 54% of correctly and 46% 𝑚 for the ticket length 𝐿 – 25 words (𝑥); 2) accepted threshold 𝑞 of incorrectly identified 𝑃𝑃𝐶 from the case study data set. The for coefficient 𝑏 – 3; 3) accepted rule set 𝑅𝑈 ∈ { 𝑅𝑈1 , 𝑅𝑈2 } for estimation of correctness was performed using the computed real 𝑃𝐴𝐸 computation is presented in Table 1; 4) accepted rule set complexity values. The case study BP workers were asked to 𝑅𝑈 ∈ { 𝑅𝑈3 , 𝑅𝑈4 , 𝑅𝑈5 } for 𝐵𝑃𝐶 computation is presented in Table critically evaluate the quality of the 𝑃𝑃𝐶 and real complexity, 2; 5) accepted rule set 𝑅𝑈 ∈ {𝑅𝑈6 , 𝑅𝑈7 , 𝑅𝑈8 } of 𝑃𝑃𝐶 is presented especially the rules and data applied for the computation of real in Table 3 (the values in each of the cell of the table represent complexity. Based on the discussions evolved with the BP workers, possible alternatives). both real complexity and 𝑃𝑃𝐶 threshold parameters and rule sets In the evaluation phase (see point 2 and 3 on Figure 2), we were adjusted. All the presented inline and in tables below 𝑃𝑃𝐶 conducted quantitative and qualitative analyses iteratively in order parameters and rule sets as well as evaluation numbers (see Table to fine tune the threshold parameters and rule sets from point 1 on 5) are based on the obtained final values. Third, in order to assess Figure 2. While discussing the 𝑃𝑃𝐶 with the case study BP the practical implications of the 𝑃𝑃𝐶 and RS, we conducted a short workers, it was discovered that there is no such a complexity Q&A session using a so-called funnel model [23], i.e. we started definition as 𝑃𝑃𝐶 in the current case study context. However, with open questions and moved towards more specific ones another type of complexity (real complexity of the ticket processing regarding possible practical value of the RS. Hereby, not only mentioned in 1b, see Section 3.1.) can be measured based on the providing “physical” recommendations in a form of templates or historical ticket data from the IT ticketing system. These data historical ticket data received a positive feedback but also the included configuration items, specifically affected applications prioritization of an incoming ticket as a dashboard for correct time (which is closely related to the number of tasks in the case study and workforce management in the team. context), number of tasks, risk type of the ticket, and implementation type (online vs offline). Real complexity can be Table 1: 𝑷𝑨𝑬 Computation Rules [21] calculated on the ordinal scale yielding to the values of “low”, “medium” and “high”, those applied in the 𝑃𝑃𝐶 computation, and # Compound Valence PAE positive (pos), neutral (neut), negative (neg) thus can be used for the evaluation of 𝑃𝑃𝐶. 𝑅𝑈1 Consequently, quantitative analysis with a new data set from 1 pos>0.2 neut>2*abs(neg) 00 neut=0 neg=0 low was performed to compute the 𝑃𝑃𝐶 of each of the ticket (see point 3 pos>2*neut neut>0 neg=0 low 4 unrecognized low 2 on Figure 2). To compute real complexity, we used mentioned 𝑅𝑈2 historical data from the IT ticketing system. Following the rules 5 pos=0 neut=1, neut=0 neg=0 medium provided by the case study BP workers, we calculated the real 6 pos>0 neut>0 neg=0 medium complexity for each of the ticket also classifying it into “low”, 7 pos>0 neut>0 00.3 cog the error in classification xerror=0.39. For this purpose, we used 𝑅𝑈4 the mentioned set of manually evaluated 60 IT tickets as a training 3 (rout=1) & (rout=0) semi-cog=0 cog=0 rout sample and data set of 4,625 tickets as a test sample. 4 rout>0.5 (semi-cog+cog)≤0.3 rout 𝑅𝑈5 5 rout=0 semi-cog=1 cog=0 semi-cog Table 5: Evaluation Statistics Based on Handcrafted and 6 rout=0 semi-cog=0 cog>0.3 semi-cog Technology Based Rules Table 3: 𝑷𝑷𝑪 Computation Rules low medium high PPC distribution 52.36% 31.7% 15.94% # 𝑐1 𝑅𝐸 𝑐2 𝑃𝐴𝐸 𝑐3 𝐵𝑃𝐶 PPC handcrafted rules 𝑅𝑈6 Real complexity distribution 87.22% 8.31% 4.46% 1 effortless low, medium, high rout low Overall precision 61.75% 2 involving effort low, medium rout low Recalls 73.9% 71.9% 40.7% 3 effortless low semi-cog low technology (CART) based rules 4 telegraphic - rout low Real complexity distribution 70.49% 11.48% 18.03% 𝑅𝑈7 Overall precision 62.27% 5 involving effort high rout medium Recalls 75.6% 61.6% 50.2% 6 effortless low cog medium 7 involving effort low semi-cog, cog medium Hereby, the distribution values show the qualitative characteristic 8 effortless medium, high semi-cog, cog medium 9 telegraphic - semi-cog medium of the data set on the total, i.e. what is the proportion of the BPs 𝑅𝑈8 with low, medium and high complexity. Overall precision is the 10 involving effort medium, high semi-cog, cog high relative number of correctly identified PPC as compared to the 11 telegraphic - cog high whole number of identified real complexity. Recalls are calculated for each of the PPC values and represent a fraction of relevant In Table 4, we present the example of a manually selected ticket. values that have been retrieved over the total amount of relevant According to the algorithm described in Section 3.2., the predicted values. As it can be concluded from the table, the values from both 𝑃𝑃𝐶 is low and recommendation 𝑅 would be 𝑀𝑒 , i.e. one-to-one approaches reveal similar evaluation results, the CART-based template from the database. method showing slightly higher (0.5% increase) precision and better recalls in case of low (1.7% increase) and high (9.5% Table 4: Multi-Criteria Knowledge-Based RS Approach on increase) values. Anonymized Ticket Example 5 Limitations and Future Work IT ticket text: "Installation of Release 001.296.01 for the application Length SAP XYZ." L=9 In this paper, we presented a multi-criteria knowledge-based RS 𝒄𝟏 𝑹𝑬: “telegraphic” approach, which exploits three core knowledge aspects of the BP 𝑥𝑃𝑜𝑆 (𝑛) count: textual descriptions to build a recommendation. The main installation, release, application, SAP XYZ 4 𝑥𝑃𝑜𝑆 (∃! 𝑛) contributions of this work are a construction of a set of criteria for count: 4 installation, release, application, SAP XYZ a recommendation problem in the context of unstructured BP texts b count: and provision of a method to efficiently extract the necessary 0 - knowledge aspects and transform them into actionable insights, 𝒄𝟐 𝑷𝑨𝑬: “medium” low (𝑥𝑝𝑜𝑠 ) count: representing a methodological guide for BP decision support. As - 0 shown in the experiments, the conceptual framework has proven to medium (𝑥𝑛𝑒𝑢𝑡 ) count: be a meaningful approach having obtained positive quantitative and installation 1 qualitative evaluation results. The main limitations are related to: high (𝑥𝑛𝑒𝑔 ) count: 1) testing of the approach in the real environment of the same case - 0 𝒄𝟑 𝑩𝑷𝑪: “routine” study, 2) applying of the framework on the case study from a 𝑥𝑟 count: different domain and 3) currently strong focus on the empirical installation, release, application 3 handcrafted rules, i.e. absence of a “learning” component of the RS. 𝑥𝑠𝑐 count: As a part of future work, we will encode the algorithms to build a - 0 𝑥𝑐 count: proof-of-concept of the suggested multi-criteria knowledge-based - 0 RS. Subsequently, the prototype will be evaluated on the case study 𝑷𝑷𝑪 index: “low” data set and by the BP workers. In parallel, we will search for a case Recommendation 𝑹: template 𝑴𝒆 study from a different domain to test the framework. Multi-Criteria Knowledge-Based Recommender System ComplexRec’19, September 20, 2019, Copenhagen, Denmark REFERENCES [17] OMG. 2013. Business Process Model and Notation (BPMN) - Version 2.0.2. [1] Gediminas Adomavicius, Nikos Manouselis, and YoungOk Kwon. 2015. Multi- Object Management Group, Needham. Criteria Recommender Systems. In Recommender Systems Handbook, Francesco [18] Vili Podgorelec, Peter Kokol, Bruno Stiglic, and Ivan Rozman. 2005. Decision Ricci, Lior Rokach and Bracha Shapira (Eds.), Springer, New York, 847-880. Trees: An Overview and Their Use in Medicine. Journal of Medical Systems 26 Electronic ISBN: 978-1-4899-7637-6. (5), 445-463. DOI: 10.1023/A:1016409317640. [2] Zhifeng Bao, Yong Zeng, H.V. Jagadish, and Tok Wang Ling. 2015. Exploratory [19] Nina Rizun, Aleksandra Revina and Vera G. Meister. 2019. Discovery of Keyword Search with Interactive Input. In SIGMOD Proceedings. ACM, Stylistic Patterns in Business Process Textual Descriptions: IT Ticket Case. In Melbourne, 871-876. 33rd International Business Information Management Association Conference [3] Allison J.B. Chaney, Brandon M. Stewart, and Barbara E. Engelhardt. 2018. How (IBIMA). Web of Science, Granada. Algorithmic Confounding in Recommendation Systems Increases Homogeneity [20] Nina Rizun, Aleksandra Revina and Vera G. Meister. 2019. Method of Decision- and Decreases Utility. In RecSys '18 Proceedings of the 12th ACM Conference Making Logic Discovery in the Business Process Textual Data. In 22nd on Recommender Systems. ACM, Vancouver, 224-232. International Conference on Business Information Systems. Springer, Seville, 70- DOI: 10.1145/3240323.3240370. 84. DOI: 10.1007/978-3-030-20485-3_6 [4] Robert Chi and Melody Kiang. 1991. An Integrated Approach of Rule-based and [21] Nina Rizun, Aleksandra Revina and Vera G. Meister. 2019. Business Sentiment. Case-based Reasoning for Decision Support. ACM, 255-267. Concept and Method for Perceived Anticipated Effort Identification. In 28th DOI: 10.1145/327164.327272. International Conference of Information Systems Development. ACM, Toulon. [5] Joel Coffman and Alfred C. Weaver. 2010. Framework for Evaluating Database [22] Bernard Roy. 1996. Multicriteria Methodology Goes Decision Aiding. Springer Keyword Search Strategie. In CIKM Proceedings. Toronto, 729-738. US, New York. DOI: 10.1007/978-1-4757-2500-1. DOI: 10.1145/1871437.1871531. [23] Per Runeson and Martin Höst. 2009. Guidelines for Conducting and Reporting [6] Daelemans Walter. 2013. Explanation in Computational Stylometry. In Case Study Research in Software Engineering. Empirical Software Engineering International Conference on Intelligent Text Processing and Computational 14(2), 131-164. DOI: 10.1007/s10664-008-9102-8. Linguistics. Springer, Samos, 451-462. DOI: 10.1007/978-3-642-37256-8_37. [24] Thomas L. Saaty, 2005. The Analytic Hierarchy and Analytic Network Processes [7] Alexander Felfernig and Robin Burke. 2008. Constraint-based Recommender for the Measurement of Intangible Criteria and for Decision-Making. In Multiple Systems: Technologies and Research Issues. In 10th International Conference on Criteria Decision Analysis: State of the Art Surveys, Salvatore Greco (Ed.), Electronic Commerce (ICEC). ICEC, Innsbruck. Springer, New York, 345-405. DOI:10.1007/b100605. DOI: 10.1145/1409540.1409544. [25] Ralph Schäfer. 2001. Rules for Using Multi-Attribute Utility Theory for [8] Alexander Felfernig, Gerhard Friedrich, Dietmar Jannach, and Markus Zanker. Estimating a User's Interests. In ABIS Workshop Adaptivität und 2011. Developing Constraint-based Recommenders. In Recommender Systems Benutzermodellierung in interaktiven Softwaresystemen. DFKI, Dortmund, 1-7. Handbook, Francesco Ricci, Lior Rokach, Bracha Shapira, Paul B. Kantor (Eds.), [26] Markus Schäfermeyer, Christoph Rosenkranz, and Roland Holten. 2012. The Springer, New York, 187-215. DOI: 10.1007/978-0-387-85820-3. Impact of Business Process Complexity on Business Process Standardization. [9] Crina Grosan and Ajith Abraham. 2011. Rule-Based Expert Systems. Intelligent Business & Information Systems Engineering, 261-270. DOI: 10.1007/s12599- Systems Reference Library, Springer, 149-185. 012-0224-6. [10] Ferdaous Hdioud, Bouchra Frikh, and Brahim Ouhbi. 2013. Multi-Criteria [27] Scorobey. 2017. A Simple Program on Python for Hyperbolic Approximation of Recommender Systems based on Multi-Attribute Decision Making. In Statistical Data. In Russian. https://m.habr.com/ru/post/322954/. International Conference on Information Integration and Web-based [28] Yi Shan. 2016. Semantic Keyword Search on Large-Scale Semi-Structured Data. Applications & Services. ACM, Vienna, 203-210. Dissertation, Arizona State University, Arizona. [11] ITIL® Service Transition. 2011. TSO, London. [29] Sam Wanekeya. 2018. Can’t Stand The Increasing Complexity Of IT [12] Conrad J. Jacoby. 2012. Understanding the Limitations of Keyword Search. Infrastructure, AI Is Here To Help. Headwaynews. White Paper, Equivio. https://www.headwaynews.org/cant-stand-the-increasing-complexity-of-it- [13] Ralph L. Keeney. 1992. Value-Focused Thinking. A Path to Creative Decision- infrastructure-ai-is-here-to-help/. Making. Harvard University Press, Cambridge MA. [30] Zhong Zeng, Zhifeng Bao, Tok Wang Ling, and Mong Li Lee. 2012. iSearch: An [14] Jie Lu, Dianshuang Wu, Mingsong Mao, Wei Wang, and Guangquan Zhang. Interpretation Based Framework for Keyword Search in Relational Databases. In 2015. Recommender System Application Developments: A Survey. Decision Proceedings of the 3rd International Workshop on Keyword Search on Support Systems (2015), 12-32. DOI:10.1016/j.dss.2015.03.008. Structured Data (KEYS 2012). ACM, Arizona, 3-10. [15] Nikos Manouselis and Constantina Costopoulou. 2007. Analysis and [31] George Kingsley Zipf. 1949. Human Behaviour and The Principle of Least Classification of Multi-Criteria Recommender Systems. World Wide Web: Effort. Addison-Wesley, Reading. Internet and Web Information Systems, Special Issue on Multi-channel Adaptive [32] George Kingsley Zipf. 1932. Selected Studies of the Principle of Relative Information Systems on the World Wide Web, 1-27. Frequency in Language. Harvard University Press, Cambridge. [16] Hien Nguyen and Peter Haddawy. 1999. DIVA: Applying Decision Theory to Collaborative Filtering. In AAAI Workshop on Recommender Systems. AAAI, Madison, 20-26.