Decision Support combining Machine Learning, Knowledge Representation and Case-Based Reasoning Carlo Mehlia, Knut Hinkelmanna,b and Stephan Jünglinga a FHNW University of Applied Sciences and Arts Northwestern Switzerland, Riggenbachstrasse 16, 4600 Olten, Switzerland b University of Pretoria, Department of Informatics, Pretoria, South Africa Abstract Knowledge and knowledge work are essential for the success of companies nowadays. Decisions are based on knowledge and better knowledge leads to more informed decisions. Therefore, the management of knowledge and support of decision making has increasingly become a source of competitive advantage for organizations. The current research uses a design science research approach (DSR) with the aim to improve the decision making of a knowledge intensive process such as the student admission process, which is done manually until now. In the awareness phase of the DSR process, the case study research method is applied to analyze the decision making and the knowledge that is needed to derive the decisions. Based on the analysis of the application scenario, suitable methods to support decision making were identified. The resulting system design is based on a combination of Case-Based Reasoning (CBR) and Machine Learning (ML). The proposed system design and prototype has been validated using triangulation evaluation, to assess the impact of the proposed system on the application scenario. The evaluation revealed that the additional hints from CBR and ML can assist the deans of the study program to improve the knowledge management and increase the quality, transparency and consistency of the decision-making process in the student application process. Furthermore, the proposed approach fosters the exchange of knowledge among the different process participants involved and codifies previously tacit knowledge to some extent and provides relevant externalized knowledge to decision makers at the required moment. The designed prototype showcases how ML and CBR methodologies can be combined to support decision making in knowledge intensive processes and finally concludes with potential recommendations for future research. Keywords 1 Case-Based Reasoning, Machine Learning, Decision Support, Knowledge Management, Knowledge Representation, Knowledge-Intensive Process 1. Introduction Knowledge and knowledge work are essential for the success of companies nowadays. It is observable that there is a shift from routine work to knowledge work [1]. Since decisions are based on knowledge and better knowledge leads to more informed decisions, governance of decision making has increasingly become a source of competitive advantage for organizations [2]. Decision making often occurs in knowledge-intensive processes, where the knowledge is responsible to determine an output [3]. This process is also referred to as decision-making process and can be classified as a knowledge- intensive process. To some extent, decisions are guesses about the future based on information available 1 In A. Martin, K. Hinkelmann, H.-G. Fill, A. Gerber, D. Lenat, R. Stolle, F. van Harmelen (Eds.), Proceedings of the AAAI 2021 Spring Symposium on Combining Machine Learning and Knowledge Engineering (AAAI-MAKE 2021) - Stanford University, Palo Alto, California, USA, March 22-24, 2021. E-mail: carlo.mehli@students.fhnw.ch, knut.hinkelmann@fhnw.ch, stephan.juengling@fhnw.ch ORCID: 0000-0002-1746-6945 (K. Hinkelmann), 0000-0002-2969-7257 (S. Jüngling) © 2021 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). CEUR Workshop Proceedings (CEUR-WS.org) at the time of the decision. As complexity builds up, decision-makers often have to rely on their intuition and judgement [4]. Therefore, the knowledge of the involved decision-makers is often implicit and not accessible for others. The transfer and flow of knowledge between different knowledge-workers is crucial to achieve a successful process completion [5]. Researchers propose many different approaches such as Ontologies, Knowledge Management, Artificial Neural Networks, Case Based Reasoning (CBR) etc. to support various knowledge intensive processes (KIP) and decision makings [3, 5–9]. CBR has been proven to be useful for decision support in several use cases, in particular also in business processes [10–13]. It helps people to make knowledge-based decisions by showing appropriate previous cases to be considered. It augments people’s ability of reasoning, since people are bad at remembering details from past cases [14]. However, knowledge acquisition for CBR systems is still an issue, since case attributes and similarity measures have to be defined when designing the system [13]. Furthermore, CBR systems are not able to deal with large amount of data [14]. Machine Learning (ML) on the other hand is able to deal with large amount of data and offers an approach to learn from data autonomously. It can identify implicit regularities based on data of previous cases and build a classification model accordingly [15]. However, the results usually lack of explainability and therefore transparency for the decision maker [16]. Furthermore, Machine Learning needs large amount of data and does not inherit general domain knowledge [17]. The objective of this research is to identify the criteria for knowledge support in a concrete application scenario and develop an appropriate combination of CBR and ML, which helps to use the knowledge of previous cases based on an explainable set of human interpretable rules, supplemented by ML which can generate recommendations from a subsymbolic perspective, in order to implicitly learn the patterns based on the features from all previous cases. After a short literature review that refers to related work in supporting human decisions in knowledge intensive processes, the particular application scenario of the student admission process as well as the improvement potential is described in the chapter 3 and 4. The system design of the prototype shows how the aspects of knowledge engineering are combined with machine learning, followed by an evaluation, outlook and conclusion. 2. Literature The importance of knowledge for companies, as well as for decision making, is well described in the literature [18, 19]. However, since even the definition of knowledge remains research worthy, the management of knowledge within companies and processes is still a challenge. The shift from routine work to knowledge work and the increasing importance of knowledge-intensive tasks and processes are crucial for businesses [8, 19]. Making the right knowledge available and accessible at the right time usually leads to better decisions [4]. Several dimensions of knowledge have been identified, some of which cannot easily be explained and shared [20, 21]. Therefore, knowledge needs to be identified first in order to manage it. However, not all types of knowledge can easily be extracted and managed, which is a particular challenge for tacit knowledge. For decisions to be consistent and knowledge to be shared and reused, tacit knowledge needs to be somehow made explicit. While knowledge-based decisions are usually better than intuition-based decisions (in an organizational environment) and knowledge-based decision making can be supported and fostered by providing the right methods, it is not clear which methods and systems are best to support certain kind of decisions [3]. Several knowledge management strategies and decision support methods are suggested for certain use cases [11, 22–24]. There are many use cases in literature which use knowledge-based systems such as CBR for decision support [10–13]. Machine Learning is considered in decision support, often in combination with knowledge-based systems [15, 17, 25]. The different strengths and weaknesses of the approaches can be beneficial when implemented for certain scenarios. However, it is not clear which combination is suitable for a specific decision and knowledge need. Therefore, the analysis of a specific knowledge-intensive business process and knowledge flow with the process-oriented knowledge management methodology can provide more insights if there is a possibility to integrate intelligent systems and identify patterns in the decision-making scenario. 3. Application Scenario In order to get a holistic view on how CBR and ML can be combined to support decisions, the knowledge-intensive process of student admission for a Master’s program at FHNW University of Applied Sciences and Arts was analyzed in depth. Universities are nowadays operating in a complex and competitive environment. Alongside identifying their uniqueness and building a sustainable strategy, one of the challenges for modern universities is to admit the right students [26]. The decision about a candidate’s admission is crucial for the university as well as the student, and the decision is therefore taken with the appropriate caution. Every candidate’s profile and application documents are analyzed individually and there is no general solution applicable for all cases. As the current admission process does mostly rely on human experts applying their judgment and intuition, transparency and consistency of the decision is not always given. Moreover, the applicants come from various countries with different educational systems, grading schemes etc., which can lead to extensive research in order to fairly assess and compare the applicants’ education background. The research can take up to several days, increasing the processing time of a case significantly. Since decisions and research about previous cases are not formally recorded, it is also possible that the research is done multiple times for similar cases while the decisions can vary, and the outcomes be different. However, since the process and the decision about admission of a candidate should be fair for all candidates, the decision-making aims to be consistent and the outcome for two candidates with similar professional background and education should be the same. Since the university admission process heavily depends on knowledge workers performing various interconnected knowledge-intensive decision-making tasks, the process itself is considered a knowledge intensive process [27]. To get a better awareness of the problem within the application scenario, multiple methods from case-study research such as interviews, assessment of data, observation of involved parties regarding decision making and a business decision maturity assessment of the business decision management were assessed. The case study resulted in the formalization of the admission process and the identification of decisions and corresponding knowledge used for decision making. The knowledge sources were further analyzed regarding knowledge coordination, knowledge development and classified in knowledge types. Finally, issues regarding the knowledge management and business decision management were identified. The case study has shown that the knowledge within the admission process is currently not sufficiently managed. Despite the decision makers having a lot of knowledge, the tacit knowledge is not captured and therefore bound to the decision-maker. Furthermore, knowledge from previous cases or investigations is not managed or stored resulting in risk of losing knowledge if people leave or are not available. However, the interviews with process participants also showed, that they would appreciate appropriate knowledge about previous cases to support decision making, if no substantial additional work is required to maintain or enter the data. The assessment also points out, that there is a lack of tool integration within the process. Several tools and sources are used, and data must be manually entered several times for different systems. This is not only time consuming and error prone, but also leads to duplicity of data, making it harder to analyze previous cases. The following issues and knowledge management methods were outlined by the case study: • There is currently no defined knowledge management strategy applied in the application process. • No knowledge base or tool other than spreadsheets is currently used to capture structured knowledge. • Decisions are often based on intuition and experience of executors. • Knowledge exchange is mainly done by socialization on request, leading to knowledge staying tacit even after transferring the knowledge. • Most knowledge sources are tacit knowledge which is currently not externalized in a structured manner. • There is no modelling standard, performance measure or decision model applied during the admission process • There is a risk of losing knowledge throughout the process, as well as in case of changing personal. Based on the insight gained by the case study, strategies to better use existing knowledge and therefore support and manage decision making within the student admission process were evaluated and developed. 4. Methods for improvement As the assessment of the current admission process revealed, that there is a vast amount of knowledge available among the decision-makers. However, this knowledge is currently not capitalized and neither actively managed. Three elements People - Process - Technology have been identified as the important influencers for successful knowledge management [28]. As the assessment of the admission process showed, there is currently no knowledge management process in place and no tool or software implemented to support knowledge management and decision making. The People component in the admission process seems not to be the issue, since the decision makers collaborate and share their knowledge. In order to improve the decision support and implement a knowledge management strategy in the admission process, the focus should be on the component Technology. In order to achieve the goals of knowledge management, there are several challenges that must be faced. The challenges in knowledge management outlined by [29] do also apply for the admission process. In order to support the decision making and manage the knowledge in the student admission process, the supporting methods must be able to face the challenges. The decisions within the admission process require different knowledge. As previously described, the challenge is not only to acquire the knowledge, but also to provide the right knowledge to the right person at the right time in an understandable way. Therefore, we assessed what kind of support methods are suitable for the decisions within the admission process. Since the decisions require different knowledge and the current knowledge is available in different types and comes from different sources, the requirements and appropriate methods to support the decision making differ. The suitability of the method depends on the decision itself. Knowledge based systems provide consistency in decision making and make expert knowledge available to less experienced personal [30]. They provide the capabilities required to achieve the previously defined goals of knowledge management. How a knowledge-based system is built, and what techniques are used are dependent on the situation and the AI techniques available [31]. Knowledge-based systems consist of a knowledge base, which represents the knowledge either as rules or as cases [32]. Therefore, there is in any case a need of a knowledge base in order to store and reuse the knowledge within the admission process. The intuition and experiential knowledge of the knowledge workers, which are mainly tacit knowledge, cannot be easily described. Among the methods that have been proven to be useful for knowledge acquisition as well as decision support, are CBR and ML. Both having different strengths and therefore can be applied in different scenarios for different kind of knowledge. Several approaches and designs of CBR and ML exist for certain use cases. However, the parallel implementation of both methods is rarely considered. As outlined in the case study of the admission process, the decision makers currently often use experiential knowledge to derive their decisions. This indicates that the experience and therefore the knowledge gained of past cases and decisions, is relevant for the decision making. CBR uses analogical reasoning methods to learn from previous cases and present similar cases. However, while humans are prone to not remember all aspects of each previous case, CBR can support human decision making by presenting the decision maker similar cases. In the admission process, this can be crucial in order to support consistent decision making. Furthermore, CBR keeps the instances (cases) in its original form and does not generalize from past cases, making it able to maintain explainability and transparency. However, in situations where knowledge must be learned inductively or universally valid rules need to be deducted from data, CBR is not suitable. In such situations, ML is suitable since it is able to learn from situations and build a generally valid model. It is especially useful in areas where not all situations can be defined upfront, or tacit knowledge cannot explicitly be explained by the decision maker, but data of previous decision is available. While CBR finds similar past situations, ML builds a model based on data in order to apply the model to newly entered data. The decisions in the student admission process were analyzed with regards to the aforementioned strengths of the AI methods, in order to identify the best supporting method for each decision. Furthermore, as described in the case study, the current knowledge is available in different types and has different forms of coordination. In order to assess the decision support method, it was determined in what form the required knowledge is available and how the knowledge can be transferred. Hence, the suitable knowledge management strategy for the decisions within the admission process was analyzed (see Table 1). During the analysis of the admission process application scenario it was observed that individuals tend to apply a certain CBR method implicitly while performing knowledge-work oriented tasks. Therefore, it is reasonable to conclude that individuals are already familiar with this type of problem- solving method. Table 1 Suggested decision support method and knowledge management strategy per decision Ontologies have been proven useful in combination with CBR and especially in knowledge- intensive environments. Therefore, with regard to business applications, the use of an ontology in an ontology-based CBR approach can be regarded as suitable for the application scenario of a student admission process. With regards to the identified knowledge management challenges, the following suggestions for the system design were derived by reviewing literature of proposed system designs [23, 33, 34]: • Structural ontology-based CBR to address knowledge intensity of the process and bridge the gap between knowledge acquisition and retrieval • Viewpoint based implementation to satisfy different knowledge needs per decisions and participants to address the challenges of knowledge publishing and retrieval • Artificial Neural Networks to learn a model to support grade conversion While the suggested decision support methods described for each decision the rationale of what method is best used, they did not cover architecture and design specific detail questions and practical applicability in a real-world setting. This will be examined in the next chapter. 5. System Design Based on the identified support methods and the insights gained in the case study, a system design, user interface design and a ML prototype were developed. 5.1. Case-Based Reasoning As suggested by several researchers [12, 33, 35] the ontology-based structural CBR approach is a successful method in knowledge-intensive environments. Based on the case study of the student admission scenario and the identified decision support methods, the following high-level objectives for the ontology-based CBR system design have been derived: • A single case base should be used for all CBR-supported decisions • The case content should contain an update functionality in order to capture knowledge of previous situations and investigations (new cases of university recognition, how to convert grades etc.) • The case should be able to contain information or at least link to data objects and documents (e.g. CV, motivation letter) • The case should be described in a structured way to allow analysis and filtering In order to build the case-base for CBR, the relevant case attributes, needed to retrieve cases and base decisions on, were defined. In the scenario of the admission process, a case represents a single student admission. We applied an ontology-based and viewpoint-based CBR approach as suggested by Martin et al. [25]. Based on the identified attributes the ontology for the admission process was developed using OWL. In Figure 1 the graphic representation of the ontology is displayed. The used ontology language OWL provides a scheme containing different properties, which are used to determine the similarity of the case. Figure 1: Admission process ontology illustrated as a graph For an ontology-based CBR approach, the main task is to compute the similarity of properties to present similar cases to the user. In the structural ontology-based CBR approach, the instances and relations representing the case characterizations of the learned cases and the query case are compared. In Figure 2 the principle of case comparison using case characterization is schematically illustrated. Figure 2: Schematic illustration of case comparison in structural CBR (adapted from [23]) As mentioned in the previous chapter, a viewpoint-based CBR approach is applied. The applied viewpoints allow the relevant cases to be retrieved by the same CBR-system from a singular case base, while considering the individual information need of the decision and decision maker. Hence, for each decision a viewpoint is applied using different attributes and similarity weights for the retrieval of similar cases. In the admission process viewpoints are the equivalence of a specific decision within the admission process, such as the grade conversion, the recognition of the university and the assessment of the work experience. Depending on the viewpoint, different attributes are of interest and therefore different cases are found. In Figure 3 the different viewpoints on a case is illustrated. Figure 3: Viewpoint example on a case (adapted from [23]) Since for each viewpoint different attributes are relevant and have to be compared, the attributes per viewpoint (decision) were identified. As not all attributes have the same relevance for the retrieval of a case, the weight per attribute was defined. The weight is defined as 0 being not relevant, 10 being the most relevant attribute for the similarity check. The identified attributes are presented to the user and will be the basis of the decision. Therefore, it is also relevant to display attributes to the decision maker, which are not relevant for case retrieval itself. Table 2 shows the relevant attributes and their weights similarity check exemplary for one decision. Table 2 Attributes and weights per viewpoint (decision) and their relevance to similarity calculation 5.2. Machine Learning Prototype In order to support and partly automate the conversion of foreign grades to Swiss grades, a machine learning (ML) prototype was implemented. The learning approach considers the data of past grade conversions done by the process executors and the conversion tables, such as the grading scales of the Universities of Hannover, Göttingen and Freiburg. All of these conversion tables are currently used for the conversion of foreign grades. However, since the conversion tables are differing, the tables are only used as indicators for the conversion. The final decision on how to convert the grade is currently up to the decision maker and his/her experience. This experience could be capitalized by ML as previous conversions can be used to train a ML model. Since deep learning is known to generalize and should therefore be able to build a useful grade conversion model, the prototype was realized with a simple (deep) artificial neural network (ANN) [36]. The prototype was developed using freely available resources, such as the R programming language and the Keras package. A limitation of the prototype is the number of available data. There exists no dataset containing the foreign grades and the converted Swiss grades of previous admission cases. Only the converted grades have been explicitly documented. Therefore, the training set has been simulated by using the data of previously mentioned conversion tables. The test dataset consists of grades of three countries (Germany, India and South Africa). In order to assess the conversion, different grades of the said countries were converted by the prototype, after training was completed. The results of the ML-converted grades are listed in Table 3 and compared to the conversion tables. The advantage of ML over conversation tables is, that ML learns the exact conversation key instead of only providing a range, as shown in row 3. Furthermore, if the conversion models differ it takes into account the different conversions. Therefore, the result takes into account all conversion tables, which have currently to be checked manually by the decision maker. Table 3 Results of ML-converted grades in comparison to conversion tables The prototype has shown that the grade conversion can be supported by applying machine learning. However, other ML-architectures could further be analyzed to assess whether the grade conversion can be done more accurately. The developed ML and CBR designs are not tangible for process executors, since the design is rather technical. In order to assess the usability of the proposed system design, to make the proposed decision support methods easily understandable for all knowledge workers and to show the integration into the admission process, some examples of a possible user interface were created using mockups. These mockups make it possible for the decision makers to understand how the system could support them and how the decision process would change. 6. Evaluation The evaluation was done by triangulation using decision maturity assessment, qualitative interview and performance analysis. The Business Decision Maturity Model (BDMM) [37] was conducted to assess the impact of the designed support system on the admission process and its decision management and to point out in what areas the proposed methods and system design could improve the decision maturity. A qualitative evaluation in form of interviews with knowledge-workers were conducted in order to assess the applicability and the impact of the system in the application scenario. Finally, the performance of the ML-prototype was tested by comparing the results of the automated grade conversion to the results derived by the currently used conversion tables. The results of the different evaluation methods revealed that the proposed system design can support knowledge management and decision making in the knowledge-intensive decision process of student admission. The conducted maturity assessment analyses the system from a business decision management perspective: • The results of the assessment show that the system can in fact improve the business decision management of the admission process. This improvement is mainly due to the fact, that by implementing the system the decisions can be retraced, knowledge is shared, and the decisions are therefore more consistent and transparent. • Also, the results of the interviews outline that the system is able to improve the knowledge management in the admission process and supports decision makers in their decision-making process. Due to the automated case retrieval, it provides decision makers with the relevant knowledge at the right moment. The system augments the natural thinking process by finding similar cases and allows decision makers to document and therefore share their knowledge. • It was furthermore argued that the system increases confidence in the correctness of preceding decisions done by other decision makers. • Finally, also the data driven results of the Machine Learning prototype evaluation point out that the prototype can be used to support conversion of grades. The combined results of the different evaluations indicate that the proposed system design addresses most of the current issues of the admission process. The only issue, which is not addressed by the system, is the definition of modelling and performance standards. However, the evaluation shows that the applied methods to address the identified issues are suitable. When the applied support methods (CBR / ML) are compared to the identified types of knowledge in the admission process, there can be no clear causality seen between the knowledge type and the applied supporting method. It becomes clear, however, that ML needs explicit, structured data in order to learn a valid model. Furthermore, ML can only be applied in cases in which a generalization based on the training data is desired. In many decisions of the student admission process, a generalization is however not suitable. The evaluation of the applied methods compared to the knowledge also outlines, that CBR is especially useful in cases in which experiential knowledge is applied for decision making. However, the suitable method to support a certain decision still remains use case specific and is dependent on many factors such as available data, kind of decision, available knowledge, decision making process etc. Nevertheless, the decision making within a knowledge-intensive decision process can be improved by combining CBR and ML, can be considered proven by the results of the evaluation. 7. Conclusion and Outlook The management of knowledge and the choice of most appropriate methods to support the decision- making process is regarded as a daunting task throughout the literature [1, 14, 38]. This is due to the fact that knowledge occurs in different forms and can be difficult to describe [5]. Furthermore, the methodology mix to support a specific knowledge intensive process and its decisions, remains use case specific [12, 25, 39, 40]. Therefore, there is still a considerable lack of specific decision support design patterns that fit best for specific knowledge-intensive and complex decision problems. In order to potentially identify a correlation between decision or knowledge patterns and different supporting methods and tools, a system design including the combination of CBR and Machine Learning approach has been proposed and applied to the application scenario of the student admission at our university. The collected data through maturity assessments and qualitative interviews was used to evaluate the applicability of the proposed system design for the chosen application scenario. The result of the evaluation has shown that decision making within a knowledge-intensive decision process for university student admission can be improved by combining Case-Based Reasoning and Machine Learning. The conducted user experiments with the decision makers revealed that the proposed system design supports most information needs for their decisions. The conducted maturity model also confirmed that the system design improves various factors of decision making and knowledge management in the admission process. Namely, the system can contribute to reduce the risk of knowledge loss, increase the transparency of decision making, enhance knowledge sharing among decision makers and support consistent decision making. Furthermore, existing tacit knowledge can be made explicit and duplication of work in the time-intensive research activities can be reduced due to previous admission cases with the help of CBR. CBR is especially useful in cases where experiential knowledge is applied and can augment the natural thinking process of the decision maker. ML on the other hand is suitable to support decisions, in which training data is available, generalization is desired and no detailed explanation of the derived result is required. Hence, CBR and ML can complement each other and can be combined in various ways, which eventually can lead to specific design patterns in knowledge-intensive decisions processes. However, more data and experience need to be collected. The current prototype has shown promising results in a real-world setting, how some of the knowledge management issues can be solved and how the consistency and transparency of the admission process can be improved. The research has shown that [[29]29][29]a desired shift from intuition-based decision making, to fact-based decision making as described by [18], can be achieved by providing clearly arranged, sufficient knowledge to the decision maker. Finally, the research also validated theory [3] that the coordination and support of single decisions within a business process leads to more business value creation. As the scope of this research was limited to the knowledge-intensive decision process in a university student admission scenario, it could investigate into a feasible combination of CBR and ML for this specific problem only. However, more research remains to be done in order to work out best practices in how to combine and better understand the correlations of both approaches. The final goal, to design reusable decision systems that can serve as AI-related design patterns for knowledge-intensive decision processes, will need further investigations. 8. References [1] Davenport, T., Prusak, L.: Working knowledge: Managing what your organization knows. Harvard Bus. Sch. Press. 1–15 (2000) [2] Taylor, J., Raden, N.: Smart (Enough) Systems: How to Deliver Competitive Advantage by Automating the Decisions Hidden in Your Business. Prentice Hall Press, USA (2007) [3] Bromberg, D.: BPM for Knowledge Workers Inside Decision Intensive Processes (DIPs): Knowledge, Practice, Context, and Characteristics. BPTrends. 1–10 (2007) [4] Bennet, A., Bennet, D.: The Decision-Making Process in a Complex Situation. In: Handbook on Decision Support Systems 1. pp. 3–20. Springer Berlin Heidelberg, Berlin, Heidelberg (2008) [5] Hagen, C., Ratz, D., Povalej, R.: Towards Self-Organizing Knowledge Intensive Processes. J. Univers. Knowl. Manag. 0, 148–169 (2005) [6] Aamodt, A.: A knowledge-intensive, integrated approach to problem solving and sustained learning, (1991) [7] Aamodt, A.: Knowledge-Intensive Case-Based Reasoning in CREEK. In: Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). pp. 1–15 (2004) [8] Hinkelmann, K., Karagiannis, D., Telesko, R.: PROMOTE — Methodologie und Werkzeug für geschäftsprozessorientiertes Wissensmanagement. In: Geschäftsprozessorientiertes Wissensmanagement. pp. 65–90. Springer-Verlag, Berlin Heidelberg (2002) [9] Santos França, J.B. dos, Netto, J.M., do E. S. Carvalho, J., Santoro, F.M., Baião, F.A., Pimentel, M.: KIPO: the knowledge-intensive process ontology. Softw. Syst. Model. 14, 1127–1157 (2015). https://doi.org/10.1007/s10270-014-0397-1 [10] Bergmann, R., Freßmann, A., Maximini, K., Maximini, R., Sauer, T.: Case-Based Support for Collaborative Business. In: Roth-Berghofer, T., Göker, M., and Güyenir, H. (eds.) Proceedings of the 8th European Conference, ECCBR 2006 Fethive Turkey. pp. 519–533. Springeer Heidelberg, Fethive, Turkey (2006) [11] Kolodner, J.L.: Improving human decision making through case-based decision aiding. AI Mag. 12, 52–68 (1991) [12] Martin, A., Hinkelmann, K.: Case-Based Reasoning for Process Experience. In: Dornberger, R. (ed.) Business Information Systems and Technology 4.0. pp. 47–63. Springer International Publishing, Cham (2018) [13] Richter, M.M., Weber, R.O.: Case-Based Reasoning. Springer Berlin Heidelberg, Berlin, Heidelberg (2013) [14] Bergmann, R., Althoff, K.-D., Breen, S., Göker, M., Manago, M., Traphöner, R., Wess, S.: Developing Industrial Case-Based Reasoning Applications. Springer-Verlag Berlin Heidelberg (2003) [15] Mitchell, T.M.: Machine Learning. McGraw-Hill Companies Inc., Singapore (1997) [16] Russell, S., Norvig, P.: Artificial Intelligence: A Modern Approach. Pearson Education, New Jersey, USA (2020) [17] Van Harmelen, F., Ten Teije, A.: A boxology of design patterns for hybrid learning and reasoning systems. CEUR Workshop Proc. 2491, (2019). https://doi.org/10.13052/jwe1540- 9589.18133 [18] Milkman, K.L., Chugh, D., Bazerman, M.H.: How Can Decision Making Be Improved? Perspect. Psychol. Sci. 4, 379–383 (2009). https://doi.org/10.1111/j.1745-6924.2009.01142.x [19] Probst, G.J.B.: Practical Knowledge Management - A Model That Works. Prism. 17–29 (1998) [20] Hinkelmann, K., Blaser, M., Faust, O., Horst, A., Mehli, C.: Virtual Bartender: Dialog System Combining Data-Driven and Knowledge-Based Recommendation. In: Martin, A., Hinkelmann, K., Gerber, A., Lenat, D., van Harmelen, F., and Clark, P. (eds.) AAAI Spring Symposium: Combining Machine Learning with Knowledge Engineering 2019. CEUR-WS.org, Palo Alto (2019) [21] Nonaka, I., Takeuchi, H.: The Knowledge-Creating Company: How Japanese Companies Create the Dynamics of Innovation. Oxford University Press (1995) [22] Burca, D., Schüller, M., Zlabinger, J.: Case-based Reasoning and Machine Learning. , Vienna (2018) [23] Martin, A., Emmenegger, S., Hinkelmann, K., Thönssen, B.: A viewpoint-based case-based reasoning approach utilising an enterprise architecture ontology for experience management. Enterp. Inf. Syst. 11, 551–575 (2017). https://doi.org/10.1080/17517575.2016.1161239 [24] Wabwoba, F., Mwakondo, F.M.: Students selection for university course admission at the joint admissions board (Kenya) using trained neural networks. J. Inf. Technol. Educ. 10, 333–347 (2011). https://doi.org/10.28945/1529 [25] Burca, D., Schüller, M., Zlabinger, J.: Case-Based Reasoning and Machine Learning, (2018) [26] Kabakchieva, D.: Predicting student performance by using data mining methods for classification. Cybern. Inf. Technol. 13, 61–72 (2013). https://doi.org/10.2478/cait-2013-0006 [27] Di Ciccio, C., Marrella, A., Russo, A.: Knowledge-Intensive Processes: Characteristics, Requirements and Analysis of Contemporary Approaches. J. Data Semant. 4, 29–57 (2015). https://doi.org/10.1007/s13740-014-0038-4 [28] Bhojaraju, G.: Knowledge management: Why do we need it for corporates. Malaysian J. Libr. Inf. Sci. 10, 37–50 (2005). https://doi.org/10.2139/ssrn.3375572 [29] Shadbolt, N., O’Hara, K.: AKTuality: An Overview of the Aims, Ambitions and Assumptions of the Advanced Knowledge Technologies Interdisciplinary Research Collaboration. AKT Sel. Pap. 1–11 (2003). https://doi.org/10.1017/CBO9781107415324.004 [30] Smith, R.G.: Knowledge-Based Systems Concepts, Techniques, Examples. Proc. Can. High Technol. Show. 1–84 (1985) [31] Aamodt, A., Plaza, E.: Case-based reasoning: Foundational issues, methodological variations, and system approaches. Artif. Intell. Commun. 7, 39–59 (1994). https://doi.org/10.3390/s120811154 [32] Hopgood, A.: The State of Artificial Intelligence. Adv. Comput. 65, 1–75 (2005). https://doi.org/10.1016/S0065-2458(05)65001-2 [33] Bergmann, R., Schaaf, M.: Structural case-based reasoning and ontology-based knowledge management: A perfect match? J. Univers. Comput. Sci. 9, 608–626 (2003). https://doi.org/10.3217/jucs-009-07-0608 [34] Recio-García, J.A., González-Calero, P.A., Díaz-Agudo, B.: Jcolibri2: A framework for building Case-based reasoning systems. Sci. Comput. Program. 79, 126–145 (2014). https://doi.org/10.1016/j.scico.2012.04.002 [35] Recio-García, J.A., Díaz-Agudo, B., González-Calero, P.A.: The COLIBRI Platform: Tools, Features and Working Examples. In: Montani, S. and Jain, L.C. (eds.) Successful Case-based Reasoning Applications-2. pp. 55–85. Springer Berlin Heidelberg, Berlin, Heidelberg (2014) [36] Gavrishchaka, V. V., Yang, Z., Miao, R.: Advantages of hybrid deep learning frameworks in applications with limited data. Int. J. Mach. Learn. Comput. 8, 549–558 (2018). https://doi.org/10.18178/ijmlc.2018.8.6.744 [37] von Halle, B., Goldberg, L.: The Decision Model - A Business Logic Framework Linking Business and Technology. Auerbach Publications (2010) [38] Burstein, F., Carlsson, S.A.: Decision Support Through Knowledge Management. In: Handbook on Decision Support Systems 1. pp. 103–120. Springer Berlin Heidelberg, Berlin, Heidelberg (2008) [39] Haque, B.U., Belecheanu, R.A., Barson, R.J., Pawar, R.J.: Towards the Application of Case Based Reasoning to Decision-Making in Concurrent Product Development (Concurrent Engineering). In: Ellis, R., Moulton, M., and Coenen, F. (eds.) Applications and Innovations in Intelligent Systems XII. pp. 81–101. Springer-Verlag London Limited 2000, Nottingham, United Kingdom (2005) [40] Yeow, W.L., Mahmud, R., Raj, R.G.: An application of case-based reasoning with machine learning for forensic autopsy. Expert Syst. Appl. 41, 3497–3505 (2014). https://doi.org/10.1016/j.eswa.2013.10.054