In the rapidly evolving landscape of Artificial Intelligence (AI), the rise of black box models has raised significant concerns regarding the transparency, interpretability, and accountability of automated decision-making processes. Currently used machine learning (and, in particular, deep learning) models operate as "black boxes", making decisions based on intricate patterns and relationships within large volumes of data, without providing insight into the underlying decision-making logic. This lack of transparency places at risk trust and confidence in automated decision-making processes. Stakeholders, including end-users, regulators, and policymakers, are often left in the dark regarding how decisions are made, leading to skepticism, uncertainty, and potential resistance to AI adoption. Without visibility into the decision-making process, assessing the fairness, equity, and reliability of automated systems becomes a challenge.

In addition, the opacity of black box models can conceal biases, errors, and unintended consequences that may arise from the data or the model's architecture. Biases embedded in training data can perpetuate discrimination and inequality, leading to unfair outcomes for certain individuals or groups. Without transparency into how decisions are made, it becomes difficult to detect and mitigate these biases, posing risks to fairness and ethical compliance. This issue is particularly crucial in domains, such as healthcare, justice and finance, where decisions can have a deep impact on individuals' lives.

In this context, eXplainable AI (XAI) algorithms have received growing interest thanks to their ability to extract useful insights also from complex models. Nonetheless XAI techniques usually highlight the features that mostly impact the decisions made by AI. Although this is very helpful and easy to understand in many situations, it may become a limitation when decisions involve more complex relationships between parameters. In fact, in many situations a feature alone is not determining, but rather the combination of multiple variables leads to a decision.

Usually, humans synthesize such complex relationships in the form of rules. Rules are how humans make decisions. Whether it is the decision on a therapeutic treatment or on the need to change the parameter of a plant, humans usually reason by considering thresholds on the variables involved and combine them in multivariate conditions. In a transition from humanbased to AI-based decision making, the incompatibility between logic-based human reasoning and maths-based machine reasoning may become an issue. As a matter of fact, rules defined by humans are usually difficult to integrate into black box models thus making the transition to AI-based systems disruptive.

For this reason, AI systems based on rules [1,2,3] could represent a valid alternative to classic black boxes, thanks to their ability to catch complex relationships and synthesize them in a comprehensible format. In this paper, we present a rule-based machine learning method highlighting how the generated rules can be integrated with a priori knowledge by expert users. While the rule generation algorithm has been presented and used in other works [4,5,6], the focus here is more on the integration with expert rules and human interface. A real-world application in the fintech sector is presented as well to underline the practical implications of this approach.

Rule-based models are very effective to explain the reasons behind a decision. A rule can be defined in different formats; the simplest one is the combination of a premise and a consequence: IF <premise> THEN <consequence>. The premise contains one or more conditions about the inputs while the consequence contains the output of the system, i.e. the quantity to be predicted. This could be an example of rule:

As the example shows, conditions could involve both categorical variables (i.e. for which an ordering cannot be imposed) and numerical attributes. Also the output could be categorical (in this case the problem is referred to as classification) or numerical (regression).

Humans are very good at creating rules, but usually can manage at most 2/3 conditions simultaneously. More complex premises are harder to handle, in particular when the possible combinations of variables and thresholds are numerous. Rule-based machine learning can combine the intuition of humans with the computational power of machines.

Decision Trees (DT) [7] are probably the best known methods for generating rules from a dataset. Their structure is quite easy to understand since it is based on the recursive partition of the input space in smaller subspaces. Despite their success in addressing many practical problems, decision trees suffer from certain limitations that diminish their accuracy and the knowledge they provide. For instance, decision trees consistently divide patterns into two (or more) branches without accounting for nuances. Furthermore, the initial split is often critical, as all subsequent splits depend on it. The accuracy of Decision Trees is often improved by combining several of them into a so-called Random Forest; in this way the accuracy is considerably improved but the decision mechanism becomes less clear since several sets of rules are weighted to make the prediction.

Other approaches have been proposed as well: in this paper we propose the Logic Learning Machine (LLM), a technique able to generate effective rules directly from data. LLM is based on Switching Neural Network [8], a connectionist model made up of four different layers: (i) a discretizer that transforms numerical variables into a discrete domain, (ii) a latticizer that converts data into a binary domain, (iii) a Boolean function synthesizer and (iv) a rule extractor. By transforming the original dataset into a binary domain, it is possible to convert a mathematical problem into a logical one. Step (iii) is crucial and requires proper algorithms to build a Boolean function that maximizes the accuracy of the model. The Shadow Clustering algorithm [9] is usually adopted for Boolean function reconstruction: it is based on the combination of several metrics aimed at maximizing the ability of the rule to describe the data as well as at minimizing its complexity (i.e. the number of conditions it includes). The trade-off between the adherence to data and the complexity of the rule is often crucial since it may affect the overfitting or underfitting of rules. Overfitting happens when rules describe the available data very well but fail in generalizing on other data, while underfitting occurs when rules do not describe the available data with enough accuracy. Besides the set of rules, LLM also generates a ranking of the features that mostly influences the classification/regression together with a score for each of them.

Even the most accurate of machine learning algorithms are of limited use if they are unable to provide results in a comprehensible format. Creating interfaces which allow people to inspect the results of machine learning, and to interact and enrich them with their knowledge is becoming imperative. For this reason, Rulex Platform has been designed to reduce the complexity burden for the user allowing them to focus on what is really important: the meaning of the models and the consequent predictions.

Rulex Platform [10] is a visual programming tool that allows users to graphically build data elaboration and machine learning flows of any complexity, by connecting elementary blocks (tasks), as shown in Figure 1. The Rulex task library includes 80+ blocks which each performs a distinct operation ranging from data elaboration and manipulation (e.g. join, concatenation, queries, formulas,...), to machine learning (e.g. classification, clustering, regression,...) and The task library allows users to perform operations typically done in various tools in a highly efficient and flexible manner. Computational efficiency is ensured through the use of an ad-hoc programming language named GOLD, which combines optimal computational performance with high-level programming features. Additionally, users can add their own code in GOLD, as well as in R and Python, to leverage existing algorithms and libraries.

Other features include the ability to create macro-blocks that encapsulate several tasks, the generation of a history of performed operations for tracking purposes, and various productionoriented facilities such as scheduling, alerting, and API calls.

In particular, Rulex includes some tasks specifically dedicated to XAI, namely:

• the Rule Engine through which user-defined rules (also involving features derived by the input variables by means of formulas) can be inserted directly from a file or by means of a simple GUI. • the Logic Learning Machine for classification/regression generates classification/regression rules directly from the data. • the Apply Model allows the application of rules (either user-defined or automatically generated) on new cases, making a prediction. • the Rule Manager visualizes and edits rules in a tabular format.

• the Rule Viewer shows the rules and their properties in an interactive pie chart.

• the Feature Ranking lists the features that have the greatest impact on predictions, providing relevance scores for attributes and values.

By combining these tasks (as well as other tasks for data manipulation) it is possible to easily implement XAI-based flows. For example, users can define their own rules, see how they perform and integrate them with rules generated by LLM. Once LLM has generated its rules, users can verify both their performance (e.g. the prediction accuracy and precision) in a cross-validation experiment and their meaning and coherence with the knowledge about the problem. If results are not satisfactory, changing some inputs or parameters is straightforward: it is possible to remove an attribute that should not be included in rules or to change some parameters of LLM. As a matter of fact, LLM has several customization options to refine the performance of the set of rules: for example, it is possible to change the error allowed in each rule or the maximum number of conditions to be included in each premise. This not only leads to enhancing performance but also to rendering rules more readable, comprehensible, and ultimately more applicable to real-world problems.

In this section we briefly introduce a use case as an example on how rule-based decision models can improve real-world processes. The use case is derived from an application we have implemented for one of the most important insurance companies in Italy. The goal of this use case is to reduce the impact of frauds in car accident insurance claims. The client, like other insurance companies, had a list of rules that highlighted if an accident was potentially fraudulent. These rules had been collected over the years by expert practitioners based on their domain experience. Even if the rules were often effective in many situations, this approach presented some drawbacks: (i) rules were strongly influenced by the individuals that created them, and often lacked in objectivity, (ii) for this reason, rules were hard to maintain, in particular if the expert were to leave the company, (iii) since fraud strategies continuously change, these rules quickly became obsolete and required constant updating (iv) in general, the process of gathering the data and applying the rules included many time-consuming and error-prone manual steps.

To overcome these issues, a machine learning approach can be adopted: in this way, the model for detecting fraud is based on objective data and can be updated quickly as soon as new data are available. The use of machine learning for fraud detection in the field of vehicle insurance has been explored in several recent works [11,12,13]. Despite the good accuracy and precision that these systems can achieve, it is often difficult for the insurance expert to completely trust it. This is due to the fact the the mechanisms behind machine learning reasoning are different from those of humans. Moreover, adopting a machine learning-based approach often means casting off old human-driven systems, reducing trust in the system further.

Due to these reasons, the insurance company decided to choose a hybrid system: (i) initially the human-driven rules were kept and Rulex was used to automate the decision process by building a workflow that combined all relevant data and applied the rules defined in a Rule Engine task; (ii) in a subsequent step, new rules were derived from the data and integrated with the existing ones. This two-step procedure enabled a seamless transition from a human-based approach to an AI-powered system, making the most out of the two systems.

Figure 2 shows a simplified version of the workflow, which implemented the decision process. The actual workflow, used in a real-world context, cannot be shown due to restrictions imposed by confidentiality. The upper branch implements the training of the model: historical data (gray box), after some cleaning (red box), are used to build rules (brown box) that are merged (purple In the considered scenario, four a priori rules were inserted by an expert practitioner, while further 71 rules were generated by LLM. The LLM rules exhibit a strong average covering, with the top-performing rule achieving a value of 60%. This means that 60% of the not fraudulent claims are described by a single rule. Some rules are quite simple, including only one condition while others are more complex with up to 5 conditions (a cap was set to avoid too complex rules). Just as an example, these are some of the generated rules: The combination of expert and AI rules allow the detection of frauds with a good accuracy, as shown in the confusion matrix (Table 1), which refers to the predictions made by rules on a test set composed by 41458 claims. As indicated in the table, and as is typical in such cases, the Fraud class is underrepresented. However, we achieved a high accuracy for both classes, with a Youden index [14] of 0.69.

Based on their frequency in the rules, LLM also produces a ranking of the most importance features, with the associated relevance score, which is shown in Figure 3. For this problem the most relevant features were the policy class, the damage class and the accident date (in particular, compared with the request date).

In the realm of Artificial Intelligence (AI), transparency is paramount to foster trust and accountability in decision-making processes. On the other hand, rules are how humans reason and make decisions. For this reason, rules play a crucial role in achieving transparency by providing a clear and interpretable framework for decision-making. Unlike black-box algorithms that operate with little visibility into their inner workings, rule-based systems offer a transparent and understandable approach to decision-making. Since rules provide a straightforward way to understand decision logic, they enable stakeholders to understand how and why certain outcomes are reached. By allowing stakeholders to evaluate the fairness and consistency of decisions, this transparency is particularly valuable in high-stakes domains such as healthcare, finance, and criminal justice, where the implications of decisions can have significant real-world consequences.

Moreover, rules enable human oversight and intervention, allowing experts to integrate their domain knowledge, validate decisions and correct errors when necessary. This human-in-theloop approach ensures that AI systems remain aligned with ethical and regulatory standards, mitigating the risk of unintended biases or discriminatory outcomes.