This paper presents the implementation of a classi cation system based on learning of association rules in conjunction with Drools rule engine. The rules are interactively discovered with a web-based data mining system EasyMiner.eu. The rules are approved and edited by the domain expert before they are deployed for classi cation.
Today, there is an increasing demand for decision support systems (DSS). The penetration of DSS solutions to many domains is sti ed by the fact that building a DSS requires a signi cant amount of time from users, who need to be not only domain experts, but also skilled knowledge engineers. While algorithmic support for decision making support is a complex endeavour, many problems can be cast as a classi cation task for which a model can be induced automatically from data. Example of such tasks include answering questions such as: "Is this product appropriate for this customer?", or "Is the user interested in this media content?"
While a plethora of machine learning algorithms exist, most of them produce \black box" models which are di cult to comprehend and manipulate. Decision tree induction and rule induction su er from this problem, perhaps, the least. The advantage of these models that the they consists of individual units of knowledge (rules),1 which can be separately viewed, evaluated and manipulated { deleted or even edited.
This paper presents a complete implementation of a prototype decision support system, which: { automatically learns decision rules from data, { allows the domain expert to edit the resulting rule base, { deploys the rules to a Business Rule Management System (BRMS), which can apply the rules to score incoming objects. 1 Decision trees can be converted to rules.
The system is built on top of the data mining system EasyMiner and a BRMS system Drools Expert.
This paper is organized as follows: Section 2 presents the motivation and related work. Section 3 describes individual steps of the preparation process of a classi cation rule set. Section 4 covers the implementation. The conclusion presents the technical details of the demo and with an outlook for further work. 2
We motivate our technological solution with the following use case.
Use case example. Alice works as a business analyst for a small community bank. She is assigned a task to prepare a credit scoring application. She has at her disposal a dataset describing loans awarded and the rating of the loan after a one year period. Applying a rule learning algorithm yields a classi cation model consisting of 100 rules.
Looking at the rules, Alice is not absolutely satis ed. She deletes some of the rules, which seem to be artifacts of the data, and introduces new rules that represent well-known relations between client demographics and rating that were omitted by the learning algorithm. After verifying that the accuracy of the edited rule set increased on a holdout dataset, she deploys the model.
If Alice would have to rely on commodity rule learning systems, she would have to probably export the rule learning results into the PMML format2, edit the resulting XML, and import the result to a scoring engine. The system presented in this paper allows to accomplish the task through a chain of web-based graphical tools, which are built on top of industry standard BRMS software and open formats.
Research of "business rules" generation from data mining results is a current,
yet not very exposed, research topic. To the best of our knowledge, the only
publicly available software solution to business rule learning is Rule learner [
However, there are several experimental systems developed in the machine
learning community with related functionality. MIME Framework [
The EasyMiner system [
BigML5 is perhaps the most well-known representative of web-based tools. BigML uses decision tree induction, but allows to export the resulting tree also into a rule set.
Here we present EasyMiner-BR, as an extension of the EasyMiner system. Compared to other software packages described above, EasyMiner-BR covers the entire work ow in a single environment. Within one web application, the user can launch multiple rule learning tasks, export selected rules to the knowledge base (business rule base), edit the saved rules and apply the rules on test data.
The brCBA rule learning algorithm used in EasyMiner-BR for building
classi ers from association rules is evaluated and described in detail in [
The process of preparing a rule base in EasyMiner-BR can be divided into several consecutive steps which are presented on Figure 1: 1) data preparation, 2) association rule learning, 3) rule selection, 4) classi cation model testing, 5) rule set editing and 6) deployment of rules to BRMS. These steps, which can be performed with a standard web browser, are described in the following. 3.1
The user starts with a dataset for training of the classi cation model. Typically, the dataset is divided into a training and a testing part. Currently, the system does not support an automatic split of the input table into a testing and training subset, the user needs to perform this task with other tools.
The dataset for model training (association rule learning) can be stored in a CSV le or in a MySQL database. The user inputs connection information or uploads a CSV le. While uploading, the user is able to select le encoding, separators etc.
For association rule learning, it is usually necessary to discretize numerical
columns. The system supports several types of preprocessing. For numerical
columns, the user can de ne preprocessing using intervals. For nominal as well
as numerical columns, the user can merge values by enumeration, creating bins
of values. More details can be found in [
Fig. 1. Rule base preparation for the classi cation task 3.2
As a rule learning algorithm, the current version uses the GUHA procedure
ASSOC implemented in LISp-Miner system [
To start association rule learning in EasyMiner6, the user is required to input a rule pattern de nition. The user selects attributes from attributes palette and places them into a rule pattern pane, which de nes the pattern that the discovered rules must match. By default, the attributes are connected with a conjunction, but the system supports also disjunction and negation connectives.
For each attribute in the rule pattern it is also possible to de ne the set of its values considered during mining. Supported options are xed value (the user selects one value or value group de ned in the preprocessing phase), simple wildcard (the system tries all possible values) and dynamic binning wildcard (the system creates new bins by merging those created during preprocessing). 7
Additionally, the user needs to select at least one interest measure. The system supports the standard con dence, support, in addition to several other measures. 3.3
After association rule learning has been executed, the user is presented with a list of rules that have been discovered in the data. From this list, the user can select association rules and save them into the Rule clipboard. The user can then change the parameters of the rule learning task, typically by making a change in the rule pattern, and re-execute the learning. In this way, the rule clipboard accumulates rules discovered during multiple data mining tasks. The rules are saved grouped by data mining tasks.
For classi cation, it is vital that the rules that enter the clipboard are chosen so as to maximize accuracy on the data. To support the user in creating the best mix of rules, the system o ers a "Check model" link, which displays the accuracy of the rule set.
The accuracy is computed either on the training dataset or on a separate test dataset (uploaded as a CSV le) depending on the user's choice. Upon clicking on the check model link, the system exports the rules saved in the Rule Clipboard to the Model Tester component. Model Tester converts the rules into business rules form (speci cally DRL) and uses the Drools Expert 8 inference engine to perform the classi cation.
In case of a rule con ict, which occurs if one object is matched by multiple rules, the con ict resolution strategy is as follows: matching rules are sorted according to their con dence, support and antecedent length (in this order). Out of the matching rules, only the highest ranked one is used for classi cation.
7 Dynamic binning is called "coe cient" setup in the LISp-Miner system, and is
described in greater detail e.g. in [
The conversion to the DRL format is described in detail in [
The result from the Model Tester is presented to the user in a graphical form (Figure 2). The user can see the counts of rows classi ed as true positive and false positive, as well as accuracy and accuracy (excl. unmatched) . The di erence between the latter two statistics is that accuracy (excl. unmatched) does not consider objects that were not matched by any rule as misclassi ed, while accuracy does. To aid the user in optimizing the rule base, the system shows also detailed results for each rule from the Rule Clipboard. 3.4
For classi cation model building, it is suitable to combine results (association rules) selected from more than one data mining task. The user can export rules from the Rule Clipboard into the "business rules" base. For practical purposes, the rules are internally saved in an XML format. The system allows to convert the rules into the Drools DRL format.
The user can display the rules saved in the "business rules" knowledge base and edit them using a web interface. We developed a simple rule editor, which presents the rules in a user-friendly, SBVR-like,9 textual form (see Listing 1.1). 9 Semantics of Business Vocabulary and Rules - http://www.omg.org/spec/SBVR/
Listing 1.1. Format of rules supported in BR Editor When age i s i n [ 4 6 ; 5 0 . 5 ) then r a t i n g i s C .
When age i s g r e a t e r than o r e q u a l s 46
and i s l o w e r than 5 0 . 5 then r a t i n g i s C .
The rule editor (Fig. 3) allows the user to edit the rules exported from the rule clipboard, delete them, or add new rules.
Finally, the business rules base can be used for classi cation. The classi
cation engine, available as a REST service, is based on Drools.
EasyMiner (easyminer.eu) is a web-based data mining system [
LISp-Miner (lispminer.vse.cz) is used for rule learning. It is a Windows
desktop application written in C++. The EasyMiner LMConnect module,
implemented in C#, exposes LISp-Miner via REST web services. The communication
is performed in an extension of PMML [
Model tester is written in Java EE. The system uses Drools Expert classes
and the functionality is accessible through REST web services in XML or JSON.
The system creates a brCBA classi er [
Business rules editor is implemented as a JavaScript application, which uses jQuery JavaScript library and HTML5, with PHP backend.
The demo of EasyMiner-BR, including screencasts, is accessible at http:// easyminer.eu/. We are currently nishing the two-way integration of Business rules editor with EasyMiner system. The goal of this integration is to present the user-de ned rules along the mined rules in Rule clipboard, which would allow a more informed selection of rules to the clipboard.
In the future work, we would like to explore the possibility of using the
business rule knowledge base already in the process of data mining of association
rules, e.g. as investigated at [
This work is supported by the University of Economics in Prague by grants no. IGA 20/2013 and institutional support (IP 400040).