This paper was accepted for publication in the proceedings of PAKDD 2023 [1]. Many areas in which classifiers are applied do not only require high predictive performance, but also interpretability of the predictions. An important category of interpretable symbolic classifiers are rule-based models. Most rule-based algorithms use heuristics to learn the models and are designed for specific classification tasks. Recent studies have explored the use of gradient-based approaches from the neural network literature to learn this type of classifier, combining the strengths of both neural and symbolic learning to leverage the literature of neural networks for rule learning. However, these studies focused on (non-ordered) rule sets, while much of the literature on rule learning uses (ordered) NeSy 2023, 17th International Workshop on Neural-Symbolic Learning and Reasoning, Certosa di Pontignano, Siena, ∗Corresponding author.
rule lists. Rule sets are based on a I F ... T H E N P o s i t i v e ; I F ... T H E N P o s i t i v e ; ( ...) ; E L S E N e g a t i v e structure, where every rule predicts the same class, while rule list follow a I F ... T H E N ...; E L S E I F ... T H E N ...; ( ...) ; E L S E ... pattern, where every rule can predict a diferent class. An interesting aspect of rule lists is that they remain fully interpretable for multi-class classification tasks.
In this work, we extended Qiao et al.’s neural network for learning rule sets [
Our neural network for rule learning (RL-Net) utilizes the neural network’s structure and its gradient-based learning optimization to learn rule lists. It is composed of four layers, as depicted in Figure 1. The first one receives the dataset’s binary features. The input is then connected to the rule layer that learns the conditions composing the diferent rules. The next Italy Input features
Rule layer
Hierarchy layer ReLU ReLU ReLU ReLU
Output layer 0 1 2
IF A and C and not D THEN 1 ELSE IF not B and not E THEN 2 ELSE IF A and D and E THEN 0 ELSE 2 Ternary weights {-,0,+}
Fixed weights {-1,0,1}
Free weights layer expresses the hierarchy among the rules, which is necessary to learn rule lists instead of rule sets. Finally, the output layer gives a class label to each rule. To remain fully interpretable, the activations and weights of the model’s nodes are mostly binary or ternary in order to mimic the behavior of the logical operation A N D and the hierarchy between the rules. The four diferent layers are described in detail in the original paper and the source code is available online1.
RL-Net performs similarly to the state-of-the-art rule-based algorithms on binary and multiclass classification problems. Moreover, we showed that RL-Net can easily be adapted to multi-label learning tasks but cannot fully compete with the widely used algorithms yet. A presentation and discussion of the experiments is provided in the original paper.
Further research includes working on the multi-label classification, taking better advantage of the maximal number of rules that are allowed by the user, and integrating our RL-Net in some developed fields of the neural network literature.
Acknowledgments This work was supported by Service Public de Wallonie Recherche under grant n°2010235 – ARIAC by DIGITALWALLONIA4.AI, and n°2110107 - SERENITY2 by WIN2WAL. We also thank FAPESP, Brazil (Grants No. 2017/21174-8 and 2020/00123-9) for the financial support.