Recent eforts in interpreting Convolutional Neural Networks (CNNs) focus on translating the activation of CNN filters into a stratified Answer Set Program (ASP) rule-sets. The CNN filters are known to capture high-level image concepts, thus the predicates in the rule-set are mapped to the concept that their corresponding filter represents. Hence, the rule-set exemplifies the decision-making process of the CNN w.r.t the concepts that it learns for any image classification task. These rule-sets help understand the biases in CNNs, although correcting the biases remains a challenge. We introduce a neurosymbolic framework called NeSyBiCor for bias correction in a trained CNN. Given symbolic concepts, as ASP constraints, that the CNN is biased towards, we convert the concepts to their corresponding vector representations. Then, the CNN is retrained using our novel semantic similarity loss that pushes the iflters away from (or towards) learning the desired/undesired concepts. The final ASP rule-set obtained after retraining, satisfies the constraints to a high degree, thus showing the revision in the knowledge of the CNN. We demonstrate that our NeSyBiCor framework successfully corrects the biases of CNNs trained with subsets of classes from the Places dataset while sacrificing minimal accuracy and improving interpretability.
Deep learning models can be biased based on the training data. One infamous illustration of this
bias is exemplified by the “wolf in the snow” problem [
Recent works have shown that it is possible to obtain the knowledge of a trained CNN in the
form of a symbolic rule-set, more specifically as a stratified Answer Set Program ([
We introduce the NeSyBiCor (Neuro-Symbolic Bias Correction) framework, to aid in correcting pre-identified biases of the CNN. The ASP rule-set generated by NeSyFOLD, from the biascorrected CNN serves to validate the efectiveness of the framework. The pre-identified biases are presented as semantic constraints based on concepts that should and should not be used to predict the class of an image. These concepts can be selected by scrutinizing the ruleset generated by NeSyFOLD. We map the undesirable/desirable semantic concepts to their corresponding vector representations learnt by the filters. Next, we retrain the CNN with a novel semantic similarity loss function which is a core component of our framework. The loss function is designed to minimize the similarity of the representations learnt by the filters with the undesirable concepts and maximize the similarity to the desirable ones. Once the retraining is complete, we use the NeSyFOLD framework again to extract the refined rule-set. Hence, our approach provides a way to refine a given ASP rule-set subject to some semantic constraints. Figure 1 illustrates the framework.
Computing Concept Representation Vectors: The first step is to obtain concept representation vectors for each desired and undesired concept specified in the semantic constraints of each target class. A CNN filter can be flattened into a vector representing the detected patterns. For example, in the rul-set (blue box) shown in Fig. 1, the filter associated with the predicate ‘sky1/1’ detects blue sky patterns, while ‘sky2/1’ detects evening sky patterns in desert road images. To compute the concept representation vector for sky, we calculate the filter representation vectors for all predicates containing sky in their name and positively linked to the ‘desert road’ class. For example, ‘sky1/1’ and ‘sky2/1’ have two such filters. Their filter representation vectors are computed by averaging the output vectors of the top-10 images these filters activate in the training set. The final concept representation vector for sky is the mean of the ‘sky1/1’ and ‘sky2/1’ vectors. This process is repeated for every desired and undesired concepts. Calculating the Semantic Similarity Loss: The semantic similarity loss, ℒ for a training set with images and a CNN with filters in the last layer, is defined as:
⎡ ⎡ ⎤⎤ ℒ = ∑︁ ⎣∑︁ ⎣ ∑︁ _(r , r) − ∑︁ _(r , r)⎦⎦ =1 =1 ∈B ∈G (1) Here, cos_sim calculates the cosine similarity between two vectors. r represents the filter output from the ℎ filter of the ℎ image, while r and r are the concept representation vectors for undesired concepts ∈ B and desired concepts ∈ G, respectively. and are hyperparameters that balance the influence of these terms.
The loss increases when filter vectors resemble undesired concepts and decreases when they are closer to desired concepts, similar to the word2vec loss function. As training progresses, the iflters are encouraged to learn desired concepts while avoiding undesired ones. The total loss is defined as the sum of crossentropy loss ℒℰ and semantic similarity loss ℒ , ℒ ℒ = Recalibrating the Concept Representation Vectors: We propose rectifying all the concept representation vectors for each class after every epochs during training. We do this by running the NeSyFOLD framework after every epochs and obtaining a new rule set from the partially retrained CNN. We then aggregate the new concept representation vectors with the old concept representation vectors by taking their mean.
We train a CNN on subsets of the Places [
The undesired concepts for the ‘desert road’ class are ‘sky’ and ‘building’. In RULE-SET 1, rule 2 uses the ‘sky1/1’ predicate to determine if the image belongs to the ‘desert road’ class. In the bias-corrected rule-set, RULE-SET 1*, there is no ‘sky’ based predicate. Moreover, the only predicate positively linked with the ‘desert road’ class is the ‘ground1_road1/1’ predicate which is based on the desired concept ‘ground’ and refers to the corresponding filter, learning a pattern comprising of specific type of patches of ‘ground’ and ‘road’. Thus it is clear that at the end of the bias correction, very few/none of the filters learn representations of the ‘sky’ or ‘building’ concepts, hence correcting the “bias” of the CNN towards them while predicting the “desert road” class.
Our framework, in addition to correcting the bias of a CNN also allows the user to fine-tune the bias based on general concepts according to their specific needs or application. To the best of our knowledge, this is the first method that does bias correction by using the learnt representations of the CNN filters in a targeted manner. We show through our experiments that the bias-corrected rule-set is highly efective at avoiding the classification of images based on undesired concepts. It is also more likely to classify the images based on the desired concepts.
Finally, the work we presented here may be used to extend implementations of loss functions
based on Diferentiable Logics: [