Automatic fault type classi cation is an important ingredient of smart electrical grids. Similar to other machine-learning models, methods developed for fault classi cation su er from the issue of lack of transparency. This work sheds light on preliminary insights of an ongoing study, in which we show how feature importance measurement and feature interaction visualization using partial dependence plots (PDPs) can help interpretability of the classi cation outcomes. While the former, measures the role of each feature on the nal predictions in isolation, the latter focuses on mutual interaction between pairs of features. We show the merits of these two complementary feature analysis mechanisms in facilitating interpretability of the fault type classi cation task.
Smart grids (SGs) are recognized as power distribution systems (PDSs) that
need to possess traits including high reliability, e ciency, and penetration of
renewable energy sources [
A common characteristic of the prior literature is that the nature of the empirical experiments carried out orients toward the prediction aspect of the fault event, aiming to nd an answer to questions such as \is it possible to detect a fault using ML techniques reliably "? or \which classi cation technique can more accurately predict a class type? " and so forth. Regretfully, such trends for full automation of PDS's self-healing capability are not designed to inform human operators who have relied on manual/visual awareness for a long time. To keep humans involved in the control loop, it is crucial to design interpretable ML models that can replace these black-box prediction models and to produce rules that can be understood with little inspection.
Motivated by this observation, the work at hand puts its attention outside the subject of proposing another classi cation method for fault prediction, instead it tries to focus on the central question \Given popular classi cation techniques already recognized by the community, is it possible to exploit the results of predictions in order to obtain more interpretable outcomes? "
The contributions of this work are two-fold:
1. Feature extraction and representation: we rely on features extracted
from the three-phase voltage signals, represented in both time and frequency
(transform) domains. For feature representation, we compute the n-th
moment of the probability distribution functions (PDFs) [
The results of our empirical study show that in general, the computed
features in this work are not only descriminative for our classi cation scenario,
but are also easily interpretable, making the classi cation process transparent.
While previous works have exploited features coming from signal or transform
domains [
IEEE-13 Node test feeder (distribution grid)
Fault injection Measurements of three-phase voltage signals from the FZ
Multi-class single label
Classification AG BG CG AB BC AC ABC
Faulty zone (FZ):
671-680
Extracting Signal-level features
Extracting
Transform-domain features Measurement of features importance Visualization of features interaction impact The goal of the proposed method is two-fold: (i) fault-type classi cation, and (ii) interpretability achieved via feature importance measurement and data visualization. The main processing stages involved in the proposed system are presented in Figure 1. The input to the system is the IEEE-13 node test feeder, while the output is one of the seven fault types, namely: line-to-ground (AG, BG,CG), Line-to-Line (AB, AC, BC), and three-phase fault (ABC). We chose IEEE-13 node test feeder, which includes a voltage generator of 4:16 kvlt and 13 buses for the simulation of fault and measurement of three-phase signals. One can divide this distribution system into four critical zones, zone 1: 632-671, zone 2: 632-633, zone 3: 692-675, and zone 4: 671-680. To collect data, faults were injected to one arbitrarily chosen zone, in this case zone 4, and then features were collected from three-phase voltage signals of this zone. We injected all the 7 di erent faults (i.e., AG, BG, CG, AB, BC, AC, ABC). These faults have been applied at a certain start time t = 0:01 and revoked at time t = 0:02 for all of the fault simulations. Thus, tf = [0:01 0:02] represents the faulty period while th = [0 0:01] characterizes the non-faulty (healthy) period. All the features that were extracted were taken from the faulty period tf were normalized by the same feature extracted from the healthy period th to obtain a relative score. The following two classes of features were extracted: { Signal-level features: Six features were extracted from raw voltage data of three phases. They include the 1st to 4-th moments: mean, standard deviation, skewness, kurtosis together with the energy and the maximum level of the signal. { Transform-domain features: In addition, we extracted features based on discrete Fourier transform (DFT), to obtain richer information about frequency of the signals. After applying DFT, from the computed spectrum we extracted similar features as signal-level features.
In total, 12 (6+6) features were collected to represent the features in our labelled
training dataset. These two set of features constitute the backbone of many
ML systems [
Fault type classi cation and interpretability analysis Fault type classi cation was done by using two main classi ers: decision tree and k-nearest neighbors. We model the classi cation task as a multi-class signal label classi cation | instead of multi-label | since there are more classi ers' choices available for the single-label classi cation task. For interpretability experiment (see next section), we only use decision tree to keep the discussion simple.
Finding important variables (features) helps to discover the main drivers in a supervised learning classi cation task. However, this approach does not produce information about the relationship between input variables and how this relationship impacts the ML model outcome (predictions). The approach envisioned in this work contemplates using: (i) a classical feature importance technique to show the contribution of each feature on predictions individually, and (ii) a partial dependence plot (PDP) to understand the relationship between pairs of input variables and predictions. PDP is calculated after the model is tted on the training data; thus, it is a model-speci c feature importance analysis technique (rather than model-agnostic). For example, in our context a PDP can show whether the probability of certain fault increases with signal energy and kurtosis of the frequency signal, a question whose answer does not seem to be trivial. Furthermore, PDP can establish the type relationship between two features: monotonic, linear, or not related. These are important cues that can help the human operator to better inspect/interpret the black-box fault classi cation predictions with little supervision. 3
The discussion of results is organized into two sections. First, we describe the results of classi cation and next, we describe the impact of two feature analysis techniques on the interpretability of classi cation predictions.
Classi cation: Table 1 summarizes the classi cation results using two classi ers, namely decision tree and k-nearest neighbors, on the basis of a hold-out setting (80%-20%) for training and test set. We can notice that in all the considered experimental cases the average classi cation accuracy is more than 92%, indicating the discriminative power of the features chosen. The best classi cation outcome is achieved for the decision tree with the accuracy of 96.42%. Thus, we use decision tree for the next step.
Feature analysis and interpretability: Results of feature importance analysis are shown in Fig. 2. In particular, Fig 2-a shows the impact of individual features on fault type classi cation predictions. According to the results, the most informative features are (i) from signal-level features : energy, mean and kurtosis, while (ii) from frequency-level features : energy and mean. Thus, the information that this analysis provides is that both signal-level and frequency-level features can play a role in the classi cation predictions.
Fig 2-b and Fig 2-c however provide a more meticulous interpretation of the results. These plots are results of utilizing the PDP approach (see Section 2.2) and visualize the impact of mutual feature interactions on the classi cation outcome. We can note that the two selected features (as an example) in Fig 2-b, i.e., mean dft and energy sig are NOT mutually informative; in other words, a change in the values of both of these features does not lead to the increase or decrease in the classi cation outcome. This is equal to say that mean dft has all the necessary information encoded in the set fmean dft, energy sigg. Thus, we can safely use mean dft for the classi cation task and expect to obtain good classi cation results. However, as shown in Fig 2-c, for what concerns the interaction between features fmean dft, kurtosis sigg a di erent relation is obtained. We can note that, in this case, both of the features monotonically impact the classi cation predictions. The highest classi cation is achieved when feature values are in the bottom-left portion of the gure.
We round o this discussion by highlighting that the results of our study show that the information provided by the PDP analysis for the SG fault type classi cation task o er new insights that could not be obtained from the classical feature importance analysis technique, as shown in Fig 2-a. For example, while Fig 2-a reports on the impact of the 12 employed features as a group, it does not provide speci c insights if the same results could be obtained when a smaller set of features are used. We can see that while some pairs of features are mutually complementary such as mean dft and energy sig, there exist other (a)
Low mean_dft (b) High
Low
High mean_dft (c) feature pairs that are correlated. This information could eventually be used by the system designer to know (i) which feature(s) to focus on for the extraction phase from the SG signals, (ii) how to represent the feature to obtain more informative features (e.g., n-th PDF moment we used), and (iii) by the system human operator to understand the root of speci c faults in the system. 4
This work presented preliminary results of a large study, in which we focused
on the central question of interpretability of ML models in the context of fault
prediction for smart grids. First, we classi ed fault types using two di erent
classi ers, k-nearest neighbors and decision tree, and identi ed decision tree as
the best choice; afterwards, for the interpretability task, we studied the role
of two complementary feature analysis techniques, namely feature importance
measurement and feature interaction visualization using partial dependence plots
(PDPs). We provided insights that can be obtained from the PDP technique
on the relationship between features, that could not be found in the classical
approach. Our study acknowledges merits of the two complementary feature
analysis mechanisms in facilitating o ering explanations. For the future work,
we plan to extend our dataset by injecting fault to other critical zones, and using
a wider set of features. We plan to experiment with larger electrical grids, e.g.,
IEEE-34, 37 and 123 that are commonly used in the literature [
This work has been partially funded by e-distribuzione S.p.A company, Italy, through a PhD scholarship granted to Fatemeh Nazary.