Bayesian methods have become a popular way to incorporate prior knowledge and a notion of uncertainty into machine learning models. At the same time, the complexity of modern machine learning makes it challenging to comprehend a model's reasoning process, let alone express specific prior assumptions in a rigorous manner. While primarily interested in the former issue, recent developments in transparent machine learning could also broaden the range of prior information that we can provide to complex Bayesian models. Inspired by the idea of self-explaining models, this paper introduces a corresponding concept for variational Gaussian Processes. While the proposed method is inherently transparent, the bayesian nature of the underlying Gaussian Process allows to incorporate prior knowledge about the underlying problem. In one sentence, the goal is to let the human expert explain how to solve a supervised learning problem in a language that both the model and the user understand. For now, we evaluate these capabilities on simple problems.
models lack the complexity characteristic as regression coeficients can be used by humans to understand the underlying decision process. On the other hand, we can encode existing knowledge about the modelling task in such models. This is usually done either via Bayesian priors over the coeficients or constrained optimization.
Problem. For complex machine learning models, it is usually not as straightforward to encode existing knowledge as in the linear example. Our goal for this paper is therefore twofold. First, we want to derive an approach that can model complex problems in a transparent manner. Subsequently, we want to be able to exploit the transparent representation to encode existing prior knowledge and use it in the model’s training procedure.
Let us split these goals further into three concrete requirements: Transparency - The solution needs to provide insights into its decision process that can be understood by a suficiently trained domain expert. Flexibility - In order to be useful for complex problems, the proposed approach needs to be flexible, i.e. be able to handle a broad range of functional relations between input and target variables. Teachability - Finally, we need to be able to use an interpretable representation of existing knowledge and align the model’s decision process with that knowledge.
Apart from that, a practically relevant solution should also be able to handle with real-world problem. This implies, in particular, that scalability to reasonably large datasets has to be possible.
Contribution. To achieve the above desiderata, this paper proposes self-explaining variational GPs (SEVGPs). The self-explanatory component aims to solve the transparency requirement. By using the right kernel functions, GPs can handle complex functional relations as demanded under the flexibility specification. Since GPs are part of the family of Bayesian models, they are naturally able to incorporate prior knowledge, i.e. they also fall under the idea teachability. The primary limitation in this regard is the representations in which we are able to express our prior knowledge.
While GP models in their original form are unable to deal with large datasets, there exist many scalable solutions nowadays. Our approach will apply the concept of sparse variational GPs (SVGPs) in order to achieve scalability to big data problems as well.
Related work. The results of [2, 3, 4] directly inspired this approach from an explainability and transparency point of view. In fact, the approach [3] relates to this work in a similar way as GPs relate to SVGPs. However, as will be seen, this paper does not merely provide a scalable variant of the former work via SVGPs.
In addition to the transparency component, our aim is to also create a tool that can be used to provide human expert knowledge via transparent representations. [5, 6, 7, 8] all discuss the potentially beneficial role of expert and domain knowledge in machine learning, yet either mention Bayesian methods only briefly or not at all. Nevertheless, Bayesian non-parametrics have already been applied successfully in countless classical statistical modeling problems with an emphasis on incorporating prior knowledge - see [9] for a variety of examples.
Recent work on functional variational inference as discussed particularly in [10, 11] could be a fruitful step towards a synthesis of meaningful prior models and modern Machine Learning architectures.
Outline. In the next section we conduct a brief recap on transparent machine learning with focus on varying coeficient and self-explaining methods. Thereafter, we proceed similarly for GPs and SVGPs. The fourth section marks the main contribution of this paper where the primary formulas of our approach are exposed and discussed. Experimental validation of the approach is conducted in section five. Finally, we discuss limitations and potential extensions of our methodology in the last section. Proofs and derivations, as well as additional details can be found in the appendix.
In regards to transparency in machine learning, terms like interpretable machine learning or explainable artificial intelligence (XAI) have become quite widespread and popular. However, up to this date, there is still no uniquely accepted definition for many terms in this field. In our context, where we consider supervised learning problems, we will use the following definitions of interpretation and explanation from [12]: Definition 1. An interpretation is the mapping of an abstract concept into a domain that the human can make sense of. An explanation is the collection of features of the interpretable domain, that have contributed for a given example to produce a decision.
The corresponding authors particularly name images and text as interpretable domains. Explanations, on the other hand, could be visualizations that highlight image regions or certain words that contributed in favour of or against a given decision.
As we will see, it makes sense to allow for explanations to also quantify the strength of contribution per interpretable feature. For example, consider a fixed grey-scale image and denote the corresponding vector of the image’s pixels, encoded in the range [0, 1], as ∈ [0, 1]. By introducing a coeficient vector ∈ R with the same dimensionality as , we can derive the usual linear model for a single example
The outcome scalar ∈ R could then be mapped to a valid probability via some monotone,
increasing function : R ↦→ (
= ∑︁ () ()
=1
With respect to the mentioned classification problem, (
Image pixels where () () > 0 contribute towards a positive classification whereas pixels
where () () < 0 contribute towards a negative classification 2. Also pixels where |() ()|
close to zero provide almost no contribution to the outcome and pixel where |() ()| is large
provide large contribution. From now on, let us explicitly name the product () () as the
contribution of the -th feature.
2Notice that we might have to add a constant term to this representation in order to account for cases where
() = 0. Otherwise, the contribution of those features will always be zero. For simplicity though, we will only
consider the model as in (
Obviously, the contribution of each pixel must be able to difer for diferent images. Even
under a mere translation of some baseline image, the corresponding contributions must also shift
accordingly. As a result, the static coeficients as implied in (
= ()
Equation (
More recent work around these ’varying coeficient’ models has been done in [ 2], who considered them, under the umbrella term self-explaining models, for modern machine learning problems like image or text classification. The most important novelty is the replacement of regression splines to model (· ) with a feedforward neural network with output neurons.
The building blocks of GPs, see [14], are a prior distribution over functions, ( ), and a likelihood (| ). Using Bayes’ law, we are interested in a posterior distribution ( |) obtained as ( |) = (| )( ) ()
.
( ) = ( |(· ), (· , · ))
(
The prior distribution is a Gaussian Process, fully specified by (· ) : ↦→ R, typically () = 0, and covariance kernel function (· , · ) : × ↦→ R0+:
We assume the input domain for to be a bounded subset of the real numbers, ⊂ R.
Technically, this invalidates (
Most importantly, we denote the × matrix of input data-points as and the corresponding marginal GP output as = ( ). This allows us to discuss GPs either at their multivariate Gaussian marginal output or as actual random functions. We will switch between both concepts depending on the situation.
A common choice for (· , · ) is the ARD3-kernel
(, ′) = · (− 0.5( − ′)Σ( − ′)))
(
where Σ = (12, ..., 2 ) is a diagonal matrix with entries in R0+ and > 0. For = 1, (
Provided that ( | ) = ∏︀
=1 (|, 2), i.e. training observations are i.i.d. univariate Gaussian conditioned on , it is possible to directly calculate a corresponding posterior distribution for new inputs * as
(* | ) = (* |Λ* , * − Λ* ( + 2)Λ* ) where Λ* = * ( + 2)− 1, * ,() = (* , ), * ,() = (* , * ); is the identity matrix with according dimension.
In order to make GPs feasible for large datasets, the work of [15, 16, 17] developed and refined Sparse Variational Gaussian Processes (SVGPs). SVGPs, introduce a set of so called inducing locations ⊂ and corresponding inducing variables . The resulting posterior distribution, (, |), is then approximated through a variational distribution (, ) = ( | )( ) - often ( ) = ( |, ), = - by maximizing the evidence lower bound (ELBO):
The preceding two sections easily motivate the replacement of the feedforward neural network in self-explaining models by a GP model. For a given matrix of training data and target vector , we obtain the following likelihood model: 4Squared Exponential
= ∑︁ E(| )( ) [log (|)] − ( (, )|| ( , ))
=1
(
(* ) = (* |Λ˜ * , * − Λ˜ * ( − )Λ˜ * ) where Λ˜ * = * −1 . Also, we will write ˜* := Λ˜ * and ˜* := * − Λ˜ * ( − )Λ˜ * . If two input matrices, and each consist of a single datapoint, ˜, ˜ and ˜ can be viewed as the mean and kernel functions of the variational GP, evaluated at and . We then denote the implicit GP mean and kernel functions as ˜(· ) = Λ˜ · and ˜(· , · ) = · − Λ˜ · ( − )Λ˜ · .
This allows us to hide the underlying dependencies on and in our notation and treat the
variational GP as a separate entity from the original GP whose posterior distribution we are
trying to approximate.
(
.
⎡ 1( ) ⎤
⎣ ( )
⎥
⎦
and we explicitly included the input matrix to exemplify the relation to self-explaining
models. Also, let us require independence between the individual GPs. Now, we are dealing
with a linear combination of independent GPs instead of a single one. Combining (
This directly implies the following ELBO:
= E(1,) [︁log (︁ ⃒⃒
1,)︁]︁
−
∑︁ ( (, )|| ( , ))
(
(* |* * 1,)(* 1,)* 1, ⃒ =1 = ︃( ⃒⃒⃒ ∑︁ * ⊙ ˜, ∑︁ (︁
︂(
* ⃒ * *
︁)
⊙ ˜ )︂
*
⊙ + 2 ·
)︃
(
denotes element-wise multiplication, is a unit-diagonal matrix of according dimension and 2 is the variance hyperparameter of the Gaussian likelihood. Finally, we can calculate a posterior distribution of the contribution of the -th feature for a given input vector ∼
︁) · ˜(), ()2 · ˜(, )
Now, let us introduce GPs - ˜ 1, ..., ˜ - with the following finite dimensional marginal distributions: ˜ * ∼ * ⊙ ˜*, * (︁ ︁) ⊙ ˜ )︂ * *
*
with ˜, ˜ the mean vector and kernel Gram-matrices per GP as defined in (
∫︁ =1 =1 * * ˜* ∼ ︃( ∑︁ * ⊙ ˜*, ∑︁ * (︁ * ︁) ⊙ ˜*
)︃ =1 =1
Notice that ˜ yields a self-explaining GP whose -th attribution can easily be queried via the corresponding summand GP, ˜ *.
Our goal now is to use ˜* as a variational posterior distribution for an arbitrary GP by ifnding a set of parameters, namely , (and potential hyperparameters for (· ), (· , · )), for ˜* that minimize
(˜( )||( |)) (16)
with ˜(· ) the GP distribution as defined in (
= E( )[( | )] − (17) where is a so-called measurement set, obtained by sampling uniformly from the space of all possible inputs, . (,) then denotes the union of and via row-wise stacking, i.e.
E() ︀[ (((,))||((,)))]︀
(,) = ︂[ ]︂ . By applying the fELBO to our prior and variational processe, we obtain
inducing variables. Clearly, (
By summing up the ˜ , we obtain yet another GP, ˜, with trivial marginal distribution: (15) (18) (19) 1 =
E˜( )[log ( | )] − E() ︀[ ( (˜,(,), ˜,(,)(,))|| ((,), (,)(,)))]︀ where ˜,(,), ˜,(,)(,) denote the evaluation of the mean vector and Kernel-gram matrix from (15), evaluated at (,). (,), (,)(,) denote the mean vector and Kernelgram matrix of the prior GP, evaluated accordingly.
In essence, this approach allows us to encode functional prior knowledge via the prior GP as usual. By decomposing the variational posterior GP after optimizing (18) into its summand attribution GPs, we obtain a transparent approximation of the true posterior distribution in the tradition of varying coeficient or self-explaining models.
Another promising use-case arises, when we place prior distributions on the attribution GPs themselves, e.g. for arbitrary input :
:= ∼ ( (· ), (· , · ))5 5 can be seen a linear operator on that transforms all finite-dimensional marginals of via ⊙ .
If the respective mean and kernel functions can be decomposed as · () and · (, ), (19) is a GPX problem as discussed before. If this not the case, however, and if we want to retain transparency of the respective posterior distribution, we can approximate the attribution GPs by ˜ 1, ..., ˜ . As in (16) we want to minimize
︁( ˜1 ,...,˜ ︀( 1 , ..., )︀ ⃒⃒⃒ ⃒⃒⃒ ︁( 1 , ..., ⃒⃒⃒ )︁
By invoking (17) again and by the fact that the KL-divergence of the joint distribution between prior and variational GPs decomposes as the sum of the KL divergences for mutually independent GPs, we get: 2 =
[︁ ⃒
E˜1,...,˜ (1 ,...,) log (︁ ⃒⃒ 1 , ..., )︁]︁ − E() [︃ ︁(
︁( ∑︁ ˜ =1 ,(,) , ˜ ,(,) ⃒⃒ ⃒⃒ ︁) ⃒ ⃒ ︁( ,(,), ,(,)(,)) ︁) ]︃
As a brief example, we could choose (· ) << 0 to exemplify the prior belief that the attribution of the -th feature is negative with high probability. Obviously, potential priors could be much more complex. In fact, it might be fruitful to consider implicit processes as introduced in [18] as a prior and use our self-explaining posterior as an approximation.
In this section, we evaluate the proposed method on several experimental tasks. In particular, we are interested in the explanations generated by our method, its ability to incorporate prior assumption and its predictive performance. All experiments were conducted on regression problems, where the likelihood could be assumed to be Gaussian.
Extended implementation details can be found in Appendix B.
In addition to point values for the varying coeficients, the SVGP components allow to also evaluate the variance of varying coeficients. In accordance with the typical interpretation of posterior variance in Bayesian models, this can be interpreted as a measure of coeficient uncertainty or explanation uncertainty.
To evaluate these measures, the coeficient means and variances of a trained SEVGP model
(via (
CHAS). The respective outputs can be used to check for hidden biases or erroneous reasoning in the respective model. (20) (21)
To verify the model’s capability to incorporate existing prior knowledge, a random sample from a quadratic function with gaussian noise was created in the interval [− 2, 2]. A model that is able to handle knowledge about the underlying quadratic function should be able to extrapolate accordingly beyond the range of the observed data (often termed out-of-distribution problem).
In order to validate this claim for our approach, the three models implied in (
The results in Figure 3 indicate that the model is able to correctly handle the functional prior knowledge about the underlying quadratic function. It can be see, that both models that were trained with additional prior knowledge (middle and right) were able to correctly extrapolate the quadratic function. Without such prior knowledge (left model), the resulting posterior predictive distribution only fits the in-sample data but is unable to extrapolate out of distribution.
To validate the predictive performance of the proposed method, it was evaluated over four regression datasets (boston housing, concrete, wine red and wine white) via five-fold cross validation. For comparison, standard SVGP was also trained and evaluated on the same folds. Table 1 shows average MSE and MSE standard deviation over the folds. All GP models used an ARD covariance kernel and zero-mean prior functions.
Since SEVGP uses one SVGP per coeficient, the amount of inducing points in the SVGP was increased accordingly to account for the increased model capacity of SEVGP. See Appendix B for more details.
It can be seen that our proposed method achieves comparable performance to SVGP. This implies that problems where the latter perform well, allow for the SVGP to be replaced by SEVGP in case the discussed benefits are deemed advantageous.
This paper presented a method that combines GPs and recent developments in varyingcoeficient/self-explaining methods for machine learning. By taking advantage of the Bayesian properties of GPs it is also possible to inject prior knowledge into respective models. One area where both the transparency and the teachability aspects can be helpful is the field of fair and unbiased machine learning. On the one hand, transparency allows to detect biased or discriminating results on a per instance basis. On the other hand, teachability could help prevent or eliminate potential biases by carefully encoding non-biasing prior knowledge into the model. While this would certainly not be a silver bullet, there might nevertheless be considerable, general potential at the intersection of explainable and human-in-the-loop machine learning.
A clear limitation is the fact that the idea of explainability that we considered in this paper is a statistical one, with focus on local, per-pixel explanations. In complex problems like image classification, this might not sufice if a class is inferred from multiple symbolic relations of diferent objects that are present in a given image instance. Nevertheless, statistical approaches have recently been shown to be quite successful on such complex problems despite possessing no inherent capabilities for logic deduction.
Future work on the proposed method should try to find a way to make the proposed method scalable to other, potentially high dimensional, supervised learning problems. Particularly problems with image inputs, like image classification or reinforcement learning might greatly benefit from external prior knowledge when training data is only sparsely available. 09421. arXiv:2011.09421. 2017.10.011. doi:10.1016/j.dsp.2017.10.011. [15] M. K. Titsias, Variational learning of inducing variables in sparse gaussian processes, in: D. A. V. Dyk, M. Welling (Eds.), Proceedings of the Twelfth International Conference on Artificial Intelligence and Statistics, AISTATS 2009, Clearwater Beach, Florida, USA, April 16-18, 2009, volume 5 of JMLR Proceedings, JMLR.org, 2009, pp. 567–574. URL: http: [16] J. Hensman, N. Fusi, N. D. Lawrence, Gaussian processes for big data, in: A. Nicholson, P. Smyth (Eds.), Proceedings of the Twenty-Ninth Conference on Uncertainty in Artificial Intelligence, UAI 2013, Bellevue, WA, USA, August 11-15, 2013, AUAI Press, [17] J. Hensman, A. G. de G. Matthews, Z. Ghahramani, Scalable variational gaussian process classification, in: G. Lebanon, S. V. N. Vishwanathan (Eds.), Proceedings of the Eighteenth International Conference on Artificial Intelligence and Statistics, AISTATS 2015, San Diego, California, USA, May 9-12, 2015, volume 38 of JMLR Workshop and Conference Proceedings, JMLR.org, 2015. URL: http://proceedings.mlr.press/v38/hensman15.html. [18] C. Ma, Y. Li, J. M. Hernández-Lobato, Variational implicit processes, in: International Conference on Machine Learning, PMLR, 2019, pp. 4222–4233. URL: https://proceedings. mlr.press/v97/ma19b.html.
A. Derivation of (
︁(
︁( 1,,
⃒ ⃒ 1,)︁ ⃒ ⃒ ︁(
⃒ ⃒ 1,, 1,⃒⃒ ︁) = ∫︁
log ︁( ∫︁ = log
︁( 1, ⃒⃒ ︁( ︁) log log ︁( ∫︁ ︁( ︁( ︁( ︁( ︁( ︁( ︁( ︁( ︁( ︁( ⃒ ⃒ since depends on 1, only via 1, and only on the marginals given by . ⃒ ⃒ ∑︁ ( (, )|| ( , ))
E (˜ 1, ) ∑︁ ( (, )|| ( , )) ∑︁ ( (, )|| ( , )) by marginalizing out 1, and writing ˜ 1, by independence of prior and variational GPs and by standard i.i.d. assumption about observed We write ( 1, ) = ( 1, ..., ) and (
1, ) = (1 , ..., ). Notice that ( 1, ) does technically not exist as it involves the infinite dimensional stochastic processes where densities don’t exist. As these objects will cancel out anyway and since such notation is commonly seen in the GP literature, we will keep it here for simplicity. Otherwise, to be notationally exact, we would have to work with KL divergences over probability measures which would make the results much less convenient to derive.
︁( 1, , ⃒ ⃒ ⃒ ⃒ 1, , 1, ⃒⃒ ︁) = ∫︁ log
︁( 1, ⃒⃒ ︁(
︁( log
︁( 1, ⃒⃒
︁(
︁( log
︁( ︁( ⃒⃒ 1, ,
︁(
︁( 1, )︁ 1, ⃒⃒
︁( ⃒ ⃒ ⃒ ⃒ log ⃒ ⃒ ︁( since depends on 1,
only via 1, and only on the marginals given by .
by independence of prior and variational GPs and by standard i.i.d. assumption about observed datapoints by marginalizing out and writing ˜ 1,
for clarity as explained before.
,
)) −
E ) [︁ () ≥ [︁ log log ︁( ⃒ ˜ ⃒ − ( ( , )|| ( ,
)) ( , )|| (