We introduce the concept of piecewise-constant reflectance spectra as a tool for modelling autocorrelation statistics in the context of reflectance and colour-signal spectra. By representing reflectance spectra as piecewiseconstant functions, we show that the corresponding spectral reflectance autocorrelation matrices are very similar to those found for real datasets and, moreover, we provide a tuning parameter so that the fit is almost perfect. We apply the idea in the context of camera characterisation and demonstrate that algorithms founded on these new piecewise-constant reflectances, via their autocorrelation, deliver excellent colour calibration, on par with using the measured reflectances themselves. The importance of this work is that we can exactly characterise the set of all piecewise constant reflectances, which is useful because we cannot measure all possible reflectances that might be encountered in the real world.
Piecewise-constant functions have a variety of applications in mathematics and science. For example, it has recently been shown that the autocorrelation of albedo pixel values found in paths through real images can be modelled as the autocorrelation of paths through Mondrians [1]. That is, the real paths of pixel values in images can be smooth (piecewise linear), yet from an autocorrelation perspective can be modelled as being piecewise constant. An application of this result lies in lightness processing, where the optimal filter chosen can be formulated as a least-squares fit that depends only on the autocorrelation matrices [2]. Importantly, we can, by mathematical means, model all possible Mondrians but we cannot record all possible scenes.
In this paper we transfer the idea of paths in images, by analogy, to the spectral domain where we now represent reflectances by piecewise-constant functions. In the context of reflectance and colour-signal spectra, Logvinenko previously used a specific type of piecewise-constant function to model metameric reflectance spectra [ 3], but here we introduce the concept of general piecewise-constant spectra as a tool for modelling autocorrelation statistics.
One important application is device characterisation. Colour devices typically output a set of RGB values in response to colour stimuli. For a device to be colorimetric, these outputs should be linearly related to the XYZ tristimulus values of the colour stimuli, which is known as the Luther-Ives condition [4]. In practice, colour devices are only approximately colorimetric. Nevertheless, techniques such as least-squares regression can be used to find the transform that best maps device RGB values to XYZ. A common approach is to determine the best 3× 3 matrix, which is known as a device characterisation or colour correction matrix [5]. Significantly, the regression depends only upon the autocorrelation statistics of the reflectance and illumination spectra.
In the case of a camera, the device RGBs of interest are the linear raw RGBs. There are two main approaches to determining the matrix transform, both of which are described in the ISO 17321 stan R 3 × matrix containing the camera spectral responsivities. Each row is a colour channel. 3 × matrix containing the ( ), ( ), ( ) colour-matching functions. × matrix, where each row is a reflectance spectrum and each column is a particular wavelength. × diagonal matrix containing the illuminant. × colour-signal matrix defined by = . × 3 matrix where each row is a triplet of XYZ tristimulus values. × 3 matrix where each row is a triplet of linear raw RGB values.
3 × 3 camera characterisation matrix. dard [6]. One approach is to photograph a chart of colour patches under specified illumination. The disadvantage is that only a relatively small number of reflectance spectra can be selectively included in the optimisation, and an experimental set-up is needed for the illumination. However, an alternative computational/statistical approach can be taken provided we have knowledge of the camera spectral responsivities (camera response functions). If these are known, then an infinite set of spectra can be included in the optimisation. The power of this approach is that desirable characteristics can be imposed on the set of spectra that could, for example, be related to assumed real-world conditions [7, 8, 9, 10, 11, 12, 13].
It is the above statistical approach that we address in this paper as a first application of our piecewiseconstant reflectance spectra. Below, we complete this introduction by summarising the statistical approach and describing existing methods such as Maximum Ignorance [7], Maximum Ignorance with Positivity [8], and Minimal Knowledge [11]. In the Method section we introduce a statistical model for our piecewise-constant spectra and derive an expression for the autocorrelation matrix. A key property of our method is that the autocorrelation matrix models an infinite amount of spectra that adhere to our model assumptions. In the Results section, we tune our model and evaluate the colour characterisation performance in relation to that obtained using real spectral reflectance datasets. We show that the autocorrelation matrix of the spectral reflectance datasets often used in colour science can be represented by that of piecewise-constant spectra. 1.1. Summary: Least-squares camera characterisation (statistical method) Given the notation listed in Table 1, let us assume that we have a set of colour-calibration reflectance spectra contained in an × matrix , where each of the columns represents a wavelength sample in the spectral passband. Under an illuminant, which mathematically we model by an × diagonal matrix where the power per wavelength is on the matrix diagonal, the human observer responses are proportional to the following × 3 matrix of responses, ∝ X⊤, ∝ R⊤, where X is the 3 × matrix containing the elements of the CIE colour-matching functions. The ⊤ symbol denotes the matrix transpose. Each of the rows of are a triplet of XYZ values.
Analogously, the camera response to the same set of reflectance spectra under the illuminant is
proportional to the × 3 matrix
where R is a 3 × matrix recording the camera spectral responsivities. Each of the rows of are a
triplet of linear raw RGB values.
(
⊤ = ()⊤ = ⊤(⊤) = (⊤), where the final step utilises the fact that is diagonal. Substituting into Eq. (5) yields the following ifnal result, = ︁( R (⊤) R⊤ ︁) − 1
R (⊤) X⊤.
The absolute values of the entries in will depend upon the values of the constants of proportionality
appearing in Eqs. (
From Eq. (7), it can be seen that the matrix that contains the reflectance spectra only appears in the form of its × inner product or autocorrelation matrix, ⊤. Rather than describe the individual reflectance spectra themselves, ⊤ describes how statistically correlated they are at diferent wavelengths. Its matrix elements are defined by (3) (4) (5) (6) (7) (8) (9) (10)
The goal of camera characterisation, sometimes referred to as colour correction, is to find a 3 × 3 matrix that best maps the rows of linear RGB values to the rows of XYZ values under the characterisation illuminant, The least-squares solution is given by
≈ .
= (⊤)− 1 ⊤. ⎡ 1 ⎢⎢ 21 ⎢ 2 1 = ⎢ .
⎢ . ⎢ . ⎢ ⎣ 1 1 12 1 2 · · · · · · 1 = ︁( R ⊤ R⊤︁) − 1
As mentioned in the introduction, it has been argued that we might define these matrices by appealing to mathematical and physical arguments rather than by calculating them from a single representative spectral dataset. The advantage of this theoretical approach is that an infinite set of spectral data can be modelled [7, 13].
[︁⊤]︁
= + · , = · · , where is the autocovariance between the spectra at wavelengths and , and is the mean of the spectra at wavelength [11]. Here autocorrelation has been defined to include normalisation by the number of reflectance spectra, .
The autocovariance can also be written in terms of Pearson’s correlation coeficients, , as follows, where is the standard deviation of the spectra at wavelength . These coeficients can be interpreted as the autocovariance normalised with respect to the standard deviations, and so the correlation matrix, , has the following form,
In order to populate the autocorrelation matrix, the simplest assumption is Maximum Ignorance (MI) [7], which assumes that an infinite set of reflectance spectra are created by a random process, are equally likely, uncorrelated, and bounded between -1 and 1 [8]. In this case, the characterisation matrix turns out simply to be the linear combination of the camera response functions that lie closest (in terms of minimising error) to the ( ), ( ), ( ) colour-matching functions [7, 8].
For a uniform probability distribution in the range [− 1, 1], the mean = 0 and variance 2 = 1/2. Since = 0 for ̸=, this leads to an autocorrelation matrix with value 1/2 along the main diagonal and 0 everywhere else. 1.4. Maximum Ignorance with Positivity (MIP) Maximum Ignorance with Positivity (MIP) [8] enforces the condition that only positive (i.e. physically realisable) powers are included. This leads to better results and a more accurate white calibration [12]. For a uniform probability distribution in the range [0, 1], the mean = 1/2 and the variance 2 = 1/12, and so the autocorrelation matrix has values of 1/3 on the diagonal and 1/4 elsewhere. This is illustrated in Fig. 1(a).
Note that MI and MIP have traditionally been applied to colour signal spectra [7, 8], which include the illumination. In other words, the MI and MIP assumptions have been applied to the colour signal autocorrelation matrix ⊤, in which case the resulting characterisation matrix will be illumination independent. However, by separating out a fixed illuminant using Eq. (6), MI and MIP can be applied only to the reflectance spectra, in which case will be illumination dependent.
The autocorrelation matrices for both the MI and MIP assumptions are examples of Toeplitz matrices [15], where elements along a given matrix diagonal take the same value. A “diagonal” could be the main diagonal with the elements 1,1, 2,2, · · · , or it could, for example, be ,1, +1,2, . . . , ,− +1. In this paper, all Toeplitz matrices are symmetric and so corresponding diagonals each side of the main diagonal are identical.
It has been argued that real-world reflectance spectra tend to have certain smoothness properties and hence are not completely uncorrelated [11]. The Minimal Knowledge (MK) [11] model by Viggiano includes short-range correlations between the reflectance spectra, > 0 for ̸=, that only depend upon the magnitude of the diference between wavelengths rather than their location in the spectral passband. For example, the diference between spectral reflectance at 500 and 510 nm is likely to ⊤ remains symmetric Toeplitz so that elements along the same diagonal again take the same value. Stronger correlation implies that the matrix elements will require a longer separation from the matrix main diagonal before dropping to their minimum value of 1/4.
MK arbitrarily uses the following Cauchy form for the Pearson correlation matrix elements, =
2 2 + ( − ) 2 , where is an adjustable parameter that controls the wavelength interval at which drops from 1 to 1/2 [11]. This can be substituted into Eqs. (8) and (9) to obtain the autocorrelation matrix, ⊤, under the MK assumptions. For a constant mean = 1/2 and variance 2 = 1/12, this will also be a Toeplitz matrix, [︁⊤]︁ = 1 4 + 1 ︃(
2 12 2 + ( − ) 2 )︃
Finlayson and Paul [12] observed that the autocorrelation matrix for real spectral reflectance datasets is not actually Toeplitz in general. For example, Fig. 2(a) shows the autocorrelation matrix for the object170 dataset, which is the set of 170 object reflectances measured by Vrhel et al. [16], the dataset used by Viggiano in the original MK publication [11]. Although the MK autocorrelation matrix with a suitable choice of is clearly closer to the real matrix than both the MI and MIP matrices are, the real matrix clearly has higher autocorrelation at higher wavelengths, which is not seen in the Toeplitz MK structure. This was not revealed in Ref. [11].
However, Finlayson and Paul [12] showed that if we calculate the Pearson correlation matrix defined by Eq. (10) for the real dataset as shown in Fig. 2(b), this does indeed turn out to look much more like a Toeplitz matrix [12]. In other words, the MK assumption is valid if we think in terms of Pearson correlation. This can be attributed to the fact that MK assumes the mean and standard deviation terms appearing in Eq. (8), and , to be constants, but this is clearly not true for real data. (11) (12)
In a future publication, we intend to develop a model that allows and to vary as a function of wavelength, as this would enable the height of the autocorrelation matrix to increase as the wavelength increases [17]. However, an alternative way forward, which was the approach adopted by Finlayson and Paul [12] and is also the approach we will adopt in this paper, is to apply the MK assumptions only to the Pearson correlation matrix. Moreover, the conversion of autocorrelation matrix for real data into its corresponding Pearson correlation matrix has a physical interpretation. Since real datasets, such as that shown in Fig. 2(a), have higher autocorrelation in the longer wavelengths, if we illuminate the reflectance spectra with a bluish light, this dampens down the autocorrelation and, in efect, transforms the data to be more Toeplitz. A slightly modified version of this process can be expressed as follows, (⊤ − ⊤ ) = , as a diagonal matrix with elements defined by where is the 1 × vector containing the mean reflectance spectrum, and denotes the blueish light = 1
. ︂) − 1 Consequently, the characterisation matrix described by Eq. (7) can be rewritten as = ︂(
R (︁ + ( ⊤ ) ︁)
R⊤
R (︁ + ( ⊤ ) ︁)
X⊤. (Note that Finlayson and Paul [12] set to be zero in their calculations by including − in the set of reflectance spectra). In other words, if we wish to calculate a camera characterisation matrix optimised for an illuminant denoted such as D65, then we can apply the MK assumptions to the Pearson correlation matrix, , instead of ⊤ provided we use a normalised illuminant / . This is significant because for real data is approximately Toeplitz, as illustrated in Fig. 2(b), and so the MK assumptions now hold.
(13) (14) (15) (16) MK has been shown to give improved results compared to MIP [11], particularly when using normalised illumination [12]. Nevertheless, one unsatisfactory aspect is the arbitrary manner in which correlation is included via Eq. (11). For example, the Cauchy function does not have any obvious physical basis, and correlation cannot be greater than 1/2 for the maximum wavelength diference of 300 nm. Indeed, the actual spectra that would lead to correlation with this functional form remain unspecified.
As mentioned in the introduction, it was recently shown that the autocorrelation of albedo pixel values found in paths (scan lines) through real image datasets can be modelled as the autocorrelation of paths through Mondrian images [1]. This means that in terms of autocorrelation, real paths, which are generally smooth (piecewise linear), can actually be modelled as being piecewise constant. The degree of correlation, which is measured by the expected length of the piecewise-constant regions, can be related to the degree of correlation in real image datasets.
In order to transfer the above idea to the spectral domain, let be the number of wavelengths in the sampled reflectance spectra. For example, = 301 if the spectral passband is taken to be between 400-700 nm and sampled at 1 nm intervals. Let ( ) denote the spectral reflectance value at wavelength sample . In this example, these can take the following values,
= 399 + , = 1, 2 · · · 301.
Let us normalise all reflectance spectra to the range [, ],
≤ ( ) ≤ .
In order to generate an infinite set of piecewise-constant reflectance spectra, we introduce the following construction.
Let ( 1) take a random value in the range [, ] according to a specified probability density function, (). Now let us build in correlation between adjacent samples by imposing the condition that the probability ( 2) (where 2 is the subsequent wavelength) takes the same value as ( 1) be , where (Here we use rather than as used in Ref. [1] in order to avoid confusion with the used by Viggiano in Eq. (10)). The probability that ( 2) takes a diferent value to ( 1) (drawn from the range [, ] according to the probability density function ()) is given by Clearly, Eq. (18) describes a “step” (of length two samples), whereas Eq. (20) describes a “jump”. Repeating the above process at all wavelength samples in the range specified by Eq. (16) defines an algorithm that can be implemented numerically in order to randomly generate as many reflectance spectra as desired. The degree of correlation between reflectance values at diferent wavelength samples will depend upon the value assigned to . (This will afect the expected or average value for the step length, which will be discussed further below). Figure 3 shows examples of randomly generated piecewise constant spectral reflectances for two diferent values.
In order to derive a closed-form expression for the spectral reflectance autocorrelation matrix elements, consider the process described above in the context of two general samples and . The probability that ( ) will be the same as ( ) for all || < ≤ | |, i.e. we have a step of length | − | samples, is given by
(( ) = ( ), || < ≤ | |) = |− |.
The probability that ( ) is diferent to ( ) is necessarily (( ) ̸= ( )) = 1 − |− |.
(( 2) = ( 1)) = , (( 2) ̸= ( 1)) = 1 − . (18) (19) (20) (21)
For a uniform probability distribution, the probability density function is given by () =
1 − , with mean = (+)/2. Substituting into Eq. (23) and performing the integration leads to the following result, If [, ] = [0, 1], then [︁⊤]︁ = [︁⊤]︁ 2 + + 2 3 = 1 3 |− | + ( + )2 (︁ 4
1 − |− |)︁ . |− | + 1 (︁ 4 1 − |− |)︁ .
(23) (24) (25) (26) (27) (28) Note that |− | = 1 if = 0 and = . If piecewise-constant reflectance spectra are generated according to the above prescription, the autocorrelation matrix calculated numerically rapidly converges towards this closed-form solution as increases. if = . Therefore
It is possible to derive an expression for the expected step length (denoted here as ⟨⟩) in terms of . To proceed, consider a randomly generated reflectance value, ( ), at wavelength sample . The probability that this reflectance is not extended to a longer step is 1 − , in which case ⟨⟩ = 1. The probability that this reflectance value is only extended by one wavelength sample is (1 − ), in which case ⟨⟩ = 2. Continuing this argument, if ( ) is located wavelength samples away from , the probability that ( ) is extended to a step with total length ⟨⟩ = is − 1(1 − ) if < and − 1 ⟨⟩ = ︃( − 1 =1 (1 − ) ∑︁ − 1
+ − 1 = )︃ 1 − 1 − . the following result, Finite step lengths are obtained provided that 0 ≤ < 1, in which case taking the limit → ∞ yields ⟨⟩ =
1 1 − .
In our calculations, we use Eqs. (25) and (28) in place of Eq. (12). We also use normalised illumination.
For our experiments we used four publicly available spectral reflectance datasets, namely the 170 objects [16], the 1995 compiled surfaces [19], the 1269 matt Munsell chips at 1 nm increments [20], and the 1056 Forest colours (birch, pine, and spruce combined) [21]. All data was restricted to the 400-700 nm range (so that = 301) and interpolated to 1 nm increments.
In order to generate a set of reference XYZ tristimulus values, , for each of the four cases above via
Eq. (
The above data was used to calculate a camera characterisation matrix , optimised for D65 illumination, via Eq. (7). When applied to the same set of linear RGB values above, this matrix will yield a set of approximate XYZ values. We compared these with the reference XYZ values by calculating the colour error Δ in the CIELAB (1976) colour space. The second column of Table 2 lists the mean error for each dataset, which represents the best possible colour characterisation for each set of reflectance spectra using the standard least-squares (LS) method.
Dataset
(
We then sought to tune our model based on piecewise-constant reflectance spectra to obtain the best possible colour calibration results for each of the four cases above. This was achieved by using Eq. (26), which is a Toeplitz matrix, to model the Pearson correlation matrix. Rather than adjust the probability distribution and the limits [, ], we applied scale and ofset parameters to Eq. (26) instead. For example, Fig. 5(a) shows the Pearson correlation matrix for the 170 objects dataset, and Fig. 5(b) shows the closest (in a least-squares sense) Toeplitz matrix based on piecewise-constant spectra. For a more detailed comparison, Fig. 6 shows the cross-section along the main diagonal of the Pearson correlation matrix in relation to the closest Toeplitz matrix. For each of the four spectral reflectance datasets, the closest Toeplitz matrix, along with normalised illumination, was then used to calculate the camera characterisation matrix, , via Eq. (15), again for D65 illumination. In each case, this was applied to the linear RGB values to determine the set of approximate XYZ values, and again these were compared with the reference values.
The third column of Table 2 shows the mean colour error for each dataset using our Toeplitz
approximation with normalised illumination (Toeplitz/norm). The values of and corresponding
expected step length, respectively, used to obtain the fit in each case were as follows: (
We have introduced the concept of general piecewise-constant spectra as a tool for evaluating autocorrelation statistics in the context of colour signals. As an application, we looked at the statistical method for camera characterisation and determined the best possible colour calibration using piecewise-constant spectra for four real-world spectral reflectance datasets. Performance was found to be nearly as good as using the real reflectance datasets themselves. As a future investigation, we could determine an autocorrelation matrix determined from piecewise-constant reflectance spectra that gives the best overall performance for a wide range of real-world conditions.
This study was funded by the University of East Anglia and EPSRC grant EP/S028730/1.