=Paper=
{{Paper
|id=Vol-2688/paper7
|storemode=property
|title=Linear, Subtractive and Logarithmic Optical Mixing Models in Oil Painting
|pdfUrl=https://ceur-ws.org/Vol-2688/paper7.pdf
|volume=Vol-2688
|authors=Federico Grillini,Jean-Baptiste Thomas,Sony George
|dblpUrl=https://dblp.org/rec/conf/cvcs/GrilliniTG20
}}
==Linear, Subtractive and Logarithmic Optical Mixing Models in Oil Painting==
https://ceur-ws.org/Vol-2688/paper7.pdf
Linear, Subtractive and Logarithmic Optical
Mixing Models in Oil Painting
Federico Grillini, Jean-Baptiste Thomas, and Sony George
The Norwegian Colour and Visual Computing Laboratory, Norwegian University of
Science and Technology, GjΓΈvik, Norway
federico.grillini@gmail.com, jean.b.thomas@ntnu.no, sony.george@ntnu.no
Abstract. Identifying the pigments and their abundances in the mix-
tures of one artistβs masterpiece is of fundamental importance for the
preservation of the artifact. The reflectance spectrum of mixtures of
pigments can be described by modeling the spectral signature of each
component, following different rules and physical laws. We analyze and
invert nine different mixing models, in order to perform Spectral Un-
mixing, using as targets two sets of mock-ups. Based on the results of
the spectral reconstruction errors, we are able to point out that three
models are best suited to describe the phenomenon: subtractive model,
its derivation with extra parameters, and the linear model adapted with
extra parameters.
Keywords: Optical mixing models Β· Spectral Unmixing Β· Pigment map-
ping.
1 Introduction
The conservation of Cultural Heritage (CH) is a vital cornerstone on which mod-
ern society is based upon, since it enables the current community to transmit
the inestimable value of knowledge, traditions, uses, and art to the next genera-
tions [1]. During the Age of Post-Enlightenment in the 18π‘β century, CH started
to be valued, as the first museums started their activity in England [2], and
the methods of conservation slowly developed, using the resources coming from
fields of the natural sciences. At the present times, a large body of research
focuses on non-invasive and non-disruptive techniques, to investigate the arti-
facts without making contact with them, and without the extraction of samples.
Techniques such as Infra-Red photography [3], X-Ray Fluorescence [4], Particle
Induced X-ray Emission [5], Fourier Transform Infra-Red spectroscopy [6], Opti-
cal Coherence Tomography [7] and Raman Spectroscopy [8] have been deployed
to study the physical and chemical properties of the artifacts. Some of them rely
on a punctual response, whereas others are scanning methods that provide an
Copyright c 2020 for this paper by its authors. Use permitted under Creative Com-
mons License Attribution 4.0 International (CC BY 4.0). Colour and Visual Com-
puting Symposium 2020, GjΓΈvik, Norway, September 16-17, 2020.
2 F. Grillini et al.
image of the object, carrying information about the specific feature investigated.
The main purpose for which these methodologies are applied is to get the precise
composition of an object; this information can help the conservators to adopt
the best suitable procedure for flawless preservation. These enlisted methods
can carry out examinations and provide results with a high degree of accuracy,
but the instrumentation required is often quite expensive, and the acquisition
sessions complex and long-lasting.
This work focuses on Hyperspectral Imaging (HSI) in the Visible-Near In-
frared (VNIR) region of the electromagnetic spectrum. The mixture of pigments
in oil painting is studied. Historically, in oil painting, pigments in powder form
are blended together with a fluid medium, called binder, usually linseed oil. The
mixture is then applied on a stretched canvas or wooden support, previously
primed, i.e. covered with a preparatory layer, usually made of gesso, to facilitate
the drying process [9].
HSI does not generally allow the examination in depth of an object in the
visible region [10], thus only the radiant light of the outer layers of the artifact
can be observed. The paints applied to create paintings are usually mixtures of
different pigments. The optical properties of the mixture will carry information
about the properties of each one of the components, denominated endmembers
or primaries. Retrieving the endmembers present in a mixture in their rela-
tive concentrations boils down to an inversion problem called Spectral Unmix-
ing (SU). In order to solve this task, several procedures have been proposed,
with the main classification made between methods that exploit the existence
of a spectral library containing the possible endmembers, and methods that re-
quire the implementation of endmember extracting algorithms [11]. The model
that eventually is inverted needs to describe the physical properties of the phe-
nomenon that takes place when different pigments are mixed together. That is
why we strongly believe that the success of SU depends on the accuracy of the
mixing model on which it is based upon. In remote sensing applications, unmix-
ing is often addressed inverting a linear model [12], assuming that the mixture
mainly takes place at the camera level and is the result of optical blending. How-
ever, when pigments are mixed together, the nature of the mixture is intimate,
meaning that the single components are not discernible even with sophisticated
imaging technologies. In close-range applications, the optical resolution is high
enough to safely discard the optical blending at the camera level and consider
only the intimate mixture.
We analyze nine mixing models, and we invert them to perform SU. The
investigations of the models are conducted by creating mockups of mixtures of
different pigments in approximated known proportions, which are captured in
an HS set-up. We then can observe that the linear model and subtractive model,
modified to include extra parameters, describe with a good degree of accuracy
the mixture of pigments in oil painting.
The remainder of this paper is organized as follows: Sec.2 introduces the pro-
posed mixing models, Sec.3 explains the strategies adopted in order to complete
the task of spectral unmixing, Sec.4 describes the materials and methods ap-
Linear, Subtractive and Logarithmic Optical Mixing Models in Oil Painting 3
plied, Sec.5 provides an overview of the results, and finally, Sec.6 gathers a few
concluding remarks.
2 Optical Mixing Models
The radiant light incoming on a sensor is described by the Radiative Transfer
Model, identifying two main components: an illuminant and an object. The ra-
diation emitted by the light source, denominated Spectral Power Distribution
(SPD), impinges on the surface of the object, which is assumed to be Lam-
bertian. Part of the radiation is absorbed by the material, while the remaining
is reflected according to the physical properties of the surface reflectance π(π)
and captured by the sensor. The radiance πΈ(π) impinging on the sensor can be
expressed with the formula:
πΈ(π) = πΏ(π) β
π(π) (1)
This quantity interacts with the sensorβs response functions, which can be the
Camera Sensitivity Function (CSF) in the case of a single-lens reflex camera, to
output a matrix of pixel values. For equal illuminants and sensor responses, π(π)
is the material-related quantity that allows the perception of different colours,
and can be easily extracted in conditions of a controlled environment, by invert-
ing the radiative transfer model.
Mixing is defined when π or more endmembers are combined together in
different concentrations πΌ. The concentration array πΆ = {πΌ1 , πΌ2 , ..., πΌπ } needs to
comply with two constraints dictated by the physics:
1. Non-negativity Constraint (NC). Since negative concentrations donβt have
any physical meaning, all of the concentrations must be equal to or greater
than 0:
β πΌπ β₯ 0, π = 1, ..., π (2)
2. Sum-to-one Constraint (SC). The sum of the concentrations must be equal
to one. If this condition is not met, there would be missing or exceeding
matter in the mix.
β
π
πΌπ = 1 (3)
π=1
For some applications, it is possible to relax the constraints in order to introduce
levels of error tolerance and have the algorithms work faster. However, in this
research work, both NC and SC are treated as hard constraints. The output
of the mixture is a new reflectance that carries information about the single
endmembers and therefore should comply with the fundamental properties of
spectra, such as being included in the interval [0, 1] and being a smooth curve.
It is well known that when the mixing of pigments and dyes is treated, it is
often referred to as a subtractive mixing. The choice of considering an additive
model might then sound incoherent. Although the linear mixing model cannot
4 F. Grillini et al.
accurately describe the intimate type of mixing that occurs at the pigments level
[13], it has been extensively used to solve tasks such as spectral unmixing and
pigment mapping [14], which is why this model has been addressed and labelled
with the code π1 .
A purely subtractive model, labelled π2 , treats images and media as fil-
ters, accordingly to the transmittance model. The reflectance of the mixture is
then obtained as the consecutive products of the single endmembersβ spectra,
implementing the concentrations in a geometric mean fashion [15].
The transmittance model seems appropriate when studying the mixture tak-
ing place on a canvas, since pigments have well-known properties of transmit-
tance, especially in the NIR region. This is why for this study we decided to
adapt the models π1 and π2 to the rules dictated by the Logarithmic Image
Processing (LIP) framework [16]. LIP has a solid physical and human-vision ba-
sis, which allows the completion of several image processing-related tasks with
a good degree of perceptual fidelity. The main contribution is the re-thinking
of an image π (π₯, π¦) as an intensity filter, therefore having an intrinsic transmis-
sion function ππ (π₯, π¦). The adoption of this framework allows to never exceed
the intensity boundaries [0, π] dictated by the encoding system adopted. In LIP,
the basic operators of addition, multiplication by a scalar, and multiplication
between images [17] are revisited and applied to fulfill tasks such as high dy-
namic range, feature extraction, and noise removal [18]. The new operators are
reported in Tab.1.
Table 1: Operators of the Logarithmic Image Processing framework. By adopting the
modifications to standard operators, the maximum encoding value π is never exceeded.
Name Symbol Equation
Image π π (π₯, π¦) = π(1 β ππ (π₯, π¦))
Addition β¨Ή π β¨Ή π = π + π β πππ
( )π
Multiplication by a scalar π ⨻ π ⨻ π = π β π 1 β ππ
π β π = πβ1 (π(π ) β
π(π))
( )
Multiplication between images β π(π₯) = βπ β
ππ 1 β ππ₯
( ( ))
πβ1 (π₯) = π 1 β ππ₯π β ππ₯
The LIP framework, originally thought for 8-bit images with π = 255, can be
translated to reflectances with significant simplifications, since in this new case
Linear, Subtractive and Logarithmic Optical Mixing Models in Oil Painting 5
π = 1. The models π3 and π4 are derived from π1 and π2 and are therefore
called LIP additive and LIP subtractive, respectively.
Purely additive and purely subtractive models are defined as ideal, and a variety
of intermediate models between the two can be formulated. These new models
rely on a parameter that explains the balance between an additive and a sub-
tractive mixture. The mixing parameter π can span between a value of 0, to
represent a purely subtractive model, to a value of 1, which indicates a purely
additive one. Three intermediate models are formulated in the literature:
β π5 Yule-Nielsen model [19]. It has been proposed in the study of half-toning,
since neither subtractive nor additive models could explain such colors. It
approximates the subtractive models when π approaches 0 asymptotically
(when π is exactly zero, the function goes to the infinity),
β π6 Additive-Subtractive model [20]. It privileges the additive mixture, even
if π takes on values β€ 0.5,
β π7 Subtractive-Additive model [20]. Conversely to the previous one, it priv-
ileges the subtractive mixtures.
From a preliminary observation of the spectra we acquired, we noted a prop-
erty that is never achievable by the models proposed so far. One common trait
followed by these models is that the reflectance curve of a mixture is always in-
cluded within the boundaries of the endmembersβ curves. In real-case scenarios,
this is generally the case (Fig.1a), but there exist instances in which this rule
is not followed (Fig.1b). For this reason, we propose 2 new models, labeled π8
0.7 1
BY Rb
0.6 B R
Y 0.8 B
0.5
0.6
0.4
( )
( )
0.3
0.4
0.2
0.2
0.1
0 0
400 500 600 700 800 900 1000 400 500 600 700 800 900 1000
[nm] [nm]
(a) (b)
Fig. 1: Comparison of acquired spectra. The endmembers are reported with dashed
lines, whereas their mixtures in an approximated 1:1 ratio are represented with solid
lines. (a): the reflectance of the mixture is included between the two endmembers. (b):
the mixtureβs spectrum overshoots the boundaries of the endmembers and therefore
presents unpredictable (for models π1 through π7 ) high values in the NIR region.
and π9 which are equal to π1 and π2 , except the fact that each component
now will present an extra factor that can allow the mixtureβs reflectance to go
out-of-bound. These new factors are assumed to be pigment-related constants
6 F. Grillini et al.
that can account for some degree of dominance in the mixture or some extra-
absorbance/scattering, similar to Kubelka-Munk theory [21]. The formulae of
the models investigated in this research work are reported in Table 2
Table 2: Proposed optical mixing models. The classification is based according to the
nature of the models: additive (A), subtractive (S), hybrid (H). The models π6 and π7
are indeed hybrid, but present strong additive and subtractive tendencies, respectively.
Label Name Formula Category
βπ
π1 Additive π = π=1
ππ πΌπ A
βπ πΌ
π2 Subtractive π = ππ
π=1 π
S
βπ
π3 LIP additive π = 1 β π=1 (1 β ππ )πΌπ A
[ β [ ]πΌ ]
π
π4 LIP subtractive π = 1 β ππ₯π β π=1 βπππ(1 β ππ ) π S
(β )1
π π
π5 Yule-Nielsen π = πΌ ππ
π=1 π π
H
βπ βπ πΌ
π6 Add-Sub π = π π=1 πΌπ ππ + (1 β π) ππ
π=1 π
H/A
(β ) (β )
π π πΌ (1βπ)
π7 Sub-Add π = πΌ ππ
π=1 π π
ππ
π=1 π
H/S
βπ
π8 Linear extra π = π=1
ππ πΌπ ππ A
βπ πΌ
π9 Subtractive extra π = π=1
ππ π ππ S
3 Spectral Unmixing
Spectral Unmixing is a task whose goal is to invert the following problem defini-
tion (Eq.4) for the array of concentrations πΆ, knowing the target spectrum π₯(π)
and the spectral library of endmembers πΈ. At the same time, the constraints of
NC and SC must be respected.
β
π
π₯[πβ
1] = π (πΈ[πβ
π] , πΆ[πβ
1] ) , βπΆ β₯ 0 , πΆ=1 (4)
The notation π represents the number of spectral bands, while π is the number of
endmembers contained in the library. The function π will be in turn one of the
proposed mixing models, so the algorithm should be able to invert a constrained
non-linear function, in most of the instances. In this study, the algorithm that
inverts the objective function is the Nelder-Mead optimization [22]. In order to
solve the optimization problem, the objective function is slightly modified in or-
der to contain a cost function between the target spectrum and the reconstructed
Linear, Subtractive and Logarithmic Optical Mixing Models in Oil Painting 7
one. The new objective function takes thus in input the target reflectance π₯, the
spectral library πΈ, and the reconstructed spectrum π¦Μ, retrieving πΆ by maximiz-
ing the Peak Signal to Noise Ratio (PSNR) between π₯ and π¦Μ, complying with
NC and SC.
[ ]
ππππ
= 20 πππ10 [πππ₯(π₯)] β 10 πππ10 πππΈ(π₯, π¦Μ) (5)
βπ
Μπ )2
π=1 (π₯π β π¦
πππΈ(π₯, π¦Μ) = (6)
π
The assumption that we formulate hereby is that when the reconstructed spec-
trum approaches the target with a high value of PSNR, it means that the correct
endmembers have been selected, in the appropriate relative concentrations as
well. This assumption bears within some risks, and overfitting is one of those. In
this scenario, overfitting can happen when the target spectrum is reconstructed
with a good degree of accuracy, but analyzing the concentration array we discover
that a large number of endmembers are included in significant proportions. Since
we performed the mixtures on canvas, we know that only a limited number of
pigments are composing the ground truth, thus high PSNR values can sometimes
hide a significant classification error. In order to tackle this issue, each unmixing
algorithm produces an estimated label of the target mixture, according to the es-
timated concentrations. If an endmember classifies with a concentration > 35%,
its correspondent capital letter will be included in the estimation, while with a
concentration > 5% a lower case letter is provided. To be selected as reliable,
one model needs to produce high PSNR values and a correct label, which are
anyway two traits highly correlated in most of the cases.
4 Materials and Methods
For the investigation of the proposed models, two sets of mock-ups are created.
Both sets are performed on pre-primed stretched canvases of size 27π₯22 ππ.
This article reports the preliminary results of a larger work that eventually
involves pigments in powder form. However, for the moment, only pigments
contained in commercially available pre-binded tubes are considered. The first
set of mockups includes the following primaries: Vermilion (R), Viridian green
(G), Ultramarine Blue (B), Lemon Yellow (Y), and Titanium White (W). The
exact chemical compositions of the tubes are not investigated, therefore the
proportions of pigments and binding medium are unknown. The canvas was
covered with a layer of white paint and after drying, 24 patches of dimensions
2, 5π₯2, 5 ππ each are painted, including 4 patches for the pure R, G, B, and
Y paints. The remaining patches are some of the possible combinations of the
primaries in ratios approximately 1:1 when the label reports two capital letters,
and approximately 2:1 when one capital and one lower-case letter are reported.
Fig.2 reports the original set photographed in a non-controlled environment and
its graphical representation, obtained computing the color from the reflectance
spectrum under the standard illuminant π·65 . The second set is made up of
8 F. Grillini et al.
(a) (b)
Fig. 2: First set of mock-ups. (a): photograph taken in an uncontrolled environment.
(b): graphical representation of the mock-ups. Each spectrum is plotted in the range
[418, 963 ππ], while the colors on which the spectra are plotted are computed in the
range [420, 780 ππ] assuming a π·65 standard illuminant.
Linear, Subtractive and Logarithmic Optical Mixing Models in Oil Painting 9
2 canvases and 111 patches. This time a preparatory layer of white was not
applied, and some of the tubes used are changed to include pigments with higher
reflectance properties: Cerulean Blue (B), Titanium White (W), Scarlet Lake
(R), Lemon Yellow (Y), Orange yellow (O), and Emerald Green (G). Mixtures
of 3 pigments are included in a 1:1:1 ratio when all 3 letters are reported as
capital, whereas an approximate 2:1:1 ratio is assumed when only one of the
letter of the group is capital (Fig.3).
Fig. 3: Second set of mock-ups. All the possible combinations of 6 pigments, in 5
concentration levels [0, 0.33, 0.5, 0.67, 1], are performed, producing 111 patches. Each
mixture is annotated with its correspondent label.
A push-broom hyperspectral camera HySpex VNIR-1800 produced by Norsko
Elektro Optikk has been used to capture all the HS images included in this
research work. This line scanner employs a diffraction grating and results in
generating 186 images across the electromagnetic spectrum, from 400 ππ to
1000 ππ, at steps of approximately 3.19 ππ. The acquisition distance is set to
30 ππ, which translates into a field of view of 10 ππ and allows to reach an
optical resolution of approximately 0.06ππ. The canvases are illuminated by a
halogen Smart Light 3900e produced by Illumination Technologies, guided on
the scene via optical fibers, projecting lights at 45β¦ with respect to the camera.
At each acquisition, a Spectralon R calibration target with a known reflectance
factor, is included in the scene. The target will serve to estimate the illuminantβs
spectrum and to compute the reflectance at the pixel level. The software enables
the user to select an option that performs radiometric correction, in order to
10 F. Grillini et al.
treat sensor and dark current errors. Flat field correction is performed using the
same Spectralon reference target captured during the acquisition [23].
When the spectral cubes are processed, slices are selected manually at the
locations of the patches and averaged at each wavelength band, in order to
obtain a mean radiance spectrum. The same is done at the spatial location of
the Spectralon target, to retrieve the SPD of the illuminant. The reflectance
of each patch is then obtained inverting Eq.1 and considering the Spectralonβs
reflectance factor.
The two acquired sets of reflectances undergo the same workflow but are
treated independently. Two tasks are performed in order to study the proposed
optical models:
1. Prediction of the concentrations: the information contained in the label of
each mixture is exploited to obtain the relative proportions of the endmem-
bers involved. This task can be seen as a facilitated unmixing, since the
algorithm is forced to use a limited set of endmembers.
2. Unmixing: the a priori knowledge is not used and any endmember can be
picked out of the provided spectral library.
5 Results
The two sets of mockups are analyzed independently, although they are subject
to the same examination process. First, the information contained in the ground
truth label is used to compose a reduced spectral library, containing only the
endmembers involved in the mixture. This task is referred to as prediction of the
concentrations. When all the primaries are included in the spectral library, the
task is then a full unmixing. When unmixing is performed, we are both interested
in the accuracy of the classification and of the reconstruction. The aim is then
to produce an estimation of the label for every analyzed spectrum.
First Set of Mock-ups
The first set included 24 mock-ups and was realized on a single canvas. Fig.4
reports the graphical representation of the canvas for both tasks of prediction and
unmixing. On each patch 3 spectra are plotted: measurement (black), prediction
(dark gray), unmixing reconstruction (light gray).
It is observable that in most instances the latter two are generally faithful
reconstructions of the ground truth, and almost overlap each other. This can be
confirmed by looking at the RMSE and GFC values in Table 3. From this Table,
it is observable that slightly better reconstructions are obtained with unmixing,
indicating that the best models achieve more accurate reconstructions if they
are allowed to select primaries originally not included in the ground truth. Each
patch reports also the colors computed under illuminant π·65 , from left to right:
prediction, ground truth, unmixing. It is noticeable that the errors in the color
computation do not follow any pattern, as sometimes the ground truth is darker
Linear, Subtractive and Logarithmic Optical Mixing Models in Oil Painting 11
Fig. 4: Graphical representation of the first set of mock-ups undergoing the tasks of
prediction and unmixing. Each patch plots in black the ground truth, in dark gray the
predicted spectrum when the endmembers involved are known, and in light gray the
unmixing reconstruction. Colors from left to right refer to prediction, ground truth,
unmixing. The labels indicate the ground truth (left) and estimated after unmixing
(right).
and some other it is brighter. Finally, the labels of each patch report the ground
truth (first) and then the estimated label from the unmixing task. If we exclude
the pure patches (R, G, B, Y), 15 instances detect all the primaries included,
with only 3 of them matching the ground truth label perfectly, the remaining
cases are able to select only 1 of the endmembers correctly (misclassifying the
second one or not selecting it at all). A detailed overview of the results obtained
with the first set is reported in Tab. 4.
From this last Table, we can finally see which are the models that produced
better results. In the prediction task, the linear extra model π8 resulted the
most selected, followed by the purely subtractive model π2 and LIP subtractive
π4 . When π8 is considered, the extra constants seem to be pigment-dependent,
12 F. Grillini et al.
Table 3: Spectral metrics related to the tasks of prediction and unmixing on the first
set of mock-ups. Slightly better results in all categories are obtained with unmixing.
SET 1 Metric Mean Median Min Max π©%ππ π©%ππ
RMSE 0.0551 0.0455 0.0240 0.0971 0.0300 0.0855
PRED
GFC 0.9870 0.9925 0.9670 0.9981 0.9676 0.9974
PSNR 25.86 26.85 20.26 32.39 21.36 30.50
ΞπΈ2000 15.02 14.14 5.57 28.30 6.63 24.74
RMSE 0.0453 0.0444 0.0152 0.0886 0.0203 0.0733
UNMIX
GFC 0.9903 0.9953 0.9688 0.9991 0.9715 0.9985
PSNR 27.90 27.05 21.05 36.34 22.70 33.85
ΞπΈ2000 13.13 12.47 2.35 30.62 5.44 20.87
Table 4: Results of prediction and unmixing on the first set of mock-ups. The con-
centrations πΌ in the prediction section refer only to the endmembers reported in the
annotated ground truth, whereas in the unmixing section they refer to all the pri-
maries included in the spectral library. In several cases, 3 endmembers are selected by
the unmixing algorithm, although only 2 are originally contained in the ground truth.
PRED UNMIXING
Label πΌπ πΌπ PSNR Model πΌπ πΌπ πΌπ πΌπ πΌπ PSNR Model Est label
Rw 0.81 0.19 26.5 π4 0.71 0 0 0.15 0.14 26.8 π4 Ryw
Gw 0.76 0.24 28.6 π4 0 0.83 0 0 0.17 28.8 π9 Gw
Rb 0.57 0.43 28.2 π8 0.61 0 0.20 0.19 0 33.5 π9 Rby
Ry 0.86 0.14 22.8 π8 0.68 0 0 0.32 0 24.7 π9 Ry
By 0.83 0.17 27.2 π8 0 0.08 0.54 0.38 0 32.8 π9 gBY
Yg 0.06 0.94 23.3 π8 0 0.72 0 0.28 0 25.7 π9 Gy
RW 0.54 0.46 21.7 π8 0.59 0 0 0.28 0.13 25.4 π9 Ryw
GW 0.94 0.06 22.0 π8 0 0.65 0 0.02 0.33 22.8 π9 Gw
RB 0.44 0.56 28.4 π8 0.45 0 0.54 0 0 28.5 π8 RB
RY 0.80 0.20 23.1 π8 0.82 0 0 0.18 0 23.1 π8 Ry
BY 0.42 0.58 28.0 π2 0 0.05 0.49 0.46 0 32.3 π9 BY
YG 0.49 0.51 28.5 π8 0 0.80 0 0.20 0 30.4 π9 Gy
Wr 0.50 0.50 23.8 π4 0.42 0 0 0.21 0.37 26.4 π9 RyW
gW 0.92 0.08 21.0 π8 0 0.91 0.03 0 0.06 21.1 π8 Gw
rB 0.30 0.70 29.5 π8 0.36 0.01 0.48 0.15 0 36.3 π9 RBy
rY 0.70 0.30 20.3 π8 0.48 0 0 0.51 0.01 22.6 π9 RY
bY 0.78 0.22 23.2 π8 0.09 0 0.73 0.18 0 23.4 π8 rBy
yG 0.50 0.50 32.4 π8 0 0.86 0 0.14 0 34.2 π9 Gy
YW 0.73 0.27 27.3 π8 0 0.04 0 0.75 0.21 27.3 π8 Yw
BW 0.40 0.60 31.5 π2 0 0.06 0.43 0 0.51 32.0 π4 gBW
especially for G and B, which present values of 3.58Β±0.26 and 1.54Β±0.17, respec-
Linear, Subtractive and Logarithmic Optical Mixing Models in Oil Painting 13
tively. In the unmixing task, the extra constants models π9 and π8 are mainly
selected. The constants of π8 point out for specificity of pigments R and Y this
time, with values 1.46 Β± 0.22 and 1.33 Β± 0.15. Concerning π9 , this model some-
times is failed to be inverted, thus producing very poor results. However, when
it is inverted correctly, the PSNR values reached are the best. The constants do
not point out for material specificity, indeed the mean value of all constants is
1.23 Β± 0.17.
Second Set of Mock-ups
With 111 mock-ups, it is not possible to provide a graphical representation of
the results. However, an indication of how the proposed models are selected and
how they behave is presented in Fig.5 and Fig.6 for the tasks of prediction and
unmixing, respectively. Once again, π8 resulted the most selected model when
the primaries included in the mixture are known a priori, followed closely by π2 .
The PSNR values are plotted as Gaussian functions in the (b) side of the figures.
For the prediction task, π9 is by far the worst model, having its PSNR distri-
bution out of scale. The Gaussian curves of all models except π8 are clustered
in a tight region of PSNR, indicating that their behavior is very similar. Little
differences in PSNR do not imply significant changes in the reconstructions, but
with the workflow adopted, they can lead to different pigment classifications.
60
0.1
M1 M9
50 M2
0.08 M3
M4
40
M5
0.06
M6
N
30
M7
0.04 M8
20 5 10
PSNR
0.02
10
0 0
M1 M2 M3 M4 M5 M6 M7 M8 M9 10 15 20 25 30 35 40
PSNR
(a) (b)
Fig. 5: Analysis of the prediction task on the second set of mock-ups. (a): number of
times each model is selected with the highest PSNR. π8 and π2 produce clearly the
best results, while the intermediate models and π9 are never selected. (b): PSNR of
each model expressed as normal distributions.
In the unmixing task, the 3 most selected models are π2 , π8 and π9 . As
stated before, π9 sometimes is failed to be inverted, thus is explained the flat
normal distribution in Fig.6b. However, it is somewhat surprising that it gets
selected in almost 1β3 of the instances. As we can see from the plot, the PSNR
values that this model reaches are not amongst the highest of the lot, meaning
that it is able to describe mixtures that are complex for all the other models.
The parameters of models π8 and π9 for the unmixing task do not suggest a
14 F. Grillini et al.
40 0.1
M1
M2
0.08
30 M3
M4
0.06 M5
M6
N
20
0.04 M7
M8
10 M9
0.02
0
0 10 15 20 25 30 35 40 45
M1 M2 M3 M4 M5 M6 M7 M8 M9
PSNR
(a) (b)
Fig. 6: Analysis of the unmixing task. (a): number of times each model is selected
with the highest PSNR. π2 , π8 and π9 are the most selected. All models except the
intermediate π5 and π7 are selected at least once. (b): PSNR of each model expressed
as normal distributions.
material-specificity. As a matter of fact, π8 produces an overall 1.00 Β± 0.15 value
for the constants, explaining the fact that the linear model describes fairly well
the mixtures, and just needs little adjustments to reach better reconstructions.
The same goes for model π9 , which presents an overall value for the constants of
1.16 Β± 0.08. In the case of prediction of concentrations, the values for π8 and π9
are 0.97Β±1.00 and 0.2Β±0.65, respectively. The higher variance indicates instability
in predicting the mixtures correctly. By observing Table 5 for the spectral metrics
we can indeed notice that better reconstruction results are achieved in the task
of unmixing. The better values of the spectral metrics, and the fact that π8 and
π9 are unstable in the prediction and not in the unmixing, leads us to state
that generally, more pigments than reported in the ground truth are needed, in
order to obtain satisfactory results in terms of spectral reconstruction. In fact,
we can observe that the estimated labels (not reported for readability) present
more pigments than the ground truth in 24% of the instances.
6 Conclusion
In this paper, we presented an investigation of optical mixing models in the
specific case of oil painting. Nine models are analyzed by performing the task
of predicting the concentrations of the primaries using prior knowledge, as well
as the task of unmixing, on two sets of mock-ups realized for the occasion. The
models that describe best each mixture are retrieved, comparing the PSNR of
each spectral reconstruction, while keeping an eye to the newly produced la-
bel. The subtractive model and the linear model adapted with extra parameters
resulted the best in the task of prediction, whereas both, plus the subtractive
model adapted with extra parameters, produced the best results in the unmix-
ing task. The role of the parameters needs to be further evaluated, but from
Linear, Subtractive and Logarithmic Optical Mixing Models in Oil Painting 15
Table 5: Spectral metrics related to the tasks of prediction and unmixing performed
on the second set of mock-ups. The improvement of the metrics in the unmixing task is
more significant than the one registered in the case of the first set, supporting the state-
ment that the analysed models require generally more endmembers than the number
reported in the annotated labels to obtain more accurate spectral reconstructions.
SET 2 Metric Mean Median Min Max π©%ππ π©%ππ
RMSE 0.0492 0.0490 0.0076 0.1418 0.0171 0.0806
PRED
GFC 0.9932 0.9950 0.9698 0.9999 0.9847 0.9995
PSNR 27.55 26.19 16.96 42.43 21.87 35.36
ΞπΈ2000 8.73 6.51 0.29 32.56 1.37 18.84
RMSE 0.0380 0.0352 0.0068 0.0942 0.0134 0.0687
UNMIX
GFC 0.9956 0.9970 0.9834 0.9999 0.9882 0.9997
PSNR 29.76 29.07 20.52 43.29 23.26 37.47
ΞπΈ2000 7.20 5.70 0.24 28.80 1.10 17.30
our results, it does not suggests a pigment-specificity for both the additive and
subtractive models.
The aim for future works will regard the performing of mixtures with prior
information on their concentrations of pigments in powder form, as well as the
implementation of more complex models that can include multiple layers of a
painting, and the application on hyperspectral images in pixel-based investiga-
tions.
References
1. Conference, U.G.: Draft Medium-term Plan, 1990-1995: General Conference,
Twenty-fifth Session, Paris, 1989. Paris, France: Unesco (1989)
2. Abt, J.: The origins of the public museum. A companion to museum studies (2006)
115β134
3. Strojnik, M., Paez, G., Ortega, A.: Near IR diodes as illumination sources to
remotely detect under-drawings on century-old paintings. In: 22nd Congress of
the International Commission for Optics: Light for the Development of the World.
Volume 8011., International Society for Optics and Photonics (2011) 801177
4. Mantler, M., Schreiner, M.: X-ray fluorescence spectrometry in art and archaeology.
X-Ray Spectrometry: An International Journal 29 (2000) 3β17
5. Grassi, N., Migliori, A., MandoΜ, P., Calvo Del Castillo, H.: Differential pixe
measurements for the stratigraphic analysis of the painting madonna dei fusi by
leonardo da vinci. X-Ray Spectrometry: An International Journal 34 (2005) 306β
309
6. Franquelo, M.L., Duran, A., Herrera, L.K., De Haro, M.J., Perez-Rodriguez, J.:
Comparison between micro-raman and micro-ftir spectroscopy techniques for the
characterization of pigments from southern spain cultural heritage. Journal of
Molecular structure 924 (2009) 404β412
16 F. Grillini et al.
7. Targowski, P., Iwanicka, M.: Optical coherence tomography: its role in the non-
invasive structural examination and conservation of cultural heritage objectsβa
review. Applied Physics A 106 (2012) 265β277
8. GarcΔ±Μa-Bucio, M.A., Casanova-GonzaΜlez, E., Ruvalcaba-Sil, J.L., Arroyo-Lemus,
E., Mitrani-Viggiano, A.: Spectroscopic characterization of sixteenth century panel
painting references using raman, surface-enhanced raman spectroscopy and helium-
raman system for in situ analysis of ibero-american colonial paintings. Philosoph-
ical Transactions of the Royal Society A: Mathematical, Physical and Engineering
Sciences 374 (2016) 20160051
9. Wrapson, L., Rose, J., Miller, R., Bucklow, S.: In Artistsβ Footsteps, the recon-
struction of pigments and paintings. 1 edn. Archetype Publications (2012)
10. Cucci, C., Casini, A., Picollo, M., Stefani, L.: Extending hyperspectral imaging
from vis to nir spectral regions: a novel scanner for the in-depth analysis of poly-
chrome surfaces. In: Optics for Arts, Architecture, and Archaeology IV. Volume
8790., International Society for Optics and Photonics (2013) 879009
11. Bioucas-Dias, J.M., Plaza, A., Dobigeon, N., Parente, M., Du, Q., Gader, P.,
Chanussot, J.: Hyperspectral unmixing overview: Geometrical, statistical, and
sparse regression-based approaches. IEEE journal of selected topics in applied
earth observations and remote sensing 5 (2012) 354β379
12. Keshava, N., Mustard, J.F.: Spectral unmixing. IEEE signal processing magazine
19 (2002) 44β57
13. Zhao, Y.: Image segmentation and pigment mapping of cultural heritage
based on spectral imaging. PhD thesis, Rochester Institute of Technology,
http://scholarworks.rit.edu/theses (2008)
14. Deborah, H., George, S., Hardeberg, J.Y.: Pigment mapping of the scream (1893)
based on hyperspectral imaging. In: International Conference on image and Signal
processing, Springer (2014) 247β256
15. Burns, S.A.: Subtractive color mixture computation. arXiv preprint
arXiv:1710.06364 (2017)
16. Jourlin, M., Pinoli, J.C.: Logarithmic image processing: the mathematical and
physical framework for the representation and processing of transmitted images.
In: Advances in imaging and electron physics. Volume 115. Elsevier (2001) 129β196
17. Panetta, K., Wharton, E., Agaian, S.: Parameterization of logarithmic image pro-
cessing models. IEEE Tran. Systems, Man, and Cybernetics, Part A: Systems and
Humans (2007) 1β12
18. Pinoli, J.C.: ModeΜlisation et traitment des images logarithmiques: TheΜorie et ap-
plications fondamentales. Report No.6 (1992)
19. Yule, J., Nielsen, W.: The penetration of light into paper and its effect on halftone
reproduction. In: Proc. TAGA. Volume 3. (1951) 65β76
20. Simonot, L., HeΜbert, M.: Between additive and subtractive color mixings: inter-
mediate mixing models. JOSA A 31 (2014) 58β66
21. Yang, L., Kruse, B.: Revised kubelkaβmunk theory. i. theory and application.
JOSA A 21 (2004) 1933β1941
22. Nelder, J.A., Mead, R.: A simplex method for function minimization. The com-
puter journal 7 (1965) 308β313
23. Pillay, R., Hardeberg, J.Y., George, S.: Hyperspectral imaging of art: Acquisition
and calibration workflows. Journal of the American Institute for Conservation 58
(2019) 3β15