<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta />
    <article-meta>
      <title-group>
        <article-title>Image Statistics as Glossiness and Translucency Predictor in Photographs of Real-world Ob jects</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Davit Gigilashvili</string-name>
          <email>davit.gigilashvili@ntnu.no</email>
          <xref ref-type="aff" rid="aff1">1</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Midori Tanaka</string-name>
          <email>midori@chiba-u.jp</email>
          <xref ref-type="aff" rid="aff0">0</xref>
          <xref ref-type="aff" rid="aff1">1</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Marius Pedersen</string-name>
          <email>marius.pedersen@ntnu.no</email>
          <xref ref-type="aff" rid="aff1">1</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Jon Yngve Hardeberg</string-name>
          <email>jon.hardeberg@ntnu.no</email>
          <xref ref-type="aff" rid="aff1">1</xref>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          <institution>Chiba University, Department of Imaging Sciences; Chiba</institution>
          ,
          <country country="JP">Japan</country>
        </aff>
        <aff id="aff1">
          <label>1</label>
          <institution>The Norwegian Colour and Visual Computing Laboratory, Norwegian University of Science and Technology</institution>
          ,
          <addr-line>Gj vik</addr-line>
          ,
          <country country="NO">Norway</country>
        </aff>
      </contrib-group>
      <abstract>
        <p>We interpret our surrounding based on the visual stimuli, and perceive objects and materials around us to have various attributes, like color, glossiness, and translucency. We analyze the three-dimensional world based on the two-dimensional images detected by our retina. The state-of-the-art works conclude that the human visual system has a poor ability to fully understand and invert the complex optical nature of light and matter interaction. Some authors rather propose that the human brain calculates image statistics to perceive appearance, demonstrating correlation between perceptual attributes and various statistical metrics. However, the illustrated examples are usually unrealistic nearly-perfect stimuli, making real-life robustness of the ndings questionable. In this study, we analyzed image statistics of photos of real world objects, and assessed the performance of statistical image metrics proposedly used by the human visual system. We identi ed very interesting trends, as well as limitations.</p>
      </abstract>
      <kwd-group>
        <kwd>Material appearance</kwd>
        <kwd>image statistics</kwd>
        <kwd>gloss</kwd>
        <kwd>translucency</kwd>
      </kwd-group>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>-</title>
      <p>
        Appearance is a complex psychovisual phenomenon that implies attributing
particular characteristics to surrounding objects based on the interpretation of the
visual data. CIE 175:2006 [
        <xref ref-type="bibr" rid="ref23">23</xref>
        ] (as quoted in [
        <xref ref-type="bibr" rid="ref5">5</xref>
        ]) de nes appearance as "the visual
sensation through which an object is perceived to have attributes as size, shape,
colour, texture, gloss, transparency, opacity, etc." The CIE identi es color, gloss,
translucency and texture as four major appearance attributes [
        <xref ref-type="bibr" rid="ref23">23</xref>
        ]. Appearance
measurement has been developed towards hard metrology, i.e. instrumental
measurements [
        <xref ref-type="bibr" rid="ref12 ref18 ref30">12, 18, 30</xref>
        ], and soft metrology relying on psychophysics [
        <xref ref-type="bibr" rid="ref22 ref27 ref8">8, 22, 27</xref>
        ].
      </p>
      <p>
        While we orient ourselves in the 3-dimensional world we still interpret the
environment based on the 2D retinal images. And we do pretty well: humans with
normal vision can easily distinguish glossy and matte or translucent and opaque
objects; furthermore, we are good at identifying materials, easily distinguishing
ceramics from wax or human esh from plastic dummies. Although multisensory
information, like tactile or auditory, facilitate this process, the crucial amount of
information is extracted from the above-mentioned 2D retinal images. Fleming
and Bultho [
        <xref ref-type="bibr" rid="ref6">6</xref>
        ] have proposed that the human visual system (HVS) has poor
optics inversion abilities, and that it relies on simple image cues to interpret
material properties. Motoyoshi [
        <xref ref-type="bibr" rid="ref21">21</xref>
        ] tried to correlate image statistics with
material properties, and found indications that skewness, or a similar measure of
luminance histogram asymmetry, might be used by the HVS to judge surface
properties. The nding is further supported and manifested by Landy [
        <xref ref-type="bibr" rid="ref15">15</xref>
        ].
Marlow and Adelson [
        <xref ref-type="bibr" rid="ref16">16</xref>
        ] demonstrated that sharpness, contrast and coverage area
of the highlights are correlated with perceived level of glossiness. Qi et al. [
        <xref ref-type="bibr" rid="ref24">24</xref>
        ]
tried to nd correlation between perceived glossiness and various statistics of
specular highlights, like spread, size, number, strength, and percentage
coverage, and found a statistically signi cant correlation between the percentage of
the highlight coverage and perceived glossiness.
      </p>
      <p>
        Image statistics have been used for studying perceived translucency as well.
Motoyoshi [
        <xref ref-type="bibr" rid="ref20">20</xref>
        ] manipulated images of various materials and concluded that
\spatial and contrast relationship between specular highlights and non-specular
shading patterns is a robust cue for perceived translucency of three-dimensional
objects". On the other hand, it has been also shown that image statistics alone
do not entirely explain the complex nature of appearance perception and they
are usually subject to multiple photo-geometric constraints [
        <xref ref-type="bibr" rid="ref13 ref14 ref16 ref2">2, 13, 14, 16</xref>
        ].
      </p>
      <p>
        Although the above-mentioned ndings are interesting, they are oftentimes
based either on the synthetic stimuli, rendered in constrained and unrealistic
environments, or few photographs taken in limited conditions. The studies using
large photograph databases have no access to the physical ground truth of the
material (e.g. [
        <xref ref-type="bibr" rid="ref25 ref31">25, 31</xref>
        ]), while wherever the ground truth is available, the number
is stimuli is low (e.g. [
        <xref ref-type="bibr" rid="ref21">21</xref>
        ]).
      </p>
      <p>The novelty of this study is using a photograph dataset with full access
to the ground truth physical stimuli. We had a particular motivation for using
photographs in this study. The vast majority of the authors using computer
generated stimuli do not account for imperfections and artifacts present in the real
world. As computer vision emerges, with autonomous vehicles among the most
prominent applications, in-the-wild performance of particular metrics becomes
vitally important for material identi cation. Therefore, we decided to extract
image statistics not from the synthetic stimuli, but from photographs of real
world objects coming with unintended artifacts, and to study the robustness of
image statistics, as predictors for actual material appearance. We photographed
objects with varying degree of gloss and translucency and described them with
statistical metrics known to be correlated with them.</p>
      <p>The paper is organized as follows: in the next section, we present the
acquisition setup and methodology. The results are presented and discussed in Sections
3 and 4, respectively. Finally, we draw conclusions and summarize the potential
directions for the the future work.
2
2.1</p>
    </sec>
    <sec id="sec-2">
      <title>Methodology</title>
      <sec id="sec-2-1">
        <title>Stimuli</title>
        <p>
          We photographed spherical resin objects from the Plastique collection [
          <xref ref-type="bibr" rid="ref28">28</xref>
          ]. The
objects have been created by an independent artist with an intention to be used
in material appearance research. The resin substrate material is colored with
di erent combinations of blue, yellow and white colorants, followed by di erent
levels of surface processing (polishing). The objects come in three levels of surface
coarseness that a ects apparent gloss of the materials. We photographed 30
spheres in total with 3 di erent levels of surface roughness, 3 hues, and various
levels of translucency (Fig. 1). It is worth mentioning that the objects have
several visible artifacts, like scratches on the surface and bubbles in the volume,
that make them good targets for testing the robustness of image-based metrics.
The close-ups of some of the objects are shown in Fig. 2. Renderings of spherical
objects are very commonly used in computer graphics for studying appearance
(e.g. [
          <xref ref-type="bibr" rid="ref22 ref29">22, 29</xref>
          ]), and a simple curved shape of a sphere ensures apparent specular
re ections, as well as distinctness of image gloss, that are very widely used cues
for glossiness assessment by the HVS [
          <xref ref-type="bibr" rid="ref22 ref8 ref9">8, 9, 22</xref>
          ].
2.2
        </p>
      </sec>
      <sec id="sec-2-2">
        <title>Image Acquisition</title>
        <p>The objects were photographed in a GretagMacbeth Spectralight III viewing
booth under di use D50 illumination with around 4900K color temperature. The
illuminant is placed on the ceiling of the viewing booth, placing all objects under
top-lit geometry - the most commonly encountered illumination geometry, both
outdoors under sunlight, as well as in an o ce environment. The light intensity
on the bottom of the viewing booth was 1858 lux, as it was directly exposed
to the light, while it was 900 lux on the background. The acquisition setup is
shown in Fig. 3.</p>
        <p>The objects were placed on a white matte paper. Metal rings were used to x
the position of the spherical objects. In order to avoid possible bias from highly
specular metal rings, they were covered with a white tape sticker. The
immediate background of the object was white for opaque objects, while translucent
objects were photographed twice, with black and white backgrounds. A Nikon
D3200 camera was used with ISO 100, shutter speed 1/250 sec., F-stop 2.2, and
50mm focal distance. The object was located around 50 centimeters away from
the camera. The camera was characterized using a MacBeth ColorChecker. The
estimates of CIE XYZ values were obtained by a regression-based method
using manufacturer-provided and camera-acquired color coordinates of the color
checker patches. The color correction matrix was found by the least squares
approximation. The spheres were segmented from the images of 3008 2000 pixels.</p>
        <p>We are aware of the limitations related to the acquisition pipeline. Although
the camera response function (CRF) has not been measured or estimated, the
non-linearity of the CRF that is typical to consumer cameras might have a ected
the results. It is especially worth highlighting that the limited dynamic range
of the acquisition system and clipping of the high luminance information could
have impacted the recorded luminance histogram and its statistical moments.
2.3</p>
      </sec>
      <sec id="sec-2-3">
        <title>Analysis of the Data</title>
        <p>
          Only manually segmented images were studied and the background is not
included in the statistics. It has been proposed that chromatic information has
negligible impact on gloss perception [
          <xref ref-type="bibr" rid="ref22 ref27 ref9">9, 22, 27</xref>
          ]. In depth analysis of this is
beyond the scope of this work. We assume that the vital portion of the information
needed for glossiness estimation is embedded in luminance, and therefore,
analyze the luminance channel Y from CIE XYZ. We found luminance histograms
for each of the segmented objects and calculated the rst four moments of it.
Finally, the following statistical measures have been considered for the analysis:
skewness and kurtosis of the luminance histogram, coverage of the highlights,
mean luminance of the object, and standard deviation of the luminance
distribution. The coverage was de ned as the percent of the total surface covered by
the areas which were larger than 20 pixels and had luminance value above 0.9
(luminance is normalized to 0-1 range, 1 corresponding to the largest luminance
recorded by the acquisition system. We do not report cm/m2 measurements). A
correlation between gloss and the size of the highlights has been reported in the
literature [
          <xref ref-type="bibr" rid="ref16 ref17">16, 17</xref>
          ]. Finally, we used these ve statistical metrics for clustering the
objects.
3
        </p>
      </sec>
    </sec>
    <sec id="sec-3">
      <title>Results</title>
      <p>The images of the 30 objects are shown in Figure 1. Objects shown in rows 11
and 12 are the same as the ones in rows 6 and 8, respectively, but photographed
4
7
10
13</p>
      <p>16 19 22
Object number
25
28
31
34
with the black background. Two major histogram asymmetry metrics have been
studied: skewness and kurtosis. How the luminance histogram varies among
different colors and levels of surface roughness is illustrated in Fig. 4. The results
for skewness are shown in Fig. 5-6. As we see from the plots, the luminance
histogram of the objects with smoother surface, i.e. higher gloss (di erence in
perceptual glossiness is apparent among the three levels of surface coarseness,
although not quanti ed psychophysically), has always a positive skew, and the
skewness is higher than that of the rougher, i.e. less glossy objects. Skewness
di erence between the two other surface levels is visible, but not large. A clear
regular pattern for the triplets is visible in Fig. 5. If we refer to rows 1-4 in Fig. 1,
the objects vary from darkest blue in row one, to lightest blue in row four. As
we increase lightness of the object, the skewness of the luminance histogram
decreases. Row 7 stands out on the plot with its low histogram skewness. This
can be explained with the fact that the object is white, close to the illumination
color. As the specular re ections on the surface are also whitish, they cause less
skew in the luminance distribution, than for the low luminance bluish objects,
where the tail of the distribution was high luminance specular highlights.
A</p>
      <p>B</p>
      <p>C</p>
      <p>6 7 8
Row in the figure
9
10
11
12</p>
      <p>While skewness measures asymmetry towards particular direction, either
positive, or negative skew, kurtosis measures general "tailedness" of the distribution
in both directions. Kurtosis for glossiest class of the objects is highest, and
generally follows the same pattern, as it is for the skewness (refer to Fig. 7). However,
the distinction between the two other classes is negligible with this measure.</p>
      <p>The surface coverage by specular highlights was equal to zero for all rougher
objects (columns B and C in Figure 1). The only exception was row 7, where the
whitish color of the object biased our calculations and led to unreasonably many
false positives. On the other hand, the coverage did not di er signi cantly among
the smooth objects (column A), and the specular highlights covered around 0.8%
of the total visible area of the sphere.</p>
      <p>
        Mean luminance for each object is summarized in Fig. 8. Studying mean
luminance can be interesting for two reasons: rst of all, overall shininess of the
object, as observed in [
        <xref ref-type="bibr" rid="ref9">9</xref>
        ], can evoke gloss perception in itself; secondly, it has
been demonstrated [
        <xref ref-type="bibr" rid="ref16 ref22">16, 22</xref>
        ] that contrast between specular and di use areas, has
signi cant impact on perceived gloss. Considering that specular highlights are
white and nearly identical on all objects, we assume that mean luminance of
the object is inversely correlated with the contrast gloss - i.e. higher the mean
A B C
      </p>
      <p>A B C
60
s
is50
o
t
ru40
K
30
20
10
0
luminance of the entire object, lesser is the contrast between specular and di use
areas. The objects with smoothest surface have less mean luminance than objects
made of the identical material but with rougher surfaces. This can be explained
with the fact that the substrate white material is exposed to the surface due to
scratches, artifacts and irregularities presented on the rough surface.
1
2
3
4
5</p>
      <p>6 7
Row in the figure
8
9
10
11
12</p>
      <p>
        In contrast with the ndings by Wiebel et al. [
        <xref ref-type="bibr" rid="ref31">31</xref>
        ], standard deviation of
the luminance distribution is a poor predictor for surface coarseness class in
our study (refer to Fig. 9). However, it signi cantly rises when the background
of the object is changed from white to black. This might be a good indication
that impact of the background on the luminance variance is a result of volume
scattering - thus, used as a predictor of translucency for see-through objects.
      </p>
      <p>In addition to this, we hypothesize that complexity of the scene might impact
the statistics. We photographed objects in one additional condition placing a
checkerboard-covered cube close to the object (the setup is shown in Fig. 10).
The general trend is that rougher the surface is, the smaller the impact of the
cube on statistical measures. This can be explained with the fact that the smooth
surface has a good distinctness-of-image re ection, and the image re ected from
the surface signi cantly impacts the statistics, while rough surfaces di use the
light and no pattern is visible on the surface re ections. This trend deserves
further attention.</p>
      <p>
        Afterwards, we compared the metrics for the identical objects between white
and black background photographing conditions (the results for rows 6 and 8 are
compared with the results for rows 11 and 12, respectively). Interestingly,
skewness and kurtosis decrease when the background is changed to black. To some
extent, this can be accounted for many white-colored artifacts of the object which
are visible on the black background only. As expected, mean luminance is
decreased for black background due to absorption of the energy by the background,
and thus, less back-re ections. Finding the ratio of the luminance measured on
white and black backgrounds is an established technique for transmittance
measurement of the at objects (e.g. [
        <xref ref-type="bibr" rid="ref10">10</xref>
        ]). This observation holds at some extent
for spherical objects as well. Also, as already discussed above, standard
deviation changes signi cantly due to change in the complexity of the background.
It has been demonstrated [
        <xref ref-type="bibr" rid="ref9">9</xref>
        ] that translucency, when objects are placed on a
white background, can make objects look glossier. Here we observe that white
background leads to more skewed luminance histograms that itself is proposedly
related to gloss. Therefore, there might be a gloss-translucency cross-attribute
interaction that is described by changes in image statistics. However, this needs
further experimental evidence.
      </p>
      <p>
        Finally, we conducted clustering to validate our hypothesis that the ve
statistical measures are good predictors for object class (smooth, moderately rough,
and highly rough surfaces). We used k-means clustering with 3 clusters. Falsely
detected highlight coverages for objects in the seventh row were manually set
to 0. The cluster was de ned as the centroid being the mean of all points in that
particular cluster. Maximum number of iterations was set to 1000. Cluster
centroids were initialized using k-means++ algorithm [
        <xref ref-type="bibr" rid="ref3">3</xref>
        ]. All objects with rougher
surfaces (columns B and C) ended up in the same cluster. A small separate
cluster was objects 1A and 2A, i.e. dark blue objects with low mean luminance, with
the highest positive skew in luminance histogram. Four smooth-surfaced objects
7A, 10A, 11A, and 12A were clustered together with rough objects. While all
other smooth objects were grouped together in a separate cluster. Clustering
gives us an indication that ve variables, the ve statistical descriptors we use,
might be enough to separate very smooth and glossy objects from rougher and
less glossy objects. However, they fail describing intra-group di erences.
      </p>
      <p>
        0.2
0.18
0.16
We have observed that as the surface becomes rougher, skewness and kurtosis
of the luminance histogram decrease, and the distribution becomes less tailed.
While glossy objects look solid opaque, like billiard balls, rougher surfaces look
milkier, and at some extent evoke illusion of subsurface scattering via surface
scattering only that is not surprising considering that the HVS has poor optics
inversion ability [
        <xref ref-type="bibr" rid="ref6">6</xref>
        ]. This can be an indication in support for Motoyoshi's
proposal [
        <xref ref-type="bibr" rid="ref20">20</xref>
        ] that blurring non-specular regions, i.e. squeezing the tails towards the
center, re ected in decreased skewness and kurtosis, can enhance translucency
perception. On the other hand, it has been observed earlier [
        <xref ref-type="bibr" rid="ref7">7</xref>
        ] that
translucency perception declines with blurring the entire object or image, i.e. when we
decrease variance and histogram asymmetry. However, this proposal certainly
needs validation psychophysically.
      </p>
      <p>
        Although specular highlights are small and very simple in texture (just
saturated blobs), covering less than 1% of the total visible area of the sphere, they
strongly skew the luminance histogram, and evoke strong perception of gloss.
Interestingly, van Assen et al. [
        <xref ref-type="bibr" rid="ref4">4</xref>
        ] have studied photographs with various
patterns of highlights (disk, square, window etc.), and found that simpler specular
highlights evoke stronger gloss perception than more complex ones. However,
the role of the highlights should not be exaggerated, as the perception of gloss is
a complex cognitive process and neither specular re ections are the only source
of the highlights, nor all highlights evoke perception of glossiness. To
demonstrate this, we have superimposed specular highlights of a smooth surface to
a rougher surface of the identical materials (Fig. 11). In one case, the target
rough object has relatively homogeneous texture, while in the other case, there
are very apparent scratches and visual artifacts that help observers deduce the
surface composition of the object. While glossiness for the former object looks
reasonably realistic, the latter object does not look glossy as the highlights start
looking more like artifacts. Presence of roughness cues limit perception of
glossiness, although the statistical metrics are similar to that of glossy objects. This
once again demonstrates photo-geometric constraints limiting the usage of image
statistics as an appearance predictor. Interestingly, the HVS can still be tricked
in particular scenarios when additional cues are missing (the manifestation of
this phenomenon is the viral glossy legs illusion [
        <xref ref-type="bibr" rid="ref19">19</xref>
        ]).
      </p>
      <p>
        Hunter [
        <xref ref-type="bibr" rid="ref11">11</xref>
        ] names contrast gloss, i.e. contrast between specular and di use
areas, among one of the types, or dimensions of gloss. Pellacini et al. [
        <xref ref-type="bibr" rid="ref22">22</xref>
        ] have
demonstrated that darker objects look glossier than lighter ones, and this e ect
has also been observed in other studies [
        <xref ref-type="bibr" rid="ref16 ref29 ref9">9, 16, 29</xref>
        ]. Although we did not have a
direct measure for contrast in this work, considering that highlights were nearly
identical among objects, we assumed that mean luminance of the entire object is
inversely correlated with the contrast gloss. It has been demonstrated that up-to
some threshold rough and light surfaces might look glossy [
        <xref ref-type="bibr" rid="ref24 ref8">8, 24</xref>
        ]. Moreover, it
has been proposed that luminance information associated with shininess might
signi cantly increase perceived glossiness [
        <xref ref-type="bibr" rid="ref9">9</xref>
        ]. Although mean luminance alone
cannot be a good predictor for apparent gloss of the materials, it might carry
rich information regarding contrast and distinctness-of-image (another
dimension of gloss according to Hunter [
        <xref ref-type="bibr" rid="ref11 ref12">11, 12</xref>
        ]), and could be eventually included in
the perceptual gloss model.
      </p>
      <p>
        Standard deviation turned out a poor predictor of surface roughness class
in our study. This interestingly contradicts with Wiebel et al. [
        <xref ref-type="bibr" rid="ref31">31</xref>
        ], who studied
natural images, observed a strong positive correlation between standard
deviation of luminace histogram and gloss, and found it a better predictor for gloss
than skewness. Although we have not conducted perceptual measurements of
our stimuli, we can draw some qualitative parallels. The inconsistency can be
explained with the type of objects depicted in authors' natural images. If we
examine the images illustrated in [
        <xref ref-type="bibr" rid="ref31">31</xref>
        ], we notice that images considered glossy
consist of large segments of contrasting luminances, i.e. photographed complex
shaped objects yield high number of pixels with highlights and also high number
of dark pixels with shading - leading to large standard deviation. Unlike theirs,
the highlights covered less than 1% of our stimuli, while the luminance gradient
on the rest of the sphere was relatively homogeneous. This led to strong skew but
was not enough for yielding high standard deviation in the luminance histogram.
      </p>
      <p>Distinctness-of-the-re ected-image, the mirror image of the surrounding we
can see on very glossy surfaces is another cue for glossiness. The background
and surrounding vary dramatically in dynamic scenes, and hardly ever are as
simple in real life, as studied in the laboratory conditions. Image statistics are
neither static, nor consistent among di erent conditions. We observed that even
a minor change in the environment (Fig. 10) can a ect image statistics that
makes its possible use by the HVS and even by machines, questionable. On the
other hand, appearance is also dynamic; even though the HVS has ability to
perceive some appearance attributes consistently across di erent conditions, i.e.
demonstrates some constancy (e.g. color constancy), the constancy is valid
upto certain extent only, and completely fails in many conditions. While the vast
majority of the studies trying to explain appearance with image statistics rely
on a few images in very limited conditions, it remains an open question how
appearance and image statistics co-vary. We have shown above that particular
image statistics are promising and deserve further attention, but for more solid
conclusions, psychometric measurements are needed. Understanding how image
statistics correlate with perceived appearance can be bene cial in two ways:
{ It can unveil further mechanisms that are used by the HVS to interpret the
surrounding.
{ It can have commercially signi cant applications in computer vision. Many
image statistical metrics are extremely e cient computationally, and might
be used for material identi cation and quality assurance. Moreover,
generality across di erent conditions might not be the mandatory requirement for
image statistics. Many computer vision applications are limited to very
speFig. 12. The object shown in the left and right photos is the same. However, even a
slight change in illumination angle leads to dramatic changes in its appearance. If the
smoothest (region A) and the roughest (region B) surfaces are distinguishable in the
left image, they (regions C and D) look nearly identical in the right one due to dynamic
range limitations.</p>
      <p>ci c conditions by nature, and limits of particular statistics might not have
vital importance, as long as a correlation between statistics and appearance
is established for given (application-speci c) conditions.</p>
      <p>
        Finally, we should mention that above-discussed variation in luminance
distributions was observed due to the curvature of the spherical objects. Our
ndings might not be applicable to other surfaces, especially to the planar ones.
To demonstrate this, we tried photographing at plastic and metallic samples
from the JIDA Standard Sample dataset [
        <xref ref-type="bibr" rid="ref1">1</xref>
        ]. We have observed two interesting
phenomena that made studying image statistics of these samples unreliable:
{ Because the surface is at, all points on small objects are under
approximately the same illumination geometry that makes it impossible to see
specular and di use areas separately, and the entire part of the patch looks rather
homogeneous, essentially cutting down the luminance histogram to a single
luminance value. This can be seen in the left image of Fig. 12, where the
leftmost part of the patch (region A) is smooth and glossy, albeit homogeneous
under given conditions.
{ The samples, especially the metallic ones, are extremely prone to
appearance changes even with a slight change in illumination geometry. This is
demonstrated in Fig. 12.
      </p>
      <p>Although haze and absence-of-textures on low gazing angles (further
dimensions or types of gloss) could be observed on the at patches, these phenomena
are beyond the scope of this work and should be addressed in the future.
5</p>
    </sec>
    <sec id="sec-4">
      <title>Conclusion and Future Work</title>
      <p>
        We have taken photographs of real world objects and studied correlation between
image statistics and actual physical surface properties. Although very clear
positive skew of the luminance histogram is characteristic for smooth (and
presumably glossy) surfaces, the robustness of the metric is challenged by complexity
of the surrounding and semantic understanding of the scene and surface
geometry. Furthermore, mean luminance can be correlated with contrast gloss, while
change in variance across di erent conditions can be a predictor for translucency.
It is worth mentioning that the dynamic range of our acquisition system was
limited, and analysis of the high dynamic range data could reveal further interesting
trends. Complex shapes and wider range of the materials should also be covered.
While di erence in perceptual gloss was assumed between smooth and rough
surfaces, the statistics should be correlated with actual psychophysical measures in
the future. Finally, more statistical measures, like entropy, and chromatic
information should also be included in future studies and the performance of simple
image statistics should be compared with that of the complex machine learning
(e.g. deep learning ) models. It has been demonstrated recently [
        <xref ref-type="bibr" rid="ref26">26</xref>
        ] that
unsupervised learning techniques outperform image statistics and even supervised
learning techniques in prediction of human perception. This is an interesting
avenue that not only provides basis for reliable computer vision systems, but can
also reveal curious mechanisms of the human vision.
      </p>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          1.
          <string-name>
            <given-names>JIDA</given-names>
            <surname>Standard</surname>
          </string-name>
          <article-title>Samples</article-title>
          . (Accessed on 21/11/2019 at), https://www.jidaonline.shop/product/5
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          2.
          <string-name>
            <surname>Anderson</surname>
            ,
            <given-names>B.L.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Kim</surname>
          </string-name>
          , J.:
          <article-title>Image statistics do not explain the perception of gloss and lightness</article-title>
          .
          <source>Journal of Vision</source>
          <volume>9</volume>
          (
          <issue>11</issue>
          :10),
          <volume>1</volume>
          {
          <fpage>17</fpage>
          (
          <year>2009</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          3.
          <string-name>
            <surname>Arthur</surname>
            ,
            <given-names>D.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Vassilvitskii</surname>
            ,
            <given-names>S.:</given-names>
          </string-name>
          <article-title>k-means++: The advantages of careful seeding</article-title>
          .
          <source>In: Proceedings of the eighteenth annual ACM-SIAM symposium on Discrete algorithms</source>
          . pp.
          <volume>1027</volume>
          {
          <fpage>1035</fpage>
          .
          <string-name>
            <surname>Society</surname>
          </string-name>
          for Industrial and Applied Mathematics (
          <year>2007</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          4. van Assen,
          <string-name>
            <given-names>J.J.R.</given-names>
            ,
            <surname>Wijntjes</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.W.</given-names>
            ,
            <surname>Pont</surname>
          </string-name>
          ,
          <string-name>
            <surname>S.C.</surname>
          </string-name>
          :
          <article-title>Highlight shapes and perception of gloss for real and photographed objects</article-title>
          .
          <source>Journal of Vision</source>
          <volume>16</volume>
          (
          <issue>6</issue>
          :6),
          <volume>1</volume>
          {
          <fpage>14</fpage>
          (
          <year>2016</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          5.
          <string-name>
            <surname>Eugene</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          :
          <article-title>Measurement of "total visual appearance": a CIE challenge of soft metrology</article-title>
          .
          <source>In: 12th IMEKO TC1 &amp; TC7 Joint Symposium on Man, Science &amp; Measurement</source>
          . pp.
          <volume>61</volume>
          {
          <issue>65</issue>
          (
          <year>2008</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          6.
          <string-name>
            <surname>Fleming</surname>
            ,
            <given-names>R.W.</given-names>
          </string-name>
          , Bultho , H.H.:
          <article-title>Low-level image cues in the perception of translucent materials</article-title>
          .
          <source>ACM Transactions on Applied Perception (TAP) 2</source>
          (
          <issue>3</issue>
          ),
          <volume>346</volume>
          {
          <fpage>382</fpage>
          (
          <year>2005</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          7.
          <string-name>
            <surname>Gigilashvili</surname>
            ,
            <given-names>D.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Pedersen</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Hardeberg</surname>
          </string-name>
          , J.Y.:
          <article-title>Blurring impairs translucency perception</article-title>
          .
          <source>In: Color and Imaging Conference</source>
          . vol.
          <year>2018</year>
          , pp.
          <volume>377</volume>
          {
          <fpage>382</fpage>
          .
          <article-title>Society for Imaging Science and Technology (</article-title>
          <year>2018</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          8.
          <string-name>
            <surname>Gigilashvili</surname>
            ,
            <given-names>D.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Thomas</surname>
            ,
            <given-names>J.B.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Hardeberg</surname>
            ,
            <given-names>J.Y.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Pedersen</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          :
          <article-title>Behavioral investigation of visual appearance assessment</article-title>
          .
          <source>In: Color and Imaging Conference</source>
          . vol.
          <source>No. 1</source>
          , pp.
          <volume>294</volume>
          {
          <fpage>299</fpage>
          .
          <article-title>Society for Imaging Science and Technology (</article-title>
          <year>2018</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          9.
          <string-name>
            <surname>Gigilashvili</surname>
            ,
            <given-names>D.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Thomas</surname>
            ,
            <given-names>J.B.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Pedersen</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Hardeberg</surname>
          </string-name>
          , J.Y.:
          <article-title>Perceived glossiness: Beyond surface properties</article-title>
          .
          <source>In: Color and Imaging Conference</source>
          . pp.
          <volume>37</volume>
          {
          <fpage>42</fpage>
          . No.
          <article-title>1, Society for Imaging Science and Technology (</article-title>
          <year>2019</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          10.
          <string-name>
            <surname>Hajipour</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Shams</surname>
            <given-names>Nateri</given-names>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            ,
            <surname>Sadr</surname>
          </string-name>
          <string-name>
            <surname>Momtaz</surname>
          </string-name>
          ,
          <string-name>
            <surname>A.</surname>
          </string-name>
          :
          <article-title>Estimation of fabric opacity by scanner</article-title>
          .
          <source>Sensor Review</source>
          <volume>34</volume>
          (
          <issue>4</issue>
          ),
          <volume>404</volume>
          {
          <fpage>409</fpage>
          (
          <year>2014</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref11">
        <mixed-citation>
          11.
          <string-name>
            <surname>Hunter</surname>
            ,
            <given-names>R.S.:</given-names>
          </string-name>
          <article-title>Methods of determining gloss</article-title>
          .
          <source>NBS Research paper RP 958</source>
          (
          <year>1937</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref12">
        <mixed-citation>
          12.
          <string-name>
            <surname>Hunter</surname>
            ,
            <given-names>R.S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Harold</surname>
            ,
            <given-names>R.W.:</given-names>
          </string-name>
          <article-title>The measurement of appearance</article-title>
          . John Wiley &amp; Sons (
          <year>1987</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref13">
        <mixed-citation>
          13.
          <string-name>
            <surname>Kim</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Anderson</surname>
            ,
            <given-names>B.L.</given-names>
          </string-name>
          :
          <article-title>Image statistics and the perception of surface gloss and lightness</article-title>
          .
          <source>Journal of Vision</source>
          <volume>10</volume>
          (
          <issue>9</issue>
          :3),
          <volume>1</volume>
          {
          <fpage>17</fpage>
          (
          <year>2010</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref14">
        <mixed-citation>
          14.
          <string-name>
            <surname>Kim</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Marlow</surname>
            ,
            <given-names>P.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Anderson</surname>
            ,
            <given-names>B.L.:</given-names>
          </string-name>
          <article-title>The perception of gloss depends on highlight congruence with surface shading</article-title>
          .
          <source>Journal of Vision</source>
          <volume>11</volume>
          (
          <issue>9</issue>
          :4),
          <volume>1</volume>
          {
          <fpage>19</fpage>
          (
          <year>2011</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref15">
        <mixed-citation>
          15.
          <string-name>
            <surname>Landy</surname>
            ,
            <given-names>M.S.:</given-names>
          </string-name>
          <article-title>Visual perception: A gloss on surface properties</article-title>
          .
          <source>Nature</source>
          <volume>447</volume>
          (
          <issue>7141</issue>
          ),
          <volume>158</volume>
          {
          <fpage>159</fpage>
          (
          <year>2007</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref16">
        <mixed-citation>
          16.
          <string-name>
            <surname>Marlow</surname>
            ,
            <given-names>P.J.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Anderson</surname>
            ,
            <given-names>B.L.</given-names>
          </string-name>
          :
          <article-title>Generative constraints on image cues for perceived gloss</article-title>
          .
          <source>Journal of vision 13(14:2)</source>
          ,
          <volume>1</volume>
          {
          <fpage>23</fpage>
          (
          <year>2013</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref17">
        <mixed-citation>
          17.
          <string-name>
            <surname>Marlow</surname>
            ,
            <given-names>P.J.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Kim</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Anderson</surname>
            ,
            <given-names>B.L.:</given-names>
          </string-name>
          <article-title>The perception and misperception of specular surface re ectance</article-title>
          .
          <source>Current Biology</source>
          <volume>22</volume>
          (
          <issue>20</issue>
          ),
          <year>1909</year>
          {
          <year>1913</year>
          (
          <year>2012</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref18">
        <mixed-citation>
          18.
          <string-name>
            <surname>Marschner</surname>
            ,
            <given-names>S.R.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Westin</surname>
            ,
            <given-names>S.H.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Lafortune</surname>
            ,
            <given-names>E.P.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Torrance</surname>
            ,
            <given-names>K.E.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Greenberg</surname>
            ,
            <given-names>D.P.</given-names>
          </string-name>
          :
          <article-title>Image-based brdf measurement including human skin</article-title>
          .
          <source>In: Rendering Techniques' 99</source>
          , pp.
          <volume>131</volume>
          {
          <fpage>144</fpage>
          . Springer (
          <year>1999</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref19">
        <mixed-citation>
          19.
          <string-name>
            <surname>Molloy</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          :
          <article-title>Confused by this shiny leg optical illusion? Here's how it works</article-title>
          .
          <source>The Telegraph (Accessed on 04/01</source>
          /2019 at https://wwwtelegraphcouk/news/2016/10/31/confused-by
          <article-title>-this-shiny-leg-opticalillusion-heres-how-it-works)</article-title>
        </mixed-citation>
      </ref>
      <ref id="ref20">
        <mixed-citation>
          20.
          <string-name>
            <surname>Motoyoshi</surname>
            ,
            <given-names>I.</given-names>
          </string-name>
          :
          <article-title>Highlight{shading relationship as a cue for the perception of translucent and transparent materials</article-title>
          .
          <source>Journal of vision 10(9:6)</source>
          ,
          <volume>1</volume>
          {
          <fpage>11</fpage>
          (
          <year>2010</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref21">
        <mixed-citation>
          21.
          <string-name>
            <surname>Motoyoshi</surname>
            ,
            <given-names>I.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Nishida</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Sharan</surname>
            ,
            <given-names>L.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Adelson</surname>
            ,
            <given-names>E.H.</given-names>
          </string-name>
          :
          <article-title>Image statistics and the perception of surface qualities</article-title>
          .
          <source>Nature</source>
          <volume>447</volume>
          (
          <issue>7141</issue>
          ),
          <volume>206</volume>
          (
          <year>2007</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref22">
        <mixed-citation>
          22.
          <string-name>
            <surname>Pellacini</surname>
            ,
            <given-names>F.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Ferwerda</surname>
            ,
            <given-names>J.A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Greenberg</surname>
            ,
            <given-names>D.P.</given-names>
          </string-name>
          :
          <article-title>Toward a psychophysically-based light re ection model for image synthesis</article-title>
          .
          <source>In: Proceedings of the 27th annual conference on Computer graphics and interactive techniques</source>
          . pp.
          <volume>55</volume>
          {
          <fpage>64</fpage>
          . ACM Press/Addison-Wesley Publishing Co.
          <article-title>(</article-title>
          <year>2000</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref23">
        <mixed-citation>
          23.
          <string-name>
            <surname>Pointer</surname>
            ,
            <given-names>M.:</given-names>
          </string-name>
          <article-title>A framework for the measurement of visual appearance</article-title>
          .
          <source>CIE</source>
          Publication pp.
          <volume>175</volume>
          {
          <year>2006</year>
          (
          <year>2006</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref24">
        <mixed-citation>
          24.
          <string-name>
            <surname>Qi</surname>
            ,
            <given-names>L.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Chantler</surname>
            ,
            <given-names>M.J.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Siebert</surname>
            ,
            <given-names>J.P.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Dong</surname>
          </string-name>
          , J.:
          <article-title>Why do rough surfaces appear glossy? JOSA A 31(5</article-title>
          ),
          <volume>935</volume>
          {
          <fpage>943</fpage>
          (
          <year>2014</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref25">
        <mixed-citation>
          25.
          <string-name>
            <surname>Sharan</surname>
            ,
            <given-names>L.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Li</surname>
            ,
            <given-names>Y.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Motoyoshi</surname>
            ,
            <given-names>I.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Nishida</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Adelson</surname>
            ,
            <given-names>E.H.</given-names>
          </string-name>
          :
          <article-title>Image statistics for surface re ectance perception</article-title>
          .
          <source>JOSA A</source>
          <volume>25</volume>
          (
          <issue>4</issue>
          ),
          <volume>846</volume>
          {
          <fpage>865</fpage>
          (
          <year>2008</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref26">
        <mixed-citation>
          26.
          <string-name>
            <surname>Storrs</surname>
            ,
            <given-names>K.R.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Fleming</surname>
          </string-name>
          , R.W.:
          <article-title>Unsupervised learning predicts human perception and misperception of specular surface re ectance</article-title>
          . bioRxiv p.
          <volume>25</volume>
          <fpage>pages</fpage>
          (
          <year>2020</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref27">
        <mixed-citation>
          27.
          <string-name>
            <surname>Tanaka</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Horiuchi</surname>
            ,
            <given-names>T.</given-names>
          </string-name>
          :
          <article-title>Investigating perceptual qualities of static surface appearance using real materials and displayed images</article-title>
          .
          <source>Vision research 115</source>
          ,
          <volume>246</volume>
          {
          <fpage>258</fpage>
          (
          <year>2015</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref28">
        <mixed-citation>
          28.
          <string-name>
            <surname>Thomas</surname>
            ,
            <given-names>J.B.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Deniel</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Hardeberg</surname>
            ,
            <given-names>J.Y.</given-names>
          </string-name>
          :
          <article-title>The plastique collection: A set of resin objects for material appearance research</article-title>
          .
          <source>XIV Conferenza del Colore</source>
          , Florence, Italy p.
          <fpage>12</fpage>
          <lpage>pages</lpage>
          (
          <year>2018</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref29">
        <mixed-citation>
          29.
          <string-name>
            <surname>Thomas</surname>
            ,
            <given-names>J.B.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Hardeberg</surname>
            ,
            <given-names>J.Y.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Simone</surname>
            ,
            <given-names>G.</given-names>
          </string-name>
          :
          <article-title>Image contrast measure as a gloss material descriptor</article-title>
          .
          <source>In: International Workshop on Computational Color Imaging</source>
          . pp.
          <volume>233</volume>
          {
          <fpage>245</fpage>
          . Springer (
          <year>2017</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref30">
        <mixed-citation>
          30.
          <string-name>
            <surname>Wetlaufer</surname>
            ,
            <given-names>L.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Scott</surname>
            ,
            <given-names>W.:</given-names>
          </string-name>
          <article-title>The measurement of gloss</article-title>
          .
          <source>Industrial &amp; Engineering Chemistry Analytical Edition</source>
          <volume>12</volume>
          (
          <issue>11</issue>
          ),
          <volume>647</volume>
          {
          <fpage>652</fpage>
          (
          <year>1940</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref31">
        <mixed-citation>
          31.
          <string-name>
            <surname>Wiebel</surname>
            ,
            <given-names>C.B.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Toscani</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Gegenfurtner</surname>
            ,
            <given-names>K.R.</given-names>
          </string-name>
          :
          <article-title>Statistical correlates of perceived gloss in natural images</article-title>
          .
          <source>Vision Research</source>
          <volume>115</volume>
          ,
          <volume>175</volume>
          {
          <fpage>187</fpage>
          (
          <year>2015</year>
          )
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>