This paper proposes a novel appearance-manipulation technique that parametrically manipulates the visible anisotropic reflection property with illumination projection from multiple projectors. This method obtains a reflectance matrix corresponding to the bidirectional reflectance distribution function (BRDF) from images captured using multiple cameras. The reflectance matrix was then fitted to the Ashikhmin BRDF model to estimate its parameters of the BRDF model. The reflectance matrix corresponding to the target BRDF was then calculated by manipulating the estimated parameters. The anisotropic reflection was manipulated based on the optical model by projecting images from multiple projectors that changed the texture represented by the reflectance matrix calculated in this manner.
The angular light intensity distribution on the
surface is formed by its properties (e.g.,
bidirectional reflectance distribution function (BRDF),
bidirectional transmittance distribution function
(BTDF)) and represents rich materiality, such as
glossy metallic reflection, clear glass caustic, and
beautiful structural color. Meanwhile, precisely
designed light-field projection, instead of normal
environmental illumination, has the potential to
manipulate light angular distribution and alter our
perception of materiality [
Horizontally aligned projectors and a screen
composed of lenticular lenses with a diffusing
screen can achieve a projection-based
autostereoscopic display [
Amano et al. [
In this paper, we propose a method to parameterize the VDDAM applying the Ashikhmin BRDF model to the previously acquired reflection characteristics. With this parameterization, we enhanced or reduced the anisotropy and then recreated the reflectance. Finally, we calculated the manipulation references for each viewing direction using the recreated reflectance. 2.3.
Murakami and Amano proposed a response
model for multiple projectors and cameras that
considers color [
B = CCC , where " = $%!", %#", … , %$"(" , + = $,!", ,#", … , ,$"(" , (5) !! * = , ⋮"! !" ⋯ !# 6!! 6!" ⋯ 6!# "" ⋯ ⋮"#0 , * = ⎛6"! 6"" ⋯ 6"#⎞ . (6)
⋮ ⋱ ⋮ ⋮ ⋱ ⋮ $! $" ⋯ $# ⎝6$! 6$" ⋯ 6$#⎠ Hereafter, we regard the color spaces as calibrated, and we write CC as C in the following sections. 2.4.
(1) (2) (3) (4)
In this section, we introduce the Ashikhmin BRDF model to fit the reflectance matrix. The Ashikhmin BRDF model is described by the sum of the specular and diffuse components, with the specular component = defined as follows: !(", #) ('!()"+'#()") = ( ($ + 1)(% + 1) () ("-()")
8 () 3("), (#)4 5! + (1 − !) 31 − ()4/8, (7) where > and ? represent normalized vector to the light and viewer. represent normalized halfvector between > and ? . represent surface normal to macroscopic surface. @ and A represent two phong-like exponents that control the specular lobe shape. The larger the value of @, the higher the directivity of reflection in the u direction. Similarly, the larger the value of A, the higher the directivity of reflection in the v direction.
Our proposed method obtained a reflectance matrix representing the optical response between projectors and cameras corresponding to a roughly sampled BRDF on every single point on the object's surface with the experimental devices shown in Figure 1. Subsequently, we fitted the reflectance matrix with the Ashikhmin BRDF model and parameterized the reflectance relationship. Then, the parameters were manipulated to design a desired anisotropic reflection, yielding a recreated reflection matrix. Finally, the VDDAM based on the optical model achieved the desired appearance that the reflectance matrix represented by projecting images from multiple projectors.
In this paper, we employed 7 cameras (Ximea， MQ013CGE2, resolution: 1280 × 1024 ) and 7 projectors (EPSON,EB-W05, resolution:1280 × 800) in order to achieve high quality perceptual BRDF manipulation with complex reflection characteristics. The cameras and projectors were placed in front of the target object and the other projectors are placed radially at 15. intervals around the projector 4 (Figure 1). The cameras are placed close to each projector. This arrangement takes into account the measurement and manipulation of anisotropic reflections.
To obtain the reflectance matrix, first capture the red projection from one projector with all cameras. Similarly, the green and blue projection from one projector is captured by all cameras. This process is repeated with seven projectors. The reflectance is obtained by dividing the image thus acquired by the RGB of the projection image. 3.2.
We used a drawing foil of Nishijin silk textile, which contains patterns of birds, flowers, clouds, and a mountain with rivers, as the manipulation target. Various threads, including gold thread and dyed thread, are used in this Nishijin silk textile, and differences in gloss can be seen, such as the gold thread being more reflective than the dyed thread. The weaving method also causes differences in reflectance characteristics. In the case of twill weave, the ratio of warp to weft threads on the surface is close, resulting in isotropic reflections. On the other hand, a satin weave has a higher ratio of warp threads than weft threads on the surface, resulting in anisotropic reflections. In this study, we regard the target object as a plane. 3.3.
We employed the Levenberg-Marquardt method, a nonlinear optimization scheme, and obtained anisotropic reflection parameters by minimizing the error function as follows: (!"#, $"#, %"#, ""#) , * where ∈ (, , ) , ∈ (, , ) , and &"'# represents the reflectance of the color component of the color projection of projector captured by camera . @=B, A=B represent anisotropic scattering for each direction, and u, v, C=B, ==B represent the intensities of the diffuse and specular components, respectively. Because the four parameters @, A, =, and C must be positive, we applied a nonnegative condition to the Levenberg-Marquardt method. This allows us to obtain the four parameters of the Ashikhmin model for a single pixel from the reflectance matrix at a single pixel.
Figure 2 shows the estimated @ and A. Because there is no significant difference the color channel, this figure shows only the R channel. Brightness expresses the value of each parameter, and the brighter area has a sharp specular reflection along each direction. A small difference between the @ and A values indicates isotropy, whereas a large difference indicates anisotropy. Area (a) in the figure shows a twill weave using gold threads and has almost isotropic reflections. Area (b) is a satin weave that uses gold threads and exhibits strong anisotropy. Area (c) is a satin weave using dyed threads, and it has weak anisotropy. Area (d) is a twill weave using dyed threads and exhibits diffuse reflection. 3.4.
Our anisotropic manipulation aims to enhance or reduce its reflection of the anisotropic reflection optically while maintaining the glossiness of the isotropic reflection. Based on this, we updated the parameters as follows: :; = : + (: − <), <; = < (: ≥ <) . (9) = )&+( )'+((&"'# − ((', )&; !"#, $"#, %"#, ""#))) , (8) !:; = :, <; = < + (< − :) (: < <) (a) Anisotropic reduction (b) Anisotropic enhancement
Figure 3:Target images.
(a) Anisotropic reduction (b) Anisotropic enhancement
Figure 4: Projection images. (a) Anisotropic reduction (c) Anisotropic enhancement (b) Original appearance
Figure 5: Projection results.
If @was greater than A, the difference, which is an anisotropy, was added to @with a multiplication of the scaling factor α. Otherwise, the difference was added to A as well. When > 1, the difference was expanded, and anisotropy was enhanced. On the contrary, the anisotropy was completely removed when = −1.
We applied this manipulation to the estimated parameters and obtained the desired target images for the entire manipulation area. 3.5.
From the reflectance matrix and the target images BB = (BB%>, BB%?, … , BB%@)% , we obtained the projection images CB = (CB%>, CB%?, … , CB%@)% for each projector. However, the projection images B must be positive. Therefore, in this paper, a C non-negative conditional optimization problem min ∥ CCB − BB ∥?,
D
#
ℎ B"> ≥ 0, B> ≥ 0, B$> ≥ 0, … , B$A ≥ 0. (10)
used the Lawson-Hanson algorithm [
The target images were created by updating @ and A, and by manipulating the value of in Eq. (9), the projection images were obtained using Eq. (10), and then projected.
Figures 3 and 4 shows examples of the target and projection images, respectively. In Figure 5, the projection results are arranged to correspond to each camera position. All of these images are shown in identical aspects by geometrical transformation to the common coordinate (cam 4).
Figure 3 shows the target images created using = 8. We solved the non-negative optimization problem described in Section 3.3 and obtained the projection images shown in Figure 4(b). The manipulation results from the projection are shown in Figure 5(c). It should be noted that a significant difference in glossiness between viewing directions along u and v was observed. Figure 6 shows the brightness change from cam1 to cam5 and cam3 to cam7 when the viewpoint is moved from left to right. Figure 6(a) shows the average brightness of the 3 × 3 pixels in the area shown in Figure 2(b). When the viewpoint is moved horizontally in the range of cam1 to cam5 and cam3 to cam7, the gloss change of the anisotropy enhanced image is sharper than that of the original appearance. These results confirmed the enhanced anisotropy.
Figure 6(b) shows the average brightness values of the 3 × 3 pixels in the area shown in Figure 2(a). When the viewpoint is moved horizontally in the range of cam1 to cam5 and cam3 to cam7, it can be confirmed that the anisotropic enhancement follows the brightness change of the original appearance, although the brightness of the anisotropic enhancement is reduced compared to the original appearance. This confirms that the isotropic reflection was maintained in the region of the gold thread twill weave. 4.2.
even with anisotropic reduction in the region of the gold thread twill weave.
In this paper, we propose a method for estimating the parameters of the Ashikhmin model from the reflectance matrix. Furthermore, we propose a parameter manipulation method that can enhance and reduce anisotropy. Projection images were calculated using non-negative conditional optimization. The projection results showed that anisotropy could be enhanced and reduced in areas with anisotropic reflections. In areas with isotropic reflections, although the brightness was slightly reduced, the isotropic reflections were maintained.