In recent years, spatial augmented reality (SAR) technology, which adds information through light projection, has been used in projection mapping. Kimura et al. proposed a method on SAR research to reproduce color changes due to viewpoint shifts by reflecting Bidirectional Reflectance Distribution Function (BRDF) data through light field projection, This method achieved adaptive BRDF representation under the assumption that surface normal vectors are oriented upwards, thereby ignoring the actual shape of the projection surface. This simplification introduced inconsistencies, such as reversed gradation, compression artifacts, and other distortions around areas of specular reflection. Despite these issues, identifying such inconsistencies is challenging, and they may have minimal impact on the perception of material properties. In this study, we clarified through a subjective evaluation experiment that even with such inaccuracies in reproducing reflected light, viewers do not perceive these errors in using Kimura et al.'s method.
In recent years, spatial augmented reality (SAR) technology, which adds information to buildings and objects through light projection, has been actively utilized in projection mapping. In SAR research, material appearance manipulation has been proposed to alter the perceived texture of objects in the real world, with applications in product design, art exhibitions in museums, and product displays.
Amano et al.[
To solve this problem, Kimura et al.[
However, when the target object is not a plane, the method fails to correctly present the direction of specular reflection or the order of colors around specular highlights. Nevertheless, viewers do not easily perceive these inaccuracies. This raises the question: do we accurately understand light reflection and the gradation of structural colors when judging material appearance? In this study, we aim to clarify how the incorrect color presentation caused by BRDF display on non-planar objects using Kimura et al.’s method afects the perception of BRDF-based material appearance.
source vector illuminates a point on the surface of an object whose reflective properties are represented by BRDF , the colors observed from viewpoints and ′ can be expressed as ( , ) and ( , ′), respectively, using the incident and viewing angles.
In Kimura et al.’s method, as illustrated on the right of Fig. 1, the reflection of the object is assumed to be specular reflection, and by projecting ( , ) and ( , ′) from the specular reflection direction of the viewpoint, the BRDF is reproduced. However, to accurately reproduce BRDF, information about the shape and position of the target object is necessary, making it dificult to achieve dynamic BRDF presentation. Therefore, in vertically upward.
In BRDF reproduction, by performing this projection in
the assumed observation direction, the target BRDF can
be presented. However, it is dificult to predetermine the
viewing directions. Therefore, it is necessary to project
light rays with high angular resolution to accommodate
various viewpoints. To address this, Kimura et al. used
the light field projection system developed by Amano
and Kubo [
The stage, where the projection object is placed, measures 300mm square, and an average of 13 light rays are projected per 1mm square on the stage, thanks to the combination of projectors and mirrors. Since the light rays projected onto the stage can be geometrically determined based on which projector emits the light and which mirror reflects it to reach the stage, it is possible to present the target color from the specular reflection direction based on the BRDF.
Fig. 3 shows the results of BRDF reproduction using Kimura et al.’s method. Changes in the specular highlight positions and the gradation around the specular highlights can be observed with viewpoint changes, successfully reproducing the illumination distribution along to the BRDF model. Moreover, it can also be reproduced for various projection targets.
vertically upward normal vector, which leads to incorrect presentation of the color order around specular highlights when presenting BRDF textures on non-planar surfaces. For example, when projecting onto a convex object in Fig. 4, the reflection direction changes significantly from 1 to 2. Additionally, when projecting onto a concave object, the direction of reflection also changes. This results in compression or inversion of gradation around specular reflection on the image. Although this method cannot achieve precise reproduction of BRDF, the results from Kimura et al. suggest that viewers may not readily perceive these gradation inconsistencies. Therefore, we aim to investigate whether presenting BRDF textures on non-planar surfaces is perceived as having diferent material appearances.
perceive precise color changes when perceiving material appearance using Kimura et al.’s method. Since material appearance is subjective, it is efective to compare results through visual experiments with participants.
For such evaluations, methods like Thurstone’s paired
comparison method[
Specifically, we projected the image onto the target the following options: "Projection 1 is correct," "Projecshown Fig. 5 and evaluated whether participants per- tion 1 is somewhat correct," "Cannot say," "Projection 2 is ceived accurate color changes as follows: somewhat correct," or "Projection 2 is correct." 1. For a planar object(Fig. 5(a)), convex object(Fig.
5(b)), and concave object(Fig. 5(c)), we will make 4. Experimental results projections that match the shape of each object: a projection aligned with the planar shape (here- We conducted an evaluation experiment with eight parafter referred to as "planar projection"), a pro- ticipants in their 20s who had normal or corrected-tojection aligned with the convex shape (hereafter normal vision and possessed basic knowledge of comreferred to as "convex projection"), and a projec- puter graphics concepts such as parallel light sources, tion aligned with the concave shape (hereafter gradation, and specular highlights. The results are shown referred to as "concave projection"). in Fig. 7. 2. We use the Schefe’s paired comparison method to Fig. 7(a) showed significant diferences when comparsurvey participants on which specular highlight ing the convex projection with the concave projection or gradation they consider correct. for both the convex and concave objects. No significant 3. If there is no significant diference between diferences were found when comparing the other projecKimura et al.’s planar projection and the opti- tions in Fig. 7(a). There were no significant diferences in cally correct projection that matches the shape any of the combinations shown in Fig. 7(b). As shown in of each target object, it will be concluded that the Fig. 7(c), significant diferences were observed when comdiferences in specular highlights and gradation paring the planar projection with the concave projection changes due to shape are not perceived. and the convex projection with the concave projection for the convex object. Similarly, significant diferences were found when comparing the planar projection with the convex projection and the convex projection with the concave projection for the concave object. No signifi1. The experimenter explains to the participants the cant diferences were found when comparing the other direction of the light source and the shape of the projections.
(a) Evaluation results of golden yellow (b) Evaluation results of paper yellow (c) Evaluation results of silk blue
In Fig. 7, the planar projection represents the method used by Kimura et al., and the area enclosed by the white dotted line indicates the optically correct projection for that shape. Since no significant diferences were observed in any of the combinations comparing the planar projection with the optically correct projection, we confirmed that Kimura et al.’s method can also present the BRDF material appearance for non-planar objects.
In the combinations of projections where significant diferences were observed in Fig. 7(a), both selected projections were optically correct. In Fig. 7(b), no significant diferences were observed among all the projections. Many participants expressed the opinion that they "could not discern the diferences in specular highlights and gradation." This may be due to the fact that "paper yellow" represents a BRDF data with a matte appearance, resulting in minimal changes across the projections, making it dificult to determine which was correct.
In Fig. 7(c), it is hypothesized that, when comparing the planar projection with the concave projection for the convex object, the planar projection was chosen because it exhibited a smaller change in the normal direction relative to the convex object. Furthermore, it was confirmed that the optically correct projection was selected when comparing the convex projection with the concave projection for the convex object. However, when comparing the planar projection with the convex projection for the concave object, the convex projection was selected instead of the planar projection, which also showed a smaller change in the normal direction. Participants commented, "The overall appearance was darker, and the convex projection felt more correct because it was brighter than the planar projection," which is believed to have led to this result for that reason. When comparing the convex projection with the concave projection for the concave object, the convex projection was selected instead of the optically correct concave projection. Participants noted that "the specular highlights were bright, making the concave projection feel similar to the convex projection," which is likely the factor contributing to this observed result.
In the projections assuming a vertically upward normal vector, significant diferences could not be discerned compared to the correct combinations. Therefore, for projections onto curved surfaces within the curvature range of − 0.111 to 0.111, we confirmed that the optical errors caused by Kimura et al.’s method are imperceptible, allowing for the efective presentation of BRDF material appearances.
ation experiments that variations in specular highlights and gradation due to shape do not afect material perception when presenting color changes based on BRDF. We assessed whether observers could notice the incorrect presentation of specular reflection directions and the order of colors around specular highlights when performing projections assuming a planar shape, as proposed by Kimura et al., on convex and concave objects. The results indicated that even when projecting onto convex and concave objects, which have simple shape changes, participants could not perceive diferences in specular highlights or gradation. This shows that it is possible to present BRDF material appearances using Kimura et al.’s method.