Synthesis methods for realistic images of three-dimensional scenes containing media with a refractive index gradient

Abstract

The paper presents the results of a study of the possibility of implementing an effective and physically correct stochastic ray tracing in gradient media based on the Runge-Kutta method. For implementation in the photorealistic rendering system, the specifics of the ray tracing method in complex three-dimensional scenes were considered. One of the main features of ray tracing in geometrically complex scenes is the large volume of geometric primitives that need to be tested for the intersection of the ray segment with the primitives. A method of ray propagation in voxel space of the scene is proposed. The method allows significant speeding up the process of searching for ray intersections with geometry primitives. To implement these ray tracing features the special program interface for gradient media was proposed, which can become the basic interface for a media of all types. Methods for calculating the luminance of all lighting components in gradient media were considered. The results of modeling the propagation of rays and image synthesis in a fiber with a refractive index gradient are presented.

1. Introduction

The solution to the problem of realistic visualization of optically complex scenes and virtual prototyping of optical devices in a real environment is based on the construction of models of physically correct propagation of light radiation in an optically complex environment. Within the framework of constructing models for the interaction of light radiation with scene objects and optical devices included in this scene, two main models are distinguished: - firstly, the conversion of light radiation at the boundaries of objects and, secondly, the propagation of light radiation in the space between the boundaries of scene objects.

Models of the conversion of light radiation at the boundaries of objects, for example, reflection and refraction of light at the boundary of dielectrics, scattering of light on the surface, described by a bidirectional scattering function, a change in polarization at the boundary of dielectrics, birefringence, etc., have gained a lot of attention in computer graphics and computational optics. Models of light propagation in a medium, as a rule, are limited by attenuation of light radiation and, in some cases, by modeling such effects as volume scattering and fluorescence. However, all these models are based on the assumption that the propagation of light is rectilinear or straightforward. Even modeling of such effects as volume scattering and fluorescence are also based on the assumption that the propagation of light is straightforward. The specificity of these models is that the straightness of light radiation is limited by extinction events that occur when a beam “hits” a scattering particle. In this case, the particles are not defined explicitly but are reduced to such parameters as the extinction cross-section, which determines the probability of the beam “being captured” by the scattering particle, and the phase function, which determines the character of the light scattering by the particle and plays the role of a bidirectional scattering function of the surface. As a result, the ray path in a scattering or fluorescent medium is a broken line, consisting of rectilinear segments.

If the optical properties of the medium (refractive index) change continuously, then, following the Fermat principle, the ray path takes the form of a curved line having a minimum optical path from the start to endpoints of the path. The ray path is determined by the eikonal equation [1], for which, in general, there are numerical solutions [2, 3]. In computing optics, solutions are used to calculate the ray path in a gradient lens environment. However, the solutions used in computational optics are used for simple geometric shapes that bound the gradient lens and, in most cases, the laws of change in the refractive index are analytical functions that have simple solutions.

The ray tracing methods used in computer graphics are fundamentally different from the methods of computational optics. The main difference is the number of geometric objects in the scene. If in computing optics the number of geometric primitives that limit the gradient medium is generally measured by units, then in computer graphics systems this number can reach tens of millions. Besides, in computer graphics systems, the gradient of the refractive index may not be an analytical function, but rather be an analog of a three-dimensional texture that varies the refractive index of the medium. These differences lead to significant changes in software interfaces and ray tracing algorithms. Also, computer graphics systems are not limited to ray tracing. Their task is to calculate the apparent luminance of the scene, and gradient media make it impossible to use standard algorithms for calculating the luminance components of direct, secondary and caustic illumination.

In this paper, we consider methods of ray tracing in gradient media inside a complex geometric environment, methods for calculating the luminance components of direct, secondary, and caustic illumination and solutions for the unification of ray tracing methods in gradient media for computer graphics systems and computational optics. 2.

Materials and method

In the approximation of geometric optics the law of light propagation in a gradient medium is derived from the Maxwell equations: or eikonal. where (⃗ ) = ∫ 0 (⃗′ ) ∙⃗

Passing to the geometric approximation, the eikonal equation in vector form can be expressed as the following: (⃗ , ) = (⃗ ) − (⃗ , ) = (⃗ ) − = 0(⃗ ) = 0(⃗ ) ′ represents the optical path ⃗ (⃗ ) = ∇ (⃗ ) ( 2 ) coordinate⃗ and thus 2 ⃗ turns into a straight line. where (⃗ ) is the refraction index of the medium at the point⃗ , and ⃗(⃗ ) = ⃗ is the direction (unit vector) of the propagation of light energy. Fig. 1 shows the curved trajectory of the light ray and the vector of its direction at the point⃗ .

Obviously, in a homogeneous medium, the refraction index of the medium (⃗ ) does not depend on the space 2 = 0. As a result, the ray path

Therefore, in most cases, another approach is used to form the ray path.

To solve the differential equation ( 2 ), an approach based on the Runge-Kutta method is used. We introduce the following notation: = ⎧ ⎪ ⎪ ⎩ ⎨ ⃗(⃗ ) = ⃗

= (⃗ ) ⃗(⃗ ) = (⃗ ) ⃗( ⃗) = ( ⃗)∇ ( ⃗) ⃗ ( 3 ) ( 1 ) ( 4 ) ( 5 ) ( 6 ) ∫ ∫ 0 0 ( ⃗′)∙ ⃗′− ( ⃗′)∙ ⃗′− = 0(⃗ ) ( ⃗) = 0(⃗ ) ( ⃗) of the refractive index. where is the reduced parameter of the ray path, ⃗(⃗ ) is the optical ray vector, ⃗(⃗ ) is the parameter of variation

As a result, after substituting expressions ( 3 ) into equation ( 2 ), the eikonal equation is transformed to a first-order differential equation, which can be solved numerically by the Runge-Kutta method.

( ⃗)

= (⃗ ) varies from 0 to N.

To solve this equation, discretization is performed along with the curved segments of the ray path, where i The initial parameters of the ray are known: ⃗( 0, ⃗0(⃗ 0), (⃗ 0)) and as a result of successive iterations, the parameters of the ray at the endpoint N can be calculated: ⃗( , ⃗

(⃗ ), (⃗ )).

An algorithm

for constructing a ray path can be represented as follows: ( 1 )

We specify a certain increment Δ of the ray path, which can be selected based on considerations of variation of the refraction index in the region of the or proximity to the boundaries of the ( 2 ) Following the Runge-Kutta method, the following parameters are calculated recursively, starting from point ⃗ 0 medium. point = 0: ⎧ ⎪ ⎩ ⎪⎪ ⃗ = Δ ⃗ ⃗ +

⃗(⃗ ) + ⎨ ⃗ = Δ ⃗ ⃗ + Δ ⃗(⃗ ) + ⃗ = Δ ⃗(⃗ ) Δ 2 ⎪ ⃗(⃗ +1 ) = ⃗(⃗ ) + 1 6 Δ 8 Δ 2 ⃗ ⃗ ⃗ + 4 ⃗ + ⃗ ( 3 ) Following the calculated parameters, the ray is transferred to the point i + 1 and at this point, its parameters are calculated: coordinates, energy propagation direction and eikonal: ⃗ +1 =⃗ + Δ ⃗(⃗ ) + ⃗ + 2 ⃗ ⎧ ⎪ ⎪ ⎪ ⎨ ⎪ ⎪ ⎪ ⎩ 0Δ 2

6 point. 1 6 ⃗(⃗ +1 ) (⃗ +1 ) ⃗+1 = 0Δ

2 ⃗(⃗ +1 ) ⃗(⃗ +1 ) − ⃗(⃗ ) ⃗(⃗ ) (⃗ +1 ) = (⃗ +1 ) + [ 2(⃗ +1 ) + 2(⃗ )] − ( 4 ) The process is repeated until the ray reaches a given

This algorithm provides high accuracy of ray transfer in a gradient medium, ensuring the continuity of its trajectory. Fig. 2 (b) illustrates the specifics of the ray tracing algorithm in a gradient medium.

Ray tracing algorithms

The above approaches allow ray tracing in media with a refractive index gradient. However, these methods are suitable for unlimited environments. In reality, all media are limited and it is necessary to take into account the shape of the geometric objects that bound this medium. Two main types of constraints of the gradient medium can be distinguished. Firstly, these are simple optical elements, for example, gradient lenses [4-7]. The peculiarity of these objects is a small number of geometric shapes that limit this environment. As a rule, these are simple analytical objects, such as planes, cylinders, and spheres. Secondly, these are complex three-dimensional scenes that can form the limitation of a gradient medium consisting of millions of independent triangles. Naturally, the search algorithms for the intersection of the curved path of the beam with the boundary of the gradient medium will be specific for these two cases.

In the first case, it is enough to implement an additional method for a geometric object, which will inform you on which side of the surface there is a point offset from the current position by a distance Δ . If the point remains in the gradient medium, then the ray tracing process (formulas ( 5 ) and ( 6 )) continues. If the point leaves the gradient medium, then the iterative process of refinement of the search for the point of intersection of the ray with the boundary surface begins. Наиболее The simplest process is to search for the intersection of the straight segment of the ray formed either by the chord ( ⃗ +1 − ⃗ ), or tangent to the ray path ( ⃗ ). The obtained distance is converted into the parameter Δ and the calculation of the new position of the point i+1 starts from point i. This process is repeated until the point i+1 approaches the surface so close that the last approximation can be replaced by a simple rectilinear segment of the ray path. As a rule, two or three iterations are enough to find the point of intersection of the beam with optical accuracy.

In the second case, when ray tracing in a threedimensional scene containing millions of triangles, the situation is completely different. The main reason is the spatial partitioning of the scene. The ray is not traced directly from the surface to the surface. The beam propagates in a voxel space, which divides the medium into some volumes, usually in the form of a parallelepiped. These volumes may or may not contain elements of the boundary of the scattering medium. Before reaching the boundary of the medium, the ray must sequentially cross and process all the voxels located on its path. Therefore, in addition to the algorithm for searching for the intersection of a ray with geometry (which does not fundamentally differ from the algorithm considered in the first case), it is necessary to implement an algorithm for ray tracing in voxel space. Fig. 3 illustrates the problem of ray tracing in a spatially partitioned gradient media. Geometric objects in the scene are tied to spatial voxels and, to accelerate the ray tracing process, the search for the point where the ray meets these objects is carried out only when the ray enters the corresponding voxel. The algorithm proposed for finding the point of intersection of the ray with the surface is not applicable for searching for the entry point to the voxel, since the voxel found after transferring the ray to the point ⃗ +1 may not be the next one. It may ultimately lead to the omission of a geometric object. The use of a chord or tangent segment of a ray can also lead to the problem of skipping a geometric object. Fig. 3 illustrates this possibility. Therefore, to search for the next voxel and its entry point, it is necessary to find the point of intersection of the ray with the boundary of the current voxel. Since the voxel, as a rule, has the shape of a parallelepiped with planes parallel to the coordinate planes, the algorithm for finding the intersection point with its boundaries is greatly simplified: ( 1 ) We specify the starting point ( ⃗0, ⃗0( ⃗0)) and (based on the parameters of the gradient medium) the ray transfer parameter Δ . ( 2 ) Parameters ⃗, ⃗, ⃗, ⃗( ⃗ +1) are calculated by the formula ( 5 ), and then ⃗ +1 by formula ( 6 ). ( 3 ) If the point ⃗ +1 lies inside the voxel, then the ray transfer parameter Δ is taken as the initial parameter to search for the point where the beam meets the geometric objects inside the voxel. ( 4 ) If the point ⃗ +1 is outside the voxel boundary, then an iterative approaching is made to the voxel boundary, the task of which is to find a point on the boundary and determine the beam transfer parameter to this point Δ . In this case, the point ⃗ +1 and the transfer parameter Δ are chosen in such a way that the point turned out to be a small distance beyond the voxel border. However, if no intersection with the geometry inside the voxel was found, then to search for a new border inside the next voxel, the point and the transfer parameter return to a short distance inside the current voxel. ( 5 ) If an intersection with a geometric object inside the voxel was found, then the beam is converted at the boundary of the geometric object. And, if the beam remains in the gradient medium, the procedure for searching for the intersection of the beam with the boundary of the current voxel is repeated.

To implement the ray tracing method in threedimensional scenes containing gradient media, a gradient media program interface was implemented, which provided the basic functionality necessary for ray tracing. The main methods of the environment interface should: the point ⃗ in the direction ⃗ ( ⃗ ). If the medium does not have gradient properties, then the transport parameter is set to infinity and a direct ray tracing is realized. The beam transfer parameter can take into account the spatial partitioning properties of the scene and, if necessary, be calculated up to the border of the nearest voxel. ( 2 ) For the set 0, 1, 2, … of wavelengths determine the subset of

wavelengths for which ray tracing along one path is possible, i.e. no dispersion. ( 3 ) Calculate the refraction index of the medium at the point ⃗ .

medium at the point ⃗ .

⃗ +1 трассы луча ⃗ +1( ⃗ +1). ( 5 ) Осуществлять перенос луча из точки ⃗ в току и вычислять новое направление в конце ( 6 ) Calculate the optical path and geometric path of the ray from the point ⃗ to the point ⃗ +1.

( 4 ) Calculate the gradient of the refraction index of the process.

transferred from the point ⃗ to the point ⃗ +1.

These interfaces are enough to implement ray tracing in gradient media of optical devices and threedimensional scenes. Besides, the implementation of these programming interfaces at a basic level will allow the implementation of ray tracing methods that will not depend on the properties of the environment in which they are distributed. This solution will greatly simplify the implementation of image synthesis methods and, in some cases, will avoid the need to impose additional conditions on the scene parameters in the rendering 4.

Luminance calculation algorithms

The methods for calculating the luminance of the scene's scattering surfaces located in gradient media have their specifics. The visible luminance of the scene object is determined by the well-known formula [8]: ( , ⃗, ⃗) = ( , ) ( , ⃗) ′( , ⃗) 1 4

0( , ⃗, ⃗) + where: 0( , ⃗, ⃗) is the own luminance of the observed object at a wavelength , at a point ⃗ and in the direction ⃗, ( , ) – medium transmission at wavelength and on the path t from the luminance source to the observer, ( , ⃗) – index of refraction at the observer point, ′( , ⃗) – index of refraction at the point of formation of luminance, ( , ⃗, ⃗′) - the luminance of the light source illuminating the surface at a wavelength , at a point ⃗ in the direction ⃗′, ⃗ – the direction of the local normal to the surface

at ( , ⃗, ⃗, ⃗′) the point of

illumination The bidirectional scattering ⃗, distribution function of the surface (that is how many times the brightness of a surface under given lighting and observation conditions differs from the brightness of an ideal diffuser) at wavelength , at a point ⃗, in direction of illumination ⃗′ and in the direction of observation ⃗. The integration of luminance is carried out over the entire hemisphere of the illumination of the observation surface.

For the computing

method, the luminance can be represented as the sum of the four components visually presented in Fig. 4: ( 1 ) The luminance of direct vision is the intrinsic luminance of the surface that the observer sees directly or through a series of "mirror" surfaces. For surfaces in gradient media, this luminance component can be calculated directly, for example, by the ray tracing method. ( 2 ) The luminance of caustic illumination is the brightness of the surface that the observer sees directly or through a series of “mirror” surfaces illuminated calculate through the “mirror”

surfaces. this luminance component, the

To most suitable method would be a method based on the use of photon maps [9]. Caustic lighting maps are created by stochastic rays emitted from light sources, stored, and then “read” in accordance with equation ( 7 ) as the intrinsic luminance of the observed object. This approach is technically applicable for surfaces in gradient media, and requires only additional analysis of the ray hit the caustic map. ( 3 ) The luminance of direct illumination is the luminance of the surface that the observer sees directly or through a series of “mirror” surfaces directly illuminated by light sources. To calculate this brightness component, as a rule, the method of multiple importance sampling is used, weighting the brightness learned from the choice of points on the light source (light sampling - which allows you to calculate the luminance of direct illumination using radiometric ratios) and the choice of direction in the bidirectional scattering function (BDF sampling allowing the method of calculating shadow rays to find the brightness of visible light sources) [10]. In most cases, the main contribution is made by the method of choosing points on the light source, however, this method cannot be applied to the case of surface illumination through gradient media (radiometric ratios do not allow calculating luminance efficiently and correctly). The method of choosing directions for the bidirectional scattering function has several serious limitations, for example, it cannot be applied to scenes containing small-sized light sources. Therefore, to calculate the brightness when illuminating a surface through a gradient medium, it is necessary to use the photon map method, which is technically implemented as a method for calculating the luminance of caustic illumination. If there are extended light sources of large size, weigh it with the method of selecting directions according to the bidirectional scattering function. ( 4 ) The luminance of the secondary illumination is the brightness of the surface that the observer sees directly or through a series of “mirror” surfaces illuminated by light scattered on the diffuse surfaces of the scene. To calculate this luminance component, forward ray tracing method, backward ray tracing methods, path tracing methods [11], or various options based on bi-directional ray tracing methods [12] are used. If the scene contains gradient media, then the use of ray tracing methods and methods of path tracing in most cases becomes ineffective. The main reason is the low probability that the rays hit the observer’s receiver (in forward ray tracing methods) or the light source (in backward ray tracing methods). The classical methods of bi-directional ray tracing can also be inefficient since they allow the possibility of connecting the paths of forward and backward rays through gradient media, which cannot be effectively implemented in the physically correct approximation of ray tracing. Therefore, the most suitable solution to the problem of calculating the brightness of secondary illumination is the method of bi-directional ray tracing using photon maps. From a practical point of view, this method is implemented similarly to the method for calculating the brightness of caustic illumination. The only difference is that photonic maps are formed at the second and further distant diffuse scattering events.

The above solutions allow you to implement physically correct rendering of scenes containing gradient media. 5.

Results

The ray tracing method and photorealistic rendering, based on the forward stochastic ray tracing method in scenes containing gradient media, was implemented as part of the Lumicept computer-based photorealistic image synthesis system [13]. Besides, a method for visualizing ray paths propagating in a medium with a refractive index gradient was implemented.

As an example, a scene was constructed consisting of a cylindrical fiber with a refractive index varying from axis to edge (as shown in Fig. 5), a small-sized LED source illuminating the end of the fiber, and a radiation receiver which detects the component of caustic illumination on the opposite end of the fiber. As an alternative, a scene was built consisting of a series of cylinders in optical contact. The radius of the cylinder was determined from the condition that the refraction index changes by 0.005. Modeling was carried out at various parameters of the beam displacement. The number of steps varied from 20 to 100. The ray path remained practically unchanged and the synthesized image remained unchanged. The simulation results showed a match with the simulation results for the alternative scene. Images of several ray paths and the distribution of illumination behind the end of the fiber are shown in Fig. 5.

Fig. 5. Variation of the refractive index from the center to the edge of the fiber, the beam path inside the fiber and the distribution of illumination behind the end of the fiber (from left to right)

The coincidence of the simulation results obtained in various ways indirectly confirms the correctness of the chosen implementation of the ray tracing method.

6. Conclusion

In the framework of this study, an effective and physically correct method of ray tracing in gradient media was proposed. For implementation in the photorealistic rendering system, a gradient software interface was proposed, which can become the basic interface for all types of media. Methods were proposed for calculating the luminance of all lighting components in gradient media.

The main tasks that are planned as part of the expansion of the proposed approach are, firstly, to determine the optimal parameter for beam movement in free space, which should provide a given accuracy of the position of the new point in space and the new direction of beam propagation, and, secondly, to develop an effective method for determining the parameter of ray tracing to the boundary of the spatial cell of the scene (voxel).

Asknowledges References

The work was supported by RFBR, Grants № 18-0801484, 19-01-00435, 20-01-00547.

References

[1] Max Born and Emil Wolf , Principles of Optics. Pergamon Press, Fourth edition , 1970 .

[2] Handbook of optics / sponsored by the Optical Society of America; Michael Bass , editor in chief. - 2nd ed., McGraw-Hill , Inc., 1995 , ch.9.

[3] R. K. Luneburg , Mathematical Theory of Optics , University of California Press, 1966 , pp. 182 - 195 .

[4] D. T Moore , "Gradient-Index Optics: A Review," Appl. Opt . 19 , 1038 - 1035 ( 1980 )

[5] E. W. Marchand , [Gradient Index Optics], Academic Press, New York, NY, ( 1978 ).

[6] Applied Digital Optics: From Micro-optics to Nanophotonics Bernard C. Kress and Patrick Meyrueis 2009 John Wiley & Sons, Ltd

[7] Sawyer D. Campbell , Jogender Nagar, Donovan E. Brocker, John A. Easum, Jeremiah P. Turpin, Douglas H. Werner , "Advanced gradient-index lens design tools to maximize system performance and reduce SWaP," Proc. SPIE 9822 , 98220P ( 17 May 2016 ); doi: 10.1117/12.2223040

[8] Kajiya , James T. ( 1986 ), "The rendering equation" (PDF) , Siggraph 1986 : 143 - 150 , doi: 10.1145/15922.15902

[9] Jensen H.W. ( 1996 ) Global Illumination using Photon Maps . In: Rendering Techniques ' 96 . Eurographics, pp. 21 - 30 . Springer, Vienna. https://doi.org/10.1007/978-3- 7091 -7484- 5 _ 3

[10] Vincent Pegoraro, [ Handbook of Digital Image Synthesis: Scientific Foundations of Rendering] , CRC Press, 2017

[11] Kang , C. , Wang , L. , Xu , Y. et al. A survey of photon mapping state-of-the-art research and future challenges . Frontiers Inf Technol Electronic Eng 17 , 185 - 199 ( 2016 ). https://doi.org/10.1631/FITEE.1500251

[12] Georgiev , I. , Krivanek , J. , Slusallek , P. : Bidirectional light transport with vertex merging . In: SIGGRAPH Asia 2011 , pp. 27 : 1 - 27 : 2 , ACM , New York, NY, USA ( 2011 ).

[13] Hybrid Light Simulation Software Lumicept // Integra.jp. URL: https://integra.jp/en/products/lumicept (date of call 10.03 . 2020 ) Zhdanov Dmitry D., PhD , Head of the Visualization Technology Chair of ITMO University. E-mail: [e-mail]. Potemin Igor S., PhD , Assistant of the Visualization Technology Chair of ITMO University. E-mail: [e-mail]. Zhdanov Andrey D., Ph .D student at ITMO University, Visualization Technology chair. E-mail: [e-mail] Galaktionov Vladimir A., Professor, Doctor of Science in physics and mathematics, head of the Department of computer graphics and computational optics at Keldysh Institute of Applied Mathematics RAS . E-mail: [e-mail] Garbul Alexey A., researcher of Department of computer graphics and computational optics at Keldysh Institute of Applied Mathematics RAS . E-mail: tnmik@gin .keldysh.ru