Development of a Neural Network for the Synthesis of Freeform Optical Elements with a Flat Wavefront* Iana Mazur[0000-0001-7089-9099], Anna Voznesenskaya[0000-0002-4074-4341], Alexander Trifanov[0000-0002-5109-2086], Mikhail Svintsov[0000-0001-7772-7222] ITMO University, 197101, Russia, St. Petersburg, Kronverksky av., 49 ianamazur@itmo.ru,voznesenskaya@itmo.ru, alextrifanov@itmo.ru, misha.svintsow@gmail.com Abstract. In this work, a direct algorithm for modeling optical systems using freeform surfaces is considered, which allows you to form a given illumination distribution of illuminating image systems of diffraction quality. Using the pro- posed ray tracing algorithm based on the laws of geometric optics, a database of optical systems for further training of the neural network is formed. To increase efficiency, the algorithm is tested on a sample of 10,000 pairs of various optical systems. Using a neural network, the inverse problem of calculating optical sys- tems is solved - according to the given parameters of the object and image, the neural network generates a result in the form of a design of freeform optical ele- ments. Further training of the neural network will speed up the design of new optical systems, and the potential for its learning opens up new opportunities for the development of better and more efficient optical systems. Keywords: Freeform Surfaces, Freeform, Design of Optical Systems, Ray Tracing, Geometric Optics, Imaging Optics, Machine Learning, Neural Net- works. 1 Introduction At the end of the 17th century, optics formed two theories of light: wave (the light is the wave process) and corpuscular (light is the flow of particles). For several centuries, these theories were considered the opposite, but based on many experiments, it was recognized that light has a dual nature. At the same time, a new section of optics appeared - geometric optics. It studies the laws of light propagation in transparent media and the rules for constructing images during the passage of light in optical systems without taking into account its wave prop- erties. Geometrical optics introduces the concepts of a light beam that determines the direction of the energy flow, and a beam of rays - the channels within which light prop- agates. Geometric optics can be considered as the limiting case of wave optics at a wavelength of light tending to zero. Optical experiments of recent years have proved Copyright Β© 2020 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). 2 I. Mazur, A. Voznesenskaya, A. Trifanov, et al. that geometric optics cannot accurately describe the propagation of light when interfer- ence and diffraction of light become important. Moreover, the geometric approach is most common when developing optical systems for imaging and non-imaging optics for various purposes. Such functions as wave aberration, point spread function, optical transfer function, and others are used to evaluate image quality. Using computer optimization of optical systems, aberrations are minimized, however, the search for alternative methods for improving the quality of imaging optics continues. In this regard, one of the solutions is the use of freeform elements. Freeform optical surfaces have no symmetry and can replace complex optical sys- tems. Due to the use of freeform elements, energy is redistributed according to given quality criteria and, in general, the number of optical elements and their dimensions is reduced. [1] Among modern methods for calculating freeform surfaces, one can distinguish such as the Monge-AmpΓ¨re method (solving the Monge-AmpΓ¨re equation can lead to an optimal display between the source and the screen, but it is very laborious to solve these non-linear partial differential equations that require very careful numerical sampling schemes), Simultaneous Multiple Surfaces (SMS) method (can help in creating optical systems that connect a pair of incoming wavefront and corresponding outgoing wave- front, but cannot be used in creating optical systems with one optical surface, such as freeform reflectors), Ray Mapping Method (very fast and efficient when designing op- tical surfaces with a point source, but has some difficulties for setting the purpose of lighting). [2–4] Currently, machine learning technologies are actively developing - artificial intelli- gence and neural networks. The paper proposes to develop and apply a neural network for the synthesis of optical systems with freeform elements. 2 The synthesis algorithm of imaging optical systems Rays of light coming from the plane of the object with a flat wavefront pass through the freeform element and fall into the image plane (fig. 1). Development of a Neural Network for the Synthesis of Freeform Optical Elements 3 Fig. 1. Optical system with freeform element We will use machine learning to create a freeform optical element, but first we need to train a neural network. For this we need to create a lot of test objects. The required objects will be designed based on axisymmetric simple optical systems. To develop samples of optical systems for further training of neural networks, the following algo- rithm was used (fig. 2). Fig. 2. Optical Element Design Algorithm Initially, the lens surfaces are defined and the focal length of the resulting lens is deter- mined: 1 1 1 (π‘›βˆ’1)βˆ™π‘‘ = (𝑛 βˆ’ 1) βˆ™ ( βˆ’ + ) (1) 𝑓 π‘Ÿ1 π‘Ÿ2 π‘›βˆ™π‘Ÿ1 βˆ™π‘Ÿ2 where f is the focus of the lens, n is the refractive index r1, r2 are the radii of the first and second surfaces. Then the coordinates of the object (x0, y0, z0) located at the double focal distance in front of the optical element are specified. Then the point of intersection of the beam coming from the object and the first sur- face (x1, y1, z1) is calculated and the tangent and normal n to the surface at the specified point (x1, y1, z1) is determined, for which the derivative is found at the point (x1, y1, z1). The next step is to find the ray vector from the object to the first surface. 𝒂 = {π‘₯1𝑖 βˆ’ π‘₯0𝑖 ; 𝑦1𝑖 βˆ’ 𝑦0𝑖 ; 𝑧1𝑖 βˆ’ 𝑧0𝑖 } (2) The angle of incidence of the ray Ξ± on the first surface is calculated (the angle between the ray vector a and the normal n) 𝛂 = 𝐚𝐫𝐜𝐨𝐬(𝐜𝐨𝐬(𝐚,̂𝐧)) (3) 4 I. Mazur, A. Voznesenskaya, A. Trifanov, et al. The ray refraction angle Ξ² is calculated based on Snell's law and the transverse dis- placement h (fig. 3): 𝒏 𝜷 = π’‚π’“π’„π’”π’Šπ’ ( 𝟏 βˆ™ π’”π’Šπ’ 𝜢) (4) π’πŸ 𝒉 = 𝒅 βˆ™ π’•π’ˆ(𝜢 βˆ’ 𝜷) (5) where d is the distance from the surface to the screen. As a result, the offset direction is determined s: (π’š) (𝒙) 𝒔 = (𝒏𝒛 ; βˆ’π’π’› ) (6) Fig. 3. The passage of optical rays of the surface of the lens After executing this algorithm, it is possible to obtain new points, which after refraction of the rays through the lens surface will be in the coordinates found by the formulas: π’™πŸ = π’™πŸ + 𝒑𝒉 (7) π’šπŸ = π’šπŸ + 𝒒𝒉 (8) (π’š) (𝒙) 𝒛𝒏 𝒛 𝒏 𝒑 = |𝒔| , 𝒒 = βˆ’ |𝒔| (9) (π’š) (𝒙) 𝒔 = √(𝒏𝒛 )𝟐 + (𝒏𝒛 )𝟐 (10) The next transition step is to find the intersection points of the rays and the second surface. To do this, solve the system: π’™βˆ’π’™πŸ π’šβˆ’π’šπŸ π’›βˆ’π’›πŸ = = 𝑷𝒙 π‘·π’š 𝑷𝒛 { (11) 𝒛 = βˆ’βˆšπ’“πŸ βˆ’ π’™πŸ βˆ’ π’šπŸ Development of a Neural Network for the Synthesis of Freeform Optical Elements 5 Fig. 4. Result of simulating one of the optical element samples to create a sample for neural network training After executing this algorithm, it is necessary to repeat the same steps for the second surface. As a result, we can get points that belong to our image. This algorithm makes it possible to calculate the ray tracing of the optical system. These test objects must be created in order to train the neural network. As a consequence, we will be able to create more complex optical systems using free-form surfaces. Based on this algorithm, an optical system was developed (fig. 4). 3 Application of neural networks Artificial neural networks are based on the work of biological neural networks and are capable of learning with or without a β€œteacher”. Networks similar to the brain of a hu- man or animal are a complex structure. They allow you to work with information, namely to store, transmit and process it. The main characteristic of the brain is its plas- ticity, that is, the ability to adapt to the environment. The basis of the structure of this algorithm is a neural network, so you can apply various approaches: 1. Input and output data: at the input, it is necessary to set an object and an image, which is an array of data - the coordinates of the points, and the response of the neural network will be an optical surface that has a different shape depending on the specific task. 2. A neural network is a multilayer perceptron in which there is an input layer, an out- put layer, and one or two hidden layers. The input layer is a set of neurons, the num- ber of which corresponds to the set of points of the object and image. The output 6 I. Mazur, A. Voznesenskaya, A. Trifanov, et al. layer is represented by an optical surface and an additional neuron corresponding to the distance from the optical surface to the image. 3. The loss function is a mean squared error (MSE). To calculate the MSE, you need to take the difference between the predicted values and the true ones, square and average it over the entire data set. [5] 1 2 𝑀𝑆𝐸 = βˆ‘π‘š Μ‚ (β…ˆ) βˆ’ 𝑦 (β…ˆ) β€– 𝑖=1‖𝑦 (12) π‘š 4. In addition, the Adam optimization method was used with the default parameters presented in the Flux library: learning rate = 0.001, beta1 = 0.9, beta2 = 0.999. For efficiency, this algorithm is tested on a sample consisting of 10,000 pairs (source, optical surface). After that, the arithmetic average of the loss functions of each pair is calculated. The radius of the nuclei depends on the number of points N as 1 π‘Ÿ(𝑁) = 𝑁 βˆ’5 (13) A common possibility for a set of algorithms is to adjust the hyperparameters of the neural network. The number of hidden layers, neurons in them, the number of source points (and images, respectively), the number of nuclei (or RBF neurons using the RBF network) is adjusted, and the activation function for each layer is selected. [6] The training of such a neural network is reduced to the selection of weights 𝑀𝑖, shifts 𝑐𝑖 and stretches 𝛼𝑖 of the basis function. 𝑦(π‘₯) = βˆ‘π‘š 𝑖=1 πœ”π‘– πœ™π‘– (π‘₯) (14) where ϕ𝑖(π‘₯) is the function of the selected basis, πœ”π‘– is the weight of the i-th basis func- tion in a linear combination. πœ‘π‘– = πœ‘(π‘Žπ‘– |π‘₯ βˆ’ 𝑐𝑖 |2 ) (15) where ci is shifts, and 𝛼𝑖 is stretches of the basis function. RBF networks are of great importance for the formation of a specific type of freeform surface due to the fact that such architectures provide a smooth solution, and this, in turn, is a necessary condition for the effectiveness of the developed algorithm [7]. The loss function for all algorithms can be determined in various ways: 1. MSE of input image points from image points obtained using the predicted freeform surface 2. MSE of predicted freeform surface parameters from training sample parameters Development of a Neural Network for the Synthesis of Freeform Optical Elements 7 3. Linear combination of previous items. Fig. 5. Structure of RBF network 4 Results To begin with, in this work, we need to solve the direct problem of modeling an optical image using the algorithm presented in the previous chapter. To do this, you need to create a large number of optical systems to further train the neural network that can solve the inverse problem, that is, we will set the optical object and image, and the program will give us the necessary optical element. Therefore, to develop such a train- ing base, we need to set various optical objects and lenses. For example, in one of the cases, a circle of radius 1 with the number of points equal to N = 64 was used as a source. The optical surface is defined as spherical (fig. 6). The radius of the first lens surface is negative and is set as 𝑑 2 +1 π‘Ÿ=βˆ’ (16) 2βˆ™β„Ž where d is the diameter of the lens. The concave surface of the lens is described by the function: 𝑆(π‘₯, 𝑦, 𝑧) = βˆ’βˆšπ‘Ÿ 2 βˆ’ π‘₯ 2 βˆ’ 𝑦 2 βˆ’ 𝑧 (17) On given surfaces, an image on the screen is built. Then, knowing the object and image, the predicted surface shape is built (fig. 7). The loss function is the sum of the MSE points of the training image from the pre- dicted image and the MSE of the predicted shape of the optical surface. 8 I. Mazur, A. Voznesenskaya, A. Trifanov, et al. Fig. 6. Predicted surface shape Fig. 7. The resulting surface shape 5 Conclusion This work is devoted to the design of freeform optical surfaces based on the neural network algorithm. The focus is on accurate geometric modeling of the optical system. From the point of view of the developer of optical systems, solving this inverse math- ematical problem presents the greatest difficulty in the entire design process. This paper presents a method for synthesizing an optical surface using a machine learning algo- rithm for a plane wavefront. In the future, it is planned to develop an algorithm for modeling a freeform optical element for a spherical wavefront. References 1. Mescheder, H., Hermens, U: Design and Function for Freeform Surfaces. Laser Technik Journal, 15(1), 29-31 (2018). 2. Brix, K., Hafizogullari, Y., Platen, A.: Solving the Monge–AmpΓ¨re equations for the inverse reflector problem. Mathematical Models and Methods in Applied Sciences, 25(05), 803– 837 (2015). Development of a Neural Network for the Synthesis of Freeform Optical Elements 9 3. Satzer, B., Richter, U., Lippmann, U., Metzner, G. S., Notni, G., & Gross, H.: Using the 3D- SMS for finding starting configurations in imaging systems with freeform surfaces. Optical Systems Design 2015: Optical Design and Engineering VI. Jena, Germany (2015). DOI:10.1117/12.2191240 4. Ma D., Feng Z., Liang R.: Freeform illumination lens design using 5. composite ray mapping. Appl. Opt. 54(3), 498-503 (2015). 6. Rashid T.: Make Your Own Neural Network. 1st edn. CreateSpace Independent Publishing Platform, Scotts Valley, California (2016). 7. Goodfellow I., Bengio Y., Courville A.: Deep Learning (Adaptive Computation and Ma- chine Learning series) The MIT Press, Cambridge, Massachusetts (2016). 8. Dash C. S. K., Behera A. K., Dehuri S., Cho S.-B.: Radial basis function neural networks: a topical state-of-the-art survey. Open Computer Science 6(1) (2016). DOI: 10.1515/comp- 2016-0005