Computer Modeling of an Image of the Optical-Electronic System for Reference Mark Position Control Tuan Pham Ngoc, Aleksandr Vasilev, Alexander Timofeev, Valery Korotaev, and Anton Maraev ITMO University, Saint Petersburg, Russia {ngoctuan,a-s-vasilev,timofeev,vvkorotaev,aamaraev}@itmo.ru Abstract. An impact of evaluation error of reference mark image en- ergy center on accuracy of optical-electronic system for reference mark position control (OES RMPC) is considered. Basic problems related to computer modelling of reference mark image, wherein reference mark is a light source, and needed for OES RMPC accuracy estimation are an- alyzed. A general algorithm for reference mark image description taking into account its relative motion is presented. A numerical experiment of image restoration using Matlab is shown. It is demonstrated that, when information is processed by OES RMPC, image restoration algorithm by Wiener parametric filtration and Tikhonov regularization are the most effective. Keywords: Computer modeling · Image blur · Position control · Refer- ence mark· deconvolution image· Matlab 1 Introduction When a railway track is built or repaired with modern high-performance track machines, definition of the actual railway track position and estimation of the re- sult with technical means are important aspects for improvement of an operation of railway track placing in a required position [1, 2, 9]. An optical-electronic sys- tem for reference mark position control (hereinafter referred to as OES RMPC) developed at ITMO (St. Petersburg, Russia) enables to define the railway track position in the vertical plane (surfacing), vertical relative position of track rails (transverse gradient) and horizontal position (realigning) relative to reference marks conjugated to geodetic network coordinates, as the track renewal train runs. The minimal error is limited by the accuracy of reference mark image energy center definition [3, 10, 11] on the receiving senor array. This error de- pends on the number of parameters, such as video camera pixel size, optical system aberration characteristics, background luminance distribution and refer- ence mark (RM) image smear. The objective of the paper is to form a computer Copyright c 2019 for this paper by its authors. Use permitted under Creative Com- mons License Attribution 4.0 International (CC BY 4.0). 2 Pham N.T et al. model, which synthesizes a smeared RM image in different background lumi- nance distributions, and to estimate the error of coordinate definition in the restored smear-free image, considering RM relative motion. 2 Impact of evaluation error of RM image energy center on OES RMPC accuracy In the category of systems we are considering [12, 4, 5], to define the energy center (EC) of the RM image the weighted summation method is used, the method is expressed with formulas (1), (2) and provides an accuracy of less than 0.1–0.01 pixel [6, 7].  ,  M X X N XM X N XEC =  (xi,j · Qi,j )  Qi,j  (1) i=1 j=1 i=1 j=1  ,  X N M X XM X N YEC =  (yi,j · Qi,j )  Qi,j  (2) i=1 j=1 i=1 j=1 where XEC ; YEC are coordinates of the RM energy center on the sensor array, Q is a total signal from array elements. However, the error of RM image coordinate evaluation, when the receiver is exposed to the background radiation, proper detector noise and camera motion, rises dramatically, thus increasing the total error of the system. Attenuation of image distortion influence generally consists in using special- ized optical systems or high-speed cameras, which makes implementation of the system more complex. Due to use of software it is possible to eliminate a number of optical distortions by means of mathematical processing of the images, it is possible to lessen requirements to imaging system hardware [13]. 3 Restoration methods of the distorted image For an effective energy center coordinates definition, the following known meth- ods for distorted RM image restoration can be used [17, 14, 18]: • inverse filtering _ N (u, v) F (u, v) = F (u, v) + (3) H(u, v) • optimal Wiener filtration   2 _ 1 |H(u, v)| F (u, v) =   .G(u, v) (4) H(u, v) |H(u, v)|2 + Sn (u,v) Sf (u,v) • smoothing functional method (Tikhonov method) H ∗ (u, v)   _ F (u, v) = .G(u, v) (5) |H(u, v)|2 + γ|P (u, v)|2 Title Suppressed Due to Excessive Length 3 • Lucy-Richardson method ! _ _ g(x, y) f k+1 (x, y) = f k (x, y) h(−x, −y) ∗ _ (6) h(x, y) ∗ f k (x, y) _ where F (u, v) is a Fourier transform (FT) of the original RM image; N (u, v) is a FT of random value of noise; H(u, v) is a FT of a distorting operator; G(u, v) is a FT of RM image; Sn (u, v) is a noise energy spectrum n(x, y); XEC is an energy spectrum of the original image f(x,y); XEC is a FT of the Laplacian operator; Sf (u, v) - is a regularization parameter; f, g, h are F, G, H function in the spatial domain, respectively. The methods mentioned above are based on an a priory defined distortion operator H(u, v), however, when real RM images are processed, an exact point spread function (PSF) is not known or known approximately as a result of image analysis by distinct fragments [15]. For RM image processing in case the PSF is unknown a range of blind deconvolution methods can be applied [19]. Restoration of distorted images with the methods considered doesn’t prevent emergence of edge effects in restored images (e.g., false waves effect, Gibbs effect), which require additional solutions to eliminate them. The majority of distorted RM image reconstruction methods are not adequate to physical essence of the smear phenomenon and don’t take into account motion speed and RM position up to a fraction of a pixel. In the present work we consider a mathematical modelling of the distortion function, which allows to understand the essence of distortion effect, which is necessary for estimation of RM image restoration algorithms, when image coordinates are defined. 4 Computer model of the reference mark image To develop and study an image model of the RM as a light source, a mathematical model has been created, its general structure is shown in Fig. 1. To set the RM spatial position on the image with a precision up to a hundredth of a pixel, it is necessary to apply the shift property of the Fourier transform to the original object form. To do this, the original spectrum must by multiplied by the phase component with shift parameters x0 , y0 . The transfer function for RM spatial position setting can be defined as: Hd (u, v) = e−i2π(ux0 +vy0 ) (7) Results of accuracy estimation of RM position setting obtained with weighted summation using Matlab are presented in table 1. It should be understood, that when this method is used, RM image mustn’t be converted in a binary image (this procedure decreases measurement accuracy). From the results in the table above we see that accuracy of setting RM shifts using transfer function H(u, v) (7) provides an error less than a hundredth of sensor array pixel. The transfer 4 Pham N.T et al. Fig. 1. Basic information transformations in the mathematical model of RM image formation in the OES RMPC Table 1. Results of accuracy estimation of RM position setting obtained with weighted summation. Actual RM image shift, px Average measured shift, px 0.1 0.1094 0.01 0.0117 0.001 0.0016 function of the optical system, in turn, can be found through FT of function [8]: +∞ Z Z +∞ 1 − x22 − y22 −i2π(ux+uv) Hop (u, v) = e a e b e dxdy (8) 2πab −∞ −∞ When RM image f(x,y) moves relative to OES RMPC according to functions x0 (t) and y0 (t) (along spatial axes x and y respectively), the total exposure when RM image is acquired will be defined as time integral of instant exposure T. In this case, the image smear model transfer function can be found through FT as: T Hl (u, v) = sin (π(ua + vb)) e−iπ(ua+vb) (9) π(ua + vb) where a, b - are RM image shifts during exposure time T. Besides the functions shown above, RM image formation model also considers the following parame- ters: RM type, its shape and size (see Fig. 2a), background radiation distribution (see Fig. 2b), observation conditions (see Fig. 2c), properties of a detector of op- tical radiation. Based on proposed elements of generalized OES RMPC imitation model a synthesis function of RM digital image is obtained [16]: Title Suppressed Due to Excessive Length 5 Fig. 2. Examples of an object (a), background radiation (b) and radiation attenuation by the air path (c), generated by the developed model. Φ−1 n o F (x, y) = f (x, y)M ∗ [Hop (u, v).Hc (u, v).Hd (u, v)] + ηb (x, y) . (10) .ηw (x, y) + ηe (x, y) where f (x, y) - the function of the object form; M - is the function of the object scaling taking into account the distance H and focal distance f’ of the optical system; Hd (u, v) - transfer function for determining the spatial position of an object in an image with the translational property of the time delay of the Fourier transform, which ensures the spatial position of the object with the highest possible accuracy; Hop (u, v) - is the weight function of the optical system; Hc (u, v) - transfer function image blur; Φ−1 – Inverse Fourier transform; ηb (x, y) – the function of forming the distribution of background radiation, taking into account the influence of the average ambient temperature; ηw (x, y)–is the forming function of the background irradiation, ηe (x, y) - the receiver noises OES RMPC. Based on the resulting function using Matlab and a library IPT (Image Pro- cessing Toolbox) [20] a program of RM image synthesis in OES RMPC was written. Modeling results are illustrated in Fig. 3. 5 Performance analysis of restoration methods of distorted RM images synthesized by the computer model Modeling of the original RM position at 20 px smear length with RM image coordinates (319.15 px, 239.92 px) has demonstrated that the error of RM image coordinates definition for restored images is 0.0118 px for Wiener method, 0.0136 px for Tikhonov method, 0.0148 px for Lucy-Richardson method, and 0.0149 px for blind deconvolution method. Results of applying restoration methods (3) – (6) of RM images are illustrated in Fig. 4. 6 Pham N.T et al. Fig. 3. RM image synthesized by a computer model in Matlab: a) an ideal RM image without noise; b) RM image distorted by noise and background; c) an image exposed to a smear caused by relative motion of OES RMPC and RM. Fig. 4. Results of RM image coordinates definition with Matlab: a) coordinates of the smeared RM image energy center is (320.05 px, 240.57 px); RM image energy center coordinates in the image restored by b) inverse filtration are not defined; c) Wiener parametric filtration are (319.17 px; 239.93 px); d) Tikhonov method are (319.15 px; 239.93 px); e) Lucy-Richardson method are (319.13 px; 240.52 px); f) blind deconvo- lution are (319.12 px; 240.52 px). Title Suppressed Due to Excessive Length 7 Thus, application of different methods for RM image restoration has demon- strated that: - inverse filtration (Fig. 3b) demonstrates significant distortions and doesn’t allow to define energy center coordinates after image restoration; - applying of Wiener parametric filtration (Fig. 3c) and the method of smooth- ing functional minimization (Tikhonov regularization) (Fig 3d) for image restora- tion has demonstrated the best results and advisability of their use in OES RMPC. Besides this, the developed computer model can be used for estimation of energy center shift evaluation, with irradiance in the receiver plane consid- ered, as well as for optimization of OES RMPC parameters in accordance to operational conditions. 6 Conclusion In the paper it is proven that the shift transfer function (7) can set a RM image position with the precision up to a hundredth of a pixel. Mathematical descrip- tion of RM distorted image, with RM motion relative to the system considered, is developed. It is discovered that applying of Wiener parametric filtration and the method of smoothing functional minimization (Tikhonov regularization) for image restora- tion demonstrate the best restoration results and can be recommended for use in the information processing system in OES RMPC. Efficiency of applying the developed computer model to estimation of dis- torted RM image restoration methods is proven. References 1. Kalaimathi M., Ilakya P., Sathiavathy E.: Innovative railway track surveying with sensors and controlled by wireless communication. International Journal of Ad- vanced Electrical and Electronics Engineering,(IJAEEE) pp.2(3), 2278-8948 (2013) 2. Samratov U. D., Sakovich L. A., Krivdin D. G.:On the accuracy of determining the geometric parameters of a railway track using the agro-industrial complex. Geoprofi .3, 28 (2007) 3. Pantyushin A. V., Serikova M. G., Timofeev A. N.: Optoelectronic system for mon- itoring displacements, based on LED fiducial marks. Journal of Optical Technology 76(8), 507-510 (2009). https://doi.org/10.1364/JOT.76.000507 4. Yoneyama S., Ueda H.: Bridge deflection measurement using digital image cor- relation with camera movement correction. Materials transactions 53(2), 285-290 (2012). https://doi.org/10.2320/matertrans.I-M2011843 5. Feng D., Feng M. Q: Computer vision for SHM of civil infrastructure: From dynamic response measurement to damage detection–A review. Engineering Structures 156, 105-117 (2018). https://doi.org/10.1016/j.engstruct.2017.11.018 6. Tsai, Roger.: A versatile camera calibration technique for high-accuracy 3D machine vision metrology using off-the-shelf TV cameras and lenses. IEEE Journal on Robotics and Automation 3(4), 323-344 (1987) https://doi.org/10.1109/JRA.1987.1087109 8 Pham N.T et al. 7. Bing, Pan, et al:Performance of sub-pixel registration algorithms in digital image correlation. Measurement Science and Technology 17(6), 1615 (2006) https://doi.org/10.1088/0957-0233/17/6/045 8. Sjoerd Stallinga and Bernd Rieger: Accuracy of the Gaussian Point Spread Function model in 2D localization microscopy. Optics Express 18(24), 24461-24476 (2010) https://doi.org/10.1364/OE.18.024461 9. Anton V. N., Timofeev A. N., Nekrylov I. S.: The definition of the railway position control error in the plane and profile using the optical-electronic system. vol.9525, pp. 1–13. Optical Measurement Systems for Industrial Inspection IX, 95253N, Mu- nich, Germany (2015). https://doi.org/10.1117/12.2184720 10. Shavrygina M. A., Konyakhin I. A., Timofeev A. N., and Ekaterina A. V.: Optical- electronic system controlling the position of a railway track with the help of reference marks. In: Optical Modelling and Design IV, vol. 9889, p. 988916. International So- ciety for Optics and Photonics, Brussels (2016) https://doi.org/10.1117/12.2227611 11. Pham N. T., Timofeev A. N., Nekrylov I. S.:Study of the errors of stereoscopic optical-electronic system for railroad track position. In: Optical Measurement Sys- tems for Industrial Inspection XI, – T.11056. – C. 110562F. International Society for Optics and Photonics, Munich (2019) https://doi.org/10.1117/12.2526081 12. Read J. C. A.: What is stereoscopic vision good for? In: Stereoscopic Displays and Applications XXVI, T. 9391. – C. 93910N. International Society for Optics and Photonics, San Francisco (2015) https://doi.org/10.1117/12.2184988 13. Pantyushin A., and Korotaev V. V.: Control measurement system for railway track position. In: Current Developments in Lens Design and Optical Engineer- ing XIII.In Current Developments in Lens Design and Optical Engineering XIII, vol. 8486, p. 84861B. International Society for Optics and Photonics, San Diego (2012) https://doi.org/10.1117/12.930503 14. Sizikov V., Kondulukova D., Sergienko A: Improving the accuracy of restoring a dis- torted image via determining the distortion parameters from the Fourier spectrum. In: 10th Majorov International Conference on Software Engineering and Computer Systems, Saint Petersburg (2018) 15. Konukhov A.L., Kostevitch A.G., Kouryatchy M.I.:Determination of the point spread function for the characteristic fragments of images. In: 4th International Conference on Recognition-99,Tomsk (1999) (In Russian) 16. Vasilev A. S., Korotaev V. V.: Research of the fusion methods of the multispectral optoelectronic systems images. In: SPIE Proceedings Vol. 9530 on Automated Vi- sual Inspection and Machine Vision, 953007. International Society for Optics and Photonics, Jürgen Beyerer (2015) https://doi.org/10.1117/12.2184554 17. Sizikov V.S., Lavrov A.V: Stable methods of mathematical-computer processing of images and spectra. ITMO University,St. Petersburg (2018) (in Russian) 18. Gonzales R. C., Woods R. E.: Digital image processing. 4nd edn. Prentice Hall,New York (2002) 19. Campisi, Patrizio, and Karen Egiazarian, eds.: Blind image deconvolution: theory and applications. 1nd edn. CRC pressr, Boca Raton (2007) 20. Gonzales R., Woods R. Eddins. S: Digital image processing using MATLAB. 2nd edn. Pearson Education India (2004)