Image-based System for 3D Visualization of Flow in Hydrodynamic Tunnel ? Vladimir Knyaz1,2[0000−0002−4466−244X] , Mikhail Novikov3[0000−0003−0626−793X] , Vladimir Kniaz1,2[0000−0003−2912−9986] , Vladimir Mizginov2[0000−0003−1885−3346] , and Eugeny Ippolitov3[0000−0002−0622−6727] 1 Moscow Institute of Physics and Technology (MIPT), Dolgoprudny, Russia 2 State Research Institute of Aviation System (GosNIIAS), 125319 Moscow, Russia {knyaz,vl.kniaz,vl.mizginov}@gosniias.ru http://www.mipt.ru 3 ILIT RAS – Branch of the FSRC “Crystallography and Photonics” of Russian Academy of Sciences, Shatura, Russia novikov@rambler.ru,ippevg@ya.ru Abstract. Aircraft safety depends on wing flow process, so the study of air flow in different flight conditions is one of the most important parts of aircraft design and exploiting. The effective method of aerodynamic processes modeling is ex- periment in wind (aerodynamic) tunnel or water (hydrodynamic) tunnel. They allow to perform experiments with a scaled model of an aircraft and to visu- alize the wing flow process. A visualization and video registration of the wing flow yields useful qualitative information about flow, but it is more important to retrieve quantitative 3D data of flow for 3D visualization and analysis. The pre- sented study addresses to creating an image-based system for accurate 3D flow acquisition for further diverse 3D visualization and quantitate evaluation of 3D flow parameters in a hydrodynamic tunnel for aircraft icing influence exploration. Being an initial part of a long-term research project, this study is aimed at de- veloping stereolithography (SLA) modeling technique for flow visualization in hydrodynamic tunnel and a photogrammetric system for accurate flow 3D cap- tion. The results of first experiments of the system calibration and application are given along with preliminary results of flow jets 3D reconstruction. Keywords: Image-based 3D Measurements, Hydrodynamic Tunnel, Aircraft Ic- ing, Multimedia Imaging, 3D Visualization. 1 Introduction A comprehensive study of flow behavior is crucially important for aircraft performance and safety. Nowadays a scaled modeling of flow in aerodynamic or hydrodynamic tun- nel confirmed to be an effective tool for flow analysis. Copyright © 2020 for this paper by its authors. Use permitted under Creative Commons Li- cense Attribution 4.0 International (CC BY 4.0). ? The reported study was funded by Russian Foundation for Basic Research (RFBR) according to the research project 19-29-13040. 2 V. Knyaz et al. Physical flow modeling in aerodynamic or hydrodynamic tunnel plays important role in the process of an aircraft design due to possibility of controlled study of flow behavior for various flight conditions, including critical attack angles or wing icing [1]. The basis for such modeling is the similitude concept. The adequateness of modeling is provided by keeping the similarity of a model geometry and dynamic characteristics for real and model processes. Similitude is widely applied in aerodynamic and hydro- dynamic to study flow behavior using scaled models. The similitude concept allows to use liquid instead of air when studying flow be- havior. Such possibility arises from the key condition of similarity real and model flow process expressed in the equivalence of main dimensionless similarity criteria such as Reynolds number, Mach number, Prandtl number. Flow modeling in a hydrodynamic tunnel has the important advantage of relatively slow velocity of flow, comparing with aerodynamic case. This feature makes hydrodynamic modeling to be an attractive tool for flow visualization and studying. While well-established techniques for 2D flow registration and visualization based on colored smoke or liquid exist, the accurate 3D flow registration and measurement are more challenging tasks. The main challenges for the accurate 3D measurement by photogrammetric techniques are imaging through optical interfaces and image-based 3D reconstruction of flow shape. The paper presents the developed photogrammetric system for accurate flow 3D registration and processing aimed at further 3D visualization of the flow. 2 Related work Vision-based methods of registration, visualization, and analysis have proven to be an effective tool for qualitative and quantitative flow characteristics study in aerodynamics and hydrodynamics. 3D laser scanning is successfully applied for ice accretion geometry acquiring and further applying the obtained results for disturbed flow investigation. A commercial 3D laser scanning system was used for recording ice accretion geometry in the NASA Icing Research Tunnel [2] with the aim of the geometric assessment of the 3D laser scanning system on a 2D (straight wing) and a 3D (swept wing) airfoil geometries. The comparison of scanned ice accretion with castings of the same ice accretion has proved the possibility of wing icing modeling. The ice-accretion molds has been used to produce one set of artificial ice shapes from polyurethane castings [3]. The laser-scan data has been utilized to fabricate an- other set of artificial ice shapes, using rapid prototype manufacturing (stereolithogra- phy). The iced-airfoil results with both sets of artificial ice shapes were compared to estimate the aerodynamic simulation accuracy of the laser-scan data. For four of the six ice-accretion cases, there was excellent agreement in the iced-airfoil aerodynamic performance between the casting and laser-scan based simulations. Mid-infrared (MIR) laser scanning demonstrated high measuring performance for acquiring the 3D ice shape for various types of ice: clear ice, rime ice, mixed ice, or even supercooled water droplets or films [4]. The advantage of these wave is the property of mid-infrared radiation to penetrate ice and water only within a depth of less than 10 Image-based System for 3D Visualization of Flow in Hydrodynamic Tunnel 3 micrometers. Several sequential scans were applied to trace the 3D shape evolution of the continuous ice accretion on an airfoil in an icing wind tunnel. The ice growth process can be well observed in the results. The mid-infrared scan shows a good agreement with the traditional visible laser scan on a plastic replication of the final ice shape made by the mold and casting method. To obtain accurate 3D geometric parameters of the flow in a hydrodynamic tun- nel it is required to account for multimedia optical working space, including air, glass and liquid (oil or water). Special models of imaging and calibration of an optical mea- surement system provides reliable and accurate 3D data. Various approaches to optical system calibration for measurement through multimedia optical interfaces can be found in [5,6,7,8], including development of distance dependent distortion or application of special ports for camera [9,10] to reduce refraction effects. A special technique for pho- togrammetric system calibration directly considering imaging through two media inter- faces [11] allows to perform accurate 3D measurements in a hydrodynamic tube. This calibration technique provides not only qualitative but also quantitative characteristics of a flow that are needed for icing process study. The standard approach to densely reconstruct the flow motion utilizes high-contrast particles for flow visualization. Then a set of synchronized high-speed cameras ac- quire video sequences of flow motion for further processing. The acquired registra- tions are processed in two separate steps, utilizing either Eulerian or Lagrangian ap- proach [12,13,14,15]. Eulerian methods perform a voxel-based reconstruction of parti- cles per time step, followed by 3D motion estimation, with some form of dense match- ing between the precomputed voxel grids from different time steps [16,12]. Alterna- tively, Lagrangian methods reconstruct an explicit, sparse set of particles and track the individual particles over time. Physical constraints can only be incorporated in a post- processing step when interpolating the particle tracks to a dense motion field [13]. Recently methods for object shape 3D reconstruction appears that incorporate deep learning for multi-view or even single-view 3D shape reconstruction [17,18,19,20]. These methods look promising for further research in 3D flow analysis. 3 Hardware and algorithms 3.1 Hydrodynamic tunnel The possibility of studying flow motion at low flow speed makes hydrodynamic tun- nel (Figure 1) an attractive solution for complicated cases, when high velocity (when modeling in aerodynamic tunnel) prevents for detailed consideration of a process. The adequateness of such modeling to real air flow is provided by correspondence of the similitude criteria such as Reynolds number R, indicating the ratio of inertial forces to viscous forces within a fluid. Having equal Reynolds number liquid flow speed is significantly lower than the speed of air flow for the same flow characteristics. So hydrodynamic tunnel allows to work with very low flow velocity (flow velocity V = 2 − 10 cm/s) for the considered conditions. An aircraft icing process and its influence on aircraft flying characteristic is studied in a very low-speed hydrodynamic tunnel HDT-400 of the Central Aerohydrodynamic 4 V. Knyaz et al. Fig. 1. Flow visualization in hydrodynamic tunnel Institute (TsAGI). HDT-400 has a working part size of 400 × 400 mm2 . This hydro- dynamic tunnel has been successfully applied for the analysis of unsteady localization phenomena of “explosion” of vortices. To visualize flow jets a specially designed model of a plane and colored liquid was used. The aircraft model during the test in HDT-400 is shown in Figure 1. Special channels inside the aircraft model provide coloring of flow jets for observation. 3.2 SLA-technique for flow visualization To visualize flow jets in a hydrodynamic tunnel, it is necessary that the test model has internal channels for the passage of paint, which will color the studied flow. One of the most accurate and fast ways to obtain such a model is additive manufacturing using laser stereolithography (SLA). Fig. 2. SLA model for flow study in HDT-400 Image-based System for 3D Visualization of Flow in Hydrodynamic Tunnel 5 This technology allows to make thin sealed channels of any shape, without using the support structure, and the remains of the photo-polymerized composition can be removed by purging and washing, provided that the polymer has sufficient fluidity. Another advantage of modeling in hydrodynamic tunnel is relatively low dynamic loads on an aircraft model, so models produced by laser stereolithography technology can be directly used for study. Stereolithography allows to produce very complicated forms with high accuracy, including producing complex inner topology of a model (Fig- ure 2). The complicated shape of the inner channels requires special means for aircraft model producing. The powerful technique for creating such complicated models is ad- ditive technology. The model of the aircraft shown in Figure 2 is produced with the aid of SLA-150 equipment developed by Institute of Laser Information Technology (ILIT) RAS. 3.3 Optical 3D measurement system Image-based system for 3D flow motion registration leverages photogrammetric tech- niques. Its core is 3D motion capture system ”Mosca” [21] extended for 3D measure- ments through several optical media interfaces and 3D reconstruction of flow jets (Fig- ure 3). Fig. 3. The laboratory 3D measurements setup. The scalable 3D motion capture system ”Mosca” provides high speed image ac- quisition and high accuracy of photogrammetric 3D measurements. For flow study in HDT-400 it has been equipped with DMK 37AUX273 USB 3.1 cameras and adjusted for working space of 300 × 300 × 300 mm. Main characteristics of DMK 37AUX273 camera is shown in Table 1. The developed original software supports image acquisition from 2 . . . 4 cameras in synchronization mode at frame rate up to 200 frames per second and accurate 3D motion 6 V. Knyaz et al. Table 1. Main characteristics of DMK 37AUX273 camera. Parameter Value Vision Standard USB3 Vision Dynamic Range 10 bit Resolution 1440x1080 Frame Rate at Full Resolution 238 Pixel Formats 8-Bit Monochrome 12-Bit Packed Monochrome 16-Bit Monochrome Optical Interface IR-Cut filter No Sensor Type Sony IMX273LLR-C Shutter Type Global Sensor Format 1/2.9 inch Pixel Size 3.45 µm Lens Mount C/CS Electrical Interface Interface USB 3.1gen1 Supply voltage 4.75 VDC to 5.25 VDC Current consumption approx 380 mA @ 5 VDC Mechanical Data Dimensions H: 36 mm, W: 36 mm, L: 25 mm Mass 70 g Adjustments Shutter 1 µs to 30 s Gain 0 dB to 48 dB reconstruction based on original calibration and point tracking procedures [22]. The ”Mosca” system has been configured for image acquisition in hydrodynamic tunnel. The laboratory setup for system calibration and 3D measurements is shown in Figure 3. For applying ”Mosca” system for 3D measurements in hydrodynamic tunnel it is necessary to account for a phenomenon of light refraction at interfaces of different op- tical media. The original techniques for calibration and 3D measurements, based on the detailed analysis of light passing through the different optical media has been devel- oped and evaluated [11]. The developed techniques provide accurate measurements of 3D coordinates in working space 300 × 300 × 300 mm with the accuracy of 0.03 mm. 4 Visualization of the captured data Test video sequences of flow modeling have demonstrated reasonable quality of flow visualization for qualitative 2D analysis of flow behavior for given modeling conditions. Figure 4 presents samples of images acquired in HDT-400 during SLA-model testing. At current stage of the study the imaging configuration of 3D flow motion registra- tion is determined. The laboratory experiments proved high accuracy of 3D measure- Image-based System for 3D Visualization of Flow in Hydrodynamic Tunnel 7 Fig. 4. Registration of an image sequence of a flow ment of the developed system. The system demonstrates necessary technical character- istics for its applying for flow registration in HDT-400 hydrodynamic tunnel. Although an accurate and fast-speed registration of a flow by the developed optical 3D measurement system in HDT-400 hydrodynamic tunnel is the next step of the study, the preliminary experiments for flow 3D reconstruction and 3D visualization were car- ried out, using available flow registration (Figure 4). As video sequence acquisition has been made by cameras with unknown parameters of interior and exterior orientation, only qualitative 3D reconstruction could be performed. A 3D Shape from Silhouette [23,24,25] technique has been applied for qualitative 3D flow jets reconstruction. The silhouette, or occluding contour of a shape in an image contains some information about the 3D shape of the object. Specifically, an object silhouette in an image defines the solid angle based on the projection of this object to the image plane. This solid angle can be found by back-projecting the area inside the silhouette using the parameters of camera orientation. An exterior orientation parameters include camera location X, Y, Z and angular orientation α, ω, κ in some reference system of coordinates. For the cameras used for acquiring test images (Figure 4) this information is not available. So the following tech- nique for determining exterior orientation parameters has been applied. The geometry of the aircraft stereolithography model applied for the experiment corresponds (with high level of the accuracy) to the digital CAD-model, that serves for the stereolithography 3D printing process. This CAD model contains full and accurate information about the geometry of the aircraft model. Using CAD information about a set of aircraft specific points as reference data, cameras orientation parameters in CAD- model coordinate system has been estimated. These parameters then were used for 3D shape estimation of the jets. As the number of cameras was minimal (only two), the additional supposition has been used for shape reconstruction, specifically, that flow jets have symmetry relatively the central trace of a jet. Another word, it has been supposed that two more (virtual) cameras has been located symmetrically to the real ones, and they has acquired two additional (symmetrical) images. 8 V. Knyaz et al. Figure 5 presents the results of the 3D reconstruction of colored flow jets by voxel- based Shape from Silhouette techniques. Fig. 5. 3D visualization of flow jets. Standard voxel-based approach has been used for a flow jets shape reconstruction. It includes the following main steps: 1. Split up the working space of HDT-400 into a 3D voxel grid of given voxel size; 2. Determine a flow jet borders (silhouette) in the acquired image; 3. For each camera calculate an intersection of silhouette solid angle from the projec- tion center with the 3D voxel grid; 4. Find the resulting flow jet voxel 3D shape as an intersection of the all solid angles. More detailed description of the pipeline for a flow jet 3D shape reconstruction is presented as Algorithm 1. The results of preliminary flow jets 3D reconstruction demonstrate the reasonable quality. With increasing the number of registering cameras and system accurate cal- ibrating, the quality of the developed 3D reconstruction technique allows to receive quantitative data needed for flow motion analysis. 5 Conclusion The developed image-based system for accurate 3D flow acquisition and 3D analy- sis utilizes a set of calibrated high-speed cameras for synchronized flow motion reg- istration. A specially developed original calibration technique allows to carry out 3D measurement in hydrodynamic tunnel with relative accuracy at the level of 1/10000 of working space. Stereolithography technology is used for producing accurate an aircraft 3D model with interior channels, that serve for colored jets injecting needed for the visualization and further 3D reconstruction of flow jet shape. The results of preliminary flow jets 3D reconstruction using the Shape from Silhouette technique demonstrate the quality, that corresponds the requirements of the icing process study. Image-based System for 3D Visualization of Flow in Hydrodynamic Tunnel 9 Algorithm 1: Flow jet 3D shape reconstruction Input: exterior orientations of acquisition cameras {Pkeo }, array of K current images of flow {Ick } came from acquisition cameras {Ck } Output: voxel shape of flow jet SF j for current image array {Ick } 1 Split working space into voxel grid V of Nx × Ny × Nz volume elements (voxels) vi ; 2 for each camera C k of acquisition system do 3 Detect borders of the flow jet F j in image Ick ; 4 Determine projecting ray rb for every point of the border b in the image of the flow jet F j ; 5 Determine solid angle ΩF j of flow jet F j for camera Ck as array of rb ; 6 for each voxel vi of voxel grid V do 7 if vi ∈ ΩF j then 8 save vi in flow jet voxel array VFkj ; 9 else 10 skip; 11 end 12 end 13 store the flow jet voxel array VFkj 14 end T k 15 Find shape of flow jet as intersection of flow jet voxel arrays: SF j = VF j ; k Acknowledgements The reported study was funded by Russian Foundation for Basic Research (RFBR) according to the research project 19-29-13040. References 1. Cao, Y., Tan, W., Wu, Z.: Aircraft icing: An ongoing threat to aviation safety. Aerospace Science and Technology 75, 353 – 385 (2018), http://www.sciencedirect.com/science/article/ pii/S1270963817317601 2. Lee, S., Broeren, A.P., Kreeger, R.E., Potapczuk, M.G., Utt, L.: Implementation and Valida- tion of 3-D Ice Accretion Measurement Methodology. https://arc.aiaa.org/doi/abs/10.2514/ 6.2014-2613 3. Broeren, A.P., Addy, H.E., Lee, S., Monastero, M.C.: Validation of 3-D Ice Accretion Mea- surement Methodology for Experimental Aerodynamic Simulation. https://arc.aiaa.org/doi/ abs/10.2514/6.2014-2614 4. Gong, X., Bansmer, S.: 3-d ice shape measurements using mid-infrared laser scanning. Opt. Express 23(4), 4908–4926 (Feb 2015), http://www.opticsexpress.org/abstract.cfm?URI= oe-23-4-4908 5. Shortis, M.: Calibration techniques for accurate measurements by underwater camera sys- tems. Sensors 15(12), 30810–30826 (2015), https://www.mdpi.com/1424-8220/15/12/29831 10 V. Knyaz et al. 6. Bräuer-Burchardt, C., Kühmstedt, P., Notni, G.: Combination of air- and water-calibration for a fringe projection based underwater 3d-scanner. In: Azzopardi, G., Petkov, N. (eds.) Computer Analysis of Images and Patterns. pp. 49–60. Springer International Publishing, Cham (2015) 7. Telem, G., Filin, S.: Photogrammetric modeling of underwater environments. ISPRS Journal of Photogrammetry and Remote Sensing 65(5), 433 – 444 (2010), http://www.sciencedirect. com/science/article/pii/S0924271610000444 8. Truscott, T., Belden, J., Nielson, J., Daily, D., Thomson, S.: Determining 3d flow fields via multi-camera light field imaging. Journal of Visualized Experiments 73(e4325), 1–9 (2013) 9. Menna, F., Nocerino, E., Fassi, F., Remondino, F.: Geometric and optic characterization of a hemispherical dome port for underwater photogrammetry. Sensors 16(1) (2016), https: //www.mdpi.com/1424-8220/16/1/48 10. Raffel, M., Willert, C.E., Scarano, F., Kähler, C.J., Wereley, S.T., Kompenhans, J.: Stereo- scopic PIV, pp. 285–307. Springer International Publishing, Cham (2018), https://doi.org/10. 1007/978-3-319-68852-7{ }8 11. Knyaz, V.A., Stepaniants, D.G., Ippolitov, E.V., Novikov, M.M.: Optical 3d measurements in hydrodynamic tunnel for aircraft icing study. ISPRS - International Archives of the Pho- togrammetry, Remote Sensing and Spatial Information Sciences XLIII-B2-2020, 963–968 (2020), https://www.int-arch-photogramm-remote-sens-spatial-inf-sci.net/XLIII-B2-2020/ 963/2020/ 12. Lasinger, K., Vogel, C., Schindler, K.: Volumetric flow estimation for incompressible fluids using the stationary stokes equations. In: 2017 IEEE International Conference on Computer Vision (ICCV). pp. 2584–2592 (Oct 2017) 13. Schanz, D., Gesemann, S., Schröder, A.: Shake-the-box: Lagrangian particle tracking at high particle image densities. Experiments in Fluids 57(5), 70 (2016), https://doi.org/10.1007/ s00348-016-2157-1 14. Lasinger, K., Vogel, C., Pock, T., Schindler, K.: 3d fluid flow estimation with integrated particle reconstruction. International Journal of Computer Vision 128(4), 1012–1027 (2020), https://doi.org/10.1007/s11263-019-01261-6 15. Rubbert, A., Schröder, W.: Iterative particle matching for three-dimensional particle- tracking velocimetrys. Experiments in Fluids 61(2), 58 (2020), https://doi.org/10.1007/ s00348-020-2891-2 16. Barbu, I., Herzet, C., Mémin, E.: Joint Estimation of Volume and Velocity in TomoPIV. In: 10TH INTERNATIONAL SYMPOSIUM ON PARTICLE IMAGE VELOCIMETRY - PIV13. p. 45. Delft, Netherlands (Jul 2013), https://hal.archives-ouvertes.fr/hal-00880712 17. Huang, Q., Wang, H., Koltun, V.: Single-view reconstruction via joint analysis of image and shape collections. ACM Trans. Graph. 34(4) (Jul 2015), https://doi.org/10.1145/2766890 18. Roth, S., Richter, S.R.: Matryoshka networks: Predicting 3d geometry via nested shape lay- ers. In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 1936– 1944 (June 2018) 19. Kniaz, V.V., Remondino, F., Knyaz, V.A.: Generative adversarial networks for single photo 3d reconstruction. ISPRS - International Archives of the Photogrammetry, Re- mote Sensing and Spatial Information Sciences XLII-2/W9, 403–408 (2019), https://www. int-arch-photogramm-remote-sens-spatial-inf-sci.net/XLII-2-W9/403/2019/ 20. Knyaz, V.: Machine learning for scene 3d reconstruction using a single image. Proc. SPIE 11353, Optics, Photonics and Digital Technologies for Imaging Applications VI 11353, 1135321 (2020), https://doi.org/10.1117/12.2556122 21. Knyaz, V.A.: Scalable photogrammetric motion capture system ”mosca”: Develop- ment and application. ISPRS - International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences XL-5/W6, 43–49 (May 2015), https://www. int-arch-photogramm-remote-sens-spatial-inf-sci.net/XL-5-W6/43/2015/ Image-based System for 3D Visualization of Flow in Hydrodynamic Tunnel 11 22. Knyaz, V.A., Zheltov, S.Y., Vishnyakov, B.V.: Robust object tracking techniques for vision- based 3D motion analysis applications. In: Schelkens, P., Ebrahimi, T., Cristóbal, G., Truchetet, F., Saarikko, P. (eds.) Optics, Photonics and Digital Technologies for Imaging Applications IV. vol. 9896, pp. 334 – 342. International Society for Optics and Photonics, SPIE (2016), https://doi.org/10.1117/12.2227879 23. Jiang Yu Zheng: Acquiring 3-d models from sequences of contours. IEEE Transactions on Pattern Analysis and Machine Intelligence 16(2), 163–178 (Feb 1994) 24. Matusik, W., Buehler, C., McMillan, L.: Polyhedral visual hulls for real-time rendering. In: Gortler, S.J., Myszkowski, K. (eds.) Rendering Techniques 2001. pp. 115–125. Springer Vi- enna, Vienna (2001) 25. Di, X., Dahyot, R., Prasad, M.: Deep shape from a low number of silhouettes. In: Hua, G., Jégou, H. (eds.) Computer Vision – ECCV 2016 Workshops. pp. 251–265. Springer International Publishing, Cham (2016)