Proceedings of the VIII International Conference "Distributed Computing and Grid-technologies in Science and Education" (GRID 2018), Dubna, Moscow region, Russia, September 10 - 14, 2018 TEXTURE GENERATION FOR ARCHAEOLOGICAL RECONSTRUCTIONS O.O. Iakushkin a, D.M. Malevanniy, A.B. Degtyarev, D.A. Selivanov, A.I. Fatkina Saint Petersburg State University, 7/9 Universitetskaya nab., St. Petersburg, 199034, Russia E-mail: a o.yakushkin@spbu.ru The paper describes a solution that reconstructs the texture in 3D models of archeological monuments and performs their visualization. The software we have developed allows to model the outward surface of objects that exhibit various degrees of destruction. The drawings and photographs of preserved wall fragments and stonework elements are used in the modelling process. Our work resulted in the development of a texturing system that reconstructs textures of a given object based on photographs and fragments of drawings. The major distinguishing feature of the system is that it can reconstruct textures using limited and low-quality input data. Specifically, the input data fed to the system may consist of photographs of an object taken with an ordinary camera (e.g., with a smartphone). We used OpenCV, CGAL and AwesomeBump open source computer vision packages to develop the system. Keywords: archeological, reconstruction, material, texture © 2018 Oleg O. Iakushkin, Daniil M. Malevanniy, Alexander B. Degtyarev, Dmitrii A. Selivanov, Anna I. Fatkina 462 Proceedings of the VIII International Conference "Distributed Computing and Grid-technologies in Science and Education" (GRID 2018), Dubna, Moscow region, Russia, September 10 - 14, 2018 1. Introduction This paper describes an algorithm used to reconstruct objects that have a uniform surface structure. We reduced the number of steps required for reconstruction by replacing a full scan of an object’s surface with reconstruction of a texture fragment. Our algorithm can create a texture fragment from a single photo of an object. It can work with photos of both individual stones and masonry. 2. Approach The algorithm has two main parts: 1. It uses a source image to isolate a square segment that will serve as as the basis for image reconstruction. 2. It creates the normal, relief, shading and reflection maps that altogether serve as an input for the PBR rendering. 2.1 The base texture generation The material can be built from a square part of a given photo or source image that does not contain foreign objects. The size of the patch directly affects the quality of the final result. Thus, the first part of the algorithm splits the image into the foreground and the background in order to identify the maximum square part that belongs to the foreground. The foreground refers to the part of the image consisting solely of the object on which the material is based. The SLIC algorithm [3] is used for the initial segmentation of the source image: it divides the image into a large number of uniformly distributed segments—i.e., superpixels whose borders closely match the borders of objects in the image. The segmentation is further simplified by constructing a weighted graph of image regions adjacency where the weight of each edge corresponds to the proximity of the average colours of adjacent regions. This way the task is reduced to the division of a section of the graph into two parts—each of the parts corresponds either to the foreground, or the background (as shown in Figure 1 and Figure 2). Figure 1. Photo segmentation using the SLIC method 463 Proceedings of the VIII International Conference "Distributed Computing and Grid-technologies in Science and Education" (GRID 2018), Dubna, Moscow region, Russia, September 10 - 14, 2018 Figure 2. The graph of adjacency regions superimposed on the photo. The colour of the face corresponds to its weight The vertex corresponding to the centre of the photo is considered to belong reliably to the foreground, and all the vertices corresponding to the outer border of the photo are considered to be on the background. We iterate along the vertices adjacent to the foreground to separate the foreground from the background by calculating the average weight difference between each given superpixel and the foreground. The border of the foreground obtained by this method is then additionally filtered (Figure 3): the algorithm deletes all the foreground and background sections that do not border on the main array of vertices of the respective type with at least two edges. Figure 3. The foreground/background splitting of the image before and after filtering the mask After obtaining a graph cut, a binary mask is constructed, where the true values correspond to the foreground. Then the maximum square is inscribed into the obtained foreground mask. If it is impossible to build a section of the graph, a square with an edge equal to the smaller side of the photo is built around its centre. 2.2 PBR Material Reconstruction The PBR materials used in realistic rendering of three-dimensional scenes consist of textures that describe various properties of the surface and its interaction with light. The photo-generated material consists of five images: 1. The scatter map (Diffuse map) that describes the visible colour of the surface. 2. The normal map that describes the surface relief in form of the normal values encoded as an RGB image. It is calculated as the value of the Sobel operator for the diffusion map. 464 Proceedings of the VIII International Conference "Distributed Computing and Grid-technologies in Science and Education" (GRID 2018), Dubna, Moscow region, Russia, September 10 - 14, 2018 3. The height map of the surface that describes the deviation of the surface point from the average level. The height map is created based on the normal map using the method described in [4]. 4. The ambient occlusion map that describes shadows created by small surface details. This map is created using a variation of the well-known SSAO technique [5] that utilizes the height and the normal maps. 5. The specular map that describes the intensity of the light reflected by a surface. This map is calculated as Gaussian difference for the previously constructed scatter map. The examples of the original image and PBR material texture maps generated from it are shown in Figure 5. When building a photorealistic texture, it is necessary to take into account that it should be uniform across the entire area of an object. To achieve seamlessness, the dispersion map is built based on the square image obtained in the first part of the algorithm, according to the scheme shown in Figure 6. Figure 5. The results of the algorithm. The basic fragment of a photo (top left); then the scattering, normal, irregularities, shading and reflection maps (in sequence from left to right) Figure 6. Generation of a seamless texture 5. Acknowledgement This research was partially supported by the Russian Foundation for Basic Research grants (projects no. 17-29-04288, 16-07-01113). The authors would like to acknowledge the reviewers for the valuable recommendations that helped improve this paper. 465 Proceedings of the VIII International Conference "Distributed Computing and Grid-technologies in Science and Education" (GRID 2018), Dubna, Moscow region, Russia, September 10 - 14, 2018 6. Conclusion The described algorithm allows to reconstruct a surface material form a single photo of a scanned object that has a uniform surface. The algorithm is ready for use in the PBR render and heavily relies on the homogeneity of the source object texture. The material may be built using a photo of a stone with a noisy background or a photo of a masonry wall—the logic of image processing in the two instances remains the same. References [1] Iakushkin, O., Fatkina, A., Plaksin, V., Sedova, O., Degtyarev, A.,Uteshev, A. / Reconstruction of stone walls in form of polygonal meshes from archaeological studies. Lecture Notes in Computer Science, Springer, 2018, Vol. 10963, pp. 136-148. [2] Oleg Iakushkin, Dmitrii Selivanov, Liliia Tazieva, Anna Fatkina, Valery Grishkin, Alexei Uteshev 3D Reconstruction of Landscape Models and Archaeological Objects Based on Photo and Video Materials // Lecture Notes in Computer Science. Springer, 2018, Vol. 10963. pp. 160-169. [3] Stutz, D., Hermans, A., & Leibe, B. Superpixels: an evaluation of the state-of-the-art // Computer Vision and Image Understanding, Vol. 166, pp. 1-27. [4] K.Kolasinski: AwesomeBump v1.0 // url:http://awesomebump.besaba.com/wp- content/uploads/2015/01/ABoverwiev.pdf [5] Louis Bavoil, Miguel Sainz, Screen Space Ambient Occlusion, September 2008 466