The paper describes a solution that reconstructs the texture in 3D models of archeological monuments and performs their visualization. The software we have developed allows to model the outward surface of objects that exhibit various degrees of destruction. The drawings and photographs of preserved wall fragments and stonework elements are used in the modelling process. Our work resulted in the development of a texturing system that reconstructs textures of a given object based on photographs and fragments of drawings. The major distinguishing feature of the system is that it can reconstruct textures using limited and low-quality input data. Specifically, the input data fed to the system may consist of photographs of an object taken with an ordinary camera (e.g., with a smartphone). We used OpenCV, CGAL and AwesomeBump open source computer vision packages to develop the system.
This paper describes an algorithm used to reconstruct objects that have a uniform surface structure. We reduced the number of steps required for reconstruction by replacing a full scan of an object’s surface with reconstruction of a texture fragment. Our algorithm can create a texture fragment from a single photo of an object. It can work with photos of both individual stones and masonry.
The algorithm has two main parts: 1. It uses a source image to isolate a square segment that will serve as as the basis for image reconstruction. 2. It creates the normal, relief, shading and reflection maps that altogether serve as an input for the PBR rendering.
The material can be built from a square part of a given photo or source image that does not contain foreign objects. The size of the patch directly affects the quality of the final result. Thus, the first part of the algorithm splits the image into the foreground and the background in order to identify the maximum square part that belongs to the foreground. The foreground refers to the part of the image consisting solely of the object on which the material is based.
The SLIC algorithm [
The segmentation is further simplified by constructing a weighted graph of image regions adjacency where the weight of each edge corresponds to the proximity of the average colours of adjacent regions. This way the task is reduced to the division of a section of the graph into two parts—each of the parts corresponds either to the foreground, or the background (as shown in Figure 1 and Figure 2).
The vertex corresponding to the centre of the photo is considered to belong reliably to the foreground, and all the vertices corresponding to the outer border of the photo are considered to be on the background. We iterate along the vertices adjacent to the foreground to separate the foreground from the background by calculating the average weight difference between each given superpixel and the foreground.
The border of the foreground obtained by this method is then additionally filtered (Figure 3): the algorithm deletes all the foreground and background sections that do not border on the main array of vertices of the respective type with at least two edges. After obtaining a graph cut, a binary mask is constructed, where the true values correspond to the foreground. Then the maximum square is inscribed into the obtained foreground mask. If it is impossible to build a section of the graph, a square with an edge equal to the smaller side of the photo is built around its centre.
The PBR materials used in realistic rendering of three-dimensional scenes consist of textures
that describe various properties of the surface and its interaction with light. The photo-generated
material consists of five images:
1. The scatter map (Diffuse map) that describes the visible colour of the surface.
2. The normal map that describes the surface relief in form of the normal values encoded
as an RGB image. It is calculated as the value of the Sobel operator for the diffusion
map.
3. The height map of the surface that describes the deviation of the surface point from
the average level. The height map is created based on the normal map using the
method described in [
This map is created using a variation of the well-known SSAO technique [
The examples of the original image and PBR material texture maps generated from it are shown in Figure 5. When building a photorealistic texture, it is necessary to take into account that it should be uniform across the entire area of an object. To achieve seamlessness, the dispersion map is built based on the square image obtained in the first part of the algorithm, according to the scheme shown in Figure 6.
This research was partially supported by the Russian Foundation for Basic Research grants (projects no. 17-29-04288, 16-07-01113). The authors would like to acknowledge the reviewers for the valuable recommendations that helped improve this paper.
The described algorithm allows to reconstruct a surface material form a single photo of a scanned object that has a uniform surface. The algorithm is ready for use in the PBR render and heavily relies on the homogeneity of the source object texture. The material may be built using a photo of a stone with a noisy background or a photo of a masonry wall—the logic of image processing in the two instances remains the same.