The paper is devoted to The decision of actual problem of development of methodology of photogrammetric survey of the small-sized objects with the purpose of receiving the correct data on them for creation of photorealistic 3D-models without use of the specialized equipment. Structure-from-motion methods applications of small-size objects are considered in the paper. Small-sized objects are justified by the narrow application of 3D models obtained by photogrammetry in marketing. A distinctive feature of the study is the use of images taken by mobile phone cameras as input data for the 3D-reconstruction. The main factors influencing the accuracy of object reconstruction are considered. An analysis of the determination of distortion in mobile phone camera images is carried out. The geometric parameters of different mobile phone cameras are considered in order to determine the deviations in reconstruction accuracy. General recommendations on creation of 3D-models for source data from mobile phone cameras are given. The work has been carried out using the central axis method. And methodical guidelines for obtaining input data for photogrammetry for different object sizes are also given. Quantitative indicators for image series are derived experimentally in order to reduce the time it takes to shoot. camera focal length, mobile phone camera, radial distortion, mobile device, tangential distortion, geometric GraphiCon 2021: 31st International Conference on Computer Graphics and Vision, September 27-30, 2021, Nizhny Novgorod, Russia
In the engineering and 3D reconstruction stages, it is necessary to decide on the methods and software tools for product creation. In this case, we are talking about creating a 3D model for subsequent use in marketing. The area of application of these models requires the objects photorealism, which imposes certain conditions both on the objects themselves and on the methods used in their creation.
The emergence of a whole class of digital devices (digital cameras, tomographs, 3D-scanners) that
allow to obtain digital twins of material objects has greatly simplified the creation and representation
of object models of any complexity: from a small coin to a multi-storey building. Digital scanning is
potentially a new paradigm that extends the possibilities of designing three-dimensional objects,
shifting the design process from working with projective geometry to working directly with points of
an object in three-dimensional space [
One reasonably accessible technology for non-contact scanning of objects is photogrammetry [
2021 Copyright for this paper by its authors.
Digital photogrammetry is having an increasing impact due to its simplicity and low cost, as well as
its combination of advanced image processing techniques. It produces results of the same quality and
resolution as traditional 3D modelling and 3D scanning methods. On the other hand, the use of this
technology to reconstruct small objects with centimetre, millimetre and submillimetre details, is more
complex and hardly mentioned in the literature [
Photogrammetry is more developed in architecture [
The methodology being developed considers alternative methods and tools for creating photorealistic 3D models. The methodology includes recommendations for the process of photogrammetric survey based on object parameters and survey equipment used, recommendations for lighting and other characteristics of the scene.
Small-size objects (size not less than 3x3x3 cm and not more than 30x30x30 cm) are used in this method, since such objects are most often used for advertising applications.
The basic principle of all photogrammetric measurements is the geometric-mathematical reconstruction of the ray paths from the object to the sensor at the moment of exposure. Therefore, the most fundamental element is the knowledge of the geometric characteristics of the individual photograph [15].
The development of triangulation and image extraction algorithms has improved the quality of photogrammetric reconstruction methods. Some methods that have emerged in the last 20 years are computer vision stereo reconstruction, structure from shadow, structure from focus, structure from texture, structure from motion, structured light, light and range detection (LIDAR) and time-lapse cameras. Each of these methods obtains range data using different technologies. Each of the listed methods has advantages and disadvantages.
For the investigation, we decided to focus on a method that allows us to have both low cost and high accuracy of results.
In particular, methods that require a single camera, simple lighting and that enable close-range imaging were considered. Simplicity in implementation was also a criterion, as it was necessary to ensure that the technique to be developed was accessible to the masses. Structure from motion [16] was the method that was decided to use for the final qualification work.
The main factors affecting the accuracy of 3D reconstructions include: photo resolution; camera calibration; angles; quality of photo orientation; photo redundancy; targets/precision of markings.
The combination of these factors affects the accuracy of the reconstruction in different ways. Different examples of the correlation of these factors are shown in Fig. 1.
The main advantage of consumer-grade digital cameras is their convenience, portability and low cost. These cameras are not specifically designed for photogrammetric measurements due to their unstable geometry. Measurement errors can be partially eliminated by calibration.
Camera calibration is the process required to extract accurate and reliable 3D information from images.
The use of mobile phone in reconstruction tasks is reflected in [17, 18, 19, 20, 21]. These studies focus on the application of SfM based automatic image orientation techniques. Although these methods provide automation and ease of use for users in data processing, they do not provide accurate and reliable final results as the georeferencing process and spatial data creation are missing.
Packages that support SfM allow lens calibration and orientation processes to be incorporated into the computational process of 3D reconstruction, without the need for additional devices and software, opening up a broader application of photogrammetry for non-specialist users. But sometimes this approach will not meet the requirements for high metric quality. Therefore, it is necessary to determine the feasibility of a comprehensive sensor calibration before reconstruction.
Research on the geometric accuracy of high-resolution smartphone camera images and the ability to reconstruct three-dimensional objects is underrepresented in the literature.
Let us perform geometric accuracy testing for four cameras of mobile devices using a 3D reference field and compare the results.
These days, the market for smartphones and other mobile devices offers the user a huge range of brands and models. In order to investigate the geometric accuracy of images from such devices, several leading models were selected. Table 1 lists the camera specifications.
To assess the geometric accuracy of the images, a calibration table, a flat image displayed on a personal computer screen, representing a chessboard image (Figure 2), was used. The size and number of cells can be changed. We have chosen a scale of 6x4 black squares, size ~ 30x30 mm.
Twenty photos were taken of each of the devices, from approximately the same angles, from a distance of ~50 cm. The format of all the images was .JPG.
The camera calibration toolbar for Matlab [22]. were used to estimate tangential and radial distortion [23].
Radial distortion causes straight lines to appear curved. Radial distortion increases as points are moved away from the centre of the image. Radial distortion can be described as follows [23]: (1) (2) (3) (4) (5) (6)
Similarly, tangential distortion arises because the image capture lens is not aligned perfectly parallel to the image plane. Thus, some areas of the image may appear closer than expected. The amount of tangential distortion can be represented as follows [23]:
That is, five parameters, known as distortion coefficients, must be found: _ = ( 1 2 1 2 3) two parameters of decentralised distortion. where 1, 2, 3 are the first three parameters of radially symmetrical distortion and 1, 2 are the first
Also, to estimate camera distortion, internal camera parameters are needed: focal length and optical centres , . The focal length and optical centres can be used to create a camera matrix, which can be used to remove distortion caused by a particular camera lens. A camera matrix is unique to a particular device. Therefore, once calculated, it can be reapplied to other images taken with the same camera. This is expressed as a 3x3 matrix: = (1 + 1 2 + 2 4 + 3 6) = (1 + 1 2 + 2 4 + 3 6) = + [2 1 = + [2 2 + 2( 2 + 2 2)] + 1( 2 + 2 2)] _ = [ 0 0 0 0 ] 1
The focal length projections and are close to the focal length because they are not performed on the image plane, but on the ZX and ZY planes of the local coordinate system of the image. This refinement is important for determining the focal length value.
Also the internal characteristics of the camera include the conditions for differential scaling and nonorthogonality between the 1, 2 axes [24]. a three-dimensional point into a coordinate system.
The external parameters correspond to rotation and shift vectors, which translate the coordinates of Coordinates of image centre for each camera and their deviations are given in Table 2.
The highest values of the optical centre position were obtained for the Apple IPhone 8 and Sony Xperia E5, as confirmed by the distortion plots.
The evaluation of the geometric model leads to the conclusion that mobile devices can be used in photogrammetric measurements requiring high accuracy. Appropriate image configuration, calibration and performance of the software to process this data, together with the SfM approach, suggest that this class of imaging device can be used in low-budget digitisation, documentation, etc. projects.
The mobile phone camera continues to be a versatile means of taking pictures, with high technical specifications and at the same time affordable for the mass user. Technological advances affect the quality and performance of mobile devices, which means that the quality of images captured by a smartphone camera will also improve. Thus, it will be possible to obtain high quality results for 3D modelling applications that use these devices separately as tools for collecting and processing photogrammetric data, at least for smaller projects.
The relative accuracy of mobile device cameras is estimated relative to the size of the object using a standard deviation of point coordinates. This deviation is equal to one . For example, "1 part in 100" means that if an object has a maximum size of about 3 m, the accuracy will be 3 m/100 or 3 cm. That is, the position of the points will be accurate to 0.3 mm "1 in 5,000" means 3 m/5000 or 0.6 mm. The accuracy values of "1 part in 30,000" in this table are one sigma standard deviation accuracy. At 1 part in 30,000 on a 3 m object, the position of the points will be accurate to 0.1 mm with a probability of 68 % (one sigma). These values are approximate.
For absolute accuracy, the design must be scaled and/or have reference points. The accuracy of these scales and reference points then affects the absolute accuracy. The higher the accuracy (absolute correspondence to standard or true values) of the scale or reference data, the higher the absolute accuracy (with constant relative accuracy).
The geometric evaluation of the accuracy of the images leads to a number of recommendations for the characteristics of the imager (Table 3).
The general algorithm for creating a 3D model and then applying it is shown in Fig. 3. The algorithm consists of four blocks of modules: 1. Initial images Acquisition - object shooting; 2. Image Processing - Structure-from-Motion; 3. Editing Resulting 3D-model; 4. 3D model Processing and Storage.
The raw imagery phase affects the time and quality characteristics of the 3D reconstruction. Therefore, it is necessary to select the optimum imaging method that satisfies the SfM conditions. The centre-axis method was chosen as such a tool. It consists in using a conditionally static camera (stationary on the horizontal axis and moving on the vertical axis) and a rotating surface, on which the object is mounted (Fig. 4).
The focal length directly affects the angle of view of the camera. The higher the angle of view, the more overlap the camera provides, the higher the percentage of subject reconstruction.
the smaller the area covered and the finer the detail
sensor determines the resolution (image detail).
The offset pitch of the camera in the vertical axis is determined according to the size of the object. The main prerequisite for this is that the triple overlap zone is maintained.
The survey can be performed in several steps, depending on the characteristics of the object. For each object, the survey trajectory will be calculated according to its size, presence/absence of symmetry and fine details. Thus, for each object the shooting trajectory will be individual. Depending on the size of the object, the number of bursts per capture block will be calculated.
Table 4 calculates the number of photos for a single shooting block at the lowest possible camera settings of the mobile device. The values of the convergence angle (table angle) and the number of shots per route will be constant.
The number of blocks, or shooting paths, is determined by the configuration of the object, depending on the number of faces that can serve as the base of the model (upon which the object will have a stable position for the duration of the photo capture).
The pitch of the camera along the vertical axis will not always be uniform. For digital cameras the object will be in the camera's field of view with a camera pitch of 30°, which corresponds to only three or four camera positions in the 0-1m range. But because mobile cameras have a much shorter focal length, the cameras will have to be "positioned" more often.
Thus, depending on the type of object and its characteristics as well as the camera configuration, a camera path is formed.
Using several examples of objects, we will make a survey and subsequent 3D reconstruction of the objects.
In the first two cases, ceramic figures with dimensions up to 10 cm on all sides will be considered. As these objects have only one facet-base, mono-block photography is performed for them. Shooting was carried out with the change of position of the camera in relation to the object fixed at the moment of shooting.
For the first figure (fig. 5a), a photograph was taken around the object from four heights. A total of 98 photographs were taken. The size of this figure is 6x5x6 cm. The 3D model shown in the figure contains 263,613 polygons. It has a build time of 1 hour 11 minutes.
For the second figure (fig. 5b) the number of heights was 6, which is explained by the size of the object - 5,5x5,5x9 cm. Number of frames is 157. The 3D model shown in the figure contains 618,089 polygons. It has a build time of 2 hour 53 seconds.
For each of the figures, the number of frames taken in a circle at the same height was 24. As optimally calculated to create a triple overlapping area. The rotation of the figures was 15°.
a This number of images was sufficient to reconstruct a high-polygon 3D model.
Increasing the number of photos would have increased the processing time of the images, but not the quality of the model. All the fine details of the figures were reflected in the resulting 3D models, eliminating the redundancy of the number of photos for the 3D reconstruction.
An object with glare surfaces and many fine details has also been considered (fig. 6). The camera shown in the figure has several faces that can serve as a base. Therefore, multi-block photography was carried out for it. A total of 392 photographs were taken, in three different positions of the subject.
Number of polygons is 1 239 083. It has a build time of 3 hour 53 seconds.
You can see that the number of shooting bursts increases as the overall size increases.
The number of both routes and shots per route will be affected by the size of the subject. As the overall size increases, the number of series of shots increases.
To minimise the number of shots in a series, it is important to consider the symmetry of the object. Full symmetry of the object avoids duplication of shooting routes, which reduces the number of shots and significantly reduces processing time.
It is important to take into account the presence of small details on the object. Such features should be captured in separate, close-up shots that do not fit into the main survey route.
In the design and redesign stages of the marketing, film and games industry, VR/AR, architectural heritage and 3D printing industries, this research can be used to demonstrate the concept of manufactured products with high fidelity and realism. And also 3D content can be used to demonstrate and re-engineer cultural heritage objects.
[15] J. S. Aber, I. Marzolff, J. Ries, Small-format aerial photography: Principles, techniques and geoscience applications, Elsevier, 2010. [16] M.J. Westoby, J. Brasington, N.F. Glasser, M.J.Hambrey, J.M. Reynolds, Structure-from-Motion’ photogrammetry: A low-cost, effective tool for geoscience applications, Geomorphology, 2012, pp. 300-314. [17] D. Akca, A. Gruen, Comparative geometric and radiometric evaluation of mobile phone and still video cameras, The Photogrammetric Record 24(127) (2009) 217-245. [18] N. A. Azhar, A. Ahmad, Comparative geometric and radiometric evaluation of mobile phone, compact and DSLR cameras, in: Proceedings of the 2013 IEEE 9th International Colloquium on Signal Processing and its Applications, Kuala Lumpur, Malaysia, March 2013. [19] P. Tanskanen, K. Kolev, L. Meier, F. Camposeco, O. Saurer, M. Pollefeys, Live metric 3D reconstruction on mobile phones. In ICCV, 2013. [20] K. Kolev, P. Tanskanen, P. Speciale, and M. Pollefeys. Turning mobile phones into 3D scanners.
In CVPR, 2014. [21] O. Muratov, Y. Slynko, V. Chernov, M. Lyubimtseva, A. Shamsuarov, V. Bucha, 3DCapture: 3D Reconstruction for a Smartphone: In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, Las Vegas, NV, USA, 2016, pp. 75–82. [22] Camera calibration toolbar for Matlab. URL: http://www.vision.caltech.edu/bouguetj/calib_doc/. [23] OpenCV. URL: https://docs.opencv.org/master/dc/dbb/tutorial_py_calibration.html. [24] C.S. Fraser, K.L. Edmundson, Design and implementation of a computational processing system for off-line digital close-range photogrammetry, ISPRS Journal of Photogrammetry and Remote Sensing 55(2) (2000) 94-104. [25] B. A DeWitt, P.R. Wolf, Elements of Photogrammetry (with Applications in GIS), 2000. [26] D.C. Brown, Close-range camera calibration, Photogrammetric Engineering, Engineering and Remote Sensing 37(8) (1971) 855-866.