This work presents a method of rendering of 360° panoramas based on real-world DEM (Digital Elevation Model) data using graphic processor (GPU). Given geographic coordinates as input this method generates a panorama image and also a distance map with a distance from the point of view to every pixel of the rendered panorama. The details on estimating these distances and the results of accuracy evaluation and error measurements are provided. 3D panorama rendering; GPU rendering; GPU z-buffer; estimation of distances up to objects of relief; digital elevation model; geo-information systems.
КЛЮЧЕВЫЕ СЛОВА
related techniques and approaches aiming at usage of spatial data emerged. The great contribution was made by the NASA's Shuttle Radar Topography Mission (SRTM) giving the public access to highly detailed geospatial data of the whole surface of our planet. Among all the fields of application of digital elevation model (DEM) data [1] it seems rational to highlight the field of comparison between real-life collected data and the model to be able to produce the new, previously non-existent information and conclusions [2,3].
interesting to consider the simultaneous rendering of the panorama together with the depth map. Any modern GPU is equipped with z-buffer, which stores the information about the distance to all the objects of the 3D scene. In order to reconstruct linear depth values from the values obtained from z-buffer one needs to reverse engineer the pipeline [4] which data goes through before it gets into the z-buffer.
Let us start from the basics and follow the pipeline. The OpenGL projection matrix * Proceedings of the XI International scientific-practical conference «Modern information technologies and IT-education» (SITITO’2016), Moscow, Russia, November 25 - 26, 2016 where where (GL_PROJECTION) looks like:
⎝ 0 0 −1 0 ⎠ Multiplied by homogeneous point (xe, ye, ze, we) in eye space it gives us this point in clip space: = − . since one only cares about depth component and taking into account the fact that in eye space we equal to 1 the result is: = ( ) +
.
= −
( ) ⎝ 0 0 −1 0 ⎠ Multiplied by homogeneous point (xe, ye, ze, we) in eye space it gives us this point in clip space: = = ( ) + ( , + ,
Then we recover ze: 0 0 0 0 0 = = = ( (
) ) (
) −1 + + + 0 ⎟⎞.
⎟ , 0
⎛ 0 ⎞ ⎜ ⎟⎟ ⎜
⎜ 0 ⎠ ⎝ ⎞ ⎟⎟, ⎟ ⎠ = ) + ,
, = ( ) +
, = − , since one only cares about depth component and taking into account the fact that in eye space we equal to
= − .
( ) = = ( ) + ( ) + ,
Then we recover ze: =
( = ( ) ) ( ( = ) = = ( + (
) − ) , , = ) = = ( − ) , = ( ( ).
. )
The whole system is implemented in Java programming language. Java gives us the advantage of using the code of the system in Desktop applications, web-services and mobile devices. The rendering of the 3D scenes is performed through the jME3 game engine library with some core classes rewritten in the way that provides the access to the underlying layer of LWJGL library. Access to the underlying layer is needed to be able to directly receive the data from z-buffer of GPU hardware, such low-level functionality is normally not needed to game and conventional 3D software developers.
jME3 itself uses LWJGL (Lightweight Java Game Library). The library accesses native C code through the Java Native Interface (JNI). LWJGL provides bindings to OpenGL. The library acts as a wrapper over the OpenGL native libraries and provides API similar to the OpenGL APIs of other lower lever languages running directly on hardware (as opposed to Java running inside the Java Virtual Machine) such as C\C++. Platformspecific lwjgl.dll file should be accessible to the Java code. Though it’s not a big constraint since libraries for most popular operation systems (Linux/Windows) and platforms (x86/x64/ARM) are available. The first attempt was made to implement the renderer using JOGL OpenGL Java library. Although the 360 degrees panorama rendering was successful, it was not possible to access z-buffer data. It forced us to switch to jME3 and LWJGL libraries. Another advantage of jME3 over the JOGL library is the support of DEM height maps out of the box.
integrated to the pipeline of the comprising project. The code can anyway be started through command line.
takes geographical coordinates representing the centre of the desired area and the desired radius as an input. Since the DEM data is stored in the big amount of files one square degree of latitude\longitude per file the module can sew together terrain data from up to 9 files producing the square terrain.
represented by two different classes: OffscreenRenderer and OnscreenRenderer. OffscreenRenderer provides the 360 degree panorama generation functionality. OnscreenRenderer provides the ability to fly over the 3D landscape in a flight simulator-like way using keyboard and mouse controls. Simple GUI is provided to choose screen resolution and some other rendering parameters. OnscreenRenderer is extremely useful for debug purposes and for detection of anomalies in DEM data.
Based on the coordinates the set of files containing relevant DEM data is determined and read into the memory. Then these data are concatenated in the proper order to form one continuous terrain surface. After this the terrain is positioned and cut to the requested coordinate the new centre of the terrain surface. Next the DEM heightmap (two-dimensional array of heights in meters) is translated into the 3D model (vertices, edges, faces, and polygons) and the altitude of the landscape in the desired coordinate is determined based on the DEM data. Then camera is positioned into the desired coordinate and on the determined latitude.
22.5° (half of 45°). Then the camera is rotated by 45° eight times. After each rotation we make a snapshot of what camera sees and a snapshot of z-buffer content. At the end all the data is sewed up together to form 360° view panorama and distance panorama and this data is returned to the calling function to be used in subsequent processing or just be saved as a file for later use.
EXPERIMENTAL EVALUATION OF THE PROPOSED METHOD
model against the mountain view generator of Dr. Ulrich Deuschle (hereinafter referred to as Udeuschle) which we take as a reference model.
First three datasets were collected by comparing three pairs of panoramas and handpicking the mountain peaks which were clearly visible and distinguishable both on picture generated by developed system and picture generated by Udeuschle generator. After plotting of all three datasets on one plot together with the line of perfect correlation it became obvious that all the observations could be naturally divided into two groups. The first group includes the observations made by the camera located lower than the peak, the second group – observations made by the camera located higher than the peak (Figure 1).
а) б) Figure 1. Correlation of distance prediction results between our model and Udeuschle model for cases when: а) camera is located higher than peaks; б) camera is located lower than peaks а) б) Figure 2. The histogram of the residuals on the evaluation data for cases when: a) camera is located higher than peaks; b) camera is located lower than peaks
As for the residuals, it can be concluded that a residual for an observation in the evaluation data is the difference between the true target and the predicted target. Residuals represent the portion of the target that the model is unable to predict (Figure 2). A positive residual indicates that the model is underestimating the target (the actual target is larger than the predicted target). A negative residual indicates an overestimation (the actual target is smaller than the predicted target). The histogram of the residuals on the evaluation data when distributed in a bell shape and centred at zero indicates that the model makes mistakes in a random manner and does not systematically over or under predict any particular range of target values (the histogram follows the form of the normal distribution).
one has done an evaluation L of the distance to the peak from the panorama. Based on this value you should give the shortest interval of the distances, which will include the real distance with probability 80%. If one knows that the peak is higher than the camera, than one could say that the distance to the mountain is L±O.25L with probability 80%. If one has no idea about the height of the peak, he should provide an interval L±O.3L in order to be 80% sure. Now one could try to improve the accuracy of the predictions for the case when the camera is located lower than the peak by calculation of the systematic error.
The dataset was separated into two parts. One part will be used as training set (80%), the second is as test set (20%). The linear regression will be built to find the systematic shift between real and predicted value of the distance. In other words the best coefficients b and c will be found, such that: dist_real= b * dist_predicted + c + ε, where: dist_real – real distance from the peak to the camera, dist_predicted – distance predicted by our model, ε – error.
performance are tested on the test set. The best b and c are found using cross validation in order to avoid overfitting. The dataset is divided into k folds (in our case k=7), k-1 folds used as a training set, 1 fold used as a test set. Among this k pair of coefficients the one is chosen, which gives the best value of the R squared statistics. The best values are b=0.898891146468 and c=-1015.66880806.
predicted by our model
determination (Rsquared)
So building the linear regression could be a good solution to increase the accuracy of the predictions in cases when it is impossible to install the camera higher than the peaks. But this method could not be used without having some knowledge about real distances to the peak and distances measured by algorithm in order to have enough data to build the regression.
was proposed. The specific technique of computing the distance mask based on the transformed data from z-buffer is presented. The experimental results revealed the correlation between the camera-peak altitude gap and the distance error. The evaluation of the accuracy of distance measurements against the reference model is provided. The tables with applied distance accuracy probabilities are computed and directions on usage are given.
the measured terrain area should be positioned lower than the camera, in other words it should be directly observable by the camera. This logical requirement is stipulated by the method of the estimation which is based on the data obtained from the z-buffer of the GPU. Yet there is still a great field for improvements in two different areas: rendering optimization and accuracy enhancement.
This work was supported by the Ministry of Education and Science of Russian Federation (project 2014/112 №35).