In this extended abstract we present our early work on structure from motion data compression for image geocoding. We address the advantages of image geocoding over standard trillateration solutions and identify desired characteristics for an image geocoding system such as accuracy, speed and scalability. We hypothesize that scalability impacts both speed and accuracy and should be further researched. Hence, in this thesis we would like to know which is the minimal representation of structure from motion to efficiently compute the location and orientation of new photographs.
The recent explosion of images on social media has lead Computer Vision researchers to an increased interest on image processing algorithms. From those, structure from motion (SFM) has gained an increased relevance due to its applications. With a couple of photographs from the same scene, this algorithm is able to retrieve the 3D structure (point clouds) by processing the motion from photograph to photograph as stereo vision. Providing the correct focal lengths and distortion parameters, this algorithm can achieve an impressing precision on building point clouds. Moreover, new photographs can be added at any time, allowing the ability to update models to this ever changing world.
The application of SFM can vary from simple visualization purposes to more complex tasks such 3D modeling and geographical location recognition. On the later, photographs with unknown location are compared with a geocoded database to compute their GPS coordinates.
Early work on image geocoding started with methods based on a database of
geocoded photographs [
Since two-view geometry is often computationally expensive as it usually relies
in RANSAC based methods to compute the relative pose of images, there was an
increased interest on using structure from motion to support the geocoding process.
Image features are extracted from new photographs, but are directly compared to 3D
point clouds. Rather than performing two-view geometry, it is computed a projection
matrix which validates 2D (query photograph) to 3D points (database point clouds).
Techniques such as nearest-neighbor feature matching [
The advantages of image geocoding are clear when compared to other geocoding systems. First of all image geocoding does not rely on a trilateration process, which means it can compute coordinates on indoor environments as long as it has a WiFi connection to issue the geocoding request. Also, image geocoding has access to the heading of the queried photographs allowing the calculus of the direction in which the photographs were taken, in a single query. This reinforce the utility of image geocoding on supporting guiding systems. Lastly, the only (and ideal) device required for image geocoding is a smartphone, since it contains both camera and WiFi connection. As they are now omnipresent in our society, the cost for deploying an image geocoding solution is greatly decreased.
Extended Abstract on: Minimal SfM Representation for Image Geocoding
In order to image geocoding replace standard trilateration solutions, three characteristics are desired: accuracy, speed and scalability. Starting from accuracy, a good pose estimation is attained when there is related data within the database to the queried photograph. Feature matching is used to ascertain which data to use when computing the pose. Assuming that we are facing the best case scenario and the focal length and distortion parameters are known, then an impressive precision can be achieved with a single photograph.
Speed is defined by how fast can we find the correct database data to geocode the queried photograph. Image processing requires expensive matrix operations as every image pixel is relevant to compute visual features. Additionally, high resolution images deliver better image features but also increase the computational time on extracting and matching those. However, the constant evolution of hardware and the parallelization of image processing algorithms is progressively breaking the barrier of real time processing.
Being scalable means that neither speed and accuracy are hindered with the growth of the geocoded database, which is not quite the case. Assuming that each SFM model contains millions of points, and each point is related to at least two image descriptors and associated 2D data, a massive amount of visual data is required to support an image geocoding system. Consequently, querying the geocoded database gets slower. Besides, more information means having an higher amount of similar features, which may confuse the image geocoding system into geocoding photographs miles way from their correct location.
Facing the scalability problem described in the previous section, in this thesis we question which is the minimal scene representation of structure from motion to allow a good geocoding rate.
Our main objective will be study and benchmark different state of art SFM based geocoding systems, to enhance existing SFM compression strategies or to develop alternative compression methods. We want compression rates able to maintain the geocoding speed and rate, while allowing the scalability of the geocoding system to wider areas. Also, rather than delivering a perfect 100% geocoding rate, we are only interested in avoiding hindering this rate due to aggressive compression.
We are aware that there is state of art research concerning SFM compression
[