Land use maps are very important references for the urban planning and management. However, it is difficult and time-consuming to get high-resolution urban land use maps. In this study, we propose a new method to derive land use information at building block level based on machine learning and geo-tagged street-level imagery - Google Street View images. Several commonly used generic image features (GIST, HoG, and SIFT-Fisher) are used to represent street-level images of different cityscapes in a case study area of New York City. Machine learning is further used to categorize different images based on the calculated image features of different street-level images. Accuracy assessment results show that the method developed in this study is a promising method for land use mapping at building block level in future.
Land use maps are very important references for urban planning and other urban practices
in cities
Different from the overhead view of remotely sensed imagery, Google Street View
(GSV) images capture the profile view of streetscapes. GSV images have already been used
to studying human perception of physical environment on ground
In the past decade, the advancement in the computer vision makes it possible to categorize and semantically classify images. In this study, we propose to bring scene classification algorithms in computer vision community to derive land use information of building blocks in cities based on geo-tagged GSV images. Multiple commonly used image features are calculated for representation of street-level images, which capture façades of different types of building blocks. Support vector machine classifier is then trained based on the calculated image features and ground truth land use labels and applied to predict land use types of different building blocks.
A small case study area in Brooklyn, New York City is chosen in this study. The study area includes various land use types, which is very suitable for testing the method using streetlevel images for land use mapping. Figure 1 shows the location and land use map of the study area.
Google Street View panoramas are distributed discretely along the streets. In general, about every 12 meters has one GSV panorama along the street. Therefore, in this study we first create sample sites along streets every 5 meters using ArcGIS 10.2 in order to collect all available GSV panoramas along streets. We then retrieval the GSV panorama ID, coordinates information using Google Maps JavaScript API by inputting the coordinates of those sample sites. In this way, we collect the metadata (panorama ID and coordinate of panorama) of all available GSV panoramas along streets in the study area. Figure 2(a) shows the discrepancy of the distributions of created sample sites and location of GSV panoramas along streets in a small area of study area.
Based on the panorama ID information, we can download GSV images for different heading angles using Google Street View static Image API. Figure 2(b) shows four static GSV images of one site with panorama ID “on66Bt1B37qRlYVxiC7J9g” at different heading angles. By specifying appropriate fov and heading parameters, the GSV images can capture the façades of building blocks along streets, which makes it possible to differentiate different types of building blocks based on their different appearances. For each building block along a street, the closest GSV panorama is chosen. Based on the geometrical model between the location of GSV site (Gx, Gy) and the footprint of building block (see Figure 2(c)), we calculated the field of view (fov) angle by equation (1): fov arctan(
V1 V2 || V1 || || V2 || ) (1) The vectors V1 and V2 are,
V1 = (x1 - Gx, y1 - Gy),
V2 = (x2 - Gx, y2 - Gy) Where (x1, y1) and (x2, y2) are the coordinates of two endpoints of a building façade. In order to decrease distortion in the static GSV images, the fov cannot be too large, therefore, for those building blocks with fov larger than 90, the fov is set to 90. In addition, the minimum fov is set to 30 empirically, although some building blocks may have their fov less than 30. This is because if the fov is too small, the GSV image may not capture the spatial pattern of building block. The heading angle heading is set to the angle between heading direction and the true north direction, and ranges from 0 to 360. Figure 3 shows several collected GSV images with their corresponding land use types in the study area.
The image features developed in computer vision community make it possible to represent
and categorize street-level images of different cityscapes. Image features, which are
calculated based on the texture and geometrical information of images, are insensitive to the
variance of spectral information or the illumination conditions. In this study, several
commonly used generic image features are used to indicate the characteristics of different
street-level images. The image features used in this study include GIST, HoG, and
SIFTFisher. Table 1 summarizes the descriptions of these image features. These features have
already been tested on scene classification
i 0,i 1,2,..., N where W is the support vector, ξi are slack variables introduced to account for the nonseparability of data, N is the number of training samples, constant C represents a penalty parameter that allows to control the penalty assigned to errors.
The trained SVM classifier is then applied to the testing images and compared with ground truth land use of these images in testing set to cross-validate the classification results.
We collect 1048 static GSV images with different land use types in the study area. We randomly split these images into training set and test set 10 times to cross-validate the proposed method. Table 2 summarizes the cross-validation results using different image features. The SIFT-Fisher feature outperforms other two image features in the classification of residential building and non-residential building. The overall accuracy of the residential building vs non-residential building classification result is 91.82% using SIFT-Fisher image feature. The GIST and HoG features get lower classification results, with accuracy of 83.88% and 60.34% respectively.
The classification accuracy of one-two family residential building vs multi-family residential building has lower accuracy compared with the classification result of the residential building vs non-residential building. This is not difficult to understand, since the appearance difference between the one-two family residential buildings and multi-family buildings is not as obvious as the difference between the residential buildings and nonresidential buildings. The selected three image features have similar performances in the classification of one-two family residential building vs multi-family residential building. SIFT-Fisher outperforms other two image features, with overall accuracy of 74%.
This study brings scene classification algorithms in computer vision community to geospatial information retrieval based on publicly accessible data on the web. Different with previous studies using overhead view dataset for urban land use mapping, we first use streetlevel images, which capture the profile view of cityscapes, for land use classification at building block level. Accuracy assessment results show that using the combination of scene classification algorithms and street-level image is a very promising method for urban land use mapping. While this study demonstrates the feasibility of using GSV images for building block level land use information retrieval, there are still some limitations that need to be solved in the future studies. The basic idea of this study is to differentiate different types of land use types based on the different physical appearances of different types of buildings. However, the definition of different land use types is not based on the physical appearances of buildings, but the social functions of buildings. Therefore, in future studies, more attention need to be paid on how to make the semantic classification system to be applicable in real urban planning practices and theoretically recognizable at same time. Future work would focus on choosing better image features and combinations of image features to classify more land use types and get more accurate land use classification results. Human reasoning and new kinds of data should also been considered to get better classification results in future studies.