Code repository for deep learning urban visual characteristics analysis
The goal of this research project is to evaluate how well cities/districts can be characterized by their visual properties using the readily available city imaginary data set provided by Google street view.
In order to find out the image features that would affect a deep learning algorithm to distinguish between different cities, we trained several image categorization neural networks using AlexNet and VGG-S CNN structures, and analyzed the visual clues that helps the algorithm make categorization decisions. We found out that the algorithm precisely grasps the differences between different city types, and has almost 92% accuracy* on test set for the best model.
As an exploration of latent urban characteristics that can be reflected in the visual aspect, the research proposes an alternative paradigm of computational urban morphological research where environmental affordances revealed in streetlevel urban imagery are taken into consideration.
About 160,000 images were requested using Googole Streetview API (data requesting code here). Samples of the dataset can be found in the "data" folder. 2/3 of the images were used for training, 1/6 for validating, and 1/6 for testing.
The project is built within MatConvNet framework. To run the demo, please
-
Clone the repository.
-
Follow this guide to install MatConvNet.
-
Download the trained model from here
-
Run
matconvnet/examples/run_city_demo.mto predict the the city of a given image (make sure you change the paths of the trained model as well as the image to point to the model you just downloaded and the image you want to test).
This project is a work in progress and you can find the latest updates in visual feature extraction and analysis mainly in the scripts/ folder.
(* Please refer to the paper for details and caveats, etc.)