CNN Feature Extractor

A Matlab class implementing CNN Feature Extractor based on Caffe, as originally presented by Ross Girschick et al. [1], and adapted for various computer vision tasks [2].

The code is inspired by the original R-CNN implementation [3], but is limited to only the feature extractor part. Thus, it offers a flexible CNN feature extraction pipeline that can be used to extract responses from desired network's layer (e.g., FC7 for regular feature extraction, soft-max for final predictions, etc.). Responses can be obtained either from whole image or one or more specified regions of interest (e.g., bounding boxes from region proposals).

While superseded by faster and improved approaches (Fast R-CNN, Faster R-CNN, etc.), having a minimal flexible CNN feature extraction pipeline may still prove useful, especially in application of CNN features in various specialized application domains.

Installation

The project depends on Caffe; for the sake of self-containment, the appropriate version is bundled as a git submodule. Due to Caffe dependency, only linux is officially supported right now.

The Caffe should be compiled and installed via CMake; the project expects to find it installed in external/caffe-bin folder. For convenience, it includes a build script that compiles and deploys Caffe. The script was written and used on Fedora linux, so it may or may not work on other distributions - if you have problems, try to manually execute its steps.

Note that because MatCaffe bindings are dynamically linked against libcaffe.so, the latter needs to be in LD_LIBRARY_PATH before Matlab is started.

Dependencies

Make sure that Caffe dependencies are installed; the list of packages for Fedora is (assuming Negativo17's repository is used for CUDA):

sudo dnf install cmake gcc-c++ boost-devel glog-devel gflags-devel \
    protobuf-devel hdf5-devel lmdb-devel leveldb-devel snappy-devel \
    openblas-devel python-devel python2-numpy
sudo dnf install  cuda-devel cudnn-devel cuda-gcc-c++

and for Ubuntu:

sudo apt-get install cmake build-essential libboost-all-dev libgflags-dev \
    libgoogle-glog-dev libprotobuf-dev protobuf-compiler libhdf5-dev \
    liblmdb-dev libleveldb-dev libsnappy-dev libopenblas-dev python-dev \
    python-numpy
sudo apt-get install nvidia-cuda-toolkit

Build commands

# Checkout the source
git clone https://github.com/rokm/cnn-feature-extractor.git
cd cnn-feature-extractor
git submodule update --init --recursive

# Build and deploy dependencies
export MATLABDIR=/usr/local/MATLAB/R2016b
./build_all.sh

# Start Matlab
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/path/to/project/external/caffe-bin/lib64
${MATLABDIR}/bin/matlab

References

1. Girshick, Ross, et al. "Rich feature hierarchies for accurate object detection and semantic segmentation." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2014.
2. Sharif Razavian, Ali, et al. "CNN features off-the-shelf: an astounding baseline for recognition." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. 2014.
3. https://github.com/rbgirshick/rcnn