0411@2018.06.25:

shenyunhang · Jun 25, 2018 · cb9f185 · cb9f185
1 parent 142cc2a
commit cb9f185
Show file tree

Hide file tree

Showing 74 changed files with 1,914 additions and 7,423 deletions.
diff --git a/.gitmodules b/.gitmodules
@@ -4,8 +4,8 @@
 	branch = fast-rcnn
 [submodule "caffe-wsl"]
 	path = caffe-wsl
-	url = https://shenyunhang@bitbucket.org/shenyunhang/caffe-wsl.git
+	url = https://github.com/shenyunhang/caffe-wsl.git
 	branch = wsl
 [submodule "caffe-fwsl"]
 	path = caffe-fwsl
-	url = https://shenyunhang@bitbucket.org/shenyunhang/ssd.git
+	url = https://github.com/shenyunhang/caffe-fwsl.git
diff --git a/LICENSE b/LICENSE
@@ -1,4 +1,4 @@
-Faster R-CNN
+CPG
 
 The MIT License (MIT)
 
@@ -21,61 +21,3 @@ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
 LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
 OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
 THE SOFTWARE.
-
-************************************************************************
-
-THIRD-PARTY SOFTWARE NOTICES AND INFORMATION
-
-This project, Faster R-CNN, incorporates material from the project(s)
-listed below (collectively, "Third Party Code").  Microsoft is not the
-original author of the Third Party Code.  The original copyright notice
-and license under which Microsoft received such Third Party Code are set
-out below. This Third Party Code is licensed to you under their original
-license terms set forth below.  Microsoft reserves all other rights not
-expressly granted, whether by implication, estoppel or otherwise.
-
-1.	Caffe, (https://github.com/BVLC/caffe/)
-
-COPYRIGHT
-
-All contributions by the University of California:
-Copyright (c) 2014, 2015, The Regents of the University of California (Regents)
-All rights reserved.
-
-All other contributions:
-Copyright (c) 2014, 2015, the respective contributors
-All rights reserved.
-
-Caffe uses a shared copyright model: each contributor holds copyright
-over their contributions to Caffe. The project versioning records all
-such contribution and copyright details. If a contributor wants to
-further mark their specific copyright on a particular contribution,
-they should indicate their copyright solely in the commit message of
-the change when it is committed.
-
-The BSD 2-Clause License
-
-Redistribution and use in source and binary forms, with or without
-modification, are permitted provided that the following conditions
-are met:
-
-1. Redistributions of source code must retain the above copyright notice,
-this list of conditions and the following disclaimer.
-
-2. Redistributions in binary form must reproduce the above copyright
-notice, this list of conditions and the following disclaimer in the
-documentation and/or other materials provided with the distribution.
-
-THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
-"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
-LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
-A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
-HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
-SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED
-TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
-PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
-LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
-NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
-SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
-
-************END OF THIRD-PARTY SOFTWARE NOTICES AND INFORMATION**********
diff --git a/README.md b/README.md
@@ -1,50 +1,11 @@
-### Disclaimer
-
-The official Faster R-CNN code (written in MATLAB) is available [here](https://github.com/ShaoqingRen/faster_rcnn).
-If your goal is to reproduce the results in our NIPS 2015 paper, please use the [official code](https://github.com/ShaoqingRen/faster_rcnn).
-
-This repository contains a Python *reimplementation* of the MATLAB code.
-This Python implementation is built on a fork of [Fast R-CNN](https://github.com/rbgirshick/fast-rcnn).
-There are slight differences between the two implementations.
-In particular, this Python port
- - is ~10% slower at test-time, because some operations execute on the CPU in Python layers (e.g., 220ms / image vs. 200ms / image for VGG16)
- - gives similar, but not exactly the same, mAP as the MATLAB version
- - is *not compatible* with models trained using the MATLAB code due to the minor implementation differences
- - **includes approximate joint training** that is 1.5x faster than alternating optimization (for VGG16) -- see these [slides](https://www.dropbox.com/s/xtr4yd4i5e0vw8g/iccv15_tutorial_training_rbg.pdf?dl=0) for more information
-
-# *Faster* R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
-
-By Shaoqing Ren, Kaiming He, Ross Girshick, Jian Sun (Microsoft Research)
-
-This Python implementation contains contributions from Sean Bell (Cornell) written during an MSR internship.
-
-Please see the official [README.md](https://github.com/ShaoqingRen/faster_rcnn/blob/master/README.md) for more details.
-
-Faster R-CNN was initially described in an [arXiv tech report](http://arxiv.org/abs/1506.01497) and was subsequently published in NIPS 2015.
-
-### License
-
-Faster R-CNN is released under the MIT License (refer to the LICENSE file for details).
-
-### Citing Faster R-CNN
-
-If you find Faster R-CNN useful in your research, please consider citing:
-
-    @inproceedings{renNIPS15fasterrcnn,
-        Author = {Shaoqing Ren and Kaiming He and Ross Girshick and Jian Sun},
-        Title = {Faster {R-CNN}: Towards Real-Time Object Detection
-                 with Region Proposal Networks},
-        Booktitle = {Advances in Neural Information Processing Systems ({NIPS})},
-        Year = {2015}
-    }
+# CPG
+Object-Aware Spatial Constraint for Weakly Supervised Detection
 
 ### Contents
 1. [Requirements: software](#requirements-software)
 2. [Requirements: hardware](#requirements-hardware)
-3. [Basic installation](#installation-sufficient-for-the-demo)
-4. [Demo](#demo)
-5. [Beyond the demo: training and testing](#beyond-the-demo-installation-for-training-and-testing-models)
-6. [Usage](#usage)
+3. [Installation](#installation)
+4. [Usage](#usage)
 
 ### Requirements: software
 
@@ -58,44 +19,42 @@ If you find Faster R-CNN useful in your research, please consider citing:
   # Unrelatedly, it's also recommended that you use CUDNN
   USE_CUDNN := 1
   ```
-
-  You can download my [Makefile.config](https://dl.dropboxusercontent.com/s/6joa55k64xo2h68/Makefile.config?dl=0) for reference.
+
 2. Python packages you might not have: `cython`, `python-opencv`, `easydict`
 3. [Optional] MATLAB is required for **official** PASCAL VOC evaluation only. The code now includes unofficial Python evaluation code.
 
 ### Requirements: hardware
 
-1. For training smaller networks (ZF, VGG_CNN_M_1024) a good GPU (e.g., Titan, K20, K40, ...) with at least 3G of memory suffices
-2. For training Fast R-CNN with VGG16, you'll need a K40 (~11G of memory)
-3. For training the end-to-end version of Faster R-CNN with VGG16, 3G of GPU memory is sufficient (using CUDNN)
+1. For training smaller networks (VGG_CNN_F, VGG_CNN_M_1024), a GPU with about 6G of memory suffices.
+2. For training lager networks (VGG16), you'll need a GPU with about 8G of memory.
 
-### Installation (sufficient for the demo)
+### Installation
 
-1. Clone the Faster R-CNN repository
+1. Clone the CPG repository
   ```Shell
   # Make sure to clone with --recursive
-  git clone --recursive https://github.com/rbgirshick/py-faster-rcnn.git
+  git clone --recursive https://github.com/shenyunhang/CPG.git
   ```
 
-2. We'll call the directory that you cloned Faster R-CNN into `FRCN_ROOT`
+2. We'll call the directory that you cloned CPG into `CPG_ROOT`
 
    *Ignore notes 1 and 2 if you followed step 1 above.*
 
-   **Note 1:** If you didn't clone Faster R-CNN with the `--recursive` flag, then you'll need to manually clone the `caffe-fast-rcnn` submodule:
+   **Note 1:** If you didn't clone CPG with the `--recursive` flag, then you'll need to manually clone the `caffe-wsl` submodule:
     ```Shell
     git submodule update --init --recursive
     ```
-    **Note 2:** The `caffe-fast-rcnn` submodule needs to be on the `faster-rcnn` branch (or equivalent detached state). This will happen automatically *if you followed step 1 instructions*.
+    **Note 2:** The `caffe-wsl` submodule needs to be on the `wsl` branch (or equivalent detached state). This will happen automatically *if you followed step 1 instructions*.
 
 3. Build the Cython modules
     ```Shell
-    cd $FRCN_ROOT/lib
+    cd $CPG_ROOT/lib
     make
     ```
 
 4. Build Caffe and pycaffe
     ```Shell
-    cd $FRCN_ROOT/caffe-fast-rcnn
+    cd $CPG_ROOT/caffe-wsl
     # Now follow the Caffe installation instructions here:
     #   http://caffe.berkeleyvision.org/installation.html
 
@@ -104,44 +63,23 @@ If you find Faster R-CNN useful in your research, please consider citing:
     make -j8 && make pycaffe
     ```
 
-5. Download pre-computed Faster R-CNN detectors
-    ```Shell
-    cd $FRCN_ROOT
-    ./data/scripts/fetch_faster_rcnn_models.sh
-    ```
-
-    This will populate the `$FRCN_ROOT/data` folder with `faster_rcnn_models`. See `data/README.md` for details.
-    These models were trained on VOC 2007 trainval.
-
-### Demo
-
-*After successfully completing [basic installation](#installation-sufficient-for-the-demo)*, you'll be ready to run the demo.
-
-To run the demo
-```Shell
-cd $FRCN_ROOT
-./tools/demo.py
-```
-The demo performs detection using a VGG16 network trained for detection on PASCAL VOC 2007.
-
-### Beyond the demo: installation for training and testing models
-1. Download the training, validation, test data and VOCdevkit
+5. Download the training, validation, test data and VOCdevkit
 
 	```Shell
 	wget http://host.robots.ox.ac.uk/pascal/VOC/voc2007/VOCtrainval_06-Nov-2007.tar
 	wget http://host.robots.ox.ac.uk/pascal/VOC/voc2007/VOCtest_06-Nov-2007.tar
 	wget http://host.robots.ox.ac.uk/pascal/VOC/voc2007/VOCdevkit_08-Jun-2007.tar
 	```
 
-2. Extract all of these tars into one directory named `VOCdevkit`
+6. Extract all of these tars into one directory named `VOCdevkit`
 
 	```Shell
 	tar xvf VOCtrainval_06-Nov-2007.tar
 	tar xvf VOCtest_06-Nov-2007.tar
 	tar xvf VOCdevkit_08-Jun-2007.tar
 	```
 
-3. It should have this basic structure
+7. It should have this basic structure
 
 	```Shell
   	$VOCdevkit/                           # development kit
@@ -150,61 +88,55 @@ The demo performs detection using a VGG16 network trained for detection on PASCA
   	# ... and several other directories ...
   	```
 
-4. Create symlinks for the PASCAL VOC dataset
+8. Create symlinks for the PASCAL VOC dataset
 
 	```Shell
-    cd $FRCN_ROOT/data
+    cd $CPG_ROOT/data
     ln -s $VOCdevkit VOCdevkit2007
     ```
     Using symlinks is a good idea because you will likely want to share the same PASCAL dataset installation between multiple projects.
-5. [Optional] follow similar steps to get PASCAL VOC 2010 and 2012
-6. [Optional] If you want to use COCO, please see some notes under `data/README.md`
-7. Follow the next sections to download pre-trained ImageNet models
+9. [Optional] follow similar steps to get PASCAL VOC 2010 and 2012
+10. [Optional] If you want to use COCO, please see some notes under `data/README.md`
+11. Follow the next sections to download pre-trained ImageNet models
+
+### Download object proposals
+1. Selective Search: [original matlab code](http://disi.unitn.it/~uijlings/MyHomepage/index.php#page=projects1), [python wrapper](https://github.com/sergeyk/selective_search_ijcv_with_python)
+2. EdgeBoxes: [matlab code](https://github.com/pdollar/edges)
+3. MCG: [matlab code](http://www.eecs.berkeley.edu/Research/Projects/CS/vision/grouping/mcg/)
+
 
 ### Download pre-trained ImageNet models
 
 Pre-trained ImageNet models can be downloaded for the three networks described in the paper: ZF and VGG16.
 
 ```Shell
-cd $FRCN_ROOT
+cd $CPG_ROOT
 ./data/scripts/fetch_imagenet_models.sh
 ```
-VGG16 comes from the [Caffe Model Zoo](https://github.com/BVLC/caffe/wiki/Model-Zoo), but is provided here for your convenience.
-ZF was trained at MSRA.
 
 ### Usage
 
-To train and test a Faster R-CNN detector using the **alternating optimization** algorithm from our NIPS 2015 paper, use `experiments/scripts/faster_rcnn_alt_opt.sh`.
-Output is written underneath `$FRCN_ROOT/output`.
+To train and test a CPG detector, use `experiments/scripts/cpg.sh`.
+Output is written underneath `$CPG_ROOT/output`.
 
 ```Shell
-cd $FRCN_ROOT
-./experiments/scripts/faster_rcnn_alt_opt.sh [GPU_ID] [NET] [--set ...]
+cd $CPG_ROOT
+./experiments/scripts/cpg.sh [GPU_ID] [NET] [--set ...]
 # GPU_ID is the GPU you want to train on
-# NET in {ZF, VGG_CNN_M_1024, VGG16} is the network arch to use
-# --set ... allows you to specify fast_rcnn.config options, e.g.
+# NET in {VGG_CNN_F, VGG_CNN_M_1024, VGG16} is the network arch to use
+# --set ... allows you to specify configure options, e.g.
 #   --set EXP_DIR seed_rng1701 RNG_SEED 1701
 ```
 
-("alt opt" refers to the alternating optimization training algorithm described in the NIPS paper.)
-
-To train and test a Faster R-CNN detector using the **approximate joint training** method, use `experiments/scripts/faster_rcnn_end2end.sh`.
-Output is written underneath `$FRCN_ROOT/output`.
+Example:
 
 ```Shell
-cd $FRCN_ROOT
-./experiments/scripts/faster_rcnn_end2end.sh [GPU_ID] [NET] [--set ...]
-# GPU_ID is the GPU you want to train on
-# NET in {ZF, VGG_CNN_M_1024, VGG16} is the network arch to use
-# --set ... allows you to specify fast_rcnn.config options, e.g.
-#   --set EXP_DIR seed_rng1701 RNG_SEED 1701
+./experiments/scripts/cpg.sh 0 VGG16 pascal_voc --set EXP_DIR cpg
 ```
 
-This method trains the RPN module jointly with the Fast R-CNN network, rather than alternating between training the two. It results in faster (~ 1.5x speedup) training times and similar detection accuracy. See these [slides](https://www.dropbox.com/s/xtr4yd4i5e0vw8g/iccv15_tutorial_training_rbg.pdf?dl=0) for more details.
+This will reproduction the VGG16 result in paper.
 
-Artifacts generated by the scripts in `tools` are written in this directory.
-
-Trained Fast R-CNN networks are saved under:
+Trained CPG networks are saved under:
 
 ```
 output/<experiment directory>/<dataset name>/
@@ -215,3 +147,26 @@ Test outputs are saved under:
 ```
 output/<experiment directory>/<dataset name>/<network snapshot name>/
 ```
+
+### Other method
+
+WSDDN:
+
+```Shell
+./experiments/scripts/wsddn.sh 0 VGG16 pascal_voc --set EXP_DIR wsddn
+```
+or
+```Shell
+./experiments/scripts/wsddn_x.sh 0 VGG16 pascal_voc --set EXP_DIR wsddn_x
+```
+
+ContextLocNet:
+
+```Shell
+./experiments/scripts/contextlocnet.sh 0 VGG16 pascal_voc --set EXP_DIR contextlocnet
+```
+or
+```Shell
+./experiments/scripts/contextlocnet_x.sh 0 VGG16 pascal_voc --set EXP_DIR contextlocnet_x
+```
+
diff --git a/caffe-wsl b/caffe-wsl
diff --git a/data/.gitignore b/data/.gitignore
@@ -10,3 +10,4 @@ faster_rcnn_models*
 *.mat
 *.caffemodel
 *.xml
+coco*
diff --git a/data/VOC2010/create_data_noise.sh b/data/VOC2010/create_data_noise.sh
+3 −1		include/caffe/layers/center_loss_layer.hpp
+7 −5		src/caffe/layers/center_loss_layer.cpp
+17 −6		src/caffe/layers/center_loss_layer.cu
+6 −6		src/caffe/layers/statistics_layer.cpp
+1 −0		src/caffe/proto/caffe.proto
-Original file line number
+Diff line change
@@ Expand Up / @@ -10,3 +10,4 @@ faster_rcnn_models* @@
     *.mat
     *.caffemodel
     *.xml
+    coco*