Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Further simplify the data layers and extend the MemoryDataLayer #995

Merged
merged 19 commits into from
Sep 3, 2014
Merged

Further simplify the data layers and extend the MemoryDataLayer #995

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
fea9f0c
Create base data layer and base prefetching data layer
kloudkl Aug 28, 2014
ee65a97
Extract common data layer functionalities out of the DataLayer
kloudkl Aug 28, 2014
6833dc0
Remove duplicate codes from the ImageDataLayer
kloudkl Aug 28, 2014
b794cf9
Simplify the WindowDataLayer using the base class
kloudkl Aug 28, 2014
5af0d24
The BasePrefetchingDataLayer shouldn't join the thread
kloudkl Aug 28, 2014
4f7c9b4
Implement Forward_gpu in the base prefetching data layer
kloudkl Aug 28, 2014
4c35ad2
Add transformer to the memory data layer
kloudkl Aug 28, 2014
156a5a2
Remove pthread which has been replaced with boost thread
kloudkl Aug 29, 2014
3c9a13c
Move transform param one level up in the proto to reduce redundancy
kloudkl Aug 29, 2014
05ade81
Test adding images w/o resizing to the memory data layer
kloudkl Aug 29, 2014
20c5992
Move the rest duplicate codes of the data layers into their base class
kloudkl Aug 29, 2014
09cfe1c
Fix conflict between nvcc and boost for cmake
kloudkl Aug 30, 2014
74fa879
Add lint rule for caffe data layer setup
kloudkl Aug 31, 2014
725e98d
Remove OpenCV stuffs from the memory data layer and io utils
kloudkl Sep 2, 2014
ab1f9b5
Add leveldb header back to util/io.cpp
kloudkl Sep 2, 2014
25ce7f5
Place InternalThreadEntry lower in the {,Image,Window}DataLayer.cpp
kloudkl Sep 2, 2014
4761072
Add and transform Datum vector in the MemeoryDataLayer
kloudkl Sep 2, 2014
858ad41
Correct the datum size checking conditions of the data layers
kloudkl Sep 3, 2014
a08f111
Initialize the transformer rng in the base data layer
kloudkl Sep 3, 2014
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 6 additions & 6 deletions examples/cifar10/cifar10_full_train_test.prototxt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,9 +7,9 @@ layers {
data_param {
source: "examples/cifar10/cifar10_train_leveldb"
batch_size: 100
transform_param {
mean_file: "examples/cifar10/mean.binaryproto"
}
}
transform_param {
mean_file: "examples/cifar10/mean.binaryproto"
}
include: { phase: TRAIN }
}
Expand All @@ -21,9 +21,9 @@ layers {
data_param {
source: "examples/cifar10/cifar10_test_leveldb"
batch_size: 100
transform_param {
mean_file: "examples/cifar10/mean.binaryproto"
}
}
transform_param {
mean_file: "examples/cifar10/mean.binaryproto"
}
include: { phase: TEST }
}
Expand Down
12 changes: 6 additions & 6 deletions examples/cifar10/cifar10_quick_train_test.prototxt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,9 +7,9 @@ layers {
data_param {
source: "examples/cifar10/cifar10_train_leveldb"
batch_size: 100
transform_param {
mean_file: "examples/cifar10/mean.binaryproto"
}
}
transform_param {
mean_file: "examples/cifar10/mean.binaryproto"
}
include: { phase: TRAIN }
}
Expand All @@ -21,9 +21,9 @@ layers {
data_param {
source: "examples/cifar10/cifar10_test_leveldb"
batch_size: 100
transform_param {
mean_file: "examples/cifar10/mean.binaryproto"
}
}
transform_param {
mean_file: "examples/cifar10/mean.binaryproto"
}
include: { phase: TEST }
}
Expand Down
10 changes: 5 additions & 5 deletions examples/feature_extraction/imagenet_val.prototxt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,11 +9,11 @@ layers {
batch_size: 50
new_height: 256
new_width: 256
transform_param {
crop_size: 227
mean_file: "$CAFFE_DIR/data/ilsvrc12/imagenet_mean.binaryproto"
mirror: false
}
}
transform_param {
crop_size: 227
mean_file: "$CAFFE_DIR/data/ilsvrc12/imagenet_mean.binaryproto"
mirror: false
}
}
layers {
Expand Down
20 changes: 10 additions & 10 deletions examples/imagenet/alexnet_train_val.prototxt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,11 +7,11 @@ layers {
data_param {
source: "examples/imagenet/ilsvrc12_train_leveldb"
batch_size: 256
transform_param {
crop_size: 227
mean_file: "data/ilsvrc12/imagenet_mean.binaryproto"
mirror: true
}
}
transform_param {
crop_size: 227
mean_file: "data/ilsvrc12/imagenet_mean.binaryproto"
mirror: true
}
include: { phase: TRAIN }
}
Expand All @@ -23,11 +23,11 @@ layers {
data_param {
source: "examples/imagenet/ilsvrc12_val_leveldb"
batch_size: 50
transform_param {
crop_size: 227
mean_file: "data/ilsvrc12/imagenet_mean.binaryproto"
mirror: false
}
}
transform_param {
crop_size: 227
mean_file: "data/ilsvrc12/imagenet_mean.binaryproto"
mirror: false
}
include: { phase: TEST }
}
Expand Down
20 changes: 10 additions & 10 deletions examples/imagenet/imagenet_train_val.prototxt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,11 +7,11 @@ layers {
data_param {
source: "examples/imagenet/ilsvrc12_train_leveldb"
batch_size: 256
transform_param {
crop_size: 227
mean_file: "data/ilsvrc12/imagenet_mean.binaryproto"
mirror: true
}
}
transform_param {
crop_size: 227
mean_file: "data/ilsvrc12/imagenet_mean.binaryproto"
mirror: true
}
include: { phase: TRAIN }
}
Expand All @@ -23,11 +23,11 @@ layers {
data_param {
source: "examples/imagenet/ilsvrc12_val_leveldb"
batch_size: 50
transform_param {
crop_size: 227
mean_file: "data/ilsvrc12/imagenet_mean.binaryproto"
mirror: false
}
}
transform_param {
crop_size: 227
mean_file: "data/ilsvrc12/imagenet_mean.binaryproto"
mirror: false
}
include: { phase: TEST }
}
Expand Down
18 changes: 9 additions & 9 deletions examples/mnist/lenet_consolidated_solver.prototxt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -53,9 +53,9 @@ net_param {
source: "examples/mnist/mnist_train_lmdb"
backend: LMDB
batch_size: 64
transform_param {
scale: 0.00390625
}
}
transform_param {
scale: 0.00390625
}
include: { phase: TRAIN }
}
Expand All @@ -68,9 +68,9 @@ net_param {
source: "examples/mnist/mnist_test_lmdb"
backend: LMDB
batch_size: 100
transform_param {
scale: 0.00390625
}
}
transform_param {
scale: 0.00390625
}
include: {
phase: TEST
Expand All @@ -86,9 +86,9 @@ net_param {
source: "examples/mnist/mnist_train_lmdb"
backend: LMDB
batch_size: 100
transform_param {
scale: 0.00390625
}
}
transform_param {
scale: 0.00390625
}
include: {
phase: TEST
Expand Down
12 changes: 6 additions & 6 deletions examples/mnist/lenet_train_test.prototxt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,9 +8,9 @@ layers {
source: "examples/mnist/mnist_train_lmdb"
backend: LMDB
batch_size: 64
transform_param {
scale: 0.00390625
}
}
transform_param {
scale: 0.00390625
}
include: { phase: TRAIN }
}
Expand All @@ -23,9 +23,9 @@ layers {
source: "examples/mnist/mnist_test_lmdb"
backend: LMDB
batch_size: 100
transform_param {
scale: 0.00390625
}
}
transform_param {
scale: 0.00390625
}
include: { phase: TEST }
}
Expand Down
12 changes: 6 additions & 6 deletions examples/mnist/mnist_autoencoder.prototxt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,9 +7,9 @@ layers {
source: "examples/mnist/mnist_train_lmdb"
backend: LMDB
batch_size: 100
transform_param {
scale: 0.0039215684
}
}
transform_param {
scale: 0.0039215684
}
include: { phase: TRAIN }
}
Expand All @@ -21,9 +21,9 @@ layers {
source: "examples/mnist/mnist_train_lmdb"
backend: LMDB
batch_size: 100
transform_param {
scale: 0.0039215684
}
}
transform_param {
scale: 0.0039215684
}
include: {
phase: TEST
Expand Down
6 changes: 6 additions & 0 deletions include/caffe/common.hpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,9 +6,14 @@
#include <glog/logging.h>

#include <cmath>
#include <fstream> // NOLINT(readability/streams)
#include <iostream> // NOLINT(readability/streams)
#include <map>
#include <set>
#include <sstream>
#include <string>
#include <utility> // pair
#include <vector>

#include "caffe/util/device_alternate.hpp"

Expand Down Expand Up @@ -88,6 +93,7 @@ using std::ostringstream;
using std::pair;
using std::set;
using std::string;
using std::stringstream;
using std::vector;

// A global initialization function that you should call in your main function.
Expand Down
Loading