Caffe Successful but not Caffe-distri

[andytraumueller]

Hi,

i have the following maven error:

[exec] g++ src/main/cpp/jni/JniMatVector.cpp -MMD -MP -pthread -fPIC -DNDEBUG -O2 -I/usr/include/python2.7 -I/usr/lib/python2.7/dist-packages/numpy/core/include -I/usr/local/include -I/usr/include/hdf5/serial/ -I/usr/lib/jvm/default-java/include -I/usr/lib/jvm/java-7-oracle/include -I.build_release/src -I./include -I../caffe-public/distribute/include -I../caffe-public/src -I/usr/local/cuda/include -I/usr/lib/jvm/java-7-oracle/include/linux -Wall -Wno-sign-compare -c -o .build_release/src/main/cpp/jni/JniMatVector.o 2> .build_release/src/main/cpp/jni/JniMatVector.o.warnings.txt
[exec] || (cat .build_release/src/main/cpp/jni/JniMatVector.o.warnings.txt; exit 1)
[exec] CXX src/main/cpp/jni/JniFloatDataTransformer.cpp
[exec] g++ src/main/cpp/jni/JniFloatDataTransformer.cpp -MMD -MP -pthread -fPIC -DNDEBUG -O2 -I/usr/include/python2.7 -I/usr/lib/python2.7/dist-packages/numpy/core/include -I/usr/local/include -I/usr/include/hdf5/serial/ -I/usr/lib/jvm/default-java/include -I/usr/lib/jvm/java-7-oracle/include -I.build_release/src -I./include -I../caffe-public/distribute/include -I../caffe-public/src -I/usr/local/cuda/include -I/usr/lib/jvm/java-7-oracle/include/linux -Wall -Wno-sign-compare -c -o .build_release/src/main/cpp/jni/JniFloatDataTransformer.o 2> .build_release/src/main/cpp/jni/JniFloatDataTransformer.o.warnings.txt
[exec] || (cat .build_release/src/main/cpp/jni/JniFloatDataTransformer.o.warnings.txt; exit 1)
[exec] make[1]: *** [.build_release/src/main/cpp/jni/JniFloatDataTransformer.o] Error 1
[exec] src/main/cpp/jni/JniFloatDataTransformer.cpp: In function ‘void Java_com_yahoo_ml_jcaffe_FloatDataTransformer_transform(JNIEnv_, jobject, jobject, jobject)’:
[exec] src/main/cpp/jni/JniFloatDataTransformer.cpp:69:52: error: no matching function for call to ‘caffe::DataTransformer::Transform(std::vectorcv::Mat&, caffe::Blob&)’
[exec] xformer->Transform((* mat_vector_ptr), blob_ptr);
[exec] ^
[exec] src/main/cpp/jni/JniFloatDataTransformer.cpp:69:52: note: candidates are:
[exec] In file included from src/main/cpp/jni/JniFloatDataTransformer.cpp:8:0:
[exec] ../caffe-public/distribute/include/caffe/data_transformer.hpp:38:8: note: void caffe::DataTransformer::Transform(const caffe::Datum&, caffe::Blob) [with Dtype = float]
[exec] void Transform(const Datum& datum, Blob_ transformed_blob);
[exec] ^
[exec] ../caffe-public/distribute/include/caffe/data_transformer.hpp:38:8: note: no known conversion for argument 1 from ‘std::vectorcv::Mat’ to ‘const caffe::Datum&’
[exec] ../caffe-public/distribute/include/caffe/data_transformer.hpp:50:8: note: void caffe::DataTransformer::Transform(const std::vectorcaffe::Datum&, caffe::Blob) [with Dtype = float]
[exec] void Transform(const vector & datum_vector,
[exec] ^
[exec] ../caffe-public/distribute/include/caffe/data_transformer.hpp:50:8: note: no known conversion for argument 1 from ‘std::vectorcv::Mat’ to ‘const std::vectorcaffe::Datum&’
[exec] ../caffe-public/distribute/include/caffe/data_transformer.hpp:91:8: note: void caffe::DataTransformer::Transform(caffe::Blob, caffe::Blob) [with Dtype = float]
[exec] void Transform(Blob input_blob, Blob* transformed_blob);
[exec] ^
[exec] ../caffe-public/distribute/include/caffe/data_transformer.hpp:91:8: note: no known conversion for argument 1 from ‘std::vectorcv::Mat’ to ‘caffe::Blob’
[exec] ../caffe-public/distribute/include/caffe/data_transformer.hpp:141:8: note: void caffe::DataTransformer::Transform(const caffe::Datum&, Dtype) [with Dtype = float]
[exec] void Transform(const Datum& datum, Dtype* transformed_data);
[exec] ^
[exec] ../caffe-public/distribute/include/caffe/data_transformer.hpp:141:8: note: no known conversion for argument 1 from ‘std::vectorcv::Mat’ to ‘const caffe::Datum&’
[exec] Makefile:413: recipe for target '.build_release/src/main/cpp/jni/JniFloatDataTransformer.o' failed
[exec] make[1]: Leaving directory '/opt/CaffeOnSpark/caffe-distri'
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] caffe ............................................. SUCCESS [0.001s]
[INFO] caffe-distri ...................................... FAILURE [53.341s]
[INFO] caffe-grid ........................................ SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 53.811s
[INFO] Finished at: Mon May 09 17:21:45 CEST 2016
[INFO] Final Memory: 16M/340M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-antrun-plugin:1.7:run (make) on project caffe-distri: An Ant BuildException has occured: exec returned: 2
[ERROR] around Ant part ...... @ 5:83 in /opt/CaffeOnSpark/caffe-distri/target/antrun/build-make.xml
[ERROR] -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR] mvn -rf :caffe-distri
Makefile:9: recipe for target 'build' failed
make: *** [build] Error 1

I am running Java version 1.7.0_80-b15, scala version 2.11.8, maven 3.0.5

[anfeng]

Please make sure that you don't disable OPENCV.
caffe-public/Makefile.config should have the following one commented out.

USE_OPENCV := 0

If you look at caffe-public/distribute/include/caffe/data_transformer.hpp,
you will see the following signature for Transformer:

#ifdef USE_OPENCV
...

/**

• @brief Applies the transformation defined in the data layer's
• transform_param block to a cv::Mat
  *
• @param cv_img
• cv::Mat containing the data to be transformed.
• @param transformed_blob
• This is destination blob. It can be part of top blob's data if
• set_cpu_data() is used. See image_data_layer.cpp for an example.
  /
  *void Transform(const cv::Mat& cv_img, Blob transformed_blob);*
  #endif // USE_OPENCV

Andy

On Mon, May 9, 2016 at 8:38 AM, TheSwaine notifications@github.com wrote:

Hi,

i have the following maven error:

[exec] g++ src/main/cpp/jni/JniMatVector.cpp -MMD -MP -pthread -fPIC
-DNDEBUG -O2 -I/usr/include/python2.7
-I/usr/lib/python2.7/dist-packages/numpy/core/include -I/usr/local/include
-I/usr/include/hdf5/serial/ -I/usr/lib/jvm/default-java/include
-I/usr/lib/jvm/java-7-oracle/include -I.build_release/src -I./include
-I../caffe-public/distribute/include -I../caffe-public/src
-I/usr/local/cuda/include -I/usr/lib/jvm/java-7-oracle/include/linux -Wall
-Wno-sign-compare -c -o .build_release/src/main/cpp/jni/JniMatVector.o 2>
.build_release/src/main/cpp/jni/JniMatVector.o.warnings.txt
[exec] || (cat
.build_release/src/main/cpp/jni/JniMatVector.o.warnings.txt; exit 1)
[exec] CXX src/main/cpp/jni/JniFloatDataTransformer.cpp
[exec] g++ src/main/cpp/jni/JniFloatDataTransformer.cpp -MMD -MP -pthread
-fPIC -DNDEBUG -O2 -I/usr/include/python2.7
-I/usr/lib/python2.7/dist-packages/numpy/core/include -I/usr/local/include
-I/usr/include/hdf5/serial/ -I/usr/lib/jvm/default-java/include
-I/usr/lib/jvm/java-7-oracle/include -I.build_release/src -I./include
-I../caffe-public/distribute/include -I../caffe-public/src
-I/usr/local/cuda/include -I/usr/lib/jvm/java-7-oracle/include/linux -Wall
-Wno-sign-compare -c -o
.build_release/src/main/cpp/jni/JniFloatDataTransformer.o 2>
.build_release/src/main/cpp/jni/JniFloatDataTransformer.o.warnings.txt
[exec] || (cat
.build_release/src/main/cpp/jni/JniFloatDataTransformer.o.warnings.txt;
exit 1)
[exec] make[1]: ***
[.build_release/src/main/cpp/jni/JniFloatDataTransformer.o] Error 1
[exec] src/main/cpp/jni/JniFloatDataTransformer.cpp: In function ‘void
Java_com_yahoo_ml_jcaffe_FloatDataTransformer_transform(JNIEnv
, jobject, jobject, jobject)’: [exec]
src/main/cpp/jni/JniFloatDataTransformer.cpp:69:52: error: no matching
function for call to
‘caffe::DataTransformer::Transform(std::vectorcv::Mat&, caffe::Blob&)’
[exec] xformer->Transform((* mat_vector_ptr), blob_ptr);
[exec] ^
[exec] src/main/cpp/jni/JniFloatDataTransformer.cpp:69:52: note:
candidates are:
[exec] In file included from
src/main/cpp/jni/JniFloatDataTransformer.cpp:8:0:
[exec] ../caffe-public/distribute/include/caffe/data_transformer.hpp:38:8:
note: void caffe::DataTransformer::Transform(const caffe::Datum&,
caffe::Blob
) [with Dtype = float] [exec] void Transform(const Datum& datum, Blob
transformed_blob);
[exec] ^
[exec] ../caffe-public/distribute/include/caffe/data_transformer.hpp:38:8:
note: no known conversion for argument 1 from ‘std::vectorcv::Mat’ to
‘const caffe::Datum&’
[exec] ../caffe-public/distribute/include/caffe/data_transformer.hpp:50:8:
note: void caffe::DataTransformer::Transform(const
std::vectorcaffe::Datum&, caffe::Blob

) [with Dtype = float] [exec] void Transform(const vector & datum_vector,
[exec] ^ [exec]
../caffe-public/distribute/include/caffe/data_transformer.hpp:50:8: note:
no known conversion for argument 1 from ‘std::vectorcv::Mat’ to ‘const
std::vectorcaffe::Datum&’ [exec]
../caffe-public/distribute/include/caffe/data_transformer.hpp:91:8: note:
void caffe::DataTransformer::Transform(caffe::Blob, caffe::Blob
) [with Dtype = float] [exec] void Transform(Blob input_blob, Blob*
transformed_blob);
[exec] ^
[exec] ../caffe-public/distribute/include/caffe/data_transformer.hpp:91:8:
note: no known conversion for argument 1 from ‘std::vectorcv::Mat’ to
‘caffe::Blob
’ [exec]
../caffe-public/distribute/include/caffe/data_transformer.hpp:141:8: note:
void caffe::DataTransformer::Transform(const caffe::Datum&, Dtype) [with
Dtype = float]
[exec] void Transform(const Datum& datum, Dtype* transformed_data);
[exec] ^
[exec]
../caffe-public/distribute/include/caffe/data_transformer.hpp:141:8: note:
no known conversion for argument 1 from ‘std::vectorcv::Mat’ to ‘const
caffe::Datum&’
[exec] Makefile:413: recipe for target
'.build_release/src/main/cpp/jni/JniFloatDataTransformer.o' failed
[exec] make[1]: Leaving directory '/opt/CaffeOnSpark/caffe-distri'

[INFO]

[INFO] Reactor Summary:
[INFO]
[INFO] caffe ............................................. SUCCESS [0.001s]
[INFO] caffe-distri ...................................... FAILURE
[53.341s]
[INFO] caffe-grid ........................................ SKIPPED

[INFO]

[INFO] BUILD FAILURE

[INFO]

[INFO] Total time: 53.811s
[INFO] Finished at: Mon May 09 17:21:45 CEST 2016
[INFO] Final Memory: 16M/340M

[INFO]

[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-antrun-plugin:1.7:run (make) on project
caffe-distri: An Ant BuildException has occured: exec returned: 2
[ERROR] around Ant part ...... @ 5:83 in
/opt/CaffeOnSpark/caffe-distri/target/antrun/build-make.xml
[ERROR] -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the
-e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions,
please read the following articles:
[ERROR] [Help 1]
http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the
command
[ERROR] mvn -rf :caffe-distri
Makefile:9: recipe for target 'build' failed
make: *** [build] Error 1

I am running Java version 1.7.0_80-b15, scala version 2.11.8, maven 3.0.5

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub
#65

[andytraumueller]

Hi and Thanks for the quick answer, but it still fails.
I am using OpenCV3.1 and it runs smoothly with just caffe.

exec] g++ -shared -o .build_release/lib/libcaffedistri.so .build_release/src/main/cpp/common.o .build_release/src/main/cpp/CaffeNet.o .build_release/src/main/cpp/util/socket_sync.o .build_release/src/main/cpp/util/blocking_queue_extra.o .build_release/src/main/cpp/util/MemoryInputAdapter.o .build_release/src/main/cpp/util/socket_sync_cpu.o .build_release/src/main/cpp/util/socket.o .build_release/src/main/cpp/util/InputAdapter.o .build_release/src/main/cpp/util/parallel_cpu.o .build_release/src/main/cpp/util/rdma.o .build_release/src/main/cpp/util/mini_cluster.o .build_release/src/main/cpp/util/rdma_sync.o .build_release/src/main/cpp/jni/JniMatVector.o .build_release/src/main/cpp/jni/JniFloatDataTransformer.o .build_release/src/main/cpp/jni/JniMat.o .build_release/src/main/cpp/jni/JniFloatArray.o .build_release/src/main/cpp/jni/JniCaffeNet.o .build_release/src/main/cpp/jni/JniFloatBlob.o -pthread -fPIC -DNDEBUG -O2 -DUSE_OPENCV -I/usr/include/python2.7 -I/usr/lib/python2.7/dist-packages/numpy/core/include -I/usr/local/include -I/usr/include/hdf5/serial/ -I/usr/lib/jvm/default-java/include -I/usr/lib/jvm/java-7-oracle/include -I.build_release/src -I./include -I../caffe-public/distribute/include -I../caffe-public/src -I/usr/local/cuda/include -I/usr/lib/jvm/java-7-oracle/include/linux -Wall -Wno-sign-compare -L/usr/lib -L/usr/local/lib -L/usr/lib -L/usr/lib/x86_64-linux-gnu/hdf5/serial/ -L../caffe-public/distribute/lib -L/usr/local/cuda/lib64 -L/usr/local/cuda/lib -L.build_release/lib -lcudart -lcublas -lcurand -lglog -lgflags -lprotobuf -lboost_system -lm -lcaffe -lhdf5_hl -lhdf5 -lopencv_core -lopencv_highgui -lopencv_imgproc -lopencv_imgcodecs -lboost_thread -lstdc++ -lcblas -latlas
[exec] CXX/LD -o .build_release/src/main/cpp/tools/caffe_mini_cluster.bin
[exec] g++ .build_release/src/main/cpp/tools/caffe_mini_cluster.o -o .build_release/src/main/cpp/tools/caffe_mini_cluster.bin -pthread -fPIC -DNDEBUG -O2 -DUSE_OPENCV -I/usr/include/python2.7 -I/usr/lib/python2.7/dist-packages/numpy/core/include -I/usr/local/include -I/usr/include/hdf5/serial/ -I/usr/lib/jvm/default-java/include -I/usr/lib/jvm/java-7-oracle/include -I.build_release/src -I./include -I../caffe-public/distribute/include -I../caffe-public/src -I/usr/local/cuda/include -I/usr/lib/jvm/java-7-oracle/include/linux -Wall -Wno-sign-compare -lcaffedistri -L/usr/lib -L/usr/local/lib -L/usr/lib -L/usr/lib/x86_64-linux-gnu/hdf5/serial/ -L../caffe-public/distribute/lib -L/usr/local/cuda/lib64 -L/usr/local/cuda/lib -L.build_release/lib -lcudart -lcublas -lcurand -lglog -lgflags -lprotobuf -lboost_system -lm -lcaffe -lhdf5_hl -lhdf5 -lopencv_core -lopencv_highgui -lopencv_imgproc -lopencv_imgcodecs -lboost_thread -lstdc++ -lcblas -latlas
[exec] -Wl,-rpath,$ORIGIN/../lib
[exec] .build_release/lib/libcaffedistri.so: Nicht definierter Verweis auf `caffe::DataTransformer::Transform(std::vector<cv::Mat, std::allocatorcv::Mat > const&, caffe::Blob)'
[exec] collect2: error: ld returned 1 exit status
[exec] make[1]: ** [.build_release/src/main/cpp/tools/caffe_mini_cluster.bin] Error 1
[exec] Makefile:423: recipe for target '.build_release/src/main/cpp/tools/caffe_mini_cluster.bin' failed
[exec] make[1]: Leaving directory '/opt/CaffeOnSpark/caffe-distri'
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] caffe ............................................. SUCCESS [0.002s]
[INFO] caffe-distri ...................................... FAILURE [56.000s]
[INFO] caffe-grid ........................................ SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------

*=======================================================================

*## Refer to http://caffe.berkeleyvision.org/installation.html
*# Contributions simplifying and improving our build system are welcome!

*# cuDNN acceleration switch (uncomment to build with cuDNN).
*# USE_CUDNN := 1

*# CPU-only switch (uncomment to build without GPU support).
*# CPU_ONLY := 1

*# Parallelization over InfiniBand or RoCE
*# INFINIBAND := 1

*# uncomment to disable IO dependencies and corresponding data layers
*#USE_OPENCV := 0
*#SE_LEVELDB := 0
*#USE_LMDB := 0

*# uncomment to allow MDB_NOLOCK when reading LMDB files (only if necessary)
*# You should not set this flag if you will be reading LMDBs with any
*# possibility of simultaneous read and write
*# ALLOW_LMDB_NOLOCK := 1

*# Uncomment if you're using OpenCV 3
*OPENCV_VERSION := 3

*# To customize your choice of compiler, uncomment and set the following.
*# N.B. the default for Linux is g++ and the default for OSX is clang++
*# CUSTOM_CXX := g++

*# CUDA directory contains bin/ and lib/ directories that we need.
*CUDA_DIR := /usr/local/cuda
*# On Ubuntu 14.04, if cuda tools are installed via
*# "sudo apt-get install nvidia-cuda-toolkit" then use this instead:
*# CUDA_DIR := /usr

*# CUDA architecture setting: going with all of them.
*# For CUDA < 6.0, comment the *_50 lines for compatibility.
*CUDA_ARCH := -gencode arch=compute_20,code=sm_20 \

•   -gencode arch=compute_20,code=sm_21 \
  

•   -gencode arch=compute_30,code=sm_30 \
  

•   -gencode arch=compute_35,code=sm_35 \
  

•   -gencode arch=compute_50,code=sm_50 \
  

•   -gencode arch=compute_50,code=compute_50
  

*# BLAS choice:
*# atlas for ATLAS (default)
*# mkl for MKL
*# open for OpenBlas
*# BLAS := atlas
*# Custom (MKL/ATLAS/OpenBLAS) include and lib directories.
*# Leave commented to accept the defaults for your choice of BLAS
*# (which should work)!
*# BLAS_INCLUDE := /path/to/your/blas
*# BLAS_LIB := /path/to/your/blas

*# Homebrew puts openblas in a directory that is not on the standard search path
*# BLAS_INCLUDE := $(shell brew --prefix openblas)/include
*# BLAS_LIB := $(shell brew --prefix openblas)/lib

*# This is required only if you will compile the matlab interface.
*# MATLAB directory should contain the mex binary in /bin.
*# MATLAB_DIR := /usr/local
*# MATLAB_DIR := /Applications/MATLAB_R2012b.app

*# NOTE: this is required only if you will compile the python interface.
*# We need to be able to find Python.h and numpy/arrayobject.h.
*PYTHON_INCLUDE := /usr/include/python2.7 \

•   /usr/lib/python2.7/dist-packages/numpy/core/include
  

  *# Anaconda Python distribution is quite popular. Include path:
  *# Verify anaconda location, sometimes it's in root.
  *# ANACONDA_HOME := $(HOME)/anaconda
  *# PYTHON_INCLUDE := $(ANACONDA_HOME)/include \

•   # $(ANACONDA_HOME)/include/python2.7 \
  

•   # $(ANACONDA_HOME)/lib/python2.7/site-packages/numpy/core/include \
  

*# Uncomment to use Python 3 (default is Python 2)
*# PYTHON_LIBRARIES := boost_python3 python3.5m
*# PYTHON_INCLUDE := /usr/include/python3.5m
*# /usr/lib/python3.5/dist-packages/numpy/core/include

*# We need to be able to find libpythonX.X.so or .dylib.
*PYTHON_LIB := /usr/lib
*# PYTHON_LIB := $(ANACONDA_HOME)/lib

*# Homebrew installs numpy in a non standard path (keg only)
*# PYTHON_INCLUDE += $(dir $(shell python -c 'import numpy.core; *print(numpy.core.file)'))/include
*# PYTHON_LIB += $(shell brew --prefix numpy)/lib

*# Uncomment to support layers written in Python (will link against Python libs)
*# WITH_PYTHON_LAYER := 1

*# Whatever else you find you need goes here.
*INCLUDE_DIRS := $(PYTHON_INCLUDE) /usr/local/include /usr/include/hdf5/serial/
*LIBRARY_DIRS := $(PYTHON_LIB) /usr/local/lib /usr/lib /usr/lib/x86_64-linux-gnu/hdf5/serial/

*# If Homebrew is installed at a non standard location (for example your home directory) and you use it *for general dependencies
*# INCLUDE_DIRS += $(shell brew --prefix)/include
*# LIBRARY_DIRS += $(shell brew --prefix)/lib

*# Uncomment to use pkg-config to specify OpenCV library paths.
*# (Usually not necessary -- OpenCV libraries are normally installed in one of the above *$LIBRARY_DIRS.)
*# USE_PKG_CONFIG := 1

*BUILD_DIR := build
*DISTRIBUTE_DIR := distribute

*# Uncomment for debugging. Does not work on OSX due to BVLC/caffe#171
*# DEBUG := 1

*# The ID of the GPU that 'make runtest' will use to run unit tests.
*TEST_GPUID := 0

*# enable pretty build (comment to see full commands)
*# Q ?= @
*INCLUDE_DIRS += /usr/lib/jvm/default-java/include

[anfeng]

• make clean
• make build

Please do a clean build. That method is defined in caffe-public/src/caffe/data_transformer.cpp.

template
void DataTransformer::Transform(const vectorcv::Mat & mat_vector,
Blob* transformed_blob);