org.apache.maven.plugins:maven-antrun-plugin:1.7

[ai408]

root@ubuntu:~/GitProgram/Spark-Program/CaffeOnSpark# make build
cd caffe-public; make proto; make -j4 -e distribute; cd ..
make[1]: Entering directory /root/GitProgram/Spark-Program/CaffeOnSpark/caffe-public' PROTOC src/caffe/proto/caffe.proto make[1]: protoc: Command not found make[1]: *** [.build_release/src/caffe/proto/caffe.pb.cc] Error 127 make[1]: Leaving directory/root/GitProgram/Spark-Program/CaffeOnSpark/caffe-public'
make[1]: Entering directory /root/GitProgram/Spark-Program/CaffeOnSpark/caffe-public' PROTOC src/caffe/proto/caffe.proto make[1]: protoc: Command not found make[1]: *** [.build_release/src/caffe/proto/caffe.pb.h] Error 127 make[1]: *** Waiting for unfinished jobs.... make[1]: Leaving directory/root/GitProgram/Spark-Program/CaffeOnSpark/caffe-public'
export LD_LIBRARY_PATH="/usr/local/cuda/lib64::/root/GitProgram/Spark-Program/CaffeOnSpark/caffe-public/distribute/lib:/root/GitProgram/Spark-Program/CaffeOnSpark/caffe-distri/distribute/lib:/usr/lib64:/lib64 "; mvn -B package
[INFO] Scanning for projects...
[WARNING]
[WARNING] Some problems were encountered while building the effective model for com.yahoo.ml:caffe-grid:jar:0.1-SNAPSHOT
[WARNING] The expression ${version} is deprecated. Please use ${project.version} instead.
[WARNING]
[WARNING] It is highly recommended to fix these problems because they threaten the stability of your build.
[WARNING]
[WARNING] For this reason, future Maven versions might no longer support building such malformed projects.
[WARNING]
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Build Order:
[INFO]
[INFO] caffe
[INFO] caffe-distri
[INFO] caffe-grid
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building caffe 0.1-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building caffe-distri 0.1-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] --- maven-antrun-plugin:1.7:run (proto) @ caffe-distri ---
[INFO] Executing tasks

protoc:
[exec] make[1]: Entering directory /root/GitProgram/Spark-Program/CaffeOnSpark/caffe-distri' [exec] make[1]: Leaving directory/root/GitProgmake[1]: *** No rule to make target ../caffe-public/distribute/proto/caffe.proto', needed bysrc/main/java/caffe/Caffe.java'. ram/Spark-Program/CaffeOnSpark/caffe-distri'
[exec] Stop.
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] caffe .............................................. SUCCESS [ 0.003 s]
[INFO] caffe-distri ....................................... FAILURE [ 1.268 s]
[INFO] caffe-grid ......................................... SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 1.591 s
[INFO] Finished at: 2016-05-16T18:20:36+08:00
[INFO] Final Memory: 9M/176M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-antrun-plugin:1.7:run (proto) on project caffe-distri: An Ant BuildException has occured: exec returned: 2
[ERROR] around Ant part ...... @ 5:109 in /root/GitProgram/Spark-Program/CaffeOnSpark/caffe-distri/target/antrun/build-protoc.xml
[ERROR] -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR] mvn -rf :caffe-distri
make: *** [build] Error 1

[anfeng]

PROTOC src/caffe/proto/caffe.proto
make[1]:

protoc: Command not found
Please install protocol buffer 2.5.0

Andy

On Mon, May 16, 2016 at 3:28 AM, ShengshengWang notifications@github.com
wrote:

root@ubuntu:~/GitProgram/Spark-Program/CaffeOnSpark# make build
cd caffe-public; make proto; make -j4 -e distribute; cd ..
make[1]: Entering directory
/root/GitProgram/Spark-Program/CaffeOnSpark/caffe-public'
PROTOC src/caffe/proto/caffe.proto
make[1]: protoc: Command not found
make[1]: *** [.build_release/src/caffe/proto/caffe.pb.cc] Error 127
make[1]: Leaving directory
/root/GitProgram/Spark-Program/CaffeOnSpark/caffe-public'
make[1]: Entering directory
/root/GitProgram/Spark-Program/CaffeOnSpark/caffe-public'
PROTOC src/caffe/proto/caffe.proto
make[1]: protoc: Command not found
make[1]: *** [.build_release/src/caffe/proto/caffe.pb.h] Error 127
make[1]: *** Waiting for unfinished jobs....
make[1]: Leaving directory
/root/GitProgram/Spark-Program/CaffeOnSpark/caffe-public'
export
LD_LIBRARY_PATH="/usr/local/cuda/lib64::/root/GitProgram/Spark-Program/CaffeOnSpark/caffe-public/distribute/lib:/root/GitProgram/Spark-Program/CaffeOnSpark/caffe-distri/distribute/lib:/usr/lib64:/lib64
"; mvn -B package
[INFO] Scanning for projects...
[WARNING]
[WARNING] Some problems were encountered while building the effective
model for com.yahoo.ml:caffe-grid:jar:0.1-SNAPSHOT
[WARNING] The expression ${version} is deprecated. Please use
${project.version} instead.
[WARNING]
[WARNING] It is highly recommended to fix these problems because they
threaten the stability of your build.
[WARNING]
[WARNING] For this reason, future Maven versions might no longer support
building such malformed projects.
[WARNING]

[INFO]

[INFO] Reactor Build Order:
[INFO]
[INFO] caffe
[INFO] caffe-distri
[INFO] caffe-grid
[INFO]

[INFO]

[INFO] Building caffe 0.1-SNAPSHOT

[INFO]

[INFO]

[INFO]

[INFO] Building caffe-distri 0.1-SNAPSHOT

[INFO]

[INFO]
[INFO] --- maven-antrun-plugin:1.7:run (proto) @ caffe-distri ---
[INFO] Executing tasks

protoc:
[exec] make[1]: Entering directory
/root/GitProgram/Spark-Program/CaffeOnSpark/caffe-distri'
[exec] make[1]: Leaving directory/root/GitProgmake[1]: *** No rule to
make target ../caffe-public/distribute/proto/caffe.proto', needed bysrc/main/java/caffe/Caffe.java'.
ram/Spark-Program/CaffeOnSpark/caffe-distri'
[exec] Stop.

[INFO]

[INFO] Reactor Summary:
[INFO]
[INFO] caffe .............................................. SUCCESS [
0.003 s]
[INFO] caffe-distri ....................................... FAILURE [
1.268 s]
[INFO] caffe-grid ......................................... SKIPPED

[INFO]

[INFO] BUILD FAILURE

[INFO]

[INFO] Total time: 1.591 s
[INFO] Finished at: 2016-05-16T18:20:36+08:00
[INFO] Final Memory: 9M/176M

[INFO]

[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-antrun-plugin:1.7:run (proto) on project
caffe-distri: An Ant BuildException has occured: exec returned: 2
[ERROR] around Ant part ...... @ 5:109 in
/root/GitProgram/Spark-Program/CaffeOnSpark/caffe-distri/target/antrun/build-protoc.xml
[ERROR] -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the
-e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions,
please read the following articles:
[ERROR] [Help 1]
http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the
command
[ERROR] mvn -rf :caffe-distri
make: *** [build] Error 1

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub
#66

[javadba]

In my case protoc is already installed

$brew install protobuf
Warning: protobuf-2.6.1 already installed
11:37:56/CaffeOnSpark $which protoc
/usr/local/bin/protoc

[javadba]

This is the error for me:

protoc:
[exec] /bin/sh: /bin/nvcc: No such file or directory
[exec] awk: syntax error at source line 1
[exec] context is
[exec] {exit >>> < <<< 7.0;}
[exec] awk: illegal statement at source line 1
[exec] make[1]: *** No rule to make target ../caffe-public/distribute/proto/caffe.proto', needed bysrc/main/java/caffe/Caffe.java'. Stop.

[mriduljain]

Uninstall protobuf 2.6 and install 2.5.0. Then inside caffe-public , try
make proto to check. 2.6 creates problems

On Tue, Jun 7, 2016 at 11:39 AM, StephenBoesch notifications@github.com
wrote:

This is the error for me:

protoc:
[exec] /bin/sh: /bin/nvcc: No such file or directory
[exec] awk: syntax error at source line 1
[exec] context is
[exec] {exit >>> < <<< 7.0;}
[exec] awk: illegal statement at source line 1
[exec] make[1]: *** No rule to make target ../caffe-public/distribute/proto/caffe.proto',
needed bysrc/main/java/caffe/Caffe.java'. Stop.

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
#66 (comment),
or mute the thread
https://github.com/notifications/unsubscribe/ACCTVZ9id8jXo-udOCnd9Hagyy95kUEmks5qJbrKgaJpZM4IfNUA
.

[javadba]

OK the real problem was I had not done git with --recursive. Building now. Thanks for your help.

[mriduljain]

If this works, please close the bug

[javadba]

Do I have that capability? Whoever does please close the bug.

[tavisbarr]

Hi. Just to note I had a similar issue building on a fairly plain vanilla Ubuntu 16.04, which comes with only protobuf 2.6.1 in the standard repositories. I changed the version in the pom file for caffe-distri to 2.6.1; the latter compiled, but then I also had to add the dependency to the pom file for caffe-grid. [And also as a completely different issue I had to add /usr/local/include /usr/include/hdf5/serial/ to INCLUDED_DIRS, and also change the Scala version in the pom file to match the Scala version that Spark was compiled with]. Then it compiled without any problems. Don't know if it runs yet, but will let you know if it doesn't. :) So I might suggest addiing a variable to set the version of protobuf in the pom file to the installed version, and also adding it to the dependencies in the pom file for caffe-grid?

[junshi15]

I think protobuf version needs to be consistent among caffe-public, caffe-distri and caffe-grid. POM only covers scala/java, not c++.

[MarcGodard]

I am having the same error. I am using protobuf 2.5 (tried the most current as well) and I did a recursive clone of the repo.

[MarcGodard]

Because I am only trying this on one of my fun toy clusters as a fun side project, I have to move on to other things for now. However, here is the build error and the tail of the maven logs if anyone can help.

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-antrun-plugin:1.7:run (proto) on project caffe-distri: An Ant BuildException has occured: exec returned: 2
[ERROR] around Ant part ...<exec failonerror="true" dir="/Users/marcgodard/CaffeOnSpark/caffe-distri" executable="make">... @ 5:96 in /Users/marcgodard/CaffeOnSpark/caffe-distri/target/antrun/build-protoc.xml
[ERROR] -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
[ERROR] 
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn <goals> -rf :caffe-distri

[ERROR] No goals have been specified for this build. You must specify a valid lifecycle phase or a goal in the format <plugin-prefix>:<goal> or <plugin-group-id>:<plugin-artifact-id>[:<plugin-version>]:<goal>. Available lifecycle phases are: validate, initialize, generate-sources, process-sources, generate-resources, process-resources, compile, process-classes, generate-test-sources, process-test-sources, generate-test-resources, process-test-resources, test-compile, process-test-classes, test, prepare-package, package, pre-integration-test, integration-test, post-integration-test, verify, install, deploy, pre-clean, clean, post-clean, pre-site, site, post-site, site-deploy. -> [Help 1]
org.apache.maven.lifecycle.NoGoalSpecifiedException: No goals have been specified for this build. You must specify a valid lifecycle phase or a goal in the format <plugin-prefix>:<goal> or <plugin-group-id>:<plugin-artifact-id>[:<plugin-version>]:<goal>. Available lifecycle phases are: validate, initialize, generate-sources, process-sources, generate-resources, process-resources, compile, process-classes, generate-test-sources, process-test-sources, generate-test-resources, process-test-resources, test-compile, process-test-classes, test, prepare-package, package, pre-integration-test, integration-test, post-integration-test, verify, install, deploy, pre-clean, clean, post-clean, pre-site, site, post-site, site-deploy.
    at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:94)
    at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:307)
    at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:193)
    at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:106)
    at org.apache.maven.cli.MavenCli.execute(MavenCli.java:863)
    at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:288)
    at org.apache.maven.cli.MavenCli.main(MavenCli.java:199)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)
    at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)
    at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)

[guiyang882]

I hava the same Problem,

[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-antrun-plugin:1.7:run (make) on project caffe-distri: An Ant BuildException has occured: exec returned: 2
[ERROR] around Ant part ...<exec failonerror="true" dir="/root/caffe/CaffeOnSpark/caffe-distri" executable="make">... @ 5:90 in /root/caffe/CaffeOnSpark/caffe-distri/target/antrun/build-make.xml
[ERROR] -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn <goals> -rf :caffe-distri
make: *** [build] Error 1

and the protoc version is right:

[root@2e7ae0fffc69 CaffeOnSpark]# protoc --version
libprotoc 2.5.0

[junshi15]

what's your maven version? Also is this the entire error message? I am wondering if errors appeared earlier? it would be nice if you can post the entire log.

[guiyang882]

@junshi15

[root@9c190733302e CaffeOnSpark]# mvn --version
Apache Maven 3.3.9 (bb52d8502b132ec0a5a3f4c09453c07478323dc5; 2015-11-10T16:41:47+00:00)
Maven home: /root/caffe/apache-maven-3.3.9
Java version: 1.8.0_111, vendor: Oracle Corporation
Java home: /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.111-1.b15.el7_2.x86_64/jre
Default locale: en_US, platform encoding: ANSI_X3.4-1968
OS name: "linux", version: "4.2.0-16-generic", arch: "amd64", family: "unix"

[guiyang882]

@junshi15

I have fix this problem.
I split the Makefile into shell command Script and run the command one by one !
I can build this program on docker !
thx !

[junshi15]

@liuguiyangnwpu Great you fixed it. Would you please share your solution? It may benefit other users who experience the same problem.

[guiyang882]

@junshi15
OK !

First, the Makefile in the caffe-public, look like it

HOME ?=/home/${USER}
ifeq ($(shell which spark-submit),)
     SPARK_HOME ?=/home/y/share/spark
else
     SPARK_HOME ?=$(shell which spark-submit 2>&1 | sed 's/\/bin\/spark-submit//g')
endif
CAFFE_ON_SPARK ?=$(shell pwd)
LD_LIBRARY_PATH ?=/home/y/lib64:/home/y/lib64/mkl/intel64
LD_LIBRARY_PATH2=${LD_LIBRARY_PATH}:${CAFFE_ON_SPARK}/caffe-public/distribute/lib:${CAFFE_ON_SPARK}/caffe-distri/distribute/lib:/usr/lib64:/lib64
DYLD_LIBRARY_PATH ?=/home/y/lib64:/home/y/lib64/mkl/intel64
DYLD_LIBRARY_PATH2=${DYLD_LIBRARY_PATH}:${CAFFE_ON_SPARK}/caffe-public/distribute/lib:${CAFFE_ON_SPARK}/caffe-distri/distribute/lib:/usr/lib64:/lib64

export SPARK_VERSION=$(shell ${SPARK_HOME}/bin/spark-submit --version 2>&1 | grep version | awk '{print $$5}' | cut -d'.' -f1)
ifeq (${SPARK_VERSION}, 2)
    export MVN_SPARK_FLAG=-Dspark2
endif

I ues the docker to virutal Centos7, the default user is root, so I change the path into that

LD_LIBRARY_PATH=/usr/lib64
LD_LIBRARY_PATH2=${LD_LIBRARY_PATH}:${CAFFE_ON_SPARK}/caffe-public/distribute/lib:${CAFFE_ON_SPARK}/caffe-distri/distribute/lib:/usr/lib64:/lib64
DYLD_LIBRARY_PATH=/usr/lib64
DYLD_LIBRARY_PATH2=${DYLD_LIBRARY_PATH}:${CAFFE_ON_SPARK}/caffe-public/distribute/lib:${CAFFE_ON_SPARK}/caffe-distri/distribute/lib:/usr/lib64:/lib64

I guess the problem is not fetch the Spark-Version-ID

export SPARK_VERSION=2
export MVN_SPARK_FLAG=-Dspark2

add above scripts into one shell script, and source it. Then run the Make Build script One By One.

cd caffe-public
make proto
make -j4 -e distribute
cd ..
export LD_LIBRARY_PATH=${LD_LIBRARY_PATH2}
mvn ${MVN_SPARK_FLAG} -B package -DskipTests
jar -xvf caffe-grid/target/caffe-grid-0.1-SNAPSHOT-jar-with-dependencies.jar META-INF/native/linux64/liblmdbjni.so
mv META-INF/native/linux64/liblmdbjni.so ${CAFFE_ON_SPARK}/caffe-distri/distribute/lib
${CAFFE_ON_SPARK}/scripts/setup-mnist.sh
export LD_LIBRARY_PATH=${LD_LIBRARY_PATH2}
mvn ${MVN_SPARK_FLAG} -B package
cp -r ${CAFFE_ON_SPARK}/caffe-public/python/caffe ${CAFFE_ON_SPARK}/caffe-grid/src/main/python/
cd ${CAFFE_ON_SPARK}/caffe-grid/src/main/python/
zip -r caffeonsparkpythonapi  *
mv caffeonsparkpythonapi.zip ${CAFFE_ON_SPARK}/caffe-grid/target/;cd ${CAFFE_ON_SPARK}
export LD_LIBRARY_PATH=${LD_LIBRARY_PATH2}
export SPARK_HOME=${SPARK_HOME}
${CAFFE_ON_SPARK}/caffe-grid/src/test/python/PythonTest.sh

you can finished it !

By the way, I guess the bug is:

export SPARK_VERSION=$(shell ${SPARK_HOME}/bin/spark-submit --version 2>&1 | grep version | awk '{print $$5}' | cut -d'.' -f1)
ifeq (${SPARK_VERSION}, 2)
    export MVN_SPARK_FLAG=-Dspark2
endif

should fix it like that:

export SPARK_VERSION=$(shell ${SPARK_HOME}/bin/spark-submit --version 2>&1 | grep version | awk '{print $5}' | cut -d'.' -f1)
ifeq (${SPARK_VERSION}, 2)
    export MVN_SPARK_FLAG=-Dspark2
endif

thx !