Skip to content
This repository has been archived by the owner on Nov 16, 2019. It is now read-only.

Run with spark local model problem #113

Closed
yamadi opened this issue Jul 18, 2016 · 2 comments
Closed

Run with spark local model problem #113

yamadi opened this issue Jul 18, 2016 · 2 comments

Comments

@yamadi
Copy link

yamadi commented Jul 18, 2016

Hi ,
When I run the scrip:
spark-submit --master local
--files /home/caffeOnSpark/CaffeOnSpark/data/lenet_memory_solver.prototxt,/home/caffeOnSpark/CaffeOnSpark/data/lenet_memory_train_test.prototxt
--conf spark.cores.max=1
--conf spark.task.cpus=1
--conf spark.driver.extraLibraryPath="$LD_LIBRARY_PATH"
--conf spark.executorEnv.LD_LIBRARY_PATH="$LD_LIBRARY_PATH"
--class com.yahoo.ml.caffe.CaffeOnSpark
/home/caffeOnSpark/CaffeOnSpark/caffe-grid/target/caffe-grid-0.1-SNAPSHOT-jar-with-dependencies.jar
-train
-features accuracy,loss -label label
-conf lenet_memory_solver.prototxt
-clusterSize 1
-devices 1
-connection ethernet
-model file:/home/caffeOnSpark/CaffeOnSpark/mnist_lenet.model
-output file:/home/caffeOnSpark/CaffeOnSpark/lenet_features_result

It occur the problem:
Caused by: org.apache.spark.SparkException: File /tmp/spark-aa8bdf19-346e-49cf-bf0d-595896e5dc79/userFiles-285a8e4b-78bb-417a-9c9f-f5c5adaba6f8/mnist_train_lmdb exists and does not match contents of file:/home/caffeOnSpark/CaffeOnSpark/data/mnist_train_lmdb/

Can you help me ?

@junshi15
Copy link
Collaborator

junshi15 commented Jul 19, 2016

maybe you need to delete the following file before launching it again?
/tmp/spark-aa8bdf19-346e-49cf-bf0d-595896e5dc79/userFiles-285a8e4b-78bb-417a-9c9f-f5c5adaba6f8/mnist_train_lmdb

@yamadi
Copy link
Author

yamadi commented Jul 20, 2016

I can't find the files.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants