LIVY-205. sparkr.zip is unzipped on executor side #181
Conversation
|
@vanzin Although this PR can make it work, but it needs some configuration change on spark-defaults.conf. Because livy use |
|
Never mind, I think I may made a mistake in #177. R is used in spark as well for interactive mode while RScript is used for batch mode. |
|
@vanzin I have a Spark PR (apache/spark#14744) to allow setting sparkr shell command through configuration. And I updated this livy PR based on the spark PR. |
Current coverage is 69.18% (diff: 50.00%)@@ master #181 diff @@
==========================================
Files 79 81 +2
Lines 3867 3956 +89
Methods 0 0
Messages 0 0
Branches 645 664 +19
==========================================
+ Hits 2702 2737 +35
- Misses 792 834 +42
- Partials 373 385 +12
|
|
ping @vanzin |
| val rArchivesFile = new File(path) | ||
| require(rArchivesFile.exists(), "sparkr.zip not found; cannot run sparkr application.") | ||
| rArchivesFile.getAbsolutePath() | ||
| rArchivesFile.getAbsolutePath() + "#sparkr" |
There was a problem hiding this comment.
Don't you need to add this suffix when the user sets SPARKR_PACKAGE too? Avoids having to force the user to do it manually.
|
ping @vanzin |
alex-the-man
force-pushed
the
master
branch
2 times, most recently
from
39a5162 to
2d6e026
Compare
|
Merging. |
3 participants
This PR is to unzipped sparkr.zip on executor side.