[TEST][SPARKR][CORE] Fix broken SparkSubmitSuite #18283
Conversation
| val main = MavenCoordinate("my.great.lib", "mylib", "0.1") | ||
| val sparkHome = sys.props.getOrElse("spark.test.home", fail("spark.test.home is not set!")) | ||
| val rScriptDir = | ||
| Seq(sparkHome, "R", "pkg", "inst", "tests", "packageInAJarTest.R").mkString(File.separator) |
There was a problem hiding this comment.
hmm, this path wasn't correct to begin with
|
Test build #77959 has finished for PR 18283 at commit
|
|
Test build #77960 has finished for PR 18283 at commit
|
## What changes were proposed in this pull request? Fix test file path. This is broken in #18264 and undetected since R-only changes don't build core and subsequent post-commit with the change built fine (again because it wasn't building core) actually appveyor builds everything but it's not running scala suites ... ## How was this patch tested? jenkins srowen gatorsmile Author: Felix Cheung <felixcheung_m@hotmail.com> Closes #18283 from felixcheung/rsubmitsuite. (cherry picked from commit 278ba7a) Signed-off-by: Felix Cheung <felixcheung@apache.org>
|
merged to master/2.2 like the other PR. |
|
@felixcheung I'm seeing different failures; are these real failures or something wrong with the machine? It happened twice, FWIW: |
|
Hmm odd. jar command not found would indicate a machine issue...
|
|
I guess that script can be changed to use JAVA_HOME instead of blindly assume it's accessible... |
|
well, if you look at that build's environment variables (left side of the page), you'll see that JAVA_HOME is indeed set: it also looks like the config was modified not too long before things went green (but nothing in the spark build config repo): @JoshRosen any ideas? FYI: these are also the same jenkins worker nodes that we've been using for the past ~3 years... no changes. i'll keep an eye on things and check build logs for more errors like this, but it seems like it was transient... |
|
@shaneknapp right - this script (install-dev.sh) has been assuming it can find let me know if it recurs - I could just fix it by checking JAVA_HOME. |
## What changes were proposed in this pull request? Fix test file path. This is broken in apache#18264 and undetected since R-only changes don't build core and subsequent post-commit with the change built fine (again because it wasn't building core) actually appveyor builds everything but it's not running scala suites ... ## How was this patch tested? jenkins srowen gatorsmile Author: Felix Cheung <felixcheung_m@hotmail.com> Closes apache#18283 from felixcheung/rsubmitsuite.
|
I guess the question is more whether |
|
ok, i just took a closer look on that worker (amp-jenkins-worker-08) and somehow the /etc/alternatives/jar entry is missing (which is in turn linked to /usr/bin/jar). all the other workers are properly set up. i'll get amp-jenkins-worker-08 fixed right now. sidenote: new jenkins coming soon, and i can't wait to get rid of these crusty old centos 6 boxen for shiny new ubuntu 16 LTS. |
|
done. |
## What changes were proposed in this pull request? Fix test file path. This is broken in apache#18264 and undetected since R-only changes don't build core and subsequent post-commit with the change built fine (again because it wasn't building core) actually appveyor builds everything but it's not running scala suites ... ## How was this patch tested? jenkins srowen gatorsmile Author: Felix Cheung <felixcheung_m@hotmail.com> Closes apache#18283 from felixcheung/rsubmitsuite. (cherry picked from commit 278ba7a) Signed-off-by: Felix Cheung <felixcheung@apache.org>
4 participants
What changes were proposed in this pull request?
Fix test file path. This is broken in #18264 and undetected since R-only changes don't build core and subsequent post-commit with the change built fine (again because it wasn't building core)
actually appveyor builds everything but it's not running scala suites ...
How was this patch tested?
jenkins
@srowen @gatorsmile