[SPARK-49920][INFRA] Install R for ubuntu 24.04 when GA run k8s-integration-tests
#48406
Conversation
panbingkun
changed the title
R for ubuntu 24.04
.github/workflows/build_and_test.yml
Outdated
| if: fromJson(needs.precondition.outputs.required).k8s-integration-tests == 'true' | ||
| name: Run Spark on Kubernetes Integration test | ||
| runs-on: ubuntu-latest | ||
| runs-on: ubuntu-24.04 |
There was a problem hiding this comment.
actions/runner-images#10636
Because the GitHub community is still in the process of migration (sometimes it's ubuntu-22.04, sometimes it's ubuntu-22.04), adding this is to verify the issue. After obtaining approve, I will remove it.
|
|
||
| private[spark] trait BasicTestsSuite { k8sSuite: KubernetesSuite => | ||
|
|
||
|
|
There was a problem hiding this comment.
Just to trigger k8s-integration-tests, after obtaining approve, I will remove it.
LuciferYang
left a comment
LuciferYang
left a comment
There was a problem hiding this comment.
+1, LGTM (pending tests)
Thanks @panbingkun , this solved the issue I encountered recently.
.github/workflows/build_and_test.yml
Outdated
| if: fromJson(needs.precondition.outputs.required).k8s-integration-tests == 'true' | ||
| name: Run Spark on Kubernetes Integration test | ||
| runs-on: ubuntu-latest | ||
| runs-on: ubuntu-24.04 |
There was a problem hiding this comment.
I think you should fix https://github.com/apache/spark/blob/master/dev/infra/Dockerfile?
There was a problem hiding this comment.
hmm... do you mean we should migrate k8s-integration-tests into container first? k8s-integration-tests does not rely on the base image for execution now.
There was a problem hiding this comment.
Line 61 in 38d66fd
This Docker file already has r-base installed.
There was a problem hiding this comment.
Yes, currently it is not running in a container
spark/.github/workflows/build_and_test.yml
Lines 1073 to 1078 in 38d66fd
There was a problem hiding this comment.
Perhaps we could try moving the tests into the container, and then install minikube by default, but only start it when running k8s-integration-tests?
However, this configuration will be reused for the daily tests of branch-3.x, so we need to carefully consider how to ensure compatibility.
|
GA has passed, I will restore some testing logic. |
LuciferYang
changed the title
R for ubuntu 24.04R for ubuntu 24.04 when GA run k8s-integration-tests
LuciferYang
changed the title
R for ubuntu 24.04 when GA run k8s-integration-testsR for ubuntu 24.04 when GA run k8s-integration-tests
all test passed |
|
Merged into master. Thanks @panbingkun @HyukjinKwon |
3 participants
What changes were proposed in this pull request?
The pr aims to install
Rforubuntu 24.04when GA runk8s-integration-tests.Why are the changes needed?
As the GitHub community switches the default version of
ubuntu-latestfromubuntu-22.04toubuntu-24.04.Ubuntu-latest workflows will use Ubuntu-24.04 image actions/runner-images#10636
In
ubuntu-24.04,Risnot installedby defaultA.
ubuntu-24.04(Risnot installedby default)https://github.com/actions/runner-images/blob/main/images/ubuntu/Ubuntu2404-Readme.md#tools
B.
ubuntu-22.04(Risinstalledby default)https://github.com/actions/runner-images/blob/main/images/ubuntu/Ubuntu2204-Readme.md#tools
Fix the failure issue of GA
https://github.com/LuciferYang/spark/actions/runs/11268158324/job/31334445659
Does this PR introduce any user-facing change?
No, only for tests.
How was this patch tested?
Pass GA.
Was this patch authored or co-authored using generative AI tooling?
No.