-
Notifications
You must be signed in to change notification settings - Fork 235
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
ADD 321cdh shim to rapids and remove 311cdh shim #5346
Conversation
@sririshindra please do a commit with -s to signoff: https://github.com/NVIDIA/spark-rapids/blob/branch-22.06/CONTRIBUTING.md#sign-your-work |
sql-plugin/src/main/320/scala/com/nvidia/spark/rapids/shims/ArrowShims.scala
Outdated
Show resolved
Hide resolved
sql-plugin/src/main/320/scala/com/nvidia/spark/rapids/shims/ArrowShims.scala
Outdated
Show resolved
Hide resolved
sql-plugin/src/main/320/scala/com/nvidia/spark/rapids/shims/ArrowShims.scala
Outdated
Show resolved
Hide resolved
sql-plugin/src/main/320/scala/com/nvidia/spark/rapids/shims/ArrowShims.scala
Outdated
Show resolved
Hide resolved
sql-plugin/src/main/320until330-cdh/scala/com/nvidia/spark/rapids/shims/OrcShims.scala
Outdated
Show resolved
Hide resolved
sql-plugin/src/main/320until330-cdh/scala/com/nvidia/spark/rapids/shims/OrcShims.scala
Outdated
Show resolved
Hide resolved
751e3f8
to
61b83ad
Compare
...rc/main/320until330-noncdh/scala/com/nvidia/spark/rapids/shims/OrcShims320untilAllBase.scala
Outdated
Show resolved
Hide resolved
sql-plugin/src/main/321cdh/scala/com/nvidia/spark/rapids/shims/OrcShims311until320Base.scala
Outdated
Show resolved
Hide resolved
sql-plugin/src/main/321cdh/scala/com/nvidia/spark/rapids/shims/SparkShims.scala
Outdated
Show resolved
Hide resolved
sql-plugin/src/main/321cdh/scala/com/nvidia/spark/rapids/spark321cdh/RapidsShuffleManager.scala
Outdated
Show resolved
Hide resolved
...21cdh/scala/org/apache/spark/sql/rapids/shims/spark321cdh/RapidsShuffleInternalManager.scala
Show resolved
Hide resolved
Also please upmerge to the latest branch-22.06 |
...c/main/321cdh/scala/com/nvidia/spark/rapids/shims/spark321cdh/SparkShimServiceProvider.scala
Outdated
Show resolved
Hide resolved
Signed-off-by: rishi <sririshindra@gmail.com>
Signed-off-by: rishi <sririshindra@gmail.com>
d50bb2b
to
4d11395
Compare
Build |
it looks like we missed the premerge script being updated. @sririshindra can you update https://github.com/NVIDIA/spark-rapids/blob/branch-22.06/jenkins/spark-premerge-build.sh#L43 to change 311cdh to 321cdh Could you also update our docs to have 321cdh instead of 311cdh: |
Trying to run locally the unit tests still don't run , it looks like the arrow-vector is pulling in an older version of jackso, so we need to exclude it:
It would also be nice to make a arrow.version variable in top level pom for the 2.0.0 |
Signed-off-by: rishi <sririshindra@gmail.com>
build |
thanks @sririshindra |
This PR adds cdh321 shim and removes 311cdh shim.
contributes to #5281 and fixes #5404