SPARKC-704 - spark 3.5.0 support #1363
Conversation
|
RC2 is out, don't anticipate any issues, but I'll bump this up and resolve conflicts soon regardless. |
JackBuggins
changed the title
|
This pull request is now ready for review, due to publishing of spark 3.5 GA |
|
@jtgrabowski, would you mind looking over this one? |
|
@jtgrabowski could you please have a look? |
|
빨리 좀 머지해쥬셩 |
|
why not marged & released? |
|
@jtgrabowski Sorry to tag you twice, I noticed that recently you had reviewed another PR. Do you think you would be in a position to conduct a review/facilitate the pre req of cutting branch-3.4 any time soon? Much appreciated if you can carve the time. |
README.md
Outdated
| @@ -45,15 +45,17 @@ corresponds to the 1.6 release. The "master" branch will normally contain | |||
| development for the next connector release in progress. | |||
|
|
|||
| Currently, the following branches are actively supported: | |||
| 3.4.x ([master](https://github.com/datastax/spark-cassandra-connector/tree/master)), | |||
| 3.5.x ([master](https://github.com/datastax/spark-cassandra-connector/tree/master)), | |||
| 3.4.x ([b3.4](https://github.com/datastax/spark-cassandra-connector/tree/master/b3.4)), | |||
There was a problem hiding this comment.
Please remove master here, it should be https://github.com/datastax/spark-cassandra-connector/tree/b3.4
|
Sorry, I think I caused a conflict by changing the |
|
Is this pull request crucial for Spark 3.5 connecting to Cassandra db? |
|
Thank you @JackBuggins ! |
|
@jtgrabowski Could you please create a release? Thank You! |
|
The new release will come out next week. |
|
Thank You! |
|
The release is out. Thank you all! |
Description
Upgrades core dependencies against the spark 3.5.0 RC1.
How did the Spark Cassandra Connector Work or Not Work Before this Patch
Changes required for compatibility with API changes present in the spark 3.5.0 RC1, see
https://github.com/apache/spark/pull/39627,https://github.com/apache/spark/pull/41928,General Design of the patch
How the fix is accomplished, were new parameters or classes added? Why did you
pursue this particular fix?
This has been persued to aid the community in preparing a release, with a goal that we can rapidly deliver an update on the 3.5.* stream as well as potentially publish RC's of the connector in line with Sparks RC phases. This can help ease engagement of voting on spark for users of this project, by making this dependency available for them.
Provides: SPARKC-704
How Has This Been Tested?
The upgrade has been tested via unit/integration tests locally and via CI, and a limited run of external private integration tests; more validation is welcome via RC stages by other users to extend the scope. The staging repository was used to fetch Spark 3.5* release candidate builds. As this progresses from a RC state, we should seek to remove that registry from the set of resolvers for the project. This could be carried out in another branch.
Checklist: