update actions version #3
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Keep deps (JNI + private) dependencies as 24.04-SNAPSHOT util they're available next week. Added TODO (NVIDIA#10256) to remind us to bump up deps version to 24.06.0-SNAPSHOT. Signed-off-by: Tim Liu <timl@nvidia.com>
[auto-merge] branch-24.04 to branch-24.06 [skip ci] [bot]
Signed-off-by: YanxuanLiu <yanxuanl@nvidia.com>
* Add fix for removing internal metadata information from 350 shim Signed-off-by: Partho Sarthi <psarthi@nvidia.com> * Create a shim helper class wth the relevant change instead of duplicating code Signed-off-by: Partho Sarthi <psarthi@nvidia.com> * Remove extra whitespace Signed-off-by: Partho Sarthi <psarthi@nvidia.com> --------- Signed-off-by: Partho Sarthi <psarthi@nvidia.com>
[auto-merge] branch-24.04 to branch-24.06 [skip ci] [bot]
* Use new kernel for getJsonObject Signed-off-by: Haoyang Li <haoyangl@nvidia.com> * Use table to pass parsed path Signed-off-by: Haoyang Li <haoyangl@nvidia.com> * use list/vector of instruction objects Signed-off-by: Haoyang Li <haoyangl@nvidia.com> * fallback when nested too long Signed-off-by: Haoyang Li <haoyangl@nvidia.com> * cancel xfail cases Signed-off-by: Haoyang Li <haoyangl@nvidia.com> * cancel xfail cases Signed-off-by: Haoyang Li <haoyangl@nvidia.com> * generated and modified docs Signed-off-by: Haoyang Li <haoyangl@nvidia.com> * wip Signed-off-by: Haoyang Li <haoyangl@nvidia.com> * wip Signed-off-by: Haoyang Li <haoyangl@nvidia.com> * apply jni change and remove xpass Signed-off-by: Haoyang Li <haoyangl@nvidia.com> * Adds test cases Signed-off-by: Haoyang Li <haoyangl@nvidia.com> --------- Signed-off-by: Haoyang Li <haoyangl@nvidia.com>
[auto-merge] branch-24.04 to branch-24.06 [skip ci] [bot]
Signed-off-by: Andy Grove <andygrove@nvidia.com>
[auto-merge] branch-24.04 to branch-24.06 [skip ci] [bot]
Signed-off-by: Jim Brennan <jimb@nvidia.com>
[auto-merge] branch-24.04 to branch-24.06 [skip ci] [bot]
…10637) * Cleanup async state when multi-threaded shuffle readers fail Signed-off-by: Alessandro Bellina <abellina@nvidia.com> --------- Signed-off-by: Alessandro Bellina <abellina@nvidia.com>
[auto-merge] branch-24.04 to branch-24.06 [skip ci] [bot]
To fix: NVIDIA#10624 As latest maven-gpg-plugin has the issue "error 401 Unauthorized" when deploy files onto Sonatype repo, we pin it to the stable version 3.1.0 to unblock our release process. Signed-off-by: Tim Liu <timl@nvidia.com>
[auto-merge] branch-24.04 to branch-24.06 [skip ci] [bot]
)" (NVIDIA#10657) This reverts commit c28c7fa. Signed-off-by: Jason Lowe <jlowe@nvidia.com>
[auto-merge] branch-24.04 to branch-24.06 [skip ci] [bot]
Fixes NVIDIA#10487 Signed-off-by: Gera Shegalov <gera@apache.org>
Signed-off-by: Tim Liu <timl@nvidia.com>
Signed-off-by: Tim Liu <timl@nvidia.com>
[auto-merge] branch-24.04 to branch-24.06 [skip ci] [bot]
Signed-off-by: Niranjan Artal <nartal@nvidia.com>
To fix: NVIDIA#10665 Signed-off-by: Tim Liu <timl@nvidia.com>
Signed-off-by: Jason Lowe <jlowe@nvidia.com>
Signed-off-by: liurenjie1024 <liurenjie2008@gmail.com>
Signed-off-by: YanxuanLiu <yanxuanl@nvidia.com>
…/spark-rapids into update-action-version
Fix merge conflict with branch-24.04 [skip ci]
* Add an config to switch back to old impl for getJsonObject Signed-off-by: Haoyang Li <haoyangl@nvidia.com> * Update sql-plugin/src/main/scala/com/nvidia/spark/rapids/GpuGetJsonObject.scala Co-authored-by: Gera Shegalov <gshegalov@nvidia.com> * update doc Signed-off-by: Haoyang Li <haoyangl@nvidia.com> * update doc Signed-off-by: Haoyang Li <haoyangl@nvidia.com> * verify and address comments Signed-off-by: Haoyang Li <haoyangl@nvidia.com> --------- Signed-off-by: Haoyang Li <haoyangl@nvidia.com> Co-authored-by: Gera Shegalov <gshegalov@nvidia.com>
[auto-merge] branch-24.04 to branch-24.06 [skip ci] [bot]
[auto-merge] branch-24.04 to branch-24.06 [skip ci] [bot]
…0681) Wait for the pre-merge CI job to SUCCEED Signed-off-by: jenkins <jenkins@localhost> Co-authored-by: jenkins <jenkins@localhost>
* Update latest changelog [skip ci] Update change log with CLI: \n\n scripts/generate-changelog --token=<GIT_TOKEN> --releases=24.02,24.04 Signed-off-by: jenkins <jenkins@localhost> * Update according to review Signed-off-by: Tim Liu <timl@nvidia.com> * Update change log to latest Signed-off-by: Tim Liu <timl@nvidia.com> * Update change log Update change log Change roadmap to 24.04 for PRs NVIDIA#10158,NVIDIA#10369,NVIDIA#10358,NVIDIA#10360, to move them into 24.04 PR list from 24.02 Signed-off-by: Tim Liu <timl@nvidia.com> --------- Signed-off-by: jenkins <jenkins@localhost> Signed-off-by: Tim Liu <timl@nvidia.com> Co-authored-by: jenkins <jenkins@localhost> Co-authored-by: Tim Liu <timl@nvidia.com>
…-10704 Fix auto merge conflict 10704 [skip ci]
Signed-off-by: Chong Gao <res_life@163.com> Signed-off-by: Robert (Bobby) Evans <bobby@apache.org> Co-authored-by: Chong Gao <res_life@163.com>
Signed-off-by: Hongbin Ma (Mahone) <mahongbin@apache.org>
* Removing some authorizations for departed users Signed-off-by: Mike Wilson <knobby@burntsheep.com> Co-authored-by: Sameer Raheja <sameerz@users.noreply.github.com>
* Upgrade to jucx 1.16.0 Signed-off-by: Alessandro Bellina <abellina@nvidia.com> --------- Signed-off-by: Alessandro Bellina <abellina@nvidia.com>
…10718) * Refactor Parquet reader Signed-off-by: Nghia Truong <nghiat@nvidia.com> * Update config * Add back the deprecated config Signed-off-by: Nghia Truong <nghiat@nvidia.com> * Fix config Signed-off-by: Nghia Truong <nghiat@nvidia.com> * Change message for the deprecated config Signed-off-by: Nghia Truong <nghiat@nvidia.com> * Rename variable Signed-off-by: Nghia Truong <nghiat@nvidia.com> * Change the logic of reading conf Signed-off-by: Nghia Truong <nghiat@nvidia.com> * Add example and mark conf as `internal()` Signed-off-by: Nghia Truong <nghiat@nvidia.com> * Reformat code Signed-off-by: Nghia Truong <nghiat@nvidia.com> * Update docs Signed-off-by: Nghia Truong <nghiat@nvidia.com> * Change configs Signed-off-by: Nghia Truong <nghiat@nvidia.com> * Update docs Signed-off-by: Nghia Truong <nghiat@nvidia.com> * Change variables into functions Signed-off-by: Nghia Truong <nghiat@nvidia.com> * Change functions back into `lazy val` Signed-off-by: Nghia Truong <nghiat@nvidia.com> --------- Signed-off-by: Nghia Truong <nghiat@nvidia.com>
…debugging UT in IDEA (NVIDIA#10733) * wip for test Signed-off-by: Haoyang Li <haoyangl@nvidia.com> * update comment Signed-off-by: Haoyang Li <haoyangl@nvidia.com> --------- Signed-off-by: Haoyang Li <haoyangl@nvidia.com>
NVIDIA#10716) Signed-off-by: Robert (Bobby) Evans <bobby@apache.org>
* generate shims * Generate Scala 2.13 poms Signed-off-by: Raza Jafri <rjafri@nvidia.com> * undo bad change to the supportedExprs.csv * Fixed copyrights and removed snapshot * Update copyrights on SparkShimsSuite --------- Signed-off-by: Raza Jafri <rjafri@nvidia.com>
* set fixed seed for some random failed tests Signed-off-by: Haoyang Li <haoyangl@nvidia.com> * add import Signed-off-by: Haoyang Li <haoyangl@nvidia.com> --------- Signed-off-by: Haoyang Li <haoyangl@nvidia.com>
Signed-off-by: Haoyang Li <haoyangl@nvidia.com>
* Shimplify 400 Signed-off-by: Raza Jafri <rjafri@nvidia.com> * Added 400 to all.buildvers * addressed review comments, moved the shim declaration to all.buildvers * added 2.13 pom --------- Signed-off-by: Raza Jafri <rjafri@nvidia.com>
…ldcard path (NVIDIA#10734) Signed-off-by: Chong Gao <res_life@163.com> Co-authored-by: Chong Gao <res_life@163.com>
* Use parse_url kernel for PATH parsing Signed-off-by: Haoyang Li <haoyangl@nvidia.com> * verify Signed-off-by: Haoyang Li <haoyangl@nvidia.com> --------- Signed-off-by: Haoyang Li <haoyangl@nvidia.com>
Signed-off-by: Robert (Bobby) Evans <bobby@apache.org>
…0755) * minor doc change Signed-off-by: cindyyuanjiang <cindyj@nvidia.com> * better wording Co-authored-by: Jason Lowe <jlowe@nvidia.com> --------- Signed-off-by: cindyyuanjiang <cindyj@nvidia.com> Co-authored-by: Jason Lowe <jlowe@nvidia.com>
Related to NVIDIA/spark-rapids-jni#1843 Signed-off-by: Gera Shegalov <gera@apache.org>
Closes NVIDIA#10721 ```json { "Event": "com.nvidia.spark.rapids.SparkRapidsBuildInfoEvent", "sparkRapidsBuildInfo": { "url": "https://github.com/NVIDIA/spark-rapids", "branch": "gerashegalov/issue10721", "date": "2024-04-30T04:47:11Z", "cudf_version": "24.06.0-SNAPSHOT", "user": "gshegalov", "revision": "d321e6b5555b1ad16dc515e70d1082e65172cefd", "version": "24.06.0-SNAPSHOT" }, "sparkRapidsJniBuildInfo": { "url": "https://github.com/NVIDIA/spark-rapids-jni.git", "branch": "HEAD", "gpu_architectures": "70;75;80;86;90", "date": "2024-04-28T05:18:44Z", "user": "root", "revision": "f3d5933601860c766a15b34968fa79af21ecaa86", "version": "24.06.0-SNAPSHOT" }, "cudfBuildInfo": { "url": "https://github.com/rapidsai/cudf.git", "branch": "HEAD", "gpu_architectures": "70;75;80;86;90", "date": "2024-04-28T05:18:41Z", "user": "root", "revision": "064dd7b02166cc67e882b708d66621bc3fafd70b", "version": "24.06.0-SNAPSHOT" }, "sparkRapidsPrivateBuildInfo": { "url": "https://gitlab-master.nvidia.com/nvspark/spark-rapids-private.git", "branch": "HEAD", "date": "2024-04-28T04:40:09Z", "user": "", "revision": "b8ec681d75ce850404b9340dc39d290ef735b3ec", "version": "" } } ``` Signed-off-by: Gera Shegalov <gera@apache.org>
…tabricks] (NVIDIA#10756) * Use cached ThreadPoolExecutor * Revert "Fix Multithreaded Readers working with Unity Catalog on Databricks [databricks] (NVIDIA#8296)" * Signing off Signed-off-by: Raza Jafri <rjafri@nvidia.com> * Removed spark311 version of ReaderUtils.scala --------- Signed-off-by: Raza Jafri <rjafri@nvidia.com>
…VIDIA#10644) * Add loggings to Integration test runs - Set explain all - Working logs - Added logger.FileHandler only for xdist workers * undo setting explain * added documentation and did some refactoring * Signing off Signed-off-by: Raza Jafri <rjafri@nvidia.com> * addressed review comments * added documentation to clarify ambiguity in multiple log4j props file * renamed xdist_it_log4j.properties to be more generic * Added log4j to the flow with TEST_PARALLEL < 2 * Added comments * Updated README.md * added more comments and updated copyrights * Made changes so the worker logs only get generated in local mode * Updated docs * added logs to local-cluster mode as well * Update integration_tests/run_pyspark_from_build.sh Co-authored-by: Gera Shegalov <gshegalov@nvidia.com> * Update integration_tests/run_pyspark_from_build.sh Co-authored-by: Gera Shegalov <gshegalov@nvidia.com> --------- Signed-off-by: Raza Jafri <rjafri@nvidia.com> Co-authored-by: Gera Shegalov <gshegalov@nvidia.com>
Signed-off-by: Partho Sarthi <psarthi@nvidia.com>
Signed-off-by: Robert (Bobby) Evans <bobby@apache.org>
…0767) fix NVIDIA#10751 A cuDF Table requires non empyt columns, so need to check the number of columns when converting a batch to a cuDF table. This PR adds the support for rows-only batches in RebatchingRoundoffIterator. --------- Signed-off-by: Firestarman <firestarmanllc@gmail.com>
…IA#10743) * initial PR for the framework resuing Vanila Spark's UTs (borrow from Gluten) Signed-off-by: Hongbin Ma (Mahone) <mahongbin@apache.org> * fix shim complain Signed-off-by: Hongbin Ma (Mahone) <mahongbin@apache.org> * fix other shim compile issue Signed-off-by: Hongbin Ma (Mahone) <mahongbin@apache.org> * fix Signed-off-by: Hongbin Ma (Mahone) <mahongbin@apache.org> * fix for scala 2.13 Signed-off-by: Hongbin Ma (Mahone) <mahongbin@apache.org> * more to shim Signed-off-by: Hongbin Ma (Mahone) <mahongbin@apache.org> * clean pom Signed-off-by: Hongbin Ma (Mahone) <mahongbin@apache.org> * move everything to shim Signed-off-by: Hongbin Ma (Mahone) <mahongbin@apache.org> * 2.13 Signed-off-by: Hongbin Ma (Mahone) <mahongbin@apache.org> * move to spark330 Signed-off-by: Hongbin Ma (Mahone) <mahongbin@apache.org> * fix Signed-off-by: Hongbin Ma (Mahone) <mahongbin@apache.org> * fix bug Signed-off-by: Hongbin Ma (Mahone) <mahongbin@apache.org> * fix blossom ci Signed-off-by: Hongbin Ma (Mahone) <mahongbin@apache.org> * address license comment Signed-off-by: Hongbin Ma (Mahone) <mahongbin@apache.org> * address suites util package Signed-off-by: Hongbin Ma (Mahone) <mahongbin@apache.org> * address reason comment Signed-off-by: Hongbin Ma (Mahone) <mahongbin@apache.org> * address other comments Signed-off-by: Hongbin Ma (Mahone) <mahongbin@apache.org> * fix bug Signed-off-by: Hongbin Ma (Mahone) <mahongbin@apache.org> * address 2rd round of comments Signed-off-by: Hongbin Ma (Mahone) <mahongbin@apache.org> * get rid of UNKNOW_ISSUE Signed-off-by: Hongbin Ma (Mahone) <mahongbin@apache.org> --------- Signed-off-by: Hongbin Ma (Mahone) <mahongbin@apache.org>
Signed-off-by: YanxuanLiu <yanxuanl@nvidia.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.