Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Forward-merge branch-24.12 into branch-25.02 #17343

Open
wants to merge 14 commits into
base: branch-25.02
Choose a base branch
from

Commits on Nov 15, 2024

  1. add telemetry setup to test (#16924)

    This is a prototype implementation of rapidsai/build-infra#139
    
    The work that this builds on:
    * rapidsai/gha-tools#118, which adds a shell wrapper that automatically creates spans for the commands that it wraps. It also uses the `opentelemetry-instrument` command to set up monkeypatching for supported Python libraries, if the command is python-based
    * https://github.com/rapidsai/shared-workflows/tree/add-telemetry, which installs the gha-tools work from above and sets necessary environment variables. This is only done for the conda-cpp-build.yaml shared workflow at the time of submitting this PR.
    
    The goal of this PR is to observe telemetry data sent from a GitHub Actions build triggered by this PR as a proof of concept. Once it all works, the remaining work is:
    
    * merge rapidsai/gha-tools#118
    * Move the opentelemetry-related install stuff in https://github.com/rapidsai/shared-workflows/compare/add-telemetry?expand=1#diff-ca6188672785b5d214aaac2bf77ce0528a48481b2a16b35aeb78ea877b2567bcR118-R125 into https://github.com/rapidsai/ci-imgs, and rebuild ci-imgs
    * expand coverage to other shared workflows
    * Incorporate the changes from this PR to other jobs and to other repos
    
    Authors:
      - Mike Sarahan (https://github.com/msarahan)
    
    Approvers:
      - Bradley Dice (https://github.com/bdice)
    
    URL: #16924
    msarahan authored Nov 15, 2024
    Configuration menu
    Copy the full SHA
    8664fad View commit details
    Browse the repository at this point in the history
  2. Update cmake to 3.28.6 in JNI Dockerfile (#17342)

    Updates cmake to 3.28.6 in the JNI Dockerfile used to build the cudf jar.  This helps avoid a bug in older cmake where FindCUDAToolkit can fail to find cufile libraries.
    
    Authors:
      - Jason Lowe (https://github.com/jlowe)
    
    Approvers:
      - Nghia Truong (https://github.com/ttnghia)
      - Gera Shegalov (https://github.com/gerashegalov)
    
    URL: #17342
    jlowe authored Nov 15, 2024
    Configuration menu
    Copy the full SHA
    e683647 View commit details
    Browse the repository at this point in the history

Commits on Nov 16, 2024

  1. Use pylibcudf contiguous split APIs in cudf python (#17246)

    Apart of #15162
    
    Authors:
      - Matthew Murray (https://github.com/Matt711)
    
    Approvers:
      - Lawrence Mitchell (https://github.com/wence-)
    
    URL: #17246
    Matt711 authored Nov 16, 2024
    Configuration menu
    Copy the full SHA
    9cc9071 View commit details
    Browse the repository at this point in the history

Commits on Nov 18, 2024

  1. Move strings translate benchmarks to nvbench (#17325)

    Moves `cpp/benchmarks/string/translate.cpp` implementation from google-bench to nvbench.
    This is benchmark for the `cudf::strings::translate` API.
    
    Authors:
      - David Wendt (https://github.com/davidwendt)
    
    Approvers:
      - Vukasin Milovanovic (https://github.com/vuule)
      - Nghia Truong (https://github.com/ttnghia)
    
    URL: #17325
    davidwendt authored Nov 18, 2024
    Configuration menu
    Copy the full SHA
    e4de8e4 View commit details
    Browse the repository at this point in the history
  2. Move cudf._lib.unary to cudf.core._internals (#17318)

    Contributes to #17317
    
    Authors:
      - Matthew Roeschke (https://github.com/mroeschke)
    
    Approvers:
      - GALI PREM SAGAR (https://github.com/galipremsagar)
    
    URL: #17318
    mroeschke authored Nov 18, 2024
    Configuration menu
    Copy the full SHA
    aeb6a30 View commit details
    Browse the repository at this point in the history
  3. Reading multi-source compressed JSONL files (#17161)

    Fixes #17068 
    Fixes #12299
    
    This PR introduces a new datasource for compressed inputs which enables batching and byte range reading of multi-source JSONL files using the reallocate-and-retry policy. Moreover. instead of using a 4:1 compression ratio heuristic, the device buffer size is estimated accurately for GZIP, ZIP, and SNAPPY compression types. For remaining types, the files are first decompressed then batched.
    
    ~~TODO: Reuse existing JSON tests but with an additional compression parameter to verify correctness.~~
    ~~Handled by #17219, which implements compressed JSON writer required for the above test.~~
    Multi-source compressed input tests added!
    
    Authors:
      - Shruti Shivakumar (https://github.com/shrshi)
    
    Approvers:
      - Vukasin Milovanovic (https://github.com/vuule)
      - Kyle Edwards (https://github.com/KyleFromNVIDIA)
      - Karthikeyan (https://github.com/karthikeyann)
    
    URL: #17161
    shrshi authored Nov 18, 2024
    Configuration menu
    Copy the full SHA
    03ac845 View commit details
    Browse the repository at this point in the history
  4. Test the full matrix for polars and dask wheels on nightlies (#17320)

    This PR ensures that we have nightly coverage of more of the CUDA/Python/arch versions that we claim to support for dask-cudf and cudf-polars wheels.
    
    In addition, this PR ensures that we do not attempt to run the dbgen executable in the Polars repository on systems with too old of a glibc to support running them.
    
    Authors:
      - Vyas Ramasubramani (https://github.com/vyasr)
    
    Approvers:
      - Bradley Dice (https://github.com/bdice)
    
    URL: #17320
    vyasr authored Nov 18, 2024
    Configuration menu
    Copy the full SHA
    d514517 View commit details
    Browse the repository at this point in the history
  5. Fix reading Parquet string cols when nrows and input_pass_limit >…

    … 0 (#17321)
    
    This PR fixes reading string columns in Parquet using chunked parquet reader when `nrows` and `input_pass_limit` are > 0.
    
    Closes #17311
    
    Authors:
      - Muhammad Haseeb (https://github.com/mhaseeb123)
    
    Approvers:
      - Vukasin Milovanovic (https://github.com/vuule)
      - Ed Seidl (https://github.com/etseidl)
      - Lawrence Mitchell (https://github.com/wence-)
      - Bradley Dice (https://github.com/bdice)
      - https://github.com/nvdbaranec
      - GALI PREM SAGAR (https://github.com/galipremsagar)
    
    URL: #17321
    mhaseeb123 authored Nov 18, 2024
    Configuration menu
    Copy the full SHA
    43f2f68 View commit details
    Browse the repository at this point in the history
  6. Configuration menu
    Copy the full SHA
    18b40dc View commit details
    Browse the repository at this point in the history
  7. Configuration menu
    Copy the full SHA
    ba21673 View commit details
    Browse the repository at this point in the history
  8. Configuration menu
    Copy the full SHA
    02c35bf View commit details
    Browse the repository at this point in the history
  9. Configuration menu
    Copy the full SHA
    302e625 View commit details
    Browse the repository at this point in the history

Commits on Nov 19, 2024

  1. Support polars 1.14 (#17355)

    1.13 was yanked for some reason, but 1.14 doesn't bring anything new and difficult.
    
    Authors:
      - Lawrence Mitchell (https://github.com/wence-)
      - GALI PREM SAGAR (https://github.com/galipremsagar)
    
    Approvers:
      - Vyas Ramasubramani (https://github.com/vyasr)
      - https://github.com/brandon-b-miller
      - GALI PREM SAGAR (https://github.com/galipremsagar)
    
    URL: #17355
    wence- authored Nov 19, 2024
    Configuration menu
    Copy the full SHA
    5f9a97f View commit details
    Browse the repository at this point in the history
  2. Writing compressed output using JSON writer (#17323)

    Depends on #17161 for implementations of compression and decompression functions (`io/comp/comp.cu`, `io/comp/comp.hpp`, `io/comp/io_uncomp.hpp` and `io/comp/uncomp.cpp`)
    
    Adds support for writing GZIP- and SNAPPY-compressed JSON to the JSON writer.
    Verifies correctness using a parameterized test in `tests/io/json/json_writer.cpp`
    
    Authors:
      - Shruti Shivakumar (https://github.com/shrshi)
      - Vukasin Milovanovic (https://github.com/vuule)
    
    Approvers:
      - Kyle Edwards (https://github.com/KyleFromNVIDIA)
      - Karthikeyan (https://github.com/karthikeyann)
      - Vukasin Milovanovic (https://github.com/vuule)
    
    URL: #17323
    shrshi authored Nov 19, 2024
    Configuration menu
    Copy the full SHA
    384abae View commit details
    Browse the repository at this point in the history