[RELEASE] cuspatial v21.08 #439
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Fix merge conflicts [skip ci]
Bumps the version of cuSpatial in CMakeLists.txt. Authors: - Mark Harris (https://github.com/harrism) Approvers: - H. Thomson Comer (https://github.com/thomcom) URL: #410
# Summary Add support for space in workspace Authors: - Joseph (https://github.com/jolorunyomi) - Paul Taylor (https://github.com/trxcllnt) Approvers: - AJ Schmidt (https://github.com/ajschmidt8) - Paul Taylor (https://github.com/trxcllnt) URL: #385
[gpuCI] Forward-merge branch-21.06 to branch-21.08 [skip ci]
Fixes #319 Authors: - H. Thomson Comer (https://github.com/thomcom) Approvers: - Paul Taylor (https://github.com/trxcllnt) URL: #411
Fix `21.08` forward-merge conflicts
According to [this issue](sphinx-doc/sphinx#7747), `add_stylesheet` has been replaced with `add_css_file`. This has caused some errors in recent doc builds. This PR replaces `add_stylesheet` with `add_css_file`. Authors: - AJ Schmidt (https://github.com/ajschmidt8) Approvers: - H. Thomson Comer (https://github.com/thomcom) URL: #421
`parallel_search` was producing the wrong indexes for deciding which coefficients to use for interpolation. This was hidden by the test cases, but exposed via python as I was researching another bug. This PR resolves the issue with parallel_search so that it produces the correct indexes for interpolation. I'm improving the testing a bit more before asking for reviews. Authors: - H. Thomson Comer (https://github.com/thomcom) - Paul Taylor (https://github.com/trxcllnt) Approvers: - Mark Harris (https://github.com/harrism) - Paul Taylor (https://github.com/trxcllnt) URL: #405
…ing (#430) This PR contains three distinct changes required to get cuspatial builds working and tests passing again: 1. RMM switched to rapids-cmake (rapidsai/rmm#800), which requires CMake 3.20.1, so this PR includes the required updates for that. 2. The Arrow upgrade in cudf also moved the location of testing utilities (rapidsai/cudf#7495). Long term cuspatial needs to move away from use of the testing utilities, which are not part of cudf's public API, but we are currently blocked by rapidsai/cudf#8646, so this PR just imports the internal `assert_eq` method as a stopgap to get tests passing. 3. The changes in rapidsai/cudf#8373 altered the way that metadata was propagated to libcudf outputs from previously existing cuDF Python objects. The new code paths require cuspatial to override metadata copying at the GeoDataFrame rather than the GeoColumn level in order to ensure that information about column types is lost in the libcudf round trip and the metadata copying functions are now called on the output DataFrame rather than the input one. This PR supersedes #427, #428, and #429, all of which can now be closed. Authors: - Vyas Ramasubramani (https://github.com/vyasr) - GALI PREM SAGAR (https://github.com/galipremsagar) Approvers: - AJ Schmidt (https://github.com/ajschmidt8) - Christopher Harris (https://github.com/cwharris) URL: #430
Fixes #393 We switched to the exclusive scan approach to Hausdorff because certain benchmarks indicated better performance. Apparently those benchmarks were inadequate or just plain badly written (by me), and performance was in fact worse. This became apparent while fixing the OOM error reported in #393. I copied the 0.14 implementation in to the 21.08 branch to re-benchmark. here are the results: cuspatial@0.14: ``` ------------------------------------------------------------------------------------------------------------ Benchmark Time CPU Iterations UserCounters... ------------------------------------------------------------------------------------------------------------ HausdorffBenchmark/hausdorff/100/64/manual_time 1.62 ms 1.78 ms 428 items_per_second=23.9898G/s HausdorffBenchmark/hausdorff/512/64/manual_time 43.9 ms 44.1 ms 16 items_per_second=23.6053G/s HausdorffBenchmark/hausdorff/4096/64/manual_time 2810 ms 2810 ms 1 items_per_second=23.6845G/s HausdorffBenchmark/hausdorff/6000/64/manual_time 6148 ms 6148 ms 1 items_per_second=23.2318G/s HausdorffBenchmark/hausdorff/100/100/manual_time 3.31 ms 3.47 ms 210 items_per_second=29.0333G/s HausdorffBenchmark/hausdorff/512/100/manual_time 88.9 ms 89.1 ms 8 items_per_second=28.7737G/s HausdorffBenchmark/hausdorff/4096/100/manual_time 5842 ms 5842 ms 1 items_per_second=28.132G/s HausdorffBenchmark/hausdorff/6000/100/manual_time 12698 ms 12698 ms 1 items_per_second=27.7783G/s ``` cuspatial@21.08 (with fix for OOM, as seen in previous commits of this PR) ``` ------------------------------------------------------------------------------------------------------------ Benchmark Time CPU Iterations UserCounters... ------------------------------------------------------------------------------------------------------------ HausdorffBenchmark/hausdorff/100/64/manual_time 17.4 ms 17.6 ms 38 items_per_second=2.2391G/s HausdorffBenchmark/hausdorff/512/64/manual_time 489 ms 490 ms 2 items_per_second=2.11979G/s HausdorffBenchmark/hausdorff/4096/64/manual_time 37120 ms 37119 ms 1 items_per_second=1.79299G/s HausdorffBenchmark/hausdorff/6000/64/manual_time 82732 ms 82729 ms 1 items_per_second=1.7265G/s HausdorffBenchmark/hausdorff/100/100/manual_time 43.4 ms 43.7 ms 16 items_per_second=2.21402G/s HausdorffBenchmark/hausdorff/512/100/manual_time 1341 ms 1341 ms 1 items_per_second=1.90885G/s HausdorffBenchmark/hausdorff/4096/100/manual_time 94898 ms 94894 ms 1 items_per_second=1.7319G/s HausdorffBenchmark/hausdorff/6000/100/manual_time 199120 ms 199115 ms 1 items_per_second=1.77138G/s ``` The performance is bad, and this regression is my fault. Fortunately I was able to quickly reverse this regression and improve performance while getting rid of a bunch of code (and learning a lot in the process). This PR re-implements Hausdorff as a straightforward custom kernel that requires zero intermediate memory. this pr: ``` ------------------------------------------------------------------------------------------------------------ Benchmark Time CPU Iterations UserCounters... ------------------------------------------------------------------------------------------------------------ HausdorffBenchmark/hausdorff/100/64/manual_time 1.31 ms 1.47 ms 526 items_per_second=29.6763G/s HausdorffBenchmark/hausdorff/512/64/manual_time 23.2 ms 23.3 ms 30 items_per_second=44.7567G/s HausdorffBenchmark/hausdorff/4096/64/manual_time 1589 ms 1590 ms 1 items_per_second=41.8747G/s HausdorffBenchmark/hausdorff/6000/64/manual_time 3170 ms 3170 ms 1 items_per_second=45.0638G/s HausdorffBenchmark/hausdorff/100/100/manual_time 2.92 ms 3.08 ms 239 items_per_second=32.8852G/s HausdorffBenchmark/hausdorff/512/100/manual_time 55.8 ms 55.8 ms 12 items_per_second=45.8415G/s HausdorffBenchmark/hausdorff/4096/100/manual_time 3547 ms 3547 ms 1 items_per_second=46.3317G/s HausdorffBenchmark/hausdorff/6000/100/manual_time 7658 ms 7658 ms 1 items_per_second=46.0564G/s ``` Authors: - Christopher Harris (https://github.com/cwharris) Approvers: - Mark Harris (https://github.com/harrism) - Paul Taylor (https://github.com/trxcllnt) - AJ Schmidt (https://github.com/ajschmidt8) URL: #424
With rapidsai/integration#286, the version of `isort` running on gpuCI will be bumped to 5.6.4, allowing us to enforce the sorting of packages in Cython (pyx, pxd) files. This PR intends to: - Enable these checks in the gpuCI style script - Enable Cython package resorting in the pre-commit hook - Resort all the Cython files in this repo so they pass the newly enabled checks These checks are optional, meaning that even without this being merged, gpuCI should still pass on style checks even when rapidsai/integration#286 is merged. Authors: - Charles Blackmon-Luca (https://github.com/charlesbluca) Approvers: - H. Thomson Comer (https://github.com/thomcom) - AJ Schmidt (https://github.com/ajschmidt8) URL: #419
Follow up PR to: rapidsai/cudf#6695. Performing the same changes but for `rapidsai/cuspatial` Depends on: rapidsai/integration#304 Authors: - Conor Hoekstra (https://github.com/codereport) Approvers: - Dillon Cullinan (https://github.com/dillon-cullinan) - H. Thomson Comer (https://github.com/thomcom) URL: #426
rapidsai/cudf#8666 modified `cudf::test` APIs to accept a verbosity enum as a parameter to control output, which is backwards incompatible with the previously boolean parameter. Authors: - Vyas Ramasubramani (https://github.com/vyasr) Approvers: - Mark Harris (https://github.com/harrism) - Paul Taylor (https://github.com/trxcllnt) URL: #433
As of rapidsai/cudf#7495 the `cudf.tests.utils` module (and in particular the `assert_eq` function) are no longer part of the public API. This PR switches tests to use the public testing functions in the `cudf.testing` subpackage. This PR is currently blocked by #430 and rapidsai/cudf#8646. Authors: - Vyas Ramasubramani (https://github.com/vyasr) Approvers: - Christopher Harris (https://github.com/cwharris) - Paul Taylor (https://github.com/trxcllnt) URL: #431
github-actions
bot
added
cmake
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
❄️ Code freeze for
branch-21.08and v21.08 release