Remove GeoArrow glue code replacing gpu storage with cudf.Series and host storage with pyarrow #585
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Check out this pull request on See visual diffs & provide feedback on Jupyter Notebooks. Powered by ReviewNB |
github-actions
bot
added
Python
6 tasks
thomcom
added
bug
github-actions
bot
added
cmake
This was referenced Jul 15, 2022
|
All comments are handle. In the last push I added |
|
I think it also closes #592 ? |
|
So is this ready to merge? Good work @thomcom and thanks! |
trxcllnt
approved these changes
Jul 28, 2022
|
@gpucibot merge |
ajschmidt8
added a commit
to ajschmidt8/cuxfilter
that referenced
this pull request
Aug 3, 2022
Similarly to rapidsai/cuspatial#585, this PR updates the `geopandas` version specifier to ensure compatibility with the rest of RAPIDS.
rapids-bot bot
pushed a commit
to rapidsai/cuxfilter
that referenced
this pull request
Aug 3, 2022
Similarly to rapidsai/cuspatial#585, this PR updates the `geopandas` version specifier to ensure compatibility with the rest of RAPIDS. Authors: - AJ Schmidt (https://github.com/ajschmidt8) Approvers: - Jordan Jacobelli (https://github.com/Ethyling) URL: #390
rapids-bot bot
pushed a commit
that referenced
this pull request
Sep 8, 2022
This PR introduces 3 documents for developer guides. The overall goal for these docs is to provide a place to centralize best practices and provide guidance to developers of all levels to bootstrap development for cuspatial. This PR deprecates internals.md. The design of geocolumn has improved dramatically since #585 , rendering many of the description obsolete. Further, examples of geoarrow and detail description of geoarrow should be hosted in [geoarrow specification](https://github.com/geopandas/geo-arrow-spec). `library_design.md` replaces `internal.md`. `CONTRIBUTING.md` is deprecated and only leaves a pointer to `contributing_guide.md`. The contents are highly identical to the overall RAPIDS contribution guide: https://docs.rapids.ai/contributing. This link is included in the contributing guide and is thus removed to maintain a single source of truth. Authors: - Michael Wang (https://github.com/isVoid) Approvers: - Paul Taylor (https://github.com/trxcllnt) - H. Thomson Comer (https://github.com/thomcom) - Mark Harris (https://github.com/harrism) URL: #625
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR: Closes #575, #584, #588, #591, and #592.
This PR removes all the old
GeoArrowBufferslogic that was implementing GeoArrow from scratch.Now, new geometry objects are first parsed into a
pyarrowDenseUnion, then copied to the GPU asListSeries. ADenseUnion/LikeclassGeoSeriesimplements theinput_typesandunion_offsetsfor indexing into the four separateListSeries.When host indexing, the
ListColumnsandtypesandoffsetsare copied into a pyarrowDenseUnionthat is used for accessing the host side coordinates. At present indexing always copies to host.This deletes three times as many loc as it creates and trims down the GeoArrow code to nearly the minimum. It is ready for review.
The next issue will describe the next four tasks that I or @isVoid will implement:
dataarguments types that can be made to theGeoSeriesconstructor.x = cuspatial.from_arrow(dense_union); dense_union = x.to_arrow()__getitem__method needs to be optimized for zero-copy and memory-direct buffer transfers. Presently it always copies the data into new Shapely objects for serialization.This closes the original issue #575. I'll create another issue to validate that the new format is correct and notify/compare it with the existing error report and geopandas team.
@trxcllnt @exactlyallan @isVoid @harrism