-
Notifications
You must be signed in to change notification settings - Fork 200
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add complete set of resource ref aliases #1479
Add complete set of resource ref aliases #1479
Conversation
…:mr::resource_ref and cudf::mr::async_resource_ref, respectively.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Only requesting changes so we can discuss question below.
* @brief Alias for a `cuda::mr::async_resource_ref` with the property | ||
* `cuda::mr::host_accessible`. | ||
*/ | ||
using host_async_resource_ref = cuda::mr::async_resource_ref<cuda::mr::host_accessible>; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think it will be common for users of async_resource_ref
for pinned host memory to also want device_accessible
. (I expect you will want it in your cuDF usage eventually too...) My question is, do we want to just add cuda::mr::device_accessible
to this alias, or add another one:
using host_device_async_resource_ref =
cuda::mr::async_resource_ref<cuda::mr::host_accessible, cuda::mr::device_accessible>;
Thoughts @nvdbaranec @Missco @jrhemstad ?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I would not add device_accessible
to the host_async_resource_ref
as that would all potential memory resource to satisfy device_accessible
On the other hand an async resource has no real value without device_accessible
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good point. However I think an async resource ref that is host-only has value because it specifies that it will only be used on the host, but it may still be backed by pinned memory. And then it benefits from stream-ordered allocation (e.g. can be used in a stream-ordered pool) and also can be used as the host memory for SOL cudaMemcpyAsync
.
So really for completeness I think we probably need the full set:
using host_resource_ref = cuda::mr::resource_ref<cuda::mr::host_accessible>;
using device_resource_ref = cuda::mr::resource_ref<cuda::mr::device_accessible>;
using host_device_resource_ref = cuda::mr::resource_ref<cuda::mr::host_accessible, cuda::mr::device_accessible>;
using host_async_resource_ref = cuda::mr::async_resource_ref<cuda::mr::host_accessible>;
using device_async_resource_ref = cuda::mr::async_resource_ref<cuda::mr::device_accessible>;
using host_device_async_resource_ref = cuda::mr::async_resource_ref<cuda::mr::host_accessible, cuda::mr::device_accessible>;
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The use case for this right now in cudf is strictly in the hostdevice_vector
class, whose sole purpose is to force explicit host<->device copying onto the user. That is: entirely avoiding the device touching pinned memory directly. So in that sense, stream ordering at the allocator level isn't necessary.
As for whether cudf at large will ever need this, I'm not sure. At the moment it definitely prefers to be explicit about all host<-> device handoffs.
Opened #1480 to cover this work. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you @nvdbaranec
/merge |
Adds new aliases to
resource_ref.hpp
using host_resource_ref = cuda::mr::resource_ref<cuda::mr::host_accessible>;
using host_async_resource_ref = cuda::mr::async_resource_ref<cuda::mr::host_accessible>;
Checklist