[BUG] deprecation warning on rmm::mr::device_memory_resource::supports_streams

[abellina]

This is something I saw in my local build of spark-rapids-jni and think we should adjust SparkResourceAdaptorJni to use the new async_memory_resource concept:

INFO]      [exec] [15/36] Building CXX object CMakeFiles/spark_rapids_jni.dir/src/SparkResourceAdaptorJni.cpp.o
[INFO]      [exec] spark-rapids-jni/src/main/cpp/src/SparkResourceAdaptorJni.cpp: In member function 'virtual bool {anonymous}::spark_resource_adaptor::supports_streams() const':
[INFO]      [exec] spark-rapids-jni/src/main/cpp/src/SparkResourceAdaptorJni.cpp:404:86: warning: 'virtual bool rmm::mr::device_memory_resource::supports_streams() const' is deprecated: Functionality removed in favor of cuda::mr::async_memory_resource. [-Wdeprecated-declarations]
[INFO]      [exec]   404 |   bool supports_streams() const noexcept override { return resource->supports_streams(); }
[INFO]      [exec]       |                                                            ~~~~~~~~~~~~~~~~~~~~~~~~~~^~
[INFO]      [exec] In file included from spark-rapids-jni/src/main/cpp/src/SparkResourceAdaptorJni.cpp:17:
[INFO]      [exec] spark-rapids-jni/target/libcudf-install/include/rmm/mr/device/device_memory_resource.hpp:306:3: note: declared here
[INFO]      [exec]   306 |   supports_streams() const noexcept
[INFO]      [exec]       |   ^~~~~~~~~~~~~~~~

[mythrocks]

Seeking clarity on this issue. There seem to be a couple of tasks implied here:

1. Addressing the deprecation warning: libcudf has addressed this by removing the override. (See https://github.com/rapidsai/cudf/pull/14857/files.) This needs addressing before supports_streams() is removed in 24.04.
2. Reimplement with async_memory_resource.

Is the intention for both to be addressed here? (I'm assuming yes, for now.)

[mythrocks]

#1847 should resolve the deprecation warning.

2. Reimplement with async_memory_resource.

Actually, I'm not fully clear on this point. asynch_memory_resource is final, so we're not asking for spark_memory_resource to extend it.

Is the suggestion for spark_memory_resource to take an async_memory_resource explicitly? It doesn't currently use much beyond what's in the device_memory_resource interface, so I'm guessing not.

If Rmm.getCurrentDeviceResource() were to return an async_memory_resource, wouldn't spark_memory_resource work unchanged?

[abellina]

Sorry for the confusion, the deprecation is what I meant. I was confused (still am a bit) on the async_memory_resource concepts, and misunderstood when I filed this whether there was a type hierarchy, which turns out to be false.

[mythrocks]

Thank you for clarifying, @abellina.