[FEA] RMM Pool by default and ease of use

[jrhemstad]

Is your feature request related to a problem? Please describe.
It would be nice if RMM pool mode was enabled by default.

Furthermore, the steps required to enable/disabled pool mode are a bit cumbersome.

from librmm_cffi import librmm_config as rmm_cfg

rmm_cfg.use_pool_allocator = True

Describe the solution you'd like
Pool mode should be enabled by default.

It should be easier to configure RMM, something like what cuPY has:

cupy.cuda.set_allocator(allocator=None)

[jrhemstad]

Note that this should tie into the new APIs in RMM for setting the default memory resource: https://github.com/rapidsai/rmm/blob/branch-0.10/include/rmm/mr/default_memory_resource.hpp#L51

[kkraus14]

I'd be in support of this as long as we limit the size of the initial pool to be 0 / minimal so as to avoid blocking out other libraries unnecessarily. I think it would be worth the risk of fragmentation / small performance loss in this respect.

[jrhemstad]

I think it would be worth the risk of fragmentation / small performance loss in this respect.

@harrism and I just discussed, and when RMM has a pool size of 0 1 byte, it effectively turns into a caching sub-allocator. It can reuse previous allocations that were allocated with cudaMalloc. However, as it is now, there is still significant risk of fragmentation. There is no way to "clean up" unused allocations.

See rapidsai/rmm#75 for an example of this fragmentation causing OOMs.

[harrism]

BTW, setting the RMM pool size argument to 0 has a semantic meaning other than "zero bytes" -- specifically, it sets the initial pool to 1/2 gpu total memory size. There is no way to set it to zero bytes.

[jrhemstad]

Closed by #2682