Support memory pools (introduced in CUDA 11.2)

[eyalroz]

CUDA 11.2 introduced a "memory pool" mechanism; we should support it:

Full API documentation here.

​cudaError_t cudaMallocFromPoolAsync ( void** ptr, size_t size, cudaMemPool_t memPool, cudaStream_t stream )
​cudaError_t cudaMemPoolCreate ( cudaMemPool_t* memPool, const cudaMemPoolProps* poolProps )
​cudaError_t cudaMemPoolDestroy ( cudaMemPool_t memPool )
​cudaError_t cudaMemPoolExportPointer ( cudaMemPoolPtrExportData* exportData, void* ptr )
​cudaError_t cudaMemPoolExportToShareableHandle ( void* shareableHandle, cudaMemPool_t memPool, cudaMemAllocationHandleType handleType, unsigned int  flags )
​cudaError_t cudaMemPoolGetAccess ( enum cudaMemAccessFlags* flags, cudaMemPool_t memPool, cudaMemLocation* location )
​cudaError_t cudaMemPoolGetAttribute ( cudaMemPool_t memPool, cudaMemPoolAttr attr, void* value )
​cudaError_t cudaMemPoolImportFromShareableHandle ( cudaMemPool_t* memPool, void* shareableHandle, cudaMemAllocationHandleType handleType, unsigned int  flags )
​cudaError_t cudaMemPoolImportPointer ( void** ptr, cudaMemPool_t memPool, cudaMemPoolPtrExportData* exportData )
​cudaError_t cudaMemPoolSetAccess ( cudaMemPool_t memPool, const cudaMemAccessDesc* descList, size_t count )
cudaError_t cudaMemPoolSetAttribute ( cudaMemPool_t memPool, cudaMemPoolAttr attr, void* value )
​cudaError_t cudaMemPoolTrimTo ( cudaMemPool_t memPool, size_t minBytesToKeep )

[eyalroz]

So, what would this entail?:

• A cuda::memory::pool_t class (perhaps cuda::memory::device::pool_t?)
• atttribute_t and attributes_t inner classes of pool_t
• Possible changes to:
  • pointer class
  • the peer_to_peer namespace
  • Another stream_t::enqueue_t method (override of allocate perhaps?)
  • Some more freestanding functions in cuda::memory::device? Perhaps only within the detail_ namespace?

[eyalroz]

Note to self: Remember to add the CUDA-graph-with-memory-node example when implementing this!