You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
For #41665 (implemented for Array in #42112 / #42113), we currently use the following assertion to check if the data is on CPU (and thus supports the operation in question that access the data's address):
raiseNotImplementedError("Implemented only for data on CPU device")
This checks explicitly for the CPU device allocation type.
However, this means that for example data with a CUDA_HOST device type, which is actually accessible from the CPU, will trigger this error:
importnumpyasnpimportpyarrowaspafrompyarrowimportcuda# create Array with CudaHost bufferbuf=cuda.new_host_buffer(5*8)
np.frombuffer(buf, dtype=np.int64)[:] =range(5)
arr=pa.Array.from_buffers(pa.int64(), size, [None, buf])
# inspect the array>>>arr<pyarrow.lib.Int64Arrayobjectat0x7f24b6e02e00>
[
0,
1,
2,
3,
4
]
>>>arr.device_type<DeviceAllocationType.CUDA_HOST: 3># calling a method that checks _assert_cpu errors>>>arr.sum()
...
NotImplementedError: ImplementedonlyfordataonCPUdevice# but the underlying buffer itself "is_cpu">>>arr.buffers()[1]
<pyarrow.Bufferaddress=0x7f24c1600400size=80is_cpu=Trueis_mutable=True>>>>arr.buffers()[1].is_cpuTrue>>>arr.buffers()[1].device_type<DeviceAllocationType.CUDA_HOST: 3>
At the buffer level we have this is_cpu attribute available, but currently on the Array level we only have device_type(). We could add CUDA_HOST device allocation type explicitly to the check above, but ideally we would use something more general?
For #41665 (implemented for Array in #42112 / #42113), we currently use the following assertion to check if the data is on CPU (and thus supports the operation in question that access the data's address):
arrow/python/pyarrow/array.pxi
Lines 2035 to 2037 in d4d92e4
This checks explicitly for the CPU device allocation type.
However, this means that for example data with a CUDA_HOST device type, which is actually accessible from the CPU, will trigger this error:
At the buffer level we have this
is_cpu
attribute available, but currently on the Array level we only havedevice_type()
. We could add CUDA_HOST device allocation type explicitly to the check above, but ideally we would use something more general?(cc @danepitkin)
The text was updated successfully, but these errors were encountered: