diff --git a/docs/source/format/CDeviceDataInterface.rst b/docs/source/format/CDeviceDataInterface.rst
index b5b7229a679e1..dd8b7e98e1cba 100644
--- a/docs/source/format/CDeviceDataInterface.rst
+++ b/docs/source/format/CDeviceDataInterface.rst
@@ -256,6 +256,10 @@ has the following fields:
     type are on the system. The semantics of the id will be hardware dependent,
     but we use an ``int64_t`` to future-proof the id as devices change over time.
 
+    For device types that do not have an intrinsic notion of a device identifier (e.g.,
+    ``ARROW_DEVICE_CPU``), it is recommended to use a ``device_id`` of -1 as a
+    convention.
+
 .. c:member:: ArrowDeviceType ArrowDeviceArray.device_type
 
     *Mandatory.* The type of the device which can access the buffers in the array.
@@ -689,4 +693,4 @@ Any incompatible changes should be part of a new specification, for example
 .. _Vulkan: https://www.vulkan.org/
 .. _Metal: https://developer.apple.com/metal/
 .. _ROCm: https://www.amd.com/en/graphics/servers-solutions-rocm
-.. _oneAPI: https://www.intel.com/content/www/us/en/developer/tools/oneapi/overview.html
\ No newline at end of file
+.. _oneAPI: https://www.intel.com/content/www/us/en/developer/tools/oneapi/overview.html