RFC: add a unified inspection API namespace

[kgryte]

This RFC is intended to supersede #635, #636, #637, and #638, based on initial consortium feedback, and proposes to consolidate inspection APIs into a single non-public namespace.

Proposal

info = xp.__array_namespace_info__()

Returns a namespace with Array API namespace inspection utilities.

Note: the x.__array_namespace__() API supports a version kwarg. We don't follow suit here, as we assume backward compatibility, whereby a consumer may ask for a particular version of the array API standard and a conforming library may return a newer version. That version should still be compliant with the older specified version (backward compat guarantee), and thus, the inspection API return values should still be applicable. Stated more succinctly: conforming array libraries are expected to be "evergreen" and only have a single Array API implementation, not separate implementations for each revision of the standard.

Methods

info.capabilities() -> dict[str,bool]

info.capabilities() -> dict[str,bool]

Returns a dictionary of static "capabilities", such as whether a conforming library supports boolean indexing or data-dependent output shapes.

Key values should be either False (does not support) or True (does support). For example,

{
    'boolean_indexing': False | True,
    'data_dependent_shapes': False | True
}

info.default_device() -> device

info.default_device()

Returns an object for the default device.

info.default_dtypes( /, *, device: device) -> dict[str, dtype]

info.default_dtypes() -> dict[str, dtype]
info.default_dtypes(device: device) -> dict[str, dtype]

Returns a dictionary having the following keys:

• real floating: default real floating-point dtype.
• complex floating: default complex floating-point dtype.
• integral: default integral dtype.
• indexing: default index dtype.

>>> info.default_dtypes()
{
    'real floating': <dtype>,
    'complex floating': <dtype>,
    'integral': <dtype>,
    'indexing': <dtype>
}
>>> info.default_dtypes(device=<device>)
{
    'real floating': <dtype>,
    'complex floating': <dtype>,
    'integral': <dtype>,
    'indexing': <dtype>
}

More keys could be added in the future, depending on evolution of the standard.

If not provided a device argument, the function would return a dictionary based on the current device (context). If provided a device argument, the function would return a dictionary as described above, but specific to the specified device.

info.dtypes( /, *, device: device, kind: Union[str, Tuple[str, ...]]) -> dict[str,dtype]

info.dtypes() -> dict[str,dtype]
info.dtypes(device: device) -> dict[str,dtype]
info.dtypes(kind: Union[dtype, str, Tuple[Union[dtype, str], ...]]) -> dict[str,dtype]
info.dtypes(kind: Union[dtype, str, Tuple[Union[dtype, str], ...]], device: device) -> dict[str,dtype]

Returns a dictionary of supported Array API data types (in canonical form; e.g., float64, float32, int32, etc).

>>> info.dtypes()
{"float64": xp.float64, "float32": xp.float32, "int32": xp.int32, ...}

Similar to the isdtype API, the function would support a "kind" kwarg for returning a dictionary of supported Array API data types belonging to the specified kind. The following kinds should be supported, matching the isdtype API:

• 'bool': boolean data types (e.g., bool).
• 'signed integer': signed integer data types (e.g., int8, int16, int32, int64).
• 'unsigned integer': unsigned integer data types (e.g., uint8, uint16, uint32, uint64).
• 'integral': integer data types. Shorthand for ('signed integer', 'unsigned integer').
• 'real floating': real-valued floating-point data types (e.g., float32, float64).
• 'complex floating': complex floating-point data types (e.g., complex64, complex128).
• 'numeric': numeric data types. Shorthand for ('integral', 'real floating', 'complex floating').

>>> info.dtypes(kind="real floating")
{"float64": xp.float64, "float32": xp.float32}

If not provided a device argument, the function should return a dictionary of supported data types for the current device (context). If provided a device argument, the function should return a dictionary of supported data types for the specified device.

>>> info.dtypes(device=<device>)
{"float32": xp.float32, "int32": xp.int32, "int8": xp.int8, "uint8": xp.uint8, ...}
>>> info.dtypes(kind="real floating", device=<device>)
{"float32": xp.float32}

Note: a hasattr check is not sufficient as a means of determining dtype support, as dtype support can vary depending on the device. A conforming array library may be fully compliant for its default device, but unable to be fully compliant on other supported devices. In which case, simply checking whether a dtype object exists in the main namespace is insufficient to determine whether a downstream consumer can actually use a specific dtype object for a particular device/execution context.

info.devices()

info.devices() -> List[device]

Returns a list of supported devices.

Related Links

• ENH: Should we allow querying certain implementation details #499

[rgommers]

This looks quite good to me, thanks @kgryte.

Attribute values would be either 0 (does not support) or 1 (does support).

Minor: True/False please, rather than 0/1.

Returns a dictionary having the following keys: real_floating_point: ...

Minor: probably better if those keys exactly match the string names used by isdtype.

[kgryte]

Minor: True/False please, rather than 0/1.

Updated in OP.

Minor: probably better if those keys exactly match the string names used by isdtype.

Agreed. I had a momentary lapse and got my programming languages mixed up. Updated.

[kgryte]

During the previous workgroup meeting focusing on the array API standard (15 June 2023), no objections were raised to this RFC and adding the proposed namespace was greenlit to move forward for inclusion in the 2023 revision of the array API specification.

As #635, #636, #637, and #638 are superseded by this RFC, they are no longer applicable and have been closed.

[betatim]

Should we (as in I'd very much like that) add a statement about how dtypes should be ordered?

In particular should it be:

>>> info.dtypes(kind="real floating")
[xp.float64, xp.float32]

or

>>> info.dtypes(kind="real floating")
[xp.float32, xp.float64]

and similarly when you have a mix of integers and floats.

The use case for having a specified ordering would be being able to answer questions like "is there a float64 and if not, what is the next highest precision type?"

[rgommers]

A few points we discussed last week:

• this should be supported something; in practice the main case of interest is "if float64 is not enabled, give me float32".
• there doesn't seem to be a real-world situation where float32 is also missing; it's not impossible that that appears but it's not clear that silently getting float16 as the most precise floating-point dtype is useful (an exception is likely better)
• ordering may be a little tricky, not everything may be ordered (e.g., float16 vs. bfloat16 in the future, or if you ask about both signed and unsigned integers)
• it's tricky to query the returned dtypes directly. Nothing can be assumed about the dtypes (they're opaque objects) and if, e.g., if float64 exists in the library but not on a particular device, then hasattr(xp, 'float64') doesn't help).
• hence, returned a dict with string names ({'float64': xp.float64, ...}) may be more useful. that way one can both iterate over all supported dtypes with the dtype objects in the dict values, and introspect the available names via the dict keys