Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

RFC: Adding complex number support to the specification #373

Closed
kgryte opened this issue Jan 20, 2022 · 5 comments
Closed

RFC: Adding complex number support to the specification #373

kgryte opened this issue Jan 20, 2022 · 5 comments
Labels
API extension Adds new functions or objects to the API. topic: Complex Data Types Complex number data types.
Milestone
v2022

Comments

@kgryte
Copy link
Contributor

kgryte commented Jan 20, 2022 •
edited
Loading

Complex Number Support

Plan for complex number support in the array API specification.

What follows is a plan for adding complex number support to the 2022 array API specification. This RFC is comprised of the following sections:

  • Prior discussions
  • High-level summary concerns and questions
  • Individual API changes

Prior Discussions

General Concerns

New APIs

real
conj (complex conjugate)
imag
arg/angle/phase (phase angle)
linalg.eig
linalg.eigvals

Decisions

  • initial data types: complex64 and complex128?

    • Update: only complex64 and complex128 to be added, thus mirroring float32 and float64.

  • rounding complex numbers (component-wise)

    • NumPy supports round, but not ceil, floor, and trunc.
    • Update: ceiling, flooring, and truncating complex numbers is not well-defined. The only rounding operation which is commonly supported is round.

  • ordering (lexicographic?)

  • branch cut policy?

    • branch cut reference: https://en.wikipedia.org/wiki/Branch_point#Branch_cuts
    • should we strictly follow NumPy/C99, or should we allow for implementation flexibility?
    • if we strictly follow something other than NumPy/C99, obvious backward compat concern
    • Update: branch cuts will be included in the 2022 revision under a provisional status. Provided no issues arise during adoption and implementation, the provisional status may be lifted in a future revision of the standard.

  • casting from complex to real

    • discard imaginary component (as in C, NumPy)
    • other choice would to require user to explicitly invoke real before casting
    • Update: decision was to require that array API consumers be explicit in specifying which component they want to cast when casting from complex to real.

  • complex numbers with components which are infinity and/or NaN

    • in Python, a complex number can be both infinite and NaN (according to cmath.isinf, cmath.isnan)

      In [1]: z = complex(float('inf'), float('nan'))

In [2]: z
Out[2]: (inf+nanj)

In [3]: cmath.isinf(z)
Out[3]: True

In [4]: cmath.isnan(z)
Out[4]: True

    • in NumPy (v1.23.5), behavior is consistent with Python

      In [1]: z = np.complex128(complex(float('inf'), float('nan')))

In [2]: z
Out[2]: (inf+nanj)

In [3]: np.isnan([z])
Out[3]: array([True])

In [4]: np.isinf([z])
Out[4]: array([True])

    • in C99 (see "complex floating types" section), one infinity model (e.g., inf + nan*j == inf)

      • "In order to support the one-infinity model of complex number arithmetic, C regards any complex value with at least one infinite part as an infinity even if its other part is a NaN..."
      • See C99 rationale 7.3.9.4 and SO

    • update: decision was made to allow implementation-dependent behavior for modeling complex NaNs and infinities. In particular, for complex multiplication and division, handling of complex infinities and NaNs may not be consistent across implementations.

Creation Functions

arange

asarray

empty

  • No changes. Output array data type has no restrictions.

empty_like

  • No changes. Output array data type has no restrictions.

eye

from_dlpack

  • No changes.

full

full_like

linspace

meshgrid

ones

ones_like

tril

  • No changes necessary. No restrictions on input array data types.

triu

  • No changes necessary. No restrictions on input array data types.

zeros

  • No changes necessary. No restrictions on output array data types.

zeros_like

  • No changes necessary. No restrictions on output array data types.

Data Type Functions

astype

  • Add note stating that, when casting from a complex data type to a real data type stating, the imaginary component is discarded and casting to integral data types is unspecified and thus implementation-dependent.

    • Discarding the imaginary component follows C and is done in NumPy.
    • Could also disallow/omit/not explicitly support casting from complex to real and require users to explicitly call real if they want to discard the imaginary component.
    • Update: casting from a complex data type to a real-valued data type was disallowed in the specification in favor of API consumers being explicit wrt which component should be cast.

  • status: implemented

broadcast_arrays

  • No changes necessary.

broadcast_to

  • No changes necessary.

can_cast

  • No changes necessary. Follows promotion rules for complex numbers.

finfo

  • No changes necessary.

iinfo

  • No changes necessary.

result_type

  • No changes necessary. Follows promotion rules for complex numbers.

Data Types

  • Add complex64 and complex128 following precedent where the numeric suffix specifies the number of bits.

    • The real and imaginary components should be IEEE 754 floating-point numbers.

  • status: implemented

Default Data Types

  • The default complex number data type is dependent on the default floating-point data type. If the latter is float32, the default complex data type must be complex64.

    • PyTorch already does this.

  • status: implemented

Data Type Categories

  • Add complex data types to list of numeric data types.

  • Add "Real Data Types" category which should be equal to the current list of numeric data types.

  • Add "Complex Data Types" category which only includes the complex number data types.

  • status: implemented

Element-wise Functions

abs

acos

acosh

add

asin

asinh

atan

atan2

  • No complex number support.
  • No changes necessary as already limited to real number data types.

atanh

bitwise_and

  • No changes. Complex dtypes are not allowed.

bitwise_left_shift

  • No changes. Complex dtypes are not allowed.

bitwise_invert

  • No changes. Complex dtypes are not allowed.

bitwise_or

  • No changes. Complex dtypes are not allowed.

bitwise_right_shift

  • No changes. Complex dtypes are not allowed.

bitwise_xor

  • No changes. Complex dtypes are not allowed.

ceil

  • Add complex number support. Independently round components (e.g., as in MATLAB).

    • NumPy does not support in np.ceil.
    • No complex number support due to implementation ambiguity.

cos

cosh

divide

equal

exp

expm1

floor

  • Add complex number support. Independently round components (e.g., as in MATLAB).

    • NumPy does not support in np.floor.
    • No complex number support due to implementation ambiguity.

floor_divide

  • No complex number support.
  • No changes necessary as already limited to real number data types.

greater

greater_equal

isfinite

isinf

isnan

  • Add support for complex numbers.

    • Care should be taken for infinities and NaNs.
      • how to classify a complex number as NaN?
      • Update: if any component is NaN, the complex number is considered "NaN"

  • status: implemented

less

less_equal

log

log1p

log2

log10

logaddexp

  • No complex number support. This function is mainly useful for stats.
  • No changes necessary as already limited to real-valued data types.

logical_and

  • No changes necessary. Input arrays expected to have boolean data type.

logical_not

  • No changes necessary. Input arrays expected to have boolean data type.

logical_or

  • No changes necessary. Input arrays expected to have boolean data type.

logical_xor

  • No changes necessary. Input arrays expected to have boolean data type.

multiply

negative

not_equal

positive

pow

remainder

  • No complex number support.
  • No changes necessary as already limited to real-valued data types.

round

sign

sin

sinh

square

sqrt

subtract

tan

tanh

trunc

  • Add complex number support. Independently truncate components (e.g., as in MATLAB).

    • NumPy does not support in np.trunc.
    • No complex number support due to implementation ambiguity.

Linear Algebra Functions

matmul

  • Add complex number support.

    • complex number addition is well-defined.
    • complex number multiplication is less well-defined and depends on how an implementation chooses to model complex infinity.

  • status: implemented

matrix_transpose

  • No changes necessary.

tensordot

  • Add complex number support.

    • complex number addition is well-defined.
    • complex number multiplication is less well-defined and depends on how an implementation chooses to model complex infinity.

  • status: implemented

vecdot

Manipulation Functions

concat

  • No changes necessary.

expand_dims

  • No changes necessary.

flip

  • No changes necessary.

permute_dims

  • No changes necessary.

reshape

  • No changes necessary.

roll

  • No changes necessary.

squeeze

  • No changes necessary.

stack

  • No changes necessary.

Searching Functions

argmax

  • Require that complex numbers be ordered in lexicographic order?

    • While not compatible with multiplication, lexicographic order is a common total order. E.g., NumPy uses lexicographic order.
    • Another alternative is by magnitude and then phase angle. E.g., MATLAB supports this order.
    • The specification could default to lexicographic, and revisit the ordering relation in the future with possible support for optionally specifying an alternative comparison method.
    • Some care needs to be given to sorting when one or more components is NaN.
    • Update: no support for complex numbers due to a lack of a natural order.

  • status: implemented

argmin

nonzero

where

  • No changes necessary.

Set Functions

unique_all

  • Specify complex number value equality.

    • In C, "In order to support the one-infinity model of complex number arithmetic, C regards any complex value with at least one infinite part as an infinity even if its other part is a NaN..."
    • Need to resolve isinf and isnan duality
    • Update: base equality comparison on equal.

  • status: implemented

unique_counts

unique_inverse

unique_values

Sorting Functions

argsort

sort

Statistical Functions

max

mean

  • No complex number support.
  • Change to only real number data types.

min

prod

  • Add support for complex numbers. Complex number multiplication is well-defined.

    • The dtype option needs to be updated to accommodate complex numbers.

  • status: implemented

std

  • No complex number support.
  • Change to only real number data types.

sum

  • Add support for complex numbers. Complex number addition is well-defined.

    • The dtype option needs to be updated to accommodate complex numbers.

  • status: implemented

var

  • No complex number support.
  • Change to only real number data types.

Type Promotion

Rules

Mixing arrays with Python Scalars

Utility Functions

all

any

Linear Algebra Extension

cholesky

cross

det (determinant)

diagonal

  • No changes necessary.

eigh

eigvalsh

inv

matmul

  • Support added in main namespace.

matrix_norm

matrix_power

matrix_rank

matrix_transpose

  • No changes necessary.

outer

pinv

qr

slogdet

solve

svd

svdvals

trace

vecdot

  • Changes made in main namespace.

vector_norm
@kgryte kgryte added API extension Adds new functions or objects to the API. topic: Complex Data Types Complex number data types. labels Jan 20, 2022
@kgryte kgryte added this to the v2022 milestone Jan 20, 2022
@honno
Copy link
Member

honno commented Jan 21, 2022 •
edited
Loading

Would we want a data type function for complex dtypes i.e. cinfo()? Currently NumPy and PyTorch don't have a dedicate method, but complex dtypes can fallback on finfo().

Interestingly there was a little discussion of a torch.cinfo() at pytorch/pytorch#35954 (comment)

@kgryte kgryte mentioned this issue Apr 18, 2022
Merged
@leofang
Copy link
Contributor

leofang commented Apr 18, 2022

  • branch cut policy?

I don't know why I missed it... CuPy uses Thrust (and it's likely being used by many other projects that depends on CUDA; IIUC the implementation was ported to libcudacxx) so it's worth mentioning.

Thrust's complex math funcs were based on FreeBSD's. In each FreeBSD man page the branch cut is clearly documented so it's a good starting point. However, as Thrust didn't document it clearly (I asked internally and was told to inspect the source directly, which is challenging given my lack of bandwidth) and it's unclear to me if the Thrust impl was a one-to-one translation of FreeBSD's since I don't have access to the FreeBSD source code, I'd proceed with caution.

@kgryte kgryte mentioned this issue May 5, 2022
Closed
@honno honno mentioned this issue May 16, 2022
Closed
@kgryte
Copy link
Contributor Author

kgryte commented May 23, 2022 •
edited
Loading

Tracking changes to existing APIs within the array API specification:

New APIs

@kgryte kgryte mentioned this issue Jun 22, 2022
Closed
@kgryte
Copy link
Contributor Author

kgryte commented Dec 13, 2022

At this point, PRs have been submitted for all existing APIs for which we currently expect complex number support.

@rgommers
Copy link
Member

All PRs have been merged - adding complex number support to the API is complete. Thanks a lot to everyone who contributed - and in particular @kgryte for all the spec writing.

@rgommers rgommers mentioned this issue Dec 14, 2022
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
API extension Adds new functions or objects to the API. topic: Complex Data Types Complex number data types.
Projects
None yet
Milestone
v2022
Development

No branches or pull requests
4 participants
@rgommers @kgryte @leofang @honno