Add differential_type, to test for non-differentiability

[mcabbott]

This adds a function to check whether a given type is non-differentiable. The purpose is to let you test whether to take the trivial path for some rule.

It goes by whether ProjectTo(x) infers to be a trivial projector. This means it should always be costless, rather than iterating through x = Any[true, false] it will give up and return false.

Possibly this should return something whose type is the indication, like Val(true)? Or return one of AbstractZero / Number / Any, or some trait struct IsNonDiff, seems a bit heavyweight...

[nickrobinson251]

🚲 Naming-wise, i wonder if "differentiable" should be reserved for functions (/callables), like in @non_differentiable f(x, y) @non_differentiable T(x)? Partly because of the existence of the @non_differentiable macro; on naming alone, i'd guessed this was related to @non_differentiable (not so much ProjectTo), i.e. is_non_differentiable(f, args...) if and only if we had @non_differentiable f(args...).

I think "perturb[able]" is a word that's been used in this package when talking about types (e.g. in CRTestUtils here and here). So that could be an option if we wanted to avoid "differentiable" for types.

(That said, a quick search suggests that Switft for TF thought "differentiable types" was a good name for "types that can be used as arguments and results of differentiable functions" https://github.com/tensorflow/swift/blob/f0d6c74ef5d016046afc1eac0b07a2f6b74b8fdf/docs/DifferentiableTypes.md)

Also, i wonder if we want to flip the sign (for ease of understanding) e.g. is_differentiable not is_non_differentiable (double-negatives like "is_non_diff(T) == false so T is not non-differentiable" can be hard to follow sometimes)?

[mcabbott]

I don't love the name.

One reason for the apparent double negative is that the present implementation fails to false. It answers the question "are we certain we can take the trivial path here?"

[oxinabox]

Is there a reason this is seperated from the others?

[mcabbott]

Long long ago that's where the acorn happened to land in the soil...

[oxinabox]

I kinda want to run this the other way around.
Can we make it so that the is_non_differentiable is the canonical one and ProjectTo falls back to that?

[mcabbott]

That would make some sense. It's a slightly bigger API change I guess, as this is then what you ought to define (although I imagine 3rd-party non-diff types are pretty rare).

What this way around does do is re-use the logic for e.g. deciding whether all elements of a tuple are non-diff, hence the whole thing is. ProjectTo(::Tuple) already has to have code for recursing into things. If we reverse it, then is_non_differentiable will also want to have such code.

[oxinabox]

Suggested change

	Returns `true` if `x` is known from its type not to have derivatives, else `false`.
	Returns `true` if `x` is known from its type that has no tangent space, else `false`.

[mcabbott]

Latest commit changes this to return a type, not a function. This means you test differential_type(x) <: AbstractZero to see whether x is known to have no derivative.

In general it returns the T in ProjectTo{T}. I'm not certain that has any other uses. But at least it's more obviously something read out from the projection machinery, rather than being an independent concept.