fix(types): replace 506 occurrences of 'Float64' with 'AbstractFloat'

[bionicles]

This potentially breaks a bunch of stuff, but I'm keen to make a change like this because it would allow us to do probabilistic programming with neural networks... @andreasnoack and @matbesancon and others can you check this out? If we allow more flexibility with the Float type then we can more easily integrate with other libraries. I'm not familiar with this code base so please let's discuss what to look at to see if this is OK

Philosophically, it doesn't make sense to require extreme 64-bit precision numbers to talk about uncertainty, although I can understand why this could be required for numerical algorithms. Is there a safe way for us to convert between types of floats and preserve gradients etc?

If an algorithm did require 64bit numbers, and it converted 32bit inputs to 64bit, ran the procedure, and converted it back to 32bit, would this screw up Zygote and Chain Rules? (cc @MikeInnes and @oxinabox )

[bionicles]

Hmmm...VSCode seems to have screwed this up. I'll try again with Atom

[oxinabox]

Why stop at AbstractFloat ?
Seems better to go all the way to Real and then can use Rational number and fixed point numbers etc.

But yes at very least AbstractFloat is a good idea, cutting down to Float32 for e.g. covarience matrixes will halve the amount of memory used

[oxinabox]

If an algorithm did require 64bit numbers, and it converted 32bit inputs to 64bit, ran the procedure, and converted it back to 32bit, would this screw up Zygote and Chain Rules? (cc @MikeInnes and @oxinabox )

Its fine.

[johnczito]

These changes are much too blunt, at least in the test files. Here (^) for example, where the point is to loop through different concrete eltypes.

[oxinabox]

Probably the tests should have been excluded from the regex.

[MikeInnes]

This is probably better done by hand (and perhaps one or two features at a time to make it easier to review). From a quick scan most of the changes are false positives or incorrect, e.g. changing the type parameters of output in examples or dispatch constraints like Vector{AbstractFoat} which aren't doing what you want.

Most likely significant work is needed to be truly generic (eg propagating the eltype of inputs) and changing some instances of Float64 is a minority of it.

[oxinabox]

Ideally, SourceWalk.jl would be usable for this rough first pass rather than by hand, which would be a bit smarter, but still a manual pass would be needed after

[bionicles]

Ok, I swapped in the old test directory, next we can look at examples and dispatch constraints

Anyone know a better approach to get this done efficiently? I'm really not familiar with this codebase and have only like 1 week of julia experience so it feels wrong for me to be tinkering here, but it would be super useful to hook distributions up to flux models

[bionicles]

I didn't even know Real type was a thing, I'll look at that. Guidance appreciated

[bionicles]

if we use Real, would that require us to convert Int to Float?

[bionicles]

hmmm, it looks like multiple dispatch freaks out. I thought AbstractFloat would accept Float64 but apparently a method defined for AbstractArray{AbstractFloat} doesn't match Array{Float64} ... super weird... are we really expected to define everything for every data type? That would be a disgusting amount of busywork. Perhaps we should just delete the types alltogether

[bionicles]

AbstractMatrix{AbstractFloat} -> AbstractMatrix

[kmsquire]

it looks like multiple dispatch freaks out. I thought AbstractFloat would accept Float64 but apparently a method defined for AbstractArray{AbstractFloat} doesn't match Array{Float64} ... super weird... are we really expected to define everything for every data type? That would be a disgusting amount of busywork. Perhaps we should just delete the types alltogether

See https://docs.julialang.org/en/v1/manual/types/#Parametric-Composite-Types-1 for an explanation as to why this is true.

Instead of using AbstractArray{AbstractFloat}, you can use AbstractArray{<:AbstractFloat}.

[andreasnoack]

As mentioned in #1126 (comment), this has to be done more carefully. E.g. some of these functions only work correctly for Float64 inputs so I'm closing this one.