Added CPUStatic backend and implemented new initialparameters interface.

[benedict-96]

CPUStatic was added as a Backend.

Further changes:

• AbstractInitializer was renamed to Initializer.
• init keyword was renamed to initializer.
• initialparameters now always takes five input arguments: rng, initializer, model, backend, type.

This pr is breaking for GeometricMachineLearning and SymbolicNeuralNetworks and should therefore be released as v0.5.

Fixes #18.
Fixes #19.

[codecov]

Codecov Report

Attention: Patch coverage is 26.86567% with 49 lines in your changes missing coverage. Please review.

Project coverage is 42.96%. Comparing base (2657df1) to head (4ff3219).
Report is 13 commits behind head on main.

Files with missing lines	Patch %	Lines
src/static_cpu_backend.jl	12.50%	21 Missing ⚠️
src/utils/changebackend.jl	0.00%	13 Missing ⚠️
src/neural_network.jl	44.44%	5 Missing ⚠️
src/cells/recurrent.jl	0.00%	2 Missing ⚠️
src/neural_network_backend.jl	50.00%	2 Missing ⚠️
src/cells/grid.jl	0.00%	1 Missing ⚠️
src/cells/gru.jl	0.00%	1 Missing ⚠️
src/cells/identity.jl	0.00%	1 Missing ⚠️
src/cells/lstm.jl	0.00%	1 Missing ⚠️
src/initializer.jl	80.00%	1 Missing ⚠️
... and 1 more

Additional details and impacted files

@@             Coverage Diff             @@
##             main      #22       +/-   ##
===========================================
- Coverage   85.28%   42.96%   -42.32%     
===========================================
  Files          22       25        +3     
  Lines         367      391       +24     
===========================================
- Hits          313      168      -145     
- Misses         54      223      +169     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[michakraus]

Should CPUStatic be defined as a subtype of KernelAbstractions.Backend ?

struct CPUStatic <: KernelAbstractions.Backend end

My understanding was that this is considered a mild form of type piracy.

[benedict-96]

Should CPUStatic be defined as a subtype of KernelAbstractions.Backend ?

struct CPUStatic <: KernelAbstractions.Backend end

My understanding was that this is considered a mild form of type piracy.

Ok, I see I shouldn't have opened this pr that quickly ...

It is true that in the KernelAbstractions documentation it says (somewhere) that custom Backends should only be defined for custom array types, and MArray does not constitute such a custom array type. From a pure Julia perspective the following extensions do however not constitute type piracy:

• KernelAbstractions.ones(::CPUStatic, ::Type{T}, dims::Tuple) where T
• KernelAbstractions.zeros(::CPUStatic, ::Type{T}, dims::Tuple) where T
• KernelAbstractions.allocate(::CPUStatic, ::Type{T}, dims::Tuple) where T
• KernelAbstractions.copyto!(::CPUStatic, x::MArray, y::AbstractArray)

The only real case of type piracy is:

• KernelAbstractions.get_backend(::MArray)

If the get_backend type piracy issue is too big, we could solve it by:

1. Defining a new backend type, something like AbstractNeuralNetworks.NeuralNetworkBackend,
2. Keeping KernelAbstractions.Backend as is but define a new array type, e.g. NetworkStaticArray.

If we opt for the latter the array type would look something like:

struct NetworkStaticArray{T, AT <: MArray{T}} <: AbstractArray{T}
    A::AT
end

I think the second option is more reasonable.

[michakraus]

Defining a custom wrapper around MArray seems a bit over the top, especially given the potentially very large number of functions you'd have to wrap for this to work properly and as efficiently as MArray.

[benedict-96]

Ok, then I'll just do
const NetworkBackend = Union{KernelAbstractions.Backend, CPUStatic} and export this 👍