rustc_codegen_ssa/src/traits/autodiff.rs

[ZuseZ4]

We can change it later, but still should try to get it right in the first run.
The larger design limitation will anyway be the user-facing ad macro, which limits which information we get.
We can take some inspiration from other traits like CoverageInfo, AsmBuilderMethods, which are implemented in both gcc and llvm.

Motivation:
Allow adding AD support for different codegen backends.
cg_llvm will be the first user, cg_cuda probably next.
We should keep in mind other possible ad tools (e.g. gcc based), as well as a possible MIR based ad tool.
The last one would be feature wise an awesome fallback, even if it can't compete performance-wise with backend based AD tools.

Assumptions:

• No integer ad
• Only Forward and Reverse Mode (or combinations), ignore theoretically possible other evaluation orders.
• Add explicit support for higher orders, allowing ad tools to use possibly more efficient implementations.

Some inspiration: https://jax.readthedocs.io/en/latest/notebooks/autodiff_cookbook.html

There are a few things to keep in mind for the Design:

• Checkpointing, possibly as a non-exhaustive enum? It should always be safe for a backend to ignore it.
• Vectorization information (width). Always pass as a NonZeroUnsignedInteger?
• Custom derivatives.
• (minor): Don't block parallel fnc generations from Enzyme.