Stop gradient feature

[Red-Portal]

Hi,

Stopping/dropping gradients is quite a common feature for a lot of applications. It would be great to have that in Tapir as well. Currently, ChainRulesCore provides ignore_gradient and non_differentiable as a unified interface, but pretty much only Zygote supports it. However, ReverseDiff provides its own macro: ReverseDiff.@skip.

[willtebbutt]

This should be straightforward, albeit slightly different from how ChainRules / Zygote do it, because we do a lot of in-place incrementation and propagation of memory locations at which to increment on the forwards-pass of AD.

I think this could be done by adding a rule which

1. replaces the fdata associated to the thing whose gradient we're dropping with a copy that doesn't live at the same memory address, and
2. returning zero_rdata always.