-
Notifications
You must be signed in to change notification settings - Fork 62
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
"Writing good rules" docs should comment on type constraints #155
Comments
I definitely agree that the above is necessary, but I'm not sure that it's sufficient. For example, you probably don't want to be implementing |
Yeah, we might want to add a note about performance considerations with that as an example |
Yes, Sometimes you get to combine them via |
I don't buy that. The issue is that you're saying "for no subtype of AbstractMatrix should you look inside this function and figure out how to differentiate it". This just isn't the behaviour that you want from an AD tool in general. It really should be able to automatically exploit eg. that a matrix is AFAICT the defining difference between writing normal Julia method and writing code for a rule is that when writing a normal Julia method generic fallbacks are better than nothing. There's no other code, so what else could you hope to do? Conversely, the generic fallback to "just do AD" is often exactly what you would have wanted to do, and custom rules can get in the way of this, causing significantly worse performance than the fallback. I'm not saying that it's impossible to write normal Julia methods that specialise and cause bad performance, but it seems to be much less of a problem in practice. A better approach would be a recommendation like "only write a rule if you're confident that it's the right way to compute the rule for every reasonably-implemented subtype". e.g. it's probably reasonable to implement a rule for The alternative is to require that you implement custom rules for all methods of edit: In short, err on the side of caution when implementing rules. Too specific is likely to cause fewer problems than overly general. |
I agree with Will. But also I think (hope?) this worry is only significant for very generic function (i.e. that have many specialisation and may be extended by many packages) and maybe even those which are performance critical. So |
You make a good point Will, I retract my claim of the generality of my statement. We do want a thing saying that rrule methods should be no more general than the methods they are written for. |
the
frule
/rrule
method signatures should have the same type constraints as the primal function (JuliaDiff/ChainRules.jl#175 (comment))For example if your pacakge defines two methods for
foo
:prefer defining
to defining
The text was updated successfully, but these errors were encountered: