Improvements to numeric exponent rules #224
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR makes a few improvements to the rules for
^(::Number, ::Number)
.For the general rule, it slightly improves the efficiency by removing the second
^
call.It also adds a new real rule that avoids unnecessarily complexifying the tangents and cotangents. The general rule embeds the numbers in the complex plane for complex differentiation. However, for real negative base, exponentiation is undefined (literally throws an error) unless the exponent is exactly an integer. So for negative base, the derivative wrt the exponent is actually undefined (hence we can't even call FD on it). The new rule adopts the subgradient convention when the base is negative.
Oh, and since the rules are defined in
fastmath.jl
, I moved the test to the corresponding test file.Here are are a few examples:
Example 1:
frule
with positive real base, real exponentOnly the type has changed. Instead of getting a purely real complex tangent, we just get the equivalent real tangent. The
rrule
has the same behavior.on master:
this pr:
Example 2:
frule
with negative real base, integer exponentWe get the same tangent as we would have gotten had the input tangent on
p
beenZero()
on master:
this pr:
Example 3:
rrule
with negative real base, integer exponentThe cotangent on
p
is 0.on master:
this pr: