Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement backend for Tracker #44

Merged
merged 7 commits into from
Feb 2, 2022
Merged

Conversation

sethaxen
Copy link
Member

Relates #40

@sethaxen
Copy link
Member Author

Tracker's an interesting case because I think you want to avoid calling Tracker.data on the outputs only if you're nesting the AD (note the Hessian failures). So here we may do something like add a type argument to the backend to indicate whether nesting is happening and then lowest would set that to true, and based on that we would unwrap the outputs or not.

@sethaxen
Copy link
Member Author

Actually, even if we track nesting in the constructor, we still have the problem that the test suite uses prod to test Hessians, but Tracker computes the wrong Hessian for prod: FluxML/Tracker.jl#120. Perhaps the testing functions should allow the user to provide a function and inputs to be used for testing.

struct TrackerBackend <: AbstractReverseMode end

function second_lowest(::TrackerBackend)
return throw(ArgumentError("Tracker backend does not support nested differentiation."))
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the best way I could think of to signal that Tracker should not be used for higher order AD.

@sethaxen sethaxen marked this pull request as ready for review January 28, 2022 22:59
@codecov-commenter
Copy link

codecov-commenter commented Jan 28, 2022

Codecov Report

Merging #44 (6b7f2f1) into master (d9b3983) will increase coverage by 0.61%.
The diff coverage is 100.00%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master      #44      +/-   ##
==========================================
+ Coverage   82.39%   83.00%   +0.61%     
==========================================
  Files           4        5       +1     
  Lines         443      459      +16     
==========================================
+ Hits          365      381      +16     
  Misses         78       78              
Impacted Files Coverage Δ
src/AbstractDifferentiation.jl 78.51% <100.00%> (+0.05%) ⬆️
src/tracker.jl 100.00% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d9b3983...6b7f2f1. Read the comment docs.

@sethaxen
Copy link
Member Author

This PR is now ready for review.

Our version of jacobian is slower than Tracker's, but it seems to be less brittle. That being said, Tracker as a whole seems pretty brittle (e.g. Tracker.jacobian(cumsum, x) fails) and is likely not used much. Still, I think it should be included for now for completeness.

Copy link
Member

@mohamed82008 mohamed82008 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good!

@mohamed82008 mohamed82008 merged commit be3de03 into JuliaDiff:master Feb 2, 2022
@mohamed82008
Copy link
Member

Thanks Seth and sorry for the delay.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants