Add Enzyme support

[devmotion]

I wanted to add some very quick and rough support for Enzyme, to be able to avoid the hacks in the Turing.jl PR. I think the code is really not what one would want to use in the long term (probably should use AbstractDifferentiation/LogDensityProblemsAD/some Turing AD meta package).

@torfjelde I noticed that there are no tests of the AD backends at all currently? Maybe we should set them up for the existing backends first?

[coveralls]

Pull Request Test Coverage Report for Build 4227260462

• 1 of 4 (25.0%) changed or added relevant lines in 2 files are covered.
• No unchanged relevant lines lost coverage.
• Overall coverage decreased (-1.05%) to 61.268%

---

Changes Missing Coverage	Covered Lines	Changed/Added Lines	%
src/compat/enzyme.jl	0	3	0.0%

Totals
Change from base Build 4088196069:	-1.05%
Covered Lines:	87
Relevant Lines:	142

---

💛 - Coveralls

[codecov]

Codecov Report

Base: 62.31% // Head: 61.26% // Decreases project coverage by -1.06% ⚠️

Coverage data is based on head (e7265af) compared to base (64ef0ee).
Patch coverage: 25.00% of modified lines in pull request are covered.

Additional details and impacted files

@@            Coverage Diff             @@
##           master      #42      +/-   ##
==========================================
- Coverage   62.31%   61.26%   -1.06%     
==========================================
  Files           8        9       +1     
  Lines         138      142       +4     
==========================================
+ Hits           86       87       +1     
- Misses         52       55       +3     

Impacted Files	Coverage Δ
src/compat/enzyme.jl	0.00% <0.00%> (ø)
src/AdvancedVI.jl	73.46% <100.00%> (+0.55%)	⬆️

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

[yebai]

@devmotion, we plan to improve AdvancedVI during the summer significantly, so feel free to move on and get this merged even without the AD tests.

[torfjelde]

(probably should use AbstractDifferentiation/LogDensityProblemsAD/some Turing AD meta package)

Yeah this is what I was thinking we should be doing. Have you tried using AbstractDifferentiation @devmotion ? The last time I gave it a go it was somewhat lackluster and didn't really seem to be under active development. The alternative is to use LogDensityProblems.jl and define something like evaluate_and_gradient, abusing the interface a bit.

[devmotion]

Yes, I still have pretty much the same impression. I encountered too many issues and annoyances, it seems more like an initial draft/proof of concept IMO. I think someone has to spend time on the package but since I have to hand in my thesis in a few weeks it won't be me in the near future 😄