Test Enzyme and reexport `ADTypes.AutoEnzyme` #1887

devmotion · 2022-09-28T10:39:48Z

Note: This does not work yet

I opened this PR to make it easier to debug (and possibly fix) issues with Enzyme.

Currently, the following example does ~~not~~ work (note that the snippet does not require the PR which solely reexports AutoEnzyme at this point):

using Turing
using Enzyme
using ADTypes
Enzyme.API.runtimeActivity!(true);
Enzyme.API.typeWarning!(false);

@model function model()
    m ~ Normal(0, 1)
    s ~ InverseGamma()
    x ~ Normal(m, s)
end

sample(model() | (; x=0.5), NUTS(; adtype = ADTypes.AutoEnzyme()), 10)

~~With Enzyme#main my Julia (1.8.1) segfaults. An incomplete (it filled my whole terminal) output: https://gist.github.com/devmotion/1352197f2354c6fecddd7b778ec4bcf7#file-log-txt~~

The example works (latest releases of Turing, Enzyme, and ADTypes on Julia 1.10.0) but the following warnings show up:

warning: didn't implement memmove, using memcpy as fallback which can result in errors
warning: didn't implement memmove, using memcpy as fallback which can result in errors

coveralls · 2022-11-13T02:26:42Z

Pull Request Test Coverage Report for Build 12122531930

Details

0 of 0 changed or added relevant lines in 0 files are covered.
4 unchanged lines in 2 files lost coverage.
Overall coverage decreased (-0.3%) to 37.831%

Files with Coverage Reduction	New Missed Lines	%
src/mcmc/Inference.jl	2	52.44%
src/mcmc/abstractmcmc.jl	2	92.86%

Totals
Change from base Build 12110228665:	-0.3%
Covered Lines:	586
Relevant Lines:	1549

💛 - Coveralls

codecov · 2022-11-13T02:26:48Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 44.46%. Comparing base (c0a4ee9) to head (8bff239).

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #1887      +/-   ##
==========================================
- Coverage   44.72%   44.46%   -0.26%     
==========================================
  Files          22       22              
  Lines        1554     1554              
==========================================
- Hits          695      691       -4     
- Misses        859      863       +4

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

src/essential/ad.jl

wsmoses · 2023-06-26T18:13:14Z

Also if you want to disable the warnings you can set it like so (https://github.com/EnzymeAD/Enzyme.jl/blob/c29e6119c7963ddb22f1363726f762455748e193/src/api.jl#L414
)

Enzyme.API.typeWarning!(false)

wsmoses · 2023-06-26T18:15:25Z

You also may want to set the version to 0.11.2 since your CI currently is running at 0.11.0 (⌃ [7da242da] Enzyme v0.11.0)

wsmoses · 2023-06-27T05:12:50Z

@devmotion this PR (EnzymeAD/Enzyme.jl#914) should fix the immediate issues you see on CI if you want to try.

mhauru · 2024-09-12T14:34:07Z

Merging this before/at the same time as Enzyme's Turing CI makes sense. However, I think we need EnzymeAD/Enzyme.jl#1811 and EnzymeAD/Enzyme.jl#1812 fixed before merging this. I'm surprised that the Turing test suite didn't catch them, need to investigate why, maybe add some tests here.

yebai · 2024-10-24T11:25:42Z

src/essential/Essential.jl


 using AdvancedPS: AdvancedPS

 include("container.jl")

 export @model,
    @varname,
+    AutoEnzyme,


Let's export this as Turing.Experimental.AutoEnzyme until Enzyme becomes more stable.

What is the threshold for being considered stable here?

@mhauru and @penelopeysm probably have a lot more experience on this.

My heuristic threshold:

Enzyme passes all Distributions.jl and Turing.jl tests

No known segfaults for Enzyme

for a continuous period of 8 weeks.

mhauru · 2024-10-24T12:42:51Z

I've merged the latest master and upgraded to Enzyme v0.12. We are still being held back from v0.13 by Bijectors.jl. There are a number of new test failures, because

There seem to be some regressions, tests that used to pass but now don't. I need to investigate.
We had previously only tested Enzyme on the HMC and SGHMC tests. I hadn't realised/had forgotten that we weren't testing Enzyme with the full test suite. I've now added it to test/mcmc/gibbs.jl,test/mcmc/abstractmcmc.jl, and a couple of others as well. Everywhere where there is a loop over AD backends.
Runtime activity is no longer a global setting in Enzyme. Having to change how we set runtime activity I think is a good opportunity to take stock of its effect, so I've just removed using it for now. Let's try to get to a point where the only test failures are ones where Enzyme says "you may need to use runtime activity", see how many there are, and only then enable it.

Getting Bijectors.jl to support Enzyme v0.13 I think has to be the next step, because otherwise any of the failures we see here might already be fixed on v0.13, and thus minimising and reporting them is pointless.

wsmoses · 2024-11-06T17:43:22Z

gentle bump here

yebai · 2024-11-11T16:51:57Z

It would be good to address EnzymeAD/Enzyme.jl#1812, #2307 and TuringLang/Bijectors.jl#341 before merging this PR.

mhauru · 2024-11-12T11:17:04Z

TuringLang/Bijectors.jl#341 at the very least needs addressing, because it's currently holding us back from running a recent Enzyme version here, and thus we don't know if the test suite would pass on a recent version. TuringLang/Bijectors.jl#341 passes tests on 1.10, but on 1.11 the Enzyme tests fail because of the accursed extension load order issue. The fix for that is currently waiting on this one: TuringLang/Bijectors.jl#346 EDIT and consequently this: TuringLang/Bijectors.jl#349

wsmoses · 2024-11-29T06:30:21Z

With the bijectors fix landed, I suppose this is (again) ready to go?

wsmoses · 2024-11-29T06:31:54Z

separately @yebai you appear to have removedd my permissions to run tests, if that can be restored

yebai · 2024-12-01T21:03:57Z

separately @yebai you appear to have removedd my permissions to run tests, if that can be restored

I don't know what happened precisely -- some changes were made to the TuringLang repos permissions to make CI work more robustly.

wsmoses · 2024-12-02T16:14:20Z

┌ Warning: Could not use exact versions of packages in manifest, re-resolving
└ @ Pkg.Operations /opt/hostedtoolcache/julia/1.11.1/x86/share/julia/stdlib/v1.11/Pkg/src/Operations.jl:1[9](https://github.com/TuringLang/Turing.jl/actions/runs/12122531930/job/33795922092?pr=1887#step:8:10)02
ERROR: Unsatisfiable requirements detected for package DynamicPPL [366bfd00]:
 DynamicPPL [366bfd00] log:
 ├─possible versions are: 0.1.0 - 0.31.0 or uninstalled
 ├─restricted to versions [0.29, 0.30.4 - 0.31] by Turing [fce5fe82], leaving only versions: [0.29.0 - 0.29.2, 0.30.4 - 0.31.0]
 │ └─Turing [fce5fe82] log:
 │   ├─possible versions are: 0.35.3 or uninstalled
 │   └─Turing [fce5fe82] is fixed to version 0.35.3
 ├─restricted by compatibility requirements with Mooncake [da2b9cff] to versions: 0.29.0 - 0.30.5 or uninstalled, leaving only versions: [0.29.0 - 0.29.2, 0.30.4 - 0.30.5]
 │ └─Mooncake [da2b9cff] log:
 │   ├─possible versions are: 0.3.0 - 0.4.53 or uninstalled
 │   └─restricted to versions 0.4.19 - 0.4 by project [23fc8c3f], leaving only versions: 0.4.19 - 0.4.53
 │     └─project [23fc8c3f] log:
 │       ├─possible versions are: 0.0.0 or uninstalled
 │       └─project [23fc8c3f] is fixed to version 0.0.0
 └─restricted by compatibility requirements with Bijectors [76274a88] to versions: 0.31.0 or uninstalled — no versions left
   └─Bijectors [76274a88] log:
     ├─possible versions are: 0.1.0 - 0.15.2 or uninstalled
     ├─restricted to versions 0.14 - 0.15 by Turing [fce5fe82], leaving only versions: 0.14.0 - 0.15.2
     │ └─Turing [fce5fe82] log: see above
     └─restricted by compatibility requirements with Enzyme [7da242da] to versions: [0.1.0 - 0.[13](https://github.com/TuringLang/Turing.jl/actions/runs/12122531930/job/33795922092?pr=1887#step:8:14).16, 0.15.0 - 0.15.2] or uninstalled, leaving only versions: 0.15.0 - 0.15.2
       └─Enzyme [7da242da] log:
         ├─possible versions are: 0.1.0 - 0.13.18 or uninstalled
         └─restricted to versions 0.13 by project [23fc8c3f], leaving only versions: 0.13.0 - 0.13.18
           └─project [23fc8c3f] log: see above
Stacktrace:

I'd try to help, but I don't have permission to edit things or rerun CI xD

penelopeysm · 2024-12-02T16:16:52Z

It should resolve with Mooncake 0.4.54 as that allows for DPPL=0.31.0. Don't know why CI isn't picking up the new version.

└─Mooncake [da2b9cff] log:
     ├─possible versions are: 0.3.0 - 0.4.53 or uninstalled

0.4.54 should have been available a few hours ago.

mhauru · 2024-12-04T10:21:59Z

The registry issue is sorted, now merrily running with the latest Enzyme.

wsmoses · 2024-12-04T17:47:39Z

Check ADType: Error During Test at /home/runner/work/Turing.jl/Turing.jl/test/mcmc/hmc.jl:334
  Got exception outside of a @test
  ArgumentError: Unsupported ADType: ADTypes.AutoEnzyme{Nothing, Nothing}
  Stacktrace:
    [1] Main.ADUtils.ADTypeCheckContext(adbackend::ADTypes.AutoEnzyme{Nothing, Nothing}, child::DynamicPPL.DefaultContext)
      @ Main.ADUtils ~/work/Turing.jl/Turing.jl/test/test_utils/ad_utils.jl:102
    [2] macro expansion
      @ ~/work/Turing.jl/Turing.jl/test/mcmc/hmc.jl:336 [inlined]
    [3] macro expansion
      @ /opt/hostedtoolcache/julia/1.11.2/x86/share/julia/stdlib/v1.11/Test/src/Test.jl:1704 [inlined]
    [4] macro expansion
      @ ~/work/Turing.jl/Turing.jl/test/mcmc/hmc.jl:335 [inlined]
    [5] macro expansion
      @ /opt/hostedtoolcache/julia/1.11.2/x86/share/julia/stdlib/v1.11/Test/src/Test.jl:1793 [inlined]
    [6] top-level scope
      @ ~/work/Turing.jl/Turing.jl/test/mcmc/hmc.jl:22
    [7] include(fname::String)
      @ Main ./sysimg.jl:38
    [8] macro expansion
      @ ~/.julia/packages/TimerOutputs/6KVfH/src/TimerOutput.jl:237 [inlined]
    [9] macro expansion
      @ ~/work/Turing.jl/Turing.jl/test/runtests.jl:26 [inlined]
   [10] macro expansion
      @ /opt/hostedtoolcache/julia/1.11.2/x86/share/julia/stdlib/v1.11/Test/src/Test.jl:1704 [inlined]
   [11] macro expansion
      @ ~/work/Turing.jl/Turing.jl/test/runtests.jl:56 [inlined]
   [12] macro expansion
      @ ~/.julia/packages/TimerOutputs/6KVfH/src/TimerOutput.jl:237 [inlined]
   [13] macro expansion
      @ ~/work/Turing.jl/Turing.jl/test/runtests.jl:54 [inlined]
   [14] macro expansion
      @ /opt/hostedtoolcache/julia/1.11.2/x86/share/julia/stdlib/v1.11/Test/src/Test.jl:1704 [inlined]
   [15] top-level scope
      @ ~/work/Turing.jl/Turing.jl/test/runtests.jl:34
   [16] include(fname::String)
      @ Main ./sysimg.jl:38
   [17] top-level scope
      @ none:6
   [18] eval
      @ ./boot.jl:430 [inlined]
   [19] exec_options(opts::Base.JLOptions)
      @ Base ./client.jl:296
   [20] _start()
      @ Base ./client.jl:531

Looks like something in turing needs to be updated?

Add support for Enzyme

29017e7

devmotion mentioned this pull request Sep 28, 2022

Segmentation Fault (turing) EnzymeAD/Enzyme.jl#457

Closed

yebai added 2 commits November 7, 2022 21:51

Merge branch 'master' into dw/enzyme

fdf0d43

Merge branch 'master' into dw/enzyme

d7ef23e

Merge branch 'master' into dw/enzyme

874b9e7

devmotion commented Dec 23, 2022

View reviewed changes

src/essential/ad.jl Outdated Show resolved Hide resolved

devmotion commented Dec 23, 2022

View reviewed changes

src/essential/ad.jl Outdated Show resolved Hide resolved

devmotion and others added 9 commits December 23, 2022 22:47

Apply suggestions from code review

43ef4c4

Add Enzyme to test dependencies

3e5841f

Test Enzyme

66bce4e

Update ad.jl

7890134

Merge branch 'master' into dw/enzyme

120e9f5

Update Project.toml

f4bd1bf

Update advi.jl

c8e01d0

Merge branch 'master' into dw/enzyme

d8d7729

Merge branch 'master' into dw/enzyme

1d1dba0

sethaxen mentioned this pull request Mar 5, 2023

Enzyme segfaults on Turing model EnzymeAD/Enzyme.jl#650

Closed

devmotion and others added 6 commits March 7, 2023 10:20

Do not call Bijectors.setadbackend

946e594

Merge branch 'master' into dw/enzyme

edd19a4

Update Project.toml

e9eedd1

Merge branch 'master' into dw/enzyme

9cfb589

Merge branch 'master' into dw/enzyme

cf03624

Merge branch 'master' into dw/enzyme

0a5c42b

devmotion added 2 commits June 27, 2023 00:24

Address comments

8d8d031

Update runtests.jl

e591630

mhauru mentioned this pull request Oct 4, 2024

More autoformatting #2359

Merged

wsmoses mentioned this pull request Oct 22, 2024

bump Enzyme version in v0.2 TuringLang/AdvancedVI.jl#132

Merged

mhauru added 4 commits October 24, 2024 11:05

Merge remote-tracking branch 'origin/master' into dw/enzyme

1c7319b

Bump Enzyme to 0.13 in tests

79d057c

Run JuliaFormatter on more files, remove trailing whitespace

c98bbc9

Merge branch 'mhauru/more-formatting' into dw/enzyme

aa9abd6

mhauru mentioned this pull request Oct 24, 2024

Run JuliaFormatter on more files, remove trailing whitespace #2374

Merged

Restore compat with Enzyme v0.12

9d39193

yebai reviewed Oct 24, 2024

View reviewed changes

mhauru added 2 commits October 24, 2024 13:11

Import Enzyme in abstractmcmc and gibbs tests

66cd80a

Add Enzyme imports to a couple of other tests files

ec34e41

mhauru and others added 2 commits October 24, 2024 14:21

Remove unnecessary version conditions in tests

120230a

Merge branch 'master' into dw/enzyme

98f2c2b

devmotion and others added 3 commits November 29, 2024 09:30

Merge branch 'master' into dw/enzyme

2c88d59

Update Project.toml

f9165fa

Merge branch 'master' into dw/enzyme

e77aa54

mhauru added 2 commits December 2, 2024 15:25

Dump DPPL to 0.31

5698212

Merge remote-tracking branch 'origin/master' into dw/enzyme

8bff239

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Test Enzyme and reexport `ADTypes.AutoEnzyme` #1887

Test Enzyme and reexport `ADTypes.AutoEnzyme` #1887

devmotion commented Sep 28, 2022 •

edited

Loading

coveralls commented Nov 13, 2022 •

edited

Loading

codecov bot commented Nov 13, 2022 •

edited

Loading

wsmoses commented Jun 26, 2023

wsmoses commented Jun 26, 2023

wsmoses commented Jun 27, 2023

mhauru commented Sep 12, 2024

yebai Oct 24, 2024

wsmoses Oct 24, 2024

yebai Oct 28, 2024

mhauru commented Oct 24, 2024

wsmoses commented Nov 6, 2024

yebai commented Nov 11, 2024 •

edited

Loading

mhauru commented Nov 12, 2024 •

edited

Loading

wsmoses commented Nov 29, 2024

wsmoses commented Nov 29, 2024

yebai commented Dec 1, 2024

wsmoses commented Dec 2, 2024

penelopeysm commented Dec 2, 2024 •

edited

Loading

mhauru commented Dec 4, 2024

wsmoses commented Dec 4, 2024

Test Enzyme and reexport ADTypes.AutoEnzyme #1887

Are you sure you want to change the base?

Test Enzyme and reexport ADTypes.AutoEnzyme #1887

Conversation

devmotion commented Sep 28, 2022 • edited Loading

coveralls commented Nov 13, 2022 • edited Loading

Pull Request Test Coverage Report for Build 12122531930

Details

💛 - Coveralls

codecov bot commented Nov 13, 2022 • edited Loading

Codecov Report

wsmoses commented Jun 26, 2023

wsmoses commented Jun 26, 2023

wsmoses commented Jun 27, 2023

mhauru commented Sep 12, 2024

yebai Oct 24, 2024

Choose a reason for hiding this comment

wsmoses Oct 24, 2024

Choose a reason for hiding this comment

yebai Oct 28, 2024

Choose a reason for hiding this comment

mhauru commented Oct 24, 2024

wsmoses commented Nov 6, 2024

yebai commented Nov 11, 2024 • edited Loading

mhauru commented Nov 12, 2024 • edited Loading

wsmoses commented Nov 29, 2024

wsmoses commented Nov 29, 2024

yebai commented Dec 1, 2024

wsmoses commented Dec 2, 2024

penelopeysm commented Dec 2, 2024 • edited Loading

mhauru commented Dec 4, 2024

wsmoses commented Dec 4, 2024

Test Enzyme and reexport `ADTypes.AutoEnzyme` #1887

Test Enzyme and reexport `ADTypes.AutoEnzyme` #1887

devmotion commented Sep 28, 2022 •

edited

Loading

coveralls commented Nov 13, 2022 •

edited

Loading

codecov bot commented Nov 13, 2022 •

edited

Loading

yebai commented Nov 11, 2024 •

edited

Loading

mhauru commented Nov 12, 2024 •

edited

Loading

penelopeysm commented Dec 2, 2024 •

edited

Loading