Adapt to Enzyme v0.13

[ExpandingMan]

EDIT by @gdalle: see final description below

I'm a little bewildered by testing of this package, so I haven't checked this thoroughly yet and likely something somewhere will break, but it seems to mostly work.

[codecov-commenter]

Codecov Report

Attention: Patch coverage is 96.37306% with 7 lines in your changes missing coverage. Please review.

Project coverage is 97.83%. Comparing base (ee11b70) to head (8dfdeab).
Report is 1 commits behind head on main.

Files with missing lines	Patch %	Lines
...ace/ext/DifferentiationInterfaceEnzymeExt/utils.jl	88.00%	6 Missing ⚠️
...ifferentiationInterfaceEnzymeExt/reverse_onearg.jl	98.48%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #471      +/-   ##
==========================================
- Coverage   98.51%   97.83%   -0.68%     
==========================================
  Files         107      106       -1     
  Lines        4298     4305       +7     
==========================================
- Hits         4234     4212      -22     
- Misses         64       93      +29     

Flag	Coverage Δ
DI	98.30% <96.37%> (-0.28%)	⬇️
DIT	96.88% <ø> (-1.49%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[gdalle]

Yeah the testing is a bit chaotic because I have a separate GitHub Actions workflow and a separate set of dependencies for each AD backend. Otherwise it's very hard to instantiate an environment with all 12 of them, especially since some are nearly unmaintained. As a result, testing locally kinda sucks, but if you become more involved in DI development we may be able to figure out something smarter.
Thanks for this contribution in any case! I'll try to take it from here, but do you happen to have a summary of the breaking changes in Enzyme v0.13? I didn't see one on the repo's release notes.

[ExpandingMan]

I have not seen an explicit list of breaking changes, I just went through and tested all the basic cases myself. Seems to be almost done, but as you can see it's not quite there.

Note that, as per our conversation on slack, this is going to have to flatten the Enzyme jacobians, which, as of writing, is not in yet.

[ExpandingMan]

I now have this flattening the jacobians. Tests now seem to mostly pass, but there are still some cases where it's mysteriously returning all zeros.

Btw, it's unfortunate that throwing test errors doesn't actually tell you what failed, it just shows you what the result is (because it's just showing the standard @test failure message) and it's not particularly easy to figure out what's going on by looking in the source code. (Just something to think about as I saw you had another PR for cleaning up testing.)

[gdalle]

DI extensions (Enzyme)

• Remove the deferred mechanism (no longer needed)
• Implement utilities to modify the Mode object (with/without primal, splitting, error analysis)
• Revamp pushforward and pullback with these new modes. Use autodiff_thunk everywhere for out-of-place functions in reverse mode (see also Syntactic sugar for vjp EnzymeAD/Enzyme.jl#1853).
• Implement batched versions (Vector / chunked / batch mode #312)
  • for forward mode (in-place and out-of-place functions)
  • for reverse mode (only in-place functions until Batch mode with autodiff_thunk EnzymeAD/Enzyme.jl#1883)
• Adjust to new syntax for Enzyme.gradient and Enzyme.jacobian. Reshape the Jacobian accordingly when it is a higher-dimensional tensor.

DI source

• Remove mention of Enzyme deferred in DI.nested docstring (nested is useless for now)

DI tests

• Enzyme: Deactivate type stability tests (broken by Enzyme update).
• Temporarily deactivate Lux tests (until Update to Lux v1 #484, because of compat conflicts)

DIT tests

• Temporarily deactivate Lux tests

[ExpandingMan]

Thanks for finishing this.