Support for exception handling

[MikeInnes]

The easiest way is to assume that try blocks are successful, and if a catch block ever runs we just disable back propagation entirely. Longer term we can do something smarter here, but it generally seems like exceptions should not be too common in people's numerical code.

I doubt we'll ever be able to differentiate through thrown exceptions without continuations, or other similar heroics in the compiler. It might be possible to throw a backpropagator + call information somehow, but I haven't thought about it much yet.

[tpapp]

My use case for this is the following:

1. Consider a log density on a subset D ⊂ ℝⁿ. The program tests for x ∈ D, and for x ∉ D, returns -Inf. I thought it best to do this by throwing an exception and unwinding the stack, since for complicated functions that calculate the log density by parts (calling other functions), the callers then don't have to check for -Inf to return early. This is costly, but happens only during adaptation, until the algorithm finds the typical set.

2. A special case of this is a log density which is only defined on some specific set (eg a simplex), and values outside are invalid. But transformations accumulate numerical errors so this can happen in corner cases. Then the function could just throw an error, which could be caught.

As noted in the other issue, currently I implement this with this function.

I have a macro for checking -Inf and returning early, so I could refactor my own code easily. The problem is that exceptions in other packages are hard to avoid, eg simple uses of @assert and ArgCheck.@argcheck.

Ideally, a solution to this should allow replacing the exception with a real value (as if returned by the original function), typically -Inf, with undefined or 0 gradient.

[MikeInnes]

Presumably other packages would not be throwing a RejectLogDensity? Throwing other exceptions, which wouldn't be caught by that try/catch, is fine.

Ideally, a solution to this should allow replacing the exception with a real value (as if returned by the original function), typically -Inf, with undefined or 0 gradient.

You could actually get this behaviour pretty easily with a custom adjoint. I think the macro approach is better myself but I'd nonetheless be happy to give advice on this, maybe on a new thread.

[tpapp]

Thanks for clarifying --- I now understand that the issue is the try ... catch block, not the throw.

It also occurred to me that I can just catch outside the AD invocation.

[schlichtanders]

Just stumbled upon this when trying to replicate the Trebuchet example. It turns out that there try/catch is used, but only thanks to @warn.

@info / @warn / @error all introduce a try/catch and hence all code with such logging support is currently not differentiable using reverse zygote.

I think it would be really helpful if at least these get support.

[MikeInnes]

See #466. It's a bit unfortunate that the logging code dumps so much stuff, but I guess we'd better support it.

Also, note that there are several ways around this. You can use ignore to ignore logging code. For example like the trebuchet I think you need a custom adjoint for the function that has the try/catch, but it's not too hard to do.