Add error boundary support for rules

[kaos]

The idea: to allow @rules to declare "error boundaries", by introducing a new _error_boundary: bool = False param to Get()s and Effect()s. :) (using underscore prefix on _error_boundary to indicate this as a private feature, at least for now ;) )

Example:

@rule 
async def i_want_this_to_be_unfallible(...) -> CertainResult:
  try:
    not_so_sure = await Get(Data, UnreliableSource(), _error_boundary=True)
    ...
  except Exception as e:
    return CertainResult(error=f"Oh no! ERROR: {e}")
  ...
  return CertainResult(result="Yay!")

Outdated example

@rule 
async def i_want_this_to_be_unfallible(...) -> CertainResult:
  not_so_sure = await Get(Data, UnreliableSource(), _safe=True)
  if isinstance(not_so_sure, Exception):
    return CertainResult(error=f"Oh no! ERROR: {not_so_sure}")
  ...
  return CertainResult(result="Yay!")

This allows rules to not have to worry about wrapping return values as "optional foo" etc, which can be very invasive especially when the optional but is buried deep in the rule graph, all intermediate types must also support, and handle, the optional result values every where.

I'm still not sure how to adjust the type hints to accommodate the new result type of a "safe" Get, ought to be _Output | Exception to force consumers to address the possibility of the error in the result value.
Edit: no need, throwing errors at the error boundary instead side-steps the typing issue altogether.

An alternative approach, presented by @thejcannon, could be to support a Result enum like of type where one leg is a the value and the other the error. Not sure what that would look like (in UX nor in implementation).

[thejcannon]

This makes me think of Rust's Result type (which is stolen from other places as well). Maybe we just lean into that?

Let's say we had:

_ValType = TypeVar("_ValType")
_ErrType = TypeVar("_ErrType")

@dataclass(frozen=True)
class Result(Generic[_ValType, _ErrType]):
    @property
    def maybe_value(self) -> _ValType | _ErrType:
        ...

    @property
    def value(self) -> _ValType:
        val = self.maybe_value
        # Some assertion that its the right type...
        return val

Then you could have your rules return

async def my_rule(...) -> Result[TypeICareAbout]:
    ...

so then it just becomes a question of how to make that "work" in the Pants' little type scraper, right?

[thejcannon]

Also, can you give some links to in-Pants-repo examples of the behavior you're trying to improve upon? That'd help visual folks like me 😄

[kaos]

Thanks @thejcannon I'll have a proper look at your comments later today/tonight.

Having stepped away for a minute, I realize I actually think this might work better (except perhaps if used in a MultiGet..?):

@rule 
async def i_want_this_to_be_unfallible(...) -> CertainResult:
  try:
    not_so_sure = await Get(Data, UnreliableSource(), _error_boundary=True)
    ...
  except Exception as e:
    return CertainResult(error=f"Oh no! ERROR: {e}")
  ...
  return CertainResult(result="Yay!")

That way there's no need to adjust the type hints at all, and things will just continue to work as now, except that any exceptions from invoked rules will be captured rather than propagate out as an engine error to the root result.

[thejcannon]

The CertainResult type still looks like a Result type to me.

[kaos]

One thing this would allow, is to get rid of (or at least not require) the use of all the "description_of_origin" fields.
I'm not sure if this is the most relevant thread for this, but serves as example: #15789

The idea being, that in case of errors, you can throw the error with the information available at hand, and the callsite (the current origin for the value of description_of_origin) may use an error boundary on the Get, to augment the error with additional context information when re-raising it.

This serves two things: one is to make requests slimmer and improve cacheability as we remove the request origin as the response will be the same regardless of where the request came from, and secondly it allows the origin/callsite to decide if it is a recoverable error which is not possible today with the description_of_origin solution. Say we want to fetch all targets based on a list of addresses provided by the user. This may fail. Now we have ways to mitigate the issue with malformed addresses and other issues, but this is very much on a case by case basis where as with error boundaries it would just work.

I don't have a bigger example code snippet at hand right now, hope the above will shed some light on the motivation for this nonetheless. :)

[kaos]

Btw, I do like the Result analogy, and I could see us going down that road as well, but it limits the application of the pattern to the rules adopting it for their return values. So callsites can't use any rule safely, only rules that return its value using this new Result type.

Edit: Ok, I see you could introduce thin "wrapper" rules around regular rules to capture any error.. but that seems more boilerplatey.

[stuhood]

Some prior art in this area: #10954

I believe that I would be in favor of having exceptions raised inside of @rule code.

But it's also worth noting that some error context will now be added automatically without catching/re-throwing by implementing EngineAwareParameter.debug_hint and giving @rules descriptions: see #17119.

[kaos]

But it's also worth noting that some error context will now be added automatically without catching/re-throwing by implementing EngineAwareParameter.debug_hint and giving @rules descriptions: see #17119.

Nice 👍🏽 I'll have to use that more. It does not solve the issue with allowing a rule to continue in face of errors though (due to bad input values or other issues)

Some prior art in this area: #10954

👀 🙏🏽

I believe that I would be in favor of having exceptions raised inside of @rule code.

Care to elaborate a bit on this point?

[stuhood]

I believe that I would be in favor of having exceptions raised inside of @rule code.

Care to elaborate a bit on this point?

i.e.: allowing for try-catch. Currently a @rule cannot catch an error from a dependency @rule, because we do not feed the generator an error, and it instead just never wakes up. That's what the linked PR implemented.

[kaos]

I believe that I would be in favor of having exceptions raised inside of @rule code.

Care to elaborate a bit on this point?

i.e.: allowing for try-catch. Currently a @rule cannot catch an error from a dependency @rule, because we do not feed the generator an error, and it instead just never wakes up. That's what the linked PR implemented.

Ah, right. Yes, so that's what I've implemented here then, as suggested in #17745 (comment)

(not updated the PR desc yet)

[kaos]

Looking at #10954 the idea here is pretty close to that, but with an opt-in for the new behaviour, and also doesn't mask the exception type but passes the original error on as-is. Maybe would be a good idea to wrap it in an engine specific error to be able to dress it with more context about the origins of the exception.. ?

Could steal most of the tests however with only minor adjustments I think... :)

[stuhood]

Generators have a generator.throw method which should probably be used rather than sending an Exception value via generator.send: otherwise Gets will not be able to request values which subclass Exception and act as error boundaries (niche, I know).

[kaos]

Oh, cool. Yea, I did consider that return values couldn't be based on Exception, but was unaware of that solution. Neat.

[kaos]

@stuhood Well, that's neat, and would be cool to support (however niche), but not even the prior art you linked to supported this (although using the generator.throw idiom) as it was switching to it based on if isinstance(arg, Exception):

https://github.com/pantsbuild/pants/pull/10954/files#diff-156757f090beeedd1a880772f81cae2da74e4275b55d392b2cc06a44543c3ee9R106

I'm working on reviving #10954 ...

[thejcannon]

Also, careful with Exception v BaseException. I never know when it's appropriate to use Base instead of Exception.

Edit: ah I think it's exceptions that terminate don't derive from Exception ?

[kaos]

Also, careful with Exception v BaseException. I never know when it's appropriate to use Base instead of Exception.

Edit: ah I think it's exceptions that terminate don't derive from Exception ?

Yea, I think I recall having read that Exception is everything except SystemExit and KeyboardInterrupt pretty much..

[kaos]

@stuhood With this, I think I may be able to solve the need I have for #17674. If the lint goal uses an error boundary when invoking all the linters, then it doesn't matter if the visibility rules checker throws for all the other linters.. as the visibility report will get produced properly any way by the visibility linter.

[kaos]

@thejcannon

The CertainResult type still looks like a Result type to me.

And here is another example:

https://github.com/pantsbuild/pants/pull/17347/files#r1059664930

[kaos]

Superseded by #17911