-
Notifications
You must be signed in to change notification settings - Fork 1.7k
Description
What
Make sure that error types are reasonably small (in bytes).
Why
Currently our error types are huge, esp. DataFusionError -- which is used all over the place -- measures over 100 bytes. For comparison: that's the size of over 4 Strings. Now allocation-wise, this usually wouldn't be an issue on the cold path, but the issue is that large errors also blow up Result enums which are allocated on stack and within async state machines. So you pay for that on the "happy path" as well.
How
Place a few careful Boxes within the respective error variants and remove the #[expect(clippy::large_enum_variant). Since the Box allocation only happens on the error path, this makes the happy path strictly better. And the error path is only a bit slower due to that1
Footnotes
-
If you look at all the
Strings andVecs in our error types, I would argue that we don't optimize for error performance anyways. And it gets even worse when thebacktracefeature is enabled. ↩