Skip to content

Shrink error types sizes #16652

@crepererum

Description

@crepererum

What

Make sure that error types are reasonably small (in bytes).

Why

Currently our error types are huge, esp. DataFusionError -- which is used all over the place -- measures over 100 bytes. For comparison: that's the size of over 4 Strings. Now allocation-wise, this usually wouldn't be an issue on the cold path, but the issue is that large errors also blow up Result enums which are allocated on stack and within async state machines. So you pay for that on the "happy path" as well.

How

Place a few careful Boxes within the respective error variants and remove the #[expect(clippy::large_enum_variant). Since the Box allocation only happens on the error path, this makes the happy path strictly better. And the error path is only a bit slower due to that1

Footnotes

  1. If you look at all the Strings and Vecs in our error types, I would argue that we don't optimize for error performance anyways. And it gets even worse when the backtrace feature is enabled.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions