Skip to content

Conversation

scovich
Copy link
Contributor

@scovich scovich commented Aug 20, 2025

Which issue does this PR close?

Rationale for this change

Today, ValueBuilder::[try_]append_variant unconditionally creates and uses a ParentState::Variant, but that is incorrect when the caller is a ListBuilder or ObjectBuilder. Rework the API so that the caller passes their parent state, thus ensuring proper rollback in all situations.

This is also a building block that will eventually let us simplify VariantArrayBuilder to use a ValueBuilder directly, instead of a VariantBuilder.

What changes are included in this PR?

Several methods become associated functions.

Are these changes tested?

Existing unit tests cover this refactor.

Are there any user-facing changes?

No

@github-actions github-actions bot added the parquet-variant parquet-variant* crates label Aug 20, 2025
Variant::Uuid(v) => self.append_uuid(v),
Variant::Object(obj) => self.append_object(metadata_builder, obj),
Variant::List(list) => self.append_list(metadata_builder, list),
Variant::Time(v) => self.append_time_micros(v),
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FYI I moved Time up by Date for better logical grouping.

(again below)

Variant::Float(v) => self.append_float(v),
Variant::Double(v) => self.append_double(v),
Variant::Binary(v) => self.append_binary(v),
Variant::Uuid(v) => self.append_uuid(v),
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FYI I moved Uuid below ShortString to match append_variant above

@scovich
Copy link
Contributor Author

scovich commented Aug 21, 2025

@codephage2020 and @klion26 -- this is a fast-follow PR to the one you just approved.
Currently stacked, but will rebase once the other merges.

Copy link
Member

@klion26 klion26 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@scovich Thanks for your contribution, LGTM % nit inline comment.

Variant::Object(obj) => return Self::append_object(state, obj),
Variant::List(list) => return Self::append_list(state, list),
}
state.finish();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure if it's better to move this line to the caller, the finish now is associated with ParentState, maybe creating and finishing in the same place is better?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The nested builders have to take ownership of their parent state (else many call sites would illegally return a reference to a temporary). So here, we have an owned value that sometimes needs finish and sometimes doesn't. Returning Option<ParentState<'_>> seems ugly and annoying?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If there is interest in trying the alternative, how about we make a follow on PR

Copy link
Contributor

@codephage2020 codephage2020 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Nice work!

@scovich
Copy link
Contributor Author

scovich commented Aug 21, 2025

@alamb -- ready for final review+merge

Copy link
Contributor

@alamb alamb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like an improvement to me -- thank you @scovich and @klion26 and @codephage2020

Let's keep the code flowing and we can continue improving it in follow on PRs

@alamb alamb merged commit cec24a0 into apache:main Aug 21, 2025
13 checks passed
@scovich
Copy link
Contributor Author

scovich commented Aug 21, 2025

Let's keep the code flowing and we can continue improving it in follow on PRs

As you wish!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
parquet-variant parquet-variant* crates
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Rework ValueBuilder API to work with ParentState for reliable nested rollbacks
4 participants