Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add StructArray Constructors (#3879) #4064

Merged
merged 5 commits into from
Apr 25, 2023

Conversation

tustvold
Copy link
Contributor

@tustvold tustvold commented Apr 12, 2023

Which issue does this PR close?

Part of #3879

Rationale for this change

What changes are included in this PR?

Are there any user-facing changes?

The TryFrom constructor no longer creates a null buffer if all children are null, this should not only be faster, but is also significantly less confusing - I had no idea it did this 😅

@github-actions github-actions bot added the arrow Changes to the arrow crate label Apr 12, 2023
@tustvold tustvold force-pushed the struct-array-constructors branch 2 times, most recently from 60086c7 to ad76714 Compare April 12, 2023 15:33
@tustvold tustvold force-pushed the struct-array-constructors branch from ad76714 to 92e9f8a Compare April 12, 2023 19:30
@tustvold tustvold marked this pull request as ready for review April 12, 2023 19:45
Copy link
Contributor

@alamb alamb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @tustvold -- this is quite a train of PRs you have here

/// * `arrays[i].len() != arrays[j].len()`
/// * `arrays[i].len() != nulls.len()`
/// * `!fields[i].is_nullable() && !nulls.contains(arrays[i].nulls())`
pub fn try_new(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As in the similar PRs, I think we should add test coverage for these new constructors, especially the error conditions

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I verified that the error conditions are now covered (via should_panic messages). 👍

@@ -359,37 +485,10 @@ impl std::fmt::Debug for StructArray {

impl From<(Vec<(Field, ArrayRef)>, Buffer)> for StructArray {
fn from(pair: (Vec<(Field, ArrayRef)>, Buffer)) -> Self {
let capacity = pair.0.len();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it is really nice that you are making the construction (and validation) of the various array types consistent ❤️

@tustvold
Copy link
Contributor Author

I've added tests, well re-used the existing tests 😄

@tustvold tustvold added the api-change Changes to the arrow API label Apr 25, 2023
@tustvold tustvold force-pushed the struct-array-constructors branch from 3b852da to b66a8c7 Compare April 25, 2023 15:31
@@ -309,66 +309,18 @@ impl TryFrom<Vec<(&str, ArrayRef)>> for StructArray {
type Error = ArrowError;

/// builds a StructArray from a vector of names and arrays.
/// This errors if the values have a different length.
/// An entry is set to Null when all values are null.
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This doc comment isn't rendered in docs.rs so this somewhat peculiar behaviour was effectively not documented

Copy link
Contributor

@alamb alamb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM -- thanks @tustvold

/// * `arrays[i].len() != arrays[j].len()`
/// * `arrays[i].len() != nulls.len()`
/// * `!fields[i].is_nullable() && !nulls.contains(arrays[i].nulls())`
pub fn try_new(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I verified that the error conditions are now covered (via should_panic messages). 👍

@tustvold tustvold merged commit 8d166a1 into apache:master Apr 25, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api-change Changes to the arrow API arrow Changes to the arrow crate
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants