Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add boolean, date, timestamp & binary partition types #1180

Merged
merged 7 commits into from
Mar 6, 2023

Conversation

marijncv
Copy link
Contributor

@marijncv marijncv commented Feb 26, 2023

Description

Adds boolean, date, timestamp & binary partition value types

Related Issue(s)

closes #1170

Signed-off-by: Marijn Valk <marijncv@hotmail.com>
@github-actions github-actions bot added binding/rust Issues for the Rust crate rust labels Feb 26, 2023
Signed-off-by: Marijn Valk <marijncv@hotmail.com>
@marijncv marijncv marked this pull request as ready for review February 26, 2023 17:21
@marijncv marijncv changed the title add boolean, date & timestamp partition types add boolean, date, timestamp & binary partition types Feb 26, 2023
Signed-off-by: Marijn Valk <marijncv@hotmail.com>
Copy link
Collaborator

@wjones127 wjones127 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for working on this @marijncv!

Just one comment, but otherwise looks great.

rust/src/writer/utils.rs Outdated Show resolved Hide resolved
Co-authored-by: Will Jones <willjones127@gmail.com>
@marijncv marijncv requested review from wjones127 and removed request for rtyler, xianwill, houqp, mosyp, fvaleye and roeap March 3, 2023 06:01
fvaleye
fvaleye previously approved these changes Mar 3, 2023
Copy link
Collaborator

@fvaleye fvaleye left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @marijncv
LGTM 👍

Copy link
Collaborator

@wjones127 wjones127 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry for the delay. I meant my earlier comment about not needing more than 1 value in the test to apply to all the test cases, not just the one I had made a suggestion on.

Added a few other comments to make the test cases a little more compact.

Comment on lines 300 to 311
(
Arc::new(Int8Array::from(vec![Some(1), Some(2)])),
Some(String::from("1")),
),
(
Arc::new(Int16Array::from(vec![Some(1), Some(2)])),
Some(String::from("1")),
),
(
Arc::new(Int32Array::from(vec![Some(1), Some(2)])),
Some(String::from("1")),
),
Copy link
Collaborator

@wjones127 wjones127 Mar 6, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This can be applied to all the test cases: You shouldn't need the Some inside the array.

Suggested change
(
Arc::new(Int8Array::from(vec![Some(1), Some(2)])),
Some(String::from("1")),
),
(
Arc::new(Int16Array::from(vec![Some(1), Some(2)])),
Some(String::from("1")),
),
(
Arc::new(Int32Array::from(vec![Some(1), Some(2)])),
Some(String::from("1")),
),
(Arc::new(Int8Array::from(vec![1])), Some("1")),
(Arc::new(Int16Array::from(vec![1])), Some("1")),
(Arc::new(Int32Array::from(vec![1])), Some("1")),


#[test]
fn test_stringified_partition_value() {
let reference_pairs: Vec<(Arc<dyn Array>, Option<String>)> = vec![
Copy link
Collaborator

@wjones127 wjones127 Mar 6, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can use &str instead to avoid needing String::from() in every test case.

Suggested change
let reference_pairs: Vec<(Arc<dyn Array>, Option<String>)> = vec![
let reference_pairs: Vec<(Arc<dyn Array>, Option<&str>)> = vec![

),
];
for (vals, result) in reference_pairs {
assert_eq!(stringified_partition_value(&vals).unwrap(), result)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can use as_deref() to turn Option<String> into Option<&str>:

Suggested change
assert_eq!(stringified_partition_value(&vals).unwrap(), result)
assert_eq!(
stringified_partition_value(&vals).unwrap().as_deref(),
result
)

Comment on lines 332 to 338
(
Arc::new(StringArray::from(vec![
Some(String::from("1")),
Some(String::from("2")),
])),
Some(String::from("1")),
),
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The StringArray::from implementation can take a &str:

Suggested change
(
Arc::new(StringArray::from(vec![
Some(String::from("1")),
Some(String::from("2")),
])),
Some(String::from("1")),
),
(Arc::new(StringArray::from(vec!["1"])), Some("1")),

marijncv added 3 commits March 6, 2023 09:06
Signed-off-by: Marijn Valk <marijncv@hotmail.com>
Signed-off-by: Marijn Valk <marijncv@hotmail.com>
Copy link
Collaborator

@roeap roeap left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

Copy link
Collaborator

@wjones127 wjones127 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great! Thanks @marijncv

@wjones127 wjones127 merged commit bb9f040 into delta-io:main Mar 6, 2023
chitralverma pushed a commit to chitralverma/delta-rs that referenced this pull request Mar 17, 2023
# Description
Adds boolean, date, timestamp & binary partition value types

# Related Issue(s)
closes delta-io#1170

---------

Signed-off-by: Marijn Valk <marijncv@hotmail.com>
Co-authored-by: Will Jones <willjones127@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
binding/rust Issues for the Rust crate rust
Projects
None yet
Development

Successfully merging this pull request may close these issues.

feat(rust): support additional types for partition values
4 participants