Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Copying inappropriately aligned buffer in ipc reader #2883

Merged
merged 10 commits into from
Oct 16, 2022
Merged

Copying inappropriately aligned buffer in ipc reader #2883

Changes from 1 commit
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
aba2874
Fix ptr alignment error.
viirya Oct 16, 2022
e184bad
Rewrite test
viirya Oct 16, 2022
705aefa
Add values
viirya Oct 16, 2022
d2ccba5
Copy buffer if it is not aligned properly
viirya Oct 16, 2022
400cf81
Move to a function
viirya Oct 16, 2022
6a56057
Cover IntervalMonthDayNanoType too
viirya Oct 16, 2022
30ffb5c
Remove unnecessary change
viirya Oct 16, 2022
5dc264a
For review
viirya Oct 16, 2022
1538455
Make it generic for i256
viirya Oct 16, 2022
96577a6
Lift into parent match and use minimum length.
viirya Oct 16, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions arrow-array/src/array/list_array.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -878,6 +878,7 @@ mod tests {
assert_eq!(array.len(), 2);
assert_eq!(array.value(0), 0);
assert_eq!(array.value(1), 0);
assert_eq!(array.values(), &[0, 0]);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You need to call array.values() on the unaligned PrimitiveArray


// Not aligned.
// `ArrayData::build` checks buffer length.
Expand All @@ -889,6 +890,7 @@ mod tests {
let array = Int32Array::from(array_data);
assert_eq!(array.len(), 1);
assert_eq!(array.value(0), 0);
Copy link
Contributor

@tustvold tustvold Oct 16, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is now UB, as this violates the safety requirement in PrimitiveArray::values.

I'm not sure why MIRI isn't catching this...

Edit: array.value doesn't call array.values so this test isn't UB. If you add a call to array.values() in this test, MIRI will fail

assert_eq!(array.values(), &[0]);
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For https://github.com/apache/arrow-rs/pull/2883/files#r996382807, I think you mean here?


ArrayData::builder(DataType::Int32)
.add_buffer(buf2)
Expand Down