Skip to content

Conversation

@rluvaton
Copy link
Member

@rluvaton rluvaton commented Oct 23, 2025

Which issue does this PR close?

N/A

Rationale for this change

overriding this function improve performance over the fallback implementation

What changes are included in this PR?

Override implementation of:

  • count which is not optimized away even when ExactSizeIterator is implemented
  • nth to avoid calling next n + 1 times (which is also used when doing .skip)
  • nth_back
  • last
  • max

Are these changes tested?

Yes, I've added a lot of tests

Are there any user-facing changes?

Nope

@github-actions github-actions bot added the arrow Changes to the arrow crate label Oct 23, 2025
Copy link
Contributor

@alamb alamb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @rluvaton -- I went through this code and tests carefully and it looks really nice. I had some small comments, but nothing needed

I also ran llvm-cov and verified that all new functions were covered by the tests

}

// Go to the one before the last bit
self.current_offset = self.end_offset - 1;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note to self: we don't need to check for self.end_offset > 0 because if it were zero so would current_offset and the code would exit above 👍

fn get_value<T: SharedBetweenBitIteratorAndSliceIter>(iter: T) -> Self::Output;
}

fn assert_cases<O: Op>() {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it would be nice if there were some comments here that explained at a high level what this function did

I think it basically calls various next() / last(), etc functions on two iterators and then asserts they produce the same result

However, it is a very nice setup and easy to follow

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updated

.into_iter()
}

fn setup_and_assert(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the nice tests -- since they are always using BitIterator and Vec<bool> I bet there is a way to reduce some of the generics, but that is just a personal preference.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In order for the generic to be removed we need to have 2 functions, 1 for BitIterator and 1 for Vec<bool>, I did not want to have 2 functions to avoid possible bugs in the tests where the setup is different.

let mut expected = expected.clone();
for _ in 0..expected.len() {
let actual_val = actual.nth_back(2);
let expected_val = expected.nth_back(2);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

these go off the back of the iterators (which return None) which I think is fine

Copy link
Member Author

@rluvaton rluvaton Oct 24, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah I know, this is on purpose, thanks

@alamb
Copy link
Contributor

alamb commented Oct 26, 2025

@rluvaton - please let me know if you would like to address comments in this PR before I merge it

@rluvaton
Copy link
Member Author

Yes, addressing now, thanks

@rluvaton
Copy link
Member Author

@alamb updated

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

arrow Changes to the arrow crate performance

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants