Skip to content

Conversation

@gatesn
Copy link
Contributor

@gatesn gatesn commented Oct 27, 2025

FLUPs:

  • Arithmetic array to invert lhs/rhs for constant checks
  • Special handling for vectors with many nulls (iterate only the sparse non-null indices)

Signed-off-by: Nicholas Gates <nick@nickgates.com>
Signed-off-by: Nicholas Gates <nick@nickgates.com>
Signed-off-by: Nicholas Gates <nick@nickgates.com>
Signed-off-by: Nicholas Gates <nick@nickgates.com>
Signed-off-by: Nicholas Gates <nick@nickgates.com>
Signed-off-by: Nicholas Gates <nick@nickgates.com>
Signed-off-by: Nicholas Gates <nick@nickgates.com>
@gatesn gatesn requested a review from robert3005 October 27, 2025 22:39
@gatesn gatesn added the feature Release label indicating a new feature or request label Oct 27, 2025
@gatesn gatesn enabled auto-merge (squash) October 27, 2025 22:42
@codspeed-hq
Copy link

codspeed-hq bot commented Oct 27, 2025

CodSpeed Performance Report

Merging #5096 will degrade performances by 10.35%

Comparing ngates/arithmetic-buffer-compute (d042d45) with develop (60ce4b5)1

Summary

⚡ 90 improvements
❌ 3 regressions
✅ 1225 untouched
🆕 7 new
⏩ 29 skipped2

⚠️ Please fix the performance issues or acknowledge them on CodSpeed.

Benchmarks breakdown

Benchmark BASE HEAD Change
chunked_dict_primitive_canonical_into[f32, (1000, 10, 10)] 107.8 µs 92.5 µs +16.45%
chunked_dict_primitive_canonical_into[f32, (1000, 10, 100)] 837.1 µs 715.7 µs +16.97%
chunked_dict_primitive_canonical_into[f32, (1000, 100, 10)] 106.6 µs 91.5 µs +16.59%
chunked_dict_primitive_canonical_into[f32, (1000, 100, 100)] 840.7 µs 721.1 µs +16.59%
chunked_dict_primitive_canonical_into[f32, (1000, 1000, 10)] 127.9 µs 112.4 µs +13.8%
chunked_dict_primitive_canonical_into[f32, (1000, 1000, 100)] 1,023.2 µs 900.2 µs +13.67%
chunked_dict_primitive_canonical_into[f64, (1000, 10, 10)] 136.1 µs 120.3 µs +13.15%
chunked_dict_primitive_canonical_into[f64, (1000, 10, 100)] 1,071.3 µs 947.2 µs +13.1%
chunked_dict_primitive_canonical_into[f64, (1000, 100, 10)] 139.6 µs 123.9 µs +12.7%
chunked_dict_primitive_canonical_into[f64, (1000, 100, 100)] 1.1 ms 1 ms +12.42%
chunked_dict_primitive_canonical_into[f64, (1000, 1000, 10)] 172.1 µs 156.2 µs +10.2%
chunked_dict_primitive_canonical_into[u32, (1000, 10, 10)] 108.3 µs 92.9 µs +16.62%
chunked_dict_primitive_canonical_into[u32, (1000, 10, 100)] 840.7 µs 718.1 µs +17.06%
chunked_dict_primitive_canonical_into[u32, (1000, 100, 10)] 109.6 µs 94.1 µs +16.42%
chunked_dict_primitive_canonical_into[u32, (1000, 100, 100)] 845.1 µs 724.1 µs +16.7%
chunked_dict_primitive_canonical_into[u32, (1000, 1000, 10)] 128.6 µs 112.9 µs +13.84%
chunked_dict_primitive_canonical_into[u32, (1000, 1000, 100)] 1,004.3 µs 880.1 µs +14.12%
chunked_dict_primitive_canonical_into[u64, (1000, 10, 10)] 135.9 µs 120.2 µs +13.1%
chunked_dict_primitive_canonical_into[u64, (1000, 10, 100)] 1,105 µs 978 µs +12.99%
chunked_dict_primitive_canonical_into[u64, (1000, 100, 10)] 137.7 µs 122.1 µs +12.78%
... ... ... ... ...

ℹ️ Only the first 20 benchmarks are displayed. Go to the app to view all benchmarks.

Footnotes

  1. No successful run was found on develop (d4bab58) during the generation of this report, so 60ce4b5 was used instead as the comparison base. There might be some changes unrelated to this pull request in this report.

  2. 29 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

@codecov
Copy link

codecov bot commented Oct 27, 2025

Codecov Report

❌ Patch coverage is 65.99462% with 253 lines in your changes missing coverage. Please review.
✅ Project coverage is 85.58%. Comparing base (60ce4b5) to head (d042d45).
⚠️ Report is 4 commits behind head on develop.

Files with missing lines Patch % Lines
vortex-array/src/compute/arrays/arithmetic.rs 44.31% 142 Missing ⚠️
vortex-compute/src/arithmetic/mod.rs 50.00% 21 Missing ⚠️
vortex-compute/src/arithmetic/buffer_checked.rs 88.72% 15 Missing ⚠️
vortex-vector/src/primitive/vector.rs 0.00% 14 Missing ⚠️
vortex-vector/src/primitive/vector_mut.rs 0.00% 14 Missing ⚠️
vortex-compute/src/arithmetic/pvector.rs 84.81% 12 Missing ⚠️
vortex-compute/src/arithmetic/buffer.rs 84.37% 10 Missing ⚠️
vortex-array/src/array/operator.rs 46.15% 7 Missing ⚠️
vortex-compute/src/logical/and.rs 0.00% 6 Missing ⚠️
vortex-compute/src/logical/or.rs 0.00% 6 Missing ⚠️
... and 2 more

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@gatesn gatesn requested a review from connortsui20 October 28, 2025 13:51
Comment on lines +58 to +60
// TODO(ngates): should we automatically cast non-null to nullable if required?
assert!(matches!(lhs.dtype(), DType::Primitive(..)));
assert_eq!(lhs.dtype(), rhs.dtype());
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is just a question: how do we handle add where values can be null currently? Do we even need to worry about this if we are just taking the OR of the masks?

Comment on lines +84 to +89
#[allow(clippy::mem_forget)]
static ENCODINGS: LazyLock<EnumMap<ArithmeticOperator, EncodingRef>> = LazyLock::new(|| {
enum_map! {
operator => ArithmeticEncoding { operator }.to_encoding(),
}
});
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure I understand the purpose of this? Could you instead just implement From<ArithmeticOperator> for ArithmeticEncoding?

Comment on lines +131 to +134
fn array_hash<H: Hasher>(array: &ArithmeticArray, state: &mut H, precision: Precision) {
array.lhs.array_hash(state, precision);
array.rhs.array_hash(state, precision);
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also hash the encoding since it contains which operation?

Signed-off-by: Nicholas Gates <nick@nickgates.com>
Copy link
Contributor

@connortsui20 connortsui20 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Signed-off-by: Nicholas Gates <nick@nickgates.com>
@gatesn gatesn merged commit fed289c into develop Oct 28, 2025
38 of 39 checks passed
@gatesn gatesn deleted the ngates/arithmetic-buffer-compute branch October 28, 2025 20:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

feature Release label indicating a new feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants