After merging your PR, Conbench analyzed the 7 benchmarking runs that have been run so far on merge-commit 1815a67.

Benchmarks with performance regressions

There was 1 possible performance regression, according to the lookback z-score method.

Benchmarks with regressions:

• Commit Run on ec2-m5-4xlarge-us-east-2 at 2024-06-27 00:42:01Z
  • partitioned-dataset-filter (R) with dataset=dataset-taxi-parquet, language=R, query=vignette

All benchmark runs analyzed:

• Commit Run on ec2-c6a-4xlarge-us-east-2 at 2024-06-26 17:09:53Z
• Commit Run on test-mac-arm at 2024-06-26 19:52:15Z
• Commit Run on ursa-thinkcentre-m75q at 2024-06-26 20:40:40Z
• Commit Run on ec2-r5-4xlarge-us-east-2 at 2024-06-27 00:46:54Z
• Commit Run on arm64-t4g-linux-compute at 2024-06-26 23:54:19Z
• Commit Run on arm64-m6g-linux-compute at 2024-06-27 00:42:45Z
• Commit Run on ec2-m5-4xlarge-us-east-2 at 2024-06-27 00:42:01Z

Unstable benchmarks with performance regressions

The following benchmark results indicate a possible performance regression, but are known to sometimes produce false positives when applying the lookback z-score analysis.

• Commit Run on ec2-c6a-4xlarge-us-east-2 at 2024-06-26 17:09:53Z

  • DoubleColumnMajorMatrixConversionFixture (C++) with params=/ConvertToSparseCSCMatrixInt32, source=cpp-micro, suite=arrow-tensor-conversion-benchmark
  • DoubleStridedMatrixConversionFixture (C++) with params=/ConvertToSparseCSCMatrixInt32, source=cpp-micro, suite=arrow-tensor-conversion-benchmark
  • DoubleRowMajorMatrixConversionFixture (C++) with params=/ConvertToSparseCSCMatrixInt32, source=cpp-micro, suite=arrow-tensor-conversion-benchmark

• Commit Run on ursa-thinkcentre-m75q at 2024-06-26 20:40:40Z

  • BM_DeltaLengthDecodingByteArray (C++) with params=max-string-length:64/batch-size:2048, source=cpp-micro, suite=parquet-encoding-benchmark
  • FloatColumnMajorTensorConversionFixture (C++) with params=/ConvertToSparseCOOTensorInt64, source=cpp-micro, suite=arrow-tensor-conversion-benchmark
  • FloatRowMajorTensorConversionFixture (C++) with params=/ConvertToSparseCOOTensorInt64, source=cpp-micro, suite=arrow-tensor-conversion-benchmark
  • BM_DeltaLengthDecodingSpacedByteArray (C++) with params=max-string-length:64/batch-size:512, source=cpp-micro, suite=parquet-encoding-benchmark
  • FilterStringFilterNoNulls (C++) with params=524288/0, source=cpp-micro, suite=arrow-compute-vector-selection-benchmark
  • BM_DeltaLengthDecodingSpacedByteArray (C++) with params=max-string-length:64/batch-size:2048, source=cpp-micro, suite=parquet-encoding-benchmark
  • ReverseSetBitRunReader (C++) with params=0, source=cpp-micro, suite=arrow-bit-util-benchmark
  • ArrayArrayKernel (C++) with params=<Subtract, Int32Type>/size:524288/inverse_null_proportion:0, source=cpp-micro, suite=arrow-compute-scalar-arithmetic-benchmark

• Commit Run on ec2-m5-4xlarge-us-east-2 at 2024-06-27 00:42:01Z

  • ArrayArrayKernel (C++) with params=/1048576/100, source=cpp-micro, suite=arrow-compute-scalar-boolean-benchmark
  • ArrayArrayKernel (C++) with params=/32768/2, source=cpp-micro, suite=arrow-compute-scalar-boolean-benchmark
  • BinaryMathOpAdd256 (C++) with source=cpp-micro, suite=arrow-decimal-benchmark
  • ArrayRangeEqualsBoolean (C++) with params=32768/0, source=cpp-micro, suite=arrow-compare-benchmark
  • ArrayArrayKernel (C++) with params=/32768/10, source=cpp-micro, suite=arrow-compute-scalar-boolean-benchmark
  • BitmapEqualsWithOffset (C++) with params=8192, source=cpp-micro, suite=arrow-bit-util-benchmark
  • ArrayArrayKernel (C++) with params=/1048576/10000, source=cpp-micro, suite=arrow-compute-scalar-boolean-benchmark
  • ArrayArrayKernel (C++) with params=/1048576/1, source=cpp-micro, suite=arrow-compute-scalar-boolean-benchmark
  • ChunkCSVFlightsExample (C++) with source=cpp-micro, suite=arrow-csv-parser-benchmark
  • BenchmarkTemporalRounding (C++) with params=<RoundTemporal, non_zoned, round_1_week>/1048576/0, source=cpp-micro, suite=arrow-compute-scalar-temporal-benchmark
  • ArrayArrayKernel (C++) with params=/32768/1, source=cpp-micro, suite=arrow-compute-scalar-boolean-benchmark
  • ArrayRankStringNarrow (C++) with params=32768/2/tiebreaker:3, source=cpp-micro, suite=arrow-compute-vector-sort-benchmark
  • ArrayArrayKernel (C++) with params=/32768/10000, source=cpp-micro, suite=arrow-compute-scalar-boolean-benchmark
  • WriteCsvStringWithQuote (C++) with params=1, source=cpp-micro, suite=arrow-csv-writer-benchmark
  • ChunkedArraySortIndicesInt64Narrow (C++) with params=32768/1, source=cpp-micro, suite=arrow-compute-vector-sort-benchmark
  • BM_DecodeArrowBooleanPlain (C++) with params=DecodeArrowNonNull/32768, source=cpp-micro, suite=parquet-encoding-benchmark
  • ArrayArrayKernel (C++) with params=/1048576/10, source=cpp-micro, suite=arrow-compute-scalar-boolean-benchmark
  • BM_DecodeArrowBooleanPlain (C++) with params=DecodeArrow/32768, source=cpp-micro, suite=parquet-encoding-benchmark
  • CheckIndexBoundsInt32 (C++) with params=32768/0, source=cpp-micro, suite=arrow-int-util-benchmark
  • CheckIndexBoundsInt32 (C++) with params=1048576/0, source=cpp-micro, suite=arrow-int-util-benchmark
  • ArrayRankStringNarrow (C++) with params=32768/2/tiebreaker:2, source=cpp-micro, suite=arrow-compute-vector-sort-benchmark
  • BM_DecodeArrowBooleanPlain (C++) with params=DecodeArrow/65536, source=cpp-micro, suite=parquet-encoding-benchmark
  • CheckIndexBoundsUInt32 (C++) with params=1048576/1000, source=cpp-micro, suite=arrow-int-util-benchmark
  • CheckIndexBoundsUInt32 (C++) with params=32768/1000, source=cpp-micro, suite=arrow-int-util-benchmark
  • ArrayArrayKernel (C++) with params=/32768/100, source=cpp-micro, suite=arrow-compute-scalar-boolean-benchmark
  • BM_DecodeArrowBooleanPlain (C++) with params=DecodeArrowNonNull/65536, source=cpp-micro, suite=parquet-encoding-benchmark
  • WriteCsvStringRejectQuote (C++) with params=50, source=cpp-micro, suite=arrow-csv-writer-benchmark
  • ArrayRankStringNarrow (C++) with params=32768/10/tiebreaker:2, source=cpp-micro, suite=arrow-compute-vector-sort-benchmark
  • WriteCsvStringWithQuote (C++) with params=0, source=cpp-micro, suite=arrow-csv-writer-benchmark
  • ArrayArrayKernel (C++) with params=/1048576/2, source=cpp-micro, suite=arrow-compute-scalar-boolean-benchmark
  • ChunkedArraySortIndicesInt64Wide (C++) with params=32768/1, source=cpp-micro, suite=arrow-compute-vector-sort-benchmark