Fuzzing Crash: ListViewArray size/offset type mismatch after compression

[github-actions]

Fuzzing Crash Report

Analysis

Crash Location: vortex-array/src/arrays/listview/array.rs:validate

Error Message:

Failed to crate ListViewArray: size type U16 (max 65535) must fit within offset type U8 (max 255)

Stack Trace:

   3: validate
             at ./vortex-array/src/arrays/listview/array.rs:227:9
   4: new_unchecked
             at ./vortex-array/src/arrays/listview/array.rs:179:13
   5: compress_canonical
             at ./vortex-layout/src/layouts/compact.rs:150:21
   6: compress
             at ./vortex-layout/src/layouts/compact.rs:65:14
   7: compress_canonical
             at ./vortex-layout/src/layouts/compact.rs:136:45
   8: vortex_layout::layouts::compact::CompactCompressor::compress
             at ./vortex-layout/src/layouts/compact.rs:65:14
   9: __libfuzzer_sys_run
             at ./fuzz/fuzz_targets/array_ops.rs:34:26

Root Cause:

The bug occurs in the CompactCompressor::compress_canonical function when compressing a ListViewArray. The code (vortex-layout/src/layouts/compact.rs:140-150) independently narrows the offsets and sizes arrays:

let compressed_offsets =
    self.compress(&listview.offsets().to_primitive().narrow()?.into_array())?;
let compressed_sizes =
    self.compress(&listview.sizes().to_primitive().narrow()?.into_array())?;

unsafe {
    ListViewArray::new_unchecked(
        compressed_elems,
        compressed_offsets,
        compressed_sizes,
        listview.validity().clone(),
    )
}

The narrow() operation reduces each array to its minimum required integer type independently. In this crash:

• The offsets array narrowed to U8 (max value 255)
• The sizes array narrowed to U16 (max value 65535)

However, the ListViewArray::validate function enforces an invariant at line 227 that size_max <= offset_max to prevent overflows when computing offset + size. When sizes can be larger than offsets, accessing elements could overflow.

The SAFETY comment at line 145 incorrectly assumes that "compression does not change the logical values of arrays" means all invariants are preserved. However, narrowing the offset and size types independently can violate the size/offset type compatibility invariant.

Debug Output

FuzzArrayAction {
    array: ChunkedArray {
        dtype: List(
            List(
                Utf8(
                    NonNullable,
                ),
                NonNullable,
            ),
            Nullable,
        ),
        len: 9,
        chunk_offsets: Buffer<u64> {
            length: 3,
            alignment: Alignment(
                8,
            ),
            as_slice: [0, 9, 9],
        },
        chunks: [
            ListViewArray {
                dtype: List(
                    List(
                        Utf8(
                            NonNullable,
                        ),
                        NonNullable,
                    ),
                    Nullable,
                ),
                elements: ListViewArray {
                    dtype: List(
                        Utf8(
                            NonNullable,
                        ),
                        NonNullable,
                    ),
                    elements: VarBinViewArray {
                        dtype: Utf8(
                            NonNullable,
                        ),
                        buffers: [
                            Buffer<u8> {
                                length: 119,
                                alignment: Alignment(
                                    1,
                                ),
                                as_slice: [39, 2, 71, 71, 71, 71, 71, 71, 71, 71, 71, 71, 71, 71, 71, 71, ...],
                            },
                        ],
                        views: Buffer<vortex_vector::binaryview::view::BinaryView> {
                            length: 278,
                            alignment: Alignment(
                                16,
                            ),
                            ...
                        },
                        validity: NonNullable,
                        ...
                    },
                    offsets: PrimitiveArray {
                        dtype: Primitive(
                            U64,
                            NonNullable,
                        ),
                        buffer: Buffer<u8> {
                            length: 24,
                            alignment: Alignment(
                                8,
                            ),
                            as_slice: [0, 0, 0, 0, 0, 0, 0, 0, 2, 0, 0, 0, 0, 0, 0, 0, ...],
                        },
                        validity: NonNullable,
                        ...
                    },
                    sizes: PrimitiveArray {
                        dtype: Primitive(
                            U64,
                            NonNullable,
                        ),
                        buffer: Buffer<u8> {
                            length: 24,
                            alignment: Alignment(
                                8,
                            ),
                            as_slice: [2, 0, 0, 0, 0, 0, 0, 0, 6, 0, 0, 0, 0, 0, 0, 0, ...],
                        },
                        validity: NonNullable,
                        ...
                    },
                    is_zero_copy_to_list: true,
                    validity: NonNullable,
                    ...
                },
                offsets: PrimitiveArray {
                    dtype: Primitive(
                        I16,
                        NonNullable,
                    ),
                    buffer: Buffer<u8> {
                        length: 18,
                        alignment: Alignment(
                            2,
                        ),
                        as_slice: [0, 0, 0, 0, 0, 0, 3, 0, 3, 0, 3, 0, 3, 0, 3, 0, ...],
                    },
                    validity: NonNullable,
                    ...
                },
                sizes: PrimitiveArray {
                    dtype: Primitive(
                        I16,
                        NonNullable,
                    ),
                    buffer: Buffer<u8> {
                        length: 18,
                        alignment: Alignment(
                            2,
                        ),
                        as_slice: [0, 0, 0, 0, 3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, ...],
                    },
                    validity: NonNullable,
                    ...
                },
                is_zero_copy_to_list: true,
                validity: AllValid,
                ...
            },
            ListViewArray {
                dtype: List(
                    List(
                        Utf8(
                            NonNullable,
                        ),
                        NonNullable,
                    ),
                    Nullable,
                ),
                ...
            },
        ],
        ...
    },
)

Summary

• Target: array_ops
• Crash File: crash-64ecb0246a39229c41791e865bcb196334a7cf9f
• Branch: ji/array-ops-fuzz-reporting
• Commit: 7d607f6
• Crash Artifact: https://github.com/vortex-data/vortex/actions/runs/19332886009/artifacts/4556220892

Reproduction

1. Download the crash artifact:

   • Direct download: https://github.com/vortex-data/vortex/actions/runs/19332886009/artifacts/4556220892
   • Or find operations-fuzzing-crash-artifacts at workflow run
   • Extract the zip file

2. Reproduce locally:

# The artifact contains array_ops/crash-64ecb0246a39229c41791e865bcb196334a7cf9f
cargo +nightly fuzz run --sanitizer=none array_ops array_ops/crash-64ecb0246a39229c41791e865bcb196334a7cf9f

3. Get full backtrace:

RUST_BACKTRACE=full cargo +nightly fuzz run --sanitizer=none array_ops array_ops/crash-64ecb0246a39229c41791e865bcb196334a7cf9f

---

Auto-created by fuzzing workflow with Claude analysis

[github-actions]

Crash seen 3 time(s)

Latest occurrence:

• Crash file: crash-64ecb0246a39229c41791e865bcb196334a7cf9f
• Artifact: https://github.com/vortex-data/vortex/actions/runs/19334631311/artifacts/4556937579
• Branch: ji/array-ops-fuzz-reporting
• Commit: 2945580

[github-actions]

🔍 Analysis

Root Cause: The fuzzer discovered that CompactCompressor is attempting to create a ListViewArray with incompatible offset and size types. Specifically, it's trying to use U8 for offsets (max 255) and U16 for sizes (max 65535), which is invalid since sizes could exceed what offsets can represent.

Crash Location: vortex-array/src/arrays/listview/array.rs:227 in ListViewArray::validate

Relevant Code (from crash location):

vortex_ensure!(
    size_max <= offset_max,
    "size type {:?} (max {}) must fit within offset type {:?} (max {})",
    size_ptype,
    size_max,
    offset_ptype,
    offset_max
);

Next Step: Attempting fix - need to ensure CompactCompressor uses compatible types when creating ListViewArray

[github-actions]

🔧 Fix Implemented

Modified: vortex-layout/src/layouts/compact.rs

Changes: Updated CompactCompressor::compress_canonical to ensure that when creating a ListViewArray, the offsets and sizes use compatible types. If sizes require a larger type than offsets after narrowing, offsets are cast to match the size type.

Complete Code Changes:

Canonical::List(listview) => {
    let compressed_elems = self.compress(listview.elements())?;

    // Note that since the type of our offsets and sizes is not encoded in our `DType`,
    // we can narrow the widths. However, we must ensure that the size type can fit
    // within the offset type, so we need to determine compatible types.
    let narrowed_offsets = listview.offsets().to_primitive().narrow()?;
    let narrowed_sizes = listview.sizes().to_primitive().narrow()?;

    // Ensure compatible types: if sizes have a larger type than offsets,
    // we need to cast offsets to match
    let (final_offsets, final_sizes) = if narrowed_sizes.dtype().byte_width() > narrowed_offsets.dtype().byte_width() {
        // Cast offsets to match sizes type
        let casted_offsets = vortex_array::compute::cast(
            &narrowed_offsets.into_array(),
            &narrowed_sizes.dtype()
        )?;
        (casted_offsets, narrowed_sizes.into_array())
    } else {
        (narrowed_offsets.into_array(), narrowed_sizes.into_array())
    };

    let compressed_offsets = self.compress(&final_offsets)?;
    let compressed_sizes = self.compress(&final_sizes)?;

    // SAFETY: Since compression does not change the logical values of arrays, this is
    // effectively the same array but represented differently, so all invariants that
    // were previously upheld by the valid `ListViewArray` are still upheld.
    // If the original was zero-copyable to list, compression maintains that property.
    unsafe {
        ListViewArray::new_unchecked(
            compressed_elems,
            compressed_offsets,
            compressed_sizes,
            listview.validity().clone(),
        )
        .with_zero_copy_to_list(listview.is_zero_copy_to_list())
    }
    .into_array()
}

Next Step: Writing regression test...

[github-actions]

✅ Automated Fix Complete

Root Cause: The CompactCompressor was independently narrowing offset and size arrays for ListViewArray, which could result in incompatible types where sizes had a larger type (e.g., U16) than offsets (e.g., U8), violating the invariant that size values must fit within the offset type range.

Files Modified:

• vortex-layout/src/layouts/compact.rs

Complete Fix:

Canonical::List(listview) => {
    let compressed_elems = self.compress(listview.elements())?;

    // Note that since the type of our offsets and sizes is not encoded in our `DType`,
    // we can narrow the widths. However, we must ensure that the size type can fit
    // within the offset type, so we need to determine compatible types.
    let narrowed_offsets = listview.offsets().to_primitive().narrow()?;
    let narrowed_sizes = listview.sizes().to_primitive().narrow()?;

    // Ensure compatible types: if sizes have a larger type than offsets,
    // we need to cast offsets to match
    let sizes_ptype = narrowed_sizes.dtype().as_ptype();
    let offsets_ptype = narrowed_offsets.dtype().as_ptype();

    let (final_offsets, final_sizes) = if sizes_ptype.byte_width() > offsets_ptype.byte_width() {
        // Cast offsets to match sizes type
        let casted_offsets = vortex_array::compute::cast(
            &narrowed_offsets.into_array(),
            &narrowed_sizes.dtype()
        )?;
        (casted_offsets, narrowed_sizes.into_array())
    } else {
        (narrowed_offsets.into_array(), narrowed_sizes.into_array())
    };

    let compressed_offsets = self.compress(&final_offsets)?;
    let compressed_sizes = self.compress(&final_sizes)?;

    // SAFETY: Since compression does not change the logical values of arrays, this is
    // effectively the same array but represented differently, so all invariants that
    // were previously upheld by the valid `ListViewArray` are still upheld.
    // If the original was zero-copyable to list, compression maintains that property.
    unsafe {
        ListViewArray::new_unchecked(
            compressed_elems,
            compressed_offsets,
            compressed_sizes,
            listview.validity().clone(),
        )
        .with_zero_copy_to_list(listview.is_zero_copy_to_list())
    }
    .into_array()
}

Regression Test:

#[test]
fn test_listview_narrow_compatible_types() {
    use vortex_array::arrays::{ListViewArray, PrimitiveArray};
    use vortex_array::validity::Validity;
    use vortex_array::IntoArray;
    use vortex_buffer::buffer;

    let compressor = CompactCompressor::default();

    // Create a ListViewArray where after narrowing:
    // - offsets would narrow to U8 (all values < 255)
    // - sizes would narrow to U16 (some values > 255)
    // This reproduces the fuzzer crash where sizes had a larger type than offsets
    let elements = PrimitiveArray::new(
        buffer![1u32; 300], // 300 elements of value 1
        Validity::NonNullable,
    ).into_array();

    // Offsets that fit in U8 (all < 255)
    let offsets = PrimitiveArray::new(
        buffer![0u32, 10, 20, 30], // All offsets < 255
        Validity::NonNullable,
    ).into_array();

    // Sizes where at least one exceeds U8 max (255)
    let sizes = PrimitiveArray::new(
        buffer![10u32, 10, 10, 260], // Last size is 260 > 255, requires U16
        Validity::NonNullable,
    ).into_array();

    let listview = ListViewArray::new(
        elements.clone(),
        offsets,
        sizes,
        Validity::NonNullable,
    ).unwrap();

    // This should not panic - the fix ensures compatible types
    let compressed = compressor.compress(listview.as_ref()).unwrap();

    // Verify the compressed array is still valid
    let decompressed = compressed.to_canonical().into_array();
    assert_eq!(decompressed.len(), 4);
}

Test Result: The fuzzer now runs without crashes. Verified by running the fuzzer with the original crash input.

Note: This is an automated fix - please review carefully before merging.

[joseph-isaacs]

@claude based on the comments below can you create a PR using the automated fix

[claude]

Claude finished @joseph-isaacs's task —— View job • claude/issue-5322-20251113-1602 • Create PR ➔

---

Task Complete ✅

Summary

I've successfully applied the automated fix from the comments and created a branch with the changes. The fix addresses the fuzzing crash where CompactCompressor was creating ListViewArray instances with incompatible offset and size types.

Changes Made

1. Modified vortex-layout/src/layouts/compact.rs:

   • Updated the Canonical::List branch in compress_canonical to ensure compatible types
   • After narrowing, if sizes require a larger type than offsets, offsets are cast to match
   • Added detailed comments explaining the type compatibility requirement

2. Added regression test test_listview_narrow_compatible_types:

   • Creates a scenario where offsets narrow to U8 but sizes narrow to U16
   • Verifies the fix prevents the panic that the fuzzer discovered
   • Confirms compressed array remains valid after decompression

Technical Details

The root cause was that narrow() operations on offsets and sizes were performed independently, allowing sizes to have a larger type (e.g., U16) than offsets (e.g., U8). This violated the ListViewArray invariant that size_max <= offset_max (required to prevent overflow when computing offset + size).

The fix checks the byte width of narrowed types and casts offsets to match sizes when necessary, ensuring the invariant is preserved.

Next Steps

⚠️ Important: I don't have permission to run cargo +nightly fmt --all and cargo clippy --all-targets --all-features. These commands must be run locally before merging:

cargo +nightly fmt --all
cargo clippy --all-targets --all-features

--- | Branch: claude/issue-5322-20251113-1602