Support building on Stable Rust

[robert3005]

Currently we use a number of nightly features. We should evaluate whether any of them is strictly necessary and whether there's a simpler alternative. More importantly removing one nightly feature doesn't offer an improvement unless we can get rid of all of them. Hence, if anyone wants to look into this issue they should start with generic_const_exprs

• vortex-buffer
• #![feature(min_specialization)]
• #![feature(trusted_len)]
• encodings/fastlanes
• #![feature(generic_const_exprs)]

[gatesn]

A workable alternative here is that we actually provide a Rust API over Vortex FFI.

For many out-of-the-box use-cases (I.e. not doing any custom trait impls) this would be sufficiently powerful.

[a10y]

Hmm, so an FFI-portable API over something like abi_stable could be interesting. But I feel like even the simpler case of using Vortex inside another codebase without forcing the entire project to nightly toolchain is also very useful. DataFusion and LanceDB both build on stable, we're probably putting ourselves at an adoption disadvantage by forcing users to nightly.

[robert3005]

I think if we dynamically load encodings then we can have the core library be built on stable and then all the unstable use cases can be distributed as shared library if consumers don’t want use nightly

[gatesn]

The big ones imo are trusted_len and min_specialization.

These are used in the stdlib to make vectors fast. We use the almost identically to make buffers fast. Removing them means our buffers aren't fast 🤷‍♂️

[robert3005]

For that we can do the thing that arrow does and add some unsafe functions since trustedlen is a marker interface. However, users would need to know to call it. Right now just extend has the correct and optimal behaviour without any new apis

[a10y]

At the end of last week I started working on another part of this, which I think is slightly controversial so I want to get people on board before I spend more time on it: I think we should move our portable_simd code over to instead use architecture-specific kernels, which get selected based on runtime feature detection (something that is already a problem which I made a ticket for in #3580).

Currently there's only one function we need to port, which is the TakeFn for PrimitiveArray. This mostly affects the performance of dict-encoded primitive arrays and is tracked via the dict_encode microbenchmark.

In the case of this particular function, the only target I'd expect a performance boost from SIMD is AVX2 and possibly SSE2, since the gather instruction is where most of the juice comes from, and ARM ISAs don't seem to have an equivalent.

According to CodSpeed the hand-rolled avx2 kernels can improve dict decode performance somewhere between 50-75%. Right now it adds a few hundred lines of ptype-specific code, I think we can potentially centralize that via some macro magic.

I can get the PR over the line with its current strategy, but as a checkpoint wanted to clarify that there are potential alternatives/downsides:

1. I know that we also want to have a world where compute functions can be written in Rust or other languages and linked into end-user binaries, and that there is probably a way with portable_simd and multiversion to build separate nightly crates as cdylib and then let library users bring them in with a build.rs supplying sufficient linker args.
2. Maintenance of hand-rolled SIMD kernels is harder than maintenance of portable_simd code

I think for (1) I'd say that this fails to maintain ease of use for the common case of Rust app builders wanting to use Vortex Rust bindings. It's not unprecedented though, linking is used by other crates particularly all of the *-sys family of crates but those generally link other libraries rather than linking Rust crate outputs. It creates some complexity on the build system to pull this off.

For (2), I think we can hedge against maintenance burden with sufficient unit tests, or possibly externalizing some of these common kernels out to a new crate. I'm trying to find good examples but surprisingly coming up a bit short. The best bet I can find is miri so long as you're running on the right architecture since that interprets all instructions including the SIMD intrinsics for each platform.

[robert3005]

After recent round of changes we are down to 3 unstable flags. vortex-buffer ones are easy to eliminate but provide a strictly worse user experience where calling extend on a BufferMut would be strictly worse than some new method like extend_from_trusted_len.

For fastlanes we could abandon all safety and get rid of generic const exprs but not convinced we want it

[gatesn]

I don't hate abandoning the const exprs for fastlanes since the callers have to basically do unsafe transmute of slices into arrays in most cases anyway

[a10y]

I've opened spiraldb/fastlanes#65 to remove the use of generic_const_expr and target fastlanes to stable rust, and I've verified that no published crate which depends on fastlanes uses the safe APIs anyway.