Introduce codegen optimizations for primitives and fixed-sized-arrays

[jleibs]

What

This implements 2 optimizations:

• The first is ArrowBuffer optimization returns an inner Buffer directly when we know that the type itself it just an array of primitives. This is useful for zero-copy returns for dense data such as Tensors.
• The second is the optimizations from: Proof-of-concept: Hand-crafted optimizations to pave the way forward for code-gen #2954 . For this, we identify cases where we know the inner arrays are not nullable and instead of using validity-iterators map directly to slices.

Significant speedups for batch queries:

TODO:

• We should be able to check that the contents don't actually contain a validity map with non-nulls and return a deserialization error in that case.
• Add handling for other ArrowBuffer types.

Checklist

• I have read and agree to Contributor Guide and the Code of Conduct
• I've included a screenshot or gif (if applicable)
• I have tested demo.rerun.io (if applicable)

• PR Build Summary
• Docs preview
• Examples preview

[emilk]

I think ArrowBuffer::len is dangerously misnamed (even though it follows what arrow2 does), otherwise LGTM!

[Wumpf]

quite unfortunate that we need this both on the arrow and on the idl type :(

[jleibs]

Yeah, I was bothered by that as well. I'd like to refactor the codegen to always use our own type-wrappers, but that turned into a bigger change.

[Wumpf]

I made this mistake in C++ codegen for a while; going to our own type wherever possible (which is everywhere for c++ but won't work for Rust) made it a lot better :)

[Wumpf]

why not obj.fields[0].is_nullable, then you don't need the arrow registry

[jleibs]

Because if it's a Struct or Union type we can't apply this optimization either. Renaming the check /adding documents to clarify: should_optimize_buffer_slice_deserialize