MapArray does not work with RecordBatchWriter (fix: update to arrow-rs 37)

[balbok0]

Environment

Delta-rs version: 0.9.0

Binding:

Environment:

• Cloud provider: N/A
• OS: Linux (Arch, Kernel 5.15.106)
• Other: N/A

---

Bug

What happened: When writing MapArray to a parquet file following error happens

thread 'main' panicked at 'not implemented: Take not supported for data type Map(Field { name: "entries", data_type: Struct([Field { name: "keys", data_type: Utf8, nullable: false, dict_id: 0, dict_is_ordered: false, metadata: {} }, Field { name: "values", data_type: Binary, nullable: false, dict_id: 0, dict_is_ordered: false, metadata: {} }]), nullable: true, dict_id: 0, dict_is_ordered: false, metadata: {} }, false)', /home/balbok/.cargo/registry/src/github.com-1ecc6299db9ec823/arrow-select-36.0.0/src/take.rs:211:14

What you expected to happen: Successful write

How to reproduce it: Take the record batch test from this PR and attempt to write it with RecordBatch.

More details: Fixed in arrow 37 on this PR by @wjones127. I took a stab on updating the repo to arrow 37 on this forked branch and confirmed that record batch writer works with MapArray. I'm new to Rust though, so probably not a production code.

[rtyler]

This is a known issue, see: #1226

Unfortunately we're kind of stuck until Data Fusion upgrades to arrow 37 in their next release. The web of dependencies here is ... not ideal 😄

[balbok0]

Makes sense, thanks for the explanation, and sorry for the duplicate!