perf: set compat_level when calling to_arrow

[ruihe774]

This depends on pola-rs/polars#17371.

In my naive benchmark, with large string columns, it improves the speed to ~10x.

[ruihe774]

I'm still wondering how to ensure forward compatibility with future=True. When a old version of pyo3-polars is used together with a newer version of python polars which has introduced incompatible arrow data structure updates, it can be broken. One solution is to check the version of polars before setting future=True, and another solution is to introduce versioning like future=1.

As for backward compatibility, we can simply retry with future=True unset if it fails.

[ruihe774]

pola-rs/polars#17421

[ruihe774]

pola-rs/polars#17421

I've made this PR work with it. In PySeries::into_py and PySeries::extract_bound, newest compatibility level supported by both sides is negotiated and selected automatically. In this way, both backward compatibility and forward compatibility are achieved.

[ritchie46]

Can we point this to the main branch until 0.42 is released.

[ruihe774]

Cargo only recognizes patches defined in the root project and not in dependencies. pyo3-polars is meant to be a dependency of projects that use it. So unless they declare patches in their Cargo.toml, patches are not in effect.

Patches declared here are only for testing. We can only wait for 0.42.

[ritchie46]

Alright. 0.42 is planned in ca. 2 weeks.