Panic when accessing RecordBatch from pyarrow

[adriangb]

This was tricky to pinpoint but I think I've got a MRE, and I'm guessing the issue is in PyArrowConvert. See:

• Repo with code demonstrating that the issue exists when passing in data from Python but not when using pure Rust: https://github.com/adriangb/misc/tree/11c245a5850582c15ebd654e34847e04a0668f1e
• A CI run confirming this finding and with a bit of debug info: https://github.com/adriangb/misc/actions/runs/4067509464/jobs/7004939916

[tustvold]

I don't appear to have access to that repo

[adriangb]

Apologies just made it public

[tustvold]

I will add this to my list to look into, thank you for the report

[adriangb]

Thanks for fixing this so quickly!

[tustvold]

label_issue.py automatically added labels {'arrow'} from #3675