chore(rust): separate adbc_core and adbc_datafusion arrow version requirements

[eitsupi]

Context #2739
Closes #2524

Since datafusion tends to require newer versions of arrow than adbc_core, the version of arrow used by adbc_core will be bumped by adbc_datafusion when compiling the entire workspace.

This PullRequest separates the arrow requirements for adbc_datafusion and others.
Since the newer arrow that meets adbc_datafusion requirements will be used when the entire workspace is compiled, a job to test other than adbc_datafuion by generating a file that locks the lower requirements on CI is added.
It make sure adbc_core will work with the older arrow.

[eitsupi]

Oops, I intended to check the behavior on my fork, but obviously I made a PullRequest to the wrong place...

[lidavidm]

Thanks, LGTM. If it's possible it would be nice to see if the Arrow version could be relaxed further as Felipe suggested.

[eitsupi]

The workspace arrow requirement has been lowered to 53.1.0 (this is copied from #2525 and I have not examined the validity of this choice. It could be lowered more)

I believe this will likely lower the MSRV as well, but I have not done the work because it is tedious to go through the MSRVs for each dependent package. (So far, half 2.6.0 requires 1.81)
If requested, we can look into lowering the MSRV later.

[lidavidm]

This looks good to me. I agree we can punt on changing the MSRV until later. @felipecrv does this solve your problem?

[felipecrv]

Yes @lidavidm, this solves my problem.