CI: Force newer S3FS

[lithomas1]

• closes CI: newer version of s3fs is pulled causing builds to fail #48271 (Replace xxxx with the Github issue number)
• Tests added and passed if fixing a bug or adding a new feature
• All code checks passed.
• Added type annotations to new arguments/methods/functions.
• Added an entry in the latest doc/source/whatsnew/vX.X.X.rst file if fixing a bug or adding a new feature.

[phofl]

Please remove aiobotocore if not pinned, it's not a dependency of us

[mroeschke]

Could we enforce botocore>=1.23.0 instead? Looks like the six changes occured there: boto/botocore@5d4ecca

Not totally ideal IMO that this s3fs pinning in these files is higher than our minimum version.

[lithomas1]

Is there a reason to pin down s3fs?

mamba is pulling really old versions of s3fs by chance(I think it's happening because old s3fs are not pinning down aiobotocore, allowing for mamba to pull down the newest aiobotocore versions which are not actually compatible with s3fs). Really old s3fs working with the newest aiobotocore is more luck than anything else.

This PR is just forcing mamba not to do this by setting a lower bound to a s3fs that pins down aiobotocore to a version it can support. The minimum version is still getting tested in the min version build so I don't think we're losing coverage.

Could we enforce botocore>=1.23.0 instead? Looks like the six changes occured there: boto/botocore@5d4ecca

I haven't tested this, but I'm pretty sure aiobotocore pins/controls the botocore version(IIUC it messes around with botocore internals so they have to pin down to a specific botocore version), so doing that would probably just result in an unsolvable environment.

(As for why we care about aiobotocore, its a dependency of s3fs)

I agree with you on the point of bumping s3fs though(I think we should probably do it ASAP for 2.0).

[phofl]

Yeah the actual issue is that our minimum version of s3fs is not pinning aiobotocore, causing failures, hence we have to pin aiobotocore too causing the python 3.10 issues…

[phofl]

Yeah the actual issue is that our minimum version of s3fs is not pinning aiobotocore, causing failures, hence we have to pin aiobotocore too causing the python 3.10 issues…

[mroeschke]

I'll check the environment solving by lower pinning botocore quickly in #48297

Is there a reason to pin down s3fs?

Not really. I think it was May when I updated the min version of several packages using a 1-year-release-prior rule of thumb. IMO I think it would be okay to bump s3fs for 1.5 to 2021.08.0 since we advertise our min versions of optional dependencies as the ones we test with, and lower versions may very well still work.

[phofl]

Yeah then let’s bump this for the rc, if the solve fails

[phofl]

#48299, if this works out then we can close here