Bump fsspec

[antonymilne]

Description

Resolves #1804.

A reminder that we should at some point we should:

• reconsider how we handle upper bounds for requirements
• consider using dependabot to handle this sort of thing automatically - would be nice to try and get Use dependabot to bump Python dependencies automatically kedro-viz#814 done and then implement it here also if it seems to work well for us

Checklist

• Read the contributing guidelines
• Opened this PR as a 'Draft Pull Request' if it is work-in-progress
• Updated the documentation to reflect the code changes
• Added a description of this change in the RELEASE.md file
• Added tests to cover my changes

[merelcht]

Any reason why you pinned it to 2022.7.1 and not just 2022.7?

[antonymilne]

2022.7.1 is the latest release that fixes bugs in 2022.7. Pinning to <2022.8 wouldn't make sense here since they're following calver.

[noklam]

reposting @datajoely comment here.

I appreciate we're taking a conservative approach here, but why not widen the scope to all of 2022 or beyond? We've not had any breaking issues since fsspec moved to CalVer and users only report this sort of issue when it's too late.

[antonymilne]

@datajoely:

I appreciate we're taking a conservative approach here, but why not widen the scope to all of 2022 or beyond? We've not had any breaking issues since fsspec moved to CalVer and users only report this sort of issue when it's too late.

This is a very good point I think. Personally I think we are generally too conservative with upper bounding dependencies at the moment and should consider changing our policy on it, but I'm pretty sure @idanov would disagree strongly with this 😅

When fsspec first introduced CalVer I think we allowed a 6 month forward-looking window. I'd be happy to go to <2023 or even consider no upper bound. But again I think @idanov would hate this.

@MerelTheisenQB - thoughts? I'm happy to go with whatever you think is best here. @deepyaman I know thinks about these things and so would have an opinion here also 🙏 Maybe the least controversial solution would be to go along with our current conservative strategy for now but reconsider the policy in general and try to push forward with something like dependabot to automatically bump dependencies.

[antonymilne]

Also @jenikovsky - as a kedro user, any thoughts on this? I see you ❤️ ed @datajoely's comment so would probably be in favour of a more relaxed upper bound so we we're not always playing catch up and relying on users to report when things aren't compatible? It's tricky to know what to do with these calver libraries.

[merelcht]

@MerelTheisenQB - thoughts? I'm happy to go with whatever you think is best here. @deepyaman I know thinks about these things and so would have an opinion here also 🙏 Maybe the least controversial solution would be to go along with our current conservative strategy for now but reconsider the policy in general and try to push forward with something like dependabot to automatically bump dependencies.

I think it's definitely worth revisiting our bumping strategy, but I'm not sure we can apply one strategy to all dependencies. fsspec is quite a fundamental dependency for our datasets so I'd be more careful in how loosely we pin it. We've also seen in the past that not all libraries stick to releasing non-breaking changes in major versions only, so not having any bounds will just lead to an even greater amount of issues related to version bumps in dependencies.

[datajoely]

Could we look at dependabot auto-opening PRs for this library? Presumably if the test suite passes we're good to go.

[noklam]

Thanks, I agree dependent bot will be a good fit here.

[Galileo-Galilei]

Can't agree more with using dependabot and being confident in your test suite which is very solid! These conflicts between kedro dependencies and other packages may be very annoying and it would be very nice to update upper bounds much more frequently.

[jenikovsky]

Keeping up using dependabot makes a lot of sense.
Thank you guys for your super quick action on this request. When could this change be roughly released?

[antonymilne]

Cool, thanks for the comments everyone. Let's do as I originally suggested then: keep this bound <=2022.7.1 and make a new ticket for us to try out dependabot.

@jenikovsky I think we should be doing a release pretty soon - within the next week or so would be my guess.

[jenikovsky]

@jenikovsky I think we should be doing a release pretty soon - within the next week or so would be my guess.

Hi @AntonyMilneQB, would you have any updates on when the next release might be happening? Thank you

[antonymilne]

Hi @jenikovsky, sorry for the delay on this - it's taking longer than expected indeed. We are still hoping to do one very shortly though 🤞