CI: Catch importwarning _SixMetaPathImporter.find_spec

[phofl]

Lets see if this helps somehow

I can't see how this could be related to our s3fs and aiobotocore pins. The following is the version diff from before and after the pin:

                 Name_old     Name_new Version_old Version_new
3             aiobotocore  aiobotocore       2.4.0       1.4.2
26                  boto3        boto3     1.24.59    1.17.106
27               botocore     botocore     1.27.59    1.20.106
76                 fsspec       fsspec    2022.7.1    2021.7.0
77                  gcsfs        gcsfs    2022.7.1    2021.7.0
98   google-cloud-storage          NaN       2.5.0         NaN
119              jmespath     jmespath       1.0.1      0.10.0
286                  s3fs         s3fs       0.6.0    2021.7.0
287            s3transfer   s3transfer       0.6.0       0.4.2

[mroeschke]

I wouldn't be opposed to just adding this to filterwarnings in pyproject.toml since pandas doesn't explicitly use six so it should be from one of the dependencies: benjaminp/six#341

[phofl]

Yes, definitely. Its a bit hard to find out from which one though.

Just want to see if this helps. Can add to pyproject.toml afterwards

[phofl]

Added to pyproject.toml, lets see if it works. Can not reproduce the warning locally

[phofl]

It looks like filterwarnings does not work well together with assert_produces_warning. Could not get tests to pass locally when filtering the warning with pytest and checking for no warning

[phofl]

cc @mroeschke

This is greenish now. Check is ugly though

[lithomas1]

So I did a little debugging, and the warning is coming from botocore(they vendor six), and it looks like the pinning downgraded botocore, cuasing this warning to come back.

[lithomas1]

This is a symptom of pinning aiobotocore(see comments in #48272)

[phofl]

Yeah but this is necessary, see #48272 (comment)

[mroeschke]

I would be okay with this PR as is as a temporary fix if pinning botocore doesn't work #48291 (comment). (It is really strange behavior that a s3fs code path is being hit when it should have nothing to do with it?)

I would prefer not to pin s3fs higher than our minimum version if possible (and generally not have our dep files have different pins in general).

[lithomas1]

The thing is, this does have to do with it. The way I understand it is s3fs pins aiobotocore which pins down botocore. As I mentioned, pinning aiobotocore in the other PR is pinning down botocore to an old version that vendors an old six doesn't address the warning.

So I would be -1 on this, since this ImportWarning is telling us that pinning aiobotocore was not the right move.

[phofl]

S3fs is not pinning aiobotocore, but we have to pin since the old s3fs is not compatible with aiobotocore > 2.0. pinning botocore will most likely cause non solvable environments

i think pinning aiobotocore is the right move here in a sense, not having a higher s3fs minimum requirement is the actual issue. The old s3fs is not compatible with 3.10

can we bump this in the rc? Or do we have to follow a certain version Schema there?

[mroeschke]

can we bump this in the rc? Or do we have to follow a certain version Schema there?

As mentioned in #48291, I would be +1 to simply bumping s3fs in the 1.5.0rc. I have been following a "min version that has been released one year prior" rule of thumb for most optional dependencies.

[phofl]

Thx, closing in favour of #48299