Update pins in aws-c-* ecosystem

[h-vetinari]

A couple of pent-up PRs that haven't been dealt with. Those run through fine with automerge, provided the different migrators don't deadlock, so order them with wait_for_migrators

Closes #3980
Closes #3977
Closes #3953
Closes #3952
Closes #3950
Closes #3944
Closes #3942
Closes #3941
Closes #3940
Closes #3939
Closes #3922
Closes #3914

CC @xhochy

[conda-forge-webservices]

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.

[xhochy]

Thanks for doing this. I was a bit out-of-office in the conda-forge world, so these pile up quite quickly 🙈

[jakirkham]

There's discussion in issue ( conda-forge/aws-sdk-cpp-feedstock#662 ) on how to better handle these because they seem to be coming in too quick (even when people are here 😉)

[h-vetinari]

There's discussion in issue ( conda-forge/aws-sdk-cpp-feedstock#662 ) on how to better handle these because they seem to be coming in too quick (even when people are here 😉)

It's a somewhat separate issue. The SDK shields its consumers (e.g. arrow) from rebuilds for the aws-c-* libs, though the sdk itself will get rebuilt (compatibly). This is how things have been running for the aws-sdk-cpp 1.9 cycle without issue.

As of 1.10, there's also an apparent dependency on aws-crt-cpp (so nominally another trigger for arrow-rebuilds), but there the situation is broadly the same - we don't need to migrate it nearly as often as the component libs.

Also, once we get cuda cross compilation sorted out, the arrow-feedstock can essentially run on auto-pilot (and with more reasonable CI size too, now that openssl 1.1.1 has been dropped - main will have 10 CI jobs per run).

[xhochy]

now that openssl 1.1.1 has been dropped

Oh, can you redirect me to where we made that decision?

[h-vetinari]

Oh, can you redirect me to where we made that decision?

Of course: #3838 / #3892

[xhochy]

Thanks! I was a bit shocked because I have some hard pins on 1.1.1 in various places but the argumentation there makes sense (i.e. I should rather invest the heavy work into getting rid of those pins).

[h-vetinari]

I was a bit shocked because I have some hard pins on 1.1.1 in various places

If those are in conda-forge, then I tried very hard to hunt for them (see #3838). Let me know if I missed some. 🙃

Other than that it's been discussed in several core syncs over the last months, and today we finally pulled the trigger! :)

[xhochy]

If those are in conda-forge, then I tried very hard to hunt for them (see #3838). Let me know if I missed some. 🙃

No binary repacks I cannot upload for license reasons to conda-forge 🙈

Other than that it's been discussed in several core syncs over the last months, and today we finally pulled the trigger! :)

core meeting are sadly not accessible to me (I cannot join at that time) and I only see meeting notes some time later.

[jakirkham]

now that openssl 1.1.1 has been dropped

Oh, can you redirect me to where we made that decision?

We discussed at the conda-forge/core meeting (see notes) and concluded things had come far enough to close out OpenSSL 3. Jaime reiterated that conclusion on GH.

Though if we've missed something, please let us know 🙂

[jakirkham]

There's discussion in issue ( conda-forge/aws-sdk-cpp-feedstock#662 ) on how to better handle these because they seem to be coming in too quick (even when people are here 😉)

It's a somewhat separate issue. The SDK shields its consumers (e.g. arrow) from rebuilds for the aws-c-* libs, though the sdk itself will get rebuilt (compatibly). This is how things have been running for the aws-sdk-cpp 1.9 cycle without issue.

As of 1.10, there's also an apparent dependency on aws-crt-cpp (so nominally another trigger for arrow-rebuilds), but there the situation is broadly the same - we don't need to migrate it nearly as often as the component libs.

Also, once we get cuda cross compilation sorted out, the arrow-feedstock can essentially run on auto-pilot (and with more reasonable CI size too, now that openssl 1.1.1 has been dropped - main will have 10 CI jobs per run).

I've not dug into this deeply. However think if we are getting a migration PR a day that is way too many. We can't keep up with that.

It is probably simplest to discuss all of this in one issue. So pointed to that one that you shared with me recently. If we would rather construct a new meta issue, that's fine too, but these issues seem interlinked so discussing in one place would be helpful (particularly for those learning about it).

[h-vetinari]

It is probably simplest to discuss all of this in one issue.

Sure, no problem.