gcc 12.1.0

[fxcoudert]

GCC 12.1.0 will be released in about a week: https://gcc.gnu.org/pipermail/gcc/2022-April/238628.html

ARM support was partly, but not fully, integrated into upstream (GCC has a slow development cycle). We can use a patch, based on the maintainer's branch at https://github.com/iains/gcc-darwin-arm64

That was meant to be a comment

[fxcoudert]

ARM patch needs to be rebased on top of the actual RC, I'll do that. Let's see how gcc 12 rc goes on Intel.

[fxcoudert]

D front-end now needs to be bootstrapped. It has so few users, remove for now.

[fxcoudert]

This PR needs a brew change before we can test formulas that depend on GCC: Homebrew/brew#13221

[fxcoudert]

So, both Intel and ARM builds seem to proceed okay, most depends build OK, I've completed the list of revision bumps and this should now look almost like the final thing. (Well, when GCC is released in a week.)

[fxcoudert]

On ARM macOS 12:

• dynare build failure: https://git.dynare.org/Dynare/dynare/-/issues/1852
• nwchem existing build failure
• lots of files with missing rpath in openmodelica linkage test
• khiva has a file with missing rpath
• highs msc-generator pythran will also need a rev-bump

[danielnachun]

Build the gcc formula using glibc@2.13 instead

This is what I want to do but it will take a while to figure out as it's a bit more complicated than the other formula. I am very confident we can make the glibc bootstrapping work but I feel bad that it's blocking our migration. I am personally very invested in putting in the work to make CentOS 7 support as seamless as possible because I am one of the people using Homebrew on that platform. But I can't give an exact timeline for when I can get this last snag fixed for bootstrapping.

If this looks infeasible in the short term then I'm okay with leaving it to the user to get a new enough version of GCC to build the glibc formula during bootstrapping. This isn't the easiest thing to do but given that I hope to make glibc relocatable eventually I hope it will be come irrelevant.

Can I propose a slightly more involved solution for migrating to GCC 12 that would allow us to save the bottles we already made and not force an immediate upgrade to a newer CI image? I am willing to take full responsibility for the work it would require and would start on it immediately if we're okay with it. These are the steps we would take:

1. Revision bump GCC 11 to add #{opt_lib}/gcc/current as a symlink to the GCC 11 formula without removing #{opt_lib}/gcc/11, and replace #{opt_lib}/gcc/11 with #{opt_lib}/gcc/current in the RPATHs we add to the GCC spec on Linux. This will mean that bottles currently built with GCC 11 will still work and that any new bottles built with it will look for libstdc++ in #{opt_lib}/gcc/current.
2. Revision bump all GCC dependents in individual PRs without dependent testing. While I know this is a pain, we only have to do it once, and we will only need long builds for a handful of formulae. The sole purpose of this is to update their RPATH so that it looks for libstdc++ in #{opt_lib}/gcc/current instead of #{opt_lib}/gcc/11.
3. In the GCC 12, PR, revert GCC 11 back the way it was and use the #{opt_lib}/gcc/current path in GCC 12 instead.

Although we will have vastly fewer GCC Linux dependents in the near future, once we've switched to the #{opt_lib}/gcc/current structure, we won't ever have to revision bump Linux-specific GCC dependents on GCC version upgrades again.

[sjackman]

I suggest…

• upgrade the glibc formula to 2.35 (or the version provided by the chosen Ubuntu version)
• update the bottle builder image to Ubuntu 22.04 (or 18.04 or 20.04)
• unbottle the ~300 formulae that depend on gcc on Linux and remove the gcc dependency
• merge this PR
• rebuild the unbottled formulae using the host compiler

[carlocab]

I am very confident we can make the glibc bootstrapping work but I feel bad that it's blocking our migration.

This is where we disagree: I think glibc bootstrapping can and should be fixed properly after migration rather than before. In particular, it should not be a blocker at all.

Instead of worrying about making glibc bootstrapping seamless for users on older systems, we should provide a viable (even if a bit rocky) migration path for them to follow. I suspect most of these users already know their way around a Linux system, so it doesn't need to be as smooth as it should be for the average user.

If this looks infeasible in the short term then I'm okay with leaving it to the user to get a new enough version of GCC to build the glibc formula during bootstrapping. This isn't the easiest thing to do but given that I hope to make glibc relocatable eventually I hope it will be come irrelevant.

Yes, this is one example of a viable migration path that I reference above.

Can I propose a slightly more involved solution for migrating to GCC 12 that would allow us to save the bottles we already made and not force an immediate upgrade to a newer CI image? I am willing to take full responsibility for the work it would require and would start on it immediately if we're okay with it. These are the steps we would take:

1. Revision bump GCC 11 to add #{opt_lib}/gcc/current as a symlink to the GCC 11 formula without removing #{opt_lib}/gcc/11, and replace #{opt_lib}/gcc/11 with #{opt_lib}/gcc/current in the RPATHs we add to the GCC spec on Linux. This will mean that bottles currently built with GCC 11 will still work and that any new bottles built with it will look for libstdc++.
2. Revision bump all GCC dependents in individual PRs without dependent testing. While I know this is a pain, we only have to do it once, and we will only need long builds for a handful of formulae. The sole purpose of this is to update their RPATH so that it looks for libstdc++ in #{opt_lib}/gcc/current instead of #{opt_lib}/gcc/11.
3. In the GCC 12, PR, revert GCC 11 back the way it was and use the #{opt_lib}/gcc/current path in GCC 12 instead.

This plan requires nearly twice as many rebuilds and/or revision bumps, and both are costly. Rebuilds take CI time, which is in short supply these days. revision bumps trigger cascading formula upgrades that we get very many complaints about.

[sjackman]

Here's a PR to upgrade Glibc to version 2.35 to get the ball rolling.

• glibc 2.35 #106837

[MikeMcQuaid]

Instead of worrying about making glibc bootstrapping seamless for users on older systems, we should provide a viable (even if a bit rocky) migration path for them to follow. I suspect most of these users already know their way around a Linux system, so it doesn't need to be as smooth as it should be for the average user.
...
Yes, this is one example of a viable migration path that I reference above.
...
This plan requires nearly twice as many rebuilds and/or revision bumps, and both are costly. Rebuilds take CI time, which is in short supply these days. revision bumps trigger cascading formula upgrades that we get very many complaints about.

Agreed with all of this. Would suggest we take discussion of a plan into a dedicated issue.

• unbottle the ~300 formulae that depend on gcc on Linux and remove the gcc dependency

This feels like something we could/should do now to get things rolling.

[sjackman]

Agreed with all of this. Would suggest we take discussion of a plan into a dedicated issue.

• Update bottling Linux distribution to Ubuntu 22.04 brew#13619

unbottle the ~300 formulae that depend on gcc on Linux and remove the gcc dependency

This feels like something we could/should do now to get things rolling.

I agree!

[carlocab]

• unbottle the ~300 formulae that depend on gcc on Linux and remove the gcc dependency

This feels like something we could/should do now to get things rolling.

Yes, we can see which formulae need this based on CI from #106755.

[chandlerc]

Can we make sure we add a comment to any formulae where the bottle is removed so that we know it was done deliberately?

Will these bottles get added back?

I'm worried because some things have had the bottle removed that are exceedingly expensive to build from source. There has been a huge push that this isn't something we should be solving and instead we should rely on bottles, but if they go away this whole idea breaks down.

For example, LLVM no longer has a bottle and that is going to make things very painful for users trying to install it as LLVM can be very slow and resource intensive to build.

[carlocab]

Will these bottles get added back?

Yes. We intend to do so as soon as possible. Sorry for the inconvenience!

[chandlerc]

Will these bottles get added back?

Yes. We intend to do so as soon as possible. Sorry for the inconvenience!

Any estimates for the timescale of "soon" here?

Our continuous builds on Linux are currently broken due to lack of bottles, so we'll have to work around this if its even on the order of days. We've been fine with the periodic bottle re-build that takes several hours in the worst case, but it seems like this is going to be a longer lasting problem than that.

We also need to give users instructions as if they are on a laptop for example they may not realistically be able to build LLVM from source.

[carlocab]

I'll add this back in #107087. (Merged in 19dc6a6.)

Note: the llvm bottles will break when I merge #106755. This may happen in the next three days. Fixing it requires one of:

• rebuilding llvm from source
• installing GCC 11+ into /usr
• installing gcc@11, but that needs gcc@11 11.3.0 (new formula) #106972.

I'll give you some warning before that happens. I'll work on merging #107018 so that users who cannot do any of the above can try using llvm@13 in the meantime.

Does that work for you?

[chandlerc]

We could temporarily go back to llvm@13 if we needed to, or better yet when landed manually install gcc@11.

Also happy to know how to get bottles working again w/ the new GCC everywhere, I don't want to leave you all stuck in a weird state either.

[chandlerc]

FWIW, if its useful for us to pin everything for a few days (or even a week) while the world rebuilds against GCC-12, I think that'd be pretty easy for us at this point.

[carlocab]

Just as an update, we've now worked out a smoother transition plan that involves much less breakage.

See:
Homebrew/brew#13631
Homebrew/brew#13633
#107148

There will almost certainly still be some, but it will likely not be to any painful-to-build formulae. In particular, I don't expect any of the workarounds I mentioned above to be necessary. But feel free to shout if we break something again.

[carlocab]

@chandlerc GCC 12 will land in the next hour or so. If something breaks for you, let me know. Anything that does should be fixed by a brew reinstall. Please do let me know even if brew reinstall fixed it, and especially if it didn't.

If there is breakage, brew may attempt to eagerly rebuild a formula from source to try to fix it. You can feel free to interrupt the rebuild attempt and then do a brew reinstall from a bottle.

[chandlerc]

@chandlerc GCC 12 will land in the next hour or so. If something breaks for you, let me know. Anything that does should be fixed by a brew reinstall. Please do let me know even if brew reinstall fixed it, and especially if it didn't.

If there is breakage, brew may attempt to eagerly rebuild a formula from source to try to fix it. You can feel free to interrupt the rebuild attempt and then do a brew reinstall from a bottle.

Just to close the loop, everything worked great with this approach!