Balancing Linux distro CI coverage with container version vs latest

[richlander]

I have found myself updating CI coverage in release/10.0 to Ubuntu 26.04 and release/9.0-staging and release/8.0-staging to Debian 13. This raises significant tension. This is new since we haven't previously been as aggressive about coverage.

Core tensions:

• We want bleeding-edge coverage to ensure users have a good experience on new distros versions.
• We want to have very high confidence that users have a good experience using our 8.0, 9.0, and 10.0 container images. These are fixed for the lifetime of the release on a specific distro version.
• We want to keep our CI costs at a minimum (with some definition of "minimum").

Context:

• 8.0 and 9.0 tag images are based on Debian 12 (Bookworm) and will never upgrade to Debian 13 (Trixie).
• 10.0 tag images are based on Ubuntu 24.04 (Noble) and will never upgrade to Ubuntu 26.04 (Resolute).
• 11.0 targ images will be based on Ubuntu 26.04 (Resolute) ...

The simpler answer is to test both old and new. If everyone is good with that, this is simple.

IMO, it is a priority inversion to chase the future when we're clearly needing to satisfy a cloud-native reliability contract with these container images. That statement only applies if we for some reason conclude that we can only support one Debian version (for 8.0 and 9.0 branches) and one Ubuntu version (for 10.0 branches). This is similar to how we test two Azure Linux versions in our branches.

The proposal is that we test multiple versions in CI for this case.

Related:

• [release/9.0-staging] Update Debian 12 to Debian 13 helix references #122326 (comment)
• [release/10.0] Update Ubuntu container references from 25.10 to 26.04 #122130

[richlander]

@agocke @jkotas @MichaelSimons

[dotnet-policy-service]

Tagging subscribers to this area: @dotnet/runtime-infrastructure
See info in area-owners.md if you want to be subscribed.

[jkotas]

#120337 (comment) is a discussion about the same topic.

• The Linux distro mix only matters for some areas (e.g. libraries tests). Large number of tests (e.g. codegen tests) run on a single Linux distro and the version does not matter. It is fine to push the version to latest in main.
• For areas sensitive to Linux distro mix, we want to test on the full mix in the outer loop runs at least. The subset of this mix to run in CI by default is cost/benefit tradeoff.
• In servicing, we want to test more by default since we are more risk averse and less cost sensitive there.