Decide what to do about failing tests on beta channel

[philipcraig]

#817 shows that integration tests fail intermittently on the rustup beta channel.

Seems like it's very likely related to rust-lang/rust#92595

If they backport the fix as they are discussing, this will go away.

If they don't and promote to stable, autocxx will be thoroughly unreliable. Or maybe just its tests?

Mentioned the bug here as they seem to be keen to promote the bug to stable "as-is" rust-lang/rust#94124 (comment)

[pnkfelix]

Hi there @philipcraig ! I was very curious about your comment on rust-lang/rust#94124 (comment) .

In particular, one of our on-going assumptions is that CI users may be testing beta but do not tend to test incremental compilation, because CI systems tend to employ "from scratch" builds.

Is your CI system actually reusing previous build states, or otherwise deliberating exercising incremental compilation?
(If it is, that's very interesting, and I'm happy to hear it.)

If to your knowledge your CI system is not employing any kind of incremental reuse, and is in principle employing "from scratch" builds, then that is also very important for us to hear, because I think one of the big assumptions for us on rust-lang/rust#94124 is that we have been assuming that it fundementally is an incremental compilation bug.

[philipcraig]

Is your CI system actually reusing previous build states, or otherwise deliberating exercising incremental compilation? (If it is, that's very interesting, and I'm happy to hear it.)

It's my understanding that the CI is deliberately exercising incremental compilation. I cannot make one of the failing tests fail on its own, though it often fails when build just two tests. It is using the trybuild crate a few hundred times to compile a few hundred Rust integration tests, each in its own compilation unit. However, each test doesn't start with an empty directory, so, incremental.

[pnkfelix]

okay. we're disabling incremental compilation's default settings in 1.59.0, as described here: rust-lang/rust@6af5faf

[philipcraig]

okay. we're disabling incremental compilation's default settings in 1.59.0, as described here: rust-lang/rust@6af5faf

@pnkfelix Thanks for this information. When should that hit the beta rust channel?

[pnkfelix]

@philipcraig The compiler team has not decided to disable incremental compilation in the beta channel; that change is restricted to the stable channel alone, for now.

The problems that you're seeing in the current beta are believed to be fixed in the nightly channel. The nightly-to-beta promotion is underway as we speak, in rust-lang/rust#94228; after that lands, then the new beta should be available sometime soon thereafter (within a day at most I expect, but in this case the release team might launch the build sooner than the cron job would automatically trigger), and we believe that new beta will also address your problem.

[philipcraig]

The problems that you're seeing in the current beta are believed to be fixed in the nightly channel. The nightly-to-beta promotion is underway as we speak, in rust-lang/rust#94228; after that lands, then the new beta should be available sometime soon thereafter (within a day at most I expect, but in this case the release team might launch the build sooner than the cron job would automatically trigger), and we believe that new beta will also address your problem.

@pnkfelix Thanks for the update. I can confirm that the CI here currently does not see these issues on nightly.