Attempt to fix VM llvm-project submodule contamination #1605
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
We're seeing test runs fail due to apparent submodule contamination. For example, #1604 didn't change
llvm-projectat all, but failed with:We believe this is because #1594, which did change
llvm-project, ran on the same VM. (Timeline: #1594 ran build 20210201.6 starting at 3:53 PM Pacific Time, followed by #1604 running build 20210202.1 starting at 4:27 PM. I can't find enough timestamps to see exactly when the first build finished, given our 10 minute idle VM cleanup setting, but this indeed looks like a contamination scenario.)After investigating, we determined that the cause was:
STL/azure-devops/checkout-sources.yml
Lines 45 to 48 in 355f8f5
These commands will perform a sparse checkout of a given SHA - but if they're run again, on the same VM but asked to check out a different SHA, they won't update the
llvm-projectworking tree. The result is running new STL bits against oldllvm-projectbits, or vice versa. (In this case, the "unexpected keyword argument" was caused by a change inllvm-projectwith a corresponding change in the STL.)(We saw this after merging #1505, but the issue was introduced in #1513 which was merged earlier.)
After reproing this locally, I believe that a reasonable fix is:
resetbetweenfetchand setting up thesparse-checkout.sparse-checkout,git reset --hardwill set the working tree to the sparse set of files that we want, regardless of whether this is the first run, a later run with the same SHA, or a later run with a different SHA.git clean. This will remove any files left over from previous builds, although they weren't responsible for today's problem.I've locally verified that this is capable of switching SHAs, and that it doesn't result in a full checkout.