[wasm] AOT tests *build* timing out on Linux #97044
Comments
radical
added
arch-wasm
|
Tagging subscribers to 'arch-wasm': @lewing Issue DetailsAOT, and sometimes EAT(EnableAggressiveTrimming) builds have been timing out on linux. The first failing rolling AOT build was 33446fb1 . Note that the corresponding EAT build did not fail. The last successful build was 4c500699 . The changes responsible should be 4c50069...33446fb , or on the outside - 4c50069...22ba7d6 .
|
ghost
added
the
|
@vitek-karas this appears to be a linker hang, it is difficult to see in action because AZDO clears the data when it cancels the task |
lewing
added
the
area-Tools-ILLink
|
Tagging subscribers to this area: @agocke, @sbomer, @vitek-karas Issue DetailsAOT, and sometimes EAT(EnableAggressiveTrimming) builds have been timing out on linux. The first failing rolling AOT build was 33446fb1 . Note that the corresponding EAT build did not fail. The last successful build was 4c500699 . The changes responsible should be 4c50069...33446fb , or on the outside - 4c50069...22ba7d6 . Known Issue Error MessageFill the error message using step by step known issues guidance. {
"ErrorMessage": "",
"ErrorPattern": "",
"BuildRetry": false,
"ExcludeConsoleLog": false
}ReportSummary
|
This is the last log from LibraryTests_AOT line, before it gets cleared by Azdo |
|
As a workaround, would it be possible to stop building tests in parallel in these lanes ? |
|
We're going to try #97491 to work around this |
|
Sorry for the delay, I tried to repro this but it's hard to tell. Locally (on a DevBox in WSL) the Full AOT build took 18+ minutes, but it fails when packing the tests (no idea why), so it hasn't really finished. But it did run all of the trimming. I didn't see any specific trimming take really long time. What we would probably need is to try this on bits before it started to fail and after. It's possible something caused trimming to be slower in general and overall the build just crosses some threshold. @matouskozak would you be able to try to get this data? |
|
@matouskozak I guess we could disable the parallel build there too? Do we know if this is just excessive slowness or something else. The EAT lanes just trim without doing AOT so I wouldn't expect them to timeout |
|
@rmarinho you mentioned you were seeing something like this in the preview 2 builds you were testing, can you try again and let us know if setting the cpus to 1 avoids your issue as well |
AOT, and sometimes EAT(EnableAggressiveTrimming) builds have been timing out on linux. The first failing rolling AOT build was 33446fb1 . Note that the corresponding EAT build did not fail.
The first EAT failure was this for 22ba7d60 .
The last successful build was 4c500699 .
The changes responsible should be 4c50069...33446fb , or on the outside - 4c50069...22ba7d6 .
Known Issue Error Message
Fill the error message using step by step known issues guidance.
{ "ErrorMessage": "", "ErrorPattern": "", "BuildRetry": false, "ExcludeConsoleLog": false }Report
Summary
The text was updated successfully, but these errors were encountered: