Rebranch: Silent assembler failure on arm64 Windows

[Steelskin]

tl;dr version

There are 3 issues at play here:

1. LLVM is failing to generate SEH end prologue properly for arm64 in some cases.
2. The error message is never surfaced and the compilation silently succeeds, with an empty object file being generated.
3. It seems likely that the SIL LICM pass is doing something to generate LLVM IR that the LLVM arm64 frame lowering code does not handle properly.

I have a potential fix for issue 1 but I still need to figure out a reproducer for it and land it in upstream LLVM main.

Long version

Symptom

The Swift toolchain rebranch is currently failing on Windows arm64 because a target in swift-build is failing to link. Further analysis has revealed that this is due to an empty object file. Here is the diagnostic output from the linker:

FAILED: bin/SWBTaskExecution.dll lib/SWBTaskExecution.lib
C:\WINDOWS\system32\cmd.exe /C "cd . && S:\b\5\bin\swiftc.exe -target aarch64-unknown-windows-msvc -j 10 -num-threads 10 -emit-library -use-ld=lld-link -sdk "S:/Program Files/Swift/Platforms/Windows.platform/Developer/SDKs/Windows.sdk" -g -debug-info-format=dwarf -use-ld=lld-link -Xlinker /DEBUG:DWARF -Xlinker /INCREMENTAL:NO -Xlinker /OPT:REF -Xlinker /OPT:ICF -O -swift-version 5 -libc MD   -Xlinker -implib:lib\SWBTaskExecution.lib   -o bin\SWBTaskExecution.dll @CMakeFiles\SWBTaskExecution.rsp && cd ."
<unknown>:0: warning: using (deprecated) legacy driver, Swift installation does not contain swift-driver at: 'C:\Users\fabrice\src\toolchain\b\5\bin\swift-driver-new.exe'
lld-link: error: Sources\SWBTaskExecution\CMakeFiles\SWBTaskExecution.dir\BuildDescriptionManager.swift.obj: unknown file type
clang: error: linker command failed with exit code 1 (use -v to see invocation)
[...]
$ file ./Sources/SWBTaskExecution/CMakeFiles/SWBTaskExecution.dir/BuildDescriptionManager.swift.obj
./Sources/SWBTaskExecution/CMakeFiles/SWBTaskExecution.dir/BuildDescriptionManager.swift.obj: empty

Investigation

Dumping diagnostic information in LLVM has revealed that the issue comes from this block in LLVM:

uint64_t WinCOFFObjectWriter::writeObject() {
  // If the assember had an error, then layout will not have completed, so we
  // cannot write an object file.
  if (getContext().hadError())
    return 0;
  // [...]
}

After adding more diagnostic in MCContext::reportError(), it turns out that the following error message was silently dropped, this is issue 2:

error: starting epilogue (.seh_startepilogue) before prologue has ended (.seh_endprologue) in $s16SWBTaskExecution23BuildDescriptionManagerC09constructcD0_11planRequest9signature11inDirectory2fs17bypassActualTasks19planningDiagnostics8delegateAA0cD0CSg0A12Construction0C4PlanC_AO0ctH0V7SWBUtil10ByteStringVAT4PathVAT7FSProxy_pSbSDy7SWBCore16ConfiguredTargetCSgSayAT10DiagnosticVGGAA0cdS8Delegate_ptYaKFZTf4nnnnnnnnd_n
error: Stray .seh_endepilogue in $s16SWBTaskExecution23BuildDescriptionManagerC09constructcD0_11planRequest9signature11inDirectory2fs17bypassActualTasks19planningDiagnostics8delegateAA0cD0CSg0A12Construction0C4PlanC_AO0ctH0V7SWBUtil10ByteStringVAT4PathVAT7FSProxy_pSbSDy7SWBCore16ConfiguredTargetCSgSayAT10DiagnosticVGGAA0cdS8Delegate_ptYaKFZTf4nnnnnnnnd_n

For reference, here are the corresponding ASM and LLVM IR dumps for this file:
BuildDescriptionManager.swift.asm.txt
BuildDescriptionManager.swift.llvm-ir.txt

It turns out that the problematic function does not have a .seh_endprologue section, resulting in the assembler failure, this is issue 1.

The issue seems to be triggered by a Swift optimization pass, since adding @_optimize(none) on the problematic function fixes the issue. Given recent changes, it seems likely that the SIL LICM pass is doing something that the frame lowering code does not like, this is issue 3.

[Steelskin]

I am still knee deep into this issue, this is what I have figured out so far:

• The prologue/epilogue emitters have a property called HasWinCFI. This controls whether the MachineFunction has its HasWinCFI property set. This is set in the prologue here and in the epilogue here. As a side-note, the Aarch64 prologue emitter emits the .seh_endprologue pseudo-instruction.
• If the MachineFunction has the HasWinCFI property set, it emits the .seh_proc pseudo-instruction. This is done here.
• The problem we are encountering stems from HasWinCFI being set in the epilogue, but not in the prologue. Because the logic controlling whether .seh_proc is emitted is separate from the logic controlling whether the .seh_endprologue pseudo-instruction is emitted, we end up never emitting the .seh_endprologue pseudo-instruction.
• The logic controlling HasWinCFI is different in the prologue and in the epilogue, hence the discrepancy.

I can't figure out how to fix it without breaking a lot of other things. This is the logic resulting in HasWinCFI being set to true in the epilogue in the faulty case:

1. This call to emitFrameOffset() happens in the else block for the AFL.canUseRedZone(MF) test.
2. emitFrameOffset() calls emitFrameOffsetAdj(). In that function, this specific branch sets HasWinCFI to true.

[Steelskin]

The fix in LLVM is at llvm#158173
It will need to be back-ported to the 21.x branch.