Clean up crossgen outerloops

[agocke]

We have at least the following:

• runtime-coreclr r2r
• runtime-coreclr r2r-extra
• runtime-coreclr crossgen2
• runtime-coreclr crossgen2 outerloop
• runtime-coreclr crossgen2-composite
• runtime-coreclr crossgen2-composite-gcstress

Which of these are needed? Can we fold at least some of them together?

[jkoritzinsky]

Here's a summary of the different legs covered by each of these pipelines:

runtime-coreclr r2r

• Pri1 outerloop with R2R (normal scenario) for the following platforms:
  - linux_arm
  - linux_arm64
  - linux_x64
  - osx_arm64
  - windows_arm64
  - windows_x64
  - windows_x86

runtime-coreclr r2r-extra

• Pri1 outerloop with R2R for the following platforms and scenarios:

  • Platforms

    • linux_arm
    • linux_arm64
    • linux_x64
    • win_x86
    • win_x64
    • win_arm64
    • osx_arm64

  • Scenarios
    - jitstress1
    - jitstress2
    - jitstress1_tiered
    - jitstress2_tiered
    - jitstressregs1
    - jitstressregs2
    - jitstressregs3
    - jitstressregs4
    - jitstressregs8
    - jitstressregs0x10
    - jitstressregs0x80
    - jitstressregs0x1000
    - jitstressregs0x2000
    - jitminopts
    - forcerelocs
    - gcstress0xc

runtime-coreclr crossgen2

• Pri0 with R2R (normal scenario) for the following platforms
  - linux_x64
  - linux_arm64
  - osx_arm64
  - osx_x64
  - windows_x64
  - windows_arm64

Note

This coverage is fully covered by the Pri1 runs in runtime-coreclr crossgen2-outerloop (also by runtime-coreclr r2r excluding osx-x64)

• Libraries tests with R2R for windows/linux/mac x64 (will be fixed by Override ResolveReadyToRunCompilers for TestReadyToRun=true test builds #123734)

• Pri0 with R2R with hot-cold splitting for the windows/linux/mac x64

runtime-coreclr crossgen2-outerloop

• Pri1 outerloop with R2R for the following platforms:

      - linux_arm
      - linux_arm64
      - linux_x64
      - osx_x64
      - osx_arm64
      - windows_arm64
      - windows_x64
      - windows_x86
  

Note

Looks like the only unique platform here (not covered by runtime-coreclr r2r) is osx-x64.

• Pri1 outerloop with composite R2R for the following platforms:

      - linux_x64
      - linux_arm64
      - osx_arm64
      - osx_x64
      - windows_x64
      - windows_arm64
  

• CrossGen2 codegen comparison (crossgen2 running on build machine vs crossgen2 on target machine generate the same code).

runtime-coreclr crossgen2-composite

• Pri0 outerloop with composite R2R for the following platforms:

      - linux_arm
      - linux_x64
      - linux_arm64
      - osx_arm64
      - osx_x64
      - windows_x86
      - windows_x64
      - windows_arm64
  

runtime-coreclr crossgen2-composite-gcstress

• Pri1 outerloop with composite R2R for the following platforms and scenarios:

  • Platforms
    - linux_x64
    - linux_arm64
    - osx_arm64
    - windows_x64
    - windows_arm64

  • Scenarios
    - heapverify1
    - gcstress0xc_disabler2r
    - gcstress0xc_disabler2r_jitstress2
    - gcstress0xc_disabler2r_heapverify1
    - gcstress0xc_jitstress1
    - gcstress0xc_jitstress2
    - gcstress0xc_tailcallstress
    - gcstress0xc_jitminopts_heapverify1

Note

Why are we running a bunch of disabler2r lanes on an r2r stress pipeline? I can understand having one to validate that disabling r2r works, but running 3 scenarios seems not helpful.

I believe we can likely combine many of these pipelines together (basically everything that's not using a special scenario can likely go in the same pipeline). The two pipelines that run the extra scenarios are currently scheduled to run on weekends at 6pm (r2r-extra) and 10pm (composite-gcstress), likely to help avoid putting too much load on Helix at once.

Something that should be investigated as part of combining these pipelines together: Do we even pass through the different JIT stress options to crossgen2 correctly for the codegen? Otherwise we're getting significantly less useful coverage (as we're only testing the stress modes on code that is ineligible for r2r, which isn't particularly helpful).

Also, how much coverage of things like composite R2R do we want/need? Especially as all of these lanes are using the PE format for R2R (none are using the Mach-O support we added recently).