Certain build configurations are "less efficient"

[tannergooding]

Issue

The repo has two main places where vcvarsall.bat is called and therefore where the C++ environment is setup. These are:

• https://github.com/dotnet/runtime/blob/main/eng/native/init-vs-env.cmd
• https://github.com/dotnet/runtime/blob/main/src/coreclr/build-runtime.cmd

To summarize, vcvarsall.bat allows a parameter in the form of host_target and therefore x86_x64 is host=x86, target=x64.

Examples

In the case of init-vs-env it defaults to assuming that the host (that is the machine we are building on) is always x86 and therefore restricts us to using the 32-bit x86 versions of the compilers and related tools.

In the case of build-runtime it has some complex handling that ultimately defaults to host=x64, target=x64 and otherwise uses host=x86, target=* where typically target=%__HostArch%. __HostArch itself defaults to __TargetArch unless the user passes in -hostarch *. __TargetArch matches the default -arch command line passed in.

Solution - init-vs-env

For init-vs-env we should check %PROCESSOR_ARCHITECTURE% and default back to x86 if unrecognized. This winds up with the most portability but also the least amount of emulation on platforms like Arm/Arm64. It likewise means we can make full advantage of modern build machines.

if /i "%PROCESSOR_ARCHITECTURE%"=="AMD64" (
    if /i "%~1" == "x86"   (set __VCBuildArch=x64_x86)
    if /i "%~1" == "x64"   (set __VCBuildArch=x64)
    if /i "%~1" == "arm"   (set __VCBuildArch=x64_arm)
    if /i "%~1" == "arm64" (set __VCBuildArch=x64_arm64)
    if /i "%~1" == "wasm"  (set __VCBuildArch=x64)
) else if /i "%PROCESSOR_ARCHITECTURE%"=="ARM64" (
    if /i "%~1" == "x86"   (set __VCBuildArch=arm64_x86)
    if /i "%~1" == "x64"   (set __VCBuildArch=arm64_x64)
    if /i "%~1" == "arm"   (set __VCBuildArch=arm64_arm)
    if /i "%~1" == "arm64" (set __VCBuildArch=arm64)
    if /i "%~1" == "wasm"  (set __VCBuildArch=arm64_x64)
) else if /i "%PROCESSOR_ARCHITECTURE%"=="ARM" (
    if /i "%~1" == "x86"   (set __VCBuildArch=arm_x86)
    if /i "%~1" == "x64"   (set __VCBuildArch=arm_x64)
    if /i "%~1" == "arm"   (set __VCBuildArch=arm)
    if /i "%~1" == "arm64" (set __VCBuildArch=arm_arm64)
    if /i "%~1" == "wasm"  (set __VCBuildArch=arm_x64)
) else (
    if /i "%~1" == "x86"   (set __VCBuildArch=x86)
    if /i "%~1" == "x64"   (set __VCBuildArch=x86_x64)
    if /i "%~1" == "arm"   (set __VCBuildArch=x86_arm)
    if /i "%~1" == "arm64" (set __VCBuildArch=x86_arm64)
    if /i "%~1" == "wasm"  (set __VCBuildArch=x86_x64)
)

Solution - build-runtime

For build-runtime the fix should ultimately be similar. I don't think we need __HostArch at all and instead should just detect it implicitly, requiring users to run the batch script from the correct command line environment.

If we wanted to keep __HostArch, it should correctly default to matching PROCESSOR_ARCHITECTURE if otherwise unspecified. It should likewise be used for the host portion of host_target. __TargetArch should then be used for the target portion.

[ghost]

Tagging subscribers to this area: @hoyosjs
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Issue

The repo has two main places where vcvarsall.bat is called and therefore where the C++ environment is setup. These are:

• https://github.com/dotnet/runtime/blob/main/eng/native/init-vs-env.cmd
• https://github.com/dotnet/runtime/blob/main/src/coreclr/build-runtime.cmd

To summarize, vcvarsall.bat allows a parameter in the form of host_target and therefore x86_x64 is host=x86, target=x64.

Examples

In the case of init-vs-env it defaults to assuming that the host (that is the machine we are building on) is always x86 and therefore restricts us to using the 32-bit x86 versions of the compilers and related tools.

In the case of build-runtime it has some complex handling that ultimately defaults to host=x64, target=x64 and otherwise uses host=x86, target=* where typically target=%__HostArch%. __HostArch itself defaults to __TargetArch unless the user passes in -hostarch *. __TargetArch matches the default -arch command line passed in.

Solution - init-vs-env

For init-vs-env we should check %PROCESSOR_ARCHITECTURE% and default back to x86 if unrecognized. This winds up with the most portability but also the least amount of emulation on platforms like Arm/Arm64. It likewise means we can make full advantage of modern build machines.

if /i "%PROCESSOR_ARCHITECTURE%"=="AMD64" (
    if /i "%~1" == "x86"   (set __VCBuildArch=x64_x86)
    if /i "%~1" == "x64"   (set __VCBuildArch=x64)
    if /i "%~1" == "arm"   (set __VCBuildArch=x64_arm)
    if /i "%~1" == "arm64" (set __VCBuildArch=x64_arm64)
    if /i "%~1" == "wasm"  (set __VCBuildArch=x64)
) else if /i "%PROCESSOR_ARCHITECTURE%"=="ARM64" (
    if /i "%~1" == "x86"   (set __VCBuildArch=arm64_x86)
    if /i "%~1" == "x64"   (set __VCBuildArch=arm64_x64)
    if /i "%~1" == "arm"   (set __VCBuildArch=arm64_arm)
    if /i "%~1" == "arm64" (set __VCBuildArch=arm64)
    if /i "%~1" == "wasm"  (set __VCBuildArch=arm64_x64)
) else if /i "%PROCESSOR_ARCHITECTURE%"=="ARM" (
    if /i "%~1" == "x86"   (set __VCBuildArch=arm_x86)
    if /i "%~1" == "x64"   (set __VCBuildArch=arm_x64)
    if /i "%~1" == "arm"   (set __VCBuildArch=arm)
    if /i "%~1" == "arm64" (set __VCBuildArch=arm_arm64)
    if /i "%~1" == "wasm"  (set __VCBuildArch=arm_x64)
) else (
    if /i "%~1" == "x86"   (set __VCBuildArch=x86)
    if /i "%~1" == "x64"   (set __VCBuildArch=x86_x64)
    if /i "%~1" == "arm"   (set __VCBuildArch=x86_arm)
    if /i "%~1" == "arm64" (set __VCBuildArch=x86_arm64)
    if /i "%~1" == "wasm"  (set __VCBuildArch=x86_x64)
)

Solution - build-runtime

For build-runtime the fix should ultimately be similar. I don't think we need __HostArch at all and instead should just detect it implicitly, requiring users to run the batch script from the correct command line environment.

If we wanted to keep __HostArch, it should correctly default to matching PROCESSOR_ARCHITECTURE if otherwise unspecified. It should likewise be used for the host portion of host_target. __TargetArch should then be used for the target portion.

Author:	tannergooding
Assignees:	-
Labels:	area-Infrastructure-coreclr, untriaged
Milestone:	-

[tannergooding]

Ultimately, I'd like to build "natively" for a given platform. So on Arm64 I should build for Arm64 and using the Arm64 tools that now exist by default. I should likewise build for x64 and using the x64 tools by default on x64; and for x86 and using the x86 tools by default on x86.

Likewise, I'd prefer using native build tools for cross-compilation scenarios. So building for Arm64 on x64 should use the x64_arm64 cross toolchain. The same could be extrapolated for other architectures where host_target should default to the actual host and actual target, with users being able to optionally override.

[filipnavara]

Related: #76079

[jkotas]

Ultimately, I'd like to build "natively" for a given platform

I would like to build the runtime using the fastest compilers flavor available. One of the reasons behind the current setup is that x86_x64 compilers used to be faster and took less memory than x64 compilers. Are x64 compilers faster than the x86_x64 compilers these days?

[tannergooding]

The TL;DR is that across a range of computers/laptops I tested on, there is a slight to significant advantage for using the native tools, more-so when compiling for release where it was nearly 25-30s faster on average for -subset clr.

One of the reasons behind the current setup is that x86_x64 compilers used to be faster and took less memory than x64 compilers

The JIT itself builds using amd64 and is "native" already. Only the paths which use init-vs-env.cmd end up with x86_x64. Almost everything else in the build process also defaults to 64-bit from .NET itself, to VS, CMake, Ninja, and beyond.

As for being faster, that's not really been the case for many years and you'll be hard pressed to find a scenario that shows x86 wins out in perf due to having half-width pointers. x64 does take 2x the space per pointer, but that is often offset due to having more registers and a more efficient calling convention, which ultimately can allow less overall memory access and less total computation.

As for taking "less memory", this really depends. For processes that scale up to the number of cores, 4GB of RAM is restrictive on high end systems and can cause a lot of thrashing. When they scale out to multiple processes instead this is less of a concern. In our case, there isn't that measurable a difference because everything else is 64-bit already and the main .NET Host that is driving everything will sit between 2-10GB depending on which stage of the build it's doing (this is just for -subset clr).

Are x64 compilers faster than the x86_x64 compilers these days?

As with anything "it depends". For debug/checked/release on Windows we're within the difference of milliseconds for just building jit and nothing else. We're within the realm of about 10 (debug) to 30 (release) seconds for building -subset clr (tested across multiple machines, including laptops). In both cases this is in favor of the x64 host compiler.

Most of our memory consumption comes from the .NET Host which is 64-bit anyways and consumes somewhere between 5-10GB on my box.

In the case of other more complex builds (like say building LLVM/Clang itself), their project is complex enough it requires a 64-bit host linker or it will OOM: https://clang.llvm.org/get_started.html (search for -Thost=x64). Clang in general is more resource intensive and scales up more than MSVC. We ultimately use the "native" host compiler by default on all non-Windows setups.

[jkotas]

We're within the realm of about 10 (debug) to 30 (release) seconds for building -subset clr

I do not see significant difference on my standard-issue HP desktop (Intel i9, 128GB). build-runtime.cmd -x64 -release takes 155sec (+-2 sec) with both x64 and x86_amd64 toolsets for me. It takes more memory, but it is not slower at least.