[Proposal] Add an LARGE_ADDRESS_AWARE switch to the .NET compiler

[Unknown6656]

I sometimes write applications and/or libraries which need the LARGE_ADDRESS_AWARE-flag set. I usually solve this by calling editbin.exe as postbuild-event.

Would it be an good idea to add a corresponding switch directly to the C#/F#/.... compiler?
It would have the following pros:

• one could save a few miliseconds/seconds of compilation time and postbuild-event calls
• it is easier to use for regular users (when properly integrated into VisualStudio)
• would not influence the compiler speed a lot - as it is only one argument which must be parsed and one bit which must be written
• reduce the amount of possible error sources during execution and project build (e.g. when some uses editbin incorrectly)

Or am I the sole one appending the editbin-postbuild to every second project of mine?

[sharwell]

This is already implemented:

roslyn/src/Compilers/Core/Portable/Compilation/Compilation.cs

Lines 1644 to 1646 in 0846ba6

	// Large address aware (the standard says never to set this, the linker team says otherwise).
	// The loader team says that this is not overridden for managed binaries and will be respected if set.
	characteristics |= Characteristics.LargeAddressAware;

• Use X86 platform to get a binary that runs as a 32-bit image and is not large address aware
• Use Any CPU+Prefer32Bit to get a binary that runs as a 32-bit image and is large address aware

[AArnott]

Thanks, @sharwell. That's a very simple workaround.

How does it work when you're compiling for .NET Native though, where one must choose a CPU architecture rather than "any cpu"?

[tannergooding]

I agree, there should be a way to explicitly set these PE header flags as part of the compilation.

[sharwell]

@AArnott I've never used .NET Native. Sounds like a bug should be filed there to work with Any CPU outputs.

[tannergooding]

@sharwell, it also seems like a bug that you cannot set LAA for explicit x86, but it magically gets set for AnyCPU/Prefer32Bit.

There are native libraries you could potentially be interoping with that are not LAA, and will fail with that assumption.

There are also cases where you are explicitly targeting x86 because AnyCPU is invalid (explicit x86 prevents the binary from being loaded in a 64-bit processes, AnyCPU+Prefer32Bit does not) but where you may still want LAA.

[AArnott]

I'm not sure if .NET Native has a bug or not. I was just wondering about that scenario.

+1 to @tannergooding's points. We compile with x86 sometimes because we know we'll be loading x86 native binaries, so x64 is not an option. The compiler/build knows this and will actually emit warnings if we reference x86 assemblies while targeting AnyCPU ourselves. So yes, there should be a way to require x86 and still be LAA.

[monty241]

From the header I understand that this also might affect dll libraries. We create various vsto applications which should ship as anycpu if possible since some users have excel, word or outlook 32-bit and some 64-bit.

Is there a good way to have these libraries also run in Excel versions with LAA and use the memory advantage, without loosing the option to use them with 64-bit office?

[AraHaan]

I agree, I also load up an dll (using reflection) made in C++ /clr and using AnyCPU+Prefer32Bit on an 64 bit OS with all the other .NET Assemblies AnyCPU and that is then loaded as an 64 bit process, now when that c++/clr dll is loaded using reflection (as a 32 bit dll) it then fails.

So then I changed the exe to be X86 however because like was said above I think the compiler should set /largeaddressaware as a switch that we pass in instead.

[AraHaan]

Because of the fact that I made dotnet/project-system#6829 I would like to take a crack at this for the next compiler version, then an option can be added to the dotnet/Sdk respositroy, so then dotnet/project-system#6829 can then be done.

Why, because I feel that the X86 and AnyCPU and x64 would have no harm in setting /largeaddressaware, on 64 bit processes largeaddessaware makes it up to use up to 8 GB of RAM right on single objects?

So either way they can be benefits for not only 32 bit, but also 64 bit programs if added on the compiler as a switch instead of setting it explicitly with AnyCPU+Prefer32Bit. So I think one should remove it from AnyCPU+Prefer32Bit and instead extract it to a switch (so that the dotnet sdk can then add support for it), and then the project system can then add an setting to enable it with a check mark.

[AraHaan]

I see the latest C# compiler has the needed stuff in https://github.com/dotnet/roslyn/blob/master/src/Compilers/Core/Portable/Compilation/Compilation.cs#L2041-L2043

However where is the Visual Basic Compiler's copy for it, and where do I go to add compilation options to the compiler?

[Youssef1313]

I see the latest C# compiler has the needed stuff in https://github.com/dotnet/roslyn/blob/master/src/Compilers/Core/Portable/Compilation/Compilation.cs#L2041-L2043

However where is the Visual Basic Compiler's copy for it, and where do I go to add compilation options to the compiler?

@AraHaan, The Compilation.cs file you're referencing is both for C# and VB compilers. This is the abstract base class that's used by both languages. See:

roslyn/src/Compilers/CSharp/Portable/Compilation/CSharpCompilation.cs

Line 41 in f735d0a

public sealed partial class CSharpCompilation : Compilation

roslyn/src/Compilers/VisualBasic/Portable/Compilation/VisualBasicCompilation.vb

Lines 36 to 37 in f735d0a

	Public NotInheritable Class VisualBasicCompilation
	Inherits Compilation

[AraHaan]

Sweet so now all I need to do is figure out on adding /laa switch to both compilers and then having the thing call both methods in that Compilation class when it is specified explicitly.

Now for users targeting AnyCPU+Prefer32Bit should it call into that new method then to keep ABI as much as possible or should LAA only be set when adding the switch to the compiler instead?

[AraHaan]

After thinking more on this, I think LAA should be set unconditionally for every target (since it does it anyway for x64 too if I read the code correctly so I see why not set it in every target) I can't think of places where setting it at all would break things.

But before I do @tannergooding can you think of any places where changing the compiler to set LAA unconditionally could break any targets at all?

Also perhaps the unconditional setting of LAA could be disabled as well via an environment variable too (to restore what the method originally did if the change is accepted) (DOTNET_DISABLE_LARGEADDRESSAWARE=1)

[michaelcfanning]

If we introduce /largeaddressaware, the problem is that we have differing current default behaviors, /largeaddressaware- for requires 32-bit, /largeaddressaware+ otherwise. We could preserve this behavior in your change, however, and still allow users to explicitly provide /largeaddressaware+ for the former (requires 32-bit) scenario. I think that's probably the right choice.

Because otherwise, as you clearly understand, a compiler update might unexpectedly opt 32-bit code into LAA causing runtime issues that aren't caught in testing. And I'm not sure an environment variable can provide a good mitigation. For one thing, I think the OS loader would actually need to honor that variable. We'd also need to think about whether this environment variable was inherited by child processes. That could be bad in many cases, for correctness.

[NOTE: I edited this comment to remove concerns around /highentropyva and this change. I now believe requires32bit, when enabled, will result in /highentropyva+ being irrelevant, no matter what /largeaddressaware is set to.]