Add DOTNET_HOST_PATH property when initializing for framework-dependent apps

[elinor-fung]

Add DOTNET_HOST_PATH runtime property to represent the path to the dotnet executable corresponding to the runtime being used to run the application. Applications will be able to retrieve this value with AppContext.GetData("DOTNET_HOST_PATH").

This goes through the properties passed directly at initialization, instead of just the callback on the host-runtime contract. Currently, AppContext is populated with the runtime properties from initialization and does not flow through to calling the host-runtime callback. Long-term, I think it should, but that is a much more impactful change.

cc @dotnet/appmodel @AaronRobinsonMSFT @baronfel @jaredpar @richlander

See #88754 (comment)

[Copilot]

Pull Request Overview

This PR adds a new DOTNET_HOST_PATH runtime property that represents the path to the dotnet executable corresponding to the runtime being used to run framework-dependent applications. The property is only set for framework-dependent apps, not self-contained ones, since self-contained apps don't have a corresponding dotnet executable.

Key changes:

• Adds the runtime property during hostpolicy initialization for framework-dependent apps only
• Refactors executable extension handling to use consistent macros across platforms
• Adds comprehensive test coverage for the new property in various scenarios

Reviewed Changes

Copilot reviewed 13 out of 13 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
src/native/corehost/hostpolicy/hostpolicy_context.cpp	Implements the logic to set DOTNET_HOST_PATH property for framework-dependent apps
src/native/corehost/hostpolicy/coreclr.h	Adds DotNetHostPath enum value to common_property
src/native/corehost/hostpolicy/coreclr.cpp	Maps DotNetHostPath enum to "DOTNET_HOST_PATH" string
src/native/corehost/hostmisc/utils.cpp	Refactors to use EXE_FILE_EXT macro instead of pal::exe_suffix()
src/native/corehost/hostmisc/pal.h	Defines EXE_FILE_EXT macro and removes exe_suffix() functions
src/installer/tests/TestUtils/Constants.cs	Adds constant for the new runtime property name
src/installer/tests/TestUtils/Assertions/CommandResultExtensions.cs	Makes class partial to support additional assertion methods
src/installer/tests/TestUtils/Assertions/CommandResultExtensions.RuntimeProperty.cs	Adds helper methods for asserting runtime property presence/absence
src/installer/tests/HostActivation.Tests/SelfContainedAppLaunch.cs	Tests that self-contained apps don't have the property
src/installer/tests/HostActivation.Tests/RuntimeProperties.cs	Comprehensive tests for the new property in various scenarios
src/installer/tests/HostActivation.Tests/NativeHosting/GetFunctionPointer.cs	Tests property behavior in native hosting scenarios
src/installer/tests/HostActivation.Tests/NativeHosting/ApplicationExecution.cs	Tests property in native application execution
src/installer/tests/AppHost.Bundle.Tests/AppLaunch.cs	Tests property behavior for bundled applications

[jaredpar]

@baronfel at the .NET SDK tool level DOTNET_HOST_PATH will be available via both AppContext and environment variables. In which order should tools consult these values? My expectation is environment first and AppContext second but want to confirm. Or possibly in SDK the host, msbuild, will always set the environment variable?

@jjonescz, @JoeRobich

[am11]

My expectation is environment first and AppContext second but want to confirm.

It makes sense. Traditionally, tools in various ecosystems use this order of precedence:

1. Configuration file (e.g., .csproj, JSON config, etc.)
2. Environment variables
3. Command-line arguments (highest precedence)

So environment variables overriding AppContext (from .csproj) is consistent with common patterns.

[richlander]

One scenario where I've had challenges is accessing copy-local files. dotnet run has a different working directory than ./bin/Debug/.../myapp. I rely on typeof(Program).Assembly.Location to get the containing directory. I am guessing that this new ENV won't change that. Is that correct?

Another way of asking this is that there multiple typeof(Blah).Assembly.Location patterns. Does this replace any of them?

[elinor-fung]

I rely on typeof(Program).Assembly.Location to get the containing directory. I am guessing that this new ENV won't change that. Is that correct?

Correct - this will not change that or other typeof(<..>).Assembly.Location uses. (For the app directory specifically, I'd go with AppContext.BaseDirectory)

[agocke]

For the app directory specifically, I'd go with AppContext.BaseDirectory

Agreed. Note that Assembly.Location doesn't work for anything without assemblies, like single-file (self-contained or FX-dependent), assemblies loaded from memory, or Native AOT. Also, for working directory I'd use Environment.CurrentDirectory.

[richlander]

Got it. That context helps. Thanks!

Do we have a first internal user for this ENV so that we can validate the design?

[baronfel]

My expectation is environment first and AppContext second but want to confirm.

It makes sense. Traditionally, tools in various ecosystems use this order of precedence:

1. Configuration file (e.g., `.csproj`, JSON config, etc.)

2. Environment variables

3. Command-line arguments (highest precedence)

So environment variables overriding AppContext (from .csproj) is consistent with common patterns.

@jaredpar agree entirely with @elinor-fung here. AppContext would be a last-resort, and the dotnet CLI (and MSBuild SDK Resolver in VS scenarios) would be responsible for setting the env var for 'general' usage by tools.

[richlander]

Are there any other CLI ENVs that should be generally productized. Whenever we create those, it is typically an indication that there is a legit problem to be solved and it will often be the case that it is not unique to the CLI/SDK.

[jkotas]

AppContext would be a last-resort, and the dotnet CLI (and MSBuild SDK Resolver in VS scenarios) would be responsible for setting the env var for 'general' usage by tools.

What's the user scenario that this last resort is fixing - given that CLI and MSBuild has to be doing what they have been doing so far?

[jaredpar]

What's the user scenario that this last resort is fixing - given that CLI and MSBuild has to be doing what they have been doing so far?

Components inside of Visual Studio / VS Code that launch dotnet processes. Concrete example: test explorer.

[jkotas]

Components inside of Visual Studio / VS Code that launch dotnet processes. Concrete example: test explorer.

Could you please elaborate on the details how this scenario works?

I assume that there are two runtimes involved in the test explorer scenario: VS runs on the private .NET runtime bundled in the VS and the user code runs in whatever runtime the user test project references. Where are the process launches where this property helps? It cannot help for launching user code from the VS.

[baronfel]

@jkotas Check out this section in a doc I wrote for some details: https://hackmd.io/pKvDHW89TwOKPbG7FvnLWw?view#Cracks-in-the-sidewalk

The problem happens when build time components are enlightened about local SDKs, and things of that nature, but the apphosts and other tools that need a Runtime are not. This hits developers working on dotnet-org repos fairly often, but is a general problem.

[elinor-fung]

Are there any other CLI ENVs that should be generally productized.

Not that I could think of - but I also didn't know DOTNET_HOST_PATH was a thing until two years ago 🫤. I did a quick search through the sdk repo and no other environment variables I found really stood out to me as general runtime usage.

[AaronRobinsonMSFT]

This looks good from my perspective. Waiting on @jkotas for any additional queries.

[jkotas]

I am missing the end to end here. I expected that this is going to make something significantly simpler, but I hear that it is only going to make things even more complicated. It would be useful to see several PRs that show end-to-end. I am not sure whether getting this in at the last minute .NET 10, without having time to validate that it actually solves any problems end-to-end, is a good idea.

Check out this section in a doc I wrote for some details: https://hackmd.io/pKvDHW89TwOKPbG7FvnLWw?view#Cracks-in-the-sidewalk

This doc talks about the same problem as what the local SDK feature is trying to solve. As far as I know, we are going to edit PATH and set DOTNET_ROOT to make the local SDKs work. Why can't this be our one and only way to deal with this? I understand that it is no prettiest, but nobody was able to come with anything better that actually works.

Also, I think we should be moving towards launching processes using myapp.exe whenever possible instead of dotnet myapp.dll. dotnet myapp.dll comes with worse security - dotnet binary has to have the least common denominator of security mitigations such as CET since the same .exe has to work for old runtimes, bad diagnosability - you see a ton of dotnet.exes running and it is hard to tell what purpose is each one for, and experience dichotomy for self-contained or native AOT compiled. Creating more first-class support for DOTNET_HOST_PATH is going to make us gravitate towards dotnet myapp.dll more that I do not think is the best place to be in.

[jaredpar]

@jaredpar agree entirely with @elinor-fung here. AppContext would be a last-resort, and the dotnet CLI (and MSBuild SDK Resolver in VS scenarios) would be responsible for setting the env var for 'general' usage by tools.

@baronfel, @rainersigwald do we have an issue tracking doing this work? Want to make sure this doesn't get lost in the shuffle

[agocke]

I thought the goal here was to replace DOTNET_HOST_PATH entirely, not supplement it.

The problem happens when build time components are enlightened about local SDKs, and things of that nature, but the apphosts and other tools that need a Runtime are not. This hits developers working on dotnet-org repos fairly often, but is a general problem.

I don't understand this, actually. The example given in the doc will not be any better -- in order to run testhost.exe and have it find a local runtime there's only one option: DOTNET_ROOT. This variable doesn't help with finding DOTNET_ROOT. In fact, even if you found the current runtime location, the current runtime is often the wrong choice since you might be running tests on a different architecture than the one you launched dotnet with.

[elinor-fung]

I thought the goal here was to replace DOTNET_HOST_PATH entirely, not supplement it.

I had thought that it would replace how the SDK computes that value in the short term. And it would be a path to replacing DOTNET_HOST_PATH in the longer term - given some sort of breaking change / notice that the SDK will stop setting it and AppContext should be used instead.

Creating more first-class support for DOTNET_HOST_PATH is going to make us gravitate towards dotnet myapp.dll more that I do not think is the best place to be in.

Great point about dotnet vs apphost. When @agocke and I had discussed this, we wanted it to be better support, but not quite first-class. Perhaps a concrete version of this is: would we be fine with / rather say 'if you want to do this, do something like below'

string dotnetDir = Path.GetDirectoryName(Path.GetDirectoryName(Path.GetDirectoryName(RuntimeEnvironment.GetRuntimeDirectory())));
string dotnetPath = string.IsNullOrEmptry(dotnetDir) ? null : Path.Join(dotnetDir, "dotnet");

[jkotas]

if you want to do this, do something like below

First, I would recommend invoking the .exe without going through dotnet (it assumes that you have DOTNET_ROOT set for local runtimes).

If they really want to go through dotnet for reasons, the snippet you have suggested works for framework dependent apps running on shared runtime. There are other snippets that work in different situations. For example, the following works for any app as long as the environment is setup for local runtimes properly:

string dotnetRoot = Environment.GetEnvironmentVariable("DOTNET_ROOT");
return (dotnetRoot != null) ? Path.Join(dotnetRoot, "dotnet") : "dotnet";

[elinor-fung]

If they really want to go through dotnet for reasons,

Yes, 'want to do this' would be want to actually go through dotnet - but the recommendation being going through the apphost if possible.

There are other snippets that work in different situations.

I think relative to the runtime directory should handle any framework-dependent app (local SDK via global.json, local runtime via DOTNET_ROOT, custom search via publish-time apphost configuration, default global/registered). Self-contained seems to me questionable as a scenario for this - trying to launch a framework-dependent app making it use the runtime of a self-contained app should be a non-scenario and launching a self-contained app should be via apphost where there should be no need to do anything other than run it. And anything would probably want some fallback per your snippet of just dotnet (whatever is on the path).

[jkotas]

Self-contained seems to me questionable as a scenario for this

We would like to start NAOT-compiling parts of the SDK. If SDK has a need to find dotnet to launch child processes, it is likely that we are going to run into it in the part that we want to NAOT-compile, sooner or later. How are we going to solve that? Also, how are we going to find that it is broken in the first place given that AppContext.GetData("DOTNET_HOST_PATH") is not going to generate any AOT compat warnings?

[agocke]

SDK has a need to find dotnet to launch child processes, it is likely that we are going to run into it in the part that we want to NAOT-compile, sooner or later

In this particular situation the launching exe is the host, so presumably it could pass this information directly into the AOT binary when it does the pinvoke. I agree that this would be an sdk-specific contract, though.

[elinor-fung]

I was referring to the self-contained scenario as a self-contained regular app trying to launch another app using its own self-contained runtime. Native AOT SDK would be trying to launch another app using another (specific) runtime - not its own. And I do still think that is not a generic scenario we would try to enable.

#88754 (comment) also acknowledged that native AOT SDK would need something different. None of RuntimeEnvironment, DOTNET_ROOT, or AppContext would cover that case.

[jkotas]

None of RuntimeEnvironment, DOTNET_ROOT, or AppContext would cover that case.

DOTNET_ROOT would cover that case in the SDK scenario.

[elinor-fung]

How? Running SDK commands does not require a user setting DOTNET_ROOT, so unless we say the sdk-specific contract @agocke referred to is that the host has to go and make sure it is always set only for native AOT SDK case, it won't be set.

[jkotas]

You are right that you do not need to set DOTNET_ROOT when executing SDK commands. SDK sets the DOTNET_ROOT for you as needed to make things work. For example, try dotnet run a program that prints all environment variables - you should see DOTNET_ROOT.* being set.

Why is it not enough to just set DOTNET_ROOT for all scenarios (SDK can set it if it is not set already)?

[elinor-fung]

Since environment variables just propagate, I don't know that the host/runtime should assume that because you are running a .NET binary, all child processes should use the same runtime root. It generally feels off to me to have the host/runtime set environment variables, particularly with broad effects.

[jkotas]

I am not suggesting host/runtime to set DOTNET_ROOT. I agree that it is not be desirable for the reasons you have mentioned.

I am suggesting SDK to set DOTNET_ROOT for all child processes. SDK sets DOTNET_ROOT for some child processes today. Can it set DOTNET_ROOT for all child processes?

[elinor-fung]

I think the SDK could do that (maybe DOTNET_ROOT_<arch> instead of DOTNET_ROOT). I feel like that's come up before, but I don't recall the discussion/outcome - @baronfel / @nagilson?

It would still have the current problem of determining what the value should be though. That computation could be 'use AppContext.("DOTNET_HOST_PATH")' (this change) or 'use path relative to RuntimeDirectory' (no runtime change, guidance to explicitly rely on relative path from runtime directory for framework-dependent).

[jkotas]

Is dotnet the only external SDK entrypoint? Current DOTNET root is Path.GetDirectoryName(Environment.ProcessPath) for dotnet process, no assumptions about directory structure are needed. Child processes launched internally by the SDK would be covered by DOTNET_ROOT.* set by the external SDK entrypoint.

[elinor-fung]

Is dotnet the only external SDK entrypoint?

I think so - as in the entrypoint being the dotnet executable running the dotnet.dll SDK assembly. @baronfel can correct me if that is not the case.

With local SDKs via global.json, the current dotnet process isn't necessarily the one that corresponds to the SDK/runtime being used though.

[marcpopMSFT]

Is the proposal that the SDK inside of program.cs set DOTNET_ROOT if it's not already set or are you looking for us to find all places we launch child processes to specifically add this there?

[jkotas]

Is the proposal that the SDK inside of program.cs set DOTNET_ROOT if it's not already set

Yes, it would be the idea.

The higher level goal is to make the extra logic required to support local runtime and SDK scenarios as simple as possible. The original reason behind this PR was to simplify it, but #118249 (comment) comment suggested that it would make it even more complicated than it is today.

[agocke]

Note: I think this API is still useful for users who want to start a managed app from a managed app (completely separate from the SDK). For apps without an apphost they would use the apphsot path directly, while apphost apps would use Path.GetDirectoryName(AppContext.GetData("DOTNET_HOST_PATH")). If we want to encourage the use of the apphost we could switch this around and create an AppContext variable for DOTNET_ROOT, but that feels like it might be confusing since we also have the DOTNET_ROOT environment variable (and they could both be set). Since it's easy to move between host path and DOTNET_ROOT when you know one or the other, it seems the most important thing is to have some API to easily access one or the other.

[richlander]

I seem to remember we had a long very similar conversation a maybe five years ago. @jaredpar was a key part of that. The conclusion then was that we could only design a system that had no side-effects and didn't require special documentation and warnings. We found that very hard to design and I think gave up.

[AaronRobinsonMSFT]

I think this API is still useful for users who want to start a managed app from a managed app

I agree with this in principle, but aside from the SDK, who is doing this? Do we have telemetry or non-SDK customer feedback that indicates this mechanism is needed? It appears like a cheap/easy decision, but given the duration and complexity of this conversation I'm suspicious the answer is all that clear.

[agocke]

It sounds like xoofx was running into this, and presumably not in an SDK scenario. I’ve hit this in MSBuild tasks. I’m not sure whether MSBuild tasks count as an SDK scenario.

[nagilson]

aside from the SDK, who is doing this?

I could see this being used in C#DK, and we also found a use for an internal project (can't mention that here), though this is also managed by the SDK Team, it's not the SDK 😁

[jkotas]

I think this API is still useful for users

If we can convince ourselves that this is worth having public API for, I would rather build it as a proper API. Python has API like that: sys.executable. Proper API makes it 100% pay-for-play and gives us full flexibility in the implementation. We can consider giving it more self-describing name in the process. I do not particularly like Host in the current name since "Host" is overloaded term.

I could see this being used in C#DK

Could you please share a link where you would use it in C# SDK?

[nagilson]

I think this API is still useful for users

If we can convince ourselves that this is worth having public API for, I would rather build it as a proper API. Python has API like that: sys.executable. Proper API makes it 100% pay-for-play and gives us full flexibility in the implementation. We can consider giving it more self-describing name in the process. I do not particularly like Host in the current name since "Host" is overloaded term.

I could see this being used in C#DK

Could you please share a link where you would use it in C# SDK?

Sure, but I mean C# DevKit. https://github.com/dotnet/vscode-dotnet-runtime/blob/6805569d16253ec4535be1666c6caf5e4759031a/vscode-dotnet-runtime-library/src/Acquisition/DotnetPathFinder.ts#L403 is in node.js but @JakeRadMSFT is thinking of having some of this logic in a separate .NET application for VS and VS Code that can find .NET runtimes if the PATH is not set and a user is using the global.json paths feature. In this use case, we rely on the fact that dotnet --list-runtimes will always use the real assembly path to output the runtime folder, and we could instead leverage an API like this. We needed to do this because a path such as '/usr/bin/snap'
is a polymorphic executable where the realpath of it doesn't resolve correctly to dotnet.

[jkotas]

separate .NET application for VS and VS Code that can find .NET runtimes if the PATH is not set and a user is using the global.json paths feature

How is the separate .NET application going to find the runtime to run on? It would be useful to see the actual code for all this.

[rainersigwald]

Is dotnet the only external SDK entrypoint?

Client applications using the MSBuild API to load and/or build projects are also an "SDK" entrypoint IMO, so we should extend the DOTNET_ROOT setting to them (that'll be an MSBuild change).

I’m not sure whether MSBuild tasks count as an SDK scenario.

I'd say "yes" in general, and include Framework MSBuild-from-VS launching a .NET application in the scenario list (though we hope to reduce the need for those with .NET TaskHosts).

[rainersigwald]

I don't think I have strong feelings about this PR, but as I read it it doesn't address the main scenario I am concerned about, which is IDEs.

An IDE

1. Runs on some runtime of its own (.NET Framework for VS, .NET for C#DK)
2. Loads MSBuild projects (using the .NET SDK resolver, which now handles local SDKs with local runtimes)
3. Builds projects (launches MSBuild workers that run tasks that may launch tools from the resolved SDK)
4. Launches the built projects for run, test or debugging (this can be MyProject.exe or dotnet MyProject.dll depending on project config, but needs to be able to locate the appropriate runtime--from a local SDK if used)

With SDK resolver changes we've gotten 3 worked out pretty well, but quickly ran into problems with 4 since it's decoupled from build.

What should IDEs do when they want to launch the application-under-development?

[jkotas]

What should IDEs do when they want to launch the application-under-development?

If they want to launch the application on a local runtime, they need to set DOTNET_ROOT environment variable to point to that runtime before launching the application.

I do not think there is any magic how an application can find the runtime in a custom location. The custom location has to be communicated to the app somehow. DOTNET_ROOT is that communication mechanism today. We can invent additional communication mechanisms, but I am not sure it would make things any easier.