Enable building with a dotnet not on PATH

[eerhardt]

In #68918 we removed inspecting DOTNET_HOST_PATH environment variable. This broke the scenario where a specific dotnet "hive" was installed to a location not on the PATH.

When the .NET SDK commands invoke a sub-process (for example MSBuild), it sets the DOTNET_HOST_PATH environment variable to tell the sub-process "this is where the dotnet.exe that invoked this command is located". See #21237 and dotnet/cli#7311 for more info.

This change reverts the behavior back to respect DOTNET_HOST_PATH, and if it isn't set it will just use "dotnet" and let the OS take care of finding the executable on the PATH.

Fix #69150

cc @jaredpar @333fred @ericstj @vitek-karas @dsplaisted

[am11]

Thank you! Just ran into this error with rc1 daily build on osx-arm64. The workaround was to symlink the sfx dir like this:

# download daily build at ~/.dotnet8 (with `alias dotnet8=~/.dotnet8/dotnet`)
# expect 'dotent8 publish' etc. to work as it's always been..

# (hopefully a temporary) workaround
sudo ln -s ~/.dotnet8/shared/Microsoft.NETCore.App/8.0.0-rc.1.23371.3 /usr/local/share/dotnet/shared/Microsoft.NETCore.App/8.0.0-rc.1.23371.3

[333fred]

Before I approve a PR here I think I need a complete explanation of how dotnet is supposed to be found by a subprocess. DOTNET_HOST_PATH is not a variable that has been documented anywhere AFAICT, and from what I can recall @jaredpar was mentioning that it was an arcade invention (though I could be misremembering this and he was referring to a different variable?).

Additionally, if this is a revert of a prior commit, I would prefer it was actually a revert commit, not a new change.

[am11]

DOTNET_HOST_PATH is internally set by SDK for the entire toolchain. I think, it is not supposed to be set publicly but .NET Foundation tools are encouraged/free to rely on it.

[vitek-karas]

I think I need a complete explanation of how dotnet is supposed to be found by a subprocess.

The problem is that we really don't have one: dotnet/runtime#88754
Changes to this are problematic as there's no clear "right way" of doing things. In my opinion it's better to keep things as they are, or apply only targeted fixes, until we come up with a good higher-level design.

Adding @agocke for some more context as well.

[eerhardt]

I can't speak to the documentation, but this has been the way things work for ~6 years. Multiple places across the product use this mechanism.

• Multiple tools built into the .NET SDK
• NuGet
• ILLink in dotnet/runtime
• F#
• Xamarin
• Uno

@marcpopMSFT @dsplaisted @baronfel - is it possible to document this environment variable to address @333fred's concerns?

[eerhardt]

In my opinion it's better to keep things as they are, or apply only targeted fixes, until we come up with a good higher-level design.

@vitek-karas - by "keep things as they are" do you mean as they currently are in dotnet/roslyn main? Or "as they are" in .NET 8.0-preview6?

[vitek-karas]

@vitek-karas - by "keep things as they are" do you mean as they currently are in dotnet/roslyn main? Or "as they are" in .NET 8.0-preview6?

Sorry for not being clear - I meant the behavior before the break (basically what we shipped in .NET 7).

[baronfel]

Many community tools for making use of the SDK have added this as well - I'm thinking of Buildalyzer and Ionide.ProjInfo here specifically. We should document it as a way to ensure consistency of experience across a single scope/lifetime - often a single dotnet CLI command, but could also be things like using user-local SDKs with language tooling, for example. Overall the guidance for tools that want to use the same dotnet as a theoretical parent caller seems like it should be:

• check DOTNET_HOST_PATH and use that explicitly if set, else
• probe PATH according to the rules of your platform

[eerhardt]

Additionally, if this is a revert of a prior commit, I would prefer it was actually a revert commit, not a new change.

@333fred, there were multiple things being changed/fixed in #68918. If we reverted that commit, we would then need to re-open and re-fix the bug it was intended to be fixing - #67102.

[marcpopMSFT]

Many community tools for making use of the SDK have added this as well - I'm thinking of Buildalyzer and Ionide.ProjInfo here specifically. We should document it as a way to ensure consistency of experience across a single scope/lifetime - often a single dotnet CLI command, but could also be things like using user-local SDKs with language tooling, for example. Overall the guidance for tools that want to use the same dotnet as a theoretical parent caller seems like it should be:

• check DOTNET_HOST_PATH and use that explicitly if set, else
• probe PATH according to the rules of your platform

Broader usage of the env variable appears to be somewhat limited: https://github.com/search?q=DOTNET_HOST_PATH++&type=code Doesn't mean we shouldn't support it as an alternative to the path though as it has come up a few times (some customers have requested SDK path be part of global.json to solve this). Seems like an area worth investing in.