[UseDotNetV2] Not setting "no_proxy" nuget config on Windows agents

[gigi81]

Entering this information will route you directly to the right team and expedite traction.

Question, Bug, or Feature?
Type: Bug

Enter Task Name: UseDotNet

Environment

• Server - Azure Pipelines or TFS on-premises? Azure Pipelines
• Agent - Hosted or Private: Private

Issue Description

When UseDotNet (V2) runs on a Windows agent, it runs a series of nuget.exe commands at the end to setup the proxy configuration. However these series of commands do not set the "no_proxy" parameter resulting in a wrong configuration and following dotnet restore step failing.

Task logs

Setting up proxy configuration for NuGet.
D:\ado\agents\agent-1_work_tool\NuGet\4.4.1\x64\nuget.exe config -set http_proxy=http://myproxy.com:8080
D:\ado\agents\agent-1_work_tool\NuGet\4.4.1\x64\nuget.exe config -set http_proxy.user=undefined
D:\ado\agents\agent-1_work_tool\NuGet\4.4.1\x64\nuget.exe config -set http_proxy.password=undefined

Work Around

Added this step after UseDotNet@2 to make it work.

- task: CmdLine@2
  displayName: 'Nuget config work around'
  condition: eq( variables['Agent.OS'], 'Windows_NT' )
  inputs:
    script: 'nuget.exe config -set no_proxy=.mydomain.com'

Fix

The issues is in nugetinstaller.ts
Which is not using the proxyBypassHosts parameter to set the no_proxy nuget parameter.
There should be another command issued to set the no_proxy parameter.

[github-actions]

This issue is stale because it has been open for 180 days with no activity. Remove the stale label or comment on the issue otherwise this will be closed in 5 days

[gigi81]

Issue still present

[github-actions]

This issue is stale because it has been open for 180 days with no activity. Remove the stale label or comment on the issue otherwise this will be closed in 5 days

[gigi81]

Still an issue

[wkz123]

For the love of... please fix this!

[danko-d]

You cannot simply use the content of .proxybypass for the no_proxy setting of nuget.config, because they are different formats:

• In .proxybypass you use a regex, according to https://github.com/microsoft/azure-pipelines-agent/blob/master/docs/design/proxy.md
• The no_proxy setting in nuget.config is a comma-separated list of domains that bypass the proxy server. (see https://learn.microsoft.com/en-us/nuget/reference/nuget-config-file)

For example, a valid .proxybypass like my\.domain\.org wouldn't work for the no_proxy setting.

[mhudasch]

Big thank you @danko-d for mentioning this.

It took me several days to investigate that:

Hosts that should be bypassed by the agent are defined according to https://learn.microsoft.com/en-us/azure/devops/pipelines/agents/proxy?view=azure-devops&tabs=windows#specify-proxy-bypass-urls
As described these expressions are in ECMA Script regex escaped format (for example my\.bypasshost\.org) so that those lines can easily used by the pipeline tasks running in node (so the expressions must be in this format or else handling by other tasks will break)

Later those setting are string joined with comma and added to the service user nuget.config (%AppData%/NuGet/NuGet.Config) with using nuget config set no_proxy as suggested by @gigi81.
This is problematic for several reasons (and for a long time at that). When running a server with multiple agent services using the same service user changes to the NuGet.config file on user-level impact all paralell executing jobs by the agent services.
The 'workaround' by @gigi81 using nuget config set no_proxy before restore becomes a racecondition to edit the NuGet.config file by multiple jobs and might fail randomly (and very hard to reproduce).
It would be way better to take the agent settings and set the environment variables that are supported by nuget.exe for the best scoping of these settings.

Finally nuget.exe (since v4+) supports proxy settings in NuGet.config but only on user-level, and http_proxy / no_proxy environment variables. As you can see here https://github.com/NuGet/NuGet.Client/blob/a32bce39889f724fbd11cfd12e946f802168b583/src/NuGet.Core/NuGet.Configuration/Proxy/ProxyCache.cs#L80 if there is http_proxy setting present in the users' NuGet.config file, all settings (inlcuding no_proxy) take precedence over environment variables. The previously joined no_proxy values get splitted again and put into the bypassList. Later those values are used here https://github.com/NuGet/NuGet.Client/blob/a32bce39889f724fbd11cfd12e946f802168b583/src/NuGet.Core/NuGet.Configuration/Proxy/WebProxy.cs#L76. As you can see in the WildcardToRegex method, those values in the list are regex-escaped again which causes a 'double-escaping' turning our sample my\.bypasshost\.org to my\\.bypasshost\\.org which in turn never matches the incoming host of a nuget package source.

I recognize this is an edge case but I would really like a fix for this issue and maybe in the span of two years no less.