Handle nested failures in HttpRequestException

`HttpClientHandler` will automatically resubmit an HTTP request that
fails in certain circumstances. One such circumstance is when
`UseDefaultCredentials` is set to `true``, alternative credentials were
provided on the request, and the request failed with "401 Unauthorized",
then it will resubmit using the default credentials. If an exception is
thrown when getting the default credentials that exception is not
handled, but is wrapped in `HttpRequestExceptio`n and thrown by
`SendAsync` instead of returning the original response with the failing
status code. See the following code snippet from `HttpWebRequest`:
https://referencesource.microsoft.com/#System/net/System/Net/HttpWebRequest.cs,5535

When this happens in GVFS, the result is that GVFS does not recognize
the failure as being authentication related, so it does not refresh the
credential. Instead, it loops through all its retries, and eventually
fails the request. This is typically visible to users as a file system
exception (e.g. file not found) if the GVFS trigger was accessing a
virtual file or other operation on an individual file, or various errors
(including "this repository requires the GVFS protocol") for a `git
pull`. The symptoms will continue for the user until they remount the
GVFS enlistment, which forces GVFS to refresh its credential.

This commit adds an exception handler for `HttpRequestException`` to
`client.SendAsync`, which attempts to find the original failed response
embedded in the inner exception properties. If it does so, it logs a
warning and continues processing using the original failed response,
which will trigger the logic for handling the various possible status
codes. If it can't extract the original failed response, then it will
let the exception bubble up.

Signed-off-by: Tyrie Vella <tyrielv@gmail.com>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>

GVFS/GVFS.Common/Http/HttpRequestor.cs

@@ -134,6 +134,14 @@ protected GitEndPointResponseData SendRequest(
                 {
                     response = this.client.SendAsync(request, HttpCompletionOption.ResponseHeadersRead, cancellationToken).GetAwaiter().GetResult();
                 }
+                catch (HttpRequestException httpRequestException) when (TryGetResponseMessageFromHttpRequestException(httpRequestException, request, out response))
+                {
+                    /* HttpClientHandler will automatically resubmit in certain circumstances, such as a 401 unauthorized response when UseDefaultCredentials
+                     * is true but another credential was provided. This resubmit can throw (instead of returning a proper status code) in some case cases, such
+                     * as when there is an exception loading the default credentials.
+                     * If we can extract the original response message from the exception, we can continue and process the original failed status code. */
+                    Tracer.RelatedWarning(responseMetadata, $"An exception occurred while resubmitting the request, but the original response is available.");
+                }
                 finally
                 {
                     responseWaitTime = requestStopwatch.Elapsed;
@@ -278,5 +286,48 @@ private static string GetSingleHeaderOrEmpty(HttpHeaders headers, string headerN
 
             return string.Empty;
         }
+
+        /// <summary>
+        /// This method is based on a private method System.Net.Http.HttpClientHandler.CreateResponseMessage
+        /// </summary>
+        private static bool TryGetResponseMessageFromHttpRequestException(HttpRequestException httpRequestException, HttpRequestMessage request, out HttpResponseMessage httpResponseMessage)
+        {
+            var webResponse = (httpRequestException?.InnerException as WebException)?.Response as HttpWebResponse;
+            if (webResponse == null)
+            {
+                httpResponseMessage = null;
+                return false;
+            }
+
+            httpResponseMessage = new HttpResponseMessage(webResponse.StatusCode);
+            httpResponseMessage.ReasonPhrase = webResponse.StatusDescription;
+            httpResponseMessage.Version = webResponse.ProtocolVersion;
+            httpResponseMessage.RequestMessage = request;
+            httpResponseMessage.Content = new StreamContent(webResponse.GetResponseStream());
+            request.RequestUri = webResponse.ResponseUri;
+            WebHeaderCollection rawHeaders = webResponse.Headers;
+            HttpContentHeaders responseContentHeaders = httpResponseMessage.Content.Headers;
+            HttpResponseHeaders responseHeaders = httpResponseMessage.Headers;
+            if (webResponse.ContentLength >= 0)
+            {
+                responseContentHeaders.ContentLength = webResponse.ContentLength;
+            }
+
+            for (int i = 0; i < rawHeaders.Count; i++)
+            {
+                string key = rawHeaders.GetKey(i);
+                if (string.Compare(key, "Content-Length", StringComparison.OrdinalIgnoreCase) != 0)
+                {
+                    string[] values = rawHeaders.GetValues(i);
+                    if (!responseHeaders.TryAddWithoutValidation(key, values))
+                    {
+                        bool flag = responseContentHeaders.TryAddWithoutValidation(key, values);
+                    }
+                }
+            }
+
+            return true;
+
+        }
     }
 }