AndroidMessageHandler doesn't allow arbitrary HTTP methods

[tipa]

Android application type

Android for .NET (net6.0-android, etc.)

Affected platform version

VS 2022

Description

I am using HttpClient to send and receive data from servers via WebDAV.
To do so, Http methods like "PROPFIND" or "MKCOL" are required.
Using these http methods causes crashes in HttpClient.SendAsync:
System.Net.WebException: 'Expected one of [OPTIONS, GET, HEAD, POST, PUT, DELETE, TRACE, PATCH] but was PROPFIND'
It crashes in AndroidMessageHandler here: https://github.com/xamarin/xamarin-android/blob/f348163bdca996645ffbc5d47dee3d908eff640c/src/Mono.Android/Xamarin.Android.Net/AndroidMessageHandler.cs#L979

There has been an issue filed previously: #3544
Unfortunately it was closed and no workaround was presented.

Steps to Reproduce

var handler = new HttpClientHandler();
var request = new HttpRequestMessage(new HttpMethod("PROPFIND"), "https://test.com");
await new HttpClient(handler).SendAsync(request);

Did you find any workaround?

In legacy Xamarin Android I found a workaround by explicitly providing an instance of HttpClientHandler to the HttpClient constructor (instead of the default AndroidMessageHandler/AndroidClientHandler).
However, this workaround stopped working with .NET6 Android as apparently, the HttpClientHandler internally uses a AndroidMessageHandler, causing the problem to occur again.

My current "workaround" is to set the UseNativeHttpHandler in my .csproj to false. I can still manually create AndroidMessageHandler and set use them to initialize HttpClients, but when using a third-party library, that library might not offer a way to provide my own HttpClient, so it keeps using HttpClientHandler, even though it could use AndroidMessageHandler

So ideally, the AndroidMessageHandler could be changed so that it accepts arbitrary HTTP methods.
Alternatively, it would be good if there was a way to force a HttpClientHandler to be used without having to set the project-wide UseNativeHttpHandler property to false.

[grendello]

@tipa Unfortunately, the answer hasn't changed since #3544 was closed - AndroidMessageHandler uses Java's HttpURLConnection (that was the whole point of creating the handler) which does NOT support any other methods than listed here.
AndroidMessageHandler is merely a glorified proxy for the Java code, so we are in essence restricted by what it supports.

Alternatively, it would be good if there was a way to force a HttpClientHandler to be used without having to set the project-wide UseNativeHttpHandler property to false.

@simonrozsival can you think of a way to to do it in a compatible way?

[tipa]

@grendello Ok, that's what I expected :(
In your answer in the other issue you mention that okhttp is deprecated, is that really the case? I guess there are other reasons why okhttp cannot be used (maybe because it increases the app size as it is an additional dependency)?

And I assume something like this would be considered too hacky? It would be enough if I could subclass AndroidMessageHandler and implement the hack in my code (e.g. a virtual SetRequestMethod method)

var httpmethod = new System.Net.Http.HttpMethod("PROPFIND");
try { conn.RequestMethod = httpmethod.ToString(); }
catch (ProtocolException)
{
        Field field = Class.FromType(typeof(HttpURLConnection)).GetDeclaredField("method");
        field.Accessible = true;
        field.Set(conn, httpmethod.ToString());
}

[simonrozsival]

My current "workaround" is to set the UseNativeHttpHandler in my .csproj to false. I can still manually create AndroidMessageHandler and set use them to initialize HttpClients, but when using a third-party library, that library might not offer a way to provide my own HttpClient, so it keeps using HttpClientHandler, even though it could use AndroidMessageHandler.

@tipa I would suggest keeping $(UseNativeHttpHandler)=true and for the WebDAV requests create a new HttpClient with System.Net.Http.SocketsHttpHandler (that's the managed handler that HttpClientHandler uses under the hood when UseNativeHttpHandler is false) which should support any HTTP method you want. Third party libraries will keep using the native handler.

Does this solve your problem?

I guess there are other reasons why okhttp cannot be used (maybe because it increases the app size as it is an additional dependency)?

AFAIK okhttp is used internally on Android, but it's not available in the standard APIs and to use it in an app directly, the app needs to add a dependency on the library and app size will increase. There is also the Cronet library that can be loaded from Google Play Services so it doesn't increase the app size, but it isn't available on all Android devices. I think that we are stuck with HttpURLConnection for the time being.

[grendello]

@grendello Ok, that's what I expected :( In your answer in the other issue you mention that okhttp is deprecated, is that really the case? I guess there are other reasons why okhttp cannot be used (maybe because it increases the app size as it is an additional dependency)?

I meant the version of okhttp that ships with Android (I think it was/is v2?). The standalone version should be fine, there are bindings available on nuget.org, but I have never tested them.

And I assume something like this would be considered too hacky? It would be enough if I could subclass AndroidMessageHandler and implement the hack in my code (e.g. a virtual SetRequestMethod method)

var httpmethod = new System.Net.Http.HttpMethod("PROPFIND");
try { conn.RequestMethod = httpmethod.ToString(); }
catch (ProtocolException)
{
        Field field = Class.FromType(typeof(HttpURLConnection)).GetDeclaredField("method");
        field.Accessible = true;
        field.Set(conn, httpmethod.ToString());
}

That would probably break at runtime, because the Java side would still compare the method against its list of accepted ones. It would be better if you were able to use one of the okhttp bindings from nuget.org, I suppose.

[tipa]

@tipa I would suggest keeping $(UseNativeHttpHandler)=true and for the WebDAV requests create a new HttpClient with System.Net.Http.SocketsHttpHandler (that's the managed handler that HttpClientHandler uses under the hood when UseNativeHttpHandler is false) which should support any HTTP method you want. Third party libraries will keep using the native handler.

Does this solve your problem?

The problem with this is that I cannot provide a ServerCertificateCustomValidationCallback any more, which was possible with HttpClientHandler. And is it possible that ServerCertificateCustomValidationCallback also stopped working when using HttpClientHandler and UseNativeHttpHandler=false in .NET6. Is there any replacement for ignoring SSL errors?

Edit: I found I can use this on SocketsHttpHandler and hope that it use works as expected :)

httpHandler.SslOptions.RemoteCertificateValidationCallback = (reqMsg, cert, cetChain, policyErrors) => true;

That would probably break at runtime, because the Java side would still compare the method against its list of accepted ones.

I could test if that's actually the case. Would you consider implementing this hack in case it worked?

[grendello]

That would probably break at runtime, because the Java side would still compare the method against its list of accepted ones.

I could test if that's actually the case. Would you consider implementing this hack in case it worked?

I'm sorry but no. If setting the method this way worked, we could probably subclass HttpURLConnection (in our Java runtime package) to add a way to set that field using a custom function.

[simonrozsival]

@tipa the RemoteCertificateValidationCallback won't work as expected in .NET 6, because the current implementation doesn't override the OS validation. You'll have to use network_security_config.xml to trust your self-signed certificates instead.

[tipa]

@simonrozsival Oh, thanks for that info. Are there plans to port the old functionality to .NET 6? Is there an open issue that I can subscribe to?
Using network_security_config.xml is not an option as in my app, users can connect to their WebDAV servers which not necessarily have a trusted SSL certificate, so I implemented the option to ignore SSL errors. I already had a customer contact me because it stopped working for him after I released an update after I ported my app to .NET 6.
Setting UseNativeHttpHandler=false will also not allow me to ignore SSL validation, I would have to backport to legacy Android, right?

[simonrozsival]

@tipa I see. We have a tracking issue dotnet/runtime#45741 that is now assigned to the .NET 8 milestone, so we will likely improve the Android Crypto integration it in the next releases (.NET 8 should be released in November 2023), but there's no short-term solution for your app in .NET 6 itself I'm afraid. Maybe the OkHttp NuGet that @grendello suggested supports the combination of custom HTTP methods and ignoring invalid certificates? I'm not familiar with it so I can't promise it has the capabilities.

@grendello although we didn't resolve @tipa's problem, I think we should close this issue now, because we won't be allowing arbitrary HTTP methods in AndroidMessageHandler due to HttpURLConnection limitations.

[grendello]

@grendello although we didn't resolve @tipa's problem, I think we should close this issue now, because we won't be allowing arbitrary HTTP methods in AndroidMessageHandler due to HttpURLConnection limitations.

Agreed. @tipa, we'll definitely think about some form of solution for your problem, but I can't promise any time frame since our options with Android are quite limited atm.