Skip to content
This repository has been archived by the owner on Jan 23, 2023. It is now read-only.

Pass Uri.IdnHost to WinHttpConnect #28818

Closed
wants to merge 0 commits into from
Closed

Pass Uri.IdnHost to WinHttpConnect #28818

wants to merge 0 commits into from

Conversation

MarcoRossignoli
Copy link
Member

fixes #28722

@davidsh
Copy link
Contributor

davidsh commented Apr 4, 2018

How did you verify this change? Did you do any manual testing and confirm what is being sent to WinHTTP APIs (via debugger) or on the wire (Wireshark etc.)?

@davidsh davidsh requested a review from a team April 4, 2018 18:43
@MarcoRossignoli MarcoRossignoli changed the title Pass Uri.IdnHost to WinHttpConnect [WIP] Pass Uri.IdnHost to WinHttpConnect Apr 4, 2018
@MarcoRossignoli
Copy link
Member Author

MarcoRossignoli commented Apr 4, 2018

no @davidsh i read only docs and followed thread discussion. I put this PR in WIP and i'll do manual verify, thank you for you advice/review.

@rmkerr
Copy link
Contributor

rmkerr commented Apr 4, 2018

Based on the documentation here, this does seem like the correct behavior:

pswzServerName [in]
Pointer to a null-terminated string that contains the host name of an HTTP server. Alternately, the 
string can contain the IP address of the site in ASCII, for example, 10.0.1.45. Note that WinHttp
does not accept international host names without converting them first to Punycode. For more 
information, see Handling Internationalized Domain Names (IDNs).

@Caesar1995 has been looking into this though and can probably provide the best feedback.

@davidsh
Copy link
Contributor

davidsh commented Apr 4, 2018

no @davidsh i read only docs and followed thread discussion. I put this PR in WIP and i'll do manual verify, thank you for you advice/review.

Thanks! In terms of the proposed fix, it is probably correct.

But we would like to see some sort of validation even with a simulated end-to-end scenario that might be manually done due to the difficulties of doing an IDN related automated test in the CI system.

@caesar-chen
Copy link
Contributor

caesar-chen commented Apr 4, 2018

Probably this PR will fail with the tests I added.

You will need to append [] around idnHost as well.

@davidsh
Copy link
Contributor

davidsh commented Apr 4, 2018

You will need to append [] around idnHost as well.

In terms of calling the WinHTTP API itself, I'm pretty sure it is ok to simply use Uri.IdnHost as input to the WinHTTP API call, even if it is an IPv6 literal wo/ enclosing brackets. In fact, adding brackets might generate an error with calling the WinHTTP API.

@caesar-chen
Copy link
Contributor

In fact, adding brackets might generate an error with calling the WinHTTP API.

Current implementation pass state.RequestMessage.RequestUri.Host for WinHttpConnect, and uri.Host will contain the brackets around IPv6 address.

What error/concern we may have here?

@davidsh
Copy link
Contributor

davidsh commented Apr 4, 2018

Normally, the brackets are not part of the actual IPv6 literal. They are used when IPv6 literals are part of a Uri which is something that is defined in the Uri RFC's.

See:https://www.ietf.org/rfc/rfc2732.txt

This document incudes an update to the generic syntax for Uniform
Resource Identifiers defined in RFC 2396 [URL]. It defines a syntax
for IPv6 addresses and allows the use of "[" and "]" within a URI
explicitly for this reserved purpose.

But IPv6 literals in general are not defined as having brackets in them:

See: https://tools.ietf.org/html/rfc5952

It's possible that WinHTTP APIs are resilient enough to ignore any brackets when an IPv6 literal is passed. Doing testing on this is recommended to validate our assumptions. But, therefore, I don't think we need to add any code to explicitly check for and add brackets if not present.

@caesar-chen
Copy link
Contributor

Doing testing on this is recommended to validate our assumptions.

I agree. I'm curious about what WinHttp will send if [] not specified in WinHttpConnect.

@MarcoRossignoli
Copy link
Member Author

MarcoRossignoli commented Apr 4, 2018

@davidsh @Caesar1995
i'm creating manual test using this site http://www.i18nguy.com/markup/idna-examples.html
Bangladesh http://বাংলাদেশ.icom.museum

        [Fact]
        [Trait("MyTrait", "MyTrait")]
        public async Task ManualTest()
        {
            var idna = System.Text.Encoding.UTF8.GetString(Convert.FromBase64String("aHR0cDovL+CmrOCmvuCmguCmsuCmvuCmpuCnh+Cmti5pY29tLm11c2V1bQ=="));
            var handler = new WinHttpHandler();
            handler.Proxy = new WebProxy("127.0.0.1", 8888);
            handler.WindowsProxyUsePolicy = WindowsProxyUsePolicy.UseCustomProxy;
            using (HttpClient client = new HttpClient(handler))
            {                                
                var response = await client.GetAsync(idna);
            }
        }

using codeBase64 to avoid VS ask to re-save file.
I can confirm with Fiddler that Agent translation(Chrome) and WinHttp translation are equal http://xn--54b6eqazv8bc7e.icom.museum/.
Could be great write a test to get log from WinHttp, i tried with DelegatingHandler, but it's too high on pipeline.

@rmkerr
Copy link
Contributor

rmkerr commented Apr 4, 2018

You can use online tools like this one to decode the IDN and verify that it is equal. I think the important thing is that the IDN encoded string really is being sent on the network. Did you see the IDN encoded version in fiddler?

@MarcoRossignoli
Copy link
Member Author

MarcoRossignoli commented Apr 4, 2018

@rmkerr sorry i replaced my post, if i setup Fiddler as proxy i can see tranlsated host.
idn

@rmkerr
Copy link
Contributor

rmkerr commented Apr 4, 2018

Awesome! Based on that, the IDN encoded host is getting sent.

It looks like the tests are hanging though. @Caesar1995 do you think that could be related to the ipv6 bracket issue?

@@ -785,7 +785,7 @@ private void EnsureSessionHandleExists(WinHttpRequestState state)
// Specify an HTTP server.
connectHandle = Interop.WinHttp.WinHttpConnect(
_sessionHandle,
state.RequestMessage.RequestUri.Host,
state.RequestMessage.RequestUri.HostNameType == UriHostNameType.IPv6 ? $"[{state.RequestMessage.RequestUri.IdnHost}]" : state.RequestMessage.RequestUri.IdnHost,
Copy link
Member Author

@MarcoRossignoli MarcoRossignoli Apr 4, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Caesar1995 @davidsh @rmkerr i added [] for IPv6 because i get 'The URL is invalid' for http://[::1234] address. In case my idea is ok do you prefer "["+state.RequestMessage.RequestUri.IdnHost+"]"(less allocations)?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

because i get 'The URL is invalid' for http://[::1234] address.

Could you elaborate on this? What part of the code will complain "The URL is invalid"? What test did you do?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you get that error for http://[::1234] or for http://::1234?

@MarcoRossignoli
Copy link
Member Author

MarcoRossignoli commented Apr 4, 2018

@davidsh @rmkerr @Caesar1995 if i create a fake IWebProxy i could try to add a unit test to verify traslation(at the moment i don't know how write a web proxy but i can investigate). Is this reasonable?

@MarcoRossignoli MarcoRossignoli changed the title [WIP] Pass Uri.IdnHost to WinHttpConnect Pass Uri.IdnHost to WinHttpConnect Apr 4, 2018
@MarcoRossignoli
Copy link
Member Author

MarcoRossignoli commented Apr 5, 2018

sorry @davidsh @rmkerr @Caesar1995 i did some mess with rebase(i wanted to integrate new tests on ipv6) i'll create a new PR, i don't know how to reopen a PR(but i think i can't do that).

@MarcoRossignoli MarcoRossignoli deleted the winhttp branch April 12, 2018 07:29
@karelz karelz added this to the 2.1.0 milestone Apr 12, 2018
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

WinHttp should accept Uri.IdnHost for WinHttpConnect()
5 participants