-
Notifications
You must be signed in to change notification settings - Fork 29.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
dns: default to verbatim=true in dns.lookup() #37931
Conversation
Switch the default from false (reorder the result so that IPv4 addresses come before IPv6 addresses) to true (return them exactly as the resolver sent them to us.) Fixes: nodejs#31566 Refs: nodejs#6307 Refs: nodejs#20710 Reissue of nodejs#31567 Reissue of nodejs#37681
5bef0de
to
90a5e93
Compare
So, I went again through any single test that failed on my machine and tried to fix it. I found the following tests to fail and did the following to fix them. Let's also have a CI job to see what problems exist on the other setups. Explanations:
|
Explanations: ------------- test/parallel/test-cluster-message.js: add `family: 4` to `connect` because without it will default to `localhost` which could resolve to `::1` if IPv6 is available, while server listen address is hardcoded legacy IPv4. test/parallel/test-http-localaddress.js: add `family: 4,` to `connect` because `localhost` could resolve to `::1` if IPv6 is available, while server listen address is hardcoded legacy IPv4. test/parallel/test-http-upgrade-client.js: add `family: 4,` to `http.get` because `http.get` without specified host uses `localhost` which could resolve to `::1` if IPv6 is available, while server listen address is hardcoded legacy IPv4. test/parallel/test-http2-connect-options.js: add `family: 4` to `http2.connect` `options` because `localhost` might resolve to `::1` if IPv6 is available, causing `EINVAL` when binding to `127.0.0.2`, and server listen address is hardcoded legacy IPv4. test/parallel/test-https-localaddress.js: add `family: 4` to `https.request` `options` because `localhost` might resolve to `::1` if IPv6 is available, causing `EINVAL` when binding to `127.0.0.2`, and server listen address is hardcoded legacy IPv4. test/common/inspector-helper.js: add `family: 4` to `http.get` options to make sure to connect via legacy IPv4 `localhost`, because inspector by default listens on legacy IPv4 `localhost`. Fixes: test/parallel/test-inspect-async-hook-setup-at-inspect.js test/parallel/test-inspector-esm.js test/parallel/test-inspector-inspect-brk-node.js test/parallel/test-inspector-multisession-ws.js test/parallel/test-inspector-reported-host.js test/parallel/test-inspector-wait-for-connection.js test/parallel/test-inspector-waiting-for-disconnect.js test/sequential/test-inspector.js test/sequential/test-inspector-async-hook-setup-at-inspect-brk.js test/sequential/test-inspector-async-hook-setup-at-signal.js test/sequential/test-inspector-async-stack-traces-promise-then.js test/sequential/test-inspector-async-stack-traces-set-interval.js test/sequential/test-inspector-break-e.js test/sequential/test-inspector-break-when-eval.js test/sequential/test-inspector-console.js test/sequential/test-inspector-debug-brk-flag.js test/sequential/test-inspector-debug-end.js test/sequential/test-inspector-exception.js test/sequential/test-inspector-not-blocked-on-idle.js test/sequential/test-inspector-scriptparsed-context.js test/sequential/test-inspector-stop-profile-after-done.js test/sequential/test-inspector-stress-http.js test/parallel/test-net-dns-lookup.js: listen on `common.localhostIPv4` and add `family: 4` to `connect` to force legacy IPv4, because it is required by the assertion tests. Could be changed to DualStack compatibility by using `assert.match(ip, /^(127\.0\.0\.1|::1)$/);` and `assert.match(type.toString, /^(4|6)$/);`. test/parallel/test-net-remote-address-port.js: add `::1` to `remoteAddrCandidates`. Why does this fix it? test/parallel/test-tcp-wrap-listen.js: add ternary test with `hasIPv6` and use `bind` or `bind6` accordingly. test/parallel/test-timers-socket-timeout-removes-other-socket-unref-timer.js: add `family: 4` to `connect` because it will default to localhost being `::1` if IPv6 is available, and listen address is hardcoded to be legacy IPv4 address for `localhost`. test/parallel/test-tls-client-getephemeralkeyinfo.js: add `family: 4` to `connect` because without it will default to `localhost` which could resolve to `::1` if IPv6 is available, while server listen address is hardcoded legacy IPv4. test/parallel/test-tls-client-mindhsize.js: add `family: 4` to `connect` because without it will default to `localhost` which could resolve to `::1` if IPv6 is available, while server listen address is hardcoded legacy IPv4. test/parallel/test-tls-wrap-econnreset-localaddress.js: add `family: 4` to `connect` because without it will default to `localhost` which could resolve to `::1` if IPv6 is available, while server listen address is hardcoded legacy IPv4. test/sequential/test-https-connect-localport.js: remove `family: 4` from connect because listen uses `localhost` which maybe resolves to `::1`? NEEDS TESTING! If it fails --> force legacy IPv4 with `common.localhostIPv4`. test/sequential/test-inspector-open.js: add `family: 4` to `connect` because without it will default to `localhost` which could resolve to `::1` if IPv6 is available, while inspector is listening on legacy IPv4 `localhost` by default. test/sequential/test-net-connect-local-error.js: add `family: 4` to `optionsIPv4` and `family: 6` to `optionsIPv6`, because it seems connecting on IPv4 will default to `localhost` which then resolves to `::1` if IPv6 is available and will cause an error.
5f24bef
to
bdedf3a
Compare
Alright, so the preliminary results confirm something that I was suspecting. While we could replace Thus, I see two options:
In my opinion, the way to go currently is approach 2. Approach 1 should be something implemented in a separate PR that tries to be more DualStack-aware. In approach 1 it would probably also make sense to review the implementation of DNS resolution on listening and connecting and to make sure they resolve to the same values. |
SmartOS fixes:
|
Explanations: test/parallel/test-net-connect-options-port.js: add `family: 4` at various places to force `localhost` or default `localhost` to be resolved to legacy IPv4 addresses. test/parallel/test-net-pingpong.js: change `localhost` to `127.0.0.1` because test is aimed at legacy IPv4, so it can be forced to literal IPv4 address without problem. NEEDS TESTING: does `pingPongTest(0);` pass? test/parallel/test-net-remote-address-port.js: change `localhost` to `127.0.0.1` and force IPv4 by adding `family: 4` because SmartOS will resolve `localhost` or no address automatically to legacy IPv4 address (`127.0.0.1`). test/parallel/test-net-writable.js: listen on `common.localhostIPv4` and add `family: 4` to connect to force IPv4. because SmartOS will resolve `localhost` or no address automatically to legacy IPv4 address (`127.0.0.1`). test/sequential/test-https-connect-localport.js: revert previous change, i.e. readd `family: 4` to `https.get`. Change `localhost` on listen to `common.localhostIPv4` to force legacy IPv4.
130edc7
to
a45f066
Compare
FWIW I don't think modifying the tests just so they can pass on SmartOS is OK. Depending on the cause of the failure:
Either way, I don't think modifying failing tests is the right thing to do here. @nodejs/platform-smartos wdyt? The changes in bdedf3a LGTM though (thanks for the detailed explanation BTW, that's really good work). |
I think it's both. Refusing to use On the other hand, I don't understand why it is binding fine to
Without the explanations I probably would have lost track myself. It was necessary to have an overview over what needs to be addressed. But thanks for the acknowledgement. |
Do the Linux CIs have IPv6 connectivity? Even though I don't really want to admit it...if I remove the public IPv6 address on my system, I observe the same problem as on SmartOS: listening on I am not sure if this is related to the DNS module or to the resolver of the OS. If I do Edit: Thinking a bit about it I think we should adjust the tests. There is the chance (in theory) that having an IPv6 address configured or not might change during runtime. Since we can't listen DualStack on Edit2: It's related to |
I think we should ditch On another node, @bnoordhuis also suggested ditching A commit will follow shortly. |
Can we get another CI job? |
Great! So, what else needs to be discussed now in order to merge this? What should be mentioned in the documentation: usage of |
Co-authored-by: Antoine du Hamel <duhamelantoine1995@gmail.com>
I'm in agreement with @mcollina suggestion - #37931 (comment) as a way to make landing the change feasible. |
Is this kind of syntax supported by NodeJS/Javascript? |
Yes. That being said, this might be a case where an environment variable seems like a better choice (because it is configurable from outside the process, which is relevant because this affects users just as much as it does developers, and because that way it is inherited by Workers and child processes by default). |
Thx. Unfortunately, I haven't been able to find some example code how to do this. Still, I can't even find any documentation of this syntax.
Yes. But maybe we can implement both solutions. The inside process option to make it easier for the devs (though I feel this might lead to some devs just setting it 'to be sure' and never touching it again), and the outside option with higher precedence to eventually let the user decide. |
I am not very well versed in Javascript. I believe we have to do sth like this:
But that throws an error/warning if I couldn't find a way to make |
Implementation is done in #38099, once it has landed it should unblock this PR. |
Was discussed in the TSC meeting today. From those who were there (only a small subset) consensus was that the opt-in approach would be ok. That can be landed after 16.x is released, likely too late for a SemVer major version as it's unlikely that we can reach consensus. @mcollina should this still stay on the TSC agenda or is that the feedback you were looking for. |
removed! |
So as I understand 'opt-in' means v16 will allow global configuration of the |
Yes that is correct |
I think it would be great to have a way for the user to configure the behavior via environment-variables until this PR lands. |
I'm +1 for this landing as the default in Node v17. |
Of course. I have loosely been monitoring the other PR. But it might take me some days until I can have a look at it, understand what exactly the other PR did, and adjust to the changes. |
@treysis as a heads up, Node.js v17.x semver cutoff is getting close (around September 19th I think, the release date is scheduled for October 19th). If you were able to rebase your branch on top on |
@aduh85 Thx. I was about to dive into it again already the past days but didn't find the time/motivation yet. The upcoming deadline might be just what is needed :) |
Before I submit a new PR...where is the |
Switch the default from false (reorder the result so that IPv4 addresses come before IPv6 addresses) to true (return them exactly as the resolver sent them to us.) Fixes: nodejs#31566 Refs: nodejs#6307 Refs: nodejs#20710 Reissue of nodejs#31567 Reissue of nodejs#37681 Reissue of nodejs#37931
Switch the default from false (reorder the result so that IPv4 addresses come before IPv6 addresses) to true (return them exactly as the resolver sent them to us.) Fixes: nodejs#31566 Refs: nodejs#6307 Refs: nodejs#20710 Reissue of nodejs#31567 Reissue of nodejs#37681 Reissue of nodejs#37931
Switch the default from false (reorder the result so that IPv4 addresses come before IPv6 addresses) to true (return them exactly as the resolver sent them to us.) Fixes: nodejs#31566 Refs: nodejs#6307 Refs: nodejs#20710 Reissue of nodejs#31567 Reissue of nodejs#37681 Reissue of nodejs#37931
Switch the default from false (reorder the result so that IPv4 addresses come before IPv6 addresses) to true (return them exactly as the resolver sent them to us.) Fixes: nodejs#31566 Refs: nodejs#6307 Refs: nodejs#20710 Refs: nodejs#38099 Reissue of nodejs#31567 Reissue of nodejs#37681 Reissue of nodejs#37931
Switch the default from false (reorder the result so that IPv4 addresses come before IPv6 addresses) to true (return them exactly as the resolver sent them to us.) Fixes: #31566 Refs: #6307 Refs: #20710 Refs: #38099 Reissue of #31567 Reissue of #37681 Reissue of #37931 PR-URL: #39987 Refs: #6307 Refs: #20710 Refs: #38099 Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com> Reviewed-By: Matteo Collina <matteo.collina@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Michael Dawson <midawson@redhat.com>
Superseded by 1b2749e. |
Switch the default from false (reorder the result so that IPv4 addresses come before IPv6 addresses) to true (return them exactly as the resolver sent them to us.)
After many tests failed due to differently resolving
localhost
to::1
or127.0.0.1
and after it became impossible to keep track of what worked and what didn't in the CI jobs during running the test suite, this is a fresh start from what we learned in, and effectively a reissue of #37681.Fixes: #31566
Refs: #6307
Refs: #20710
Reissue of #31567