investigate flaky inspector/test-inspector on Windows

[Trott]

Fails intermittently on Windows 2012r2 / VS 2015 on CI.

https://ci.nodejs.org/job/node-test-binary-windows/4057/RUN_SUBSET=1,VS_VERSION=vs2015,label=win2012r2/console:

not ok 296 inspector/test-inspector
# Error: read ECONNRESET
#     at exports._errnoException (util.js:1026:11)
#     at TCP.onread (net.js:572:26)
# [err] Debugger listening on port 9229.
# [err] Warning: This is an experimental feature and could change at any time.
# [err] To start debugging, open the following URL in Chrome:
# [err]     chrome-devtools://devtools/remote/serve_file/@60cd6e859b9f557d2312f5bf532f6aec5f284980/inspector.html?experiments=true&v8only=true&ws=localhost:9229/node
# [err] 
# [test] Verifying debugger stops on start (--debug-brk option)
# [test] Setting a breakpoint and verifying it is hit
# [out] A message 5
# [out] 
# [test] Verify we can read current application state
# [test] Verify node waits for the frontend to disconnect
# [out] Outputed message #1
# [out] 
# [err] Debugger attached.
# [err] Waiting for the debugger to disconnect...
# [err] 
# [test] Connection terminated
  ---
  duration_ms: 0.536

/cc @eugeneo

[mcollina]

This might be related: #8155.

[Trott]

Alas, #8155 is fixed but we're still seeing this.

Any progress with https://github.com/eugeneo/node/commit/3d3ec6fd960b52e9ae920c2c7b387c0c37bf398d? (Or did that already land in some format?)

Here's a failure from today which has error output that looks slightly different than previous failures. Not sure if Jenkins changed or if the inspector code did or what....

https://ci.nodejs.org/job/node-test-binary-windows/4783/RUN_SUBSET=2,VS_VERSION=vs2015,label=win2008r2/console

not ok 302 inspector/test-inspector # TODO : Fix flaky test
  ---
  duration_ms: 0.522
  severity: flaky
  stack: |-
    [err] Debugger listening on port 9229.
    [err] Warning: This is an experimental feature and could change at any time.
    [err] To start debugging, open the following URL in Chrome:
    [err]     chrome-devtools://devtools/bundled/inspector.html?experiments=true&v8only=true&ws=127.0.0.1:9229/63a05fcc-03d6-43f3-bf70-550e1e4101ce
    [err] 
    [test] Verifying debugger stops on start (--debug-brk option)
    [test] Setting a breakpoint and verifying it is hit
    [out] A message 5
    [out] 
    [test] Verify we can read current application state
    [test] Verify sending and receiving UTF8 characters
    [out] טֶ字и
    [out] 
    [test] Verify node waits for the frontend to disconnect
    [out] Outputed message #1
    [out] 
    [err] Debugger attached.
    [err] Waiting for the debugger to disconnect...
    [err] 
    [test] Connection terminated
    Error: read ECONNRESET
        at exports._errnoException (util.js:1022:11)
        at TCP.onread (net.js:572:26)
  ...

[eugeneo]

I am trying some fixes from time to time, no actual progress yet...

[Trott]

The "ECONNRESET showing up only on Windows" has certainly happened elsewhere, such as #5386. /cc @gibfahn in case they have any light to shed

Maybe someone in @nodejs/platform-windows knows something about why that is and what the appropriate way of dealing with it is. (Swallow ECONNRESET on Windows only?)

@nodejs/testing

[gibfahn]

@Trott I think the problem is that no-one ever got to the bottom of why the ECONNRESET actually happens. Unless we know why it's happening, I don't think we should just ignore what could well be a legitimate error.

Maybe if we could work out the simplest reproducible test case, we could have a test that specifically checks for this, mark that one as flaky, and then swallow ECONNRESET on all the other tests (e.g. this one).

[Trott]

@Trott I think the problem is that no-one ever got to the bottom of why the ECONNRESET actually happens.

Hmmm...someone who cares about tests and understands Node.js tests...and who understands Windows and Node.js Windows quirks... Maybe @joaocgreis would have some ideas as to how we might trigger the ECONNRESET-on-Windows issue reliably?

[Trott]

I ran the stress test 100 times in succession and it failed 93 times. Yikes! https://ci.nodejs.org/job/node-stress-single-test/1063/nodes=win10/console

[Trott]

Changing socket.end() to socket.destroy() seems to fix the flakiness on Windows:

https://ci.nodejs.org/job/node-stress-single-test/1066/nodes=win10/console

Any reason not to go with something like that?

[Trott]

PR to fix (if the destroy() solution is OK): #9727