http2 benchmark seems broken

[aduh95]

• Test: http2/compat.js http2/simple.js
• Platform: benchmark-node-micro-benchmarks
• Console Output:

"old", "http2/compat.js", "duration=5 benchmarker='test-double-http2' clients=2 streams=100 requests=1000", 2024, 5.239167185
/w/bnch-comp/node/benchmark/_test-double-benchmarker.js:49
    client.on('error', (e) => { throw e; });
                                ^

Error: connect EADDRNOTAVAIL 127.0.0.1:12346 - Local (127.0.0.1:0)
    at internalConnect (node:net:911:16)
    at defaultTriggerAsyncIdScope (node:internal/async_hooks:430:12)
    at node:net:1002:9
    at processTicksAndRejections (node:internal/process/task_queues:76:11) {
  errno: -99,
  code: 'EADDRNOTAVAIL',
  syscall: 'connect',
  address: '127.0.0.1',
  port: 12346
}
Error: test-double-http2 failed with 1.
    at ChildProcess.<anonymous> (/w/bnch-comp/node/benchmark/_http-benchmarkers.js:242:16)
    at Object.onceWrapper (node:events:486:26)
    at ChildProcess.emit (node:events:379:20)
    at maybeClose (node:internal/child_process:1063:16)
    at Socket.<anonymous> (node:internal/child_process:449:11)
    at Socket.emit (node:events:379:20)
    at Pipe.<anonymous> (node:net:667:12)

• Build Links:
  • https://ci.nodejs.org/view/Node.js%20benchmark/job/benchmark-node-micro-benchmarks/852/
  • https://ci.nodejs.org/view/Node.js%20benchmark/job/benchmark-node-micro-benchmarks/856/
  • https://ci.nodejs.org/view/Node.js%20benchmark/job/benchmark-node-micro-benchmarks/859/
  • https://ci.nodejs.org/view/Node.js%20benchmark/job/benchmark-node-micro-benchmarks/862/

[Trott]

Not having this problem locally. I wonder if what's going on is that port 12346 is used by the test suite and the server is running tests at the same time as the http2 benchmark. This would seem to be borne out by the fact that it fails with the issue at different times in the test suite. Another possibility is that the benchmark isn't waiting for the previous server to fully close before opening the next one.

[Trott]

I was able to replicate this locally. It just took much much longer. And I wasn't doing anything else on the computer, so it's something within the benchmark itself apparently and not an interaction with a test suite. Maybe _http-benchmarkers.js closes the server somewhere but doesn't wait for a callback.

[Trott]

I think I see the bug in the benchmarks themselves. They call close() but don't wait for the callback, so the next benchmark can start too soon. I think the problem took longer to happen locally for me because it is probably more likely to happen on a fast machine than a slow one and the CI machine is probably very fast.

I'll open a PR to fix this by using port 0 for the benchmarks so that we fix this issue without shutdown time becoming part of the benchmark measurement.

[Trott]

Proposed fix is in #36871