Node.js debugging - cannot attach debugger to subprocesses

[wywywywy]

Basically I am having trouble attaching the VS Code debugger to the subprocesses I have in my Node app. It only attaches to the master process.

I have already read and followed the official documentation here - https://code.visualstudio.com/docs/nodejs/nodejs-debugging#_automatically-attach-debugger-to-nodejs-subprocesses

My Node code uses the cluster module to fork a few worker. Pretty standard really and it follows the cluster example code.

if (cluster.isMaster) {
	console.log(`Master process ${process.pid} starting...`);
	for (let i = 0; i < numCPUs && i < maxNumThreads; i++) {
		cluster.fork(); // Create a worker
	}
} else {
	startClusterWorkers();
}

function startClusterWorkers() {
	// xxxxxx
}

And my launch.json already has autoAttachChildProcesses defined.

{
	"version": "0.2.0",
	"configurations": [
		{
			"type": "node",
			"request": "launch",
			"name": "Launch app.js",
			"program": "${workspaceFolder}\\app.js",
			"restart": true,
			"autoAttachChildProcesses": true,
			// "console": "integratedTerminal"
		},
		{
			"type": "node",
			"request": "launch",
			"name": "Launch test.js",
			"program": "${workspaceFolder}\\test.js"
		}
	]
}

But the child processes don't show up in the call stack or the floating debugger control, unlike the official documentation. The debugger console doesn't say anything about the child processes either. And of course breakpoints set in the child process (the startClusterWorkers() function above) don't work.

Debugging with inspector protocol because Node.js v8.9.1 was detected.
node --inspect-brk=31761 app.js 
Debugger listening on ws://127.0.0.1:31761/54919117-38ae-4455-82c6-e21ade314bdd
Master process 10096 starting...

Any ideas what I am doing wrong please?

VS Code ver 1.20.1 & 1.21.0 (latest)
Node ver 8.9.1

[weinand]

What version of the node-debug extension are you using?
Go to the Extension viewlet and run the "Show Installed Extensions" query.

Why am I asking this?
In recent versions of VS Code we have removed the message "Debugging with inspector protocol because Node.js v8.9.1 was detected."
Since you are still seeing the message I am speculating that you have installed an older version of the node debugger that does not have the auto-attach feature.

[wywywywy]

It's built in isn't it?

Node Debug 1.21.2
Node Debug (legacy) 1.21.8

[weinand]

@wywywywy yes, node-debug is builtin but you can override it if you have another version in your extensions folder.

And you are not seeing any Node Debug extensions under "Show Installed Extensions"?

[wywywywy]

That's right. Only in show built-in.

By the way I have tested this on 2 PCs (both Win 10) and both behave the same.

[weinand]

And you are seeing the debug console message "Debugging with inspector protocol because Node.js v8.9.1 was detected." on both PCs?

[weinand]

Could you try the latest Insiders?

[wywywywy]

Just tried it with the Insiders. Same.

node --inspect-brk=15122 test.js
Debugger listening on ws://127.0.0.1:15122/55639fc6-8cea-4e49-97ab-1a81ecafbc2e
Master 29496 is running

[weinand]

Do you actually see the node.js child processes in Windows Task Manager?

[weinand]

Could you try this code (which works fine for me on all platforms):

const cluster = require('cluster');
const http = require('http');
const numCPUs = require('os').cpus().length;

if (cluster.isMaster) {
	console.log(`Master ${process.pid} is running`);

	// Fork workers.
	for (let i = 0; i < numCPUs; i++) {
		cluster.fork();
	}

	cluster.on('exit', (worker, code, signal) => {
		console.log(`worker ${worker.process.pid} died`);
	});
} else {
	// Workers can share any TCP connection
	// In this case it is an HTTP server
	http.createServer((req, res) => {
		res.writeHead(200);
		res.end('hello world\n');
	}).listen(8000);

	console.log(`Worker ${process.pid} started`);
}

[wywywywy]

Yes this is already the code I am using, from your test plan.

And this is the launch.json for the test plan code.

	{
		"type": "node",
		"request": "launch",
		"name": "Launch test.js",
		"program": "${workspaceFolder}\\test.js",
		"autoAttachChildProcesses": true,
	}

[weinand]

And do you see the node.js child processes in Windows Task Manager?

[wywywywy]

Yes that is correct.

Also if I just run the test.js in console without debug I do see the "worker xxx started" messages, so the code runs fine. I just don't understand why the debugger won't attach.

[weinand]

Please show screenshots!
E.g. the debug console output and the CALL STACK viewer.

It is very difficult for me to understand what is going on without having the full picture.
For the first time you mentioned the "worker xxx started" message!
You never mentioned that before.
The debug console output you showed always stopped at the "Master 29496 is running"...

[weinand]

Could you please configure your Task Manager to show the command line arguments (see http://www.winhelponline.com/blog/configure-task-manager-to-display-full-path-of-running-processes/) and add a screenshot of the child processes and their arguments.

This is what I am seeing:

[wywywywy]

Debugging

Running in terminal without debug

Task Manager

Sooooo it looks like --inspect-brk is looking at the wrong port? Is that right?

[weinand]

No, if you compare the node.js arguments in our screenshot, there is no difference. But you have way more child processes. Please try to limit the number to 4 and see whether this makes a difference.

[wywywywy]

Just tried and max 4 has not made a difference.

[weinand]

Thanks for trying!
For the 4 processes case could you please post a screenshot that shows all child processes of VS Code (like mine from above). I'm interested to see the "wmic processes ..."

[wywywywy]

So this morning I tried it on my office PC, and without any change to the code, it started working! The only change is a reboot.

I agree that it's gotta be something to do with "wmic process get CommandLine,ParentProcessId,ProcessId", which I will try on my other PC when I get home.

My hunch is that perhaps another program has "--debug-port" in its command line. But I will be able to confirm later today.

I think that if these lines below were changed to have the word "node" in it, or even better the filename of the JS file being executed, it will make the process more reliable.

const DEBUG_PORT_PATTERN = /\s--(inspect|debug)-port=(\d+)/;
const DEBUG_FLAGS_PATTERN = /\s--(inspect|debug)(-brk)?(=(\d+))?/;

[weinand]

In addition I've tried to understand what is going on in your first screenshot of #45320 (comment).

There is an active debug session but no stack trace and no indication that the program is stopped. In addition I see an error squiggly for argument "signal".

When I do the same on Windows I see this:

Are you able to stop on the breakpoint on the line with the "cluster.fork()"?

[weinand]

I'm glad that you found the issue. I should have suggested a reboot earlier, but since I'm a macOS user, a reboot never comes to my mind...

We do not match on "node" because we want to allow debugging of node-like programs that do not use the word "node" in their executable name.
In addition trying to attach to false positives that happen to use the same pattern should do no harm because all debuggers are separate processes that just time-out if they cannot connect to a debuggee successfully.

I'm currently looking into a replacement for the 'wmic'. May be that helps to avoid the problem you had.

[wywywywy]

Wait please don't close this yet! I am still having the same problem with one of the PCs.

I have of course rebooted. And I check wmic process get CommandLine,ParentProcessId,ProcessId and everything looks ok, disproving my previous theory.

To answer your question, when stopped at a breakpoint, it does show the call stack and everything. When I press F5 the call stack and everything else disappears.

[wywywywy]

You'll be pleased to know that I've found the real reason!!!! 💃 🍡 🗡

	exec(CMD, { maxBuffer: 1000 * 1024 }, (err, stdout, stderr) => {

The buffer is set to 1000*1024, but my wmic output is much bigger than that, mainly consumed by blank spaces (services I guess?) and Chrome and Firefox. Hence the code wasn't able to find the node processes.

My suggestions as a fix -

1. Make the maxBuffer bigger

2. Or more optimally, change the command to wmic process get CommandLine,ParentProcessId,ProcessId|find "--" (or similarly use findstr with regex) so that it skips the junk which will improve the performance as well.

Is this reasonable?

[weinand]

Great find! Thanks a lot for your continued investigation.

This probably explains why a reboot helped: after the reboot the number of processes was small enough to fit in the 1 MB buffer.

Yes, the output of wmic is horrible: if there is a single long entry all other entries are padded with spaces to the same width.

Yes, your suggestions make sense and as I said previously are in line with my plan to replace 'wmic' by using a native library on Windows.

[weinand]

As a first step I've eliminated the fixed size buffer. Please give it a try in the next Insiders build.

[wywywywy]

Nice that library sounds just perfect for the job!

And yes sure I will give it a try in a couple of days and report back.

Many thanks for your help.

[wywywywy]

I am currently on the latest Insiders build, but it does not seem to work. I am not familiar with Insiders, do I need to update Node Debug manually/separately? The built-in Node Debug (Legacy) is version 1.22.1.

Version 1.22.0-insider
Commit 5450fdc79778b5a76a27ea7e5fd68e9ebbe44107
Date 2018-03-12T05:15:51.483Z
Shell 1.7.9
Renderer 58.0.3029.110
Node 7.9.0
Architecture x64

[weinand]

The fix will be in node-debug 1.22.2 which will be part of tomorrow's Insiders.
(I forgot to include it in yesterday's build).

There is nothing special about the Insiders build. No need to update node-debug explicitly.

[wywywywy]

Hello. I'm here to report that it works great 🔫😁🔫

Thank you very much for your help.

Usually how long does it take for a feature in Insiders to go into the main branch?

[weinand]

@wywywywy thanks for helping investigating this!
I'll add the Verified label for this issue.

Insiders becomes Stable at the end of the month (or beginning of the next month). You can find the schedule in our Iteration Plan item: #45632.