Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Debugger breakpoints using incorrect URL for a remote debugging session and therefore failing to pause the process #103

Closed
treffynnon opened this issue May 25, 2018 · 42 comments

Comments

@treffynnon
Copy link

treffynnon commented May 25, 2018

Symptoms

This is related to the closed ticket #99, which describes the symptoms of this very problem.

You set a breakpoint, but the process isn't paused when that breakpoint is encountered and carries merrily on.

Environment

This is a remote debugging session between a Windows client (running Chrome devtools) and a remote Linux server (running node with --inspect=0.0.0.0).

What's wrong

By watching the websocket communication with Wireshark it is apparent that the Chrome Dev tools are using an incorrect filepath/URL when calling Debugger.setBreakpointByUrl.

What I am seeing

Request

{
  "id": 17,
  "method": "Debugger.setBreakpointByUrl",
  "params": {
    "lineNumber": 67,
    "url": "EFS\\DEV-efsbastion\\username\\application\\projects\\service\\module\\index.js",
    "columnNumber": 18,
    "condition": ""
  }
}

Response

{
  "id": 17,
  "result": {
    "breakpointId": "EFS\\DEV-efsbastion\\username\\application\\projects\\service\\module\\index.js:67:18",
    "locations": []
  }
}

Note the empty locations array. I am no node debugger expert, but it feels like this locations array should be checked to ensure it is not empty after setting a new breakpoint marker before then showing the marker as active in the UI.

What I should be seeing

Request

{
  "id": 17,
  "method": "Debugger.setBreakpointByUrl",
  "params": {
    "lineNumber": 67,
    "url": "/EFS/DEV-efsbastion/username/application/projects/service/module/index.js",
    "columnNumber": 18,
    "condition": ""
  }
}

Response

{
  "id": 17,
  "result": {
    "breakpointId": "/EFS/DEV-efsbastion/username/application/projects/service/module/index.js:67:18",
    "locations": [{
      "scriptId":"104",
      "lineNumber":67,
      "columnNumber":18
    }]
  }
}

Note the Linux path separators instead of Windows and the initial root slash on the path.

But isn't this a node bug?

No, it is not from what I can tell because of the following two discoveries.

PHPStorm

PHPStorm/WebStorm debugger continues to work just fine using it's URL regexes

Request

{
  "id": 17,
  "method": "Debugger.setBreakpointByUrl",
  "params": {
    "lineNumber": 67,
    "urlRegex": "[/\\\\][iI][nN][dD][eE][xX]\\.[jJ][sS]([;?#!].*)?$"
  }
}

Response

{
  "id": 17,
  "result": {
    "breakpointId": "/[/\\\\][iI][nN][dD][eE][xX]\\.[jJ][sS]([;?#!].*)?$/:67:18",
    "locations": [{
      "scriptId":"104",
      "lineNumber":67,
      "columnNumber":18
    }]
  }
}

Manually recreating the request successfully

If I manually set the breakpoint myself I see a success response back over the websocket. For example consider the following code run from the Chrome console:

let id, key, socket

id = 1000
key = '<debugger-key-goes-here>'

const connect = userSuppliedKey => {
	socket = new WebSocket(`ws://<ip-address-goes-here>:9229/${userSuppliedKey || key}`)
	socket.addEventListener('message', e => {
		const d = JSON.parse(e.data)
		if (d) {
			if (d.method === 'Debugger.scriptParsed') return console.log('scriptParsed', JSON.stringify({ scriptId: d.params.scriptId, url: d.params.url }))
			if (d.method === 'Debugger.breakpointResolved') return console.log('breakpointResolved', JSON.stringify(Object.assign({ breakpointId: d.params.breakpointId }, d.params.location)))
		}
		console.info(`${e.data}`)
	})
}
const send = (method, x) => socket.send(JSON.stringify({ id: ++id, method: `Debugger.${method}`, params: x }))
const _getPossibleBps = (start, end) => send('getPossibleBreakpoints', { start, end })
const getPossibleBps = (scriptId, start, end) => _getPossibleBps({ scriptId, lineNumber: start }, { scriptId, lineNumber: end })
const setBp = (lineNumber, url) => send('setBreakpointByUrl', { lineNumber, url, })
const removeBp = (lineNumber, url, columnNumber) => send('removeBreakpoint', { breakpointId: `${url}:${lineNumber}:${columnNumber || 0}` })
const enable = () => send('enable')

Which gives us this when run in the Chrome console

connect('<debugger-key-goes-here>')
// '<debugger-key-goes-here>'

enable()
// ...
// scriptParsed {"scriptId":"104","url":"/EFS/DEV-efsbastion/username/application/projects/service/module/index.js"}
// ...

getPossibleBps('104', 9, 15)
// {"id":1016,"result":{"locations":[{"scriptId":"104","lineNumber":9,"columnNumber":15},{"scriptId":"104","lineNumber":11,"columnNumber":15},{"scriptId":"104","lineNumber":12,"columnNumber":16},{"scriptId":"104","lineNumber":14,"columnNumber":4},{"scriptId":"104","lineNumber":15,"columnNumber":0},{"scriptId":"104","lineNumber":17,"columnNumber":4},{"scriptId":"104","lineNumber":18,"columnNumber":14}]}}

setBp(11, '/EFS/DEV-efsbastion/username/application/projects/service/module/index.js')
// {"id":1024,"result":{"breakpointId":"/EFS/DEV-efsbastion/username/application/projects/service/module/index.js:11:0","locations":[{"scriptId":"104","lineNumber":11,"columnNumber":15}]}}

removeBp(11, '/EFS/DEV-efsbastion/username/application/projects/service/module/index.js')
// {"id":1019,"result":{}}

Possible introduction of the issue

https://chromium.googlesource.com/chromium/src/+/2140ffeadf3e081e2e0821fc8e1ee040373b205d%5E%21/third_party/WebKit/Source/devtools/front_end/sdk/DebuggerModel.js

in combination with https://chromium.googlesource.com/chromium/src/+blame/master/third_party/blink/renderer/devtools/front_end/common/ParsedURL.js#103

@treffynnon treffynnon changed the title Debugger breakpoints using incorrect URL for a remote debugging session Debugger breakpoints using incorrect URL for a remote debugging session and therefore failing to pause the process May 25, 2018
@treffynnon
Copy link
Author

treffynnon commented May 25, 2018

@alexkozy
Copy link

Thanks for great report!

Inside V8 on inspector backend side we try to match breakpoint url and script url. In this case we send incorrect url from frontend side and when we receive new parsed script on backend side we do not set breakpoint properly.

I will do following two points next week:

  • check why we send incorrect url from frontend side,
  • consider some kind of breakpoint url normalization on backend side to make our protocol a little more flexible.

I will update this issue as soon as I get some results.

(PhpStorm actually uses great regexp)

@mathieulj
Copy link

Just to corroborate that this is due to a change on the chrome side and provide a workaround. We experienced this recently running windows workstations with a linux development VM. We had 3 versions of chrome(64, 66, 68) connecting one at a time to a debug session and only 64 was working normally. This was true for node 6, 8 and 10.

As a workaround for those who need to debug, we found that using the id and manually visiting the link chrome-devtools://devtools/bundled/inspector.html?experiments=true&v8only=true&ws=${ip}:${port}/${id} works as expected even in the newer chrome versions. If, for example, you get the message Debugger listening on ws://0.0.0.0:3020/4dacc23a-f296-4b6d-bb47-18af21dfef67 during inspect start, the URL would be chrome-devtools://devtools/bundled/inspector.html?experiments=true&v8only=true&ws=${ip}:3020/4dacc23a-f296-4b6d-bb47-18af21dfef67 where ip is the ip of the remote machine. We also found that the plugin nodejs-v8-inspector-manag(https://chrome.google.com/webstore/detail/nodejs-v8-inspector-manag/gnhhdgbaldcilmgcpfddgdbkhjohddkj) seems to still work.

@brucejo75
Copy link

brucejo75 commented Jun 4, 2018

@tonymet
Copy link

tonymet commented Jun 17, 2018

I'm experiencing the lack-of-breakpoints with Windows 10 + WSL (aka Ubuntu Bash for Windows) + Node 8.11.3

@mathieulj 's workaround by manually building the chrome-devtools:// URL works for me.

Thanks for the help guys this was driving me bonkers for a while :D

Node v8.11.3
Windows 10 PRo 1803
Version 67.0.3396.87 (Official Build) (64-bit)

@tonymet
Copy link

tonymet commented Jun 17, 2018

Also mentioned in the chromium bug, Chrome Extensions "Inspector Manager" is a simpler workaround for now

https://chrome.google.com/webstore/detail/nodejs-v8-inspector-manag/gnhhdgbaldcilmgcpfddgdbkhjohddkj

@treffynnon
Copy link
Author

@ak239 is there any progress being made with this issue?

Unfortunately, [Web/PHP]Storm has it's own issues with simply choosing the first file it sees with the name the breakpoint is set against. In the case of files named index.js this is an issue as so many modules use this filename that the breakpoint often ends up being set against the wrong file.

@alexkozy
Copy link

alexkozy commented Aug 1, 2018

I landed following CL recently. It should fix provisional breakpoints in Node.js targets. At least it helps ndb.
I have another CL to land that fixes breakpoints in experimental Node.js modules: CL. I am going to land it soon.
It would be nice if someone could verify that issue is fixed in latest Canary.

@treffynnon
Copy link
Author

treffynnon commented Aug 1, 2018

I downloaded the canary with version: Version 70.0.3508.0 (Official Build) canary (64-bit) and it appears to still have the same problem if that helps.

@mathieulj
Copy link

It seems to have gotten worse with the release of chrome 69. The workarounds I posted above no longer work with chrome 69.0.3497.81(stable) and with chrome 71.0.3548.0 (canary). Workstations still running 68.0.3440.106 do work with both workarounds.

@mathieulj
Copy link

For those with a Webstorm licence, version 2018.2.3 seems to break correctly on breakpoints.

@treffynnon
Copy link
Author

treffynnon commented Sep 13, 2018

For those without a WebStorm licence you can use the Early Access Programme (EAP) builds for free with the following notice:

This is an early access version of the product. You expressly acknowledge that this version of the product may not be reliable, may not work as intended and may contain errors. Any use of the EAP product is at your own risk.

-- JetBrains

NOTE: I am not affiliated with JetBrains

@alexkozy
Copy link

alexkozy commented Sep 14, 2018

The root of this issue is using platform-specific file paths in our protocol as was mentioned before. I landed V8 patch and close to merging Node patch that will migrate Node starting Node 11 to file urls: nodejs/node#22251.
At the same time I am working on better solution for remote debugging as part of ndb project. I will update this issue as soon as it is ready for Node 8 and Node 10.

@alexkozy
Copy link

fileURL patches were merged to Node 10. So I expect this issue to be fixed in next Node 10 release, I will update this bug when release happens.

@GummyDonut
Copy link

GummyDonut commented Sep 26, 2018

@ak239 Will this be updated for node 8 at well? As I just recently encountered the bug after Chrome updated XD

@june07
Copy link

june07 commented Oct 5, 2018

The most recent version of NiM actually does work as a fix to this. Different DevTools versions can be selected along with a custom version as the video shows. https://youtu.be/hjf6esnqOJQ

So in essence when using NiM it doesn't matter which version of Node or Chrome you use.

https://june07.com/nim

@sdd
Copy link

sdd commented Oct 12, 2018

@june07 ?I'm using NiM 1.1.0, Node 10.12 (remote, linux), Chrome (69, windows) and I still get this problem

@june07
Copy link

june07 commented Oct 12, 2018 via email

@sdd
Copy link

sdd commented Oct 12, 2018

@june07 thanks for the quick response.

I'v tried changing NiM settings but it is not clear what to use for the value of custom devtools.

I have tried a few values but none work, and now if I try to turn off "use alternate devtools", it does not open an inspector at all.

@alexkozy
Copy link

@sdd, could you please try DevTools frontend in Canary with Node 10.12?

@smeijer
Copy link

smeijer commented Oct 12, 2018

For those with a Webstorm licence, version 2018.2.3 seems to break correctly on breakpoints.

Strange, just upgraded from 2018.1 to 2018.2, but doesn't change anything here. Also tried the 2018.3, but still not hitting any breakpoint.

Using NiM turned out to work, but it doesn't work as sweet as the Webstorm debugger.

@june07
Copy link

june07 commented Oct 12, 2018 via email

@smeijer
Copy link

smeijer commented Oct 12, 2018

Okay, let me rephrase that. WebStorm does kind of work. Some breakpoints can be hit, other ones not at all. And others don't seem to be hit, but my frontend does block. So node is pausing and waiting for the debugger, but I don't have control over it.

So I basically have 3 types of breakpoints.

  1. The ones that function as expected
  2. Those that cannot be hit
  3. Those that are hit, but not given control over?

@brucejo75
Copy link

brucejo75 commented Oct 13, 2018

@ak239, Should I assume that Node 8 will not be supported?

I've just tried NiM 1.1.0 with devtools set to: https://chrome-devtools-frontend.appspot.com/serve_file/@a10b9cedb40738cb152f8148ddab4891df876959/inspector.html

on Node 8.11.3 and Version 69.0.3497.100
(Official Build) (64-bit)
Result: breakpoints are not hit.

@alexkozy
Copy link

I definitely would like to get if fixed, but I need your help.
@brucejo75, could you provide more details about your setup, I assume that you use Windows and can reproduce this bug with any node script with node 8? (I tried on mac and it works).
Could you confirm that with latest node 10.12 it works to you? And that it does not work with dedicated frontend from latest Google Chrome Canary?

@june07
Copy link

june07 commented Oct 14, 2018

@brucejo75

https://youtu.be/ZfpZ3wGvEm4

I literally just setup my environment to match yours and confirmed again that things are working with NiM. Maybe you'll see some changes between our environments from the video?

@alexkozy
Copy link

Could we focus on fixing DevTools and DevTools protocol in this issue please? A lot of workarounds were already mentioned and I believe that now we can focus on finding solution for root issue.

@brucejo75
Copy link

@june07,

I had the devtools set correctly but failed to turn them on, D'oh.

When I use it now, I notice that the debug console uses v66 of chrome and all works as expected.

@brucejo75
Copy link

@ak239,

Looks like canary (v72) fixes the problem! (at least for me).

Environment

Windows 10: Version 1803 (OS Build 17134.345)
node: 8.11.3 & 10.12 (using WSL)
chrome: 69.0.3497.100 & 72.0.3580.0 (Official Build) canary (64-bit)

Note: I am running WSL on windows, which is the Ubuntu linux subsystem.

node script index.js:

1 console.log('hello World');
2
3 console.log('Hit 1?');
4 debugger;
5 console.log('Hit 2?');

Repro steps

  1. node --inspect-brk index.js
  2. in chrome browser go to chrome://inspect, click on Open dedicated DevTools for Node
  3. Set breakpoint on Line 3.
  4. in dedicated DevTools, resume script execution (e.g. hit F8)

Fails with chrome 69, tested with node 8.11.3 & node 10.12

Result: debugger stops on line 4, missing breakpoint at line 3.

Workaround with NIM tools works because essentially it uses v66 of the devTools.

Verified fixed with chrome 72, tested with node 8.11.3 & node 10.12

Result: debugger stops on breakpoint at line 3
Yay!

@june07
Copy link

june07 commented Oct 15, 2018

@june07,

I had the devtools set correctly but failed to turn them on, D'oh.

When I use it now, I notice that the debug console uses v66 of chrome and all works as expected.

Actually updated NiM (1.2.0) to use the most up to date version of DevTools (Chrome 71) as well as adding info on how to better use the custom setting https://june07.com/blog/nim-custom-devtools-url.

@ak239 This will be my last post to this thread. Cheers.

@brucejo75
Copy link

brucejo75 commented Oct 15, 2018

Ignore?

This may be expected behavior. Before I can set a breakpoint on a required file, the file has to be loaded. Sorry for the clutter.

Entered issue

Oops! spoke too soon. devTools fails to stop on a breakpoint in a required module.
@ak239, fails with Canary :-(

Environment

Windows 10: Version 1803 (OS Build 17134.345)
node: 8.11.3 (using WSL)
chrome: 72.0.3580.0 (Official Build) canary (64-bit)

Note: I am running WSL on windows, which is the Ubuntu linux subsystem.

node script index.js:

1 let foo = require('./foo');
2 console.log('hello World');
3 foo();

node script foo.js

1 function foo() {
2   console.log('Hit 1?');
3   debugger;
4   console.log('Hit 2?');
5 }
6
7 module.exports = foo;

Repro steps

  1. node --inspect-brk index.js
  2. in chrome browser go to chrome://inspect, click on Open dedicated DevTools for Node
  3. Set breakpoint on foo.js Line 2.
  4. in dedicated devTools, resume script execution (e.g. hit F8)

Result: devTools fails to stop on the breakpoint, stops on foo.js line 3.

Workaround

  1. node --inspect-brk index.js
  2. in chrome browser go to chrome://inspect, click on Open dedicated DevTools for Node
  3. set breakpoint on index.js line 2
  4. set breakpoint on foo.js line 2.
  5. in dedicated devTools, resume script execution (e.g. hit F8)

Result: devTools will now stop on both breakpoints.

@cronvel
Copy link

cronvel commented Nov 22, 2018

On Chrome/Chromium v69, UI breakpoints (not debugger statement) doesn't work anymore since node v10.
It was working on node v8 (and I confirm it's still working when I downgrade). I believe it's a bug on the devTools side nonetheless. Also in Node v10, the devTools doesn't skip Node.js source code anymore, which is pretty annoying...

@brucejo75
Copy link

I verified that Chrome release version: Version 71.0.3578.80 (Official Build) (64-bit) now has this repaired. Maybe should be closed?

@daniele-orlando
Copy link

I confirm that Chrome 71.0.3578.98 works fine.

@louderspace
Copy link

I'm using v8 8.0.426 and am still seeing this issue with the latest Chrome and VS Code. Installing NiM did fix the problem on the mac, but not on windows (though, I'm not convinced NiM is correctly installed on windows because I'm not seeing the syntax coloring). I did verify that all of my paths are correctly specified within the v8::ScriptOrigin.

Here's an example:

Ticks = 23110 <17356> <5> messageFromDebugger : /1, payload : {"id":17,"method":"Debugger.setBreakpointByUrl","params":{"urlRegex":"[Dd]:\\dev\\apd\\<...redacted...>\.js|file:\/\/\/[Dd]:\/dev\/apd\/<...redacted...>\.js","lineNumber":430,"columnNumber":13}}

@june07
Copy link

june07 commented May 20, 2020

What version of DevTools, NiM, and Node.js are you running on Windows?

@louderspace
Copy link

louderspace commented May 20, 2020

Node.js v12.16.3
NiM 2.3.4
Google Chrome | 81.0.4044.138
VS Code 1.45.1

Note, I've embedded v8 into a desktop application directly.

Updating Node.js yesterday helped resolve one of my problems on windows, but I'm still seeing the same problem within VSCode.

In the failure case, I'm unable to step into some functions that are in other .js files.

@june07
Copy link

june07 commented May 20, 2020

Well regarding

(though, I'm not convinced NiM is correctly installed on windows because I'm not seeing the syntax coloring)

I can confirm running with the same versions as you that syntax coloring is working in both Chrome 81.0.4044.138 and Edge Version 83.0.478.28 (using both currently).

If that is still an issue you may want to double check your actual DevTools version, as in the case of NiM it's possible you have it set several different ways.:
image

Of course this same point could also very well affect your not being able to step into some functions, as that is a function of the DevTools debugger.

@louderspace
Copy link

I'm getting a new machine today and will try setting it up again from scratch. Here's what it currently looks like.

image

@alexkozy
Copy link

@louderspace If you are embedding V8 directly, your ScriptOrigin might contain properly formed file URLs like file://... - in this case, everything should work out of the box.

The solution that was implemented on the DevTools side only works for Node.js. So how do you start DevTools to debug your embedded V8?

@louderspace
Copy link

I am setting the script origins:

Added tracing shows "SetScriptPath = D:\dev\p4<redacted>\nml\nml_include.js

Should this be in the form file://dev/... ?

@louderspace
Copy link

I tried various flavors and haven't had any success:

CompileAndRun script : file:///D:/dev/p4/nml/nml_include.js

I do have NiM working on windows now and running Chrome 8.3

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests