sourceReference should be 0 for jupyter cells

[rchiodo]

Environment data

• DEBUGPY version: '1.0.0b5'
• PTVSD version: '4.3.2'
• OS and version: Windows 10
• Python version (& distribution if applicable, e.g. Anaconda): Python 3.7
• Using VS Code or Visual Studio: VS code

Actual behavior

Stepping into functions doesn't work. Opens bogus file.

Expected behavior

Stepping into functions defined in other cells would open the appropriate file with the cell.

Steps to reproduce:

1. Create file like so

# %% 
def baz():
   print('baz')

# %%
print('foo')
baz()

2. Run first cell in the interactive window in VS code
3. Debug second cell
4. Step into the baz call
5. It will not go to where the first cell is defined.

I'll attach logs in a second.

[rchiodo]

PTVSD working:
ptvsd-25956.log

DEBUGPY Not (on the same code, doing the same steps)
debugpy.server-35680.log
debugpy.adapter-14836.log
debugpy.pydevd.35680.log

[int19h]

It looks like it did the mapping back to .py correctly, but when reporting that, also provided a "sourceReference":

D+00004.547: /handling #33 request "stackTrace" from Client[1]/
             Client[1] <-- {
                 "seq": 71,
                 "type": "response",
                 "request_seq": 33,
                 "success": true,
                 "command": "stackTrace",
                 "body": {
                     "stackFrames": [
                         {
                             "id": 26,
                             "name": "baz",
                             "line": 3,
                             "column": 1,
                             "source": {
                                 "path": "d:\\Training\\SnakePython\\foo.py",
                                 "sourceReference": 15
                             }
                         }
                     ],
                     "totalFrames": 2
                 }
             }

This causes VSCode to request the source directly instead of trying to open the file:

D+00004.625: Client[1] --> {
                 "seq": 34,
                 "type": "request",
                 "command": "source",
                 "arguments": {
                     "sourceReference": 15,
                     "source": {
                         "path": "d:\\Training\\SnakePython\\foo.py",
                         "sourceReference": 15
                     }
                 }
             }

D+00004.625: /handling #34 request "source" from Client[1]/
             Client[1] <-- {
                 "seq": 72,
                 "type": "response",
                 "request_seq": 34,
                 "success": true,
                 "command": "source",
                 "body": {
                     "content": "def baz():\n    print('baz')\n"
                 }
             }

@fabioz, could any of the recent changes to https://github.com/microsoft/debugpy/blame/master/src/debugpy/_vendored/pydevd/pydevd_file_utils.py regressed this?

[int19h]

I think it's this bit that regressed it, in 8fc238e. The actual path mapping code in file utils does map it correctly, producing "sourceReference": 0; but then here it is replaced. If this entire conditional is removed, everything works fine.

debugpy/src/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_net_command_factory_json.py

Lines 233 to 246 in 0dd33dd

	source_reference = pydevd_file_utils.get_client_filename_source_reference(filename_in_utf8)
	if not source_reference:
	# Check if someone added a source reference to the linecache (Python attrs does this).
	if linecache.getline(original_filename, 0):
	source_reference = pydevd_file_utils.create_source_reference_for_linecache(
	original_filename, filename_in_utf8)
	if not source_reference:
	if getattr(frame.f_code, 'co_lnotab', None):
	if not os.path.exists(original_filename):
	# Create a source-reference to be used where we provide the source by decompiling the code.
	source_reference = pydevd_file_utils.create_source_reference_for_frame_id(frame_id)

[fabioz]

Humm, you're right... just removing that condition will break other things, but I guess that checking if the file exists before proceeding should work...

i.e.:

if (not source_reference) and (not os.path.exists(original_filename)):
   ...

I'll provide a pull request with the fix tomorrow (with proper tests).