Unhelpful parse error in decoder.py

[Smit-tay]

Bug report

Bug description:

This is the essentially useless output from attempting to parse a malformed json :

File "/usr/lib/python3.10/json/decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

In my case the json file - written by hand - looked something like this:

### A comment here
{
    "name": "value"
}

As you can see, the first line is invalid JSON, since, for reasons left to be imagined, the designers of JSON didn't like long existing UNIX standard comment indicators.

I suspect a more usefull error message when parsing might be something like :
json.decoder.JSONDecodeError: Invalid JSON "### A comment here" , Expecting value: line 1 column 1 (char 0)

This at least gives someone NOT INTERESTED in opening a debugger - i.e. a naive and casual user - a hint that the JSON is malformed - somehow.

CPython versions tested on:

3.10

Operating systems tested on:

Linux

[brianschubert]

Related: #121479

[ZeroIntensity]

This is a feature, and as far as I know, it's been rejected previously. I think there was a DPO thread somewhere.

cc @nineteendo, who had a venture with this. Maybe it's time to revisit it?

[nineteendo]

Well, yeah I would like to revisit this in January. The idea is raising a subclass of SyntaxError:

Traceback (most recent call last):
  File "<stdin>", line 1, column 1
    ### A comment here
    ^
jsonyx.JSONSyntaxError: Expecting value

But some adjustments are necessary, which is why this needs to be discussed.
Also, if you wrote a C style comment, we could even display a more helpful error message:

Traceback (most recent call last):
  File "<stdin>", line 1, column 1-18
    // A comment here
    ^^^^^^^^^^^^^^^^^
jsonyx.JSONSyntaxError: Comments are not allowed

In the meantime, you can take a look at https://pypi.org/project/jsonyx (it can parse these comments, if you opt-in).

[Smit-tay]

Thanks:

I might add - this is exactly the reason I really dislike working with python - in general.
Every time I encounter any problem, I am given a stack trace that is nearly useless, along with a final message (the root of the stack) that doesn't indicate, in any helpful message, what the issue is (i.e. doesn't require debugging) .

In this case, I know :

json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

What I don't know is the general issue - namely: the decoder encounter malformed JSON, or what the offending JSON might be. Since this message comes deep into stack trace, I didn't even realize this was a result of a file parse, until I debugged it.

Not being a python guy, I wasted a frustrating hour learning how to launch the same command in a python debugger, learning how to set breakpoints, learning how to display locals, etc, etc. just to discover that my problem was an invalid JSON file that was trying to be parsed.

Imagine how much time a non techie person is going to waste on this !

NONE - it doesn't work, this package is broke, let's try something else.

Instead of

Hmm, It looks like there some kind of problem in XXX file, where a properly formed JSON is expected, but instead it encounters "#### COMMENT string" - Oh, I recognize that, comment string; I just edited that, and added the comment, so that I knew what I was doing - how weird that a JSON parser doesn't respect decades long tradition of how to make a comment line. I guess I have to correct that !

Until almost human error messages are produced your python utility - useful as it might be - is going to be hated by all , and rightly so.

Error messages are for humans - emit them for humans !

[ZeroIntensity]

Generally, it's a bad idea to complain about a language when you're talking to the people who maintain it :)

Anyways, I don't really agree with the current message being hard to debug or recognize. I'm not sure where you got the idea that a JSONDecodeError meant a bug in your code and not an issue with the JSON itself. Better yet, it tells me exactly where the JSON is wrong. When you see line 1 column 1 (char 0), where were you looking if not the JSON?

[nineteendo]

It doesn't tell you which JSON file is wrong though, which is very important information if you're parsing many files.

Also, if one of my users shows me this error with addition of the filename, I still can't tell them what's wrong. I need to ask them to send me the file to check the line in order to tell them how they can fix it. If we included the offending line in the error message, it would make helping people on e.g. StackOverflow easier.