JSON API: Missing bits and pieces

[enkore]

From #2249

Although some minor / not overly important bits and pieces are still missing, I'd have to make a survey to find them all. Thinking recreate might not do the stats or file list right (or does it?), perhaps adding a few more tests, and working a more general intro / summary into the docs and more [comprehensive] reference material.

Also, feedback if someone wants to make a test drive would be valuable, before we freeze this interface and define compatibility policies etc.

[Abogical]

Excuse me if it's a dumb question, but why not msgpack?

[ThomasWaldmann]

@Abogical json is easier for this and we do not have the high efficiency / throughput requirements here.

[enkore]

• Pro JSON: way more tooling (e.g. jq), more widely known, easier to introspect (since it's printable text)
• Contra JSON: has issues with binary contents ("Since JSON can only encode text, any string representing a file system path may miss non-text parts." [1] -- for now, might add workaround, e.g. base64 for elements that could contain binary stuff)
• Pro msgpack: no problems with binary contents
• Contra msgpack: less tooling, harder to introspect / handle
  • difficult to put in a mailing list or Github issue
  • could show a decoded form, e.g. Python repr, but that is not universal
  • also harder to document -- with JSON we can just put the literal JSON in the docs as examples, which doesn't work with msgpack
• Streaming is easier with msgpack, API-wise, and is well-specified, while JSONlines is more of a colloquial invention (though widely used)

Because possibly-binary contents are so far an exception I chose JSON for this first implementation (e.g. not-entirely-complete paths in a progress update are not important - plus, a binary path can't be really rendered as text anyway –– but they might be annoying for "borg list --json", for example).

[1] https://borgbackup.readthedocs.io/en/latest/internals/frontends.html

[ThomasWaldmann]

Guess concerning msgpack vs json, the decision was made long ago.

About binary items: we have a more specific ticket about this. #6151

Considering all else, this ticket is too unspecific.