Need to escape ASCII control characters

[raphlinus]

Serializing control characters such as "\x01" results in these characters being passed through, ie "\"\x01\"" rather than the correct "\"\\u0001\"". The JSON spec is quite clear that these characters must be escaped, and at least one JSON implementation (Swift's) rejects the unescaped variant as invalid JSON.

I'm happy to provide a patch, not sure if you prefer that or to write your own.

[dtolnay]

A PR would be appreciated!

[raphlinus]

I have a minimal patch for this, but it looks like #58 is a completely different approach to fixing it. If you still want the PR to escape_bytes lemme know and I'm happy to upload it.

[dtolnay]

Thanks but based on the discussion in #58 we plan to remove escape_bytes entirely (see #60). We are waiting to hear from @erickt regarding #58.

[raphlinus]

So, looking over the discussion, I have some concerns. As a user of serde-json, I would have a very negative reaction to anything that slowed down serialization. "You should use msgpack instead of json" is already one of the major pieces of feedback I get - I depend on serializing large blocks of text performantly. I'm also not a fan of escaping things not required by the spec. To me, "other json encoders do it" is not a good enough reason. Lastly, I'm not a fan of adding complexity to deal with unicode when it's not required.

So I uploaded #65, which is what I'd personally like to see. I'm also in favor of removing escape_bytes, as I agree that it doesn't make sense (it's impossible to safely serialize binary to json), but I didn't put that in my PR because it kinda seems like a separate issue. I'd be happy to update my PR if there's agreement.

Totally up to @erickt what he prefers, but I figured I would give him the option and present my reasoning behind it.

[PaulGrandperrin]

After my PR would have merged, I was about to propose to add an option to the serializer.
This option would have been about escaping or not non-ascii characters. It can be useful to generate ascii-compatible Jsons.
In Python, this option is called ensure_ascii.
To address @raphlinus needs, we could make this option tristate like this:

enum Escaping {
    Minimal, // according to ECMA-404
    AllControlCharacters, // plus DEL and C1 characters
    AllNonAscii, // everything above 127 (C1 characters are above 127)
}