Provide an alternatives to &str for other encodings and potentially malformed strings

[shadaj]

We should support UTF-16 strings an also provide a MaybeValidStr for situations where the string may not decode properly. Right now, we just panic if the string is not UTF-8 encoded.

[CBenoit]

Regarding the FIXME that is linking to this issue:

diplomat/macro/src/lib.rs

Line 87 in 2e015a3

// TODO(#57): don't just unwrap? or should we assume that the other side gives us a good value?

My take on this is that it's okay to assume the other side is giving a good value and unwrap the result of core::str::from_utf8. If the function is taking a &str we already know UTF-8 is required when generating the code for the other side (using alternative types for UTF-16 and other would follow the same logic).
For the same reason, I also think we could use from_utf8_unchecked at least in release build (using the safe version in debug build could be useful when debugging backend code).

I'm not sure about the usefulness of MaybeValidStr though. What would be the advantage over &[u8] (or other as appropriate)?

[Manishearth]

I think this is probably worth doing at a per-backend level; for example C and C++ can be asked to provide valid utf8, but JS/.NET/etc can enforce it so that we don't have crashes. This means that at the base C layer we always from_utf8_unchecked or from_utf8 + unwrap, but backends for safe languages perform additional checks if necessary. In many cases (e.g. JS) the string will need to be synthesized at the boundary anyway.

I'm not sure about the usefulness of MaybeValidStr though. What would be the advantage over &[u8] (or other as appropriate)?

Probably a cleaner generated API, but unsure.