string: implement API to access raw string data

[indygreg]

With the recent implementation of non-limited unicode APIs, we're
able to query Python's low-level state to access the raw bytes that
Python is using to store string objects.

This commit implements a safe Rust API for obtaining a view into
Python's internals and representing the raw bytes Python is using
to store strings.

Not only do we allow accessing what Python has stored internally,
but we also support coercing this data to a Cow<str>.

Closes #1776.

[davidhewitt]

Thanks, this looks useful to me! I wonder whether eventually we'd use this internally in PyO3 too to potentially avoid causing strings to generate (and cache) their UTF-8 representation.

I have a bunch of small nitty comments below.

I'd also very much like to see some more tests of PyStringData::to_string() - it would be good to cover both the success case (which looks like it's covered) and the error cases (which don't appear to be) for each of the three character widths.

[davidhewitt]

As it doesn't look like any of these methods need to refer to 'a, I think you can just do:

Suggested change

	impl<'a> PyStringData<'a> {
	impl PyStringData<'_> {

[davidhewitt]

Also, types::string is currently not a public module, which is why CI is unhappy. I don't personally see the harm in making it public? I think I'd rather users wrote use pyo3::types::string::PyStringData over use pyo3::types:::PyStringData - the latter seems misleading.

[indygreg]

OK. I think the most recent push addresses most (all?) of your feedback.

I did disagree with you on exposing PyStringData under pyo3::types::string. If you insist on that, suggest a commit and I'll apply it.

I added tests for verifying .to_string() errors. This required using PyUnicode_FromKindAndData() to construct instances with invalid byte sequences, which Python appears to accept without validation.

As part of writing the tests, I noticed there may be a c_int / c_uint mismatch in the ffi definitions. When I implemented the FFI bindings, I attempted to copy signedness from the C headers. However, some of these things were macros and lacked type annotations. I may have used the wrong types for symbols like PyUnicode_1BYTE_KIND :/

[davidhewitt]

Nice work, this looks good to me!