Allow handlers to return user-defined error types

[hawkw]

Currently, all endpoint handler functions must return a
Result<T, HttpError>, where T implements HttpResponse. This is
unfortunate, as it limits how error return values are represented in
the API. There isn't presently a mechanism for an endpoint to return
a structured error value of its own which is part of the OpenAPI spec
for that endpoint. This is discussed at length in issue #39.

This branch relaxes this requirement, and instead allows endpoint
handler functions to return any type that implements HttpResponse.
This includes Result<T, HttpError> where T: HttpResponse, but it also
includes any Result<T, E> where T implements HttpResponse and E
implements a new HttpResponseError trait, which defines how to convert
a value into an error response.

Incidentally, changing the return type of handler functions from
Result<T: HttpResponse, HttpError> to just any T: HttpResponse also
permits endpoint handler functions to be infallible and just always
return a response. This was not a primary goal of this change, but it
seems kind of nice to have anyway.

The HttpResponseError trait defines how to produce an error response
for an error value. This is implemented by dropshot's HttpError
type, but it may also be implemented by user errors. Types implementing
this trait must implement HttpResponseContent, to determine how to
generate the response body and define its schema, and they must also
implement a method HttpResponseError::status_code to provide the
status code to use for the error response. This is somewhat different
from the existing HttpCodedResponse trait, which allows successful
responses to indicate at compile time that they will always have a
particular status code, such as 201 Created. Errors are a bit different:
we would like to be able to return any number of different error status
codes, but would still like to ensure that they represent errors, in
order to correctly generate an OpenAPI document where the error schemas
are returned only for error responses (see this comment for
details). As discussed here, we ensure this by providing new
ErrorStatusCode and ClientErrorStatusCode types, which are newtypes
around http::StatusCode that only contain a 4xx or 5xx status (in the
case of ErrorStatusCode), or only contain a 4xx (in the case of
ClientErrorStatusCode). These types may be fallibly converted from an
http::StatusCode at runtime, but we also provide constants for
well-known 4xx and 5xx statuses, which can be used infallibly. The
HttpResponseError::status_code method returns an ErrorStatusCode
rather than a http::StatusCode, allowing us to ensure that error types
always have error statuses and generate a correct OpenAPI document.

Additionally, while adding ErrorStatusCodes, I've gone ahead and
changed the dropshot::HttpError type to also use it, and changed the
HttpError::for_client_error and HttpError::for_status constructors
to take a ClientErrorStatusCode. Although this is a breaking change,
it resolves a long-standing issue with these APIs: currently, they
assert that the provided status code is a 4xx internally, which is often
surprising to the user. Thus, this PR fixes #693.

Fixes #700
Fixes #39
Fixes #693
Fixes #801

This branch is a second attempt at the change originally proposed in PR
#1164, so this closes #1164. This design is substantially simpler, and
only addresses the ability for handler functions to return
user-defined types. Other changes made in #1164, such as a way to
specify a global handler for dropshot-generated errors, and adding
headers to HttpError responses, can be addressed separately. For now,
all extractors and internal errors still produce dropshot::HttpErrors.
A subsequent change will implement a mechanism for providing alternate
presentation for such errors (such as an HTML 404 page).

[hawkw]

it occurs to me that HttpResponseHeaders might want to forward an implementation of HttpResponseError when the wrapped type implements that. I'll add one.

[ahl]

thanks for doing this

[ahl]

Alternatively, we could have added to components.responses as before and then referenced that. I can see the inline approach you've taken as potentially simpler, though it does bloat up the json output...

[hawkw]

So, I'd like to put them in components.responses, too. The reason I didn't is that it might be a bit annoying to determine the name for each response schema. schemars internally disambiguates colliding schema names by turning subsequent ones into like Error2 or whatever, but (AFAICT) we only get that when we actually generate the schema and it gives us back a reference (into components.schemas). We could then try to parse that reference and get the name back out to then use it to generate a components.responses entry for that response, which seems possible, I just thought it seemed annoying enough that I didn't really want to bother with it. Do you think it's worth doing?

[ahl]

What if we name the response based on <T as JsonSchema>::schema_name()? Might that work?

Do I think it's worth doing? I think it's worth trying. It might make the code worse, but it might make the output simpler. At a minimum it will make the diffs against current json simpler. These together--I think--at least warrant giving it a shot.

[hawkw]

I don't believe that deduplication is applied to JsonSchema::schema_name(); as
far as I can tell, it only happens once a schema has already been generated,
because that's when the generator can check if the name already exists in the
set of schemas that have been generated so far?

[ahl]

should we be firing a USDT probe here?

[hawkw]

We weren't previously, but yeah, we probably should. I can add it in this PR if that makes sense?

[ahl]

Probably doesn't... (unless you've done it already). We can file an issue

[ahl]

I like this direction