Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RPC handlers can directly set HTTP response fields #921

Merged
merged 48 commits into from
Mar 6, 2020

Conversation

eddyashton
Copy link
Member

Opening as a draft so I can do a sanity check on the diff, and update documentation.

Apologies for another monster-sized PR, hopefully this is the last for a while.

This PR lets RPC handlers set the status, headers, and body of the HTTP responses. As a result we no longer return JSON-RPC objects - clients must check the HTTP status and headers to know whether the body is an error (string) or successful result (generally JSON). The CCF version information (term, commit, global-commit) are now headers in the response, rather than part of the payload. I've converted all the existing JSON-RPC error codes to HTTP statuses (often Bad Request, sometimes Not Found, Forbidden, or Unauthorized), but feel free to disagree with any of these mappings.

Since the existing JSON requests/responses are still useful and widely used, we have json_adapter functions which handle the parsing in a standard way. In particular, they read the content-type header to work out whether the payload is JSON or msgpack, and pack the result (if successful) in the same way.

To be concrete, between this and #852, we've gone from this:

request:
POST /users/LOG_record HTTP/1.1
content-type: application/json
<...>
{
  "jsonrpc": "2.0",
  "id": 0,
  "method": "users/LOG_record",
  "params":
  {
    "id": 42,
    "msg": "Log me"
  }
}
response:
HTTP/1.1 200 OK
content-type: application/json
<...>
{
  "jsonrpc": "2.0",
  "id": 0,
  "result": true
}

or

HTTP/1.1 200 OK
content-type: application/json
<...>
{
  "jsonrpc": "2.0",
  "id": 0,
  "error": "Cannot record an empty log message"
}

to this:

request:
POST /users/LOG_record HTTP/1.1
content-type: application/json
<...>
{
  "id": 42,
  "msg": "Log me"
}
response:
HTTP/1.1 200 OK
content-type: application/json
<...>
true

or

HTTP/1.1 400 Bad Request
content-type: text/plain
<...>
Cannot record an empty log message

(NB: Handwritten samples, imprecise)

@codecov-io
Copy link

codecov-io commented Mar 4, 2020

Codecov Report

Merging #921 into master will decrease coverage by 16.83%.
The diff coverage is 16.11%.

Impacted file tree graph

@@             Coverage Diff             @@
##           master     #921       +/-   ##
===========================================
- Coverage   69.07%   52.24%   -16.83%     
===========================================
  Files         100       95        -5     
  Lines        7836     6815     -1021     
===========================================
- Hits         5412     3560     -1852     
- Misses       2424     3255      +831
Flag Coverage Δ
#unit_BFT 52.24% <16.11%> (+0.05%) ⬆️
#unit_CFT ?
Impacted Files Coverage Δ
src/node/members.h 20% <ø> (-70%) ⬇️
src/node/script.h 57.14% <ø> (-42.86%) ⬇️
src/consensus/pbft/pbftrequests.h 0% <ø> (ø) ⬆️
src/node/genesisgen.h 16.39% <ø> (-34.43%) ⬇️
src/node/rpc/handlerregistry.h 71.15% <ø> (-19.76%) ⬇️
src/node/clientsignatures.h 0% <ø> (-100%) ⬇️
src/node/rpc/memberfrontend.h 0% <0%> (-67.6%) ⬇️
src/node/rpc/rpcexception.h 0% <0%> (-50%) ⬇️
src/luainterp/txscriptrunner.h 0% <0%> (-95.83%) ⬇️
src/http/http_builder.h 72.22% <0%> (-10.73%) ⬇️
... and 49 more

@ghost
Copy link

ghost commented Mar 4, 2020

non_jsonrpc_response@5741 aka 20200305.24 vs master ewma over 30 builds from 5362 to 5751
images

{
throw std::runtime_error(
"No result in verification response: " + response.dump());
throw std::runtime_error(fmt::format(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does anyone else want a THROW_FMT() or is it just me?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe - we do repeat this pattern a lot. I'd be tempted to keep the throw outside of the macro so that flow control is clear:

throw FMT_ERROR(std::runtime_error, "expected {}, got {}", expected, actual);

return jsonrpc::unpack(ctx->get_request_body(), pack);
}

static nlohmann::json get_params_from_query(
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is an early sketch of what this might look like, and needs some more thought. Basically for GET requests, we should be able to write /users/LOG_get?id=42 rather than {"id": 42} in the body. This does that parsing, but we need to decide how complicated we allow these queries to be, how they're escaped, and how this hooks into the OpenAPI spec (each endpoint should declare whether it expects the params in the query or in the body).

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should maybe look at off-the-shelf options, this one has tests and is allocation-free for example: https://github.com/jacketizer/libyuarel/blob/master/yuarel.h

Copy link
Member Author

@eddyashton eddyashton Mar 5, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree we should look, but I don't think there's anything that gets us very far. yuarel for instance is merely splitting at &s - we can do that trivially, and what we're really interested in is handling escaped-values and type-converting the separated strings.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You’re right, but perhaps we can at least get some tests that way.

@eddyashton eddyashton marked this pull request as ready for review March 5, 2020 14:43
@eddyashton eddyashton requested a review from a team as a code owner March 5, 2020 14:43
Copy link
Contributor

@jumaffre jumaffre left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's awesome!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants