Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Graceful shutdown #1079

Closed
wants to merge 56 commits into from
Closed

Graceful shutdown #1079

wants to merge 56 commits into from

Conversation

jhpratt
Copy link
Contributor

@jhpratt jhpratt commented Aug 7, 2019

This PR implements a mandatory graceful shutdown on a Rocket instance, which can be called at runtime.

Calling .get_shutdown_handle() before launch will return a ShutdownHandle. As shutting things down before starting isn't too useful most of the time, there is also rocket::shutdown::ShutdownHandle, which implements FromRequest, so it can be used as a request guard. In either situation, calling .shutdown() will signal the server to shut down.

Per a preexisting comment in the code, I've also converted rocket.launch() to return Result<(), LaunchError>.

Resolves #180.

schrieveslaach and others added 22 commits July 19, 2019 17:40
- Use hyper's MakeService implementation with futures API
- Use tokio runtime to serve HTTP backend
This lets us keep support for keep-alive and remote address while doing
other work on async, at the cost of TLS. Abstracting over the connection
type will be done more thoroughly later.
This requires some awkward channel and spawning work because
Body might contain borrowed data.
Adds body_string_wait and body_bytes_wait functions to LocalRequest for convenience.
This is required to be able to do anything useful with the body in the
outgoing response. Request fairings do not appear to need to be async
as everything on Data that returns a future moves self and on_request only
gets &Data, but the same change in this commit should work for on_request
if desired.
Like with response fairings, this is required to be able to do anything
useful with the body.
Also update 'static_files' example.
@jhpratt jhpratt marked this pull request as ready for review August 8, 2019 02:35
core/lib/src/rocket.rs Outdated Show resolved Hide resolved
@jhpratt
Copy link
Contributor Author

jhpratt commented Aug 10, 2019

Just a note, it won't be too difficult to add signal handling (ctrl + C), as the signal-hook crate provides for simple abstractions. There's also tokio-signal, which is similar. That can be done in this PR if wanted.

Copy link
Collaborator

@jebrosen jebrosen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

currently able to be called using the rocket.shutdown() method

I think that's currently the biggest flaw with this PR -- nothing can currently call rocket.shutdown() once the server is started except for Rocket's own request/response plumbing, which definitely shouldn't be calling it. The sender needs to either be threaded through Request or put in managed state in order to be usable.

As a proof of concept, I think we need a ShutdownHandle type that implements FromRequest. That way, someone could write a #[post("/shutdown")] route or similar. I think we will also need a way to get a ShutdownHandle for a not-yet-launched rocket instance, which can be sent to another thread.

Just a note, it won't be too difficult to add signal handling

Agreed. Let's look at what it takes to add it once we know it can be made to work in a route or fairing.

core/lib/src/rocket.rs Outdated Show resolved Hide resolved
@jhpratt
Copy link
Contributor Author

jhpratt commented Aug 11, 2019

Is there anything currently using managed stated within Rocket itself? If not, wouldn't it be best to preserve that expectation?

Having a ShutdownHandle is definitely feasible, I'd actually just need to revert one of the commits where I included the changes. Off the top of my head, an mpsc sender is cloneable, so it should be relatively easy to implement FromRequest. We could then expose ShutdownHandle (the exact path TBD), and use it like any other guard from the client's perspective.

With regard to handling the sender/receiver errors, should the channel be left open after receiving a shutdown signal? I see no reason to, as it would provide a clear indication that the method is being called more than once.

@jebrosen
Copy link
Collaborator

Let's use this error type for launch() for the time being:

enum Error {
    Launch(LaunchError),
    Run(Box<dyn std::error::Error>),
}

@jhpratt
Copy link
Contributor Author

jhpratt commented Aug 26, 2019

Would it be better to use hyper::Error over Box<dyn std::error::Error>, given that we know that's the type it will be? After taking a quick look at things, hyper::Error is actually already public via LaunchErrorKind, so it wouldn't be newly exposed.

.spawn_on() could also return hyper::Error over the boxed, since you had wanted to return a Result<impl Future<Result<(), ??>, LaunchError>, per previous discussion. The advantage of that signature is not having to .await to determine if launch failed.

@jebrosen
Copy link
Collaborator

Interesting, I really thought hyper was "erased" from the error type. In that case, I don't mind hyper::Error instead of the boxed error after all.

@jhpratt
Copy link
Contributor Author

jhpratt commented Aug 26, 2019

Updated to use an enum that can contain a LaunchError or a hyper::Error. I haven't written documentation for it, but it would be similar to LaunchError's.

@jhpratt
Copy link
Contributor Author

jhpratt commented Aug 27, 2019

For anyone that might be following this PR, I'm going to attempt adding signal handling once tokio-rs/tokio#1498 is figured out. I believe it should just amount to a couple lines added.

Copy link
Collaborator

@jebrosen jebrosen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This does increase our reliance on tokio; I think we will want to make Ctrl-C handling optional somehow. The overall architecture of this looks fine, but I haven't actually run the code yet myself.

core/lib/src/rocket.rs Outdated Show resolved Hide resolved
core/lib/src/rocket.rs Outdated Show resolved Hide resolved
core/lib/src/rocket.rs Outdated Show resolved Hide resolved
core/lib/src/rocket.rs Outdated Show resolved Hide resolved
core/lib/src/error.rs Outdated Show resolved Hide resolved
core/lib/src/shutdown.rs Outdated Show resolved Hide resolved
Since the `signal` feature in Tokio is only used when we want the ctrl+c
handling, it's not necessary to compile that bit of code if we're not
using it.
Copy link
Collaborator

@jebrosen jebrosen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some doc nits, and then I think I'm pretty much happy with this once I test it.

core/lib/src/error.rs Outdated Show resolved Hide resolved
core/lib/src/rocket.rs Outdated Show resolved Hide resolved
core/lib/src/rocket.rs Outdated Show resolved Hide resolved
core/lib/src/rocket.rs Show resolved Hide resolved
core/lib/src/shutdown.rs Show resolved Hide resolved
Use a newtype wrapper that the client doesn't have access to, ensuring
they're not able to access it.
@jebrosen
Copy link
Collaborator

jebrosen commented Sep 9, 2019

Merged as 49709a6, with some error handling followup in a3719c4. Thanks for all the hard work on this!

@jebrosen jebrosen closed this Sep 9, 2019
@jebrosen jebrosen added the pr: merged This pull request was merged manually. label Sep 9, 2019
@jhpratt jhpratt deleted the graceful-shutdown branch September 9, 2019 00:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
pr: merged This pull request was merged manually.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants