Can't stop gracefully multiple server instances in the same application

[aoudiamoncef]

Hello,

Currently, I'm working with jsonrpsee and I started 3 instances A B and C in the same application.

When I send a request to stop the application in A, we call stop() method in the three instances, but with actual implementation, B and C will never stop. We have to send additional requests to B and C to trigger the graceful shutdown.

To trigger a shutdown, we must create a connection to B and C to trigger the stop_handle.

A simple CTRL+C will never trigger the graceful shutdown.

More detail massalabs/massa#3759)

I'm waiting for feedback, if we could contribute upstream, we'll be happy to help.

Thanks

[niklasad1]

Hey @aoudiamoncef

Which versions of tokio are you using?
We noticed that tokio v1.27 broke our interval waker which was fixed recently #1062 but we haven't made a new release yet.

If you can downgrade tokio to 1.26 that should fix it temporary I think.

[aoudiamoncef]

Hi,

We are using the last stable release of jsonrpsee = 0.16.2 and tokio = 1.23

[niklasad1]

gotcha, something else then I suppose.

Can you elaborate what do you mean by "three instances"? Are these separate servers i.e, you call Server::start or something else?

[aoudiamoncef]

The problem is the fact that to trigger a graceful shutdown, we should send a request to trigger It.

A simple CTRL+C can't kill properly the API and It keeps listening to the opened port.

The workaround is: OK, I'll send a request to stop my running API. But if I have multiples instances, I have to send n queries to stop each one.

We should be able to trigger the stop handle without sending a request.

[niklasad1]

We changed this recently on master to to handle connections tasks in tokio::spawn instead of the FutureDriver which may not be waken up until a new connection occurs. However, really weird that is works on server A but not on server B and C.

Thus, I think this is already fixed on master but could be wrong. Would be great if you could test it 🙏

[aoudiamoncef]

Hi @niklasad1

I confirm that switching to 118acc3 commit in master branch solves the issue.

Thank you so much 👍🏻

[niklasad1]

Cool thanks for confirming, just be bare in mind that master has backpressure which may affect your connections, subscriptions and similar.

We will prep a new release soon

[niklasad1]

This is fixed by #1080 and #1062