Hard to shutdown, shows stack trace

[ethanfrey]

Summary of Bug

If you run wasmd, it works fine, but fails to shut down on Ctrl+C. If I hit Ctrl+C a second time, it will shut down with a big stack trace.

Version

0.5.2

Steps to Reproduce

See above. I assume this is an issue with shutting down the rust lib and tendermint signal handling.

---

For Admin Use

• Not duplicate issue
• Appropriate labels applied
• Appropriate contributors tagged
• Contributor assigned/self-assigned

[workshub]

This issue is now published on WorksHub. If you would like to work on this issue you can
start work on the WorksHub Issue Details page.

[ethanfrey]

I came across this while debuging some weird test failures in the 0.6 update.

When I hit a nil pointer panic in go, killing the process, I got a giant rust stacktrace. I was tracing down weird ffi stuff, til I finally realized it was really just a go bug. In that debugging, I found the "culprit":

https://github.com/wasmerio/wasmer/blob/master/lib/runtime-core/src/fault.rs

Notably signal_trap_handler, which is set by ensure_sighandler.

This is called in two places in wasmer code, notably in the singlepass backend, which we use. (Also in run_tiering which I have no idea if we invoke.

This probably interacts with the tendermint-install signal handlers in not so great ways...

[workshub]

@MDM-General started working on this issue via WorksHub.

[workshub]

A user started working on this issue via WorksHub.

[yun-yeo]

Seems the SIGINT is not passed to cosmos TrapSignal function due to Wasmer signal handler.

This can disturb Tendermint & Cosmos side clean shutdown. We must solve this problem before being used in production.

[yun-yeo]

Maybe related with this golang/go#21905 (comment)

[hanjukim]

Related issue: wasmerio/wasmer#842

[ethanfrey]

Thank you @hanjukim

That is a good description and exactly what I had thought was happening (but they did investigate more in depth, with different backends having different signal schemes). It does give a good place to start investigating

[hanjukim]

From approach above, I have successfully called terra signal handler from wasmer-runtime-core. I will push my branch soon for reference. 🚀

[ethanfrey]

Is this a PR on wasmer? That would be great.

I was thinking to somehow cache the signal handlers in go before calling rust, then reset them when returning from rust, but that is a bit more ugly than just improving wasmer.

[hanjukim]

wasmerio/wasmer@master...hanjukim:interrupt-signal-propagation

Should I push this PR to wasmer? I am not sure that they will accept this sometime soon. I have added propagation codes for SIGINT only.