Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Segfault in alpine docker image while initializing client #1072

Closed
Hakukano opened this issue Oct 29, 2020 · 7 comments
Closed

Segfault in alpine docker image while initializing client #1072

Hakukano opened this issue Oct 29, 2020 · 7 comments

Comments

@Hakukano
Copy link

This issue can only be reproduced in Alpine docker image: alpine or Rust alpine image: rust:alpine. Debian based images work just fine with the same code.

Min code

Running in actix-web::main

use reqwest::Client;
...
let client = Client::new(); // segfault (docker exist code 139) here
// Or using builder
let client = Client::builder().build(); // segfault (docker exist code 139) here
...

Error log

There isn't any log whatsoever. According to docker exit code 139, there is a segfault occurred.

Possible cause

I don't think it's caused by missing libraries or the log should show what's missing but instead, there are no logs.
Maybe something wrong with alpine's musl libc vs glibc?

@seanmonstar
Copy link
Owner

Well that's pretty strange. Could you try debugging a little more, perhaps using strace to see what calls the program gets to before crashing?

@Hakukano
Copy link
Author

Here is the strace:

Before crash

...
rt_sigprocmask(SIG_BLOCK, [QUIT], [], 8) = 0
brk(0x55555714e000)                     = 0x55555714e000
rt_sigaction(SIGQUIT, {sa_handler=0x7f64849224e0, sa_mask=[], sa_flags=SA_RESTORER|SA_RESTART|SA_SIGINFO|SA_NOCLDSTOP, sa_restorer=0x7f648496510f}, {sa_handler=SIG_DFL, sa_mask=[], sa_flags=0}, 8) = 0
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
fcntl(46, F_DUPFD_CLOEXEC, 0)           = 51
fcntl(51, F_SETFD, FD_CLOEXEC)          = 0
fcntl(51, F_GETFD)                      = 0x1 (flags FD_CLOEXEC)
epoll_ctl(3, EPOLL_CTL_ADD, 51, {EPOLLIN|EPOLLPRI|EPOLLOUT|EPOLLET, {u32=3, u64=3}}) = 0
epoll_pwait(3, [{EPOLLOUT, {u32=0, u64=0}}, {EPOLLOUT, {u32=1, u64=1}}, {EPOLLOUT, {u32=2, u64=2}}, {EPOLLOUT, {u32=3, u64=3}}], 1024, -1, NULL, 8) = 4

Crash

epoll_pwait(3,  <unfinished ...>)       = ?
+++ killed by SIGSEGV (core dumped) +++

@Hakukano
Copy link
Author

Additionally, I have Valgrind log here:

==1== Thread 2 actix-rt:worker::
==1== Jump to the invalid address stated on the next line
==1==    at 0x0: ???
==1==    by 0x31EC3D: native_tls::imp::TlsConnector::new (in /fwcheck)
==1==    by 0x31F2B4: native_tls::TlsConnectorBuilder::build (in /fwcheck)
==1==    by 0x31C423: reqwest::connect::Connector::new_default_tls (in /fwcheck)
==1==    by 0x305706: reqwest::async_impl::client::ClientBuilder::build (in /fwcheck)
==1==    by 0x20A7F0: <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll (in /fwcheck)
==1==    by 0x1B06A1: <actix_web::handler::HandlerServiceResponse<T,R> as core::future::future::Future>::poll (in /fwcheck)
==1==    by 0x1BA4A8: <actix_web::handler::ExtractResponse<T,S> as core::future::future::Future>::poll (in /fwcheck)
==1==    by 0x1BACC9: <actix_web::handler::ExtractResponse<T,S> as core::future::future::Future>::poll (in /fwcheck)
==1==    by 0x1D6556: <futures_util::future::future::map::Map<Fut,F> as core::future::future::Future>::poll (in /fwcheck)
==1==    by 0x1EBAA8: <futures_util::future::future::Map<Fut,F> as core::future::future::Future>::poll (in /fwcheck)
==1==    by 0x357780: <core::pin::Pin<P> as core::future::future::Future>::poll (in /fwcheck)
==1==  Address 0x0 is not stack'd, malloc'd or (recently) free'd
==1==
==1==
==1== Process terminating with default action of signal 11 (SIGSEGV): dumping core
==1==  Bad permissions for mapped region at address 0x0
==1==    at 0x0: ???
==1==    by 0x31EC3D: native_tls::imp::TlsConnector::new (in /fwcheck)
==1==    by 0x31F2B4: native_tls::TlsConnectorBuilder::build (in /fwcheck)
==1==    by 0x31C423: reqwest::connect::Connector::new_default_tls (in /fwcheck)
==1==    by 0x305706: reqwest::async_impl::client::ClientBuilder::build (in /fwcheck)
==1==    by 0x20A7F0: <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll (in /fwcheck)
==1==    by 0x1B06A1: <actix_web::handler::HandlerServiceResponse<T,R> as core::future::future::Future>::poll (in /fwcheck)
==1==    by 0x1BA4A8: <actix_web::handler::ExtractResponse<T,S> as core::future::future::Future>::poll (in /fwcheck)
==1==    by 0x1BACC9: <actix_web::handler::ExtractResponse<T,S> as core::future::future::Future>::poll (in /fwcheck)
==1==    by 0x1D6556: <futures_util::future::future::map::Map<Fut,F> as core::future::future::Future>::poll (in /fwcheck)
==1==    by 0x1EBAA8: <futures_util::future::future::Map<Fut,F> as core::future::future::Future>::poll (in /fwcheck)
==1==    by 0x357780: <core::pin::Pin<P> as core::future::future::Future>::poll (in /fwcheck)
==1==
==1== HEAP SUMMARY:
==1==     in use at exit: 0 bytes in 0 blocks
==1==   total heap usage: 0 allocs, 0 frees, 0 bytes allocated
==1==
==1== All heap blocks were freed -- no leaks are possible
==1==
==1== Use --track-origins=yes to see where uninitialised values come from
==1== For lists of detected and suppressed errors, rerun with: -s
==1== ERROR SUMMARY: 5 errors from 2 contexts (suppressed: 0 from 0)

valgrind: the 'impossible' happened:
   main(): signal was supposed to be fatal

host stacktrace:
==1==    at 0x58031AD2: show_sched_status_wrk (m_libcassert.c:388)
==1==    by 0x58031D26: report_and_quit (m_libcassert.c:459)
==1==    by 0x58031F3D: panic (m_libcassert.c:535)
==1==    by 0x58031F3D: vgPlain_core_panic_at (m_libcassert.c:540)
==1==    by 0x58031F45: vgPlain_core_panic (m_libcassert.c:545)
==1==    by 0x58081326: shutdown_actions_NORETURN (m_main.c:2268)
==1==    by 0x580AA3C7: run_a_thread_NORETURN (syswrap-linux.c:203)
==1==    by 0x580AA465: vgModuleLocal_start_thread_NORETURN (syswrap-linux.c:320)
==1==    by 0x5807FA72: ??? (in /usr/lib/valgrind/memcheck-amd64-linux)
==1==    by 0xDEADBEEFDEADBEEE: ???
==1==    by 0xDEADBEEFDEADBEEE: ???
==1==    by 0xDEADBEEFDEADBEEE: ???

sched status:
  running_tid=2


Note: see also the FAQ in the source distribution.
It contains workarounds to several common problems.
In particular, if Valgrind aborted or crashed after
identifying problems in your program, there's a good chance
that fixing those problems will prevent Valgrind aborting or
crashing, especially if it happened in m_mallocfree.c.

If that doesn't help, please report this bug to: www.valgrind.org

In the bug report, send all the above text, the valgrind
version, and what OS and version you are using.  Thanks.

@seanmonstar
Copy link
Owner

Ah interesting. It could be the crash is from native-tls... You could file an issue there. Also, you could try disabling native-tls in reqwest and enabling rustls.

@onelson
Copy link

onelson commented Oct 29, 2020

In the past there have been caveats cross-compiling with musl around getting openssl to link correctly. Workarounds involved explicitly adding an extern crate openssl; to all the binary targets in your project and somehow the ordering was significant... emk/rust-musl-builder#69 (comment)

Using rustls should sidestep the issue since you'd no longer need to link openssl.

@Hakukano
Copy link
Author

Enabling rustls works for me! Thanks guys!

@Daktic
Copy link

Daktic commented Jan 23, 2024

Can you expand on what you mean by enable rustls to sidestep the tls issue?

I have been stuck on this for days, and can't get my head wrapped around how to fix this.

I have a really simple implementation here where I just create an actix-web route, initialize a new reqwest client, and I segfault when calling the route. I can see that it is related to the TLS failing to do something, but can't seem to get anything to work to fix it.

https://github.com/Daktic/mock_reqwest

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants