libgcrypt crashing when connecting to Mac hosts [Trace: basic_initialization, _gcry_rngfips_initialize] #45

krf · 2014-12-14T14:34:46Z

Heya,

this has been already discussed at the beginning of 2014:
http://sourceforge.net/p/libvncserver/mailman/message/32165820/

This is exactly the same issue, but I'm resending this in hope to get this
fixed. KRDC is unusable for me now.

I'm also getting a crash when using KRDC to connect to a Mac host (running OSX
10.7, if that helps). Backtrace is attached, see [1]

Error message is:

Fatal: failed to create the RNG lock: Invalid argument
FATAL: failed to acquire the FSM lock in libgrypt: Invalid argument

libgcrypt version is: 1.6.1-2ubuntu1

I also tried compiling libvncserver with SSL support and without gcrypt
support but compilation fails in this mode:

../libvncserver/.libs/libvncserver.so: error: undefined reference to 'gcry_md_open'
../libvncserver/.libs/libvncserver.so: error: undefined reference to 'gcry_md_write'
../libvncserver/.libs/libvncserver.so: error: undefined reference to 'gcry_md_ctl'
../libvncserver/.libs/libvncserver.so: error: undefined reference to 'gcry_md_get_algo_dlen'
../libvncserver/.libs/libvncserver.so: error: undefined reference to 'gcry_md_read' collect2: error: ld returned 1 exit status

Note: I have no real clue about VNC/libvncserver/libgcrypt altogether, but
this seems like a real issue people are having which has been around for more
than a year now. Let's get this fixed, happy to help.

Greets

[1] Backtrace:

(gdb) bt
#0  0x00007ffff3e41d27 in __GI_raise (sig=sig@entry=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:56
#1  0x00007ffff3e43418 in __GI_abort () at abort.c:89
#2  0x00007fffe391e8f4 in lock_fsm () at fips.c:235
#3  0x00007fffe391ec01 in fips_new_state (new_state=STATE_FATALERROR) at fips.c:759
#4  0x00007fffe391f0b8 in _gcry_fips_signal_error (srcfile=0x7fffe39b49f0 "misc.c", srcline=140, srcfunc=0x7fffe39b4b00 <__FUNCTION__.7625> 
"_gcry_logv", is_fatal=1, description=0x7fffe39b4a3b "internal
 error (fatal or bug)") at fips.c:732
#5  0x00007fffe3917efa in _gcry_logv (level=level@entry=40, fmt=fmt@entry=0x7fffe39d24e8 "failed to create the RNG lock: %s\n", 
arg_ptr=arg_ptr@entry=0x7fffe0bd68f8) at misc.c:140
#6  0x00007fffe3918274 in _gcry_log_fatal (fmt=fmt@entry=0x7fffe39d24e8 "failed to create the RNG lock: %s\n") at misc.c:230
#7  0x00007fffe39a9665 in basic_initialization () at random-fips.c:201
#8  0x00007fffe39aa2e1 in _gcry_rngfips_initialize (full=1) at random-fips.c:746
#9  _gcry_rngfips_randomize (buffer=0x7fffd4003a90, length=16, level=GCRY_STRONG_RANDOM) at random-fips.c:835
#10 0x00007fffe39a7c80 in _gcry_random_bytes (nbytes=nbytes@entry=16, level=GCRY_STRONG_RANDOM) at random.c:324
#11 0x00007fffe39b23d7 in _gcry_mpi_randomize (w=0x7fffd4003a50, nbits=<optimized out>, level=<optimized out>) at mpiutil.c:586
#12 0x00007fffe3bff25c in InitialiseRFBConnection () from /usr/lib/x86_64-linux-gnu/libvncclient.so.0
#13 0x00007fffe3c01064 in rfbInitClient () from /usr/lib/x86_64-linux-gnu/libvncclient.so.0
#14 0x00007fffe3e17e84 in VncClientThread::clientCreate 
(this=this@entry=0xb25220, reinitialising=reinitialising@entry=false) at ../../vnc/vncclientthread.cpp:555
#15 0x00007fffe3e182b0 in VncClientThread::run (this=0xb25220) at ../../vnc/vncclientthread.cpp:463
#16 0x00007ffff5fe239f in ?? () from /usr/lib/x86_64-linux-gnu/libQtCore.so.4
#17 0x00007ffff23320a5 in start_thread (arg=0x7fffe0bd7700) at pthread_create.c:309
#18 0x00007ffff3f0577d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111

The text was updated successfully, but these errors were encountered:

bk138 · 2014-12-14T17:51:18Z

Tbh, no idea what's going on there. I will provide a minimum Program that triggers the issue and that will hopefully help us to track down the problem.

bk138 · 2014-12-22T16:04:58Z

Ok, can you please try with the provided sample clients in client_examples? I suppose it's got to do with multithreading...

bk138 · 2014-12-22T16:13:59Z

Also, I guess that this might be a bug within krdc as it's not calling gcry_control() to setup gcrypt threading. A note in the libvncclient README would be nice though...

krf · 2014-12-23T08:43:18Z

I figured that I could use gtkvncviewer to try to connect. However: My target host requires a unix-like login, with username + password. I don't see a way to specify the username using that binary(?).

So, in fact, rfbCredential.userCredential.username needs to be filled for me... (I think)

bk138 · 2014-12-23T16:24:41Z

You can try https://github.com/bk138/multivnc which has the GetCredential callback implemented.

bk138 · 2014-12-24T08:48:11Z

Also, http://remmina.sourceforge.net/ 1.0 should also work by looking at its code...

krf · 2014-12-25T09:18:48Z

FYI: I got the same crash when trying to connect using multivnc:

Fatal: failed to create the RNG lock: Invalid argument
FATAL: failed to acquire the FSM lock in libgrypt: Invalid argument

Program received signal SIGABRT, Aborted.
0x00007ffff46d0d27 in __GI_raise (sig=sig@entry=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:56
56      ../nptl/sysdeps/unix/sysv/linux/raise.c: No such file or directory.
(gdb) bt
#0  0x00007ffff46d0d27 in __GI_raise (sig=sig@entry=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:56
#1  0x00007ffff46d2418 in __GI_abort () at abort.c:89
#2  0x00007ffff57c38f4 in lock_fsm () at fips.c:235
#3  0x00007ffff57c3c01 in fips_new_state (new_state=STATE_FATALERROR) at fips.c:759
#4  0x00007ffff57c40b8 in _gcry_fips_signal_error (srcfile=0x7ffff58599f0 "misc.c", srcline=140, srcfunc=0x7ffff5859b00 <__FUNCTION__.7625> "_gcry_logv", is_fatal=1, description=0x7ffff5859a3b "internal
 error (fatal or bug)") at fips.c:732
#5  0x00007ffff57bcefa in _gcry_logv (level=level@entry=40, fmt=fmt@entry=0x7ffff58774e8 "failed to create the RNG lock: %s\n", arg_ptr=arg_ptr@entry=0x7fffffffb9e8) at misc.c:140
#6  0x00007ffff57bd274 in _gcry_log_fatal (fmt=fmt@entry=0x7ffff58774e8 "failed to create the RNG lock: %s\n") at misc.c:230
#7  0x00007ffff584e665 in basic_initialization () at random-fips.c:201
#8  0x00007ffff584f2e1 in _gcry_rngfips_initialize (full=1) at random-fips.c:746
#9  _gcry_rngfips_randomize (buffer=0x82fa80, length=16, level=GCRY_STRONG_RANDOM) at random-fips.c:835
#10 0x00007ffff584cc80 in _gcry_random_bytes (nbytes=nbytes@entry=16, level=GCRY_STRONG_RANDOM) at random.c:324
#11 0x00007ffff58573d7 in _gcry_mpi_randomize (w=0x7078b0, nbits=<optimized out>, level=<optimized out>) at mpiutil.c:586
#12 0x000000000048a0f6 in InitialiseRFBConnection ()
#13 0x000000000048c5f4 in rfbInitClient ()
#14 0x0000000000425c33 in VNCConn::Init(wxString const&, wxString const&, int, int, bool, int, int) ()
#15 0x000000000043aa60 in MyFrameMain::spawn_conn(wxString, int) ()
#16 0x000000000043c1a4 in MyFrameMain::machine_connect(wxCommandEvent&) ()
#17 0x00007ffff5f628be in wxAppConsoleBase::CallEventHandler(wxEvtHandler*, wxEventFunctor&, wxEvent&) const () from /usr/lib/x86_64-linux-gnu/libwx_baseu-3.0.so.0
#18 0x00007ffff60f4368 in wxEvtHandler::ProcessEventIfMatchesId(wxEventTableEntryBase const&, wxEvtHandler*, wxEvent&) () from /usr/lib/x86_64-linux-gnu/libwx_baseu-3.0.so.0
#19 0x00007ffff60f446b in wxEventHashTable::HandleEvent(wxEvent&, wxEvtHandler*) () from /usr/lib/x86_64-linux-gnu/libwx_baseu-3.0.so.0
#20 0x00007ffff60f4818 in wxEvtHandler::TryHereOnly(wxEvent&) () from /usr/lib/x86_64-linux-gnu/libwx_baseu-3.0.so.0
#21 0x00007ffff60f48a3 in wxEvtHandler::ProcessEventLocally(wxEvent&) () from /usr/lib/x86_64-linux-gnu/libwx_baseu-3.0.so.0
#22 0x00007ffff60f4905 in wxEvtHandler::ProcessEvent(wxEvent&) () from /usr/lib/x86_64-linux-gnu/libwx_baseu-3.0.so.0
#23 0x00007ffff6843b48 in wxWindowBase::TryAfter(wxEvent&) () from /usr/lib/x86_64-linux-gnu/libwx_gtk2u_core-3.0.so.0
#24 0x00007ffff60f4677 in wxEvtHandler::SafelyProcessEvent(wxEvent&) () from /usr/lib/x86_64-linux-gnu/libwx_baseu-3.0.so.0
#25 0x00007ffff6818ec0 in wxToolBarBase::OnLeftClick(int, bool) () from /usr/lib/x86_64-linux-gnu/libwx_gtk2u_core-3.0.so.0
#26 0x00007ffff7295474 in _g_closure_invoke_va (closure=0x372e, closure@entry=0x6e3b40, return_value=0x372e, return_value@entry=0x0, instance=0x6, instance@entry=0x573b70, args=0xffffffffffffffff, args@
entry=0x7fffffffcea8, n_params=0, param_types=0x85a010) at /build/buildd/glib2.0-2.42.1/./gobject/gclosure.c:831
#27 0x00007ffff72af057 in g_signal_emit_valist (instance=0x573b70, signal_id=<optimized out>, detail=0, var_args=0x7fffffffcea8) at /build/buildd/glib2.0-2.42.1/./gobject/gsignal.c:3218
#28 0x00007ffff72afefa in g_signal_emit_by_name (instance=0x372e, detailed_signal=0x372e <error: Cannot access memory at address 0x372e>) at /build/buildd/glib2.0-2.42.1/./gobject/gsignal.c:3405
#29 0x00007ffff7295474 in _g_closure_invoke_va (closure=0x372e, closure@entry=0x6dab40, return_value=0x372e, return_value@entry=0x0, instance=0x6, instance@entry=0x6c9930, args=0xffffffffffffffff, args@
entry=0x7fffffffd1c0, n_params=0, param_types=0x85a010) at /build/buildd/glib2.0-2.42.1/./gobject/gclosure.c:831
#30 0x00007ffff72af057 in g_signal_emit_valist (instance=0x6c9930, signal_id=<optimized out>, detail=0, var_args=var_args@entry=0x7fffffffd1c0) at /build/buildd/glib2.0-2.42.1/./gobject/gsignal.c:3218
#31 0x00007ffff72af9af in g_signal_emit (instance=<optimized out>, signal_id=<optimized out>, detail=<optimized out>) at /build/buildd/glib2.0-2.42.1/./gobject/gsignal.c:3365
#32 0x00007ffff7819425 in ?? () from /usr/lib/x86_64-linux-gnu/libgtk-x11-2.0.so.0
#33 0x00007ffff7295474 in _g_closure_invoke_va (closure=0x372e, closure@entry=0x6d5f80, return_value=0x372e, return_value@entry=0x0, instance=0x6, instance@entry=0x6c9930, args=0xffffffffffffffff, args@
entry=0x7fffffffd490, n_params=0, param_types=0x85a010) at /build/buildd/glib2.0-2.42.1/./gobject/gclosure.c:831
#34 0x00007ffff72af057 in g_signal_emit_valist (instance=0x6c9930, signal_id=<optimized out>, detail=0, var_args=var_args@entry=0x7fffffffd490) at /build/buildd/glib2.0-2.42.1/./gobject/gsignal.c:3218
#35 0x00007ffff72af9af in g_signal_emit (instance=<optimized out>, signal_id=<optimized out>, detail=<optimized out>) at /build/buildd/glib2.0-2.42.1/./gobject/gsignal.c:3365

bk138 · 2014-12-28T14:15:40Z

What about remmina?

maxnet · 2014-12-29T14:46:15Z

Also, I guess that this might be a bug within krdc as it's not calling gcry_control() to setup gcrypt threading.

Can't libvncclient just assume pthreads are used on unix platforms, and set it up for that?
It seems to be already making that same assumption when setting up other dependent libraries like OpenSSL (in tls_openssl.c)

bk138 · 2014-12-29T18:28:59Z

Might be an option as per https://www.gnupg.org/documentation/manuals/gcrypt/Multi_002dThreading.html, yep.

maxnet · 2015-01-02T15:01:16Z

libgcrypt version is: 1.6.1-2ubuntu1

Might be an option as per https://www.gnupg.org/documentation/manuals/gcrypt/Multi_002dThreading.html,

Functions mentioned there are dummies in 1.6.x

gcrypt.h

/* NOTE: Since Libgcrypt 1.6 the thread callbacks are not anymore
   used.  However we keep it to allow for some source code
   compatibility if used in the standard way.  */

So problem must be something different.

bk138 · 2015-01-02T17:33:15Z

Should be fixed by #51

krf · 2015-01-06T15:28:02Z

Confirmed. Works for me.

ghost · 2015-08-13T08:48:30Z

Hi krf!

I have the same issue connecting to Mac mini from Ubuntu 15.04.

Remmina just closes without any error message.

gvncviewer dies with
"Fatal: failed to create the RNG lock: Invalid argument
FATAL: failed to acquire the FSM lock in libgrypt: Invalid argument"

vncviewer (xtightvncviewer) works.

Could you please help how to work around?

…sion 0.9.11 0.9.11 - Overall changes: * LibVNCServer/LibVNCClient development now uses continous intregration, provided by TravisCI. - LibVNCClient: * Now initializes libgcrypt before use if the application did not do it. Fixes a crash when connection to Mac hosts (LibVNC/libvncserver#45). * Various fixes that result in more stable handling of malicious or broken servers. * Removed broken and unmaintained H264 decoding. * Some documentation fixes. * Added hooks to WriteToTLS() for optional protection by mutex.

bk138 added the bug label Dec 14, 2014

maxnet mentioned this issue Jan 2, 2015

Initialize libgcrypt before use #51

Merged

bk138 closed this as completed Jan 2, 2015

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

libgcrypt crashing when connecting to Mac hosts [Trace: basic_initialization, _gcry_rngfips_initialize] #45

libgcrypt crashing when connecting to Mac hosts [Trace: basic_initialization, _gcry_rngfips_initialize] #45

krf commented Dec 14, 2014

bk138 commented Dec 14, 2014

bk138 commented Dec 22, 2014

bk138 commented Dec 22, 2014

krf commented Dec 23, 2014

bk138 commented Dec 23, 2014

bk138 commented Dec 24, 2014

krf commented Dec 25, 2014

bk138 commented Dec 28, 2014

maxnet commented Dec 29, 2014

bk138 commented Dec 29, 2014

maxnet commented Jan 2, 2015

bk138 commented Jan 2, 2015

krf commented Jan 6, 2015

ghost commented Aug 13, 2015

libgcrypt crashing when connecting to Mac hosts [Trace: basic_initialization, _gcry_rngfips_initialize] #45

libgcrypt crashing when connecting to Mac hosts [Trace: basic_initialization, _gcry_rngfips_initialize] #45

Comments

krf commented Dec 14, 2014

bk138 commented Dec 14, 2014

bk138 commented Dec 22, 2014

bk138 commented Dec 22, 2014

krf commented Dec 23, 2014

bk138 commented Dec 23, 2014

bk138 commented Dec 24, 2014

krf commented Dec 25, 2014

bk138 commented Dec 28, 2014

maxnet commented Dec 29, 2014

bk138 commented Dec 29, 2014

maxnet commented Jan 2, 2015

bk138 commented Jan 2, 2015

krf commented Jan 6, 2015

ghost commented Aug 13, 2015