Segfault in UMA shutdown / destructor path

[erikarn]

After running for a while, a clean shutdown ends with a segfault:

Core was generated by `blockd'.
Program terminated with signal 11, Segmentation fault.
#0 uma_zfree_arg (zone=0x802e58d80, item=0x802505000, udata=0x0) at /home/adrian/git/github/erikarn/libuinet/lib/libuinet/../../sys/vm/uma_core.c:2788

2788 bucket = cache->uc_freebucket;
(gdb) bt
#0 uma_zfree_arg (zone=0x802e58d80, item=0x802505000, udata=0x0) at /home/adrian/git/github/erikarn/libuinet/lib/libuinet/../../sys/vm/uma_core.c:2788
#1 0x000000000049485c in zone_free_item (zone=0x802e57000, item=0x8024f5b00, udata=0x0, skip=, flags=0)

at /home/adrian/git/github/erikarn/libuinet/lib/libuinet/../../sys/vm/uma_core.c:2932

#2 0x0000000000493342 in bucket_drain (zone=0x802e57000, bucket=0x80278f418) at /home/adrian/git/github/erikarn/libuinet/lib/libuinet/../../sys/vm/uma_core.c:660
#3 local_cache_drain (zone=0x802e57000, cache=) at /home/adrian/git/github/erikarn/libuinet/lib/libuinet/../../sys/vm/uma_core.c:670
#4 uma_tls_destructor (arg=0x81c00f000) at /home/adrian/git/github/erikarn/libuinet/lib/libuinet/../../sys/vm/uma_core.c:1653
#5 0x000000080229dd25 in ?? () from /lib/libthr.so.3
#6 0x00000008022a533e in ?? () from /lib/libthr.so.3
#7 0x00000008022a5289 in ?? () from /lib/libthr.so.3
#8 0x00000008022a50fb in pthread_exit () from /lib/libthr.so.3
#9 0x000000080229a4fd in ?? () from /lib/libthr.so.3
#10 0x0000000000000000 in ?? ()

(gdb) frame 0
#0 uma_zfree_arg (zone=0x802e58d80, item=0x802505000, udata=0x0) at /home/adrian/git/github/erikarn/libuinet/lib/libuinet/../../sys/vm/uma_core.c:2788

2788 bucket = cache->uc_freebucket;
(gdb) print cache
$1 =
(gdb) print bucket
$2 =
(gdb) print zone
$3 = (uma_zone_t) 0x802e58d80
(gdb) print *zone
$4 = {uz_name = 0x4b0ff5 "mbuf_cluster", uz_lock = 0x802e598d0, uz_link = {le_next = 0x0, le_prev = 0x802e59908}, uz_full_bucket = {lh_first = 0x0}, uz_free_bucket = {lh_first = 0x0}, uz_kegs = {
lh_first = 0x802e58db8}, uz_klink = {kl_link = {le_next = 0x0, le_prev = 0x802e58db0}, kl_keg = 0x802e598c0}, uz_slab = 0x4963b0 <zone_fetch_slab>, uz_ctor = 0x432540 <mb_ctor_clust>,
uz_dtor = 0x432620 <mb_dtor_clust>, uz_init = 0x0, uz_fini = 0x0, uz_flags = 9224, uz_size = 2048, uz_allocs = 245, uz_frees = 0, uz_fails = 0, uz_sleeps = 0, uz_fills = 0, uz_count = 24,
uz_cacheidx = 2}
(gdb) print uma_tls_key
$5 = 3
(gdb) print uhi_tls_get(uma_tls_key)
You can't do that without a process to debug.
(gdb) frame 4
#4 uma_tls_destructor (arg=0x81c00f000) at /home/adrian/git/github/erikarn/libuinet/lib/libuinet/../../sys/vm/uma_core.c:1653

1653 local_cache_drain(tls->ut_caches[i].uc_zone, &tls->ut_caches[i]);
(gdb)

I'll recompile with -O0 and retry but I bet it's NULL.

[erikarn]

Ok, with -O0:

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 802c09c00 (LWP 100519 nm_rx: netmap0)]
0x000000000050244a in uma_zfree_arg (zone=0x802e59d80, item=0x8008ca800, udata=0x0) at /home/adrian/git/github/erikarn/libuinet/lib/libuinet/../../sys/vm/uma_core.c:2788
2788 bucket = cache->uc_freebucket;
(gdb) print cache
$1 = (uma_cache_t) 0x180
(gdb)

[erikarn]

(gdb) print cache
$1 = (uma_cache_t) 0x180
(gdb) print *zone
$2 = {uz_name = 0x52a6a3 "mbuf_cluster", uz_lock = 0x802e5a8d0, uz_link = {le_next = 0x0, le_prev = 0x802e5a908}, uz_full_bucket = {lh_first = 0x0}, uz_free_bucket = {lh_first = 0x0}, uz_kegs = {lh_first = 0x802e59db8}, uz_klink = {kl_link = {le_next = 0x0, le_prev = 0x802e59db0},
kl_keg = 0x802e5a8c0}, uz_slab = 0x504bb0 <zone_fetch_slab>, uz_ctor = 0x4514e0 <mb_ctor_clust>, uz_dtor = 0x451690 <mb_dtor_clust>, uz_init = 0x0, uz_fini = 0x0, uz_flags = 9224, uz_size = 2048, uz_allocs = 245, uz_frees = 0, uz_fails = 0, uz_sleeps = 0, uz_fills = 0,
uz_count = 24, uz_cacheidx = 2}
(gdb) info threads
[New Thread 84800f400 (LWP 100527)]
[New Thread 802c0b800 (LWP 100526)]
[New Thread 802c0b400 (LWP 100525)]
[New Thread 802c0b000 (LWP 100524)]
[New Thread 802c0ac00 (LWP 100523)]
[New Thread 802c0a800 (LWP 100522)]
[New Thread 802c0a400 (LWP 100521)]
[New Thread 802c0a000 (LWP 100520)]
[New Thread 802c09800 (LWP 100518 nm_tx: netmap0)]
[New Thread 802c09400 (LWP 100517 one_sighandler)]
[New Thread 802c09000 (LWP 100516 shutdown_helper)]
[New Thread 802c08400 (LWP 100515 intr: newborn)]
[New Thread 802c08000 (LWP 100514 intr: newborn)]
[New Thread 802c07c00 (LWP 100513 intr: newborn)]
[New Thread 802c07800 (LWP 100512 thread taskq)]
[New Thread 802c07400 (LWP 100511 kqueue taskq)]
[New Thread 802c07000 (LWP 100510 clock)]
[New Thread 802c06c00 (LWP 100509 intr: newborn)]
[New Thread 802c06400 (LWP 100139)]
Id Target Id Frame
21 Thread 802c06400 (LWP 100139) 0x00000008023268cc in ?? () from /lib/libthr.so.3
20 Thread 802c06c00 (LWP 100509 intr: newborn) 0x00000008023268cc in ?? () from /lib/libthr.so.3
19 Thread 802c07000 (LWP 100510 clock) 0x0000000801c958ba in nanosleep () from /lib/libc.so.7
18 Thread 802c07400 (LWP 100511 kqueue taskq) 0x00000008023268cc in ?? () from /lib/libthr.so.3
17 Thread 802c07800 (LWP 100512 thread taskq) 0x00000008023268cc in ?? () from /lib/libthr.so.3
16 Thread 802c07c00 (LWP 100513 intr: newborn) 0x00000008023268cc in ?? () from /lib/libthr.so.3
15 Thread 802c08000 (LWP 100514 intr: newborn) 0x00000008023268cc in ?? () from /lib/libthr.so.3
14 Thread 802c08400 (LWP 100515 intr: newborn) 0x00000008023268cc in ?? () from /lib/libthr.so.3
13 Thread 802c09000 (LWP 100516 shutdown_helper) 0x00000008023268cc in ?? () from /lib/libthr.so.3
12 Thread 802c09400 (LWP 100517 one_sighandler) 0x0000000801c958ba in nanosleep () from /lib/libc.so.7
11 Thread 802c09800 (LWP 100518 nm_tx: netmap0) 0x00000008023268cc in ?? () from /lib/libthr.so.3
10 Thread 802c0a000 (LWP 100520) 0x00000008023268cc in ?? () from /lib/libthr.so.3
9 Thread 802c0a400 (LWP 100521) 0x0000000801bdfa9a in _sigwait () from /lib/libc.so.7
8 Thread 802c0a800 (LWP 100522) 0x0000000801c407ea in accept () from /lib/libc.so.7
7 Thread 802c0ac00 (LWP 100523) 0x0000000801cb7bfa in read () from /lib/libc.so.7
6 Thread 802c0b000 (LWP 100524) 0x0000000801c958ba in nanosleep () from /lib/libc.so.7
5 Thread 802c0b400 (LWP 100525) 0x0000000801c9704a in kevent () from /lib/libc.so.7
4 Thread 802c0b800 (LWP 100526) 0x0000000801c407ea in accept () from /lib/libc.so.7
3 Thread 84800f400 (LWP 100527) 0x0000000801cb7bfa in read () from /lib/libc.so.7

• 2 Thread 802c09c00 (LWP 100519 nm_rx: netmap0) 0x000000000050244a in uma_zfree_arg (zone=0x802e59d80, item=0x8008ca800, udata=0x0) at /home/adrian/git/github/erikarn/libuinet/lib/libuinet/../../sys/vm/uma_core.c:2788
  (gdb)

[pkelsey]

This is fixed by cdb8e06.