library hangs on deleting the instance

[syedtoah]

Hi Edenhill,

I am seeing the issue of hanging again, it is hanging on the ua_idle. I have set the timeout message.timeout.ms to be less (20s) but still this particular loop goes on. I am not sure how to reopen #923 issue.

I am on 0.9.2

BT

[Current thread is 1 (Thread 0x7ffff7fea200 (LWP 2226))]
(gdb) where
#0 0x00007ffff571a4d8 in __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:55
#1 0x00007ffff571d5aa in __GI_abort () at abort.c:89
#2 0x00000000004aeb5e in crash_handler (sig=6) at crash.c:80
#3
#4 0x00007ffff634760b in pthread_join (threadid=140737176553216, thread_return=0x7fffffff75e0) at pthread_join.c:90
#5 0x0000000000477179 in thrd_join (thr=140737176553216, res=0x0) at tinycthread.c:738
#6 0x000000000040eabf in rd_kafka_destroy_app (rk=0x8d7420, blocking=1) at rdkafka.c:581
#7 0x000000000040eb3d in rd_kafka_destroy (rk=0x8d7420) at rdkafka.c:589
#8 0x00000000004080e0 in kafkaClientDestroyTopicAndHandler () at kafka_client.c:443
#9 0x0000000000408303 in kafkaClientConfigInit () at kafka_client.c:561
#10 0x0000000000409670 in kafkaClientCliCmdCallback (cmSession=0x7fffffffb250, cmdId=1239, noForm=0, argc=6, argv=0x7fffffffaa48) at kafka_client_cli.c:193
#11 0x0000000000489279 in cm_run_command (user_function=0x40943f , pmsg=0x7fffffffbf20, sock=15) at climux_lib.c:1344
#12 0x000000000040904c in kafkaClientCliProcessMsg () at kafka_client_cli.c:70
#13 0x000000000040bbe1 in main (argc=1, argv=0x7fffffffe898) at kafka_client_main.c:271
(gdb) info thre
Id Target Id Frame
3 Thread 0x7fffecea3700 (LWP 10074) pthread_cond_timedwait () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:225
2 Thread 0x7fffed6a4700 (LWP 10073) 0x00007ffff634760d in pthread_join (threadid=140737168160512, thread_return=0x7fffed6a12a0) at pthread_join.c:90

1 Thread 0x7ffff7fea200 (LWP 2226) 0x00007ffff571a4d8 in __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:55
(gdb)
Id Target Id Frame
3 Thread 0x7fffecea3700 (LWP 10074) pthread_cond_timedwait () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:225
2 Thread 0x7fffed6a4700 (LWP 10073) 0x00007ffff634760d in pthread_join (threadid=140737168160512, thread_return=0x7fffed6a12a0) at pthread_join.c:90
1 Thread 0x7ffff7fea200 (LWP 2226) 0x00007ffff571a4d8 in __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:55
(gdb)
Id Target Id Frame
3 Thread 0x7fffecea3700 (LWP 10074) pthread_cond_timedwait () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:225
2 Thread 0x7fffed6a4700 (LWP 10073) 0x00007ffff634760d in pthread_join (threadid=140737168160512, thread_return=0x7fffed6a12a0) at pthread_join.c:90
1 Thread 0x7ffff7fea200 (LWP 2226) 0x00007ffff571a4d8 in __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:55
(gdb) thr 2
[Switching to thread 2 (Thread 0x7fffed6a4700 (LWP 10073))]
#0 0x00007ffff634760d in pthread_join (threadid=140737168160512, thread_return=0x7fffed6a12a0) at pthread_join.c:90
90 lll_wait_tid (pd->tid);
(gdb) where
#0 0x00007ffff634760d in pthread_join (threadid=140737168160512, thread_return=0x7fffed6a12a0) at pthread_join.c:90
#1 0x0000000000477179 in thrd_join (thr=140737168160512, res=0x0) at tinycthread.c:738
#2 0x000000000040ee4b in rd_kafka_destroy_internal (rk=0x8d7420) at rdkafka.c:665
#3 0x000000000041141e in rd_kafka_thread_main (arg=0x8d7420) at rdkafka.c:1082
#4 0x0000000000476f1d in _thrd_wrapper_function (aArg=0x8f4140) at tinycthread.c:613
#5 0x00007ffff63463c4 in start_thread (arg=0x7fffed6a4700) at pthread_create.c:337
#6 0x00007ffff57cfa0d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109
(gdb) thr 3
[Switching to thread 3 (Thread 0x7fffecea3700 (LWP 10074))]
#0 pthread_cond_timedwait () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:225
225 62: movq %rax, %r14
(gdb) where
#0 pthread_cond_timedwait () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:225
#1 0x0000000000476e9a in cnd_timedwait_ms (cnd=0x8b9378, mtx=0x8b9350, timeout_ms=10) at tinycthread.c:500
#2 0x00000000004488db in rd_kafka_q_pop_serve (rkq=0x8b9350, timeout_ms=10, version=0, cb_type=0, callback=0x0, opaque=0x0) at rdkafka_queue.c:320
#3 0x00000000004489fc in rd_kafka_q_pop (rkq=0x8b9350, timeout_ms=10, version=0) at rdkafka_queue.c:353
#4 0x0000000000426c19 in rd_kafka_broker_serve (rkb=0x8d7c90, timeout_ms=10) at rdkafka_broker.c:3159
#5 0x0000000000426e55 in rd_kafka_broker_ua_idle (rkb=0x8d7c90, timeout_ms=0) at rdkafka_broker.c:3220
#6 0x000000000042eec9 in rd_kafka_broker_thread_main (arg=0x8d7c90) at rdkafka_broker.c:4544
#7 0x0000000000476f1d in _thrd_wrapper_function (aArg=0x8f43b0) at tinycthread.c:613
#8 0x00007ffff63463c4 in start_thread (arg=0x7fffecea3700) at pthread_create.c:337
#9 0x00007ffff57cfa0d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109
(gdb) thr 1
[Switching to thread 1 (Thread 0x7ffff7fea200 (LWP 2226))]
#0 0x00007ffff571a4d8 in __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:55
55 return INLINE_SYSCALL (tgkill, 3, pid, selftid, sig);
(gdb) where
#0 0x00007ffff571a4d8 in __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:55
#1 0x00007ffff571d5aa in __GI_abort () at abort.c:89
#2 0x00000000004aeb5e in crash_handler (sig=6) at crash.c:80
#3
#4 0x00007ffff634760b in pthread_join (threadid=140737176553216, thread_return=0x7fffffff75e0) at pthread_join.c:90
#5 0x0000000000477179 in thrd_join (thr=140737176553216, res=0x0) at tinycthread.c:738
#6 0x000000000040eabf in rd_kafka_destroy_app (rk=0x8d7420, blocking=1) at rdkafka.c:581
#7 0x000000000040eb3d in rd_kafka_destroy (rk=0x8d7420) at rdkafka.c:589
#8 0x00000000004080e0 in kafkaClientDestroyTopicAndHandler () at kafka_client.c:443
#9 0x0000000000408303 in kafkaClientConfigInit () at kafka_client.c:561
#10 0x0000000000409670 in kafkaClientCallback (cmSession=0x7fffffffb250, cmdId=1239, noForm=0, argc=6, argv=0x7fffffffaa48) at kafka_client_cli.c:193
#11 0x0000000000489279 in cm_run_command (user_function=0x40943f , pmsg=0x7fffffffbf20, sock=15) at mux_lib.c:1344
#12 0x000000000040904c in kafkaClientProcessMsg () at kafka_client_cli.c:70
#13 0x000000000040bbe1 in main (argc=1, argv=0x7fffffffe898) at kafka_client_main.c:271
(gdb)

[edenhill]

Are you destroying your topic_t objects prior to calling rd_kafka_destroy()?

[syedtoah]

Yes, below is my destroy code

if (rk)
    rd_kafka_poll(rk, 0);

/* Wait for messages to be delivered */
while (rk && rd_kafka_outq_len(rk) > 0)
    rd_kafka_poll(rk, 0);

/* Destroy topic */

if(rkt_vT1 != 0)
{   
    rd_kafka_topic_destroy(rkt_vT1);

    rkt_vT1 = 0;
}

if(rkt_vT2 != 0)
{   
    rd_kafka_topic_destroy(rkt_vT2);
    rkt_vT2 = 0;
}

/* Destroy the handle */

if (rk != 0)
{   
    rd_kafka_destroy(rk);

    rk = 0;
}

int r;

while ((r = rd_kafka_thread_cnt()) && timeout-- >= 0)
{  
        while (rd_kafka_wait_destroyed(100) == -1)
        continue;
}

[syedtoah]

Hello Magnus,

Any pointers?

[edenhill]

Sorry, can you reproduce this with debug=all and provide me the logs?

[edenhill]

ping