ATS9 crash on inactivity timeout

[shinrich]

I just did the experiment again of getting ASF 9.0.x and applying our local commits and outstanding PRs. Good news is the number of additions is around 8. The bad news is that it crashes within 5 minutes of production traffic with the following stack. In frame 3, vc->read.vio.cont is null and vc->write.vio.cont is not null but it doesn't dereference to anything interesting. The do_io_shutdown() will null out the continuation for the associated operation.

I am running with half_closed and HTTP/2 disabled to narrow things down a bit.

I did this experiment 9 days ago, and all was good.

I'm out tomorrow, so probably won't get back to debugging this until Monday.

(gdb) bt
#0  0x000000000072bf46 in handleEvent (data=0x2adf78565d50, event=105, this=0x2adf6aa57000)
    at /home/shinrich/build/_build/build_release_posix-x86_64/trafficserver9/build/../../../../_vcs/trafficserver9/iocore/eventsystem/I_Continuation.h:189
#1  write_signal_and_update (event=105, vc=0x2adf78565b00) at ../../../../../../_vcs/trafficserver9/iocore/net/UnixNetVConnection.cc:115
#2  0x000000000072d709 in UnixNetVConnection::mainEvent(int, Event*) () at ../../../../../../_vcs/trafficserver9/iocore/net/UnixNetVConnection.cc:1167
#3  0x00000000007243e6 in InactivityCop::check_inactivity (this=0x2addcbccb000, event=<optimized out>, e=<optimized out>) at ../../../../../../_vcs/trafficserver9/iocore/net/UnixNet.cc:93
#4  0x000000000077080d in handleEvent (data=0x2addcbc00060, event=2, this=0x2addcbccb000) at ../../../../../../_vcs/trafficserver9/iocore/eventsystem/I_Continuation.h:190
#5  handleEvent (data=0x2addcbc00060, event=2, this=0x2addcbccb000) at ../../../../../../_vcs/trafficserver9/iocore/eventsystem/I_Continuation.h:186
#6  EThread::process_event(Event*, int) () at ../../../../../../_vcs/trafficserver9/iocore/eventsystem/UnixEThread.cc:127
#7  0x00000000007716c4 in EThread::execute_regular (this=this@entry=0x2addc1d9a440) at ../../../../../../_vcs/trafficserver9/iocore/eventsystem/UnixEThread.cc:236
#8  0x0000000000771a02 in execute (this=0x2addc1d9a440) at ../../../../../../_vcs/trafficserver9/iocore/eventsystem/UnixEThread.cc:327
#9  EThread::execute (this=0x2addc1d9a440) at ../../../../../../_vcs/trafficserver9/iocore/eventsystem/UnixEThread.cc:305
#10 0x000000000076ff69 in spawn_thread_internal (a=0x2addbe27e080) at ../../../../../../_vcs/trafficserver9/iocore/eventsystem/Thread.cc:92
#11 0x00002addbc71bdd5 in start_thread () from /lib64/libpthread.so.0
#12 0x00002addbd450ead in clone () from /lib64/libc.so.6

[juergengp]

i have many crashes on 10.0 with centos 8 how can i help ?? where should i upload crash-logs and debug-output ?

[shinrich]

With this stack? You can add stacks/debug output on this issue.

If you have a different stack, please create a new issue.

[shinrich]

Addressed by PR #6889