ATS 7.1.8: traffic_server crashed when stop the service

[duanfeixiang]

core文件堆栈：

[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
Core was generated by `/opt/oss/share/NCECOMMONE/APINotifyProxyService/ts/bin/traffic_server --bind_st'.
Program terminated with signal 6, Aborted.
#0 0x00002ad26c2301d7 in raise () from /lib64/libc.so.6
Missing separate debuginfos, use: debuginfo-install glibc-2.17-157.el7.x86_64 libgcc-4.8.5-11.el7.x86_64 xz-libs-5.2.2-1.el7.x86_64
(gdb) bt
#0 0x00002ad26c2301d7 in raise () from /lib64/libc.so.6
#1 0x00002ad26c231a08 in abort () from /lib64/libc.so.6
#2 0x00002ad26c26ff07 in __libc_message () from /lib64/libc.so.6
#3 0x00002ad26c277503 in _int_free () from /lib64/libc.so.6
#4 0x00002ad26c233a70 in __run_exit_handlers () from /lib64/libc.so.6
#5 0x00002ad26c233a95 in exit () from /lib64/libc.so.6
#6 0x00002ad26c21cb3c in __libc_start_main () from /lib64/libc.so.6
#7 0x00002ad2690ecc60 in _start ()

traffic.out

Fatal: [ProcessManager::processSignalQueue] Error writing message!
traffic_server: received signal 11 (Segmentation fault)
traffic_server - STACK TRACE:
*** Error in /opt/oss/share/NCECOMMONE/APINotifyProxyService/ts/bin/traffic_server': malloc(): memory corruption: 0x00002aed903817c7 *** traffic_server: received signal 11 (Segmentation fault) traffic_server - STACK TRACE: traffic_server: received signal 15 (Terminated) traffic_server: received signal 15 (Terminated) [traffic_server: received signal 15 (Terminated) traffic_server: using root directory '/opt/oss/share/NCECOMMONE/APINotifyProxyService/ts' traffic_server: received signal 15 (Terminated) traffic_server: received signal 15 (Terminated) traffic_server: received signal 15 (Terminated) Fatal: [ProcessManager::processSignalQueue] Error writing message! traffic_server: received signal 11 (Segmentation fault) traffic_server - STACK TRACE: *** Error in /opt/oss/share/NCECOMMONE/APINotifyProxyService/ts/bin/traffic_server': free(): corrupted unsorted chunks: 0x00002ad269d83bd0 ***
======= Backtrace: =========
/lib64/libc.so.6(+0x7c503)[0x2ad26c277503]
/lib64/libc.so.6(+0x38a70)[0x2ad26c233a70]
/lib64/libc.so.6(+0x38a95)[0x2ad26c233a95]
/lib64/libc.so.6(__libc_start_main+0xfc)[0x2ad26c21cb3c]

[shinrich]

We have seen such shutdown crashes internally. @duke8253 made some changes to fix or at least reduce these shutdown crashes for us. @duke8253 can you point out the changes that got pushed to open source?

[duanfeixiang]

@shinrich How is this issue handled? Will core files be generated when stoppping the service ?

[bryancall]

@duanfeixiang Yes, it will create a core file if traffic_server or traffic_manager crashes. There have been a lot of fixes since this issue on cleaning up the shutdown. Please reopen or file another issue if this is still a problem.