Fix a crash in PluginVC, reschedule other_side with core_lock_retry_event instead of sm_lock

[oknet]

We met a crash in PluginVC

(gdb) bt full
#0  0x00002b29f722cdf0 in ?? ()
No symbol table info available.
#1  0x00002b29ef4ac3fb in PluginVC::process_close (this=0x2b2a19def180) at PluginVC.cc:700
        __FUNCTION__ = "process_close"
#2  0x00002b29ef4acc10 in PluginVC::main_handler (this=0x2b2a19def180, event=<optimized out>, data=0x2b2a1c13ecc0) at PluginVC.cc:216
        __FUNCTION__ = "main_handler"
        my_ethread = <optimized out>
        call_event = 0x2b2a1c13ecc0
        read_mutex_held = true
        write_mutex_held = true
        read_side_mutex = {m_ptr = 0x2b2a1c1449a0}
        write_side_mutex = {m_ptr = 0x2b2a1c1449a0}
#3  0x00002b29ef792680 in handleEvent (data=0x2b2a1c13ecc0, event=2, this=<optimized out>) at I_Continuation.h:146
No locals.
#4  EThread::process_event (this=this@entry=0x2b29f7132050, e=e@entry=0x2b2a1c13ecc0, calling_code=2) at UnixEThread.cc:130
        c_temp = <optimized out>
        lock = {m = {m_ptr = 0x2b2a1c1449f0}, lock_acquired = true}
#5  0x00002b29ef793283 in EThread::execute (this=0x2b29f7132050) at UnixEThread.cc:212
        done_one = true
        e = 0x2b2a1c13ecc0
        NegativeQueue = {<DLL<Event, Event::Link_link>> = {head = 0x2b29f722dd50}, tail = 0x2b29f722d090}
        next_time = 1492131740118840109
#6  0x00002b29ef79196a in spawn_thread_internal (a=0x2b29f711efd0) at Thread.cc:85
        p = 0x2b29f711efd0
#7  0x00002b29f37cfb50 in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#8  0x00002b29f351a30d in clone () from /lib/x86_64-linux-gnu/libc.so.6
No symbol table info available.
#9  0x0000000000000000 in ?? ()
No symbol table info available.

the local side and other side share one sm_lock_retry_event

(gdb) p this
$1 = (PluginVC * const) 0x2b2a19def180
(gdb) p this->sm_lock_retry_event
$2 = (Event *) 0x2b29f722c7f0
(gdb) p other_side->sm_lock_retry_event
$3 = (Event *) 0x2b29f722c7f0

And the event was cancelled

(gdb) p *(Event *) 0x2b29f722c7f0
$4 = {<Action> = {_vptr.Action = 0x2b29f722c1f0, continuation = 0x2b2a19def180, mutex = {m_ptr = 0x0}, cancelled = 1}, 
  ethread = 0x2b29f7132050, in_the_prot_queue = 0, in_the_priority_queue = 0, immediate = 1, globally_allocated = 0, in_heap = 0, 
  callback_event = 1, timeout_at = 0, period = 0, cookie = 0x0, link = {<SLink<Event>> = {next = 0x0}, prev = 0x0}}

The callback event is inactive_event

(gdb) p inactive_event
$5 = (Event *) 0x0
(gdb) p *call_event
$6 = {<Action> = {_vptr.Action = 0x2b29efabd2b0, continuation = 0x2b2a19def180, mutex = {m_ptr = 0x2b2a1c1449f0}, cancelled = 1}, 
  ethread = 0x2b29f7132050, in_the_prot_queue = 0, in_the_priority_queue = 0, immediate = 0, globally_allocated = 1, in_heap = 0, 
  callback_event = 2, timeout_at = 1492131902133937916, period = 1000000000, cookie = 0x0, link = {<SLink<Event>> = {next = 0x0}, 
    prev = 0x0}}

To reproduce it with heavy load and make request to stats_over_http plugin then wait for inactive_timeout.

[mingzym]

this pull may or may not fix the very old bug: https://issues.apache.org/jira/browse/TS-2462 hopes this will lead us a good future of PluginVC.

[atsci]

RAT check successful! https://ci.trafficserver.apache.org/job/RAT-github/215/

[atsci]

clang format successful! https://ci.trafficserver.apache.org/job/clang-format-github/201/

[atsci]

clang-analyzer build successful! https://ci.trafficserver.apache.org/job/clang-analyzer-github/458/

[atsci]

AU check failed! https://ci.trafficserver.apache.org/job/autest-github/197/

[atsci]

FreeBSD11 build successful! https://ci.trafficserver.apache.org/job/freebsd-github/1896/

[atsci]

Intel CC build successful! https://ci.trafficserver.apache.org/job/icc-github/326/

[atsci]

Linux build successful! https://ci.trafficserver.apache.org/job/linux-github/1787/

[atsci]

clang-analyzer build successful! https://ci.trafficserver.apache.org/job/clang-analyzer-github/459/

[oknet]

Consider this is a better fix for TS-3235 ( PR #164 )

[zwoop]

For issues on Jira, that are relevant, if you can make new Github Issues instead, that would be super helpful. Not required, but it really makes things easier to track (since Jira is now read-only).

[zwoop]

@oknet Is this a candidate for v7.1.x ?

[oknet]

@zwoop Is there any complains about the Interception ? Please backport this to 7.1.x if the Interception is widely used.

[zwoop]

Shall we land this?

[oknet]

Hi Leif, Please do not merge this PR and leave it for me. I am working on the analysis of PluginVC and still not finished. Oknet Xu

…

在 2017年4月28日，09:12，Bryan Call ***@***.***> 写道： @bryancall approved this pull request. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[zwoop]

Marking as WIP, please keep this updated, so we know where we're taking this.

[PSUdaemon]

[approve ci]

[oknet]

This is not a bug, the other_side_call == true means it is called from other side and has mutex locked already.