Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Core dump removing old SIP session #377

Closed
shrhoads opened this issue Nov 10, 2015 · 4 comments
Closed

Core dump removing old SIP session #377

shrhoads opened this issue Nov 10, 2015 · 4 comments

Comments

@shrhoads
Copy link

It looks like there may be a reference leak in old_sessions in the sip plugin.

In my logs I see:

Checking 1 old SIP sessions...
Checking 1 old SIP sessions...
Cleaning up handle 2390786515...
[2390786515] WebRTC resources freed
[2390786515] Handle and related resources freed
Checking 1 old SIP sessions...
Checking 1 old SIP sessions...
Checking 1 old SIP sessions...
Checking 1 old SIP sessions...
Freeing old SIP session

then later, when a second session is unsuccessful in registering with a sip server I see:

Checking 2 old SIP sessions...
[nua_r_shutdown]: 101 Shutdown in progress
[janus.c:janus_cleanup_session:305] Cleaning up session 3918755889...
Destroying session 3918755889
Checking 2 old SIP sessions...
Checking 2 old SIP sessions...
[nua_r_shutdown]: 101 Shutdown in progress
Cleaning up handle 313132925...
[313132925] WebRTC resources freed
[313132925] Handle and related resources freed
Cleaning up handle 3017998674...
[3017998674] WebRTC resources freed
[3017998674] Handle and related resources freed
Checking 2 old SIP sessions...
Checking 2 old SIP sessions...
[nua_r_shutdown]: 101 Shutdown in progress
Checking 2 old SIP sessions...
Freeing old SIP session
Freeing old SIP session
[nua_r_shutdown]: 101 Shutdown in progress
[nua_r_shutdown]: 101 Shutdown in progress
[nua_r_shutdown]: 101 Shutdown in progress
[nua_r_shutdown]: 101 Shutdown in progress
[nua_r_shutdown]: 200 Shutdown successful
Segmentation fault (core dumped)

This happens when the second session error occurs (there are only 2 sessions).

The backtrace is:

(gdb) bt full
#0 0x0000000100000019 in ?? ()

No symbol table info available.
#1 0x00007f69541937a6 in janus_sip_sofia_callback (event=, status=200, phrase=0x7f690c0028f0 "Shutdown successful",

nua=<optimized out>, magic=0x7f6918001af0, nh=0x0, hmagic=0x0, sip=0x0, tags=0x7f690c0028e0) at plugins/janus_sip.c:1514
    ti = <optimized out>
    callstate = <optimized out>
    session = 0x7f6918001af0
    ssip = <optimized out>
    __FUNCTION__ = "janus_sip_sofia_callback"

#2 0x00007f699604df65 in ?? () from /usr/lib/libsofia-sip-ua.so.0

No symbol table info available.
#3 0x00007f699609f302 in ?? () from /usr/lib/libsofia-sip-ua.so.0

No symbol table info available.
#4 0x00007f699609f8f9 in su_base_port_run () from /usr/lib/libsofia-sip-ua.so.0

No symbol table info available.
#5 0x00007f695418910a in janus_sip_sofia_thread (user_data=0x7f6918001af0) at plugins/janus_sip.c:2057

    session = 0x7f6918001af0
    sip_url = "sip:1.2.3.4:*\000i\177\000\000 \000\000\004i\177\000\000\000\b\000\000\000\000\000\000\360\a\000\000\000\000\000\000\220+\000\004i\177\000\000\004\000\000\000\000\000\000\000\035\000\000\000\000\000\000\000\370M֔i\177\000\000\267VE\000\000\000\000\000\360\a\000\000\000\000\000\000\241c\200Ζ\200\377\377\200\000\000\000\000\000\000\000 \000\000\000\000\000\000\000P\000\000\000\000\000\000"
    sips_url = "sips:1.2.3.4:*", '\000' <repeats 20 times>, "n\000\000\000w", '\000' <repeats 11 times>, "_\234\177\061i\177\000\000p\364\001\000\000\000\000\000\000\214\242\063\267i\242\204|\000\000\000i\177\000\000\320\325\341\001\000\000\000\000@\203f\000\000\000\000\000\360\355\343\001\000\000\000\000\260\234\177\061i\177\000\000Qc\200Ζ\200\377\377"
    ipv6 = <optimized out>
    outbound_options = "use-rport no-validate options-keepalive no-natify", '\000' <repeats 206 times>

#6 0x00007f6996dfaf05 in ?? () from /lib/x86_64-linux-gnu/libglib-2.0.so.0

No symbol table info available.
#7 0x00007f69950b2182 in start_thread (arg=0x7f69317fa700) at pthread_create.c:312

    __res = <optimized out>
    pd = 0x7f69317fa700
    now = <optimized out>
    unwind_buf = {cancel_jmp_buf = {{jmp_buf = {140089778743040, -5542106898072312968, 0, 0, 140089778743744, 140089778743040,
            5603632271235507064, 5603974603630338936}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x0}, data = {prev = 0x0,
          cleanup = 0x0, canceltype = 0}}}
    not_first_call = <optimized out>

---Type to continue, or q to quit---
pagesize_m1 =
sp =
freesize =
PRETTY_FUNCTION = "start_thread"
#8 0x00007f6994ddf47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111

No locals.

It seems like the error is in json_is_boolean()?
@lminiero
Copy link
Member

What commit of Janus is this? At line 2057 I see something different.
Why do you think json_is_boolean is involved?

@shrhoads
Copy link
Author

I did a pull and ran my test again.

I was looking at plugins/janus_sip.c:1514, which made me think it was accessing invalid memory at this point?

So the way I can make it happen from the client side is to:catch the onerror event in javascript after a failed sip REGISTER, then invoke Janus.destroy() followed by Janus.init() and then try to register again.

I've worked around it by not doing the destroy()/init() combo, but probably that shouldn't cause the server to fail.

(gdb) bt full
#0 0x0000000100000019 in ?? ()
No symbol table info available.
#1 0x00007f69541937a6 in janus_sip_sofia_callback (event=, status=200, phrase=0x7f690c0028f0 "Shutdown successful",
nua=, magic=0x7f6918001af0, nh=0x0, hmagic=0x0, sip=0x0, tags=0x7f690c0028e0) at plugins/janus_sip.c:1514
ti =
callstate =
session = 0x7f6918001af0
ssip =
FUNCTION = "janus_sip_sofia_callback"
#2 0x00007f699604df65 in ?? () from /usr/lib/libsofia-sip-ua.so.0
No symbol table info available.
#3 0x00007f699609f302 in ?? () from /usr/lib/libsofia-sip-ua.so.0
No symbol table info available.
#4 0x00007f699609f8f9 in su_base_port_run () from /usr/lib/libsofia-sip-ua.so.0
No symbol table info available.
#5 0x00007f695418910a in janus_sip_sofia_thread (user_data=0x7f6918001af0) at plugins/janus_sip.c:2057
session = 0x7f6918001af0
sip_url = "sip:1.2.3.4:\000i\177\000\000 \000\000\004i\177\000\000\000\b\000\000\000\000\000\000\360\a\000\000\000\000\000\000\220+\000\004i\177\000\000\004\000\000\000\000\000\000\000\035\000\000\000\000\000\000\000\370M֔i\177\000\000\267VE\000\000\000\000\000\360\a\000\000\000\000\000\000\241c\200Ζ\200\377\377\200\000\000\000\000\000\000\000 \000\000\000\000\000\000\000P\000\000\000\000\000\000"
sips_url = "sips:1.2.3.4:", '\000' <repeats 20 times>, "n\000\000\000w", '\000' <repeats 11 times>, "_\234\177\061i\177\000\000p\364\001\000\000\000\000\000\000\214\242\063\267i\242\204|\000\000\000i\177\000\000\320\325\341\001\000\000\000\000@\203f\000\000\000\000\000\360\355\343\001\000\000\000\000\260\234\177\061i\177\000\000Qc\200Ζ\200\377\377"
ipv6 =
outbound_options = "use-rport no-validate options-keepalive no-natify", '\000' <repeats 206 times>
#6 0x00007f6996dfaf05 in ?? () from /lib/x86_64-linux-gnu/libglib-2.0.so.0
No symbol table info available.
#7 0x00007f69950b2182 in start_thread (arg=0x7f69317fa700) at pthread_create.c:312
__res =
pd = 0x7f69317fa700
now =
unwind_buf = {cancel_jmp_buf = {{jmp_buf = {140089778743040, -5542106898072312968, 0, 0, 140089778743744, 140089778743040,
5603632271235507064, 5603974603630338936}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x0}, data = {prev = 0x0,
cleanup = 0x0, canceltype = 0}}}
not_first_call =
pagesize_m1 =
sp =
freesize =
PRETTY_FUNCTION = "start_thread"
#8 0x00007f6994ddf47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111
No locals.

@lminiero
Copy link
Member

A destroy+init means the handle is removed as well and you're creating another one, which means the resources from the previous one are in the process of being get rid of. Not sure if this might cause a race condition of some kind when you try to register the same user immediately thereafter.

But I still can't match the lines of your dump with the code we have. It talks about something happening in janus_sip_sofia_callback at line 1514, but on the repo that method only starts at line 1737:

https://github.com/meetecho/janus-gateway/blob/master/plugins/janus_sip.c#L1737

There's a mention of something at line 2057 in janus_sip_sofia_thread, but that method only starts at line 2421:

https://github.com/meetecho/janus-gateway/blob/master/plugins/janus_sip.c#L2421

It looks like you're still using an older (don't know how older, different anyway) version of the plugin. Did you do a make clean + make + make install after the pull?

@shrhoads
Copy link
Author

You are correct my make install was directed at a different location. I fix and reopen if this is still an issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@shrhoads @lminiero