rawrouter segfaults

[prymitive]

I might be misusing it with this setup, but still a segfault is a segfault.

Start rawrouter without any node

lukasz.mierzwa@sudoku-admin:~/uwsgi$ ./uwsgi --logdate --rawrouter :2200 -M --rawrouter-subscription-server 127.0.0.1:2100 --rawrouter-stats :2203 --rawrouter-max-retries 2 --rawrouter-use-cluster
Fri Jan 25 10:46:50 2013 - *** Starting uWSGI 1.5-dev-e88d4bb (64bit) on [Fri Jan 25 10:46:50 2013] ***
Fri Jan 25 10:46:50 2013 - compiled with version: 4.4.3 on 25 January 2013 10:19:46
Fri Jan 25 10:46:50 2013 - os: Linux-3.0.0-23-virtual #39~lucid1-Ubuntu SMP Fri Jul 20 19:37:29 UTC 2012
Fri Jan 25 10:46:50 2013 - nodename: sudoku-admin
Fri Jan 25 10:46:50 2013 - machine: x86_64
Fri Jan 25 10:46:50 2013 - clock source: unix
Fri Jan 25 10:46:50 2013 - pcre jit disabled
Fri Jan 25 10:46:50 2013 - detected number of CPU cores: 4
Fri Jan 25 10:46:50 2013 - current working directory: /home/lukasz.mierzwa/uwsgi
Fri Jan 25 10:46:50 2013 - detected binary path: /home/lukasz.mierzwa/uwsgi/uwsgi
Fri Jan 25 10:46:50 2013 - your memory page size is 4096 bytes
Fri Jan 25 10:46:50 2013 - detected max file descriptor number: 1024
Fri Jan 25 10:46:50 2013 - lock engine: pthread robust mutexes
Fri Jan 25 10:46:50 2013 - uWSGI rawrouter bound on :2200 fd 3
Fri Jan 25 10:46:50 2013 - uWSGI rawrouter subscription server bound on 127.0.0.1:2100 fd 4
Fri Jan 25 10:46:50 2013 - Python version: 2.6.5 (r265:79063, Oct  1 2012, 22:16:31)  [GCC 4.4.3]
Fri Jan 25 10:46:50 2013 - *** Python threads support is disabled. You can enable it with --enable-threads ***
Fri Jan 25 10:46:50 2013 - Python main interpreter initialized at 0x23e1840
Fri Jan 25 10:46:50 2013 - *** Operational MODE: no-workers ***
Fri Jan 25 10:46:50 2013 - spawned uWSGI master process (pid: 12533)
Fri Jan 25 10:46:50 2013 - spawned uWSGI rawrouter 1 (pid: 12534)
Fri Jan 25 10:46:50 2013 - *** rawrouter stats server enabled on :2203 fd: 9 ***

Stats socket shows:

nc localhost 2203
{
        "version":"1.5-dev-e88d4bb",
        "pid":12534,
        "uid":10062,
        "gid":100,
        "cwd":"/home/lukasz.mierzwa/uwsgi",
        "active_sessions":0,
        "rawrouter":[
":2200",
"127.0.0.1:2100"
        ],
        "subscriptions":[

        ],
        "cheap":0
}

Run ab:

lukasz.mierzwa@sudoku-admin:~$ ab -rn 1000 http://127.0.0.1:2200/
This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking 127.0.0.1 (be patient)
Completed 100 requests
Completed 200 requests
Completed 300 requests
Completed 400 requests
Completed 500 requests
Completed 600 requests
Completed 700 requests
Completed 800 requests
Completed 900 requests
apr_poll: The timeout specified has expired (70007)
Total of 989 requests completed

During ab run it will hang at ~900 requests, stats socket from rawrouter show that there is a bunch of active sessions:

lukasz.mierzwa@sudoku-admin:~/uwsgi$ nc localhost 2203
{
        "version":"1.5-dev-e88d4bb",
        "pid":12537,
        "uid":10062,
        "gid":100,
        "cwd":"/home/lukasz.mierzwa/uwsgi",
        "active_sessions":1918,
        "rawrouter":[
":2200",
"127.0.0.1:2100"
        ],
        "subscriptions":[

        ],
        "cheap":0
}

And rawrouter segfaults:

Fri Jan 25 10:48:15 2013 - !!! uWSGI process 12534 got Segmentation Fault !!!
Fri Jan 25 10:48:15 2013 - *** backtrace of 12534 ***
Fri Jan 25 10:48:15 2013 - ./uwsgi(uwsgi_backtrace+0x25) [0x44d905]
Fri Jan 25 10:48:15 2013 - ./uwsgi(uwsgi_segfault+0x21) [0x44d9e1]
Fri Jan 25 10:48:15 2013 - /lib/libc.so.6(+0x33ba0) [0x7f603ade5ba0]
Fri Jan 25 10:48:15 2013 - ./uwsgi(uwsgi_cr_peer_del+0x2a) [0x475a0a]
Fri Jan 25 10:48:15 2013 - ./uwsgi(corerouter_close_session+0x1b) [0x475b9b]
Fri Jan 25 10:48:15 2013 - ./uwsgi() [0x476006]
Fri Jan 25 10:48:15 2013 - ./uwsgi(uwsgi_corerouter_loop+0x2fb) [0x47631b]
Fri Jan 25 10:48:15 2013 - ./uwsgi(gateway_respawn+0x206) [0x448d86]
Fri Jan 25 10:48:15 2013 - ./uwsgi(master_loop+0x3dd) [0x42d83d]
Fri Jan 25 10:48:15 2013 - ./uwsgi(uwsgi_start+0x13a6) [0x44cff6]
Fri Jan 25 10:48:15 2013 - ./uwsgi(main+0x1027) [0x450547]
Fri Jan 25 10:48:15 2013 - /lib/libc.so.6(__libc_start_main+0xfd) [0x7f603add0c4d]
Fri Jan 25 10:48:15 2013 - ./uwsgi() [0x41d359]
Fri Jan 25 10:48:15 2013 - *** end of backtrace ***
Fri Jan 25 10:48:16 2013 - respawned uWSGI rawrouter 1 (pid: 12537)
Fri Jan 25 10:48:16 2013 - *** rawrouter stats server enabled on :2203 fd: 9 ***

Sessions hangs, than after a while segfault occurs. If I continue to run ab it segfaults more.

[prymitive]

./uwsgi --logdate --rawrouter :2200 -M --rawrouter-stats :2203 is enough options

[prymitive]

looks fixed with latest commit