python plugin routing support

[gber]

Virtual hosting with the python plugin is already possible through routing as in http://article.gmane.org/gmane.comp.python.wsgi.uwsgi.general/5153. However, there seems to be no way to specify different virtualenvs for each application, for that it is still necessary to configure a reverse proxy in front of uwsgi as in http://projects.unbit.it/uwsgi/wiki/Example#TwoPinaxsiteintwovirtualenvintwovirtualhostwithonlyoneuWSGIinstance. It would be desirable to have uwsgi handle this on its own so the owner of an uwsgi vassal who is not in control of the reverse proxy in front of uwsgi can configure this on his own. As discussed, routing targets are the current approach to virtualhosting in uwsgi, in this case the python: target should allow the specification of the UWSGI_PYHOME, UWSGI_SCRIPT, and UWSGI_CHDIR parameters.

[unbit]

Added chdir,setapp and sethome actions:

[uwsgi]

http-socket = :9090

mount = topogigio=werkzeug.testapp:test_app
mount = topolino=hello.py

route = ^/foo chdir:/tmp
route = ^/foo setapp:topogigio

route = ^/bar chdir:/var
route = ^/bar setapp:topolino

i have not tested sethome, let me know if it works

[gber]

Thanks, chdir and setapp work fine. With sethome I'm not sure, it should be the equivalent to UWSGI_PYHOME or --virtualenv, right? With the following config, the virtualenv in /home/gber/venv1 is not active (importing a module installed there does not work):

[uwsgi]
plugins = 0:notfound,16:python,router_uwsgi
http-socket = :8000
no-default-app = true
mount = app1=/home/gber/venv1/app1.py
mount = app2=/home/gber/venv2/app2.py

route = ^/foo chdir:/home/gber/venv1
route = ^/foo setapp:app1
route = ^/foo sethome:/home/gber/venv1
route = ^/foo uwsgi:,16,0

route = ^/bar chdir:/home/gber/venv2
route = ^/bar setapp:app2
route = ^/bar sethome:/home/gber/venv2
route = ^/bar uwsgi:,16,0

A request of http://127.0.0.1:8000/foo shows the following debug output:

add uwsgi var: REQUEST_METHOD = GET
add uwsgi var: REQUEST_URI = /foo
add uwsgi var: PATH_INFO = /foo
add uwsgi var: QUERY_STRING =
add uwsgi var: SERVER_PROTOCOL = HTTP/1.1
add uwsgi var: SCRIPT_NAME =
add uwsgi var: SERVER_NAME = hal.local.invalid
add uwsgi var: SERVER_PORT = 8000
add uwsgi var: REMOTE_ADDR = 127.0.0.1
[uWSGI DEBUG] PATH_INFO=/foo
[uWSGI DEBUG] SCRIPT_NAME=
[uWSGI DEBUG] SERVER_NAME=hal.local.invalid
[uWSGI DEBUG] HTTP_HOST=127.0.0.1:8000
searching for app1 in app1 0x7fcdaee0e500
[uWSGI DEBUG] REQUEST_METHOD: GET
0x7fcdad1e48b8 14 0x7fcdad1e6f58 3
[uWSGI DEBUG] REQUEST_URI: /foo
0x7fcdad1e1b70 11 0x7fcdacee3ed0 4
[uWSGI DEBUG] PATH_INFO: /foo
0x7fcdacee3f00 9 0x7fcdacee3f30 4
[uWSGI DEBUG] QUERY_STRING:
0x7fcdad1e4998 12 0x7fcdaeede508 0
[uWSGI DEBUG] SERVER_PROTOCOL: HTTP/1.1
0x7fcdad1e4dc0 15 0x7fcdacee3f60 8
[uWSGI DEBUG] SCRIPT_NAME:
0x7fcdacee3cf0 11 0x7fcdaeede508 0
[uWSGI DEBUG] SERVER_NAME: hal.local.invalid
0x7fcdacee5270 11 0x7fcdad1e4df8 17
[uWSGI DEBUG] SERVER_PORT: 8000
0x7fcdacee52a0 11 0x7fcdacee52d0 4
[uWSGI DEBUG] REMOTE_ADDR: 127.0.0.1
0x7fcdacee5300 11 0x7fcdacee5330 9
[uWSGI DEBUG] HTTP_USER_AGENT: curl/7.28.1
0x7fcdacee6148 15 0x7fcdacee5360 11
[uWSGI DEBUG] HTTP_HOST: 127.0.0.1:8000
0x7fcdacee5390 9 0x7fcdacee6180 14
[uWSGI DEBUG] HTTP_ACCEPT: */*
0x7fcdacee53c0 11 0x7fcdad1e6f80 3
[uWSGI DEBUG] UWSGI_APPID: app1
0x7fcdacee53f0 11 0x7fcdacee5420 4
[uWSGI DEBUG] UWSGI_HOME: /home/gber/venv1
0x7fcdacee5450 10 0x7fcdad1e0eb0 25
[uWSGI DEBUG] called 0x7fcdaee0e500 0x7fcdaee01d40 1
[pid: 31558|app: 0|req: 1/1] 127.0.0.1 () {28 vars in 323 bytes} [Thu Apr  4 13:32:03 2013] GET /foo => generated 28 bytes in 0 msecs (HTTP/1.1 200) 2 headers in 64 bytes (1 switches on core 0)

Adding

virtualenv = /home/gber/venv1

makes it work, though.

[unbit]

just realized that with mount we preload the applications, so dynamic virtualenv are not possibile. I have added the "setfile" intruction to allows dynamic loading of application:

[uwsgi]

http-socket = :9090

route = ^/foo chdir:/tmp
route = ^/foo log:SCRIPT_NAME=${SCRIPT_NAME}
route = ^/foo log:URI=${REQUEST_URI}
route = ^/foo sethome:/var/uwsgi/venv001
route = ^/foo setfile:/var/uwsgi/app001.py
route = ^/foo break:

route = ^/bar chdir:/var
route = ^/bar addvar:SCRIPT_NAME=/bar
route = ^/bar/(.+) addvar:PATH_INFO=/$1
route = ^/bar sethome:/var/uwsgi/venv002
route = ^/bar setfile:/var/uwsgi/app002.py
route = ^/bar break:

route = ^/test setapp:/var/uwsgi/app002.py
route = ^/test addvar:SCRIPT_NAME=/test
route = ^/test/(.+) addvar:PATH_INFO=/$1

in the second application we even modify SCRIPT_NAME and PATH_INFO dynamically (this should allow another series of tricks). In the third application we reference the app loaded in the second one (dynamic apps are mounted under a virtual path named like the full path of the file) but redefining SCRIPT_NAME and PATH_INFO accordingly

[gber]

With the new setfile I'm able to get multiple apps with multiple virtualenvs to work, although differently from your above example.

no-default-app is ignored, using only sethome and setfile as above will result in the first application handling all paths, so adding setapp seems to be required when using more than one app:

[uwsgi]
plugins = 0:notfound,16:python,cgi,router_uwsgi
http-socket = :8000

no-default-app = true

route = ^/app1 sethome:/home/gber/venv1
route = ^/app1 setfile:/home/gber/venv1/app1.py
route = ^/app1 setapp:/home/gber/venv1/app1.py
route = ^/app1 uwsgi:,16,0

route = ^/app2 sethome:/home/gber/venv2
route = ^/app2 setfile:/home/gber/venv2/app2.py
route = ^/app2 setapp:/home/gber/venv2/app2.py
route = ^/app2 uwsgi:,16,0

There are a few oddities with the environment:

• PATH_INFO is set to the full REQUEST_URI, SCRIPT_NAME is always empty
• UWSGI_HOME is set to the path of the python callable, e.g. /home/gber/venv2/app2.py rather than the path set by sethome.

Fortunately they are easy to work around (based on your example above) by adding:

route = ^/app1 addvar:SCRIPT_NAME=/app1
route = ^/app1 addvar:UWSGI_HOME=/home/gber/venv1
route = ^/app1(/.*|)$ addvar:PATH_INFO=$1

route = ^/app2 addvar:SCRIPT_NAME=/app2
route = ^/app2 addvar:UWSGI_HOME=/home/gber/venv2
route = ^/app2(/.*|)$ addvar:PATH_INFO=$1

Can you comment on whether the above is expected/intended?

[unbit]

Yes it is the expected behaviours, the routing subsystem does not change SCRIPT_NAME/PATH_INFO by itself, they must br voluntary rewritten. What is annoying is having to specify setapp and setfile, i do not know it if is a good approach to imply setapp in setfile

[gber]

OK, I think the setapp/setfile is no big deal, just something to be aware of. What about UWSGI_HOME though? Shouldn't it be set to whatever is specified by sethome? I also noticed that I cannot correct it via addvar.

[unbit]

UWSGI_HOME (as well as the others UWSGI_ vars) is parsed before routing happens. They are used by frontend to pass infos to uWSGI. Internally they have no scope (the wsgi_request structure is used). By the way, you mean you cannot set it with:

route-run = addvar:UWSGI_HOME=pippo

?

[gber]

No problem if the UWSGI_HOME isn't relevant, I just noticed it in the environment of the WSGI app. And no, it cannot be set via addvar.

[gber]

Is the inability to set UWSGI_HOME via addvar a bug? If not we can probably close this bug.

[unbit]

No, variables (from the internal routing point of view) are not related to internal behaviours. UWSGI_HOME set wsgi_req->home only at parsing phase (if passed by a webserver) after that phase it lose meaning