Improve documentation on running gunicorn from Python with custom applications

[duffsterlp]

I'm running gunicorn using the falcon web framework and would like to run gunicorn from the Python interpreter directly, similar to this issue here:
#91

I now know how to accomplish this from a combination of these links as well as a conversation in the IRC room.
http://docs.gunicorn.org/en/stable/custom.html
https://gist.github.com/quamilek/4fd1f246feb6149dd1dd/

The latter link really shed light that the "Custom Application" section of the documentation was the appropriate place to connect Gunicorn with existing WSGI frameworks. The Gunicorn documentation alone did not provide ample indication of this, in my opinion.

I'm looking for the following improvements in the Custom Application page:

1. Something that explicitly indicates how to run Gunicorn from Python using existing WSGI frameworks. The example only indicates how to connect a custom application that implements the WSGI specification itself.
2. Some more documentation on the Gunicorn app base class. The "Custom Application" section provides an example of how to use this base class, but there should be an API that indicates how to use the app base class. Without this API, users are forced to look at the source code to infer how to use it. For example, there is nothing in the documentation that says that the load_config() and load() functions need to be overridden.

[benoitc]

maybe @matrixise can help there since he's supposed to maintain that feature?

[berkerpeksag]

For 1), perhaps we can provide a factory function that returns an application class?

We can use 2) as an opportunity to refactor the API (if needed) and document it.

[benoitc]

@duffsterlp out of curiosity, what's the reaon to not simply use the gunicorn command line?

[aldanor]

@duffsterlp out of curiosity, what's the reaon to not simply use the gunicorn command line?

For example, you have a CLI which you need to pass some parameters to in order to start the server.

[jakethedev]

Bringing this back from the dead, I've been on a bit of a witch hunt, and I think this would solve my issue.

Context: We're a microservice type company, and we have virtual machines that act as containers, but container management software can't be adopted in full at $work yet. To cut a huge amount of complexity from $configmanager and $deployer, we're taking the PEX deployable route for python web services, and these apps generally exist as Flask services around here, though Django would be a nice to have.

Unfortunately, PEX is... difficult... to deal with in wsgi-land. You can't set up wsgi to serve a PEX, or add a PEX to a path, or any of the normal pythonic goodness, but given that Gunicorn is in python, it seems to stand to reason that one could instantiate a Flask app and pass that to Gunicorn in a python context. As PEX is just an executable zip file, it requires a single entry point that should take over and run the application - and it seems like a reasonable thing to set up gunicorn within a PEX file, but I'm not seeing any information on writing an entry point script that extends Gunicorn and runs a Flask application (or other wsgi compatible app that isn't totally homebrewed).

All that outta the way - is there information on writing a python script that effectively runs Gunicorn(FlaskApp)? Perhaps someone here has a gist or other example of how to boilerplate this?

Side note: I'm not seeing much on various forums about PEX usage, but long story short for those who haven't messed with it, it's Twitter's Python equivalent to static Go binaries and Java's uberjars. It wraps up a virtualenv, zips it all up as a single executable file, and allows you to deploy an app and run it without significant fuss, which allows dependency management at build time and avoids the need for installing/maintaining virtualenvs

[jakethedev]

Nevermind that last question, I still think it warrants a bit more TLC to the docs regarding running wsgi-compatible apps from a pure python context but I made a bit of headway and can move forward. Perhaps I spent too long reading in circles around it, but I never found it in the docs that I could use this as a 'run.py':

#!/usr/bin/env python
from gunicorn.app.wsgiapp import WSGIApplication
WSGIApplication("%(prog)s [OPTIONS] [APP_MODULE]").run()

...which then can be used like "run.py flaskOrWhatev:entry_point". Pertaining to PEX, in case anyone comes across this, you can set the entry point to the PEX file as 'run' in this example, compile the PEX, and run 'myapp.pex flaskOrWhatev:entry_point' - plus any other appropriate cmd options - and it works like a portable gunicorn charm.

I had to search the source repo for use of "APP_MODULE" to figure this out though 😐 Assuming that's an appropriate usage of the WSGIApplication class, is there somewhere in particular that bit of information might fit best in the docs? I'd be happy to PR it. Also, is it worth creating an issue for optionally setting APP_MODULE via env variable or config file, or would that not be considered for implementation for some reason?

[joenap]

@jakethedev I also ran into this with pex files. Your solution was everything I needed! Thanks for posting this.

[jakethedev]

Glad it helped!
After a bit of discussion on the docs case linked above, there's a nicer way to do the same thing - let's say your WSGI is at yourpackage:appinstance. If you can run the app from a virtualenv, you can just call gunicorn.app.wsgiapp yourpackage:appinstance directly from the shell, or in a python context, just import gunicorn.app.wsgiapp and run it in a separate main script:

import gunicorn.app.wsgiapp as wsgi
wsgi.run()

and pass your app identifier (yourpackage:appinstance) as a param, and you should be good to go

[jakethedev]

Well, I haven't felt this dumb since calculus, but I've learned quite a bit since this last comment...

To everyone who cares - you literally just run gunicorn myapp:appinstance. That's what benoitc meant in 2017 by the gunicorn command line. My May 16th comment is still relevant if you're running python that should then run Gunicorn, but there's simpler ways. For those dealing with PEX: check out this PEX issue, the TLDR is that you build pex like pex -c gunicorn -o app.pex, then you invoke it like ./app.pex mymodule:appinstance [-c optional_gun_config.py].

Once gunicorn is installed as a dep or on the system, the entry_points it declares in setup.py install the CLI shortcut gunicorn as a clean and maintained way to invoke the gunicorn.app.wsgiapp code. The way entry_points works isn't documented in any of the setuptools or official python docs, but it hit me like a brick reading the python packaging docs.

Anyways. Hope this clears things up for anyone that hits this thread and maybe hasn't done significant work with setup.py files, or is building out python support in their CI with no python dev knowledge like I am 😅

[benoitc]

closing as answered. thanks for the thread and PR.