Tinman adds a little more stack to Tornado. It is designed to speed development of Tornado applications. It includes an application wrapper and a toolbox of decorators and utilities.
The configuration file syntax has changed and the structure of the package has changed. Please test your applications if you are upgrading from 0.4 or previous.
- A full featured application wrapper
- Standard configuration for applications
- Built-in configurable session management
- Lightweight model system for NoSQL systems
- Authentication Mixins for Github and StackExchange
- A command-line tool, tinman-init that will create a skeleton app structure with the initial package and setup.py file.
- Network address whitelisting decorator
- Method/Function debug logging decorator
- Handlers with automated connection setup for PostgreSQL, RabbitMQ and Redis
- Support for a External Template Loaders including Tinman's CouchDB Template Loader
- Flexible logging configuration allowing for custom formatters, filters handlers and setting logging levels for individual packages.
- Built in support for NewRelic's Python agent library
- RequestHandler output caching/memoization
Install via pip or easy_install:
pip install tinman
- tinman
- application: Application extends tornado.web.Application, handling the auto-loading of configuration for routes, logging, translations, etc.
- auth: Authentication Mixins for GitHub, StackExchange, and HTTP Digest Authentication.
- controller: Core tinman application controller.
- couchdb: A CouchDB based template loader module
- decorators: Authentication, memoization and whitelisting decorators.
- exceptions: Tinman specific exceptions
- handlers: Request handlers which may be used as the base handler or mix-ins.
- base: Base request handlers including the SessionRequestHandler
- mixins: Request Handlers mixins including support for Redis, RabbitMQ and Model API Request Handlers
- model: Model system with base model class and various base model classes for supported storage backends.
- process: Invoked by the controller, each Tinman process is tied to a specific HTTP server port.
- session: Session object and storage mixins
- utilities: Command line utilities
- helper
- ipaddr
- pyyaml
- Heapy: guppy,
- LDAP: python-ldap,
- MsgPack Sessions: msgpack,
- NewRelic: newrelic>=1.12.0',
- PostgreSQL: psycopg2,
- RabbitMQ: pika>=0.9.13,
- Redis: tornado-redis,
- Redis Sessions: redis
Use pip to install dependencies:
pip install 'tinman[Dependency Name]'
For example:
pip install 'tinman[RabbitMQ]'
The tinman application runner works off a YAML configuration file format and provides a convenient interface for running tornado applications interactively or as a daemon.
Command Line Syntax:
Usage: usage: tinman -c <configfile> [options]
Tinman adds a little more stack to Tornado
Options:
-h, --help show this help message and exit
-c CONFIGURATION, --config=CONFIGURATION
Path to the configuration file
-f, --foreground Run interactively in console
-p PATH, --path=PATH Path to prepend to the Python system path
Sessions will automatically load on prepare and save on finish. If you extend the SessionHandler and need to use prepare or finish, make sure to call super(YourClass, self).prepare() and super(YourClass, self).on_finish() in your extended methods. By using the session mixins you can change the default session behavior to use different types of storage backends and serializers.
from tinman.handlers import session
from tornado import web
class Handler(session.SessionHandler):
@web.asynchronous
def get(self, *args, **kwargs):
# Set a session attribute
self._session.username = 'foo'
self._session.your_variable = 'bar'
self.write({'session_id': self._session.id,
'your_variable': self._session.your_variable})
self.finish()
def prepare(self):
super(Handler, self).on_finished()
# Do other stuff here
def prepare(self):
super(Handler, self).prepare()
# Do other stuff here
The Heapy handler uses the guppy library to inspect the memory stack of your running Tinman application, providing a JSON document back with the results. It is very slow and blocking and can take many MINUTES to complete so it should be used very sparingly and if used on a production application, with the whitelist decorator.
To use the Heapy handler, just add the route to your configuration:
- [/heapy, tinman.handlers.heapy.HeapyRequestHandler]
The following is a very abbreviated repport:
{
"rows": [
{
"count": {
"percent": 40,
"value": 45068
},
"cumulative": {
"percent": 29,
"value": 4159016
},
"item": "types.CodeType",
"referrers": {
"rows": [
{
"count": {
"percent": 96,
"value": 7290
},
"cumulative": {
"percent": 96,
"value": 874800
},
"item": "function",
"size": {
"percent": 96,
"value": 874800
}
}
],
"title": "Referrers by Kind (class / dict of class)",
"total_bytes": 911160,
"total_objects": 7593
},
"size": {
"percent": 29,
"value": 4159016
}
}
],
"title": "Referrers by Kind (class / dict of class)",
"total_bytes": 14584240,
"total_objects": 113444
}
The following are the keys that are available to be used for your Tinman/Tornado application.
- cookie_secret: A salt for signing cookies when using secure cookies
- debug: Toggle tornado.Application's debug mode
- login_url: Login URL when using Tornado's @authenticated decorator
- newrelic_ini: Path to newrelic Python .ini file for enabling newrelic support
- paths:
- base: The root of the files for the application
- static: The path to static files
- templates: The path to template files
- translations: The path to translation files
- redis: If using tinman.handlers.redis.RedisRequestHandler to auto-connect to redis.
- host: The redis server IP address
- port: The port number
- db: The database number
- rabbitmq:
- host: the hostname
- port: the server port
- virtual_host: the virtual host
- username: the username
- password: the password
- session: Configuration if using tinman.handlers.session.SessionRequestHandler
- adapter:
- class: The classname for the adapter. One of FileSessionAdapter, RedisSessionAdapter
- configuration: SessionAdapter specific configuration
- cookie:
- name: The cookie name for the session ID
- duration: The duration in seconds for the session lifetime
- adapter:
- template_loader: The python module.Class to override the default template loader with
- transforms: A list of transformation objects to add to the application in module.Class format
- ui_modules: Module for the UI modules classes, can be a single module, a mapping of modules (dict) or a list of modules.
- xsrf_cookies: Enable xsrf_cookie mode for forms
- whitelist: List of IP addresses in CIDR notation if whitelist decorator is to be used
The tinman-init script will create a skeleton Tinman directory structure for your project and create a setup.py that will put the static and template files in /usr/share/.
If you are not going to install your app as a python package, you should set a base_path so that tinman knows what directory to insert into the Python path to be able to load your request handlers and such.
Configure the tornado.httpserver.HTTPServer with the following options:
- no_keep_alive: Enable/Disable keep-alives
- ports: Ports to listen to, one process per port will be spawned
- ssl_options: SSL Options to pass to the HTTP Server
- certfile: Path to the certificate file
- keyfile: Path to the keyfile
- cert_reqs: Certificicate required?
- ca_certs: One of none, optional or required
- xheaders: Enable X-Header support in tornado.httpserver.HTTPServer
Logging uses the dictConfig format as specified at
http://docs.python.org/library/logging.config.html#dictionary-schema-details
But is able to do a minimal logging config thanks to clihelper defaults. The following is the minimal logging configuration required:
Logging:
tinman:
handlers: [console]
propagate: True
formatter: verbose
level: DEBUG
tornado:
handlers: [console]
propagate: True
formatter: verbose
level: INFO
The route list is specified using the top-level Routes keyword. Routes consist of a list of individual route items that may consist of mulitiple items in a route tuple.
The traditional route tuple, as expected by Tornado is a two or three item tuple that includes the route to match on, the python module specified Class to use to respond to requests on that route and an optional route settings dictionary.
Routes:
- [/, myapp.Homepage]
- [/images, tornado.web.StaticFileHandler, {'path': '/var/www/user_images'}]
In order to facilitate complex regex that does not break YAML files, Tinman supports a "re" flag in a Route tuple. If you find that your regex is breaking your route definition, insert the string "re" before the route regex.
Routes:
-
- re
- /(c[a-f0-9]f[a-f0-9]{1,3}-[a-f0-9]{8}).gif
- test.example.Pixel
The TemplateLoader configuration option is detailed the External Template Loading section of the document.
The following is a "full" example tinman application configuration:
%YAML 1.2
---
Application:
path:
base: /home/foo/mywebsite
debug: True
xsrf_cookies: False
wake_interval: 60
# Any other vaidate Tornado application setting item
Daemon:
pidfile: /tmp/myapp.pid
user: www-data
group: www-data
HTTPServer:
no_keep_alive: False
ports: [8000,8001]
xheaders: True
Logging:
version: 1
formatters:
verbose:
format: '%(levelname) -10s %(asctime)s %(process)-6d %(processName) -20s %(name) -20s %(funcName) -25s: %(message)s'
datefmt: '%Y-%m-%d %H:%M:%S'
syslog:
format: ' %(levelname)s <PID %(process)d:%(processName)s> %(name)s.%(funcName)s(): %(message)s'
handlers:
console:
class: logging.StreamHandler
debug_only: True
formatter: verbose
error:
filename: /Users/gmr/Source/Tinman/logs/error.log
class: logging.handlers.RotatingFileHandler
maxBytes: 104857600
backupCount: 6
formatter: verbose
file:
filename: /Users/gmr/Source/Tinman/logs/tinman.log
class: logging.handlers.RotatingFileHandler
maxBytes: 104857600
backupCount: 6
formatter: verbose
syslog:
class: logging.handlers.SysLogHandler
facility: local6
address: /var/run/syslog
formatter: syslog
loggers:
clihelper:
handlers: [console]
level: DEBUG
propagate: True
formatter: verbose
tinman:
handlers: [console, file]
propagate: True
formatter: verbose
level: DEBUG
tornado:
handlers: [console, file]
propagate: True
formatter: verbose
level: INFO
root:
handlers: [error]
formatter: verbose
level: ERROR
disable_existing_loggers: True
Routes:
-[/, test.example.Home]
-
# /c1f1-7c5d9e0f.gif
- re
- /(c[a-f0-9]f[a-f0-9]{1,3}-[a-f0-9]{8}).gif
- test.example.Pixel
-
- .*
- tornado.web.RedirectHandler
- {"url": "http://www.github.com/gmr/tinman"}
The tinman application runner has a built in test application. To see if the module is setup correctly simply run:
tinman -f
In your console you should see output similar to:
Configuration not specified, running Tinman Test Application
utils # 247 INFO 2011-06-11 23:25:26,164 Log level set to 10
cli # 145 INFO 2011-06-11 23:25:26,164 Starting Tinman v0.2.1 process for port 8000
cli # 154 DEBUG 2011-06-11 23:25:26,169 All children spawned
application # 106 DEBUG 2011-06-11 23:25:26,170 Initializing route: / with tinman.test.DefaultHandler
application # 36 INFO 2011-06-11 23:25:26,171 Appending handler: ('/', <class 'tinman.test.DefaultHandler'>)
cli # 171 INFO 2011-06-11 23:25:26,174 Starting Tornado v1.2.1 HTTPServer on port 8000
web # 1235 INFO 2011-06-11 23:25:32,782 200 GET / (127.0.0.1) 1.24ms
You should now be able to access a test webpage on port 8000. CTRL-C will exit.
Tinman has decorators to make web development easier.
Vaidates the requesting IP address against a list of ip address blocks specified in Application.settings
from tinman.decorators import whitelist
from tornado import web
# Define the whitelist as part of your application settings
settings['whitelist'] = ['10.0.0.0/8',
'192.168.1.0/24',
'1.2.3.4/32']
application = Application(routes, **settings)
# Specify the decorator in each method of your RequestHandler class
# where you'd like to enforce the whitelist
class MyClass(web.RequestHandler):
@whitelist.Whitelisted
def get(self):
self.write("IP was whitelisted")
In addition you may add the whitelist right into the configuration file:
Application:
whitelist:
- 10.0.0.0/8
- 192.168.1.0/24
- 1.2.3.4/32
A local in-memory cache decorator. RequestHandler class method calls are cached by name and arguments. Note that this monkey-patches the RequestHandler class on execution and will cache the total output created including all of the template rendering if there is anything. Local cache can be flushed with _tinman.decorators.memoize.flush()
from tornado import web
from tinman.decorators import whitelist
class MyClass(web.RequestHandler):
@tinman.memoize
def get(self, content_id):
self.write("Hello, World")
Tinman includes tinman.loaders.couchdb.CouchDBLoader to enable the storage of templates in CouchDB to provide a common stemplate storage mechanism across multiple servers.
Templates stored in CouchDB are stored with the full path as the document key and the template value is stored in the document using the key "template"
When storing templates in CouchDB it is important that the forward-slash (/) is replaced with a colon (:) as the key value in CouchDB, or it will not find the stored file.
For example a template with a filesystem path of /foo/bar/template.html would be stored with a key of foo:bar:template.html in CouchDB but still referenced as /foo/bar/template.html everywhere else.
{
"_id": "base.html",
"_rev": "1-18d104181a15f617a929c221d01423da",
"template": "<html>\n <head>\n <title>{% block \"title\" %}Title{% end %}</title>\n </head>\n <body>\n <h1>Hello, World!</h1>\n </body>\n</html>"
},
{
"_id": "pages:home.html",
"_rev": "2-c3c06f5a6d6a7b8149e0a700c67aeb41",
"template": "{% extends \"base.html\" %} \n{% block title %}Homepage{% end %}"
}