StackStorm Single Host Deployment

Disclaimer: This is an un-official, unfinished, Ubuntu-14.04 based guide. Will be [eventually] translated to the proper docs.

StackStorm's single host reference deployment is delivered by All-In-One installer aka AIO.

If you're looking for quick smooth start with StackStorm, go to AWS, create Ubuntu AMI ami-5189a661 and run the installer. If you care to learn what's under the hood, read on.

Here I describe some details of this reference deployment with examples of configurations, and pointers to puppet modules in puppet-st2 and st2workroom that are used to deploy it. Using this as a map of StackStorm's dependencies, components, and configurations, you can reason about the setup and configuration, troubleshoot problems, contribute patches, or reverse engineer installer for DYI deployment, either using StackStorm Puppet modules directly or as a reference points. NOTE: this description is for Ubuntu 14.04; RedHat deployment is similar but OS specific details obviously differ; puppet pointers handle both systems so you could figure it out.

Big Picture

TODO: short word description

External dependencies:

• RabbitMQ (AIO installs uses as a docker image)
• MongoDB 2.4 (AIO installs as a docker image)
• gunicorn or uwsgi
• PostgreSQL
• nodejs, npm, hubot (optional, required for Chatops)
• Reddis or Zookeeper (optional, required for Policies)
• LDAP (optional, for LDAP based authentication)

Frontent

1. nginx provides SSL termination, redirects HTTP to HTTPS, serves WebUI as static HTML, and proxies REST API endpoints to st2* web services. Configurations /etc/nginx/, or see puppet

   CORS: Currently, api and auth endpoints are served off port 9100 and 9101. As these endpoints are accessed from a browser, CORS need to be properly handled on nginx. You can see quite a bit of settings to handle brower's pre-flight OPTION requests. Coming soon: we are moving to serving everything off a single HTTPS port.

   SSL certificate is at /etc/ssl/st2/ (st2.crt, st2.key). The certificate commonname or alt names shall match the external address used to access StackStorm on the box.

2. StackStorm WebUI (st2web and flow) is served as static HTML. Nginx configuration at /etc/nginx/sites-enabled/st2webui.conf. Static content is under /opt/stackstorm/static/webui/. The config.js there defines the API endponts URL according to rfc1808, as well as the relative path to flow.

3. gunicorn and uwsgi are used to run st2 web services. Installed via pip, globally. Hint: use gunicorn - we are migrating away from uwsgi.

   • gunicorn: runs st2api, will soon be used to run st2install, st2auth, and mistral api. Upstart script template for st2* services - see puppet
   • uwsgi: still used to run st2auth. Upstart script template for uwsgi - see puppet

st2* components

• st2* are python st2 service components that comprise main StackStorm functionality. They are installed from .deb or .rpm packages at https://downloads.stackstorm.net. Currently the packages do not include python pip dependencies. Therefore, python pip requirements must be installed from pip during deployment. (example here).
  • stackstorm-st2 on PuppetForge.
  • Puppet code under StackStorm/puppet-st2.
• Most st2* components are connected to RabbitMQ and MongoDB; the connections are configured in /etc/st2.conf (see docs).
• Other common configurations in /etc/st2.conf. Note that some parameters are used in "debug" mode and not relevant when run under nginx/gunicorn.
• upstart scripts are in /etc/init - (see puppet)
• st2ctl is a convenience command to operate StackStorm.
  • NOTE: when you run st2ctl status and see st2web is not running - don't worry: it's not used in the ref deployment as WebUI and Flow are served by nginx.

1. st2installer

   st2installer is a small Pecan app that serves a graphical setup for All-In-One installer over HTTPS. Only used by all-in-one installer.

2. st2auth

st2auth is an authentication Web service. It is a Pecan app, running behind nginx on port 9100, with gunicorn (recommended, will update soon) or uwsgi (current). * nginx configuration in /etc/nginx/sites-enabled/st2auth.conf and /etc/nginx/uwsgi_params * uwsgi upstart script /etc/init/st2auth.conf - (puppet template) * the configuration is in [auth] section of /etc/st2/st2.conf. Note that use_ssl=False is because nginx is doing SSL termination, so st2auth doesn't have to. * Reference deployment uses PAM auth backend; st2-auth-backend-pam needs to be installed from downloads.stackstorm.net/st2community (puppet); * With PAM auth backend, any local user with a password can access StackStorm. However, if RBAC is enabled, you'll see little. To give user an access to StackStorm, go to /opt/stackstorm/rbac/assigments and create an assignment for your newly created user, than apply it by running st2-apply-rbac-definitions command - see details in docs * to set up a different authentication backends, follow docs

3. st2api

st2api is the REST API Web service, also serves webhooks for webhook triggers. It's a pecan app running behind nginx on 9101 via gunicorn. * nginx configuration in /etc/nginx/sites-enabled/st2api.conf Note that we are switching to proxying it over HTTPS:443 soon. * gunicorn upstart script /etc/init/st2api.conf - (puppet template)

4. st2sensorcontainer

st2sensorcontainer runs sensor plugins from /opt/stackstorm/packs. Sensors from the packs need to be registered; this is done when pack is installed by packs.install, or manually manually by run st2ctl reload --register-sensors. Deployed with st2reactor package.

5. st2rulesengine

This picks triggers from the RabbitMQ, matches them against the active Rules, and sends an action requests to RabbitMQ. Deployed with st2reactor package.

6. st2actionrunner

st2actionrunner runs action plugins from /opt/stackstorm/packs via a variety of runners. Deployed with st2actions package. Different runners relies on different configuration settings to operate. * Running 4 st2actionrunner processes per core is a good empirics. With Ubuntu upstart: number of st2actionrunner-worker* is created in /etc/init/ which are all controlled by /etc/init/st2actionrunner.conf - (see puppet). * Actions based on remote-shell-runner and remote-command-runner require proper SSH setup per docs; [system_user], [ssh_runner] in , * Mistral runner needs access to Mistral API endpoint (see docs for more options) * Python runner creates a virtualenv environment per pack under /opt/stackstorm/virtualenvs; it is handled by packs.install; set it up manually with packs.setup_virtualenv when writing your custom pack. * If policies used, a coordination service is needed. Redis is recommended for for single-box deployment, or use an external Redis or Zookeepr.

7. st2resultstracker

A service for tracking long-running workflow executions. Deployed as part of st2actions package. Calls Mistral API endpoint.

8. st2notifier

A service for providing notifications that fires core.st2.generic.notifytrigger on action completions. Deployed as part of st2actions package.

9. st2client

CLI client. Works against the st2 API, installed on the box for convenience, can be installed remotely.Obviously requires access to api and auth endpoints. Details in the docs. * ST2_API_URL and ST2_AUTH_URL env variables used to specify api and auth endpoints. AIO installer sets them up in /etc/profiles.d/st2.sh. * * ~/.st2/conf can be used to set up default URL, silence SSL warnings for self-signed certificates, overrides of api and auth endpoints, keeping credentials if desired, etc.

Mistral

Mistral workflow service is running workflows. It consists of the three components: api, executor, and worker, as well as st2mistral - a proxy plugin that executor uses to call StackStorm actions. Here is how it works:

1. mistral runner from st2actionrunner requests mistral api to create a workflow and start a workflow execution.
2. api calls engine to start workflow execution
3. engine computes the next tasks and passes it to an executor to call appropriate actions.
4. executor passes all requests to StackStorm actions to st2mistral proxy action.
5. st2mistral action calls st2api to run an action
6. st2resultstracker queries mistral api for workflow execution progress.

• Configurations are in /etc/mistral/mistral.conf. Mistral uses RabbitMQ for communication between components, and PostgreSQL DB to keep the state of the workflow.
• Deployed to /opt/openstack/mistral from StackStorm's fork on github.
• runs with it's own .venv virtualenv with it's own set of pip dependencies, per requirements.txt. st2mistral must be also installed into the same virtualenv. See puppet for details.
• By default, Mistral api is running via SimpleHTTPservice. For any serious workload, separate mistral-api under gunicorn; use this sample upstart script mistral-api.conf Coming soon: api under gunicorn will be the default.
• Mistral CLI client is also installed to help debugging, try mistral --help.

Chatops

• Preferably, get stackstorm/hubot docker with all pre-installed, see puppet.
• Alternatively, follow docs or see puppet:
  • install hubot and hubot-stackstorm - requires nodjs, npm, installed to /opt/stackstorm/hubot/hubot
  • upstart script /etc/init.d/hubot
  • hubot configurations under /opt/hubot/hubot/hubot.env

TODO: add Chatops wiring details