Containerized Deployment

[christiansmith]

A major goal for Anvil Connect has always been to make it very fast and dead simple for a new user to get an auth server up and running. Ideally, all it would take to launch a new instance into production is a single command. With recent Docker-related contributions from @tomkersten and @oren we're much closer to that goal (thanks guys!). There's still a great deal of work to be done. As usual, the devil's in the details.

Prebuilt Images

Official images for nginx and Redis are available that spare us from having to roll our own. However, as @oren has demonstrated, we can obtain much (MUCH!) smaller images by starting with Alpine Linux. This distro is "designed for power users who appreciate security, simplicity and resource efficiency". Oren's anvil-connect-docker repo uses Alpine for all the images.

We can go a step further by pushing officially supported images for these dependencies to Docker Hub, saving users the time of building them. Aside from building on Alpine base images, we can make additional improvements.

Redis

First, Redis needs to be configured properly for durability. The config file is something we may want to build right into the image to discourage misuse in the context of Connect. The official Redis Dockerfiles have a few extra features we ought to incorporate, like the use of gosu. Redis containers should also expose two volumes: one for data and another for logs.

nginx

nginx performs several tasks on behalf of Connect:

• SSL termination
• Load balancing
• Adding proxy headers for secure cookies
• Serving static assets (public directory)

To support these functions, along with scaling the number of Connect containers, it may be best to use read-only volumes for getting config files, SSL certificates, and static assets inside the container. This would make it easy to update upstream entries for the load balancer, rotate certs, etc, without having to rebuild the nginx image. Restarting the containers would be sufficient.

Connect

When it comes to Connect, it's an open question whether we can and should provide a prebuilt image. Using volumes we can load most of the what users might want to customize. Perhaps it's possible to base a custom image on a stock image? Let's see how far we can push it.

Changes to Connect

There are also some changes we can make to Connect itself to simplify initial setup and deployment. First, it would be helpful to minimize the need to edit Connect's config file by making providers and OIDC settings part of the HTTP API. Secondly, nv migrate could possibly be deprecated. This command is currently only useful when initializing a fresh Redis database. We could build a check into the Connect boot process that ensures required values are present.

Project organization

When a user generates a new project with nv init, we want to provide everything needed to get them into production fast. Now that we're going to ship support for Redis and nginx containers, there's a question of how to best organize the project directory. There are (at least) two options to consider:

Option 1: Container based organization

This option is straightforward from an ops perspective:

├── docker-compose.yaml
├── connect
│   ├── config
│   ├── logs
│   ├── public
│   └── views
├── nginx
│   ├── certs
│   ├── conf.d
│   ├── logs
│   └── sites-available
└── redis
    ├── data
    └── logs

Option 2: Connect based organization

This option is close to what we have now and perhaps more convenient for someone planning to customize Connect (and run it locally without docker).

├── docker-compose.yaml
├── config
│   ├── connect
│   ├── nginx
│   └── redis
├── data
│   └── redis
├── logs
│   ├── connect
│   ├── nginx
│   └── redis
├── public
└── views

Feedback

Please join today's hangout at 9AM Pacific time to discuss, or comment here if you can't make it. Thanks in advance!

[christiansmith]

Consensus in the hangout seemed to be "Option 1" for project organization.

[bnb]

CLI: Add a feature to the CLI to generate the/a docker script that is cross platform to initialize the docker container.

Hope I got that right.

[christiansmith]

@bnb the issue is that we can generate all the config files needed by nginx, connect, and redis from the Nodejs CLI (nv init), rather than from bash scripts that may not run in other shells and operating systems.

[christiansmith]

With 18eb3d8, nv init now generates a containerized deployment setup very similar to @oren's repo. Still a couple things to do before this can make it into a release.

• prompt the user for various config properties and render files from templates
• generate new SSL self-signed certs for getting started (eventually use letsencrypt.org?)
• initialize the redis database
• verify redis data and logs volumes are working as expected
• verify Connect views volume is working as expected
• audit for security/best practices
• try to get the Connect image size a bit smaller if we can. (is it npm dependencies causing the bloat?)
• ...

Love to get some feedback from anyone who wants to try this out as I continue working. All it takes is ...

$ mkdir PATH
$ ~/PATH/TO/SRC/bin/nv init
$ docker-compose up -d

Thanks!

[topperge]

I know we discussed this at last week's hangout, but the more I think about it the more I think this should be a separate repo that pulls from the core connect repo. If I want to deploy to AWS (EC2 or EC2 Container), VirtualBox, etc. I'm stuck pulling this apart and building on my own. I know a lot of orgs that don't trust the docker container model yet. By baking this in we're making big assumptions about an org's deployment environment.

In other environments we've used vagrant to define our environments and then create different playbooks for different environments at different scale.

[christiansmith]

@topperge that's a really good point you bring up. We don't want to discourage people from using Connect by forcing technology choices on them (any more than we have to). One thing I've considered is having init generate the project from different "profiles", depending on where and how you might want to deploy it.

I can imagine shipping support for a variety of scenarios, including AMIs, Ansible playbooks, etc. We definitely want to have at least one completely secured, turn-key option. The danger of providing more choices is that we end up spreading ourselves very thin with the burden of maintenance.

For now, I do want to push the envelope with the Docker setup and see how close we can get to a one command deployment. This looks to be the easiest way to support multiple infrastructure environments from the cloud to the corporate data center. We're not too far away from this right now. A few more commits should wrap it up.

Perhaps the best way to do this for the time being is to have init generate a "vanilla" auth server and make another command to generate the Docker version. nv init --docker?

[christiansmith]

I'm thinking this might be a good time to start working earnestly on the new independent CLI. I was planning to make this a quick update to nv init. But if we're going to start conditionally generating Docker vs non-Docker projects, I'd rather just write that once and get it over with. That would also motivate us to make some other improvements to the API to facilitate remote administration. Anyone want to jump in and pair on it?

[christiansmith]

With recent commits, the new CLI implements nv init. The command prompts users for configuration details and deployment options and generates a project tree based on those choices.

NOTE: At the moment, this code is not published to npm. To run it you'll need to clone the repo and invoke ~/PATH/TO/REPO/bin/nv init directly. Do this from a separate, empty directory.

$ ~/GitHub/anvilresearch/connect-cli/bin/nv init
? What (sub)domain will you use? laptop-connect.anvil.io
? Would you like to use Docker? Yes
? Would you like to run Redis? Yes
? Would you like to run nginx? Yes
? Would you like to create a self-signed SSL cert? Yes
? Country Name (2 letter code) US
? State or Province Name (full name) South Dakota
? Locality Name (eg, city) Rapid City
? Organization Name (eg, company) Internet Widgits Pty Ltd
...

Assuming the default choices, all it takes to run a (almost) production ready server after initializing the project is:

$ docker-compose up -d

We'd love to have some folks try this out and give us feedback. There's plenty of opportunity to add support for different environments like Vagrant. If you have ideas or specific requirements, please comment here.

Thanks everyone!

[christiansmith]

Update. The initialization feature now lives in the new CLI repo. It has been released and you can try it out by installing that package globally with npm:

$ npm install -g anvil-connect-cli

The command is named nvl to avoid clashing with the existing nv command that ships with the server package. Run nvl init in a new directory and follow the prompts to see how it works.

$ nvl init
? What would you like to name your Connect instance? myconnect
? What (sub)domain will you use? myconnect.example.com
? Would you like to use Docker? Yes
? Would you like to run Redis? Yes
? Would you like to run nginx? Yes
? Would you like to create a self-signed SSL cert? Yes
? Country Name (2 letter code) US
? State or Province Name (full name) South Dakota
? Locality Name (eg, city) Rapid City
? Organization Name (eg, company) Anvil Research, Inc.

The command will generate all the files you need to customize and run Anvil Connect, with or without Docker, Redis and nginx, depending on your prompt selections.

Initialized empty Git repository in /Users/smith/Code/2015-08-11-nvl/.git/
Generating RSA private key, 4096 bit long modulus
.........................++
.............................................................................++
e is 65537 (0x10001)
writing RSA key
Generating a 4096 bit RSA private key
........................................++
.....................................................................................++
writing new private key to '/Users/smith/Code/2015-08-11-nvl/nginx/certs/nginx.key'
-----
   create .gitignore
   create README.md
   create docker-compose.yml
   create connect/public/images/anvil.svg
   create connect/public/javascript/session.js
   create connect/public/stylesheets/app.css
   create connect/public/stylesheets/providers.css
   create connect/views/authorize.jade
   create connect/views/session.jade
   create connect/views/signin.jade
   create connect/views/signup.jade
   create connect/server.js
   create connect/config/production.json
   create connect/package.json
   create connect/.bowerrc
   create connect/bower.json
   create connect/Dockerfile
   create nginx/nginx.conf
   create nginx/conf.d/default.conf
   create nginx/conf.d/upstream.conf
   create nginx/Dockerfile
   create redis/etc/redis.conf
   create redis/Dockerfile
Connect all the things :)

A README is generated for the project containing instructions on how to run the server with Docker Compose. We need to expand on this with alternative instructions for running without Docker, using a different SSL front-end, and using hosted Redis elsewhere.

I'm also thinking to ship support for Vagrant, Ansible, and deployment in environments like AWS, DigitalOcean, and Google Cloud Platform.

I'm closing this issue since it seems to be more or less solved in terms of the big picture. Please open new ones for specific enhancements.