docker-compose addons

A set of command line tools to supplement the features already available in docker-compose. These tools generally focus on development or testing environment use-cases.

Contents

• Install
• dcao-include
  • Use Cases
  • Working with Includes
  • Example
  • Usage
• dcao-namespace
  • Example
  • Usage
• dcao-merge
  • Use Cases
  • Usage

Install

Currently the only install option is pip

pip install compose-addons

dcao-include

Given a docker-compose.yml file, fetch each configuration in the include section and merge it into a base docker-compose.yml. If any of the included files have include sections continue to fetch and merge each of them until there are no more files to include.

Use Cases

• If you have a service-oriented architecture, where each of your services is developed and deployed in a separate code repo, and each has its own docker-compose.yml. When you want to create a full testing or development environment for an individual service, you need to include all the downstream services. Instead of duplicating the topology of each downstream service, you can include the docker-compose.yml from the downstream service. Including (instead of duplicating) this topology allows you to change dependencies in a single place without worrying about breaking the test suite of dependent services.
• If the scope of your composition can change based on the task you're performing. Your application might have a "core" set of services that are always run, and some adhoc, or administrative services that are only run sometimes. You can split your composition into two (or more) files. The core docker-compose.yml would only contain the core services. The compose-admin.yml would include the docker-compose.yml and add extra services which could link to or use volumes from the core services, without having to duplicate any of the service configuration.
• If your composition varies by environment (dev vs prod). Similar to the case above, the core docker-compose.yml would remain the same for all environments, but docker-compose-dev.yml could include the "core" services, and add additional service, like database or proxies.

Working with Includes

dcao-include works with a configuration that is different from the docker-compose config in a few ways:

• an optional top level include key, which contains a list of urls (which may be local file paths, http(s) urls, or s3 paths)
• a required top level namespace key, which is used by a config to link to services in an included file. For example, if a config includes http://example.com/compositions/servicea.yaml which has a namespace of servicea, all "public" services in servicea.yaml should start with servicea..
• since configuration can be included from a remote url, or different directories, the configuration should not include anything that depends on the host. There should be no build keys in any service, and no host volumes.

Example

An example composition file with includes:

include:
    - http://example.com/compositions/servicea.yaml
    - http://example.com/compositions/serviceb.yaml

namespace: core

web:
    image: example/service_a:latest
    links: ['servicea.web', 'serviceb.api']

servicea.yaml might look something like this

namespace: servicea

servicea.web:
    image: services/a:latest

serviceb.yaml might look something like this

namespace: serviceb

serviceb.api:
    image: services/b:latest

Usage

To use dcao-include with docker-compose you have a couple options:

Use it with a pipe to stdin:

dcao-include compose-with-includes.yml | docker-compose -f - up -d

Use it once to generate a new file:

dcao-include -o docker-compose.yml compose-with-includes.yml
docker-compose up -d
docker-compose ps

dcao-namespace

Given a standard docker-compose.yml file, add a namespace key, and prefix all instances of service names with that namespace. This command is used to prepare a standard docker-compose.yml file for being used as an include by dcao-include. This could be considered the "export" step required before a compose file can be included by another project.

Example

Given a docker-compose.yml:

web:
    image: example.com/web:latest
    links: ['db']
    volumes_from: ['configs']
db:
    image: example.com/db:latest
configs:
    image: example.com/configs:latest

running dcao-namespace docker-compose.yml myservice would produce

namespace: myservice
myservice.web:
    image: example.com/web:latest
    links: ['myservice.db:db']
    volumes_from: ['myservice.configs']
myservice.db:
    image: example.com/db:latest
myservice.configs:
    image: example.com/configs:latest

Usage

First generate the namespaced config

dcao-namespace -o myservice.yml docker-compose.yml myservice

Next you'll want to make myservice.yml available to other services. In this example we'll assume we're using an s3 bucket

aws s3 cp myservice.yml s3://some-bucket/compose-registry/myservice.yml

Now we can use that configuration as an include in another service. In a different services compose-with-includes.yml (which will be consumed by dcao-include)

include:
    - s3://some-bucket/compose-registry/myservice.yml

dcao-merge

Merge docker-compose.yml configuration files by overriding values in the base configuration with values from other files. It is used to transform a configuration without having to duplicate any fields that should remain consistent.

Use Cases

• Often in development you'll want to include code using a volume for faster iteration, but for testing on a CI you want to include the source in the container with ADD (or COPY). You could use an overrides-dev.yml to add volumes to the configuration, and skip that step during CI.
• If the composition is running on a shared host each developer needs to use a different host port. This variation can be included in a file maintained by each developer outside of version control.
• If a docker-compose.yml contains build directives for local development, but needs image directives in other environments (testing, stage, prod, etc), merge can be used to rewrite build to image with the correct image tag.

Usage

To rewrite a configuration to use image instead of build, and remove any host specific configuration:

dcao-merge -o export.yml docker-compose.yml compose-overrides.yml

Where docker-compose.yml is:

web:
    build: .
    links: ['db']
    volumes: ['./logs:/app/logs']
db:
    build: database/

and compose-overrides.yml:

web:
    image: example.com/web:latest
    volumes: []
db:
    image: example.com/db:latest

would produce an export.yml

web:
    image: example.com/web:latest
    links: ['db']
    volumes: []
db:
    image: example.com/db:latest