confd is a great project for building dynamic configurations from data stores like etcd and zookeeper, among many others. It can be run once to generate the config files or continuously, watching for data changes to regenerate the config files and reload their dependent services. These capabilities are vital to modern distributed architectures since many services require their configurations to live on disk and lack support for these more advanced configuration stores. confd bridges that gap cleanly.
At Brand Networks, we like to containerize everything. This constraint may seem like a nuisance at first glance, but it enforces clean boundaries between services. When a service comes online, whether that service be a Node or Python API or a data store like Cassandra, it registers itself with etcd. Using confd, we have dynamic service discovery, and the individual services can be kept clean.
Assuming you are running boot2docker, running ./up.sh should bring up a working installation of both a static and dynamic confd container.
The dynamic container will immediately generate configs from the current data in etcd.
You can see the file it generated by running docker exec -it confd_dynamic /bin/sh in a new shell followed by cat /etc/confd/dynamic.cfg.
About 60 seconds after, the data in etcd will change.
The script will then show you a tail of the logs.
About 60 seconds into watching not a whole lot, confd will poll and receive the changes, causing it to regenerate /etc/confd/dynamic.cfg and restarting the dummy test container.
Running the docker exec command mentioned previously will show you the new data.
Running docker ps will show you that the test container has recently restarted.
Much less interesting, but still important is the static configuration.
The static confd will run and immediately exit.
To see the configuration that it generated, run docker run --rm -it --volumes-from confd_static ubuntu /bin/bash.
You should have a generated file in /etc/confd/static.cfg.
This is utilizing a fairly simple container pattern: data containers.
Typically, the data in that type of container is used for persistence across service restarts.
Here, we are applying that to a service's configuration as well.
There are two types of images at play here: confd, which never changes, and a confd config image.
The confd config image simply exposes as a volume /etc/confd.
Add to that whatever you would to run confd yourself.
The command that image runs is /bin/true since it's whole purpose is to be a data container.
The confd container will then use the volume from that config data container to make another directory that it will need to expose as a volume for future containers like Cassandra or your API.
For a full demo, see the up.sh and down.sh scripts.
To use this, you'll need to make your own config data image.
See either confd-config-dynamic or confd-config-static for examples.
Basically, you'll use the same Dockerfile, but the contents of confd-config will be whatever you want confd to do.
When you run the confd container, the command line options are the same as confd itself since confd was set as the ENTRYPOINT of the image. If you need to restart a docker image, make sure you have docker.sock mounted as in the example.