Skip to content

Remove example, make Nginx config extensible, simplify configurations #37

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 13 commits into from
Closed

Remove example, make Nginx config extensible, simplify configurations #37

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
13 commits
Select commit Hold shift + click to select a range
c4a4ed6
Add nginx-ssl interface (non-public)
jasonpincin Oct 19, 2016
602e0b5
Removing example
jasonpincin Oct 21, 2016
24ed42d
Exit without error when not leader. Added watch, which starts consul-…
jasonpincin Oct 21, 2016
80300a2
Seperated generate-config and reload for clarity in containerpilot.json
jasonpincin Oct 21, 2016
a70f486
Broke nginx config out into multiple files, suitable for partial over…
jasonpincin Oct 21, 2016
f7a8a63
Do not do debug logging by default. Axed absolute paths for readabili…
jasonpincin Oct 21, 2016
3789a55
Updated for restructured Nginx config and nginx config templates dire…
jasonpincin Oct 21, 2016
4eb70c0
Removed .sh extensions for consistency
jasonpincin Oct 21, 2016
06491cb
Pin Nginx container version
jasonpincin Oct 21, 2016
15fc76d
Update to ContainerPilot 2.4.4
jasonpincin Nov 7, 2016
ac6aeea
Link to mozilla SSL best practices
jasonpincin Nov 7, 2016
faf5c39
reduce calls to consul-template
jasonpincin Nov 7, 2016
855c6bc
No -k option on curl tests. Use LE Staging CA Cert for curl when ACME…
jasonpincin Nov 8, 2016
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 8 additions & 5 deletions Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
# A minimal Nginx container including ContainerPilot and a simple virtualhost config
FROM nginx:latest
# A minimal Nginx container including ContainerPilot
FROM nginx:1.11
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As long as we're doing an update we might consider tossing in ContainerPilot 2.4.4 which reduces some log spam.


# Add some stuff via apt-get
RUN apt-get update \
Expand Down Expand Up @@ -33,10 +33,10 @@ RUN export CONSUL_TEMPLATE_VERSION=0.14.0 \
&& rm /tmp/consul-template.zip

# Add Containerpilot and set its configuration
ENV CONTAINERPILOT_VER 2.4.3
ENV CONTAINERPILOT_VER 2.4.4
ENV CONTAINERPILOT file:///etc/containerpilot.json

RUN export CONTAINERPILOT_CHECKSUM=2c469a0e79a7ac801f1c032c2515dd0278134790 \
RUN export CONTAINERPILOT_CHECKSUM=6194ee482dae95844046266dcec2150655ef80e9 \
&& curl -Lso /tmp/containerpilot.tar.gz \
"https://github.com/joyent/containerpilot/releases/download/${CONTAINERPILOT_VER}/containerpilot-${CONTAINERPILOT_VER}.tar.gz" \
&& echo "${CONTAINERPILOT_CHECKSUM} /tmp/containerpilot.tar.gz" | sha1sum -c \
Expand All @@ -56,7 +56,10 @@ RUN export JQ_VERSION=1.5 \
&& chmod a+x /usr/local/bin/jq

# Add our configuration files and scripts
COPY etc /etc
RUN rm -f /etc/nginx/conf.d/default.conf
COPY etc/acme /etc/acme
COPY etc/containerpilot.json /etc/
COPY etc/nginx /etc/nginx/templates
COPY bin /usr/local/bin

# Usable SSL certs written here
Expand Down
21 changes: 13 additions & 8 deletions bin/acme
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,7 @@ function renewConsulSession () {

function createConsulSession () {
printf "Creating Consul session... "
local SID=$(curl -sX PUT -d '{"LockDelay":"0s","Name":"acme-lock","Behavior":"release","TTL":"600s"}' ${CONSUL_ROOT}/session/create | awk -F '"' '{print $4}')
local SID=$(curl -sX PUT -d '{"LockDelay":"0s","Name":"acme-lock","Behavior":"release","TTL":"5400s"}' ${CONSUL_ROOT}/session/create | awk -F '"' '{print $4}')
rc=$?
if [[ $rc -ne 0 ]]; then
echo "failed"
Expand Down Expand Up @@ -106,10 +106,10 @@ function updateKeys () {
local TEMP_PRIVKEY="${TEMP_CERT_DIR}/privkey.pem"
local FULLCHAIN="${CERT_DIR}/fullchain.pem"
local PRIVKEY="${CERT_DIR}/privkey.pem"
if [ -f ${TEMP_FULLCHAIN} -a -f ${TEMP_PRIVKEY} -a "$(cat ${TEMP_FULLCHAIN})" != "" -a "$(cat ${TEMP_PRIVKEY})" != "" ]; then
if [ -f ${TEMP_FULLCHAIN} -a -f ${TEMP_PRIVKEY} -a "$(cat ${TEMP_FULLCHAIN} 2> /dev/null)" != "" -a "$(cat ${TEMP_PRIVKEY} 2> /dev/null)" != "" ]; then
cp -f $TEMP_FULLCHAIN $FULLCHAIN
cp -f $TEMP_PRIVKEY $PRIVKEY
$SCRIPTPATH/reload.sh
$SCRIPTPATH/reload
fi
}

Expand All @@ -126,9 +126,8 @@ case "$1" in
acquire-leader)
acquireLeader
;;
checkin)
renewConsulSession &&
( acquireLeader || exit 0 )
watch)
/usr/local/bin/consul-template -config /etc/acme/watch.hcl -consul $CONSUL_HOST:8500
;;
init)
if [ -f ${CERT_DIR}/fullchain.pem -a -f ${CERT_DIR}/privkey.pem ]; then
Expand All @@ -142,13 +141,13 @@ case "$1" in
renew-certs)
shift
renewConsulSession &&
acquireLeader &&
(acquireLeader || exit 0) &&
${SCRIPTPATH}/dehydrated --cron --domain ${ACME_DOMAIN} --hook /etc/acme/dehydrated/hook.sh --config /etc/acme/dehydrated/config.${ACME_ENV}
;;
clean-certs)
shift
renewConsulSession &&
acquireLeader &&
(acquireLeader || exit 0) &&
${SCRIPTPATH}/dehydrated --cleanup --domain ${ACME_DOMAIN} --hook /etc/acme/dehydrated/hook.sh --config /etc/acme/dehydrated/config.${ACME_ENV}
;;
generate-challenge-token)
Expand All @@ -157,6 +156,12 @@ case "$1" in
update-keys)
updateKeys
;;
checkin)
renewConsulSession &&
( acquireLeader || exit 0 ) &&
${SCRIPTPATH}/dehydrated --cron --domain ${ACME_DOMAIN} --hook /etc/acme/dehydrated/hook.sh --config /etc/acme/dehydrated/config.${ACME_ENV} &&
${SCRIPTPATH}/dehydrated --cleanup --domain ${ACME_DOMAIN} --hook /etc/acme/dehydrated/hook.sh --config /etc/acme/dehydrated/config.${ACME_ENV}
;;
*)
echo $"Usage: $0 [ {get,create,renew}-consul-session | acquire-leader | init | checkin | renew-certs | clean-certs | generate-challenge-token ]"
exit 1
Expand Down
31 changes: 31 additions & 0 deletions bin/generate-config
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
#!/bin/bash
set -e

SERVICE_NAME=${SERVICE_NAME:-nginx}
CONSUL=${CONSUL:-consul}
CERT_DIR="/var/www/ssl"

# Determine whether ACME SSL certificates have been acquired
SSL_READY="false"
if [ -f ${CERT_DIR}/fullchain.pem -a -f ${CERT_DIR}/privkey.pem ]; then
export SSL_READY="true"
fi

# Generate a conf.d config file for every corresponding cond.d Consul template
for f in $(ls -1 /etc/nginx/templates/conf.d/)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure if you missed it in the consul-template docs but we can render multiple files from a single invocation, which should reduce the number of API calls to Consul. Their example is as follows:

$ consul-template \
  -consul my.consul.internal:6124 \
  -template "/tmp/nginx.ctmpl:/var/nginx/nginx.conf:service nginx restart" \
  -template "/tmp/redis.ctmpl:/var/redis/redis.conf:service redis restart" \
  -template "/tmp/haproxy.ctmpl:/var/haproxy/haproxy.conf"

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I updated the 2nd block to use this approach, as there's a static list of files there. For the first block, the files would primarily be user supplied, so we need to loop over the list.

do
consul-template \
-once \
-dedup \
-consul ${CONSUL}:8500 \
-template "/etc/nginx/templates/conf.d/${f}:/etc/nginx/conf.d/${f}"
done

# Render Nginx configuration template using values from Consul
consul-template \
-once \
-dedup \
-consul ${CONSUL}:8500 \
-template "/etc/nginx/templates/health.conf:/etc/nginx/health.conf" \
-template "/etc/nginx/templates/ssl.conf:/etc/nginx/ssl.conf" \
-template "/etc/nginx/templates/nginx.conf:/etc/nginx/nginx.conf"
17 changes: 17 additions & 0 deletions bin/health-check
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
#!/usr/bin/env bash

LE_STAGING_CACERT=/etc/acme/ca/lets-encrypt-staging.crt
ACME_ENV=${ACME_ENV:-staging}

SCHEME=$1

if [[ $SCHEME == "https" ]]; then
grep -q $ACME_DOMAIN /etc/hosts || printf "\n\n127.0.0.1 ${ACME_DOMAIN}\n" >> /etc/hosts
if [[ "$ACME_ENV" == "staging" ]]; then
curl --cacert $LE_STAGING_CACERT -fsSo /dev/null "https://${ACME_DOMAIN}/nginx-health"
else
curl -fsSo /dev/null "https://${ACME_DOMAIN}/nginx-health"
fi
else
curl -fsSo /dev/null "http://localhost/nginx-health"
fi
5 changes: 5 additions & 0 deletions bin/reload
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
#!/bin/bash
set -e

generate-config
nginx -s reload
48 changes: 0 additions & 48 deletions bin/reload.sh
View file
Open in desktop

This file was deleted.

0 bin/sensor.sh → bin/sensor
View file
Open in desktop
File renamed without changes.
53 changes: 0 additions & 53 deletions docker-compose.yml
View file
Open in desktop

This file was deleted.

29 changes: 29 additions & 0 deletions etc/acme/ca/lets-encrypt-staging.crt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
-----BEGIN CERTIFICATE-----
MIIFATCCAumgAwIBAgIRAKc9ZKBASymy5TLOEp57N98wDQYJKoZIhvcNAQELBQAw
GjEYMBYGA1UEAwwPRmFrZSBMRSBSb290IFgxMB4XDTE2MDMyMzIyNTM0NloXDTM2
MDMyMzIyNTM0NlowGjEYMBYGA1UEAwwPRmFrZSBMRSBSb290IFgxMIICIjANBgkq
hkiG9w0BAQEFAAOCAg8AMIICCgKCAgEA+pYHvQw5iU3v2b3iNuYNKYgsWD6KU7aJ
diddtZQxSWYzUI3U0I1UsRPTxnhTifs/M9NW4ZlV13ZfB7APwC8oqKOIiwo7IwlP
xg0VKgyz+kT8RJfYr66PPIYP0fpTeu42LpMJ+CKo9sbpgVNDZN2z/qiXrRNX/VtG
TkPV7a44fZ5bHHVruAxvDnylpQxJobtCBWlJSsbIRGFHMc2z88eUz9NmIOWUKGGj
EmP76x8OfRHpIpuxRSCjn0+i9+hR2siIOpcMOGd+40uVJxbRRP5ZXnUFa2fF5FWd
O0u0RPI8HON0ovhrwPJY+4eWKkQzyC611oLPYGQ4EbifRsTsCxUZqyUuStGyp8oa
aoSKfF6X0+KzGgwwnrjRTUpIl19A92KR0Noo6h622OX+4sZiO/JQdkuX5w/HupK0
A0M0WSMCvU6GOhjGotmh2VTEJwHHY4+TUk0iQYRtv1crONklyZoAQPD76hCrC8Cr
IbgsZLfTMC8TWUoMbyUDgvgYkHKMoPm0VGVVuwpRKJxv7+2wXO+pivrrUl2Q9fPe
Kk055nJLMV9yPUdig8othUKrRfSxli946AEV1eEOhxddfEwBE3Lt2xn0hhiIedbb
Ftf/5kEWFZkXyUmMJK8Ra76Kus2ABueUVEcZ48hrRr1Hf1N9n59VbTUaXgeiZA50
qXf2bymE6F8CAwEAAaNCMEAwDgYDVR0PAQH/BAQDAgEGMA8GA1UdEwEB/wQFMAMB
Af8wHQYDVR0OBBYEFMEmdKSKRKDm+iAo2FwjmkWIGHngMA0GCSqGSIb3DQEBCwUA
A4ICAQBCPw74M9X/Xx04K1VAES3ypgQYH5bf9FXVDrwhRFSVckria/7dMzoF5wln
uq9NGsjkkkDg17AohcQdr8alH4LvPdxpKr3BjpvEcmbqF8xH+MbbeUEnmbSfLI8H
sefuhXF9AF/9iYvpVNC8FmJ0OhiVv13VgMQw0CRKkbtjZBf8xaEhq/YqxWVsgOjm
dm5CAQ2X0aX7502x8wYRgMnZhA5goC1zVWBVAi8yhhmlhhoDUfg17cXkmaJC5pDd
oenZ9NVhW8eDb03MFCrWNvIh89DDeCGWuWfDltDq0n3owyL0IeSn7RfpSclpxVmV
/53jkYjwIgxIG7Gsv0LKMbsf6QdBcTjhvfZyMIpBRkTe3zuHd2feKzY9lEkbRvRQ
zbh4Ps5YBnG6CKJPTbe2hfi3nhnw/MyEmF3zb0hzvLWNrR9XW3ibb2oL3424XOwc
VjrTSCLzO9Rv6s5wi03qoWvKAQQAElqTYRHhynJ3w6wuvKYF5zcZF3MDnrVGLbh1
Q9ePRFBCiXOQ6wPLoUhrrbZ8LpFUFYDXHMtYM7P9sc9IAWoONXREJaO08zgFtMp4
8iyIYUyQAbsvx8oD2M8kRvrIRSrRJSl6L957b4AFiLIQ/GgV2curs0jje7Edx34c
idWw1VrejtwclobqNMVtG3EiPUIpJGpbMcJgbiLSmKkrvQtGng==
-----END CERTIFICATE-----
53 changes: 19 additions & 34 deletions etc/containerpilot.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,43 +1,43 @@
{
"consul": "{{ if .CONSUL_AGENT }}localhost{{ else }}{{ if .CONSUL }}{{ .CONSUL }}{{ else }}consul{{ end }}{{ end }}:8500",
"preStart": "/usr/local/bin/reload.sh preStart",
"logging": {"level": "DEBUG"},
"preStart": "generate-config",
"services": [
{
"name": "nginx",
"port": 80,
"health": "/usr/bin/curl --fail --silent --show-error --output /dev/null http://localhost/nginx-health",
"health": "health-check http",
"poll": 10,
"ttl": 25,
"interfaces": ["eth0"]
},
{
"name": "nginx-public",
"port": 80,
"health": "/usr/bin/curl --fail --silent --show-error --output /dev/null http://localhost/nginx-health",
"health": "health-check http",
"poll": 10,
"ttl": 25,
"interfaces": ["eth1", "eth0"]
}{{ if .ACME_DOMAIN }},
{
"name": "nginx-ssl",
"port": 443,
"health": "acme init && health-check https",
"poll": 10,
"ttl": 25,
"interfaces": ["eth0"]
},
{
"name": "nginx-public-ssl",
"port": 443,
"health": "/usr/local/bin/acme init && /usr/bin/curl --insecure --fail --silent --show-error --output /dev/null --header \"HOST: {{ .ACME_DOMAIN }}\" https://localhost/nginx-health",
"health": "health-check https",
"poll": 10,
"ttl": 25,
"interfaces": ["eth1", "eth0"]
}{{ end }}
],
"backends": [
{
"name": "{{ .BACKEND }}",
"poll": 7,
"onChange": "/usr/local/bin/reload.sh"
}
],
"coprocesses": [{{ if .CONSUL_AGENT }}
{
"command": ["/usr/local/bin/consul", "agent",
"command": ["consul", "agent",
"-data-dir=/var/lib/consul",
"-config-dir=/etc/consul",
"-rejoin",
Expand All @@ -49,9 +49,7 @@
{{ if and .CONSUL_AGENT .ACME_DOMAIN }},{{ end }}
{{ if .ACME_DOMAIN }}
{
"command": ["/usr/local/bin/consul-template",
"-config", "/etc/acme/watch.hcl",
"-consul", "{{ if .CONSUL_AGENT }}localhost{{ else }}{{ if .CONSUL }}{{ .CONSUL }}{{ else }}consul{{ end }}{{ end }}:8500"],
"command": ["acme", "watch"],
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not just use /usr/local/bin/consul-template -config /etc/acme/watch.hcl -consul $CONSUL_HOST:8500 directly here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We certainly can. I just did it this way so that all the moving parts related to certificate acquisition was contained within the acme script, so that when you're looking at that you see it all, versus some pieces here, some there. Things like the CONSUL_AGENT are tucked away cleanly in the acme script, etc. Cleans up containerpilot.json a bit. Admittedly, this is purely stylistic.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, that's totally reasonable. Let's keep it as you've got it here.

"restarts": "unlimited"
}{{ end }}],
"telemetry": {
Expand All @@ -62,35 +60,22 @@
"help": "Number of accepted connnections that were not handled",
"type": "gauge",
"poll": 5,
"check": ["/usr/local/bin/sensor.sh", "unhandled"]
"check": ["sensor", "unhandled"]
},
{
"name": "nginx_connections_load",
"help": "Ratio of active connections (less waiting) to the maximum worker connections",
"type": "gauge",
"poll": 5,
"check": ["/usr/local/bin/sensor.sh", "connections_load"]
"check": ["sensor", "connections_load"]
}
]
},
"tasks": [{{ if .ACME_DOMAIN }}
{
"name": "acme-checkin",
"command": [ "/usr/local/bin/acme", "checkin" ],
"frequency": "5m",
"command": [ "acme", "checkin" ],
"frequency": "1h",
"timeout": "10s"
},
{
"name": "acme-renew-certs",
"command": [ "/usr/local/bin/acme", "renew-certs" ],
"frequency": "12h",
"timeout": "10m"
},
{
"name": "clean-unused-certs",
"command": ["/usr/local/bin/acme", "clean-certs" ],
"frequency": "24h",
"timeout": "10m"
}{{ end }}
]
}{{ end }}]
}
Loading