MUPX Feature Discussion - proxy, vhost, force-ssl, nginx and docker

[stephentcannon]

My application is a financial application that touches consumer credit card data mag stripe data and must use Meteor's forcessl.

As well, each new customer gets their own instance of the Meteor application. Considering that it is a mission critical financial application I cannot consider multi-homing multiple clients in a single application or database. It would just be irresponsible. I don't care that Square, Intuit Quickbooks POS or other point of sale companies do it. It isn't right.

That creates another requirement, namely to be able to handle requests with a wildcard SSL certificate thereby complicating things a little bit.

So, mupx is great for deploying and managing multiple instances. Except that it wasn't supporting force-ssl,

Currently, when deploying with mupx it forces your meteor app to launch on port 80.

Even if you use mupx ssl termination, it only handles 443.

This issue opens up a discussions of how can we implement force-ssl while still using mupx's excellent ssl handling and how we might implement proxying and vhosts and maybe even load balancing.

I have successfully used mupx to deploy nginx on port 80 and 443 for ssl termination and my meteor app on another port (58080).

I did it through some rough hacking and hard coding because I didn't not want to force a solution on the community.

Rather I think it would be good to get Arunoda and his team as well as the community to steer this project to a suitable solution in order to solve these common set of issues.

So, I cloned the docker containers for MeteorD, Mup Front End Server and Mupx and made manual static changes.

You can see for yourself.

https://registry.hub.docker.com/u/etherpos/meteord/

https://registry.hub.docker.com/u/etherpos/mup-frontend-server/

As well, I cloned mupx and made some static changes just to get this to work.

https://github.com/stephentcannon/meteor-up/tree/mupx

https://github.com/EtherPOS/mup-frontend-server/tree/forcessl

https://github.com/EtherPOS/mup-frontend-server/tree/forcessl

Here is what I did to get it to work in a very static manual manner.

1. I made some manual changes to mup-frontend-server/lib/nginx.conf
   This causes nginx to listen on 80, forces ssl and proxies to 58080 properly while still using mupx to handle ssl certification installation, etc... because it is all dockerized.

   upstream site{
      server backend:58080;
    }

    server {
      listen 80  default_server;
      server_name *.etherpos.com;
      return 302 https://$host$request_uri;
    }

1. Then I made static changes to mup-frontend-server/run.sh by adding the publish of 80:80 for nginx

docker run -it \
  --volume=/opt/nginx/conf/bundle.pem:/bundle.crt \
  --volume=/opt/nginx/conf/private.key:/private.key \
  --link=meteor:backend \
  --publish=80:80 \
  --publish=443:443 \
  meteorhacks:mup-ssl-server /start.sh

1. Then I changed the Docker file for MeteorD specifically exposing 58080 instead of how it is defaulted to 80.

FROM debian:wheezy
MAINTAINER EtherPOS, LLC

ENV METEORD_DIR /opt/meteord
COPY scripts $METEORD_DIR

RUN bash $METEORD_DIR/init.sh

EXPOSE 58080
ENTRYPOINT bash $METEORD_DIR/run_app.sh

1. Then I changed my forcessl mupx as follows. See that it points to the branched docker containers that I edited above and has some changes related to how the docker containers are launched specifically setting port 80 alongside the SSL.

set -e
docker pull etherpos/meteord:forcessl

if [ "$USE_LOCAL_MONGO" == "1" ]; then
  docker run \
    -d \
    --restart=always \
    --publish=$PORT:$PORT \
    --volume=$BUNDLE_PATH:/bundle \
    --env-file=$ENV_FILE \
    --link=mongodb:mongodb \
    --hostname="$HOSTNAME-$APPNAME" \
    --env=MONGO_URL=mongodb://mongodb:27017/$APPNAME \
    --name=$APPNAME \
    etherpos/meteord:forcessl
else
  docker run \
    -d \
    --restart=always \
    --publish=$PORT:$PORT \
    --volume=$BUNDLE_PATH:/bundle \
    --hostname="$HOSTNAME-$APPNAME" \
    --env-file=$ENV_FILE \
    --name=$APPNAME \
    etherpos/meteord:forcessl
fi

<% if(typeof sslConfig === "object")  { %>
  docker pull etherpos/mup-frontend-server:forcessl
  docker run \
    -d \
    --restart=always \
    --volume=/opt/$APPNAME/config/bundle.crt:/bundle.crt \
    --volume=/opt/$APPNAME/config/private.key:/private.key \
    --link=$APPNAME:backend \
    --publish=80:80 \
    --publish=<%= sslConfig.port %>:443 \
    --name=$APPNAME-frontend \
    etherpos/mup-frontend-server:forcessl /start.sh
<% } %>

Then I deployed and it worked great. You can test it by visiting http://base.etherpos.com and you will see that nginx is listening on 80, forcing to 443 and proxying requests to 58080. Also, it is preventing direct access to 58080 either with or without SSL which is what force-ssl should do.

So, what is the strategy for implementing this?

Strategy 1 - Implement force-ssl only to comply with the Meteor package

Maybe have an environment variable

"env": {
    "FORCESSL": false, // true or false
    "HOST": *.domain.com // used to write the nginx.conf file either wildcard or specific host
    "backendPort": 58080 //already exists in mup in SSL config area...maybe leave, maybe put here?
}

I think with these 3 pieces of information we could at least implement force-ssl where nginx listens on 80, forces to 443 and proxies to the backendPort easily enough.

Strategy 2 - Use mupx and mup.json to configure ports, proxies, etc...

We could have new environment elements.

I am honestly not even sure how we would begin to handle all of these requirements from the mup.json

It starts to get quite complicated when we start thinking about vhosts, proxying, load balancing etc...

Strategy 3 - Use mupx and mup.json to configure which docker container to use as the front end and the meteord.

Instead maybe mupx does what it does best. Help you deploy but lets you decide what docker container you want to use for the Mup Front End docker container and the MeteorD docker container.

This could really open up some possibilities for people that want to do load balancing, vhosts, proxying and routing to multiple apps based on varying requirements coming into nginx as well as all the good stuff like ssl termination and certificate handling.

Maybe something like...

"env": {
    METEORD_CONTAINER : "etherpos/meteord:forcessl",
    MUPX_FRONTEND_CONTAINER: "etherpos/mup-frontend-server:forcessl"
}

To be used in meteor-up/templates/linux/start.sh

docker pull $METEORD_CONTAINER

if [ "$USE_LOCAL_MONGO" == "1" ]; then
  docker run \
    -d \
    --restart=always \
    --publish=$PORT:$PORT \
    --volume=$BUNDLE_PATH:/bundle \
    --env-file=$ENV_FILE \
    --link=mongodb:mongodb \
    --hostname="$HOSTNAME-$APPNAME" \
    --env=MONGO_URL=mongodb://mongodb:27017/$APPNAME \
    --name=$APPNAME \
    $METEORD_CONTAINER
else
  docker run \
    -d \
    --restart=always \
    --publish=$PORT:$PORT \
    --volume=$BUNDLE_PATH:/bundle \
    --hostname="$HOSTNAME-$APPNAME" \
    --env-file=$ENV_FILE \
    --name=$APPNAME \
    $METEORD_CONTAINER
fi

<% if(typeof sslConfig === "object")  { %>
  docker pull $MUPX_FRONTEND_CONTAINER
  docker run \
    -d \
    --restart=always \
    --volume=/opt/$APPNAME/config/bundle.crt:/bundle.crt \
    --volume=/opt/$APPNAME/config/private.key:/private.key \
    --link=$APPNAME:backend \
    --publish=80:80 \
    --publish=<%= sslConfig.port %>:443 \
    --name=$APPNAME-frontend \
    $MUPX_FRONTEND_CONTAINER /start.sh
<% } %>

I am not sure how we would set the sslConfig part though. Currently, mupx doesn't have the line. I tossed that in there because I am using force-ssl,

--publish=80:80 \

Another advantage of using this last strategy is that it would allow people to package other applications with their deployment if they want. It could go so far as having a way to support deployment of other docker containers inside the task runner like email servers, etc... but that might be out of scope.

Next Steps

This started as an email discussion between Arunoda and myself and I sent him some details on what I did to get this working. It would be easy if for only force-ssl via another environment parameter.

I firmly believe supporting force-ssl is absolutely necessary because it is a core package. Mupx not supporting core Meteor features seems important and that was my primary goal.

While exploring this I saw other people asking about vhosts, proxying, load balancing, etc... and thought that all of it is probably within reach if a strategy can be agreed upon.

So, with that said, comments and help everyone?

[arunoda]

Right now we are using force-ssl meteor package and it works pretty well for us.

But, in the long run we need to support www redirect also. So, I think it's better to handle it via nginx layer. We should allow it by default, but should be possible to turn it off. (If the user is managing SSL and other stuff)

Right now we are not proxying port 80 with nginx. But it the user is asking to implement force-ssl and www-redirect via mup.json file, I think we can do the proxying.

[stephentcannon]

@arunoda It seems implementing force-ssl with nginx would be the most straightforward solution.

But there is some overlap on #454 in terms of how @4shaw implemented a volume on the docker image for nginx to serve static files that is in line with strategy 4 I listed above.

[arunoda]

That's another issue and we won't support it.

On Mon, Jun 1, 2015 at 5:13 PM steeve cannon notifications@github.com
wrote:

@arunoda https://github.com/arunoda It seems implementing force-ssl
with nginx would be the most straightforward solution.

But there is some overlap on #454
zodern/meteor-up#454 in terms of how @4shaw
https://github.com/4shaw implemented a volume on the docker image for
nginx to serve static files that is in line with strategy 4 I listed above.

—
Reply to this email directly or view it on GitHub
zodern/meteor-up#458 (comment).

[stephentcannon]

That is valid.

Would you like me to take a stab at support for force-ssl then?

If so, will the strategy be through mupx and mup.json and not via configurable docker containers?

Should nginx always listen 80 and proxy? Regardless of whether ssl or force-ssl is being used, it seems valid that nginx should always listen on 80 and proxy to a backendPort.

Although that forces use of mup-frontend-server even if you aren't using SSL. Right now if you aren't using SSL then mup-frontend-server is not used.

[arunoda]

We can start by listening to port 80 by default. We still need to use
docker. I
We can use your version of mup-front-end server for that.

But, we need to pass the app when running the docker container via env vars.
Then we need to use it.
There is no easy way to access env vars in nginx conf file, so, you've sed
it.
See: http://serverfault.com/a/582794

On Mon, Jun 1, 2015 at 7:14 PM steeve cannon notifications@github.com
wrote:

That is valid.

Would you like me to take a stab at support for force-ssl then?

If so, will the strategy be through mupx and mup.json and not via
configurable docker containers?

Should nginx always listen 80 and proxy? Regardless of whether ssl or
force-ssl is being used, it seems valid that nginx should always listen on
80 and proxy to a backendPort.

Although that forces use of mup-frontend-server even if you aren't using
SSL. Right now if you aren't using SSL then mup-frontend-server is not used.

—
Reply to this email directly or view it on GitHub
zodern/meteor-up#458 (comment).

[bytesandwich]

@arunoda @stephentcannon have you made any progress on forcessl?

[sahanDissanayake]

is there a way to remove force-ssl ? I have a widget and sometimes its a curse to have https.. any thoughts on this ?

[MasterJames]

If you integrate LetsEncrypt your free keys should be no reason to not use ssl. Without it passwords can be seen.

[sahanDissanayake]

Yes I do know that. so what i did was remove MUPX SSL and setup my own nginx setup on the server, So if the user visit http then the server serves http. but for login signup i will have the links setup for https so the user will request the https

server_tokens off; # for security-by-obscurity: stop displaying nginx version


# HTTP
server {
    listen 80 default_server; # if this is not a default server, remove "default_server"
    listen [::]:80 default_server ipv6only=on;

    server_name myapp.com; # the domain on which we want to host the application. Since we set "default_server" previously, nginx will answer all hosts anyway.

    add_header Strict-Transport-Security "max-age=0;";

    # redirect non-SSL to SSL
    location / {
        proxy_pass http://127.0.0.1:3000;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection 'upgrade';
        proxy_set_header X-Forwarded-For $remote_addr;
    }

}

# HTTPS server
server {
    listen 443 ssl spdy; # we enable SPDY here
    server_name myapp.com; # this domain must match Common Name (CN) in the SSL certificate


    ssl_certificate /etc/letsencrypt/archive/myapp.com/fullchain1.pem; # full path to SSL certificate and CA certificate concatenated together
    ssl_certificate_key /etc/letsencrypt/archive/myapp.com/privkey1.pem; # full path to SSL key

    # performance enhancement for SSL
    ssl_stapling on;
    ssl_session_cache shared:SSL:10m;
    ssl_session_timeout 5m;

    # safety enhancement to SSL: make sure we actually use a safe cipher
    ssl_prefer_server_ciphers on;
    ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
    ssl_ciphers 'ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:ECDHE-RSA-RC4-SHA:ECDHE-ECDSA-RC4-SHA:RC4-SHA:HIGH:!aNULL:!eNULL:!EXPORT:!DES:!3DES:!MD5:!PSK';

    # config to enable HSTS(HTTP Strict Transport Security) https://developer.mozilla.org/en-US/docs/Security/HTTP_Strict_Transport_Security
    # to avoid ssl stripping https://en.wikipedia.org/wiki/SSL_stripping#SSL_stripping
    add_header Strict-Transport-Security "max-age=0;";

    # If your application is not compatible with IE <= 10, this will redirect visitors to a page advising a browser update
    # This works because IE 11 does not present itself as MSIE anymore

    # pass all requests to Meteor
    location / {
        proxy_pass http://127.0.0.1:3000;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
        proxy_set_header Host $host;
        proxy_set_header X-Forwarded-For $remote_addr; # preserve client IP


        # this setting allows the browser to cache the application in a way compatible with Meteor
        # on every applicaiton update the name of CSS and JS file is different, so they can be cache infinitely (here: 30 days)
        # the root path (/) MUST NOT be cached
        if ($uri != '/') {
            expires 30d;
        }
    }
}