browserSync requires cumbersome work-arounds to function

[bretmette]

I thought I would create an issue here to discuss a better workaround / solution for browserSync. Let me know if you are interested in discussing that. I can move the issue to my own fork if it is not something you want to pursue with me.

Currently when running gulp watch which triggers browserSync the connection is null-routed. I have a really ugly work around that I think we can improve upon.

[bretmette]

Currently I issue the following command at the command line --

bmette$ docker-compose run -p 3000:3000 -p 3001:3001 --rm --name projectname_frontend-dev_1 frontend-dev gulp watch

And here is my docker-compose.yml

nginx:
    build: ./.docker/nginx/
    ports:
        - "80:80"
    expose:
        - "80"
    links:
        - php
    volumes:
        - ./:/var/www/html

php:
    build: ./.docker/php/
    expose:
        - "9000"
    links:
        - db
    volumes:
        - ./:/var/www/html

frontend-dev:
    image: mkenney/npm
    links:
        - nginx:local.dev
    volumes:
        - ./:/src:rw

db:
    image: mariadb:10.0.19
    ports:
        - "3306:3306"
    expose:
        - "3306"
    env_file:
        - ./.docker/mysql/.env.db
    volumes:
        - /usr/local/var/mysql:/var/lib/mysql

I also need to force my proxy in the browserSync config to local.dev --

gulp.task('watch', function() {
  browserSync.init({
    files: ['{lib,templates}/**/*.php', '*.php'],
    host: 'local.dev',
    proxy: 'http://local.dev',
    browser: config.browser,
    open: false,
    snippetOptions: {
      whitelist: ['/wp-admin/admin-ajax.php'],
      blacklist: ['/wp-admin/**']
    }
  });
  gulp.watch([path.source + 'styles/**/*'], ['styles']);
  gulp.watch([path.source + 'scripts/**/*'], ['jshint', 'scripts']);
  gulp.watch([path.source + 'fonts/**/*'], ['fonts']);
  gulp.watch([path.source + 'images/**/*'], ['images']);
  gulp.watch(['bower.json', 'assets/manifest.json'], ['build']);
});

Additionally I have to add an entry in my /etc/hosts file --

127.0.0.1    local.dev

While this works, it is an awful work-around with the following problems:

1. Configuration is not project agnostic.

2. Forces browserSync to run on a specific domain for WordPress development, meaning if I used http://localhost as my proxy it will not work (because localhost conflicts with the /etc/hosts file on the container). The container tries to route the request back to itself. Adding nginx:local.dev as a link in the docker-compose.yml will put an entry in the frontend-dev container's /etc/host so that it will properly forward the request to nginx. The downside of this is 2-fold, WordPress will re-write the URL if you try to connect to anything but local.dev and it requires environment specific configuration to be added to the docker-compose.yml file. Nasty stuff.

3. That command to bring up the gulp watch is obviously very long and difficult to remember (even if you do remember it, who wants to type all that). Making a script in each project folder sounds like a garbage solution as well.

I am sure this can be cleaned up to "just-work". However, I am too new to docker to solve this issue on my own just yet. Thought this might be a good place to start the discussion as I have a feeling you and I are like minded in wanting to do as little project-specific of environment-specific configuration as possible along with not having to remember to run a bunch of commands to get something to work just because of docker.

[mkenney]

@bretmette I'm not familiar with browser-sync but that's an interesting problem. Taking a brief look, I'm not sure that there's a simple way to use localhost to accomplish this. It might be easier with some kind of wildcard DNS (I like to use jwilder/nginx-proxy with that setup) but I'm sure I'm missing something.

Since I'm not familiar with this setup, can you put together a reduced test case I can use to reproduce the issue? I don't think I need anything around the db, or the watcher processes, but if you could explain what you're doing with the nginx proxy and the php container that would be helpful. It would be interesting to dig into.

[mkenney]

Here's what did work for me (I'm sure I'm missing pieces of the puzzle).

I used this package.js:

{
  "name": "test-project",
  "version": "0.0.1",
  "devDependencies": {
    "browser-sync": "^2.16.0",
    "grunt": "~0.4.5",
    "grunt-contrib-jshint": "^1.0.0",
    "gulp": "~3.9.1",
    "gulp-jshint": "~2.0.1",
    "jshint": "~2.9.2"
  }
}

I reduced the gulpfile to

var gulp = require('gulp');
var browserSync = require('browser-sync');
gulp.task('watch', function() {
  browserSync.init({
    files: ['{lib,templates}/**/*.php', '*.php'],
    open: false,
    snippetOptions: {
      whitelist: ['/wp-admin/admin-ajax.php'],
      blacklist: ['/wp-admin/**']
    }
  });
});

And I executed it with

docker run --rm -ti -p 3001:3001 -v $(pwd):/src:rw -v $HOME/.ssh:/home/dev/.ssh:ro mkenney/npm:latest /run-as-user /usr/local/bin/gulp watch

Navigating to that host (or localhost) on port 3001 brought up the browsersync UI.

[bretmette]

@mkenney Browsersync is amazing, it really speeds up development (not as much as it could due to the docker OS X filesystem performance issues...).

The ability to access the Browsersync UI does not tell the full story. The issue is that Browsersync itself is a proxy (among other things). The UI is run on the gulp container and accessed via port 3001, no problem there. But when you go to access the site (default on port 3000) Browsersync doesn't know how to handle the request.

Say you have http://localhost:3000/ when you hit that you first are sent to the Browsersync container, Browsersync will then attempt to access http://localhost:80/ rewrite some URLs and send the data back on port 3000 to the browser. The problem is your http server container is not the same as the one running gulp and Browsersync. So when Browsersync tries to access http://localhost:80/ it gets blackholed (since the gulp container is not serving any content on port 80, the httpd container is).

So my solution is to stitch them together using the "local.dev" hostname via this section of the docker-compose.yml.

frontend-dev:
    image: mkenney/npm
    links:
        - nginx:local.dev

This adds an entry in the frontend-dev container's /etc/hosts file for local.dev and points it to the nginx (http server) container's IP.

So now when I hit http://local.dev:3000/ the proxy path is

Browser (http://local.dev:3000/) <--> frontend-dev:3000 (Browsersync) <--> nginx:80

vs using http://localhost:3000/

Browser (http://localhost:3000) <--> frontend-dev:3000 (Browsersync) --> frontend-dev:80

The reason the Browsersync UI works without a problem on port 3001 is because Browsersync doesn't proxy the UI, it serves it from it's own container so it works without an issue. It's when Browsersync tries to access localhost:80 that it runs into a dead end.

Let me know if that makes sense. So for now my bad workaround is in this comment.

[mkenney]

@bretmette Ahh, ok, that does makes sense. I'm not sure there's a better way to do that without implementing some kind of DNS resolution that your proxy can take advantage of, I haven't done that on a Mac in years but it's relatively straightforward these days.

I do most of my development on remote boxes with wildcard DNS resolution which makes connecting that stuff together a bit easier. /etc/hosts isn't really equipped to do that, but with DNS resolution I think it could work with just your docker-compose.yml and and your browserSync config.

[mkenney]

@bretmette So I had a thought.

I think you can add net: "host" to your docker-compose.yml file for frontend-dev instead of using links, and then in your gulpfile if you set your host and proxy to 127.0.0.1 instead of local.dev, I think that might work. It might work with localhost instead of 127.0.0.1 at that point too.

If you give that a try, let me know how it works out.

[bretmette]

@mkenney my setup has become slightly more complex now. I am running a reverse-proxy using nginx so that I can run multiple apps at once.

So I have a container I start outside of the app's docker-compose called nginx-proxy which checks for containers with the VIRTUAL_HOST=myapp.dev environment variable set. More on this on reverse-proxy container's github page.

What I do now is setup the frontend-dev container to use the same hostname as what is specified in the VIRTUAL_HOST such as:

    links:
        - nginx:myapp.dev

Where myapp.dev is application specific and unique. I still have to run the frontend container on it's own using a bash script I setup called gulp-watch, but it seems to be working well so far.

There is more to it, using external networks, etc so that everything can talk to each other, but it seems to be working. I don't think I could run more than one frontend container at once, but that shouldn't be an issue. I could always switch up the port so each one has a unique port if I need to do that.

[mkenney]

@bretmette that proxy is awesome, I use it all the time. Again, I'm not totally sure how your setup, but I'm reasonably sure you could setup as many projects as you want. I use the proxy to run 12 different hosts on my dev server at the moment. They're all on random docker ports but I access them all on port 80 through the proxy.

I'm going to go ahead and close this, thanks for letting me know!

[bretmette]

@mkenney yes, I have been setting up all my projects to use the reverse proxy and it kicks ass. I have a container that is set to restart: always for the proxy along with one for mysql. Then I setup a custom network and have all my containers attach to that when they need the reverse proxy.

I think the bottom line is BrowserSync will require a little bit of additional setup and depending on the individual's workflow they will need to configure things accordingly.

The main reason it does not work "out-of-the-box" is because the docker-npm container does not expose ports 3000 & 3001 to the host (which are the default for BrowserSync). Even if it did, there would be multiple problems with routing associated with it, (i.e. localhost is not the same on the host as it is in the container so some sort of middleware is needed to fix the routing).

I think using a custom binary to run gulp watch when I want to use BrowserSync is the best course of action at this time.

I believe to make this work docker-npm would have to detect if BrowserSync is being used in gulp or grunt and then modify the container at run-time to expose the correct ports to the host. Additionally it would have to setup custom routing in the docker network associated with docker-npm.

That seems out of scope for this project.

[necevil]

@bretmette any chance you could post your current Docker-Compose file that makes use of the jwilder nginx proxy? I love that thing but I am trying to get browsersync to work and I think I am attempting to solve the same problems you solved to get your setup rolling.

I am working with ghost.org instead of wordpress but essentially everything else is the same. I assume I could make things work using your initial docker-compose / gulp setup but it seems like you have evolved since then!

Thanks in advance!

[titaniumbones]

@bretmette and others I would love to see a solution to this problem. I am trying to set up a very simple environment that my students can use on both mac and windows. I think setting a hostname in /etc/hosts is beyond their skill level. If someone else has figured out a solution I'd be very pleased to hear it.

[mkenney]

@titaniumbones I don't know that there is a self-contained solution that doesn't require configuring hosts or something more complicated.

But, I don't really understand the BrowserSync workflow. If someone would like to put together a small demo project with a note on what to expect when running it I'd be happy to look closer for a way to try and make it work.