Updated docker-compose to 3.0, use of .env file, and use of a custom network #195

joserprieto · 2017-11-11T20:07:07Z

Hi;

This may be useful for your;

I updated the docker-compose.yml file, to docker-compose 3.0 specs.
Avoid the use of "enviroment" key, and isolate the enviroment vars in a separated file (I versioned a .env.example only).
Use of a custom network, for better management in the host running docker.

If you want, i can edit the readme and document the use and changes, and why is better this approach.

Regards.

- Enable the use of a .env file, and remove the enviroment sections on the docker-compose.yml file. - Enable a custom network, for the complete stack.

# Conflicts: # .env.sample

- Enable the use of a .env file, and remove the enviroment sections on the docker-compose.yml file. - Enable a custom network, for the complete stack.

joserprieto · 2017-11-11T20:08:53Z

Tested in:

Linux Mint 18.2, and last versions of Docker and docker-compose.

Debian 9 and last versions of Docker and docker-compose.

mattermod · 2017-11-11T20:09:34Z

Thanks @joserprieto for the pull request!

Per the CONTRIBUTING.md file displayed when you created this pull request, we need to add you to the list of approved contributors for the Mattermost project.

Please help complete the Mattermost contribution license agreement?

This is a standard procedure for many open source projects. Your form should be processed within 24 hours and reviewers for your pull request will be able to proceed.

Please let us know if you have any questions.

We are very happy to have you join our growing community! If you're not yet a member, please consider joining our Contributors community channel to meet other contributors and discuss new opportunities with the core team.

pichouk · 2017-11-11T20:17:25Z

Thanks @joserprieto !
I'm quite busy for now but I promise I'll take a look :)

pichouk · 2017-11-14T21:41:25Z

docker-compose.yml

    restart: unless-stopped
+    links:


We removed links in PR #185 because it is deprecated. Should remove it I guess.

The links is because you can assign one of more aliases for the different hosts; i think that, for the moment, docker-compose will mantain retrocompatibility

pichouk · 2017-11-14T21:46:48Z

So we're back on the same question again @jasonblais and @xcompass : should we move to Docker Compose 3 ?

Maybe it is a good idea. Actually, the docker-compose.yml file provided in this repository is just a cool tool we provide to help people deploying with Docker. In fact, it is quite easy to build Docker images (using Dockerfiles) and deploy without Docker Compose (or with an old version). I did it for my production server because I have some custom needs, and this is not something really hard if you have some basics with Docker.
My point is : the docker-compose.yml file on this repository is not that critical. In few minutes you can easily create a new one, matching your needs. So changing version of Docker Compose is okay as soon as we provide correct documentation on how to build the Docker images.

jasonblais · 2017-11-14T21:51:39Z

@pichouk I forget the previous discussion, but what were the potential side effects of moving to Docker Compose v3? (Link to a previous post is okay, I wasn't able to find it)

And conversely, what is the benefit of moving to v3? At first it was to support swarm cluster I think, but that is resolved with #176

xcompass · 2017-11-14T21:54:48Z

docker-compose.yml

-      # args:
-      #   - edition=team
+      args:
+        - edition=team


we should not change the default behavior.

Ok.
For, then, the docker-compose.yml don't be versioned.

xcompass · 2017-11-14T21:55:09Z

docker-compose.yml

-    #  - AWS_REGION=us-east-1
+    env_file:
+      - .env
+    networks:


What's the rationale to use custom network?

For many reasons, you can want to have under control the IPs and the ports exposed outside, and so on.

xcompass · 2017-11-14T21:59:49Z

.env.sample

@@ -0,0 +1,29 @@
+#!/usr/bin/env bash


Could you also add some instruction to copy the sample file to .env?

xcompass · 2017-11-14T22:06:34Z

docker-compose.yml

@@ -1,53 +1,68 @@
-version: "2"
+version: '3'


Which docker-compose directive requires v3 format? If we upgrade the compose files to v3, it will break backward compatibility. Not sure if it is good time to upgrade to v3 and if we decide to go with v3, we need to list it as a breaking change.

pichouk · 2017-11-14T22:07:42Z

@jasonblais We add some discussions about this subject on #151 and this forum post (and a little on #104 and #147).

Yes the main benefit to moving to v3 is the swarm support, but it was solved. So, the "only" benefit now is to be up to date on our default configuration to deploy with Docker. I have no particular interest to moving on v3. It just not the first time someone submit the idea ;)

jasonblais · 2017-11-16T19:30:33Z

Thanks! I'm okay switching to v3, don't see major issues and seems to be the preference of the community @pichouk

jasonblais · 2017-11-16T19:34:08Z

As long as we update the documentation accordingly, which you've taken care of here :)

Can you also help take a look at

to see if anything needs to be updated?

pichouk · 2017-11-18T15:53:12Z

@jasonblais Great :)
We will work on this PR with @joserprieto since there is some things to correct/change, but when it will be ready I'll take particular attention to Mattermost documentation and will update it :)

joserprieto · 2017-11-30T21:01:34Z

I read all the discussions, and i think that here are a debate of use version 2 or version 3 of docker-compose files.

The solution can be easy; put docker-compose.v2.yml.sample and docker-compose.v3.yml.sample, and the correspondent instructions on the README.

I make a PR right now with this approach in mind.

….gitignore

…ml, and updated readme, and .gitignore

joserprieto · 2017-11-30T21:39:48Z

Changes done!

Please, review, @pichouk @xcompass

Regards.

joserprieto · 2017-11-30T21:44:45Z

@pichouk about the main documentation, that remarks @jasonblais, i think that:

https://docs.mattermost.com/install/prod-docker.html

Deploy the Mattermost Production Docker setup by running:

git clone https://github.com/mattermost/mattermost-docker.git
cd mattermost-docker
docker-compose build
docker-compose up -d

Can be:

Deploy the Mattermost Production Docker setup by running:

git clone https://github.com/mattermost/mattermost-docker.git
cd mattermost-docker
open the README and follow instructions to select docker-compose version and for the enviroment vars.
docker-compose build
docker-compose up -d

The other link:

https://docs.mattermost.com/install/docker-local-machine.html

Is for use on a dev environment, not for make all the stack and so on.

Regards!

xcompass · 2017-12-01T01:00:30Z

@joserprieto Thanks. I would prefer to leave a docker-compose yaml file as default one (maybe v3 one). And let people to copy v2 one or use -f option if they need v2 compose file.

pichouk · 2017-12-05T21:15:54Z

I agree with @joserprieto now. We will never achieve to have a "perfect docker-compose.yml that match everyone needs". Maybe we should provide multiple example files and let users choose what they want to use.
Also it will force users to understand what they are doing, and avoid to just run git clone && docker-compose up -d, that can lead to situations where a user doesn't understand how to upgrade/where data is stored, etc.

It will not be a breaking change. Current users will still have their docker-compose.yml file so it will not break their setup.

xcompass · 2017-12-07T00:50:45Z

@pichouk when you remove the docker-compose.yml from this repo, wouldn't it also remove from the local repo when one pulls the change (assume no change in the file)?

pichouk · 2017-12-08T20:32:49Z

@xcompass Hum you're right. I just tested it and if we want to remove the docker-compose.yml file and add it to .gitignore, it will remove the file on local repositories when pulling only if you never change the file.
If you modify this file (to change default password for example) you will not be able to pull, so it will not remove it. So everybody have to backup the file, pull the change, and restore it.

joserprieto · 2017-12-09T10:39:37Z

I'm agree with @pichouk.
And, remember, this docker / stack is for production purpose; so, the normal behaviour is to change default passwords on the docker-compose.yml (passwords that will be isolated, on a .env file, like in my PR).

I think that a votation may resolve this question; logically, i can't vote, but the mattermost team's votes can resolve all of this.

Regards.

pichouk · 2017-12-12T20:02:59Z

Good for me. If @jasonblais and @xcompass are ok, then you can update your PR to match this new organization.
We will take care that it didn't broke current users installations and that documentation is correctly updated.

joserprieto · 2017-12-12T21:17:53Z

About of to not broke current users installations; i can see two scenarios:

User changed the docker-compose.yml file (the default behaviour, i hope...); so, this user can't make a git pull and not loses his configuration (on his docker-compose.yml file); when he makes a git pull, it has to force it, and discard changes on his docker-compose.yml file (so, this new approach is good for him).
User not changed the docker-compose.yml file; yes, in this case, if we put the docker-compose.yml on the .gitignore file, they docker-compose.yml file will "disappears" whith git pull.. But i think that is a very helpful behaviour, because forces the user to change default users and passwords (that are in the docker-compose.yml file already).

A correct approach (imho), it's to isolate the vars outside the docker-compose.yml file (on a .env file gitignored too), and isolate the production ready docker-compose.yml, that defines the stack (so, it's a configuration file, in a certain way..)

I think that is this approach is reflected on the documentation, the user will not have trouble with the change.

Regards.

pdemagny · 2017-12-15T20:37:48Z

Any news on a merge for this PR ? Once its merged i'll start to work on #203 :)

pichouk · 2017-12-15T20:59:31Z

@joserprieto can submit changes describe in this comment and, as soon as @xcompass and/or @jasonblais approve this changes, we can merge :)

joserprieto · 2017-12-16T10:31:21Z

The use of docker volumes, or Traefik, i think that is a personal election.

For example; on the current stack definition, we have:

    volumes:
      - ./volumes/app/mattermost/config:/mattermost/config:rw
      - ./volumes/app/mattermost/data:/mattermost/data:rw
      - ./volumes/app/mattermost/logs:/mattermost/logs:rw
      - /etc/localtime:/etc/localtime:ro

    volumes:
      # This directory must have cert files if you want to enable SSL
      - ./volumes/web/cert:/cert:ro
      - /etc/localtime:/etc/localtime:ro

And yes, we can use something like that:

version: "3"
services:
(...)
    volumes:
      - mttm-config:/mattermost/config:rw
      - mttm-data:/mattermost/data:rw
      - mttm-logs:/mattermost/logs:rw
      - etc-localtime:/etc/localtime:ro
(...)

volumes:
  mttm-config:
  mttm-data:
  mttm-logs:
  web-cert:
  etc-localtime:

But it's the same of the discussion on other threads; is better to use volumes? is better to use custom networks? is better to use compose files v2 or v3?

The answer is easy; depends of the use case.

With traefick, is the same.

I think that put sample docker-compose files, is the best choice; and, with this, any user can take is own decissions, taking his/her use case.

joserprieto · 2017-12-16T10:38:15Z

@pichouk , if i'm not wrong, i made the changes.

I resume changes made:

Created two separated files; docker-compose-v3.yml.sample and docker-compose-v2.yml-sample.
Updated the v2 to get the advantange of isolated environment vars in a .env file.
Updated the README with the instructions to use the docker-compose that you choose, and the .env file that you need.
gitignored files .env and docker-compose.yml

I think that is all done... But, if any change is needed, please, tell me.

joserprieto · 2017-12-16T10:47:02Z

Another changes that can will add, is the explanation of how isolated the .env file and docker-compose.yml file.

One option (as valid as another), is take advantage of git hooks. And do something like:

Put the .env and the docker-compose.yml files outside the repo directory.
Make symlinks to this files, on the repo dir.
Before a pull/push, remove this symlinks.
After the pull/push, make the symlinks again.

Another form, is make a simple bash script outside the repo dir, that do something like "setup.sh --enable" and "setup.sh --disable", for put symlinks / delete symlinks, before/after pull/push.

Maybe useful to explain this on the README. Or, it can be useful on the docs. As you wish 😉

pdemagny · 2017-12-16T13:14:28Z

Well, for "simple" compose files (debate v2/v3), i think that going with the flow is important here, v2 is getting really old and should be present only for backward compatible reasons.

Regarding the "Volumes" debate, a quick look at docker doc is enough for me:

Volumes are the preferred mechanism for persisting data generated by and used by Docker containers. While bind mounts are dependent on the directory structure of the host machine, volumes are completely managed by Docker. Volumes have several advantages over bind mounts ...

I read a lot of people being reluctant to move to docker volumes because they think its easier to backup, why would it be ? Docker volumes are stored in a predictible directory on the host ... (/var/lib/docker/volumes) and are named like you named them in the compose file.
They offer better integration with docker, better isolation & avoid dirty permissions hacks.
And btw using local mounts tied to one host makes no sense in a multi-node swarm where you want your storage to be distributed & redundant over multiple nodes, or at least portable.

Same goes for custom networks, they offer better isolation for services, while still allowing them to talk to each other if the user wants to.

As for the env file, it is a best practice that i'm going to apply on docker-stack.yml file, but we also should promote the use of docker secrets for passwords in this file.

And for Traefik, yes it is a personal choice, but it offers this choice to the users of this repo, and doesn't prevent from adding more choices for them in the future by adding more compose/stack file with support for more load-balancers ...
Having a single node swarm is almost pointless, people running swarms have these kind of services, we simply provide them with a good example here.
Traefik is a container ready solution, unlike things such as nginx-proxy, and offers a quick and nice way of handling https by automating creation & renewal of ssl certificates.

For all of these reasons, i completely back the solution with multiple yaml "sample" files in the root of the repo, with something like:

docker-compose.yml (v3, volumes)
docker-compose-v2.yml (v2, mounts, for backward compatibility)
docker-stack.yml (v3 with volumes, configs, secrets, env_file, network)
docker-stack-with_traefik.yml (same but with traefik & letsencrypt)

I'm willing to do this after the merge of this PR by extending the work i've done on docker-stack.yml file, depending of what you decide, just let me know ;)

pichouk · 2017-12-16T18:53:48Z

.env.sample

+
+# See https://docs.docker.com/compose/environment-variables/#the-env-file
+
+# PostgreSQL Enviroment Variables:


Typo Enviroment => environment

pichouk · 2017-12-16T18:54:06Z

.env.sample

+POSTGRES_DB=mattermost
+#PGDATA="/path/to/your/postgresql/data/files | default: /var/lib/postgresql/data/pgdata optional"
+#POSTGRES_DB="name_of_your_postgresql_db | default: postgres_user value | optional
+# uncomment the following to enable backupe


pichouk · 2017-12-16T18:56:17Z

docker-compose.v2.yml.sample

@@ -50,4 +41,4 @@ services:
      - /etc/localtime:/etc/localtime:ro
    # Uncomment for SSL
    # environment:
-    #  - MATTERMOST_ENABLE_SSL=true
+    #  - MATTERMOST_ENABLE_SSL=true


Please keep the newline :)

pichouk · 2017-12-16T18:59:28Z

.gitignore

+
+# Specific files:
+
+docker-compose.yml


Please keep a newline :)

pichouk · 2017-12-16T19:00:32Z

README.md

+* https://forum.mattermost.org/t/mattermost-with-docker-discussion-about-requirements/3518
+* https://github.com/mattermost/mattermost-docker/issues/151
+
+You only have to do is:


is -> this

pichouk · 2017-12-16T19:01:43Z

README.md

@@ -83,6 +132,14 @@ Put your SSL certificate as `./volumes/web/cert/cert.pem` and the private key th
 no password as `./volumes/web/cert/key-no-password.pem`. If you don't have
 them you may generate a self-signed SSL certificate.

+### Building images
+
+Before start the containers, we must to build the images:


we must to build -> you must build

pichouk · 2017-12-16T19:02:03Z

@joserprieto Great :)
I made a review, I'll like @xcompass to review too.
Also, it is important to update documentation on this page. So if you can submit a PR on this file too I'll appreciate :)

To add new features (as @pdemagny suggest) I prefer to wait for another PR, but it is a good idea :)

Also, thanks for your proposal, it is a great improvement :)

joserprieto · 2017-12-17T13:32:19Z

@pichouk i think that i made all fixes.
Review and if you see more errors, please, tell me.

About the documentation, i can propose here the new doc page, and if you like it, i wll make the PR.

Do you agree in proceed on this way?

Regards.

pichouk · 2017-12-17T18:23:59Z

LGTM, thanks a lot :)
@joserprieto I'm okay to do it this way !

joserprieto · 2018-02-02T13:28:51Z

But, this PR is still open?
Why?

pichouk · 2018-02-02T18:18:13Z

@joserprieto You proposed to submit a PR to Mattermost documentation before merging this one.

joserprieto · 2018-02-02T18:45:18Z

Ah, ok.
I'll try to do it this weekend

mattermod · 2019-03-12T00:00:23Z

This issue has been automatically labelled "stale" because it hasn't had recent activity.
A core team member will check in on the status of the PR to help with questions.
Thank you for your contribution!

/cc @jasonblais @hanzei

hanzei · 2019-03-12T07:24:23Z

Hey @pichouk,

What is the status of this PR? Can it be merged after it has been synced with master?

Would you help with the changes required in the docs?

pichouk · 2019-03-12T07:39:37Z

We are dropping the support of the docker-compose.yml file so I guess we can close this PR.

joserprieto added 5 commits November 9, 2017 14:26

Added "links" section, for link and alias of hosts.

ca4999c

- Updated docker-compose.yml to 3.0 format.

54ff94b

- Enable the use of a .env file, and remove the enviroment sections on the docker-compose.yml file. - Enable a custom network, for the complete stack.

Merge remote-tracking branch 'origin/master'

3d0b515

# Conflicts: # .env.sample

Updated travis link.

b970a71

- Updated docker-compose.yml to 3.0 format.

6239e8a

- Enable the use of a .env file, and remove the enviroment sections on the docker-compose.yml file. - Enable a custom network, for the complete stack.

pichouk requested review from xcompass and pichouk November 11, 2017 20:16

pichouk added Enhancement 2: Dev Review Requires review by a core committer waiting-for-cla labels Nov 11, 2017

pichouk suggested changes Nov 14, 2017

View reviewed changes

xcompass suggested changes Nov 14, 2017

View reviewed changes

pichouk added Awaiting Submitter Action Blocked on the author and removed waiting-for-cla labels Nov 19, 2017

joserprieto added 3 commits November 30, 2017 22:26

Move docker-compose files to "sample" files, updated readme, and the …

67da891

….gitignore

Move docker-compose files to "sample" files, removed docker-compose.y…

5769527

…ml, and updated readme, and .gitignore

Move docker-compose files to "sample" files, removed docker-compose.y…

c33863a

…ml, and updated readme, and .gitignore

jasonblais mentioned this pull request Dec 12, 2017

Images App Alpine, optimization #208

Merged

pdemagny mentioned this pull request Dec 15, 2017

Use Docker volumes #203

Open

pichouk suggested changes Dec 16, 2017

View reviewed changes

fixed typo and style errors

df6d579

pichouk approved these changes Dec 17, 2017

View reviewed changes

pichouk mentioned this pull request Mar 1, 2018

Production Hardening #241

Merged

pichouk mentioned this pull request Feb 9, 2019

WIP Deprecate the web image and the default Docker Compose file #366

Closed

5 tasks

mattermod added the Lifecycle/1:stale label Mar 12, 2019

pichouk closed this Mar 12, 2019


		# See https://docs.docker.com/compose/environment-variables/#the-env-file

		# PostgreSQL Enviroment Variables:

Updated docker-compose to 3.0, use of .env file, and use of a custom network #195

Updated docker-compose to 3.0, use of .env file, and use of a custom network #195

Conversation

joserprieto commented Nov 11, 2017 • edited Loading

joserprieto commented Nov 11, 2017

mattermod commented Nov 11, 2017

pichouk commented Nov 11, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

pichouk commented Nov 14, 2017

jasonblais commented Nov 14, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

pichouk commented Nov 14, 2017

jasonblais commented Nov 16, 2017

jasonblais commented Nov 16, 2017

pichouk commented Nov 18, 2017

joserprieto commented Nov 30, 2017

joserprieto commented Nov 30, 2017

joserprieto commented Nov 30, 2017

xcompass commented Dec 1, 2017

pichouk commented Dec 5, 2017

xcompass commented Dec 7, 2017

pichouk commented Dec 8, 2017

joserprieto commented Dec 9, 2017

pichouk commented Dec 12, 2017

joserprieto commented Dec 12, 2017 • edited Loading

pdemagny commented Dec 15, 2017

pichouk commented Dec 15, 2017

joserprieto commented Dec 16, 2017 • edited Loading

joserprieto commented Dec 16, 2017 • edited Loading

joserprieto commented Dec 16, 2017

pdemagny commented Dec 16, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

pichouk commented Dec 16, 2017 • edited Loading

joserprieto commented Dec 17, 2017

pichouk commented Dec 17, 2017

joserprieto commented Feb 2, 2018

pichouk commented Feb 2, 2018

joserprieto commented Feb 2, 2018

mattermod commented Mar 12, 2019

hanzei commented Mar 12, 2019 • edited Loading

pichouk commented Mar 12, 2019

joserprieto commented Nov 11, 2017 •

edited

Loading

joserprieto commented Dec 12, 2017 •

edited

Loading

joserprieto commented Dec 16, 2017 •

edited

Loading

joserprieto commented Dec 16, 2017 •

edited

Loading

pdemagny commented Dec 16, 2017 •

edited

Loading

pichouk commented Dec 16, 2017 •

edited

Loading

hanzei commented Mar 12, 2019 •

edited

Loading