Add docker-compose development environment #3947
Conversation
There was a problem hiding this comment.
Hey @waiting-for-dev, this is great. I'm not a Docker expert but this seems to be well done, thanks!
I left some preliminary questions + I think we should improve the documentation by adding some notes about Docker setup in Solidus Guides here: https://guides.solidus.io/developers/getting-started/develop-solidus.html.
|
Thanks for your review @kennyadsl .
I updated the guides, thanks for pointing me to them. I have addressed the issues separately in different commits. If it's ok with you I'll squash them together when everything is ready for release. |
|
There are some good pointers here https://evilmartians.com/chronicles/ruby-on-whales-docker-for-ruby-rails-development and https://evilmartians.com/chronicles/reusable-development-containers-with-docker-compose-and-dip that we could implement here too. Keeping the docker image small and running fast is key for adaption I think. Using volumes and the correct layering gives a lot of benefit. Having this available is awesome, thanks for that |
Thank you @peterberkenbosch for sharing, that's really good stuff. I think we could adopt some of the ideas, while I'd ignore others. I'll try to go through all of them so we can all agree on what to take. Please, tell me if I overlooked something. ❌ Using dip. It's an opinionated tool that anyway can be used orthogonally with our setup (but looks really cool, I'll study adopting it myself, so thanks for letting me know about it). ✔️ Using slim image. SURE! ❌ Specifying Debian distribution. I'm not sure it's a good idea. It adds more maintenance burden for a benefit that's not too clear for me. ✔️ Installing postgres-client in the image. Yeah, makes sense. ✔️ Setting default values for build arguments from docker-compose. ✔️ Lock yarn & node versions. ✔️ Lock bundler version. ❌ Install git. I don't think most people use git from within the container. But more on this later. ✔️ Remove apt records. Makes sense. ✔️ noninteractive apt frontend. Makes sense. ✔️ Don't install apt recommendations. Makes sense. ✔️ & ❌ Using the ✔️ Use ❌ Define multiple services based on ✔️ Tagging the image. Definitely!!! ✔️ ✔️ Keep psql history. Cool! ✔️ Keep bash history. Even cooler! ❌ Setting default editor. It's user-dependent and can be configured using the ✔️ Health checks. I didn't know about it. Makes sense! |
|
Awesome @waiting-for-dev, this helps a lot for development. 👍 Also really like your reply and rational for your choices here. Love it. |
|
@peterberkenbosch I updated the configuration following some of the tips from the article. Some changes from what I said in the previous comment:
Please, look at 737464d |
There was a problem hiding this comment.
Nice work @waiting-for-dev!
I just added some minor considerations but looks good to me.
docker-compose.yml
Outdated
| POSTGRES_USER: root | ||
| POSTGRES_PASSWORD: password | ||
| volumes: | ||
| - postgres:/var/lib/postgresql/data |
There was a problem hiding this comment.
A curiosity here: is there a special reason to have a named volume for DB data?
There was a problem hiding this comment.
It's so that we can keep the data in case we remove the container.
waiting-for-dev
force-pushed
the
docker
branch
from
6a01699
to
591f6cc
Compare
|
In 591f6cc I have adapted the I have removed the reference to Travis as I think we're not using it anymore. I also have removed a duplicated configuration for sqlite. As @kennyadsl said, cleaning and unifying how the |
See solidusio/solidus#3947 for details on the decisions that have been taken. We are also decoupling the source of solidus & solidus_i18n dependencies, and providing more flexibility by allowing the specification of both repository & branch for each project (therefore allowing working with forks).
|
I added a new commit that allows configuring the exposed port (defaults to SANDBOX_PORT=4000 docker-compose up -d
docker-compose exec app rails server --binding 0.0.0.0 --port 4000 |
|
I added a new commit that generates the sandbox application taking into account what is configured as host/username/password for the given database engine. |
See solidusio/solidus#3947 for details on the decisions that have been taken. We are also decoupling the source of solidus & solidus_i18n dependencies, and providing more flexibility by allowing the specification of both repository & branch for each project (therefore allowing working with forks).
There was a problem hiding this comment.
I'm not a docker user but this seems legit if we want to be more friendly with who does. Thanks, @waiting-for-dev!
This allows a quick hands-on approach to contributing to Solidus:
```bash
docker-compose up -d
```
The ruby version in the image can be configured through the
`ruby_version` docker build argument (at this point, 2.7.2 is used by
default):
```bash
docker-compose build --build-arg ruby_version=2.6 app
docker-compose up -d
```
The rails version can be set when the project is booted up:
```bash
RAILS_VERSION='~> 5.0' docker-compose up -d
```
Tests can be run setting the corresponding `DB` environment variable:
```bash
docker-compose exec app bin/rspec
docker-compose exec app env DB=postgres bin/rspec
docker-compose exec app env DB=mysql bin/rspec
```
By default, port `3000` is exposed to allow installing the sandbox
application as usual. However, it can be configured through the
`SANDBOX_PORT` environment variable:
```bash
SANDBOX_PORT=4000 docker-compose up -d
docker-compose exec app bin/sandbox
docker-compose exec app bin/rails server --binding 0.0.0.0 --port 4000
```
Some considerations:
- The file `database.yml` in the Dummy app has been changed to
accommodate the docker-compose setup. Taking the occasion it has been
refactor to be more consistent.
- The simplest configuration forces us to use `root` as the postgres
user. MySQL's docker image creates the `root` user and grants all the
privileges to it. Using the same user (and password) for both engines
simplifies `database.yml` configuration. In case another username was to
be chosen, MySQL wouldn't allow it to create the databases needed to do
the testing. If we decide to change it in the future, we'll need to
provision a SQL script to the image on boot time:
```
services:
mysql:
# ...
volumes:
- ./docker/provision/mysql/init:/docker-entrypoint-initdb.d
```
```sql
-- docker/provision/mysql/init/01_databases.sql
CREATE DATABASE IF NOT EXISTS `solidus_api_test`;
CREATE DATABASE IF NOT EXISTS `solidus_backend_test`;
CREATE DATABASE IF NOT EXISTS `solidus_core_test`;
CREATE DATABASE IF NOT EXISTS `solidus_frontend_test`;
GRANT ALL ON `solidus_api_test`.* TO 'solidus_user'@'%';
GRANT ALL ON `solidus_backend_test`.* TO 'solidus_user'@'%';
GRANT ALL ON `solidus_core_test`.* TO 'solidus_user'@'%';
GRANT ALL ON `solidus_frontend_test`.* TO 'solidus_user'@'%'
```
- Dummy application's `database.yml` file no longer checks whether the
`CI` environment variable is set. Instead, it's CircleCI configuration
the one that sets `DB_USERNAME` to `root`. This is part of the work of
making the file more consistent.
- MySQL and Postgres hosts are now configurable through `DB_MYSQL_HOST` &
`DB_POSTGRES_HOST`. This allows us link to both services from the
`app` container.
- At first, I explored having two different images, one for each database
engine. But I find the final solution simpler. However, it's another
option to consider.
- The `Gemfile` has been modified to allow mysql2, postgres &
sqlite3/fast_sqlite gems to be installed at the same time. They will
install everything in case the `DB_ALL` environment variable is set (as
we are doing in the docker-compose file). This change should be backward
compatible.
- A new selenium capybara driver has been added to make tests pass
inside the docker container.
Thanks @kennyadsl . I rebased from master and I added a new line to the Dockerfile: It adds gem executables to the PATH so that we can, for instance, run |
See solidusio/solidus#3947 for details on the decisions that have been taken. We are also decoupling the source of solidus & solidus_i18n dependencies, and providing more flexibility by allowing the specification of both repository & branch for each project (therefore allowing working with forks).
See solidusio/solidus#3947 for details on the decisions that have been taken. We are also decoupling the source of solidus & solidus_i18n dependencies, and providing more flexibility by allowing the specification of both repository & branch for each project (therefore allowing working with forks).
See solidusio/solidus#3947 for some of the decisions taken
This allows a quick hands-on approach to contributing to Solidus:
The ruby version in the image can be configured through the
ruby_versiondocker build argument (2.7.2 is used by default):The rails version can be set when the project is booted:
RAILS_VERSION='~> 5.0' docker-compose up -dTests can be run setting the corresponding
DBenvironment variable:Port
3000is exposed to allow installing the sandbox application asusual.
Some noticeable considerations:
The file
database.ymlin the Dummy app has been changed toaccommodate the docker-compose setup. Taking the occasion it has been
refactor to be more consistent. The changes should not affect any local
setup, but it's worth that some people check their setup keeps working
without issues.
The simplest configuration forces us to use
rootas the postgresuser. MySQL's docker image creates the
rootuser and grants all theprivileges to it. Using the same user (and password) for both engines
simplifies
database.ymlconfiguration. In case another username was tobe chosen, MySQL wouldn't allow it to create the databases needed to do
the testing. If we decide to change it in the future, we would need to
provision a SQL script to the image on boot time:
Dummy application's
database.ymlfile no longer checks whether theCIenvironment variable is set. Instead, it's CircleCI configurationthe one that sets
DB_USERNAMEtoroot. This is part of the work ofmaking the file more consistent.
At first, I explored having two different images, one for each database
engine. But I found the final solution simpler. However, it's another
option to consider.
The
Gemfilehas been modified to allow mysql2, postgres &sqlite3/fast_sqlite gems to be installed at the same time. They will
install everything in case the
DB_ALLenvironment variable is set (aswe are doing in the docker-compose file). This change should be backward
compatible.
A new selenium capybara driver has been added to make tests pass
inside the docker container.
Description
Checklist: