Add docker-compose development environment

This allows a quick hands-on approach to contributing to Solidus: ```bash docker-compose up -d ``` The ruby version in the image can be configured through the `ruby_version` docker build argument (at this point, 2.7.2 is used by default): ```bash docker-compose build --build-arg ruby_version=2.6 app docker-compose up -d ``` The rails version can be set when the project is booted up: ```bash RAILS_VERSION='~> 5.0' docker-compose up -d ``` Tests can be run setting the corresponding `DB` environment variable: ```bash docker-compose exec app bin/rspec docker-compose exec app env DB=postgres bin/rspec docker-compose exec app env DB=mysql bin/rspec ``` By default, port `3000` is exposed to allow installing the sandbox application as usual. However, it can be configured through the `SANDBOX_PORT` environment variable: ```bash SANDBOX_PORT=4000 docker-compose up -d docker-compose exec app bin/sandbox docker-compose exec app bin/rails server --binding 0.0.0.0 --port 4000 ``` Some considerations: - The file `database.yml` in the Dummy app has been changed to accommodate the docker-compose setup. Taking the occasion it has been refactor to be more consistent. - The simplest configuration forces us to use `root` as the postgres user. MySQL's docker image creates the `root` user and grants all the privileges to it. Using the same user (and password) for both engines simplifies `database.yml` configuration. In case another username was to be chosen, MySQL wouldn't allow it to create the databases needed to do the testing. If we decide to change it in the future, we'll need to provision a SQL script to the image on boot time: ``` services: mysql: # ... volumes: - ./docker/provision/mysql/init:/docker-entrypoint-initdb.d ``` ```sql -- docker/provision/mysql/init/01_databases.sql CREATE DATABASE IF NOT EXISTS `solidus_api_test`; CREATE DATABASE IF NOT EXISTS `solidus_backend_test`; CREATE DATABASE IF NOT EXISTS `solidus_core_test`; CREATE DATABASE IF NOT EXISTS `solidus_frontend_test`; GRANT ALL ON `solidus_api_test`.* TO 'solidus_user'@'%'; GRANT ALL ON `solidus_backend_test`.* TO 'solidus_user'@'%'; GRANT ALL ON `solidus_core_test`.* TO 'solidus_user'@'%'; GRANT ALL ON `solidus_frontend_test`.* TO 'solidus_user'@'%' ``` - Dummy application's `database.yml` file no longer checks whether the `CI` environment variable is set. Instead, it's CircleCI configuration the one that sets `DB_USERNAME` to `root`. This is part of the work of making the file more consistent. - MySQL and Postgres hosts are now configurable through `DB_MYSQL_HOST` & `DB_POSTGRES_HOST`. This allows us link to both services from the `app` container. - At first, I explored having two different images, one for each database engine. But I find the final solution simpler. However, it's another option to consider. - The `Gemfile` has been modified to allow mysql2, postgres & sqlite3/fast_sqlite gems to be installed at the same time. They will install everything in case the `DB_ALL` environment variable is set (as we are doing in the docker-compose file). This change should be backward compatible. - A new selenium capybara driver has been added to make tests pass inside the docker container.
solidusio · Apr 19, 2021 · 7888d51 · 7888d51
1 parent 3ba6bc6
commit 7888d51
Show file tree

Hide file tree

Showing 13 changed files with 415 additions and 61 deletions.
diff --git a/.circleci/config.yml b/.circleci/config.yml
@@ -32,6 +32,7 @@ executors:
       <<: *environment
       DB: mysql
       DB_HOST: 127.0.0.1
+      DB_USERNAME: root
     docker:
       - image: *image
       - image: circleci/mysql:5.7-ram

diff --git a/.dockerdev/.psqlrc b/.dockerdev/.psqlrc
@@ -0,0 +1 @@
+\set HISTFILE ~/history/psql_history
diff --git a/.dockerdev/Dockerfile b/.dockerdev/Dockerfile
@@ -0,0 +1,57 @@
+ARG RUBY_VERSION
+FROM ruby:$RUBY_VERSION-slim-buster
+
+ARG PG_VERSION
+ARG MYSQL_VERSION
+ARG NODE_VERSION
+ARG BUNDLER_VERSION
+
+RUN apt-get update -qq \
+  && DEBIAN_FRONTEND=noninteractive apt-get install -yq --no-install-recommends \
+    build-essential \
+    gnupg2 \
+    curl \
+    git \
+    imagemagick \
+    libmariadb-dev \
+    sqlite3 \
+    libsqlite3-dev \
+    chromium \
+    chromium-driver \
+  && rm -rf /var/cache/apt/lists/*
+
+RUN curl -sSL https://www.postgresql.org/media/keys/ACCC4CF8.asc | apt-key add - \
+  && echo 'deb http://apt.postgresql.org/pub/repos/apt/ buster-pgdg main' $PG_VERSION > /etc/apt/sources.list.d/pgdg.list
+
+RUN apt-key adv --keyserver ha.pool.sks-keyservers.net --recv-keys 8C718D3B5072E1F5 \
+ && echo "deb http://repo.mysql.com/apt/debian/ buster mysql-"$MYSQL_VERSION > /etc/apt/sources.list.d/mysql.list
+
+RUN curl -sSL https://deb.nodesource.com/setup_$NODE_VERSION.x | bash -
+
+RUN apt-get update -qq && DEBIAN_FRONTEND=noninteractive apt-get -yq dist-upgrade && \
+  DEBIAN_FRONTEND=noninteractive apt-get install -yq --no-install-recommends \
+    libpq-dev \
+    postgresql-client-$PG_VERSION \
+    mysql-client \
+    nodejs \
+  &&  rm -rf /var/lib/apt/lists/*
+
+ENV APP_USER=solidus_user \
+    LANG=C.UTF-8 \
+    BUNDLE_JOBS=4 \
+    BUNDLE_RETRY=3
+ENV GEM_HOME=/home/$APP_USER/gems
+ENV APP_HOME=/home/$APP_USER/app
+ENV PATH=$PATH:$GEM_HOME/bin
+
+RUN useradd -ms /bin/bash $APP_USER
+
+RUN gem update --system \
+  && gem install bundler:$BUNDLER_VERSION \
+  && chown -R $APP_USER:$(id -g $APP_USER) /home/$APP_USER/gems
+
+USER $APP_USER
+
+RUN mkdir -p /home/$APP_USER/history
+
+WORKDIR /home/$APP_USER/app
diff --git a/.gitignore b/.gitignore
@@ -30,3 +30,4 @@ tags
 node_modules
 yarn.lock
 package-lock.json
+.env
diff --git a/Gemfile b/Gemfile
@@ -19,12 +19,13 @@ group :backend, :frontend, :core, :api do
   gem 'sprockets', '~> 3'
 
   platforms :ruby do
-    case ENV['DB']
-    when /mysql/
+    if /mysql/.match?(ENV['DB']) || ENV['DB_ALL']
       gem 'mysql2', '~> 0.5.0', require: false
-    when /postgres/
+    end
+    if /postgres/.match?(ENV['DB']) || ENV['DB_ALL']
       gem 'pg', '~> 1.0', require: false
-    else
+    end
+    if ENV['DB_ALL'] || !/mysql|postgres/.match?(ENV['DB'])
       gem 'sqlite3', require: false
       gem 'fast_sqlite', require: false
     end

diff --git a/README.md b/README.md
@@ -238,6 +238,8 @@ and/or customizations to the Solidus admin. Use at your own risk.
   cd solidus
   ```
 
+### Without Docker
+
 * Install the gem dependencies
 
   ```bash
@@ -256,6 +258,60 @@ and/or customizations to the Solidus admin. Use at your own risk.
   bin/setup
   ```
 
+### With Docker
+
+```bash
+docker-compose up -d
+```
+
+Wait for all the gems to be installed (progress can be checked through `docker-compose logs -f app`).
+
+You can provide the ruby version you want your image to use:
+
+```bash
+docker-compose build --build-arg RUBY_VERSION=2.6 app
+docker-compose up -d
+```
+
+The rails version can be customized at runtime through `RAILS_VERSION` environment variable:
+
+```bash
+RAILS_VERSION='~> 5.0' docker-compose up -d
+```
+
+Running tests:
+
+```bash
+# sqlite
+docker-compose exec app bin/rspec
+# postgres
+docker-compose exec app env DB=postgres bin/rspec
+# mysql
+docker-compose exec app env DB=mysql bin/rspec
+```
+
+Accessing the databases:
+
+```bash
+# sqlite
+docker-compose exec app sqlite3 /path/to/db
+# postgres
+docker-compose exec app env PGPASSWORD=password psql -U root -h postgres
+# mysql
+docker-compose exec app mysql -u root -h mysql -ppassword
+```
+
+In order to be able to access the [sandbox application](#sandbox), just make
+sure to provide the appropriate `--binding` option to `rails server`. By
+default, port `3000` is exposed, but you can change it through `SANDBOX_PORT`
+environment variable:
+
+```bash
+SANDBOX_PORT=4000 docker-compose up -d
+docker-compose exec app bin/sandbox
+docker-compose exec app bin/rails server --binding 0.0.0.0 --port 4000
+```
+
 ### Sandbox
 
 Solidus is meant to be run within the context of Rails application. You can

diff --git a/backend/spec/spec_helper.rb b/backend/spec/spec_helper.rb
@@ -54,6 +54,15 @@
   browser_options.args << '--window-size=1920,1080'
   Capybara::Selenium::Driver.new(app, browser: :chrome, options: browser_options)
 end
+Capybara.register_driver :selenium_chrome_headless_docker_friendly do |app|
+  browser_options = ::Selenium::WebDriver::Chrome::Options.new
+  browser_options.args << '--headless'
+  browser_options.args << '--disable-gpu'
+  # Sandbox cannot be used inside unprivileged Docker container
+  browser_options.args << '--no-sandbox'
+  browser_options.args << '--window-size=1240,1400'
+  Capybara::Selenium::Driver.new(app, browser: :chrome, options: browser_options)
+end
 
 Capybara.javascript_driver = (ENV['CAPYBARA_DRIVER'] || :selenium_chrome_headless).to_sym
 

diff --git a/bin/sandbox b/bin/sandbox
@@ -7,9 +7,15 @@ set -e
 case "$DB" in
 postgres|postgresql)
   RAILSDB="postgresql"
+  HOST=${DB_POSTGRES_HOST:-${DB_HOST}}
+  USERNAME=$DB_USERNAME
+  PASSWORD=$DB_PASSWORD
   ;;
 mysql)
   RAILSDB="mysql"
+  HOST=${DB_MYSQL_HOST:-${DB_HOST}}
+  USERNAME=$DB_USERNAME
+  PASSWORD=$DB_PASSWORD
   ;;
 sqlite|'')
   RAILSDB="sqlite3"
@@ -59,6 +65,27 @@ group :test, :development do
 end
 RUBY
 
+replace_in_database_yml() {
+  if [ $RAILSDB = "postgresql" ]; then
+    sed -i.bck "/^  adapter:/a \ \ $1:  $2" config/database.yml
+  elif [ $RAILSDB = "mysql" ]; then
+    sed -i.bck "s/^  $1:.*/\ \ $1: $2/" config/database.yml
+  fi
+  if [ -f config/database.yml.bck ]; then
+    rm -f config/database.yml.bck
+  fi
+}
+
+if [ ${HOST} ]; then
+  replace_in_database_yml "host" $HOST
+fi
+if [ ${USERNAME} ]; then
+  replace_in_database_yml "username" $USERNAME
+fi
+if [ ${PASSWORD} ]; then
+  replace_in_database_yml "password" $PASSWORD
+fi
+
 unbundled bundle install --gemfile Gemfile
 unbundled bin/rails db:drop db:create
 unbundled bin/rails generate solidus:install \

diff --git a/core/lib/generators/spree/dummy/templates/rails/database.yml b/core/lib/generators/spree/dummy/templates/rails/database.yml
@@ -1,66 +1,111 @@
 <% if agent_number = ENV['TC_AGENT_NUMBER']
 database_prefix = agent_number + '_'
 end %>
+<% db = case ENV['DB']
+        when 'mysql'
+          'mysql'
+        when 'postgres', 'postgresql'
+          'postgres'
+        when 'sqlite', '', nil
+          'sqlite'
+        else
+          raise "Invalid DB specified: #{ENV['DB']}"
+        end %>
+<% db_host = case db
+             when 'mysql'
+               ENV['DB_MYSQL_HOST'] || ENV['DB_HOST']
+             when 'postgres'
+               ENV['DB_POSTGRES_HOST'] || ENV['DB_HOST']
+             else
+               ENV['DB_HOST']
+             end %>
+<% db_username = ENV['DB_USERNAME'] %>
+<% db_password = ENV['DB_PASSWORD'] %>
+
+
+
+
 <% case ENV['DB']
-  when 'sqlite' %>
-development:
-  adapter: sqlite3
-  database: db/solidus_development.sqlite3
-test:
-  adapter: sqlite3
-  database: db/solidus_test.sqlite3
-  timeout: 10000
-production:
-  adapter: sqlite3
-  database: db/solidus_production.sqlite3
-<% when 'mysql' %>
+  when 'mysql' %>
 development:
   adapter: mysql2
   database: <%= database_prefix %><%= options[:lib_name] %>_solidus_development
+  <% unless db_username.blank? %>
+  username: <%= db_username %>
+  <% end %>
+  <% unless db_password.blank? %>
+  password: <%= db_password %>
+  <% end %>
+  <% unless db_host.blank? %>
+  host: <%= db_host %>
+  <% end %>
   encoding: utf8
 test:
   adapter: mysql2
-  <% if ENV['TRAVIS'] %>
-  username: root
-  password:
-  <% end %>
   database: <%= database_prefix %><%= options[:lib_name] %>_solidus_test
+  <% unless db_username.blank? %>
+  username: <%= db_username %>
+  <% end %>
+  <% unless db_password.blank? %>
+  password: <%= db_password %>
+  <% end %>
+  <% unless db_host.blank? %>
+  host: <%= db_host %>
+  <% end %>
   encoding: utf8
 production:
   adapter: mysql2
   database: <%= database_prefix %><%= options[:lib_name] %>_solidus_production
+  <% unless db_username.blank? %>
+  username: <%= db_username %>
+  <% end %>
+  <% unless db_password.blank? %>
+  password: <%= db_password %>
+  <% end %>
+  <% unless db_host.blank? %>
+  host: <%= db_host %>
+  <% end %>
   encoding: utf8
 <% when 'postgres', 'postgresql' %>
-<% db_host = ENV['DB_HOST'] -%>
-<% db_username = ENV['DB_USERNAME'] -%>
-<% db_password = ENV['DB_PASSWORD'] -%>
 development:
   adapter: postgresql
   database: <%= database_prefix %><%= options[:lib_name] %>_solidus_development
-  username: postgres
-  min_messages: warning
-<% unless db_host.blank? %>
+  <% unless db_username.blank? %>
+  username: <%= db_username %>
+  <% end %>
+  <% unless db_password.blank? %>
+  password: <%= db_password %>
+  <% end %>
+  <% unless db_host.blank? %>
   host: <%= db_host %>
-<% end %>
+  <% end %>
+  min_messages: warning
 test:
   adapter: postgresql
   database: <%= database_prefix %><%= options[:lib_name] %>_solidus_test
-  username: <%= db_username || 'postgres' %>
-<% unless db_password.blank? %>
+  <% unless db_username.blank? %>
+  username: <%= db_username %>
+  <% end %>
+  <% unless db_password.blank? %>
   password: <%= db_password %>
-<% end %>
-  min_messages: warning
-<% unless db_host.blank? %>
+  <% end %>
+  <% unless db_host.blank? %>
   host: <%= db_host %>
-<% end %>
+  <% end %>
+  min_messages: warning
 production:
   adapter: postgresql
   database: <%= database_prefix %><%= options[:lib_name] %>_solidus_production
-  username: postgres
-  min_messages: warning
-<% unless db_host.blank? %>
+  <% unless db_username.blank? %>
+  username: <%= db_username %>
+  <% end %>
+  <% unless db_password.blank? %>
+  password: <%= db_password %>
+  <% end %>
+  <% unless db_host.blank? %>
   host: <%= db_host %>
-<% end %>
+  <% end %>
+  min_messages: warning
 <% when 'sqlite', '', nil %>
 development:
   adapter: sqlite3
@@ -71,6 +116,4 @@ test:
 production:
   adapter: sqlite3
   database: db/solidus_production.sqlite3
-<% else %>
-  <% raise "Invalid DB specified: #{ENV['DB']}" %>
 <% end %>