# magento-docker `magento-docker` is Docker environment for easy to setup, configure, debug Magento2 + Storefront instance with varnish, elasticsearch, redis, rabbit and mftf tests. ### Requirements * [Git](https://git-scm.com/book/en/v2/Getting-Started-Installing-Git) * [Docker](https://docs.docker.com/) * [Docker Compose](https://docs.docker.com/compose/install/) * Setup SSH-keys on your github account. (see [docs](https://help.github.com/en/github/authenticating-to-github/generating-a-new-ssh-key-and-adding-it-to-the-ssh-agent) for [help](https://help.github.com/en/github/authenticating-to-github/adding-a-new-ssh-key-to-your-github-account)) * (optional - for Mutagen installation only) Install Mutagen [docs](https://mutagen.io/documentation/introduction/installation) * Ensure you do not have `dnsmasq` installed/enabled locally (will be auto-installed if you've use Valet+ to install Magento) ### How to install #### Steps 1. Create a directory where all repositories will be cloned (used in your IDE) Proposed structure: ``` ~/projects/storefront/magento-docker # this repo ~/projects/storefront/repos # directory with repositories ``` 2. Update MAGENTO_PATH in .env with recently created directory path 3. Add $MAGENTO_DOMAIN from .env to hosts, e.g.: ``` sudo -- sh -c "echo '127.0.0.1 magento.test >> /etc/hosts" ``` 4. Add Magento authentication keys to access the Magento Commerce repository * copy the contents of `etc/php/auth.json.dist` to new `etc/php/auth.json` file and replace placeholders with your credentials You have two options to install the project #### docker-compose based installation This approach will use only docker-compose to install Storefront project. Files between the host and guest are synced with "delegated" options (see FILE_SYNC in .env) - RUN `git checkout docker-compose.yml` - optional: reset changes if any present in docker-compose.yml - RUN `bash ./init_project` - clone repos (if RECLONE=YES in .evn) - RUN `docker-compose up -d` - up services - RUN `bash ./reinstall` - install/reinstall Magento with repositories provided in INSTALLED_REPOS (see .env) #### mutagen based installation This approach will use mutagen service to share code between the host and guest. Mutagen will significantly speedup work of Magento instance Before run, set `MUTAGEN_INSTALLATION=YES` in .env 4. RUN `mutagen project start` Note, for the first installation (when you don't have cloned repositories yes) please change settings "RECLONE" to "yes" in ".env" file ### Configuration MAGENTO_PATH=/magento/magento-docker-install # local directory to clone repos into RECLONE=no # flag indicate whether do re-clon of all repos or no MAGENTO_EDITION=EE # EE|B2B Notices: ### gRPC performance testing Preconditions: - Product with id "1" is created - Verify `DOCKER_PATH` in the `.env` file Ro run gRPC benchmark run `mutagen project run grpc-run-bench` or `docker-compose -f bundles/grpc-testing.yml up` Change test scenario if needed in magento-docker/etc/grpc/bench-test-config.json ### Troubleshooting 1. Add MAGENTO_PATH path to Docker sharing folders (Docker preferences) in case docker-error ## Scenarios ### 1. Enter container * Run `docker-compose exec app bash`. ### 2. Relaunch container * Run `docker-compose scale <container_name>=0 && docker-compose scale <container_name>=1`. For example: `docker-compose scale app=0 && docker-compose scale app=1`. ### 3. Run tests 1. `docker-compose exec app magento prepare_tests` 2. `docker-compose exec app bin/magento dev:tests:run (unit, integration)` 3. `docker-compose exec app bash` 4. `cd dev/tests/acceptance/ and vendor/bin/codecept run (mftf)` 5. `cd dev/tests/functional/ and vendor/bin/phpunit run (mtf)` 6. `vnc://localhost:5900 pass:secret (mftf or mtf)` ### 4. Enable/disable Xdebug * Enable: `mutagen project run xdebug-enable` * Disable: `mutagen project run xdebug-disable` :warning: Enabled Xdebug may slow your environment. ### 5. Magento (Re)-Installation * `mutagen project run reinstall` ### 6. Optimization host 1. Redis optimization ``` docker run -it --rm --privileged ubuntu /bin/bash echo never | tee /sys/kernel/mm/transparent_hugepage/enabled echo never | tee /sys/kernel/mm/transparent_hugepage/defrag ``` 2. [Optimization for MacOS](https://gist.github.com/tombigel/d503800a282fcadbee14b537735d202c) ### FAQ 1. If docker containers do not go up, check errors in console, run `docker-compose down`, fix issue and run `docker-compose up` again. 2. If `Overwrite the existing configuration for db-ssl-verify?[Y/n]` prompts in console, type `Y`. 3. If magento installation fails, run `docker-compose exec app magento reinstall` ### TODO list https://github.com/duhon/magento-docker/projects ### gRPC set up: As currently we don't have ability to generate Storefront API on fly - we put magento.proto file to the root of `catalog-storefront:develop` branch. It's needed to run gRPC server and client. #### Steps for manual setup 1. PHP image should be built with "grpc" extension and `rr-grpc` - see details in `./build/php/fpm-grpc` 2. enter into a container and run magento CLI command `bin/magento storefront:grpc:init \\Magento\\CatalogStorefrontApi\\Api\\CatalogProxyServer \\Magento\\CatalogStorefrontApi\\Api\\VariantServiceProxyServer` 3. gRPC server can be executed now: `./vendor/bin/grpc-server` #### Automated setup 1. Run `mutagen project run grpc-server-start` command to execute `etc/php/tools/grpc` script which does the following: - replaces `magento.proto` with a file from `catalog-storfront` if it's a link - runs magento CLI command `bin/magento storefront:grpc:init \\Magento\\CatalogStorefrontApi\\Api\\CatalogProxyServer \\Magento\\CatalogStorefrontApi\\Api\\VariantServiceProxyServer` that does the following: - copies certain files to `vendor/bin` (if they don't exist) - creates list of gRPC services and puts it to `./generated/code/grpc_services_map.php` file (if not created yet) - runs gRPC server via executing of `./vendor/bin/grpc-server` - Please NOTE: Port **9001** should be opened to allow external connections to the server. 2. Run gRPC client (can be executed from any instance which has access to **app:9001**): - Uncomment following code in docker-compose.yml: ```yaml grpcui: image: wongnai/grpcui ports: - "8080:8080" volumes: - code:/var/www entrypoint: ["grpcui", "-plaintext", "-proto", "magento.proto", "-port", "8080", "-bind", "0.0.0.0", "-import-path", "/var/www/magento2ce", "app:9001"] ``` - Make sure that paths to Magento root in entry point are correct (**/var/www/magento2ce**) - if no, replace them by correct paths. - Port **8080** should be opened to allow external connections to the client. - Run grpcui container: `docker-compose up grpcui` ### Standalone services setup Catalog Storefront contains services that can be run separately. For example `storefront-message-broker` is being started [as separate application](https://github.com/duhon/magento-docker/blob/8f02567fa9cd4c106d1419f198930c8383c270c4/reinstall#L27) after Magento Monolith installed. Other services are optional (`pricing`, `search`, `reviews`) and can be started after Magento launched. For example `pricing` service can be started by the following steps: 1. Make sure [container with pricing is available](https://github.com/duhon/magento-docker/blob/8f02567fa9cd4c106d1419f198930c8383c270c4/bundles/storefront.yml#L86). and not commented before install. 2. In `reinstall` file [uncomment catalog-pricing installation command](https://github.com/duhon/magento-docker/blob/8f02567fa9cd4c106d1419f198930c8383c270c4/reinstall#L29) or run this command if Magento already installed. `docker-compose exec app-pricing magento reinstall_storefront_pricing` 3. To enter the container run the following command: `docker-compose exec app-pricing bash` To start `grpc-server` within pricing application do the following: 1. Navigate to `.env` file and set `GRPC_SERVICE_CLASS` to the value [mentioned in the example](https://github.com/duhon/magento-docker/blob/6437cac1dc6c97c7c66ef46aacdbcd983ece269c/.env#L13): ``` GRPC_SERVICE_CLASS="\\Magento\\PricingStorefrontApi\\Api\\PriceBookServiceProxyServer" ``` 2. From docker project root start the following command: `docker-compose exec app-pricing magento grpc` 3. To send requests to `pricing` application start grpcui. Uncomment `grpcui-pricing` section in `docker-composer.yml` and start via docker-compose: `docker-compose up grpcui-pricing` 4. Open `http://0.0.0.0:8082/` in browser. `magento.pricingStorefrontApi.proto.PriceBookService` service must be present in services list. ### Q/A 1. Unable to start `grpcui`. If project uses symlinks then `magento.proto` in root folder will not work correctly with `grpcui`. The following errors can occur: ``` Attaching to storefront_grpcui_1 grpcui_1 | Failed to process proto source files.: /var/www/magento2ce/magento.proto does not reside in any import path storefront_grpcui_1 exited with code 1 ``` Connect to project, unlink proto and copy from `catalog-storefront`, then start container again: - `unlink magento.proto` - `cp ../catalog-storefront/magento.proto magento.proto`