There is no documentation about deploy commands queue and share folders and files

[luckyraul]

have a look magento/magento2#2423

[ghost]

I've read this and do not understand your question. Can you clarify and give an example?

[luckyraul]

Which folders and files should be shared between releases(for example for m1 var, media and app/etc/local.xml are shared) and how and what order should I use bin/magento setup commands during rollout release

[buskamuza]

Hi @luckyraul ,

could you, please, clarify what do you mean under "folders/files shared between releases"? Are those storing user data?

[luckyraul]

Usually it is everything that not changed between releases

For instance, I already know that pub/media and app/etc/env.php should be shared

/var/www/example.com/     # The deploy_to path
 |-  releases/              # Holds releases, one subdir per release
 |   |- 1/
 |   |- 2/
 |   |- 3/
 |   '- ...
 |-  shared/                # Holds files shared between releases
 |   |- logs/               # Log files are usually stored here
 |   `- ...
 '-  current/               # A symlink to the current release in releases/

[hostep]

Hi

I also believe Magento should write some documentation about recommended deployment scenario's to/on a production server.
There already exists a page about this, but this only talks about the initial setup. Not about deploying updates of your project to the production server.

We need to know exactly which steps (commands) we need to perform and in what order (git clone, composer install, setup:di:compile, setup:static-content:deploy, setting up correct permissions, setup:upgrade, ...).
Also some commands need to have access to the database, so we would need to know at what stage in the deployment process we need to provide access to the database (preferable as late as possible). Then we also need to know when to put the current webshop in maintenance mode (preferable for the shortest time possible), before running the upgrade scripts and when we can disable maintenance mode again.

The deployment directories @luckyraul points out is what a lot of deployments scripts (capistrano, fabric, deployer, ...) prefer to use, so would be ideal if the documentation tries to use a similar setup.

As a clarification, this is about documentation for a deployment process where you execute a bunch of commands on the production server itself. This isn't about a separate build server which produces an artifact which you then need to upload to a production server. Although the steps would probably be very similar for that scenario.

Thanks!

[erikhansen]

@xcomSteveJohnson I agree with @hostep that the devdocs should be updated to include the recommend order of steps to deploy. It might also be useful to link to community projects that can facilitate production deployments, such as @davidalger's capistrano-magento2 Gem.

Whoever is writing this documentation may want to reference the sequence of commands in the capistrano-magento2 scripts: https://github.com/davidalger/capistrano-magento2/blob/master/lib/capistrano/tasks/deploy.rake#L14

[maderlock]

As usual we're trying to work this out ourselves but it is a lot more difficult to get working than with Magento 1 which in very well understood by this point. c.f.
http://magento.stackexchange.com/questions/102144/magento-2-deployment-process
http://magento.stackexchange.com/questions/124166/order-of-magento-2-deployment-tasks

[ghost]

Tracking with internal issue MAGETWO-55366.

Expect some questions from me as I get started.

[erikhansen]

@xcomSteveJohnson I'd be happy to answer questions as you have them.

[maderlock]

Ditto. We have our first production site going live next week and we've reverted to manual commands due to uncertainty over ordering of commands. Great to see some movement on this.

[rob1256]

Can I ask what commands people are using and in what order for deployment in production mode, with minimal down-time? I don't think I'm going to be able to have a build server to compile etc. on to then sync across to the production server (the idea of this seems a bit ridiculous anyway).

For example, say there is an update to one or more less files in my theme, as far as I can tell, you need to eithere delete the entire pub/static folder (thus breaking the site) or find the files that have been updated and then run the static-content:deploy command. Is this correct?

My thoughts on how deployment in general might work (be it for less files or for module code updates etc.) would be:

1. Push code to production server
2. setup:di:compile
3. rm -rf var/view_preprocessed/*
4. setup:static-content:deploy
5. setup:upgrade --keep-generated (though I know this is broken currently)
6. cache:flush

Is the above anywhere near correct?

[davidalger]

@rob1256 Classy Llama is using the capistrano-magento2 gem (which I personally wrote and published on my github) for all code deployments to staging and production environments: https://rubygems.org/gems/capistrano-magento2 (more info in the README and at this blog post)

[rob1256]

@davidalger Thanks for that information, looks like a useful deployment tool, do you know of anyway of doing deployments with no down-time and without having to do release folders etc?

I'm getting relatively close but it looks like I may need to run magento setup:upgrade --keep-generated as soon as any new code is pushed to make sure that the setup_modules table is up-to-date to stop Magento throwing errors. Are there any potential issues with doing that and then running all the compilation steps?

[mbrehme]

+1

[hostep]

Just a heads up for people working on deployment workflows, because something important has changed between Magento 2.1.0 and 2.1.1.
Magento now requires a file pub/static/deployed_version.txt to exist before running bin/magento setup:static-content:deploy

So if you are using a current/releases/shared directory setup, you'll need to copy that file from your current directory into your release directory before running bin/magento setup:static-content:deploy.

I don't know if this is an intentional change, or if this is a new bug introduced in 2.1.1, but using the above workaround, you can get around it.

[davidalger]

@hostep

Magento now requires a file pub/static/deployed_version.txt to exist before running bin/magento setup:static-content:deploy

Also interesting tidbit is that issue only causes trouble if the versioned URLs have been enabled. I've worked around that in the capistrano-magento2 gem by adding a call to touch the deploy_version.txt file prior to running static asset deployment. See davidalger/capistrano-magento2@d61d6983

[luckyraul]

@xcomSteveJohnson

Hi, do have any updates to this ticket?

[ghost]

@luckyraul Sorry about the delay. I should know something soon.

[ghost]

@luckyraul We should have a better story about deployment to production in 2.2 but I don't know the ETA for that release.

[ghost]

As with issue #739 we're planning to address this in 2.2 but I don't know the ETA for that release.

[tschifftner]

We are currently trying to find a good deployment setup. You are welcome to use and participate:

https://github.com/tschifftner/magento2-deployscripts

[ghost]

Thanks. I passed the link along.

[Khaleel]

magento/magento2#6131

Is this related?

[ghost]

@Khaleel Yes, thanks!

[DavidLambauer]

Matthias Walter published a series of our (netz98) deployment process. This might be interesting too!

https://dev98.de/2017/01/06/deploying-magento2-history-overview-14/

[aaronsturm]

I'm using Docker to build a deployable image with any changes from a previous release. I wanted to state that I found if I run bin/magento setup:upgrade and then bin/magento deploy:mode:set production, that that seems to remove the need for the individual steps others have listed above.

[luckyraul]

@aaronsturm
magento deploy:mode:set production enables maintenance mode, you can't do it on the production

[aaronsturm]

I'm using a load balanced environment with rolling updates, that's how I bypass that. The Docker image doesn't even launch the web server until that's complete.

[luckyraul]

@aaronsturm if you put production site to maintenance mode, lb will not help )
You code may be okay, but only to building images

[aaronsturm]

You can remove the instance from the load balancer before updating, of course. In my case, I use AWS' immutable policy to create a new instance, deploy the Docker image, add the instance to the load balancer, and destroy the others after it's successful. Using deploy:mode:set production won't work in every case, I was just adding input to hopefully help someone else out there.

[nickrigby]

I wrote a blog post about deploying Magento 2 using envoyer.io. Hopefully it is useful to someone, even if you're not using Envoyer.

[ipascual]

My two cents, this is how I'm currently deploying using JENKINS:

echo "Removing files in case they exists in the repo"
rm -rf var
rm -rf pub/media

echo "Creating the required symbolic links"
ln -s $WORKSPACE/../env/var ./var
ln -s $WORKSPACE/../env/media ./pub/media

echo "Changing permissions of the temporary folder to www-data user"
sudo chown -R www-data:www-data $WORKSPACE/../*
sudo chmod -R 770 $WORKSPACE/../*

echo "Dependencies"
composer install --no-dev

echo "MAINTENANCE STARTS"
sudo chmod u+x ./bin/magento
sudo -H -u www-data bash -c "./bin/magento maintenance:enable"

echo "Setup upgrade"
sudo -H -u www-data bash -c "./bin/magento setup:upgrade"

echo "Application mode"
sudo -H -u www-data bash -c "./bin/magento deploy:mode:set production"

echo "Flush cache"
sudo -H -u www-data bash -c "./bin/magento cache:flush"

Use jenkins configuration files for env.php and config.php and custom workspace to point to HTML folder.

[dooblem]

Come on, @magento-team , it's been months and this issue is still there. How are you proceeding for deployments in Magento Cloud Edition? Can you share it with us?

http://devdocs.magento.com/guides/v2.1/config-guide/prod/prod_deploy.html

We’re currently working with internal teams to research and publish best practice recommendations for deploying the Magento application to production.

The documentation page has not evolved in months!

It really gives the users a bad image of Magento 2.. like it's not ready for production.

@davidalger capistrano is the best workflow currently, to my opinion, but there are still things to optimise I think (like static assets and di:compile done on target servers...).

Thanks in advance,
Marc from Clever-Age Toulouse

[davidalger]

@dooblem If you're having speed related issues with DI compile and static asset gen, make sure you're on the latest version of M2 and have php70u-process (this is from IUS repo, name according to rpm providing PHP 7 for your server) installed on the server. With this installed, we have deploys running in less than 5 minutes (the static content / DI compilation will become multi-threaded with 4 threads by default; major speed increase!).

[hostep]

@davidalger: unfortunately due to a new bug introduced in Magento 2.1.3 we can no longer benefit from the multi-threaded static content deploy, since it produces incorrect JS translations when you generate multiple languages at once. And for as far as I'm aware, the di:compile step isn't multi-threaded (yet), but there were some good performance improvements made in 2.1.3.

Just FYI :)

@ all the others: @xcomSteveJohnson said they would publish documentation for this when Magento 2.2 is released, which I expect will happen during the Imagine 2017 conference, and that's only a couple of weeks away (let's hope it is true :))

[Khaleel]

Why the hell are they doing JSON translation files? On top of that it is wrapped inside mixed localStorage. It is so poor the mix mash solution. The front end lead architecture of Magento 2 has no clue.

…

On 18 Mar 2017 21:09, "Pieter Hoste" ***@***.***> wrote: @davidalger <https://github.com/davidalger>: unfortunately due to a new bug <magento/magento2#7862> introduced in Magento 2.1.3 we can no longer benefit from the multi-threaded static content deploy, since it produces incorrect JS translations when you compile multiple languages at once. And for as far as I'm aware, the di:compile step isn't multi-threaded (yet), but there were some good performance improvements <#937> made in 2.1.3. Just FYI :) @ all the others: @xcomSteveJohnson <https://github.com/xcomSteveJohnson> said they would publish documentation for this when Magento 2.2 is released, which I expect will happen during the Imagine 2017 conference, and that's only a couple of weeks away (let's hope it is true :)) — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#419 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAVsCCXESXEfDrxbY1vEVz2-tJ9dukSHks5rnA_YgaJpZM4GlfzQ> .

[ghost]

@dooblem Magento Enterprise Cloud Edition deployment is discussed here. If you want a discussion of deployment per se, see this topic.

@dooblem @hostep The only thing we can say for certain is that for us to officially document reduced-downtime deployment to production, code changes have to be made in Magento 2. We're expecting those changes in 2.2 but I cannot give you an ETA either for the release or for documentation about deployment.

If you need that information now, you should contact one of the helpful community members who have responded to this issue such as @davidalger

When we know more we'll let you know here.

[dooblem]

Thanks all for you answers.
@xcomSteveJohnson can we use the "magento-cloud-configuration" module in an Enterprise Edition ?

[ghost]

@dooblem No, only if you have Cloud Edition.

[jcalcaben]

Closing due to lack of activity.