Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Less verbose output for the migrate command #834

Closed
stof opened this issue Jun 26, 2019 · 6 comments
Closed

Less verbose output for the migrate command #834

stof opened this issue Jun 26, 2019 · 6 comments
Assignees
Milestone

Comments

@stof
Copy link
Member

stof commented Jun 26, 2019

Feature Request

Q A
New Feature no
RFC yes
BC Break no (unless some tools parse the text output of the migrate command)

Summary

Currently, running the migrate commands in normal verbosity displays a lot of output (it shows each SQL query plus some details about each migration execution). And the only lower verbosity is --quiet which won't display anything at all (an uncaught exception would still be rendered by the Console component, but that's all).
In my project, this made us face an issue due to the size of this output: our Travis CI setup runs migrations (from a blank state) to initialize the DB (which helps us ensuring that migrations are correct at least regarding schema changes and the validity of other data-migrating queries). The project currently contains 538 DB migrations. And the migrate command alone generates ~7600 lines of output. This makes our CI output reach the Travis log limit, which truncates the logs (and so hides the output about which test is failing...). I know that I could dump the schema and rollup migrations to limit how many we have (and I will at some point, but I need to freeze development changing the DB schema to perform it, and find a way to hook the rollup command in my development process at that point and then revert to migrate) but that would happen again in the future.

Would you consider moving the existing output to the verbose level, and having a much more compact progress report at normal level (maybe with a dot progress, as done by PHPUnit when running tests ?) ?

@jwage
Copy link
Member

jwage commented Jun 26, 2019

I would be fine with this change. I faced the same problem in one of my projects in the past and I would periodically rollup the starting point with a schema.sql file that I load first, and then run migrations from that point. Running all the migrations from 10 years of development really took a long time so we made it a regular process/practice to rollup the migrations.

@stof
Copy link
Member Author

stof commented Jun 26, 2019

Well, the fact that such pattern took a very long time in the past was the reason why I contributed #323 (which was then replaced by #407 as a better way to achieve the same optimization result). Time stays under control now (running these 538 migrations takes 10s on Travis)
Migrations 2.0 now supports this dump+rollup feature, but changing my deployment process to run rollup instead of migrate for that particular deployment is something I haven't figured out yet.

On a side note, this has another benefit for Travis: when loading the logs of a build, the collapsing of the setup command happens only when the full output of that command is received (as the end marker is at the end of this output) and a 7600 lines output covers many chunks and so stays uncollapsed long enough to annoy someone starting to scroll past it. A more compact output will help here.

@jwage
Copy link
Member

jwage commented Jun 26, 2019

It makes sense. 10 seconds isn't bad at all for all those migrations so that isn't an issue for you. I am 👍

@stof
Copy link
Member Author

stof commented Jul 12, 2019

Another benefit of this: the output is actually slowing down the migration (due to waiting on the output stream).

When running the command in quiet mode on Travis, it goes down from 10s to 6s.

@mssimi
Copy link

mssimi commented Sep 15, 2019

Opened pull request to improve this.

Example output:

97v8wEy

@goetas goetas self-assigned this Dec 10, 2019
@goetas goetas added this to the 3.0.0 milestone Dec 10, 2019
@goetas
Copy link
Member

goetas commented Mar 6, 2020

The verbosity of the migrations in 3.x can be controlled suing the -v, -vv or -vvv command parameters

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants