-
Notifications
You must be signed in to change notification settings - Fork 125
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Don't sync the schema_migrations_ran table when adding it #383
Don't sync the schema_migrations_ran table when adding it #383
Conversation
I was able to reproduce the conflicts we saw in the I was able to verify our suspicion that the entire contents of the remote region's After applying this patch there were no more conflicts in the PG logs and the migration also completed without issue. |
This really does need to be backported, but I have no idea what the implications of that would be ... |
@carbonin Do we need to test again with this change that the only records in the table after running this migration on the global are the old records? |
@bdunne I think that's not exactly how it would work. It sounds like you think that What actually happens is that all of the changes which can be applied to the current schema from the remote to the global are replicated. Put another way, replication runs until it blows up. What we should expect is that the global So the short answer is ☝️ is what I reproduced, but with the |
The purpose of the table is to track when migrations are run on remote regions. If we sync the whole table outside of the scope of the transaction based replication then we're violating the assumption that the migration version number only appears in the global region when the data from that migration was replicated up to the global region. This was causing a timing issue where the sync could take place just before the global region runs its migrations. Then the global would blaze past migrations without successfully replicating up the data needed for a particular schema version. This could lead to an irrecoverable situation in replication where the subscription would have to be removed and re-added. Fixes https://bugzilla.redhat.com/show_bug.cgi?id=1721596
f2866b6
to
4f106e5
Compare
Checked commit carbonin@4f106e5 with ruby 2.3.3, rubocop 0.69.0, haml-lint 0.20.0, and yamllint 1.10.0 |
…_ran Don't sync the schema_migrations_ran table when adding it (cherry picked from commit 88dfe84) Fixes https://bugzilla.redhat.com/show_bug.cgi?id=1722540
Hammer backport details:
|
The purpose of the table is to track when migrations are run on
remote regions. If we sync the whole table outside of the scope
of the transaction based replication then we're violating the
assumption that the migration version number only appears in the
global region when the data from that migration was replicated
up to the global region.
This was causing a timing issue where the sync could take place
just before the global region runs its migrations. Then the global
would blaze past migrations without successfully replicating up
the data needed for a particular schema version.
This could lead to an irrecoverable situation in replication where
the subscription would have to be removed and re-added.
Fixes https://bugzilla.redhat.com/show_bug.cgi?id=1721596