Use a sequence to generate AS transaction IDs, drop `last_txn` AS state #12209

Fizzadar · 2022-03-11T08:21:19Z

Drafting this pending any concerns and will then look into tests. Aims here:

fix SerializationFailure on appservice outgoing transaction (updating last_txn) #11620 without altering transaction isolation levels
precursor to parallising the appservice workload across multiple worker instances

Switching to a sequence means there's no need to track last_txn on the AS state table to generate new TXN IDs. This also means that there is no longer contention between the AS scheduler and AS handler on updates to the application_services_state table, which will prevent serialization errors during the complete AS txn transaction. This should properly fix #11620 without the need for dropping the TXN isolation levels.

NOTE: we have been running the TXN isolation level change in #12032 successfully for a couple of weeks now. Since making the change we have not seen a single instance of the debug line removed in this PR. My conclusion is that the tracking of last TXN ID and the associated debug statement actually cause the problem due to the contention on the AS state table writes. (I believe this all pre-dates the use of sequences).

Another thing of note is this means all AS's share the same sequence; I don't think there's any actual issues with this other than the TXN ID "jumping" between as specific AS' transactions - the spec doesn't specify these need to be sequential. (if needed one could generate a sequence-per-AS, but that feels wrong!).

If this is all good, I'll update the tests and work on the required DB migration so this can be rolled out.

Signed off by Nick Mills-Barrett nick@beeper.com

Pull Request Checklist

Pull request is based on the develop branch
Pull request includes a changelog file.
Pull request includes a sign off
Code style is correct

Switching to a sequence means there's no need to track `last_txn` on the AS state table to generate new TXN IDs. This also means that there is no longer contention between the AS scheduler and AS handler on updates to the `application_services_state` table, which will prevent serialization errors during the complete AS txn transaction.

clokep · 2022-03-11T16:05:46Z

@Fizzadar requested an initial review on this:

basically a quick confirm I've not missed something obvious that would prevent switching to a sequence generator before I look into tests and the DB migration required as I suspect that'll take longer than the changes so far!

richvdh

sorry for the delay on this, @Fizzadar.

The approach here looks very sensible. I think the only reason we're using a field in application_services_state is that this code is so old it predates the use of sequences anywhere in Synapse.

synapse/storage/databases/main/appservice.py

Writing this column can be removed in a later release once the migration to the sequence has been completed.

The sequence generators are already tested elsewhere with implementation specifics for each DB type, so no point duplicating that here.

Fizzadar · 2022-03-28T14:30:45Z

Thank you for looking at this @richvdh - I've addressed comments, added the migration and updated the tests.

One thing to note - the DB migration requires that any appservice worker be stopped before the main process is upgraded, otherwise a new AS TNX may be generated and break the initial sequence value. Is this something that should be included in the changelog file? Or is there a different approach here?

richvdh

lgtm otherwise (though I'm afraid a couple of conflicts have slipped in).

synapse/storage/schema/main/delta/68/06as_txn_seq.py

richvdh · 2022-03-30T15:27:36Z

synapse/storage/schema/main/delta/68/06as_txn_seq.py

@@ -0,0 +1,48 @@
+# Copyright 2022 Beeper


it would be good to bump synapse.storage.schema.SCHEMA_VERSION (and move this to 69/). Then in future, when we remove the upsert on application_services_state, we can bump SCHEMA_COMPAT_VERSION to ensure that nobody tries rolling back to a version of synapse that relies on it.

richvdh · 2022-03-30T15:29:16Z

One thing to note - the DB migration requires that any appservice worker be stopped before the main process is upgraded, otherwise a new AS TNX may be generated and break the initial sequence value. Is this something that should be included in the changelog file? Or is there a different approach here?

yeah, that's awkward. I don't think we have a great way to handle this, so let's just call it out in the changelog and upgrade notes.

Fizzadar · 2022-03-30T20:04:44Z

Updated per comments, merged in develop and added a note to the changelog about the upgrade!

changelog.d/12209.misc

richvdh

thanks very much for this, @Fizzadar!

I've added an entry to the upgrade notes to warn about shutting down the AS worker.

Synapse 1.57.0 (2022-04-19) =========================== This version includes a [change](matrix-org/synapse#12209) to the way transaction IDs are managed for application services. If your deployment uses a dedicated worker for application service traffic, **it must be stopped** when the database is upgraded (which normally happens when the main process is upgraded), to ensure the change is made safely without any risk of reusing transaction IDs. See the [upgrade notes](https://github.com/matrix-org/synapse/blob/v1.57.0rc1/docs/upgrade.md#upgrading-to-v1570) for more details. No significant changes since 1.57.0rc1. Synapse 1.57.0rc1 (2022-04-12) ============================== Features -------- - Send device list changes to application services as specified by [MSC3202](matrix-org/matrix-spec-proposals#3202), using unstable prefixes. The `msc3202_transaction_extensions` experimental homeserver config option must be enabled and `org.matrix.msc3202: true` must be present in the application service registration file for device list changes to be sent. The "left" field is currently always empty. ([\#11881](matrix-org/synapse#11881)) - Optimise fetching large quantities of missing room state over federation. ([\#12040](matrix-org/synapse#12040)) - Offload the `update_client_ip` background job from the main process to the background worker, when using Redis-based replication. ([\#12251](matrix-org/synapse#12251)) - Move `update_client_ip` background job from the main process to the background worker. ([\#12252](matrix-org/synapse#12252)) - Add a module callback to react to new 3PID (email address, phone number) associations. ([\#12302](matrix-org/synapse#12302)) - Add a configuration option to remove a specific set of rooms from sync responses. ([\#12310](matrix-org/synapse#12310)) - Add a module callback to react to account data changes. ([\#12327](matrix-org/synapse#12327)) - Allow setting user admin status using the module API. Contributed by Famedly. ([\#12341](matrix-org/synapse#12341)) - Reduce overhead of restarting synchrotrons. ([\#12367](matrix-org/synapse#12367), [\#12372](matrix-org/synapse#12372)) - Update `/messages` to use historic pagination tokens if no `from` query parameter is given. ([\#12370](matrix-org/synapse#12370)) - Add a module API for reading and writing global account data. ([\#12391](matrix-org/synapse#12391)) - Support the stable `v1` endpoint for `/relations`, per [MSC2675](matrix-org/matrix-spec-proposals#2675). ([\#12403](matrix-org/synapse#12403)) - Include bundled aggregations in search results ([MSC3666](matrix-org/matrix-spec-proposals#3666)). ([\#12436](matrix-org/synapse#12436)) Bugfixes -------- - Fix a long-standing bug where updates to the server notices user profile (display name/avatar URL) in the configuration would not be applied to pre-existing rooms. Contributed by Jorge Florian. ([\#12115](matrix-org/synapse#12115)) - Fix a long-standing bug where events from ignored users were still considered for bundled aggregations. ([\#12235](matrix-org/synapse#12235), [\#12338](matrix-org/synapse#12338)) - Fix non-member state events not resolving for historical events when used in [MSC2716](matrix-org/matrix-spec-proposals#2716) `/batch_send` `state_events_at_start`. ([\#12329](matrix-org/synapse#12329)) - Fix a long-standing bug affecting URL previews that would generate a 500 response instead of a 403 if the previewed URL includes a port that isn't allowed by the relevant blacklist. ([\#12333](matrix-org/synapse#12333)) - Default to `private` room visibility rather than `public` when a client does not specify one, according to spec. ([\#12350](matrix-org/synapse#12350)) - Fix a spec compliance issue where requests to the `/publicRooms` federation API would specify `limit` as a string. ([\#12364](matrix-org/synapse#12364), [\#12410](matrix-org/synapse#12410)) - Fix a bug introduced in Synapse 1.49.0 which caused the `synapse_event_persisted_position` metric to have invalid values. ([\#12390](matrix-org/synapse#12390)) Updates to the Docker image --------------------------- - Bundle locked versions of dependencies into the Docker image. ([\#12385](matrix-org/synapse#12385), [\#12439](matrix-org/synapse#12439)) - Fix up healthcheck generation for workers docker image. ([\#12405](matrix-org/synapse#12405)) Improved Documentation ---------------------- - Clarify documentation for running SyTest against Synapse, including use of Postgres and worker mode. ([\#12271](matrix-org/synapse#12271)) - Document the behaviour of `LoggingTransaction.call_after` and `LoggingTransaction.call_on_exception` methods when transactions are retried. ([\#12315](matrix-org/synapse#12315)) - Update dead links in `check-newsfragment.sh` to point to the correct documentation URL. ([\#12331](matrix-org/synapse#12331)) - Upgrade the version of `mdbook` in CI to 0.4.17. ([\#12339](matrix-org/synapse#12339)) - Updates to the Room DAG concepts development document to clarify that we mark events as outliers because we don't have any state for them. ([\#12345](matrix-org/synapse#12345)) - Update the link to Redis pub/sub documentation in the workers documentation. ([\#12369](matrix-org/synapse#12369)) - Remove documentation for converting a legacy structured logging configuration to the new format. ([\#12392](matrix-org/synapse#12392)) Deprecations and Removals ------------------------- - Remove the unused and unstable `/aggregations` endpoint which was removed from [MSC2675](matrix-org/matrix-spec-proposals#2675). ([\#12293](matrix-org/synapse#12293)) Internal Changes ---------------- - Remove lingering unstable references to MSC2403 (knocking). ([\#12165](matrix-org/synapse#12165)) - Avoid trying to calculate the state at outlier events. ([\#12191](matrix-org/synapse#12191), [\#12316](matrix-org/synapse#12316), [\#12330](matrix-org/synapse#12330), [\#12332](matrix-org/synapse#12332), [\#12409](matrix-org/synapse#12409)) - Omit sending "offline" presence updates to application services after they are initially configured. ([\#12193](matrix-org/synapse#12193)) - Switch to using a sequence to generate AS transaction IDs. Contributed by Nick @ Beeper. If running synapse with a dedicated appservice worker, this MUST be stopped before upgrading the main process and database. ([\#12209](matrix-org/synapse#12209)) - Add missing type hints for storage. ([\#12267](matrix-org/synapse#12267)) - Add missing type definitions for scripts in docker folder. Contributed by Jorge Florian. ([\#12280](matrix-org/synapse#12280)) - Move [MSC2654](matrix-org/matrix-spec-proposals#2654) support behind an experimental configuration flag. ([\#12295](matrix-org/synapse#12295)) - Update docstrings to explain how to decipher live and historic pagination tokens. ([\#12317](matrix-org/synapse#12317)) - Add ground work for speeding up device list updates for users in large numbers of rooms. ([\#12321](matrix-org/synapse#12321)) - Fix typechecker problems exposed by signedjson 1.1.2. ([\#12326](matrix-org/synapse#12326)) - Remove the `tox` packaging job: it will be redundant once #11537 lands. ([\#12334](matrix-org/synapse#12334)) - Ignore `.envrc` for `direnv` users. ([\#12335](matrix-org/synapse#12335)) - Remove the (broadly unused, dev-only) dockerfile for pg tests. ([\#12336](matrix-org/synapse#12336)) - Remove redundant `get_success` calls in test code. ([\#12346](matrix-org/synapse#12346)) - Add type annotations for `tests/unittest.py`. ([\#12347](matrix-org/synapse#12347)) - Move single-use methods out of `TestCase`. ([\#12348](matrix-org/synapse#12348)) - Remove broken and unused development scripts. ([\#12349](matrix-org/synapse#12349), [\#12351](matrix-org/synapse#12351), [\#12355](matrix-org/synapse#12355)) - Convert `Linearizer` tests from `inlineCallbacks` to async. ([\#12353](matrix-org/synapse#12353)) - Update docstrings for `ReadWriteLock` tests. ([\#12354](matrix-org/synapse#12354)) - Refactor `Linearizer`, convert methods to async and use an async context manager. ([\#12357](matrix-org/synapse#12357)) - Fix a long-standing bug where `Linearizer`s could get stuck if a cancellation were to happen at the wrong time. ([\#12358](matrix-org/synapse#12358)) - Make `StreamToken.from_string` and `RoomStreamToken.parse` propagate cancellations instead of replacing them with `SynapseError`s. ([\#12366](matrix-org/synapse#12366)) - Add type hints to tests files. ([\#12371](matrix-org/synapse#12371)) - Allow specifying the Postgres database's port when running unit tests with Postgres. ([\#12376](matrix-org/synapse#12376)) - Remove temporary pin of signedjson<=1.1.1 that was added in Synapse 1.56.0. ([\#12379](matrix-org/synapse#12379)) - Add opentracing spans to calls to external cache. ([\#12380](matrix-org/synapse#12380)) - Lay groundwork for using `poetry` to manage Synapse's dependencies. ([\#12381](matrix-org/synapse#12381), [\#12407](matrix-org/synapse#12407), [\#12412](matrix-org/synapse#12412), [\#12418](matrix-org/synapse#12418)) - Make missing `importlib_metadata` dependency explicit. ([\#12384](matrix-org/synapse#12384), [\#12400](matrix-org/synapse#12400)) - Update type annotations for compatiblity with prometheus_client 0.14. ([\#12389](matrix-org/synapse#12389)) - Remove support for the unstable identifiers specified in [MSC3288](matrix-org/matrix-spec-proposals#3288). ([\#12398](matrix-org/synapse#12398)) - Add missing type hints to configuration classes. ([\#12402](matrix-org/synapse#12402)) - Add files used to build the Docker image used for complement testing into the Synapse repository. ([\#12404](matrix-org/synapse#12404)) - Do not include groups in the sync response when disabled. ([\#12408](matrix-org/synapse#12408)) - Improve type hints related to HTTP query parameters. ([\#12415](matrix-org/synapse#12415)) - Stop maintaining a list of lint targets. ([\#12420](matrix-org/synapse#12420)) - Make `synapse._scripts` pass type checks. ([\#12421](matrix-org/synapse#12421), [\#12422](matrix-org/synapse#12422)) - Add some type hints to datastore. ([\#12423](matrix-org/synapse#12423)) - Enable certificate checking during complement tests. ([\#12435](matrix-org/synapse#12435)) - Explicitly specify the `tls` extra for Twisted dependency. ([\#12444](matrix-org/synapse#12444))

Synapse 1.57.0 (2022-04-19) =========================== This version includes a [change](matrix-org#12209) to the way transaction IDs are managed for application services. If your deployment uses a dedicated worker for application service traffic, **it must be stopped** when the database is upgraded (which normally happens when the main process is upgraded), to ensure the change is made safely without any risk of reusing transaction IDs. See the [upgrade notes](https://github.com/matrix-org/synapse/blob/v1.57.0rc1/docs/upgrade.md#upgrading-to-v1570) for more details. No significant changes since 1.57.0rc1. Synapse 1.57.0rc1 (2022-04-12) ============================== Features -------- - Send device list changes to application services as specified by [MSC3202](matrix-org/matrix-spec-proposals#3202), using unstable prefixes. The `msc3202_transaction_extensions` experimental homeserver config option must be enabled and `org.matrix.msc3202: true` must be present in the application service registration file for device list changes to be sent. The "left" field is currently always empty. ([\matrix-org#11881](matrix-org#11881)) - Optimise fetching large quantities of missing room state over federation. ([\matrix-org#12040](matrix-org#12040)) - Offload the `update_client_ip` background job from the main process to the background worker, when using Redis-based replication. ([\matrix-org#12251](matrix-org#12251)) - Move `update_client_ip` background job from the main process to the background worker. ([\matrix-org#12252](matrix-org#12252)) - Add a module callback to react to new 3PID (email address, phone number) associations. ([\matrix-org#12302](matrix-org#12302)) - Add a configuration option to remove a specific set of rooms from sync responses. ([\matrix-org#12310](matrix-org#12310)) - Add a module callback to react to account data changes. ([\matrix-org#12327](matrix-org#12327)) - Allow setting user admin status using the module API. Contributed by Famedly. ([\matrix-org#12341](matrix-org#12341)) - Reduce overhead of restarting synchrotrons. ([\matrix-org#12367](matrix-org#12367), [\matrix-org#12372](matrix-org#12372)) - Update `/messages` to use historic pagination tokens if no `from` query parameter is given. ([\matrix-org#12370](matrix-org#12370)) - Add a module API for reading and writing global account data. ([\matrix-org#12391](matrix-org#12391)) - Support the stable `v1` endpoint for `/relations`, per [MSC2675](matrix-org/matrix-spec-proposals#2675). ([\matrix-org#12403](matrix-org#12403)) - Include bundled aggregations in search results ([MSC3666](matrix-org/matrix-spec-proposals#3666)). ([\matrix-org#12436](matrix-org#12436)) Bugfixes -------- - Fix a long-standing bug where updates to the server notices user profile (display name/avatar URL) in the configuration would not be applied to pre-existing rooms. Contributed by Jorge Florian. ([\matrix-org#12115](matrix-org#12115)) - Fix a long-standing bug where events from ignored users were still considered for bundled aggregations. ([\matrix-org#12235](matrix-org#12235), [\matrix-org#12338](matrix-org#12338)) - Fix non-member state events not resolving for historical events when used in [MSC2716](matrix-org/matrix-spec-proposals#2716) `/batch_send` `state_events_at_start`. ([\matrix-org#12329](matrix-org#12329)) - Fix a long-standing bug affecting URL previews that would generate a 500 response instead of a 403 if the previewed URL includes a port that isn't allowed by the relevant blacklist. ([\matrix-org#12333](matrix-org#12333)) - Default to `private` room visibility rather than `public` when a client does not specify one, according to spec. ([\matrix-org#12350](matrix-org#12350)) - Fix a spec compliance issue where requests to the `/publicRooms` federation API would specify `limit` as a string. ([\matrix-org#12364](matrix-org#12364), [\matrix-org#12410](matrix-org#12410)) - Fix a bug introduced in Synapse 1.49.0 which caused the `synapse_event_persisted_position` metric to have invalid values. ([\matrix-org#12390](matrix-org#12390)) Updates to the Docker image --------------------------- - Bundle locked versions of dependencies into the Docker image. ([\matrix-org#12385](matrix-org#12385), [\matrix-org#12439](matrix-org#12439)) - Fix up healthcheck generation for workers docker image. ([\matrix-org#12405](matrix-org#12405)) Improved Documentation ---------------------- - Clarify documentation for running SyTest against Synapse, including use of Postgres and worker mode. ([\matrix-org#12271](matrix-org#12271)) - Document the behaviour of `LoggingTransaction.call_after` and `LoggingTransaction.call_on_exception` methods when transactions are retried. ([\matrix-org#12315](matrix-org#12315)) - Update dead links in `check-newsfragment.sh` to point to the correct documentation URL. ([\matrix-org#12331](matrix-org#12331)) - Upgrade the version of `mdbook` in CI to 0.4.17. ([\matrix-org#12339](matrix-org#12339)) - Updates to the Room DAG concepts development document to clarify that we mark events as outliers because we don't have any state for them. ([\matrix-org#12345](matrix-org#12345)) - Update the link to Redis pub/sub documentation in the workers documentation. ([\matrix-org#12369](matrix-org#12369)) - Remove documentation for converting a legacy structured logging configuration to the new format. ([\matrix-org#12392](matrix-org#12392)) Deprecations and Removals ------------------------- - Remove the unused and unstable `/aggregations` endpoint which was removed from [MSC2675](matrix-org/matrix-spec-proposals#2675). ([\matrix-org#12293](matrix-org#12293)) Internal Changes ---------------- - Remove lingering unstable references to MSC2403 (knocking). ([\matrix-org#12165](matrix-org#12165)) - Avoid trying to calculate the state at outlier events. ([\matrix-org#12191](matrix-org#12191), [\matrix-org#12316](matrix-org#12316), [\matrix-org#12330](matrix-org#12330), [\matrix-org#12332](matrix-org#12332), [\matrix-org#12409](matrix-org#12409)) - Omit sending "offline" presence updates to application services after they are initially configured. ([\matrix-org#12193](matrix-org#12193)) - Switch to using a sequence to generate AS transaction IDs. Contributed by Nick @ Beeper. If running synapse with a dedicated appservice worker, this MUST be stopped before upgrading the main process and database. ([\matrix-org#12209](matrix-org#12209)) - Add missing type hints for storage. ([\matrix-org#12267](matrix-org#12267)) - Add missing type definitions for scripts in docker folder. Contributed by Jorge Florian. ([\matrix-org#12280](matrix-org#12280)) - Move [MSC2654](matrix-org/matrix-spec-proposals#2654) support behind an experimental configuration flag. ([\matrix-org#12295](matrix-org#12295)) - Update docstrings to explain how to decipher live and historic pagination tokens. ([\matrix-org#12317](matrix-org#12317)) - Add ground work for speeding up device list updates for users in large numbers of rooms. ([\matrix-org#12321](matrix-org#12321)) - Fix typechecker problems exposed by signedjson 1.1.2. ([\matrix-org#12326](matrix-org#12326)) - Remove the `tox` packaging job: it will be redundant once matrix-org#11537 lands. ([\matrix-org#12334](matrix-org#12334)) - Ignore `.envrc` for `direnv` users. ([\matrix-org#12335](matrix-org#12335)) - Remove the (broadly unused, dev-only) dockerfile for pg tests. ([\matrix-org#12336](matrix-org#12336)) - Remove redundant `get_success` calls in test code. ([\matrix-org#12346](matrix-org#12346)) - Add type annotations for `tests/unittest.py`. ([\matrix-org#12347](matrix-org#12347)) - Move single-use methods out of `TestCase`. ([\matrix-org#12348](matrix-org#12348)) - Remove broken and unused development scripts. ([\matrix-org#12349](matrix-org#12349), [\matrix-org#12351](matrix-org#12351), [\matrix-org#12355](matrix-org#12355)) - Convert `Linearizer` tests from `inlineCallbacks` to async. ([\matrix-org#12353](matrix-org#12353)) - Update docstrings for `ReadWriteLock` tests. ([\matrix-org#12354](matrix-org#12354)) - Refactor `Linearizer`, convert methods to async and use an async context manager. ([\matrix-org#12357](matrix-org#12357)) - Fix a long-standing bug where `Linearizer`s could get stuck if a cancellation were to happen at the wrong time. ([\matrix-org#12358](matrix-org#12358)) - Make `StreamToken.from_string` and `RoomStreamToken.parse` propagate cancellations instead of replacing them with `SynapseError`s. ([\matrix-org#12366](matrix-org#12366)) - Add type hints to tests files. ([\matrix-org#12371](matrix-org#12371)) - Allow specifying the Postgres database's port when running unit tests with Postgres. ([\matrix-org#12376](matrix-org#12376)) - Remove temporary pin of signedjson<=1.1.1 that was added in Synapse 1.56.0. ([\matrix-org#12379](matrix-org#12379)) - Add opentracing spans to calls to external cache. ([\matrix-org#12380](matrix-org#12380)) - Lay groundwork for using `poetry` to manage Synapse's dependencies. ([\matrix-org#12381](matrix-org#12381), [\matrix-org#12407](matrix-org#12407), [\matrix-org#12412](matrix-org#12412), [\matrix-org#12418](matrix-org#12418)) - Make missing `importlib_metadata` dependency explicit. ([\matrix-org#12384](matrix-org#12384), [\matrix-org#12400](matrix-org#12400)) - Update type annotations for compatiblity with prometheus_client 0.14. ([\matrix-org#12389](matrix-org#12389)) - Remove support for the unstable identifiers specified in [MSC3288](matrix-org/matrix-spec-proposals#3288). ([\matrix-org#12398](matrix-org#12398)) - Add missing type hints to configuration classes. ([\matrix-org#12402](matrix-org#12402)) - Add files used to build the Docker image used for complement testing into the Synapse repository. ([\matrix-org#12404](matrix-org#12404)) - Do not include groups in the sync response when disabled. ([\matrix-org#12408](matrix-org#12408)) - Improve type hints related to HTTP query parameters. ([\matrix-org#12415](matrix-org#12415)) - Stop maintaining a list of lint targets. ([\matrix-org#12420](matrix-org#12420)) - Make `synapse._scripts` pass type checks. ([\matrix-org#12421](matrix-org#12421), [\matrix-org#12422](matrix-org#12422)) - Add some type hints to datastore. ([\matrix-org#12423](matrix-org#12423)) - Enable certificate checking during complement tests. ([\matrix-org#12435](matrix-org#12435)) - Explicitly specify the `tls` extra for Twisted dependency. ([\matrix-org#12444](matrix-org#12444))

This column is unused as of #12209, so let's stop writing to it.

This makes sure we that the port script sets up the "application_services_txns" sequence (matrix-org#12209). and the "un_partial_stated_event_stream_sequence" sequence (matrix-org#14545), which fixes errors like the one below from appearing when trying to connect to the postgresql database after running the script: Postgres sequence 'application_services_txn_id_seq' is inconsistent with associated table 'application_services_txns'. This can happen if Synapse has been downgraded and then upgraded again, or due to a bad migration. To fix this error, shut down Synapse (including any and all workers) and run the following SQL: SELECT setval('application_services_txn_id_seq', ( SELECT GREATEST(MAX(txn_id), 0) FROM application_services_txns )); See docs/postgres.md for more information. Signed-off-by: Guacamolie <guac@amolie.nl>

Fizzadar changed the title ~~Use a sequence to generate AS transaction IDs, drop last_txn AS state~~ Draft: Use a sequence to generate AS transaction IDs, drop last_txn AS state Mar 11, 2022

clokep requested a review from a team March 11, 2022 16:05

richvdh self-assigned this Mar 24, 2022

richvdh reviewed Mar 25, 2022

View reviewed changes

synapse/storage/databases/main/appservice.py Show resolved Hide resolved

synapse/storage/databases/main/appservice.py Outdated Show resolved Hide resolved

Fizzadar added 8 commits March 28, 2022 11:22

Show warning when running slow AS sequence query

cceb2b9

Restore writing AS state last_txn for this release

722f6ce

Writing this column can be removed in a later release once the migration to the sequence has been completed.

Add changelog file

81f5220

Fix database variable

6d35ceb

Update AS storage tests for new sequence generated TXN IDs

ccc2e63

Add database migration to create postgres sequence

ca0daba

Lint migration

ca4a78a

Remove AS txn sequence test as tested elsewhere

c9c0b06

The sequence generators are already tested elsewhere with implementation specifics for each DB type, so no point duplicating that here.

Fizzadar marked this pull request as ready for review March 28, 2022 14:28

Fizzadar changed the title ~~Draft: Use a sequence to generate AS transaction IDs, drop last_txn AS state~~ Use a sequence to generate AS transaction IDs, drop last_txn AS state Mar 28, 2022

richvdh self-requested a review March 28, 2022 14:48

richvdh reviewed Mar 30, 2022

View reviewed changes

Fizzadar added 3 commits March 30, 2022 20:57

Merge branch 'develop' into use-sequence-for-appservice-transactions

c919194

Update DB schema migration to version 69

d62adee

Update changelog w/note about shutting AS worker down before upgrade

175f4d3

Fizzadar requested a review from richvdh March 30, 2022 20:04

richvdh reviewed Apr 1, 2022

View reviewed changes

changelog.d/12209.misc Outdated Show resolved Hide resolved

richvdh added 2 commits April 1, 2022 13:53

Update changelog.d/12209.misc

5803f05

Add notes to upgrade.md

226820b

richvdh approved these changes Apr 1, 2022

View reviewed changes

richvdh enabled auto-merge (squash) April 1, 2022 13:05

richvdh mentioned this pull request Apr 1, 2022

Remove code that updates application_services_state.last_txn #12352

Closed

richvdh merged commit 993d90f into matrix-org:develop Apr 1, 2022

Fizzadar deleted the use-sequence-for-appservice-transactions branch April 1, 2022 13:37

spantaleev mentioned this pull request Apr 19, 2022

update synapse 1.56.0 -> 1.57.0 spantaleev/matrix-docker-ansible-deploy#1766

Merged

Fizzadar mentioned this pull request Apr 26, 2022

AS transaction isolation tweaks #12032

Closed

4 tasks

turt2live mentioned this pull request Apr 26, 2022

newly-configured AS didn't receive new events #11629

Open

richvdh added a commit that referenced this pull request May 9, 2022

Remove code which updates application_services_state.last_txn

d053855

This column is unused as of #12209, so let's stop writing to it.

richvdh mentioned this pull request May 9, 2022

Remove code which updates application_services_state.last_txn #12680

Merged

richvdh added a commit that referenced this pull request May 9, 2022

Remove code which updates application_services_state.last_txn

12fb065

This column is unused as of #12209, so let's stop writing to it.

richvdh added a commit that referenced this pull request May 9, 2022

Remove code which updates application_services_state.last_txn

9bf085d

This column is unused as of #12209, so let's stop writing to it.

richvdh added a commit that referenced this pull request May 17, 2022

Remove code which updates application_services_state.last_txn (#12680)

24b590d

This column is unused as of #12209, so let's stop writing to it.

DMRobertson mentioned this pull request Aug 25, 2022

Drop unused column application_services_state.last_txn #13627

Merged

AndrewFerr mentioned this pull request Nov 27, 2022

Appservice transaction IDs reset with sqlite3 #14556

Open

vanguacamolie mentioned this pull request Jul 20, 2023

Update synapse_port_db for new sequences #15965

Closed

4 tasks

clokep mentioned this pull request Aug 1, 2023

Properly setup more sequences in portdb #16043

Merged

matrixbot mentioned this pull request Dec 21, 2023

Appservice transaction IDs reset with sqlite3 element-hq/synapse#14556

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Use a sequence to generate AS transaction IDs, drop `last_txn` AS state #12209

Use a sequence to generate AS transaction IDs, drop `last_txn` AS state #12209

Fizzadar commented Mar 11, 2022 •

edited

Loading

clokep commented Mar 11, 2022

richvdh left a comment

Fizzadar commented Mar 28, 2022

richvdh left a comment

richvdh Mar 30, 2022

Fizzadar Mar 30, 2022

richvdh commented Mar 30, 2022

Fizzadar commented Mar 30, 2022

richvdh left a comment •

edited by babolivier

Loading

Use a sequence to generate AS transaction IDs, drop last_txn AS state #12209

Use a sequence to generate AS transaction IDs, drop last_txn AS state #12209

Conversation

Fizzadar commented Mar 11, 2022 • edited Loading

Pull Request Checklist

clokep commented Mar 11, 2022

richvdh left a comment

Choose a reason for hiding this comment

Fizzadar commented Mar 28, 2022

richvdh left a comment

Choose a reason for hiding this comment

richvdh Mar 30, 2022

Choose a reason for hiding this comment

Fizzadar Mar 30, 2022

Choose a reason for hiding this comment

richvdh commented Mar 30, 2022

Fizzadar commented Mar 30, 2022

richvdh left a comment • edited by babolivier Loading

Choose a reason for hiding this comment

Use a sequence to generate AS transaction IDs, drop `last_txn` AS state #12209

Use a sequence to generate AS transaction IDs, drop `last_txn` AS state #12209

Fizzadar commented Mar 11, 2022 •

edited

Loading

richvdh left a comment •

edited by babolivier

Loading