Use vector clocks for room stream tokens. #8439

erikjohnston · 2020-10-01T15:28:17Z

Currently when using multiple event persisters we (in the worst case) don't tell clients about events until all event persisters have persisted new events after the original event. This is a suboptimal, especially if one of the event persisters goes down.

To handle this, we encode the position of each event persister in the room tokens so that we can send events to clients immediately. To reduce the size of the token we do two things:

We create a unique immutable persistent mapping between instance names and a generated small integer ID, which we can encode in the tokens instead of the instance name; and
We encode the "persisted upto position" of the room token and then only explicitly include instances that have positions strictly greater than that.

The new tokens look something like: m3478.1~3488.2~3489, where the first number is the min position, and the subsequent - separated pairs are the instance ID to positions map. (We use . and ~ as separators as they're URL safe and not already used by StreamToken).

Unit tests and SyTest support are incoming

anoadragon453

words

synapse/storage/databases/main/stream.py

synapse/storage/databases/main/schema/delta/58/19instance_map.sql.postgres

synapse/storage/databases/main/stream.py

synapse/types.py

Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>

…_stream_token

synapse/storage/databases/main/schema/delta/58/19instance_map.sql.postgres

synapse/storage/databases/main/stream.py

synapse/types.py

richvdh · 2020-10-05T16:57:23Z

synapse/storage/databases/main/stream.py

+    # We will have already filtered by the topological tokens, so we don't
+    # bother checking topological token bounds again.
+    if from_token and from_token.topological:
+        from_token = None
+
+    if to_token and to_token.topological:
+        to_token = None


I'm somewhat failing to convince myself that this is correct. Why does the presence of a topological component mean that we have to ignore that bound?

ok, it kinda makes sense having stared at _paginate_room_events_txn for a while, but in terms of this function having a comprehensible interface, I think you need to say it only deals with stream orderings, and to set to_token/from_token = None in _paginate_room_events_txn.

I've changed it so that _filter_results now checks historical tokens as well, that seemed easiest and most consistent.

synapse/storage/databases/main/stream.py

richvdh · 2020-10-05T17:13:05Z

synapse/types.py

    def as_tuple(self) -> Tuple[Optional[int], int]:
        return (self.topological, self.stream)


I think if we're going to exclude self.instance_map here, we better document what it actually means.

Actually, given its only used in one place I think I might just move it inline, as it makes more sense there why we are excluding instance map part

I then proceeded to use it to fix the other issues, so have kept it

synapse/storage/databases/main/stream.py

Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>

erikjohnston · 2020-10-06T11:02:18Z

synapse/storage/databases/main/stream.py

+            if stream_ordering <= lower_token.get_stream_pos_for_instance(
+                instance_name
+            ):
+                return False


I wonder if we should lift these comparisons into PersistedEventPosition (and give PersistedEventPosition the event's topological ordering), then we could just do lower_token < event_pos < upper_token?

One problem with overloading < in that way is that, given the different ways we have of comparing token orderings (ref the existence of PersistedEventPosition.persisted_after), < fails to make it clear which mechanism we are using.

So yes, I think it might be nice to move the comparison into PersistedEventPosition, but it might be better to do it with explicit methods rather than <.

(also, note that lower_token < event_pos would call RoomStreamToken.__lt__, so you'd have to put logic in RoomStreamToken too)

(I believe persisted_after has the same semantics)

richvdh

generally seems good. a few nits.

richvdh · 2020-10-07T11:27:16Z

synapse/types.py

+    @instance_map.validator
+    def _validate_topological_or_live(self, attribute, value):
+        """Validates that both `topological` and `instance_map` aren't set.
+        """
+
+        if value and self.topological:
+            raise ValueError(
+                "Cannot set both 'topological' and 'instance_map' on 'RoomStreamToken'."
+            )


for symmetry, we strictly ought to have a validator for topological which checks that it isn't set after instance_map.

But better, since the token is frozen, just have an __attrs_post_init__ which checks for consistency.

synapse/types.py

richvdh · 2020-10-07T11:32:05Z

synapse/types.py

@@ -428,7 +431,19 @@ class RoomStreamToken:
    )
    stream = attr.ib(type=int, validator=attr.validators.instance_of(int))

-    instance_map = attr.ib(type=Dict[str, int], factory=dict)
+    instance_map = attr.ib(
+        type=Dict[str, int], factory=dict, validator=attr.validators.instance_of(dict)


Suggested change

type=Dict[str, int], factory=dict, validator=attr.validators.instance_of(dict)

type=Dict[str, int], factory=dict, validator=attr.validators.deep_mapping(

key_validator=attr.validators.instance_of(str),

value_validator=attr.validators.instance_of(int),

mapping_validator=attr.validators.instance_of(dict),

),

😇

richvdh · 2020-10-07T12:04:26Z

synapse/types.py

        return (self.topological, self.stream)

    def get_stream_pos_for_instance(self, instance_name: str) -> int:
-        """Get the stream position for the instance
+        """Get the stream position that the writer was at at this token.


Suggested change

"""Get the stream position that the writer was at at this token.

"""Get the stream position that the given writer was at at this token.

richvdh · 2020-10-07T12:09:17Z

synapse/storage/databases/main/stream.py

+            if stream_ordering <= lower_token.get_stream_pos_for_instance(
+                instance_name
+            ):
+                return False


One problem with overloading < in that way is that, given the different ways we have of comparing token orderings (ref the existence of PersistedEventPosition.persisted_after), < fails to make it clear which mechanism we are using.

So yes, I think it might be nice to move the comparison into PersistedEventPosition, but it might be better to do it with explicit methods rather than <.

(also, note that lower_token < event_pos would call RoomStreamToken.__lt__, so you'd have to put logic in RoomStreamToken too)

richvdh

lgtm

…_stream_token

These are tests for #8439

@Akkowicz

Synapse 1.22.0rc1 (2020-10-22) ============================== Features -------- - Add a configuration option for always using the "userinfo endpoint" for OpenID Connect. This fixes support for some identity providers, e.g. GitLab. Contributed by Benjamin Koch. ([\#7658](#7658)) - Add ability for `ThirdPartyEventRules` modules to query and manipulate whether a room is in the public rooms directory. ([\#8292](#8292), [\#8467](#8467)) - Add support for olm fallback keys ([MSC2732](matrix-org/matrix-spec-proposals#2732)). ([\#8312](#8312), [\#8501](#8501)) - Add support for running background tasks in a separate worker process. ([\#8369](#8369), [\#8458](#8458), [\#8489](#8489), [\#8513](#8513), [\#8544](#8544), [\#8599](#8599)) - Add support for device dehydration ([MSC2697](matrix-org/matrix-spec-proposals#2697)). ([\#8380](#8380)) - Add support for [MSC2409](matrix-org/matrix-spec-proposals#2409), which allows sending typing, read receipts, and presence events to appservices. ([\#8437](#8437), [\#8590](#8590)) - Change default room version to "6", per [MSC2788](matrix-org/matrix-spec-proposals#2788). ([\#8461](#8461)) - Add the ability to send non-membership events into a room via the `ModuleApi`. ([\#8479](#8479)) - Increase default upload size limit from 10M to 50M. Contributed by @Akkowicz. ([\#8502](#8502)) - Add support for modifying event content in `ThirdPartyRules` modules. ([\#8535](#8535), [\#8564](#8564)) Bugfixes -------- - Fix a longstanding bug where invalid ignored users in account data could break clients. ([\#8454](#8454)) - Fix a bug where backfilling a room with an event that was missing the `redacts` field would break. ([\#8457](#8457)) - Don't attempt to respond to some requests if the client has already disconnected. ([\#8465](#8465)) - Fix message duplication if something goes wrong after persisting the event. ([\#8476](#8476)) - Fix incremental sync returning an incorrect `prev_batch` token in timeline section, which when used to paginate returned events that were included in the incremental sync. Broken since v0.16.0. ([\#8486](#8486)) - Expose the `uk.half-shot.msc2778.login.application_service` to clients from the login API. This feature was added in v1.21.0, but was not exposed as a potential login flow. ([\#8504](#8504)) - Fix error code for `/profile/{userId}/displayname` to be `M_BAD_JSON`. ([\#8517](#8517)) - Fix a bug introduced in v1.7.0 that could cause Synapse to insert values from non-state `m.room.retention` events into the `room_retention` database table. ([\#8527](#8527)) - Fix not sending events over federation when using sharded event writers. ([\#8536](#8536)) - Fix a long standing bug where email notifications for encrypted messages were blank. ([\#8545](#8545)) - Fix increase in the number of `There was no active span...` errors logged when using OpenTracing. ([\#8567](#8567)) - Fix a bug that prevented errors encountered during execution of the `synapse_port_db` from being correctly printed. ([\#8585](#8585)) - Fix appservice transactions to only include a maximum of 100 persistent and 100 ephemeral events. ([\#8606](#8606)) Updates to the Docker image --------------------------- - Added multi-arch support (arm64,arm/v7) for the docker images. Contributed by @maquis196. ([\#7921](#7921)) - Add support for passing commandline args to the synapse process. Contributed by @samuel-p. ([\#8390](#8390)) Improved Documentation ---------------------- - Update the directions for using the manhole with coroutines. ([\#8462](#8462)) - Improve readme by adding new shield.io badges. ([\#8493](#8493)) - Added note about docker in manhole.md regarding which ip address to bind to. Contributed by @maquis196. ([\#8526](#8526)) - Document the new behaviour of the `allowed_lifetime_min` and `allowed_lifetime_max` settings in the room retention configuration. ([\#8529](#8529)) Deprecations and Removals ------------------------- - Drop unused `device_max_stream_id` table. ([\#8589](#8589)) Internal Changes ---------------- - Check for unreachable code with mypy. ([\#8432](#8432)) - Add unit test for event persister sharding. ([\#8433](#8433)) - Allow events to be sent to clients sooner when using sharded event persisters. ([\#8439](#8439), [\#8488](#8488), [\#8496](#8496), [\#8499](#8499)) - Configure `public_baseurl` when using demo scripts. ([\#8443](#8443)) - Add SQL logging on queries that happen during startup. ([\#8448](#8448)) - Speed up unit tests when using PostgreSQL. ([\#8450](#8450)) - Remove redundant database loads of stream_ordering for events we already have. ([\#8452](#8452)) - Reduce inconsistencies between codepaths for membership and non-membership events. ([\#8463](#8463)) - Combine `SpamCheckerApi` with the more generic `ModuleApi`. ([\#8464](#8464)) - Additional testing for `ThirdPartyEventRules`. ([\#8468](#8468)) - Add `-d` option to `./scripts-dev/lint.sh` to lint files that have changed since the last git commit. ([\#8472](#8472)) - Unblacklist some sytests. ([\#8474](#8474)) - Include the log level in the phone home stats. ([\#8477](#8477)) - Remove outdated sphinx documentation, scripts and configuration. ([\#8480](#8480)) - Clarify error message when plugin config parsers raise an error. ([\#8492](#8492)) - Remove the deprecated `Handlers` object. ([\#8494](#8494)) - Fix a threadsafety bug in unit tests. ([\#8497](#8497)) - Add user agent to user_daily_visits table. ([\#8503](#8503)) - Add type hints to various parts of the code base. ([\#8407](#8407), [\#8505](#8505), [\#8507](#8507), [\#8547](#8547), [\#8562](#8562), [\#8609](#8609)) - Remove unused code from the test framework. ([\#8514](#8514)) - Apply some internal fixes to the `HomeServer` class to make its code more idiomatic and statically-verifiable. ([\#8515](#8515)) - Factor out common code between `RoomMemberHandler._locally_reject_invite` and `EventCreationHandler.create_event`. ([\#8537](#8537)) - Improve database performance by executing more queries without starting transactions. ([\#8542](#8542)) - Rename `Cache` to `DeferredCache`, to better reflect its purpose. ([\#8548](#8548)) - Move metric registration code down into `LruCache`. ([\#8561](#8561), [\#8591](#8591)) - Replace `DeferredCache` with the lighter-weight `LruCache` where possible. ([\#8563](#8563)) - Add virtualenv-generated folders to `.gitignore`. ([\#8566](#8566)) - Add `get_immediate` method to `DeferredCache`. ([\#8568](#8568)) - Fix mypy not properly checking across the codebase, additionally, fix a typing assertion error in `handlers/auth.py`. ([\#8569](#8569)) - Fix `synmark` benchmark runner. ([\#8571](#8571)) - Modify `DeferredCache.get()` to return `Deferred`s instead of `ObservableDeferred`s. ([\#8572](#8572)) - Adjust a protocol-type definition to fit `sqlite3` assertions. ([\#8577](#8577)) - Support macOS on the `synmark` benchmark runner. ([\#8578](#8578)) - Update `mypy` static type checker to 0.790. ([\#8583](#8583), [\#8600](#8600)) - Re-organize the structured logging code to separate the TCP transport handling from the JSON formatting. ([\#8587](#8587)) - Remove extraneous unittest logging decorators from unit tests. ([\#8592](#8592)) - Minor optimisations in caching code. ([\#8593](#8593), [\#8594](#8594))

@Akkowicz

Synapse 1.22.0 (2020-10-27) =========================== No significant changes. Synapse 1.22.0rc2 (2020-10-26) ============================== Bugfixes -------- - Fix bugs where ephemeral events were not sent to appservices. Broke in v1.22.0rc1. ([\#8648](matrix-org/synapse#8648), [\#8656](matrix-org/synapse#8656)) - Fix `user_daily_visits` table to not have duplicate rows per user/device due to multiple user agents. Broke in v1.22.0rc1. ([\#8654](matrix-org/synapse#8654)) Synapse 1.22.0rc1 (2020-10-22) ============================== Features -------- - Add a configuration option for always using the "userinfo endpoint" for OpenID Connect. This fixes support for some identity providers, e.g. GitLab. Contributed by Benjamin Koch. ([\#7658](matrix-org/synapse#7658)) - Add ability for `ThirdPartyEventRules` modules to query and manipulate whether a room is in the public rooms directory. ([\#8292](matrix-org/synapse#8292), [\#8467](matrix-org/synapse#8467)) - Add support for olm fallback keys ([MSC2732](matrix-org/matrix-spec-proposals#2732)). ([\#8312](matrix-org/synapse#8312), [\#8501](matrix-org/synapse#8501)) - Add support for running background tasks in a separate worker process. ([\#8369](matrix-org/synapse#8369), [\#8458](matrix-org/synapse#8458), [\#8489](matrix-org/synapse#8489), [\#8513](matrix-org/synapse#8513), [\#8544](matrix-org/synapse#8544), [\#8599](matrix-org/synapse#8599)) - Add support for device dehydration ([MSC2697](matrix-org/matrix-spec-proposals#2697)). ([\#8380](matrix-org/synapse#8380)) - Add support for [MSC2409](matrix-org/matrix-spec-proposals#2409), which allows sending typing, read receipts, and presence events to appservices. ([\#8437](matrix-org/synapse#8437), [\#8590](matrix-org/synapse#8590)) - Change default room version to "6", per [MSC2788](matrix-org/matrix-spec-proposals#2788). ([\#8461](matrix-org/synapse#8461)) - Add the ability to send non-membership events into a room via the `ModuleApi`. ([\#8479](matrix-org/synapse#8479)) - Increase default upload size limit from 10M to 50M. Contributed by @Akkowicz. ([\#8502](matrix-org/synapse#8502)) - Add support for modifying event content in `ThirdPartyRules` modules. ([\#8535](matrix-org/synapse#8535), [\#8564](matrix-org/synapse#8564)) Bugfixes -------- - Fix a longstanding bug where invalid ignored users in account data could break clients. ([\#8454](matrix-org/synapse#8454)) - Fix a bug where backfilling a room with an event that was missing the `redacts` field would break. ([\#8457](matrix-org/synapse#8457)) - Don't attempt to respond to some requests if the client has already disconnected. ([\#8465](matrix-org/synapse#8465)) - Fix message duplication if something goes wrong after persisting the event. ([\#8476](matrix-org/synapse#8476)) - Fix incremental sync returning an incorrect `prev_batch` token in timeline section, which when used to paginate returned events that were included in the incremental sync. Broken since v0.16.0. ([\#8486](matrix-org/synapse#8486)) - Expose the `uk.half-shot.msc2778.login.application_service` to clients from the login API. This feature was added in v1.21.0, but was not exposed as a potential login flow. ([\#8504](matrix-org/synapse#8504)) - Fix error code for `/profile/{userId}/displayname` to be `M_BAD_JSON`. ([\#8517](matrix-org/synapse#8517)) - Fix a bug introduced in v1.7.0 that could cause Synapse to insert values from non-state `m.room.retention` events into the `room_retention` database table. ([\#8527](matrix-org/synapse#8527)) - Fix not sending events over federation when using sharded event writers. ([\#8536](matrix-org/synapse#8536)) - Fix a long standing bug where email notifications for encrypted messages were blank. ([\#8545](matrix-org/synapse#8545)) - Fix increase in the number of `There was no active span...` errors logged when using OpenTracing. ([\#8567](matrix-org/synapse#8567)) - Fix a bug that prevented errors encountered during execution of the `synapse_port_db` from being correctly printed. ([\#8585](matrix-org/synapse#8585)) - Fix appservice transactions to only include a maximum of 100 persistent and 100 ephemeral events. ([\#8606](matrix-org/synapse#8606)) Updates to the Docker image --------------------------- - Added multi-arch support (arm64,arm/v7) for the docker images. Contributed by @maquis196. ([\#7921](matrix-org/synapse#7921)) - Add support for passing commandline args to the synapse process. Contributed by @samuel-p. ([\#8390](matrix-org/synapse#8390)) Improved Documentation ---------------------- - Update the directions for using the manhole with coroutines. ([\#8462](matrix-org/synapse#8462)) - Improve readme by adding new shield.io badges. ([\#8493](matrix-org/synapse#8493)) - Added note about docker in manhole.md regarding which ip address to bind to. Contributed by @maquis196. ([\#8526](matrix-org/synapse#8526)) - Document the new behaviour of the `allowed_lifetime_min` and `allowed_lifetime_max` settings in the room retention configuration. ([\#8529](matrix-org/synapse#8529)) Deprecations and Removals ------------------------- - Drop unused `device_max_stream_id` table. ([\#8589](matrix-org/synapse#8589)) Internal Changes ---------------- - Check for unreachable code with mypy. ([\#8432](matrix-org/synapse#8432)) - Add unit test for event persister sharding. ([\#8433](matrix-org/synapse#8433)) - Allow events to be sent to clients sooner when using sharded event persisters. ([\#8439](matrix-org/synapse#8439), [\#8488](matrix-org/synapse#8488), [\#8496](matrix-org/synapse#8496), [\#8499](matrix-org/synapse#8499)) - Configure `public_baseurl` when using demo scripts. ([\#8443](matrix-org/synapse#8443)) - Add SQL logging on queries that happen during startup. ([\#8448](matrix-org/synapse#8448)) - Speed up unit tests when using PostgreSQL. ([\#8450](matrix-org/synapse#8450)) - Remove redundant database loads of stream_ordering for events we already have. ([\#8452](matrix-org/synapse#8452)) - Reduce inconsistencies between codepaths for membership and non-membership events. ([\#8463](matrix-org/synapse#8463)) - Combine `SpamCheckerApi` with the more generic `ModuleApi`. ([\#8464](matrix-org/synapse#8464)) - Additional testing for `ThirdPartyEventRules`. ([\#8468](matrix-org/synapse#8468)) - Add `-d` option to `./scripts-dev/lint.sh` to lint files that have changed since the last git commit. ([\#8472](matrix-org/synapse#8472)) - Unblacklist some sytests. ([\#8474](matrix-org/synapse#8474)) - Include the log level in the phone home stats. ([\#8477](matrix-org/synapse#8477)) - Remove outdated sphinx documentation, scripts and configuration. ([\#8480](matrix-org/synapse#8480)) - Clarify error message when plugin config parsers raise an error. ([\#8492](matrix-org/synapse#8492)) - Remove the deprecated `Handlers` object. ([\#8494](matrix-org/synapse#8494)) - Fix a threadsafety bug in unit tests. ([\#8497](matrix-org/synapse#8497)) - Add user agent to user_daily_visits table. ([\#8503](matrix-org/synapse#8503)) - Add type hints to various parts of the code base. ([\#8407](matrix-org/synapse#8407), [\#8505](matrix-org/synapse#8505), [\#8507](matrix-org/synapse#8507), [\#8547](matrix-org/synapse#8547), [\#8562](matrix-org/synapse#8562), [\#8609](matrix-org/synapse#8609)) - Remove unused code from the test framework. ([\#8514](matrix-org/synapse#8514)) - Apply some internal fixes to the `HomeServer` class to make its code more idiomatic and statically-verifiable. ([\#8515](matrix-org/synapse#8515)) - Factor out common code between `RoomMemberHandler._locally_reject_invite` and `EventCreationHandler.create_event`. ([\#8537](matrix-org/synapse#8537)) - Improve database performance by executing more queries without starting transactions. ([\#8542](matrix-org/synapse#8542)) - Rename `Cache` to `DeferredCache`, to better reflect its purpose. ([\#8548](matrix-org/synapse#8548)) - Move metric registration code down into `LruCache`. ([\#8561](matrix-org/synapse#8561), [\#8591](matrix-org/synapse#8591)) - Replace `DeferredCache` with the lighter-weight `LruCache` where possible. ([\#8563](matrix-org/synapse#8563)) - Add virtualenv-generated folders to `.gitignore`. ([\#8566](matrix-org/synapse#8566)) - Add `get_immediate` method to `DeferredCache`. ([\#8568](matrix-org/synapse#8568)) - Fix mypy not properly checking across the codebase, additionally, fix a typing assertion error in `handlers/auth.py`. ([\#8569](matrix-org/synapse#8569)) - Fix `synmark` benchmark runner. ([\#8571](matrix-org/synapse#8571)) - Modify `DeferredCache.get()` to return `Deferred`s instead of `ObservableDeferred`s. ([\#8572](matrix-org/synapse#8572)) - Adjust a protocol-type definition to fit `sqlite3` assertions. ([\#8577](matrix-org/synapse#8577)) - Support macOS on the `synmark` benchmark runner. ([\#8578](matrix-org/synapse#8578)) - Update `mypy` static type checker to 0.790. ([\#8583](matrix-org/synapse#8583), [\#8600](matrix-org/synapse#8600)) - Re-organize the structured logging code to separate the TCP transport handling from the JSON formatting. ([\#8587](matrix-org/synapse#8587)) - Remove extraneous unittest logging decorators from unit tests. ([\#8592](matrix-org/synapse#8592)) - Minor optimisations in caching code. ([\#8593](matrix-org/synapse#8593), [\#8594](matrix-org/synapse#8594))

Add an immutable mapping from instance name to ID

a23a313

erikjohnston force-pushed the erikj/vector_clock_stream_token branch from 2dfaae4 to a1ca8ed Compare October 1, 2020 15:32

erikjohnston added 3 commits October 1, 2020 16:41

Create new token format for encoding instance positions

b2172da

Wire up new token when fetching events streams

d7da8ca

Newsfile

fdf6c05

erikjohnston force-pushed the erikj/vector_clock_stream_token branch from a1ca8ed to fdf6c05 Compare October 1, 2020 15:42

erikjohnston requested a review from a team October 1, 2020 15:59

anoadragon453 reviewed Oct 1, 2020

View reviewed changes

anoadragon453 requested a review from a team October 1, 2020 16:06

erikjohnston and others added 2 commits October 2, 2020 10:49

Typos

eed3cf5

Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>

Merge remote-tracking branch 'origin/develop' into erikj/vector_clock…

2067dc7

…_stream_token

erikjohnston marked this pull request as ready for review October 5, 2020 11:25

erikjohnston mentioned this pull request Oct 5, 2020

Reduce duplication in event ordering lookup methods #8453

Closed

richvdh reviewed Oct 5, 2020

View reviewed changes

erikjohnston and others added 4 commits October 6, 2020 09:59

Apply suggestions from code review

f709da9

Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>

Fix up RoomStreamToken

e91f0e9

Fix up _filter_results

604e33f

Fixup get_room_max_token

6425f43

erikjohnston force-pushed the erikj/vector_clock_stream_token branch from 505b373 to 6425f43 Compare October 6, 2020 10:55

erikjohnston added 2 commits October 6, 2020 11:57

Fix always False condition to just be False

93c8091

Switch '.' and '~' in new token format

a321252

erikjohnston commented Oct 6, 2020

View reviewed changes

erikjohnston requested a review from richvdh October 6, 2020 11:05

richvdh reviewed Oct 7, 2020

View reviewed changes

Improve RoomStreamToken

cb7c48c

erikjohnston requested a review from richvdh October 7, 2020 13:13

richvdh approved these changes Oct 7, 2020

View reviewed changes

erikjohnston added 2 commits October 7, 2020 14:17

Merge remote-tracking branch 'origin/develop' into erikj/vector_clock…

c0858cc

…_stream_token

Fix unit tests

05c2058

erikjohnston merged commit 52a50e8 into develop Oct 7, 2020

erikjohnston deleted the erikj/vector_clock_stream_token branch October 7, 2020 14:15

erikjohnston mentioned this pull request Oct 7, 2020

Add basic tests for sync/pagination with vector clock tokens. #8488

Merged

erikjohnston added a commit that referenced this pull request Oct 14, 2020

Add basic tests for sync/pagination with vector clock tokens. (#8488)

1264c8a

These are tests for #8439

richvdh mentioned this pull request Jun 9, 2021

Sync requests immediately return with empty payload #8518

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Use vector clocks for room stream tokens. #8439

Use vector clocks for room stream tokens. #8439

erikjohnston commented Oct 1, 2020 •

edited by richvdh

Loading

anoadragon453 left a comment

richvdh Oct 5, 2020

richvdh Oct 5, 2020

erikjohnston Oct 6, 2020

richvdh Oct 5, 2020

erikjohnston Oct 6, 2020

erikjohnston Oct 6, 2020

erikjohnston Oct 6, 2020

richvdh Oct 7, 2020

erikjohnston Oct 7, 2020

richvdh left a comment

richvdh Oct 7, 2020

richvdh Oct 7, 2020

richvdh Oct 7, 2020

richvdh Oct 7, 2020

richvdh left a comment

		def as_tuple(self) -> Tuple[Optional[int], int]:
		return (self.topological, self.stream)

-        type=Dict[str, int], factory=dict, validator=attr.validators.instance_of(dict)
+        type=Dict[str, int], factory=dict, validator=attr.validators.deep_mapping(
+            key_validator=attr.validators.instance_of(str),
+            value_validator=attr.validators.instance_of(int),
+            mapping_validator=attr.validators.instance_of(dict),
+       ),

	"""Get the stream position that the writer was at at this token.
	"""Get the stream position that the given writer was at at this token.

Use vector clocks for room stream tokens. #8439

Use vector clocks for room stream tokens. #8439

Conversation

erikjohnston commented Oct 1, 2020 • edited by richvdh Loading

anoadragon453 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

richvdh left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

richvdh left a comment

Choose a reason for hiding this comment

erikjohnston commented Oct 1, 2020 •

edited by richvdh

Loading