This repository has been archived by the owner on Apr 26, 2024. It is now read-only.
Refactor /context to reuse pagination storage functions #3193
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
richvdh
reviewed
May 8, 2018
synapse/storage/stream.py
Outdated
| direction='b', limit=-1, event_filter=None): | ||
| Returns: | ||
| tuple[list[dict], str]: Returns the results as a list of dicts and | ||
| a token that points to the end of the result set. The dicts haveq |
synapse/storage/stream.py
Outdated
| Returns: | ||
| tuple[list[dict], str]: Returns the results as a list of dicts and | ||
| a token that points to the end of the result set. The dicts haveq | ||
| the keys "event_id", "toplogical_ordering" and "stream_orderign". |
synapse/storage/stream.py
Outdated
| @@ -738,38 +678,49 @@ def update_federation_out_pos(self, typ, stream_id): | |||
| def has_room_changed_since(self, room_id, stream_id): | |||
| return self._events_stream_cache.has_entity_changed(room_id, stream_id) | |||
|
|
|||
| def paginate_room_events_txn(self, txn, room_id, from_token, to_token=None, | |||
synapse/storage/stream.py
Outdated
| @@ -738,38 +678,49 @@ def update_federation_out_pos(self, typ, stream_id): | |||
| def has_room_changed_since(self, room_id, stream_id): | |||
| return self._events_stream_cache.has_entity_changed(room_id, stream_id) | |||
|
|
|||
| def paginate_room_events_txn(self, txn, room_id, from_token, to_token=None, | |||
There was a problem hiding this comment.
Is there any change in functionality here, or is it just:
- move paginate_room_events to StreamWorkerStore instead of StreamStore
- pull paginate_room_events_txn out to a member function rather than being a nested
fwithin paginate_room_events - do some of the stuff that was in paginate_room_events in paginate_room_events_txn?
I think it's the latter but I'm not sure if I'm missing something.
There was a problem hiding this comment.
Yup, there shouldn't be any functional change here.
(I really wish GH made it clearer that stuff is mainly white space change, e.g. 06c0d0e?w=1 makes this a lot clearer imo)
|
Woops sorry for not splitting up in to two PRs |
neilisfragile
added a commit
that referenced
this pull request
May 18, 2018
Changes in synapse v0.29.1 (2018-05-17) ========================================== Changes: * Update docker documentation (PR #3222) Changes in synapse v0.29.0 (2018-05-16) =========================================== Not changes since v0.29.0-rc1 Changes in synapse v0.29.0-rc1 (2018-05-14) =========================================== Notable changes, a docker file for running Synapse (Thanks to @kaiyou!) and a closed spec bug in the Client Server API. Additionally further prep for Python 3 migration. Potentially breaking change: * Make Client-Server API return 401 for invalid token (PR #3161). This changes the Client-server spec to return a 401 error code instead of 403 when the access token is unrecognised. This is the behaviour required by the specification, but some clients may be relying on the old, incorrect behaviour. Thanks to @NotAFile for fixing this. Features: * Add a Dockerfile for synapse (PR #2846) Thanks to @kaiyou! Changes - General: * nuke-room-from-db.sh: added postgresql option and help (PR #2337) Thanks to @rubo77! * Part user from rooms on account deactivate (PR #3201) * Make 'unexpected logging context' into warnings (PR #3007) * Set Server header in SynapseRequest (PR #3208) * remove duplicates from groups tables (PR #3129) * Improve exception handling for background processes (PR #3138) * Add missing consumeErrors to improve exception handling (PR #3139) * reraise exceptions more carefully (PR #3142) * Remove redundant call to preserve_fn (PR #3143) * Trap exceptions thrown within run_in_background (PR #3144) Changes - Refactors: * Refactor /context to reuse pagination storage functions (PR #3193) * Refactor recent events func to use pagination func (PR #3195) * Refactor pagination DB API to return concrete type (PR #3196) * Refactor get_recent_events_for_room return type (PR #3198) * Refactor sync APIs to reuse pagination API (PR #3199) * Remove unused code path from member change DB func (PR #3200) * Refactor request handling wrappers (PR #3203) * transaction_id, destination defined twice (PR #3209) Thanks to @damir-manapov! * Refactor event storage to prepare for changes in state calculations (PR #3141) * Set Server header in SynapseRequest (PR #3208) * Use deferred.addTimeout instead of time_bound_deferred (PR #3127, #3178) * Use run_in_background in preference to preserve_fn (PR #3140) Changes - Python 3 migration: * Construct HMAC as bytes on py3 (PR #3156) Thanks to @NotAFile! * run config tests on py3 (PR #3159) Thanks to @NotAFile! * Open certificate files as bytes (PR #3084) Thanks to @NotAFile! * Open config file in non-bytes mode (PR #3085) Thanks to @NotAFile! * Make event properties raise AttributeError instead (PR #3102) Thanks to @NotAFile! * Use six.moves.urlparse (PR #3108) Thanks to @NotAFile! * Add py3 tests to tox with folders that work (PR #3145) Thanks to @NotAFile! * Don't yield in list comprehensions (PR #3150) Thanks to @NotAFile! * Move more xrange to six (PR #3151) Thanks to @NotAFile! * make imports local (PR #3152) Thanks to @NotAFile! * move httplib import to six (PR #3153) Thanks to @NotAFile! * Replace stringIO imports with six (PR #3154, #3168) Thanks to @NotAFile! * more bytes strings (PR #3155) Thanks to @NotAFile! Bug Fixes: * synapse fails to start under Twisted >= 18.4 (PR #3157) * Fix a class of logcontext leaks (PR #3170) * Fix a couple of logcontext leaks in unit tests (PR #3172) * Fix logcontext leak in media repo (PR #3174) * Escape label values in prometheus metrics (PR #3175, #3186) * Fix 'Unhandled Error' logs with Twisted 18.4 (PR #3182) Thanks to @Half-Shot! * Fix logcontext leaks in rate limiter (PR #3183) * notifications: Convert next_token to string according to the spec (PR #3190) Thanks to @mujx! * nuke-room-from-db.sh: fix deletion from search table (PR #3194) Thanks to @rubo77! * add guard for None on purge_history api (PR #3160) Thanks to @krombel!
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The aim here is to get all code that tries to return events in topological order going through the same storage function so that we can later change the way we order events in the DB.
This also removes a query optimisation for SQLite for
/contextAPI. Ideally we'd bump the minimum version of SQLite to 3.15 and use tuple comparison as we do for postgres, however that turns out to be faffy and can be done separately (if at all). Since we don't have this optimisation in place for/messagesit feels acceptable to remove it for now.