Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix bug in sliding sync when using old DB. (default instance_name to "master") #17398

Merged
merged 10 commits into from
Jul 8, 2024
Merged

Fix bug in sliding sync when using old DB. (default instance_name to "master") #17398

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
59314d5
Fix bug in sliding sync when using old DB.
erikjohnston Jul 4, 2024
c742b74
Newsfile
erikjohnston Jul 4, 2024
b2880b6
Update changelog.d/17398.bugfix
erikjohnston Jul 4, 2024
b177307
Also fixup internal metadata
erikjohnston Jul 4, 2024
5a47fed
Remove unused function
erikjohnston Jul 4, 2024
24444a4
Lint
erikjohnston Jul 4, 2024
d15d8ab
Have _filter_results handle null instance_name
erikjohnston Jul 5, 2024
5f1612e
Have _filter_results_by_stream handle null instance_name
erikjohnston Jul 5, 2024
fb2fc47
Update synapse/storage/databases/main/events_worker.py
erikjohnston Jul 5, 2024
c8f1411
Remove assertions
erikjohnston Jul 8, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions changelog.d/17398.bugfix
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Fix bug in experimenntal sliding sync support when using an old database.
erikjohnston marked this conversation as resolved.
Show resolved Hide resolved
6 changes: 5 additions & 1 deletion synapse/storage/databases/main/roommember.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -494,7 +494,11 @@ def _get_rooms_for_local_user_where_membership_is_txn(
sender=sender,
membership=membership,
event_id=event_id,
event_pos=PersistedEventPosition(instance_name, stream_ordering),
event_pos=PersistedEventPosition(
# If instance_name is null we default to "master"
instance_name or "master",
Copy link
Contributor

@MadLittleMods MadLittleMods Jul 4, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should probably have some constant to represent "master" to avoid typos (subtle bugs)

(can iterate in another PR)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The typing from the SQL queries makes this type of thing too easy. Kinda wish we had to duck-type all of it to narrow things down (forced by the linter instead of letting it slide because Any)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's no real easy way of annotating the types of the returned rows that aren't error prone, e.g. here we'd probably assume instance_name column was NON NULL.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess we just need to use the disallow-any-xxx options in mypy so we can't just assign Any to something that expects a str.

Copy link
Contributor

@reivilibre reivilibre Jul 5, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have thought about this in the past, the best idea I came up with is: during the trial tests, have Postgres return the type information of the SQL statements (and combine this with an API change on the fetch functions where you tell it what type you're expecting and this is used as the return type). Then it could assert that the Postgres type information matches what is expected. (You'd disable this at real runtime to avoid any performance hits)
Ultimately this is loosely inspired by the sqlx library in Rust (where type information flows out from the database at or ahead of compile-time) but I think for us it'd be too much work for too little benefit now.

stream_ordering,
),
room_version_id=room_version,
)
erikjohnston marked this conversation as resolved.
Show resolved Hide resolved
for room_id, sender, membership, event_id, instance_name, stream_ordering, room_version in txn
Expand Down
10 changes: 7 additions & 3 deletions synapse/storage/databases/main/stream.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -936,7 +936,8 @@ def f(txn: LoggingTransaction) -> List[CurrentStateDeltaMembership]:
# Event
erikjohnston marked this conversation as resolved.
Show resolved Hide resolved
event_id=event_id,
event_pos=PersistedEventPosition(
instance_name=instance_name,
# If instance_name is null we default to "master"
instance_name=instance_name or "master",
MadLittleMods marked this conversation as resolved.
Show resolved Hide resolved
erikjohnston marked this conversation as resolved.
Show resolved Hide resolved
stream=stream_ordering,
),
# When `s.event_id = null`, we won't be able to get respective
Expand All @@ -952,7 +953,8 @@ def f(txn: LoggingTransaction) -> List[CurrentStateDeltaMembership]:
prev_event_id=prev_event_id,
prev_event_pos=(
PersistedEventPosition(
instance_name=prev_instance_name,
# If instance_name is null we default to "master"
instance_name=prev_instance_name or "master",
erikjohnston marked this conversation as resolved.
Show resolved Hide resolved
stream=prev_stream_ordering,
)
if (
erikjohnston marked this conversation as resolved.
Show resolved Hide resolved
Expand Down Expand Up @@ -1257,7 +1259,9 @@ def get_last_event_pos_in_room_before_stream_ordering_txn(
stream_ordering=stream_ordering,
):
return event_id, PersistedEventPosition(
instance_name, stream_ordering
# If instance_name is null we default to "master"
instance_name or "master",
erikjohnston marked this conversation as resolved.
Show resolved Hide resolved
stream_ordering,
)

erikjohnston marked this conversation as resolved.
Show resolved Hide resolved
return None
erikjohnston marked this conversation as resolved.
Show resolved Hide resolved
Expand Down
Loading