Skip to content
This repository has been archived by the owner on Nov 25, 2024. It is now read-only.

Peeking via MSC2753 #1370

Merged
merged 66 commits into from
Sep 10, 2020
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
66 commits
Select commit Hold shift + click to select a range
b9342d9
a very very WIP first cut of peeking via MSC2753.
ara4n Aug 30, 2020
d7bdf71
make PeekingDeviceSet private
ara4n Aug 30, 2020
cfa0be5
merge master
ara4n Aug 31, 2020
9b79f9a
add server_name param
ara4n Aug 31, 2020
d343b8f
blind stab at adding a `peek` section to /sync
ara4n Aug 31, 2020
c4e5f60
make it build
ara4n Aug 31, 2020
d1e4d66
make it launch
ara4n Aug 31, 2020
f006b37
add peeking to getResponseWithPDUsForCompleteSync
ara4n Aug 31, 2020
6c3a896
cancel any peeks when we join a room
ara4n Aug 31, 2020
7b38d48
spell out how to runoutside of docker if you want speed
ara4n Aug 31, 2020
e589984
fix SQL
ara4n Aug 31, 2020
0bb2c2c
remove unnecessary txn for SelectPeeks
ara4n Aug 31, 2020
28219c6
Merge branch 'master' into matthew/peeking
ara4n Sep 1, 2020
86e9736
fix s/join/peek/ cargocult fail
ara4n Sep 1, 2020
d0d5f70
Merge branch 'master' into matthew/peeking
ara4n Sep 1, 2020
bfecc8e
HACK: Track goroutine IDs to determine when we write by the wrong thread
kegsay Sep 1, 2020
7bf2a27
Track partition offsets and only log unsafe for non-selects
kegsay Sep 1, 2020
fcdb90c
Put redactions in the writer goroutine
kegsay Sep 1, 2020
6410b70
Update filters on writer goroutine
kegsay Sep 1, 2020
ed4b3a5
Merge branch 'kegan/redact-txn' into matthew/peeking
ara4n Sep 1, 2020
3cebd8d
Merge branch 'kegan/HACK-goid-sqlite-db-is-locked' into matthew/peeking
ara4n Sep 1, 2020
5d7f688
wrap peek storage in goid hack
ara4n Sep 1, 2020
6424117
use exclusive writer, and MarkPeeksAsOld more efficiently
ara4n Sep 1, 2020
85bce11
don't log ascii in binary at sql trace...
ara4n Sep 1, 2020
75b91ac
strip out empty roomd deltas
ara4n Sep 1, 2020
b6cc441
re-add txn to SelectPeeks
ara4n Sep 2, 2020
f6af656
re-add accidentally deleted field
ara4n Sep 2, 2020
8712ea3
Merge branch 'master' into matthew/peeking
ara4n Sep 3, 2020
eda84cd
reject peeks for non-worldreadable rooms
ara4n Sep 3, 2020
da3742c
move perform_peek
ara4n Sep 3, 2020
3c5e079
fix package
ara4n Sep 3, 2020
c775643
correctly refactor perform_peek
ara4n Sep 3, 2020
4f5c8ca
Merge branch 'master' into matthew/peeking
ara4n Sep 3, 2020
2b8f0b8
Merge branch 'master' into matthew/peeking
neilalexander Sep 4, 2020
56d772f
Cancel peeks on join again
neilalexander Sep 4, 2020
9ad05e7
Merge branch 'master' into matthew/peeking
neilalexander Sep 4, 2020
ca0406e
Fix unit test
neilalexander Sep 4, 2020
83c559c
Merge branch 'master' into matthew/peeking
neilalexander Sep 4, 2020
64fe274
Merge branch 'master' into matthew/peeking
neilalexander Sep 4, 2020
5c04c52
Add PerformPeek full-HTTP serverside
neilalexander Sep 4, 2020
ff65f0e
Revert "Merge branch 'kegan/HACK-goid-sqlite-db-is-locked' into matth…
neilalexander Sep 4, 2020
2ccd4fa
Clear a whole bunch of lint issues
neilalexander Sep 4, 2020
8bbccab
Merge branch 'matthew/peeking' of github.com:matrix-org/dendrite into…
neilalexander Sep 4, 2020
843b7a7
Fix PerformPeekPath
neilalexander Sep 4, 2020
55c7f2c
add (broken) postgres; advance streampos whenever sync output changes
ara4n Sep 7, 2020
034ff32
move peek-cancelling to consumer
ara4n Sep 8, 2020
7f41f39
remove erroneous commentary
ara4n Sep 8, 2020
9f2bc62
(broken) rewrite to use SelectPeeksInRange rather than MarkPeeksAsOld…
ara4n Sep 8, 2020
b96a31d
track transitions more clearly
ara4n Sep 8, 2020
7a76f49
notify new streampos after joining peeked rooms
ara4n Sep 8, 2020
56001d0
Only call Membership() on membership events
neilalexander Sep 8, 2020
6fe281a
Don't return sql.ErrNoRows on DeletePeeks since that is not an error …
neilalexander Sep 8, 2020
bcbe651
Merge branch 'master' into matthew/peeking
neilalexander Sep 8, 2020
faa070b
Don't panic
neilalexander Sep 8, 2020
de53608
fix nightmare bug where sqlite doesn't let you use out of order sub s…
ara4n Sep 9, 2020
b45436a
handle exclusive writer txn for cleanliness
ara4n Sep 9, 2020
052351e
go fmt
ara4n Sep 9, 2020
2300549
fix lint
ara4n Sep 9, 2020
d2a0bad
Merge branch 'master' into matthew/peeking
ara4n Sep 9, 2020
f09afe0
Remove accidental formerly-untracked file
neilalexander Sep 9, 2020
15349d8
Return errors from SQL statements to handle rollbacks correctly
kegsay Sep 9, 2020
34b8961
Fix sqlite DeletePeeks API to match postgres; fix bug which incorrect…
kegsay Sep 9, 2020
af47237
Add peeking tests to whitelist
kegsay Sep 9, 2020
8cec7e0
fix lots of places where we didn't rollback txns upon go errs (#1417)
ara4n Sep 9, 2020
b1d1ccd
allow duplicate stream ids in the peeks table
ara4n Sep 10, 2020
9561843
stop the peek table growing by reusing rows correctly
ara4n Sep 10, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 3 additions & 2 deletions syncapi/storage/postgres/peeks_table.go
Original file line number Diff line number Diff line change
Expand Up @@ -27,13 +27,14 @@ import (

const peeksSchema = `
CREATE TABLE IF NOT EXISTS syncapi_peeks (
id BIGINT PRIMARY KEY DEFAULT nextval('syncapi_stream_id'),
id BIGINT DEFAULT nextval('syncapi_stream_id'),
room_id TEXT NOT NULL,
user_id TEXT NOT NULL,
device_id TEXT NOT NULL,
deleted BOOL NOT NULL DEFAULT false,
-- When the peek was created in UNIX epoch ms.
creation_ts BIGINT NOT NULL
creation_ts BIGINT NOT NULL,
UNIQUE(room_id, user_id, device_id)
);

CREATE INDEX IF NOT EXISTS syncapi_peeks_room_id_idx ON syncapi_peeks(room_id);
Expand Down
5 changes: 3 additions & 2 deletions syncapi/storage/sqlite3/peeks_table.go
Original file line number Diff line number Diff line change
Expand Up @@ -27,13 +27,14 @@ import (

const peeksSchema = `
CREATE TABLE IF NOT EXISTS syncapi_peeks (
id INTEGER PRIMARY KEY,
id INTEGER,
room_id TEXT NOT NULL,
user_id TEXT NOT NULL,
device_id TEXT NOT NULL,
deleted BOOL NOT NULL DEFAULT false,
-- When the peek was created in UNIX epoch ms.
creation_ts INTEGER NOT NULL
creation_ts INTEGER NOT NULL,
UNIQUE(room_id, user_id, device_id)
ara4n marked this conversation as resolved.
Show resolved Hide resolved
);

CREATE INDEX IF NOT EXISTS syncapi_peeks_room_id_idx ON syncapi_peeks(room_id);
Expand Down