garbage collect forgotten rooms

[ara4n]

if everyone on a server has left and forgotten a given room, we should GC it from the DB

[richvdh]

I'm surprised we don't already have this, but I can't find it if we do.

Related:

• RoomDirectory: use SimpleRoomHeader instead of RoomHeader element-hq/element-web#1307
• By default, don't do URL previews in a room for m.notices element-hq/element-web#2338
• https://github.com/vector-im/riot-web/issues/6978

[aaronraimist]

This should probably get a Mozilla label https://bugzilla.mozilla.org/show_bug.cgi?id=1622403

[schildbach]

This seems like no-brainer to me. I just deleted gigabytes of data from my postgres DB just by manually removing abandoned rooms.

[peterhoeg]

Do you have a (relatively) safe script to do that @schildbach ?

[ptman]

@peterhoeg you can get a list of rooms and remove them via the admin api: https://github.com/matrix-org/synapse/blob/develop/docs/admin_api/rooms.md

Directly modifying the database is error prone and should not be done.

Also synapse admin ui can let you click around instead of writing code for the API: https://github.com/Awesome-Technologies/synapse-admin

[schildbach]

@peterhoeg I used the instructions provided here. Yes, it uses the admin API for write operations. Still, I think synapse could automatically purge a room if the last local user left it.

[clokep]

I suspect to have this done automatically we would want to:

1. Add a background update that searches for rooms with 0 joined members.
2. Iterates over them and calls purge.

There's a couple of tricky pieces here:

1. Do we care how long ago this had no members in it? (Do you want to immediately purge the room or wait a few days?)
2. Some logic to ensure only a single background update is running at once, as purging can take a while.

[schildbach]

Alternatively, it could simply purge immediately, within the "last member left" operation. For me, waiting a few days doesn't add any value.

[olmari]

How about following set retention minimum policy? Would be most logical, so in case of fould play it would still be there to dig if so needed within same retention policy numbers as any other related event.

[ara4n]

I suspect to have this done automatically we would want to:

1. Add a background update that searches for rooms with 0 joined members.

N.B. a room with 0 joined members is distinct from a room where all the local members have explicitly forgotten it. You should only purge/gc ones which have been explicitly forgotten, rather than just parted (given we let users read their historical rooms unless they forget them - like a recycle bin in a filemanager).

[richvdh]

Note that #11521 would be a blocker for this.

[richvdh]

This is also somewhat related to GDPR: if all users in a room are deactivated, the room itself should be garbage collected (element-hq/riot-meta#165).

[richvdh]

Obviously, when a room is deleted, we should also delete any media only referenced from that room (see also #1263)

[ghost]

@richvdh Would it be possible to transfer #element-hq/riot-meta#165 to element-meta, if the issue is still relevant?

[richvdh]

@richvdh Would it be possible to transfer vector-im/riot-meta#165 to element-meta, if the issue is still relevant?

Not without unarchiving the project, which is too much like hard work.

[DemiMarie]

Spaces also need to be subject to garbage collecition.

[olmari]

@DemiMarie Well, space is an room, so I'd imagine all this will touch spaces too(?)

[DemiMarie]

@olmari from a user perspective the two are not the same.

[Malix-Labs]

Spaces also need to be subject to garbage collection.

Can someone confirm it currently is / isn't ?

[Malix-Labs]

Also, is the garbage collection process instant after the last user forgot the room?
If not, how often does the garbage collection process occur?

[schildbach]

It seems like you need to add

forgotten_room_retention_period: 1d

to your homeserver.yaml and that implies it isn't instant.

[Malix-Labs]

It seems like you need to add

forgotten_room_retention_period: 1d

to your homeserver.yaml and that implies it isn't instant.

Nice to know.
Is 1d the minimum value for forgotten_room_retention_period?
Do you have documentation about it?

[sebix]

https://matrix-org.github.io/synapse/latest/usage/configuration/config_documentation.html?highlight=forgotten_room_retention_period#forgotten_room_retention_period

[clokep]

Spaces also need to be subject to garbage collection.

Can someone confirm it currently is / isn't ?

Spaces will also be cleaned up, they're just rooms.