We need a way to be able to expire data from a HS. (SPEC-141)

[matrixbot]

This could be implementation-specific, or via a standard admin API or per-room state configuration.

Eitherway, we need a way to tell HSes to:

• no longer store this room
• only keep N messages for this room
• expire more than N messages for this room
• only keep X days of messages for this room
• purge state from this room (as opposed to messages)?

(Imported from https://matrix.org/jira/browse/SPEC-141)

(Reported by @ara4n)

[matrixbot]

Jira watchers: @ara4n

[matrixbot]

Links exported from Jira:

relates to SPEC-224
relates to https://github.com/matrix-org/matrix-doc/issues/440
relates to SPEC-216
relates to SYN-361
relates to SYN-601

[matrixbot]

The "No longer store this room" use case is now important given the history of rooms we have left are now returned to clients - we need a way to tell the server "no, really, leave and forget this conversation forever please".

-- @ara4n

[matrixbot]

(A workaround for this in the short term would be to tag rooms you want to purge as 'purge' or something via SPEC-218 and filter them out on the client, but clearly this should be done serverside)

-- @ara4n

[ara4n]

Bumping up the priority on this, as it's causing large homeserver admins (e.g. disroot.org) to consider discontinuing the service as their DB keeps growing and reaping it is hard.

In practice, i think what's needed is a way for ops to specify the desired history retention scheme for a given room - and for server admins to specify a maximum history retention in general.

(synapse bug matrix-org/synapse#1480)

[monkz]

I like to request a warning of some sort for this feature, so people can export their chatlogs before the culling.

[verymilan]

Maybe it's not worth mentioning but i'd like to add that the current way of deleting the chathistory is not very effective in terms of space usage as i ran against many big rooms only keeping 30days with synpurge and after a reindex only got like 20-30G free.
...so a more effective way would be very nice for such a feature.

[ara4n]

This is solved by matrix-org/matrix-spec-proposals#1763, which is implemented in matrix-org/synapse#5815. This is currently being ported into mainline synapse.

[MurzNN]

Does messages removed fully with all metadata (so no one record kept in database about that message was exist before), or only message content?

[benyanke]

Any progress on this? This is a big win for some for my usecases.

[X448NAX]

Any progress on this? This is a big win for some for my usecases.

It has been implemented upstream and can be set in the server config:

https://github.com/matrix-org/synapse/blob/develop/docs/message_retention_policies.md

[Mikaela]

I don't think Synapse is matrix-doc upstream and it's not the only server implementation either.

[natecovington]

This might not be the right way to request such a thing, but if I could wave my magic wand, I'd build this into Synapse Admin, so I can delete an individual users' media:

And something like this into the "room settings" so I can control the room message / media history settings right from Element:

[catfromplan9]

This might not be the right way to request such a thing, but if I could wave my magic wand, I'd build this into Synapse Admin, so I can delete an individual users' media:

And something like this into the "room settings" so I can control the room message / media history settings right from Element:

We need this implementation

[natecovington]

As far as I know, this would be a custom development job, I don't see the entire Matrix ecosystem rolling this out. The closest you can get is with Synapse, you can configure server-wide retention policies:
https://github.com/matrix-org/synapse/blob/develop/docs/message_retention_policies.md

Actually, I'm seeing retention policy set on a per-room basis here:
https://github.com/matrix-org/synapse/blob/develop/docs/message_retention_policies.md

But it's not something you'd set from within Element. I think you'd need command line access to the server.

[catfromplan9]

As far as I know, this would be a custom development job, I don't see the entire Matrix ecosystem rolling this out. The closest you can get is with Synapse, you can configure server-wide retention policies: https://github.com/matrix-org/synapse/blob/develop/docs/message_retention_policies.md

Actually, I'm seeing retention policy set on a per-room basis here: https://github.com/matrix-org/synapse/blob/develop/docs/message_retention_policies.md

But it's not something you'd set from within Element. I think you'd need command line access to the server.

I saw that the reason it is a server-specific option is because of some issues with possible database corruption. Has anyone looked into this and tried to fix it? Why is it still an issue years after it was implemented in synapse?

[natecovington]

Can you give more details about the alleged database corruption? I think the general challenge here is what happens if you have two entirely separate Matrix servers sharing the same room(s) and they don't "agree" on how long to retain messages. If you're in a closed ecosystem (e.g. no federation) this is probably less of an issue.

[turt2live]

matrix-org/synapse#13476 is the corruption issue.

[natecovington]

Interesting. I wonder if that's where having multiple matrix servers meshed together would save from that corruption issue, or does it make it worse?