Synapse doesn't limit data/time for http previews

[Rafaeltheraven]

Description

I have an Icecast stream running and a while back decided to send a link to it to someone over Matrix. A couple of days later I check the admin panel and noticed the following:

It seems that, while attempting to make the preview for the url, synapse started listening to the stream and it never stopped, because the stream never ends. Now this is a very specific case which could be solved by me properly disallowing preview fetching on my home network, but imagine if there were other homeservers in that room.

This points to the fact that synapse doesn't properly limit itself when trying to generate http previews, causing it to keep busy forever. In fact, this seems to happen multiple times as at the time of writing there are 3 synapse clients listening to the stream.

Steps to reproduce

• Have something with an infinite stream (such as http radio)
• Post a link to this stream in a room
• Any server having previews enabled will now be listening to the stream indefinitely.

Expected Behaviour

It would probably be best if Synapse kills the "thread" when it notices it can't make the preview/it takes too long. Looking at the logs, it seems Synapse does notice that the stream is too large, but it still keeps listening:

2020-09-12 10:29:43,512 - synapse.http.client - 339 - INFO - GET-3961617 - Received response to GET https://mkmkm.thebias.nl/: 200
2020-09-12 10:33:34,790 - synapse.http.server - 75 - INFO - GET-3961617 - <XForwardedForRequest at 0x7f5c555f3710 method='GET' uri='/_matrix/media/r0/preview_url?url=https%3A%2F%2Fmkmkm.thebias.nl%2F&ts=1599673680000' clientproto='HTTP/1.0' site=8008> SynapseError: 502 - Requested file is too large > 10485760 bytes
2020-09-12 10:33:34,790 - synapse.http.server - 531 - WARNING - GET-3961617 - Not sending response to request <XForwardedForRequest at 0x7f5c555f3710 method='GET' uri='/_matrix/media/r0/preview_url?url=https%3A%2F%2Fmkmkm.thebias.nl%2F&ts=1599673680000' clientproto='HTTP/1.0' site=8008>, already disconnected.
2020-09-12 10:33:34,791 - synapse.access.http.8008 - 311 - INFO - GET-3961617 - 2001:67c:2564:331:5d2:b8cb:8cf7:a474 - 8008 - {@rafael:thebias.nl} Processed request: 231.856sec/-171.857sec (0.004sec, 0.000sec) (0.000sec/0.002sec/1) 0B 200! "GET /_matrix/media/r0/preview_url?url=https%3A%2F%2Fmkmkm.thebias.nl%2F&ts=1599673680000 HTTP/1.0" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:81.0) Gecko/20100101 Firefox/81.0" [0 dbevts]

Version information

• Homeserver: Personal Homeserver

• Version: 1.19.1

• Install method: debian package

• Platform: debian 10

[erikjohnston]

Looks like we only time out fetching response headers, not reading the body

[laama1]

I can confirm, this issue still exists on Synapse/1.44.0

[squahtx]

A side effect of this issue is that files and folders in the url_cache directory don't get cleaned up properly: #10863 (comment)

[anarcat]

as a workaround, i have a nginx frontend in front of my icecast and adedd the following discussion to block synapse clients from hogging the feeds:

# Synapse/1.50.0rc1 (b=matrix-org-hotfixes,8e8a00829)
if ($http_user_agent ~ "^Synapse/.*"){
      return 403;
}

[meskio]

I guess another work around is to disable url previews completelly in homeserver.yaml:

url_preview_enabled: False

[clokep]

Should be fixed by #11784.

[jvoisin]

Shouldn't this DoS be considered as a security issue?

[anarcat]

as a workaround, i have a nginx frontend in front of my icecast and adedd the following discussion to block synapse clients from hogging the feeds:

# Synapse/1.50.0rc1 (b=matrix-org-hotfixes,8e8a00829)
if ($http_user_agent ~ "^Synapse/.*"){
      return 403;
}

for what it's worth, that didn't actually work at all and a proper workaround is:

    if ($http_user_agent ~ Synapse){
      return 403;
    }

i'll note that I still see Synapse clients on my streams, latest of which being Synapse (bot; +https://github.com/matrix-org/synapse). woot.