exception on sync with filter

[L0ric0]

Description

on a sync of the mautrix-whatsapp bridge synapse errors out and returns a 500 M_UNKNOWN.

sync filter:

{
    "account_data": { "types": [] },
    "presence": {
        "senders": [
            "YOUR_USER_ID"
        ],
        "types": [
            "m.presence"
        ]
    },
    "room": {
        "ephemeral": {
            "types": [
                "m.typing",
                "m.receipt"
            ]
        },
        "include_leave": false,
        "account_data": { "types": [] },
        "state": { "types": [] },
        "timeline": { "types": [] }
    }
}

homeserver logs:

Jul 09 18:23:52 h2790514.stratoserver.net mautrix-whatsapp[22860]: [Jul  9, 2019 18:23:52] [Puppet/491757895950@s.whatsapp.net/WARN] Sync error: msg=Failed to GET JSON to /_matrix/client/r0/sync code=500 wrapped=M_UNKNOWN: Internal server error                                                                                                                                           
Jul 09 18:23:52 h2790514.stratoserver.net /opt/venvs/matrix-synapse/lib/python3.7/site-packages/synapse/app/homeserver.py[424]: synapse.http.server: [GET-204505] Failed handle request via 'SyncRestServlet': <XForwardedForRequest at 0x7f03580fe470 method='GET' uri='/_matrix/client/r0/sync?filter=4&set_presence=offline&since=s300907_14266824_10205_320779_11588_48_254_28635_1&timeout
=30000' clientproto='HTTP/1.1' site=8008>                                                                                                                                                                                                                                                                                                                                                      
Traceback (most recent call last):                                                                                                                                                                                                                             
  File "/opt/venvs/matrix-synapse/lib/python3.7/site-packages/twisted/internet/defer.py", line 1418, in _inlineCallbacks                                                                                                                                       
    result = g.send(result)                                                                                                                                                                                                                                    
  File "/opt/venvs/matrix-synapse/lib/python3.7/site-packages/synapse/storage/_base.py", line 542, in runWithConnection                                                                                                                                        
    defer.returnValue(result)                                                                                                                                                                                                                                  
  File "/opt/venvs/matrix-synapse/lib/python3.7/site-packages/twisted/internet/defer.py", line 1362, in returnValue                                                                                                                                            
    raise _DefGen_Return(val)                                                                                                                                                                                                                                  
twisted.internet.defer._DefGen_Return: [_EventDictReturn(event_id='$1561535751197oQWeK:uraziel.de', topological_ordering=None, stream_ordering=325956), _EventDictReturn(event_id='$1561535742190JVgnP:uraziel.de', topological_ordering=None, stream_ordering=
325946), _EventDictReturn(event_id='$1561535742189deCHS:uraziel.de', topological_ordering=None, stream_ordering=325945), _EventDictReturn(event_id='$1561535741188Ghdag:uraziel.de', topological_ordering=None, stream_ordering=325944), _EventDictReturn(event_id='$1561535741187PoNuN:uraziel.de', topological_ordering=None, stream_ordering=325943), _EventDictReturn(event_id='$1561535741
186fCqNl:uraziel.de', topological_ordering=None, stream_ordering=325942), _EventDictReturn(event_id='$1561535737184Glhed:uraziel.de', topological_ordering=None, stream_ordering=325940), _EventDictReturn(event_id='$1561395019102HLLju:uraziel.de', topological_ordering=None, stream_ordering=324380), _EventDictReturn(event_id='$1561395019101HbbMb:uraziel.de', topological_ordering=None
, stream_ordering=324379), _EventDictReturn(event_id='$156138018987BXoqd:uraziel.de', topological_ordering=None, stream_ordering=324067), _EventDictReturn(event_id='$156138018986YmMJm:uraziel.de', topological_ordering=None, stream_ordering=324066), _EventDictReturn(event_id='$156138016685szLjf:uraziel.de', topological_ordering=None, stream_ordering=324065), _EventDictReturn(event_
id='$156136169666IPtOr:uraziel.de', topological_ordering=None, stream_ordering=323619), _EventDictReturn(event_id='$156136158765BBJbg:uraziel.de', topological_ordering=None, stream_ordering=323618), _EventDictReturn(event_id='$156136153064hZASD:uraziel.de', topological_ordering=None, stream_ordering=323617), _EventDictReturn(event_id='$156132773156RmtFA:uraziel.de', topological_or
dering=None, stream_ordering=323428), _EventDictReturn(event_id='$15611524502756grOPJ:uraziel.de', topological_ordering=None, stream_ordering=321448), _EventDictReturn(event_id='$15611453662729rdBpt:uraziel.de', topological_ordering=None, stream_ordering=321337), _EventDictReturn(event_id='$15611314042708jKJum:uraziel.de', topological_ordering=None, stream_ordering=321183), _Event
DictReturn(event_id='$15611126612674gQoba:uraziel.de', topological_ordering=None, stream_ordering=320844), _EventDictReturn(event_id='$15611039982658sMXTk:uraziel.de', topological_ordering=None, stream_ordering=320725)]                                                                                                                                                                    

During handling of the above exception, another exception occurred:

Traceback (most recent call last):                                                                                                                                                                                                                             
  File "/opt/venvs/matrix-synapse/lib/python3.7/site-packages/twisted/internet/defer.py", line 1418, in _inlineCallbacks                                                                                                                                       
    result = g.send(result)                                                                                                                                                                                                                                    
  File "/opt/venvs/matrix-synapse/lib/python3.7/site-packages/synapse/storage/_base.py", line 502, in runInteraction                                                                                                                                           
    defer.returnValue(result)                                                                                                                                                                                                                                  
  File "/opt/venvs/matrix-synapse/lib/python3.7/site-packages/twisted/internet/defer.py", line 1362, in returnValue                                                                                                                                            
    raise _DefGen_Return(val)                                                                                                                                                                                                                                  
twisted.internet.defer._DefGen_Return: [_EventDictReturn(event_id='$1561535751197oQWeK:uraziel.de', topological_ordering=None, stream_ordering=325956), _EventDictReturn(event_id='$1561535742190JVgnP:uraziel.de', topological_ordering=None, stream_ordering=
325946), _EventDictReturn(event_id='$1561535742189deCHS:uraziel.de', topological_ordering=None, stream_ordering=325945), _EventDictReturn(event_id='$1561535741188Ghdag:uraziel.de', topological_ordering=None, stream_ordering=325944), _EventDictReturn(event_id='$1561535741187PoNuN:uraziel.de', topological_ordering=None, stream_ordering=325943), _EventDictReturn(event_id='$1561535741186fCqNl:uraziel.de', topological_ordering=None, stream_ordering=325942), _EventDictReturn(event_id='$1561535737184Glhed:uraziel.de', topological_ordering=None, stream_ordering=325940), _EventDictReturn(event_id='$1561395019102HLLju:uraziel.de', topological_ordering=None, stream_ordering=324380), _EventDictReturn(event_id='$1561395019101HbbMb:uraziel.de', topological_ordering=Non$, stream_ordering=324379), _EventDictReturn(event_id='$156138018987BXoqd:uraziel.de', topological_ordering=None, stream_ordering=324067), _EventDictReturn(event_id='$156138018986YmMJm:uraziel.de', topological_ordering=None, stream_ordering=324066), _EventDictReturn(event_id='$156138016685szLjf:uraziel.de', topological_ordering=None, stream_ordering=324065), _EventDictReturn(event$id='$156136169666IPtOr:uraziel.de', topological_ordering=None, stream_ordering=323619), _EventDictReturn(event_id='$156136158765BBJbg:uraziel.de', topological_ordering=None, stream_ordering=323618), _EventDictReturn(event_id='$156136153064hZASD:uraziel.de', topological_ordering=None, stream_ordering=323617), _EventDictReturn(event_id='$156132773156RmtFA:uraziel.de', topological_o$dering=None, stream_ordering=323428), _EventDictReturn(event_id='$15611524502756grOPJ:uraziel.de', topological_ordering=None, stream_ordering=321448), _EventDictReturn(event_id='$15611453662729rdBpt:uraziel.de', topological_ordering=None, stream_ordering=321337), _EventDictReturn(event_id='$15611314042708jKJum:uraziel.de', topological_ordering=None, stream_ordering=321183), _Even$DictReturn(event_id='$15611126612674gQoba:uraziel.de', topological_ordering=None, stream_ordering=320844), _EventDictReturn(event_id='$15611039982658sMXTk:uraziel.de', topological_ordering=None, stream_ordering=320725)]                                                                                                                                                                   

During handling of the above exception, another exception occurred:                                                                                                                                                                                           

Traceback (most recent call last):
  File "/opt/venvs/matrix-synapse/lib/python3.7/site-packages/twisted/internet/defer.py", line 1418, in _inlineCallbacks                                                                                                                                      
    result = g.send(result)
  File "/opt/venvs/matrix-synapse/lib/python3.7/site-packages/synapse/storage/stream.py", line 410, in get_room_events_stream_for_room                                                                                                                        
    defer.returnValue((ret, key))
  File "/opt/venvs/matrix-synapse/lib/python3.7/site-packages/twisted/internet/defer.py", line 1362, in returnValue                                                                                                                                           
    raise _DefGen_Return(val)
twisted.internet.defer._DefGen_Return: ([<FrozenEvent event_id='$15611039982658sMXTk:uraziel.de', type='m.room.message', state_key='None'>, <FrozenEvent event_id='$15611126612674gQoba:uraziel.de', type='m.room.message', state_key='None'>, <FrozenEvent ev$nt_id='$15611314042708jKJum:uraziel.de', type='m.room.message', state_key='None'>, <FrozenEvent event_id='$15611453662729rdBpt:uraziel.de', type='m.room.member', state_key='@whatsapp_4917678171128:uraziel.de'>, <FrozenEvent event_id='$15611524502756grOPJ:uraziel.de', type='m.room.member', state_key='@whatsapp_4917678171128:uraziel.de'>, <FrozenEvent event_id='$156132773156RmtFA:u$aziel.de', type='m.room.member', state_key='@whatsapp_491638804698:uraziel.de'>, <FrozenEvent event_id='$156136153064hZASD:uraziel.de', type='m.room.message', state_key='None'>, <FrozenEvent event_id='$156136158765BBJbg:uraziel.de', type='m.room.message', state_key='None'>, <FrozenEvent event_id='$156136169666IPtOr:uraziel.de', type='m.room.message', state_key='None'>, <FrozenEve$t event_id='$156138016685szLjf:uraziel.de', type='m.room.message', state_key='None'>, <FrozenEvent event_id='$156138018986YmMJm:uraziel.de', type='m.room.message', state_key='None'>, <FrozenEvent event_id='$156138018987BXoqd:uraziel.de', type='m.room.message', state_key='None'>, <FrozenEvent event_id='$1561395019101HbbMb:uraziel.de', type='m.room.message', state_key='None'>, <Fro$enEvent event_id='$1561395019102HLLju:uraziel.de', type='m.room.message', state_key='None'>, <FrozenEvent event_id='$1561535737184Glhed:uraziel.de', type='m.room.member', state_key='@whatsapp_491771450233:uraziel.de'>, <FrozenEvent event_id='$1561535741186fCqNl:uraziel.de', type='m.room.message', state_key='None'>, <FrozenEvent event_id='$1561535741187PoNuN:uraziel.de', type='m.r$om.message', state_key='None'>, <FrozenEvent event_id='$1561535741188Ghdag:uraziel.de', type='m.room.message', state_key='None'>, <FrozenEvent event_id='$1561535742189deCHS:uraziel.de', type='m.room.message', state_key='None'>, <FrozenEvent event_id='$1561535742190JVgnP:uraziel.de', type='m.room.message', state_key='None'>, <FrozenEvent event_id='$1561535751197oQWeK:uraziel.de', $ype='m.room.member', state_key='@whatsapp_4915128258429:uraziel.de'>], 's320725')

During handling of the above exception, another exception occurred:                                                                                                                                                                                           

Traceback (most recent call last):
  File "/opt/venvs/matrix-synapse/lib/python3.7/site-packages/twisted/internet/defer.py", line 1418, in _inlineCallbacks                                                                                                                                      
    result = g.send(result)
  File "/opt/venvs/matrix-synapse/lib/python3.7/site-packages/synapse/handlers/sync.py", line 497, in _load_filtered_recents                                                                                                                                  
    limited=limited or newly_joined_room,
  File "/opt/venvs/matrix-synapse/lib/python3.7/site-packages/twisted/internet/defer.py", line 1362, in returnValue                                                                                                                                           
    raise _DefGen_Return(val)
twisted.internet.defer._DefGen_Return: TimelineBatch(prev_batch=StreamToken(room_key='s345127', presence_key=16128690, typing_key=1401, receipt_key=379462, account_data_key=16724, push_rules_key=54, to_device_key=312, device_list_key=43253, groups_key=2)$ events=[], limited=True)

During handling of the above exception, another exception occurred:                                                                                                                                                                                           

Traceback (most recent call last):
  File "/opt/venvs/matrix-synapse/lib/python3.7/site-packages/synapse/http/server.py", line 76, in wrapped_request_handler                                                                                                                                    
    await h(self, request)
  File "/opt/venvs/matrix-synapse/lib/python3.7/site-packages/synapse/http/server.py", line 301, in _async_render                                                                                                                                             
    callback_return = await callback_return
IndexError: list index out of range

Version information

• Homeserver: uraziel.de

• Version: matrix-synapse-py3 1.1.0+sid1rebuild1

• Install method: apt

• Platform: debian 10

[richvdh]

I think this is a duplicate of #5407, though the stacktrace is opaque.