push support

[enkore]

Clients poll. A lot. Tornado can do WS oob. Already requires Redis (as a sync backend).

• Dual-homing again, with FCM support as well?
  • In this case it would make sense to spin the FCM backend used in q-drop off in a reusable fashion. Possibly as a separate service?
  • We already saw in q-drop that the FCM HTTP API is slow as buggers. Using the XMPP API (our end is a XMPP client to FCM) may help
  • Along these lines: using custom FCM wrapper based on async HTTP client tech may mitigate the large latency. If XMPP doesn't have that in the first place, then an async XMPP client would likely work even better.
• What needs to be pushed - everything, or is just the DM enough to have the client figure out what has to be pulled?

[enkore]

• Only websocket support
• /api/v0/websocket/<prefix>
  • Watch whole prefix, *do not include any URL containing "blocks"`
  • Requires client to own the prefix
  • Updated contents are sent in-line
• /api/v0/websocket/<prefix>/<file>
  • Watch single file
  • No authorization required (to support shares)
  • URLs containing "blocks" are not eligible (=HTTP 405 Method Not Allowed)
  • Updated contents are sent in-line

Both use the same protocol. It's probably a good idea to use a WS subprotocol from the start, eg. qabel-block v0.ws.block.qabel.de. This works by the client saying what subprotocols it supports, and the server choosing one.

The protocol needs to support (semantically, not as-is on-wire-format, that wouldn't work):

• POST <path> <contents> (create/update operation; create only for prefixes)
• DELETE <path>

Since <contents> will be binary, JSON is a bit wasteful but may be good enough for a first version. Otherwise any binary framing protocol would work great, eg. msgpack, protobufs or RFC-standardized CBOR.

[enkore]

For now we'll forego sending the contents in-line.

Protocol is JSON:

{
    "operation": STR in ("POST", "DELETE"),
    "prefix": STR,
    "path": STR,
    ["etag": STR -- optional, new etag when operation is POST],
}

One JSON object per WebSocket message. Messages are sent via text frames.

WebSocket subprotocol name will be v0.ws.block.qabel.de (see https://tools.ietf.org/html/rfc6455#section-11.5 for details on that).

This permits us to change the protocol in the future, including backwards incompatible changes and changes of the framing protocol, since WS subprotocol negotiation takes place before any data is exchanged between parties.

The server will ignore messages sent from the client to the server via the v0.ws.block.qabel.de protocol.

[enkore]

Here's a sample JSON:

{
    "operation": "POST",
    "etag": "1476865590312410415",
    "prefix": "123451234",
    "path": "123451234/testfile"
}

{
    "operation": "DELETE",
    "prefix": "123451234",
    "path": "123451234/testfile"
}

Implementation notes:

• There are different message types in WS. As noted above, these come in via text frames.
• Don't forget to set the subprotocol header (Sec-WebSocket-Protocol)

  • The block server prints a warning if it's wrong, see the docker console (docker attach ...):

    Subprotocol negotiation will fail: Our protocol 'v0.ws.block.qabel.de' was not proposed by client: ['']
    

  • Check that the server returns the correct protocol. This is the Sec-WebSocket-Protocol header in the HTTP/101 Switching Protocols response. Abort if that's not the one you wanna talk.

• A correct Origin header is a hard requirement. E.g. block server at http://127.0.0.1:9696 => Origin: http://127.0.0.1:9696 not http://127.0.0.1. The protocol (HTTP vs HTTPS) must match as well. This is because JS speaks WS, so everyone has to make some effort for JS, even when we don't use it.