Realtime docs

Context

Definitions

What couchdb offers

Couchdb supports with its _changes API both events feeds types:

Couchdb also offers a _db_updates route, which give us continuous changes at the database level. This routes does not support a since parameter, as there is no global seq_number.

Other events will be generated from the stack itself, such as session close or jobs activity.

Performance limitation

We cannot have a continuous _changes feed open to every databases

Use cases for interval events feeds

Replication

Couchdb replication algorithm can work in one-shot mode, where it replicates changes since last sync up until now, or in continuous mode where it replicates changes as they happens.

Sharing

Our sharing is based on couchdb replication rules, so also depends on _changes feed to ensure all changes have been applied.

Considering these use cases, there is no need for non-couchdb event to be part of the interval events feed.

Use cases for continuous events feeds

Realtime

Some events should be send to the client to update views as data change.

@event jobs trigger

Some event will trigger the activation of a job (ie. When a photo has been uploaded, generate thumbnails and extract EXIF metadatas). This should be done as soon as possible after the events

Sharing ?

While not absolutely necessary, having cozy A notify cozy B when a shared document is changed allows for both better user experience (faster propagation) and better performance (no need to poll every X minutes, the N cozy we are sharing from).

Client realtime tech choice

Options

Choice = Websocket

While SSE appears at first glance like a better fit for our use case, its limitation and lack of browser priority makes us choose websocket. In the event older browser supports becomes necessary we can use SockJS.

optimization paths (future)

Go/Stack architecture

We create a realtime.Event interface, which we call in other packages. We accept websocket connection and bind them to a realtime.Dispatcher object.

Small cozy version

It all happens in RAM, realtime.Event are immediately transmited to the dispatcher.

Big cozy version (ie. multiple stack instance)

Redis pub/sub

Websocket API

We start with a normal websocket handshake.

Websockets include a protocol description in handshake, the protocol described below is hereby named io.cozy.websocket.

Changes to the websocket protocol should be given versions, support for older version should be maintained when reasonable.

GET /realtime/ HTTP/1.1
Host: mycozy.example.com
Upgrade: websocket
Connection: Upgrade
Origin: http://calendar.mycozy.example.com
Sec-WebSocket-Key: x3JrandomLkh9GBhXDw==
Sec-WebSocket-Protocol: io.cozy.websocket
Sec-WebSocket-Version: 13

Then messages are sent using json

client > {"method": "AUTH",
          "payload": "xxAppOrAuthTokenxx="}
client > {"method": "SUBSCRIBE",
          "payload": {"type": "io.cozy.files"}}
client > {"method": "SUBSCRIBE",
          "payload": {"type": "io.cozy.contacts"}}
server > {"event": "UPDATED",
          "payload": {"id": "idA", "rev": "2-705...", "type": "io.cozy.contacts", "doc": {embeded doc ...}}}
server > {"event": "DELETED",
          "payload": {"id": "idA", "rev": "3-541...", "type": "io.cozy.contacts"}}
server > {"event": "UPDATED",
          "payload": {"id": "idB", "rev": "6-457...", "type": "io.cozy.files", "doc": {embeded doc ...}}}

AUTH

It must be the first command to be sent. The client gives its token with this command, and the stack will use it to know which are the permissions of the app.

{"method": "AUTH", "payload": "eyJhbGciOiJIUzUxMiIsInR5cCI6IkpXVCJ9.eyJhdWQiOiJhcHAiLCJpYXQiOjE0OTg4MTY1OTEsImlzcyI6ImNvenkudG9vbHM6ODA4MCIsInN1YiI6Im1pbmkifQ.eH9DhoHz7rg8gR7noAiKfeo8eL3Q_PzyuskO_x3T8Hlh9q_IV-4zqoGtjTiO7luD6_VcLboEU-6o3XBek84VTg"}

SUBSCRIBE

A client can send a SUBSCRIBE request to be notified of changes. The payload is a selector for the events it wishes to receive For now the only possible selector is on type & optionaly id

{"method": "SUBSCRIBE", "payload": {"type": "[desired doctype]"}}
{"method": "SUBSCRIBE", "payload": {"type": "[desired doctype]", "id": "idA"}}

In order to subscribe, a client must have permission GET on the passed selector. Otherwise an error is passed in the message feed.

server > {"event": "error",
          "payload": {
            "status": "403 Forbidden"
            "code": "forbidden"
            "title":"The Application can't subscribe to io.cozy.files"
            "source": {"method": "SUBSCRIBE", "payload": {"type":"io.cozy.files"} }
          }}