This document was written in january 2017.
- Event: Something happening in the stack. Most of them will come from couchdb, some jobs and user actions might also trigger them.
- Events feed: the feed of occurring events. There is two types of feeds:
- continuous allows to follow events as they occurs
- interval allow the see the history of the feed from any given time
- Realtime: user experienced updates of the interface from change happening from another source. Ie, I have a folder opened in cozy-files on the browser, I take some pictures from my smartphone, the pictures appears in the folder without me needing to refresh the browser tab.
What couchdb offers¶
Couchdb supports with its
_changes API both events feeds types:
since=now&continuous=truewe get all events continuously as they happen (SSE)
since=(last known seq_number)we get all changes in the interval between last known
Couchdb also offers a
_db_updates route, which give us continuous changes
at the database level. This routes does not support a since parameter, as there
is no global
Other events will be generated from the stack itself, such as session close or jobs activity.
We cannot have a continuous
_changes feed open to every databases
Use cases for interval events feeds¶
Couchdb replication algorithm can work in one-shot mode, where it replicates changes since last sync up until now, or in continuous mode where it replicates changes as they happens.
- The stack will not allow continuous mode for replication.
- This is already supported with the
Our sharing is based on couchdb replication rules, so also depends on
feed to ensure all changes have been applied.
Considering these use cases, there is no need for non-couchdb event to be part of the interval events feed.
Use cases for continuous events feeds¶
Some events should be send to the client to update views as data change.
@event jobs trigger¶
Some event will trigger the activation of a job (ie. When a photo has been uploaded, generate thumbnails and extract EXIF metadatas). This should be done as soon as possible after the events
While not absolutely necessary, having cozy A notify cozy B when a shared document is changed allows for both better user experience (faster propagation) and better performance (no need to poll every X minutes, the N cozy we are sharing from).
Client realtime tech choice¶
- Polling: regularly ask the server what happened since last time.
- COMET: Leaving a normal HTTP connection open sending data and heartbeets regularly to keep it open, reading xhr.responseText at intervals without waiting for readyState == 4. Restart the connection when it breaks.
- SSE: Normalized & standardized version of COMET with half-decent browser support (86% users) but easily polyfillable (it’s just COMET). It is simpler and easier to debug. It has some limitations (no HTTP headers in JS api, counts toward the maximum number of http connection per domain).
- Websocket: keep a socket open, it allows 2 way data communication which we do not need, has better server support (92% users) but is impossible to polyfill client side, more popular, there is a better golang package
Choice = Websocket¶
While SSE appears at first glance like a better fit for our use case, its limitation and lack of browser priority makes us choose websocket. In the event older browser supports becomes necessary we can use SockJS.
optimization paths (future)¶
- bandwidth Limiting the number of events sent by allowing the client to specified it is only interested in events matching a selector (files app only care about changes in the files of the current folder view)
- number of connections Instead of 1 socket / tab, we can probably make 1 socket / browser using some hackish combination of SharedWorker / iframe.postMessage and a client-side demultiplexer.
- both No need for realtime if the user is not using the tab (for most usecases), we could cut the realtime feed depending on Page Visibility API
- We assume all couchdb changes will originate from the stack
- Events are generated at the stack level
- We do NOT rely on couchdb
We create a realtime.Event interface, which we call in other packages. We accept websocket connection and bind them to a realtime.Dispatcher object.
Small cozy version¶
It all happens in RAM, realtime.Event are immediately transmited to the dispatcher.
Big cozy version (ie. multiple stack instance)¶