

Log analysis

Debugging synchronization issues mostly means analysing logs. Logs are in JSON lines format, generated by winston, and can be quite huge (this is a known issue).

We use jq to filter them. See jq manual to get all the basic and advanced filters. A yarn jq script is also available which automatically includes a bunch of custom ones:

yarn jq 'some|filter' path/to/logs

The script doesn’t use --slurp so JSON lines are filtered one by one and memory usage is kept below 100MB. Which means you can filter multiple log files at once, even huge ones:

yarn jq 'some|filter' path/to/logs*

Start with the issues filter to find out main issues:

yarn jq issues path/to/logs*

Shorten the output with jq‘s -c option and the short filter so you get a better overview:

yarn jq -c 'issues|short' path/to/logs*

You can combine even more filters to focus on some kind of events:

yarn jq -c 'issues|no_gui|xls|short' path/to/logs*

Then you can check what happened at some file path:

yarn jq -c 'path(foo/bar)|short' path/to/logs*

Please note that since the path filter takes a regular expression pattern, you’ll have to double-escape backslashes in Windows paths:

yarn jq 'path("foo\\\\bar")' path/to/logs*

Then get more details (e.g. dropping -c option and short filter):

yarn jq 'select(...)' path/to/logs*

To get an error stack trace as you would expect, you can use jq‘s -r (raw) option on the .stack property:

yarn jq -r 'select(.time == "2018-06-21T13:57:40.699Z")|.err|.stack' path/to/logs

See the .jq file at the root of the repository to get the complete list of custom filters.