API

Modules

addData

This function saves the data into the cozy blindly without check You need at least the POST permission for the given doctype in your manifest, to be able to use this function.

Parameters:

  • documents: an array of objects corresponding to the data you want to save in the cozy
  • doctype (string): the doctype where you want to save data (ex: 'io.cozy.bills')
const documents = [
  {
    name: 'toto',
    height: 1.8
  },
  {
    name: 'titi',
    height: 1.7
  }
]

return addData(documents, 'io.cozy.height')
cozyClient

This is a cozy-client-js instance already initialized and ready to use

If you want to access cozy-client-js directly, this method gives you directly an instance of it, initialized according to COZY_URL and COZY_CREDENTIALS environment variable given by cozy-stack You can refer to the cozy-client-js documentation for more information.

Example :

const {cozyClient} = require('cozy-konnector-libs')

cozyClient.data.defineIndex('my.doctype', ['_id'])
filterData

This function filters the passed array from data already present in the cozy so that there is not duplicated data in the cozy. You need at least the GET permission for the given doctype in your manifest, to be able to use this function.

Parameters:

  • documents: an array of objects corresponding to the data you want to save in the cozy
  • doctype (string): the doctype where you want to save data (ex: 'io.cozy.bills')
  • options :
    • keys (array) : List of keys used to check that two items are the same. By default it is set to `['id']'.
    • index (optionnal) : Return value returned by cozy.data.defineIndex, the default will correspond to all documents of the selected doctype.
    • selector (optionnal object) : Mango request to get records. Default is built from the keys {selector: {_id: {"$gt": null}}} to get all the records.
const documents = [
  {
    name: 'toto',
    height: 1.8
  },
  {
    name: 'titi',
    height: 1.7
  }
]

return filterData(documents, 'io.cozy.height', {
  keys: ['name']
}).then(filteredDocuments => addData(filteredDocuments, 'io.cozy.height'))

linkBankOperations

linkBankOperations ( entries, doctype, fields, options = {} )

This function will soon move to a dedicated service. You should not use it. The goal of this function is to find links between bills and bank operations.

mkdirp

Creates a directory and its missing ancestors as needed.

Options :

  • ...pathComponents: one or many path components to be joined
await mkdirp('/foo') // Creates /foo
await mkdirp('/foo') // Does nothing as /foo already exists
await mkdirp('/bar/baz') // Creates /bar, then /bar/baz
await mkdirp('/foo/bar/baz') // Creates /foo/bar, then /foo/bar/baz, not /foo
await mkdirp('/') // Does nothing
await mkdirp('/qux', 'qux2/qux3', 'qux4') // Creates /qux, then /qux/qux2,
                                          // then /qux/qux2/qux3 and
                                          // finally /qux/qux2/qux3/qux4

The function will automatically add a leading slash when missing:

await mkdirp('foo', 'bar') // Creates /foo, then /foo/bar
normalizeFilename

Returns the given name, replacing characters that could be an issue when used in a filename with spaces.

Replaced characters include:

  • Those forbidden on one or many popular OS or filesystem: <>:"/\|?*
  • Those forbidden by the cozy-stack \0, \r and \n
  • Multiple spaces and/or tabs are replaced with a single space
  • Leading & trailing spaces and/or tabs are removed

An exception will be thrown in case there is not any filename-compatible character in the given name.

Parameters:

  • basename is whatever string you want to generate the filename from
  • ext is an optional file extension, with or without leading dot
const { normalizeFilename } = require('cozy-konnector-libs')

const filename = normalizeFilename('*foo/bar: <baz> \\"qux"\t???', '.txt')
// `filename` === `foo bar baz qux.txt`
requestFactory

This is a function which returns an instance of request-promise initialized with defaults often used in connector development.

// Showing defaults
req = requestFactory({
  cheerio: false,
  jar: true,
  json: true
})

Options :

  • cheerio: will parse automatically the response.body in a cheerio instance
req = requestFactory({ cheerio: true })
req('http://github.com', $ => {
  const repos = $('#repo_listing .repo')
})
  • jar: is passed to request options. Remembers cookies for future use.
  • json: will parse the response.body as JSON
  • json: will parse the response.body as JSON
  • resolveWithFullResponse: The full response will be return in the promise. It is compatible with cheerio and json options.
req = requestFactory({
   resolveWithFullResponse: true,
   cheerio: true
})
req('http://github.com', response => {
  console.log(response.statusCode)
  const $ = response.body
  const repos = $('#repo_listing .repo')
})

You can find the full list of available options in request-promise and request documentations.

saveBills

Combines the features of saveFiles, filterData, addData and linkBankOperations for a common case: bills. Will create io.cozy.bills objects. The default deduplication keys are ['date', 'amount', 'vendor']. You need the full permission on io.cozy.bills, full permission on io.cozy.files and also full permission on io.cozy.bank.operations in your manifest, to be able to * use this function.

Parameters:

  • documents is an array of objects with any attributes with some mandatory attributes :
    • amount (Number): the amount of the bill used to match bank operations
    • date (Date): the date of the bill also used to match bank operations
    • vendor (String): the name of the vendor associated to the bill. Ex: 'trainline' You can also pass attributes expected by saveFiles Please take a look at io.cozy.bills doctype documentation
  • fields (object) this is the first parameter given to BaseKonnector's constructor
  • options is passed directly to saveFiles, hydrateAndFilter, addData and linkBankOperations.
const { BaseKonnector, saveBills } = require('cozy-konnector-libs')

module.exports = new BaseKonnector(function fetch (fields) {
  const documents = []
  // some code which fills documents
  return saveBills(documents, fields, {
    identifiers: ['vendorj']
  })
})
saveFiles

The goal of this function is to save the given files in the given folder via the Cozy API. You need the full permission on io.cozy.files in your manifest to use this function.

  • files is an array of { fileurl, filename } :

    • fileurl: The url of the file. This attribute is mandatory or this item will be ignored
    • filename : The file name of the item written on disk. This attribute is optional and as default value, the file name will be "smartly" guessed by the function. Use this attribute if the guess is not smart enough for you.
  • folderPath (string) is relative to the main path given by the cozy-collect application to the connector. If the connector is run in standalone mode, the main path is the path of the connector.

  • options (object) is optional. Possible options :

    • timeout (timestamp) can be used if your connector needs to fetch a lot of files and if the stack does not give enough time to your connector to fetch it all. It could happen that the connector is stopped right in the middle of the download of the file and the file will be broken. With the timeout option, the saveFiles function will check if the timeout has passed right after downloading each file and then will be sure to be stopped cleanly if the timeout is not too long. And since it is really fast to check that a file has already been downloaded, on the next run of the connector, it will be able to download some more files, and so on. If you want the timeout to be in 10s, do Date.now() + 10*1000. You can try it in the previous code.
signin

The goal of this function is to provide an handy method to log the user in, on html form pages. On success, it resolves a promise with a parsed body.

Errors:

  • LOGIN_FAILED if the validate predicate is false
  • INVALID_FORM if the element matched by formSelector is not a form or has no action attribute
  • UNKNOWN_PARSING_STRATEGY if parse is not one of the accepted values: raw, cheerio, json.
  • VENDOR_DOWN if a request throws a RequestError, or StatusCodeError

It does not submit values provided through select tags, except if populated by user with formData.

  • url is the url to access the html form

  • formSelector is used by cheerio to uniquely identify the form in which to log in

  • formData is an object { name: value, … }. It is used to populate the form, in the proper inputs with the same name as the properties of this object, before submitting it. It can also be a function that returns this object. The page at url would be given as argument, right after having been parsed through cheerio.

  • parse allow the user to resolve signin with a preparsed body. The choice of the strategy for the parsing is one of : raw, json or cheerio. cheerio being the default.

  • validate is a predicate taking two arguments statusCode and parsedBody. If it is false, LOGIN_FAILED is thrown, otherwise the signin resolves with parsedBody value.

  • requestOpts allows to pass eventual options to the signin's requestFactory. It could be useful for pages using latin1 encoding for instance.

updateOrCreate

The goal of this function is create or update the given entries according to if they already exist in the cozy or not You need the full permission for the given doctype in your manifest, to be able to use this function.

Parameters:

  • entries is an array of objects with any attributes :
  • doctype (string) is the cozy doctype where the entries should be saved
  • matchingAttributes (array of strings) is the list of attributes in each entry should be used to check if an entry is already saved in the cozy

Classes

BaseKonnector

The class from which all the connectors must inherit. It takes a fetch function in parameter that must return a Promise. You need at least the GET permission on io.cozy.accounts in your manifest to allow it to fetch account information for your connector.

Document

Simple Model for Documents. Allows to specify shouldSave, shouldUpdate as methods.

Has useful isEqual method

Constants

LOGIN_FAILED : String

The konnector could not login

NOT_EXISTING_DIRECTORY : String

The folder specified as folder_to_save does not exist (checked by BaseKonnector)

VENDOR_DOWN : String

The vendor's website is down

USER_ACTION_NEEDED : String

There was an unexpected error, please take a look at the logs to know what happened

FILE_DOWNLOAD_FAILED : String

There was a problem while downloading a file

SAVE_FILE_FAILED : String

There was a problem while saving a file

DISK_QUOTA_EXCEEDED : String

Could not save a file to the cozy because of disk quota exceeded

Functions

mkSpec()

Declarative scraping.

Describe your items attributes and where to find/parse them instead of imperatively building them.

Heavily inspired by artoo scraping method.

scrape($, spec(s), [childSelector])object | array

Scrape a cheerio object for properties

addData

This function saves the data into the cozy blindly without check You need at least the POST permission for the given doctype in your manifest, to be able to use this function.

Parameters:

const documents = [
  {
    name: 'toto',
    height: 1.8
  },
  {
    name: 'titi',
    height: 1.7
  }
]

return addData(documents, 'io.cozy.height')

cozyClient

This is a cozy-client-js instance already initialized and ready to use

If you want to access cozy-client-js directly, this method gives you directly an instance of it, initialized according to COZY_URL and COZY_CREDENTIALS environment variable given by cozy-stack You can refer to the cozy-client-js documentation for more information.

Example :

const {cozyClient} = require('cozy-konnector-libs')

cozyClient.data.defineIndex('my.doctype', ['_id'])

filterData

This function filters the passed array from data already present in the cozy so that there is not duplicated data in the cozy. You need at least the GET permission for the given doctype in your manifest, to be able to use this function.

Parameters:

const documents = [
  {
    name: 'toto',
    height: 1.8
  },
  {
    name: 'titi',
    height: 1.7
  }
]

return filterData(documents, 'io.cozy.height', {
  keys: ['name']
}).then(filteredDocuments => addData(filteredDocuments, 'io.cozy.height'))

filterData~suitableCall()

Since we can use methods or basic functions for shouldSave and shouldUpdate we pass the appropriate this and arguments.

If funcOrMethod is a method, it will be called with args[0] as this and the rest as arguments Otherwise, this will be null and args will be passed as arguments.

Kind: inner method of filterData

linkBankOperations

linkBankOperations ( entries, doctype, fields, options = {} )

This function will soon move to a dedicated service. You should not use it. The goal of this function is to find links between bills and bank operations.

mkdirp

Creates a directory and its missing ancestors as needed.

Options :

await mkdirp('/foo') // Creates /foo
await mkdirp('/foo') // Does nothing as /foo already exists
await mkdirp('/bar/baz') // Creates /bar, then /bar/baz
await mkdirp('/foo/bar/baz') // Creates /foo/bar, then /foo/bar/baz, not /foo
await mkdirp('/') // Does nothing
await mkdirp('/qux', 'qux2/qux3', 'qux4') // Creates /qux, then /qux/qux2,
                                          // then /qux/qux2/qux3 and
                                          // finally /qux/qux2/qux3/qux4

The function will automatically add a leading slash when missing:

await mkdirp('foo', 'bar') // Creates /foo, then /foo/bar

normalizeFilename

Returns the given name, replacing characters that could be an issue when used in a filename with spaces.

Replaced characters include:

An exception will be thrown in case there is not any filename-compatible character in the given name.

Parameters:

const { normalizeFilename } = require('cozy-konnector-libs')

const filename = normalizeFilename('*foo/bar: <baz> \\"qux"\t???', '.txt')
// `filename` === `foo bar baz qux.txt`

requestFactory

This is a function which returns an instance of request-promise initialized with defaults often used in connector development.

// Showing defaults
req = requestFactory({
  cheerio: false,
  jar: true,
  json: true
})

Options :

req = requestFactory({ cheerio: true })
req('http://github.com', $ => {
  const repos = $('#repo_listing .repo')
})
req = requestFactory({
   resolveWithFullResponse: true,
   cheerio: true
})
req('http://github.com', response => {
  console.log(response.statusCode)
  const $ = response.body
  const repos = $('#repo_listing .repo')
})

You can find the full list of available options in request-promise and request documentations.

saveBills

Combines the features of saveFiles, filterData, addData and linkBankOperations for a common case: bills. Will create io.cozy.bills objects. The default deduplication keys are ['date', 'amount', 'vendor']. You need the full permission on io.cozy.bills, full permission on io.cozy.files and also full permission on io.cozy.bank.operations in your manifest, to be able to * use this function.

Parameters:

const { BaseKonnector, saveBills } = require('cozy-konnector-libs')

module.exports = new BaseKonnector(function fetch (fields) {
  const documents = []
  // some code which fills documents
  return saveBills(documents, fields, {
    identifiers: ['vendorj']
  })
})

saveFiles

The goal of this function is to save the given files in the given folder via the Cozy API. You need the full permission on io.cozy.files in your manifest to use this function.

signin

The goal of this function is to provide an handy method to log the user in, on html form pages. On success, it resolves a promise with a parsed body.

Errors:

It does not submit values provided through select tags, except if populated by user with formData.

updateOrCreate

The goal of this function is create or update the given entries according to if they already exist in the cozy or not You need the full permission for the given doctype in your manifest, to be able to use this function.

Parameters:

BaseKonnector

The class from which all the connectors must inherit. It takes a fetch function in parameter that must return a Promise. You need at least the GET permission on io.cozy.accounts in your manifest to allow it to fetch account information for your connector.

Kind: global class

new BaseKonnector(fetch)

Its role is twofold :

⚠️ A promise should be returned from the fetch function otherwise the konnector cannot know that asynchronous code has been called.

this.terminate('LOGIN_FAILED')
Param Type Description
fetch function Function to be run automatically after account data is fetched. This function will be binded to the current connector. If not fetch function is given. The connector will have to handle itself it’s own exection and error handling

Example

const { BaseKonnector } = require('cozy-konnector-libs')

module.exports = new BaseKonnector(function fetch () {
 // use this to access the instance of the konnector to
 // store any information that needs to be passed to
 // different stages of the konnector
 return request('http://ameli.fr')
   .then(computeReimbursements)
   .then(saveBills)
})

baseKonnector.end()

Hook called when the connector is ended

Kind: instance method of BaseKonnector

baseKonnector.fail()

Hook called when the connector fails

Kind: instance method of BaseKonnector

baseKonnector.init() ⇒ Promise

Initializes the current connector with data comming from the associated account

Kind: instance method of BaseKonnector
Returns: Promise - with the fields as an object

baseKonnector.saveAccountData(data, options) ⇒ Promise

Saves data to the account that is passed to the konnector. Use it to persist data that needs to be passed to each konnector run.

By default, the data is merged to the remote data, use options.merge = false to overwrite the data.

The data is saved under the .data attribute of the cozy account.

Kind: instance method of BaseKonnector

Param Type Description
data object Attributes to be merged
options object { merge: true

baseKonnector.getAccountData() ⇒ object

Get the data saved by saveAccountData

Kind: instance method of BaseKonnector

baseKonnector.terminate(message)

Send a special error code which is interpreted by the cozy stack to terminate the execution of the connector now

Kind: instance method of BaseKonnector

Param Type Description
message string The error code to be saved as connector result see [docs/ERROR_CODES.md]

Document

Simple Model for Documents. Allows to specify shouldSave, shouldUpdate as methods.

Has useful isEqual method

Kind: global class

document.isEqual()

Compares to another document deeply.

_id and _rev are by default ignored in the comparison.

By default, will compare dates loosely since you often compare existing documents (dates in ISO string) with documents that just have been scraped where dates are Dates.

Kind: instance method of Document

LOGIN_FAILED : String

The konnector could not login

Kind: global constant

NOT_EXISTING_DIRECTORY : String

The folder specified as folder_to_save does not exist (checked by BaseKonnector)

Kind: global constant

VENDOR_DOWN : String

The vendor’s website is down

Kind: global constant

USER_ACTION_NEEDED : String

There was an unexpected error, please take a look at the logs to know what happened

Kind: global constant

FILE_DOWNLOAD_FAILED : String

There was a problem while downloading a file

Kind: global constant

SAVE_FILE_FAILED : String

There was a problem while saving a file

Kind: global constant

DISK_QUOTA_EXCEEDED : String

Could not save a file to the cozy because of disk quota exceeded

Kind: global constant

mkSpec()

Declarative scraping.

Describe your items attributes and where to find/parse them instead of imperatively building them.

Heavily inspired by artoo scraping method.

Kind: global function

scrape($, spec(s), [childSelector]) ⇒ object | array

Scrape a cheerio object for properties

Kind: global function
Returns: object | array - - Item(s) scraped

Param Type Description
$ cheerio Cheerio node which will be scraped
spec(s) object | string Options object describing what you want to scrape
[childSelector] string If passed, scrape will return an array of items

Example
scrape can be used to declaratively extract data :

const item = scrape($('#item'), {
  title: '.title',
  content: '.content'
})
const items = scrape($('#content'), {
  title: '.title',
  content: '.content'
}, '.item')

For more power, you can use objects for each retriever :

const items = scrape($('#content'), {
  title: '.title',
  content: '.content',
  link: {
    sel: 'a',
    attr: 'href'
  },
}, '.item')

Here the href attribute of the a inside .items would have been put into the link attribute of the items returned by scrape.

Available options :

⚠ Permissions

Please note that some classes require some permissions: