cozy-apps-registry

What’s Cozy?

cozy


Cozy is a platform that brings all your web services in the same private space. With it, your webapps and your devices can share data easily, providing you with a new experience. You can install Cozy on your own hardware where no one’s tracking you.

Table of contents

What about this repository?

The cozy-apps-registry is a go project that implements the registry API described to work with the cozy-stack.

To work properly, it requires: - Couchdb >= 2.0 - Openstack Object Storage (Swift)

How to develop with a cozy-apps-registry working in local environment

Before starting, you will need to have a couchdb running already. That can be the one used by the local cozy-stack if you have one. For this tutorial, couchdb will be running on the default port 5984.

You also must have an OpenStack Object Storage (Swift) up and running. You can follow install instructions on the official website

1) Install and configure the local cozy-apps-registry

Since this is a golang project, you can install it using go with the followed command:

go get -u github.com/cozy/cozy-apps-registry
cd $GOPATH/src/github.com/cozy/cozy-apps-registry/

Then you will need a session secret key to run the registry (this key will be store in the sessionsecret.key file of your working directory) using a password:

cozy-apps-registry gen-session-secret --passphrase sessionsecret.key

This password is important, you will need it to run the cozy-apps-registry again later. If you lose it, you will have to start again these steps.

2) Configure the registry with cozy-registry.yml

Before running the registry, you have to configure it correctly. Don’t worry, here is the .yaml file to copy and paste in your cozy-registry.yml (should be in the root folder of cozy-registry/):

:bulb: You can find an example of this configuration file at the root of the directory.

# server host (serve command) - flag --host
host: "localhost"
# server port (serve command) - flag --port
port: 8081

couchdb:
  # CouchDB server url - flag --couchdb-url
  url: http://localhost:5984
  # CouchDB user - flag --couchdb-user
  user: ''
  # CouchDB password - flag --couchdb-password
  password: ''
  # CouchDB prefix for the registries databases - flag --couchdb-prefix
  prefix: registry1

swift:
  # Swift auth URL (provided by keystone)
  auth_url: http://172.28.128.3/identity/v3
  # Swift username
  username: admin
  # Swift password
  api_key: secret
  # Endpoint type (public/admin/internal)
  endpointy_type: public
  # Project name
  tenant: demo
  # Swift domain
  domain: default

# Path to the session secret file containing the master secret to generate
# session token.
#
# Should be generated with the "gen-session-secret" command.
session-secret: sessionsecret.key

Feel free to change it if some configurations change in your case (the couchdb user, the server parameters or the databases prefix for example).

Notices : - Here the generated session token (step 1) is stored in the sessionsecret.key file of the working directory, this is so the value of the property session-secret at the end of the configuration file. - By default, the local couchdb allow all admin accesses without creating users, so there is no user and password here. But if an admin user has been created, you have to use the properties user and password in the couchdb part to provide these informations.

It’s also possible to use env variables for configuration. You can take the key from the configuration file and add the COZY_REGISTRY prefix. For example, you can run:

COZY_REGISTRY_PORT=8081 cozy-apps-registry serve

There is also the REGISTRY_SESSION_PASS env variable for the password for the session secret.

3) Run the registry to serve the apps

Run the registry using this followed command (here our configuration file is a cozy-registry.yml in the current working directory):

cozy-apps-registry serve -c cozy-registry.yml

At this step, you should have a cozy-apps-registry running and ready for the development.

If you runnig the registry for the first time you can see the next steps to create a new editor and to configure your local cozy-stack to run with this new registry.

The -c options is always mandatory if you want to specify a config file like here.

4) Create an editor

This step is required to submit new applications in the registry. In another terminal, run these commands to create an editor (the one that will submit the application(s)) and generate an access token:

cozy-apps-registry add-editor Cozy -c cozy-registry.yml
cozy-apps-registry gen-token --master Cozy -c cozy-registry.yml

Here we go, you have now an registry ready and running on the 8081 port.

The -c options is always mandatory if you want to specify a config file like here.

5) Configure cozy-stack with the registry

On the cozy-stack, you have to add the new registry in the stack configuration file cozy.yaml (registries property):

...
registries:
  default:
    - http://localhost:8081/
...

Now restart your local cozy-stack to take this new configuration in consideration (stop and run again cozy-stack serve) and you’re ready to work with the cozy-apps-registry!

Publish your application on the registry

If you need more details about the registry you can go to the official registry documentation

:warning: Important: In this whole documentation, by the term application, we mean a web application or a konnector. Indeed, for the registry, all entities are applications and they can be either webapp or konnector type.

1) Prepare your application

For an application

To be publishable, your application requires some informations in its manifest.webapp, the manifest of a Cozy application. Here is an example for the application Drive:

wzxhzdk:7
For a connector

To be publishable, your konnector requires some informations in its manifest.konnector, the manifest of a Cozy konnector. Here is an example for the konnector cozy-konnector-trainline:

wzxhzdk:8
Properties meaning (reference)

Here are all properties meaning for the manifest file (for webapp and konnectors) sorted alphabetically:

Field Description
aggregator Object containing aggregator data. Typically { accountId: 'aggregator-service' }.
categories     array of categories for your apps (see authorized categories), it will be ['others'] by default if empty
data_types (konnector specific) Array of the data type the konnector will manage
developer name and url for the developer
editor the editor’s name to display on the cozy-bar (REQUIRED)
fields (konnector specific) JSON object describing the fields need by the konnector (except folder path). Used to generate a form. See collect documentation
folders (konnector specific) A list of folders required by the konnector to store files according to datatype (see the specific documentation below)
frequency (konnector specific) A human readable value between monthly, weekly, daily, indicating the interval of time between two runs of the konnector. Default : weekly.
icon path to the icon for the home (path in the build)
intents (application specific) a list of intents provided by this app (see cozy-stack intents doc for more details)
langs Languages available in your app (can be different from locales)
language (konnector specific) the konnector development language used (ex: node)
license the SPDX license identifier
locales an object with language slug as property, each name property is an object of localized informations (see the second part below)
manifest_version The current manifest version used. This is a versioning for the manifest and allow better retrocompatiblity when processing app manifest
messages (konnector specific) Array of message identifiers, which can be used by application to display information at known areas. See example in collect documentation.
name the name to display on the home (REQUIRED)
name_prefix the prefix to display with the name
oauth (konnector specific) JSON object containing oAuth information, like scope. If a manifest provides an oauth property, it is considered as an OAuth konnector.
parameters (konnector specific) Additional parameters which should be passed to the konnector. Used for example for bank konnectors to pass a bankId parameter.
partnership an object to provide informations (to display in the Store for example) about a partnership related to the application (icon description, name and domain)
permissions a map of permissions needed by the app (see see cozy-stack permissions doc for more details)
platforms     (application specific) List of objects for platform native applications. For now there are only two properties: type (i.e. 'ios' or 'linux') and the optional url to reach this application page.
routes (application specific) a map of routes for the app (see cozy-stack routes doc for more details) (REQUIRED)
screenshots an array of paths to the screenshots of the application (paths in the build)
services (application specific) a map of the services associated with the app (see cozy-stack services doc for more details)
slug the default slug that should never change (alpha-numeric lowercase) (REQUIRED)
source where the files of the app can be downloaded (by default it will look for the branch build)
tags a list a tags describing your application and features (useful for indexing and search)
terms an object defining properties for terms that need to be displayed/accepted by the user when installing the application (more-info-below)
time_interval (konnector specific) By defaults, konnector triggers are scheduled randomly between 00:00 AM and 05:00 AM. Those two values can be overwritten thanks to this property, by passing an array containing two values: first is the interval start hour, second is the interval end hour. Example: [15, 21] will randomly schedule the konnector trigger between 15:00 (03:00 PM) and 21:00 (09:00 PM).
type type of application (konnector or webapp) (REQUIRED)
version the current version number (REQUIRED)
vendor_link (konnector specific) URL to editor or service website

Notices: - All images paths (icon, partnership.icon and screenshots) should be relative to the build directory. For example, here, the icon.svg is stored in the build root directory and all screenshots are store in a folder screenshots in the build directory. Therefore, if you use a bundler (like webpack) be sure to know exactly where the bundler will store these assets in the build directory (and change it in the manifest if needed). - All properties in locales objects will override the matched property of the main manifest.webapp body, if a property is not found in locales it will fallback to the main body one. - We use to have the en locale as default one if the one wanted by the user doesn’t exist. Be sure to have, at least, that locale complete with the name and all descriptions. - In your build files, this manifest.webapp file must be at the root.

Translated manifest fields

Here are the properties that you can override using locales (we recommand to automatically build these properties according to your locales files if you’re using a translating tool like transifex) :

Application terms

You can provide a related terms property if you want to display and make the user accept some terms (ToS for example) just before installing the application. Here are all properties allowed and used:

Konnectors folders handling

The folders property is a list of objects with these following properties:

2) Add a new application in the registry

Our official apps registry

Official registry URL: https://apps-registry.cozycloud.cc

In order to use our official repository, you need a token for a specific editor. To do so, contact us directly at the address contact@cozycloud.cc with a mail using the following title prefix: [registry] and provide us these folowing information (not changeable after): - slug of your application - editor name that you want

We will provide you with the correct token.

Custom registry

See the details below to know how to add a new application in a custom cozy-apps-registry instance (local or not).

Prequisites: - For this step, you will need your editor token access generated when you created your editor (see below). You have to replace all {{EDITOR_TOKEN}} in this documentation by this token. - The communication with the registry is done through HTTP requests for now. So we will use the curl command line tool to register our application here.

Now you can add your application in the registry. This step is splitted in two septs: - Add a new application (without versions) - Add a version of the application

:pushpin: A new application without registered version won’t be displayed by the cozy-store application.

To add a new application, you have to do a POST request which all the informations needed to the route registryAddress/registry with registryAddress your registry address (for example here http://localhost:8081).

Let’s add the Collect application as example:

# {{EDITOR_TOKEN}} -> your generated editor access token
curl -X "POST" "http://localhost:8081/registry" \
     -H "Authorization: Token {{EDITOR_TOKEN}}" \
     -H "Content-Type: application/json" \
     -d $'{
  "slug": "collect",
  "type": "webapp",
  "editor": "{{EDITOR}}"
}'
Field Description
slug your application unique ID
type kind of application (it can be only webapp or konnector)
editor       Name of the editor matching the {{EDITOR_TOKEN}}

:warning: Here the slug is the unique ID of the application in the registry, so it can’t be changed after the application is already registered.

3) Add a new version of a registered application

Via cozy-app-publish (highly recommanded)

Here we will show the classical way to add a version using the manual mode of cozy-app-publish as reference. But you may need to look at the Automation CI part of this documentation instead.

Prequisites: - For this step, you will need your editor token access generated when you created your editor (see below). You have to replace all {{EDITOR_TOKEN}} in this documentation by this token. - Don’t forget to build your application first (run yarn build), cozy-app-publish will read the manifest from your build

Firstly, install the cozy-app-publish package using yarn or npm:

yarn add cozy-app-publish --dev

You can also install the package as a global package, but don’t forget to update it frequently

Then use it to publish your application, here is an example for collect:

yarn cozy-app-publish \
--token {{EDITOR_TOKEN}} \
--build-url https://github.com/cozy/cozy-collect/archive/042cef26d9d33ea604fe4364eaab569980b500c9.tar.gz \
--manual-version 1.0.2-dev.042cef26d9d33ea604fe4364eaab569980b500c9

If you need more information about this tool, you can go to the official cozy-app-publish documentation.

Via curl

Here we will show the classical way to add a version using curl as reference. But you may need to look at our dedicated tool cozy-app-publish or the Automation CI part of this documentation instead.

Prequisites: - For this step, you will need your editor token access generated when you created your editor (see below). You have to replace all {{EDITOR_TOKEN}} in this documentation by this token. - The communication with the registry is done through HTTP requests for now. So we will use the curl command line tool to register our application here.

To add a new application, you have to do a POST request which all the informations needed to the route registryAddress/registry/:appSlug with registryAddress your registry address (for example here http://localhost:8081), :appName your application slug (here drive).

Let’s add the version 1.0.1 of the Collect application as example:

# {{EDITOR_TOKEN}} -> your generated editor access token
curl -X "POST" "http://localhost:8081/registry/collect" \
     -H "Authorization: Token {{EDITOR_TOKEN}}" \
     -H "Content-Type: application/json" \
     -d $'{
  "url": "https://github.com/cozy/cozy-collect/archive/1.0.1.tar.gz",
  "sha256": "96212bf53ab618808da0a92c7b6d9f2867b1f9487ba7c1c29606826b107041b5",
  "version": "1.0.1",
  "type": "webapp",
  "editor": "{{EDITOR}}"
}'
Field Description
url the archive source of your application, it will be downloaded and checked with the sha256 property
sha256 the sha256 hash of your source archive matching the archive in url (see the notice below)
version version of the application, must match the one in the manifest (see the notice below)
type kind of application (it can be only webapp or konnector)
editor       Name of the editor matching the {{EDITOR_TOKEN}}

:warning: Important notices: - The version must match the one in the manifest.webapp file for stable release. For beta (X.X.X-betaX) or dev releases (X.X.X-dev.hash256), the version before the cyphen must match the one in the manifest.webapp. - For better integrity, the sha256 provided must match the sha256 of the archive provided in url. If it’s not the case, that will be considered as an error and the version won’t be registered.

Automation (CI)

The following tutorial explains how to connect your continuous integration based on Travis to automatically publish new versions on the apps registry.

In this tutorial, we assume:

You first need to add the token to your travis configuration file .travis.yml. To do so, you need the travis utility to encrypt its value.

$ travis encrypt REGISTRY_TOKEN={{EDITOR_TOKEN}} -r myname/cozy-example --org
Please add the following to your .travis.yml file:

  secure: "jUAjk..LOOOOONG_ENCRYPTED_STRING.....jdk89="

Like said, you need to add this block of ciphered data in the .travis.yml (if it’s not already done automatically). This will allow you to use the REGISTRY_TOKEN variable in your deployment script.

Then, you can add the publish script in your package.json in order to be used by Travis:

...
"publish:cozy": "git fetch origin ${DEPLOY_BRANCH:-build}:${DEPLOY_BRANCH:-build} && cozy-app-publish --token $REGISTRY_TOKEN --build-commit $(git rev-parse ${DEPLOY_BRANCH:-build})"
...

This script will fetch your last commit from the build branch to publish to the registry. If you push a tag, be sure to wait the last build branch Travis build finished in order to have the real last commit to publish.

Finally, you can add this script to your .travis.yml to publish your app using our publishing tool cozy-app-publish during the deploy process:

...
before_deploy:
- yarn add cozy-app-publish
deploy:
  - provider: script
    repo: myname/cozy-example
    skip-cleanup: true
    script: export DEPLOY_BRANCH=build && yarn deploy && yarn publish:cozy
    on:
      branch: master
  - provider: script
    repo: myname/cozy-example
    skip-cleanup: true
    script: export DEPLOY_BRANCH=build && yarn publish:cozy
    on:
      tags: true
...

Important notices: - A commit push to the branch master will publish your application in the dev channel of the registry. - A tag push (Github release) will publish a stable version (ex: 1.0.0) or a beta version (ex: 1.0.1-beta2) to the registry (automatically handled by the registry). - cozy-app-publish will use the github archive URL computing to get the application tarball. If your applicaiton is not on Github, you may need to use the manual mode of the command.

Access to our official apps registry

Official registry URL: https://apps-registry.cozycloud.cc

In order to use our official repository, you need a token for a specific editor. To do so, contact us directly at the address contact@cozycloud.cc with a mail using the following title prefix: [registry] and provide us these folowing information (not changeable after): - slug of your application - editor name that you want

We will provide you with the correct token.

Access control and tokens

The read-only routes of the registry are all public and do not require any access-contre. However routes allowing to create applications and versions have access-control policies associated to them.

The registry has two types of access permissions, that are associated to two different tokens:

Editor tokens can be specific to one or more applications names that they are allowed to publish.

In order to create tokens, the binary offers a gen-token command-line. Here are some examples to illustrates some usages:

# Generating tokens
  # generate an editor token for the editor "cozy", for any application
  $ cozy-apps-registry gen-token cozy
  # generate an editor token for the editor "cozy" expiring after 30 days
  $ cozy-apps-registry gen-token cozy --max-age 30d
  # generate an editor token for the editor "cozy" for application "collect" and "drive"
  $ cozy-apps-registry gen-token cozy --apps collect,drive
  # generate a master token associated with the editor "cozy" expiring after 30 days
  $ cozy-apps-registry gen-token cozy --master --max-age 30d

# Verifying tokens
  # verify the editor token "XXX" for the editor "cozy"
  $ cozy-apps-registry verify-token cozy "XXX"
  # verify the master token "XXX" associated with the editor "cozy"
  $ cozy-apps-registry verify-token cozy "XXX" --master

# Revoking tokens
  # revoke all editors tokens for the cozy editor
  $ cozy-apps-registry revoke-tokens cozy
  # revoke all master tokens associated with the cozy editor
  $ cozy-apps-registry revoke-tokens cozy --master

Maintenance

In order to set/unset an application into maintenance mode, the binary offers a maintenance command-line. Here are some examples of how to use it:

# Activate maintenance mode for the application 'bank' of space 'myspace'
# Available flagged options:
#  --infra            specify a maintenance specific to our infra
#  --no-manual-exec   specify a maintenance disallowing manual execution
#  --short            specify a short maintenance
$ cozy-apps-registry maintenance activate bank --space myspace

# Deactivate maintenance mode for the application 'bank' of space 'myspace'
$ cozy-apps-registry maintenance deactivate bank --space myspace

Or using a cURL request and a master token:

curl -XPUT \
  -H"Authorization: Token $COZY_REGISTRY_ADMIN_TOKEN" \
  -H"Content-Type: application/json" \
  -d'{"flag_infra_maintenance": false,"flag_short_maintenance": false,"flag_disallow_manual_exec": false,"messages": {"fr": {"long_message": "Bla bla bla","short_message": "Bla"},"en": {"long_message": "Yadi yadi yada","short_message": "Yada"}}}' \
  https://apps-registry.cozycloud.cc/myspace/registry/maintenance/bank/activate

curl -XPUT \
  -H"Authorization: Token $COZY_REGISTRY_ADMIN_TOKEN" \
  https://apps-registry.cozycloud.cc/registry/maintenance/bank/deactivate

Application confidence grade / labelling

The confidence grade of an applications can be specified by specifying the data_usage_commitment and data_usage_commitment_by fields of the application document.

Possible values for these properties are:

To do that, a command line and admin API are available, and be used as follow:

$ cozy-apps-registry modify-app banks --space my_space --data-usage-commitment user_reserved --data-usage-commitment-by editor

Or using a cURL request and a master token:

curl -XPATCH \
  -H"Authorization: Token $COZY_REGISTRY_ADMIN_TOKEN" \
  -H"Content-Type: application/json" \
  -d'{"data_usage_commitment": "user_reserved", "data_usage_commitment_by": "editor"}
  https://apps-registry.cozycloud.cc/my_space/registry/banks

Community

You can reach the Cozy Community by: