S3 Storage Backend¶

cozy-stack supports S3-compatible object storage as a file system backend, alongside the existing local filesystem (afero) and OpenStack Swift backends. It supports S3-compatible providers such as OVH, MinIO and Scaleway.

Configuration¶

Set fs.url to the default S3 connection and configure destinations in fs.s3.buckets. Each entry contains a bucket name and an optional url. Bucket names are never generated from prefixes or organization IDs.

fs:
  url: s3://s3.example.net?access_key=ACCESS&secret_key=SECRET&region=gra
  s3:
    buckets:
      default:
        name: company-storage

This stores everything in one bucket, with separate object paths for each storage type. All organizations share the files bucket; each instance has its own path beneath files/.

URL parameter	Description	Default
`access_key`	S3 access key ID	—
`secret_key`	S3 secret access key	—
`region`	Region for requests and bucket creation	SDK discovery/default
`use_ssl`	Use HTTPS	`true`

URL-encode credentials containing reserved query characters. Specify region in every connection URL to avoid location discovery requests.

Separate or mixed buckets¶

Configure a bucket name and connection together for each storage type:

fs:
  url: s3://s3.example.net?access_key=DEFAULT_KEY&secret_key=DEFAULT_SECRET&region=gra
  s3:
    auto_create_buckets: false
    buckets:
      default:
        name: company-storage
      files:
        name: company-files
        url: s3://files.example.net?access_key=FILES_KEY&secret_key=FILES_SECRET&region=rbx
      exports:
        name: company-backups

The supported types are files, apps_web, apps_konnectors, assets, previews, and exports. Types without an entry use the complete default entry. To use only separate destinations, omit default and specify all six types. Several types can use the same bucket name and URL.

Every entry requires name. Its optional url supplies the endpoint, credentials, region and TLS setting; omitting it uses fs.url. An explicit entry does not inherit fields from default. Entries with the same URL reuse one connection, and each bucket on that connection is initialized once. The same bucket name can also be used on different connections. Copies between different connections stream through the stack.

Invalid names, unknown storage types, missing destinations and invalid connection URLs cause startup errors. Configuration errors do not print connection credentials.

Bucket provisioning¶

auto_create_buckets defaults to true: startup creates each distinct bucket once, using its connection’s region. Existing buckets are accepted. Credentials must allow bucket creation; bucket enumeration is not used.

With auto_create_buckets: false, create the configured buckets before starting the stack. Startup checks each distinct bucket with HeadBucket, without listing or creating buckets. Missing or inaccessible buckets fail startup with a bucket-specific error. All startup bucket operations have a 30-second timeout. Loading or initializing an instance does not probe S3.

Credentials must allow HeadBucket on each configured bucket and the object operations used by the stack, including listing, reads, writes, deletes and multipart uploads. Consult your storage provider’s documentation for the corresponding permissions. Permission to enumerate or create buckets is not required when creation is disabled. Dynamic asset health checks also use HeadBucket and honor shorter caller deadlines.

Environment variables override YAML:

COZY_FS_S3_BUCKETS_DEFAULT_NAME sets the default bucket name.
COZY_FS_S3_BUCKETS_FILES_NAME and COZY_FS_S3_BUCKETS_FILES_URL configure the files destination; the other types follow the same pattern, such as COZY_FS_S3_BUCKETS_APPS_WEB_NAME.
COZY_FS_S3_AUTO_CREATE_BUCKETS=false disables creation; true enables it.

Local development with MinIO¶

This tutorial explains how to set up a local S3 backend using MinIO for development and testing.

1. Start MinIO with Docker:

docker run -d --name minio \
  -p 9000:9000 \
  -p 9001:9001 \
  -e MINIO_ROOT_USER=minioadmin \
  -e MINIO_ROOT_PASSWORD=minioadmin \
  quay.io/minio/minio:RELEASE.2025-02-28T09-55-16Z server /data --console-address ":9001"

MinIO is now running: - S3 API: http://localhost:9000 - Web console: http://localhost:9001 (login: minioadmin / minioadmin)

2. Configure cozy-stack:

Buckets are created automatically at startup. No manual bucket creation is needed.

Edit your ~/.cozy/cozy.yaml:

fs:
  url: s3://localhost:9000?access_key=minioadmin&secret_key=minioadmin&use_ssl=false
  s3:
    buckets:
      default:
        name: company-storage

3. Build and start:

go build -o ~/go/bin/cozy-stack .
~/go/bin/cozy-stack serve

You should see in the logs:

Successfully connected to S3 endpoint localhost:9000

4. (Re)install your apps:

When switching from a different storage backend (e.g. file://), you need to reinstall the apps so their assets are stored in S3:

cozy-stack apps uninstall drive --domain your.domain.localhost:8080
cozy-stack apps install drive --domain your.domain.localhost:8080
cozy-stack apps uninstall home --domain your.domain.localhost:8080
cozy-stack apps install home --domain your.domain.localhost:8080

5. Verify:

Check that objects appear in MinIO:

docker exec minio mc ls --recursive local/company-storage/apps-web/

Upload a file via the Drive UI or the API:

TOKEN=$(cozy-stack instances token-cli your.domain.localhost:8080 io.cozy.files)
curl -X POST \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: text/plain" \
  "http://your.domain.localhost:8080/files/io.cozy.files.root-dir?Type=file&Name=test.txt" \
  -d "Hello S3!"

Verify the file is in MinIO:

docker exec minio mc ls --recursive local/company-storage/files/

6. Switching back to local filesystem:

Comment out the S3 URL in your config and restart cozy-stack:

fs:
  # url: s3://localhost:9000?access_key=minioadmin&secret_key=minioadmin&use_ssl=false

Note: files uploaded to S3 won’t be accessible when using the local filesystem backend, and vice versa. Each backend has its own storage.

Migrating an instance from Swift to S3¶

Instances can be moved from Swift to S3 one at a time, without changing the storage backend for the rest of the fleet. This is useful to validate S3 on a few instances before committing the whole platform to it.

1. Configure the migration target¶

Set fs.migration_target to the S3 URL, keeping fs.url on swift://. Configure the target buckets in fs.s3.buckets. Both connections (Swift and S3) are then initialized at startup. Bucket entries without their own url use fs.migration_target:

fs:
  url: swift://openstack/?UserName=...
  migration_target: s3://s3.example.net?access_key=ACCESS&secret_key=SECRET&region=rbx
  s3:
    auto_create_buckets: false
    buckets:
      default:
        name: company-storage

With auto_create_buckets: false, create the configured buckets before starting the stack. See fs.migration_target in the configuration documentation for details on this key.

2. Run the migration¶

cozy-stack instances migrate-storage <domain> --to s3

Use --dry-run to preview file and version counts, their combined byte size, and avatar presence. It does not copy data, create target containers, block the instance, or switch backends. This also applies with --flag-only --force. The instance remains active, so these counts can change before migration. Target verification happens only during an actual migration.

Without --dry-run, the command:

blocks HTTP access to the instance for the duration of the migration;
compares the file and version databases’ update_seq values across a five-second quiet period, aborting before copying if either changes;
holds the instance’s VFS write lock through copying, verification, backend switching and optional source cleanup;
copies the files, file versions, and the user’s avatar to the S3 target (thumbnails and installed apps are not copied; they regenerate on the target backend);
verifies the copied objects against the source;
flips the instance’s storage backend to S3 and restores its previous block state.

By default the Swift source is kept as-is after a successful migration, so it stays available for a rollback.

The CLI waits for completion without the usual 30-minute admin-client timeout. Follow the stack logs for the instance domain and the storagemigration namespace to see each phase and the final success or error. During copying, completed file, version and byte counts are logged when an object finishes and at least 30 seconds have passed since the last progress update. A single large object can therefore take longer without a progress update.

Inspect the instance in another terminal:

cozy-stack instances show <domain>

The output includes fs_scheme, blocked and blocking_reason when set. A missing fs_scheme means the global fs.url scheme is used. A switch to the target means copy verification and cutover succeeded; optional source cleanup or block-state restoration may still be pending or have failed. blocking_reason: "MOVING" marks the migration block, but is not proof that the operation is still running.

If the CLI disconnects, the server can continue migrating. Check the logs and instance state before retrying, unblocking the instance, or purging source data. Migration progress is not persisted as a job and cannot resume after a server restart.

Known limitations: the quiet-period check cannot detect uploads that are still streaming or avatar changes. The VFS lock only protects operations that use it; an upload already in progress can finalize against the source after it is released. Run migrations during low-activity periods until this is addressed.

3. Roll back if needed¶

If something looks wrong shortly after the switch, before any real write has landed on the S3 target, flip the instance back to the retained Swift source instantly, without copying anything back:

cozy-stack instances migrate-storage <domain> --to swift --flag-only --force

--force is required with --flag-only because any writes made against S3 since the cutover are lost.

If real data now lives on S3 and needs to be preserved, run a full migration back to Swift instead, which copies the data:

cozy-stack instances migrate-storage <domain> --to swift

4. Reclaim the source¶

Once confident the instance is stable on its new backend, delete the retained source objects:

cozy-stack instances migrate-storage <domain> --to s3 --purge-source

Since the instance already uses s3, this runs in purge-only mode: nothing is copied, verified, or flipped, and the previously retained Swift data is simply deleted. The instance is not blocked for this step. Running the same command again is safe and is the way to retry a purge that failed right after an earlier switch.

(--purge-source can also be supplied on the initial migration if no rollback window is needed.)

5. Switch the global default¶

After the whole fleet has been migrated to S3, change the global fs.url to the S3 URL and remove fs.migration_target. From then on, the per-instance backend flag set by earlier migrations simply matches the global default.

Storage namespaces¶

The same object paths are used whether types share a bucket or use separate buckets. Every list and deletion is restricted to its type or instance path.

Storage type	Object prefix	Content
`files`	`files/<DBPrefix>/`	Files, versions, avatars and thumbnails
`apps_web`	`apps-web/`	Installed web applications, markers and tarballs
`apps_konnectors`	`apps-konnectors/`	Installed konnectors, markers and tarballs
`assets`	`assets/`	Dynamic assets, grouped by context
`previews`	`previews/`	PDF previews and icons
`exports`	`exports/`	Instance export archives, grouped by domain

Object key structure¶

Within a bucket, each instance’s data is isolated by a key prefix derived from DBPrefix() (typically the instance domain or a CouchDB prefix).

VFS files¶

files/<DBPrefix>/<docID_part1>/<docID_part2>/<docID_part3>/<internalID>

The document ID (a 32-character UUID v7 hex string) is split into virtual subfolders to avoid flat hierarchies:

files/cozy218def.../019d35b1-9dc3-78ec-994d-f5/44336/7f1b6/e0AbCdEfGh123456
                    ^^^^^^^^^^^^^^^^^^^^^^       ^^^^^  ^^^^^  ^^^^^^^^^^^^^^^^
                    first 22 chars               5 ch   5 ch   16-char internalID

This structure mirrors the Swift V3 layout (MakeObjectNameV3).

Thumbnails¶

files/<DBPrefix>/thumbs/<docID_split>-<format>

Formats: small, medium, large.

Avatar¶

files/<DBPrefix>/avatar

Memory consumption¶

The S3 backend is designed to have comparable memory usage to Swift:

Scenario	Memory per upload
Known size, file < 5 GiB	~32 KB (single PUT, stream)
Unknown size (rare)	~5 MiB (multipart, PartSize=5MiB, NumThreads=1)

When ByteSize is known on the file document (the common case for drive uploads), the backend passes the exact size to PutObject, which uses a single PUT request that streams directly to S3 with minimal buffering — the same behavior as Swift’s ObjectCreate.

Multipart upload is only used for files with unknown size or exceeding 5 GiB, with PartSize=5MiB and NumThreads=1 to limit memory.

Encryption at rest¶

The S3 backend does not implement client-side encryption. Encryption should be configured at the infrastructure level (S3 bucket default encryption / SSE-S3), the same approach used for the Swift backend.

Differences from Swift¶

Aspect	Swift	S3
Container/Bucket	One per instance	One per organization (shared)
Instance isolation	Container name	Key prefix within bucket
Delete instance	Delete entire container	Delete all objects with key prefix
File streaming	Native `io.WriteCloser`	`io.Pipe` + `PutObject` goroutine
Bulk delete	`BulkDelete` API	`RemoveObjects` channel API
Server-side copy	`ObjectCopy`	`CopyObject` (same endpoint only)

Testing¶

The VFS integration tests run against all three backends (afero, swift, s3) using a table-driven approach. The S3 tests use testcontainers-go with a MinIO container that is started automatically.

# Run VFS tests (requires CouchDB + Docker)
COZY_COUCHDB_URL=http://admin:admin@localhost:5984/ \
  go test ./model/vfs/ -run TestVfs -v -count=1 -timeout 300s

# Run naming unit tests (no external deps)
go test ./model/vfs/vfss3/ -run "TestMakeObjectKey|TestMakeDocID" -v

# Exercise all storage types in one bucket (requires Docker, no CouchDB)
go test ./model/move/ -run TestS3SharedStorage -v -count=1 -timeout 2m