Configure off-box encrypted backups (single point of failure) #12

Closed
opened 2026-10-01 15:22:34 +00:00 by inference-bot · 7 comments
Owner

What's the idea?

Configure off-box, encrypted backups for the Cloudron server. Today, nightly box backups run (verified: 2026-09-30, 09-29, 09-14) but provider: None — they are stored on the same VPS disk (96 GB, 41% used) as everything they back up. A disk failure, a provider issue, or a botched migration would destroy: the members DB, LiteLLM Postgres (spend history, budgets, credits ledger), chat histories, git repos, email, and all app data — with no recovery path.

This becomes more acute with the planned hosting migration (issue #2): moving to a new provider with no off-site backup is the riskiest possible moment to have none.

Why does it matter?

  • Single point of failure currently protects the entire co-op's state
  • Members' chat histories and API usage records would be unrecoverable
  • The credits ledger (financial audit trail) lives only on this box
  • Migration risk (issue #2) compounds it

How might it work?

  1. Choose an S3-compatible backup target independent of the app server:
    • Infomaniak Object Storage (Swiss, hydro-powered, employee-owned — matches the co-op's values story and would pair well with the Infomaniak hosting migration)
    • Alternatives: Hetzner Storage Box (cheap, EU), Wasabi, any S3-compatible
  2. Configure in Cloudron: Settings → Backups → S3-compatible provider, set a schedule (daily) and retention (e.g. 14–30 days)
  3. Enable backup encryption (Cloudron supports encrypted backups; the members DB and LiteLLM keys make this important)
  4. Verify by restoring at least one app to a scratch instance once

Estimated cost: ~CHF/€ 1–5/month for object storage at our data volume (~40 GB, deduplicated likely much less).

Acceptance

  • Nightly backups land on external storage (verified in Cloudron UI)
  • Backup encryption enabled
  • One successful test restore of a critical app (portal or gateway)
  • Retention configured
  • Documented in co-op/docs (runbook: how to restore)

Filed from the 2026-10-01 security review. This was the top-priority finding.

## What's the idea? Configure **off-box, encrypted backups** for the Cloudron server. Today, nightly box backups run (verified: 2026-09-30, 09-29, 09-14) but `provider: None` — they are stored **on the same VPS disk** (96 GB, 41% used) as everything they back up. A disk failure, a provider issue, or a botched migration would destroy: the members DB, LiteLLM Postgres (spend history, budgets, credits ledger), chat histories, git repos, email, and all app data — with no recovery path. This becomes more acute with the planned hosting migration (issue #2): moving to a new provider with no off-site backup is the riskiest possible moment to have none. ## Why does it matter? - Single point of failure currently protects the entire co-op's state - Members' chat histories and API usage records would be unrecoverable - The credits ledger (financial audit trail) lives only on this box - Migration risk (issue #2) compounds it ## How might it work? 1. Choose an S3-compatible backup target independent of the app server: - **Infomaniak Object Storage** (Swiss, hydro-powered, employee-owned — matches the co-op's values story and would pair well with the Infomaniak hosting migration) - Alternatives: Hetzner Storage Box (cheap, EU), Wasabi, any S3-compatible 2. Configure in Cloudron: Settings → Backups → S3-compatible provider, set a schedule (daily) and retention (e.g. 14–30 days) 3. Enable backup encryption (Cloudron supports encrypted backups; the members DB and LiteLLM keys make this important) 4. Verify by restoring at least one app to a scratch instance once Estimated cost: ~CHF/€ 1–5/month for object storage at our data volume (~40 GB, deduplicated likely much less). ## Acceptance - [ ] Nightly backups land on external storage (verified in Cloudron UI) - [ ] Backup encryption enabled - [ ] One successful test restore of a critical app (portal or gateway) - [ ] Retention configured - [ ] Documented in co-op/docs (runbook: how to restore) *Filed from the 2026-10-01 security review. This was the top-priority finding.*

I'm interested to help with this, @ntnsndr .

I'd like to get experience with Infomaniak and/or Hetzner Storage Box.

I'm interested to help with this, @ntnsndr . I'd like to get experience with Infomaniak and/or Hetzner Storage Box.

@dckc Thanks for the interest! this is related to this: #2

I'm really interested in Infomaniak for its foundation-owned model: https://thefinancialanalyst.net/2026/08/02/sovereign-cloud-champion-takes-on-big-tech-with-infomaniaks-swiss-listing/ That seems like the best-aligned ownership model in the industry for us.

@dckc Thanks for the interest! this is related to this: https://git.inference.coop/co-op/support/issues/2 I'm really interested in Infomaniak for its foundation-owned model: https://thefinancialanalyst.net/2026/08/02/sovereign-cloud-champion-takes-on-big-tech-with-infomaniaks-swiss-listing/ That seems like the best-aligned ownership model in the industry for us.

darn; I just noticed your reply. I'm struggling with notification settings.

I'm really interested in Infomaniak for its foundation-owned model

concur. I surfed around their stuff and there's a lot to like.

_darn; I just noticed your reply. I'm struggling with notification settings._ > I'm really interested in Infomaniak for its foundation-owned model concur. I surfed around their stuff and there's a lot to like.

What do you think would be a good setup on their system? We're currently running Hostinger's KVM2 setup.

What do you think would be a good setup on their system? We're currently running Hostinger's KVM2 setup.

Their acronis backup product is 6,33€ / month / device, if I'm reading this right.

But I'm somewhat inclined to use them as an S3 provider and use something like borg (p.s. I see cloudron makes borg unnecessary).
2,05 € / month /device for Up to 1000 TB. p.s. oops; I see S3 is already in the issue description.

I don't have a lot of experience in this area, so I would want to talk it over with screensharing and such. I might do some research with bots too.

Their [acronis backup product](https://www.infomaniak.com/en/swiss-backup/acronis) is 6,33€ / month / device, if I'm reading this right. But I'm somewhat inclined to use them as an S3 provider and use something like borg (_p.s. I see cloudron makes borg unnecessary_). 2,05 € / month /device for Up to 1000 TB. _p.s. oops; I see S3 is already in the issue description._ I don't have a lot of experience in this area, so I would want to talk it over with screensharing and such. I might do some research with bots too.

I've just started a free trial with Infomaniak and am testing Cloudron full backup on their S3 service. I'll report on any updates.

I've just started a free trial with Infomaniak and am testing Cloudron full backup on their S3 service. I'll report on any updates.
Author
Owner

Resolved — off-box encrypted backups are live and restore-verified

Closing the loop on the Oct 19 trial note: the Infomaniak Swiss Backup (S3) setup is configured and verified end-to-end — not just "backups exist," but a real restore was run and passed.

Final configuration

  • Off-box tier: Infomaniak Swiss Backup S3 — s3.swiss-backup02.infomaniak.com, bucket default, prefix cloudron-encrypted/, backups encrypted (v2). Schedule 2×/day (23:00 + 11:00 UTC), retention 7 days, plan doubled to 100 GB. Keys rotated post-setup.
  • On-box tier (kept, re-purposed): the local backup site now fires only before app updates (schedule "never") — a fast local rollback tier. Scheduled durability lives entirely at Infomaniak, which resolves the single-point-of-failure this issue was about.

How it was verified

  1. Bucket contents listed directly with the S3 credentials: 72 objects, ~5.6 GB per run (each run stores a session dir + a snapshot/ mirror — size plans for ~2× the data).
  2. Cloudron's backup integrity check (downloads the full set and hashes everything): passed.
  3. Restore test: the admin panel was restored from the encrypted Infomaniak backup via the dashboard — completed successfully, app serving after.

Gotchas worth keeping (all documented in our ops skill)

  • Infomaniak's manager surfaces Swift-style fields (user name, tenant, domain) — the S3 setup only needs endpoint + bucket default + access/secret keys. Also: the usage meter can read 0 while data is landing; verify the bucket directly.
  • Cloudron runs a backup-level integrity check before any restore; on our 3 GB-RAM box it needs swap. A crashed check blocks later restores until re-run.
  • Encrypted restores must go through the dashboard (the CLI cannot supply the encryption password).

@dckc — your S3 instinct was right; Cloudron handles the encryption on top. Thanks for the pointers.

## Resolved — off-box encrypted backups are live and restore-verified Closing the loop on the Oct 19 trial note: the Infomaniak Swiss Backup (S3) setup is **configured and verified end-to-end** — not just "backups exist," but a real restore was run and passed. **Final configuration** - **Off-box tier:** Infomaniak Swiss Backup S3 — `s3.swiss-backup02.infomaniak.com`, bucket `default`, prefix `cloudron-encrypted/`, backups **encrypted** (v2). Schedule **2×/day** (23:00 + 11:00 UTC), retention **7 days**, plan doubled to **100 GB**. Keys rotated post-setup. - **On-box tier (kept, re-purposed):** the local backup site now fires *only before app updates* (schedule "never") — a fast local rollback tier. Scheduled durability lives entirely at Infomaniak, which resolves the single-point-of-failure this issue was about. **How it was verified** 1. Bucket contents listed directly with the S3 credentials: 72 objects, ~5.6 GB per run (each run stores a session dir + a `snapshot/` mirror — size plans for ~2× the data). 2. Cloudron's backup integrity check (downloads the full set and hashes everything): **passed**. 3. **Restore test:** the admin panel was restored from the encrypted Infomaniak backup via the dashboard — completed successfully, app serving after. **Gotchas worth keeping (all documented in our ops skill)** - Infomaniak's manager surfaces Swift-style fields (user name, tenant, domain) — the S3 setup only needs endpoint + bucket `default` + access/secret keys. Also: the usage meter can read 0 while data is landing; verify the bucket directly. - Cloudron runs a backup-level integrity check before any restore; on our 3 GB-RAM box it needs swap. A crashed check blocks later restores until re-run. - Encrypted restores must go through the dashboard (the CLI cannot supply the encryption password). @dckc — your S3 instinct was right; Cloudron handles the encryption on top. Thanks for the pointers.
Sign in to join this conversation.
3 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: co-op/support#12