operator-token-runbook: add the credential rotation policy and reminder
Records the 12-month cadence and the trigger rules agreed in co-op/support#35, a last-rotated table (first entry: the Open Collective token, rotated 2026-10-27; other rows blank because nothing is recorded), and the account map behind the credentials. Adds scripts/rotation_due.py, which reads the table and prints only when a credential is within 30 days of its due date.
This commit is contained in:
1 parent
63d14156ec
commit
43f8f8c3fe
2 files changed
+237
-2
No files matched your search
@@ -3,6 +3,85 @@
|
||||
How the co-op's Cloudron credentials are organized, and how to rotate them
|
||||
without breaking the stack.
|
||||
|
||||
## Credential rotation policy
|
||||
|
||||
**Cadence: 12 months for every credential.** One interval, no exceptions for now.
|
||||
Twelve months is a starting point, not a principle — revisit it once the token
|
||||
list is stable (co-op/support#35).
|
||||
|
||||
**Rotate on a trigger immediately, off the calendar, when any of these happens:**
|
||||
|
||||
- **Offboarding** — anyone who had access to the credential leaves or changes
|
||||
role, or the device/account it lived on is lost.
|
||||
- **Suspected exposure** — the value was seen somewhere it should not be: a log,
|
||||
a chat, a commit, a screenshot.
|
||||
- **After any incident, or any diagnostic session on the agent host in which a
|
||||
secret could have been read.** Reading a live secret to debug with it is itself
|
||||
the trigger; it does not have to be a confirmed breach.
|
||||
- **After any restore** — the credential travels in the backup, so a restored
|
||||
host re-materialises whatever values the snapshot held.
|
||||
|
||||
**The GM controls all tokens for now** (operator, 2026-11-04), so this record
|
||||
needs **no owner column**: every row is rotated by the GM. Provider keys are
|
||||
minted on the provider account, but the value held in the co-op's environment is
|
||||
still the GM's to rotate.
|
||||
|
||||
### Last rotated
|
||||
|
||||
One row per credential, or per credential group — a group rotates together, so it
|
||||
dates together. A blank cell means **no rotation has been recorded**, not "not
|
||||
rotated". Dates are UTC, `YYYY-MM-DD`.
|
||||
|
||||
The reminder reads this exact table: `scripts/rotation_due.py` in this repository
|
||||
(deployed on the agent host at `~/.hermes/scripts/rotation_due.py`) prints a line
|
||||
only when a due date is within 30 days, and prints nothing otherwise. Keep the
|
||||
table machine-legible — one `YYYY-MM-DD` date per row.
|
||||
|
||||
| Credential or group | Last rotated (UTC) |
|
||||
|---|---|
|
||||
| Open Collective token (`OC_PERSONAL_TOKEN`) | 2026-10-27 |
|
||||
| `LITELLM_MASTER_KEY` / `GW_MASTER_KEY` (one value, four holders) | |
|
||||
| Cloudron API tokens — portal + agent (both on `inference-bot`) | |
|
||||
| `REGISTRY_BUILD_TOKEN` (on `inference-admin`; moves to CI, co-op/support#30) | |
|
||||
| Gitea token (`inference-bot`) | |
|
||||
| `WEBHOOK_TOKEN` (portal env + Open Collective webhook URL) | |
|
||||
| `ADMIN_TOKEN` (portal + admin panel + agent host) | |
|
||||
| `BROKER_SECRET` and `PORTAL_SECRET` | |
|
||||
| Matrix tokens — portal `MATRIX_BOT_TOKEN` + agent `MATRIX_ADMIN_TOKEN` (one account, `@inference-admin`) | |
|
||||
| Matrix token — steward agent `@inferencebot` | |
|
||||
| Loomio API keys (`LOOMIO_API_KEY`, `LOOMIO_B3_KEY`) | |
|
||||
| Listmonk API (`LISTMONK_API_USER`, `LISTMONK_API_TOKEN`) | |
|
||||
| Provider keys — Tinfoil, GreenPT, Regolo, and the GreenPT fallback | |
|
||||
| Restic repository passphrase; Infomaniak Swiss Backup S3 keys | |
|
||||
|
||||
### Accounts behind the credentials
|
||||
|
||||
A credential belongs to an **account**, which is what a rotation is actually
|
||||
performed against — ask the provider, do not read the variable name. Two
|
||||
credentials on one account are a single hazard: rotating one and not the other is
|
||||
the silent break.
|
||||
|
||||
| Credential | Account | Basis |
|
||||
|---|---|---|
|
||||
| `LITELLM_MASTER_KEY` / `GW_MASTER_KEY` | self-issued by the gateway | one literal, four holders |
|
||||
| Cloudron API tokens — portal and agent | `inference-bot` | verified (co-op/support#35) |
|
||||
| `REGISTRY_BUILD_TOKEN` | `inference-admin` | verified (co-op/support#35) |
|
||||
| `MATRIX_BOT_TOKEN` (portal) + `MATRIX_ADMIN_TOKEN` (agent) | `@inference-admin:inference.coop` — two tokens, one account | verified, homeserver `whoami` 2026-11-04 |
|
||||
| Steward agent's `MATRIX_ACCESS_TOKEN` | `@inferencebot:inference.coop` | verified, homeserver `whoami` 2026-11-04 |
|
||||
| `OC_PERSONAL_TOKEN` | `info@inference.coop` | from the 2026-10-27 rotation record (inference) |
|
||||
| `TINFOIL_API_KEY` | `info@inference.coop` | operator-confirmed |
|
||||
| `GREENPT_API_KEY` (gateway) and `GREENPT_FALLBACK_API_KEY` (agent) | the GreenPT account — the **same value** in both | verified: the "fallback" isolates nothing |
|
||||
| `REGOLO_API_KEY` | Regolo account | not yet identified |
|
||||
| Gitea token | `inference-bot` | inferred, not verified |
|
||||
| `BROKER_SECRET`, `PORTAL_SECRET`, `WEBHOOK_TOKEN`, `ADMIN_TOKEN`, `LOOMIO_*`, `LISTMONK_*` | self-issued / service account | — |
|
||||
| Restic passphrase; Infomaniak Swiss Backup S3 keys | self-issued; the Infomaniak account | — |
|
||||
|
||||
The two couplings to watch: **one account with several credentials** (the
|
||||
`@inference-admin` Matrix pair, the two Cloudron tokens, the four holders of the
|
||||
master key) and **one value with several holders** (the GreenPT key, which is the
|
||||
only such case found). Full account map, including the hash comparison behind the
|
||||
GreenPT finding, is held with the security working notes.
|
||||
|
||||
## The two-token split
|
||||
|
||||
There are **two separate** Cloudron API tokens, deliberately decoupled so
|
||||
@@ -202,7 +281,7 @@ row that names the same credential: several are held by more than one app.
|
||||
| `WEBHOOK_TOKEN` | Secret in the Open Collective webhook URL | Portal app env, and the webhook URL on Open Collective | GM | Rotating means re-creating the webhook on OC as well |
|
||||
| `LITELLM_MASTER_KEY` | Create gateway teams and keys for members | Portal app env (copy of the gateway's) | GM | Three holders: portal, gateway, steward agent. Rotate all together |
|
||||
| `LITELLM_DB_*` | Read spend logs from the gateway database | Portal app env | Cloudron (addon) | Postgres credentials for the gateway's database |
|
||||
| `MATRIX_BOT_TOKEN` | `@inference-bot:inference.coop` admin account: invites, space membership | Portal app env | GM | The account has no password. Paired with the steward agent's `MATRIX_LOCAL_BOT_TOKEN`: same account, separate token on a separate device (verified). Revoke or rotate both together |
|
||||
| `MATRIX_BOT_TOKEN` | `@inference-admin:inference.coop` admin account: invites, space membership | Portal app env | GM | The account has no password. Paired with the steward agent's `MATRIX_ADMIN_TOKEN`: same account, separate token on a separate device (verified by homeserver `whoami`, 2026-11-04). Revoke or rotate both together |
|
||||
| `LOOMIO_API_KEY`, `LOOMIO_B3_KEY` | Forum (Loomio) membership sync | Portal app env | GM | |
|
||||
| `LISTMONK_API_USER`, `LISTMONK_API_TOKEN` | Newsletter list sync | Portal app env | GM | |
|
||||
| `OWUI_API_KEY`, `OWUI_JWT_SECRET` | Open WebUI knowledge refresh and SSO | Portal app env | GM | Chat moved to LibreChat in October; confirm whether still needed, and retire if not |
|
||||
@@ -236,7 +315,7 @@ granularity for the public half. Highest privilege first.
|
||||
| Credential | Purpose | Where it lives | Who rotates | Notes |
|
||||
|---|---|---|---|---|
|
||||
| Gateway master key (`GW_MASTER_KEY`) | Admin over every gateway team, key and spend record | Steward agent `.env` | GM | **Third holder**, with the portal and the gateway. The row most in need of coordinated rotation |
|
||||
| `MATRIX_LOCAL_BOT_TOKEN` | The `@inference-bot:inference.coop` automation account: room administration in every co-op room | Steward agent `.env` | GM | Same account as the portal's `MATRIX_BOT_TOKEN`, but a separate token on a separate device (verified). Revoking one does not revoke the other; revoke or rotate both together |
|
||||
| `MATRIX_ADMIN_TOKEN` | The `@inference-admin:inference.coop` automation account: room administration in every co-op room | Steward agent `.env` | GM | Same account as the portal's `MATRIX_BOT_TOKEN`, but a separate token on a separate device (verified by homeserver `whoami`, 2026-11-04). Revoking one does not revoke the other; revoke or rotate both together |
|
||||
| Cloudron agent token | Cloudron API | Steward agent `.env` | GM | See "The two-token split" above |
|
||||
| `REGISTRY_BUILD_TOKEN` (and its username) | Image build and push for the custom apps | Steward agent `.env` | GM | Moves to CI (co-op/support#30); the agent stops holding it then |
|
||||
| Gitea token | Commits, issues, PRs as `inference-bot` | Steward agent `.env` | GM | |
|
||||
|
||||
Reference in new issue
Block a user