From d7cee510cb5518c931cd2b8f53984d098918201c Mon Sep 17 00:00:00 2001 From: InferenceBot Date: Wed, 28 Oct 2026 13:55:28 +0000 Subject: [PATCH] runbook: admin/webhook token split, metrics_snapshot task, correct sweep auth --- operator-token-runbook.md | 41 +++++++++++++++++++++++++++++++++++---- 1 file changed, 37 insertions(+), 4 deletions(-) diff --git a/operator-token-runbook.md b/operator-token-runbook.md index 6fabfc1..583444e 100644 --- a/operator-token-runbook.md +++ b/operator-token-runbook.md @@ -39,6 +39,34 @@ copy, the portal's Cloudron calls start returning `401` — which breaks provisioning (new members can't be created) and the member dashboard (its usage/keys data is resolved through the portal's Cloudron calls). +## Admin token vs webhook token (portal) + +Two more, separate from the Cloudron tokens above: + +| Token | Purpose | Where it lives | +|-------|---------|----------------| +| **`ADMIN_TOKEN`** | Authenticates `/admin/*`; sent ONLY as an `X-Admin-Token` header | Portal env; the **admin panel's own env has a separate copy**; `~/.hermes/.env` on the agent host | +| **`WEBHOOK_TOKEN`** | Authenticates Open Collective's inbound webhook | Portal env, and inside the webhook URL configured on the collective | + +They were once one value, so anything that learned the webhook URL — OC admins, +OC's logs, our access logs, a Referer — got portal admin. Split 2026-10-27; the +cutover completed 2026-10-28 (`ADMIN_TOKEN_LEGACY=0`), verified on the wire: +`ADMIN_TOKEN` → HTTP 200, `WEBHOOK_TOKEN` → 403, no header → 403. + +- **`WEBHOOK_TOKEN` is not retired and cannot be.** OC's generic webhooks cannot + send headers, so a secret in the URL is the standard authentication. Its only + power now is authenticating that one inbound route, which is the intended end + state. +- **Rotating `ADMIN_TOKEN` touches THREE places** — the portal env (shared by the + app and the scheduler addon), the admin panel's separate copy, and the agent's + `~/.hermes/.env`. Miss one and that caller 403s silently. +- **Rotating `WEBHOOK_TOKEN` needs Open Collective admin access**, because the + value lives in the webhook URL on the collective — the same blocker as moving + `OC_PERSONAL_TOKEN` (below) to a co-op-owned account. +- **No caller may fall back from `ADMIN_TOKEN` to `WEBHOOK_TOKEN`.** The legacy + token 403s, so a fallback turns "unset" into a confusing mid-run failure + instead of a clear abort. + ## Chat image: spaCy model for document parsing (EPUB/DOCX) Open WebUI's document parser (`unstructured`) lazily downloads the spaCy model @@ -64,10 +92,14 @@ The portal and gateway run their own housekeeping via the Cloudron scheduler addon, declared in each app's `CloudronManifest.json` under `addons.scheduler`: - **portal**: `membership_sweep` (nightly 3am, `/app/code/sweep.sh` → - `POST /admin/sweep` with the `WEBHOOK_TOKEN` header) and `rag_refresh` + `POST /admin/sweep` with the **`ADMIN_TOKEN`** header — NOT `WEBHOOK_TOKEN`, + which returns 403 on `/admin/*` since the 2026-10-28 cutover), `rag_refresh` (nightly 3:30am, `/app/code/rag_refresh.py` — pulls docs from the Gitea raw API and refreshes the chat knowledge base; needs `OWUI_API_KEY` + - `DOCS_KNOWLEDGE_ID` in the app env). + `DOCS_KNOWLEDGE_ID` in the app env), and `metrics_snapshot` (nightly 3:15am, + `/app/code/metrics_snapshot.py` — writes the day's members / usage / energy / + carbon row into the portal's `metrics_daily` table; `--backfill` rebuilds + history and is idempotent). - **gateway**: `audio_keepalive` (every 10 minutes, `/app/code/audio_keepalive.py` — pings the Tinfoil audio models locally to avoid the ~24s TEE cold-start; reads the master key from `/app/data/.master_key`, which `start.sh` writes — @@ -109,8 +141,9 @@ Env vars for custom apps are set via: ## Open Collective: the bot token needs the `email` scope (Sept 2026) -The portal identifies members by email, read via the bot's OC personal token -(`OC_PERSONAL_TOKEN`) as `collective.members.nodes[].account.email` with the +The portal identifies members by email, read via a **personal** OC token +(`OC_PERSONAL_TOKEN`) — currently Nathan's own rather than a co-op-owned +account's (open item F5); the fix is a co-op account plus a token minted there as `collective.members.nodes[].account.email` with the `... on Individual { email }` inline fragment. **The `email` scope is load-bearing.** The bot's personal token must be minted