From 00f60f137427f6b06db576cc487483630e72df0c Mon Sep 17 00:00:00 2001 From: Inferencebot Date: Thu, 5 Nov 2026 16:26:16 +0000 Subject: [PATCH] ops: bound allowance_log growth; correct stale README claims (audit L4+L5) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit allowance_log: the sweep and every report-mode run used to append a row per member per run (~500/day measured). The insert is now an upsert per (member, cycle, decision) — a re-derived decision updates the row's evidence and timestamp; a decision CHANGE still gets its own row. No index or migration; the latest-decision readers are unaffected; history is not rewritten. README: scheduler runs 3x/day (0 3,11,19 UTC), not nightly-3am; the sweep's auth is the ADMIN_TOKEN X-Admin-Token header, not WEBHOOK_TOKEN; the retired free/member/supporter tier budgets replaced by the actual single MEMBER_BUDGET model; the unread OPENCOLLECTIVE_WEBHOOK_SECRET env row removed. --- README.md | 27 +++++++++++++++------------ app/main.py | 23 +++++++++++++++++++---- 2 files changed, 34 insertions(+), 16 deletions(-) diff --git a/README.md b/README.md index e858033..677081b 100644 --- a/README.md +++ b/README.md @@ -45,19 +45,20 @@ LibreChat ──→ Member Portal ──→ LiteLLM ──→ backend | `MEMBERS_GROUP_ID` | Cloudron group ID for members (grants chat access) | | `LITELLM_BASE` | LiteLLM gateway URL | | `LITELLM_MASTER_KEY` | LiteLLM master key | -| `OPENCOLLECTIVE_WEBHOOK_SECRET` | Optional webhook signature secret | | `PORTAL_KEK` | **Required.** Base64-encoded 32-byte key-encryption key. Member chat keys (`members.key_token`) are stored AES-256-GCM-encrypted at rest (`enc1:` prefix) and decrypted in memory only when served (dashboard, LibreChat key feed, LiteLLM management). Generate with `python3 -c "import base64,os;print(base64.b64encode(os.urandom(32)).decode())"`. If unset, the portal runs but refuses to encrypt new keys, keeps legacy plaintext, and logs CRITICAL. Rotating it requires re-encrypting every row (decrypt with the old key, encrypt with the new). | +| `ADMIN_TOKEN` | The portal's admin token (`X-Admin-Token` header) for the `/admin/*` endpoints the panel calls. Never the webhook URL token — that one cannot unlock `/admin/*` by design. | | `ALLOWANCE_POLICY` | `report` (default) \| `on`. `report` computes every allowance decision and changes nothing; `on` performs renewals and sends the one reminder. Deactivation is gated separately by `ALLOWANCE_DEACTIVATE`. | | `ALLOWANCE_DEACTIVATE` | `off` (default) \| `on`. When `on`, a member who was reminded and is still unpaid after the grace window is moved to the inactive group. **Defaults off so renewals/reminders can run before anyone loses access.** | | `PAYMENT_GRACE_DAYS` | Days after the single reminder before an unpaid member is deactivated (default `7`). | -## Tier budgets +## Budget model -Governance decision (set in Loomio). Defaults: - -- `free` — $2/month of tokens -- `member` — $15/month -- `supporter` — $30/month +Governance decision (set in Loomio): a single sliding-scale Membership tier +($5–$20/month flexible) with a FLAT allowance for every member — +`MEMBER_BUDGET` (default `15`), reset on each membership payment and extendable +with non-expiring purchased credit packs. The per-tier budgets the portal once +shipped (free/member/supporter) are retired — there are no per-tier portal +settings; the single env knob is `MEMBER_BUDGET`. ## LibreChat wiring @@ -98,18 +99,20 @@ points. Remaining work before production: Three jobs run inside the container via `CloudronManifest.json` → `addons.scheduler`: -- **membership_sweep** (nightly 3am) — `sweep.sh` calls the portal's own - `/admin/sweep` (auth: `X-Admin-Token` = `WEBHOOK_TOKEN`) to reconcile members +- **membership_sweep** (`0 3,11,19 * * *` UTC — three times a day) — `sweep.sh` + calls the portal's own `/admin/sweep` (auth: the `X-Admin-Token` header with + the portal's `ADMIN_TOKEN` — never `WEBHOOK_TOKEN`, which is the webhook's + URL token and returns 403 on `/admin/*`) to reconcile members against the live Open Collective list: provisions missed signups, deactivates lapsed members, syncs Loomio + Matrix. Idempotent; a missed webhook - self-heals within 24 hours. The same sweep also runs the payment-driven + self-heals within a day. The same sweep also runs the payment-driven allowance reconciliation (below). -- **rag_refresh** (nightly 3:30am) — `rag_refresh.py` pulls this repo's +- **rag_refresh** (daily 3:30am UTC) — `rag_refresh.py` pulls this repo's sibling docs from the Gitea raw API (public, no credentials), uploads changed files to Open WebUI (`OWUI_API_KEY`), attaches them to the shared knowledge collection (`DOCS_KNOWLEDGE_ID`), reindexes, and re-grants the members group. State persists in `/app/data/rag_refresh_state.json`. -- **metrics_snapshot** (nightly 3:15am) — `metrics_snapshot.py` records one row per +- **metrics_snapshot** (daily 3:15am UTC) — `metrics_snapshot.py` records one row per day into the `metrics_daily` table: members, newsletter subscribers, Matrix membership, spend, request counts, and energy/CO₂. A source that cannot be read yields NULL for that metric rather than a zero. `spend_usd` is the co-op-wide diff --git a/app/main.py b/app/main.py index ede3be7..f2bb98f 100644 --- a/app/main.py +++ b/app/main.py @@ -5026,11 +5026,26 @@ async def reconcile_allowances(send: bool = False) -> dict: await revoke_member_litellm_access(email) await listmonk_sync(email, subscribe=False) - conn.execute( - "INSERT INTO allowance_log (email, cycle, decision, evidence, anchor_day)" - " VALUES (?, ?, ?, ?, ?)", - (email, cycle_key, decision, evidence, anchor_day), + # L4 (audit 2026-11-05): one row per (member, cycle, decision) — the + # same decision re-derived on a later run UPDATES that row's evidence + # and timestamp instead of appending a duplicate. The sweep runs 3x/day + # and report-mode runs also write here; without this the table grew + # ~500 rows/day (measured 2026-11-05: 1741 -> 2257). A decision CHANGE + # within a cycle still gets its own row, so the "latest decision per + # member" readers are unaffected and no history is rewritten. + cur = conn.execute( + "UPDATE allowance_log SET evidence = ?, anchor_day = ?, " + "decided_at = datetime('now') " + "WHERE id = (SELECT id FROM allowance_log WHERE email = ? AND cycle = ? " + "AND decision = ? ORDER BY id DESC LIMIT 1)", + (evidence, anchor_day, email, cycle_key, decision), ) + if cur.rowcount == 0: + conn.execute( + "INSERT INTO allowance_log (email, cycle, decision, evidence, anchor_day)" + " VALUES (?, ?, ?, ?, ?)", + (email, cycle_key, decision, evidence, anchor_day), + ) conn.commit() decisions.append({"email": email, "slug": oc_slug, "anchor_day": anchor_day, "cycle": cycle_key, "decision": decision, "evidence": evidence})