ops: bound allowance_log growth; correct stale README claims (audit L4+L5)
allowance_log: the sweep and every report-mode run used to append a row per member per run (~500/day measured). The insert is now an upsert per (member, cycle, decision) — a re-derived decision updates the row's evidence and timestamp; a decision CHANGE still gets its own row. No index or migration; the latest-decision readers are unaffected; history is not rewritten. README: scheduler runs 3x/day (0 3,11,19 UTC), not nightly-3am; the sweep's auth is the ADMIN_TOKEN X-Admin-Token header, not WEBHOOK_TOKEN; the retired free/member/supporter tier budgets replaced by the actual single MEMBER_BUDGET model; the unread OPENCOLLECTIVE_WEBHOOK_SECRET env row removed.
This commit is contained in:
1 parent
1127b41ba3
commit
00f60f1374
2 files changed
+34
-16
No files matched your search
@@ -45,19 +45,20 @@ LibreChat ──→ Member Portal ──→ LiteLLM ──→ backend
|
|||||||
| `MEMBERS_GROUP_ID` | Cloudron group ID for members (grants chat access) |
|
| `MEMBERS_GROUP_ID` | Cloudron group ID for members (grants chat access) |
|
||||||
| `LITELLM_BASE` | LiteLLM gateway URL |
|
| `LITELLM_BASE` | LiteLLM gateway URL |
|
||||||
| `LITELLM_MASTER_KEY` | LiteLLM master key |
|
| `LITELLM_MASTER_KEY` | LiteLLM master key |
|
||||||
| `OPENCOLLECTIVE_WEBHOOK_SECRET` | Optional webhook signature secret |
|
|
||||||
| `PORTAL_KEK` | **Required.** Base64-encoded 32-byte key-encryption key. Member chat keys (`members.key_token`) are stored AES-256-GCM-encrypted at rest (`enc1:` prefix) and decrypted in memory only when served (dashboard, LibreChat key feed, LiteLLM management). Generate with `python3 -c "import base64,os;print(base64.b64encode(os.urandom(32)).decode())"`. If unset, the portal runs but refuses to encrypt new keys, keeps legacy plaintext, and logs CRITICAL. Rotating it requires re-encrypting every row (decrypt with the old key, encrypt with the new). |
|
| `PORTAL_KEK` | **Required.** Base64-encoded 32-byte key-encryption key. Member chat keys (`members.key_token`) are stored AES-256-GCM-encrypted at rest (`enc1:` prefix) and decrypted in memory only when served (dashboard, LibreChat key feed, LiteLLM management). Generate with `python3 -c "import base64,os;print(base64.b64encode(os.urandom(32)).decode())"`. If unset, the portal runs but refuses to encrypt new keys, keeps legacy plaintext, and logs CRITICAL. Rotating it requires re-encrypting every row (decrypt with the old key, encrypt with the new). |
|
||||||
|
| `ADMIN_TOKEN` | The portal's admin token (`X-Admin-Token` header) for the `/admin/*` endpoints the panel calls. Never the webhook URL token — that one cannot unlock `/admin/*` by design. |
|
||||||
| `ALLOWANCE_POLICY` | `report` (default) \| `on`. `report` computes every allowance decision and changes nothing; `on` performs renewals and sends the one reminder. Deactivation is gated separately by `ALLOWANCE_DEACTIVATE`. |
|
| `ALLOWANCE_POLICY` | `report` (default) \| `on`. `report` computes every allowance decision and changes nothing; `on` performs renewals and sends the one reminder. Deactivation is gated separately by `ALLOWANCE_DEACTIVATE`. |
|
||||||
| `ALLOWANCE_DEACTIVATE` | `off` (default) \| `on`. When `on`, a member who was reminded and is still unpaid after the grace window is moved to the inactive group. **Defaults off so renewals/reminders can run before anyone loses access.** |
|
| `ALLOWANCE_DEACTIVATE` | `off` (default) \| `on`. When `on`, a member who was reminded and is still unpaid after the grace window is moved to the inactive group. **Defaults off so renewals/reminders can run before anyone loses access.** |
|
||||||
| `PAYMENT_GRACE_DAYS` | Days after the single reminder before an unpaid member is deactivated (default `7`). |
|
| `PAYMENT_GRACE_DAYS` | Days after the single reminder before an unpaid member is deactivated (default `7`). |
|
||||||
|
|
||||||
## Tier budgets
|
## Budget model
|
||||||
|
|
||||||
Governance decision (set in Loomio). Defaults:
|
Governance decision (set in Loomio): a single sliding-scale Membership tier
|
||||||
|
($5–$20/month flexible) with a FLAT allowance for every member —
|
||||||
- `free` — $2/month of tokens
|
`MEMBER_BUDGET` (default `15`), reset on each membership payment and extendable
|
||||||
- `member` — $15/month
|
with non-expiring purchased credit packs. The per-tier budgets the portal once
|
||||||
- `supporter` — $30/month
|
shipped (free/member/supporter) are retired — there are no per-tier portal
|
||||||
|
settings; the single env knob is `MEMBER_BUDGET`.
|
||||||
|
|
||||||
## LibreChat wiring
|
## LibreChat wiring
|
||||||
|
|
||||||
@@ -98,18 +99,20 @@ points. Remaining work before production:
|
|||||||
|
|
||||||
Three jobs run inside the container via `CloudronManifest.json` → `addons.scheduler`:
|
Three jobs run inside the container via `CloudronManifest.json` → `addons.scheduler`:
|
||||||
|
|
||||||
- **membership_sweep** (nightly 3am) — `sweep.sh` calls the portal's own
|
- **membership_sweep** (`0 3,11,19 * * *` UTC — three times a day) — `sweep.sh`
|
||||||
`/admin/sweep` (auth: `X-Admin-Token` = `WEBHOOK_TOKEN`) to reconcile members
|
calls the portal's own `/admin/sweep` (auth: the `X-Admin-Token` header with
|
||||||
|
the portal's `ADMIN_TOKEN` — never `WEBHOOK_TOKEN`, which is the webhook's
|
||||||
|
URL token and returns 403 on `/admin/*`) to reconcile members
|
||||||
against the live Open Collective list: provisions missed signups, deactivates
|
against the live Open Collective list: provisions missed signups, deactivates
|
||||||
lapsed members, syncs Loomio + Matrix. Idempotent; a missed webhook
|
lapsed members, syncs Loomio + Matrix. Idempotent; a missed webhook
|
||||||
self-heals within 24 hours. The same sweep also runs the payment-driven
|
self-heals within a day. The same sweep also runs the payment-driven
|
||||||
allowance reconciliation (below).
|
allowance reconciliation (below).
|
||||||
- **rag_refresh** (nightly 3:30am) — `rag_refresh.py` pulls this repo's
|
- **rag_refresh** (daily 3:30am UTC) — `rag_refresh.py` pulls this repo's
|
||||||
sibling docs from the Gitea raw API (public, no credentials), uploads changed
|
sibling docs from the Gitea raw API (public, no credentials), uploads changed
|
||||||
files to Open WebUI (`OWUI_API_KEY`), attaches them to the shared knowledge
|
files to Open WebUI (`OWUI_API_KEY`), attaches them to the shared knowledge
|
||||||
collection (`DOCS_KNOWLEDGE_ID`), reindexes, and re-grants the members group.
|
collection (`DOCS_KNOWLEDGE_ID`), reindexes, and re-grants the members group.
|
||||||
State persists in `/app/data/rag_refresh_state.json`.
|
State persists in `/app/data/rag_refresh_state.json`.
|
||||||
- **metrics_snapshot** (nightly 3:15am) — `metrics_snapshot.py` records one row per
|
- **metrics_snapshot** (daily 3:15am UTC) — `metrics_snapshot.py` records one row per
|
||||||
day into the `metrics_daily` table: members, newsletter subscribers, Matrix
|
day into the `metrics_daily` table: members, newsletter subscribers, Matrix
|
||||||
membership, spend, request counts, and energy/CO₂. A source that cannot be read
|
membership, spend, request counts, and energy/CO₂. A source that cannot be read
|
||||||
yields NULL for that metric rather than a zero. `spend_usd` is the co-op-wide
|
yields NULL for that metric rather than a zero. `spend_usd` is the co-op-wide
|
||||||
|
|||||||
+19
-4
@@ -5026,11 +5026,26 @@ async def reconcile_allowances(send: bool = False) -> dict:
|
|||||||
await revoke_member_litellm_access(email)
|
await revoke_member_litellm_access(email)
|
||||||
await listmonk_sync(email, subscribe=False)
|
await listmonk_sync(email, subscribe=False)
|
||||||
|
|
||||||
conn.execute(
|
# L4 (audit 2026-11-05): one row per (member, cycle, decision) — the
|
||||||
"INSERT INTO allowance_log (email, cycle, decision, evidence, anchor_day)"
|
# same decision re-derived on a later run UPDATES that row's evidence
|
||||||
" VALUES (?, ?, ?, ?, ?)",
|
# and timestamp instead of appending a duplicate. The sweep runs 3x/day
|
||||||
(email, cycle_key, decision, evidence, anchor_day),
|
# and report-mode runs also write here; without this the table grew
|
||||||
|
# ~500 rows/day (measured 2026-11-05: 1741 -> 2257). A decision CHANGE
|
||||||
|
# within a cycle still gets its own row, so the "latest decision per
|
||||||
|
# member" readers are unaffected and no history is rewritten.
|
||||||
|
cur = conn.execute(
|
||||||
|
"UPDATE allowance_log SET evidence = ?, anchor_day = ?, "
|
||||||
|
"decided_at = datetime('now') "
|
||||||
|
"WHERE id = (SELECT id FROM allowance_log WHERE email = ? AND cycle = ? "
|
||||||
|
"AND decision = ? ORDER BY id DESC LIMIT 1)",
|
||||||
|
(evidence, anchor_day, email, cycle_key, decision),
|
||||||
)
|
)
|
||||||
|
if cur.rowcount == 0:
|
||||||
|
conn.execute(
|
||||||
|
"INSERT INTO allowance_log (email, cycle, decision, evidence, anchor_day)"
|
||||||
|
" VALUES (?, ?, ?, ?, ?)",
|
||||||
|
(email, cycle_key, decision, evidence, anchor_day),
|
||||||
|
)
|
||||||
conn.commit()
|
conn.commit()
|
||||||
decisions.append({"email": email, "slug": oc_slug, "anchor_day": anchor_day,
|
decisions.append({"email": email, "slug": oc_slug, "anchor_day": anchor_day,
|
||||||
"cycle": cycle_key, "decision": decision, "evidence": evidence})
|
"cycle": cycle_key, "decision": decision, "evidence": evidence})
|
||||||
|
|||||||
Reference in new issue
Block a user