The member dashboard's usage chart reads per-day spend + requests for the
signed-in member, bounded by the existing days param (0 = all time, the
unchanged default). Same source and day convention as the co-op daily series
(to_char(startTime)); interior zero days are explicit so chart spacing is
true; requests excludes failed attempts; spend sums every row. Additive and
backward compatible: every existing field untouched. Tested in
tests/test_broker_usage_daily.py (route table, SQL shape, fallback twin,
fail-soft degradation).
The 2026-11-12 hole was NOT a missed sample: the 03:15 run recorded
oc_balance_usd=1357.39, and the evening repair backfill's full-row
upsert of the then-current day — whose dict carries no balance key —
nulled it on conflict. Nothing ever revisited a day's balance, so the
hole was permanent.
Two fixes:
- upsert() can no longer erase a balance it did not measure: the
conflict clause COALESCEs, so a NULL (a failed OC read) keeps the
previously recorded value.
- the nightly snapshot heals any recent day (within a week, after the
series' first recorded balance) whose balance is missing: the value is
reconstructed from Open Collective's transaction ledger
(balance(cutoff) = current - net flow after the 03:15 sample moment,
ledger window clamped to the balance read), matching the daily sample
semantics exactly. It is written balance-column-only, so membership
and usage series are untouched. Days before 2026-11-08 stay NULL by
design. Fail-soft: an OC failure prints and retries next night.
The reconstruction was verified to the cent against OC's own ledger:
full-ledger signed sum == stats.balance (1470.44 on 2026-11-13), and
1470.44 - 113.05 (net flow 2026-11-12 03:15 -> 2026-11-13 03:15:58)
== 1357.39, exactly the sample the 03:15 run had recorded.
snapshot() fetched the full spend window (since = yesterday) but kept only
spend.get(TODAY): the day's first hours, written at 03:15 and never revisited.
Every metrics_daily row after the last backfill under-counted its day by the
hours after 03:15 (Nov 7 recorded $2.19/691 requests against a true
$29.06/3384). The completed day's aggregates were computed and thrown away.
Now each run completes the day that just ENDED — full-day spend, requests,
tokens and impact, written to the usage columns only (upsert_usage: membership
columns are point-in-time measurements and are never rewritten) — and then
records today's live partial as before.
backfill() gains --since YYYY-MM-DD: a targeted usage-only repair that cannot
touch the published membership curve.
Regression test: tests/test_metrics_completion.py (day completion, membership
preservation, since-mode repair boundaries, full-backfill parity).
Single derivation point (tested once, here): 'invited' while the Cloudron
invite is unaccepted, else the guests-ledger status (the record of record),
membership state only as a no-ledger-row fallback. The panel and dashboard
become pure consumers of `state` — their local invited/active derivation
goes away in their own repos next. Live-behaviour note: today's four live
guests render byte-identically; the change only shows for ended guests
(ledger 'expired' instead of the bare inactive flag), which is what the
panel already displayed via its own guest_status preference.
The one-per-30-days rule is a READ of the guests ledger, provisioning
takes seconds of external calls, and the ledger row that closes the
window is written LAST — so two PARALLEL direct calls (curl, bypassing
the dashboard JS) could both pass guest_request_guard before either row
exists and slip two guests past the rule. The member invite path now
runs guard+provision under _GUEST_INVITE_LOCK (single-process uvicorn),
making check-and-send atomic: the second caller re-runs the guard after
the first caller's ledger row exists and is refused 429.
tests/test_guest_concurrency.py proves both halves on the fixture DB:
unguarded, two parallel invites both provision (the test has teeth);
with the lock, exactly one provisions, one is refused 429, one ledger
row, one account, one email; sequential semantics unchanged.
A guest invitation provisions immediately, but the Cloudron account is not
ACCEPTED until the guest clicks the setup link (inviteAccepted=false, no
username) — until then they cannot log in, so no surface may read them as
active. guest_invite_pending() derives it from the same Cloudron signal the
overview's activated flag uses; unseen-record (users fetch failed) is never
pending, so the state falls back instead of asserting what the data does
not show.
/admin/overview: guest rows report state='invited' with an 'awaiting
setup' reason (invite date named when the ledger has it); /broker/guests
trail entries carry state='invited' the same way, status stays the ledger
record of record. Display/state semantics only — access rules, the $2 cap
and the expiry box are unchanged.
Tests: new test_guest_invited_state.py (derivation, overview flip,
fetch-failure fallback, member rows untouched, trail state);
test_guest_requests.py gains the trail invited/accepted assertions and a
constructible-but-loud fake client so the stubbed users fetch runs.
Operator decision 2026-11-08: the remaining protections (active member,
1 guest per inviter per rolling 30 days, one guest per recipient ever,
no existing portal row, MANUAL_MEMBERS refusal, the $2 cap and the
GUEST_DAYS box) are sufficient without the approval step.
* POST /broker/guests provisions through provision_guest_member at once —
no pending state; no guest_requests row is ever written.
* GUEST_AUTO_APPROVE flag removed (the live env never carried it).
* guest_request_guard keeps the member-path rules and drops the
trial-era pending-request checks; the 30-day window is counted from
the guests ledger.
* GET /admin/guest-requests + approve/reject removed; GET /admin/guest-ledger
serves the panel's duplicate-send guard.
* The member trail keeps legacy rejected requests as history only.
Built on the sibling $2-cap commit 6be8e2d.
Operator edits:
- The co-op is described only as a member-governed AI cooperative; the
renewable-energy claim is removed (not all providers are renewable).
- Membership pitch now states the pricing exactly: pay on a sliding
scale, $15/month allowance plus optional credit packs, plus a say in
how it's run.
The admin panel's /admin/provision-guest now takes ONE field (the guest's
email): no name is collected and none is derived from the email — the
guest chooses their own display name and username during Cloudron account
setup, and the invitation email greets neutrally when no name is known.
The guests ledger records the co-op's ADMIN account (ADMIN_INVITER_EMAIL,
default info@inference.coop) as the inviter. Admin invites provision
immediately (the operator IS the approver) and are not bound by the
member's one-per-30-days cap; recipient-side guards (one guest per
recipient ever, no existing portal row) still run. The member paths
(broker invite, request approval) keep every protection and likewise pass
no derived name.
POST /broker/guests (member broker identity — guests cannot reach it) creates
a guest REQUEST; GET /broker/guests returns the member's invitation allowance
(rolling one-per-30-days, counting pending + approved requests) and their own
guest trail. guest_request_guard enforces every existing protection plus the
member-side ones: recipient never an existing member, a prior guest, a pending
request, or a MANUAL_MEMBERS address; the inviter's window counts requests too.
Trial gate (GUEST_AUTO_APPROVE, default off): requests land PENDING in the new
guest_requests table; the operator approves or rejects them via
GET/POST /admin/guest-requests (X-Admin-Token). Approval fires the full
provision_guest_member pipeline (Cloudron user, $5 credit, GUEST_DAYS box,
invitation email naming the inviter, newsletter, ledger row); a guard refusal
at approve-time is recorded on the request and surfaced. The member sees
pending/not-approved status in their own trail — the recipient is contacted
only on approval. With the flag on, a member's request provisions
immediately, no code change needed.
38-check fixture suite (tests/test_guest_requests.py) proves the gate, every
refusal path, the approve/reject actions and auto-approve with the network
replaced by a raising guard.
Two defects found by booting the renamed app against a fixture of the
pre-rename database (the live portal's exact state):
1. members.gift_expires_at: the RENAME above uses the stale
snapshot, so the plain
below fired right after a successful rename and crashed boot with
'duplicate column name' — the portal would have failed its deploy
health check on the live database. Make the two branches exclusive.
2. The ledger's own expiry column: ALTER TABLE guests RENAME moved the
table but kept its legacy gift_expires_at column, which the provision
INSERT (writing guest_expires_at) would fail on at first use. Rename
the column with the table.
Also moves the credit-ledger dedupe references ('gift:<email>' /
'gift-return:<email>') to their guest prefixes at boot — returns first,
since 'gift-return:…' also matches 'gift:%' — so a re-grant after the
rename cannot double-credit.
Verified: migration harness 10/10 on a legacy-state fixture (renames,
value conversion, ref migration, idempotent second boot, readback
helpers); full portal suite 9/9 files green.
Pure rename, no behaviour change, matching the renamed panel and dashboard
builds. Includes one-time idempotent migrations for existing databases
(gifts->guests table, gift_expires_at->guest_expires_at column,
account_type 'gift'->'guest' values, gift_lapse_notices->guest_lapse_notices)
and four transitional alias routes (admin/provision-gift, admin/gift-convert,
broker/gift-status, broker/cancel-gift) so pre-rename dashboard/panel builds
keep working until they are redeployed.
POST /broker/cancel-gift (dashboard broker identity, gift accounts only,
idempotent) + GET /broker/gift-status for the ended-state view. Semantics
match the gift lapse: row deactivated, account kept for a clean resume,
gateway budget zeroed, newsletter off — plus the return a lapse does not
do: unspent credit back to the co-op as a NEGATIVE credit_ledger row whose
reason names the cancellation and the original gift reference, deduped on
its own reference. Gifts ledger records inviter → recipient → 'cancelled
by recipient'. The user leaves the guests group for GUESTS_ENDED_GROUP_ID
(dashboard-only visibility for the ended state); the reaching-limits
notice is pre-marked so a self-cancelled guest is never told their
credits were spent. Tests: tests/test_gift_cancel.py (19 fixture checks,
network-guarded).
- gift_welcome_email becomes gift_invitation_email: the ONE email a gift
sends (the invitation and the guest welcome merged, not two mails at
once). Subject 'You've been invited to the Inference Cooperative'; names
the inviter (member slug), the $5 credits, the setup link, the
Become-a-member button on BECOME_MEMBER_URL. Uses the shared
PINE_LOGO_SVG instead of an inline copy.
- reaching-limits email (gift_limits_email, 'Your guest access has ended —
become a member to continue') wired into the sweep's gift deactivation:
sent once per guest, gated by a gift_lapse_notices sent-state row (same
pattern as mark_shortfall_notice_sent). Says the access ended, the
account and data are kept, and membership ($15/mo via Open Collective)
restores a full monthly allowance + community spaces.
- catch-up pass in reconcile_gift_lifecycles: guests deactivated before the
email shipped (or a failed send) get exactly one notice on a later sweep;
converted guests are excluded by account_type.
- sweep summary now reports gift_lapse_notices.
- test_gift_lifecycle: invitation-subject assertion; send_email stub
returns True (the bool is the gate's contract); new once-only + catch-up
+ healthy-untouched checks. 43/43.
/admin/overview now returns gift (guest) accounts in a separate guests[]
list instead of inside members[], and excludes them from totals.members,
totals.active and the member spend totals. Guests get their own totals
breakout (guests, guests_active, guest_spend, guest_all_time_spend) so
their spend stays visible without ever reading as member activity — the
same reasoning as the operator (admins) and agent categories.
Builds on the gift feature's account_type marker: a guest is any row with
account_type='gift'. The panel reads guests[] and ignores gift rows in
its member count either way.
Operator feature: a member gives a friend access with $5 of credits, no OC
membership. Cloudron user lands in the guests group (chat + dashboard only),
LiteLLM team capped at GIFT_BUDGET with NO monthly reset, portal row marked
account_type='gift' with gift_expires_at set, gift welcome email, newsletter
subscription, and a gifts ledger recording inviter -> recipient (one gift per
inviter per 30 days, one per recipient ever).
Sweep safety: reconcile_memberships never deactivates a gift for lacking an
OC contribution; reconcile_gift_lifecycles lapses gifts by their own rules
(credits spent out, or the GIFT_DAYS time box) with member-deactivation
semantics. Guests are excluded from Loomio/Matrix invite lists, the reinvite
path, budget notices, the allowance reconciliation and the metrics member
counts.
Conversion: a gift whose OC identity appears under its own email converts
automatically (webhook + sweep + /admin/provision all funnel through
provision_member): group guests->members, cap lifted to allowance + carried
credits, anchor day kept, congrats email, full member onboarding.
/admin/gift-convert links a membership paid under a different OC email
(members.oc_email, matched by the sweep and webhook).
Fixture test: tests/test_gift_lifecycle.py (37 checks, network replaced by
a raising guard).
The operator removed all caveats from the finance chart; the balance-history
note is gone from the API payload too so it cannot reappear anywhere.
balance_history_start, the series, totals, sources and errors are unchanged.
- /admin/money-series (X-Admin-Token, 30/min): three series for the admin
panel's Money chart, hourly in-memory cache with stale-on-error fallback.
revenue = OC contribution transactions (CREDIT/CONTRIBUTION/non-refund,
monthly + one-time, org-backed included), cumulative from 2026-09-06;
spend = one GROUP BY day pass over the full LiteLLM SpendLogs (no team
filter, same definition as metrics_snapshot.spend_by_day); balance = OC
current balance + daily snapshots.
- OC exposes only the CURRENT balance — no historical series exists — so
metrics_snapshot.py now writes oc_balance_usd into metrics_daily daily
(from 2026-11-08 onward; balance history cannot be backfilled). The
endpoint seeds an existing row's balance with the live value without
touching the snapshot's other columns.
- Aggregate totals only: no member emails or names selected or returned.
New members were skipped by the syncs until their Cloudron account was
activated (deliberately - never invite a half-created account), but nothing
fired the syncs at the moment they DID activate: the 3x/day sweep was the
only pickup, so a newly-activated member could wait hours for forum/space
access (limdingwen waited ~5h on 2026-11-06).
Hook the broker identity path (_verify_broker_secret): after an active
member resolves, if loomio_invited_at is still null, confirm their Cloudron
account is activated (inviteAccepted, fail-CLOSED - an unreadable user list
never fires) and run the same idempotent syncs the sweep runs, as a
background task off the response. Established members pay one indexed
SELECT; concurrent first-logins collapse into one run via an inflight flag;
each service is independently best-effort so a Loomio outage cannot block
the Matrix invite or the response.
Tests: tests/test_first_login_sync.py (18 checks, fixture db + stubbed
network). Also fix test_allowance_shortfall, whose expected exclusion
reference still asserted the pre-a88ca86 placeholder.
Sweep, provisioning and deactivation paths are untouched.
The deployed portal image (20261105-ca3b8db) was built from a working tree
carrying the real transaction id; commit 22c0167 over-redacted it to a
placeholder that matches nothing, so the next build would lose the exclusion
that prevents double-crediting the manually-remediated 2026-11-04 shortfall
charge. Sync HEAD to the deployed bytes (md5-identical to the running
container). The tx id alone is the H2-approved form: it identifies the OC
transaction, carries no member email.
allowance_log: the sweep and every report-mode run used to append a row
per member per run (~500/day measured). The insert is now an upsert per
(member, cycle, decision) — a re-derived decision updates the row's
evidence and timestamp; a decision CHANGE still gets its own row. No
index or migration; the latest-decision readers are unaffected; history
is not rewritten.
README: scheduler runs 3x/day (0 3,11,19 UTC), not nightly-3am; the
sweep's auth is the ADMIN_TOKEN X-Admin-Token header, not WEBHOOK_TOKEN;
the retired free/member/supporter tier budgets replaced by the actual
single MEMBER_BUDGET model; the unread OPENCOLLECTIVE_WEBHOOK_SECRET env
row removed.
/admin/librechat/keys deliberately serves every member's chat key on one
sync token (the LibreChat sandbox seeds from it; a per-member fetch would
require changing the sandbox's identity model — decision recorded in
ops-docs maintenance/librechat-key-feed.md). Every fetch now logs a
count-only WARNING (no key material; the email redactor strips addresses)
with caller + user-agent, and the daily digest watches the rate against
the sandbox's ~10-min poll cadence.
The _RedactWebhookPath filter now also strips email-shaped strings from
every log record — messages, %-format args (recursing into tuple/list/
dict args), and any exception text already attached to the record — the
same retention discipline the webhook token got in ops/security#2.
_send_email recipients, listmonk failures, Loomio invite lists and
provisioning/credit lines carry member addresses into container logs,
which persist on the host and ride the backups. MXIDs, counts, slugs and
amounts stay readable; the daily digest's own sanitisation is unchanged.
Filter keeps its never-break-logging guarantee (all redaction inside
try/except; non-string args untouched).
Live pre-state measured: 99 email-bearing lines in the current app log.
M6: a single-page ?per_page=100 fetch silently truncates past 100 users —
the box is at 99. New shared walkers (async + sync) page through
/api/v1/users like the Matrix paths, capped at 50 pages. All six single-
page call sites now walk: duplicate-email check (provisioning 409s),
reinvite loop, Loomio activation filter, admin overview, member-email
resolution, local MXID resolution. Failure behavior per site is preserved
(page-1 failure fails loudly; later-page failure keeps what was read).
M5: provisioning-time held-credit grant moved into grant_held_credits():
per hold, the status flip, balance move and ledger row commit in ONE
transaction (the old code committed the flip before add_credits, so an
exception between them left hold=granted and the member uncredited).
Mirrors admin_held_grant, keyed on held:<hold_id> so re-runs are no-ops;
mixed-case emails normalized. Tests in tests/test_portal_hardening.py.