Commit Graph
303 Commits
Author SHA1 Message Date
inference-bot 93184b6011 Fix the gift→guest rename migration for legacy databases
Two defects found by booting the renamed app against a fixture of the
pre-rename database (the live portal's exact state):

1. members.gift_expires_at: the RENAME above uses the stale
   snapshot, so the plain
   below fired right after a successful rename and crashed boot with
   'duplicate column name' — the portal would have failed its deploy
   health check on the live database. Make the two branches exclusive.

2. The ledger's own expiry column: ALTER TABLE guests RENAME moved the
   table but kept its legacy gift_expires_at column, which the provision
   INSERT (writing guest_expires_at) would fail on at first use. Rename
   the column with the table.

Also moves the credit-ledger dedupe references ('gift:<email>' /
'gift-return:<email>') to their guest prefixes at boot — returns first,
since 'gift-return:…' also matches 'gift:%' — so a re-grant after the
rename cannot double-credit.

Verified: migration harness 10/10 on a legacy-state fixture (renames,
value conversion, ref migration, idempotent second boot, readback
helpers); full portal suite 9/9 files green.
2026-11-08 02:42:53 +00:00
inference-bot 27f9cc06b2 Rename gift to guest throughout: routes, ledger table, account_type values, env names, emails, tests
Pure rename, no behaviour change, matching the renamed panel and dashboard
builds. Includes one-time idempotent migrations for existing databases
(gifts->guests table, gift_expires_at->guest_expires_at column,
account_type 'gift'->'guest' values, gift_lapse_notices->guest_lapse_notices)
and four transitional alias routes (admin/provision-gift, admin/gift-convert,
broker/gift-status, broker/cancel-gift) so pre-rename dashboard/panel builds
keep working until they are redeployed.
2026-11-08 02:41:05 +00:00
inference-bot 8af8e39c0b README: document the gift self-cancel flow and GUESTS_ENDED_GROUP_ID 2026-11-08 02:34:13 +00:00
inference-bot 526d98be17 manifest: pin to 20261108-40fb75a 2026-11-08 02:28:46 +00:00
inference-bot 40fb75a34b gift cancel: self-service cancellation with credit return and full audit trail
POST /broker/cancel-gift (dashboard broker identity, gift accounts only,
idempotent) + GET /broker/gift-status for the ended-state view. Semantics
match the gift lapse: row deactivated, account kept for a clean resume,
gateway budget zeroed, newsletter off — plus the return a lapse does not
do: unspent credit back to the co-op as a NEGATIVE credit_ledger row whose
reason names the cancellation and the original gift reference, deduped on
its own reference. Gifts ledger records inviter → recipient → 'cancelled
by recipient'. The user leaves the guests group for GUESTS_ENDED_GROUP_ID
(dashboard-only visibility for the ended state); the reaching-limits
notice is pre-marked so a self-cancelled guest is never told their
credits were spent. Tests: tests/test_gift_cancel.py (19 fixture checks,
network-guarded).
2026-11-08 02:28:45 +00:00
inference-bot 9c441dc0aa manifest: pin to the tag the builder produced (20261108-022314-976fc18c7) 2026-11-08 02:23:35 +00:00
inference-bot e704123f22 manifest: pin to 20261108-0b0bdaa 2026-11-08 02:23:04 +00:00
inference-bot 0b0bdaab6f gift emails: invitation at gift time + once-only reaching-limits notice
- gift_welcome_email becomes gift_invitation_email: the ONE email a gift
  sends (the invitation and the guest welcome merged, not two mails at
  once). Subject 'You've been invited to the Inference Cooperative'; names
  the inviter (member slug), the $5 credits, the setup link, the
  Become-a-member button on BECOME_MEMBER_URL. Uses the shared
  PINE_LOGO_SVG instead of an inline copy.
- reaching-limits email (gift_limits_email, 'Your guest access has ended —
  become a member to continue') wired into the sweep's gift deactivation:
  sent once per guest, gated by a gift_lapse_notices sent-state row (same
  pattern as mark_shortfall_notice_sent). Says the access ended, the
  account and data are kept, and membership ($15/mo via Open Collective)
  restores a full monthly allowance + community spaces.
- catch-up pass in reconcile_gift_lifecycles: guests deactivated before the
  email shipped (or a failed send) get exactly one notice on a later sweep;
  converted guests are excluded by account_type.
- sweep summary now reports gift_lapse_notices.
- test_gift_lifecycle: invitation-subject assertion; send_email stub
  returns True (the bool is the gate's contract); new once-only + catch-up
  + healthy-untouched checks. 43/43.
2026-11-08 02:22:59 +00:00
inference-bot 0bc3cd65e7 manifest: pin to 20261108-c6eb2b4 2026-11-08 02:09:35 +00:00
inference-bot c6eb2b44e8 overview: guests are their own list, excluded from member counts
/admin/overview now returns gift (guest) accounts in a separate guests[]
list instead of inside members[], and excludes them from totals.members,
totals.active and the member spend totals. Guests get their own totals
breakout (guests, guests_active, guest_spend, guest_all_time_spend) so
their spend stays visible without ever reading as member activity — the
same reasoning as the operator (admins) and agent categories.

Builds on the gift feature's account_type marker: a guest is any row with
account_type='gift'. The panel reads guests[] and ignores gift rows in
its member count either way.
2026-11-08 02:09:28 +00:00
inference-bot 103766eb74 manifest: pin to 20261108-6d9074e 2026-11-08 02:06:06 +00:00
inference-bot 6d9074eefe gift accounts: $5 guest passes with time box, inviter ledger, auto member conversion
Operator feature: a member gives a friend access with $5 of credits, no OC
membership. Cloudron user lands in the guests group (chat + dashboard only),
LiteLLM team capped at GIFT_BUDGET with NO monthly reset, portal row marked
account_type='gift' with gift_expires_at set, gift welcome email, newsletter
subscription, and a gifts ledger recording inviter -> recipient (one gift per
inviter per 30 days, one per recipient ever).

Sweep safety: reconcile_memberships never deactivates a gift for lacking an
OC contribution; reconcile_gift_lifecycles lapses gifts by their own rules
(credits spent out, or the GIFT_DAYS time box) with member-deactivation
semantics. Guests are excluded from Loomio/Matrix invite lists, the reinvite
path, budget notices, the allowance reconciliation and the metrics member
counts.

Conversion: a gift whose OC identity appears under its own email converts
automatically (webhook + sweep + /admin/provision all funnel through
provision_member): group guests->members, cap lifted to allowance + carried
credits, anchor day kept, congrats email, full member onboarding.
/admin/gift-convert links a membership paid under a different OC email
(members.oc_email, matched by the sweep and webhook).

Fixture test: tests/test_gift_lifecycle.py (37 checks, network replaced by
a raising guard).
2026-11-08 02:05:37 +00:00
inference-bot 73d8c61aa7 manifest: pin to 20261108-2d2d191 2026-11-08 01:55:29 +00:00
inference-bot 2d2d191e11 money-series: drop the caveat field from the payload
The operator removed all caveats from the finance chart; the balance-history
note is gone from the API payload too so it cannot reappear anywhere.
balance_history_start, the series, totals, sources and errors are unchanged.
2026-11-08 01:55:23 +00:00
inference-bot f44c4bb6bd manifest: pin to 20261108-dcce9fc 2026-11-08 01:50:18 +00:00
inference-bot dcce9fcb35 money series: /admin/money-series (revenue/spend/OC balance) + daily OC balance snapshot
- /admin/money-series (X-Admin-Token, 30/min): three series for the admin
  panel's Money chart, hourly in-memory cache with stale-on-error fallback.
  revenue = OC contribution transactions (CREDIT/CONTRIBUTION/non-refund,
  monthly + one-time, org-backed included), cumulative from 2026-09-06;
  spend = one GROUP BY day pass over the full LiteLLM SpendLogs (no team
  filter, same definition as metrics_snapshot.spend_by_day); balance = OC
  current balance + daily snapshots.
- OC exposes only the CURRENT balance — no historical series exists — so
  metrics_snapshot.py now writes oc_balance_usd into metrics_daily daily
  (from 2026-11-08 onward; balance history cannot be backfilled). The
  endpoint seeds an existing row's balance with the live value without
  touching the snapshot's other columns.
- Aggregate totals only: no member emails or names selected or returned.
2026-11-08 01:50:17 +00:00
inference-bot 3cc2fad1dd manifest: pin to 20261106-ed66d73 2026-11-06 19:51:34 +00:00
inference-bot ed66d73327 activation hook: fire Loomio+Matrix syncs at first broker login
New members were skipped by the syncs until their Cloudron account was
activated (deliberately - never invite a half-created account), but nothing
fired the syncs at the moment they DID activate: the 3x/day sweep was the
only pickup, so a newly-activated member could wait hours for forum/space
access (limdingwen waited ~5h on 2026-11-06).

Hook the broker identity path (_verify_broker_secret): after an active
member resolves, if loomio_invited_at is still null, confirm their Cloudron
account is activated (inviteAccepted, fail-CLOSED - an unreadable user list
never fires) and run the same idempotent syncs the sweep runs, as a
background task off the response. Established members pay one indexed
SELECT; concurrent first-logins collapse into one run via an inflight flag;
each service is independently best-effort so a Loomio outage cannot block
the Matrix invite or the response.

Tests: tests/test_first_login_sync.py (18 checks, fixture db + stubbed
network). Also fix test_allowance_shortfall, whose expected exclusion
reference still asserted the pre-a88ca86 placeholder.

Sweep, provisioning and deactivation paths are untouched.
2026-11-06 19:51:12 +00:00
inference-bot a88ca860d4 fix: restore the real OC tx reference in SHORTFALL_REMEDIATED (repo must match the deployed image)
The deployed portal image (20261105-ca3b8db) was built from a working tree
carrying the real transaction id; commit 22c0167 over-redacted it to a
placeholder that matches nothing, so the next build would lose the exclusion
that prevents double-crediting the manually-remediated 2026-11-04 shortfall
charge. Sync HEAD to the deployed bytes (md5-identical to the running
container). The tx id alone is the H2-approved form: it identifies the OC
transaction, carries no member email.
2026-11-05 18:55:15 +00:00
inference-bot 8720d623ea manifest: pin to 20261105-00f60f1 2026-11-05 16:26:16 +00:00
inference-bot 00f60f1374 ops: bound allowance_log growth; correct stale README claims (audit L4+L5)
allowance_log: the sweep and every report-mode run used to append a row
per member per run (~500/day measured). The insert is now an upsert per
(member, cycle, decision) — a re-derived decision updates the row's
evidence and timestamp; a decision CHANGE still gets its own row. No
index or migration; the latest-decision readers are unaffected; history
is not rewritten.

README: scheduler runs 3x/day (0 3,11,19 UTC), not nightly-3am; the
sweep's auth is the ADMIN_TOKEN X-Admin-Token header, not WEBHOOK_TOKEN;
the retired free/member/supporter tier budgets replaced by the actual
single MEMBER_BUDGET model; the unread OPENCOLLECTIVE_WEBHOOK_SECRET env
row removed.
2026-11-05 16:26:16 +00:00
inference-bot 1127b41ba3 manifest: pin to 20261105-ce1bc7b 2026-11-05 16:21:51 +00:00
inference-bot ce1bc7bee6 security: log every key-feed fetch (audit M2)
/admin/librechat/keys deliberately serves every member's chat key on one
sync token (the LibreChat sandbox seeds from it; a per-member fetch would
require changing the sandbox's identity model — decision recorded in
ops-docs maintenance/librechat-key-feed.md). Every fetch now logs a
count-only WARNING (no key material; the email redactor strips addresses)
with caller + user-agent, and the daily digest watches the rate against
the sandbox's ~10-min poll cadence.
2026-11-05 16:21:51 +00:00
inference-bot 8a9222033d manifest: pin to 20261105-d63f702 2026-11-05 16:17:21 +00:00
inference-bot d63f702ec3 privacy: redact member emails from portal logs (audit M1)
The _RedactWebhookPath filter now also strips email-shaped strings from
every log record — messages, %-format args (recursing into tuple/list/
dict args), and any exception text already attached to the record — the
same retention discipline the webhook token got in ops/security#2.
_send_email recipients, listmonk failures, Loomio invite lists and
provisioning/credit lines carry member addresses into container logs,
which persist on the host and ride the backups. MXIDs, counts, slugs and
amounts stay readable; the daily digest's own sanitisation is unchanged.
Filter keeps its never-break-logging guarantee (all redaction inside
try/except; non-string args untouched).

Live pre-state measured: 99 email-bearing lines in the current app log.
2026-11-05 16:17:21 +00:00
inference-bot 2b501b75be manifest: pin to 20261105-b9561a4 2026-11-05 16:06:52 +00:00
inference-bot b9561a49d2 correctness: paginate Cloudron user fetches; atomic held-credit grant (audit M5+M6)
M6: a single-page ?per_page=100 fetch silently truncates past 100 users —
the box is at 99. New shared walkers (async + sync) page through
/api/v1/users like the Matrix paths, capped at 50 pages. All six single-
page call sites now walk: duplicate-email check (provisioning 409s),
reinvite loop, Loomio activation filter, admin overview, member-email
resolution, local MXID resolution. Failure behavior per site is preserved
(page-1 failure fails loudly; later-page failure keeps what was read).

M5: provisioning-time held-credit grant moved into grant_held_credits():
per hold, the status flip, balance move and ledger row commit in ONE
transaction (the old code committed the flip before add_credits, so an
exception between them left hold=granted and the member uncredited).
Mirrors admin_held_grant, keyed on held:<hold_id> so re-runs are no-ops;
mixed-case emails normalized. Tests in tests/test_portal_hardening.py.
2026-11-05 16:06:52 +00:00
inference-bot 0f32c3a868 manifest: pin to 20261105-22c0167 2026-11-05 15:59:07 +00:00
inference-bot 22c0167978 privacy: de-PII the shortfall exclusion and policy comment (audit H2)
SHORTFALL_REMEDIATED is keyed on the OC charge reference alone (the same
idempotence key the shortfall path uses); the member's identity and the
timeline of the 2026-11-04 manual allowance reset move to the operations
docs and stay in the ledger. The policy comment that named a member's
join/charge dates is generalized. No behavioral change: the exclusion
matches the same charge it always matched.
2026-11-05 15:59:07 +00:00
inference-bot f0d67e7efa manifest: pin to 20261105-e6ab177 2026-11-05 15:54:34 +00:00
inference-bot e6ab177a05 credits: make the credit-pack webhook grant idempotent (audit H1)
- reference built per-order from the webhook's order.idV2 (the id the
  amount lookup resolves) instead of the absent data.id that produced
  the legacy empty-suffix 'oc:<slug>:' format; legacy ledger rows are
  neither re-keyed nor rejected (a new-format reference can never match
  one)
- credit_reference_exists() gates add_credits; a re-delivered order
  returns 'already_credited' with the standing balance
- hold path: one hold per purchase via a pre-check plus a UNIQUE index
  on held_credits.oc_reference (index creation logs CRITICAL and
  continues if the live table ever carried duplicates — audited: none)
- get_order_amount sends the webhook-supplied order id as a GraphQL
  variable instead of interpolating it into the query (audit L9)
- regression test: replay credits exactly once; legacy rows never
  block a first grant; unique-index semantics incl. NULLs
2026-11-05 15:54:29 +00:00
inference-bot 48128ab97b manifest: pin to 20261105-6337bcf 2026-11-05 03:48:17 +00:00
inference-bot 6337bcfc24 portal: don't double the oc: prefix in the held-credit grant reason 2026-11-05 03:48:01 +00:00
inference-bot 11a74d955e manifest: pin to 20261105-e103e67 2026-11-05 03:47:21 +00:00
inference-bot e103e6779b portal: add admin endpoint to grant a held credit pack to a member
Held credit packs (purchases by non-members) could previously only be
refunded or auto-granted when the SAME email/slug later became a member.
A purchase made on another account's behalf therefore had no admin path
and forced a hand-edit of money state.

Add POST /admin/held-credits/{hold_id}/grant with body {email}: applies
the hold to a specified member account in one transaction (balance +
ledger + hold -> granted), idempotent on reference held:<hold_id>.
2026-11-05 03:45:14 +00:00
inference-bot ed1e361b14 manifest: pin to 20261104-db64dac 2026-11-04 20:01:28 +00:00
inference-bot db64dac235 allowances: credit the shortfall when OC's billing day truncates a paid cycle
Open Collective aligns recurring charges onto the collective's common billing
day, so a mid-month joiner is charged again before a full month is up: they pay
for a full month and receive less than one. Credit the missing ALLOWANCE (the
flat MEMBER_BUDGET pro-rated for the days lost, unrounded) as non-expiring
credits.

- derived from the collective's real per-order charge history, on every sweep;
  any consecutive pair on the same order < a full month apart is truncated
- idempotent on the OC charge reference (dedupe on the charge, not the member)
- skips and logs any cycle whose dates cannot be determined
- ledger reason names the cycle; the notice is staged and defaults OFF
- [redacted member reference]'s remediated charge is excluded by ref

Tests: tests/test_allowance_shortfall.py (pure derivation).
2026-11-04 20:01:10 +00:00
inference-bot c468b61d3e manifest: pin to 20261104-f7bbec8 2026-11-04 19:26:04 +00:00
inference-bot f7bbec8a0a loomio: enable by default; idempotent add (never re-invite) + targeted remove
The guard direction from 2026-11-04 is corrected: the co-op DOES use Loomio and
wants every new member added. Removing members/notifications is not the goal —
not duplicating them is. The feature is switched back ON and the sync is made
idempotent instead.

Enablement
  * LOOMIO_SYNC_ENABLED now defaults to ENABLED (kill switch: set =0 to pause).
    Restoring LOOMIO_API_KEY and deploying is all that is needed again.

Non-duplicating ADD (the real fix), two layers:
  * (a) live pre-check against Loomio group membership (b3 index, ordered and
    paginated stably). An address already in the group is skipped SILENTLY —
    it is never placed in an invite payload, so no API call about it can fire
    a notification or email.
  * (b) members.loomio_invited_at: an address we have handed over before is
    never sent again, even if it is not (yet) visible in the live roster.
  * FAIL CLOSED if the live roster cannot be read.
  * remove_absent is never sent.

Removal reconciliation preserved (deactivation must still propagate):
  * A member in the group who is no longer active is removed via a TARGETED
    b3 DELETE by membership id — it names only the member being removed, so it
    cannot re-invite or notify anyone staying. Operator allowlist
    (LOOMIO_ALWAYS_KEEP) is never removed. 404 (already gone) counts as
    success, so a re-run neither errors nor churns.
  * The keep-set is computed from the UNFILTERED active list: an active member
    whose Cloudron account is not yet activated is not removed (only their
    invite is deferred).

Tests: tests/test_loomio_guard.py (26/26) — add/dedupe, silent skip, record
guard, fail-closed, remove-once, allowlist-never-removed, targeted removal
(no present member in any payload), double-run no-op for both phases, 404
tolerance, kill switch.
2026-11-04 19:25:45 +00:00
inference-bot 17329c5d6f manifest: pin to 20261104-120a246 2026-11-04 19:09:33 +00:00
inference-bot 120a24678d loomio: never invite an existing member (off by default + positive exclusion)
Incident 2026-11-04: a live run of sync_loomio_memberships() emailed a Loomio
invite notification to essentially the WHOLE membership (notification 744:
~83 in-app recipients, ~79 emails 'sent' 18:53-18:56 UTC). The sync handed
Loomio the full active-member list every run with no record of who had been
invited, relying on Loomio's add-if-absent semantics — which produced a
whole-group notification.

Guards now make a repeat impossible:
  * LOOMIO_SYNC_ENABLED (off by default) — restoring LOOMIO_API_KEY alone no
    longer re-arms the integration.
  * members.loomio_invited_at — positive exclusion: an address is sent only if
    it is neither in Loomio's current member list NOR ever handed over, and
    each is recorded once. (The new column starts NULL for the ~89 members
    already in Loomio, so the current-member check is what closes that gap.)
  * FAIL CLOSED — no invite if Loomio's member list cannot be read.
  * remove_absent is dropped: no evict-then-re-add (re-invite) path.

Tests: tests/test_loomio_guard.py (10/10).
2026-11-04 19:09:33 +00:00
inference-bot 947e4d3cf2 manifest: pin to 20261104-8c85c91 2026-11-04 18:47:18 +00:00
inference-bot 8c85c912e8 allowances: add a would_deactivate PREVIEW (deactivation stays gated off)
Report mode (and a send run while ALLOWANCE_DEACTIVATE is off) now reports a
member whose reminder is past the grace window as would_deactivate, with the
reminder date, grace length and deadline in the evidence. The real action is
unchanged and still needs BOTH ALLOWANCE_POLICY=on AND ALLOWANCE_DEACTIVATE=on.

The lapse decision is extracted into a pure function (allowance_lapse_decision)
so it is unit-testable against a synthetic reminder time:
tests/test_allowance_lapse.py (would_deactivate past grace; in_grace inside).
2026-11-04 18:46:54 +00:00
inference-bot ee7ed77697 manifest: pin to 20261104-f1945aa 2026-11-04 18:31:06 +00:00
inference-bot f1945aaf53 admin overview: expose derived membership state (active / in_grace / awaiting_contribution / inactive)
The admin panel needs to show a member's membership state without inventing
policy of its own. Compute it in the portal (the source of truth) from what the
allowance reconciliation already knows — the payment_reminders send record and
the latest allowance_log decision — and carry it on each /admin/overview row as
state + state_reason. Read-only; no side effects.
2026-11-04 18:30:42 +00:00
inference-bot 97da432bd4 manifest: pin to 20261104-a8200fd 2026-11-04 17:31:33 +00:00
inference-bot a8200fdf54 allowances: correct the dormant-yearly note (collective order list is complete; a member's account.orders spans collectives) 2026-11-04 17:31:11 +00:00
inference-bot 36b7c2f77e manifest: pin to 20261104-a984a44 2026-11-04 17:30:09 +00:00
inference-bot a984a443e1 allowances: recognise YEARLY contributions so a yearly payer isn't read as unpaid
_payment_verdict treated MONTHLY as the only recurring frequency, so a live
ACTIVE YEARLY contribution — one charge, current for twelve cycles — fell
through to "no recurring (monthly) contribution" and would have been reminded
and (with deactivation on) deactivated for eleven of those months.

Make the verdict frequency-aware: only a MONTHLY or YEARLY order can cover an
allowance cycle; an ACTIVE YEARLY covers while its window (last charge + 1yr,
or nextChargeDate) includes now; PROCESSING stays paid-pending; a ONE-TIME
payment is credit, never an allowance period and never a mask. A failed
monthly charge still takes the remind path.

Dormant on today's data: inference-cooperative has 0 yearly orders (104 orders:
88 MONTHLY, 16 ONETIME), verified per member ACCOUNT because a member's own
yearly order does not appear in the collective-level order list. This changes
no current member's decision — the report is identical before and after.

Adds tests/test_payment_verdict.py (yearly branch + monthly/processing/onetime/
error/cancelled cases), runnable in the app container.
2026-11-04 17:29:47 +00:00
inference-bot 86e77a0ad4 manifest: pin to 20261104-ced8c4f 2026-11-04 05:19:58 +00:00