Commit Graph
308 Commits
Author SHA1 Message Date
inference-bot 97da432bd4 manifest: pin to 20261104-a8200fd 2026-11-04 17:31:33 +00:00
inference-bot a8200fdf54 allowances: correct the dormant-yearly note (collective order list is complete; a member's account.orders spans collectives) 2026-11-04 17:31:11 +00:00
inference-bot 36b7c2f77e manifest: pin to 20261104-a984a44 2026-11-04 17:30:09 +00:00
inference-bot a984a443e1 allowances: recognise YEARLY contributions so a yearly payer isn't read as unpaid
_payment_verdict treated MONTHLY as the only recurring frequency, so a live
ACTIVE YEARLY contribution — one charge, current for twelve cycles — fell
through to "no recurring (monthly) contribution" and would have been reminded
and (with deactivation on) deactivated for eleven of those months.

Make the verdict frequency-aware: only a MONTHLY or YEARLY order can cover an
allowance cycle; an ACTIVE YEARLY covers while its window (last charge + 1yr,
or nextChargeDate) includes now; PROCESSING stays paid-pending; a ONE-TIME
payment is credit, never an allowance period and never a mask. A failed
monthly charge still takes the remind path.

Dormant on today's data: inference-cooperative has 0 yearly orders (104 orders:
88 MONTHLY, 16 ONETIME), verified per member ACCOUNT because a member's own
yearly order does not appear in the collective-level order list. This changes
no current member's decision — the report is identical before and after.

Adds tests/test_payment_verdict.py (yearly branch + monthly/processing/onetime/
error/cancelled cases), runnable in the app container.
2026-11-04 17:29:47 +00:00
inference-bot 86e77a0ad4 manifest: pin to 20261104-ced8c4f 2026-11-04 05:19:58 +00:00
inference-bot ced8c4f082 email: set an explicit Message-ID on outbound mail so a send is traceable 2026-11-04 05:19:43 +00:00
inference-bot 75959e6045 manifest: pin to 20261104-5237e63 2026-11-04 05:18:54 +00:00
inference-bot 5237e6352b allowances: add mode=test to /admin/allowance to verify a reminder really sends 2026-11-04 05:18:39 +00:00
inference-bot f68e4fb652 manifest: pin to 20261104-a8bdb70 2026-11-04 05:18:09 +00:00
inference-bot a8bdb70e08 allowances: fix decision-label mapping (would_renew not would_renewed) 2026-11-04 05:17:54 +00:00
inference-bot 326fec9214 manifest: pin to 20261104-d878c32 2026-11-04 05:17:26 +00:00
inference-bot d878c32d68 allowances: restore fetch_member_orders/_payment_verdict/payment_reminder_email dropped in the slug fix 2026-11-04 05:17:12 +00:00
inference-bot ea4660690d manifest: pin to 20261104-f8b3eaa 2026-11-04 05:16:46 +00:00
inference-bot f8b3eaacd6 allowances: resolve OC slug from the member's email, not the stored column
The first report run matched orders on members.slug, which holds a stale
guest-… slug from first provisioning for 18 members — every one of them read
as unpaid while actually being current payers. Resolve the live slug from the
member's email via the OC member list, refresh the stored slug, and treat
members absent from the OC member list as unverifiable (skip, never punish).
2026-11-04 05:16:32 +00:00
inference-bot 985b55f6d9 manifest: pin to 20261104-ccce643 2026-11-04 05:15:41 +00:00
inference-bot ccce643b39 allowances: payment-driven renewal anchored to the member's join day
Replaces the payment-date re-anchoring that let Open Collective's common
billing day move a member's renewal and skip a cycle ([redacted member reference]:
[redacted timeline].

- Reconcile against OC orders directly on the sweep's schedule: a live
  recurring contribution (ACTIVE/PROCESSING) renews; otherwise one reminder
  email, then deactivation only after PAYMENT_GRACE_DAYS.
- Anchor each member's reset to their join day (members.anchor_day), not the
  payment date; the webhook now only confirms the cycle.
- Decision log (allowance_log) records renewed/reminded/deactivated with the
  payment evidence.
- Report-only by default (ALLOWANCE_POLICY=report); deactivation gated
  separately by ALLOWANCE_DEACTIVATE=off.
- POST /admin/allowance?mode=report|send.
2026-11-04 05:15:24 +00:00
inference-bot 5da1981823 manifest: pin to 20261104-a89c464 2026-11-04 01:05:43 +00:00
inference-bot a89c464834 sync: retire identities from all invite rooms, not just Space+General
The retirement half of sync_matrix_space() iterated [MATRIX_SPACE_ID] + the
General room while the invite half used member_invite_rooms() (MATRIX_INVITE_ROOMS,
7 rooms including Skillshare). A retired local identity was therefore left in
Design, Community, Infrastructure, Updates and Skillshare — a privacy problem in
member-only rooms. Iterate member_invite_rooms() in both halves so they cannot
drift again. Retirement logic (kick joined / revoke invite-state) is unchanged.
2026-11-04 01:05:41 +00:00
inference-bot d5b4a2e03a manifest: pin to 20261103-c53112f
The registry builder is working again (the build pushed cleanly), so
restore the dockerImage pin the portal dropped in f45ca19. Tag carries
the Requests-count fix in app/main.py (commit c53112f).
2026-11-03 23:40:33 +00:00
inference-bot c53112fbaf model usage: Requests counts only successful calls
The Requests column summed every spend-log row (COUNT(*)), so a model's
call count included refused/failed attempts — e.g. Metis showed 26 calls
when only 16 succeeded (the other 10 were 401s on /v1/systemone).

Count only rows whose status is not 'failure', and report the refused
count separately as 'refused' (per model and in totals) so a failure
cannot inflate use without disappearing from view. spend and tokens stay
summed over all rows: a call that failed after consuming tokens still
cost money.

Fixes the panel and dashboard model tables, which both read this
aggregation (panel via /admin/model-usage, dashboard via
/broker/model-leaderboard).
2026-11-03 23:39:01 +00:00
inference-bot 8bcb7ff049 sync: invite members to configured MATRIX_INVITE_ROOMS (Skillshare)
sync_matrix_space() now invites every active member to the rooms in
member_invite_rooms(): MATRIX_INVITE_ROOMS when set, else the co-op Space +
General room; legacy MATRIX_EXTRA_ROOMS is still appended. The set is explicit
configuration, never "every child of the Space", so operator-only rooms are not
pushed at members. Skillshare (!NSPfCdVP7TWTtFoAz1TxVOcqnoDIpMoJDiPb_XS79ys)
was missing from the invite set, so no member who joined after it was created
was ever told about it; including it here makes the nightly sweep self-heal.
2026-11-02 19:51:11 +00:00
inference-bot ea648aab95 README: document operator_spend_usd in the metrics snapshot 2026-11-02 15:51:27 +00:00
inference-bot f45ca199cf manifest: build on-server (drop the registry image pin)
The remote registry builder is currently unusable from this Cloudron host:
its in-build `docker login registry.inference.coop` resolves the registry to
127.0.0.1:443 and is refused, so no new image can be pushed to the registry
(the box's own docker CAN reach the registry — a pull of a missing tag returns
a clean 404 — so this is the builder's container network, not the registry).
The panel and dashboard are already unpinned and build on-server via
`cloudron update` from the repo dir; the portal joins them so its code can ship
the operator-spend breakout the dashboard reads. Restore the pin when the
builder is fixed.
2026-11-02 15:49:11 +00:00
inference-bot 8fc670b240 manifest: pin to 20261102-5f8bed3 2026-11-02 15:42:38 +00:00
inference-bot 5f8bed33e6 metrics: break out the co-op's own (operator) spend in the daily series
metrics_daily.spend_usd is the co-op-wide total and deliberately spans every
team, operators included — the numbers were already right. What was missing
was a way to tell how much of that total is the co-op's OWN usage, so a
member-facing chart could not be misread as member-only activity.

Adds metrics_daily.operator_spend_usd: the per-day portion of spend_usd that
belongs to the accounts in OPERATOR_EMAILS (teams aliased member:<email>, like
anyone else). NULL = not measured (never a confident 0), mirroring spend_usd.

- metrics_snapshot.py: operator_spend_by_day() + write it on snapshot/backfill
- app/main.py: expose it in metrics_series (so /broker/metrics and
  /admin/metrics/series carry it), and add operator_spend /
  operator_all_time_spend to /admin/overview totals as the explicit breakout
  of total_spend / total_all_time_spend.

spend_usd and every existing column are untouched.
2026-11-02 15:42:30 +00:00
InferenceBot ddeefdebe0 manifest: pin to 20261101-6a7eaf0 2026-11-01 18:00:30 +00:00
InferenceBot 6a7eaf0ff8 broker: per-key spend (chat separate) in /broker/usage
Members need to see how much each API key has spent. LiteLLM's per-key
'spend' counter is cycle-only (resets at budget_reset_at), so derive it
from the LiteLLM_SpendLogs table instead: sum spend grouped by api_key,
matching rows to member_api_keys.hash by name and labelling unmatched
rows 'chat' (the portal-held chat key is never in member_api_keys).

broker_get_usage now returns key_spend: [{name, is_chat, spend, tokens,
requests}], API keys by spend desc with the chat row last.
2026-11-01 18:00:25 +00:00
inference-bot b065b7301f manifest: correct the registry path in the pin
I had rewritten co-op/member-portal to code/member-portal while bumping the tag. The repository path in a live pin is the registry location the image is stored and fetched from; changing it points Cloudron at a repo that does not hold this app. Only the tag should move.
2026-11-01 17:17:13 +00:00
inference-bot abdc56e84c manifest: pin to 20261101-e1ec25f
Tag names the commit it was computed from, after committing the agent all-time-spend change.
2026-11-01 17:17:00 +00:00
inference-bot e1ec25f1e4 admin overview: give agent teams an all-time spend, not just the cycle counter
The team spend counter is cycle-only and resets at budget_reset_at, which for the agent teams means the 1st of the month. The panel's agents table read that field directly, so on 1 November the co-op's own bot accounts appeared to have had their spend zeroed out. Members already had an all-time figure for exactly this reason; agent teams did not, because the all-time query filtered to member:% only. Widen it to agent:% and expose all_time_spend on agent rows.
2026-11-01 17:16:33 +00:00
inference-bot 5beffaf7d5 manifest: pin to 20261031-6f6957c 2026-10-31 16:58:03 +00:00
inference-bot 6f6957c779 portal: budget notice copy — drop the reassurance, button says 'Buy credit packs'
Removes 'nothing else about your account changes' from both the body and the footer; it goes without saying. Button label becomes 'Buy credit packs', and the sentence above it says the same thing so the two do not disagree.
2026-10-31 16:57:40 +00:00
inference-bot d2b6f1551f manifest: pin to 20261031-b998d8d 2026-10-31 16:53:50 +00:00
inference-bot b998d8d535 portal: budget notices gain mode=test (email one sample, no ledger row)
Lets the letter be read in a mail client before the sender is armed. Uses the member's real numbers when the address matches one, writes nothing to budget_notices so the once-per-crossing latch is not consumed, and works while BUDGET_NOTICES is off.
2026-10-31 16:53:22 +00:00
inference-bot c21421a89b manifest: pin to 20261031-7d2507a 2026-10-31 16:50:01 +00:00
inference-bot 7d2507a927 portal: 90% threshold for budget notices; sweep 3x a day
Trigger becomes proportional (90% of the balance used) instead of a $2 floor, so a member on $15 and one on $35 are treated the same. The once-per-crossing latch keys on the same fraction. membership_sweep moves from '0 3 * * *' to '0 3,11,19 * * *' so the notice arrives within hours of crossing rather than up to a day later. BUDGET_NOTICES remains off.
2026-10-31 16:49:22 +00:00
inference-bot 26be6156ae manifest: pin to 20261031-d5fbb2e (previous commit dropped the date prefix and 404ed the pull) 2026-10-31 16:44:23 +00:00
inference-bot bb864055e6 manifest: pin to d5fbb2e (computed before the commit again — the code commit is 49e79aa) 2026-10-31 16:35:48 +00:00
inference-bot d5fbb2ed84 portal: budget notice -> generic low-balance wording, $2 floor, username, once per crossing
Wording no longer names an allowance or a credit pack: it says balance, so it is true whether the member is short on allowance, on purchased credit, or both. Trigger is an absolute $2 floor on remaining budget rather than a percentage, so it means the same at any budget size. Recipient greeting uses members.slug (the username) instead of the email local part, which produced 'Hi n,'. The once-only latch now clears when the balance rises back above the floor, so it fires once per crossing rather than once per budget cycle. BUDGET_NOTICES stays off.
2026-10-31 16:35:35 +00:00
inference-bot 49e79aadcc manifest: pin to 20261031-87e7c5b (the earlier pin named the pre-commit hash) 2026-10-31 16:25:51 +00:00
inference-bot 87e7c5b30e portal: budget notices by email, off by default
Warns a member at 80% of their monthly cap and again when it is exhausted, using the welcome email's template shell. Spend and budget come from LiteLLM's Postgres, the same source the dashboard shows. One notice per level per budget cycle. BUDGET_NOTICES=off|dryrun|on gates sending; POST /admin/budget-notices previews at any time and refuses to send unless the flag is on. Rides the existing sweep cron, wrapped so it cannot break reconciliation.
2026-10-31 16:25:36 +00:00
inference-bot 76d59b3fb6 manifest: pin dockerImage to 20261030-6475a51 2026-10-30 19:22:05 +00:00
portal-dev 6475a51067 metrics: expose + persist measured-vs-estimated provenance (#4, #6)
Thread the per-factor source through impact_for() instead of discarding it.
Numeric output (energy_wh, co2e_g) is byte-identical — this is a reporting
change, not a recalculation.

#6 persistence: additive columns tokens_measured, tokens_estimated and
factor_revision (a stable hash of the factor table) on metrics_daily, added via
the existing guarded ALTER TABLE idiom in both app/main.py and
metrics_snapshot.py. Existing rows keep NULL.

#4 API: metrics_series() selects the new columns and derives a per-row
provenance indicator (measured | mixed | estimated | none) from the split; both
/admin/metrics/series and /broker/metrics return it. NULL/unknown splits report
'none', never a confident 'measured'.

docs/energy-metrics.md documents the indicator, the new columns, and the
forward-only-vs-backfill behaviour.
2026-10-30 19:12:20 +00:00
inference-bot e5c6b6060e manifest: pin dockerImage to 20261029-9f198fc 2026-10-29 21:20:19 +00:00
inference-bot 9f198fcf4c overview: exclude sandbox teams from the agents list, and name agents for display
sandbox:librechat-test is a throwaway test account rather than an agent of the co-op,
so it drops out of the panel. Its spend then appears nowhere — accepted knowingly.

Agents also carry a display `name` (the alias minus its "agent:" prefix), so the panel
does not have to strip prefixes itself.
2026-10-29 21:19:21 +00:00
inference-bot a0040f52e0 manifest: pin dockerImage to 20261029-592cf83 2026-10-29 20:57:38 +00:00
inference-bot 592cf83656 metrics + overview: stop counting utility accounts as members, and surface agent usage
The panel has always excluded operator/utility emails from member counts
(OPERATOR_EMAILS), but the metrics counted every row in the members table, so the
panel said 75 while every metric surface said 76. The one affected row is
info@inference.coop — the reading mailbox, not a person.

- metrics_snapshot.py now applies the same OPERATOR_EMAILS definition (same variable
  name, same default) so the two cannot drift apart again. No backfill: history stays
  as published and the daily job writes the corrected value going forward.
- admin_overview now also returns an `agents` list, built from the LiteLLM teams it
  already fetches (anything not member:*), so bot and sandbox spend is visible rather
  than structurally invisible. Kept out of member counts and member aggregates, like
  the operator category.
2026-10-29 20:57:10 +00:00
inference-bot b11552fa49 docs: write down how the energy/CO2 metrics actually work (#3)
The calculation lived only in code comments. Adds docs/energy-metrics.md and wires it
into the README, per the issue's proposed scope. No calculation changes.

Contents: the input (completion_tokens, output tokens only, grouped on model_group)
and why output-only understates total compute; the factor table with its three
provenance tiers (measured / same-family / estimated) and the fact that the tiers are
never exposed; the formulas; NULL-not-zero and the reason (the first backfill claimed
every day used no energy); the implied-grid-intensity tripwire and the 1000x carbon
error it exists to catch; the 03:15 schedule and that --backfill REWRITES history.

States plainly what is measured and what is not: GreenPT is measured, glm-5.3 is
assumed from its flash sibling, and all Tinfoil and Regolo traffic currently falls to
the proxy estimate — with the consequence that the 250 gCO2e/kWh proxy OVERSTATES
Regolo, which runs on renewable energy.

README: "Two jobs" -> "Three", the metrics_snapshot task documented, and a short
Energy & carbon metrics subsection linking the new doc. Also corrects "listing both
tasks" in the post-change verification step.
2026-10-29 20:15:10 +00:00
inference-bot 42cf43a125 manifest: pin dockerImage to 20261029-42bdfa2 2026-10-29 04:19:33 +00:00
inference-bot 42bdfa25d3 portal: record that the admin-token cutover is complete (comment only)
The comment above ADMIN_TOKEN still said "set ADMIN_TOKEN_LEGACY=0 and rotate
both". Stale — and the direct reason a report claimed a rotation was
outstanding: ADMIN_TOKEN was minted new and WEBHOOK_TOKEN rotated on 2026-10-27,
and the legacy path was closed on 2026-10-28.

Comment only, no behaviour change. Rebranched onto the current main: an earlier
attempt had pinned an image built from a tree that predated the key-encryption
work, which would have reverted it.
2026-10-29 04:19:17 +00:00