Commit Graph
3 Commits
Author SHA1 Message Date
inference-bot cf3dbb7489 metrics: complete YESTERDAY's row at 03:15 — every day was frozen at its first ~3 hours
ci/woodpecker/push/woodpecker Pipeline was successful
snapshot() fetched the full spend window (since = yesterday) but kept only
spend.get(TODAY): the day's first hours, written at 03:15 and never revisited.
Every metrics_daily row after the last backfill under-counted its day by the
hours after 03:15 (Nov 7 recorded $2.19/691 requests against a true
$29.06/3384). The completed day's aggregates were computed and thrown away.

Now each run completes the day that just ENDED — full-day spend, requests,
tokens and impact, written to the usage columns only (upsert_usage: membership
columns are point-in-time measurements and are never rewritten) — and then
records today's live partial as before.

backfill() gains --since YYYY-MM-DD: a targeted usage-only repair that cannot
touch the published membership curve.

Regression test: tests/test_metrics_completion.py (day completion, membership
preservation, since-mode repair boundaries, full-backfill parity).
2026-11-12 18:40:31 +00:00
portal-dev 6475a51067 metrics: expose + persist measured-vs-estimated provenance (#4, #6)
Thread the per-factor source through impact_for() instead of discarding it.
Numeric output (energy_wh, co2e_g) is byte-identical — this is a reporting
change, not a recalculation.

#6 persistence: additive columns tokens_measured, tokens_estimated and
factor_revision (a stable hash of the factor table) on metrics_daily, added via
the existing guarded ALTER TABLE idiom in both app/main.py and
metrics_snapshot.py. Existing rows keep NULL.

#4 API: metrics_series() selects the new columns and derives a per-row
provenance indicator (measured | mixed | estimated | none) from the split; both
/admin/metrics/series and /broker/metrics return it. NULL/unknown splits report
'none', never a confident 'measured'.

docs/energy-metrics.md documents the indicator, the new columns, and the
forward-only-vs-backfill behaviour.
2026-10-30 19:12:20 +00:00
inference-bot b11552fa49 docs: write down how the energy/CO2 metrics actually work (#3)
The calculation lived only in code comments. Adds docs/energy-metrics.md and wires it
into the README, per the issue's proposed scope. No calculation changes.

Contents: the input (completion_tokens, output tokens only, grouped on model_group)
and why output-only understates total compute; the factor table with its three
provenance tiers (measured / same-family / estimated) and the fact that the tiers are
never exposed; the formulas; NULL-not-zero and the reason (the first backfill claimed
every day used no energy); the implied-grid-intensity tripwire and the 1000x carbon
error it exists to catch; the 03:15 schedule and that --backfill REWRITES history.

States plainly what is measured and what is not: GreenPT is measured, glm-5.3 is
assumed from its flash sibling, and all Tinfoil and Regolo traffic currently falls to
the proxy estimate — with the consequence that the 250 gCO2e/kWh proxy OVERSTATES
Regolo, which runs on renewable energy.

README: "Two jobs" -> "Three", the metrics_snapshot task documented, and a short
Energy & carbon metrics subsection linking the new doc. Also corrects "listing both
tasks" in the post-change verification step.
2026-10-29 20:15:10 +00:00