metrics: complete YESTERDAY's row at 03:15 — every day was frozen at its first ~3 hours
ci/woodpecker/push/woodpecker Pipeline was successful
ci/woodpecker/push/woodpecker Pipeline was successful
snapshot() fetched the full spend window (since = yesterday) but kept only spend.get(TODAY): the day's first hours, written at 03:15 and never revisited. Every metrics_daily row after the last backfill under-counted its day by the hours after 03:15 (Nov 7 recorded $2.19/691 requests against a true $29.06/3384). The completed day's aggregates were computed and thrown away. Now each run completes the day that just ENDED — full-day spend, requests, tokens and impact, written to the usage columns only (upsert_usage: membership columns are point-in-time measurements and are never rewritten) — and then records today's live partial as before. backfill() gains --since YYYY-MM-DD: a targeted usage-only repair that cannot touch the published membership curve. Regression test: tests/test_metrics_completion.py (day completion, membership preservation, since-mode repair boundaries, full-backfill parity).
This commit is contained in:
1 parent
022e193cc2
commit
cf3dbb7489
4 files changed
+365
-61
No files matched your search
+10
-1
@@ -155,10 +155,19 @@ than by us.
|
||||
`members_total`, `members_new`, `newsletter_subs`, `matrix_space_members`,
|
||||
`spend_usd`, `requests`, `out_tokens`, `energy_wh`, `co2e_g`, `tokens_measured`,
|
||||
`tokens_estimated`, `factor_revision`, `recorded_at`.
|
||||
- **Each 03:15 run completes YESTERDAY, then records today live.** The day that
|
||||
just ended is rewritten with its FULL spend/requests/tokens/impact aggregates
|
||||
(usage columns only — the membership columns are point-in-time measurements
|
||||
that are not reconstructible afterwards). Today's row carries the first hours
|
||||
of live data and is completed by the next run. (Until 2026-11-12 the run
|
||||
wrote only today's partial and never revisited it, which froze every day at
|
||||
its first ~3 hours — the November under-count.)
|
||||
- **`metrics_snapshot.py --backfill` rebuilds history** from the sources and is
|
||||
idempotent — it rewrites a day rather than duplicating it. It is the tool to run
|
||||
after changing a factor, and the reason a published chart can move: history is
|
||||
**rewritten**, not appended to.
|
||||
**rewritten**, not appended to. `--backfill --since YYYY-MM-DD` limits the
|
||||
rewrite to days from that date and touches ONLY the usage/impact columns, so
|
||||
a targeted repair of recent days cannot alter the published membership curve.
|
||||
|
||||
## What is measured, and what is not
|
||||
|
||||
|
||||
Reference in new issue
Block a user