metrics: complete YESTERDAY's row at 03:15 — every day was frozen at its first ~3 hours
ci/woodpecker/push/woodpecker Pipeline was successful
ci/woodpecker/push/woodpecker Pipeline was successful
snapshot() fetched the full spend window (since = yesterday) but kept only spend.get(TODAY): the day's first hours, written at 03:15 and never revisited. Every metrics_daily row after the last backfill under-counted its day by the hours after 03:15 (Nov 7 recorded $2.19/691 requests against a true $29.06/3384). The completed day's aggregates were computed and thrown away. Now each run completes the day that just ENDED — full-day spend, requests, tokens and impact, written to the usage columns only (upsert_usage: membership columns are point-in-time measurements and are never rewritten) — and then records today's live partial as before. backfill() gains --since YYYY-MM-DD: a targeted usage-only repair that cannot touch the published membership curve. Regression test: tests/test_metrics_completion.py (day completion, membership preservation, since-mode repair boundaries, full-backfill parity).
This commit is contained in:
1 parent
022e193cc2
commit
cf3dbb7489
4 files changed
+365
-61
No files matched your search
@@ -200,7 +200,10 @@ Three jobs run inside the container via `CloudronManifest.json` → `addons.sche
|
||||
- **metrics_snapshot** (daily 3:15am UTC) — `metrics_snapshot.py` records one row per
|
||||
day into the `metrics_daily` table: members, newsletter subscribers, Matrix
|
||||
membership, spend, request counts, energy/CO₂, and the Open Collective
|
||||
balance. A source that cannot be read
|
||||
balance. Each run **completes yesterday's row** with its full-day
|
||||
spend/requests/tokens/impact aggregates, then records today's row live (its
|
||||
usage columns carry the hours so far and are completed by the next run's
|
||||
completion pass). A source that cannot be read
|
||||
yields NULL for that metric rather than a zero. `spend_usd` is the co-op-wide
|
||||
total and deliberately spans every team, the co-op's own operator accounts
|
||||
included; `operator_spend_usd` is the breakout of that portion, served through
|
||||
@@ -209,7 +212,9 @@ Three jobs run inside the container via `CloudronManifest.json` → `addons.sche
|
||||
OC balance (the OC API has no historical series, so balance history accrues
|
||||
from these snapshots only — 2026-11-08 onward). `--backfill` rebuilds history
|
||||
idempotently (balance history cannot be backfilled: past balances were never
|
||||
recorded). Methodology: [Energy and carbon metrics](docs/energy-metrics.md).
|
||||
recorded); `--backfill --since YYYY-MM-DD` repairs recent days' usage columns
|
||||
only, leaving the membership curve as the nightly snapshots measured it.
|
||||
Methodology: [Energy and carbon metrics](docs/energy-metrics.md).
|
||||
|
||||
Verify after any manifest change: the deploy log must show
|
||||
`setupAddons: ... scheduler ...` listing all three tasks, and each script must
|
||||
|
||||
Reference in new issue
Block a user