member-portal/docs/energy-metrics.md

Energy and carbon metrics

How the portal's daily energy_wh and co2e_g figures are produced, what is measured versus estimated, and how to change them. Written because the calculation previously lived only in code comments (see code/member-portal#3).

For what members are told, see co-op/docs/energy.md. This document is the developer-facing counterpart and is deliberately blunter about the limits.

Where the numbers come from

Input: output tokens only, per day, per model.

• Source: LiteLLM's spend logs (LiteLLM_SpendLogs), column completion_tokens.
• Grouped on model_group — our alias, e.g. greenpt/deepseek-v4.1-flash — not model, which carries the upstream name (openai/deepseek-v4.1-flash). Grouping on the wrong column matches no factor and yields zeros for every day. That mistake shipped once during development and is the reason this is called out.

Only output tokens count, matching the unit GreenPT publishes (mWh per 100 output tokens). This is a deliberate choice to line up with the provider's own metric, not an oversight — and it means these figures understate total compute, because input tokens consume energy too. Treat the numbers as comparable and directionally true rather than as a total energy accounting.

The factor table

Factors live in metrics_snapshot.py (IMPACT_FACTORS, IMPACT_PROXY). Matching is str.startswith(), so the longest prefix must come first — order matters.

model prefix	Wh / 1k output tokens	gCO2e / kWh	provenance
greenpt/deepseek-v4.1-flash	0.3641	193.0	MEASURED
greenpt/glm-5.3-flash	1.0782	193.0	MEASURED
greenpt/glm-5.3	1.0782	193.0	SAME_FAMILY
greenpt/green-r	0.5535	23.6	MEASURED
anything else	0.5000	250.0	ESTIMATED

The three provenance tiers

• MEASURED — fitted from live calls to GreenPT's API, which returns an impact object on every request (energy in watt-milliseconds, emissions in µgCO2e). Energy was regressed on output tokens across prompt sizes (R² ≥ 0.999); grid intensity is what their own emissions figures imply. Fitted 2026-10-28.
• SAME_FAMILY — measured for glm-5.3-flash and assumed to hold for glm-5.3. An assumption of similarity, not a measurement.
• ESTIMATED — the proxy row: an order-of-magnitude figure near the middle of the measured range, with an EU grid average. It is applied to every provider that does not disclose impact, deliberately, so that a day's total is never quietly missing models. It is not a number we know.

No tier is exposed on its own, but every row now carries a provenance read derived from the measured/estimated token split — see "Provenance in the API" below. A consumer can tell how much of a day's figure rested on a measurement.

Provenance in the API

Every row returned by /admin/metrics/series and /broker/metrics carries the measured-vs-estimated split, so a consumer can tell how much of a day's figure rested on a real measurement. Four fields:

field	meaning
tokens_measured	output tokens that day attributed to a factor whose source is MEASURED — a direct provider measurement.
tokens_estimated	the rest: SAME_FAMILY (borrowed from a sibling) and the flat proxy. A borrowed or proxied factor is an assumption, so it is not counted as measured.
factor_revision	a short, stable hash of the factor table (IMPACT_FACTORS + IMPACT_PROXY) in effect when the row was computed.
provenance	the derived read, below.

tokens_measured + tokens_estimated = out_tokens for any day with tokens. A day with no output tokens has NULL in both (fail-closed, same rule as energy/carbon).

provenance is one of:

• measured — every token rested on a measured factor.
• estimated — every token rested on a non-measured factor (borrowed or proxy).
• mixed — both are present; the day's total is a blend.
• none — no provenance. This is either a day with no output tokens or a row written before these columns existed (older rows keep NULL). It is deliberately not measured: an unknown split must never read as a confident measurement. This is the whole point of the indicator — failure reads as unknown.

factor_revision is a deterministic hash, not a timestamp: two runs against the same table produce the same string. It changes if any factor value or source label changes, or if the matching order changes (order is meaningful — longest prefix first). It is "what the numbers were computed with", not "what was known then".

Forward-only, then backfill

The columns are added by an additive migration — a guarded ALTER TABLE metrics_daily ADD COLUMN …, the same idiom used when the impact columns first arrived. On an existing database this is safe: it never errors, existing rows survive untouched, and they simply keep NULL in the new columns. Nothing is guessed for history on migration; those rows read as provenance: "none".

To give past rows a real split, run metrics_snapshot.py --backfill (or re-run the affected days). Backfill rewrites each day from the current LiteLLM token split and the current factor table, so it also updates factor_revision to today's revision — a backfilled historical row records the revision used to recompute it, not the one that was current at the time. Honest for a chart label; not a historical audit trail.

Formulas

energy_wh = Σ_models ( out_tokens / 1000 × wh_per_1k_out )

co2e_g    = Σ_models ( out_tokens / 1000 × wh_per_1k_out / 1000 × gco2e_per_kwh )
                      └────────────── Wh ──────────────┘   └─ Wh → kWh ─┘

Carbon was once understated by exactly 1000× here — a /1e6 where /1e3 was intended — which is why the tripwire below exists.

NULL, never zero

impact_for() returns an all-NULL impact block for a day with no measured tokens, and the row is written with NULL energy and carbon. It never writes 0.0:

a zero reads as "the co-op used no energy", which is a claim we cannot support, and it would survive into a chart as a fact.

The first backfill shipped exactly that bug — it summed the wrong column, every model totalled zero, and every day in history claimed to have used no energy.

A NULL day renders as a bare gap in the charts, with nothing explaining it. That is deliberate: the surfaces carry no caveat text. Accepted knowingly.

The same fail-closed rule applies to every metric in the snapshot, not only energy: an unreadable source (Listmonk, Matrix, LiteLLM) yields NULL for that metric and prints to the log, rather than a number that looks real.

The tripwire

After summing, impact_for() computes the grid intensity its own mix implies and compares it against a plausible band:

implied = co2e_g / energy_wh × 1000      # gCO2e per kWh
if not (5.0 <= implied <= 500.0):
    warn "suspect a unit error before trusting these numbers"

It warns; it never alters the numbers or suppresses the row. Its purpose is narrow and earned: the 1000× carbon error kept the energy column plausible, so nothing looked wrong, and a silent error of that kind would otherwise be found by a member rather than by us.

Schedule and backfill

• metrics_snapshot runs daily at 03:15, from Cloudron's scheduler addon (CloudronManifest.json → addons.scheduler), via /app/code/metrics_snapshot.py. It runs after membership_sweep (03:00) and rag_refresh (03:30).
• It writes one row per day into the metrics_daily table, in the same SQLite database as the members table (members.db). Columns: day, members_active, members_total, members_new, newsletter_subs, matrix_space_members, spend_usd, requests, out_tokens, energy_wh, co2e_g, tokens_measured, tokens_estimated, factor_revision, recorded_at.
• metrics_snapshot.py --backfill rebuilds history from the sources and is idempotent — it rewrites a day rather than duplicating it. It is the tool to run after changing a factor, and the reason a published chart can move: history is rewritten, not appended to.

What is measured, and what is not

Stated plainly, because a mixed total is easy to over-read:

• Measured: GreenPT models — the providers whose API reports impact.
• Assumed equal to a measurement: greenpt/glm-5.3, from its flash sibling.
• Estimated: everything else. Currently that means all Tinfoil and Regolo traffic, which is a real share of co-op usage.

So a day's figure is a blend, and the split is now recorded per day — tokens_measured and tokens_estimated say how much of the total rested on a measurement, and the API derives a provenance read from them. Before those columns existed the split was not recorded; such rows carry NULL and read as provenance: "none". See "Provenance in the API".

One consequence worth knowing: the proxy assumes a 250 gCO2e/kWh grid, while Regolo runs on 100% renewable energy, so the proxy overstates Regolo's carbon. Per-provider factors are a follow-up; until then, Regolo and Tinfoil figures should be read as upper bounds rather than measurements.

Changing the factors

Edit the table in metrics_snapshot.py and redeploy — refining a factor is a data edit, not a code change. Then run --backfill if the change should apply to history. The energy factors are also recorded in co-op/docs/energy.md for members, and the two should not disagree.