member-portal/docs/energy-metrics.md

Energy and carbon metrics

How the portal's daily energy_wh and co2e_g figures are produced, what is measured versus estimated, and how to change them. Written because the calculation previously lived only in code comments (see code/member-portal#3).

For what members are told, see co-op/docs/energy.md. This document is the developer-facing counterpart and is deliberately blunter about the limits.

Where the numbers come from

Input: output tokens only, per day, per model.

• Source: LiteLLM's spend logs (LiteLLM_SpendLogs), column completion_tokens.
• Grouped on model_group — our alias, e.g. greenpt/deepseek-v4.1-flash — not model, which carries the upstream name (openai/deepseek-v4.1-flash). Grouping on the wrong column matches no factor and yields zeros for every day. That mistake shipped once during development and is the reason this is called out.

Only output tokens count, matching the unit GreenPT publishes (mWh per 100 output tokens). This is a deliberate choice to line up with the provider's own metric, not an oversight — and it means these figures understate total compute, because input tokens consume energy too. Treat the numbers as comparable and directionally true rather than as a total energy accounting.

The factor table

Factors live in metrics_snapshot.py (IMPACT_FACTORS, IMPACT_PROXY). Matching is str.startswith(), so the longest prefix must come first — order matters.

model prefix	Wh / 1k output tokens	gCO2e / kWh	provenance
greenpt/deepseek-v4.1-flash	0.3641	193.0	MEASURED
greenpt/glm-5.3-flash	1.0782	193.0	MEASURED
greenpt/glm-5.3	1.0782	193.0	SAME_FAMILY
greenpt/green-r	0.5535	23.6	MEASURED
anything else	0.5000	250.0	ESTIMATED

The three provenance tiers

• MEASURED — fitted from live calls to GreenPT's API, which returns an impact object on every request (energy in watt-milliseconds, emissions in µgCO2e). Energy was regressed on output tokens across prompt sizes (R² ≥ 0.999); grid intensity is what their own emissions figures imply. Fitted 2026-10-28.
• SAME_FAMILY — measured for glm-5.3-flash and assumed to hold for glm-5.3. An assumption of similarity, not a measurement.
• ESTIMATED — the proxy row: an order-of-magnitude figure near the middle of the measured range, with an EU grid average. It is applied to every provider that does not disclose impact, deliberately, so that a day's total is never quietly missing models. It is not a number we know.

No tier is exposed anywhere. The endpoints (/admin/metrics/series, /broker/metrics) return bare numbers, so a consumer cannot tell measured from estimated. Exposing provenance is tracked as a separate issue.

Formulas

energy_wh = Σ_models ( out_tokens / 1000 × wh_per_1k_out )

co2e_g    = Σ_models ( out_tokens / 1000 × wh_per_1k_out / 1000 × gco2e_per_kwh )
                      └────────────── Wh ──────────────┘   └─ Wh → kWh ─┘

Carbon was once understated by exactly 1000× here — a /1e6 where /1e3 was intended — which is why the tripwire below exists.

NULL, never zero

impact_for() returns None, None for a day with no measured tokens, and the row is written with NULL energy and carbon. It never writes 0.0:

a zero reads as "the co-op used no energy", which is a claim we cannot support, and it would survive into a chart as a fact.

The first backfill shipped exactly that bug — it summed the wrong column, every model totalled zero, and every day in history claimed to have used no energy.

A NULL day renders as a bare gap in the charts, with nothing explaining it. That is deliberate: the surfaces carry no caveat text. Accepted knowingly.

The same fail-closed rule applies to every metric in the snapshot, not only energy: an unreadable source (Listmonk, Matrix, LiteLLM) yields NULL for that metric and prints to the log, rather than a number that looks real.

The tripwire

After summing, impact_for() computes the grid intensity its own mix implies and compares it against a plausible band:

implied = co2e_g / energy_wh × 1000      # gCO2e per kWh
if not (5.0 <= implied <= 500.0):
    warn "suspect a unit error before trusting these numbers"

It warns; it never alters the numbers or suppresses the row. Its purpose is narrow and earned: the 1000× carbon error kept the energy column plausible, so nothing looked wrong, and a silent error of that kind would otherwise be found by a member rather than by us.

Schedule and backfill

• metrics_snapshot runs daily at 03:15, from Cloudron's scheduler addon (CloudronManifest.json → addons.scheduler), via /app/code/metrics_snapshot.py. It runs after membership_sweep (03:00) and rag_refresh (03:30).
• It writes one row per day into the metrics_daily table, in the same SQLite database as the members table (members.db). Columns: day, members_active, members_total, members_new, newsletter_subs, matrix_space_members, spend_usd, requests, out_tokens, energy_wh, co2e_g, recorded_at.
• metrics_snapshot.py --backfill rebuilds history from the sources and is idempotent — it rewrites a day rather than duplicating it. It is the tool to run after changing a factor, and the reason a published chart can move: history is rewritten, not appended to.

What is measured, and what is not

Stated plainly, because a mixed total is easy to over-read:

• Measured: GreenPT models — the providers whose API reports impact.
• Assumed equal to a measurement: greenpt/glm-5.3, from its flash sibling.
• Estimated: everything else. Currently that means all Tinfoil and Regolo traffic, which is a real share of co-op usage.

So a day's figure is a blend, and the split is not recorded per day — you cannot tell from metrics_daily how much of a total rested on the proxy. Persisting that split is a separate follow-up.

One consequence worth knowing: the proxy assumes a 250 gCO2e/kWh grid, while Regolo runs on 100% renewable energy, so the proxy overstates Regolo's carbon. Per-provider factors are a follow-up; until then, Regolo and Tinfoil figures should be read as upper bounds rather than measurements.

Changing the factors

Edit the table in metrics_snapshot.py and redeploy — refining a factor is a data edit, not a code change. Then run --backfill if the change should apply to history. The energy factors are also recorded in co-op/docs/energy.md for members, and the two should not disagree.