diff --git a/energy.md b/energy.md new file mode 100644 index 0000000..0a35a8d --- /dev/null +++ b/energy.md @@ -0,0 +1,80 @@ +# Energy and carbon + +The co-op shows members how much energy its inference uses, and what carbon that +represents. This page says where those numbers come from and where they are weak. +The [model list](models.md) covers which providers we use and their energy +sources; this covers how we count. + +## What we measure + +**Output tokens, per model, per day** — read from our own gateway logs. Nothing +is required of your client, and no per-member figure is published: the dashboards +show co-op totals. + +We count **output tokens only**, deliberately. That is the unit +[GreenPT's published metric](https://greenpt.com/blog/introducing-a-new-ai-metric-to-drive-sustainability) +uses ("mWh per 100 tokens"); input tokens are excluded so comparisons hold across +prompts, workloads and models. It also means these figures are **not** total +compute. + +## How tokens become energy and carbon + +``` +energy = output tokens × (Wh per 1,000 output tokens) +carbon = energy × grid intensity of the electricity (gCO2e per kWh) +``` + +Both factors are held per provider in one table, with the source recorded for +each row, in the portal's +[`metrics_snapshot.py`](https://git.inference.coop/code/member-portal/src/branch/main/metrics_snapshot.py). + +### Measured: GreenPT + +GreenPT returns an `impact` object with every call — energy in watt-milliseconds, +emissions in µgCO2e — so these are fitted from their own numbers rather than +imported from a third party. Measured 2026-10-28 by regressing energy against +output tokens across prompt sizes (R² ≥ 0.999; there is a real per-request +overhead, because their energy derives from inference *time*): + +| Model | Wh per 1,000 output tokens | Implied grid intensity | +|---|---|---| +| `greenpt/deepseek-v4.1-flash` | 0.36 | ~193 gCO2e/kWh | +| `greenpt/glm-5.3-flash` | 1.08 | ~193 gCO2e/kWh | +| `greenpt/green-r` | 0.55 | ~24 gCO2e/kWh | + +`green-r` is roughly an order of magnitude cleaner per unit of energy, which is +where GreenPT's claim of up to 40% lower CO2 per query comes from. + +### Estimated: everyone else + +Providers that do not disclose impact — Tinfoil, Regolo, PublicAI — use a +**documented order-of-magnitude proxy**: 0.5 Wh per 1,000 output tokens at an EU +grid average of 250 gCO2e/kWh. It exists so a day's total is never quietly +missing models. It is an estimate, and it is labelled as one everywhere it +appears. Refining a factor is a data edit, not a code change. + +## What this does not cover + +- **It is a floor, not a total.** Embedding, audio and image requests carry no + completion tokens, so their compute is not counted at all. This is the largest + known gap. +- **Estimates for non-disclosing providers.** The proxy is an order-of-magnitude + figure, not a measurement of what a given provider actually draws. +- **GreenPT's energy is itself estimated** — from time and documented hardware + power — so our per-token figures are an efficiency ratio, not a per-call meter + reading. +- **Training emissions are not included**, for any model we serve. Nobody in this + market can verify them yet. + +## Where to see the numbers + +The [member dashboard](https://dashboard.inference.coop) has them under +"Co-op usage" — by day, alongside membership, requests and spend. Operators see +the same series on the admin panel's Trends tab. + +## Units, for the curious + +`Wh` is a watt-hour (a 100 W bulb for one hour is 100 Wh). `kWh` is 1,000 Wh. +`gCO2e` is grams of carbon-dioxide equivalent. Converting between them is where +mistakes hide: µgCO2e per mWh and gCO2e per kWh are the same ratio, and Wh to kWh +is a factor of 1,000.