Add energy data to dashboards #26

New Issue

2026-10-19T20:57:57Z

ntnsndr commented

2026-10-19 20:57:57 +00:00

What's the idea?

Visibilize energy use and carbon emissions to members, not just cost and tokens, by presenting it with their usage data at dashboard.inference.coop.

Here, for instance, is GreenPT's dashboard on our collective usage:

Why does it matter?

Sustainability is an important goal for our co-op, and making energy impact visible is a way of helping guide our collective learning and behavior.

Who is it for?

Members and the planet.

How might it work?

For GreenPT, we can get energy usage data from the messages streamed to us, so we can capture and record that. For other providers, we might have to use industry-average estimates, unless they share data.

We could also attempt to gather data on the training runs of models we use, but that is likely going to be more difficult.

## What's the idea? Visibilize energy use and carbon emissions to members, not just cost and tokens, by presenting it with their usage data at dashboard.inference.coop. Here, for instance, is GreenPT's dashboard on our collective usage: ![image.png](attachments/633bc3be-f07e-4316-97fc-ff810beb7bdc) ## Why does it matter? Sustainability is an important goal for our co-op, and making energy impact visible is a way of helping guide our collective learning and behavior. ## Who is it for? Members and the planet. ## How might it work? For GreenPT, we can get energy usage data from the messages streamed to us, so we can capture and record that. For other providers, we might have to use industry-average estimates, unless they share data. We could also attempt to gather data on the training runs of models we use, but that is likely going to be more difficult.

image.png

98 KiB

ed commented

2026-10-21 19:38:57 +00:00

Sadly litellm instance strips the impact information that is sent when interacting with a GreenPT model.

GreenPT

FIrst here's where you can see it coming back when talking directly to GreenPT:

curl -sN https://api.greenpt.ai/v1/chat/completions \
    -H "Authorization: Bearer $GREENPT_KEY" \
    -H 'Content-Type: application/json' \
    -d '{"model":"deepseek-v4.1-flash","max_tokens":5,"stream":true,"stream_options":{"include_usage":true},"messages":[{"role":"user","content":"hi"}]}'                                                                                                                             data: {"id":"69fed020415f4189870644d806082fd3","object":"chat.completion.chunk","created":1792611294,"model":"deepseek-v4.1-flash","choices":[{"index":0,"delta":{"role":"assistant","content":"","tool_calls":null,"reasoning_content":null},"finish_reason":null}]}

data: {"id":"69fed020415f4189870644d806082fd3","object":"chat.completion.chunk","created":1792611294,"model":"deepseek-v4.1-flash","choices":[{"index":0,"delta":{"role":null,"content":"Hi! How can","tool_calls":null,"reasoning_content":null},"finish_reason":null}]}

data: {"id":"69fed020415f4189870644d806082fd3","object":"chat.completion.chunk","created":1792611294,"model":"deepseek-v4.1-flash","choices":[{"index":0,"delta":{"role":null,"content":null,"tool_calls":null,"reasoning_content":null},"finish_reason":"length"}]}

data: {"id":"69fed020415f4189870644d806082fd3","object":"chat.completion.chunk","created":1792611294,"model":"deepseek-v4.1-flash","choices":[{"index":0,"delta":{"role":null,"content":null,"tool_calls":null,"reasoning_content":null},"finish_reason":null}]}

data: {"id":"69fed020415f4189870644d806082fd3","object":"chat.completion.chunk","created":1792611294,"model":"deepseek-v4.1-flash","choices":[]}

data: {"id":"69fed020415f4189870644d806082fd3","object":"chat.completion.chunk","created":1792611294,"model":"deepseek-v4.1-flash","choices":[],"usage":{"prompt_tokens":31,"completion_tokens":5,"total_tokens":36,"prompt_tokens_details":{"cached_tokens":0},"completion_tokens_details":{"reasoning_tokens":1},"inferenceTiming":{"inferenceTimeMs":276.746509000659,"ttftMs":240}},"impact":{"version":"20250922","inferenceTime":{"total":277,"unit":"ms"},"energy":{"total":7234,"unit":"Wms"},"emissions":{"total":388,"unit":"ugCO2e"}}}

Notice the impact information that includes the energy and emissions information?

LiteLLM

However when you talk to LiteLLM at gateway.inference.coop this information is getting stripped off:

$ curl -N -s https://gateway.inference.coop/v1/chat/completions \
  -H "Authorization: Bearer $INFERENCE_COOP_KEY" \
  -H 'Content-Type: application/json' \
  -d '{
    "model":"greenpt/deepseek-v4.1-flash",
    "stream":true,
    "stream_options":{"include_usage":true},
    "messages":[{"role":"user","content":"hi"}]
  }'
data: {"id":"3b0a60117aca4706908b44c422248350","object":"chat.completion.chunk","created":1792611035,"model":"greenpt/deepseek-v4.1-flash","choices":[{"index":0,"delta":{"role":"assistant","content":""}}]}

data: {"id":"3b0a60117aca4706908b44c422248350","created":1792611035,"model":"greenpt/deepseek-v4.1-flash","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"reasoning_content":"The"}}]}

data: {"id":"3b0a60117aca4706908b44c422248350","created":1792611035,"model":"greenpt/deepseek-v4.1-flash","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"reasoning_content":" user just said \"hi"}}]}

data: {"id":"3b0a60117aca4706908b44c422248350","created":1792611035,"model":"greenpt/deepseek-v4.1-flash","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"reasoning_content":"\". This is a\n"}}]}

data: {"id":"3b0a60117aca4706908b44c422248350","created":1792611035,"model":"greenpt/deepseek-v4.1-flash","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"reasoning_content":"simple"}}]}

data: {"id":"3b0a60117aca4706908b44c422248350","created":1792611035,"model":"greenpt/deepseek-v4.1-flash","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"reasoning_content":" greeting. I should"}}]}

data: {"id":"3b0a60117aca4706908b44c422248350","created":1792611035,"model":"greenpt/deepseek-v4.1-flash","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"reasoning_content":" respond\n"}}]}

data: {"id":"3b0a60117aca4706908b44c422248350","created":1792611035,"model":"greenpt/deepseek-v4.1-flash","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"reasoning_content":"warmly"}}]}

data: {"id":"3b0a60117aca4706908b44c422248350","created":1792611035,"model":"greenpt/deepseek-v4.1-flash","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"reasoning_content":" and naturally,"}}]}

data: {"id":"3b0a60117aca4706908b44c422248350","created":1792611035,"model":"greenpt/deepseek-v4.1-flash","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"reasoning_content":" and offer"}}]}

data: {"id":"3b0a60117aca4706908b44c422248350","created":1792611035,"model":"greenpt/deepseek-v4.1-flash","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"reasoning_content":" to\n"}}]}

data: {"id":"3b0a60117aca4706908b44c422248350","created":1792611035,"model":"greenpt/deepseek-v4.1-flash","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"reasoning_content":"help. No"}}]}

data: {"id":"3b0a60117aca4706908b44c422248350","created":1792611035,"model":"greenpt/deepseek-v4.1-flash","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"reasoning_content":" need for tools"}}]}

data: {"id":"3b0a60117aca4706908b44c422248350","created":1792611035,"model":"greenpt/deepseek-v4.1-flash","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"reasoning_content":" or complex"}}]}

data: {"id":"3b0a60117aca4706908b44c422248350","created":1792611035,"model":"greenpt/deepseek-v4.1-flash","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"reasoning_content":"\nreason"}}]}

data: {"id":"3b0a60117aca4706908b44c422248350","created":1792611035,"model":"greenpt/deepseek-v4.1-flash","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"reasoning_content":"ing. Keep it brief"}}]}

data: {"id":"3b0a60117aca4706908b44c422248350","created":1792611035,"model":"greenpt/deepseek-v4.1-flash","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"reasoning_content":" and friendly,\n"}}]}

data: {"id":"3b0a60117aca4706908b44c422248350","created":1792611035,"model":"greenpt/deepseek-v4.1-flash","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"reasoning_content":"maybe"}}]}

data: {"id":"3b0a60117aca4706908b44c422248350","created":1792611035,"model":"greenpt/deepseek-v4.1-flash","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"reasoning_content":" ask"}}]}

data: {"id":"3b0a60117aca4706908b44c422248350","created":1792611035,"model":"greenpt/deepseek-v4.1-flash","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"reasoning_content":" what they'd like to do"}}]}

data: {"id":"3b0a60117aca4706908b44c422248350","created":1792611035,"model":"greenpt/deepseek-v4.1-flash","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"reasoning_content":"."}}]}

data: {"id":"3b0a60117aca4706908b44c422248350","object":"chat.completion.chunk","created":1792611035,"model":"greenpt/deepseek-v4.1-flash","choices":[{"index":0,"delta":{"content":"Hi! How"}}]}

data: {"id":"3b0a60117aca4706908b44c422248350","object":"chat.completion.chunk","created":1792611035,"model":"greenpt/deepseek-v4.1-flash","choices":[{"index":0,"delta":{"content":" can I help you today?"}}]}

data: {"id":"3b0a60117aca4706908b44c422248350","object":"chat.completion.chunk","created":1792611035,"model":"greenpt/deepseek-v4.1-flash","choices":[{"index":0,"delta":{},"finish_reason":"stop"}]}

data: {"id":"3b0a60117aca4706908b44c422248350","created":1792611035,"model":"greenpt/deepseek-v4.1-flash","object":"chat.completion.chunk","choices":[]}

data: {"id":"3b0a60117aca4706908b44c422248350","created":1792611035,"model":"greenpt/deepseek-v4.1-flash","object":"chat.completion.chunk","choices":[{"index":0,"delta":{}}],"usage":{"completion_tokens":66,"prompt_tokens":31,"total_tokens":97,"completion_tokens_details":{"reasoning_tokens":56},"prompt_tokens_details":{"cached_tokens":0},"cost":0.00007942000000000001}}

data: [DONE]

See how that second to last chunk lacks the impact? It might be difficult to see since you'll need to scroll to spot it.

litellm PR

With the help of AI I worked up a PR to litellm to preserve this information when it is sent. I'm having some trouble getting the attention of maintainers on the project. Perhaps upvoting the PR will help bring attention to it?

https://github.com/BerriAI/litellm/pull/46616

Sadly litellm instance strips the `impact` information that is sent when interacting with a GreenPT model. ## GreenPT FIrst here's where you can see it coming back when talking directly to GreenPT: ``` curl -sN https://api.greenpt.ai/v1/chat/completions \ -H "Authorization: Bearer $GREENPT_KEY" \ -H 'Content-Type: application/json' \ -d '{"model":"deepseek-v4.1-flash","max_tokens":5,"stream":true,"stream_options":{"include_usage":true},"messages":[{"role":"user","content":"hi"}]}' data: {"id":"69fed020415f4189870644d806082fd3","object":"chat.completion.chunk","created":1792611294,"model":"deepseek-v4.1-flash","choices":[{"index":0,"delta":{"role":"assistant","content":"","tool_calls":null,"reasoning_content":null},"finish_reason":null}]} data: {"id":"69fed020415f4189870644d806082fd3","object":"chat.completion.chunk","created":1792611294,"model":"deepseek-v4.1-flash","choices":[{"index":0,"delta":{"role":null,"content":"Hi! How can","tool_calls":null,"reasoning_content":null},"finish_reason":null}]} data: {"id":"69fed020415f4189870644d806082fd3","object":"chat.completion.chunk","created":1792611294,"model":"deepseek-v4.1-flash","choices":[{"index":0,"delta":{"role":null,"content":null,"tool_calls":null,"reasoning_content":null},"finish_reason":"length"}]} data: {"id":"69fed020415f4189870644d806082fd3","object":"chat.completion.chunk","created":1792611294,"model":"deepseek-v4.1-flash","choices":[{"index":0,"delta":{"role":null,"content":null,"tool_calls":null,"reasoning_content":null},"finish_reason":null}]} data: {"id":"69fed020415f4189870644d806082fd3","object":"chat.completion.chunk","created":1792611294,"model":"deepseek-v4.1-flash","choices":[]} data: {"id":"69fed020415f4189870644d806082fd3","object":"chat.completion.chunk","created":1792611294,"model":"deepseek-v4.1-flash","choices":[],"usage":{"prompt_tokens":31,"completion_tokens":5,"total_tokens":36,"prompt_tokens_details":{"cached_tokens":0},"completion_tokens_details":{"reasoning_tokens":1},"inferenceTiming":{"inferenceTimeMs":276.746509000659,"ttftMs":240}},"impact":{"version":"20250922","inferenceTime":{"total":277,"unit":"ms"},"energy":{"total":7234,"unit":"Wms"},"emissions":{"total":388,"unit":"ugCO2e"}}} ``` Notice the `impact` information that includes the energy and emissions information? ## LiteLLM However when you talk to LiteLLM at gateway.inference.coop this information is getting stripped off: ``` $ curl -N -s https://gateway.inference.coop/v1/chat/completions \ -H "Authorization: Bearer $INFERENCE_COOP_KEY" \ -H 'Content-Type: application/json' \ -d '{ "model":"greenpt/deepseek-v4.1-flash", "stream":true, "stream_options":{"include_usage":true}, "messages":[{"role":"user","content":"hi"}] }' data: {"id":"3b0a60117aca4706908b44c422248350","object":"chat.completion.chunk","created":1792611035,"model":"greenpt/deepseek-v4.1-flash","choices":[{"index":0,"delta":{"role":"assistant","content":""}}]} data: {"id":"3b0a60117aca4706908b44c422248350","created":1792611035,"model":"greenpt/deepseek-v4.1-flash","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"reasoning_content":"The"}}]} data: {"id":"3b0a60117aca4706908b44c422248350","created":1792611035,"model":"greenpt/deepseek-v4.1-flash","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"reasoning_content":" user just said \"hi"}}]} data: {"id":"3b0a60117aca4706908b44c422248350","created":1792611035,"model":"greenpt/deepseek-v4.1-flash","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"reasoning_content":"\". This is a\n"}}]} data: {"id":"3b0a60117aca4706908b44c422248350","created":1792611035,"model":"greenpt/deepseek-v4.1-flash","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"reasoning_content":"simple"}}]} data: {"id":"3b0a60117aca4706908b44c422248350","created":1792611035,"model":"greenpt/deepseek-v4.1-flash","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"reasoning_content":" greeting. I should"}}]} data: {"id":"3b0a60117aca4706908b44c422248350","created":1792611035,"model":"greenpt/deepseek-v4.1-flash","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"reasoning_content":" respond\n"}}]} data: {"id":"3b0a60117aca4706908b44c422248350","created":1792611035,"model":"greenpt/deepseek-v4.1-flash","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"reasoning_content":"warmly"}}]} data: {"id":"3b0a60117aca4706908b44c422248350","created":1792611035,"model":"greenpt/deepseek-v4.1-flash","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"reasoning_content":" and naturally,"}}]} data: {"id":"3b0a60117aca4706908b44c422248350","created":1792611035,"model":"greenpt/deepseek-v4.1-flash","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"reasoning_content":" and offer"}}]} data: {"id":"3b0a60117aca4706908b44c422248350","created":1792611035,"model":"greenpt/deepseek-v4.1-flash","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"reasoning_content":" to\n"}}]} data: {"id":"3b0a60117aca4706908b44c422248350","created":1792611035,"model":"greenpt/deepseek-v4.1-flash","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"reasoning_content":"help. No"}}]} data: {"id":"3b0a60117aca4706908b44c422248350","created":1792611035,"model":"greenpt/deepseek-v4.1-flash","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"reasoning_content":" need for tools"}}]} data: {"id":"3b0a60117aca4706908b44c422248350","created":1792611035,"model":"greenpt/deepseek-v4.1-flash","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"reasoning_content":" or complex"}}]} data: {"id":"3b0a60117aca4706908b44c422248350","created":1792611035,"model":"greenpt/deepseek-v4.1-flash","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"reasoning_content":"\nreason"}}]} data: {"id":"3b0a60117aca4706908b44c422248350","created":1792611035,"model":"greenpt/deepseek-v4.1-flash","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"reasoning_content":"ing. Keep it brief"}}]} data: {"id":"3b0a60117aca4706908b44c422248350","created":1792611035,"model":"greenpt/deepseek-v4.1-flash","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"reasoning_content":" and friendly,\n"}}]} data: {"id":"3b0a60117aca4706908b44c422248350","created":1792611035,"model":"greenpt/deepseek-v4.1-flash","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"reasoning_content":"maybe"}}]} data: {"id":"3b0a60117aca4706908b44c422248350","created":1792611035,"model":"greenpt/deepseek-v4.1-flash","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"reasoning_content":" ask"}}]} data: {"id":"3b0a60117aca4706908b44c422248350","created":1792611035,"model":"greenpt/deepseek-v4.1-flash","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"reasoning_content":" what they'd like to do"}}]} data: {"id":"3b0a60117aca4706908b44c422248350","created":1792611035,"model":"greenpt/deepseek-v4.1-flash","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"reasoning_content":"."}}]} data: {"id":"3b0a60117aca4706908b44c422248350","object":"chat.completion.chunk","created":1792611035,"model":"greenpt/deepseek-v4.1-flash","choices":[{"index":0,"delta":{"content":"Hi! How"}}]} data: {"id":"3b0a60117aca4706908b44c422248350","object":"chat.completion.chunk","created":1792611035,"model":"greenpt/deepseek-v4.1-flash","choices":[{"index":0,"delta":{"content":" can I help you today?"}}]} data: {"id":"3b0a60117aca4706908b44c422248350","object":"chat.completion.chunk","created":1792611035,"model":"greenpt/deepseek-v4.1-flash","choices":[{"index":0,"delta":{},"finish_reason":"stop"}]} data: {"id":"3b0a60117aca4706908b44c422248350","created":1792611035,"model":"greenpt/deepseek-v4.1-flash","object":"chat.completion.chunk","choices":[]} data: {"id":"3b0a60117aca4706908b44c422248350","created":1792611035,"model":"greenpt/deepseek-v4.1-flash","object":"chat.completion.chunk","choices":[{"index":0,"delta":{}}],"usage":{"completion_tokens":66,"prompt_tokens":31,"total_tokens":97,"completion_tokens_details":{"reasoning_tokens":56},"prompt_tokens_details":{"cached_tokens":0},"cost":0.00007942000000000001}} data: [DONE] ``` See how that second to last chunk lacks the `impact`? It might be difficult to see since you'll need to scroll to spot it. ## litellm PR With the help of AI I worked up a PR to litellm to preserve this information when it is sent. I'm having some trouble getting the attention of maintainers on the project. Perhaps upvoting the PR will help bring attention to it? https://github.com/BerriAI/litellm/pull/46616

inference-bot commented

2026-10-22 03:40:22 +00:00

Verified the strip on our side too: LiteLLM drops impact on both streaming and non-streaming (only inferenceTiming survives — a field the Usage model knows). Root cause is Usage serialization dropping undeclared fields; the chunk-level preservation that exists today can't catch it because impact lives inside usage.

Two things we can do without waiting upstream:

Passthrough — a ~1-line patch on our gateway (allow extra fields on Usage) lets impact reach clients on both paths. @ed's PR (#46616) is the proper upstream version of the same idea.
Capture — record impact per request at the gateway (user, model, tokens, energy, emissions) into our own store, and have the dashboard aggregate from there. That's what this issue actually needs: member-visible aggregates shouldn't depend on clients persisting anything.

Other providers: the same capture table can hold estimated values — a per-model Wh / CO₂e-per-token factor applied at ingestion (or dashboard-side) for providers that don't report impact. GreenPT shows measured data; others show estimates, labeled as such — the dashboard treats measured-vs-estimated as a data-quality flag, not a special case.

Verified the strip on our side too: LiteLLM drops `impact` on **both** streaming and non-streaming (only `inferenceTiming` survives — a field the `Usage` model knows). Root cause is `Usage` serialization dropping undeclared fields; the chunk-level preservation that exists today can't catch it because `impact` lives *inside* `usage`. Two things we can do without waiting upstream: 1. **Passthrough** — a ~1-line patch on our gateway (allow extra fields on `Usage`) lets `impact` reach clients on both paths. @ed's PR (#46616) is the proper upstream version of the same idea. 2. **Capture** — record `impact` per request at the gateway (user, model, tokens, energy, emissions) into our own store, and have the dashboard aggregate from there. That's what this issue actually needs: member-visible aggregates shouldn't depend on clients persisting anything. **Other providers**: the same capture table can hold *estimated* values — a per-model Wh / CO₂e-per-token factor applied at ingestion (or dashboard-side) for providers that don't report impact. GreenPT shows measured data; others show estimates, labeled as such — the dashboard treats measured-vs-estimated as a data-quality flag, not a special case.

ed commented

2026-10-22 11:25:03 +00:00

A one line patch doesn't sound too bad! I do worry that these patches will become technical debt that prevents us from easily upgrading litellm (we are already behind)? But as long as we flag it as maintenance that needs to happen on some cadence I think it should be fine.

I am unsure about estimating information that greenpt is offering as a value add. Presumably they have done some research of their own, to determine the best way to do this? I also think their numbers are based on an understanding of the underlying infrastructure that inference is running on?

My prefence would be to start by adding what we are being provided, and flag it as a research project to add more?

A one line patch doesn't sound too bad! I do worry that these patches will become technical debt that prevents us from easily upgrading litellm (we are already behind)? But as long as we flag it as maintenance that needs to happen on some cadence I think it should be fine. I am unsure about estimating information that greenpt is offering as a value add. Presumably they have done some research of their own, to determine the best way to do this? I also think their numbers are based on an understanding of the underlying infrastructure that inference is running on? My prefence would be to start by adding what we are being provided, and flag it as a research project to add more?

ntnsndr commented

2026-10-22 15:42:23 +00:00

I wonder if it would be cleaner for now to just use per-token energy/emissions estimates based on what GreenPT publishes, similar to what we do for other providers.

👍 1

inference-bot referenced this issue from a commit

2026-10-28 04:32:07 +00:00

metrics: energy and carbon per day, from measured GreenPT factors

inference-bot referenced this issue from a commit

2026-10-28 04:36:15 +00:00

charts: add Energy and Carbon (co-op/support#26)

inference-bot commented

2026-10-28 04:36:23 +00:00

Nathan's steer (per-token estimates based on what GreenPT publishes) is now implemented and live in the data layer. Summary of what it does, and what it does not yet cover.

How the numbers are produced

Output tokens per model per day come from our own spend logs (completion_tokens, grouped by model_group), so member-visible impact does not depend on any client persisting an impact object — and it covers our whole history immediately rather than from whenever the passthrough lands. That sidesteps the maintenance-debt concern @ed raised about patching LiteLLM.

Factors are applied from one table, each row carrying its source:

Measured for greenpt/*: fitted from live calls to their API. Their impact object gave energy (watt-ms) and emissions (µgCO2e) per call, so energy was regressed on output tokens across prompt sizes (R² ≥ 0.999; the intercept is real — their energy derives from inference time, so short calls are mostly overhead). Carbon uses the grid intensity their own emissions imply. Per 1,000 output tokens: deepseek-v4.1-flash 0.36 Wh, glm-5.3-flash 1.08 Wh, green-r 0.55 Wh. Implied intensity ~193 gCO₂e/kWh for the first two, ~24 for green-r — which is where their "up to 40% lower CO₂ per query" claim comes from.
Estimated for everyone else (Tinfoil, Regolo, PublicAI do not disclose). Those fall back to a documented order-of-magnitude proxy so a day's total is never silently missing models. I deliberately did not manufacture per-model precision we do not have; refining a factor is a data edit, not a code change.

What it reports

55 days backfilled, through 2026-10-28: 21.2M output tokens, 14.35 kWh, 2.9 kgCO₂e. The most recent week is ~5.6 kWh and ~1.1 kgCO₂e. Energy and carbon now appear as metrics on the admin panel's Trends tab and the member dashboard's co-op view.

Three caveats worth recording before anyone quotes these figures

Output tokens only — that is GreenPT's definition, chosen so comparisons hold across prompts, and it means our figures are not total compute.
This is a floor, not a total. About a third of spend-log rows — embeddings, audio, image generation — carry no completion tokens at all, so their compute is not counted. A separate factor for those workloads is the obvious next improvement.
One day (2026-09-13) has requests but no recorded completion tokens, so it reads NULL ("not measured") rather than zero.

The estimate is guarded by a tripwire: if the implied grid intensity falls outside 5–500 gCO₂e/kWh, the job prints a warning instead of quietly emitting numbers. That exists because three unit/column errors shipped in the first pass — the carbon column was 1,000× low at one point, and nothing looked broken until the arithmetic was checked against the energy column.

Open question

The sustainability page says per-account metrics are exportable from the GreenPT dashboard. If the co-op has that account, an export would replace my per-token extrapolation with real measured totals. Does anyone have access to it?

— inference-bot

Nathan's steer (per-token estimates based on what GreenPT publishes) is now implemented and live in the data layer. Summary of what it does, and what it does not yet cover. ## How the numbers are produced Output tokens per model per day come from **our own spend logs** (`completion_tokens`, grouped by `model_group`), so member-visible impact does not depend on any client persisting an `impact` object — and it covers our whole history immediately rather than from whenever the passthrough lands. That sidesteps the maintenance-debt concern @ed raised about patching LiteLLM. Factors are applied from one table, each row carrying its source: - **Measured** for `greenpt/*`: fitted from live calls to their API. Their `impact` object gave energy (watt-ms) and emissions (µgCO2e) per call, so energy was regressed on output tokens across prompt sizes (R² ≥ 0.999; the intercept is real — their energy derives from inference *time*, so short calls are mostly overhead). Carbon uses the grid intensity their own emissions imply. Per 1,000 output tokens: `deepseek-v4.1-flash` 0.36 Wh, `glm-5.3-flash` 1.08 Wh, `green-r` 0.55 Wh. Implied intensity ~193 gCO₂e/kWh for the first two, ~24 for `green-r` — which is where their "up to 40% lower CO₂ per query" claim comes from. - **Estimated** for everyone else (Tinfoil, Regolo, PublicAI do not disclose). Those fall back to a documented order-of-magnitude proxy so a day's total is never silently missing models. I deliberately did **not** manufacture per-model precision we do not have; refining a factor is a data edit, not a code change. ## What it reports 55 days backfilled, through 2026-10-28: **21.2M output tokens, 14.35 kWh, 2.9 kgCO₂e**. The most recent week is ~5.6 kWh and ~1.1 kgCO₂e. Energy and carbon now appear as metrics on the admin panel's Trends tab and the member dashboard's co-op view. ## Three caveats worth recording before anyone quotes these figures 1. **Output tokens only** — that is GreenPT's definition, chosen so comparisons hold across prompts, and it means our figures are *not* total compute. 2. **This is a floor, not a total.** About a third of spend-log rows — embeddings, audio, image generation — carry no completion tokens at all, so their compute is not counted. A separate factor for those workloads is the obvious next improvement. 3. One day (2026-09-13) has requests but no recorded completion tokens, so it reads NULL ("not measured") rather than zero. The estimate is guarded by a tripwire: if the implied grid intensity falls outside 5–500 gCO₂e/kWh, the job prints a warning instead of quietly emitting numbers. That exists because three unit/column errors shipped in the first pass — the carbon column was 1,000× low at one point, and nothing looked broken until the arithmetic was checked against the energy column. ## Open question The sustainability page says per-account metrics are **exportable from the GreenPT dashboard**. If the co-op has that account, an export would replace my per-token extrapolation with real measured totals. Does anyone have access to it? — inference-bot

inference-bot referenced this issue from a commit

2026-10-28 04:36:43 +00:00

charts: add Energy and Carbon (co-op/support#26)

ntnsndr commented

2026-10-28 16:20:41 +00:00

I do, but not sure we want to manually export them regularly.

ntnsndr commented

2026-10-28 16:43:44 +00:00

Closing for now!

ntnsndr closed this issue

2026-10-28 16:43:44 +00:00

ed commented

2026-10-30 16:39:07 +00:00

In case you are looking at this later and wondering how the analytics work there is now documentation for it as well:

https://git.inference.coop/code/member-portal/src/branch/main/docs/energy-metrics.md

In case you are looking at this later and wondering how the analytics work there is now documentation for it as well: https://git.inference.coop/code/member-portal/src/branch/main/docs/energy-metrics.md

Sign in to join this conversation.

3 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: co-op/support#26