gateway: record token counts on Metis pass-through spend rows

The pass-through row for /v1/systemone carried the correct spend but
prompt_tokens/completion_tokens/total_tokens were all 0: the generic
pass-through success handler hands the spend writer a
StandardPassThroughResponseObject with no usage, and the generic branch
builds no standard_logging_object usage, so anything summing tokens for
the route under-reported.

patch_passthrough_token_cost.py now also sets combined_usage_object (a
litellm.Usage carrying prompt_tokens=input_tokens, completion_tokens=0,
total_tokens=input_tokens) on the logging object -- the same override the
streaming handler uses for recovered partial usage, which the spend
writer reads before the response object. Spend is unchanged.

Defect 2 (a second zero-spend row with call_type=/v1/systemone) is NOT
duplication: it is upstream LiteLLM's failure row for a rejected
(missing/invalid key) request, written by
ProxyLogging._handle_logging_proxy_only_error on the pristine image. One
authenticated call produces exactly one priced row. Documented in the
README; no code change.

README.md

@@ -167,7 +167,9 @@ expect a 15–20 min maintenance window and **never panic-restart mid-migration*
 - Check the [LiteLLM changelog](https://github.com/BerriAI/litellm/releases) for breaking changes
 - The `start.sh` and `config.yaml` may need adjustment if LiteLLM changes its env-var or config schema
 - **Build-time patches ride on LiteLLM internals — verify each one after an upgrade.** `patch_budget_message.py`, `patch_image_cost.py` (keeps image-gen pricing on the shared cost-map key) and `patch_audio_request_cost.py` (adds the declared per-request charge on the audio paths: Tinfoil bills `voxtral-tts` $0.002 and `whisper-large-v3-turbo` $0.01 per request, which the cost map cannot express). Each prints `[coop-patch]` in the build log. They are non-fatal by design: a missing anchor warns and the build still succeeds, so the behaviour can vanish silently on a version bump. Related: code/litellm#5.
-- **`patch_passthrough_token_cost.py`** prices a generic pass-through route from the token count it reports in its own response body. A `pass_through_endpoints` target whose host matches none of vertex/anthropic/openai/gemini resolves to `EndpointType.GENERIC`: it is logged, but with no model on the route LiteLLM only knows the flat `cost_per_request`, which is a budget estimate, not an invoice. The patch bills any route listed in `general_settings.coop_passthrough_cost_per_token` (USD per input token, keyed by path) as `usage.input_tokens × rate` — Metis (`/v1/systemone`) at $0.12 per 1M input tokens. Same rules: non-fatal, `[coop-patch]` in the build log, verify after every upgrade. `cost_per_request` stays as the fallback for a listed route whose response carries no token count. The block prints `[coop-passthrough-cost] input_tokens=… rate=… -> response_cost=…` on each priced call, so the log is the field check that it fired.
+- **`patch_passthrough_token_cost.py`** prices a generic pass-through route from the token count it reports in its own response body. A `pass_through_endpoints` target whose host matches none of vertex/anthropic/openai/gemini resolves to `EndpointType.GENERIC`: it is logged, but with no model on the route LiteLLM only knows the flat `cost_per_request`, which is a budget estimate, not an invoice. The patch bills any route listed in `general_settings.coop_passthrough_cost_per_token` (USD per input token, keyed by path) as `usage.input_tokens × rate` — Metis (`/v1/systemone`) at $0.12 per 1M input tokens — and records the same count on the row (see below). Same rules: non-fatal, `[coop-patch]` in the build log, verify after every upgrade. `cost_per_request` stays as the fallback for a listed route whose response carries no token count. The block prints `[coop-passthrough-cost] input_tokens=… rate=… -> response_cost=…` on each priced call, so the log is the field check that it fired.
+- **Pass-through spend rows also carry the token counts (Metis defect fix).** Pricing the row from the body was not enough: the row recorded the correct `spend` but `prompt_tokens`/`completion_tokens`/`total_tokens` stayed `0`, because the pass-through success handler hands the spend writer a `StandardPassThroughResponseObject` whose `usage` is empty and the generic branch builds no `standard_logging_object` usage. Anything summing tokens for the route (panel, dashboard, energy reporting) then under-reported. The same patch now sets `combined_usage_object` (a `litellm.Usage`) on the logging object — the override the streaming handler already uses for recovered partial usage — which the spend writer reads before the response object (`spend_tracking_utils.get_logging_payload`, `StandardLoggingPayloadSetup.get_usage_as_dict`). The row now records `prompt_tokens = usage.input_tokens`, `completion_tokens = 0`, `total_tokens = usage.input_tokens`, with spend unchanged.
+- **A zero-spend row with `call_type = /v1/<path>` is NOT a duplicate of the priced row — it is the 401 refusal.** Every *rejected* pass-through request (missing/invalid key) is logged by upstream LiteLLM as a failure row whose `call_type` is the route path and whose `spend` is `0`, via `ProxyLogging._handle_logging_proxy_only_error` (`litellm/proxy/utils.py`, `original_function=route`). That is upstream behaviour on the pristine image, not something our patches add (neither `patch_passthrough_key_access.py` nor `patch_passthrough_token_cost.py` writes a row). A single *authenticated* call produces exactly **one** row (`call_type=pass_through_endpoint`, the real spend). Measured: one 200 → one row; one unauthenticated 401 → one failure row. Count requests for the route by `call_type = pass_through_endpoint`, not by summing all rows carrying the model name.
 - The CORS env var is `LITELLM_CORS_ORIGINS` (upstream-native since 1.84.0); the old `LITELLM_CORS_ALLOWED_ORIGINS` + sed patch are dead — don't re-add them
 - Test on a staging instance if the version jump is large
 

patch_passthrough_token_cost.py

@@ -39,6 +39,22 @@ payload — the ``general_settings`` dict is read straight from the proxy module
 the same way ``patch_passthrough_key_access.py`` reads
 ``coop_passthrough_open_to_keys``.
 
+TOKEN COUNTS. Pricing the row from the body is not enough: the row still
+recorded ``prompt_tokens=0``, ``completion_tokens=0``, ``total_tokens=0``,
+because the pass-through success handler hands the spend writer a
+``StandardPassThroughResponseObject`` whose text carries the body but whose
+``usage`` is empty, and the generic pass-through branch builds no
+``standard_logging_object`` usage either. Any per-token reporting (panel,
+dashboard, energy) that sums tokens for this route then under-reports. So the
+patch also carries the count onto the logging object as
+``combined_usage_object`` — the same ``litellm.Usage`` override the streaming
+handler uses to recover partial usage. The spend writer reads
+``combined_usage_object`` (``spend_tracking_utils.get_logging_payload`` and
+``StandardLoggingPayloadSetup.get_usage_as_dict``) before it falls back to the
+response object, so the row records ``prompt_tokens=input_tokens``,
+``completion_tokens=0``, ``total_tokens=input_tokens`` while spend stays exactly
+``input_tokens × rate``.
+
 The exact figure wins; ``cost_per_request`` stays as the fallback for a listed
 route whose body carries no usable input-token count, and for every route not
 listed. An upstream that reports its own totals via the x-litellm-* headers still
@@ -83,7 +99,8 @@ NEW = (
     "        # token count rather than cost_per_request's flat estimate. GreenPT's\n"
     "        # Metis decision route (/v1/systemone) is the first such route: it\n"
     "        # returns usage.input_tokens and always 0 output tokens, billed at\n"
-    "        # $0.12 per 1M input tokens. Inert unless a route is listed; a listed\n"
+    "        # $0.12 per 1M input tokens and recorded on the spend row with the\n"
+    "        # same count. Inert unless a route is listed; a listed\n"
     "        # route whose body carries no token count falls through to\n"
     "        # cost_per_request below, and every unlisted route is unchanged.\n"
     "        try:\n"
@@ -130,6 +147,30 @@ NEW = (
     "                    _coop_cost = float(_coop_tokens) * float(_coop_rate)\n"
     "                    kwargs[\"response_cost\"] = _coop_cost\n"
     "                    logging_obj.model_call_details[\"response_cost\"] = _coop_cost\n"
+    "                    # [coop-patch] Record the true token counts on the row too.\n"
+    "                    # The spend writer reads usage from combined_usage_object\n"
+    "                    # (the override the streaming handler already uses for\n"
+    "                    # recovered partial usage); without it the generic\n"
+    "                    # pass-through response object carries no usage and the\n"
+    "                    # row's prompt/completion/total tokens stay 0.\n"
+    "                    _coop_usage = None\n"
+    "                    try:\n"
+    "                        from litellm.types.utils import Usage as _CoopUsage\n"
+    "\n"
+    "                        _coop_usage = _CoopUsage(\n"
+    "                            prompt_tokens=int(_coop_tokens),\n"
+    "                            completion_tokens=0,\n"
+    "                            total_tokens=int(_coop_tokens),\n"
+    "                        )\n"
+    "                    except Exception as _coop_usage_exc:\n"
+    "                        print(\n"
+    "                            \"[coop-passthrough-cost] usage object failed: %s: %s\"\n"
+    "                            % (type(_coop_usage_exc).__name__, _coop_usage_exc),\n"
+    "                            flush=True,\n"
+    "                        )\n"
+    "                    if _coop_usage is not None:\n"
+    "                        kwargs[\"combined_usage_object\"] = _coop_usage\n"
+    "                        logging_obj.model_call_details[\"combined_usage_object\"] = _coop_usage\n"
     "                    print(\n"
     "                        \"[coop-passthrough-cost] input_tokens=%s rate=%s -> response_cost=%s\"\n"
     "                        % (_coop_tokens, _coop_rate, _coop_cost),\n"