Files
litellm/patch_expose_cost_headers.py
inference-bot 9e25578ee6 upgrade: pin LiteLLM v1.108.0 (candidate for 2026-11-08 window)
- Dockerfile FROM v1.103.2 -> v1.108.0; manifest upstreamVersion follows.
- patch_expose_cost_headers.py: upstream 1.108.0 refactored the CORS
  middleware call (expose_headers now also spreads
  trace_response_expose_headers()), so the 1.103.x anchor no longer
  matched and the patch would have SILENTLY stopped applying. The script
  now handles both anchor shapes (1.103.x first, 1.108.x fallback).
- All 8 build patches verified against the real v1.108.0 sources: every
  anchor matched, every patched file compiles, re-runs are idempotent.
- 36 new migrations, all additive (no unguarded DROP/DELETE/UPDATE);
  budget_reset_at unchanged on LiteLLM_TeamTable; SpendLogs gains only
  billing_agent_id.
2026-11-06 03:55:36 +00:00

116 lines
4.7 KiB
Python

"""Expose LiteLLM's per-request cost headers to cross-origin browser clients.
MEMBER REQUEST (co-op/support#39, 2026-10-31). Browser JS calling the gateway from
another origin cannot read a response header unless the server names it in
Access-Control-Expose-Headers. LiteLLM sets
expose_headers=LITELLM_UI_ALLOW_HEADERS # through 1.103.x
expose_headers=[*LITELLM_UI_ALLOW_HEADERS,
*trace_response_expose_headers()] # from 1.108.0
which covers seven names (semantic-filter, guardrail, cache-key, ...) and none of
the cost ones — so `x-litellm-response-cost*` reads as null in JS even though the
proxy sends those headers on the response. Upstream closed the request
(BerriAI/litellm#33123) as stale, so it has to be handled here at the app layer,
exactly like the other patches in this directory.
Additive only. It lets a caller read per-request pricing for requests they made
with their own API key — numbers already present in the response body for
non-streaming calls, and already visible in their dashboard. It exposes no prompt,
key, or account data, and no other member's usage.
Run at BUILD time (the runtime rootfs is read-only under Cloudron). Non-fatal, per
house rule: if upstream moves or rewrites this call, WARN and exit 0 — a missing
convenience header must never keep the gateway from booting.
Regression check: after a deploy, a browser request from an unrelated origin should
see the cost headers listed in Access-Control-Expose-Headers on the response.
"""
from pathlib import Path
import sys
# The header names the proxy can set, taken from the running image rather than
# guessed: grep -rhoE 'x-litellm-[a-z0-9-]*cost[a-z0-9-]*' over the litellm package.
EXPOSED_HEADERS = [
"x-litellm-response-cost",
"x-litellm-response-cost-input",
"x-litellm-response-cost-output",
"x-litellm-response-cost-original",
"x-litellm-response-cost-discount-amount",
"x-litellm-response-cost-margin-amount",
"x-litellm-response-cost-margin-percent",
"x-litellm-response-cost-cache-creation",
"x-litellm-response-cost-cache-read",
"x-litellm-response-cost-reasoning",
"x-litellm-response-cost-tool-usage",
"x-litellm-classifier-cost",
# Telemetry names, requested in co-op/support#39 (2026-10-31): model identity,
# call id, the caller's own key spend, timings and retry/fallback counts.
"x-litellm-model-name",
"x-litellm-call-id",
"x-litellm-key-spend",
"x-litellm-response-duration-ms",
"x-litellm-overhead-duration-ms",
"x-litellm-callback-duration-ms",
"x-litellm-attempted-retries",
"x-litellm-attempted-fallbacks",
]
OLD = " expose_headers=LITELLM_UI_ALLOW_HEADERS,\n"
NEW = (
" expose_headers=LITELLM_UI_ALLOW_HEADERS\n"
" + [\n"
+ "".join(f' "{h}",\n' for h in EXPOSED_HEADERS)
+ " ],\n"
)
# 1.108.0 refactored the middleware call (upstream added the trace-response
# header set to expose_headers). Same operation, new anchor; tried when the
# 1.103.2 shape is absent so the script works against either base image.
OLD_V1108 = (
" expose_headers=[*LITELLM_UI_ALLOW_HEADERS, *trace_response_expose_headers()],\n"
)
NEW_V1108 = (
" expose_headers=[\n"
" *LITELLM_UI_ALLOW_HEADERS,\n"
" *trace_response_expose_headers(),\n"
+ "".join(f' "{h}",\n' for h in EXPOSED_HEADERS)
+ " ],\n"
)
def main() -> int:
# Glob rather than pin a Python version: the venv path moves when the base
# image bumps its interpreter, and a stale path would silently skip the patch.
candidates = sorted(Path("/app/.venv/lib").glob("python3*/site-packages/litellm/proxy/proxy_server.py"))
if not candidates:
print("WARN expose-cost-headers: proxy_server.py not found under /app/.venv/lib — skipping")
return 0
target = candidates[-1]
src = target.read_text()
if "x-litellm-response-cost-input" in src:
print("expose-cost-headers: already applied")
return 0
for old, new, shape in ((OLD, NEW, "1.103.x"), (OLD_V1108, NEW_V1108, "1.108.x")):
found = src.count(old)
if found == 1:
target.write_text(src.replace(old, new))
print(
f"expose-cost-headers: appended {len(EXPOSED_HEADERS)} cost header names "
f"to expose_headers (anchor {shape}) in {target}"
)
return 0
if found > 1:
print(f"WARN expose-cost-headers: expected exactly 1 {shape} CORS middleware call, found {found} — skipping")
return 0
print(
"WARN expose-cost-headers: no known expose_headers anchor found "
"(neither 1.103.x nor 1.108.x shape) — skipping"
)
return 0
if __name__ == "__main__":
sys.exit(main())