inference-bot
8ddd023786
LiteLLM copies every upstream response header onto its own response prefixed
'llm_provider-'. For our providers that hands callers operational detail about the
co-op's own account — verified live: llm_provider-x-credits-remaining carried the
remaining provider balance in every member's response.
Build-time patch makes _get_llm_provider_headers() return {}. The four OpenAI-style
x-ratelimit-* headers a provider may send are still forwarded: those describe the
caller's own request, not the co-op's account.
76 lines
3.3 KiB
Python
76 lines
3.3 KiB
Python
"""Stop LiteLLM from forwarding the provider's response headers to callers.
|
|
|
|
SECURITY (2026-10-27, member-disclosure finding 4): LiteLLM's proxy copies every
|
|
upstream response header onto its own response as `llm_provider-<name>` — see
|
|
litellm_core_utils/llm_response_utils/get_headers.py::_get_llm_provider_headers,
|
|
whose docstring reads "Forward all headers as llm_provider-{header}". Verified
|
|
live on the co-op gateway: one chat completion returned 27 such headers, among
|
|
them `llm_provider-x-credits-remaining: 16.52` — the co-op's remaining balance
|
|
with the provider, readable by any member API key, and (read across calls) its
|
|
burn rate.
|
|
|
|
This patch makes _get_llm_provider_headers() return {} so no provider header is
|
|
copied. get_response_headers() still forwards the four OpenAI-style x-ratelimit-*
|
|
headers WHEN a provider sends them; those describe the caller's own request
|
|
rather than the co-op's account, and no upstream in our config sends them today.
|
|
|
|
Run at BUILD time (the runtime rootfs is read-only under Cloudron). Non-fatal, per
|
|
house rule: if upstream moves or rewrites the function, WARN and exit 0 — a
|
|
cosmetic leak must never keep the gateway from booting. Regression check: the
|
|
daily litellm watcher asserts the gateway returns no `llm_provider-*` header.
|
|
"""
|
|
import glob
|
|
import os
|
|
import sys
|
|
|
|
ROOT = os.environ.get("COOP_PATCH_ROOT", "/app")
|
|
TARGET = "/litellm/litellm_core_utils/llm_response_utils/get_headers.py"
|
|
ANCHOR = ''' for k, v in response_headers.items():
|
|
if "llm_provider" not in k:
|
|
_key = "{}-{}".format("llm_provider", k)
|
|
llm_provider_headers[_key] = v
|
|
else:
|
|
llm_provider_headers[k] = v
|
|
return llm_provider_headers'''
|
|
REPLACEMENT = ''' # CO-OP PATCH (2026-10-27): do NOT forward provider headers to callers.
|
|
# Upstream prefixes EVERY response header with "llm_provider-", which leaked
|
|
# the co-op's remaining provider balance (llm_provider-x-credits-remaining)
|
|
# to any member API key. Delete this patch from the Dockerfile to revert.
|
|
return {}'''
|
|
|
|
candidates = glob.glob(f"{ROOT}/.venv/lib/python*/site-packages{TARGET}")
|
|
if not candidates:
|
|
candidates = glob.glob(f"{ROOT}/**{TARGET}", recursive=True)
|
|
|
|
patched_any = False
|
|
for p in candidates:
|
|
src = open(p).read()
|
|
if "CO-OP PATCH (2026-10-27)" in src:
|
|
print(f"INFO: provider-header strip already present in {p}")
|
|
patched_any = True
|
|
continue
|
|
if ANCHOR not in src:
|
|
print(f"WARN: anchor not found in {p} (upstream changed it); "
|
|
"provider headers are still forwarded (non-fatal)", file=sys.stderr)
|
|
continue
|
|
new = src.replace(ANCHOR, REPLACEMENT, 1)
|
|
try:
|
|
compile(new, p, "exec")
|
|
except SyntaxError as e:
|
|
print(f"WARN: patched source would not compile ({e}); not written "
|
|
"(non-fatal)", file=sys.stderr)
|
|
continue
|
|
with open(p, "w") as f:
|
|
f.write(new)
|
|
print("[coop-patch] applied: provider response headers are no longer "
|
|
f"forwarded to callers ({p})")
|
|
patched_any = True
|
|
|
|
if not candidates:
|
|
print("WARN: get_headers.py not found; provider headers are still forwarded "
|
|
"(non-fatal)", file=sys.stderr)
|
|
elif not patched_any:
|
|
print("WARN: no copy of get_headers.py patched (non-fatal)", file=sys.stderr)
|
|
|
|
sys.exit(0)
|