start.sh hardcoded and exported LITELLM_CORS_ORIGINS, so the Cloudron app env that is meant to control it was silently overridden — the change I made earlier tonight looked applied and was inert. It now defaults only when unset, so the app env wins, and the co-op decision to allow all origins (co-op/support#38) actually takes effect. Comment rewritten for the current version and the reasoning. Also adds patch_expose_cost_headers.py (wired into the Dockerfile) so x-litellm-response-cost* can be read by cross-origin clients (co-op/support#39); upstream closed the request as stale, so it is handled at the app layer like the other patches.
LiteLLM copies every upstream response header onto its own response prefixed
'llm_provider-'. For our providers that hands callers operational detail about the
co-op's own account — verified live: llm_provider-x-credits-remaining carried the
remaining provider balance in every member's response.
Build-time patch makes _get_llm_provider_headers() return {}. The four OpenAI-style
x-ratelimit-* headers a provider may send are still forwarded: those describe the
caller's own request, not the co-op's account.
The 3301 'address already in use' on process-level restarts left an unmanaged
old proxy serving (pre-update code/flags) while the new one died. Also: if the
proxy never binds, exit 1 instead of exec-ing LiteLLM against a dead 3301 —
that masqueraded as a provider outage. Cache continuity: with
TINFOIL_USER_CACHE_SECRET now pinned (env, 2026-10-22), restarts keep Tinfoil's
prompt-cache warm either way.
The 2026-10-02 outage had three causes the runbook now prevents:
- floating tag drift (README now mandates exact-tag pinning)
- 256MB default cgroup cap OOM-killing the prisma migration engine
(manifest now sets memoryLimit: 2147483648)
- no _prisma_migrations ledger -> P3005 crash-loop on boot
(start.sh now exports LITELLM_MIGRATION_DIR=/app/data/migrations)
README 'Updating LiteLLM' section rewritten from the old (wrong)
floating-tag procedure into a runbook with rules + verify checklist.
- Dockerfile: pin ghcr.io/berriai/litellm:v1.84.0 (was floating
main-v1.74.0-stable, which silently moved to a newer digest).
v1.84.0 is the fix line for CVE-2026-35029 (auth bypass on
/config/update, fixed 1.83.0) and CVE-2026-59822 (MCP session
auth bypass, fixed 1.84.0, CISA KEV).
- Drop the v1.74.0-era sed CORS patch: upstream v1.84.0 moved the
file and now reads LITELLM_CORS_ORIGINS natively.
- start.sh: export LITELLM_CORS_ORIGINS (the native v1.84.0 var)
instead of LITELLM_CORS_ALLOWED_ORIGINS (the old sed-patch var
that v1.84.0 ignores) — keeps CORS locked to chat.inference.coop.
- CloudronManifest: upstreamVersion 1.74.0 -> 1.84.0.
Live gateway already runs v1.84.0 (image digest
sha256:dc532d896ba8..., built 2026-10-02 04:50 UTC); this commit
makes the repo mirror the deployed packaging per the no-drift rule.
LITELLM_CORS_ORIGINS also set as a Cloudron env var on the app so
the running container honours it without a rebuild.
num_retries=2 (retries transient connection errors), retry_policy with
TimeoutErrorRetries=0 (a timed-out generation is already billed by Tinfoil, so
re-hitting it re-bills). Combined with timeout=180, this hedges connection
errors without the silent re-billing that caused the Tinfoil spend gap.
DeepSeek V4.1 Flash is a reasoning model; 30s timeout caused LiteLLM to
abandon in-flight generations (falling back to GPT-OSS) while Tinfoil still
completed and billed them. num_retries re-hit Tinfoil up to 3x. Result: Tinfoil
billed ~2x the tokens LiteLLM logged. Generous timeout + zero retries stops
the silent re-billing.
inference-bot
inference-bot