- Dockerfile FROM v1.103.2 -> v1.108.3; manifest upstreamVersion follows.
- Carries main's newer build patches (patch_transcription_response_format.py,
patch_audio_default_mp3.py, audio_keepalive.py) that the v1.108.0 branch
predates, plus the 1.108.x CORS anchor fallback from that branch.
- Re-validated all ten patch scripts against the real v1.108.3 wheel sources:
every anchor matched and the patched files compile. patch_expose_cost_headers
applies via its 1.108.x anchor; patch_audio_request_cost needs
LITELLM_COST_CALCULATOR when run outside a full install (noted in README).
- Stop tracking __pycache__.
- patch_audio_default_mp3.py: default /v1/audio/speech to response_format=mp3
when the caller omits it. LibreChat sends no response_format, and Tinfoil's
voxtral-tts then serves WAV: measured ~8-17s warm (145-300KB) vs ~2.6-3.1s
for mp3 (28-39KB, provider streams it). LibreChat labels the reply
audio/mpeg regardless, so WAV bytes rode under an mp3 label. Explicit
response_format from any caller still wins.
- start.sh: scheduler tasks arrive as ARGUMENTS to this ENTRYPOINT script
(docker run <image> <command>); they were booting a second tinfoil-proxy +
LiteLLM and dying on the bind FATAL every run, so audio_keepalive never
executed. Detect the arg and exec the task directly.
- start.sh: own the warm path in the app container regardless of the
scheduler container: ping tinfoil/voxtral-tts every 480s in the background
(silent on success; failures append to /app/data/keepalive.log).
LiteLLM's openai transcription transformation force-injects
response_format=verbose_json when the caller omits it (to obtain a duration
for per-second pricing). Tinfoil rejects verbose_json for
voxtral-small-24b with a 400, so every bare transcription call failed —
including the file+model shape members and Open WebUI dictation send.
patch_transcription_response_format.py (new build-time patch, non-fatal):
skip the injection for deployments whose model_info declares
coop_no_verbose_json, with an explicit model-name gate as the reliable
mechanism. Its billing (per-token via chat, per-request via the audio
patch) does not need the duration. whisper-large-v3-turbo is untouched:
Tinfoil serves verbose_json for it and dictation depends on the injection.
The /v1/systemone pass-through is an EndpointType.GENERIC route: LiteLLM
logs it, but with no model on the route the only cost it can apply is the
flat cost_per_request (a budget estimate, not an invoice). So decisions
were billed at a fixed 0.0001 instead of the real $0.12 per 1M input
tokens GreenPT reports back in usage.input_tokens.
Price the route from its own response body via a new build-time patch,
patch_passthrough_token_cost.py, in the house style of the existing
patch_*.py files: any route listed in
general_settings.coop_passthrough_cost_per_token (USD per input token,
keyed by path) is billed as usage.input_tokens x rate. cost_per_request
stays as the fallback for a listed route whose response carries no token
count, and every unlisted route is unchanged. Auth is untouched.
config.yaml: list /v1/systemone at 0.00000012 ($0.12 per 1M input
tokens); update the metering note.
LiteLLM gates auth=true pass-through routes behind allowed_passthrough_routes,
a LiteLLM Enterprise field: on this community-edition proxy the management API
refuses to set it, so no member key can ever be granted access, and auth:false
would instead expose the route (and our GreenPT key) to anonymous callers.
Add patch_passthrough_key_access.py (house style: build-time, non-fatal,
idempotent, compile-checked) so routes the operator names in
general_settings.coop_passthrough_open_to_keys are callable by ANY
authenticated key. Authentication itself is unchanged: auth=true still installs
Depends(user_api_key_auth), so unauthenticated requests are still rejected.
Validated locally against LiteLLM 1.103.2: a plain generated key gets 200 with
real Metis answers; an unauthenticated POST gets 401.
start.sh hardcoded and exported LITELLM_CORS_ORIGINS, so the Cloudron app env that is meant to control it was silently overridden — the change I made earlier tonight looked applied and was inert. It now defaults only when unset, so the app env wins, and the co-op decision to allow all origins (co-op/support#38) actually takes effect. Comment rewritten for the current version and the reasoning. Also adds patch_expose_cost_headers.py (wired into the Dockerfile) so x-litellm-response-cost* can be read by cross-origin clients (co-op/support#39); upstream closed the request as stale, so it is handled at the app layer like the other patches.
LiteLLM copies every upstream response header onto its own response prefixed
'llm_provider-'. For our providers that hands callers operational detail about the
co-op's own account — verified live: llm_provider-x-credits-remaining carried the
remaining provider balance in every member's response.
Build-time patch makes _get_llm_provider_headers() return {}. The four OpenAI-style
x-ratelimit-* headers a provider may send are still forwarded: those describe the
caller's own request, not the co-op's account.
The first audio deploy proved the config and the cost-calculator patch were both
live — /model/info served input_cost_per_request, and the build log confirmed
'[coop-patch] applied' — yet a real /v1/audio/speech and
/v1/audio/transcriptions pair still billed exactly $0.000000, with no spend row
anywhere. Diagnosis: the audio cost paths resolve pricing from the SHARED
cost-map key, and the router strips per-deployment pricing from that key, so the
lookup raises and the spend write is skipped. Same mechanism as the
image-generation case patch_image_cost.py already fixes.
The cost-calculator patch cannot cover this on its own: it adds a per-request
charge on top of a cost the normal path computes, and that path never got there.
This patch re-adds the pricing for audio_speech and audio_transcription
deployments, explicitly allowing input_cost_per_request through — LiteLLM does
not list it among its CustomPricing fields, which is precisely why the
cost-calculator patch has to read it.
Deploying the audio-pricing change proved the gap: the image built and the app
restarted, but /model/info still served the old prices — /app/data is a
persistent volume, so the Dockerfile's COPY config.yaml /app/data/config.yaml is
shadowed by the volume's file from the first install, and start.sh only wrote a
default when the file was missing. Config changes in this repo therefore never
reached a running gateway; the audit's byte-identical md5 check was a manual
sync.
Fix: ship a second copy at /app/code/config.yaml and have start.sh sync it over
the volume at boot whenever they differ. The repo stays the source of truth, the
legacy first-boot default remains, and the README documents the flow.
Tinfoil bills voxtral-tts $0.002 and whisper-large-v3-turbo $0.01 per REQUEST,
with $0 per token for both. Reading only the two token prices in its model list
made TTS look free and left STT billed at an older catalog's $0.05/1M tokens, so
neither matched what the co-op actually pays.
LiteLLM cannot express a per-request charge on the audio paths: per-request
pricing exists only on the search/rerank/MCP paths via input_cost_per_query, and
input_cost_per_request is read only by the proxy's cost-tracking settings, never
by the model cost calculator.
So patch_audio_request_cost.py, a build-time patch in the same shape as
patch_image_cost.py, makes response_cost_calculator honour a declared
input_cost_per_request. That function is the single funnel for every call type —
the proxy dispatches audio as `aspeech` and `transcription` and reaches it
through Logging._response_cost_calculator — so one insertion covers both. The
charge resolves from the per-deployment custom_pricing the logging layer builds
from our model_info, then the shared cost map, then call metadata, and the patch
is inert for any model that declares nothing.
config.yaml declares the real charges: $0.002/request for voxtral-tts, and
$0.01/request for whisper with its token price corrected to $0.
Verified before committing: the patch applies to the installed 1.103.2 anchors
and compiles; the helper's resolution order passes unit checks (all branches,
plus inert and garbage cases); the resulting config validates as YAML with all
12 entries intact. End-to-end spend is proven after deploy by a real TTS/STT call.
Refs code/litellm#5
- Dockerfile: pin ghcr.io/berriai/litellm:v1.84.0 (was floating
main-v1.74.0-stable, which silently moved to a newer digest).
v1.84.0 is the fix line for CVE-2026-35029 (auth bypass on
/config/update, fixed 1.83.0) and CVE-2026-59822 (MCP session
auth bypass, fixed 1.84.0, CISA KEV).
- Drop the v1.74.0-era sed CORS patch: upstream v1.84.0 moved the
file and now reads LITELLM_CORS_ORIGINS natively.
- start.sh: export LITELLM_CORS_ORIGINS (the native v1.84.0 var)
instead of LITELLM_CORS_ALLOWED_ORIGINS (the old sed-patch var
that v1.84.0 ignores) — keeps CORS locked to chat.inference.coop.
- CloudronManifest: upstreamVersion 1.74.0 -> 1.84.0.
Live gateway already runs v1.84.0 (image digest
sha256:dc532d896ba8..., built 2026-10-02 04:50 UTC); this commit
makes the repo mirror the deployed packaging per the no-drift rule.
LITELLM_CORS_ORIGINS also set as a Cloudron env var on the app so
the running container honours it without a rebuild.