The first audio deploy proved the config and the cost-calculator patch were both
live — /model/info served input_cost_per_request, and the build log confirmed
'[coop-patch] applied' — yet a real /v1/audio/speech and
/v1/audio/transcriptions pair still billed exactly $0.000000, with no spend row
anywhere. Diagnosis: the audio cost paths resolve pricing from the SHARED
cost-map key, and the router strips per-deployment pricing from that key, so the
lookup raises and the spend write is skipped. Same mechanism as the
image-generation case patch_image_cost.py already fixes.
The cost-calculator patch cannot cover this on its own: it adds a per-request
charge on top of a cost the normal path computes, and that path never got there.
This patch re-adds the pricing for audio_speech and audio_transcription
deployments, explicitly allowing input_cost_per_request through — LiteLLM does
not list it among its CustomPricing fields, which is precisely why the
cost-calculator patch has to read it.
Deploying the audio-pricing change proved the gap: the image built and the app
restarted, but /model/info still served the old prices — /app/data is a
persistent volume, so the Dockerfile's COPY config.yaml /app/data/config.yaml is
shadowed by the volume's file from the first install, and start.sh only wrote a
default when the file was missing. Config changes in this repo therefore never
reached a running gateway; the audit's byte-identical md5 check was a manual
sync.
Fix: ship a second copy at /app/code/config.yaml and have start.sh sync it over
the volume at boot whenever they differ. The repo stays the source of truth, the
legacy first-boot default remains, and the README documents the flow.
Tinfoil bills voxtral-tts $0.002 and whisper-large-v3-turbo $0.01 per REQUEST,
with $0 per token for both. Reading only the two token prices in its model list
made TTS look free and left STT billed at an older catalog's $0.05/1M tokens, so
neither matched what the co-op actually pays.
LiteLLM cannot express a per-request charge on the audio paths: per-request
pricing exists only on the search/rerank/MCP paths via input_cost_per_query, and
input_cost_per_request is read only by the proxy's cost-tracking settings, never
by the model cost calculator.
So patch_audio_request_cost.py, a build-time patch in the same shape as
patch_image_cost.py, makes response_cost_calculator honour a declared
input_cost_per_request. That function is the single funnel for every call type —
the proxy dispatches audio as `aspeech` and `transcription` and reaches it
through Logging._response_cost_calculator — so one insertion covers both. The
charge resolves from the per-deployment custom_pricing the logging layer builds
from our model_info, then the shared cost map, then call metadata, and the patch
is inert for any model that declares nothing.
config.yaml declares the real charges: $0.002/request for voxtral-tts, and
$0.01/request for whisper with its token price corrected to $0.
Verified before committing: the patch applies to the installed 1.103.2 anchors
and compiles; the helper's resolution order passes unit checks (all branches,
plus inert and garbage cases); the resulting config validates as YAML with all
12 entries intact. End-to-end spend is proven after deploy by a real TTS/STT call.
Refs code/litellm#5
Ubuntu