Speech billed $0.000000 across three deploys while transcription billed
$0.010000 exactly, so the patch works for one audio call type and not the other.
Three hypotheses have now been retired (hidden-params early return; missing
character price; lookup miss) — the remaining question is WHERE the speech cost
is lost, and guessing is more expensive than measuring.
The helper now prints, for TTS/STT models only, whether it ran at all, what it
could see (custom_pricing keys, shared cost-map hits) and what it resolved, into
the app log. If those lines never appear for a speech call, the request never
reaches the model cost calculator and the loss is upstream of this patch.
Also fixes a real defect in the previous revision: the addition was wrapped in a
bare except, so a TypeError from a None base cost would have been swallowed
silently — which is exactly the sort of silence that made this take three
deploys. It now coerces a None base to 0.0 and reports any failure.
Speech still billed nothing after the router patch put its pricing on the shared
cost map. Cause, read out of cost_per_token: for a speech model
select_cost_metric_for_model picks "cost_per_character", and that branch raises
ValueError("cost for tts call is None") when no character price is declared. The
raise aborts the whole cost calculation, so the spend write is skipped and the
per-request patch never runs — the entry being unpriced per character made the
per-request charge unreachable.
Declaring input_cost_per_character: 0 lets the path complete with a zero
per-unit cost, after which the patch adds the real $0.002 per request.
Measured, not assumed: transcription bills $0.010000 exactly (whisper's token
price is $0, so that is entirely the per-request charge), while speech sat at
$0.000000 across three deploys.
The first audio deploy proved the config and the cost-calculator patch were both
live — /model/info served input_cost_per_request, and the build log confirmed
'[coop-patch] applied' — yet a real /v1/audio/speech and
/v1/audio/transcriptions pair still billed exactly $0.000000, with no spend row
anywhere. Diagnosis: the audio cost paths resolve pricing from the SHARED
cost-map key, and the router strips per-deployment pricing from that key, so the
lookup raises and the spend write is skipped. Same mechanism as the
image-generation case patch_image_cost.py already fixes.
The cost-calculator patch cannot cover this on its own: it adds a per-request
charge on top of a cost the normal path computes, and that path never got there.
This patch re-adds the pricing for audio_speech and audio_transcription
deployments, explicitly allowing input_cost_per_request through — LiteLLM does
not list it among its CustomPricing fields, which is precisely why the
cost-calculator patch has to read it.
Deploying the audio-pricing change proved the gap: the image built and the app
restarted, but /model/info still served the old prices — /app/data is a
persistent volume, so the Dockerfile's COPY config.yaml /app/data/config.yaml is
shadowed by the volume's file from the first install, and start.sh only wrote a
default when the file was missing. Config changes in this repo therefore never
reached a running gateway; the audit's byte-identical md5 check was a manual
sync.
Fix: ship a second copy at /app/code/config.yaml and have start.sh sync it over
the volume at boot whenever they differ. The repo stays the source of truth, the
legacy first-boot default remains, and the README documents the flow.
Tinfoil bills voxtral-tts $0.002 and whisper-large-v3-turbo $0.01 per REQUEST,
with $0 per token for both. Reading only the two token prices in its model list
made TTS look free and left STT billed at an older catalog's $0.05/1M tokens, so
neither matched what the co-op actually pays.
LiteLLM cannot express a per-request charge on the audio paths: per-request
pricing exists only on the search/rerank/MCP paths via input_cost_per_query, and
input_cost_per_request is read only by the proxy's cost-tracking settings, never
by the model cost calculator.
So patch_audio_request_cost.py, a build-time patch in the same shape as
patch_image_cost.py, makes response_cost_calculator honour a declared
input_cost_per_request. That function is the single funnel for every call type —
the proxy dispatches audio as `aspeech` and `transcription` and reaches it
through Logging._response_cost_calculator — so one insertion covers both. The
charge resolves from the per-deployment custom_pricing the logging layer builds
from our model_info, then the shared cost map, then call metadata, and the patch
is inert for any model that declares nothing.
config.yaml declares the real charges: $0.002/request for voxtral-tts, and
$0.01/request for whisper with its token price corrected to $0.
Verified before committing: the patch applies to the installed 1.103.2 anchors
and compiles; the helper's resolution order passes unit checks (all branches,
plus inert and garbage cases); the resulting config validates as YAML with all
12 entries intact. End-to-end spend is proven after deploy by a real TTS/STT call.
Refs code/litellm#5
Ubuntu