Tinfoil's catalog prices exactly one audio model per token: voxtral-small-24b
($0.20 in / $0.60 out per 1M, $0 per request). Wiring it gives members an
STT route without the whisper $0.01/request floor (co-op/support#48).
Two billing shapes on one model entry:
- chat route (input_audio): returns real token usage -> bills exactly per token.
- /v1/audio/transcriptions: Tinfoil returns NO token counts (duration only),
so per-token billing cannot fire; Tinfoil also tokenizes audio in ~30s
chunks (~375-380 tokens each regardless of clip length), making the
chunk-equivalent ~$0.00008/request. Declared input_cost_per_request with
that measured value, added by patch_audio_request_cost.py.
patch_audio_request_cost.py: scope the per-request charge to the audio call
types (transcription/atranscription/speech/aspeech) so this model's chat
calls bill per token and are not double-billed; extend the diagnostic
name-gate to 'voxtral'.
Measured on the live catalog + sidecar 2026-11-06 (see issue #48 follow-up).
Speech billed $0.000000 across three deploys while transcription billed
$0.010000 exactly, so the patch works for one audio call type and not the other.
Three hypotheses have now been retired (hidden-params early return; missing
character price; lookup miss) — the remaining question is WHERE the speech cost
is lost, and guessing is more expensive than measuring.
The helper now prints, for TTS/STT models only, whether it ran at all, what it
could see (custom_pricing keys, shared cost-map hits) and what it resolved, into
the app log. If those lines never appear for a speech call, the request never
reaches the model cost calculator and the loss is upstream of this patch.
Also fixes a real defect in the previous revision: the addition was wrapped in a
bare except, so a TypeError from a None base cost would have been swallowed
silently — which is exactly the sort of silence that made this take three
deploys. It now coerces a None base to 0.0 and reports any failure.
Tinfoil bills voxtral-tts $0.002 and whisper-large-v3-turbo $0.01 per REQUEST,
with $0 per token for both. Reading only the two token prices in its model list
made TTS look free and left STT billed at an older catalog's $0.05/1M tokens, so
neither matched what the co-op actually pays.
LiteLLM cannot express a per-request charge on the audio paths: per-request
pricing exists only on the search/rerank/MCP paths via input_cost_per_query, and
input_cost_per_request is read only by the proxy's cost-tracking settings, never
by the model cost calculator.
So patch_audio_request_cost.py, a build-time patch in the same shape as
patch_image_cost.py, makes response_cost_calculator honour a declared
input_cost_per_request. That function is the single funnel for every call type —
the proxy dispatches audio as `aspeech` and `transcription` and reaches it
through Logging._response_cost_calculator — so one insertion covers both. The
charge resolves from the per-deployment custom_pricing the logging layer builds
from our model_info, then the shared cost map, then call metadata, and the patch
is inert for any model that declares nothing.
config.yaml declares the real charges: $0.002/request for voxtral-tts, and
$0.01/request for whisper with its token price corrected to $0.
Verified before committing: the patch applies to the installed 1.103.2 anchors
and compiles; the helper's resolution order passes unit checks (all branches,
plus inert and garbage cases); the resulting config validates as YAML with all
12 entries intact. End-to-end spend is proven after deploy by a real TTS/STT call.
Refs code/litellm#5
inference-bot
Ubuntu