gateway: add tinfoil/voxtral-small-24b, Tinfoil's token-priced STT model

Tinfoil's catalog prices exactly one audio model per token: voxtral-small-24b
($0.20 in / $0.60 out per 1M, $0 per request). Wiring it gives members an
STT route without the whisper $0.01/request floor (co-op/support#48).

Two billing shapes on one model entry:
- chat route (input_audio): returns real token usage -> bills exactly per token.
- /v1/audio/transcriptions: Tinfoil returns NO token counts (duration only),
  so per-token billing cannot fire; Tinfoil also tokenizes audio in ~30s
  chunks (~375-380 tokens each regardless of clip length), making the
  chunk-equivalent ~$0.00008/request. Declared input_cost_per_request with
  that measured value, added by patch_audio_request_cost.py.

patch_audio_request_cost.py: scope the per-request charge to the audio call
types (transcription/atranscription/speech/aspeech) so this model's chat
calls bill per token and are not double-billed; extend the diagnostic
name-gate to 'voxtral'.

Measured on the live catalog + sidecar 2026-11-06 (see issue #48 follow-up).
This commit is contained in:
inference-bot committed 2026-11-06 22:33:17 +00:00
1 parent 54925cae97
commit 67df968a04
2 files changed
+61 -9

No files matched your search

+27
View File
@@ -11,6 +11,8 @@
# tinfoil/gpt-oss-120b: $0.15 in / $0.60 out per 1M
# tinfoil/glm-5.3: $1.80 in / $5.75 out / $0.45 cached per 1M
# greenpt/qwen3-embedding-8b: $0.25 in / $0 out per 1M (embeddings, 32K ctx)
# tinfoil/voxtral-small-24b: $0.20 in / $0.60 out per 1M (token-priced audio;
# +$0.00008/request on /v1/audio/transcriptions)
# Without these, LiteLLM can't price these custom models, so spend shows $0.00
# and the team budget cap is not enforced.
#
@@ -151,6 +153,8 @@ model_list:
# picker, but they remain fully callable via /v1/audio/* for API users.
# whisper-large-v3-turbo: $0/1M tokens + $0.01 per request.
# voxtral-tts: $0/1M tokens + $0.002 per request.
# voxtral-small-24b: $0.20/1M in + $0.60/1M out tokens, $0 per request —
# the only TOKEN-PRICED audio model in their catalog (its block below).
# Tinfoil bills these per REQUEST (its model list carries a `requestPrice`);
# reading only the two token prices made TTS look free and left STT billed off
# an older catalog. LiteLLM cannot price per request on the audio paths, so
@@ -172,6 +176,29 @@ model_list:
input_cost_per_token: 0
input_cost_per_request: 0.01
# voxtral-small-24b — Tinfoil's TOKEN-PRICED STT model: the only audio entry
# in their catalog with real token prices ($0.20 in / $0.60 out per 1M, $0
# per request). The chat route (audio sent as input_audio) returns real
# token usage and bills exactly at those rates. The /v1/audio/transcriptions
# route returns NO token counts (duration only), so per-token billing cannot
# fire there; Tinfoil tokenizes audio in ~30s chunks (~375-380 tokens each —
# measured: a 1-second clip and a 6-second clip both cost ~380 prompt
# tokens), so for the short dictation requests this route serves the
# chunk-equivalent is ~$0.00008/request — declared as input_cost_per_request
# below, added by patch_audio_request_cost.py, which is scoped to the audio
# call types so chat calls to this model are NOT double-billed. Audio longer
# than ~30s should go through the chat route, which meters exactly.
- model_name: tinfoil/voxtral-small-24b
litellm_params:
model: openai/voxtral-small-24b
api_base: http://127.0.0.1:3301/v1
api_key: os.environ/TINFOIL_API_KEY
model_info:
mode: audio_transcription
input_cost_per_token: 0.0000002
output_cost_per_token: 0.0000006
input_cost_per_request: 0.00008
- model_name: tinfoil/voxtral-tts
litellm_params:
model: openai/voxtral-tts