Commit Graph
1 Commits
Author SHA1 Message Date
inference-bot 943701581c gateway: make /v1/audio/transcriptions work bare for voxtral-small-24b
LiteLLM's openai transcription transformation force-injects
response_format=verbose_json when the caller omits it (to obtain a duration
for per-second pricing). Tinfoil rejects verbose_json for
voxtral-small-24b with a 400, so every bare transcription call failed —
including the file+model shape members and Open WebUI dictation send.

patch_transcription_response_format.py (new build-time patch, non-fatal):
skip the injection for deployments whose model_info declares
coop_no_verbose_json, with an explicit model-name gate as the reliable
mechanism. Its billing (per-token via chat, per-request via the audio
patch) does not need the duration. whisper-large-v3-turbo is untouched:
Tinfoil serves verbose_json for it and dictation depends on the injection.
2026-11-06 22:40:53 +00:00