gateway: make /v1/audio/transcriptions work bare for voxtral-small-24b
LiteLLM's openai transcription transformation force-injects response_format=verbose_json when the caller omits it (to obtain a duration for per-second pricing). Tinfoil rejects verbose_json for voxtral-small-24b with a 400, so every bare transcription call failed — including the file+model shape members and Open WebUI dictation send. patch_transcription_response_format.py (new build-time patch, non-fatal): skip the injection for deployments whose model_info declares coop_no_verbose_json, with an explicit model-name gate as the reliable mechanism. Its billing (per-token via chat, per-request via the audio patch) does not need the duration. whisper-large-v3-turbo is untouched: Tinfoil serves verbose_json for it and dictation depends on the injection.
This commit is contained in:
1 parent
abaeb255b2
commit
943701581c
3 files changed
+125
No files matched your search
@@ -197,6 +197,11 @@ model_list:
|
||||
api_key: os.environ/TINFOIL_API_KEY
|
||||
model_info:
|
||||
mode: audio_transcription
|
||||
# Opt out of LiteLLM's forced response_format=verbose_json injection on
|
||||
# transcription calls (patch_transcription_response_format.py): Tinfoil
|
||||
# rejects verbose_json for this model with a 400, and nothing in its
|
||||
# billing needs the duration the injection exists to obtain.
|
||||
coop_no_verbose_json: true
|
||||
input_cost_per_token: 0.0000002
|
||||
output_cost_per_token: 0.0000006
|
||||
input_cost_per_request: 0.00008
|
||||
|
||||
Reference in new issue
Block a user