gateway: make /v1/audio/transcriptions work bare for voxtral-small-24b
LiteLLM's openai transcription transformation force-injects response_format=verbose_json when the caller omits it (to obtain a duration for per-second pricing). Tinfoil rejects verbose_json for voxtral-small-24b with a 400, so every bare transcription call failed — including the file+model shape members and Open WebUI dictation send. patch_transcription_response_format.py (new build-time patch, non-fatal): skip the injection for deployments whose model_info declares coop_no_verbose_json, with an explicit model-name gate as the reliable mechanism. Its billing (per-token via chat, per-request via the audio patch) does not need the duration. whisper-large-v3-turbo is untouched: Tinfoil serves verbose_json for it and dictation depends on the injection.
This commit is contained in:
1 parent
abaeb255b2
commit
943701581c
3 files changed
+125
No files matched your search
@@ -92,6 +92,15 @@ RUN python3 /app/code/patch_passthrough_token_cost.py
|
||||
COPY audio_keepalive.py /app/code/audio_keepalive.py
|
||||
RUN chmod +x /app/code/audio_keepalive.py
|
||||
|
||||
# Skip LiteLLM's forced verbose_json on transcription requests for opting-out
|
||||
# models: Tinfoil's voxtral-small-24b rejects verbose_json with a 400, which
|
||||
# broke every bare /v1/audio/transcriptions call to it (file + model — the
|
||||
# shape members and dictation send). Its billing does not need the duration
|
||||
# the injection exists for. Non-fatal; check the "[coop-patch] applied" line
|
||||
# in the build log on every LiteLLM upgrade.
|
||||
COPY patch_transcription_response_format.py /app/code/patch_transcription_response_format.py
|
||||
RUN python3 /app/code/patch_transcription_response_format.py
|
||||
|
||||
# Override the base image's ENTRYPOINT (which is `litellm`) so our
|
||||
# startup script runs instead of being passed as an argument to litellm.
|
||||
ENTRYPOINT ["/app/code/start.sh"]
|
||||
|
||||
Reference in new issue
Block a user