gateway: fix TTS latency + restore audio warm path
- patch_audio_default_mp3.py: default /v1/audio/speech to response_format=mp3 when the caller omits it. LibreChat sends no response_format, and Tinfoil's voxtral-tts then serves WAV: measured ~8-17s warm (145-300KB) vs ~2.6-3.1s for mp3 (28-39KB, provider streams it). LibreChat labels the reply audio/mpeg regardless, so WAV bytes rode under an mp3 label. Explicit response_format from any caller still wins. - start.sh: scheduler tasks arrive as ARGUMENTS to this ENTRYPOINT script (docker run <image> <command>); they were booting a second tinfoil-proxy + LiteLLM and dying on the bind FATAL every run, so audio_keepalive never executed. Detect the arg and exec the task directly. - start.sh: own the warm path in the app container regardless of the scheduler container: ping tinfoil/voxtral-tts every 480s in the background (silent on success; failures append to /app/data/keepalive.log).
This commit is contained in:
1 parent
943701581c
commit
5d38403892
3 files changed
+139
No files matched your search
@@ -101,6 +101,14 @@ RUN chmod +x /app/code/audio_keepalive.py
|
||||
COPY patch_transcription_response_format.py /app/code/patch_transcription_response_format.py
|
||||
RUN python3 /app/code/patch_transcription_response_format.py
|
||||
|
||||
# TTS default response format: LibreChat sends no response_format on its TTS
|
||||
# request and Tinfoil's voxtral-tts then serves WAV (~8-17s warm measured) —
|
||||
# while LibreChat labels the reply audio/mpeg regardless. Default to mp3
|
||||
# (OpenAI's documented default, ~3s warm measured). Non-fatal; check the
|
||||
# "[coop-patch] applied" line in the build log on every LiteLLM upgrade.
|
||||
COPY patch_audio_default_mp3.py /app/code/patch_audio_default_mp3.py
|
||||
RUN python3 /app/code/patch_audio_default_mp3.py
|
||||
|
||||
# Override the base image's ENTRYPOINT (which is `litellm`) so our
|
||||
# startup script runs instead of being passed as an argument to litellm.
|
||||
ENTRYPOINT ["/app/code/start.sh"]
|
||||
|
||||
Reference in new issue
Block a user