Files
litellm/audio_keepalive.py
T
inference-bot 64d7e5afc5 gateway: single TTS warm path — drop the 10-min scheduler task, in-container loop every 15 min
The scheduler task ran audio_keepalive.py in its own container every 10 min
in parallel with the in-container loop (8 min): two warm paths billing the
same ping twice, and the scheduler container's access to 127.0.0.1:4000 and
/app/data was never proven. Keep the proven in-container path only, at a
15-minute interval (sleep 900).
2026-11-09 14:33:52 +00:00

52 lines
1.9 KiB
Python
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
#!/usr/bin/env python3
"""Audio keep-alive: ping the Tinfoil TEE audio models through localhost LiteLLM
so voice mode doesn't pay the ~24s cold-start after idle.
Runs INSIDE the gateway container, invoked by the keep-alive loop in
start.sh (every 15 min; the Cloudron scheduler task was removed
2026-11-09 so the loop is the only warm path). Reads the master key from
env or /app/data/.master_key. TTS-only (STT pings cost
$0.01/request each; TTS $0.002 — see docs, keep-alive economics). Silent on
success; prints + exits non-zero on failure so keepalive.log captures it.
"""
import json, os, urllib.request, urllib.error, sys
PORT = os.environ.get("PORT", "4000")
def master_key():
"""The gateway's start.sh generates the master key into /app/data/.master_key
and exports LITELLM_MASTER_KEY for the litellm process — scheduler tasks run
in a separate process tree, so read the key file directly."""
key = os.environ.get("LITELLM_MASTER_KEY", "")
if key:
return key
try:
with open("/app/data/.master_key") as f:
return f.read().strip()
except OSError:
return ""
KEY = master_key()
if not KEY:
print("keepalive: no master key (env or /app/data/.master_key)", file=sys.stderr)
sys.exit(1)
fail = 0
# TTS warm (voxtral-tts) — $0.002/request (Tinfoil request-priced)
req = urllib.request.Request(
f"http://127.0.0.1:{PORT}/v1/audio/speech",
data=json.dumps({"model": "tinfoil/voxtral-tts", "input": "ok",
"voice": "casual_female"}).encode(),
method="POST",
headers={"Authorization": f"Bearer {KEY}", "Content-Type": "application/json"})
try:
urllib.request.urlopen(req, timeout=30)
except Exception as e:
print(f"keepalive: TTS ping failed: {e}", file=sys.stderr)
fail = 1
# STT warm disabled 2026-10-11: $0.01/STT request × 6/hr ≈ $52/mo. Revisit only
# with a cheaper warm strategy.
sys.exit(fail)