gateway: single TTS warm path — drop the 10-min scheduler task, in-container loop every 15 min

The scheduler task ran audio_keepalive.py in its own container every 10 min
in parallel with the in-container loop (8 min): two warm paths billing the
same ping twice, and the scheduler container's access to 127.0.0.1:4000 and
/app/data was never proven. Keep the proven in-container path only, at a
15-minute interval (sleep 900).
This commit is contained in:
inference-bot committed 2026-11-09 14:33:52 +00:00
1 parent c43d45c455
commit 64d7e5afc5
4 files changed
+19 -22

No files matched your search

+1 -7
View File
@@ -16,13 +16,7 @@
"addons": {
"postgresql": {},
"redis": {},
"localstorage": {},
"scheduler": {
"audio_keepalive": {
"schedule": "*/10 * * * *",
"command": "/app/code/audio_keepalive.py"
}
}
"localstorage": {}
},
"tags": [
"ai",
+2 -2
View File
@@ -87,8 +87,8 @@ RUN python3 /app/code/patch_passthrough_key_access.py
COPY patch_passthrough_token_cost.py /app/code/patch_passthrough_token_cost.py
RUN python3 /app/code/patch_passthrough_token_cost.py
# Audio keep-alive: scheduled pings keep the Tinfoil TEE audio enclave
# warm so voice mode skips the ~24s cold-start (see audio_keepalive.py header).
# Audio keep-alive: the in-container loop in start.sh pings tinfoil/voxtral-tts
# every 15 min so voice mode skips the ~24s cold-start (see audio_keepalive.py).
COPY audio_keepalive.py /app/code/audio_keepalive.py
RUN chmod +x /app/code/audio_keepalive.py
+5 -3
View File
@@ -2,10 +2,12 @@
"""Audio keep-alive: ping the Tinfoil TEE audio models through localhost LiteLLM
so voice mode doesn't pay the ~24s cold-start after idle.
Runs INSIDE the gateway container via the Cloudron scheduler addon. Uses the
LITELLM_MASTER_KEY already in the container env. TTS-only (STT pings cost
Runs INSIDE the gateway container, invoked by the keep-alive loop in
start.sh (every 15 min; the Cloudron scheduler task was removed
2026-11-09 so the loop is the only warm path). Reads the master key from
env or /app/data/.master_key. TTS-only (STT pings cost
$0.01/request each; TTS $0.002 — see docs, keep-alive economics). Silent on
success; prints + exits non-zero on failure so scheduler logs capture it.
success; prints + exits non-zero on failure so keepalive.log captures it.
"""
import json, os, urllib.request, urllib.error, sys
+11 -10
View File
@@ -239,20 +239,21 @@ if [ "$proxy_up" != "1" ]; then
fi
echo "Tinfoil proxy is up on 127.0.0.1:3301"
# --- In-container audio keep-alive loop ---
# The scheduler task (handled above in scheduler-task mode) runs in a SEPARATE
# container whose ability to reach 127.0.0.1:4000 and read /app/data is not
# guaranteed, so the warm path is owned HERE — the app container, where the
# network namespace, the master key and the sidecar are all proven. Every 8
# minutes ping tinfoil/voxtral-tts ($0.002/request) so voice mode skips the
# ~24s Tinfoil cold-start after idle. audio_keepalive.py is silent on success
# and prints on failure, so /app/data/keepalive.log only ever grows on failure.
# --- In-container audio keep-alive loop (the ONLY warm path) ---
# A separate scheduler-task container was removed 2026-11-09: its container
# could not be proven to reach 127.0.0.1:4000 and /app/data, and two warm
# paths billed twice for the same ping. The warm path is owned HERE — the
# app container, where the network namespace, the master key and the
# sidecar are all proven. Every 15 minutes ping tinfoil/voxtral-tts
# ($0.002/request) so voice mode skips the ~24s Tinfoil cold-start after
# idle. audio_keepalive.py is silent on success and prints on failure, so
# /app/data/keepalive.log only ever grows on failure.
(
while true; do
python3 /app/code/audio_keepalive.py >> /app/data/keepalive.log 2>&1 || true
sleep 480
sleep 900
done
) &
echo "Audio keep-alive loop started (every 480s)"
echo "Audio keep-alive loop started (every 900s)"
exec litellm --config "$CONFIG_FILE" --port 4000 --host 0.0.0.0