gateway: single TTS warm path — drop the 10-min scheduler task, in-container loop every 15 min
The scheduler task ran audio_keepalive.py in its own container every 10 min in parallel with the in-container loop (8 min): two warm paths billing the same ping twice, and the scheduler container's access to 127.0.0.1:4000 and /app/data was never proven. Keep the proven in-container path only, at a 15-minute interval (sleep 900).
This commit is contained in:
1 parent
c43d45c455
commit
64d7e5afc5
4 files changed
+19
-22
No files matched your search
@@ -16,13 +16,7 @@
|
||||
"addons": {
|
||||
"postgresql": {},
|
||||
"redis": {},
|
||||
"localstorage": {},
|
||||
"scheduler": {
|
||||
"audio_keepalive": {
|
||||
"schedule": "*/10 * * * *",
|
||||
"command": "/app/code/audio_keepalive.py"
|
||||
}
|
||||
}
|
||||
"localstorage": {}
|
||||
},
|
||||
"tags": [
|
||||
"ai",
|
||||
|
||||
+2
-2
@@ -87,8 +87,8 @@ RUN python3 /app/code/patch_passthrough_key_access.py
|
||||
COPY patch_passthrough_token_cost.py /app/code/patch_passthrough_token_cost.py
|
||||
RUN python3 /app/code/patch_passthrough_token_cost.py
|
||||
|
||||
# Audio keep-alive: scheduled pings keep the Tinfoil TEE audio enclave
|
||||
# warm so voice mode skips the ~24s cold-start (see audio_keepalive.py header).
|
||||
# Audio keep-alive: the in-container loop in start.sh pings tinfoil/voxtral-tts
|
||||
# every 15 min so voice mode skips the ~24s cold-start (see audio_keepalive.py).
|
||||
COPY audio_keepalive.py /app/code/audio_keepalive.py
|
||||
RUN chmod +x /app/code/audio_keepalive.py
|
||||
|
||||
|
||||
+5
-3
@@ -2,10 +2,12 @@
|
||||
"""Audio keep-alive: ping the Tinfoil TEE audio models through localhost LiteLLM
|
||||
so voice mode doesn't pay the ~24s cold-start after idle.
|
||||
|
||||
Runs INSIDE the gateway container via the Cloudron scheduler addon. Uses the
|
||||
LITELLM_MASTER_KEY already in the container env. TTS-only (STT pings cost
|
||||
Runs INSIDE the gateway container, invoked by the keep-alive loop in
|
||||
start.sh (every 15 min; the Cloudron scheduler task was removed
|
||||
2026-11-09 so the loop is the only warm path). Reads the master key from
|
||||
env or /app/data/.master_key. TTS-only (STT pings cost
|
||||
$0.01/request each; TTS $0.002 — see docs, keep-alive economics). Silent on
|
||||
success; prints + exits non-zero on failure so scheduler logs capture it.
|
||||
success; prints + exits non-zero on failure so keepalive.log captures it.
|
||||
"""
|
||||
import json, os, urllib.request, urllib.error, sys
|
||||
|
||||
|
||||
@@ -239,20 +239,21 @@ if [ "$proxy_up" != "1" ]; then
|
||||
fi
|
||||
echo "Tinfoil proxy is up on 127.0.0.1:3301"
|
||||
|
||||
# --- In-container audio keep-alive loop ---
|
||||
# The scheduler task (handled above in scheduler-task mode) runs in a SEPARATE
|
||||
# container whose ability to reach 127.0.0.1:4000 and read /app/data is not
|
||||
# guaranteed, so the warm path is owned HERE — the app container, where the
|
||||
# network namespace, the master key and the sidecar are all proven. Every 8
|
||||
# minutes ping tinfoil/voxtral-tts ($0.002/request) so voice mode skips the
|
||||
# ~24s Tinfoil cold-start after idle. audio_keepalive.py is silent on success
|
||||
# and prints on failure, so /app/data/keepalive.log only ever grows on failure.
|
||||
# --- In-container audio keep-alive loop (the ONLY warm path) ---
|
||||
# A separate scheduler-task container was removed 2026-11-09: its container
|
||||
# could not be proven to reach 127.0.0.1:4000 and /app/data, and two warm
|
||||
# paths billed twice for the same ping. The warm path is owned HERE — the
|
||||
# app container, where the network namespace, the master key and the
|
||||
# sidecar are all proven. Every 15 minutes ping tinfoil/voxtral-tts
|
||||
# ($0.002/request) so voice mode skips the ~24s Tinfoil cold-start after
|
||||
# idle. audio_keepalive.py is silent on success and prints on failure, so
|
||||
# /app/data/keepalive.log only ever grows on failure.
|
||||
(
|
||||
while true; do
|
||||
python3 /app/code/audio_keepalive.py >> /app/data/keepalive.log 2>&1 || true
|
||||
sleep 480
|
||||
sleep 900
|
||||
done
|
||||
) &
|
||||
echo "Audio keep-alive loop started (every 480s)"
|
||||
echo "Audio keep-alive loop started (every 900s)"
|
||||
|
||||
exec litellm --config "$CONFIG_FILE" --port 4000 --host 0.0.0.0
|
||||
Reference in new issue
Block a user