gateway: single TTS warm path — drop the 10-min scheduler task, in-container loop every 15 min

The scheduler task ran audio_keepalive.py in its own container every 10 min
in parallel with the in-container loop (8 min): two warm paths billing the
same ping twice, and the scheduler container's access to 127.0.0.1:4000 and
/app/data was never proven. Keep the proven in-container path only, at a
15-minute interval (sleep 900).
This commit is contained in:
inference-bot committed 2026-11-09 14:33:52 +00:00
1 parent c43d45c455
commit 64d7e5afc5
4 files changed
+19 -22

No files matched your search

+1 -7
View File
@@ -16,13 +16,7 @@
"addons": { "addons": {
"postgresql": {}, "postgresql": {},
"redis": {}, "redis": {},
"localstorage": {}, "localstorage": {}
"scheduler": {
"audio_keepalive": {
"schedule": "*/10 * * * *",
"command": "/app/code/audio_keepalive.py"
}
}
}, },
"tags": [ "tags": [
"ai", "ai",
+2 -2
View File
@@ -87,8 +87,8 @@ RUN python3 /app/code/patch_passthrough_key_access.py
COPY patch_passthrough_token_cost.py /app/code/patch_passthrough_token_cost.py COPY patch_passthrough_token_cost.py /app/code/patch_passthrough_token_cost.py
RUN python3 /app/code/patch_passthrough_token_cost.py RUN python3 /app/code/patch_passthrough_token_cost.py
# Audio keep-alive: scheduled pings keep the Tinfoil TEE audio enclave # Audio keep-alive: the in-container loop in start.sh pings tinfoil/voxtral-tts
# warm so voice mode skips the ~24s cold-start (see audio_keepalive.py header). # every 15 min so voice mode skips the ~24s cold-start (see audio_keepalive.py).
COPY audio_keepalive.py /app/code/audio_keepalive.py COPY audio_keepalive.py /app/code/audio_keepalive.py
RUN chmod +x /app/code/audio_keepalive.py RUN chmod +x /app/code/audio_keepalive.py
+5 -3
View File
@@ -2,10 +2,12 @@
"""Audio keep-alive: ping the Tinfoil TEE audio models through localhost LiteLLM """Audio keep-alive: ping the Tinfoil TEE audio models through localhost LiteLLM
so voice mode doesn't pay the ~24s cold-start after idle. so voice mode doesn't pay the ~24s cold-start after idle.
Runs INSIDE the gateway container via the Cloudron scheduler addon. Uses the Runs INSIDE the gateway container, invoked by the keep-alive loop in
LITELLM_MASTER_KEY already in the container env. TTS-only (STT pings cost start.sh (every 15 min; the Cloudron scheduler task was removed
2026-11-09 so the loop is the only warm path). Reads the master key from
env or /app/data/.master_key. TTS-only (STT pings cost
$0.01/request each; TTS $0.002 — see docs, keep-alive economics). Silent on $0.01/request each; TTS $0.002 — see docs, keep-alive economics). Silent on
success; prints + exits non-zero on failure so scheduler logs capture it. success; prints + exits non-zero on failure so keepalive.log captures it.
""" """
import json, os, urllib.request, urllib.error, sys import json, os, urllib.request, urllib.error, sys
+11 -10
View File
@@ -239,20 +239,21 @@ if [ "$proxy_up" != "1" ]; then
fi fi
echo "Tinfoil proxy is up on 127.0.0.1:3301" echo "Tinfoil proxy is up on 127.0.0.1:3301"
# --- In-container audio keep-alive loop --- # --- In-container audio keep-alive loop (the ONLY warm path) ---
# The scheduler task (handled above in scheduler-task mode) runs in a SEPARATE # A separate scheduler-task container was removed 2026-11-09: its container
# container whose ability to reach 127.0.0.1:4000 and read /app/data is not # could not be proven to reach 127.0.0.1:4000 and /app/data, and two warm
# guaranteed, so the warm path is owned HERE — the app container, where the # paths billed twice for the same ping. The warm path is owned HERE — the
# network namespace, the master key and the sidecar are all proven. Every 8 # app container, where the network namespace, the master key and the
# minutes ping tinfoil/voxtral-tts ($0.002/request) so voice mode skips the # sidecar are all proven. Every 15 minutes ping tinfoil/voxtral-tts
# ~24s Tinfoil cold-start after idle. audio_keepalive.py is silent on success # ($0.002/request) so voice mode skips the ~24s Tinfoil cold-start after
# and prints on failure, so /app/data/keepalive.log only ever grows on failure. # idle. audio_keepalive.py is silent on success and prints on failure, so
# /app/data/keepalive.log only ever grows on failure.
( (
while true; do while true; do
python3 /app/code/audio_keepalive.py >> /app/data/keepalive.log 2>&1 || true python3 /app/code/audio_keepalive.py >> /app/data/keepalive.log 2>&1 || true
sleep 480 sleep 900
done done
) & ) &
echo "Audio keep-alive loop started (every 480s)" echo "Audio keep-alive loop started (every 900s)"
exec litellm --config "$CONFIG_FILE" --port 4000 --host 0.0.0.0 exec litellm --config "$CONFIG_FILE" --port 4000 --host 0.0.0.0