gateway: put audio pricing on the shared cost-map key (no SpendLogs row at all)

The first audio deploy proved the config and the cost-calculator patch were both
live — /model/info served input_cost_per_request, and the build log confirmed
'[coop-patch] applied' — yet a real /v1/audio/speech and
/v1/audio/transcriptions pair still billed exactly $0.000000, with no spend row
anywhere. Diagnosis: the audio cost paths resolve pricing from the SHARED
cost-map key, and the router strips per-deployment pricing from that key, so the
lookup raises and the spend write is skipped. Same mechanism as the
image-generation case patch_image_cost.py already fixes.

The cost-calculator patch cannot cover this on its own: it adds a per-request
charge on top of a cost the normal path computes, and that path never got there.

This patch re-adds the pricing for audio_speech and audio_transcription
deployments, explicitly allowing input_cost_per_request through — LiteLLM does
not list it among its CustomPricing fields, which is precisely why the
cost-calculator patch has to read it.
This commit is contained in:
Ubuntu committed 2026-10-26 21:24:42 +00:00
1 parent facf01d578
commit c03abaa330
2 files changed
+120

No files matched your search

+8
View File
@@ -46,6 +46,14 @@ RUN python3 /app/code/patch_image_cost.py
COPY patch_audio_request_cost.py /app/code/patch_audio_request_cost.py
RUN python3 /app/code/patch_audio_request_cost.py
# The audio cost paths read pricing from the SHARED cost-map key, which the
# router strips per-deployment pricing from — so the lookup raised, the spend
# write was skipped, and audio produced no SpendLogs row at all. This re-adds the
# pricing (and input_cost_per_request) for the audio modes. Same mechanism as
# patch_image_cost.py above.
COPY patch_audio_router_pricing.py /app/code/patch_audio_router_pricing.py
RUN python3 /app/code/patch_audio_router_pricing.py
# Audio keep-alive: scheduled pings keep the Tinfoil TEE audio enclave
# warm so voice mode skips the ~24s cold-start (see audio_keepalive.py header).
COPY audio_keepalive.py /app/code/audio_keepalive.py