gateway: put audio pricing on the shared cost-map key (no SpendLogs row at all)
The first audio deploy proved the config and the cost-calculator patch were both live — /model/info served input_cost_per_request, and the build log confirmed '[coop-patch] applied' — yet a real /v1/audio/speech and /v1/audio/transcriptions pair still billed exactly $0.000000, with no spend row anywhere. Diagnosis: the audio cost paths resolve pricing from the SHARED cost-map key, and the router strips per-deployment pricing from that key, so the lookup raises and the spend write is skipped. Same mechanism as the image-generation case patch_image_cost.py already fixes. The cost-calculator patch cannot cover this on its own: it adds a per-request charge on top of a cost the normal path computes, and that path never got there. This patch re-adds the pricing for audio_speech and audio_transcription deployments, explicitly allowing input_cost_per_request through — LiteLLM does not list it among its CustomPricing fields, which is precisely why the cost-calculator patch has to read it.
This commit is contained in:
1 parent
facf01d578
commit
c03abaa330
2 files changed
+120
No files matched your search
@@ -46,6 +46,14 @@ RUN python3 /app/code/patch_image_cost.py
|
||||
COPY patch_audio_request_cost.py /app/code/patch_audio_request_cost.py
|
||||
RUN python3 /app/code/patch_audio_request_cost.py
|
||||
|
||||
# The audio cost paths read pricing from the SHARED cost-map key, which the
|
||||
# router strips per-deployment pricing from — so the lookup raised, the spend
|
||||
# write was skipped, and audio produced no SpendLogs row at all. This re-adds the
|
||||
# pricing (and input_cost_per_request) for the audio modes. Same mechanism as
|
||||
# patch_image_cost.py above.
|
||||
COPY patch_audio_router_pricing.py /app/code/patch_audio_router_pricing.py
|
||||
RUN python3 /app/code/patch_audio_router_pricing.py
|
||||
|
||||
# Audio keep-alive: scheduled pings keep the Tinfoil TEE audio enclave
|
||||
# warm so voice mode skips the ~24s cold-start (see audio_keepalive.py header).
|
||||
COPY audio_keepalive.py /app/code/audio_keepalive.py
|
||||
|
||||
Reference in new issue
Block a user