litellm: put Tinfoil's per-request audio prices in the shared cost map (#5)
The per-request helper reads litellm.model_cost, which is built from model_prices_and_context_window.json — not from model_info. Router-side patching therefore never priced voxtral-tts speech, and every call logged $0. This build step writes the prices where the calculator looks, and fails the build if the file is missing instead of warning.
This commit is contained in:
1 parent
d96df6fd5a
commit
b81177c3ad
2 files changed
+71
No files matched your search
@@ -59,6 +59,13 @@ RUN python3 /app/code/patch_audio_request_cost.py
|
|||||||
COPY patch_audio_router_pricing.py /app/code/patch_audio_router_pricing.py
|
COPY patch_audio_router_pricing.py /app/code/patch_audio_router_pricing.py
|
||||||
RUN python3 /app/code/patch_audio_router_pricing.py
|
RUN python3 /app/code/patch_audio_router_pricing.py
|
||||||
|
|
||||||
|
# Audio per-request prices must live in the SHARED cost map, which is built from
|
||||||
|
# model_prices_and_context_window.json — the per-request helper reads that map, not
|
||||||
|
# model_info, so a router-only patch left every speech call at $0 (issue #5).
|
||||||
|
# This patch fails the build rather than warning.
|
||||||
|
COPY patch_audio_cost_map.py /app/code/patch_audio_cost_map.py
|
||||||
|
RUN python3 /app/code/patch_audio_cost_map.py
|
||||||
|
|
||||||
# SECURITY (2026-10-27, member-disclosure finding 4): LiteLLM forwards every
|
# SECURITY (2026-10-27, member-disclosure finding 4): LiteLLM forwards every
|
||||||
# provider response header to callers as `llm_provider-*`, which leaked the
|
# provider response header to callers as `llm_provider-*`, which leaked the
|
||||||
# co-op's remaining provider balance (x-credits-remaining) to any member API key.
|
# co-op's remaining provider balance (x-credits-remaining) to any member API key.
|
||||||
|
|||||||
@@ -0,0 +1,64 @@
|
|||||||
|
"""Put Tinfoil's per-request audio prices in the SHARED cost map, at BUILD time.
|
||||||
|
|
||||||
|
Why this exists (issue #5): the cost calculator's per-request helper reads
|
||||||
|
``litellm.model_cost`` — the map built from model_prices_and_context_window.json —
|
||||||
|
and our config's ``input_cost_per_request`` never reaches it. Patching the router
|
||||||
|
to keep per-deployment pricing on the shared model_info entry was not enough,
|
||||||
|
because the helper consults the JSON-backed map, not model_info. Result: every
|
||||||
|
successful voxtral-tts speech call logged cost 0.0 (~$0.22/day of unlogged spend).
|
||||||
|
|
||||||
|
This is the same mechanism as patch_image_cost.py: re-state the price where the
|
||||||
|
calculator actually looks. It FAILS THE BUILD rather than warning, because a silent
|
||||||
|
"billed at $0" is exactly the failure mode we are trying to close.
|
||||||
|
|
||||||
|
Prices from the Tinfoil catalogue (2026-11-17): voxtral-tts $0.002/request,
|
||||||
|
whisper-large-v3-turbo $0.01/request, both $0 per token.
|
||||||
|
"""
|
||||||
|
import glob
|
||||||
|
import json
|
||||||
|
import sys
|
||||||
|
|
||||||
|
ENTRIES = {
|
||||||
|
"openai/voxtral-tts": {
|
||||||
|
"mode": "audio_speech",
|
||||||
|
"input_cost_per_request": 0.002,
|
||||||
|
"input_cost_per_token": 0.0,
|
||||||
|
"output_cost_per_token": 0.0,
|
||||||
|
},
|
||||||
|
"openai/whisper-large-v3-turbo": {
|
||||||
|
"mode": "audio_transcription",
|
||||||
|
"input_cost_per_request": 0.01,
|
||||||
|
"input_cost_per_token": 0.0,
|
||||||
|
"output_cost_per_token": 0.0,
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def main() -> int:
|
||||||
|
files = glob.glob("/app/.venv/lib/python*/site-packages/litellm/"
|
||||||
|
"model_prices_and_context_window.json")
|
||||||
|
if not files:
|
||||||
|
print("[coop-patch] FAIL model_prices_and_context_window.json not found")
|
||||||
|
return 1
|
||||||
|
path = files[0]
|
||||||
|
with open(path, encoding="utf-8") as handle:
|
||||||
|
data = json.load(handle)
|
||||||
|
changed = []
|
||||||
|
for key, entry in ENTRIES.items():
|
||||||
|
current = data.get(key) or {}
|
||||||
|
if all(current.get(k) == v for k, v in entry.items()):
|
||||||
|
continue
|
||||||
|
merged = {**current, **entry}
|
||||||
|
data[key] = merged
|
||||||
|
changed.append(key)
|
||||||
|
if not changed:
|
||||||
|
print(f"[coop-patch] audio cost map already correct ({len(ENTRIES)} entries)")
|
||||||
|
return 0
|
||||||
|
with open(path, "w", encoding="utf-8") as handle:
|
||||||
|
json.dump(data, handle, indent=2)
|
||||||
|
print("[coop-patch] applied audio cost map: " + ", ".join(changed))
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
sys.exit(main())
|
||||||
Reference in new issue
Block a user