Files
litellm/patch_audio_cost_map.py
T
inference-admin b81177c3ad litellm: put Tinfoil's per-request audio prices in the shared cost map (#5)
The per-request helper reads litellm.model_cost, which is built from
model_prices_and_context_window.json — not from model_info. Router-side patching
therefore never priced voxtral-tts speech, and every call logged $0. This build
step writes the prices where the calculator looks, and fails the build if the file
is missing instead of warning.
2026-11-17 08:32:56 +00:00

65 lines
2.3 KiB
Python

"""Put Tinfoil's per-request audio prices in the SHARED cost map, at BUILD time.
Why this exists (issue #5): the cost calculator's per-request helper reads
``litellm.model_cost`` — the map built from model_prices_and_context_window.json —
and our config's ``input_cost_per_request`` never reaches it. Patching the router
to keep per-deployment pricing on the shared model_info entry was not enough,
because the helper consults the JSON-backed map, not model_info. Result: every
successful voxtral-tts speech call logged cost 0.0 (~$0.22/day of unlogged spend).
This is the same mechanism as patch_image_cost.py: re-state the price where the
calculator actually looks. It FAILS THE BUILD rather than warning, because a silent
"billed at $0" is exactly the failure mode we are trying to close.
Prices from the Tinfoil catalogue (2026-11-17): voxtral-tts $0.002/request,
whisper-large-v3-turbo $0.01/request, both $0 per token.
"""
import glob
import json
import sys
ENTRIES = {
"openai/voxtral-tts": {
"mode": "audio_speech",
"input_cost_per_request": 0.002,
"input_cost_per_token": 0.0,
"output_cost_per_token": 0.0,
},
"openai/whisper-large-v3-turbo": {
"mode": "audio_transcription",
"input_cost_per_request": 0.01,
"input_cost_per_token": 0.0,
"output_cost_per_token": 0.0,
},
}
def main() -> int:
files = glob.glob("/app/.venv/lib/python*/site-packages/litellm/"
"model_prices_and_context_window.json")
if not files:
print("[coop-patch] FAIL model_prices_and_context_window.json not found")
return 1
path = files[0]
with open(path, encoding="utf-8") as handle:
data = json.load(handle)
changed = []
for key, entry in ENTRIES.items():
current = data.get(key) or {}
if all(current.get(k) == v for k, v in entry.items()):
continue
merged = {**current, **entry}
data[key] = merged
changed.append(key)
if not changed:
print(f"[coop-patch] audio cost map already correct ({len(ENTRIES)} entries)")
return 0
with open(path, "w", encoding="utf-8") as handle:
json.dump(data, handle, indent=2)
print("[coop-patch] applied audio cost map: " + ", ".join(changed))
return 0
if __name__ == "__main__":
sys.exit(main())