gateway: price the Metis pass-through route from its reported tokens

The /v1/systemone pass-through is an EndpointType.GENERIC route: LiteLLM
logs it, but with no model on the route the only cost it can apply is the
flat cost_per_request (a budget estimate, not an invoice). So decisions
were billed at a fixed 0.0001 instead of the real $0.12 per 1M input
tokens GreenPT reports back in usage.input_tokens.

Price the route from its own response body via a new build-time patch,
patch_passthrough_token_cost.py, in the house style of the existing
patch_*.py files: any route listed in
general_settings.coop_passthrough_cost_per_token (USD per input token,
keyed by path) is billed as usage.input_tokens x rate. cost_per_request
stays as the fallback for a listed route whose response carries no token
count, and every unlisted route is unchanged. Auth is untouched.

config.yaml: list /v1/systemone at 0.00000012 ($0.12 per 1M input
tokens); update the metering note.
This commit is contained in:
inference-bot committed 2026-11-03 18:58:07 +00:00
1 parent 557999a16f
commit da02f8eb32
4 files changed
+221 -6

No files matched your search

+11
View File
@@ -76,6 +76,17 @@ RUN python3 /app/code/patch_expose_cost_headers.py
COPY patch_passthrough_key_access.py /app/code/patch_passthrough_key_access.py
RUN python3 /app/code/patch_passthrough_key_access.py
# Exact per-input-token pricing for generic pass-through routes. A route whose
# target matches none of vertex/anthropic/openai/gemini resolves to
# EndpointType.GENERIC: it IS logged, but LiteLLM has no model to price it from,
# so the only cost it knows is the flat cost_per_request. Metis (/v1/systemone)
# is billed at $0.12 per 1M input tokens and returns usage.input_tokens, so this
# patch prices a route listed in general_settings.coop_passthrough_cost_per_token
# from that count and writes the exact figure to member spend. Non-fatal; check
# the "[coop-patch] applied" line in the build log on every LiteLLM upgrade.
COPY patch_passthrough_token_cost.py /app/code/patch_passthrough_token_cost.py
RUN python3 /app/code/patch_passthrough_token_cost.py
# Audio keep-alive: scheduled pings keep the Tinfoil TEE audio enclave
# warm so voice mode skips the ~24s cold-start (see audio_keepalive.py header).
COPY audio_keepalive.py /app/code/audio_keepalive.py