gateway: price the Metis pass-through route from its reported tokens
The /v1/systemone pass-through is an EndpointType.GENERIC route: LiteLLM logs it, but with no model on the route the only cost it can apply is the flat cost_per_request (a budget estimate, not an invoice). So decisions were billed at a fixed 0.0001 instead of the real $0.12 per 1M input tokens GreenPT reports back in usage.input_tokens. Price the route from its own response body via a new build-time patch, patch_passthrough_token_cost.py, in the house style of the existing patch_*.py files: any route listed in general_settings.coop_passthrough_cost_per_token (USD per input token, keyed by path) is billed as usage.input_tokens x rate. cost_per_request stays as the fallback for a listed route whose response carries no token count, and every unlisted route is unchanged. Auth is untouched. config.yaml: list /v1/systemone at 0.00000012 ($0.12 per 1M input tokens); update the metering note.
This commit is contained in:
1 parent
557999a16f
commit
da02f8eb32
4 files changed
+221
-6
No files matched your search
+11
@@ -76,6 +76,17 @@ RUN python3 /app/code/patch_expose_cost_headers.py
|
||||
COPY patch_passthrough_key_access.py /app/code/patch_passthrough_key_access.py
|
||||
RUN python3 /app/code/patch_passthrough_key_access.py
|
||||
|
||||
# Exact per-input-token pricing for generic pass-through routes. A route whose
|
||||
# target matches none of vertex/anthropic/openai/gemini resolves to
|
||||
# EndpointType.GENERIC: it IS logged, but LiteLLM has no model to price it from,
|
||||
# so the only cost it knows is the flat cost_per_request. Metis (/v1/systemone)
|
||||
# is billed at $0.12 per 1M input tokens and returns usage.input_tokens, so this
|
||||
# patch prices a route listed in general_settings.coop_passthrough_cost_per_token
|
||||
# from that count and writes the exact figure to member spend. Non-fatal; check
|
||||
# the "[coop-patch] applied" line in the build log on every LiteLLM upgrade.
|
||||
COPY patch_passthrough_token_cost.py /app/code/patch_passthrough_token_cost.py
|
||||
RUN python3 /app/code/patch_passthrough_token_cost.py
|
||||
|
||||
# Audio keep-alive: scheduled pings keep the Tinfoil TEE audio enclave
|
||||
# warm so voice mode skips the ~24s cold-start (see audio_keepalive.py header).
|
||||
COPY audio_keepalive.py /app/code/audio_keepalive.py
|
||||
|
||||
Reference in new issue
Block a user