Commit Graph
20 Commits
Author SHA1 Message Date
inference-admin d1d342d8e6 manifest: pin to 20261117-0f7cf17 2026-11-17 15:19:08 +00:00
inference-admin 0f7cf17cd8 litellm(config): comments match the prices members are actually billed
The greenpt/deepseek-v4.1-flash block still advertised the pre-1017 rates
($0.22/$1.10/$0.011, and in euros). It now states $0.36/$1.45/$0.04 with the
snapshot and date. The audio note no longer says voxtral-tts is 'listed $0/$0';
it states the per-request charges and points at #5, now closed with a spend row.
2026-11-17 15:16:05 +00:00
inference-admin 4b6583c3c0 manifest: pin to 20261117-2483715 2026-11-17 08:45:27 +00:00
inference-admin 2483715ef3 litellm: register audio per-request prices at interpreter startup (#5)
Build-time edits of the cost-map JSON never reached the running gateway, so the
audio prices now go in through sitecustomize — imported before LiteLLM prices
anything. Idempotent, non-fatal.
2026-11-17 08:44:41 +00:00
inference-admin f66bedfb55 manifest: pin to 20261117-0e8517b 2026-11-17 08:37:26 +00:00
inference-admin 0e8517b6be litellm: target *_backup.json — LiteLLM's local cost map, not the file name we guessed 2026-11-17 08:36:53 +00:00
inference-admin d35ee1760f manifest: revert pin — the audio cost-map build failed, image never existed 2026-11-17 08:34:38 +00:00
inference-admin faff547ff4 manifest: pin to 20261117-b81177c 2026-11-17 08:33:12 +00:00
inference-admin b81177c3ad litellm: put Tinfoil's per-request audio prices in the shared cost map (#5)
The per-request helper reads litellm.model_cost, which is built from
model_prices_and_context_window.json — not from model_info. Router-side patching
therefore never priced voxtral-tts speech, and every call logged $0. This build
step writes the prices where the calculator looks, and fails the build if the file
is missing instead of warning.
2026-11-17 08:32:56 +00:00
inference-admin d96df6fd5a manifest: pin to 20261116-7a167a8 2026-11-16 19:56:17 +00:00
inference-admin 7a167a8354 audio patch: bill speech too — the provider-cost early return pre-empted the tail patch
response_cost_calculator returns provider_response_cost as soon as the response
carries one in hidden params and no custom pricing is attached. That return sits
ABOVE the per-request splice, so tts calls exited at 0.0 and never reached it —
which is why whisper billed $0.01/request and speech billed nothing, with no
[coop-audio] debug line to show for it.

The early return now adds the declared per-request charge. Guarded by the
DECLARATION (input_cost_per_request), not the call type, so token-priced models
like voxtral-small-24b stay inert.
2026-11-16 19:55:43 +00:00
inference-admin a1613401c3 manifest: pin to 20261116-77e38d8 2026-11-16 15:55:57 +00:00
inference-admin 77e38d81e0 gateway: serve the DeepSeek V4.1 Flash 1017 snapshot under the existing names
GreenPT no longer recommends the base V4.1 Flash release, so the default model now
serves the 1017 snapshot. The deployment keeps model_name greenpt/deepseek-v4.1-flash
(and the deepseek-flash-green alias): only the upstream pointer and the cost move.

  openai/deepseek-v4.1-flash -> openai/deepseek-v4.1-flash-1017
  $0.22/$1.10/$0.011 per 1M   -> $0.36/$1.45/$0.04 per 1M

Why not the rename PR #8 proposes: two days of spend logs show 3,014 calls on the
prefixed id against 84 on the alias, spread across member keys. A rename turns every
one of those into 400 Invalid model name, and member client configs are not visible to
us. Keeping the name means members get the new snapshot and price with no config edit.

Credit: slomo, who found the deprecation and supplied the rates (co-op/support#7, PR #8).
2026-11-16 15:55:23 +00:00
inference-admin d877bc8eee manifest: pin to 20261116-4377611 2026-11-16 15:53:35 +00:00
inference-admin 437761171c manifest: pin to 20261114-84a1918 2026-11-14 18:56:23 +00:00
inference-admin 84a191851f gateway: glm-green -> glm-flash-green, for symmetry with deepseek-flash-green
The -green/-tee pair should read as the same model on two routes; 'glm-green'
named the route but not the tier, so it sat oddly beside deepseek-flash-green.
2026-11-14 18:55:53 +00:00
inference-admin 1ec589cf8b manifest: pin to 20261114-9266f9b 2026-11-14 18:46:24 +00:00
inference-admin 9266f9bc3c gateway: aliases name the route, not the version (-green / -tee)
deepseek-flash-latest -> deepseek-flash-green, glm-flash-latest -> glm-green.
'latest' described how fresh a version is; what a member actually needs to choose
between is the route, and -green / -tee says it. The version-free property is
unchanged — it comes from the alias being re-pointed, not from the word in the
name — and is now stated in the comment rather than encoded in the label.
Renamed while the previous names were hours old and nothing depended on them.
2026-11-14 18:45:53 +00:00
inference-admin 02ff60bfbf manifest: pin to 20261113-5986ee7 2026-11-14 18:40:47 +00:00
inference-admin 5986ee7987 gateway: stable model aliases members can pin across transitions
Four version-free names, mapped to the models we serve today:

  deepseek-flash-latest -> greenpt/deepseek-v4.1-flash  (renewable, policy-private)
  deepseek-flash-tee    -> tinfoil/deepseek-v4-1-flash  (TEE)
  glm-flash-latest      -> greenpt/glm-5.3-flash        (renewable, policy-private)
  glm-tee               -> tinfoil/glm-5.3              (TEE; Tinfoil deprecated flash)

Config-only, per the proposal in co-op/support#47: a model_name is a label, so an
alias is another entry pointing at the same upstream model. Pricing is stated at
the alias, since LiteLLM bills the model_info of the name that was called — a
transition that changes price is then visible in the model list and in budgets.
glm-tee rather than glm-flash-tee: the TEE route for this family is the flagship.

Validator: config OK (litellm), 21 models, no duplicate model_name.
2026-11-14 18:40:10 +00:00