Commit Graph
26 Commits
Author SHA1 Message Date
inference-admin c8816b58c1 gateway: convert the route aliases too — members call those, not the prefixed ids
deepseek-flash-green and glm-flash-green carried their own old model_info; the
deployment entries were converted but the aliases kept EUR numerals, so live spend
still billed the old rate.
2026-11-18 17:29:29 +00:00
inference-admin 718644c363 manifest: pin to 20261118-870b165 2026-11-18 17:26:03 +00:00
inference-admin 870b165321 scripts: check EUR->USD conversions in model_info (FX guard) 2026-11-18 17:24:57 +00:00
inference-admin aa9331cff3 gateway: convert GreenPT and Regolo EUR prices to USD at FX 1.16 (corrected)
Previous attempt forgot to multiply by the FX and could match a prefix sibling; reverted
and redone with exact model-name anchors. EUR remains the source of truth in comments
and docs; model_info now carries EUR x 1.16 USD so member billing covers cost.
2026-11-18 17:24:39 +00:00
inference-admin 6f6bb73d16 gateway: convert GreenPT and Regolo EUR list prices to USD at FX 1.16
LiteLLM's cost fields are dollar-shaped, but both providers quote EUR; the config
carried EUR numerals as USD, so members were billed ~14% under cost and the co-op
silently absorbed the difference. EUR stays the source of truth; model_info now
carries EUR x 1.16 USD, with the policy recorded in the pricing header. No backfill:
historical spend rows keep the old understated figures and are flagged in docs.
2026-11-18 17:24:23 +00:00
inference-admin 9c81df48b8 gateway: raise Cloudron memory limit 2 GB -> 3 GB
The container was restarted for OOM at the 2 GB cap; a fresh process is already at
1.74 GB peak, so the limit is the binding constraint, not a one-off spike.
2026-11-18 14:13:15 +00:00
inference-admin d1d342d8e6 manifest: pin to 20261117-0f7cf17 2026-11-17 15:19:08 +00:00
inference-admin 0f7cf17cd8 litellm(config): comments match the prices members are actually billed
The greenpt/deepseek-v4.1-flash block still advertised the pre-1017 rates
($0.22/$1.10/$0.011, and in euros). It now states $0.36/$1.45/$0.04 with the
snapshot and date. The audio note no longer says voxtral-tts is 'listed $0/$0';
it states the per-request charges and points at #5, now closed with a spend row.
2026-11-17 15:16:05 +00:00
inference-admin 4b6583c3c0 manifest: pin to 20261117-2483715 2026-11-17 08:45:27 +00:00
inference-admin 2483715ef3 litellm: register audio per-request prices at interpreter startup (#5)
Build-time edits of the cost-map JSON never reached the running gateway, so the
audio prices now go in through sitecustomize — imported before LiteLLM prices
anything. Idempotent, non-fatal.
2026-11-17 08:44:41 +00:00
inference-admin f66bedfb55 manifest: pin to 20261117-0e8517b 2026-11-17 08:37:26 +00:00
inference-admin 0e8517b6be litellm: target *_backup.json — LiteLLM's local cost map, not the file name we guessed 2026-11-17 08:36:53 +00:00
inference-admin d35ee1760f manifest: revert pin — the audio cost-map build failed, image never existed 2026-11-17 08:34:38 +00:00
inference-admin faff547ff4 manifest: pin to 20261117-b81177c 2026-11-17 08:33:12 +00:00
inference-admin b81177c3ad litellm: put Tinfoil's per-request audio prices in the shared cost map (#5)
The per-request helper reads litellm.model_cost, which is built from
model_prices_and_context_window.json — not from model_info. Router-side patching
therefore never priced voxtral-tts speech, and every call logged $0. This build
step writes the prices where the calculator looks, and fails the build if the file
is missing instead of warning.
2026-11-17 08:32:56 +00:00
inference-admin d96df6fd5a manifest: pin to 20261116-7a167a8 2026-11-16 19:56:17 +00:00
inference-admin 7a167a8354 audio patch: bill speech too — the provider-cost early return pre-empted the tail patch
response_cost_calculator returns provider_response_cost as soon as the response
carries one in hidden params and no custom pricing is attached. That return sits
ABOVE the per-request splice, so tts calls exited at 0.0 and never reached it —
which is why whisper billed $0.01/request and speech billed nothing, with no
[coop-audio] debug line to show for it.

The early return now adds the declared per-request charge. Guarded by the
DECLARATION (input_cost_per_request), not the call type, so token-priced models
like voxtral-small-24b stay inert.
2026-11-16 19:55:43 +00:00
inference-admin a1613401c3 manifest: pin to 20261116-77e38d8 2026-11-16 15:55:57 +00:00
inference-admin 77e38d81e0 gateway: serve the DeepSeek V4.1 Flash 1017 snapshot under the existing names
GreenPT no longer recommends the base V4.1 Flash release, so the default model now
serves the 1017 snapshot. The deployment keeps model_name greenpt/deepseek-v4.1-flash
(and the deepseek-flash-green alias): only the upstream pointer and the cost move.

  openai/deepseek-v4.1-flash -> openai/deepseek-v4.1-flash-1017
  $0.22/$1.10/$0.011 per 1M   -> $0.36/$1.45/$0.04 per 1M

Why not the rename PR #8 proposes: two days of spend logs show 3,014 calls on the
prefixed id against 84 on the alias, spread across member keys. A rename turns every
one of those into 400 Invalid model name, and member client configs are not visible to
us. Keeping the name means members get the new snapshot and price with no config edit.

Credit: slomo, who found the deprecation and supplied the rates (co-op/support#7, PR #8).
2026-11-16 15:55:23 +00:00
inference-admin d877bc8eee manifest: pin to 20261116-4377611 2026-11-16 15:53:35 +00:00
inference-admin 437761171c manifest: pin to 20261114-84a1918 2026-11-14 18:56:23 +00:00
inference-admin 84a191851f gateway: glm-green -> glm-flash-green, for symmetry with deepseek-flash-green
The -green/-tee pair should read as the same model on two routes; 'glm-green'
named the route but not the tier, so it sat oddly beside deepseek-flash-green.
2026-11-14 18:55:53 +00:00
inference-admin 1ec589cf8b manifest: pin to 20261114-9266f9b 2026-11-14 18:46:24 +00:00
inference-admin 9266f9bc3c gateway: aliases name the route, not the version (-green / -tee)
deepseek-flash-latest -> deepseek-flash-green, glm-flash-latest -> glm-green.
'latest' described how fresh a version is; what a member actually needs to choose
between is the route, and -green / -tee says it. The version-free property is
unchanged — it comes from the alias being re-pointed, not from the word in the
name — and is now stated in the comment rather than encoded in the label.
Renamed while the previous names were hours old and nothing depended on them.
2026-11-14 18:45:53 +00:00
inference-admin 02ff60bfbf manifest: pin to 20261113-5986ee7 2026-11-14 18:40:47 +00:00
inference-admin 5986ee7987 gateway: stable model aliases members can pin across transitions
Four version-free names, mapped to the models we serve today:

  deepseek-flash-latest -> greenpt/deepseek-v4.1-flash  (renewable, policy-private)
  deepseek-flash-tee    -> tinfoil/deepseek-v4-1-flash  (TEE)
  glm-flash-latest      -> greenpt/glm-5.3-flash        (renewable, policy-private)
  glm-tee               -> tinfoil/glm-5.3              (TEE; Tinfoil deprecated flash)

Config-only, per the proposal in co-op/support#47: a model_name is a label, so an
alias is another entry pointing at the same upstream model. Pricing is stated at
the alias, since LiteLLM bills the model_info of the name that was called — a
transition that changes price is then visible in the model list and in budgets.
glm-tee rather than glm-flash-tee: the TEE route for this family is the flagship.

Validator: config OK (litellm), 21 models, no duplicate model_name.
2026-11-14 18:40:10 +00:00