gateway: add greenpt/qwen3-embedding-8b (member-facing embeddings)
A member with API-spend history for tinfoil/nomic-embed-text reported the model 'missing'. It was never removed: it has been admin-team scoped (model_info.team_id) since it was added 2026-10-07 for the co-op's own docs-RAG, and team-scoped models are hidden from every /v1/models listing by design — the member's dashboard row was their own refused call. Tinfoil still serves nomic-embed-text upstream; nothing to restore. What WAS missing is a member-callable embedding model. GreenPT serves qwen3-embedding-8b (id verbatim, type EMBEDDINGS): $0.25/1M input from GreenPT's own /v1/pricing, no output charge, 32K context. Added as a regular model_list entry (no team_id) so any member key can call it via /v1/embeddings; the portal filters mode:embedding out of the chat picker. max_input_tokens declared from the provider's contextWindowTokens. Additive change only: every pre-existing model_info byte-identical, fallbacks and pass-through untouched; boot-proved on a spare port in the container before deploy.
This commit is contained in:
1 parent
e161db9ec8
commit
54925cae97
1 file changed
+23
+23
@@ -10,6 +10,7 @@
|
|||||||
# tinfoil/deepseek-v4-1-flash: $0.65 in / $1.45 out / $0.13 cached per 1M
|
# tinfoil/deepseek-v4-1-flash: $0.65 in / $1.45 out / $0.13 cached per 1M
|
||||||
# tinfoil/gpt-oss-120b: $0.15 in / $0.60 out per 1M
|
# tinfoil/gpt-oss-120b: $0.15 in / $0.60 out per 1M
|
||||||
# tinfoil/glm-5.3: $1.80 in / $5.75 out / $0.45 cached per 1M
|
# tinfoil/glm-5.3: $1.80 in / $5.75 out / $0.45 cached per 1M
|
||||||
|
# greenpt/qwen3-embedding-8b: $0.25 in / $0 out per 1M (embeddings, 32K ctx)
|
||||||
# Without these, LiteLLM can't price these custom models, so spend shows $0.00
|
# Without these, LiteLLM can't price these custom models, so spend shows $0.00
|
||||||
# and the team budget cap is not enforced.
|
# and the team budget cap is not enforced.
|
||||||
#
|
#
|
||||||
@@ -208,6 +209,28 @@ model_list:
|
|||||||
input_cost_per_token: 0.00000005
|
input_cost_per_token: 0.00000005
|
||||||
output_cost_per_token: 0.0
|
output_cost_per_token: 0.0
|
||||||
|
|
||||||
|
# --- Embedding model (GreenPT, member-facing) ---
|
||||||
|
# qwen3-embedding-8b — text embeddings via /v1/embeddings, callable by any
|
||||||
|
# member key (NOT team-scoped, unlike nomic above). Added 2026-11-06 after a
|
||||||
|
# member asked for an embedding model: the private (Tinfoil) embedding is an
|
||||||
|
# operator-only docs-RAG resource, so this fills the member-facing gap on
|
||||||
|
# renewable EU hosting. The portal filters mode:embedding out of the chat
|
||||||
|
# picker, so it is API-only — same shape as the audio models above.
|
||||||
|
# Pricing per GreenPT /v1/pricing: $0.25/1M input, no output charge
|
||||||
|
# (embeddings return vectors; GreenPT lists maxOutputTokens null), 32K
|
||||||
|
# context. max_input_tokens declared from the provider's contextWindowTokens
|
||||||
|
# so clients don't invent a larger one (the null-caps defect).
|
||||||
|
- model_name: greenpt/qwen3-embedding-8b
|
||||||
|
litellm_params:
|
||||||
|
model: openai/qwen3-embedding-8b
|
||||||
|
api_base: https://api.greenpt.ai/v1
|
||||||
|
api_key: os.environ/GREENPT_API_KEY
|
||||||
|
model_info:
|
||||||
|
mode: embedding
|
||||||
|
input_cost_per_token: 0.00000025
|
||||||
|
output_cost_per_token: 0.0
|
||||||
|
max_input_tokens: 32000
|
||||||
|
|
||||||
# Fallback: if DeepSeek is down, use GPT-OSS
|
# Fallback: if DeepSeek is down, use GPT-OSS
|
||||||
# NOTE: DeepSeek V4.1 Flash is a reasoning model — generations routinely run
|
# NOTE: DeepSeek V4.1 Flash is a reasoning model — generations routinely run
|
||||||
# longer than 30s. A 30s timeout caused LiteLLM to abandon in-flight DeepSeek
|
# longer than 30s. A 30s timeout caused LiteLLM to abandon in-flight DeepSeek
|
||||||
|
|||||||
Reference in new issue
Block a user