gateway: make tinfoil/nomic-embed-text member-facing; add greenpt/green-embeddings; fix nomic price to $0.016/1M
- Remove the team_id scoping on tinfoil/nomic-embed-text: member keys can now call the TEE-protected embedding the co-op's own docs-RAG runs on (co-op traffic unaffected). - Correct nomic's declared price to Tinfoil's published rate ($0.016/1M in, was declared $0.05) and declare its 8K context. - Add greenpt/green-embeddings ($0.20/1M, 2560-dim, 32K ctx) — the only genuinely distinct other embedding model in GreenPT's catalog; their green-embedding slug is the same model (identical vectors verified live). - 15 models served; docs models.md updated in the co-op/docs repo.
This commit is contained in:
1 parent
67df968a04
commit
abaeb255b2
1 file changed
+40
-13
+40
-13
@@ -11,6 +11,8 @@
|
||||
# tinfoil/gpt-oss-120b: $0.15 in / $0.60 out per 1M
|
||||
# tinfoil/glm-5.3: $1.80 in / $5.75 out / $0.45 cached per 1M
|
||||
# greenpt/qwen3-embedding-8b: $0.25 in / $0 out per 1M (embeddings, 32K ctx)
|
||||
# greenpt/green-embeddings: $0.20 in / $0 out per 1M (embeddings, 32K ctx)
|
||||
# tinfoil/nomic-embed-text: $0.016 in / $0 out per 1M (embeddings, 8K ctx, TEE)
|
||||
# tinfoil/voxtral-small-24b: $0.20 in / $0.60 out per 1M (token-priced audio;
|
||||
# +$0.00008/request on /v1/audio/transcriptions)
|
||||
# Without these, LiteLLM can't price these custom models, so spend shows $0.00
|
||||
@@ -218,13 +220,17 @@ model_list:
|
||||
input_cost_per_character: 0
|
||||
input_cost_per_request: 0.002
|
||||
|
||||
# --- Embedding model (Tinfoil, in-TEE) ---
|
||||
# --- Embedding models (Tinfoil, in-TEE; and GreenPT) ---
|
||||
# nomic-embed-text — 768-dim text embeddings for RAG/semantic search.
|
||||
# NOT a chat model: it powers /v1/embeddings only. Scoped to the admin
|
||||
# account's team (model_info.team_id) so member keys cannot call it — it is
|
||||
# an operator resource for the co-op's own docs-RAG, not a member-facing
|
||||
# model. The portal also filters it out of the chat-facing /v1/models list by
|
||||
# its `mode: embedding`. Pricing per Tinfoil catalog: $0.05/1M input, $0 out.
|
||||
# NOT a chat model: it powers /v1/embeddings only. MEMBER-FACING since
|
||||
# 2026-11-06 — the team_id scoping (operator-account-only) was removed so
|
||||
# members get a TEE-protected embedding option. This is the same model the
|
||||
# co-op's own docs-RAG runs on inside the TEE (Open WebUI's knowledge base);
|
||||
# that traffic is unaffected by members also being able to call it. The
|
||||
# portal filters `mode: embedding` out of the chat-facing /v1/models list, so
|
||||
# it stays API-only. Pricing per Tinfoil catalog (inference.tinfoil.sh
|
||||
# /v1/models): $0.016/1M input, $0 out — corrected 2026-11-06 from a stale
|
||||
# $0.05 declaration; 8K context declared so clients don't invent larger.
|
||||
- model_name: tinfoil/nomic-embed-text
|
||||
litellm_params:
|
||||
model: openai/nomic-embed-text
|
||||
@@ -232,17 +238,17 @@ model_list:
|
||||
api_key: os.environ/TINFOIL_API_KEY
|
||||
model_info:
|
||||
mode: embedding
|
||||
team_id: cdfdf422-633f-4432-8893-cffee82a8325
|
||||
input_cost_per_token: 0.00000005
|
||||
max_input_tokens: 8192
|
||||
input_cost_per_token: 0.000000016
|
||||
output_cost_per_token: 0.0
|
||||
|
||||
# --- Embedding model (GreenPT, member-facing) ---
|
||||
# qwen3-embedding-8b — text embeddings via /v1/embeddings, callable by any
|
||||
# member key (NOT team-scoped, unlike nomic above). Added 2026-11-06 after a
|
||||
# member asked for an embedding model: the private (Tinfoil) embedding is an
|
||||
# operator-only docs-RAG resource, so this fills the member-facing gap on
|
||||
# renewable EU hosting. The portal filters mode:embedding out of the chat
|
||||
# picker, so it is API-only — same shape as the audio models above.
|
||||
# member key. Added 2026-11-06 after a member asked for an embedding model,
|
||||
# filling the member-facing gap on renewable EU hosting (Tinfoil's private
|
||||
# embedding was opened to members the same day). The portal filters
|
||||
# mode:embedding out of the chat picker, so it is API-only — same shape as
|
||||
# the audio models above.
|
||||
# Pricing per GreenPT /v1/pricing: $0.25/1M input, no output charge
|
||||
# (embeddings return vectors; GreenPT lists maxOutputTokens null), 32K
|
||||
# context. max_input_tokens declared from the provider's contextWindowTokens
|
||||
@@ -258,6 +264,27 @@ model_list:
|
||||
output_cost_per_token: 0.0
|
||||
max_input_tokens: 32000
|
||||
|
||||
# --- Embedding model (GreenPT, member-facing) ---
|
||||
# green-embeddings — GreenPT's own embedding model (2560-dim, measured live).
|
||||
# Added 2026-11-06 after a catalog survey: the only genuinely distinct
|
||||
# embedding model on GreenPT besides qwen3-embedding-8b. Their catalog also
|
||||
# lists a `green-embedding` (singular) slug — verified the SAME model
|
||||
# (identical vectors on the same input), so only green-embeddings is served.
|
||||
# Cheaper per token than qwen3-embedding-8b; the reranker sibling
|
||||
# (green-rerank) is a different capability and is NOT served.
|
||||
# Same shape: /v1/embeddings only, filtered out of the chat picker.
|
||||
# Pricing per GreenPT /v1/pricing: $0.20/1M input, no output charge, 32K ctx.
|
||||
- model_name: greenpt/green-embeddings
|
||||
litellm_params:
|
||||
model: openai/green-embeddings
|
||||
api_base: https://api.greenpt.ai/v1
|
||||
api_key: os.environ/GREENPT_API_KEY
|
||||
model_info:
|
||||
mode: embedding
|
||||
input_cost_per_token: 0.0000002
|
||||
output_cost_per_token: 0.0
|
||||
max_input_tokens: 32000
|
||||
|
||||
# Fallback: if DeepSeek is down, use GPT-OSS
|
||||
# NOTE: DeepSeek V4.1 Flash is a reasoning model — generations routinely run
|
||||
# longer than 30s. A 30s timeout caused LiteLLM to abandon in-flight DeepSeek
|
||||
|
||||
Reference in new issue
Block a user