gateway: make tinfoil/nomic-embed-text member-facing; add greenpt/green-embeddings; fix nomic price to $0.016/1M
- Remove the team_id scoping on tinfoil/nomic-embed-text: member keys can now call the TEE-protected embedding the co-op's own docs-RAG runs on (co-op traffic unaffected). - Correct nomic's declared price to Tinfoil's published rate ($0.016/1M in, was declared $0.05) and declare its 8K context. - Add greenpt/green-embeddings ($0.20/1M, 2560-dim, 32K ctx) — the only genuinely distinct other embedding model in GreenPT's catalog; their green-embedding slug is the same model (identical vectors verified live). - 15 models served; docs models.md updated in the co-op/docs repo.
This commit is contained in:
1 parent
67df968a04
commit
abaeb255b2
1 file changed
+40
-13
+40
-13
@@ -11,6 +11,8 @@
|
|||||||
# tinfoil/gpt-oss-120b: $0.15 in / $0.60 out per 1M
|
# tinfoil/gpt-oss-120b: $0.15 in / $0.60 out per 1M
|
||||||
# tinfoil/glm-5.3: $1.80 in / $5.75 out / $0.45 cached per 1M
|
# tinfoil/glm-5.3: $1.80 in / $5.75 out / $0.45 cached per 1M
|
||||||
# greenpt/qwen3-embedding-8b: $0.25 in / $0 out per 1M (embeddings, 32K ctx)
|
# greenpt/qwen3-embedding-8b: $0.25 in / $0 out per 1M (embeddings, 32K ctx)
|
||||||
|
# greenpt/green-embeddings: $0.20 in / $0 out per 1M (embeddings, 32K ctx)
|
||||||
|
# tinfoil/nomic-embed-text: $0.016 in / $0 out per 1M (embeddings, 8K ctx, TEE)
|
||||||
# tinfoil/voxtral-small-24b: $0.20 in / $0.60 out per 1M (token-priced audio;
|
# tinfoil/voxtral-small-24b: $0.20 in / $0.60 out per 1M (token-priced audio;
|
||||||
# +$0.00008/request on /v1/audio/transcriptions)
|
# +$0.00008/request on /v1/audio/transcriptions)
|
||||||
# Without these, LiteLLM can't price these custom models, so spend shows $0.00
|
# Without these, LiteLLM can't price these custom models, so spend shows $0.00
|
||||||
@@ -218,13 +220,17 @@ model_list:
|
|||||||
input_cost_per_character: 0
|
input_cost_per_character: 0
|
||||||
input_cost_per_request: 0.002
|
input_cost_per_request: 0.002
|
||||||
|
|
||||||
# --- Embedding model (Tinfoil, in-TEE) ---
|
# --- Embedding models (Tinfoil, in-TEE; and GreenPT) ---
|
||||||
# nomic-embed-text — 768-dim text embeddings for RAG/semantic search.
|
# nomic-embed-text — 768-dim text embeddings for RAG/semantic search.
|
||||||
# NOT a chat model: it powers /v1/embeddings only. Scoped to the admin
|
# NOT a chat model: it powers /v1/embeddings only. MEMBER-FACING since
|
||||||
# account's team (model_info.team_id) so member keys cannot call it — it is
|
# 2026-11-06 — the team_id scoping (operator-account-only) was removed so
|
||||||
# an operator resource for the co-op's own docs-RAG, not a member-facing
|
# members get a TEE-protected embedding option. This is the same model the
|
||||||
# model. The portal also filters it out of the chat-facing /v1/models list by
|
# co-op's own docs-RAG runs on inside the TEE (Open WebUI's knowledge base);
|
||||||
# its `mode: embedding`. Pricing per Tinfoil catalog: $0.05/1M input, $0 out.
|
# that traffic is unaffected by members also being able to call it. The
|
||||||
|
# portal filters `mode: embedding` out of the chat-facing /v1/models list, so
|
||||||
|
# it stays API-only. Pricing per Tinfoil catalog (inference.tinfoil.sh
|
||||||
|
# /v1/models): $0.016/1M input, $0 out — corrected 2026-11-06 from a stale
|
||||||
|
# $0.05 declaration; 8K context declared so clients don't invent larger.
|
||||||
- model_name: tinfoil/nomic-embed-text
|
- model_name: tinfoil/nomic-embed-text
|
||||||
litellm_params:
|
litellm_params:
|
||||||
model: openai/nomic-embed-text
|
model: openai/nomic-embed-text
|
||||||
@@ -232,17 +238,17 @@ model_list:
|
|||||||
api_key: os.environ/TINFOIL_API_KEY
|
api_key: os.environ/TINFOIL_API_KEY
|
||||||
model_info:
|
model_info:
|
||||||
mode: embedding
|
mode: embedding
|
||||||
team_id: cdfdf422-633f-4432-8893-cffee82a8325
|
max_input_tokens: 8192
|
||||||
input_cost_per_token: 0.00000005
|
input_cost_per_token: 0.000000016
|
||||||
output_cost_per_token: 0.0
|
output_cost_per_token: 0.0
|
||||||
|
|
||||||
# --- Embedding model (GreenPT, member-facing) ---
|
# --- Embedding model (GreenPT, member-facing) ---
|
||||||
# qwen3-embedding-8b — text embeddings via /v1/embeddings, callable by any
|
# qwen3-embedding-8b — text embeddings via /v1/embeddings, callable by any
|
||||||
# member key (NOT team-scoped, unlike nomic above). Added 2026-11-06 after a
|
# member key. Added 2026-11-06 after a member asked for an embedding model,
|
||||||
# member asked for an embedding model: the private (Tinfoil) embedding is an
|
# filling the member-facing gap on renewable EU hosting (Tinfoil's private
|
||||||
# operator-only docs-RAG resource, so this fills the member-facing gap on
|
# embedding was opened to members the same day). The portal filters
|
||||||
# renewable EU hosting. The portal filters mode:embedding out of the chat
|
# mode:embedding out of the chat picker, so it is API-only — same shape as
|
||||||
# picker, so it is API-only — same shape as the audio models above.
|
# the audio models above.
|
||||||
# Pricing per GreenPT /v1/pricing: $0.25/1M input, no output charge
|
# Pricing per GreenPT /v1/pricing: $0.25/1M input, no output charge
|
||||||
# (embeddings return vectors; GreenPT lists maxOutputTokens null), 32K
|
# (embeddings return vectors; GreenPT lists maxOutputTokens null), 32K
|
||||||
# context. max_input_tokens declared from the provider's contextWindowTokens
|
# context. max_input_tokens declared from the provider's contextWindowTokens
|
||||||
@@ -258,6 +264,27 @@ model_list:
|
|||||||
output_cost_per_token: 0.0
|
output_cost_per_token: 0.0
|
||||||
max_input_tokens: 32000
|
max_input_tokens: 32000
|
||||||
|
|
||||||
|
# --- Embedding model (GreenPT, member-facing) ---
|
||||||
|
# green-embeddings — GreenPT's own embedding model (2560-dim, measured live).
|
||||||
|
# Added 2026-11-06 after a catalog survey: the only genuinely distinct
|
||||||
|
# embedding model on GreenPT besides qwen3-embedding-8b. Their catalog also
|
||||||
|
# lists a `green-embedding` (singular) slug — verified the SAME model
|
||||||
|
# (identical vectors on the same input), so only green-embeddings is served.
|
||||||
|
# Cheaper per token than qwen3-embedding-8b; the reranker sibling
|
||||||
|
# (green-rerank) is a different capability and is NOT served.
|
||||||
|
# Same shape: /v1/embeddings only, filtered out of the chat picker.
|
||||||
|
# Pricing per GreenPT /v1/pricing: $0.20/1M input, no output charge, 32K ctx.
|
||||||
|
- model_name: greenpt/green-embeddings
|
||||||
|
litellm_params:
|
||||||
|
model: openai/green-embeddings
|
||||||
|
api_base: https://api.greenpt.ai/v1
|
||||||
|
api_key: os.environ/GREENPT_API_KEY
|
||||||
|
model_info:
|
||||||
|
mode: embedding
|
||||||
|
input_cost_per_token: 0.0000002
|
||||||
|
output_cost_per_token: 0.0
|
||||||
|
max_input_tokens: 32000
|
||||||
|
|
||||||
# Fallback: if DeepSeek is down, use GPT-OSS
|
# Fallback: if DeepSeek is down, use GPT-OSS
|
||||||
# NOTE: DeepSeek V4.1 Flash is a reasoning model — generations routinely run
|
# NOTE: DeepSeek V4.1 Flash is a reasoning model — generations routinely run
|
||||||
# longer than 30s. A 30s timeout caused LiteLLM to abandon in-flight DeepSeek
|
# longer than 30s. A 30s timeout caused LiteLLM to abandon in-flight DeepSeek
|
||||||
|
|||||||
Reference in new issue
Block a user