gateway: make tinfoil/nomic-embed-text member-facing; add greenpt/green-embeddings; fix nomic price to $0.016/1M

- Remove the team_id scoping on tinfoil/nomic-embed-text: member keys can now
  call the TEE-protected embedding the co-op's own docs-RAG runs on (co-op
  traffic unaffected).
- Correct nomic's declared price to Tinfoil's published rate ($0.016/1M in,
  was declared $0.05) and declare its 8K context.
- Add greenpt/green-embeddings ($0.20/1M, 2560-dim, 32K ctx) — the only
  genuinely distinct other embedding model in GreenPT's catalog; their
  green-embedding slug is the same model (identical vectors verified live).
- 15 models served; docs models.md updated in the co-op/docs repo.
This commit is contained in:
inference-bot committed 2026-11-06 22:35:58 +00:00
1 parent 67df968a04
commit abaeb255b2
1 file changed
+40 -13
+40 -13
View File
@@ -11,6 +11,8 @@
# tinfoil/gpt-oss-120b: $0.15 in / $0.60 out per 1M # tinfoil/gpt-oss-120b: $0.15 in / $0.60 out per 1M
# tinfoil/glm-5.3: $1.80 in / $5.75 out / $0.45 cached per 1M # tinfoil/glm-5.3: $1.80 in / $5.75 out / $0.45 cached per 1M
# greenpt/qwen3-embedding-8b: $0.25 in / $0 out per 1M (embeddings, 32K ctx) # greenpt/qwen3-embedding-8b: $0.25 in / $0 out per 1M (embeddings, 32K ctx)
# greenpt/green-embeddings: $0.20 in / $0 out per 1M (embeddings, 32K ctx)
# tinfoil/nomic-embed-text: $0.016 in / $0 out per 1M (embeddings, 8K ctx, TEE)
# tinfoil/voxtral-small-24b: $0.20 in / $0.60 out per 1M (token-priced audio; # tinfoil/voxtral-small-24b: $0.20 in / $0.60 out per 1M (token-priced audio;
# +$0.00008/request on /v1/audio/transcriptions) # +$0.00008/request on /v1/audio/transcriptions)
# Without these, LiteLLM can't price these custom models, so spend shows $0.00 # Without these, LiteLLM can't price these custom models, so spend shows $0.00
@@ -218,13 +220,17 @@ model_list:
input_cost_per_character: 0 input_cost_per_character: 0
input_cost_per_request: 0.002 input_cost_per_request: 0.002
# --- Embedding model (Tinfoil, in-TEE) --- # --- Embedding models (Tinfoil, in-TEE; and GreenPT) ---
# nomic-embed-text — 768-dim text embeddings for RAG/semantic search. # nomic-embed-text — 768-dim text embeddings for RAG/semantic search.
# NOT a chat model: it powers /v1/embeddings only. Scoped to the admin # NOT a chat model: it powers /v1/embeddings only. MEMBER-FACING since
# account's team (model_info.team_id) so member keys cannot call it — it is # 2026-11-06 — the team_id scoping (operator-account-only) was removed so
# an operator resource for the co-op's own docs-RAG, not a member-facing # members get a TEE-protected embedding option. This is the same model the
# model. The portal also filters it out of the chat-facing /v1/models list by # co-op's own docs-RAG runs on inside the TEE (Open WebUI's knowledge base);
# its `mode: embedding`. Pricing per Tinfoil catalog: $0.05/1M input, $0 out. # that traffic is unaffected by members also being able to call it. The
# portal filters `mode: embedding` out of the chat-facing /v1/models list, so
# it stays API-only. Pricing per Tinfoil catalog (inference.tinfoil.sh
# /v1/models): $0.016/1M input, $0 out — corrected 2026-11-06 from a stale
# $0.05 declaration; 8K context declared so clients don't invent larger.
- model_name: tinfoil/nomic-embed-text - model_name: tinfoil/nomic-embed-text
litellm_params: litellm_params:
model: openai/nomic-embed-text model: openai/nomic-embed-text
@@ -232,17 +238,17 @@ model_list:
api_key: os.environ/TINFOIL_API_KEY api_key: os.environ/TINFOIL_API_KEY
model_info: model_info:
mode: embedding mode: embedding
team_id: cdfdf422-633f-4432-8893-cffee82a8325 max_input_tokens: 8192
input_cost_per_token: 0.00000005 input_cost_per_token: 0.000000016
output_cost_per_token: 0.0 output_cost_per_token: 0.0
# --- Embedding model (GreenPT, member-facing) --- # --- Embedding model (GreenPT, member-facing) ---
# qwen3-embedding-8b — text embeddings via /v1/embeddings, callable by any # qwen3-embedding-8b — text embeddings via /v1/embeddings, callable by any
# member key (NOT team-scoped, unlike nomic above). Added 2026-11-06 after a # member key. Added 2026-11-06 after a member asked for an embedding model,
# member asked for an embedding model: the private (Tinfoil) embedding is an # filling the member-facing gap on renewable EU hosting (Tinfoil's private
# operator-only docs-RAG resource, so this fills the member-facing gap on # embedding was opened to members the same day). The portal filters
# renewable EU hosting. The portal filters mode:embedding out of the chat # mode:embedding out of the chat picker, so it is API-only — same shape as
# picker, so it is API-only — same shape as the audio models above. # the audio models above.
# Pricing per GreenPT /v1/pricing: $0.25/1M input, no output charge # Pricing per GreenPT /v1/pricing: $0.25/1M input, no output charge
# (embeddings return vectors; GreenPT lists maxOutputTokens null), 32K # (embeddings return vectors; GreenPT lists maxOutputTokens null), 32K
# context. max_input_tokens declared from the provider's contextWindowTokens # context. max_input_tokens declared from the provider's contextWindowTokens
@@ -258,6 +264,27 @@ model_list:
output_cost_per_token: 0.0 output_cost_per_token: 0.0
max_input_tokens: 32000 max_input_tokens: 32000
# --- Embedding model (GreenPT, member-facing) ---
# green-embeddings — GreenPT's own embedding model (2560-dim, measured live).
# Added 2026-11-06 after a catalog survey: the only genuinely distinct
# embedding model on GreenPT besides qwen3-embedding-8b. Their catalog also
# lists a `green-embedding` (singular) slug — verified the SAME model
# (identical vectors on the same input), so only green-embeddings is served.
# Cheaper per token than qwen3-embedding-8b; the reranker sibling
# (green-rerank) is a different capability and is NOT served.
# Same shape: /v1/embeddings only, filtered out of the chat picker.
# Pricing per GreenPT /v1/pricing: $0.20/1M input, no output charge, 32K ctx.
- model_name: greenpt/green-embeddings
litellm_params:
model: openai/green-embeddings
api_base: https://api.greenpt.ai/v1
api_key: os.environ/GREENPT_API_KEY
model_info:
mode: embedding
input_cost_per_token: 0.0000002
output_cost_per_token: 0.0
max_input_tokens: 32000
# Fallback: if DeepSeek is down, use GPT-OSS # Fallback: if DeepSeek is down, use GPT-OSS
# NOTE: DeepSeek V4.1 Flash is a reasoning model — generations routinely run # NOTE: DeepSeek V4.1 Flash is a reasoning model — generations routinely run
# longer than 30s. A 30s timeout caused LiteLLM to abandon in-flight DeepSeek # longer than 30s. A 30s timeout caused LiteLLM to abandon in-flight DeepSeek