diff --git a/api-access.md b/api-access.md index 0457627..a85283a 100644 --- a/api-access.md +++ b/api-access.md @@ -88,16 +88,18 @@ if your base URL already ends in `/v1`, that's correct and you should not add ## What the gateway guarantees (and what it doesn't) -Checked against `gateway.inference.coop` (LiteLLM 1.103.2) with a member key -on 2026-10-19. If any of this drifts, open a PR against this file. +Checked against `gateway.inference.coop` (LiteLLM 1.103.2 at the time of +writing) with a member key on 2026-10-19, re-checked 2026-10-26/27. If any of this drifts, open a PR against this file. - **Supported route:** `POST /v1/chat/completions`. The audio, image and embedding routes work for the models marked for them in the - [model list](models.md). `POST /v1/messages` (Anthropic format) also - answers: it is LiteLLM's built-in translation layer, not something the co-op - tests or supports. It carries tools and tool results correctly, but it drops - the `cost` field and any reasoning content, so prefer chat completions for - anything you meter. + [model list](models.md). `POST /v1/messages` (Anthropic format) answers for + some models (`greenpt/deepseek-v4.1-flash`, `tinfoil/deepseek-v4-1-flash`, + `tinfoil/glm-5.3`, `tinfoil/gpt-oss-120b`) and returns 404 for others + (`greenpt/glm-5.3-flash`): it is LiteLLM's built-in translation layer, not + something the co-op tests or supports. Where it works it carries tools and + tool results correctly, but it drops the `cost` field and any reasoning + content, so prefer chat completions for anything you meter. - **Fallbacks are silent in the body, visible in the headers.** Some models have a fallback chain (today `greenpt/deepseek-v4.1-flash` → `tinfoil/deepseek-v4-1-flash` → `tinfoil/gpt-oss-120b`). If the first @@ -113,13 +115,17 @@ on 2026-10-19. If any of this drifts, open a PR against this file. - **Timeouts and retries.** 180 seconds per request. Connection errors are retried up to twice; timeouts are never retried, because the provider has already billed the generation. -- **Reasoning is on by default** for `tinfoil/deepseek-v4-1-flash` and - `tinfoil/glm-5.3`, and reasoning tokens count against `max_tokens`. A - 700-token `max_tokens` can come back as `finish_reason: "length"` with 700 - reasoning tokens and no content. To turn it off per request, send +- **Reasoning is on by default** on every chat model we tested, and reasoning + tokens count against `max_tokens`. How much varies: the GreenPT GLM models + think for a dozen tokens or so and then answer, while the Tinfoil models and + DeepSeek on either provider can think for hundreds. A 700-token `max_tokens` + on `tinfoil/glm-5.3` or `tinfoil/gpt-oss-120b` can come back as + `finish_reason: "length"` with 700 reasoning tokens and no content. To turn it off per request, send `"reasoning_effort": "none"` (or - `"chat_template_kwargs": {"enable_thinking": false}`). Anthropic-style - `thinking: {...}` is dropped without an error. + `"chat_template_kwargs": {"enable_thinking": false}`); `"off"` is rejected + with a 400, and `tinfoil/gpt-oss-120b` rejects `"none"` too, so thinking + can't be disabled on that model. Anthropic-style `thinking: {...}` is + dropped without an error. - **Unknown parameters are dropped, not rejected** (`drop_params` is on). If a parameter seems to have no effect, that is probably why. - **Usage.** With `"stream_options": {"include_usage": true}` the last chunk @@ -139,7 +145,8 @@ on 2026-10-19. If any of this drifts, open a PR against this file. (`GET /key/info`) and its own budget container (`GET /team/info?team_id=…`, with the `team_id` from `/key/info`): spend, `max_budget`, and `budget_reset_at`. Handy for a client that wants to stop - before the gateway does. `/spend/logs` is admin-only. + before the gateway does. The scoping is enforced: asking for another team's + `team_id` returns 403. `/spend/logs` is admin-only. - **No rate limits beyond the budget, and no request-size cap we could find below 1.5 MB.** There is one shared container; please be a good neighbour. - **No uptime guarantee** (Terms, "Service"). `GET /health/liveliness` and diff --git a/models.md b/models.md index e34411f..c1173f2 100644 --- a/models.md +++ b/models.md @@ -43,8 +43,8 @@ that **you choose** which axis matters more for a given task. cost. - **Hard reasoning, coding, agentic work** → `tinfoil/glm-5.3` (TEE-protected) or `tinfoil/deepseek-v4-1-flash` for long documents and tool-driven tasks. - `greenpt/glm-5.3` is the renewable flagship for reasoning and drafting, but - tool calls currently return empty through the gateway (see the table). + `greenpt/glm-5.3` is the same model on renewable energy; choose between the + two by privacy posture, not capability (see the table). - **Images in, text out (vision)** → `tinfoil/deepseek-v4-1-flash` or the Apertus models — see the ✅ in the table below. - **Cheap and cheerful** → `tinfoil/gpt-oss-120b` or `greenpt/glm-5.3-flash`. @@ -80,8 +80,8 @@ The emoji in the model name marks its **value tier** (the same icons used in the | `🔒 tinfoil/deepseek-v4-1-flash` | Tinfoil | architectural | undisclosed | ✅ | ✅ | 1M | $0.65 / $1.45 / $0.13 | TEE-protected version of the default; image input, tool calling, long docs/agents. Reasoning on by default. | | `🔒 tinfoil/glm-5.3` | Tinfoil | architectural | undisclosed | — | ✅ | 1M | $1.80 / $5.75 / $0.45 | GLM flagship, TEE-protected; strongest private-tier model for agentic coding and long-horizon tasks. Reasoning on by default; the fastest model we measured. | | `🔒 tinfoil/gpt-oss-120b` | Tinfoil | architectural | undisclosed | — | ? | 131K | $0.15 / $0.60 / — | Lightweight open-weight reasoning; cheapest private tier. | -| `🌱 greenpt/glm-5.3-flash` | GreenPT | policy | ✅ 100% renewable | — | ? | 1M | $0.11 / $0.44 / $0.022 | Fast GLM model, renewable energy. | -| `🌱 greenpt/glm-5.3` | GreenPT | policy | ✅ 100% renewable | — | ⚠️ | 1M | $1.10 / $4.40 / $0.275 | GLM flagship, renewable energy. Tool calls currently return empty through the gateway (one reasoning token, then stop; 3 of 3 runs on 2026-10-19). Use `tinfoil/glm-5.3` for agentic work until this is fixed. | +| `🌱 greenpt/glm-5.3-flash` | GreenPT | policy | ✅ 100% renewable | — | ✅ | 1M | $0.11 / $0.44 / $0.022 | Fast GLM model, renewable energy. Reasoning on by default. Tool calls verified 2026-10-26. | +| `🌱 greenpt/glm-5.3` | GreenPT | policy | ✅ 100% renewable | — | ✅ | 1M | $1.10 / $4.40 / $0.275 | GLM flagship, renewable energy. A successful tool call comes back with empty `content` and one reasoning token, so read `tool_calls`, not `content`. Tool calls failed 3 of 3 on 2026-10-19 and passed 15 of 15 on 2026-10-26/27; if you see the failure again, note it here. | | `🌱 regolo/apertus-70b` | Regolo | policy + ZDR | ✅ 100% renewable | ✅ | ? | 65K | $0.46 / $2.44 / — | Fully open sovereign model (Apache-2.0), large/capable, image understanding. | | `🌱 regolo/qwen3.5-122b` | Regolo | policy + ZDR | ✅ 100% renewable | ✅ | ? | 262K | $1.16 / $4.87 / — | Large vision+reasoning MoE (Qwen 3.5, 122B-A10B); 262K context. Thinking is forced off in the gateway config. |