api-access, models: apply review (greenpt/glm-5.3 tool calls work again, /v1/messages per model, /team/info 403, reasoning defaults re-checked 2026-10-27)

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
solo.istandClaude Fable 5.1 committed 2026-10-27 01:34:12 -04:00
1 parent 72361b2363
commit 730204bcd0
2 files changed
+25 -18

No files matched your search

+21 -14
View File
@@ -88,16 +88,18 @@ if your base URL already ends in `/v1`, that's correct and you should not add
## What the gateway guarantees (and what it doesn't)
Checked against `gateway.inference.coop` (LiteLLM 1.103.2) with a member key
on 2026-10-19. If any of this drifts, open a PR against this file.
Checked against `gateway.inference.coop` (LiteLLM 1.103.2 at the time of
writing) with a member key on 2026-10-19, re-checked 2026-10-26/27. If any of this drifts, open a PR against this file.
- **Supported route:** `POST /v1/chat/completions`. The audio, image and
embedding routes work for the models marked for them in the
[model list](models.md). `POST /v1/messages` (Anthropic format) also
answers: it is LiteLLM's built-in translation layer, not something the co-op
tests or supports. It carries tools and tool results correctly, but it drops
the `cost` field and any reasoning content, so prefer chat completions for
anything you meter.
[model list](models.md). `POST /v1/messages` (Anthropic format) answers for
some models (`greenpt/deepseek-v4.1-flash`, `tinfoil/deepseek-v4-1-flash`,
`tinfoil/glm-5.3`, `tinfoil/gpt-oss-120b`) and returns 404 for others
(`greenpt/glm-5.3-flash`): it is LiteLLM's built-in translation layer, not
something the co-op tests or supports. Where it works it carries tools and
tool results correctly, but it drops the `cost` field and any reasoning
content, so prefer chat completions for anything you meter.
- **Fallbacks are silent in the body, visible in the headers.** Some models
have a fallback chain (today `greenpt/deepseek-v4.1-flash` →
`tinfoil/deepseek-v4-1-flash` → `tinfoil/gpt-oss-120b`). If the first
@@ -113,13 +115,17 @@ on 2026-10-19. If any of this drifts, open a PR against this file.
- **Timeouts and retries.** 180 seconds per request. Connection errors are
retried up to twice; timeouts are never retried, because the provider has
already billed the generation.
- **Reasoning is on by default** for `tinfoil/deepseek-v4-1-flash` and
`tinfoil/glm-5.3`, and reasoning tokens count against `max_tokens`. A
700-token `max_tokens` can come back as `finish_reason: "length"` with 700
reasoning tokens and no content. To turn it off per request, send
- **Reasoning is on by default** on every chat model we tested, and reasoning
tokens count against `max_tokens`. How much varies: the GreenPT GLM models
think for a dozen tokens or so and then answer, while the Tinfoil models and
DeepSeek on either provider can think for hundreds. A 700-token `max_tokens`
on `tinfoil/glm-5.3` or `tinfoil/gpt-oss-120b` can come back as
`finish_reason: "length"` with 700 reasoning tokens and no content. To turn it off per request, send
`"reasoning_effort": "none"` (or
`"chat_template_kwargs": {"enable_thinking": false}`). Anthropic-style
`thinking: {...}` is dropped without an error.
`"chat_template_kwargs": {"enable_thinking": false}`); `"off"` is rejected
with a 400, and `tinfoil/gpt-oss-120b` rejects `"none"` too, so thinking
can't be disabled on that model. Anthropic-style `thinking: {...}` is
dropped without an error.
- **Unknown parameters are dropped, not rejected** (`drop_params` is on). If
a parameter seems to have no effect, that is probably why.
- **Usage.** With `"stream_options": {"include_usage": true}` the last chunk
@@ -139,7 +145,8 @@ on 2026-10-19. If any of this drifts, open a PR against this file.
(`GET /key/info`) and its own budget container
(`GET /team/info?team_id=…`, with the `team_id` from `/key/info`): spend,
`max_budget`, and `budget_reset_at`. Handy for a client that wants to stop
before the gateway does. `/spend/logs` is admin-only.
before the gateway does. The scoping is enforced: asking for another team's
`team_id` returns 403. `/spend/logs` is admin-only.
- **No rate limits beyond the budget, and no request-size cap we could find
below 1.5 MB.** There is one shared container; please be a good neighbour.
- **No uptime guarantee** (Terms, "Service"). `GET /health/liveliness` and