docs/api-access.md

API access

Members can use the co-op's models programmatically through an OpenAI-compatible API. It's the same infrastructure that powers the chat — any OpenAI-compatible client works.

Quick facts

• Base URL: https://gateway.inference.coop/v1
• Auth: a bearer API key (create one in the Member Dashboard)
• Format: OpenAI chat completions — POST https://gateway.inference.coop/v1/chat/completions
• Models: see the model list.

API usage draws from the same monthly allowance as chat — there's no separate quota.

Get a key

Log in to the Member Dashboard and create an API key. The key is an sk-… string you pass as a bearer token.

Plain curl

curl https://gateway.inference.coop/v1/chat/completions \
  -H "Authorization: Bearer sk-your-key" \
  -H "Content-Type: application/json" \
  -d '{"model": "tinfoil/deepseek-v4-1-flash", "messages": [{"role": "user", "content": "Hello"}]}'

OpenAI SDK (Python, Node, etc.)

Point the SDK at our base URL and use your member key:

from openai import OpenAI

client = OpenAI(
    base_url="https://gateway.inference.coop/v1",
    api_key="sk-your-key",
)
resp = client.chat.completions.create(
    model="tinfoil/deepseek-v4-1-flash",
    messages=[{"role": "user", "content": "Hello"}],
)

Any OpenAI-compatible client works the same way: set the base URL to https://gateway.inference.coop/v1 and the API key to your member key.

Coding agents and other OpenAI-compatible clients

Any tool that speaks the OpenAI API works. Where a tool asks for provider or endpoint configuration:

• Base URL / endpoint: https://gateway.inference.coop/v1
• API type / protocol: "OpenAI compatible" (or openai-completions where a protocol string must be named) — not "Ollama", "Anthropic", or "Responses"
• API key: your member key (the sk-… string)
• Model: one from the model list, e.g. tinfoil/deepseek-v4-1-flash or greenpt/glm-5.3-flash

A typical provider entry looks like:

{
  "providers": {
    "inference-coop": {
      "baseUrl": "https://gateway.inference.coop/v1",
      "api": "openai-completions",
      "apiKey": "sk-your-key",
      "models": [
        { "id": "tinfoil/deepseek-v4-1-flash" },
        { "id": "tinfoil/gpt-oss-120b" },
        { "id": "tinfoil/glm-5.3" },
        { "id": "greenpt/glm-5.3-flash" }
      ]
    }
  }
}

If a tool's own docs show a local-server example (e.g. baseUrl: http://localhost:11434/v1 with api: "openai-completions"), substitute our base URL and your member key for apiKey — the protocol value stays the same. Some tools append /chat/completions to the base URL themselves; if your base URL already ends in /v1, that's correct and you should not add /chat/completions.

What the gateway guarantees (and what it doesn't)

Checked against gateway.inference.coop (LiteLLM 1.103.2 at the time of writing) with a member key on 2026-10-19, re-checked 2026-10-26/27. If any of this drifts, open a PR against this file.

• Supported route: POST /v1/chat/completions. The audio, image and embedding routes work for the models marked for them in the model list. POST /v1/messages (Anthropic format) answers for some models (greenpt/deepseek-v4.1-flash, tinfoil/deepseek-v4-1-flash, tinfoil/glm-5.3, tinfoil/gpt-oss-120b) and returns 404 for others (greenpt/glm-5.3-flash): it is LiteLLM's built-in translation layer, not something the co-op tests or supports. Where it works it carries tools and tool results correctly, but it drops the cost field and any reasoning content, so prefer chat completions for anything you meter.
• Fallbacks are silent in the body, visible in the headers. Some models have a fallback chain (today greenpt/deepseek-v4.1-flash → tinfoil/deepseek-v4-1-flash → tinfoil/gpt-oss-120b). If the first provider fails, the next one serves your request at its own price and its own privacy posture. Every response carries x-litellm-model-group (what you asked for), x-litellm-model-name and x-litellm-model-api-base (what answered), and x-litellm-attempted-fallbacks (how many hops). Read them if the provider matters to you.
• Cost per call is in the headers too: x-litellm-response-cost in USD, split into -input, -output and -reasoning, and x-litellm-key-spend for your key's running total. With streaming, usage.cost arrives in the final chunk instead.
• Timeouts and retries. 180 seconds per request. Connection errors are retried up to twice; timeouts are never retried, because the provider has already billed the generation.
• Reasoning is on by default on every chat model we tested, and reasoning tokens count against max_tokens. How much varies: the GreenPT GLM models think for a dozen tokens or so and then answer, while the Tinfoil models and DeepSeek on either provider can think for hundreds. A 700-token max_tokens on tinfoil/glm-5.3 or tinfoil/gpt-oss-120b can come back as finish_reason: "length" with 700 reasoning tokens and no content. To turn it off per request, send "reasoning_effort": "none" (or "chat_template_kwargs": {"enable_thinking": false}); "off" is rejected with a 400, and tinfoil/gpt-oss-120b rejects "none" too, so thinking can't be disabled on that model. Anthropic-style thinking: {...} is dropped without an error.
• Unknown parameters are dropped, not rejected (drop_params is on). If a parameter seems to have no effect, that is probably why.
• Usage. With "stream_options": {"include_usage": true} the last chunk carries prompt_tokens, completion_tokens, completion_tokens_details.reasoning_tokens, prompt_tokens_details.cached_tokens, and cost. Some providers add an impact block (energy in Wms, emissions in µgCO₂e) on non-streaming responses.
• Model metadata is readable. GET /model/info with your key returns every model's model_info (pricing, and whatever capability flags are declared there), so a client can pick a model by what it supports.
• Budget exhausted. When your allowance is spent, the gateway returns HTTP 400 with error.type: "budget_exceeded" before any stream starts. Match on the type, not the status. Your allowance resets on your billing date; the dashboard shows when.
• Read your own usage. Your key can read its own record (GET /key/info) and its own budget container (GET /team/info?team_id=…, with the team_id from /key/info): spend, max_budget, and budget_reset_at. Handy for a client that wants to stop before the gateway does. The scoping is enforced: asking for another team's team_id returns 403. /spend/logs is admin-only.
• No rate limits beyond the budget, and no request-size cap we could find below 1.5 MB. There is one shared container; please be a good neighbour.
• No uptime guarantee (Terms, "Service"). GET /health/liveliness and GET /health/readiness answer without auth, so a client can check before it falls back to something else.