docs/api-access.md

API access

Members can use the co-op's models programmatically through an OpenAI-compatible API. It's the same infrastructure that powers the chat — any OpenAI-compatible client works.

Quick facts

• Base URL: https://gateway.inference.coop/v1
• Auth: a bearer API key (create one in the Member Dashboard)
• Format: OpenAI chat completions — POST https://gateway.inference.coop/v1/chat/completions
• Models: see the model list.

API usage draws from the same monthly allowance as chat — there's no separate quota.

Get a key

Log in to the Member Dashboard and create an API key. The key is an sk-… string you pass as a bearer token.

Plain curl

curl https://gateway.inference.coop/v1/chat/completions \
  -H "Authorization: Bearer sk-your-key" \
  -H "Content-Type: application/json" \
  -d '{"model": "tinfoil/deepseek-v4-1-flash", "messages": [{"role": "user", "content": "Hello"}]}'

OpenAI SDK (Python, Node, etc.)

Point the SDK at our base URL and use your member key:

from openai import OpenAI

client = OpenAI(
    base_url="https://gateway.inference.coop/v1",
    api_key="sk-your-key",
)
resp = client.chat.completions.create(
    model="tinfoil/deepseek-v4-1-flash",
    messages=[{"role": "user", "content": "Hello"}],
)

Any OpenAI-compatible client works the same way: set the base URL to https://gateway.inference.coop/v1 and the API key to your member key.

Coding agents and other OpenAI-compatible clients

Any tool that speaks the OpenAI API works. Where a tool asks for provider or endpoint configuration:

• Base URL / endpoint: https://gateway.inference.coop/v1
• API type / protocol: "OpenAI compatible" (or openai-completions where a protocol string must be named) — not "Ollama", "Anthropic", or "Responses"
• API key: your member key (the sk-… string)
• Model: one from the model list, e.g. tinfoil/deepseek-v4-1-flash or greenpt/glm-5.3-flash

A typical provider entry looks like:

{
  "providers": {
    "inference-coop": {
      "baseUrl": "https://gateway.inference.coop/v1",
      "api": "openai-completions",
      "apiKey": "sk-your-key",
      "models": [
        { "id": "tinfoil/deepseek-v4-1-flash" },
        { "id": "tinfoil/gpt-oss-120b" },
        { "id": "tinfoil/glm-5.3" },
        { "id": "greenpt/glm-5.3-flash" }
      ]
    }
  }
}

If a tool's own docs show a local-server example (e.g. baseUrl: http://localhost:11434/v1 with api: "openai-completions"), substitute our base URL and your member key for apiKey — the protocol value stays the same. Some tools append /chat/completions to the base URL themselves; if your base URL already ends in /v1, that's correct and you should not add /chat/completions.

What the gateway guarantees (and what it doesn't)

Checked against gateway.inference.coop (LiteLLM 1.103.2) with a member key on 2026-10-19. If any of this drifts, open a PR against this file.

• Supported route: POST /v1/chat/completions. The audio, image and embedding routes work for the models marked for them in the model list. POST /v1/messages (Anthropic format) also answers: it is LiteLLM's built-in translation layer, not something the co-op tests or supports. It carries tools and tool results correctly, but it drops the cost field and any reasoning content, so prefer chat completions for anything you meter.
• Fallbacks are silent in the body, visible in the headers. Some models have a fallback chain (today greenpt/deepseek-v4.1-flash → tinfoil/deepseek-v4-1-flash → tinfoil/gpt-oss-120b). If the first provider fails, the next one serves your request at its own price and its own privacy posture. Every response carries x-litellm-model-group (what you asked for), x-litellm-model-name and x-litellm-model-api-base (what answered), and x-litellm-attempted-fallbacks (how many hops). Read them if the provider matters to you.
• Cost per call is in the headers too: x-litellm-response-cost in USD, split into -input, -output and -reasoning, and x-litellm-key-spend for your key's running total. With streaming, usage.cost arrives in the final chunk instead.
• Timeouts and retries. 180 seconds per request. Connection errors are retried up to twice; timeouts are never retried, because the provider has already billed the generation.
• Reasoning is on by default for tinfoil/deepseek-v4-1-flash and tinfoil/glm-5.3, and reasoning tokens count against max_tokens. A 700-token max_tokens can come back as finish_reason: "length" with 700 reasoning tokens and no content. To turn it off per request, send "reasoning_effort": "none" (or "chat_template_kwargs": {"enable_thinking": false}). Anthropic-style thinking: {...} is dropped without an error.
• Unknown parameters are dropped, not rejected (drop_params is on). If a parameter seems to have no effect, that is probably why.
• Usage. With "stream_options": {"include_usage": true} the last chunk carries prompt_tokens, completion_tokens, completion_tokens_details.reasoning_tokens, prompt_tokens_details.cached_tokens, and cost. Some providers add an impact block (energy in Wms, emissions in µgCO₂e) on non-streaming responses.
• Model metadata is readable. GET /model/info with your key returns every model's model_info (pricing, and whatever capability flags are declared there), so a client can pick a model by what it supports.
• Budget exhausted. When your allowance is spent, the gateway returns HTTP 400 with error.type: "budget_exceeded" before any stream starts. Match on the type, not the status. Your allowance resets on your billing date; the dashboard shows when.
• Read your own usage. Your key can read its own record (GET /key/info) and its own budget container (GET /team/info?team_id=…, with the team_id from /key/info): spend, max_budget, and budget_reset_at. Handy for a client that wants to stop before the gateway does. /spend/logs is admin-only.
• No rate limits beyond the budget, and no request-size cap we could find below 1.5 MB. There is one shared container; please be a good neighbour.
• No uptime guarantee (Terms, "Service"). GET /health/liveliness and GET /health/readiness answer without auth, so a client can check before it falls back to something else.