docs/api-access.md

# API access

Members can use the co-op's models programmatically through an OpenAI-compatible
API. It's the same infrastructure that powers the chat — any OpenAI-compatible
client works.

## Quick facts

- **Base URL:** `https://gateway.inference.coop/v1`
- **Auth:** a bearer API key (create one in the [Member Dashboard](https://dashboard.inference.coop))
- **Format:** OpenAI chat completions — `POST https://gateway.inference.coop/v1/chat/completions`
- **Models:** see the [model list](models.md).

API usage draws from the **same monthly allowance as chat** — there's no separate
quota.

## Get a key

Log in to the [Member Dashboard](https://dashboard.inference.coop) and create an
API key. The key is an `sk-…` string you pass as a bearer token.

## Plain `curl`

```bash
curl https://gateway.inference.coop/v1/chat/completions \
  -H "Authorization: Bearer sk-your-key" \
  -H "Content-Type: application/json" \
  -d '{"model": "tinfoil/deepseek-v4-1-flash", "messages": [{"role": "user", "content": "Hello"}]}'
```

## OpenAI SDK (Python, Node, etc.)

Point the SDK at our base URL and use your member key:

```python
from openai import OpenAI

client = OpenAI(
    base_url="https://gateway.inference.coop/v1",
    api_key="sk-your-key",
)
resp = client.chat.completions.create(
    model="tinfoil/deepseek-v4-1-flash",
    messages=[{"role": "user", "content": "Hello"}],
)
```

Any OpenAI-compatible client works the same way: set the base URL to
`https://gateway.inference.coop/v1` and the API key to your member key.

## Coding agents and other OpenAI-compatible clients

Any tool that speaks the OpenAI API works. Where a tool asks for provider or
endpoint configuration:

- **Base URL / endpoint:** `https://gateway.inference.coop/v1`
- **API type / protocol:** "OpenAI compatible" (or `openai-completions` where a
  protocol string must be named) — not "Ollama", "Anthropic", or "Responses"
- **API key:** your member key (the `sk-…` string)
- **Model:** one from the [model list](models.md), e.g. `tinfoil/deepseek-v4-1-flash` or `greenpt/glm-5.3-flash`

A typical provider entry looks like:

```json
{
  "providers": {
    "inference-coop": {
      "baseUrl": "https://gateway.inference.coop/v1",
      "api": "openai-completions",
      "apiKey": "sk-your-key",
      "models": [
        { "id": "tinfoil/deepseek-v4-1-flash" },
        { "id": "tinfoil/gpt-oss-120b" },
        { "id": "tinfoil/glm-5.3" },
        { "id": "greenpt/glm-5.3-flash" }
      ]
    }
  }
}
```

If a tool's own docs show a local-server example (e.g.
`baseUrl: http://localhost:11434/v1` with `api: "openai-completions"`),
substitute our base URL and your member key for `apiKey` — the protocol value
stays the same. Some tools append `/chat/completions` to the base URL themselves;
if your base URL already ends in `/v1`, that's correct and you should not add
`/chat/completions`.

## What the gateway guarantees (and what it doesn't)

Checked against `gateway.inference.coop` (LiteLLM 1.103.2) with a member key
on 2026-10-19. If any of this drifts, open a PR against this file.

- **Supported route:** `POST /v1/chat/completions`. The audio, image and
  embedding routes work for the models marked for them in the
  [model list](models.md). `POST /v1/messages` (Anthropic format) also
  answers: it is LiteLLM's built-in translation layer, not something the co-op
  tests or supports. It carries tools and tool results correctly, but it drops
  the `cost` field and any reasoning content, so prefer chat completions for
  anything you meter.
- **Fallbacks are silent in the body, visible in the headers.** Some models
  have a fallback chain (today `greenpt/deepseek-v4.1-flash` →
  `tinfoil/deepseek-v4-1-flash` → `tinfoil/gpt-oss-120b`). If the first
  provider fails, the next one serves your request at its own price and its
  own privacy posture. Every response carries `x-litellm-model-group` (what
  you asked for), `x-litellm-model-name` and `x-litellm-model-api-base` (what
  answered), and `x-litellm-attempted-fallbacks` (how many hops). Read them
  if the provider matters to you.
- **Cost per call is in the headers too:** `x-litellm-response-cost` in USD,
  split into `-input`, `-output` and `-reasoning`, and `x-litellm-key-spend`
  for your key's running total. With streaming, `usage.cost` arrives in the
  final chunk instead.
- **Timeouts and retries.** 180 seconds per request. Connection errors are
  retried up to twice; timeouts are never retried, because the provider has
  already billed the generation.
- **Reasoning is on by default** for `tinfoil/deepseek-v4-1-flash` and
  `tinfoil/glm-5.3`, and reasoning tokens count against `max_tokens`. A
  700-token `max_tokens` can come back as `finish_reason: "length"` with 700
  reasoning tokens and no content. To turn it off per request, send
  `"reasoning_effort": "none"` (or
  `"chat_template_kwargs": {"enable_thinking": false}`). Anthropic-style
  `thinking: {...}` is dropped without an error.
- **Unknown parameters are dropped, not rejected** (`drop_params` is on). If
  a parameter seems to have no effect, that is probably why.
- **Usage.** With `"stream_options": {"include_usage": true}` the last chunk
  carries `prompt_tokens`, `completion_tokens`,
  `completion_tokens_details.reasoning_tokens`,
  `prompt_tokens_details.cached_tokens`, and `cost`. Some providers add an
  `impact` block (energy in Wms, emissions in µgCO₂e) on non-streaming
  responses.
- **Model metadata is readable.** `GET /model/info` with your key returns
  every model's `model_info` (pricing, and whatever capability flags are
  declared there), so a client can pick a model by what it supports.
- **Budget exhausted.** When your allowance is spent, the gateway returns
  HTTP `400` with `error.type: "budget_exceeded"` before any stream starts.
  Match on the type, not the status. Your allowance resets on your billing
  date; the dashboard shows when.
- **Read your own usage.** Your key can read its own record
  (`GET /key/info`) and its own budget container
  (`GET /team/info?team_id=…`, with the `team_id` from `/key/info`): spend,
  `max_budget`, and `budget_reset_at`. Handy for a client that wants to stop
  before the gateway does. `/spend/logs` is admin-only.
- **No rate limits beyond the budget, and no request-size cap we could find
  below 1.5 MB.** There is one shared container; please be a good neighbour.
- **No uptime guarantee** (Terms, "Service"). `GET /health/liveliness` and
  `GET /health/readiness` answer without auth, so a client can check before
  it falls back to something else.