Files
docs/api-access.md
T
inference-bot cdb62fc247 docs: Metis decisions now count toward the allowance
Metering on /v1/systemone is live: each call is billed at $0.12 per 1M
input tokens, computed from the exact token count GreenPT returns and
attributed to the calling member key. Replace the "not yet metered"
warnings in models.md and api-access.md with the accurate position.

(metering verified 2026-11-03; code/litellm patch_passthrough_token_cost.py)
2026-11-03 19:03:27 +00:00

129 lines
4.9 KiB
Markdown

# API access
Members can use the co-op's models programmatically through an OpenAI-compatible
API. It's the same infrastructure that powers the chat — any OpenAI-compatible
client works.
## Quick facts
- **Base URL:** `https://gateway.inference.coop/v1`
- **Auth:** a bearer API key (create one in the [Member Dashboard](https://dashboard.inference.coop))
- **Format:** OpenAI chat completions — `POST https://gateway.inference.coop/v1/chat/completions`
- **Models:** see the [model list](models.md).
- **Not only chat:** the gateway also serves a decision endpoint,
`POST https://gateway.inference.coop/v1/systemone` — see
[Decisions](models.md#decisions). It takes a different request shape, so
it is not reachable through an OpenAI SDK's `chat.completions` call.
API usage draws from the **same monthly allowance as chat** — there's no separate
quota.
## What the co-op stores about your requests
Usage records, and nothing else: the model, token counts, cost, and which key made
the request — that is how your monthly metering works. **Your prompts and the
responses are not stored on our side.** The API path keeps no conversation, so there
is no transcript to delete later.
Two things worth being precise about:
- The **chat interface** is different — it saves your conversations so you can revisit
them. If you want the API's behaviour there instead, use a **temporary chat**.
- The **provider** is a separate question. Whether a model's operator retains anything
is set by that provider, not by us, and it varies by model — the
[model list](models.md) distinguishes policy-based zero retention from
enclave-based protection.
Full detail, including what we can and cannot see: [privacy policy](privacy-policy.md).
## Get a key
Log in to the [Member Dashboard](https://dashboard.inference.coop) and create an
API key. The key is an `sk-…` string you pass as a bearer token.
## Plain `curl`
```bash
curl https://gateway.inference.coop/v1/chat/completions \
-H "Authorization: Bearer sk-your-key" \
-H "Content-Type: application/json" \
-d '{"model": "tinfoil/deepseek-v4-1-flash", "messages": [{"role": "user", "content": "Hello"}]}'
```
## OpenAI SDK (Python, Node, etc.)
Point the SDK at our base URL and use your member key:
```python
from openai import OpenAI
client = OpenAI(
base_url="https://gateway.inference.coop/v1",
api_key="sk-your-key",
)
resp = client.chat.completions.create(
model="tinfoil/deepseek-v4-1-flash",
messages=[{"role": "user", "content": "Hello"}],
)
```
Any OpenAI-compatible client works the same way: set the base URL to
`https://gateway.inference.coop/v1` and the API key to your member key.
## Decisions (Metis)
Alongside the chat models, the gateway exposes GreenPT's **Metis** decision model
at `POST https://gateway.inference.coop/v1/systemone`. It is the same key and the
same base URL — only the path and the request body differ. For what it is, the
three question types, and a full `curl` example, see
[Decisions in the model list](models.md#decisions).
Two practical notes:
- **It is not an OpenAI chat endpoint.** You send a `state` plus typed `questions`
and receive probabilities; there is nothing for an OpenAI SDK's
`chat.completions` to parse. Post the body directly (`curl`, `requests`,
`httpx`, or an SDK's generic POST).
- **Its usage is metered like any other call.** Each decision is billed at
**$0.12 per 1M input tokens**, figured from the exact token count GreenPT
returns (Metis generates no prose, so there is no output cost). Decisions draw
on the same monthly allowance as chat.
## Coding agents and other OpenAI-compatible clients
Any tool that speaks the OpenAI API works. Where a tool asks for provider or
endpoint configuration:
- **Base URL / endpoint:** `https://gateway.inference.coop/v1`
- **API type / protocol:** "OpenAI compatible" (or `openai-completions` where a
protocol string must be named) — not "Ollama", "Anthropic", or "Responses"
- **API key:** your member key (the `sk-…` string)
- **Model:** one from the [model list](models.md), e.g. `tinfoil/deepseek-v4-1-flash` or `greenpt/glm-5.3-flash`
A typical provider entry looks like:
```json
{
"providers": {
"inference-coop": {
"baseUrl": "https://gateway.inference.coop/v1",
"api": "openai-completions",
"apiKey": "sk-your-key",
"models": [
{ "id": "tinfoil/deepseek-v4-1-flash" },
{ "id": "tinfoil/gpt-oss-120b" },
{ "id": "tinfoil/glm-5.3" },
{ "id": "greenpt/glm-5.3-flash" }
]
}
}
}
```
If a tool's own docs show a local-server example (e.g.
`baseUrl: http://localhost:11434/v1` with `api: "openai-completions"`),
substitute our base URL and your member key for `apiKey` — the protocol value
stays the same. Some tools append `/chat/completions` to the base URL themselves;
if your base URL already ends in `/v1`, that's correct and you should not add
`/chat/completions`.