docs: reorganise into Guides / Operations / Governance
- Guides/: getting started, guest access, models, API access, energy & carbon - Operations/: agentic systems (was 'working with the agent'), infrastructure, roles, steward policy - Governance/: unchanged, now with its overview page Lateral links and the homepage updated; redirects keep the old root URLs alive.
This commit is contained in:
1 parent
a6142d68f2
commit
ac8cac02c6
13 files changed
+66
-29
No files matched your search
@@ -0,0 +1,151 @@
|
||||
# API access
|
||||
|
||||
Members can use the co-op's models programmatically through an OpenAI-compatible
|
||||
API. It's the same infrastructure that powers the chat — any OpenAI-compatible
|
||||
client works.
|
||||
|
||||
## Quick facts
|
||||
|
||||
- **Base URL:** `https://gateway.inference.coop/v1`
|
||||
- **Auth:** a bearer API key (create one in the [Member Dashboard](https://dashboard.inference.coop))
|
||||
- **Format:** OpenAI chat completions — `POST https://gateway.inference.coop/v1/chat/completions`
|
||||
- **Models:** see the [model list](models.md).
|
||||
- **Not only chat:** the gateway also serves a decision endpoint,
|
||||
`POST https://gateway.inference.coop/v1/systemone` — see
|
||||
[Decisions](models.md#decisions). It takes a different request shape, so
|
||||
it is not reachable through an OpenAI SDK's `chat.completions` call.
|
||||
|
||||
API usage draws from the **same monthly allowance as chat** — there's no separate
|
||||
quota.
|
||||
|
||||
## What the co-op stores about your requests
|
||||
|
||||
Usage records, and nothing else: the model, token counts, cost, and which key made
|
||||
the request — that is how your monthly metering works. **Your prompts and the
|
||||
responses are not stored on our side.** The API path keeps no conversation, so there
|
||||
is no transcript to delete later.
|
||||
|
||||
Two things worth being precise about:
|
||||
|
||||
- The **chat interface** is different — it saves your conversations so you can revisit
|
||||
them. If you want the API's behaviour there instead, use a **temporary chat**.
|
||||
- The **provider** is a separate question. Whether a model's operator retains anything
|
||||
is set by that provider, not by us, and it varies by model — the
|
||||
[model list](models.md) distinguishes policy-based zero retention from
|
||||
enclave-based protection.
|
||||
|
||||
Full detail, including what we can and cannot see: [privacy policy](../governance/privacy-policy.md).
|
||||
|
||||
## Get a key
|
||||
|
||||
Log in to the [Member Dashboard](https://dashboard.inference.coop)) and create an
|
||||
API key. The key is an `sk-…` string you pass as a bearer token.
|
||||
|
||||
## Plain `curl`
|
||||
|
||||
```bash
|
||||
curl https://gateway.inference.coop/v1/chat/completions \
|
||||
-H "Authorization: Bearer sk-your-key" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"model": "tinfoil/deepseek-v4-1-flash", "messages": [{"role": "user", "content": "Hello"}]}'
|
||||
```
|
||||
|
||||
## OpenAI SDK (Python, Node, etc.)
|
||||
|
||||
Point the SDK at our base URL and use your member key:
|
||||
|
||||
```python
|
||||
from openai import OpenAI
|
||||
|
||||
client = OpenAI(
|
||||
base_url="https://gateway.inference.coop/v1",
|
||||
api_key="sk-your-key",
|
||||
)
|
||||
resp = client.chat.completions.create(
|
||||
model="tinfoil/deepseek-v4-1-flash",
|
||||
messages=[{"role": "user", "content": "Hello"}],
|
||||
)
|
||||
```
|
||||
|
||||
Any OpenAI-compatible client works the same way: set the base URL to
|
||||
`https://gateway.inference.coop/v1` and the API key to your member key.
|
||||
|
||||
## Decisions (Metis)
|
||||
|
||||
Alongside the chat models, the gateway exposes GreenPT's **Metis** decision model
|
||||
at `POST https://gateway.inference.coop/v1/systemone`. It is the same key and the
|
||||
same base URL — only the path and the request body differ. For what it is, the
|
||||
three question types, and a full `curl` example, see
|
||||
[Decisions in the model list](models.md#decisions).
|
||||
|
||||
Two practical notes:
|
||||
|
||||
- **It is not an OpenAI chat endpoint.** You send a `state` plus typed `questions`
|
||||
and receive probabilities; there is nothing for an OpenAI SDK's
|
||||
`chat.completions` to parse. Post the body directly (`curl`, `requests`,
|
||||
`httpx`, or an SDK's generic POST).
|
||||
- **Its usage is metered like any other call.** Each decision is billed at
|
||||
**$0.12 per 1M input tokens**, figured from the exact token count GreenPT
|
||||
returns (Metis generates no prose, so there is no output cost). Decisions draw
|
||||
on the same monthly allowance as chat.
|
||||
|
||||
## Coding agents and other OpenAI-compatible clients
|
||||
|
||||
Any tool that speaks the OpenAI API works. Where a tool asks for provider or
|
||||
endpoint configuration:
|
||||
|
||||
- **Base URL / endpoint:** `https://gateway.inference.coop/v1`
|
||||
- **API type / protocol:** "OpenAI compatible" (or `openai-completions` where a
|
||||
protocol string must be named) — not "Ollama", "Anthropic", or "Responses"
|
||||
- **API key:** your member key (the `sk-…` string)
|
||||
- **Model:** a **stable alias** — `deepseek-flash-green`, `glm-flash-green`, `deepseek-flash-tee`, or `glm-tee`
|
||||
— or any id from the [model list](models.md), e.g. `tinfoil/deepseek-v4-1-flash`
|
||||
|
||||
A typical provider entry looks like:
|
||||
|
||||
```json
|
||||
{
|
||||
"providers": {
|
||||
"inference-coop": {
|
||||
"baseUrl": "https://gateway.inference.coop/v1",
|
||||
"api": "openai-completions",
|
||||
"apiKey": "sk-your-key",
|
||||
"models": [
|
||||
{ "id": "deepseek-flash-green" },
|
||||
{ "id": "glm-flash-green" },
|
||||
{ "id": "deepseek-flash-tee" },
|
||||
{ "id": "glm-tee" }
|
||||
]
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
The four names in `models` are **stable aliases**: we re-point them when a
|
||||
provider retires or replaces a model, so a config that pins one keeps working
|
||||
across transitions. The suffix says which route: `-green` is the default
|
||||
(renewable energy, policy-based privacy), `-tee` serves the same family inside
|
||||
attested hardware. Each alias is paired, so the two routes sit side by side in the
|
||||
model list.
|
||||
Pricing is set at the alias, so a transition that changes price appears in the
|
||||
model list rather than arriving as a surprise. If you need one particular version,
|
||||
use the provider-prefixed id from the [model list](models.md) instead.
|
||||
|
||||
Where the aliases point today:
|
||||
|
||||
| Alias | Serves today | Route |
|
||||
| --- | --- | --- |
|
||||
| `deepseek-flash-green` | `greenpt/deepseek-v4.1-flash` | Renewable energy; policy-based privacy |
|
||||
| `glm-flash-green` | `greenpt/glm-5.3-flash` | Renewable energy; policy-based privacy |
|
||||
| `deepseek-flash-tee` | `tinfoil/deepseek-v4-1-flash` | Confidential computing (TEE) |
|
||||
| `glm-tee` | `tinfoil/glm-5.3` | Confidential computing (TEE) |
|
||||
|
||||
*Points as of 2026-11-14. We re-point these when a provider retires or replaces a
|
||||
model; the alias names stay as they are, and this table changes with them.*
|
||||
|
||||
If a tool's own docs show a local-server example (e.g.
|
||||
`baseUrl: http://localhost:11434/v1` with `api: "openai-completions"`),
|
||||
substitute our base URL and your member key for `apiKey` — the protocol value
|
||||
stays the same. Some tools append `/chat/completions` to the base URL themselves;
|
||||
if your base URL already ends in `/v1`, that's correct and you should not add
|
||||
`/chat/completions`.
|
||||
@@ -0,0 +1,80 @@
|
||||
# Energy and carbon
|
||||
|
||||
The co-op shows members how much energy its inference uses, and what carbon that
|
||||
represents. This page says where those numbers come from and where they are weak.
|
||||
The [model list](models.md) covers which providers we use and their energy
|
||||
sources; this covers how we count.
|
||||
|
||||
## What we measure
|
||||
|
||||
**Output tokens, per model, per day** — read from our own gateway logs. Nothing
|
||||
is required of your client, and no per-member figure is published: the dashboards
|
||||
show co-op totals.
|
||||
|
||||
We count **output tokens only**, deliberately. That is the unit
|
||||
[GreenPT's published metric](https://greenpt.com/blog/introducing-a-new-ai-metric-to-drive-sustainability)
|
||||
uses ("mWh per 100 tokens"); input tokens are excluded so comparisons hold across
|
||||
prompts, workloads and models. It also means these figures are **not** total
|
||||
compute.
|
||||
|
||||
## How tokens become energy and carbon
|
||||
|
||||
```
|
||||
energy = output tokens × (Wh per 1,000 output tokens)
|
||||
carbon = energy × grid intensity of the electricity (gCO2e per kWh)
|
||||
```
|
||||
|
||||
Both factors are held per provider in one table, with the source recorded for
|
||||
each row, in the portal's
|
||||
[`metrics_snapshot.py`](https://git.inference.coop/code/member-portal/src/branch/main/metrics_snapshot.py).
|
||||
|
||||
### Measured: GreenPT
|
||||
|
||||
GreenPT returns an `impact` object with every call — energy in watt-milliseconds,
|
||||
emissions in µgCO2e — so these are fitted from their own numbers rather than
|
||||
imported from a third party. Measured 2026-10-28 by regressing energy against
|
||||
output tokens across prompt sizes (R² ≥ 0.999; there is a real per-request
|
||||
overhead, because their energy derives from inference *time*):
|
||||
|
||||
| Model | Wh per 1,000 output tokens | Implied grid intensity |
|
||||
|---|---|---|
|
||||
| `greenpt/deepseek-v4.1-flash` | 0.36 | ~193 gCO2e/kWh |
|
||||
| `greenpt/glm-5.3-flash` | 1.08 | ~193 gCO2e/kWh |
|
||||
| `greenpt/green-r` | 0.55 | ~24 gCO2e/kWh |
|
||||
|
||||
`green-r` is roughly an order of magnitude cleaner per unit of energy, which is
|
||||
where GreenPT's claim of up to 40% lower CO2 per query comes from.
|
||||
|
||||
### Estimated: everyone else
|
||||
|
||||
Providers that do not disclose impact — Tinfoil, Regolo, PublicAI — use a
|
||||
**documented order-of-magnitude proxy**: 0.5 Wh per 1,000 output tokens at an EU
|
||||
grid average of 250 gCO2e/kWh. It exists so a day's total is never quietly
|
||||
missing models. It is an estimate, and it is labelled as one everywhere it
|
||||
appears. Refining a factor is a data edit, not a code change.
|
||||
|
||||
## What this does not cover
|
||||
|
||||
- **It is a floor, not a total.** Embedding, audio and image requests carry no
|
||||
completion tokens, so their compute is not counted at all. This is the largest
|
||||
known gap.
|
||||
- **Estimates for non-disclosing providers.** The proxy is an order-of-magnitude
|
||||
figure, not a measurement of what a given provider actually draws.
|
||||
- **GreenPT's energy is itself estimated** — from time and documented hardware
|
||||
power — so our per-token figures are an efficiency ratio, not a per-call meter
|
||||
reading.
|
||||
- **Training emissions are not included**, for any model we serve. Nobody in this
|
||||
market can verify them yet.
|
||||
|
||||
## Where to see the numbers
|
||||
|
||||
The [member dashboard](https://dashboard.inference.coop) has them under
|
||||
"Co-op usage" — by day, alongside membership, requests and spend. Operators see
|
||||
the same series on the admin panel's Trends tab.
|
||||
|
||||
## Units, for the curious
|
||||
|
||||
`Wh` is a watt-hour (a 100 W bulb for one hour is 100 Wh). `kWh` is 1,000 Wh.
|
||||
`gCO2e` is grams of carbon-dioxide equivalent. Converting between them is where
|
||||
mistakes hide: µgCO2e per mWh and gCO2e per kWh are the same ratio, and Wh to kWh
|
||||
is a factor of 1,000.
|
||||
@@ -0,0 +1,209 @@
|
||||
# Getting started
|
||||
|
||||
Welcome. You're joining a **member-governed** AI cooperative — not signing up
|
||||
for a service from a vendor. You'll help direct the project while getting
|
||||
private AI chat and API access in return.
|
||||
|
||||
This guide covers the three things most new members want: becoming a member,
|
||||
using the chat, and getting involved in governance.
|
||||
|
||||
The co-op's meetings — including the regular member meetings — are all on the
|
||||
shared [co-op calendar](https://cloud.inference.coop/apps/calendar/p/Ltb6LAbt7NWHH7Td),
|
||||
where we share dates and details. Check it for what's coming up.
|
||||
|
||||
## 1. Become a member
|
||||
|
||||
Membership is a **sliding-scale contribution starting at $5/month** through
|
||||
Open Collective (the suggested level is $15 — pay what fits your
|
||||
circumstances).
|
||||
|
||||
1. Go to [opencollective.com/inference-cooperative](https://opencollective.com/inference-cooperative).
|
||||
2. Choose a contribution amount ($5 minimum) and subscribe.
|
||||
3. After you contribute, you'll land on a "check your inbox" page.
|
||||
|
||||
**Within a few minutes, you'll get a welcome email** with a link to set up your
|
||||
account. Click it and create your login.
|
||||
|
||||
> **One thing that matters:** the email address you use on Open Collective and
|
||||
> the email address you use to set up your account **must match**. Membership is
|
||||
> verified by matching those two emails. If they differ, you won't be recognized
|
||||
> as a member. (Contributed without an Open Collective account? You're recognized
|
||||
> by the email you entered at checkout.)
|
||||
|
||||
**On payment timing:** Open Collective charges your contribution on the
|
||||
collective's billing day, not the day you joined — so your second charge can
|
||||
arrive sooner than a month after the first. When that leaves a paid month
|
||||
shorter than a month, we credit the difference automatically as non-expiring
|
||||
credits, pro-rated to the days lost. It's automatic: no need to ask, claim it,
|
||||
or contact anyone.
|
||||
|
||||
### Your membership includes
|
||||
|
||||
- **Private AI chat** through the co-op's shared gateway.
|
||||
- **API access** for your own tools and projects (see [API access](api-access.md)).
|
||||
- **A monthly usage allowance** of $15 of credit, shared between chat and API.
|
||||
- **Credit packs** — buy extra non-expiring credits when you need more.
|
||||
- **Guest access** — give a friend $2 of credit and 60 days of the co-op.
|
||||
See [Guest access](guest-access.md).
|
||||
- **A voice in governance** — how the project is run and where it goes next.
|
||||
|
||||
## 2. Use the chat
|
||||
|
||||
The chat lives at **[chat.inference.coop](https://chat.inference.coop)**. Log in
|
||||
with the account you created in step 1.
|
||||
|
||||
- **Choose your model.** You can pick not just *which* model but *where* it
|
||||
runs: **private** (Tinfoil, runs in encrypted enclaves) or **green**
|
||||
(GreenPT and Regolo, 100% renewable energy — Regolo adds a written
|
||||
zero-data-retention policy). See the [model list](models.md) for the full
|
||||
list.
|
||||
- **Web search** — click the **globe** next to the message box to ground
|
||||
answers in current web results (self-hosted search, no external dependency).
|
||||
- **File uploads** — attach a document with the **paperclip** and ask the
|
||||
chat about it.
|
||||
- **Tools** — the **Tools menu** next to the message box lists everything the
|
||||
chat can do: generate images, search the co-op documentation, and more. If
|
||||
there's a tool you'd like available, ask in #general.
|
||||
- **Voice** works in both directions: speak to the chat with the microphone
|
||||
icon, and have replies read aloud.
|
||||
- **Temporary chats** — for conversations you don't want kept on our servers
|
||||
at all. Click the **Temporary Chat** toggle (in the chat header) and the
|
||||
conversation is never saved: it disappears when you leave the page, and
|
||||
nothing lands in your history or our database. To make this your default,
|
||||
open **Settings → Conversations** and turn on **Default Temporary Chat**.
|
||||
For anything sensitive, we recommend a temporary chat — ideally paired with
|
||||
a 🔒 enclave model. See [Privacy](../governance/privacy-policy.md#4-what-we-can-see-an-honest-caveat)).
|
||||
|
||||
Your usage draws from your monthly allowance; you can see it on the
|
||||
[Member Dashboard](https://dashboard.inference.coop).
|
||||
|
||||
If the chat ever says your balance is out, that's the allowance, not an error —
|
||||
see "Your allowance" below.
|
||||
|
||||
> **Moved from the old chat?** The previous Open WebUI instance remains
|
||||
> readable at [chat-legacy.inference.coop](https://chat-legacy.inference.coop)
|
||||
> — your full history is here in the new chat, but files you attached in the
|
||||
> old one are still accessible only there.
|
||||
|
||||
### Add the chat to your phone
|
||||
|
||||
The chat works like an app on your phone — no app store needed. Open
|
||||
[chat.inference.coop](https://chat.inference.coop) in your phone's browser and
|
||||
log in, then:
|
||||
|
||||
- **iPhone (Safari):** tap the **Share** button (the square with an arrow),
|
||||
then **Add to Home Screen**.
|
||||
- **Android (Chrome):** tap the **⋮** menu, then **Add to Home Screen**
|
||||
(or **Install app**).
|
||||
|
||||
You'll get an icon on your home screen that opens the chat full-screen, like
|
||||
any other app. Your login stays active — just tap and chat.
|
||||
|
||||
## 3. Get involved in governance
|
||||
|
||||
You don't just use the co-op; you help run it. Decisions about models, pricing,
|
||||
privacy, and direction happen on our **Loomio forum**:
|
||||
|
||||
- **[forum.inference.coop](https://forum.inference.coop)** — where members discuss and decide.
|
||||
- **[matrix.inference.coop](https://matrix.inference.coop)** — our Matrix chat server. Log in
|
||||
with your co-op account (Cloudron SSO); your Matrix ID is `@username:inference.coop`.
|
||||
Every member is invited to the **Inference Cooperative** space automatically — check your
|
||||
invites on first login.
|
||||
|
||||
**Using it from any Matrix client** — the server speaks the standard Matrix protocol,
|
||||
so you can connect from any client:
|
||||
|
||||
- **Element** (web, desktop, iOS, Android): on the sign-in screen, click *Edit* next to
|
||||
"Homeserver" and enter `https://matrix.inference.coop`, then sign in with your co-op
|
||||
account. (On [app.element.io](https://app.element.io) or the desktop/mobile apps alike.)
|
||||
**Note:** the new **Element X** apps (iOS/Android/macOS) are *not* currently supported —
|
||||
they only work with the newer Matrix Authentication Service, which this server doesn't
|
||||
run yet. Use the classic Element apps (Element Web / Element Desktop / "Element" on the
|
||||
app stores) or another client below.
|
||||
- **FluffyChat, Cinny, SchildiChat** or any other client: look for the "custom homeserver"
|
||||
/ "other server" option in the client's login screen and enter the same address.
|
||||
- Your co-op account and your existing Matrix account on another server (e.g.
|
||||
matrix.org or a personal homeserver) are separate identities. **Pick your
|
||||
identity (optional):** by default, invitations go to your co-op identity
|
||||
(`@username:inference.coop`). To use an account you already have instead:
|
||||
|
||||
1. Open your [dashboard](https://dashboard.inference.coop) and find the
|
||||
**Matrix chat** card. It shows the Matrix ID currently registered for
|
||||
you — your co-op identity, or `pending` if your co-op Matrix account
|
||||
hasn't been created yet (it is created the first time you log in to
|
||||
Matrix).
|
||||
2. Click **Use external account**, enter your existing Matrix ID in the
|
||||
format `@username:server.org`, and click **Save**.
|
||||
3. Accept the space invitation that arrives in that account — usually
|
||||
instantly. If the account can't be reached right away, the invitation
|
||||
is retried automatically each night.
|
||||
|
||||
That's all it takes: invitations (and the space) follow whichever identity
|
||||
you've chosen. Switching back is one click — **Use
|
||||
@username:inference.coop** on the same card. One identity is active at a
|
||||
time, and both work equally well in every co-op room.
|
||||
|
||||
**Members-only, readable history.** The space is invite-only and unencrypted
|
||||
so every member can read the full history (including messages from before
|
||||
they joined). Invites are restricted to admins — so while any member is
|
||||
welcome to bring a collaborator or guest, an admin sends the invite rather
|
||||
than members handing the space to outsiders directly.
|
||||
|
||||
- **#general** is the main room — say hello, ask questions, start
|
||||
conversations. Beyond it, the space holds a handful of topical rooms
|
||||
(infrastructure, design, an automated co-op updates feed) — explore the
|
||||
space and join whatever is interesting to you. New rooms appear there as
|
||||
the co-op grows.
|
||||
|
||||
- The updates feed is posted by the co-op's automation account. See
|
||||
[Working with the agent](../operations/agentic-systems.md) for the agents behind
|
||||
it, what they do, and how members interact with them.
|
||||
|
||||
- See the [Charter](../governance/charter.md) for how governance is structured.
|
||||
- See [Open questions](../governance/open-questions.md) for the decisions currently on the table.
|
||||
|
||||
|
||||
### Help build it
|
||||
|
||||
Everything we run is open source, and members are welcome to contribute code,
|
||||
documentation, and design. See [Infrastructure](../operations/infrastructure.md) for the map
|
||||
and the "Contributing" section for how to propose changes through
|
||||
[git.inference.coop](https://git.inference.coop).
|
||||
|
||||
## Your allowance and credits
|
||||
|
||||
Membership includes **$15 of usage credit each month** (same for every member,
|
||||
regardless of what you pay), shared between chat and API. It's a starting
|
||||
value, not a hard rule:
|
||||
|
||||
- The allowance resets each month on your personal billing date (the day your
|
||||
membership payment recurs) — not on a fixed calendar day.
|
||||
- The allowance can be adjusted per member — contact info@inference.coop.
|
||||
|
||||
### Need more than the allowance?
|
||||
|
||||
Members can buy **credit packs** — non-expiring credits that kick in
|
||||
automatically when your monthly allowance runs out:
|
||||
|
||||
- Credits **never expire**. Unused credits carry over month to month.
|
||||
- They're drawn only after the monthly allowance is exhausted, so your
|
||||
allowance always gets used first.
|
||||
- Buy a credit pack on our
|
||||
[Open Collective page](https://opencollective.com/inference-cooperative/contribute)
|
||||
(choose the "Credit pack" option). Credits are applied to your account
|
||||
automatically — usually within minutes.
|
||||
- Credit packs are for members. If you buy one before joining, we'll hold it
|
||||
and it will be applied to your account when you join (or refunded if you
|
||||
prefer — just email info@inference.coop).
|
||||
|
||||
You can see both buckets — allowance left this cycle and non-expiring credits —
|
||||
on the [Member Dashboard](https://dashboard.inference.coop).
|
||||
|
||||
## Need help?
|
||||
|
||||
- **Questions about the service or your account:** info@inference.coop
|
||||
- **Governance and decisions:** the [forum](https://forum.inference.coop)
|
||||
- **Building on the API:** [API access](api-access.md)
|
||||
- **Leaving:** cancel your subscription on Open Collective; your account is
|
||||
deactivated and your data preserved for reactivation. You may request full
|
||||
deletion.
|
||||
@@ -0,0 +1,44 @@
|
||||
# Guest access
|
||||
|
||||
Any member can **give a friend guest access to the co-op**: **$2 of AI usage
|
||||
for 60 days**, spent in the chat and the API. It doesn't come out of your own
|
||||
balance. You can send it once every 30 days, and an address can receive it
|
||||
only once.
|
||||
|
||||
## How to send guest access
|
||||
|
||||
On your [dashboard](https://dashboard.inference.coop), open the **Guests**
|
||||
card, enter your friend's email address (name optional), and click
|
||||
**Send invite**. There's no approval step: the account is provisioned right
|
||||
away, and your friend gets an **invitation email** from the co-op, naming
|
||||
you, with a link to set up their account. Setting it up takes about a minute.
|
||||
|
||||
New guests show as **Invited — awaiting setup** until they accept it. The
|
||||
card also lists the guests you've sent.
|
||||
|
||||
## What your guest gets
|
||||
|
||||
- **$2 of credit** for the chat and the API — a hard cap: usage stops when
|
||||
it's spent.
|
||||
- **Their own dashboard** at
|
||||
[dashboard.inference.coop](https://dashboard.inference.coop) showing what's
|
||||
left.
|
||||
- **The weekly newsletter** and these public docs.
|
||||
|
||||
Guest access is not membership — no Matrix, governance forum, or vote.
|
||||
|
||||
## When it ends
|
||||
|
||||
When **the $2 is spent or the 60 days are up**, whichever comes first, your
|
||||
friend gets an email saying so. Access stops; **the account and its data are
|
||||
kept** — and a guest can end it early with the **Cancel guest access** link
|
||||
on their dashboard, returning any unspent credit to the co-op.
|
||||
|
||||
## Becoming a member
|
||||
|
||||
Contribute on
|
||||
[Open Collective](https://opencollective.com/inference-cooperative/contribute)
|
||||
and the account converts automatically — same login, same dashboard. Any
|
||||
unspent guest credit carries over. See
|
||||
[Become a member](getting-started.md#1-become-a-member) for how membership
|
||||
works.
|
||||
@@ -0,0 +1,12 @@
|
||||
# Guides
|
||||
|
||||
How to join, use the chat, choose a model, call the API, and understand what that
|
||||
usage costs in energy and emissions.
|
||||
|
||||
## In this section
|
||||
|
||||
- [Getting started](getting-started.md) — join, use the chat, and get involved in governance.
|
||||
- [Guest access](guest-access.md) — inviting someone without a full membership.
|
||||
- [Models](models.md) — the catalogue of providers and models.
|
||||
- [API access](api-access.md) — use the models programmatically.
|
||||
- [Energy & carbon](energy.md) — the energy and emissions behind the service.
|
||||
@@ -0,0 +1,316 @@
|
||||
# Models
|
||||
|
||||
The co-op serves models from multiple providers. This is the single list — both
|
||||
the chat and the API draw from it, so it's the one place to look (and the one
|
||||
place to update).
|
||||
|
||||
## Providers
|
||||
|
||||
Models are named `provider/model-name`, so you can choose not just *which* model
|
||||
but *where* it runs. Providers differ along two independent axes: **privacy**
|
||||
(how protected your prompts and responses are) and **green energy** (how the
|
||||
compute is powered). These are gradations, not binary switches — and no
|
||||
provider today is best on both.
|
||||
|
||||
| Provider | Value | Privacy | Green energy | What that means |
|
||||
|---|---|---|---|---|
|
||||
| **Tinfoil** | *private* | **Strongest — architectural** | Not a green claim | Runs in hardware enclaves (TEEs) with end-to-end encryption (EHBP), verifiable via remote attestation. Neither we nor Tinfoil can read your prompts or responses at inference time. Energy mix is not disclosed. |
|
||||
| **GreenPT** | *green* | **Policy-based** | **Strongest — 100% renewable** | Standard (non-enclave) hosting in the EU; privacy rests on GreenPT's commitments and EU data rules, not on cryptography. Compute runs on 100% renewable energy. |
|
||||
| **Regolo** | *green* | Policy-based — **zero data retention** | **100% renewable** | EU-hosted (Italy, Seeweb) with a stated zero-data-retention policy: prompts are not stored or reused. Compute runs on 100% renewable energy. |
|
||||
|
||||
**Reading the gradations:**
|
||||
|
||||
- **Privacy:** *architectural* (Tinfoil) beats *policy-based* (GreenPT,
|
||||
Regolo). An enclave protects you even if the provider is compromised; a
|
||||
policy protects you only as far as the provider keeps its word. Both are
|
||||
better than a provider with neither — and Regolo adds a written
|
||||
zero-data-retention commitment on top of its policy.
|
||||
How the co-op measures its own energy use and carbon emissions is on the
|
||||
[energy and carbon page](energy.md) — including what those figures cover and what
|
||||
they miss.
|
||||
|
||||
- **Green energy:** *100% renewable* (GreenPT, Regolo) beats *undisclosed*
|
||||
(Tinfoil). GreenPT and Regolo both run on 100% renewable energy; Tinfoil's
|
||||
mix is not disclosed.
|
||||
|
||||
Trade-offs are real: the most private option (Tinfoil) isn't the greenest, and
|
||||
the greenest option (GreenPT) isn't the most private. The point of the co-op is
|
||||
that **you choose** which axis matters more for a given task.
|
||||
|
||||
**How to pick, quick guide:**
|
||||
|
||||
- **Default** → `greenpt/deepseek-v4.1-flash`: DeepSeek's efficient multimodal
|
||||
reasoning model on renewable energy, and the model the chat starts on. For the
|
||||
TEE-protected (private) version of the same model, use `tinfoil/deepseek-v4-1-flash`.
|
||||
- **Everyday chat and drafting** → a fast model: `greenpt/glm-5.3-flash`.
|
||||
These handle the bulk of what people use chat for, at a fraction of flagship
|
||||
cost.
|
||||
- **Hard reasoning, coding, agentic work** → a flagship: `greenpt/glm-5.3`
|
||||
(renewable) or `tinfoil/glm-5.3` (TEE-protected) — the GLM flagship on the
|
||||
provider of your choice — or `tinfoil/deepseek-v4-1-flash` for long documents
|
||||
and tool-driven tasks.
|
||||
- **Images in, text out (vision)** → `tinfoil/deepseek-v4-1-flash` or the
|
||||
Apertus models — see the ✅ in the table below.
|
||||
- **Cheap and cheerful** → `tinfoil/gpt-oss-120b` or `greenpt/glm-5.3-flash`.
|
||||
- **Model origins matter to you** → the Apertus models (publicly developed;
|
||||
see below).
|
||||
- **Semantic search / RAG (API)** → an embedding model: `greenpt/qwen3-embedding-8b`
|
||||
or `greenpt/green-embeddings` (renewable energy), or `tinfoil/nomic-embed-text`
|
||||
(TEE-protected — the model the co-op's own docs search runs on). API-only; see
|
||||
[Embedding models](#embedding-models) below.
|
||||
|
||||
## Current models
|
||||
|
||||
Pricing is per 1 million tokens, input / output / **cached input** (what the
|
||||
co-op pays the provider — your own cost is your monthly allowance, not
|
||||
per-token billing). The cached-input rate applies when a model reuses a
|
||||
shared prompt prefix (multi-turn conversations, agent loops); most providers
|
||||
charge a steep discount for it. Models with `—` don't offer a distinct
|
||||
cache-read price. **Vision** means the model accepts images as input
|
||||
(screenshots, photos, documents to reason about). **Image generation** is
|
||||
available via the chat's image feature (the 🎨 button) using
|
||||
`regolo/qwen-image`.
|
||||
|
||||
The emoji in the model name marks its **value tier** (the same icons used in the chat picker):
|
||||
|
||||
- 🔒 **Private** — Tinfoil: architecturally private (TEE enclaves)
|
||||
- 🌱 **Green** — GreenPT and Regolo: renewable-powered EU hosts (Regolo adds
|
||||
a zero-data-retention commitment). The 🌱 emoji marks renewable-powered
|
||||
providers, not a specific company.
|
||||
|
||||
| Model | Provider | Privacy | Green | Vision | Context | Pricing (in / out / cached) | Notes |
|
||||
|---|---|---|---|---|---|---|---|
|
||||
| `🌱 greenpt/deepseek-v4.1-flash` | GreenPT | policy | ✅ 100% renewable | ✅ | 1M | $0.36 / $1.45 / $0.04 | **Default.** DeepSeek's multimodal reasoning model, renewable energy; TEE twin below. Serves the **1017 snapshot** (17 Oct 2026) as of 2026-11-16; GreenPT no longer recommends the base release. The id keeps its version-free name so existing configs keep working. |
|
||||
| `🔒 tinfoil/deepseek-v4-1-flash` | Tinfoil | architectural | undisclosed | ✅ | 1M | $0.65 / $1.45 / $0.13 | TEE-protected version of the default; image input, tool calling, long docs/agents. |
|
||||
| `🔒 tinfoil/glm-5.3` | Tinfoil | architectural | undisclosed | — | 1M | $1.80 / $5.75 / $0.45 | GLM flagship, TEE-protected; strongest private-tier model for agentic coding and long-horizon tasks. |
|
||||
| `🔒 tinfoil/gpt-oss-120b` | Tinfoil | architectural | undisclosed | — | 131K | $0.15 / $0.60 / — | Lightweight open-weight reasoning; cheapest private tier. |
|
||||
| `🌱 greenpt/glm-5.3-flash` | GreenPT | policy | ✅ 100% renewable | — | 1M | $0.11 / $0.44 / $0.022 | Fast GLM model, renewable energy. |
|
||||
| `🌱 greenpt/glm-5.3` | GreenPT | policy | ✅ 100% renewable | — | 1M | $1.10 / $4.40 / $0.275 | GLM flagship — strongest agentic model in the green tier. |
|
||||
| `🌱 regolo/apertus-70b` | Regolo | policy + ZDR | ✅ 100% renewable | ✅ | 262K | $1.16 / $3.94 / — | Fully open sovereign model (Apache-2.0), large/capable, image understanding. |
|
||||
| `🌱 regolo/qwen3.5-122b` | Regolo | policy + ZDR | ✅ 100% renewable | ✅ | 262K | $0.70 / $3.94 / — | Large vision+reasoning MoE (Qwen 3.5, 122B-A10B); 262K context. |
|
||||
| `🌱 regolo/gemma4-31b` | Regolo | policy + ZDR | ✅ 100% renewable | ✅ | 262K | $0.40 / $0.60 / — | Google's dense multimodal Gemma 4 (31B); image input, native function calling. Served from Regolo's EU renewable-powered infrastructure. |
|
||||
| `🌱 regolo/mistral-small-4-119b` | Regolo | policy + ZDR | ✅ 100% renewable | ✅ | 262K | $0.20 / $0.70 / — | Mistral's hybrid Small 4 (119B) — instruct + reasoning + coding unified; image input, tool calling. Served from Regolo's EU renewable-powered infrastructure. |
|
||||
|
||||
Vision is verified working through the co-op gateway (base64 image input via
|
||||
`/v1/chat/completions`, standard OpenAI format). In the chat UI, just attach
|
||||
an image to your message while using a ✅ model.
|
||||
|
||||
Context figures are **measured on the served model**, not copied from a
|
||||
provider's catalogue: a prompt of that size is sent through the gateway and the
|
||||
token count is read back. Where the two disagree, the measured figure is the one
|
||||
shown here — and on Regolo they disagree a lot. Its catalogue advertises a
|
||||
**30K input limit** for `regolo/apertus-70b`, **60K** for
|
||||
`regolo/mistral-small-4-119b` and **100K** for `regolo/gemma4-31b`; all four
|
||||
Regolo chat models the co-op serves accept **262,144 tokens** and reject anything
|
||||
above it (measured 2026-11-13 by probing at 262,144 and above, so the ceiling is
|
||||
exact rather than approximate). If a limit matters to you, ask us to probe the
|
||||
model you intend to use rather than trusting a published number.
|
||||
|
||||
**Reading the Privacy column:** *architectural* means the protection is built
|
||||
into the infrastructure (TEE enclaves) — even the provider can't read your
|
||||
prompts. *Policy* means the provider commits, by contract and policy, not to
|
||||
store or reuse your data. *ZDR* (zero data retention) is a stronger policy:
|
||||
the provider keeps no copy of prompts or outputs at all after the response is
|
||||
delivered.
|
||||
|
||||
**Same model, two providers:** `glm-5.3` (Z.ai's GLM flagship) is offered both
|
||||
on GreenPT (renewable, $1.10/$4.40) and on Tinfoil (TEE-protected, $1.80/$5.75)
|
||||
— pick the axis that matters to you. The fast `glm-5.3-flash` variant is
|
||||
GreenPT-only (Tinfoil deprecated their flash variant in October 2026).
|
||||
|
||||
### Audio models
|
||||
|
||||
These are **not chat models** — they power the chat's voice mode (dictation
|
||||
and spoken replies) and are available through the API on `/v1/audio/*` routes
|
||||
for members building speech into their own tools. They're hidden from the chat
|
||||
model picker to avoid confusion.
|
||||
|
||||
| Model | Provider | Use | Pricing |
|
||||
|---|---|---|---|
|
||||
| `🔒 tinfoil/whisper-large-v3-turbo` | Tinfoil | Speech-to-text | $0.01 per request (flat; zero per-token) |
|
||||
| `🔒 tinfoil/voxtral-small-24b` | Tinfoil | Speech-to-text (token-priced) | $0.20 / 1M input + $0.60 / 1M output tokens |
|
||||
| `🔒 tinfoil/voxtral-tts` | Tinfoil | Text-to-speech | $0.002 per request (zero per-token) |
|
||||
|
||||
All three run inside Tinfoil's enclaves, so voice data gets the same
|
||||
architectural privacy as the private chat models. TTS voices available:
|
||||
`neutral_female`, `neutral_male`, `casual_female/male`, `cheerful_female`,
|
||||
plus French, German, Spanish, Italian, Portuguese, Dutch, and Hindi variants.
|
||||
|
||||
Tinfoil bills these at cost, but its audio API returns **no token usage** to
|
||||
the gateway — so dashboard rows for these models read `$0.01 · 0 tokens`: the
|
||||
charge is real, the token columns stay empty.
|
||||
|
||||
`whisper-large-v3-turbo` and `voxtral-tts` charge **flat per request**.
|
||||
`voxtral-small-24b` is token-priced and the better pick for small requests:
|
||||
sent through the chat API with `input_audio` it meters exactly per token, and
|
||||
on `/v1/audio/transcriptions` (the route dictation uses) the gateway bills
|
||||
Tinfoil's measured chunk-equivalent — their audio encoder tokenizes in ~30 s
|
||||
chunks, so a short request costs ≈**$0.0001**, about 1/125th of the whisper
|
||||
floor. For audio longer than ~30 s, prefer the chat route, which meters
|
||||
exactly.
|
||||
|
||||
```bash
|
||||
# Speech-to-text
|
||||
curl https://gateway.inference.coop/v1/audio/transcriptions \
|
||||
-H "Authorization: Bearer ***" \
|
||||
-F file=@your-audio.mp3 \
|
||||
-F model=tinfoil/whisper-large-v3-turbo
|
||||
|
||||
# Speech-to-text (token-priced model, chat route — meters exactly per token)
|
||||
curl https://gateway.inference.coop/v1/chat/completions \
|
||||
-H "Authorization: Bearer ***" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"model": "tinfoil/voxtral-small-24b", "messages": [{"role": "user", "content": [
|
||||
{"type": "text", "text": "Transcribe this audio."},
|
||||
{"type": "input_audio", "input_audio": {"data": "<base64 audio>", "format": "wav"}}
|
||||
]}]}'
|
||||
|
||||
# Text-to-speech (returns WAV audio)
|
||||
curl https://gateway.inference.coop/v1/audio/speech \
|
||||
-H "Authorization: Bearer ***" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"model": "tinfoil/voxtral-tts", "input": "Hello from the co-op", "voice": "neutral_female"}' \
|
||||
--output speech.wav
|
||||
|
||||
# Vision (image input to a multimodal chat model)
|
||||
curl https://gateway.inference.coop/v1/chat/completions \
|
||||
-H "Authorization: Bearer ***" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"model": "tinfoil/deepseek-v4-1-flash", "messages": [{"role": "user", "content": [
|
||||
{"type": "text", "text": "What is in this image?"},
|
||||
{"type": "image_url", "image_url": {"url": "data:image/png;base64,..."}}
|
||||
]}]}'
|
||||
```
|
||||
|
||||
### Embedding models
|
||||
|
||||
Embeddings turn text into a numeric vector — the input to semantic search,
|
||||
clustering and RAG. They're called on `/v1/embeddings` with your co-op API key
|
||||
and are **not chat models**: they never appear in the chat picker, the API is
|
||||
the interface (the same shape as the audio models above). You choose between
|
||||
them the same way as for chat: a TEE-protected option (Tinfoil) or
|
||||
renewable-energy options (GreenPT).
|
||||
|
||||
| Model | Provider | Privacy | Green | Dimensions | Max input | Pricing |
|
||||
|---|---|---|---|---|---|---|
|
||||
| `🌱 greenpt/green-embeddings` | GreenPT | policy | ✅ 100% renewable | 2560 (measured) | 32K tokens (per GreenPT's catalog) | $0.20 / 1M input tokens (no output charge) |
|
||||
| `🌱 greenpt/qwen3-embedding-8b` | GreenPT | policy | ✅ 100% renewable | 4096 (measured) | 32K tokens (per GreenPT's catalog) | $0.25 / 1M input tokens (no output charge) |
|
||||
| `🔒 tinfoil/nomic-embed-text` | Tinfoil | **architectural (TEE)** | — (mix not disclosed) | 768 (measured) | 8K tokens (per Tinfoil's catalog) | $0.016 / 1M input tokens (no output charge) |
|
||||
|
||||
```bash
|
||||
curl https://gateway.inference.coop/v1/embeddings \
|
||||
-H "Authorization: Bearer ***" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"model": "tinfoil/nomic-embed-text", "input": "the text to embed"}'
|
||||
```
|
||||
|
||||
The response is standard OpenAI shape: `data[0].embedding` holds the vector,
|
||||
and `usage.prompt_tokens` is what meters against your allowance at the rate
|
||||
above. Embeddings bill input tokens only — there is no output charge.
|
||||
|
||||
Vectors from different models are not interchangeable — pick one per search
|
||||
index and stay with it (re-index if you switch). All three are callable with a
|
||||
member key; for reference, the co-op's own documentation search runs on
|
||||
`tinfoil/nomic-embed-text` inside the TEE.
|
||||
|
||||
### Image generation
|
||||
|
||||
`🌱 regolo/qwen-image` (Regolo, EU-hosted, zero data retention) generates
|
||||
images from a text prompt. It runs the December 2025 "2512" checkpoint of
|
||||
Qwen-Image — not the newer Qwen Image 2.1 (September 2026), which Regolo does
|
||||
not list (as of 2026-11-09). In the chat, press the **Integrations** button
|
||||
below the message box, choose **Image**, and describe what you want — the
|
||||
generated image appears in the conversation. Via the API:
|
||||
|
||||
```bash
|
||||
curl https://gateway.inference.coop/v1/images/generations \
|
||||
-H "Authorization: Bearer ***" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"model": "regolo/qwen-image", "prompt": "a watercolor painting of a mountain lake", "n": 1, "size": "512x512"}'
|
||||
```
|
||||
|
||||
The response returns base64-encoded JPEG(s). Billed per pixel generated.
|
||||
|
||||
### About the Apertus models
|
||||
|
||||
Apertus is the **Swiss AI Initiative's** open foundation model, built by a
|
||||
public collaboration between **EPFL, ETH Zurich, and the Swiss National
|
||||
Supercomputing Centre (CSCS)**. It's "fully open" in the strict sense — not just
|
||||
downloadable weights:
|
||||
|
||||
- **Open weights & code** — released under the **Apache-2.0** license, so anyone
|
||||
can use, modify, and redistribute it, commercially or not.
|
||||
- **Open training data** — trained only on *fully open* data (unlike most
|
||||
proprietary models, which keep their training set secret). The data, code,
|
||||
methods, and alignment principles are all published and reproducible.
|
||||
- **Open values** — governed by a public **Apertus Charter** that documents the
|
||||
model's values and principles rather than leaving them implicit.
|
||||
- **Sovereign & public-good** — designed to European data-protection norms
|
||||
(GDPR, EU AI Act, Swiss law) as an example of "AI as a public good," not a
|
||||
proprietary product. Trained across 1,500+ languages (40% non-English).
|
||||
|
||||
The v1.5 generation adds image understanding, a longer 262K-token context
|
||||
window, improved tool use, and instruction-following. It's the *publicly
|
||||
developed* counterpart to the private (Tinfoil) and renewable (GreenPT)
|
||||
offerings — sovereignty as a third axis of member choice.
|
||||
|
||||
For more information on open-source AI and the Apertus models, see the
|
||||
[European Open Source AI Index](https://osai-index.eu/database/).
|
||||
|
||||
*Source: [apertus-ai.org](https://www.apertus-ai.org/) and the
|
||||
[Swiss AI Hugging Face](https://huggingface.co/swiss-ai) model cards.*
|
||||
|
||||
Some models also have a discounted cached-input rate (when a prompt is reused
|
||||
and doesn't need to be re-read) — see the `code/litellm` config for the exact
|
||||
numbers. Pricing here mirrors the `model_info` in the LiteLLM config
|
||||
(`code/litellm/config.yaml`), which is the authoritative source — the two are
|
||||
kept in sync.
|
||||
|
||||
## Decisions
|
||||
|
||||
Most models here generate text. **Metis**, from GreenPT, makes *decisions*
|
||||
instead: you give it a `state` (the situation, in plain text) plus up to 32
|
||||
typed questions, and it returns **probabilities, choices, and scores** — never
|
||||
prose. Use it when you want a calibrated judgement rather than a written
|
||||
answer: "should we…?", "which option…?", "how confident is that?".
|
||||
|
||||
It's on the gateway at `/v1/systemone`, called with your normal co-op API key
|
||||
(the same key you use for chat):
|
||||
|
||||
```bash
|
||||
curl https://gateway.inference.coop/v1/systemone \
|
||||
-H "Authorization: Bearer ***" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"model": "metis",
|
||||
"state": "Your situation, described in plain text.",
|
||||
"questions": {
|
||||
"go": {"type": "noul", "instructions": "Should we proceed?",
|
||||
"criteria": {"true": "proceed", "false": "do not proceed"}},
|
||||
"which": {"type": "choice", "instructions": "Which factor matters most?",
|
||||
"criteria": {"cost": "price", "speed": "latency"}},
|
||||
"confidence": {"type": "score", "instructions": "How confident are you?",
|
||||
"criteria": ["very low", "low", "medium", "high", "very high"]}
|
||||
}
|
||||
}'
|
||||
```
|
||||
|
||||
Metis is hosted by **GreenPT** — the same policy-based privacy, EU hosting, and
|
||||
100% renewable energy as the 🌱 chat models. **Decisions count toward your
|
||||
monthly allowance**, like every other call: each request is metered at
|
||||
**$0.12 per 1M input tokens**, computed from the exact token count GreenPT
|
||||
reports back, and appears in your usage record alongside your chat calls.
|
||||
(Metis generates no prose, so there is no output charge.)
|
||||
|
||||
## Limitations on privacy
|
||||
|
||||
Only **Tinfoil** offers architectural privacy. The other providers are chosen
|
||||
for their value (renewable energy, public models) but do not run in enclaves —
|
||||
prompts and responses pass through them in the ordinary way. Your **chat
|
||||
history** is also stored on our server so you can revisit it, and that stored
|
||||
history is not encrypted in a way that prevents us from technically reading it —
|
||||
we commit not to. Requests made through the **API** with your own key are a
|
||||
different matter: those are **not stored on our side at all**, beyond the usage
|
||||
record needed for metering ([API access](api-access.md)). The full distinction —
|
||||
what's architecturally private versus what's a policy commitment — is in the
|
||||
[Privacy Policy](../governance/privacy-policy.md).
|
||||
Reference in new issue
Block a user