Compare commits

...
21 Commits
Author SHA1 Message Date
inference-bot 2422432b85 Dark mode: search form wrapper also dark (the input's parent was still light) 2026-11-09 18:20:09 +00:00
inference-bot 5402b9e758 Dark mode: darker tab stripe (dark-border green) and darker pine search surfaces 2026-11-09 18:15:50 +00:00
inference-bot 96b1201f72 guest-access.md: dashboard self-serve invites replace the email path, note awaiting-setup state and credit return on cancel 2026-11-09 18:04:33 +00:00
inference-bot 852e52dc35 models.md: pin exact upstream versions of the default chat model and the image model
- greenpt/deepseek-v4.1-flash routes to GreenPT's base deepseek-v4.1-flash
  id, not the deepseek-v4.1-flash-1017 snapshot (17 Oct 2026) that GreenPT
  lists separately. Confirmed from the gateway registry mapping and
  GreenPT's model card / model list, 2026-11-09.
- regolo/qwen-image serves Regolo's single Qwen-Image id, which Regolo's
  catalog describes as the (2512) checkpoint — the December 2025 update,
  not the newer Qwen Image 2.1 (2026-09-20). Regolo does not list 2.1.
2026-11-09 15:44:44 +00:00
inference-bot ac75c504f6 Dark-mode search input: deep green instead of pine (stands out less) 2026-11-09 06:33:26 +00:00
inference-admin c6d03c8f85 Removing some open questions that have been closed 2026-11-09 06:28:34 +00:00
inference-bot fb14577626 Dark-mode search input: dark background with light text (was white box, light-on-light) 2026-11-09 06:27:23 +00:00
inference-bot 9119e6726a Dark header bar: #171717 with white text, matching the chat overlay's dark menu 2026-11-09 06:24:40 +00:00
inference-bot 70c87e47c2 Guest access: $5 -> $2 of credits; trim page to essentials 2026-11-08 20:12:28 +00:00
inference-bot 342c353ebc Rename 'Gift access' to 'Guest access' — pure terminology rename, page and nav moved to guest-access.md 2026-11-08 02:30:27 +00:00
inference-bot 0786d97286 Add 'Gift access' — documenting the member gift-account feature 2026-11-08 02:23:42 +00:00
inference-bot af4f96ad13 models: three member-facing embedding models (nomic un-scoped, green-embeddings added)
- tinfoil/nomic-embed-text is now member-facing ($0.016/1M per Tinfoil's
  catalog, TEE-protected, 768-dim, 8K ctx) — the model the co-op's own docs
  search runs on inside the TEE.
- greenpt/green-embeddings added ($0.20/1M, 2560-dim measured, 32K ctx) — the
  only genuinely distinct other embedding model in GreenPT's catalog.
- Embedding table now lists all three with pricing; quick-guide bullet updated.
2026-11-06 22:39:07 +00:00
inference-bot b263e125c4 models: correct audio pricing rows; add tinfoil/voxtral-small-24b
- whisper-large-v3-turbo: $0.01 per request (flat, zero per-token) — the old
  $0.05/1M-input row was stale (co-op/support#48)
- voxtral-tts: $0.002 per request (was 'listed $0 (verify on invoice)')
- new: tinfoil/voxtral-small-24b, Tinfoil's token-priced STT model
  ($0.20/1M in + $0.60/1M out), with the chunk-equivalent note for the
  transcription route and a chat-with-input_audio example
- note why dashboard audio rows read '$0.01 · 0 tokens' (Tinfoil returns no
  token usage for audio)
2026-11-06 22:33:42 +00:00
inference-bot fa4ba8d989 models: document greenpt/qwen3-embedding-8b — the member-facing embeddings API
New gateway model (code/litellm 54925ca): /v1/embeddings, GreenPT-hosted,
4096-dim (measured through the gateway), $0.25/1M input from GreenPT's own
pricing, 32K input cap (provider catalog). Not a chat model — API-only like
the audio models, hidden from the chat picker. Also states plainly that
Tinfoil's nomic-embed-text is the operator-only docs-RAG embedding, since a
member read its absence from the model list as removal.
2026-11-06 20:35:04 +00:00
inference-bot 2a89e6a19d Add the co-op meeting calendar to the homepage and Getting Started 2026-11-06 17:44:34 +00:00
inference-admin 5eacc1c116 Made explicit Nathan Schneider's founding role 2026-11-05 04:47:21 +00:00
inference-admin df722f81b0 A little tightening 2026-11-04 21:33:35 +00:00
docs-bot 278c1fda1a infrastructure: make the page member-facing; lighten the light-mode menu-bar stripe
- condense the page to what a member needs: where services run (Infomaniak),
  what backups mean to them, the .coop domain via domains.coop, mail via
  Scaleway, the Matrix space and the accounts they may meet
- drop operator detail (migration narration, DNS record inventory, scheduled-task
  internals, admin-only components, internal account/credential identifiers) to
  co-op/ops-docs; fix a heading glitch where a line began with '#infrastructure'
- brand.css: light-mode .md-tabs uses the header's pale wash #e8f5e8 (was the
  saturated --green-deep) with a warm hairline; dark scheme unchanged
2026-11-04 21:29:06 +00:00
inference-bot 869d16cc65 docs: note on Open Collective payment timing and automatic credit 2026-11-04 21:15:41 +00:00
inference-admin 07d5a60377 Adjusted framing from "private" to "ethical" 2026-11-04 20:54:48 +00:00
inference-bot 33b5cf73e8 docs site: label the site as 'docs' and link the co-op homepage
Make clear this is the documentation, not the co-op homepage:
- site_name -> 'Inference Cooperative docs' (drives the <title> and the
  header brand on every page)
- homepage h1 -> 'Inference Cooperative docs'
- footer gains a 'Co-op homepage' link to https://inference.coop/, added
  in the existing copyright.html override alongside the Gitea source link
2026-11-04 20:42:07 +00:00
10 changed files with 290 additions and 281 deletions

No files matched your search

+4 -3
View File
@@ -1,9 +1,9 @@
# Inference Cooperative
# Inference Cooperative docs
**AI, governed together.**
The Inference Cooperative is a member-governed project providing private AI
inference. Members pay dues on a sliding scale and, in return, get private AI
The Inference Cooperative is a member-governed project providing more ethical AI
inference. Members pay dues on a sliding scale and, in return, get AI
chat and API access through shared, cooperatively governed infrastructure. We
are fiscally sponsored by [Metagov](https://metagov.org), a nonprofit, and
funded through our [Open Collective](https://opencollective.com/inference-cooperative).
@@ -26,6 +26,7 @@ funded through our [Open Collective](https://opencollective.com/inference-cooper
- **Matrix chat:** [matrix.inference.coop](https://matrix.inference.coop)
- **Member Dashboard:** [dashboard.inference.coop](https://dashboard.inference.coop)
- **Governance forum:** [forum.inference.coop](https://forum.inference.coop)
- **Co-op calendar:** [cloud.inference.coop](https://cloud.inference.coop/apps/calendar/p/Ltb6LAbt7NWHH7Td)
- **Source code:** [git.inference.coop](https://git.inference.coop)
- **Contact:** [info@inference.coop](mailto:info@inference.coop)
+5 -1
View File
@@ -16,11 +16,15 @@ for our governance model, see the [Charter](charter.md).
The cooperative launched in September 2026, growing out of research on democratic
digital infrastructure through Metagov's [Collective Governance for AI:
Points of Intervention](https://metagov.org/cg-ai/).
Points of Intervention](https://metagov.org/cg-ai/). It was founded by
[Nathan Schneider](https://nathanschneider.info), who formerly co-founded
Social.coop, Start.coop, and the Media Economies Design Lab at the University
of Colorado Boulder.
It began with a single privacy-first provider and a deliberately small
toolset: a chat, an API gateway, and the basics for membership and payments.
From there it grew the way co-ops do: in response to what members asked for.
Green-energy providers were added, along with more diverse models.
A member dashboard made usage and spending transparent. A federated
[Matrix](https://matrix.inference.coop) server, a
[forum](https://forum.inference.coop), and a newsletter gave members places to
+13
View File
@@ -7,6 +7,10 @@ private AI chat and API access in return.
This guide covers the three things most new members want: becoming a member,
using the chat, and getting involved in governance.
The co-op's meetings — including the regular member meetings — are all on the
shared [co-op calendar](https://cloud.inference.coop/apps/calendar/p/Ltb6LAbt7NWHH7Td),
where we share dates and details. Check it for what's coming up.
## 1. Become a member
Membership is a **sliding-scale contribution starting at $5/month** through
@@ -26,12 +30,21 @@ account. Click it and create your login.
> as a member. (Contributed without an Open Collective account? You're recognized
> by the email you entered at checkout.)
**On payment timing:** Open Collective charges your contribution on the
collective's billing day, not the day you joined — so your second charge can
arrive sooner than a month after the first. When that leaves a paid month
shorter than a month, we credit the difference automatically as non-expiring
credits, pro-rated to the days lost. It's automatic: no need to ask, claim it,
or contact anyone.
### Your membership includes
- **Private AI chat** through the co-op's shared gateway.
- **API access** for your own tools and projects (see [API access](api-access.md)).
- **A monthly usage allowance** of $15 of credit, shared between chat and API.
- **Credit packs** — buy extra non-expiring credits when you need more.
- **Guest access** — give a friend $2 of credit and 60 days of the co-op.
See [Guest access](guest-access.md).
- **A voice in governance** — how the project is run and where it goes next.
## 2. Use the chat
+44
View File
@@ -0,0 +1,44 @@
# Guest access
Any member can **give a friend guest access to the co-op**: **$2 of AI usage
for 60 days**, spent in the chat and the API. It doesn't come out of your own
balance. You can send it once every 30 days, and an address can receive it
only once.
## How to send guest access
On your [dashboard](https://dashboard.inference.coop), open the **Guests**
card, enter your friend's email address (name optional), and click
**Send invite**. There's no approval step: the account is provisioned right
away, and your friend gets an **invitation email** from the co-op, naming
you, with a link to set up their account. Setting it up takes about a minute.
New guests show as **Invited — awaiting setup** until they accept it. The
card also lists the guests you've sent.
## What your guest gets
- **$2 of credit** for the chat and the API — a hard cap: usage stops when
it's spent.
- **Their own dashboard** at
[dashboard.inference.coop](https://dashboard.inference.coop) showing what's
left.
- **The weekly newsletter** and these public docs.
Guest access is not membership — no Matrix, governance forum, or vote.
## When it ends
When **the $2 is spent or the 60 days are up**, whichever comes first, your
friend gets an email saying so. Access stops; **the account and its data are
kept** — and a guest can end it early with the **Cancel guest access** link
on their dashboard, returning any unspent credit to the co-op.
## Becoming a member
Contribute on
[Open Collective](https://opencollective.com/inference-cooperative/contribute)
and the account converts automatically — same login, same dashboard. Any
unspent guest credit carries over. See
[Become a member](getting-started.md#1-become-a-member) for how membership
works.
+112 -226
View File
@@ -7,11 +7,11 @@ and contributors who want to work on it.
Members reach the co-op through two front doors: the **chat** (LibreChat) and
the **API** (the LiteLLM gateway). Both route through a single gateway to
multiple LLM providers — so members choose what they value for each task:
architecturally private inference in **TEE-protected enclaves** (Tinfoil),
100% renewable energy (GreenPT, Regolo). A thin layer of custom middleware —
the member portal — ties Open
Collective membership to Cloudron accounts and LiteLLM keys.
multiple LLM providers, so members can choose what they value for each task:
architecturally private inference in **TEE-protected enclaves** (Tinfoil), or
100% renewable energy (GreenPT, Regolo). A thin layer of custom middleware — the
member portal — ties Open Collective membership to Cloudron accounts and LiteLLM
keys.
```
┌─> Tinfoil (TEE) 🔒 private
@@ -28,228 +28,130 @@ single sign-on, and per-app isolation.
## Where it runs
The co-op's services run on a server the co-op owns and pays for directly, at
**Infomaniak** — a Swiss sovereign cloud provider whose data centres are in
Geneva, run on renewable energy, and certified to ISO 14001, ISO 50001 and
ISO 27001. The stack moved there from a personal Hostinger account on
**2026-11-02**; nothing about how members use the co-op changed in the move.
The co-op's member-facing services run on a server the co-op owns and pays for
directly, at **Infomaniak** — a Swiss sovereign cloud provider whose data centres
are in Geneva, run on renewable energy, and are certified to ISO 14001, ISO
50001 and ISO 27001.
One thing runs elsewhere: the co-op's **steward bot** — the automation behind the
digests and the newsletter drafts — runs on a separate machine, not on the
Cloudron server, and was not part of the migration.
### DNS
`inference.coop` is managed by hand at **domains.coop** (nameservers
`ns1/ns2.yoursrs.com`). The zone is deliberately small — **eleven records**. Most
of the co-op hangs off just a few of them: the apex (`inference.coop`), `www`,
and a wildcard (`*.inference.coop`) point at the server, so each application
hostname — chat, portal, dashboard, gateway, git, newsletter, vault and the rest
— is served by the wildcard rather than having a record of its own. The DNS
change sheet listing all eleven records is public, in
[`co-op/runbooks`](https://git.inference.coop/co-op/runbooks).
The co-op's **automation runs on a separate server**, also hosted on Infomaniak
and deliberately kept apart from the member-facing applications. The steward bot
— the automation behind the digests and the newsletter drafts — and the co-op's
other agents live there. Keeping them on their own host means an agent problem
stays an agent problem: it cannot reach the chat, the gateway or member data.
### Backups
Cloudron writes **encrypted** backups to **Infomaniak's object storage** (the
"Infomaniak Encrypted S3" site, in Switzerland) every night, and keeps the last
**seven days**. The steward-bot host backs up its own selected home directories
to the same object storage.
Cloudron writes **encrypted** backups to Infomaniak's object storage in
Switzerland every night and keeps the last **seven days**, so a failed upgrade
or a lost server can be restored. The automation host backs up its own data to
the same object storage.
### Domain and DNS
The co-op's domain is **inference.coop** — a `.coop` domain. The co-op registers
and manages its domains with **domains.coop**, the domain provider run by the
cooperative movement, and holds them in the co-op's own name rather than in any
individual's. DNS itself is deliberately simple, and the co-op's own nameservers
answer it.
## Components
| Component | Purpose | URL | Code |
|-----------|---------|-----|------|
| **LibreChat** | Member-facing chat (models, web search, image generation, voice) | chat.inference.coop | [co-op/librechat-custom](https://git.inference.coop/co-op/librechat-custom) |
| **Open WebUI** | Legacy chat instance — read-only reference (old history + file attachments) | chat-legacy.inference.coop | — |
| **LiteLLM** | AI gateway — keys, metering, model routing | gateway.inference.coop | [code/litellm](https://git.inference.coop/code/litellm) |
| **Member Portal** | Membership middleware — onboarding, key provisioning, Loomio sync | portal.inference.coop | [code/member-portal](https://git.inference.coop/code/member-portal) |
| **Member Dashboard** | Member-facing usage view + API key management | dashboard.inference.coop | [code/member-dashboard](https://git.inference.coop/code/member-dashboard) |
| **Admin Panel** | Admin-only member overview, spend, balance management | panel.inference.coop | [code/admin-panel](https://git.inference.coop/code/admin-panel) |
| **SearXNG** | Self-hosted web search (feeds the chat's search) | search.inference.coop | — |
| **Loomio** | Member governance | forum.inference.coop | — |
| **Matrix (Synapse)** | Member chat server (federated; SSO login via Cloudron) | matrix.inference.coop | — |
| **Gitea** | Git hosting (code + docs) | git.inference.coop | — |
| **Surfer** | Static hosting (landing page, thanks page) | inference.coop | [co-op/website](https://git.inference.coop/co-op/website) |
| **Listmonk** | Member newsletter (auto-synced from membership) | newsletter.inference.coop | — |
| **Vaultwarden** | Password manager (admin credentials) | vault.inference.coop | — |
| **SnappyMail** | Webmail | email.inference.coop | — |
| Component | Purpose | URL |
|-----------|---------|-----|
| **LibreChat** | Member-facing chat (models, web search, image generation, voice) | chat.inference.coop |
| **Open WebUI** | Legacy chat instance — read-only reference (old history and file attachments) | chat-legacy.inference.coop |
| **LiteLLM** | AI gateway — keys, metering and model routing, for chat and API | gateway.inference.coop |
| **Member portal** | Membership middleware — onboarding, key provisioning, Loomio sync | portal.inference.coop |
| **Member dashboard** | Your usage view and API keys | dashboard.inference.coop |
| **Loomio** | Member governance | forum.inference.coop |
| **Matrix (Synapse)** | Member chat server, federated, SSO login via Cloudron | matrix.inference.coop |
| **Gitea** | Git hosting for the co-op's code and this documentation | git.inference.coop |
| **Listmonk** | Member newsletter (synced from membership) | newsletter.inference.coop |
| **SnappyMail** | Webmail | email.inference.coop |
The co-op's software behind all of these is open source — see [The
code](#the-code) below for where each piece lives.
Open Collective (billing and fiscal sponsorship) is the one external service in
the flow — see [opencollective.com/inference-cooperative](https://opencollective.com/inference-cooperative).
the flow — see
[opencollective.com/inference-cooperative](https://opencollective.com/inference-cooperative).
## Mail
The `inference.coop` mail domain is the co-op's, and the mail itself runs on the
co-op's Cloudron server.
The co-op runs its own `@inference.coop` mail. **Incoming** mail arrives directly
on the co-op's server, and webmail and mail apps reach it over the usual
encrypted ports (IMAP and SMTP over TLS). **Outgoing** mail is sent through
**Scaleway**, an EU-based email service, authenticated for our domain.
**Incoming** mail is received directly by that server — the domain's MX record
points at it, it accepts mail on the standard SMTP port, and webmail and mail
apps reach it over the usual encrypted ports (IMAP and SMTP over TLS). If mail
to your `@inference.coop` address stops arriving, that is a fault on our side to
fix, not a limitation of how the co-op is set up.
**Outgoing** mail travels by two different paths, on purpose:
- The **newsletter** (Listmonk) sends through **Scaleway Transactional Email**,
an EU-based SMTP relay, rather than from our own server. Our domain is
authenticated to that relay with its own DKIM key, and a real message has been
checked end to end: SPF, DKIM and DMARC all pass under a `p=reject` policy —
so anything claiming to be from us that the relay hasn't authenticated is
rejected outright rather than quietly filed away.
- The **member portal's** transactional email — welcome messages, budget
notices, the confirmation the nightly membership sweep sends — still sends from
the co-op's own server, through the mail service Cloudron runs for us.
Both paths send as **Inference Cooperative `<info@inference.coop>`**.
Why split them: sending the newsletter through a relay lets a provider with
established sending IPs carry a list this size, instead of leaning on one
freshly-migrated server's reputation. Receiving we keep in-house, on our own
hardware.
You can reach your mailbox through webmail at email.inference.coop, or add it to
a standard mail app on your phone or computer. If mail to your `@inference.coop`
address stops arriving, that is a fault on our side to fix.
## Matrix
Members get a federated Matrix account on our self-hosted Synapse
(matrix.inference.coop, user IDs `@username:inference.coop`) using their
Cloudron SSO login. The portal invites every active member — and Nathan's
federated account — to the private **Inference Cooperative** Space (with a
**General** room) on every membership sync, so new members land in the space
automatically. Invites honor rate limits; external/federated members can be
added via the portal's `MATRIX_EXTRA_MXIDS` env.
Members get a federated Matrix account on the co-op's self-hosted Synapse
(matrix.inference.coop, user IDs `@username:inference.coop`), signing in with the
same Cloudron SSO as the chat. The co-op invites every active member to the
private **Inference Cooperative** space (with a **General** room) automatically,
so new members land there as they join. Federation is on, so you can also join
from any homeserver you already use.
Federation is enabled (delegated via `.well-known/matrix/server` on the bare
domain, port 443), so members can also join from any homeserver they already
use.
### Accounts you may meet
### Which account is which
A few accounts in the space are automation rather than people:
Several automated accounts touch this homeserver, and two of them once differed
by a single character — which caused real confusion, including a bot appearing to
announce itself. The distinction is deliberate:
- **`@inferencebot` — the steward.** It posts the digests, newsletter notes and
reminders. It holds no more power than posting needs.
- **`@inference-admin`** — the administrator of record for the co-op's rooms
(#general, #community, #design, #skillshare, #updates and #infrastructure) and
their membership. It **never speaks in a member room** — a message from it is
a mistake, not a feature.
- **`@inferencebot:inference.coop` — the steward.** The bot members actually meet:
it posts digests, newsletter notes and reminders. It is *not* a Synapse server
admin, and holds no room power beyond what it needs to post.
- **`@inference-admin:inference.coop` — the room and server administrator.** A
Synapse server admin, used for provisioning rooms, managing membership, and
retiring rooms. It **never speaks in a member room**; a message from it is a
mistake, not a feature.
- **`@infrastructure-bot:inference.coop` — the infrastructure working group's bot.**
Answers infrastructure questions in **#infrastructure** when mentioned.
- **`@ops_audit:inference.coop`** — a server-admin account created for the October
2026 security review. Retained only until its tokens are revoked.
The house rule: **power belongs to `@inference-admin`, voice belongs to
`@inferencebot`.** Please don't create an account whose name differs from an
existing one by a character or two — spotting lookalikes is exactly what this
convention is for.
**House rule: power belongs to `@inference-admin`, voice belongs to `@inferencebot`.**
Please don't create another account whose name differs from an existing one by a
character or two; that is the specific confusion this section exists to undo.
## Membership
**`@inference-bot:inference.coop` is retired (2026-10-29).** It was the original
room-administration account (it created the space and most of the rooms) and held a
Synapse server-admin flag. Its room power went to `@inference-admin`, which is now
administrator of record for #general, #community, #design, #skillshare, #updates and
#infrastructure, **and holds the portal's member-invite credential**
(`MATRIX_BOT_TOKEN`) under its own token. The old account's admin flag was dropped,
it was deactivated, and every device and token was deleted. Nothing should use it
again.
Two loose ends, recorded so nobody re-investigates them from scratch: it remains
joined to two rooms it cannot leave (they reject every write — `500` on leave), and
those rooms are also why they can't be deleted; and the old account's *name* is still
the tempting one, so check `whoami` rather than the name before trusting any account
that looks like it.
## How the apps run themselves
The portal and gateway are self-maintaining: their recurring housekeeping is
built into the apps as scheduled tasks, not run from any member's or operator's
personal machine.
- **Member portal** — two scheduled jobs (Cloudron scheduler):
- *Membership sweep* (nightly, 3am) — reconciles members against the live
Open Collective list: provisions new members who signed up while a webhook
was missed, deactivates lapsed members, and syncs Loomio + Matrix
membership. A missed webhook self-heals within 24 hours.
- *Docs knowledge refresh* (nightly, 3:30am) — pulls this documentation from
the public Git repo and updates the shared knowledge base behind the chat's
models, so answers grounded in the docs stay current. Changes to the docs
appear in the chat within a day, no manual step.
- **Gateway** — an *audio keep-alive* (every 10 minutes) sends a tiny text-to-
speech ping to the private enclave so voice mode responds quickly even after
long quiet periods, without compromising the end-to-end encryption model.
Scheduled tasks are declared in each app's `CloudronManifest.json` (see the
`code/` repos) and survive redeploys. If a task fails, it shows up in the
app's logs on the admin panel; the portal also exposes a manual trigger for
the membership sweep.
## How membership is wired
Membership flows through the member portal, which is the single source of truth
for "who is a member":
Membership runs through the member portal, the single source of truth for who is
a member:
1. A member contributes on Open Collective.
2. Open Collective fires a webhook to the portal, which provisions a Cloudron
account, a LiteLLM key, and a welcome email.
3. A nightly sweep reconciles against the live Open Collective list, so a missed
webhook self-heals within 24 hours.
2. Open Collective notifies the portal, which provisions a Cloudron account, a
LiteLLM key, and a welcome email.
3. A nightly reconciliation against the live Open Collective list means a missed
notification self-heals within a day.
The portal database is authoritative for member emails and balances; LiteLLM's
team budget is the source of truth for a member's allowance, shared between chat
and API.
### The bot account on Open Collective
You may notice **"Inference Co-op Bot"** listed as an admin of the collective on
Open Collective. This is infrastructure, not a member or a person:
- **What it is:** a dedicated Open Collective account owned by the co-op. It
holds a **Personal Token** that the member portal uses to read membership
data — the roster, tiers (needed to route credit-pack purchases correctly),
and contributor emails (including guests, which only admins can see).
- **What it can do:** read member and tier data via Open Collective's API.
It cannot move money, issue refunds, or change the collective — Open
Collective restricts those to dashboard actions by human admins. Its API
access was verified read-mostly during the October 2026 security review.
- **Why it has admin:** reading guest contributor emails requires admin on
Open Collective; without it, membership reconciliation and credit-pack
routing can't function.
- **Who supervises it:** all of its code is open source in
[code/member-portal](https://git.inference.coop/code/member-portal) — every
API call it makes is documented there. Nathan Schneider (collective admin)
oversees its use and can revoke or rotate its token at any time from the
Open Collective dashboard.
Nothing on Open Collective is charged, transferred, or refunded by automation:
payments always come from a human's own contribution action, and any automated
system only *reads* the resulting records.
You may see **"Inference Co-op Bot"** listed as an admin of the collective on
Open Collective. It is infrastructure, not a person or a member: a dedicated
account the portal uses, **read-only**, to check the member roster and tiers. It
cannot move money, issue refunds or change the collective — payments always come
from a human's own contribution, and automation only reads the result. Its code
is open source in
[code/member-portal](https://git.inference.coop/code/member-portal), and a
collective admin can revoke its access at any time.
## The code
The cooperative's software is open source, in the [`code`](https://git.inference.coop/code)
organization on Gitea:
The co-op's software is open source, in the [`code`](https://git.inference.coop/code)
and [`co-op`](https://git.inference.coop/co-op) organizations on Gitea:
- [**code/member-portal**](https://git.inference.coop/code/member-portal) — membership middleware (Python).
- [**code/member-dashboard**](https://git.inference.coop/code/member-dashboard) — member-facing usage + API keys (Python).
- [**code/admin-panel**](https://git.inference.coop/code/admin-panel) — admin overview (Python).
- [**code/member-dashboard**](https://git.inference.coop/code/member-dashboard) — member dashboard: usage and API keys.
- [**code/litellm**](https://git.inference.coop/code/litellm) — the gateway packaged as a Cloudron app.
Plus the `co-op` organization:
- [**co-op/website**](https://git.inference.coop/co-op/website) — landing page and static site.
- [**co-op/docs**](https://git.inference.coop/co-op/docs) — this documentation.
- [**co-op/design-assets**](https://git.inference.coop/co-op/design-assets) — logos, fonts, brand materials.
## Contributing (the Git is yours, too)
Everything the co-op runs is open source and lives at
[git.inference.coop](https://git.inference.coop) — the website, this
documentation, the member portal, the dashboard, the admin panel, and the
gateway. As a member, you can browse all of it and propose changes.
The chat, portal and other apps live in `code/` and `co-op/` too; everything is
browsable at [git.inference.coop](https://git.inference.coop). Everything the co-op runs
is open source, so as a member you can browse all of
it and propose changes.
**Who can sign in.** Git access is tied to your membership: you log into the Git
with the same single sign-on you use for the chat. Members are in automatically
(through the `members` group); when a membership lapses, Git access lapses with it.
(through the `members` group); when a membership lapses, Git access lapses with
it.
**How to contribute.** Anyone can read the repos without an account. To make a
change, fork the repo, edit in your fork, and open a **pull request**:
@@ -260,55 +162,39 @@ change, fork the repo, edit in your fork, and open a **pull request**:
4. Open a **pull request** back to the original repo, describing your change.
**All changes are reviewed.** Nothing merges directly into `main` — every change
comes in as a pull request that an admin reviews and approves first. You don't
need to be able to create repos or organizations to contribute; you just fork,
edit, and ask for review.
Questions about the Git itself? Write to info@inference.coop.
comes in as a pull request an admin reviews and approves first. You don't need
to be able to create repos or organizations to contribute; you just fork, edit
and ask for review.
## Models and providers
The co-op serves models from multiple providers (Tinfoil for privacy, GreenPT
for renewable energy, PublicAI coming soon). See the [model list](models.md) for
the current catalog — that's the single source of truth for which models are
available and how they're named.
One infra detail worth knowing: a local Tinfoil proxy sidecar handles the
enclave encryption on our side of the gateway — only Tinfoil traffic goes
through it; the other providers are called directly.
The co-op serves models from multiple providers — Tinfoil for architectural
privacy, GreenPT and Regolo for renewable energy. See the [model list](models.md),
the single source of truth for which models are available and how they're named.
## Security posture (plain language)
- **Token-gated control plane.** The portal is publicly reachable, but every
sensitive surface requires a secret token (admin, broker, webhook). All fail
closed — unauthenticated requests get `401`.
- **Separated secrets.** Chat, broker, admin, and model-layer secrets are
- **Member isolation.** Each API key is scoped to its owner's team and budget; a
member can only ever act on their own keys.
- **Separated secrets.** Chat, portal, admin and model-layer secrets are
independent and independently rotatable, so a leak has bounded scope.
- **Member isolation.** Each API key is scoped to its owner's LiteLLM team and
budget; a member can only ever act on their own keys.
- **Rate limiting** on control-plane endpoints; the inference path is capped by
each member's budget instead.
- **No secrets in version control.** Credentials live in Cloudron env vars, never
in the repos.
- **No secrets in version control.** Credentials live in the hosting platform's
environment, never in the repos.
- **Accepted trade-offs** (documented, not hidden): member keys are stored in
plaintext in the portal database (functionally necessary), and the gateway
must stay publicly reachable for API access (its admin UI and docs are
disabled; only the `/v1/*` API is exposed).
the portal database so the co-op can serve them, and the API gateway must stay
publicly reachable. Its admin surfaces are disabled; only the public API is
exposed.
## For operators
Maintaining the co-op's infrastructure — rotating Cloudron tokens, editing the
portal's environment, and troubleshooting the provisioning pipeline — is
documented in the [operator
runbook](https://git.inference.coop/co-op/ops-docs/src/branch/main/operator-token-runbook.md).
All operator-facing material (that runbook, the maintenance notes, and incident
write-ups) lives in the git-only
[`co-op/ops-docs`](https://git.inference.coop/co-op/ops-docs) repository, which
is deliberately not published on this site.
Maintaining the co-op's infrastructure — token rotation, app configuration and
troubleshooting — is documented in the git-only
[`co-op/ops-docs`](https://git.inference.coop/co-op/ops-docs) repository, which is
deliberately not published on this site. It is the home for the operator runbook,
the maintenance notes, incident write-ups, and the record of past server
migrations. Anyone may read it; operators are the audience.
## Incident log
Write-ups for outages live in the operator docs, in
[`incidents/`](https://git.inference.coop/co-op/ops-docs/src/branch/main/incidents).
One so far:
[2026-10-02 — gateway outage after a security-upgrade rebuild](https://git.inference.coop/co-op/ops-docs/src/branch/main/incidents/2026-10-02-gateway-cve-upgrade.md).
+2 -1
View File
@@ -13,7 +13,7 @@
# * fonts are self-hosted (publishing/fonts/) and OFL-licensed (Figtree).
# * no CDN / third-party host appears anywhere in the output.
site_name: Inference Cooperative
site_name: Inference Cooperative docs
site_description: >-
Member-readable documentation for the Inference Cooperative: how to join and
use the service, the model catalogue, API access, governance and policies.
@@ -69,6 +69,7 @@ theme:
nav:
- Home: README.md
- Getting started: getting-started.md
- Guest access: guest-access.md
- Models: models.md
- API access: api-access.md
- Energy & carbon: energy.md
+68 -8
View File
@@ -54,6 +54,10 @@ that **you choose** which axis matters more for a given task.
- **Cheap and cheerful** → `tinfoil/gpt-oss-120b` or `greenpt/glm-5.3-flash`.
- **Model origins matter to you** → the Apertus models (publicly developed;
see below).
- **Semantic search / RAG (API)** → an embedding model: `greenpt/qwen3-embedding-8b`
or `greenpt/green-embeddings` (renewable energy), or `tinfoil/nomic-embed-text`
(TEE-protected — the model the co-op's own docs search runs on). API-only; see
[Embedding models](#embedding-models) below.
## Current models
@@ -76,7 +80,7 @@ The emoji in the model name marks its **value tier** (the same icons used in the
| Model | Provider | Privacy | Green | Vision | Context | Pricing (in / out / cached) | Notes |
|---|---|---|---|---|---|---|---|
| `🌱 greenpt/deepseek-v4.1-flash` | GreenPT | policy | ✅ 100% renewable | ✅ | 1M | $0.22 / $1.10 / $0.011 | **Default.** DeepSeek's multimodal reasoning model, renewable energy; TEE twin below. |
| `🌱 greenpt/deepseek-v4.1-flash` | GreenPT | policy | ✅ 100% renewable | ✅ | 1M | $0.22 / $1.10 / $0.011 | **Default.** DeepSeek's multimodal reasoning model, renewable energy; TEE twin below. Base V4.1 Flash release — not GreenPT's 17 Oct 2026 "1017" snapshot (as of 2026-11-09). |
| `🔒 tinfoil/deepseek-v4-1-flash` | Tinfoil | architectural | undisclosed | ✅ | 1M | $0.65 / $1.45 / $0.13 | TEE-protected version of the default; image input, tool calling, long docs/agents. |
| `🔒 tinfoil/glm-5.3` | Tinfoil | architectural | undisclosed | — | 1M | $1.80 / $5.75 / $0.45 | GLM flagship, TEE-protected; strongest private-tier model for agentic coding and long-horizon tasks. |
| `🔒 tinfoil/gpt-oss-120b` | Tinfoil | architectural | undisclosed | — | 131K | $0.15 / $0.60 / — | Lightweight open-weight reasoning; cheapest private tier. |
@@ -114,13 +118,27 @@ model picker to avoid confusion.
| Model | Provider | Use | Pricing |
|---|---|---|---|
| `🔒 tinfoil/whisper-large-v3-turbo` | Tinfoil | Speech-to-text | $0.05 / 1M input tokens |
| `🔒 tinfoil/voxtral-tts` | Tinfoil | Text-to-speech | listed $0 (verify on invoice) |
| `🔒 tinfoil/whisper-large-v3-turbo` | Tinfoil | Speech-to-text | $0.01 per request (flat; zero per-token) |
| `🔒 tinfoil/voxtral-small-24b` | Tinfoil | Speech-to-text (token-priced) | $0.20 / 1M input + $0.60 / 1M output tokens |
| `🔒 tinfoil/voxtral-tts` | Tinfoil | Text-to-speech | $0.002 per request (zero per-token) |
Both run inside Tinfoil's enclaves, so voice data gets the same architectural
privacy as the private chat models. TTS voices available: `neutral_female`,
`neutral_male`, `casual_female/male`, `cheerful_female`, plus French, German,
Spanish, Italian, Portuguese, Dutch, and Hindi variants.
All three run inside Tinfoil's enclaves, so voice data gets the same
architectural privacy as the private chat models. TTS voices available:
`neutral_female`, `neutral_male`, `casual_female/male`, `cheerful_female`,
plus French, German, Spanish, Italian, Portuguese, Dutch, and Hindi variants.
Tinfoil bills these at cost, but its audio API returns **no token usage** to
the gateway — so dashboard rows for these models read `$0.01 · 0 tokens`: the
charge is real, the token columns stay empty.
`whisper-large-v3-turbo` and `voxtral-tts` charge **flat per request**.
`voxtral-small-24b` is token-priced and the better pick for small requests:
sent through the chat API with `input_audio` it meters exactly per token, and
on `/v1/audio/transcriptions` (the route dictation uses) the gateway bills
Tinfoil's measured chunk-equivalent — their audio encoder tokenizes in ~30 s
chunks, so a short request costs ≈**$0.0001**, about 1/125th of the whisper
floor. For audio longer than ~30 s, prefer the chat route, which meters
exactly.
```bash
# Speech-to-text
@@ -129,6 +147,15 @@ curl https://gateway.inference.coop/v1/audio/transcriptions \
-F file=@your-audio.mp3 \
-F model=tinfoil/whisper-large-v3-turbo
# Speech-to-text (token-priced model, chat route — meters exactly per token)
curl https://gateway.inference.coop/v1/chat/completions \
-H "Authorization: Bearer ***" \
-H "Content-Type: application/json" \
-d '{"model": "tinfoil/voxtral-small-24b", "messages": [{"role": "user", "content": [
{"type": "text", "text": "Transcribe this audio."},
{"type": "input_audio", "input_audio": {"data": "<base64 audio>", "format": "wav"}}
]}]}'
# Text-to-speech (returns WAV audio)
curl https://gateway.inference.coop/v1/audio/speech \
-H "Authorization: Bearer ***" \
@@ -146,10 +173,43 @@ curl https://gateway.inference.coop/v1/chat/completions \
]}]}'
```
### Embedding models
Embeddings turn text into a numeric vector — the input to semantic search,
clustering and RAG. They're called on `/v1/embeddings` with your co-op API key
and are **not chat models**: they never appear in the chat picker, the API is
the interface (the same shape as the audio models above). You choose between
them the same way as for chat: a TEE-protected option (Tinfoil) or
renewable-energy options (GreenPT).
| Model | Provider | Privacy | Green | Dimensions | Max input | Pricing |
|---|---|---|---|---|---|---|
| `🌱 greenpt/green-embeddings` | GreenPT | policy | ✅ 100% renewable | 2560 (measured) | 32K tokens (per GreenPT's catalog) | $0.20 / 1M input tokens (no output charge) |
| `🌱 greenpt/qwen3-embedding-8b` | GreenPT | policy | ✅ 100% renewable | 4096 (measured) | 32K tokens (per GreenPT's catalog) | $0.25 / 1M input tokens (no output charge) |
| `🔒 tinfoil/nomic-embed-text` | Tinfoil | **architectural (TEE)** | — (mix not disclosed) | 768 (measured) | 8K tokens (per Tinfoil's catalog) | $0.016 / 1M input tokens (no output charge) |
```bash
curl https://gateway.inference.coop/v1/embeddings \
-H "Authorization: Bearer ***" \
-H "Content-Type: application/json" \
-d '{"model": "tinfoil/nomic-embed-text", "input": "the text to embed"}'
```
The response is standard OpenAI shape: `data[0].embedding` holds the vector,
and `usage.prompt_tokens` is what meters against your allowance at the rate
above. Embeddings bill input tokens only — there is no output charge.
Vectors from different models are not interchangeable — pick one per search
index and stay with it (re-index if you switch). All three are callable with a
member key; for reference, the co-op's own documentation search runs on
`tinfoil/nomic-embed-text` inside the TEE.
### Image generation
`🌱 regolo/qwen-image` (Regolo, EU-hosted, zero data retention) generates
images from a text prompt. In the chat, press the **Integrations** button
images from a text prompt. It runs the December 2025 "2512" checkpoint of
Qwen-Image — not the newer Qwen Image 2.1 (September 2026), which Regolo does
not list (as of 2026-11-09). In the chat, press the **Integrations** button
below the message box, choose **Image**, and describe what you want — the
generated image appears in the conversation. Via the API:
+1 -27
View File
@@ -1,20 +1,11 @@
# Open Questions for Member Discussion
A living list of decisions the cooperative needs to make. Each question is framed for discussion on [Loomio](https://forum.inference.coop). As questions are resolved, they move to a "Decisions" log (to be created) rather than being deleted.
A living list of some of the decisions the cooperative needs to make.
---
## Privacy & Infrastructure
### Search: self-managed or Tinfoil's encrypted search?
We currently run our own [SearXNG](https://searxng.github.io/searxng/) instance (`search.inference.coop`) to power web search in the chat. Tinfoil also offers a `websearch` model — a private, encrypted search that runs inside their enclave.
- **Self-managed (SearXNG):** full control, no external dependency, but search queries are visible to our own infrastructure (and to the search engines SearXNG queries).
- **Tinfoil's encrypted search:** queries are encrypted end-to-end, but it's a third-party dependency and may cost per-query.
**Question:** Do we prioritize self-hosting (control, no external deps) or encrypted search (stronger privacy, but a dependency on Tinfoil)?
### Green energy vs. privacy — when we can't have both?
Our inference runs through Tinfoil (TEE-protected, verifiable privacy), but Tinfoil's energy mix is not disclosed. Self-hosting on renewable-powered hardware (e.g. Hetzner hydropower, Scaleway wind+hydro) would be greener but would lose the TEE privacy guarantee at our current scale.
@@ -24,12 +15,6 @@ Our inference runs through Tinfoil (TEE-protected, verifiable privacy), but Tinf
**Question:** When the two conflict, which value wins? Is there a threshold (e.g. member count, cost) at which we'd switch?
### Should we adopt Tinfoil's other models (embeddings, private search, audio)?
Tinfoil's catalog includes `nomic-embed-text` (embeddings), `websearch` (private search), and audio models (Whisper, TTS). We currently use Open WebUI's bundled local embedding model for file uploads.
**Question:** Do we standardize on Tinfoil for all model needs (consistency, privacy), or keep the local embedding model (no per-token cost, no dependency)?
### Which models should we offer?
We currently expose three chat models: DeepSeek V4 Flash (default), GPT-OSS 120B, and GLM-5.3 Flash. Tinfoil's full catalog also includes GLM-5.3 (full), Kimi K3, Llama 3.3 70B, Gemma 4 31B, and others.
@@ -57,22 +42,11 @@ Members currently pay dues on a sliding scale (from $5/month) with a flat $15/mo
**Question:** What pricing model best balances fairness, simplicity, and the cooperative's solidarity values? How do we handle a member who consistently exceeds their budget?
### What happens when a member's budget runs out mid-month?
Members have a **flexible balance**, not a hardcoded $15/month. The balance starts at $15 but can be topped up by payments (via Open Collective or another system) through the portal's `/admin/set-balance` endpoint, which pushes the new cap to the member's LiteLLM team budget (chat + API draw from the same balance). LiteLLM enforces the cap — requests stop when it's exhausted.
**Question:** Should members be able to top up self-serve? Should there be a grace period? Should the co-op pool surplus to cover light users? (The *mechanism* is built — these are now governance decisions, not technical blockers.)
---
## Governance & Membership
### What does "member-governed" mean in practice before 50 members?
The charter says the General Manager (Nathan) has full governance until 50 members, then transitions to member governance. But members can already deliberate on Loomio.
**Question:** Which decisions should be opened to member input *now* (even if non-binding), versus reserved for the General Manager during the pilot?
### How do we handle institutional members?
AI Potluck's "government-as-contributor" framing suggests a path for institutions (universities, municipalities) to contribute compute or data without owning the whole.
+4
View File
@@ -4,6 +4,10 @@
{{ config.copyright }}
</div>
{% endif %}
<div class="md-copyright__highlight">
Co-op homepage:
<a href="https://inference.coop/" rel="noopener">inference.coop</a>
</div>
<div class="md-copyright__highlight">
Documentation source:
<a href="https://git.inference.coop/co-op/docs" target="_blank" rel="noopener">co-op/docs on Gitea</a>
+37 -15
View File
@@ -90,14 +90,13 @@
--md-default-fg-color--lighter: #6f7a68;
--md-default-fg-color--lightest: rgba(148, 160, 140, 0.12);
/* The header was an unreadably dark green. Green-mid with near-black pine
* text keeps the bar clearly green, clearly distinct from the page, and
* legible (measured below). */
--md-primary-fg-color: var(--green-mid);
--md-primary-fg-color--light: var(--green-light);
--md-primary-fg-color--dark: var(--green-deep);
--md-primary-bg-color: #16201a;
--md-primary-bg-color--light: rgba(22, 32, 26, 0.7);
/* Header bar now matches the chat overlay's dark menu: #171717 with white
* text (operator request 2026-11-09) — neutral dark, not green. */
--md-primary-fg-color: #171717;
--md-primary-fg-color--light: #171717;
--md-primary-fg-color--dark: #171717;
--md-primary-bg-color: #ffffff;
--md-primary-bg-color--light: rgba(255, 255, 255, 0.7);
--md-accent-fg-color: var(--green-light);
--md-typeset-a-color: var(--green-light);
@@ -185,19 +184,42 @@
font-weight: 600;
}
/* Tabs and the search field pick up the brand green. */
/* The menu-bar (tab) strip. In light it is the same pale green WASH as the
* header — #e8f5e8, the homepage's "open source" pill — not a saturated fill:
* the deep greens are accents (links, active navigation, rules), and a large
* --green-deep bar clashed with the rest of the light palette. A warm hairline
* separates the wash from the cream page below it. Dark is unchanged. */
.md-tabs {
background-color: var(--green-deep);
background-color: #e8f5e8;
border-bottom: 1px solid var(--border);
}
/* In dark, keep the tab row the same green as the header bar so the text on it
* clears 4.5:1 against green-mid (green-deep against near-black text is only
* ~4.2:1 — marginal for the small tab labels). */
/* In dark, the tab row drops to the sheet's dark warm-green (dark-border
* #4a5240): quieter than green-mid, white labels still clear 7:1. */
[data-md-color-scheme="slate"] .md-tabs {
background-color: var(--green-mid);
background-color: var(--dark-border);
border-bottom: none;
}
.md-search__input {
/* Search input: the white field is a light-mode choice only. In dark mode the
* search surfaces use the pine family — input on the chips tone, light text. */
html:not([data-md-color-scheme="slate"]) .md-search__input {
background-color: rgba(255, 255, 255, 0.92);
}
[data-md-color-scheme="slate"] .md-search__form,
[data-md-color-scheme="slate"] .md-search__input {
background-color: var(--pine-3);
color: var(--dark-text);
}
[data-md-color-scheme="slate"] .md-search__form .md-icon,
[data-md-color-scheme="slate"] .md-search__input::placeholder {
color: var(--dark-muted);
}
[data-md-color-scheme="slate"] .md-search__scrollwrap,
[data-md-color-scheme="slate"] .md-search-result__article {
background-color: var(--pine-2);
}
[data-md-color-scheme="slate"] .md-search-result__meta {
color: var(--dark-muted);
}
/* Admonitions get the co-op's washes instead of stock greys. */
.md-typeset .admonition,