Compare commits
21
Commits
98cd517b8a
...
main
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
2422432b85 | ||
|
|
5402b9e758 | ||
|
|
96b1201f72 | ||
|
|
852e52dc35 | ||
|
|
ac75c504f6 | ||
|
|
c6d03c8f85 | ||
|
|
fb14577626 | ||
|
|
9119e6726a | ||
|
|
70c87e47c2 | ||
|
|
342c353ebc | ||
|
|
0786d97286 | ||
|
|
af4f96ad13 | ||
|
|
b263e125c4 | ||
|
|
fa4ba8d989 | ||
|
|
2a89e6a19d | ||
|
|
5eacc1c116 | ||
|
|
df722f81b0 | ||
|
|
278c1fda1a | ||
|
|
869d16cc65 | ||
|
|
07d5a60377 | ||
|
|
33b5cf73e8 |
No files matched your search
@@ -1,9 +1,9 @@
|
|||||||
# Inference Cooperative
|
# Inference Cooperative docs
|
||||||
|
|
||||||
**AI, governed together.**
|
**AI, governed together.**
|
||||||
|
|
||||||
The Inference Cooperative is a member-governed project providing private AI
|
The Inference Cooperative is a member-governed project providing more ethical AI
|
||||||
inference. Members pay dues on a sliding scale and, in return, get private AI
|
inference. Members pay dues on a sliding scale and, in return, get AI
|
||||||
chat and API access through shared, cooperatively governed infrastructure. We
|
chat and API access through shared, cooperatively governed infrastructure. We
|
||||||
are fiscally sponsored by [Metagov](https://metagov.org), a nonprofit, and
|
are fiscally sponsored by [Metagov](https://metagov.org), a nonprofit, and
|
||||||
funded through our [Open Collective](https://opencollective.com/inference-cooperative).
|
funded through our [Open Collective](https://opencollective.com/inference-cooperative).
|
||||||
@@ -26,6 +26,7 @@ funded through our [Open Collective](https://opencollective.com/inference-cooper
|
|||||||
- **Matrix chat:** [matrix.inference.coop](https://matrix.inference.coop)
|
- **Matrix chat:** [matrix.inference.coop](https://matrix.inference.coop)
|
||||||
- **Member Dashboard:** [dashboard.inference.coop](https://dashboard.inference.coop)
|
- **Member Dashboard:** [dashboard.inference.coop](https://dashboard.inference.coop)
|
||||||
- **Governance forum:** [forum.inference.coop](https://forum.inference.coop)
|
- **Governance forum:** [forum.inference.coop](https://forum.inference.coop)
|
||||||
|
- **Co-op calendar:** [cloud.inference.coop](https://cloud.inference.coop/apps/calendar/p/Ltb6LAbt7NWHH7Td)
|
||||||
- **Source code:** [git.inference.coop](https://git.inference.coop)
|
- **Source code:** [git.inference.coop](https://git.inference.coop)
|
||||||
- **Contact:** [info@inference.coop](mailto:info@inference.coop)
|
- **Contact:** [info@inference.coop](mailto:info@inference.coop)
|
||||||
|
|
||||||
|
|||||||
+5
-1
@@ -16,11 +16,15 @@ for our governance model, see the [Charter](charter.md).
|
|||||||
|
|
||||||
The cooperative launched in September 2026, growing out of research on democratic
|
The cooperative launched in September 2026, growing out of research on democratic
|
||||||
digital infrastructure through Metagov's [Collective Governance for AI:
|
digital infrastructure through Metagov's [Collective Governance for AI:
|
||||||
Points of Intervention](https://metagov.org/cg-ai/).
|
Points of Intervention](https://metagov.org/cg-ai/). It was founded by
|
||||||
|
[Nathan Schneider](https://nathanschneider.info), who formerly co-founded
|
||||||
|
Social.coop, Start.coop, and the Media Economies Design Lab at the University
|
||||||
|
of Colorado Boulder.
|
||||||
|
|
||||||
It began with a single privacy-first provider and a deliberately small
|
It began with a single privacy-first provider and a deliberately small
|
||||||
toolset: a chat, an API gateway, and the basics for membership and payments.
|
toolset: a chat, an API gateway, and the basics for membership and payments.
|
||||||
From there it grew the way co-ops do: in response to what members asked for.
|
From there it grew the way co-ops do: in response to what members asked for.
|
||||||
|
Green-energy providers were added, along with more diverse models.
|
||||||
A member dashboard made usage and spending transparent. A federated
|
A member dashboard made usage and spending transparent. A federated
|
||||||
[Matrix](https://matrix.inference.coop) server, a
|
[Matrix](https://matrix.inference.coop) server, a
|
||||||
[forum](https://forum.inference.coop), and a newsletter gave members places to
|
[forum](https://forum.inference.coop), and a newsletter gave members places to
|
||||||
|
|||||||
@@ -7,6 +7,10 @@ private AI chat and API access in return.
|
|||||||
This guide covers the three things most new members want: becoming a member,
|
This guide covers the three things most new members want: becoming a member,
|
||||||
using the chat, and getting involved in governance.
|
using the chat, and getting involved in governance.
|
||||||
|
|
||||||
|
The co-op's meetings — including the regular member meetings — are all on the
|
||||||
|
shared [co-op calendar](https://cloud.inference.coop/apps/calendar/p/Ltb6LAbt7NWHH7Td),
|
||||||
|
where we share dates and details. Check it for what's coming up.
|
||||||
|
|
||||||
## 1. Become a member
|
## 1. Become a member
|
||||||
|
|
||||||
Membership is a **sliding-scale contribution starting at $5/month** through
|
Membership is a **sliding-scale contribution starting at $5/month** through
|
||||||
@@ -26,12 +30,21 @@ account. Click it and create your login.
|
|||||||
> as a member. (Contributed without an Open Collective account? You're recognized
|
> as a member. (Contributed without an Open Collective account? You're recognized
|
||||||
> by the email you entered at checkout.)
|
> by the email you entered at checkout.)
|
||||||
|
|
||||||
|
**On payment timing:** Open Collective charges your contribution on the
|
||||||
|
collective's billing day, not the day you joined — so your second charge can
|
||||||
|
arrive sooner than a month after the first. When that leaves a paid month
|
||||||
|
shorter than a month, we credit the difference automatically as non-expiring
|
||||||
|
credits, pro-rated to the days lost. It's automatic: no need to ask, claim it,
|
||||||
|
or contact anyone.
|
||||||
|
|
||||||
### Your membership includes
|
### Your membership includes
|
||||||
|
|
||||||
- **Private AI chat** through the co-op's shared gateway.
|
- **Private AI chat** through the co-op's shared gateway.
|
||||||
- **API access** for your own tools and projects (see [API access](api-access.md)).
|
- **API access** for your own tools and projects (see [API access](api-access.md)).
|
||||||
- **A monthly usage allowance** of $15 of credit, shared between chat and API.
|
- **A monthly usage allowance** of $15 of credit, shared between chat and API.
|
||||||
- **Credit packs** — buy extra non-expiring credits when you need more.
|
- **Credit packs** — buy extra non-expiring credits when you need more.
|
||||||
|
- **Guest access** — give a friend $2 of credit and 60 days of the co-op.
|
||||||
|
See [Guest access](guest-access.md).
|
||||||
- **A voice in governance** — how the project is run and where it goes next.
|
- **A voice in governance** — how the project is run and where it goes next.
|
||||||
|
|
||||||
## 2. Use the chat
|
## 2. Use the chat
|
||||||
|
|||||||
@@ -0,0 +1,44 @@
|
|||||||
|
# Guest access
|
||||||
|
|
||||||
|
Any member can **give a friend guest access to the co-op**: **$2 of AI usage
|
||||||
|
for 60 days**, spent in the chat and the API. It doesn't come out of your own
|
||||||
|
balance. You can send it once every 30 days, and an address can receive it
|
||||||
|
only once.
|
||||||
|
|
||||||
|
## How to send guest access
|
||||||
|
|
||||||
|
On your [dashboard](https://dashboard.inference.coop), open the **Guests**
|
||||||
|
card, enter your friend's email address (name optional), and click
|
||||||
|
**Send invite**. There's no approval step: the account is provisioned right
|
||||||
|
away, and your friend gets an **invitation email** from the co-op, naming
|
||||||
|
you, with a link to set up their account. Setting it up takes about a minute.
|
||||||
|
|
||||||
|
New guests show as **Invited — awaiting setup** until they accept it. The
|
||||||
|
card also lists the guests you've sent.
|
||||||
|
|
||||||
|
## What your guest gets
|
||||||
|
|
||||||
|
- **$2 of credit** for the chat and the API — a hard cap: usage stops when
|
||||||
|
it's spent.
|
||||||
|
- **Their own dashboard** at
|
||||||
|
[dashboard.inference.coop](https://dashboard.inference.coop) showing what's
|
||||||
|
left.
|
||||||
|
- **The weekly newsletter** and these public docs.
|
||||||
|
|
||||||
|
Guest access is not membership — no Matrix, governance forum, or vote.
|
||||||
|
|
||||||
|
## When it ends
|
||||||
|
|
||||||
|
When **the $2 is spent or the 60 days are up**, whichever comes first, your
|
||||||
|
friend gets an email saying so. Access stops; **the account and its data are
|
||||||
|
kept** — and a guest can end it early with the **Cancel guest access** link
|
||||||
|
on their dashboard, returning any unspent credit to the co-op.
|
||||||
|
|
||||||
|
## Becoming a member
|
||||||
|
|
||||||
|
Contribute on
|
||||||
|
[Open Collective](https://opencollective.com/inference-cooperative/contribute)
|
||||||
|
and the account converts automatically — same login, same dashboard. Any
|
||||||
|
unspent guest credit carries over. See
|
||||||
|
[Become a member](getting-started.md#1-become-a-member) for how membership
|
||||||
|
works.
|
||||||
+112
-226
@@ -7,11 +7,11 @@ and contributors who want to work on it.
|
|||||||
|
|
||||||
Members reach the co-op through two front doors: the **chat** (LibreChat) and
|
Members reach the co-op through two front doors: the **chat** (LibreChat) and
|
||||||
the **API** (the LiteLLM gateway). Both route through a single gateway to
|
the **API** (the LiteLLM gateway). Both route through a single gateway to
|
||||||
multiple LLM providers — so members choose what they value for each task:
|
multiple LLM providers, so members can choose what they value for each task:
|
||||||
architecturally private inference in **TEE-protected enclaves** (Tinfoil),
|
architecturally private inference in **TEE-protected enclaves** (Tinfoil), or
|
||||||
100% renewable energy (GreenPT, Regolo). A thin layer of custom middleware —
|
100% renewable energy (GreenPT, Regolo). A thin layer of custom middleware — the
|
||||||
the member portal — ties Open
|
member portal — ties Open Collective membership to Cloudron accounts and LiteLLM
|
||||||
Collective membership to Cloudron accounts and LiteLLM keys.
|
keys.
|
||||||
|
|
||||||
```
|
```
|
||||||
┌─> Tinfoil (TEE) 🔒 private
|
┌─> Tinfoil (TEE) 🔒 private
|
||||||
@@ -28,228 +28,130 @@ single sign-on, and per-app isolation.
|
|||||||
|
|
||||||
## Where it runs
|
## Where it runs
|
||||||
|
|
||||||
The co-op's services run on a server the co-op owns and pays for directly, at
|
The co-op's member-facing services run on a server the co-op owns and pays for
|
||||||
**Infomaniak** — a Swiss sovereign cloud provider whose data centres are in
|
directly, at **Infomaniak** — a Swiss sovereign cloud provider whose data centres
|
||||||
Geneva, run on renewable energy, and certified to ISO 14001, ISO 50001 and
|
are in Geneva, run on renewable energy, and are certified to ISO 14001, ISO
|
||||||
ISO 27001. The stack moved there from a personal Hostinger account on
|
50001 and ISO 27001.
|
||||||
**2026-11-02**; nothing about how members use the co-op changed in the move.
|
|
||||||
|
|
||||||
One thing runs elsewhere: the co-op's **steward bot** — the automation behind the
|
The co-op's **automation runs on a separate server**, also hosted on Infomaniak
|
||||||
digests and the newsletter drafts — runs on a separate machine, not on the
|
and deliberately kept apart from the member-facing applications. The steward bot
|
||||||
Cloudron server, and was not part of the migration.
|
— the automation behind the digests and the newsletter drafts — and the co-op's
|
||||||
|
other agents live there. Keeping them on their own host means an agent problem
|
||||||
### DNS
|
stays an agent problem: it cannot reach the chat, the gateway or member data.
|
||||||
|
|
||||||
`inference.coop` is managed by hand at **domains.coop** (nameservers
|
|
||||||
`ns1/ns2.yoursrs.com`). The zone is deliberately small — **eleven records**. Most
|
|
||||||
of the co-op hangs off just a few of them: the apex (`inference.coop`), `www`,
|
|
||||||
and a wildcard (`*.inference.coop`) point at the server, so each application
|
|
||||||
hostname — chat, portal, dashboard, gateway, git, newsletter, vault and the rest
|
|
||||||
— is served by the wildcard rather than having a record of its own. The DNS
|
|
||||||
change sheet listing all eleven records is public, in
|
|
||||||
[`co-op/runbooks`](https://git.inference.coop/co-op/runbooks).
|
|
||||||
|
|
||||||
### Backups
|
### Backups
|
||||||
|
|
||||||
Cloudron writes **encrypted** backups to **Infomaniak's object storage** (the
|
Cloudron writes **encrypted** backups to Infomaniak's object storage in
|
||||||
"Infomaniak Encrypted S3" site, in Switzerland) every night, and keeps the last
|
Switzerland every night and keeps the last **seven days**, so a failed upgrade
|
||||||
**seven days**. The steward-bot host backs up its own selected home directories
|
or a lost server can be restored. The automation host backs up its own data to
|
||||||
to the same object storage.
|
the same object storage.
|
||||||
|
|
||||||
|
### Domain and DNS
|
||||||
|
|
||||||
|
The co-op's domain is **inference.coop** — a `.coop` domain. The co-op registers
|
||||||
|
and manages its domains with **domains.coop**, the domain provider run by the
|
||||||
|
cooperative movement, and holds them in the co-op's own name rather than in any
|
||||||
|
individual's. DNS itself is deliberately simple, and the co-op's own nameservers
|
||||||
|
answer it.
|
||||||
|
|
||||||
## Components
|
## Components
|
||||||
|
|
||||||
| Component | Purpose | URL | Code |
|
| Component | Purpose | URL |
|
||||||
|-----------|---------|-----|------|
|
|-----------|---------|-----|
|
||||||
| **LibreChat** | Member-facing chat (models, web search, image generation, voice) | chat.inference.coop | [co-op/librechat-custom](https://git.inference.coop/co-op/librechat-custom) |
|
| **LibreChat** | Member-facing chat (models, web search, image generation, voice) | chat.inference.coop |
|
||||||
| **Open WebUI** | Legacy chat instance — read-only reference (old history + file attachments) | chat-legacy.inference.coop | — |
|
| **Open WebUI** | Legacy chat instance — read-only reference (old history and file attachments) | chat-legacy.inference.coop |
|
||||||
| **LiteLLM** | AI gateway — keys, metering, model routing | gateway.inference.coop | [code/litellm](https://git.inference.coop/code/litellm) |
|
| **LiteLLM** | AI gateway — keys, metering and model routing, for chat and API | gateway.inference.coop |
|
||||||
| **Member Portal** | Membership middleware — onboarding, key provisioning, Loomio sync | portal.inference.coop | [code/member-portal](https://git.inference.coop/code/member-portal) |
|
| **Member portal** | Membership middleware — onboarding, key provisioning, Loomio sync | portal.inference.coop |
|
||||||
| **Member Dashboard** | Member-facing usage view + API key management | dashboard.inference.coop | [code/member-dashboard](https://git.inference.coop/code/member-dashboard) |
|
| **Member dashboard** | Your usage view and API keys | dashboard.inference.coop |
|
||||||
| **Admin Panel** | Admin-only member overview, spend, balance management | panel.inference.coop | [code/admin-panel](https://git.inference.coop/code/admin-panel) |
|
| **Loomio** | Member governance | forum.inference.coop |
|
||||||
| **SearXNG** | Self-hosted web search (feeds the chat's search) | search.inference.coop | — |
|
| **Matrix (Synapse)** | Member chat server, federated, SSO login via Cloudron | matrix.inference.coop |
|
||||||
| **Loomio** | Member governance | forum.inference.coop | — |
|
| **Gitea** | Git hosting for the co-op's code and this documentation | git.inference.coop |
|
||||||
| **Matrix (Synapse)** | Member chat server (federated; SSO login via Cloudron) | matrix.inference.coop | — |
|
| **Listmonk** | Member newsletter (synced from membership) | newsletter.inference.coop |
|
||||||
| **Gitea** | Git hosting (code + docs) | git.inference.coop | — |
|
| **SnappyMail** | Webmail | email.inference.coop |
|
||||||
| **Surfer** | Static hosting (landing page, thanks page) | inference.coop | [co-op/website](https://git.inference.coop/co-op/website) |
|
|
||||||
| **Listmonk** | Member newsletter (auto-synced from membership) | newsletter.inference.coop | — |
|
The co-op's software behind all of these is open source — see [The
|
||||||
| **Vaultwarden** | Password manager (admin credentials) | vault.inference.coop | — |
|
code](#the-code) below for where each piece lives.
|
||||||
| **SnappyMail** | Webmail | email.inference.coop | — |
|
|
||||||
|
|
||||||
Open Collective (billing and fiscal sponsorship) is the one external service in
|
Open Collective (billing and fiscal sponsorship) is the one external service in
|
||||||
the flow — see [opencollective.com/inference-cooperative](https://opencollective.com/inference-cooperative).
|
the flow — see
|
||||||
|
[opencollective.com/inference-cooperative](https://opencollective.com/inference-cooperative).
|
||||||
|
|
||||||
## Mail
|
## Mail
|
||||||
|
|
||||||
The `inference.coop` mail domain is the co-op's, and the mail itself runs on the
|
The co-op runs its own `@inference.coop` mail. **Incoming** mail arrives directly
|
||||||
co-op's Cloudron server.
|
on the co-op's server, and webmail and mail apps reach it over the usual
|
||||||
|
encrypted ports (IMAP and SMTP over TLS). **Outgoing** mail is sent through
|
||||||
|
**Scaleway**, an EU-based email service, authenticated for our domain.
|
||||||
|
|
||||||
**Incoming** mail is received directly by that server — the domain's MX record
|
You can reach your mailbox through webmail at email.inference.coop, or add it to
|
||||||
points at it, it accepts mail on the standard SMTP port, and webmail and mail
|
a standard mail app on your phone or computer. If mail to your `@inference.coop`
|
||||||
apps reach it over the usual encrypted ports (IMAP and SMTP over TLS). If mail
|
address stops arriving, that is a fault on our side to fix.
|
||||||
to your `@inference.coop` address stops arriving, that is a fault on our side to
|
|
||||||
fix, not a limitation of how the co-op is set up.
|
|
||||||
|
|
||||||
**Outgoing** mail travels by two different paths, on purpose:
|
|
||||||
|
|
||||||
- The **newsletter** (Listmonk) sends through **Scaleway Transactional Email**,
|
|
||||||
an EU-based SMTP relay, rather than from our own server. Our domain is
|
|
||||||
authenticated to that relay with its own DKIM key, and a real message has been
|
|
||||||
checked end to end: SPF, DKIM and DMARC all pass under a `p=reject` policy —
|
|
||||||
so anything claiming to be from us that the relay hasn't authenticated is
|
|
||||||
rejected outright rather than quietly filed away.
|
|
||||||
- The **member portal's** transactional email — welcome messages, budget
|
|
||||||
notices, the confirmation the nightly membership sweep sends — still sends from
|
|
||||||
the co-op's own server, through the mail service Cloudron runs for us.
|
|
||||||
|
|
||||||
Both paths send as **Inference Cooperative `<info@inference.coop>`**.
|
|
||||||
|
|
||||||
Why split them: sending the newsletter through a relay lets a provider with
|
|
||||||
established sending IPs carry a list this size, instead of leaning on one
|
|
||||||
freshly-migrated server's reputation. Receiving we keep in-house, on our own
|
|
||||||
hardware.
|
|
||||||
|
|
||||||
## Matrix
|
## Matrix
|
||||||
|
|
||||||
Members get a federated Matrix account on our self-hosted Synapse
|
Members get a federated Matrix account on the co-op's self-hosted Synapse
|
||||||
(matrix.inference.coop, user IDs `@username:inference.coop`) using their
|
(matrix.inference.coop, user IDs `@username:inference.coop`), signing in with the
|
||||||
Cloudron SSO login. The portal invites every active member — and Nathan's
|
same Cloudron SSO as the chat. The co-op invites every active member to the
|
||||||
federated account — to the private **Inference Cooperative** Space (with a
|
private **Inference Cooperative** space (with a **General** room) automatically,
|
||||||
**General** room) on every membership sync, so new members land in the space
|
so new members land there as they join. Federation is on, so you can also join
|
||||||
automatically. Invites honor rate limits; external/federated members can be
|
from any homeserver you already use.
|
||||||
added via the portal's `MATRIX_EXTRA_MXIDS` env.
|
|
||||||
|
|
||||||
Federation is enabled (delegated via `.well-known/matrix/server` on the bare
|
### Accounts you may meet
|
||||||
domain, port 443), so members can also join from any homeserver they already
|
|
||||||
use.
|
|
||||||
|
|
||||||
### Which account is which
|
A few accounts in the space are automation rather than people:
|
||||||
|
|
||||||
Several automated accounts touch this homeserver, and two of them once differed
|
- **`@inferencebot` — the steward.** It posts the digests, newsletter notes and
|
||||||
by a single character — which caused real confusion, including a bot appearing to
|
reminders. It holds no more power than posting needs.
|
||||||
announce itself. The distinction is deliberate:
|
- **`@inference-admin`** — the administrator of record for the co-op's rooms
|
||||||
|
(#general, #community, #design, #skillshare, #updates and #infrastructure) and
|
||||||
|
their membership. It **never speaks in a member room** — a message from it is
|
||||||
|
a mistake, not a feature.
|
||||||
|
|
||||||
- **`@inferencebot:inference.coop` — the steward.** The bot members actually meet:
|
The house rule: **power belongs to `@inference-admin`, voice belongs to
|
||||||
it posts digests, newsletter notes and reminders. It is *not* a Synapse server
|
`@inferencebot`.** Please don't create an account whose name differs from an
|
||||||
admin, and holds no room power beyond what it needs to post.
|
existing one by a character or two — spotting lookalikes is exactly what this
|
||||||
- **`@inference-admin:inference.coop` — the room and server administrator.** A
|
convention is for.
|
||||||
Synapse server admin, used for provisioning rooms, managing membership, and
|
|
||||||
retiring rooms. It **never speaks in a member room**; a message from it is a
|
|
||||||
mistake, not a feature.
|
|
||||||
- **`@infrastructure-bot:inference.coop` — the infrastructure working group's bot.**
|
|
||||||
Answers infrastructure questions in **#infrastructure** when mentioned.
|
|
||||||
- **`@ops_audit:inference.coop`** — a server-admin account created for the October
|
|
||||||
2026 security review. Retained only until its tokens are revoked.
|
|
||||||
|
|
||||||
**House rule: power belongs to `@inference-admin`, voice belongs to `@inferencebot`.**
|
## Membership
|
||||||
Please don't create another account whose name differs from an existing one by a
|
|
||||||
character or two; that is the specific confusion this section exists to undo.
|
|
||||||
|
|
||||||
**`@inference-bot:inference.coop` is retired (2026-10-29).** It was the original
|
Membership runs through the member portal, the single source of truth for who is
|
||||||
room-administration account (it created the space and most of the rooms) and held a
|
a member:
|
||||||
Synapse server-admin flag. Its room power went to `@inference-admin`, which is now
|
|
||||||
administrator of record for #general, #community, #design, #skillshare, #updates and
|
|
||||||
#infrastructure, **and holds the portal's member-invite credential**
|
|
||||||
(`MATRIX_BOT_TOKEN`) under its own token. The old account's admin flag was dropped,
|
|
||||||
it was deactivated, and every device and token was deleted. Nothing should use it
|
|
||||||
again.
|
|
||||||
|
|
||||||
Two loose ends, recorded so nobody re-investigates them from scratch: it remains
|
|
||||||
joined to two rooms it cannot leave (they reject every write — `500` on leave), and
|
|
||||||
those rooms are also why they can't be deleted; and the old account's *name* is still
|
|
||||||
the tempting one, so check `whoami` rather than the name before trusting any account
|
|
||||||
that looks like it.
|
|
||||||
|
|
||||||
## How the apps run themselves
|
|
||||||
|
|
||||||
The portal and gateway are self-maintaining: their recurring housekeeping is
|
|
||||||
built into the apps as scheduled tasks, not run from any member's or operator's
|
|
||||||
personal machine.
|
|
||||||
|
|
||||||
- **Member portal** — two scheduled jobs (Cloudron scheduler):
|
|
||||||
- *Membership sweep* (nightly, 3am) — reconciles members against the live
|
|
||||||
Open Collective list: provisions new members who signed up while a webhook
|
|
||||||
was missed, deactivates lapsed members, and syncs Loomio + Matrix
|
|
||||||
membership. A missed webhook self-heals within 24 hours.
|
|
||||||
- *Docs knowledge refresh* (nightly, 3:30am) — pulls this documentation from
|
|
||||||
the public Git repo and updates the shared knowledge base behind the chat's
|
|
||||||
models, so answers grounded in the docs stay current. Changes to the docs
|
|
||||||
appear in the chat within a day, no manual step.
|
|
||||||
- **Gateway** — an *audio keep-alive* (every 10 minutes) sends a tiny text-to-
|
|
||||||
speech ping to the private enclave so voice mode responds quickly even after
|
|
||||||
long quiet periods, without compromising the end-to-end encryption model.
|
|
||||||
|
|
||||||
Scheduled tasks are declared in each app's `CloudronManifest.json` (see the
|
|
||||||
`code/` repos) and survive redeploys. If a task fails, it shows up in the
|
|
||||||
app's logs on the admin panel; the portal also exposes a manual trigger for
|
|
||||||
the membership sweep.
|
|
||||||
|
|
||||||
## How membership is wired
|
|
||||||
|
|
||||||
Membership flows through the member portal, which is the single source of truth
|
|
||||||
for "who is a member":
|
|
||||||
|
|
||||||
1. A member contributes on Open Collective.
|
1. A member contributes on Open Collective.
|
||||||
2. Open Collective fires a webhook to the portal, which provisions a Cloudron
|
2. Open Collective notifies the portal, which provisions a Cloudron account, a
|
||||||
account, a LiteLLM key, and a welcome email.
|
LiteLLM key, and a welcome email.
|
||||||
3. A nightly sweep reconciles against the live Open Collective list, so a missed
|
3. A nightly reconciliation against the live Open Collective list means a missed
|
||||||
webhook self-heals within 24 hours.
|
notification self-heals within a day.
|
||||||
|
|
||||||
The portal database is authoritative for member emails and balances; LiteLLM's
|
You may see **"Inference Co-op Bot"** listed as an admin of the collective on
|
||||||
team budget is the source of truth for a member's allowance, shared between chat
|
Open Collective. It is infrastructure, not a person or a member: a dedicated
|
||||||
and API.
|
account the portal uses, **read-only**, to check the member roster and tiers. It
|
||||||
|
cannot move money, issue refunds or change the collective — payments always come
|
||||||
### The bot account on Open Collective
|
from a human's own contribution, and automation only reads the result. Its code
|
||||||
|
is open source in
|
||||||
You may notice **"Inference Co-op Bot"** listed as an admin of the collective on
|
[code/member-portal](https://git.inference.coop/code/member-portal), and a
|
||||||
Open Collective. This is infrastructure, not a member or a person:
|
collective admin can revoke its access at any time.
|
||||||
|
|
||||||
- **What it is:** a dedicated Open Collective account owned by the co-op. It
|
|
||||||
holds a **Personal Token** that the member portal uses to read membership
|
|
||||||
data — the roster, tiers (needed to route credit-pack purchases correctly),
|
|
||||||
and contributor emails (including guests, which only admins can see).
|
|
||||||
- **What it can do:** read member and tier data via Open Collective's API.
|
|
||||||
It cannot move money, issue refunds, or change the collective — Open
|
|
||||||
Collective restricts those to dashboard actions by human admins. Its API
|
|
||||||
access was verified read-mostly during the October 2026 security review.
|
|
||||||
- **Why it has admin:** reading guest contributor emails requires admin on
|
|
||||||
Open Collective; without it, membership reconciliation and credit-pack
|
|
||||||
routing can't function.
|
|
||||||
- **Who supervises it:** all of its code is open source in
|
|
||||||
[code/member-portal](https://git.inference.coop/code/member-portal) — every
|
|
||||||
API call it makes is documented there. Nathan Schneider (collective admin)
|
|
||||||
oversees its use and can revoke or rotate its token at any time from the
|
|
||||||
Open Collective dashboard.
|
|
||||||
|
|
||||||
Nothing on Open Collective is charged, transferred, or refunded by automation:
|
|
||||||
payments always come from a human's own contribution action, and any automated
|
|
||||||
system only *reads* the resulting records.
|
|
||||||
|
|
||||||
## The code
|
## The code
|
||||||
|
|
||||||
The cooperative's software is open source, in the [`code`](https://git.inference.coop/code)
|
The co-op's software is open source, in the [`code`](https://git.inference.coop/code)
|
||||||
organization on Gitea:
|
and [`co-op`](https://git.inference.coop/co-op) organizations on Gitea:
|
||||||
|
|
||||||
- [**code/member-portal**](https://git.inference.coop/code/member-portal) — membership middleware (Python).
|
- [**code/member-portal**](https://git.inference.coop/code/member-portal) — membership middleware (Python).
|
||||||
- [**code/member-dashboard**](https://git.inference.coop/code/member-dashboard) — member-facing usage + API keys (Python).
|
- [**code/member-dashboard**](https://git.inference.coop/code/member-dashboard) — member dashboard: usage and API keys.
|
||||||
- [**code/admin-panel**](https://git.inference.coop/code/admin-panel) — admin overview (Python).
|
|
||||||
- [**code/litellm**](https://git.inference.coop/code/litellm) — the gateway packaged as a Cloudron app.
|
- [**code/litellm**](https://git.inference.coop/code/litellm) — the gateway packaged as a Cloudron app.
|
||||||
|
|
||||||
Plus the `co-op` organization:
|
|
||||||
|
|
||||||
- [**co-op/website**](https://git.inference.coop/co-op/website) — landing page and static site.
|
- [**co-op/website**](https://git.inference.coop/co-op/website) — landing page and static site.
|
||||||
- [**co-op/docs**](https://git.inference.coop/co-op/docs) — this documentation.
|
- [**co-op/docs**](https://git.inference.coop/co-op/docs) — this documentation.
|
||||||
- [**co-op/design-assets**](https://git.inference.coop/co-op/design-assets) — logos, fonts, brand materials.
|
|
||||||
|
|
||||||
## Contributing (the Git is yours, too)
|
The chat, portal and other apps live in `code/` and `co-op/` too; everything is
|
||||||
|
browsable at [git.inference.coop](https://git.inference.coop). Everything the co-op runs
|
||||||
Everything the co-op runs is open source and lives at
|
is open source, so as a member you can browse all of
|
||||||
[git.inference.coop](https://git.inference.coop) — the website, this
|
it and propose changes.
|
||||||
documentation, the member portal, the dashboard, the admin panel, and the
|
|
||||||
gateway. As a member, you can browse all of it and propose changes.
|
|
||||||
|
|
||||||
**Who can sign in.** Git access is tied to your membership: you log into the Git
|
**Who can sign in.** Git access is tied to your membership: you log into the Git
|
||||||
with the same single sign-on you use for the chat. Members are in automatically
|
with the same single sign-on you use for the chat. Members are in automatically
|
||||||
(through the `members` group); when a membership lapses, Git access lapses with it.
|
(through the `members` group); when a membership lapses, Git access lapses with
|
||||||
|
it.
|
||||||
|
|
||||||
**How to contribute.** Anyone can read the repos without an account. To make a
|
**How to contribute.** Anyone can read the repos without an account. To make a
|
||||||
change, fork the repo, edit in your fork, and open a **pull request**:
|
change, fork the repo, edit in your fork, and open a **pull request**:
|
||||||
@@ -260,55 +162,39 @@ change, fork the repo, edit in your fork, and open a **pull request**:
|
|||||||
4. Open a **pull request** back to the original repo, describing your change.
|
4. Open a **pull request** back to the original repo, describing your change.
|
||||||
|
|
||||||
**All changes are reviewed.** Nothing merges directly into `main` — every change
|
**All changes are reviewed.** Nothing merges directly into `main` — every change
|
||||||
comes in as a pull request that an admin reviews and approves first. You don't
|
comes in as a pull request an admin reviews and approves first. You don't need
|
||||||
need to be able to create repos or organizations to contribute; you just fork,
|
to be able to create repos or organizations to contribute; you just fork, edit
|
||||||
edit, and ask for review.
|
and ask for review.
|
||||||
|
|
||||||
Questions about the Git itself? Write to info@inference.coop.
|
|
||||||
|
|
||||||
## Models and providers
|
## Models and providers
|
||||||
|
|
||||||
The co-op serves models from multiple providers (Tinfoil for privacy, GreenPT
|
The co-op serves models from multiple providers — Tinfoil for architectural
|
||||||
for renewable energy, PublicAI coming soon). See the [model list](models.md) for
|
privacy, GreenPT and Regolo for renewable energy. See the [model list](models.md),
|
||||||
the current catalog — that's the single source of truth for which models are
|
the single source of truth for which models are available and how they're named.
|
||||||
available and how they're named.
|
|
||||||
|
|
||||||
One infra detail worth knowing: a local Tinfoil proxy sidecar handles the
|
|
||||||
enclave encryption on our side of the gateway — only Tinfoil traffic goes
|
|
||||||
through it; the other providers are called directly.
|
|
||||||
|
|
||||||
## Security posture (plain language)
|
## Security posture (plain language)
|
||||||
|
|
||||||
- **Token-gated control plane.** The portal is publicly reachable, but every
|
- **Member isolation.** Each API key is scoped to its owner's team and budget; a
|
||||||
sensitive surface requires a secret token (admin, broker, webhook). All fail
|
member can only ever act on their own keys.
|
||||||
closed — unauthenticated requests get `401`.
|
- **Separated secrets.** Chat, portal, admin and model-layer secrets are
|
||||||
- **Separated secrets.** Chat, broker, admin, and model-layer secrets are
|
|
||||||
independent and independently rotatable, so a leak has bounded scope.
|
independent and independently rotatable, so a leak has bounded scope.
|
||||||
- **Member isolation.** Each API key is scoped to its owner's LiteLLM team and
|
- **No secrets in version control.** Credentials live in the hosting platform's
|
||||||
budget; a member can only ever act on their own keys.
|
environment, never in the repos.
|
||||||
- **Rate limiting** on control-plane endpoints; the inference path is capped by
|
|
||||||
each member's budget instead.
|
|
||||||
- **No secrets in version control.** Credentials live in Cloudron env vars, never
|
|
||||||
in the repos.
|
|
||||||
- **Accepted trade-offs** (documented, not hidden): member keys are stored in
|
- **Accepted trade-offs** (documented, not hidden): member keys are stored in
|
||||||
plaintext in the portal database (functionally necessary), and the gateway
|
the portal database so the co-op can serve them, and the API gateway must stay
|
||||||
must stay publicly reachable for API access (its admin UI and docs are
|
publicly reachable. Its admin surfaces are disabled; only the public API is
|
||||||
disabled; only the `/v1/*` API is exposed).
|
exposed.
|
||||||
|
|
||||||
## For operators
|
## For operators
|
||||||
|
|
||||||
Maintaining the co-op's infrastructure — rotating Cloudron tokens, editing the
|
Maintaining the co-op's infrastructure — token rotation, app configuration and
|
||||||
portal's environment, and troubleshooting the provisioning pipeline — is
|
troubleshooting — is documented in the git-only
|
||||||
documented in the [operator
|
[`co-op/ops-docs`](https://git.inference.coop/co-op/ops-docs) repository, which is
|
||||||
runbook](https://git.inference.coop/co-op/ops-docs/src/branch/main/operator-token-runbook.md).
|
deliberately not published on this site. It is the home for the operator runbook,
|
||||||
All operator-facing material (that runbook, the maintenance notes, and incident
|
the maintenance notes, incident write-ups, and the record of past server
|
||||||
write-ups) lives in the git-only
|
migrations. Anyone may read it; operators are the audience.
|
||||||
[`co-op/ops-docs`](https://git.inference.coop/co-op/ops-docs) repository, which
|
|
||||||
is deliberately not published on this site.
|
|
||||||
|
|
||||||
## Incident log
|
## Incident log
|
||||||
|
|
||||||
Write-ups for outages live in the operator docs, in
|
Write-ups for outages live in the operator docs, in
|
||||||
[`incidents/`](https://git.inference.coop/co-op/ops-docs/src/branch/main/incidents).
|
[`incidents/`](https://git.inference.coop/co-op/ops-docs/src/branch/main/incidents).
|
||||||
One so far:
|
|
||||||
[2026-10-02 — gateway outage after a security-upgrade rebuild](https://git.inference.coop/co-op/ops-docs/src/branch/main/incidents/2026-10-02-gateway-cve-upgrade.md).
|
|
||||||
+2
-1
@@ -13,7 +13,7 @@
|
|||||||
# * fonts are self-hosted (publishing/fonts/) and OFL-licensed (Figtree).
|
# * fonts are self-hosted (publishing/fonts/) and OFL-licensed (Figtree).
|
||||||
# * no CDN / third-party host appears anywhere in the output.
|
# * no CDN / third-party host appears anywhere in the output.
|
||||||
|
|
||||||
site_name: Inference Cooperative
|
site_name: Inference Cooperative docs
|
||||||
site_description: >-
|
site_description: >-
|
||||||
Member-readable documentation for the Inference Cooperative: how to join and
|
Member-readable documentation for the Inference Cooperative: how to join and
|
||||||
use the service, the model catalogue, API access, governance and policies.
|
use the service, the model catalogue, API access, governance and policies.
|
||||||
@@ -69,6 +69,7 @@ theme:
|
|||||||
nav:
|
nav:
|
||||||
- Home: README.md
|
- Home: README.md
|
||||||
- Getting started: getting-started.md
|
- Getting started: getting-started.md
|
||||||
|
- Guest access: guest-access.md
|
||||||
- Models: models.md
|
- Models: models.md
|
||||||
- API access: api-access.md
|
- API access: api-access.md
|
||||||
- Energy & carbon: energy.md
|
- Energy & carbon: energy.md
|
||||||
|
|||||||
@@ -54,6 +54,10 @@ that **you choose** which axis matters more for a given task.
|
|||||||
- **Cheap and cheerful** → `tinfoil/gpt-oss-120b` or `greenpt/glm-5.3-flash`.
|
- **Cheap and cheerful** → `tinfoil/gpt-oss-120b` or `greenpt/glm-5.3-flash`.
|
||||||
- **Model origins matter to you** → the Apertus models (publicly developed;
|
- **Model origins matter to you** → the Apertus models (publicly developed;
|
||||||
see below).
|
see below).
|
||||||
|
- **Semantic search / RAG (API)** → an embedding model: `greenpt/qwen3-embedding-8b`
|
||||||
|
or `greenpt/green-embeddings` (renewable energy), or `tinfoil/nomic-embed-text`
|
||||||
|
(TEE-protected — the model the co-op's own docs search runs on). API-only; see
|
||||||
|
[Embedding models](#embedding-models) below.
|
||||||
|
|
||||||
## Current models
|
## Current models
|
||||||
|
|
||||||
@@ -76,7 +80,7 @@ The emoji in the model name marks its **value tier** (the same icons used in the
|
|||||||
|
|
||||||
| Model | Provider | Privacy | Green | Vision | Context | Pricing (in / out / cached) | Notes |
|
| Model | Provider | Privacy | Green | Vision | Context | Pricing (in / out / cached) | Notes |
|
||||||
|---|---|---|---|---|---|---|---|
|
|---|---|---|---|---|---|---|---|
|
||||||
| `🌱 greenpt/deepseek-v4.1-flash` | GreenPT | policy | ✅ 100% renewable | ✅ | 1M | $0.22 / $1.10 / $0.011 | **Default.** DeepSeek's multimodal reasoning model, renewable energy; TEE twin below. |
|
| `🌱 greenpt/deepseek-v4.1-flash` | GreenPT | policy | ✅ 100% renewable | ✅ | 1M | $0.22 / $1.10 / $0.011 | **Default.** DeepSeek's multimodal reasoning model, renewable energy; TEE twin below. Base V4.1 Flash release — not GreenPT's 17 Oct 2026 "1017" snapshot (as of 2026-11-09). |
|
||||||
| `🔒 tinfoil/deepseek-v4-1-flash` | Tinfoil | architectural | undisclosed | ✅ | 1M | $0.65 / $1.45 / $0.13 | TEE-protected version of the default; image input, tool calling, long docs/agents. |
|
| `🔒 tinfoil/deepseek-v4-1-flash` | Tinfoil | architectural | undisclosed | ✅ | 1M | $0.65 / $1.45 / $0.13 | TEE-protected version of the default; image input, tool calling, long docs/agents. |
|
||||||
| `🔒 tinfoil/glm-5.3` | Tinfoil | architectural | undisclosed | — | 1M | $1.80 / $5.75 / $0.45 | GLM flagship, TEE-protected; strongest private-tier model for agentic coding and long-horizon tasks. |
|
| `🔒 tinfoil/glm-5.3` | Tinfoil | architectural | undisclosed | — | 1M | $1.80 / $5.75 / $0.45 | GLM flagship, TEE-protected; strongest private-tier model for agentic coding and long-horizon tasks. |
|
||||||
| `🔒 tinfoil/gpt-oss-120b` | Tinfoil | architectural | undisclosed | — | 131K | $0.15 / $0.60 / — | Lightweight open-weight reasoning; cheapest private tier. |
|
| `🔒 tinfoil/gpt-oss-120b` | Tinfoil | architectural | undisclosed | — | 131K | $0.15 / $0.60 / — | Lightweight open-weight reasoning; cheapest private tier. |
|
||||||
@@ -114,13 +118,27 @@ model picker to avoid confusion.
|
|||||||
|
|
||||||
| Model | Provider | Use | Pricing |
|
| Model | Provider | Use | Pricing |
|
||||||
|---|---|---|---|
|
|---|---|---|---|
|
||||||
| `🔒 tinfoil/whisper-large-v3-turbo` | Tinfoil | Speech-to-text | $0.05 / 1M input tokens |
|
| `🔒 tinfoil/whisper-large-v3-turbo` | Tinfoil | Speech-to-text | $0.01 per request (flat; zero per-token) |
|
||||||
| `🔒 tinfoil/voxtral-tts` | Tinfoil | Text-to-speech | listed $0 (verify on invoice) |
|
| `🔒 tinfoil/voxtral-small-24b` | Tinfoil | Speech-to-text (token-priced) | $0.20 / 1M input + $0.60 / 1M output tokens |
|
||||||
|
| `🔒 tinfoil/voxtral-tts` | Tinfoil | Text-to-speech | $0.002 per request (zero per-token) |
|
||||||
|
|
||||||
Both run inside Tinfoil's enclaves, so voice data gets the same architectural
|
All three run inside Tinfoil's enclaves, so voice data gets the same
|
||||||
privacy as the private chat models. TTS voices available: `neutral_female`,
|
architectural privacy as the private chat models. TTS voices available:
|
||||||
`neutral_male`, `casual_female/male`, `cheerful_female`, plus French, German,
|
`neutral_female`, `neutral_male`, `casual_female/male`, `cheerful_female`,
|
||||||
Spanish, Italian, Portuguese, Dutch, and Hindi variants.
|
plus French, German, Spanish, Italian, Portuguese, Dutch, and Hindi variants.
|
||||||
|
|
||||||
|
Tinfoil bills these at cost, but its audio API returns **no token usage** to
|
||||||
|
the gateway — so dashboard rows for these models read `$0.01 · 0 tokens`: the
|
||||||
|
charge is real, the token columns stay empty.
|
||||||
|
|
||||||
|
`whisper-large-v3-turbo` and `voxtral-tts` charge **flat per request**.
|
||||||
|
`voxtral-small-24b` is token-priced and the better pick for small requests:
|
||||||
|
sent through the chat API with `input_audio` it meters exactly per token, and
|
||||||
|
on `/v1/audio/transcriptions` (the route dictation uses) the gateway bills
|
||||||
|
Tinfoil's measured chunk-equivalent — their audio encoder tokenizes in ~30 s
|
||||||
|
chunks, so a short request costs ≈**$0.0001**, about 1/125th of the whisper
|
||||||
|
floor. For audio longer than ~30 s, prefer the chat route, which meters
|
||||||
|
exactly.
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
# Speech-to-text
|
# Speech-to-text
|
||||||
@@ -129,6 +147,15 @@ curl https://gateway.inference.coop/v1/audio/transcriptions \
|
|||||||
-F file=@your-audio.mp3 \
|
-F file=@your-audio.mp3 \
|
||||||
-F model=tinfoil/whisper-large-v3-turbo
|
-F model=tinfoil/whisper-large-v3-turbo
|
||||||
|
|
||||||
|
# Speech-to-text (token-priced model, chat route — meters exactly per token)
|
||||||
|
curl https://gateway.inference.coop/v1/chat/completions \
|
||||||
|
-H "Authorization: Bearer ***" \
|
||||||
|
-H "Content-Type: application/json" \
|
||||||
|
-d '{"model": "tinfoil/voxtral-small-24b", "messages": [{"role": "user", "content": [
|
||||||
|
{"type": "text", "text": "Transcribe this audio."},
|
||||||
|
{"type": "input_audio", "input_audio": {"data": "<base64 audio>", "format": "wav"}}
|
||||||
|
]}]}'
|
||||||
|
|
||||||
# Text-to-speech (returns WAV audio)
|
# Text-to-speech (returns WAV audio)
|
||||||
curl https://gateway.inference.coop/v1/audio/speech \
|
curl https://gateway.inference.coop/v1/audio/speech \
|
||||||
-H "Authorization: Bearer ***" \
|
-H "Authorization: Bearer ***" \
|
||||||
@@ -146,10 +173,43 @@ curl https://gateway.inference.coop/v1/chat/completions \
|
|||||||
]}]}'
|
]}]}'
|
||||||
```
|
```
|
||||||
|
|
||||||
|
### Embedding models
|
||||||
|
|
||||||
|
Embeddings turn text into a numeric vector — the input to semantic search,
|
||||||
|
clustering and RAG. They're called on `/v1/embeddings` with your co-op API key
|
||||||
|
and are **not chat models**: they never appear in the chat picker, the API is
|
||||||
|
the interface (the same shape as the audio models above). You choose between
|
||||||
|
them the same way as for chat: a TEE-protected option (Tinfoil) or
|
||||||
|
renewable-energy options (GreenPT).
|
||||||
|
|
||||||
|
| Model | Provider | Privacy | Green | Dimensions | Max input | Pricing |
|
||||||
|
|---|---|---|---|---|---|---|
|
||||||
|
| `🌱 greenpt/green-embeddings` | GreenPT | policy | ✅ 100% renewable | 2560 (measured) | 32K tokens (per GreenPT's catalog) | $0.20 / 1M input tokens (no output charge) |
|
||||||
|
| `🌱 greenpt/qwen3-embedding-8b` | GreenPT | policy | ✅ 100% renewable | 4096 (measured) | 32K tokens (per GreenPT's catalog) | $0.25 / 1M input tokens (no output charge) |
|
||||||
|
| `🔒 tinfoil/nomic-embed-text` | Tinfoil | **architectural (TEE)** | — (mix not disclosed) | 768 (measured) | 8K tokens (per Tinfoil's catalog) | $0.016 / 1M input tokens (no output charge) |
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl https://gateway.inference.coop/v1/embeddings \
|
||||||
|
-H "Authorization: Bearer ***" \
|
||||||
|
-H "Content-Type: application/json" \
|
||||||
|
-d '{"model": "tinfoil/nomic-embed-text", "input": "the text to embed"}'
|
||||||
|
```
|
||||||
|
|
||||||
|
The response is standard OpenAI shape: `data[0].embedding` holds the vector,
|
||||||
|
and `usage.prompt_tokens` is what meters against your allowance at the rate
|
||||||
|
above. Embeddings bill input tokens only — there is no output charge.
|
||||||
|
|
||||||
|
Vectors from different models are not interchangeable — pick one per search
|
||||||
|
index and stay with it (re-index if you switch). All three are callable with a
|
||||||
|
member key; for reference, the co-op's own documentation search runs on
|
||||||
|
`tinfoil/nomic-embed-text` inside the TEE.
|
||||||
|
|
||||||
### Image generation
|
### Image generation
|
||||||
|
|
||||||
`🌱 regolo/qwen-image` (Regolo, EU-hosted, zero data retention) generates
|
`🌱 regolo/qwen-image` (Regolo, EU-hosted, zero data retention) generates
|
||||||
images from a text prompt. In the chat, press the **Integrations** button
|
images from a text prompt. It runs the December 2025 "2512" checkpoint of
|
||||||
|
Qwen-Image — not the newer Qwen Image 2.1 (September 2026), which Regolo does
|
||||||
|
not list (as of 2026-11-09). In the chat, press the **Integrations** button
|
||||||
below the message box, choose **Image**, and describe what you want — the
|
below the message box, choose **Image**, and describe what you want — the
|
||||||
generated image appears in the conversation. Via the API:
|
generated image appears in the conversation. Via the API:
|
||||||
|
|
||||||
|
|||||||
+1
-27
@@ -1,20 +1,11 @@
|
|||||||
# Open Questions for Member Discussion
|
# Open Questions for Member Discussion
|
||||||
|
|
||||||
A living list of decisions the cooperative needs to make. Each question is framed for discussion on [Loomio](https://forum.inference.coop). As questions are resolved, they move to a "Decisions" log (to be created) rather than being deleted.
|
A living list of some of the decisions the cooperative needs to make.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## Privacy & Infrastructure
|
## Privacy & Infrastructure
|
||||||
|
|
||||||
### Search: self-managed or Tinfoil's encrypted search?
|
|
||||||
|
|
||||||
We currently run our own [SearXNG](https://searxng.github.io/searxng/) instance (`search.inference.coop`) to power web search in the chat. Tinfoil also offers a `websearch` model — a private, encrypted search that runs inside their enclave.
|
|
||||||
|
|
||||||
- **Self-managed (SearXNG):** full control, no external dependency, but search queries are visible to our own infrastructure (and to the search engines SearXNG queries).
|
|
||||||
- **Tinfoil's encrypted search:** queries are encrypted end-to-end, but it's a third-party dependency and may cost per-query.
|
|
||||||
|
|
||||||
**Question:** Do we prioritize self-hosting (control, no external deps) or encrypted search (stronger privacy, but a dependency on Tinfoil)?
|
|
||||||
|
|
||||||
### Green energy vs. privacy — when we can't have both?
|
### Green energy vs. privacy — when we can't have both?
|
||||||
|
|
||||||
Our inference runs through Tinfoil (TEE-protected, verifiable privacy), but Tinfoil's energy mix is not disclosed. Self-hosting on renewable-powered hardware (e.g. Hetzner hydropower, Scaleway wind+hydro) would be greener but would lose the TEE privacy guarantee at our current scale.
|
Our inference runs through Tinfoil (TEE-protected, verifiable privacy), but Tinfoil's energy mix is not disclosed. Self-hosting on renewable-powered hardware (e.g. Hetzner hydropower, Scaleway wind+hydro) would be greener but would lose the TEE privacy guarantee at our current scale.
|
||||||
@@ -24,12 +15,6 @@ Our inference runs through Tinfoil (TEE-protected, verifiable privacy), but Tinf
|
|||||||
|
|
||||||
**Question:** When the two conflict, which value wins? Is there a threshold (e.g. member count, cost) at which we'd switch?
|
**Question:** When the two conflict, which value wins? Is there a threshold (e.g. member count, cost) at which we'd switch?
|
||||||
|
|
||||||
### Should we adopt Tinfoil's other models (embeddings, private search, audio)?
|
|
||||||
|
|
||||||
Tinfoil's catalog includes `nomic-embed-text` (embeddings), `websearch` (private search), and audio models (Whisper, TTS). We currently use Open WebUI's bundled local embedding model for file uploads.
|
|
||||||
|
|
||||||
**Question:** Do we standardize on Tinfoil for all model needs (consistency, privacy), or keep the local embedding model (no per-token cost, no dependency)?
|
|
||||||
|
|
||||||
### Which models should we offer?
|
### Which models should we offer?
|
||||||
|
|
||||||
We currently expose three chat models: DeepSeek V4 Flash (default), GPT-OSS 120B, and GLM-5.3 Flash. Tinfoil's full catalog also includes GLM-5.3 (full), Kimi K3, Llama 3.3 70B, Gemma 4 31B, and others.
|
We currently expose three chat models: DeepSeek V4 Flash (default), GPT-OSS 120B, and GLM-5.3 Flash. Tinfoil's full catalog also includes GLM-5.3 (full), Kimi K3, Llama 3.3 70B, Gemma 4 31B, and others.
|
||||||
@@ -57,22 +42,11 @@ Members currently pay dues on a sliding scale (from $5/month) with a flat $15/mo
|
|||||||
|
|
||||||
**Question:** What pricing model best balances fairness, simplicity, and the cooperative's solidarity values? How do we handle a member who consistently exceeds their budget?
|
**Question:** What pricing model best balances fairness, simplicity, and the cooperative's solidarity values? How do we handle a member who consistently exceeds their budget?
|
||||||
|
|
||||||
### What happens when a member's budget runs out mid-month?
|
|
||||||
|
|
||||||
Members have a **flexible balance**, not a hardcoded $15/month. The balance starts at $15 but can be topped up by payments (via Open Collective or another system) through the portal's `/admin/set-balance` endpoint, which pushes the new cap to the member's LiteLLM team budget (chat + API draw from the same balance). LiteLLM enforces the cap — requests stop when it's exhausted.
|
|
||||||
|
|
||||||
**Question:** Should members be able to top up self-serve? Should there be a grace period? Should the co-op pool surplus to cover light users? (The *mechanism* is built — these are now governance decisions, not technical blockers.)
|
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## Governance & Membership
|
## Governance & Membership
|
||||||
|
|
||||||
### What does "member-governed" mean in practice before 50 members?
|
|
||||||
|
|
||||||
The charter says the General Manager (Nathan) has full governance until 50 members, then transitions to member governance. But members can already deliberate on Loomio.
|
|
||||||
|
|
||||||
**Question:** Which decisions should be opened to member input *now* (even if non-binding), versus reserved for the General Manager during the pilot?
|
|
||||||
|
|
||||||
### How do we handle institutional members?
|
### How do we handle institutional members?
|
||||||
|
|
||||||
AI Potluck's "government-as-contributor" framing suggests a path for institutions (universities, municipalities) to contribute compute or data without owning the whole.
|
AI Potluck's "government-as-contributor" framing suggests a path for institutions (universities, municipalities) to contribute compute or data without owning the whole.
|
||||||
|
|||||||
@@ -4,6 +4,10 @@
|
|||||||
{{ config.copyright }}
|
{{ config.copyright }}
|
||||||
</div>
|
</div>
|
||||||
{% endif %}
|
{% endif %}
|
||||||
|
<div class="md-copyright__highlight">
|
||||||
|
Co-op homepage:
|
||||||
|
<a href="https://inference.coop/" rel="noopener">inference.coop</a>
|
||||||
|
</div>
|
||||||
<div class="md-copyright__highlight">
|
<div class="md-copyright__highlight">
|
||||||
Documentation source:
|
Documentation source:
|
||||||
<a href="https://git.inference.coop/co-op/docs" target="_blank" rel="noopener">co-op/docs on Gitea</a>
|
<a href="https://git.inference.coop/co-op/docs" target="_blank" rel="noopener">co-op/docs on Gitea</a>
|
||||||
|
|||||||
+37
-15
@@ -90,14 +90,13 @@
|
|||||||
--md-default-fg-color--lighter: #6f7a68;
|
--md-default-fg-color--lighter: #6f7a68;
|
||||||
--md-default-fg-color--lightest: rgba(148, 160, 140, 0.12);
|
--md-default-fg-color--lightest: rgba(148, 160, 140, 0.12);
|
||||||
|
|
||||||
/* The header was an unreadably dark green. Green-mid with near-black pine
|
/* Header bar now matches the chat overlay's dark menu: #171717 with white
|
||||||
* text keeps the bar clearly green, clearly distinct from the page, and
|
* text (operator request 2026-11-09) — neutral dark, not green. */
|
||||||
* legible (measured below). */
|
--md-primary-fg-color: #171717;
|
||||||
--md-primary-fg-color: var(--green-mid);
|
--md-primary-fg-color--light: #171717;
|
||||||
--md-primary-fg-color--light: var(--green-light);
|
--md-primary-fg-color--dark: #171717;
|
||||||
--md-primary-fg-color--dark: var(--green-deep);
|
--md-primary-bg-color: #ffffff;
|
||||||
--md-primary-bg-color: #16201a;
|
--md-primary-bg-color--light: rgba(255, 255, 255, 0.7);
|
||||||
--md-primary-bg-color--light: rgba(22, 32, 26, 0.7);
|
|
||||||
|
|
||||||
--md-accent-fg-color: var(--green-light);
|
--md-accent-fg-color: var(--green-light);
|
||||||
--md-typeset-a-color: var(--green-light);
|
--md-typeset-a-color: var(--green-light);
|
||||||
@@ -185,19 +184,42 @@
|
|||||||
font-weight: 600;
|
font-weight: 600;
|
||||||
}
|
}
|
||||||
|
|
||||||
/* Tabs and the search field pick up the brand green. */
|
/* The menu-bar (tab) strip. In light it is the same pale green WASH as the
|
||||||
|
* header — #e8f5e8, the homepage's "open source" pill — not a saturated fill:
|
||||||
|
* the deep greens are accents (links, active navigation, rules), and a large
|
||||||
|
* --green-deep bar clashed with the rest of the light palette. A warm hairline
|
||||||
|
* separates the wash from the cream page below it. Dark is unchanged. */
|
||||||
.md-tabs {
|
.md-tabs {
|
||||||
background-color: var(--green-deep);
|
background-color: #e8f5e8;
|
||||||
|
border-bottom: 1px solid var(--border);
|
||||||
}
|
}
|
||||||
/* In dark, keep the tab row the same green as the header bar so the text on it
|
/* In dark, the tab row drops to the sheet's dark warm-green (dark-border
|
||||||
* clears 4.5:1 against green-mid (green-deep against near-black text is only
|
* #4a5240): quieter than green-mid, white labels still clear 7:1. */
|
||||||
* ~4.2:1 — marginal for the small tab labels). */
|
|
||||||
[data-md-color-scheme="slate"] .md-tabs {
|
[data-md-color-scheme="slate"] .md-tabs {
|
||||||
background-color: var(--green-mid);
|
background-color: var(--dark-border);
|
||||||
|
border-bottom: none;
|
||||||
}
|
}
|
||||||
.md-search__input {
|
/* Search input: the white field is a light-mode choice only. In dark mode the
|
||||||
|
* search surfaces use the pine family — input on the chips tone, light text. */
|
||||||
|
html:not([data-md-color-scheme="slate"]) .md-search__input {
|
||||||
background-color: rgba(255, 255, 255, 0.92);
|
background-color: rgba(255, 255, 255, 0.92);
|
||||||
}
|
}
|
||||||
|
[data-md-color-scheme="slate"] .md-search__form,
|
||||||
|
[data-md-color-scheme="slate"] .md-search__input {
|
||||||
|
background-color: var(--pine-3);
|
||||||
|
color: var(--dark-text);
|
||||||
|
}
|
||||||
|
[data-md-color-scheme="slate"] .md-search__form .md-icon,
|
||||||
|
[data-md-color-scheme="slate"] .md-search__input::placeholder {
|
||||||
|
color: var(--dark-muted);
|
||||||
|
}
|
||||||
|
[data-md-color-scheme="slate"] .md-search__scrollwrap,
|
||||||
|
[data-md-color-scheme="slate"] .md-search-result__article {
|
||||||
|
background-color: var(--pine-2);
|
||||||
|
}
|
||||||
|
[data-md-color-scheme="slate"] .md-search-result__meta {
|
||||||
|
color: var(--dark-muted);
|
||||||
|
}
|
||||||
|
|
||||||
/* Admonitions get the co-op's washes instead of stock greys. */
|
/* Admonitions get the co-op's washes instead of stock greys. */
|
||||||
.md-typeset .admonition,
|
.md-typeset .admonition,
|
||||||
|
|||||||
Reference in new issue
Block a user