docs/README.md

# Inference Cooperative

**Private AI, governed together.**

The Inference Cooperative is a member-governed project providing private AI inference. We are fiscally sponsored by [Metagov](https://metagov.org), a nonprofit, and funded through our [Open Collective](https://opencollective.com/inference-cooperative).

- **Website:** https://inference.coop
- **AI Chat:** https://chat.inference.coop
- **Open Collective:** https://opencollective.com/inference-cooperative
- **Contact:** info@inference.coop

## The model

The Inference Cooperative is a **member-governed** AI inference utility. Members contribute on a sliding scale (currently $10–20/month) and, in return, get access to private AI inference through a shared, cooperatively governed infrastructure.

Because we are legally under a nonprofit—[Metagov](https://metagov.org/) is our fiscal sponsor—we are member-governed rather than member-owned: members direct the project through governance, while the nonprofit holds the legal and financial structure.

### Membership

- **Sliding scale dues:** $10–20/month
- **Access:** private AI inference through the cooperative's gateway
- **Governance:** members participate in decisions about models, priorities, and direction

**Important:** your email address on Open Collective and on Cloudron must match. Membership is verified by matching the email you use to contribute on Open Collective against the email you use to log in. If they differ, you won't be recognized as a member. (Guest contributors — those who contribute without an Open Collective account — are recognized by the email they entered at checkout.)

**Access control:** the chat app (Open WebUI) and the governance app (Loomio) are restricted to Cloudron's **`members`** group. Provisioned members are added to this group; lapsed members are moved to the `inactive` group (which has no app access). Both apps must be listed in the `members` group's app list — if either is missing, members can't reach it.

### Founder membership (free, invite-only)

There is a **free "Co-founder" membership** for early adopters — people who have been involved in earlier work on the project. It is **invite-only**: we reach out to trusted contributors directly rather than accepting requests from the general public.

Founder members are onboarded **manually** (Open Collective cannot process a $0 recurring subscription — recurring contributions require an automatic payment method, and a $0 tier has none):

1. We invite the founder and collect their name and email.
2. Their email is added to the portal's `MANUAL_MEMBERS` allowlist (a comma-separated env var). This exempts them from the daily sweep's deactivation, since they have no Open Collective membership to lapse.
3. They are provisioned via the portal's `/admin/provision` endpoint (Cloudron user + LiteLLM key + welcome email + Loomio sync).

## The stack

The Inference Cooperative runs a self-hosted stack on [Cloudron](https://cloudron.io), with all inference routed through a single gateway to cloud-based LLM providers.

### Components

| Component | Purpose | URL |
|-----------|---------|-----|
| **Open WebUI** | Member-facing chat interface (with web search) | chat.inference.coop |
| **LiteLLM** | AI gateway — keys, metering, model routing | gateway.inference.coop |
| **Member Portal** | Membership middleware — onboarding, key provisioning, Loomio sync | portal.inference.coop |
| **Member Dashboard** | Member-facing usage view + API key management | dashboard.inference.coop |
| **Admin Panel** | Admin-only member overview, spend, and balance management | panel.inference.coop |
| **SearXNG** | Self-hosted web search (feeds the chat's search tool) | search.inference.coop |
| **Gitea** | Git hosting (code + docs) | git.inference.coop |
| **Loomio** | Member governance | forum.inference.coop |
| **Open Collective** | Membership billing + fiscal sponsorship | opencollective.com/inference-cooperative |

The cooperative's software is open source and lives in the [`code`](https://git.inference.coop/code) organization on our Gitea instance.

### Models

The gateway currently exposes three models (all served through Tinfoil's TEE-protected enclaves):

- **DeepSeek V4.1 Flash** — default model, best for agentic tasks (1M context, tool calling)
- **GPT-OSS 120B** — lightweight fallback
- **GLM-5.3 Flash** — fast, efficient MoE model

### Features

- **Web search** — enabled by default, backed by a self-hosted SearXNG instance (no external search API or scraper key required).
- **File uploads (RAG)** — works out of the box via Open WebUI's bundled local embedding model (`sentence-transformers/all-MiniLM-L6-v2`), no external embedding API.

### API access

Members can use the co-op's models programmatically through an OpenAI-compatible API:

- **Endpoint:** `https://gateway.inference.coop/v1`
- **Auth:** a bearer API key (created in the [Member Dashboard](https://dashboard.inference.coop))
- **Format:** standard OpenAI chat completions (`POST /v1/chat/completions`)

Example:

```bash
curl https://gateway.inference.coop/v1/chat/completions \
  -H "Authorization: Bearer sk-..." \
  -H "Content-Type: application/json" \
  -d '{"model": "deepseek-v4-1-flash", "messages": [{"role": "user", "content": "Hello"}]}'
```

API usage draws from the **same monthly balance as chat** — there is no separate quota. The gateway is publicly reachable (member keys authenticate it), but its admin UI and API documentation are disabled; admin access is via the internal dashboard or the gateway's admin endpoints.

### Privacy

Privacy is a core value. Inference now runs through [Tinfoil](https://tinfoil.sh/inference), which provides **architectural** privacy: models run inside hardware enclaves (TEEs), and request/response bodies are encrypted end-to-end with the Encrypted HTTP Body Protocol (EHBP), so even Tinfoil's own infrastructure cannot read them. This is verifiable via remote attestation — not just a policy promise.

Because Tinfoil requires EHBP-encrypted request bodies (it rejects plaintext with `426 EHBP_REQUIRED`), the LiteLLM gateway routes through a local [Tinfoil proxy](https://github.com/tinfoilsh/tinfoil-proxy) sidecar, which verifies the enclave attestation and handles the encryption. LiteLLM talks plaintext OpenAI to the proxy; the proxy encrypts and forwards to the enclave.


## Governance

Members govern the project through [Loomio](https://www.loomio.com) and the Open Collective. See the [Charter](charter.md) for the cooperative's values, membership terms, and governance structure, and [Open Questions](open-questions.md) for the decisions currently open for member discussion.

During the pilot phase, Nathan Schneider serves as Managing Director and has sole final discretion on decisions.

## Legal

- [Terms of Service](terms-of-service.md) — the agreement between the cooperative and its members about the service.
- [Privacy Policy](privacy-policy.md) — how we handle your data, including what we can and can't see.

By creating your account, you agree to these terms.

## Planning

- [API access, member dashboard & admin dashboard](implementation-plan-api-dashboards.md) — the plan for API access, key management, and dashboards (Steps 1–5 complete). 

---

*This documentation lives in a public git repository. Contributions welcome.*