Split docs into README TOC + getting-started, api-access, infrastructure guides
This commit is contained in:
inference-bot committed
1 parent
39b6bfa770
commit
4cccc521e8
4 files changed
+298
-172
No files matched your search
@@ -2,184 +2,33 @@
|
||||
|
||||
**Private AI, governed together.**
|
||||
|
||||
The Inference Cooperative is a member-governed project providing private AI inference. We are fiscally sponsored by [Metagov](https://metagov.org), a nonprofit, and funded through our [Open Collective](https://opencollective.com/inference-cooperative).
|
||||
The Inference Cooperative is a member-governed project providing private AI
|
||||
inference. Members pay dues on a sliding scale and, in return, get private AI
|
||||
chat and API access through shared, cooperatively governed infrastructure. We
|
||||
are fiscally sponsored by [Metagov](https://metagov.org), a nonprofit, and
|
||||
funded through our [Open Collective](https://opencollective.com/inference-cooperative).
|
||||
|
||||
## Start here
|
||||
|
||||
- **New here?** → [Getting started](getting-started.md) — join, use the chat, and get involved in governance.
|
||||
- **Want API access?** → [API access](api-access.md) — use our models programmatically.
|
||||
- **Curious how it works?** → [Infrastructure](infrastructure.md) — a map of the stack and the codebase.
|
||||
|
||||
## Key links
|
||||
|
||||
- **Website:** https://inference.coop
|
||||
- **AI Chat:** https://chat.inference.coop
|
||||
- **Open Collective:** https://opencollective.com/inference-cooperative
|
||||
- **Member Dashboard:** https://dashboard.inference.coop
|
||||
- **Governance forum:** https://forum.inference.coop
|
||||
- **Source code:** https://git.inference.coop
|
||||
- **Contact:** info@inference.coop
|
||||
|
||||
## The model
|
||||
## Governance & legal
|
||||
|
||||
The Inference Cooperative is a **member-governed** AI inference utility. Members contribute on a sliding scale (currently $10–20/month) and, in return, get access to private AI inference through a shared, cooperatively governed infrastructure.
|
||||
|
||||
Because we are legally under a nonprofit—[Metagov](https://metagov.org/) is our fiscal sponsor—we are member-governed rather than member-owned: members direct the project through governance, while the nonprofit holds the legal and financial structure.
|
||||
|
||||
### Membership
|
||||
|
||||
- **Sliding scale dues:** $10–20/month
|
||||
- **Access:** private AI inference through the cooperative's gateway
|
||||
- **Governance:** members participate in decisions about models, priorities, and direction
|
||||
|
||||
**Important:** your email address on Open Collective and on Cloudron must match. Membership is verified by matching the email you use to contribute on Open Collective against the email you use to log in. If they differ, you won't be recognized as a member. (Guest contributors — those who contribute without an Open Collective account — are recognized by the email they entered at checkout.)
|
||||
|
||||
**Access control:** the chat app (Open WebUI) and the governance app (Loomio) are restricted to Cloudron's **`members`** group. Provisioned members are added to this group; lapsed members are moved to the `inactive` group (which has no app access). Both apps must be listed in the `members` group's app list — if either is missing, members can't reach it.
|
||||
|
||||
### Founder membership (free, invite-only)
|
||||
|
||||
There is a **free "Co-founder" membership** for early adopters — people who have been involved in earlier work on the project. It is **invite-only**: we reach out to trusted contributors directly rather than accepting requests from the general public.
|
||||
|
||||
Founder members are onboarded **manually** (Open Collective cannot process a $0 recurring subscription — recurring contributions require an automatic payment method, and a $0 tier has none):
|
||||
|
||||
1. We invite the founder and collect their name and email.
|
||||
2. Their email is added to the portal's `MANUAL_MEMBERS` allowlist (a comma-separated env var). This exempts them from the daily sweep's deactivation, since they have no Open Collective membership to lapse.
|
||||
3. They are provisioned via the portal's `/admin/provision` endpoint (Cloudron user + LiteLLM key + welcome email + Loomio sync).
|
||||
|
||||
## The stack
|
||||
|
||||
The Inference Cooperative runs a self-hosted stack on [Cloudron](https://cloudron.io), with all inference routed through a single gateway to cloud-based LLM providers.
|
||||
|
||||
### Components
|
||||
|
||||
| Component | Purpose | URL |
|
||||
|-----------|---------|-----|
|
||||
| **Open WebUI** | Member-facing chat interface (with web search) | chat.inference.coop |
|
||||
| **LiteLLM** | AI gateway — keys, metering, model routing | gateway.inference.coop |
|
||||
| **Member Portal** | Membership middleware — onboarding, key provisioning, Loomio sync | portal.inference.coop |
|
||||
| **Member Dashboard** | Member-facing usage view + API key management | dashboard.inference.coop |
|
||||
| **Admin Panel** | Admin-only member overview, spend, and balance management | panel.inference.coop |
|
||||
| **SearXNG** | Self-hosted web search (feeds the chat's search tool) | search.inference.coop |
|
||||
| **Gitea** | Git hosting (code + docs) | git.inference.coop |
|
||||
| **Loomio** | Member governance | forum.inference.coop |
|
||||
| **Open Collective** | Membership billing + fiscal sponsorship | opencollective.com/inference-cooperative |
|
||||
| **Listmonk** | Member newsletter (list auto-synced from membership) | newsletter.inference.coop |
|
||||
|
||||
The cooperative's software is open source and lives in the [`code`](https://git.inference.coop/code) organization on our Gitea instance.
|
||||
|
||||
### Newsletter
|
||||
|
||||
Member newsletters are sent via **Listmonk** (self-hosted, at `newsletter.inference.coop`).
|
||||
|
||||
- **Single source of truth:** the portal database. Listmonk is *not* managed by hand.
|
||||
- **Auto-sync:** the portal subscribes a member to the "Members" list on provisioning, and unsubscribes them on deactivation (both in the webhook path and the nightly sweep). It's best-effort — a newsletter outage can't break onboarding.
|
||||
- **Opt-in model:** single opt-in. Members consent by joining, so they're auto-confirmed rather than double-opted-in (no second confirmation email at signup).
|
||||
- **Sender:** `info@inference.coop` via Cloudron's mail addon, inheriting the domain's SPF/DKIM/DMARC alignment.
|
||||
- **Machine credential:** the portal authenticates to listmonk using a dedicated API user (`portal-sync`) with a token stored in the portal's Cloudron env vars (`LISTMONK_API_TOKEN`), not a password. Human admins log into listmonk via Cloudron SSO; password login is disabled.
|
||||
|
||||
### Models
|
||||
|
||||
The gateway currently exposes three models (all served through Tinfoil's TEE-protected enclaves):
|
||||
|
||||
- **DeepSeek V4.1 Flash** — default model, best for agentic tasks (1M context, tool calling)
|
||||
- **GPT-OSS 120B** — lightweight fallback
|
||||
- **GLM-5.3 Flash** — fast, efficient MoE model
|
||||
|
||||
### Features
|
||||
|
||||
- **Web search** — enabled by default, backed by a self-hosted SearXNG instance (no external search API or scraper key required).
|
||||
- **File uploads (RAG)** — works out of the box via Open WebUI's bundled local embedding model (`sentence-transformers/all-MiniLM-L6-v2`), no external embedding API.
|
||||
|
||||
### API access
|
||||
|
||||
Members can use the co-op's models programmatically through an OpenAI-compatible API:
|
||||
|
||||
- **Endpoint (base URL):** `https://gateway.inference.coop/v1`
|
||||
- **Auth:** a bearer API key (create one in the [Member Dashboard](https://dashboard.inference.coop))
|
||||
- **Format:** OpenAI chat completions — `POST https://gateway.inference.coop/v1/chat/completions`
|
||||
- **Models:** `deepseek-v4-1-flash` (default), `gpt-oss-120b`, `glm-5-3-flash`
|
||||
|
||||
API usage draws from the **same monthly balance as chat** — there is no separate quota. The gateway is publicly reachable (member keys authenticate it), but its admin UI and API documentation are disabled; admin access is via the internal dashboard or the gateway's admin endpoints.
|
||||
|
||||
#### Plain `curl`
|
||||
|
||||
```bash
|
||||
curl https://gateway.inference.coop/v1/chat/completions \
|
||||
-H "Authorization: Bearer sk-your-key" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"model": "deepseek-v4-1-flash", "messages": [{"role": "user", "content": "Hello"}]}'
|
||||
```
|
||||
|
||||
#### OpenAI SDK (Python, Node, etc.)
|
||||
|
||||
Point the SDK at our base URL and use the member key:
|
||||
|
||||
```python
|
||||
from openai import OpenAI
|
||||
|
||||
client = OpenAI(
|
||||
base_url="https://gateway.inference.coop/v1",
|
||||
api_key="sk-your-key",
|
||||
)
|
||||
resp = client.chat.completions.create(
|
||||
model="deepseek-v4-1-flash",
|
||||
messages=[{"role": "user", "content": "Hello"}],
|
||||
)
|
||||
```
|
||||
|
||||
Any OpenAI-compatible client works the same way: set the base URL to `https://gateway.inference.coop/v1` and the API key to your member key.
|
||||
|
||||
#### Coding agents and other OpenAI-compatible clients
|
||||
|
||||
Any client that speaks the OpenAI API works the same way. Where a tool asks for a provider/endpoint configuration:
|
||||
|
||||
- **Base URL / endpoint:** `https://gateway.inference.coop/v1`
|
||||
- **API type / protocol:** "OpenAI compatible" (or `openai-completions` where a protocol must be named) — not "Ollama", "Anthropic", or "Responses"
|
||||
- **API key:** your member key (the `sk-…` string)
|
||||
- **Model:** one of `deepseek-v4-1-flash`, `gpt-oss-120b`, `glm-5-3-flash`
|
||||
|
||||
A typical JSON provider entry looks like:
|
||||
|
||||
```json
|
||||
{
|
||||
"providers": {
|
||||
"inference-coop": {
|
||||
"baseUrl": "https://gateway.inference.coop/v1",
|
||||
"api": "openai-completions",
|
||||
"apiKey": "sk-your-key",
|
||||
"models": [
|
||||
{ "id": "deepseek-v4-1-flash" },
|
||||
{ "id": "gpt-oss-120b" },
|
||||
{ "id": "glm-5-3-flash" }
|
||||
]
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
If a tool's own docs show a local-server example (e.g. `baseUrl: http://localhost:11434/v1` with `api: "openai-completions"`), substitute our base URL and your member key for `apiKey` — the protocol value stays the same. Some tools append `/chat/completions` to the base URL themselves; if your base URL already ends in `/v1`, that's correct and you should not add `/chat/completions`.
|
||||
|
||||
### Privacy
|
||||
|
||||
Privacy is a core value. Inference now runs through [Tinfoil](https://tinfoil.sh/inference), which provides **architectural** privacy: models run inside hardware enclaves (TEEs), and request/response bodies are encrypted end-to-end with the Encrypted HTTP Body Protocol (EHBP), so even Tinfoil's own infrastructure cannot read them. This is verifiable via remote attestation — not just a policy promise.
|
||||
|
||||
Because Tinfoil requires EHBP-encrypted request bodies (it rejects plaintext with `426 EHBP_REQUIRED`), the LiteLLM gateway routes through a local [Tinfoil proxy](https://github.com/tinfoilsh/tinfoil-proxy) sidecar, which verifies the enclave attestation and handles the encryption. LiteLLM talks plaintext OpenAI to the proxy; the proxy encrypts and forwards to the enclave.
|
||||
|
||||
|
||||
### Security posture
|
||||
|
||||
A plain-language summary of how the co-op's infrastructure is secured, and the trade-offs we've deliberately accepted.
|
||||
|
||||
- **Public-but-token-gated control plane.** The member portal (`portal.inference.coop`) is publicly reachable but every sensitive surface is gated: the admin endpoints require an `X-Admin-Token` header, the broker requires `X-Broker-Secret`, and the Open Collective webhook requires a secret token in its URL (Open Collective's webhooks aren't HMAC-signed, so a secret URL is the standard mechanism). All endpoints fail closed — unauthenticated requests get `401`. The portal stays public because the webhook is an external caller that can't go through Cloudron SSO.
|
||||
- **Secret separation.** Chat, broker, admin, and model-layer secrets are independent and independently rotatable, so a leak in one has bounded scope.
|
||||
- **Member isolation.** A member's API key is scoped to their own LiteLLM team and budget; the broker can only ever operate on the authenticated member's own keys.
|
||||
- **Rate limiting.** Control-plane endpoints (admin, broker, webhook, the model list) are rate-limited per IP. The member inference path is *not* rate-limited — it's capped by each member's budget instead, so legitimate long generations aren't throttled.
|
||||
- **No secrets in version control.** All credentials live in Cloudron environment variables, never in the repos.
|
||||
- **Accepted trade-off — keys at rest.** Member chat keys and API keys are stored in plaintext in the portal's database. This is functionally necessary (the injector needs the chat key per request; the broker must display a freshly-created API key once), and Cloudron app volumes aren't encrypted at rest. Access to that volume is limited to the portal app itself.
|
||||
- **Accepted trade-off — gateway surface.** The LiteLLM gateway must stay publicly reachable for member API access. Its admin UI and docs (`/docs`, `/redoc`, Swagger) are disabled; only the `/v1/*` API is exposed, which requires a bearer key.
|
||||
|
||||
|
||||
## Governance
|
||||
|
||||
Members govern the project through [Loomio](https://www.loomio.com) and the Open Collective. See the [Charter](charter.md) for the cooperative's values, membership terms, and governance structure, and [Open Questions](open-questions.md) for the decisions currently open for member discussion.
|
||||
|
||||
During the pilot phase, Nathan Schneider serves as Managing Director and has sole final discretion on decisions.
|
||||
|
||||
## Legal
|
||||
|
||||
- [Terms of Service](terms-of-service.md) — the agreement between the cooperative and its members about the service.
|
||||
- [Privacy Policy](privacy-policy.md) — how we handle your data, including what we can and can't see.
|
||||
|
||||
By creating your account, you agree to these terms.
|
||||
- [Charter](charter.md) — values, membership, and governance.
|
||||
- [Open questions](open-questions.md) — decisions currently up for member discussion.
|
||||
- [Terms of Service](terms-of-service.md)
|
||||
- [Privacy Policy](privacy-policy.md)
|
||||
|
||||
---
|
||||
|
||||
|
||||
@@ -0,0 +1,89 @@
|
||||
# API access
|
||||
|
||||
Members can use the co-op's models programmatically through an OpenAI-compatible
|
||||
API. It's the same infrastructure that powers the chat — any OpenAI-compatible
|
||||
client works.
|
||||
|
||||
## Quick facts
|
||||
|
||||
- **Base URL:** `https://gateway.inference.coop/v1`
|
||||
- **Auth:** a bearer API key (create one in the [Member Dashboard](https://dashboard.inference.coop))
|
||||
- **Format:** OpenAI chat completions — `POST https://gateway.inference.coop/v1/chat/completions`
|
||||
- **Models:**
|
||||
- `deepseek-v4-1-flash` — default, best for agentic tasks (1M context, tool calling)
|
||||
- `gpt-oss-120b` — lightweight fallback
|
||||
- `glm-5-3-flash` — fast, efficient MoE model
|
||||
|
||||
API usage draws from the **same monthly allowance as chat** — there's no separate
|
||||
quota.
|
||||
|
||||
## Get a key
|
||||
|
||||
Log in to the [Member Dashboard](https://dashboard.inference.coop) and create an
|
||||
API key. The key is an `sk-…` string you pass as a bearer token.
|
||||
|
||||
## Plain `curl`
|
||||
|
||||
```bash
|
||||
curl https://gateway.inference.coop/v1/chat/completions \
|
||||
-H "Authorization: Bearer sk-your-key" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"model": "deepseek-v4-1-flash", "messages": [{"role": "user", "content": "Hello"}]}'
|
||||
```
|
||||
|
||||
## OpenAI SDK (Python, Node, etc.)
|
||||
|
||||
Point the SDK at our base URL and use your member key:
|
||||
|
||||
```python
|
||||
from openai import OpenAI
|
||||
|
||||
client = OpenAI(
|
||||
base_url="https://gateway.inference.coop/v1",
|
||||
api_key="sk-your-key",
|
||||
)
|
||||
resp = client.chat.completions.create(
|
||||
model="deepseek-v4-1-flash",
|
||||
messages=[{"role": "user", "content": "Hello"}],
|
||||
)
|
||||
```
|
||||
|
||||
Any OpenAI-compatible client works the same way: set the base URL to
|
||||
`https://gateway.inference.coop/v1` and the API key to your member key.
|
||||
|
||||
## Coding agents and other OpenAI-compatible clients
|
||||
|
||||
Any tool that speaks the OpenAI API works. Where a tool asks for provider or
|
||||
endpoint configuration:
|
||||
|
||||
- **Base URL / endpoint:** `https://gateway.inference.coop/v1`
|
||||
- **API type / protocol:** "OpenAI compatible" (or `openai-completions` where a
|
||||
protocol string must be named) — not "Ollama", "Anthropic", or "Responses"
|
||||
- **API key:** your member key (the `sk-…` string)
|
||||
- **Model:** one of `deepseek-v4-1-flash`, `gpt-oss-120b`, `glm-5-3-flash`
|
||||
|
||||
A typical provider entry looks like:
|
||||
|
||||
```json
|
||||
{
|
||||
"providers": {
|
||||
"inference-coop": {
|
||||
"baseUrl": "https://gateway.inference.coop/v1",
|
||||
"api": "openai-completions",
|
||||
"apiKey": "sk-your-key",
|
||||
"models": [
|
||||
{ "id": "deepseek-v4-1-flash" },
|
||||
{ "id": "gpt-oss-120b" },
|
||||
{ "id": "glm-5-3-flash" }
|
||||
]
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
If a tool's own docs show a local-server example (e.g.
|
||||
`baseUrl: http://localhost:11434/v1` with `api: "openai-completions"`),
|
||||
substitute our base URL and your member key for `apiKey` — the protocol value
|
||||
stays the same. Some tools append `/chat/completions` to the base URL themselves;
|
||||
if your base URL already ends in `/v1`, that's correct and you should not add
|
||||
`/chat/completions`.
|
||||
@@ -0,0 +1,76 @@
|
||||
# Getting started
|
||||
|
||||
Welcome. You're joining a **member-governed** AI cooperative — not signing up
|
||||
for a service from a vendor. You'll help direct the project while getting
|
||||
private AI chat and API access in return.
|
||||
|
||||
This guide covers the three things most new members want: becoming a member,
|
||||
using the chat, and getting involved in governance.
|
||||
|
||||
## 1. Become a member
|
||||
|
||||
Membership is a **sliding-scale contribution of $10–20/month** through Open
|
||||
Collective. What you pay is up to you.
|
||||
|
||||
1. Go to [opencollective.com/inference-cooperative](https://opencollective.com/inference-cooperative).
|
||||
2. Choose a contribution amount in the $10–20 range and subscribe.
|
||||
3. After you contribute, you'll land on a "check your inbox" page.
|
||||
|
||||
**Within a few minutes, you'll get a welcome email** with a link to set up your
|
||||
account. Click it and create your login.
|
||||
|
||||
> **One thing that matters:** the email address you use on Open Collective and
|
||||
> the email address you use to set up your account **must match**. Membership is
|
||||
> verified by matching those two emails. If they differ, you won't be recognized
|
||||
> as a member. (Contributed without an Open Collective account? You're recognized
|
||||
> by the email you entered at checkout.)
|
||||
|
||||
### Your membership includes
|
||||
|
||||
- **Private AI chat** through the co-op's shared gateway.
|
||||
- **API access** for your own tools and projects (see [API access](api-access.md)).
|
||||
- **A monthly usage allowance** of $15 of credit, shared between chat and API.
|
||||
- **A voice in governance** — how the project is run and where it goes next.
|
||||
|
||||
## 2. Use the chat
|
||||
|
||||
The chat lives at **[chat.inference.coop](https://chat.inference.coop)**. Log in
|
||||
with the account you created in step 1.
|
||||
|
||||
- **Web search** is enabled by default (self-hosted, no external dependency).
|
||||
- **File uploads** work out of the box — ask the chat about a document you attach.
|
||||
- Your usage draws from your monthly allowance; you can see it on the
|
||||
[Member Dashboard](https://dashboard.inference.coop).
|
||||
|
||||
If the chat ever says your balance is out, that's the allowance, not an error —
|
||||
see "Your allowance" below.
|
||||
|
||||
## 3. Get involved in governance
|
||||
|
||||
You don't just use the co-op; you help run it. Decisions about models, pricing,
|
||||
privacy, and direction happen on our **Loomio forum**:
|
||||
|
||||
- **[forum.inference.coop](https://forum.inference.coop)** — where members discuss and decide.
|
||||
- See the [Charter](charter.md) for how governance is structured.
|
||||
- See [Open questions](open-questions.md) for the decisions currently on the table.
|
||||
|
||||
During the pilot phase, Nathan Schneider serves as General Manager and holds
|
||||
final discretion — but members deliberate now, and full member governance is the
|
||||
goal as the co-op grows.
|
||||
|
||||
## Your allowance
|
||||
|
||||
Membership includes **$15 of usage credit each month**, shared between chat and
|
||||
API. It's a starting value, not a hard rule:
|
||||
|
||||
- The allowance can be adjusted per member — contact info@inference.coop.
|
||||
- When it's exhausted, requests pause until the next cycle.
|
||||
|
||||
## Need help?
|
||||
|
||||
- **Questions about the service or your account:** info@inference.coop
|
||||
- **Governance and decisions:** the [forum](https://forum.inference.coop)
|
||||
- **Building on the API:** [API access](api-access.md)
|
||||
- **Leaving:** cancel your subscription on Open Collective; your account is
|
||||
deactivated and your data preserved for reactivation. You may request full
|
||||
deletion.
|
||||
@@ -0,0 +1,112 @@
|
||||
# Infrastructure
|
||||
|
||||
How the Inference Cooperative is built, for members who want to understand it —
|
||||
and contributors who want to work on it.
|
||||
|
||||
## The big picture
|
||||
|
||||
Members reach the co-op through two front doors: the **chat** (Open WebUI) and
|
||||
the **API** (the LiteLLM gateway). Both route through a single gateway to cloud
|
||||
LLM providers running inside **TEE-protected enclaves** (Tinfoil), so inference
|
||||
is private end-to-end. A thin layer of custom middleware — the member portal —
|
||||
ties Open Collective membership to Cloudron accounts and LiteLLM keys.
|
||||
|
||||
```
|
||||
Members ──> Open WebUI (chat) ─┐
|
||||
├──> LiteLLM gateway ──> Tinfoil (TEE) ──> models
|
||||
Members ──> your own tools ────┘
|
||||
(API)
|
||||
```
|
||||
|
||||
The whole stack runs on [Cloudron](https://cloudron.io), which handles hosting,
|
||||
single sign-on, and per-app isolation.
|
||||
|
||||
## Components
|
||||
|
||||
| Component | Purpose | URL | Code |
|
||||
|-----------|---------|-----|------|
|
||||
| **Open WebUI** | Member-facing chat (web search, file uploads) | chat.inference.coop | — |
|
||||
| **LiteLLM** | AI gateway — keys, metering, model routing | gateway.inference.coop | [code/litellm](https://git.inference.coop/code/litellm) |
|
||||
| **Member Portal** | Membership middleware — onboarding, key provisioning, Loomio sync | portal.inference.coop | [code/member-portal](https://git.inference.coop/code/member-portal) |
|
||||
| **Member Dashboard** | Member-facing usage view + API key management | dashboard.inference.coop | [code/member-dashboard](https://git.inference.coop/code/member-dashboard) |
|
||||
| **Admin Panel** | Admin-only member overview, spend, balance management | panel.inference.coop | [code/admin-panel](https://git.inference.coop/code/admin-panel) |
|
||||
| **SearXNG** | Self-hosted web search (feeds the chat's search) | search.inference.coop | — |
|
||||
| **Loomio** | Member governance | forum.inference.coop | — |
|
||||
| **Gitea** | Git hosting (code + docs) | git.inference.coop | — |
|
||||
| **Surfer** | Static hosting (landing page, thanks page) | inference.coop | [co-op/website](https://git.inference.coop/co-op/website) |
|
||||
| **Listmonk** | Member newsletter (auto-synced from membership) | newsletter.inference.coop | — |
|
||||
| **Vaultwarden** | Password manager (admin credentials) | vault.inference.coop | — |
|
||||
| **SnappyMail** | Webmail | mail.inference.coop | — |
|
||||
|
||||
Open Collective (billing and fiscal sponsorship) is the one external service in
|
||||
the flow — see [opencollective.com/inference-cooperative](https://opencollective.com/inference-cooperative).
|
||||
|
||||
## How membership is wired
|
||||
|
||||
Membership flows through the member portal, which is the single source of truth
|
||||
for "who is a member":
|
||||
|
||||
1. A member contributes on Open Collective.
|
||||
2. Open Collective fires a webhook to the portal, which provisions a Cloudron
|
||||
account, a LiteLLM key, and a welcome email.
|
||||
3. A nightly sweep reconciles against the live Open Collective list, so a missed
|
||||
webhook self-heals within 24 hours.
|
||||
|
||||
The portal database is authoritative for member emails and balances; LiteLLM's
|
||||
team budget is the source of truth for a member's allowance, shared between chat
|
||||
and API.
|
||||
|
||||
## The code
|
||||
|
||||
The cooperative's software is open source, in the [`code`](https://git.inference.coop/code)
|
||||
organization on Gitea:
|
||||
|
||||
- [**code/member-portal**](https://git.inference.coop/code/member-portal) — membership middleware (Python).
|
||||
- [**code/member-dashboard**](https://git.inference.coop/code/member-dashboard) — member-facing usage + API keys (Python).
|
||||
- [**code/admin-panel**](https://git.inference.coop/code/admin-panel) — admin overview (Python).
|
||||
- [**code/litellm**](https://git.inference.coop/code/litellm) — the gateway packaged as a Cloudron app.
|
||||
|
||||
Plus the `co-op` organization:
|
||||
|
||||
- [**co-op/website**](https://git.inference.coop/co-op/website) — landing page and static site.
|
||||
- [**co-op/docs**](https://git.inference.coop/co-op/docs) — this documentation.
|
||||
- [**co-op/design-assets**](https://git.inference.coop/co-op/design-assets) — logos, fonts, brand materials.
|
||||
|
||||
## Models and privacy
|
||||
|
||||
Inference runs through [Tinfoil](https://tinfoil.sh/inference), which provides
|
||||
**architectural** privacy: models run inside hardware enclaves (TEEs), and
|
||||
request/response bodies are encrypted end-to-end (EHBP), so even Tinfoil's own
|
||||
infrastructure can't read them. This is verifiable via remote attestation — not
|
||||
just a policy promise. A local Tinfoil proxy sidecar handles the encryption on
|
||||
our side of the gateway.
|
||||
|
||||
Three models are exposed:
|
||||
|
||||
- **DeepSeek V4.1 Flash** — default, agentic tasks.
|
||||
- **GPT-OSS 120B** — lightweight fallback.
|
||||
- **GLM-5.3 Flash** — fast, efficient.
|
||||
|
||||
The honest caveat: your **chat history** is stored on our server so you can
|
||||
revisit it, and that stored history is not encrypted in a way that prevents us
|
||||
from technically reading it. We commit not to. The full distinction — what's
|
||||
architecturally private versus what's a policy commitment — is in the
|
||||
[Privacy Policy](privacy-policy.md).
|
||||
|
||||
## Security posture (plain language)
|
||||
|
||||
- **Token-gated control plane.** The portal is publicly reachable, but every
|
||||
sensitive surface requires a secret token (admin, broker, webhook). All fail
|
||||
closed — unauthenticated requests get `401`.
|
||||
- **Separated secrets.** Chat, broker, admin, and model-layer secrets are
|
||||
independent and independently rotatable, so a leak has bounded scope.
|
||||
- **Member isolation.** Each API key is scoped to its owner's LiteLLM team and
|
||||
budget; a member can only ever act on their own keys.
|
||||
- **Rate limiting** on control-plane endpoints; the inference path is capped by
|
||||
each member's budget instead.
|
||||
- **No secrets in version control.** Credentials live in Cloudron env vars, never
|
||||
in the repos.
|
||||
- **Accepted trade-offs** (documented, not hidden): member keys are stored in
|
||||
plaintext in the portal database (functionally necessary), and the gateway
|
||||
must stay publicly reachable for API access (its admin UI and docs are
|
||||
disabled; only the `/v1/*` API is exposed).
|
||||
Reference in new issue
Block a user