docs-bot
278c1fda1a
- condense the page to what a member needs: where services run (Infomaniak), what backups mean to them, the .coop domain via domains.coop, mail via Scaleway, the Matrix space and the accounts they may meet - drop operator detail (migration narration, DNS record inventory, scheduled-task internals, admin-only components, internal account/credential identifiers) to co-op/ops-docs; fix a heading glitch where a line began with '#infrastructure' - brand.css: light-mode .md-tabs uses the header's pale wash #e8f5e8 (was the saturated --green-deep) with a warm hairline; dark scheme unchanged
208 lines
10 KiB
Markdown
208 lines
10 KiB
Markdown
# Infrastructure
|
|
|
|
How the Inference Cooperative is built, for members who want to understand it —
|
|
and contributors who want to work on it.
|
|
|
|
## The big picture
|
|
|
|
Members reach the co-op through two front doors: the **chat** (LibreChat) and
|
|
the **API** (the LiteLLM gateway). Both route through a single gateway to
|
|
multiple LLM providers, so members can choose what they value for each task:
|
|
architecturally private inference in **TEE-protected enclaves** (Tinfoil), or
|
|
100% renewable energy (GreenPT, Regolo). A thin layer of custom middleware — the
|
|
member portal — ties Open Collective membership to Cloudron accounts and LiteLLM
|
|
keys.
|
|
|
|
```
|
|
┌─> Tinfoil (TEE) 🔒 private
|
|
Members ──> LibreChat (chat) ──┐ │
|
|
├──> LiteLLM ──┼─> GreenPT (renewable) 🌱 green
|
|
Members ──> your own tools ────┘ gateway │
|
|
(API) (keys, └─> Regolo (renewable) 🌱 green
|
|
metering,
|
|
routing)
|
|
```
|
|
|
|
The whole stack runs on [Cloudron](https://cloudron.io), which handles hosting,
|
|
single sign-on, and per-app isolation.
|
|
|
|
## Where it runs
|
|
|
|
The co-op's member-facing services run on a server the co-op owns and pays for
|
|
directly, at **Infomaniak** — a Swiss sovereign cloud provider whose data centres
|
|
are in Geneva, run on renewable energy, and are certified to ISO 14001, ISO
|
|
50001 and ISO 27001.
|
|
|
|
The co-op's **automation runs on a separate server**, also hosted on Infomaniak
|
|
and deliberately kept apart from the member-facing applications. The steward bot
|
|
— the automation behind the digests and the newsletter drafts — and the co-op's
|
|
other agents live there. Keeping them on their own host means an agent problem
|
|
stays an agent problem: it cannot reach the chat, the gateway or member data.
|
|
|
|
### Backups
|
|
|
|
Cloudron writes **encrypted** backups to Infomaniak's object storage in
|
|
Switzerland every night and keeps the last **seven days**, so a failed upgrade
|
|
or a lost server can be restored. The automation host backs up its own data to
|
|
the same object storage.
|
|
|
|
### Domain and DNS
|
|
|
|
The co-op's domain is **inference.coop** — a `.coop` domain. The co-op registers
|
|
and manages its domains with **domains.coop**, the domain provider run by the
|
|
cooperative movement, and holds them in the co-op's own name rather than in any
|
|
individual's. DNS itself is deliberately simple, and the co-op's own nameservers
|
|
answer it.
|
|
|
|
## Components
|
|
|
|
| Component | Purpose | URL |
|
|
|-----------|---------|-----|
|
|
| **LibreChat** | Member-facing chat (models, web search, image generation, voice) | chat.inference.coop |
|
|
| **Open WebUI** | Legacy chat instance — read-only reference (old history and file attachments) | chat-legacy.inference.coop |
|
|
| **LiteLLM** | AI gateway — keys, metering and model routing, for chat and API | gateway.inference.coop |
|
|
| **Member portal** | Membership middleware — onboarding, key provisioning, Loomio sync | portal.inference.coop |
|
|
| **Member dashboard** | Your usage view and API keys | dashboard.inference.coop |
|
|
| **Loomio** | Member governance | forum.inference.coop |
|
|
| **Matrix (Synapse)** | Member chat server, federated, SSO login via Cloudron | matrix.inference.coop |
|
|
| **Gitea** | Git hosting for the co-op's code and this documentation | git.inference.coop |
|
|
| **Listmonk** | Member newsletter (synced from membership) | newsletter.inference.coop |
|
|
| **SnappyMail** | Webmail | email.inference.coop |
|
|
|
|
The co-op's software behind all of these is open source — see [The
|
|
code](#the-code) below for where each piece lives.
|
|
|
|
Open Collective (billing and fiscal sponsorship) is the one external service in
|
|
the flow — see
|
|
[opencollective.com/inference-cooperative](https://opencollective.com/inference-cooperative).
|
|
|
|
## Mail
|
|
|
|
The co-op runs its own `@inference.coop` mail. **Incoming** mail arrives directly
|
|
on the co-op's server, and webmail and mail apps reach it over the usual
|
|
encrypted ports (IMAP and SMTP over TLS). **Outgoing** mail is sent through
|
|
**Scaleway**, an EU-based email service, authenticated for our domain.
|
|
|
|
You can reach your mailbox through webmail at email.inference.coop, or add it to
|
|
a standard mail app on your phone or computer. If mail to your `@inference.coop`
|
|
address stops arriving, that is a fault on our side to fix.
|
|
|
|
## Matrix
|
|
|
|
Members get a federated Matrix account on the co-op's self-hosted Synapse
|
|
(matrix.inference.coop, user IDs `@username:inference.coop`), signing in with the
|
|
same Cloudron SSO as the chat. The co-op invites every active member to the
|
|
private **Inference Cooperative** space (with a **General** room) automatically,
|
|
so new members land there as they join. Federation is on, so you can also join
|
|
from any homeserver you already use.
|
|
|
|
### Accounts you may meet
|
|
|
|
A few accounts in the space are automation rather than people:
|
|
|
|
- **`@inferencebot` — the steward.** It posts the digests, newsletter notes and
|
|
reminders. It holds no more power than posting needs.
|
|
- **`@inference-admin`** — the administrator of record for the co-op's rooms
|
|
(#general, #community, #design, #skillshare, #updates and #infrastructure) and
|
|
their membership. It **never speaks in a member room** — a message from it is
|
|
a mistake, not a feature.
|
|
|
|
The house rule: **power belongs to `@inference-admin`, voice belongs to
|
|
`@inferencebot`.** Please don't create an account whose name differs from an
|
|
existing one by a character or two — spotting lookalikes is exactly what this
|
|
convention is for.
|
|
|
|
## Membership
|
|
|
|
Membership runs through the member portal, the single source of truth for who is
|
|
a member:
|
|
|
|
1. A member contributes on Open Collective.
|
|
2. Open Collective notifies the portal, which provisions a Cloudron account, a
|
|
LiteLLM key, and a welcome email.
|
|
3. A nightly reconciliation against the live Open Collective list means a missed
|
|
notification self-heals within a day.
|
|
|
|
You may see **"Inference Co-op Bot"** listed as an admin of the collective on
|
|
Open Collective. It is infrastructure, not a person or a member: a dedicated
|
|
account the portal uses, **read-only**, to check the member roster and tiers. It
|
|
cannot move money, issue refunds or change the collective — payments always come
|
|
from a human's own contribution, and automation only reads the result. Its code
|
|
is open source in
|
|
[code/member-portal](https://git.inference.coop/code/member-portal), and a
|
|
collective admin can revoke its access at any time.
|
|
|
|
## The code
|
|
|
|
The co-op's software is open source, in the [`code`](https://git.inference.coop/code)
|
|
and [`co-op`](https://git.inference.coop/co-op) organizations on Gitea:
|
|
|
|
- [**code/member-portal**](https://git.inference.coop/code/member-portal) — membership middleware (Python).
|
|
- [**code/member-dashboard**](https://git.inference.coop/code/member-dashboard) — member dashboard: usage and API keys.
|
|
- [**code/litellm**](https://git.inference.coop/code/litellm) — the gateway packaged as a Cloudron app.
|
|
- [**co-op/website**](https://git.inference.coop/co-op/website) — landing page and static site.
|
|
- [**co-op/docs**](https://git.inference.coop/co-op/docs) — this documentation.
|
|
|
|
The chat, portal and other apps live in `code/` and `co-op/` too; everything is
|
|
browsable at [git.inference.coop](https://git.inference.coop).
|
|
|
|
## Contributing (the Git is yours, too)
|
|
|
|
Everything the co-op runs is open source, so as a member you can browse all of
|
|
it and propose changes.
|
|
|
|
**Who can sign in.** Git access is tied to your membership: you log into the Git
|
|
with the same single sign-on you use for the chat. Members are in automatically
|
|
(through the `members` group); when a membership lapses, Git access lapses with
|
|
it.
|
|
|
|
**How to contribute.** Anyone can read the repos without an account. To make a
|
|
change, fork the repo, edit in your fork, and open a **pull request**:
|
|
|
|
1. Log into [git.inference.coop](https://git.inference.coop) with your co-op account.
|
|
2. **Fork** the repo you want to change (e.g. `co-op/docs`).
|
|
3. Make your edits in your fork and commit them.
|
|
4. Open a **pull request** back to the original repo, describing your change.
|
|
|
|
**All changes are reviewed.** Nothing merges directly into `main` — every change
|
|
comes in as a pull request an admin reviews and approves first. You don't need
|
|
to be able to create repos or organizations to contribute; you just fork, edit
|
|
and ask for review.
|
|
|
|
Questions? Write to info@inference.coop.
|
|
|
|
## Models and providers
|
|
|
|
The co-op serves models from multiple providers — Tinfoil for architectural
|
|
privacy, GreenPT and Regolo for renewable energy. See the [model list](models.md),
|
|
the single source of truth for which models are available and how they're named.
|
|
|
|
## Security posture (plain language)
|
|
|
|
- **Member isolation.** Each API key is scoped to its owner's team and budget; a
|
|
member can only ever act on their own keys.
|
|
- **Separated secrets.** Chat, portal, admin and model-layer secrets are
|
|
independent and independently rotatable, so a leak has bounded scope.
|
|
- **No secrets in version control.** Credentials live in the hosting platform's
|
|
environment, never in the repos.
|
|
- **Accepted trade-offs** (documented, not hidden): member keys are stored in
|
|
the portal database so the co-op can serve them, and the API gateway must stay
|
|
publicly reachable. Its admin surfaces are disabled; only the public API is
|
|
exposed.
|
|
|
|
## For operators
|
|
|
|
Maintaining the co-op's infrastructure — token rotation, app configuration and
|
|
troubleshooting — is documented in the git-only
|
|
[`co-op/ops-docs`](https://git.inference.coop/co-op/ops-docs) repository, which is
|
|
deliberately not published on this site. It is the home for the operator runbook,
|
|
the maintenance notes, incident write-ups, and the record of past server
|
|
migrations. Anyone may read it; operators are the audience.
|
|
|
|
## Incident log
|
|
|
|
Write-ups for outages live in the operator docs, in
|
|
[`incidents/`](https://git.inference.coop/co-op/ops-docs/src/branch/main/incidents).
|
|
One so far:
|
|
[2026-10-02 — gateway outage after a security-upgrade rebuild](https://git.inference.coop/co-op/ops-docs/src/branch/main/incidents/2026-10-02-gateway-cve-upgrade.md).
|