docs: reorganise into Guides / Operations / Governance
- Guides/: getting started, guest access, models, API access, energy & carbon - Operations/: agentic systems (was 'working with the agent'), infrastructure, roles, steward policy - Governance/: unchanged, now with its overview page Lateral links and the homepage updated; redirects keep the old root URLs alive.
This commit is contained in:
1 parent
a6142d68f2
commit
ac8cac02c6
13 files changed
+66
-29
No files matched your search
@@ -0,0 +1,200 @@
|
||||
# Infrastructure
|
||||
|
||||
How the Inference Cooperative is built, for members who want to understand it —
|
||||
and contributors who want to work on it.
|
||||
|
||||
## The big picture
|
||||
|
||||
Members reach the co-op through two front doors: the **chat** (LibreChat) and
|
||||
the **API** (the LiteLLM gateway). Both route through a single gateway to
|
||||
multiple LLM providers, so members can choose what they value for each task:
|
||||
architecturally private inference in **TEE-protected enclaves** (Tinfoil), or
|
||||
100% renewable energy (GreenPT, Regolo). A thin layer of custom middleware — the
|
||||
member portal — ties Open Collective membership to Cloudron accounts and LiteLLM
|
||||
keys.
|
||||
|
||||
```
|
||||
┌─> Tinfoil (TEE) 🔒 private
|
||||
Members ──> LibreChat (chat) ──┐ │
|
||||
├──> LiteLLM ──┼─> GreenPT (renewable) 🌱 green
|
||||
Members ──> your own tools ────┘ gateway │
|
||||
(API) (keys, └─> Regolo (renewable) 🌱 green
|
||||
metering,
|
||||
routing)
|
||||
```
|
||||
|
||||
The whole stack runs on [Cloudron](https://cloudron.io), which handles hosting,
|
||||
single sign-on, and per-app isolation.
|
||||
|
||||
## Where it runs
|
||||
|
||||
The co-op's member-facing services run on a server the co-op owns and pays for
|
||||
directly, at **Infomaniak** — a Swiss sovereign cloud provider whose data centres
|
||||
are in Geneva, run on renewable energy, and are certified to ISO 14001, ISO
|
||||
50001 and ISO 27001.
|
||||
|
||||
The co-op's **automation runs on a separate server**, also hosted on Infomaniak
|
||||
and deliberately kept apart from the member-facing applications. The steward bot
|
||||
— the automation behind the digests and the newsletter drafts — and the co-op's
|
||||
other agents live there. Keeping them on their own host means an agent problem
|
||||
stays an agent problem: it cannot reach the chat, the gateway or member data.
|
||||
|
||||
### Backups
|
||||
|
||||
Cloudron writes **encrypted** backups to Infomaniak's object storage in
|
||||
Switzerland every night and keeps the last **seven days**, so a failed upgrade
|
||||
or a lost server can be restored. The automation host backs up its own data to
|
||||
the same object storage.
|
||||
|
||||
### Domain and DNS
|
||||
|
||||
The co-op's domain is **inference.coop** — a `.coop` domain. The co-op registers
|
||||
and manages its domains with **domains.coop**, the domain provider run by the
|
||||
cooperative movement, and holds them in the co-op's own name rather than in any
|
||||
individual's. DNS itself is deliberately simple, and the co-op's own nameservers
|
||||
answer it.
|
||||
|
||||
## Components
|
||||
|
||||
| Component | Purpose | URL |
|
||||
|-----------|---------|-----|
|
||||
| **LibreChat** | Member-facing chat (models, web search, image generation, voice) | chat.inference.coop |
|
||||
| **Open WebUI** | Legacy chat instance — read-only reference (old history and file attachments) | chat-legacy.inference.coop |
|
||||
| **LiteLLM** | AI gateway — keys, metering and model routing, for chat and API | gateway.inference.coop |
|
||||
| **Member portal** | Membership middleware — onboarding, key provisioning, Loomio sync | portal.inference.coop |
|
||||
| **Member dashboard** | Your usage view and API keys | dashboard.inference.coop |
|
||||
| **Loomio** | Member governance | forum.inference.coop |
|
||||
| **Matrix (Synapse)** | Member chat server, federated, SSO login via Cloudron | matrix.inference.coop |
|
||||
| **Gitea** | Git hosting for the co-op's code and this documentation | git.inference.coop |
|
||||
| **Listmonk** | Member newsletter (synced from membership) | newsletter.inference.coop |
|
||||
| **SnappyMail** | Webmail | email.inference.coop |
|
||||
|
||||
The co-op's software behind all of these is open source — see [The
|
||||
code](#the-code) below for where each piece lives.
|
||||
|
||||
Open Collective (billing and fiscal sponsorship) is the one external service in
|
||||
the flow — see
|
||||
[opencollective.com/inference-cooperative](https://opencollective.com/inference-cooperative).
|
||||
|
||||
## Mail
|
||||
|
||||
The co-op runs its own `@inference.coop` mail. **Incoming** mail arrives directly
|
||||
on the co-op's server, and webmail and mail apps reach it over the usual
|
||||
encrypted ports (IMAP and SMTP over TLS). **Outgoing** mail is sent through
|
||||
**Scaleway**, an EU-based email service, authenticated for our domain.
|
||||
|
||||
You can reach your mailbox through webmail at email.inference.coop, or add it to
|
||||
a standard mail app on your phone or computer. If mail to your `@inference.coop`
|
||||
address stops arriving, that is a fault on our side to fix.
|
||||
|
||||
## Matrix
|
||||
|
||||
Members get a federated Matrix account on the co-op's self-hosted Synapse
|
||||
(matrix.inference.coop, user IDs `@username:inference.coop`), signing in with the
|
||||
same Cloudron SSO as the chat. The co-op invites every active member to the
|
||||
private **Inference Cooperative** space (with a **General** room) automatically,
|
||||
so new members land there as they join. Federation is on, so you can also join
|
||||
from any homeserver you already use.
|
||||
|
||||
### Accounts you may meet
|
||||
|
||||
A few accounts in the space are automation rather than people:
|
||||
|
||||
- **`@inferencebot` — the steward.** It posts the digests, newsletter notes and
|
||||
reminders. It holds no more power than posting needs.
|
||||
- **`@inference-admin`** — the administrator of record for the co-op's rooms
|
||||
(#general, #community, #design, #skillshare, #updates and #infrastructure) and
|
||||
their membership. It **never speaks in a member room** — a message from it is
|
||||
a mistake, not a feature.
|
||||
|
||||
The house rule: **power belongs to `@inference-admin`, voice belongs to
|
||||
`@inferencebot`.** Please don't create an account whose name differs from an
|
||||
existing one by a character or two — spotting lookalikes is exactly what this
|
||||
convention is for.
|
||||
|
||||
## Membership
|
||||
|
||||
Membership runs through the member portal, the single source of truth for who is
|
||||
a member:
|
||||
|
||||
1. A member contributes on Open Collective.
|
||||
2. Open Collective notifies the portal, which provisions a Cloudron account, a
|
||||
LiteLLM key, and a welcome email.
|
||||
3. A nightly reconciliation against the live Open Collective list means a missed
|
||||
notification self-heals within a day.
|
||||
|
||||
You may see **"Inference Co-op Bot"** listed as an admin of the collective on
|
||||
Open Collective. It is infrastructure, not a person or a member: a dedicated
|
||||
account the portal uses, **read-only**, to check the member roster and tiers. It
|
||||
cannot move money, issue refunds or change the collective — payments always come
|
||||
from a human's own contribution, and automation only reads the result. Its code
|
||||
is open source in
|
||||
[code/member-portal](https://git.inference.coop/code/member-portal), and a
|
||||
collective admin can revoke its access at any time.
|
||||
|
||||
## The code
|
||||
|
||||
The co-op's software is open source, in the [`code`](https://git.inference.coop/code)
|
||||
and [`co-op`](https://git.inference.coop/co-op) organizations on Gitea:
|
||||
|
||||
- [**code/member-portal**](https://git.inference.coop/code/member-portal) — membership middleware (Python).
|
||||
- [**code/member-dashboard**](https://git.inference.coop/code/member-dashboard) — member dashboard: usage and API keys.
|
||||
- [**code/litellm**](https://git.inference.coop/code/litellm) — the gateway packaged as a Cloudron app.
|
||||
- [**co-op/website**](https://git.inference.coop/co-op/website) — landing page and static site.
|
||||
- [**co-op/docs**](https://git.inference.coop/co-op/docs) — this documentation.
|
||||
|
||||
The chat, portal and other apps live in `code/` and `co-op/` too; everything is
|
||||
browsable at [git.inference.coop](https://git.inference.coop). Everything the co-op runs
|
||||
is open source, so as a member you can browse all of
|
||||
it and propose changes.
|
||||
|
||||
**Who can sign in.** Git access is tied to your membership: you log into the Git
|
||||
with the same single sign-on you use for the chat. Members are in automatically
|
||||
(through the `members` group); when a membership lapses, Git access lapses with
|
||||
it.
|
||||
|
||||
**How to contribute.** Anyone can read the repos without an account. To make a
|
||||
change, fork the repo, edit in your fork, and open a **pull request**:
|
||||
|
||||
1. Log into [git.inference.coop](https://git.inference.coop) with your co-op account.
|
||||
2. **Fork** the repo you want to change (e.g. `co-op/docs`).
|
||||
3. Make your edits in your fork and commit them.
|
||||
4. Open a **pull request** back to the original repo, describing your change.
|
||||
|
||||
**All changes are reviewed.** Nothing merges directly into `main` — every change
|
||||
comes in as a pull request an admin reviews and approves first. You don't need
|
||||
to be able to create repos or organizations to contribute; you just fork, edit
|
||||
and ask for review.
|
||||
|
||||
## Models and providers
|
||||
|
||||
The co-op serves models from multiple providers — Tinfoil for architectural
|
||||
privacy, GreenPT and Regolo for renewable energy. See the [model list](../guides/models.md),
|
||||
the single source of truth for which models are available and how they're named.
|
||||
|
||||
## Security posture (plain language)
|
||||
|
||||
- **Member isolation.** Each API key is scoped to its owner's team and budget; a
|
||||
member can only ever act on their own keys.
|
||||
- **Separated secrets.** Chat, portal, admin and model-layer secrets are
|
||||
independent and independently rotatable, so a leak has bounded scope.
|
||||
- **No secrets in version control.** Credentials live in the hosting platform's
|
||||
environment, never in the repos.
|
||||
- **Accepted trade-offs** (documented, not hidden): member keys are stored in
|
||||
the portal database so the co-op can serve them, and the API gateway must stay
|
||||
publicly reachable. Its admin surfaces are disabled; only the public API is
|
||||
exposed.
|
||||
|
||||
## For operators
|
||||
|
||||
Maintaining the co-op's infrastructure — token rotation, app configuration and
|
||||
troubleshooting — is documented in the git-only
|
||||
[`co-op/ops-docs`](https://git.inference.coop/co-op/ops-docs) repository, which is
|
||||
deliberately not published on this site. It is the home for the operator runbook,
|
||||
the maintenance notes, incident write-ups, and the record of past server
|
||||
migrations. Anyone may read it; operators are the audience.
|
||||
|
||||
## Incident log
|
||||
|
||||
Write-ups for outages live in the operator docs, in
|
||||
[`incidents/`](https://git.inference.coop/co-op/ops-docs/src/branch/main/incidents).
|
||||
Reference in new issue
Block a user