co-op/docs
4
0 Fork 3
Code Issues Pull Requests 1 Actions Packages Projects Releases Wiki Activity

infrastructure: make the page member-facing; lighten the light-mode menu-bar stripe

Browse Source 
- condense the page to what a member needs: where services run (Infomaniak),
  what backups mean to them, the .coop domain via domains.coop, mail via
  Scaleway, the Matrix space and the accounts they may meet
- drop operator detail (migration narration, DNS record inventory, scheduled-task
  internals, admin-only components, internal account/credential identifiers) to
  co-op/ops-docs; fix a heading glitch where a line began with '#infrastructure'
- brand.css: light-mode .md-tabs uses the header's pale wash #e8f5e8 (was the
  saturated --green-deep) with a warm hairline; dark scheme unchanged
This commit is contained in:
docs-bot committed 2026-11-04 21:29:06 +00:00
1 parent 869d16cc65
commit 278c1fda1a
2 files changed
+122 -223
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

infrastructure.md
+114 -221
View File
@@ -7,11 +7,11 @@ and contributors who want to work on it.
Members reach the co-op through two front doors: the **chat** (LibreChat) and Members reach the co-op through two front doors: the **chat** (LibreChat) and
the **API** (the LiteLLM gateway). Both route through a single gateway to the **API** (the LiteLLM gateway). Both route through a single gateway to
multiple LLM providers — so members choose what they value for each task: multiple LLM providers, so members can choose what they value for each task:
architecturally private inference in **TEE-protected enclaves** (Tinfoil), architecturally private inference in **TEE-protected enclaves** (Tinfoil), or
100% renewable energy (GreenPT, Regolo). A thin layer of custom middleware — 100% renewable energy (GreenPT, Regolo). A thin layer of custom middleware — the
the member portal — ties Open member portal — ties Open Collective membership to Cloudron accounts and LiteLLM
Collective membership to Cloudron accounts and LiteLLM keys. keys.
``` ```
┌─> Tinfoil (TEE) 🔒 private ┌─> Tinfoil (TEE) 🔒 private
@@ -28,228 +28,133 @@ single sign-on, and per-app isolation.
## Where it runs ## Where it runs
The co-op's services run on a server the co-op owns and pays for directly, at The co-op's member-facing services run on a server the co-op owns and pays for
**Infomaniak** — a Swiss sovereign cloud provider whose data centres are in directly, at **Infomaniak** — a Swiss sovereign cloud provider whose data centres
Geneva, run on renewable energy, and certified to ISO 14001, ISO 50001 and are in Geneva, run on renewable energy, and are certified to ISO 14001, ISO
ISO 27001. The stack moved there from a personal Hostinger account on 50001 and ISO 27001.
**2026-11-02**; nothing about how members use the co-op changed in the move.
One thing runs elsewhere: the co-op's **steward bot** — the automation behind the The co-op's **automation runs on a separate server**, also hosted on Infomaniak
digests and the newsletter drafts — runs on a separate machine, not on the and deliberately kept apart from the member-facing applications. The steward bot
Cloudron server, and was not part of the migration. — the automation behind the digests and the newsletter drafts — and the co-op's
other agents live there. Keeping them on their own host means an agent problem
### DNS stays an agent problem: it cannot reach the chat, the gateway or member data.
`inference.coop` is managed by hand at **domains.coop** (nameservers
`ns1/ns2.yoursrs.com`). The zone is deliberately small — **eleven records**. Most
of the co-op hangs off just a few of them: the apex (`inference.coop`), `www`,
and a wildcard (`*.inference.coop`) point at the server, so each application
hostname — chat, portal, dashboard, gateway, git, newsletter, vault and the rest
— is served by the wildcard rather than having a record of its own. The DNS
change sheet listing all eleven records is public, in
[`co-op/runbooks`](https://git.inference.coop/co-op/runbooks).
### Backups ### Backups
Cloudron writes **encrypted** backups to **Infomaniak's object storage** (the Cloudron writes **encrypted** backups to Infomaniak's object storage in
"Infomaniak Encrypted S3" site, in Switzerland) every night, and keeps the last Switzerland every night and keeps the last **seven days**, so a failed upgrade
**seven days**. The steward-bot host backs up its own selected home directories or a lost server can be restored. The automation host backs up its own data to
to the same object storage. the same object storage.
### Domain and DNS
The co-op's domain is **inference.coop** — a `.coop` domain. The co-op registers
and manages its domains with **domains.coop**, the domain provider run by the
cooperative movement, and holds them in the co-op's own name rather than in any
individual's. DNS itself is deliberately simple, and the co-op's own nameservers
answer it.
## Components ## Components
| Component | Purpose | URL | Code | | Component | Purpose | URL |
|-----------|---------|-----|------| |-----------|---------|-----|
| **LibreChat** | Member-facing chat (models, web search, image generation, voice) | chat.inference.coop | [co-op/librechat-custom](https://git.inference.coop/co-op/librechat-custom) | | **LibreChat** | Member-facing chat (models, web search, image generation, voice) | chat.inference.coop |
| **Open WebUI** | Legacy chat instance — read-only reference (old history + file attachments) | chat-legacy.inference.coop | — | | **Open WebUI** | Legacy chat instance — read-only reference (old history and file attachments) | chat-legacy.inference.coop |
| **LiteLLM** | AI gateway — keys, metering, model routing | gateway.inference.coop | [code/litellm](https://git.inference.coop/code/litellm) | | **LiteLLM** | AI gateway — keys, metering and model routing, for chat and API | gateway.inference.coop |
| **Member Portal** | Membership middleware — onboarding, key provisioning, Loomio sync | portal.inference.coop | [code/member-portal](https://git.inference.coop/code/member-portal) | | **Member portal** | Membership middleware — onboarding, key provisioning, Loomio sync | portal.inference.coop |
| **Member Dashboard** | Member-facing usage view + API key management | dashboard.inference.coop | [code/member-dashboard](https://git.inference.coop/code/member-dashboard) | | **Member dashboard** | Your usage view and API keys | dashboard.inference.coop |
| **Admin Panel** | Admin-only member overview, spend, balance management | panel.inference.coop | [code/admin-panel](https://git.inference.coop/code/admin-panel) | | **Loomio** | Member governance | forum.inference.coop |
| **SearXNG** | Self-hosted web search (feeds the chat's search) | search.inference.coop | — | | **Matrix (Synapse)** | Member chat server, federated, SSO login via Cloudron | matrix.inference.coop |
| **Loomio** | Member governance | forum.inference.coop | — | | **Gitea** | Git hosting for the co-op's code and this documentation | git.inference.coop |
| **Matrix (Synapse)** | Member chat server (federated; SSO login via Cloudron) | matrix.inference.coop | — | | **Listmonk** | Member newsletter (synced from membership) | newsletter.inference.coop |
| **Gitea** | Git hosting (code + docs) | git.inference.coop | — | | **SnappyMail** | Webmail | email.inference.coop |
| **Surfer** | Static hosting (landing page, thanks page) | inference.coop | [co-op/website](https://git.inference.coop/co-op/website) |
| **Listmonk** | Member newsletter (auto-synced from membership) | newsletter.inference.coop | — | The co-op's software behind all of these is open source — see [The
| **Vaultwarden** | Password manager (admin credentials) | vault.inference.coop | — | code](#the-code) below for where each piece lives.
| **SnappyMail** | Webmail | email.inference.coop | — |
Open Collective (billing and fiscal sponsorship) is the one external service in Open Collective (billing and fiscal sponsorship) is the one external service in
the flow — see [opencollective.com/inference-cooperative](https://opencollective.com/inference-cooperative). the flow — see
[opencollective.com/inference-cooperative](https://opencollective.com/inference-cooperative).
## Mail ## Mail
The `inference.coop` mail domain is the co-op's, and the mail itself runs on the The co-op runs its own `@inference.coop` mail. **Incoming** mail arrives directly
co-op's Cloudron server. on the co-op's server, and webmail and mail apps reach it over the usual
encrypted ports (IMAP and SMTP over TLS). **Outgoing** mail is sent through
**Scaleway**, an EU-based email service, authenticated for our domain.
**Incoming** mail is received directly by that server — the domain's MX record You can reach your mailbox through webmail at email.inference.coop, or add it to
points at it, it accepts mail on the standard SMTP port, and webmail and mail a standard mail app on your phone or computer. If mail to your `@inference.coop`
apps reach it over the usual encrypted ports (IMAP and SMTP over TLS). If mail address stops arriving, that is a fault on our side to fix.
to your `@inference.coop` address stops arriving, that is a fault on our side to
fix, not a limitation of how the co-op is set up.
**Outgoing** mail travels by two different paths, on purpose:
- The **newsletter** (Listmonk) sends through **Scaleway Transactional Email**,
an EU-based SMTP relay, rather than from our own server. Our domain is
authenticated to that relay with its own DKIM key, and a real message has been
checked end to end: SPF, DKIM and DMARC all pass under a `p=reject` policy —
so anything claiming to be from us that the relay hasn't authenticated is
rejected outright rather than quietly filed away.
- The **member portal's** transactional email — welcome messages, budget
notices, the confirmation the nightly membership sweep sends — still sends from
the co-op's own server, through the mail service Cloudron runs for us.
Both paths send as **Inference Cooperative `<info@inference.coop>`**.
Why split them: sending the newsletter through a relay lets a provider with
established sending IPs carry a list this size, instead of leaning on one
freshly-migrated server's reputation. Receiving we keep in-house, on our own
hardware.
## Matrix ## Matrix
Members get a federated Matrix account on our self-hosted Synapse Members get a federated Matrix account on the co-op's self-hosted Synapse
(matrix.inference.coop, user IDs `@username:inference.coop`) using their (matrix.inference.coop, user IDs `@username:inference.coop`), signing in with the
Cloudron SSO login. The portal invites every active member — and Nathan's same Cloudron SSO as the chat. The co-op invites every active member to the
federated account — to the private **Inference Cooperative** Space (with a private **Inference Cooperative** space (with a **General** room) automatically,
**General** room) on every membership sync, so new members land in the space so new members land there as they join. Federation is on, so you can also join
automatically. Invites honor rate limits; external/federated members can be from any homeserver you already use.
added via the portal's `MATRIX_EXTRA_MXIDS` env.
Federation is enabled (delegated via `.well-known/matrix/server` on the bare ### Accounts you may meet
domain, port 443), so members can also join from any homeserver they already
use.
### Which account is which A few accounts in the space are automation rather than people:
Several automated accounts touch this homeserver, and two of them once differed - **`@inferencebot` — the steward.** It posts the digests, newsletter notes and
by a single character — which caused real confusion, including a bot appearing to reminders. It holds no more power than posting needs.
announce itself. The distinction is deliberate: - **`@inference-admin`** — the administrator of record for the co-op's rooms
(#general, #community, #design, #skillshare, #updates and #infrastructure) and
their membership. It **never speaks in a member room** — a message from it is
a mistake, not a feature.
- **`@inferencebot:inference.coop` — the steward.** The bot members actually meet: The house rule: **power belongs to `@inference-admin`, voice belongs to
it posts digests, newsletter notes and reminders. It is *not* a Synapse server `@inferencebot`.** Please don't create an account whose name differs from an
admin, and holds no room power beyond what it needs to post. existing one by a character or two — spotting lookalikes is exactly what this
- **`@inference-admin:inference.coop` — the room and server administrator.** A convention is for.
Synapse server admin, used for provisioning rooms, managing membership, and
retiring rooms. It **never speaks in a member room**; a message from it is a
mistake, not a feature.
- **`@infrastructure-bot:inference.coop` — the infrastructure working group's bot.**
Answers infrastructure questions in **#infrastructure** when mentioned.
- **`@ops_audit:inference.coop`** — a server-admin account created for the October
2026 security review. Retained only until its tokens are revoked.
**House rule: power belongs to `@inference-admin`, voice belongs to `@inferencebot`.** ## Membership
Please don't create another account whose name differs from an existing one by a
character or two; that is the specific confusion this section exists to undo.
**`@inference-bot:inference.coop` is retired (2026-10-29).** It was the original Membership runs through the member portal, the single source of truth for who is
room-administration account (it created the space and most of the rooms) and held a a member:
Synapse server-admin flag. Its room power went to `@inference-admin`, which is now
administrator of record for #general, #community, #design, #skillshare, #updates and
#infrastructure, **and holds the portal's member-invite credential**
(`MATRIX_BOT_TOKEN`) under its own token. The old account's admin flag was dropped,
it was deactivated, and every device and token was deleted. Nothing should use it
again.
Two loose ends, recorded so nobody re-investigates them from scratch: it remains
joined to two rooms it cannot leave (they reject every write — `500` on leave), and
those rooms are also why they can't be deleted; and the old account's *name* is still
the tempting one, so check `whoami` rather than the name before trusting any account
that looks like it.
## How the apps run themselves
The portal and gateway are self-maintaining: their recurring housekeeping is
built into the apps as scheduled tasks, not run from any member's or operator's
personal machine.
- **Member portal** — two scheduled jobs (Cloudron scheduler):
- *Membership sweep* (nightly, 3am) — reconciles members against the live
Open Collective list: provisions new members who signed up while a webhook
was missed, deactivates lapsed members, and syncs Loomio + Matrix
membership. A missed webhook self-heals within 24 hours.
- *Docs knowledge refresh* (nightly, 3:30am) — pulls this documentation from
the public Git repo and updates the shared knowledge base behind the chat's
models, so answers grounded in the docs stay current. Changes to the docs
appear in the chat within a day, no manual step.
- **Gateway** — an *audio keep-alive* (every 10 minutes) sends a tiny text-to-
speech ping to the private enclave so voice mode responds quickly even after
long quiet periods, without compromising the end-to-end encryption model.
Scheduled tasks are declared in each app's `CloudronManifest.json` (see the
`code/` repos) and survive redeploys. If a task fails, it shows up in the
app's logs on the admin panel; the portal also exposes a manual trigger for
the membership sweep.
## How membership is wired
Membership flows through the member portal, which is the single source of truth
for "who is a member":
1. A member contributes on Open Collective. 1. A member contributes on Open Collective.
2. Open Collective fires a webhook to the portal, which provisions a Cloudron 2. Open Collective notifies the portal, which provisions a Cloudron account, a
account, a LiteLLM key, and a welcome email. LiteLLM key, and a welcome email.
3. A nightly sweep reconciles against the live Open Collective list, so a missed 3. A nightly reconciliation against the live Open Collective list means a missed
webhook self-heals within 24 hours. notification self-heals within a day.
The portal database is authoritative for member emails and balances; LiteLLM's You may see **"Inference Co-op Bot"** listed as an admin of the collective on
team budget is the source of truth for a member's allowance, shared between chat Open Collective. It is infrastructure, not a person or a member: a dedicated
and API. account the portal uses, **read-only**, to check the member roster and tiers. It
cannot move money, issue refunds or change the collective — payments always come
### The bot account on Open Collective from a human's own contribution, and automation only reads the result. Its code
is open source in
You may notice **"Inference Co-op Bot"** listed as an admin of the collective on [code/member-portal](https://git.inference.coop/code/member-portal), and a
Open Collective. This is infrastructure, not a member or a person: collective admin can revoke its access at any time.
- **What it is:** a dedicated Open Collective account owned by the co-op. It
holds a **Personal Token** that the member portal uses to read membership
data — the roster, tiers (needed to route credit-pack purchases correctly),
and contributor emails (including guests, which only admins can see).
- **What it can do:** read member and tier data via Open Collective's API.
It cannot move money, issue refunds, or change the collective — Open
Collective restricts those to dashboard actions by human admins. Its API
access was verified read-mostly during the October 2026 security review.
- **Why it has admin:** reading guest contributor emails requires admin on
Open Collective; without it, membership reconciliation and credit-pack
routing can't function.
- **Who supervises it:** all of its code is open source in
[code/member-portal](https://git.inference.coop/code/member-portal) — every
API call it makes is documented there. Nathan Schneider (collective admin)
oversees its use and can revoke or rotate its token at any time from the
Open Collective dashboard.
Nothing on Open Collective is charged, transferred, or refunded by automation:
payments always come from a human's own contribution action, and any automated
system only *reads* the resulting records.
## The code ## The code
The cooperative's software is open source, in the [`code`](https://git.inference.coop/code) The co-op's software is open source, in the [`code`](https://git.inference.coop/code)
organization on Gitea: and [`co-op`](https://git.inference.coop/co-op) organizations on Gitea:
- [**code/member-portal**](https://git.inference.coop/code/member-portal) — membership middleware (Python). - [**code/member-portal**](https://git.inference.coop/code/member-portal) — membership middleware (Python).
- [**code/member-dashboard**](https://git.inference.coop/code/member-dashboard) — member-facing usage + API keys (Python). - [**code/member-dashboard**](https://git.inference.coop/code/member-dashboard) — member dashboard: usage and API keys.
- [**code/admin-panel**](https://git.inference.coop/code/admin-panel) — admin overview (Python).
- [**code/litellm**](https://git.inference.coop/code/litellm) — the gateway packaged as a Cloudron app. - [**code/litellm**](https://git.inference.coop/code/litellm) — the gateway packaged as a Cloudron app.
Plus the `co-op` organization:
- [**co-op/website**](https://git.inference.coop/co-op/website) — landing page and static site. - [**co-op/website**](https://git.inference.coop/co-op/website) — landing page and static site.
- [**co-op/docs**](https://git.inference.coop/co-op/docs) — this documentation. - [**co-op/docs**](https://git.inference.coop/co-op/docs) — this documentation.
- [**co-op/design-assets**](https://git.inference.coop/co-op/design-assets) — logos, fonts, brand materials.
The chat, portal and other apps live in `code/` and `co-op/` too; everything is
browsable at [git.inference.coop](https://git.inference.coop).
## Contributing (the Git is yours, too) ## Contributing (the Git is yours, too)
Everything the co-op runs is open source and lives at Everything the co-op runs is open source, so as a member you can browse all of
[git.inference.coop](https://git.inference.coop) — the website, this it and propose changes.
documentation, the member portal, the dashboard, the admin panel, and the
gateway. As a member, you can browse all of it and propose changes.
**Who can sign in.** Git access is tied to your membership: you log into the Git **Who can sign in.** Git access is tied to your membership: you log into the Git
with the same single sign-on you use for the chat. Members are in automatically with the same single sign-on you use for the chat. Members are in automatically
(through the `members` group); when a membership lapses, Git access lapses with it. (through the `members` group); when a membership lapses, Git access lapses with
it.
**How to contribute.** Anyone can read the repos without an account. To make a **How to contribute.** Anyone can read the repos without an account. To make a
change, fork the repo, edit in your fork, and open a **pull request**: change, fork the repo, edit in your fork, and open a **pull request**:
@@ -260,51 +165,39 @@ change, fork the repo, edit in your fork, and open a **pull request**:
4. Open a **pull request** back to the original repo, describing your change. 4. Open a **pull request** back to the original repo, describing your change.
**All changes are reviewed.** Nothing merges directly into `main` — every change **All changes are reviewed.** Nothing merges directly into `main` — every change
comes in as a pull request that an admin reviews and approves first. You don't comes in as a pull request an admin reviews and approves first. You don't need
need to be able to create repos or organizations to contribute; you just fork, to be able to create repos or organizations to contribute; you just fork, edit
edit, and ask for review. and ask for review.
Questions about the Git itself? Write to info@inference.coop. Questions? Write to info@inference.coop.
## Models and providers ## Models and providers
The co-op serves models from multiple providers (Tinfoil for privacy, GreenPT The co-op serves models from multiple providers — Tinfoil for architectural
for renewable energy, PublicAI coming soon). See the [model list](models.md) for privacy, GreenPT and Regolo for renewable energy. See the [model list](models.md),
the current catalog — that's the single source of truth for which models are the single source of truth for which models are available and how they're named.
available and how they're named.
One infra detail worth knowing: a local Tinfoil proxy sidecar handles the
enclave encryption on our side of the gateway — only Tinfoil traffic goes
through it; the other providers are called directly.
## Security posture (plain language) ## Security posture (plain language)
- **Token-gated control plane.** The portal is publicly reachable, but every - **Member isolation.** Each API key is scoped to its owner's team and budget; a
sensitive surface requires a secret token (admin, broker, webhook). All fail member can only ever act on their own keys.
closed — unauthenticated requests get `401`. - **Separated secrets.** Chat, portal, admin and model-layer secrets are
- **Separated secrets.** Chat, broker, admin, and model-layer secrets are
independent and independently rotatable, so a leak has bounded scope. independent and independently rotatable, so a leak has bounded scope.
- **Member isolation.** Each API key is scoped to its owner's LiteLLM team and - **No secrets in version control.** Credentials live in the hosting platform's
budget; a member can only ever act on their own keys. environment, never in the repos.
- **Rate limiting** on control-plane endpoints; the inference path is capped by
each member's budget instead.
- **No secrets in version control.** Credentials live in Cloudron env vars, never
in the repos.
- **Accepted trade-offs** (documented, not hidden): member keys are stored in - **Accepted trade-offs** (documented, not hidden): member keys are stored in
plaintext in the portal database (functionally necessary), and the gateway the portal database so the co-op can serve them, and the API gateway must stay
must stay publicly reachable for API access (its admin UI and docs are publicly reachable. Its admin surfaces are disabled; only the public API is
disabled; only the `/v1/*` API is exposed). exposed.
## For operators ## For operators
Maintaining the co-op's infrastructure — rotating Cloudron tokens, editing the Maintaining the co-op's infrastructure — token rotation, app configuration and
portal's environment, and troubleshooting the provisioning pipeline — is troubleshooting — is documented in the git-only
documented in the [operator [`co-op/ops-docs`](https://git.inference.coop/co-op/ops-docs) repository, which is
runbook](https://git.inference.coop/co-op/ops-docs/src/branch/main/operator-token-runbook.md). deliberately not published on this site. It is the home for the operator runbook,
All operator-facing material (that runbook, the maintenance notes, and incident the maintenance notes, incident write-ups, and the record of past server
write-ups) lives in the git-only migrations. Anyone may read it; operators are the audience.
[`co-op/ops-docs`](https://git.inference.coop/co-op/ops-docs) repository, which
is deliberately not published on this site.
## Incident log ## Incident log
publishing/brand.css
+8 -2
View File
@@ -185,15 +185,21 @@
font-weight: 600; font-weight: 600;
} }
/* Tabs and the search field pick up the brand green. */ /* The menu-bar (tab) strip. In light it is the same pale green WASH as the
* header — #e8f5e8, the homepage's "open source" pill — not a saturated fill:
* the deep greens are accents (links, active navigation, rules), and a large
* --green-deep bar clashed with the rest of the light palette. A warm hairline
* separates the wash from the cream page below it. Dark is unchanged. */
.md-tabs { .md-tabs {
background-color: var(--green-deep); background-color: #e8f5e8;
border-bottom: 1px solid var(--border);
} }
/* In dark, keep the tab row the same green as the header bar so the text on it /* In dark, keep the tab row the same green as the header bar so the text on it
* clears 4.5:1 against green-mid (green-deep against near-black text is only * clears 4.5:1 against green-mid (green-deep against near-black text is only
* ~4.2:1 — marginal for the small tab labels). */ * ~4.2:1 — marginal for the small tab labels). */
[data-md-color-scheme="slate"] .md-tabs { [data-md-color-scheme="slate"] .md-tabs {
background-color: var(--green-mid); background-color: var(--green-mid);
border-bottom: none;
} }
.md-search__input { .md-search__input {
background-color: rgba(255, 255, 255, 0.92); background-color: rgba(255, 255, 255, 0.92);