docs: reorganise into Guides / Operations / Governance
- Guides/: getting started, guest access, models, API access, energy & carbon - Operations/: agentic systems (was 'working with the agent'), infrastructure, roles, steward policy - Governance/: unchanged, now with its overview page Lateral links and the homepage updated; redirects keep the old root URLs alive.
This commit is contained in:
1 parent
a6142d68f2
commit
ac8cac02c6
13 files changed
+66
-29
No files matched your search
@@ -0,0 +1,69 @@
|
||||
# Working with the agent
|
||||
|
||||
The co-op runs **Inferencebot** to help steward its infrastructure. It is
|
||||
isolated as its own OS user, with its own gateway key — budget, model
|
||||
allowlist and spend enforced at the gateway. It runs on co-op infrastructure
|
||||
under a co-op identity, uses the co-op's own gateway for inference, and its
|
||||
skills, scripts and configuration are mirrored to a Gitea repository nightly.
|
||||
This page is for members who read its output or give it instructions.
|
||||
|
||||
## Where it speaks and listens
|
||||
|
||||
- **#inferencebot-updates** is read-only: daily digests, model changes,
|
||||
upgrades, maintenance, outages. Posts are anonymized.
|
||||
- **Instructing the agent** is limited to members on its allowlist of Matrix
|
||||
IDs (kept in its configuration, published here once settled), and only in
|
||||
the rooms where it is configured to answer — most require a mention. The
|
||||
General Manager decides who is on the list.
|
||||
- The **retired `@inference-bot:inference.coop`** automation account (with a
|
||||
hyphen) no longer holds an access token; some historical references and old
|
||||
room roles still carry the name.
|
||||
|
||||
## What to ask for
|
||||
|
||||
- Operational work that a runbook already describes, or that ends in a PR, an
|
||||
issue or a post members can see.
|
||||
- Research and summaries that would otherwise land in an issue.
|
||||
- Status: what is running, what failed, what changed.
|
||||
|
||||
## What not to ask in a shared room
|
||||
|
||||
- Anything involving member-identifying data. That work happens in a session a
|
||||
named steward opened, and the result is reported without the data.
|
||||
- Anything that moves money or changes who is a member. Those go through the
|
||||
portal and a human.
|
||||
- Credentials, in either direction.
|
||||
|
||||
## What they do without asking
|
||||
|
||||
Read, monitor, draft, open issues and PRs, post updates, run the scheduled jobs
|
||||
listed in [Infrastructure](infrastructure.md), and deploy bug and security
|
||||
fixes immediately, reporting each one in #inferencebot-updates.
|
||||
|
||||
## What always needs a human
|
||||
|
||||
- Member-visible, destructive or risky changes to the gateway, the portal or
|
||||
the chat: confirmed before they deploy.
|
||||
- Changing the model catalog, prices or budgets.
|
||||
- Minting or rotating credentials.
|
||||
- Anything a runbook marks "confirm".
|
||||
|
||||
This matches the standing policy the agents operate under today. A stricter
|
||||
norm is a policy decision for the [forum](https://forum.inference.coop).
|
||||
|
||||
## Attribution and the off switch
|
||||
|
||||
Every instruction an agent acts on is attributable to the member who gave it,
|
||||
in the room where it was given. If an agent misbehaves, the operator, or anyone
|
||||
holding gateway-admin and Matrix account access, can stop it: revoke its
|
||||
gateway key, then its Matrix access token. Stewards generally do not hold that
|
||||
access today, so say what you saw in #general and the operator acts. Stopping
|
||||
is not restarting: the backup and the git mirror restore state, not uptime,
|
||||
and someone has to start the agent again. The steps, and the steps for
|
||||
rotating everything an agent holds, are in the agent's runbook; the credential
|
||||
inventory in the [operator token
|
||||
runbook](https://git.inference.coop/co-op/ops-docs/src/branch/main/operator-token-runbook.md)
|
||||
says what that is.
|
||||
|
||||
Changes to this page are policy: propose them in the
|
||||
[forum](https://forum.inference.coop).
|
||||
+4
-2
@@ -1,7 +1,7 @@
|
||||
# Operations
|
||||
|
||||
How the co-op runs its own work: who holds which role, what they are responsible for,
|
||||
and the agreements that come with it.
|
||||
How the co-op runs its own work: who holds which role, the agreements that come
|
||||
with it, the stack it runs on, and the agents that work in it.
|
||||
|
||||
## In this section
|
||||
|
||||
@@ -11,3 +11,5 @@ and the agreements that come with it.
|
||||
and rotated, their duties and privileged access, stipends, accountability and handover.
|
||||
|
||||
More operational documents will land here as roles and procedures are agreed.
|
||||
- [Agentic systems](agentic-systems.md) — what the co-op's agents do, who can instruct them, and what always needs a human.
|
||||
- [Infrastructure](infrastructure.md) — a map of the stack and the codebase.
|
||||
@@ -0,0 +1,200 @@
|
||||
# Infrastructure
|
||||
|
||||
How the Inference Cooperative is built, for members who want to understand it —
|
||||
and contributors who want to work on it.
|
||||
|
||||
## The big picture
|
||||
|
||||
Members reach the co-op through two front doors: the **chat** (LibreChat) and
|
||||
the **API** (the LiteLLM gateway). Both route through a single gateway to
|
||||
multiple LLM providers, so members can choose what they value for each task:
|
||||
architecturally private inference in **TEE-protected enclaves** (Tinfoil), or
|
||||
100% renewable energy (GreenPT, Regolo). A thin layer of custom middleware — the
|
||||
member portal — ties Open Collective membership to Cloudron accounts and LiteLLM
|
||||
keys.
|
||||
|
||||
```
|
||||
┌─> Tinfoil (TEE) 🔒 private
|
||||
Members ──> LibreChat (chat) ──┐ │
|
||||
├──> LiteLLM ──┼─> GreenPT (renewable) 🌱 green
|
||||
Members ──> your own tools ────┘ gateway │
|
||||
(API) (keys, └─> Regolo (renewable) 🌱 green
|
||||
metering,
|
||||
routing)
|
||||
```
|
||||
|
||||
The whole stack runs on [Cloudron](https://cloudron.io), which handles hosting,
|
||||
single sign-on, and per-app isolation.
|
||||
|
||||
## Where it runs
|
||||
|
||||
The co-op's member-facing services run on a server the co-op owns and pays for
|
||||
directly, at **Infomaniak** — a Swiss sovereign cloud provider whose data centres
|
||||
are in Geneva, run on renewable energy, and are certified to ISO 14001, ISO
|
||||
50001 and ISO 27001.
|
||||
|
||||
The co-op's **automation runs on a separate server**, also hosted on Infomaniak
|
||||
and deliberately kept apart from the member-facing applications. The steward bot
|
||||
— the automation behind the digests and the newsletter drafts — and the co-op's
|
||||
other agents live there. Keeping them on their own host means an agent problem
|
||||
stays an agent problem: it cannot reach the chat, the gateway or member data.
|
||||
|
||||
### Backups
|
||||
|
||||
Cloudron writes **encrypted** backups to Infomaniak's object storage in
|
||||
Switzerland every night and keeps the last **seven days**, so a failed upgrade
|
||||
or a lost server can be restored. The automation host backs up its own data to
|
||||
the same object storage.
|
||||
|
||||
### Domain and DNS
|
||||
|
||||
The co-op's domain is **inference.coop** — a `.coop` domain. The co-op registers
|
||||
and manages its domains with **domains.coop**, the domain provider run by the
|
||||
cooperative movement, and holds them in the co-op's own name rather than in any
|
||||
individual's. DNS itself is deliberately simple, and the co-op's own nameservers
|
||||
answer it.
|
||||
|
||||
## Components
|
||||
|
||||
| Component | Purpose | URL |
|
||||
|-----------|---------|-----|
|
||||
| **LibreChat** | Member-facing chat (models, web search, image generation, voice) | chat.inference.coop |
|
||||
| **Open WebUI** | Legacy chat instance — read-only reference (old history and file attachments) | chat-legacy.inference.coop |
|
||||
| **LiteLLM** | AI gateway — keys, metering and model routing, for chat and API | gateway.inference.coop |
|
||||
| **Member portal** | Membership middleware — onboarding, key provisioning, Loomio sync | portal.inference.coop |
|
||||
| **Member dashboard** | Your usage view and API keys | dashboard.inference.coop |
|
||||
| **Loomio** | Member governance | forum.inference.coop |
|
||||
| **Matrix (Synapse)** | Member chat server, federated, SSO login via Cloudron | matrix.inference.coop |
|
||||
| **Gitea** | Git hosting for the co-op's code and this documentation | git.inference.coop |
|
||||
| **Listmonk** | Member newsletter (synced from membership) | newsletter.inference.coop |
|
||||
| **SnappyMail** | Webmail | email.inference.coop |
|
||||
|
||||
The co-op's software behind all of these is open source — see [The
|
||||
code](#the-code) below for where each piece lives.
|
||||
|
||||
Open Collective (billing and fiscal sponsorship) is the one external service in
|
||||
the flow — see
|
||||
[opencollective.com/inference-cooperative](https://opencollective.com/inference-cooperative).
|
||||
|
||||
## Mail
|
||||
|
||||
The co-op runs its own `@inference.coop` mail. **Incoming** mail arrives directly
|
||||
on the co-op's server, and webmail and mail apps reach it over the usual
|
||||
encrypted ports (IMAP and SMTP over TLS). **Outgoing** mail is sent through
|
||||
**Scaleway**, an EU-based email service, authenticated for our domain.
|
||||
|
||||
You can reach your mailbox through webmail at email.inference.coop, or add it to
|
||||
a standard mail app on your phone or computer. If mail to your `@inference.coop`
|
||||
address stops arriving, that is a fault on our side to fix.
|
||||
|
||||
## Matrix
|
||||
|
||||
Members get a federated Matrix account on the co-op's self-hosted Synapse
|
||||
(matrix.inference.coop, user IDs `@username:inference.coop`), signing in with the
|
||||
same Cloudron SSO as the chat. The co-op invites every active member to the
|
||||
private **Inference Cooperative** space (with a **General** room) automatically,
|
||||
so new members land there as they join. Federation is on, so you can also join
|
||||
from any homeserver you already use.
|
||||
|
||||
### Accounts you may meet
|
||||
|
||||
A few accounts in the space are automation rather than people:
|
||||
|
||||
- **`@inferencebot` — the steward.** It posts the digests, newsletter notes and
|
||||
reminders. It holds no more power than posting needs.
|
||||
- **`@inference-admin`** — the administrator of record for the co-op's rooms
|
||||
(#general, #community, #design, #skillshare, #updates and #infrastructure) and
|
||||
their membership. It **never speaks in a member room** — a message from it is
|
||||
a mistake, not a feature.
|
||||
|
||||
The house rule: **power belongs to `@inference-admin`, voice belongs to
|
||||
`@inferencebot`.** Please don't create an account whose name differs from an
|
||||
existing one by a character or two — spotting lookalikes is exactly what this
|
||||
convention is for.
|
||||
|
||||
## Membership
|
||||
|
||||
Membership runs through the member portal, the single source of truth for who is
|
||||
a member:
|
||||
|
||||
1. A member contributes on Open Collective.
|
||||
2. Open Collective notifies the portal, which provisions a Cloudron account, a
|
||||
LiteLLM key, and a welcome email.
|
||||
3. A nightly reconciliation against the live Open Collective list means a missed
|
||||
notification self-heals within a day.
|
||||
|
||||
You may see **"Inference Co-op Bot"** listed as an admin of the collective on
|
||||
Open Collective. It is infrastructure, not a person or a member: a dedicated
|
||||
account the portal uses, **read-only**, to check the member roster and tiers. It
|
||||
cannot move money, issue refunds or change the collective — payments always come
|
||||
from a human's own contribution, and automation only reads the result. Its code
|
||||
is open source in
|
||||
[code/member-portal](https://git.inference.coop/code/member-portal), and a
|
||||
collective admin can revoke its access at any time.
|
||||
|
||||
## The code
|
||||
|
||||
The co-op's software is open source, in the [`code`](https://git.inference.coop/code)
|
||||
and [`co-op`](https://git.inference.coop/co-op) organizations on Gitea:
|
||||
|
||||
- [**code/member-portal**](https://git.inference.coop/code/member-portal) — membership middleware (Python).
|
||||
- [**code/member-dashboard**](https://git.inference.coop/code/member-dashboard) — member dashboard: usage and API keys.
|
||||
- [**code/litellm**](https://git.inference.coop/code/litellm) — the gateway packaged as a Cloudron app.
|
||||
- [**co-op/website**](https://git.inference.coop/co-op/website) — landing page and static site.
|
||||
- [**co-op/docs**](https://git.inference.coop/co-op/docs) — this documentation.
|
||||
|
||||
The chat, portal and other apps live in `code/` and `co-op/` too; everything is
|
||||
browsable at [git.inference.coop](https://git.inference.coop). Everything the co-op runs
|
||||
is open source, so as a member you can browse all of
|
||||
it and propose changes.
|
||||
|
||||
**Who can sign in.** Git access is tied to your membership: you log into the Git
|
||||
with the same single sign-on you use for the chat. Members are in automatically
|
||||
(through the `members` group); when a membership lapses, Git access lapses with
|
||||
it.
|
||||
|
||||
**How to contribute.** Anyone can read the repos without an account. To make a
|
||||
change, fork the repo, edit in your fork, and open a **pull request**:
|
||||
|
||||
1. Log into [git.inference.coop](https://git.inference.coop) with your co-op account.
|
||||
2. **Fork** the repo you want to change (e.g. `co-op/docs`).
|
||||
3. Make your edits in your fork and commit them.
|
||||
4. Open a **pull request** back to the original repo, describing your change.
|
||||
|
||||
**All changes are reviewed.** Nothing merges directly into `main` — every change
|
||||
comes in as a pull request an admin reviews and approves first. You don't need
|
||||
to be able to create repos or organizations to contribute; you just fork, edit
|
||||
and ask for review.
|
||||
|
||||
## Models and providers
|
||||
|
||||
The co-op serves models from multiple providers — Tinfoil for architectural
|
||||
privacy, GreenPT and Regolo for renewable energy. See the [model list](../guides/models.md),
|
||||
the single source of truth for which models are available and how they're named.
|
||||
|
||||
## Security posture (plain language)
|
||||
|
||||
- **Member isolation.** Each API key is scoped to its owner's team and budget; a
|
||||
member can only ever act on their own keys.
|
||||
- **Separated secrets.** Chat, portal, admin and model-layer secrets are
|
||||
independent and independently rotatable, so a leak has bounded scope.
|
||||
- **No secrets in version control.** Credentials live in the hosting platform's
|
||||
environment, never in the repos.
|
||||
- **Accepted trade-offs** (documented, not hidden): member keys are stored in
|
||||
the portal database so the co-op can serve them, and the API gateway must stay
|
||||
publicly reachable. Its admin surfaces are disabled; only the public API is
|
||||
exposed.
|
||||
|
||||
## For operators
|
||||
|
||||
Maintaining the co-op's infrastructure — token rotation, app configuration and
|
||||
troubleshooting — is documented in the git-only
|
||||
[`co-op/ops-docs`](https://git.inference.coop/co-op/ops-docs) repository, which is
|
||||
deliberately not published on this site. It is the home for the operator runbook,
|
||||
the maintenance notes, incident write-ups, and the record of past server
|
||||
migrations. Anyone may read it; operators are the audience.
|
||||
|
||||
## Incident log
|
||||
|
||||
Write-ups for outages live in the operator docs, in
|
||||
[`incidents/`](https://git.inference.coop/co-op/ops-docs/src/branch/main/incidents).
|
||||
Reference in new issue
Block a user