Compare commits
13
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
a3fabdd53c | ||
|
|
c9a6e23154 | ||
|
|
12da5c1268 | ||
|
|
1f3f0ce9b8 | ||
|
|
fde5d143c5 | ||
|
|
0e708a99c6 | ||
|
|
467d19357d | ||
|
|
6a0d092e53 | ||
|
|
bac352f50e | ||
|
|
34a23a34f3 | ||
|
|
fa979a5ed9 | ||
|
|
5bb99b8592 | ||
|
|
e7c86b16ba |
No files matched your search
+47
-16
@@ -39,11 +39,17 @@ account. Click it and create your login.
|
||||
The chat lives at **[chat.inference.coop](https://chat.inference.coop)**. Log in
|
||||
with the account you created in step 1.
|
||||
|
||||
- **Choose your model.** You can pick not just *which* model but *where* it runs:
|
||||
**private** (Tinfoil, runs in encrypted enclaves) or **green** (GreenPT, 100%
|
||||
renewable energy). See the [model list](models.md) for the full list.
|
||||
- **Choose your model.** You can pick not just *which* model but *where* it
|
||||
runs: **private** (Tinfoil, runs in encrypted enclaves) or **green**
|
||||
(GreenPT and Regolo, 100% renewable energy — Regolo adds a written
|
||||
zero-data-retention policy). See the [model list](models.md) for the full
|
||||
list.
|
||||
- **Web search** is enabled by default (self-hosted, no external dependency).
|
||||
- **File uploads** work out of the box — ask the chat about a document you attach.
|
||||
- **Image generation** is built in — ask the chat for an image (for example,
|
||||
"draw a pine tree in flat green circles") and it renders one.
|
||||
- **Voice** works in both directions: speak to the chat with the microphone
|
||||
icon, and have replies read aloud.
|
||||
- **External tool servers.** The chat can connect to external tool services —
|
||||
things that let the AI do like search marketplaces, query other services,
|
||||
or pull in specialized data. Members can add their own **OpenAPI** services:
|
||||
@@ -62,6 +68,20 @@ Your usage draws from your monthly allowance; you can see it on the
|
||||
If the chat ever says your balance is out, that's the allowance, not an error —
|
||||
see "Your allowance" below.
|
||||
|
||||
### Add the chat to your phone
|
||||
|
||||
The chat works like an app on your phone — no app store needed. Open
|
||||
[chat.inference.coop](https://chat.inference.coop) in your phone's browser and
|
||||
log in, then:
|
||||
|
||||
- **iPhone (Safari):** tap the **Share** button (the square with an arrow),
|
||||
then **Add to Home Screen**.
|
||||
- **Android (Chrome):** tap the **⋮** menu, then **Add to Home Screen**
|
||||
(or **Install app**).
|
||||
|
||||
You'll get an icon on your home screen that opens the chat full-screen, like
|
||||
any other app. Your login stays active — just tap and chat.
|
||||
|
||||
## 3. Get involved in governance
|
||||
|
||||
You don't just use the co-op; you help run it. Decisions about models, pricing,
|
||||
@@ -86,10 +106,25 @@ privacy, and direction happen on our **Loomio forum**:
|
||||
- **FluffyChat, Cinny, SchildiChat** or any other client: look for the "custom homeserver"
|
||||
/ "other server" option in the client's login screen and enter the same address.
|
||||
- Your co-op account and your existing Matrix account on another server (e.g.
|
||||
matrix.org or a personal homeserver) are separate identities. If you'd
|
||||
rather use an existing account for co-op rooms, set it on your
|
||||
[dashboard](https://dashboard.inference.coop) under **Matrix chat** (one
|
||||
extra account) — invitations will go there instead. Both work either way.
|
||||
matrix.org or a personal homeserver) are separate identities. **Pick your
|
||||
identity (optional):** by default, invitations go to your co-op identity
|
||||
(`@username:inference.coop`). To use an account you already have instead:
|
||||
|
||||
1. Open your [dashboard](https://dashboard.inference.coop) and find the
|
||||
**Matrix chat** card. It shows the Matrix ID currently registered for
|
||||
you — your co-op identity, or `pending` if your co-op Matrix account
|
||||
hasn't been created yet (it is created the first time you log in to
|
||||
Matrix).
|
||||
2. Click **Use external account**, enter your existing Matrix ID in the
|
||||
format `@username:server.org`, and click **Save**.
|
||||
3. Accept the space invitation that arrives in that account — usually
|
||||
instantly. If the account can't be reached right away, the invitation
|
||||
is retried automatically each night.
|
||||
|
||||
That's all it takes: invitations (and the space) follow whichever identity
|
||||
you've chosen. Switching back is one click — **Use
|
||||
@username:inference.coop** on the same card. One identity is active at a
|
||||
time, and both work equally well in every co-op room.
|
||||
|
||||
**Members-only, readable history.** The space is invite-only and unencrypted
|
||||
so every member can read the full history (including messages from before
|
||||
@@ -97,15 +132,11 @@ privacy, and direction happen on our **Loomio forum**:
|
||||
welcome to bring a collaborator or guest, an admin sends the invite rather
|
||||
than members handing the space to outsiders directly.
|
||||
|
||||
- **#inferencebot-updates** — an automated feed inside the space where the
|
||||
co-op's stewarding bot posts daily updates on infrastructure work: model
|
||||
changes, gateway upgrades, maintenance, and outages. Posts are anonymized
|
||||
(never any member-identifying details) and only the bot and admins can
|
||||
write — it's a read-and-follow channel. Questions go to the main space
|
||||
room.
|
||||
|
||||
- **#design** — an open member room for co-op design discussions: website,
|
||||
docs, dashboard, branding. All members can post; pull up a chair.
|
||||
- **#general** is the main room — say hello, ask questions, start
|
||||
conversations. Beyond it, the space holds a handful of topical rooms
|
||||
(infrastructure, design, an automated co-op updates feed) — explore the
|
||||
space and join whatever is interesting to you. New rooms appear there as
|
||||
the co-op grows.
|
||||
|
||||
- See the [Charter](charter.md) for how governance is structured.
|
||||
- See [Open questions](open-questions.md) for the decisions currently on the table.
|
||||
|
||||
@@ -0,0 +1,27 @@
|
||||
# Chat task models (titles, tags, suggestions)
|
||||
|
||||
Both chat front doors generate small helper outputs — conversation titles,
|
||||
tags, retrieval queries — with a model separate from the one answering the
|
||||
member. These tasks are pinned to a small, inexpensive model so they never
|
||||
bill against the member's chosen (larger) model.
|
||||
|
||||
## Open WebUI (chat.inference.coop)
|
||||
|
||||
- Title generation and tag generation are **enabled**; follow-up reply
|
||||
suggestions and input autocomplete are **disabled** (October 2026).
|
||||
- The task model (`task.model.default` in the Postgres `config` table) is
|
||||
pinned to `greenpt/glm-5.3-flash` — a small, inexpensive model — so titles
|
||||
and tags always run on it regardless of which model the member is chatting
|
||||
with. Before October 2026 it was empty, which meant helper tasks ran on the
|
||||
member's selected model.
|
||||
- Config lives in the app's Postgres database (`config` table, key/value rows
|
||||
with JSON values) and is read at startup: change a row, then restart the
|
||||
app (`cloudron restart --app <id>`; note `restart` takes no `--timeout`).
|
||||
|
||||
## LibreChat (librechat-test.inference.coop)
|
||||
|
||||
- Title generation is configured in `/app/data/librechat.yaml`:
|
||||
`titleConvo: true`, `titleModel: greenpt/glm-5.3-flash`,
|
||||
`titleMethod: completion` — the same small GreenPT model.
|
||||
- LibreChat v0.8.8 has no reply-suggestion feature; there is nothing to
|
||||
disable on that front.
|
||||
@@ -0,0 +1,156 @@
|
||||
# LibreChat sandbox maintenance
|
||||
|
||||
The LibreChat evaluation instance at **librechat-test.inference.coop**
|
||||
(Cloudron app `4023ccbf-d57a-4500-97df-14de9cf3516e`, appstore package
|
||||
`ai.librechat.cloudronapp@1.4.1`, which bundles **LibreChat v0.8.8**).
|
||||
|
||||
The single source of truth for the deployment is the repo
|
||||
**[co-op/librechat-custom](https://git.inference.coop/co-op/librechat-custom)**
|
||||
— its README is the full runbook (config deploy, image build/deploy, rollback,
|
||||
env vars, branding overlay, known state). This doc is the operator-facing
|
||||
summary.
|
||||
|
||||
## Architecture: what lives where
|
||||
|
||||
- **App config** — `/app/data/librechat.yaml` (persistent volume, mirrored
|
||||
byte-identically in the repo). Holds the "Inference Co-op" custom endpoint
|
||||
(gateway), the 8-model chat picker, speech, web search, and the two MCP
|
||||
servers. Volume edits deploy via `cloudron exec` (base64), never via the
|
||||
image — the image never touches `/app/data`.
|
||||
- **Env file** — `/app/data/env` (persistent volume). **`start.sh` sources it
|
||||
AFTER the process env, so it wins over `cloudron env`** — that is where
|
||||
`APP_TITLE="Inference Co-op"`, `ALLOW_EMAIL_LOGIN=false` (password login
|
||||
removed; Cloudron SSO is the only login path), and
|
||||
`SCHEDULES_SINGLE_PROCESS=true` must live.
|
||||
- **Other env vars** — via `cloudron env`: `TTS_API_KEY`/`STT_API_KEY`
|
||||
(dedicated LiteLLM virtual key, alias `librechat-speech`, $5/30d, audio
|
||||
models only — never the gateway master key), `IMAGE_GEN_OAI_BASEURL` +
|
||||
`IMAGE_GEN_OAI_MODEL` (agent image tool → gateway), `SEARXNG_INSTANCE_URL`.
|
||||
- **MCP servers** — `/app/data/mcp/server.py` (docs search, `search_docs`,
|
||||
starts at boot) and `/app/data/mcp/image_server.py` (image generation,
|
||||
`generate_image`, `startup: false`); both mirrored in the repo under
|
||||
`mcp/`, stdlib-only Python.
|
||||
- **Branding overlay** (logo, favicons, PWA icons, `<title>`, `custom.css`,
|
||||
`custom2.js`) — baked into a pinned overlay image `FROM` the official
|
||||
appstore image. Deployed tag (last verified 2026-10-19):
|
||||
`registry.inference.coop/co-op/librechat-custom:20261019-fc9b11e`.
|
||||
- **Branding CSS/JS** — `custom.css` overrides LibreChat's Tailwind theme
|
||||
variables on `html.light`/`html.dark` using the HOMEPAGE's canonical
|
||||
palette (`--cream #faf8f5`, `--wash #eef4ea`, `--green-deep #5b8c5a`,
|
||||
`--green-mid #6ba86b`, `--ink #2d3327`, `--muted #6b7a62`) and brands
|
||||
the loading flash (`#loading-container`); the login logo is the green
|
||||
pine-dots vector (white-plate variant was invisible on cream — the
|
||||
'flash on load' glitch), cache-busted via `?v=3` in the client bundle.
|
||||
and hides the register/sign-up prompt via
|
||||
`p:has(> a[href*="register"])`; `custom2.js` is a MutationObserver that
|
||||
hides the prompt when React renders it (the auth form renders after
|
||||
`load`). The `interface.theme` block in the yaml remains
|
||||
forward-compatible only (inert on v0.8.8).
|
||||
|
||||
## Image generation (two member-facing paths)
|
||||
|
||||
1. **MCP tool (default, no key ever)**: the `inference-image` MCP server
|
||||
exposes `generate_image` in the composer's **+** palette — no agent setup,
|
||||
no key prompt. Per-member billing via `customUserVars.IC_API_KEY` (seeded
|
||||
from the portal) + `env: IC_API_KEY: "{{IC_API_KEY}}"`; a system-key
|
||||
fallback (`system-image-fallback`, $5/30d) covers the rare empty-value
|
||||
case. The `customUserVars` gate blocks a not-yet-seeded member until the
|
||||
next sync (≤10 min) — accepted as a short delay over billing everything
|
||||
to the system account.
|
||||
2. **Agent tool**: LibreChat's built-in "OpenAI Image Tools"
|
||||
(`IMAGE_GEN_OAI_BASEURL`/`IMAGE_GEN_OAI_MODEL` → gateway; members paste
|
||||
their own key — v0.8.8 limitation).
|
||||
|
||||
## Update policy
|
||||
|
||||
- The app is **image-sourced** (the `--force` update moved it off the
|
||||
appstore source), so Cloudron will not auto-update it. This is deliberate —
|
||||
same lockdown model as LiteLLM.
|
||||
- **Image deploys**: build `--no-cache`, write the new tag into
|
||||
`CloudronManifest.json.dockerImage` (the only editable field), commit, then
|
||||
`cloudron update --app <id> --force`. **`--image` alone is IGNORED** when
|
||||
the manifest sits in the deploy directory. **The remote builder's cache can
|
||||
serve stale COPY layers even when file content changed** (six consecutive
|
||||
"Build successful" runs did on 2026-10-19) — so after every image deploy,
|
||||
verify: the Cloudron API's `manifest.dockerImage`, container-file md5 vs
|
||||
repo md5, the served HTML, and the rendered DOM. Full checklist in the
|
||||
repo README.
|
||||
- **When a new appstore package ships**: pull its manifest copy into the repo
|
||||
(`GET /api/v1/apps/<id>` → `.manifest`), repoint `dockerImage` to the
|
||||
rebuilt overlay, bump the Dockerfile `FROM` to the new `manifest.dockerImage`
|
||||
(pinned), rebuild, verify in the sandbox, then update.
|
||||
- **Rollback**: repoint `CloudronManifest.json.dockerImage` to the previous
|
||||
tag and rerun `cloudron update --force` (or the original appstore image).
|
||||
Cloudron also snapshots the app before every image update.
|
||||
|
||||
## Hard-won pitfalls (do not re-learn these)
|
||||
|
||||
- `cloudron update` deploys **`manifest.dockerImage`** from the manifest in
|
||||
the deploy directory — the `--image` flag alone is ignored (and the update
|
||||
output still prints the `--image` value, so it *looks* right). The repo
|
||||
manifest is a byte-identical copy of the OFFICIAL appstore manifest except
|
||||
`dockerImage`; editing anything else in it (or hand-writing one that omits
|
||||
the addons) strips `postgresql`/`mongodb`/`oidc`/`sendmail`/`localstorage`
|
||||
from the app and crash-loops it (`mkdir: Read-only file system`). That is
|
||||
what broke the sandbox on 2026-10-18.
|
||||
- The remote builder's layer cache can pin COPY steps to old file content
|
||||
even across "successful" rebuilds — use `--no-cache` after asset edits and
|
||||
confirm by comparing container-file md5 against the repo file.
|
||||
- Deploy `librechat.yaml` only via base64 round-trip. A `sed` insert once
|
||||
broke the YAML indentation and LibreChat **silently fell back to defaults**
|
||||
(all custom branding/models/MCP gone) with a single log line.
|
||||
- Reading large files from the container over `cloudron exec` with a plain
|
||||
`cat` resets the connection (`ECONNRESET`) — pipe through
|
||||
`gzip | base64 | fold` and decode locally.
|
||||
- `cloudron env set` alone does NOT disable the password form —
|
||||
`ALLOW_EMAIL_LOGIN` must be flipped in `/app/data/env` (precedence above).
|
||||
- Image generation in LibreChat is the **MCP tool** (composer **+** palette);
|
||||
the agent "OpenAI Image Tools" path also exists. Image models in the chat
|
||||
picker were the source of the old 500 errors and are deliberately excluded.
|
||||
- No `branding:` block exists in LibreChat's config schema, and the login
|
||||
page has no `.login-page`/`.primary` classes to target — theming happens by
|
||||
overriding the Tailwind CSS variables on `html.light`/`html.dark`
|
||||
(`--surface-primary`, `--accent-primary`, …) from the overlay `custom.css`.
|
||||
|
||||
|
||||
## Per-member key auto-load (no API-key prompts)
|
||||
|
||||
Members never enter an API key. The portal provisions each member's LiteLLM
|
||||
key at onboarding; the sandbox seeds it into LibreChat's user-key store:
|
||||
|
||||
- `GET /admin/librechat/keys` on the portal returns active members' chat keys
|
||||
(dedicated `X-Sync-Token` auth — `LIBRECHAT_SYNC_TOKEN` on both apps).
|
||||
- `/app/data/sync-keys.js` (in the LibreChat container, mirrored in the repo
|
||||
as `sync-keys.js`) fetches that feed and seeds the `keys` collection
|
||||
(chat endpoint, `apiKey: "user_provided"`) and `pluginauths`
|
||||
(`mcp:inference-image` → `IC_API_KEY`, injected per-user via
|
||||
`customUserVars` so image generation bills to the member's own team).
|
||||
- Members without a sandbox account are skipped until they first log in; run
|
||||
the sync after onboarding waves and on a schedule (the command is in the
|
||||
repo README). Idempotent.
|
||||
- Encryption matches LibreChat's own (`CREDS_KEY`/`CREDS_IV`, read from
|
||||
`/proc/1/environ` — exec sessions don't inherit the app env).
|
||||
|
||||
> Terminal output redacts values after `apiKey:` as `***` even for harmless
|
||||
> strings (`user_provided`, `${TTS_API_KEY}`). Probe lengths/counts instead
|
||||
> of trusting redacted output.
|
||||
|
||||
## Verification (after any change)
|
||||
|
||||
```bash
|
||||
cloudron logs --app 4023ccbf-d57a-4500-97df-14de9cf3516e --lines 60 | \
|
||||
grep -iE 'custom config|error|invalid|MCP|schedules'
|
||||
curl -s https://librechat-test.inference.coop/api/config # appTitle, emailLoginEnabled
|
||||
```
|
||||
|
||||
Expected: `Custom config file loaded`, `[MCP][inference-docs]` initialized
|
||||
with tool `search_docs`, `inference-image` registered (its `Tools: undefined`
|
||||
boot line is expected — per-user startup), `[schedules] engine started`,
|
||||
`appTitle: Inference Co-op`, `emailLoginEnabled: false`. The login page shows
|
||||
the co-op logo, the co-op favicon, a single "Login with Inference
|
||||
Cooperative" (Cloudron SSO) path with no email/password form, **no
|
||||
"Don't have an account?" line**, and the cream/green co-op color scheme.
|
||||
|
||||
After an IMAGE deploy, additionally run the four-step verification in the
|
||||
repo README (API `manifest.dockerImage`, container md5 vs repo md5, served
|
||||
HTML tags, rendered DOM colors).
|
||||
@@ -79,7 +79,7 @@ The emoji in the model name marks its **value tier** (the same icons used in the
|
||||
| `🌱 greenpt/glm-5.3-flash` | GreenPT | policy | ✅ 100% renewable | — | 1M | $0.11 / $0.44 / $0.022 | Fast GLM model, renewable energy. |
|
||||
| `🌱 greenpt/glm-5.3` | GreenPT | policy | ✅ 100% renewable | — | 1M | $1.10 / $4.40 / $0.275 | GLM flagship — strongest agentic model in the green tier. |
|
||||
| `🌱 regolo/apertus-70b` | Regolo | policy + ZDR | ✅ 100% renewable | ✅ | 65K | $0.46 / $2.44 / — | Fully open sovereign model (Apache-2.0), large/capable, image understanding. |
|
||||
| `🌱 regolo/qwen3.5-122b` | Regolo | policy + ZDR | ✅ 100% renewable | ✅ | 262K | $1.16 / $4.87 / — | Large vision+reasoning MoE (Qwen 3.5, 122B-A10B); biggest context in the roster. |
|
||||
| `🌱 regolo/qwen3.5-122b` | Regolo | policy + ZDR | ✅ 100% renewable | ✅ | 262K | $1.16 / $4.87 / — | Large vision+reasoning MoE (Qwen 3.5, 122B-A10B); 262K context. |
|
||||
|
||||
Vision is verified working through the co-op gateway (base64 image input via
|
||||
`/v1/chat/completions`, standard OpenAI format). In the chat UI, just attach
|
||||
|
||||
Reference in new issue
Block a user