getting-started: focus on #general, encourage exploring the space instead of listing rooms

Documents the verified setup: config/env/MCP on the persistent volume,
brand assets via the pinned overlay image, /app/data/env precedence over
cloudron env, the manifest.dockerImage deploy mechanism, image-gen as an
agent tool, and the 2026-10-18 crash-loop root cause (hand-edited manifest).

getting-started.md

@@ -39,11 +39,17 @@ account. Click it and create your login.
 The chat lives at **[chat.inference.coop](https://chat.inference.coop)**. Log in
 with the account you created in step 1.
 
-- **Choose your model.** You can pick not just *which* model but *where* it runs:
+- **Choose your model.** You can pick not just *which* model but *where* it
-  **private** (Tinfoil, runs in encrypted enclaves) or **green** (GreenPT, 100%
+  runs: **private** (Tinfoil, runs in encrypted enclaves) or **green**
-  renewable energy). See the [model list](models.md) for the full list.
+  (GreenPT and Regolo, 100% renewable energy — Regolo adds a written
+  zero-data-retention policy). See the [model list](models.md) for the full
+  list.
 - **Web search** is enabled by default (self-hosted, no external dependency).
 - **File uploads** work out of the box — ask the chat about a document you attach.
+- **Image generation** is built in — ask the chat for an image (for example,
+  "draw a pine tree in flat green circles") and it renders one.
+- **Voice** works in both directions: speak to the chat with the microphone
+  icon, and have replies read aloud.
 - **External tool servers.** The chat can connect to external tool services —
   things that let the AI do like search marketplaces, query other services,
   or pull in specialized data. Members can add their own **OpenAPI** services:
@@ -62,6 +68,20 @@ Your usage draws from your monthly allowance; you can see it on the
 If the chat ever says your balance is out, that's the allowance, not an error —
 see "Your allowance" below.
 
+### Add the chat to your phone
+
+The chat works like an app on your phone — no app store needed. Open
+[chat.inference.coop](https://chat.inference.coop) in your phone's browser and
+log in, then:
+
+- **iPhone (Safari):** tap the **Share** button (the square with an arrow),
+  then **Add to Home Screen**.
+- **Android (Chrome):** tap the **⋮** menu, then **Add to Home Screen**
+  (or **Install app**).
+
+You'll get an icon on your home screen that opens the chat full-screen, like
+any other app. Your login stays active — just tap and chat.
+
 ## 3. Get involved in governance
 
 You don't just use the co-op; you help run it. Decisions about models, pricing,
@@ -86,10 +106,25 @@ privacy, and direction happen on our **Loomio forum**:
   - **FluffyChat, Cinny, SchildiChat** or any other client: look for the "custom homeserver"
     / "other server" option in the client's login screen and enter the same address.
   - Your co-op account and your existing Matrix account on another server (e.g.
-    matrix.org or a personal homeserver) are separate identities. If you'd
+    matrix.org or a personal homeserver) are separate identities. **Pick your
-    rather use an existing account for co-op rooms, set it on your
+    identity (optional):** by default, invitations go to your co-op identity
-    [dashboard](https://dashboard.inference.coop) under **Matrix chat** (one
+    (`@username:inference.coop`). To use an account you already have instead:
-    extra account) — invitations will go there instead. Both work either way.
+
+    1. Open your [dashboard](https://dashboard.inference.coop) and find the
+       **Matrix chat** card. It shows the Matrix ID currently registered for
+       you — your co-op identity, or `pending` if your co-op Matrix account
+       hasn't been created yet (it is created the first time you log in to
+       Matrix).
+    2. Click **Use external account**, enter your existing Matrix ID in the
+       format `@username:server.org`, and click **Save**.
+    3. Accept the space invitation that arrives in that account — usually
+       instantly. If the account can't be reached right away, the invitation
+       is retried automatically each night.
+
+    That's all it takes: invitations (and the space) follow whichever identity
+    you've chosen. Switching back is one click — **Use
+    @username:inference.coop** on the same card. One identity is active at a
+    time, and both work equally well in every co-op room.
 
   **Members-only, readable history.** The space is invite-only and unencrypted
   so every member can read the full history (including messages from before
@@ -97,15 +132,11 @@ privacy, and direction happen on our **Loomio forum**:
   welcome to bring a collaborator or guest, an admin sends the invite rather
   than members handing the space to outsiders directly.
 
-- **#inferencebot-updates** — an automated feed inside the space where the
+- **#general** is the main room — say hello, ask questions, start
-  co-op's stewarding bot posts daily updates on infrastructure work: model
+  conversations. Beyond it, the space holds a handful of topical rooms
-  changes, gateway upgrades, maintenance, and outages. Posts are anonymized
+  (infrastructure, design, an automated co-op updates feed) — explore the
-  (never any member-identifying details) and only the bot and admins can
+  space and join whatever is interesting to you. New rooms appear there as
-  write — it's a read-and-follow channel. Questions go to the main space
+  the co-op grows.
-  room.
-
-- **#design** — an open member room for co-op design discussions: website,
-  docs, dashboard, branding. All members can post; pull up a chair.
 
 - See the [Charter](charter.md) for how governance is structured.
 - See [Open questions](open-questions.md) for the decisions currently on the table.

maintenance/chat-task-models.md

@@ -0,0 +1,27 @@
+# Chat task models (titles, tags, suggestions)
+
+Both chat front doors generate small helper outputs — conversation titles,
+tags, retrieval queries — with a model separate from the one answering the
+member. These tasks are pinned to a small, inexpensive model so they never
+bill against the member's chosen (larger) model.
+
+## Open WebUI (chat.inference.coop)
+
+- Title generation and tag generation are **enabled**; follow-up reply
+  suggestions and input autocomplete are **disabled** (October 2026).
+- The task model (`task.model.default` in the Postgres `config` table) is
+  pinned to `greenpt/glm-5.3-flash` — a small, inexpensive model — so titles
+  and tags always run on it regardless of which model the member is chatting
+  with. Before October 2026 it was empty, which meant helper tasks ran on the
+  member's selected model.
+- Config lives in the app's Postgres database (`config` table, key/value rows
+  with JSON values) and is read at startup: change a row, then restart the
+  app (`cloudron restart --app <id>`; note `restart` takes no `--timeout`).
+
+## LibreChat (librechat-test.inference.coop)
+
+- Title generation is configured in `/app/data/librechat.yaml`:
+  `titleConvo: true`, `titleModel: greenpt/glm-5.3-flash`,
+  `titleMethod: completion` — the same small GreenPT model.
+- LibreChat v0.8.8 has no reply-suggestion feature; there is nothing to
+  disable on that front.

maintenance/librechat-sandbox.md

@@ -0,0 +1,156 @@
+# LibreChat sandbox maintenance
+
+The LibreChat evaluation instance at **librechat-test.inference.coop**
+(Cloudron app `4023ccbf-d57a-4500-97df-14de9cf3516e`, appstore package
+`ai.librechat.cloudronapp@1.4.1`, which bundles **LibreChat v0.8.8**).
+
+The single source of truth for the deployment is the repo
+**[co-op/librechat-custom](https://git.inference.coop/co-op/librechat-custom)**
+— its README is the full runbook (config deploy, image build/deploy, rollback,
+env vars, branding overlay, known state). This doc is the operator-facing
+summary.
+
+## Architecture: what lives where
+
+- **App config** — `/app/data/librechat.yaml` (persistent volume, mirrored
+  byte-identically in the repo). Holds the "Inference Co-op" custom endpoint
+  (gateway), the 8-model chat picker, speech, web search, and the two MCP
+  servers. Volume edits deploy via `cloudron exec` (base64), never via the
+  image — the image never touches `/app/data`.
+- **Env file** — `/app/data/env` (persistent volume). **`start.sh` sources it
+  AFTER the process env, so it wins over `cloudron env`** — that is where
+  `APP_TITLE="Inference Co-op"`, `ALLOW_EMAIL_LOGIN=false` (password login
+  removed; Cloudron SSO is the only login path), and
+  `SCHEDULES_SINGLE_PROCESS=true` must live.
+- **Other env vars** — via `cloudron env`: `TTS_API_KEY`/`STT_API_KEY`
+  (dedicated LiteLLM virtual key, alias `librechat-speech`, $5/30d, audio
+  models only — never the gateway master key), `IMAGE_GEN_OAI_BASEURL` +
+  `IMAGE_GEN_OAI_MODEL` (agent image tool → gateway), `SEARXNG_INSTANCE_URL`.
+- **MCP servers** — `/app/data/mcp/server.py` (docs search, `search_docs`,
+  starts at boot) and `/app/data/mcp/image_server.py` (image generation,
+  `generate_image`, `startup: false`); both mirrored in the repo under
+  `mcp/`, stdlib-only Python.
+- **Branding overlay** (logo, favicons, PWA icons, `<title>`, `custom.css`,
+  `custom2.js`) — baked into a pinned overlay image `FROM` the official
+  appstore image. Deployed tag (last verified 2026-10-19):
+  `registry.inference.coop/co-op/librechat-custom:20261019-fc9b11e`.
+- **Branding CSS/JS** — `custom.css` overrides LibreChat's Tailwind theme
+  variables on `html.light`/`html.dark` using the HOMEPAGE's canonical
+  palette (`--cream #faf8f5`, `--wash #eef4ea`, `--green-deep #5b8c5a`,
+  `--green-mid #6ba86b`, `--ink #2d3327`, `--muted #6b7a62`) and brands
+  the loading flash (`#loading-container`); the login logo is the green
+  pine-dots vector (white-plate variant was invisible on cream — the
+  'flash on load' glitch), cache-busted via `?v=3` in the client bundle.
+  and hides the register/sign-up prompt via
+  `p:has(> a[href*="register"])`; `custom2.js` is a MutationObserver that
+  hides the prompt when React renders it (the auth form renders after
+  `load`). The `interface.theme` block in the yaml remains
+  forward-compatible only (inert on v0.8.8).
+
+## Image generation (two member-facing paths)
+
+1. **MCP tool (default, no key ever)**: the `inference-image` MCP server
+   exposes `generate_image` in the composer's **+** palette — no agent setup,
+   no key prompt. Per-member billing via `customUserVars.IC_API_KEY` (seeded
+   from the portal) + `env: IC_API_KEY: "{{IC_API_KEY}}"`; a system-key
+   fallback (`system-image-fallback`, $5/30d) covers the rare empty-value
+   case. The `customUserVars` gate blocks a not-yet-seeded member until the
+   next sync (≤10 min) — accepted as a short delay over billing everything
+   to the system account.
+2. **Agent tool**: LibreChat's built-in "OpenAI Image Tools"
+   (`IMAGE_GEN_OAI_BASEURL`/`IMAGE_GEN_OAI_MODEL` → gateway; members paste
+   their own key — v0.8.8 limitation).
+
+## Update policy
+
+- The app is **image-sourced** (the `--force` update moved it off the
+  appstore source), so Cloudron will not auto-update it. This is deliberate —
+  same lockdown model as LiteLLM.
+- **Image deploys**: build `--no-cache`, write the new tag into
+  `CloudronManifest.json.dockerImage` (the only editable field), commit, then
+  `cloudron update --app <id> --force`. **`--image` alone is IGNORED** when
+  the manifest sits in the deploy directory. **The remote builder's cache can
+  serve stale COPY layers even when file content changed** (six consecutive
+  "Build successful" runs did on 2026-10-19) — so after every image deploy,
+  verify: the Cloudron API's `manifest.dockerImage`, container-file md5 vs
+  repo md5, the served HTML, and the rendered DOM. Full checklist in the
+  repo README.
+- **When a new appstore package ships**: pull its manifest copy into the repo
+  (`GET /api/v1/apps/<id>` → `.manifest`), repoint `dockerImage` to the
+  rebuilt overlay, bump the Dockerfile `FROM` to the new `manifest.dockerImage`
+  (pinned), rebuild, verify in the sandbox, then update.
+- **Rollback**: repoint `CloudronManifest.json.dockerImage` to the previous
+  tag and rerun `cloudron update --force` (or the original appstore image).
+  Cloudron also snapshots the app before every image update.
+
+## Hard-won pitfalls (do not re-learn these)
+
+- `cloudron update` deploys **`manifest.dockerImage`** from the manifest in
+  the deploy directory — the `--image` flag alone is ignored (and the update
+  output still prints the `--image` value, so it *looks* right). The repo
+  manifest is a byte-identical copy of the OFFICIAL appstore manifest except
+  `dockerImage`; editing anything else in it (or hand-writing one that omits
+  the addons) strips `postgresql`/`mongodb`/`oidc`/`sendmail`/`localstorage`
+  from the app and crash-loops it (`mkdir: Read-only file system`). That is
+  what broke the sandbox on 2026-10-18.
+- The remote builder's layer cache can pin COPY steps to old file content
+  even across "successful" rebuilds — use `--no-cache` after asset edits and
+  confirm by comparing container-file md5 against the repo file.
+- Deploy `librechat.yaml` only via base64 round-trip. A `sed` insert once
+  broke the YAML indentation and LibreChat **silently fell back to defaults**
+  (all custom branding/models/MCP gone) with a single log line.
+- Reading large files from the container over `cloudron exec` with a plain
+  `cat` resets the connection (`ECONNRESET`) — pipe through
+  `gzip | base64 | fold` and decode locally.
+- `cloudron env set` alone does NOT disable the password form —
+  `ALLOW_EMAIL_LOGIN` must be flipped in `/app/data/env` (precedence above).
+- Image generation in LibreChat is the **MCP tool** (composer **+** palette);
+  the agent "OpenAI Image Tools" path also exists. Image models in the chat
+  picker were the source of the old 500 errors and are deliberately excluded.
+- No `branding:` block exists in LibreChat's config schema, and the login
+  page has no `.login-page`/`.primary` classes to target — theming happens by
+  overriding the Tailwind CSS variables on `html.light`/`html.dark`
+  (`--surface-primary`, `--accent-primary`, …) from the overlay `custom.css`.
+
+
+## Per-member key auto-load (no API-key prompts)
+
+Members never enter an API key. The portal provisions each member's LiteLLM
+key at onboarding; the sandbox seeds it into LibreChat's user-key store:
+
+- `GET /admin/librechat/keys` on the portal returns active members' chat keys
+  (dedicated `X-Sync-Token` auth — `LIBRECHAT_SYNC_TOKEN` on both apps).
+- `/app/data/sync-keys.js` (in the LibreChat container, mirrored in the repo
+  as `sync-keys.js`) fetches that feed and seeds the `keys` collection
+  (chat endpoint, `apiKey: "user_provided"`) and `pluginauths`
+  (`mcp:inference-image` → `IC_API_KEY`, injected per-user via
+  `customUserVars` so image generation bills to the member's own team).
+- Members without a sandbox account are skipped until they first log in; run
+  the sync after onboarding waves and on a schedule (the command is in the
+  repo README). Idempotent.
+- Encryption matches LibreChat's own (`CREDS_KEY`/`CREDS_IV`, read from
+  `/proc/1/environ` — exec sessions don't inherit the app env).
+
+> Terminal output redacts values after `apiKey:` as `***` even for harmless
+> strings (`user_provided`, `${TTS_API_KEY}`). Probe lengths/counts instead
+> of trusting redacted output.
+
+## Verification (after any change)
+
+```bash
+cloudron logs --app 4023ccbf-d57a-4500-97df-14de9cf3516e --lines 60 | \
+  grep -iE 'custom config|error|invalid|MCP|schedules'
+curl -s https://librechat-test.inference.coop/api/config   # appTitle, emailLoginEnabled
+```
+
+Expected: `Custom config file loaded`, `[MCP][inference-docs]` initialized
+with tool `search_docs`, `inference-image` registered (its `Tools: undefined`
+boot line is expected — per-user startup), `[schedules] engine started`,
+`appTitle: Inference Co-op`, `emailLoginEnabled: false`. The login page shows
+the co-op logo, the co-op favicon, a single "Login with Inference
+Cooperative" (Cloudron SSO) path with no email/password form, **no
+"Don't have an account?" line**, and the cream/green co-op color scheme.
+
+After an IMAGE deploy, additionally run the four-step verification in the
+repo README (API `manifest.dockerImage`, container md5 vs repo md5, served
+HTML tags, rendered DOM colors).

models.md

@@ -79,7 +79,7 @@ The emoji in the model name marks its **value tier** (the same icons used in the
 | `🌱 greenpt/glm-5.3-flash` | GreenPT | policy | ✅ 100% renewable | — | 1M | $0.11 / $0.44 / $0.022 | Fast GLM model, renewable energy. |
 | `🌱 greenpt/glm-5.3` | GreenPT | policy | ✅ 100% renewable | — | 1M | $1.10 / $4.40 / $0.275 | GLM flagship — strongest agentic model in the green tier. |
 | `🌱 regolo/apertus-70b` | Regolo | policy + ZDR | ✅ 100% renewable | ✅ | 65K | $0.46 / $2.44 / — | Fully open sovereign model (Apache-2.0), large/capable, image understanding. |
-| `🌱 regolo/qwen3.5-122b` | Regolo | policy + ZDR | ✅ 100% renewable | ✅ | 262K | $1.16 / $4.87 / — | Large vision+reasoning MoE (Qwen 3.5, 122B-A10B); biggest context in the roster. |
+| `🌱 regolo/qwen3.5-122b` | Regolo | policy + ZDR | ✅ 100% renewable | ✅ | 262K | $1.16 / $4.87 / — | Large vision+reasoning MoE (Qwen 3.5, 122B-A10B); 262K context. |
 
 Vision is verified working through the co-op gateway (base64 image input via
 `/v1/chat/completions`, standard OpenAI format). In the chat UI, just attach