Sustainable LibreChat setup: official manifest, config layer, brand overlay

- CloudronManifest.json: byte-identical copy of the OFFICIAL appstore manifest
  (the hand-edited one stripped addons and crash-looped the app on 2026-10-18)
- Dockerfile: pinned base (202610150224320000), brand assets only + title sed
- librechat.yaml: schema 1.3.17; chat models only in picker (qwen-image moved
  to the agent image tool); speech via gateway with capped key refs; SearXNG
  web search; inference-docs MCP server; forward-compatible interface.theme
- mcp/server.py: docs MCP server (stdio, stdlib-only, tool search_docs)
- assets/: co-op logo + favicons/PWA icons

README.md

@@ -1,34 +1,133 @@
-# LibreChat sandbox maintenance
+# LibreChat sandbox — Inference Co-op customization
 
-**Current image**
-- Docker image: `cloudron/ai.librechat.cloudronapp:202610150224320000`
-- Deployed as app **0752e430‑aa36‑436e‑8668‑ef18da474f47** (LibreChat sandbox at `https://librechat-test.inference.coop`).
+Source of truth for the LibreChat evaluation instance at
+**librechat-test.inference.coop** (Cloudron app `4023ccbf-d57a-4500-97df-14de9cf3516e`,
+appstore package `ai.librechat.cloudronapp@1.4.1`, which bundles **LibreChat
+v0.8.8** on image `cloudron/ai.librechat.cloudronapp:202610150224320000`).
 
-**Automatic updates**
-- The "Automatic updates" toggle in the Cloudron UI has been turned **off** (as of 2026‑10‑18). This prevents the sandbox from pulling newer Docker images automatically.
+This repo exists so the sandbox never drifts from what is deployed: the config
+file here is byte-identical to `/app/data/librechat.yaml`, the MCP server here
+is byte-identical to `/app/data/mcp/server.py`, and the Docker image is built
+from this directory.
 
-**Image‑generation button**
-- Added the `regolo/qwen-image` model to `modelSpecs.list` in `librechat.yaml`. This makes LibreChat display a permanent **📷 Image generation** button for users, allowing easy prompt‑based image creation via the gateway.
+## Architecture: what lives where
 
-**Version‑pinning workflow**
-1. **Create a version tag** when a new LibreChat release is approved (e.g. `cloudron/ai.librechat.cloudronapp:202610250100000000`).
-2. **Push the tagged image** to the co‑op registry (e.g. `registry.inference.coop/librechat:sandbox‑2026‑10‑25`).
-   ```bash
-   docker pull cloudron/ai.librechat.cloudronapp:<new‑tag>
-   docker tag cloudron/ai.librechat.cloudronapp:<new‑tag> registry.inference.coop/librechat:sandbox‑<date>
-   docker login registry.inference.coop   # use registry‑bot credentials
-   docker push registry.inference.coop/librechat:sandbox‑<date>
-   ```
-3. **Redeploy** the sandbox with the pinned tag:
-   ```bash
-   cloudron update --app 0752e430-aa36-436e-8668-ef18da474f47 \
-       --image registry.inference.coop/librechat:sandbox‑<date>
-   ```
-4. **Verify** the app is running (`cloudron status --app <id>`) and that the image field reflects the new tag.
-5. **Record** the change in this document (date, tag, reason) and optionally open a GitHub/Gitea issue linking to the upstream LibreChat release.
+| Layer | Location | Survives image update? | Repo mirror |
+|---|---|---|---|
+| App config | `/app/data/librechat.yaml` (volume) | yes | `librechat.yaml` |
+| Env vars (env-file wins!) | `/app/data/env` (volume, sourced LAST by start.sh) | yes | changed lines documented below |
+| Env vars (Cloudron-managed) | `cloudron env` (process env) | yes | documented below |
+| MCP docs server | `/app/data/mcp/server.py` (volume) | yes | `mcp/server.py` |
+| Brand assets (logo, favicons) | baked into image overlay | no — rebuilt per image | `assets/` |
+| CloudronManifest.json | repo only (used by `cloudron update`) | n/a | byte-identical copy of the OFFICIAL appstore manifest |
 
-**Rollback**
-- To roll back, repeat step 3 with the previous image tag.
+**Precedence gotcha:** `start.sh` does `source /run/librechat/env` AFTER the
+process env, and `/app/data/env` is appended into it — so a value defined in
+`/app/data/env` **overrides** the same `cloudron env` variable. That is why
+`ALLOW_EMAIL_LOGIN=false` must be set in `/app/data/env`, not (only) via
+`cloudron env`.
 
-**Future maintainers**
-- All steps are captured in this doc and in the corresponding issue in `co‑op/support` (Issue #24). The `co‑op/docs` repository should contain this file; keep it up‑to‑date whenever the sandbox image changes or the automatic‑update flag is toggled.
+## Deploy a config change (no image rebuild needed)
+
+```bash
+B64=$(base64 -w0 librechat.yaml)
+cloudron --server my.inference.coop --token "$CLOUDRON_TOKEN" exec \
+  --app 4023ccbf-d57a-4500-97df-14de9cf3516e \
+  -- bash -c "echo '$B64' | base64 -d > /app/data/librechat.yaml"
+cloudron --server my.inference.coop --token "$CLOUDRON_TOKEN" restart \
+  --app 4023ccbf-d57a-4500-97df-14de9cf3516e
+```
+
+Always `diff` the repo file against the live one after deploying. Always
+deploy via base64 — heredocs and `sed` inserts have corrupted this file twice
+(literal `$(...)` / bad indentation → LibreChat silently falls back to
+defaults with only one log line).
+
+Verify after every change:
+
+```bash
+cloudron logs --app <id> --lines 60 | grep -iE 'custom config|error|invalid|MCP'
+curl -s https://librechat-test.inference.coop/api/config   # appTitle, emailLoginEnabled, socialLogins
+```
+
+## Build + deploy the image (brand assets only)
+
+```bash
+cd /tmp/librechat-custom          # THIS dir: cloudron update reads CloudronManifest.json here
+git add -A && git commit -m "..." && git push
+SHA=$(git rev-parse --short HEAD)
+cloudron builder build --repository registry.inference.coop/co-op/librechat-custom \
+  --tag 20261019-$SHA
+cloudron --server my.inference.coop --token "$CLOUDRON_TOKEN" update \
+  --app 4023ccbf-d57a-4500-97df-14de9cf3516e \
+  --image registry.inference.coop/co-op/librechat-custom:20261019-$SHA
+```
+
+Rollback: rerun the `update` with the previous tag (or the appstore image
+`cloudron/ai.librechat.cloudronapp:<original-tag>`).
+
+**NEVER edit `CloudronManifest.json`.** `cloudron update` reads it from this
+directory and applies it to the app. A hand-edited manifest that omits the
+addons (`postgresql`, `mongodb`, `oidc`, `sendmail`, `localstorage`) strips
+them from the running app and crash-loops it (`mkdir: Read-only file system`).
+That is what broke the sandbox on 2026-10-18 — the manifest, not the overlay.
+When the appstore package updates, refresh this file with a new copy pulled
+from the Cloudron API (`GET /api/v1/apps/<id> → .manifest`) and bump the
+`FROM` tag in the Dockerfile to the new `manifest.dockerImage`.
+
+## Env vars (live state)
+
+Set in `/app/data/env` (volume — these WIN over cloudron env):
+
+- `APP_TITLE="Inference Co-op"` (was `"LibreChat"`)
+- `ALLOW_EMAIL_LOGIN=false` (was `true`) — password login removed; Cloudron
+  SSO (OIDC) is the only login path
+- `SCHEDULES_SINGLE_PROCESS=true` — sandbox runs a single replica; silences
+  the scheduler's Redis guard error
+
+Set via `cloudron env set` (process env):
+
+- `TTS_API_KEY` / `STT_API_KEY` — **dedicated LiteLLM virtual key** (alias
+  `librechat-speech`, $5 / 30d budget, restricted to `tinfoil/voxtral-tts` +
+  `tinfoil/whisper-large-v3-turbo`). Regenerate with the gateway master key:
+  `POST /key/generate {"key_alias":"librechat-speech","models":[...],"max_budget":5,"budget_duration":"30d"}`
+  — never put the master key here.
+- `IMAGE_GEN_OAI_BASEURL=https://gateway.inference.coop/v1` and
+  `IMAGE_GEN_OAI_MODEL=regolo/qwen-image` — the agent "OpenAI Image Tools"
+  points at the co-op gateway. No `IMAGE_GEN_OAI_API_KEY`: members enter their
+  own co-op key in the tool UI (per-member billing, same pattern as the chat
+  endpoint's `apiKey: user_provided`).
+- `SEARXNG_INSTANCE_URL=https://search.inference.coop` — referenced from the
+  YAML as `${SEARXNG_INSTANCE_URL}` (self-hosted search; keyless JSON API).
+
+## Feature notes (LibreChat v0.8.8)
+
+- **Chat models**: custom endpoint "Inference Co-op" → gateway, per-member
+  keys, 8 chat modelSpecs (emoji convention shared with models.md).
+  `regolo/qwen-image` is deliberately NOT in the chat picker (image models
+  are not chat models; chat-calling it was the source of the old 500s).
+- **Image generation**: LibreChat exposes image gen through an **Agent with
+  the "OpenAI Image Tools" tool**, not a chat-picker button. Members create
+  an agent, add the tool, and paste their co-op key on first use.
+- **MCP**: `inference-docs` (stdio, `/app/data/mcp/server.py`, tool
+  `search_docs`) is admin-configured; members may add their own via the MCP
+  Settings panel (the "bring your own tools" pattern). Permissions seeded via
+  `interface.mcpServers` (use/create true, share false).
+- **Speech**: TTS `tinfoil/voxtral-tts` (voice `casual_female`), STT
+  `tinfoil/whisper-large-v3-turbo`, both via the gateway with the capped key.
+- **Web search**: SearXNG (self-hosted) + keyless Keenable scraper, no reranker.
+- **Colors**: `interface.theme` in the YAML is **forward-compatible only** —
+  the theme engine ships in LibreChat releases AFTER 0.8.8 (merged to dev,
+  not tagged). The block is silently ignored today and activates by itself
+  when the appstore package bundles a newer LibreChat. No other runtime
+  theming exists in 0.8.8.
+
+## Known state / limitations
+
+- `patches/`, `logo.png`, `logo-coop.png`, `logo.svg` (repo root) are legacy
+  leftovers from the abandoned 2026-10-18 hand-built-image attempt — unused,
+  pending cleanup.
+- The `[schedules]` Redis guard error is resolved by
+  `SCHEDULES_SINGLE_PROCESS=true` (above).
+- The `RAG API not reachable` warning is expected (no rag_api addon); it only
+  affects LibreChat's own file-RAG, which we do not use.