97 lines
5.3 KiB
Markdown
97 lines
5.3 KiB
Markdown
# LibreChat sandbox maintenance
|
||
|
||
The LibreChat evaluation instance at **librechat-test.inference.coop**
|
||
(Cloudron app `4023ccbf-d57a-4500-97df-14de9cf3516e`, appstore package
|
||
`ai.librechat.cloudronapp@1.4.1`, which bundles **LibreChat v0.8.8**).
|
||
|
||
The single source of truth for the deployment is the repo
|
||
**[co-op/librechat-custom](https://git.inference.coop/co-op/librechat-custom)**
|
||
— its README is the full runbook (config deploy, image build/deploy, rollback,
|
||
env vars, known state). This doc is the operator-facing summary.
|
||
|
||
## Architecture: what lives where
|
||
|
||
- **App config** — `/app/data/librechat.yaml` (persistent volume, mirrored
|
||
byte-identically in the repo). Holds the "Inference Co-op" custom endpoint
|
||
(gateway + per-member keys), the 8-model chat picker, speech, web search,
|
||
the docs MCP server, and a forward-compatible theme block (see below).
|
||
- **Env file** — `/app/data/env` (persistent volume). **`start.sh` sources it
|
||
AFTER the process env, so it wins over `cloudron env`** — that is where
|
||
`APP_TITLE="Inference Co-op"`, `ALLOW_EMAIL_LOGIN=false` (password login
|
||
removed; Cloudron SSO is the only login path), and
|
||
`SCHEDULES_SINGLE_PROCESS=true` must live.
|
||
- **Other env vars** — via `cloudron env`: `TTS_API_KEY`/`STT_API_KEY`
|
||
(dedicated LiteLLM virtual key, alias `librechat-speech`, $5/30d, audio
|
||
models only — never the gateway master key), `IMAGE_GEN_OAI_BASEURL` +
|
||
`IMAGE_GEN_OAI_MODEL` (agent image tool → gateway), `SEARXNG_INSTANCE_URL`.
|
||
- **MCP servers** — `/app/data/mcp/server.py` (docs search, `search_docs`,
|
||
starts at boot) and `/app/data/mcp/image_server.py` (image generation,
|
||
`generate_image`, per-user startup — see below); both mirrored in the repo
|
||
under `mcp/`, stdlib-only Python.
|
||
- **Brand assets** (login logo, favicons, PWA icons, page title) — baked into
|
||
a pinned overlay image `FROM` the official appstore image, built from the
|
||
repo via `cloudron builder build` and deployed with
|
||
`cloudron update --image ... --force`. Deployed tag:
|
||
`registry.inference.coop/co-op/librechat-custom:20261019-5e214e4`.
|
||
|
||
## Image generation (two member-facing paths)
|
||
|
||
1. **MCP tool (easiest)**: the `inference-image` MCP server exposes
|
||
`generate_image` in the composer's **+** palette — no agent setup. Each
|
||
member's own co-op API key is injected per-user (LibreChat
|
||
`customUserVars` → `env`, `startup: false` because the key only exists
|
||
after the member saves it in the MCP panel), so images bill to the
|
||
member's own balance (~1–2¢ per 1024² image via `regolo/qwen-image`).
|
||
2. **Agent tool**: LibreChat's built-in "OpenAI Image Tools"
|
||
(`IMAGE_GEN_OAI_BASEURL`/`IMAGE_GEN_OAI_MODEL` → gateway; per-user keys).
|
||
|
||
## Update policy
|
||
|
||
- The app is now **image-sourced** (the `--force` update moved it off the
|
||
appstore source), so Cloudron will not auto-update it. This is deliberate —
|
||
same lockdown model as LiteLLM.
|
||
- **When a new appstore package ships**: pull its manifest copy into the repo
|
||
(`GET /api/v1/apps/<id> → .manifest`), bump the Dockerfile `FROM` to the new
|
||
`manifest.dockerImage`, rebuild, verify in the sandbox, then update. A
|
||
newer LibreChat also activates the already-present `interface.theme` block
|
||
(co-op colors) automatically — the theme engine does not exist in v0.8.8
|
||
and the block is silently ignored today.
|
||
- **Rollback**: `cloudron update --app <id> --image <previous-tag> --force`
|
||
(or the original appstore image). Cloudron also snapshots the app before
|
||
every image update.
|
||
|
||
## Hard-won pitfalls (do not re-learn these)
|
||
|
||
- `cloudron update` deploys **`manifest.dockerImage`** from the manifest in
|
||
the deploy directory — the `--image` flag alone is ignored. The repo
|
||
manifest is a byte-identical copy of the OFFICIAL appstore manifest except
|
||
`dockerImage`; editing anything else in it (or hand-writing one that omits
|
||
the addons) strips `postgresql`/`mongodb`/`oidc`/`sendmail`/`localstorage`
|
||
from the app and crash-loops it (`mkdir: Read-only file system`). That is
|
||
what broke the sandbox on 2026-10-18.
|
||
- Deploy `librechat.yaml` only via base64 round-trip. A `sed` insert once
|
||
broke the YAML indentation and LibreChat **silently fell back to defaults**
|
||
(all custom branding/models/MCP gone) with a single log line.
|
||
- `cloudron env set` alone does NOT disable the password form —
|
||
`ALLOW_EMAIL_LOGIN` must be flipped in `/app/data/env` (precedence above).
|
||
- Image generation in LibreChat is an **Agent tool ("OpenAI Image Tools")**,
|
||
not a chat-picker button; image models in the chat picker were the source
|
||
of the old 500 errors and are deliberately excluded.
|
||
- No `branding:` block exists in LibreChat's config schema; the login logo
|
||
and favicons come from the image overlay.
|
||
|
||
## Verification (after any change)
|
||
|
||
```bash
|
||
cloudron logs --app 4023ccbf-d57a-4500-97df-14de9cf3516e --lines 60 | \
|
||
grep -iE 'custom config|error|invalid|MCP|schedules'
|
||
curl -s https://librechat-test.inference.coop/api/config # appTitle, emailLoginEnabled
|
||
```
|
||
|
||
Expected: `Custom config file loaded`, `[MCP][inference-docs]` initialized
|
||
with tool `search_docs`, `inference-image` registered (its `Tools: undefined`
|
||
boot line is expected — per-user startup), `[schedules] engine started`,
|
||
`appTitle: Inference Co-op`, `emailLoginEnabled: false`. The login page shows
|
||
the co-op logo, the co-op favicon, and a single "Login with Inference
|
||
Cooperative" (Cloudron SSO) path with no email/password form.
|