# LibreChat sandbox maintenance

The LibreChat evaluation instance at **librechat-test.inference.coop**
(Cloudron app `4023ccbf-d57a-4500-97df-14de9cf3516e`, appstore package
`ai.librechat.cloudronapp@1.4.1`, which bundles **LibreChat v0.8.8**).

The single source of truth for the deployment is the repo
**[co-op/librechat-custom](https://git.inference.coop/co-op/librechat-custom)**
— its README is the full runbook (config deploy, image build/deploy, rollback,
env vars, known state). This doc is the operator-facing summary.

## Architecture: what lives where

- **App config** — `/app/data/librechat.yaml` (persistent volume, mirrored
  byte-identically in the repo). Holds the "Inference Co-op" custom endpoint
  (gateway + per-member keys), the 8-model chat picker, speech, web search,
  the docs MCP server, and a forward-compatible theme block (see below).
- **Env file** — `/app/data/env` (persistent volume). **`start.sh` sources it
  AFTER the process env, so it wins over `cloudron env`** — that is where
  `APP_TITLE="Inference Co-op"`, `ALLOW_EMAIL_LOGIN=false` (password login
  removed; Cloudron SSO is the only login path), and
  `SCHEDULES_SINGLE_PROCESS=true` must live.
- **Other env vars** — via `cloudron env`: `TTS_API_KEY`/`STT_API_KEY`
  (dedicated LiteLLM virtual key, alias `librechat-speech`, $5/30d, audio
  models only — never the gateway master key), `IMAGE_GEN_OAI_BASEURL` +
  `IMAGE_GEN_OAI_MODEL` (agent image tool → gateway), `SEARXNG_INSTANCE_URL`.
- **Docs MCP server** — `/app/data/mcp/server.py` (volume, mirrored in the
  repo at `mcp/server.py`; stdlib-only Python; tool `search_docs`).
- **Brand assets** (login logo, favicons, PWA icons, page title) — baked into
  a pinned overlay image `FROM` the official appstore image, built from the
  repo via `cloudron builder build` and deployed with
  `cloudron update --image ... --force`. Deployed tag:
  `registry.inference.coop/co-op/librechat-custom:20261019-070dec3`.

## Update policy

- The app is now **image-sourced** (the `--force` update moved it off the
  appstore source), so Cloudron will not auto-update it. This is deliberate —
  same lockdown model as LiteLLM.
- **When a new appstore package ships**: pull its manifest copy into the repo
  (`GET /api/v1/apps/<id> → .manifest`), bump the Dockerfile `FROM` to the new
  `manifest.dockerImage`, rebuild, verify in the sandbox, then update. A
  newer LibreChat also activates the already-present `interface.theme` block
  (co-op colors) automatically — the theme engine does not exist in v0.8.8
  and the block is silently ignored today.
- **Rollback**: `cloudron update --app <id> --image <previous-tag> --force`
  (or the original appstore image). Cloudron also snapshots the app before
  every image update.

## Hard-won pitfalls (do not re-learn these)

- `cloudron update` deploys **`manifest.dockerImage`** from the manifest in
  the deploy directory — the `--image` flag alone is ignored. The repo
  manifest is a byte-identical copy of the OFFICIAL appstore manifest except
  `dockerImage`; editing anything else in it (or hand-writing one that omits
  the addons) strips `postgresql`/`mongodb`/`oidc`/`sendmail`/`localstorage`
  from the app and crash-loops it (`mkdir: Read-only file system`). That is
  what broke the sandbox on 2026-10-18.
- Deploy `librechat.yaml` only via base64 round-trip. A `sed` insert once
  broke the YAML indentation and LibreChat **silently fell back to defaults**
  (all custom branding/models/MCP gone) with a single log line.
- `cloudron env set` alone does NOT disable the password form —
  `ALLOW_EMAIL_LOGIN` must be flipped in `/app/data/env` (precedence above).
- Image generation in LibreChat is an **Agent tool ("OpenAI Image Tools")**,
  not a chat-picker button; image models in the chat picker were the source
  of the old 500 errors and are deliberately excluded.
- No `branding:` block exists in LibreChat's config schema; the login logo
  and favicons come from the image overlay.

## Verification (after any change)

```bash
cloudron logs --app 4023ccbf-d57a-4500-97df-14de9cf3516e --lines 60 | \
  grep -iE 'custom config|error|invalid|MCP|schedules'
curl -s https://librechat-test.inference.coop/api/config   # appTitle, emailLoginEnabled
```

Expected: `Custom config file loaded`, `[MCP] Initialized with 1 configured
server and 1 tool`, `[schedules] engine started`, `appTitle: Inference
Co-op`, `emailLoginEnabled: false`. The login page shows the co-op logo, the
co-op favicon, and a single "Login with Inference Cooperative" (Cloudron SSO)
path with no email/password form.