docs/maintenance/librechat-sandbox.md

LibreChat sandbox maintenance

The LibreChat evaluation instance at librechat-test.inference.coop (Cloudron app 4023ccbf-d57a-4500-97df-14de9cf3516e, appstore package ai.librechat.cloudronapp@1.4.1, which bundles LibreChat v0.8.8).

The single source of truth for the deployment is the repo co-op/librechat-custom — its README is the full runbook (config deploy, image build/deploy, rollback, env vars, known state). This doc is the operator-facing summary.

Architecture: what lives where

• App config — /app/data/librechat.yaml (persistent volume, mirrored byte-identically in the repo). Holds the "Inference Co-op" custom endpoint (gateway + per-member keys), the 8-model chat picker, speech, web search, the docs MCP server, and a forward-compatible theme block (see below).
• Env file — /app/data/env (persistent volume). start.sh sources it AFTER the process env, so it wins over cloudron env — that is where APP_TITLE="Inference Co-op", ALLOW_EMAIL_LOGIN=false (password login removed; Cloudron SSO is the only login path), and SCHEDULES_SINGLE_PROCESS=true must live.
• Other env vars — via cloudron env: TTS_API_KEY/STT_API_KEY (dedicated LiteLLM virtual key, alias librechat-speech, $5/30d, audio models only — never the gateway master key), IMAGE_GEN_OAI_BASEURL + IMAGE_GEN_OAI_MODEL (agent image tool → gateway), SEARXNG_INSTANCE_URL.
• Docs MCP server — /app/data/mcp/server.py (volume, mirrored in the repo at mcp/server.py; stdlib-only Python; tool search_docs).
• Brand assets (login logo, favicons, PWA icons, page title) — baked into a pinned overlay image FROM the official appstore image, built from the repo via cloudron builder build and deployed with cloudron update --image ... --force. Deployed tag: registry.inference.coop/co-op/librechat-custom:20261019-070dec3.

Update policy

• The app is now image-sourced (the --force update moved it off the appstore source), so Cloudron will not auto-update it. This is deliberate — same lockdown model as LiteLLM.
• When a new appstore package ships: pull its manifest copy into the repo (GET /api/v1/apps/<id> → .manifest), bump the Dockerfile FROM to the new manifest.dockerImage, rebuild, verify in the sandbox, then update. A newer LibreChat also activates the already-present interface.theme block (co-op colors) automatically — the theme engine does not exist in v0.8.8 and the block is silently ignored today.
• Rollback: cloudron update --app <id> --image <previous-tag> --force (or the original appstore image). Cloudron also snapshots the app before every image update.

Hard-won pitfalls (do not re-learn these)

• cloudron update deploys manifest.dockerImage from the manifest in the deploy directory — the --image flag alone is ignored. The repo manifest is a byte-identical copy of the OFFICIAL appstore manifest except dockerImage; editing anything else in it (or hand-writing one that omits the addons) strips postgresql/mongodb/oidc/sendmail/localstorage from the app and crash-loops it (mkdir: Read-only file system). That is what broke the sandbox on 2026-10-18.
• Deploy librechat.yaml only via base64 round-trip. A sed insert once broke the YAML indentation and LibreChat silently fell back to defaults (all custom branding/models/MCP gone) with a single log line.
• cloudron env set alone does NOT disable the password form — ALLOW_EMAIL_LOGIN must be flipped in /app/data/env (precedence above).
• Image generation in LibreChat is an Agent tool ("OpenAI Image Tools"), not a chat-picker button; image models in the chat picker were the source of the old 500 errors and are deliberately excluded.
• No branding: block exists in LibreChat's config schema; the login logo and favicons come from the image overlay.

Verification (after any change)

cloudron logs --app 4023ccbf-d57a-4500-97df-14de9cf3516e --lines 60 | \
  grep -iE 'custom config|error|invalid|MCP|schedules'
curl -s https://librechat-test.inference.coop/api/config   # appTitle, emailLoginEnabled

Expected: Custom config file loaded, [MCP] Initialized with 1 configured server and 1 tool, [schedules] engine started, appTitle: Inference Co-op, emailLoginEnabled: false. The login page shows the co-op logo, the co-op favicon, and a single "Login with Inference Cooperative" (Cloudron SSO) path with no email/password form.