Nathan Schneider
070dec3d29
- CloudronManifest.json: byte-identical copy of the OFFICIAL appstore manifest (the hand-edited one stripped addons and crash-looped the app on 2026-10-18) - Dockerfile: pinned base (202610150224320000), brand assets only + title sed - librechat.yaml: schema 1.3.17; chat models only in picker (qwen-image moved to the agent image tool); speech via gateway with capped key refs; SearXNG web search; inference-docs MCP server; forward-compatible interface.theme - mcp/server.py: docs MCP server (stdio, stdlib-only, tool search_docs) - assets/: co-op logo + favicons/PWA icons
134 lines
6.7 KiB
Markdown
134 lines
6.7 KiB
Markdown
# LibreChat sandbox — Inference Co-op customization
|
|
|
|
Source of truth for the LibreChat evaluation instance at
|
|
**librechat-test.inference.coop** (Cloudron app `4023ccbf-d57a-4500-97df-14de9cf3516e`,
|
|
appstore package `ai.librechat.cloudronapp@1.4.1`, which bundles **LibreChat
|
|
v0.8.8** on image `cloudron/ai.librechat.cloudronapp:202610150224320000`).
|
|
|
|
This repo exists so the sandbox never drifts from what is deployed: the config
|
|
file here is byte-identical to `/app/data/librechat.yaml`, the MCP server here
|
|
is byte-identical to `/app/data/mcp/server.py`, and the Docker image is built
|
|
from this directory.
|
|
|
|
## Architecture: what lives where
|
|
|
|
| Layer | Location | Survives image update? | Repo mirror |
|
|
|---|---|---|---|
|
|
| App config | `/app/data/librechat.yaml` (volume) | yes | `librechat.yaml` |
|
|
| Env vars (env-file wins!) | `/app/data/env` (volume, sourced LAST by start.sh) | yes | changed lines documented below |
|
|
| Env vars (Cloudron-managed) | `cloudron env` (process env) | yes | documented below |
|
|
| MCP docs server | `/app/data/mcp/server.py` (volume) | yes | `mcp/server.py` |
|
|
| Brand assets (logo, favicons) | baked into image overlay | no — rebuilt per image | `assets/` |
|
|
| CloudronManifest.json | repo only (used by `cloudron update`) | n/a | byte-identical copy of the OFFICIAL appstore manifest |
|
|
|
|
**Precedence gotcha:** `start.sh` does `source /run/librechat/env` AFTER the
|
|
process env, and `/app/data/env` is appended into it — so a value defined in
|
|
`/app/data/env` **overrides** the same `cloudron env` variable. That is why
|
|
`ALLOW_EMAIL_LOGIN=false` must be set in `/app/data/env`, not (only) via
|
|
`cloudron env`.
|
|
|
|
## Deploy a config change (no image rebuild needed)
|
|
|
|
```bash
|
|
B64=$(base64 -w0 librechat.yaml)
|
|
cloudron --server my.inference.coop --token "$CLOUDRON_TOKEN" exec \
|
|
--app 4023ccbf-d57a-4500-97df-14de9cf3516e \
|
|
-- bash -c "echo '$B64' | base64 -d > /app/data/librechat.yaml"
|
|
cloudron --server my.inference.coop --token "$CLOUDRON_TOKEN" restart \
|
|
--app 4023ccbf-d57a-4500-97df-14de9cf3516e
|
|
```
|
|
|
|
Always `diff` the repo file against the live one after deploying. Always
|
|
deploy via base64 — heredocs and `sed` inserts have corrupted this file twice
|
|
(literal `$(...)` / bad indentation → LibreChat silently falls back to
|
|
defaults with only one log line).
|
|
|
|
Verify after every change:
|
|
|
|
```bash
|
|
cloudron logs --app <id> --lines 60 | grep -iE 'custom config|error|invalid|MCP'
|
|
curl -s https://librechat-test.inference.coop/api/config # appTitle, emailLoginEnabled, socialLogins
|
|
```
|
|
|
|
## Build + deploy the image (brand assets only)
|
|
|
|
```bash
|
|
cd /tmp/librechat-custom # THIS dir: cloudron update reads CloudronManifest.json here
|
|
git add -A && git commit -m "..." && git push
|
|
SHA=$(git rev-parse --short HEAD)
|
|
cloudron builder build --repository registry.inference.coop/co-op/librechat-custom \
|
|
--tag 20261019-$SHA
|
|
cloudron --server my.inference.coop --token "$CLOUDRON_TOKEN" update \
|
|
--app 4023ccbf-d57a-4500-97df-14de9cf3516e \
|
|
--image registry.inference.coop/co-op/librechat-custom:20261019-$SHA
|
|
```
|
|
|
|
Rollback: rerun the `update` with the previous tag (or the appstore image
|
|
`cloudron/ai.librechat.cloudronapp:<original-tag>`).
|
|
|
|
**NEVER edit `CloudronManifest.json`.** `cloudron update` reads it from this
|
|
directory and applies it to the app. A hand-edited manifest that omits the
|
|
addons (`postgresql`, `mongodb`, `oidc`, `sendmail`, `localstorage`) strips
|
|
them from the running app and crash-loops it (`mkdir: Read-only file system`).
|
|
That is what broke the sandbox on 2026-10-18 — the manifest, not the overlay.
|
|
When the appstore package updates, refresh this file with a new copy pulled
|
|
from the Cloudron API (`GET /api/v1/apps/<id> → .manifest`) and bump the
|
|
`FROM` tag in the Dockerfile to the new `manifest.dockerImage`.
|
|
|
|
## Env vars (live state)
|
|
|
|
Set in `/app/data/env` (volume — these WIN over cloudron env):
|
|
|
|
- `APP_TITLE="Inference Co-op"` (was `"LibreChat"`)
|
|
- `ALLOW_EMAIL_LOGIN=false` (was `true`) — password login removed; Cloudron
|
|
SSO (OIDC) is the only login path
|
|
- `SCHEDULES_SINGLE_PROCESS=true` — sandbox runs a single replica; silences
|
|
the scheduler's Redis guard error
|
|
|
|
Set via `cloudron env set` (process env):
|
|
|
|
- `TTS_API_KEY` / `STT_API_KEY` — **dedicated LiteLLM virtual key** (alias
|
|
`librechat-speech`, $5 / 30d budget, restricted to `tinfoil/voxtral-tts` +
|
|
`tinfoil/whisper-large-v3-turbo`). Regenerate with the gateway master key:
|
|
`POST /key/generate {"key_alias":"librechat-speech","models":[...],"max_budget":5,"budget_duration":"30d"}`
|
|
— never put the master key here.
|
|
- `IMAGE_GEN_OAI_BASEURL=https://gateway.inference.coop/v1` and
|
|
`IMAGE_GEN_OAI_MODEL=regolo/qwen-image` — the agent "OpenAI Image Tools"
|
|
points at the co-op gateway. No `IMAGE_GEN_OAI_API_KEY`: members enter their
|
|
own co-op key in the tool UI (per-member billing, same pattern as the chat
|
|
endpoint's `apiKey: user_provided`).
|
|
- `SEARXNG_INSTANCE_URL=https://search.inference.coop` — referenced from the
|
|
YAML as `${SEARXNG_INSTANCE_URL}` (self-hosted search; keyless JSON API).
|
|
|
|
## Feature notes (LibreChat v0.8.8)
|
|
|
|
- **Chat models**: custom endpoint "Inference Co-op" → gateway, per-member
|
|
keys, 8 chat modelSpecs (emoji convention shared with models.md).
|
|
`regolo/qwen-image` is deliberately NOT in the chat picker (image models
|
|
are not chat models; chat-calling it was the source of the old 500s).
|
|
- **Image generation**: LibreChat exposes image gen through an **Agent with
|
|
the "OpenAI Image Tools" tool**, not a chat-picker button. Members create
|
|
an agent, add the tool, and paste their co-op key on first use.
|
|
- **MCP**: `inference-docs` (stdio, `/app/data/mcp/server.py`, tool
|
|
`search_docs`) is admin-configured; members may add their own via the MCP
|
|
Settings panel (the "bring your own tools" pattern). Permissions seeded via
|
|
`interface.mcpServers` (use/create true, share false).
|
|
- **Speech**: TTS `tinfoil/voxtral-tts` (voice `casual_female`), STT
|
|
`tinfoil/whisper-large-v3-turbo`, both via the gateway with the capped key.
|
|
- **Web search**: SearXNG (self-hosted) + keyless Keenable scraper, no reranker.
|
|
- **Colors**: `interface.theme` in the YAML is **forward-compatible only** —
|
|
the theme engine ships in LibreChat releases AFTER 0.8.8 (merged to dev,
|
|
not tagged). The block is silently ignored today and activates by itself
|
|
when the appstore package bundles a newer LibreChat. No other runtime
|
|
theming exists in 0.8.8.
|
|
|
|
## Known state / limitations
|
|
|
|
- `patches/`, `logo.png`, `logo-coop.png`, `logo.svg` (repo root) are legacy
|
|
leftovers from the abandoned 2026-10-18 hand-built-image attempt — unused,
|
|
pending cleanup.
|
|
- The `[schedules]` Redis guard error is resolved by
|
|
`SCHEDULES_SINGLE_PROCESS=true` (above).
|
|
- The `RAG API not reachable` warning is expected (no rag_api addon); it only
|
|
affects LibreChat's own file-RAG, which we do not use.
|