Nathan Schneider
5e214e4d7e
- mcp/image_server.py: generate_image tool via the co-op gateway; per-member key injected through customUserVars (IC_API_KEY) -> env; verified standalone (handshake, tools/list, real 1024x1024 + 1536x1024 images, error paths) - librechat.yaml: register inference-image with startup:false (per-user credentials), 120s timeout - assets/: real favicon sizes (16/32/192), opaque apple-touch 180, padded maskable 512
142 lines
7.3 KiB
Markdown
142 lines
7.3 KiB
Markdown
# LibreChat sandbox — Inference Co-op customization
|
||
|
||
Source of truth for the LibreChat evaluation instance at
|
||
**librechat-test.inference.coop** (Cloudron app `4023ccbf-d57a-4500-97df-14de9cf3516e`,
|
||
appstore package `ai.librechat.cloudronapp@1.4.1`, which bundles **LibreChat
|
||
v0.8.8** on image `cloudron/ai.librechat.cloudronapp:202610150224320000`).
|
||
|
||
This repo exists so the sandbox never drifts from what is deployed: the config
|
||
file here is byte-identical to `/app/data/librechat.yaml`, the MCP server here
|
||
is byte-identical to `/app/data/mcp/server.py`, and the Docker image is built
|
||
from this directory.
|
||
|
||
## Architecture: what lives where
|
||
|
||
| Layer | Location | Survives image update? | Repo mirror |
|
||
|---|---|---|---|
|
||
| App config | `/app/data/librechat.yaml` (volume) | yes | `librechat.yaml` |
|
||
| Env vars (env-file wins!) | `/app/data/env` (volume, sourced LAST by start.sh) | yes | changed lines documented below |
|
||
| Env vars (Cloudron-managed) | `cloudron env` (process env) | yes | documented below |
|
||
| MCP servers | `/app/data/mcp/server.py` + `/app/data/mcp/image_server.py` (volume) | yes | `mcp/` |
|
||
| Brand assets (logo, favicons) | baked into image overlay | no — rebuilt per image | `assets/` |
|
||
| CloudronManifest.json | repo only (used by `cloudron update`) | n/a | byte-identical copy of the OFFICIAL appstore manifest |
|
||
|
||
**Precedence gotcha:** `start.sh` does `source /run/librechat/env` AFTER the
|
||
process env, and `/app/data/env` is appended into it — so a value defined in
|
||
`/app/data/env` **overrides** the same `cloudron env` variable. That is why
|
||
`ALLOW_EMAIL_LOGIN=false` must be set in `/app/data/env`, not (only) via
|
||
`cloudron env`.
|
||
|
||
## Deploy a config change (no image rebuild needed)
|
||
|
||
```bash
|
||
B64=$(base64 -w0 librechat.yaml)
|
||
cloudron --server my.inference.coop --token "$CLOUDRON_TOKEN" exec \
|
||
--app 4023ccbf-d57a-4500-97df-14de9cf3516e \
|
||
-- bash -c "echo '$B64' | base64 -d > /app/data/librechat.yaml"
|
||
cloudron --server my.inference.coop --token "$CLOUDRON_TOKEN" restart \
|
||
--app 4023ccbf-d57a-4500-97df-14de9cf3516e
|
||
```
|
||
|
||
Always `diff` the repo file against the live one after deploying. Always
|
||
deploy via base64 — heredocs and `sed` inserts have corrupted this file twice
|
||
(literal `$(...)` / bad indentation → LibreChat silently falls back to
|
||
defaults with only one log line).
|
||
|
||
Verify after every change:
|
||
|
||
```bash
|
||
cloudron logs --app <id> --lines 60 | grep -iE 'custom config|error|invalid|MCP'
|
||
curl -s https://librechat-test.inference.coop/api/config # appTitle, emailLoginEnabled, socialLogins
|
||
```
|
||
|
||
## Build + deploy the image (brand assets only)
|
||
|
||
```bash
|
||
cd /tmp/librechat-custom # THIS dir: cloudron update reads CloudronManifest.json here
|
||
git add -A && git commit -m "..." && git push
|
||
SHA=$(git rev-parse --short HEAD)
|
||
cloudron builder build --repository registry.inference.coop/co-op/librechat-custom \
|
||
--tag 20261019-$SHA
|
||
cloudron --server my.inference.coop --token "$CLOUDRON_TOKEN" update \
|
||
--app 4023ccbf-d57a-4500-97df-14de9cf3516e \
|
||
--image registry.inference.coop/co-op/librechat-custom:20261019-$SHA
|
||
```
|
||
|
||
Rollback: rerun the `update` with the previous tag (or the appstore image
|
||
`cloudron/ai.librechat.cloudronapp:<original-tag>`).
|
||
|
||
**NEVER edit `CloudronManifest.json`.** `cloudron update` reads it from this
|
||
directory and applies it to the app. A hand-edited manifest that omits the
|
||
addons (`postgresql`, `mongodb`, `oidc`, `sendmail`, `localstorage`) strips
|
||
them from the running app and crash-loops it (`mkdir: Read-only file system`).
|
||
That is what broke the sandbox on 2026-10-18 — the manifest, not the overlay.
|
||
When the appstore package updates, refresh this file with a new copy pulled
|
||
from the Cloudron API (`GET /api/v1/apps/<id> → .manifest`) and bump the
|
||
`FROM` tag in the Dockerfile to the new `manifest.dockerImage`.
|
||
|
||
## Env vars (live state)
|
||
|
||
Set in `/app/data/env` (volume — these WIN over cloudron env):
|
||
|
||
- `APP_TITLE="Inference Co-op"` (was `"LibreChat"`)
|
||
- `ALLOW_EMAIL_LOGIN=false` (was `true`) — password login removed; Cloudron
|
||
SSO (OIDC) is the only login path
|
||
- `SCHEDULES_SINGLE_PROCESS=true` — sandbox runs a single replica; silences
|
||
the scheduler's Redis guard error
|
||
|
||
Set via `cloudron env set` (process env):
|
||
|
||
- `TTS_API_KEY` / `STT_API_KEY` — **dedicated LiteLLM virtual key** (alias
|
||
`librechat-speech`, $5 / 30d budget, restricted to `tinfoil/voxtral-tts` +
|
||
`tinfoil/whisper-large-v3-turbo`). Regenerate with the gateway master key:
|
||
`POST /key/generate {"key_alias":"librechat-speech","models":[...],"max_budget":5,"budget_duration":"30d"}`
|
||
— never put the master key here.
|
||
- `IMAGE_GEN_OAI_BASEURL=https://gateway.inference.coop/v1` and
|
||
`IMAGE_GEN_OAI_MODEL=regolo/qwen-image` — the agent "OpenAI Image Tools"
|
||
points at the co-op gateway. No `IMAGE_GEN_OAI_API_KEY`: members enter their
|
||
own co-op key in the tool UI (per-member billing, same pattern as the chat
|
||
endpoint's `apiKey: user_provided`).
|
||
- `SEARXNG_INSTANCE_URL=https://search.inference.coop` — referenced from the
|
||
YAML as `${SEARXNG_INSTANCE_URL}` (self-hosted search; keyless JSON API).
|
||
|
||
## Feature notes (LibreChat v0.8.8)
|
||
|
||
- **Chat models**: custom endpoint "Inference Co-op" → gateway, per-member
|
||
keys, 8 chat modelSpecs (emoji convention shared with models.md).
|
||
`regolo/qwen-image` is deliberately NOT in the chat picker (image models
|
||
are not chat models; chat-calling it was the source of the old 500s).
|
||
- **Image generation — two paths**:
|
||
1. **MCP tool (easiest)**: `inference-image` server (tool `generate_image`)
|
||
— members select it in the composer's **+** palette and just ask. No
|
||
agent setup. First use: save your co-op key in the MCP panel
|
||
(customUserVars `IC_API_KEY`, injected per-user into the server process
|
||
env; `startup: false` because the key only exists per-user after they
|
||
configure it). Bills to the member's own key (~1–2¢/image at 1024²).
|
||
2. **Agent tool**: LibreChat's built-in "OpenAI Image Tools" (env
|
||
`IMAGE_GEN_OAI_BASEURL`/`IMAGE_GEN_OAI_MODEL` → gateway; no admin key,
|
||
members paste their own). Note LibreChat v0.8.8 renders the generated
|
||
image only inside the agent conversation.
|
||
- **MCP**: `inference-docs` (tool `search_docs`, starts at boot) and
|
||
`inference-image` (tool `generate_image`, per-user startup). Members may
|
||
add their own via the MCP Settings panel ("bring your own tools").
|
||
Permissions seeded via `interface.mcpServers` (use/create true, share false).
|
||
- **Speech**: TTS `tinfoil/voxtral-tts` (voice `casual_female`), STT
|
||
`tinfoil/whisper-large-v3-turbo`, both via the gateway with the capped key.
|
||
- **Web search**: SearXNG (self-hosted) + keyless Keenable scraper, no reranker.
|
||
- **Colors**: `interface.theme` in the YAML is **forward-compatible only** —
|
||
the theme engine ships in LibreChat releases AFTER 0.8.8 (merged to dev,
|
||
not tagged). The block is silently ignored today and activates by itself
|
||
when the appstore package bundles a newer LibreChat. No other runtime
|
||
theming exists in 0.8.8.
|
||
|
||
## Known state / limitations
|
||
|
||
- The `[schedules]` Redis guard error is resolved by
|
||
`SCHEDULES_SINGLE_PROCESS=true` (above).
|
||
- The `RAG API not reachable` warning is expected (no rag_api addon); it only
|
||
affects LibreChat's own file-RAG, which we do not use.
|
||
- The `inference-image` MCP server logs `Tools: undefined` at boot — expected:
|
||
with `startup: false` it registers but only spawns per-user after a member
|
||
saves their key and reinitializes in the MCP panel.
|