co-op/librechat-custom
2
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

README: document manifest-pinned deploy flow, builder-cache trap, branding overlay, keyless image path; mirror live volume files

Browse Source
This commit is contained in:
Nathan Schneider committed 2026-10-18 22:52:23 -06:00
1 parent 571084d8bf
commit 91e97b00ea
2 files changed
+180 -74
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

README.md
+176 -65
View File
@@ -5,29 +5,49 @@ Source of truth for the LibreChat evaluation instance at
appstore package `ai.librechat.cloudronapp@1.4.1`, which bundles **LibreChat
v0.8.8** on image `cloudron/ai.librechat.cloudronapp:202610150224320000`).
This repo exists so the sandbox never drifts from what is deployed: the config
file here is byte-identical to `/app/data/librechat.yaml`, the MCP server here
is byte-identical to `/app/data/mcp/server.py`, and the Docker image is built
from this directory.
The deployment has **two independent layers** — know which one you are changing:
1. **Persistent volume** (`/app/data/…`) — behavior: `librechat.yaml`, `env`,
`mcp/*.py`. Survives image updates. Edited via `cloudron exec`, then
mirrored into this repo (see "Volume changes" below).
2. **Overlay image** (this repo's Dockerfile) — appearance only: logo,
favicons, `custom.css`, `custom2.js`, `<title>`. Rebuilt and redeployed to
change branding.
The md5s of `librechat.yaml`, `mcp/server.py`, and `mcp/image_server.py` in
this repo are kept byte-identical to the live volume files. Verify with:
```bash
APP=4023ccbf-d57a-4500-97df-14de9cf3516e
cloudron --server my.inference.coop --token "$CLOUDRON_TOKEN" exec --app $APP \
-- md5sum /app/data/librechat.yaml /app/data/mcp/server.py /app/data/mcp/image_server.py
md5sum librechat.yaml mcp/server.py mcp/image_server.py
```
## Architecture: what lives where
| Layer | Location | Survives image update? | Repo mirror |
|---|---|---|---|
| App config | `/app/data/librechat.yaml` (volume) | yes | `librechat.yaml` |
| Env vars (env-file wins!) | `/app/data/env` (volume, sourced LAST by start.sh) | yes | changed lines documented below |
| Env vars (env-file wins!) | `/app/data/env` (volume, sourced LAST by start.sh) | yes | documented below |
| Env vars (Cloudron-managed) | `cloudron env` (process env) | yes | documented below |
| MCP servers | `/app/data/mcp/server.py` + `/app/data/mcp/image_server.py` (volume) | yes | `mcp/` |
| Brand assets (logo, favicons) | baked into image overlay | no — rebuilt per image | `assets/` |
| CloudronManifest.json | repo only (used by `cloudron update`) | n/a | byte-identical copy of the OFFICIAL appstore manifest |
| MCP servers | `/app/data/mcp/server.py` + `mcp/image_server.py` (volume) | yes | `mcp/` |
| Brand assets + branding CSS/JS | baked into image overlay | no — rebuilt per image | `assets/` |
| `CloudronManifest.json` | repo only (controls what `cloudron update` deploys) | n/a | official appstore manifest **except `dockerImage`** |
**Precedence gotcha:** `start.sh` does `source /run/librechat/env` AFTER the
**Precedence gotcha:** `start.sh` sources `/run/librechat/env` AFTER the
process env, and `/app/data/env` is appended into it — so a value defined in
`/app/data/env` **overrides** the same `cloudron env` variable. That is why
`ALLOW_EMAIL_LOGIN=false` must be set in `/app/data/env`, not (only) via
`cloudron env`.
## Deploy a config change (no image rebuild needed)
## Volume changes (librechat.yaml, mcp/*.py, env)
> **Volume warning:** edits you make to the files in this repo do **not**
> deploy with the image. The image never touches `/app/data`. Apply changes
> to the live files first, then mirror them here.
Deploy a config change (no image rebuild needed):
```bash
B64=$(base64 -w0 librechat.yaml)
@@ -35,45 +55,102 @@ cloudron --server my.inference.coop --token "$CLOUDRON_TOKEN" exec \
--app 4023ccbf-d57a-4500-97df-14de9cf3516e \
-- bash -c "echo '$B64' | base64 -d > /app/data/librechat.yaml"
cloudron --server my.inference.coop --token "$CLOUDRON_TOKEN" restart \
--app 4023ccbf-d57a-4500-97df-14de9cf3516e
--app 4023ccbf-d57a-4500-97df-14de9cf3516e --timeout 900
```
Always `diff` the repo file against the live one after deploying. Always
deploy via base64 — heredocs and `sed` inserts have corrupted this file twice
(literal `$(...)` / bad indentation → LibreChat silently falls back to
defaults with only one log line).
- Always deploy via base64 — heredocs and `sed` inserts have corrupted this
file twice (literal `$(...)` / bad indentation → LibreChat silently falls
back to defaults with only one log line).
- Always `diff` the repo file against the live one after deploying.
- **Reading large live files back over `cloudron exec`** cannot be done with
a plain `cat` (a single large stdout stream resets the connection,
`ECONNRESET`). Pipe it small instead:
Verify after every change:
```bash
cloudron logs --app <id> --lines 60 | grep -iE 'custom config|error|invalid|MCP'
curl -s https://librechat-test.inference.coop/api/config # appTitle, emailLoginEnabled, socialLogins
```
## Build + deploy the image (brand assets only)
```bash
cd /tmp/librechat-custom # THIS dir: cloudron update reads CloudronManifest.json here
git add -A && git commit -m "..." && git push
SHA=$(git rev-parse --short HEAD)
cloudron builder build --repository registry.inference.coop/co-op/librechat-custom \
--tag 20261019-$SHA
cloudron --server my.inference.coop --token "$CLOUDRON_TOKEN" update \
```bash
cloudron --server my.inference.coop --token "$CLOUDRON_TOKEN" exec \
--app 4023ccbf-d57a-4500-97df-14de9cf3516e \
--image registry.inference.coop/co-op/librechat-custom:20261019-$SHA
-- bash -c "gzip -c /app/data/librechat.yaml | base64 -w0 | fold -w 100" > /tmp/live.b64
tr -d '\n' < /tmp/live.b64 | base64 -d | gunzip > /tmp/live-librechat.yaml
```
Verify after every config change:
```bash
cloudron logs --app 4023ccbf-d57a-4500-97df-14de9cf3516e --lines 60 | grep -iE 'custom config|error|invalid|MCP'
curl -s https://librechat-test.inference.coop/api/config # appTitle, emailLoginEnabled
```
Rollback: rerun the `update` with the previous tag (or the appstore image
`cloudron/ai.librechat.cloudronapp:<original-tag>`).
## Build + deploy the image (branding only)
**NEVER edit `CloudronManifest.json`.** `cloudron update` reads it from this
directory and applies it to the app. A hand-edited manifest that omits the
addons (`postgresql`, `mongodb`, `oidc`, `sendmail`, `localstorage`) strips
them from the running app and crash-loops it (`mkdir: Read-only file system`).
That is what broke the sandbox on 2026-10-18 — the manifest, not the overlay.
When the appstore package updates, refresh this file with a new copy pulled
from the Cloudron API (`GET /api/v1/apps/<id> → .manifest`) and bump the
`FROM` tag in the Dockerfile to the new `manifest.dockerImage`.
> **Two traps have each silently eaten deploys of this overlay.** Both look
> like success. Run the verification block below after every deploy.
1. **`cloudron update --image <tag>` is IGNORED when this directory contains
`CloudronManifest.json`.** The CLI deploys `manifest.dockerImage` instead —
while still printing your `--image` value in its output, so it *looks*
right. The tag you build MUST be written into
`CloudronManifest.json.dockerImage` before updating.
2. **The remote builder's cache can serve stale COPY layers even when file
content changed** (observed 2026-10-19: six consecutive "Build successful"
runs produced images with old assets). Build with `--no-cache` whenever
anything under `assets/` changed.
Full deploy sequence:
```bash
cd /path/to/librechat-custom # the dir containing CloudronManifest.json
# 1. edit assets/ … then:
git add -A && git commit -m "…" && git push
SHA=$(git rev-parse --short HEAD)
# 2. build fresh layers (no-cache after asset edits)
cloudron builder build --no-cache \
--repository registry.inference.coop/co-op/librechat-custom \
--tag 20261019-$SHA
# 3. pin the manifest to the new tag — the ONLY field you may change
# (edit "dockerImage" in CloudronManifest.json to :20261019-$SHA)
git add CloudronManifest.json && git commit -m "Pin manifest to $SHA" && git push
# 4. deploy — the manifest drives it
cloudron --server my.inference.coop --token "$CLOUDRON_TOKEN" update \
--app 4023ccbf-d57a-4500-97df-14de9cf3516e --force
```
Rollback: repoint `CloudronManifest.json.dockerImage` to the previous tag
(every build is tagged `20261019-<git-sha>`; the base image
`cloudron/ai.librechat.cloudronapp:<tag>` is the pre-overlay fallback) and
rerun step 4.
### Verify after every image deploy (all four, or it didn't happen)
```bash
APP=4023ccbf-d57a-4500-97df-14de9cf3516e
# 1. What is ACTUALLY running? (CLI update output is not evidence)
curl -s -H "Authorization: Bearer $CLOUDRON_TOKEN" \
https://my.inference.coop/api/v1/apps/$APP \
| python3 -c "import json,sys; print(json.load(sys.stdin)['app']['manifest']['dockerImage'])"
# 2. Container files match the repo (catches stale builder cache)
cloudron --server my.inference.coop --token "$CLOUDRON_TOKEN" exec --app $APP \
-- md5sum /app/code/librechat/client/dist/assets/custom.css
md5sum assets/custom.css
# 3. Served HTML carries the injection
curl -s https://librechat-test.inference.coop/login | grep -Eo 'assets/custom2?\.js|assets/custom\.css'
# 4. Rendered DOM shows it (browser devtools or a browser tool):
# computed background of div.bg-surface-primary == rgb(250, 248, 245);
# the "Don't have an account?" <p> computes display: none.
```
### `CloudronManifest.json` rules
- The **only** field you may change is `dockerImage` — it must point at the
overlay tag you want deployed.
- Everything else stays a byte-identical copy of the OFFICIAL appstore
manifest. A hand-edited manifest that omits the addons (`postgresql`,
`mongodb`, `oidc`, `sendmail`, `localstorage`) strips them from the running
app and crash-loops it (`mkdir: Read-only file system`). That broke the
sandbox on 2026-10-18 — the manifest, not the overlay, was the bug.
- When the appstore package updates, refresh the file with a new copy from
the Cloudron API (`GET /api/v1/apps/<id>` → `.manifest`), repoint
`dockerImage` to your rebuilt overlay, and bump the `FROM` tag in the
Dockerfile to the new appstore `dockerImage` (pinned — never a floating tag).
## Env vars (live state)
@@ -93,42 +170,68 @@ Set via `cloudron env set` (process env):
`POST /key/generate {"key_alias":"librechat-speech","models":[...],"max_budget":5,"budget_duration":"30d"}`
— never put the master key here.
- `IMAGE_GEN_OAI_BASEURL=https://gateway.inference.coop/v1` and
`IMAGE_GEN_OAI_MODEL=regolo/qwen-image` — the agent "OpenAI Image Tools"
points at the co-op gateway. No `IMAGE_GEN_OAI_API_KEY`: members enter their
own co-op key in the tool UI (per-member billing, same pattern as the chat
endpoint's `apiKey: user_provided`).
`IMAGE_GEN_OAI_MODEL=regolo/qwen-image` — the built-in agent "OpenAI Image
Tools" points at the co-op gateway. No admin key is configured; members
using the AGENT path paste their own key in the tool UI (v0.8.8
limitation). The MCP path below needs no key at all.
- `SEARXNG_INSTANCE_URL=https://search.inference.coop` — referenced from the
YAML as `${SEARXNG_INSTANCE_URL}` (self-hosted search; keyless JSON API).
## Branding overlay (custom.css / custom2.js)
Injected into the built SPA by the Dockerfile (`<link>` + `<script>` after
the `<title>` sed). What it does:
- **Colors** — LibreChat's Tailwind theme resolves every color through CSS
variables set on `html.light` / `html.dark` (space-separated RGB):
`--surface-primary`, `--surface-secondary/-tertiary/-hover/-submit/-submit-hover`,
`--accent-primary/-hover`, `--ring-primary`, `--link`,
`--text-primary/-secondary`, `--border-light`. `custom.css` overrides them
(light: cream `250 248 245` surfaces + co-op green `91 140 90` accents;
dark: matching co-op dark palette). **To rebrand, edit the `html.light` /
`html.dark` blocks** — do not invent class names; `.login-page`,
`.primary` etc. do not exist in the DOM. Read current values in devtools
via `getComputedStyle(document.documentElement).getPropertyValue('--surface-primary')`.
- **No sign-up prompt** — the "Don't have an account? Sign up" line is
`<p><a href="/register">Sign up</a></p>`; hidden with
`p:has(> a[href*="register"]) { display: none !important; }` (hiding only
the `<a>` leaves the text). Any residual register/signup links are hidden
too.
- **`custom2.js`** — the auth form is client-rendered AFTER `load`, so a
plain load-time query matches nothing; a **MutationObserver** re-checks on
every DOM change and hides the sign-up paragraph whenever React renders it.
There is also an `interface.theme` block in `librechat.yaml` — it is
**forward-compatible only**: the theme engine ships in LibreChat releases
AFTER 0.8.8 and the block is silently ignored today. When the appstore
package bundles a newer LibreChat, revisit whether the CSS overlay is still
needed.
## Feature notes (LibreChat v0.8.8)
- **Chat models**: custom endpoint "Inference Co-op" → gateway, per-member
keys, 8 chat modelSpecs (emoji convention shared with models.md).
`regolo/qwen-image` is deliberately NOT in the chat picker (image models
are not chat models; chat-calling it was the source of the old 500s).
- **Chat models**: custom endpoint "Inference Co-op" → gateway, 8 chat
modelSpecs (emoji convention shared with models.md). `regolo/qwen-image`
is deliberately NOT in the chat picker (image models are not chat models;
chat-calling it was the source of the old 500s).
- **Image generation — two paths**:
1. **MCP tool (easiest)**: `inference-image` server (tool `generate_image`)
— members select it in the composer's **+** palette and just ask. No
agent setup. First use: save your co-op key in the MCP panel
(customUserVars `IC_API_KEY`, injected per-user into the server process
env; `startup: false` because the key only exists per-user after they
configure it). Bills to the member's own key (~1–2¢/image at 1024²).
1. **MCP tool (default, no key ever)**: `inference-image` server (tool
`generate_image`) — members select it in the composer's **+** palette
and just ask. The server uses the member's `IC_API_KEY` if LibreChat
injected one, otherwise sends a **keyless request** (the gateway bills
the request; per-member billing is NOT enforced on this path).
`customUserVars` were removed from the live yaml on 2026-10-19 —
members are never asked for a key.
2. **Agent tool**: LibreChat's built-in "OpenAI Image Tools" (env
`IMAGE_GEN_OAI_BASEURL`/`IMAGE_GEN_OAI_MODEL` → gateway; no admin key,
members paste their own). Note LibreChat v0.8.8 renders the generated
image only inside the agent conversation.
- **MCP**: `inference-docs` (tool `search_docs`, starts at boot) and
`inference-image` (tool `generate_image`, per-user startup). Members may
`inference-image` (tool `generate_image`, `startup: false`). Members may
add their own via the MCP Settings panel ("bring your own tools").
Permissions seeded via `interface.mcpServers` (use/create true, share false).
- **Speech**: TTS `tinfoil/voxtral-tts` (voice `casual_female`), STT
`tinfoil/whisper-large-v3-turbo`, both via the gateway with the capped key.
- **Web search**: SearXNG (self-hosted) + keyless Keenable scraper, no reranker.
- **Colors**: `interface.theme` in the YAML is **forward-compatible only** —
the theme engine ships in LibreChat releases AFTER 0.8.8 (merged to dev,
not tagged). The block is silently ignored today and activates by itself
when the appstore package bundles a newer LibreChat. No other runtime
theming exists in 0.8.8.
## Known state / limitations
@@ -136,6 +239,14 @@ Set via `cloudron env set` (process env):
`SCHEDULES_SINGLE_PROCESS=true` (above).
- The `RAG API not reachable` warning is expected (no rag_api addon); it only
affects LibreChat's own file-RAG, which we do not use.
- The `inference-image` MCP server logs `Tools: undefined` at boot — expected:
with `startup: false` it registers but only spawns per-user after a member
saves their key and reinitializes in the MCP panel.
- The `inference-image` MCP server logs `Tools: undefined` at boot —
expected: with `startup: false` it registers at boot and spawns per-user
on first use in a conversation.
- Members' chat usage bills per-member via their endpoint key; the MCP image
path is currently keyless and bills to the gateway (not enforced
per-member). Wiring per-member key injection into LibreChat chat is an
open task.
- Repo ↔ live parity last verified 2026-10-19: `librechat.yaml`,
`mcp/server.py`, `mcp/image_server.py` md5s match the live volume; overlay
image `registry.inference.coop/co-op/librechat-custom:20261019-fc9b11e` is
deployed (container asset md5s match this repo).
mcp/image_server.py
+4 -9
View File
@@ -37,16 +37,11 @@ def log(msg):
def tool_generate_image(args):
# Seamless: use the member's own key if LibreChat injected one; otherwise
# fall back to a keyless request so members who never saved a key can
# still generate images (the gateway bills their shared co-op allowance).
key = (os.environ.get("IC_API_KEY") or "").strip()
# If the user has not set a personal API key, fall back to an unauthenticated request.
# The gateway will bill the request to the member's account based on the session
# (the LibreChat backend already knows the member identity). This mirrors the
# behaviour of the chat endpoint, which does not require the user to manually
# provide a key.
if not key:
auth_header = {}
else:
auth_header = {"Authorization": f"Bearer {key}"}
auth_header = {"Authorization": f"Bearer {key}"} if key else {}
prompt = str(args.get("prompt") or "").strip()
if not prompt: