co-op/librechat-custom
2
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

README: document manifest-pinned deploy flow, builder-cache trap, branding overlay, keyless image path; mirror live volume files

Browse Source
This commit is contained in:
Nathan Schneider committed 2026-10-18 22:52:23 -06:00
1 parent 571084d8bf
commit 91e97b00ea
2 files changed
+180 -74
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

README.md
+176 -65
View File
@@ -5,29 +5,49 @@ Source of truth for the LibreChat evaluation instance at
appstore package `ai.librechat.cloudronapp@1.4.1`, which bundles **LibreChat appstore package `ai.librechat.cloudronapp@1.4.1`, which bundles **LibreChat
v0.8.8** on image `cloudron/ai.librechat.cloudronapp:202610150224320000`). v0.8.8** on image `cloudron/ai.librechat.cloudronapp:202610150224320000`).
This repo exists so the sandbox never drifts from what is deployed: the config The deployment has **two independent layers** — know which one you are changing:
file here is byte-identical to `/app/data/librechat.yaml`, the MCP server here
is byte-identical to `/app/data/mcp/server.py`, and the Docker image is built 1. **Persistent volume** (`/app/data/…`) — behavior: `librechat.yaml`, `env`,
from this directory. `mcp/*.py`. Survives image updates. Edited via `cloudron exec`, then
mirrored into this repo (see "Volume changes" below).
2. **Overlay image** (this repo's Dockerfile) — appearance only: logo,
favicons, `custom.css`, `custom2.js`, `<title>`. Rebuilt and redeployed to
change branding.
The md5s of `librechat.yaml`, `mcp/server.py`, and `mcp/image_server.py` in
this repo are kept byte-identical to the live volume files. Verify with:
```bash
APP=4023ccbf-d57a-4500-97df-14de9cf3516e
cloudron --server my.inference.coop --token "$CLOUDRON_TOKEN" exec --app $APP \
-- md5sum /app/data/librechat.yaml /app/data/mcp/server.py /app/data/mcp/image_server.py
md5sum librechat.yaml mcp/server.py mcp/image_server.py
```
## Architecture: what lives where ## Architecture: what lives where
| Layer | Location | Survives image update? | Repo mirror | | Layer | Location | Survives image update? | Repo mirror |
|---|---|---|---| |---|---|---|---|
| App config | `/app/data/librechat.yaml` (volume) | yes | `librechat.yaml` | | App config | `/app/data/librechat.yaml` (volume) | yes | `librechat.yaml` |
| Env vars (env-file wins!) | `/app/data/env` (volume, sourced LAST by start.sh) | yes | changed lines documented below | | Env vars (env-file wins!) | `/app/data/env` (volume, sourced LAST by start.sh) | yes | documented below |
| Env vars (Cloudron-managed) | `cloudron env` (process env) | yes | documented below | | Env vars (Cloudron-managed) | `cloudron env` (process env) | yes | documented below |
| MCP servers | `/app/data/mcp/server.py` + `/app/data/mcp/image_server.py` (volume) | yes | `mcp/` | | MCP servers | `/app/data/mcp/server.py` + `mcp/image_server.py` (volume) | yes | `mcp/` |
| Brand assets (logo, favicons) | baked into image overlay | no — rebuilt per image | `assets/` | | Brand assets + branding CSS/JS | baked into image overlay | no — rebuilt per image | `assets/` |
| CloudronManifest.json | repo only (used by `cloudron update`) | n/a | byte-identical copy of the OFFICIAL appstore manifest | | `CloudronManifest.json` | repo only (controls what `cloudron update` deploys) | n/a | official appstore manifest **except `dockerImage`** |
**Precedence gotcha:** `start.sh` does `source /run/librechat/env` AFTER the **Precedence gotcha:** `start.sh` sources `/run/librechat/env` AFTER the
process env, and `/app/data/env` is appended into it — so a value defined in process env, and `/app/data/env` is appended into it — so a value defined in
`/app/data/env` **overrides** the same `cloudron env` variable. That is why `/app/data/env` **overrides** the same `cloudron env` variable. That is why
`ALLOW_EMAIL_LOGIN=false` must be set in `/app/data/env`, not (only) via `ALLOW_EMAIL_LOGIN=false` must be set in `/app/data/env`, not (only) via
`cloudron env`. `cloudron env`.
## Deploy a config change (no image rebuild needed) ## Volume changes (librechat.yaml, mcp/*.py, env)
> **Volume warning:** edits you make to the files in this repo do **not**
> deploy with the image. The image never touches `/app/data`. Apply changes
> to the live files first, then mirror them here.
Deploy a config change (no image rebuild needed):
```bash ```bash
B64=$(base64 -w0 librechat.yaml) B64=$(base64 -w0 librechat.yaml)
@@ -35,45 +55,102 @@ cloudron --server my.inference.coop --token "$CLOUDRON_TOKEN" exec \
--app 4023ccbf-d57a-4500-97df-14de9cf3516e \ --app 4023ccbf-d57a-4500-97df-14de9cf3516e \
-- bash -c "echo '$B64' | base64 -d > /app/data/librechat.yaml" -- bash -c "echo '$B64' | base64 -d > /app/data/librechat.yaml"
cloudron --server my.inference.coop --token "$CLOUDRON_TOKEN" restart \ cloudron --server my.inference.coop --token "$CLOUDRON_TOKEN" restart \
--app 4023ccbf-d57a-4500-97df-14de9cf3516e --app 4023ccbf-d57a-4500-97df-14de9cf3516e --timeout 900
``` ```
Always `diff` the repo file against the live one after deploying. Always - Always deploy via base64 — heredocs and `sed` inserts have corrupted this
deploy via base64 — heredocs and `sed` inserts have corrupted this file twice file twice (literal `$(...)` / bad indentation → LibreChat silently falls
(literal `$(...)` / bad indentation → LibreChat silently falls back to back to defaults with only one log line).
defaults with only one log line). - Always `diff` the repo file against the live one after deploying.
- **Reading large live files back over `cloudron exec`** cannot be done with
a plain `cat` (a single large stdout stream resets the connection,
`ECONNRESET`). Pipe it small instead:
Verify after every change: ```bash
cloudron --server my.inference.coop --token "$CLOUDRON_TOKEN" exec \
```bash
cloudron logs --app <id> --lines 60 | grep -iE 'custom config|error|invalid|MCP'
curl -s https://librechat-test.inference.coop/api/config # appTitle, emailLoginEnabled, socialLogins
```
## Build + deploy the image (brand assets only)
```bash
cd /tmp/librechat-custom # THIS dir: cloudron update reads CloudronManifest.json here
git add -A && git commit -m "..." && git push
SHA=$(git rev-parse --short HEAD)
cloudron builder build --repository registry.inference.coop/co-op/librechat-custom \
--tag 20261019-$SHA
cloudron --server my.inference.coop --token "$CLOUDRON_TOKEN" update \
--app 4023ccbf-d57a-4500-97df-14de9cf3516e \ --app 4023ccbf-d57a-4500-97df-14de9cf3516e \
--image registry.inference.coop/co-op/librechat-custom:20261019-$SHA -- bash -c "gzip -c /app/data/librechat.yaml | base64 -w0 | fold -w 100" > /tmp/live.b64
tr -d '\n' < /tmp/live.b64 | base64 -d | gunzip > /tmp/live-librechat.yaml
```
Verify after every config change:
```bash
cloudron logs --app 4023ccbf-d57a-4500-97df-14de9cf3516e --lines 60 | grep -iE 'custom config|error|invalid|MCP'
curl -s https://librechat-test.inference.coop/api/config # appTitle, emailLoginEnabled
``` ```
Rollback: rerun the `update` with the previous tag (or the appstore image ## Build + deploy the image (branding only)
`cloudron/ai.librechat.cloudronapp:<original-tag>`).
**NEVER edit `CloudronManifest.json`.** `cloudron update` reads it from this > **Two traps have each silently eaten deploys of this overlay.** Both look
directory and applies it to the app. A hand-edited manifest that omits the > like success. Run the verification block below after every deploy.
addons (`postgresql`, `mongodb`, `oidc`, `sendmail`, `localstorage`) strips
them from the running app and crash-loops it (`mkdir: Read-only file system`). 1. **`cloudron update --image <tag>` is IGNORED when this directory contains
That is what broke the sandbox on 2026-10-18 — the manifest, not the overlay. `CloudronManifest.json`.** The CLI deploys `manifest.dockerImage` instead —
When the appstore package updates, refresh this file with a new copy pulled while still printing your `--image` value in its output, so it *looks*
from the Cloudron API (`GET /api/v1/apps/<id> → .manifest`) and bump the right. The tag you build MUST be written into
`FROM` tag in the Dockerfile to the new `manifest.dockerImage`. `CloudronManifest.json.dockerImage` before updating.
2. **The remote builder's cache can serve stale COPY layers even when file
content changed** (observed 2026-10-19: six consecutive "Build successful"
runs produced images with old assets). Build with `--no-cache` whenever
anything under `assets/` changed.
Full deploy sequence:
```bash
cd /path/to/librechat-custom # the dir containing CloudronManifest.json
# 1. edit assets/ … then:
git add -A && git commit -m "…" && git push
SHA=$(git rev-parse --short HEAD)
# 2. build fresh layers (no-cache after asset edits)
cloudron builder build --no-cache \
--repository registry.inference.coop/co-op/librechat-custom \
--tag 20261019-$SHA
# 3. pin the manifest to the new tag — the ONLY field you may change
# (edit "dockerImage" in CloudronManifest.json to :20261019-$SHA)
git add CloudronManifest.json && git commit -m "Pin manifest to $SHA" && git push
# 4. deploy — the manifest drives it
cloudron --server my.inference.coop --token "$CLOUDRON_TOKEN" update \
--app 4023ccbf-d57a-4500-97df-14de9cf3516e --force
```
Rollback: repoint `CloudronManifest.json.dockerImage` to the previous tag
(every build is tagged `20261019-<git-sha>`; the base image
`cloudron/ai.librechat.cloudronapp:<tag>` is the pre-overlay fallback) and
rerun step 4.
### Verify after every image deploy (all four, or it didn't happen)
```bash
APP=4023ccbf-d57a-4500-97df-14de9cf3516e
# 1. What is ACTUALLY running? (CLI update output is not evidence)
curl -s -H "Authorization: Bearer $CLOUDRON_TOKEN" \
https://my.inference.coop/api/v1/apps/$APP \
| python3 -c "import json,sys; print(json.load(sys.stdin)['app']['manifest']['dockerImage'])"
# 2. Container files match the repo (catches stale builder cache)
cloudron --server my.inference.coop --token "$CLOUDRON_TOKEN" exec --app $APP \
-- md5sum /app/code/librechat/client/dist/assets/custom.css
md5sum assets/custom.css
# 3. Served HTML carries the injection
curl -s https://librechat-test.inference.coop/login | grep -Eo 'assets/custom2?\.js|assets/custom\.css'
# 4. Rendered DOM shows it (browser devtools or a browser tool):
# computed background of div.bg-surface-primary == rgb(250, 248, 245);
# the "Don't have an account?" <p> computes display: none.
```
### `CloudronManifest.json` rules
- The **only** field you may change is `dockerImage` — it must point at the
overlay tag you want deployed.
- Everything else stays a byte-identical copy of the OFFICIAL appstore
manifest. A hand-edited manifest that omits the addons (`postgresql`,
`mongodb`, `oidc`, `sendmail`, `localstorage`) strips them from the running
app and crash-loops it (`mkdir: Read-only file system`). That broke the
sandbox on 2026-10-18 — the manifest, not the overlay, was the bug.
- When the appstore package updates, refresh the file with a new copy from
the Cloudron API (`GET /api/v1/apps/<id>` → `.manifest`), repoint
`dockerImage` to your rebuilt overlay, and bump the `FROM` tag in the
Dockerfile to the new appstore `dockerImage` (pinned — never a floating tag).
## Env vars (live state) ## Env vars (live state)
@@ -93,42 +170,68 @@ Set via `cloudron env set` (process env):
`POST /key/generate {"key_alias":"librechat-speech","models":[...],"max_budget":5,"budget_duration":"30d"}` `POST /key/generate {"key_alias":"librechat-speech","models":[...],"max_budget":5,"budget_duration":"30d"}`
— never put the master key here. — never put the master key here.
- `IMAGE_GEN_OAI_BASEURL=https://gateway.inference.coop/v1` and - `IMAGE_GEN_OAI_BASEURL=https://gateway.inference.coop/v1` and
`IMAGE_GEN_OAI_MODEL=regolo/qwen-image` — the agent "OpenAI Image Tools" `IMAGE_GEN_OAI_MODEL=regolo/qwen-image` — the built-in agent "OpenAI Image
points at the co-op gateway. No `IMAGE_GEN_OAI_API_KEY`: members enter their Tools" points at the co-op gateway. No admin key is configured; members
own co-op key in the tool UI (per-member billing, same pattern as the chat using the AGENT path paste their own key in the tool UI (v0.8.8
endpoint's `apiKey: user_provided`). limitation). The MCP path below needs no key at all.
- `SEARXNG_INSTANCE_URL=https://search.inference.coop` — referenced from the - `SEARXNG_INSTANCE_URL=https://search.inference.coop` — referenced from the
YAML as `${SEARXNG_INSTANCE_URL}` (self-hosted search; keyless JSON API). YAML as `${SEARXNG_INSTANCE_URL}` (self-hosted search; keyless JSON API).
## Branding overlay (custom.css / custom2.js)
Injected into the built SPA by the Dockerfile (`<link>` + `<script>` after
the `<title>` sed). What it does:
- **Colors** — LibreChat's Tailwind theme resolves every color through CSS
variables set on `html.light` / `html.dark` (space-separated RGB):
`--surface-primary`, `--surface-secondary/-tertiary/-hover/-submit/-submit-hover`,
`--accent-primary/-hover`, `--ring-primary`, `--link`,
`--text-primary/-secondary`, `--border-light`. `custom.css` overrides them
(light: cream `250 248 245` surfaces + co-op green `91 140 90` accents;
dark: matching co-op dark palette). **To rebrand, edit the `html.light` /
`html.dark` blocks** — do not invent class names; `.login-page`,
`.primary` etc. do not exist in the DOM. Read current values in devtools
via `getComputedStyle(document.documentElement).getPropertyValue('--surface-primary')`.
- **No sign-up prompt** — the "Don't have an account? Sign up" line is
`<p><a href="/register">Sign up</a></p>`; hidden with
`p:has(> a[href*="register"]) { display: none !important; }` (hiding only
the `<a>` leaves the text). Any residual register/signup links are hidden
too.
- **`custom2.js`** — the auth form is client-rendered AFTER `load`, so a
plain load-time query matches nothing; a **MutationObserver** re-checks on
every DOM change and hides the sign-up paragraph whenever React renders it.
There is also an `interface.theme` block in `librechat.yaml` — it is
**forward-compatible only**: the theme engine ships in LibreChat releases
AFTER 0.8.8 and the block is silently ignored today. When the appstore
package bundles a newer LibreChat, revisit whether the CSS overlay is still
needed.
## Feature notes (LibreChat v0.8.8) ## Feature notes (LibreChat v0.8.8)
- **Chat models**: custom endpoint "Inference Co-op" → gateway, per-member - **Chat models**: custom endpoint "Inference Co-op" → gateway, 8 chat
keys, 8 chat modelSpecs (emoji convention shared with models.md). modelSpecs (emoji convention shared with models.md). `regolo/qwen-image`
`regolo/qwen-image` is deliberately NOT in the chat picker (image models is deliberately NOT in the chat picker (image models are not chat models;
are not chat models; chat-calling it was the source of the old 500s). chat-calling it was the source of the old 500s).
- **Image generation — two paths**: - **Image generation — two paths**:
1. **MCP tool (easiest)**: `inference-image` server (tool `generate_image`) 1. **MCP tool (default, no key ever)**: `inference-image` server (tool
— members select it in the composer's **+** palette and just ask. No `generate_image`) — members select it in the composer's **+** palette
agent setup. First use: save your co-op key in the MCP panel and just ask. The server uses the member's `IC_API_KEY` if LibreChat
(customUserVars `IC_API_KEY`, injected per-user into the server process injected one, otherwise sends a **keyless request** (the gateway bills
env; `startup: false` because the key only exists per-user after they the request; per-member billing is NOT enforced on this path).
configure it). Bills to the member's own key (~1–2¢/image at 1024²). `customUserVars` were removed from the live yaml on 2026-10-19 —
members are never asked for a key.
2. **Agent tool**: LibreChat's built-in "OpenAI Image Tools" (env 2. **Agent tool**: LibreChat's built-in "OpenAI Image Tools" (env
`IMAGE_GEN_OAI_BASEURL`/`IMAGE_GEN_OAI_MODEL` → gateway; no admin key, `IMAGE_GEN_OAI_BASEURL`/`IMAGE_GEN_OAI_MODEL` → gateway; no admin key,
members paste their own). Note LibreChat v0.8.8 renders the generated members paste their own). Note LibreChat v0.8.8 renders the generated
image only inside the agent conversation. image only inside the agent conversation.
- **MCP**: `inference-docs` (tool `search_docs`, starts at boot) and - **MCP**: `inference-docs` (tool `search_docs`, starts at boot) and
`inference-image` (tool `generate_image`, per-user startup). Members may `inference-image` (tool `generate_image`, `startup: false`). Members may
add their own via the MCP Settings panel ("bring your own tools"). add their own via the MCP Settings panel ("bring your own tools").
Permissions seeded via `interface.mcpServers` (use/create true, share false). Permissions seeded via `interface.mcpServers` (use/create true, share false).
- **Speech**: TTS `tinfoil/voxtral-tts` (voice `casual_female`), STT - **Speech**: TTS `tinfoil/voxtral-tts` (voice `casual_female`), STT
`tinfoil/whisper-large-v3-turbo`, both via the gateway with the capped key. `tinfoil/whisper-large-v3-turbo`, both via the gateway with the capped key.
- **Web search**: SearXNG (self-hosted) + keyless Keenable scraper, no reranker. - **Web search**: SearXNG (self-hosted) + keyless Keenable scraper, no reranker.
- **Colors**: `interface.theme` in the YAML is **forward-compatible only** —
the theme engine ships in LibreChat releases AFTER 0.8.8 (merged to dev,
not tagged). The block is silently ignored today and activates by itself
when the appstore package bundles a newer LibreChat. No other runtime
theming exists in 0.8.8.
## Known state / limitations ## Known state / limitations
@@ -136,6 +239,14 @@ Set via `cloudron env set` (process env):
`SCHEDULES_SINGLE_PROCESS=true` (above). `SCHEDULES_SINGLE_PROCESS=true` (above).
- The `RAG API not reachable` warning is expected (no rag_api addon); it only - The `RAG API not reachable` warning is expected (no rag_api addon); it only
affects LibreChat's own file-RAG, which we do not use. affects LibreChat's own file-RAG, which we do not use.
- The `inference-image` MCP server logs `Tools: undefined` at boot — expected: - The `inference-image` MCP server logs `Tools: undefined` at boot —
with `startup: false` it registers but only spawns per-user after a member expected: with `startup: false` it registers at boot and spawns per-user
saves their key and reinitializes in the MCP panel. on first use in a conversation.
- Members' chat usage bills per-member via their endpoint key; the MCP image
path is currently keyless and bills to the gateway (not enforced
per-member). Wiring per-member key injection into LibreChat chat is an
open task.
- Repo ↔ live parity last verified 2026-10-19: `librechat.yaml`,
`mcp/server.py`, `mcp/image_server.py` md5s match the live volume; overlay
image `registry.inference.coop/co-op/librechat-custom:20261019-fc9b11e` is
deployed (container asset md5s match this repo).
mcp/image_server.py
+4 -9
View File
@@ -37,16 +37,11 @@ def log(msg):
def tool_generate_image(args): def tool_generate_image(args):
# Seamless: use the member's own key if LibreChat injected one; otherwise
# fall back to a keyless request so members who never saved a key can
# still generate images (the gateway bills their shared co-op allowance).
key = (os.environ.get("IC_API_KEY") or "").strip() key = (os.environ.get("IC_API_KEY") or "").strip()
# If the user has not set a personal API key, fall back to an unauthenticated request. auth_header = {"Authorization": f"Bearer {key}"} if key else {}
# The gateway will bill the request to the member's account based on the session
# (the LibreChat backend already knows the member identity). This mirrors the
# behaviour of the chat endpoint, which does not require the user to manually
# provide a key.
if not key:
auth_header = {}
else:
auth_header = {"Authorization": f"Bearer {key}"}
prompt = str(args.get("prompt") or "").strip() prompt = str(args.get("prompt") or "").strip()
if not prompt: if not prompt: