Compare commits
11
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
12da5c1268 | ||
|
|
1f3f0ce9b8 | ||
|
|
fde5d143c5 | ||
|
|
0e708a99c6 | ||
|
|
467d19357d | ||
|
|
6a0d092e53 | ||
|
|
bac352f50e | ||
|
|
34a23a34f3 | ||
|
|
fa979a5ed9 | ||
|
|
5bb99b8592 | ||
|
|
e7c86b16ba |
No files matched your search
@@ -62,6 +62,20 @@ Your usage draws from your monthly allowance; you can see it on the
|
|||||||
If the chat ever says your balance is out, that's the allowance, not an error —
|
If the chat ever says your balance is out, that's the allowance, not an error —
|
||||||
see "Your allowance" below.
|
see "Your allowance" below.
|
||||||
|
|
||||||
|
### Add the chat to your phone
|
||||||
|
|
||||||
|
The chat works like an app on your phone — no app store needed. Open
|
||||||
|
[chat.inference.coop](https://chat.inference.coop) in your phone's browser and
|
||||||
|
log in, then:
|
||||||
|
|
||||||
|
- **iPhone (Safari):** tap the **Share** button (the square with an arrow),
|
||||||
|
then **Add to Home Screen**.
|
||||||
|
- **Android (Chrome):** tap the **⋮** menu, then **Add to Home Screen**
|
||||||
|
(or **Install app**).
|
||||||
|
|
||||||
|
You'll get an icon on your home screen that opens the chat full-screen, like
|
||||||
|
any other app. Your login stays active — just tap and chat.
|
||||||
|
|
||||||
## 3. Get involved in governance
|
## 3. Get involved in governance
|
||||||
|
|
||||||
You don't just use the co-op; you help run it. Decisions about models, pricing,
|
You don't just use the co-op; you help run it. Decisions about models, pricing,
|
||||||
|
|||||||
@@ -0,0 +1,27 @@
|
|||||||
|
# Chat task models (titles, tags, suggestions)
|
||||||
|
|
||||||
|
Both chat front doors generate small helper outputs — conversation titles,
|
||||||
|
tags, retrieval queries — with a model separate from the one answering the
|
||||||
|
member. These tasks are pinned to a small, inexpensive model so they never
|
||||||
|
bill against the member's chosen (larger) model.
|
||||||
|
|
||||||
|
## Open WebUI (chat.inference.coop)
|
||||||
|
|
||||||
|
- Title generation and tag generation are **enabled**; follow-up reply
|
||||||
|
suggestions and input autocomplete are **disabled** (October 2026).
|
||||||
|
- The task model (`task.model.default` in the Postgres `config` table) is
|
||||||
|
pinned to `greenpt/glm-5.3-flash` — a small, inexpensive model — so titles
|
||||||
|
and tags always run on it regardless of which model the member is chatting
|
||||||
|
with. Before October 2026 it was empty, which meant helper tasks ran on the
|
||||||
|
member's selected model.
|
||||||
|
- Config lives in the app's Postgres database (`config` table, key/value rows
|
||||||
|
with JSON values) and is read at startup: change a row, then restart the
|
||||||
|
app (`cloudron restart --app <id>`; note `restart` takes no `--timeout`).
|
||||||
|
|
||||||
|
## LibreChat (librechat-test.inference.coop)
|
||||||
|
|
||||||
|
- Title generation is configured in `/app/data/librechat.yaml`:
|
||||||
|
`titleConvo: true`, `titleModel: greenpt/glm-5.3-flash`,
|
||||||
|
`titleMethod: completion` — the same small GreenPT model.
|
||||||
|
- LibreChat v0.8.8 has no reply-suggestion feature; there is nothing to
|
||||||
|
disable on that front.
|
||||||
@@ -0,0 +1,156 @@
|
|||||||
|
# LibreChat sandbox maintenance
|
||||||
|
|
||||||
|
The LibreChat evaluation instance at **librechat-test.inference.coop**
|
||||||
|
(Cloudron app `4023ccbf-d57a-4500-97df-14de9cf3516e`, appstore package
|
||||||
|
`ai.librechat.cloudronapp@1.4.1`, which bundles **LibreChat v0.8.8**).
|
||||||
|
|
||||||
|
The single source of truth for the deployment is the repo
|
||||||
|
**[co-op/librechat-custom](https://git.inference.coop/co-op/librechat-custom)**
|
||||||
|
— its README is the full runbook (config deploy, image build/deploy, rollback,
|
||||||
|
env vars, branding overlay, known state). This doc is the operator-facing
|
||||||
|
summary.
|
||||||
|
|
||||||
|
## Architecture: what lives where
|
||||||
|
|
||||||
|
- **App config** — `/app/data/librechat.yaml` (persistent volume, mirrored
|
||||||
|
byte-identically in the repo). Holds the "Inference Co-op" custom endpoint
|
||||||
|
(gateway), the 8-model chat picker, speech, web search, and the two MCP
|
||||||
|
servers. Volume edits deploy via `cloudron exec` (base64), never via the
|
||||||
|
image — the image never touches `/app/data`.
|
||||||
|
- **Env file** — `/app/data/env` (persistent volume). **`start.sh` sources it
|
||||||
|
AFTER the process env, so it wins over `cloudron env`** — that is where
|
||||||
|
`APP_TITLE="Inference Co-op"`, `ALLOW_EMAIL_LOGIN=false` (password login
|
||||||
|
removed; Cloudron SSO is the only login path), and
|
||||||
|
`SCHEDULES_SINGLE_PROCESS=true` must live.
|
||||||
|
- **Other env vars** — via `cloudron env`: `TTS_API_KEY`/`STT_API_KEY`
|
||||||
|
(dedicated LiteLLM virtual key, alias `librechat-speech`, $5/30d, audio
|
||||||
|
models only — never the gateway master key), `IMAGE_GEN_OAI_BASEURL` +
|
||||||
|
`IMAGE_GEN_OAI_MODEL` (agent image tool → gateway), `SEARXNG_INSTANCE_URL`.
|
||||||
|
- **MCP servers** — `/app/data/mcp/server.py` (docs search, `search_docs`,
|
||||||
|
starts at boot) and `/app/data/mcp/image_server.py` (image generation,
|
||||||
|
`generate_image`, `startup: false`); both mirrored in the repo under
|
||||||
|
`mcp/`, stdlib-only Python.
|
||||||
|
- **Branding overlay** (logo, favicons, PWA icons, `<title>`, `custom.css`,
|
||||||
|
`custom2.js`) — baked into a pinned overlay image `FROM` the official
|
||||||
|
appstore image. Deployed tag (last verified 2026-10-19):
|
||||||
|
`registry.inference.coop/co-op/librechat-custom:20261019-fc9b11e`.
|
||||||
|
- **Branding CSS/JS** — `custom.css` overrides LibreChat's Tailwind theme
|
||||||
|
variables on `html.light`/`html.dark` using the HOMEPAGE's canonical
|
||||||
|
palette (`--cream #faf8f5`, `--wash #eef4ea`, `--green-deep #5b8c5a`,
|
||||||
|
`--green-mid #6ba86b`, `--ink #2d3327`, `--muted #6b7a62`) and brands
|
||||||
|
the loading flash (`#loading-container`); the login logo is the green
|
||||||
|
pine-dots vector (white-plate variant was invisible on cream — the
|
||||||
|
'flash on load' glitch), cache-busted via `?v=3` in the client bundle.
|
||||||
|
and hides the register/sign-up prompt via
|
||||||
|
`p:has(> a[href*="register"])`; `custom2.js` is a MutationObserver that
|
||||||
|
hides the prompt when React renders it (the auth form renders after
|
||||||
|
`load`). The `interface.theme` block in the yaml remains
|
||||||
|
forward-compatible only (inert on v0.8.8).
|
||||||
|
|
||||||
|
## Image generation (two member-facing paths)
|
||||||
|
|
||||||
|
1. **MCP tool (default, no key ever)**: the `inference-image` MCP server
|
||||||
|
exposes `generate_image` in the composer's **+** palette — no agent setup,
|
||||||
|
no key prompt. Per-member billing via `customUserVars.IC_API_KEY` (seeded
|
||||||
|
from the portal) + `env: IC_API_KEY: "{{IC_API_KEY}}"`; a system-key
|
||||||
|
fallback (`system-image-fallback`, $5/30d) covers the rare empty-value
|
||||||
|
case. The `customUserVars` gate blocks a not-yet-seeded member until the
|
||||||
|
next sync (≤10 min) — accepted as a short delay over billing everything
|
||||||
|
to the system account.
|
||||||
|
2. **Agent tool**: LibreChat's built-in "OpenAI Image Tools"
|
||||||
|
(`IMAGE_GEN_OAI_BASEURL`/`IMAGE_GEN_OAI_MODEL` → gateway; members paste
|
||||||
|
their own key — v0.8.8 limitation).
|
||||||
|
|
||||||
|
## Update policy
|
||||||
|
|
||||||
|
- The app is **image-sourced** (the `--force` update moved it off the
|
||||||
|
appstore source), so Cloudron will not auto-update it. This is deliberate —
|
||||||
|
same lockdown model as LiteLLM.
|
||||||
|
- **Image deploys**: build `--no-cache`, write the new tag into
|
||||||
|
`CloudronManifest.json.dockerImage` (the only editable field), commit, then
|
||||||
|
`cloudron update --app <id> --force`. **`--image` alone is IGNORED** when
|
||||||
|
the manifest sits in the deploy directory. **The remote builder's cache can
|
||||||
|
serve stale COPY layers even when file content changed** (six consecutive
|
||||||
|
"Build successful" runs did on 2026-10-19) — so after every image deploy,
|
||||||
|
verify: the Cloudron API's `manifest.dockerImage`, container-file md5 vs
|
||||||
|
repo md5, the served HTML, and the rendered DOM. Full checklist in the
|
||||||
|
repo README.
|
||||||
|
- **When a new appstore package ships**: pull its manifest copy into the repo
|
||||||
|
(`GET /api/v1/apps/<id>` → `.manifest`), repoint `dockerImage` to the
|
||||||
|
rebuilt overlay, bump the Dockerfile `FROM` to the new `manifest.dockerImage`
|
||||||
|
(pinned), rebuild, verify in the sandbox, then update.
|
||||||
|
- **Rollback**: repoint `CloudronManifest.json.dockerImage` to the previous
|
||||||
|
tag and rerun `cloudron update --force` (or the original appstore image).
|
||||||
|
Cloudron also snapshots the app before every image update.
|
||||||
|
|
||||||
|
## Hard-won pitfalls (do not re-learn these)
|
||||||
|
|
||||||
|
- `cloudron update` deploys **`manifest.dockerImage`** from the manifest in
|
||||||
|
the deploy directory — the `--image` flag alone is ignored (and the update
|
||||||
|
output still prints the `--image` value, so it *looks* right). The repo
|
||||||
|
manifest is a byte-identical copy of the OFFICIAL appstore manifest except
|
||||||
|
`dockerImage`; editing anything else in it (or hand-writing one that omits
|
||||||
|
the addons) strips `postgresql`/`mongodb`/`oidc`/`sendmail`/`localstorage`
|
||||||
|
from the app and crash-loops it (`mkdir: Read-only file system`). That is
|
||||||
|
what broke the sandbox on 2026-10-18.
|
||||||
|
- The remote builder's layer cache can pin COPY steps to old file content
|
||||||
|
even across "successful" rebuilds — use `--no-cache` after asset edits and
|
||||||
|
confirm by comparing container-file md5 against the repo file.
|
||||||
|
- Deploy `librechat.yaml` only via base64 round-trip. A `sed` insert once
|
||||||
|
broke the YAML indentation and LibreChat **silently fell back to defaults**
|
||||||
|
(all custom branding/models/MCP gone) with a single log line.
|
||||||
|
- Reading large files from the container over `cloudron exec` with a plain
|
||||||
|
`cat` resets the connection (`ECONNRESET`) — pipe through
|
||||||
|
`gzip | base64 | fold` and decode locally.
|
||||||
|
- `cloudron env set` alone does NOT disable the password form —
|
||||||
|
`ALLOW_EMAIL_LOGIN` must be flipped in `/app/data/env` (precedence above).
|
||||||
|
- Image generation in LibreChat is the **MCP tool** (composer **+** palette);
|
||||||
|
the agent "OpenAI Image Tools" path also exists. Image models in the chat
|
||||||
|
picker were the source of the old 500 errors and are deliberately excluded.
|
||||||
|
- No `branding:` block exists in LibreChat's config schema, and the login
|
||||||
|
page has no `.login-page`/`.primary` classes to target — theming happens by
|
||||||
|
overriding the Tailwind CSS variables on `html.light`/`html.dark`
|
||||||
|
(`--surface-primary`, `--accent-primary`, …) from the overlay `custom.css`.
|
||||||
|
|
||||||
|
|
||||||
|
## Per-member key auto-load (no API-key prompts)
|
||||||
|
|
||||||
|
Members never enter an API key. The portal provisions each member's LiteLLM
|
||||||
|
key at onboarding; the sandbox seeds it into LibreChat's user-key store:
|
||||||
|
|
||||||
|
- `GET /admin/librechat/keys` on the portal returns active members' chat keys
|
||||||
|
(dedicated `X-Sync-Token` auth — `LIBRECHAT_SYNC_TOKEN` on both apps).
|
||||||
|
- `/app/data/sync-keys.js` (in the LibreChat container, mirrored in the repo
|
||||||
|
as `sync-keys.js`) fetches that feed and seeds the `keys` collection
|
||||||
|
(chat endpoint, `apiKey: "user_provided"`) and `pluginauths`
|
||||||
|
(`mcp:inference-image` → `IC_API_KEY`, injected per-user via
|
||||||
|
`customUserVars` so image generation bills to the member's own team).
|
||||||
|
- Members without a sandbox account are skipped until they first log in; run
|
||||||
|
the sync after onboarding waves and on a schedule (the command is in the
|
||||||
|
repo README). Idempotent.
|
||||||
|
- Encryption matches LibreChat's own (`CREDS_KEY`/`CREDS_IV`, read from
|
||||||
|
`/proc/1/environ` — exec sessions don't inherit the app env).
|
||||||
|
|
||||||
|
> Terminal output redacts values after `apiKey:` as `***` even for harmless
|
||||||
|
> strings (`user_provided`, `${TTS_API_KEY}`). Probe lengths/counts instead
|
||||||
|
> of trusting redacted output.
|
||||||
|
|
||||||
|
## Verification (after any change)
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cloudron logs --app 4023ccbf-d57a-4500-97df-14de9cf3516e --lines 60 | \
|
||||||
|
grep -iE 'custom config|error|invalid|MCP|schedules'
|
||||||
|
curl -s https://librechat-test.inference.coop/api/config # appTitle, emailLoginEnabled
|
||||||
|
```
|
||||||
|
|
||||||
|
Expected: `Custom config file loaded`, `[MCP][inference-docs]` initialized
|
||||||
|
with tool `search_docs`, `inference-image` registered (its `Tools: undefined`
|
||||||
|
boot line is expected — per-user startup), `[schedules] engine started`,
|
||||||
|
`appTitle: Inference Co-op`, `emailLoginEnabled: false`. The login page shows
|
||||||
|
the co-op logo, the co-op favicon, a single "Login with Inference
|
||||||
|
Cooperative" (Cloudron SSO) path with no email/password form, **no
|
||||||
|
"Don't have an account?" line**, and the cream/green co-op color scheme.
|
||||||
|
|
||||||
|
After an IMAGE deploy, additionally run the four-step verification in the
|
||||||
|
repo README (API `manifest.dockerImage`, container md5 vs repo md5, served
|
||||||
|
HTML tags, rendered DOM colors).
|
||||||
@@ -79,7 +79,7 @@ The emoji in the model name marks its **value tier** (the same icons used in the
|
|||||||
| `🌱 greenpt/glm-5.3-flash` | GreenPT | policy | ✅ 100% renewable | — | 1M | $0.11 / $0.44 / $0.022 | Fast GLM model, renewable energy. |
|
| `🌱 greenpt/glm-5.3-flash` | GreenPT | policy | ✅ 100% renewable | — | 1M | $0.11 / $0.44 / $0.022 | Fast GLM model, renewable energy. |
|
||||||
| `🌱 greenpt/glm-5.3` | GreenPT | policy | ✅ 100% renewable | — | 1M | $1.10 / $4.40 / $0.275 | GLM flagship — strongest agentic model in the green tier. |
|
| `🌱 greenpt/glm-5.3` | GreenPT | policy | ✅ 100% renewable | — | 1M | $1.10 / $4.40 / $0.275 | GLM flagship — strongest agentic model in the green tier. |
|
||||||
| `🌱 regolo/apertus-70b` | Regolo | policy + ZDR | ✅ 100% renewable | ✅ | 65K | $0.46 / $2.44 / — | Fully open sovereign model (Apache-2.0), large/capable, image understanding. |
|
| `🌱 regolo/apertus-70b` | Regolo | policy + ZDR | ✅ 100% renewable | ✅ | 65K | $0.46 / $2.44 / — | Fully open sovereign model (Apache-2.0), large/capable, image understanding. |
|
||||||
| `🌱 regolo/qwen3.5-122b` | Regolo | policy + ZDR | ✅ 100% renewable | ✅ | 262K | $1.16 / $4.87 / — | Large vision+reasoning MoE (Qwen 3.5, 122B-A10B); biggest context in the roster. |
|
| `🌱 regolo/qwen3.5-122b` | Regolo | policy + ZDR | ✅ 100% renewable | ✅ | 262K | $1.16 / $4.87 / — | Large vision+reasoning MoE (Qwen 3.5, 122B-A10B); 262K context. |
|
||||||
|
|
||||||
Vision is verified working through the co-op gateway (base64 image input via
|
Vision is verified working through the co-op gateway (base64 image input via
|
||||||
`/v1/chat/completions`, standard OpenAI format). In the chat UI, just attach
|
`/v1/chat/completions`, standard OpenAI format). In the chat UI, just attach
|
||||||
|
|||||||
Reference in new issue
Block a user