diff --git a/CloudronManifest.json b/CloudronManifest.json index 2036ded..c72a0fa 100644 --- a/CloudronManifest.json +++ b/CloudronManifest.json @@ -11,7 +11,7 @@ "memoryLimit": 2147483648, "manifestVersion": 2, "website": "https://litellm.ai", - "contactEmail": "botbot@hermes", + "contactEmail": "info@inference.coop", "icon": "file://logo.png", "addons": { "postgresql": {}, diff --git a/README.md b/README.md index 6664eb3..7a701f1 100644 --- a/README.md +++ b/README.md @@ -16,16 +16,16 @@ LiteLLM is the single gateway between members and the inference backends. It han - **Cloudron Redis** for rate limiting and caching - **Automatic SSL, backups, and sandboxing** (Cloudron-managed) - **Master key + salt key** auto-generated on first start, persisted in `/app/data` -- Hardened for the public internet: admin UI and API docs disabled at the proxy; CORS locked to the chat origin +- Hardened for the public internet: LiteLLM's built-in API docs surfaces are disabled (`NO_DOCS`/`NO_REDOC`/`NO_OPENAPI` → live 404s). The static `/ui/` admin-UI bundle **is served** on this version (see *Admin access*); logging into it still requires the master key. CORS is deliberately open (`LITELLM_CORS_ORIGINS=*`, co-op/support#38) — auth is the per-member bearer key, not origin checking. ## Architecture ``` -Members → Open WebUI (chat) / own tools (API) → member portal (key injection) +Members → LibreChat (chat) / own tools (API) → member portal (key injection) → LiteLLM gateway (this app) ├→ Tinfoil (TEE enclaves — architectural privacy) ├→ GreenPT (100% renewable energy) - └→ PublicAI (publicly developed, sovereign models) + └→ Regolo (EU-hosted, renewable + zero data retention) ``` LiteLLM is the control plane (keys, metering, routing). It does **not** run models itself — it forwards to providers, each chosen by members for a different value. A [tinfoil-proxy](https://github.com/tinfoilsh/tinfoil-proxy) sidecar provides EHBP end-to-end encryption to Tinfoil's enclaves. @@ -76,7 +76,7 @@ After install, set the provider API keys in Cloudron → app → Settings → En - `TINFOIL_API_KEY` — Tinfoil (https://tinfoil.sh/) - `GREENPT_API_KEY` — GreenPT (https://greenpt.ai/) -- `PUBLICAI_API_KEY` — PublicAI (https://publicai.co/) +- `REGOLO_API_KEY` — Regolo (regolo.ai; replaced PublicAI, removed 2026-10-16) The following are auto-generated and should **not** be set manually: @@ -95,7 +95,26 @@ configuration. Change the repo and redeploy; never edit the volume copy directly ### Admin access -The LiteLLM admin UI and API docs are **disabled** on the public gateway (hardened by default; see `start.sh`). Administrative operations run through the [member portal](https://git.inference.coop/code/member-portal), which holds the master key server-side. If direct LiteLLM admin access is ever needed, re-enable the UI in `start.sh`, redeploy, and use the master key from `/app/data/.master_key` — then disable it again. +The state of the admin surfaces on the pinned v1.103.2, wire-verified 2026-11-05: + +- The built-in **API docs are disabled** (`NO_DOCS`/`NO_REDOC`/`NO_OPENAPI` in + `start.sh`) — `/docs`, `/redoc` and `/openapi.json` return 404. +- The **static admin UI bundle is still served at `/ui/`** — upstream's + `DISABLE_ADMIN_UI` env (which we set) only disables the UI's *SSO login + redirect* routes in this version; nothing on v1.103.2 stops the unconditional + `app.mount("/ui", …)`. So `GET /ui/` serves the login page. **Logging in + still requires the master key** (auto-generated here, never published), and + administrative operations normally run through the + [member portal](https://git.inference.coop/code/member-portal), which holds + the master key server-side. +- If the login surface itself should disappear, it needs a build-time patch + (strip the `/ui` mount from `proxy_server` in the Dockerfile, the same + non-fatal patch pattern the other 8 build patches use) — there is no env/flag + for it on this version. Until then treat `/ui/` as a master-key-only + endpoint, like any other control-plane route. + +If direct LiteLLM admin access is ever needed, log in at `/ui/` with the +master key from `/app/data/.master_key` and lock it away again afterwards. --- @@ -159,7 +178,7 @@ expect a 15–20 min maintenance window and **never panic-restart mid-migration* 2. Generate a temp key with the master key → if `/key/generate` 500s, the schema is incomplete; fix the migration before anything else 3. Use the **temp key** (not master) for `GET /v1/models` → full catalog 4. Use the **temp key** for a small `POST /v1/chat/completions` → content + usage -5. OPTIONS preflight from a foreign origin → 400; from `chat.inference.coop` → allowed +5. OPTIONS preflight: `LITELLM_CORS_ORIGINS` is set at the app level (`*` by co-op decision, co-op/support#38) — a preflight from any origin answers with the matching `access-control-allow-origin` header rather than a 400. Auth is the bearer key, not the origin. 6. Cloudron app health = healthy, then delete the temp key ### Before updating @@ -181,13 +200,13 @@ expect a 15–20 min maintenance window and **never panic-restart mid-migration* |----------|----------|---------| | `TINFOIL_API_KEY` | Yes | Tinfoil provider (TEE inference) | | `GREENPT_API_KEY` | Yes | GreenPT provider (renewable energy) | -| `PUBLICAI_API_KEY` | Yes | PublicAI provider (public models) | -| `LITELLM_CORS_ORIGINS` | Set by start.sh | CORS allowlist (chat origin only) | +| `REGOLO_API_KEY` | Yes | Regolo provider (EU, renewable + ZDR) | +| `LITELLM_CORS_ORIGINS` | Set at app level | Currently `*` by co-op decision (co-op/support#38); `start.sh` falls back to the chat origin only if the app env omits it | | `LITELLM_MIGRATION_DIR` | Set by start.sh | Persistent Prisma migrations ledger | ## Connecting chat clients -Open WebUI does **not** talk to this gateway directly — it routes through the [member portal](https://git.inference.coop/code/member-portal), which authenticates the member and injects their personal LiteLLM key per request. Members building their own tools create API keys on the [member dashboard](https://dashboard.inference.coop) and use: +LibreChat (chat) does **not** talk to this gateway directly — it routes through the [member portal](https://git.inference.coop/code/member-portal), which authenticates the member and injects their personal LiteLLM key per request. Members building their own tools create API keys on the [member dashboard](https://dashboard.inference.coop) and use: - **API Base URL:** `https://gateway.inference.coop/v1` (or the portal's proxy endpoint for chat-integrated tools) - **API Key:** their own virtual key from the dashboard diff --git a/start.sh b/start.sh index 060d346..be94550 100644 --- a/start.sh +++ b/start.sh @@ -77,10 +77,13 @@ export LITELLM_MIGRATION_DIR="/app/data/migrations" export UI_USERNAME="admin" export UI_PASSWORD="${LITELLM_MASTER_KEY}" -# DISABLE the admin UI and API docs. The gateway must stay publicly reachable -# for the /v1 API (member keys), but we don't want to advertise a login-gated -# admin console (/ui) or LiteLLM's Swagger/ReDoc/openapi surfaces. Admin -# visibility comes from the admin dashboard / the API directly. +# Nothing disables the static /ui admin bundle on the pinned v1.103.2 (there +# is no flag for it): DISABLE_ADMIN_UI="True" only turns off the UI's SSO +# login-redirect routes upstream — GET /ui/ still serves the static login +# page. Logging into it still requires the master key, and the docs surfaces +# below ARE fully disabled (NO_DOCS/NO_REDOC/NO_OPENAPI → 404s). See README +# "Admin access" for the wire-verified state and the patch needed to remove +# the bundle itself. Kept set because the SSO-redirect disable is still real. export DISABLE_ADMIN_UI="True" export NO_DOCS="True" export NO_REDOC="True" @@ -166,6 +169,14 @@ if [ -f /app/code/config.yaml ] && ! cmp -s /app/code/config.yaml "$CONFIG_FILE" echo "Updating /app/data/config.yaml from the image (repo is the source of truth)" cp /app/code/config.yaml "$CONFIG_FILE" fi +# NOTE on the fallback config written above: it can never be SERVED. The very +# same boot's sync above overwrites it with the image's (repo) config whenever +# they differ — including the boot right after the fallback ran. It is dead +# by construction; it exists only so the container is never left with an +# empty config file if the volume copy vanishes. Historically this block +# carried stale api.tinfoil.sh models that could have resurrected — that +# cannot happen through this path, and the shipped /app/code/config.yaml is +# the live catalog. # --- Start LiteLLM --- echo "Starting LiteLLM on port 4000..."