co-op/support
3
0 Fork 0
Code Issues 13 Pull Requests Actions Packages Projects Releases Wiki Activity

CORS allow-list on the LiteLLM gateway: open to all origins, or app-specific? (decision needed) #38

New Issue
Closed
opened 2026-10-30 23:33:54 +00:00 by inference-bot · 2 comments
inference-bot commented 2026-10-30 23:33:54 +00:00
Owner
Copy link
Copy source

A member raised this on Loomio. Nathan's read is that it's an ops decision rather than a governance one, so it lives here rather than as a poll. Filing the facts I verified live, so the discussion doesn't start from guesses.

What was asked

Whether gateway.inference.coop should send CORS exceptions for more apps — specifically, whether to open it to all origins (LITELLM_CORS_ORIGINS=*) so members can use browser-based apps we don't host, versus naming apps as needed.

What is configured today (verified 2026-10-30, not from memory)

LITELLM_CORS_ORIGINS=https://chat.inference.coop — one origin, not two.

Live preflight against https://gateway.inference.coop/v1/chat/completions:

  • Origin: https://chat.inference.coop → 200, access-control-allow-origin: https://chat.inference.coop
  • Origin: https://evil.example → 400, no allow-origin header
  • Origin: null → 400 — correctly refused (null is the sandboxed-iframe / file:// origin and a standard bypass attempt)

Correction worth noting: chat-legacy.inference.coop is not in the list — it returns 400. It is also not an older LibreChat: it runs com.openwebui.cloudronapp (Open WebUI), a separate app that is up and serving. [#14] records that this env var was renamed (LITELLM_CORS_ALLOWED_ORIGINS → LITELLM_CORS_ORIGINS) with "re-verify CORS after the upgrade" as a follow-up, which is the likely origin of the belief that two origins are set. Open WebUI normally calls the gateway server-side, where CORS does not apply, so this is only a real gap if members are using its Direct Connections mode. Worth confirming rather than assuming.

The security reasoning (which I think is sound)

  • CORS is not an authentication boundary here. Requests carry a per-member bearer key; the browser does not attach anything implicitly.
  • A leaked key is curl-able regardless of the allow-list. If a member pastes a key into a third-party web app, that app can use it directly — an origin list does not change that.
  • What actually protects the co-op is per-key budgets, model scoping and revocation, all of which we already enforce. With *, the budget is the thing standing between a leaked key and an unbounded bill.
  • Most large providers do send permissive CORS. The notable exception: Anthropic requires an explicit anthropic-dangerous-direct-browser-access opt-in rather than opening up. So a narrow allow-list is a defensible choice, not an odd one.
  • The honest downside of opening it: it removes friction, which makes casual key-sharing more likely. That argues for a docs note, not for keeping the list closed.

If we open it — the change and its risks

One env var on the gateway app. Two things to know:

  1. It restarts the app (a few seconds of gateway downtime). Doing it during the migration window costs nothing, since the box restarts anyway.
  2. The gateway also sends access-control-allow-credentials: true, and * together with that is normally invalid in browsers — which would break chat if the middleware sent a literal *. I checked Starlette 1.3.1's CORSMiddleware in the running container: with allow_origins=["*"] and credentials enabled it echoes the requesting origin instead of sending *, which stays valid. So the change should work without breaking chat — but it must be re-verified, because that behaviour is a library implementation detail, not a contract.

Acceptance criteria (if we proceed)

  • Preflight from an unrelated origin returns 200 with access-control-allow-origin echoing that origin
  • chat.inference.coop still returns 200 with its origin echoed, and a real browser completion from chat still works
  • Noted, not necessarily fixed: null will also be echoed under * — reachable only by a page that already holds a key
  • A docs line stating that pasting a key into a third-party web app exposes it; use budgeted, revocable keys
  • chat-legacy.inference.coop explicitly added or its Direct-Connections use confirmed unnecessary

Options

  1. Open to all origins (recommended): one var, matches mainstream providers, no real loss given bearer keys + budgets.
  2. Keep it app-specific, adding chat-legacy. if that use is intended: least change; leaves members unable to use third-party browser apps.
  3. Request-and-add process: middle ground, negligible security gain for ongoing ops cost.

Reversible either way, and small. [#14]: #14
[#24]: #24

A member raised this on Loomio. Nathan's read is that it's an **ops decision** rather than a governance one, so it lives here rather than as a poll. Filing the facts I verified live, so the discussion doesn't start from guesses. ## What was asked Whether `gateway.inference.coop` should send CORS exceptions for more apps — specifically, whether to open it to **all origins** (`LITELLM_CORS_ORIGINS=*`) so members can use browser-based apps we don't host, versus naming apps as needed. ## What is configured today (verified 2026-10-30, not from memory) `LITELLM_CORS_ORIGINS=https://chat.inference.coop` — **one** origin, not two. Live preflight against `https://gateway.inference.coop/v1/chat/completions`: - `Origin: https://chat.inference.coop` → **200**, `access-control-allow-origin: https://chat.inference.coop` - `Origin: https://evil.example` → **400**, no allow-origin header - `Origin: null` → **400** — correctly refused (`null` is the sandboxed-iframe / `file://` origin and a standard bypass attempt) **Correction worth noting:** `chat-legacy.inference.coop` is **not** in the list — it returns 400. It is also not an older LibreChat: it runs `com.openwebui.cloudronapp` (Open WebUI), a separate app that is up and serving. [#14] records that this env var was *renamed* (`LITELLM_CORS_ALLOWED_ORIGINS` → `LITELLM_CORS_ORIGINS`) with "re-verify CORS after the upgrade" as a follow-up, which is the likely origin of the belief that two origins are set. Open WebUI normally calls the gateway **server-side**, where CORS does not apply, so this is only a real gap if members are using its *Direct Connections* mode. Worth confirming rather than assuming. ## The security reasoning (which I think is sound) - **CORS is not an authentication boundary here.** Requests carry a per-member bearer key; the browser does not attach anything implicitly. - **A leaked key is curl-able regardless of the allow-list.** If a member pastes a key into a third-party web app, that app can use it directly — an origin list does not change that. - **What actually protects the co-op is per-key budgets, model scoping and revocation**, all of which we already enforce. With `*`, the budget is the thing standing between a leaked key and an unbounded bill. - Most large providers do send permissive CORS. The notable exception: **Anthropic requires an explicit `anthropic-dangerous-direct-browser-access` opt-in** rather than opening up. So a narrow allow-list is a defensible choice, not an odd one. - **The honest downside of opening it:** it removes friction, which makes casual key-sharing more likely. That argues for a docs note, not for keeping the list closed. ## If we open it — the change and its risks One env var on the gateway app. Two things to know: 1. **It restarts the app** (a few seconds of gateway downtime). Doing it during the migration window costs nothing, since the box restarts anyway. 2. **The gateway also sends `access-control-allow-credentials: true`**, and `*` together with that is normally invalid in browsers — which would break chat if the middleware sent a literal `*`. I checked Starlette 1.3.1's `CORSMiddleware` in the running container: with `allow_origins=["*"]` **and** credentials enabled it **echoes the requesting origin** instead of sending `*`, which stays valid. So the change should work without breaking chat — but it must be re-verified, because that behaviour is a library implementation detail, not a contract. ## Acceptance criteria (if we proceed) - [ ] Preflight from an unrelated origin returns 200 with `access-control-allow-origin` echoing that origin - [ ] `chat.inference.coop` still returns 200 with its origin echoed, and a real browser completion from chat still works - [ ] Noted, not necessarily fixed: `null` will also be echoed under `*` — reachable only by a page that already holds a key - [ ] A docs line stating that pasting a key into a third-party web app exposes it; use budgeted, revocable keys - [ ] `chat-legacy.inference.coop` explicitly added **or** its Direct-Connections use confirmed unnecessary ## Options 1. **Open to all origins** (recommended): one var, matches mainstream providers, no real loss given bearer keys + budgets. 2. **Keep it app-specific**, adding `chat-legacy.` if that use is intended: least change; leaves members unable to use third-party browser apps. 3. **Request-and-add process**: middle ground, negligible security gain for ongoing ops cost. Reversible either way, and small. [#14]: https://git.inference.coop/co-op/support/issues/14 [#24]: https://git.inference.coop/co-op/support/issues/24
bo commented 2026-10-31 12:19:11 +00:00
Copy link
Copy source

I agree with this proposal. As I understand it, CORS is designed for cookies and adds almost no security for API keys anyways.

I agree with this proposal. As I understand it, CORS is designed for cookies and adds almost no security for API keys anyways.
inference-bot commented 2026-10-31 17:22:23 +00:00
Author
Owner
Copy link
Copy source

Implemented and verified on the gateway (image 1.0.0-1793467213663).

Why the earlier attempt didn't work, which is worth recording: start.sh in the app hardcoded
export LITELLM_CORS_ORIGINS="https://chat.inference.coop". The export overrode the Cloudron app
env, so setting LITELLM_CORS_ORIGINS on the app read back correctly and did nothing. It now reads
${LITELLM_CORS_ORIGINS:-https://chat.inference.coop}, so the app env drives it and the old lock
remains only as a fallback. Thanks to @staeiou for raising it and to @bo for the reasoning on CORS
and cookies.

Decision: allow all origins. Set via the app env, and the comment in start.sh now says why —
CORS is not an auth boundary here; every request carries a per-member bearer key the browser never
attaches implicitly, so an origin list buys nothing against a party that already holds a key. The
control that matters is the per-key budget, model scoping and revocation.

Verified live (before → after):

  • Origin: https://some-other-app.example — HTTP 400, no allow-origin → 200, *
  • Origin: https://chat-legacy.inference.coop — HTTP 400 → 200, * (the Open WebUI app now works cross-origin, which closes that part of the report)
  • Origin: https://chat.inference.coop — HTTP 200 → 200, * (no regression)

On the credentials caveat: a bare * is invalid in browsers if Access-Control-Allow-Credentials
is also sent — and it is not sent now, so * is valid and chat keeps working. Had it been, this
change would have broken the existing setup rather than widened it.

Implemented and verified on the gateway (image `1.0.0-1793467213663`). **Why the earlier attempt didn't work, which is worth recording:** `start.sh` in the app hardcoded `export LITELLM_CORS_ORIGINS="https://chat.inference.coop"`. The `export` overrode the Cloudron app env, so setting `LITELLM_CORS_ORIGINS` on the app read back correctly and did nothing. It now reads `${LITELLM_CORS_ORIGINS:-https://chat.inference.coop}`, so the app env drives it and the old lock remains only as a fallback. Thanks to @staeiou for raising it and to @bo for the reasoning on CORS and cookies. **Decision: allow all origins.** Set via the app env, and the comment in `start.sh` now says why — CORS is not an auth boundary here; every request carries a per-member bearer key the browser never attaches implicitly, so an origin list buys nothing against a party that already holds a key. The control that matters is the per-key budget, model scoping and revocation. **Verified live (before → after):** - `Origin: https://some-other-app.example` — HTTP **400, no allow-origin** → **200, `*`** - `Origin: https://chat-legacy.inference.coop` — HTTP **400** → **200, `*`** (the Open WebUI app now works cross-origin, which closes that part of the report) - `Origin: https://chat.inference.coop` — HTTP **200** → **200, `*`** (no regression) **On the credentials caveat:** a bare `*` is invalid in browsers if `Access-Control-Allow-Credentials` is also sent — and it is *not* sent now, so `*` is valid and chat keeps working. Had it been, this change would have broken the existing setup rather than widened it.
Sign in to join this conversation.
No Branch/Tag Specified
Labels
Clear labels
Community
Compute
Design
Finance
Governance
Infrastructure
No labels
Milestone
No items
No Milestone
Projects
Clear projects
No projects
Assignees
Clear assignees
inference-admin (Admin) inference-bot (InferenceBot)
No Assignees
2 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: co-op/support#38