rate limits: one bucket per ROUTE, not one per proxy address
ci/woodpecker/push/woodpecker Pipeline was canceled
ci/woodpecker/push/woodpecker Pipeline was canceled
Every request arrives through Cloudrons
This commit is contained in:
1 parent
6f6261eb1f
commit
bed7f03063
1 file changed
+9
-1
+9
-1
@@ -183,7 +183,15 @@ app = FastAPI(title="Inference Cooperative Member Portal")
|
|||||||
# LiteLLM budget, NOT by rate — a member streaming a long conversation must
|
# LiteLLM budget, NOT by rate — a member streaming a long conversation must
|
||||||
# never be throttled. Rate limiting here covers the control-plane surfaces:
|
# never be throttled. Rate limiting here covers the control-plane surfaces:
|
||||||
# admin endpoints, the broker, the webhook, and the unauthenticated model list.
|
# admin endpoints, the broker, the webhook, and the unauthenticated model list.
|
||||||
limiter = Limiter(key_func=get_remote_address)
|
# Every request reaches this app through Cloudron's proxy, so get_remote_address
|
||||||
|
# returns the SAME address (172.18.0.1) for the panel, the dashboard, our own
|
||||||
|
# scheduler jobs and a member's browser alike. An IP-only key therefore makes each
|
||||||
|
# limit a co-op-wide budget rather than a per-caller one: on 2026-11-13 a guest
|
||||||
|
# invite was refused with 429 because the bucket had been drained by a minute of
|
||||||
|
# unrelated admin traffic (my own verification probes among it), and the operator
|
||||||
|
# saw a failure for an action nobody had abused. Keying on the PATH as well gives
|
||||||
|
# each route its own bucket, which is what a per-route limit is supposed to mean.
|
||||||
|
limiter = Limiter(key_func=lambda request: f"{get_remote_address(request)}:{request.url.path}")
|
||||||
app.state.limiter = limiter
|
app.state.limiter = limiter
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
Reference in new issue
Block a user