rate limits: one bucket per ROUTE, not one per proxy address
ci/woodpecker/push/woodpecker Pipeline was canceled

Every request arrives through Cloudrons
This commit is contained in:
inference-admin committed 2026-11-13 22:45:48 +00:00
1 parent 6f6261eb1f
commit bed7f03063
1 file changed
+9 -1
+9 -1
View File
@@ -183,7 +183,15 @@ app = FastAPI(title="Inference Cooperative Member Portal")
# LiteLLM budget, NOT by rate — a member streaming a long conversation must # LiteLLM budget, NOT by rate — a member streaming a long conversation must
# never be throttled. Rate limiting here covers the control-plane surfaces: # never be throttled. Rate limiting here covers the control-plane surfaces:
# admin endpoints, the broker, the webhook, and the unauthenticated model list. # admin endpoints, the broker, the webhook, and the unauthenticated model list.
limiter = Limiter(key_func=get_remote_address) # Every request reaches this app through Cloudron's proxy, so get_remote_address
# returns the SAME address (172.18.0.1) for the panel, the dashboard, our own
# scheduler jobs and a member's browser alike. An IP-only key therefore makes each
# limit a co-op-wide budget rather than a per-caller one: on 2026-11-13 a guest
# invite was refused with 429 because the bucket had been drained by a minute of
# unrelated admin traffic (my own verification probes among it), and the operator
# saw a failure for an action nobody had abused. Keying on the PATH as well gives
# each route its own bucket, which is what a per-route limit is supposed to mean.
limiter = Limiter(key_func=lambda request: f"{get_remote_address(request)}:{request.url.path}")
app.state.limiter = limiter app.state.limiter = limiter