Add implementation plan: API access, member dashboard, admin dashboard (Teams + broker architecture)
This commit is contained in:
inference-bot committed
1 parent
72afa8c78f
commit
f158135342
2 files changed
+116
-1
No files matched your search
@@ -82,7 +82,11 @@ During the pilot phase, Nathan Schneider serves as Managing Director and has sol
|
||||
- [Terms of Service](terms-of-service.md) — the agreement between the cooperative and its members about the service.
|
||||
- [Privacy Policy](privacy-policy.md) — how we handle your data, including what we can and can't see.
|
||||
|
||||
By creating your account, you agree to these terms.
|
||||
By creating your account, you agree to these terms.
|
||||
|
||||
## Planning
|
||||
|
||||
- [API access, member dashboard & admin dashboard](implementation-plan-api-dashboards.md) — the plan for API access, key management, and dashboards (not yet implemented).
|
||||
|
||||
---
|
||||
|
||||
|
||||
@@ -0,0 +1,111 @@
|
||||
# Implementation Plan: API Access, Member Dashboard & Admin Dashboard
|
||||
|
||||
**Status:** Planning (not yet implemented)
|
||||
**Last updated:** September 13, 2026
|
||||
|
||||
## Goal
|
||||
|
||||
Give members direct API access to the cooperative's models (alongside chat), a dashboard to view their usage and manage their own API keys, and an admin dashboard for co-op-wide monitoring.
|
||||
|
||||
## Guiding principles
|
||||
|
||||
1. **One unified budget.** Chat and API draw from the same $15/month credit pool. No separate accounting.
|
||||
2. **The member surface holds no master key.** A member-facing app must never touch the LiteLLM master key, the Cloudron token, or the Open Collective token. Key creation happens through a tightly-scoped broker.
|
||||
3. **Leverage LiteLLM's native primitives.** Teams for budgets/spend, spend-log endpoints for usage. Don't reinvent what the gateway already does.
|
||||
4. **No commercial dependencies for now.** Stay on the open-source LiteLLM build.
|
||||
|
||||
## Verified facts (from research)
|
||||
|
||||
- **LiteLLM Teams** (open-source) provide: per-team budget, per-team spend tracking, and key attribution to both user and team. This is the abstraction layer for "unified account + multiple keys."
|
||||
- **Spend endpoints** are live on our gateway: `/global/spend`, `/spend/logs`, `/user/info`, `/team/list` (all return 200).
|
||||
- **LiteLLM is v1.74.0, with Postgres** — so budgets and teams are fully supported.
|
||||
- **Key self-serve requires Premium.** The `team_admin` role and "team member can create keys" permission are both **✨ Premium Features**. On the OSS build, only the master key can mint keys. Hence the broker.
|
||||
- **Team members can still *view* key info on OSS** — read-only usage/key visibility is viable without Premium.
|
||||
|
||||
## Target architecture
|
||||
|
||||
```
|
||||
Member (browser) ──SSO──▶ Member Dashboard (new Cloudron app, "members" group)
|
||||
│ read-only: my spend, my keys, my usage
|
||||
│ writes: "create/revoke key" → Broker
|
||||
▼
|
||||
Broker (portal or admin app; holds master key)
|
||||
│ mints/revokes a key scoped to the member's team
|
||||
▼
|
||||
LiteLLM Gateway (Teams + budgets + spend logs)
|
||||
```
|
||||
|
||||
- **Chat key:** portal-held, per member, under their team, hidden from the member.
|
||||
- **API keys:** member-managed, under their team, visible in the member dashboard.
|
||||
- Both draw from the same team's $15/30d budget.
|
||||
|
||||
## Phases
|
||||
|
||||
### Phase 1 — Teams migration (foundation)
|
||||
|
||||
- Create a **Team per member** (`team_alias: member:<email>`) with `$15/30d` budget and the three models.
|
||||
- Two keys per team:
|
||||
- **Chat key** — created/handled by the portal injector (existing `member:<email>` key becomes this).
|
||||
- **API key(s)** — created via the broker on member request.
|
||||
- Update `provision_member()` to create the team + chat key (instead of a bare key).
|
||||
- Migrate existing members (Nathan, Liz, Dan, Ed, Lee).
|
||||
- Verify: chat still works end-to-end; spend now attributed per-team.
|
||||
|
||||
### Phase 2 — Broker (key management)
|
||||
|
||||
- A token-protected endpoint that, given a member's email + a requested key name:
|
||||
- Verifies the caller (SSO identity or portal JWT).
|
||||
- Mints a key scoped to that member's team (via master key).
|
||||
- Revokes a key on request.
|
||||
- Lives in the **portal** (which already holds the master key) or the **admin app**.
|
||||
- This is the only place the master key is used for key creation.
|
||||
|
||||
### Phase 3 — Member dashboard (new Cloudron app)
|
||||
|
||||
- Installed in the Cloudron **"members"** group → SSO-gated (same pattern as OpenWebUI/Loomio). No custom auth to build; trust the forwarded user identity.
|
||||
- **Read-only views** (via team-scoped endpoints, no master key):
|
||||
- My current spend vs. $15.
|
||||
- My API keys (list, with the chat key hidden).
|
||||
- Recent usage (from `/spend/logs` scoped to my key/team).
|
||||
- **Write actions** (call the broker):
|
||||
- Create API key.
|
||||
- Revoke API key.
|
||||
- Holds **no** master key, Cloudron token, or OC token.
|
||||
|
||||
### Phase 4 — Admin dashboard
|
||||
|
||||
- Co-op-wide view (admin-gated, holds master key):
|
||||
- All members + status (active/inactive/founder).
|
||||
- Aggregate spend + per-model breakdown.
|
||||
- Budget adjustment, key/team management.
|
||||
- Either a section of the portal or a distinct app. Decision deferred until Phase 3 is built (the admin view is lower urgency — LiteLLM's built-in UI already covers some of it).
|
||||
|
||||
### Phase 5 — Documentation
|
||||
|
||||
- Document the API endpoint (`https://gateway.inference.coop/v1`), OpenAI-compatible format, and "shares your $15 budget with chat."
|
||||
- Document the member dashboard (how to get/rotate keys).
|
||||
- Add an open question to `open-questions.md`: how to split/communicate the chat-vs-API budget if members want separate limits (future governance decision).
|
||||
|
||||
## Security model (the load-bearing part)
|
||||
|
||||
| Component | Holds master key? | Notes |
|
||||
|-----------|-------------------|-------|
|
||||
| Portal (control plane) | Yes | Existing: provisioning, key injection, webhook, admin |
|
||||
| Broker | Yes | Scoped: only mints/revokes team keys |
|
||||
| Member dashboard | **No** | SSO-gated, team-scoped, read-only + broker calls |
|
||||
| Admin dashboard | Yes | Admin-only |
|
||||
|
||||
The invariant: **a member's session can only ever (a) read their own team's data, or (b) ask the broker to act on their own team.** No member-path code can enumerate or touch another member's keys.
|
||||
|
||||
## Open decisions (to resolve before/at build time)
|
||||
|
||||
1. **Broker location** — portal (simplest, already has master key) vs. admin app. Leaning portal.
|
||||
2. **Member dashboard framework** — a small custom app (FastAPI, reusing the portal's patterns) vs. an app-store tool. Leaning custom: the app is a thin read/write wrapper, not a BI dashboard, and existing tools (Metabase/Superset/Baserow) don't fit "create my key."
|
||||
3. **Key naming/metadata** — what a member sees when they create a key (name, purpose tag) for their own cost tracking.
|
||||
4. **Premium later?** If the co-op grows an enterprise offering, LiteLLM Premium unlocks true key self-serve (drop the broker). Deferred.
|
||||
|
||||
## Risks
|
||||
|
||||
- **Broker is a new attack surface** — must be narrowly scoped and rate-limited.
|
||||
- **Migration touches existing members** — needs careful idempotency (reuse existing keys by alias).
|
||||
- **Spend-log scoping** — must verify `/spend/logs` can be filtered per-key/per-team so the member dashboard can't see others' usage.
|
||||
Reference in new issue
Block a user