portal-dev 6475a51067 metrics: expose + persist measured-vs-estimated provenance (#4, #6)
Thread the per-factor source through impact_for() instead of discarding it.
Numeric output (energy_wh, co2e_g) is byte-identical — this is a reporting
change, not a recalculation.

#6 persistence: additive columns tokens_measured, tokens_estimated and
factor_revision (a stable hash of the factor table) on metrics_daily, added via
the existing guarded ALTER TABLE idiom in both app/main.py and
metrics_snapshot.py. Existing rows keep NULL.

#4 API: metrics_series() selects the new columns and derives a per-row
provenance indicator (measured | mixed | estimated | none) from the split; both
/admin/metrics/series and /broker/metrics return it. NULL/unknown splits report
'none', never a confident 'measured'.

docs/energy-metrics.md documents the indicator, the new columns, and the
forward-only-vs-backfill behaviour.
2026-10-30 19:12:20 +00:00

Member Portal — membership middleware for the Inference Cooperative

Reconciles three systems into one membership lifecycle:

System Role Source of truth for
Open Collective Billing Who is a paying member
Cloudron Identity/SSO Who can log in
LiteLLM Inference Who can use models, and how much

Architecture

Open Collective (billing)
        │ webhook
        ▼
┌─────────────────────────────┐
│  Member Portal (this app)   │
│  - OC webhook handler       │
│  - Cloudron user management │
│  - LiteLLM key management   │
│  - key-injection proxy      │
└─────────────────────────────┘
        ▲
        │ X-User-Email header (LibreChat forwards identity)
        │
LibreChat ──→ Member Portal ──→ LiteLLM ──→ backend

The three responsibilities

  1. Webhook handler (POST /webhook/opencollective) — listens for Open Collective membership events and activates/deactivates members.
  2. Key injector (/v1/*) — reads the member's email from the X-User-Email header, looks up their LiteLLM key, and forwards the request to the gateway with that key.
  3. Admin (/health, /) — health and status.

Configuration (environment variables)

Variable Purpose
CLOUDRON_API Cloudron API origin (auto-set by Cloudron)
CLOUDRON_TOKEN Cloudron API token (Read+Write)
MEMBERS_GROUP_ID Cloudron group ID for members (grants chat access)
LITELLM_BASE LiteLLM gateway URL
LITELLM_MASTER_KEY LiteLLM master key
OPENCOLLECTIVE_WEBHOOK_SECRET Optional webhook signature secret
PORTAL_KEK Required. Base64-encoded 32-byte key-encryption key. Member chat keys (members.key_token) are stored AES-256-GCM-encrypted at rest (enc1: prefix) and decrypted in memory only when served (dashboard, LibreChat key feed, LiteLLM management). Generate with python3 -c "import base64,os;print(base64.b64encode(os.urandom(32)).decode())". If unset, the portal runs but refuses to encrypt new keys, keeps legacy plaintext, and logs CRITICAL. Rotating it requires re-encrypting every row (decrypt with the old key, encrypt with the new).

Tier budgets

Governance decision (set in Loomio). Defaults:

  • free — $2/month of tokens
  • member — $15/month
  • supporter — $30/month

LibreChat wiring

Point LibreChat's custom endpoint at this portal (not directly at LiteLLM), and add the identity header:

endpoints:
  custom:
    - name: "Inference Cooperative"
      apiKey: "${LITELLM_KEY}"
      baseURL: "https://portal.inference.coop/v1"
      headers:
        X-User-Email: "{{LIBRECHAT_USER_EMAIL}}"
      models:
        default: ["deepseek-v4-flash", "gpt-oss-120b"]
        fetch: true

Status

Scaffold — the three handlers are stubbed with the correct integration points. Remaining work before production:

  • Verify Open Collective webhook event names + payload shape → order.processed (every payment), new member (first only), firstPayment flag
  • Verify Cloudron user-creation API payload (role/group assignment) → POST /api/v1/users with {username, email, displayName, role, active} → group assignment via PUT /api/v1/users/:userId/groups with {groupIds: [...]}
  • Add a persistent store (SQLite) for email→key mapping → key/list returns token strings (not objects), so we store email→token locally
  • Verify LiteLLM key/generate + key/list payload shapes against live gateway → key/generate returns {key: "sk-..."}; key/list returns {keys: ["<token>", ...]}
  • Add HMAC signature verification for the OC webhook (if OC supports it)
  • Wire the OIDC addon for admin access

Scheduled tasks (scheduler addon)

Three jobs run inside the container via CloudronManifest.json → addons.scheduler:

  • membership_sweep (nightly 3am) — sweep.sh calls the portal's own /admin/sweep (auth: X-Admin-Token = WEBHOOK_TOKEN) to reconcile members against the live Open Collective list: provisions missed signups, deactivates lapsed members, syncs Loomio + Matrix. Idempotent; a missed webhook self-heals within 24 hours.
  • rag_refresh (nightly 3:30am) — rag_refresh.py pulls this repo's sibling docs from the Gitea raw API (public, no credentials), uploads changed files to Open WebUI (OWUI_API_KEY), attaches them to the shared knowledge collection (DOCS_KNOWLEDGE_ID), reindexes, and re-grants the members group. State persists in /app/data/rag_refresh_state.json.
  • metrics_snapshot (nightly 3:15am) — metrics_snapshot.py records one row per day into the metrics_daily table: members, newsletter subscribers, Matrix membership, spend, request counts, and energy/CO₂. A source that cannot be read yields NULL for that metric rather than a zero. --backfill rebuilds history idempotently. Methodology: Energy and carbon metrics.

Verify after any manifest change: the deploy log must show setupAddons: ... scheduler ... listing all three tasks, and each script must succeed when run manually inside the container.

Energy & carbon metrics

The daily snapshot also computes energy (Wh) and carbon (gCO₂e) from output tokens, using per-model factors whose provenance varies: measured for GreenPT, assumed equal for one sibling model, and an explicit order-of-magnitude estimate for providers that disclose nothing. Only output tokens count, and a day with no measured tokens is recorded as NULL rather than zero.

Energy and carbon metrics documents the factor table, the formulas, the tripwire that catches unit errors, the schedule, and — most importantly — which parts are measurements and which are estimates. What members are told is in co-op/docs/energy.md.

S
Description
Membership middleware: syncs Open Collective members with Cloudron users and LiteLLM keys
Readme
4.9 MiB
0 Stars 2 Watchers 0 Forks
Languages
Python 99%
Shell 0.7%
Dockerfile 0.3%