inference-bot
2eb88f55b4
Audit of the operator docs against live reality (2026-11-04): - app-inventory.md: all 19 Cloudron apps with id, fqdn and registry image pin. Corrects the portal path (code/member-portal, not co-op/member-portal) and the panel path (local/coop.inference.admin-panel, not code/admin-panel). - maintenance/backups.md: the two backup systems; the deliberate per-user restic cache split and the --group-by "" retention fix, with why. - maintenance/woodpecker-ci.md: CI server + agent, image build/pin, manual-only trigger, git-lfs requirement, and its current not-yet-green state (agent's job-collection long-poll gets HTTP 500 from the server). - provider-pricing/: index + GreenPT, Tinfoil (local EHBP proxy architecture) and the Metis /v1/systemone decision-model route; regolo.md updated to record the gateway prices were applied. - agent-fleet.md: steward, infrastructure-bot and socialbot (Unix users, gateway units, teams); member-bot tier 2 planned but not yet provisioned. - README: file list; operator-token-runbook: correct the membership_sweep cadence (3x daily, not nightly); migration doc: post-migration addendum (16->19 apps, DNS split resolved, old box still on with apps stopped).
2.8 KiB
The decision-model route — GreenPT "Metis" (/v1/systemone)
The gateway exposes a decision model — GreenPT's Metis (System One API) — as a pass-through route, not a chat model.
What it is, and why it is a route and not a model
Metis is a decision endpoint, not a chat completion: it takes a state plus up
to 32 typed questions and returns probabilities / choices / scores. It
generates no text (output_tokens is always 0), so it cannot be a model_list
entry — LiteLLM's chat pipeline has nothing to parse. It is therefore exposed as
a pass-through:
path: /v1/systemone
target: https://api.greenpt.ai/v1/systemone
auth: true # members call it with their normal co-op API key
- It is API-only and deliberately NOT a chat-picker model. It never appears in the chat model list. Do not add it there.
- Members authenticate with their own co-op key, never the provider key — the
route's
Authorizationheader injects the provider key server-side. - Hosted by GreenPT (policy-based privacy, EU, 100% renewable), same
GREENPT_API_KEYthe green chat models use.
Pricing and the metering fix
- $0.12 per 1M input tokens; output is free.
Metering is exact, from the token count GreenPT reports back. Because Metis is
not a model_list entry, it is priced through the
coop_passthrough_cost_per_token map, keyed by the route path
(/v1/systemone: 0.00000012). patch_passthrough_token_cost.py (built into the
image) reads usage.input_tokens from the Metis response body at logging time and
writes the exact figure to member spend.
Why this exists — record it, it will recur: generic pass-through routes
originally logged nothing. A route that is not a model_list entry gets no
automatic pricing, so its calls were metered as $0 — a paid feature silently free.
The pass-through accounting had to be fixed so spend lands against the calling
member's key. If another pass-through route is ever added, it needs the same
treatment or it bills nothing.
cost_per_request: 0.0001 on the route is now only a fallback — it applies if
a response arrives without a usable token count, so a call is never silently free.
Keep the two in sync.
Access control gotcha (community edition)
The route is opened to any authenticated member key via
coop_passthrough_open_to_keys (consumed at build time by
patch_passthrough_key_access.py). LiteLLM's own gate for auth=true
pass-through routes (allowed_passthrough_routes) is an Enterprise-only field,
so on this community-edition proxy it can never be granted to member keys. The
only alternative, auth: false, would expose the route publicly and forward the
co-op's GreenPT key to anonymous callers. Authentication is unchanged — the
endpoint still requires a valid key.