inference-bot
2eb88f55b4
Audit of the operator docs against live reality (2026-11-04): - app-inventory.md: all 19 Cloudron apps with id, fqdn and registry image pin. Corrects the portal path (code/member-portal, not co-op/member-portal) and the panel path (local/coop.inference.admin-panel, not code/admin-panel). - maintenance/backups.md: the two backup systems; the deliberate per-user restic cache split and the --group-by "" retention fix, with why. - maintenance/woodpecker-ci.md: CI server + agent, image build/pin, manual-only trigger, git-lfs requirement, and its current not-yet-green state (agent's job-collection long-poll gets HTTP 500 from the server). - provider-pricing/: index + GreenPT, Tinfoil (local EHBP proxy architecture) and the Metis /v1/systemone decision-model route; regolo.md updated to record the gateway prices were applied. - agent-fleet.md: steward, infrastructure-bot and socialbot (Unix users, gateway units, teams); member-bot tier 2 planned but not yet provisioned. - README: file list; operator-token-runbook: correct the membership_sweep cadence (3x daily, not nightly); migration doc: post-migration addendum (16->19 apps, DNS split resolved, old box still on with apps stopped).
59 lines
2.8 KiB
Markdown
59 lines
2.8 KiB
Markdown
# The decision-model route — GreenPT "Metis" (`/v1/systemone`)
|
|
|
|
The gateway exposes a **decision model** — GreenPT's **Metis** (System One API) —
|
|
as a **pass-through route**, not a chat model.
|
|
|
|
## What it is, and why it is a route and not a model
|
|
|
|
Metis is a **decision** endpoint, not a chat completion: it takes a `state` plus up
|
|
to 32 typed questions and returns **probabilities / choices / scores**. It
|
|
generates no text (`output_tokens` is always 0), so **it cannot be a `model_list`
|
|
entry** — LiteLLM's chat pipeline has nothing to parse. It is therefore exposed as
|
|
a pass-through:
|
|
|
|
```
|
|
path: /v1/systemone
|
|
target: https://api.greenpt.ai/v1/systemone
|
|
auth: true # members call it with their normal co-op API key
|
|
```
|
|
|
|
- **It is API-only and deliberately NOT a chat-picker model.** It never appears in
|
|
the chat model list. Do not add it there.
|
|
- **Members authenticate with their own co-op key**, never the provider key — the
|
|
route's `Authorization` header injects the provider key server-side.
|
|
- Hosted by GreenPT (policy-based privacy, EU, 100% renewable), same
|
|
`GREENPT_API_KEY` the green chat models use.
|
|
|
|
## Pricing and the metering fix
|
|
|
|
- **$0.12 per 1M input tokens; output is free.**
|
|
|
|
Metering is **exact, from the token count GreenPT reports back**. Because Metis is
|
|
not a `model_list` entry, it is priced through the
|
|
`coop_passthrough_cost_per_token` map, keyed by the route path
|
|
(`/v1/systemone: 0.00000012`). `patch_passthrough_token_cost.py` (built into the
|
|
image) reads `usage.input_tokens` from the Metis response body at logging time and
|
|
writes the exact figure to member spend.
|
|
|
|
**Why this exists — record it, it will recur:** **generic pass-through routes
|
|
originally logged nothing.** A route that is not a `model_list` entry gets no
|
|
automatic pricing, so its calls were metered as $0 — a paid feature silently free.
|
|
The pass-through accounting had to be fixed so spend lands against the **calling
|
|
member's key**. If another pass-through route is ever added, it needs the same
|
|
treatment or it bills nothing.
|
|
|
|
`cost_per_request: 0.0001` on the route is now only a **fallback** — it applies if
|
|
a response arrives without a usable token count, so a call is never silently free.
|
|
Keep the two in sync.
|
|
|
|
## Access control gotcha (community edition)
|
|
|
|
The route is opened to any authenticated member key via
|
|
`coop_passthrough_open_to_keys` (consumed at build time by
|
|
`patch_passthrough_key_access.py`). **LiteLLM's own gate for `auth=true`
|
|
pass-through routes (`allowed_passthrough_routes`) is an Enterprise-only field**,
|
|
so on this community-edition proxy it can never be granted to member keys. The
|
|
only alternative, `auth: false`, would expose the route publicly and forward the
|
|
co-op's GreenPT key to anonymous callers. **Authentication is unchanged** — the
|
|
endpoint still requires a valid key.
|