Compare commits
3
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
  solo.istandClaude Fable 5.1
|
3302ae35f4 | ||
|
solo.ist
|
11ea40485c | ||
|
solo.ist
|
5aa6098e5b |
No files matched your search
@@ -85,70 +85,3 @@ substitute our base URL and your member key for `apiKey` — the protocol value
|
||||
stays the same. Some tools append `/chat/completions` to the base URL themselves;
|
||||
if your base URL already ends in `/v1`, that's correct and you should not add
|
||||
`/chat/completions`.
|
||||
|
||||
## What the gateway guarantees (and what it doesn't)
|
||||
|
||||
Checked against `gateway.inference.coop` (LiteLLM 1.103.2 at the time of
|
||||
writing) with a member key on 2026-10-19, re-checked 2026-10-26/27. If any of this drifts, open a PR against this file.
|
||||
|
||||
- **Supported route:** `POST /v1/chat/completions`. The audio, image and
|
||||
embedding routes work for the models marked for them in the
|
||||
[model list](models.md). `POST /v1/messages` (Anthropic format) answers for
|
||||
some models (`greenpt/deepseek-v4.1-flash`, `tinfoil/deepseek-v4-1-flash`,
|
||||
`tinfoil/glm-5.3`, `tinfoil/gpt-oss-120b`) and returns 404 for others
|
||||
(`greenpt/glm-5.3-flash`): it is LiteLLM's built-in translation layer, not
|
||||
something the co-op tests or supports. Where it works it carries tools and
|
||||
tool results correctly, but it drops the `cost` field and any reasoning
|
||||
content, so prefer chat completions for anything you meter.
|
||||
- **Fallbacks are silent in the body, visible in the headers.** Some models
|
||||
have a fallback chain (today `greenpt/deepseek-v4.1-flash` →
|
||||
`tinfoil/deepseek-v4-1-flash` → `tinfoil/gpt-oss-120b`). If the first
|
||||
provider fails, the next one serves your request at its own price and its
|
||||
own privacy posture. Every response carries `x-litellm-model-group` (what
|
||||
you asked for), `x-litellm-model-name` and `x-litellm-model-api-base` (what
|
||||
answered), and `x-litellm-attempted-fallbacks` (how many hops). Read them
|
||||
if the provider matters to you.
|
||||
- **Cost per call is in the headers too:** `x-litellm-response-cost` in USD,
|
||||
split into `-input`, `-output` and `-reasoning`, and `x-litellm-key-spend`
|
||||
for your key's running total. With streaming, `usage.cost` arrives in the
|
||||
final chunk instead.
|
||||
- **Timeouts and retries.** 180 seconds per request. Connection errors are
|
||||
retried up to twice; timeouts are never retried, because the provider has
|
||||
already billed the generation.
|
||||
- **Reasoning is on by default** on every chat model we tested, and reasoning
|
||||
tokens count against `max_tokens`. How much varies: the GreenPT GLM models
|
||||
think for a dozen tokens or so and then answer, while the Tinfoil models and
|
||||
DeepSeek on either provider can think for hundreds. A 700-token `max_tokens`
|
||||
on `tinfoil/glm-5.3` or `tinfoil/gpt-oss-120b` can come back as
|
||||
`finish_reason: "length"` with 700 reasoning tokens and no content. To turn it off per request, send
|
||||
`"reasoning_effort": "none"` (or
|
||||
`"chat_template_kwargs": {"enable_thinking": false}`); `"off"` is rejected
|
||||
with a 400, and `tinfoil/gpt-oss-120b` rejects `"none"` too, so thinking
|
||||
can't be disabled on that model. Anthropic-style `thinking: {...}` is
|
||||
dropped without an error.
|
||||
- **Unknown parameters are dropped, not rejected** (`drop_params` is on). If
|
||||
a parameter seems to have no effect, that is probably why.
|
||||
- **Usage.** With `"stream_options": {"include_usage": true}` the last chunk
|
||||
carries `prompt_tokens`, `completion_tokens`,
|
||||
`completion_tokens_details.reasoning_tokens`,
|
||||
`prompt_tokens_details.cached_tokens`, and `cost`. Some providers add an
|
||||
`impact` block (energy in Wms, emissions in µgCO₂e) on non-streaming
|
||||
responses.
|
||||
- **Model metadata is readable.** `GET /model/info` with your key returns
|
||||
every model's `model_info` (pricing, and whatever capability flags are
|
||||
declared there), so a client can pick a model by what it supports.
|
||||
- **Budget exhausted.** When your allowance is spent, the gateway returns
|
||||
HTTP `400` with `error.type: "budget_exceeded"` before any stream starts.
|
||||
Match on the type, not the status. Your allowance resets on your billing
|
||||
date; the dashboard shows when.
|
||||
- **Read your own usage.** Your key can read its own record
|
||||
(`GET /key/info`) and its own budget container
|
||||
(`GET /team/info?team_id=…`, with the `team_id` from `/key/info`): spend,
|
||||
`max_budget`, and `budget_reset_at`. Handy for a client that wants to stop
|
||||
before the gateway does. The scoping is enforced: asking for another team's
|
||||
`team_id` returns 403. `/spend/logs` is admin-only.
|
||||
- **No rate limits beyond the budget, and no request-size cap we could find
|
||||
below 1.5 MB.** There is one shared container; please be a good neighbour.
|
||||
- **No uptime guarantee** (Terms, "Service"). `GET /health/liveliness` and
|
||||
`GET /health/readiness` answer without auth, so a client can check before
|
||||
it falls back to something else.
|
||||
@@ -41,10 +41,10 @@ that **you choose** which axis matters more for a given task.
|
||||
- **Everyday chat and drafting** → a fast model: `greenpt/glm-5.3-flash`.
|
||||
These handle the bulk of what people use chat for, at a fraction of flagship
|
||||
cost.
|
||||
- **Hard reasoning, coding, agentic work** → `tinfoil/glm-5.3` (TEE-protected)
|
||||
or `tinfoil/deepseek-v4-1-flash` for long documents and tool-driven tasks.
|
||||
`greenpt/glm-5.3` is the same model on renewable energy; choose between the
|
||||
two by privacy posture, not capability (see the table).
|
||||
- **Hard reasoning, coding, agentic work** → a flagship: `greenpt/glm-5.3`
|
||||
(renewable) or `tinfoil/glm-5.3` (TEE-protected) — the GLM flagship on the
|
||||
provider of your choice — or `tinfoil/deepseek-v4-1-flash` for long documents
|
||||
and tool-driven tasks.
|
||||
- **Images in, text out (vision)** → `tinfoil/deepseek-v4-1-flash` or the
|
||||
Apertus models — see the ✅ in the table below.
|
||||
- **Cheap and cheerful** → `tinfoil/gpt-oss-120b` or `greenpt/glm-5.3-flash`.
|
||||
@@ -63,10 +63,6 @@ cache-read price. **Vision** means the model accepts images as input
|
||||
available via the chat's image feature (the 🎨 button) using
|
||||
`regolo/qwen-image`.
|
||||
|
||||
**Tools** means the model made a valid streaming tool call through the co-op
|
||||
gateway when we tested it (✅), failed to (⚠️, with the date), or hasn't been
|
||||
tested yet (?). If you test one, add the result by PR.
|
||||
|
||||
The emoji in the model name marks its **value tier** (the same icons used in the chat picker):
|
||||
|
||||
- 🔒 **Private** — Tinfoil: architecturally private (TEE enclaves)
|
||||
@@ -74,16 +70,16 @@ The emoji in the model name marks its **value tier** (the same icons used in the
|
||||
a zero-data-retention commitment). The 🌱 emoji marks renewable-powered
|
||||
providers, not a specific company.
|
||||
|
||||
| Model | Provider | Privacy | Green | Vision | Tools | Context | Pricing (in / out / cached) | Notes |
|
||||
|---|---|---|---|---|---|---|---|---|
|
||||
| `🌱 greenpt/deepseek-v4.1-flash` | GreenPT | policy | ✅ 100% renewable | ✅ | ✅ | 1M | $0.22 / $1.10 / $0.011 | **Default.** DeepSeek's multimodal reasoning model, renewable energy; TEE twin below. Falls back to the Tinfoil twin, then `tinfoil/gpt-oss-120b`, if GreenPT fails. |
|
||||
| `🔒 tinfoil/deepseek-v4-1-flash` | Tinfoil | architectural | undisclosed | ✅ | ✅ | 1M | $0.65 / $1.45 / $0.13 | TEE-protected version of the default; image input, tool calling, long docs/agents. Reasoning on by default. |
|
||||
| `🔒 tinfoil/glm-5.3` | Tinfoil | architectural | undisclosed | — | ✅ | 1M | $1.80 / $5.75 / $0.45 | GLM flagship, TEE-protected; strongest private-tier model for agentic coding and long-horizon tasks. Reasoning on by default; the fastest model we measured. |
|
||||
| `🔒 tinfoil/gpt-oss-120b` | Tinfoil | architectural | undisclosed | — | ? | 131K | $0.15 / $0.60 / — | Lightweight open-weight reasoning; cheapest private tier. |
|
||||
| `🌱 greenpt/glm-5.3-flash` | GreenPT | policy | ✅ 100% renewable | — | ✅ | 1M | $0.11 / $0.44 / $0.022 | Fast GLM model, renewable energy. Reasoning on by default. Tool calls verified 2026-10-26. |
|
||||
| `🌱 greenpt/glm-5.3` | GreenPT | policy | ✅ 100% renewable | — | ✅ | 1M | $1.10 / $4.40 / $0.275 | GLM flagship, renewable energy. A successful tool call comes back with empty `content` and one reasoning token, so read `tool_calls`, not `content`. Tool calls failed 3 of 3 on 2026-10-19 and passed 15 of 15 on 2026-10-26/27; if you see the failure again, note it here. |
|
||||
| `🌱 regolo/apertus-70b` | Regolo | policy + ZDR | ✅ 100% renewable | ✅ | ? | 65K | $0.46 / $2.44 / — | Fully open sovereign model (Apache-2.0), large/capable, image understanding. |
|
||||
| `🌱 regolo/qwen3.5-122b` | Regolo | policy + ZDR | ✅ 100% renewable | ✅ | ? | 262K | $1.16 / $4.87 / — | Large vision+reasoning MoE (Qwen 3.5, 122B-A10B); 262K context. Thinking is forced off in the gateway config. |
|
||||
| Model | Provider | Privacy | Green | Vision | Context | Pricing (in / out / cached) | Notes |
|
||||
|---|---|---|---|---|---|---|---|
|
||||
| `🌱 greenpt/deepseek-v4.1-flash` | GreenPT | policy | ✅ 100% renewable | ✅ | 1M | $0.22 / $1.10 / $0.011 | **Default.** DeepSeek's multimodal reasoning model, renewable energy; TEE twin below. |
|
||||
| `🔒 tinfoil/deepseek-v4-1-flash` | Tinfoil | architectural | undisclosed | ✅ | 1M | $0.65 / $1.45 / $0.13 | TEE-protected version of the default; image input, tool calling, long docs/agents. |
|
||||
| `🔒 tinfoil/glm-5.3` | Tinfoil | architectural | undisclosed | — | 1M | $1.80 / $5.75 / $0.45 | GLM flagship, TEE-protected; strongest private-tier model for agentic coding and long-horizon tasks. |
|
||||
| `🔒 tinfoil/gpt-oss-120b` | Tinfoil | architectural | undisclosed | — | 131K | $0.15 / $0.60 / — | Lightweight open-weight reasoning; cheapest private tier. |
|
||||
| `🌱 greenpt/glm-5.3-flash` | GreenPT | policy | ✅ 100% renewable | — | 1M | $0.11 / $0.44 / $0.022 | Fast GLM model, renewable energy. |
|
||||
| `🌱 greenpt/glm-5.3` | GreenPT | policy | ✅ 100% renewable | — | 1M | $1.10 / $4.40 / $0.275 | GLM flagship — strongest agentic model in the green tier. |
|
||||
| `🌱 regolo/apertus-70b` | Regolo | policy + ZDR | ✅ 100% renewable | ✅ | 65K | $0.46 / $2.44 / — | Fully open sovereign model (Apache-2.0), large/capable, image understanding. |
|
||||
| `🌱 regolo/qwen3.5-122b` | Regolo | policy + ZDR | ✅ 100% renewable | ✅ | 262K | $1.16 / $4.87 / — | Large vision+reasoning MoE (Qwen 3.5, 122B-A10B); 262K context. |
|
||||
|
||||
Vision is verified working through the co-op gateway (base64 image input via
|
||||
`/v1/chat/completions`, standard OpenAI format). In the chat UI, just attach
|
||||
|
||||
@@ -133,3 +133,83 @@ email" problem — it is not; verify the scope first.)
|
||||
revoked). Fix: re-add the `email` scope to the token on Open Collective
|
||||
(Settings → Developers → Personal Tokens). No token rotation, no re-granting of
|
||||
admin — just the scope.
|
||||
|
||||
## Credential inventory (draft, 2026-10-26)
|
||||
|
||||
Every credential the agents and the member-facing apps hold, compiled from the
|
||||
public code and docs, corrected against what the agent holds (PR #7 review). **No secret values appear here, only names and roles.**
|
||||
"Where it lives" and "Who rotates" are filled in where the docs say; blank
|
||||
cells are for the operator to complete. When rotating anything, check every
|
||||
row that names the same credential: several are held by more than one app.
|
||||
|
||||
### Member portal (`portal.inference.coop`)
|
||||
|
||||
| Credential | Purpose | Where it lives | Who rotates | Notes |
|
||||
|---|---|---|---|---|
|
||||
| `CLOUDRON_TOKEN` (portal token) | Create users and groups, invite links | Portal app env | GM | See "The two-token split" above |
|
||||
| `OC_PERSONAL_TOKEN` | Read the member list from Open Collective GraphQL | Portal app env, and the vault | GM, as `info@inference.coop` | Co-op-owned since 2026-10-27: minted by `info@inference.coop`, an Admin of the collective (OC returns member emails only to admins). Scopes `collective`, `individual`, `email`, read-only; the credit-pack order lookup is an anonymous query and needs no token. No expiry set; if one is added, record it here. Rotate on suspected exposure or when the account's admin role changes. The previous personal token should be revoked |
|
||||
| `WEBHOOK_TOKEN` | Secret in the Open Collective webhook URL | Portal app env, and the webhook URL on Open Collective | GM | Rotating means re-creating the webhook on OC as well |
|
||||
| `LITELLM_MASTER_KEY` | Create gateway teams and keys for members | Portal app env (copy of the gateway's) | GM | Three holders: portal, gateway, steward agent. Rotate all together |
|
||||
| `LITELLM_DB_*` | Read spend logs from the gateway database | Portal app env | Cloudron (addon) | Postgres credentials for the gateway's database |
|
||||
| `MATRIX_BOT_TOKEN` | `@inference-bot:inference.coop` admin account: invites, space membership | Portal app env | GM | The account has no password. Paired with the steward agent's `MATRIX_LOCAL_BOT_TOKEN`: same account, separate token on a separate device (verified). Revoke or rotate both together |
|
||||
| `LOOMIO_API_KEY`, `LOOMIO_B3_KEY` | Forum (Loomio) membership sync | Portal app env | GM | |
|
||||
| `LISTMONK_API_USER`, `LISTMONK_API_TOKEN` | Newsletter list sync | Portal app env | GM | |
|
||||
| `OWUI_API_KEY`, `OWUI_JWT_SECRET` | Open WebUI knowledge refresh and SSO | Portal app env | GM | Chat moved to LibreChat in October; confirm whether still needed, and retire if not |
|
||||
| `BROKER_SECRET` | Shared secret between the portal and the member dashboard | Portal and dashboard app env | GM | Held by both; rotate together |
|
||||
| `PORTAL_SECRET` | Session signing | Portal app env | GM | |
|
||||
| `CLOUDRON_MAIL_SMTP_*` | Outbound mail | Cloudron mail addon | Cloudron | Managed by the platform |
|
||||
|
||||
### Gateway (`gateway.inference.coop`)
|
||||
|
||||
| Credential | Purpose | Where it lives | Who rotates | Notes |
|
||||
|---|---|---|---|---|
|
||||
| `LITELLM_MASTER_KEY` | Gateway admin | `/app/data/.master_key`, exported into the litellm process by `start.sh` | GM | Also held by the portal (above), the steward agent (below) and the audio keep-alive job |
|
||||
| Salt key (`SALT_KEY_FILE`) | Encrypts provider keys stored in the gateway database | `/app/data` | GM | Losing it means re-entering every provider key |
|
||||
| `TINFOIL_API_KEY` | Provider account | Gateway app env | Provider account owner | Billing lives at the provider |
|
||||
| `GREENPT_API_KEY` | Provider account | Gateway app env | Provider account owner | Billing lives at the provider |
|
||||
| `REGOLO_API_KEY` | Provider account | Gateway app env | Provider account owner | Billing lives at the provider |
|
||||
| `CLOUDRON_POSTGRESQL_*`, `CLOUDRON_REDIS_*` | Database and cache | Cloudron addons | Cloudron | Managed by the platform |
|
||||
|
||||
### Chat (LibreChat, `chat.inference.coop`)
|
||||
|
||||
| Credential | Purpose | Where it lives | Who rotates | Notes |
|
||||
|---|---|---|---|---|
|
||||
| LibreChat encryption keys and app secrets | App secrets; the per-member key sync runs in the app's own Cloudron scheduler | App data directory | GM | Not held by the agent |
|
||||
|
||||
### Agents (on the agent host)
|
||||
|
||||
Exact file paths are deliberately not listed; "agent `.env`" is the right
|
||||
granularity for the public half. Highest privilege first.
|
||||
|
||||
| Credential | Purpose | Where it lives | Who rotates | Notes |
|
||||
|---|---|---|---|---|
|
||||
| Gateway master key (`GW_MASTER_KEY`) | Admin over every gateway team, key and spend record | Steward agent `.env` | GM | **Third holder**, with the portal and the gateway. The row most in need of coordinated rotation |
|
||||
| `MATRIX_LOCAL_BOT_TOKEN` | The `@inference-bot:inference.coop` automation account: room administration in every co-op room | Steward agent `.env` | GM | Same account as the portal's `MATRIX_BOT_TOKEN`, but a separate token on a separate device (verified). Revoking one does not revoke the other; revoke or rotate both together |
|
||||
| Cloudron agent token | Cloudron API | Steward agent `.env` | GM | See "The two-token split" above |
|
||||
| `REGISTRY_BUILD_TOKEN` (and its username) | Image build and push for the custom apps | Steward agent `.env` | GM | Moves to CI (co-op/support#30); the agent stops holding it then |
|
||||
| Gitea token | Commits, issues, PRs as `inference-bot` | Steward agent `.env` | GM | |
|
||||
| Matrix access token (`@inferencebot:inference.coop`) | Reading and posting in rooms | Agent host, with its E2EE key store | GM | The off switch in [Working with the agent](working-with-the-agent.md) revokes this second |
|
||||
| Steward agent's gateway key (`api:info@inference.coop:inferencebots`) | The steward agent's own inference | Steward agent `.env` | GM | Today: empty model allowlist and no per-key budget; draws on the co-op account team's budget. To be scoped like the per-agent keys below. The off switch revokes this first |
|
||||
| Per-agent gateway keys (`infrastructure-bot`, later agents) | That agent's inference | That agent's `.env` | GM | Model allowlist and monthly budget enforced at the gateway |
|
||||
| `GREENPT_FALLBACK_API_KEY` | Direct-to-provider fallback inference during a gateway outage (live since 2026-10-26) | Steward agent `.env` | Provider account owner | Rotation-coupled to the gateway's `GREENPT_API_KEY`: rotate one without the other and the fallback silently breaks. A dedicated key is the cleaner end state |
|
||||
| `HERMES_CUSTOM_API_GREENPT_AI_API_KEY` | Provider key, still present | Steward agent `.env` | Provider account owner | Confirm whether still needed; retire if not |
|
||||
| Restic repository passphrase | Nightly encrypted backup of the agent's state | Agent host and the co-op vault | GM | |
|
||||
| Infomaniak Swiss Backup S3 keys (agent) | Backup target | Agent host | Infomaniak account owner | |
|
||||
|
||||
### Accounts (not tokens, but the things tokens come from)
|
||||
|
||||
| Account | Purpose | Owner today | Notes |
|
||||
|---|---|---|---|
|
||||
| Cloudron admin (`my.inference.coop`) | Everything on the main server | | |
|
||||
| Hostinger | Hosts the main Cloudron server | Personal account (co-op/support#2) | Moving to Infomaniak |
|
||||
| Infomaniak | Agent VPS, Swiss Backup S3 for Cloudron and the agent | | S3 keys for Cloudron backups rotated after setup (co-op/support#12) |
|
||||
| Open Collective (collective admins) | Dues, webhooks, the personal token above | | |
|
||||
| Provider accounts (Tinfoil, GreenPT, Regolo) | Billing and API keys | | |
|
||||
| Domain registrar and DNS for `inference.coop` | | | |
|
||||
| Gitea, Matrix, Loomio, Listmonk admin accounts | Service administration | | Cloudron SSO where applicable |
|
||||
|
||||
### Keeping this current
|
||||
|
||||
Add a row when a new integration lands, and strike one when a service is
|
||||
retired. A credential held by two apps is the thing most likely to break on
|
||||
rotation; the "Notes" column marks each one.
|
||||
Reference in new issue
Block a user