From 2fdda5a201f18c38fb393cd6a82b8b9f26df4bc5 Mon Sep 17 00:00:00 2001 From: Inferencebot Date: Thu, 29 Oct 2026 18:21:01 +0000 Subject: [PATCH] privacy: state what the API path actually stores (and add it where members read) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The policy described chat history and enclave models carefully but never addressed the API path, and §2 read as though all content is retained — which is wrong for members using their own keys. Verified before writing, rather than restating policy language: a request with a distinctive marker was sent through the gateway and searched for afterwards. It appears nowhere in the gateway's logs, `litellm_settings` enables no content logging, and the request records carry usage fields only (model, tokens, cost, key). - privacy-policy.md §2: content retention now depends on the surface, with a pointer. - privacy-policy.md §4: new subsection on API requests, including the two limits that stop it being read as more than it says — we can still link a request to a key and therefore a member, and "not stored by us" says nothing about the provider. - api-access.md: a "what we store" section, since that is the page an API user reads before worrying about it. - models.md: the privacy-limitations section now distinguishes chat from API. --- api-access.md | 18 ++++++++++++++++++ models.md | 7 +++++-- privacy-policy.md | 24 +++++++++++++++++++++++- 3 files changed, 46 insertions(+), 3 deletions(-) diff --git a/api-access.md b/api-access.md index acfcb7f..b01c122 100644 --- a/api-access.md +++ b/api-access.md @@ -14,6 +14,24 @@ client works. API usage draws from the **same monthly allowance as chat** — there's no separate quota. +## What the co-op stores about your requests + +Usage records, and nothing else: the model, token counts, cost, and which key made +the request — that is how your monthly metering works. **Your prompts and the +responses are not stored on our side.** The API path keeps no conversation, so there +is no transcript to delete later. + +Two things worth being precise about: + +- The **chat interface** is different — it saves your conversations so you can revisit + them. If you want the API's behaviour there instead, use a **temporary chat**. +- The **provider** is a separate question. Whether a model's operator retains anything + is set by that provider, not by us, and it varies by model — the + [model list](models.md) distinguishes policy-based zero retention from + enclave-based protection. + +Full detail, including what we can and cannot see: [privacy policy](privacy-policy.md). + ## Get a key Log in to the [Member Dashboard](https://dashboard.inference.coop) and create an diff --git a/models.md b/models.md index 6c4dbb8..2f0cca6 100644 --- a/models.md +++ b/models.md @@ -204,5 +204,8 @@ for their value (renewable energy, public models) but do not run in enclaves — prompts and responses pass through them in the ordinary way. Your **chat history** is also stored on our server so you can revisit it, and that stored history is not encrypted in a way that prevents us from technically reading it — -we commit not to. The full distinction — what's architecturally private versus -what's a policy commitment — is in the [Privacy Policy](privacy-policy.md). +we commit not to. Requests made through the **API** with your own key are a +different matter: those are **not stored on our side at all**, beyond the usage +record needed for metering ([API access](api-access.md)). The full distinction — +what's architecturally private versus what's a policy commitment — is in the +[Privacy Policy](privacy-policy.md). diff --git a/privacy-policy.md b/privacy-policy.md index 742c9e3..94e4f75 100644 --- a/privacy-policy.md +++ b/privacy-policy.md @@ -23,7 +23,7 @@ The Inference Cooperative is a member-governed project. We are **not** an ad-sup - **Account data:** your email and name (from Open Collective). - **Membership data:** your contribution status and role. - **Usage data:** token counts, model used, and timestamps (for metering your monthly budget). -- **Content:** your prompts and the AI's responses (stored in your chat history). +- **Content:** your prompts and the AI's responses. Whether we keep them depends on how you use the service: in the **chat interface** they are saved as your chat history, while through the **API** they are not stored at all. See §4. - **Billing data:** handled by Open Collective — we do not see full payment details. ## 3. Privacy depends on the model you choose @@ -45,6 +45,28 @@ We commit not to read it. But this is a **policy commitment**, not an architectu Note the boundary precisely: enclave models protect your conversation **while it is being processed** — the model provider and we cannot read the exchange at inference time. Stored history is a separate leg: once a conversation is saved, it sits on our server in a form we can technically read. So for sensitive material, the strongest combination available today is a **Temporary Chat with an enclave model** — protected in transit and at inference, and never stored afterwards. +### If you use the API, nothing you send is stored + +This is the part that surprises people, so it is worth stating plainly. If you call +the co-op's API with your own key and your own client, there is no conversation on +our side to save. Our gateway records **that** a request happened and what it cost: +the model, the token counts, the cost, and which key made the request. It does not +record **what you asked or what came back**. + +We measured this rather than asserting it. A request with a distinctive marker was +sent through the gateway and then searched for: it appears nowhere in the gateway's +logs, the gateway's configuration enables no content logging, and the request records +themselves carry usage fields only. So the API path keeps no text. + +Two boundaries on that statement, because it would be easy to read it as more than +it is: + +- **We can still tell that a given key made a request**, and therefore which member + it belongs to. That link is how metering works — it is not something we hide. +- **"Not stored by us" is not the same as "not stored."** The provider's handling is + a separate question from ours, and it depends on the model you choose — see §3. Not + being stored on our side says nothing about the far end of the request. + ### Temporary chats (recommended for anything sensitive) The chat has a **Temporary Chat** mode designed exactly for this: