The gateway handles the prompt in memory before the Tinfoil proxy encrypts it; say that, rather than implying the co-op never sees plaintext.
147 lines
8.0 KiB
Markdown
147 lines
8.0 KiB
Markdown
# What happens with your data?
|
|
|
|
A request passes through up to three layers, and each has a different
|
|
guarantee:
|
|
|
|
1. **Our gateway** routes the request and records what it cost.
|
|
2. **The model provider** runs the model.
|
|
3. **The chat server** saves the conversation, if you are using chat.
|
|
|
|
This page brings the privacy details together. The formal commitments are in
|
|
the [Privacy Policy](../governance/privacy-policy.md).
|
|
|
|
## The short version
|
|
|
|
- **In chat**, your conversation is stored on our server so you can revisit it.
|
|
Temporary chats are the exception: hidden from your history immediately,
|
|
deleted about an hour after creation, and only recoverable from backups until
|
|
those backups age out.
|
|
- **Through the API**, our gateway keeps a usage record and no text: the model,
|
|
token counts, cost, and which key made the request.
|
|
- **At the provider**, protection depends on the model you choose. Tinfoil runs
|
|
inference inside hardware enclaves. GreenPT and Regolo run in the EU under
|
|
GDPR, and Regolo's zero-retention claim is independently certified.
|
|
- We do not train on your data or sell it.
|
|
|
|
## The router: what it sees and what it keeps
|
|
|
|
The gateway is **LiteLLM**, running on the co-op's own server in Switzerland. It
|
|
has to handle your prompt in memory to send it to the provider. It does not
|
|
write prompts or responses to its logs or database. What it keeps is the usage
|
|
record that meters your allowance.
|
|
|
|
We test that instead of asserting it. A request containing a distinctive marker
|
|
is sent through the gateway, then the marker is searched for in the gateway's
|
|
logs, its request records, and its logging configuration. The marker does not
|
|
appear; the records carry usage fields only. The gateway's configuration and
|
|
code are public at [code/litellm](https://git.inference.coop/code/litellm), so
|
|
the logging settings can be read directly.
|
|
|
|
Two limits on that statement, because it is easy to read as more than it is:
|
|
|
|
- The gateway still knows **that a given key made a request**, and therefore
|
|
which member it belongs to. That link is how metering works.
|
|
- **"Not stored by us" is not "not stored."** What the provider keeps is a
|
|
separate question, answered in the next section.
|
|
|
|
## Why this is not a marketplace router
|
|
|
|
A common AI router sits in a chain: your request goes through the router, then
|
|
possibly another router, then a marketplace seller, then the provider. Every hop
|
|
can see the prompt, and some marketplaces retain prompts through ordinary
|
|
platform logging.
|
|
|
|
Our path has one hop we operate:
|
|
|
|
**Your client or our chat → our gateway → the provider.**
|
|
|
|
There is no marketplace, no reseller, and no second router in the path. We
|
|
contract each provider directly. Fewer parties handling the request means fewer
|
|
places it can be copied.
|
|
|
|
That is a reduction in exposure, not a guarantee about any provider. Dragos
|
|
researchers sent roughly two million prompts containing canary tokens through
|
|
commercial AI services and found prompt theft at marketplace resellers, providers
|
|
ignoring privacy settings, and breaches at three hosting providers — including
|
|
large ones
|
|
([their write-up](https://www.dragos.com/blog/ai-supply-chain-security-canary-tokens)).
|
|
Size is not the safeguard. Verifiable commitments, fewer intermediaries, and the
|
|
law are.
|
|
|
|
## What the providers keep
|
|
|
|
| Provider | Runs where | What is committed | Basis |
|
|
|---|---|---|---|
|
|
| **Tinfoil** | United States | Prompts and responses are not retained after the response is returned, apart from temporary prompt caching in encrypted enclave memory. Once the request reaches the enclave, neither Tinfoil nor the co-op can read it in plaintext; our gateway still handles it in memory before encrypting. | Hardware enclaves, verifiable through remote attestation — a technical guarantee, not only a policy. |
|
|
| **GreenPT** | EU data centres in France and Finland | API payloads are processed in memory only and not stored. Error logs and monitoring metrics contain no prompt content. A DPA is available; its model-hosting sub-processors are in Finland. | EU jurisdiction, GDPR, and a contractual DPA. Policy-based: protection depends on GreenPT keeping its commitments. |
|
|
| **Regolo** (Seeweb S.r.l.) | Italy, EU | Prompts and outputs are not stored, logged, or reused for training. General account and connection metadata is still covered by its privacy policy. | EU jurisdiction and GDPR, plus a zero-data-retention certificate issued by an independent auditor on 24 July 2026. Policy-based with an external audit. |
|
|
|
|
Sources: [Tinfoil's terms](https://tinfoil.sh/terms),
|
|
[GreenPT's privacy policy](https://greenpt.com/privacy-policy), and
|
|
[Regolo's zero-data-retention page](https://regolo.ai/zero-data-retention/).
|
|
|
|
**Architectural versus policy-based.** An enclave protects you even if the
|
|
provider is compromised, because the provider cannot read the plaintext. A
|
|
policy protects you only as far as the provider keeps its word and the law is
|
|
enforced. Regolo's independent audit makes its policy claim checkable; GreenPT's
|
|
DPA and EU location give you a legal route if it is broken. Neither is the same
|
|
as cryptography.
|
|
|
|
## What the co-op can still see
|
|
|
|
- **Which member made a request.** Gateway usage records link a request to a key
|
|
and therefore to an account. This is how metering and budgets work.
|
|
- **Chat history.** Saved conversations sit in the chat database and its search
|
|
index, which the co-op can technically read. We commit not to; that is a policy
|
|
commitment, not an architectural guarantee. Temporary chats reduce the window
|
|
but do not remove the operator's technical access while the chat exists.
|
|
- **Operational logs.** Account provisioning, welcome emails, errors and access
|
|
logs contain the email addresses and connection metadata needed to run the
|
|
service. They are kept briefly and are not used to profile members.
|
|
- **Backups.** Cloudron takes whole-app encrypted backups nightly and keeps
|
|
seven days. A temporary chat present when a backup runs can remain recoverable
|
|
until that backup expires.
|
|
|
|
## Retention at a glance
|
|
|
|
| What | How long |
|
|
|---|---|
|
|
| Saved chats and their search entries | Until you delete them |
|
|
| Temporary chats | Hidden immediately; deleted from the database about an hour after creation; may persist in backups until they expire |
|
|
| API prompts and responses | Not stored by the co-op; provider retention is per the table above |
|
|
| Usage records (model, tokens, cost, key) | Kept for billing and account administration |
|
|
| Web-search queries | Stored with the conversation; DuckDuckGo and Brave receive the query when you use search |
|
|
| Operational logs | Kept briefly for running and debugging the service |
|
|
|
|
## What we do not claim
|
|
|
|
- **We do not claim "we never see your messages."** The gateway sits between you
|
|
and the provider by design, so prompts pass through it in memory.
|
|
- **Privacy is not uniform across the catalogue.** Only Tinfoil offers
|
|
architectural privacy. GreenPT and Regolo are chosen for EU jurisdiction,
|
|
contractual commitments and renewable energy, not for enclaves.
|
|
- **A temporary chat does not change who processes the request.** The model you
|
|
choose still receives your messages.
|
|
- **EU law is a constraint, not a force field.** GDPR gives you rights and a
|
|
complaints route; it does not make misuse impossible.
|
|
|
|
## Check it yourself
|
|
|
|
- Read the gateway configuration and patch history at
|
|
[code/litellm](https://git.inference.coop/code/litellm).
|
|
- Check Tinfoil's attestation documents, Regolo's zero-retention certificate,
|
|
and GreenPT's DPA against the links above.
|
|
- Ask us to repeat the marker test on the live gateway. Write to
|
|
[info@inference.coop](mailto:info@inference.coop) and we will run it and report
|
|
what we find.
|
|
|
|
## Formal policy and provider terms
|
|
|
|
- [Privacy Policy](../governance/privacy-policy.md) — the co-op's formal policy,
|
|
including your rights and how to request deletion.
|
|
- [Terms of Service](../governance/terms-of-service.md) — acceptable use and the
|
|
commitments that apply to members.
|
|
- Provider terms: [Tinfoil](https://tinfoil.sh/terms),
|
|
[GreenPT](https://greenpt.com/terms-of-service),
|
|
[Regolo](https://regolo.ai/terms/).
|