From 3fb725d1a345f0f5b0262fe021ffbbcb625b7c67 Mon Sep 17 00:00:00 2001 From: inference-bot Date: Wed, 9 Sep 2026 13:28:29 -0600 Subject: [PATCH] Document Tinfoil TEE switch + proxy sidecar --- README.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index d8d6e73..99df3de 100644 --- a/README.md +++ b/README.md @@ -53,7 +53,9 @@ The gateway currently exposes two models: ### Privacy -Privacy is a core value. The current LLM backend (Ollama Cloud) is a policy-based privacy model. The roadmap is to move to [TEE-protected inference (via Tinfoil)](https://tinfoil.sh/inference), which provides architectural—not just policy—privacy guarantees. +Privacy is a core value. Inference now runs through [Tinfoil](https://tinfoil.sh/inference), which provides **architectural** privacy: models run inside hardware enclaves (TEEs), and request/response bodies are encrypted end-to-end with the Encrypted HTTP Body Protocol (EHBP), so even Tinfoil's own infrastructure cannot read them. This is verifiable via remote attestation — not just a policy promise. + +Because Tinfoil requires EHBP-encrypted request bodies (it rejects plaintext with `426 EHBP_REQUIRED`), the LiteLLM gateway routes through a local [Tinfoil proxy](https://github.com/tinfoilsh/tinfoil-proxy) sidecar, which verifies the enclave attestation and handles the encryption. LiteLLM talks plaintext OpenAI to the proxy; the proxy encrypts and forwards to the enclave. ## Governance