From c2f9c97fce15f54530bd083d66a6300a43fadedf Mon Sep 17 00:00:00 2001 From: inference-bot Date: Fri, 4 Sep 2026 16:04:00 -0600 Subject: [PATCH] Initial docs: model and stack overview --- README.md | 69 +++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 69 insertions(+) create mode 100644 README.md diff --git a/README.md b/README.md new file mode 100644 index 0000000..452320d --- /dev/null +++ b/README.md @@ -0,0 +1,69 @@ +# Inference Cooperative + +**Private AI, governed together.** + +The Inference Cooperative is a member-governed project providing private AI inference. We are fiscally sponsored by [Metagov](https://metagov.org), a nonprofit, and funded through our [Open Collective](https://opencollective.com/inference-cooperative). + +- **Website:** https://inference.coop +- **Chat:** https://chat.inference.coop +- **Open Collective:** https://opencollective.com/inference-cooperative +- **Contact:** info@inference.coop + +--- + +## The model + +The Inference Cooperative is a **member-governed** AI inference utility. Members contribute on a sliding scale (currently $10–20/month) and, in return, get access to private AI inference through a shared, cooperatively governed infrastructure. + +Because we are legally under a nonprofit (Metagov is our fiscal sponsor), we are **member-governed** rather than member-owned: members direct the project through governance, while the nonprofit holds the legal and financial structure. + +### Membership + +- **Sliding scale:** $10–20/month +- **Governance:** members participate in decisions about models, privacy, and direction +- **Access:** private AI inference through the cooperative's gateway + +--- + +## The stack + +The Inference Cooperative runs a self-hosted stack on [Cloudron](https://cloudron.io), with all inference routed through a single gateway. + +``` +Members → LibreChat (chat.inference.coop) + ↓ + LiteLLM gateway (gateway.inference.coop) + ↓ + Inference backend (Ollama Cloud today; TEE-protected Tinfoil planned) +``` + +### Components + +| Component | Purpose | URL | +|-----------|---------|-----| +| **LibreChat** | Member-facing chat interface | chat.inference.coop | +| **LiteLLM** | AI gateway — keys, metering, model routing | gateway.inference.coop | +| **Gitea** | Git hosting (code + docs) | git.inference.coop | +| **Loomio** | Member governance / decision-making | (forum) | +| **Open Collective** | Membership billing + fiscal sponsorship | opencollective.com/inference-cooperative | + +### Models + +The gateway currently exposes two models: + +- **DeepSeek V4 Flash** — default model, best for agentic tasks +- **GPT-OSS 120B** — lightweight fallback + +### Privacy + +Privacy is a core value. The current backend (Ollama Cloud) is a policy-based privacy model. The roadmap is to move to **TEE-protected inference** (via Tinfoil), which provides architectural — not just policy — privacy guarantees. + +--- + +## Governance + +The Inference Cooperative is fiscally sponsored by [Metagov](https://metagov.org). Members govern the project through [Loomio](https://www.loomio.com) and the Open Collective. + +--- + +*This documentation lives in a public git repository. Contributions welcome.*