1 Commits
Author SHA1 Message Date
bengansky 1827f7c55e Add guide: using the co-op privately from a local chat app
Walk-through for going from zero (no IC account/membership) to using IC-provided models on your own device (desktop or phone) via local apps.
2026-11-19 20:06:58 +00:00
9 changed files with 164 additions and 155 deletions

No files matched your search

+40 -22
View File
@@ -1,25 +1,33 @@
# Woodpecker CI pipeline for co-op/docs — THIN and SECRET-FREE on purpose.
# Woodpecker CI pipeline for co-op/docs — THIN on purpose.
#
# It does NOT deploy. Publishing is host-side and split:
# It does NOT re-implement the build. The repository's publishing layer is the
# single source of truth and this pipeline only wires it up:
#
# /usr/local/sbin/docs-poll host-owned poller (root) — pulls main
# publishing/build.sh runs as the unprivileged `docsbuild` user,
# with no credentials and no access to
# /home/ubuntu
# /usr/local/sbin/docs-publish root-owned publish step — runs no repo code,
# holds the Cloudron token from a root-only file
# publishing/requirements.txt pinned MkDocs + Material toolchain
# publishing/build.sh stage Markdown -> .site-src/, render .site-build/
# publishing/deploy.sh push .site-build/ into the Cloudron Surfer app
#
# CI exists only to prove the repository builds. It runs in a throwaway venv
# inside the pipeline workspace, never the agent image's shared /opt/docs-venv:
# a push to ANY repo must not be able to change a toolchain that another
# pipeline uses. No secret is wired into this pipeline, and none should be
# until the Woodpecker agent supports per-pipeline isolation.
# Any change to how the site is built belongs in publishing/, never here.
#
# Backend note: the co-op's Woodpecker agent is a Cloudron app with no Docker
# socket, so it uses Woodpecker's `local` backend. Each step runs as a shell
# command INSIDE the agent container; the `image:` field selects the SHELL, not
# a container image. That is exactly why this file must stay secret-free: the
# container is shared by every pipeline.
# Run model: MANUAL ONLY for now. The Gitea webhook that fires this on push is
# deliberately NOT wired — the poll watcher (docs-deploy-poll.service on the
# agent host) remains the live auto-deploy path until a manual run has proven
# the pipeline. Once it has, add `push` to the `when` block below and register
# the webhook in Gitea.
#
# Backend note (IMPORTANT — this is why there are no container images here):
# The co-op's Woodpecker agent runs as a Cloudron app, which has no Docker
# socket, so it uses Woodpecker's `local` backend. Under `local` each step is
# run as a shell command INSIDE the agent container, and the step's `image:`
# field selects the SHELL, not a container image. Every tool the steps need is
# therefore baked into the agent image (co-op/woodpecker-agent):
# python3 + the pinned MkDocs toolchain (venv at /opt/docs-venv, mirroring
# publishing/requirements.txt), node + the cloudron CLI, bash, git, tar.
# If the toolchain pin changes, rebuild that image — see its Dockerfile.
#
# Secrets required (Woodpecker repo secret, name `cloudron_token`):
# cloudron_token = the Cloudron API token used by publishing/deploy.sh
# Do not put credentials in this file.
when:
- event: manual
@@ -28,7 +36,17 @@ steps:
- name: build
image: bash
commands:
# Ephemeral toolchain per run — no shared venv, no /opt/docs-venv.
- python3 -m venv .ci-venv
- .ci-venv/bin/pip install -q -r publishing/requirements.txt
- publishing/build.sh .ci-venv
# Reuses the toolchain pre-installed in the agent image; build.sh only
# creates/installs a venv if the one given is missing, which it is not.
- publishing/build.sh /opt/docs-venv
- name: deploy
image: bash
commands:
# deploy.sh drives the Cloudron `cloudron` CLI (a node program) via
# `cloudron exec`; node, the CLI, bash, git and tar are all in the agent
# image. The build it publishes is the shared workspace .site-build/.
- publishing/deploy.sh
environment:
CLOUDRON_TOKEN:
from_secret: cloudron_token
+5 -25
View File
@@ -83,31 +83,13 @@ Our server software keeps standard operational logs (dates, error messages, and
## 5. Search
Our self-hosted **SearXNG** instance at
[search.inference.coop](https://search.inference.coop) handles web search. For
chat searches it forwards queries to **DuckDuckGo** and **Brave**; those
services receive the query from our server and may see it. Search queries and
results are part of the conversation record, so they follow that conversation's
retention: deleted when you delete a saved chat, or about an hour after a
Temporary Chat is created. Operational details are in
[Infrastructure](../operations/infrastructure.md#chat-retention-and-web-search).
To read the pages behind the results, our own page-fetch service fetches the
top result URLs. It runs on co-op infrastructure at fetch.inference.coop. The
service does not log the URLs or the page text and keeps no copy: the extracted
text goes to the model for that answer and is not stored by the fetcher. It
refuses private, loopback, and internal addresses, so it cannot be used to reach
our own network. Your chat messages and account identity stay with us.
Web search is optional. Leave the globe off for sensitive topics. Note that the
search engines in §7 still receive your query when you use it.
Web search runs through our self-hosted search instance. For chat searches we use DuckDuckGo and Brave; those services receive your query from our server, and may see it. This is inherent to how web search works. Search queries and results are part of the conversation record, so they follow that conversation's retention: deleted when you delete a saved chat, or about an hour after a Temporary Chat is created. Operational details are in [Infrastructure](../operations/infrastructure.md#chat-retention-and-web-search).
## 6. What we do NOT do
- We do **not** train models on your data.
- We do **not** sell or rent your data.
- We do **not** share data for advertising or profiling. The service providers
in §5 and §7 process data to run the service.
- We do **not** sell, rent, or share your data.
- We do **not** use your data for advertising or profiling.
## 7. Third parties
@@ -115,8 +97,7 @@ The models we serve are provided by third parties that change over time, each wi
- **Open Collective** — handles billing and membership (their own privacy policy applies).
- **Cloudron** — our self-hosted platform; no third-party cloud provider.
- **Search engines.** DuckDuckGo and Brave receive your search queries (see §5).
Page fetching runs on co-op infrastructure, not at a third party.
- **Search engines** — receive your search queries (see §5).
## 8. Retention and deletion
@@ -129,8 +110,7 @@ The models we serve are provided by third parties that change over time, each wi
- Self-hosted on Cloudron (sandboxed apps, SSL, backups).
- Per-member API keys; access gated on live membership.
- Core infrastructure has no external dependencies. Web search sends queries
to the search engines named in §5; page fetching runs on co-op infrastructure.
- No external dependencies for core infrastructure.
## 10. Your rights
+2 -20
View File
@@ -56,10 +56,8 @@ with the account you created in step 1.
(GreenPT and Regolo, 100% renewable energy — Regolo adds a written
zero-data-retention policy). See the [model list](models.md) for the full
list.
- **Web search.** Click the **globe** next to the message box to base answers
on current web results. Searches use our self-hosted SearXNG instance, and
pages are read by our own fetcher. See [what happens during a
search](#what-happens-when-you-use-web-search).
- **Web search** — click the **globe** next to the message box to ground
answers in current web results (self-hosted search, no external dependency).
- **File uploads** — attach a document with the **paperclip** and ask the
chat about it.
- **Tools** — the **Tools menu** next to the message box lists everything the
@@ -86,22 +84,6 @@ see "Your allowance" below.
> — your full history is here in the new chat, but files you attached in the
> old one are still accessible only there.
### What happens when you use web search
Web search runs in two stages.
1. Our self-hosted **SearXNG** instance at
[search.inference.coop](https://search.inference.coop) sends your query to
**DuckDuckGo** and **Brave**.
2. LibreChat sends the top result URLs to the co-op's **own page fetcher**. It
reads each page and extracts its text. The model receives those passages,
and the answer cites the sources.
Both SearXNG and the page fetcher run on co-op infrastructure. The fetcher does
not log the URLs or the page text and keeps no copy. The
[privacy policy](../governance/privacy-policy.md#5-search) describes this. Web
search is optional. Leave the globe off if you do not want pages fetched.
### Add the chat to your phone
The chat works like an app on your phone — no app store needed. Open
+23
View File
@@ -1,5 +1,28 @@
# Using the co-op privately on your phone or computer
## Index
- [Why use a local app](#why-use-a-local-app)
- [Which app should I use?](#which-app-should-i-use)
- [What you need](#what-you-need)
- [Part 1. Join the co-op (everyone)](#part-1-join-the-co-op-everyone)
- [Step 1. Become a member](#step-1-become-a-member)
- [Step 2. Create your co-op login](#step-2-create-your-co-op-login)
- [Step 3. Make your API key](#step-3-make-your-api-key)
- [Part 2A. Jan on a computer](#part-2a-jan-on-a-computer)
- [4. Install Jan](#jan-step-4-install-jan)
- [5. Connect Jan to the co-op](#jan-step-5-connect-jan-to-the-co-op)
- [6. Turn on tool access](#jan-step-6-turn-on-tool-access)
- [7. Say hello](#jan-step-7-say-hello)
- [Part 2B. Chatbox on a phone or computer](#part-2b-chatbox-on-a-phone-or-computer)
- [4. Install Chatbox](#chatbox-step-4-install-chatbox)
- [5. Connect Chatbox to the co-op](#chatbox-step-5-connect-chatbox-to-the-co-op)
- [6. Turn on images and tools](#chatbox-step-6-turn-on-images-and-tools)
- [7. Say hello](#chatbox-step-7-say-hello)
- [Which model should I use?](#which-model-should-i-use)
- [Keeping an eye on your allowance](#keeping-an-eye-on-your-allowance)
- [If something goes wrong](#if-something-goes-wrong)
## Why use a local app
*A step-by-step walkthrough for members who aren't developers/super technical. Whole process takes about 15 minutes. Drafted for [co-op/support#56](https://git.inference.coop/co-op/support/issues/56).*
-1
View File
@@ -71,7 +71,6 @@ nav:
- Guides:
- Overview: guides/index.md
- Getting started: guides/getting-started.md
- Local chat apps: guides/local-chat-apps.md
- Pricing: guides/pricing.md
- Guest access: guides/guest-access.md
- Models: guides/models.md
+2 -13
View File
@@ -34,22 +34,11 @@ This page is for members who read its output or give it instructions.
portal and a human.
- Credentials, in either direction.
## Content is data, never instructions
Text an agent reads — issue descriptions, pull-request comments, web pages,
fetched documents, and messages from members in a room — is **data**. The agent
may quote, summarise or act on it once the operator asks, but an instruction
embedded in that text is never executed. Only the operator's own direct
instruction authorises a credentialed action: a deploy, a credential change, or
a budget change. This rule exists because the same process that reads an issue
can reach the infrastructure the issue is about.
## What they do without asking
Read, monitor, draft, open issues and PRs, post updates, run the scheduled jobs
listed in [Infrastructure](infrastructure.md), and deploy fixes after flagging
them to the operator. The bot can deploy without waiting for a reply, and the
operator can stop a deploy; anything irreversible waits for confirmation.
listed in [Infrastructure](infrastructure.md), and deploy bug and security
fixes immediately, reporting each one in #inferencebot-updates.
## What always needs a human
+1 -9
View File
@@ -73,10 +73,8 @@ The co-op's software behind all of these is open source — see [The
code](#the-code) below for where each piece lives.
Open Collective (billing and fiscal sponsorship) is the one external service in
the membership flow. See
the flow — see
[opencollective.com/inference-cooperative](https://opencollective.com/inference-cooperative).
Web search uses external services too; see
[Chat retention and web search](#chat-retention-and-web-search).
## Mail
@@ -201,12 +199,6 @@ contain query text when an upstream engine request fails; those log lines carry
no member identity or IP, and are retained only briefly. The reverse proxy does
not log search queries.
LibreChat reads the pages behind the results through the co-op's **own
page-fetch service** (`fetch.inference.coop`), a small app on our Cloudron. It
runs with no access log and keeps no copy of URLs or page text; it refuses
private, loopback, and internal addresses. The
[privacy policy](../governance/privacy-policy.md#5-search) describes it.
## Security posture (plain language)
- **Member isolation.** Each API key is scoped to its owner's team and budget; a
+69 -42
View File
@@ -17,11 +17,11 @@ repository root.
| `logo.svg` | Co-op logo (shared with the website repo) |
| `build.sh` | Stage the Markdown into `.site-src/`, render to `.site-build/` |
| `hooks.py` | MkDocs hooks: directory-link rewriting + content-hashed stylesheet URLs |
| `deploy.sh` | Manual publish of the build into the Surfer app (needs a Cloudron token) |
| `poll-watcher.sh` | Legacy poller; no longer installed — superseded by the host-owned poller |
| `docs-deploy-poll.service` | systemd unit for the poller (runs the host-owned script) |
| `webhook-receiver.py` | Webhook receiver; not installed (see below) |
| `webhook-receiver.service` | systemd unit for the receiver; not installed |
| `deploy.sh` | Push the build into the Surfer app container |
| `poll-watcher.sh` | Polls Gitea main every 60s; deploys on change (agent host) |
| `docs-deploy-poll.service` | systemd unit for the poller |
| `webhook-receiver.py` | Gitea webhook → pull → build → deploy (agent host) |
| `webhook-receiver.service` | systemd unit for the receiver |
| `requirements.txt` | Pinned Python build toolchain |
Nothing external is loaded by the site: no CDN, no third-party font service.
@@ -71,52 +71,79 @@ Override the destination with `SURFER_APP` (the app's Cloudron location) and
## Automatic rebuilds (push to main)
`docs-deploy-poll.service` polls Gitea every 60 seconds and deploys whenever
`main` moves, so a push appears on the site within about a minute with no human
action. Edit, commit, push — that is the whole deployment story.
Two mechanisms keep the site in step with `main`; both run on the agent host
and deploy into the docs Cloudron app above.
**The credential-bearing steps are host-owned, not repository code.** The
poller runs a root-owned script (`/usr/local/sbin/docs-poll`) that does two
separate jobs:
**Polling (running now).** `docs-deploy-poll.service` polls Gitea every 60
seconds and deploys whenever `main` moves. A push appears on the site within
about a minute, with no human action. The Gitea webhook alone was not enough
because the agent host's cloud firewall only admits inbound SSH — a webhook
POST is silently dropped — while outbound access to Gitea works.
1. **Build as `docsbuild`.** The repository's build logic (`build.sh` → MkDocs
→ `hooks.py`) runs as an unprivileged system user with a scrubbed
environment, from its own clean clone at `/var/lib/docs-publisher/repo`
with its own venv. That user cannot read `/home/ubuntu` (mode 750), so repo
code cannot see any co-op credential — the Cloudron token, the gateway
master key, the agent's env file.
2. **Publish with a root-owned script.** `/usr/local/sbin/docs-publish` runs
no repository code. It takes only the built directory, writes `meta.json`,
and pushes it into the Surfer app using `CLOUDRON_TOKEN` from
`/etc/docs-deploy.env` (root-only, mode 0600).
The service builds from a **dedicated clean clone** at
`/home/ubuntu/.docs-publisher`, not from anyone's working copy, so uncommitted
edits in a working clone can never block a deploy. Edit, commit, push — that
is the whole deployment story.
A commit that lands on `main` therefore cannot execute anything with the
Cloudron token. It can only influence the bytes of the site.
**Webhook (wired, waiting on one port).** A Gitea webhook is registered on
`co-op/docs` (push events, branch `main`) pointing at the receiver below. The
moment the Infomaniak panel allows inbound TCP to the agent host — port 8787
from the Cloudron box 179.237.86.116 is enough — start the receiver and pushes
deploy instantly instead of within a minute:
**The webhook receiver is installed but not running.** `docs-deploy-webhook`
is inactive and nothing listens on 8787. It predates this split, loads the
agent's env file, and runs repo scripts with the Cloudron token; do not enable
it without porting it to the same host-owned build/publish split. The Gitea
webhook registration is harmless while the receiver is down.
```sh
sudo systemctl enable --now docs-deploy-webhook
tail -f /home/ubuntu/docs-deploy.log # what both mechanisms are doing
```
The receiver authenticates webhooks with an HMAC-SHA256 shared secret
(`DEPLOY_WEBHOOK_SECRET` in the agent host's env file) and only accepts pushes
to `main` of `co-op/docs` from the Cloudron box. Woodpecker CI (Cloudron
catalog) could do the same job inside Cloudron, at the cost of running another
app; the agent-host mechanisms were chosen because they add no new Cloudron app.
Each deploy writes `meta.json` into the site with the commit it was built
from, so a deploy can be verified from the served site itself:
https://docs.inference.coop/meta.json
If a deploy breaks, roll back with the saved unit
(`/root/docs-deploy-poll.service.before-split`) and the previous build in
`/var/lib/docs-publisher/repo/.site-build`; the poller retries every 60 seconds
and sends one Matrix alert after 3 consecutive failures.
## One build environment (2026-11-12)
## Build environment
The deploy used to have TWO build environments — this repo's `.venv-docs` and
a second private venv in the poller's checkout at
`/home/ubuntu/.docs-publisher/.venv-docs`. They diverged twice, each time
silently delaying a deploy (the brand.css cache-busting confusion, then a
missing `mkdocs-redirects` that failed every build for ~15 minutes while the
poller retried without telling anyone). Consolidated:
`publishing/requirements.txt` pins every package the build needs (`mkdocs`,
`mkdocs-material`, `mkdocs-redirects`). `build.sh` verifies the build venv
against it before every build and installs anything missing or off-version
(fast local no-op when current); it also fails loudly if `mkdocs.yml`
configures a plugin that `requirements.txt` does not pin.
- **ONE venv.** Every trigger — the poller, the (dormant) webhook receiver, a
manual `publishing/build.sh`, CI — builds with the canonical checkout's
venv: `~/repos/docs/.venv-docs`. The poller's private venv was deleted.
- **The dedicated checkout stays, by design.** The poller still builds from
`/home/ubuntu/.docs-publisher`, its own clean clone: uncommitted edits in a
working clone must never block a deploy or leak into the published site.
- **Drift check.** Before every build the poller compares `publishing/` and
`mkdocs.yml` between its checkout and the canonical one; any mismatch fails
the deploy loudly (with a diff and the fix — `git -C ~/repos/docs pull
--ff-only`), and the failure counts toward the alert below. A canonical
checkout that lags behind main trips this too, on purpose: a deploy of
logic the canonical source cannot reproduce is the failure mode this exists
to prevent. A missing canonical checkout is fatal as well.
- **Plugin gate + self-heal.** `publishing/requirements.txt` pins every
package the build needs (`mkdocs`, `mkdocs-material`, `mkdocs-redirects`).
`build.sh` verifies the venv against it before every build and installs
anything missing or off-version (fast local no-op when current); it also
fails loudly if `mkdocs.yml` configures a plugin that `requirements.txt`
does not pin. The poller runs this as its pre-deploy check via
`build.sh --check-only`.
- **Failure alerting.** After 3 consecutive deploy failures the poller writes
a marker (`~/.docs-deploy.alert`) and sends ONE message to the InferenceBot
HQ Matrix room naming the failing commit — never more than one message per
hour (`~/.docs-deploy.last-alert`). The counter (`~/.docs-deploy.failcount`)
resets on the next successful deploy. A deploy that succeeds silently
resets everything; nothing announces routine successes.
The deployed poller keeps one build venv at
`/var/lib/docs-publisher/repo/.venv-docs`, owned by `docsbuild`. A manual
`publishing/build.sh` uses `.venv-docs` in whatever checkout you run it from;
that is fine for local previews and does not affect deploys.
Files: `~/.docs-deploy.state` (last deployed commit), `~/.docs-deploy.lock`
(deploy lock), `~/.docs-deploy.failcount`, `~/.docs-deploy.alert`,
`~/.docs-deploy.last-alert`. Manual deploys should respect the lock:
`flock -n ~/.docs-deploy.lock bash publishing/build.sh && flock -n
~/.docs-deploy.lock bash publishing/deploy.sh`.
+22 -23
View File
@@ -1,42 +1,41 @@
# systemd unit for the docs auto-deploy poller (agent host).
#
# docs-deploy-poll.service ALWAYS ON. Polls Gitea every 60s; deploys on
# any change to main.
# any change to main. Works with the agent
# host's current firewall (outbound only).
#
# docs-deploy-webhook.service INSTALLED, INACTIVE. It predates the
# build/publish split, loads the agent's env
# file, and runs repo scripts with the Cloudron
# token. Do not enable it until it is ported to
# the same split.
# docs-deploy-webhook.service INSTALLED, INACTIVE UNTIL THE PORT IS OPEN.
# Instant push -> deploy once the Infomaniak
# panel allows inbound TCP to the agent host
# (the Gitea webhook is already registered).
#
# Install:
# sudo install -o root -g root -m 0644 publishing/docs-deploy-poll.service \
# /etc/systemd/system/docs-deploy-poll.service
# sudo cp publishing/docs-deploy-poll.service /etc/systemd/system/
# sudo systemctl daemon-reload
# sudo systemctl enable --now docs-deploy-poll
#
# The unit runs a HOST-OWNED script, /usr/local/sbin/docs-poll, which is not
# copied from this repository. That script:
# * builds as the unprivileged `docsbuild` user from its own checkout at
# /var/lib/docs-publisher/repo (a scrubbed environment, no read access to
# /home/ubuntu, so repo code cannot see any co-op credential);
# * publishes with /usr/local/sbin/docs-publish, a root-owned script that
# runs no repository code and reads the Cloudron token from
# /etc/docs-deploy.env (root-only, mode 0600).
# poll-watcher.sh itself needs NO manual copy: the service runs the copy in
# the dedicated checkout (/home/ubuntu/.docs-publisher), which git pull keeps
# at main on every deploy, and the watcher re-execs itself when a deploy
# changed it.
#
# Repo code (build.sh, hooks.py) therefore never executes with a credential.
# See publishing/README.md.
# ONE BUILD ENVIRONMENT: the poller builds from its own dedicated checkout
# (uncommitted edits in a working clone must never block or leak into a
# deploy) but with the ONE canonical venv, DOCS_CANONICAL_DIR/.venv-docs, and
# drift-checks its publishing/ tree against the canonical checkout before
# every build. See publishing/README.md.
[Unit]
Description=Inference docs: poll Gitea main, build unprivileged, publish without repo code
Description=Inference docs: poll Gitea main and deploy to Surfer
After=network-online.target
Wants=network-online.target
[Service]
User=root
Environment=DOCS_REPO_DIR=/var/lib/docs-publisher/repo
Environment=DOCS_DEPLOY_LOG=/var/log/docs-deploy.log
ExecStart=/usr/local/sbin/docs-poll
User=ubuntu
WorkingDirectory=/home/ubuntu/.docs-publisher
Environment=DOCS_REPO_DIR=/home/ubuntu/.docs-publisher
Environment=DOCS_CANONICAL_DIR=/home/ubuntu/repos/docs
ExecStart=/bin/bash /home/ubuntu/.docs-publisher/publishing/poll-watcher.sh
Restart=on-failure
RestartSec=10