pip show exits non-zero for an absent package; under set -euo pipefail the command substitution aborted build.sh before the self-heal could run — the gate then failed the deploy with no message at all. Tolerate the missing package (that is the case being healed) and report it.
Publishing the documentation site
The Markdown in this repository is the single source of truth. This directory is the publishing layer that turns it into a member-readable static site and puts it on the co-op's docs Surfer app at https://docs.inference.coop/. Documents are never edited here — a change to a document is a normal commit or pull request against the Markdown at the repository root.
What is built
| File | Purpose |
|---|---|
../mkdocs.yml |
Site configuration (MkDocs + Material for MkDocs) |
brand.css |
Co-op palette + self-hosted Figtree webfont (SIL OFL 1.1) |
fonts/ |
Figtree woff2 files and the OFL licence text |
logo.svg |
Co-op logo (shared with the website repo) |
build.sh |
Stage the Markdown into .site-src/, render to .site-build/ |
hooks.py |
MkDocs hooks: directory-link rewriting + content-hashed stylesheet URLs |
deploy.sh |
Push the build into the Surfer app container |
poll-watcher.sh |
Polls Gitea main every 60s; deploys on change (agent host) |
docs-deploy-poll.service |
systemd unit for the poller |
webhook-receiver.py |
Gitea webhook → pull → build → deploy (agent host) |
webhook-receiver.service |
systemd unit for the receiver |
requirements.txt |
Pinned Python build toolchain |
Nothing external is loaded by the site: no CDN, no third-party font service. Fonts are served from this repository's own output.
Cache-proof stylesheets
brand.css and typography.css are listed under extra_css, so at a fixed URL
they would be vulnerable to a stale browser cache after a deploy — a warm cache
keeps serving the old file because the URL never changes. hooks.py's
on_post_build fixes this at build time: it copies each stylesheet to
<name>.<sha256-8>.css in the output and rewrites the <link> in every built
page to the hashed name. A change to a stylesheet's content therefore always
produces a new URL. The authored sources keep their names and locations; the
hashed copies exist only in the build output. The hashed copy sits in the same
directory as the original, so the @font-face url("fonts/…") references keep
resolving. The deploy is unchanged.
Build
publishing/build.sh # creates .venv-docs on first run
Output lands in .site-build/ (gitignored). mkdocs build --strict is what
CI-style runs should use, so broken links fail the build.
Deploy
CLOUDRON_TOKEN=... publishing/deploy.sh
Writes the build into the docs Surfer app (Cloudron app docs.inference.coop,
io.cloudron.surfer) at:
/app/data/public/— served at https://docs.inference.coop/
That is now the one canonical location. The docs used to be a pair of
directories inside the marketing Surfer app (inference.coop) as well, at
/app/data/public-docs/ and /app/data/public/docs/; both were deleted when
this app took over, so https://inference.coop/docs/ no longer serves the docs
(nothing on the marketing site linked to it).
Override the destination with SURFER_APP (the app's Cloudron location) and
DOCS_DEST (path inside the container) if you ever need to.
Automatic rebuilds (push to main)
Two mechanisms keep the site in step with main; both run on the agent host
and deploy into the docs Cloudron app above.
Polling (running now). docs-deploy-poll.service polls Gitea every 60
seconds and deploys whenever main moves. A push appears on the site within
about a minute, with no human action. The Gitea webhook alone was not enough
because the agent host's cloud firewall only admits inbound SSH — a webhook
POST is silently dropped — while outbound access to Gitea works.
The service builds from a dedicated clean clone at
/home/ubuntu/.docs-publisher, not from anyone's working copy, so uncommitted
edits in a working clone can never block a deploy. Edit, commit, push — that
is the whole deployment story.
Webhook (wired, waiting on one port). A Gitea webhook is registered on
co-op/docs (push events, branch main) pointing at the receiver below. The
moment the Infomaniak panel allows inbound TCP to the agent host — port 8787
from the Cloudron box 179.237.86.116 is enough — start the receiver and pushes
deploy instantly instead of within a minute:
sudo systemctl enable --now docs-deploy-webhook
tail -f /home/ubuntu/docs-deploy.log # what both mechanisms are doing
The receiver authenticates webhooks with an HMAC-SHA256 shared secret
(DEPLOY_WEBHOOK_SECRET in the agent host's env file) and only accepts pushes
to main of co-op/docs from the Cloudron box. Woodpecker CI (Cloudron
catalog) could do the same job inside Cloudron, at the cost of running another
app; the agent-host mechanisms were chosen because they add no new Cloudron app.
Each deploy writes meta.json into the site with the commit it was built
from, so a deploy can be verified from the served site itself:
https://docs.inference.coop/meta.json
One build environment (2026-11-12)
The deploy used to have TWO build environments — this repo's .venv-docs and
a second private venv in the poller's checkout at
/home/ubuntu/.docs-publisher/.venv-docs. They diverged twice, each time
silently delaying a deploy (the brand.css cache-busting confusion, then a
missing mkdocs-redirects that failed every build for ~15 minutes while the
poller retried without telling anyone). Consolidated:
- ONE venv. Every trigger — the poller, the (dormant) webhook receiver, a
manual
publishing/build.sh, CI — builds with the canonical checkout's venv:~/repos/docs/.venv-docs. The poller's private venv was deleted. - The dedicated checkout stays, by design. The poller still builds from
/home/ubuntu/.docs-publisher, its own clean clone: uncommitted edits in a working clone must never block a deploy or leak into the published site. - Drift check. Before every build the poller compares
publishing/andmkdocs.ymlbetween its checkout and the canonical one; any mismatch fails the deploy loudly (with a diff and the fix —git -C ~/repos/docs pull --ff-only), and the failure counts toward the alert below. A canonical checkout that lags behind main trips this too, on purpose: a deploy of logic the canonical source cannot reproduce is the failure mode this exists to prevent. A missing canonical checkout is fatal as well. - Plugin gate + self-heal.
publishing/requirements.txtpins every package the build needs (mkdocs,mkdocs-material,mkdocs-redirects).build.shverifies the venv against it before every build and installs anything missing or off-version (fast local no-op when current); it also fails loudly ifmkdocs.ymlconfigures a plugin thatrequirements.txtdoes not pin. The poller runs this as its pre-deploy check viabuild.sh --check-only. - Failure alerting. After 3 consecutive deploy failures the poller writes
a marker (
~/.docs-deploy.alert) and sends ONE message to the InferenceBot HQ Matrix room naming the failing commit — never more than one message per hour (~/.docs-deploy.last-alert). The counter (~/.docs-deploy.failcount) resets on the next successful deploy. A deploy that succeeds silently resets everything; nothing announces routine successes.
Files: ~/.docs-deploy.state (last deployed commit), ~/.docs-deploy.lock
(deploy lock), ~/.docs-deploy.failcount, ~/.docs-deploy.alert,
~/.docs-deploy.last-alert. Manual deploys should respect the lock:
flock -n ~/.docs-deploy.lock bash publishing/build.sh && flock -n ~/.docs-deploy.lock bash publishing/deploy.sh.