Files
docs/publishing/README.md
T
inference-bot 20d384be87 publishing: one canonical build venv; drift check; deploy-failure alerting
The poller's private venv (.docs-publisher/.venv-docs) diverged from the
canonical one twice, each time silently delaying a deploy. Consolidate:

- poll-watcher.sh builds with the ONE canonical venv
  (~/repos/docs/.venv-docs) while keeping its dedicated checkout (uncommitted
  edits in a working clone must never block or leak into a deploy).
- New drift check: publishing/ + mkdocs.yml are compared with the canonical
  checkout before every build; a mismatch fails the deploy loudly instead of
  shipping logic the canonical source cannot reproduce.
- Pre-deploy gate: build.sh --check-only verifies the pinned toolchain and
  every configured plugin (self-heals missing pins, fails loudly naming an
  unpinned plugin).
- Alerting: 3 consecutive deploy failures -> marker file + ONE Matrix message
  to the InferenceBot HQ room (hourly cap); counter resets on next success.
- The watcher re-execs itself when a deploy changed poll-watcher.sh, so the
  running loop always matches the deployed one.
- webhook receiver (dormant) repointed at the canonical venv too; unit files
  and README document the layout.
2026-11-12 03:23:33 +00:00

150 lines
7.5 KiB
Markdown

# Publishing the documentation site
The Markdown in this repository is the single source of truth. This directory
is the publishing layer that turns it into a member-readable static site and
puts it on the co-op's docs Surfer app at https://docs.inference.coop/.
Documents are never edited here — a change
to a document is a normal commit or pull request against the Markdown at the
repository root.
## What is built
| File | Purpose |
|---|---|
| `../mkdocs.yml` | Site configuration (MkDocs + Material for MkDocs) |
| `brand.css` | Co-op palette + self-hosted Figtree webfont (SIL OFL 1.1) |
| `fonts/` | Figtree woff2 files and the OFL licence text |
| `logo.svg` | Co-op logo (shared with the website repo) |
| `build.sh` | Stage the Markdown into `.site-src/`, render to `.site-build/` |
| `hooks.py` | MkDocs hooks: directory-link rewriting + content-hashed stylesheet URLs |
| `deploy.sh` | Push the build into the Surfer app container |
| `poll-watcher.sh` | Polls Gitea main every 60s; deploys on change (agent host) |
| `docs-deploy-poll.service` | systemd unit for the poller |
| `webhook-receiver.py` | Gitea webhook → pull → build → deploy (agent host) |
| `webhook-receiver.service` | systemd unit for the receiver |
| `requirements.txt` | Pinned Python build toolchain |
Nothing external is loaded by the site: no CDN, no third-party font service.
Fonts are served from this repository's own output.
### Cache-proof stylesheets
`brand.css` and `typography.css` are listed under `extra_css`, so at a fixed URL
they would be vulnerable to a stale browser cache after a deploy — a warm cache
keeps serving the old file because the URL never changes. `hooks.py`'s
`on_post_build` fixes this at build time: it copies each stylesheet to
`<name>.<sha256-8>.css` in the output and rewrites the `<link>` in every built
page to the hashed name. A change to a stylesheet's content therefore always
produces a new URL. The authored sources keep their names and locations; the
hashed copies exist only in the build output. The hashed copy sits in the same
directory as the original, so the `@font-face` `url("fonts/…")` references keep
resolving. The deploy is unchanged.
## Build
```sh
publishing/build.sh # creates .venv-docs on first run
```
Output lands in `.site-build/` (gitignored). `mkdocs build --strict` is what
CI-style runs should use, so broken links fail the build.
## Deploy
```sh
CLOUDRON_TOKEN=... publishing/deploy.sh
```
Writes the build into the **docs Surfer app** (Cloudron app `docs.inference.coop`,
`io.cloudron.surfer`) at:
- `/app/data/public/` — served at **https://docs.inference.coop/**
That is now the one canonical location. The docs used to be a pair of
directories inside the *marketing* Surfer app (`inference.coop`) as well, at
`/app/data/public-docs/` and `/app/data/public/docs/`; both were deleted when
this app took over, so `https://inference.coop/docs/` no longer serves the docs
(nothing on the marketing site linked to it).
Override the destination with `SURFER_APP` (the app's Cloudron location) and
`DOCS_DEST` (path inside the container) if you ever need to.
## Automatic rebuilds (push to main)
Two mechanisms keep the site in step with `main`; both run on the agent host
and deploy into the docs Cloudron app above.
**Polling (running now).** `docs-deploy-poll.service` polls Gitea every 60
seconds and deploys whenever `main` moves. A push appears on the site within
about a minute, with no human action. The Gitea webhook alone was not enough
because the agent host's cloud firewall only admits inbound SSH — a webhook
POST is silently dropped — while outbound access to Gitea works.
The service builds from a **dedicated clean clone** at
`/home/ubuntu/.docs-publisher`, not from anyone's working copy, so uncommitted
edits in a working clone can never block a deploy. Edit, commit, push — that
is the whole deployment story.
**Webhook (wired, waiting on one port).** A Gitea webhook is registered on
`co-op/docs` (push events, branch `main`) pointing at the receiver below. The
moment the Infomaniak panel allows inbound TCP to the agent host — port 8787
from the Cloudron box 179.237.86.116 is enough — start the receiver and pushes
deploy instantly instead of within a minute:
```sh
sudo systemctl enable --now docs-deploy-webhook
tail -f /home/ubuntu/docs-deploy.log # what both mechanisms are doing
```
The receiver authenticates webhooks with an HMAC-SHA256 shared secret
(`DEPLOY_WEBHOOK_SECRET` in the agent host's env file) and only accepts pushes
to `main` of `co-op/docs` from the Cloudron box. Woodpecker CI (Cloudron
catalog) could do the same job inside Cloudron, at the cost of running another
app; the agent-host mechanisms were chosen because they add no new Cloudron app.
Each deploy writes `meta.json` into the site with the commit it was built
from, so a deploy can be verified from the served site itself:
https://docs.inference.coop/meta.json
## One build environment (2026-11-12)
The deploy used to have TWO build environments — this repo's `.venv-docs` and
a second private venv in the poller's checkout at
`/home/ubuntu/.docs-publisher/.venv-docs`. They diverged twice, each time
silently delaying a deploy (the brand.css cache-busting confusion, then a
missing `mkdocs-redirects` that failed every build for ~15 minutes while the
poller retried without telling anyone). Consolidated:
- **ONE venv.** Every trigger — the poller, the (dormant) webhook receiver, a
manual `publishing/build.sh`, CI — builds with the canonical checkout's
venv: `~/repos/docs/.venv-docs`. The poller's private venv was deleted.
- **The dedicated checkout stays, by design.** The poller still builds from
`/home/ubuntu/.docs-publisher`, its own clean clone: uncommitted edits in a
working clone must never block a deploy or leak into the published site.
- **Drift check.** Before every build the poller compares `publishing/` and
`mkdocs.yml` between its checkout and the canonical one; any mismatch fails
the deploy loudly (with a diff and the fix — `git -C ~/repos/docs pull
--ff-only`), and the failure counts toward the alert below. A canonical
checkout that lags behind main trips this too, on purpose: a deploy of
logic the canonical source cannot reproduce is the failure mode this exists
to prevent. A missing canonical checkout is fatal as well.
- **Plugin gate + self-heal.** `publishing/requirements.txt` pins every
package the build needs (`mkdocs`, `mkdocs-material`, `mkdocs-redirects`).
`build.sh` verifies the venv against it before every build and installs
anything missing or off-version (fast local no-op when current); it also
fails loudly if `mkdocs.yml` configures a plugin that `requirements.txt`
does not pin. The poller runs this as its pre-deploy check via
`build.sh --check-only`.
- **Failure alerting.** After 3 consecutive deploy failures the poller writes
a marker (`~/.docs-deploy.alert`) and sends ONE message to the InferenceBot
HQ Matrix room naming the failing commit — never more than one message per
hour (`~/.docs-deploy.last-alert`). The counter (`~/.docs-deploy.failcount`)
resets on the next successful deploy. A deploy that succeeds silently
resets everything; nothing announces routine successes.
Files: `~/.docs-deploy.state` (last deployed commit), `~/.docs-deploy.lock`
(deploy lock), `~/.docs-deploy.failcount`, `~/.docs-deploy.alert`,
`~/.docs-deploy.last-alert`. Manual deploys should respect the lock:
`flock -n ~/.docs-deploy.lock bash publishing/build.sh && flock -n
~/.docs-deploy.lock bash publishing/deploy.sh`.