The poller's private venv (.docs-publisher/.venv-docs) diverged from the canonical one twice, each time silently delaying a deploy. Consolidate: - poll-watcher.sh builds with the ONE canonical venv (~/repos/docs/.venv-docs) while keeping its dedicated checkout (uncommitted edits in a working clone must never block or leak into a deploy). - New drift check: publishing/ + mkdocs.yml are compared with the canonical checkout before every build; a mismatch fails the deploy loudly instead of shipping logic the canonical source cannot reproduce. - Pre-deploy gate: build.sh --check-only verifies the pinned toolchain and every configured plugin (self-heals missing pins, fails loudly naming an unpinned plugin). - Alerting: 3 consecutive deploy failures -> marker file + ONE Matrix message to the InferenceBot HQ room (hourly cap); counter resets on next success. - The watcher re-execs itself when a deploy changed poll-watcher.sh, so the running loop always matches the deployed one. - webhook receiver (dormant) repointed at the canonical venv too; unit files and README document the layout.
150 lines
7.5 KiB
Markdown
150 lines
7.5 KiB
Markdown
# Publishing the documentation site
|
|
|
|
The Markdown in this repository is the single source of truth. This directory
|
|
is the publishing layer that turns it into a member-readable static site and
|
|
puts it on the co-op's docs Surfer app at https://docs.inference.coop/.
|
|
Documents are never edited here — a change
|
|
to a document is a normal commit or pull request against the Markdown at the
|
|
repository root.
|
|
|
|
## What is built
|
|
|
|
| File | Purpose |
|
|
|---|---|
|
|
| `../mkdocs.yml` | Site configuration (MkDocs + Material for MkDocs) |
|
|
| `brand.css` | Co-op palette + self-hosted Figtree webfont (SIL OFL 1.1) |
|
|
| `fonts/` | Figtree woff2 files and the OFL licence text |
|
|
| `logo.svg` | Co-op logo (shared with the website repo) |
|
|
| `build.sh` | Stage the Markdown into `.site-src/`, render to `.site-build/` |
|
|
| `hooks.py` | MkDocs hooks: directory-link rewriting + content-hashed stylesheet URLs |
|
|
| `deploy.sh` | Push the build into the Surfer app container |
|
|
| `poll-watcher.sh` | Polls Gitea main every 60s; deploys on change (agent host) |
|
|
| `docs-deploy-poll.service` | systemd unit for the poller |
|
|
| `webhook-receiver.py` | Gitea webhook → pull → build → deploy (agent host) |
|
|
| `webhook-receiver.service` | systemd unit for the receiver |
|
|
| `requirements.txt` | Pinned Python build toolchain |
|
|
|
|
Nothing external is loaded by the site: no CDN, no third-party font service.
|
|
Fonts are served from this repository's own output.
|
|
|
|
### Cache-proof stylesheets
|
|
|
|
`brand.css` and `typography.css` are listed under `extra_css`, so at a fixed URL
|
|
they would be vulnerable to a stale browser cache after a deploy — a warm cache
|
|
keeps serving the old file because the URL never changes. `hooks.py`'s
|
|
`on_post_build` fixes this at build time: it copies each stylesheet to
|
|
`<name>.<sha256-8>.css` in the output and rewrites the `<link>` in every built
|
|
page to the hashed name. A change to a stylesheet's content therefore always
|
|
produces a new URL. The authored sources keep their names and locations; the
|
|
hashed copies exist only in the build output. The hashed copy sits in the same
|
|
directory as the original, so the `@font-face` `url("fonts/…")` references keep
|
|
resolving. The deploy is unchanged.
|
|
|
|
## Build
|
|
|
|
```sh
|
|
publishing/build.sh # creates .venv-docs on first run
|
|
```
|
|
|
|
Output lands in `.site-build/` (gitignored). `mkdocs build --strict` is what
|
|
CI-style runs should use, so broken links fail the build.
|
|
|
|
## Deploy
|
|
|
|
```sh
|
|
CLOUDRON_TOKEN=... publishing/deploy.sh
|
|
```
|
|
|
|
Writes the build into the **docs Surfer app** (Cloudron app `docs.inference.coop`,
|
|
`io.cloudron.surfer`) at:
|
|
|
|
- `/app/data/public/` — served at **https://docs.inference.coop/**
|
|
|
|
That is now the one canonical location. The docs used to be a pair of
|
|
directories inside the *marketing* Surfer app (`inference.coop`) as well, at
|
|
`/app/data/public-docs/` and `/app/data/public/docs/`; both were deleted when
|
|
this app took over, so `https://inference.coop/docs/` no longer serves the docs
|
|
(nothing on the marketing site linked to it).
|
|
|
|
Override the destination with `SURFER_APP` (the app's Cloudron location) and
|
|
`DOCS_DEST` (path inside the container) if you ever need to.
|
|
|
|
## Automatic rebuilds (push to main)
|
|
|
|
Two mechanisms keep the site in step with `main`; both run on the agent host
|
|
and deploy into the docs Cloudron app above.
|
|
|
|
**Polling (running now).** `docs-deploy-poll.service` polls Gitea every 60
|
|
seconds and deploys whenever `main` moves. A push appears on the site within
|
|
about a minute, with no human action. The Gitea webhook alone was not enough
|
|
because the agent host's cloud firewall only admits inbound SSH — a webhook
|
|
POST is silently dropped — while outbound access to Gitea works.
|
|
|
|
The service builds from a **dedicated clean clone** at
|
|
`/home/ubuntu/.docs-publisher`, not from anyone's working copy, so uncommitted
|
|
edits in a working clone can never block a deploy. Edit, commit, push — that
|
|
is the whole deployment story.
|
|
|
|
**Webhook (wired, waiting on one port).** A Gitea webhook is registered on
|
|
`co-op/docs` (push events, branch `main`) pointing at the receiver below. The
|
|
moment the Infomaniak panel allows inbound TCP to the agent host — port 8787
|
|
from the Cloudron box 179.237.86.116 is enough — start the receiver and pushes
|
|
deploy instantly instead of within a minute:
|
|
|
|
```sh
|
|
sudo systemctl enable --now docs-deploy-webhook
|
|
tail -f /home/ubuntu/docs-deploy.log # what both mechanisms are doing
|
|
```
|
|
|
|
The receiver authenticates webhooks with an HMAC-SHA256 shared secret
|
|
(`DEPLOY_WEBHOOK_SECRET` in the agent host's env file) and only accepts pushes
|
|
to `main` of `co-op/docs` from the Cloudron box. Woodpecker CI (Cloudron
|
|
catalog) could do the same job inside Cloudron, at the cost of running another
|
|
app; the agent-host mechanisms were chosen because they add no new Cloudron app.
|
|
|
|
Each deploy writes `meta.json` into the site with the commit it was built
|
|
from, so a deploy can be verified from the served site itself:
|
|
https://docs.inference.coop/meta.json
|
|
|
|
## One build environment (2026-11-12)
|
|
|
|
The deploy used to have TWO build environments — this repo's `.venv-docs` and
|
|
a second private venv in the poller's checkout at
|
|
`/home/ubuntu/.docs-publisher/.venv-docs`. They diverged twice, each time
|
|
silently delaying a deploy (the brand.css cache-busting confusion, then a
|
|
missing `mkdocs-redirects` that failed every build for ~15 minutes while the
|
|
poller retried without telling anyone). Consolidated:
|
|
|
|
- **ONE venv.** Every trigger — the poller, the (dormant) webhook receiver, a
|
|
manual `publishing/build.sh`, CI — builds with the canonical checkout's
|
|
venv: `~/repos/docs/.venv-docs`. The poller's private venv was deleted.
|
|
- **The dedicated checkout stays, by design.** The poller still builds from
|
|
`/home/ubuntu/.docs-publisher`, its own clean clone: uncommitted edits in a
|
|
working clone must never block a deploy or leak into the published site.
|
|
- **Drift check.** Before every build the poller compares `publishing/` and
|
|
`mkdocs.yml` between its checkout and the canonical one; any mismatch fails
|
|
the deploy loudly (with a diff and the fix — `git -C ~/repos/docs pull
|
|
--ff-only`), and the failure counts toward the alert below. A canonical
|
|
checkout that lags behind main trips this too, on purpose: a deploy of
|
|
logic the canonical source cannot reproduce is the failure mode this exists
|
|
to prevent. A missing canonical checkout is fatal as well.
|
|
- **Plugin gate + self-heal.** `publishing/requirements.txt` pins every
|
|
package the build needs (`mkdocs`, `mkdocs-material`, `mkdocs-redirects`).
|
|
`build.sh` verifies the venv against it before every build and installs
|
|
anything missing or off-version (fast local no-op when current); it also
|
|
fails loudly if `mkdocs.yml` configures a plugin that `requirements.txt`
|
|
does not pin. The poller runs this as its pre-deploy check via
|
|
`build.sh --check-only`.
|
|
- **Failure alerting.** After 3 consecutive deploy failures the poller writes
|
|
a marker (`~/.docs-deploy.alert`) and sends ONE message to the InferenceBot
|
|
HQ Matrix room naming the failing commit — never more than one message per
|
|
hour (`~/.docs-deploy.last-alert`). The counter (`~/.docs-deploy.failcount`)
|
|
resets on the next successful deploy. A deploy that succeeds silently
|
|
resets everything; nothing announces routine successes.
|
|
|
|
Files: `~/.docs-deploy.state` (last deployed commit), `~/.docs-deploy.lock`
|
|
(deploy lock), `~/.docs-deploy.failcount`, `~/.docs-deploy.alert`,
|
|
`~/.docs-deploy.last-alert`. Manual deploys should respect the lock:
|
|
`flock -n ~/.docs-deploy.lock bash publishing/build.sh && flock -n
|
|
~/.docs-deploy.lock bash publishing/deploy.sh`.
|