publishing: build without credentials, publish from a host-owned script

Repo build logic now runs as the unprivileged docsbuild user with a scrubbed
environment; the Cloudron token lives only in a root-owned publish script that
executes no repository code.
This commit is contained in:
inference-admin committed 2026-11-19 21:51:05 +00:00
1 parent f2e74564df
commit b39c9b9289
2 files changed
+65 -91

No files matched your search

+42 -69
View File
@@ -17,11 +17,11 @@ repository root.
| `logo.svg` | Co-op logo (shared with the website repo) | | `logo.svg` | Co-op logo (shared with the website repo) |
| `build.sh` | Stage the Markdown into `.site-src/`, render to `.site-build/` | | `build.sh` | Stage the Markdown into `.site-src/`, render to `.site-build/` |
| `hooks.py` | MkDocs hooks: directory-link rewriting + content-hashed stylesheet URLs | | `hooks.py` | MkDocs hooks: directory-link rewriting + content-hashed stylesheet URLs |
| `deploy.sh` | Push the build into the Surfer app container | | `deploy.sh` | Manual publish of the build into the Surfer app (needs a Cloudron token) |
| `poll-watcher.sh` | Polls Gitea main every 60s; deploys on change (agent host) | | `poll-watcher.sh` | Legacy poller; no longer installed — superseded by the host-owned poller |
| `docs-deploy-poll.service` | systemd unit for the poller | | `docs-deploy-poll.service` | systemd unit for the poller (runs the host-owned script) |
| `webhook-receiver.py` | Gitea webhook → pull → build → deploy (agent host) | | `webhook-receiver.py` | Webhook receiver; not installed (see below) |
| `webhook-receiver.service` | systemd unit for the receiver | | `webhook-receiver.service` | systemd unit for the receiver; not installed |
| `requirements.txt` | Pinned Python build toolchain | | `requirements.txt` | Pinned Python build toolchain |
Nothing external is loaded by the site: no CDN, no third-party font service. Nothing external is loaded by the site: no CDN, no third-party font service.
@@ -71,79 +71,52 @@ Override the destination with `SURFER_APP` (the app's Cloudron location) and
## Automatic rebuilds (push to main) ## Automatic rebuilds (push to main)
Two mechanisms keep the site in step with `main`; both run on the agent host `docs-deploy-poll.service` polls Gitea every 60 seconds and deploys whenever
and deploy into the docs Cloudron app above. `main` moves, so a push appears on the site within about a minute with no human
action. Edit, commit, push — that is the whole deployment story.
**Polling (running now).** `docs-deploy-poll.service` polls Gitea every 60 **The credential-bearing steps are host-owned, not repository code.** The
seconds and deploys whenever `main` moves. A push appears on the site within poller runs a root-owned script (`/usr/local/sbin/docs-poll`) that does two
about a minute, with no human action. The Gitea webhook alone was not enough separate jobs:
because the agent host's cloud firewall only admits inbound SSH — a webhook
POST is silently dropped — while outbound access to Gitea works.
The service builds from a **dedicated clean clone** at 1. **Build as `docsbuild`.** The repository's build logic (`build.sh` → MkDocs
`/home/ubuntu/.docs-publisher`, not from anyone's working copy, so uncommitted → `hooks.py`) runs as an unprivileged system user with a scrubbed
edits in a working clone can never block a deploy. Edit, commit, push — that environment, from its own clean clone at `/var/lib/docs-publisher/repo`
is the whole deployment story. with its own venv. That user cannot read `/home/ubuntu` (mode 750), so repo
code cannot see any co-op credential — the Cloudron token, the gateway
master key, the agent's env file.
2. **Publish with a root-owned script.** `/usr/local/sbin/docs-publish` runs
no repository code. It takes only the built directory, writes `meta.json`,
and pushes it into the Surfer app using `CLOUDRON_TOKEN` from
`/etc/docs-deploy.env` (root-only, mode 0600).
**Webhook (wired, waiting on one port).** A Gitea webhook is registered on A commit that lands on `main` therefore cannot execute anything with the
`co-op/docs` (push events, branch `main`) pointing at the receiver below. The Cloudron token. It can only influence the bytes of the site.
moment the Infomaniak panel allows inbound TCP to the agent host — port 8787
from the Cloudron box 179.237.86.116 is enough — start the receiver and pushes
deploy instantly instead of within a minute:
```sh **The webhook receiver is installed but not running.** `docs-deploy-webhook`
sudo systemctl enable --now docs-deploy-webhook is inactive and nothing listens on 8787. It predates this split, loads the
tail -f /home/ubuntu/docs-deploy.log # what both mechanisms are doing agent's env file, and runs repo scripts with the Cloudron token; do not enable
``` it without porting it to the same host-owned build/publish split. The Gitea
webhook registration is harmless while the receiver is down.
The receiver authenticates webhooks with an HMAC-SHA256 shared secret
(`DEPLOY_WEBHOOK_SECRET` in the agent host's env file) and only accepts pushes
to `main` of `co-op/docs` from the Cloudron box. Woodpecker CI (Cloudron
catalog) could do the same job inside Cloudron, at the cost of running another
app; the agent-host mechanisms were chosen because they add no new Cloudron app.
Each deploy writes `meta.json` into the site with the commit it was built Each deploy writes `meta.json` into the site with the commit it was built
from, so a deploy can be verified from the served site itself: from, so a deploy can be verified from the served site itself:
https://docs.inference.coop/meta.json https://docs.inference.coop/meta.json
## One build environment (2026-11-12) If a deploy breaks, roll back with the saved unit
(`/root/docs-deploy-poll.service.before-split`) and the previous build in
`/var/lib/docs-publisher/repo/.site-build`; the poller retries every 60 seconds
and sends one Matrix alert after 3 consecutive failures.
The deploy used to have TWO build environments — this repo's `.venv-docs` and ## Build environment
a second private venv in the poller's checkout at
`/home/ubuntu/.docs-publisher/.venv-docs`. They diverged twice, each time
silently delaying a deploy (the brand.css cache-busting confusion, then a
missing `mkdocs-redirects` that failed every build for ~15 minutes while the
poller retried without telling anyone). Consolidated:
- **ONE venv.** Every trigger — the poller, the (dormant) webhook receiver, a `publishing/requirements.txt` pins every package the build needs (`mkdocs`,
manual `publishing/build.sh`, CI — builds with the canonical checkout's `mkdocs-material`, `mkdocs-redirects`). `build.sh` verifies the build venv
venv: `~/repos/docs/.venv-docs`. The poller's private venv was deleted. against it before every build and installs anything missing or off-version
- **The dedicated checkout stays, by design.** The poller still builds from (fast local no-op when current); it also fails loudly if `mkdocs.yml`
`/home/ubuntu/.docs-publisher`, its own clean clone: uncommitted edits in a configures a plugin that `requirements.txt` does not pin.
working clone must never block a deploy or leak into the published site.
- **Drift check.** Before every build the poller compares `publishing/` and
`mkdocs.yml` between its checkout and the canonical one; any mismatch fails
the deploy loudly (with a diff and the fix — `git -C ~/repos/docs pull
--ff-only`), and the failure counts toward the alert below. A canonical
checkout that lags behind main trips this too, on purpose: a deploy of
logic the canonical source cannot reproduce is the failure mode this exists
to prevent. A missing canonical checkout is fatal as well.
- **Plugin gate + self-heal.** `publishing/requirements.txt` pins every
package the build needs (`mkdocs`, `mkdocs-material`, `mkdocs-redirects`).
`build.sh` verifies the venv against it before every build and installs
anything missing or off-version (fast local no-op when current); it also
fails loudly if `mkdocs.yml` configures a plugin that `requirements.txt`
does not pin. The poller runs this as its pre-deploy check via
`build.sh --check-only`.
- **Failure alerting.** After 3 consecutive deploy failures the poller writes
a marker (`~/.docs-deploy.alert`) and sends ONE message to the InferenceBot
HQ Matrix room naming the failing commit — never more than one message per
hour (`~/.docs-deploy.last-alert`). The counter (`~/.docs-deploy.failcount`)
resets on the next successful deploy. A deploy that succeeds silently
resets everything; nothing announces routine successes.
Files: `~/.docs-deploy.state` (last deployed commit), `~/.docs-deploy.lock` The deployed poller keeps one build venv at
(deploy lock), `~/.docs-deploy.failcount`, `~/.docs-deploy.alert`, `/var/lib/docs-publisher/repo/.venv-docs`, owned by `docsbuild`. A manual
`~/.docs-deploy.last-alert`. Manual deploys should respect the lock: `publishing/build.sh` uses `.venv-docs` in whatever checkout you run it from;
`flock -n ~/.docs-deploy.lock bash publishing/build.sh && flock -n that is fine for local previews and does not affect deploys.
~/.docs-deploy.lock bash publishing/deploy.sh`.
+23 -22
View File
@@ -1,41 +1,42 @@
# systemd unit for the docs auto-deploy poller (agent host). # systemd unit for the docs auto-deploy poller (agent host).
# #
# docs-deploy-poll.service ALWAYS ON. Polls Gitea every 60s; deploys on # docs-deploy-poll.service ALWAYS ON. Polls Gitea every 60s; deploys on
# any change to main. Works with the agent # any change to main.
# host's current firewall (outbound only).
# #
# docs-deploy-webhook.service INSTALLED, INACTIVE UNTIL THE PORT IS OPEN. # docs-deploy-webhook.service INSTALLED, INACTIVE. It predates the
# Instant push -> deploy once the Infomaniak # build/publish split, loads the agent's env
# panel allows inbound TCP to the agent host # file, and runs repo scripts with the Cloudron
# (the Gitea webhook is already registered). # token. Do not enable it until it is ported to
# the same split.
# #
# Install: # Install:
# sudo cp publishing/docs-deploy-poll.service /etc/systemd/system/ # sudo install -o root -g root -m 0644 publishing/docs-deploy-poll.service \
# /etc/systemd/system/docs-deploy-poll.service
# sudo systemctl daemon-reload # sudo systemctl daemon-reload
# sudo systemctl enable --now docs-deploy-poll # sudo systemctl enable --now docs-deploy-poll
# #
# poll-watcher.sh itself needs NO manual copy: the service runs the copy in # The unit runs a HOST-OWNED script, /usr/local/sbin/docs-poll, which is not
# the dedicated checkout (/home/ubuntu/.docs-publisher), which git pull keeps # copied from this repository. That script:
# at main on every deploy, and the watcher re-execs itself when a deploy # * builds as the unprivileged `docsbuild` user from its own checkout at
# changed it. # /var/lib/docs-publisher/repo (a scrubbed environment, no read access to
# /home/ubuntu, so repo code cannot see any co-op credential);
# * publishes with /usr/local/sbin/docs-publish, a root-owned script that
# runs no repository code and reads the Cloudron token from
# /etc/docs-deploy.env (root-only, mode 0600).
# #
# ONE BUILD ENVIRONMENT: the poller builds from its own dedicated checkout # Repo code (build.sh, hooks.py) therefore never executes with a credential.
# (uncommitted edits in a working clone must never block or leak into a # See publishing/README.md.
# deploy) but with the ONE canonical venv, DOCS_CANONICAL_DIR/.venv-docs, and
# drift-checks its publishing/ tree against the canonical checkout before
# every build. See publishing/README.md.
[Unit] [Unit]
Description=Inference docs: poll Gitea main and deploy to Surfer Description=Inference docs: poll Gitea main, build unprivileged, publish without repo code
After=network-online.target After=network-online.target
Wants=network-online.target Wants=network-online.target
[Service] [Service]
User=ubuntu User=root
WorkingDirectory=/home/ubuntu/.docs-publisher Environment=DOCS_REPO_DIR=/var/lib/docs-publisher/repo
Environment=DOCS_REPO_DIR=/home/ubuntu/.docs-publisher Environment=DOCS_DEPLOY_LOG=/var/log/docs-deploy.log
Environment=DOCS_CANONICAL_DIR=/home/ubuntu/repos/docs ExecStart=/usr/local/sbin/docs-poll
ExecStart=/bin/bash /home/ubuntu/.docs-publisher/publishing/poll-watcher.sh
Restart=on-failure Restart=on-failure
RestartSec=10 RestartSec=10