publishing: build without credentials, publish from a host-owned script
Repo build logic now runs as the unprivileged docsbuild user with a scrubbed environment; the Cloudron token lives only in a root-owned publish script that executes no repository code.
This commit is contained in:
1 parent
f2e74564df
commit
b39c9b9289
2 files changed
+65
-91
No files matched your search
+42
-69
@@ -17,11 +17,11 @@ repository root.
|
|||||||
| `logo.svg` | Co-op logo (shared with the website repo) |
|
| `logo.svg` | Co-op logo (shared with the website repo) |
|
||||||
| `build.sh` | Stage the Markdown into `.site-src/`, render to `.site-build/` |
|
| `build.sh` | Stage the Markdown into `.site-src/`, render to `.site-build/` |
|
||||||
| `hooks.py` | MkDocs hooks: directory-link rewriting + content-hashed stylesheet URLs |
|
| `hooks.py` | MkDocs hooks: directory-link rewriting + content-hashed stylesheet URLs |
|
||||||
| `deploy.sh` | Push the build into the Surfer app container |
|
| `deploy.sh` | Manual publish of the build into the Surfer app (needs a Cloudron token) |
|
||||||
| `poll-watcher.sh` | Polls Gitea main every 60s; deploys on change (agent host) |
|
| `poll-watcher.sh` | Legacy poller; no longer installed — superseded by the host-owned poller |
|
||||||
| `docs-deploy-poll.service` | systemd unit for the poller |
|
| `docs-deploy-poll.service` | systemd unit for the poller (runs the host-owned script) |
|
||||||
| `webhook-receiver.py` | Gitea webhook → pull → build → deploy (agent host) |
|
| `webhook-receiver.py` | Webhook receiver; not installed (see below) |
|
||||||
| `webhook-receiver.service` | systemd unit for the receiver |
|
| `webhook-receiver.service` | systemd unit for the receiver; not installed |
|
||||||
| `requirements.txt` | Pinned Python build toolchain |
|
| `requirements.txt` | Pinned Python build toolchain |
|
||||||
|
|
||||||
Nothing external is loaded by the site: no CDN, no third-party font service.
|
Nothing external is loaded by the site: no CDN, no third-party font service.
|
||||||
@@ -71,79 +71,52 @@ Override the destination with `SURFER_APP` (the app's Cloudron location) and
|
|||||||
|
|
||||||
## Automatic rebuilds (push to main)
|
## Automatic rebuilds (push to main)
|
||||||
|
|
||||||
Two mechanisms keep the site in step with `main`; both run on the agent host
|
`docs-deploy-poll.service` polls Gitea every 60 seconds and deploys whenever
|
||||||
and deploy into the docs Cloudron app above.
|
`main` moves, so a push appears on the site within about a minute with no human
|
||||||
|
action. Edit, commit, push — that is the whole deployment story.
|
||||||
|
|
||||||
**Polling (running now).** `docs-deploy-poll.service` polls Gitea every 60
|
**The credential-bearing steps are host-owned, not repository code.** The
|
||||||
seconds and deploys whenever `main` moves. A push appears on the site within
|
poller runs a root-owned script (`/usr/local/sbin/docs-poll`) that does two
|
||||||
about a minute, with no human action. The Gitea webhook alone was not enough
|
separate jobs:
|
||||||
because the agent host's cloud firewall only admits inbound SSH — a webhook
|
|
||||||
POST is silently dropped — while outbound access to Gitea works.
|
|
||||||
|
|
||||||
The service builds from a **dedicated clean clone** at
|
1. **Build as `docsbuild`.** The repository's build logic (`build.sh` → MkDocs
|
||||||
`/home/ubuntu/.docs-publisher`, not from anyone's working copy, so uncommitted
|
→ `hooks.py`) runs as an unprivileged system user with a scrubbed
|
||||||
edits in a working clone can never block a deploy. Edit, commit, push — that
|
environment, from its own clean clone at `/var/lib/docs-publisher/repo`
|
||||||
is the whole deployment story.
|
with its own venv. That user cannot read `/home/ubuntu` (mode 750), so repo
|
||||||
|
code cannot see any co-op credential — the Cloudron token, the gateway
|
||||||
|
master key, the agent's env file.
|
||||||
|
2. **Publish with a root-owned script.** `/usr/local/sbin/docs-publish` runs
|
||||||
|
no repository code. It takes only the built directory, writes `meta.json`,
|
||||||
|
and pushes it into the Surfer app using `CLOUDRON_TOKEN` from
|
||||||
|
`/etc/docs-deploy.env` (root-only, mode 0600).
|
||||||
|
|
||||||
**Webhook (wired, waiting on one port).** A Gitea webhook is registered on
|
A commit that lands on `main` therefore cannot execute anything with the
|
||||||
`co-op/docs` (push events, branch `main`) pointing at the receiver below. The
|
Cloudron token. It can only influence the bytes of the site.
|
||||||
moment the Infomaniak panel allows inbound TCP to the agent host — port 8787
|
|
||||||
from the Cloudron box 179.237.86.116 is enough — start the receiver and pushes
|
|
||||||
deploy instantly instead of within a minute:
|
|
||||||
|
|
||||||
```sh
|
**The webhook receiver is installed but not running.** `docs-deploy-webhook`
|
||||||
sudo systemctl enable --now docs-deploy-webhook
|
is inactive and nothing listens on 8787. It predates this split, loads the
|
||||||
tail -f /home/ubuntu/docs-deploy.log # what both mechanisms are doing
|
agent's env file, and runs repo scripts with the Cloudron token; do not enable
|
||||||
```
|
it without porting it to the same host-owned build/publish split. The Gitea
|
||||||
|
webhook registration is harmless while the receiver is down.
|
||||||
The receiver authenticates webhooks with an HMAC-SHA256 shared secret
|
|
||||||
(`DEPLOY_WEBHOOK_SECRET` in the agent host's env file) and only accepts pushes
|
|
||||||
to `main` of `co-op/docs` from the Cloudron box. Woodpecker CI (Cloudron
|
|
||||||
catalog) could do the same job inside Cloudron, at the cost of running another
|
|
||||||
app; the agent-host mechanisms were chosen because they add no new Cloudron app.
|
|
||||||
|
|
||||||
Each deploy writes `meta.json` into the site with the commit it was built
|
Each deploy writes `meta.json` into the site with the commit it was built
|
||||||
from, so a deploy can be verified from the served site itself:
|
from, so a deploy can be verified from the served site itself:
|
||||||
https://docs.inference.coop/meta.json
|
https://docs.inference.coop/meta.json
|
||||||
|
|
||||||
## One build environment (2026-11-12)
|
If a deploy breaks, roll back with the saved unit
|
||||||
|
(`/root/docs-deploy-poll.service.before-split`) and the previous build in
|
||||||
|
`/var/lib/docs-publisher/repo/.site-build`; the poller retries every 60 seconds
|
||||||
|
and sends one Matrix alert after 3 consecutive failures.
|
||||||
|
|
||||||
The deploy used to have TWO build environments — this repo's `.venv-docs` and
|
## Build environment
|
||||||
a second private venv in the poller's checkout at
|
|
||||||
`/home/ubuntu/.docs-publisher/.venv-docs`. They diverged twice, each time
|
|
||||||
silently delaying a deploy (the brand.css cache-busting confusion, then a
|
|
||||||
missing `mkdocs-redirects` that failed every build for ~15 minutes while the
|
|
||||||
poller retried without telling anyone). Consolidated:
|
|
||||||
|
|
||||||
- **ONE venv.** Every trigger — the poller, the (dormant) webhook receiver, a
|
`publishing/requirements.txt` pins every package the build needs (`mkdocs`,
|
||||||
manual `publishing/build.sh`, CI — builds with the canonical checkout's
|
`mkdocs-material`, `mkdocs-redirects`). `build.sh` verifies the build venv
|
||||||
venv: `~/repos/docs/.venv-docs`. The poller's private venv was deleted.
|
against it before every build and installs anything missing or off-version
|
||||||
- **The dedicated checkout stays, by design.** The poller still builds from
|
(fast local no-op when current); it also fails loudly if `mkdocs.yml`
|
||||||
`/home/ubuntu/.docs-publisher`, its own clean clone: uncommitted edits in a
|
configures a plugin that `requirements.txt` does not pin.
|
||||||
working clone must never block a deploy or leak into the published site.
|
|
||||||
- **Drift check.** Before every build the poller compares `publishing/` and
|
|
||||||
`mkdocs.yml` between its checkout and the canonical one; any mismatch fails
|
|
||||||
the deploy loudly (with a diff and the fix — `git -C ~/repos/docs pull
|
|
||||||
--ff-only`), and the failure counts toward the alert below. A canonical
|
|
||||||
checkout that lags behind main trips this too, on purpose: a deploy of
|
|
||||||
logic the canonical source cannot reproduce is the failure mode this exists
|
|
||||||
to prevent. A missing canonical checkout is fatal as well.
|
|
||||||
- **Plugin gate + self-heal.** `publishing/requirements.txt` pins every
|
|
||||||
package the build needs (`mkdocs`, `mkdocs-material`, `mkdocs-redirects`).
|
|
||||||
`build.sh` verifies the venv against it before every build and installs
|
|
||||||
anything missing or off-version (fast local no-op when current); it also
|
|
||||||
fails loudly if `mkdocs.yml` configures a plugin that `requirements.txt`
|
|
||||||
does not pin. The poller runs this as its pre-deploy check via
|
|
||||||
`build.sh --check-only`.
|
|
||||||
- **Failure alerting.** After 3 consecutive deploy failures the poller writes
|
|
||||||
a marker (`~/.docs-deploy.alert`) and sends ONE message to the InferenceBot
|
|
||||||
HQ Matrix room naming the failing commit — never more than one message per
|
|
||||||
hour (`~/.docs-deploy.last-alert`). The counter (`~/.docs-deploy.failcount`)
|
|
||||||
resets on the next successful deploy. A deploy that succeeds silently
|
|
||||||
resets everything; nothing announces routine successes.
|
|
||||||
|
|
||||||
Files: `~/.docs-deploy.state` (last deployed commit), `~/.docs-deploy.lock`
|
The deployed poller keeps one build venv at
|
||||||
(deploy lock), `~/.docs-deploy.failcount`, `~/.docs-deploy.alert`,
|
`/var/lib/docs-publisher/repo/.venv-docs`, owned by `docsbuild`. A manual
|
||||||
`~/.docs-deploy.last-alert`. Manual deploys should respect the lock:
|
`publishing/build.sh` uses `.venv-docs` in whatever checkout you run it from;
|
||||||
`flock -n ~/.docs-deploy.lock bash publishing/build.sh && flock -n
|
that is fine for local previews and does not affect deploys.
|
||||||
~/.docs-deploy.lock bash publishing/deploy.sh`.
|
|
||||||
@@ -1,41 +1,42 @@
|
|||||||
# systemd unit for the docs auto-deploy poller (agent host).
|
# systemd unit for the docs auto-deploy poller (agent host).
|
||||||
#
|
#
|
||||||
# docs-deploy-poll.service ALWAYS ON. Polls Gitea every 60s; deploys on
|
# docs-deploy-poll.service ALWAYS ON. Polls Gitea every 60s; deploys on
|
||||||
# any change to main. Works with the agent
|
# any change to main.
|
||||||
# host's current firewall (outbound only).
|
|
||||||
#
|
#
|
||||||
# docs-deploy-webhook.service INSTALLED, INACTIVE UNTIL THE PORT IS OPEN.
|
# docs-deploy-webhook.service INSTALLED, INACTIVE. It predates the
|
||||||
# Instant push -> deploy once the Infomaniak
|
# build/publish split, loads the agent's env
|
||||||
# panel allows inbound TCP to the agent host
|
# file, and runs repo scripts with the Cloudron
|
||||||
# (the Gitea webhook is already registered).
|
# token. Do not enable it until it is ported to
|
||||||
|
# the same split.
|
||||||
#
|
#
|
||||||
# Install:
|
# Install:
|
||||||
# sudo cp publishing/docs-deploy-poll.service /etc/systemd/system/
|
# sudo install -o root -g root -m 0644 publishing/docs-deploy-poll.service \
|
||||||
|
# /etc/systemd/system/docs-deploy-poll.service
|
||||||
# sudo systemctl daemon-reload
|
# sudo systemctl daemon-reload
|
||||||
# sudo systemctl enable --now docs-deploy-poll
|
# sudo systemctl enable --now docs-deploy-poll
|
||||||
#
|
#
|
||||||
# poll-watcher.sh itself needs NO manual copy: the service runs the copy in
|
# The unit runs a HOST-OWNED script, /usr/local/sbin/docs-poll, which is not
|
||||||
# the dedicated checkout (/home/ubuntu/.docs-publisher), which git pull keeps
|
# copied from this repository. That script:
|
||||||
# at main on every deploy, and the watcher re-execs itself when a deploy
|
# * builds as the unprivileged `docsbuild` user from its own checkout at
|
||||||
# changed it.
|
# /var/lib/docs-publisher/repo (a scrubbed environment, no read access to
|
||||||
|
# /home/ubuntu, so repo code cannot see any co-op credential);
|
||||||
|
# * publishes with /usr/local/sbin/docs-publish, a root-owned script that
|
||||||
|
# runs no repository code and reads the Cloudron token from
|
||||||
|
# /etc/docs-deploy.env (root-only, mode 0600).
|
||||||
#
|
#
|
||||||
# ONE BUILD ENVIRONMENT: the poller builds from its own dedicated checkout
|
# Repo code (build.sh, hooks.py) therefore never executes with a credential.
|
||||||
# (uncommitted edits in a working clone must never block or leak into a
|
# See publishing/README.md.
|
||||||
# deploy) but with the ONE canonical venv, DOCS_CANONICAL_DIR/.venv-docs, and
|
|
||||||
# drift-checks its publishing/ tree against the canonical checkout before
|
|
||||||
# every build. See publishing/README.md.
|
|
||||||
|
|
||||||
[Unit]
|
[Unit]
|
||||||
Description=Inference docs: poll Gitea main and deploy to Surfer
|
Description=Inference docs: poll Gitea main, build unprivileged, publish without repo code
|
||||||
After=network-online.target
|
After=network-online.target
|
||||||
Wants=network-online.target
|
Wants=network-online.target
|
||||||
|
|
||||||
[Service]
|
[Service]
|
||||||
User=ubuntu
|
User=root
|
||||||
WorkingDirectory=/home/ubuntu/.docs-publisher
|
Environment=DOCS_REPO_DIR=/var/lib/docs-publisher/repo
|
||||||
Environment=DOCS_REPO_DIR=/home/ubuntu/.docs-publisher
|
Environment=DOCS_DEPLOY_LOG=/var/log/docs-deploy.log
|
||||||
Environment=DOCS_CANONICAL_DIR=/home/ubuntu/repos/docs
|
ExecStart=/usr/local/sbin/docs-poll
|
||||||
ExecStart=/bin/bash /home/ubuntu/.docs-publisher/publishing/poll-watcher.sh
|
|
||||||
Restart=on-failure
|
Restart=on-failure
|
||||||
RestartSec=10
|
RestartSec=10
|
||||||
|
|
||||||
|
|||||||
Reference in new issue
Block a user