diff --git a/publishing/README.md b/publishing/README.md index 131f5df..ac2b2bd 100644 --- a/publishing/README.md +++ b/publishing/README.md @@ -16,6 +16,8 @@ repository root. | `logo.svg` | Co-op logo (shared with the website repo) | | `build.sh` | Stage the Markdown into `.site-src/`, render to `.site-build/` | | `deploy.sh` | Push the build into the Surfer app container | +| `poll-watcher.sh` | Polls Gitea main every 60s; deploys on change (agent host) | +| `docs-deploy-poll.service` | systemd unit for the poller | | `webhook-receiver.py` | Gitea webhook → pull → build → deploy (agent host) | | `webhook-receiver.service` | systemd unit for the receiver | | `requirements.txt` | Pinned Python build toolchain | @@ -48,21 +50,31 @@ canonical one. ## Automatic rebuilds (push to main) -A push to `main` on `co-op/docs` fires a Gitea webhook (repo settings → -webhooks) to the receiver on the agent host, which pulls, rebuilds and -redeploys. On the agent host: +Two mechanisms keep the site in step with `main`; both run on the agent host +and neither adds a Cloudron app. + +**Polling (running now).** `docs-deploy-poll.service` polls Gitea every 60 +seconds and deploys whenever `main` moves. A push appears on the site within +about a minute, with no human action. The Gitea webhook alone was not enough +because the agent host's cloud firewall only admits inbound SSH — a webhook +POST is silently dropped — while outbound access to Gitea works. + +**Webhook (wired, waiting on one port).** A Gitea webhook is registered on +`co-op/docs` (push events, branch `main`) pointing at the receiver below. The +moment the Infomaniak panel allows inbound TCP to the agent host — port 8787 +from the Cloudron box 179.237.86.116 is enough — start the receiver and pushes +deploy instantly instead of within a minute: ```sh -sudo cp publishing/webhook-receiver.service /etc/systemd/system/docs-deploy-webhook.service -sudo systemctl daemon-reload && sudo systemctl enable --now docs-deploy-webhook -tail -f /home/ubuntu/docs-deploy.log # what the receiver is doing +sudo systemctl enable --now docs-deploy-webhook +tail -f /home/ubuntu/docs-deploy.log # what both mechanisms are doing ``` The receiver authenticates webhooks with an HMAC-SHA256 shared secret (`DEPLOY_WEBHOOK_SECRET` in the agent host's env file) and only accepts pushes to `main` of `co-op/docs` from the Cloudron box. Woodpecker CI (Cloudron catalog) could do the same job inside Cloudron, at the cost of running another -app; the receiver was chosen because it adds no new Cloudron app. +app; the agent-host mechanisms were chosen because they add no new Cloudron app. Each deploy writes `meta.json` into the site with the commit it was built from, so a deploy can be verified from the served site itself: diff --git a/publishing/docs-deploy-poll.service b/publishing/docs-deploy-poll.service new file mode 100644 index 0000000..8740b4b --- /dev/null +++ b/publishing/docs-deploy-poll.service @@ -0,0 +1,35 @@ +# systemd units for the docs deploy automation (agent host). +# +# Two mechanisms, one job — keep https://docs.inference.coop/ in step with +# co-op/docs main: +# +# docs-deploy-poll.service ALWAYS ON. Polls Gitea every 60s; deploys on +# any change to main. Works with the agent +# host's current firewall (outbound only). +# +# docs-deploy-webhook.service INSTALLED, INACTIVE UNTIL THE PORT IS OPEN. +# Instant push -> deploy once the Infomaniak +# panel allows inbound TCP to the agent host +# (the Gitea webhook is already registered). +# +# Install both: +# sudo cp publishing/poll-watcher.sh publishing/webhook-receiver.py /home/ubuntu/repos/docs/publishing/ +# sudo cp publishing/docs-deploy-poll.service publishing/webhook-receiver.service /etc/systemd/system/ +# sudo systemctl daemon-reload +# sudo systemctl enable --now docs-deploy-poll +# sudo systemctl enable --now docs-deploy-webhook # when the port is open + +[Unit] +Description=Inference docs: poll Gitea main and deploy to Surfer +After=network-online.target +Wants=network-online.target + +[Service] +User=ubuntu +WorkingDirectory=/home/ubuntu/repos/docs +ExecStart=/bin/bash /home/ubuntu/repos/docs/publishing/poll-watcher.sh +Restart=on-failure +RestartSec=10 + +[Install] +WantedBy=multi-user.target diff --git a/publishing/poll-watcher.sh b/publishing/poll-watcher.sh new file mode 100755 index 0000000..81b2efc --- /dev/null +++ b/publishing/poll-watcher.sh @@ -0,0 +1,66 @@ +#!/usr/bin/env bash +# Poll Gitea for new commits on main and deploy the docs site when it moves. +# +# Why polling: the agent host's cloud firewall only admits inbound SSH, so a +# Gitea webhook cannot reach the receiver (publishing/webhook-receiver.py) — +# every port is silently dropped. Outbound to git.inference.coop works, so the +# agent host polls instead: a push to main is picked up within ~a minute and +# deployed with no human action. When the operator opens a TCP port to the +# agent host in the Infomaniak panel, the webhook takes over (its unit is +# already installed); until then this watcher is the automation. +# +# Runs on the agent host as a systemd service (docs-deploy-poll.service). +# Deploy steps are serialised with the same lock the webhook receiver uses, so +# the two mechanisms can never deploy concurrently. + +set -uo pipefail + +REPO_DIR="${DOCS_REPO_DIR:-/home/ubuntu/repos/docs}" +ENV_FILE="${DOCS_ENV_FILE:-/home/ubuntu/.hermes/.env}" +LOG_FILE="${DOCS_DEPLOY_LOG:-/home/ubuntu/docs-deploy.log}" +LOCK_FILE="${DOCS_DEPLOY_LOCK:-/home/ubuntu/.docs-deploy.lock}" +INTERVAL="${DOCS_POLL_INTERVAL:-60}" + +cd "$REPO_DIR" + +log() { echo "$(date -u +%Y-%m-%dT%H:%M:%SZ) [poll] $*" | tee -a "$LOG_FILE" >&2; } + +# The Cloudron token travels by environment only; it is read from the env file +# and never echoed. +CLOUDRON_TOKEN="$(sed -n 's/^CLOUDRON_TOKEN=//p' "$ENV_FILE" | head -1 | tr -d '"' | tr -d "'")" +export CLOUDRON_TOKEN +if [ -z "$CLOUDRON_TOKEN" ]; then + log "FATAL: CLOUDRON_TOKEN not found in $ENV_FILE" + exit 1 +fi + +log "watcher started: polling $REPO_DIR every ${INTERVAL}s" + +while true; do + if ! git fetch --quiet origin main 2>>"$LOG_FILE"; then + log "git fetch failed; will retry" + sleep "$INTERVAL" + continue + fi + + LOCAL="$(git rev-parse HEAD 2>/dev/null || echo none)" + REMOTE="$(git rev-parse origin/main 2>/dev/null || echo none)" + + if [ "$LOCAL" != "$REMOTE" ]; then + log "main moved: $LOCAL -> $REMOTE; deploying" + # flock -n shares the receiver's lock: one deploy at a time, ever. + if flock -n "$LOCK_FILE" bash -c ' + set -e + cd "$1" + git pull --ff-only origin main + bash publishing/build.sh + bash publishing/deploy.sh + ' _ "$REPO_DIR" >>"$LOG_FILE" 2>&1; then + log "deploy finished for $REMOTE" + else + log "deploy FAILED for $REMOTE (see log above)" + fi + fi + + sleep "$INTERVAL" +done