docs site: add MkDocs publishing layer (build, deploy, push-to-main webhook)
Renders the repository's Markdown (which stays where it is) into a
member-readable static site with Material for MkDocs: co-op brand colours
and logo, self-hosted OFL Figtree, offline search, no external hosts.
- publishing/build.sh stage .site-src, render .site-build (mkdocs --strict)
- publishing/deploy.sh write the build into the Surfer app container
- publishing/webhook-receiver.py + .service Gitea push hook on main
triggers pull -> build -> deploy (agent host, no new
Cloudron app)
This commit is contained in:
1 parent
cdb62fc247
commit
62158c16cc
15 files changed
+849
No files matched your search
@@ -0,0 +1,69 @@
|
||||
# Publishing the documentation site
|
||||
|
||||
The Markdown in this repository is the single source of truth. This directory
|
||||
is the publishing layer that turns it into a member-readable static site and
|
||||
puts it on the co-op's Surfer app. Documents are never edited here — a change
|
||||
to a document is a normal commit or pull request against the Markdown at the
|
||||
repository root.
|
||||
|
||||
## What is built
|
||||
|
||||
| File | Purpose |
|
||||
|---|---|
|
||||
| `../mkdocs.yml` | Site configuration (MkDocs + Material for MkDocs) |
|
||||
| `brand.css` | Co-op palette + self-hosted Figtree webfont (SIL OFL 1.1) |
|
||||
| `fonts/` | Figtree woff2 files and the OFL licence text |
|
||||
| `logo.svg` | Co-op logo (shared with the website repo) |
|
||||
| `build.sh` | Stage the Markdown into `.site-src/`, render to `.site-build/` |
|
||||
| `deploy.sh` | Push the build into the Surfer app container |
|
||||
| `webhook-receiver.py` | Gitea webhook → pull → build → deploy (agent host) |
|
||||
| `webhook-receiver.service` | systemd unit for the receiver |
|
||||
| `requirements.txt` | Pinned Python build toolchain |
|
||||
|
||||
Nothing external is loaded by the site: no CDN, no third-party font service.
|
||||
Fonts are served from this repository's own output.
|
||||
|
||||
## Build
|
||||
|
||||
```sh
|
||||
publishing/build.sh # creates .venv-docs on first run
|
||||
```
|
||||
|
||||
Output lands in `.site-build/` (gitignored). `mkdocs build --strict` is what
|
||||
CI-style runs should use, so broken links fail the build.
|
||||
|
||||
## Deploy
|
||||
|
||||
```sh
|
||||
CLOUDRON_TOKEN=... publishing/deploy.sh
|
||||
```
|
||||
|
||||
Writes the build into the Surfer app (Cloudron app `inference.coop`) at:
|
||||
|
||||
- `/app/data/public-docs/` — served at **https://docs.inference.coop/**
|
||||
- `/app/data/public/docs/` — served at **https://inference.coop/docs/**
|
||||
|
||||
Both are written so the site stays reachable either way; the hostname is the
|
||||
canonical one.
|
||||
|
||||
## Automatic rebuilds (push to main)
|
||||
|
||||
A push to `main` on `co-op/docs` fires a Gitea webhook (repo settings →
|
||||
webhooks) to the receiver on the agent host, which pulls, rebuilds and
|
||||
redeploys. On the agent host:
|
||||
|
||||
```sh
|
||||
sudo cp publishing/webhook-receiver.service /etc/systemd/system/docs-deploy-webhook.service
|
||||
sudo systemctl daemon-reload && sudo systemctl enable --now docs-deploy-webhook
|
||||
tail -f /home/ubuntu/docs-deploy.log # what the receiver is doing
|
||||
```
|
||||
|
||||
The receiver authenticates webhooks with an HMAC-SHA256 shared secret
|
||||
(`DEPLOY_WEBHOOK_SECRET` in the agent host's env file) and only accepts pushes
|
||||
to `main` of `co-op/docs` from the Cloudron box. Woodpecker CI (Cloudron
|
||||
catalog) could do the same job inside Cloudron, at the cost of running another
|
||||
app; the receiver was chosen because it adds no new Cloudron app.
|
||||
|
||||
Each deploy writes `meta.json` into the site with the commit it was built
|
||||
from, so a deploy can be verified from the served site itself:
|
||||
https://docs.inference.coop/meta.json
|
||||
@@ -0,0 +1,145 @@
|
||||
/* Inference Cooperative brand layer for the documentation site.
|
||||
*
|
||||
* Source of truth: the co-op's website (repos/website/index.html) — its CSS
|
||||
* custom properties and its Figtree webfont. This file reuses those values so
|
||||
* the documentation reads as the same organisation, not a default theme.
|
||||
*
|
||||
* Standing constraint: everything here is local. No third-party font service
|
||||
* (the Google Fonts and gstatic hosts), no CDN, no other external host.
|
||||
*/
|
||||
|
||||
:root {
|
||||
/* -- the co-op's palette (copied from repos/website/index.html) -- */
|
||||
--cream: #faf8f5;
|
||||
--white: #ffffff;
|
||||
--ink: #2d3327;
|
||||
--muted: #6b7a62;
|
||||
--light-muted: #94a08c;
|
||||
--green-deep: #5b8c5a;
|
||||
--green-mid: #6ba86b;
|
||||
--green-light: #7ab87a;
|
||||
--border: #e8e2d8;
|
||||
--wash: #eef4ea;
|
||||
--earth-red: #a04b3a;
|
||||
--earth-red-wash: #f3e6e1;
|
||||
--bronze: #a97a4a;
|
||||
--bronze-wash: #f1e8da;
|
||||
}
|
||||
|
||||
/* ---------- Material for MkDocs: map the palette onto the theme ---------- */
|
||||
|
||||
:root,
|
||||
[data-md-color-scheme="default"] {
|
||||
--md-default-bg-color: var(--cream);
|
||||
--md-primary-fg-color: var(--green-deep);
|
||||
--md-primary-fg-color--light: var(--green-mid);
|
||||
--md-primary-fg-color--dark: #4a7a49;
|
||||
--md-primary-bg-color: var(--cream);
|
||||
--md-accent-fg-color: var(--earth-red);
|
||||
--md-typeset-a-color: var(--green-deep);
|
||||
--md-typeset-color: var(--ink);
|
||||
--md-footer-bg-color: var(--wash);
|
||||
--md-footer-fg-color: var(--ink);
|
||||
--md-footer-fg-color--light: var(--muted);
|
||||
--md-code-bg-color: #f2efe9;
|
||||
--md-code-fg-color: var(--ink);
|
||||
--md-admonition-bg-color: var(--wash);
|
||||
}
|
||||
|
||||
/* Dark scheme keeps the same accents; light stays the default. */
|
||||
[data-md-color-scheme="slate"] {
|
||||
--md-primary-fg-color: #3f6b3e;
|
||||
--md-accent-fg-color: #d98a75;
|
||||
--md-typeset-a-color: var(--green-light);
|
||||
--md-code-bg-color: #2e3328;
|
||||
--md-code-fg-color: #e8e2d8;
|
||||
}
|
||||
|
||||
/* ---------- Typography: Figtree, self-hosted (SIL OFL 1.1) ---------- */
|
||||
|
||||
@font-face {
|
||||
font-family: "Figtree";
|
||||
font-style: normal;
|
||||
font-weight: 400;
|
||||
font-display: swap;
|
||||
src: url("fonts/figtree-400.woff2") format("woff2");
|
||||
}
|
||||
@font-face {
|
||||
font-family: "Figtree";
|
||||
font-style: normal;
|
||||
font-weight: 500;
|
||||
font-display: swap;
|
||||
src: url("fonts/figtree-500.woff2") format("woff2");
|
||||
}
|
||||
@font-face {
|
||||
font-family: "Figtree";
|
||||
font-style: normal;
|
||||
font-weight: 600;
|
||||
font-display: swap;
|
||||
src: url("fonts/figtree-600.woff2") format("woff2");
|
||||
}
|
||||
@font-face {
|
||||
font-family: "Figtree";
|
||||
font-style: normal;
|
||||
font-weight: 700;
|
||||
font-display: swap;
|
||||
src: url("fonts/figtree-700.woff2") format("woff2");
|
||||
}
|
||||
|
||||
:root {
|
||||
--md-text-font-family: "Figtree", system-ui, -apple-system, "Segoe UI",
|
||||
Roboto, "Helvetica Neue", Arial, sans-serif;
|
||||
--md-code-font-family: ui-monospace, "SFMono-Regular", "Cascadia Mono",
|
||||
"Segoe UI Mono", Menlo, Consolas, monospace;
|
||||
}
|
||||
|
||||
/* ---------- Friendly details ---------- */
|
||||
|
||||
/* Links read like the website's: brand green, semibold, underline on hover. */
|
||||
.md-typeset a {
|
||||
font-weight: 600;
|
||||
}
|
||||
.md-typeset a:hover,
|
||||
.md-typeset a:focus {
|
||||
text-decoration: underline;
|
||||
}
|
||||
|
||||
/* Rounder cards and tables; a wash instead of grey where the theme has one. */
|
||||
.md-typeset .grid.cards > ul > li,
|
||||
.md-typeset .grid > .card,
|
||||
.md-typeset table:not([class]) {
|
||||
border-radius: 0.6rem;
|
||||
}
|
||||
.md-typeset table:not([class]) {
|
||||
border: 1px solid var(--border);
|
||||
}
|
||||
|
||||
/* Header: co-op green with the cream wordmark. */
|
||||
.md-header {
|
||||
box-shadow: none;
|
||||
border-bottom: 1px solid rgba(255, 255, 255, 0.25);
|
||||
}
|
||||
.md-header .md-header__title {
|
||||
font-weight: 600;
|
||||
}
|
||||
|
||||
/* Tabs and the search field pick up the wash. */
|
||||
.md-tabs {
|
||||
background-color: var(--green-deep);
|
||||
}
|
||||
.md-search__input {
|
||||
background-color: rgba(255, 255, 255, 0.92);
|
||||
}
|
||||
|
||||
/* Admonitions get the co-op's washes instead of stock greys. */
|
||||
.md-typeset .admonition,
|
||||
.md-typeset details {
|
||||
border-radius: 0.6rem;
|
||||
}
|
||||
.md-typeset .admonition.tip,
|
||||
.md-typeset .admonition.note {
|
||||
border-color: var(--green-mid);
|
||||
}
|
||||
.md-typeset .admonition.warning {
|
||||
border-color: var(--earth-red);
|
||||
}
|
||||
Executable
+55
@@ -0,0 +1,55 @@
|
||||
#!/usr/bin/env bash
|
||||
# Build the member-readable documentation site from this repository.
|
||||
#
|
||||
# The repository is the single source of truth. This script:
|
||||
# 1. stages a mirror of the documents into .site-src/ (generated, gitignored),
|
||||
# because MkDocs cannot read the repository root directly;
|
||||
# 2. renders .site-src/ to .site-build/ with MkDocs + Material.
|
||||
#
|
||||
# No document is moved, renamed, or rewritten — the staging copy is a build
|
||||
# artifact and disappears with `--clean`.
|
||||
#
|
||||
# Usage:
|
||||
# publishing/build.sh # create/use .venv-docs and build
|
||||
# publishing/build.sh /path/to/venv # use an existing venv
|
||||
#
|
||||
# In CI (Woodpecker), install the toolchain and run MkDocs directly:
|
||||
# pip install -r publishing/requirements.txt && mkdocs build --strict
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
REPO_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
|
||||
cd "$REPO_DIR"
|
||||
|
||||
VENV="${1:-$REPO_DIR/.venv-docs}"
|
||||
SITE_SRC="$REPO_DIR/.site-src"
|
||||
|
||||
# 1. Stage the documents (Markdown at the root, maintenance/, incidents/) plus
|
||||
# the publishing assets (fonts, CSS, icon). Exclude tooling and build output.
|
||||
rm -rf "$SITE_SRC"
|
||||
mkdir -p "$SITE_SRC"
|
||||
tar cf - \
|
||||
--exclude='.git' \
|
||||
--exclude='.site-src' \
|
||||
--exclude='.site-build' \
|
||||
--exclude='.venv-docs' \
|
||||
--exclude='*.sh' \
|
||||
--exclude='publishing/webhook-receiver.py' \
|
||||
--exclude='publishing/webhook-receiver.service' \
|
||||
--exclude='mkdocs.yml' \
|
||||
--exclude='.gitignore' \
|
||||
--exclude='requirements.txt' \
|
||||
. | tar xf - -C "$SITE_SRC"
|
||||
|
||||
# 2. Build.
|
||||
if [ ! -x "$VENV/bin/mkdocs" ]; then
|
||||
echo "Creating build venv at $VENV"
|
||||
python3 -m venv "$VENV"
|
||||
"$VENV/bin/pip" install --quiet --upgrade pip
|
||||
"$VENV/bin/pip" install --quiet -r "$REPO_DIR/publishing/requirements.txt"
|
||||
fi
|
||||
|
||||
echo "Building site -> $REPO_DIR/.site-build"
|
||||
"$VENV/bin/mkdocs" build --clean
|
||||
|
||||
echo "Done: $REPO_DIR/.site-build"
|
||||
Executable
+84
@@ -0,0 +1,84 @@
|
||||
#!/usr/bin/env bash
|
||||
# Publish the built documentation site to the Cloudron Surfer app.
|
||||
#
|
||||
# Surfer serves static files from inside its app container. The build is
|
||||
# written to two places:
|
||||
# /app/data/public-docs/ the Surfer site bound to docs.inference.coop
|
||||
# (served at the root of that hostname)
|
||||
# /app/data/public/docs/ a subdirectory of the marketing site, so the docs
|
||||
# stay viewable at https://inference.coop/docs/
|
||||
#
|
||||
# Mechanism (evidence: Cloudron app "Surfer" io.cloudron.surfer 7.4.1, and
|
||||
# repos/website/deploy.sh, which already deploys the marketing site this way):
|
||||
# pipe a tar of the build output into the container via `cloudron exec`, which
|
||||
# needs only the Cloudron API token — no WebDAV token, SFTP credentials or SSH.
|
||||
#
|
||||
# Usage:
|
||||
# CLOUDRON_TOKEN=... publishing/deploy.sh
|
||||
#
|
||||
# Env:
|
||||
# CLOUDRON_TOKEN (required) Cloudron API token
|
||||
# CLOUDRON_SERVER default my.inference.coop
|
||||
# SURFER_APP default inference.coop (the Surfer app's location)
|
||||
# SITE_DIR default <repo>/.site-build
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
REPO_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
|
||||
SITE_DIR="${SITE_DIR:-$REPO_DIR/.site-build}"
|
||||
SERVER="${CLOUDRON_SERVER:-my.inference.coop}"
|
||||
APP="${SURFER_APP:-inference.coop}"
|
||||
DEST_SITE="${DOCS_DEST_SITE:-/app/data/public-docs}" # docs.inference.coop
|
||||
DEST_PATH="${DOCS_DEST_PATH:-/app/data/public/docs}" # inference.coop/docs/
|
||||
|
||||
CLOUDRON_BIN="${CLOUDRON_BIN:-$(command -v cloudron || true)}"
|
||||
if [ -z "$CLOUDRON_BIN" ] && [ -x "$HOME/.local/bin/cloudron" ]; then
|
||||
CLOUDRON_BIN="$HOME/.local/bin/cloudron"
|
||||
fi
|
||||
: "${CLOUDRON_BIN:?cloudron CLI not found (set CLOUDRON_BIN or add it to PATH)}"
|
||||
|
||||
# The cloudron CLI is a node script (`#!/usr/bin/env node`); under systemd the
|
||||
# PATH may not contain the host's node, so look in the usual places.
|
||||
if ! command -v node >/dev/null 2>&1; then
|
||||
for dir in "$HOME"/.hermes/tools/node*/bin /usr/local/bin; do
|
||||
if [ -x "$dir/node" ]; then export PATH="$dir:$PATH"; break; fi
|
||||
done
|
||||
fi
|
||||
|
||||
: "${CLOUDRON_TOKEN:?CLOUDRON_TOKEN is required (the Cloudron API token)}"
|
||||
|
||||
if [ ! -f "$SITE_DIR/index.html" ]; then
|
||||
echo "No build found at $SITE_DIR — run publishing/build.sh first." >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Build metadata: written into the output so a deploy can be verified from the
|
||||
# served site itself (which push produced it, and when).
|
||||
COMMIT="$(git -C "$REPO_DIR" rev-parse HEAD 2>/dev/null || echo unknown)"
|
||||
COMMIT_SHORT="$(git -C "$REPO_DIR" rev-parse --short HEAD 2>/dev/null || echo unknown)"
|
||||
BUILT_AT="$(date -u +%Y-%m-%dT%H:%M:%SZ)"
|
||||
cat > "$SITE_DIR/meta.json" <<EOF
|
||||
{
|
||||
"commit": "$COMMIT",
|
||||
"commit_short": "$COMMIT_SHORT",
|
||||
"built_at": "$BUILT_AT",
|
||||
"generator": "mkdocs-material (publishing/build.sh)",
|
||||
"source": "https://git.inference.coop/co-op/docs"
|
||||
}
|
||||
EOF
|
||||
|
||||
echo "Deploying $SITE_DIR (commit $COMMIT_SHORT) -> $SERVER:$APP"
|
||||
echo " $DEST_SITE (docs.inference.coop)"
|
||||
echo " $DEST_PATH (inference.coop/docs/)"
|
||||
|
||||
tar cf - -C "$SITE_DIR" . | \
|
||||
"$CLOUDRON_BIN" --server "$SERVER" --token "$CLOUDRON_TOKEN" exec --app "$APP" -- \
|
||||
sh -c "set -e
|
||||
cat > /tmp/docs-site.tar
|
||||
rm -rf '$DEST_SITE' && mkdir -p '$DEST_SITE' && tar xf /tmp/docs-site.tar -C '$DEST_SITE'
|
||||
rm -rf '$DEST_PATH' && mkdir -p '$DEST_PATH' && tar xf /tmp/docs-site.tar -C '$DEST_PATH'
|
||||
rm -f /tmp/docs-site.tar"
|
||||
|
||||
echo "Done:"
|
||||
echo " https://docs.inference.coop/"
|
||||
echo " https://inference.coop/docs/"
|
||||
@@ -0,0 +1,93 @@
|
||||
Copyright 2022 The Figtree Project Authors (https://github.com/erikdkennedy/figtree)
|
||||
|
||||
This Font Software is licensed under the SIL Open Font License, Version 1.1.
|
||||
This license is copied below, and is also available with a FAQ at:
|
||||
http://scripts.sil.org/OFL
|
||||
|
||||
|
||||
-----------------------------------------------------------
|
||||
SIL OPEN FONT LICENSE Version 1.1 - 26 February 2007
|
||||
-----------------------------------------------------------
|
||||
|
||||
PREAMBLE
|
||||
The goals of the Open Font License (OFL) are to stimulate worldwide
|
||||
development of collaborative font projects, to support the font creation
|
||||
efforts of academic and linguistic communities, and to provide a free and
|
||||
open framework in which fonts may be shared and improved in partnership
|
||||
with others.
|
||||
|
||||
The OFL allows the licensed fonts to be used, studied, modified and
|
||||
redistributed freely as long as they are not sold by themselves. The
|
||||
fonts, including any derivative works, can be bundled, embedded,
|
||||
redistributed and/or sold with any software provided that any reserved
|
||||
names are not used by derivative works. The fonts and derivatives,
|
||||
however, cannot be released under any other type of license. The
|
||||
requirement for fonts to remain under this license does not apply
|
||||
to any document created using the fonts or their derivatives.
|
||||
|
||||
DEFINITIONS
|
||||
"Font Software" refers to the set of files released by the Copyright
|
||||
Holder(s) under this license and clearly marked as such. This may
|
||||
include source files, build scripts and documentation.
|
||||
|
||||
"Reserved Font Name" refers to any names specified as such after the
|
||||
copyright statement(s).
|
||||
|
||||
"Original Version" refers to the collection of Font Software components as
|
||||
distributed by the Copyright Holder(s).
|
||||
|
||||
"Modified Version" refers to any derivative made by adding to, deleting,
|
||||
or substituting -- in part or in whole -- any of the components of the
|
||||
Original Version, by changing formats or by porting the Font Software to a
|
||||
new environment.
|
||||
|
||||
"Author" refers to any designer, engineer, programmer, technical
|
||||
writer or other person who contributed to the Font Software.
|
||||
|
||||
PERMISSION & CONDITIONS
|
||||
Permission is hereby granted, free of charge, to any person obtaining
|
||||
a copy of the Font Software, to use, study, copy, merge, embed, modify,
|
||||
redistribute, and sell modified and unmodified copies of the Font
|
||||
Software, subject to the following conditions:
|
||||
|
||||
1) Neither the Font Software nor any of its individual components,
|
||||
in Original or Modified Versions, may be sold by itself.
|
||||
|
||||
2) Original or Modified Versions of the Font Software may be bundled,
|
||||
redistributed and/or sold with any software, provided that each copy
|
||||
contains the above copyright notice and this license. These can be
|
||||
included either as stand-alone text files, human-readable headers or
|
||||
in the appropriate machine-readable metadata fields within text or
|
||||
binary files as long as those fields can be easily viewed by the user.
|
||||
|
||||
3) No Modified Version of the Font Software may use the Reserved Font
|
||||
Name(s) unless explicit written permission is granted by the corresponding
|
||||
Copyright Holder. This restriction only applies to the primary font name as
|
||||
presented to the users.
|
||||
|
||||
4) The name(s) of the Copyright Holder(s) or the Author(s) of the Font
|
||||
Software shall not be used to promote, endorse or advertise any
|
||||
Modified Version, except to acknowledge the contribution(s) of the
|
||||
Copyright Holder(s) and the Author(s) or with their explicit written
|
||||
permission.
|
||||
|
||||
5) The Font Software, modified or unmodified, in part or in whole,
|
||||
must be distributed entirely under this license, and must not be
|
||||
distributed under any other license. The requirement for fonts to
|
||||
remain under this license does not apply to any document created
|
||||
using the Font Software.
|
||||
|
||||
TERMINATION
|
||||
This license becomes null and void if any of the above conditions are
|
||||
not met.
|
||||
|
||||
DISCLAIMER
|
||||
THE FONT SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
|
||||
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTIES OF
|
||||
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT
|
||||
OF COPYRIGHT, PATENT, TRADEMARK, OR OTHER RIGHT. IN NO EVENT SHALL THE
|
||||
COPYRIGHT HOLDER BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
|
||||
INCLUDING ANY GENERAL, SPECIAL, INDIRECT, INCIDENTAL, OR CONSEQUENTIAL
|
||||
DAMAGES, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
|
||||
FROM, OUT OF THE USE OR INABILITY TO USE THE FONT SOFTWARE OR FROM
|
||||
OTHER DEALINGS IN THE FONT SOFTWARE.
|
||||
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
@@ -0,0 +1,25 @@
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 130 130">
|
||||
<rect width="130" height="130" rx="24" fill="#faf8f5"/>
|
||||
<circle cx="65" cy="14" r="5" fill="#5b8c5a"/>
|
||||
<circle cx="58" cy="30" r="6" fill="#7ab87a"/>
|
||||
<circle cx="74" cy="30" r="4" fill="#6ba86b"/>
|
||||
<circle cx="51.5" cy="46" r="6.5" fill="#5b8c5a"/>
|
||||
<circle cx="65" cy="46" r="5" fill="#7ab87a"/>
|
||||
<circle cx="81" cy="46" r="4" fill="#6ba86b"/>
|
||||
<circle cx="44" cy="62" r="7" fill="#6ba86b"/>
|
||||
<circle cx="61" cy="62" r="5.5" fill="#5b8c5a"/>
|
||||
<circle cx="76" cy="62" r="4.5" fill="#7ab87a"/>
|
||||
<circle cx="89.5" cy="62" r="3.5" fill="#6ba86b"/>
|
||||
<circle cx="36.5" cy="78" r="7.5" fill="#5b8c5a"/>
|
||||
<circle cx="54" cy="78" r="6" fill="#7ab87a"/>
|
||||
<circle cx="70" cy="78" r="5" fill="#6ba86b"/>
|
||||
<circle cx="84" cy="78" r="4" fill="#5b8c5a"/>
|
||||
<circle cx="97.5" cy="78" r="3.5" fill="#7ab87a"/>
|
||||
<circle cx="44" cy="94" r="6.5" fill="#6ba86b"/>
|
||||
<circle cx="61" cy="94" r="5.5" fill="#5b8c5a"/>
|
||||
<circle cx="76" cy="94" r="4.5" fill="#7ab87a"/>
|
||||
<circle cx="89.5" cy="94" r="3.5" fill="#6ba86b"/>
|
||||
<circle cx="51.5" cy="110" r="5.5" fill="#5b8c5a"/>
|
||||
<circle cx="65" cy="110" r="4.5" fill="#7ab87a"/>
|
||||
<circle cx="81" cy="110" r="3.5" fill="#6ba86b"/>
|
||||
</svg>
|
||||
|
After Width: | Height: | Size: 1.2 KiB |
@@ -0,0 +1,3 @@
|
||||
# Pinned build toolchain for the documentation site.
|
||||
# Reproducible: `pip install -r publishing/requirements.txt && mkdocs build`
|
||||
mkdocs-material==9.8.0
|
||||
@@ -0,0 +1,225 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Gitea webhook receiver that rebuilds and republishes the docs site.
|
||||
|
||||
Flow: a push to main in co-op/docs fires a Gitea webhook -> this receiver
|
||||
verifies it -> pulls the latest main -> runs publishing/build.sh ->
|
||||
publishing/deploy.sh (which writes the build into the Surfer app).
|
||||
|
||||
Runs on the agent host (no new Cloudron app). Standard library only.
|
||||
|
||||
Security model:
|
||||
* Only POSTs from the Cloudron box (git.inference.coop's host) are accepted.
|
||||
* The payload must carry a valid HMAC-SHA256 signature (X-Gitea-Signature,
|
||||
shared secret DEPLOY_WEBHOOK_SECRET). The secret itself is never sent.
|
||||
* Only pushes to refs/heads/main of co-op/docs trigger a deploy.
|
||||
* The receiver binds 0.0.0.0; an iptables rule installed by the systemd unit
|
||||
narrows the port to the Cloudron box as well (defence in depth).
|
||||
|
||||
Secrets are read from /home/ubuntu/.hermes/.env and are never logged.
|
||||
"""
|
||||
|
||||
import fcntl
|
||||
import hashlib
|
||||
import hmac
|
||||
import ipaddress
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
import subprocess
|
||||
import sys
|
||||
import threading
|
||||
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
|
||||
|
||||
REPO_DIR = os.environ.get("DOCS_REPO_DIR", "/home/ubuntu/repos/docs")
|
||||
ENV_FILE = os.environ.get("DOCS_ENV_FILE", "/home/ubuntu/.hermes/.env")
|
||||
LOG_FILE = os.environ.get("DOCS_DEPLOY_LOG", "/home/ubuntu/docs-deploy.log")
|
||||
LOCK_FILE = os.environ.get("DOCS_DEPLOY_LOCK", "/home/ubuntu/.docs-deploy.lock")
|
||||
PORT = int(os.environ.get("DOCS_WEBHOOK_PORT", "8787"))
|
||||
EXPECT_REPO = "co-op/docs"
|
||||
EXPECT_BRANCH = "refs/heads/main"
|
||||
ALLOWED_SOURCES = [
|
||||
ipaddress.ip_network(n)
|
||||
for n in os.environ.get(
|
||||
"DOCS_WEBHOOK_SOURCES",
|
||||
# The Cloudron box (hosts git.inference.coop) plus loopback for local tests.
|
||||
"179.237.86.116/32,2001:1600:18:206::89/128,127.0.0.1/32,::1/128",
|
||||
).split(",")
|
||||
if n.strip()
|
||||
]
|
||||
|
||||
log = logging.getLogger("docs-deploy")
|
||||
deploy_lock_file = open(LOCK_FILE, "a")
|
||||
|
||||
|
||||
def load_env_file(path):
|
||||
"""Return KEY=VALUE pairs from the agent host's env file (values stay local)."""
|
||||
values = {}
|
||||
try:
|
||||
with open(path) as handle:
|
||||
for line in handle:
|
||||
line = line.strip()
|
||||
if not line or line.startswith("#") or "=" not in line:
|
||||
continue
|
||||
key, _, value = line.partition("=")
|
||||
values[key.strip()] = value.strip().strip('"').strip("'")
|
||||
except OSError:
|
||||
pass
|
||||
return values
|
||||
|
||||
|
||||
_FILE_ENV = load_env_file(ENV_FILE)
|
||||
|
||||
|
||||
def secret(name):
|
||||
return os.environ.get(name) or _FILE_ENV.get(name) or ""
|
||||
|
||||
|
||||
def source_allowed(address):
|
||||
try:
|
||||
ip = ipaddress.ip_address(address.split("%")[0])
|
||||
except ValueError:
|
||||
return False
|
||||
return any(ip in net for net in ALLOWED_SOURCES)
|
||||
|
||||
|
||||
def run(cmd, timeout, cwd=None):
|
||||
log.info("exec: %s", " ".join(cmd))
|
||||
# Hand the build/deploy scripts the credentials they need (from the env
|
||||
# file, if not already set). Values are passed by environment only and are
|
||||
# never logged.
|
||||
env = dict(os.environ)
|
||||
for key in ("CLOUDRON_TOKEN", "CLOUDRON_SERVER", "SURFER_APP", "GIT_ASKPASS"):
|
||||
if not env.get(key) and secret(key):
|
||||
env[key] = secret(key)
|
||||
env.setdefault("GIT_ASKPASS", "/home/ubuntu/.hermes/scripts/git_askpass.sh")
|
||||
env.setdefault("PATH", os.environ.get("PATH", ""))
|
||||
result = subprocess.run(
|
||||
cmd, cwd=cwd or REPO_DIR, capture_output=True, text=True, timeout=timeout,
|
||||
env=env,
|
||||
)
|
||||
if result.stdout.strip():
|
||||
log.info("stdout: %s", result.stdout.strip()[-2000:])
|
||||
if result.returncode != 0 and result.stderr.strip():
|
||||
log.error("stderr: %s", result.stderr.strip()[-2000:])
|
||||
return result
|
||||
|
||||
|
||||
def deploy(commit_hint=""):
|
||||
"""Pull main, rebuild, republish. Returns (ok, log_tail)."""
|
||||
try:
|
||||
with open("/dev/null") as devnull:
|
||||
fcntl.flock(deploy_lock_file, fcntl.LOCK_EX | fcntl.LOCK_NB)
|
||||
except BlockingIOError:
|
||||
return False, "another deploy is already running; skipped"
|
||||
|
||||
tail = []
|
||||
try:
|
||||
steps = [
|
||||
(["git", "pull", "--ff-only", "origin", "main"], 120),
|
||||
(["bash", "publishing/build.sh"], 900),
|
||||
(["bash", "publishing/deploy.sh"], 600),
|
||||
]
|
||||
for cmd, timeout in steps:
|
||||
result = run(cmd, timeout)
|
||||
out = (result.stdout or "") + (result.stderr or "")
|
||||
tail.append(f"$ {' '.join(cmd)} -> exit {result.returncode}")
|
||||
tail.extend(out.strip().splitlines()[-12:])
|
||||
if result.returncode != 0:
|
||||
log.error("step failed: %s", " ".join(cmd))
|
||||
return False, "\n".join(tail)
|
||||
log.info("deploy complete%s", f" ({commit_hint})" if commit_hint else "")
|
||||
return True, "\n".join(tail)
|
||||
except Exception as exc: # noqa: BLE001 - report, never crash the receiver
|
||||
log.exception("deploy failed")
|
||||
tail.append(f"error: {exc}")
|
||||
return False, "\n".join(tail)
|
||||
|
||||
|
||||
class Handler(BaseHTTPRequestHandler):
|
||||
server_version = "docs-deploy/1.0"
|
||||
|
||||
def log_message(self, format, *args): # noqa: A002 - stdlib signature
|
||||
log.info("%s - %s", self.client_address[0], format % args)
|
||||
|
||||
def _json(self, code, payload):
|
||||
body = json.dumps(payload).encode()
|
||||
self.send_response(code)
|
||||
self.send_header("Content-Type", "application/json")
|
||||
self.send_header("Content-Length", str(len(body)))
|
||||
self.end_headers()
|
||||
self.wfile.write(body)
|
||||
|
||||
def do_GET(self):
|
||||
if self.path == "/healthz":
|
||||
return self._json(200, {"status": "ok"})
|
||||
return self._json(404, {"error": "not found"})
|
||||
|
||||
def do_POST(self):
|
||||
peer = self.client_address[0]
|
||||
if not source_allowed(peer):
|
||||
log.warning("rejected: source %s not allowed", peer)
|
||||
return self._json(403, {"error": "forbidden"})
|
||||
|
||||
if self.headers.get("X-Gitea-Event") != "push":
|
||||
return self._json(400, {"error": "not a push event"})
|
||||
|
||||
length = int(self.headers.get("Content-Length") or 0)
|
||||
if length <= 0 or length > 1_000_000:
|
||||
return self._json(400, {"error": "bad payload size"})
|
||||
body = self.rfile.read(length)
|
||||
|
||||
signature = self.headers.get("X-Gitea-Signature") or ""
|
||||
expected = hmac.new(
|
||||
secret("DEPLOY_WEBHOOK_SECRET").encode(), body, hashlib.sha256
|
||||
).hexdigest()
|
||||
if not signature or not hmac.compare_digest(signature, expected):
|
||||
log.warning("rejected: bad signature from %s", peer)
|
||||
return self._json(403, {"error": "invalid signature"})
|
||||
|
||||
try:
|
||||
payload = json.loads(body)
|
||||
except json.JSONDecodeError:
|
||||
return self._json(400, {"error": "invalid json"})
|
||||
|
||||
repo = (payload.get("repository") or {}).get("full_name", "")
|
||||
ref = payload.get("ref", "")
|
||||
commit = (payload.get("after") or "")[:12]
|
||||
pusher = ((payload.get("pusher") or {}).get("login")) or "unknown"
|
||||
if repo != EXPECT_REPO or ref != EXPECT_BRANCH:
|
||||
log.info("ignored: push to %s %s (not %s %s)", repo, ref, EXPECT_REPO, EXPECT_BRANCH)
|
||||
return self._json(200, {"status": "ignored", "reason": "not main on co-op/docs"})
|
||||
|
||||
log.info("push received: %s %s commit=%s pusher=%s", repo, ref, commit, pusher)
|
||||
threading.Thread(target=self._run_deploy, args=(commit,), daemon=True).start()
|
||||
return self._json(202, {"status": "accepted", "commit": commit})
|
||||
|
||||
def _run_deploy(self, commit):
|
||||
ok, tail = deploy(commit)
|
||||
log.info("deploy result for %s: %s", commit, "ok" if ok else "FAILED")
|
||||
log.info("tail:\n%s", tail[-1500:])
|
||||
|
||||
|
||||
def main():
|
||||
logging.basicConfig(
|
||||
level=logging.INFO,
|
||||
format="%(asctime)s %(levelname)s %(message)s",
|
||||
handlers=[
|
||||
logging.FileHandler(LOG_FILE),
|
||||
logging.StreamHandler(sys.stdout),
|
||||
],
|
||||
)
|
||||
if not secret("DEPLOY_WEBHOOK_SECRET"):
|
||||
log.error("DEPLOY_WEBHOOK_SECRET is not set; refusing to start")
|
||||
sys.exit(1)
|
||||
if not secret("CLOUDRON_TOKEN"):
|
||||
log.warning("CLOUDRON_TOKEN not set; deploys will fail")
|
||||
server = ThreadingHTTPServer(("0.0.0.0", PORT), Handler)
|
||||
log.info("listening on 0.0.0.0:%d, repo %s", PORT, REPO_DIR)
|
||||
try:
|
||||
server.serve_forever()
|
||||
except KeyboardInterrupt:
|
||||
pass
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,28 @@
|
||||
# systemd unit for the docs deploy webhook receiver (agent host).
|
||||
#
|
||||
# Install:
|
||||
# sudo cp publishing/webhook-receiver.service /etc/systemd/system/
|
||||
# sudo systemctl daemon-reload && sudo systemctl enable --now docs-deploy-webhook
|
||||
#
|
||||
# The ExecStartPre lines narrow the webhook port to the Cloudron box (which
|
||||
# hosts git.inference.coop); the receiver additionally checks the source address
|
||||
# and the payload signature, so a scanner hitting the port gets nothing.
|
||||
|
||||
[Unit]
|
||||
Description=Inference docs: Gitea webhook -> build -> Surfer deploy
|
||||
After=network-online.target
|
||||
Wants=network-online.target
|
||||
|
||||
[Service]
|
||||
User=ubuntu
|
||||
WorkingDirectory=/home/ubuntu/repos/docs
|
||||
ExecStartPre=+/bin/sh -c '/usr/sbin/iptables -C INPUT -i lo -p tcp --dport 8787 -j ACCEPT 2>/dev/null || /usr/sbin/iptables -I INPUT 1 -i lo -p tcp --dport 8787 -j ACCEPT'
|
||||
ExecStartPre=+/bin/sh -c '/usr/sbin/iptables -C INPUT -p tcp --dport 8787 -s 179.237.86.116 -j ACCEPT 2>/dev/null || /usr/sbin/iptables -I INPUT 2 -p tcp --dport 8787 -s 179.237.86.116 -j ACCEPT'
|
||||
ExecStartPre=+/bin/sh -c '/usr/sbin/iptables -C INPUT -p tcp --dport 8787 ! -s 179.237.86.116 -j DROP 2>/dev/null || /usr/sbin/iptables -I INPUT 3 -p tcp --dport 8787 ! -s 179.237.86.116 -j DROP'
|
||||
ExecStartPre=+/bin/sh -c '/usr/sbin/ip6tables -C INPUT -p tcp --dport 8787 -j DROP 2>/dev/null || /usr/sbin/ip6tables -I INPUT -p tcp --dport 8787 -j DROP'
|
||||
ExecStart=/home/ubuntu/repos/docs/.venv-docs/bin/python /home/ubuntu/repos/docs/publishing/webhook-receiver.py
|
||||
Restart=on-failure
|
||||
RestartSec=5
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
Reference in new issue
Block a user