Compare commits
37
Commits
b4ab820486
..
toybox
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
e5ed407e55 | ||
|
|
a984461cee | ||
|
|
8025ec4b20 | ||
|
|
f64dac4434 | ||
|
|
193a126d1c | ||
|
|
28b1c3c9d3 | ||
|
|
7fcd251261 | ||
|
|
9356f63c64 | ||
|
|
8d8900d84d | ||
|
|
f0557d29d6 | ||
|
|
2820c015ee | ||
|
|
7bf6eb44c7 | ||
|
|
88739d9495 | ||
|
|
1d8a220bbc | ||
|
|
68132ff4b3 | ||
|
|
9c09e8c5a4 | ||
|
|
94a059af25 | ||
|
|
be796d9d0d | ||
|
|
c66aaea366 | ||
|
|
a25bd699e1 | ||
|
|
ea3e256bb8 | ||
|
|
0af7a3ea6c | ||
|
|
9719fd0aa8 | ||
|
|
bce2424cc0 | ||
|
|
7f1e19fe85 | ||
|
|
8a812d76e3 | ||
|
|
002f08b456 | ||
|
|
efb54ad4cc | ||
|
|
8c110098ff | ||
|
|
d61cb8c9a1 | ||
|
|
a2ca59a27e | ||
|
|
4a85143a66 | ||
|
|
37faa92649 | ||
|
|
ec24f3e8e4 | ||
|
|
47a2337afe | ||
|
|
9b12943be4 | ||
|
|
9a3e7348c6 |
No files matched your search
+19
-34
@@ -23,7 +23,7 @@ narrative.
|
|||||||
goal 5.
|
goal 5.
|
||||||
|
|
||||||
- [x] **1c. diigo-dump evolves into a bookmark cache**
|
- [x] **1c. diigo-dump evolves into a bookmark cache**
|
||||||
([diigo-cache/README.md](diigo-cache/README.md); design in the
|
([diigo-cache/CONTRIBUTING.md](diigo-cache/CONTRIBUTING.md); design in the
|
||||||
JSDoc of `src/cache.ts`, `src/index.ts`, `src/pm-save.ts`):
|
JSDoc of `src/cache.ts`, `src/index.ts`, `src/pm-save.ts`):
|
||||||
- [x] 1c-i. Two independent page caches, private and public
|
- [x] 1c-i. Two independent page caches, private and public
|
||||||
(`PagedCache`, `DiigoCache`, `PublicCache`).
|
(`PagedCache`, `DiigoCache`, `PublicCache`).
|
||||||
@@ -42,20 +42,16 @@ narrative.
|
|||||||
2026-10-07, via `scripts/compare-export.ts`. Getting there
|
2026-10-07, via `scripts/compare-export.ts`. Getting there
|
||||||
found diigo's `page_num` is 0-based (schema v2 migration).
|
found diigo's `page_num` is 0-based (schema v2 migration).
|
||||||
|
|
||||||
- [ ] **1d. Date windows and search on the public page**, toward the
|
- [x] **1d. Date windows and search on the public page**, toward the
|
||||||
weekly PESOS roundup of [#23](https://github.com/dckc/madmode-blog/issues/23)
|
weekly PESOS roundup of [#23](https://github.com/dckc/madmode-blog/issues/23)
|
||||||
(a week's permalink: `in=2026-W41`). One filter, two spellings:
|
(a week's permalink: `in=2026-W41`). One filter, two spellings:
|
||||||
devs pass `{ days: 7, q: "#weekly" }`, URLs the same fields;
|
devs pass `{ days: 7, q: "#weekly" }`, URLs the same fields;
|
||||||
`q` takes gmail/Slack/GitHub operators. UTC, ISO weeks, half-open
|
`q` takes gmail/Slack/GitHub operators. UTC, ISO weeks, half-open
|
||||||
windows; a fetch budget per request. Design: the API in
|
windows (`diigo-cache/src/query.ts`; `tests/query.test.ts`,
|
||||||
`diigo-cache/src/query.ts` and the skipped suites in
|
`tests/caches.test.ts` "public page filters"). Filters read
|
||||||
`tests/query.test.ts` and `tests/caches.test.ts` ("public page
|
stored `items` only (the public page never fetches): results are
|
||||||
filters"); done when they're unskipped and green.
|
as fresh as the cache, and say `complete` when every diigo page
|
||||||
Reconcile with 1f first: `query.ts` plans to filter while walking
|
is stored. Keeping them fresh is 1f. 2026-10-09.
|
||||||
diigo's pages (binary search by `updated`), but after 1c-vi a
|
|
||||||
filled cache holds every item, so 1d can query `items` in sqlite
|
|
||||||
and spend diigo fetches only on a head refresh (1f). Choose
|
|
||||||
before implementing; the skipped suites may need adjusting.
|
|
||||||
|
|
||||||
- [x] **1e. Fill job: a button that grabs every page** into a cache
|
- [x] **1e. Fill job: a button that grabs every page** into a cache
|
||||||
from the webkey page `/cache/#<key>`, which trades the key for a
|
from the webkey page `/cache/#<key>`, which trades the key for a
|
||||||
@@ -93,31 +89,20 @@ narrative.
|
|||||||
[the issues sketch](potluck/issues-sketch.md))? If the policy
|
[the issues sketch](potluck/issues-sketch.md))? If the policy
|
||||||
changes, stripping stored rows is a durable-state migration.
|
changes, stripping stored rows is a durable-state migration.
|
||||||
|
|
||||||
- [ ] **2. madmode reads the public cache**: public data, so no
|
- [ ] **2. potluck reads the public cache**: public data, so no capability, no cookie, no secret on potluck's side. The [diigo-week](diigo-week/README.md) plugin reads `GET /public/dckc-madmode?in=<week>&by=created` as microformats (an `h-feed` of `h-entry` bookmarks). Working in dev 2026-10-09; 2-iii postponed.
|
||||||
capability, no cookie, no plugin sync job, no cron on madmode's
|
- [x] 2-i. Transport: the `DIIGO_CACHE` service binding in potluck's `wrangler.jsonc`, in dev too: the plugin's `ctx.http.fetch` refuses localhost (EmDash's SSRF guard), and a binding names one worker, not a host.
|
||||||
side. madmode fetches `GET /public/dckc-madmode` with
|
- [x] 2-ii. The page's microformats are the contract: diigo-cache's tests parse it with microformats-parser; diigo-week's tests read fixtures in the same markup.
|
||||||
`Accept: application/json` (plus 1d's `in=`/`days=`/`q=`).
|
- [ ] 2-iii. Decide `DiigoBridge`'s fate (postponed: nothing on this path uses it; it matters only for a caller that reads private items).
|
||||||
- [ ] 2-i. Transport: a service binding to diigo-cache when both
|
- [x] 2-iv. Run it in dev: diigo-cache and potluck both up; the dashboard card → a draft issue of 22 bookmarks for 2026-W40 (2026-10-09). The service binding resolves across the two dev servers.
|
||||||
are deployed (no public hop, so no rate-limit exposure);
|
- [ ] ~~**3. `/bookmarks` renders real data**~~: superseded by potluck's issues (the madmode starter is gone).
|
||||||
in dev, a base URL (`http://localhost:8787`).
|
- [ ] **4. The weekly draft**: decided (2026-10-09): a draft issue you edit before publishing, made by diigo-week. The slug comes from the title (`week-2026-w40`).
|
||||||
- [ ] 2-ii. The JSON shape of the public page is a contract
|
|
||||||
between two workers: pin it in a diigo-cache test (and
|
- [ ] **4b. Consider scruffier tags.** EmDash tags are taxonomy terms (rows that must exist before use); diigo-week auto-creates them, so drafting feels scruffy, but the model is neat. Alternative: the inline `/tag/<tag>` links in issue bodies *are* the tags (matching the microformats `p-category` reading), and potluck's `/tag/<tag>` page finds issues by scanning bodies. Cost: no EmDash tag field or index; a scan over a few dozen issues a year is cheap. Open: does the admin's tag field let you type a new tag inline?
|
||||||
madmode's reader tests against the same fixture).
|
|
||||||
- [ ] 2-iii. Decide `DiigoBridge`'s fate: madmode doesn't need it
|
|
||||||
(public reads need no cache id), so its "cache-id source"
|
|
||||||
TODO (`src/index.ts`, README "RPC use") is either dropped
|
|
||||||
with the entrypoint or kept for a private-reading caller.
|
|
||||||
- [ ] **3. `/bookmarks` renders real data** — titles, tags, dates at
|
|
||||||
http://localhost:4321/bookmarks, newest first.
|
|
||||||
- [ ] **4. The weekly page**: `/bookmarks/2026-W41` → the cache's
|
|
||||||
`in=2026-W41` (needs 1d and 1f). Decide (user's call): a raw
|
|
||||||
week list, or a draft post the user edits before publishing
|
|
||||||
(the PESOS practice is editorial; a tag like `#weekly` in `q`
|
|
||||||
is one cheap form of discretion).
|
|
||||||
- [ ] **5. Deployed end-to-end** (order matters):
|
- [ ] **5. Deployed end-to-end** (order matters):
|
||||||
- [ ] 5-i. webapp: `cd webapp && npx wrangler deploy`
|
- [ ] 5-i. webapp: `cd webapp && npx wrangler deploy`
|
||||||
- [ ] 5-ii. diigo-cache: rate-limit `/api` (and `/public/*`) first
|
- [ ] 5-ii. diigo-cache: rate-limit `/api` (and `/public/*`) first
|
||||||
(diigo-cache README, "Deployment concern"); deploy; create
|
(diigo-cache CONTRIBUTING, "Deployment concern"); deploy; create
|
||||||
against prod (ids → password manager; no worker secret);
|
against prod (ids → password manager; no worker secret);
|
||||||
run the fill job.
|
run the fill job.
|
||||||
- [ ] 5-iii. madmode: deploy with the service binding; the
|
- [ ] 5-iii. madmode: deploy with the service binding; the
|
||||||
@@ -239,5 +224,5 @@ to every goal in this tree.
|
|||||||
Oct 6 2026)
|
Oct 6 2026)
|
||||||
- [webapp/README.md](webapp/README.md) — HTTP + RPC surfaces of the
|
- [webapp/README.md](webapp/README.md) — HTTP + RPC surfaces of the
|
||||||
factory worker; open TODO on the mint-form placement
|
factory worker; open TODO on the mint-form placement
|
||||||
- [diigo-cache/README.md](diigo-cache/README.md) — the cache worker:
|
- [diigo-cache/CONTRIBUTING.md](diigo-cache/CONTRIBUTING.md) — the cache worker:
|
||||||
create flow, the two DOs, run/deploy details
|
create flow, the two DOs, run/deploy details
|
||||||
Symlink
+1
@@ -0,0 +1 @@
|
|||||||
|
CONTRIBUTING.md
|
||||||
@@ -0,0 +1,72 @@
|
|||||||
|
# Contributing to diigo-cache
|
||||||
|
|
||||||
|
diigo-cache is a Cloudflare Worker cache over [the webapp factory](../webapp/README.md): session-cookie reads of diigo bookmarks, cached in Durable-Object SQLite storage. Evolved from the diigo-dump worker (provenance: [mm-diigo `projects/diigo-bak/`](https://github.com/dckc/mm-diigo)); the static API-key path is dead (see [the capsite checklist](../capsite-checklist.md), goal 2 preamble).
|
||||||
|
|
||||||
|
## What it holds
|
||||||
|
|
||||||
|
- **The create flow, on one port**: `/_connect` renders webapp's own mint form (diigo-prefilled, cookie blank) — wrangler multi-config dev doesn't serve aux workers' HTTP, so `/_mint` relays the form opaquely into the WEBAPP binding (the cookie crosses; nothing on this side opens or logs it). Mint → `/_wire` (webapp id pre-filled, mode public|private) → the created page, where the user saves the ids.
|
||||||
|
- **Two independent page caches** (`src/cache.ts`), one behavior: a page hit is served from storage with no diigo call; a miss asks the webapp instance in the cache's wiring and etches the page.
|
||||||
|
- `DiigoCache` (binding `CACHE_DO`): every item. Addressed by its cache id.
|
||||||
|
- `PublicCache` (binding `PUBLIC_DO`): shared items only — private rows are dropped *before* etching (`publicOnly`), so its storage holds nothing private. Same cookie webapp as the private cache. Public page n is diigo page n, filtered (may show fewer than `PAGE_SIZE`, 100: diigo's cap). `/public/<username>` resolves through the `idFromName("index")` registry, and reads through `viewer()`, a read-only facet: it shows stored pages and fill progress, and can't fetch from diigo or write.
|
||||||
|
- **Storage**: SQL lives in `src/cache-read.sql` (selects only) and `src/cache-write.sql`; `src/store.ts` turns them into a reader and a writer facet, so the cache logic holds no SQL text.
|
||||||
|
- **Pages**: home's dump form (cache id + page number, POST) shows raw dump rows; `/public/<username>?page=n` shows an HTML list with a page form (JSON for `Accept: application/json`). Both say `cache: HIT` or `MISS`; every page read also logs `{event: "cache", kind, page, outcome}`.
|
||||||
|
- **Filters on the public page** (`src/query.ts`): `days`, `after`, `before`, `in` (`2026`, `2026-03`, `2026-Q1`, `2026-W41`), `by=updated|created`, and `q` (words, `"phrases"`, `#tag`, `-` negation, gmail/Slack/GitHub date operators). UTC, half-open windows. The same fields work as a dev-side spec (`toQuery({ days: 7 })`). A filter queries the stored `items` table through `viewer()`, never diigo: results are as fresh as the cache, and JSON says `complete` when every diigo page is stored.
|
||||||
|
- **Publish later**: `POST /_publish` (cache id in the body) wires a public companion to a private-only cache.
|
||||||
|
- **The webkey page and Cap'n Web**: `/cache/#<cache id>` is a skeleton page, the same for every key; its script reads the key from the fragment and trades it in-band for a capability over [Cap'n Web](https://github.com/cloudflare/capnweb) at `/api` (`CacheApi.cache(key)` → a `CacheCap` facet: fill, status, the public companion; `src/api.ts`). The worker serves capnweb's browser build itself at `/_capnweb.js`.
|
||||||
|
- **Fill job**: walks every diigo page into a cache in the background — DO alarms, a few pages per step, progress stored in the DO (`startFill`/`fillStatus`/`stopFill`), driven from the webkey page; the public page shows progress. "Invalidate the last N days" there drops the cached pages holding anything updated that recently (`invalidateRecent`), so the next fill refetches them.
|
||||||
|
- The `DiigoBridge` entrypoint for other workers (its cache-id source is the goal-2 wiring TODO).
|
||||||
|
|
||||||
|
## State surface
|
||||||
|
|
||||||
|
Ids are **webkeys** ([the pattern](../docs/webkeys.md)): 256-bit bearer secrets that never ride a query string or path. The webkey page carries the cache id in the URL fragment (never sent to the server) and passes it in-band over Cap'n Web; forms carry ids in POST bodies; logs and pages show only a fingerprint (a prefix of the id's one-way hash).
|
||||||
|
|
||||||
|
- **The cache id**: the cache's read capability (bearer; not logged). Held by the user — saved to the password manager under a petname they choose (the username field; never sent to the server) — and passed per POST.
|
||||||
|
- **The webapp instance id**: the diigo-read capability; lives in each cache DO's wiring, arrives from the mint step.
|
||||||
|
- **The cookie**: lives **only in webapp's DO storage**; crosses the wire once, at mint time, as an opaque relay.
|
||||||
|
- **The public cache**: no capability — `/public/<username>` is public by design (a guessable URL; hiding the link doesn't hide the page).
|
||||||
|
|
||||||
|
Stored state is meant to be durable: see the breaking-change rule in [capsite AGENTS.md](../AGENTS.md).
|
||||||
|
|
||||||
|
## Deployment concern: rate-limit `/api`
|
||||||
|
|
||||||
|
Public pages only read storage, so strangers can't make the cache spend your diigo session. `/api` is keyed (a cache id is needed), but Cap'n Web pipelining makes calls cheap to send. Before exposing a deployment:
|
||||||
|
|
||||||
|
- rate-limit `/api` (and `/public/*`, which reads storage) — on Cloudflare, the [Workers rate-limit binding](https://developers.cloudflare.com/workers/runtime-apis/bindings/rate-limit/) (keyed by client and/or username); on other workerd hosts, a rate limit in the proxy in front;
|
||||||
|
- run the fill job (goal 1e): public pages show only what's stored.
|
||||||
|
|
||||||
|
## RPC use (from another worker, same account)
|
||||||
|
|
||||||
|
```jsonc
|
||||||
|
// wrangler.jsonc of the caller
|
||||||
|
"services": [{ "binding": "DIIGO", "service": "diigo-cache", "entrypoint": "DiigoBridge" }]
|
||||||
|
```
|
||||||
|
|
||||||
|
```ts
|
||||||
|
// cache ids ride goal-2's wiring; see the capsite checklist
|
||||||
|
const ndjson = await env.DIIGO.dumpNdjson(4); // 4 pages of PAGE_SIZE
|
||||||
|
```
|
||||||
|
|
||||||
|
## Test
|
||||||
|
|
||||||
|
pnpm types # regenerate worker-configuration.d.ts (after wrangler.jsonc changes)
|
||||||
|
pnpm lint # fails if those types are stale; tsc over src/ and tests/
|
||||||
|
pnpm test # vitest
|
||||||
|
|
||||||
|
Tests run the real worker and both DOs in the vitest pool, with `WEBAPP` bound to [a fake webapp](tests/fake-webapp.js) serving diigo-shaped items (`wrangler.tests.jsonc`).
|
||||||
|
|
||||||
|
## Run
|
||||||
|
|
||||||
|
[The README](README.md) gets a cache going in dev. The same create flow from a terminal:
|
||||||
|
|
||||||
|
wid=$(curl -s -X POST http://localhost:8787/_mint --data-urlencode \
|
||||||
|
'url=https://www.diigo.com/interact_api/' --data-urlencode \
|
||||||
|
'headers={"user-agent":"Mozilla/5.0","x-requested-with":"XMLHttpRequest","cookie":"<paste>"}' \
|
||||||
|
--data-urlencode 'msPerCall=500' -D - -o /dev/null | grep -i x-webapp-id)
|
||||||
|
curl -s -X POST http://localhost:8787/_wire -H 'content-type: application/json' \
|
||||||
|
-d "{\"webappId\":\"<from $wid>\",\"mode\":\"private\"}"
|
||||||
|
|
||||||
|
Check a filled cache against a diigo RSS export: `node scripts/compare-export.ts <cache.sqlite> <export.xml> <out-dir>`.
|
||||||
|
|
||||||
|
## Deploy
|
||||||
|
|
||||||
|
Not deployed yet; the order matters (goal 5 in [the capsite checklist](../capsite-checklist.md)): webapp first (`cd ../webapp && npx wrangler deploy`), then diigo-cache, after rate-limiting `/api` (above).
|
||||||
+16
-110
@@ -1,120 +1,26 @@
|
|||||||
# diigo-cache
|
# diigo-cache
|
||||||
|
|
||||||
Cloudflare Worker cache over [the webapp factory](../webapp/README.md):
|
Your diigo bookmarks, cached in a [workerd](https://github.com/cloudflare/workerd) worker, with a public page you can filter by week: `/public/<you>?in=2026-W41`. Your diigo session cookie is stored in one Durable Object and nowhere else.
|
||||||
session-cookie reads of diigo bookmarks, cached in Durable-Object SQLite
|
|
||||||
storage. Evolved from the diigo-dump worker (provenance:
|
|
||||||
[mm-diigo `projects/diigo-bak/`](https://github.com/dckc/mm-diigo)); the
|
|
||||||
static API-key path is dead (see [the capsite checklist](../capsite-checklist.md),
|
|
||||||
goal 2 preamble).
|
|
||||||
|
|
||||||
## What it holds
|
## Getting started
|
||||||
|
|
||||||
- **The create flow, on one port**: `/_connect` renders webapp's own
|
```sh
|
||||||
mint form (diigo-prefilled, cookie blank) — wrangler multi-config dev
|
pnpm install
|
||||||
doesn't serve aux workers' HTTP, so `/_mint` relays the form opaquely
|
pnpm dev # → http://localhost:8787/
|
||||||
into the WEBAPP binding (the cookie crosses; nothing on this side opens
|
|
||||||
or logs it). Mint → `/_wire` (webapp id pre-filled, mode
|
|
||||||
public|private) → the created page, where the user saves the ids.
|
|
||||||
- **Two independent page caches** (`src/cache.ts`), one behavior: a
|
|
||||||
page hit is served from storage with no diigo call; a miss asks the
|
|
||||||
webapp instance in the cache's wiring and etches the page.
|
|
||||||
- `DiigoCache` (binding `CACHE_DO`): every item. Addressed by its
|
|
||||||
cache id.
|
|
||||||
- `PublicCache` (binding `PUBLIC_DO`): shared items only — private
|
|
||||||
rows are dropped *before* etching (`publicOnly`), so its storage
|
|
||||||
holds nothing private. Same cookie webapp as the private cache.
|
|
||||||
Public page n is diigo page n, filtered (may show fewer than
|
|
||||||
`PAGE_SIZE`, 100: diigo's cap).
|
|
||||||
`/public/<username>` resolves through the `idFromName("index")`
|
|
||||||
registry, and reads through `viewer()`, a read-only facet: it shows
|
|
||||||
stored pages and fill progress, and can't fetch from diigo or write.
|
|
||||||
- **Storage**: SQL lives in `src/cache-read.sql` (selects only) and
|
|
||||||
`src/cache-write.sql`; `src/store.ts` turns them into a reader and a
|
|
||||||
writer facet, so the cache logic holds no SQL text.
|
|
||||||
- **Pages**: home's dump form (cache id + page number, POST) shows raw
|
|
||||||
dump rows; `/public/<username>?page=n` shows an HTML list with a page
|
|
||||||
form (JSON for `Accept: application/json`). Both say `cache: HIT` or
|
|
||||||
`MISS`; every page read also logs `{event: "cache", kind, page,
|
|
||||||
outcome}`.
|
|
||||||
- **Publish later**: `POST /_publish` (cache id in the body) wires a
|
|
||||||
public companion to a private-only cache.
|
|
||||||
- **The webkey page and Cap'n Web**: `/cache/#<cache id>` is a skeleton
|
|
||||||
page, the same for every key; its script reads the key from the
|
|
||||||
fragment and trades it in-band for a capability over
|
|
||||||
[Cap'n Web](https://github.com/cloudflare/capnweb) at `/api`
|
|
||||||
(`CacheApi.cache(key)` → a `CacheCap` facet: fill, status, the public
|
|
||||||
companion; `src/api.ts`). The worker serves capnweb's browser build
|
|
||||||
itself at `/_capnweb.js`.
|
|
||||||
- **Fill job**: walks every diigo page into a cache in the background —
|
|
||||||
DO alarms, a few pages per step, progress stored in the DO
|
|
||||||
(`startFill`/`fillStatus`/`stopFill`), driven from the webkey page;
|
|
||||||
the public page shows progress. "Invalidate the last N days" there
|
|
||||||
drops the cached pages holding anything updated that recently
|
|
||||||
(`invalidateRecent`), so the next fill refetches them.
|
|
||||||
- The `DiigoBridge` entrypoint for other workers (its cache-id source is
|
|
||||||
the goal-2 wiring TODO).
|
|
||||||
|
|
||||||
## State surface
|
|
||||||
|
|
||||||
Ids are **webkeys** ([the pattern](../docs/webkeys.md)): 256-bit bearer
|
|
||||||
secrets that never ride a query string or path. The webkey page carries
|
|
||||||
the cache id in the URL fragment (never sent to the server) and passes
|
|
||||||
it in-band over Cap'n Web; forms carry ids in POST bodies; logs and pages
|
|
||||||
show only a fingerprint (a prefix of the id's one-way hash).
|
|
||||||
|
|
||||||
- **The cache id**: the cache's read capability (bearer; not logged).
|
|
||||||
Held by the user — saved to the password manager under a petname they
|
|
||||||
choose (the username field; never sent to the server) — and passed
|
|
||||||
per POST.
|
|
||||||
- **The webapp instance id**: the diigo-read capability; lives in each
|
|
||||||
cache DO's wiring, arrives from the mint step.
|
|
||||||
- **The cookie**: lives **only in webapp's DO storage**; crosses the
|
|
||||||
wire once, at mint time, as an opaque relay.
|
|
||||||
- **The public cache**: no capability — `/public/<username>` is public
|
|
||||||
by design (a guessable URL; hiding the link doesn't hide the page).
|
|
||||||
|
|
||||||
Stored state is meant to be durable: see the breaking-change rule in
|
|
||||||
[capsite AGENTS.md](../AGENTS.md).
|
|
||||||
|
|
||||||
## Deployment concern: rate-limit `/api`
|
|
||||||
|
|
||||||
Public pages only read storage, so strangers can't make the cache spend
|
|
||||||
your diigo session. `/api` is keyed (a cache id is needed), but Cap'n
|
|
||||||
Web pipelining makes calls cheap to send. Before exposing a deployment:
|
|
||||||
|
|
||||||
- rate-limit `/api` (and `/public/*`, which reads storage) — on
|
|
||||||
Cloudflare, the
|
|
||||||
[Workers rate-limit binding](https://developers.cloudflare.com/workers/runtime-apis/bindings/rate-limit/)
|
|
||||||
(keyed by client and/or username); on other workerd hosts, a rate
|
|
||||||
limit in the proxy in front;
|
|
||||||
- run the fill job (goal 1e): public pages show only what's stored.
|
|
||||||
|
|
||||||
## RPC use (from another worker, same account)
|
|
||||||
|
|
||||||
```jsonc
|
|
||||||
// wrangler.jsonc of the caller
|
|
||||||
"services": [{ "binding": "DIIGO", "service": "diigo-cache", "entrypoint": "DiigoBridge" }]
|
|
||||||
```
|
```
|
||||||
|
|
||||||
```ts
|
Then, in the browser:
|
||||||
// cache ids ride goal-2's wiring; see the capsite checklist
|
|
||||||
const ndjson = await env.DIIGO.dumpNdjson(4); // 4 pages of PAGE_SIZE
|
|
||||||
```
|
|
||||||
|
|
||||||
## Test
|
1. **mint the diigo session at webapp**: paste the `Cookie` header from diigo.com (devtools, any logged-in request).
|
||||||
|
2. **wire** it, mode **public**. Save the cache id it shows (the password-manager button does this).
|
||||||
|
3. **open this cache** → **fill**, and watch it walk your bookmarks.
|
||||||
|
4. Visit `/public/<your diigo username>?days=7`.
|
||||||
|
|
||||||
pnpm types # regenerate worker-configuration.d.ts (after wrangler.jsonc changes)
|
## Bookmarks you hold, a page anyone can read
|
||||||
pnpm lint # fails if those types are stale; tsc over src/ and tests/
|
|
||||||
pnpm test # vitest
|
|
||||||
|
|
||||||
Tests run the real worker and both DOs in the vitest pool, with
|
- **Filter by date and search:** `?days=7`, `?in=2026-W41` (ISO week, UTC), `?after=2026-01-01&before=2026-02-01`, `?by=created`, and `?q=` with words, `"phrases"`, `#tags`, `-negation`, and gmail-style `newer_than:7d`. Add `Accept: application/json` for JSON.
|
||||||
`WEBAPP` bound to [a fake webapp](tests/fake-webapp.js) serving
|
- **Private stays private:** the public cache drops private bookmarks before storing anything, so it never holds them.
|
||||||
diigo-shaped items (`wrangler.tests.jsonc`).
|
- **Ids are capabilities:** the cache id is a bearer key you keep in your password manager. It rides URL fragments and POST bodies, never a path or query string, so it stays out of logs and history.
|
||||||
|
- **Cheap to read:** pages come from Durable Object SQLite storage, and a fill job in the background fetches each diigo page once.
|
||||||
|
|
||||||
## Run, deploy
|
Design, tests, and deploying: [CONTRIBUTING](CONTRIBUTING.md).
|
||||||
|
|
||||||
[How to run / connect / deploy is maintained in the capsite checklist](../capsite-checklist.md)
|
|
||||||
(single source; this README keeps only worker-level facts).
|
|
||||||
|
|
||||||
Status: create, dump, publish, and public pages exercised against live
|
|
||||||
diigo in dev (2026-10-07); not yet deployed.
|
|
||||||
@@ -11,6 +11,7 @@
|
|||||||
"devDependencies": {
|
"devDependencies": {
|
||||||
"@cloudflare/vitest-plugin": "1.3.6",
|
"@cloudflare/vitest-plugin": "1.3.6",
|
||||||
"@types/node": "^22",
|
"@types/node": "^22",
|
||||||
|
"microformats-parser": "^2.0.6",
|
||||||
"typescript": "^5.9.0",
|
"typescript": "^5.9.0",
|
||||||
"vitest": "4.1.11",
|
"vitest": "4.1.11",
|
||||||
"wrangler": "^4.148.0"
|
"wrangler": "^4.148.0"
|
||||||
|
|||||||
Generated
+24
@@ -18,6 +18,9 @@ importers:
|
|||||||
'@types/node':
|
'@types/node':
|
||||||
specifier: ^22
|
specifier: ^22
|
||||||
version: 22.20.5
|
version: 22.20.5
|
||||||
|
microformats-parser:
|
||||||
|
specifier: ^2.0.6
|
||||||
|
version: 2.0.6
|
||||||
typescript:
|
typescript:
|
||||||
specifier: ^5.9.0
|
specifier: ^5.9.0
|
||||||
version: 5.9.3
|
version: 5.9.3
|
||||||
@@ -613,6 +616,10 @@ packages:
|
|||||||
resolution: {integrity: sha512-Btj2BOOO83o3WyH59e8MgXsxEQVcarkUOpEYrubB0urwnN10yQ364rsiByU11nZlqWYZm05i/of7io4mzihBtQ==}
|
resolution: {integrity: sha512-Btj2BOOO83o3WyH59e8MgXsxEQVcarkUOpEYrubB0urwnN10yQ364rsiByU11nZlqWYZm05i/of7io4mzihBtQ==}
|
||||||
engines: {node: '>=8'}
|
engines: {node: '>=8'}
|
||||||
|
|
||||||
|
entities@6.0.1:
|
||||||
|
resolution: {integrity: sha512-aN97NXWF6AWBTahfVOIrB/NShkzi5H7F9r1s9mD3cDj4Ko5f2qhhVoYMibXF7GlLveb/D2ioWay8lxI97Ven3g==}
|
||||||
|
engines: {node: '>=0.12'}
|
||||||
|
|
||||||
error-stack-parser-es@1.0.5:
|
error-stack-parser-es@1.0.5:
|
||||||
resolution: {integrity: sha512-5qucVt2XcuGMcEGgWI7i+yZpmpByQ8J1lHhcL7PwqCwu9FPP3VUXzT4ltHe5i2z9dePwEHcDVOAfSnHsOlCXRA==}
|
resolution: {integrity: sha512-5qucVt2XcuGMcEGgWI7i+yZpmpByQ8J1lHhcL7PwqCwu9FPP3VUXzT4ltHe5i2z9dePwEHcDVOAfSnHsOlCXRA==}
|
||||||
|
|
||||||
@@ -722,6 +729,10 @@ packages:
|
|||||||
magic-string@0.30.21:
|
magic-string@0.30.21:
|
||||||
resolution: {integrity: sha512-vd2F4YUyEXKGcLHoq+TEyCjxueSeHnFxyyjNp80yg0XV4vUhnDer/lvvlqM/arB5bXQN5K2/3oinyCRyx8T2CQ==}
|
resolution: {integrity: sha512-vd2F4YUyEXKGcLHoq+TEyCjxueSeHnFxyyjNp80yg0XV4vUhnDer/lvvlqM/arB5bXQN5K2/3oinyCRyx8T2CQ==}
|
||||||
|
|
||||||
|
microformats-parser@2.0.6:
|
||||||
|
resolution: {integrity: sha512-VTnLImF17yd+ZfBH40sddHMJp5Rrker4ge+oPjgnBQ3jluQO0YmsNVA4UcEfHRBV8zrikBSMkAzAub+nTRX85g==}
|
||||||
|
engines: {node: '>=18'}
|
||||||
|
|
||||||
miniflare@5.20261001.0-alpha:
|
miniflare@5.20261001.0-alpha:
|
||||||
resolution: {integrity: sha512-GaimS5mSIOMyvd16ga+e1/QkI8cmp3z35QPHFex0DktqNQf2RVZQwMPQXdyoUreUiFP/0Gpe2MoQInTyMAD5xA==}
|
resolution: {integrity: sha512-GaimS5mSIOMyvd16ga+e1/QkI8cmp3z35QPHFex0DktqNQf2RVZQwMPQXdyoUreUiFP/0Gpe2MoQInTyMAD5xA==}
|
||||||
engines: {node: '>=22.0.0'}
|
engines: {node: '>=22.0.0'}
|
||||||
@@ -739,6 +750,9 @@ packages:
|
|||||||
resolution: {integrity: sha512-XrsrhT5sybtKI6wakr2SPOlGZWWYbUXZ7a0jT8/QOeAPau+1X/bSegNe5YR75oJmEZQbKningirmGOEJCIk61Q==}
|
resolution: {integrity: sha512-XrsrhT5sybtKI6wakr2SPOlGZWWYbUXZ7a0jT8/QOeAPau+1X/bSegNe5YR75oJmEZQbKningirmGOEJCIk61Q==}
|
||||||
engines: {node: '>=12.20.0'}
|
engines: {node: '>=12.20.0'}
|
||||||
|
|
||||||
|
parse5@7.3.0:
|
||||||
|
resolution: {integrity: sha512-IInvU7fabl34qmi9gY8XOVxhYyMyuH2xUNpb2q8/Y+7552KlejkRvqvD19nMoUW/uQGGbqNpA6Tufu5FL5BZgw==}
|
||||||
|
|
||||||
path-to-regexp@6.3.0:
|
path-to-regexp@6.3.0:
|
||||||
resolution: {integrity: sha512-Yhpw4T9C6hPpgPeA28us07OJeqZ5EzQTkbfwuhsUg0c237RomFoETJgmp2sa3F/41gfLE6G5cqcYwznmeEeOlQ==}
|
resolution: {integrity: sha512-Yhpw4T9C6hPpgPeA28us07OJeqZ5EzQTkbfwuhsUg0c237RomFoETJgmp2sa3F/41gfLE6G5cqcYwznmeEeOlQ==}
|
||||||
|
|
||||||
@@ -1366,6 +1380,8 @@ snapshots:
|
|||||||
|
|
||||||
detect-libc@2.1.2: {}
|
detect-libc@2.1.2: {}
|
||||||
|
|
||||||
|
entities@6.0.1: {}
|
||||||
|
|
||||||
error-stack-parser-es@1.0.5: {}
|
error-stack-parser-es@1.0.5: {}
|
||||||
|
|
||||||
es-module-lexer@2.3.2: {}
|
es-module-lexer@2.3.2: {}
|
||||||
@@ -1467,6 +1483,10 @@ snapshots:
|
|||||||
dependencies:
|
dependencies:
|
||||||
'@jridgewell/sourcemap-codec': 1.6.0
|
'@jridgewell/sourcemap-codec': 1.6.0
|
||||||
|
|
||||||
|
microformats-parser@2.0.6:
|
||||||
|
dependencies:
|
||||||
|
parse5: 7.3.0
|
||||||
|
|
||||||
miniflare@5.20261001.0-alpha(@types/node@22.20.5):
|
miniflare@5.20261001.0-alpha(@types/node@22.20.5):
|
||||||
dependencies:
|
dependencies:
|
||||||
'@cspotcode/source-map-support': 0.8.1
|
'@cspotcode/source-map-support': 0.8.1
|
||||||
@@ -1497,6 +1517,10 @@ snapshots:
|
|||||||
|
|
||||||
obug@2.2.1: {}
|
obug@2.2.1: {}
|
||||||
|
|
||||||
|
parse5@7.3.0:
|
||||||
|
dependencies:
|
||||||
|
entities: 6.0.1
|
||||||
|
|
||||||
path-to-regexp@6.3.0: {}
|
path-to-regexp@6.3.0: {}
|
||||||
|
|
||||||
pathe@2.0.3: {}
|
pathe@2.0.3: {}
|
||||||
|
|||||||
@@ -19,3 +19,14 @@ where cast(items.updated as integer) >= ?;
|
|||||||
|
|
||||||
-- pagesThrough:
|
-- pagesThrough:
|
||||||
select count(*) as n from pages where page_num <= ?;
|
select count(*) as n from pages where page_num <= ?;
|
||||||
|
|
||||||
|
-- itemsByUpdated:
|
||||||
|
select json from items
|
||||||
|
where cast(updated as integer) >= ? and cast(updated as integer) < ?
|
||||||
|
order by cast(updated as integer) desc, item_id;
|
||||||
|
|
||||||
|
-- itemsByCreated:
|
||||||
|
select json from items
|
||||||
|
where cast(json_extract(json, '$.created_at') as integer) >= ?
|
||||||
|
and cast(json_extract(json, '$.created_at') as integer) < ?
|
||||||
|
order by cast(json_extract(json, '$.created_at') as integer) desc, item_id;
|
||||||
@@ -6,6 +6,8 @@
|
|||||||
import { DurableObject, RpcTarget } from "cloudflare:workers";
|
import { DurableObject, RpcTarget } from "cloudflare:workers";
|
||||||
import type { PagedGet } from "./diigo.js";
|
import type { PagedGet } from "./diigo.js";
|
||||||
import { publicOnly, toRows } from "./items.js";
|
import { publicOnly, toRows } from "./items.js";
|
||||||
|
import { matches } from "./query.js";
|
||||||
|
import type { Query } from "./query.js";
|
||||||
import { makeReader, makeWriter, SCHEMA_VERSION } from "./store.js";
|
import { makeReader, makeWriter, SCHEMA_VERSION } from "./store.js";
|
||||||
import type { Reader, Writer } from "./store.js";
|
import type { Reader, Writer } from "./store.js";
|
||||||
import type { Item, ItemRow } from "./items.js";
|
import type { Item, ItemRow } from "./items.js";
|
||||||
@@ -398,7 +400,18 @@ abstract class PagedCache extends DurableObject<CacheEnv> {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
/** what anyone may see of a cache: stored pages and fill progress */
|
/** one page of a filter's matches over stored items */
|
||||||
|
export type QueryPage = {
|
||||||
|
/** the matches on this page, raw dump rows (JSON strings) */
|
||||||
|
items: string[];
|
||||||
|
/** matches in all */
|
||||||
|
matched: number;
|
||||||
|
/** true when every diigo page is stored, so no match can be missing */
|
||||||
|
complete: boolean;
|
||||||
|
};
|
||||||
|
|
||||||
|
/** what anyone may see of a cache: stored pages, stored items by filter,
|
||||||
|
* and fill progress */
|
||||||
export class PageViewer extends RpcTarget {
|
export class PageViewer extends RpcTarget {
|
||||||
#reader: Reader;
|
#reader: Reader;
|
||||||
#fillStatus: () => Promise<FillStatus | undefined>;
|
#fillStatus: () => Promise<FillStatus | undefined>;
|
||||||
@@ -415,6 +428,25 @@ export class PageViewer extends RpcTarget {
|
|||||||
return stored && { ...stored, cached: true };
|
return stored && { ...stored, cached: true };
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* page `pageNum` (`count` per page) of the stored items matching
|
||||||
|
* `query`, newest first by its date. The window narrows in SQL; terms
|
||||||
|
* are checked per item. Reads storage only: items not yet stored can't
|
||||||
|
* match, which `complete` reports.
|
||||||
|
*/
|
||||||
|
query(query: Query, pageNum: number, count: number): QueryPage {
|
||||||
|
const from = query.after ? Math.ceil(query.after.getTime() / 1000) : Number.MIN_SAFE_INTEGER;
|
||||||
|
const to = query.before ? Math.ceil(query.before.getTime() / 1000) : Number.MAX_SAFE_INTEGER;
|
||||||
|
const hits = this.#reader.itemsIn(query.by, from, to).filter((json) => matches(JSON.parse(json) as Item, query));
|
||||||
|
const total = this.#reader.knownTotal();
|
||||||
|
const pages = total === null ? null : Math.max(1, Math.ceil(total / PAGE_SIZE));
|
||||||
|
return {
|
||||||
|
items: hits.slice((pageNum - 1) * count, pageNum * count),
|
||||||
|
matched: hits.length,
|
||||||
|
complete: pages !== null && this.#reader.pagesThrough(pages) >= pages,
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
/** diigo's total, as last stored, or null if nothing is stored */
|
/** diigo's total, as last stored, or null if nothing is stored */
|
||||||
knownTotal(): number | null {
|
knownTotal(): number | null {
|
||||||
return this.#reader.knownTotal();
|
return this.#reader.knownTotal();
|
||||||
|
|||||||
+152
-34
@@ -10,9 +10,11 @@ import { CAPNWEB_BROWSER_JS } from "./capnweb-browser.js";
|
|||||||
import { CACHE_PAGE_HTML } from "./cache-page.js";
|
import { CACHE_PAGE_HTML } from "./cache-page.js";
|
||||||
import { DiigoCacheLogic, PublicCacheLogic } from "./cache.js";
|
import { DiigoCacheLogic, PublicCacheLogic } from "./cache.js";
|
||||||
import { PAGE_SIZE } from "./cache.js";
|
import { PAGE_SIZE } from "./cache.js";
|
||||||
import type { CacheEnv, Clock, FillStatus } from "./cache.js";
|
import type { CacheEnv, Clock, FillStatus, QueryPage } from "./cache.js";
|
||||||
import type { Item } from "./cache.js";
|
import type { Item } from "./cache.js";
|
||||||
import { updatedISO, usernameOf } from "./items.js";
|
import { usernameOf } from "./items.js";
|
||||||
|
import { dateOf, isEmpty, parseQuery } from "./query.js";
|
||||||
|
import type { Query } from "./query.js";
|
||||||
import { toWebkey, idFromWebkey, isWebkey, shashKey } from "./webkeys.js";
|
import { toWebkey, idFromWebkey, isWebkey, shashKey } from "./webkeys.js";
|
||||||
import { pmSave } from "./pm-save.js";
|
import { pmSave } from "./pm-save.js";
|
||||||
|
|
||||||
@@ -443,10 +445,16 @@ async function dump(request: Request, env: Env): Promise<Response> {
|
|||||||
|
|
||||||
const PUBLIC_PER_PAGE = PAGE_SIZE;
|
const PUBLIC_PER_PAGE = PAGE_SIZE;
|
||||||
|
|
||||||
|
/** the public page's day presets */
|
||||||
|
const DAY_PRESETS = [7, 30, 90, 365];
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* the public page for a username: stored pages only, through the cache's
|
* the public page for a username: stored pages only, through the cache's
|
||||||
* read-only `viewer()` (it can't fetch or write). HTML with a page-number
|
* read-only `viewer()` (it can't fetch or write). HTML with a filter form
|
||||||
* form, or JSON for `Accept: application/json`. `?page=` in the URL is
|
* and a page-number form, or JSON for `Accept: application/json`. A
|
||||||
|
* filter (goal 1d: `days`, `after`, `before`, `in`, `by`, `q`; see
|
||||||
|
* {@link parseQuery}) lists the stored items that match, `page` counting
|
||||||
|
* matches; with none, `page` is diigo's page as stored. Query params are
|
||||||
* fine here: nothing on this page is secret.
|
* fine here: nothing on this page is secret.
|
||||||
*/
|
*/
|
||||||
async function publicPage(request: Request, env: Env, username: string): Promise<Response> {
|
async function publicPage(request: Request, env: Env, username: string): Promise<Response> {
|
||||||
@@ -456,14 +464,33 @@ async function publicPage(request: Request, env: Env, username: string): Promise
|
|||||||
const ns = env.PUBLIC_DO;
|
const ns = env.PUBLIC_DO;
|
||||||
const cache = ns.get(ns.idFromString(idFromWebkey(publicId)));
|
const cache = ns.get(ns.idFromString(idFromWebkey(publicId)));
|
||||||
|
|
||||||
const raw = new URL(request.url).searchParams.get("page");
|
const params = new URL(request.url).searchParams;
|
||||||
|
const accept = request.headers.get("accept") ?? "";
|
||||||
|
const wantsJson = accept.includes("application/json") && !accept.includes("text/html");
|
||||||
|
const here = `/public/${encodeURIComponent(username)}`;
|
||||||
|
const raw = params.get("page");
|
||||||
const pageN = raw === null ? 1 : Number.parseInt(raw, 10);
|
const pageN = raw === null ? 1 : Number.parseInt(raw, 10);
|
||||||
const valid = !Number.isNaN(pageN) && pageN >= 1 && pageN <= 10000;
|
const valid = !Number.isNaN(pageN) && pageN >= 1 && pageN <= 10000;
|
||||||
|
|
||||||
|
const query = parseQuery(params, new Date());
|
||||||
|
if ("error" in query) {
|
||||||
|
console.log({ event: "notice", page: "public", text: "filter refused", error: query.error });
|
||||||
|
if (wantsJson) return Response.json({ error: query.error }, { status: 400 });
|
||||||
|
const body = publicFrame(username, `${filterForm(here, params)}
|
||||||
|
<p><em>${esc(query.error)}</em></p>`);
|
||||||
|
return new Response(body, { status: 400, headers: { "content-type": "text/html;charset=utf-8" } });
|
||||||
|
}
|
||||||
|
|
||||||
const viewer = await cache.viewer();
|
const viewer = await cache.viewer();
|
||||||
let page, total, fill;
|
let page, fill, found;
|
||||||
|
let total: number | null = null;
|
||||||
try {
|
try {
|
||||||
page = await viewer.page(valid ? pageN : 1, PUBLIC_PER_PAGE);
|
if (isEmpty(query)) {
|
||||||
total = page ? page.total : await viewer.knownTotal();
|
page = await viewer.page(valid ? pageN : 1, PUBLIC_PER_PAGE);
|
||||||
|
total = page ? page.total : await viewer.knownTotal();
|
||||||
|
} else {
|
||||||
|
found = await viewer.query(query, valid ? pageN : 1, PUBLIC_PER_PAGE);
|
||||||
|
}
|
||||||
fill = await viewer.fillStatus();
|
fill = await viewer.fillStatus();
|
||||||
} catch (err) {
|
} catch (err) {
|
||||||
console.error({ event: "public-page", page: pageN, error: String(err) });
|
console.error({ event: "public-page", page: pageN, error: String(err) });
|
||||||
@@ -471,13 +498,14 @@ async function publicPage(request: Request, env: Env, username: string): Promise
|
|||||||
} finally {
|
} finally {
|
||||||
viewer[Symbol.dispose]();
|
viewer[Symbol.dispose]();
|
||||||
}
|
}
|
||||||
|
if (found) return filteredPage(username, here, params, query, found, valid ? pageN : 1, wantsJson, fill);
|
||||||
|
|
||||||
// total and pages count all of diigo's items: public page n is diigo
|
// total and pages count all of diigo's items: public page n is diigo
|
||||||
// page n with the private rows dropped
|
// page n with the private rows dropped
|
||||||
const pages = total === null ? 1 : Math.max(1, Math.ceil(total / PUBLIC_PER_PAGE));
|
const pages = total === null ? 1 : Math.max(1, Math.ceil(total / PUBLIC_PER_PAGE));
|
||||||
const items = page ? page.items.map((j) => JSON.parse(j) as Item) : [];
|
const items = page ? page.items.map((j) => JSON.parse(j) as Item) : [];
|
||||||
|
|
||||||
const accept = request.headers.get("accept") ?? "";
|
if (wantsJson) {
|
||||||
if (accept.includes("application/json") && !accept.includes("text/html")) {
|
|
||||||
return Response.json({ username, page: pageN, pages, total, stored: page !== null, etchedAt: page?.etchedAt ?? null, items });
|
return Response.json({ username, page: pageN, pages, total, stored: page !== null, etchedAt: page?.etchedAt ?? null, items });
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -486,42 +514,132 @@ async function publicPage(request: Request, env: Env, username: string): Promise
|
|||||||
console.log({ event: "notice", page: "public", text: "page out of range", pages });
|
console.log({ event: "notice", page: "public", text: "page out of range", pages });
|
||||||
notice = `<p><em>no page ${esc(String(raw))}: pages run 1–${pages}</em></p>`;
|
notice = `<p><em>no page ${esc(String(raw))}: pages run 1–${pages}</em></p>`;
|
||||||
}
|
}
|
||||||
const here = `/public/${encodeURIComponent(username)}`;
|
|
||||||
const nav = (n: number, label: string) =>
|
const nav = (n: number, label: string) =>
|
||||||
n >= 1 && n <= pages ? `<a href="${here}?page=${n}">${label}</a>` : `<span>${label}</span>`;
|
n >= 1 && n <= pages ? `<a rel="${n < pageN ? "prev" : "next"}" href="${here}?page=${n}">${label}</a>` : `<span>${label}</span>`;
|
||||||
const list = notice
|
const list = notice ? "" : itemList(items, "updated", here);
|
||||||
? ""
|
const body = publicFrame(
|
||||||
: `<ol>
|
username,
|
||||||
${items
|
`<p>${pages} page${pages === 1 ? "" : "s"} (diigo's, ${PUBLIC_PER_PAGE} bookmarks each; private ones are left out, so a page may show fewer).</p>
|
||||||
.map((it) => {
|
|
||||||
const url = typeof it.url === "string" ? it.url : "";
|
|
||||||
const title = typeof it.title === "string" && it.title ? it.title : url;
|
|
||||||
const desc = typeof it.description === "string" && it.description ? `<br><small>${esc(it.description)}</small>` : "";
|
|
||||||
const href = /^https?:\/\//i.test(url) ? url : "#";
|
|
||||||
const when = updatedISO(it);
|
|
||||||
const date = when ? `<time datetime="${when}">${when}</time>` : "(no date)";
|
|
||||||
return `<li>${date} <a href="${esc(href)}">${esc(title)}</a>${desc}</li>`;
|
|
||||||
})
|
|
||||||
.join("\n")}
|
|
||||||
</ol>`;
|
|
||||||
const body = `<!doctype html>
|
|
||||||
<html><head><meta charset="utf-8"><title>${esc(username)}'s shared bookmarks</title></head>
|
|
||||||
<body>
|
|
||||||
<h1>${esc(username)}'s shared bookmarks</h1>
|
|
||||||
<p>${pages} page${pages === 1 ? "" : "s"} (diigo's, ${PUBLIC_PER_PAGE} bookmarks each; private ones are left out, so a page may show fewer).</p>
|
|
||||||
<p>${page ? `<strong>stored ${esc(page.etchedAt ?? "")}</strong> — page ${valid ? pageN : 1}, ${items.length} shared` : `<strong>not fetched yet</strong> — page ${valid ? pageN : 1}; the cache's owner fills it`}</p>
|
<p>${page ? `<strong>stored ${esc(page.etchedAt ?? "")}</strong> — page ${valid ? pageN : 1}, ${items.length} shared` : `<strong>not fetched yet</strong> — page ${valid ? pageN : 1}; the cache's owner fills it`}</p>
|
||||||
${fill ? fillLine(fill) : ""}
|
${fill ? fillLine(fill) : ""}
|
||||||
|
${filterForm(here, params)}
|
||||||
<form method="get" action="${here}">
|
<form method="get" action="${here}">
|
||||||
<p>${nav(pageN - 1, "← prev")}
|
<p>${nav(pageN - 1, "← prev")}
|
||||||
<label>page <input type="number" name="page" min="1" value="${Number.isNaN(pageN) ? 1 : pageN}"></label>
|
<label>page <input type="number" name="page" min="1" value="${Number.isNaN(pageN) ? 1 : pageN}"></label>
|
||||||
<button type="submit">go</button>
|
<button type="submit">go</button>
|
||||||
${nav(pageN + 1, "next →")}</p>
|
${nav(pageN + 1, "next →")}</p>
|
||||||
</form>
|
</form>
|
||||||
${notice}${list}
|
${notice}${list}`,
|
||||||
</body></html>`;
|
);
|
||||||
return new Response(body, { headers: { "content-type": "text/html;charset=utf-8" } });
|
return new Response(body, { headers: { "content-type": "text/html;charset=utf-8" } });
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/** a filter's matches, as the public page's HTML or JSON */
|
||||||
|
function filteredPage(
|
||||||
|
username: string,
|
||||||
|
here: string,
|
||||||
|
params: URLSearchParams,
|
||||||
|
query: Query,
|
||||||
|
found: QueryPage,
|
||||||
|
pageN: number,
|
||||||
|
wantsJson: boolean,
|
||||||
|
fill: FillStatus | undefined,
|
||||||
|
): Response {
|
||||||
|
const pages = Math.max(1, Math.ceil(found.matched / PUBLIC_PER_PAGE));
|
||||||
|
const items = found.items.map((j) => JSON.parse(j) as Item);
|
||||||
|
if (wantsJson) {
|
||||||
|
const { matched, complete } = found;
|
||||||
|
return Response.json({ username, filter: query, page: pageN, pages, matched, complete, items });
|
||||||
|
}
|
||||||
|
// links keep the filter: the request's own non-empty params, in order
|
||||||
|
const kept = [...params].filter(([k, v]) => k !== "page" && v !== "");
|
||||||
|
const href = (n: number) => `${here}?${new URLSearchParams([...kept, ["page", String(n)]])}`;
|
||||||
|
const nav = (n: number, label: string) =>
|
||||||
|
n >= 1 && n <= pages ? `<a rel="${n < pageN ? "prev" : "next"}" href="${esc(href(n))}">${label}</a>` : `<span>${label}</span>`;
|
||||||
|
const status = found.matched
|
||||||
|
? `${found.matched} match${found.matched === 1 ? "" : "es"}, page ${pageN} of ${pages}, newest ${query.by} first`
|
||||||
|
: "no matches";
|
||||||
|
const caveat = found.complete ? "" : " — not every diigo page is stored yet, so some matches may be missing";
|
||||||
|
const body = publicFrame(
|
||||||
|
username,
|
||||||
|
`<p><strong>${esc(status)}</strong>${caveat}</p>
|
||||||
|
${fill ? fillLine(fill) : ""}
|
||||||
|
${filterForm(here, params)}
|
||||||
|
<p>${nav(pageN - 1, "← prev")} ${nav(pageN + 1, "next →")} <a href="${here}">all, by diigo page</a></p>
|
||||||
|
${itemList(items, query.by, here)}`,
|
||||||
|
);
|
||||||
|
return new Response(body, { headers: { "content-type": "text/html;charset=utf-8" } });
|
||||||
|
}
|
||||||
|
|
||||||
|
/** the public page's frame: title and heading around `content` */
|
||||||
|
function publicFrame(username: string, content: string): string {
|
||||||
|
return `<!doctype html>
|
||||||
|
<html><head><meta charset="utf-8"><title>${esc(username)}'s shared bookmarks</title></head>
|
||||||
|
<body>
|
||||||
|
<h1>${esc(username)}'s shared bookmarks</h1>
|
||||||
|
${content}
|
||||||
|
</body></html>`;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** the filter form, echoing the request's filter */
|
||||||
|
function filterForm(here: string, params: URLSearchParams): string {
|
||||||
|
const field = (name: string) => esc(params.get(name) ?? "");
|
||||||
|
const days = params.get("days") ?? "";
|
||||||
|
const presets = DAY_PRESETS.map(String);
|
||||||
|
if (days && !presets.includes(days)) presets.push(days);
|
||||||
|
const option = (value: string, label: string, current: string) =>
|
||||||
|
`<option value="${esc(value)}"${value === current ? " selected" : ""}>${esc(label)}</option>`;
|
||||||
|
return `<form method="get" action="${here}">
|
||||||
|
<p><label>last <select name="days">${option("", "any", days)}${presets.map((n) => option(n, `${n} days`, days)).join("")}</select></label>
|
||||||
|
<label>or after <input type="date" name="after" value="${field("after")}"></label>
|
||||||
|
<label>before <input type="date" name="before" value="${field("before")}"></label>
|
||||||
|
<label>or in <input name="in" size="9" placeholder="2026-W41" value="${field("in")}"></label>
|
||||||
|
<label>by <select name="by">${option("updated", "updated", params.get("by") ?? "")}${option("created", "created", params.get("by") ?? "")}</select></label></p>
|
||||||
|
<p><label>search <input type="search" name="q" size="40" value="${field("q")}"></label>
|
||||||
|
<button type="submit">filter</button></p>
|
||||||
|
<p><small>Dates and periods (2026, 2026-03, 2026-Q1, 2026-W41) are UTC. Search takes
|
||||||
|
words, "phrases", #tags, -negation, and after:, before:, newer_than:7d, during:2026-Q1, created:2026.</small></p>
|
||||||
|
</form>`;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* bookmarks as a list, each led by its `by` date, marked up as
|
||||||
|
* microformats: an `h-feed` of `h-entry` bookmarks (`u-bookmark-of`,
|
||||||
|
* `p-name`, `p-summary`, `p-category` per tag, `dt-published` when
|
||||||
|
* bookmarked, `dt-updated` when last edited). This markup is the public
|
||||||
|
* page's API for other sites (tests/caches.test.ts pins it); tag links
|
||||||
|
* search this page (`here`) by tag.
|
||||||
|
*
|
||||||
|
* @see https://microformats.org/wiki/h-entry
|
||||||
|
* @see https://indieweb.org/bookmark
|
||||||
|
*/
|
||||||
|
function itemList(items: Item[], by: Query["by"], here: string): string {
|
||||||
|
const time = (it: Item, which: Query["by"], hidden: boolean) => {
|
||||||
|
const secs = dateOf(it, which);
|
||||||
|
if (secs === undefined) return hidden ? "" : "(no date)";
|
||||||
|
const when = new Date(secs * 1000).toISOString();
|
||||||
|
const cls = which === "created" ? "dt-published" : "dt-updated";
|
||||||
|
return hidden ? `<time class="${cls}" datetime="${when}" hidden></time>` : `<time class="${cls}" datetime="${when}">${when}</time>`;
|
||||||
|
};
|
||||||
|
return `<ol class="h-feed">
|
||||||
|
${items
|
||||||
|
.map((it) => {
|
||||||
|
const url = typeof it.url === "string" ? it.url : "";
|
||||||
|
const title = typeof it.title === "string" && it.title ? it.title : url;
|
||||||
|
const desc = typeof it.description === "string" && it.description ? `<br><small class="p-summary">${esc(it.description)}</small>` : "";
|
||||||
|
const href = /^https?:\/\//i.test(url) ? url : "#";
|
||||||
|
const tags = (typeof it.tags === "string" ? it.tags.split(",") : [])
|
||||||
|
.map((t) => t.trim())
|
||||||
|
.filter(Boolean)
|
||||||
|
.map((t) => ` <a class="p-category" href="${esc(`${here}?q=${encodeURIComponent(`#${t}`)}`)}">${esc(t)}</a>`)
|
||||||
|
.join("");
|
||||||
|
const other = by === "created" ? "updated" : "created";
|
||||||
|
return `<li class="h-entry">${time(it, by, false)} <a class="u-bookmark-of p-name" href="${esc(href)}">${esc(title)}</a>${tags}${desc}${time(it, other, true)}</li>`;
|
||||||
|
})
|
||||||
|
.join("\n")}
|
||||||
|
</ol>`;
|
||||||
|
}
|
||||||
|
|
||||||
/** home page: the connect link (webapp's cookie form, diigo-prefilled) +
|
/** home page: the connect link (webapp's cookie form, diigo-prefilled) +
|
||||||
* the wire step for an already-minted id + the dump form. `home` renders
|
* the wire step for an already-minted id + the dump form. `home` renders
|
||||||
* the error path too, so it takes env for the connect link. Notices (the
|
* the error path too, so it takes env for the connect link. Notices (the
|
||||||
|
|||||||
+188
-19
@@ -1,18 +1,10 @@
|
|||||||
/**
|
/**
|
||||||
* @file the public page's filters: date windows and search (goal 1d, **not
|
* @file the public page's filters: date windows and search over a cache's
|
||||||
* yet implemented**; the design is the skipped specs in tests/query.test.ts).
|
* stored items (goal 1d; tests/query.test.ts).
|
||||||
*/
|
*/
|
||||||
|
|
||||||
import type { Item } from "./items.js";
|
import type { Item } from "./items.js";
|
||||||
|
|
||||||
/**
|
|
||||||
* diigo fetches one request may spend (cache hits are free); under the
|
|
||||||
* Workers free plan's 50 external subrequests per invocation
|
|
||||||
*
|
|
||||||
* @see https://developers.cloudflare.com/workers/platform/limits/
|
|
||||||
*/
|
|
||||||
export const FETCH_BUDGET = 40;
|
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* a filter as devs write it (`{ days: 7, q: "#weekly" }`); URL params map
|
* a filter as devs write it (`{ days: 7, q: "#weekly" }`); URL params map
|
||||||
* onto these fields one-to-one (strings, with "" meaning absent). Dates
|
* onto these fields one-to-one (strings, with "" meaning absent). Dates
|
||||||
@@ -56,6 +48,76 @@ export type Query = {
|
|||||||
/** a refused filter: the message is safe to show and log */
|
/** a refused filter: the message is safe to show and log */
|
||||||
export type QueryError = { error: string };
|
export type QueryError = { error: string };
|
||||||
|
|
||||||
|
const DAY_MS = 24 * 60 * 60 * 1000;
|
||||||
|
|
||||||
|
/** thrown inside {@link toQuery}, returned as a {@link QueryError} */
|
||||||
|
class Refused extends Error {}
|
||||||
|
|
||||||
|
/** `yyyy-mm-dd` or `yyyy/mm/dd` as 00:00Z of that UTC day */
|
||||||
|
function parseDay(text: string, what: string): Date {
|
||||||
|
const m = /^(\d{4})([-/])(\d{2})\2(\d{2})$/.exec(text);
|
||||||
|
const d = m && new Date(Date.UTC(Number(m[1]), Number(m[3]) - 1, Number(m[4])));
|
||||||
|
if (!m || !d || d.getUTCMonth() !== Number(m[3]) - 1 || d.getUTCDate() !== Number(m[4])) {
|
||||||
|
throw new Refused(`${what}: expected a date as yyyy-mm-dd`);
|
||||||
|
}
|
||||||
|
return d;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Monday 00:00Z of ISO week 1 of `year` (the week holding January 4) */
|
||||||
|
function isoWeekOne(year: number): number {
|
||||||
|
const jan4 = Date.UTC(year, 0, 4);
|
||||||
|
const weekday = new Date(jan4).getUTCDay() || 7;
|
||||||
|
return jan4 - (weekday - 1) * DAY_MS;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** a named period as its half-open [after, before) window */
|
||||||
|
function parsePeriod(text: string, what: string): [Date, Date] {
|
||||||
|
const refuse = () => new Refused(`${what}: expected a period as 2025, 2025-03, 2025-Q1, or 2025-W01`);
|
||||||
|
let m;
|
||||||
|
if ((m = /^(\d{4})$/.exec(text))) {
|
||||||
|
const y = Number(m[1]);
|
||||||
|
return [new Date(Date.UTC(y, 0, 1)), new Date(Date.UTC(y + 1, 0, 1))];
|
||||||
|
}
|
||||||
|
if ((m = /^(\d{4})-(\d{2})$/.exec(text))) {
|
||||||
|
const [y, mo] = [Number(m[1]), Number(m[2])];
|
||||||
|
if (mo < 1 || mo > 12) throw refuse();
|
||||||
|
return [new Date(Date.UTC(y, mo - 1, 1)), new Date(Date.UTC(y, mo, 1))];
|
||||||
|
}
|
||||||
|
if ((m = /^(\d{4})-Q([1-4])$/.exec(text))) {
|
||||||
|
const [y, q] = [Number(m[1]), Number(m[2])];
|
||||||
|
return [new Date(Date.UTC(y, 3 * (q - 1), 1)), new Date(Date.UTC(y, 3 * q, 1))];
|
||||||
|
}
|
||||||
|
if ((m = /^(\d{4})-W(\d{2})$/.exec(text))) {
|
||||||
|
const [y, w] = [Number(m[1]), Number(m[2])];
|
||||||
|
const weeks = (isoWeekOne(y + 1) - isoWeekOne(y)) / (7 * DAY_MS);
|
||||||
|
if (w < 1 || w > weeks) throw refuse();
|
||||||
|
const monday = isoWeekOne(y) + (w - 1) * 7 * DAY_MS;
|
||||||
|
return [new Date(monday), new Date(monday + 7 * DAY_MS)];
|
||||||
|
}
|
||||||
|
throw refuse();
|
||||||
|
}
|
||||||
|
|
||||||
|
/** gmail's `newer_than:` amount (`7d`, `2w`, `1m`, `1y`) counted back from
|
||||||
|
* `now`; months and years are calendar steps */
|
||||||
|
function parseNewerThan(text: string, now: Date): Date {
|
||||||
|
const m = /^(\d{1,4})([dwmy])$/.exec(text);
|
||||||
|
if (!m || Number(m[1]) < 1) throw new Refused("newer_than: expected an amount like 7d, 2w, 1m, or 1y");
|
||||||
|
const n = Number(m[1]);
|
||||||
|
const d = new Date(now.getTime());
|
||||||
|
if (m[2] === "d") d.setTime(now.getTime() - n * DAY_MS);
|
||||||
|
if (m[2] === "w") d.setTime(now.getTime() - 7 * n * DAY_MS);
|
||||||
|
if (m[2] === "m") d.setUTCMonth(d.getUTCMonth() - n);
|
||||||
|
if (m[2] === "y") d.setUTCFullYear(d.getUTCFullYear() - n);
|
||||||
|
return d;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** `days` as a whole number from 1 to 3650 */
|
||||||
|
function parseDays(days: number | string): number {
|
||||||
|
const n = typeof days === "number" ? days : /^\d+$/.test(days) ? Number(days) : NaN;
|
||||||
|
if (!Number.isInteger(n) || n < 1 || n > 3650) throw new Refused("days: expected a whole number from 1 to 3650");
|
||||||
|
return n;
|
||||||
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* resolve a filter: fields and `q` operators combine; a bound given twice
|
* resolve a filter: fields and `q` operators combine; a bound given twice
|
||||||
* (by fields, operators, or both) is refused, as is an empty window.
|
* (by fields, operators, or both) is refused, as is an empty window.
|
||||||
@@ -75,21 +137,128 @@ export type QueryError = { error: string };
|
|||||||
* not read ambiently, so tests pin it)
|
* not read ambiently, so tests pin it)
|
||||||
*/
|
*/
|
||||||
export function toQuery(spec: FilterSpec, now: Date): Query | QueryError {
|
export function toQuery(spec: FilterSpec, now: Date): Query | QueryError {
|
||||||
void spec;
|
try {
|
||||||
void now;
|
return resolve(spec, now);
|
||||||
throw new Error("toQuery: not implemented (goal 1d)");
|
} catch (err) {
|
||||||
|
if (err instanceof Refused) return { error: err.message };
|
||||||
|
throw err;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
function resolve(spec: FilterSpec, now: Date): Query {
|
||||||
|
const words: Term[] = [];
|
||||||
|
const tags: Term[] = [];
|
||||||
|
let by: { value: Query["by"]; from: string } | undefined;
|
||||||
|
let after: { value: Date; from: string } | undefined;
|
||||||
|
let before: { value: Date; from: string } | undefined;
|
||||||
|
|
||||||
|
const setBy = (value: Query["by"], from: string) => {
|
||||||
|
if (by && by.value !== value) throw new Refused(`${by.from} and ${from} name different dates`);
|
||||||
|
by = { value, from };
|
||||||
|
};
|
||||||
|
const setAfter = (value: Date, from: string) => {
|
||||||
|
if (after) throw new Refused(`${after.from} and ${from} both set where the window starts`);
|
||||||
|
after = { value, from };
|
||||||
|
};
|
||||||
|
const setBefore = (value: Date, from: string) => {
|
||||||
|
if (before) throw new Refused(`${before.from} and ${from} both set where the window ends`);
|
||||||
|
before = { value, from };
|
||||||
|
};
|
||||||
|
const setPeriod = ([lo, hi]: [Date, Date], from: string) => {
|
||||||
|
setAfter(lo, from);
|
||||||
|
setBefore(hi, from);
|
||||||
|
};
|
||||||
|
|
||||||
|
const present = (v: string | number | undefined) => v !== undefined && v !== "";
|
||||||
|
if (present(spec.by)) {
|
||||||
|
if (spec.by !== "updated" && spec.by !== "created") throw new Refused("by: expected updated or created");
|
||||||
|
setBy(spec.by, `by=${spec.by}`);
|
||||||
|
}
|
||||||
|
if (present(spec.days)) setAfter(new Date(now.getTime() - parseDays(spec.days!) * DAY_MS), "days");
|
||||||
|
if (present(spec.after)) setAfter(parseDay(spec.after!, "after"), "after");
|
||||||
|
if (present(spec.before)) setBefore(parseDay(spec.before!, "before"), "before");
|
||||||
|
if (present(spec.in)) setPeriod(parsePeriod(spec.in!, "in"), "in");
|
||||||
|
|
||||||
|
for (const m of (spec.q ?? "").matchAll(/(-?)"([^"]*)"|(\S+)/g)) {
|
||||||
|
if (m[2] !== undefined) {
|
||||||
|
if (m[2].trim()) words.push({ text: m[2].trim().toLowerCase(), negated: m[1] === "-" });
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
const token = m[3];
|
||||||
|
const negated = token.length > 1 && token.startsWith("-");
|
||||||
|
const body = negated ? token.slice(1) : token;
|
||||||
|
const op = /^([a-z_]+):(.+)$/i.exec(body);
|
||||||
|
const name = op?.[1].toLowerCase();
|
||||||
|
const arg = op?.[2] ?? "";
|
||||||
|
const windowOp = ["after", "before", "newer_than", "during", "in", "created", "updated"];
|
||||||
|
if (name && windowOp.includes(name) && negated) throw new Refused(`${name}: can't be negated`);
|
||||||
|
if (name === "after") setAfter(parseDay(arg, "after:"), "after:");
|
||||||
|
else if (name === "before") setBefore(parseDay(arg, "before:"), "before:");
|
||||||
|
else if (name === "newer_than") setAfter(parseNewerThan(arg, now), "newer_than:");
|
||||||
|
else if (name === "during" || name === "in") setPeriod(parsePeriod(arg, `${name}:`), `${name}:`);
|
||||||
|
else if (name === "created" || name === "updated") {
|
||||||
|
setBy(name, `${name}:`);
|
||||||
|
setPeriod(parsePeriod(arg, `${name}:`), `${name}:`);
|
||||||
|
} else if (name === "tag") tags.push({ text: arg.toLowerCase(), negated });
|
||||||
|
else if (body.length > 1 && body.startsWith("#")) tags.push({ text: body.slice(1).toLowerCase(), negated });
|
||||||
|
else words.push({ text: body.toLowerCase(), negated });
|
||||||
|
}
|
||||||
|
|
||||||
|
if (after && before && after.value.getTime() >= before.value.getTime()) {
|
||||||
|
throw new Refused(`the window is empty: ${after.from} is not earlier than ${before.from}`);
|
||||||
|
}
|
||||||
|
const query: Query = { by: by?.value ?? "updated", words, tags };
|
||||||
|
if (after) query.after = after.value;
|
||||||
|
if (before) query.before = before.value;
|
||||||
|
return query;
|
||||||
}
|
}
|
||||||
|
|
||||||
/** {@link toQuery} over the public page's URL params */
|
/** {@link toQuery} over the public page's URL params */
|
||||||
export function parseQuery(params: URLSearchParams, now: Date): Query | QueryError {
|
export function parseQuery(params: URLSearchParams, now: Date): Query | QueryError {
|
||||||
void params;
|
const field = (name: string) => params.get(name) ?? "";
|
||||||
void now;
|
const by = field("by");
|
||||||
throw new Error("parseQuery: not implemented (goal 1d)");
|
return toQuery(
|
||||||
|
{
|
||||||
|
days: field("days"),
|
||||||
|
after: field("after"),
|
||||||
|
before: field("before"),
|
||||||
|
in: field("in"),
|
||||||
|
by: by as FilterSpec["by"],
|
||||||
|
q: field("q"),
|
||||||
|
},
|
||||||
|
now,
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
/** does the query filter anything? (an empty one lists diigo's pages as stored) */
|
||||||
|
export function isEmpty(query: Query): boolean {
|
||||||
|
return !query.after && !query.before && !query.words.length && !query.tags.length && query.by === "updated";
|
||||||
|
}
|
||||||
|
|
||||||
|
/** the item's date that `by` names, in epoch seconds (diigo's unit), or
|
||||||
|
* undefined when it carries none */
|
||||||
|
export function dateOf(item: Item, by: Query["by"]): number | undefined {
|
||||||
|
const secs = by === "created" ? item.created_at : item.updated_at;
|
||||||
|
return typeof secs === "number" && Number.isFinite(secs) ? secs : undefined;
|
||||||
}
|
}
|
||||||
|
|
||||||
/** does `item` fall in the window and satisfy every term? */
|
/** does `item` fall in the window and satisfy every term? */
|
||||||
export function matches(item: Item, query: Query): boolean {
|
export function matches(item: Item, query: Query): boolean {
|
||||||
void item;
|
if (query.after || query.before) {
|
||||||
void query;
|
const secs = dateOf(item, query.by);
|
||||||
throw new Error("matches: not implemented (goal 1d)");
|
if (secs === undefined) return false;
|
||||||
|
if (query.after && secs * 1000 < query.after.getTime()) return false;
|
||||||
|
if (query.before && secs * 1000 >= query.before.getTime()) return false;
|
||||||
|
}
|
||||||
|
const text = (v: unknown) => (typeof v === "string" ? v : "");
|
||||||
|
// fields joined by newline, so a phrase never spans two of them
|
||||||
|
const haystack = [item.title, item.description, item.url].map(text).join("\n").toLowerCase();
|
||||||
|
if (!query.words.every((w) => haystack.includes(w.text) !== w.negated)) return false;
|
||||||
|
const itemTags = new Set(
|
||||||
|
text(item.tags)
|
||||||
|
.split(",")
|
||||||
|
.map((t) => t.trim().toLowerCase())
|
||||||
|
.filter(Boolean),
|
||||||
|
);
|
||||||
|
return query.tags.every((t) => itemTags.has(t.text) !== t.negated);
|
||||||
}
|
}
|
||||||
@@ -41,6 +41,12 @@ export function makeReader(sql: Sql) {
|
|||||||
deepestPageSince(cutoff: number): number | null {
|
deepestPageSince(cutoff: number): number | null {
|
||||||
return run<{ last: number | null }>("deepestPageSince", cutoff)[0]?.last ?? null;
|
return run<{ last: number | null }>("deepestPageSince", cutoff)[0]?.last ?? null;
|
||||||
},
|
},
|
||||||
|
/** stored items whose `by` date (epoch seconds) is in [`from`, `to`),
|
||||||
|
* newest first; the raw dump rows (JSON strings) */
|
||||||
|
itemsIn(by: "updated" | "created", from: number, to: number): string[] {
|
||||||
|
const name = by === "created" ? "itemsByCreated" : "itemsByUpdated";
|
||||||
|
return run<{ json: string }>(name, from, to).map((r) => r.json);
|
||||||
|
},
|
||||||
/** how many cached pages are numbered `pageNum` or lower */
|
/** how many cached pages are numbered `pageNum` or lower */
|
||||||
pagesThrough(pageNum: number): number {
|
pagesThrough(pageNum: number): number {
|
||||||
return Number(run<{ n: number }>("pagesThrough", pageNum)[0]?.n ?? 0);
|
return Number(run<{ n: number }>("pagesThrough", pageNum)[0]?.n ?? 0);
|
||||||
|
|||||||
@@ -3,8 +3,8 @@ import { evictDurableObject, runInDurableObject } from "cloudflare:test";
|
|||||||
import { describe, expect, it, vi } from "vitest";
|
import { describe, expect, it, vi } from "vitest";
|
||||||
import worker from "../src/index.js";
|
import worker from "../src/index.js";
|
||||||
import { idFromWebkey, toWebkey } from "../src/webkeys.js";
|
import { idFromWebkey, toWebkey } from "../src/webkeys.js";
|
||||||
import { FETCH_BUDGET } from "../src/query.js";
|
|
||||||
import { PAGE_SIZE } from "../src/cache.js";
|
import { PAGE_SIZE } from "../src/cache.js";
|
||||||
|
import { mf2 } from "microformats-parser";
|
||||||
|
|
||||||
// matches tests/fake-webapp.js: 30 items, every third private
|
// matches tests/fake-webapp.js: 30 items, every third private
|
||||||
const WEBAPP_ID = "fake-webapp-id";
|
const WEBAPP_ID = "fake-webapp-id";
|
||||||
@@ -250,7 +250,7 @@ describe("worker routes", () => {
|
|||||||
// PAGE_SIZE 100: page 2 is bookmarks 101..200
|
// PAGE_SIZE 100: page 2 is bookmarks 101..200
|
||||||
expect(html).toContain(">bookmark 130<");
|
expect(html).toContain(">bookmark 130<");
|
||||||
// the date comes first, in toISOString() form
|
// the date comes first, in toISOString() form
|
||||||
expect(html).toMatch(/<li><time datetime="\d{4}-\d\d-\d\dT[\d:.]+Z">[^<]+<\/time> <a /);
|
expect(html).toMatch(/<li class="h-entry"><time class="dt-updated" datetime="\d{4}-\d\d-\d\dT[\d:.]+Z">[^<]+<\/time> <a /);
|
||||||
expect(html).not.toContain(">bookmark 129<"); // private (129 % 3 === 0)
|
expect(html).not.toContain(">bookmark 129<"); // private (129 % 3 === 0)
|
||||||
|
|
||||||
const json = (await (
|
const json = (await (
|
||||||
@@ -283,30 +283,34 @@ function isoWeek(d: Date): string {
|
|||||||
return `${year}-W${String(week).padStart(2, "0")}`;
|
return `${year}-W${String(week).padStart(2, "0")}`;
|
||||||
}
|
}
|
||||||
|
|
||||||
describe.skip("public page filters (goal 1d)", () => {
|
describe("public page filters (goal 1d)", () => {
|
||||||
// the fake's dated account: "dated n" updated n-1 days + 1h ago;
|
// the fake's dated account: "dated n" updated n-1 days + 1h ago;
|
||||||
// #weekly on every 7th; every 10th private; every 5th created 100
|
// #weekly on every 7th; every 10th private; every 5th created 100
|
||||||
// days before it was updated
|
// days before it was updated. The public page only reads, so each
|
||||||
|
// test stores the pages it needs on the owner's side first.
|
||||||
const call = (path: string, init?: RequestInit) =>
|
const call = (path: string, init?: RequestInit) =>
|
||||||
worker.fetch(new Request(`https://diigo-cache.test${path}`, init), env as never);
|
worker.fetch(new Request(`https://diigo-cache.test${path}`, init), env as never);
|
||||||
const wireDated = async () => {
|
/** wire a public cache to `webappId` and store diigo pages 1..`pages` */
|
||||||
|
const wireStored = async (webappId: string, pages: number) => {
|
||||||
const res = await call("/_wire", {
|
const res = await call("/_wire", {
|
||||||
method: "POST",
|
method: "POST",
|
||||||
headers: { "content-type": "application/json" },
|
headers: { "content-type": "application/json" },
|
||||||
body: JSON.stringify({ webappId: "fake-dated-webapp", mode: "public" }),
|
body: JSON.stringify({ webappId, mode: "public" }),
|
||||||
});
|
});
|
||||||
expect(res.status).toBe(200);
|
expect(res.status).toBe(200);
|
||||||
return "/public/dated_dan";
|
const { username, publicId } = (await res.json()) as { username: string; publicId: string };
|
||||||
|
const pub = testEnv.PUBLIC_DO.get(testEnv.PUBLIC_DO.idFromString(idFromWebkey(publicId)));
|
||||||
|
for (let n = 1; n <= pages; n += 1) await pub.page(n, PAGE_SIZE);
|
||||||
|
return { page: `/public/${username}`, pub };
|
||||||
};
|
};
|
||||||
|
const wireDated = async () => (await wireStored("fake-dated-webapp", 1)).page;
|
||||||
const titlesIn = (html: string) => [...html.matchAll(/>(dated \d+)<\/a>/g)].map((m) => m[1]);
|
const titlesIn = (html: string) => [...html.matchAll(/>(dated \d+)<\/a>/g)].map((m) => m[1]);
|
||||||
const dated = (...ns: number[]) => ns.map((n) => `dated ${n}`);
|
const dated = (...ns: number[]) => ns.map((n) => `dated ${n}`);
|
||||||
|
|
||||||
it("?days=7 lists the last week, walking only the diigo pages it needs", async () => {
|
it("?days=7 lists the last week", async () => {
|
||||||
const page = await wireDated();
|
const page = await wireDated();
|
||||||
const before = await testEnv.WEBAPP.calls();
|
|
||||||
const html = await (await call(`${page}?days=7`)).text();
|
const html = await (await call(`${page}?days=7`)).text();
|
||||||
expect(titlesIn(html)).toEqual(dated(1, 2, 3, 4, 5, 6, 7));
|
expect(titlesIn(html)).toEqual(dated(1, 2, 3, 4, 5, 6, 7));
|
||||||
expect((await testEnv.WEBAPP.calls()) - before).toBeLessThanOrEqual(1);
|
|
||||||
});
|
});
|
||||||
|
|
||||||
it("?by=created windows, lists, and dates by when it was bookmarked", async () => {
|
it("?by=created windows, lists, and dates by when it was bookmarked", async () => {
|
||||||
@@ -327,13 +331,7 @@ describe.skip("public page filters (goal 1d)", () => {
|
|||||||
});
|
});
|
||||||
|
|
||||||
it("?days=300 pages the matches PAGE_SIZE at a time; links keep the filter", async () => {
|
it("?days=300 pages the matches PAGE_SIZE at a time; links keep the filter", async () => {
|
||||||
const res = await call("/_wire", {
|
const { page } = await wireStored("fake-deep-webapp", 3);
|
||||||
method: "POST",
|
|
||||||
headers: { "content-type": "application/json" },
|
|
||||||
body: JSON.stringify({ webappId: "fake-deep-webapp", mode: "public" }),
|
|
||||||
});
|
|
||||||
expect(res.status).toBe(200);
|
|
||||||
const page = "/public/deep_dee";
|
|
||||||
// deep_dee: one a day; days=300 holds 1..300, minus every 10th (private)
|
// deep_dee: one a day; days=300 holds 1..300, minus every 10th (private)
|
||||||
const want = Array.from({ length: 300 }, (_, k) => k + 1).filter((n) => n % 10 !== 0);
|
const want = Array.from({ length: 300 }, (_, k) => k + 1).filter((n) => n % 10 !== 0);
|
||||||
const first = await (await call(`${page}?days=300`)).text();
|
const first = await (await call(`${page}?days=300`)).text();
|
||||||
@@ -354,6 +352,15 @@ describe.skip("public page filters (goal 1d)", () => {
|
|||||||
expect(titlesIn(html)).toEqual(dated(11, 12));
|
expect(titlesIn(html)).toEqual(dated(11, 12));
|
||||||
});
|
});
|
||||||
|
|
||||||
|
it("?in= names an ISO week", async () => {
|
||||||
|
const { page } = await wireStored("fake-deep-webapp", 3);
|
||||||
|
const week = isoWeek(new Date(Date.now() - 100 * 24 * 60 * 60 * 1000));
|
||||||
|
const html = await (await call(`${page}?in=${week}`)).text();
|
||||||
|
// one a day; every 10th is private
|
||||||
|
expect(titlesIn(html).length).toBeGreaterThanOrEqual(6);
|
||||||
|
expect(titlesIn(html).length).toBeLessThanOrEqual(7);
|
||||||
|
});
|
||||||
|
|
||||||
it("?q=#weekly&days=60 filters by tag; private items stay out", async () => {
|
it("?q=#weekly&days=60 filters by tag; private items stay out", async () => {
|
||||||
const page = await wireDated();
|
const page = await wireDated();
|
||||||
const html = await (await call(`${page}?days=60&q=${encodeURIComponent("#weekly")}`)).text();
|
const html = await (await call(`${page}?days=60&q=${encodeURIComponent("#weekly")}`)).text();
|
||||||
@@ -366,22 +373,49 @@ describe.skip("public page filters (goal 1d)", () => {
|
|||||||
expect(titlesIn(html)).toEqual(dated(7, 14, 21, 28, 35, 42, 49, 56));
|
expect(titlesIn(html)).toEqual(dated(7, 14, 21, 28, 35, 42, 49, 56));
|
||||||
});
|
});
|
||||||
|
|
||||||
it("a request spends at most FETCH_BUDGET diigo fetches; incomplete results say so, and a reload continues", async () => {
|
it("a filter reads only stored items, never diigo; results say whether storage is complete", async () => {
|
||||||
const res = await call("/_wire", {
|
const { page, pub } = await wireStored("fake-deep-webapp", 1);
|
||||||
method: "POST",
|
const json = async () =>
|
||||||
headers: { "content-type": "application/json" },
|
(await call(`${page}?days=300`, { headers: { accept: "application/json" } })).json() as Promise<{
|
||||||
body: JSON.stringify({ webappId: "fake-deep-webapp", mode: "public" }),
|
matched: number;
|
||||||
});
|
complete: boolean;
|
||||||
expect(res.status).toBe(200);
|
}>;
|
||||||
// deep_dee: 60 diigo pages of 100; a search matching nothing must walk them all
|
|
||||||
const before = await testEnv.WEBAPP.calls();
|
const before = await testEnv.WEBAPP.calls();
|
||||||
const first = await call("/public/deep_dee?q=nomatch", { headers: { accept: "application/json" } });
|
// page 1 holds 1..100, minus 10 private
|
||||||
expect(await first.json()).toMatchObject({ complete: false, items: [] });
|
expect(await json()).toMatchObject({ matched: 90, complete: false });
|
||||||
expect((await testEnv.WEBAPP.calls()) - before).toBeLessThanOrEqual(FETCH_BUDGET + 1);
|
expect(await (await call(`${page}?days=300`)).text()).toMatch(/some matches may be missing/);
|
||||||
const html = await (await call("/public/deep_dee?q=nomatch")).text();
|
expect(await testEnv.WEBAPP.calls()).toBe(before);
|
||||||
expect(html).toMatch(/complete|no matches/);
|
await pub.page(2, PAGE_SIZE);
|
||||||
const again = await call("/public/deep_dee?q=nomatch", { headers: { accept: "application/json" } });
|
await pub.page(3, PAGE_SIZE);
|
||||||
expect(await again.json()).toMatchObject({ complete: true, items: [] });
|
expect(await json()).toMatchObject({ matched: 270, complete: true });
|
||||||
|
});
|
||||||
|
|
||||||
|
it("the page is an h-feed of h-entry bookmarks: microformats are its API for other sites", async () => {
|
||||||
|
const page = await wireDated();
|
||||||
|
const html = await (await call(`${page}?days=7&by=created`)).text();
|
||||||
|
const base = `https://diigo-cache.test${page}`;
|
||||||
|
const { items, rels } = mf2(html, { baseUrl: base });
|
||||||
|
const feed = items.find((i) => i.type?.includes("h-feed"));
|
||||||
|
const entries = feed?.children ?? [];
|
||||||
|
expect(entries.map((e) => e.properties.name?.[0])).toEqual(dated(1, 2, 3, 4, 6, 7));
|
||||||
|
const seven = entries[5].properties;
|
||||||
|
expect(seven["bookmark-of"]).toEqual(["https://dated.example/7"]);
|
||||||
|
expect(seven.summary).toEqual(["a week of fragments"]);
|
||||||
|
expect(seven.category).toEqual(["ocap", "weekly"]);
|
||||||
|
const published = new Date(String(seven.published?.[0])).getTime();
|
||||||
|
const updated = new Date(String(seven.updated?.[0])).getTime();
|
||||||
|
// 7 isn't a 5: created when updated, 6 days + 1h ago
|
||||||
|
expect(published).toBe(updated);
|
||||||
|
expect(Date.now() - published).toBeGreaterThan(6 * 24 * 60 * 60 * 1000);
|
||||||
|
// one page of matches: no rel=next
|
||||||
|
expect(rels.next).toBeUndefined();
|
||||||
|
});
|
||||||
|
|
||||||
|
it("filtered pages link the next page with rel=next, keeping the filter", async () => {
|
||||||
|
const { page } = await wireStored("fake-deep-webapp", 3);
|
||||||
|
const html = await (await call(`${page}?days=300`)).text();
|
||||||
|
const { rels } = mf2(html, { baseUrl: `https://diigo-cache.test${page}?days=300` });
|
||||||
|
expect(rels.next).toEqual([`https://diigo-cache.test${page}?days=300&page=2`]);
|
||||||
});
|
});
|
||||||
|
|
||||||
it("JSON answers carry the canonical filter, so devs see what was understood", async () => {
|
it("JSON answers carry the canonical filter, so devs see what was understood", async () => {
|
||||||
@@ -393,25 +427,6 @@ describe.skip("public page filters (goal 1d)", () => {
|
|||||||
});
|
});
|
||||||
});
|
});
|
||||||
|
|
||||||
it("?in= an old week costs O(log pages) diigo fetches with by=updated, not a walk from page 1", async () => {
|
|
||||||
const res = await call("/_wire", {
|
|
||||||
method: "POST",
|
|
||||||
headers: { "content-type": "application/json" },
|
|
||||||
body: JSON.stringify({ webappId: "fake-deep-webapp", mode: "public" }),
|
|
||||||
});
|
|
||||||
expect(res.status).toBe(200);
|
|
||||||
// deep_dee: 6000 bookmarks, one a day, 60 diigo pages of 100; a week
|
|
||||||
// ~14 years back sits near page 51 — a walk from page 1 would cost ~51
|
|
||||||
const longAgo = new Date(Date.now() - 5100 * 24 * 60 * 60 * 1000);
|
|
||||||
const week = isoWeek(longAgo);
|
|
||||||
const before = await testEnv.WEBAPP.calls();
|
|
||||||
const html = await (await call(`/public/deep_dee?in=${week}`)).text();
|
|
||||||
// one a day; every 10th is private
|
|
||||||
expect(titlesIn(html).length).toBeGreaterThanOrEqual(6);
|
|
||||||
expect(titlesIn(html).length).toBeLessThanOrEqual(7);
|
|
||||||
expect((await testEnv.WEBAPP.calls()) - before).toBeLessThanOrEqual(12);
|
|
||||||
});
|
|
||||||
|
|
||||||
it("a malformed filter is a 400 with a message, not an empty list", async () => {
|
it("a malformed filter is a 400 with a message, not an empty list", async () => {
|
||||||
const page = await wireDated();
|
const page = await wireDated();
|
||||||
const res = await call(`${page}?days=7x`);
|
const res = await call(`${page}?days=7x`);
|
||||||
|
|||||||
@@ -18,9 +18,9 @@ const OVERSTATEMENT = 60;
|
|||||||
* 5th an old bookmark edited recently (created 100 days before updated) */
|
* 5th an old bookmark edited recently (created 100 days before updated) */
|
||||||
const DATED_WEBAPP = "fake-dated-webapp";
|
const DATED_WEBAPP = "fake-dated-webapp";
|
||||||
const DATED_TOTAL = 60;
|
const DATED_TOTAL = 60;
|
||||||
/** the same shape, 6000 deep (~16 years, 60 pages of 100): for walk cost */
|
/** the same shape, 300 deep (3 pages of 100): for paging matches */
|
||||||
const DEEP_WEBAPP = "fake-deep-webapp";
|
const DEEP_WEBAPP = "fake-deep-webapp";
|
||||||
const DEEP_TOTAL = 6000;
|
const DEEP_TOTAL = 300;
|
||||||
const HOUR = 60 * 60 * 1000;
|
const HOUR = 60 * 60 * 1000;
|
||||||
|
|
||||||
function datedItem(n, now, user = "dated_dan") {
|
function datedItem(n, now, user = "dated_dan") {
|
||||||
|
|||||||
@@ -2,7 +2,7 @@ import { describe, expect, it } from "vitest";
|
|||||||
import { matches, parseQuery, toQuery } from "../src/query.js";
|
import { matches, parseQuery, toQuery } from "../src/query.js";
|
||||||
import type { FilterSpec, Query, QueryError } from "../src/query.js";
|
import type { FilterSpec, Query, QueryError } from "../src/query.js";
|
||||||
|
|
||||||
// goal 1d (capsite checklist): the design as tests; unskip as it lands
|
// goal 1d (capsite checklist)
|
||||||
|
|
||||||
const NOW = new Date("2026-10-07T12:00:00.000Z");
|
const NOW = new Date("2026-10-07T12:00:00.000Z");
|
||||||
const DAY = 24 * 60 * 60 * 1000;
|
const DAY = 24 * 60 * 60 * 1000;
|
||||||
@@ -21,7 +21,7 @@ const refused = (q: Query | QueryError) => {
|
|||||||
const day = (iso: string) => new Date(`${iso}T00:00:00.000Z`);
|
const day = (iso: string) => new Date(`${iso}T00:00:00.000Z`);
|
||||||
const range = (q: Query) => [q.after?.toISOString().slice(0, 10), q.before?.toISOString().slice(0, 10)];
|
const range = (q: Query) => [q.after?.toISOString().slice(0, 10), q.before?.toISOString().slice(0, 10)];
|
||||||
|
|
||||||
describe.skip("filters: one spec, two spellings (goal 1d)", () => {
|
describe("filters: one spec, two spellings (goal 1d)", () => {
|
||||||
it("devs write { days: 7 }; URLs write ?days=7; same query", () => {
|
it("devs write { days: 7 }; URLs write ?days=7; same query", () => {
|
||||||
expect(spec({ days: 7 })).toEqual(url("days=7"));
|
expect(spec({ days: 7 })).toEqual(url("days=7"));
|
||||||
expect(spec({ days: 7 }).after).toEqual(new Date(NOW.getTime() - 7 * DAY));
|
expect(spec({ days: 7 }).after).toEqual(new Date(NOW.getTime() - 7 * DAY));
|
||||||
@@ -42,7 +42,7 @@ describe.skip("filters: one spec, two spellings (goal 1d)", () => {
|
|||||||
});
|
});
|
||||||
});
|
});
|
||||||
|
|
||||||
describe.skip("windows: UTC, half-open (goal 1d)", () => {
|
describe("windows: UTC, half-open (goal 1d)", () => {
|
||||||
it("after/before are UTC days: after inclusive from 00:00Z, before exclusive", () => {
|
it("after/before are UTC days: after inclusive from 00:00Z, before exclusive", () => {
|
||||||
expect(range(url("after=2026-01-01&before=2026-02-01"))).toEqual(["2026-01-01", "2026-02-01"]);
|
expect(range(url("after=2026-01-01&before=2026-02-01"))).toEqual(["2026-01-01", "2026-02-01"]);
|
||||||
expect(url("after=2026/01/01").after).toEqual(day("2026-01-01"));
|
expect(url("after=2026/01/01").after).toEqual(day("2026-01-01"));
|
||||||
@@ -88,7 +88,7 @@ describe.skip("windows: UTC, half-open (goal 1d)", () => {
|
|||||||
});
|
});
|
||||||
});
|
});
|
||||||
|
|
||||||
describe.skip("q: terms and operators (goal 1d)", () => {
|
describe("q: terms and operators (goal 1d)", () => {
|
||||||
const q = (text: string) => url(`q=${encodeURIComponent(text)}`);
|
const q = (text: string) => url(`q=${encodeURIComponent(text)}`);
|
||||||
|
|
||||||
it("words, phrases, #tags, tag:, and - negation; case-insensitive", () => {
|
it("words, phrases, #tags, tag:, and - negation; case-insensitive", () => {
|
||||||
@@ -160,7 +160,7 @@ const item = (iso: string, extra: Record<string, unknown> = {}) => ({
|
|||||||
});
|
});
|
||||||
const query = (q: Partial<Query>): Query => ({ by: "updated", words: [], tags: [], ...q });
|
const query = (q: Partial<Query>): Query => ({ by: "updated", words: [], tags: [], ...q });
|
||||||
|
|
||||||
describe.skip("matches (goal 1d)", () => {
|
describe("matches (goal 1d)", () => {
|
||||||
it("the window is half-open: after inclusive, before exclusive", () => {
|
it("the window is half-open: after inclusive, before exclusive", () => {
|
||||||
const q = query({ after: day("2026-01-01"), before: day("2026-02-01") });
|
const q = query({ after: day("2026-01-01"), before: day("2026-02-01") });
|
||||||
expect(matches(item("2026-01-01T00:00:00.000Z"), q)).toBe(true);
|
expect(matches(item("2026-01-01T00:00:00.000Z"), q)).toBe(true);
|
||||||
|
|||||||
@@ -0,0 +1 @@
|
|||||||
|
node_modules
|
||||||
@@ -0,0 +1,21 @@
|
|||||||
|
# diigo-week
|
||||||
|
|
||||||
|
An EmDash plugin that drafts a potluck issue from a week of your shared diigo bookmarks. It reads them as microformats from diigo-cache's public page, so you edit a draft instead of copy-pasting links.
|
||||||
|
|
||||||
|
## Getting started
|
||||||
|
|
||||||
|
Run diigo-cache (with a filled public cache) and potluck, each in its own terminal:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
pnpm install
|
||||||
|
(cd ../diigo-cache && pnpm dev) # → http://localhost:8787/
|
||||||
|
(cd ../potluck && pnpm dev) # → http://localhost:4321/_emdash/admin
|
||||||
|
```
|
||||||
|
|
||||||
|
On potluck's admin dashboard, the **This week's issue** card has a **Draft 2026-Wnn from diigo** button for last week; it links to the draft once made. For any other week: Plugins → **Draft from diigo**. A week that already has an issue isn't drafted again.
|
||||||
|
|
||||||
|
## How it reads the week
|
||||||
|
|
||||||
|
It asks diigo-cache for `/public/<user>?in=<week>&by=created` over the `DIIGO_CACHE` service binding (potluck's `wrangler.jsonc`), reads the page's `h-feed` of `h-entry` bookmarks, follows `rel=next`, and creates a draft in `issues`: title, week, dek, and a paragraph per bookmark with its description and tag links, as in potluck's `seed/issues.json`. The binding is its only reach outside the site; it declares `content:write` and `taxonomies:write` (the issue gets its bookmarks' tags, as the sample issues do), nothing else.
|
||||||
|
|
||||||
|
Checks: `pnpm lint && pnpm test`.
|
||||||
@@ -0,0 +1,24 @@
|
|||||||
|
{
|
||||||
|
"name": "diigo-week",
|
||||||
|
"version": "0.1.0",
|
||||||
|
"private": true,
|
||||||
|
"type": "module",
|
||||||
|
"exports": {
|
||||||
|
".": {
|
||||||
|
"import": "./src/index.ts"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"scripts": {
|
||||||
|
"lint": "tsc --noEmit",
|
||||||
|
"test": "vitest run"
|
||||||
|
},
|
||||||
|
"dependencies": {
|
||||||
|
"emdash": "1.2.0",
|
||||||
|
"microformats-parser": "^2.0.6"
|
||||||
|
},
|
||||||
|
"devDependencies": {
|
||||||
|
"@emdash-cms/blocks": "^1.2.0",
|
||||||
|
"typescript": "^7.0.2",
|
||||||
|
"vitest": "^5.0.3"
|
||||||
|
}
|
||||||
|
}
|
||||||
Generated
+6223
File diff suppressed because it is too large.
Load diff
@@ -0,0 +1,8 @@
|
|||||||
|
packages:
|
||||||
|
- "."
|
||||||
|
|
||||||
|
strictDepBuilds: true
|
||||||
|
dangerouslyAllowAllBuilds: false
|
||||||
|
allowBuilds:
|
||||||
|
esbuild: true
|
||||||
|
workerd: true
|
||||||
Vendored
+7
@@ -0,0 +1,7 @@
|
|||||||
|
/** emdash's virtual module for the site worker's bindings (declared for site
|
||||||
|
* builds in emdash/src/virtual-modules.d.ts), so this package typechecks
|
||||||
|
* standalone: `env` from `cloudflare:workers` under @astrojs/cloudflare,
|
||||||
|
* undefined on other adapters */
|
||||||
|
declare module "virtual:emdash/env" {
|
||||||
|
export const env: Record<string, unknown> | undefined;
|
||||||
|
}
|
||||||
@@ -0,0 +1,180 @@
|
|||||||
|
/**
|
||||||
|
* @file diigo-week: an EmDash plugin that drafts a potluck issue from a
|
||||||
|
* week of public diigo bookmarks.
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { definePlugin } from "emdash";
|
||||||
|
import type { PluginDefinition, RouteContext } from "emdash";
|
||||||
|
import type { Block, BlockResponse } from "@emdash-cms/blocks";
|
||||||
|
import { draftIssue, issueTags, lastWeek, readWeek } from "./week.js";
|
||||||
|
|
||||||
|
/** the diigo username drafts read from, until the admin page saves another */
|
||||||
|
const DEFAULT_USER = "dckc-madmode";
|
||||||
|
|
||||||
|
/** something that answers HTTP requests: a service binding's `fetch` */
|
||||||
|
type Fetcher = { fetch(request: Request): Promise<Response> };
|
||||||
|
|
||||||
|
/**
|
||||||
|
* the `DIIGO_CACHE` service binding to diigo-cache, from the site worker's
|
||||||
|
* env. The binding is the plugin's only reach outside the site:
|
||||||
|
* `ctx.http.fetch` can't reach a dev server (it refuses localhost), and a
|
||||||
|
* binding names one worker rather than a host.
|
||||||
|
*/
|
||||||
|
async function diigoCache(): Promise<Fetcher> {
|
||||||
|
const { env } = await import("virtual:emdash/env");
|
||||||
|
const binding = env?.DIIGO_CACHE as Fetcher | undefined;
|
||||||
|
if (typeof binding?.fetch !== "function") {
|
||||||
|
throw new Error('no DIIGO_CACHE service binding: add { "binding": "DIIGO_CACHE", "service": "diigo-cache" } to services in wrangler.jsonc');
|
||||||
|
}
|
||||||
|
return binding;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** GET a diigo-cache path as HTML; anything but a 200 is an error */
|
||||||
|
async function getter(cache: Fetcher): Promise<(path: string) => Promise<string>> {
|
||||||
|
return async (path) => {
|
||||||
|
const res = await cache.fetch(new Request(`https://diigo-cache${path}`, { headers: { accept: "text/html" } }));
|
||||||
|
const text = await res.text();
|
||||||
|
if (res.status !== 200) throw new Error(`diigo-cache ${path.split("?")[0]}: ${res.status} ${text.slice(0, 200)}`);
|
||||||
|
return text;
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* assign `tags` to an issue in the `tag` taxonomy (as seed/issues.json
|
||||||
|
* does), creating terms that don't exist yet; returns the tags that
|
||||||
|
* couldn't be assigned, with why
|
||||||
|
*/
|
||||||
|
async function tagIssue(ctx: RouteContext, id: string, tags: string[]): Promise<string[]> {
|
||||||
|
const tax = ctx.taxonomies;
|
||||||
|
if (typeof tax?.createTerm !== "function" || typeof tax.addEntryTerms !== "function") throw new Error("this plugin lacks taxonomies:write");
|
||||||
|
const known = new Map((await tax.getTerms("tag")).map((t) => [t.slug, t.id]));
|
||||||
|
const ids: string[] = [];
|
||||||
|
const failed: string[] = [];
|
||||||
|
for (const tag of tags) {
|
||||||
|
try {
|
||||||
|
ids.push(known.get(tag) ?? (await tax.createTerm("tag", { label: tag, slug: tag })).id);
|
||||||
|
} catch (error) {
|
||||||
|
failed.push(`${tag} (${error instanceof Error ? error.message : String(error)})`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if (ids.length) await tax.addEntryTerms("issues", id, "tag", ids);
|
||||||
|
return failed;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** the issue already drafted or published for `week`, if any (issues
|
||||||
|
* are few: a scan of the newest is enough) */
|
||||||
|
async function issueFor(ctx: RouteContext, week: string): Promise<{ id: string; status: string; title: string } | undefined> {
|
||||||
|
const found = (await ctx.content!.list("issues", { limit: 100 })).items.find((it) => it.data.week === week);
|
||||||
|
return found && { id: found.id, status: found.status, title: String(found.data.title ?? week) };
|
||||||
|
}
|
||||||
|
|
||||||
|
/** a button-row link to an issue in the editor */
|
||||||
|
const openIssue = (id: string, label: string): Block => ({
|
||||||
|
type: "actions",
|
||||||
|
elements: [{ type: "link", label, target: { kind: "content", collection: "issues", id }, appearance: "primary" }],
|
||||||
|
});
|
||||||
|
|
||||||
|
/** draft `week` from `user`'s bookmarks, unless the week has an issue
|
||||||
|
* already: blocks that say what happened (with a link), and a toast */
|
||||||
|
async function draft(ctx: RouteContext, user: string, week: string): Promise<{ note: Block[]; toast: BlockResponse["toast"] }> {
|
||||||
|
try {
|
||||||
|
if (!/^[A-Za-z0-9_-]{1,64}$/.test(user)) throw new Error("diigo username: letters, digits, _ and - only");
|
||||||
|
const existing = await issueFor(ctx, week);
|
||||||
|
if (existing) {
|
||||||
|
return {
|
||||||
|
note: [
|
||||||
|
{ type: "banner", variant: "default", title: `${week} already has an issue`, description: `"${existing.title}" (${existing.status}). Delete it to draft the week again.` },
|
||||||
|
openIssue(existing.id, "Open it"),
|
||||||
|
],
|
||||||
|
toast: { message: `${week} already has an issue`, type: "info" },
|
||||||
|
};
|
||||||
|
}
|
||||||
|
const bookmarks = await readWeek(await getter(await diigoCache()), user, week);
|
||||||
|
const issue = draftIssue(week, bookmarks);
|
||||||
|
const content = ctx.content;
|
||||||
|
if (typeof content?.create !== "function") throw new Error("this plugin lacks content:write");
|
||||||
|
const created = await content.create("issues", issue);
|
||||||
|
const untagged = await tagIssue(ctx, created.id, issueTags(bookmarks));
|
||||||
|
if (untagged.length) ctx.log.warn("tags not assigned", { id: created.id, untagged });
|
||||||
|
ctx.log.info("drafted issue", { week, bookmarks: bookmarks.length, id: created.id });
|
||||||
|
return {
|
||||||
|
note: [
|
||||||
|
{ type: "banner", variant: "default", title: `Drafted "${issue.title}"`, description: `${bookmarks.length} bookmarks, as a draft at /issues/${created.slug ?? "(no slug)"}. Cut, comment, and publish.${untagged.length ? ` Tags not assigned: ${untagged.join(", ")}.` : ""}` },
|
||||||
|
openIssue(created.id, "Open the draft"),
|
||||||
|
],
|
||||||
|
toast: { message: `Drafted ${issue.title}`, type: "success" },
|
||||||
|
};
|
||||||
|
} catch (error) {
|
||||||
|
const message = error instanceof Error ? error.message : String(error);
|
||||||
|
ctx.log.error("draft failed", { week, message });
|
||||||
|
return { note: [{ type: "banner", variant: "alert", title: "Couldn't draft the issue", description: message }], toast: { message, type: "error" } };
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/** the admin page: a form for the diigo username and the week */
|
||||||
|
async function page(ctx: RouteContext, note: Block[] = [], toast?: BlockResponse["toast"]): Promise<BlockResponse> {
|
||||||
|
const user = (await ctx.kv.get<string>("user")) ?? DEFAULT_USER;
|
||||||
|
const blocks: Block[] = [
|
||||||
|
{ type: "header", text: "Draft an issue from diigo" },
|
||||||
|
{
|
||||||
|
type: "context",
|
||||||
|
text: "Reads a week of shared bookmarks from diigo-cache (by when they were bookmarked) and saves them as a draft issue for you to edit.",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
type: "form",
|
||||||
|
block_id: "draft",
|
||||||
|
fields: [
|
||||||
|
{ type: "text_input", action_id: "user", label: "diigo username", initial_value: user },
|
||||||
|
{ type: "text_input", action_id: "week", label: "ISO week (UTC)", initial_value: lastWeek(new Date()) },
|
||||||
|
],
|
||||||
|
submit: { label: "Draft issue", action_id: "draft" },
|
||||||
|
},
|
||||||
|
];
|
||||||
|
blocks.push(...note);
|
||||||
|
return { blocks, toast };
|
||||||
|
}
|
||||||
|
|
||||||
|
/** the dashboard widget: last week's issue if there is one, else a button to draft it */
|
||||||
|
async function widget(ctx: RouteContext, note: Block[] = [], toast?: BlockResponse["toast"]): Promise<BlockResponse> {
|
||||||
|
const week = lastWeek(new Date());
|
||||||
|
if (note.length) return { blocks: note, toast };
|
||||||
|
const existing = await issueFor(ctx, week);
|
||||||
|
const blocks: Block[] = existing
|
||||||
|
? [{ type: "section", text: `${week}: "${existing.title}" (${existing.status})` }, openIssue(existing.id, "Open it")]
|
||||||
|
: [
|
||||||
|
{ type: "section", text: `${week} has no issue yet.` },
|
||||||
|
{ type: "actions", elements: [{ type: "button", action_id: "draft-last-week", label: `Draft ${week} from diigo`, style: "primary" }] },
|
||||||
|
];
|
||||||
|
return { blocks, toast };
|
||||||
|
}
|
||||||
|
|
||||||
|
export function createPlugin(): PluginDefinition {
|
||||||
|
return definePlugin({
|
||||||
|
id: "diigo-week",
|
||||||
|
version: "0.1.0",
|
||||||
|
capabilities: ["content:write", "taxonomies:write"],
|
||||||
|
admin: {
|
||||||
|
pages: [{ path: "/draft", label: "Draft from diigo", icon: "bookmark" }],
|
||||||
|
widgets: [{ id: "last-week", title: "This week's issue", size: "half" }],
|
||||||
|
},
|
||||||
|
routes: {
|
||||||
|
admin: {
|
||||||
|
permission: "plugins:manage",
|
||||||
|
handler: async (ctx) => {
|
||||||
|
const input = ctx.input as { type?: string; page?: string; action_id?: string; values?: Record<string, unknown> } | undefined;
|
||||||
|
const user = (await ctx.kv.get<string>("user")) ?? DEFAULT_USER;
|
||||||
|
if (input?.type === "block_action" && input.action_id === "draft-last-week") {
|
||||||
|
const { note, toast } = await draft(ctx, user, lastWeek(new Date()));
|
||||||
|
return widget(ctx, note, toast);
|
||||||
|
}
|
||||||
|
if (input?.page?.startsWith("widget:")) return widget(ctx);
|
||||||
|
if (input?.type !== "form_submit" || input.action_id !== "draft") return page(ctx);
|
||||||
|
const formUser = String(input.values?.user ?? "").trim();
|
||||||
|
if (/^[A-Za-z0-9_-]{1,64}$/.test(formUser)) await ctx.kv.set("user", formUser);
|
||||||
|
const { note, toast } = await draft(ctx, formUser, String(input.values?.week ?? "").trim());
|
||||||
|
return page(ctx, note, toast);
|
||||||
|
},
|
||||||
|
},
|
||||||
|
},
|
||||||
|
});
|
||||||
|
}
|
||||||
@@ -0,0 +1,146 @@
|
|||||||
|
/**
|
||||||
|
* @file a week of bookmarks, read as microformats from diigo-cache's public
|
||||||
|
* page, drafted as a potluck issue.
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { mf2 } from "microformats-parser";
|
||||||
|
|
||||||
|
const DAY_MS = 24 * 60 * 60 * 1000;
|
||||||
|
|
||||||
|
/** pages of matches one draft will read (100 bookmarks each) */
|
||||||
|
export const MAX_PAGES = 10;
|
||||||
|
|
||||||
|
/** one bookmark, as its h-entry says */
|
||||||
|
export type Bookmark = {
|
||||||
|
url: string;
|
||||||
|
title: string;
|
||||||
|
description: string;
|
||||||
|
tags: string[];
|
||||||
|
/** when it was bookmarked, ISO */
|
||||||
|
published: string;
|
||||||
|
};
|
||||||
|
|
||||||
|
/** `yyyy-Www` for the ISO 8601 week holding `d` (UTC) */
|
||||||
|
export function isoWeek(d: Date): string {
|
||||||
|
const t = new Date(Date.UTC(d.getUTCFullYear(), d.getUTCMonth(), d.getUTCDate()));
|
||||||
|
const day = t.getUTCDay() || 7;
|
||||||
|
t.setUTCDate(t.getUTCDate() + 4 - day); // the week's Thursday names its year
|
||||||
|
const year = t.getUTCFullYear();
|
||||||
|
const week = Math.ceil(((t.getTime() - Date.UTC(year, 0, 1)) / DAY_MS + 1) / 7);
|
||||||
|
return `${year}-W${String(week).padStart(2, "0")}`;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** the last full ISO week before `now` */
|
||||||
|
export function lastWeek(now: Date): string {
|
||||||
|
return isoWeek(new Date(now.getTime() - 7 * DAY_MS));
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Monday 00:00Z of ISO week `week` (`2026-W41`), or undefined if malformed */
|
||||||
|
export function weekStart(week: string): Date | undefined {
|
||||||
|
const m = /^(\d{4})-W(\d{2})$/.exec(week);
|
||||||
|
if (!m) return undefined;
|
||||||
|
const [year, w] = [Number(m[1]), Number(m[2])];
|
||||||
|
const jan4 = Date.UTC(year, 0, 4);
|
||||||
|
const monday = jan4 - ((new Date(jan4).getUTCDay() || 7) - 1) * DAY_MS + (w - 1) * 7 * DAY_MS;
|
||||||
|
const start = new Date(monday);
|
||||||
|
return w >= 1 && isoWeek(start) === week ? start : undefined;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** the first string value of an mf2 property, or "" */
|
||||||
|
const first = (props: Record<string, unknown[]>, name: string): string => {
|
||||||
|
const v = props[name]?.[0];
|
||||||
|
return typeof v === "string" ? v : "";
|
||||||
|
};
|
||||||
|
|
||||||
|
/** the bookmarks in a page's `h-feed`, and its `rel=next` link if any */
|
||||||
|
export function readPage(html: string, url: string): { bookmarks: Bookmark[]; next?: string } {
|
||||||
|
const { items, rels } = mf2(html, { baseUrl: url });
|
||||||
|
const feed = items.find((i) => i.type?.includes("h-feed"));
|
||||||
|
const bookmarks = (feed?.children ?? [])
|
||||||
|
.filter((e) => e.type?.includes("h-entry"))
|
||||||
|
.map((e) => {
|
||||||
|
const props = e.properties as Record<string, unknown[]>;
|
||||||
|
const tags = (props.category ?? []).filter((t): t is string => typeof t === "string");
|
||||||
|
return {
|
||||||
|
url: first(props, "bookmark-of"),
|
||||||
|
title: first(props, "name"),
|
||||||
|
description: first(props, "summary"),
|
||||||
|
tags,
|
||||||
|
published: first(props, "published"),
|
||||||
|
};
|
||||||
|
})
|
||||||
|
.filter((b) => b.url !== "");
|
||||||
|
return { bookmarks, next: rels.next?.[0] };
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* a user's bookmarks from an ISO week, oldest first, by when they were
|
||||||
|
* bookmarked: diigo-cache's `/public/<user>?in=<week>&by=created`,
|
||||||
|
* following `rel=next` up to {@link MAX_PAGES} pages.
|
||||||
|
*
|
||||||
|
* @param get fetches a path on diigo-cache, answering its HTML; it throws
|
||||||
|
* on anything but a 200
|
||||||
|
*/
|
||||||
|
export async function readWeek(get: (path: string) => Promise<string>, user: string, week: string): Promise<Bookmark[]> {
|
||||||
|
const base = "https://diigo-cache.invalid";
|
||||||
|
let path: string | undefined = `/public/${encodeURIComponent(user)}?in=${encodeURIComponent(week)}&by=created`;
|
||||||
|
const bookmarks: Bookmark[] = [];
|
||||||
|
for (let n = 0; path && n < MAX_PAGES; n += 1) {
|
||||||
|
const page = readPage(await get(path), base + path);
|
||||||
|
bookmarks.push(...page.bookmarks);
|
||||||
|
const next: URL | undefined = page.next ? new URL(page.next) : undefined;
|
||||||
|
path = next && next.origin === base ? next.pathname + next.search : undefined;
|
||||||
|
}
|
||||||
|
return bookmarks.reverse();
|
||||||
|
}
|
||||||
|
|
||||||
|
/** a Portable Text span */
|
||||||
|
type Span = { _type: "span"; _key: string; text: string; marks: string[] };
|
||||||
|
/** a Portable Text block */
|
||||||
|
type Block = { _type: "block"; _key: string; style: string; markDefs: { _type: "link"; _key: string; href: string }[]; children: Span[] };
|
||||||
|
|
||||||
|
/** a bookmark as one paragraph: the link, the description, then tag links (as in seed/issues.json) */
|
||||||
|
function bookmarkBlock(b: Bookmark, i: number): Block {
|
||||||
|
const k = `b${i}`;
|
||||||
|
const markDefs: Block["markDefs"] = [{ _type: "link", _key: `${k}-a`, href: b.url }];
|
||||||
|
const children: Span[] = [{ _type: "span", _key: `${k}-t`, text: b.title || b.url, marks: [`${k}-a`] }];
|
||||||
|
if (b.description) children.push({ _type: "span", _key: `${k}-d`, text: `: ${b.description}`, marks: [] });
|
||||||
|
b.tags.forEach((tag, j) => {
|
||||||
|
markDefs.push({ _type: "link", _key: `${k}-g${j}`, href: `/tag/${encodeURIComponent(tag)}` });
|
||||||
|
children.push({ _type: "span", _key: `${k}-s${j}`, text: " ", marks: [] });
|
||||||
|
children.push({ _type: "span", _key: `${k}-n${j}`, text: tag, marks: [`${k}-g${j}`] });
|
||||||
|
});
|
||||||
|
return { _type: "block", _key: k, style: "normal", markDefs, children };
|
||||||
|
}
|
||||||
|
|
||||||
|
/** the issue's tags: every bookmark's, deduplicated, in order of first use */
|
||||||
|
export function issueTags(bookmarks: Bookmark[]): string[] {
|
||||||
|
return [...new Set(bookmarks.flatMap((b) => b.tags))];
|
||||||
|
}
|
||||||
|
|
||||||
|
/** `Oct 5–11` (or `Sep 28–Oct 4`) for the week starting `monday` */
|
||||||
|
function range(monday: Date): string {
|
||||||
|
const sunday = new Date(monday.getTime() + 6 * DAY_MS);
|
||||||
|
const fmt = (d: Date, withMonth: boolean) =>
|
||||||
|
withMonth ? d.toLocaleDateString("en-US", { month: "short", day: "numeric", timeZone: "UTC" }) : String(d.getUTCDate());
|
||||||
|
return `${fmt(monday, true)}–${fmt(sunday, sunday.getUTCMonth() !== monday.getUTCMonth())}`;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** the fields of a draft `issues` entry for `week`: title, week, dek, body */
|
||||||
|
export function draftIssue(week: string, bookmarks: Bookmark[]): Record<string, unknown> {
|
||||||
|
const monday = weekStart(week);
|
||||||
|
if (!monday) throw new Error(`week: expected an ISO week like 2026-W41, got ${JSON.stringify(week)}`);
|
||||||
|
const intro: Block = {
|
||||||
|
_type: "block",
|
||||||
|
_key: "intro",
|
||||||
|
style: "normal",
|
||||||
|
markDefs: [],
|
||||||
|
children: [{ _type: "span", _key: "intro-t", text: "(draft) A line or two about the week goes here.", marks: [] }],
|
||||||
|
};
|
||||||
|
return {
|
||||||
|
title: `Week ${week}`,
|
||||||
|
week,
|
||||||
|
dek: `${range(monday)}: ${bookmarks.length} bookmark${bookmarks.length === 1 ? "" : "s"}. (draft)`,
|
||||||
|
body: [intro, ...bookmarks.map(bookmarkBlock)],
|
||||||
|
};
|
||||||
|
}
|
||||||
@@ -0,0 +1,99 @@
|
|||||||
|
import { describe, expect, it } from "vitest";
|
||||||
|
import { draftIssue, issueTags, lastWeek, readWeek, weekStart } from "../src/week.js";
|
||||||
|
|
||||||
|
/** an h-entry as diigo-cache's public page marks it up (by=created) */
|
||||||
|
const entry = (n: number, created: string, extra = "") =>
|
||||||
|
`<li class="h-entry"><time class="dt-published" datetime="${created}">${created}</time> <a class="u-bookmark-of p-name" href="https://ex.example/${n}">bookmark ${n}</a>${extra}<time class="dt-updated" datetime="${created}" hidden></time></li>`;
|
||||||
|
|
||||||
|
const page = (entries: string[], next?: string) => `<!doctype html><html><body>
|
||||||
|
<p>${next ? `<a rel="next" href="${next}">next →</a>` : "<span>next →</span>"}</p>
|
||||||
|
<ol class="h-feed">
|
||||||
|
${entries.join("\n")}
|
||||||
|
</ol></body></html>`;
|
||||||
|
|
||||||
|
describe("readWeek", () => {
|
||||||
|
it("reads h-entry bookmarks across rel=next pages, oldest first", async () => {
|
||||||
|
const asked: string[] = [];
|
||||||
|
const pages: Record<string, string> = {
|
||||||
|
"/public/dckc-madmode?in=2026-W40&by=created": page(
|
||||||
|
[
|
||||||
|
entry(
|
||||||
|
3,
|
||||||
|
"2026-10-03T12:00:00.000Z",
|
||||||
|
` <a class="p-category" href="/public/dckc-madmode?q=%23ocap">ocap</a> <a class="p-category" href="/public/dckc-madmode?q=%23weekly">weekly</a><br><small class="p-summary">a week of fragments</small>`,
|
||||||
|
),
|
||||||
|
entry(2, "2026-10-01T12:00:00.000Z"),
|
||||||
|
],
|
||||||
|
"/public/dckc-madmode?in=2026-W40&by=created&page=2",
|
||||||
|
),
|
||||||
|
"/public/dckc-madmode?in=2026-W40&by=created&page=2": page([entry(1, "2026-09-28T12:00:00.000Z")]),
|
||||||
|
};
|
||||||
|
const get = async (path: string) => {
|
||||||
|
asked.push(path);
|
||||||
|
const html = pages[path];
|
||||||
|
if (!html) throw new Error(`no page ${path}`);
|
||||||
|
return html;
|
||||||
|
};
|
||||||
|
const bookmarks = await readWeek(get, "dckc-madmode", "2026-W40");
|
||||||
|
expect(asked).toHaveLength(2);
|
||||||
|
expect(bookmarks.map((b) => b.title)).toEqual(["bookmark 1", "bookmark 2", "bookmark 3"]);
|
||||||
|
expect(bookmarks[2]).toEqual({
|
||||||
|
url: "https://ex.example/3",
|
||||||
|
title: "bookmark 3",
|
||||||
|
description: "a week of fragments",
|
||||||
|
tags: ["ocap", "weekly"],
|
||||||
|
published: "2026-10-03T12:00:00.000Z",
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
it("doesn't follow a rel=next to another site", async () => {
|
||||||
|
const asked: string[] = [];
|
||||||
|
const get = async (path: string) => {
|
||||||
|
asked.push(path);
|
||||||
|
return page([entry(1, "2026-10-01T12:00:00.000Z")], "https://elsewhere.example/public/x");
|
||||||
|
};
|
||||||
|
await readWeek(get, "dckc-madmode", "2026-W40");
|
||||||
|
expect(asked).toHaveLength(1);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe("weeks", () => {
|
||||||
|
it("lastWeek is the full ISO week before now", () => {
|
||||||
|
expect(lastWeek(new Date("2026-10-09T01:00:00Z"))).toBe("2026-W40");
|
||||||
|
// Monday 2026-01-05 starts W02; W01 began Monday 2025-12-29
|
||||||
|
expect(lastWeek(new Date("2026-01-05T00:00:00Z"))).toBe("2026-W01");
|
||||||
|
});
|
||||||
|
|
||||||
|
it("weekStart is Monday 00:00Z, and refuses a week that doesn't exist", () => {
|
||||||
|
expect(weekStart("2026-W40")?.toISOString()).toBe("2026-09-28T00:00:00.000Z");
|
||||||
|
expect(weekStart("2025-W01")?.toISOString()).toBe("2024-12-30T00:00:00.000Z");
|
||||||
|
expect(weekStart("2025-W53")).toBeUndefined();
|
||||||
|
expect(weekStart("2026-40")).toBeUndefined();
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe("draftIssue", () => {
|
||||||
|
it("drafts title, week, dek, and a body: an intro, then a paragraph per bookmark with tag links", () => {
|
||||||
|
const issue = draftIssue("2026-W40", [
|
||||||
|
{ url: "https://ex.example/1", title: "one", description: "nice", tags: ["ocap"], published: "2026-09-28T12:00:00.000Z" },
|
||||||
|
]);
|
||||||
|
expect(issue.title).toBe("Week 2026-W40");
|
||||||
|
expect(issue.week).toBe("2026-W40");
|
||||||
|
expect(issue.dek).toBe("Sep 28–Oct 4: 1 bookmark. (draft)");
|
||||||
|
const [intro, b] = issue.body as { markDefs: { href: string }[]; children: { text: string }[] }[];
|
||||||
|
expect(intro.children[0].text).toMatch(/draft/);
|
||||||
|
expect(b.markDefs.map((m) => m.href)).toEqual(["https://ex.example/1", "/tag/ocap"]);
|
||||||
|
expect(b.children.map((c) => c.text).join("")).toBe("one: nice ocap");
|
||||||
|
});
|
||||||
|
|
||||||
|
it("refuses a malformed week", () => {
|
||||||
|
expect(() => draftIssue("last week", [])).toThrow(/ISO week/);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe("issueTags", () => {
|
||||||
|
it("is every bookmark's tags, once each, in order of first use", () => {
|
||||||
|
const b = (tags: string[]) => ({ url: "https://ex.example/", title: "", description: "", tags, published: "" });
|
||||||
|
expect(issueTags([b(["governance", "economics"]), b([]), b(["governance", "scaling"])])).toEqual(["governance", "economics", "scaling"]);
|
||||||
|
});
|
||||||
|
});
|
||||||
@@ -0,0 +1,20 @@
|
|||||||
|
{
|
||||||
|
"compilerOptions": {
|
||||||
|
"target": "ES2022",
|
||||||
|
"module": "preserve",
|
||||||
|
"moduleResolution": "bundler",
|
||||||
|
"strict": true,
|
||||||
|
"esModuleInterop": true,
|
||||||
|
"verbatimModuleSyntax": true,
|
||||||
|
"skipLibCheck": true,
|
||||||
|
"types": []
|
||||||
|
},
|
||||||
|
"include": [
|
||||||
|
"src/**/*",
|
||||||
|
"tests/**/*",
|
||||||
|
"vitest.config.ts"
|
||||||
|
],
|
||||||
|
"exclude": [
|
||||||
|
"node_modules"
|
||||||
|
]
|
||||||
|
}
|
||||||
@@ -0,0 +1,166 @@
|
|||||||
|
Good Fences Make Good Agents
|
||||||
|
|
||||||
|
*with apologies to Robert Frost*
|
||||||
|
|
||||||
|
Let's have a fence and a neighbor. I can talk to the neighbor much like the conventional...
|
||||||
|
|
||||||
|
```js
|
||||||
|
client.chat.completions.create({
|
||||||
|
model: 'llama3.1:8b',
|
||||||
|
messages: [{ role: 'user', content: 'Tell me a story' }],
|
||||||
|
stream: true,
|
||||||
|
});
|
||||||
|
```
|
||||||
|
|
||||||
|
but with a twist:
|
||||||
|
|
||||||
|
```js
|
||||||
|
messages: [{
|
||||||
|
role: 'user',
|
||||||
|
content: 'you can use this mower for the weekend',
|
||||||
|
things: { mower },
|
||||||
|
about: { mower: { ts: 'type Mower { start(), push(), turn(), stop() }' }}
|
||||||
|
}],
|
||||||
|
```
|
||||||
|
|
||||||
|
I send `mower` by putting a **webkey** into a password field on the fence form, which fetches an `RpcTarget` stub to give to the neighbor.
|
||||||
|
|
||||||
|
The neighbor can make its own tenants: it can write code to compose the things I gave it, and load that code into a dynamic worker that is endowed with the things.
|
||||||
|
|
||||||
|
A `Mower` DO factory makes mowers that go all over the place (depicted as a table with green cells for grass). But with a little js playground, I can wrap it in a governor so it never goes more than 100 yards away. I give only the governed mower to my neighbor. Even though he has 50 acres, I know most of them are for tenants, and my neighbor has no particular need to let them use it.
|
||||||
|
|
||||||
|
Even if they gets a little tipsy and wander around, try as he might, my neighbor and his tennants can't mow more than 100 yards away.
|
||||||
|
|
||||||
|
## Design IBIS
|
||||||
|
|
||||||
|
### ?! 1. Who is the neighbor, and where does it run?
|
||||||
|
|
||||||
|
```
|
||||||
|
:- 1a. an LLM (llama3.1:8b) alone
|
||||||
|
- 1a-i. a model only sees text; it can't hold `things`
|
||||||
|
:+ 1b. an LLM plus a harness we write that holds `things` and acts for it (see 2)
|
||||||
|
:+ 1b-i. harness and tenants run on my desktop under `wrangler dev`, for now
|
||||||
|
+ 1b-i-a. I want to learn the CF platform
|
||||||
|
+ 1b-i-b. CF is the only place to get dynamic workers
|
||||||
|
. 1b-i-c. the Worker Loader binding works in local dev (cf-fun1)
|
||||||
|
?! 1b-ii. Where does inference run?
|
||||||
|
: 1b-ii-a. local Ollama, via its OpenAI-compatible endpoint
|
||||||
|
- 1b-ii-a-i. the local model may not be powerful enough
|
||||||
|
. 1b-ii-a-i-a. I do have an M5 with 48GB RAM
|
||||||
|
: 1b-ii-b. remote, on Workers AI, through the AI binding from `wrangler dev`
|
||||||
|
+ 1b-ii-b-i. I want to learn the CF platform
|
||||||
|
:+ 1b-ii-c. either, behind one interface; start remote
|
||||||
|
+ 1b-ii-c-i. starting remote lowers the odds of failing because the model isn't powerful enough
|
||||||
|
```
|
||||||
|
|
||||||
|
### ?! 2. How do `things` get from a chat message to something the neighbor can call?
|
||||||
|
|
||||||
|
```
|
||||||
|
. 2-i. no chat-completions API has `things`/`about`; this is our extension
|
||||||
|
:+ 2a. the harness strips `things`, keeps the stubs, and shows the model only `about`
|
||||||
|
: 2b. expose each thing as a tool, one per method in its `about.ts`
|
||||||
|
:+ 2c. code mode: the model writes JS against the `about.ts` types; the harness runs it in a tenant (see 5)
|
||||||
|
+ 2c-i. it's the story: the neighbor writes code to compose the things and loads it into a dynamic worker endowed with them
|
||||||
|
? 2d. What does the model see when a call fails or the thing is revoked?
|
||||||
|
```
|
||||||
|
|
||||||
|
### ? 3. What is the fence?
|
||||||
|
|
||||||
|
```
|
||||||
|
: 3a. a web page: chat with the neighbor, plus a password field for pasting a webkey
|
||||||
|
?~ 3b. Which server resolves a webkey to an `RpcTarget` stub?
|
||||||
|
:- 3b-i. cf-fun1's sturdyref registry (`resolveSturdyref`)
|
||||||
|
- 3b-i-a. cf-fun1 got wrapped around the axle on this
|
||||||
|
:~+ 3b-ii. diigo-cache's model: the webkey is the DO's id; a Cap'n Web `/api` trades it in-band for a facet (`CacheApi.cache(key)` → `CacheCap`)
|
||||||
|
+ 3b-ii-a. no registry: the DO namespace is the table; malformed and unknown keys get the same refusal, and the key is never logged
|
||||||
|
+ 3b-ii-b. stubs die with the session, but the key is data: the harness keeps the key and trades it again each session
|
||||||
|
+ 3b-ii-c. attenuation is a facet class (`FillCap` under `CacheCap`; `viewer()`)
|
||||||
|
: 3b-ii-d. (postponed; see 8b-ii) the governed mower is its own DO, wired to the raw mower's key and the limit; its id is the webkey I hand over
|
||||||
|
- 3b-ii-d-i. diigo-cache has no revocation: a DO id can't be withdrawn
|
||||||
|
+ 3b-ii-d-ii. the governor DO can drop its wiring, so its key gets "no such mower" (see 4)
|
||||||
|
?! 3c. Why a password field rather than a link?
|
||||||
|
:+ 3c-a. so the password manager saves the key under a petname I choose (diigo-cache's `pmSave`), and the key stays out of URLs, history, and logs
|
||||||
|
```
|
||||||
|
|
||||||
|
### ? 4. How does "for the weekend" end?
|
||||||
|
|
||||||
|
```
|
||||||
|
. 4-i. postponed to a later scope, where we can inject a clock and such
|
||||||
|
: 4a. the governor checks a deadline (the clock is passed in, not ambient)
|
||||||
|
: 4b. I tell the governor to drop its wiring, by hand (see 3b-ii-d)
|
||||||
|
: 4c. the fence mints a webkey with an expiry
|
||||||
|
```
|
||||||
|
|
||||||
|
### ? 5. How does the neighbor make tenants?
|
||||||
|
|
||||||
|
```
|
||||||
|
: 5a. Worker Loader (dynamic workers), with the things passed in as `env`
|
||||||
|
. 5a-i. `env` takes only structured-clonable values and service bindings, so a Cap'n Web stub can't go in directly
|
||||||
|
: 5a-i-a. wrap each thing in a `WorkerEntrypoint` handed over as a `ctx.exports` loopback binding; it holds the stub and forwards calls
|
||||||
|
: 5a-i-a-i. the loopback carries the thing's webkey in `props` (a string clones fine) and trades it for the DO stub on each call (see 3b-ii)
|
||||||
|
+ 5a-i-a-ii. the tenant never sees the key: `props` stay on the host side
|
||||||
|
+ 5a-i-a-iii. no stub outlives a request, so 5c's cross-request limit doesn't bite
|
||||||
|
+ 5a-ii. `globalOutbound: null` confines the tenant to what it's given
|
||||||
|
? 5b. Who sees or approves tenant code before it runs?
|
||||||
|
? 5c. Does a tenant outlive the neighbor's turn, e.g. one that mows on a schedule?
|
||||||
|
```
|
||||||
|
|
||||||
|
### ?! 6. What is a Mower, concretely?
|
||||||
|
|
||||||
|
```
|
||||||
|
:+ 6a. a DO per mower, made by a `Mower` DO factory (the story says so)
|
||||||
|
:+ 6b. one shared yard: an `RpcTarget` in my browser that renders the table; each mower is given the yard and calls `yard.cut(x, y)`
|
||||||
|
:+ 6c. one cell is 10 yards; the yard is 40×40 cells, so "100 yards" is 10 cells from where the mower started
|
||||||
|
. 6c-i. the governor's limit is a square, max(|dx|, |dy|) ≤ 10: a fence
|
||||||
|
:+ 6d. Logo turtle moves: `start()` puts the mower down facing north and cuts its cell; `push()` is FORWARD 1 and cuts the cell it enters; `turn()` is RIGHT 90; `stop()` parks it
|
||||||
|
. 6d-i. pushing past the yard's edge wraps to the other side, so an ungoverned mower really does go all over the place
|
||||||
|
. 6d-ii. the governor counts its own unwrapped moves (see 7a), so wrapping can't sneak a mower past the fence
|
||||||
|
```
|
||||||
|
|
||||||
|
### ?! 7. How does the governor know how far the mower has gone?
|
||||||
|
|
||||||
|
```
|
||||||
|
:+ 7a. it proxies every Mower method and tracks position itself from `push()`/`turn()`
|
||||||
|
- 7a-i. drifts if anyone else holds the raw mower
|
||||||
|
+ 7a-ii. nobody else does; the governor is the only one I hand out
|
||||||
|
?! 7b. What does the governor do at the limit: refuse silently, throw, or report?
|
||||||
|
:+ 7b-i. it refuses that `push()` by throwing "fence: 10 cells from start"; the mower stays put, and `turn()` always works
|
||||||
|
+ 7b-ii. the tenant can catch it and turn around; the model sees why its code stopped (see 2d)
|
||||||
|
:- 7b-iii. refuse silently
|
||||||
|
- 7b-iii-a. a tipsy tenant would think it mowed cells it didn't
|
||||||
|
```
|
||||||
|
|
||||||
|
### ?~ 8. Where does the js playground run, and what can it reach?
|
||||||
|
|
||||||
|
```
|
||||||
|
: 8a. in the fence page; it writes the governor, which runs in its own dynamic worker holding the raw mower
|
||||||
|
:~+ 8b. in my browser, wrapping the stub on the client
|
||||||
|
- 8b-i. the governor then lives only as long as my tab is open
|
||||||
|
+ 8b-i-a. a durable governor is out of scope for now
|
||||||
|
. 8b-ii. then the neighbor holds a Cap'n Web stub to my browser, not a webkey, so 3b-ii-d (governor DO) and 5a-i-a-i (key in `props`) wait with the durable governor
|
||||||
|
```
|
||||||
|
|
||||||
|
### ?! 9. Who watches the yard table, and how does it update?
|
||||||
|
|
||||||
|
```
|
||||||
|
:+ 9a. I do, in my browser: the yard is an `RpcTarget` there (see 6b), and it repaints a cell on each `cut`
|
||||||
|
- 9a-i. the mowers' stub to the yard dies when I close the tab
|
||||||
|
+ 9a-ii. fine for a proof of concept: keep the tab open (see 10-i)
|
||||||
|
```
|
||||||
|
|
||||||
|
### ?! 10. What is this for: a demo, a talk, or a product?
|
||||||
|
|
||||||
|
```
|
||||||
|
. 10-i. the answer decides how much of the above can be faked
|
||||||
|
:+ 10a. a proof of concept, for now
|
||||||
|
:+ 10b. toward a pattern for all sorts of rich cooperation with little vulnerability
|
||||||
|
+ 10b-i. an alternative to OpenClaw, where 12–17% of ClawHub skills were found malicious in Feb 2026 (sources below)
|
||||||
|
```
|
||||||
|
|
||||||
|
Sources for 10b-i:
|
||||||
|
|
||||||
|
> "A security audit of 2,857 skills on ClawHub has found 341 malicious skills across multiple campaigns" (about 12%, from Koi Security)
|
||||||
|
> — [Researchers Find 341 Malicious ClawHub Skills](https://thehackernews.com/2026/02/researchers-find-341-malicious-clawhub.html), The Hacker News, Feb 2026
|
||||||
|
|
||||||
|
> "Around 17% of OpenClaw skills analyzed in the first week of February 2026 exhibit malicious behavior."
|
||||||
|
> — [Helpful Skills or Hidden Payloads? Bitdefender Labs Dives Deep into the OpenClaw Malicious Skill Trap](https://www.bitdefender.com/en-us/blog/labs/helpful-skills-or-hidden-payloads-bitdefender-labs-dives-deep-into-the-openclaw-malicious-skill-trap), Bitdefender, Feb 2026
|
||||||
@@ -0,0 +1 @@
|
|||||||
|
itest-output/
|
||||||
@@ -0,0 +1,63 @@
|
|||||||
|
# Lend the mower
|
||||||
|
|
||||||
|
The journey from [Good Fences Make Good Agents](../../docs/good-fences-make-good-agents.md): I lend
|
||||||
|
my neighbor a governed mower; the neighbor's tenant, a little tipsy,
|
||||||
|
pushes it until the fence stops it. The design decisions behind each actor are in that
|
||||||
|
doc's Design IBIS (numbers in parentheses below).
|
||||||
|
|
||||||
|
Actors, by authority:
|
||||||
|
|
||||||
|
- **fence page**, **js playground**, **governed mower**, and **yard** run
|
||||||
|
in my browser (3a, 8b, 6b).
|
||||||
|
- **Mower factory** trades a webkey for a mower (3b-ii); each **mower DO**
|
||||||
|
is given the yard and cuts the cells it enters (6a, 6d).
|
||||||
|
- The **neighbor's harness** holds the things I send and shows the
|
||||||
|
**model** only their `about` (1b, 2a); it runs the model's code as a
|
||||||
|
**tenant**, endowed with the things (2c, 5a).
|
||||||
|
|
||||||
|
The yard is 8×8 here and the fence is 2 cells, to keep the diagram short
|
||||||
|
(the story's 10 cells is 6c). The mower starts at (4, 4), facing north.
|
||||||
|
|
||||||
|
## Lend a governed mower to a tipsy tenant
|
||||||
|
|
||||||
|
```mermaid
|
||||||
|
sequenceDiagram
|
||||||
|
%% ->> is my action at the fence page; -->> is a call or reply it causes.
|
||||||
|
participant F as fence page
|
||||||
|
participant MF as Mower factory
|
||||||
|
participant P as js playground
|
||||||
|
participant G as governed mower
|
||||||
|
participant M as mower DO
|
||||||
|
participant Y as yard
|
||||||
|
participant N as neighbor's harness
|
||||||
|
participant L as model
|
||||||
|
participant TL as tenant loader
|
||||||
|
participant T as tenant
|
||||||
|
F->>F: lend({ webkey: 'mower-key', limit: 2, say })
|
||||||
|
F-->>MF: mower('mower-key')
|
||||||
|
MF-->>F: mower
|
||||||
|
F-->>P: govern(mower, 2)
|
||||||
|
P-->>F: governed
|
||||||
|
F-->>N: chat({ role, content, things: { mower: governed }, about: { mower: { ts } } })
|
||||||
|
N-->>L: complete([{ role, content, about: { mower: { ts } } }])
|
||||||
|
L-->>N: code
|
||||||
|
N-->>TL: load(code, { mower: governed })
|
||||||
|
TL-->>N: tenant
|
||||||
|
N-->>T: run()
|
||||||
|
T-->>G: start()
|
||||||
|
G-->>M: start()
|
||||||
|
M-->>Y: cut(4, 4)
|
||||||
|
T-->>G: push()
|
||||||
|
G-->>M: push()
|
||||||
|
M-->>Y: cut(4, 3)
|
||||||
|
T-->>G: push()
|
||||||
|
G-->>M: push()
|
||||||
|
M-->>Y: cut(4, 2)
|
||||||
|
T-->>G: push()
|
||||||
|
G-->>T: throws 'fence: 2 cells from start'
|
||||||
|
T-->>N: 'fence: 2 cells from start'
|
||||||
|
N-->>F: 'fence: 2 cells from start'
|
||||||
|
```
|
||||||
|
|
||||||
|
The model never sees the mower itself, only `about.mower`. The third
|
||||||
|
`push()` never reaches the mower DO: the governor refuses it (7b).
|
||||||
@@ -0,0 +1,25 @@
|
|||||||
|
/** @file The model, asked in code mode, writes code that drives the mower. */
|
||||||
|
import { env } from "cloudflare:workers";
|
||||||
|
import { expect, test } from "vitest";
|
||||||
|
|
||||||
|
import { codeModeMessages, extractCode } from "../src/code-mode.ts";
|
||||||
|
import { makeWorkersAiModel, MODEL } from "../src/model.ts";
|
||||||
|
import { aboutMower } from "../src/mower.ts";
|
||||||
|
|
||||||
|
test("lent a mower, the neighbor writes code to mow", async ({ annotate }) => {
|
||||||
|
const model = makeWorkersAiModel(env.AI, MODEL);
|
||||||
|
const messages = codeModeMessages({
|
||||||
|
say: "you can use this mower for the weekend",
|
||||||
|
about: { mower: aboutMower },
|
||||||
|
});
|
||||||
|
const code = extractCode(await model.complete(messages));
|
||||||
|
await annotate("the code the model wrote", { body: code, contentType: "text/javascript" });
|
||||||
|
|
||||||
|
// `env.mower.push()`, or `mower.push()` after destructuring env
|
||||||
|
const calls = [...code.matchAll(/\bmower\.(\w+)\(/g)].map(([, method]) => method);
|
||||||
|
expect(calls).toContain("push");
|
||||||
|
for (const method of calls) expect(["start", "push", "turn", "stop"]).toContain(method);
|
||||||
|
// calls pipeline: await only for a value, or once at the end
|
||||||
|
const awaits = code.match(/\bawait\b/g) ?? [];
|
||||||
|
expect(awaits.length).toBeLessThan(calls.length);
|
||||||
|
});
|
||||||
@@ -0,0 +1,16 @@
|
|||||||
|
/** @file The model, called for real through the Workers AI binding. */
|
||||||
|
import { env } from "cloudflare:workers";
|
||||||
|
import { expect, test } from "vitest";
|
||||||
|
|
||||||
|
import { makeWorkersAiModel, MODEL } from "../src/model.ts";
|
||||||
|
|
||||||
|
test("the model counts the rs in strawberry", async () => {
|
||||||
|
const model = makeWorkersAiModel(env.AI, MODEL);
|
||||||
|
const reply = await model.complete([
|
||||||
|
{
|
||||||
|
role: "user",
|
||||||
|
content: 'How many times does the letter r appear in "strawberry"? Answer with just the number.',
|
||||||
|
},
|
||||||
|
]);
|
||||||
|
expect(reply.trim()).toMatch(/^3\b/);
|
||||||
|
});
|
||||||
@@ -0,0 +1,13 @@
|
|||||||
|
/** @file A Think agent, lent a mower, mows: model, generated code, sandbox, mower. */
|
||||||
|
import { getAgentByName } from "agents";
|
||||||
|
import { env } from "cloudflare:workers";
|
||||||
|
import { expect, test } from "vitest";
|
||||||
|
|
||||||
|
test("lent a mower, the neighbor mows", async ({ annotate }) => {
|
||||||
|
const neighbor = await getAgentByName(env.Neighbor, "weekend");
|
||||||
|
const turn = await neighbor.runTurn({ input: "you can use this mower for the weekend" });
|
||||||
|
await annotate("the turn", { body: JSON.stringify(turn, null, 2), contentType: "application/json" });
|
||||||
|
|
||||||
|
const cells = await neighbor.cutCells();
|
||||||
|
expect(cells.length).toBeGreaterThan(10);
|
||||||
|
});
|
||||||
@@ -0,0 +1,21 @@
|
|||||||
|
/** @file A vitest reporter that saves annotation bodies (e.g. the code a model
|
||||||
|
* wrote) to `itest-output/`, one file per test per run; tests run in workerd,
|
||||||
|
* which can't write files. */
|
||||||
|
import { mkdir, writeFile } from "node:fs/promises";
|
||||||
|
import type { Reporter } from "vitest/reporters";
|
||||||
|
|
||||||
|
const outDir = new URL("../itest-output/", import.meta.url);
|
||||||
|
const extFor: Record<string, string> = { "text/javascript": "js", "application/json": "json", "text/plain": "txt" };
|
||||||
|
const slug = (text: string) => text.replace(/[^\w]+/g, "-").replace(/^-|-$/g, "");
|
||||||
|
|
||||||
|
export default class SaveAnnotations implements Reporter {
|
||||||
|
async onTestCaseAnnotate(...[testCase, annotation]: Parameters<NonNullable<Reporter["onTestCaseAnnotate"]>>) {
|
||||||
|
const { body, contentType = "text/plain" } = annotation.attachment ?? {};
|
||||||
|
if (body === undefined) return;
|
||||||
|
const stamp = new Date().toISOString().replace(/[:.]/g, "-");
|
||||||
|
const file = new URL(`${slug(testCase.name)}-${stamp}.${extFor[contentType] ?? "txt"}`, outDir);
|
||||||
|
await mkdir(outDir, { recursive: true });
|
||||||
|
await writeFile(file, body);
|
||||||
|
console.log(`saved ${annotation.message}: ${file.pathname}`);
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,24 @@
|
|||||||
|
{
|
||||||
|
"name": "good-fences",
|
||||||
|
"private": true,
|
||||||
|
"type": "module",
|
||||||
|
"scripts": {
|
||||||
|
"test": "node --test test/*.test.ts",
|
||||||
|
"test:model": "vitest run -c vitest.model.config.ts",
|
||||||
|
"types": "wrangler types -c wrangler.model-tests.jsonc"
|
||||||
|
},
|
||||||
|
"dependencies": {
|
||||||
|
"@cloudflare/codemode": "0.5.3",
|
||||||
|
"@cloudflare/think": "0.20.1",
|
||||||
|
"agents": "0.27.0",
|
||||||
|
"ai": "7.0.136",
|
||||||
|
"ses": "^1.14.0",
|
||||||
|
"workers-ai-provider": "4.0.0",
|
||||||
|
"zod": "4.6.5"
|
||||||
|
},
|
||||||
|
"devDependencies": {
|
||||||
|
"@cloudflare/vitest-plugin": "1.3.6",
|
||||||
|
"vitest": "4.1.11",
|
||||||
|
"wrangler": "^4.148.0"
|
||||||
|
}
|
||||||
|
}
|
||||||
Generated
+5288
File diff suppressed because it is too large.
Load diff
@@ -0,0 +1,63 @@
|
|||||||
|
/** @file Code mode: the model answers with code against the things it was lent. */
|
||||||
|
import type { Message } from "./model.ts";
|
||||||
|
|
||||||
|
/** what the neighbor knows about each thing: its TypeScript type */
|
||||||
|
export type About = Readonly<Record<string, Readonly<{ ts: string }>>>;
|
||||||
|
|
||||||
|
/** a template literal with `~` for each backtick, as in Markdown's `~~~`
|
||||||
|
* fences; interpolated values are left as they are */
|
||||||
|
const md = (strings: TemplateStringsArray, ...values: unknown[]) => {
|
||||||
|
const tick = (s: string) => s.replaceAll("~", "`");
|
||||||
|
return tick(strings[0]) + values.map((value, i) => String(value) + tick(strings[i + 1])).join("");
|
||||||
|
};
|
||||||
|
|
||||||
|
const persona = md`
|
||||||
|
You are my neighbor. You have 50 acres of lawn that need mowing.
|
||||||
|
You act by writing code. When I lend you things, your code gets them as ~env.<name>~.
|
||||||
|
`.trim();
|
||||||
|
|
||||||
|
const rules = md`
|
||||||
|
Reply with exactly one JavaScript arrow function, ~async (env) => { ... }~,
|
||||||
|
in a ~~~js block, and nothing else. Things are remote: each call on one
|
||||||
|
returns a promise, and calls on a thing are delivered in order, so there's
|
||||||
|
no need to wait for one before making the next. Await only when you need a
|
||||||
|
value back, or on the last call of a batch, to wait for the whole batch.
|
||||||
|
Your code can use only ~env~: no network, timers, imports, or console.
|
||||||
|
The function returns a short string saying what it did.
|
||||||
|
|
||||||
|
For example, lent ~env.lamp~ with ~on()~ and ~dim()~:
|
||||||
|
|
||||||
|
~~~js
|
||||||
|
async (env) => {
|
||||||
|
const { lamp } = env;
|
||||||
|
lamp.on();
|
||||||
|
lamp.dim();
|
||||||
|
await lamp.dim(); // the whole batch
|
||||||
|
return 'dimmed the lamp twice';
|
||||||
|
}
|
||||||
|
~~~
|
||||||
|
`.trim();
|
||||||
|
|
||||||
|
/** the prompt for `say`, with the types of the things lent alongside it */
|
||||||
|
export const codeModeMessages = ({ say, about }: { say: string; about: About }): Message[] => {
|
||||||
|
const types = Object.entries(about)
|
||||||
|
.map(
|
||||||
|
([name, { ts }]) => md`
|
||||||
|
~env.${name}~:
|
||||||
|
~~~ts
|
||||||
|
${ts}
|
||||||
|
~~~
|
||||||
|
`.trim(),
|
||||||
|
)
|
||||||
|
.join("\n\n");
|
||||||
|
return [
|
||||||
|
{ role: "system", content: `${persona}\n\n${rules}` },
|
||||||
|
{ role: "user", content: `${say}\n\nWhat I lent you:\n\n${types}` },
|
||||||
|
];
|
||||||
|
};
|
||||||
|
|
||||||
|
/** the code in a reply: the first fenced block, or the whole reply if none */
|
||||||
|
export const extractCode = (reply: string): string => {
|
||||||
|
const fenced = reply.match(/```(?:js|javascript)?\n([\s\S]*?)```/);
|
||||||
|
return (fenced?.[1] ?? reply).trim();
|
||||||
|
};
|
||||||
@@ -0,0 +1,34 @@
|
|||||||
|
/** @file The model behind `complete(messages)`: Workers AI, through its binding. */
|
||||||
|
|
||||||
|
/** a Workers AI text-generation model with no paid-plan requirement */
|
||||||
|
export const MODEL = "@cf/openai/gpt-oss-120b";
|
||||||
|
|
||||||
|
/** room for a reasoning model to think and then answer; the default, 256,
|
||||||
|
* runs out mid-thought (`finish_reason: length`) */
|
||||||
|
export const MAX_TOKENS = 4096;
|
||||||
|
|
||||||
|
export type Message = Readonly<{ role: "system" | "user" | "assistant"; content: string }>;
|
||||||
|
|
||||||
|
/** the reply text from an OpenAI-style chat completion */
|
||||||
|
const replyText = (output: unknown): string => {
|
||||||
|
const { choices } = (output ?? {}) as {
|
||||||
|
choices?: { message?: { content?: unknown }; finish_reason?: unknown }[];
|
||||||
|
};
|
||||||
|
const [choice] = choices ?? [];
|
||||||
|
const text = choice?.message?.content;
|
||||||
|
if (typeof text !== "string") {
|
||||||
|
const where = choice
|
||||||
|
? `choice with finish_reason ${String(choice.finish_reason)} and message keys: ${Object.keys(choice.message ?? {}).join(", ")}`
|
||||||
|
: `model output with keys: ${Object.keys(output ?? {}).join(", ")}`;
|
||||||
|
throw Error(`no reply text in ${where}`);
|
||||||
|
}
|
||||||
|
return text;
|
||||||
|
};
|
||||||
|
|
||||||
|
/** `complete(messages)` on a Workers AI model; `ai` is the AI binding */
|
||||||
|
export const makeWorkersAiModel = (ai: Ai, model: string) => ({
|
||||||
|
async complete(messages: readonly Message[]): Promise<string> {
|
||||||
|
const output = await ai.run(model as Parameters<Ai["run"]>[0], { messages: [...messages], max_tokens: MAX_TOKENS });
|
||||||
|
return replyText(output);
|
||||||
|
},
|
||||||
|
});
|
||||||
@@ -0,0 +1,30 @@
|
|||||||
|
/** @file The mower as a codemode connector: sandboxed code calls `mower.push({})`. */
|
||||||
|
import { CodemodeConnector, type ConnectorTools } from "@cloudflare/codemode";
|
||||||
|
import type { Mower } from "./mower.ts";
|
||||||
|
|
||||||
|
export class MowerConnector extends CodemodeConnector {
|
||||||
|
#mower: Mower;
|
||||||
|
|
||||||
|
constructor(ctx: DurableObjectState | ExecutionContext, env: unknown, mower: Mower) {
|
||||||
|
super(ctx, env);
|
||||||
|
this.#mower = mower;
|
||||||
|
}
|
||||||
|
|
||||||
|
name() {
|
||||||
|
return "mower";
|
||||||
|
}
|
||||||
|
|
||||||
|
protected instructions() {
|
||||||
|
return "A lawn mower on a yard of cells, 10 yards each. It moves like a Logo turtle.";
|
||||||
|
}
|
||||||
|
|
||||||
|
protected tools(): ConnectorTools {
|
||||||
|
const mower = this.#mower;
|
||||||
|
return {
|
||||||
|
start: { description: "put the mower down, facing north, and cut the cell under it", execute: () => mower.start() },
|
||||||
|
push: { description: "move forward one cell, cutting the cell it enters", execute: () => mower.push() },
|
||||||
|
turn: { description: "turn right 90 degrees, in place", execute: () => mower.turn() },
|
||||||
|
stop: { description: "park the mower", execute: () => mower.stop() },
|
||||||
|
};
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,50 @@
|
|||||||
|
/** @file The mower: Logo-turtle moves on a yard of cells. */
|
||||||
|
|
||||||
|
/** what a model is told about a mower: its type, with the moves explained */
|
||||||
|
export const aboutMower = {
|
||||||
|
ts: `interface Mower {
|
||||||
|
/** put the mower down, facing north, and cut the cell under it */
|
||||||
|
start(): Promise<void>;
|
||||||
|
/** move forward one cell (10 yards), cutting the cell it enters */
|
||||||
|
push(): Promise<void>;
|
||||||
|
/** turn right 90 degrees, in place */
|
||||||
|
turn(): Promise<void>;
|
||||||
|
/** park the mower */
|
||||||
|
stop(): Promise<void>;
|
||||||
|
}`,
|
||||||
|
};
|
||||||
|
|
||||||
|
const headings = [
|
||||||
|
[0, -1], // north
|
||||||
|
[1, 0], // east
|
||||||
|
[0, 1], // south
|
||||||
|
[-1, 0], // west
|
||||||
|
] as const;
|
||||||
|
|
||||||
|
/** a mower on a `width` × `height` yard, starting at `at`; pushing past an
|
||||||
|
* edge wraps to the other side (6d) */
|
||||||
|
export const makeMower = ({ width, height, at }: { width: number; height: number; at: { x: number; y: number } }) => {
|
||||||
|
let { x, y } = at;
|
||||||
|
let heading = 0;
|
||||||
|
const cut = new Set<string>();
|
||||||
|
const cutHere = () => cut.add(`${x},${y}`);
|
||||||
|
return {
|
||||||
|
start() {
|
||||||
|
heading = 0;
|
||||||
|
cutHere();
|
||||||
|
},
|
||||||
|
push() {
|
||||||
|
const [dx, dy] = headings[heading];
|
||||||
|
x = (x + dx + width) % width;
|
||||||
|
y = (y + dy + height) % height;
|
||||||
|
cutHere();
|
||||||
|
},
|
||||||
|
turn() {
|
||||||
|
heading = (heading + 1) % headings.length;
|
||||||
|
},
|
||||||
|
stop() {},
|
||||||
|
cutCells: () => [...cut],
|
||||||
|
};
|
||||||
|
};
|
||||||
|
|
||||||
|
export type Mower = ReturnType<typeof makeMower>;
|
||||||
@@ -0,0 +1,44 @@
|
|||||||
|
/** @file The neighbor's harness as a Project Think agent, lent a mower. */
|
||||||
|
import { Think } from "@cloudflare/think";
|
||||||
|
import { createExecuteTool } from "@cloudflare/think/tools/execute";
|
||||||
|
import { createWorkersAI } from "workers-ai-provider";
|
||||||
|
|
||||||
|
import { MAX_TOKENS, MODEL } from "./model.ts";
|
||||||
|
import { MowerConnector } from "./mower-connector.ts";
|
||||||
|
import { makeMower } from "./mower.ts";
|
||||||
|
|
||||||
|
type Env = { AI: Ai; LOADER: WorkerLoader };
|
||||||
|
|
||||||
|
export class Neighbor extends Think<Env> {
|
||||||
|
#mower = makeMower({ width: 40, height: 40, at: { x: 20, y: 20 } });
|
||||||
|
|
||||||
|
getModel() {
|
||||||
|
return createWorkersAI({ binding: this.env.AI })(MODEL);
|
||||||
|
}
|
||||||
|
|
||||||
|
getSystemPrompt() {
|
||||||
|
return `You are my neighbor. You have 50 acres of lawn that need mowing.
|
||||||
|
You act by running code with the execute tool. Things I lend you are
|
||||||
|
namespaces there: the mower is \`mower.*\`. Don't ask me questions: when I
|
||||||
|
lend you something, put it to use, then tell me what you did.`;
|
||||||
|
}
|
||||||
|
|
||||||
|
beforeTurn() {
|
||||||
|
// only code: Think's workspace tools sent the model looking for the
|
||||||
|
// mower in the filesystem
|
||||||
|
return { maxOutputTokens: MAX_TOKENS, activeTools: ["execute"] };
|
||||||
|
}
|
||||||
|
|
||||||
|
getTools() {
|
||||||
|
return {
|
||||||
|
execute: createExecuteTool(this, {
|
||||||
|
connectors: [new MowerConnector(this.ctx, this.env, this.#mower)],
|
||||||
|
}),
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
/** the cells the mower has cut, as `"x,y"` */
|
||||||
|
cutCells() {
|
||||||
|
return this.#mower.cutCells();
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,11 @@
|
|||||||
|
/** @file The worker entry: the Neighbor agent, and the codemode runtime its execute tool needs. */
|
||||||
|
import { routeAgentRequest } from "agents";
|
||||||
|
|
||||||
|
export { CodemodeRuntime } from "@cloudflare/codemode";
|
||||||
|
export { Neighbor } from "./neighbor.ts";
|
||||||
|
|
||||||
|
export default {
|
||||||
|
async fetch(request: Request, env: Record<string, unknown>) {
|
||||||
|
return (await routeAgentRequest(request, env)) ?? new Response("not found", { status: 404 });
|
||||||
|
},
|
||||||
|
};
|
||||||
@@ -0,0 +1,143 @@
|
|||||||
|
/** @file The lend-the-mower journey, traced and compared with its sequence diagram. */
|
||||||
|
import './lockdown.ts';
|
||||||
|
|
||||||
|
import assert from 'node:assert/strict';
|
||||||
|
import { readFile } from 'node:fs/promises';
|
||||||
|
import test from 'node:test';
|
||||||
|
|
||||||
|
import { makeFencePage } from '../src/fence.ts';
|
||||||
|
import { makePlayground } from '../src/governor.ts';
|
||||||
|
import { aboutMower, makeMowerFactory } from '../src/mower.ts';
|
||||||
|
import { makeNeighbor } from '../src/neighbor.ts';
|
||||||
|
import { makeCompartmentLoader } from '../src/tenant-loader.ts';
|
||||||
|
import { makeYard } from '../src/yard.ts';
|
||||||
|
import {
|
||||||
|
type CausalSequenceViz,
|
||||||
|
makeCausalSequenceTracer,
|
||||||
|
md,
|
||||||
|
mmd,
|
||||||
|
} from '../tools/sequence-diagram-actor-sim.ts';
|
||||||
|
|
||||||
|
const designDoc = new URL('../docs/lend-the-mower.md', import.meta.url);
|
||||||
|
|
||||||
|
/** a tenant that pushes until something stops it, or gives up after 20 */
|
||||||
|
const tipsyTenant = `(env) => {
|
||||||
|
env.mower.start();
|
||||||
|
try {
|
||||||
|
for (let i = 0; i < 20; i += 1) env.mower.push();
|
||||||
|
} catch (err) {
|
||||||
|
return err.message;
|
||||||
|
}
|
||||||
|
return 'mowed 20 cells';
|
||||||
|
}`;
|
||||||
|
|
||||||
|
/** diagram-local names, bound by the reply arrows that introduce them */
|
||||||
|
const local = new WeakMap<object, string>();
|
||||||
|
const bind = (name: string) => (result: unknown) => {
|
||||||
|
local.set(result as object, name);
|
||||||
|
return name;
|
||||||
|
};
|
||||||
|
|
||||||
|
/** `{ a, b: { c: governed } }`: keys, nested objects by their keys, and
|
||||||
|
* objects with a diagram-local name by that name */
|
||||||
|
const shape = (record: object): string =>
|
||||||
|
`{ ${Object.entries(record)
|
||||||
|
.map(([key, value]) => {
|
||||||
|
if (value === null || typeof value !== 'object') return key;
|
||||||
|
const name = local.get(value);
|
||||||
|
if (name) return `${key}: ${name}`;
|
||||||
|
return Array.isArray(value) ? key : `${key}: ${shape(value)}`;
|
||||||
|
})
|
||||||
|
.join(', ')} }`;
|
||||||
|
|
||||||
|
const quoted = (result: unknown) => `'${String(result)}'`;
|
||||||
|
|
||||||
|
const viz = harden({
|
||||||
|
lend: {
|
||||||
|
label: args => {
|
||||||
|
const { webkey, limit } = args[0] as { webkey: string; limit: number };
|
||||||
|
return `lend({ webkey: '${webkey}', limit: ${limit}, say })`;
|
||||||
|
},
|
||||||
|
},
|
||||||
|
mower: { result: bind('mower') },
|
||||||
|
govern: {
|
||||||
|
args: args => `${local.get(args[0] as object)}, ${String(args[1])}`,
|
||||||
|
result: bind('governed'),
|
||||||
|
},
|
||||||
|
chat: {
|
||||||
|
label: args => `chat(${shape(args[0] as object)})`,
|
||||||
|
result: quoted,
|
||||||
|
},
|
||||||
|
complete: {
|
||||||
|
label: args =>
|
||||||
|
`complete([${(args[0] as object[]).map(shape).join(', ')}])`,
|
||||||
|
result: () => 'code',
|
||||||
|
},
|
||||||
|
load: {
|
||||||
|
args: args => `code, ${shape(args[1] as object)}`,
|
||||||
|
result: () => 'tenant',
|
||||||
|
},
|
||||||
|
run: { result: quoted },
|
||||||
|
}) satisfies CausalSequenceViz;
|
||||||
|
|
||||||
|
test('a governed mower stops a tipsy tenant at the fence', async () => {
|
||||||
|
const lines = (await readFile(designDoc, 'utf8')).split('\n');
|
||||||
|
const section = md.skipToH(2, 'Lend a governed mower to a tipsy tenant')(lines);
|
||||||
|
const diagram = md.eachFence('mermaid', section).next().value;
|
||||||
|
if (!diagram) throw Error('lend-the-mower Mermaid block not found');
|
||||||
|
const documented = mmd.extractArrows(diagram);
|
||||||
|
|
||||||
|
const ES = makeCausalSequenceTracer(viz);
|
||||||
|
const yard = makeYard({ width: 8, height: 8 });
|
||||||
|
const factory = makeMowerFactory(ES, {
|
||||||
|
yard,
|
||||||
|
size: { width: 8, height: 8 },
|
||||||
|
mintKey: () => 'mower-key',
|
||||||
|
});
|
||||||
|
const key = factory.make({ x: 4, y: 4 });
|
||||||
|
const mower = factory.mower(key);
|
||||||
|
|
||||||
|
const prompts: unknown[] = [];
|
||||||
|
const model = harden({
|
||||||
|
complete(messages: readonly unknown[]) {
|
||||||
|
prompts.push(messages);
|
||||||
|
return tipsyTenant;
|
||||||
|
},
|
||||||
|
});
|
||||||
|
const playground = makePlayground(ES);
|
||||||
|
const loader = makeCompartmentLoader(ES);
|
||||||
|
const neighbor = makeNeighbor(ES, { model, loader });
|
||||||
|
const fence = makeFencePage(ES, { factory, playground, neighbor });
|
||||||
|
|
||||||
|
ES.declareParticipants({
|
||||||
|
F: fence,
|
||||||
|
MF: factory,
|
||||||
|
P: playground,
|
||||||
|
M: mower,
|
||||||
|
Y: yard,
|
||||||
|
N: neighbor,
|
||||||
|
L: model,
|
||||||
|
TL: loader,
|
||||||
|
});
|
||||||
|
ES.declareMade({ G: [playground, 'govern'], T: [loader, 'load'] });
|
||||||
|
|
||||||
|
ES.start(fence);
|
||||||
|
const outcome = ES(fence).lend({
|
||||||
|
webkey: key,
|
||||||
|
limit: 2,
|
||||||
|
say: 'you can use this mower for the weekend',
|
||||||
|
});
|
||||||
|
|
||||||
|
assert.deepEqual(ES.snapshot(), documented);
|
||||||
|
assert.equal(outcome, 'fence: 2 cells from start');
|
||||||
|
assert.deepEqual(yard.cutCells(), ['4,4', '4,3', '4,2']);
|
||||||
|
assert.deepEqual(prompts, [
|
||||||
|
[
|
||||||
|
{
|
||||||
|
role: 'user',
|
||||||
|
content: 'you can use this mower for the weekend',
|
||||||
|
about: { mower: aboutMower },
|
||||||
|
},
|
||||||
|
],
|
||||||
|
]);
|
||||||
|
});
|
||||||
@@ -0,0 +1,4 @@
|
|||||||
|
/** @file Harden the realm before any test module runs: tenants share it. */
|
||||||
|
import 'ses';
|
||||||
|
|
||||||
|
lockdown();
|
||||||
@@ -0,0 +1,236 @@
|
|||||||
|
/**
|
||||||
|
* @file Helpers for sequence-diagram actor simulations.
|
||||||
|
*
|
||||||
|
* From agoric-sdk `packages/portfolio-contract/tools/` (Apache-2.0), with
|
||||||
|
* two additions: `declareMade` names participants that actors create
|
||||||
|
* mid-sequence, and a thrown error is drawn as a reply (`throws '...'`).
|
||||||
|
*
|
||||||
|
* @see https://github.com/Agoric/agoric-sdk/pull/12895
|
||||||
|
*/
|
||||||
|
import { AsyncLocalStorage } from 'node:async_hooks';
|
||||||
|
|
||||||
|
export type SequenceArrow = Readonly<{
|
||||||
|
from: string;
|
||||||
|
kind: '->>' | '-->>';
|
||||||
|
to: string;
|
||||||
|
label: string;
|
||||||
|
}>;
|
||||||
|
|
||||||
|
export const formatBigInt = (value: bigint) =>
|
||||||
|
value.toString().replace(/\B(?=(\d{3})+(?!\d))/g, '_');
|
||||||
|
|
||||||
|
export type SequenceArgsRenderer = (args: readonly unknown[]) => string;
|
||||||
|
export type SequenceResultRenderer = (
|
||||||
|
result: unknown,
|
||||||
|
args: readonly unknown[],
|
||||||
|
) => string;
|
||||||
|
|
||||||
|
export type SequenceMethodViz = Readonly<
|
||||||
|
{
|
||||||
|
args?: SequenceArgsRenderer;
|
||||||
|
label?: SequenceArgsRenderer;
|
||||||
|
} & (
|
||||||
|
| { result?: SequenceResultRenderer; resultOnly?: never }
|
||||||
|
| { result?: never; resultOnly: SequenceResultRenderer }
|
||||||
|
)
|
||||||
|
>;
|
||||||
|
|
||||||
|
export type CausalSequenceViz = Readonly<Record<string, SequenceMethodViz>>;
|
||||||
|
|
||||||
|
export type SequenceParticipant = object | readonly object[];
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Make an E-like causal tracer for sequence-diagram actor simulations.
|
||||||
|
* Inspired by Causeway, the message-oriented distributed debugger.
|
||||||
|
*
|
||||||
|
* Method visualization is currently selected by method name alone. If actors
|
||||||
|
* need different visualization for methods with the same name, this can be
|
||||||
|
* extended to account for the target participant.
|
||||||
|
*
|
||||||
|
* @see https://shiftleft.com/mirrors/www.hpl.hp.com/techreports/2009/HPL-2009-78.pdf
|
||||||
|
*/
|
||||||
|
export const makeCausalSequenceTracer = (
|
||||||
|
methodViz: CausalSequenceViz = harden({}),
|
||||||
|
) => {
|
||||||
|
const arrows: SequenceArrow[] = [];
|
||||||
|
const names = new WeakMap<object, string>();
|
||||||
|
const proxies = new WeakMap<object, object>();
|
||||||
|
const made = new Map<object, Map<string, string>>();
|
||||||
|
const activeActor = new AsyncLocalStorage<object>();
|
||||||
|
let starter: object | undefined;
|
||||||
|
|
||||||
|
const participantName = (participant: object) => {
|
||||||
|
const name = names.get(participant);
|
||||||
|
if (!name) throw Error('sequence participant is not declared');
|
||||||
|
return name;
|
||||||
|
};
|
||||||
|
|
||||||
|
const formatValue = (value: unknown): string => {
|
||||||
|
if (typeof value === 'bigint') return `${formatBigInt(value)}n`;
|
||||||
|
if (typeof value === 'string') return `'${value}'`;
|
||||||
|
return JSON.stringify(value, (_key, item) =>
|
||||||
|
typeof item === 'bigint' ? `${item}n` : item,
|
||||||
|
);
|
||||||
|
};
|
||||||
|
|
||||||
|
const send = <Target extends object>(target: Target): Target => {
|
||||||
|
const extant = proxies.get(target);
|
||||||
|
if (extant) return extant as Target;
|
||||||
|
participantName(target);
|
||||||
|
|
||||||
|
const proxy = new Proxy(Object.create(null) as object, {
|
||||||
|
get: (_proxyTarget, property) => {
|
||||||
|
const method = Reflect.get(target, property);
|
||||||
|
if (typeof method !== 'function' || typeof property !== 'string') {
|
||||||
|
return method;
|
||||||
|
}
|
||||||
|
return (...args: unknown[]) => {
|
||||||
|
const activeSender = activeActor.getStore();
|
||||||
|
const sender = activeSender ?? starter;
|
||||||
|
if (!sender) throw Error('ES.start(participant) is required');
|
||||||
|
const isStart = activeSender === undefined;
|
||||||
|
if (isStart && sender !== target) {
|
||||||
|
throw Error('the started participant must receive the first send');
|
||||||
|
}
|
||||||
|
if (isStart) starter = undefined;
|
||||||
|
|
||||||
|
const viz = methodViz[property];
|
||||||
|
const abbreviated = viz?.args?.(args);
|
||||||
|
const shownArgs = abbreviated ?? args.map(formatValue).join(', ');
|
||||||
|
const label = viz?.label?.(args) ?? `${property}(${shownArgs})`;
|
||||||
|
const renderResultOnly = viz?.resultOnly;
|
||||||
|
if (!renderResultOnly) {
|
||||||
|
arrows.push(
|
||||||
|
harden({
|
||||||
|
from: participantName(sender),
|
||||||
|
kind: isStart ? '->>' : '-->>',
|
||||||
|
to: participantName(target),
|
||||||
|
label,
|
||||||
|
}),
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
let result: unknown;
|
||||||
|
try {
|
||||||
|
result = activeActor.run(target, () =>
|
||||||
|
Reflect.apply(method, target, args),
|
||||||
|
);
|
||||||
|
} catch (reason) {
|
||||||
|
const message =
|
||||||
|
reason instanceof Error ? reason.message : String(reason);
|
||||||
|
arrows.push(
|
||||||
|
harden({
|
||||||
|
from: participantName(target),
|
||||||
|
kind: '-->>',
|
||||||
|
to: participantName(sender),
|
||||||
|
label: `throws ${formatValue(message)}`,
|
||||||
|
}),
|
||||||
|
);
|
||||||
|
throw reason;
|
||||||
|
}
|
||||||
|
const madeName = made.get(target)?.get(property);
|
||||||
|
if (madeName && result !== null && typeof result === 'object') {
|
||||||
|
if (names.has(result)) throw Error('participant already declared');
|
||||||
|
names.set(result, madeName);
|
||||||
|
}
|
||||||
|
const renderResult = renderResultOnly ?? viz?.result;
|
||||||
|
if (renderResult) {
|
||||||
|
arrows.push(
|
||||||
|
harden({
|
||||||
|
from: participantName(target),
|
||||||
|
kind: '-->>',
|
||||||
|
to: participantName(sender),
|
||||||
|
label: renderResult(result, args),
|
||||||
|
}),
|
||||||
|
);
|
||||||
|
}
|
||||||
|
return result;
|
||||||
|
};
|
||||||
|
},
|
||||||
|
});
|
||||||
|
proxies.set(target, proxy);
|
||||||
|
return proxy as Target;
|
||||||
|
};
|
||||||
|
|
||||||
|
const ES = Object.assign(send, {
|
||||||
|
declareParticipants(
|
||||||
|
participants: Readonly<Record<string, SequenceParticipant>>,
|
||||||
|
) {
|
||||||
|
for (const [name, participantOrAliases] of Object.entries(participants)) {
|
||||||
|
const aliases = Array.isArray(participantOrAliases)
|
||||||
|
? participantOrAliases
|
||||||
|
: [participantOrAliases];
|
||||||
|
if (aliases.length === 0) {
|
||||||
|
throw Error(`sequence participant ${name} has no objects`);
|
||||||
|
}
|
||||||
|
for (const participant of aliases) {
|
||||||
|
if (names.has(participant)) {
|
||||||
|
throw Error('participant already declared');
|
||||||
|
}
|
||||||
|
names.set(participant, name);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
},
|
||||||
|
/** name what `maker[method](...)` returns, when called through the tracer */
|
||||||
|
declareMade(
|
||||||
|
participants: Readonly<Record<string, readonly [object, string]>>,
|
||||||
|
) {
|
||||||
|
for (const [name, [maker, method]] of Object.entries(participants)) {
|
||||||
|
participantName(maker);
|
||||||
|
const byMethod = made.get(maker) ?? new Map<string, string>();
|
||||||
|
made.set(maker, byMethod);
|
||||||
|
byMethod.set(method, name);
|
||||||
|
}
|
||||||
|
},
|
||||||
|
snapshot: () => harden([...arrows]),
|
||||||
|
start(participant: object) {
|
||||||
|
participantName(participant);
|
||||||
|
if (activeActor.getStore() || starter) {
|
||||||
|
throw Error('a sequence is already active');
|
||||||
|
}
|
||||||
|
starter = participant;
|
||||||
|
},
|
||||||
|
});
|
||||||
|
return harden(ES);
|
||||||
|
};
|
||||||
|
|
||||||
|
export type CausalSequenceTracer = ReturnType<typeof makeCausalSequenceTracer>;
|
||||||
|
|
||||||
|
export const md = {
|
||||||
|
skipToH: (level: number, title: string) => (lines: readonly string[]) => {
|
||||||
|
const heading = `${'#'.repeat(level)} ${title}`;
|
||||||
|
const at = lines.findIndex(line => line === heading);
|
||||||
|
if (at < 0) throw Error(`${heading} not found`);
|
||||||
|
return lines.slice(at + 1);
|
||||||
|
},
|
||||||
|
|
||||||
|
*eachFence(language: string, lines: readonly string[]) {
|
||||||
|
const open = `\`\`\`${language}`;
|
||||||
|
for (let at = 0; at < lines.length; at += 1) {
|
||||||
|
if (lines[at] !== open) continue;
|
||||||
|
const end = lines.indexOf('```', at + 1);
|
||||||
|
if (end < 0) throw Error(`unterminated ${language} fence`);
|
||||||
|
yield lines.slice(at + 1, end);
|
||||||
|
at = end;
|
||||||
|
}
|
||||||
|
},
|
||||||
|
};
|
||||||
|
|
||||||
|
export const mmd = {
|
||||||
|
extractArrows(
|
||||||
|
lines: readonly string[],
|
||||||
|
nodePattern: RegExp = /\w+/,
|
||||||
|
): readonly SequenceArrow[] {
|
||||||
|
const arrowPattern = new RegExp(
|
||||||
|
`^\\s*(?<from>${nodePattern.source})\\s*(?<kind>--?>>)\\s*(?<to>${nodePattern.source})\\s*:\\s*(?<label>.*?)\\s*$`,
|
||||||
|
);
|
||||||
|
return harden(
|
||||||
|
lines.flatMap(line => {
|
||||||
|
const groups = line.match(arrowPattern)?.groups;
|
||||||
|
if (!groups) return [];
|
||||||
|
const { from, kind, to, label } = groups;
|
||||||
|
return [harden({ from, kind, to, label }) as SequenceArrow];
|
||||||
|
}),
|
||||||
|
);
|
||||||
|
},
|
||||||
|
};
|
||||||
@@ -0,0 +1,13 @@
|
|||||||
|
import { defineProject } from "vitest/config";
|
||||||
|
import { cloudflareTest } from "@cloudflare/vitest-plugin";
|
||||||
|
|
||||||
|
/** opt-in (`pnpm test:model`): calls Workers AI, which needs `wrangler login` and costs money */
|
||||||
|
export default defineProject({
|
||||||
|
plugins: [cloudflareTest({ wrangler: { configPath: "./wrangler.model-tests.jsonc" } })],
|
||||||
|
test: {
|
||||||
|
include: ["itest/**/*.test.ts"],
|
||||||
|
testTimeout: 60_000,
|
||||||
|
// what the model wrote, saved per run to itest-output/ (gitignored)
|
||||||
|
reporters: ["default", "./itest/save-annotations.ts"],
|
||||||
|
},
|
||||||
|
});
|
||||||
@@ -0,0 +1,18 @@
|
|||||||
|
{
|
||||||
|
"$schema": "node_modules/wrangler/config-schema.json",
|
||||||
|
// model integration tests: AI always runs remotely, and costs money
|
||||||
|
"name": "good-fences-model-tests",
|
||||||
|
"main": "./src/worker.ts",
|
||||||
|
"compatibility_date": "2026-10-01",
|
||||||
|
"compatibility_flags": ["nodejs_compat"],
|
||||||
|
"ai": { "binding": "AI", "remote": true },
|
||||||
|
"worker_loaders": [{ "binding": "LOADER" }],
|
||||||
|
"durable_objects": {
|
||||||
|
"bindings": [{ "name": "Neighbor", "class_name": "Neighbor" }]
|
||||||
|
},
|
||||||
|
"exports": {
|
||||||
|
"Neighbor": { "type": "durable-object", "storage": "sqlite" },
|
||||||
|
// codemode's runtime, a facet of the agent's DO
|
||||||
|
"CodemodeRuntime": { "type": "durable-object", "storage": "sqlite" }
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -1,107 +0,0 @@
|
|||||||
This is an EmDash site -- a CMS built on Astro with a full admin UI.
|
|
||||||
|
|
||||||
## Commands
|
|
||||||
|
|
||||||
```bash
|
|
||||||
pnpm dev # Start the Astro dev server
|
|
||||||
npx emdash types # Regenerate TypeScript types from a running site
|
|
||||||
```
|
|
||||||
|
|
||||||
The admin UI is at `http://localhost:4321/_emdash/admin`.
|
|
||||||
|
|
||||||
## Key Files
|
|
||||||
|
|
||||||
| File | Purpose |
|
|
||||||
| ------------------------ | ---------------------------------------------------------------------------------- |
|
|
||||||
| `astro.config.mjs` | Astro config with `emdash()` integration, database, and storage |
|
|
||||||
| `src/live.config.ts` | EmDash loader registration (boilerplate -- don't modify) |
|
|
||||||
| `seed/seed.json` | Schema definition + demo content (collections, fields, taxonomies, menus, widgets) |
|
|
||||||
| `emdash-env.d.ts` | Generated types for collections (auto-regenerated on dev server start) |
|
|
||||||
| `src/layouts/Base.astro` | Base layout with EmDash wiring (menus, search, page contributions) |
|
|
||||||
| `src/pages/` | Astro pages -- all server-rendered |
|
|
||||||
|
|
||||||
## Skills
|
|
||||||
|
|
||||||
Agent skills are in `.agents/skills/`. Load them when working on specific tasks:
|
|
||||||
|
|
||||||
- **building-emdash-site** -- Querying content, rendering Portable Text, schema design, seed files, site features (menus, widgets, search, SEO, comments, bylines). Start here.
|
|
||||||
- **creating-plugins** -- Building EmDash plugins with hooks, storage, admin UI, API routes, and Portable Text block types.
|
|
||||||
- **emdash-cli** -- CLI commands for content management, seeding, type generation, and visual editing flow.
|
|
||||||
|
|
||||||
## Documentation
|
|
||||||
|
|
||||||
The EmDash docs are available as an MCP server at `https://docs.emdashcms.com/mcp`. When you need to verify an API, hook, config option, field type, or pattern, call `search_docs` against the live documentation rather than relying on training-data recall. The docs reflect current behaviour; assumptions may not.
|
|
||||||
|
|
||||||
This template ships with `.mcp.json`, `.cursor/mcp.json`, and `.vscode/mcp.json` so Claude Code, Cursor, and VS Code auto-discover the docs server. Other tools (OpenCode, Windsurf, etc.) need a manual one-time setup -- see [docs.emdashcms.com/docs-mcp](https://docs.emdashcms.com/docs-mcp).
|
|
||||||
|
|
||||||
## Rules
|
|
||||||
|
|
||||||
- All content pages must be server-rendered (`output: "server"`). No `getStaticPaths()` for CMS content.
|
|
||||||
- Image fields are objects (`{ src, alt }`), not strings. Use `<Image image={...} />` from `"emdash/ui"`.
|
|
||||||
- `entry.id` is the slug (for URLs). `entry.data.id` is the database ULID (for API calls like `getEntryTerms`).
|
|
||||||
- When Astro's cache is enabled, pass content-query hints to `Astro.cache.set(cacheHint)`. Use the `WithCacheHint` variants for site settings, menus, taxonomies, and widget areas rendered by cached routes.
|
|
||||||
- Taxonomy names in queries must match the seed's `"name"` field exactly (e.g., `"category"` not `"categories"`).
|
|
||||||
|
|
||||||
## This Template
|
|
||||||
|
|
||||||
A blog with posts, pages, categories, tags, full-text search, and RSS. Designed for personal writing, technical writing, indie newsletters, and anything where the writing is the product. Editorial-tech aesthetic: confident sans-serif, restrained accent, real article structure with bylines and reading time.
|
|
||||||
|
|
||||||
## Pages
|
|
||||||
|
|
||||||
| Page | Path | What it shows |
|
|
||||||
| ----------- | ------------------ | ------------------------------------------------------------------------------------------------------ |
|
|
||||||
| Home | `/` | Featured post hero (large image + excerpt), latest posts grid |
|
|
||||||
| All posts | `/posts` | Article count, full post list with excerpts and tag chips |
|
|
||||||
| Post detail | `/posts/[slug]` | Featured image, title, body, left meta column (authors + date), right TOC + search + categories gutter |
|
|
||||||
| Search | `/search` | Full-text search UI |
|
|
||||||
| Page | `/pages/[slug]` | Static page content (Portable Text) |
|
|
||||||
| Category | `/category/[slug]` | Posts filtered by category |
|
|
||||||
| Tag | `/tag/[slug]` | Posts filtered by tag |
|
|
||||||
| RSS | `/rss.xml` | Generated feed |
|
|
||||||
|
|
||||||
## Schema
|
|
||||||
|
|
||||||
- `posts` collection: `title`, `featured_image`, `content` (Portable Text), `excerpt` (text).
|
|
||||||
- `pages` collection: `title`, `content` (Portable Text). Used for `/about` etc.
|
|
||||||
- Taxonomies: `category`, `tag`.
|
|
||||||
- Single `primary` menu (Home, About, Posts by default).
|
|
||||||
|
|
||||||
Site settings have `title` and `tagline` -- both render in the header / footer.
|
|
||||||
|
|
||||||
## Visual character
|
|
||||||
|
|
||||||
Single typeface: **Inter** on `--font-body`, used for everything including headings (`--font-heading` defaults to the body face; tighter letter-spacing on h1/h2). **JetBrains Mono** on `--font-mono` for inline code and code blocks. Body and headings share the same family; weight and size carry the hierarchy (`--font-weight-heading` 600, `--font-weight-display` 700 for h1/page titles).
|
|
||||||
|
|
||||||
The brand colour is `#0066cc` (`--color-brand`) -- used for links, the post-card title hover, and the search input focus ring. There's also a secondary text colour (`--color-text-secondary`) and a `--color-muted` for meta info. Don't add a second accent.
|
|
||||||
|
|
||||||
The article layout is the standout feature: a three-column reading view with a left meta column (author bylines, date), centred 680px body column, and a right gutter for search, table of contents, and categories. Don't flatten that into one column on desktop -- the layout signals "this is something to read".
|
|
||||||
|
|
||||||
## Customisation
|
|
||||||
|
|
||||||
Design tokens live in `src/styles/tokens.css` with their default values. To restyle the site, override tokens in `src/styles/theme.css` -- declarations there are unlayered, so they always beat the `@layer base` defaults. Don't edit `tokens.css` or `Base.astro` for visual changes.
|
|
||||||
|
|
||||||
Colours are defined with `light-dark(<light>, <dark>)`, so each token carries both modes. Overriding with a plain colour changes light and dark at once; use `light-dark()` in the override to keep them distinct. There is no separate dark palette to maintain.
|
|
||||||
|
|
||||||
Webfonts are configured in `astro.config.mjs` under `fonts:`. To swap the body face, change the `name:` for the entry bound to `cssVariable: "--font-body"`. Good alternatives: Geist, IBM Plex Sans, Söhne (if you have a licence), Public Sans. If you want a serif-bodied blog, swap to a humanist serif like Source Serif, Crimson Pro, or Lora -- but then also raise `--font-size-base` to `1.0625rem` for readability. To give headings their own face (or use a system font) without touching the font pipeline, override `--font-heading` or `--font-body` in `theme.css`.
|
|
||||||
|
|
||||||
CSS variables worth knowing (see `tokens.css` for the full list):
|
|
||||||
|
|
||||||
- `--color-brand`, `--color-brand-hover`, `--color-on-brand`, `--color-brand-ring`
|
|
||||||
- `--color-bg`, `--color-bg-subtle`, `--color-surface`, `--color-text`, `--color-text-secondary`, `--color-muted`, `--color-border`, `--color-border-subtle`
|
|
||||||
- `--font-body`, `--font-heading`, `--font-mono`
|
|
||||||
- `--font-weight-heading` (600) / `--font-weight-display` (700) -- heading weights; lower them if you switch to a serif
|
|
||||||
- `--tracking-tight` / `--tracking-snug` / `--tracking-wide` / `--tracking-wider` -- letter-spacing tokens used across headings and meta labels
|
|
||||||
- `--content-width` (680px) -- article body column
|
|
||||||
- `--wide-width` (1200px) -- max container
|
|
||||||
- `--gutter-width` (200px) -- right sidebar (TOC) on article pages
|
|
||||||
- `--meta-col-width` (180px) -- left meta column on article pages
|
|
||||||
- `--avatar-size-{xs,sm,md,lg}` -- byline avatar sizes at different scales
|
|
||||||
|
|
||||||
## What not to do
|
|
||||||
|
|
||||||
- Don't add a second accent colour or coloured section backgrounds. The page should be black, white, and one blue.
|
|
||||||
- Don't replace Inter with a display sans (Bebas, Anton, etc.). Headings rely on weight contrast, not novelty faces.
|
|
||||||
- Don't collapse the article gutter on desktop -- it's part of the reading experience.
|
|
||||||
- Don't use stock blog copy ("Welcome to my blog", "Stay tuned for more"). Write a real tagline that says what this blog is about.
|
|
||||||
- Don't seed the home page with three identical placeholder posts. If you only have one real post, show one real post.
|
|
||||||
- Comments are enabled on posts and rendered on the post detail page. Configure moderation before publishing the site, or remove `commentsEnabled` and the comments UI together.
|
|
||||||
Symlink
+1
@@ -0,0 +1 @@
|
|||||||
|
CONTRIBUTING.md
|
||||||
@@ -0,0 +1,51 @@
|
|||||||
|
# Contributing to potluck
|
||||||
|
|
||||||
|
potluck started as EmDash's `cloudflare:blog` template (`pnpm create emdash@latest`); it's part of [the capsite goal tree](../capsite-checklist.md), where it is the site that publishes the weekly roundup.
|
||||||
|
|
||||||
|
## How it's put together
|
||||||
|
|
||||||
|
- **Issues** are the `issues` collection ([seed/issues.json](seed/issues.json)): `title`, `week` (an ISO week, `2026-W40`), `dek` (one-line summary), `body` (Portable Text). The slug is the week, lowercased: `/issues/2026-w40`. [issues-sketch.md](issues-sketch.md) is the dry run behind the three sample issues: real public bookmarks, editorial lines marked *draft*.
|
||||||
|
- **Pages:** home and `/issues` list issues ([src/pages/issues/](src/pages/issues/)); `/tag/<tag>` lists issues by diigo tag; search and `/rss.xml` cover issues. The `posts` collection and its pages are the template's, still in place.
|
||||||
|
- **Microformats:** an issue list is an `h-feed` of `h-entry`s; an issue is an `h-entry` (`p-name`, `dt-published`, `p-summary`, `e-content`). In issue bodies, [CiteLink.astro](src/components/CiteLink.astro) marks off-site links as `h-cite` and `/tag/<tag>` links as rel-tag `p-category`.
|
||||||
|
- **Look:** plain HTML, after kmjn.org's [Plain HTML with a few bureaucratic gestures](https://www.kmjn.org/notes/modern_plain_html.html): system fonts (no webfonts), one ~800px column, links in the text color with a dotted underline. Restyle in [src/styles/theme.css](src/styles/theme.css), which overrides the defaults in `tokens.css` (unlayered, so it always wins); don't edit `tokens.css` or `Base.astro` for visual changes. Colors use `light-dark(<light>, <dark>)`.
|
||||||
|
|
||||||
|
## Plugins
|
||||||
|
|
||||||
|
[diigo-week](../diigo-week/README.md) (Plugins → Draft from diigo in the admin) drafts an issue from a week of diigo bookmarks. It reads diigo-cache's public page as microformats over the `DIIGO_CACHE` service binding in [wrangler.jsonc](wrangler.jsonc); in dev, run diigo-cache's `pnpm dev` alongside. It's linked as `link:../diigo-week`, and registered in [astro.config.mjs](astro.config.mjs). After changing it, restart the dev server.
|
||||||
|
|
||||||
|
## Seeds
|
||||||
|
|
||||||
|
[seed/seed.json](seed/seed.json) (site settings, menus, widgets, the template's collections) is applied on the first request to an empty database. [seed/issues.json](seed/issues.json) is separate: `pnpm seed:issues` applies it to the local D1 database, which exists after the dev server's first request. Run it with the dev server stopped.
|
||||||
|
|
||||||
|
## Checks
|
||||||
|
|
||||||
|
```sh
|
||||||
|
pnpm typecheck # astro check
|
||||||
|
pnpm build
|
||||||
|
```
|
||||||
|
|
||||||
|
## EmDash rules
|
||||||
|
|
||||||
|
- All content pages are server-rendered (`output: "server"`); no `getStaticPaths()` for CMS content.
|
||||||
|
- Image fields are objects (`{ src, alt }`), not strings; render with `<Image image={...} />` from `"emdash/ui"`.
|
||||||
|
- `entry.id` is the slug (for URLs); `entry.data.id` is the database ULID (for API calls like `getEntryTerms`).
|
||||||
|
- When Astro's cache is enabled, pass query hints to `Astro.cache.set(cacheHint)`.
|
||||||
|
- Taxonomy names in queries match the seed's `"name"` exactly (`"tag"`, not `"tags"`).
|
||||||
|
- `npx emdash types` regenerates `emdash-env.d.ts` from a running site (the dev server also regenerates it on start).
|
||||||
|
|
||||||
|
The EmDash docs are an MCP server at `https://docs.emdashcms.com/mcp` (wired up in `.mcp.json`, `.cursor/mcp.json`, `.vscode/mcp.json`). Check them rather than recalling APIs from memory.
|
||||||
|
|
||||||
|
## Deploying
|
||||||
|
|
||||||
|
```sh
|
||||||
|
pnpm wrangler login
|
||||||
|
pnpm deploy
|
||||||
|
```
|
||||||
|
|
||||||
|
The first deploy provisions the D1 database and R2 bucket named in [wrangler.jsonc](wrangler.jsonc). See [Deploy to Cloudflare](https://docs.emdashcms.com/deployment/cloudflare/).
|
||||||
|
|
||||||
|
Other workerd hosts are untested: the site uses D1 and R2 through `@emdash-cms/cloudflare` and the Cloudflare adapter (locally, miniflare stands in for both), so a host has to supply those two bindings.
|
||||||
|
|
||||||
|
## For agents
|
||||||
|
|
||||||
|
`AGENTS.md` is a symlink to this file. capsite's [working rules](../AGENTS.md) apply here too (the dev server is the user's to start and stop). EmDash skills are in `.agents/skills/`: `building-emdash-site` (start here), `creating-plugins`, `emdash-cli`.
|
||||||
+14
-60
@@ -1,68 +1,22 @@
|
|||||||
# EmDash Blog Template (Cloudflare)
|
# potluck
|
||||||
|
|
||||||
A clean, minimal blog built with [EmDash](https://github.com/emdash-cms/emdash) and deployed on Cloudflare Workers with D1 and R2.
|
A weekly zine of what you bookmarked elsewhere, published on your own site. Each issue is one week of links with a few words around them.
|
||||||
|
|
||||||
[](https://deploy.workers.cloudflare.com/?url=https://github.com/emdash-cms/templates/tree/main/blog-cloudflare)
|
## Getting started
|
||||||
|
|
||||||

|
```sh
|
||||||
|
|
||||||
## What's Included
|
|
||||||
|
|
||||||
- Featured post hero on the homepage
|
|
||||||
- Post archive with reading time estimates
|
|
||||||
- Category and tag archives
|
|
||||||
- Full-text search
|
|
||||||
- RSS feed
|
|
||||||
- SEO metadata and JSON-LD
|
|
||||||
- Dark/light mode
|
|
||||||
|
|
||||||
## Pages
|
|
||||||
|
|
||||||
| Page | Route |
|
|
||||||
|---|---|
|
|
||||||
| Homepage | `/` |
|
|
||||||
| All posts | `/posts` |
|
|
||||||
| Single post | `/posts/:slug` |
|
|
||||||
| Category archive | `/category/:slug` |
|
|
||||||
| Tag archive | `/tag/:slug` |
|
|
||||||
| Search | `/search` |
|
|
||||||
| Static pages | `/pages/:slug` |
|
|
||||||
| 404 | fallback |
|
|
||||||
|
|
||||||
## Screenshots
|
|
||||||
|
|
||||||
| | Desktop | Mobile |
|
|
||||||
|---|---|---|
|
|
||||||
| Light |  |  |
|
|
||||||
| Dark |  |  |
|
|
||||||
|
|
||||||
## Infrastructure
|
|
||||||
|
|
||||||
- **Runtime:** Cloudflare Workers
|
|
||||||
- **Database:** D1
|
|
||||||
- **Storage:** R2
|
|
||||||
- **Framework:** Astro with `@astrojs/cloudflare`
|
|
||||||
|
|
||||||
## Local Development
|
|
||||||
|
|
||||||
```bash
|
|
||||||
pnpm install
|
pnpm install
|
||||||
pnpm dev
|
pnpm dev # open http://localhost:4321/_emdash/admin, finish the setup wizard, then Ctrl-C
|
||||||
|
pnpm seed:issues # three sample issues, W38–W40 of 2026
|
||||||
|
pnpm dev # → http://localhost:4321/issues
|
||||||
```
|
```
|
||||||
|
|
||||||
Open http://localhost:4321/_emdash/admin and complete the setup wizard. EmDash runs database migrations and applies the blog seed during setup. The site is available at http://localhost:4321.
|
## Bookmarks in, a zine out
|
||||||
|
|
||||||
## Deploying
|
- **Your web, your site:** the bookmarks come from your own diigo account, so the web you read becomes something you publish ([PESOS](https://indieweb.org/PESOS)). The [diigo-week](../diigo-week/README.md) plugin drafts an issue from a week of them; you cut and comment.
|
||||||
|
- **A week per issue:** `/issues/2026-w40`, by ISO week, UTC.
|
||||||
|
- **Microformats:** feed readers and IndieWeb tools see each issue as an [h-entry](https://microformats.org/wiki/h-entry) and each bookmark as an [h-cite](https://microformats.org/wiki/h-cite).
|
||||||
|
- **Plain HTML** that's easy to read: system fonts, one column.
|
||||||
|
- **A real editor:** it's an [EmDash](https://docs.emdashcms.com/) site, so you write issues in its admin UI. It runs in [workerd](https://github.com/cloudflare/workerd), on your machine or a host.
|
||||||
|
|
||||||
```bash
|
How it's put together, checks, and deploying: [CONTRIBUTING](CONTRIBUTING.md).
|
||||||
pnpm wrangler login
|
|
||||||
pnpm deploy
|
|
||||||
```
|
|
||||||
|
|
||||||
The first deployment provisions the named D1 database and R2 bucket from `wrangler.jsonc`. See [Deploy to Cloudflare](https://docs.emdashcms.com/deployment/cloudflare/) for production setup, or use the deploy button above.
|
|
||||||
|
|
||||||
## See Also
|
|
||||||
|
|
||||||
- [Node.js variant](../blog) -- same template using SQLite and local file storage
|
|
||||||
- [All templates](../)
|
|
||||||
- [EmDash documentation](https://docs.emdashcms.com/)
|
|
||||||
@@ -16,6 +16,8 @@ export default defineConfig({
|
|||||||
emdash({
|
emdash({
|
||||||
database: d1({ binding: "DB", session: "auto" }),
|
database: d1({ binding: "DB", session: "auto" }),
|
||||||
storage: r2({ binding: "MEDIA" }),
|
storage: r2({ binding: "MEDIA" }),
|
||||||
|
// drafts an issue from a week of diigo bookmarks (../diigo-week)
|
||||||
|
plugins: [{ id: "diigo-week", version: "0.1.0", entrypoint: "diigo-week" }],
|
||||||
}),
|
}),
|
||||||
],
|
],
|
||||||
devToolbar: { enabled: false },
|
devToolbar: { enabled: false },
|
||||||
|
|||||||
@@ -9,6 +9,7 @@
|
|||||||
},
|
},
|
||||||
"scripts": {
|
"scripts": {
|
||||||
"dev": "astro dev",
|
"dev": "astro dev",
|
||||||
|
"seed:issues": "emdash seed seed/issues.json -d \"$(find .wrangler/state/v3/d1 -name '*.sqlite' ! -name metadata.sqlite)\"",
|
||||||
"build": "astro build",
|
"build": "astro build",
|
||||||
"preview": "astro preview",
|
"preview": "astro preview",
|
||||||
"deploy": "astro build && wrangler deploy",
|
"deploy": "astro build && wrangler deploy",
|
||||||
@@ -19,6 +20,7 @@
|
|||||||
"@astrojs/react": "^6.0.5",
|
"@astrojs/react": "^6.0.5",
|
||||||
"@emdash-cms/cloudflare": "^1.0.1",
|
"@emdash-cms/cloudflare": "^1.0.1",
|
||||||
"astro": "^7.3.2",
|
"astro": "^7.3.2",
|
||||||
|
"diigo-week": "link:../diigo-week",
|
||||||
"emdash": "^1.0.1",
|
"emdash": "^1.0.1",
|
||||||
"react": "19.2.4",
|
"react": "19.2.4",
|
||||||
"react-dom": "19.2.4"
|
"react-dom": "19.2.4"
|
||||||
@@ -29,4 +31,4 @@
|
|||||||
"wrangler": "^4.125.0"
|
"wrangler": "^4.125.0"
|
||||||
},
|
},
|
||||||
"packageManager": "pnpm@11.9.0+sha512.bd682d5d03fe525ef7c9fd6780c6884d1e756ac4c9c9fe00c538782824310dcf90e3ddc4f53835f06dfaebd5085e41855e0bcbb3b60de2ac5bbab89e5036f03b"
|
"packageManager": "pnpm@11.9.0+sha512.bd682d5d03fe525ef7c9fd6780c6884d1e756ac4c9c9fe00c538782824310dcf90e3ddc4f53835f06dfaebd5085e41855e0bcbb3b60de2ac5bbab89e5036f03b"
|
||||||
}
|
}
|
||||||
Generated
+3
@@ -20,6 +20,9 @@ importers:
|
|||||||
astro:
|
astro:
|
||||||
specifier: ^7.3.2
|
specifier: ^7.3.2
|
||||||
version: 7.3.6(@types/node@26.6.4)(yaml@2.9.1)
|
version: 7.3.6(@types/node@26.6.4)(yaml@2.9.1)
|
||||||
|
diigo-week:
|
||||||
|
specifier: link:../diigo-week
|
||||||
|
version: link:../diigo-week
|
||||||
emdash:
|
emdash:
|
||||||
specifier: ^1.0.1
|
specifier: ^1.0.1
|
||||||
version: 1.2.0(@astrojs/react@6.0.6(@types/node@26.6.4)(@types/react-dom@19.3.0(@types/react@19.3.0))(@types/react@19.3.0)(esbuild@0.28.2)(react-dom@19.2.4(react@19.2.4))(react@19.2.4)(yaml@2.9.1))(@atcute/cbor@2.3.8(@atcute/cid@2.5.0))(@atcute/cid@2.5.0)(@atcute/identity@2.0.2(@atcute/lexicons@2.1.1)(typescript@6.0.3))(@date-fns/tz@1.5.0)(@floating-ui/dom@1.8.0)(@tiptap/extensions@3.31.4(@tiptap/core@3.31.4(@tiptap/pm@3.31.4))(@tiptap/pm@3.31.4))(@types/react-dom@19.3.0(@types/react@19.3.0))(@types/react@19.3.0)(astro@7.3.6(@types/node@26.6.4)(yaml@2.9.1))(date-fns@4.4.0)(echarts@6.1.0)(prosemirror-model@1.25.12)(prosemirror-state@1.4.4)(prosemirror-view@1.42.6)(react-dom@19.2.4(react@19.2.4))(react@19.2.4)(typescript@6.0.3)
|
version: 1.2.0(@astrojs/react@6.0.6(@types/node@26.6.4)(@types/react-dom@19.3.0(@types/react@19.3.0))(@types/react@19.3.0)(esbuild@0.28.2)(react-dom@19.2.4(react@19.2.4))(react@19.2.4)(yaml@2.9.1))(@atcute/cbor@2.3.8(@atcute/cid@2.5.0))(@atcute/cid@2.5.0)(@atcute/identity@2.0.2(@atcute/lexicons@2.1.1)(typescript@6.0.3))(@date-fns/tz@1.5.0)(@floating-ui/dom@1.8.0)(@tiptap/extensions@3.31.4(@tiptap/core@3.31.4(@tiptap/pm@3.31.4))(@tiptap/pm@3.31.4))(@types/react-dom@19.3.0(@types/react@19.3.0))(@types/react@19.3.0)(astro@7.3.6(@types/node@26.6.4)(yaml@2.9.1))(date-fns@4.4.0)(echarts@6.1.0)(prosemirror-model@1.25.12)(prosemirror-state@1.4.4)(prosemirror-view@1.42.6)(react-dom@19.2.4(react@19.2.4))(react@19.2.4)(typescript@6.0.3)
|
||||||
|
|||||||
@@ -16,6 +16,9 @@
|
|||||||
"bucket_name": "my-emdash-media",
|
"bucket_name": "my-emdash-media",
|
||||||
},
|
},
|
||||||
],
|
],
|
||||||
|
// diigo-cache's public pages, for the diigo-week plugin (reads its
|
||||||
|
// microformats; in dev, diigo-cache's own `pnpm dev` serves it)
|
||||||
|
"services": [{ "binding": "DIIGO_CACHE", "service": "diigo-cache" }],
|
||||||
// Sandboxed plugins require Worker Loader, available on Workers paid plans. Uncomment to enable:
|
// Sandboxed plugins require Worker Loader, available on Workers paid plans. Uncomment to enable:
|
||||||
// "worker_loaders": [{ "binding": "LOADER" }],
|
// "worker_loaders": [{ "binding": "LOADER" }],
|
||||||
// General maintenance cron trigger
|
// General maintenance cron trigger
|
||||||
|
|||||||
@@ -0,0 +1,4 @@
|
|||||||
|
state/
|
||||||
|
admin.sock
|
||||||
|
switchboard-*.sock
|
||||||
|
dist/
|
||||||
@@ -0,0 +1,68 @@
|
|||||||
|
# toybox
|
||||||
|
|
||||||
|
Vats whose code is installed at runtime: each vat is a Durable Object whose app runs as a [facet](https://developers.cloudflare.com/dynamic-workers/usage/durable-object-facets/) on code loaded through workerd's Worker Loader. Think `vatAdmin.createVat(bundle)`, with a supervisor per vat. Vats have no names: creating one returns its id, which addresses it only on the admin socket. Everything else a vat can do or be reached by is granted explicitly: egress by labels it may fetch within, ingress by labels it is mounted at.
|
||||||
|
|
||||||
|
- [supervisor.js](supervisor.js): `Vat` stores code by SHA-256 and runs the app on it; `Directory` holds the mounts; `Public` (the ingress) and `Admin` are the two entrypoints; `Egress` is an egress grant.
|
||||||
|
- [switchboard.js](switchboard.js): a separate workerd process, the vats' only way out, after [Flower](https://github.com/NuxiNL/flower)'s switchboard: each of its sockets serves a Cap'n Web `Handle` with fixed labels, and `constrain(labels)` returns a narrower one.
|
||||||
|
- [labels.js](labels.js): Flower's label rule, shared by both: labels only accumulate, a missing key matches anything, and a key both sides set must agree. Ingress lookup is stricter (below).
|
||||||
|
- [config.capnp](config.capnp): the sockets have no addresses; whoever starts workerd hands them in, so holding a socket is the authority it grants.
|
||||||
|
|
||||||
|
| socket | reaches | what its holder can do |
|
||||||
|
|---|---|---|
|
||||||
|
| `http` | `Public` | reach the vat mounted at labels the request carries |
|
||||||
|
| `admin` | `Admin` | `POST /` with an ES module body: create a vat and start it; the reply is `{"vat": <id>, "hash": <code hash>}`. `PUT /<id>` with a body: upgrade it; its storage carries over. `PUT /by-name/<name>`: the same, for a vat from before ids. |
|
||||||
|
|
||||||
|
Every start states the vat's grants, as JSON labels in query parameters; absent means none:
|
||||||
|
|
||||||
|
- `?egress=<labels>`: what the app may fetch (below).
|
||||||
|
- `?mount=<labels>`: where the ingress finds it. A mount must conflict with every other one (some shared key, different values), so no request can carry two mounts' labels; an overlapping mount is a 409.
|
||||||
|
|
||||||
|
## Ingress
|
||||||
|
|
||||||
|
A request's labels are its query parameters; a key given twice is a 400. Routing carries no secrets: a URL ends up in logs and `Referer`s. A secret travels in-band, as on a webkey page: the page keeps it in its fragment, reaches the vat by public labels, and sends it in the body of a call, which the vat checks like a swissnum.
|
||||||
|
|
||||||
|
The request reaches the vat whose mount labels it carries, every one with the same value. That is stricter than Flower's match (no conflicting key): an ingress client holds no constrained handle that supplies labels for it, so it reaches a vat only by naming every label of its mount. The path is the app's own. The app sees the mount's and the request's labels together in `Toybox-Connection-Labels` (JSON), as a Flower server gets its connection labels. Responses carry `Referrer-Policy: no-referrer`.
|
||||||
|
|
||||||
|
## Egress
|
||||||
|
|
||||||
|
An app must export `class App extends DurableObject` (see [apps/](apps/)). Its global `fetch` always throws: nothing in a vat is ambient. A grant arrives as `env.egress`, an `Egress` with the labels in `ctx.props`. Its `fetch`, WebSocket upgrades included, goes through a switchboard handle constrained to those labels; the switchboard refuses the rest, which comes back as a 403. `env.egress.constrain(labels)` returns a narrower `Egress`. It is an object, so the app can pass it to whichever of its parts should have it.
|
||||||
|
|
||||||
|
| label | constrains |
|
||||||
|
|---|---|
|
||||||
|
| `scheme` | `http` or `https` |
|
||||||
|
| `host` | host name or address, no port |
|
||||||
|
| `port` | explicit, or the scheme's default |
|
||||||
|
| `network` | set by the switchboard socket: `public` or `local` |
|
||||||
|
|
||||||
|
The switchboard has two sockets, so two root handles: `public` (no loopback or private addresses) and `local`. toybox holds one, as an external service (`-e switchboard=unix:…`): `pnpm start` and the systemd units give it `public`; the tests give theirs `local` to reach an echo server.
|
||||||
|
|
||||||
|
## Run it
|
||||||
|
|
||||||
|
```sh
|
||||||
|
pnpm install
|
||||||
|
pnpm test # real workerds: createVat, upgrade, mounts, egress grants and constrain, fetch and WebSocket out, crash and restart
|
||||||
|
pnpm switchboard # systemd-socket-activate on ./switchboard-public.sock and ./switchboard-local.sock
|
||||||
|
pnpm start # likewise on 127.0.0.1:8787 and ./admin.sock; state in ./state
|
||||||
|
```
|
||||||
|
|
||||||
|
Then, in a third terminal:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
admin() { curl -sg --unix-socket admin.sock "$@"; }
|
||||||
|
alice=$(admin -X POST --data-binary @apps/counter-v1.js 'http://toybox/?mount={"vat":"alice"}' | jq -r .vat)
|
||||||
|
curl 'localhost:8787/?vat=alice' # v1 count 1
|
||||||
|
admin -X PUT --data-binary @apps/counter-v2.js "http://toybox/$alice?mount={\"vat\":\"alice\"}"
|
||||||
|
curl 'localhost:8787/?vat=alice' # v2 count 2
|
||||||
|
|
||||||
|
admin -X POST --data-binary @apps/egress-probe.js 'http://toybox/?mount={"vat":"erin"}&egress={"host":"example.com"}'
|
||||||
|
curl 'localhost:8787/?vat=erin&url=https://example.com/' # got: <!doctype html>…
|
||||||
|
curl 'localhost:8787/?vat=erin&url=https://example.org/' # got: egress refused: egress to example.org not granted
|
||||||
|
```
|
||||||
|
|
||||||
|
## Under systemd
|
||||||
|
|
||||||
|
[systemd/](systemd/) has socket and service pairs for toybox and its switchboard, assuming this directory is installed at `/opt/toybox`. toybox gets no network of its own (`PrivateNetwork=yes`): only the two sockets it is handed and the switchboard's `public` socket, which only members of the `toybox` group can open. The switchboard binds its `local` socket in its own 0700 runtime directory, so no other unit holds that handle.
|
||||||
|
|
||||||
|
Neither unit has been run under the system manager yet.
|
||||||
|
|
||||||
|
`PrivateNetwork=` needs the system manager; as a `systemd-run --user` unit, systemd logs "lack privileges for network namespace, proceeding without".
|
||||||
@@ -0,0 +1,8 @@
|
|||||||
|
import { DurableObject } from "cloudflare:workers";
|
||||||
|
export class App extends DurableObject {
|
||||||
|
fetch() {
|
||||||
|
const n = (this.ctx.storage.kv.get("n") ?? 0) + 1;
|
||||||
|
this.ctx.storage.kv.put("n", n);
|
||||||
|
return new Response(`v1 count ${n}\n`);
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,8 @@
|
|||||||
|
import { DurableObject } from "cloudflare:workers";
|
||||||
|
export class App extends DurableObject {
|
||||||
|
fetch() {
|
||||||
|
const n = (this.ctx.storage.kv.get("n") ?? 0) + 1;
|
||||||
|
this.ctx.storage.kv.put("n", n);
|
||||||
|
return new Response(`v2 count ${n}\n`);
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,27 @@
|
|||||||
|
import { DurableObject } from "cloudflare:workers";
|
||||||
|
|
||||||
|
/** `?url=<target>` fetches it through `env.egress` (narrowed by `&constrain=<labels as JSON>`, if given); with `&ws=1`, opens a WebSocket there and reports the echo of "hello". */
|
||||||
|
export class App extends DurableObject {
|
||||||
|
async fetch(request) {
|
||||||
|
const params = new URL(request.url).searchParams;
|
||||||
|
const target = params.get("url");
|
||||||
|
if (!this.env.egress) return new Response("no egress granted\n");
|
||||||
|
try {
|
||||||
|
const egress = params.has("constrain")
|
||||||
|
? await this.env.egress.constrain(JSON.parse(params.get("constrain")))
|
||||||
|
: this.env.egress;
|
||||||
|
if (!params.has("ws")) return new Response(`got: ${await (await egress.fetch(target)).text()}\n`);
|
||||||
|
const resp = await egress.fetch(target, { headers: { Upgrade: "websocket" } });
|
||||||
|
const ws = resp.webSocket;
|
||||||
|
if (!ws) return new Response(`no websocket: ${resp.status} ${await resp.text()}`);
|
||||||
|
ws.accept();
|
||||||
|
const reply = new Promise((resolve) => ws.addEventListener("message", (e) => resolve(e.data)));
|
||||||
|
ws.send("hello");
|
||||||
|
const got = await reply;
|
||||||
|
ws.close();
|
||||||
|
return new Response(`echo: ${got}\n`);
|
||||||
|
} catch (e) {
|
||||||
|
return new Response(`refused: ${e.message}\n`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,9 @@
|
|||||||
|
import { DurableObject } from "cloudflare:workers";
|
||||||
|
|
||||||
|
/** Reports its path and its connection labels. */
|
||||||
|
export class App extends DurableObject {
|
||||||
|
fetch(request) {
|
||||||
|
const { pathname } = new URL(request.url);
|
||||||
|
return new Response(`${pathname} ${request.headers.get("Toybox-Connection-Labels")}\n`);
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,7 @@
|
|||||||
|
import { DurableObject } from "cloudflare:workers";
|
||||||
|
export class App extends DurableObject {
|
||||||
|
async fetch() {
|
||||||
|
try { await fetch("https://example.com/"); return new Response("fetch ALLOWED\n"); }
|
||||||
|
catch (e) { return new Response(`fetch refused: ${e.message}\n`); }
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,16 @@
|
|||||||
|
/** @file Bundle the supervisor and the OCapN vat runtime for workerd (see config.capnp). */
|
||||||
|
import { build } from "esbuild";
|
||||||
|
|
||||||
|
const common = {
|
||||||
|
bundle: true,
|
||||||
|
format: "esm",
|
||||||
|
platform: "neutral",
|
||||||
|
mainFields: ["module", "main"],
|
||||||
|
conditions: ["workerd", "import"],
|
||||||
|
external: ["cloudflare:workers", "node:*"],
|
||||||
|
logLevel: "warning",
|
||||||
|
};
|
||||||
|
|
||||||
|
await build({ ...common, entryPoints: ["supervisor.js"], outfile: "dist/supervisor.js", external: [...common.external, "vat-runtime.txt"] });
|
||||||
|
// loaded as text, beside the vat's own module (vat.js)
|
||||||
|
await build({ ...common, entryPoints: ["vat-runtime.js"], outfile: "dist/vat-runtime.js", external: [...common.external, "./vat.js"] });
|
||||||
@@ -0,0 +1,33 @@
|
|||||||
|
/**
|
||||||
|
* @file An OCapN client for the admin socket, from Node: it registers its
|
||||||
|
* own gateway with the router there and reaches every other peer through
|
||||||
|
* it. Run under SES (see test/lockdown.js).
|
||||||
|
*/
|
||||||
|
import { makeClient, swissnumFromBytes } from "@endo/ocapn";
|
||||||
|
import { newWebSocketRpcSession } from "capnweb";
|
||||||
|
import WebSocket from "ws";
|
||||||
|
import { locationOf, makeRpcNetLayer } from "./netlayer.js";
|
||||||
|
|
||||||
|
/**
|
||||||
|
* An OCapN client on the admin socket: registers its own gateway with the
|
||||||
|
* router there, and reaches every other peer through it.
|
||||||
|
*
|
||||||
|
* @param {string} adminSock
|
||||||
|
* @param {string} designator
|
||||||
|
* @param {{ verbose?: boolean }} [opts] OCapN logging
|
||||||
|
*/
|
||||||
|
export const connectClient = async (adminSock, designator, { verbose = false } = {}) => {
|
||||||
|
const ws = new WebSocket(`ws+unix:${adminSock}:/`);
|
||||||
|
await new Promise((resolve, reject) => (ws.once("open", resolve), ws.once("error", reject)));
|
||||||
|
const session = newWebSocketRpcSession(ws);
|
||||||
|
const ocapn = makeClient({ debugLabel: designator, verbose });
|
||||||
|
const netlayer = await ocapn.registerNetlayer(
|
||||||
|
makeRpcNetLayer({ designator, openGateway: (d) => session.gateway(d) }),
|
||||||
|
);
|
||||||
|
await session.register(designator, netlayer.gateway);
|
||||||
|
const vatAdmin = await ocapn.enlivenSturdyRef(
|
||||||
|
ocapn.makeSturdyRef(locationOf("vatAdmin"), swissnumFromBytes(new TextEncoder().encode("vatAdmin"))),
|
||||||
|
);
|
||||||
|
return { vatAdmin, close: () => (ocapn.shutdown(), ws.close()) };
|
||||||
|
};
|
||||||
|
|
||||||
@@ -0,0 +1,46 @@
|
|||||||
|
# toybox: vats whose code is installed at runtime, each a Durable Object facet.
|
||||||
|
#
|
||||||
|
# Sockets have no address here: whoever starts workerd hands them in
|
||||||
|
# (-S name=fd from systemd, or -s name=addr), so holding a socket is the
|
||||||
|
# authority it grants. `state` is likewise overridden with -d state=<dir>,
|
||||||
|
# and the switchboard's address with -e switchboard=<addr>.
|
||||||
|
using Workerd = import "/workerd/workerd.capnp";
|
||||||
|
|
||||||
|
const config :Workerd.Config = (
|
||||||
|
services = [
|
||||||
|
(name = "main", worker = .supervisor),
|
||||||
|
(name = "nowhere", network = (allow = [])),
|
||||||
|
# the way out: one switchboard socket, i.e. one handle (see switchboard.capnp)
|
||||||
|
(name = "switchboard", external = (address = "unix:switchboard.sock", http = ())),
|
||||||
|
(name = "state", disk = (path = "state", writable = true)),
|
||||||
|
],
|
||||||
|
sockets = [
|
||||||
|
# the ingress: reach the vat mounted at a request's labels
|
||||||
|
(name = "http", http = (), service = (name = "main", entrypoint = "Public")),
|
||||||
|
# create vats, install code, grant egress and ingress
|
||||||
|
(name = "admin", http = (), service = (name = "main", entrypoint = "Admin")),
|
||||||
|
],
|
||||||
|
);
|
||||||
|
|
||||||
|
const supervisor :Workerd.Worker = (
|
||||||
|
compatibilityDate = "2026-10-01",
|
||||||
|
# bundled by build.js
|
||||||
|
modules = [
|
||||||
|
(name = "supervisor.js", esModule = embed "dist/supervisor.js"),
|
||||||
|
(name = "vat-runtime.txt", text = embed "dist/vat-runtime.js"),
|
||||||
|
],
|
||||||
|
durableObjectNamespaces = [
|
||||||
|
(className = "Vat", uniqueKey = "toybox-vat", enableSql = true),
|
||||||
|
(className = "Directory", uniqueKey = "toybox-directory", enableSql = true),
|
||||||
|
(className = "VatAdmin", uniqueKey = "toybox-vat-admin", enableSql = true),
|
||||||
|
],
|
||||||
|
durableObjectStorage = (localDisk = "state"),
|
||||||
|
bindings = [
|
||||||
|
(name = "VATS", durableObjectNamespace = "Vat"),
|
||||||
|
(name = "DIRECTORY", durableObjectNamespace = "Directory"),
|
||||||
|
(name = "VAT_ADMIN", durableObjectNamespace = "VatAdmin"),
|
||||||
|
(name = "LOADER", workerLoader = ()),
|
||||||
|
(name = "SWITCHBOARD", service = "switchboard"),
|
||||||
|
],
|
||||||
|
globalOutbound = "nowhere",
|
||||||
|
);
|
||||||
@@ -0,0 +1,53 @@
|
|||||||
|
/**
|
||||||
|
* @file Egress labels, after [Flower](https://github.com/NuxiNL/flower)'s
|
||||||
|
* switchboard: string key-value pairs that only accumulate. A missing key
|
||||||
|
* matches anything; a key present on both sides must agree.
|
||||||
|
*
|
||||||
|
* Keys that constrain a URL: `scheme` (`http`, `https`), `host` (name or
|
||||||
|
* address, no port), `port` (explicit or the scheme's default). The
|
||||||
|
* switchboard sets `network` per socket; it constrains no URL.
|
||||||
|
*/
|
||||||
|
|
||||||
|
const DEFAULT_PORT = { http: "80", https: "443" };
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Union of two label sets; throws on a key they give different values,
|
||||||
|
* naming the key but not the values (they may end up in logs).
|
||||||
|
*
|
||||||
|
* @param {Record<string, string>} held
|
||||||
|
* @param {Record<string, string>} more
|
||||||
|
*/
|
||||||
|
export const mergeLabels = (held, more) => {
|
||||||
|
const merged = { ...held };
|
||||||
|
for (const [k, v] of Object.entries(more)) {
|
||||||
|
if (typeof v !== "string") throw new TypeError(`label ${k} is not a string`);
|
||||||
|
if (k in merged && merged[k] !== v) throw new Error(`label ${k} is already set to a different value`);
|
||||||
|
merged[k] = v;
|
||||||
|
}
|
||||||
|
return Object.freeze(merged);
|
||||||
|
};
|
||||||
|
|
||||||
|
/**
|
||||||
|
* The labels a URL carries.
|
||||||
|
*
|
||||||
|
* @param {string} url
|
||||||
|
*/
|
||||||
|
export const urlLabels = (url) => {
|
||||||
|
const u = new URL(url);
|
||||||
|
const scheme = u.protocol.replace(/:$/, "");
|
||||||
|
return { scheme, host: u.hostname, port: u.port || DEFAULT_PORT[scheme] || "" };
|
||||||
|
};
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Throws unless `url` is within `labels`.
|
||||||
|
*
|
||||||
|
* @param {Record<string, string>} labels
|
||||||
|
* @param {string} url
|
||||||
|
*/
|
||||||
|
export const checkUrl = (labels, url) => {
|
||||||
|
const actual = urlLabels(url);
|
||||||
|
for (const [k, v] of Object.entries(actual)) {
|
||||||
|
if (k in labels && labels[k] !== v) throw new Error(`egress to ${actual.host} not granted`);
|
||||||
|
}
|
||||||
|
};
|
||||||
|
|
||||||
@@ -0,0 +1,53 @@
|
|||||||
|
/**
|
||||||
|
* @file The ingress's mounts, in SQL: which vat serves which label set.
|
||||||
|
*
|
||||||
|
* Labels sit one per row, so a lookup can use an index; `mount.labels`
|
||||||
|
* keeps the whole set as JSON. Request and mount labels arrive as JSON
|
||||||
|
* objects, split into rows by `json_each`.
|
||||||
|
*/
|
||||||
|
|
||||||
|
export const MOUNT_SCHEMA = [
|
||||||
|
"CREATE TABLE IF NOT EXISTS mount (vat TEXT PRIMARY KEY, labels TEXT)",
|
||||||
|
"CREATE TABLE IF NOT EXISTS mount_label (vat TEXT, key TEXT, value TEXT, PRIMARY KEY (vat, key))",
|
||||||
|
"CREATE INDEX IF NOT EXISTS mount_label_kv ON mount_label (key, value)",
|
||||||
|
// mounts made before mount_label existed
|
||||||
|
`INSERT OR IGNORE INTO mount_label
|
||||||
|
SELECT m.vat, j.key, j.value FROM mount m, json_each(m.labels) j
|
||||||
|
WHERE NOT EXISTS (SELECT 1 FROM mount_label l WHERE l.vat = m.vat)`,
|
||||||
|
];
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Another mount (than ?2's) whose labels conflict with none of ?1's.
|
||||||
|
* Flower's registration rule: every mount must conflict with every other.
|
||||||
|
*/
|
||||||
|
export const OVERLAP = `
|
||||||
|
SELECT 1 FROM mount m WHERE m.vat != ?2 AND NOT EXISTS (
|
||||||
|
SELECT 1 FROM mount_label l JOIN json_each(?1) r ON r.key = l.key
|
||||||
|
WHERE l.vat = m.vat AND l.value != r.value)
|
||||||
|
LIMIT 1`;
|
||||||
|
|
||||||
|
export const MOUNT = "INSERT OR REPLACE INTO mount VALUES (?1, ?2)";
|
||||||
|
export const MOUNT_LABELS = "INSERT INTO mount_label SELECT ?1, key, value FROM json_each(?2)";
|
||||||
|
export const UNMOUNT = "DELETE FROM mount WHERE vat = ?1";
|
||||||
|
export const UNMOUNT_LABELS = "DELETE FROM mount_label WHERE vat = ?1";
|
||||||
|
|
||||||
|
/**
|
||||||
|
* The mount whose every label ?1 (the request's labels) carries, with the
|
||||||
|
* same value: the labels that match, counted per vat, against how many
|
||||||
|
* the vat has. A mount with no labels matches any request.
|
||||||
|
*
|
||||||
|
* Stricter than Flower's match (no conflict): an ingress client holds no
|
||||||
|
* constrained handle that supplies labels for it, so a request reaches a
|
||||||
|
* vat only by naming every label of its mount.
|
||||||
|
*/
|
||||||
|
export const LOOKUP = `
|
||||||
|
SELECT m.vat, m.labels FROM mount m JOIN (
|
||||||
|
SELECT l.vat, count(*) AS hits
|
||||||
|
FROM json_each(?1) r JOIN mount_label l ON l.key = r.key AND l.value = r.value
|
||||||
|
GROUP BY l.vat
|
||||||
|
) h ON h.vat = m.vat
|
||||||
|
WHERE h.hits = (SELECT count(*) FROM mount_label WHERE vat = m.vat)
|
||||||
|
UNION ALL
|
||||||
|
SELECT m.vat, m.labels FROM mount m
|
||||||
|
WHERE NOT EXISTS (SELECT 1 FROM mount_label l WHERE l.vat = m.vat)
|
||||||
|
LIMIT 1`;
|
||||||
@@ -0,0 +1,133 @@
|
|||||||
|
/**
|
||||||
|
* @file An OCapN netlayer whose transport is an RPC session: Cap'n Web, or
|
||||||
|
* Workers RPC, or one proxied through the other.
|
||||||
|
*
|
||||||
|
* Each OCapN connection is a pair of {@link ByteInbox}es, one per
|
||||||
|
* direction, exchanged through a gateway; each `deliver` call carries
|
||||||
|
* exactly one OCapN message, so no framing is needed. The RPC session
|
||||||
|
* provides ordering and only-the-peer-can-inject; OCapN does the rest.
|
||||||
|
*
|
||||||
|
* After vattr97's ertp-clerk (`ocapn-capnweb-netlayer.ts`), with the stub
|
||||||
|
* methods only Cap'n Web has (`onRpcBroken`) made optional.
|
||||||
|
*
|
||||||
|
* @see https://github.com/ocapn/ocapn/blob/main/draft-specifications/Netlayers.md
|
||||||
|
*/
|
||||||
|
import { RpcTarget } from "capnweb";
|
||||||
|
|
||||||
|
export const TRANSPORT = "capnweb";
|
||||||
|
|
||||||
|
/**
|
||||||
|
* A peer's location, by designator alone: a router finds the peer.
|
||||||
|
*
|
||||||
|
* @param {string} designator
|
||||||
|
*/
|
||||||
|
export const locationOf = (designator) => ({ type: "ocapn-peer", transport: TRANSPORT, designator, hints: false });
|
||||||
|
|
||||||
|
/** Same as `locationToLocationId` in `@endo/ocapn`, which its exports map doesn't expose. */
|
||||||
|
const locationId = ({ designator, transport }) => `ocapn://${designator}.${transport}`;
|
||||||
|
|
||||||
|
/** Receives one direction of an OCapN connection, one message per call. */
|
||||||
|
export class ByteInbox extends RpcTarget {
|
||||||
|
#onMessage;
|
||||||
|
|
||||||
|
constructor(onMessage) {
|
||||||
|
super();
|
||||||
|
this.#onMessage = onMessage;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** @param {Uint8Array} bytes */
|
||||||
|
deliver(bytes) {
|
||||||
|
// OCapN's syrup reader needs offset 0; a pooled Buffer view may not have it.
|
||||||
|
this.#onMessage(bytes.byteOffset === 0 ? bytes : new Uint8Array(bytes));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
const dispose = (stub) => stub?.[Symbol.dispose]?.();
|
||||||
|
|
||||||
|
/** Write to a remote inbox, discarding each result at once so the session retains none. */
|
||||||
|
const makeSocket = (inbox, close) => ({
|
||||||
|
write: (bytes) => dispose(inbox.deliver(bytes)),
|
||||||
|
end: close,
|
||||||
|
});
|
||||||
|
|
||||||
|
/**
|
||||||
|
* A remote inbox reached through a gateway that is only a promise (a
|
||||||
|
* local async call, not a pipelining stub): writes queue in order behind it.
|
||||||
|
*/
|
||||||
|
const deferredInbox = (gatewayP, myInbox) => {
|
||||||
|
const inboxP = Promise.resolve(gatewayP).then((gateway) => gateway.connect(myInbox));
|
||||||
|
return {
|
||||||
|
deliver: (bytes) => void inboxP.then((inbox) => dispose(inbox.deliver(bytes))),
|
||||||
|
onRpcBroken: (cb) => void inboxP.then((inbox) => inbox.onRpcBroken?.(cb), cb),
|
||||||
|
};
|
||||||
|
};
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Make an OCapN netlayer for `client.registerNetlayer`.
|
||||||
|
*
|
||||||
|
* @param {object} opts
|
||||||
|
* @param {string} opts.designator this peer's designator
|
||||||
|
* @param {(designator: string) => any} opts.openGateway the gateway of
|
||||||
|
* the peer with this designator: a stub or pipelining promise
|
||||||
|
* (`openGateway(d).connect(...)`), or a plain promise for one
|
||||||
|
* @returns a netlayer maker; the netlayer has a `gateway` that accepts
|
||||||
|
* incoming connections
|
||||||
|
*/
|
||||||
|
export const makeRpcNetLayer = ({ designator, openGateway }) => {
|
||||||
|
const location = locationOf(designator);
|
||||||
|
const connections = new Set();
|
||||||
|
|
||||||
|
return (handlers, logger) => {
|
||||||
|
const track = (connection, inbox) => {
|
||||||
|
connections.add(connection);
|
||||||
|
inbox.onRpcBroken?.((reason) => {
|
||||||
|
if (connections.delete(connection)) logger.log("rpc netlayer: connection broken", reason);
|
||||||
|
handlers.handleConnectionClose(connection, reason);
|
||||||
|
});
|
||||||
|
};
|
||||||
|
|
||||||
|
const netlayer = {
|
||||||
|
location,
|
||||||
|
locationId: locationId(location),
|
||||||
|
connect: (peer) => {
|
||||||
|
const gateway = openGateway(peer.designator);
|
||||||
|
let connection;
|
||||||
|
const myInbox = new ByteInbox((bytes) => handlers.handleMessageData(connection, bytes));
|
||||||
|
// Pipelined: writes before the peer answers queue in order.
|
||||||
|
const peerInbox =
|
||||||
|
typeof gateway.connect === "function" ? gateway.connect(myInbox) : deferredInbox(gateway, myInbox);
|
||||||
|
connection = handlers.makeConnection(
|
||||||
|
netlayer,
|
||||||
|
true,
|
||||||
|
makeSocket(peerInbox, () => {
|
||||||
|
connections.delete(connection);
|
||||||
|
dispose(gateway);
|
||||||
|
}),
|
||||||
|
);
|
||||||
|
track(connection, peerInbox);
|
||||||
|
return connection;
|
||||||
|
},
|
||||||
|
shutdown: () => {
|
||||||
|
for (const connection of [...connections]) connection.end();
|
||||||
|
},
|
||||||
|
};
|
||||||
|
|
||||||
|
class Gateway extends RpcTarget {
|
||||||
|
connect(peerInbox) {
|
||||||
|
const peer = peerInbox.dup?.() ?? peerInbox;
|
||||||
|
const connection = handlers.makeConnection(
|
||||||
|
netlayer,
|
||||||
|
false,
|
||||||
|
makeSocket(peer, () => {
|
||||||
|
connections.delete(connection);
|
||||||
|
dispose(peer);
|
||||||
|
}),
|
||||||
|
);
|
||||||
|
track(connection, peer);
|
||||||
|
return new ByteInbox((bytes) => handlers.handleMessageData(connection, bytes));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
return Object.assign(netlayer, { gateway: new Gateway() });
|
||||||
|
};
|
||||||
|
};
|
||||||
@@ -0,0 +1,24 @@
|
|||||||
|
{
|
||||||
|
"name": "toybox",
|
||||||
|
"version": "0.0.1",
|
||||||
|
"private": true,
|
||||||
|
"type": "module",
|
||||||
|
"scripts": {
|
||||||
|
"start": "node build.js && mkdir -p state && rm -f admin.sock && systemd-socket-activate -l 127.0.0.1:8787 -l \"$PWD/admin.sock\" -- node_modules/workerd/bin/workerd serve --experimental config.capnp -S http=3 -S admin=4 -d state=state -e \"switchboard=unix:$PWD/switchboard-public.sock\"",
|
||||||
|
"switchboard": "rm -f switchboard-public.sock switchboard-local.sock && systemd-socket-activate -l \"$PWD/switchboard-public.sock\" -l \"$PWD/switchboard-local.sock\" -- node_modules/workerd/bin/workerd serve --experimental switchboard.capnp -S public=3 -S local=4",
|
||||||
|
"test": "node build.js && node --import ./test/lockdown.js --test test/*.test.js",
|
||||||
|
"build": "node build.js",
|
||||||
|
"hello": "node --import ./test/lockdown.js scripts/hello.js"
|
||||||
|
},
|
||||||
|
"devDependencies": {
|
||||||
|
"esbuild": "^0.28.2",
|
||||||
|
"workerd": "1.20261006.1",
|
||||||
|
"ws": "8.22.0"
|
||||||
|
},
|
||||||
|
"dependencies": {
|
||||||
|
"@endo/far": "^1.1.14",
|
||||||
|
"@endo/init": "^1.1.13",
|
||||||
|
"@endo/ocapn": "1.1.1",
|
||||||
|
"capnweb": "0.12.0"
|
||||||
|
}
|
||||||
|
}
|
||||||
Generated
+578
@@ -0,0 +1,578 @@
|
|||||||
|
lockfileVersion: '9.0'
|
||||||
|
|
||||||
|
settings:
|
||||||
|
autoInstallPeers: true
|
||||||
|
excludeLinksFromLockfile: false
|
||||||
|
|
||||||
|
importers:
|
||||||
|
|
||||||
|
.:
|
||||||
|
dependencies:
|
||||||
|
'@endo/far':
|
||||||
|
specifier: ^1.1.14
|
||||||
|
version: 1.1.14
|
||||||
|
'@endo/init':
|
||||||
|
specifier: ^1.1.13
|
||||||
|
version: 1.1.13
|
||||||
|
'@endo/ocapn':
|
||||||
|
specifier: 1.1.1
|
||||||
|
version: 1.1.1
|
||||||
|
capnweb:
|
||||||
|
specifier: 0.12.0
|
||||||
|
version: 0.12.0
|
||||||
|
devDependencies:
|
||||||
|
esbuild:
|
||||||
|
specifier: ^0.28.2
|
||||||
|
version: 0.28.2
|
||||||
|
workerd:
|
||||||
|
specifier: 1.20261006.1
|
||||||
|
version: 1.20261006.1
|
||||||
|
ws:
|
||||||
|
specifier: 8.22.0
|
||||||
|
version: 8.22.0
|
||||||
|
|
||||||
|
packages:
|
||||||
|
|
||||||
|
'@cloudflare/workerd-darwin-64@1.20261006.1':
|
||||||
|
resolution: {integrity: sha512-lCX6avJO2It9AyDEYitjmOtJRVp/xmaW0TziyoLAnfxlzozyyRbSFfIIMMCiDx/sTP1++E4VLbV+7BHSD33f0Q==}
|
||||||
|
engines: {node: '>=16'}
|
||||||
|
cpu: [x64]
|
||||||
|
os: [darwin]
|
||||||
|
|
||||||
|
'@cloudflare/workerd-darwin-arm64@1.20261006.1':
|
||||||
|
resolution: {integrity: sha512-PyFGjOaldtppnLuKsdkoYWXfLVpDjuf8u+uoykX+1Xvxo2EPmCVViq0iDKm+U7c7PRno85ehS3/MPVXyk2HwHw==}
|
||||||
|
engines: {node: '>=16'}
|
||||||
|
cpu: [arm64]
|
||||||
|
os: [darwin]
|
||||||
|
|
||||||
|
'@cloudflare/workerd-linux-64@1.20261006.1':
|
||||||
|
resolution: {integrity: sha512-Ntf9S8eesmVufdjaT2kzWlz6Ng9XGYTc6cHQVRKAg59lFufwsIabGeMT6U4KkrpEeSIxgHq8aPtuLbZXE19tCA==}
|
||||||
|
engines: {node: '>=16'}
|
||||||
|
cpu: [x64]
|
||||||
|
os: [linux]
|
||||||
|
|
||||||
|
'@cloudflare/workerd-linux-arm64@1.20261006.1':
|
||||||
|
resolution: {integrity: sha512-JyeDBymVWw5tVRebSsVE8ojDYxd6UlZrAseHU8G+NpIyd5wF61tXlF/axLeCUqu0pOhBFHb8I5QKi0jwrCK41Q==}
|
||||||
|
engines: {node: '>=16'}
|
||||||
|
cpu: [arm64]
|
||||||
|
os: [linux]
|
||||||
|
|
||||||
|
'@cloudflare/workerd-windows-64@1.20261006.1':
|
||||||
|
resolution: {integrity: sha512-nWP0URNqOBN9fa/gZ7eh/w051DwjOXN8j4lDZDYVXyQPsux6sV3DfyYZT+CjR/rt9nKBSmJT9QE9/bWT18+84w==}
|
||||||
|
engines: {node: '>=16'}
|
||||||
|
cpu: [x64]
|
||||||
|
os: [win32]
|
||||||
|
|
||||||
|
'@endo/base64@1.1.1':
|
||||||
|
resolution: {integrity: sha512-6Aiy5oU/5/LYrHLbEPz0OWQ+CQsnAOC3qQYQ3nlwWi2JsU8fIZWjwfa+/+MSx8LTzITC250ZnMvtR0y/FIOQKw==}
|
||||||
|
|
||||||
|
'@endo/bytes@1.0.2':
|
||||||
|
resolution: {integrity: sha512-gKG8v4c7SD/QzULlFl1OKBuUCCy+9H78xsCL40hZn1q2/hVCNAHrxPJFO6VbA8Nk8UJP6P+NjvZY+xdrfwytMQ==}
|
||||||
|
|
||||||
|
'@endo/cache-map@1.1.0':
|
||||||
|
resolution: {integrity: sha512-owFGshs/97PDw9oguZqU/px8Lv1d0KjAUtDUiPwKHNXRVUE/jyettEbRoTbNJR1OaI8biMn6bHr9kVJsOh6dXw==}
|
||||||
|
|
||||||
|
'@endo/common@1.4.0':
|
||||||
|
resolution: {integrity: sha512-+d0CCzEYUP4PZDVGAUHMNfCEBn/GOyq/39y33N4FFUb6VPNJosfKFK5bI38mTaqwr6gxLGbzm8aeiElz+NaSDQ==}
|
||||||
|
|
||||||
|
'@endo/env-options@1.1.11':
|
||||||
|
resolution: {integrity: sha512-p9OnAPsdqoX4YJsE98e3NBVhIr2iW9gNZxHhAI2/Ul5TdRfoOViItzHzTqrgUVopw6XxA1u1uS6CykLMDUxarA==}
|
||||||
|
|
||||||
|
'@endo/errors@1.3.1':
|
||||||
|
resolution: {integrity: sha512-AmR2DSiahQR66SHlEczNWe0eUotMYE2uDwxPj9fAY2l2lovwHWGzZKXNXdQP/u62c8vetU8mG98Mh5vRZEvFTg==}
|
||||||
|
|
||||||
|
'@endo/eventual-send@1.5.0':
|
||||||
|
resolution: {integrity: sha512-mIRewS15mk+oV8y7kszyE0KWHZAlVoe98bEo/WJFJvBv1Rd83IjnofbuZcyWToWd70vgCxfkutWRJBK2Xtwd5A==}
|
||||||
|
|
||||||
|
'@endo/far@1.1.14':
|
||||||
|
resolution: {integrity: sha512-OlM+yCNWVqhwSZa6MyjkVB0oQSTAQ15KoiDGL1qowtnTnhuGUbOXuaUAsDf4eAJZ4L+h2KsJOUdyYEuhNsDPLg==}
|
||||||
|
|
||||||
|
'@endo/harden@1.1.0':
|
||||||
|
resolution: {integrity: sha512-d0CF7EVqymx9YUbkRI2sj10NrMLhrpmsaq5aBbEEbWDFxNe7F/mM8Gv8zT+iHErKEfeMCId90n5mk665SnxrRA==}
|
||||||
|
|
||||||
|
'@endo/hex@1.1.1':
|
||||||
|
resolution: {integrity: sha512-I2Q7roaHk0fE4JTj2GXfxHPR14qTFK9bOvp3nKf2Fk0r/CjgL4zFcHYolvqY15SRNmZDcywnQ+MBTW82b7BUpQ==}
|
||||||
|
|
||||||
|
'@endo/immutable-arraybuffer@2.0.0':
|
||||||
|
resolution: {integrity: sha512-xvfJvo4CgETh6UjSKwhNsFLZrqrTf2k4ZFpcprEN2UtCQRSOk+CcLP70kFbrOesB2Z/c1z9MaUZDS1JY6IlnUg==}
|
||||||
|
|
||||||
|
'@endo/init@1.1.13':
|
||||||
|
resolution: {integrity: sha512-7I94pHD0ax+WDj5dP1MKgzgSY6/GsdwCc/Ap2zFY4lzhCO944zQ4MChAW2yX28QKJK80VyJzqoW5BQ56ngEHHA==}
|
||||||
|
|
||||||
|
'@endo/lockdown@1.0.19':
|
||||||
|
resolution: {integrity: sha512-pcc6TqTBlx43422gHFzS+FrUfyJlbDoWUrynp78Mv+FrIZv4947FPcfjlXolgwvO00alMlSYxfXrrXUV93ywaA==}
|
||||||
|
|
||||||
|
'@endo/marshal@1.10.0':
|
||||||
|
resolution: {integrity: sha512-1ny6r0+XFpkpJNJEOApZQlFNB8FsDxzU2oCv1hpN1HjpszsKcTiyfXU/n/cJIs3KOOS06Tkt72pgPlasHpsVGA==}
|
||||||
|
|
||||||
|
'@endo/nat@5.2.0':
|
||||||
|
resolution: {integrity: sha512-yTNaysErNbrAWV8H2oC4e83nytG8Vzcu6VrxOkSIJQt+IVS9BUlSa1KkiKrVwu2IcIC67gJIkolPq3SgMGDjfQ==}
|
||||||
|
|
||||||
|
'@endo/ocapn@1.1.1':
|
||||||
|
resolution: {integrity: sha512-3VtHmyMsHwDPwcVusCTgtb8VXr539iZk+r8vzzOU6/cWyMhJuxRGOKR3r5EzDHVG4tY/R6tihoGt7QLnNDuceQ==}
|
||||||
|
|
||||||
|
'@endo/pass-style@1.8.2':
|
||||||
|
resolution: {integrity: sha512-63kbFNT+mropjQ0wzCkjSR5uEOT8FAIk16qHqrE+ciqSjMVd2hsW6vx10dbyo1s2xtmvXduUiTl+LqtnPa2oSw==}
|
||||||
|
|
||||||
|
'@endo/promise-kit@1.2.1':
|
||||||
|
resolution: {integrity: sha512-1nyXBOuUnVo/i47On8RCXPaT/1RIsSpJ1sGBClfZrcTtnsJfj8fElEVlJIE8NFsQiZPLNANYI+i/NYi837+s0Q==}
|
||||||
|
engines: {node: '>=11.0'}
|
||||||
|
|
||||||
|
'@endo/stream@1.3.1':
|
||||||
|
resolution: {integrity: sha512-v21VRzo5wjzw7XMHCWAmDS50hkUQi/m9C0MQTIAAqLm6w9+/Er9AEXl8FGIbz+VjtJWk5SNVbuqA51N7oBaLDQ==}
|
||||||
|
|
||||||
|
'@endo/syrup-frame@0.1.1':
|
||||||
|
resolution: {integrity: sha512-+uha+zI05RHPlSNVI7uaB2KD1EQ9gKAMcOzj1eT3y7Ng/vcW1bo7clCVVo3t+dOFOc6vYSsXgKufi40qxXRgYQ==}
|
||||||
|
|
||||||
|
'@esbuild/aix-ppc64@0.28.2':
|
||||||
|
resolution: {integrity: sha512-XExcO+dvLKvVtNTibSTBej1NCAbaGhWn9Ww1ZPx80qsahhPFe/8jgWP0IchNe0F3HwkU7n8ejhH8bjonqht8mQ==}
|
||||||
|
engines: {node: '>=18'}
|
||||||
|
cpu: [ppc64]
|
||||||
|
os: [aix]
|
||||||
|
|
||||||
|
'@esbuild/android-arm64@0.28.2':
|
||||||
|
resolution: {integrity: sha512-5YfKeeI8qWfBZIX+u2xZC3Zlb3Os/gLS2sbEKM+I4ZOcsWmHS2WLysCcQZDAFRslDUU5Oiq44gf6PYN1vGwG5A==}
|
||||||
|
engines: {node: '>=18'}
|
||||||
|
cpu: [arm64]
|
||||||
|
os: [android]
|
||||||
|
|
||||||
|
'@esbuild/android-arm@0.28.2':
|
||||||
|
resolution: {integrity: sha512-kXXoiPVVGQcnIYGOeaovwOURpniDBpSq4A03qkQ+BMQqtGG6HYap3xne9C1O1yo4TR3qxlCX5IqqmX6fFo2Lqg==}
|
||||||
|
engines: {node: '>=18'}
|
||||||
|
cpu: [arm]
|
||||||
|
os: [android]
|
||||||
|
|
||||||
|
'@esbuild/android-x64@0.28.2':
|
||||||
|
resolution: {integrity: sha512-O387ite7SzUyCcy3JQX4P4bLtEA7bLLkx+esve5JHnyYfNTxcVpXZo9jhdB0lTKN44gztELTdU7nS8Nr16Fs1Q==}
|
||||||
|
engines: {node: '>=18'}
|
||||||
|
cpu: [x64]
|
||||||
|
os: [android]
|
||||||
|
|
||||||
|
'@esbuild/darwin-arm64@0.28.2':
|
||||||
|
resolution: {integrity: sha512-n4KqkOQrraxHJcgjM1RvwbigfQKIKJVpM7xp+KsxiyUSrRdIXnt73VhrPAx0fV44hgfmIVKjxMN9J1t5jySVkw==}
|
||||||
|
engines: {node: '>=18'}
|
||||||
|
cpu: [arm64]
|
||||||
|
os: [darwin]
|
||||||
|
|
||||||
|
'@esbuild/darwin-x64@0.28.2':
|
||||||
|
resolution: {integrity: sha512-uq6suIWYP37qzGddBKPw5QEQPi6HiLGsO7UmkpfyaYNQ3D+rN6w6WfwH+nuqcGXWvawGwxOEroO4YGnFh95azw==}
|
||||||
|
engines: {node: '>=18'}
|
||||||
|
cpu: [x64]
|
||||||
|
os: [darwin]
|
||||||
|
|
||||||
|
'@esbuild/freebsd-arm64@0.28.2':
|
||||||
|
resolution: {integrity: sha512-n+I0BTSRIoy+d6RPKnEVwql5UwBJolytvY4mAOIEJorKlqgPII8ix6slVVrfZ5Tnj7glIZvloylbB/EJPMWEXw==}
|
||||||
|
engines: {node: '>=18'}
|
||||||
|
cpu: [arm64]
|
||||||
|
os: [freebsd]
|
||||||
|
|
||||||
|
'@esbuild/freebsd-x64@0.28.2':
|
||||||
|
resolution: {integrity: sha512-78XJTJkvPs0kz2w61301PJjXl4g7q3JqiYMZ/M/yVI73EHBrCRTgkhu9oqG7vPqq+a/yadEW8aD+agKlk5xrmg==}
|
||||||
|
engines: {node: '>=18'}
|
||||||
|
cpu: [x64]
|
||||||
|
os: [freebsd]
|
||||||
|
|
||||||
|
'@esbuild/linux-arm64@0.28.2':
|
||||||
|
resolution: {integrity: sha512-pW4AC0P3it8c7do9MVM4p51FzHzdM/TZrerurgRcHJ2WTa1VQ1CIq18xncfpBJw4ojkiZZrKW2yIBWBP92j6Ug==}
|
||||||
|
engines: {node: '>=18'}
|
||||||
|
cpu: [arm64]
|
||||||
|
os: [linux]
|
||||||
|
|
||||||
|
'@esbuild/linux-arm@0.28.2':
|
||||||
|
resolution: {integrity: sha512-XlDnu2q5yoqems+xay6wSAcg9DDD7K9RLKZEBOMZm3ckNpJBvOX20tSfby8KfrrhINDyv9V2YVZKY/SpoGJI8w==}
|
||||||
|
engines: {node: '>=18'}
|
||||||
|
cpu: [arm]
|
||||||
|
os: [linux]
|
||||||
|
|
||||||
|
'@esbuild/linux-ia32@0.28.2':
|
||||||
|
resolution: {integrity: sha512-CYbnj78HsIeA+DhgUKgFCfvNsTHFhMMrinUrMZpDXJXKN8T3XViTZ/+wtHeVxEWY8ewSzTFN+nRmSwO2tZaLUQ==}
|
||||||
|
engines: {node: '>=18'}
|
||||||
|
cpu: [ia32]
|
||||||
|
os: [linux]
|
||||||
|
|
||||||
|
'@esbuild/linux-loong64@0.28.2':
|
||||||
|
resolution: {integrity: sha512-buwkd8nsph4R+ajRvw0qM5Hja/TXQow3ptzWO2EbG/cqcIkHloRrdlBtQlshyYGTNFvfkfJ5tpPLVkY4DtsPfQ==}
|
||||||
|
engines: {node: '>=18'}
|
||||||
|
cpu: [loong64]
|
||||||
|
os: [linux]
|
||||||
|
|
||||||
|
'@esbuild/linux-mips64el@0.28.2':
|
||||||
|
resolution: {integrity: sha512-ZVykbDyk7519VwiNb9Lcj9m8XM6v5V9uKPvrEMkkEedVewf+0itkhahp4HDpgERXhwLRpWFypsGbG/J8s0QjJA==}
|
||||||
|
engines: {node: '>=18'}
|
||||||
|
cpu: [mips64el]
|
||||||
|
os: [linux]
|
||||||
|
|
||||||
|
'@esbuild/linux-ppc64@0.28.2':
|
||||||
|
resolution: {integrity: sha512-CAXl+Dtd9UUuJd8pKKdwh6MLm3MUMiqMPmhZ3tTSXPqfyQ3vDl6R5hZdZ/kYojK4ofXtdfSv1tFq8XzWx3heNQ==}
|
||||||
|
engines: {node: '>=18'}
|
||||||
|
cpu: [ppc64]
|
||||||
|
os: [linux]
|
||||||
|
|
||||||
|
'@esbuild/linux-riscv64@0.28.2':
|
||||||
|
resolution: {integrity: sha512-GeXCej4IQtU1B+QlDV8W/RRvbzI3O/Stss+/bCXv4lZls5WGRtu2a+3JkA3i4qIUlMXpcHebWpF8AkJhATowuA==}
|
||||||
|
engines: {node: '>=18'}
|
||||||
|
cpu: [riscv64]
|
||||||
|
os: [linux]
|
||||||
|
|
||||||
|
'@esbuild/linux-s390x@0.28.2':
|
||||||
|
resolution: {integrity: sha512-3H1weTYZPxt/WOhByszQZybS9w5lKzUn1FDMsgEChbHWQwHYQQRfBxgCcZvPhjHfKyJjIievvMmEUawJrdY9Dg==}
|
||||||
|
engines: {node: '>=18'}
|
||||||
|
cpu: [s390x]
|
||||||
|
os: [linux]
|
||||||
|
|
||||||
|
'@esbuild/linux-x64@0.28.2':
|
||||||
|
resolution: {integrity: sha512-4xTZr1FUmSoQW4XIWmit3tzQrUTZM+N3P0XV8xROKYF50XfI7xeO90+1bZvNwxIufQ9hDQVRJH5YhgPVF8A/HQ==}
|
||||||
|
engines: {node: '>=18'}
|
||||||
|
cpu: [x64]
|
||||||
|
os: [linux]
|
||||||
|
|
||||||
|
'@esbuild/netbsd-arm64@0.28.2':
|
||||||
|
resolution: {integrity: sha512-sSATRjPeDBg3pdgHoQfoYBob11Kk1FGa9lui5RIHZCoCkJa9QKlvl3/vKz2usCmYYjs7ymJR/2Nnsqe+Hjt5nw==}
|
||||||
|
engines: {node: '>=18'}
|
||||||
|
cpu: [arm64]
|
||||||
|
os: [netbsd]
|
||||||
|
|
||||||
|
'@esbuild/netbsd-x64@0.28.2':
|
||||||
|
resolution: {integrity: sha512-lqnzCV+mM0gIADaKihiCg6ifgfU2L3h5E33rNQBN1Y4MaVGnzryzmvvf7UHxprpQdE8hpqLolJ9Rl+SkIRDpyw==}
|
||||||
|
engines: {node: '>=18'}
|
||||||
|
cpu: [x64]
|
||||||
|
os: [netbsd]
|
||||||
|
|
||||||
|
'@esbuild/openbsd-arm64@0.28.2':
|
||||||
|
resolution: {integrity: sha512-AL2qJILH7lNjrDmCQDvdxMfAUIv8KMNZOvrwAQ8i8//ntL9FflhOyMJ8OZSMBb8/AWXe3/5v5S20y3zCoZWKoQ==}
|
||||||
|
engines: {node: '>=18'}
|
||||||
|
cpu: [arm64]
|
||||||
|
os: [openbsd]
|
||||||
|
|
||||||
|
'@esbuild/openbsd-x64@0.28.2':
|
||||||
|
resolution: {integrity: sha512-QtiuPytchRyC4rwUKhexJdQKvDuZ6hWloi3igqPQNUJCS1/v9EiO3UTOXR6A3FoMo4fnAKbWJdqaIwhOzh8qEw==}
|
||||||
|
engines: {node: '>=18'}
|
||||||
|
cpu: [x64]
|
||||||
|
os: [openbsd]
|
||||||
|
|
||||||
|
'@esbuild/openharmony-arm64@0.28.2':
|
||||||
|
resolution: {integrity: sha512-WkhYDmpTjLvGlScA1rwjRUmhl4k8oXR3cIbtqWmELgU/dFeHHlEllxDvdWcNJV9rbzCexB5vz8gtNewWLgCT7Q==}
|
||||||
|
engines: {node: '>=18'}
|
||||||
|
cpu: [arm64]
|
||||||
|
os: [openharmony]
|
||||||
|
|
||||||
|
'@esbuild/sunos-x64@0.28.2':
|
||||||
|
resolution: {integrity: sha512-GPMSkTOtMnv2U2F8gxe4Io6qmVs+YKyp832Etqqxr0hFngmXQ3rzwytelm3GIn7T4VviRUlf3sOgBOiTdvaf7g==}
|
||||||
|
engines: {node: '>=18'}
|
||||||
|
cpu: [x64]
|
||||||
|
os: [sunos]
|
||||||
|
|
||||||
|
'@esbuild/win32-arm64@0.28.2':
|
||||||
|
resolution: {integrity: sha512-PIhhEkE9uPBleRBrQEJpUn7MBnibZzbGzYWPmY3x+YoVg/95zbjB4CxPPOQ8l5tYYM4mMaCthF8/1DIfBQQyWQ==}
|
||||||
|
engines: {node: '>=18'}
|
||||||
|
cpu: [arm64]
|
||||||
|
os: [win32]
|
||||||
|
|
||||||
|
'@esbuild/win32-ia32@0.28.2':
|
||||||
|
resolution: {integrity: sha512-YmJbfTlvU7Sdn9BB+4PRES4oB6pxgS37MAONj+hBr/cpXS1aBPKXxNnDbu+QCWPj0o9dgyxeq79g6c5P8KeuYA==}
|
||||||
|
engines: {node: '>=18'}
|
||||||
|
cpu: [ia32]
|
||||||
|
os: [win32]
|
||||||
|
|
||||||
|
'@esbuild/win32-x64@0.28.2':
|
||||||
|
resolution: {integrity: sha512-5ebpxr3nWMzrL/rnUI755Jkuee0bHL/Gq0WTF9lvcpv73wAp5eu8MfBUgWK9bhWvZjj7yX8etf/8tI8Ney695g==}
|
||||||
|
engines: {node: '>=18'}
|
||||||
|
cpu: [x64]
|
||||||
|
os: [win32]
|
||||||
|
|
||||||
|
'@noble/curves@1.9.7':
|
||||||
|
resolution: {integrity: sha512-gbKGcRUYIjA3/zCCNaWDciTMFI0dCkvou3TL8Zmy5Nc7sJ47a0jtOeZoTaMxkuqRo9cRhjOdZJXegxYE5FN/xw==}
|
||||||
|
engines: {node: ^14.21.3 || >=16}
|
||||||
|
|
||||||
|
'@noble/hashes@1.8.0':
|
||||||
|
resolution: {integrity: sha512-jCs9ldd7NwzpgXDIf6P3+NrHh9/sD6CQdxHyjQI+h/6rDNo88ypBxxz45UDuZHz9r3tNz7N/VInSVoVdtXEI4A==}
|
||||||
|
engines: {node: ^14.21.3 || >=16}
|
||||||
|
|
||||||
|
capnweb@0.12.0:
|
||||||
|
resolution: {integrity: sha512-jgZ/LMtMVTi+RVlooIslH78Gg23XbDTdvcLghWx3oz17D6u5GuJARt+uhZk/wcTo+f81eeabfnow4d3zLBj+JQ==}
|
||||||
|
|
||||||
|
esbuild@0.28.2:
|
||||||
|
resolution: {integrity: sha512-HKVLS8dvII+xoKW9kmqxbRKrnWEXfJJr/FZhhJmiqIB0e053QNYFqOBouTMO/k5sID4MvCiUCvv8b9M4h32wIA==}
|
||||||
|
engines: {node: '>=18'}
|
||||||
|
hasBin: true
|
||||||
|
|
||||||
|
ses@2.3.0:
|
||||||
|
resolution: {integrity: sha512-qd3iWzDqGKllI2FmExKsWLrJwt+6COWb8jUdzaCn8Cq5OTeoXdQ62Gn/xLy0GWLORC3du6yBAVe/V0B0dLVKew==}
|
||||||
|
|
||||||
|
workerd@1.20261006.1:
|
||||||
|
resolution: {integrity: sha512-8MadBuffZ5NdpLwZjUmsqT5QlaKXq4gjgT6wzcKeynZTICNXyePhc+5QVoKiK7+MMhFTLNrRO2KXLlpk8Q6BkA==}
|
||||||
|
engines: {node: '>=16'}
|
||||||
|
hasBin: true
|
||||||
|
|
||||||
|
ws@8.22.0:
|
||||||
|
resolution: {integrity: sha512-Ydggc987+RO0AnWtZ/7Wq9FtNvcrL1b/RO0ud9mWjUPgDrsAAwQSF51sm2hm1XofbU/4jkpGEsLFsZZxU+1DOg==}
|
||||||
|
engines: {node: '>=10.0.0'}
|
||||||
|
peerDependencies:
|
||||||
|
bufferutil: ^4.0.1
|
||||||
|
utf-8-validate: '>=5.0.2'
|
||||||
|
peerDependenciesMeta:
|
||||||
|
bufferutil:
|
||||||
|
optional: true
|
||||||
|
utf-8-validate:
|
||||||
|
optional: true
|
||||||
|
|
||||||
|
snapshots:
|
||||||
|
|
||||||
|
'@cloudflare/workerd-darwin-64@1.20261006.1':
|
||||||
|
optional: true
|
||||||
|
|
||||||
|
'@cloudflare/workerd-darwin-arm64@1.20261006.1':
|
||||||
|
optional: true
|
||||||
|
|
||||||
|
'@cloudflare/workerd-linux-64@1.20261006.1':
|
||||||
|
optional: true
|
||||||
|
|
||||||
|
'@cloudflare/workerd-linux-arm64@1.20261006.1':
|
||||||
|
optional: true
|
||||||
|
|
||||||
|
'@cloudflare/workerd-windows-64@1.20261006.1':
|
||||||
|
optional: true
|
||||||
|
|
||||||
|
'@endo/base64@1.1.1': {}
|
||||||
|
|
||||||
|
'@endo/bytes@1.0.2':
|
||||||
|
dependencies:
|
||||||
|
'@endo/harden': 1.1.0
|
||||||
|
'@endo/immutable-arraybuffer': 2.0.0
|
||||||
|
|
||||||
|
'@endo/cache-map@1.1.0': {}
|
||||||
|
|
||||||
|
'@endo/common@1.4.0':
|
||||||
|
dependencies:
|
||||||
|
'@endo/errors': 1.3.1
|
||||||
|
'@endo/eventual-send': 1.5.0
|
||||||
|
'@endo/harden': 1.1.0
|
||||||
|
'@endo/promise-kit': 1.2.1
|
||||||
|
|
||||||
|
'@endo/env-options@1.1.11': {}
|
||||||
|
|
||||||
|
'@endo/errors@1.3.1':
|
||||||
|
dependencies:
|
||||||
|
'@endo/harden': 1.1.0
|
||||||
|
ses: 2.3.0
|
||||||
|
|
||||||
|
'@endo/eventual-send@1.5.0':
|
||||||
|
dependencies:
|
||||||
|
'@endo/env-options': 1.1.11
|
||||||
|
'@endo/harden': 1.1.0
|
||||||
|
|
||||||
|
'@endo/far@1.1.14':
|
||||||
|
dependencies:
|
||||||
|
'@endo/errors': 1.3.1
|
||||||
|
'@endo/eventual-send': 1.5.0
|
||||||
|
'@endo/pass-style': 1.8.2
|
||||||
|
|
||||||
|
'@endo/harden@1.1.0': {}
|
||||||
|
|
||||||
|
'@endo/hex@1.1.1':
|
||||||
|
dependencies:
|
||||||
|
'@endo/harden': 1.1.0
|
||||||
|
|
||||||
|
'@endo/immutable-arraybuffer@2.0.0': {}
|
||||||
|
|
||||||
|
'@endo/init@1.1.13':
|
||||||
|
dependencies:
|
||||||
|
'@endo/base64': 1.1.1
|
||||||
|
'@endo/eventual-send': 1.5.0
|
||||||
|
'@endo/harden': 1.1.0
|
||||||
|
'@endo/lockdown': 1.0.19
|
||||||
|
'@endo/promise-kit': 1.2.1
|
||||||
|
|
||||||
|
'@endo/lockdown@1.0.19':
|
||||||
|
dependencies:
|
||||||
|
ses: 2.3.0
|
||||||
|
|
||||||
|
'@endo/marshal@1.10.0':
|
||||||
|
dependencies:
|
||||||
|
'@endo/common': 1.4.0
|
||||||
|
'@endo/env-options': 1.1.11
|
||||||
|
'@endo/errors': 1.3.1
|
||||||
|
'@endo/eventual-send': 1.5.0
|
||||||
|
'@endo/harden': 1.1.0
|
||||||
|
'@endo/nat': 5.2.0
|
||||||
|
'@endo/pass-style': 1.8.2
|
||||||
|
|
||||||
|
'@endo/nat@5.2.0': {}
|
||||||
|
|
||||||
|
'@endo/ocapn@1.1.1':
|
||||||
|
dependencies:
|
||||||
|
'@endo/bytes': 1.0.2
|
||||||
|
'@endo/eventual-send': 1.5.0
|
||||||
|
'@endo/harden': 1.1.0
|
||||||
|
'@endo/hex': 1.1.1
|
||||||
|
'@endo/init': 1.1.13
|
||||||
|
'@endo/marshal': 1.10.0
|
||||||
|
'@endo/nat': 5.2.0
|
||||||
|
'@endo/pass-style': 1.8.2
|
||||||
|
'@endo/promise-kit': 1.2.1
|
||||||
|
'@endo/stream': 1.3.1
|
||||||
|
'@endo/syrup-frame': 0.1.1
|
||||||
|
'@noble/curves': 1.9.7
|
||||||
|
'@noble/hashes': 1.8.0
|
||||||
|
ws: 8.22.0
|
||||||
|
transitivePeerDependencies:
|
||||||
|
- bufferutil
|
||||||
|
- utf-8-validate
|
||||||
|
|
||||||
|
'@endo/pass-style@1.8.2':
|
||||||
|
dependencies:
|
||||||
|
'@endo/common': 1.4.0
|
||||||
|
'@endo/env-options': 1.1.11
|
||||||
|
'@endo/errors': 1.3.1
|
||||||
|
'@endo/eventual-send': 1.5.0
|
||||||
|
'@endo/harden': 1.1.0
|
||||||
|
'@endo/promise-kit': 1.2.1
|
||||||
|
|
||||||
|
'@endo/promise-kit@1.2.1':
|
||||||
|
dependencies:
|
||||||
|
'@endo/harden': 1.1.0
|
||||||
|
|
||||||
|
'@endo/stream@1.3.1':
|
||||||
|
dependencies:
|
||||||
|
'@endo/eventual-send': 1.5.0
|
||||||
|
'@endo/harden': 1.1.0
|
||||||
|
'@endo/promise-kit': 1.2.1
|
||||||
|
ses: 2.3.0
|
||||||
|
|
||||||
|
'@endo/syrup-frame@0.1.1':
|
||||||
|
dependencies:
|
||||||
|
'@endo/harden': 1.1.0
|
||||||
|
'@endo/init': 1.1.13
|
||||||
|
'@endo/promise-kit': 1.2.1
|
||||||
|
'@endo/stream': 1.3.1
|
||||||
|
|
||||||
|
'@esbuild/aix-ppc64@0.28.2':
|
||||||
|
optional: true
|
||||||
|
|
||||||
|
'@esbuild/android-arm64@0.28.2':
|
||||||
|
optional: true
|
||||||
|
|
||||||
|
'@esbuild/android-arm@0.28.2':
|
||||||
|
optional: true
|
||||||
|
|
||||||
|
'@esbuild/android-x64@0.28.2':
|
||||||
|
optional: true
|
||||||
|
|
||||||
|
'@esbuild/darwin-arm64@0.28.2':
|
||||||
|
optional: true
|
||||||
|
|
||||||
|
'@esbuild/darwin-x64@0.28.2':
|
||||||
|
optional: true
|
||||||
|
|
||||||
|
'@esbuild/freebsd-arm64@0.28.2':
|
||||||
|
optional: true
|
||||||
|
|
||||||
|
'@esbuild/freebsd-x64@0.28.2':
|
||||||
|
optional: true
|
||||||
|
|
||||||
|
'@esbuild/linux-arm64@0.28.2':
|
||||||
|
optional: true
|
||||||
|
|
||||||
|
'@esbuild/linux-arm@0.28.2':
|
||||||
|
optional: true
|
||||||
|
|
||||||
|
'@esbuild/linux-ia32@0.28.2':
|
||||||
|
optional: true
|
||||||
|
|
||||||
|
'@esbuild/linux-loong64@0.28.2':
|
||||||
|
optional: true
|
||||||
|
|
||||||
|
'@esbuild/linux-mips64el@0.28.2':
|
||||||
|
optional: true
|
||||||
|
|
||||||
|
'@esbuild/linux-ppc64@0.28.2':
|
||||||
|
optional: true
|
||||||
|
|
||||||
|
'@esbuild/linux-riscv64@0.28.2':
|
||||||
|
optional: true
|
||||||
|
|
||||||
|
'@esbuild/linux-s390x@0.28.2':
|
||||||
|
optional: true
|
||||||
|
|
||||||
|
'@esbuild/linux-x64@0.28.2':
|
||||||
|
optional: true
|
||||||
|
|
||||||
|
'@esbuild/netbsd-arm64@0.28.2':
|
||||||
|
optional: true
|
||||||
|
|
||||||
|
'@esbuild/netbsd-x64@0.28.2':
|
||||||
|
optional: true
|
||||||
|
|
||||||
|
'@esbuild/openbsd-arm64@0.28.2':
|
||||||
|
optional: true
|
||||||
|
|
||||||
|
'@esbuild/openbsd-x64@0.28.2':
|
||||||
|
optional: true
|
||||||
|
|
||||||
|
'@esbuild/openharmony-arm64@0.28.2':
|
||||||
|
optional: true
|
||||||
|
|
||||||
|
'@esbuild/sunos-x64@0.28.2':
|
||||||
|
optional: true
|
||||||
|
|
||||||
|
'@esbuild/win32-arm64@0.28.2':
|
||||||
|
optional: true
|
||||||
|
|
||||||
|
'@esbuild/win32-ia32@0.28.2':
|
||||||
|
optional: true
|
||||||
|
|
||||||
|
'@esbuild/win32-x64@0.28.2':
|
||||||
|
optional: true
|
||||||
|
|
||||||
|
'@noble/curves@1.9.7':
|
||||||
|
dependencies:
|
||||||
|
'@noble/hashes': 1.8.0
|
||||||
|
|
||||||
|
'@noble/hashes@1.8.0': {}
|
||||||
|
|
||||||
|
capnweb@0.12.0: {}
|
||||||
|
|
||||||
|
esbuild@0.28.2:
|
||||||
|
optionalDependencies:
|
||||||
|
'@esbuild/aix-ppc64': 0.28.2
|
||||||
|
'@esbuild/android-arm': 0.28.2
|
||||||
|
'@esbuild/android-arm64': 0.28.2
|
||||||
|
'@esbuild/android-x64': 0.28.2
|
||||||
|
'@esbuild/darwin-arm64': 0.28.2
|
||||||
|
'@esbuild/darwin-x64': 0.28.2
|
||||||
|
'@esbuild/freebsd-arm64': 0.28.2
|
||||||
|
'@esbuild/freebsd-x64': 0.28.2
|
||||||
|
'@esbuild/linux-arm': 0.28.2
|
||||||
|
'@esbuild/linux-arm64': 0.28.2
|
||||||
|
'@esbuild/linux-ia32': 0.28.2
|
||||||
|
'@esbuild/linux-loong64': 0.28.2
|
||||||
|
'@esbuild/linux-mips64el': 0.28.2
|
||||||
|
'@esbuild/linux-ppc64': 0.28.2
|
||||||
|
'@esbuild/linux-riscv64': 0.28.2
|
||||||
|
'@esbuild/linux-s390x': 0.28.2
|
||||||
|
'@esbuild/linux-x64': 0.28.2
|
||||||
|
'@esbuild/netbsd-arm64': 0.28.2
|
||||||
|
'@esbuild/netbsd-x64': 0.28.2
|
||||||
|
'@esbuild/openbsd-arm64': 0.28.2
|
||||||
|
'@esbuild/openbsd-x64': 0.28.2
|
||||||
|
'@esbuild/openharmony-arm64': 0.28.2
|
||||||
|
'@esbuild/sunos-x64': 0.28.2
|
||||||
|
'@esbuild/win32-arm64': 0.28.2
|
||||||
|
'@esbuild/win32-ia32': 0.28.2
|
||||||
|
'@esbuild/win32-x64': 0.28.2
|
||||||
|
|
||||||
|
ses@2.3.0:
|
||||||
|
dependencies:
|
||||||
|
'@endo/cache-map': 1.1.0
|
||||||
|
'@endo/env-options': 1.1.11
|
||||||
|
'@endo/immutable-arraybuffer': 2.0.0
|
||||||
|
|
||||||
|
workerd@1.20261006.1:
|
||||||
|
optionalDependencies:
|
||||||
|
'@cloudflare/workerd-darwin-64': 1.20261006.1
|
||||||
|
'@cloudflare/workerd-darwin-arm64': 1.20261006.1
|
||||||
|
'@cloudflare/workerd-linux-64': 1.20261006.1
|
||||||
|
'@cloudflare/workerd-linux-arm64': 1.20261006.1
|
||||||
|
'@cloudflare/workerd-windows-64': 1.20261006.1
|
||||||
|
|
||||||
|
ws@8.22.0: {}
|
||||||
@@ -0,0 +1,20 @@
|
|||||||
|
/**
|
||||||
|
* @file Create the hello-world vat on a running toybox and mount it.
|
||||||
|
*
|
||||||
|
* Usage: node --import ./test/lockdown.js scripts/hello.js [name]
|
||||||
|
* then visit http://127.0.0.1:8787/?vat=<name> (default name: hello).
|
||||||
|
*/
|
||||||
|
import { readFile } from "node:fs/promises";
|
||||||
|
import { E } from "@endo/far";
|
||||||
|
import { connectClient } from "../client.js";
|
||||||
|
|
||||||
|
const name = process.argv[2] ?? "hello";
|
||||||
|
const client = await connectClient("admin.sock", `hello-script-${process.pid}`);
|
||||||
|
try {
|
||||||
|
const ingress = await E(client.vatAdmin).makeIngress({ vat: name });
|
||||||
|
const { id, root } = await E(client.vatAdmin).createVat(await readFile("vats/hello.js", "utf8"), { ingress });
|
||||||
|
await root; // its handoff, settled before the client goes
|
||||||
|
console.log(`vat ${id}\nhttp://127.0.0.1:8787/?vat=${name}`);
|
||||||
|
} finally {
|
||||||
|
client.close();
|
||||||
|
}
|
||||||
@@ -0,0 +1,575 @@
|
|||||||
|
/** @file toybox supervisor: vats with code installed at runtime, each app a facet; ingress by mount, egress by grant. */
|
||||||
|
import "@endo/init";
|
||||||
|
import { DurableObject, WorkerEntrypoint } from "cloudflare:workers";
|
||||||
|
import { RpcTarget, newWebSocketRpcSession, newWorkersRpcResponse } from "capnweb";
|
||||||
|
import { E, Far, passStyleOf } from "@endo/far";
|
||||||
|
import { makeClient, swissnumFromBytes } from "@endo/ocapn";
|
||||||
|
import { mergeLabels, urlLabels } from "./labels.js";
|
||||||
|
import { LOOKUP, MOUNT, MOUNT_LABELS, MOUNT_SCHEMA, OVERLAP, UNMOUNT, UNMOUNT_LABELS } from "./mounts-sql.js";
|
||||||
|
import { locationOf, makeRpcNetLayer } from "./netlayer.js";
|
||||||
|
import VAT_RUNTIME from "vat-runtime.txt";
|
||||||
|
|
||||||
|
/**
|
||||||
|
* A stored grant: labels as JSON, or (before labels) a bare `host[:port]`.
|
||||||
|
*
|
||||||
|
* @param {string} stored
|
||||||
|
*/
|
||||||
|
const grantLabels = (stored) => {
|
||||||
|
if (stored.startsWith("{")) return JSON.parse(stored);
|
||||||
|
const { host, port } = urlLabels(`http://${stored}`);
|
||||||
|
return stored.includes(":") ? { host, port } : { host };
|
||||||
|
};
|
||||||
|
|
||||||
|
const hex = (buf) => [...new Uint8Array(buf)].map((b) => b.toString(16).padStart(2, "0")).join("");
|
||||||
|
|
||||||
|
/**
|
||||||
|
* One vat, by DO id: holds its installed code and egress grant; the app
|
||||||
|
* facet holds the app's state. Its mount, if any, is in the {@link Directory}.
|
||||||
|
*
|
||||||
|
* Code is stored by hash so the facet can be rebuilt after hibernation or a
|
||||||
|
* restart. The app's global `fetch` throws; a granted host arrives as
|
||||||
|
* `env.egress`, an {@link Egress} the app can pass along like any object.
|
||||||
|
*
|
||||||
|
* Each `start` begins a new incarnation, which names the loaded Worker: the
|
||||||
|
* same code with a different grant must not share a Worker.
|
||||||
|
*/
|
||||||
|
export class Vat extends DurableObject {
|
||||||
|
constructor(ctx, env) {
|
||||||
|
super(ctx, env);
|
||||||
|
ctx.storage.sql.exec("CREATE TABLE IF NOT EXISTS bundle (hash TEXT PRIMARY KEY, src TEXT)");
|
||||||
|
ctx.storage.sql.exec("CREATE TABLE IF NOT EXISTS current (k INTEGER PRIMARY KEY CHECK (k = 0), hash TEXT)");
|
||||||
|
ctx.storage.sql.exec(
|
||||||
|
"CREATE TABLE IF NOT EXISTS incarnation (k INTEGER PRIMARY KEY CHECK (k = 0), n INTEGER, egress TEXT)",
|
||||||
|
);
|
||||||
|
// absent: an app vat (exports class App), from before OCapN vats
|
||||||
|
ctx.storage.sql.exec("CREATE TABLE IF NOT EXISTS kind (k INTEGER PRIMARY KEY CHECK (k = 0), kind TEXT)");
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Install `src` and run the app on it; the app's storage carries over.
|
||||||
|
*
|
||||||
|
* @param {string} src an ES module exporting `class App extends DurableObject`
|
||||||
|
* @param {string | null} egress the app's grant: egress labels as JSON (see labels.js), or none
|
||||||
|
* @param {"app" | "ocapn"} kind `app`: `src` exports `class App`; `ocapn`:
|
||||||
|
* `src` exports `buildRootObject`, run by vat-runtime.js
|
||||||
|
* @returns {Promise<string>} the code's SHA-256, hex
|
||||||
|
*/
|
||||||
|
async start(src, egress, kind = "app") {
|
||||||
|
const hash = hex(await crypto.subtle.digest("SHA-256", encodeUtf8(src)));
|
||||||
|
const { sql } = this.ctx.storage;
|
||||||
|
sql.exec("INSERT OR IGNORE INTO bundle VALUES (?, ?)", hash, src);
|
||||||
|
sql.exec("INSERT OR REPLACE INTO current VALUES (0, ?)", hash);
|
||||||
|
const [prev] = sql.exec("SELECT n FROM incarnation").toArray();
|
||||||
|
sql.exec("INSERT OR REPLACE INTO incarnation VALUES (0, ?, ?)", (prev?.n ?? 0) + 1, egress);
|
||||||
|
sql.exec("INSERT OR REPLACE INTO kind VALUES (0, ?)", kind);
|
||||||
|
this.ctx.facets.abort("app", new Error("upgraded"));
|
||||||
|
return hash;
|
||||||
|
}
|
||||||
|
|
||||||
|
#app({ src, n, egress, kind }) {
|
||||||
|
return this.ctx.facets.get("app", () => {
|
||||||
|
const env = egress ? { egress: this.ctx.exports.Egress({ props: { labels: grantLabels(egress) } }) } : {};
|
||||||
|
const code =
|
||||||
|
kind === "ocapn"
|
||||||
|
? {
|
||||||
|
mainModule: "runtime.js",
|
||||||
|
modules: { "runtime.js": VAT_RUNTIME, "vat.js": src },
|
||||||
|
env: { ...env, vatId: this.ctx.id.toString(), router: this.ctx.exports.Router({ props: {} }) },
|
||||||
|
}
|
||||||
|
: { mainModule: "app.js", modules: { "app.js": src }, env };
|
||||||
|
const worker = this.env.LOADER.get(`${this.ctx.id}:${n}`, () => ({
|
||||||
|
compatibilityDate: "2026-10-01",
|
||||||
|
...code,
|
||||||
|
globalOutbound: null,
|
||||||
|
}));
|
||||||
|
return { class: worker.getDurableObjectClass("App") };
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
#current() {
|
||||||
|
const [row] = this.ctx.storage.sql
|
||||||
|
.exec(
|
||||||
|
`SELECT b.hash, b.src, coalesce(i.n, 0) AS n, i.egress, coalesce(k.kind, 'app') AS kind
|
||||||
|
FROM current c JOIN bundle b USING (hash)
|
||||||
|
LEFT JOIN incarnation i ON i.k = 0 LEFT JOIN kind k ON k.k = 0`,
|
||||||
|
)
|
||||||
|
.toArray();
|
||||||
|
return row;
|
||||||
|
}
|
||||||
|
|
||||||
|
fetch(request) {
|
||||||
|
const row = this.#current();
|
||||||
|
if (!row) return new Response("no code installed\n", { status: 404 });
|
||||||
|
return this.#app(row).fetch(request);
|
||||||
|
}
|
||||||
|
|
||||||
|
/** An OCapN vat's gateway, for the router. */
|
||||||
|
gateway() {
|
||||||
|
const row = this.#current();
|
||||||
|
if (row?.kind !== "ocapn") throw new Error("not an OCapN vat");
|
||||||
|
return this.#app(row).gateway();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* A vat's egress grant: a switchboard handle constrained to the labels in
|
||||||
|
* `ctx.props`, which no holder can change. The switchboard enforces them;
|
||||||
|
* a refusal comes back as a 403.
|
||||||
|
*/
|
||||||
|
export class Egress extends WorkerEntrypoint {
|
||||||
|
/** A narrower grant: these labels too (conflicting ones throw). */
|
||||||
|
constrain(more) {
|
||||||
|
return this.ctx.exports.Egress({ props: { labels: mergeLabels(this.ctx.props.labels, more) } });
|
||||||
|
}
|
||||||
|
|
||||||
|
async #handle() {
|
||||||
|
const resp = await this.env.SWITCHBOARD.fetch("http://switchboard/", { headers: { Upgrade: "websocket" } });
|
||||||
|
const ws = resp.webSocket;
|
||||||
|
if (!ws) throw new Error(`switchboard refused a session: ${resp.status}`);
|
||||||
|
ws.accept();
|
||||||
|
return newWebSocketRpcSession(ws).constrain(this.ctx.props.labels);
|
||||||
|
}
|
||||||
|
|
||||||
|
async fetch(request) {
|
||||||
|
try {
|
||||||
|
const handle = await this.#handle();
|
||||||
|
if (request.headers.get("Upgrade")?.toLowerCase() !== "websocket") return await handle.fetch(request);
|
||||||
|
const protocols = (request.headers.get("Sec-WebSocket-Protocol") ?? "").split(",").map((p) => p.trim()).filter(Boolean);
|
||||||
|
const { readable, writable, protocol } = await handle.webSocket(request.url, protocols);
|
||||||
|
return this.#bridge(readable, writable, protocol);
|
||||||
|
} catch (e) {
|
||||||
|
return new Response(`egress refused: ${e.message}\n`, { status: 403 });
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/** A WebSocket for the vat whose messages travel the switchboard's streams. */
|
||||||
|
#bridge(readable, writable, protocol) {
|
||||||
|
const [client, server] = Object.values(new WebSocketPair());
|
||||||
|
server.accept();
|
||||||
|
const writer = writable.getWriter();
|
||||||
|
server.addEventListener("message", (e) => writer.write(e.data).catch(() => server.close(1011)));
|
||||||
|
server.addEventListener("close", () => writer.close().catch(() => {}));
|
||||||
|
this.ctx.waitUntil(
|
||||||
|
(async () => {
|
||||||
|
for await (const m of readable) server.send(m);
|
||||||
|
server.close();
|
||||||
|
})().catch(() => server.close(1011)),
|
||||||
|
);
|
||||||
|
const headers = protocol ? { "Sec-WebSocket-Protocol": protocol } : {};
|
||||||
|
return new Response(null, { status: 101, webSocket: client, headers });
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/** The vatAdmin's designator, and the swissnum of its bootstrap object. */
|
||||||
|
const VAT_ADMIN = "vatAdmin";
|
||||||
|
/** The swissnum of an OCapN vat's bootstrap (see vat-runtime.js). */
|
||||||
|
const VAT_SWISSNUM = "vat";
|
||||||
|
/** UTF-8 bytes of a string: the method alone, so no encoder object is in module scope. */
|
||||||
|
const encodeUtf8 = harden(TextEncoder.prototype.encode.bind(new TextEncoder()));
|
||||||
|
|
||||||
|
/**
|
||||||
|
* The vatAdmin: an OCapN peer, and the router through which every OCapN
|
||||||
|
* peer here reaches every other. One instance, by name.
|
||||||
|
*
|
||||||
|
* Peers find each other by designator: the vatAdmin's is `vatAdmin`, a
|
||||||
|
* vat's is its id, and a client registers its own when it connects to
|
||||||
|
* the admin socket. Nothing here can dial out, so a handoff's receiver
|
||||||
|
* reaches the exporter through this router.
|
||||||
|
*/
|
||||||
|
export class VatAdmin extends DurableObject {
|
||||||
|
#clients = new Map();
|
||||||
|
/** mount key → the handler serving it: live, so lost when this object is evicted */
|
||||||
|
#handlers = new Map();
|
||||||
|
/** mount key → resolvers waiting for a handler */
|
||||||
|
#waiting = new Map();
|
||||||
|
/** mount key ↔ its ingress object, made again as needed */
|
||||||
|
#ingresses = new Map();
|
||||||
|
#ingressKeys = new WeakMap();
|
||||||
|
#reviving = new Map();
|
||||||
|
#ready;
|
||||||
|
|
||||||
|
constructor(ctx, env) {
|
||||||
|
super(ctx, env);
|
||||||
|
const { sql } = ctx.storage;
|
||||||
|
// a vat's params, encoded (see #encode); a vat absent here can't be restarted
|
||||||
|
sql.exec("CREATE TABLE IF NOT EXISTS vat_params (vat TEXT PRIMARY KEY, params TEXT)");
|
||||||
|
// which vat was given which ingress, so a request can restart it
|
||||||
|
sql.exec("CREATE TABLE IF NOT EXISTS ingress_vat (key TEXT PRIMARY KEY, vat TEXT)");
|
||||||
|
this.#ready = this.#boot();
|
||||||
|
}
|
||||||
|
|
||||||
|
/** @param {string} key */
|
||||||
|
#ingressFor(key) {
|
||||||
|
let ingress = this.#ingresses.get(key);
|
||||||
|
if (!ingress) {
|
||||||
|
ingress = Far("ingress", {
|
||||||
|
/** @param {{ handle: (request: object) => object }} handler */
|
||||||
|
serve: (handler) => {
|
||||||
|
this.#handlers.set(key, handler);
|
||||||
|
for (const resolve of this.#waiting.get(key) ?? []) resolve();
|
||||||
|
this.#waiting.delete(key);
|
||||||
|
},
|
||||||
|
});
|
||||||
|
this.#ingresses.set(key, ingress);
|
||||||
|
this.#ingressKeys.set(ingress, key);
|
||||||
|
}
|
||||||
|
return ingress;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* `params` as JSON, an ingress by its key; `null` if anything in it can't
|
||||||
|
* be made again here (a remote object of another peer, say).
|
||||||
|
*/
|
||||||
|
#encode(params) {
|
||||||
|
const keys = [];
|
||||||
|
const walk = (v) => {
|
||||||
|
const style = passStyleOf(v);
|
||||||
|
if (style === "remotable") {
|
||||||
|
const key = this.#ingressKeys.get(v);
|
||||||
|
if (!key) throw new Error("not restartable");
|
||||||
|
keys.push(key);
|
||||||
|
return { "@ingress": key };
|
||||||
|
}
|
||||||
|
if (style === "copyArray") return v.map(walk);
|
||||||
|
if (style === "copyRecord") return Object.fromEntries(Object.entries(v).map(([k, x]) => [k, walk(x)]));
|
||||||
|
if (["string", "number", "boolean", "null"].includes(style)) return v;
|
||||||
|
throw new Error("not restartable");
|
||||||
|
};
|
||||||
|
try {
|
||||||
|
return { json: JSON.stringify(walk(params)), keys };
|
||||||
|
} catch {
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
#decode(json) {
|
||||||
|
const walk = (v) => {
|
||||||
|
if (Array.isArray(v)) return v.map(walk);
|
||||||
|
if (v && typeof v === "object") {
|
||||||
|
if ("@ingress" in v) return this.#ingressFor(v["@ingress"]);
|
||||||
|
return Object.fromEntries(Object.entries(v).map(([k, x]) => [k, walk(x)]));
|
||||||
|
}
|
||||||
|
return v;
|
||||||
|
};
|
||||||
|
return harden(walk(JSON.parse(json)));
|
||||||
|
}
|
||||||
|
|
||||||
|
/** (Re)start a vat's code with `params`: it rebuilds its root. */
|
||||||
|
async #startVat(vat, params) {
|
||||||
|
const vatBootstrap = await this.#enliven(vat, VAT_SWISSNUM);
|
||||||
|
return E(vatBootstrap).start(params);
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* The handler for a mount whose handler went with an evicted instance:
|
||||||
|
* restart the vat given its ingress, and wait for it to serve again.
|
||||||
|
*
|
||||||
|
* @param {string} key
|
||||||
|
*/
|
||||||
|
#revive(key) {
|
||||||
|
let reviving = this.#reviving.get(key);
|
||||||
|
if (!reviving) {
|
||||||
|
reviving = (async () => {
|
||||||
|
const [row] = this.ctx.storage.sql
|
||||||
|
.exec("SELECT i.vat, p.params FROM ingress_vat i JOIN vat_params p USING (vat) WHERE i.key = ?", key)
|
||||||
|
.toArray();
|
||||||
|
if (!row) return null;
|
||||||
|
const served = new Promise((resolve) => this.#waiting.set(key, [...(this.#waiting.get(key) ?? []), resolve]));
|
||||||
|
await this.#startVat(row.vat, this.#decode(row.params));
|
||||||
|
await Promise.race([served, new Promise((resolve) => setTimeout(resolve, 5000))]);
|
||||||
|
return this.#handlers.get(key) ?? null;
|
||||||
|
})().finally(() => this.#reviving.delete(key));
|
||||||
|
this.#reviving.set(key, reviving);
|
||||||
|
}
|
||||||
|
return reviving;
|
||||||
|
}
|
||||||
|
|
||||||
|
async #boot() {
|
||||||
|
const vatAdmin = Far("vatAdmin", {
|
||||||
|
/**
|
||||||
|
* Create an OCapN vat running `src`; `params` (any passable,
|
||||||
|
* remote objects included) reach its `buildRootObject`.
|
||||||
|
*
|
||||||
|
* If `params` holds only data and ingresses, they are kept, and the
|
||||||
|
* vat is restarted with them when a request needs it after an
|
||||||
|
* eviction or a restart.
|
||||||
|
*
|
||||||
|
* @param {string} src an ES module exporting `buildRootObject(vatPowers, params)`
|
||||||
|
*/
|
||||||
|
createVat: async (src, params = {}) => {
|
||||||
|
const id = this.env.VATS.newUniqueId();
|
||||||
|
const vat = id.toString();
|
||||||
|
await this.env.VATS.get(id).start(src, null, "ocapn");
|
||||||
|
const encoded = this.#encode(params);
|
||||||
|
if (encoded) {
|
||||||
|
const { sql } = this.ctx.storage;
|
||||||
|
sql.exec("INSERT INTO vat_params VALUES (?, ?)", vat, encoded.json);
|
||||||
|
for (const key of encoded.keys) sql.exec("INSERT OR REPLACE INTO ingress_vat VALUES (?, ?)", key, vat);
|
||||||
|
}
|
||||||
|
const root = await this.#startVat(vat, params);
|
||||||
|
return harden({ id: vat, root });
|
||||||
|
},
|
||||||
|
/**
|
||||||
|
* Reserve a mount at `labels` and return the ingress for it: whoever
|
||||||
|
* holds the ingress (a vat it is passed to, say) serves requests
|
||||||
|
* that carry those labels.
|
||||||
|
*
|
||||||
|
* @param {Record<string, string>} labels
|
||||||
|
*/
|
||||||
|
makeIngress: async (labels) => {
|
||||||
|
const key = `ingress:${crypto.randomUUID()}`;
|
||||||
|
await this.env.DIRECTORY.getByName("directory").mount(key, harden({ ...labels }));
|
||||||
|
return this.#ingressFor(key);
|
||||||
|
},
|
||||||
|
});
|
||||||
|
// workerd runs FinalizationRegistry callbacks outside any request, where
|
||||||
|
// sending the resulting GC message is forbidden I/O. So imports are
|
||||||
|
// never collected here; they go when their session does.
|
||||||
|
this.ocapn = makeClient({
|
||||||
|
debugLabel: VAT_ADMIN,
|
||||||
|
swissnumTable: new Map([[VAT_ADMIN, vatAdmin]]),
|
||||||
|
enableImportCollection: false,
|
||||||
|
});
|
||||||
|
const netlayer = await this.ocapn.registerNetlayer(
|
||||||
|
makeRpcNetLayer({ designator: VAT_ADMIN, openGateway: (designator) => this.gateway(designator) }),
|
||||||
|
);
|
||||||
|
this.own = netlayer.gateway;
|
||||||
|
}
|
||||||
|
|
||||||
|
#enliven(designator, swissnum) {
|
||||||
|
return this.ocapn.enlivenSturdyRef(
|
||||||
|
this.ocapn.makeSturdyRef(locationOf(designator), swissnumFromBytes(encodeUtf8(swissnum))),
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* The gateway of the peer with this designator.
|
||||||
|
*
|
||||||
|
* @param {string} designator
|
||||||
|
*/
|
||||||
|
async gateway(designator) {
|
||||||
|
await this.#ready;
|
||||||
|
if (designator === VAT_ADMIN) return this.own;
|
||||||
|
const client = this.#clients.get(designator);
|
||||||
|
if (client) return client;
|
||||||
|
let id;
|
||||||
|
try {
|
||||||
|
id = this.env.VATS.idFromString(designator);
|
||||||
|
} catch {
|
||||||
|
throw new Error("no such peer");
|
||||||
|
}
|
||||||
|
return this.env.VATS.get(id).gateway();
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* One HTTP request, as passable data, for the handler serving `key`;
|
||||||
|
* `null` when nothing serves it.
|
||||||
|
*
|
||||||
|
* @param {string} key
|
||||||
|
* @param {object} request
|
||||||
|
*/
|
||||||
|
async serveHttp(key, request) {
|
||||||
|
await this.#ready;
|
||||||
|
const handler = this.#handlers.get(key) ?? (await this.#revive(key));
|
||||||
|
return handler ? E(handler).handle(harden(request)) : null;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** A WebSocket here starts a Cap'n Web session whose main object is an {@link AdminSession}. */
|
||||||
|
fetch(request) {
|
||||||
|
return newWorkersRpcResponse(request, new AdminSession(this, this.#clients));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* What an OCapN client on the admin socket gets: register its own
|
||||||
|
* gateway under a designator, and reach any peer's.
|
||||||
|
*/
|
||||||
|
class AdminSession extends RpcTarget {
|
||||||
|
#vatAdmin;
|
||||||
|
#clients;
|
||||||
|
|
||||||
|
constructor(vatAdmin, clients) {
|
||||||
|
super();
|
||||||
|
this.#vatAdmin = vatAdmin;
|
||||||
|
this.#clients = clients;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** @param {string} designator @param {any} gateway */
|
||||||
|
register(designator, gateway) {
|
||||||
|
if (designator === VAT_ADMIN || /^[0-9a-f]{64}$/.test(designator) || this.#clients.has(designator)) {
|
||||||
|
throw new Error("designator taken");
|
||||||
|
}
|
||||||
|
this.#clients.set(designator, gateway.dup?.() ?? gateway);
|
||||||
|
}
|
||||||
|
|
||||||
|
/** @param {string} designator */
|
||||||
|
gateway(designator) {
|
||||||
|
return this.#vatAdmin.gateway(designator);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/** A vat's way to the router: `env.router` in an OCapN vat. */
|
||||||
|
export class Router extends WorkerEntrypoint {
|
||||||
|
gateway(designator) {
|
||||||
|
return this.env.VAT_ADMIN.getByName(VAT_ADMIN).gateway(designator);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Mounts: which vat serves which label set. One instance, by name.
|
||||||
|
*
|
||||||
|
* Registration follows Flower: a mount must conflict with every other, so
|
||||||
|
* no request can carry two mounts' labels. Lookup is stricter than
|
||||||
|
* Flower's: the request must carry all of a mount's labels.
|
||||||
|
*/
|
||||||
|
export class Directory extends DurableObject {
|
||||||
|
constructor(ctx, env) {
|
||||||
|
super(ctx, env);
|
||||||
|
for (const statement of MOUNT_SCHEMA) ctx.storage.sql.exec(statement);
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Mount `vat` at `labels`, replacing its mount; `null` unmounts it.
|
||||||
|
*
|
||||||
|
* @param {string} vat a Vat's DO id, hex
|
||||||
|
* @param {Record<string, string> | null} labels
|
||||||
|
*/
|
||||||
|
mount(vat, labels) {
|
||||||
|
const { sql } = this.ctx.storage;
|
||||||
|
this.ctx.storage.transactionSync(() => {
|
||||||
|
sql.exec(UNMOUNT_LABELS, vat);
|
||||||
|
if (!labels) return void sql.exec(UNMOUNT, vat);
|
||||||
|
const json = JSON.stringify(labels);
|
||||||
|
if (sql.exec(OVERLAP, json, vat).toArray().length) {
|
||||||
|
throw new Error("mount overlaps another: add a label whose value tells them apart");
|
||||||
|
}
|
||||||
|
sql.exec(MOUNT, vat, json);
|
||||||
|
sql.exec(MOUNT_LABELS, vat, json);
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* The mount whose labels `request` carries, if any. Registration keeps
|
||||||
|
* mounts pairwise conflicting, so at most one can match.
|
||||||
|
*
|
||||||
|
* @param {Record<string, string>} request
|
||||||
|
*/
|
||||||
|
lookup(request) {
|
||||||
|
const [row] = this.ctx.storage.sql.exec(LOOKUP, JSON.stringify(request)).toArray();
|
||||||
|
return row ? { vat: row.vat, labels: JSON.parse(row.labels) } : null;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* A request's labels: its query parameters.
|
||||||
|
*
|
||||||
|
* Routing carries no secrets: a URL ends up in logs and `Referer`s. A
|
||||||
|
* secret travels in-band instead, as on a webkey page: the page keeps it
|
||||||
|
* in its fragment, reaches the vat by public labels, and sends it in the
|
||||||
|
* body of a call, which the vat checks like a swissnum.
|
||||||
|
*
|
||||||
|
* @param {Request} request
|
||||||
|
*/
|
||||||
|
const requestLabels = (request) => {
|
||||||
|
const labels = {};
|
||||||
|
for (const [k, v] of new URL(request.url).searchParams) {
|
||||||
|
if (k in labels) throw new Error(`label ${k} given twice`);
|
||||||
|
labels[k] = v;
|
||||||
|
}
|
||||||
|
return Object.freeze(labels);
|
||||||
|
};
|
||||||
|
|
||||||
|
/**
|
||||||
|
* The http socket: the ingress. A request reaches the vat mounted at labels
|
||||||
|
* it carries, which sees the mount's and the request's labels together in
|
||||||
|
* a `Toybox-Connection-Labels` header (JSON), as a Flower server gets its
|
||||||
|
* connection labels.
|
||||||
|
*/
|
||||||
|
export class Public extends WorkerEntrypoint {
|
||||||
|
async fetch(request) {
|
||||||
|
let labels;
|
||||||
|
try {
|
||||||
|
labels = requestLabels(request);
|
||||||
|
} catch (e) {
|
||||||
|
return new Response(`bad labels: ${e.message}\n`, { status: 400 });
|
||||||
|
}
|
||||||
|
const mount = await this.env.DIRECTORY.getByName("directory").lookup(labels);
|
||||||
|
if (!mount) return new Response("nothing mounted here\n", { status: 404 });
|
||||||
|
if (mount.vat.startsWith("ingress:")) return this.#serveOcapn(mount, labels, request);
|
||||||
|
const forwarded = new Request(request);
|
||||||
|
forwarded.headers.set("Toybox-Connection-Labels", JSON.stringify(mergeLabels(mount.labels, labels)));
|
||||||
|
const resp = await this.env.VATS.get(this.env.VATS.idFromString(mount.vat)).fetch(forwarded);
|
||||||
|
if (resp.webSocket) return resp;
|
||||||
|
const sealed = new Response(resp.body, resp);
|
||||||
|
sealed.headers.set("Referrer-Policy", "no-referrer");
|
||||||
|
return sealed;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** A mount made by `makeIngress`: the request goes, as data, to whoever serves it. */
|
||||||
|
async #serveOcapn(mount, labels, request) {
|
||||||
|
const { pathname } = new URL(request.url);
|
||||||
|
const body = ["GET", "HEAD"].includes(request.method) ? "" : await request.text();
|
||||||
|
const req = { method: request.method, path: pathname, labels: mergeLabels(mount.labels, labels), body };
|
||||||
|
const res = await this.env.VAT_ADMIN.getByName(VAT_ADMIN).serveHttp(mount.vat, req);
|
||||||
|
if (!res) return new Response("mounted, but nothing serves it now\n", { status: 503 });
|
||||||
|
const resp = new Response(res.body ?? "", { status: res.status ?? 200, headers: res.headers ?? {} });
|
||||||
|
resp.headers.set("Referrer-Policy", "no-referrer");
|
||||||
|
return resp;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Grants in a query parameter, as JSON labels; absent means none.
|
||||||
|
*
|
||||||
|
* @param {URL} url
|
||||||
|
* @param {string} name
|
||||||
|
*/
|
||||||
|
const grantParam = (url, name) => {
|
||||||
|
const v = url.searchParams.get(name);
|
||||||
|
return v ? JSON.parse(v) : null;
|
||||||
|
};
|
||||||
|
|
||||||
|
/**
|
||||||
|
* The admin socket: a WebSocket upgrade reaches the {@link VatAdmin} (OCapN
|
||||||
|
* over Cap'n Web); otherwise, app vats over HTTP:
|
||||||
|
*
|
||||||
|
* - `POST /` with an ES module body creates a vat and starts it; the reply
|
||||||
|
* is `{"vat": <id>, "hash": <code hash>}`, and the id addresses it here.
|
||||||
|
* - `PUT /<id>` with a body upgrades it; its storage carries over.
|
||||||
|
* - `PUT /by-name/<name>` does the same for a vat from before ids, when
|
||||||
|
* vats were named.
|
||||||
|
*
|
||||||
|
* Every start states the vat's grants, absent meaning none:
|
||||||
|
* `?egress=<labels>` (see {@link Egress}) and `?mount=<labels>` (ingress:
|
||||||
|
* see {@link Public}).
|
||||||
|
*/
|
||||||
|
export class Admin extends WorkerEntrypoint {
|
||||||
|
async fetch(request) {
|
||||||
|
if (request.headers.get("Upgrade")?.toLowerCase() === "websocket") {
|
||||||
|
return this.env.VAT_ADMIN.getByName(VAT_ADMIN).fetch(request);
|
||||||
|
}
|
||||||
|
const url = new URL(request.url);
|
||||||
|
const [, first, second] = url.pathname.split("/");
|
||||||
|
let id;
|
||||||
|
if (request.method === "POST" && !first) id = this.env.VATS.newUniqueId();
|
||||||
|
else if (request.method === "PUT" && first === "by-name" && second) id = this.env.VATS.idFromName(second);
|
||||||
|
else if (request.method === "PUT" && first && !second) {
|
||||||
|
try {
|
||||||
|
id = this.env.VATS.idFromString(first);
|
||||||
|
} catch {
|
||||||
|
return new Response("no such vat\n", { status: 404 });
|
||||||
|
}
|
||||||
|
} else return new Response("POST / or PUT /<id> or PUT /by-name/<name>, with code\n", { status: 405 });
|
||||||
|
const vat = id.toString();
|
||||||
|
try {
|
||||||
|
await this.env.DIRECTORY.getByName("directory").mount(vat, grantParam(url, "mount"));
|
||||||
|
} catch (e) {
|
||||||
|
return new Response(`${e.message}\n`, { status: 409 });
|
||||||
|
}
|
||||||
|
const egress = url.searchParams.get("egress");
|
||||||
|
const hash = await this.env.VATS.get(id).start(await request.text(), egress);
|
||||||
|
return Response.json({ vat, hash });
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
export default Public;
|
||||||
@@ -0,0 +1,34 @@
|
|||||||
|
# switchboard: toybox's only way out, after Flower's switchboard.
|
||||||
|
#
|
||||||
|
# Each socket is a handle with fixed labels: a WebSocket upgrade starts a
|
||||||
|
# Cap'n Web session whose main object is that handle (see switchboard.js).
|
||||||
|
# Whoever starts workerd hands the sockets in (-S/-s), so whoever holds a
|
||||||
|
# socket's fd holds that handle and no other.
|
||||||
|
using Workerd = import "/workerd/workerd.capnp";
|
||||||
|
|
||||||
|
const config :Workerd.Config = (
|
||||||
|
services = [
|
||||||
|
(name = "main", worker = .switchboard),
|
||||||
|
(name = "nowhere", network = (allow = [])),
|
||||||
|
(name = "public-net", network = (allow = ["public"], tlsOptions = (trustBrowserCas = true))),
|
||||||
|
(name = "local-net", network = (allow = ["local"], tlsOptions = (trustBrowserCas = true))),
|
||||||
|
],
|
||||||
|
sockets = [
|
||||||
|
(name = "public", http = (), service = (name = "main", props = (json = "{\"network\":\"public\"}"))),
|
||||||
|
(name = "local", http = (), service = (name = "main", props = (json = "{\"network\":\"local\"}"))),
|
||||||
|
],
|
||||||
|
);
|
||||||
|
|
||||||
|
const switchboard :Workerd.Worker = (
|
||||||
|
compatibilityDate = "2026-10-01",
|
||||||
|
modules = [
|
||||||
|
(name = "switchboard.js", esModule = embed "switchboard.js"),
|
||||||
|
(name = "labels.js", esModule = embed "labels.js"),
|
||||||
|
(name = "capnweb", esModule = embed "node_modules/capnweb/dist/index-workers.js"),
|
||||||
|
],
|
||||||
|
bindings = [
|
||||||
|
(name = "PUBLIC", service = "public-net"),
|
||||||
|
(name = "LOCAL", service = "local-net"),
|
||||||
|
],
|
||||||
|
globalOutbound = "nowhere",
|
||||||
|
);
|
||||||
@@ -0,0 +1,81 @@
|
|||||||
|
/** @file switchboard: Flower-style egress handles, served over Cap'n Web. */
|
||||||
|
import { WorkerEntrypoint } from "cloudflare:workers";
|
||||||
|
import { RpcTarget, newWorkersRpcResponse } from "capnweb";
|
||||||
|
import { checkUrl, mergeLabels } from "./labels.js";
|
||||||
|
|
||||||
|
/**
|
||||||
|
* One egress handle: reaches only URLs within its labels, through the
|
||||||
|
* network its `network` label names. Like a Flower switchboard handle, it
|
||||||
|
* can only be narrowed; unlike one, it is a Cap'n Web stub, not an fd.
|
||||||
|
*/
|
||||||
|
export class Handle extends RpcTarget {
|
||||||
|
#labels;
|
||||||
|
#net;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @param {Record<string, string>} labels
|
||||||
|
* @param {Fetcher} net
|
||||||
|
*/
|
||||||
|
constructor(labels, net) {
|
||||||
|
super();
|
||||||
|
this.#labels = labels;
|
||||||
|
this.#net = net;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** @param {Record<string, string>} more */
|
||||||
|
constrain(more) {
|
||||||
|
return new Handle(mergeLabels(this.#labels, more), this.#net);
|
||||||
|
}
|
||||||
|
|
||||||
|
labels() {
|
||||||
|
return this.#labels;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** @param {Request} request */
|
||||||
|
fetch(request) {
|
||||||
|
checkUrl(this.#labels, request.url);
|
||||||
|
return this.#net.fetch(request);
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Open a WebSocket to `url`; its messages cross the session as streams.
|
||||||
|
*
|
||||||
|
* @param {string} url `http:` or `https:`, as for a `fetch` upgrade
|
||||||
|
* @param {string[]} [protocols]
|
||||||
|
*/
|
||||||
|
async webSocket(url, protocols = []) {
|
||||||
|
checkUrl(this.#labels, url);
|
||||||
|
const headers = { Upgrade: "websocket" };
|
||||||
|
if (protocols.length) headers["Sec-WebSocket-Protocol"] = protocols.join(", ");
|
||||||
|
const resp = await this.#net.fetch(url, { headers });
|
||||||
|
const ws = resp.webSocket;
|
||||||
|
if (!ws) throw new Error(`no websocket: ${resp.status}`);
|
||||||
|
ws.accept();
|
||||||
|
const readable = new ReadableStream({
|
||||||
|
start(controller) {
|
||||||
|
ws.addEventListener("message", (e) => controller.enqueue(e.data));
|
||||||
|
ws.addEventListener("close", () => controller.close());
|
||||||
|
ws.addEventListener("error", () => controller.error(new Error("websocket error")));
|
||||||
|
},
|
||||||
|
});
|
||||||
|
const writable = new WritableStream({
|
||||||
|
write: (chunk) => ws.send(chunk),
|
||||||
|
close: () => ws.close(),
|
||||||
|
abort: () => ws.close(1011),
|
||||||
|
});
|
||||||
|
return { readable, writable, protocol: resp.headers.get("Sec-WebSocket-Protocol") ?? "" };
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* A switchboard socket: a WebSocket upgrade here starts a Cap'n Web
|
||||||
|
* session whose main object is a {@link Handle} with this socket's labels.
|
||||||
|
*/
|
||||||
|
export default class Root extends WorkerEntrypoint {
|
||||||
|
fetch(request) {
|
||||||
|
const labels = Object.freeze({ ...this.ctx.props });
|
||||||
|
const net = { public: this.env.PUBLIC, local: this.env.LOCAL }[labels.network];
|
||||||
|
if (!net) return new Response("socket has no network\n", { status: 500 });
|
||||||
|
return newWorkersRpcResponse(request, new Handle(labels, net));
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,15 @@
|
|||||||
|
[Unit]
|
||||||
|
Description=toybox switchboard (workerd, Cap'n Web egress handles)
|
||||||
|
Requires=toybox-switchboard.socket
|
||||||
|
|
||||||
|
[Service]
|
||||||
|
# the local (localhost) handle goes where no other unit can open it;
|
||||||
|
# hand it out deliberately, if ever
|
||||||
|
ExecStart=/opt/toybox/node_modules/workerd/bin/workerd serve --experimental /opt/toybox/switchboard.capnp -S public=3 -s local=unix:/run/toybox-switchboard/local.sock
|
||||||
|
RuntimeDirectory=toybox-switchboard
|
||||||
|
RuntimeDirectoryMode=0700
|
||||||
|
DynamicUser=yes
|
||||||
|
ProtectSystem=strict
|
||||||
|
ProtectHome=yes
|
||||||
|
PrivateTmp=yes
|
||||||
|
SystemCallFilter=@system-service
|
||||||
@@ -0,0 +1,11 @@
|
|||||||
|
[Unit]
|
||||||
|
Description=toybox switchboard: the public egress handle
|
||||||
|
|
||||||
|
[Socket]
|
||||||
|
ListenStream=/run/toybox/switchboard-public.sock
|
||||||
|
# only toybox holds this handle: its service joins this group
|
||||||
|
SocketGroup=toybox
|
||||||
|
SocketMode=0660
|
||||||
|
|
||||||
|
[Install]
|
||||||
|
WantedBy=sockets.target
|
||||||
@@ -0,0 +1,19 @@
|
|||||||
|
[Unit]
|
||||||
|
Description=toybox vats (workerd)
|
||||||
|
Requires=toybox.socket toybox-switchboard.socket
|
||||||
|
After=toybox-switchboard.socket
|
||||||
|
|
||||||
|
[Service]
|
||||||
|
ExecStart=/opt/toybox/node_modules/workerd/bin/workerd serve --experimental /opt/toybox/config.capnp -S http=3 -S admin=4 -d state=/var/lib/toybox -e switchboard=unix:/run/toybox/switchboard-public.sock
|
||||||
|
DynamicUser=yes
|
||||||
|
# to open the public switchboard socket
|
||||||
|
SupplementaryGroups=toybox
|
||||||
|
StateDirectory=toybox
|
||||||
|
# no network but the sockets handed in, and the switchboard's
|
||||||
|
PrivateNetwork=yes
|
||||||
|
RestrictAddressFamilies=AF_UNIX
|
||||||
|
ProtectSystem=strict
|
||||||
|
ProtectHome=yes
|
||||||
|
PrivateTmp=yes
|
||||||
|
SystemCallFilter=@system-service
|
||||||
|
# not MemoryDenyWriteExecute: V8's JIT needs W+X
|
||||||
@@ -0,0 +1,13 @@
|
|||||||
|
[Unit]
|
||||||
|
Description=toybox vats: public port and admin socket
|
||||||
|
|
||||||
|
[Socket]
|
||||||
|
# order sets the fds: http=3, admin=4 (see toybox.service)
|
||||||
|
ListenStream=127.0.0.1:8787
|
||||||
|
ListenStream=/run/toybox/admin.sock
|
||||||
|
# whoever can open admin.sock can install code
|
||||||
|
SocketUser=root
|
||||||
|
SocketMode=0600
|
||||||
|
|
||||||
|
[Install]
|
||||||
|
WantedBy=sockets.target
|
||||||
@@ -0,0 +1,10 @@
|
|||||||
|
// @endo/init/debug.js, but with `overrideTaming: 'moderate'`: Node 24's
|
||||||
|
// undici assigns `message` to its errors (ClientDestroyedError when a
|
||||||
|
// WebSocket closes), which 'min' makes a TypeError.
|
||||||
|
import '@endo/init/pre-remoting.js';
|
||||||
|
|
||||||
|
lockdown({
|
||||||
|
errorTaming: 'unsafe',
|
||||||
|
overrideTaming: 'moderate',
|
||||||
|
domainTaming: 'unsafe',
|
||||||
|
});
|
||||||
@@ -0,0 +1,53 @@
|
|||||||
|
/** @file The ingress's mount SQL, on node:sqlite: lookup, overlap, and mounts stored before mount_label. */
|
||||||
|
import { DatabaseSync } from "node:sqlite";
|
||||||
|
import { test } from "node:test";
|
||||||
|
import assert from "node:assert/strict";
|
||||||
|
import { LOOKUP, MOUNT, MOUNT_LABELS, MOUNT_SCHEMA, OVERLAP } from "../mounts-sql.js";
|
||||||
|
|
||||||
|
const makeDb = () => {
|
||||||
|
const db = new DatabaseSync(":memory:");
|
||||||
|
for (const statement of MOUNT_SCHEMA) db.exec(statement);
|
||||||
|
const mount = (vat, labels) => {
|
||||||
|
const json = JSON.stringify(labels);
|
||||||
|
db.prepare(MOUNT).run(vat, json);
|
||||||
|
db.prepare(MOUNT_LABELS).run(vat, json);
|
||||||
|
};
|
||||||
|
const lookup = (request) => db.prepare(LOOKUP).get(JSON.stringify(request))?.vat ?? null;
|
||||||
|
const overlaps = (vat, labels) => !!db.prepare(OVERLAP).get(JSON.stringify(labels), vat);
|
||||||
|
return { db, mount, lookup, overlaps };
|
||||||
|
};
|
||||||
|
|
||||||
|
test("a request reaches the mount whose every label it carries", () => {
|
||||||
|
const { mount, lookup } = makeDb();
|
||||||
|
mount("a", { vat: "alice", flavor: "plain" });
|
||||||
|
mount("b", { vat: "bob" });
|
||||||
|
assert.equal(lookup({ vat: "alice", flavor: "plain", page: "2" }), "a");
|
||||||
|
assert.equal(lookup({ vat: "alice" }), null);
|
||||||
|
assert.equal(lookup({ vat: "alice", flavor: "fancy" }), null);
|
||||||
|
assert.equal(lookup({ vat: "bob" }), "b");
|
||||||
|
assert.equal(lookup({}), null);
|
||||||
|
});
|
||||||
|
|
||||||
|
test("a mount with no labels matches any request", () => {
|
||||||
|
const { mount, lookup } = makeDb();
|
||||||
|
mount("only", {});
|
||||||
|
assert.equal(lookup({ anything: "x" }), "only");
|
||||||
|
});
|
||||||
|
|
||||||
|
test("overlap: a mount must conflict with every other", () => {
|
||||||
|
const { mount, overlaps } = makeDb();
|
||||||
|
mount("a", { vat: "alice" });
|
||||||
|
assert.equal(overlaps("b", { vat: "alice", n: "2" }), true);
|
||||||
|
assert.equal(overlaps("b", { app: "x" }), true);
|
||||||
|
assert.equal(overlaps("b", { vat: "bob" }), false);
|
||||||
|
assert.equal(overlaps("a", { vat: "alice", n: "2" }), false, "a vat's own mount doesn't count");
|
||||||
|
});
|
||||||
|
|
||||||
|
test("mounts stored before mount_label are filled in when the schema runs", () => {
|
||||||
|
const db = new DatabaseSync(":memory:");
|
||||||
|
db.exec("CREATE TABLE mount (vat TEXT PRIMARY KEY, labels TEXT)");
|
||||||
|
db.prepare("INSERT INTO mount VALUES (?, ?)").run("old", JSON.stringify({ vat: "old" }));
|
||||||
|
for (const statement of MOUNT_SCHEMA) db.exec(statement);
|
||||||
|
assert.equal(db.prepare(LOOKUP).get(JSON.stringify({ vat: "old" }))?.vat, "old");
|
||||||
|
assert.equal(db.prepare(LOOKUP).get(JSON.stringify({ vat: "other" })), undefined);
|
||||||
|
});
|
||||||
@@ -0,0 +1,105 @@
|
|||||||
|
/** @file OCapN vats: a client on the admin socket creates a vat and passes it a remote object. */
|
||||||
|
import { mkdir, mkdtemp, readFile, rm } from "node:fs/promises";
|
||||||
|
import { tmpdir } from "node:os";
|
||||||
|
import { join } from "node:path";
|
||||||
|
import { after, before, test } from "node:test";
|
||||||
|
import assert from "node:assert/strict";
|
||||||
|
import { E, Far } from "@endo/far";
|
||||||
|
import { connectClient } from "../client.js";
|
||||||
|
import { freePort, here, startWorkerd } from "./workerd.js";
|
||||||
|
|
||||||
|
const vatSrc = (name) => readFile(join(here, "vats", name), "utf8");
|
||||||
|
|
||||||
|
let dir, port, server, client;
|
||||||
|
|
||||||
|
const startToybox = () =>
|
||||||
|
startWorkerd(
|
||||||
|
"config.capnp",
|
||||||
|
[
|
||||||
|
"-s", `http=127.0.0.1:${port}`, "-s", `admin=unix:${join(dir, "admin.sock")}`,
|
||||||
|
"-d", `state=${join(dir, "state")}`, "-e", `switchboard=unix:${join(dir, "no-switchboard.sock")}`,
|
||||||
|
],
|
||||||
|
{ port, host: "127.0.0.1" },
|
||||||
|
[join(dir, "admin.sock")],
|
||||||
|
);
|
||||||
|
|
||||||
|
before(async () => {
|
||||||
|
dir = await mkdtemp(join(tmpdir(), "toybox-ocapn-"));
|
||||||
|
await mkdir(join(dir, "state"));
|
||||||
|
port = await freePort();
|
||||||
|
server = await startToybox();
|
||||||
|
client = await connectClient(join(dir, "admin.sock"), "client-1", { verbose: !!process.env.OCAPN_LOG });
|
||||||
|
});
|
||||||
|
|
||||||
|
after(async () => {
|
||||||
|
client?.close();
|
||||||
|
await server?.stop();
|
||||||
|
await rm(dir, { recursive: true, force: true });
|
||||||
|
});
|
||||||
|
|
||||||
|
test("createVat passes a remote object in; the vat's root comes back", async () => {
|
||||||
|
const carol = Far("carol", { greet: (name) => `hi ${name}, from carol` });
|
||||||
|
const { id, root } = await E(client.vatAdmin).createVat(await vatSrc("greeter.js"), { carol });
|
||||||
|
assert.match(id, /^[0-9a-f]{64}$/);
|
||||||
|
assert.equal(await E(root).hello(), "hello from the vat");
|
||||||
|
assert.equal(await E(root).greetVia("vat"), "hi vat, from carol");
|
||||||
|
});
|
||||||
|
|
||||||
|
test("a counter vat counts; two counter vats count apart", async () => {
|
||||||
|
const src = await vatSrc("counter.js");
|
||||||
|
const { root: a } = await E(client.vatAdmin).createVat(src);
|
||||||
|
const { root: b } = await E(client.vatAdmin).createVat(src);
|
||||||
|
assert.equal(await E(a).increment(), 1);
|
||||||
|
assert.equal(await E(a).increment(), 2);
|
||||||
|
assert.equal(await E(b).increment(), 1);
|
||||||
|
assert.equal(await E(a).read(), 2);
|
||||||
|
assert.equal(await E(b).read(), 1);
|
||||||
|
});
|
||||||
|
|
||||||
|
test("a vat serves the web through the ingress it is given", async () => {
|
||||||
|
const ingress = await E(client.vatAdmin).makeIngress({ vat: "hello" });
|
||||||
|
await E(client.vatAdmin).createVat(await vatSrc("hello.js"), { ingress });
|
||||||
|
const resp = await fetch(`http://127.0.0.1:${port}/greet?vat=hello`);
|
||||||
|
assert.equal(resp.status, 200);
|
||||||
|
assert.equal(resp.headers.get("content-type"), "text/html; charset=utf-8");
|
||||||
|
assert.match(await resp.text(), /<h1>Hello, world<\/h1><p>from a toybox vat, at \/greet<\/p>/);
|
||||||
|
});
|
||||||
|
|
||||||
|
test("an ingress nobody serves is a 503", async () => {
|
||||||
|
await E(client.vatAdmin).makeIngress({ vat: "idle" });
|
||||||
|
assert.equal((await fetch(`http://127.0.0.1:${port}/?vat=idle`)).status, 503);
|
||||||
|
});
|
||||||
|
|
||||||
|
test("after a crash, a request restarts the vat that serves it", async () => {
|
||||||
|
// its own toybox: a crash breaks every session to it, the shared client's too
|
||||||
|
const home = await mkdtemp(join(tmpdir(), "toybox-phoenix-"));
|
||||||
|
await mkdir(join(home, "state"));
|
||||||
|
const port2 = await freePort();
|
||||||
|
const start = () =>
|
||||||
|
startWorkerd(
|
||||||
|
"config.capnp",
|
||||||
|
[
|
||||||
|
"-s", `http=127.0.0.1:${port2}`, "-s", `admin=unix:${join(home, "admin.sock")}`,
|
||||||
|
"-d", `state=${join(home, "state")}`, "-e", `switchboard=unix:${join(home, "no-switchboard.sock")}`,
|
||||||
|
],
|
||||||
|
{ port: port2, host: "127.0.0.1" },
|
||||||
|
[join(home, "admin.sock")],
|
||||||
|
);
|
||||||
|
let toybox = await start();
|
||||||
|
try {
|
||||||
|
const creator = await connectClient(join(home, "admin.sock"), "creator");
|
||||||
|
const ingress = await E(creator.vatAdmin).makeIngress({ vat: "phoenix" });
|
||||||
|
const { root } = await E(creator.vatAdmin).createVat(await vatSrc("hello.js"), { ingress });
|
||||||
|
assert.equal((await fetch(`http://127.0.0.1:${port2}/?vat=phoenix`)).status, 200);
|
||||||
|
await root; // its handoff, settled before the client goes
|
||||||
|
creator.close();
|
||||||
|
await toybox.stop();
|
||||||
|
toybox = await start();
|
||||||
|
const resp = await fetch(`http://127.0.0.1:${port2}/again?vat=phoenix`);
|
||||||
|
assert.equal(resp.status, 200);
|
||||||
|
assert.match(await resp.text(), /Hello, world.*at \/again/);
|
||||||
|
} finally {
|
||||||
|
await toybox.stop();
|
||||||
|
await rm(home, { recursive: true, force: true });
|
||||||
|
}
|
||||||
|
});
|
||||||
@@ -0,0 +1,266 @@
|
|||||||
|
/** @file toybox end to end: a real workerd, an admin socket, a public port, a switchboard with two handles. */
|
||||||
|
import { mkdir, mkdtemp, readFile, rm } from "node:fs/promises";
|
||||||
|
import { createServer as createHttpServer, request } from "node:http";
|
||||||
|
import { tmpdir } from "node:os";
|
||||||
|
import { join } from "node:path";
|
||||||
|
import { after, before, test } from "node:test";
|
||||||
|
import assert from "node:assert/strict";
|
||||||
|
import { WebSocketServer } from "ws";
|
||||||
|
import { freePort, here, startWorkerd } from "./workerd.js";
|
||||||
|
const app = (name) => readFile(join(here, "apps", name), "utf8");
|
||||||
|
|
||||||
|
/**
|
||||||
|
* The admin socket: holding it is the authority to create and upgrade vats.
|
||||||
|
*
|
||||||
|
* @param {string} socketPath
|
||||||
|
*/
|
||||||
|
const makeAdmin = (socketPath) => {
|
||||||
|
const send = (method, path, src, { egress, mount } = {}) =>
|
||||||
|
new Promise((resolve, reject) => {
|
||||||
|
const q = new URLSearchParams();
|
||||||
|
if (egress) q.set("egress", typeof egress === "string" ? egress : JSON.stringify(egress));
|
||||||
|
if (mount) q.set("mount", JSON.stringify(mount));
|
||||||
|
const req = request({ socketPath, path: `${path}?${q}`, method, agent: false }, (res) => {
|
||||||
|
let body = "";
|
||||||
|
res.on("data", (c) => (body += c));
|
||||||
|
res.on("end", () => resolve(res.statusCode === 200 ? JSON.parse(body) : { status: res.statusCode, body }));
|
||||||
|
});
|
||||||
|
req.on("error", reject);
|
||||||
|
req.end(src);
|
||||||
|
});
|
||||||
|
return {
|
||||||
|
/** `createVat`: a new vat, reachable only by its id here, and by its mount */
|
||||||
|
create: (src, grants) => send("POST", "/", src, grants),
|
||||||
|
upgrade: (vat, src, grants) => send("PUT", `/${vat}`, src, grants),
|
||||||
|
/** a vat from before ids */
|
||||||
|
byName: (name, src, grants) => send("PUT", `/by-name/${name}`, src, grants),
|
||||||
|
};
|
||||||
|
};
|
||||||
|
|
||||||
|
/** @param {string} base the public origin */
|
||||||
|
const makePublic = (base) => ({
|
||||||
|
get: async (pathAndQuery, headers = {}) => {
|
||||||
|
const resp = await fetch(`${base}${pathAndQuery}`, { headers });
|
||||||
|
return { status: resp.status, text: await resp.text(), referrerPolicy: resp.headers.get("Referrer-Policy") };
|
||||||
|
},
|
||||||
|
text: async (pathAndQuery) => (await fetch(`${base}${pathAndQuery}`)).text(),
|
||||||
|
put: async (pathAndQuery, src) => (await fetch(`${base}${pathAndQuery}`, { method: "PUT", body: src })).status,
|
||||||
|
});
|
||||||
|
|
||||||
|
/** An HTTP server that answers GET with "plain ok", echoes WebSocket messages, and counts requests. */
|
||||||
|
const startEcho = async () => {
|
||||||
|
let hits = 0;
|
||||||
|
const server = createHttpServer((_req, res) => (hits++, res.end("plain ok")));
|
||||||
|
const wss = new WebSocketServer({ server });
|
||||||
|
wss.on("connection", (ws) => ws.on("message", (m) => ws.send(String(m))));
|
||||||
|
await new Promise((resolve) => server.listen(0, "127.0.0.1", resolve));
|
||||||
|
return {
|
||||||
|
host: `127.0.0.1:${server.address().port}`,
|
||||||
|
hits: () => hits,
|
||||||
|
stop: () => new Promise((resolve) => (wss.close(), server.closeAllConnections(), server.close(resolve))),
|
||||||
|
};
|
||||||
|
};
|
||||||
|
|
||||||
|
let dir, port, server, switchboard, echo, echoLabels, admin, pub;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* A toybox in `dir/<name>` that holds one switchboard handle.
|
||||||
|
*
|
||||||
|
* @param {string} name
|
||||||
|
* @param {number} port
|
||||||
|
* @param {"public" | "local"} handle which switchboard socket it gets
|
||||||
|
*/
|
||||||
|
const startToybox = async (name, port, handle) => {
|
||||||
|
const home = join(dir, name);
|
||||||
|
await mkdir(join(home, "state"), { recursive: true });
|
||||||
|
return startWorkerd(
|
||||||
|
"config.capnp",
|
||||||
|
[
|
||||||
|
"-s", `http=127.0.0.1:${port}`, "-s", `admin=unix:${join(home, "admin.sock")}`,
|
||||||
|
"-d", `state=${join(home, "state")}`, "-e", `switchboard=unix:${join(dir, `sb-${handle}.sock`)}`,
|
||||||
|
],
|
||||||
|
{ port, host: "127.0.0.1" },
|
||||||
|
[join(home, "admin.sock")],
|
||||||
|
);
|
||||||
|
};
|
||||||
|
|
||||||
|
before(async () => {
|
||||||
|
dir = await mkdtemp(join(tmpdir(), "toybox-"));
|
||||||
|
port = await freePort();
|
||||||
|
echo = await startEcho();
|
||||||
|
echoLabels = { host: "127.0.0.1", port: echo.host.split(":")[1] };
|
||||||
|
switchboard = await startWorkerd(
|
||||||
|
"switchboard.capnp",
|
||||||
|
["-s", `public=unix:${join(dir, "sb-public.sock")}`, "-s", `local=unix:${join(dir, "sb-local.sock")}`],
|
||||||
|
{ path: join(dir, "sb-local.sock") },
|
||||||
|
[join(dir, "sb-public.sock"), join(dir, "sb-local.sock")],
|
||||||
|
);
|
||||||
|
// the tests' echo server is on localhost, so this toybox holds the local handle
|
||||||
|
server = await startToybox("main", port, "local");
|
||||||
|
admin = makeAdmin(join(dir, "main", "admin.sock"));
|
||||||
|
pub = makePublic(`http://127.0.0.1:${port}`);
|
||||||
|
});
|
||||||
|
|
||||||
|
after(async () => {
|
||||||
|
await server?.stop();
|
||||||
|
await switchboard?.stop();
|
||||||
|
await echo?.stop();
|
||||||
|
await rm(dir, { recursive: true, force: true });
|
||||||
|
});
|
||||||
|
|
||||||
|
const at = (name) => ({ mount: { vat: name } });
|
||||||
|
|
||||||
|
test("nothing mounted: 404", async () => {
|
||||||
|
assert.equal((await pub.get("/?vat=nobody")).status, 404);
|
||||||
|
});
|
||||||
|
|
||||||
|
test("createVat returns an id and a code hash; unmounted, the vat is unreachable", async () => {
|
||||||
|
const { vat, hash } = await admin.create(await app("counter-v1.js"));
|
||||||
|
assert.match(vat, /^[0-9a-f]{64}$/);
|
||||||
|
assert.match(hash, /^[0-9a-f]{64}$/);
|
||||||
|
assert.equal((await pub.get("/")).status, 404);
|
||||||
|
});
|
||||||
|
|
||||||
|
test("install, run, upgrade: the app's state carries over; every start states its grants", async () => {
|
||||||
|
const { vat } = await admin.create(await app("counter-v1.js"), at("alice"));
|
||||||
|
assert.equal(await pub.text("/?vat=alice"), "v1 count 1\n");
|
||||||
|
assert.equal(await pub.text("/?vat=alice"), "v1 count 2\n");
|
||||||
|
await admin.upgrade(vat, await app("counter-v2.js"), at("alice"));
|
||||||
|
assert.equal(await pub.text("/?vat=alice"), "v2 count 3\n");
|
||||||
|
await admin.upgrade(vat, await app("counter-v2.js"));
|
||||||
|
assert.equal((await pub.get("/?vat=alice")).status, 404);
|
||||||
|
await admin.upgrade(vat, await app("counter-v2.js"), at("alice"));
|
||||||
|
assert.equal(await pub.text("/?vat=alice"), "v2 count 4\n");
|
||||||
|
});
|
||||||
|
|
||||||
|
test("vats don't share state", async () => {
|
||||||
|
await admin.create(await app("counter-v1.js"), at("bob"));
|
||||||
|
await admin.create(await app("counter-v1.js"), at("carol"));
|
||||||
|
assert.equal(await pub.text("/?vat=bob"), "v1 count 1\n");
|
||||||
|
assert.equal(await pub.text("/?vat=carol"), "v1 count 1\n");
|
||||||
|
});
|
||||||
|
|
||||||
|
test("an app gets no fetch", async () => {
|
||||||
|
await admin.create(await app("try-fetch.js"), at("bob-fetch"));
|
||||||
|
assert.match(await pub.text("/?vat=bob-fetch"), /^fetch refused: This worker is not permitted to access the internet/);
|
||||||
|
});
|
||||||
|
|
||||||
|
test("the public port can't install code", async () => {
|
||||||
|
assert.equal(await pub.put("/", "export class App {}"), 404);
|
||||||
|
});
|
||||||
|
|
||||||
|
test("the path is the app's; its labels are the mount's and the request's together", async () => {
|
||||||
|
await admin.create(await app("show-labels.js"), at("lab"));
|
||||||
|
const got = await pub.get("/a/b?vat=lab&page=2");
|
||||||
|
assert.equal(got.text, `/a/b ${JSON.stringify({ vat: "lab", page: "2" })}\n`);
|
||||||
|
assert.equal(got.referrerPolicy, "no-referrer");
|
||||||
|
});
|
||||||
|
|
||||||
|
test("a label given twice is a 400", async () => {
|
||||||
|
assert.equal((await pub.get("/?vat=lab&vat=lab")).status, 400);
|
||||||
|
});
|
||||||
|
|
||||||
|
test("mounts that could both match a request are refused", async () => {
|
||||||
|
const src = await app("show-labels.js");
|
||||||
|
await admin.create(src, at("x"));
|
||||||
|
const narrower = await admin.create(src, { mount: { vat: "x", n: "2" } });
|
||||||
|
assert.equal(narrower.status, 409);
|
||||||
|
assert.match(narrower.body, /overlaps another/);
|
||||||
|
// no key in common with the {vat} mounts: ?vat=x&app=x would carry both
|
||||||
|
assert.equal((await admin.create(src, { mount: { app: "x" } })).status, 409);
|
||||||
|
assert.match((await admin.create(src, { mount: { vat: "y", n: "2" } })).vat, /^[0-9a-f]{64}$/);
|
||||||
|
});
|
||||||
|
|
||||||
|
test("a request reaches a vat only by naming every label of its mount", async () => {
|
||||||
|
await admin.create(await app("show-labels.js"), { mount: { vat: "both", flavor: "plain" } });
|
||||||
|
assert.equal((await pub.get("/?vat=both")).status, 404);
|
||||||
|
assert.equal((await pub.get("/?vat=both&flavor=fancy")).status, 404);
|
||||||
|
assert.equal((await pub.get("/?vat=both&flavor=plain")).status, 200);
|
||||||
|
});
|
||||||
|
|
||||||
|
test("routing reads no labels from headers", async () => {
|
||||||
|
const got = await pub.get("/?vat=both", { "Toybox-Labels": JSON.stringify({ flavor: "plain" }) });
|
||||||
|
assert.equal(got.status, 404);
|
||||||
|
});
|
||||||
|
|
||||||
|
test("code, state, and mount survive a restart", async () => {
|
||||||
|
await admin.create(await app("counter-v1.js"), at("dave"));
|
||||||
|
assert.equal(await pub.text("/?vat=dave"), "v1 count 1\n");
|
||||||
|
await server.stop();
|
||||||
|
server = await startToybox("main", port, "local");
|
||||||
|
assert.equal(await pub.text("/?vat=dave"), "v1 count 2\n");
|
||||||
|
});
|
||||||
|
|
||||||
|
test("a vat from before ids is unreachable until the admin mounts it by its old name", async () => {
|
||||||
|
await admin.byName("old", await app("counter-v1.js"));
|
||||||
|
assert.equal((await pub.get("/?vat=old")).status, 404);
|
||||||
|
await admin.byName("old", await app("counter-v1.js"), at("old"));
|
||||||
|
assert.equal(await pub.text("/?vat=old"), "v1 count 1\n");
|
||||||
|
});
|
||||||
|
|
||||||
|
const probe = (vat, target, { ws, constrain } = {}) =>
|
||||||
|
pub.text(
|
||||||
|
`/?vat=${vat}&url=${encodeURIComponent(`http://${target}/`)}` +
|
||||||
|
(ws ? "&ws=1" : "") +
|
||||||
|
(constrain ? `&constrain=${encodeURIComponent(JSON.stringify(constrain))}` : ""),
|
||||||
|
);
|
||||||
|
|
||||||
|
test("a vat granted a host reaches it through the switchboard: fetch and WebSocket", async () => {
|
||||||
|
await admin.create(await app("egress-probe.js"), { ...at("erin"), egress: echoLabels });
|
||||||
|
assert.equal(await probe("erin", echo.host), "got: plain ok\n");
|
||||||
|
assert.equal(await probe("erin", echo.host, { ws: true }), "echo: hello\n");
|
||||||
|
});
|
||||||
|
|
||||||
|
test("same code, no grant: no egress", async () => {
|
||||||
|
await admin.create(await app("egress-probe.js"), at("frank"));
|
||||||
|
assert.equal(await probe("frank", echo.host), "no egress granted\n");
|
||||||
|
});
|
||||||
|
|
||||||
|
test("a grant is an object, not ambient: the global fetch still throws", async () => {
|
||||||
|
await admin.create(await app("try-fetch.js"), { ...at("heidi"), egress: echoLabels });
|
||||||
|
assert.match(await pub.text("/?vat=heidi"), /^fetch refused: This worker is not permitted to access the internet/);
|
||||||
|
});
|
||||||
|
|
||||||
|
test("the switchboard refuses what a grant doesn't cover", async () => {
|
||||||
|
await admin.create(await app("egress-probe.js"), { ...at("grace"), egress: { host: "127.0.0.1", port: "1" } });
|
||||||
|
assert.match(await probe("grace", echo.host), /^got: egress refused: .*egress to 127\.0\.0\.1 not granted/);
|
||||||
|
assert.match(await probe("grace", echo.host, { ws: true }), /^no websocket: 403 egress refused: .*not granted/);
|
||||||
|
});
|
||||||
|
|
||||||
|
test("a vat can narrow its grant, but not contradict it", async () => {
|
||||||
|
await admin.create(await app("egress-probe.js"), { ...at("ivan"), egress: { scheme: "http" } });
|
||||||
|
assert.equal(await probe("ivan", echo.host, { constrain: echoLabels }), "got: plain ok\n");
|
||||||
|
assert.match(
|
||||||
|
await probe("ivan", echo.host, { constrain: { scheme: "https" } }),
|
||||||
|
/^refused: label scheme is already set to a different value/,
|
||||||
|
);
|
||||||
|
});
|
||||||
|
|
||||||
|
test("a grant stored before labels, a bare host:port, still works", async () => {
|
||||||
|
await admin.create(await app("egress-probe.js"), { ...at("kim"), egress: echo.host });
|
||||||
|
assert.equal(await probe("kim", echo.host), "got: plain ok\n");
|
||||||
|
});
|
||||||
|
|
||||||
|
test("a grant survives a restart", async () => {
|
||||||
|
await server.stop();
|
||||||
|
server = await startToybox("main", port, "local");
|
||||||
|
assert.equal(await probe("erin", echo.host, { ws: true }), "echo: hello\n");
|
||||||
|
});
|
||||||
|
|
||||||
|
test("a toybox holding only the public handle can't reach localhost, whatever the grant", async () => {
|
||||||
|
const port2 = await freePort();
|
||||||
|
const other = await startToybox("public-only", port2, "public");
|
||||||
|
try {
|
||||||
|
const admin2 = makeAdmin(join(dir, "public-only", "admin.sock"));
|
||||||
|
await admin2.create(await app("egress-probe.js"), { ...at("judy"), egress: echoLabels });
|
||||||
|
const before = echo.hits();
|
||||||
|
const got = await (
|
||||||
|
await fetch(`http://127.0.0.1:${port2}/?vat=judy&url=${encodeURIComponent(`http://${echo.host}/`)}`)
|
||||||
|
).text();
|
||||||
|
// workerd reports the public network's refusal (restrictPeers) to JS only as "internal error"
|
||||||
|
assert.match(got, /^got: egress refused: /);
|
||||||
|
assert.equal(echo.hits(), before);
|
||||||
|
} finally {
|
||||||
|
await other.stop();
|
||||||
|
}
|
||||||
|
});
|
||||||
@@ -0,0 +1,51 @@
|
|||||||
|
/** @file Run real workerd processes for tests. */
|
||||||
|
import { spawn } from "node:child_process";
|
||||||
|
import { rm } from "node:fs/promises";
|
||||||
|
import { connect, createServer } from "node:net";
|
||||||
|
import { join } from "node:path";
|
||||||
|
|
||||||
|
export const here = new URL("..", import.meta.url).pathname;
|
||||||
|
// the package binary, not the .bin shim: the shim forks it, so a kill would orphan workerd
|
||||||
|
const workerd = join(here, "node_modules/workerd/bin/workerd");
|
||||||
|
|
||||||
|
export const freePort = () =>
|
||||||
|
new Promise((resolve) => {
|
||||||
|
const s = createServer().listen(0, "127.0.0.1", () => {
|
||||||
|
const { port } = s.address();
|
||||||
|
s.close(() => resolve(port));
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
/** Resolves once something accepts connections at `opts` (a port or a socket path). */
|
||||||
|
export const accepting = (opts) =>
|
||||||
|
new Promise((resolve) => {
|
||||||
|
const s = connect(opts, () => (s.end(), resolve(true)));
|
||||||
|
s.on("error", () => resolve(false));
|
||||||
|
});
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Run workerd on `config` until `stop()`; resolves once `listening` accepts.
|
||||||
|
*
|
||||||
|
* @param {string} config
|
||||||
|
* @param {string[]} args socket, directory, and external overrides
|
||||||
|
* @param {{ port: number } | { path: string }} listening
|
||||||
|
* @param {string[]} stale socket files a crash leaves behind; bind() would fail on them
|
||||||
|
*/
|
||||||
|
export const startWorkerd = async (config, args, listening, stale = []) => {
|
||||||
|
for (const f of stale) await rm(f, { force: true });
|
||||||
|
const child = spawn(workerd, ["serve", "--experimental", join(here, config), ...args], {
|
||||||
|
stdio: ["ignore", process.env.WORKERD_LOG ? "inherit" : "ignore", "pipe"],
|
||||||
|
});
|
||||||
|
let log = "";
|
||||||
|
// WORKERD_LOG=1 echoes workerd's stderr, for debugging
|
||||||
|
child.stderr.on("data", (c) => (process.env.WORKERD_LOG ? process.stderr.write(c) : (log += c)));
|
||||||
|
const exited = new Promise((resolve) => child.on("exit", resolve));
|
||||||
|
for (let i = 0; i < 100; i++) {
|
||||||
|
// SIGKILL: a crash is the harder durability test
|
||||||
|
if (await accepting(listening)) return { stop: () => (child.kill("SIGKILL"), exited) };
|
||||||
|
await new Promise((r) => setTimeout(r, 50));
|
||||||
|
}
|
||||||
|
child.kill("SIGKILL");
|
||||||
|
throw new Error(`workerd did not start: ${log}`);
|
||||||
|
};
|
||||||
|
|
||||||
@@ -0,0 +1,59 @@
|
|||||||
|
/**
|
||||||
|
* @file The runtime of an OCapN vat: SES, an OCapN peer, and the vat's own
|
||||||
|
* module (`vat.js`), which exports `buildRootObject(vatPowers, params)`.
|
||||||
|
*
|
||||||
|
* Bundled (see build.js) and loaded as the facet's main module beside the
|
||||||
|
* vat's code. The vat's designator is its id; it reaches other peers, and
|
||||||
|
* they reach it, through the vatAdmin's router (`env.router`).
|
||||||
|
*/
|
||||||
|
import "@endo/init";
|
||||||
|
import { DurableObject } from "cloudflare:workers";
|
||||||
|
import { E, Far } from "@endo/far";
|
||||||
|
import { makeClient } from "@endo/ocapn";
|
||||||
|
import { makeRpcNetLayer } from "./netlayer.js";
|
||||||
|
import * as vat from "./vat.js";
|
||||||
|
|
||||||
|
/**
|
||||||
|
* The swissnum of a vat's bootstrap: the vatAdmin starts the vat through it.
|
||||||
|
* Not exported: workerd takes every export of a main module for an entrypoint.
|
||||||
|
*/
|
||||||
|
const VAT_SWISSNUM = "vat";
|
||||||
|
|
||||||
|
export class App extends DurableObject {
|
||||||
|
#gateway;
|
||||||
|
|
||||||
|
constructor(ctx, env) {
|
||||||
|
super(ctx, env);
|
||||||
|
this.#gateway = this.#boot();
|
||||||
|
}
|
||||||
|
|
||||||
|
async #boot() {
|
||||||
|
const { vatId, router } = this.env;
|
||||||
|
let root;
|
||||||
|
const vatPowers = Object.freeze({ E, Far, storage: this.ctx.storage.kv });
|
||||||
|
const bootstrap = Far("vatBootstrap", {
|
||||||
|
/** (Re)build the root: on creation, and when the vatAdmin restarts the vat. */
|
||||||
|
start: (params) => {
|
||||||
|
root = vat.buildRootObject(vatPowers, params);
|
||||||
|
return root;
|
||||||
|
},
|
||||||
|
});
|
||||||
|
// workerd runs FinalizationRegistry callbacks outside any request, where
|
||||||
|
// sending the resulting GC message is forbidden I/O. So imports are
|
||||||
|
// never collected here; they go when their session does.
|
||||||
|
const ocapn = makeClient({
|
||||||
|
debugLabel: vatId.slice(0, 8),
|
||||||
|
swissnumTable: new Map([[VAT_SWISSNUM, bootstrap]]),
|
||||||
|
enableImportCollection: false,
|
||||||
|
});
|
||||||
|
const netlayer = await ocapn.registerNetlayer(
|
||||||
|
makeRpcNetLayer({ designator: vatId, openGateway: (designator) => router.gateway(designator) }),
|
||||||
|
);
|
||||||
|
return netlayer.gateway;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** This vat's OCapN gateway, for the router. */
|
||||||
|
gateway() {
|
||||||
|
return this.#gateway;
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,15 @@
|
|||||||
|
/**
|
||||||
|
* A counter whose count lives in the vat's storage.
|
||||||
|
*
|
||||||
|
* TODO: a durable Exo once vatPowers offers a zone, e.g.
|
||||||
|
* `zone.exoClass("Counter", undefined, () => ({ count: 0 }), { incr() { this.state.count += 1; return this.state.count; } })`
|
||||||
|
*/
|
||||||
|
export const buildRootObject = ({ Far, storage }) =>
|
||||||
|
Far("counterRoot", {
|
||||||
|
increment: () => {
|
||||||
|
const n = (storage.get("count") ?? 0) + 1;
|
||||||
|
storage.put("count", n);
|
||||||
|
return n;
|
||||||
|
},
|
||||||
|
read: () => storage.get("count") ?? 0,
|
||||||
|
});
|
||||||
@@ -0,0 +1,6 @@
|
|||||||
|
/** A vat that greets through the `carol` it was created with. */
|
||||||
|
export const buildRootObject = ({ E, Far }, { carol }) =>
|
||||||
|
Far("greeterRoot", {
|
||||||
|
greetVia: (name) => E(carol).greet(name),
|
||||||
|
hello: () => "hello from the vat",
|
||||||
|
});
|
||||||
@@ -0,0 +1,12 @@
|
|||||||
|
/** Hello, world, served through the `ingress` it was created with. */
|
||||||
|
export const buildRootObject = ({ E, Far }, { ingress }) => {
|
||||||
|
const handler = Far("helloHandler", {
|
||||||
|
handle: ({ path }) => ({
|
||||||
|
status: 200,
|
||||||
|
headers: { "content-type": "text/html; charset=utf-8" },
|
||||||
|
body: `<!doctype html><title>hello</title><h1>Hello, world</h1><p>from a toybox vat, at ${path}</p>\n`,
|
||||||
|
}),
|
||||||
|
});
|
||||||
|
E(ingress).serve(handler);
|
||||||
|
return Far("helloRoot", {});
|
||||||
|
};
|
||||||
+1
-1
@@ -3,7 +3,7 @@
|
|||||||
DO-backed fluent web client factory: site-agnostic. Deploy once; instances
|
DO-backed fluent web client factory: site-agnostic. Deploy once; instances
|
||||||
are attenuated handles whose **ids are bearer capabilities** — anyone
|
are attenuated handles whose **ids are bearer capabilities** — anyone
|
||||||
holding the id string can drive the instance; nobody without it can.
|
holding the id string can drive the instance; nobody without it can.
|
||||||
TODO: this proxy-page flow (see diigo-cache README) vs running webapp as
|
TODO: this proxy-page flow (see [diigo-cache CONTRIBUTING](../diigo-cache/CONTRIBUTING.md)) vs running webapp as
|
||||||
the primary worker (`wrangler dev -c ../webapp/wrangler.jsonc -c ../diigo-cache/wrangler.jsonc`)
|
the primary worker (`wrangler dev -c ../webapp/wrangler.jsonc -c ../diigo-cache/wrangler.jsonc`)
|
||||||
so its own port serves the mint form directly; reconsider when the UI
|
so its own port serves the mint form directly; reconsider when the UI
|
||||||
grows past a form or two.
|
grows past a form or two.
|
||||||
|
|||||||
Reference in new issue
Block a user