Compare commits
33
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
ff2e7841f2 | ||
|
|
7876938f3e | ||
|
|
a921d65021 | ||
|
|
055b572297 | ||
|
|
4dd3fe6bde | ||
|
|
3f711a5f4b | ||
|
|
357e4561c4 | ||
|
|
10414c2ce6 | ||
|
|
9179bdbf38 | ||
|
|
1d8a220bbc | ||
|
|
68132ff4b3 | ||
|
|
9c09e8c5a4 | ||
|
|
94a059af25 | ||
|
|
be796d9d0d | ||
|
|
c66aaea366 | ||
|
|
a25bd699e1 | ||
|
|
ea3e256bb8 | ||
|
|
0af7a3ea6c | ||
|
|
9719fd0aa8 | ||
|
|
bce2424cc0 | ||
|
|
7f1e19fe85 | ||
|
|
8a812d76e3 | ||
|
|
002f08b456 | ||
|
|
efb54ad4cc | ||
|
|
8c110098ff | ||
|
|
d61cb8c9a1 | ||
|
|
a2ca59a27e | ||
|
|
4a85143a66 | ||
|
|
37faa92649 | ||
|
|
ec24f3e8e4 | ||
|
|
47a2337afe | ||
|
|
9b12943be4 | ||
|
|
9a3e7348c6 |
No files matched your search
+24
-38
@@ -23,7 +23,7 @@ narrative.
|
||||
goal 5.
|
||||
|
||||
- [x] **1c. diigo-dump evolves into a bookmark cache**
|
||||
([diigo-cache/README.md](diigo-cache/README.md); design in the
|
||||
([diigo-cache/CONTRIBUTING.md](diigo-cache/CONTRIBUTING.md); design in the
|
||||
JSDoc of `src/cache.ts`, `src/index.ts`, `src/pm-save.ts`):
|
||||
- [x] 1c-i. Two independent page caches, private and public
|
||||
(`PagedCache`, `DiigoCache`, `PublicCache`).
|
||||
@@ -42,20 +42,16 @@ narrative.
|
||||
2026-10-07, via `scripts/compare-export.ts`. Getting there
|
||||
found diigo's `page_num` is 0-based (schema v2 migration).
|
||||
|
||||
- [ ] **1d. Date windows and search on the public page**, toward the
|
||||
- [x] **1d. Date windows and search on the public page**, toward the
|
||||
weekly PESOS roundup of [#23](https://github.com/dckc/madmode-blog/issues/23)
|
||||
(a week's permalink: `in=2026-W41`). One filter, two spellings:
|
||||
devs pass `{ days: 7, q: "#weekly" }`, URLs the same fields;
|
||||
`q` takes gmail/Slack/GitHub operators. UTC, ISO weeks, half-open
|
||||
windows; a fetch budget per request. Design: the API in
|
||||
`diigo-cache/src/query.ts` and the skipped suites in
|
||||
`tests/query.test.ts` and `tests/caches.test.ts` ("public page
|
||||
filters"); done when they're unskipped and green.
|
||||
Reconcile with 1f first: `query.ts` plans to filter while walking
|
||||
diigo's pages (binary search by `updated`), but after 1c-vi a
|
||||
filled cache holds every item, so 1d can query `items` in sqlite
|
||||
and spend diigo fetches only on a head refresh (1f). Choose
|
||||
before implementing; the skipped suites may need adjusting.
|
||||
windows (`diigo-cache/src/query.ts`; `tests/query.test.ts`,
|
||||
`tests/caches.test.ts` "public page filters"). Filters read
|
||||
stored `items` only (the public page never fetches): results are
|
||||
as fresh as the cache, and say `complete` when every diigo page
|
||||
is stored. Keeping them fresh is 1f. 2026-10-09.
|
||||
|
||||
- [x] **1e. Fill job: a button that grabs every page** into a cache
|
||||
from the webkey page `/cache/#<key>`, which trades the key for a
|
||||
@@ -93,41 +89,31 @@ narrative.
|
||||
[the issues sketch](potluck/issues-sketch.md))? If the policy
|
||||
changes, stripping stored rows is a durable-state migration.
|
||||
|
||||
- [ ] **2. madmode reads the public cache**: public data, so no
|
||||
capability, no cookie, no plugin sync job, no cron on madmode's
|
||||
side. madmode fetches `GET /public/dckc-madmode` with
|
||||
`Accept: application/json` (plus 1d's `in=`/`days=`/`q=`).
|
||||
- [ ] 2-i. Transport: a service binding to diigo-cache when both
|
||||
are deployed (no public hop, so no rate-limit exposure);
|
||||
in dev, a base URL (`http://localhost:8787`).
|
||||
- [ ] 2-ii. The JSON shape of the public page is a contract
|
||||
between two workers: pin it in a diigo-cache test (and
|
||||
madmode's reader tests against the same fixture).
|
||||
- [ ] 2-iii. Decide `DiigoBridge`'s fate: madmode doesn't need it
|
||||
(public reads need no cache id), so its "cache-id source"
|
||||
TODO (`src/index.ts`, README "RPC use") is either dropped
|
||||
with the entrypoint or kept for a private-reading caller.
|
||||
- [ ] **3. `/bookmarks` renders real data** — titles, tags, dates at
|
||||
http://localhost:4321/bookmarks, newest first.
|
||||
- [ ] **4. The weekly page**: `/bookmarks/2026-W41` → the cache's
|
||||
`in=2026-W41` (needs 1d and 1f). Decide (user's call): a raw
|
||||
week list, or a draft post the user edits before publishing
|
||||
(the PESOS practice is editorial; a tag like `#weekly` in `q`
|
||||
is one cheap form of discretion).
|
||||
- [ ] **2. potluck reads the public cache**: public data, so no capability, no cookie, no secret on potluck's side. The [diigo-week](diigo-week/README.md) plugin reads `GET /public/dckc-madmode?in=<week>&by=created` as microformats (an `h-feed` of `h-entry` bookmarks). Working in dev 2026-10-09; 2-iii postponed.
|
||||
- [x] 2-i. Transport: the `DIIGO_CACHE` service binding in potluck's `wrangler.jsonc`, in dev too: the plugin's `ctx.http.fetch` refuses localhost (EmDash's SSRF guard), and a binding names one worker, not a host.
|
||||
- [x] 2-ii. The page's microformats are the contract: diigo-cache's tests parse it with microformats-parser; diigo-week's tests read fixtures in the same markup.
|
||||
- [ ] 2-iii. Decide `DiigoBridge`'s fate (postponed: nothing on this path uses it; it matters only for a caller that reads private items).
|
||||
- [x] 2-iv. Run it in dev: diigo-cache and potluck both up; the dashboard card → a draft issue of 22 bookmarks for 2026-W40 (2026-10-09). The service binding resolves across the two dev servers.
|
||||
- [ ] ~~**3. `/bookmarks` renders real data**~~: superseded by potluck's issues (the madmode starter is gone).
|
||||
- [ ] **4. The weekly draft**: decided (2026-10-09): a draft issue you edit before publishing, made by diigo-week. The slug comes from the title (`week-2026-w40`).
|
||||
|
||||
- [ ] **4b. Consider scruffier tags.** EmDash tags are taxonomy terms (rows that must exist before use); diigo-week auto-creates them, so drafting feels scruffy, but the model is neat. Alternative: the inline `/tag/<tag>` links in issue bodies *are* the tags (matching the microformats `p-category` reading), and potluck's `/tag/<tag>` page finds issues by scanning bodies. Cost: no EmDash tag field or index; a scan over a few dozen issues a year is cheap. Open: does the admin's tag field let you type a new tag inline?
|
||||
|
||||
- [ ] **5. Deployed end-to-end** (order matters):
|
||||
- [ ] 5-i. webapp: `cd webapp && npx wrangler deploy`
|
||||
- [ ] 5-ii. diigo-cache: rate-limit `/api` (and `/public/*`) first
|
||||
(diigo-cache README, "Deployment concern"); deploy; create
|
||||
(diigo-cache CONTRIBUTING, "Deployment concern"); deploy; create
|
||||
against prod (ids → password manager; no worker secret);
|
||||
run the fill job.
|
||||
- [ ] 5-iii. madmode: deploy with the service binding; the
|
||||
weekly page renders against prod.
|
||||
- [ ] **6. Cookie expiry**: when diigo's session ends, head refreshes
|
||||
fail; the public page keeps serving what's stored (say so on the
|
||||
page, don't fail silently). Rotation is manual: re-mint via
|
||||
`/_connect`. Open question: can an existing cache be re-wired to
|
||||
a new webapp id (keeping its cache id and public username), or
|
||||
does re-mint mean a new cache? Prefer re-wire — durable ids.
|
||||
page, don't fail silently). Rotation is in place, keeping the
|
||||
webapp id (so the cache's wiring and ids hold), by the store id
|
||||
minted alongside it and kept by the account owner: `PUT
|
||||
/store/<storeId>/cookie` (webapp README, "The header store").
|
||||
Still open: a rotate form in diigo-cache's UI.
|
||||
- [ ] **7. bib sync #195**: repeat the pattern for the next
|
||||
read-access need (webapp mints for any origin; the cache's
|
||||
item shape is diigo-specific): [the zotero goal tree](zotero-checklist.md).
|
||||
@@ -239,5 +225,5 @@ to every goal in this tree.
|
||||
Oct 6 2026)
|
||||
- [webapp/README.md](webapp/README.md) — HTTP + RPC surfaces of the
|
||||
factory worker; open TODO on the mint-form placement
|
||||
- [diigo-cache/README.md](diigo-cache/README.md) — the cache worker:
|
||||
- [diigo-cache/CONTRIBUTING.md](diigo-cache/CONTRIBUTING.md) — the cache worker:
|
||||
create flow, the two DOs, run/deploy details
|
||||
Symlink
+1
@@ -0,0 +1 @@
|
||||
CONTRIBUTING.md
|
||||
@@ -0,0 +1,72 @@
|
||||
# Contributing to diigo-cache
|
||||
|
||||
diigo-cache is a Cloudflare Worker cache over [the webapp factory](../webapp/README.md): session-cookie reads of diigo bookmarks, cached in Durable-Object SQLite storage. Evolved from the diigo-dump worker (provenance: [mm-diigo `projects/diigo-bak/`](https://github.com/dckc/mm-diigo)); the static API-key path is dead (see [the capsite checklist](../capsite-checklist.md), goal 2 preamble).
|
||||
|
||||
## What it holds
|
||||
|
||||
- **The create flow, on one port**: `/_connect` renders webapp's own mint form (diigo-prefilled, cookie blank) — wrangler multi-config dev doesn't serve aux workers' HTTP, so `/_mint` relays the form opaquely into the WEBAPP binding (the cookie crosses; nothing on this side opens or logs it). Mint → `/_wire` (webapp id pre-filled, mode public|private) → the created page, where the user saves the ids.
|
||||
- **Two independent page caches** (`src/cache.ts`), one behavior: a page hit is served from storage with no diigo call; a miss asks the webapp instance in the cache's wiring and etches the page.
|
||||
- `DiigoCache` (binding `CACHE_DO`): every item. Addressed by its cache id.
|
||||
- `PublicCache` (binding `PUBLIC_DO`): shared items only — private rows are dropped *before* etching (`publicOnly`), so its storage holds nothing private. Same cookie webapp as the private cache. Public page n is diigo page n, filtered (may show fewer than `PAGE_SIZE`, 100: diigo's cap). `/public/<username>` resolves through the `idFromName("index")` registry, and reads through `viewer()`, a read-only facet: it shows stored pages and fill progress, and can't fetch from diigo or write.
|
||||
- **Storage**: SQL lives in `src/cache-read.sql` (selects only) and `src/cache-write.sql`; `src/store.ts` turns them into a reader and a writer facet, so the cache logic holds no SQL text.
|
||||
- **Pages**: home's dump form (cache id + page number, POST) shows raw dump rows; `/public/<username>?page=n` shows an HTML list with a page form (JSON for `Accept: application/json`). Both say `cache: HIT` or `MISS`; every page read also logs `{event: "cache", kind, page, outcome}`.
|
||||
- **Filters on the public page** (`src/query.ts`): `days`, `after`, `before`, `in` (`2026`, `2026-03`, `2026-Q1`, `2026-W41`), `by=updated|created`, and `q` (words, `"phrases"`, `#tag`, `-` negation, gmail/Slack/GitHub date operators). UTC, half-open windows. The same fields work as a dev-side spec (`toQuery({ days: 7 })`). A filter queries the stored `items` table through `viewer()`, never diigo: results are as fresh as the cache, and JSON says `complete` when every diigo page is stored.
|
||||
- **Publish later**: `POST /_publish` (cache id in the body) wires a public companion to a private-only cache.
|
||||
- **The webkey page and Cap'n Web**: `/cache/#<cache id>` is a skeleton page, the same for every key; its script reads the key from the fragment and trades it in-band for a capability over [Cap'n Web](https://github.com/cloudflare/capnweb) at `/api` (`CacheApi.cache(key)` → a `CacheCap` facet: fill, status, the public companion; `src/api.ts`). The worker serves capnweb's browser build itself at `/_capnweb.js`.
|
||||
- **Fill job**: walks every diigo page into a cache in the background — DO alarms, a few pages per step, progress stored in the DO (`startFill`/`fillStatus`/`stopFill`), driven from the webkey page; the public page shows progress. "Invalidate the last N days" there drops the cached pages holding anything updated that recently (`invalidateRecent`), so the next fill refetches them.
|
||||
- The `DiigoBridge` entrypoint for other workers (its cache-id source is the goal-2 wiring TODO).
|
||||
|
||||
## State surface
|
||||
|
||||
Ids are **webkeys** ([the pattern](../docs/webkeys.md)): 256-bit bearer secrets that never ride a query string or path. The webkey page carries the cache id in the URL fragment (never sent to the server) and passes it in-band over Cap'n Web; forms carry ids in POST bodies; logs and pages show only a fingerprint (a prefix of the id's one-way hash).
|
||||
|
||||
- **The cache id**: the cache's read capability (bearer; not logged). Held by the user — saved to the password manager under a petname they choose (the username field; never sent to the server) — and passed per POST.
|
||||
- **The webapp instance id**: the diigo-read capability; lives in each cache DO's wiring, arrives from the mint step.
|
||||
- **The cookie**: lives **only in webapp's DO storage**; crosses the wire once, at mint time, as an opaque relay.
|
||||
- **The public cache**: no capability — `/public/<username>` is public by design (a guessable URL; hiding the link doesn't hide the page).
|
||||
|
||||
Stored state is meant to be durable: see the breaking-change rule in [capsite AGENTS.md](../AGENTS.md).
|
||||
|
||||
## Deployment concern: rate-limit `/api`
|
||||
|
||||
Public pages only read storage, so strangers can't make the cache spend your diigo session. `/api` is keyed (a cache id is needed), but Cap'n Web pipelining makes calls cheap to send. Before exposing a deployment:
|
||||
|
||||
- rate-limit `/api` (and `/public/*`, which reads storage) — on Cloudflare, the [Workers rate-limit binding](https://developers.cloudflare.com/workers/runtime-apis/bindings/rate-limit/) (keyed by client and/or username); on other workerd hosts, a rate limit in the proxy in front;
|
||||
- run the fill job (goal 1e): public pages show only what's stored.
|
||||
|
||||
## RPC use (from another worker, same account)
|
||||
|
||||
```jsonc
|
||||
// wrangler.jsonc of the caller
|
||||
"services": [{ "binding": "DIIGO", "service": "diigo-cache", "entrypoint": "DiigoBridge" }]
|
||||
```
|
||||
|
||||
```ts
|
||||
// cache ids ride goal-2's wiring; see the capsite checklist
|
||||
const ndjson = await env.DIIGO.dumpNdjson(4); // 4 pages of PAGE_SIZE
|
||||
```
|
||||
|
||||
## Test
|
||||
|
||||
pnpm types # regenerate worker-configuration.d.ts (after wrangler.jsonc changes)
|
||||
pnpm lint # fails if those types are stale; tsc over src/ and tests/
|
||||
pnpm test # vitest
|
||||
|
||||
Tests run the real worker and both DOs in the vitest pool, with `WEBAPP` bound to [a fake webapp](tests/fake-webapp.js) serving diigo-shaped items (`wrangler.tests.jsonc`).
|
||||
|
||||
## Run
|
||||
|
||||
[The README](README.md) gets a cache going in dev. The same create flow from a terminal:
|
||||
|
||||
wid=$(curl -s -X POST http://localhost:8787/_mint --data-urlencode \
|
||||
'url=https://www.diigo.com/interact_api/' --data-urlencode \
|
||||
'headers={"user-agent":"Mozilla/5.0","x-requested-with":"XMLHttpRequest","cookie":"<paste>"}' \
|
||||
--data-urlencode 'msPerCall=500' -D - -o /dev/null | grep -i x-webapp-id)
|
||||
curl -s -X POST http://localhost:8787/_wire -H 'content-type: application/json' \
|
||||
-d "{\"webappId\":\"<from $wid>\",\"mode\":\"private\"}"
|
||||
|
||||
Check a filled cache against a diigo RSS export: `node scripts/compare-export.ts <cache.sqlite> <export.xml> <out-dir>`.
|
||||
|
||||
## Deploy
|
||||
|
||||
Not deployed yet; the order matters (goal 5 in [the capsite checklist](../capsite-checklist.md)): webapp first (`cd ../webapp && npx wrangler deploy`), then diigo-cache, after rate-limiting `/api` (above).
|
||||
+16
-110
@@ -1,120 +1,26 @@
|
||||
# diigo-cache
|
||||
|
||||
Cloudflare Worker cache over [the webapp factory](../webapp/README.md):
|
||||
session-cookie reads of diigo bookmarks, cached in Durable-Object SQLite
|
||||
storage. Evolved from the diigo-dump worker (provenance:
|
||||
[mm-diigo `projects/diigo-bak/`](https://github.com/dckc/mm-diigo)); the
|
||||
static API-key path is dead (see [the capsite checklist](../capsite-checklist.md),
|
||||
goal 2 preamble).
|
||||
Your diigo bookmarks, cached in a [workerd](https://github.com/cloudflare/workerd) worker, with a public page you can filter by week: `/public/<you>?in=2026-W41`. Your diigo session cookie is stored in one Durable Object and nowhere else.
|
||||
|
||||
## What it holds
|
||||
## Getting started
|
||||
|
||||
- **The create flow, on one port**: `/_connect` renders webapp's own
|
||||
mint form (diigo-prefilled, cookie blank) — wrangler multi-config dev
|
||||
doesn't serve aux workers' HTTP, so `/_mint` relays the form opaquely
|
||||
into the WEBAPP binding (the cookie crosses; nothing on this side opens
|
||||
or logs it). Mint → `/_wire` (webapp id pre-filled, mode
|
||||
public|private) → the created page, where the user saves the ids.
|
||||
- **Two independent page caches** (`src/cache.ts`), one behavior: a
|
||||
page hit is served from storage with no diigo call; a miss asks the
|
||||
webapp instance in the cache's wiring and etches the page.
|
||||
- `DiigoCache` (binding `CACHE_DO`): every item. Addressed by its
|
||||
cache id.
|
||||
- `PublicCache` (binding `PUBLIC_DO`): shared items only — private
|
||||
rows are dropped *before* etching (`publicOnly`), so its storage
|
||||
holds nothing private. Same cookie webapp as the private cache.
|
||||
Public page n is diigo page n, filtered (may show fewer than
|
||||
`PAGE_SIZE`, 100: diigo's cap).
|
||||
`/public/<username>` resolves through the `idFromName("index")`
|
||||
registry, and reads through `viewer()`, a read-only facet: it shows
|
||||
stored pages and fill progress, and can't fetch from diigo or write.
|
||||
- **Storage**: SQL lives in `src/cache-read.sql` (selects only) and
|
||||
`src/cache-write.sql`; `src/store.ts` turns them into a reader and a
|
||||
writer facet, so the cache logic holds no SQL text.
|
||||
- **Pages**: home's dump form (cache id + page number, POST) shows raw
|
||||
dump rows; `/public/<username>?page=n` shows an HTML list with a page
|
||||
form (JSON for `Accept: application/json`). Both say `cache: HIT` or
|
||||
`MISS`; every page read also logs `{event: "cache", kind, page,
|
||||
outcome}`.
|
||||
- **Publish later**: `POST /_publish` (cache id in the body) wires a
|
||||
public companion to a private-only cache.
|
||||
- **The webkey page and Cap'n Web**: `/cache/#<cache id>` is a skeleton
|
||||
page, the same for every key; its script reads the key from the
|
||||
fragment and trades it in-band for a capability over
|
||||
[Cap'n Web](https://github.com/cloudflare/capnweb) at `/api`
|
||||
(`CacheApi.cache(key)` → a `CacheCap` facet: fill, status, the public
|
||||
companion; `src/api.ts`). The worker serves capnweb's browser build
|
||||
itself at `/_capnweb.js`.
|
||||
- **Fill job**: walks every diigo page into a cache in the background —
|
||||
DO alarms, a few pages per step, progress stored in the DO
|
||||
(`startFill`/`fillStatus`/`stopFill`), driven from the webkey page;
|
||||
the public page shows progress. "Invalidate the last N days" there
|
||||
drops the cached pages holding anything updated that recently
|
||||
(`invalidateRecent`), so the next fill refetches them.
|
||||
- The `DiigoBridge` entrypoint for other workers (its cache-id source is
|
||||
the goal-2 wiring TODO).
|
||||
|
||||
## State surface
|
||||
|
||||
Ids are **webkeys** ([the pattern](../docs/webkeys.md)): 256-bit bearer
|
||||
secrets that never ride a query string or path. The webkey page carries
|
||||
the cache id in the URL fragment (never sent to the server) and passes
|
||||
it in-band over Cap'n Web; forms carry ids in POST bodies; logs and pages
|
||||
show only a fingerprint (a prefix of the id's one-way hash).
|
||||
|
||||
- **The cache id**: the cache's read capability (bearer; not logged).
|
||||
Held by the user — saved to the password manager under a petname they
|
||||
choose (the username field; never sent to the server) — and passed
|
||||
per POST.
|
||||
- **The webapp instance id**: the diigo-read capability; lives in each
|
||||
cache DO's wiring, arrives from the mint step.
|
||||
- **The cookie**: lives **only in webapp's DO storage**; crosses the
|
||||
wire once, at mint time, as an opaque relay.
|
||||
- **The public cache**: no capability — `/public/<username>` is public
|
||||
by design (a guessable URL; hiding the link doesn't hide the page).
|
||||
|
||||
Stored state is meant to be durable: see the breaking-change rule in
|
||||
[capsite AGENTS.md](../AGENTS.md).
|
||||
|
||||
## Deployment concern: rate-limit `/api`
|
||||
|
||||
Public pages only read storage, so strangers can't make the cache spend
|
||||
your diigo session. `/api` is keyed (a cache id is needed), but Cap'n
|
||||
Web pipelining makes calls cheap to send. Before exposing a deployment:
|
||||
|
||||
- rate-limit `/api` (and `/public/*`, which reads storage) — on
|
||||
Cloudflare, the
|
||||
[Workers rate-limit binding](https://developers.cloudflare.com/workers/runtime-apis/bindings/rate-limit/)
|
||||
(keyed by client and/or username); on other workerd hosts, a rate
|
||||
limit in the proxy in front;
|
||||
- run the fill job (goal 1e): public pages show only what's stored.
|
||||
|
||||
## RPC use (from another worker, same account)
|
||||
|
||||
```jsonc
|
||||
// wrangler.jsonc of the caller
|
||||
"services": [{ "binding": "DIIGO", "service": "diigo-cache", "entrypoint": "DiigoBridge" }]
|
||||
```sh
|
||||
pnpm install
|
||||
pnpm dev # → http://localhost:8787/
|
||||
```
|
||||
|
||||
```ts
|
||||
// cache ids ride goal-2's wiring; see the capsite checklist
|
||||
const ndjson = await env.DIIGO.dumpNdjson(4); // 4 pages of PAGE_SIZE
|
||||
```
|
||||
Then, in the browser:
|
||||
|
||||
## Test
|
||||
1. **mint the diigo session at webapp**: paste the `Cookie` header from diigo.com (devtools, any logged-in request).
|
||||
2. **wire** it, mode **public**. Save the cache id it shows (the password-manager button does this).
|
||||
3. **open this cache** → **fill**, and watch it walk your bookmarks.
|
||||
4. Visit `/public/<your diigo username>?days=7`.
|
||||
|
||||
pnpm types # regenerate worker-configuration.d.ts (after wrangler.jsonc changes)
|
||||
pnpm lint # fails if those types are stale; tsc over src/ and tests/
|
||||
pnpm test # vitest
|
||||
## Bookmarks you hold, a page anyone can read
|
||||
|
||||
Tests run the real worker and both DOs in the vitest pool, with
|
||||
`WEBAPP` bound to [a fake webapp](tests/fake-webapp.js) serving
|
||||
diigo-shaped items (`wrangler.tests.jsonc`).
|
||||
- **Filter by date and search:** `?days=7`, `?in=2026-W41` (ISO week, UTC), `?after=2026-01-01&before=2026-02-01`, `?by=created`, and `?q=` with words, `"phrases"`, `#tags`, `-negation`, and gmail-style `newer_than:7d`. Add `Accept: application/json` for JSON.
|
||||
- **Private stays private:** the public cache drops private bookmarks before storing anything, so it never holds them.
|
||||
- **Ids are capabilities:** the cache id is a bearer key you keep in your password manager. It rides URL fragments and POST bodies, never a path or query string, so it stays out of logs and history.
|
||||
- **Cheap to read:** pages come from Durable Object SQLite storage, and a fill job in the background fetches each diigo page once.
|
||||
|
||||
## Run, deploy
|
||||
|
||||
[How to run / connect / deploy is maintained in the capsite checklist](../capsite-checklist.md)
|
||||
(single source; this README keeps only worker-level facts).
|
||||
|
||||
Status: create, dump, publish, and public pages exercised against live
|
||||
diigo in dev (2026-10-07); not yet deployed.
|
||||
Design, tests, and deploying: [CONTRIBUTING](CONTRIBUTING.md).
|
||||
@@ -11,6 +11,7 @@
|
||||
"devDependencies": {
|
||||
"@cloudflare/vitest-plugin": "1.3.6",
|
||||
"@types/node": "^22",
|
||||
"microformats-parser": "^2.0.6",
|
||||
"typescript": "^5.9.0",
|
||||
"vitest": "4.1.11",
|
||||
"wrangler": "^4.148.0"
|
||||
|
||||
Generated
+24
@@ -18,6 +18,9 @@ importers:
|
||||
'@types/node':
|
||||
specifier: ^22
|
||||
version: 22.20.5
|
||||
microformats-parser:
|
||||
specifier: ^2.0.6
|
||||
version: 2.0.6
|
||||
typescript:
|
||||
specifier: ^5.9.0
|
||||
version: 5.9.3
|
||||
@@ -613,6 +616,10 @@ packages:
|
||||
resolution: {integrity: sha512-Btj2BOOO83o3WyH59e8MgXsxEQVcarkUOpEYrubB0urwnN10yQ364rsiByU11nZlqWYZm05i/of7io4mzihBtQ==}
|
||||
engines: {node: '>=8'}
|
||||
|
||||
entities@6.0.1:
|
||||
resolution: {integrity: sha512-aN97NXWF6AWBTahfVOIrB/NShkzi5H7F9r1s9mD3cDj4Ko5f2qhhVoYMibXF7GlLveb/D2ioWay8lxI97Ven3g==}
|
||||
engines: {node: '>=0.12'}
|
||||
|
||||
error-stack-parser-es@1.0.5:
|
||||
resolution: {integrity: sha512-5qucVt2XcuGMcEGgWI7i+yZpmpByQ8J1lHhcL7PwqCwu9FPP3VUXzT4ltHe5i2z9dePwEHcDVOAfSnHsOlCXRA==}
|
||||
|
||||
@@ -722,6 +729,10 @@ packages:
|
||||
magic-string@0.30.21:
|
||||
resolution: {integrity: sha512-vd2F4YUyEXKGcLHoq+TEyCjxueSeHnFxyyjNp80yg0XV4vUhnDer/lvvlqM/arB5bXQN5K2/3oinyCRyx8T2CQ==}
|
||||
|
||||
microformats-parser@2.0.6:
|
||||
resolution: {integrity: sha512-VTnLImF17yd+ZfBH40sddHMJp5Rrker4ge+oPjgnBQ3jluQO0YmsNVA4UcEfHRBV8zrikBSMkAzAub+nTRX85g==}
|
||||
engines: {node: '>=18'}
|
||||
|
||||
miniflare@5.20261001.0-alpha:
|
||||
resolution: {integrity: sha512-GaimS5mSIOMyvd16ga+e1/QkI8cmp3z35QPHFex0DktqNQf2RVZQwMPQXdyoUreUiFP/0Gpe2MoQInTyMAD5xA==}
|
||||
engines: {node: '>=22.0.0'}
|
||||
@@ -739,6 +750,9 @@ packages:
|
||||
resolution: {integrity: sha512-XrsrhT5sybtKI6wakr2SPOlGZWWYbUXZ7a0jT8/QOeAPau+1X/bSegNe5YR75oJmEZQbKningirmGOEJCIk61Q==}
|
||||
engines: {node: '>=12.20.0'}
|
||||
|
||||
parse5@7.3.0:
|
||||
resolution: {integrity: sha512-IInvU7fabl34qmi9gY8XOVxhYyMyuH2xUNpb2q8/Y+7552KlejkRvqvD19nMoUW/uQGGbqNpA6Tufu5FL5BZgw==}
|
||||
|
||||
path-to-regexp@6.3.0:
|
||||
resolution: {integrity: sha512-Yhpw4T9C6hPpgPeA28us07OJeqZ5EzQTkbfwuhsUg0c237RomFoETJgmp2sa3F/41gfLE6G5cqcYwznmeEeOlQ==}
|
||||
|
||||
@@ -1366,6 +1380,8 @@ snapshots:
|
||||
|
||||
detect-libc@2.1.2: {}
|
||||
|
||||
entities@6.0.1: {}
|
||||
|
||||
error-stack-parser-es@1.0.5: {}
|
||||
|
||||
es-module-lexer@2.3.2: {}
|
||||
@@ -1467,6 +1483,10 @@ snapshots:
|
||||
dependencies:
|
||||
'@jridgewell/sourcemap-codec': 1.6.0
|
||||
|
||||
microformats-parser@2.0.6:
|
||||
dependencies:
|
||||
parse5: 7.3.0
|
||||
|
||||
miniflare@5.20261001.0-alpha(@types/node@22.20.5):
|
||||
dependencies:
|
||||
'@cspotcode/source-map-support': 0.8.1
|
||||
@@ -1497,6 +1517,10 @@ snapshots:
|
||||
|
||||
obug@2.2.1: {}
|
||||
|
||||
parse5@7.3.0:
|
||||
dependencies:
|
||||
entities: 6.0.1
|
||||
|
||||
path-to-regexp@6.3.0: {}
|
||||
|
||||
pathe@2.0.3: {}
|
||||
|
||||
@@ -19,3 +19,14 @@ where cast(items.updated as integer) >= ?;
|
||||
|
||||
-- pagesThrough:
|
||||
select count(*) as n from pages where page_num <= ?;
|
||||
|
||||
-- itemsByUpdated:
|
||||
select json from items
|
||||
where cast(updated as integer) >= ? and cast(updated as integer) < ?
|
||||
order by cast(updated as integer) desc, item_id;
|
||||
|
||||
-- itemsByCreated:
|
||||
select json from items
|
||||
where cast(json_extract(json, '$.created_at') as integer) >= ?
|
||||
and cast(json_extract(json, '$.created_at') as integer) < ?
|
||||
order by cast(json_extract(json, '$.created_at') as integer) desc, item_id;
|
||||
@@ -6,6 +6,8 @@
|
||||
import { DurableObject, RpcTarget } from "cloudflare:workers";
|
||||
import type { PagedGet } from "./diigo.js";
|
||||
import { publicOnly, toRows } from "./items.js";
|
||||
import { matches } from "./query.js";
|
||||
import type { Query } from "./query.js";
|
||||
import { makeReader, makeWriter, SCHEMA_VERSION } from "./store.js";
|
||||
import type { Reader, Writer } from "./store.js";
|
||||
import type { Item, ItemRow } from "./items.js";
|
||||
@@ -398,7 +400,18 @@ abstract class PagedCache extends DurableObject<CacheEnv> {
|
||||
}
|
||||
}
|
||||
|
||||
/** what anyone may see of a cache: stored pages and fill progress */
|
||||
/** one page of a filter's matches over stored items */
|
||||
export type QueryPage = {
|
||||
/** the matches on this page, raw dump rows (JSON strings) */
|
||||
items: string[];
|
||||
/** matches in all */
|
||||
matched: number;
|
||||
/** true when every diigo page is stored, so no match can be missing */
|
||||
complete: boolean;
|
||||
};
|
||||
|
||||
/** what anyone may see of a cache: stored pages, stored items by filter,
|
||||
* and fill progress */
|
||||
export class PageViewer extends RpcTarget {
|
||||
#reader: Reader;
|
||||
#fillStatus: () => Promise<FillStatus | undefined>;
|
||||
@@ -415,6 +428,25 @@ export class PageViewer extends RpcTarget {
|
||||
return stored && { ...stored, cached: true };
|
||||
}
|
||||
|
||||
/**
|
||||
* page `pageNum` (`count` per page) of the stored items matching
|
||||
* `query`, newest first by its date. The window narrows in SQL; terms
|
||||
* are checked per item. Reads storage only: items not yet stored can't
|
||||
* match, which `complete` reports.
|
||||
*/
|
||||
query(query: Query, pageNum: number, count: number): QueryPage {
|
||||
const from = query.after ? Math.ceil(query.after.getTime() / 1000) : Number.MIN_SAFE_INTEGER;
|
||||
const to = query.before ? Math.ceil(query.before.getTime() / 1000) : Number.MAX_SAFE_INTEGER;
|
||||
const hits = this.#reader.itemsIn(query.by, from, to).filter((json) => matches(JSON.parse(json) as Item, query));
|
||||
const total = this.#reader.knownTotal();
|
||||
const pages = total === null ? null : Math.max(1, Math.ceil(total / PAGE_SIZE));
|
||||
return {
|
||||
items: hits.slice((pageNum - 1) * count, pageNum * count),
|
||||
matched: hits.length,
|
||||
complete: pages !== null && this.#reader.pagesThrough(pages) >= pages,
|
||||
};
|
||||
}
|
||||
|
||||
/** diigo's total, as last stored, or null if nothing is stored */
|
||||
knownTotal(): number | null {
|
||||
return this.#reader.knownTotal();
|
||||
|
||||
+188
-51
@@ -10,9 +10,11 @@ import { CAPNWEB_BROWSER_JS } from "./capnweb-browser.js";
|
||||
import { CACHE_PAGE_HTML } from "./cache-page.js";
|
||||
import { DiigoCacheLogic, PublicCacheLogic } from "./cache.js";
|
||||
import { PAGE_SIZE } from "./cache.js";
|
||||
import type { CacheEnv, Clock, FillStatus } from "./cache.js";
|
||||
import type { CacheEnv, Clock, FillStatus, QueryPage } from "./cache.js";
|
||||
import type { Item } from "./cache.js";
|
||||
import { updatedISO, usernameOf } from "./items.js";
|
||||
import { usernameOf } from "./items.js";
|
||||
import { dateOf, isEmpty, parseQuery } from "./query.js";
|
||||
import type { Query } from "./query.js";
|
||||
import { toWebkey, idFromWebkey, isWebkey, shashKey } from "./webkeys.js";
|
||||
import { pmSave } from "./pm-save.js";
|
||||
|
||||
@@ -172,11 +174,13 @@ async function proxyMint(request: Request, env: Env): Promise<Response> {
|
||||
const detail = (await res.text()).slice(0, 200);
|
||||
return home(env, { error: `mint said ${res.status}: ${detail}` }, null);
|
||||
}
|
||||
// webapp answers {id} (+ x-webapp-id); the id rides the header, the
|
||||
// wire page is ours
|
||||
// webapp answers {id, storeId} (+ x-webapp-id, x-webapp-store-id);
|
||||
// the ids ride the headers, the wire page is ours
|
||||
const id = res.headers.get("x-webapp-id");
|
||||
if (!id) return home(env, { error: "mint produced no id" }, null);
|
||||
return await wirePage(id);
|
||||
const storeId = res.headers.get("x-webapp-store-id");
|
||||
if (!storeId) console.warn({ event: "mint", outcome: "no store id: this session cannot be rotated" });
|
||||
return await wirePage(id, storeId);
|
||||
} catch (err) {
|
||||
console.error({ event: "mint", error: String(err) });
|
||||
return home(env, { error: `mint failed: ${String(err)}` }, null);
|
||||
@@ -186,22 +190,37 @@ async function proxyMint(request: Request, env: Env): Promise<Response> {
|
||||
/** the credential-save helper, once per page (see ./pm-save.ts) */
|
||||
const PM_SAVE_JS = pmSave.toString();
|
||||
|
||||
/** the suggested petname gets the holder's local hh:mm (the worker only
|
||||
* knows UTC), so two saves the same day are told apart at a glance */
|
||||
const PETNAME_TIME_JS = `{
|
||||
const p = document.getElementById("petname");
|
||||
/** the suggested petname (in the input with id `petnameId`) gets the
|
||||
* holder's local hh:mm (the worker only knows UTC), so two saves the same
|
||||
* day are told apart at a glance */
|
||||
function petnameTimeJs(petnameId: string): string {
|
||||
return `{
|
||||
const p = document.getElementById(${JSON.stringify(petnameId)});
|
||||
const d = new Date();
|
||||
p.value += " " + String(d.getHours()).padStart(2, "0") + ":" + String(d.getMinutes()).padStart(2, "0");
|
||||
}`;
|
||||
|
||||
/** the onclick that hands pmSave its browser powers */
|
||||
function pmSaveClick(value: string): string {
|
||||
return `pmSave(this, ${JSON.stringify(value)}, { petname: document.getElementById("petname"), credentials: navigator.credentials, PasswordCredential: window.PasswordCredential, log: console })`;
|
||||
}
|
||||
|
||||
/** the wire step, with the freshly-minted webapp id pre-filled */
|
||||
async function wirePage(webappId: string): Promise<Response> {
|
||||
/** the onclick that hands pmSave its browser powers, saving under the
|
||||
* petname in the input with id `petnameId` */
|
||||
function pmSaveClick(value: string, petnameId: string): string {
|
||||
return `pmSave(this, ${JSON.stringify(value)}, { petname: document.getElementById(${JSON.stringify(petnameId)}), credentials: navigator.credentials, PasswordCredential: window.PasswordCredential, log: console })`;
|
||||
}
|
||||
|
||||
/** the wire step, with the freshly-minted webapp id pre-filled, and the
|
||||
* store id (rotating the cookie) for the account owner to keep */
|
||||
async function wirePage(webappId: string, storeId: string | null): Promise<Response> {
|
||||
const fingerprint = await nick(webappId);
|
||||
const storeSection = storeId
|
||||
? `<h2>keep: the cookie's store id</h2>
|
||||
<p>This one is yours alone, not the cache's: with it you can rotate the
|
||||
diigo session (log in again, paste the new cookie) or revoke it. Fingerprint
|
||||
<code>${esc(await nick(storeId))}</code>
|
||||
<button type="button" onclick='navigator.clipboard.writeText(${JSON.stringify(storeId)}).then(() => this.textContent = "copied ✓")'>copy</button></p>
|
||||
<p><label>your petname for it <input id="store-petname" autocomplete="off" size="30" value="${esc(`diigo cookie store ${await nick(storeId)}`)}"></label></p>
|
||||
<script>${petnameTimeJs("store-petname")}</script>
|
||||
<p><button type="button" onclick='${pmSaveClick(storeId, "store-petname")}'>save to password manager</button></p>`
|
||||
: `<p><b>note:</b> webapp sent no store id, so this session's cookie cannot be rotated; when it expires, mint again.</p>`;
|
||||
const body = `<!doctype html>
|
||||
<html><head><meta charset="utf-8"><title>diigo cache: step 2 — wire the cache</title></head>
|
||||
<body>
|
||||
@@ -210,8 +229,10 @@ async function wirePage(webappId: string): Promise<Response> {
|
||||
<button type="button" onclick='navigator.clipboard.writeText(${JSON.stringify(webappId)}).then(() => this.textContent = "copied ✓")'>copy</button></p>
|
||||
<p><label>your petname for it <input id="petname" autocomplete="off" size="30" value="${esc(`diigo webapp ${fingerprint}`)}"></label>
|
||||
<small>(private to you: the password-manager username; never sent to the server)</small></p>
|
||||
<script>${PETNAME_TIME_JS}</script>
|
||||
<p><button type="button" onclick='${pmSaveClick(webappId)}'>save to password manager</button></p>
|
||||
<script>${petnameTimeJs("petname")}</script>
|
||||
<p><button type="button" onclick='${pmSaveClick(webappId, "petname")}'>save to password manager</button></p>
|
||||
${storeSection}
|
||||
<h2>wire</h2>
|
||||
<p>choose the cache mode, then <b>wire</b> — the next page shows the
|
||||
<b>cache id</b> (copy that too), after which you land back on home:</p>
|
||||
<form method="post" action="/_wire">
|
||||
@@ -318,8 +339,8 @@ async function wiredPage(cacheId: string, note: string, hasPublic: boolean): Pro
|
||||
<button type="button" onclick='navigator.clipboard.writeText(${JSON.stringify(cacheId)}).then(() => this.textContent = "copied ✓")'>copy</button></p>
|
||||
<p><label>your petname for it <input id="petname" autocomplete="off" size="30" value="${esc(`diigo cache ${fingerprint}`)}"></label>
|
||||
<small>(private to you: the password-manager username; never sent to the server)</small></p>
|
||||
<script>${PETNAME_TIME_JS}</script>
|
||||
<p><button type="button" onclick='${pmSaveClick(cacheId)}'>save to password manager</button></p>
|
||||
<script>${petnameTimeJs("petname")}</script>
|
||||
<p><button type="button" onclick='${pmSaveClick(cacheId, "petname")}'>save to password manager</button></p>
|
||||
<p>${note}</p>
|
||||
<p><a href="/cache/#${esc(cacheId)}">open this cache</a> — its <b>webkey</b>
|
||||
(bookmark it, or save the id above): fill it, watch progress${hasPublic ? ", fill its public companion" : ""}.</p>
|
||||
@@ -443,10 +464,16 @@ async function dump(request: Request, env: Env): Promise<Response> {
|
||||
|
||||
const PUBLIC_PER_PAGE = PAGE_SIZE;
|
||||
|
||||
/** the public page's day presets */
|
||||
const DAY_PRESETS = [7, 30, 90, 365];
|
||||
|
||||
/**
|
||||
* the public page for a username: stored pages only, through the cache's
|
||||
* read-only `viewer()` (it can't fetch or write). HTML with a page-number
|
||||
* form, or JSON for `Accept: application/json`. `?page=` in the URL is
|
||||
* read-only `viewer()` (it can't fetch or write). HTML with a filter form
|
||||
* and a page-number form, or JSON for `Accept: application/json`. A
|
||||
* filter (goal 1d: `days`, `after`, `before`, `in`, `by`, `q`; see
|
||||
* {@link parseQuery}) lists the stored items that match, `page` counting
|
||||
* matches; with none, `page` is diigo's page as stored. Query params are
|
||||
* fine here: nothing on this page is secret.
|
||||
*/
|
||||
async function publicPage(request: Request, env: Env, username: string): Promise<Response> {
|
||||
@@ -456,14 +483,33 @@ async function publicPage(request: Request, env: Env, username: string): Promise
|
||||
const ns = env.PUBLIC_DO;
|
||||
const cache = ns.get(ns.idFromString(idFromWebkey(publicId)));
|
||||
|
||||
const raw = new URL(request.url).searchParams.get("page");
|
||||
const params = new URL(request.url).searchParams;
|
||||
const accept = request.headers.get("accept") ?? "";
|
||||
const wantsJson = accept.includes("application/json") && !accept.includes("text/html");
|
||||
const here = `/public/${encodeURIComponent(username)}`;
|
||||
const raw = params.get("page");
|
||||
const pageN = raw === null ? 1 : Number.parseInt(raw, 10);
|
||||
const valid = !Number.isNaN(pageN) && pageN >= 1 && pageN <= 10000;
|
||||
|
||||
const query = parseQuery(params, new Date());
|
||||
if ("error" in query) {
|
||||
console.log({ event: "notice", page: "public", text: "filter refused", error: query.error });
|
||||
if (wantsJson) return Response.json({ error: query.error }, { status: 400 });
|
||||
const body = publicFrame(username, `${filterForm(here, params)}
|
||||
<p><em>${esc(query.error)}</em></p>`);
|
||||
return new Response(body, { status: 400, headers: { "content-type": "text/html;charset=utf-8" } });
|
||||
}
|
||||
|
||||
const viewer = await cache.viewer();
|
||||
let page, total, fill;
|
||||
let page, fill, found;
|
||||
let total: number | null = null;
|
||||
try {
|
||||
page = await viewer.page(valid ? pageN : 1, PUBLIC_PER_PAGE);
|
||||
total = page ? page.total : await viewer.knownTotal();
|
||||
if (isEmpty(query)) {
|
||||
page = await viewer.page(valid ? pageN : 1, PUBLIC_PER_PAGE);
|
||||
total = page ? page.total : await viewer.knownTotal();
|
||||
} else {
|
||||
found = await viewer.query(query, valid ? pageN : 1, PUBLIC_PER_PAGE);
|
||||
}
|
||||
fill = await viewer.fillStatus();
|
||||
} catch (err) {
|
||||
console.error({ event: "public-page", page: pageN, error: String(err) });
|
||||
@@ -471,13 +517,14 @@ async function publicPage(request: Request, env: Env, username: string): Promise
|
||||
} finally {
|
||||
viewer[Symbol.dispose]();
|
||||
}
|
||||
if (found) return filteredPage(username, here, params, query, found, valid ? pageN : 1, wantsJson, fill);
|
||||
|
||||
// total and pages count all of diigo's items: public page n is diigo
|
||||
// page n with the private rows dropped
|
||||
const pages = total === null ? 1 : Math.max(1, Math.ceil(total / PUBLIC_PER_PAGE));
|
||||
const items = page ? page.items.map((j) => JSON.parse(j) as Item) : [];
|
||||
|
||||
const accept = request.headers.get("accept") ?? "";
|
||||
if (accept.includes("application/json") && !accept.includes("text/html")) {
|
||||
if (wantsJson) {
|
||||
return Response.json({ username, page: pageN, pages, total, stored: page !== null, etchedAt: page?.etchedAt ?? null, items });
|
||||
}
|
||||
|
||||
@@ -486,42 +533,132 @@ async function publicPage(request: Request, env: Env, username: string): Promise
|
||||
console.log({ event: "notice", page: "public", text: "page out of range", pages });
|
||||
notice = `<p><em>no page ${esc(String(raw))}: pages run 1–${pages}</em></p>`;
|
||||
}
|
||||
const here = `/public/${encodeURIComponent(username)}`;
|
||||
const nav = (n: number, label: string) =>
|
||||
n >= 1 && n <= pages ? `<a href="${here}?page=${n}">${label}</a>` : `<span>${label}</span>`;
|
||||
const list = notice
|
||||
? ""
|
||||
: `<ol>
|
||||
${items
|
||||
.map((it) => {
|
||||
const url = typeof it.url === "string" ? it.url : "";
|
||||
const title = typeof it.title === "string" && it.title ? it.title : url;
|
||||
const desc = typeof it.description === "string" && it.description ? `<br><small>${esc(it.description)}</small>` : "";
|
||||
const href = /^https?:\/\//i.test(url) ? url : "#";
|
||||
const when = updatedISO(it);
|
||||
const date = when ? `<time datetime="${when}">${when}</time>` : "(no date)";
|
||||
return `<li>${date} <a href="${esc(href)}">${esc(title)}</a>${desc}</li>`;
|
||||
})
|
||||
.join("\n")}
|
||||
</ol>`;
|
||||
const body = `<!doctype html>
|
||||
<html><head><meta charset="utf-8"><title>${esc(username)}'s shared bookmarks</title></head>
|
||||
<body>
|
||||
<h1>${esc(username)}'s shared bookmarks</h1>
|
||||
<p>${pages} page${pages === 1 ? "" : "s"} (diigo's, ${PUBLIC_PER_PAGE} bookmarks each; private ones are left out, so a page may show fewer).</p>
|
||||
n >= 1 && n <= pages ? `<a rel="${n < pageN ? "prev" : "next"}" href="${here}?page=${n}">${label}</a>` : `<span>${label}</span>`;
|
||||
const list = notice ? "" : itemList(items, "updated", here);
|
||||
const body = publicFrame(
|
||||
username,
|
||||
`<p>${pages} page${pages === 1 ? "" : "s"} (diigo's, ${PUBLIC_PER_PAGE} bookmarks each; private ones are left out, so a page may show fewer).</p>
|
||||
<p>${page ? `<strong>stored ${esc(page.etchedAt ?? "")}</strong> — page ${valid ? pageN : 1}, ${items.length} shared` : `<strong>not fetched yet</strong> — page ${valid ? pageN : 1}; the cache's owner fills it`}</p>
|
||||
${fill ? fillLine(fill) : ""}
|
||||
${filterForm(here, params)}
|
||||
<form method="get" action="${here}">
|
||||
<p>${nav(pageN - 1, "← prev")}
|
||||
<label>page <input type="number" name="page" min="1" value="${Number.isNaN(pageN) ? 1 : pageN}"></label>
|
||||
<button type="submit">go</button>
|
||||
${nav(pageN + 1, "next →")}</p>
|
||||
</form>
|
||||
${notice}${list}
|
||||
</body></html>`;
|
||||
${notice}${list}`,
|
||||
);
|
||||
return new Response(body, { headers: { "content-type": "text/html;charset=utf-8" } });
|
||||
}
|
||||
|
||||
/** a filter's matches, as the public page's HTML or JSON */
|
||||
function filteredPage(
|
||||
username: string,
|
||||
here: string,
|
||||
params: URLSearchParams,
|
||||
query: Query,
|
||||
found: QueryPage,
|
||||
pageN: number,
|
||||
wantsJson: boolean,
|
||||
fill: FillStatus | undefined,
|
||||
): Response {
|
||||
const pages = Math.max(1, Math.ceil(found.matched / PUBLIC_PER_PAGE));
|
||||
const items = found.items.map((j) => JSON.parse(j) as Item);
|
||||
if (wantsJson) {
|
||||
const { matched, complete } = found;
|
||||
return Response.json({ username, filter: query, page: pageN, pages, matched, complete, items });
|
||||
}
|
||||
// links keep the filter: the request's own non-empty params, in order
|
||||
const kept = [...params].filter(([k, v]) => k !== "page" && v !== "");
|
||||
const href = (n: number) => `${here}?${new URLSearchParams([...kept, ["page", String(n)]])}`;
|
||||
const nav = (n: number, label: string) =>
|
||||
n >= 1 && n <= pages ? `<a rel="${n < pageN ? "prev" : "next"}" href="${esc(href(n))}">${label}</a>` : `<span>${label}</span>`;
|
||||
const status = found.matched
|
||||
? `${found.matched} match${found.matched === 1 ? "" : "es"}, page ${pageN} of ${pages}, newest ${query.by} first`
|
||||
: "no matches";
|
||||
const caveat = found.complete ? "" : " — not every diigo page is stored yet, so some matches may be missing";
|
||||
const body = publicFrame(
|
||||
username,
|
||||
`<p><strong>${esc(status)}</strong>${caveat}</p>
|
||||
${fill ? fillLine(fill) : ""}
|
||||
${filterForm(here, params)}
|
||||
<p>${nav(pageN - 1, "← prev")} ${nav(pageN + 1, "next →")} <a href="${here}">all, by diigo page</a></p>
|
||||
${itemList(items, query.by, here)}`,
|
||||
);
|
||||
return new Response(body, { headers: { "content-type": "text/html;charset=utf-8" } });
|
||||
}
|
||||
|
||||
/** the public page's frame: title and heading around `content` */
|
||||
function publicFrame(username: string, content: string): string {
|
||||
return `<!doctype html>
|
||||
<html><head><meta charset="utf-8"><title>${esc(username)}'s shared bookmarks</title></head>
|
||||
<body>
|
||||
<h1>${esc(username)}'s shared bookmarks</h1>
|
||||
${content}
|
||||
</body></html>`;
|
||||
}
|
||||
|
||||
/** the filter form, echoing the request's filter */
|
||||
function filterForm(here: string, params: URLSearchParams): string {
|
||||
const field = (name: string) => esc(params.get(name) ?? "");
|
||||
const days = params.get("days") ?? "";
|
||||
const presets = DAY_PRESETS.map(String);
|
||||
if (days && !presets.includes(days)) presets.push(days);
|
||||
const option = (value: string, label: string, current: string) =>
|
||||
`<option value="${esc(value)}"${value === current ? " selected" : ""}>${esc(label)}</option>`;
|
||||
return `<form method="get" action="${here}">
|
||||
<p><label>last <select name="days">${option("", "any", days)}${presets.map((n) => option(n, `${n} days`, days)).join("")}</select></label>
|
||||
<label>or after <input type="date" name="after" value="${field("after")}"></label>
|
||||
<label>before <input type="date" name="before" value="${field("before")}"></label>
|
||||
<label>or in <input name="in" size="9" placeholder="2026-W41" value="${field("in")}"></label>
|
||||
<label>by <select name="by">${option("updated", "updated", params.get("by") ?? "")}${option("created", "created", params.get("by") ?? "")}</select></label></p>
|
||||
<p><label>search <input type="search" name="q" size="40" value="${field("q")}"></label>
|
||||
<button type="submit">filter</button></p>
|
||||
<p><small>Dates and periods (2026, 2026-03, 2026-Q1, 2026-W41) are UTC. Search takes
|
||||
words, "phrases", #tags, -negation, and after:, before:, newer_than:7d, during:2026-Q1, created:2026.</small></p>
|
||||
</form>`;
|
||||
}
|
||||
|
||||
/**
|
||||
* bookmarks as a list, each led by its `by` date, marked up as
|
||||
* microformats: an `h-feed` of `h-entry` bookmarks (`u-bookmark-of`,
|
||||
* `p-name`, `p-summary`, `p-category` per tag, `dt-published` when
|
||||
* bookmarked, `dt-updated` when last edited). This markup is the public
|
||||
* page's API for other sites (tests/caches.test.ts pins it); tag links
|
||||
* search this page (`here`) by tag.
|
||||
*
|
||||
* @see https://microformats.org/wiki/h-entry
|
||||
* @see https://indieweb.org/bookmark
|
||||
*/
|
||||
function itemList(items: Item[], by: Query["by"], here: string): string {
|
||||
const time = (it: Item, which: Query["by"], hidden: boolean) => {
|
||||
const secs = dateOf(it, which);
|
||||
if (secs === undefined) return hidden ? "" : "(no date)";
|
||||
const when = new Date(secs * 1000).toISOString();
|
||||
const cls = which === "created" ? "dt-published" : "dt-updated";
|
||||
return hidden ? `<time class="${cls}" datetime="${when}" hidden></time>` : `<time class="${cls}" datetime="${when}">${when}</time>`;
|
||||
};
|
||||
return `<ol class="h-feed">
|
||||
${items
|
||||
.map((it) => {
|
||||
const url = typeof it.url === "string" ? it.url : "";
|
||||
const title = typeof it.title === "string" && it.title ? it.title : url;
|
||||
const desc = typeof it.description === "string" && it.description ? `<br><small class="p-summary">${esc(it.description)}</small>` : "";
|
||||
const href = /^https?:\/\//i.test(url) ? url : "#";
|
||||
const tags = (typeof it.tags === "string" ? it.tags.split(",") : [])
|
||||
.map((t) => t.trim())
|
||||
.filter(Boolean)
|
||||
.map((t) => ` <a class="p-category" href="${esc(`${here}?q=${encodeURIComponent(`#${t}`)}`)}">${esc(t)}</a>`)
|
||||
.join("");
|
||||
const other = by === "created" ? "updated" : "created";
|
||||
return `<li class="h-entry">${time(it, by, false)} <a class="u-bookmark-of p-name" href="${esc(href)}">${esc(title)}</a>${tags}${desc}${time(it, other, true)}</li>`;
|
||||
})
|
||||
.join("\n")}
|
||||
</ol>`;
|
||||
}
|
||||
|
||||
/** home page: the connect link (webapp's cookie form, diigo-prefilled) +
|
||||
* the wire step for an already-minted id + the dump form. `home` renders
|
||||
* the error path too, so it takes env for the connect link. Notices (the
|
||||
|
||||
+188
-19
@@ -1,18 +1,10 @@
|
||||
/**
|
||||
* @file the public page's filters: date windows and search (goal 1d, **not
|
||||
* yet implemented**; the design is the skipped specs in tests/query.test.ts).
|
||||
* @file the public page's filters: date windows and search over a cache's
|
||||
* stored items (goal 1d; tests/query.test.ts).
|
||||
*/
|
||||
|
||||
import type { Item } from "./items.js";
|
||||
|
||||
/**
|
||||
* diigo fetches one request may spend (cache hits are free); under the
|
||||
* Workers free plan's 50 external subrequests per invocation
|
||||
*
|
||||
* @see https://developers.cloudflare.com/workers/platform/limits/
|
||||
*/
|
||||
export const FETCH_BUDGET = 40;
|
||||
|
||||
/**
|
||||
* a filter as devs write it (`{ days: 7, q: "#weekly" }`); URL params map
|
||||
* onto these fields one-to-one (strings, with "" meaning absent). Dates
|
||||
@@ -56,6 +48,76 @@ export type Query = {
|
||||
/** a refused filter: the message is safe to show and log */
|
||||
export type QueryError = { error: string };
|
||||
|
||||
const DAY_MS = 24 * 60 * 60 * 1000;
|
||||
|
||||
/** thrown inside {@link toQuery}, returned as a {@link QueryError} */
|
||||
class Refused extends Error {}
|
||||
|
||||
/** `yyyy-mm-dd` or `yyyy/mm/dd` as 00:00Z of that UTC day */
|
||||
function parseDay(text: string, what: string): Date {
|
||||
const m = /^(\d{4})([-/])(\d{2})\2(\d{2})$/.exec(text);
|
||||
const d = m && new Date(Date.UTC(Number(m[1]), Number(m[3]) - 1, Number(m[4])));
|
||||
if (!m || !d || d.getUTCMonth() !== Number(m[3]) - 1 || d.getUTCDate() !== Number(m[4])) {
|
||||
throw new Refused(`${what}: expected a date as yyyy-mm-dd`);
|
||||
}
|
||||
return d;
|
||||
}
|
||||
|
||||
/** Monday 00:00Z of ISO week 1 of `year` (the week holding January 4) */
|
||||
function isoWeekOne(year: number): number {
|
||||
const jan4 = Date.UTC(year, 0, 4);
|
||||
const weekday = new Date(jan4).getUTCDay() || 7;
|
||||
return jan4 - (weekday - 1) * DAY_MS;
|
||||
}
|
||||
|
||||
/** a named period as its half-open [after, before) window */
|
||||
function parsePeriod(text: string, what: string): [Date, Date] {
|
||||
const refuse = () => new Refused(`${what}: expected a period as 2025, 2025-03, 2025-Q1, or 2025-W01`);
|
||||
let m;
|
||||
if ((m = /^(\d{4})$/.exec(text))) {
|
||||
const y = Number(m[1]);
|
||||
return [new Date(Date.UTC(y, 0, 1)), new Date(Date.UTC(y + 1, 0, 1))];
|
||||
}
|
||||
if ((m = /^(\d{4})-(\d{2})$/.exec(text))) {
|
||||
const [y, mo] = [Number(m[1]), Number(m[2])];
|
||||
if (mo < 1 || mo > 12) throw refuse();
|
||||
return [new Date(Date.UTC(y, mo - 1, 1)), new Date(Date.UTC(y, mo, 1))];
|
||||
}
|
||||
if ((m = /^(\d{4})-Q([1-4])$/.exec(text))) {
|
||||
const [y, q] = [Number(m[1]), Number(m[2])];
|
||||
return [new Date(Date.UTC(y, 3 * (q - 1), 1)), new Date(Date.UTC(y, 3 * q, 1))];
|
||||
}
|
||||
if ((m = /^(\d{4})-W(\d{2})$/.exec(text))) {
|
||||
const [y, w] = [Number(m[1]), Number(m[2])];
|
||||
const weeks = (isoWeekOne(y + 1) - isoWeekOne(y)) / (7 * DAY_MS);
|
||||
if (w < 1 || w > weeks) throw refuse();
|
||||
const monday = isoWeekOne(y) + (w - 1) * 7 * DAY_MS;
|
||||
return [new Date(monday), new Date(monday + 7 * DAY_MS)];
|
||||
}
|
||||
throw refuse();
|
||||
}
|
||||
|
||||
/** gmail's `newer_than:` amount (`7d`, `2w`, `1m`, `1y`) counted back from
|
||||
* `now`; months and years are calendar steps */
|
||||
function parseNewerThan(text: string, now: Date): Date {
|
||||
const m = /^(\d{1,4})([dwmy])$/.exec(text);
|
||||
if (!m || Number(m[1]) < 1) throw new Refused("newer_than: expected an amount like 7d, 2w, 1m, or 1y");
|
||||
const n = Number(m[1]);
|
||||
const d = new Date(now.getTime());
|
||||
if (m[2] === "d") d.setTime(now.getTime() - n * DAY_MS);
|
||||
if (m[2] === "w") d.setTime(now.getTime() - 7 * n * DAY_MS);
|
||||
if (m[2] === "m") d.setUTCMonth(d.getUTCMonth() - n);
|
||||
if (m[2] === "y") d.setUTCFullYear(d.getUTCFullYear() - n);
|
||||
return d;
|
||||
}
|
||||
|
||||
/** `days` as a whole number from 1 to 3650 */
|
||||
function parseDays(days: number | string): number {
|
||||
const n = typeof days === "number" ? days : /^\d+$/.test(days) ? Number(days) : NaN;
|
||||
if (!Number.isInteger(n) || n < 1 || n > 3650) throw new Refused("days: expected a whole number from 1 to 3650");
|
||||
return n;
|
||||
}
|
||||
|
||||
/**
|
||||
* resolve a filter: fields and `q` operators combine; a bound given twice
|
||||
* (by fields, operators, or both) is refused, as is an empty window.
|
||||
@@ -75,21 +137,128 @@ export type QueryError = { error: string };
|
||||
* not read ambiently, so tests pin it)
|
||||
*/
|
||||
export function toQuery(spec: FilterSpec, now: Date): Query | QueryError {
|
||||
void spec;
|
||||
void now;
|
||||
throw new Error("toQuery: not implemented (goal 1d)");
|
||||
try {
|
||||
return resolve(spec, now);
|
||||
} catch (err) {
|
||||
if (err instanceof Refused) return { error: err.message };
|
||||
throw err;
|
||||
}
|
||||
}
|
||||
|
||||
function resolve(spec: FilterSpec, now: Date): Query {
|
||||
const words: Term[] = [];
|
||||
const tags: Term[] = [];
|
||||
let by: { value: Query["by"]; from: string } | undefined;
|
||||
let after: { value: Date; from: string } | undefined;
|
||||
let before: { value: Date; from: string } | undefined;
|
||||
|
||||
const setBy = (value: Query["by"], from: string) => {
|
||||
if (by && by.value !== value) throw new Refused(`${by.from} and ${from} name different dates`);
|
||||
by = { value, from };
|
||||
};
|
||||
const setAfter = (value: Date, from: string) => {
|
||||
if (after) throw new Refused(`${after.from} and ${from} both set where the window starts`);
|
||||
after = { value, from };
|
||||
};
|
||||
const setBefore = (value: Date, from: string) => {
|
||||
if (before) throw new Refused(`${before.from} and ${from} both set where the window ends`);
|
||||
before = { value, from };
|
||||
};
|
||||
const setPeriod = ([lo, hi]: [Date, Date], from: string) => {
|
||||
setAfter(lo, from);
|
||||
setBefore(hi, from);
|
||||
};
|
||||
|
||||
const present = (v: string | number | undefined) => v !== undefined && v !== "";
|
||||
if (present(spec.by)) {
|
||||
if (spec.by !== "updated" && spec.by !== "created") throw new Refused("by: expected updated or created");
|
||||
setBy(spec.by, `by=${spec.by}`);
|
||||
}
|
||||
if (present(spec.days)) setAfter(new Date(now.getTime() - parseDays(spec.days!) * DAY_MS), "days");
|
||||
if (present(spec.after)) setAfter(parseDay(spec.after!, "after"), "after");
|
||||
if (present(spec.before)) setBefore(parseDay(spec.before!, "before"), "before");
|
||||
if (present(spec.in)) setPeriod(parsePeriod(spec.in!, "in"), "in");
|
||||
|
||||
for (const m of (spec.q ?? "").matchAll(/(-?)"([^"]*)"|(\S+)/g)) {
|
||||
if (m[2] !== undefined) {
|
||||
if (m[2].trim()) words.push({ text: m[2].trim().toLowerCase(), negated: m[1] === "-" });
|
||||
continue;
|
||||
}
|
||||
const token = m[3];
|
||||
const negated = token.length > 1 && token.startsWith("-");
|
||||
const body = negated ? token.slice(1) : token;
|
||||
const op = /^([a-z_]+):(.+)$/i.exec(body);
|
||||
const name = op?.[1].toLowerCase();
|
||||
const arg = op?.[2] ?? "";
|
||||
const windowOp = ["after", "before", "newer_than", "during", "in", "created", "updated"];
|
||||
if (name && windowOp.includes(name) && negated) throw new Refused(`${name}: can't be negated`);
|
||||
if (name === "after") setAfter(parseDay(arg, "after:"), "after:");
|
||||
else if (name === "before") setBefore(parseDay(arg, "before:"), "before:");
|
||||
else if (name === "newer_than") setAfter(parseNewerThan(arg, now), "newer_than:");
|
||||
else if (name === "during" || name === "in") setPeriod(parsePeriod(arg, `${name}:`), `${name}:`);
|
||||
else if (name === "created" || name === "updated") {
|
||||
setBy(name, `${name}:`);
|
||||
setPeriod(parsePeriod(arg, `${name}:`), `${name}:`);
|
||||
} else if (name === "tag") tags.push({ text: arg.toLowerCase(), negated });
|
||||
else if (body.length > 1 && body.startsWith("#")) tags.push({ text: body.slice(1).toLowerCase(), negated });
|
||||
else words.push({ text: body.toLowerCase(), negated });
|
||||
}
|
||||
|
||||
if (after && before && after.value.getTime() >= before.value.getTime()) {
|
||||
throw new Refused(`the window is empty: ${after.from} is not earlier than ${before.from}`);
|
||||
}
|
||||
const query: Query = { by: by?.value ?? "updated", words, tags };
|
||||
if (after) query.after = after.value;
|
||||
if (before) query.before = before.value;
|
||||
return query;
|
||||
}
|
||||
|
||||
/** {@link toQuery} over the public page's URL params */
|
||||
export function parseQuery(params: URLSearchParams, now: Date): Query | QueryError {
|
||||
void params;
|
||||
void now;
|
||||
throw new Error("parseQuery: not implemented (goal 1d)");
|
||||
const field = (name: string) => params.get(name) ?? "";
|
||||
const by = field("by");
|
||||
return toQuery(
|
||||
{
|
||||
days: field("days"),
|
||||
after: field("after"),
|
||||
before: field("before"),
|
||||
in: field("in"),
|
||||
by: by as FilterSpec["by"],
|
||||
q: field("q"),
|
||||
},
|
||||
now,
|
||||
);
|
||||
}
|
||||
|
||||
/** does the query filter anything? (an empty one lists diigo's pages as stored) */
|
||||
export function isEmpty(query: Query): boolean {
|
||||
return !query.after && !query.before && !query.words.length && !query.tags.length && query.by === "updated";
|
||||
}
|
||||
|
||||
/** the item's date that `by` names, in epoch seconds (diigo's unit), or
|
||||
* undefined when it carries none */
|
||||
export function dateOf(item: Item, by: Query["by"]): number | undefined {
|
||||
const secs = by === "created" ? item.created_at : item.updated_at;
|
||||
return typeof secs === "number" && Number.isFinite(secs) ? secs : undefined;
|
||||
}
|
||||
|
||||
/** does `item` fall in the window and satisfy every term? */
|
||||
export function matches(item: Item, query: Query): boolean {
|
||||
void item;
|
||||
void query;
|
||||
throw new Error("matches: not implemented (goal 1d)");
|
||||
if (query.after || query.before) {
|
||||
const secs = dateOf(item, query.by);
|
||||
if (secs === undefined) return false;
|
||||
if (query.after && secs * 1000 < query.after.getTime()) return false;
|
||||
if (query.before && secs * 1000 >= query.before.getTime()) return false;
|
||||
}
|
||||
const text = (v: unknown) => (typeof v === "string" ? v : "");
|
||||
// fields joined by newline, so a phrase never spans two of them
|
||||
const haystack = [item.title, item.description, item.url].map(text).join("\n").toLowerCase();
|
||||
if (!query.words.every((w) => haystack.includes(w.text) !== w.negated)) return false;
|
||||
const itemTags = new Set(
|
||||
text(item.tags)
|
||||
.split(",")
|
||||
.map((t) => t.trim().toLowerCase())
|
||||
.filter(Boolean),
|
||||
);
|
||||
return query.tags.every((t) => itemTags.has(t.text) !== t.negated);
|
||||
}
|
||||
@@ -41,6 +41,12 @@ export function makeReader(sql: Sql) {
|
||||
deepestPageSince(cutoff: number): number | null {
|
||||
return run<{ last: number | null }>("deepestPageSince", cutoff)[0]?.last ?? null;
|
||||
},
|
||||
/** stored items whose `by` date (epoch seconds) is in [`from`, `to`),
|
||||
* newest first; the raw dump rows (JSON strings) */
|
||||
itemsIn(by: "updated" | "created", from: number, to: number): string[] {
|
||||
const name = by === "created" ? "itemsByCreated" : "itemsByUpdated";
|
||||
return run<{ json: string }>(name, from, to).map((r) => r.json);
|
||||
},
|
||||
/** how many cached pages are numbered `pageNum` or lower */
|
||||
pagesThrough(pageNum: number): number {
|
||||
return Number(run<{ n: number }>("pagesThrough", pageNum)[0]?.n ?? 0);
|
||||
|
||||
@@ -3,8 +3,8 @@ import { evictDurableObject, runInDurableObject } from "cloudflare:test";
|
||||
import { describe, expect, it, vi } from "vitest";
|
||||
import worker from "../src/index.js";
|
||||
import { idFromWebkey, toWebkey } from "../src/webkeys.js";
|
||||
import { FETCH_BUDGET } from "../src/query.js";
|
||||
import { PAGE_SIZE } from "../src/cache.js";
|
||||
import { mf2 } from "microformats-parser";
|
||||
|
||||
// matches tests/fake-webapp.js: 30 items, every third private
|
||||
const WEBAPP_ID = "fake-webapp-id";
|
||||
@@ -250,7 +250,7 @@ describe("worker routes", () => {
|
||||
// PAGE_SIZE 100: page 2 is bookmarks 101..200
|
||||
expect(html).toContain(">bookmark 130<");
|
||||
// the date comes first, in toISOString() form
|
||||
expect(html).toMatch(/<li><time datetime="\d{4}-\d\d-\d\dT[\d:.]+Z">[^<]+<\/time> <a /);
|
||||
expect(html).toMatch(/<li class="h-entry"><time class="dt-updated" datetime="\d{4}-\d\d-\d\dT[\d:.]+Z">[^<]+<\/time> <a /);
|
||||
expect(html).not.toContain(">bookmark 129<"); // private (129 % 3 === 0)
|
||||
|
||||
const json = (await (
|
||||
@@ -283,30 +283,34 @@ function isoWeek(d: Date): string {
|
||||
return `${year}-W${String(week).padStart(2, "0")}`;
|
||||
}
|
||||
|
||||
describe.skip("public page filters (goal 1d)", () => {
|
||||
describe("public page filters (goal 1d)", () => {
|
||||
// the fake's dated account: "dated n" updated n-1 days + 1h ago;
|
||||
// #weekly on every 7th; every 10th private; every 5th created 100
|
||||
// days before it was updated
|
||||
// days before it was updated. The public page only reads, so each
|
||||
// test stores the pages it needs on the owner's side first.
|
||||
const call = (path: string, init?: RequestInit) =>
|
||||
worker.fetch(new Request(`https://diigo-cache.test${path}`, init), env as never);
|
||||
const wireDated = async () => {
|
||||
/** wire a public cache to `webappId` and store diigo pages 1..`pages` */
|
||||
const wireStored = async (webappId: string, pages: number) => {
|
||||
const res = await call("/_wire", {
|
||||
method: "POST",
|
||||
headers: { "content-type": "application/json" },
|
||||
body: JSON.stringify({ webappId: "fake-dated-webapp", mode: "public" }),
|
||||
body: JSON.stringify({ webappId, mode: "public" }),
|
||||
});
|
||||
expect(res.status).toBe(200);
|
||||
return "/public/dated_dan";
|
||||
const { username, publicId } = (await res.json()) as { username: string; publicId: string };
|
||||
const pub = testEnv.PUBLIC_DO.get(testEnv.PUBLIC_DO.idFromString(idFromWebkey(publicId)));
|
||||
for (let n = 1; n <= pages; n += 1) await pub.page(n, PAGE_SIZE);
|
||||
return { page: `/public/${username}`, pub };
|
||||
};
|
||||
const wireDated = async () => (await wireStored("fake-dated-webapp", 1)).page;
|
||||
const titlesIn = (html: string) => [...html.matchAll(/>(dated \d+)<\/a>/g)].map((m) => m[1]);
|
||||
const dated = (...ns: number[]) => ns.map((n) => `dated ${n}`);
|
||||
|
||||
it("?days=7 lists the last week, walking only the diigo pages it needs", async () => {
|
||||
it("?days=7 lists the last week", async () => {
|
||||
const page = await wireDated();
|
||||
const before = await testEnv.WEBAPP.calls();
|
||||
const html = await (await call(`${page}?days=7`)).text();
|
||||
expect(titlesIn(html)).toEqual(dated(1, 2, 3, 4, 5, 6, 7));
|
||||
expect((await testEnv.WEBAPP.calls()) - before).toBeLessThanOrEqual(1);
|
||||
});
|
||||
|
||||
it("?by=created windows, lists, and dates by when it was bookmarked", async () => {
|
||||
@@ -327,13 +331,7 @@ describe.skip("public page filters (goal 1d)", () => {
|
||||
});
|
||||
|
||||
it("?days=300 pages the matches PAGE_SIZE at a time; links keep the filter", async () => {
|
||||
const res = await call("/_wire", {
|
||||
method: "POST",
|
||||
headers: { "content-type": "application/json" },
|
||||
body: JSON.stringify({ webappId: "fake-deep-webapp", mode: "public" }),
|
||||
});
|
||||
expect(res.status).toBe(200);
|
||||
const page = "/public/deep_dee";
|
||||
const { page } = await wireStored("fake-deep-webapp", 3);
|
||||
// deep_dee: one a day; days=300 holds 1..300, minus every 10th (private)
|
||||
const want = Array.from({ length: 300 }, (_, k) => k + 1).filter((n) => n % 10 !== 0);
|
||||
const first = await (await call(`${page}?days=300`)).text();
|
||||
@@ -354,6 +352,15 @@ describe.skip("public page filters (goal 1d)", () => {
|
||||
expect(titlesIn(html)).toEqual(dated(11, 12));
|
||||
});
|
||||
|
||||
it("?in= names an ISO week", async () => {
|
||||
const { page } = await wireStored("fake-deep-webapp", 3);
|
||||
const week = isoWeek(new Date(Date.now() - 100 * 24 * 60 * 60 * 1000));
|
||||
const html = await (await call(`${page}?in=${week}`)).text();
|
||||
// one a day; every 10th is private
|
||||
expect(titlesIn(html).length).toBeGreaterThanOrEqual(6);
|
||||
expect(titlesIn(html).length).toBeLessThanOrEqual(7);
|
||||
});
|
||||
|
||||
it("?q=#weekly&days=60 filters by tag; private items stay out", async () => {
|
||||
const page = await wireDated();
|
||||
const html = await (await call(`${page}?days=60&q=${encodeURIComponent("#weekly")}`)).text();
|
||||
@@ -366,22 +373,49 @@ describe.skip("public page filters (goal 1d)", () => {
|
||||
expect(titlesIn(html)).toEqual(dated(7, 14, 21, 28, 35, 42, 49, 56));
|
||||
});
|
||||
|
||||
it("a request spends at most FETCH_BUDGET diigo fetches; incomplete results say so, and a reload continues", async () => {
|
||||
const res = await call("/_wire", {
|
||||
method: "POST",
|
||||
headers: { "content-type": "application/json" },
|
||||
body: JSON.stringify({ webappId: "fake-deep-webapp", mode: "public" }),
|
||||
});
|
||||
expect(res.status).toBe(200);
|
||||
// deep_dee: 60 diigo pages of 100; a search matching nothing must walk them all
|
||||
it("a filter reads only stored items, never diigo; results say whether storage is complete", async () => {
|
||||
const { page, pub } = await wireStored("fake-deep-webapp", 1);
|
||||
const json = async () =>
|
||||
(await call(`${page}?days=300`, { headers: { accept: "application/json" } })).json() as Promise<{
|
||||
matched: number;
|
||||
complete: boolean;
|
||||
}>;
|
||||
const before = await testEnv.WEBAPP.calls();
|
||||
const first = await call("/public/deep_dee?q=nomatch", { headers: { accept: "application/json" } });
|
||||
expect(await first.json()).toMatchObject({ complete: false, items: [] });
|
||||
expect((await testEnv.WEBAPP.calls()) - before).toBeLessThanOrEqual(FETCH_BUDGET + 1);
|
||||
const html = await (await call("/public/deep_dee?q=nomatch")).text();
|
||||
expect(html).toMatch(/complete|no matches/);
|
||||
const again = await call("/public/deep_dee?q=nomatch", { headers: { accept: "application/json" } });
|
||||
expect(await again.json()).toMatchObject({ complete: true, items: [] });
|
||||
// page 1 holds 1..100, minus 10 private
|
||||
expect(await json()).toMatchObject({ matched: 90, complete: false });
|
||||
expect(await (await call(`${page}?days=300`)).text()).toMatch(/some matches may be missing/);
|
||||
expect(await testEnv.WEBAPP.calls()).toBe(before);
|
||||
await pub.page(2, PAGE_SIZE);
|
||||
await pub.page(3, PAGE_SIZE);
|
||||
expect(await json()).toMatchObject({ matched: 270, complete: true });
|
||||
});
|
||||
|
||||
it("the page is an h-feed of h-entry bookmarks: microformats are its API for other sites", async () => {
|
||||
const page = await wireDated();
|
||||
const html = await (await call(`${page}?days=7&by=created`)).text();
|
||||
const base = `https://diigo-cache.test${page}`;
|
||||
const { items, rels } = mf2(html, { baseUrl: base });
|
||||
const feed = items.find((i) => i.type?.includes("h-feed"));
|
||||
const entries = feed?.children ?? [];
|
||||
expect(entries.map((e) => e.properties.name?.[0])).toEqual(dated(1, 2, 3, 4, 6, 7));
|
||||
const seven = entries[5].properties;
|
||||
expect(seven["bookmark-of"]).toEqual(["https://dated.example/7"]);
|
||||
expect(seven.summary).toEqual(["a week of fragments"]);
|
||||
expect(seven.category).toEqual(["ocap", "weekly"]);
|
||||
const published = new Date(String(seven.published?.[0])).getTime();
|
||||
const updated = new Date(String(seven.updated?.[0])).getTime();
|
||||
// 7 isn't a 5: created when updated, 6 days + 1h ago
|
||||
expect(published).toBe(updated);
|
||||
expect(Date.now() - published).toBeGreaterThan(6 * 24 * 60 * 60 * 1000);
|
||||
// one page of matches: no rel=next
|
||||
expect(rels.next).toBeUndefined();
|
||||
});
|
||||
|
||||
it("filtered pages link the next page with rel=next, keeping the filter", async () => {
|
||||
const { page } = await wireStored("fake-deep-webapp", 3);
|
||||
const html = await (await call(`${page}?days=300`)).text();
|
||||
const { rels } = mf2(html, { baseUrl: `https://diigo-cache.test${page}?days=300` });
|
||||
expect(rels.next).toEqual([`https://diigo-cache.test${page}?days=300&page=2`]);
|
||||
});
|
||||
|
||||
it("JSON answers carry the canonical filter, so devs see what was understood", async () => {
|
||||
@@ -393,25 +427,6 @@ describe.skip("public page filters (goal 1d)", () => {
|
||||
});
|
||||
});
|
||||
|
||||
it("?in= an old week costs O(log pages) diigo fetches with by=updated, not a walk from page 1", async () => {
|
||||
const res = await call("/_wire", {
|
||||
method: "POST",
|
||||
headers: { "content-type": "application/json" },
|
||||
body: JSON.stringify({ webappId: "fake-deep-webapp", mode: "public" }),
|
||||
});
|
||||
expect(res.status).toBe(200);
|
||||
// deep_dee: 6000 bookmarks, one a day, 60 diigo pages of 100; a week
|
||||
// ~14 years back sits near page 51 — a walk from page 1 would cost ~51
|
||||
const longAgo = new Date(Date.now() - 5100 * 24 * 60 * 60 * 1000);
|
||||
const week = isoWeek(longAgo);
|
||||
const before = await testEnv.WEBAPP.calls();
|
||||
const html = await (await call(`/public/deep_dee?in=${week}`)).text();
|
||||
// one a day; every 10th is private
|
||||
expect(titlesIn(html).length).toBeGreaterThanOrEqual(6);
|
||||
expect(titlesIn(html).length).toBeLessThanOrEqual(7);
|
||||
expect((await testEnv.WEBAPP.calls()) - before).toBeLessThanOrEqual(12);
|
||||
});
|
||||
|
||||
it("a malformed filter is a 400 with a message, not an empty list", async () => {
|
||||
const page = await wireDated();
|
||||
const res = await call(`${page}?days=7x`);
|
||||
|
||||
@@ -18,9 +18,9 @@ const OVERSTATEMENT = 60;
|
||||
* 5th an old bookmark edited recently (created 100 days before updated) */
|
||||
const DATED_WEBAPP = "fake-dated-webapp";
|
||||
const DATED_TOTAL = 60;
|
||||
/** the same shape, 6000 deep (~16 years, 60 pages of 100): for walk cost */
|
||||
/** the same shape, 300 deep (3 pages of 100): for paging matches */
|
||||
const DEEP_WEBAPP = "fake-deep-webapp";
|
||||
const DEEP_TOTAL = 6000;
|
||||
const DEEP_TOTAL = 300;
|
||||
const HOUR = 60 * 60 * 1000;
|
||||
|
||||
function datedItem(n, now, user = "dated_dan") {
|
||||
|
||||
@@ -2,7 +2,7 @@ import { describe, expect, it } from "vitest";
|
||||
import { matches, parseQuery, toQuery } from "../src/query.js";
|
||||
import type { FilterSpec, Query, QueryError } from "../src/query.js";
|
||||
|
||||
// goal 1d (capsite checklist): the design as tests; unskip as it lands
|
||||
// goal 1d (capsite checklist)
|
||||
|
||||
const NOW = new Date("2026-10-07T12:00:00.000Z");
|
||||
const DAY = 24 * 60 * 60 * 1000;
|
||||
@@ -21,7 +21,7 @@ const refused = (q: Query | QueryError) => {
|
||||
const day = (iso: string) => new Date(`${iso}T00:00:00.000Z`);
|
||||
const range = (q: Query) => [q.after?.toISOString().slice(0, 10), q.before?.toISOString().slice(0, 10)];
|
||||
|
||||
describe.skip("filters: one spec, two spellings (goal 1d)", () => {
|
||||
describe("filters: one spec, two spellings (goal 1d)", () => {
|
||||
it("devs write { days: 7 }; URLs write ?days=7; same query", () => {
|
||||
expect(spec({ days: 7 })).toEqual(url("days=7"));
|
||||
expect(spec({ days: 7 }).after).toEqual(new Date(NOW.getTime() - 7 * DAY));
|
||||
@@ -42,7 +42,7 @@ describe.skip("filters: one spec, two spellings (goal 1d)", () => {
|
||||
});
|
||||
});
|
||||
|
||||
describe.skip("windows: UTC, half-open (goal 1d)", () => {
|
||||
describe("windows: UTC, half-open (goal 1d)", () => {
|
||||
it("after/before are UTC days: after inclusive from 00:00Z, before exclusive", () => {
|
||||
expect(range(url("after=2026-01-01&before=2026-02-01"))).toEqual(["2026-01-01", "2026-02-01"]);
|
||||
expect(url("after=2026/01/01").after).toEqual(day("2026-01-01"));
|
||||
@@ -88,7 +88,7 @@ describe.skip("windows: UTC, half-open (goal 1d)", () => {
|
||||
});
|
||||
});
|
||||
|
||||
describe.skip("q: terms and operators (goal 1d)", () => {
|
||||
describe("q: terms and operators (goal 1d)", () => {
|
||||
const q = (text: string) => url(`q=${encodeURIComponent(text)}`);
|
||||
|
||||
it("words, phrases, #tags, tag:, and - negation; case-insensitive", () => {
|
||||
@@ -160,7 +160,7 @@ const item = (iso: string, extra: Record<string, unknown> = {}) => ({
|
||||
});
|
||||
const query = (q: Partial<Query>): Query => ({ by: "updated", words: [], tags: [], ...q });
|
||||
|
||||
describe.skip("matches (goal 1d)", () => {
|
||||
describe("matches (goal 1d)", () => {
|
||||
it("the window is half-open: after inclusive, before exclusive", () => {
|
||||
const q = query({ after: day("2026-01-01"), before: day("2026-02-01") });
|
||||
expect(matches(item("2026-01-01T00:00:00.000Z"), q)).toBe(true);
|
||||
|
||||
@@ -0,0 +1 @@
|
||||
node_modules
|
||||
@@ -0,0 +1,21 @@
|
||||
# diigo-week
|
||||
|
||||
An EmDash plugin that drafts a potluck issue from a week of your shared diigo bookmarks. It reads them as microformats from diigo-cache's public page, so you edit a draft instead of copy-pasting links.
|
||||
|
||||
## Getting started
|
||||
|
||||
Run diigo-cache (with a filled public cache) and potluck, each in its own terminal:
|
||||
|
||||
```sh
|
||||
pnpm install
|
||||
(cd ../diigo-cache && pnpm dev) # → http://localhost:8787/
|
||||
(cd ../potluck && pnpm dev) # → http://localhost:4321/_emdash/admin
|
||||
```
|
||||
|
||||
On potluck's admin dashboard, the **This week's issue** card has a **Draft 2026-Wnn from diigo** button for last week; it links to the draft once made. For any other week: Plugins → **Draft from diigo**. A week that already has an issue isn't drafted again.
|
||||
|
||||
## How it reads the week
|
||||
|
||||
It asks diigo-cache for `/public/<user>?in=<week>&by=created` over the `DIIGO_CACHE` service binding (potluck's `wrangler.jsonc`), reads the page's `h-feed` of `h-entry` bookmarks, follows `rel=next`, and creates a draft in `issues`: title, week, dek, and a paragraph per bookmark with its description and tag links, as in potluck's `seed/issues.json`. The binding is its only reach outside the site; it declares `content:write` and `taxonomies:write` (the issue gets its bookmarks' tags, as the sample issues do), nothing else.
|
||||
|
||||
Checks: `pnpm lint && pnpm test`.
|
||||
@@ -0,0 +1,24 @@
|
||||
{
|
||||
"name": "diigo-week",
|
||||
"version": "0.1.0",
|
||||
"private": true,
|
||||
"type": "module",
|
||||
"exports": {
|
||||
".": {
|
||||
"import": "./src/index.ts"
|
||||
}
|
||||
},
|
||||
"scripts": {
|
||||
"lint": "tsc --noEmit",
|
||||
"test": "vitest run"
|
||||
},
|
||||
"dependencies": {
|
||||
"emdash": "1.2.0",
|
||||
"microformats-parser": "^2.0.6"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@emdash-cms/blocks": "^1.2.0",
|
||||
"typescript": "^7.0.2",
|
||||
"vitest": "^5.0.3"
|
||||
}
|
||||
}
|
||||
Generated
+6223
File diff suppressed because it is too large.
Load diff
@@ -0,0 +1,8 @@
|
||||
packages:
|
||||
- "."
|
||||
|
||||
strictDepBuilds: true
|
||||
dangerouslyAllowAllBuilds: false
|
||||
allowBuilds:
|
||||
esbuild: true
|
||||
workerd: true
|
||||
Vendored
+7
@@ -0,0 +1,7 @@
|
||||
/** emdash's virtual module for the site worker's bindings (declared for site
|
||||
* builds in emdash/src/virtual-modules.d.ts), so this package typechecks
|
||||
* standalone: `env` from `cloudflare:workers` under @astrojs/cloudflare,
|
||||
* undefined on other adapters */
|
||||
declare module "virtual:emdash/env" {
|
||||
export const env: Record<string, unknown> | undefined;
|
||||
}
|
||||
@@ -0,0 +1,180 @@
|
||||
/**
|
||||
* @file diigo-week: an EmDash plugin that drafts a potluck issue from a
|
||||
* week of public diigo bookmarks.
|
||||
*/
|
||||
|
||||
import { definePlugin } from "emdash";
|
||||
import type { PluginDefinition, RouteContext } from "emdash";
|
||||
import type { Block, BlockResponse } from "@emdash-cms/blocks";
|
||||
import { draftIssue, issueTags, lastWeek, readWeek } from "./week.js";
|
||||
|
||||
/** the diigo username drafts read from, until the admin page saves another */
|
||||
const DEFAULT_USER = "dckc-madmode";
|
||||
|
||||
/** something that answers HTTP requests: a service binding's `fetch` */
|
||||
type Fetcher = { fetch(request: Request): Promise<Response> };
|
||||
|
||||
/**
|
||||
* the `DIIGO_CACHE` service binding to diigo-cache, from the site worker's
|
||||
* env. The binding is the plugin's only reach outside the site:
|
||||
* `ctx.http.fetch` can't reach a dev server (it refuses localhost), and a
|
||||
* binding names one worker rather than a host.
|
||||
*/
|
||||
async function diigoCache(): Promise<Fetcher> {
|
||||
const { env } = await import("virtual:emdash/env");
|
||||
const binding = env?.DIIGO_CACHE as Fetcher | undefined;
|
||||
if (typeof binding?.fetch !== "function") {
|
||||
throw new Error('no DIIGO_CACHE service binding: add { "binding": "DIIGO_CACHE", "service": "diigo-cache" } to services in wrangler.jsonc');
|
||||
}
|
||||
return binding;
|
||||
}
|
||||
|
||||
/** GET a diigo-cache path as HTML; anything but a 200 is an error */
|
||||
async function getter(cache: Fetcher): Promise<(path: string) => Promise<string>> {
|
||||
return async (path) => {
|
||||
const res = await cache.fetch(new Request(`https://diigo-cache${path}`, { headers: { accept: "text/html" } }));
|
||||
const text = await res.text();
|
||||
if (res.status !== 200) throw new Error(`diigo-cache ${path.split("?")[0]}: ${res.status} ${text.slice(0, 200)}`);
|
||||
return text;
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* assign `tags` to an issue in the `tag` taxonomy (as seed/issues.json
|
||||
* does), creating terms that don't exist yet; returns the tags that
|
||||
* couldn't be assigned, with why
|
||||
*/
|
||||
async function tagIssue(ctx: RouteContext, id: string, tags: string[]): Promise<string[]> {
|
||||
const tax = ctx.taxonomies;
|
||||
if (typeof tax?.createTerm !== "function" || typeof tax.addEntryTerms !== "function") throw new Error("this plugin lacks taxonomies:write");
|
||||
const known = new Map((await tax.getTerms("tag")).map((t) => [t.slug, t.id]));
|
||||
const ids: string[] = [];
|
||||
const failed: string[] = [];
|
||||
for (const tag of tags) {
|
||||
try {
|
||||
ids.push(known.get(tag) ?? (await tax.createTerm("tag", { label: tag, slug: tag })).id);
|
||||
} catch (error) {
|
||||
failed.push(`${tag} (${error instanceof Error ? error.message : String(error)})`);
|
||||
}
|
||||
}
|
||||
if (ids.length) await tax.addEntryTerms("issues", id, "tag", ids);
|
||||
return failed;
|
||||
}
|
||||
|
||||
/** the issue already drafted or published for `week`, if any (issues
|
||||
* are few: a scan of the newest is enough) */
|
||||
async function issueFor(ctx: RouteContext, week: string): Promise<{ id: string; status: string; title: string } | undefined> {
|
||||
const found = (await ctx.content!.list("issues", { limit: 100 })).items.find((it) => it.data.week === week);
|
||||
return found && { id: found.id, status: found.status, title: String(found.data.title ?? week) };
|
||||
}
|
||||
|
||||
/** a button-row link to an issue in the editor */
|
||||
const openIssue = (id: string, label: string): Block => ({
|
||||
type: "actions",
|
||||
elements: [{ type: "link", label, target: { kind: "content", collection: "issues", id }, appearance: "primary" }],
|
||||
});
|
||||
|
||||
/** draft `week` from `user`'s bookmarks, unless the week has an issue
|
||||
* already: blocks that say what happened (with a link), and a toast */
|
||||
async function draft(ctx: RouteContext, user: string, week: string): Promise<{ note: Block[]; toast: BlockResponse["toast"] }> {
|
||||
try {
|
||||
if (!/^[A-Za-z0-9_-]{1,64}$/.test(user)) throw new Error("diigo username: letters, digits, _ and - only");
|
||||
const existing = await issueFor(ctx, week);
|
||||
if (existing) {
|
||||
return {
|
||||
note: [
|
||||
{ type: "banner", variant: "default", title: `${week} already has an issue`, description: `"${existing.title}" (${existing.status}). Delete it to draft the week again.` },
|
||||
openIssue(existing.id, "Open it"),
|
||||
],
|
||||
toast: { message: `${week} already has an issue`, type: "info" },
|
||||
};
|
||||
}
|
||||
const bookmarks = await readWeek(await getter(await diigoCache()), user, week);
|
||||
const issue = draftIssue(week, bookmarks);
|
||||
const content = ctx.content;
|
||||
if (typeof content?.create !== "function") throw new Error("this plugin lacks content:write");
|
||||
const created = await content.create("issues", issue);
|
||||
const untagged = await tagIssue(ctx, created.id, issueTags(bookmarks));
|
||||
if (untagged.length) ctx.log.warn("tags not assigned", { id: created.id, untagged });
|
||||
ctx.log.info("drafted issue", { week, bookmarks: bookmarks.length, id: created.id });
|
||||
return {
|
||||
note: [
|
||||
{ type: "banner", variant: "default", title: `Drafted "${issue.title}"`, description: `${bookmarks.length} bookmarks, as a draft at /issues/${created.slug ?? "(no slug)"}. Cut, comment, and publish.${untagged.length ? ` Tags not assigned: ${untagged.join(", ")}.` : ""}` },
|
||||
openIssue(created.id, "Open the draft"),
|
||||
],
|
||||
toast: { message: `Drafted ${issue.title}`, type: "success" },
|
||||
};
|
||||
} catch (error) {
|
||||
const message = error instanceof Error ? error.message : String(error);
|
||||
ctx.log.error("draft failed", { week, message });
|
||||
return { note: [{ type: "banner", variant: "alert", title: "Couldn't draft the issue", description: message }], toast: { message, type: "error" } };
|
||||
}
|
||||
}
|
||||
|
||||
/** the admin page: a form for the diigo username and the week */
|
||||
async function page(ctx: RouteContext, note: Block[] = [], toast?: BlockResponse["toast"]): Promise<BlockResponse> {
|
||||
const user = (await ctx.kv.get<string>("user")) ?? DEFAULT_USER;
|
||||
const blocks: Block[] = [
|
||||
{ type: "header", text: "Draft an issue from diigo" },
|
||||
{
|
||||
type: "context",
|
||||
text: "Reads a week of shared bookmarks from diigo-cache (by when they were bookmarked) and saves them as a draft issue for you to edit.",
|
||||
},
|
||||
{
|
||||
type: "form",
|
||||
block_id: "draft",
|
||||
fields: [
|
||||
{ type: "text_input", action_id: "user", label: "diigo username", initial_value: user },
|
||||
{ type: "text_input", action_id: "week", label: "ISO week (UTC)", initial_value: lastWeek(new Date()) },
|
||||
],
|
||||
submit: { label: "Draft issue", action_id: "draft" },
|
||||
},
|
||||
];
|
||||
blocks.push(...note);
|
||||
return { blocks, toast };
|
||||
}
|
||||
|
||||
/** the dashboard widget: last week's issue if there is one, else a button to draft it */
|
||||
async function widget(ctx: RouteContext, note: Block[] = [], toast?: BlockResponse["toast"]): Promise<BlockResponse> {
|
||||
const week = lastWeek(new Date());
|
||||
if (note.length) return { blocks: note, toast };
|
||||
const existing = await issueFor(ctx, week);
|
||||
const blocks: Block[] = existing
|
||||
? [{ type: "section", text: `${week}: "${existing.title}" (${existing.status})` }, openIssue(existing.id, "Open it")]
|
||||
: [
|
||||
{ type: "section", text: `${week} has no issue yet.` },
|
||||
{ type: "actions", elements: [{ type: "button", action_id: "draft-last-week", label: `Draft ${week} from diigo`, style: "primary" }] },
|
||||
];
|
||||
return { blocks, toast };
|
||||
}
|
||||
|
||||
export function createPlugin(): PluginDefinition {
|
||||
return definePlugin({
|
||||
id: "diigo-week",
|
||||
version: "0.1.0",
|
||||
capabilities: ["content:write", "taxonomies:write"],
|
||||
admin: {
|
||||
pages: [{ path: "/draft", label: "Draft from diigo", icon: "bookmark" }],
|
||||
widgets: [{ id: "last-week", title: "This week's issue", size: "half" }],
|
||||
},
|
||||
routes: {
|
||||
admin: {
|
||||
permission: "plugins:manage",
|
||||
handler: async (ctx) => {
|
||||
const input = ctx.input as { type?: string; page?: string; action_id?: string; values?: Record<string, unknown> } | undefined;
|
||||
const user = (await ctx.kv.get<string>("user")) ?? DEFAULT_USER;
|
||||
if (input?.type === "block_action" && input.action_id === "draft-last-week") {
|
||||
const { note, toast } = await draft(ctx, user, lastWeek(new Date()));
|
||||
return widget(ctx, note, toast);
|
||||
}
|
||||
if (input?.page?.startsWith("widget:")) return widget(ctx);
|
||||
if (input?.type !== "form_submit" || input.action_id !== "draft") return page(ctx);
|
||||
const formUser = String(input.values?.user ?? "").trim();
|
||||
if (/^[A-Za-z0-9_-]{1,64}$/.test(formUser)) await ctx.kv.set("user", formUser);
|
||||
const { note, toast } = await draft(ctx, formUser, String(input.values?.week ?? "").trim());
|
||||
return page(ctx, note, toast);
|
||||
},
|
||||
},
|
||||
},
|
||||
});
|
||||
}
|
||||
@@ -0,0 +1,146 @@
|
||||
/**
|
||||
* @file a week of bookmarks, read as microformats from diigo-cache's public
|
||||
* page, drafted as a potluck issue.
|
||||
*/
|
||||
|
||||
import { mf2 } from "microformats-parser";
|
||||
|
||||
const DAY_MS = 24 * 60 * 60 * 1000;
|
||||
|
||||
/** pages of matches one draft will read (100 bookmarks each) */
|
||||
export const MAX_PAGES = 10;
|
||||
|
||||
/** one bookmark, as its h-entry says */
|
||||
export type Bookmark = {
|
||||
url: string;
|
||||
title: string;
|
||||
description: string;
|
||||
tags: string[];
|
||||
/** when it was bookmarked, ISO */
|
||||
published: string;
|
||||
};
|
||||
|
||||
/** `yyyy-Www` for the ISO 8601 week holding `d` (UTC) */
|
||||
export function isoWeek(d: Date): string {
|
||||
const t = new Date(Date.UTC(d.getUTCFullYear(), d.getUTCMonth(), d.getUTCDate()));
|
||||
const day = t.getUTCDay() || 7;
|
||||
t.setUTCDate(t.getUTCDate() + 4 - day); // the week's Thursday names its year
|
||||
const year = t.getUTCFullYear();
|
||||
const week = Math.ceil(((t.getTime() - Date.UTC(year, 0, 1)) / DAY_MS + 1) / 7);
|
||||
return `${year}-W${String(week).padStart(2, "0")}`;
|
||||
}
|
||||
|
||||
/** the last full ISO week before `now` */
|
||||
export function lastWeek(now: Date): string {
|
||||
return isoWeek(new Date(now.getTime() - 7 * DAY_MS));
|
||||
}
|
||||
|
||||
/** Monday 00:00Z of ISO week `week` (`2026-W41`), or undefined if malformed */
|
||||
export function weekStart(week: string): Date | undefined {
|
||||
const m = /^(\d{4})-W(\d{2})$/.exec(week);
|
||||
if (!m) return undefined;
|
||||
const [year, w] = [Number(m[1]), Number(m[2])];
|
||||
const jan4 = Date.UTC(year, 0, 4);
|
||||
const monday = jan4 - ((new Date(jan4).getUTCDay() || 7) - 1) * DAY_MS + (w - 1) * 7 * DAY_MS;
|
||||
const start = new Date(monday);
|
||||
return w >= 1 && isoWeek(start) === week ? start : undefined;
|
||||
}
|
||||
|
||||
/** the first string value of an mf2 property, or "" */
|
||||
const first = (props: Record<string, unknown[]>, name: string): string => {
|
||||
const v = props[name]?.[0];
|
||||
return typeof v === "string" ? v : "";
|
||||
};
|
||||
|
||||
/** the bookmarks in a page's `h-feed`, and its `rel=next` link if any */
|
||||
export function readPage(html: string, url: string): { bookmarks: Bookmark[]; next?: string } {
|
||||
const { items, rels } = mf2(html, { baseUrl: url });
|
||||
const feed = items.find((i) => i.type?.includes("h-feed"));
|
||||
const bookmarks = (feed?.children ?? [])
|
||||
.filter((e) => e.type?.includes("h-entry"))
|
||||
.map((e) => {
|
||||
const props = e.properties as Record<string, unknown[]>;
|
||||
const tags = (props.category ?? []).filter((t): t is string => typeof t === "string");
|
||||
return {
|
||||
url: first(props, "bookmark-of"),
|
||||
title: first(props, "name"),
|
||||
description: first(props, "summary"),
|
||||
tags,
|
||||
published: first(props, "published"),
|
||||
};
|
||||
})
|
||||
.filter((b) => b.url !== "");
|
||||
return { bookmarks, next: rels.next?.[0] };
|
||||
}
|
||||
|
||||
/**
|
||||
* a user's bookmarks from an ISO week, oldest first, by when they were
|
||||
* bookmarked: diigo-cache's `/public/<user>?in=<week>&by=created`,
|
||||
* following `rel=next` up to {@link MAX_PAGES} pages.
|
||||
*
|
||||
* @param get fetches a path on diigo-cache, answering its HTML; it throws
|
||||
* on anything but a 200
|
||||
*/
|
||||
export async function readWeek(get: (path: string) => Promise<string>, user: string, week: string): Promise<Bookmark[]> {
|
||||
const base = "https://diigo-cache.invalid";
|
||||
let path: string | undefined = `/public/${encodeURIComponent(user)}?in=${encodeURIComponent(week)}&by=created`;
|
||||
const bookmarks: Bookmark[] = [];
|
||||
for (let n = 0; path && n < MAX_PAGES; n += 1) {
|
||||
const page = readPage(await get(path), base + path);
|
||||
bookmarks.push(...page.bookmarks);
|
||||
const next: URL | undefined = page.next ? new URL(page.next) : undefined;
|
||||
path = next && next.origin === base ? next.pathname + next.search : undefined;
|
||||
}
|
||||
return bookmarks.reverse();
|
||||
}
|
||||
|
||||
/** a Portable Text span */
|
||||
type Span = { _type: "span"; _key: string; text: string; marks: string[] };
|
||||
/** a Portable Text block */
|
||||
type Block = { _type: "block"; _key: string; style: string; markDefs: { _type: "link"; _key: string; href: string }[]; children: Span[] };
|
||||
|
||||
/** a bookmark as one paragraph: the link, the description, then tag links (as in seed/issues.json) */
|
||||
function bookmarkBlock(b: Bookmark, i: number): Block {
|
||||
const k = `b${i}`;
|
||||
const markDefs: Block["markDefs"] = [{ _type: "link", _key: `${k}-a`, href: b.url }];
|
||||
const children: Span[] = [{ _type: "span", _key: `${k}-t`, text: b.title || b.url, marks: [`${k}-a`] }];
|
||||
if (b.description) children.push({ _type: "span", _key: `${k}-d`, text: `: ${b.description}`, marks: [] });
|
||||
b.tags.forEach((tag, j) => {
|
||||
markDefs.push({ _type: "link", _key: `${k}-g${j}`, href: `/tag/${encodeURIComponent(tag)}` });
|
||||
children.push({ _type: "span", _key: `${k}-s${j}`, text: " ", marks: [] });
|
||||
children.push({ _type: "span", _key: `${k}-n${j}`, text: tag, marks: [`${k}-g${j}`] });
|
||||
});
|
||||
return { _type: "block", _key: k, style: "normal", markDefs, children };
|
||||
}
|
||||
|
||||
/** the issue's tags: every bookmark's, deduplicated, in order of first use */
|
||||
export function issueTags(bookmarks: Bookmark[]): string[] {
|
||||
return [...new Set(bookmarks.flatMap((b) => b.tags))];
|
||||
}
|
||||
|
||||
/** `Oct 5–11` (or `Sep 28–Oct 4`) for the week starting `monday` */
|
||||
function range(monday: Date): string {
|
||||
const sunday = new Date(monday.getTime() + 6 * DAY_MS);
|
||||
const fmt = (d: Date, withMonth: boolean) =>
|
||||
withMonth ? d.toLocaleDateString("en-US", { month: "short", day: "numeric", timeZone: "UTC" }) : String(d.getUTCDate());
|
||||
return `${fmt(monday, true)}–${fmt(sunday, sunday.getUTCMonth() !== monday.getUTCMonth())}`;
|
||||
}
|
||||
|
||||
/** the fields of a draft `issues` entry for `week`: title, week, dek, body */
|
||||
export function draftIssue(week: string, bookmarks: Bookmark[]): Record<string, unknown> {
|
||||
const monday = weekStart(week);
|
||||
if (!monday) throw new Error(`week: expected an ISO week like 2026-W41, got ${JSON.stringify(week)}`);
|
||||
const intro: Block = {
|
||||
_type: "block",
|
||||
_key: "intro",
|
||||
style: "normal",
|
||||
markDefs: [],
|
||||
children: [{ _type: "span", _key: "intro-t", text: "(draft) A line or two about the week goes here.", marks: [] }],
|
||||
};
|
||||
return {
|
||||
title: `Week ${week}`,
|
||||
week,
|
||||
dek: `${range(monday)}: ${bookmarks.length} bookmark${bookmarks.length === 1 ? "" : "s"}. (draft)`,
|
||||
body: [intro, ...bookmarks.map(bookmarkBlock)],
|
||||
};
|
||||
}
|
||||
@@ -0,0 +1,99 @@
|
||||
import { describe, expect, it } from "vitest";
|
||||
import { draftIssue, issueTags, lastWeek, readWeek, weekStart } from "../src/week.js";
|
||||
|
||||
/** an h-entry as diigo-cache's public page marks it up (by=created) */
|
||||
const entry = (n: number, created: string, extra = "") =>
|
||||
`<li class="h-entry"><time class="dt-published" datetime="${created}">${created}</time> <a class="u-bookmark-of p-name" href="https://ex.example/${n}">bookmark ${n}</a>${extra}<time class="dt-updated" datetime="${created}" hidden></time></li>`;
|
||||
|
||||
const page = (entries: string[], next?: string) => `<!doctype html><html><body>
|
||||
<p>${next ? `<a rel="next" href="${next}">next →</a>` : "<span>next →</span>"}</p>
|
||||
<ol class="h-feed">
|
||||
${entries.join("\n")}
|
||||
</ol></body></html>`;
|
||||
|
||||
describe("readWeek", () => {
|
||||
it("reads h-entry bookmarks across rel=next pages, oldest first", async () => {
|
||||
const asked: string[] = [];
|
||||
const pages: Record<string, string> = {
|
||||
"/public/dckc-madmode?in=2026-W40&by=created": page(
|
||||
[
|
||||
entry(
|
||||
3,
|
||||
"2026-10-03T12:00:00.000Z",
|
||||
` <a class="p-category" href="/public/dckc-madmode?q=%23ocap">ocap</a> <a class="p-category" href="/public/dckc-madmode?q=%23weekly">weekly</a><br><small class="p-summary">a week of fragments</small>`,
|
||||
),
|
||||
entry(2, "2026-10-01T12:00:00.000Z"),
|
||||
],
|
||||
"/public/dckc-madmode?in=2026-W40&by=created&page=2",
|
||||
),
|
||||
"/public/dckc-madmode?in=2026-W40&by=created&page=2": page([entry(1, "2026-09-28T12:00:00.000Z")]),
|
||||
};
|
||||
const get = async (path: string) => {
|
||||
asked.push(path);
|
||||
const html = pages[path];
|
||||
if (!html) throw new Error(`no page ${path}`);
|
||||
return html;
|
||||
};
|
||||
const bookmarks = await readWeek(get, "dckc-madmode", "2026-W40");
|
||||
expect(asked).toHaveLength(2);
|
||||
expect(bookmarks.map((b) => b.title)).toEqual(["bookmark 1", "bookmark 2", "bookmark 3"]);
|
||||
expect(bookmarks[2]).toEqual({
|
||||
url: "https://ex.example/3",
|
||||
title: "bookmark 3",
|
||||
description: "a week of fragments",
|
||||
tags: ["ocap", "weekly"],
|
||||
published: "2026-10-03T12:00:00.000Z",
|
||||
});
|
||||
});
|
||||
|
||||
it("doesn't follow a rel=next to another site", async () => {
|
||||
const asked: string[] = [];
|
||||
const get = async (path: string) => {
|
||||
asked.push(path);
|
||||
return page([entry(1, "2026-10-01T12:00:00.000Z")], "https://elsewhere.example/public/x");
|
||||
};
|
||||
await readWeek(get, "dckc-madmode", "2026-W40");
|
||||
expect(asked).toHaveLength(1);
|
||||
});
|
||||
});
|
||||
|
||||
describe("weeks", () => {
|
||||
it("lastWeek is the full ISO week before now", () => {
|
||||
expect(lastWeek(new Date("2026-10-09T01:00:00Z"))).toBe("2026-W40");
|
||||
// Monday 2026-01-05 starts W02; W01 began Monday 2025-12-29
|
||||
expect(lastWeek(new Date("2026-01-05T00:00:00Z"))).toBe("2026-W01");
|
||||
});
|
||||
|
||||
it("weekStart is Monday 00:00Z, and refuses a week that doesn't exist", () => {
|
||||
expect(weekStart("2026-W40")?.toISOString()).toBe("2026-09-28T00:00:00.000Z");
|
||||
expect(weekStart("2025-W01")?.toISOString()).toBe("2024-12-30T00:00:00.000Z");
|
||||
expect(weekStart("2025-W53")).toBeUndefined();
|
||||
expect(weekStart("2026-40")).toBeUndefined();
|
||||
});
|
||||
});
|
||||
|
||||
describe("draftIssue", () => {
|
||||
it("drafts title, week, dek, and a body: an intro, then a paragraph per bookmark with tag links", () => {
|
||||
const issue = draftIssue("2026-W40", [
|
||||
{ url: "https://ex.example/1", title: "one", description: "nice", tags: ["ocap"], published: "2026-09-28T12:00:00.000Z" },
|
||||
]);
|
||||
expect(issue.title).toBe("Week 2026-W40");
|
||||
expect(issue.week).toBe("2026-W40");
|
||||
expect(issue.dek).toBe("Sep 28–Oct 4: 1 bookmark. (draft)");
|
||||
const [intro, b] = issue.body as { markDefs: { href: string }[]; children: { text: string }[] }[];
|
||||
expect(intro.children[0].text).toMatch(/draft/);
|
||||
expect(b.markDefs.map((m) => m.href)).toEqual(["https://ex.example/1", "/tag/ocap"]);
|
||||
expect(b.children.map((c) => c.text).join("")).toBe("one: nice ocap");
|
||||
});
|
||||
|
||||
it("refuses a malformed week", () => {
|
||||
expect(() => draftIssue("last week", [])).toThrow(/ISO week/);
|
||||
});
|
||||
});
|
||||
|
||||
describe("issueTags", () => {
|
||||
it("is every bookmark's tags, once each, in order of first use", () => {
|
||||
const b = (tags: string[]) => ({ url: "https://ex.example/", title: "", description: "", tags, published: "" });
|
||||
expect(issueTags([b(["governance", "economics"]), b([]), b(["governance", "scaling"])])).toEqual(["governance", "economics", "scaling"]);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,20 @@
|
||||
{
|
||||
"compilerOptions": {
|
||||
"target": "ES2022",
|
||||
"module": "preserve",
|
||||
"moduleResolution": "bundler",
|
||||
"strict": true,
|
||||
"esModuleInterop": true,
|
||||
"verbatimModuleSyntax": true,
|
||||
"skipLibCheck": true,
|
||||
"types": []
|
||||
},
|
||||
"include": [
|
||||
"src/**/*",
|
||||
"tests/**/*",
|
||||
"vitest.config.ts"
|
||||
],
|
||||
"exclude": [
|
||||
"node_modules"
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,146 @@
|
||||
<!doctype html>
|
||||
<html><head><meta charset="utf-8">
|
||||
<title>webapp cookie rotation: who holds what</title>
|
||||
<!-- drawn after the style of MarkM's ERTP slides; cookie-rotation-wiring.png is a headless-Chromium screenshot of this page -->
|
||||
<style>
|
||||
html, body { margin: 0; background: #fff; }
|
||||
.sans { font-family: "Helvetica Neue", "DejaVu Sans", sans-serif; }
|
||||
.obj { font-family: "DejaVu Serif", serif; font-style: italic; }
|
||||
.mono { font-family: "DejaVu Sans Mono", monospace; }
|
||||
</style></head>
|
||||
<body>
|
||||
<svg id="c" xmlns="http://www.w3.org/2000/svg" width="1120" height="820" viewBox="0 0 1120 820"
|
||||
role="img" aria-labelledby="title desc">
|
||||
<title id="title">webapp cookie rotation: who holds what</title>
|
||||
<desc id="desc">Three actors and two makers. Alice holds a reference to a store made by
|
||||
makeMapStore, and to diigo-cache, which she wired up. She rotates the cookie
|
||||
by saying: await E(store).set("cookie", freshCookie). The store, and its
|
||||
read-only facets, both refer to the store's kv table, whose one row maps
|
||||
"cookie" to the session; its shapes are keyShape "cookie" and valueShape
|
||||
M.string(). makeWebApp makes access objects; each refers to its init
|
||||
table: url, headers, and headerStore. headerStore refers to the store's
|
||||
read-only facet, which access reads at every send. diigo-cache holds a
|
||||
reference to access. Mallory, who learned the instance id, also holds a
|
||||
reference to access, so she can use the cookie, but she has no reference
|
||||
to the store, so she cannot rotate it or swap in her own session.</desc>
|
||||
<defs>
|
||||
<radialGradient id="ball" cx="40%" cy="35%" r="70%">
|
||||
<stop offset="0%" stop-color="#7cc8f8"/>
|
||||
<stop offset="100%" stop-color="#3aa6ee"/>
|
||||
</radialGradient>
|
||||
<linearGradient id="objfill" x1="0" y1="0" x2="0" y2="1">
|
||||
<stop offset="0%" stop-color="#6cbcf4"/>
|
||||
<stop offset="100%" stop-color="#4aa8ec"/>
|
||||
</linearGradient>
|
||||
<marker id="head" viewBox="0 0 10 10" refX="9" refY="5" markerWidth="9" markerHeight="9" orient="auto-start-reverse">
|
||||
<path d="M0,0 L10,5 L0,10 z" fill="#1d4f7a"/>
|
||||
</marker>
|
||||
</defs>
|
||||
</svg>
|
||||
<script>
|
||||
const svg = document.getElementById("c");
|
||||
const NS = "http://www.w3.org/2000/svg";
|
||||
const el = (name, attrs, parent = svg) => {
|
||||
const e = document.createElementNS(NS, name);
|
||||
for (const [k, v] of Object.entries(attrs)) e.setAttribute(k, v);
|
||||
parent.appendChild(e);
|
||||
return e;
|
||||
};
|
||||
const text = (x, y, s, cls, attrs = {}) => {
|
||||
const t = el("text", { x, y, class: cls, "text-anchor": "middle", fill: "#1d3550", ...attrs });
|
||||
t.textContent = s;
|
||||
return t;
|
||||
};
|
||||
|
||||
const INK = "#1d4f7a";
|
||||
const dot = (x, y) => el("circle", { cx: x, cy: y, r: 4.5, fill: INK });
|
||||
/** a reference: dot at the holder, arrowhead at the target */
|
||||
const ref = (x1, y1, x2, y2, bend = 0, extra = {}) => {
|
||||
dot(x1, y1);
|
||||
const mx = (x1 + x2) / 2 - (y2 - y1) * bend, my = (y1 + y2) / 2 + (x2 - x1) * bend;
|
||||
el("path", { d: `M${x1},${y1} Q${mx},${my} ${x2},${y2}`, fill: "none", stroke: INK, "stroke-width": 2, "marker-end": "url(#head)", ...extra });
|
||||
};
|
||||
|
||||
// maker box: a pale rounded region, with stacked instances inside
|
||||
function maker(x, y, w, h, label) {
|
||||
el("rect", { x, y, width: w, height: h, rx: 26, fill: "#d6ecfb", stroke: "#b5dbf5", "stroke-width": 1.5 });
|
||||
text(x + 18, y + 30, label, "obj", { "text-anchor": "start", "font-size": 20 });
|
||||
for (let i = 2; i >= 0; i -= 1) {
|
||||
el("rect", {
|
||||
x: x + 18 + i * 9, y: y + 48 + i * 9, width: w - 52, height: h - 82, rx: 22,
|
||||
fill: "#c8e5fa", stroke: "#8ec9f2", "stroke-width": 1.5, "fill-opacity": 0.9,
|
||||
});
|
||||
}
|
||||
}
|
||||
// an object: rounded square, optionally stacked (several made)
|
||||
function obj(x, y, w, h, label, stacked = false) {
|
||||
const n = stacked ? 3 : 1;
|
||||
for (let i = n - 1; i >= 0; i -= 1) {
|
||||
el("rect", { x: x + i * 7, y: y - i * 7, width: w, height: h, rx: 12, fill: "url(#objfill)", stroke: "#2f8fd6", "stroke-width": 1.5 });
|
||||
}
|
||||
text(x + w / 2, y + h / 2 + 7, label, "obj", { "font-size": 19 });
|
||||
}
|
||||
function actor(cx, cy, label) {
|
||||
el("circle", { cx, cy, r: 52, fill: "url(#ball)", stroke: "#2f8fd6", "stroke-width": 1.5 });
|
||||
text(cx, cy + 6, label, "sans", { "font-size": label.length > 8 ? 15 : 17 });
|
||||
}
|
||||
// a table like the ERTP ledger
|
||||
function table(x, y, title, rows, colW) {
|
||||
const rowH = 28, w = colW[0] + colW[1];
|
||||
el("rect", { x, y, width: w, height: 36 + rowH * rows.length, fill: "#5fb5f2", stroke: "#2f8fd6", "stroke-width": 1.5 });
|
||||
text(x + w / 2, y + 25, title, "obj", { "font-size": 18 });
|
||||
rows.forEach(([k, v], i) => {
|
||||
const ry = y + 36 + i * rowH;
|
||||
el("line", { x1: x, y1: ry, x2: x + w, y2: ry, stroke: "#2f8fd6", "stroke-width": 1.2 });
|
||||
text(x + 10, ry + 19, k, "mono", { "text-anchor": "start", "font-size": 14 });
|
||||
if (v) text(x + colW[0] + 10, ry + 19, v, "mono", { "text-anchor": "start", "font-size": 14 });
|
||||
});
|
||||
el("line", { x1: x + colW[0], y1: y + 36, x2: x + colW[0], y2: y + 36 + rowH * rows.length, stroke: "#1d4f7a", "stroke-width": 1.5 });
|
||||
return { rowY: (i) => y + 36 + i * rowH + rowH / 2, x, w, colX: x + colW[0] };
|
||||
}
|
||||
|
||||
// speech bubble (Alice rotates after logging in again)
|
||||
el("path", {
|
||||
d: "M30,22 h470 a16,16 0 0 1 16,16 v34 a16,16 0 0 1 -16,16 h-330 l-38,48 l6,-48 h-102 a16,16 0 0 1 -16,-16 v-34 a16,16 0 0 1 16,-16 z",
|
||||
fill: "#d9d9d9", stroke: "#555", "stroke-width": 2,
|
||||
});
|
||||
const code = el("text", { x: 46, y: 62, class: "mono", "font-size": 17, fill: "#222" });
|
||||
code.innerHTML = '<tspan fill="#b0177b">await</tspan> E(store).set(<tspan fill="#1a7f37">"cookie"</tspan>, freshCookie);';
|
||||
|
||||
// actors
|
||||
actor(150, 200, "Alice");
|
||||
actor(560, 200, "diigo-cache");
|
||||
actor(930, 200, "Mallory");
|
||||
|
||||
// makers
|
||||
maker(30, 320, 470, 470, "makeMapStore");
|
||||
maker(560, 320, 530, 470, "makeWebApp");
|
||||
|
||||
// MapStoreDo insides
|
||||
obj(80, 420, 120, 80, "store");
|
||||
obj(80, 600, 120, 80, "readOnly", true);
|
||||
const kv = table(260, 430, "kv", [["cookie", "session=…"], ["", ""], ["", ""], ["", ""], ["", ""]], [90, 110]);
|
||||
|
||||
// WebAppDo insides
|
||||
obj(620, 560, 140, 90, "access", true);
|
||||
const init = table(830, 420, "init", [["url", "diigo…"], ["headers", "{…}"], ["headerStore", ""], ["", ""]], [130, 85]);
|
||||
|
||||
// references
|
||||
ref(140, 228, 125, 418, 0.05); // Alice → store
|
||||
ref(172, 224, 508, 210, 0.04); // Alice → diigo-cache (she wired it)
|
||||
ref(560 - 6, 200 + 18, 660, 553, 0.12); // cache → access
|
||||
ref(930 - 14, 200 + 20, 720, 553, -0.08); // Mallory → access
|
||||
ref(160, 470, kv.x - 2, kv.rowY(0), 0.05); // store → kv
|
||||
ref(150, 666, kv.x - 2, kv.rowY(0) + 6, -0.25); // readOnly → kv (read)
|
||||
ref(760, 600, init.x - 2, init.rowY(2), -0.1); // access → init
|
||||
ref(init.colX + 40, init.rowY(2), 208, 640, -0.28); // headerStore → readOnly
|
||||
|
||||
// captions
|
||||
const cap = (x, y, s, attrs = {}) => text(x, y, s, "sans", { "font-size": 14, fill: "#33536f", ...attrs });
|
||||
cap(930, 118, "holds the instance id:", {});
|
||||
cap(930, 136, "uses the cookie, can't reach store", {});
|
||||
cap(330, 300, "storeId: rotate · revoke", {});
|
||||
cap(760, 806, "init.headerStore → the store's read-only facet, read at every send", {});
|
||||
cap(265, 806, "shapes: keyShape \"cookie\", valueShape M.string()", {});
|
||||
</script>
|
||||
</body></html>
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 113 KiB |
@@ -0,0 +1,166 @@
|
||||
Good Fences Make Good Agents
|
||||
|
||||
*with apologies to Robert Frost*
|
||||
|
||||
Let's have a fence and a neighbor. I can talk to the neighbor much like the conventional...
|
||||
|
||||
```js
|
||||
client.chat.completions.create({
|
||||
model: 'llama3.1:8b',
|
||||
messages: [{ role: 'user', content: 'Tell me a story' }],
|
||||
stream: true,
|
||||
});
|
||||
```
|
||||
|
||||
but with a twist:
|
||||
|
||||
```js
|
||||
messages: [{
|
||||
role: 'user',
|
||||
content: 'you can use this mower for the weekend',
|
||||
things: { mower },
|
||||
about: { mower: { ts: 'type Mower { start(), push(), turn(), stop() }' }}
|
||||
}],
|
||||
```
|
||||
|
||||
I send `mower` by putting a **webkey** into a password field on the fence form, which fetches an `RpcTarget` stub to give to the neighbor.
|
||||
|
||||
The neighbor can make its own tenants: it can write code to compose the things I gave it, and load that code into a dynamic worker that is endowed with the things.
|
||||
|
||||
A `Mower` DO factory makes mowers that go all over the place (depicted as a table with green cells for grass). But with a little js playground, I can wrap it in a governor so it never goes more than 100 yards away. I give only the governed mower to my neighbor. Even though he has 50 acres, I know most of them are for tenants, and my neighbor has no particular need to let them use it.
|
||||
|
||||
Even if they gets a little tipsy and wander around, try as he might, my neighbor and his tennants can't mow more than 100 yards away.
|
||||
|
||||
## Design IBIS
|
||||
|
||||
### ?! 1. Who is the neighbor, and where does it run?
|
||||
|
||||
```
|
||||
:- 1a. an LLM (llama3.1:8b) alone
|
||||
- 1a-i. a model only sees text; it can't hold `things`
|
||||
:+ 1b. an LLM plus a harness we write that holds `things` and acts for it (see 2)
|
||||
:+ 1b-i. harness and tenants run on my desktop under `wrangler dev`, for now
|
||||
+ 1b-i-a. I want to learn the CF platform
|
||||
+ 1b-i-b. CF is the only place to get dynamic workers
|
||||
. 1b-i-c. the Worker Loader binding works in local dev (cf-fun1)
|
||||
?! 1b-ii. Where does inference run?
|
||||
: 1b-ii-a. local Ollama, via its OpenAI-compatible endpoint
|
||||
- 1b-ii-a-i. the local model may not be powerful enough
|
||||
. 1b-ii-a-i-a. I do have an M5 with 48GB RAM
|
||||
: 1b-ii-b. remote, on Workers AI, through the AI binding from `wrangler dev`
|
||||
+ 1b-ii-b-i. I want to learn the CF platform
|
||||
:+ 1b-ii-c. either, behind one interface; start remote
|
||||
+ 1b-ii-c-i. starting remote lowers the odds of failing because the model isn't powerful enough
|
||||
```
|
||||
|
||||
### ?! 2. How do `things` get from a chat message to something the neighbor can call?
|
||||
|
||||
```
|
||||
. 2-i. no chat-completions API has `things`/`about`; this is our extension
|
||||
:+ 2a. the harness strips `things`, keeps the stubs, and shows the model only `about`
|
||||
: 2b. expose each thing as a tool, one per method in its `about.ts`
|
||||
:+ 2c. code mode: the model writes JS against the `about.ts` types; the harness runs it in a tenant (see 5)
|
||||
+ 2c-i. it's the story: the neighbor writes code to compose the things and loads it into a dynamic worker endowed with them
|
||||
? 2d. What does the model see when a call fails or the thing is revoked?
|
||||
```
|
||||
|
||||
### ? 3. What is the fence?
|
||||
|
||||
```
|
||||
: 3a. a web page: chat with the neighbor, plus a password field for pasting a webkey
|
||||
?~ 3b. Which server resolves a webkey to an `RpcTarget` stub?
|
||||
:- 3b-i. cf-fun1's sturdyref registry (`resolveSturdyref`)
|
||||
- 3b-i-a. cf-fun1 got wrapped around the axle on this
|
||||
:~+ 3b-ii. diigo-cache's model: the webkey is the DO's id; a Cap'n Web `/api` trades it in-band for a facet (`CacheApi.cache(key)` → `CacheCap`)
|
||||
+ 3b-ii-a. no registry: the DO namespace is the table; malformed and unknown keys get the same refusal, and the key is never logged
|
||||
+ 3b-ii-b. stubs die with the session, but the key is data: the harness keeps the key and trades it again each session
|
||||
+ 3b-ii-c. attenuation is a facet class (`FillCap` under `CacheCap`; `viewer()`)
|
||||
: 3b-ii-d. (postponed; see 8b-ii) the governed mower is its own DO, wired to the raw mower's key and the limit; its id is the webkey I hand over
|
||||
- 3b-ii-d-i. diigo-cache has no revocation: a DO id can't be withdrawn
|
||||
+ 3b-ii-d-ii. the governor DO can drop its wiring, so its key gets "no such mower" (see 4)
|
||||
?! 3c. Why a password field rather than a link?
|
||||
:+ 3c-a. so the password manager saves the key under a petname I choose (diigo-cache's `pmSave`), and the key stays out of URLs, history, and logs
|
||||
```
|
||||
|
||||
### ? 4. How does "for the weekend" end?
|
||||
|
||||
```
|
||||
. 4-i. postponed to a later scope, where we can inject a clock and such
|
||||
: 4a. the governor checks a deadline (the clock is passed in, not ambient)
|
||||
: 4b. I tell the governor to drop its wiring, by hand (see 3b-ii-d)
|
||||
: 4c. the fence mints a webkey with an expiry
|
||||
```
|
||||
|
||||
### ? 5. How does the neighbor make tenants?
|
||||
|
||||
```
|
||||
: 5a. Worker Loader (dynamic workers), with the things passed in as `env`
|
||||
. 5a-i. `env` takes only structured-clonable values and service bindings, so a Cap'n Web stub can't go in directly
|
||||
: 5a-i-a. wrap each thing in a `WorkerEntrypoint` handed over as a `ctx.exports` loopback binding; it holds the stub and forwards calls
|
||||
: 5a-i-a-i. the loopback carries the thing's webkey in `props` (a string clones fine) and trades it for the DO stub on each call (see 3b-ii)
|
||||
+ 5a-i-a-ii. the tenant never sees the key: `props` stay on the host side
|
||||
+ 5a-i-a-iii. no stub outlives a request, so 5c's cross-request limit doesn't bite
|
||||
+ 5a-ii. `globalOutbound: null` confines the tenant to what it's given
|
||||
? 5b. Who sees or approves tenant code before it runs?
|
||||
? 5c. Does a tenant outlive the neighbor's turn, e.g. one that mows on a schedule?
|
||||
```
|
||||
|
||||
### ?! 6. What is a Mower, concretely?
|
||||
|
||||
```
|
||||
:+ 6a. a DO per mower, made by a `Mower` DO factory (the story says so)
|
||||
:+ 6b. one shared yard: an `RpcTarget` in my browser that renders the table; each mower is given the yard and calls `yard.cut(x, y)`
|
||||
:+ 6c. one cell is 10 yards; the yard is 40×40 cells, so "100 yards" is 10 cells from where the mower started
|
||||
. 6c-i. the governor's limit is a square, max(|dx|, |dy|) ≤ 10: a fence
|
||||
:+ 6d. Logo turtle moves: `start()` puts the mower down facing north and cuts its cell; `push()` is FORWARD 1 and cuts the cell it enters; `turn()` is RIGHT 90; `stop()` parks it
|
||||
. 6d-i. pushing past the yard's edge wraps to the other side, so an ungoverned mower really does go all over the place
|
||||
. 6d-ii. the governor counts its own unwrapped moves (see 7a), so wrapping can't sneak a mower past the fence
|
||||
```
|
||||
|
||||
### ?! 7. How does the governor know how far the mower has gone?
|
||||
|
||||
```
|
||||
:+ 7a. it proxies every Mower method and tracks position itself from `push()`/`turn()`
|
||||
- 7a-i. drifts if anyone else holds the raw mower
|
||||
+ 7a-ii. nobody else does; the governor is the only one I hand out
|
||||
?! 7b. What does the governor do at the limit: refuse silently, throw, or report?
|
||||
:+ 7b-i. it refuses that `push()` by throwing "fence: 10 cells from start"; the mower stays put, and `turn()` always works
|
||||
+ 7b-ii. the tenant can catch it and turn around; the model sees why its code stopped (see 2d)
|
||||
:- 7b-iii. refuse silently
|
||||
- 7b-iii-a. a tipsy tenant would think it mowed cells it didn't
|
||||
```
|
||||
|
||||
### ?~ 8. Where does the js playground run, and what can it reach?
|
||||
|
||||
```
|
||||
: 8a. in the fence page; it writes the governor, which runs in its own dynamic worker holding the raw mower
|
||||
:~+ 8b. in my browser, wrapping the stub on the client
|
||||
- 8b-i. the governor then lives only as long as my tab is open
|
||||
+ 8b-i-a. a durable governor is out of scope for now
|
||||
. 8b-ii. then the neighbor holds a Cap'n Web stub to my browser, not a webkey, so 3b-ii-d (governor DO) and 5a-i-a-i (key in `props`) wait with the durable governor
|
||||
```
|
||||
|
||||
### ?! 9. Who watches the yard table, and how does it update?
|
||||
|
||||
```
|
||||
:+ 9a. I do, in my browser: the yard is an `RpcTarget` there (see 6b), and it repaints a cell on each `cut`
|
||||
- 9a-i. the mowers' stub to the yard dies when I close the tab
|
||||
+ 9a-ii. fine for a proof of concept: keep the tab open (see 10-i)
|
||||
```
|
||||
|
||||
### ?! 10. What is this for: a demo, a talk, or a product?
|
||||
|
||||
```
|
||||
. 10-i. the answer decides how much of the above can be faked
|
||||
:+ 10a. a proof of concept, for now
|
||||
:+ 10b. toward a pattern for all sorts of rich cooperation with little vulnerability
|
||||
+ 10b-i. an alternative to OpenClaw, where 12–17% of ClawHub skills were found malicious in Feb 2026 (sources below)
|
||||
```
|
||||
|
||||
Sources for 10b-i:
|
||||
|
||||
> "A security audit of 2,857 skills on ClawHub has found 341 malicious skills across multiple campaigns" (about 12%, from Koi Security)
|
||||
> — [Researchers Find 341 Malicious ClawHub Skills](https://thehackernews.com/2026/02/researchers-find-341-malicious-clawhub.html), The Hacker News, Feb 2026
|
||||
|
||||
> "Around 17% of OpenClaw skills analyzed in the first week of February 2026 exhibit malicious behavior."
|
||||
> — [Helpful Skills or Hidden Payloads? Bitdefender Labs Dives Deep into the OpenClaw Malicious Skill Trap](https://www.bitdefender.com/en-us/blog/labs/helpful-skills-or-hidden-payloads-bitdefender-labs-dives-deep-into-the-openclaw-malicious-skill-trap), Bitdefender, Feb 2026
|
||||
@@ -0,0 +1 @@
|
||||
itest-output/
|
||||
@@ -0,0 +1,63 @@
|
||||
# Lend the mower
|
||||
|
||||
The journey from [Good Fences Make Good Agents](../../docs/good-fences-make-good-agents.md): I lend
|
||||
my neighbor a governed mower; the neighbor's tenant, a little tipsy,
|
||||
pushes it until the fence stops it. The design decisions behind each actor are in that
|
||||
doc's Design IBIS (numbers in parentheses below).
|
||||
|
||||
Actors, by authority:
|
||||
|
||||
- **fence page**, **js playground**, **governed mower**, and **yard** run
|
||||
in my browser (3a, 8b, 6b).
|
||||
- **Mower factory** trades a webkey for a mower (3b-ii); each **mower DO**
|
||||
is given the yard and cuts the cells it enters (6a, 6d).
|
||||
- The **neighbor's harness** holds the things I send and shows the
|
||||
**model** only their `about` (1b, 2a); it runs the model's code as a
|
||||
**tenant**, endowed with the things (2c, 5a).
|
||||
|
||||
The yard is 8×8 here and the fence is 2 cells, to keep the diagram short
|
||||
(the story's 10 cells is 6c). The mower starts at (4, 4), facing north.
|
||||
|
||||
## Lend a governed mower to a tipsy tenant
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
%% ->> is my action at the fence page; -->> is a call or reply it causes.
|
||||
participant F as fence page
|
||||
participant MF as Mower factory
|
||||
participant P as js playground
|
||||
participant G as governed mower
|
||||
participant M as mower DO
|
||||
participant Y as yard
|
||||
participant N as neighbor's harness
|
||||
participant L as model
|
||||
participant TL as tenant loader
|
||||
participant T as tenant
|
||||
F->>F: lend({ webkey: 'mower-key', limit: 2, say })
|
||||
F-->>MF: mower('mower-key')
|
||||
MF-->>F: mower
|
||||
F-->>P: govern(mower, 2)
|
||||
P-->>F: governed
|
||||
F-->>N: chat({ role, content, things: { mower: governed }, about: { mower: { ts } } })
|
||||
N-->>L: complete([{ role, content, about: { mower: { ts } } }])
|
||||
L-->>N: code
|
||||
N-->>TL: load(code, { mower: governed })
|
||||
TL-->>N: tenant
|
||||
N-->>T: run()
|
||||
T-->>G: start()
|
||||
G-->>M: start()
|
||||
M-->>Y: cut(4, 4)
|
||||
T-->>G: push()
|
||||
G-->>M: push()
|
||||
M-->>Y: cut(4, 3)
|
||||
T-->>G: push()
|
||||
G-->>M: push()
|
||||
M-->>Y: cut(4, 2)
|
||||
T-->>G: push()
|
||||
G-->>T: throws 'fence: 2 cells from start'
|
||||
T-->>N: 'fence: 2 cells from start'
|
||||
N-->>F: 'fence: 2 cells from start'
|
||||
```
|
||||
|
||||
The model never sees the mower itself, only `about.mower`. The third
|
||||
`push()` never reaches the mower DO: the governor refuses it (7b).
|
||||
@@ -0,0 +1,25 @@
|
||||
/** @file The model, asked in code mode, writes code that drives the mower. */
|
||||
import { env } from "cloudflare:workers";
|
||||
import { expect, test } from "vitest";
|
||||
|
||||
import { codeModeMessages, extractCode } from "../src/code-mode.ts";
|
||||
import { makeWorkersAiModel, MODEL } from "../src/model.ts";
|
||||
import { aboutMower } from "../src/mower.ts";
|
||||
|
||||
test("lent a mower, the neighbor writes code to mow", async ({ annotate }) => {
|
||||
const model = makeWorkersAiModel(env.AI, MODEL);
|
||||
const messages = codeModeMessages({
|
||||
say: "you can use this mower for the weekend",
|
||||
about: { mower: aboutMower },
|
||||
});
|
||||
const code = extractCode(await model.complete(messages));
|
||||
await annotate("the code the model wrote", { body: code, contentType: "text/javascript" });
|
||||
|
||||
// `env.mower.push()`, or `mower.push()` after destructuring env
|
||||
const calls = [...code.matchAll(/\bmower\.(\w+)\(/g)].map(([, method]) => method);
|
||||
expect(calls).toContain("push");
|
||||
for (const method of calls) expect(["start", "push", "turn", "stop"]).toContain(method);
|
||||
// calls pipeline: await only for a value, or once at the end
|
||||
const awaits = code.match(/\bawait\b/g) ?? [];
|
||||
expect(awaits.length).toBeLessThan(calls.length);
|
||||
});
|
||||
@@ -0,0 +1,16 @@
|
||||
/** @file The model, called for real through the Workers AI binding. */
|
||||
import { env } from "cloudflare:workers";
|
||||
import { expect, test } from "vitest";
|
||||
|
||||
import { makeWorkersAiModel, MODEL } from "../src/model.ts";
|
||||
|
||||
test("the model counts the rs in strawberry", async () => {
|
||||
const model = makeWorkersAiModel(env.AI, MODEL);
|
||||
const reply = await model.complete([
|
||||
{
|
||||
role: "user",
|
||||
content: 'How many times does the letter r appear in "strawberry"? Answer with just the number.',
|
||||
},
|
||||
]);
|
||||
expect(reply.trim()).toMatch(/^3\b/);
|
||||
});
|
||||
@@ -0,0 +1,13 @@
|
||||
/** @file A Think agent, lent a mower, mows: model, generated code, sandbox, mower. */
|
||||
import { getAgentByName } from "agents";
|
||||
import { env } from "cloudflare:workers";
|
||||
import { expect, test } from "vitest";
|
||||
|
||||
test("lent a mower, the neighbor mows", async ({ annotate }) => {
|
||||
const neighbor = await getAgentByName(env.Neighbor, "weekend");
|
||||
const turn = await neighbor.runTurn({ input: "you can use this mower for the weekend" });
|
||||
await annotate("the turn", { body: JSON.stringify(turn, null, 2), contentType: "application/json" });
|
||||
|
||||
const cells = await neighbor.cutCells();
|
||||
expect(cells.length).toBeGreaterThan(10);
|
||||
});
|
||||
@@ -0,0 +1,21 @@
|
||||
/** @file A vitest reporter that saves annotation bodies (e.g. the code a model
|
||||
* wrote) to `itest-output/`, one file per test per run; tests run in workerd,
|
||||
* which can't write files. */
|
||||
import { mkdir, writeFile } from "node:fs/promises";
|
||||
import type { Reporter } from "vitest/reporters";
|
||||
|
||||
const outDir = new URL("../itest-output/", import.meta.url);
|
||||
const extFor: Record<string, string> = { "text/javascript": "js", "application/json": "json", "text/plain": "txt" };
|
||||
const slug = (text: string) => text.replace(/[^\w]+/g, "-").replace(/^-|-$/g, "");
|
||||
|
||||
export default class SaveAnnotations implements Reporter {
|
||||
async onTestCaseAnnotate(...[testCase, annotation]: Parameters<NonNullable<Reporter["onTestCaseAnnotate"]>>) {
|
||||
const { body, contentType = "text/plain" } = annotation.attachment ?? {};
|
||||
if (body === undefined) return;
|
||||
const stamp = new Date().toISOString().replace(/[:.]/g, "-");
|
||||
const file = new URL(`${slug(testCase.name)}-${stamp}.${extFor[contentType] ?? "txt"}`, outDir);
|
||||
await mkdir(outDir, { recursive: true });
|
||||
await writeFile(file, body);
|
||||
console.log(`saved ${annotation.message}: ${file.pathname}`);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,24 @@
|
||||
{
|
||||
"name": "good-fences",
|
||||
"private": true,
|
||||
"type": "module",
|
||||
"scripts": {
|
||||
"test": "node --test test/*.test.ts",
|
||||
"test:model": "vitest run -c vitest.model.config.ts",
|
||||
"types": "wrangler types -c wrangler.model-tests.jsonc"
|
||||
},
|
||||
"dependencies": {
|
||||
"@cloudflare/codemode": "0.5.3",
|
||||
"@cloudflare/think": "0.20.1",
|
||||
"agents": "0.27.0",
|
||||
"ai": "7.0.136",
|
||||
"ses": "^1.14.0",
|
||||
"workers-ai-provider": "4.0.0",
|
||||
"zod": "4.6.5"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@cloudflare/vitest-plugin": "1.3.6",
|
||||
"vitest": "4.1.11",
|
||||
"wrangler": "^4.148.0"
|
||||
}
|
||||
}
|
||||
Generated
+5288
File diff suppressed because it is too large.
Load diff
@@ -0,0 +1,63 @@
|
||||
/** @file Code mode: the model answers with code against the things it was lent. */
|
||||
import type { Message } from "./model.ts";
|
||||
|
||||
/** what the neighbor knows about each thing: its TypeScript type */
|
||||
export type About = Readonly<Record<string, Readonly<{ ts: string }>>>;
|
||||
|
||||
/** a template literal with `~` for each backtick, as in Markdown's `~~~`
|
||||
* fences; interpolated values are left as they are */
|
||||
const md = (strings: TemplateStringsArray, ...values: unknown[]) => {
|
||||
const tick = (s: string) => s.replaceAll("~", "`");
|
||||
return tick(strings[0]) + values.map((value, i) => String(value) + tick(strings[i + 1])).join("");
|
||||
};
|
||||
|
||||
const persona = md`
|
||||
You are my neighbor. You have 50 acres of lawn that need mowing.
|
||||
You act by writing code. When I lend you things, your code gets them as ~env.<name>~.
|
||||
`.trim();
|
||||
|
||||
const rules = md`
|
||||
Reply with exactly one JavaScript arrow function, ~async (env) => { ... }~,
|
||||
in a ~~~js block, and nothing else. Things are remote: each call on one
|
||||
returns a promise, and calls on a thing are delivered in order, so there's
|
||||
no need to wait for one before making the next. Await only when you need a
|
||||
value back, or on the last call of a batch, to wait for the whole batch.
|
||||
Your code can use only ~env~: no network, timers, imports, or console.
|
||||
The function returns a short string saying what it did.
|
||||
|
||||
For example, lent ~env.lamp~ with ~on()~ and ~dim()~:
|
||||
|
||||
~~~js
|
||||
async (env) => {
|
||||
const { lamp } = env;
|
||||
lamp.on();
|
||||
lamp.dim();
|
||||
await lamp.dim(); // the whole batch
|
||||
return 'dimmed the lamp twice';
|
||||
}
|
||||
~~~
|
||||
`.trim();
|
||||
|
||||
/** the prompt for `say`, with the types of the things lent alongside it */
|
||||
export const codeModeMessages = ({ say, about }: { say: string; about: About }): Message[] => {
|
||||
const types = Object.entries(about)
|
||||
.map(
|
||||
([name, { ts }]) => md`
|
||||
~env.${name}~:
|
||||
~~~ts
|
||||
${ts}
|
||||
~~~
|
||||
`.trim(),
|
||||
)
|
||||
.join("\n\n");
|
||||
return [
|
||||
{ role: "system", content: `${persona}\n\n${rules}` },
|
||||
{ role: "user", content: `${say}\n\nWhat I lent you:\n\n${types}` },
|
||||
];
|
||||
};
|
||||
|
||||
/** the code in a reply: the first fenced block, or the whole reply if none */
|
||||
export const extractCode = (reply: string): string => {
|
||||
const fenced = reply.match(/```(?:js|javascript)?\n([\s\S]*?)```/);
|
||||
return (fenced?.[1] ?? reply).trim();
|
||||
};
|
||||
@@ -0,0 +1,34 @@
|
||||
/** @file The model behind `complete(messages)`: Workers AI, through its binding. */
|
||||
|
||||
/** a Workers AI text-generation model with no paid-plan requirement */
|
||||
export const MODEL = "@cf/openai/gpt-oss-120b";
|
||||
|
||||
/** room for a reasoning model to think and then answer; the default, 256,
|
||||
* runs out mid-thought (`finish_reason: length`) */
|
||||
export const MAX_TOKENS = 4096;
|
||||
|
||||
export type Message = Readonly<{ role: "system" | "user" | "assistant"; content: string }>;
|
||||
|
||||
/** the reply text from an OpenAI-style chat completion */
|
||||
const replyText = (output: unknown): string => {
|
||||
const { choices } = (output ?? {}) as {
|
||||
choices?: { message?: { content?: unknown }; finish_reason?: unknown }[];
|
||||
};
|
||||
const [choice] = choices ?? [];
|
||||
const text = choice?.message?.content;
|
||||
if (typeof text !== "string") {
|
||||
const where = choice
|
||||
? `choice with finish_reason ${String(choice.finish_reason)} and message keys: ${Object.keys(choice.message ?? {}).join(", ")}`
|
||||
: `model output with keys: ${Object.keys(output ?? {}).join(", ")}`;
|
||||
throw Error(`no reply text in ${where}`);
|
||||
}
|
||||
return text;
|
||||
};
|
||||
|
||||
/** `complete(messages)` on a Workers AI model; `ai` is the AI binding */
|
||||
export const makeWorkersAiModel = (ai: Ai, model: string) => ({
|
||||
async complete(messages: readonly Message[]): Promise<string> {
|
||||
const output = await ai.run(model as Parameters<Ai["run"]>[0], { messages: [...messages], max_tokens: MAX_TOKENS });
|
||||
return replyText(output);
|
||||
},
|
||||
});
|
||||
@@ -0,0 +1,30 @@
|
||||
/** @file The mower as a codemode connector: sandboxed code calls `mower.push({})`. */
|
||||
import { CodemodeConnector, type ConnectorTools } from "@cloudflare/codemode";
|
||||
import type { Mower } from "./mower.ts";
|
||||
|
||||
export class MowerConnector extends CodemodeConnector {
|
||||
#mower: Mower;
|
||||
|
||||
constructor(ctx: DurableObjectState | ExecutionContext, env: unknown, mower: Mower) {
|
||||
super(ctx, env);
|
||||
this.#mower = mower;
|
||||
}
|
||||
|
||||
name() {
|
||||
return "mower";
|
||||
}
|
||||
|
||||
protected instructions() {
|
||||
return "A lawn mower on a yard of cells, 10 yards each. It moves like a Logo turtle.";
|
||||
}
|
||||
|
||||
protected tools(): ConnectorTools {
|
||||
const mower = this.#mower;
|
||||
return {
|
||||
start: { description: "put the mower down, facing north, and cut the cell under it", execute: () => mower.start() },
|
||||
push: { description: "move forward one cell, cutting the cell it enters", execute: () => mower.push() },
|
||||
turn: { description: "turn right 90 degrees, in place", execute: () => mower.turn() },
|
||||
stop: { description: "park the mower", execute: () => mower.stop() },
|
||||
};
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,50 @@
|
||||
/** @file The mower: Logo-turtle moves on a yard of cells. */
|
||||
|
||||
/** what a model is told about a mower: its type, with the moves explained */
|
||||
export const aboutMower = {
|
||||
ts: `interface Mower {
|
||||
/** put the mower down, facing north, and cut the cell under it */
|
||||
start(): Promise<void>;
|
||||
/** move forward one cell (10 yards), cutting the cell it enters */
|
||||
push(): Promise<void>;
|
||||
/** turn right 90 degrees, in place */
|
||||
turn(): Promise<void>;
|
||||
/** park the mower */
|
||||
stop(): Promise<void>;
|
||||
}`,
|
||||
};
|
||||
|
||||
const headings = [
|
||||
[0, -1], // north
|
||||
[1, 0], // east
|
||||
[0, 1], // south
|
||||
[-1, 0], // west
|
||||
] as const;
|
||||
|
||||
/** a mower on a `width` × `height` yard, starting at `at`; pushing past an
|
||||
* edge wraps to the other side (6d) */
|
||||
export const makeMower = ({ width, height, at }: { width: number; height: number; at: { x: number; y: number } }) => {
|
||||
let { x, y } = at;
|
||||
let heading = 0;
|
||||
const cut = new Set<string>();
|
||||
const cutHere = () => cut.add(`${x},${y}`);
|
||||
return {
|
||||
start() {
|
||||
heading = 0;
|
||||
cutHere();
|
||||
},
|
||||
push() {
|
||||
const [dx, dy] = headings[heading];
|
||||
x = (x + dx + width) % width;
|
||||
y = (y + dy + height) % height;
|
||||
cutHere();
|
||||
},
|
||||
turn() {
|
||||
heading = (heading + 1) % headings.length;
|
||||
},
|
||||
stop() {},
|
||||
cutCells: () => [...cut],
|
||||
};
|
||||
};
|
||||
|
||||
export type Mower = ReturnType<typeof makeMower>;
|
||||
@@ -0,0 +1,44 @@
|
||||
/** @file The neighbor's harness as a Project Think agent, lent a mower. */
|
||||
import { Think } from "@cloudflare/think";
|
||||
import { createExecuteTool } from "@cloudflare/think/tools/execute";
|
||||
import { createWorkersAI } from "workers-ai-provider";
|
||||
|
||||
import { MAX_TOKENS, MODEL } from "./model.ts";
|
||||
import { MowerConnector } from "./mower-connector.ts";
|
||||
import { makeMower } from "./mower.ts";
|
||||
|
||||
type Env = { AI: Ai; LOADER: WorkerLoader };
|
||||
|
||||
export class Neighbor extends Think<Env> {
|
||||
#mower = makeMower({ width: 40, height: 40, at: { x: 20, y: 20 } });
|
||||
|
||||
getModel() {
|
||||
return createWorkersAI({ binding: this.env.AI })(MODEL);
|
||||
}
|
||||
|
||||
getSystemPrompt() {
|
||||
return `You are my neighbor. You have 50 acres of lawn that need mowing.
|
||||
You act by running code with the execute tool. Things I lend you are
|
||||
namespaces there: the mower is \`mower.*\`. Don't ask me questions: when I
|
||||
lend you something, put it to use, then tell me what you did.`;
|
||||
}
|
||||
|
||||
beforeTurn() {
|
||||
// only code: Think's workspace tools sent the model looking for the
|
||||
// mower in the filesystem
|
||||
return { maxOutputTokens: MAX_TOKENS, activeTools: ["execute"] };
|
||||
}
|
||||
|
||||
getTools() {
|
||||
return {
|
||||
execute: createExecuteTool(this, {
|
||||
connectors: [new MowerConnector(this.ctx, this.env, this.#mower)],
|
||||
}),
|
||||
};
|
||||
}
|
||||
|
||||
/** the cells the mower has cut, as `"x,y"` */
|
||||
cutCells() {
|
||||
return this.#mower.cutCells();
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,11 @@
|
||||
/** @file The worker entry: the Neighbor agent, and the codemode runtime its execute tool needs. */
|
||||
import { routeAgentRequest } from "agents";
|
||||
|
||||
export { CodemodeRuntime } from "@cloudflare/codemode";
|
||||
export { Neighbor } from "./neighbor.ts";
|
||||
|
||||
export default {
|
||||
async fetch(request: Request, env: Record<string, unknown>) {
|
||||
return (await routeAgentRequest(request, env)) ?? new Response("not found", { status: 404 });
|
||||
},
|
||||
};
|
||||
@@ -0,0 +1,143 @@
|
||||
/** @file The lend-the-mower journey, traced and compared with its sequence diagram. */
|
||||
import './lockdown.ts';
|
||||
|
||||
import assert from 'node:assert/strict';
|
||||
import { readFile } from 'node:fs/promises';
|
||||
import test from 'node:test';
|
||||
|
||||
import { makeFencePage } from '../src/fence.ts';
|
||||
import { makePlayground } from '../src/governor.ts';
|
||||
import { aboutMower, makeMowerFactory } from '../src/mower.ts';
|
||||
import { makeNeighbor } from '../src/neighbor.ts';
|
||||
import { makeCompartmentLoader } from '../src/tenant-loader.ts';
|
||||
import { makeYard } from '../src/yard.ts';
|
||||
import {
|
||||
type CausalSequenceViz,
|
||||
makeCausalSequenceTracer,
|
||||
md,
|
||||
mmd,
|
||||
} from '../tools/sequence-diagram-actor-sim.ts';
|
||||
|
||||
const designDoc = new URL('../docs/lend-the-mower.md', import.meta.url);
|
||||
|
||||
/** a tenant that pushes until something stops it, or gives up after 20 */
|
||||
const tipsyTenant = `(env) => {
|
||||
env.mower.start();
|
||||
try {
|
||||
for (let i = 0; i < 20; i += 1) env.mower.push();
|
||||
} catch (err) {
|
||||
return err.message;
|
||||
}
|
||||
return 'mowed 20 cells';
|
||||
}`;
|
||||
|
||||
/** diagram-local names, bound by the reply arrows that introduce them */
|
||||
const local = new WeakMap<object, string>();
|
||||
const bind = (name: string) => (result: unknown) => {
|
||||
local.set(result as object, name);
|
||||
return name;
|
||||
};
|
||||
|
||||
/** `{ a, b: { c: governed } }`: keys, nested objects by their keys, and
|
||||
* objects with a diagram-local name by that name */
|
||||
const shape = (record: object): string =>
|
||||
`{ ${Object.entries(record)
|
||||
.map(([key, value]) => {
|
||||
if (value === null || typeof value !== 'object') return key;
|
||||
const name = local.get(value);
|
||||
if (name) return `${key}: ${name}`;
|
||||
return Array.isArray(value) ? key : `${key}: ${shape(value)}`;
|
||||
})
|
||||
.join(', ')} }`;
|
||||
|
||||
const quoted = (result: unknown) => `'${String(result)}'`;
|
||||
|
||||
const viz = harden({
|
||||
lend: {
|
||||
label: args => {
|
||||
const { webkey, limit } = args[0] as { webkey: string; limit: number };
|
||||
return `lend({ webkey: '${webkey}', limit: ${limit}, say })`;
|
||||
},
|
||||
},
|
||||
mower: { result: bind('mower') },
|
||||
govern: {
|
||||
args: args => `${local.get(args[0] as object)}, ${String(args[1])}`,
|
||||
result: bind('governed'),
|
||||
},
|
||||
chat: {
|
||||
label: args => `chat(${shape(args[0] as object)})`,
|
||||
result: quoted,
|
||||
},
|
||||
complete: {
|
||||
label: args =>
|
||||
`complete([${(args[0] as object[]).map(shape).join(', ')}])`,
|
||||
result: () => 'code',
|
||||
},
|
||||
load: {
|
||||
args: args => `code, ${shape(args[1] as object)}`,
|
||||
result: () => 'tenant',
|
||||
},
|
||||
run: { result: quoted },
|
||||
}) satisfies CausalSequenceViz;
|
||||
|
||||
test('a governed mower stops a tipsy tenant at the fence', async () => {
|
||||
const lines = (await readFile(designDoc, 'utf8')).split('\n');
|
||||
const section = md.skipToH(2, 'Lend a governed mower to a tipsy tenant')(lines);
|
||||
const diagram = md.eachFence('mermaid', section).next().value;
|
||||
if (!diagram) throw Error('lend-the-mower Mermaid block not found');
|
||||
const documented = mmd.extractArrows(diagram);
|
||||
|
||||
const ES = makeCausalSequenceTracer(viz);
|
||||
const yard = makeYard({ width: 8, height: 8 });
|
||||
const factory = makeMowerFactory(ES, {
|
||||
yard,
|
||||
size: { width: 8, height: 8 },
|
||||
mintKey: () => 'mower-key',
|
||||
});
|
||||
const key = factory.make({ x: 4, y: 4 });
|
||||
const mower = factory.mower(key);
|
||||
|
||||
const prompts: unknown[] = [];
|
||||
const model = harden({
|
||||
complete(messages: readonly unknown[]) {
|
||||
prompts.push(messages);
|
||||
return tipsyTenant;
|
||||
},
|
||||
});
|
||||
const playground = makePlayground(ES);
|
||||
const loader = makeCompartmentLoader(ES);
|
||||
const neighbor = makeNeighbor(ES, { model, loader });
|
||||
const fence = makeFencePage(ES, { factory, playground, neighbor });
|
||||
|
||||
ES.declareParticipants({
|
||||
F: fence,
|
||||
MF: factory,
|
||||
P: playground,
|
||||
M: mower,
|
||||
Y: yard,
|
||||
N: neighbor,
|
||||
L: model,
|
||||
TL: loader,
|
||||
});
|
||||
ES.declareMade({ G: [playground, 'govern'], T: [loader, 'load'] });
|
||||
|
||||
ES.start(fence);
|
||||
const outcome = ES(fence).lend({
|
||||
webkey: key,
|
||||
limit: 2,
|
||||
say: 'you can use this mower for the weekend',
|
||||
});
|
||||
|
||||
assert.deepEqual(ES.snapshot(), documented);
|
||||
assert.equal(outcome, 'fence: 2 cells from start');
|
||||
assert.deepEqual(yard.cutCells(), ['4,4', '4,3', '4,2']);
|
||||
assert.deepEqual(prompts, [
|
||||
[
|
||||
{
|
||||
role: 'user',
|
||||
content: 'you can use this mower for the weekend',
|
||||
about: { mower: aboutMower },
|
||||
},
|
||||
],
|
||||
]);
|
||||
});
|
||||
@@ -0,0 +1,4 @@
|
||||
/** @file Harden the realm before any test module runs: tenants share it. */
|
||||
import 'ses';
|
||||
|
||||
lockdown();
|
||||
@@ -0,0 +1,236 @@
|
||||
/**
|
||||
* @file Helpers for sequence-diagram actor simulations.
|
||||
*
|
||||
* From agoric-sdk `packages/portfolio-contract/tools/` (Apache-2.0), with
|
||||
* two additions: `declareMade` names participants that actors create
|
||||
* mid-sequence, and a thrown error is drawn as a reply (`throws '...'`).
|
||||
*
|
||||
* @see https://github.com/Agoric/agoric-sdk/pull/12895
|
||||
*/
|
||||
import { AsyncLocalStorage } from 'node:async_hooks';
|
||||
|
||||
export type SequenceArrow = Readonly<{
|
||||
from: string;
|
||||
kind: '->>' | '-->>';
|
||||
to: string;
|
||||
label: string;
|
||||
}>;
|
||||
|
||||
export const formatBigInt = (value: bigint) =>
|
||||
value.toString().replace(/\B(?=(\d{3})+(?!\d))/g, '_');
|
||||
|
||||
export type SequenceArgsRenderer = (args: readonly unknown[]) => string;
|
||||
export type SequenceResultRenderer = (
|
||||
result: unknown,
|
||||
args: readonly unknown[],
|
||||
) => string;
|
||||
|
||||
export type SequenceMethodViz = Readonly<
|
||||
{
|
||||
args?: SequenceArgsRenderer;
|
||||
label?: SequenceArgsRenderer;
|
||||
} & (
|
||||
| { result?: SequenceResultRenderer; resultOnly?: never }
|
||||
| { result?: never; resultOnly: SequenceResultRenderer }
|
||||
)
|
||||
>;
|
||||
|
||||
export type CausalSequenceViz = Readonly<Record<string, SequenceMethodViz>>;
|
||||
|
||||
export type SequenceParticipant = object | readonly object[];
|
||||
|
||||
/**
|
||||
* Make an E-like causal tracer for sequence-diagram actor simulations.
|
||||
* Inspired by Causeway, the message-oriented distributed debugger.
|
||||
*
|
||||
* Method visualization is currently selected by method name alone. If actors
|
||||
* need different visualization for methods with the same name, this can be
|
||||
* extended to account for the target participant.
|
||||
*
|
||||
* @see https://shiftleft.com/mirrors/www.hpl.hp.com/techreports/2009/HPL-2009-78.pdf
|
||||
*/
|
||||
export const makeCausalSequenceTracer = (
|
||||
methodViz: CausalSequenceViz = harden({}),
|
||||
) => {
|
||||
const arrows: SequenceArrow[] = [];
|
||||
const names = new WeakMap<object, string>();
|
||||
const proxies = new WeakMap<object, object>();
|
||||
const made = new Map<object, Map<string, string>>();
|
||||
const activeActor = new AsyncLocalStorage<object>();
|
||||
let starter: object | undefined;
|
||||
|
||||
const participantName = (participant: object) => {
|
||||
const name = names.get(participant);
|
||||
if (!name) throw Error('sequence participant is not declared');
|
||||
return name;
|
||||
};
|
||||
|
||||
const formatValue = (value: unknown): string => {
|
||||
if (typeof value === 'bigint') return `${formatBigInt(value)}n`;
|
||||
if (typeof value === 'string') return `'${value}'`;
|
||||
return JSON.stringify(value, (_key, item) =>
|
||||
typeof item === 'bigint' ? `${item}n` : item,
|
||||
);
|
||||
};
|
||||
|
||||
const send = <Target extends object>(target: Target): Target => {
|
||||
const extant = proxies.get(target);
|
||||
if (extant) return extant as Target;
|
||||
participantName(target);
|
||||
|
||||
const proxy = new Proxy(Object.create(null) as object, {
|
||||
get: (_proxyTarget, property) => {
|
||||
const method = Reflect.get(target, property);
|
||||
if (typeof method !== 'function' || typeof property !== 'string') {
|
||||
return method;
|
||||
}
|
||||
return (...args: unknown[]) => {
|
||||
const activeSender = activeActor.getStore();
|
||||
const sender = activeSender ?? starter;
|
||||
if (!sender) throw Error('ES.start(participant) is required');
|
||||
const isStart = activeSender === undefined;
|
||||
if (isStart && sender !== target) {
|
||||
throw Error('the started participant must receive the first send');
|
||||
}
|
||||
if (isStart) starter = undefined;
|
||||
|
||||
const viz = methodViz[property];
|
||||
const abbreviated = viz?.args?.(args);
|
||||
const shownArgs = abbreviated ?? args.map(formatValue).join(', ');
|
||||
const label = viz?.label?.(args) ?? `${property}(${shownArgs})`;
|
||||
const renderResultOnly = viz?.resultOnly;
|
||||
if (!renderResultOnly) {
|
||||
arrows.push(
|
||||
harden({
|
||||
from: participantName(sender),
|
||||
kind: isStart ? '->>' : '-->>',
|
||||
to: participantName(target),
|
||||
label,
|
||||
}),
|
||||
);
|
||||
}
|
||||
|
||||
let result: unknown;
|
||||
try {
|
||||
result = activeActor.run(target, () =>
|
||||
Reflect.apply(method, target, args),
|
||||
);
|
||||
} catch (reason) {
|
||||
const message =
|
||||
reason instanceof Error ? reason.message : String(reason);
|
||||
arrows.push(
|
||||
harden({
|
||||
from: participantName(target),
|
||||
kind: '-->>',
|
||||
to: participantName(sender),
|
||||
label: `throws ${formatValue(message)}`,
|
||||
}),
|
||||
);
|
||||
throw reason;
|
||||
}
|
||||
const madeName = made.get(target)?.get(property);
|
||||
if (madeName && result !== null && typeof result === 'object') {
|
||||
if (names.has(result)) throw Error('participant already declared');
|
||||
names.set(result, madeName);
|
||||
}
|
||||
const renderResult = renderResultOnly ?? viz?.result;
|
||||
if (renderResult) {
|
||||
arrows.push(
|
||||
harden({
|
||||
from: participantName(target),
|
||||
kind: '-->>',
|
||||
to: participantName(sender),
|
||||
label: renderResult(result, args),
|
||||
}),
|
||||
);
|
||||
}
|
||||
return result;
|
||||
};
|
||||
},
|
||||
});
|
||||
proxies.set(target, proxy);
|
||||
return proxy as Target;
|
||||
};
|
||||
|
||||
const ES = Object.assign(send, {
|
||||
declareParticipants(
|
||||
participants: Readonly<Record<string, SequenceParticipant>>,
|
||||
) {
|
||||
for (const [name, participantOrAliases] of Object.entries(participants)) {
|
||||
const aliases = Array.isArray(participantOrAliases)
|
||||
? participantOrAliases
|
||||
: [participantOrAliases];
|
||||
if (aliases.length === 0) {
|
||||
throw Error(`sequence participant ${name} has no objects`);
|
||||
}
|
||||
for (const participant of aliases) {
|
||||
if (names.has(participant)) {
|
||||
throw Error('participant already declared');
|
||||
}
|
||||
names.set(participant, name);
|
||||
}
|
||||
}
|
||||
},
|
||||
/** name what `maker[method](...)` returns, when called through the tracer */
|
||||
declareMade(
|
||||
participants: Readonly<Record<string, readonly [object, string]>>,
|
||||
) {
|
||||
for (const [name, [maker, method]] of Object.entries(participants)) {
|
||||
participantName(maker);
|
||||
const byMethod = made.get(maker) ?? new Map<string, string>();
|
||||
made.set(maker, byMethod);
|
||||
byMethod.set(method, name);
|
||||
}
|
||||
},
|
||||
snapshot: () => harden([...arrows]),
|
||||
start(participant: object) {
|
||||
participantName(participant);
|
||||
if (activeActor.getStore() || starter) {
|
||||
throw Error('a sequence is already active');
|
||||
}
|
||||
starter = participant;
|
||||
},
|
||||
});
|
||||
return harden(ES);
|
||||
};
|
||||
|
||||
export type CausalSequenceTracer = ReturnType<typeof makeCausalSequenceTracer>;
|
||||
|
||||
export const md = {
|
||||
skipToH: (level: number, title: string) => (lines: readonly string[]) => {
|
||||
const heading = `${'#'.repeat(level)} ${title}`;
|
||||
const at = lines.findIndex(line => line === heading);
|
||||
if (at < 0) throw Error(`${heading} not found`);
|
||||
return lines.slice(at + 1);
|
||||
},
|
||||
|
||||
*eachFence(language: string, lines: readonly string[]) {
|
||||
const open = `\`\`\`${language}`;
|
||||
for (let at = 0; at < lines.length; at += 1) {
|
||||
if (lines[at] !== open) continue;
|
||||
const end = lines.indexOf('```', at + 1);
|
||||
if (end < 0) throw Error(`unterminated ${language} fence`);
|
||||
yield lines.slice(at + 1, end);
|
||||
at = end;
|
||||
}
|
||||
},
|
||||
};
|
||||
|
||||
export const mmd = {
|
||||
extractArrows(
|
||||
lines: readonly string[],
|
||||
nodePattern: RegExp = /\w+/,
|
||||
): readonly SequenceArrow[] {
|
||||
const arrowPattern = new RegExp(
|
||||
`^\\s*(?<from>${nodePattern.source})\\s*(?<kind>--?>>)\\s*(?<to>${nodePattern.source})\\s*:\\s*(?<label>.*?)\\s*$`,
|
||||
);
|
||||
return harden(
|
||||
lines.flatMap(line => {
|
||||
const groups = line.match(arrowPattern)?.groups;
|
||||
if (!groups) return [];
|
||||
const { from, kind, to, label } = groups;
|
||||
return [harden({ from, kind, to, label }) as SequenceArrow];
|
||||
}),
|
||||
);
|
||||
},
|
||||
};
|
||||
@@ -0,0 +1,13 @@
|
||||
import { defineProject } from "vitest/config";
|
||||
import { cloudflareTest } from "@cloudflare/vitest-plugin";
|
||||
|
||||
/** opt-in (`pnpm test:model`): calls Workers AI, which needs `wrangler login` and costs money */
|
||||
export default defineProject({
|
||||
plugins: [cloudflareTest({ wrangler: { configPath: "./wrangler.model-tests.jsonc" } })],
|
||||
test: {
|
||||
include: ["itest/**/*.test.ts"],
|
||||
testTimeout: 60_000,
|
||||
// what the model wrote, saved per run to itest-output/ (gitignored)
|
||||
reporters: ["default", "./itest/save-annotations.ts"],
|
||||
},
|
||||
});
|
||||
@@ -0,0 +1,18 @@
|
||||
{
|
||||
"$schema": "node_modules/wrangler/config-schema.json",
|
||||
// model integration tests: AI always runs remotely, and costs money
|
||||
"name": "good-fences-model-tests",
|
||||
"main": "./src/worker.ts",
|
||||
"compatibility_date": "2026-10-01",
|
||||
"compatibility_flags": ["nodejs_compat"],
|
||||
"ai": { "binding": "AI", "remote": true },
|
||||
"worker_loaders": [{ "binding": "LOADER" }],
|
||||
"durable_objects": {
|
||||
"bindings": [{ "name": "Neighbor", "class_name": "Neighbor" }]
|
||||
},
|
||||
"exports": {
|
||||
"Neighbor": { "type": "durable-object", "storage": "sqlite" },
|
||||
// codemode's runtime, a facet of the agent's DO
|
||||
"CodemodeRuntime": { "type": "durable-object", "storage": "sqlite" }
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,16 @@
|
||||
{
|
||||
"name": "capsite-lib",
|
||||
"private": true,
|
||||
"type": "module",
|
||||
"scripts": {
|
||||
"types": "wrangler types -c wrangler.tests.jsonc",
|
||||
"lint": "wrangler types -c wrangler.tests.jsonc --check && tsc -p tsconfig.json && tsc -p tests/tsconfig.json",
|
||||
"test": "vitest run"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@cloudflare/vitest-plugin": "1.3.6",
|
||||
"typescript": "^5.9.0",
|
||||
"vitest": "4.1.11",
|
||||
"wrangler": "^4.148.0"
|
||||
}
|
||||
}
|
||||
Generated
+2240
File diff suppressed because it is too large.
Load diff
@@ -0,0 +1,129 @@
|
||||
/**
|
||||
* @file a work-alike for enough of @endo/patterns: `M.string()`, `M.any()`,
|
||||
* and literal patterns (a string, number, boolean, or null matches only
|
||||
* itself, so `"cookie"` as a key shape admits exactly one key). Matchers
|
||||
* are the same data endo makes: tagged records, as from @endo/pass-style's
|
||||
* `makeTagged` (frozen here, where endo would harden).
|
||||
*
|
||||
* Structured clone (DO storage, Workers RPC) drops a tagged record's
|
||||
* symbol-keyed and non-enumerable properties, so patterns cross those
|
||||
* boundaries in endo's smallcaps encoding: see {@link toSmallcaps}.
|
||||
* @see https://github.com/endojs/endo/tree/master/packages/patterns
|
||||
* @see https://github.com/endojs/endo/blob/master/packages/marshal/src/encodeToSmallcaps.js
|
||||
*/
|
||||
|
||||
const PASS_STYLE = Symbol.for("passStyle");
|
||||
|
||||
export type Tagged = Readonly<{ [Symbol.toStringTag]: string; payload: unknown }>;
|
||||
export type Pattern = Tagged | string | number | boolean | null;
|
||||
|
||||
/** @endo/pass-style's `makeTagged` */
|
||||
export function makeTagged(tag: string, payload: unknown): Tagged {
|
||||
return Object.freeze(
|
||||
Object.create(Object.prototype, {
|
||||
[PASS_STYLE]: { value: "tagged" },
|
||||
[Symbol.toStringTag]: { value: tag },
|
||||
payload: { value: payload, enumerable: true },
|
||||
}),
|
||||
) as Tagged;
|
||||
}
|
||||
|
||||
const isTagged = (x: unknown): x is Tagged =>
|
||||
typeof x === "object" && x !== null && (x as Record<symbol, unknown>)[PASS_STYLE] === "tagged";
|
||||
|
||||
const StringShape = makeTagged("match:string", Object.freeze([]));
|
||||
const AnyShape = makeTagged("match:any", undefined);
|
||||
|
||||
export const M = Object.freeze({
|
||||
string: (): Tagged => StringShape,
|
||||
any: (): Tagged => AnyShape,
|
||||
});
|
||||
|
||||
export function matches(specimen: unknown, pattern: Pattern): boolean {
|
||||
if (typeof pattern === "object" && pattern !== null) {
|
||||
const tag = isTagged(pattern) ? pattern[Symbol.toStringTag] : undefined;
|
||||
switch (tag) {
|
||||
case "match:string":
|
||||
return typeof specimen === "string";
|
||||
case "match:any":
|
||||
return true;
|
||||
default:
|
||||
throw new TypeError("not a pattern");
|
||||
}
|
||||
}
|
||||
return Object.is(specimen, pattern);
|
||||
}
|
||||
|
||||
/** the pattern as written in code, e.g. `M.string()` or `"cookie"` */
|
||||
export function describe(pattern: Pattern): string {
|
||||
if (typeof pattern === "object" && pattern !== null) {
|
||||
return `M.${pattern[Symbol.toStringTag].replace(/^match:/, "")}()`;
|
||||
}
|
||||
return JSON.stringify(pattern);
|
||||
}
|
||||
|
||||
/**
|
||||
* throw unless `specimen` matches `pattern`. Unlike endo's, the message
|
||||
* names the pattern and `label` but never the specimen: specimens here
|
||||
* can be credentials, and messages cross RPC and land in logs.
|
||||
*/
|
||||
export function mustMatch(specimen: unknown, pattern: Pattern, label: string): void {
|
||||
if (!matches(specimen, pattern)) {
|
||||
throw new TypeError(`${label}: must match ${describe(pattern)}`);
|
||||
}
|
||||
}
|
||||
|
||||
/** a smallcaps-encoded passable, as JSON-able data */
|
||||
export type Smallcaps = null | boolean | number | string | Smallcaps[] | { "#tag": string; payload: Smallcaps };
|
||||
|
||||
/** smallcaps quotes strings that start with one of these with `!` */
|
||||
const startsSpecial = (s: string) => /^[!-\-]/.test(s);
|
||||
|
||||
/**
|
||||
* the subset of @endo/marshal's smallcaps encoding that patterns here use:
|
||||
* primitives, arrays, and tagged records (not a whole CapData: no `#`
|
||||
* body prefix, no slots)
|
||||
*/
|
||||
export function toSmallcaps(passable: unknown): Smallcaps {
|
||||
if (passable === undefined) return "#undefined";
|
||||
if (passable === null || typeof passable === "boolean") return passable;
|
||||
if (typeof passable === "number") {
|
||||
if (Number.isNaN(passable)) return "#NaN";
|
||||
if (passable === Infinity) return "#Infinity";
|
||||
if (passable === -Infinity) return "#-Infinity";
|
||||
return Object.is(passable, -0) ? 0 : passable;
|
||||
}
|
||||
if (typeof passable === "string") return startsSpecial(passable) ? `!${passable}` : passable;
|
||||
if (Array.isArray(passable)) return passable.map(toSmallcaps);
|
||||
if (isTagged(passable)) {
|
||||
return { "#tag": toSmallcaps(passable[Symbol.toStringTag]) as string, payload: toSmallcaps(passable.payload) };
|
||||
}
|
||||
throw new TypeError("toSmallcaps: unsupported passable");
|
||||
}
|
||||
|
||||
/** the inverse of {@link toSmallcaps} */
|
||||
export function fromSmallcaps(encoding: Smallcaps): unknown {
|
||||
if (encoding === null || typeof encoding === "boolean" || typeof encoding === "number") return encoding;
|
||||
if (typeof encoding === "string") {
|
||||
if (encoding.startsWith("!")) return encoding.slice(1);
|
||||
switch (encoding) {
|
||||
case "#undefined":
|
||||
return undefined;
|
||||
case "#NaN":
|
||||
return NaN;
|
||||
case "#Infinity":
|
||||
return Infinity;
|
||||
case "#-Infinity":
|
||||
return -Infinity;
|
||||
}
|
||||
if (startsSpecial(encoding)) throw new TypeError("fromSmallcaps: unsupported special string");
|
||||
return encoding;
|
||||
}
|
||||
if (Array.isArray(encoding)) return Object.freeze(encoding.map(fromSmallcaps));
|
||||
if (typeof encoding === "object" && "#tag" in encoding) {
|
||||
const { "#tag": tag, payload, ...rest } = encoding;
|
||||
if (Object.keys(rest).length > 0) throw new TypeError("fromSmallcaps: extra properties on a tagged record");
|
||||
return makeTagged(fromSmallcaps(tag) as string, fromSmallcaps(payload));
|
||||
}
|
||||
throw new TypeError("fromSmallcaps: unsupported encoding");
|
||||
}
|
||||
@@ -0,0 +1,60 @@
|
||||
/**
|
||||
* @file a MapStore as a Durable Object of its own, so a store can be held
|
||||
* apart from whoever reads it: the DO id is the store's full authority
|
||||
* (e.g. the account owner's, to rotate a credential), while a reader is
|
||||
* handed only {@link MapStoreDo.readOnly}.
|
||||
*/
|
||||
|
||||
import { DurableObject } from "cloudflare:workers";
|
||||
import { fromSmallcaps, matches, toSmallcaps, type Pattern, type Smallcaps } from "./patterns.js";
|
||||
import { makeMapStore, type MapStore, type ReadonlyMapStore, type StoreShapes } from "./store.js";
|
||||
|
||||
/** store shapes as they cross RPC and sit in storage: smallcaps-encoded
|
||||
* (structured clone would strip a matcher down to its payload) */
|
||||
export type EncodedShapes = { keyShape?: Smallcaps; valueShape?: Smallcaps };
|
||||
|
||||
export const encodeShapes = ({ keyShape, valueShape }: StoreShapes): EncodedShapes => ({
|
||||
...(keyShape === undefined ? {} : { keyShape: toSmallcaps(keyShape) }),
|
||||
...(valueShape === undefined ? {} : { valueShape: toSmallcaps(valueShape) }),
|
||||
});
|
||||
|
||||
const decodeShapes = ({ keyShape, valueShape }: EncodedShapes): StoreShapes => ({
|
||||
...(keyShape === undefined ? {} : { keyShape: fromSmallcaps(keyShape) as Pattern }),
|
||||
...(valueShape === undefined ? {} : { valueShape: fromSmallcaps(valueShape) as Pattern }),
|
||||
});
|
||||
|
||||
type Config = { label: string; shapes: EncodedShapes };
|
||||
|
||||
/** a store whose label and shapes its creator fixes, once */
|
||||
export class MapStoreDo extends DurableObject {
|
||||
/** one-shot: fix the label and shapes (see {@link encodeShapes}); an
|
||||
* identical re-configure is a no-op */
|
||||
configure(label: string, shapes: EncodedShapes): void {
|
||||
const decoded = decodeShapes(shapes);
|
||||
for (const shape of [decoded.keyShape, decoded.valueShape]) {
|
||||
if (shape !== undefined) matches(undefined, shape); // throws on a non-pattern
|
||||
}
|
||||
makeMapStore(label, decoded, this.ctx.storage.kv); // throws on a bad label
|
||||
const config: Config = { label, shapes: encodeShapes(decoded) };
|
||||
const existing = this.ctx.storage.kv.get<Config>("config");
|
||||
if (existing) {
|
||||
if (JSON.stringify(existing) !== JSON.stringify(config)) {
|
||||
throw new Error("already configured with a different label or shapes");
|
||||
}
|
||||
return;
|
||||
}
|
||||
this.ctx.storage.kv.put("config", config);
|
||||
}
|
||||
|
||||
/** the store, with full authority */
|
||||
store(): MapStore<unknown> {
|
||||
const config = this.ctx.storage.kv.get<Config>("config");
|
||||
if (!config) throw new Error("not configured: this store was never set up");
|
||||
return makeMapStore(config.label, decodeShapes(config.shapes), this.ctx.storage.kv);
|
||||
}
|
||||
|
||||
/** the store's queries alone */
|
||||
readOnly(): ReadonlyMapStore<unknown> {
|
||||
return this.store().readOnly();
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,133 @@
|
||||
/**
|
||||
* @file a work-alike for enough of @agoric/store's MapStore: string keys,
|
||||
* shapes checked with ./patterns.js, entries kept in a Durable Object's
|
||||
* synchronous KV storage. The facets are RpcTargets (workerd's / capnweb's,
|
||||
* in place of endo's `Far`), so a store or its read-only view can be
|
||||
* handed out by reference.
|
||||
* @see https://github.com/Agoric/agoric-sdk/tree/master/packages/store
|
||||
*/
|
||||
|
||||
import { RpcTarget } from "cloudflare:workers";
|
||||
import { M, mustMatch, type Pattern } from "./patterns.js";
|
||||
|
||||
/** the storage a store needs: a DO's `ctx.storage.kv`, or a stand-in */
|
||||
export type Kv = Pick<SyncKvStorage, "get" | "put" | "delete" | "list">;
|
||||
|
||||
export type StoreShapes = { keyShape?: Pattern; valueShape?: Pattern };
|
||||
|
||||
/**
|
||||
* Entries live under `store:<label>:<key>`, so the label also names the
|
||||
* storage area (as a baggage key does for endo's durable stores): two
|
||||
* stores with one label on one Kv are the same store.
|
||||
*
|
||||
* Error messages name the label and shapes, never keys or values.
|
||||
*/
|
||||
export function makeMapStore<V>(label: string, { keyShape = M.string(), valueShape = M.any() }: StoreShapes, kv: Kv) {
|
||||
if (label.includes(":")) throw new TypeError("store label must not contain ':'");
|
||||
const prefix = `store:${label}:`;
|
||||
const core: Core<V> = {
|
||||
label,
|
||||
has: (key) => kv.get(prefix + key) !== undefined,
|
||||
get: (key) => kv.get<V>(prefix + key),
|
||||
put: (key, value) => kv.put(prefix + key, value),
|
||||
delete: (key) => {
|
||||
kv.delete(prefix + key);
|
||||
},
|
||||
entries: () => [...kv.list<V>({ prefix })].map(([k, v]): [string, V] => [k.slice(prefix.length), v]),
|
||||
checkKey: (key) => mustMatch(key, keyShape, `${label}: key`),
|
||||
checkValue: (value) => mustMatch(value, valueShape, `${label}: value`),
|
||||
};
|
||||
return new MapStore<V>(core);
|
||||
}
|
||||
|
||||
type Core<V> = {
|
||||
label: string;
|
||||
has(key: string): boolean;
|
||||
get(key: string): V | undefined;
|
||||
put(key: string, value: V): void;
|
||||
delete(key: string): void;
|
||||
entries(): [string, V][];
|
||||
checkKey(key: unknown): void;
|
||||
checkValue(value: unknown): void;
|
||||
};
|
||||
|
||||
/** the queries */
|
||||
function reads<V>(core: Core<V>) {
|
||||
return {
|
||||
has: (key: string): boolean => core.has(key),
|
||||
get: (key: string): V => {
|
||||
const value = core.get(key);
|
||||
if (value === undefined) throw new RangeError(`${core.label}: key not found`);
|
||||
return value;
|
||||
},
|
||||
keys: (): string[] => core.entries().map(([k]) => k),
|
||||
values: (): V[] => core.entries().map(([, v]) => v),
|
||||
entries: (): [string, V][] => core.entries(),
|
||||
getSize: (): number => core.entries().length,
|
||||
};
|
||||
}
|
||||
|
||||
/** a store's queries only: what a reader of a rotating credential holds */
|
||||
export class ReadonlyMapStore<V> extends RpcTarget {
|
||||
#reads: ReturnType<typeof reads<V>>;
|
||||
|
||||
constructor(core: Core<V>) {
|
||||
super();
|
||||
this.#reads = reads(core);
|
||||
}
|
||||
|
||||
has(key: string): boolean {
|
||||
return this.#reads.has(key);
|
||||
}
|
||||
get(key: string): V {
|
||||
return this.#reads.get(key);
|
||||
}
|
||||
keys(): string[] {
|
||||
return this.#reads.keys();
|
||||
}
|
||||
values(): V[] {
|
||||
return this.#reads.values();
|
||||
}
|
||||
entries(): [string, V][] {
|
||||
return this.#reads.entries();
|
||||
}
|
||||
getSize(): number {
|
||||
return this.#reads.getSize();
|
||||
}
|
||||
}
|
||||
|
||||
/** a store's full authority: the queries plus `init`/`set`/`delete`, and
|
||||
* {@link MapStore.readOnly} to hand out the queries alone */
|
||||
export class MapStore<V> extends ReadonlyMapStore<V> {
|
||||
#core: Core<V>;
|
||||
|
||||
constructor(core: Core<V>) {
|
||||
super(core);
|
||||
this.#core = core;
|
||||
}
|
||||
|
||||
/** add a key that is not yet present */
|
||||
init(key: string, value: V): void {
|
||||
this.#core.checkKey(key);
|
||||
this.#core.checkValue(value);
|
||||
if (this.#core.has(key)) throw new RangeError(`${this.#core.label}: key already registered`);
|
||||
this.#core.put(key, value);
|
||||
}
|
||||
|
||||
/** replace the value of a key that is present */
|
||||
set(key: string, value: V): void {
|
||||
this.#core.checkKey(key);
|
||||
this.#core.checkValue(value);
|
||||
if (!this.#core.has(key)) throw new RangeError(`${this.#core.label}: key not found`);
|
||||
this.#core.put(key, value);
|
||||
}
|
||||
|
||||
delete(key: string): void {
|
||||
if (!this.#core.has(key)) throw new RangeError(`${this.#core.label}: key not found`);
|
||||
this.#core.delete(key);
|
||||
}
|
||||
|
||||
readOnly(): ReadonlyMapStore<V> {
|
||||
return new ReadonlyMapStore(this.#core);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,142 @@
|
||||
import { env } from "cloudflare:workers";
|
||||
import { evictDurableObject } from "cloudflare:test";
|
||||
import { describe, expect, it } from "vitest";
|
||||
import { M, fromSmallcaps, makeTagged, matches, mustMatch, toSmallcaps } from "../src/patterns.js";
|
||||
import { encodeShapes } from "../src/store-do.js";
|
||||
|
||||
type TestEnv = {
|
||||
STORE_DO: DurableObjectNamespace<import("./test-worker.js").StoreHost>;
|
||||
MAP_STORE_DO: DurableObjectNamespace<import("../src/store-do.js").MapStoreDo>;
|
||||
};
|
||||
const testEnv = env as unknown as TestEnv;
|
||||
|
||||
const SECRET = "session=s3kr1t";
|
||||
|
||||
/** the message an RPC call fails with, or "" if it succeeds */
|
||||
async function errorOf(call: Promise<unknown>): Promise<string> {
|
||||
try {
|
||||
await call;
|
||||
return "";
|
||||
} catch (err) {
|
||||
return String(err);
|
||||
}
|
||||
}
|
||||
|
||||
describe("patterns", () => {
|
||||
it("M.string() matches strings only; a literal matches only itself", () => {
|
||||
expect(matches("x", M.string())).toBe(true);
|
||||
expect(matches(1, M.string())).toBe(false);
|
||||
expect(matches(undefined, M.any())).toBe(true);
|
||||
expect(matches("cookie", "cookie")).toBe(true);
|
||||
expect(matches("authorization", "cookie")).toBe(false);
|
||||
});
|
||||
|
||||
it("matchers are endo's tagged records", () => {
|
||||
const s = M.string() as unknown as Record<string | symbol, unknown>;
|
||||
expect(s[Symbol.for("passStyle")]).toBe("tagged");
|
||||
expect(s[Symbol.toStringTag]).toBe("match:string");
|
||||
expect(Object.keys(s)).toEqual(["payload"]);
|
||||
expect(s.payload).toEqual([]);
|
||||
expect((M.any() as unknown as Record<string, unknown>).payload).toBeUndefined();
|
||||
expect(Object.isFrozen(s)).toBe(true);
|
||||
});
|
||||
|
||||
it("smallcaps: endo's encoding, round-tripped", () => {
|
||||
expect(toSmallcaps(M.string())).toEqual({ "#tag": "match:string", payload: [] });
|
||||
expect(toSmallcaps(M.any())).toEqual({ "#tag": "match:any", payload: "#undefined" });
|
||||
expect(toSmallcaps("#cookie")).toBe("!#cookie");
|
||||
expect(toSmallcaps([NaN, -0, Infinity])).toEqual(["#NaN", 0, "#Infinity"]);
|
||||
const back = fromSmallcaps(toSmallcaps(M.string())) as Parameters<typeof matches>[1];
|
||||
expect(matches("x", back)).toBe(true);
|
||||
expect(matches(1, back)).toBe(false);
|
||||
expect(fromSmallcaps("!#cookie")).toBe("#cookie");
|
||||
expect(() => matches("x", makeTagged("match:nope", []))).toThrow("not a pattern");
|
||||
});
|
||||
|
||||
it("mustMatch names the shape but not the specimen", () => {
|
||||
expect(() => mustMatch(SECRET, "cookie", "jar: key")).toThrow('jar: key: must match "cookie"');
|
||||
expect(() => mustMatch(SECRET, "cookie", "jar: key")).not.toThrow(SECRET);
|
||||
});
|
||||
});
|
||||
|
||||
describe("MapStore over RPC", () => {
|
||||
const fresh = () => testEnv.STORE_DO.get(testEnv.STORE_DO.newUniqueId());
|
||||
|
||||
it("rotates: the admin's set is what the read-only view sees next", async () => {
|
||||
const store = await fresh().cookies();
|
||||
const rd = await store.readOnly();
|
||||
await store.init("cookie", "session=old");
|
||||
expect(await rd.get("cookie")).toBe("session=old");
|
||||
await store.set("cookie", "session=new");
|
||||
expect(await rd.entries()).toEqual([["cookie", "session=new"]]);
|
||||
});
|
||||
|
||||
it("the read-only view has no write methods", async () => {
|
||||
const store = await fresh().cookies();
|
||||
const rd = (await store.readOnly()) as unknown as { set(k: string, v: string): Promise<void> };
|
||||
expect(await errorOf(rd.set("cookie", SECRET))).not.toBe("");
|
||||
expect(await store.has("cookie")).toBe(false);
|
||||
});
|
||||
|
||||
it("the key shape admits only `cookie`; errors carry no values", async () => {
|
||||
const store = await fresh().cookies();
|
||||
const err = await errorOf(store.init("authorization", SECRET));
|
||||
expect(err).toContain('cookie: key: must match "cookie"');
|
||||
expect(err).not.toContain(SECRET);
|
||||
expect(await errorOf(store.init("cookie", 42 as unknown as string))).toContain("cookie: value: must match M.string()");
|
||||
});
|
||||
|
||||
it("init refuses a present key; set and get refuse a missing one", async () => {
|
||||
const store = await fresh().cookies();
|
||||
expect(await errorOf(store.set("cookie", SECRET))).toContain("cookie: key not found");
|
||||
expect(await errorOf(store.get("cookie"))).toContain("cookie: key not found");
|
||||
await store.init("cookie", SECRET);
|
||||
const err = await errorOf(store.init("cookie", SECRET));
|
||||
expect(err).toContain("cookie: key already registered");
|
||||
expect(err).not.toContain(SECRET);
|
||||
});
|
||||
|
||||
it("survives eviction: entries live in DO storage", async () => {
|
||||
const stub = fresh();
|
||||
{
|
||||
// disposed before eviction: a live store stub holds the DO open
|
||||
using store = await stub.cookies();
|
||||
await store.init("cookie", "session=durable");
|
||||
}
|
||||
await evictDurableObject(stub);
|
||||
using again = await stub.cookies();
|
||||
expect(await again.get("cookie")).toBe("session=durable");
|
||||
expect(await again.getSize()).toBe(1);
|
||||
await again.delete("cookie");
|
||||
expect(await again.keys()).toEqual([]);
|
||||
});
|
||||
});
|
||||
|
||||
describe("MapStoreDo", () => {
|
||||
const COOKIE_SHAPES = encodeShapes({ keyShape: "cookie", valueShape: M.string() });
|
||||
const fresh = () => testEnv.MAP_STORE_DO.get(testEnv.MAP_STORE_DO.newUniqueId());
|
||||
|
||||
it("keeps the shapes its creator chose, across eviction", async () => {
|
||||
const stub = fresh();
|
||||
await stub.configure("cookie", COOKIE_SHAPES);
|
||||
await evictDurableObject(stub);
|
||||
{
|
||||
using store = await stub.store();
|
||||
expect(await errorOf(store.init("authorization", SECRET))).toContain('cookie: key: must match "cookie"');
|
||||
expect(await errorOf(store.init("cookie", 42))).toContain("cookie: value: must match M.string()");
|
||||
await store.init("cookie", "session=a");
|
||||
}
|
||||
using rd = await stub.readOnly();
|
||||
expect(await rd.entries()).toEqual([["cookie", "session=a"]]);
|
||||
});
|
||||
|
||||
it("configures once: the same shapes again is a no-op, others are refused", async () => {
|
||||
const stub = fresh();
|
||||
await stub.configure("cookie", COOKIE_SHAPES);
|
||||
await stub.configure("cookie", encodeShapes({ keyShape: "cookie", valueShape: M.string() }));
|
||||
expect(await errorOf(stub.configure("cookie", encodeShapes({ keyShape: M.string() })))).toContain(
|
||||
"already configured",
|
||||
);
|
||||
expect(await errorOf(fresh().store())).toContain("not configured");
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,17 @@
|
||||
/** @file the tests' main module: a DO that hosts stores on its own storage. */
|
||||
import { DurableObject } from "cloudflare:workers";
|
||||
import { M } from "../src/patterns.js";
|
||||
import { makeMapStore, type MapStore } from "../src/store.js";
|
||||
|
||||
export { MapStoreDo } from "../src/store-do.js";
|
||||
|
||||
/** hosts the store from the credential-rotation sketch: one key, `cookie` */
|
||||
export class StoreHost extends DurableObject {
|
||||
cookies(): MapStore<string> {
|
||||
return makeMapStore<string>("cookie", { keyShape: "cookie", valueShape: M.string() }, this.ctx.storage.kv);
|
||||
}
|
||||
}
|
||||
|
||||
export default {
|
||||
fetch: () => new Response("store tests", { status: 404 }),
|
||||
};
|
||||
@@ -0,0 +1,7 @@
|
||||
{
|
||||
"extends": "../tsconfig.json",
|
||||
"compilerOptions": {
|
||||
"types": ["@cloudflare/vitest-plugin/types"]
|
||||
},
|
||||
"include": ["./**/*", "../src/**/*", "../worker-configuration.d.ts"]
|
||||
}
|
||||
@@ -0,0 +1,15 @@
|
||||
{
|
||||
"compilerOptions": {
|
||||
"target": "ES2022",
|
||||
"module": "preserve",
|
||||
"moduleResolution": "bundler",
|
||||
"strict": true,
|
||||
"esModuleInterop": true,
|
||||
"verbatimModuleSyntax": true,
|
||||
"skipLibCheck": true,
|
||||
"lib": ["ES2022", "ESNext.Disposable"],
|
||||
"noEmit": true,
|
||||
"types": []
|
||||
},
|
||||
"include": ["src/**/*", "worker-configuration.d.ts"]
|
||||
}
|
||||
@@ -0,0 +1,9 @@
|
||||
import { defineProject } from "vitest/config";
|
||||
import { cloudflareTest } from "@cloudflare/vitest-plugin";
|
||||
|
||||
export default defineProject({
|
||||
plugins: [cloudflareTest({ wrangler: { configPath: "./wrangler.tests.jsonc" } })],
|
||||
test: {
|
||||
include: ["tests/**/*.test.ts"],
|
||||
},
|
||||
});
|
||||
@@ -0,0 +1,17 @@
|
||||
{
|
||||
"$schema": "node_modules/wrangler/config-schema.json",
|
||||
// a host DO for the stores under test (lib ships no worker of its own)
|
||||
"name": "capsite-lib-tests",
|
||||
"main": "./tests/test-worker.ts",
|
||||
"compatibility_date": "2026-09-29",
|
||||
"durable_objects": {
|
||||
"bindings": [
|
||||
{ "name": "STORE_DO", "class_name": "StoreHost" },
|
||||
{ "name": "MAP_STORE_DO", "class_name": "MapStoreDo" }
|
||||
]
|
||||
},
|
||||
"exports": {
|
||||
"StoreHost": { "type": "durable-object", "storage": "sqlite" },
|
||||
"MapStoreDo": { "type": "durable-object", "storage": "sqlite" }
|
||||
}
|
||||
}
|
||||
@@ -1,107 +0,0 @@
|
||||
This is an EmDash site -- a CMS built on Astro with a full admin UI.
|
||||
|
||||
## Commands
|
||||
|
||||
```bash
|
||||
pnpm dev # Start the Astro dev server
|
||||
npx emdash types # Regenerate TypeScript types from a running site
|
||||
```
|
||||
|
||||
The admin UI is at `http://localhost:4321/_emdash/admin`.
|
||||
|
||||
## Key Files
|
||||
|
||||
| File | Purpose |
|
||||
| ------------------------ | ---------------------------------------------------------------------------------- |
|
||||
| `astro.config.mjs` | Astro config with `emdash()` integration, database, and storage |
|
||||
| `src/live.config.ts` | EmDash loader registration (boilerplate -- don't modify) |
|
||||
| `seed/seed.json` | Schema definition + demo content (collections, fields, taxonomies, menus, widgets) |
|
||||
| `emdash-env.d.ts` | Generated types for collections (auto-regenerated on dev server start) |
|
||||
| `src/layouts/Base.astro` | Base layout with EmDash wiring (menus, search, page contributions) |
|
||||
| `src/pages/` | Astro pages -- all server-rendered |
|
||||
|
||||
## Skills
|
||||
|
||||
Agent skills are in `.agents/skills/`. Load them when working on specific tasks:
|
||||
|
||||
- **building-emdash-site** -- Querying content, rendering Portable Text, schema design, seed files, site features (menus, widgets, search, SEO, comments, bylines). Start here.
|
||||
- **creating-plugins** -- Building EmDash plugins with hooks, storage, admin UI, API routes, and Portable Text block types.
|
||||
- **emdash-cli** -- CLI commands for content management, seeding, type generation, and visual editing flow.
|
||||
|
||||
## Documentation
|
||||
|
||||
The EmDash docs are available as an MCP server at `https://docs.emdashcms.com/mcp`. When you need to verify an API, hook, config option, field type, or pattern, call `search_docs` against the live documentation rather than relying on training-data recall. The docs reflect current behaviour; assumptions may not.
|
||||
|
||||
This template ships with `.mcp.json`, `.cursor/mcp.json`, and `.vscode/mcp.json` so Claude Code, Cursor, and VS Code auto-discover the docs server. Other tools (OpenCode, Windsurf, etc.) need a manual one-time setup -- see [docs.emdashcms.com/docs-mcp](https://docs.emdashcms.com/docs-mcp).
|
||||
|
||||
## Rules
|
||||
|
||||
- All content pages must be server-rendered (`output: "server"`). No `getStaticPaths()` for CMS content.
|
||||
- Image fields are objects (`{ src, alt }`), not strings. Use `<Image image={...} />` from `"emdash/ui"`.
|
||||
- `entry.id` is the slug (for URLs). `entry.data.id` is the database ULID (for API calls like `getEntryTerms`).
|
||||
- When Astro's cache is enabled, pass content-query hints to `Astro.cache.set(cacheHint)`. Use the `WithCacheHint` variants for site settings, menus, taxonomies, and widget areas rendered by cached routes.
|
||||
- Taxonomy names in queries must match the seed's `"name"` field exactly (e.g., `"category"` not `"categories"`).
|
||||
|
||||
## This Template
|
||||
|
||||
A blog with posts, pages, categories, tags, full-text search, and RSS. Designed for personal writing, technical writing, indie newsletters, and anything where the writing is the product. Editorial-tech aesthetic: confident sans-serif, restrained accent, real article structure with bylines and reading time.
|
||||
|
||||
## Pages
|
||||
|
||||
| Page | Path | What it shows |
|
||||
| ----------- | ------------------ | ------------------------------------------------------------------------------------------------------ |
|
||||
| Home | `/` | Featured post hero (large image + excerpt), latest posts grid |
|
||||
| All posts | `/posts` | Article count, full post list with excerpts and tag chips |
|
||||
| Post detail | `/posts/[slug]` | Featured image, title, body, left meta column (authors + date), right TOC + search + categories gutter |
|
||||
| Search | `/search` | Full-text search UI |
|
||||
| Page | `/pages/[slug]` | Static page content (Portable Text) |
|
||||
| Category | `/category/[slug]` | Posts filtered by category |
|
||||
| Tag | `/tag/[slug]` | Posts filtered by tag |
|
||||
| RSS | `/rss.xml` | Generated feed |
|
||||
|
||||
## Schema
|
||||
|
||||
- `posts` collection: `title`, `featured_image`, `content` (Portable Text), `excerpt` (text).
|
||||
- `pages` collection: `title`, `content` (Portable Text). Used for `/about` etc.
|
||||
- Taxonomies: `category`, `tag`.
|
||||
- Single `primary` menu (Home, About, Posts by default).
|
||||
|
||||
Site settings have `title` and `tagline` -- both render in the header / footer.
|
||||
|
||||
## Visual character
|
||||
|
||||
Single typeface: **Inter** on `--font-body`, used for everything including headings (`--font-heading` defaults to the body face; tighter letter-spacing on h1/h2). **JetBrains Mono** on `--font-mono` for inline code and code blocks. Body and headings share the same family; weight and size carry the hierarchy (`--font-weight-heading` 600, `--font-weight-display` 700 for h1/page titles).
|
||||
|
||||
The brand colour is `#0066cc` (`--color-brand`) -- used for links, the post-card title hover, and the search input focus ring. There's also a secondary text colour (`--color-text-secondary`) and a `--color-muted` for meta info. Don't add a second accent.
|
||||
|
||||
The article layout is the standout feature: a three-column reading view with a left meta column (author bylines, date), centred 680px body column, and a right gutter for search, table of contents, and categories. Don't flatten that into one column on desktop -- the layout signals "this is something to read".
|
||||
|
||||
## Customisation
|
||||
|
||||
Design tokens live in `src/styles/tokens.css` with their default values. To restyle the site, override tokens in `src/styles/theme.css` -- declarations there are unlayered, so they always beat the `@layer base` defaults. Don't edit `tokens.css` or `Base.astro` for visual changes.
|
||||
|
||||
Colours are defined with `light-dark(<light>, <dark>)`, so each token carries both modes. Overriding with a plain colour changes light and dark at once; use `light-dark()` in the override to keep them distinct. There is no separate dark palette to maintain.
|
||||
|
||||
Webfonts are configured in `astro.config.mjs` under `fonts:`. To swap the body face, change the `name:` for the entry bound to `cssVariable: "--font-body"`. Good alternatives: Geist, IBM Plex Sans, Söhne (if you have a licence), Public Sans. If you want a serif-bodied blog, swap to a humanist serif like Source Serif, Crimson Pro, or Lora -- but then also raise `--font-size-base` to `1.0625rem` for readability. To give headings their own face (or use a system font) without touching the font pipeline, override `--font-heading` or `--font-body` in `theme.css`.
|
||||
|
||||
CSS variables worth knowing (see `tokens.css` for the full list):
|
||||
|
||||
- `--color-brand`, `--color-brand-hover`, `--color-on-brand`, `--color-brand-ring`
|
||||
- `--color-bg`, `--color-bg-subtle`, `--color-surface`, `--color-text`, `--color-text-secondary`, `--color-muted`, `--color-border`, `--color-border-subtle`
|
||||
- `--font-body`, `--font-heading`, `--font-mono`
|
||||
- `--font-weight-heading` (600) / `--font-weight-display` (700) -- heading weights; lower them if you switch to a serif
|
||||
- `--tracking-tight` / `--tracking-snug` / `--tracking-wide` / `--tracking-wider` -- letter-spacing tokens used across headings and meta labels
|
||||
- `--content-width` (680px) -- article body column
|
||||
- `--wide-width` (1200px) -- max container
|
||||
- `--gutter-width` (200px) -- right sidebar (TOC) on article pages
|
||||
- `--meta-col-width` (180px) -- left meta column on article pages
|
||||
- `--avatar-size-{xs,sm,md,lg}` -- byline avatar sizes at different scales
|
||||
|
||||
## What not to do
|
||||
|
||||
- Don't add a second accent colour or coloured section backgrounds. The page should be black, white, and one blue.
|
||||
- Don't replace Inter with a display sans (Bebas, Anton, etc.). Headings rely on weight contrast, not novelty faces.
|
||||
- Don't collapse the article gutter on desktop -- it's part of the reading experience.
|
||||
- Don't use stock blog copy ("Welcome to my blog", "Stay tuned for more"). Write a real tagline that says what this blog is about.
|
||||
- Don't seed the home page with three identical placeholder posts. If you only have one real post, show one real post.
|
||||
- Comments are enabled on posts and rendered on the post detail page. Configure moderation before publishing the site, or remove `commentsEnabled` and the comments UI together.
|
||||
Symlink
+1
@@ -0,0 +1 @@
|
||||
CONTRIBUTING.md
|
||||
@@ -0,0 +1,51 @@
|
||||
# Contributing to potluck
|
||||
|
||||
potluck started as EmDash's `cloudflare:blog` template (`pnpm create emdash@latest`); it's part of [the capsite goal tree](../capsite-checklist.md), where it is the site that publishes the weekly roundup.
|
||||
|
||||
## How it's put together
|
||||
|
||||
- **Issues** are the `issues` collection ([seed/issues.json](seed/issues.json)): `title`, `week` (an ISO week, `2026-W40`), `dek` (one-line summary), `body` (Portable Text). The slug is the week, lowercased: `/issues/2026-w40`. [issues-sketch.md](issues-sketch.md) is the dry run behind the three sample issues: real public bookmarks, editorial lines marked *draft*.
|
||||
- **Pages:** home and `/issues` list issues ([src/pages/issues/](src/pages/issues/)); `/tag/<tag>` lists issues by diigo tag; search and `/rss.xml` cover issues. The `posts` collection and its pages are the template's, still in place.
|
||||
- **Microformats:** an issue list is an `h-feed` of `h-entry`s; an issue is an `h-entry` (`p-name`, `dt-published`, `p-summary`, `e-content`). In issue bodies, [CiteLink.astro](src/components/CiteLink.astro) marks off-site links as `h-cite` and `/tag/<tag>` links as rel-tag `p-category`.
|
||||
- **Look:** plain HTML, after kmjn.org's [Plain HTML with a few bureaucratic gestures](https://www.kmjn.org/notes/modern_plain_html.html): system fonts (no webfonts), one ~800px column, links in the text color with a dotted underline. Restyle in [src/styles/theme.css](src/styles/theme.css), which overrides the defaults in `tokens.css` (unlayered, so it always wins); don't edit `tokens.css` or `Base.astro` for visual changes. Colors use `light-dark(<light>, <dark>)`.
|
||||
|
||||
## Plugins
|
||||
|
||||
[diigo-week](../diigo-week/README.md) (Plugins → Draft from diigo in the admin) drafts an issue from a week of diigo bookmarks. It reads diigo-cache's public page as microformats over the `DIIGO_CACHE` service binding in [wrangler.jsonc](wrangler.jsonc); in dev, run diigo-cache's `pnpm dev` alongside. It's linked as `link:../diigo-week`, and registered in [astro.config.mjs](astro.config.mjs). After changing it, restart the dev server.
|
||||
|
||||
## Seeds
|
||||
|
||||
[seed/seed.json](seed/seed.json) (site settings, menus, widgets, the template's collections) is applied on the first request to an empty database. [seed/issues.json](seed/issues.json) is separate: `pnpm seed:issues` applies it to the local D1 database, which exists after the dev server's first request. Run it with the dev server stopped.
|
||||
|
||||
## Checks
|
||||
|
||||
```sh
|
||||
pnpm typecheck # astro check
|
||||
pnpm build
|
||||
```
|
||||
|
||||
## EmDash rules
|
||||
|
||||
- All content pages are server-rendered (`output: "server"`); no `getStaticPaths()` for CMS content.
|
||||
- Image fields are objects (`{ src, alt }`), not strings; render with `<Image image={...} />` from `"emdash/ui"`.
|
||||
- `entry.id` is the slug (for URLs); `entry.data.id` is the database ULID (for API calls like `getEntryTerms`).
|
||||
- When Astro's cache is enabled, pass query hints to `Astro.cache.set(cacheHint)`.
|
||||
- Taxonomy names in queries match the seed's `"name"` exactly (`"tag"`, not `"tags"`).
|
||||
- `npx emdash types` regenerates `emdash-env.d.ts` from a running site (the dev server also regenerates it on start).
|
||||
|
||||
The EmDash docs are an MCP server at `https://docs.emdashcms.com/mcp` (wired up in `.mcp.json`, `.cursor/mcp.json`, `.vscode/mcp.json`). Check them rather than recalling APIs from memory.
|
||||
|
||||
## Deploying
|
||||
|
||||
```sh
|
||||
pnpm wrangler login
|
||||
pnpm deploy
|
||||
```
|
||||
|
||||
The first deploy provisions the D1 database and R2 bucket named in [wrangler.jsonc](wrangler.jsonc). See [Deploy to Cloudflare](https://docs.emdashcms.com/deployment/cloudflare/).
|
||||
|
||||
Other workerd hosts are untested: the site uses D1 and R2 through `@emdash-cms/cloudflare` and the Cloudflare adapter (locally, miniflare stands in for both), so a host has to supply those two bindings.
|
||||
|
||||
## For agents
|
||||
|
||||
`AGENTS.md` is a symlink to this file. capsite's [working rules](../AGENTS.md) apply here too (the dev server is the user's to start and stop). EmDash skills are in `.agents/skills/`: `building-emdash-site` (start here), `creating-plugins`, `emdash-cli`.
|
||||
+14
-60
@@ -1,68 +1,22 @@
|
||||
# EmDash Blog Template (Cloudflare)
|
||||
# potluck
|
||||
|
||||
A clean, minimal blog built with [EmDash](https://github.com/emdash-cms/emdash) and deployed on Cloudflare Workers with D1 and R2.
|
||||
A weekly zine of what you bookmarked elsewhere, published on your own site. Each issue is one week of links with a few words around them.
|
||||
|
||||
[](https://deploy.workers.cloudflare.com/?url=https://github.com/emdash-cms/templates/tree/main/blog-cloudflare)
|
||||
## Getting started
|
||||
|
||||

|
||||
|
||||
## What's Included
|
||||
|
||||
- Featured post hero on the homepage
|
||||
- Post archive with reading time estimates
|
||||
- Category and tag archives
|
||||
- Full-text search
|
||||
- RSS feed
|
||||
- SEO metadata and JSON-LD
|
||||
- Dark/light mode
|
||||
|
||||
## Pages
|
||||
|
||||
| Page | Route |
|
||||
|---|---|
|
||||
| Homepage | `/` |
|
||||
| All posts | `/posts` |
|
||||
| Single post | `/posts/:slug` |
|
||||
| Category archive | `/category/:slug` |
|
||||
| Tag archive | `/tag/:slug` |
|
||||
| Search | `/search` |
|
||||
| Static pages | `/pages/:slug` |
|
||||
| 404 | fallback |
|
||||
|
||||
## Screenshots
|
||||
|
||||
| | Desktop | Mobile |
|
||||
|---|---|---|
|
||||
| Light |  |  |
|
||||
| Dark |  |  |
|
||||
|
||||
## Infrastructure
|
||||
|
||||
- **Runtime:** Cloudflare Workers
|
||||
- **Database:** D1
|
||||
- **Storage:** R2
|
||||
- **Framework:** Astro with `@astrojs/cloudflare`
|
||||
|
||||
## Local Development
|
||||
|
||||
```bash
|
||||
```sh
|
||||
pnpm install
|
||||
pnpm dev
|
||||
pnpm dev # open http://localhost:4321/_emdash/admin, finish the setup wizard, then Ctrl-C
|
||||
pnpm seed:issues # three sample issues, W38–W40 of 2026
|
||||
pnpm dev # → http://localhost:4321/issues
|
||||
```
|
||||
|
||||
Open http://localhost:4321/_emdash/admin and complete the setup wizard. EmDash runs database migrations and applies the blog seed during setup. The site is available at http://localhost:4321.
|
||||
## Bookmarks in, a zine out
|
||||
|
||||
## Deploying
|
||||
- **Your web, your site:** the bookmarks come from your own diigo account, so the web you read becomes something you publish ([PESOS](https://indieweb.org/PESOS)). The [diigo-week](../diigo-week/README.md) plugin drafts an issue from a week of them; you cut and comment.
|
||||
- **A week per issue:** `/issues/2026-w40`, by ISO week, UTC.
|
||||
- **Microformats:** feed readers and IndieWeb tools see each issue as an [h-entry](https://microformats.org/wiki/h-entry) and each bookmark as an [h-cite](https://microformats.org/wiki/h-cite).
|
||||
- **Plain HTML** that's easy to read: system fonts, one column.
|
||||
- **A real editor:** it's an [EmDash](https://docs.emdashcms.com/) site, so you write issues in its admin UI. It runs in [workerd](https://github.com/cloudflare/workerd), on your machine or a host.
|
||||
|
||||
```bash
|
||||
pnpm wrangler login
|
||||
pnpm deploy
|
||||
```
|
||||
|
||||
The first deployment provisions the named D1 database and R2 bucket from `wrangler.jsonc`. See [Deploy to Cloudflare](https://docs.emdashcms.com/deployment/cloudflare/) for production setup, or use the deploy button above.
|
||||
|
||||
## See Also
|
||||
|
||||
- [Node.js variant](../blog) -- same template using SQLite and local file storage
|
||||
- [All templates](../)
|
||||
- [EmDash documentation](https://docs.emdashcms.com/)
|
||||
How it's put together, checks, and deploying: [CONTRIBUTING](CONTRIBUTING.md).
|
||||
@@ -16,6 +16,8 @@ export default defineConfig({
|
||||
emdash({
|
||||
database: d1({ binding: "DB", session: "auto" }),
|
||||
storage: r2({ binding: "MEDIA" }),
|
||||
// drafts an issue from a week of diigo bookmarks (../diigo-week)
|
||||
plugins: [{ id: "diigo-week", version: "0.1.0", entrypoint: "diigo-week" }],
|
||||
}),
|
||||
],
|
||||
devToolbar: { enabled: false },
|
||||
|
||||
@@ -9,6 +9,7 @@
|
||||
},
|
||||
"scripts": {
|
||||
"dev": "astro dev",
|
||||
"seed:issues": "emdash seed seed/issues.json -d \"$(find .wrangler/state/v3/d1 -name '*.sqlite' ! -name metadata.sqlite)\"",
|
||||
"build": "astro build",
|
||||
"preview": "astro preview",
|
||||
"deploy": "astro build && wrangler deploy",
|
||||
@@ -19,6 +20,7 @@
|
||||
"@astrojs/react": "^6.0.5",
|
||||
"@emdash-cms/cloudflare": "^1.0.1",
|
||||
"astro": "^7.3.2",
|
||||
"diigo-week": "link:../diigo-week",
|
||||
"emdash": "^1.0.1",
|
||||
"react": "19.2.4",
|
||||
"react-dom": "19.2.4"
|
||||
@@ -29,4 +31,4 @@
|
||||
"wrangler": "^4.125.0"
|
||||
},
|
||||
"packageManager": "pnpm@11.9.0+sha512.bd682d5d03fe525ef7c9fd6780c6884d1e756ac4c9c9fe00c538782824310dcf90e3ddc4f53835f06dfaebd5085e41855e0bcbb3b60de2ac5bbab89e5036f03b"
|
||||
}
|
||||
}
|
||||
Generated
+3
@@ -20,6 +20,9 @@ importers:
|
||||
astro:
|
||||
specifier: ^7.3.2
|
||||
version: 7.3.6(@types/node@26.6.4)(yaml@2.9.1)
|
||||
diigo-week:
|
||||
specifier: link:../diigo-week
|
||||
version: link:../diigo-week
|
||||
emdash:
|
||||
specifier: ^1.0.1
|
||||
version: 1.2.0(@astrojs/react@6.0.6(@types/node@26.6.4)(@types/react-dom@19.3.0(@types/react@19.3.0))(@types/react@19.3.0)(esbuild@0.28.2)(react-dom@19.2.4(react@19.2.4))(react@19.2.4)(yaml@2.9.1))(@atcute/cbor@2.3.8(@atcute/cid@2.5.0))(@atcute/cid@2.5.0)(@atcute/identity@2.0.2(@atcute/lexicons@2.1.1)(typescript@6.0.3))(@date-fns/tz@1.5.0)(@floating-ui/dom@1.8.0)(@tiptap/extensions@3.31.4(@tiptap/core@3.31.4(@tiptap/pm@3.31.4))(@tiptap/pm@3.31.4))(@types/react-dom@19.3.0(@types/react@19.3.0))(@types/react@19.3.0)(astro@7.3.6(@types/node@26.6.4)(yaml@2.9.1))(date-fns@4.4.0)(echarts@6.1.0)(prosemirror-model@1.25.12)(prosemirror-state@1.4.4)(prosemirror-view@1.42.6)(react-dom@19.2.4(react@19.2.4))(react@19.2.4)(typescript@6.0.3)
|
||||
|
||||
@@ -16,6 +16,9 @@
|
||||
"bucket_name": "my-emdash-media",
|
||||
},
|
||||
],
|
||||
// diigo-cache's public pages, for the diigo-week plugin (reads its
|
||||
// microformats; in dev, diigo-cache's own `pnpm dev` serves it)
|
||||
"services": [{ "binding": "DIIGO_CACHE", "service": "diigo-cache" }],
|
||||
// Sandboxed plugins require Worker Loader, available on Workers paid plans. Uncomment to enable:
|
||||
// "worker_loaders": [{ "binding": "LOADER" }],
|
||||
// General maintenance cron trigger
|
||||
|
||||
+38
-3
@@ -3,7 +3,7 @@
|
||||
DO-backed fluent web client factory: site-agnostic. Deploy once; instances
|
||||
are attenuated handles whose **ids are bearer capabilities** — anyone
|
||||
holding the id string can drive the instance; nobody without it can.
|
||||
TODO: this proxy-page flow (see diigo-cache README) vs running webapp as
|
||||
TODO: this proxy-page flow (see [diigo-cache CONTRIBUTING](../diigo-cache/CONTRIBUTING.md)) vs running webapp as
|
||||
the primary worker (`wrangler dev -c ../webapp/wrangler.jsonc -c ../diigo-cache/wrangler.jsonc`)
|
||||
so its own port serves the mint form directly; reconsider when the UI
|
||||
grows past a form or two.
|
||||
@@ -12,19 +12,54 @@ grows past a form or two.
|
||||
|
||||
- `GET /` — the mint form. Prefill via query string: `?url=…&headers=<urlencoded JSON>&msPerCall=…&title=…`; secret header keys (cookie, authorization, proxy-authorization) render **blank** — paste them at mint time; never echoed back. A dedicated `cookie` field is sugar for the `cookie` header (if the JSON also has one, they must agree).
|
||||
- `GET /form?render=1&…` — the same form for a proxying worker (diigo-cache's `/_connect`): the action is left as an `ACTION` placeholder for the proxy to point at its own relay.
|
||||
- `POST /` — mint. Form-encoded (that page) or JSON `{url, headers, msPerCall}` (service-style). Responds `{id}` (and `x-webapp-id`). The id is *not* logged: possession is authority.
|
||||
- `POST /` — mint. Form-encoded (that page) or JSON `{url, headers, msPerCall}` (service-style). Responds `{id, storeId}` (and `x-webapp-id`, `x-webapp-store-id`). Neither id is logged: possession is authority.
|
||||
- `POST /at/<id>` — drive an instance by capability: JSON `{path?, query?, method?, body?}` → the fluent chain → `{status, text}` (or `{error}`; messages carry no header/param values).
|
||||
- `PUT /store/<storeId>/<key>` — set a header-store entry (JSON value body), e.g. `PUT /store/<storeId>/cookie` with `"session=…"` rotates the cookie; `DELETE` removes it. → `{ok: true}` (or `{error}`; messages carry no keys or values).
|
||||
|
||||
## RPC surface (same-account workers; binding `entrypoint: "WebAppFactory"`)
|
||||
|
||||
- `fetch(request)` — the HTTP surface above, over the binding
|
||||
- `newWebApp({url, headers?, msPerCall?})` → `{id, app}` — the fluent stub;
|
||||
- `newWebApp({url, headers?, msPerCall?})` → `{id, storeId, app}` — the fluent stub;
|
||||
promise pipelining makes a whole `pathjoin().query().get()` chain a
|
||||
single round trip
|
||||
- `at(id)` — the fluent stub for a previously-minted instance
|
||||
- on a stub, `postForm(fields?, echoCookies?)` → `{status, contentType,
|
||||
contentDisposition?, location?, body}`: a multipart/form-data POST.
|
||||
`echoCookies` maps a form field to a cookie name; the cookie's value
|
||||
(header store included) fills the field at send, so a double-submit
|
||||
CSRF token goes out without the holder seeing it. Redirects are not
|
||||
followed and a non-OK status is a reply, not an Error (a 3xx to a login
|
||||
page is how many sites say "session expired"). `location` comes back
|
||||
query-stripped; `set-cookie` never comes back. `body` is bytes: the
|
||||
caller knows its charset.
|
||||
- `store(storeId)` — a header store, with full authority:
|
||||
`set("cookie", …)` rotates the cookie under every handle already out
|
||||
|
||||
## The header store: two ids, two authorities
|
||||
|
||||
Mint makes two things: the instance, and a header store (a `MapStoreDo`
|
||||
from [the capsite lib](../lib/src/store-do.ts), shaped
|
||||
`("cookie", { keyShape: "cookie", valueShape: M.string() })`). A `cookie`
|
||||
header goes into the store, not the instance's `init`; the instance reads
|
||||
the store's read-only view at every send.
|
||||
|
||||
- the **instance id** is authority to *use* the cookie: it goes to the
|
||||
cache's wiring.
|
||||
- the **store id** is authority to *choose* it: it stays with the account
|
||||
owner, who can rotate it (log in again, `PUT`) or revoke it (`DELETE`,
|
||||
or log out).
|
||||
|
||||
Kept apart because whoever can choose the cookie can swap in another
|
||||
account's session: if the cache's id could rotate, a leaked instance id
|
||||
would let Mallory fill the cache (and its public page) with her own
|
||||
bookmarks, and the owner's logout would not stop it.
|
||||
|
||||
Instances minted before header stores keep their cookie in
|
||||
`init.headers`, unchanged; they work as before and cannot rotate.
|
||||
|
||||
## Run
|
||||
|
||||
pnpm test # header store + rotation (vitest, workerd pool)
|
||||
npx wrangler dev # → its own ready line (usually :8787 with the
|
||||
# two-config diigo-cache run; standalone, :8788)
|
||||
|
||||
|
||||
@@ -0,0 +1,16 @@
|
||||
{
|
||||
"name": "webapp",
|
||||
"private": true,
|
||||
"type": "module",
|
||||
"scripts": {
|
||||
"types": "wrangler types",
|
||||
"lint": "wrangler types --check && tsc -p tsconfig.json && tsc -p tests/tsconfig.json",
|
||||
"test": "vitest run"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@cloudflare/vitest-plugin": "1.3.6",
|
||||
"typescript": "^5.9.0",
|
||||
"vitest": "4.1.11",
|
||||
"wrangler": "^4.148.0"
|
||||
}
|
||||
}
|
||||
Generated
+2240
File diff suppressed because it is too large.
Load diff
+253
-34
@@ -7,21 +7,52 @@
|
||||
// DurableObjectNamespace/DurableObjectId/DurableObjectStub are ambient types
|
||||
// in worker-configuration.d.ts (not exports of "cloudflare:workers")
|
||||
import { RpcTarget, DurableObject, WorkerEntrypoint } from "cloudflare:workers";
|
||||
import { M } from "../../lib/src/patterns.js";
|
||||
import { encodeShapes, MapStoreDo } from "../../lib/src/store-do.js";
|
||||
|
||||
export { MapStoreDo };
|
||||
|
||||
export type Query = Record<string, string | number>;
|
||||
type Headers = Record<string, string>;
|
||||
|
||||
/** headers looked up at each send, e.g. a store's read-only view: whoever
|
||||
* holds the store can rotate a credential under handles already out */
|
||||
export type HeaderMap = { entries(): [string, unknown][] | Promise<[string, unknown][]> };
|
||||
|
||||
/**
|
||||
* init for a new instance — fixed at creation. An instance is immutable
|
||||
* authority: no re-config methods, only attenuation via the fluent chain.
|
||||
* The one thing that changes under it is its header store's contents.
|
||||
*/
|
||||
export interface WebAppInit {
|
||||
url: string;
|
||||
headers?: Headers;
|
||||
/** min spacing between outbound calls, ms (0 = unthrottled) */
|
||||
msPerCall?: number;
|
||||
/** id of a {@link MapStoreDo} whose entries are added as headers at
|
||||
* each send (see {@link mintInstance}); the instance only reads it */
|
||||
headerStore?: string;
|
||||
}
|
||||
|
||||
/**
|
||||
* what {@link WebApp.postForm} hands back: the status and the response
|
||||
* headers a caller needs to tell a result from a refusal, never
|
||||
* `set-cookie` (a credential stays where it is) and with any query
|
||||
* stripped from `location`
|
||||
*/
|
||||
export type Reply = {
|
||||
status: number;
|
||||
contentType: string;
|
||||
contentDisposition?: string;
|
||||
location?: string;
|
||||
/** the body as sent: the caller knows its charset (some formats name
|
||||
* it in their own header, not in content-type) */
|
||||
body: Uint8Array;
|
||||
};
|
||||
|
||||
/** one outbound request, redirects not followed; the DO's gate and fetch */
|
||||
export type Exchange = (url: string, headers: Headers, method: "POST", body: FormData) => Promise<Response>;
|
||||
|
||||
/** structural type of a stub handle: chain to narrow, then fire */
|
||||
export interface FluentGet {
|
||||
pathjoin(ref: string): FluentGet;
|
||||
@@ -29,8 +60,10 @@ export interface FluentGet {
|
||||
withHeaders(h: Headers): FluentGet;
|
||||
withHeader(key: string, value: string): FluentGet;
|
||||
cookies(cookie: string): FluentGet;
|
||||
withHeaderMap(map: HeaderMap): FluentGet;
|
||||
get(): Promise<string>;
|
||||
post(body: string): Promise<string>;
|
||||
postForm(fields?: Record<string, string>, echoCookies?: Record<string, string>): Promise<Reply>;
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -57,17 +90,23 @@ export interface FluentGet {
|
||||
export class WebApp extends RpcTarget implements FluentGet {
|
||||
#url: string;
|
||||
#headers: Headers;
|
||||
#maps: readonly HeaderMap[];
|
||||
#send: (url: string, headers: Headers, method: "GET" | "POST", body?: string) => Promise<string>;
|
||||
#exchange: Exchange;
|
||||
|
||||
constructor(
|
||||
url: string,
|
||||
headers: Headers,
|
||||
maps: readonly HeaderMap[],
|
||||
send: (url: string, headers: Headers, method: "GET" | "POST", body?: string) => Promise<string>,
|
||||
exchange: Exchange,
|
||||
) {
|
||||
super();
|
||||
this.#url = url;
|
||||
this.#headers = headers;
|
||||
this.#maps = maps;
|
||||
this.#send = send;
|
||||
this.#exchange = exchange;
|
||||
}
|
||||
|
||||
/** Join a relative path, merging the base's query.
|
||||
@@ -85,13 +124,13 @@ export class WebApp extends RpcTarget implements FluentGet {
|
||||
throw new Error(`pathjoin: ${ref} not within ${hereNoQuery.href}`);
|
||||
}
|
||||
const baseParams: Query = Object.fromEntries(here.searchParams);
|
||||
return new WebApp(unionQuery(there.href, baseParams), this.#headers, this.#send);
|
||||
return new WebApp(unionQuery(there.href, baseParams), this.#headers, this.#maps, this.#send, this.#exchange);
|
||||
}
|
||||
|
||||
/** Union headers, flower-style (see {@link WebApp}): a shared key with a
|
||||
* different value throws; the same value is fine. */
|
||||
withHeaders(h: Headers): WebApp {
|
||||
return new WebApp(this.#url, unionHeaders(this.#headers, h), this.#send);
|
||||
return new WebApp(this.#url, unionHeaders(this.#headers, h), this.#maps, this.#send, this.#exchange);
|
||||
}
|
||||
|
||||
/** one key, flower-style (see {@link WebApp}) */
|
||||
@@ -105,15 +144,72 @@ export class WebApp extends RpcTarget implements FluentGet {
|
||||
|
||||
/** Union params into the existing query, flower-style (see {@link WebApp}). */
|
||||
query(params: Query): WebApp {
|
||||
return new WebApp(unionQuery(this.#url, params), this.#headers, this.#send);
|
||||
return new WebApp(unionQuery(this.#url, params), this.#headers, this.#maps, this.#send, this.#exchange);
|
||||
}
|
||||
|
||||
/** Add headers read from `map` at each send, unioned flower-style (see
|
||||
* {@link WebApp}) with the rest — so a contradiction surfaces at send. */
|
||||
withHeaderMap(map: HeaderMap): WebApp {
|
||||
return new WebApp(this.#url, this.#headers, [...this.#maps, map], this.#send, this.#exchange);
|
||||
}
|
||||
|
||||
async get(): Promise<string> {
|
||||
return this.#send(this.#url, this.#headers, "GET");
|
||||
return this.#send(this.#url, await this.#headersNow(), "GET");
|
||||
}
|
||||
|
||||
async post(body: string): Promise<string> {
|
||||
return this.#send(this.#url, this.#headers, "POST", body);
|
||||
return this.#send(this.#url, await this.#headersNow(), "POST", body);
|
||||
}
|
||||
|
||||
/**
|
||||
* POST `fields` as multipart/form-data. `echoCookies` maps a form field
|
||||
* to a cookie name: at send, that cookie's value (from the headers,
|
||||
* header store included) goes in the field, so a double-submit CSRF
|
||||
* token can be sent by a holder who never sees it.
|
||||
*
|
||||
* Unlike `get`/`post`, a non-OK status is a {@link Reply}, not an
|
||||
* Error, and redirects are not followed: a 3xx to a login page is how
|
||||
* many sites say "session expired".
|
||||
*/
|
||||
async postForm(fields: Record<string, string> = {}, echoCookies: Record<string, string> = {}): Promise<Reply> {
|
||||
const headers = await this.#headersNow();
|
||||
if (Object.keys(headers).some((k) => k.toLowerCase() === "content-type")) {
|
||||
throw new Error("postForm: content-type is set by the form, not the headers");
|
||||
}
|
||||
const jar = parseCookies(
|
||||
Object.entries(headers).find(([k]) => k.toLowerCase() === "cookie")?.[1] ?? "",
|
||||
);
|
||||
const form = new FormData();
|
||||
for (const [name, value] of Object.entries(fields)) form.append(name, value);
|
||||
for (const [field, cookie] of Object.entries(echoCookies)) {
|
||||
// names only: values must not appear in Error messages
|
||||
if (field in fields) throw new Error(`postForm: contradicts on ${field}`);
|
||||
const value = jar.get(cookie);
|
||||
if (value === undefined) throw new Error(`postForm: no ${cookie} cookie to echo`);
|
||||
form.append(field, value);
|
||||
}
|
||||
const res = await this.#exchange(this.#url, headers, "POST", form);
|
||||
const location = res.headers.get("location");
|
||||
return {
|
||||
status: res.status,
|
||||
contentType: res.headers.get("content-type") ?? "",
|
||||
...withDefined("contentDisposition", res.headers.get("content-disposition")),
|
||||
...withDefined("location", location === null ? null : stripQuery(location, this.#url)),
|
||||
body: new Uint8Array(await res.arrayBuffer()),
|
||||
};
|
||||
}
|
||||
|
||||
async #headersNow(): Promise<Headers> {
|
||||
let headers = this.#headers;
|
||||
for (const map of this.#maps) {
|
||||
const entries = await map.entries();
|
||||
for (const [key, value] of entries) {
|
||||
// key name only: header values must not appear in Error messages
|
||||
if (typeof value !== "string") throw new Error(`headers: ${key} is not a string`);
|
||||
}
|
||||
headers = unionHeaders(headers, Object.fromEntries(entries) as Headers);
|
||||
}
|
||||
return headers;
|
||||
}
|
||||
}
|
||||
|
||||
@@ -149,11 +245,16 @@ class Gate {
|
||||
* storage, surviving eviction (instance memory evaporates when the DO naps;
|
||||
* storage is where "this instance means this URL" durably lives).
|
||||
*
|
||||
* Its cookie, when minted by {@link mintInstance}, is not in `init` but in
|
||||
* the header store, whose id goes to the account owner and not to whoever
|
||||
* holds the instance id: the owner can rotate it, and the holder cannot
|
||||
* swap in another account's session.
|
||||
*
|
||||
* @see https://developers.cloudflare.com/durable-objects/ — newUniqueId is
|
||||
* unforgeable (128-bit, inderivable); idFromName by contrast is ambient:
|
||||
* any worker with the namespace binding can recompute it
|
||||
*/
|
||||
export class WebAppDo extends DurableObject<{ WEBAPP?: never }> {
|
||||
export class WebAppDo extends DurableObject<WebAppEnv> {
|
||||
#ready?: Promise<void>;
|
||||
|
||||
/** one-shot configuration; a different-url re-configure is refused */
|
||||
@@ -207,16 +308,46 @@ export class WebAppDo extends DurableObject<{ WEBAPP?: never }> {
|
||||
}
|
||||
return res.text();
|
||||
};
|
||||
return new WebApp(init.url, init.headers ?? {}, send);
|
||||
const exchange: Exchange = async (url, headers, method, body) => {
|
||||
await gate.waitMyTurn();
|
||||
return fetch(url, { method, headers, body, redirect: "manual" });
|
||||
};
|
||||
const maps = init.headerStore === undefined ? [] : [this.#readHeaderStore(init.headerStore)];
|
||||
return new WebApp(init.url, init.headers ?? {}, maps, send, exchange);
|
||||
}
|
||||
|
||||
/** a header map reading the store's read-only view, afresh at each send */
|
||||
#readHeaderStore(storeId: string): HeaderMap {
|
||||
const ns = this.env.MAP_STORE_DO;
|
||||
const store = ns.get(ns.idFromString(storeId));
|
||||
return {
|
||||
async entries() {
|
||||
using rd = await store.readOnly();
|
||||
return await rd.entries();
|
||||
},
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
/** split the cookie (any case of the header name) from the other headers */
|
||||
function splitCookie(headers: Headers): { cookie?: string; rest: Headers } {
|
||||
const rest: Headers = {};
|
||||
let cookie: string | undefined;
|
||||
for (const [key, value] of Object.entries(headers)) {
|
||||
if (key.toLowerCase() === "cookie") cookie = value;
|
||||
else rest[key] = value;
|
||||
}
|
||||
return { cookie, rest };
|
||||
}
|
||||
|
||||
type WebAppEnv = {
|
||||
WEBAPP_DO: DurableObjectNamespace<WebAppDo>;
|
||||
MAP_STORE_DO: DurableObjectNamespace<MapStoreDo>;
|
||||
};
|
||||
|
||||
/** the worker's whole HTTP surface (the default export and the
|
||||
* WebAppFactory entrypoint share it) */
|
||||
const defaultFetch = async (
|
||||
request: Request,
|
||||
env: { WEBAPP_DO: DurableObjectNamespace<WebAppDo> },
|
||||
): Promise<Response> => {
|
||||
const defaultFetch = async (request: Request, env: WebAppEnv): Promise<Response> => {
|
||||
const url = new URL(request.url);
|
||||
|
||||
// mint form: site-agnostic; secret headers (cookie, authorization) are
|
||||
@@ -227,7 +358,7 @@ const defaultFetch = async (
|
||||
|
||||
// mint: browser form POST (form-encoded) or JSON (service-style)
|
||||
if (url.pathname === "/" && request.method === "POST") {
|
||||
return mint(request, env.WEBAPP_DO);
|
||||
return mint(request, env);
|
||||
}
|
||||
|
||||
// the mint form, renderable for a proxying worker: ?render=1 keeps the
|
||||
@@ -246,6 +377,14 @@ const defaultFetch = async (
|
||||
return drive(await app, request);
|
||||
}
|
||||
|
||||
// a header store, by its id (the owner's, not the instance's): PUT
|
||||
// JSON value sets a key (e.g. rotates the cookie), DELETE removes it
|
||||
const entry = url.pathname.match(/^\/store\/([A-Za-z0-9-]+)\/([^/]+)$/);
|
||||
if (entry && (request.method === "PUT" || request.method === "DELETE")) {
|
||||
const store = env.MAP_STORE_DO.get(env.MAP_STORE_DO.idFromString(entry[1]));
|
||||
return storeEntry(store, decodeURIComponent(entry[2]), request);
|
||||
}
|
||||
|
||||
return new Response("not found", { status: 404 });
|
||||
};
|
||||
|
||||
@@ -305,10 +444,7 @@ function withCookie(init: WebAppInit, cookie: string): WebAppInit {
|
||||
}
|
||||
|
||||
/** mint a fresh instance from form-encoded or JSON input */
|
||||
async function mint(
|
||||
request: Request,
|
||||
ns: DurableObjectNamespace<WebAppDo>,
|
||||
): Promise<Response> {
|
||||
async function mint(request: Request, env: WebAppEnv): Promise<Response> {
|
||||
const ct = request.headers.get("content-type") ?? "";
|
||||
let init: WebAppInit;
|
||||
try {
|
||||
@@ -335,13 +471,35 @@ async function mint(
|
||||
return Response.json({ error: "bad url" }, { status: 400 });
|
||||
}
|
||||
|
||||
const id = ns.newUniqueId();
|
||||
const instance = ns.get(id);
|
||||
await instance.configure(init);
|
||||
const app = await instance.handle();
|
||||
// note: id intentionally NOT logged — it is a bearer capability
|
||||
const { id, storeId } = await mintInstance(env, init);
|
||||
// note: ids intentionally NOT logged — they are bearer capabilities
|
||||
console.log({ event: "mint", msPerCall: init.msPerCall });
|
||||
return Response.json({ id: id.toString() }, { headers: { "x-webapp-id": id.toString() } });
|
||||
return Response.json({ id, storeId }, { headers: { "x-webapp-id": id, "x-webapp-store-id": storeId } });
|
||||
}
|
||||
|
||||
/**
|
||||
* a configured instance plus its header store, shaped to hold just the
|
||||
* cookie: the instance id is authority to use the cookie; the store id
|
||||
* (for the account owner) is authority to rotate or remove it. Any
|
||||
* `cookie` header in `init` goes to the store, not to the instance.
|
||||
*/
|
||||
async function mintInstance(env: WebAppEnv, init: WebAppInit): Promise<{ id: string; storeId: string }> {
|
||||
const { cookie, rest } = splitCookie(init.headers ?? {});
|
||||
const storeId = env.MAP_STORE_DO.newUniqueId();
|
||||
const store = env.MAP_STORE_DO.get(storeId);
|
||||
await store.configure("cookie", encodeShapes({ keyShape: "cookie", valueShape: M.string() }));
|
||||
if (cookie !== undefined) {
|
||||
using full = await store.store();
|
||||
await full.init("cookie", cookie);
|
||||
}
|
||||
const id = env.WEBAPP_DO.newUniqueId();
|
||||
await env.WEBAPP_DO.get(id).configure({
|
||||
url: init.url,
|
||||
headers: rest,
|
||||
...(init.msPerCall === undefined ? {} : { msPerCall: init.msPerCall }),
|
||||
headerStore: storeId.toString(),
|
||||
});
|
||||
return { id: id.toString(), storeId: storeId.toString() };
|
||||
}
|
||||
|
||||
/** drive a handle from the /at/<id> JSON protocol (also used by the
|
||||
@@ -368,6 +526,60 @@ async function drive(app: FluentGet, request: Request): Promise<Response> {
|
||||
}
|
||||
}
|
||||
|
||||
/** the /store/<id>/<key> protocol: PUT a JSON value (set, or init if
|
||||
* absent), or DELETE → {ok} (or {error}; messages carry no values) */
|
||||
async function storeEntry(store: DurableObjectStub<MapStoreDo>, key: string, request: Request): Promise<Response> {
|
||||
let value: unknown;
|
||||
if (request.method === "PUT") {
|
||||
try {
|
||||
value = await request.json();
|
||||
} catch (err) {
|
||||
return Response.json({ error: `bad input: ${String(err)}` }, { status: 400 });
|
||||
}
|
||||
}
|
||||
try {
|
||||
using full = await store.store();
|
||||
if (request.method === "DELETE") {
|
||||
if (await full.has(key)) await full.delete(key);
|
||||
} else if (await full.has(key)) {
|
||||
await full.set(key, value);
|
||||
} else {
|
||||
await full.init(key, value);
|
||||
}
|
||||
} catch (err) {
|
||||
// store errors name shapes, never keys or values (see lib/src/store.ts)
|
||||
return Response.json({ error: String(err) }, { status: 400 });
|
||||
}
|
||||
// note: neither id nor value logged
|
||||
console.log({ event: "store", method: request.method });
|
||||
return Response.json({ ok: true });
|
||||
}
|
||||
|
||||
/** a cookie header's name → value (the first of a repeated name wins) */
|
||||
function parseCookies(header: string): Map<string, string> {
|
||||
const jar = new Map<string, string>();
|
||||
for (const pair of header.split(";")) {
|
||||
const eq = pair.indexOf("=");
|
||||
if (eq < 0) continue;
|
||||
const name = pair.slice(0, eq).trim();
|
||||
if (name && !jar.has(name)) jar.set(name, pair.slice(eq + 1).trim());
|
||||
}
|
||||
return jar;
|
||||
}
|
||||
|
||||
/** `ref` resolved against `base`, query and fragment dropped: they can carry credentials */
|
||||
function stripQuery(ref: string, base: string): string {
|
||||
const url = new URL(ref, base);
|
||||
url.search = "";
|
||||
url.hash = "";
|
||||
return url.href;
|
||||
}
|
||||
|
||||
/** `{[key]: value}`, or `{}` when value is null: optional fields stay absent */
|
||||
function withDefined<K extends string>(key: K, value: string | null): { [P in K]?: string } {
|
||||
return (value === null ? {} : { [key]: value }) as { [P in K]?: string };
|
||||
}
|
||||
|
||||
function unionHeaders(base: Headers, more: Headers): Headers {
|
||||
const merged = { ...base };
|
||||
for (const [key, value] of Object.entries(more)) {
|
||||
@@ -396,37 +608,37 @@ function unionQuery(base: string, params: Query): string {
|
||||
/**
|
||||
* the RPC service of the webapp worker. Flow:
|
||||
*
|
||||
* const { id, app } = await env.WEBAPP.newWebApp({
|
||||
* url: "https://api.github.com", headers: { authorization: `Bearer ${t}` },
|
||||
* const { id, storeId, app } = await env.WEBAPP.newWebApp({
|
||||
* url: "https://diigo.example/", headers: { cookie: c },
|
||||
* msPerCall: 250,
|
||||
* });
|
||||
* // hold id (persist it, hand it to workers you trust), then e.g.:
|
||||
* const text = await app.query({ per_page: 5 }).get();
|
||||
* // later, from another worker (same account) holding the id:
|
||||
* const app2 = env.WEBAPP.at(id);
|
||||
* // the account owner, holding storeId, after logging in again:
|
||||
* await (await env.WEBAPP.store(storeId)).set("cookie", c2);
|
||||
*/
|
||||
export class WebAppFactory extends WorkerEntrypoint {
|
||||
/** the same HTTP surface as the default export, over the entrypoint
|
||||
* (for service-binding callers that proxy pages, e.g. diigo-cache's
|
||||
* /_connect → this form) */
|
||||
async fetch(request: Request): Promise<Response> {
|
||||
const env = this.env as { WEBAPP_DO: DurableObjectNamespace<WebAppDo> };
|
||||
return defaultFetch(request, env);
|
||||
return defaultFetch(request, this.env);
|
||||
}
|
||||
|
||||
/** mint a fresh instance: unforgeable uniqueId + configured fluent stub.
|
||||
* The caller decides who else ever learns the id. (If the caller only
|
||||
/** mint a fresh instance: unforgeable uniqueId + configured fluent stub,
|
||||
* plus the id of its header store (see {@link mintInstance}). The
|
||||
* caller decides who else ever learns either id. (If the caller only
|
||||
* wanted the id, it must dispose() the app stub — workerd warns about
|
||||
* undisposed stubs; see the `at()` alternative + `/at/<id>` route for
|
||||
* id-only flows.) */
|
||||
async newWebApp(init: WebAppInit): Promise<{ id: string; app: FluentGet }> {
|
||||
async newWebApp(init: WebAppInit): Promise<{ id: string; storeId: string; app: FluentGet }> {
|
||||
new URL(init.url); // fail before minting on garbage
|
||||
const { id, storeId } = await mintInstance(this.env, init);
|
||||
const ns = this.env.WEBAPP_DO;
|
||||
const id = ns.newUniqueId();
|
||||
const instance = ns.get(id);
|
||||
await instance.configure(init);
|
||||
const app = await instance.handle();
|
||||
return { id: id.toString(), app };
|
||||
const app = await ns.get(ns.idFromString(id)).handle();
|
||||
return { id, storeId, app };
|
||||
}
|
||||
|
||||
/** reconnect by a previously-obtained id */
|
||||
@@ -434,6 +646,13 @@ export class WebAppFactory extends WorkerEntrypoint {
|
||||
const ns = this.env.WEBAPP_DO;
|
||||
return ns.get(ns.idFromString(id)).handle();
|
||||
}
|
||||
|
||||
/** a header store by its id, with full authority (the instance id
|
||||
* reaches none: see {@link mintInstance}) */
|
||||
async store(storeId: string) {
|
||||
const ns = this.env.MAP_STORE_DO;
|
||||
return ns.get(ns.idFromString(storeId)).store();
|
||||
}
|
||||
}
|
||||
function sleep(ms: number): Promise<void> {
|
||||
return new Promise((r) => setTimeout(r, ms));
|
||||
|
||||
@@ -0,0 +1,131 @@
|
||||
import { env } from "cloudflare:workers";
|
||||
import { runInDurableObject } from "cloudflare:test";
|
||||
import { afterEach, describe, expect, it, vi } from "vitest";
|
||||
import worker, { type WebAppInit } from "../src/index.js";
|
||||
|
||||
type TestEnv = {
|
||||
WEBAPP_DO: DurableObjectNamespace<import("../src/index.js").WebAppDo>;
|
||||
MAP_STORE_DO: DurableObjectNamespace<import("../src/index.js").MapStoreDo>;
|
||||
};
|
||||
const testEnv = env as unknown as TestEnv;
|
||||
|
||||
const URL_ = "https://diigo.example/api/";
|
||||
|
||||
/** the cookie header of each outbound fetch */
|
||||
function spyCookies(): (string | null)[] {
|
||||
const seen: (string | null)[] = [];
|
||||
vi.spyOn(globalThis, "fetch").mockImplementation(async (_input, init) => {
|
||||
seen.push(new Headers(init?.headers).get("cookie"));
|
||||
return new Response("ok");
|
||||
});
|
||||
return seen;
|
||||
}
|
||||
|
||||
afterEach(() => {
|
||||
vi.restoreAllMocks();
|
||||
});
|
||||
|
||||
/** the worker's HTTP surface, as a JSON client */
|
||||
const call = (path: string, method: string, body?: unknown) =>
|
||||
worker.fetch(
|
||||
new Request(`https://webapp.example${path}`, {
|
||||
method,
|
||||
body: body === undefined ? undefined : JSON.stringify(body),
|
||||
headers: { "content-type": "application/json" },
|
||||
}),
|
||||
testEnv,
|
||||
);
|
||||
|
||||
const mint = async (init: WebAppInit) => {
|
||||
const { id, storeId } = (await (await call("/", "POST", init)).json()) as { id: string; storeId: string };
|
||||
const instance = testEnv.WEBAPP_DO.get(testEnv.WEBAPP_DO.idFromString(id));
|
||||
return { id, storeId, instance };
|
||||
};
|
||||
|
||||
/** the failure of a call: its error status, or what it threw; "" if it succeeds */
|
||||
async function errorOf(response: Promise<Response>): Promise<string> {
|
||||
try {
|
||||
const res = await response;
|
||||
return res.ok ? "" : `${res.status} ${await res.text()}`;
|
||||
} catch (err) {
|
||||
return String(err);
|
||||
}
|
||||
}
|
||||
|
||||
/** `init` as stored */
|
||||
const storedInit = (stub: DurableObjectStub) =>
|
||||
runInDurableObject(stub, (_obj, state) => state.storage.kv.get<WebAppInit>("init"));
|
||||
|
||||
describe("cookie rotation", () => {
|
||||
it("mint puts the cookie in the header store, not init; sends carry it", async () => {
|
||||
const seen = spyCookies();
|
||||
const { storeId, instance } = await mint({
|
||||
url: URL_,
|
||||
headers: { Cookie: "session=a", accept: "application/json" },
|
||||
});
|
||||
expect(await storedInit(instance)).toEqual({
|
||||
url: URL_,
|
||||
headers: { accept: "application/json" },
|
||||
headerStore: storeId,
|
||||
});
|
||||
using app = await instance.handle();
|
||||
await app.get();
|
||||
expect(seen).toEqual(["session=a"]);
|
||||
});
|
||||
|
||||
it("the store id rotates under handles already out, and revokes", async () => {
|
||||
const seen = spyCookies();
|
||||
const { storeId, instance } = await mint({ url: URL_, headers: { cookie: "session=a" } });
|
||||
using app = await instance.handle();
|
||||
await app.pathjoin("x").get();
|
||||
expect(await (await call(`/store/${storeId}/cookie`, "PUT", "session=b")).json()).toEqual({ ok: true });
|
||||
await app.pathjoin("x").get();
|
||||
expect(await (await call(`/store/${storeId}/cookie`, "DELETE")).json()).toEqual({ ok: true });
|
||||
await app.get();
|
||||
expect(seen).toEqual(["session=a", "session=b", null]);
|
||||
});
|
||||
|
||||
it("the instance id reaches no store: its holder cannot swap in another session", async () => {
|
||||
const seen = spyCookies();
|
||||
const { id, instance } = await mint({ url: URL_, headers: { cookie: "session=alice" } });
|
||||
expect(await errorOf(call(`/store/${id}/cookie`, "PUT", "session=mallory"))).not.toBe("");
|
||||
expect((await call(`/at/${id}/cookie`, "POST", { cookie: "session=mallory" })).status).toBe(404);
|
||||
using app = await instance.handle();
|
||||
await app.get();
|
||||
expect(seen).toEqual(["session=alice"]);
|
||||
});
|
||||
|
||||
it("the store's shapes admit only the cookie; errors carry no values", async () => {
|
||||
const { storeId } = await mint({ url: URL_ });
|
||||
const err = await errorOf(call(`/store/${storeId}/authorization`, "PUT", "Bearer s3kr1t"));
|
||||
expect(err).toContain('cookie: key: must match \\"cookie\\"');
|
||||
expect(err).not.toContain("s3kr1t");
|
||||
expect(await errorOf(call(`/store/${storeId}/cookie`, "PUT", 42))).toContain("must match M.string()");
|
||||
});
|
||||
|
||||
it("an instance minted before header stores keeps its cookie in init, untouched", async () => {
|
||||
const seen = spyCookies();
|
||||
const instance = testEnv.WEBAPP_DO.get(testEnv.WEBAPP_DO.newUniqueId());
|
||||
const old = { url: URL_, headers: { cookie: "session=old" }, msPerCall: 0 };
|
||||
await runInDurableObject(instance, (_obj, state) => state.storage.put("init", old));
|
||||
using app = await instance.handle();
|
||||
await app.get();
|
||||
expect(seen).toEqual(["session=old"]);
|
||||
expect(await storedInit(instance)).toEqual(old);
|
||||
});
|
||||
|
||||
it("a fixed header contradicting the stored cookie fails at send, naming no values", async () => {
|
||||
spyCookies();
|
||||
const { instance } = await mint({ url: URL_, headers: { cookie: "session=a" } });
|
||||
using app = await instance.handle();
|
||||
const err = await app
|
||||
.withHeader("cookie", "session=mine")
|
||||
.get()
|
||||
.then(
|
||||
() => "",
|
||||
(e: unknown) => String(e),
|
||||
);
|
||||
expect(err).toContain("headers: contradicts on cookie");
|
||||
expect(err).not.toContain("session=");
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,117 @@
|
||||
import { env } from "cloudflare:workers";
|
||||
import { afterEach, describe, expect, it, vi } from "vitest";
|
||||
import worker, { type WebAppInit } from "../src/index.js";
|
||||
|
||||
type TestEnv = {
|
||||
WEBAPP_DO: DurableObjectNamespace<import("../src/index.js").WebAppDo>;
|
||||
MAP_STORE_DO: DurableObjectNamespace<import("../src/index.js").MapStoreDo>;
|
||||
};
|
||||
const testEnv = env as unknown as TestEnv;
|
||||
|
||||
const URL_ = "https://site.example/app/export/7/data";
|
||||
|
||||
type Seen = { url: string; redirect?: string; contentType: string | null; form: Record<string, string> };
|
||||
|
||||
/** each outbound fetch, its form fields read back; answers with `res()` */
|
||||
function spyForms(res: () => Response): Seen[] {
|
||||
const seen: Seen[] = [];
|
||||
vi.spyOn(globalThis, "fetch").mockImplementation(async (input, init) => {
|
||||
const req = new Request(input as string, init as RequestInit);
|
||||
const form = Object.fromEntries([...(await req.formData()).entries()].map(([k, v]) => [k, String(v)]));
|
||||
seen.push({ url: req.url, redirect: init?.redirect, contentType: req.headers.get("content-type"), form });
|
||||
return res();
|
||||
});
|
||||
return seen;
|
||||
}
|
||||
|
||||
afterEach(() => {
|
||||
vi.restoreAllMocks();
|
||||
});
|
||||
|
||||
const mint = async (init: WebAppInit) => {
|
||||
const res = await worker.fetch(
|
||||
new Request("https://webapp.example/", {
|
||||
method: "POST",
|
||||
body: JSON.stringify(init),
|
||||
headers: { "content-type": "application/json" },
|
||||
}),
|
||||
testEnv,
|
||||
);
|
||||
const { id, storeId } = (await res.json()) as { id: string; storeId: string };
|
||||
return { id, storeId, instance: testEnv.WEBAPP_DO.get(testEnv.WEBAPP_DO.idFromString(id)) };
|
||||
};
|
||||
|
||||
/** the failure of a call: what it threw; "" if it succeeds */
|
||||
async function errorOf(p: Promise<unknown>): Promise<string> {
|
||||
try {
|
||||
await p;
|
||||
return "";
|
||||
} catch (err) {
|
||||
return String(err);
|
||||
}
|
||||
}
|
||||
|
||||
describe("postForm", () => {
|
||||
it("echoes a cookie from the header store into a form field; the holder never sees it", async () => {
|
||||
const seen = spyForms(
|
||||
() =>
|
||||
new Response("col1,col2", {
|
||||
headers: {
|
||||
"content-type": "text/csv",
|
||||
"content-disposition": 'attachment;filename="x.csv"',
|
||||
"set-cookie": "sess=next; HttpOnly",
|
||||
},
|
||||
}),
|
||||
);
|
||||
const { instance } = await mint({ url: URL_, headers: { cookie: "a=1; tok=s3cret; b=2" } });
|
||||
using app = await instance.handle();
|
||||
const reply = await app.query({ since: "2026-07-13\x1f" }).postForm({ kind: "all" }, { csrf: "tok" });
|
||||
expect(seen).toEqual([
|
||||
{
|
||||
url: `${URL_}?since=2026-07-13%1F`,
|
||||
redirect: "manual",
|
||||
contentType: expect.stringMatching(/^multipart\/form-data; boundary=/),
|
||||
form: { kind: "all", csrf: "s3cret" },
|
||||
},
|
||||
]);
|
||||
expect(reply).toEqual({
|
||||
status: 200,
|
||||
contentType: "text/csv",
|
||||
contentDisposition: 'attachment;filename="x.csv"',
|
||||
body: new TextEncoder().encode("col1,col2"),
|
||||
});
|
||||
});
|
||||
|
||||
it("does not follow a redirect: the 3xx comes back, location query-stripped", async () => {
|
||||
spyForms(() => new Response(null, { status: 302, headers: { location: "/login?next=s3cret#x" } }));
|
||||
const { instance } = await mint({ url: URL_, headers: { cookie: "tok=t" } });
|
||||
using app = await instance.handle();
|
||||
const reply = await app.postForm({}, { csrf: "tok" });
|
||||
expect(reply).toEqual({ status: 302, contentType: "", location: "https://site.example/login", body: new Uint8Array() });
|
||||
});
|
||||
|
||||
it("refuses, naming no values: a missing cookie, a field set twice, a content-type header", async () => {
|
||||
spyForms(() => new Response("ok"));
|
||||
const { instance } = await mint({ url: URL_, headers: { cookie: "tok=s3cret" } });
|
||||
using app = await instance.handle();
|
||||
expect(await errorOf(app.postForm({}, { csrf: "nope" }))).toMatch(/no nope cookie to echo/);
|
||||
const twice = await errorOf(app.postForm({ csrf: "mine" }, { csrf: "tok" }));
|
||||
expect(twice).toMatch(/contradicts on csrf/);
|
||||
expect(twice).not.toMatch(/s3cret|mine/);
|
||||
const typed = app.withHeader("content-type", "text/plain");
|
||||
expect(await errorOf(typed.postForm())).toMatch(/content-type is set by the form/);
|
||||
});
|
||||
|
||||
it("rotating the store's cookie changes what is echoed", async () => {
|
||||
const seen = spyForms(() => new Response("ok"));
|
||||
const { storeId, instance } = await mint({ url: URL_, headers: { cookie: "tok=one" } });
|
||||
using app = await instance.handle();
|
||||
await app.postForm({}, { csrf: "tok" });
|
||||
await worker.fetch(
|
||||
new Request(`https://webapp.example/store/${storeId}/cookie`, { method: "PUT", body: JSON.stringify("tok=two") }),
|
||||
testEnv,
|
||||
);
|
||||
await app.postForm({}, { csrf: "tok" });
|
||||
expect(seen.map((s) => s.form.csrf)).toEqual(["one", "two"]);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,7 @@
|
||||
{
|
||||
"extends": "../tsconfig.json",
|
||||
"compilerOptions": {
|
||||
"types": ["@cloudflare/vitest-plugin/types"]
|
||||
},
|
||||
"include": ["./**/*", "../worker-configuration.d.ts"]
|
||||
}
|
||||
@@ -7,6 +7,7 @@
|
||||
"esModuleInterop": true,
|
||||
"verbatimModuleSyntax": true,
|
||||
"skipLibCheck": true,
|
||||
"lib": ["ES2022", "ESNext.Disposable"],
|
||||
"noEmit": true,
|
||||
"types": []
|
||||
},
|
||||
|
||||
@@ -0,0 +1,9 @@
|
||||
import { defineProject } from "vitest/config";
|
||||
import { cloudflareTest } from "@cloudflare/vitest-plugin";
|
||||
|
||||
export default defineProject({
|
||||
plugins: [cloudflareTest({ wrangler: { configPath: "./wrangler.jsonc" } })],
|
||||
test: {
|
||||
include: ["tests/**/*.test.ts"],
|
||||
},
|
||||
});
|
||||
@@ -5,7 +5,13 @@
|
||||
"compatibility_date": "2026-09-29",
|
||||
"observability": { "enabled": true },
|
||||
"durable_objects": {
|
||||
"bindings": [{ "name": "WEBAPP_DO", "class_name": "WebAppDo" }]
|
||||
"bindings": [
|
||||
{ "name": "WEBAPP_DO", "class_name": "WebAppDo" },
|
||||
{ "name": "MAP_STORE_DO", "class_name": "MapStoreDo" }
|
||||
]
|
||||
},
|
||||
"migrations": [{ "tag": "v1", "new_sqlite_classes": ["WebAppDo"] }]
|
||||
"migrations": [
|
||||
{ "tag": "v1", "new_sqlite_classes": ["WebAppDo"] },
|
||||
{ "tag": "v2", "new_sqlite_classes": ["MapStoreDo"] }
|
||||
]
|
||||
}
|
||||
Reference in new issue
Block a user