Compare commits

..
23 Commits
Author SHA1 Message Date
dckc ff2e7841f2 capsite/webapp: postForm, a multipart POST that can echo a cookie into a field
Some sites check a double-submit token: the session value sent as a
cookie and again in the form body. postForm(fields, echoCookies) builds
a multipart body and copies the named cookies (from the header store,
at send time) into fields, so the caller never sees the value.

It returns a Reply instead of throwing on non-OK: status, content type,
content-disposition, the redirect location with its query stripped, and
the body as bytes (the caller decodes, e.g. per a charset the body
declares). Redirects are not followed, so an expired session shows up
as a 3xx. Set-Cookie is not passed back. Errors name keys, never values.

Generated-by: Claude Opus 5.5
2026-10-11 14:22:09 -05:00
dckc 7876938f3e capsite/docs: describe the wiring diagram for screen readers
The SVG gets role="img" with a <title> and a <desc> that says in prose
who refers to what, since the arrows carry the meaning and are invisible
to assistive tech. The rendered PNG is unchanged.

Generated-by: Claude Opus 5.5
2026-10-09 02:32:22 -05:00
dckc a921d65021 capsite/docs: cookie-rotation wiring diagram (PNG)
A headless-Chromium screenshot of docs/cookie-rotation-wiring.html.

Generated-by: Claude Opus 5.5
2026-10-09 02:32:22 -05:00
dckc 055b572297 capsite/docs: cookie-rotation wiring diagram (source)
Who holds what after mint: Alice holds the store id (rotate, revoke),
diigo-cache and a leaked-id Mallory hold only the instance, whose
init.headerStore reaches the store's read-only facet. Drawn as an SVG
page in the style of MarkM's ERTP slides.

Generated-by: Claude Opus 5.5
2026-10-09 02:32:22 -05:00
dckc 4dd3fe6bde capsite/diigo-cache: the wire page hands the owner the store id
webapp's mint now answers a store id (x-webapp-store-id) beside the
instance id: authority to rotate or revoke the diigo cookie. The wire
page shows it apart from the instance id, with its own copy button,
petname, and password-manager save; the cache never gets it. A webapp
that sends none gets a note on the page and a warning in the log.

Generated-by: Claude Opus 5.5
2026-10-09 02:32:22 -05:00
dckc 3f711a5f4b capsite/webapp: the cookie's store is the owner's, apart from the instance
Letting the instance id rotate the cookie let its holder (the cache, or
whoever learns the id) swap in another account's session, which the
owner's logout would not stop. Now mint makes two things: the instance,
and a header store (lib's MapStoreDo, shaped ("cookie", { keyShape:
"cookie", valueShape: M.string() })). The instance id is authority to
use the cookie; the store id, for the account owner, to rotate (PUT
/store/<storeId>/cookie, or WebAppFactory.store) or revoke (DELETE).
Instances read the store's read-only view at every send, via init's new
headerStore.

This replaces the in-instance cookie store and its migration: instances
minted earlier keep their cookie in init.headers, untouched, and simply
cannot rotate. The MapStoreDo class arrives as wrangler migration v2
(additive).

Generated-by: Claude Opus 5.5
2026-10-09 02:32:22 -05:00
dckc 357e4561c4 capsite/lib: endo's own matcher data, smallcaps, and MapStoreDo
Matchers are now exactly what @endo/patterns makes: tagged records as
from pass-style's makeTagged (M.string() is match:string with payload
[], M.any() is match:any with payload undefined). Structured clone drops
their symbol-keyed and non-enumerable properties, so they cross DO
storage and Workers RPC in marshal's smallcaps encoding (the subset
patterns need).

MapStoreDo is a MapStore as a DO of its own: its creator fixes label and
(encoded) shapes once; its id is full authority, and readOnly() hands
out the queries alone.

Generated-by: Claude Opus 5.5
2026-10-09 02:32:22 -05:00
dckc 10414c2ce6 capsite/webapp: rotate the cookie in place
The cookie moves out of the immutable init into a MapStore (key shape
"cookie", value M.string()) in the instance's DO storage. Handles get
its read-only view via the new WebApp.withHeaderMap, which reads at
every send, so a rotation reaches handles already out and the webapp
id, and with it the cache's wiring, stays put (checklist goal 6).

Whoever holds the id can rotate, by RPC (WebAppFactory.cookies(id)) or
POST /at/<id>/cookie; the diigo account owner revokes by logging out.

Durable state: instances minted earlier keep the cookie in
init.headers. The first handle() moves it to the store and rewrites
init without it, in one transaction. Code from before this commit
would then see no cookie, so rolling back past it needs the reverse
move.

webapp gains a package.json and a vitest workerd pool for the tests.

Generated-by: Claude Opus 5.5
2026-10-09 02:32:22 -05:00
dckc 9179bdbf38 capsite/lib: MapStore and patterns work-alikes
Enough of @agoric/store's MapStore and @endo/patterns' M for the
credential-rotation design: makeMapStore(label, { keyShape, valueShape },
kv) keeps entries in a DO's sync KV under store:<label>:, and readOnly()
hands out the queries alone. Facets are workerd RpcTargets in place of
endo's Far.

Unlike endo, error messages never include keys or values: the values
here are credentials, and messages cross RPC into logs.

Generated-by: Claude Opus 5.5
2026-10-09 02:32:22 -05:00
dckc 1d8a220bbc capsite: checklist: 2-iv, the weekly draft works in dev
Generated-by: Claude Opus 5.5
2026-10-09 02:12:41 -05:00
dckc 68132ff4b3 diigo-week: a dashboard card drafts last week in one click
The card shows last week's issue with a link to it, or a button to
draft it. Drafting a week that already has an issue (draft or
published) is refused, with a link to that issue, rather than making
a duplicate.

Generated-by: Claude Opus 5.5
2026-10-09 02:12:41 -05:00
dckc 9c09e8c5a4 capsite: checklist: 4b, consider scruffier tags
Generated-by: Claude Opus 5.5
2026-10-09 02:12:41 -05:00
dckc 94a059af25 diigo-week: draft a potluck issue from a week of diigo bookmarks
An EmDash plugin (native, content:write only) with one admin page:
a diigo username and an ISO week. It reads diigo-cache's public page
(?in=<week>&by=created) as microformats, follows rel=next, and
creates a draft in issues shaped like seed/issues.json.

Transport is a DIIGO_CACHE service binding rather than ctx.http.fetch:
EmDash's SSRF guard refuses localhost, and the binding names one
worker. Logic lives in src/week.ts with the fetcher passed in; tests
read fixtures in diigo-cache's markup. The checklist's goal 2 now
names potluck; DiigoBridge waits.

Generated-by: Claude Opus 5.5
2026-10-09 02:12:41 -05:00
dckc be796d9d0d diigo-cache: the public page is an h-feed of h-entry bookmarks
Microformats are the public page's API for other sites (potluck
first): each bookmark is an h-entry with u-bookmark-of, p-name,
p-summary, a p-category per tag (linked to a tag search),
dt-published (bookmarked) and dt-updated (last edited). Prev/next
links carry rel=prev/next, so a reader can follow a filter's pages.
Tests parse the page with microformats-parser to pin the contract.

Generated-by: Claude Opus 5.5
2026-10-09 01:49:40 -05:00
dckc c66aaea366 potluck, diigo-cache: docs one line per paragraph; potluck deploy caveat
Obsidian and GitHub render each newline as a line break, so the
hard-wrapped READMEs and CONTRIBUTINGs rendered ragged. Whitespace
only, except a note in potluck's CONTRIBUTING that other workerd
hosts are untested: they'd have to supply the D1 and R2 bindings.

Generated-by: Claude Opus 5.5
2026-10-09 01:30:56 -05:00
dckc a25bd699e1 potluck: README: workerd, not a vendor
Generated-by: Claude Opus 5.5
2026-10-09 01:29:31 -05:00
dckc ea3e256bb8 diigo-cache: README: workerd, not a vendor
Generated-by: Claude Opus 5.5
2026-10-09 01:28:50 -05:00
dckc 0af7a3ea6c diigo-cache: README sells and gets to TADA; CONTRIBUTING updated
New README: two sentences, the dev run through connect, fill, and a
filtered public page, then features. CONTRIBUTING gains the 1d
filters and its own run and deploy sections instead of pointing at
the checklist; links to the old README now point at CONTRIBUTING.

Generated-by: Claude Opus 5.5
2026-10-09 01:27:45 -05:00
dckc 9719fd0aa8 diigo-cache: README becomes CONTRIBUTING; AGENTS.md links to it
Generated-by: Claude Opus 5.5
2026-10-09 01:27:12 -05:00
dckc bce2424cc0 potluck: README: a heading that says something
Generated-by: Claude Opus 5.5
2026-10-09 01:25:47 -05:00
dckc 7f1e19fe85 potluck: README: two sentences, then getting started, then features
Generated-by: Claude Opus 5.5
2026-10-09 01:25:13 -05:00
dckc 8a812d76e3 potluck: README sells and gets to TADA; CONTRIBUTING has the rest
The README was EmDash's template blurb. Now it says what potluck is
and its first block runs from a fresh clone to /issues, using a new
`pnpm seed:issues` script that applies seed/issues.json to the local
D1 database. CONTRIBUTING covers how it's put together, seeds, checks,
the EmDash rules, and deploying; the template's stale notes (Inter,
three-column articles) are gone. AGENTS.md is now a symlink to it.

Generated-by: Claude Opus 5.5
2026-10-09 01:24:10 -05:00
dckc 002f08b456 diigo-cache: date windows and search on the public page (1d)
/public/<username> takes days, after, before, in (year, month,
quarter, ISO week), by=updated|created, and q (words, "phrases",
#tags, -negation, gmail/Slack/GitHub date operators). JSON answers
echo the parsed filter; a malformed one is a 400.

Filters query the stored items in sqlite through the read-only
viewer instead of walking diigo pages: the public page keeps no
fetch authority, and a filled cache already holds every item.
Results say `complete` when every diigo page is stored; freshness
is 1f's head refresh. So the fetch budget and the O(log pages)
walk test go, and the fake deep account shrinks to 300.

Generated-by: Claude Opus 5.5
2026-10-09 01:17:03 -05:00
54 changed files with 13194 additions and 484 deletions

No files matched your search

+24 -38
View File
@@ -23,7 +23,7 @@ narrative.
goal 5.
- [x] **1c. diigo-dump evolves into a bookmark cache**
([diigo-cache/README.md](diigo-cache/README.md); design in the
([diigo-cache/CONTRIBUTING.md](diigo-cache/CONTRIBUTING.md); design in the
JSDoc of `src/cache.ts`, `src/index.ts`, `src/pm-save.ts`):
- [x] 1c-i. Two independent page caches, private and public
(`PagedCache`, `DiigoCache`, `PublicCache`).
@@ -42,20 +42,16 @@ narrative.
2026-10-07, via `scripts/compare-export.ts`. Getting there
found diigo's `page_num` is 0-based (schema v2 migration).
- [ ] **1d. Date windows and search on the public page**, toward the
- [x] **1d. Date windows and search on the public page**, toward the
weekly PESOS roundup of [#23](https://github.com/dckc/madmode-blog/issues/23)
(a week's permalink: `in=2026-W41`). One filter, two spellings:
devs pass `{ days: 7, q: "#weekly" }`, URLs the same fields;
`q` takes gmail/Slack/GitHub operators. UTC, ISO weeks, half-open
windows; a fetch budget per request. Design: the API in
`diigo-cache/src/query.ts` and the skipped suites in
`tests/query.test.ts` and `tests/caches.test.ts` ("public page
filters"); done when they're unskipped and green.
Reconcile with 1f first: `query.ts` plans to filter while walking
diigo's pages (binary search by `updated`), but after 1c-vi a
filled cache holds every item, so 1d can query `items` in sqlite
and spend diigo fetches only on a head refresh (1f). Choose
before implementing; the skipped suites may need adjusting.
windows (`diigo-cache/src/query.ts`; `tests/query.test.ts`,
`tests/caches.test.ts` "public page filters"). Filters read
stored `items` only (the public page never fetches): results are
as fresh as the cache, and say `complete` when every diigo page
is stored. Keeping them fresh is 1f. 2026-10-09.
- [x] **1e. Fill job: a button that grabs every page** into a cache
from the webkey page `/cache/#<key>`, which trades the key for a
@@ -93,41 +89,31 @@ narrative.
[the issues sketch](potluck/issues-sketch.md))? If the policy
changes, stripping stored rows is a durable-state migration.
- [ ] **2. madmode reads the public cache**: public data, so no
capability, no cookie, no plugin sync job, no cron on madmode's
side. madmode fetches `GET /public/dckc-madmode` with
`Accept: application/json` (plus 1d's `in=`/`days=`/`q=`).
- [ ] 2-i. Transport: a service binding to diigo-cache when both
are deployed (no public hop, so no rate-limit exposure);
in dev, a base URL (`http://localhost:8787`).
- [ ] 2-ii. The JSON shape of the public page is a contract
between two workers: pin it in a diigo-cache test (and
madmode's reader tests against the same fixture).
- [ ] 2-iii. Decide `DiigoBridge`'s fate: madmode doesn't need it
(public reads need no cache id), so its "cache-id source"
TODO (`src/index.ts`, README "RPC use") is either dropped
with the entrypoint or kept for a private-reading caller.
- [ ] **3. `/bookmarks` renders real data** — titles, tags, dates at
http://localhost:4321/bookmarks, newest first.
- [ ] **4. The weekly page**: `/bookmarks/2026-W41` → the cache's
`in=2026-W41` (needs 1d and 1f). Decide (user's call): a raw
week list, or a draft post the user edits before publishing
(the PESOS practice is editorial; a tag like `#weekly` in `q`
is one cheap form of discretion).
- [ ] **2. potluck reads the public cache**: public data, so no capability, no cookie, no secret on potluck's side. The [diigo-week](diigo-week/README.md) plugin reads `GET /public/dckc-madmode?in=<week>&by=created` as microformats (an `h-feed` of `h-entry` bookmarks). Working in dev 2026-10-09; 2-iii postponed.
- [x] 2-i. Transport: the `DIIGO_CACHE` service binding in potluck's `wrangler.jsonc`, in dev too: the plugin's `ctx.http.fetch` refuses localhost (EmDash's SSRF guard), and a binding names one worker, not a host.
- [x] 2-ii. The page's microformats are the contract: diigo-cache's tests parse it with microformats-parser; diigo-week's tests read fixtures in the same markup.
- [ ] 2-iii. Decide `DiigoBridge`'s fate (postponed: nothing on this path uses it; it matters only for a caller that reads private items).
- [x] 2-iv. Run it in dev: diigo-cache and potluck both up; the dashboard card → a draft issue of 22 bookmarks for 2026-W40 (2026-10-09). The service binding resolves across the two dev servers.
- [ ] ~~**3. `/bookmarks` renders real data**~~: superseded by potluck's issues (the madmode starter is gone).
- [ ] **4. The weekly draft**: decided (2026-10-09): a draft issue you edit before publishing, made by diigo-week. The slug comes from the title (`week-2026-w40`).
- [ ] **4b. Consider scruffier tags.** EmDash tags are taxonomy terms (rows that must exist before use); diigo-week auto-creates them, so drafting feels scruffy, but the model is neat. Alternative: the inline `/tag/<tag>` links in issue bodies *are* the tags (matching the microformats `p-category` reading), and potluck's `/tag/<tag>` page finds issues by scanning bodies. Cost: no EmDash tag field or index; a scan over a few dozen issues a year is cheap. Open: does the admin's tag field let you type a new tag inline?
- [ ] **5. Deployed end-to-end** (order matters):
- [ ] 5-i. webapp: `cd webapp && npx wrangler deploy`
- [ ] 5-ii. diigo-cache: rate-limit `/api` (and `/public/*`) first
(diigo-cache README, "Deployment concern"); deploy; create
(diigo-cache CONTRIBUTING, "Deployment concern"); deploy; create
against prod (ids → password manager; no worker secret);
run the fill job.
- [ ] 5-iii. madmode: deploy with the service binding; the
weekly page renders against prod.
- [ ] **6. Cookie expiry**: when diigo's session ends, head refreshes
fail; the public page keeps serving what's stored (say so on the
page, don't fail silently). Rotation is manual: re-mint via
`/_connect`. Open question: can an existing cache be re-wired to
a new webapp id (keeping its cache id and public username), or
does re-mint mean a new cache? Prefer re-wire — durable ids.
page, don't fail silently). Rotation is in place, keeping the
webapp id (so the cache's wiring and ids hold), by the store id
minted alongside it and kept by the account owner: `PUT
/store/<storeId>/cookie` (webapp README, "The header store").
Still open: a rotate form in diigo-cache's UI.
- [ ] **7. bib sync #195**: repeat the pattern for the next
read-access need (webapp mints for any origin; the cache's
item shape is diigo-specific): [the zotero goal tree](zotero-checklist.md).
@@ -239,5 +225,5 @@ to every goal in this tree.
Oct 6 2026)
- [webapp/README.md](webapp/README.md) — HTTP + RPC surfaces of the
factory worker; open TODO on the mint-form placement
- [diigo-cache/README.md](diigo-cache/README.md) — the cache worker:
- [diigo-cache/CONTRIBUTING.md](diigo-cache/CONTRIBUTING.md) — the cache worker:
create flow, the two DOs, run/deploy details
+1
View File
@@ -0,0 +1 @@
CONTRIBUTING.md
+72
View File
@@ -0,0 +1,72 @@
# Contributing to diigo-cache
diigo-cache is a Cloudflare Worker cache over [the webapp factory](../webapp/README.md): session-cookie reads of diigo bookmarks, cached in Durable-Object SQLite storage. Evolved from the diigo-dump worker (provenance: [mm-diigo `projects/diigo-bak/`](https://github.com/dckc/mm-diigo)); the static API-key path is dead (see [the capsite checklist](../capsite-checklist.md), goal 2 preamble).
## What it holds
- **The create flow, on one port**: `/_connect` renders webapp's own mint form (diigo-prefilled, cookie blank) — wrangler multi-config dev doesn't serve aux workers' HTTP, so `/_mint` relays the form opaquely into the WEBAPP binding (the cookie crosses; nothing on this side opens or logs it). Mint → `/_wire` (webapp id pre-filled, mode public|private) → the created page, where the user saves the ids.
- **Two independent page caches** (`src/cache.ts`), one behavior: a page hit is served from storage with no diigo call; a miss asks the webapp instance in the cache's wiring and etches the page.
- `DiigoCache` (binding `CACHE_DO`): every item. Addressed by its cache id.
- `PublicCache` (binding `PUBLIC_DO`): shared items only — private rows are dropped *before* etching (`publicOnly`), so its storage holds nothing private. Same cookie webapp as the private cache. Public page n is diigo page n, filtered (may show fewer than `PAGE_SIZE`, 100: diigo's cap). `/public/<username>` resolves through the `idFromName("index")` registry, and reads through `viewer()`, a read-only facet: it shows stored pages and fill progress, and can't fetch from diigo or write.
- **Storage**: SQL lives in `src/cache-read.sql` (selects only) and `src/cache-write.sql`; `src/store.ts` turns them into a reader and a writer facet, so the cache logic holds no SQL text.
- **Pages**: home's dump form (cache id + page number, POST) shows raw dump rows; `/public/<username>?page=n` shows an HTML list with a page form (JSON for `Accept: application/json`). Both say `cache: HIT` or `MISS`; every page read also logs `{event: "cache", kind, page, outcome}`.
- **Filters on the public page** (`src/query.ts`): `days`, `after`, `before`, `in` (`2026`, `2026-03`, `2026-Q1`, `2026-W41`), `by=updated|created`, and `q` (words, `"phrases"`, `#tag`, `-` negation, gmail/Slack/GitHub date operators). UTC, half-open windows. The same fields work as a dev-side spec (`toQuery({ days: 7 })`). A filter queries the stored `items` table through `viewer()`, never diigo: results are as fresh as the cache, and JSON says `complete` when every diigo page is stored.
- **Publish later**: `POST /_publish` (cache id in the body) wires a public companion to a private-only cache.
- **The webkey page and Cap'n Web**: `/cache/#<cache id>` is a skeleton page, the same for every key; its script reads the key from the fragment and trades it in-band for a capability over [Cap'n Web](https://github.com/cloudflare/capnweb) at `/api` (`CacheApi.cache(key)` → a `CacheCap` facet: fill, status, the public companion; `src/api.ts`). The worker serves capnweb's browser build itself at `/_capnweb.js`.
- **Fill job**: walks every diigo page into a cache in the background — DO alarms, a few pages per step, progress stored in the DO (`startFill`/`fillStatus`/`stopFill`), driven from the webkey page; the public page shows progress. "Invalidate the last N days" there drops the cached pages holding anything updated that recently (`invalidateRecent`), so the next fill refetches them.
- The `DiigoBridge` entrypoint for other workers (its cache-id source is the goal-2 wiring TODO).
## State surface
Ids are **webkeys** ([the pattern](../docs/webkeys.md)): 256-bit bearer secrets that never ride a query string or path. The webkey page carries the cache id in the URL fragment (never sent to the server) and passes it in-band over Cap'n Web; forms carry ids in POST bodies; logs and pages show only a fingerprint (a prefix of the id's one-way hash).
- **The cache id**: the cache's read capability (bearer; not logged). Held by the user — saved to the password manager under a petname they choose (the username field; never sent to the server) — and passed per POST.
- **The webapp instance id**: the diigo-read capability; lives in each cache DO's wiring, arrives from the mint step.
- **The cookie**: lives **only in webapp's DO storage**; crosses the wire once, at mint time, as an opaque relay.
- **The public cache**: no capability — `/public/<username>` is public by design (a guessable URL; hiding the link doesn't hide the page).
Stored state is meant to be durable: see the breaking-change rule in [capsite AGENTS.md](../AGENTS.md).
## Deployment concern: rate-limit `/api`
Public pages only read storage, so strangers can't make the cache spend your diigo session. `/api` is keyed (a cache id is needed), but Cap'n Web pipelining makes calls cheap to send. Before exposing a deployment:
- rate-limit `/api` (and `/public/*`, which reads storage) — on Cloudflare, the [Workers rate-limit binding](https://developers.cloudflare.com/workers/runtime-apis/bindings/rate-limit/) (keyed by client and/or username); on other workerd hosts, a rate limit in the proxy in front;
- run the fill job (goal 1e): public pages show only what's stored.
## RPC use (from another worker, same account)
```jsonc
// wrangler.jsonc of the caller
"services": [{ "binding": "DIIGO", "service": "diigo-cache", "entrypoint": "DiigoBridge" }]
```
```ts
// cache ids ride goal-2's wiring; see the capsite checklist
const ndjson = await env.DIIGO.dumpNdjson(4); // 4 pages of PAGE_SIZE
```
## Test
pnpm types # regenerate worker-configuration.d.ts (after wrangler.jsonc changes)
pnpm lint # fails if those types are stale; tsc over src/ and tests/
pnpm test # vitest
Tests run the real worker and both DOs in the vitest pool, with `WEBAPP` bound to [a fake webapp](tests/fake-webapp.js) serving diigo-shaped items (`wrangler.tests.jsonc`).
## Run
[The README](README.md) gets a cache going in dev. The same create flow from a terminal:
wid=$(curl -s -X POST http://localhost:8787/_mint --data-urlencode \
'url=https://www.diigo.com/interact_api/' --data-urlencode \
'headers={"user-agent":"Mozilla/5.0","x-requested-with":"XMLHttpRequest","cookie":"<paste>"}' \
--data-urlencode 'msPerCall=500' -D - -o /dev/null | grep -i x-webapp-id)
curl -s -X POST http://localhost:8787/_wire -H 'content-type: application/json' \
-d "{\"webappId\":\"<from $wid>\",\"mode\":\"private\"}"
Check a filled cache against a diigo RSS export: `node scripts/compare-export.ts <cache.sqlite> <export.xml> <out-dir>`.
## Deploy
Not deployed yet; the order matters (goal 5 in [the capsite checklist](../capsite-checklist.md)): webapp first (`cd ../webapp && npx wrangler deploy`), then diigo-cache, after rate-limiting `/api` (above).
+16 -110
View File
@@ -1,120 +1,26 @@
# diigo-cache
Cloudflare Worker cache over [the webapp factory](../webapp/README.md):
session-cookie reads of diigo bookmarks, cached in Durable-Object SQLite
storage. Evolved from the diigo-dump worker (provenance:
[mm-diigo `projects/diigo-bak/`](https://github.com/dckc/mm-diigo)); the
static API-key path is dead (see [the capsite checklist](../capsite-checklist.md),
goal 2 preamble).
Your diigo bookmarks, cached in a [workerd](https://github.com/cloudflare/workerd) worker, with a public page you can filter by week: `/public/<you>?in=2026-W41`. Your diigo session cookie is stored in one Durable Object and nowhere else.
## What it holds
## Getting started
- **The create flow, on one port**: `/_connect` renders webapp's own
mint form (diigo-prefilled, cookie blank) — wrangler multi-config dev
doesn't serve aux workers' HTTP, so `/_mint` relays the form opaquely
into the WEBAPP binding (the cookie crosses; nothing on this side opens
or logs it). Mint → `/_wire` (webapp id pre-filled, mode
public|private) → the created page, where the user saves the ids.
- **Two independent page caches** (`src/cache.ts`), one behavior: a
page hit is served from storage with no diigo call; a miss asks the
webapp instance in the cache's wiring and etches the page.
- `DiigoCache` (binding `CACHE_DO`): every item. Addressed by its
cache id.
- `PublicCache` (binding `PUBLIC_DO`): shared items only — private
rows are dropped *before* etching (`publicOnly`), so its storage
holds nothing private. Same cookie webapp as the private cache.
Public page n is diigo page n, filtered (may show fewer than
`PAGE_SIZE`, 100: diigo's cap).
`/public/<username>` resolves through the `idFromName("index")`
registry, and reads through `viewer()`, a read-only facet: it shows
stored pages and fill progress, and can't fetch from diigo or write.
- **Storage**: SQL lives in `src/cache-read.sql` (selects only) and
`src/cache-write.sql`; `src/store.ts` turns them into a reader and a
writer facet, so the cache logic holds no SQL text.
- **Pages**: home's dump form (cache id + page number, POST) shows raw
dump rows; `/public/<username>?page=n` shows an HTML list with a page
form (JSON for `Accept: application/json`). Both say `cache: HIT` or
`MISS`; every page read also logs `{event: "cache", kind, page,
outcome}`.
- **Publish later**: `POST /_publish` (cache id in the body) wires a
public companion to a private-only cache.
- **The webkey page and Cap'n Web**: `/cache/#<cache id>` is a skeleton
page, the same for every key; its script reads the key from the
fragment and trades it in-band for a capability over
[Cap'n Web](https://github.com/cloudflare/capnweb) at `/api`
(`CacheApi.cache(key)` → a `CacheCap` facet: fill, status, the public
companion; `src/api.ts`). The worker serves capnweb's browser build
itself at `/_capnweb.js`.
- **Fill job**: walks every diigo page into a cache in the background —
DO alarms, a few pages per step, progress stored in the DO
(`startFill`/`fillStatus`/`stopFill`), driven from the webkey page;
the public page shows progress. "Invalidate the last N days" there
drops the cached pages holding anything updated that recently
(`invalidateRecent`), so the next fill refetches them.
- The `DiigoBridge` entrypoint for other workers (its cache-id source is
the goal-2 wiring TODO).
## State surface
Ids are **webkeys** ([the pattern](../docs/webkeys.md)): 256-bit bearer
secrets that never ride a query string or path. The webkey page carries
the cache id in the URL fragment (never sent to the server) and passes
it in-band over Cap'n Web; forms carry ids in POST bodies; logs and pages
show only a fingerprint (a prefix of the id's one-way hash).
- **The cache id**: the cache's read capability (bearer; not logged).
Held by the user — saved to the password manager under a petname they
choose (the username field; never sent to the server) — and passed
per POST.
- **The webapp instance id**: the diigo-read capability; lives in each
cache DO's wiring, arrives from the mint step.
- **The cookie**: lives **only in webapp's DO storage**; crosses the
wire once, at mint time, as an opaque relay.
- **The public cache**: no capability — `/public/<username>` is public
by design (a guessable URL; hiding the link doesn't hide the page).
Stored state is meant to be durable: see the breaking-change rule in
[capsite AGENTS.md](../AGENTS.md).
## Deployment concern: rate-limit `/api`
Public pages only read storage, so strangers can't make the cache spend
your diigo session. `/api` is keyed (a cache id is needed), but Cap'n
Web pipelining makes calls cheap to send. Before exposing a deployment:
- rate-limit `/api` (and `/public/*`, which reads storage) — on
Cloudflare, the
[Workers rate-limit binding](https://developers.cloudflare.com/workers/runtime-apis/bindings/rate-limit/)
(keyed by client and/or username); on other workerd hosts, a rate
limit in the proxy in front;
- run the fill job (goal 1e): public pages show only what's stored.
## RPC use (from another worker, same account)
```jsonc
// wrangler.jsonc of the caller
"services": [{ "binding": "DIIGO", "service": "diigo-cache", "entrypoint": "DiigoBridge" }]
```sh
pnpm install
pnpm dev # → http://localhost:8787/
```
```ts
// cache ids ride goal-2's wiring; see the capsite checklist
const ndjson = await env.DIIGO.dumpNdjson(4); // 4 pages of PAGE_SIZE
```
Then, in the browser:
## Test
1. **mint the diigo session at webapp**: paste the `Cookie` header from diigo.com (devtools, any logged-in request).
2. **wire** it, mode **public**. Save the cache id it shows (the password-manager button does this).
3. **open this cache** → **fill**, and watch it walk your bookmarks.
4. Visit `/public/<your diigo username>?days=7`.
pnpm types # regenerate worker-configuration.d.ts (after wrangler.jsonc changes)
pnpm lint # fails if those types are stale; tsc over src/ and tests/
pnpm test # vitest
## Bookmarks you hold, a page anyone can read
Tests run the real worker and both DOs in the vitest pool, with
`WEBAPP` bound to [a fake webapp](tests/fake-webapp.js) serving
diigo-shaped items (`wrangler.tests.jsonc`).
- **Filter by date and search:** `?days=7`, `?in=2026-W41` (ISO week, UTC), `?after=2026-01-01&before=2026-02-01`, `?by=created`, and `?q=` with words, `"phrases"`, `#tags`, `-negation`, and gmail-style `newer_than:7d`. Add `Accept: application/json` for JSON.
- **Private stays private:** the public cache drops private bookmarks before storing anything, so it never holds them.
- **Ids are capabilities:** the cache id is a bearer key you keep in your password manager. It rides URL fragments and POST bodies, never a path or query string, so it stays out of logs and history.
- **Cheap to read:** pages come from Durable Object SQLite storage, and a fill job in the background fetches each diigo page once.
## Run, deploy
[How to run / connect / deploy is maintained in the capsite checklist](../capsite-checklist.md)
(single source; this README keeps only worker-level facts).
Status: create, dump, publish, and public pages exercised against live
diigo in dev (2026-10-07); not yet deployed.
Design, tests, and deploying: [CONTRIBUTING](CONTRIBUTING.md).
+1
View File
@@ -11,6 +11,7 @@
"devDependencies": {
"@cloudflare/vitest-plugin": "1.3.6",
"@types/node": "^22",
"microformats-parser": "^2.0.6",
"typescript": "^5.9.0",
"vitest": "4.1.11",
"wrangler": "^4.148.0"
+24
View File
@@ -18,6 +18,9 @@ importers:
'@types/node':
specifier: ^22
version: 22.20.5
microformats-parser:
specifier: ^2.0.6
version: 2.0.6
typescript:
specifier: ^5.9.0
version: 5.9.3
@@ -613,6 +616,10 @@ packages:
resolution: {integrity: sha512-Btj2BOOO83o3WyH59e8MgXsxEQVcarkUOpEYrubB0urwnN10yQ364rsiByU11nZlqWYZm05i/of7io4mzihBtQ==}
engines: {node: '>=8'}
entities@6.0.1:
resolution: {integrity: sha512-aN97NXWF6AWBTahfVOIrB/NShkzi5H7F9r1s9mD3cDj4Ko5f2qhhVoYMibXF7GlLveb/D2ioWay8lxI97Ven3g==}
engines: {node: '>=0.12'}
error-stack-parser-es@1.0.5:
resolution: {integrity: sha512-5qucVt2XcuGMcEGgWI7i+yZpmpByQ8J1lHhcL7PwqCwu9FPP3VUXzT4ltHe5i2z9dePwEHcDVOAfSnHsOlCXRA==}
@@ -722,6 +729,10 @@ packages:
magic-string@0.30.21:
resolution: {integrity: sha512-vd2F4YUyEXKGcLHoq+TEyCjxueSeHnFxyyjNp80yg0XV4vUhnDer/lvvlqM/arB5bXQN5K2/3oinyCRyx8T2CQ==}
microformats-parser@2.0.6:
resolution: {integrity: sha512-VTnLImF17yd+ZfBH40sddHMJp5Rrker4ge+oPjgnBQ3jluQO0YmsNVA4UcEfHRBV8zrikBSMkAzAub+nTRX85g==}
engines: {node: '>=18'}
miniflare@5.20261001.0-alpha:
resolution: {integrity: sha512-GaimS5mSIOMyvd16ga+e1/QkI8cmp3z35QPHFex0DktqNQf2RVZQwMPQXdyoUreUiFP/0Gpe2MoQInTyMAD5xA==}
engines: {node: '>=22.0.0'}
@@ -739,6 +750,9 @@ packages:
resolution: {integrity: sha512-XrsrhT5sybtKI6wakr2SPOlGZWWYbUXZ7a0jT8/QOeAPau+1X/bSegNe5YR75oJmEZQbKningirmGOEJCIk61Q==}
engines: {node: '>=12.20.0'}
parse5@7.3.0:
resolution: {integrity: sha512-IInvU7fabl34qmi9gY8XOVxhYyMyuH2xUNpb2q8/Y+7552KlejkRvqvD19nMoUW/uQGGbqNpA6Tufu5FL5BZgw==}
path-to-regexp@6.3.0:
resolution: {integrity: sha512-Yhpw4T9C6hPpgPeA28us07OJeqZ5EzQTkbfwuhsUg0c237RomFoETJgmp2sa3F/41gfLE6G5cqcYwznmeEeOlQ==}
@@ -1366,6 +1380,8 @@ snapshots:
detect-libc@2.1.2: {}
entities@6.0.1: {}
error-stack-parser-es@1.0.5: {}
es-module-lexer@2.3.2: {}
@@ -1467,6 +1483,10 @@ snapshots:
dependencies:
'@jridgewell/sourcemap-codec': 1.6.0
microformats-parser@2.0.6:
dependencies:
parse5: 7.3.0
miniflare@5.20261001.0-alpha(@types/node@22.20.5):
dependencies:
'@cspotcode/source-map-support': 0.8.1
@@ -1497,6 +1517,10 @@ snapshots:
obug@2.2.1: {}
parse5@7.3.0:
dependencies:
entities: 6.0.1
path-to-regexp@6.3.0: {}
pathe@2.0.3: {}
+11
View File
@@ -19,3 +19,14 @@ where cast(items.updated as integer) >= ?;
-- pagesThrough:
select count(*) as n from pages where page_num <= ?;
-- itemsByUpdated:
select json from items
where cast(updated as integer) >= ? and cast(updated as integer) < ?
order by cast(updated as integer) desc, item_id;
-- itemsByCreated:
select json from items
where cast(json_extract(json, '$.created_at') as integer) >= ?
and cast(json_extract(json, '$.created_at') as integer) < ?
order by cast(json_extract(json, '$.created_at') as integer) desc, item_id;
+33 -1
View File
@@ -6,6 +6,8 @@
import { DurableObject, RpcTarget } from "cloudflare:workers";
import type { PagedGet } from "./diigo.js";
import { publicOnly, toRows } from "./items.js";
import { matches } from "./query.js";
import type { Query } from "./query.js";
import { makeReader, makeWriter, SCHEMA_VERSION } from "./store.js";
import type { Reader, Writer } from "./store.js";
import type { Item, ItemRow } from "./items.js";
@@ -398,7 +400,18 @@ abstract class PagedCache extends DurableObject<CacheEnv> {
}
}
/** what anyone may see of a cache: stored pages and fill progress */
/** one page of a filter's matches over stored items */
export type QueryPage = {
/** the matches on this page, raw dump rows (JSON strings) */
items: string[];
/** matches in all */
matched: number;
/** true when every diigo page is stored, so no match can be missing */
complete: boolean;
};
/** what anyone may see of a cache: stored pages, stored items by filter,
* and fill progress */
export class PageViewer extends RpcTarget {
#reader: Reader;
#fillStatus: () => Promise<FillStatus | undefined>;
@@ -415,6 +428,25 @@ export class PageViewer extends RpcTarget {
return stored && { ...stored, cached: true };
}
/**
* page `pageNum` (`count` per page) of the stored items matching
* `query`, newest first by its date. The window narrows in SQL; terms
* are checked per item. Reads storage only: items not yet stored can't
* match, which `complete` reports.
*/
query(query: Query, pageNum: number, count: number): QueryPage {
const from = query.after ? Math.ceil(query.after.getTime() / 1000) : Number.MIN_SAFE_INTEGER;
const to = query.before ? Math.ceil(query.before.getTime() / 1000) : Number.MAX_SAFE_INTEGER;
const hits = this.#reader.itemsIn(query.by, from, to).filter((json) => matches(JSON.parse(json) as Item, query));
const total = this.#reader.knownTotal();
const pages = total === null ? null : Math.max(1, Math.ceil(total / PAGE_SIZE));
return {
items: hits.slice((pageNum - 1) * count, pageNum * count),
matched: hits.length,
complete: pages !== null && this.#reader.pagesThrough(pages) >= pages,
};
}
/** diigo's total, as last stored, or null if nothing is stored */
knownTotal(): number | null {
return this.#reader.knownTotal();
+188 -51
View File
@@ -10,9 +10,11 @@ import { CAPNWEB_BROWSER_JS } from "./capnweb-browser.js";
import { CACHE_PAGE_HTML } from "./cache-page.js";
import { DiigoCacheLogic, PublicCacheLogic } from "./cache.js";
import { PAGE_SIZE } from "./cache.js";
import type { CacheEnv, Clock, FillStatus } from "./cache.js";
import type { CacheEnv, Clock, FillStatus, QueryPage } from "./cache.js";
import type { Item } from "./cache.js";
import { updatedISO, usernameOf } from "./items.js";
import { usernameOf } from "./items.js";
import { dateOf, isEmpty, parseQuery } from "./query.js";
import type { Query } from "./query.js";
import { toWebkey, idFromWebkey, isWebkey, shashKey } from "./webkeys.js";
import { pmSave } from "./pm-save.js";
@@ -172,11 +174,13 @@ async function proxyMint(request: Request, env: Env): Promise<Response> {
const detail = (await res.text()).slice(0, 200);
return home(env, { error: `mint said ${res.status}: ${detail}` }, null);
}
// webapp answers {id} (+ x-webapp-id); the id rides the header, the
// wire page is ours
// webapp answers {id, storeId} (+ x-webapp-id, x-webapp-store-id);
// the ids ride the headers, the wire page is ours
const id = res.headers.get("x-webapp-id");
if (!id) return home(env, { error: "mint produced no id" }, null);
return await wirePage(id);
const storeId = res.headers.get("x-webapp-store-id");
if (!storeId) console.warn({ event: "mint", outcome: "no store id: this session cannot be rotated" });
return await wirePage(id, storeId);
} catch (err) {
console.error({ event: "mint", error: String(err) });
return home(env, { error: `mint failed: ${String(err)}` }, null);
@@ -186,22 +190,37 @@ async function proxyMint(request: Request, env: Env): Promise<Response> {
/** the credential-save helper, once per page (see ./pm-save.ts) */
const PM_SAVE_JS = pmSave.toString();
/** the suggested petname gets the holder's local hh:mm (the worker only
* knows UTC), so two saves the same day are told apart at a glance */
const PETNAME_TIME_JS = `{
const p = document.getElementById("petname");
/** the suggested petname (in the input with id `petnameId`) gets the
* holder's local hh:mm (the worker only knows UTC), so two saves the same
* day are told apart at a glance */
function petnameTimeJs(petnameId: string): string {
return `{
const p = document.getElementById(${JSON.stringify(petnameId)});
const d = new Date();
p.value += " " + String(d.getHours()).padStart(2, "0") + ":" + String(d.getMinutes()).padStart(2, "0");
}`;
/** the onclick that hands pmSave its browser powers */
function pmSaveClick(value: string): string {
return `pmSave(this, ${JSON.stringify(value)}, { petname: document.getElementById("petname"), credentials: navigator.credentials, PasswordCredential: window.PasswordCredential, log: console })`;
}
/** the wire step, with the freshly-minted webapp id pre-filled */
async function wirePage(webappId: string): Promise<Response> {
/** the onclick that hands pmSave its browser powers, saving under the
* petname in the input with id `petnameId` */
function pmSaveClick(value: string, petnameId: string): string {
return `pmSave(this, ${JSON.stringify(value)}, { petname: document.getElementById(${JSON.stringify(petnameId)}), credentials: navigator.credentials, PasswordCredential: window.PasswordCredential, log: console })`;
}
/** the wire step, with the freshly-minted webapp id pre-filled, and the
* store id (rotating the cookie) for the account owner to keep */
async function wirePage(webappId: string, storeId: string | null): Promise<Response> {
const fingerprint = await nick(webappId);
const storeSection = storeId
? `<h2>keep: the cookie's store id</h2>
<p>This one is yours alone, not the cache's: with it you can rotate the
diigo session (log in again, paste the new cookie) or revoke it. Fingerprint
<code>${esc(await nick(storeId))}</code>
<button type="button" onclick='navigator.clipboard.writeText(${JSON.stringify(storeId)}).then(() => this.textContent = "copied ✓")'>copy</button></p>
<p><label>your petname for it <input id="store-petname" autocomplete="off" size="30" value="${esc(`diigo cookie store ${await nick(storeId)}`)}"></label></p>
<script>${petnameTimeJs("store-petname")}</script>
<p><button type="button" onclick='${pmSaveClick(storeId, "store-petname")}'>save to password manager</button></p>`
: `<p><b>note:</b> webapp sent no store id, so this session's cookie cannot be rotated; when it expires, mint again.</p>`;
const body = `<!doctype html>
<html><head><meta charset="utf-8"><title>diigo cache: step 2 — wire the cache</title></head>
<body>
@@ -210,8 +229,10 @@ async function wirePage(webappId: string): Promise<Response> {
<button type="button" onclick='navigator.clipboard.writeText(${JSON.stringify(webappId)}).then(() => this.textContent = "copied ✓")'>copy</button></p>
<p><label>your petname for it <input id="petname" autocomplete="off" size="30" value="${esc(`diigo webapp ${fingerprint}`)}"></label>
<small>(private to you: the password-manager username; never sent to the server)</small></p>
<script>${PETNAME_TIME_JS}</script>
<p><button type="button" onclick='${pmSaveClick(webappId)}'>save to password manager</button></p>
<script>${petnameTimeJs("petname")}</script>
<p><button type="button" onclick='${pmSaveClick(webappId, "petname")}'>save to password manager</button></p>
${storeSection}
<h2>wire</h2>
<p>choose the cache mode, then <b>wire</b> — the next page shows the
<b>cache id</b> (copy that too), after which you land back on home:</p>
<form method="post" action="/_wire">
@@ -318,8 +339,8 @@ async function wiredPage(cacheId: string, note: string, hasPublic: boolean): Pro
<button type="button" onclick='navigator.clipboard.writeText(${JSON.stringify(cacheId)}).then(() => this.textContent = "copied ✓")'>copy</button></p>
<p><label>your petname for it <input id="petname" autocomplete="off" size="30" value="${esc(`diigo cache ${fingerprint}`)}"></label>
<small>(private to you: the password-manager username; never sent to the server)</small></p>
<script>${PETNAME_TIME_JS}</script>
<p><button type="button" onclick='${pmSaveClick(cacheId)}'>save to password manager</button></p>
<script>${petnameTimeJs("petname")}</script>
<p><button type="button" onclick='${pmSaveClick(cacheId, "petname")}'>save to password manager</button></p>
<p>${note}</p>
<p><a href="/cache/#${esc(cacheId)}">open this cache</a> — its <b>webkey</b>
(bookmark it, or save the id above): fill it, watch progress${hasPublic ? ", fill its public companion" : ""}.</p>
@@ -443,10 +464,16 @@ async function dump(request: Request, env: Env): Promise<Response> {
const PUBLIC_PER_PAGE = PAGE_SIZE;
/** the public page's day presets */
const DAY_PRESETS = [7, 30, 90, 365];
/**
* the public page for a username: stored pages only, through the cache's
* read-only `viewer()` (it can't fetch or write). HTML with a page-number
* form, or JSON for `Accept: application/json`. `?page=` in the URL is
* read-only `viewer()` (it can't fetch or write). HTML with a filter form
* and a page-number form, or JSON for `Accept: application/json`. A
* filter (goal 1d: `days`, `after`, `before`, `in`, `by`, `q`; see
* {@link parseQuery}) lists the stored items that match, `page` counting
* matches; with none, `page` is diigo's page as stored. Query params are
* fine here: nothing on this page is secret.
*/
async function publicPage(request: Request, env: Env, username: string): Promise<Response> {
@@ -456,14 +483,33 @@ async function publicPage(request: Request, env: Env, username: string): Promise
const ns = env.PUBLIC_DO;
const cache = ns.get(ns.idFromString(idFromWebkey(publicId)));
const raw = new URL(request.url).searchParams.get("page");
const params = new URL(request.url).searchParams;
const accept = request.headers.get("accept") ?? "";
const wantsJson = accept.includes("application/json") && !accept.includes("text/html");
const here = `/public/${encodeURIComponent(username)}`;
const raw = params.get("page");
const pageN = raw === null ? 1 : Number.parseInt(raw, 10);
const valid = !Number.isNaN(pageN) && pageN >= 1 && pageN <= 10000;
const query = parseQuery(params, new Date());
if ("error" in query) {
console.log({ event: "notice", page: "public", text: "filter refused", error: query.error });
if (wantsJson) return Response.json({ error: query.error }, { status: 400 });
const body = publicFrame(username, `${filterForm(here, params)}
<p><em>${esc(query.error)}</em></p>`);
return new Response(body, { status: 400, headers: { "content-type": "text/html;charset=utf-8" } });
}
const viewer = await cache.viewer();
let page, total, fill;
let page, fill, found;
let total: number | null = null;
try {
page = await viewer.page(valid ? pageN : 1, PUBLIC_PER_PAGE);
total = page ? page.total : await viewer.knownTotal();
if (isEmpty(query)) {
page = await viewer.page(valid ? pageN : 1, PUBLIC_PER_PAGE);
total = page ? page.total : await viewer.knownTotal();
} else {
found = await viewer.query(query, valid ? pageN : 1, PUBLIC_PER_PAGE);
}
fill = await viewer.fillStatus();
} catch (err) {
console.error({ event: "public-page", page: pageN, error: String(err) });
@@ -471,13 +517,14 @@ async function publicPage(request: Request, env: Env, username: string): Promise
} finally {
viewer[Symbol.dispose]();
}
if (found) return filteredPage(username, here, params, query, found, valid ? pageN : 1, wantsJson, fill);
// total and pages count all of diigo's items: public page n is diigo
// page n with the private rows dropped
const pages = total === null ? 1 : Math.max(1, Math.ceil(total / PUBLIC_PER_PAGE));
const items = page ? page.items.map((j) => JSON.parse(j) as Item) : [];
const accept = request.headers.get("accept") ?? "";
if (accept.includes("application/json") && !accept.includes("text/html")) {
if (wantsJson) {
return Response.json({ username, page: pageN, pages, total, stored: page !== null, etchedAt: page?.etchedAt ?? null, items });
}
@@ -486,42 +533,132 @@ async function publicPage(request: Request, env: Env, username: string): Promise
console.log({ event: "notice", page: "public", text: "page out of range", pages });
notice = `<p><em>no page ${esc(String(raw))}: pages run 1–${pages}</em></p>`;
}
const here = `/public/${encodeURIComponent(username)}`;
const nav = (n: number, label: string) =>
n >= 1 && n <= pages ? `<a href="${here}?page=${n}">${label}</a>` : `<span>${label}</span>`;
const list = notice
? ""
: `<ol>
${items
.map((it) => {
const url = typeof it.url === "string" ? it.url : "";
const title = typeof it.title === "string" && it.title ? it.title : url;
const desc = typeof it.description === "string" && it.description ? `<br><small>${esc(it.description)}</small>` : "";
const href = /^https?:\/\//i.test(url) ? url : "#";
const when = updatedISO(it);
const date = when ? `<time datetime="${when}">${when}</time>` : "(no date)";
return `<li>${date} <a href="${esc(href)}">${esc(title)}</a>${desc}</li>`;
})
.join("\n")}
</ol>`;
const body = `<!doctype html>
<html><head><meta charset="utf-8"><title>${esc(username)}'s shared bookmarks</title></head>
<body>
<h1>${esc(username)}'s shared bookmarks</h1>
<p>${pages} page${pages === 1 ? "" : "s"} (diigo's, ${PUBLIC_PER_PAGE} bookmarks each; private ones are left out, so a page may show fewer).</p>
n >= 1 && n <= pages ? `<a rel="${n < pageN ? "prev" : "next"}" href="${here}?page=${n}">${label}</a>` : `<span>${label}</span>`;
const list = notice ? "" : itemList(items, "updated", here);
const body = publicFrame(
username,
`<p>${pages} page${pages === 1 ? "" : "s"} (diigo's, ${PUBLIC_PER_PAGE} bookmarks each; private ones are left out, so a page may show fewer).</p>
<p>${page ? `<strong>stored ${esc(page.etchedAt ?? "")}</strong> — page ${valid ? pageN : 1}, ${items.length} shared` : `<strong>not fetched yet</strong> — page ${valid ? pageN : 1}; the cache's owner fills it`}</p>
${fill ? fillLine(fill) : ""}
${filterForm(here, params)}
<form method="get" action="${here}">
<p>${nav(pageN - 1, "← prev")}
<label>page <input type="number" name="page" min="1" value="${Number.isNaN(pageN) ? 1 : pageN}"></label>
<button type="submit">go</button>
${nav(pageN + 1, "next →")}</p>
</form>
${notice}${list}
</body></html>`;
${notice}${list}`,
);
return new Response(body, { headers: { "content-type": "text/html;charset=utf-8" } });
}
/** a filter's matches, as the public page's HTML or JSON */
function filteredPage(
username: string,
here: string,
params: URLSearchParams,
query: Query,
found: QueryPage,
pageN: number,
wantsJson: boolean,
fill: FillStatus | undefined,
): Response {
const pages = Math.max(1, Math.ceil(found.matched / PUBLIC_PER_PAGE));
const items = found.items.map((j) => JSON.parse(j) as Item);
if (wantsJson) {
const { matched, complete } = found;
return Response.json({ username, filter: query, page: pageN, pages, matched, complete, items });
}
// links keep the filter: the request's own non-empty params, in order
const kept = [...params].filter(([k, v]) => k !== "page" && v !== "");
const href = (n: number) => `${here}?${new URLSearchParams([...kept, ["page", String(n)]])}`;
const nav = (n: number, label: string) =>
n >= 1 && n <= pages ? `<a rel="${n < pageN ? "prev" : "next"}" href="${esc(href(n))}">${label}</a>` : `<span>${label}</span>`;
const status = found.matched
? `${found.matched} match${found.matched === 1 ? "" : "es"}, page ${pageN} of ${pages}, newest ${query.by} first`
: "no matches";
const caveat = found.complete ? "" : " — not every diigo page is stored yet, so some matches may be missing";
const body = publicFrame(
username,
`<p><strong>${esc(status)}</strong>${caveat}</p>
${fill ? fillLine(fill) : ""}
${filterForm(here, params)}
<p>${nav(pageN - 1, "← prev")} ${nav(pageN + 1, "next →")} <a href="${here}">all, by diigo page</a></p>
${itemList(items, query.by, here)}`,
);
return new Response(body, { headers: { "content-type": "text/html;charset=utf-8" } });
}
/** the public page's frame: title and heading around `content` */
function publicFrame(username: string, content: string): string {
return `<!doctype html>
<html><head><meta charset="utf-8"><title>${esc(username)}'s shared bookmarks</title></head>
<body>
<h1>${esc(username)}'s shared bookmarks</h1>
${content}
</body></html>`;
}
/** the filter form, echoing the request's filter */
function filterForm(here: string, params: URLSearchParams): string {
const field = (name: string) => esc(params.get(name) ?? "");
const days = params.get("days") ?? "";
const presets = DAY_PRESETS.map(String);
if (days && !presets.includes(days)) presets.push(days);
const option = (value: string, label: string, current: string) =>
`<option value="${esc(value)}"${value === current ? " selected" : ""}>${esc(label)}</option>`;
return `<form method="get" action="${here}">
<p><label>last <select name="days">${option("", "any", days)}${presets.map((n) => option(n, `${n} days`, days)).join("")}</select></label>
<label>or after <input type="date" name="after" value="${field("after")}"></label>
<label>before <input type="date" name="before" value="${field("before")}"></label>
<label>or in <input name="in" size="9" placeholder="2026-W41" value="${field("in")}"></label>
<label>by <select name="by">${option("updated", "updated", params.get("by") ?? "")}${option("created", "created", params.get("by") ?? "")}</select></label></p>
<p><label>search <input type="search" name="q" size="40" value="${field("q")}"></label>
<button type="submit">filter</button></p>
<p><small>Dates and periods (2026, 2026-03, 2026-Q1, 2026-W41) are UTC. Search takes
words, "phrases", #tags, -negation, and after:, before:, newer_than:7d, during:2026-Q1, created:2026.</small></p>
</form>`;
}
/**
* bookmarks as a list, each led by its `by` date, marked up as
* microformats: an `h-feed` of `h-entry` bookmarks (`u-bookmark-of`,
* `p-name`, `p-summary`, `p-category` per tag, `dt-published` when
* bookmarked, `dt-updated` when last edited). This markup is the public
* page's API for other sites (tests/caches.test.ts pins it); tag links
* search this page (`here`) by tag.
*
* @see https://microformats.org/wiki/h-entry
* @see https://indieweb.org/bookmark
*/
function itemList(items: Item[], by: Query["by"], here: string): string {
const time = (it: Item, which: Query["by"], hidden: boolean) => {
const secs = dateOf(it, which);
if (secs === undefined) return hidden ? "" : "(no date)";
const when = new Date(secs * 1000).toISOString();
const cls = which === "created" ? "dt-published" : "dt-updated";
return hidden ? `<time class="${cls}" datetime="${when}" hidden></time>` : `<time class="${cls}" datetime="${when}">${when}</time>`;
};
return `<ol class="h-feed">
${items
.map((it) => {
const url = typeof it.url === "string" ? it.url : "";
const title = typeof it.title === "string" && it.title ? it.title : url;
const desc = typeof it.description === "string" && it.description ? `<br><small class="p-summary">${esc(it.description)}</small>` : "";
const href = /^https?:\/\//i.test(url) ? url : "#";
const tags = (typeof it.tags === "string" ? it.tags.split(",") : [])
.map((t) => t.trim())
.filter(Boolean)
.map((t) => ` <a class="p-category" href="${esc(`${here}?q=${encodeURIComponent(`#${t}`)}`)}">${esc(t)}</a>`)
.join("");
const other = by === "created" ? "updated" : "created";
return `<li class="h-entry">${time(it, by, false)} <a class="u-bookmark-of p-name" href="${esc(href)}">${esc(title)}</a>${tags}${desc}${time(it, other, true)}</li>`;
})
.join("\n")}
</ol>`;
}
/** home page: the connect link (webapp's cookie form, diigo-prefilled) +
* the wire step for an already-minted id + the dump form. `home` renders
* the error path too, so it takes env for the connect link. Notices (the
+188 -19
View File
@@ -1,18 +1,10 @@
/**
* @file the public page's filters: date windows and search (goal 1d, **not
* yet implemented**; the design is the skipped specs in tests/query.test.ts).
* @file the public page's filters: date windows and search over a cache's
* stored items (goal 1d; tests/query.test.ts).
*/
import type { Item } from "./items.js";
/**
* diigo fetches one request may spend (cache hits are free); under the
* Workers free plan's 50 external subrequests per invocation
*
* @see https://developers.cloudflare.com/workers/platform/limits/
*/
export const FETCH_BUDGET = 40;
/**
* a filter as devs write it (`{ days: 7, q: "#weekly" }`); URL params map
* onto these fields one-to-one (strings, with "" meaning absent). Dates
@@ -56,6 +48,76 @@ export type Query = {
/** a refused filter: the message is safe to show and log */
export type QueryError = { error: string };
const DAY_MS = 24 * 60 * 60 * 1000;
/** thrown inside {@link toQuery}, returned as a {@link QueryError} */
class Refused extends Error {}
/** `yyyy-mm-dd` or `yyyy/mm/dd` as 00:00Z of that UTC day */
function parseDay(text: string, what: string): Date {
const m = /^(\d{4})([-/])(\d{2})\2(\d{2})$/.exec(text);
const d = m && new Date(Date.UTC(Number(m[1]), Number(m[3]) - 1, Number(m[4])));
if (!m || !d || d.getUTCMonth() !== Number(m[3]) - 1 || d.getUTCDate() !== Number(m[4])) {
throw new Refused(`${what}: expected a date as yyyy-mm-dd`);
}
return d;
}
/** Monday 00:00Z of ISO week 1 of `year` (the week holding January 4) */
function isoWeekOne(year: number): number {
const jan4 = Date.UTC(year, 0, 4);
const weekday = new Date(jan4).getUTCDay() || 7;
return jan4 - (weekday - 1) * DAY_MS;
}
/** a named period as its half-open [after, before) window */
function parsePeriod(text: string, what: string): [Date, Date] {
const refuse = () => new Refused(`${what}: expected a period as 2025, 2025-03, 2025-Q1, or 2025-W01`);
let m;
if ((m = /^(\d{4})$/.exec(text))) {
const y = Number(m[1]);
return [new Date(Date.UTC(y, 0, 1)), new Date(Date.UTC(y + 1, 0, 1))];
}
if ((m = /^(\d{4})-(\d{2})$/.exec(text))) {
const [y, mo] = [Number(m[1]), Number(m[2])];
if (mo < 1 || mo > 12) throw refuse();
return [new Date(Date.UTC(y, mo - 1, 1)), new Date(Date.UTC(y, mo, 1))];
}
if ((m = /^(\d{4})-Q([1-4])$/.exec(text))) {
const [y, q] = [Number(m[1]), Number(m[2])];
return [new Date(Date.UTC(y, 3 * (q - 1), 1)), new Date(Date.UTC(y, 3 * q, 1))];
}
if ((m = /^(\d{4})-W(\d{2})$/.exec(text))) {
const [y, w] = [Number(m[1]), Number(m[2])];
const weeks = (isoWeekOne(y + 1) - isoWeekOne(y)) / (7 * DAY_MS);
if (w < 1 || w > weeks) throw refuse();
const monday = isoWeekOne(y) + (w - 1) * 7 * DAY_MS;
return [new Date(monday), new Date(monday + 7 * DAY_MS)];
}
throw refuse();
}
/** gmail's `newer_than:` amount (`7d`, `2w`, `1m`, `1y`) counted back from
* `now`; months and years are calendar steps */
function parseNewerThan(text: string, now: Date): Date {
const m = /^(\d{1,4})([dwmy])$/.exec(text);
if (!m || Number(m[1]) < 1) throw new Refused("newer_than: expected an amount like 7d, 2w, 1m, or 1y");
const n = Number(m[1]);
const d = new Date(now.getTime());
if (m[2] === "d") d.setTime(now.getTime() - n * DAY_MS);
if (m[2] === "w") d.setTime(now.getTime() - 7 * n * DAY_MS);
if (m[2] === "m") d.setUTCMonth(d.getUTCMonth() - n);
if (m[2] === "y") d.setUTCFullYear(d.getUTCFullYear() - n);
return d;
}
/** `days` as a whole number from 1 to 3650 */
function parseDays(days: number | string): number {
const n = typeof days === "number" ? days : /^\d+$/.test(days) ? Number(days) : NaN;
if (!Number.isInteger(n) || n < 1 || n > 3650) throw new Refused("days: expected a whole number from 1 to 3650");
return n;
}
/**
* resolve a filter: fields and `q` operators combine; a bound given twice
* (by fields, operators, or both) is refused, as is an empty window.
@@ -75,21 +137,128 @@ export type QueryError = { error: string };
* not read ambiently, so tests pin it)
*/
export function toQuery(spec: FilterSpec, now: Date): Query | QueryError {
void spec;
void now;
throw new Error("toQuery: not implemented (goal 1d)");
try {
return resolve(spec, now);
} catch (err) {
if (err instanceof Refused) return { error: err.message };
throw err;
}
}
function resolve(spec: FilterSpec, now: Date): Query {
const words: Term[] = [];
const tags: Term[] = [];
let by: { value: Query["by"]; from: string } | undefined;
let after: { value: Date; from: string } | undefined;
let before: { value: Date; from: string } | undefined;
const setBy = (value: Query["by"], from: string) => {
if (by && by.value !== value) throw new Refused(`${by.from} and ${from} name different dates`);
by = { value, from };
};
const setAfter = (value: Date, from: string) => {
if (after) throw new Refused(`${after.from} and ${from} both set where the window starts`);
after = { value, from };
};
const setBefore = (value: Date, from: string) => {
if (before) throw new Refused(`${before.from} and ${from} both set where the window ends`);
before = { value, from };
};
const setPeriod = ([lo, hi]: [Date, Date], from: string) => {
setAfter(lo, from);
setBefore(hi, from);
};
const present = (v: string | number | undefined) => v !== undefined && v !== "";
if (present(spec.by)) {
if (spec.by !== "updated" && spec.by !== "created") throw new Refused("by: expected updated or created");
setBy(spec.by, `by=${spec.by}`);
}
if (present(spec.days)) setAfter(new Date(now.getTime() - parseDays(spec.days!) * DAY_MS), "days");
if (present(spec.after)) setAfter(parseDay(spec.after!, "after"), "after");
if (present(spec.before)) setBefore(parseDay(spec.before!, "before"), "before");
if (present(spec.in)) setPeriod(parsePeriod(spec.in!, "in"), "in");
for (const m of (spec.q ?? "").matchAll(/(-?)"([^"]*)"|(\S+)/g)) {
if (m[2] !== undefined) {
if (m[2].trim()) words.push({ text: m[2].trim().toLowerCase(), negated: m[1] === "-" });
continue;
}
const token = m[3];
const negated = token.length > 1 && token.startsWith("-");
const body = negated ? token.slice(1) : token;
const op = /^([a-z_]+):(.+)$/i.exec(body);
const name = op?.[1].toLowerCase();
const arg = op?.[2] ?? "";
const windowOp = ["after", "before", "newer_than", "during", "in", "created", "updated"];
if (name && windowOp.includes(name) && negated) throw new Refused(`${name}: can't be negated`);
if (name === "after") setAfter(parseDay(arg, "after:"), "after:");
else if (name === "before") setBefore(parseDay(arg, "before:"), "before:");
else if (name === "newer_than") setAfter(parseNewerThan(arg, now), "newer_than:");
else if (name === "during" || name === "in") setPeriod(parsePeriod(arg, `${name}:`), `${name}:`);
else if (name === "created" || name === "updated") {
setBy(name, `${name}:`);
setPeriod(parsePeriod(arg, `${name}:`), `${name}:`);
} else if (name === "tag") tags.push({ text: arg.toLowerCase(), negated });
else if (body.length > 1 && body.startsWith("#")) tags.push({ text: body.slice(1).toLowerCase(), negated });
else words.push({ text: body.toLowerCase(), negated });
}
if (after && before && after.value.getTime() >= before.value.getTime()) {
throw new Refused(`the window is empty: ${after.from} is not earlier than ${before.from}`);
}
const query: Query = { by: by?.value ?? "updated", words, tags };
if (after) query.after = after.value;
if (before) query.before = before.value;
return query;
}
/** {@link toQuery} over the public page's URL params */
export function parseQuery(params: URLSearchParams, now: Date): Query | QueryError {
void params;
void now;
throw new Error("parseQuery: not implemented (goal 1d)");
const field = (name: string) => params.get(name) ?? "";
const by = field("by");
return toQuery(
{
days: field("days"),
after: field("after"),
before: field("before"),
in: field("in"),
by: by as FilterSpec["by"],
q: field("q"),
},
now,
);
}
/** does the query filter anything? (an empty one lists diigo's pages as stored) */
export function isEmpty(query: Query): boolean {
return !query.after && !query.before && !query.words.length && !query.tags.length && query.by === "updated";
}
/** the item's date that `by` names, in epoch seconds (diigo's unit), or
* undefined when it carries none */
export function dateOf(item: Item, by: Query["by"]): number | undefined {
const secs = by === "created" ? item.created_at : item.updated_at;
return typeof secs === "number" && Number.isFinite(secs) ? secs : undefined;
}
/** does `item` fall in the window and satisfy every term? */
export function matches(item: Item, query: Query): boolean {
void item;
void query;
throw new Error("matches: not implemented (goal 1d)");
if (query.after || query.before) {
const secs = dateOf(item, query.by);
if (secs === undefined) return false;
if (query.after && secs * 1000 < query.after.getTime()) return false;
if (query.before && secs * 1000 >= query.before.getTime()) return false;
}
const text = (v: unknown) => (typeof v === "string" ? v : "");
// fields joined by newline, so a phrase never spans two of them
const haystack = [item.title, item.description, item.url].map(text).join("\n").toLowerCase();
if (!query.words.every((w) => haystack.includes(w.text) !== w.negated)) return false;
const itemTags = new Set(
text(item.tags)
.split(",")
.map((t) => t.trim().toLowerCase())
.filter(Boolean),
);
return query.tags.every((t) => itemTags.has(t.text) !== t.negated);
}
+6
View File
@@ -41,6 +41,12 @@ export function makeReader(sql: Sql) {
deepestPageSince(cutoff: number): number | null {
return run<{ last: number | null }>("deepestPageSince", cutoff)[0]?.last ?? null;
},
/** stored items whose `by` date (epoch seconds) is in [`from`, `to`),
* newest first; the raw dump rows (JSON strings) */
itemsIn(by: "updated" | "created", from: number, to: number): string[] {
const name = by === "created" ? "itemsByCreated" : "itemsByUpdated";
return run<{ json: string }>(name, from, to).map((r) => r.json);
},
/** how many cached pages are numbered `pageNum` or lower */
pagesThrough(pageNum: number): number {
return Number(run<{ n: number }>("pagesThrough", pageNum)[0]?.n ?? 0);
+66 -51
View File
@@ -3,8 +3,8 @@ import { evictDurableObject, runInDurableObject } from "cloudflare:test";
import { describe, expect, it, vi } from "vitest";
import worker from "../src/index.js";
import { idFromWebkey, toWebkey } from "../src/webkeys.js";
import { FETCH_BUDGET } from "../src/query.js";
import { PAGE_SIZE } from "../src/cache.js";
import { mf2 } from "microformats-parser";
// matches tests/fake-webapp.js: 30 items, every third private
const WEBAPP_ID = "fake-webapp-id";
@@ -250,7 +250,7 @@ describe("worker routes", () => {
// PAGE_SIZE 100: page 2 is bookmarks 101..200
expect(html).toContain(">bookmark 130<");
// the date comes first, in toISOString() form
expect(html).toMatch(/<li><time datetime="\d{4}-\d\d-\d\dT[\d:.]+Z">[^<]+<\/time> <a /);
expect(html).toMatch(/<li class="h-entry"><time class="dt-updated" datetime="\d{4}-\d\d-\d\dT[\d:.]+Z">[^<]+<\/time> <a /);
expect(html).not.toContain(">bookmark 129<"); // private (129 % 3 === 0)
const json = (await (
@@ -283,30 +283,34 @@ function isoWeek(d: Date): string {
return `${year}-W${String(week).padStart(2, "0")}`;
}
describe.skip("public page filters (goal 1d)", () => {
describe("public page filters (goal 1d)", () => {
// the fake's dated account: "dated n" updated n-1 days + 1h ago;
// #weekly on every 7th; every 10th private; every 5th created 100
// days before it was updated
// days before it was updated. The public page only reads, so each
// test stores the pages it needs on the owner's side first.
const call = (path: string, init?: RequestInit) =>
worker.fetch(new Request(`https://diigo-cache.test${path}`, init), env as never);
const wireDated = async () => {
/** wire a public cache to `webappId` and store diigo pages 1..`pages` */
const wireStored = async (webappId: string, pages: number) => {
const res = await call("/_wire", {
method: "POST",
headers: { "content-type": "application/json" },
body: JSON.stringify({ webappId: "fake-dated-webapp", mode: "public" }),
body: JSON.stringify({ webappId, mode: "public" }),
});
expect(res.status).toBe(200);
return "/public/dated_dan";
const { username, publicId } = (await res.json()) as { username: string; publicId: string };
const pub = testEnv.PUBLIC_DO.get(testEnv.PUBLIC_DO.idFromString(idFromWebkey(publicId)));
for (let n = 1; n <= pages; n += 1) await pub.page(n, PAGE_SIZE);
return { page: `/public/${username}`, pub };
};
const wireDated = async () => (await wireStored("fake-dated-webapp", 1)).page;
const titlesIn = (html: string) => [...html.matchAll(/>(dated \d+)<\/a>/g)].map((m) => m[1]);
const dated = (...ns: number[]) => ns.map((n) => `dated ${n}`);
it("?days=7 lists the last week, walking only the diigo pages it needs", async () => {
it("?days=7 lists the last week", async () => {
const page = await wireDated();
const before = await testEnv.WEBAPP.calls();
const html = await (await call(`${page}?days=7`)).text();
expect(titlesIn(html)).toEqual(dated(1, 2, 3, 4, 5, 6, 7));
expect((await testEnv.WEBAPP.calls()) - before).toBeLessThanOrEqual(1);
});
it("?by=created windows, lists, and dates by when it was bookmarked", async () => {
@@ -327,13 +331,7 @@ describe.skip("public page filters (goal 1d)", () => {
});
it("?days=300 pages the matches PAGE_SIZE at a time; links keep the filter", async () => {
const res = await call("/_wire", {
method: "POST",
headers: { "content-type": "application/json" },
body: JSON.stringify({ webappId: "fake-deep-webapp", mode: "public" }),
});
expect(res.status).toBe(200);
const page = "/public/deep_dee";
const { page } = await wireStored("fake-deep-webapp", 3);
// deep_dee: one a day; days=300 holds 1..300, minus every 10th (private)
const want = Array.from({ length: 300 }, (_, k) => k + 1).filter((n) => n % 10 !== 0);
const first = await (await call(`${page}?days=300`)).text();
@@ -354,6 +352,15 @@ describe.skip("public page filters (goal 1d)", () => {
expect(titlesIn(html)).toEqual(dated(11, 12));
});
it("?in= names an ISO week", async () => {
const { page } = await wireStored("fake-deep-webapp", 3);
const week = isoWeek(new Date(Date.now() - 100 * 24 * 60 * 60 * 1000));
const html = await (await call(`${page}?in=${week}`)).text();
// one a day; every 10th is private
expect(titlesIn(html).length).toBeGreaterThanOrEqual(6);
expect(titlesIn(html).length).toBeLessThanOrEqual(7);
});
it("?q=#weekly&days=60 filters by tag; private items stay out", async () => {
const page = await wireDated();
const html = await (await call(`${page}?days=60&q=${encodeURIComponent("#weekly")}`)).text();
@@ -366,22 +373,49 @@ describe.skip("public page filters (goal 1d)", () => {
expect(titlesIn(html)).toEqual(dated(7, 14, 21, 28, 35, 42, 49, 56));
});
it("a request spends at most FETCH_BUDGET diigo fetches; incomplete results say so, and a reload continues", async () => {
const res = await call("/_wire", {
method: "POST",
headers: { "content-type": "application/json" },
body: JSON.stringify({ webappId: "fake-deep-webapp", mode: "public" }),
});
expect(res.status).toBe(200);
// deep_dee: 60 diigo pages of 100; a search matching nothing must walk them all
it("a filter reads only stored items, never diigo; results say whether storage is complete", async () => {
const { page, pub } = await wireStored("fake-deep-webapp", 1);
const json = async () =>
(await call(`${page}?days=300`, { headers: { accept: "application/json" } })).json() as Promise<{
matched: number;
complete: boolean;
}>;
const before = await testEnv.WEBAPP.calls();
const first = await call("/public/deep_dee?q=nomatch", { headers: { accept: "application/json" } });
expect(await first.json()).toMatchObject({ complete: false, items: [] });
expect((await testEnv.WEBAPP.calls()) - before).toBeLessThanOrEqual(FETCH_BUDGET + 1);
const html = await (await call("/public/deep_dee?q=nomatch")).text();
expect(html).toMatch(/complete|no matches/);
const again = await call("/public/deep_dee?q=nomatch", { headers: { accept: "application/json" } });
expect(await again.json()).toMatchObject({ complete: true, items: [] });
// page 1 holds 1..100, minus 10 private
expect(await json()).toMatchObject({ matched: 90, complete: false });
expect(await (await call(`${page}?days=300`)).text()).toMatch(/some matches may be missing/);
expect(await testEnv.WEBAPP.calls()).toBe(before);
await pub.page(2, PAGE_SIZE);
await pub.page(3, PAGE_SIZE);
expect(await json()).toMatchObject({ matched: 270, complete: true });
});
it("the page is an h-feed of h-entry bookmarks: microformats are its API for other sites", async () => {
const page = await wireDated();
const html = await (await call(`${page}?days=7&by=created`)).text();
const base = `https://diigo-cache.test${page}`;
const { items, rels } = mf2(html, { baseUrl: base });
const feed = items.find((i) => i.type?.includes("h-feed"));
const entries = feed?.children ?? [];
expect(entries.map((e) => e.properties.name?.[0])).toEqual(dated(1, 2, 3, 4, 6, 7));
const seven = entries[5].properties;
expect(seven["bookmark-of"]).toEqual(["https://dated.example/7"]);
expect(seven.summary).toEqual(["a week of fragments"]);
expect(seven.category).toEqual(["ocap", "weekly"]);
const published = new Date(String(seven.published?.[0])).getTime();
const updated = new Date(String(seven.updated?.[0])).getTime();
// 7 isn't a 5: created when updated, 6 days + 1h ago
expect(published).toBe(updated);
expect(Date.now() - published).toBeGreaterThan(6 * 24 * 60 * 60 * 1000);
// one page of matches: no rel=next
expect(rels.next).toBeUndefined();
});
it("filtered pages link the next page with rel=next, keeping the filter", async () => {
const { page } = await wireStored("fake-deep-webapp", 3);
const html = await (await call(`${page}?days=300`)).text();
const { rels } = mf2(html, { baseUrl: `https://diigo-cache.test${page}?days=300` });
expect(rels.next).toEqual([`https://diigo-cache.test${page}?days=300&page=2`]);
});
it("JSON answers carry the canonical filter, so devs see what was understood", async () => {
@@ -393,25 +427,6 @@ describe.skip("public page filters (goal 1d)", () => {
});
});
it("?in= an old week costs O(log pages) diigo fetches with by=updated, not a walk from page 1", async () => {
const res = await call("/_wire", {
method: "POST",
headers: { "content-type": "application/json" },
body: JSON.stringify({ webappId: "fake-deep-webapp", mode: "public" }),
});
expect(res.status).toBe(200);
// deep_dee: 6000 bookmarks, one a day, 60 diigo pages of 100; a week
// ~14 years back sits near page 51 — a walk from page 1 would cost ~51
const longAgo = new Date(Date.now() - 5100 * 24 * 60 * 60 * 1000);
const week = isoWeek(longAgo);
const before = await testEnv.WEBAPP.calls();
const html = await (await call(`/public/deep_dee?in=${week}`)).text();
// one a day; every 10th is private
expect(titlesIn(html).length).toBeGreaterThanOrEqual(6);
expect(titlesIn(html).length).toBeLessThanOrEqual(7);
expect((await testEnv.WEBAPP.calls()) - before).toBeLessThanOrEqual(12);
});
it("a malformed filter is a 400 with a message, not an empty list", async () => {
const page = await wireDated();
const res = await call(`${page}?days=7x`);
+2 -2
View File
@@ -18,9 +18,9 @@ const OVERSTATEMENT = 60;
* 5th an old bookmark edited recently (created 100 days before updated) */
const DATED_WEBAPP = "fake-dated-webapp";
const DATED_TOTAL = 60;
/** the same shape, 6000 deep (~16 years, 60 pages of 100): for walk cost */
/** the same shape, 300 deep (3 pages of 100): for paging matches */
const DEEP_WEBAPP = "fake-deep-webapp";
const DEEP_TOTAL = 6000;
const DEEP_TOTAL = 300;
const HOUR = 60 * 60 * 1000;
function datedItem(n, now, user = "dated_dan") {
+5 -5
View File
@@ -2,7 +2,7 @@ import { describe, expect, it } from "vitest";
import { matches, parseQuery, toQuery } from "../src/query.js";
import type { FilterSpec, Query, QueryError } from "../src/query.js";
// goal 1d (capsite checklist): the design as tests; unskip as it lands
// goal 1d (capsite checklist)
const NOW = new Date("2026-10-07T12:00:00.000Z");
const DAY = 24 * 60 * 60 * 1000;
@@ -21,7 +21,7 @@ const refused = (q: Query | QueryError) => {
const day = (iso: string) => new Date(`${iso}T00:00:00.000Z`);
const range = (q: Query) => [q.after?.toISOString().slice(0, 10), q.before?.toISOString().slice(0, 10)];
describe.skip("filters: one spec, two spellings (goal 1d)", () => {
describe("filters: one spec, two spellings (goal 1d)", () => {
it("devs write { days: 7 }; URLs write ?days=7; same query", () => {
expect(spec({ days: 7 })).toEqual(url("days=7"));
expect(spec({ days: 7 }).after).toEqual(new Date(NOW.getTime() - 7 * DAY));
@@ -42,7 +42,7 @@ describe.skip("filters: one spec, two spellings (goal 1d)", () => {
});
});
describe.skip("windows: UTC, half-open (goal 1d)", () => {
describe("windows: UTC, half-open (goal 1d)", () => {
it("after/before are UTC days: after inclusive from 00:00Z, before exclusive", () => {
expect(range(url("after=2026-01-01&before=2026-02-01"))).toEqual(["2026-01-01", "2026-02-01"]);
expect(url("after=2026/01/01").after).toEqual(day("2026-01-01"));
@@ -88,7 +88,7 @@ describe.skip("windows: UTC, half-open (goal 1d)", () => {
});
});
describe.skip("q: terms and operators (goal 1d)", () => {
describe("q: terms and operators (goal 1d)", () => {
const q = (text: string) => url(`q=${encodeURIComponent(text)}`);
it("words, phrases, #tags, tag:, and - negation; case-insensitive", () => {
@@ -160,7 +160,7 @@ const item = (iso: string, extra: Record<string, unknown> = {}) => ({
});
const query = (q: Partial<Query>): Query => ({ by: "updated", words: [], tags: [], ...q });
describe.skip("matches (goal 1d)", () => {
describe("matches (goal 1d)", () => {
it("the window is half-open: after inclusive, before exclusive", () => {
const q = query({ after: day("2026-01-01"), before: day("2026-02-01") });
expect(matches(item("2026-01-01T00:00:00.000Z"), q)).toBe(true);
+1
View File
@@ -0,0 +1 @@
node_modules
+21
View File
@@ -0,0 +1,21 @@
# diigo-week
An EmDash plugin that drafts a potluck issue from a week of your shared diigo bookmarks. It reads them as microformats from diigo-cache's public page, so you edit a draft instead of copy-pasting links.
## Getting started
Run diigo-cache (with a filled public cache) and potluck, each in its own terminal:
```sh
pnpm install
(cd ../diigo-cache && pnpm dev) # → http://localhost:8787/
(cd ../potluck && pnpm dev) # → http://localhost:4321/_emdash/admin
```
On potluck's admin dashboard, the **This week's issue** card has a **Draft 2026-Wnn from diigo** button for last week; it links to the draft once made. For any other week: Plugins → **Draft from diigo**. A week that already has an issue isn't drafted again.
## How it reads the week
It asks diigo-cache for `/public/<user>?in=<week>&by=created` over the `DIIGO_CACHE` service binding (potluck's `wrangler.jsonc`), reads the page's `h-feed` of `h-entry` bookmarks, follows `rel=next`, and creates a draft in `issues`: title, week, dek, and a paragraph per bookmark with its description and tag links, as in potluck's `seed/issues.json`. The binding is its only reach outside the site; it declares `content:write` and `taxonomies:write` (the issue gets its bookmarks' tags, as the sample issues do), nothing else.
Checks: `pnpm lint && pnpm test`.
+24
View File
@@ -0,0 +1,24 @@
{
"name": "diigo-week",
"version": "0.1.0",
"private": true,
"type": "module",
"exports": {
".": {
"import": "./src/index.ts"
}
},
"scripts": {
"lint": "tsc --noEmit",
"test": "vitest run"
},
"dependencies": {
"emdash": "1.2.0",
"microformats-parser": "^2.0.6"
},
"devDependencies": {
"@emdash-cms/blocks": "^1.2.0",
"typescript": "^7.0.2",
"vitest": "^5.0.3"
}
}
+6223
View File
File diff suppressed because it is too large. Load diff
+8
View File
@@ -0,0 +1,8 @@
packages:
- "."
strictDepBuilds: true
dangerouslyAllowAllBuilds: false
allowBuilds:
esbuild: true
workerd: true
+7
View File
@@ -0,0 +1,7 @@
/** emdash's virtual module for the site worker's bindings (declared for site
* builds in emdash/src/virtual-modules.d.ts), so this package typechecks
* standalone: `env` from `cloudflare:workers` under @astrojs/cloudflare,
* undefined on other adapters */
declare module "virtual:emdash/env" {
export const env: Record<string, unknown> | undefined;
}
+180
View File
@@ -0,0 +1,180 @@
/**
* @file diigo-week: an EmDash plugin that drafts a potluck issue from a
* week of public diigo bookmarks.
*/
import { definePlugin } from "emdash";
import type { PluginDefinition, RouteContext } from "emdash";
import type { Block, BlockResponse } from "@emdash-cms/blocks";
import { draftIssue, issueTags, lastWeek, readWeek } from "./week.js";
/** the diigo username drafts read from, until the admin page saves another */
const DEFAULT_USER = "dckc-madmode";
/** something that answers HTTP requests: a service binding's `fetch` */
type Fetcher = { fetch(request: Request): Promise<Response> };
/**
* the `DIIGO_CACHE` service binding to diigo-cache, from the site worker's
* env. The binding is the plugin's only reach outside the site:
* `ctx.http.fetch` can't reach a dev server (it refuses localhost), and a
* binding names one worker rather than a host.
*/
async function diigoCache(): Promise<Fetcher> {
const { env } = await import("virtual:emdash/env");
const binding = env?.DIIGO_CACHE as Fetcher | undefined;
if (typeof binding?.fetch !== "function") {
throw new Error('no DIIGO_CACHE service binding: add { "binding": "DIIGO_CACHE", "service": "diigo-cache" } to services in wrangler.jsonc');
}
return binding;
}
/** GET a diigo-cache path as HTML; anything but a 200 is an error */
async function getter(cache: Fetcher): Promise<(path: string) => Promise<string>> {
return async (path) => {
const res = await cache.fetch(new Request(`https://diigo-cache${path}`, { headers: { accept: "text/html" } }));
const text = await res.text();
if (res.status !== 200) throw new Error(`diigo-cache ${path.split("?")[0]}: ${res.status} ${text.slice(0, 200)}`);
return text;
};
}
/**
* assign `tags` to an issue in the `tag` taxonomy (as seed/issues.json
* does), creating terms that don't exist yet; returns the tags that
* couldn't be assigned, with why
*/
async function tagIssue(ctx: RouteContext, id: string, tags: string[]): Promise<string[]> {
const tax = ctx.taxonomies;
if (typeof tax?.createTerm !== "function" || typeof tax.addEntryTerms !== "function") throw new Error("this plugin lacks taxonomies:write");
const known = new Map((await tax.getTerms("tag")).map((t) => [t.slug, t.id]));
const ids: string[] = [];
const failed: string[] = [];
for (const tag of tags) {
try {
ids.push(known.get(tag) ?? (await tax.createTerm("tag", { label: tag, slug: tag })).id);
} catch (error) {
failed.push(`${tag} (${error instanceof Error ? error.message : String(error)})`);
}
}
if (ids.length) await tax.addEntryTerms("issues", id, "tag", ids);
return failed;
}
/** the issue already drafted or published for `week`, if any (issues
* are few: a scan of the newest is enough) */
async function issueFor(ctx: RouteContext, week: string): Promise<{ id: string; status: string; title: string } | undefined> {
const found = (await ctx.content!.list("issues", { limit: 100 })).items.find((it) => it.data.week === week);
return found && { id: found.id, status: found.status, title: String(found.data.title ?? week) };
}
/** a button-row link to an issue in the editor */
const openIssue = (id: string, label: string): Block => ({
type: "actions",
elements: [{ type: "link", label, target: { kind: "content", collection: "issues", id }, appearance: "primary" }],
});
/** draft `week` from `user`'s bookmarks, unless the week has an issue
* already: blocks that say what happened (with a link), and a toast */
async function draft(ctx: RouteContext, user: string, week: string): Promise<{ note: Block[]; toast: BlockResponse["toast"] }> {
try {
if (!/^[A-Za-z0-9_-]{1,64}$/.test(user)) throw new Error("diigo username: letters, digits, _ and - only");
const existing = await issueFor(ctx, week);
if (existing) {
return {
note: [
{ type: "banner", variant: "default", title: `${week} already has an issue`, description: `"${existing.title}" (${existing.status}). Delete it to draft the week again.` },
openIssue(existing.id, "Open it"),
],
toast: { message: `${week} already has an issue`, type: "info" },
};
}
const bookmarks = await readWeek(await getter(await diigoCache()), user, week);
const issue = draftIssue(week, bookmarks);
const content = ctx.content;
if (typeof content?.create !== "function") throw new Error("this plugin lacks content:write");
const created = await content.create("issues", issue);
const untagged = await tagIssue(ctx, created.id, issueTags(bookmarks));
if (untagged.length) ctx.log.warn("tags not assigned", { id: created.id, untagged });
ctx.log.info("drafted issue", { week, bookmarks: bookmarks.length, id: created.id });
return {
note: [
{ type: "banner", variant: "default", title: `Drafted "${issue.title}"`, description: `${bookmarks.length} bookmarks, as a draft at /issues/${created.slug ?? "(no slug)"}. Cut, comment, and publish.${untagged.length ? ` Tags not assigned: ${untagged.join(", ")}.` : ""}` },
openIssue(created.id, "Open the draft"),
],
toast: { message: `Drafted ${issue.title}`, type: "success" },
};
} catch (error) {
const message = error instanceof Error ? error.message : String(error);
ctx.log.error("draft failed", { week, message });
return { note: [{ type: "banner", variant: "alert", title: "Couldn't draft the issue", description: message }], toast: { message, type: "error" } };
}
}
/** the admin page: a form for the diigo username and the week */
async function page(ctx: RouteContext, note: Block[] = [], toast?: BlockResponse["toast"]): Promise<BlockResponse> {
const user = (await ctx.kv.get<string>("user")) ?? DEFAULT_USER;
const blocks: Block[] = [
{ type: "header", text: "Draft an issue from diigo" },
{
type: "context",
text: "Reads a week of shared bookmarks from diigo-cache (by when they were bookmarked) and saves them as a draft issue for you to edit.",
},
{
type: "form",
block_id: "draft",
fields: [
{ type: "text_input", action_id: "user", label: "diigo username", initial_value: user },
{ type: "text_input", action_id: "week", label: "ISO week (UTC)", initial_value: lastWeek(new Date()) },
],
submit: { label: "Draft issue", action_id: "draft" },
},
];
blocks.push(...note);
return { blocks, toast };
}
/** the dashboard widget: last week's issue if there is one, else a button to draft it */
async function widget(ctx: RouteContext, note: Block[] = [], toast?: BlockResponse["toast"]): Promise<BlockResponse> {
const week = lastWeek(new Date());
if (note.length) return { blocks: note, toast };
const existing = await issueFor(ctx, week);
const blocks: Block[] = existing
? [{ type: "section", text: `${week}: "${existing.title}" (${existing.status})` }, openIssue(existing.id, "Open it")]
: [
{ type: "section", text: `${week} has no issue yet.` },
{ type: "actions", elements: [{ type: "button", action_id: "draft-last-week", label: `Draft ${week} from diigo`, style: "primary" }] },
];
return { blocks, toast };
}
export function createPlugin(): PluginDefinition {
return definePlugin({
id: "diigo-week",
version: "0.1.0",
capabilities: ["content:write", "taxonomies:write"],
admin: {
pages: [{ path: "/draft", label: "Draft from diigo", icon: "bookmark" }],
widgets: [{ id: "last-week", title: "This week's issue", size: "half" }],
},
routes: {
admin: {
permission: "plugins:manage",
handler: async (ctx) => {
const input = ctx.input as { type?: string; page?: string; action_id?: string; values?: Record<string, unknown> } | undefined;
const user = (await ctx.kv.get<string>("user")) ?? DEFAULT_USER;
if (input?.type === "block_action" && input.action_id === "draft-last-week") {
const { note, toast } = await draft(ctx, user, lastWeek(new Date()));
return widget(ctx, note, toast);
}
if (input?.page?.startsWith("widget:")) return widget(ctx);
if (input?.type !== "form_submit" || input.action_id !== "draft") return page(ctx);
const formUser = String(input.values?.user ?? "").trim();
if (/^[A-Za-z0-9_-]{1,64}$/.test(formUser)) await ctx.kv.set("user", formUser);
const { note, toast } = await draft(ctx, formUser, String(input.values?.week ?? "").trim());
return page(ctx, note, toast);
},
},
},
});
}
+146
View File
@@ -0,0 +1,146 @@
/**
* @file a week of bookmarks, read as microformats from diigo-cache's public
* page, drafted as a potluck issue.
*/
import { mf2 } from "microformats-parser";
const DAY_MS = 24 * 60 * 60 * 1000;
/** pages of matches one draft will read (100 bookmarks each) */
export const MAX_PAGES = 10;
/** one bookmark, as its h-entry says */
export type Bookmark = {
url: string;
title: string;
description: string;
tags: string[];
/** when it was bookmarked, ISO */
published: string;
};
/** `yyyy-Www` for the ISO 8601 week holding `d` (UTC) */
export function isoWeek(d: Date): string {
const t = new Date(Date.UTC(d.getUTCFullYear(), d.getUTCMonth(), d.getUTCDate()));
const day = t.getUTCDay() || 7;
t.setUTCDate(t.getUTCDate() + 4 - day); // the week's Thursday names its year
const year = t.getUTCFullYear();
const week = Math.ceil(((t.getTime() - Date.UTC(year, 0, 1)) / DAY_MS + 1) / 7);
return `${year}-W${String(week).padStart(2, "0")}`;
}
/** the last full ISO week before `now` */
export function lastWeek(now: Date): string {
return isoWeek(new Date(now.getTime() - 7 * DAY_MS));
}
/** Monday 00:00Z of ISO week `week` (`2026-W41`), or undefined if malformed */
export function weekStart(week: string): Date | undefined {
const m = /^(\d{4})-W(\d{2})$/.exec(week);
if (!m) return undefined;
const [year, w] = [Number(m[1]), Number(m[2])];
const jan4 = Date.UTC(year, 0, 4);
const monday = jan4 - ((new Date(jan4).getUTCDay() || 7) - 1) * DAY_MS + (w - 1) * 7 * DAY_MS;
const start = new Date(monday);
return w >= 1 && isoWeek(start) === week ? start : undefined;
}
/** the first string value of an mf2 property, or "" */
const first = (props: Record<string, unknown[]>, name: string): string => {
const v = props[name]?.[0];
return typeof v === "string" ? v : "";
};
/** the bookmarks in a page's `h-feed`, and its `rel=next` link if any */
export function readPage(html: string, url: string): { bookmarks: Bookmark[]; next?: string } {
const { items, rels } = mf2(html, { baseUrl: url });
const feed = items.find((i) => i.type?.includes("h-feed"));
const bookmarks = (feed?.children ?? [])
.filter((e) => e.type?.includes("h-entry"))
.map((e) => {
const props = e.properties as Record<string, unknown[]>;
const tags = (props.category ?? []).filter((t): t is string => typeof t === "string");
return {
url: first(props, "bookmark-of"),
title: first(props, "name"),
description: first(props, "summary"),
tags,
published: first(props, "published"),
};
})
.filter((b) => b.url !== "");
return { bookmarks, next: rels.next?.[0] };
}
/**
* a user's bookmarks from an ISO week, oldest first, by when they were
* bookmarked: diigo-cache's `/public/<user>?in=<week>&by=created`,
* following `rel=next` up to {@link MAX_PAGES} pages.
*
* @param get fetches a path on diigo-cache, answering its HTML; it throws
* on anything but a 200
*/
export async function readWeek(get: (path: string) => Promise<string>, user: string, week: string): Promise<Bookmark[]> {
const base = "https://diigo-cache.invalid";
let path: string | undefined = `/public/${encodeURIComponent(user)}?in=${encodeURIComponent(week)}&by=created`;
const bookmarks: Bookmark[] = [];
for (let n = 0; path && n < MAX_PAGES; n += 1) {
const page = readPage(await get(path), base + path);
bookmarks.push(...page.bookmarks);
const next: URL | undefined = page.next ? new URL(page.next) : undefined;
path = next && next.origin === base ? next.pathname + next.search : undefined;
}
return bookmarks.reverse();
}
/** a Portable Text span */
type Span = { _type: "span"; _key: string; text: string; marks: string[] };
/** a Portable Text block */
type Block = { _type: "block"; _key: string; style: string; markDefs: { _type: "link"; _key: string; href: string }[]; children: Span[] };
/** a bookmark as one paragraph: the link, the description, then tag links (as in seed/issues.json) */
function bookmarkBlock(b: Bookmark, i: number): Block {
const k = `b${i}`;
const markDefs: Block["markDefs"] = [{ _type: "link", _key: `${k}-a`, href: b.url }];
const children: Span[] = [{ _type: "span", _key: `${k}-t`, text: b.title || b.url, marks: [`${k}-a`] }];
if (b.description) children.push({ _type: "span", _key: `${k}-d`, text: `: ${b.description}`, marks: [] });
b.tags.forEach((tag, j) => {
markDefs.push({ _type: "link", _key: `${k}-g${j}`, href: `/tag/${encodeURIComponent(tag)}` });
children.push({ _type: "span", _key: `${k}-s${j}`, text: " ", marks: [] });
children.push({ _type: "span", _key: `${k}-n${j}`, text: tag, marks: [`${k}-g${j}`] });
});
return { _type: "block", _key: k, style: "normal", markDefs, children };
}
/** the issue's tags: every bookmark's, deduplicated, in order of first use */
export function issueTags(bookmarks: Bookmark[]): string[] {
return [...new Set(bookmarks.flatMap((b) => b.tags))];
}
/** `Oct 5–11` (or `Sep 28–Oct 4`) for the week starting `monday` */
function range(monday: Date): string {
const sunday = new Date(monday.getTime() + 6 * DAY_MS);
const fmt = (d: Date, withMonth: boolean) =>
withMonth ? d.toLocaleDateString("en-US", { month: "short", day: "numeric", timeZone: "UTC" }) : String(d.getUTCDate());
return `${fmt(monday, true)}–${fmt(sunday, sunday.getUTCMonth() !== monday.getUTCMonth())}`;
}
/** the fields of a draft `issues` entry for `week`: title, week, dek, body */
export function draftIssue(week: string, bookmarks: Bookmark[]): Record<string, unknown> {
const monday = weekStart(week);
if (!monday) throw new Error(`week: expected an ISO week like 2026-W41, got ${JSON.stringify(week)}`);
const intro: Block = {
_type: "block",
_key: "intro",
style: "normal",
markDefs: [],
children: [{ _type: "span", _key: "intro-t", text: "(draft) A line or two about the week goes here.", marks: [] }],
};
return {
title: `Week ${week}`,
week,
dek: `${range(monday)}: ${bookmarks.length} bookmark${bookmarks.length === 1 ? "" : "s"}. (draft)`,
body: [intro, ...bookmarks.map(bookmarkBlock)],
};
}
+99
View File
@@ -0,0 +1,99 @@
import { describe, expect, it } from "vitest";
import { draftIssue, issueTags, lastWeek, readWeek, weekStart } from "../src/week.js";
/** an h-entry as diigo-cache's public page marks it up (by=created) */
const entry = (n: number, created: string, extra = "") =>
`<li class="h-entry"><time class="dt-published" datetime="${created}">${created}</time> <a class="u-bookmark-of p-name" href="https://ex.example/${n}">bookmark ${n}</a>${extra}<time class="dt-updated" datetime="${created}" hidden></time></li>`;
const page = (entries: string[], next?: string) => `<!doctype html><html><body>
<p>${next ? `<a rel="next" href="${next}">next →</a>` : "<span>next →</span>"}</p>
<ol class="h-feed">
${entries.join("\n")}
</ol></body></html>`;
describe("readWeek", () => {
it("reads h-entry bookmarks across rel=next pages, oldest first", async () => {
const asked: string[] = [];
const pages: Record<string, string> = {
"/public/dckc-madmode?in=2026-W40&by=created": page(
[
entry(
3,
"2026-10-03T12:00:00.000Z",
` <a class="p-category" href="/public/dckc-madmode?q=%23ocap">ocap</a> <a class="p-category" href="/public/dckc-madmode?q=%23weekly">weekly</a><br><small class="p-summary">a week of fragments</small>`,
),
entry(2, "2026-10-01T12:00:00.000Z"),
],
"/public/dckc-madmode?in=2026-W40&by=created&page=2",
),
"/public/dckc-madmode?in=2026-W40&by=created&page=2": page([entry(1, "2026-09-28T12:00:00.000Z")]),
};
const get = async (path: string) => {
asked.push(path);
const html = pages[path];
if (!html) throw new Error(`no page ${path}`);
return html;
};
const bookmarks = await readWeek(get, "dckc-madmode", "2026-W40");
expect(asked).toHaveLength(2);
expect(bookmarks.map((b) => b.title)).toEqual(["bookmark 1", "bookmark 2", "bookmark 3"]);
expect(bookmarks[2]).toEqual({
url: "https://ex.example/3",
title: "bookmark 3",
description: "a week of fragments",
tags: ["ocap", "weekly"],
published: "2026-10-03T12:00:00.000Z",
});
});
it("doesn't follow a rel=next to another site", async () => {
const asked: string[] = [];
const get = async (path: string) => {
asked.push(path);
return page([entry(1, "2026-10-01T12:00:00.000Z")], "https://elsewhere.example/public/x");
};
await readWeek(get, "dckc-madmode", "2026-W40");
expect(asked).toHaveLength(1);
});
});
describe("weeks", () => {
it("lastWeek is the full ISO week before now", () => {
expect(lastWeek(new Date("2026-10-09T01:00:00Z"))).toBe("2026-W40");
// Monday 2026-01-05 starts W02; W01 began Monday 2025-12-29
expect(lastWeek(new Date("2026-01-05T00:00:00Z"))).toBe("2026-W01");
});
it("weekStart is Monday 00:00Z, and refuses a week that doesn't exist", () => {
expect(weekStart("2026-W40")?.toISOString()).toBe("2026-09-28T00:00:00.000Z");
expect(weekStart("2025-W01")?.toISOString()).toBe("2024-12-30T00:00:00.000Z");
expect(weekStart("2025-W53")).toBeUndefined();
expect(weekStart("2026-40")).toBeUndefined();
});
});
describe("draftIssue", () => {
it("drafts title, week, dek, and a body: an intro, then a paragraph per bookmark with tag links", () => {
const issue = draftIssue("2026-W40", [
{ url: "https://ex.example/1", title: "one", description: "nice", tags: ["ocap"], published: "2026-09-28T12:00:00.000Z" },
]);
expect(issue.title).toBe("Week 2026-W40");
expect(issue.week).toBe("2026-W40");
expect(issue.dek).toBe("Sep 28–Oct 4: 1 bookmark. (draft)");
const [intro, b] = issue.body as { markDefs: { href: string }[]; children: { text: string }[] }[];
expect(intro.children[0].text).toMatch(/draft/);
expect(b.markDefs.map((m) => m.href)).toEqual(["https://ex.example/1", "/tag/ocap"]);
expect(b.children.map((c) => c.text).join("")).toBe("one: nice ocap");
});
it("refuses a malformed week", () => {
expect(() => draftIssue("last week", [])).toThrow(/ISO week/);
});
});
describe("issueTags", () => {
it("is every bookmark's tags, once each, in order of first use", () => {
const b = (tags: string[]) => ({ url: "https://ex.example/", title: "", description: "", tags, published: "" });
expect(issueTags([b(["governance", "economics"]), b([]), b(["governance", "scaling"])])).toEqual(["governance", "economics", "scaling"]);
});
});
+20
View File
@@ -0,0 +1,20 @@
{
"compilerOptions": {
"target": "ES2022",
"module": "preserve",
"moduleResolution": "bundler",
"strict": true,
"esModuleInterop": true,
"verbatimModuleSyntax": true,
"skipLibCheck": true,
"types": []
},
"include": [
"src/**/*",
"tests/**/*",
"vitest.config.ts"
],
"exclude": [
"node_modules"
]
}
+146
View File
@@ -0,0 +1,146 @@
<!doctype html>
<html><head><meta charset="utf-8">
<title>webapp cookie rotation: who holds what</title>
<!-- drawn after the style of MarkM's ERTP slides; cookie-rotation-wiring.png is a headless-Chromium screenshot of this page -->
<style>
html, body { margin: 0; background: #fff; }
.sans { font-family: "Helvetica Neue", "DejaVu Sans", sans-serif; }
.obj { font-family: "DejaVu Serif", serif; font-style: italic; }
.mono { font-family: "DejaVu Sans Mono", monospace; }
</style></head>
<body>
<svg id="c" xmlns="http://www.w3.org/2000/svg" width="1120" height="820" viewBox="0 0 1120 820"
role="img" aria-labelledby="title desc">
<title id="title">webapp cookie rotation: who holds what</title>
<desc id="desc">Three actors and two makers. Alice holds a reference to a store made by
makeMapStore, and to diigo-cache, which she wired up. She rotates the cookie
by saying: await E(store).set("cookie", freshCookie). The store, and its
read-only facets, both refer to the store's kv table, whose one row maps
"cookie" to the session; its shapes are keyShape "cookie" and valueShape
M.string(). makeWebApp makes access objects; each refers to its init
table: url, headers, and headerStore. headerStore refers to the store's
read-only facet, which access reads at every send. diigo-cache holds a
reference to access. Mallory, who learned the instance id, also holds a
reference to access, so she can use the cookie, but she has no reference
to the store, so she cannot rotate it or swap in her own session.</desc>
<defs>
<radialGradient id="ball" cx="40%" cy="35%" r="70%">
<stop offset="0%" stop-color="#7cc8f8"/>
<stop offset="100%" stop-color="#3aa6ee"/>
</radialGradient>
<linearGradient id="objfill" x1="0" y1="0" x2="0" y2="1">
<stop offset="0%" stop-color="#6cbcf4"/>
<stop offset="100%" stop-color="#4aa8ec"/>
</linearGradient>
<marker id="head" viewBox="0 0 10 10" refX="9" refY="5" markerWidth="9" markerHeight="9" orient="auto-start-reverse">
<path d="M0,0 L10,5 L0,10 z" fill="#1d4f7a"/>
</marker>
</defs>
</svg>
<script>
const svg = document.getElementById("c");
const NS = "http://www.w3.org/2000/svg";
const el = (name, attrs, parent = svg) => {
const e = document.createElementNS(NS, name);
for (const [k, v] of Object.entries(attrs)) e.setAttribute(k, v);
parent.appendChild(e);
return e;
};
const text = (x, y, s, cls, attrs = {}) => {
const t = el("text", { x, y, class: cls, "text-anchor": "middle", fill: "#1d3550", ...attrs });
t.textContent = s;
return t;
};
const INK = "#1d4f7a";
const dot = (x, y) => el("circle", { cx: x, cy: y, r: 4.5, fill: INK });
/** a reference: dot at the holder, arrowhead at the target */
const ref = (x1, y1, x2, y2, bend = 0, extra = {}) => {
dot(x1, y1);
const mx = (x1 + x2) / 2 - (y2 - y1) * bend, my = (y1 + y2) / 2 + (x2 - x1) * bend;
el("path", { d: `M${x1},${y1} Q${mx},${my} ${x2},${y2}`, fill: "none", stroke: INK, "stroke-width": 2, "marker-end": "url(#head)", ...extra });
};
// maker box: a pale rounded region, with stacked instances inside
function maker(x, y, w, h, label) {
el("rect", { x, y, width: w, height: h, rx: 26, fill: "#d6ecfb", stroke: "#b5dbf5", "stroke-width": 1.5 });
text(x + 18, y + 30, label, "obj", { "text-anchor": "start", "font-size": 20 });
for (let i = 2; i >= 0; i -= 1) {
el("rect", {
x: x + 18 + i * 9, y: y + 48 + i * 9, width: w - 52, height: h - 82, rx: 22,
fill: "#c8e5fa", stroke: "#8ec9f2", "stroke-width": 1.5, "fill-opacity": 0.9,
});
}
}
// an object: rounded square, optionally stacked (several made)
function obj(x, y, w, h, label, stacked = false) {
const n = stacked ? 3 : 1;
for (let i = n - 1; i >= 0; i -= 1) {
el("rect", { x: x + i * 7, y: y - i * 7, width: w, height: h, rx: 12, fill: "url(#objfill)", stroke: "#2f8fd6", "stroke-width": 1.5 });
}
text(x + w / 2, y + h / 2 + 7, label, "obj", { "font-size": 19 });
}
function actor(cx, cy, label) {
el("circle", { cx, cy, r: 52, fill: "url(#ball)", stroke: "#2f8fd6", "stroke-width": 1.5 });
text(cx, cy + 6, label, "sans", { "font-size": label.length > 8 ? 15 : 17 });
}
// a table like the ERTP ledger
function table(x, y, title, rows, colW) {
const rowH = 28, w = colW[0] + colW[1];
el("rect", { x, y, width: w, height: 36 + rowH * rows.length, fill: "#5fb5f2", stroke: "#2f8fd6", "stroke-width": 1.5 });
text(x + w / 2, y + 25, title, "obj", { "font-size": 18 });
rows.forEach(([k, v], i) => {
const ry = y + 36 + i * rowH;
el("line", { x1: x, y1: ry, x2: x + w, y2: ry, stroke: "#2f8fd6", "stroke-width": 1.2 });
text(x + 10, ry + 19, k, "mono", { "text-anchor": "start", "font-size": 14 });
if (v) text(x + colW[0] + 10, ry + 19, v, "mono", { "text-anchor": "start", "font-size": 14 });
});
el("line", { x1: x + colW[0], y1: y + 36, x2: x + colW[0], y2: y + 36 + rowH * rows.length, stroke: "#1d4f7a", "stroke-width": 1.5 });
return { rowY: (i) => y + 36 + i * rowH + rowH / 2, x, w, colX: x + colW[0] };
}
// speech bubble (Alice rotates after logging in again)
el("path", {
d: "M30,22 h470 a16,16 0 0 1 16,16 v34 a16,16 0 0 1 -16,16 h-330 l-38,48 l6,-48 h-102 a16,16 0 0 1 -16,-16 v-34 a16,16 0 0 1 16,-16 z",
fill: "#d9d9d9", stroke: "#555", "stroke-width": 2,
});
const code = el("text", { x: 46, y: 62, class: "mono", "font-size": 17, fill: "#222" });
code.innerHTML = '<tspan fill="#b0177b">await</tspan> E(store).set(<tspan fill="#1a7f37">"cookie"</tspan>, freshCookie);';
// actors
actor(150, 200, "Alice");
actor(560, 200, "diigo-cache");
actor(930, 200, "Mallory");
// makers
maker(30, 320, 470, 470, "makeMapStore");
maker(560, 320, 530, 470, "makeWebApp");
// MapStoreDo insides
obj(80, 420, 120, 80, "store");
obj(80, 600, 120, 80, "readOnly", true);
const kv = table(260, 430, "kv", [["cookie", "session=…"], ["", ""], ["", ""], ["", ""], ["", ""]], [90, 110]);
// WebAppDo insides
obj(620, 560, 140, 90, "access", true);
const init = table(830, 420, "init", [["url", "diigo…"], ["headers", "{…}"], ["headerStore", ""], ["", ""]], [130, 85]);
// references
ref(140, 228, 125, 418, 0.05); // Alice → store
ref(172, 224, 508, 210, 0.04); // Alice → diigo-cache (she wired it)
ref(560 - 6, 200 + 18, 660, 553, 0.12); // cache → access
ref(930 - 14, 200 + 20, 720, 553, -0.08); // Mallory → access
ref(160, 470, kv.x - 2, kv.rowY(0), 0.05); // store → kv
ref(150, 666, kv.x - 2, kv.rowY(0) + 6, -0.25); // readOnly → kv (read)
ref(760, 600, init.x - 2, init.rowY(2), -0.1); // access → init
ref(init.colX + 40, init.rowY(2), 208, 640, -0.28); // headerStore → readOnly
// captions
const cap = (x, y, s, attrs = {}) => text(x, y, s, "sans", { "font-size": 14, fill: "#33536f", ...attrs });
cap(930, 118, "holds the instance id:", {});
cap(930, 136, "uses the cookie, can't reach store", {});
cap(330, 300, "storeId: rotate · revoke", {});
cap(760, 806, "init.headerStore → the store's read-only facet, read at every send", {});
cap(265, 806, "shapes: keyShape \"cookie\", valueShape M.string()", {});
</script>
</body></html>
Binary file not shown.

After

Width:  |  Height:  |  Size: 113 KiB

+16
View File
@@ -0,0 +1,16 @@
{
"name": "capsite-lib",
"private": true,
"type": "module",
"scripts": {
"types": "wrangler types -c wrangler.tests.jsonc",
"lint": "wrangler types -c wrangler.tests.jsonc --check && tsc -p tsconfig.json && tsc -p tests/tsconfig.json",
"test": "vitest run"
},
"devDependencies": {
"@cloudflare/vitest-plugin": "1.3.6",
"typescript": "^5.9.0",
"vitest": "4.1.11",
"wrangler": "^4.148.0"
}
}
+2240
View File
File diff suppressed because it is too large. Load diff
+129
View File
@@ -0,0 +1,129 @@
/**
* @file a work-alike for enough of @endo/patterns: `M.string()`, `M.any()`,
* and literal patterns (a string, number, boolean, or null matches only
* itself, so `"cookie"` as a key shape admits exactly one key). Matchers
* are the same data endo makes: tagged records, as from @endo/pass-style's
* `makeTagged` (frozen here, where endo would harden).
*
* Structured clone (DO storage, Workers RPC) drops a tagged record's
* symbol-keyed and non-enumerable properties, so patterns cross those
* boundaries in endo's smallcaps encoding: see {@link toSmallcaps}.
* @see https://github.com/endojs/endo/tree/master/packages/patterns
* @see https://github.com/endojs/endo/blob/master/packages/marshal/src/encodeToSmallcaps.js
*/
const PASS_STYLE = Symbol.for("passStyle");
export type Tagged = Readonly<{ [Symbol.toStringTag]: string; payload: unknown }>;
export type Pattern = Tagged | string | number | boolean | null;
/** @endo/pass-style's `makeTagged` */
export function makeTagged(tag: string, payload: unknown): Tagged {
return Object.freeze(
Object.create(Object.prototype, {
[PASS_STYLE]: { value: "tagged" },
[Symbol.toStringTag]: { value: tag },
payload: { value: payload, enumerable: true },
}),
) as Tagged;
}
const isTagged = (x: unknown): x is Tagged =>
typeof x === "object" && x !== null && (x as Record<symbol, unknown>)[PASS_STYLE] === "tagged";
const StringShape = makeTagged("match:string", Object.freeze([]));
const AnyShape = makeTagged("match:any", undefined);
export const M = Object.freeze({
string: (): Tagged => StringShape,
any: (): Tagged => AnyShape,
});
export function matches(specimen: unknown, pattern: Pattern): boolean {
if (typeof pattern === "object" && pattern !== null) {
const tag = isTagged(pattern) ? pattern[Symbol.toStringTag] : undefined;
switch (tag) {
case "match:string":
return typeof specimen === "string";
case "match:any":
return true;
default:
throw new TypeError("not a pattern");
}
}
return Object.is(specimen, pattern);
}
/** the pattern as written in code, e.g. `M.string()` or `"cookie"` */
export function describe(pattern: Pattern): string {
if (typeof pattern === "object" && pattern !== null) {
return `M.${pattern[Symbol.toStringTag].replace(/^match:/, "")}()`;
}
return JSON.stringify(pattern);
}
/**
* throw unless `specimen` matches `pattern`. Unlike endo's, the message
* names the pattern and `label` but never the specimen: specimens here
* can be credentials, and messages cross RPC and land in logs.
*/
export function mustMatch(specimen: unknown, pattern: Pattern, label: string): void {
if (!matches(specimen, pattern)) {
throw new TypeError(`${label}: must match ${describe(pattern)}`);
}
}
/** a smallcaps-encoded passable, as JSON-able data */
export type Smallcaps = null | boolean | number | string | Smallcaps[] | { "#tag": string; payload: Smallcaps };
/** smallcaps quotes strings that start with one of these with `!` */
const startsSpecial = (s: string) => /^[!-\-]/.test(s);
/**
* the subset of @endo/marshal's smallcaps encoding that patterns here use:
* primitives, arrays, and tagged records (not a whole CapData: no `#`
* body prefix, no slots)
*/
export function toSmallcaps(passable: unknown): Smallcaps {
if (passable === undefined) return "#undefined";
if (passable === null || typeof passable === "boolean") return passable;
if (typeof passable === "number") {
if (Number.isNaN(passable)) return "#NaN";
if (passable === Infinity) return "#Infinity";
if (passable === -Infinity) return "#-Infinity";
return Object.is(passable, -0) ? 0 : passable;
}
if (typeof passable === "string") return startsSpecial(passable) ? `!${passable}` : passable;
if (Array.isArray(passable)) return passable.map(toSmallcaps);
if (isTagged(passable)) {
return { "#tag": toSmallcaps(passable[Symbol.toStringTag]) as string, payload: toSmallcaps(passable.payload) };
}
throw new TypeError("toSmallcaps: unsupported passable");
}
/** the inverse of {@link toSmallcaps} */
export function fromSmallcaps(encoding: Smallcaps): unknown {
if (encoding === null || typeof encoding === "boolean" || typeof encoding === "number") return encoding;
if (typeof encoding === "string") {
if (encoding.startsWith("!")) return encoding.slice(1);
switch (encoding) {
case "#undefined":
return undefined;
case "#NaN":
return NaN;
case "#Infinity":
return Infinity;
case "#-Infinity":
return -Infinity;
}
if (startsSpecial(encoding)) throw new TypeError("fromSmallcaps: unsupported special string");
return encoding;
}
if (Array.isArray(encoding)) return Object.freeze(encoding.map(fromSmallcaps));
if (typeof encoding === "object" && "#tag" in encoding) {
const { "#tag": tag, payload, ...rest } = encoding;
if (Object.keys(rest).length > 0) throw new TypeError("fromSmallcaps: extra properties on a tagged record");
return makeTagged(fromSmallcaps(tag) as string, fromSmallcaps(payload));
}
throw new TypeError("fromSmallcaps: unsupported encoding");
}
+60
View File
@@ -0,0 +1,60 @@
/**
* @file a MapStore as a Durable Object of its own, so a store can be held
* apart from whoever reads it: the DO id is the store's full authority
* (e.g. the account owner's, to rotate a credential), while a reader is
* handed only {@link MapStoreDo.readOnly}.
*/
import { DurableObject } from "cloudflare:workers";
import { fromSmallcaps, matches, toSmallcaps, type Pattern, type Smallcaps } from "./patterns.js";
import { makeMapStore, type MapStore, type ReadonlyMapStore, type StoreShapes } from "./store.js";
/** store shapes as they cross RPC and sit in storage: smallcaps-encoded
* (structured clone would strip a matcher down to its payload) */
export type EncodedShapes = { keyShape?: Smallcaps; valueShape?: Smallcaps };
export const encodeShapes = ({ keyShape, valueShape }: StoreShapes): EncodedShapes => ({
...(keyShape === undefined ? {} : { keyShape: toSmallcaps(keyShape) }),
...(valueShape === undefined ? {} : { valueShape: toSmallcaps(valueShape) }),
});
const decodeShapes = ({ keyShape, valueShape }: EncodedShapes): StoreShapes => ({
...(keyShape === undefined ? {} : { keyShape: fromSmallcaps(keyShape) as Pattern }),
...(valueShape === undefined ? {} : { valueShape: fromSmallcaps(valueShape) as Pattern }),
});
type Config = { label: string; shapes: EncodedShapes };
/** a store whose label and shapes its creator fixes, once */
export class MapStoreDo extends DurableObject {
/** one-shot: fix the label and shapes (see {@link encodeShapes}); an
* identical re-configure is a no-op */
configure(label: string, shapes: EncodedShapes): void {
const decoded = decodeShapes(shapes);
for (const shape of [decoded.keyShape, decoded.valueShape]) {
if (shape !== undefined) matches(undefined, shape); // throws on a non-pattern
}
makeMapStore(label, decoded, this.ctx.storage.kv); // throws on a bad label
const config: Config = { label, shapes: encodeShapes(decoded) };
const existing = this.ctx.storage.kv.get<Config>("config");
if (existing) {
if (JSON.stringify(existing) !== JSON.stringify(config)) {
throw new Error("already configured with a different label or shapes");
}
return;
}
this.ctx.storage.kv.put("config", config);
}
/** the store, with full authority */
store(): MapStore<unknown> {
const config = this.ctx.storage.kv.get<Config>("config");
if (!config) throw new Error("not configured: this store was never set up");
return makeMapStore(config.label, decodeShapes(config.shapes), this.ctx.storage.kv);
}
/** the store's queries alone */
readOnly(): ReadonlyMapStore<unknown> {
return this.store().readOnly();
}
}
+133
View File
@@ -0,0 +1,133 @@
/**
* @file a work-alike for enough of @agoric/store's MapStore: string keys,
* shapes checked with ./patterns.js, entries kept in a Durable Object's
* synchronous KV storage. The facets are RpcTargets (workerd's / capnweb's,
* in place of endo's `Far`), so a store or its read-only view can be
* handed out by reference.
* @see https://github.com/Agoric/agoric-sdk/tree/master/packages/store
*/
import { RpcTarget } from "cloudflare:workers";
import { M, mustMatch, type Pattern } from "./patterns.js";
/** the storage a store needs: a DO's `ctx.storage.kv`, or a stand-in */
export type Kv = Pick<SyncKvStorage, "get" | "put" | "delete" | "list">;
export type StoreShapes = { keyShape?: Pattern; valueShape?: Pattern };
/**
* Entries live under `store:<label>:<key>`, so the label also names the
* storage area (as a baggage key does for endo's durable stores): two
* stores with one label on one Kv are the same store.
*
* Error messages name the label and shapes, never keys or values.
*/
export function makeMapStore<V>(label: string, { keyShape = M.string(), valueShape = M.any() }: StoreShapes, kv: Kv) {
if (label.includes(":")) throw new TypeError("store label must not contain ':'");
const prefix = `store:${label}:`;
const core: Core<V> = {
label,
has: (key) => kv.get(prefix + key) !== undefined,
get: (key) => kv.get<V>(prefix + key),
put: (key, value) => kv.put(prefix + key, value),
delete: (key) => {
kv.delete(prefix + key);
},
entries: () => [...kv.list<V>({ prefix })].map(([k, v]): [string, V] => [k.slice(prefix.length), v]),
checkKey: (key) => mustMatch(key, keyShape, `${label}: key`),
checkValue: (value) => mustMatch(value, valueShape, `${label}: value`),
};
return new MapStore<V>(core);
}
type Core<V> = {
label: string;
has(key: string): boolean;
get(key: string): V | undefined;
put(key: string, value: V): void;
delete(key: string): void;
entries(): [string, V][];
checkKey(key: unknown): void;
checkValue(value: unknown): void;
};
/** the queries */
function reads<V>(core: Core<V>) {
return {
has: (key: string): boolean => core.has(key),
get: (key: string): V => {
const value = core.get(key);
if (value === undefined) throw new RangeError(`${core.label}: key not found`);
return value;
},
keys: (): string[] => core.entries().map(([k]) => k),
values: (): V[] => core.entries().map(([, v]) => v),
entries: (): [string, V][] => core.entries(),
getSize: (): number => core.entries().length,
};
}
/** a store's queries only: what a reader of a rotating credential holds */
export class ReadonlyMapStore<V> extends RpcTarget {
#reads: ReturnType<typeof reads<V>>;
constructor(core: Core<V>) {
super();
this.#reads = reads(core);
}
has(key: string): boolean {
return this.#reads.has(key);
}
get(key: string): V {
return this.#reads.get(key);
}
keys(): string[] {
return this.#reads.keys();
}
values(): V[] {
return this.#reads.values();
}
entries(): [string, V][] {
return this.#reads.entries();
}
getSize(): number {
return this.#reads.getSize();
}
}
/** a store's full authority: the queries plus `init`/`set`/`delete`, and
* {@link MapStore.readOnly} to hand out the queries alone */
export class MapStore<V> extends ReadonlyMapStore<V> {
#core: Core<V>;
constructor(core: Core<V>) {
super(core);
this.#core = core;
}
/** add a key that is not yet present */
init(key: string, value: V): void {
this.#core.checkKey(key);
this.#core.checkValue(value);
if (this.#core.has(key)) throw new RangeError(`${this.#core.label}: key already registered`);
this.#core.put(key, value);
}
/** replace the value of a key that is present */
set(key: string, value: V): void {
this.#core.checkKey(key);
this.#core.checkValue(value);
if (!this.#core.has(key)) throw new RangeError(`${this.#core.label}: key not found`);
this.#core.put(key, value);
}
delete(key: string): void {
if (!this.#core.has(key)) throw new RangeError(`${this.#core.label}: key not found`);
this.#core.delete(key);
}
readOnly(): ReadonlyMapStore<V> {
return new ReadonlyMapStore(this.#core);
}
}
+142
View File
@@ -0,0 +1,142 @@
import { env } from "cloudflare:workers";
import { evictDurableObject } from "cloudflare:test";
import { describe, expect, it } from "vitest";
import { M, fromSmallcaps, makeTagged, matches, mustMatch, toSmallcaps } from "../src/patterns.js";
import { encodeShapes } from "../src/store-do.js";
type TestEnv = {
STORE_DO: DurableObjectNamespace<import("./test-worker.js").StoreHost>;
MAP_STORE_DO: DurableObjectNamespace<import("../src/store-do.js").MapStoreDo>;
};
const testEnv = env as unknown as TestEnv;
const SECRET = "session=s3kr1t";
/** the message an RPC call fails with, or "" if it succeeds */
async function errorOf(call: Promise<unknown>): Promise<string> {
try {
await call;
return "";
} catch (err) {
return String(err);
}
}
describe("patterns", () => {
it("M.string() matches strings only; a literal matches only itself", () => {
expect(matches("x", M.string())).toBe(true);
expect(matches(1, M.string())).toBe(false);
expect(matches(undefined, M.any())).toBe(true);
expect(matches("cookie", "cookie")).toBe(true);
expect(matches("authorization", "cookie")).toBe(false);
});
it("matchers are endo's tagged records", () => {
const s = M.string() as unknown as Record<string | symbol, unknown>;
expect(s[Symbol.for("passStyle")]).toBe("tagged");
expect(s[Symbol.toStringTag]).toBe("match:string");
expect(Object.keys(s)).toEqual(["payload"]);
expect(s.payload).toEqual([]);
expect((M.any() as unknown as Record<string, unknown>).payload).toBeUndefined();
expect(Object.isFrozen(s)).toBe(true);
});
it("smallcaps: endo's encoding, round-tripped", () => {
expect(toSmallcaps(M.string())).toEqual({ "#tag": "match:string", payload: [] });
expect(toSmallcaps(M.any())).toEqual({ "#tag": "match:any", payload: "#undefined" });
expect(toSmallcaps("#cookie")).toBe("!#cookie");
expect(toSmallcaps([NaN, -0, Infinity])).toEqual(["#NaN", 0, "#Infinity"]);
const back = fromSmallcaps(toSmallcaps(M.string())) as Parameters<typeof matches>[1];
expect(matches("x", back)).toBe(true);
expect(matches(1, back)).toBe(false);
expect(fromSmallcaps("!#cookie")).toBe("#cookie");
expect(() => matches("x", makeTagged("match:nope", []))).toThrow("not a pattern");
});
it("mustMatch names the shape but not the specimen", () => {
expect(() => mustMatch(SECRET, "cookie", "jar: key")).toThrow('jar: key: must match "cookie"');
expect(() => mustMatch(SECRET, "cookie", "jar: key")).not.toThrow(SECRET);
});
});
describe("MapStore over RPC", () => {
const fresh = () => testEnv.STORE_DO.get(testEnv.STORE_DO.newUniqueId());
it("rotates: the admin's set is what the read-only view sees next", async () => {
const store = await fresh().cookies();
const rd = await store.readOnly();
await store.init("cookie", "session=old");
expect(await rd.get("cookie")).toBe("session=old");
await store.set("cookie", "session=new");
expect(await rd.entries()).toEqual([["cookie", "session=new"]]);
});
it("the read-only view has no write methods", async () => {
const store = await fresh().cookies();
const rd = (await store.readOnly()) as unknown as { set(k: string, v: string): Promise<void> };
expect(await errorOf(rd.set("cookie", SECRET))).not.toBe("");
expect(await store.has("cookie")).toBe(false);
});
it("the key shape admits only `cookie`; errors carry no values", async () => {
const store = await fresh().cookies();
const err = await errorOf(store.init("authorization", SECRET));
expect(err).toContain('cookie: key: must match "cookie"');
expect(err).not.toContain(SECRET);
expect(await errorOf(store.init("cookie", 42 as unknown as string))).toContain("cookie: value: must match M.string()");
});
it("init refuses a present key; set and get refuse a missing one", async () => {
const store = await fresh().cookies();
expect(await errorOf(store.set("cookie", SECRET))).toContain("cookie: key not found");
expect(await errorOf(store.get("cookie"))).toContain("cookie: key not found");
await store.init("cookie", SECRET);
const err = await errorOf(store.init("cookie", SECRET));
expect(err).toContain("cookie: key already registered");
expect(err).not.toContain(SECRET);
});
it("survives eviction: entries live in DO storage", async () => {
const stub = fresh();
{
// disposed before eviction: a live store stub holds the DO open
using store = await stub.cookies();
await store.init("cookie", "session=durable");
}
await evictDurableObject(stub);
using again = await stub.cookies();
expect(await again.get("cookie")).toBe("session=durable");
expect(await again.getSize()).toBe(1);
await again.delete("cookie");
expect(await again.keys()).toEqual([]);
});
});
describe("MapStoreDo", () => {
const COOKIE_SHAPES = encodeShapes({ keyShape: "cookie", valueShape: M.string() });
const fresh = () => testEnv.MAP_STORE_DO.get(testEnv.MAP_STORE_DO.newUniqueId());
it("keeps the shapes its creator chose, across eviction", async () => {
const stub = fresh();
await stub.configure("cookie", COOKIE_SHAPES);
await evictDurableObject(stub);
{
using store = await stub.store();
expect(await errorOf(store.init("authorization", SECRET))).toContain('cookie: key: must match "cookie"');
expect(await errorOf(store.init("cookie", 42))).toContain("cookie: value: must match M.string()");
await store.init("cookie", "session=a");
}
using rd = await stub.readOnly();
expect(await rd.entries()).toEqual([["cookie", "session=a"]]);
});
it("configures once: the same shapes again is a no-op, others are refused", async () => {
const stub = fresh();
await stub.configure("cookie", COOKIE_SHAPES);
await stub.configure("cookie", encodeShapes({ keyShape: "cookie", valueShape: M.string() }));
expect(await errorOf(stub.configure("cookie", encodeShapes({ keyShape: M.string() })))).toContain(
"already configured",
);
expect(await errorOf(fresh().store())).toContain("not configured");
});
});
+17
View File
@@ -0,0 +1,17 @@
/** @file the tests' main module: a DO that hosts stores on its own storage. */
import { DurableObject } from "cloudflare:workers";
import { M } from "../src/patterns.js";
import { makeMapStore, type MapStore } from "../src/store.js";
export { MapStoreDo } from "../src/store-do.js";
/** hosts the store from the credential-rotation sketch: one key, `cookie` */
export class StoreHost extends DurableObject {
cookies(): MapStore<string> {
return makeMapStore<string>("cookie", { keyShape: "cookie", valueShape: M.string() }, this.ctx.storage.kv);
}
}
export default {
fetch: () => new Response("store tests", { status: 404 }),
};
+7
View File
@@ -0,0 +1,7 @@
{
"extends": "../tsconfig.json",
"compilerOptions": {
"types": ["@cloudflare/vitest-plugin/types"]
},
"include": ["./**/*", "../src/**/*", "../worker-configuration.d.ts"]
}
+15
View File
@@ -0,0 +1,15 @@
{
"compilerOptions": {
"target": "ES2022",
"module": "preserve",
"moduleResolution": "bundler",
"strict": true,
"esModuleInterop": true,
"verbatimModuleSyntax": true,
"skipLibCheck": true,
"lib": ["ES2022", "ESNext.Disposable"],
"noEmit": true,
"types": []
},
"include": ["src/**/*", "worker-configuration.d.ts"]
}
+9
View File
@@ -0,0 +1,9 @@
import { defineProject } from "vitest/config";
import { cloudflareTest } from "@cloudflare/vitest-plugin";
export default defineProject({
plugins: [cloudflareTest({ wrangler: { configPath: "./wrangler.tests.jsonc" } })],
test: {
include: ["tests/**/*.test.ts"],
},
});
+17
View File
@@ -0,0 +1,17 @@
{
"$schema": "node_modules/wrangler/config-schema.json",
// a host DO for the stores under test (lib ships no worker of its own)
"name": "capsite-lib-tests",
"main": "./tests/test-worker.ts",
"compatibility_date": "2026-09-29",
"durable_objects": {
"bindings": [
{ "name": "STORE_DO", "class_name": "StoreHost" },
{ "name": "MAP_STORE_DO", "class_name": "MapStoreDo" }
]
},
"exports": {
"StoreHost": { "type": "durable-object", "storage": "sqlite" },
"MapStoreDo": { "type": "durable-object", "storage": "sqlite" }
}
}
-107
View File
@@ -1,107 +0,0 @@
This is an EmDash site -- a CMS built on Astro with a full admin UI.
## Commands
```bash
pnpm dev # Start the Astro dev server
npx emdash types # Regenerate TypeScript types from a running site
```
The admin UI is at `http://localhost:4321/_emdash/admin`.
## Key Files
| File | Purpose |
| ------------------------ | ---------------------------------------------------------------------------------- |
| `astro.config.mjs` | Astro config with `emdash()` integration, database, and storage |
| `src/live.config.ts` | EmDash loader registration (boilerplate -- don't modify) |
| `seed/seed.json` | Schema definition + demo content (collections, fields, taxonomies, menus, widgets) |
| `emdash-env.d.ts` | Generated types for collections (auto-regenerated on dev server start) |
| `src/layouts/Base.astro` | Base layout with EmDash wiring (menus, search, page contributions) |
| `src/pages/` | Astro pages -- all server-rendered |
## Skills
Agent skills are in `.agents/skills/`. Load them when working on specific tasks:
- **building-emdash-site** -- Querying content, rendering Portable Text, schema design, seed files, site features (menus, widgets, search, SEO, comments, bylines). Start here.
- **creating-plugins** -- Building EmDash plugins with hooks, storage, admin UI, API routes, and Portable Text block types.
- **emdash-cli** -- CLI commands for content management, seeding, type generation, and visual editing flow.
## Documentation
The EmDash docs are available as an MCP server at `https://docs.emdashcms.com/mcp`. When you need to verify an API, hook, config option, field type, or pattern, call `search_docs` against the live documentation rather than relying on training-data recall. The docs reflect current behaviour; assumptions may not.
This template ships with `.mcp.json`, `.cursor/mcp.json`, and `.vscode/mcp.json` so Claude Code, Cursor, and VS Code auto-discover the docs server. Other tools (OpenCode, Windsurf, etc.) need a manual one-time setup -- see [docs.emdashcms.com/docs-mcp](https://docs.emdashcms.com/docs-mcp).
## Rules
- All content pages must be server-rendered (`output: "server"`). No `getStaticPaths()` for CMS content.
- Image fields are objects (`{ src, alt }`), not strings. Use `<Image image={...} />` from `"emdash/ui"`.
- `entry.id` is the slug (for URLs). `entry.data.id` is the database ULID (for API calls like `getEntryTerms`).
- When Astro's cache is enabled, pass content-query hints to `Astro.cache.set(cacheHint)`. Use the `WithCacheHint` variants for site settings, menus, taxonomies, and widget areas rendered by cached routes.
- Taxonomy names in queries must match the seed's `"name"` field exactly (e.g., `"category"` not `"categories"`).
## This Template
A blog with posts, pages, categories, tags, full-text search, and RSS. Designed for personal writing, technical writing, indie newsletters, and anything where the writing is the product. Editorial-tech aesthetic: confident sans-serif, restrained accent, real article structure with bylines and reading time.
## Pages
| Page | Path | What it shows |
| ----------- | ------------------ | ------------------------------------------------------------------------------------------------------ |
| Home | `/` | Featured post hero (large image + excerpt), latest posts grid |
| All posts | `/posts` | Article count, full post list with excerpts and tag chips |
| Post detail | `/posts/[slug]` | Featured image, title, body, left meta column (authors + date), right TOC + search + categories gutter |
| Search | `/search` | Full-text search UI |
| Page | `/pages/[slug]` | Static page content (Portable Text) |
| Category | `/category/[slug]` | Posts filtered by category |
| Tag | `/tag/[slug]` | Posts filtered by tag |
| RSS | `/rss.xml` | Generated feed |
## Schema
- `posts` collection: `title`, `featured_image`, `content` (Portable Text), `excerpt` (text).
- `pages` collection: `title`, `content` (Portable Text). Used for `/about` etc.
- Taxonomies: `category`, `tag`.
- Single `primary` menu (Home, About, Posts by default).
Site settings have `title` and `tagline` -- both render in the header / footer.
## Visual character
Single typeface: **Inter** on `--font-body`, used for everything including headings (`--font-heading` defaults to the body face; tighter letter-spacing on h1/h2). **JetBrains Mono** on `--font-mono` for inline code and code blocks. Body and headings share the same family; weight and size carry the hierarchy (`--font-weight-heading` 600, `--font-weight-display` 700 for h1/page titles).
The brand colour is `#0066cc` (`--color-brand`) -- used for links, the post-card title hover, and the search input focus ring. There's also a secondary text colour (`--color-text-secondary`) and a `--color-muted` for meta info. Don't add a second accent.
The article layout is the standout feature: a three-column reading view with a left meta column (author bylines, date), centred 680px body column, and a right gutter for search, table of contents, and categories. Don't flatten that into one column on desktop -- the layout signals "this is something to read".
## Customisation
Design tokens live in `src/styles/tokens.css` with their default values. To restyle the site, override tokens in `src/styles/theme.css` -- declarations there are unlayered, so they always beat the `@layer base` defaults. Don't edit `tokens.css` or `Base.astro` for visual changes.
Colours are defined with `light-dark(<light>, <dark>)`, so each token carries both modes. Overriding with a plain colour changes light and dark at once; use `light-dark()` in the override to keep them distinct. There is no separate dark palette to maintain.
Webfonts are configured in `astro.config.mjs` under `fonts:`. To swap the body face, change the `name:` for the entry bound to `cssVariable: "--font-body"`. Good alternatives: Geist, IBM Plex Sans, Söhne (if you have a licence), Public Sans. If you want a serif-bodied blog, swap to a humanist serif like Source Serif, Crimson Pro, or Lora -- but then also raise `--font-size-base` to `1.0625rem` for readability. To give headings their own face (or use a system font) without touching the font pipeline, override `--font-heading` or `--font-body` in `theme.css`.
CSS variables worth knowing (see `tokens.css` for the full list):
- `--color-brand`, `--color-brand-hover`, `--color-on-brand`, `--color-brand-ring`
- `--color-bg`, `--color-bg-subtle`, `--color-surface`, `--color-text`, `--color-text-secondary`, `--color-muted`, `--color-border`, `--color-border-subtle`
- `--font-body`, `--font-heading`, `--font-mono`
- `--font-weight-heading` (600) / `--font-weight-display` (700) -- heading weights; lower them if you switch to a serif
- `--tracking-tight` / `--tracking-snug` / `--tracking-wide` / `--tracking-wider` -- letter-spacing tokens used across headings and meta labels
- `--content-width` (680px) -- article body column
- `--wide-width` (1200px) -- max container
- `--gutter-width` (200px) -- right sidebar (TOC) on article pages
- `--meta-col-width` (180px) -- left meta column on article pages
- `--avatar-size-{xs,sm,md,lg}` -- byline avatar sizes at different scales
## What not to do
- Don't add a second accent colour or coloured section backgrounds. The page should be black, white, and one blue.
- Don't replace Inter with a display sans (Bebas, Anton, etc.). Headings rely on weight contrast, not novelty faces.
- Don't collapse the article gutter on desktop -- it's part of the reading experience.
- Don't use stock blog copy ("Welcome to my blog", "Stay tuned for more"). Write a real tagline that says what this blog is about.
- Don't seed the home page with three identical placeholder posts. If you only have one real post, show one real post.
- Comments are enabled on posts and rendered on the post detail page. Configure moderation before publishing the site, or remove `commentsEnabled` and the comments UI together.
+1
View File
@@ -0,0 +1 @@
CONTRIBUTING.md
+51
View File
@@ -0,0 +1,51 @@
# Contributing to potluck
potluck started as EmDash's `cloudflare:blog` template (`pnpm create emdash@latest`); it's part of [the capsite goal tree](../capsite-checklist.md), where it is the site that publishes the weekly roundup.
## How it's put together
- **Issues** are the `issues` collection ([seed/issues.json](seed/issues.json)): `title`, `week` (an ISO week, `2026-W40`), `dek` (one-line summary), `body` (Portable Text). The slug is the week, lowercased: `/issues/2026-w40`. [issues-sketch.md](issues-sketch.md) is the dry run behind the three sample issues: real public bookmarks, editorial lines marked *draft*.
- **Pages:** home and `/issues` list issues ([src/pages/issues/](src/pages/issues/)); `/tag/<tag>` lists issues by diigo tag; search and `/rss.xml` cover issues. The `posts` collection and its pages are the template's, still in place.
- **Microformats:** an issue list is an `h-feed` of `h-entry`s; an issue is an `h-entry` (`p-name`, `dt-published`, `p-summary`, `e-content`). In issue bodies, [CiteLink.astro](src/components/CiteLink.astro) marks off-site links as `h-cite` and `/tag/<tag>` links as rel-tag `p-category`.
- **Look:** plain HTML, after kmjn.org's [Plain HTML with a few bureaucratic gestures](https://www.kmjn.org/notes/modern_plain_html.html): system fonts (no webfonts), one ~800px column, links in the text color with a dotted underline. Restyle in [src/styles/theme.css](src/styles/theme.css), which overrides the defaults in `tokens.css` (unlayered, so it always wins); don't edit `tokens.css` or `Base.astro` for visual changes. Colors use `light-dark(<light>, <dark>)`.
## Plugins
[diigo-week](../diigo-week/README.md) (Plugins → Draft from diigo in the admin) drafts an issue from a week of diigo bookmarks. It reads diigo-cache's public page as microformats over the `DIIGO_CACHE` service binding in [wrangler.jsonc](wrangler.jsonc); in dev, run diigo-cache's `pnpm dev` alongside. It's linked as `link:../diigo-week`, and registered in [astro.config.mjs](astro.config.mjs). After changing it, restart the dev server.
## Seeds
[seed/seed.json](seed/seed.json) (site settings, menus, widgets, the template's collections) is applied on the first request to an empty database. [seed/issues.json](seed/issues.json) is separate: `pnpm seed:issues` applies it to the local D1 database, which exists after the dev server's first request. Run it with the dev server stopped.
## Checks
```sh
pnpm typecheck # astro check
pnpm build
```
## EmDash rules
- All content pages are server-rendered (`output: "server"`); no `getStaticPaths()` for CMS content.
- Image fields are objects (`{ src, alt }`), not strings; render with `<Image image={...} />` from `"emdash/ui"`.
- `entry.id` is the slug (for URLs); `entry.data.id` is the database ULID (for API calls like `getEntryTerms`).
- When Astro's cache is enabled, pass query hints to `Astro.cache.set(cacheHint)`.
- Taxonomy names in queries match the seed's `"name"` exactly (`"tag"`, not `"tags"`).
- `npx emdash types` regenerates `emdash-env.d.ts` from a running site (the dev server also regenerates it on start).
The EmDash docs are an MCP server at `https://docs.emdashcms.com/mcp` (wired up in `.mcp.json`, `.cursor/mcp.json`, `.vscode/mcp.json`). Check them rather than recalling APIs from memory.
## Deploying
```sh
pnpm wrangler login
pnpm deploy
```
The first deploy provisions the D1 database and R2 bucket named in [wrangler.jsonc](wrangler.jsonc). See [Deploy to Cloudflare](https://docs.emdashcms.com/deployment/cloudflare/).
Other workerd hosts are untested: the site uses D1 and R2 through `@emdash-cms/cloudflare` and the Cloudflare adapter (locally, miniflare stands in for both), so a host has to supply those two bindings.
## For agents
`AGENTS.md` is a symlink to this file. capsite's [working rules](../AGENTS.md) apply here too (the dev server is the user's to start and stop). EmDash skills are in `.agents/skills/`: `building-emdash-site` (start here), `creating-plugins`, `emdash-cli`.
+14 -60
View File
@@ -1,68 +1,22 @@
# EmDash Blog Template (Cloudflare)
# potluck
A clean, minimal blog built with [EmDash](https://github.com/emdash-cms/emdash) and deployed on Cloudflare Workers with D1 and R2.
A weekly zine of what you bookmarked elsewhere, published on your own site. Each issue is one week of links with a few words around them.
[![Deploy to Cloudflare](https://deploy.workers.cloudflare.com/button)](https://deploy.workers.cloudflare.com/?url=https://github.com/emdash-cms/templates/tree/main/blog-cloudflare)
## Getting started
![Blog template homepage](https://raw.githubusercontent.com/emdash-cms/emdash/main/assets/templates/blog/latest/homepage-light-desktop.jpg)
## What's Included
- Featured post hero on the homepage
- Post archive with reading time estimates
- Category and tag archives
- Full-text search
- RSS feed
- SEO metadata and JSON-LD
- Dark/light mode
## Pages
| Page | Route |
|---|---|
| Homepage | `/` |
| All posts | `/posts` |
| Single post | `/posts/:slug` |
| Category archive | `/category/:slug` |
| Tag archive | `/tag/:slug` |
| Search | `/search` |
| Static pages | `/pages/:slug` |
| 404 | fallback |
## Screenshots
| | Desktop | Mobile |
|---|---|---|
| Light | ![homepage light desktop](https://raw.githubusercontent.com/emdash-cms/emdash/main/assets/templates/blog/latest/homepage-light-desktop.jpg) | ![homepage light mobile](https://raw.githubusercontent.com/emdash-cms/emdash/main/assets/templates/blog/latest/homepage-light-mobile.jpg) |
| Dark | ![homepage dark desktop](https://raw.githubusercontent.com/emdash-cms/emdash/main/assets/templates/blog/latest/homepage-dark-desktop.jpg) | ![homepage dark mobile](https://raw.githubusercontent.com/emdash-cms/emdash/main/assets/templates/blog/latest/homepage-dark-mobile.jpg) |
## Infrastructure
- **Runtime:** Cloudflare Workers
- **Database:** D1
- **Storage:** R2
- **Framework:** Astro with `@astrojs/cloudflare`
## Local Development
```bash
```sh
pnpm install
pnpm dev
pnpm dev # open http://localhost:4321/_emdash/admin, finish the setup wizard, then Ctrl-C
pnpm seed:issues # three sample issues, W38–W40 of 2026
pnpm dev # → http://localhost:4321/issues
```
Open http://localhost:4321/_emdash/admin and complete the setup wizard. EmDash runs database migrations and applies the blog seed during setup. The site is available at http://localhost:4321.
## Bookmarks in, a zine out
## Deploying
- **Your web, your site:** the bookmarks come from your own diigo account, so the web you read becomes something you publish ([PESOS](https://indieweb.org/PESOS)). The [diigo-week](../diigo-week/README.md) plugin drafts an issue from a week of them; you cut and comment.
- **A week per issue:** `/issues/2026-w40`, by ISO week, UTC.
- **Microformats:** feed readers and IndieWeb tools see each issue as an [h-entry](https://microformats.org/wiki/h-entry) and each bookmark as an [h-cite](https://microformats.org/wiki/h-cite).
- **Plain HTML** that's easy to read: system fonts, one column.
- **A real editor:** it's an [EmDash](https://docs.emdashcms.com/) site, so you write issues in its admin UI. It runs in [workerd](https://github.com/cloudflare/workerd), on your machine or a host.
```bash
pnpm wrangler login
pnpm deploy
```
The first deployment provisions the named D1 database and R2 bucket from `wrangler.jsonc`. See [Deploy to Cloudflare](https://docs.emdashcms.com/deployment/cloudflare/) for production setup, or use the deploy button above.
## See Also
- [Node.js variant](../blog) -- same template using SQLite and local file storage
- [All templates](../)
- [EmDash documentation](https://docs.emdashcms.com/)
How it's put together, checks, and deploying: [CONTRIBUTING](CONTRIBUTING.md).
+2
View File
@@ -16,6 +16,8 @@ export default defineConfig({
emdash({
database: d1({ binding: "DB", session: "auto" }),
storage: r2({ binding: "MEDIA" }),
// drafts an issue from a week of diigo bookmarks (../diigo-week)
plugins: [{ id: "diigo-week", version: "0.1.0", entrypoint: "diigo-week" }],
}),
],
devToolbar: { enabled: false },
+3 -1
View File
@@ -9,6 +9,7 @@
},
"scripts": {
"dev": "astro dev",
"seed:issues": "emdash seed seed/issues.json -d \"$(find .wrangler/state/v3/d1 -name '*.sqlite' ! -name metadata.sqlite)\"",
"build": "astro build",
"preview": "astro preview",
"deploy": "astro build && wrangler deploy",
@@ -19,6 +20,7 @@
"@astrojs/react": "^6.0.5",
"@emdash-cms/cloudflare": "^1.0.1",
"astro": "^7.3.2",
"diigo-week": "link:../diigo-week",
"emdash": "^1.0.1",
"react": "19.2.4",
"react-dom": "19.2.4"
@@ -29,4 +31,4 @@
"wrangler": "^4.125.0"
},
"packageManager": "pnpm@11.9.0+sha512.bd682d5d03fe525ef7c9fd6780c6884d1e756ac4c9c9fe00c538782824310dcf90e3ddc4f53835f06dfaebd5085e41855e0bcbb3b60de2ac5bbab89e5036f03b"
}
}
+3
View File
@@ -20,6 +20,9 @@ importers:
astro:
specifier: ^7.3.2
version: 7.3.6(@types/node@26.6.4)(yaml@2.9.1)
diigo-week:
specifier: link:../diigo-week
version: link:../diigo-week
emdash:
specifier: ^1.0.1
version: 1.2.0(@astrojs/react@6.0.6(@types/node@26.6.4)(@types/react-dom@19.3.0(@types/react@19.3.0))(@types/react@19.3.0)(esbuild@0.28.2)(react-dom@19.2.4(react@19.2.4))(react@19.2.4)(yaml@2.9.1))(@atcute/cbor@2.3.8(@atcute/cid@2.5.0))(@atcute/cid@2.5.0)(@atcute/identity@2.0.2(@atcute/lexicons@2.1.1)(typescript@6.0.3))(@date-fns/tz@1.5.0)(@floating-ui/dom@1.8.0)(@tiptap/extensions@3.31.4(@tiptap/core@3.31.4(@tiptap/pm@3.31.4))(@tiptap/pm@3.31.4))(@types/react-dom@19.3.0(@types/react@19.3.0))(@types/react@19.3.0)(astro@7.3.6(@types/node@26.6.4)(yaml@2.9.1))(date-fns@4.4.0)(echarts@6.1.0)(prosemirror-model@1.25.12)(prosemirror-state@1.4.4)(prosemirror-view@1.42.6)(react-dom@19.2.4(react@19.2.4))(react@19.2.4)(typescript@6.0.3)
+3
View File
@@ -16,6 +16,9 @@
"bucket_name": "my-emdash-media",
},
],
// diigo-cache's public pages, for the diigo-week plugin (reads its
// microformats; in dev, diigo-cache's own `pnpm dev` serves it)
"services": [{ "binding": "DIIGO_CACHE", "service": "diigo-cache" }],
// Sandboxed plugins require Worker Loader, available on Workers paid plans. Uncomment to enable:
// "worker_loaders": [{ "binding": "LOADER" }],
// General maintenance cron trigger
+38 -3
View File
@@ -3,7 +3,7 @@
DO-backed fluent web client factory: site-agnostic. Deploy once; instances
are attenuated handles whose **ids are bearer capabilities** — anyone
holding the id string can drive the instance; nobody without it can.
TODO: this proxy-page flow (see diigo-cache README) vs running webapp as
TODO: this proxy-page flow (see [diigo-cache CONTRIBUTING](../diigo-cache/CONTRIBUTING.md)) vs running webapp as
the primary worker (`wrangler dev -c ../webapp/wrangler.jsonc -c ../diigo-cache/wrangler.jsonc`)
so its own port serves the mint form directly; reconsider when the UI
grows past a form or two.
@@ -12,19 +12,54 @@ grows past a form or two.
- `GET /` — the mint form. Prefill via query string: `?url=…&headers=<urlencoded JSON>&msPerCall=…&title=…`; secret header keys (cookie, authorization, proxy-authorization) render **blank** — paste them at mint time; never echoed back. A dedicated `cookie` field is sugar for the `cookie` header (if the JSON also has one, they must agree).
- `GET /form?render=1&…` — the same form for a proxying worker (diigo-cache's `/_connect`): the action is left as an `ACTION` placeholder for the proxy to point at its own relay.
- `POST /` — mint. Form-encoded (that page) or JSON `{url, headers, msPerCall}` (service-style). Responds `{id}` (and `x-webapp-id`). The id is *not* logged: possession is authority.
- `POST /` — mint. Form-encoded (that page) or JSON `{url, headers, msPerCall}` (service-style). Responds `{id, storeId}` (and `x-webapp-id`, `x-webapp-store-id`). Neither id is logged: possession is authority.
- `POST /at/<id>` — drive an instance by capability: JSON `{path?, query?, method?, body?}` → the fluent chain → `{status, text}` (or `{error}`; messages carry no header/param values).
- `PUT /store/<storeId>/<key>` — set a header-store entry (JSON value body), e.g. `PUT /store/<storeId>/cookie` with `"session=…"` rotates the cookie; `DELETE` removes it. → `{ok: true}` (or `{error}`; messages carry no keys or values).
## RPC surface (same-account workers; binding `entrypoint: "WebAppFactory"`)
- `fetch(request)` — the HTTP surface above, over the binding
- `newWebApp({url, headers?, msPerCall?})` → `{id, app}` — the fluent stub;
- `newWebApp({url, headers?, msPerCall?})` → `{id, storeId, app}` — the fluent stub;
promise pipelining makes a whole `pathjoin().query().get()` chain a
single round trip
- `at(id)` — the fluent stub for a previously-minted instance
- on a stub, `postForm(fields?, echoCookies?)` → `{status, contentType,
contentDisposition?, location?, body}`: a multipart/form-data POST.
`echoCookies` maps a form field to a cookie name; the cookie's value
(header store included) fills the field at send, so a double-submit
CSRF token goes out without the holder seeing it. Redirects are not
followed and a non-OK status is a reply, not an Error (a 3xx to a login
page is how many sites say "session expired"). `location` comes back
query-stripped; `set-cookie` never comes back. `body` is bytes: the
caller knows its charset.
- `store(storeId)` — a header store, with full authority:
`set("cookie", …)` rotates the cookie under every handle already out
## The header store: two ids, two authorities
Mint makes two things: the instance, and a header store (a `MapStoreDo`
from [the capsite lib](../lib/src/store-do.ts), shaped
`("cookie", { keyShape: "cookie", valueShape: M.string() })`). A `cookie`
header goes into the store, not the instance's `init`; the instance reads
the store's read-only view at every send.
- the **instance id** is authority to *use* the cookie: it goes to the
cache's wiring.
- the **store id** is authority to *choose* it: it stays with the account
owner, who can rotate it (log in again, `PUT`) or revoke it (`DELETE`,
or log out).
Kept apart because whoever can choose the cookie can swap in another
account's session: if the cache's id could rotate, a leaked instance id
would let Mallory fill the cache (and its public page) with her own
bookmarks, and the owner's logout would not stop it.
Instances minted before header stores keep their cookie in
`init.headers`, unchanged; they work as before and cannot rotate.
## Run
pnpm test # header store + rotation (vitest, workerd pool)
npx wrangler dev # → its own ready line (usually :8787 with the
# two-config diigo-cache run; standalone, :8788)
+16
View File
@@ -0,0 +1,16 @@
{
"name": "webapp",
"private": true,
"type": "module",
"scripts": {
"types": "wrangler types",
"lint": "wrangler types --check && tsc -p tsconfig.json && tsc -p tests/tsconfig.json",
"test": "vitest run"
},
"devDependencies": {
"@cloudflare/vitest-plugin": "1.3.6",
"typescript": "^5.9.0",
"vitest": "4.1.11",
"wrangler": "^4.148.0"
}
}
+2240
View File
File diff suppressed because it is too large. Load diff
+253 -34
View File
@@ -7,21 +7,52 @@
// DurableObjectNamespace/DurableObjectId/DurableObjectStub are ambient types
// in worker-configuration.d.ts (not exports of "cloudflare:workers")
import { RpcTarget, DurableObject, WorkerEntrypoint } from "cloudflare:workers";
import { M } from "../../lib/src/patterns.js";
import { encodeShapes, MapStoreDo } from "../../lib/src/store-do.js";
export { MapStoreDo };
export type Query = Record<string, string | number>;
type Headers = Record<string, string>;
/** headers looked up at each send, e.g. a store's read-only view: whoever
* holds the store can rotate a credential under handles already out */
export type HeaderMap = { entries(): [string, unknown][] | Promise<[string, unknown][]> };
/**
* init for a new instance — fixed at creation. An instance is immutable
* authority: no re-config methods, only attenuation via the fluent chain.
* The one thing that changes under it is its header store's contents.
*/
export interface WebAppInit {
url: string;
headers?: Headers;
/** min spacing between outbound calls, ms (0 = unthrottled) */
msPerCall?: number;
/** id of a {@link MapStoreDo} whose entries are added as headers at
* each send (see {@link mintInstance}); the instance only reads it */
headerStore?: string;
}
/**
* what {@link WebApp.postForm} hands back: the status and the response
* headers a caller needs to tell a result from a refusal, never
* `set-cookie` (a credential stays where it is) and with any query
* stripped from `location`
*/
export type Reply = {
status: number;
contentType: string;
contentDisposition?: string;
location?: string;
/** the body as sent: the caller knows its charset (some formats name
* it in their own header, not in content-type) */
body: Uint8Array;
};
/** one outbound request, redirects not followed; the DO's gate and fetch */
export type Exchange = (url: string, headers: Headers, method: "POST", body: FormData) => Promise<Response>;
/** structural type of a stub handle: chain to narrow, then fire */
export interface FluentGet {
pathjoin(ref: string): FluentGet;
@@ -29,8 +60,10 @@ export interface FluentGet {
withHeaders(h: Headers): FluentGet;
withHeader(key: string, value: string): FluentGet;
cookies(cookie: string): FluentGet;
withHeaderMap(map: HeaderMap): FluentGet;
get(): Promise<string>;
post(body: string): Promise<string>;
postForm(fields?: Record<string, string>, echoCookies?: Record<string, string>): Promise<Reply>;
}
/**
@@ -57,17 +90,23 @@ export interface FluentGet {
export class WebApp extends RpcTarget implements FluentGet {
#url: string;
#headers: Headers;
#maps: readonly HeaderMap[];
#send: (url: string, headers: Headers, method: "GET" | "POST", body?: string) => Promise<string>;
#exchange: Exchange;
constructor(
url: string,
headers: Headers,
maps: readonly HeaderMap[],
send: (url: string, headers: Headers, method: "GET" | "POST", body?: string) => Promise<string>,
exchange: Exchange,
) {
super();
this.#url = url;
this.#headers = headers;
this.#maps = maps;
this.#send = send;
this.#exchange = exchange;
}
/** Join a relative path, merging the base's query.
@@ -85,13 +124,13 @@ export class WebApp extends RpcTarget implements FluentGet {
throw new Error(`pathjoin: ${ref} not within ${hereNoQuery.href}`);
}
const baseParams: Query = Object.fromEntries(here.searchParams);
return new WebApp(unionQuery(there.href, baseParams), this.#headers, this.#send);
return new WebApp(unionQuery(there.href, baseParams), this.#headers, this.#maps, this.#send, this.#exchange);
}
/** Union headers, flower-style (see {@link WebApp}): a shared key with a
* different value throws; the same value is fine. */
withHeaders(h: Headers): WebApp {
return new WebApp(this.#url, unionHeaders(this.#headers, h), this.#send);
return new WebApp(this.#url, unionHeaders(this.#headers, h), this.#maps, this.#send, this.#exchange);
}
/** one key, flower-style (see {@link WebApp}) */
@@ -105,15 +144,72 @@ export class WebApp extends RpcTarget implements FluentGet {
/** Union params into the existing query, flower-style (see {@link WebApp}). */
query(params: Query): WebApp {
return new WebApp(unionQuery(this.#url, params), this.#headers, this.#send);
return new WebApp(unionQuery(this.#url, params), this.#headers, this.#maps, this.#send, this.#exchange);
}
/** Add headers read from `map` at each send, unioned flower-style (see
* {@link WebApp}) with the rest — so a contradiction surfaces at send. */
withHeaderMap(map: HeaderMap): WebApp {
return new WebApp(this.#url, this.#headers, [...this.#maps, map], this.#send, this.#exchange);
}
async get(): Promise<string> {
return this.#send(this.#url, this.#headers, "GET");
return this.#send(this.#url, await this.#headersNow(), "GET");
}
async post(body: string): Promise<string> {
return this.#send(this.#url, this.#headers, "POST", body);
return this.#send(this.#url, await this.#headersNow(), "POST", body);
}
/**
* POST `fields` as multipart/form-data. `echoCookies` maps a form field
* to a cookie name: at send, that cookie's value (from the headers,
* header store included) goes in the field, so a double-submit CSRF
* token can be sent by a holder who never sees it.
*
* Unlike `get`/`post`, a non-OK status is a {@link Reply}, not an
* Error, and redirects are not followed: a 3xx to a login page is how
* many sites say "session expired".
*/
async postForm(fields: Record<string, string> = {}, echoCookies: Record<string, string> = {}): Promise<Reply> {
const headers = await this.#headersNow();
if (Object.keys(headers).some((k) => k.toLowerCase() === "content-type")) {
throw new Error("postForm: content-type is set by the form, not the headers");
}
const jar = parseCookies(
Object.entries(headers).find(([k]) => k.toLowerCase() === "cookie")?.[1] ?? "",
);
const form = new FormData();
for (const [name, value] of Object.entries(fields)) form.append(name, value);
for (const [field, cookie] of Object.entries(echoCookies)) {
// names only: values must not appear in Error messages
if (field in fields) throw new Error(`postForm: contradicts on ${field}`);
const value = jar.get(cookie);
if (value === undefined) throw new Error(`postForm: no ${cookie} cookie to echo`);
form.append(field, value);
}
const res = await this.#exchange(this.#url, headers, "POST", form);
const location = res.headers.get("location");
return {
status: res.status,
contentType: res.headers.get("content-type") ?? "",
...withDefined("contentDisposition", res.headers.get("content-disposition")),
...withDefined("location", location === null ? null : stripQuery(location, this.#url)),
body: new Uint8Array(await res.arrayBuffer()),
};
}
async #headersNow(): Promise<Headers> {
let headers = this.#headers;
for (const map of this.#maps) {
const entries = await map.entries();
for (const [key, value] of entries) {
// key name only: header values must not appear in Error messages
if (typeof value !== "string") throw new Error(`headers: ${key} is not a string`);
}
headers = unionHeaders(headers, Object.fromEntries(entries) as Headers);
}
return headers;
}
}
@@ -149,11 +245,16 @@ class Gate {
* storage, surviving eviction (instance memory evaporates when the DO naps;
* storage is where "this instance means this URL" durably lives).
*
* Its cookie, when minted by {@link mintInstance}, is not in `init` but in
* the header store, whose id goes to the account owner and not to whoever
* holds the instance id: the owner can rotate it, and the holder cannot
* swap in another account's session.
*
* @see https://developers.cloudflare.com/durable-objects/ — newUniqueId is
* unforgeable (128-bit, inderivable); idFromName by contrast is ambient:
* any worker with the namespace binding can recompute it
*/
export class WebAppDo extends DurableObject<{ WEBAPP?: never }> {
export class WebAppDo extends DurableObject<WebAppEnv> {
#ready?: Promise<void>;
/** one-shot configuration; a different-url re-configure is refused */
@@ -207,16 +308,46 @@ export class WebAppDo extends DurableObject<{ WEBAPP?: never }> {
}
return res.text();
};
return new WebApp(init.url, init.headers ?? {}, send);
const exchange: Exchange = async (url, headers, method, body) => {
await gate.waitMyTurn();
return fetch(url, { method, headers, body, redirect: "manual" });
};
const maps = init.headerStore === undefined ? [] : [this.#readHeaderStore(init.headerStore)];
return new WebApp(init.url, init.headers ?? {}, maps, send, exchange);
}
/** a header map reading the store's read-only view, afresh at each send */
#readHeaderStore(storeId: string): HeaderMap {
const ns = this.env.MAP_STORE_DO;
const store = ns.get(ns.idFromString(storeId));
return {
async entries() {
using rd = await store.readOnly();
return await rd.entries();
},
};
}
}
/** split the cookie (any case of the header name) from the other headers */
function splitCookie(headers: Headers): { cookie?: string; rest: Headers } {
const rest: Headers = {};
let cookie: string | undefined;
for (const [key, value] of Object.entries(headers)) {
if (key.toLowerCase() === "cookie") cookie = value;
else rest[key] = value;
}
return { cookie, rest };
}
type WebAppEnv = {
WEBAPP_DO: DurableObjectNamespace<WebAppDo>;
MAP_STORE_DO: DurableObjectNamespace<MapStoreDo>;
};
/** the worker's whole HTTP surface (the default export and the
* WebAppFactory entrypoint share it) */
const defaultFetch = async (
request: Request,
env: { WEBAPP_DO: DurableObjectNamespace<WebAppDo> },
): Promise<Response> => {
const defaultFetch = async (request: Request, env: WebAppEnv): Promise<Response> => {
const url = new URL(request.url);
// mint form: site-agnostic; secret headers (cookie, authorization) are
@@ -227,7 +358,7 @@ const defaultFetch = async (
// mint: browser form POST (form-encoded) or JSON (service-style)
if (url.pathname === "/" && request.method === "POST") {
return mint(request, env.WEBAPP_DO);
return mint(request, env);
}
// the mint form, renderable for a proxying worker: ?render=1 keeps the
@@ -246,6 +377,14 @@ const defaultFetch = async (
return drive(await app, request);
}
// a header store, by its id (the owner's, not the instance's): PUT
// JSON value sets a key (e.g. rotates the cookie), DELETE removes it
const entry = url.pathname.match(/^\/store\/([A-Za-z0-9-]+)\/([^/]+)$/);
if (entry && (request.method === "PUT" || request.method === "DELETE")) {
const store = env.MAP_STORE_DO.get(env.MAP_STORE_DO.idFromString(entry[1]));
return storeEntry(store, decodeURIComponent(entry[2]), request);
}
return new Response("not found", { status: 404 });
};
@@ -305,10 +444,7 @@ function withCookie(init: WebAppInit, cookie: string): WebAppInit {
}
/** mint a fresh instance from form-encoded or JSON input */
async function mint(
request: Request,
ns: DurableObjectNamespace<WebAppDo>,
): Promise<Response> {
async function mint(request: Request, env: WebAppEnv): Promise<Response> {
const ct = request.headers.get("content-type") ?? "";
let init: WebAppInit;
try {
@@ -335,13 +471,35 @@ async function mint(
return Response.json({ error: "bad url" }, { status: 400 });
}
const id = ns.newUniqueId();
const instance = ns.get(id);
await instance.configure(init);
const app = await instance.handle();
// note: id intentionally NOT logged — it is a bearer capability
const { id, storeId } = await mintInstance(env, init);
// note: ids intentionally NOT logged — they are bearer capabilities
console.log({ event: "mint", msPerCall: init.msPerCall });
return Response.json({ id: id.toString() }, { headers: { "x-webapp-id": id.toString() } });
return Response.json({ id, storeId }, { headers: { "x-webapp-id": id, "x-webapp-store-id": storeId } });
}
/**
* a configured instance plus its header store, shaped to hold just the
* cookie: the instance id is authority to use the cookie; the store id
* (for the account owner) is authority to rotate or remove it. Any
* `cookie` header in `init` goes to the store, not to the instance.
*/
async function mintInstance(env: WebAppEnv, init: WebAppInit): Promise<{ id: string; storeId: string }> {
const { cookie, rest } = splitCookie(init.headers ?? {});
const storeId = env.MAP_STORE_DO.newUniqueId();
const store = env.MAP_STORE_DO.get(storeId);
await store.configure("cookie", encodeShapes({ keyShape: "cookie", valueShape: M.string() }));
if (cookie !== undefined) {
using full = await store.store();
await full.init("cookie", cookie);
}
const id = env.WEBAPP_DO.newUniqueId();
await env.WEBAPP_DO.get(id).configure({
url: init.url,
headers: rest,
...(init.msPerCall === undefined ? {} : { msPerCall: init.msPerCall }),
headerStore: storeId.toString(),
});
return { id: id.toString(), storeId: storeId.toString() };
}
/** drive a handle from the /at/<id> JSON protocol (also used by the
@@ -368,6 +526,60 @@ async function drive(app: FluentGet, request: Request): Promise<Response> {
}
}
/** the /store/<id>/<key> protocol: PUT a JSON value (set, or init if
* absent), or DELETE → {ok} (or {error}; messages carry no values) */
async function storeEntry(store: DurableObjectStub<MapStoreDo>, key: string, request: Request): Promise<Response> {
let value: unknown;
if (request.method === "PUT") {
try {
value = await request.json();
} catch (err) {
return Response.json({ error: `bad input: ${String(err)}` }, { status: 400 });
}
}
try {
using full = await store.store();
if (request.method === "DELETE") {
if (await full.has(key)) await full.delete(key);
} else if (await full.has(key)) {
await full.set(key, value);
} else {
await full.init(key, value);
}
} catch (err) {
// store errors name shapes, never keys or values (see lib/src/store.ts)
return Response.json({ error: String(err) }, { status: 400 });
}
// note: neither id nor value logged
console.log({ event: "store", method: request.method });
return Response.json({ ok: true });
}
/** a cookie header's name → value (the first of a repeated name wins) */
function parseCookies(header: string): Map<string, string> {
const jar = new Map<string, string>();
for (const pair of header.split(";")) {
const eq = pair.indexOf("=");
if (eq < 0) continue;
const name = pair.slice(0, eq).trim();
if (name && !jar.has(name)) jar.set(name, pair.slice(eq + 1).trim());
}
return jar;
}
/** `ref` resolved against `base`, query and fragment dropped: they can carry credentials */
function stripQuery(ref: string, base: string): string {
const url = new URL(ref, base);
url.search = "";
url.hash = "";
return url.href;
}
/** `{[key]: value}`, or `{}` when value is null: optional fields stay absent */
function withDefined<K extends string>(key: K, value: string | null): { [P in K]?: string } {
return (value === null ? {} : { [key]: value }) as { [P in K]?: string };
}
function unionHeaders(base: Headers, more: Headers): Headers {
const merged = { ...base };
for (const [key, value] of Object.entries(more)) {
@@ -396,37 +608,37 @@ function unionQuery(base: string, params: Query): string {
/**
* the RPC service of the webapp worker. Flow:
*
* const { id, app } = await env.WEBAPP.newWebApp({
* url: "https://api.github.com", headers: { authorization: `Bearer ${t}` },
* const { id, storeId, app } = await env.WEBAPP.newWebApp({
* url: "https://diigo.example/", headers: { cookie: c },
* msPerCall: 250,
* });
* // hold id (persist it, hand it to workers you trust), then e.g.:
* const text = await app.query({ per_page: 5 }).get();
* // later, from another worker (same account) holding the id:
* const app2 = env.WEBAPP.at(id);
* // the account owner, holding storeId, after logging in again:
* await (await env.WEBAPP.store(storeId)).set("cookie", c2);
*/
export class WebAppFactory extends WorkerEntrypoint {
/** the same HTTP surface as the default export, over the entrypoint
* (for service-binding callers that proxy pages, e.g. diigo-cache's
* /_connect → this form) */
async fetch(request: Request): Promise<Response> {
const env = this.env as { WEBAPP_DO: DurableObjectNamespace<WebAppDo> };
return defaultFetch(request, env);
return defaultFetch(request, this.env);
}
/** mint a fresh instance: unforgeable uniqueId + configured fluent stub.
* The caller decides who else ever learns the id. (If the caller only
/** mint a fresh instance: unforgeable uniqueId + configured fluent stub,
* plus the id of its header store (see {@link mintInstance}). The
* caller decides who else ever learns either id. (If the caller only
* wanted the id, it must dispose() the app stub — workerd warns about
* undisposed stubs; see the `at()` alternative + `/at/<id>` route for
* id-only flows.) */
async newWebApp(init: WebAppInit): Promise<{ id: string; app: FluentGet }> {
async newWebApp(init: WebAppInit): Promise<{ id: string; storeId: string; app: FluentGet }> {
new URL(init.url); // fail before minting on garbage
const { id, storeId } = await mintInstance(this.env, init);
const ns = this.env.WEBAPP_DO;
const id = ns.newUniqueId();
const instance = ns.get(id);
await instance.configure(init);
const app = await instance.handle();
return { id: id.toString(), app };
const app = await ns.get(ns.idFromString(id)).handle();
return { id, storeId, app };
}
/** reconnect by a previously-obtained id */
@@ -434,6 +646,13 @@ export class WebAppFactory extends WorkerEntrypoint {
const ns = this.env.WEBAPP_DO;
return ns.get(ns.idFromString(id)).handle();
}
/** a header store by its id, with full authority (the instance id
* reaches none: see {@link mintInstance}) */
async store(storeId: string) {
const ns = this.env.MAP_STORE_DO;
return ns.get(ns.idFromString(storeId)).store();
}
}
function sleep(ms: number): Promise<void> {
return new Promise((r) => setTimeout(r, ms));
+131
View File
@@ -0,0 +1,131 @@
import { env } from "cloudflare:workers";
import { runInDurableObject } from "cloudflare:test";
import { afterEach, describe, expect, it, vi } from "vitest";
import worker, { type WebAppInit } from "../src/index.js";
type TestEnv = {
WEBAPP_DO: DurableObjectNamespace<import("../src/index.js").WebAppDo>;
MAP_STORE_DO: DurableObjectNamespace<import("../src/index.js").MapStoreDo>;
};
const testEnv = env as unknown as TestEnv;
const URL_ = "https://diigo.example/api/";
/** the cookie header of each outbound fetch */
function spyCookies(): (string | null)[] {
const seen: (string | null)[] = [];
vi.spyOn(globalThis, "fetch").mockImplementation(async (_input, init) => {
seen.push(new Headers(init?.headers).get("cookie"));
return new Response("ok");
});
return seen;
}
afterEach(() => {
vi.restoreAllMocks();
});
/** the worker's HTTP surface, as a JSON client */
const call = (path: string, method: string, body?: unknown) =>
worker.fetch(
new Request(`https://webapp.example${path}`, {
method,
body: body === undefined ? undefined : JSON.stringify(body),
headers: { "content-type": "application/json" },
}),
testEnv,
);
const mint = async (init: WebAppInit) => {
const { id, storeId } = (await (await call("/", "POST", init)).json()) as { id: string; storeId: string };
const instance = testEnv.WEBAPP_DO.get(testEnv.WEBAPP_DO.idFromString(id));
return { id, storeId, instance };
};
/** the failure of a call: its error status, or what it threw; "" if it succeeds */
async function errorOf(response: Promise<Response>): Promise<string> {
try {
const res = await response;
return res.ok ? "" : `${res.status} ${await res.text()}`;
} catch (err) {
return String(err);
}
}
/** `init` as stored */
const storedInit = (stub: DurableObjectStub) =>
runInDurableObject(stub, (_obj, state) => state.storage.kv.get<WebAppInit>("init"));
describe("cookie rotation", () => {
it("mint puts the cookie in the header store, not init; sends carry it", async () => {
const seen = spyCookies();
const { storeId, instance } = await mint({
url: URL_,
headers: { Cookie: "session=a", accept: "application/json" },
});
expect(await storedInit(instance)).toEqual({
url: URL_,
headers: { accept: "application/json" },
headerStore: storeId,
});
using app = await instance.handle();
await app.get();
expect(seen).toEqual(["session=a"]);
});
it("the store id rotates under handles already out, and revokes", async () => {
const seen = spyCookies();
const { storeId, instance } = await mint({ url: URL_, headers: { cookie: "session=a" } });
using app = await instance.handle();
await app.pathjoin("x").get();
expect(await (await call(`/store/${storeId}/cookie`, "PUT", "session=b")).json()).toEqual({ ok: true });
await app.pathjoin("x").get();
expect(await (await call(`/store/${storeId}/cookie`, "DELETE")).json()).toEqual({ ok: true });
await app.get();
expect(seen).toEqual(["session=a", "session=b", null]);
});
it("the instance id reaches no store: its holder cannot swap in another session", async () => {
const seen = spyCookies();
const { id, instance } = await mint({ url: URL_, headers: { cookie: "session=alice" } });
expect(await errorOf(call(`/store/${id}/cookie`, "PUT", "session=mallory"))).not.toBe("");
expect((await call(`/at/${id}/cookie`, "POST", { cookie: "session=mallory" })).status).toBe(404);
using app = await instance.handle();
await app.get();
expect(seen).toEqual(["session=alice"]);
});
it("the store's shapes admit only the cookie; errors carry no values", async () => {
const { storeId } = await mint({ url: URL_ });
const err = await errorOf(call(`/store/${storeId}/authorization`, "PUT", "Bearer s3kr1t"));
expect(err).toContain('cookie: key: must match \\"cookie\\"');
expect(err).not.toContain("s3kr1t");
expect(await errorOf(call(`/store/${storeId}/cookie`, "PUT", 42))).toContain("must match M.string()");
});
it("an instance minted before header stores keeps its cookie in init, untouched", async () => {
const seen = spyCookies();
const instance = testEnv.WEBAPP_DO.get(testEnv.WEBAPP_DO.newUniqueId());
const old = { url: URL_, headers: { cookie: "session=old" }, msPerCall: 0 };
await runInDurableObject(instance, (_obj, state) => state.storage.put("init", old));
using app = await instance.handle();
await app.get();
expect(seen).toEqual(["session=old"]);
expect(await storedInit(instance)).toEqual(old);
});
it("a fixed header contradicting the stored cookie fails at send, naming no values", async () => {
spyCookies();
const { instance } = await mint({ url: URL_, headers: { cookie: "session=a" } });
using app = await instance.handle();
const err = await app
.withHeader("cookie", "session=mine")
.get()
.then(
() => "",
(e: unknown) => String(e),
);
expect(err).toContain("headers: contradicts on cookie");
expect(err).not.toContain("session=");
});
});
+117
View File
@@ -0,0 +1,117 @@
import { env } from "cloudflare:workers";
import { afterEach, describe, expect, it, vi } from "vitest";
import worker, { type WebAppInit } from "../src/index.js";
type TestEnv = {
WEBAPP_DO: DurableObjectNamespace<import("../src/index.js").WebAppDo>;
MAP_STORE_DO: DurableObjectNamespace<import("../src/index.js").MapStoreDo>;
};
const testEnv = env as unknown as TestEnv;
const URL_ = "https://site.example/app/export/7/data";
type Seen = { url: string; redirect?: string; contentType: string | null; form: Record<string, string> };
/** each outbound fetch, its form fields read back; answers with `res()` */
function spyForms(res: () => Response): Seen[] {
const seen: Seen[] = [];
vi.spyOn(globalThis, "fetch").mockImplementation(async (input, init) => {
const req = new Request(input as string, init as RequestInit);
const form = Object.fromEntries([...(await req.formData()).entries()].map(([k, v]) => [k, String(v)]));
seen.push({ url: req.url, redirect: init?.redirect, contentType: req.headers.get("content-type"), form });
return res();
});
return seen;
}
afterEach(() => {
vi.restoreAllMocks();
});
const mint = async (init: WebAppInit) => {
const res = await worker.fetch(
new Request("https://webapp.example/", {
method: "POST",
body: JSON.stringify(init),
headers: { "content-type": "application/json" },
}),
testEnv,
);
const { id, storeId } = (await res.json()) as { id: string; storeId: string };
return { id, storeId, instance: testEnv.WEBAPP_DO.get(testEnv.WEBAPP_DO.idFromString(id)) };
};
/** the failure of a call: what it threw; "" if it succeeds */
async function errorOf(p: Promise<unknown>): Promise<string> {
try {
await p;
return "";
} catch (err) {
return String(err);
}
}
describe("postForm", () => {
it("echoes a cookie from the header store into a form field; the holder never sees it", async () => {
const seen = spyForms(
() =>
new Response("col1,col2", {
headers: {
"content-type": "text/csv",
"content-disposition": 'attachment;filename="x.csv"',
"set-cookie": "sess=next; HttpOnly",
},
}),
);
const { instance } = await mint({ url: URL_, headers: { cookie: "a=1; tok=s3cret; b=2" } });
using app = await instance.handle();
const reply = await app.query({ since: "2026-07-13\x1f" }).postForm({ kind: "all" }, { csrf: "tok" });
expect(seen).toEqual([
{
url: `${URL_}?since=2026-07-13%1F`,
redirect: "manual",
contentType: expect.stringMatching(/^multipart\/form-data; boundary=/),
form: { kind: "all", csrf: "s3cret" },
},
]);
expect(reply).toEqual({
status: 200,
contentType: "text/csv",
contentDisposition: 'attachment;filename="x.csv"',
body: new TextEncoder().encode("col1,col2"),
});
});
it("does not follow a redirect: the 3xx comes back, location query-stripped", async () => {
spyForms(() => new Response(null, { status: 302, headers: { location: "/login?next=s3cret#x" } }));
const { instance } = await mint({ url: URL_, headers: { cookie: "tok=t" } });
using app = await instance.handle();
const reply = await app.postForm({}, { csrf: "tok" });
expect(reply).toEqual({ status: 302, contentType: "", location: "https://site.example/login", body: new Uint8Array() });
});
it("refuses, naming no values: a missing cookie, a field set twice, a content-type header", async () => {
spyForms(() => new Response("ok"));
const { instance } = await mint({ url: URL_, headers: { cookie: "tok=s3cret" } });
using app = await instance.handle();
expect(await errorOf(app.postForm({}, { csrf: "nope" }))).toMatch(/no nope cookie to echo/);
const twice = await errorOf(app.postForm({ csrf: "mine" }, { csrf: "tok" }));
expect(twice).toMatch(/contradicts on csrf/);
expect(twice).not.toMatch(/s3cret|mine/);
const typed = app.withHeader("content-type", "text/plain");
expect(await errorOf(typed.postForm())).toMatch(/content-type is set by the form/);
});
it("rotating the store's cookie changes what is echoed", async () => {
const seen = spyForms(() => new Response("ok"));
const { storeId, instance } = await mint({ url: URL_, headers: { cookie: "tok=one" } });
using app = await instance.handle();
await app.postForm({}, { csrf: "tok" });
await worker.fetch(
new Request(`https://webapp.example/store/${storeId}/cookie`, { method: "PUT", body: JSON.stringify("tok=two") }),
testEnv,
);
await app.postForm({}, { csrf: "tok" });
expect(seen.map((s) => s.form.csrf)).toEqual(["one", "two"]);
});
});
+7
View File
@@ -0,0 +1,7 @@
{
"extends": "../tsconfig.json",
"compilerOptions": {
"types": ["@cloudflare/vitest-plugin/types"]
},
"include": ["./**/*", "../worker-configuration.d.ts"]
}
+1
View File
@@ -7,6 +7,7 @@
"esModuleInterop": true,
"verbatimModuleSyntax": true,
"skipLibCheck": true,
"lib": ["ES2022", "ESNext.Disposable"],
"noEmit": true,
"types": []
},
+9
View File
@@ -0,0 +1,9 @@
import { defineProject } from "vitest/config";
import { cloudflareTest } from "@cloudflare/vitest-plugin";
export default defineProject({
plugins: [cloudflareTest({ wrangler: { configPath: "./wrangler.jsonc" } })],
test: {
include: ["tests/**/*.test.ts"],
},
});
+8 -2
View File
@@ -5,7 +5,13 @@
"compatibility_date": "2026-09-29",
"observability": { "enabled": true },
"durable_objects": {
"bindings": [{ "name": "WEBAPP_DO", "class_name": "WebAppDo" }]
"bindings": [
{ "name": "WEBAPP_DO", "class_name": "WebAppDo" },
{ "name": "MAP_STORE_DO", "class_name": "MapStoreDo" }
]
},
"migrations": [{ "tag": "v1", "new_sqlite_classes": ["WebAppDo"] }]
"migrations": [
{ "tag": "v1", "new_sqlite_classes": ["WebAppDo"] },
{ "tag": "v2", "new_sqlite_classes": ["MapStoreDo"] }
]
}