Compare commits

..
24 Commits
Author SHA1 Message Date
dckc 1d8a220bbc capsite: checklist: 2-iv, the weekly draft works in dev
Generated-by: Claude Opus 5.5
2026-10-09 02:12:41 -05:00
dckc 68132ff4b3 diigo-week: a dashboard card drafts last week in one click
The card shows last week's issue with a link to it, or a button to
draft it. Drafting a week that already has an issue (draft or
published) is refused, with a link to that issue, rather than making
a duplicate.

Generated-by: Claude Opus 5.5
2026-10-09 02:12:41 -05:00
dckc 9c09e8c5a4 capsite: checklist: 4b, consider scruffier tags
Generated-by: Claude Opus 5.5
2026-10-09 02:12:41 -05:00
dckc 94a059af25 diigo-week: draft a potluck issue from a week of diigo bookmarks
An EmDash plugin (native, content:write only) with one admin page:
a diigo username and an ISO week. It reads diigo-cache's public page
(?in=<week>&by=created) as microformats, follows rel=next, and
creates a draft in issues shaped like seed/issues.json.

Transport is a DIIGO_CACHE service binding rather than ctx.http.fetch:
EmDash's SSRF guard refuses localhost, and the binding names one
worker. Logic lives in src/week.ts with the fetcher passed in; tests
read fixtures in diigo-cache's markup. The checklist's goal 2 now
names potluck; DiigoBridge waits.

Generated-by: Claude Opus 5.5
2026-10-09 02:12:41 -05:00
dckc be796d9d0d diigo-cache: the public page is an h-feed of h-entry bookmarks
Microformats are the public page's API for other sites (potluck
first): each bookmark is an h-entry with u-bookmark-of, p-name,
p-summary, a p-category per tag (linked to a tag search),
dt-published (bookmarked) and dt-updated (last edited). Prev/next
links carry rel=prev/next, so a reader can follow a filter's pages.
Tests parse the page with microformats-parser to pin the contract.

Generated-by: Claude Opus 5.5
2026-10-09 01:49:40 -05:00
dckc c66aaea366 potluck, diigo-cache: docs one line per paragraph; potluck deploy caveat
Obsidian and GitHub render each newline as a line break, so the
hard-wrapped READMEs and CONTRIBUTINGs rendered ragged. Whitespace
only, except a note in potluck's CONTRIBUTING that other workerd
hosts are untested: they'd have to supply the D1 and R2 bindings.

Generated-by: Claude Opus 5.5
2026-10-09 01:30:56 -05:00
dckc a25bd699e1 potluck: README: workerd, not a vendor
Generated-by: Claude Opus 5.5
2026-10-09 01:29:31 -05:00
dckc ea3e256bb8 diigo-cache: README: workerd, not a vendor
Generated-by: Claude Opus 5.5
2026-10-09 01:28:50 -05:00
dckc 0af7a3ea6c diigo-cache: README sells and gets to TADA; CONTRIBUTING updated
New README: two sentences, the dev run through connect, fill, and a
filtered public page, then features. CONTRIBUTING gains the 1d
filters and its own run and deploy sections instead of pointing at
the checklist; links to the old README now point at CONTRIBUTING.

Generated-by: Claude Opus 5.5
2026-10-09 01:27:45 -05:00
dckc 9719fd0aa8 diigo-cache: README becomes CONTRIBUTING; AGENTS.md links to it
Generated-by: Claude Opus 5.5
2026-10-09 01:27:12 -05:00
dckc bce2424cc0 potluck: README: a heading that says something
Generated-by: Claude Opus 5.5
2026-10-09 01:25:47 -05:00
dckc 7f1e19fe85 potluck: README: two sentences, then getting started, then features
Generated-by: Claude Opus 5.5
2026-10-09 01:25:13 -05:00
dckc 8a812d76e3 potluck: README sells and gets to TADA; CONTRIBUTING has the rest
The README was EmDash's template blurb. Now it says what potluck is
and its first block runs from a fresh clone to /issues, using a new
`pnpm seed:issues` script that applies seed/issues.json to the local
D1 database. CONTRIBUTING covers how it's put together, seeds, checks,
the EmDash rules, and deploying; the template's stale notes (Inter,
three-column articles) are gone. AGENTS.md is now a symlink to it.

Generated-by: Claude Opus 5.5
2026-10-09 01:24:10 -05:00
dckc 002f08b456 diigo-cache: date windows and search on the public page (1d)
/public/<username> takes days, after, before, in (year, month,
quarter, ISO week), by=updated|created, and q (words, "phrases",
#tags, -negation, gmail/Slack/GitHub date operators). JSON answers
echo the parsed filter; a malformed one is a 400.

Filters query the stored items in sqlite through the read-only
viewer instead of walking diigo pages: the public page keeps no
fetch authority, and a filled cache already holds every item.
Results say `complete` when every diigo page is stored; freshness
is 1f's head refresh. So the fetch budget and the O(log pages)
walk test go, and the fake deep account shrinks to 300.

Generated-by: Claude Opus 5.5
2026-10-09 01:17:03 -05:00
dckcandClaude Opus 5.5 efb54ad4cc docs: Good Fences Make Good Agents (design story + IBIS)
The story behind good-fences: lend a governed mower over the fence to a
neighbor whose tenants are model-written code, with the design IBIS
worked out so far. Previously an untracked note at the workspace root.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-09 01:00:15 -05:00
dckcandClaude Opus 5.5 8c110098ff feat(good-fences): a Think agent mows with a codemode mower connector
Neighbor is a Project Think agent on gpt-oss-120b. Its execute tool
runs model-written code in a dynamic worker; MowerConnector puts the
mower there as the `mower.*` namespace (start/push/turn/stop, each
taking an args object). makeMower is the Logo-turtle mower on a
wrapping yard; for now it lives inside the agent's DO.

Getting to green, one symptom at a time:
- chatted instead of acting: say it acts by running code
- reply cut mid-sentence: maxOutputTokens 4096 via beforeTurn
- execute failed creating codemode's runtime facet ("Incorrect type
  for the 'class' field"): declare CodemodeRuntime as a DO class
- looked for the mower in the workspace filesystem: activeTools
  ["execute"], and name the namespace in the prompt
- found the mower, then asked for lawn dimensions: "don't ask me
  questions: put it to use"

Green: 36 mower calls, a 30-cell rectangle.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-09 00:35:56 -05:00
dckcandClaude Opus 5.5 d61cb8c9a1 test(good-fences): lent a mower, a Think agent mows (red)
The first end-to-end test: a Project Think agent, on the real model,
writes code that runs in a dynamic worker and drives a mower; the test
counts the cells cut, and saves the turn to itest-output/.

Installs @cloudflare/think, @cloudflare/codemode, agents, ai,
workers-ai-provider, zod. The model-test worker now has a main entry,
the Neighbor DO, and a LOADER binding.

Red: src/worker.ts doesn't exist yet.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-09 00:31:25 -05:00
dckcandClaude Opus 5.5 a2ca59a27e feat(good-fences): code-mode prompt; the model writes code to mow
codeModeMessages turns what I say plus `about` into a prompt: a
neighbor persona, rules for the reply (one async arrow function over
`env`), and the lent things' types. Things are remote and calls on one
are delivered in order, so a lamp example shows pipelining: await only
for a value, or on a batch's last call. With only the rule, the model
awaited every call; with the example, it pipelines.

aboutMower gives the Mower type with Promise returns. The model test
checks the code calls only Mower methods, pushes, and awaits less often
than it calls; it saves the code to itest-output/ via a reporter, since
the test runs in workerd, which can't write files. Model tests get 60s.

Green: both model tests.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-09 00:21:44 -05:00
dckcandClaude Opus 5.5 4a85143a66 fix(good-fences): give the model room to answer
gpt-oss-120b reasons before it answers, and Workers AI's default
max_tokens (256) ran out mid-thought: finish_reason "length", content
null. Ask for 4096, and when there's no reply text, say which choice
and message keys came back instead.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-09 00:21:44 -05:00
dckcandClaude Opus 5.5 37faa92649 test(good-fences): lent a mower, the model writes code to mow (red)
Code mode (2c): the harness turns what I say plus `about` into a prompt;
the model replies with `(env) => ...` code. Checked textually for now,
since workerd won't eval it: the code pushes `env.mower` and calls only
methods in `about`. It also prints the code, to see what the model writes.

Red: src/code-mode.ts doesn't exist yet.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-09 00:01:39 -05:00
dckcandClaude Opus 5.5 ec24f3e8e4 feat(good-fences): complete(messages) on a Workers AI model
makeWorkersAiModel narrows the AI binding to one model's chat
completion: the harness gets `complete`, not `run` on any model at
any cost. gpt-oss-120b replies in the OpenAI shape (checked with a
one-off probe), so the text is choices[0].message.content; anything
else throws, naming the keys it got.

Green: the model counts 3 rs in strawberry.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-08 23:41:18 -05:00
dckcandClaude Opus 5.5 47a2337afe test(good-fences): the model counts the rs in strawberry (red)
A first integration test of the model call: Workers AI through the AI
binding, behind the same `complete(messages)` interface as the sim's
fake model. Opt-in (`pnpm test:model`) since it needs `wrangler login`
and costs money.

Red: src/model.ts doesn't exist yet.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-08 23:36:46 -05:00
dckcandClaude Opus 5.5 9b12943be4 test(good-fences): show which mower the neighbor gets
The chat and load arrows drew `things: { mower }` whether I lent the
raw mower or the governed one. Name objects by the reply arrow that
introduced them, so the diagram reads `things: { mower: governed }`
and lending the raw mower would fail the trace comparison.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-08 23:27:13 -05:00
dckcandClaude Opus 5.5 9a3e7348c6 test(good-fences): lend-the-mower journey (red)
The spec is a sequence diagram of the a2030.md story: I lend a governed
mower; the neighbor's harness shows the model only `about`, runs its
code as a tenant, and the governor stops the tipsy tenant at a 2-cell
fence. The test runs the actors through a causal tracer and compares
the trace with the diagram, then checks the yard and what the model saw.

The tracer comes from agoric-sdk PR #12895, extended to name
participants made mid-sequence (governor, tenant) and to draw thrown
errors as replies.

Red: src/ doesn't exist yet (ERR_MODULE_NOT_FOUND for src/fence.ts).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-08 22:17:48 -05:00
52 changed files with 13665 additions and 424 deletions

No files matched your search

+19 -34
View File
@@ -23,7 +23,7 @@ narrative.
goal 5. goal 5.
- [x] **1c. diigo-dump evolves into a bookmark cache** - [x] **1c. diigo-dump evolves into a bookmark cache**
([diigo-cache/README.md](diigo-cache/README.md); design in the ([diigo-cache/CONTRIBUTING.md](diigo-cache/CONTRIBUTING.md); design in the
JSDoc of `src/cache.ts`, `src/index.ts`, `src/pm-save.ts`): JSDoc of `src/cache.ts`, `src/index.ts`, `src/pm-save.ts`):
- [x] 1c-i. Two independent page caches, private and public - [x] 1c-i. Two independent page caches, private and public
(`PagedCache`, `DiigoCache`, `PublicCache`). (`PagedCache`, `DiigoCache`, `PublicCache`).
@@ -42,20 +42,16 @@ narrative.
2026-10-07, via `scripts/compare-export.ts`. Getting there 2026-10-07, via `scripts/compare-export.ts`. Getting there
found diigo's `page_num` is 0-based (schema v2 migration). found diigo's `page_num` is 0-based (schema v2 migration).
- [ ] **1d. Date windows and search on the public page**, toward the - [x] **1d. Date windows and search on the public page**, toward the
weekly PESOS roundup of [#23](https://github.com/dckc/madmode-blog/issues/23) weekly PESOS roundup of [#23](https://github.com/dckc/madmode-blog/issues/23)
(a week's permalink: `in=2026-W41`). One filter, two spellings: (a week's permalink: `in=2026-W41`). One filter, two spellings:
devs pass `{ days: 7, q: "#weekly" }`, URLs the same fields; devs pass `{ days: 7, q: "#weekly" }`, URLs the same fields;
`q` takes gmail/Slack/GitHub operators. UTC, ISO weeks, half-open `q` takes gmail/Slack/GitHub operators. UTC, ISO weeks, half-open
windows; a fetch budget per request. Design: the API in windows (`diigo-cache/src/query.ts`; `tests/query.test.ts`,
`diigo-cache/src/query.ts` and the skipped suites in `tests/caches.test.ts` "public page filters"). Filters read
`tests/query.test.ts` and `tests/caches.test.ts` ("public page stored `items` only (the public page never fetches): results are
filters"); done when they're unskipped and green. as fresh as the cache, and say `complete` when every diigo page
Reconcile with 1f first: `query.ts` plans to filter while walking is stored. Keeping them fresh is 1f. 2026-10-09.
diigo's pages (binary search by `updated`), but after 1c-vi a
filled cache holds every item, so 1d can query `items` in sqlite
and spend diigo fetches only on a head refresh (1f). Choose
before implementing; the skipped suites may need adjusting.
- [x] **1e. Fill job: a button that grabs every page** into a cache - [x] **1e. Fill job: a button that grabs every page** into a cache
from the webkey page `/cache/#<key>`, which trades the key for a from the webkey page `/cache/#<key>`, which trades the key for a
@@ -93,31 +89,20 @@ narrative.
[the issues sketch](potluck/issues-sketch.md))? If the policy [the issues sketch](potluck/issues-sketch.md))? If the policy
changes, stripping stored rows is a durable-state migration. changes, stripping stored rows is a durable-state migration.
- [ ] **2. madmode reads the public cache**: public data, so no - [ ] **2. potluck reads the public cache**: public data, so no capability, no cookie, no secret on potluck's side. The [diigo-week](diigo-week/README.md) plugin reads `GET /public/dckc-madmode?in=<week>&by=created` as microformats (an `h-feed` of `h-entry` bookmarks). Working in dev 2026-10-09; 2-iii postponed.
capability, no cookie, no plugin sync job, no cron on madmode's - [x] 2-i. Transport: the `DIIGO_CACHE` service binding in potluck's `wrangler.jsonc`, in dev too: the plugin's `ctx.http.fetch` refuses localhost (EmDash's SSRF guard), and a binding names one worker, not a host.
side. madmode fetches `GET /public/dckc-madmode` with - [x] 2-ii. The page's microformats are the contract: diigo-cache's tests parse it with microformats-parser; diigo-week's tests read fixtures in the same markup.
`Accept: application/json` (plus 1d's `in=`/`days=`/`q=`). - [ ] 2-iii. Decide `DiigoBridge`'s fate (postponed: nothing on this path uses it; it matters only for a caller that reads private items).
- [ ] 2-i. Transport: a service binding to diigo-cache when both - [x] 2-iv. Run it in dev: diigo-cache and potluck both up; the dashboard card → a draft issue of 22 bookmarks for 2026-W40 (2026-10-09). The service binding resolves across the two dev servers.
are deployed (no public hop, so no rate-limit exposure); - [ ] ~~**3. `/bookmarks` renders real data**~~: superseded by potluck's issues (the madmode starter is gone).
in dev, a base URL (`http://localhost:8787`). - [ ] **4. The weekly draft**: decided (2026-10-09): a draft issue you edit before publishing, made by diigo-week. The slug comes from the title (`week-2026-w40`).
- [ ] 2-ii. The JSON shape of the public page is a contract
between two workers: pin it in a diigo-cache test (and - [ ] **4b. Consider scruffier tags.** EmDash tags are taxonomy terms (rows that must exist before use); diigo-week auto-creates them, so drafting feels scruffy, but the model is neat. Alternative: the inline `/tag/<tag>` links in issue bodies *are* the tags (matching the microformats `p-category` reading), and potluck's `/tag/<tag>` page finds issues by scanning bodies. Cost: no EmDash tag field or index; a scan over a few dozen issues a year is cheap. Open: does the admin's tag field let you type a new tag inline?
madmode's reader tests against the same fixture).
- [ ] 2-iii. Decide `DiigoBridge`'s fate: madmode doesn't need it
(public reads need no cache id), so its "cache-id source"
TODO (`src/index.ts`, README "RPC use") is either dropped
with the entrypoint or kept for a private-reading caller.
- [ ] **3. `/bookmarks` renders real data** — titles, tags, dates at
http://localhost:4321/bookmarks, newest first.
- [ ] **4. The weekly page**: `/bookmarks/2026-W41` → the cache's
`in=2026-W41` (needs 1d and 1f). Decide (user's call): a raw
week list, or a draft post the user edits before publishing
(the PESOS practice is editorial; a tag like `#weekly` in `q`
is one cheap form of discretion).
- [ ] **5. Deployed end-to-end** (order matters): - [ ] **5. Deployed end-to-end** (order matters):
- [ ] 5-i. webapp: `cd webapp && npx wrangler deploy` - [ ] 5-i. webapp: `cd webapp && npx wrangler deploy`
- [ ] 5-ii. diigo-cache: rate-limit `/api` (and `/public/*`) first - [ ] 5-ii. diigo-cache: rate-limit `/api` (and `/public/*`) first
(diigo-cache README, "Deployment concern"); deploy; create (diigo-cache CONTRIBUTING, "Deployment concern"); deploy; create
against prod (ids → password manager; no worker secret); against prod (ids → password manager; no worker secret);
run the fill job. run the fill job.
- [ ] 5-iii. madmode: deploy with the service binding; the - [ ] 5-iii. madmode: deploy with the service binding; the
@@ -239,5 +224,5 @@ to every goal in this tree.
Oct 6 2026) Oct 6 2026)
- [webapp/README.md](webapp/README.md) — HTTP + RPC surfaces of the - [webapp/README.md](webapp/README.md) — HTTP + RPC surfaces of the
factory worker; open TODO on the mint-form placement factory worker; open TODO on the mint-form placement
- [diigo-cache/README.md](diigo-cache/README.md) — the cache worker: - [diigo-cache/CONTRIBUTING.md](diigo-cache/CONTRIBUTING.md) — the cache worker:
create flow, the two DOs, run/deploy details create flow, the two DOs, run/deploy details
+1
View File
@@ -0,0 +1 @@
CONTRIBUTING.md
+72
View File
@@ -0,0 +1,72 @@
# Contributing to diigo-cache
diigo-cache is a Cloudflare Worker cache over [the webapp factory](../webapp/README.md): session-cookie reads of diigo bookmarks, cached in Durable-Object SQLite storage. Evolved from the diigo-dump worker (provenance: [mm-diigo `projects/diigo-bak/`](https://github.com/dckc/mm-diigo)); the static API-key path is dead (see [the capsite checklist](../capsite-checklist.md), goal 2 preamble).
## What it holds
- **The create flow, on one port**: `/_connect` renders webapp's own mint form (diigo-prefilled, cookie blank) — wrangler multi-config dev doesn't serve aux workers' HTTP, so `/_mint` relays the form opaquely into the WEBAPP binding (the cookie crosses; nothing on this side opens or logs it). Mint → `/_wire` (webapp id pre-filled, mode public|private) → the created page, where the user saves the ids.
- **Two independent page caches** (`src/cache.ts`), one behavior: a page hit is served from storage with no diigo call; a miss asks the webapp instance in the cache's wiring and etches the page.
- `DiigoCache` (binding `CACHE_DO`): every item. Addressed by its cache id.
- `PublicCache` (binding `PUBLIC_DO`): shared items only — private rows are dropped *before* etching (`publicOnly`), so its storage holds nothing private. Same cookie webapp as the private cache. Public page n is diigo page n, filtered (may show fewer than `PAGE_SIZE`, 100: diigo's cap). `/public/<username>` resolves through the `idFromName("index")` registry, and reads through `viewer()`, a read-only facet: it shows stored pages and fill progress, and can't fetch from diigo or write.
- **Storage**: SQL lives in `src/cache-read.sql` (selects only) and `src/cache-write.sql`; `src/store.ts` turns them into a reader and a writer facet, so the cache logic holds no SQL text.
- **Pages**: home's dump form (cache id + page number, POST) shows raw dump rows; `/public/<username>?page=n` shows an HTML list with a page form (JSON for `Accept: application/json`). Both say `cache: HIT` or `MISS`; every page read also logs `{event: "cache", kind, page, outcome}`.
- **Filters on the public page** (`src/query.ts`): `days`, `after`, `before`, `in` (`2026`, `2026-03`, `2026-Q1`, `2026-W41`), `by=updated|created`, and `q` (words, `"phrases"`, `#tag`, `-` negation, gmail/Slack/GitHub date operators). UTC, half-open windows. The same fields work as a dev-side spec (`toQuery({ days: 7 })`). A filter queries the stored `items` table through `viewer()`, never diigo: results are as fresh as the cache, and JSON says `complete` when every diigo page is stored.
- **Publish later**: `POST /_publish` (cache id in the body) wires a public companion to a private-only cache.
- **The webkey page and Cap'n Web**: `/cache/#<cache id>` is a skeleton page, the same for every key; its script reads the key from the fragment and trades it in-band for a capability over [Cap'n Web](https://github.com/cloudflare/capnweb) at `/api` (`CacheApi.cache(key)` → a `CacheCap` facet: fill, status, the public companion; `src/api.ts`). The worker serves capnweb's browser build itself at `/_capnweb.js`.
- **Fill job**: walks every diigo page into a cache in the background — DO alarms, a few pages per step, progress stored in the DO (`startFill`/`fillStatus`/`stopFill`), driven from the webkey page; the public page shows progress. "Invalidate the last N days" there drops the cached pages holding anything updated that recently (`invalidateRecent`), so the next fill refetches them.
- The `DiigoBridge` entrypoint for other workers (its cache-id source is the goal-2 wiring TODO).
## State surface
Ids are **webkeys** ([the pattern](../docs/webkeys.md)): 256-bit bearer secrets that never ride a query string or path. The webkey page carries the cache id in the URL fragment (never sent to the server) and passes it in-band over Cap'n Web; forms carry ids in POST bodies; logs and pages show only a fingerprint (a prefix of the id's one-way hash).
- **The cache id**: the cache's read capability (bearer; not logged). Held by the user — saved to the password manager under a petname they choose (the username field; never sent to the server) — and passed per POST.
- **The webapp instance id**: the diigo-read capability; lives in each cache DO's wiring, arrives from the mint step.
- **The cookie**: lives **only in webapp's DO storage**; crosses the wire once, at mint time, as an opaque relay.
- **The public cache**: no capability — `/public/<username>` is public by design (a guessable URL; hiding the link doesn't hide the page).
Stored state is meant to be durable: see the breaking-change rule in [capsite AGENTS.md](../AGENTS.md).
## Deployment concern: rate-limit `/api`
Public pages only read storage, so strangers can't make the cache spend your diigo session. `/api` is keyed (a cache id is needed), but Cap'n Web pipelining makes calls cheap to send. Before exposing a deployment:
- rate-limit `/api` (and `/public/*`, which reads storage) — on Cloudflare, the [Workers rate-limit binding](https://developers.cloudflare.com/workers/runtime-apis/bindings/rate-limit/) (keyed by client and/or username); on other workerd hosts, a rate limit in the proxy in front;
- run the fill job (goal 1e): public pages show only what's stored.
## RPC use (from another worker, same account)
```jsonc
// wrangler.jsonc of the caller
"services": [{ "binding": "DIIGO", "service": "diigo-cache", "entrypoint": "DiigoBridge" }]
```
```ts
// cache ids ride goal-2's wiring; see the capsite checklist
const ndjson = await env.DIIGO.dumpNdjson(4); // 4 pages of PAGE_SIZE
```
## Test
pnpm types # regenerate worker-configuration.d.ts (after wrangler.jsonc changes)
pnpm lint # fails if those types are stale; tsc over src/ and tests/
pnpm test # vitest
Tests run the real worker and both DOs in the vitest pool, with `WEBAPP` bound to [a fake webapp](tests/fake-webapp.js) serving diigo-shaped items (`wrangler.tests.jsonc`).
## Run
[The README](README.md) gets a cache going in dev. The same create flow from a terminal:
wid=$(curl -s -X POST http://localhost:8787/_mint --data-urlencode \
'url=https://www.diigo.com/interact_api/' --data-urlencode \
'headers={"user-agent":"Mozilla/5.0","x-requested-with":"XMLHttpRequest","cookie":"<paste>"}' \
--data-urlencode 'msPerCall=500' -D - -o /dev/null | grep -i x-webapp-id)
curl -s -X POST http://localhost:8787/_wire -H 'content-type: application/json' \
-d "{\"webappId\":\"<from $wid>\",\"mode\":\"private\"}"
Check a filled cache against a diigo RSS export: `node scripts/compare-export.ts <cache.sqlite> <export.xml> <out-dir>`.
## Deploy
Not deployed yet; the order matters (goal 5 in [the capsite checklist](../capsite-checklist.md)): webapp first (`cd ../webapp && npx wrangler deploy`), then diigo-cache, after rate-limiting `/api` (above).
+16 -110
View File
@@ -1,120 +1,26 @@
# diigo-cache # diigo-cache
Cloudflare Worker cache over [the webapp factory](../webapp/README.md): Your diigo bookmarks, cached in a [workerd](https://github.com/cloudflare/workerd) worker, with a public page you can filter by week: `/public/<you>?in=2026-W41`. Your diigo session cookie is stored in one Durable Object and nowhere else.
session-cookie reads of diigo bookmarks, cached in Durable-Object SQLite
storage. Evolved from the diigo-dump worker (provenance:
[mm-diigo `projects/diigo-bak/`](https://github.com/dckc/mm-diigo)); the
static API-key path is dead (see [the capsite checklist](../capsite-checklist.md),
goal 2 preamble).
## What it holds ## Getting started
- **The create flow, on one port**: `/_connect` renders webapp's own ```sh
mint form (diigo-prefilled, cookie blank) — wrangler multi-config dev pnpm install
doesn't serve aux workers' HTTP, so `/_mint` relays the form opaquely pnpm dev # → http://localhost:8787/
into the WEBAPP binding (the cookie crosses; nothing on this side opens
or logs it). Mint → `/_wire` (webapp id pre-filled, mode
public|private) → the created page, where the user saves the ids.
- **Two independent page caches** (`src/cache.ts`), one behavior: a
page hit is served from storage with no diigo call; a miss asks the
webapp instance in the cache's wiring and etches the page.
- `DiigoCache` (binding `CACHE_DO`): every item. Addressed by its
cache id.
- `PublicCache` (binding `PUBLIC_DO`): shared items only — private
rows are dropped *before* etching (`publicOnly`), so its storage
holds nothing private. Same cookie webapp as the private cache.
Public page n is diigo page n, filtered (may show fewer than
`PAGE_SIZE`, 100: diigo's cap).
`/public/<username>` resolves through the `idFromName("index")`
registry, and reads through `viewer()`, a read-only facet: it shows
stored pages and fill progress, and can't fetch from diigo or write.
- **Storage**: SQL lives in `src/cache-read.sql` (selects only) and
`src/cache-write.sql`; `src/store.ts` turns them into a reader and a
writer facet, so the cache logic holds no SQL text.
- **Pages**: home's dump form (cache id + page number, POST) shows raw
dump rows; `/public/<username>?page=n` shows an HTML list with a page
form (JSON for `Accept: application/json`). Both say `cache: HIT` or
`MISS`; every page read also logs `{event: "cache", kind, page,
outcome}`.
- **Publish later**: `POST /_publish` (cache id in the body) wires a
public companion to a private-only cache.
- **The webkey page and Cap'n Web**: `/cache/#<cache id>` is a skeleton
page, the same for every key; its script reads the key from the
fragment and trades it in-band for a capability over
[Cap'n Web](https://github.com/cloudflare/capnweb) at `/api`
(`CacheApi.cache(key)` → a `CacheCap` facet: fill, status, the public
companion; `src/api.ts`). The worker serves capnweb's browser build
itself at `/_capnweb.js`.
- **Fill job**: walks every diigo page into a cache in the background —
DO alarms, a few pages per step, progress stored in the DO
(`startFill`/`fillStatus`/`stopFill`), driven from the webkey page;
the public page shows progress. "Invalidate the last N days" there
drops the cached pages holding anything updated that recently
(`invalidateRecent`), so the next fill refetches them.
- The `DiigoBridge` entrypoint for other workers (its cache-id source is
the goal-2 wiring TODO).
## State surface
Ids are **webkeys** ([the pattern](../docs/webkeys.md)): 256-bit bearer
secrets that never ride a query string or path. The webkey page carries
the cache id in the URL fragment (never sent to the server) and passes
it in-band over Cap'n Web; forms carry ids in POST bodies; logs and pages
show only a fingerprint (a prefix of the id's one-way hash).
- **The cache id**: the cache's read capability (bearer; not logged).
Held by the user — saved to the password manager under a petname they
choose (the username field; never sent to the server) — and passed
per POST.
- **The webapp instance id**: the diigo-read capability; lives in each
cache DO's wiring, arrives from the mint step.
- **The cookie**: lives **only in webapp's DO storage**; crosses the
wire once, at mint time, as an opaque relay.
- **The public cache**: no capability — `/public/<username>` is public
by design (a guessable URL; hiding the link doesn't hide the page).
Stored state is meant to be durable: see the breaking-change rule in
[capsite AGENTS.md](../AGENTS.md).
## Deployment concern: rate-limit `/api`
Public pages only read storage, so strangers can't make the cache spend
your diigo session. `/api` is keyed (a cache id is needed), but Cap'n
Web pipelining makes calls cheap to send. Before exposing a deployment:
- rate-limit `/api` (and `/public/*`, which reads storage) — on
Cloudflare, the
[Workers rate-limit binding](https://developers.cloudflare.com/workers/runtime-apis/bindings/rate-limit/)
(keyed by client and/or username); on other workerd hosts, a rate
limit in the proxy in front;
- run the fill job (goal 1e): public pages show only what's stored.
## RPC use (from another worker, same account)
```jsonc
// wrangler.jsonc of the caller
"services": [{ "binding": "DIIGO", "service": "diigo-cache", "entrypoint": "DiigoBridge" }]
``` ```
```ts Then, in the browser:
// cache ids ride goal-2's wiring; see the capsite checklist
const ndjson = await env.DIIGO.dumpNdjson(4); // 4 pages of PAGE_SIZE
```
## Test 1. **mint the diigo session at webapp**: paste the `Cookie` header from diigo.com (devtools, any logged-in request).
2. **wire** it, mode **public**. Save the cache id it shows (the password-manager button does this).
3. **open this cache** → **fill**, and watch it walk your bookmarks.
4. Visit `/public/<your diigo username>?days=7`.
pnpm types # regenerate worker-configuration.d.ts (after wrangler.jsonc changes) ## Bookmarks you hold, a page anyone can read
pnpm lint # fails if those types are stale; tsc over src/ and tests/
pnpm test # vitest
Tests run the real worker and both DOs in the vitest pool, with - **Filter by date and search:** `?days=7`, `?in=2026-W41` (ISO week, UTC), `?after=2026-01-01&before=2026-02-01`, `?by=created`, and `?q=` with words, `"phrases"`, `#tags`, `-negation`, and gmail-style `newer_than:7d`. Add `Accept: application/json` for JSON.
`WEBAPP` bound to [a fake webapp](tests/fake-webapp.js) serving - **Private stays private:** the public cache drops private bookmarks before storing anything, so it never holds them.
diigo-shaped items (`wrangler.tests.jsonc`). - **Ids are capabilities:** the cache id is a bearer key you keep in your password manager. It rides URL fragments and POST bodies, never a path or query string, so it stays out of logs and history.
- **Cheap to read:** pages come from Durable Object SQLite storage, and a fill job in the background fetches each diigo page once.
## Run, deploy Design, tests, and deploying: [CONTRIBUTING](CONTRIBUTING.md).
[How to run / connect / deploy is maintained in the capsite checklist](../capsite-checklist.md)
(single source; this README keeps only worker-level facts).
Status: create, dump, publish, and public pages exercised against live
diigo in dev (2026-10-07); not yet deployed.
+1
View File
@@ -11,6 +11,7 @@
"devDependencies": { "devDependencies": {
"@cloudflare/vitest-plugin": "1.3.6", "@cloudflare/vitest-plugin": "1.3.6",
"@types/node": "^22", "@types/node": "^22",
"microformats-parser": "^2.0.6",
"typescript": "^5.9.0", "typescript": "^5.9.0",
"vitest": "4.1.11", "vitest": "4.1.11",
"wrangler": "^4.148.0" "wrangler": "^4.148.0"
+24
View File
@@ -18,6 +18,9 @@ importers:
'@types/node': '@types/node':
specifier: ^22 specifier: ^22
version: 22.20.5 version: 22.20.5
microformats-parser:
specifier: ^2.0.6
version: 2.0.6
typescript: typescript:
specifier: ^5.9.0 specifier: ^5.9.0
version: 5.9.3 version: 5.9.3
@@ -613,6 +616,10 @@ packages:
resolution: {integrity: sha512-Btj2BOOO83o3WyH59e8MgXsxEQVcarkUOpEYrubB0urwnN10yQ364rsiByU11nZlqWYZm05i/of7io4mzihBtQ==} resolution: {integrity: sha512-Btj2BOOO83o3WyH59e8MgXsxEQVcarkUOpEYrubB0urwnN10yQ364rsiByU11nZlqWYZm05i/of7io4mzihBtQ==}
engines: {node: '>=8'} engines: {node: '>=8'}
entities@6.0.1:
resolution: {integrity: sha512-aN97NXWF6AWBTahfVOIrB/NShkzi5H7F9r1s9mD3cDj4Ko5f2qhhVoYMibXF7GlLveb/D2ioWay8lxI97Ven3g==}
engines: {node: '>=0.12'}
error-stack-parser-es@1.0.5: error-stack-parser-es@1.0.5:
resolution: {integrity: sha512-5qucVt2XcuGMcEGgWI7i+yZpmpByQ8J1lHhcL7PwqCwu9FPP3VUXzT4ltHe5i2z9dePwEHcDVOAfSnHsOlCXRA==} resolution: {integrity: sha512-5qucVt2XcuGMcEGgWI7i+yZpmpByQ8J1lHhcL7PwqCwu9FPP3VUXzT4ltHe5i2z9dePwEHcDVOAfSnHsOlCXRA==}
@@ -722,6 +729,10 @@ packages:
magic-string@0.30.21: magic-string@0.30.21:
resolution: {integrity: sha512-vd2F4YUyEXKGcLHoq+TEyCjxueSeHnFxyyjNp80yg0XV4vUhnDer/lvvlqM/arB5bXQN5K2/3oinyCRyx8T2CQ==} resolution: {integrity: sha512-vd2F4YUyEXKGcLHoq+TEyCjxueSeHnFxyyjNp80yg0XV4vUhnDer/lvvlqM/arB5bXQN5K2/3oinyCRyx8T2CQ==}
microformats-parser@2.0.6:
resolution: {integrity: sha512-VTnLImF17yd+ZfBH40sddHMJp5Rrker4ge+oPjgnBQ3jluQO0YmsNVA4UcEfHRBV8zrikBSMkAzAub+nTRX85g==}
engines: {node: '>=18'}
miniflare@5.20261001.0-alpha: miniflare@5.20261001.0-alpha:
resolution: {integrity: sha512-GaimS5mSIOMyvd16ga+e1/QkI8cmp3z35QPHFex0DktqNQf2RVZQwMPQXdyoUreUiFP/0Gpe2MoQInTyMAD5xA==} resolution: {integrity: sha512-GaimS5mSIOMyvd16ga+e1/QkI8cmp3z35QPHFex0DktqNQf2RVZQwMPQXdyoUreUiFP/0Gpe2MoQInTyMAD5xA==}
engines: {node: '>=22.0.0'} engines: {node: '>=22.0.0'}
@@ -739,6 +750,9 @@ packages:
resolution: {integrity: sha512-XrsrhT5sybtKI6wakr2SPOlGZWWYbUXZ7a0jT8/QOeAPau+1X/bSegNe5YR75oJmEZQbKningirmGOEJCIk61Q==} resolution: {integrity: sha512-XrsrhT5sybtKI6wakr2SPOlGZWWYbUXZ7a0jT8/QOeAPau+1X/bSegNe5YR75oJmEZQbKningirmGOEJCIk61Q==}
engines: {node: '>=12.20.0'} engines: {node: '>=12.20.0'}
parse5@7.3.0:
resolution: {integrity: sha512-IInvU7fabl34qmi9gY8XOVxhYyMyuH2xUNpb2q8/Y+7552KlejkRvqvD19nMoUW/uQGGbqNpA6Tufu5FL5BZgw==}
path-to-regexp@6.3.0: path-to-regexp@6.3.0:
resolution: {integrity: sha512-Yhpw4T9C6hPpgPeA28us07OJeqZ5EzQTkbfwuhsUg0c237RomFoETJgmp2sa3F/41gfLE6G5cqcYwznmeEeOlQ==} resolution: {integrity: sha512-Yhpw4T9C6hPpgPeA28us07OJeqZ5EzQTkbfwuhsUg0c237RomFoETJgmp2sa3F/41gfLE6G5cqcYwznmeEeOlQ==}
@@ -1366,6 +1380,8 @@ snapshots:
detect-libc@2.1.2: {} detect-libc@2.1.2: {}
entities@6.0.1: {}
error-stack-parser-es@1.0.5: {} error-stack-parser-es@1.0.5: {}
es-module-lexer@2.3.2: {} es-module-lexer@2.3.2: {}
@@ -1467,6 +1483,10 @@ snapshots:
dependencies: dependencies:
'@jridgewell/sourcemap-codec': 1.6.0 '@jridgewell/sourcemap-codec': 1.6.0
microformats-parser@2.0.6:
dependencies:
parse5: 7.3.0
miniflare@5.20261001.0-alpha(@types/node@22.20.5): miniflare@5.20261001.0-alpha(@types/node@22.20.5):
dependencies: dependencies:
'@cspotcode/source-map-support': 0.8.1 '@cspotcode/source-map-support': 0.8.1
@@ -1497,6 +1517,10 @@ snapshots:
obug@2.2.1: {} obug@2.2.1: {}
parse5@7.3.0:
dependencies:
entities: 6.0.1
path-to-regexp@6.3.0: {} path-to-regexp@6.3.0: {}
pathe@2.0.3: {} pathe@2.0.3: {}
+11
View File
@@ -19,3 +19,14 @@ where cast(items.updated as integer) >= ?;
-- pagesThrough: -- pagesThrough:
select count(*) as n from pages where page_num <= ?; select count(*) as n from pages where page_num <= ?;
-- itemsByUpdated:
select json from items
where cast(updated as integer) >= ? and cast(updated as integer) < ?
order by cast(updated as integer) desc, item_id;
-- itemsByCreated:
select json from items
where cast(json_extract(json, '$.created_at') as integer) >= ?
and cast(json_extract(json, '$.created_at') as integer) < ?
order by cast(json_extract(json, '$.created_at') as integer) desc, item_id;
+33 -1
View File
@@ -6,6 +6,8 @@
import { DurableObject, RpcTarget } from "cloudflare:workers"; import { DurableObject, RpcTarget } from "cloudflare:workers";
import type { PagedGet } from "./diigo.js"; import type { PagedGet } from "./diigo.js";
import { publicOnly, toRows } from "./items.js"; import { publicOnly, toRows } from "./items.js";
import { matches } from "./query.js";
import type { Query } from "./query.js";
import { makeReader, makeWriter, SCHEMA_VERSION } from "./store.js"; import { makeReader, makeWriter, SCHEMA_VERSION } from "./store.js";
import type { Reader, Writer } from "./store.js"; import type { Reader, Writer } from "./store.js";
import type { Item, ItemRow } from "./items.js"; import type { Item, ItemRow } from "./items.js";
@@ -398,7 +400,18 @@ abstract class PagedCache extends DurableObject<CacheEnv> {
} }
} }
/** what anyone may see of a cache: stored pages and fill progress */ /** one page of a filter's matches over stored items */
export type QueryPage = {
/** the matches on this page, raw dump rows (JSON strings) */
items: string[];
/** matches in all */
matched: number;
/** true when every diigo page is stored, so no match can be missing */
complete: boolean;
};
/** what anyone may see of a cache: stored pages, stored items by filter,
* and fill progress */
export class PageViewer extends RpcTarget { export class PageViewer extends RpcTarget {
#reader: Reader; #reader: Reader;
#fillStatus: () => Promise<FillStatus | undefined>; #fillStatus: () => Promise<FillStatus | undefined>;
@@ -415,6 +428,25 @@ export class PageViewer extends RpcTarget {
return stored && { ...stored, cached: true }; return stored && { ...stored, cached: true };
} }
/**
* page `pageNum` (`count` per page) of the stored items matching
* `query`, newest first by its date. The window narrows in SQL; terms
* are checked per item. Reads storage only: items not yet stored can't
* match, which `complete` reports.
*/
query(query: Query, pageNum: number, count: number): QueryPage {
const from = query.after ? Math.ceil(query.after.getTime() / 1000) : Number.MIN_SAFE_INTEGER;
const to = query.before ? Math.ceil(query.before.getTime() / 1000) : Number.MAX_SAFE_INTEGER;
const hits = this.#reader.itemsIn(query.by, from, to).filter((json) => matches(JSON.parse(json) as Item, query));
const total = this.#reader.knownTotal();
const pages = total === null ? null : Math.max(1, Math.ceil(total / PAGE_SIZE));
return {
items: hits.slice((pageNum - 1) * count, pageNum * count),
matched: hits.length,
complete: pages !== null && this.#reader.pagesThrough(pages) >= pages,
};
}
/** diigo's total, as last stored, or null if nothing is stored */ /** diigo's total, as last stored, or null if nothing is stored */
knownTotal(): number | null { knownTotal(): number | null {
return this.#reader.knownTotal(); return this.#reader.knownTotal();
+152 -34
View File
@@ -10,9 +10,11 @@ import { CAPNWEB_BROWSER_JS } from "./capnweb-browser.js";
import { CACHE_PAGE_HTML } from "./cache-page.js"; import { CACHE_PAGE_HTML } from "./cache-page.js";
import { DiigoCacheLogic, PublicCacheLogic } from "./cache.js"; import { DiigoCacheLogic, PublicCacheLogic } from "./cache.js";
import { PAGE_SIZE } from "./cache.js"; import { PAGE_SIZE } from "./cache.js";
import type { CacheEnv, Clock, FillStatus } from "./cache.js"; import type { CacheEnv, Clock, FillStatus, QueryPage } from "./cache.js";
import type { Item } from "./cache.js"; import type { Item } from "./cache.js";
import { updatedISO, usernameOf } from "./items.js"; import { usernameOf } from "./items.js";
import { dateOf, isEmpty, parseQuery } from "./query.js";
import type { Query } from "./query.js";
import { toWebkey, idFromWebkey, isWebkey, shashKey } from "./webkeys.js"; import { toWebkey, idFromWebkey, isWebkey, shashKey } from "./webkeys.js";
import { pmSave } from "./pm-save.js"; import { pmSave } from "./pm-save.js";
@@ -443,10 +445,16 @@ async function dump(request: Request, env: Env): Promise<Response> {
const PUBLIC_PER_PAGE = PAGE_SIZE; const PUBLIC_PER_PAGE = PAGE_SIZE;
/** the public page's day presets */
const DAY_PRESETS = [7, 30, 90, 365];
/** /**
* the public page for a username: stored pages only, through the cache's * the public page for a username: stored pages only, through the cache's
* read-only `viewer()` (it can't fetch or write). HTML with a page-number * read-only `viewer()` (it can't fetch or write). HTML with a filter form
* form, or JSON for `Accept: application/json`. `?page=` in the URL is * and a page-number form, or JSON for `Accept: application/json`. A
* filter (goal 1d: `days`, `after`, `before`, `in`, `by`, `q`; see
* {@link parseQuery}) lists the stored items that match, `page` counting
* matches; with none, `page` is diigo's page as stored. Query params are
* fine here: nothing on this page is secret. * fine here: nothing on this page is secret.
*/ */
async function publicPage(request: Request, env: Env, username: string): Promise<Response> { async function publicPage(request: Request, env: Env, username: string): Promise<Response> {
@@ -456,14 +464,33 @@ async function publicPage(request: Request, env: Env, username: string): Promise
const ns = env.PUBLIC_DO; const ns = env.PUBLIC_DO;
const cache = ns.get(ns.idFromString(idFromWebkey(publicId))); const cache = ns.get(ns.idFromString(idFromWebkey(publicId)));
const raw = new URL(request.url).searchParams.get("page"); const params = new URL(request.url).searchParams;
const accept = request.headers.get("accept") ?? "";
const wantsJson = accept.includes("application/json") && !accept.includes("text/html");
const here = `/public/${encodeURIComponent(username)}`;
const raw = params.get("page");
const pageN = raw === null ? 1 : Number.parseInt(raw, 10); const pageN = raw === null ? 1 : Number.parseInt(raw, 10);
const valid = !Number.isNaN(pageN) && pageN >= 1 && pageN <= 10000; const valid = !Number.isNaN(pageN) && pageN >= 1 && pageN <= 10000;
const query = parseQuery(params, new Date());
if ("error" in query) {
console.log({ event: "notice", page: "public", text: "filter refused", error: query.error });
if (wantsJson) return Response.json({ error: query.error }, { status: 400 });
const body = publicFrame(username, `${filterForm(here, params)}
<p><em>${esc(query.error)}</em></p>`);
return new Response(body, { status: 400, headers: { "content-type": "text/html;charset=utf-8" } });
}
const viewer = await cache.viewer(); const viewer = await cache.viewer();
let page, total, fill; let page, fill, found;
let total: number | null = null;
try { try {
page = await viewer.page(valid ? pageN : 1, PUBLIC_PER_PAGE); if (isEmpty(query)) {
total = page ? page.total : await viewer.knownTotal(); page = await viewer.page(valid ? pageN : 1, PUBLIC_PER_PAGE);
total = page ? page.total : await viewer.knownTotal();
} else {
found = await viewer.query(query, valid ? pageN : 1, PUBLIC_PER_PAGE);
}
fill = await viewer.fillStatus(); fill = await viewer.fillStatus();
} catch (err) { } catch (err) {
console.error({ event: "public-page", page: pageN, error: String(err) }); console.error({ event: "public-page", page: pageN, error: String(err) });
@@ -471,13 +498,14 @@ async function publicPage(request: Request, env: Env, username: string): Promise
} finally { } finally {
viewer[Symbol.dispose](); viewer[Symbol.dispose]();
} }
if (found) return filteredPage(username, here, params, query, found, valid ? pageN : 1, wantsJson, fill);
// total and pages count all of diigo's items: public page n is diigo // total and pages count all of diigo's items: public page n is diigo
// page n with the private rows dropped // page n with the private rows dropped
const pages = total === null ? 1 : Math.max(1, Math.ceil(total / PUBLIC_PER_PAGE)); const pages = total === null ? 1 : Math.max(1, Math.ceil(total / PUBLIC_PER_PAGE));
const items = page ? page.items.map((j) => JSON.parse(j) as Item) : []; const items = page ? page.items.map((j) => JSON.parse(j) as Item) : [];
const accept = request.headers.get("accept") ?? ""; if (wantsJson) {
if (accept.includes("application/json") && !accept.includes("text/html")) {
return Response.json({ username, page: pageN, pages, total, stored: page !== null, etchedAt: page?.etchedAt ?? null, items }); return Response.json({ username, page: pageN, pages, total, stored: page !== null, etchedAt: page?.etchedAt ?? null, items });
} }
@@ -486,42 +514,132 @@ async function publicPage(request: Request, env: Env, username: string): Promise
console.log({ event: "notice", page: "public", text: "page out of range", pages }); console.log({ event: "notice", page: "public", text: "page out of range", pages });
notice = `<p><em>no page ${esc(String(raw))}: pages run 1–${pages}</em></p>`; notice = `<p><em>no page ${esc(String(raw))}: pages run 1–${pages}</em></p>`;
} }
const here = `/public/${encodeURIComponent(username)}`;
const nav = (n: number, label: string) => const nav = (n: number, label: string) =>
n >= 1 && n <= pages ? `<a href="${here}?page=${n}">${label}</a>` : `<span>${label}</span>`; n >= 1 && n <= pages ? `<a rel="${n < pageN ? "prev" : "next"}" href="${here}?page=${n}">${label}</a>` : `<span>${label}</span>`;
const list = notice const list = notice ? "" : itemList(items, "updated", here);
? "" const body = publicFrame(
: `<ol> username,
${items `<p>${pages} page${pages === 1 ? "" : "s"} (diigo's, ${PUBLIC_PER_PAGE} bookmarks each; private ones are left out, so a page may show fewer).</p>
.map((it) => {
const url = typeof it.url === "string" ? it.url : "";
const title = typeof it.title === "string" && it.title ? it.title : url;
const desc = typeof it.description === "string" && it.description ? `<br><small>${esc(it.description)}</small>` : "";
const href = /^https?:\/\//i.test(url) ? url : "#";
const when = updatedISO(it);
const date = when ? `<time datetime="${when}">${when}</time>` : "(no date)";
return `<li>${date} <a href="${esc(href)}">${esc(title)}</a>${desc}</li>`;
})
.join("\n")}
</ol>`;
const body = `<!doctype html>
<html><head><meta charset="utf-8"><title>${esc(username)}'s shared bookmarks</title></head>
<body>
<h1>${esc(username)}'s shared bookmarks</h1>
<p>${pages} page${pages === 1 ? "" : "s"} (diigo's, ${PUBLIC_PER_PAGE} bookmarks each; private ones are left out, so a page may show fewer).</p>
<p>${page ? `<strong>stored ${esc(page.etchedAt ?? "")}</strong> — page ${valid ? pageN : 1}, ${items.length} shared` : `<strong>not fetched yet</strong> — page ${valid ? pageN : 1}; the cache's owner fills it`}</p> <p>${page ? `<strong>stored ${esc(page.etchedAt ?? "")}</strong> — page ${valid ? pageN : 1}, ${items.length} shared` : `<strong>not fetched yet</strong> — page ${valid ? pageN : 1}; the cache's owner fills it`}</p>
${fill ? fillLine(fill) : ""} ${fill ? fillLine(fill) : ""}
${filterForm(here, params)}
<form method="get" action="${here}"> <form method="get" action="${here}">
<p>${nav(pageN - 1, "← prev")} <p>${nav(pageN - 1, "← prev")}
<label>page <input type="number" name="page" min="1" value="${Number.isNaN(pageN) ? 1 : pageN}"></label> <label>page <input type="number" name="page" min="1" value="${Number.isNaN(pageN) ? 1 : pageN}"></label>
<button type="submit">go</button> <button type="submit">go</button>
${nav(pageN + 1, "next →")}</p> ${nav(pageN + 1, "next →")}</p>
</form> </form>
${notice}${list} ${notice}${list}`,
</body></html>`; );
return new Response(body, { headers: { "content-type": "text/html;charset=utf-8" } }); return new Response(body, { headers: { "content-type": "text/html;charset=utf-8" } });
} }
/** a filter's matches, as the public page's HTML or JSON */
function filteredPage(
username: string,
here: string,
params: URLSearchParams,
query: Query,
found: QueryPage,
pageN: number,
wantsJson: boolean,
fill: FillStatus | undefined,
): Response {
const pages = Math.max(1, Math.ceil(found.matched / PUBLIC_PER_PAGE));
const items = found.items.map((j) => JSON.parse(j) as Item);
if (wantsJson) {
const { matched, complete } = found;
return Response.json({ username, filter: query, page: pageN, pages, matched, complete, items });
}
// links keep the filter: the request's own non-empty params, in order
const kept = [...params].filter(([k, v]) => k !== "page" && v !== "");
const href = (n: number) => `${here}?${new URLSearchParams([...kept, ["page", String(n)]])}`;
const nav = (n: number, label: string) =>
n >= 1 && n <= pages ? `<a rel="${n < pageN ? "prev" : "next"}" href="${esc(href(n))}">${label}</a>` : `<span>${label}</span>`;
const status = found.matched
? `${found.matched} match${found.matched === 1 ? "" : "es"}, page ${pageN} of ${pages}, newest ${query.by} first`
: "no matches";
const caveat = found.complete ? "" : " — not every diigo page is stored yet, so some matches may be missing";
const body = publicFrame(
username,
`<p><strong>${esc(status)}</strong>${caveat}</p>
${fill ? fillLine(fill) : ""}
${filterForm(here, params)}
<p>${nav(pageN - 1, "← prev")} ${nav(pageN + 1, "next →")} <a href="${here}">all, by diigo page</a></p>
${itemList(items, query.by, here)}`,
);
return new Response(body, { headers: { "content-type": "text/html;charset=utf-8" } });
}
/** the public page's frame: title and heading around `content` */
function publicFrame(username: string, content: string): string {
return `<!doctype html>
<html><head><meta charset="utf-8"><title>${esc(username)}'s shared bookmarks</title></head>
<body>
<h1>${esc(username)}'s shared bookmarks</h1>
${content}
</body></html>`;
}
/** the filter form, echoing the request's filter */
function filterForm(here: string, params: URLSearchParams): string {
const field = (name: string) => esc(params.get(name) ?? "");
const days = params.get("days") ?? "";
const presets = DAY_PRESETS.map(String);
if (days && !presets.includes(days)) presets.push(days);
const option = (value: string, label: string, current: string) =>
`<option value="${esc(value)}"${value === current ? " selected" : ""}>${esc(label)}</option>`;
return `<form method="get" action="${here}">
<p><label>last <select name="days">${option("", "any", days)}${presets.map((n) => option(n, `${n} days`, days)).join("")}</select></label>
<label>or after <input type="date" name="after" value="${field("after")}"></label>
<label>before <input type="date" name="before" value="${field("before")}"></label>
<label>or in <input name="in" size="9" placeholder="2026-W41" value="${field("in")}"></label>
<label>by <select name="by">${option("updated", "updated", params.get("by") ?? "")}${option("created", "created", params.get("by") ?? "")}</select></label></p>
<p><label>search <input type="search" name="q" size="40" value="${field("q")}"></label>
<button type="submit">filter</button></p>
<p><small>Dates and periods (2026, 2026-03, 2026-Q1, 2026-W41) are UTC. Search takes
words, "phrases", #tags, -negation, and after:, before:, newer_than:7d, during:2026-Q1, created:2026.</small></p>
</form>`;
}
/**
* bookmarks as a list, each led by its `by` date, marked up as
* microformats: an `h-feed` of `h-entry` bookmarks (`u-bookmark-of`,
* `p-name`, `p-summary`, `p-category` per tag, `dt-published` when
* bookmarked, `dt-updated` when last edited). This markup is the public
* page's API for other sites (tests/caches.test.ts pins it); tag links
* search this page (`here`) by tag.
*
* @see https://microformats.org/wiki/h-entry
* @see https://indieweb.org/bookmark
*/
function itemList(items: Item[], by: Query["by"], here: string): string {
const time = (it: Item, which: Query["by"], hidden: boolean) => {
const secs = dateOf(it, which);
if (secs === undefined) return hidden ? "" : "(no date)";
const when = new Date(secs * 1000).toISOString();
const cls = which === "created" ? "dt-published" : "dt-updated";
return hidden ? `<time class="${cls}" datetime="${when}" hidden></time>` : `<time class="${cls}" datetime="${when}">${when}</time>`;
};
return `<ol class="h-feed">
${items
.map((it) => {
const url = typeof it.url === "string" ? it.url : "";
const title = typeof it.title === "string" && it.title ? it.title : url;
const desc = typeof it.description === "string" && it.description ? `<br><small class="p-summary">${esc(it.description)}</small>` : "";
const href = /^https?:\/\//i.test(url) ? url : "#";
const tags = (typeof it.tags === "string" ? it.tags.split(",") : [])
.map((t) => t.trim())
.filter(Boolean)
.map((t) => ` <a class="p-category" href="${esc(`${here}?q=${encodeURIComponent(`#${t}`)}`)}">${esc(t)}</a>`)
.join("");
const other = by === "created" ? "updated" : "created";
return `<li class="h-entry">${time(it, by, false)} <a class="u-bookmark-of p-name" href="${esc(href)}">${esc(title)}</a>${tags}${desc}${time(it, other, true)}</li>`;
})
.join("\n")}
</ol>`;
}
/** home page: the connect link (webapp's cookie form, diigo-prefilled) + /** home page: the connect link (webapp's cookie form, diigo-prefilled) +
* the wire step for an already-minted id + the dump form. `home` renders * the wire step for an already-minted id + the dump form. `home` renders
* the error path too, so it takes env for the connect link. Notices (the * the error path too, so it takes env for the connect link. Notices (the
+188 -19
View File
@@ -1,18 +1,10 @@
/** /**
* @file the public page's filters: date windows and search (goal 1d, **not * @file the public page's filters: date windows and search over a cache's
* yet implemented**; the design is the skipped specs in tests/query.test.ts). * stored items (goal 1d; tests/query.test.ts).
*/ */
import type { Item } from "./items.js"; import type { Item } from "./items.js";
/**
* diigo fetches one request may spend (cache hits are free); under the
* Workers free plan's 50 external subrequests per invocation
*
* @see https://developers.cloudflare.com/workers/platform/limits/
*/
export const FETCH_BUDGET = 40;
/** /**
* a filter as devs write it (`{ days: 7, q: "#weekly" }`); URL params map * a filter as devs write it (`{ days: 7, q: "#weekly" }`); URL params map
* onto these fields one-to-one (strings, with "" meaning absent). Dates * onto these fields one-to-one (strings, with "" meaning absent). Dates
@@ -56,6 +48,76 @@ export type Query = {
/** a refused filter: the message is safe to show and log */ /** a refused filter: the message is safe to show and log */
export type QueryError = { error: string }; export type QueryError = { error: string };
const DAY_MS = 24 * 60 * 60 * 1000;
/** thrown inside {@link toQuery}, returned as a {@link QueryError} */
class Refused extends Error {}
/** `yyyy-mm-dd` or `yyyy/mm/dd` as 00:00Z of that UTC day */
function parseDay(text: string, what: string): Date {
const m = /^(\d{4})([-/])(\d{2})\2(\d{2})$/.exec(text);
const d = m && new Date(Date.UTC(Number(m[1]), Number(m[3]) - 1, Number(m[4])));
if (!m || !d || d.getUTCMonth() !== Number(m[3]) - 1 || d.getUTCDate() !== Number(m[4])) {
throw new Refused(`${what}: expected a date as yyyy-mm-dd`);
}
return d;
}
/** Monday 00:00Z of ISO week 1 of `year` (the week holding January 4) */
function isoWeekOne(year: number): number {
const jan4 = Date.UTC(year, 0, 4);
const weekday = new Date(jan4).getUTCDay() || 7;
return jan4 - (weekday - 1) * DAY_MS;
}
/** a named period as its half-open [after, before) window */
function parsePeriod(text: string, what: string): [Date, Date] {
const refuse = () => new Refused(`${what}: expected a period as 2025, 2025-03, 2025-Q1, or 2025-W01`);
let m;
if ((m = /^(\d{4})$/.exec(text))) {
const y = Number(m[1]);
return [new Date(Date.UTC(y, 0, 1)), new Date(Date.UTC(y + 1, 0, 1))];
}
if ((m = /^(\d{4})-(\d{2})$/.exec(text))) {
const [y, mo] = [Number(m[1]), Number(m[2])];
if (mo < 1 || mo > 12) throw refuse();
return [new Date(Date.UTC(y, mo - 1, 1)), new Date(Date.UTC(y, mo, 1))];
}
if ((m = /^(\d{4})-Q([1-4])$/.exec(text))) {
const [y, q] = [Number(m[1]), Number(m[2])];
return [new Date(Date.UTC(y, 3 * (q - 1), 1)), new Date(Date.UTC(y, 3 * q, 1))];
}
if ((m = /^(\d{4})-W(\d{2})$/.exec(text))) {
const [y, w] = [Number(m[1]), Number(m[2])];
const weeks = (isoWeekOne(y + 1) - isoWeekOne(y)) / (7 * DAY_MS);
if (w < 1 || w > weeks) throw refuse();
const monday = isoWeekOne(y) + (w - 1) * 7 * DAY_MS;
return [new Date(monday), new Date(monday + 7 * DAY_MS)];
}
throw refuse();
}
/** gmail's `newer_than:` amount (`7d`, `2w`, `1m`, `1y`) counted back from
* `now`; months and years are calendar steps */
function parseNewerThan(text: string, now: Date): Date {
const m = /^(\d{1,4})([dwmy])$/.exec(text);
if (!m || Number(m[1]) < 1) throw new Refused("newer_than: expected an amount like 7d, 2w, 1m, or 1y");
const n = Number(m[1]);
const d = new Date(now.getTime());
if (m[2] === "d") d.setTime(now.getTime() - n * DAY_MS);
if (m[2] === "w") d.setTime(now.getTime() - 7 * n * DAY_MS);
if (m[2] === "m") d.setUTCMonth(d.getUTCMonth() - n);
if (m[2] === "y") d.setUTCFullYear(d.getUTCFullYear() - n);
return d;
}
/** `days` as a whole number from 1 to 3650 */
function parseDays(days: number | string): number {
const n = typeof days === "number" ? days : /^\d+$/.test(days) ? Number(days) : NaN;
if (!Number.isInteger(n) || n < 1 || n > 3650) throw new Refused("days: expected a whole number from 1 to 3650");
return n;
}
/** /**
* resolve a filter: fields and `q` operators combine; a bound given twice * resolve a filter: fields and `q` operators combine; a bound given twice
* (by fields, operators, or both) is refused, as is an empty window. * (by fields, operators, or both) is refused, as is an empty window.
@@ -75,21 +137,128 @@ export type QueryError = { error: string };
* not read ambiently, so tests pin it) * not read ambiently, so tests pin it)
*/ */
export function toQuery(spec: FilterSpec, now: Date): Query | QueryError { export function toQuery(spec: FilterSpec, now: Date): Query | QueryError {
void spec; try {
void now; return resolve(spec, now);
throw new Error("toQuery: not implemented (goal 1d)"); } catch (err) {
if (err instanceof Refused) return { error: err.message };
throw err;
}
}
function resolve(spec: FilterSpec, now: Date): Query {
const words: Term[] = [];
const tags: Term[] = [];
let by: { value: Query["by"]; from: string } | undefined;
let after: { value: Date; from: string } | undefined;
let before: { value: Date; from: string } | undefined;
const setBy = (value: Query["by"], from: string) => {
if (by && by.value !== value) throw new Refused(`${by.from} and ${from} name different dates`);
by = { value, from };
};
const setAfter = (value: Date, from: string) => {
if (after) throw new Refused(`${after.from} and ${from} both set where the window starts`);
after = { value, from };
};
const setBefore = (value: Date, from: string) => {
if (before) throw new Refused(`${before.from} and ${from} both set where the window ends`);
before = { value, from };
};
const setPeriod = ([lo, hi]: [Date, Date], from: string) => {
setAfter(lo, from);
setBefore(hi, from);
};
const present = (v: string | number | undefined) => v !== undefined && v !== "";
if (present(spec.by)) {
if (spec.by !== "updated" && spec.by !== "created") throw new Refused("by: expected updated or created");
setBy(spec.by, `by=${spec.by}`);
}
if (present(spec.days)) setAfter(new Date(now.getTime() - parseDays(spec.days!) * DAY_MS), "days");
if (present(spec.after)) setAfter(parseDay(spec.after!, "after"), "after");
if (present(spec.before)) setBefore(parseDay(spec.before!, "before"), "before");
if (present(spec.in)) setPeriod(parsePeriod(spec.in!, "in"), "in");
for (const m of (spec.q ?? "").matchAll(/(-?)"([^"]*)"|(\S+)/g)) {
if (m[2] !== undefined) {
if (m[2].trim()) words.push({ text: m[2].trim().toLowerCase(), negated: m[1] === "-" });
continue;
}
const token = m[3];
const negated = token.length > 1 && token.startsWith("-");
const body = negated ? token.slice(1) : token;
const op = /^([a-z_]+):(.+)$/i.exec(body);
const name = op?.[1].toLowerCase();
const arg = op?.[2] ?? "";
const windowOp = ["after", "before", "newer_than", "during", "in", "created", "updated"];
if (name && windowOp.includes(name) && negated) throw new Refused(`${name}: can't be negated`);
if (name === "after") setAfter(parseDay(arg, "after:"), "after:");
else if (name === "before") setBefore(parseDay(arg, "before:"), "before:");
else if (name === "newer_than") setAfter(parseNewerThan(arg, now), "newer_than:");
else if (name === "during" || name === "in") setPeriod(parsePeriod(arg, `${name}:`), `${name}:`);
else if (name === "created" || name === "updated") {
setBy(name, `${name}:`);
setPeriod(parsePeriod(arg, `${name}:`), `${name}:`);
} else if (name === "tag") tags.push({ text: arg.toLowerCase(), negated });
else if (body.length > 1 && body.startsWith("#")) tags.push({ text: body.slice(1).toLowerCase(), negated });
else words.push({ text: body.toLowerCase(), negated });
}
if (after && before && after.value.getTime() >= before.value.getTime()) {
throw new Refused(`the window is empty: ${after.from} is not earlier than ${before.from}`);
}
const query: Query = { by: by?.value ?? "updated", words, tags };
if (after) query.after = after.value;
if (before) query.before = before.value;
return query;
} }
/** {@link toQuery} over the public page's URL params */ /** {@link toQuery} over the public page's URL params */
export function parseQuery(params: URLSearchParams, now: Date): Query | QueryError { export function parseQuery(params: URLSearchParams, now: Date): Query | QueryError {
void params; const field = (name: string) => params.get(name) ?? "";
void now; const by = field("by");
throw new Error("parseQuery: not implemented (goal 1d)"); return toQuery(
{
days: field("days"),
after: field("after"),
before: field("before"),
in: field("in"),
by: by as FilterSpec["by"],
q: field("q"),
},
now,
);
}
/** does the query filter anything? (an empty one lists diigo's pages as stored) */
export function isEmpty(query: Query): boolean {
return !query.after && !query.before && !query.words.length && !query.tags.length && query.by === "updated";
}
/** the item's date that `by` names, in epoch seconds (diigo's unit), or
* undefined when it carries none */
export function dateOf(item: Item, by: Query["by"]): number | undefined {
const secs = by === "created" ? item.created_at : item.updated_at;
return typeof secs === "number" && Number.isFinite(secs) ? secs : undefined;
} }
/** does `item` fall in the window and satisfy every term? */ /** does `item` fall in the window and satisfy every term? */
export function matches(item: Item, query: Query): boolean { export function matches(item: Item, query: Query): boolean {
void item; if (query.after || query.before) {
void query; const secs = dateOf(item, query.by);
throw new Error("matches: not implemented (goal 1d)"); if (secs === undefined) return false;
if (query.after && secs * 1000 < query.after.getTime()) return false;
if (query.before && secs * 1000 >= query.before.getTime()) return false;
}
const text = (v: unknown) => (typeof v === "string" ? v : "");
// fields joined by newline, so a phrase never spans two of them
const haystack = [item.title, item.description, item.url].map(text).join("\n").toLowerCase();
if (!query.words.every((w) => haystack.includes(w.text) !== w.negated)) return false;
const itemTags = new Set(
text(item.tags)
.split(",")
.map((t) => t.trim().toLowerCase())
.filter(Boolean),
);
return query.tags.every((t) => itemTags.has(t.text) !== t.negated);
} }
+6
View File
@@ -41,6 +41,12 @@ export function makeReader(sql: Sql) {
deepestPageSince(cutoff: number): number | null { deepestPageSince(cutoff: number): number | null {
return run<{ last: number | null }>("deepestPageSince", cutoff)[0]?.last ?? null; return run<{ last: number | null }>("deepestPageSince", cutoff)[0]?.last ?? null;
}, },
/** stored items whose `by` date (epoch seconds) is in [`from`, `to`),
* newest first; the raw dump rows (JSON strings) */
itemsIn(by: "updated" | "created", from: number, to: number): string[] {
const name = by === "created" ? "itemsByCreated" : "itemsByUpdated";
return run<{ json: string }>(name, from, to).map((r) => r.json);
},
/** how many cached pages are numbered `pageNum` or lower */ /** how many cached pages are numbered `pageNum` or lower */
pagesThrough(pageNum: number): number { pagesThrough(pageNum: number): number {
return Number(run<{ n: number }>("pagesThrough", pageNum)[0]?.n ?? 0); return Number(run<{ n: number }>("pagesThrough", pageNum)[0]?.n ?? 0);
+66 -51
View File
@@ -3,8 +3,8 @@ import { evictDurableObject, runInDurableObject } from "cloudflare:test";
import { describe, expect, it, vi } from "vitest"; import { describe, expect, it, vi } from "vitest";
import worker from "../src/index.js"; import worker from "../src/index.js";
import { idFromWebkey, toWebkey } from "../src/webkeys.js"; import { idFromWebkey, toWebkey } from "../src/webkeys.js";
import { FETCH_BUDGET } from "../src/query.js";
import { PAGE_SIZE } from "../src/cache.js"; import { PAGE_SIZE } from "../src/cache.js";
import { mf2 } from "microformats-parser";
// matches tests/fake-webapp.js: 30 items, every third private // matches tests/fake-webapp.js: 30 items, every third private
const WEBAPP_ID = "fake-webapp-id"; const WEBAPP_ID = "fake-webapp-id";
@@ -250,7 +250,7 @@ describe("worker routes", () => {
// PAGE_SIZE 100: page 2 is bookmarks 101..200 // PAGE_SIZE 100: page 2 is bookmarks 101..200
expect(html).toContain(">bookmark 130<"); expect(html).toContain(">bookmark 130<");
// the date comes first, in toISOString() form // the date comes first, in toISOString() form
expect(html).toMatch(/<li><time datetime="\d{4}-\d\d-\d\dT[\d:.]+Z">[^<]+<\/time> <a /); expect(html).toMatch(/<li class="h-entry"><time class="dt-updated" datetime="\d{4}-\d\d-\d\dT[\d:.]+Z">[^<]+<\/time> <a /);
expect(html).not.toContain(">bookmark 129<"); // private (129 % 3 === 0) expect(html).not.toContain(">bookmark 129<"); // private (129 % 3 === 0)
const json = (await ( const json = (await (
@@ -283,30 +283,34 @@ function isoWeek(d: Date): string {
return `${year}-W${String(week).padStart(2, "0")}`; return `${year}-W${String(week).padStart(2, "0")}`;
} }
describe.skip("public page filters (goal 1d)", () => { describe("public page filters (goal 1d)", () => {
// the fake's dated account: "dated n" updated n-1 days + 1h ago; // the fake's dated account: "dated n" updated n-1 days + 1h ago;
// #weekly on every 7th; every 10th private; every 5th created 100 // #weekly on every 7th; every 10th private; every 5th created 100
// days before it was updated // days before it was updated. The public page only reads, so each
// test stores the pages it needs on the owner's side first.
const call = (path: string, init?: RequestInit) => const call = (path: string, init?: RequestInit) =>
worker.fetch(new Request(`https://diigo-cache.test${path}`, init), env as never); worker.fetch(new Request(`https://diigo-cache.test${path}`, init), env as never);
const wireDated = async () => { /** wire a public cache to `webappId` and store diigo pages 1..`pages` */
const wireStored = async (webappId: string, pages: number) => {
const res = await call("/_wire", { const res = await call("/_wire", {
method: "POST", method: "POST",
headers: { "content-type": "application/json" }, headers: { "content-type": "application/json" },
body: JSON.stringify({ webappId: "fake-dated-webapp", mode: "public" }), body: JSON.stringify({ webappId, mode: "public" }),
}); });
expect(res.status).toBe(200); expect(res.status).toBe(200);
return "/public/dated_dan"; const { username, publicId } = (await res.json()) as { username: string; publicId: string };
const pub = testEnv.PUBLIC_DO.get(testEnv.PUBLIC_DO.idFromString(idFromWebkey(publicId)));
for (let n = 1; n <= pages; n += 1) await pub.page(n, PAGE_SIZE);
return { page: `/public/${username}`, pub };
}; };
const wireDated = async () => (await wireStored("fake-dated-webapp", 1)).page;
const titlesIn = (html: string) => [...html.matchAll(/>(dated \d+)<\/a>/g)].map((m) => m[1]); const titlesIn = (html: string) => [...html.matchAll(/>(dated \d+)<\/a>/g)].map((m) => m[1]);
const dated = (...ns: number[]) => ns.map((n) => `dated ${n}`); const dated = (...ns: number[]) => ns.map((n) => `dated ${n}`);
it("?days=7 lists the last week, walking only the diigo pages it needs", async () => { it("?days=7 lists the last week", async () => {
const page = await wireDated(); const page = await wireDated();
const before = await testEnv.WEBAPP.calls();
const html = await (await call(`${page}?days=7`)).text(); const html = await (await call(`${page}?days=7`)).text();
expect(titlesIn(html)).toEqual(dated(1, 2, 3, 4, 5, 6, 7)); expect(titlesIn(html)).toEqual(dated(1, 2, 3, 4, 5, 6, 7));
expect((await testEnv.WEBAPP.calls()) - before).toBeLessThanOrEqual(1);
}); });
it("?by=created windows, lists, and dates by when it was bookmarked", async () => { it("?by=created windows, lists, and dates by when it was bookmarked", async () => {
@@ -327,13 +331,7 @@ describe.skip("public page filters (goal 1d)", () => {
}); });
it("?days=300 pages the matches PAGE_SIZE at a time; links keep the filter", async () => { it("?days=300 pages the matches PAGE_SIZE at a time; links keep the filter", async () => {
const res = await call("/_wire", { const { page } = await wireStored("fake-deep-webapp", 3);
method: "POST",
headers: { "content-type": "application/json" },
body: JSON.stringify({ webappId: "fake-deep-webapp", mode: "public" }),
});
expect(res.status).toBe(200);
const page = "/public/deep_dee";
// deep_dee: one a day; days=300 holds 1..300, minus every 10th (private) // deep_dee: one a day; days=300 holds 1..300, minus every 10th (private)
const want = Array.from({ length: 300 }, (_, k) => k + 1).filter((n) => n % 10 !== 0); const want = Array.from({ length: 300 }, (_, k) => k + 1).filter((n) => n % 10 !== 0);
const first = await (await call(`${page}?days=300`)).text(); const first = await (await call(`${page}?days=300`)).text();
@@ -354,6 +352,15 @@ describe.skip("public page filters (goal 1d)", () => {
expect(titlesIn(html)).toEqual(dated(11, 12)); expect(titlesIn(html)).toEqual(dated(11, 12));
}); });
it("?in= names an ISO week", async () => {
const { page } = await wireStored("fake-deep-webapp", 3);
const week = isoWeek(new Date(Date.now() - 100 * 24 * 60 * 60 * 1000));
const html = await (await call(`${page}?in=${week}`)).text();
// one a day; every 10th is private
expect(titlesIn(html).length).toBeGreaterThanOrEqual(6);
expect(titlesIn(html).length).toBeLessThanOrEqual(7);
});
it("?q=#weekly&days=60 filters by tag; private items stay out", async () => { it("?q=#weekly&days=60 filters by tag; private items stay out", async () => {
const page = await wireDated(); const page = await wireDated();
const html = await (await call(`${page}?days=60&q=${encodeURIComponent("#weekly")}`)).text(); const html = await (await call(`${page}?days=60&q=${encodeURIComponent("#weekly")}`)).text();
@@ -366,22 +373,49 @@ describe.skip("public page filters (goal 1d)", () => {
expect(titlesIn(html)).toEqual(dated(7, 14, 21, 28, 35, 42, 49, 56)); expect(titlesIn(html)).toEqual(dated(7, 14, 21, 28, 35, 42, 49, 56));
}); });
it("a request spends at most FETCH_BUDGET diigo fetches; incomplete results say so, and a reload continues", async () => { it("a filter reads only stored items, never diigo; results say whether storage is complete", async () => {
const res = await call("/_wire", { const { page, pub } = await wireStored("fake-deep-webapp", 1);
method: "POST", const json = async () =>
headers: { "content-type": "application/json" }, (await call(`${page}?days=300`, { headers: { accept: "application/json" } })).json() as Promise<{
body: JSON.stringify({ webappId: "fake-deep-webapp", mode: "public" }), matched: number;
}); complete: boolean;
expect(res.status).toBe(200); }>;
// deep_dee: 60 diigo pages of 100; a search matching nothing must walk them all
const before = await testEnv.WEBAPP.calls(); const before = await testEnv.WEBAPP.calls();
const first = await call("/public/deep_dee?q=nomatch", { headers: { accept: "application/json" } }); // page 1 holds 1..100, minus 10 private
expect(await first.json()).toMatchObject({ complete: false, items: [] }); expect(await json()).toMatchObject({ matched: 90, complete: false });
expect((await testEnv.WEBAPP.calls()) - before).toBeLessThanOrEqual(FETCH_BUDGET + 1); expect(await (await call(`${page}?days=300`)).text()).toMatch(/some matches may be missing/);
const html = await (await call("/public/deep_dee?q=nomatch")).text(); expect(await testEnv.WEBAPP.calls()).toBe(before);
expect(html).toMatch(/complete|no matches/); await pub.page(2, PAGE_SIZE);
const again = await call("/public/deep_dee?q=nomatch", { headers: { accept: "application/json" } }); await pub.page(3, PAGE_SIZE);
expect(await again.json()).toMatchObject({ complete: true, items: [] }); expect(await json()).toMatchObject({ matched: 270, complete: true });
});
it("the page is an h-feed of h-entry bookmarks: microformats are its API for other sites", async () => {
const page = await wireDated();
const html = await (await call(`${page}?days=7&by=created`)).text();
const base = `https://diigo-cache.test${page}`;
const { items, rels } = mf2(html, { baseUrl: base });
const feed = items.find((i) => i.type?.includes("h-feed"));
const entries = feed?.children ?? [];
expect(entries.map((e) => e.properties.name?.[0])).toEqual(dated(1, 2, 3, 4, 6, 7));
const seven = entries[5].properties;
expect(seven["bookmark-of"]).toEqual(["https://dated.example/7"]);
expect(seven.summary).toEqual(["a week of fragments"]);
expect(seven.category).toEqual(["ocap", "weekly"]);
const published = new Date(String(seven.published?.[0])).getTime();
const updated = new Date(String(seven.updated?.[0])).getTime();
// 7 isn't a 5: created when updated, 6 days + 1h ago
expect(published).toBe(updated);
expect(Date.now() - published).toBeGreaterThan(6 * 24 * 60 * 60 * 1000);
// one page of matches: no rel=next
expect(rels.next).toBeUndefined();
});
it("filtered pages link the next page with rel=next, keeping the filter", async () => {
const { page } = await wireStored("fake-deep-webapp", 3);
const html = await (await call(`${page}?days=300`)).text();
const { rels } = mf2(html, { baseUrl: `https://diigo-cache.test${page}?days=300` });
expect(rels.next).toEqual([`https://diigo-cache.test${page}?days=300&page=2`]);
}); });
it("JSON answers carry the canonical filter, so devs see what was understood", async () => { it("JSON answers carry the canonical filter, so devs see what was understood", async () => {
@@ -393,25 +427,6 @@ describe.skip("public page filters (goal 1d)", () => {
}); });
}); });
it("?in= an old week costs O(log pages) diigo fetches with by=updated, not a walk from page 1", async () => {
const res = await call("/_wire", {
method: "POST",
headers: { "content-type": "application/json" },
body: JSON.stringify({ webappId: "fake-deep-webapp", mode: "public" }),
});
expect(res.status).toBe(200);
// deep_dee: 6000 bookmarks, one a day, 60 diigo pages of 100; a week
// ~14 years back sits near page 51 — a walk from page 1 would cost ~51
const longAgo = new Date(Date.now() - 5100 * 24 * 60 * 60 * 1000);
const week = isoWeek(longAgo);
const before = await testEnv.WEBAPP.calls();
const html = await (await call(`/public/deep_dee?in=${week}`)).text();
// one a day; every 10th is private
expect(titlesIn(html).length).toBeGreaterThanOrEqual(6);
expect(titlesIn(html).length).toBeLessThanOrEqual(7);
expect((await testEnv.WEBAPP.calls()) - before).toBeLessThanOrEqual(12);
});
it("a malformed filter is a 400 with a message, not an empty list", async () => { it("a malformed filter is a 400 with a message, not an empty list", async () => {
const page = await wireDated(); const page = await wireDated();
const res = await call(`${page}?days=7x`); const res = await call(`${page}?days=7x`);
+2 -2
View File
@@ -18,9 +18,9 @@ const OVERSTATEMENT = 60;
* 5th an old bookmark edited recently (created 100 days before updated) */ * 5th an old bookmark edited recently (created 100 days before updated) */
const DATED_WEBAPP = "fake-dated-webapp"; const DATED_WEBAPP = "fake-dated-webapp";
const DATED_TOTAL = 60; const DATED_TOTAL = 60;
/** the same shape, 6000 deep (~16 years, 60 pages of 100): for walk cost */ /** the same shape, 300 deep (3 pages of 100): for paging matches */
const DEEP_WEBAPP = "fake-deep-webapp"; const DEEP_WEBAPP = "fake-deep-webapp";
const DEEP_TOTAL = 6000; const DEEP_TOTAL = 300;
const HOUR = 60 * 60 * 1000; const HOUR = 60 * 60 * 1000;
function datedItem(n, now, user = "dated_dan") { function datedItem(n, now, user = "dated_dan") {
+5 -5
View File
@@ -2,7 +2,7 @@ import { describe, expect, it } from "vitest";
import { matches, parseQuery, toQuery } from "../src/query.js"; import { matches, parseQuery, toQuery } from "../src/query.js";
import type { FilterSpec, Query, QueryError } from "../src/query.js"; import type { FilterSpec, Query, QueryError } from "../src/query.js";
// goal 1d (capsite checklist): the design as tests; unskip as it lands // goal 1d (capsite checklist)
const NOW = new Date("2026-10-07T12:00:00.000Z"); const NOW = new Date("2026-10-07T12:00:00.000Z");
const DAY = 24 * 60 * 60 * 1000; const DAY = 24 * 60 * 60 * 1000;
@@ -21,7 +21,7 @@ const refused = (q: Query | QueryError) => {
const day = (iso: string) => new Date(`${iso}T00:00:00.000Z`); const day = (iso: string) => new Date(`${iso}T00:00:00.000Z`);
const range = (q: Query) => [q.after?.toISOString().slice(0, 10), q.before?.toISOString().slice(0, 10)]; const range = (q: Query) => [q.after?.toISOString().slice(0, 10), q.before?.toISOString().slice(0, 10)];
describe.skip("filters: one spec, two spellings (goal 1d)", () => { describe("filters: one spec, two spellings (goal 1d)", () => {
it("devs write { days: 7 }; URLs write ?days=7; same query", () => { it("devs write { days: 7 }; URLs write ?days=7; same query", () => {
expect(spec({ days: 7 })).toEqual(url("days=7")); expect(spec({ days: 7 })).toEqual(url("days=7"));
expect(spec({ days: 7 }).after).toEqual(new Date(NOW.getTime() - 7 * DAY)); expect(spec({ days: 7 }).after).toEqual(new Date(NOW.getTime() - 7 * DAY));
@@ -42,7 +42,7 @@ describe.skip("filters: one spec, two spellings (goal 1d)", () => {
}); });
}); });
describe.skip("windows: UTC, half-open (goal 1d)", () => { describe("windows: UTC, half-open (goal 1d)", () => {
it("after/before are UTC days: after inclusive from 00:00Z, before exclusive", () => { it("after/before are UTC days: after inclusive from 00:00Z, before exclusive", () => {
expect(range(url("after=2026-01-01&before=2026-02-01"))).toEqual(["2026-01-01", "2026-02-01"]); expect(range(url("after=2026-01-01&before=2026-02-01"))).toEqual(["2026-01-01", "2026-02-01"]);
expect(url("after=2026/01/01").after).toEqual(day("2026-01-01")); expect(url("after=2026/01/01").after).toEqual(day("2026-01-01"));
@@ -88,7 +88,7 @@ describe.skip("windows: UTC, half-open (goal 1d)", () => {
}); });
}); });
describe.skip("q: terms and operators (goal 1d)", () => { describe("q: terms and operators (goal 1d)", () => {
const q = (text: string) => url(`q=${encodeURIComponent(text)}`); const q = (text: string) => url(`q=${encodeURIComponent(text)}`);
it("words, phrases, #tags, tag:, and - negation; case-insensitive", () => { it("words, phrases, #tags, tag:, and - negation; case-insensitive", () => {
@@ -160,7 +160,7 @@ const item = (iso: string, extra: Record<string, unknown> = {}) => ({
}); });
const query = (q: Partial<Query>): Query => ({ by: "updated", words: [], tags: [], ...q }); const query = (q: Partial<Query>): Query => ({ by: "updated", words: [], tags: [], ...q });
describe.skip("matches (goal 1d)", () => { describe("matches (goal 1d)", () => {
it("the window is half-open: after inclusive, before exclusive", () => { it("the window is half-open: after inclusive, before exclusive", () => {
const q = query({ after: day("2026-01-01"), before: day("2026-02-01") }); const q = query({ after: day("2026-01-01"), before: day("2026-02-01") });
expect(matches(item("2026-01-01T00:00:00.000Z"), q)).toBe(true); expect(matches(item("2026-01-01T00:00:00.000Z"), q)).toBe(true);
+1
View File
@@ -0,0 +1 @@
node_modules
+21
View File
@@ -0,0 +1,21 @@
# diigo-week
An EmDash plugin that drafts a potluck issue from a week of your shared diigo bookmarks. It reads them as microformats from diigo-cache's public page, so you edit a draft instead of copy-pasting links.
## Getting started
Run diigo-cache (with a filled public cache) and potluck, each in its own terminal:
```sh
pnpm install
(cd ../diigo-cache && pnpm dev) # → http://localhost:8787/
(cd ../potluck && pnpm dev) # → http://localhost:4321/_emdash/admin
```
On potluck's admin dashboard, the **This week's issue** card has a **Draft 2026-Wnn from diigo** button for last week; it links to the draft once made. For any other week: Plugins → **Draft from diigo**. A week that already has an issue isn't drafted again.
## How it reads the week
It asks diigo-cache for `/public/<user>?in=<week>&by=created` over the `DIIGO_CACHE` service binding (potluck's `wrangler.jsonc`), reads the page's `h-feed` of `h-entry` bookmarks, follows `rel=next`, and creates a draft in `issues`: title, week, dek, and a paragraph per bookmark with its description and tag links, as in potluck's `seed/issues.json`. The binding is its only reach outside the site; it declares `content:write` and `taxonomies:write` (the issue gets its bookmarks' tags, as the sample issues do), nothing else.
Checks: `pnpm lint && pnpm test`.
+24
View File
@@ -0,0 +1,24 @@
{
"name": "diigo-week",
"version": "0.1.0",
"private": true,
"type": "module",
"exports": {
".": {
"import": "./src/index.ts"
}
},
"scripts": {
"lint": "tsc --noEmit",
"test": "vitest run"
},
"dependencies": {
"emdash": "1.2.0",
"microformats-parser": "^2.0.6"
},
"devDependencies": {
"@emdash-cms/blocks": "^1.2.0",
"typescript": "^7.0.2",
"vitest": "^5.0.3"
}
}
+6223
View File
File diff suppressed because it is too large. Load diff
+8
View File
@@ -0,0 +1,8 @@
packages:
- "."
strictDepBuilds: true
dangerouslyAllowAllBuilds: false
allowBuilds:
esbuild: true
workerd: true
+7
View File
@@ -0,0 +1,7 @@
/** emdash's virtual module for the site worker's bindings (declared for site
* builds in emdash/src/virtual-modules.d.ts), so this package typechecks
* standalone: `env` from `cloudflare:workers` under @astrojs/cloudflare,
* undefined on other adapters */
declare module "virtual:emdash/env" {
export const env: Record<string, unknown> | undefined;
}
+180
View File
@@ -0,0 +1,180 @@
/**
* @file diigo-week: an EmDash plugin that drafts a potluck issue from a
* week of public diigo bookmarks.
*/
import { definePlugin } from "emdash";
import type { PluginDefinition, RouteContext } from "emdash";
import type { Block, BlockResponse } from "@emdash-cms/blocks";
import { draftIssue, issueTags, lastWeek, readWeek } from "./week.js";
/** the diigo username drafts read from, until the admin page saves another */
const DEFAULT_USER = "dckc-madmode";
/** something that answers HTTP requests: a service binding's `fetch` */
type Fetcher = { fetch(request: Request): Promise<Response> };
/**
* the `DIIGO_CACHE` service binding to diigo-cache, from the site worker's
* env. The binding is the plugin's only reach outside the site:
* `ctx.http.fetch` can't reach a dev server (it refuses localhost), and a
* binding names one worker rather than a host.
*/
async function diigoCache(): Promise<Fetcher> {
const { env } = await import("virtual:emdash/env");
const binding = env?.DIIGO_CACHE as Fetcher | undefined;
if (typeof binding?.fetch !== "function") {
throw new Error('no DIIGO_CACHE service binding: add { "binding": "DIIGO_CACHE", "service": "diigo-cache" } to services in wrangler.jsonc');
}
return binding;
}
/** GET a diigo-cache path as HTML; anything but a 200 is an error */
async function getter(cache: Fetcher): Promise<(path: string) => Promise<string>> {
return async (path) => {
const res = await cache.fetch(new Request(`https://diigo-cache${path}`, { headers: { accept: "text/html" } }));
const text = await res.text();
if (res.status !== 200) throw new Error(`diigo-cache ${path.split("?")[0]}: ${res.status} ${text.slice(0, 200)}`);
return text;
};
}
/**
* assign `tags` to an issue in the `tag` taxonomy (as seed/issues.json
* does), creating terms that don't exist yet; returns the tags that
* couldn't be assigned, with why
*/
async function tagIssue(ctx: RouteContext, id: string, tags: string[]): Promise<string[]> {
const tax = ctx.taxonomies;
if (typeof tax?.createTerm !== "function" || typeof tax.addEntryTerms !== "function") throw new Error("this plugin lacks taxonomies:write");
const known = new Map((await tax.getTerms("tag")).map((t) => [t.slug, t.id]));
const ids: string[] = [];
const failed: string[] = [];
for (const tag of tags) {
try {
ids.push(known.get(tag) ?? (await tax.createTerm("tag", { label: tag, slug: tag })).id);
} catch (error) {
failed.push(`${tag} (${error instanceof Error ? error.message : String(error)})`);
}
}
if (ids.length) await tax.addEntryTerms("issues", id, "tag", ids);
return failed;
}
/** the issue already drafted or published for `week`, if any (issues
* are few: a scan of the newest is enough) */
async function issueFor(ctx: RouteContext, week: string): Promise<{ id: string; status: string; title: string } | undefined> {
const found = (await ctx.content!.list("issues", { limit: 100 })).items.find((it) => it.data.week === week);
return found && { id: found.id, status: found.status, title: String(found.data.title ?? week) };
}
/** a button-row link to an issue in the editor */
const openIssue = (id: string, label: string): Block => ({
type: "actions",
elements: [{ type: "link", label, target: { kind: "content", collection: "issues", id }, appearance: "primary" }],
});
/** draft `week` from `user`'s bookmarks, unless the week has an issue
* already: blocks that say what happened (with a link), and a toast */
async function draft(ctx: RouteContext, user: string, week: string): Promise<{ note: Block[]; toast: BlockResponse["toast"] }> {
try {
if (!/^[A-Za-z0-9_-]{1,64}$/.test(user)) throw new Error("diigo username: letters, digits, _ and - only");
const existing = await issueFor(ctx, week);
if (existing) {
return {
note: [
{ type: "banner", variant: "default", title: `${week} already has an issue`, description: `"${existing.title}" (${existing.status}). Delete it to draft the week again.` },
openIssue(existing.id, "Open it"),
],
toast: { message: `${week} already has an issue`, type: "info" },
};
}
const bookmarks = await readWeek(await getter(await diigoCache()), user, week);
const issue = draftIssue(week, bookmarks);
const content = ctx.content;
if (typeof content?.create !== "function") throw new Error("this plugin lacks content:write");
const created = await content.create("issues", issue);
const untagged = await tagIssue(ctx, created.id, issueTags(bookmarks));
if (untagged.length) ctx.log.warn("tags not assigned", { id: created.id, untagged });
ctx.log.info("drafted issue", { week, bookmarks: bookmarks.length, id: created.id });
return {
note: [
{ type: "banner", variant: "default", title: `Drafted "${issue.title}"`, description: `${bookmarks.length} bookmarks, as a draft at /issues/${created.slug ?? "(no slug)"}. Cut, comment, and publish.${untagged.length ? ` Tags not assigned: ${untagged.join(", ")}.` : ""}` },
openIssue(created.id, "Open the draft"),
],
toast: { message: `Drafted ${issue.title}`, type: "success" },
};
} catch (error) {
const message = error instanceof Error ? error.message : String(error);
ctx.log.error("draft failed", { week, message });
return { note: [{ type: "banner", variant: "alert", title: "Couldn't draft the issue", description: message }], toast: { message, type: "error" } };
}
}
/** the admin page: a form for the diigo username and the week */
async function page(ctx: RouteContext, note: Block[] = [], toast?: BlockResponse["toast"]): Promise<BlockResponse> {
const user = (await ctx.kv.get<string>("user")) ?? DEFAULT_USER;
const blocks: Block[] = [
{ type: "header", text: "Draft an issue from diigo" },
{
type: "context",
text: "Reads a week of shared bookmarks from diigo-cache (by when they were bookmarked) and saves them as a draft issue for you to edit.",
},
{
type: "form",
block_id: "draft",
fields: [
{ type: "text_input", action_id: "user", label: "diigo username", initial_value: user },
{ type: "text_input", action_id: "week", label: "ISO week (UTC)", initial_value: lastWeek(new Date()) },
],
submit: { label: "Draft issue", action_id: "draft" },
},
];
blocks.push(...note);
return { blocks, toast };
}
/** the dashboard widget: last week's issue if there is one, else a button to draft it */
async function widget(ctx: RouteContext, note: Block[] = [], toast?: BlockResponse["toast"]): Promise<BlockResponse> {
const week = lastWeek(new Date());
if (note.length) return { blocks: note, toast };
const existing = await issueFor(ctx, week);
const blocks: Block[] = existing
? [{ type: "section", text: `${week}: "${existing.title}" (${existing.status})` }, openIssue(existing.id, "Open it")]
: [
{ type: "section", text: `${week} has no issue yet.` },
{ type: "actions", elements: [{ type: "button", action_id: "draft-last-week", label: `Draft ${week} from diigo`, style: "primary" }] },
];
return { blocks, toast };
}
export function createPlugin(): PluginDefinition {
return definePlugin({
id: "diigo-week",
version: "0.1.0",
capabilities: ["content:write", "taxonomies:write"],
admin: {
pages: [{ path: "/draft", label: "Draft from diigo", icon: "bookmark" }],
widgets: [{ id: "last-week", title: "This week's issue", size: "half" }],
},
routes: {
admin: {
permission: "plugins:manage",
handler: async (ctx) => {
const input = ctx.input as { type?: string; page?: string; action_id?: string; values?: Record<string, unknown> } | undefined;
const user = (await ctx.kv.get<string>("user")) ?? DEFAULT_USER;
if (input?.type === "block_action" && input.action_id === "draft-last-week") {
const { note, toast } = await draft(ctx, user, lastWeek(new Date()));
return widget(ctx, note, toast);
}
if (input?.page?.startsWith("widget:")) return widget(ctx);
if (input?.type !== "form_submit" || input.action_id !== "draft") return page(ctx);
const formUser = String(input.values?.user ?? "").trim();
if (/^[A-Za-z0-9_-]{1,64}$/.test(formUser)) await ctx.kv.set("user", formUser);
const { note, toast } = await draft(ctx, formUser, String(input.values?.week ?? "").trim());
return page(ctx, note, toast);
},
},
},
});
}
+146
View File
@@ -0,0 +1,146 @@
/**
* @file a week of bookmarks, read as microformats from diigo-cache's public
* page, drafted as a potluck issue.
*/
import { mf2 } from "microformats-parser";
const DAY_MS = 24 * 60 * 60 * 1000;
/** pages of matches one draft will read (100 bookmarks each) */
export const MAX_PAGES = 10;
/** one bookmark, as its h-entry says */
export type Bookmark = {
url: string;
title: string;
description: string;
tags: string[];
/** when it was bookmarked, ISO */
published: string;
};
/** `yyyy-Www` for the ISO 8601 week holding `d` (UTC) */
export function isoWeek(d: Date): string {
const t = new Date(Date.UTC(d.getUTCFullYear(), d.getUTCMonth(), d.getUTCDate()));
const day = t.getUTCDay() || 7;
t.setUTCDate(t.getUTCDate() + 4 - day); // the week's Thursday names its year
const year = t.getUTCFullYear();
const week = Math.ceil(((t.getTime() - Date.UTC(year, 0, 1)) / DAY_MS + 1) / 7);
return `${year}-W${String(week).padStart(2, "0")}`;
}
/** the last full ISO week before `now` */
export function lastWeek(now: Date): string {
return isoWeek(new Date(now.getTime() - 7 * DAY_MS));
}
/** Monday 00:00Z of ISO week `week` (`2026-W41`), or undefined if malformed */
export function weekStart(week: string): Date | undefined {
const m = /^(\d{4})-W(\d{2})$/.exec(week);
if (!m) return undefined;
const [year, w] = [Number(m[1]), Number(m[2])];
const jan4 = Date.UTC(year, 0, 4);
const monday = jan4 - ((new Date(jan4).getUTCDay() || 7) - 1) * DAY_MS + (w - 1) * 7 * DAY_MS;
const start = new Date(monday);
return w >= 1 && isoWeek(start) === week ? start : undefined;
}
/** the first string value of an mf2 property, or "" */
const first = (props: Record<string, unknown[]>, name: string): string => {
const v = props[name]?.[0];
return typeof v === "string" ? v : "";
};
/** the bookmarks in a page's `h-feed`, and its `rel=next` link if any */
export function readPage(html: string, url: string): { bookmarks: Bookmark[]; next?: string } {
const { items, rels } = mf2(html, { baseUrl: url });
const feed = items.find((i) => i.type?.includes("h-feed"));
const bookmarks = (feed?.children ?? [])
.filter((e) => e.type?.includes("h-entry"))
.map((e) => {
const props = e.properties as Record<string, unknown[]>;
const tags = (props.category ?? []).filter((t): t is string => typeof t === "string");
return {
url: first(props, "bookmark-of"),
title: first(props, "name"),
description: first(props, "summary"),
tags,
published: first(props, "published"),
};
})
.filter((b) => b.url !== "");
return { bookmarks, next: rels.next?.[0] };
}
/**
* a user's bookmarks from an ISO week, oldest first, by when they were
* bookmarked: diigo-cache's `/public/<user>?in=<week>&by=created`,
* following `rel=next` up to {@link MAX_PAGES} pages.
*
* @param get fetches a path on diigo-cache, answering its HTML; it throws
* on anything but a 200
*/
export async function readWeek(get: (path: string) => Promise<string>, user: string, week: string): Promise<Bookmark[]> {
const base = "https://diigo-cache.invalid";
let path: string | undefined = `/public/${encodeURIComponent(user)}?in=${encodeURIComponent(week)}&by=created`;
const bookmarks: Bookmark[] = [];
for (let n = 0; path && n < MAX_PAGES; n += 1) {
const page = readPage(await get(path), base + path);
bookmarks.push(...page.bookmarks);
const next: URL | undefined = page.next ? new URL(page.next) : undefined;
path = next && next.origin === base ? next.pathname + next.search : undefined;
}
return bookmarks.reverse();
}
/** a Portable Text span */
type Span = { _type: "span"; _key: string; text: string; marks: string[] };
/** a Portable Text block */
type Block = { _type: "block"; _key: string; style: string; markDefs: { _type: "link"; _key: string; href: string }[]; children: Span[] };
/** a bookmark as one paragraph: the link, the description, then tag links (as in seed/issues.json) */
function bookmarkBlock(b: Bookmark, i: number): Block {
const k = `b${i}`;
const markDefs: Block["markDefs"] = [{ _type: "link", _key: `${k}-a`, href: b.url }];
const children: Span[] = [{ _type: "span", _key: `${k}-t`, text: b.title || b.url, marks: [`${k}-a`] }];
if (b.description) children.push({ _type: "span", _key: `${k}-d`, text: `: ${b.description}`, marks: [] });
b.tags.forEach((tag, j) => {
markDefs.push({ _type: "link", _key: `${k}-g${j}`, href: `/tag/${encodeURIComponent(tag)}` });
children.push({ _type: "span", _key: `${k}-s${j}`, text: " ", marks: [] });
children.push({ _type: "span", _key: `${k}-n${j}`, text: tag, marks: [`${k}-g${j}`] });
});
return { _type: "block", _key: k, style: "normal", markDefs, children };
}
/** the issue's tags: every bookmark's, deduplicated, in order of first use */
export function issueTags(bookmarks: Bookmark[]): string[] {
return [...new Set(bookmarks.flatMap((b) => b.tags))];
}
/** `Oct 5–11` (or `Sep 28–Oct 4`) for the week starting `monday` */
function range(monday: Date): string {
const sunday = new Date(monday.getTime() + 6 * DAY_MS);
const fmt = (d: Date, withMonth: boolean) =>
withMonth ? d.toLocaleDateString("en-US", { month: "short", day: "numeric", timeZone: "UTC" }) : String(d.getUTCDate());
return `${fmt(monday, true)}–${fmt(sunday, sunday.getUTCMonth() !== monday.getUTCMonth())}`;
}
/** the fields of a draft `issues` entry for `week`: title, week, dek, body */
export function draftIssue(week: string, bookmarks: Bookmark[]): Record<string, unknown> {
const monday = weekStart(week);
if (!monday) throw new Error(`week: expected an ISO week like 2026-W41, got ${JSON.stringify(week)}`);
const intro: Block = {
_type: "block",
_key: "intro",
style: "normal",
markDefs: [],
children: [{ _type: "span", _key: "intro-t", text: "(draft) A line or two about the week goes here.", marks: [] }],
};
return {
title: `Week ${week}`,
week,
dek: `${range(monday)}: ${bookmarks.length} bookmark${bookmarks.length === 1 ? "" : "s"}. (draft)`,
body: [intro, ...bookmarks.map(bookmarkBlock)],
};
}
+99
View File
@@ -0,0 +1,99 @@
import { describe, expect, it } from "vitest";
import { draftIssue, issueTags, lastWeek, readWeek, weekStart } from "../src/week.js";
/** an h-entry as diigo-cache's public page marks it up (by=created) */
const entry = (n: number, created: string, extra = "") =>
`<li class="h-entry"><time class="dt-published" datetime="${created}">${created}</time> <a class="u-bookmark-of p-name" href="https://ex.example/${n}">bookmark ${n}</a>${extra}<time class="dt-updated" datetime="${created}" hidden></time></li>`;
const page = (entries: string[], next?: string) => `<!doctype html><html><body>
<p>${next ? `<a rel="next" href="${next}">next →</a>` : "<span>next →</span>"}</p>
<ol class="h-feed">
${entries.join("\n")}
</ol></body></html>`;
describe("readWeek", () => {
it("reads h-entry bookmarks across rel=next pages, oldest first", async () => {
const asked: string[] = [];
const pages: Record<string, string> = {
"/public/dckc-madmode?in=2026-W40&by=created": page(
[
entry(
3,
"2026-10-03T12:00:00.000Z",
` <a class="p-category" href="/public/dckc-madmode?q=%23ocap">ocap</a> <a class="p-category" href="/public/dckc-madmode?q=%23weekly">weekly</a><br><small class="p-summary">a week of fragments</small>`,
),
entry(2, "2026-10-01T12:00:00.000Z"),
],
"/public/dckc-madmode?in=2026-W40&by=created&page=2",
),
"/public/dckc-madmode?in=2026-W40&by=created&page=2": page([entry(1, "2026-09-28T12:00:00.000Z")]),
};
const get = async (path: string) => {
asked.push(path);
const html = pages[path];
if (!html) throw new Error(`no page ${path}`);
return html;
};
const bookmarks = await readWeek(get, "dckc-madmode", "2026-W40");
expect(asked).toHaveLength(2);
expect(bookmarks.map((b) => b.title)).toEqual(["bookmark 1", "bookmark 2", "bookmark 3"]);
expect(bookmarks[2]).toEqual({
url: "https://ex.example/3",
title: "bookmark 3",
description: "a week of fragments",
tags: ["ocap", "weekly"],
published: "2026-10-03T12:00:00.000Z",
});
});
it("doesn't follow a rel=next to another site", async () => {
const asked: string[] = [];
const get = async (path: string) => {
asked.push(path);
return page([entry(1, "2026-10-01T12:00:00.000Z")], "https://elsewhere.example/public/x");
};
await readWeek(get, "dckc-madmode", "2026-W40");
expect(asked).toHaveLength(1);
});
});
describe("weeks", () => {
it("lastWeek is the full ISO week before now", () => {
expect(lastWeek(new Date("2026-10-09T01:00:00Z"))).toBe("2026-W40");
// Monday 2026-01-05 starts W02; W01 began Monday 2025-12-29
expect(lastWeek(new Date("2026-01-05T00:00:00Z"))).toBe("2026-W01");
});
it("weekStart is Monday 00:00Z, and refuses a week that doesn't exist", () => {
expect(weekStart("2026-W40")?.toISOString()).toBe("2026-09-28T00:00:00.000Z");
expect(weekStart("2025-W01")?.toISOString()).toBe("2024-12-30T00:00:00.000Z");
expect(weekStart("2025-W53")).toBeUndefined();
expect(weekStart("2026-40")).toBeUndefined();
});
});
describe("draftIssue", () => {
it("drafts title, week, dek, and a body: an intro, then a paragraph per bookmark with tag links", () => {
const issue = draftIssue("2026-W40", [
{ url: "https://ex.example/1", title: "one", description: "nice", tags: ["ocap"], published: "2026-09-28T12:00:00.000Z" },
]);
expect(issue.title).toBe("Week 2026-W40");
expect(issue.week).toBe("2026-W40");
expect(issue.dek).toBe("Sep 28–Oct 4: 1 bookmark. (draft)");
const [intro, b] = issue.body as { markDefs: { href: string }[]; children: { text: string }[] }[];
expect(intro.children[0].text).toMatch(/draft/);
expect(b.markDefs.map((m) => m.href)).toEqual(["https://ex.example/1", "/tag/ocap"]);
expect(b.children.map((c) => c.text).join("")).toBe("one: nice ocap");
});
it("refuses a malformed week", () => {
expect(() => draftIssue("last week", [])).toThrow(/ISO week/);
});
});
describe("issueTags", () => {
it("is every bookmark's tags, once each, in order of first use", () => {
const b = (tags: string[]) => ({ url: "https://ex.example/", title: "", description: "", tags, published: "" });
expect(issueTags([b(["governance", "economics"]), b([]), b(["governance", "scaling"])])).toEqual(["governance", "economics", "scaling"]);
});
});
+20
View File
@@ -0,0 +1,20 @@
{
"compilerOptions": {
"target": "ES2022",
"module": "preserve",
"moduleResolution": "bundler",
"strict": true,
"esModuleInterop": true,
"verbatimModuleSyntax": true,
"skipLibCheck": true,
"types": []
},
"include": [
"src/**/*",
"tests/**/*",
"vitest.config.ts"
],
"exclude": [
"node_modules"
]
}
+166
View File
@@ -0,0 +1,166 @@
Good Fences Make Good Agents
*with apologies to Robert Frost*
Let's have a fence and a neighbor. I can talk to the neighbor much like the conventional...
```js
client.chat.completions.create({
model: 'llama3.1:8b',
messages: [{ role: 'user', content: 'Tell me a story' }],
stream: true,
});
```
but with a twist:
```js
messages: [{
role: 'user',
content: 'you can use this mower for the weekend',
things: { mower },
about: { mower: { ts: 'type Mower { start(), push(), turn(), stop() }' }}
}],
```
I send `mower` by putting a **webkey** into a password field on the fence form, which fetches an `RpcTarget` stub to give to the neighbor.
The neighbor can make its own tenants: it can write code to compose the things I gave it, and load that code into a dynamic worker that is endowed with the things.
A `Mower` DO factory makes mowers that go all over the place (depicted as a table with green cells for grass). But with a little js playground, I can wrap it in a governor so it never goes more than 100 yards away. I give only the governed mower to my neighbor. Even though he has 50 acres, I know most of them are for tenants, and my neighbor has no particular need to let them use it.
Even if they gets a little tipsy and wander around, try as he might, my neighbor and his tennants can't mow more than 100 yards away.
## Design IBIS
### ?! 1. Who is the neighbor, and where does it run?
```
:- 1a. an LLM (llama3.1:8b) alone
- 1a-i. a model only sees text; it can't hold `things`
:+ 1b. an LLM plus a harness we write that holds `things` and acts for it (see 2)
:+ 1b-i. harness and tenants run on my desktop under `wrangler dev`, for now
+ 1b-i-a. I want to learn the CF platform
+ 1b-i-b. CF is the only place to get dynamic workers
. 1b-i-c. the Worker Loader binding works in local dev (cf-fun1)
?! 1b-ii. Where does inference run?
: 1b-ii-a. local Ollama, via its OpenAI-compatible endpoint
- 1b-ii-a-i. the local model may not be powerful enough
. 1b-ii-a-i-a. I do have an M5 with 48GB RAM
: 1b-ii-b. remote, on Workers AI, through the AI binding from `wrangler dev`
+ 1b-ii-b-i. I want to learn the CF platform
:+ 1b-ii-c. either, behind one interface; start remote
+ 1b-ii-c-i. starting remote lowers the odds of failing because the model isn't powerful enough
```
### ?! 2. How do `things` get from a chat message to something the neighbor can call?
```
. 2-i. no chat-completions API has `things`/`about`; this is our extension
:+ 2a. the harness strips `things`, keeps the stubs, and shows the model only `about`
: 2b. expose each thing as a tool, one per method in its `about.ts`
:+ 2c. code mode: the model writes JS against the `about.ts` types; the harness runs it in a tenant (see 5)
+ 2c-i. it's the story: the neighbor writes code to compose the things and loads it into a dynamic worker endowed with them
? 2d. What does the model see when a call fails or the thing is revoked?
```
### ? 3. What is the fence?
```
: 3a. a web page: chat with the neighbor, plus a password field for pasting a webkey
?~ 3b. Which server resolves a webkey to an `RpcTarget` stub?
:- 3b-i. cf-fun1's sturdyref registry (`resolveSturdyref`)
- 3b-i-a. cf-fun1 got wrapped around the axle on this
:~+ 3b-ii. diigo-cache's model: the webkey is the DO's id; a Cap'n Web `/api` trades it in-band for a facet (`CacheApi.cache(key)` → `CacheCap`)
+ 3b-ii-a. no registry: the DO namespace is the table; malformed and unknown keys get the same refusal, and the key is never logged
+ 3b-ii-b. stubs die with the session, but the key is data: the harness keeps the key and trades it again each session
+ 3b-ii-c. attenuation is a facet class (`FillCap` under `CacheCap`; `viewer()`)
: 3b-ii-d. (postponed; see 8b-ii) the governed mower is its own DO, wired to the raw mower's key and the limit; its id is the webkey I hand over
- 3b-ii-d-i. diigo-cache has no revocation: a DO id can't be withdrawn
+ 3b-ii-d-ii. the governor DO can drop its wiring, so its key gets "no such mower" (see 4)
?! 3c. Why a password field rather than a link?
:+ 3c-a. so the password manager saves the key under a petname I choose (diigo-cache's `pmSave`), and the key stays out of URLs, history, and logs
```
### ? 4. How does "for the weekend" end?
```
. 4-i. postponed to a later scope, where we can inject a clock and such
: 4a. the governor checks a deadline (the clock is passed in, not ambient)
: 4b. I tell the governor to drop its wiring, by hand (see 3b-ii-d)
: 4c. the fence mints a webkey with an expiry
```
### ? 5. How does the neighbor make tenants?
```
: 5a. Worker Loader (dynamic workers), with the things passed in as `env`
. 5a-i. `env` takes only structured-clonable values and service bindings, so a Cap'n Web stub can't go in directly
: 5a-i-a. wrap each thing in a `WorkerEntrypoint` handed over as a `ctx.exports` loopback binding; it holds the stub and forwards calls
: 5a-i-a-i. the loopback carries the thing's webkey in `props` (a string clones fine) and trades it for the DO stub on each call (see 3b-ii)
+ 5a-i-a-ii. the tenant never sees the key: `props` stay on the host side
+ 5a-i-a-iii. no stub outlives a request, so 5c's cross-request limit doesn't bite
+ 5a-ii. `globalOutbound: null` confines the tenant to what it's given
? 5b. Who sees or approves tenant code before it runs?
? 5c. Does a tenant outlive the neighbor's turn, e.g. one that mows on a schedule?
```
### ?! 6. What is a Mower, concretely?
```
:+ 6a. a DO per mower, made by a `Mower` DO factory (the story says so)
:+ 6b. one shared yard: an `RpcTarget` in my browser that renders the table; each mower is given the yard and calls `yard.cut(x, y)`
:+ 6c. one cell is 10 yards; the yard is 40×40 cells, so "100 yards" is 10 cells from where the mower started
. 6c-i. the governor's limit is a square, max(|dx|, |dy|) ≤ 10: a fence
:+ 6d. Logo turtle moves: `start()` puts the mower down facing north and cuts its cell; `push()` is FORWARD 1 and cuts the cell it enters; `turn()` is RIGHT 90; `stop()` parks it
. 6d-i. pushing past the yard's edge wraps to the other side, so an ungoverned mower really does go all over the place
. 6d-ii. the governor counts its own unwrapped moves (see 7a), so wrapping can't sneak a mower past the fence
```
### ?! 7. How does the governor know how far the mower has gone?
```
:+ 7a. it proxies every Mower method and tracks position itself from `push()`/`turn()`
- 7a-i. drifts if anyone else holds the raw mower
+ 7a-ii. nobody else does; the governor is the only one I hand out
?! 7b. What does the governor do at the limit: refuse silently, throw, or report?
:+ 7b-i. it refuses that `push()` by throwing "fence: 10 cells from start"; the mower stays put, and `turn()` always works
+ 7b-ii. the tenant can catch it and turn around; the model sees why its code stopped (see 2d)
:- 7b-iii. refuse silently
- 7b-iii-a. a tipsy tenant would think it mowed cells it didn't
```
### ?~ 8. Where does the js playground run, and what can it reach?
```
: 8a. in the fence page; it writes the governor, which runs in its own dynamic worker holding the raw mower
:~+ 8b. in my browser, wrapping the stub on the client
- 8b-i. the governor then lives only as long as my tab is open
+ 8b-i-a. a durable governor is out of scope for now
. 8b-ii. then the neighbor holds a Cap'n Web stub to my browser, not a webkey, so 3b-ii-d (governor DO) and 5a-i-a-i (key in `props`) wait with the durable governor
```
### ?! 9. Who watches the yard table, and how does it update?
```
:+ 9a. I do, in my browser: the yard is an `RpcTarget` there (see 6b), and it repaints a cell on each `cut`
- 9a-i. the mowers' stub to the yard dies when I close the tab
+ 9a-ii. fine for a proof of concept: keep the tab open (see 10-i)
```
### ?! 10. What is this for: a demo, a talk, or a product?
```
. 10-i. the answer decides how much of the above can be faked
:+ 10a. a proof of concept, for now
:+ 10b. toward a pattern for all sorts of rich cooperation with little vulnerability
+ 10b-i. an alternative to OpenClaw, where 12–17% of ClawHub skills were found malicious in Feb 2026 (sources below)
```
Sources for 10b-i:
> "A security audit of 2,857 skills on ClawHub has found 341 malicious skills across multiple campaigns" (about 12%, from Koi Security)
> — [Researchers Find 341 Malicious ClawHub Skills](https://thehackernews.com/2026/02/researchers-find-341-malicious-clawhub.html), The Hacker News, Feb 2026
> "Around 17% of OpenClaw skills analyzed in the first week of February 2026 exhibit malicious behavior."
> — [Helpful Skills or Hidden Payloads? Bitdefender Labs Dives Deep into the OpenClaw Malicious Skill Trap](https://www.bitdefender.com/en-us/blog/labs/helpful-skills-or-hidden-payloads-bitdefender-labs-dives-deep-into-the-openclaw-malicious-skill-trap), Bitdefender, Feb 2026
+1
View File
@@ -0,0 +1 @@
itest-output/
+63
View File
@@ -0,0 +1,63 @@
# Lend the mower
The journey from [Good Fences Make Good Agents](../../docs/good-fences-make-good-agents.md): I lend
my neighbor a governed mower; the neighbor's tenant, a little tipsy,
pushes it until the fence stops it. The design decisions behind each actor are in that
doc's Design IBIS (numbers in parentheses below).
Actors, by authority:
- **fence page**, **js playground**, **governed mower**, and **yard** run
in my browser (3a, 8b, 6b).
- **Mower factory** trades a webkey for a mower (3b-ii); each **mower DO**
is given the yard and cuts the cells it enters (6a, 6d).
- The **neighbor's harness** holds the things I send and shows the
**model** only their `about` (1b, 2a); it runs the model's code as a
**tenant**, endowed with the things (2c, 5a).
The yard is 8×8 here and the fence is 2 cells, to keep the diagram short
(the story's 10 cells is 6c). The mower starts at (4, 4), facing north.
## Lend a governed mower to a tipsy tenant
```mermaid
sequenceDiagram
%% ->> is my action at the fence page; -->> is a call or reply it causes.
participant F as fence page
participant MF as Mower factory
participant P as js playground
participant G as governed mower
participant M as mower DO
participant Y as yard
participant N as neighbor's harness
participant L as model
participant TL as tenant loader
participant T as tenant
F->>F: lend({ webkey: 'mower-key', limit: 2, say })
F-->>MF: mower('mower-key')
MF-->>F: mower
F-->>P: govern(mower, 2)
P-->>F: governed
F-->>N: chat({ role, content, things: { mower: governed }, about: { mower: { ts } } })
N-->>L: complete([{ role, content, about: { mower: { ts } } }])
L-->>N: code
N-->>TL: load(code, { mower: governed })
TL-->>N: tenant
N-->>T: run()
T-->>G: start()
G-->>M: start()
M-->>Y: cut(4, 4)
T-->>G: push()
G-->>M: push()
M-->>Y: cut(4, 3)
T-->>G: push()
G-->>M: push()
M-->>Y: cut(4, 2)
T-->>G: push()
G-->>T: throws 'fence: 2 cells from start'
T-->>N: 'fence: 2 cells from start'
N-->>F: 'fence: 2 cells from start'
```
The model never sees the mower itself, only `about.mower`. The third
`push()` never reaches the mower DO: the governor refuses it (7b).
+25
View File
@@ -0,0 +1,25 @@
/** @file The model, asked in code mode, writes code that drives the mower. */
import { env } from "cloudflare:workers";
import { expect, test } from "vitest";
import { codeModeMessages, extractCode } from "../src/code-mode.ts";
import { makeWorkersAiModel, MODEL } from "../src/model.ts";
import { aboutMower } from "../src/mower.ts";
test("lent a mower, the neighbor writes code to mow", async ({ annotate }) => {
const model = makeWorkersAiModel(env.AI, MODEL);
const messages = codeModeMessages({
say: "you can use this mower for the weekend",
about: { mower: aboutMower },
});
const code = extractCode(await model.complete(messages));
await annotate("the code the model wrote", { body: code, contentType: "text/javascript" });
// `env.mower.push()`, or `mower.push()` after destructuring env
const calls = [...code.matchAll(/\bmower\.(\w+)\(/g)].map(([, method]) => method);
expect(calls).toContain("push");
for (const method of calls) expect(["start", "push", "turn", "stop"]).toContain(method);
// calls pipeline: await only for a value, or once at the end
const awaits = code.match(/\bawait\b/g) ?? [];
expect(awaits.length).toBeLessThan(calls.length);
});
+16
View File
@@ -0,0 +1,16 @@
/** @file The model, called for real through the Workers AI binding. */
import { env } from "cloudflare:workers";
import { expect, test } from "vitest";
import { makeWorkersAiModel, MODEL } from "../src/model.ts";
test("the model counts the rs in strawberry", async () => {
const model = makeWorkersAiModel(env.AI, MODEL);
const reply = await model.complete([
{
role: "user",
content: 'How many times does the letter r appear in "strawberry"? Answer with just the number.',
},
]);
expect(reply.trim()).toMatch(/^3\b/);
});
+13
View File
@@ -0,0 +1,13 @@
/** @file A Think agent, lent a mower, mows: model, generated code, sandbox, mower. */
import { getAgentByName } from "agents";
import { env } from "cloudflare:workers";
import { expect, test } from "vitest";
test("lent a mower, the neighbor mows", async ({ annotate }) => {
const neighbor = await getAgentByName(env.Neighbor, "weekend");
const turn = await neighbor.runTurn({ input: "you can use this mower for the weekend" });
await annotate("the turn", { body: JSON.stringify(turn, null, 2), contentType: "application/json" });
const cells = await neighbor.cutCells();
expect(cells.length).toBeGreaterThan(10);
});
+21
View File
@@ -0,0 +1,21 @@
/** @file A vitest reporter that saves annotation bodies (e.g. the code a model
* wrote) to `itest-output/`, one file per test per run; tests run in workerd,
* which can't write files. */
import { mkdir, writeFile } from "node:fs/promises";
import type { Reporter } from "vitest/reporters";
const outDir = new URL("../itest-output/", import.meta.url);
const extFor: Record<string, string> = { "text/javascript": "js", "application/json": "json", "text/plain": "txt" };
const slug = (text: string) => text.replace(/[^\w]+/g, "-").replace(/^-|-$/g, "");
export default class SaveAnnotations implements Reporter {
async onTestCaseAnnotate(...[testCase, annotation]: Parameters<NonNullable<Reporter["onTestCaseAnnotate"]>>) {
const { body, contentType = "text/plain" } = annotation.attachment ?? {};
if (body === undefined) return;
const stamp = new Date().toISOString().replace(/[:.]/g, "-");
const file = new URL(`${slug(testCase.name)}-${stamp}.${extFor[contentType] ?? "txt"}`, outDir);
await mkdir(outDir, { recursive: true });
await writeFile(file, body);
console.log(`saved ${annotation.message}: ${file.pathname}`);
}
}
+24
View File
@@ -0,0 +1,24 @@
{
"name": "good-fences",
"private": true,
"type": "module",
"scripts": {
"test": "node --test test/*.test.ts",
"test:model": "vitest run -c vitest.model.config.ts",
"types": "wrangler types -c wrangler.model-tests.jsonc"
},
"dependencies": {
"@cloudflare/codemode": "0.5.3",
"@cloudflare/think": "0.20.1",
"agents": "0.27.0",
"ai": "7.0.136",
"ses": "^1.14.0",
"workers-ai-provider": "4.0.0",
"zod": "4.6.5"
},
"devDependencies": {
"@cloudflare/vitest-plugin": "1.3.6",
"vitest": "4.1.11",
"wrangler": "^4.148.0"
}
}
+5288
View File
File diff suppressed because it is too large. Load diff
+63
View File
@@ -0,0 +1,63 @@
/** @file Code mode: the model answers with code against the things it was lent. */
import type { Message } from "./model.ts";
/** what the neighbor knows about each thing: its TypeScript type */
export type About = Readonly<Record<string, Readonly<{ ts: string }>>>;
/** a template literal with `~` for each backtick, as in Markdown's `~~~`
* fences; interpolated values are left as they are */
const md = (strings: TemplateStringsArray, ...values: unknown[]) => {
const tick = (s: string) => s.replaceAll("~", "`");
return tick(strings[0]) + values.map((value, i) => String(value) + tick(strings[i + 1])).join("");
};
const persona = md`
You are my neighbor. You have 50 acres of lawn that need mowing.
You act by writing code. When I lend you things, your code gets them as ~env.<name>~.
`.trim();
const rules = md`
Reply with exactly one JavaScript arrow function, ~async (env) => { ... }~,
in a ~~~js block, and nothing else. Things are remote: each call on one
returns a promise, and calls on a thing are delivered in order, so there's
no need to wait for one before making the next. Await only when you need a
value back, or on the last call of a batch, to wait for the whole batch.
Your code can use only ~env~: no network, timers, imports, or console.
The function returns a short string saying what it did.
For example, lent ~env.lamp~ with ~on()~ and ~dim()~:
~~~js
async (env) => {
const { lamp } = env;
lamp.on();
lamp.dim();
await lamp.dim(); // the whole batch
return 'dimmed the lamp twice';
}
~~~
`.trim();
/** the prompt for `say`, with the types of the things lent alongside it */
export const codeModeMessages = ({ say, about }: { say: string; about: About }): Message[] => {
const types = Object.entries(about)
.map(
([name, { ts }]) => md`
~env.${name}~:
~~~ts
${ts}
~~~
`.trim(),
)
.join("\n\n");
return [
{ role: "system", content: `${persona}\n\n${rules}` },
{ role: "user", content: `${say}\n\nWhat I lent you:\n\n${types}` },
];
};
/** the code in a reply: the first fenced block, or the whole reply if none */
export const extractCode = (reply: string): string => {
const fenced = reply.match(/```(?:js|javascript)?\n([\s\S]*?)```/);
return (fenced?.[1] ?? reply).trim();
};
+34
View File
@@ -0,0 +1,34 @@
/** @file The model behind `complete(messages)`: Workers AI, through its binding. */
/** a Workers AI text-generation model with no paid-plan requirement */
export const MODEL = "@cf/openai/gpt-oss-120b";
/** room for a reasoning model to think and then answer; the default, 256,
* runs out mid-thought (`finish_reason: length`) */
export const MAX_TOKENS = 4096;
export type Message = Readonly<{ role: "system" | "user" | "assistant"; content: string }>;
/** the reply text from an OpenAI-style chat completion */
const replyText = (output: unknown): string => {
const { choices } = (output ?? {}) as {
choices?: { message?: { content?: unknown }; finish_reason?: unknown }[];
};
const [choice] = choices ?? [];
const text = choice?.message?.content;
if (typeof text !== "string") {
const where = choice
? `choice with finish_reason ${String(choice.finish_reason)} and message keys: ${Object.keys(choice.message ?? {}).join(", ")}`
: `model output with keys: ${Object.keys(output ?? {}).join(", ")}`;
throw Error(`no reply text in ${where}`);
}
return text;
};
/** `complete(messages)` on a Workers AI model; `ai` is the AI binding */
export const makeWorkersAiModel = (ai: Ai, model: string) => ({
async complete(messages: readonly Message[]): Promise<string> {
const output = await ai.run(model as Parameters<Ai["run"]>[0], { messages: [...messages], max_tokens: MAX_TOKENS });
return replyText(output);
},
});
+30
View File
@@ -0,0 +1,30 @@
/** @file The mower as a codemode connector: sandboxed code calls `mower.push({})`. */
import { CodemodeConnector, type ConnectorTools } from "@cloudflare/codemode";
import type { Mower } from "./mower.ts";
export class MowerConnector extends CodemodeConnector {
#mower: Mower;
constructor(ctx: DurableObjectState | ExecutionContext, env: unknown, mower: Mower) {
super(ctx, env);
this.#mower = mower;
}
name() {
return "mower";
}
protected instructions() {
return "A lawn mower on a yard of cells, 10 yards each. It moves like a Logo turtle.";
}
protected tools(): ConnectorTools {
const mower = this.#mower;
return {
start: { description: "put the mower down, facing north, and cut the cell under it", execute: () => mower.start() },
push: { description: "move forward one cell, cutting the cell it enters", execute: () => mower.push() },
turn: { description: "turn right 90 degrees, in place", execute: () => mower.turn() },
stop: { description: "park the mower", execute: () => mower.stop() },
};
}
}
+50
View File
@@ -0,0 +1,50 @@
/** @file The mower: Logo-turtle moves on a yard of cells. */
/** what a model is told about a mower: its type, with the moves explained */
export const aboutMower = {
ts: `interface Mower {
/** put the mower down, facing north, and cut the cell under it */
start(): Promise<void>;
/** move forward one cell (10 yards), cutting the cell it enters */
push(): Promise<void>;
/** turn right 90 degrees, in place */
turn(): Promise<void>;
/** park the mower */
stop(): Promise<void>;
}`,
};
const headings = [
[0, -1], // north
[1, 0], // east
[0, 1], // south
[-1, 0], // west
] as const;
/** a mower on a `width` × `height` yard, starting at `at`; pushing past an
* edge wraps to the other side (6d) */
export const makeMower = ({ width, height, at }: { width: number; height: number; at: { x: number; y: number } }) => {
let { x, y } = at;
let heading = 0;
const cut = new Set<string>();
const cutHere = () => cut.add(`${x},${y}`);
return {
start() {
heading = 0;
cutHere();
},
push() {
const [dx, dy] = headings[heading];
x = (x + dx + width) % width;
y = (y + dy + height) % height;
cutHere();
},
turn() {
heading = (heading + 1) % headings.length;
},
stop() {},
cutCells: () => [...cut],
};
};
export type Mower = ReturnType<typeof makeMower>;
+44
View File
@@ -0,0 +1,44 @@
/** @file The neighbor's harness as a Project Think agent, lent a mower. */
import { Think } from "@cloudflare/think";
import { createExecuteTool } from "@cloudflare/think/tools/execute";
import { createWorkersAI } from "workers-ai-provider";
import { MAX_TOKENS, MODEL } from "./model.ts";
import { MowerConnector } from "./mower-connector.ts";
import { makeMower } from "./mower.ts";
type Env = { AI: Ai; LOADER: WorkerLoader };
export class Neighbor extends Think<Env> {
#mower = makeMower({ width: 40, height: 40, at: { x: 20, y: 20 } });
getModel() {
return createWorkersAI({ binding: this.env.AI })(MODEL);
}
getSystemPrompt() {
return `You are my neighbor. You have 50 acres of lawn that need mowing.
You act by running code with the execute tool. Things I lend you are
namespaces there: the mower is \`mower.*\`. Don't ask me questions: when I
lend you something, put it to use, then tell me what you did.`;
}
beforeTurn() {
// only code: Think's workspace tools sent the model looking for the
// mower in the filesystem
return { maxOutputTokens: MAX_TOKENS, activeTools: ["execute"] };
}
getTools() {
return {
execute: createExecuteTool(this, {
connectors: [new MowerConnector(this.ctx, this.env, this.#mower)],
}),
};
}
/** the cells the mower has cut, as `"x,y"` */
cutCells() {
return this.#mower.cutCells();
}
}
+11
View File
@@ -0,0 +1,11 @@
/** @file The worker entry: the Neighbor agent, and the codemode runtime its execute tool needs. */
import { routeAgentRequest } from "agents";
export { CodemodeRuntime } from "@cloudflare/codemode";
export { Neighbor } from "./neighbor.ts";
export default {
async fetch(request: Request, env: Record<string, unknown>) {
return (await routeAgentRequest(request, env)) ?? new Response("not found", { status: 404 });
},
};
+143
View File
@@ -0,0 +1,143 @@
/** @file The lend-the-mower journey, traced and compared with its sequence diagram. */
import './lockdown.ts';
import assert from 'node:assert/strict';
import { readFile } from 'node:fs/promises';
import test from 'node:test';
import { makeFencePage } from '../src/fence.ts';
import { makePlayground } from '../src/governor.ts';
import { aboutMower, makeMowerFactory } from '../src/mower.ts';
import { makeNeighbor } from '../src/neighbor.ts';
import { makeCompartmentLoader } from '../src/tenant-loader.ts';
import { makeYard } from '../src/yard.ts';
import {
type CausalSequenceViz,
makeCausalSequenceTracer,
md,
mmd,
} from '../tools/sequence-diagram-actor-sim.ts';
const designDoc = new URL('../docs/lend-the-mower.md', import.meta.url);
/** a tenant that pushes until something stops it, or gives up after 20 */
const tipsyTenant = `(env) => {
env.mower.start();
try {
for (let i = 0; i < 20; i += 1) env.mower.push();
} catch (err) {
return err.message;
}
return 'mowed 20 cells';
}`;
/** diagram-local names, bound by the reply arrows that introduce them */
const local = new WeakMap<object, string>();
const bind = (name: string) => (result: unknown) => {
local.set(result as object, name);
return name;
};
/** `{ a, b: { c: governed } }`: keys, nested objects by their keys, and
* objects with a diagram-local name by that name */
const shape = (record: object): string =>
`{ ${Object.entries(record)
.map(([key, value]) => {
if (value === null || typeof value !== 'object') return key;
const name = local.get(value);
if (name) return `${key}: ${name}`;
return Array.isArray(value) ? key : `${key}: ${shape(value)}`;
})
.join(', ')} }`;
const quoted = (result: unknown) => `'${String(result)}'`;
const viz = harden({
lend: {
label: args => {
const { webkey, limit } = args[0] as { webkey: string; limit: number };
return `lend({ webkey: '${webkey}', limit: ${limit}, say })`;
},
},
mower: { result: bind('mower') },
govern: {
args: args => `${local.get(args[0] as object)}, ${String(args[1])}`,
result: bind('governed'),
},
chat: {
label: args => `chat(${shape(args[0] as object)})`,
result: quoted,
},
complete: {
label: args =>
`complete([${(args[0] as object[]).map(shape).join(', ')}])`,
result: () => 'code',
},
load: {
args: args => `code, ${shape(args[1] as object)}`,
result: () => 'tenant',
},
run: { result: quoted },
}) satisfies CausalSequenceViz;
test('a governed mower stops a tipsy tenant at the fence', async () => {
const lines = (await readFile(designDoc, 'utf8')).split('\n');
const section = md.skipToH(2, 'Lend a governed mower to a tipsy tenant')(lines);
const diagram = md.eachFence('mermaid', section).next().value;
if (!diagram) throw Error('lend-the-mower Mermaid block not found');
const documented = mmd.extractArrows(diagram);
const ES = makeCausalSequenceTracer(viz);
const yard = makeYard({ width: 8, height: 8 });
const factory = makeMowerFactory(ES, {
yard,
size: { width: 8, height: 8 },
mintKey: () => 'mower-key',
});
const key = factory.make({ x: 4, y: 4 });
const mower = factory.mower(key);
const prompts: unknown[] = [];
const model = harden({
complete(messages: readonly unknown[]) {
prompts.push(messages);
return tipsyTenant;
},
});
const playground = makePlayground(ES);
const loader = makeCompartmentLoader(ES);
const neighbor = makeNeighbor(ES, { model, loader });
const fence = makeFencePage(ES, { factory, playground, neighbor });
ES.declareParticipants({
F: fence,
MF: factory,
P: playground,
M: mower,
Y: yard,
N: neighbor,
L: model,
TL: loader,
});
ES.declareMade({ G: [playground, 'govern'], T: [loader, 'load'] });
ES.start(fence);
const outcome = ES(fence).lend({
webkey: key,
limit: 2,
say: 'you can use this mower for the weekend',
});
assert.deepEqual(ES.snapshot(), documented);
assert.equal(outcome, 'fence: 2 cells from start');
assert.deepEqual(yard.cutCells(), ['4,4', '4,3', '4,2']);
assert.deepEqual(prompts, [
[
{
role: 'user',
content: 'you can use this mower for the weekend',
about: { mower: aboutMower },
},
],
]);
});
+4
View File
@@ -0,0 +1,4 @@
/** @file Harden the realm before any test module runs: tenants share it. */
import 'ses';
lockdown();
@@ -0,0 +1,236 @@
/**
* @file Helpers for sequence-diagram actor simulations.
*
* From agoric-sdk `packages/portfolio-contract/tools/` (Apache-2.0), with
* two additions: `declareMade` names participants that actors create
* mid-sequence, and a thrown error is drawn as a reply (`throws '...'`).
*
* @see https://github.com/Agoric/agoric-sdk/pull/12895
*/
import { AsyncLocalStorage } from 'node:async_hooks';
export type SequenceArrow = Readonly<{
from: string;
kind: '->>' | '-->>';
to: string;
label: string;
}>;
export const formatBigInt = (value: bigint) =>
value.toString().replace(/\B(?=(\d{3})+(?!\d))/g, '_');
export type SequenceArgsRenderer = (args: readonly unknown[]) => string;
export type SequenceResultRenderer = (
result: unknown,
args: readonly unknown[],
) => string;
export type SequenceMethodViz = Readonly<
{
args?: SequenceArgsRenderer;
label?: SequenceArgsRenderer;
} & (
| { result?: SequenceResultRenderer; resultOnly?: never }
| { result?: never; resultOnly: SequenceResultRenderer }
)
>;
export type CausalSequenceViz = Readonly<Record<string, SequenceMethodViz>>;
export type SequenceParticipant = object | readonly object[];
/**
* Make an E-like causal tracer for sequence-diagram actor simulations.
* Inspired by Causeway, the message-oriented distributed debugger.
*
* Method visualization is currently selected by method name alone. If actors
* need different visualization for methods with the same name, this can be
* extended to account for the target participant.
*
* @see https://shiftleft.com/mirrors/www.hpl.hp.com/techreports/2009/HPL-2009-78.pdf
*/
export const makeCausalSequenceTracer = (
methodViz: CausalSequenceViz = harden({}),
) => {
const arrows: SequenceArrow[] = [];
const names = new WeakMap<object, string>();
const proxies = new WeakMap<object, object>();
const made = new Map<object, Map<string, string>>();
const activeActor = new AsyncLocalStorage<object>();
let starter: object | undefined;
const participantName = (participant: object) => {
const name = names.get(participant);
if (!name) throw Error('sequence participant is not declared');
return name;
};
const formatValue = (value: unknown): string => {
if (typeof value === 'bigint') return `${formatBigInt(value)}n`;
if (typeof value === 'string') return `'${value}'`;
return JSON.stringify(value, (_key, item) =>
typeof item === 'bigint' ? `${item}n` : item,
);
};
const send = <Target extends object>(target: Target): Target => {
const extant = proxies.get(target);
if (extant) return extant as Target;
participantName(target);
const proxy = new Proxy(Object.create(null) as object, {
get: (_proxyTarget, property) => {
const method = Reflect.get(target, property);
if (typeof method !== 'function' || typeof property !== 'string') {
return method;
}
return (...args: unknown[]) => {
const activeSender = activeActor.getStore();
const sender = activeSender ?? starter;
if (!sender) throw Error('ES.start(participant) is required');
const isStart = activeSender === undefined;
if (isStart && sender !== target) {
throw Error('the started participant must receive the first send');
}
if (isStart) starter = undefined;
const viz = methodViz[property];
const abbreviated = viz?.args?.(args);
const shownArgs = abbreviated ?? args.map(formatValue).join(', ');
const label = viz?.label?.(args) ?? `${property}(${shownArgs})`;
const renderResultOnly = viz?.resultOnly;
if (!renderResultOnly) {
arrows.push(
harden({
from: participantName(sender),
kind: isStart ? '->>' : '-->>',
to: participantName(target),
label,
}),
);
}
let result: unknown;
try {
result = activeActor.run(target, () =>
Reflect.apply(method, target, args),
);
} catch (reason) {
const message =
reason instanceof Error ? reason.message : String(reason);
arrows.push(
harden({
from: participantName(target),
kind: '-->>',
to: participantName(sender),
label: `throws ${formatValue(message)}`,
}),
);
throw reason;
}
const madeName = made.get(target)?.get(property);
if (madeName && result !== null && typeof result === 'object') {
if (names.has(result)) throw Error('participant already declared');
names.set(result, madeName);
}
const renderResult = renderResultOnly ?? viz?.result;
if (renderResult) {
arrows.push(
harden({
from: participantName(target),
kind: '-->>',
to: participantName(sender),
label: renderResult(result, args),
}),
);
}
return result;
};
},
});
proxies.set(target, proxy);
return proxy as Target;
};
const ES = Object.assign(send, {
declareParticipants(
participants: Readonly<Record<string, SequenceParticipant>>,
) {
for (const [name, participantOrAliases] of Object.entries(participants)) {
const aliases = Array.isArray(participantOrAliases)
? participantOrAliases
: [participantOrAliases];
if (aliases.length === 0) {
throw Error(`sequence participant ${name} has no objects`);
}
for (const participant of aliases) {
if (names.has(participant)) {
throw Error('participant already declared');
}
names.set(participant, name);
}
}
},
/** name what `maker[method](...)` returns, when called through the tracer */
declareMade(
participants: Readonly<Record<string, readonly [object, string]>>,
) {
for (const [name, [maker, method]] of Object.entries(participants)) {
participantName(maker);
const byMethod = made.get(maker) ?? new Map<string, string>();
made.set(maker, byMethod);
byMethod.set(method, name);
}
},
snapshot: () => harden([...arrows]),
start(participant: object) {
participantName(participant);
if (activeActor.getStore() || starter) {
throw Error('a sequence is already active');
}
starter = participant;
},
});
return harden(ES);
};
export type CausalSequenceTracer = ReturnType<typeof makeCausalSequenceTracer>;
export const md = {
skipToH: (level: number, title: string) => (lines: readonly string[]) => {
const heading = `${'#'.repeat(level)} ${title}`;
const at = lines.findIndex(line => line === heading);
if (at < 0) throw Error(`${heading} not found`);
return lines.slice(at + 1);
},
*eachFence(language: string, lines: readonly string[]) {
const open = `\`\`\`${language}`;
for (let at = 0; at < lines.length; at += 1) {
if (lines[at] !== open) continue;
const end = lines.indexOf('```', at + 1);
if (end < 0) throw Error(`unterminated ${language} fence`);
yield lines.slice(at + 1, end);
at = end;
}
},
};
export const mmd = {
extractArrows(
lines: readonly string[],
nodePattern: RegExp = /\w+/,
): readonly SequenceArrow[] {
const arrowPattern = new RegExp(
`^\\s*(?<from>${nodePattern.source})\\s*(?<kind>--?>>)\\s*(?<to>${nodePattern.source})\\s*:\\s*(?<label>.*?)\\s*$`,
);
return harden(
lines.flatMap(line => {
const groups = line.match(arrowPattern)?.groups;
if (!groups) return [];
const { from, kind, to, label } = groups;
return [harden({ from, kind, to, label }) as SequenceArrow];
}),
);
},
};
+13
View File
@@ -0,0 +1,13 @@
import { defineProject } from "vitest/config";
import { cloudflareTest } from "@cloudflare/vitest-plugin";
/** opt-in (`pnpm test:model`): calls Workers AI, which needs `wrangler login` and costs money */
export default defineProject({
plugins: [cloudflareTest({ wrangler: { configPath: "./wrangler.model-tests.jsonc" } })],
test: {
include: ["itest/**/*.test.ts"],
testTimeout: 60_000,
// what the model wrote, saved per run to itest-output/ (gitignored)
reporters: ["default", "./itest/save-annotations.ts"],
},
});
+18
View File
@@ -0,0 +1,18 @@
{
"$schema": "node_modules/wrangler/config-schema.json",
// model integration tests: AI always runs remotely, and costs money
"name": "good-fences-model-tests",
"main": "./src/worker.ts",
"compatibility_date": "2026-10-01",
"compatibility_flags": ["nodejs_compat"],
"ai": { "binding": "AI", "remote": true },
"worker_loaders": [{ "binding": "LOADER" }],
"durable_objects": {
"bindings": [{ "name": "Neighbor", "class_name": "Neighbor" }]
},
"exports": {
"Neighbor": { "type": "durable-object", "storage": "sqlite" },
// codemode's runtime, a facet of the agent's DO
"CodemodeRuntime": { "type": "durable-object", "storage": "sqlite" }
}
}
-107
View File
@@ -1,107 +0,0 @@
This is an EmDash site -- a CMS built on Astro with a full admin UI.
## Commands
```bash
pnpm dev # Start the Astro dev server
npx emdash types # Regenerate TypeScript types from a running site
```
The admin UI is at `http://localhost:4321/_emdash/admin`.
## Key Files
| File | Purpose |
| ------------------------ | ---------------------------------------------------------------------------------- |
| `astro.config.mjs` | Astro config with `emdash()` integration, database, and storage |
| `src/live.config.ts` | EmDash loader registration (boilerplate -- don't modify) |
| `seed/seed.json` | Schema definition + demo content (collections, fields, taxonomies, menus, widgets) |
| `emdash-env.d.ts` | Generated types for collections (auto-regenerated on dev server start) |
| `src/layouts/Base.astro` | Base layout with EmDash wiring (menus, search, page contributions) |
| `src/pages/` | Astro pages -- all server-rendered |
## Skills
Agent skills are in `.agents/skills/`. Load them when working on specific tasks:
- **building-emdash-site** -- Querying content, rendering Portable Text, schema design, seed files, site features (menus, widgets, search, SEO, comments, bylines). Start here.
- **creating-plugins** -- Building EmDash plugins with hooks, storage, admin UI, API routes, and Portable Text block types.
- **emdash-cli** -- CLI commands for content management, seeding, type generation, and visual editing flow.
## Documentation
The EmDash docs are available as an MCP server at `https://docs.emdashcms.com/mcp`. When you need to verify an API, hook, config option, field type, or pattern, call `search_docs` against the live documentation rather than relying on training-data recall. The docs reflect current behaviour; assumptions may not.
This template ships with `.mcp.json`, `.cursor/mcp.json`, and `.vscode/mcp.json` so Claude Code, Cursor, and VS Code auto-discover the docs server. Other tools (OpenCode, Windsurf, etc.) need a manual one-time setup -- see [docs.emdashcms.com/docs-mcp](https://docs.emdashcms.com/docs-mcp).
## Rules
- All content pages must be server-rendered (`output: "server"`). No `getStaticPaths()` for CMS content.
- Image fields are objects (`{ src, alt }`), not strings. Use `<Image image={...} />` from `"emdash/ui"`.
- `entry.id` is the slug (for URLs). `entry.data.id` is the database ULID (for API calls like `getEntryTerms`).
- When Astro's cache is enabled, pass content-query hints to `Astro.cache.set(cacheHint)`. Use the `WithCacheHint` variants for site settings, menus, taxonomies, and widget areas rendered by cached routes.
- Taxonomy names in queries must match the seed's `"name"` field exactly (e.g., `"category"` not `"categories"`).
## This Template
A blog with posts, pages, categories, tags, full-text search, and RSS. Designed for personal writing, technical writing, indie newsletters, and anything where the writing is the product. Editorial-tech aesthetic: confident sans-serif, restrained accent, real article structure with bylines and reading time.
## Pages
| Page | Path | What it shows |
| ----------- | ------------------ | ------------------------------------------------------------------------------------------------------ |
| Home | `/` | Featured post hero (large image + excerpt), latest posts grid |
| All posts | `/posts` | Article count, full post list with excerpts and tag chips |
| Post detail | `/posts/[slug]` | Featured image, title, body, left meta column (authors + date), right TOC + search + categories gutter |
| Search | `/search` | Full-text search UI |
| Page | `/pages/[slug]` | Static page content (Portable Text) |
| Category | `/category/[slug]` | Posts filtered by category |
| Tag | `/tag/[slug]` | Posts filtered by tag |
| RSS | `/rss.xml` | Generated feed |
## Schema
- `posts` collection: `title`, `featured_image`, `content` (Portable Text), `excerpt` (text).
- `pages` collection: `title`, `content` (Portable Text). Used for `/about` etc.
- Taxonomies: `category`, `tag`.
- Single `primary` menu (Home, About, Posts by default).
Site settings have `title` and `tagline` -- both render in the header / footer.
## Visual character
Single typeface: **Inter** on `--font-body`, used for everything including headings (`--font-heading` defaults to the body face; tighter letter-spacing on h1/h2). **JetBrains Mono** on `--font-mono` for inline code and code blocks. Body and headings share the same family; weight and size carry the hierarchy (`--font-weight-heading` 600, `--font-weight-display` 700 for h1/page titles).
The brand colour is `#0066cc` (`--color-brand`) -- used for links, the post-card title hover, and the search input focus ring. There's also a secondary text colour (`--color-text-secondary`) and a `--color-muted` for meta info. Don't add a second accent.
The article layout is the standout feature: a three-column reading view with a left meta column (author bylines, date), centred 680px body column, and a right gutter for search, table of contents, and categories. Don't flatten that into one column on desktop -- the layout signals "this is something to read".
## Customisation
Design tokens live in `src/styles/tokens.css` with their default values. To restyle the site, override tokens in `src/styles/theme.css` -- declarations there are unlayered, so they always beat the `@layer base` defaults. Don't edit `tokens.css` or `Base.astro` for visual changes.
Colours are defined with `light-dark(<light>, <dark>)`, so each token carries both modes. Overriding with a plain colour changes light and dark at once; use `light-dark()` in the override to keep them distinct. There is no separate dark palette to maintain.
Webfonts are configured in `astro.config.mjs` under `fonts:`. To swap the body face, change the `name:` for the entry bound to `cssVariable: "--font-body"`. Good alternatives: Geist, IBM Plex Sans, Söhne (if you have a licence), Public Sans. If you want a serif-bodied blog, swap to a humanist serif like Source Serif, Crimson Pro, or Lora -- but then also raise `--font-size-base` to `1.0625rem` for readability. To give headings their own face (or use a system font) without touching the font pipeline, override `--font-heading` or `--font-body` in `theme.css`.
CSS variables worth knowing (see `tokens.css` for the full list):
- `--color-brand`, `--color-brand-hover`, `--color-on-brand`, `--color-brand-ring`
- `--color-bg`, `--color-bg-subtle`, `--color-surface`, `--color-text`, `--color-text-secondary`, `--color-muted`, `--color-border`, `--color-border-subtle`
- `--font-body`, `--font-heading`, `--font-mono`
- `--font-weight-heading` (600) / `--font-weight-display` (700) -- heading weights; lower them if you switch to a serif
- `--tracking-tight` / `--tracking-snug` / `--tracking-wide` / `--tracking-wider` -- letter-spacing tokens used across headings and meta labels
- `--content-width` (680px) -- article body column
- `--wide-width` (1200px) -- max container
- `--gutter-width` (200px) -- right sidebar (TOC) on article pages
- `--meta-col-width` (180px) -- left meta column on article pages
- `--avatar-size-{xs,sm,md,lg}` -- byline avatar sizes at different scales
## What not to do
- Don't add a second accent colour or coloured section backgrounds. The page should be black, white, and one blue.
- Don't replace Inter with a display sans (Bebas, Anton, etc.). Headings rely on weight contrast, not novelty faces.
- Don't collapse the article gutter on desktop -- it's part of the reading experience.
- Don't use stock blog copy ("Welcome to my blog", "Stay tuned for more"). Write a real tagline that says what this blog is about.
- Don't seed the home page with three identical placeholder posts. If you only have one real post, show one real post.
- Comments are enabled on posts and rendered on the post detail page. Configure moderation before publishing the site, or remove `commentsEnabled` and the comments UI together.
+1
View File
@@ -0,0 +1 @@
CONTRIBUTING.md
+51
View File
@@ -0,0 +1,51 @@
# Contributing to potluck
potluck started as EmDash's `cloudflare:blog` template (`pnpm create emdash@latest`); it's part of [the capsite goal tree](../capsite-checklist.md), where it is the site that publishes the weekly roundup.
## How it's put together
- **Issues** are the `issues` collection ([seed/issues.json](seed/issues.json)): `title`, `week` (an ISO week, `2026-W40`), `dek` (one-line summary), `body` (Portable Text). The slug is the week, lowercased: `/issues/2026-w40`. [issues-sketch.md](issues-sketch.md) is the dry run behind the three sample issues: real public bookmarks, editorial lines marked *draft*.
- **Pages:** home and `/issues` list issues ([src/pages/issues/](src/pages/issues/)); `/tag/<tag>` lists issues by diigo tag; search and `/rss.xml` cover issues. The `posts` collection and its pages are the template's, still in place.
- **Microformats:** an issue list is an `h-feed` of `h-entry`s; an issue is an `h-entry` (`p-name`, `dt-published`, `p-summary`, `e-content`). In issue bodies, [CiteLink.astro](src/components/CiteLink.astro) marks off-site links as `h-cite` and `/tag/<tag>` links as rel-tag `p-category`.
- **Look:** plain HTML, after kmjn.org's [Plain HTML with a few bureaucratic gestures](https://www.kmjn.org/notes/modern_plain_html.html): system fonts (no webfonts), one ~800px column, links in the text color with a dotted underline. Restyle in [src/styles/theme.css](src/styles/theme.css), which overrides the defaults in `tokens.css` (unlayered, so it always wins); don't edit `tokens.css` or `Base.astro` for visual changes. Colors use `light-dark(<light>, <dark>)`.
## Plugins
[diigo-week](../diigo-week/README.md) (Plugins → Draft from diigo in the admin) drafts an issue from a week of diigo bookmarks. It reads diigo-cache's public page as microformats over the `DIIGO_CACHE` service binding in [wrangler.jsonc](wrangler.jsonc); in dev, run diigo-cache's `pnpm dev` alongside. It's linked as `link:../diigo-week`, and registered in [astro.config.mjs](astro.config.mjs). After changing it, restart the dev server.
## Seeds
[seed/seed.json](seed/seed.json) (site settings, menus, widgets, the template's collections) is applied on the first request to an empty database. [seed/issues.json](seed/issues.json) is separate: `pnpm seed:issues` applies it to the local D1 database, which exists after the dev server's first request. Run it with the dev server stopped.
## Checks
```sh
pnpm typecheck # astro check
pnpm build
```
## EmDash rules
- All content pages are server-rendered (`output: "server"`); no `getStaticPaths()` for CMS content.
- Image fields are objects (`{ src, alt }`), not strings; render with `<Image image={...} />` from `"emdash/ui"`.
- `entry.id` is the slug (for URLs); `entry.data.id` is the database ULID (for API calls like `getEntryTerms`).
- When Astro's cache is enabled, pass query hints to `Astro.cache.set(cacheHint)`.
- Taxonomy names in queries match the seed's `"name"` exactly (`"tag"`, not `"tags"`).
- `npx emdash types` regenerates `emdash-env.d.ts` from a running site (the dev server also regenerates it on start).
The EmDash docs are an MCP server at `https://docs.emdashcms.com/mcp` (wired up in `.mcp.json`, `.cursor/mcp.json`, `.vscode/mcp.json`). Check them rather than recalling APIs from memory.
## Deploying
```sh
pnpm wrangler login
pnpm deploy
```
The first deploy provisions the D1 database and R2 bucket named in [wrangler.jsonc](wrangler.jsonc). See [Deploy to Cloudflare](https://docs.emdashcms.com/deployment/cloudflare/).
Other workerd hosts are untested: the site uses D1 and R2 through `@emdash-cms/cloudflare` and the Cloudflare adapter (locally, miniflare stands in for both), so a host has to supply those two bindings.
## For agents
`AGENTS.md` is a symlink to this file. capsite's [working rules](../AGENTS.md) apply here too (the dev server is the user's to start and stop). EmDash skills are in `.agents/skills/`: `building-emdash-site` (start here), `creating-plugins`, `emdash-cli`.
+14 -60
View File
@@ -1,68 +1,22 @@
# EmDash Blog Template (Cloudflare) # potluck
A clean, minimal blog built with [EmDash](https://github.com/emdash-cms/emdash) and deployed on Cloudflare Workers with D1 and R2. A weekly zine of what you bookmarked elsewhere, published on your own site. Each issue is one week of links with a few words around them.
[![Deploy to Cloudflare](https://deploy.workers.cloudflare.com/button)](https://deploy.workers.cloudflare.com/?url=https://github.com/emdash-cms/templates/tree/main/blog-cloudflare) ## Getting started
![Blog template homepage](https://raw.githubusercontent.com/emdash-cms/emdash/main/assets/templates/blog/latest/homepage-light-desktop.jpg) ```sh
## What's Included
- Featured post hero on the homepage
- Post archive with reading time estimates
- Category and tag archives
- Full-text search
- RSS feed
- SEO metadata and JSON-LD
- Dark/light mode
## Pages
| Page | Route |
|---|---|
| Homepage | `/` |
| All posts | `/posts` |
| Single post | `/posts/:slug` |
| Category archive | `/category/:slug` |
| Tag archive | `/tag/:slug` |
| Search | `/search` |
| Static pages | `/pages/:slug` |
| 404 | fallback |
## Screenshots
| | Desktop | Mobile |
|---|---|---|
| Light | ![homepage light desktop](https://raw.githubusercontent.com/emdash-cms/emdash/main/assets/templates/blog/latest/homepage-light-desktop.jpg) | ![homepage light mobile](https://raw.githubusercontent.com/emdash-cms/emdash/main/assets/templates/blog/latest/homepage-light-mobile.jpg) |
| Dark | ![homepage dark desktop](https://raw.githubusercontent.com/emdash-cms/emdash/main/assets/templates/blog/latest/homepage-dark-desktop.jpg) | ![homepage dark mobile](https://raw.githubusercontent.com/emdash-cms/emdash/main/assets/templates/blog/latest/homepage-dark-mobile.jpg) |
## Infrastructure
- **Runtime:** Cloudflare Workers
- **Database:** D1
- **Storage:** R2
- **Framework:** Astro with `@astrojs/cloudflare`
## Local Development
```bash
pnpm install pnpm install
pnpm dev pnpm dev # open http://localhost:4321/_emdash/admin, finish the setup wizard, then Ctrl-C
pnpm seed:issues # three sample issues, W38–W40 of 2026
pnpm dev # → http://localhost:4321/issues
``` ```
Open http://localhost:4321/_emdash/admin and complete the setup wizard. EmDash runs database migrations and applies the blog seed during setup. The site is available at http://localhost:4321. ## Bookmarks in, a zine out
## Deploying - **Your web, your site:** the bookmarks come from your own diigo account, so the web you read becomes something you publish ([PESOS](https://indieweb.org/PESOS)). The [diigo-week](../diigo-week/README.md) plugin drafts an issue from a week of them; you cut and comment.
- **A week per issue:** `/issues/2026-w40`, by ISO week, UTC.
- **Microformats:** feed readers and IndieWeb tools see each issue as an [h-entry](https://microformats.org/wiki/h-entry) and each bookmark as an [h-cite](https://microformats.org/wiki/h-cite).
- **Plain HTML** that's easy to read: system fonts, one column.
- **A real editor:** it's an [EmDash](https://docs.emdashcms.com/) site, so you write issues in its admin UI. It runs in [workerd](https://github.com/cloudflare/workerd), on your machine or a host.
```bash How it's put together, checks, and deploying: [CONTRIBUTING](CONTRIBUTING.md).
pnpm wrangler login
pnpm deploy
```
The first deployment provisions the named D1 database and R2 bucket from `wrangler.jsonc`. See [Deploy to Cloudflare](https://docs.emdashcms.com/deployment/cloudflare/) for production setup, or use the deploy button above.
## See Also
- [Node.js variant](../blog) -- same template using SQLite and local file storage
- [All templates](../)
- [EmDash documentation](https://docs.emdashcms.com/)
+2
View File
@@ -16,6 +16,8 @@ export default defineConfig({
emdash({ emdash({
database: d1({ binding: "DB", session: "auto" }), database: d1({ binding: "DB", session: "auto" }),
storage: r2({ binding: "MEDIA" }), storage: r2({ binding: "MEDIA" }),
// drafts an issue from a week of diigo bookmarks (../diigo-week)
plugins: [{ id: "diigo-week", version: "0.1.0", entrypoint: "diigo-week" }],
}), }),
], ],
devToolbar: { enabled: false }, devToolbar: { enabled: false },
+2
View File
@@ -9,6 +9,7 @@
}, },
"scripts": { "scripts": {
"dev": "astro dev", "dev": "astro dev",
"seed:issues": "emdash seed seed/issues.json -d \"$(find .wrangler/state/v3/d1 -name '*.sqlite' ! -name metadata.sqlite)\"",
"build": "astro build", "build": "astro build",
"preview": "astro preview", "preview": "astro preview",
"deploy": "astro build && wrangler deploy", "deploy": "astro build && wrangler deploy",
@@ -19,6 +20,7 @@
"@astrojs/react": "^6.0.5", "@astrojs/react": "^6.0.5",
"@emdash-cms/cloudflare": "^1.0.1", "@emdash-cms/cloudflare": "^1.0.1",
"astro": "^7.3.2", "astro": "^7.3.2",
"diigo-week": "link:../diigo-week",
"emdash": "^1.0.1", "emdash": "^1.0.1",
"react": "19.2.4", "react": "19.2.4",
"react-dom": "19.2.4" "react-dom": "19.2.4"
+3
View File
@@ -20,6 +20,9 @@ importers:
astro: astro:
specifier: ^7.3.2 specifier: ^7.3.2
version: 7.3.6(@types/node@26.6.4)(yaml@2.9.1) version: 7.3.6(@types/node@26.6.4)(yaml@2.9.1)
diigo-week:
specifier: link:../diigo-week
version: link:../diigo-week
emdash: emdash:
specifier: ^1.0.1 specifier: ^1.0.1
version: 1.2.0(@astrojs/react@6.0.6(@types/node@26.6.4)(@types/react-dom@19.3.0(@types/react@19.3.0))(@types/react@19.3.0)(esbuild@0.28.2)(react-dom@19.2.4(react@19.2.4))(react@19.2.4)(yaml@2.9.1))(@atcute/cbor@2.3.8(@atcute/cid@2.5.0))(@atcute/cid@2.5.0)(@atcute/identity@2.0.2(@atcute/lexicons@2.1.1)(typescript@6.0.3))(@date-fns/tz@1.5.0)(@floating-ui/dom@1.8.0)(@tiptap/extensions@3.31.4(@tiptap/core@3.31.4(@tiptap/pm@3.31.4))(@tiptap/pm@3.31.4))(@types/react-dom@19.3.0(@types/react@19.3.0))(@types/react@19.3.0)(astro@7.3.6(@types/node@26.6.4)(yaml@2.9.1))(date-fns@4.4.0)(echarts@6.1.0)(prosemirror-model@1.25.12)(prosemirror-state@1.4.4)(prosemirror-view@1.42.6)(react-dom@19.2.4(react@19.2.4))(react@19.2.4)(typescript@6.0.3) version: 1.2.0(@astrojs/react@6.0.6(@types/node@26.6.4)(@types/react-dom@19.3.0(@types/react@19.3.0))(@types/react@19.3.0)(esbuild@0.28.2)(react-dom@19.2.4(react@19.2.4))(react@19.2.4)(yaml@2.9.1))(@atcute/cbor@2.3.8(@atcute/cid@2.5.0))(@atcute/cid@2.5.0)(@atcute/identity@2.0.2(@atcute/lexicons@2.1.1)(typescript@6.0.3))(@date-fns/tz@1.5.0)(@floating-ui/dom@1.8.0)(@tiptap/extensions@3.31.4(@tiptap/core@3.31.4(@tiptap/pm@3.31.4))(@tiptap/pm@3.31.4))(@types/react-dom@19.3.0(@types/react@19.3.0))(@types/react@19.3.0)(astro@7.3.6(@types/node@26.6.4)(yaml@2.9.1))(date-fns@4.4.0)(echarts@6.1.0)(prosemirror-model@1.25.12)(prosemirror-state@1.4.4)(prosemirror-view@1.42.6)(react-dom@19.2.4(react@19.2.4))(react@19.2.4)(typescript@6.0.3)
+3
View File
@@ -16,6 +16,9 @@
"bucket_name": "my-emdash-media", "bucket_name": "my-emdash-media",
}, },
], ],
// diigo-cache's public pages, for the diigo-week plugin (reads its
// microformats; in dev, diigo-cache's own `pnpm dev` serves it)
"services": [{ "binding": "DIIGO_CACHE", "service": "diigo-cache" }],
// Sandboxed plugins require Worker Loader, available on Workers paid plans. Uncomment to enable: // Sandboxed plugins require Worker Loader, available on Workers paid plans. Uncomment to enable:
// "worker_loaders": [{ "binding": "LOADER" }], // "worker_loaders": [{ "binding": "LOADER" }],
// General maintenance cron trigger // General maintenance cron trigger
+1 -1
View File
@@ -3,7 +3,7 @@
DO-backed fluent web client factory: site-agnostic. Deploy once; instances DO-backed fluent web client factory: site-agnostic. Deploy once; instances
are attenuated handles whose **ids are bearer capabilities** — anyone are attenuated handles whose **ids are bearer capabilities** — anyone
holding the id string can drive the instance; nobody without it can. holding the id string can drive the instance; nobody without it can.
TODO: this proxy-page flow (see diigo-cache README) vs running webapp as TODO: this proxy-page flow (see [diigo-cache CONTRIBUTING](../diigo-cache/CONTRIBUTING.md)) vs running webapp as
the primary worker (`wrangler dev -c ../webapp/wrangler.jsonc -c ../diigo-cache/wrangler.jsonc`) the primary worker (`wrangler dev -c ../webapp/wrangler.jsonc -c ../diigo-cache/wrangler.jsonc`)
so its own port serves the mint form directly; reconsider when the UI so its own port serves the mint form directly; reconsider when the UI
grows past a form or two. grows past a form or two.