# capsite goal tree Nested checkboxes form the goal tree itself: each `- [ ]` is a goal; its indented children are the subgoals/steps that must pass before it (an item is only checked when **all** its children are checked — or, for leaf items, when it has passed verification itself). Working notes live under each goal; [the capsite notes](capsite-notes.md) carry the background narrative. - [ ] **madmode publishes a weekly roundup of diigo bookmarks** ([#23](https://github.com/dckc/madmode-blog/issues/23)); the diigo session stays inside webapp, so madmode holds no credential at all (background: [the capsite notes](capsite-notes.md); after that, [bib sync #195](https://github.com/dckc/madmode-blog/issues/195)) - [x] Credential discipline: no worker secrets; the cookie lives only in webapp's DO storage, ids are held by the user ([AGENTS.md](AGENTS.md), secrets section). - [x] **1. Cookie-path dump worker**: port of mm-diigo's diigo-dump.mjs (verified 2026-10-06: 200, `total: 1674`); evolved into 1c. - [x] **1b. webapp DO factory** ([webapp/README.md](webapp/README.md)); mint verified end-to-end in dev 2026-10-07 (via 1c). Prod mint is goal 5. - [x] **1c. diigo-dump evolves into a bookmark cache** ([diigo-cache/README.md](diigo-cache/README.md); design in the JSDoc of `src/cache.ts`, `src/index.ts`, `src/pm-save.ts`): - [x] 1c-i. Two independent page caches, private and public (`PagedCache`, `DiigoCache`, `PublicCache`). - [x] 1c-ii. Create flow on one port: `/_connect` → `/_mint` → `/_wire` (`mintPage`, `proxyMint`, `wire`). - [x] 1c-iii. Ids saved by explicit user action under the holder's petname (`pmSave`); pages show a fingerprint (`nick`). - [x] 1c-iv. Verified against live diigo in dev, 2026-10-07: create, dump, publish; `/public/dckc-madmode?page=4` → `cache: HIT`. - [x] 1c-v. Tests: `pnpm lint && pnpm test` (the real worker + DOs against [a fake webapp](diigo-cache/tests/fake-webapp.js)). - [x] 1c-vi. Complete: a filled cache holds everything in diigo's RSS export (16,744 of 16,744; differences are form only), 2026-10-07, via `scripts/compare-export.ts`. Getting there found diigo's `page_num` is 0-based (schema v2 migration). - [ ] **1d. Date windows and search on the public page**, toward the weekly PESOS roundup of [#23](https://github.com/dckc/madmode-blog/issues/23) (a week's permalink: `in=2026-W41`). One filter, two spellings: devs pass `{ days: 7, q: "#weekly" }`, URLs the same fields; `q` takes gmail/Slack/GitHub operators. UTC, ISO weeks, half-open windows; a fetch budget per request. Design: the API in `diigo-cache/src/query.ts` and the skipped suites in `tests/query.test.ts` and `tests/caches.test.ts` ("public page filters"); done when they're unskipped and green. Reconcile with 1f first: `query.ts` plans to filter while walking diigo's pages (binary search by `updated`), but after 1c-vi a filled cache holds every item, so 1d can query `items` in sqlite and spend diigo fetches only on a head refresh (1f). Choose before implementing; the skipped suites may need adjusting. - [x] **1e. Fill job: a button that grabs every page** into a cache from the webkey page `/cache/#`, which trades the key for a capability over Cap'n Web (`diigo-cache/src/api.ts`; `startFill`/`fillStatus`/`stopFill` in `src/cache.ts`; `tests/fill.test.ts`). `/cache/` without a key is a login form the password manager fills. - [ ] **1f. Cached pages go stale: the head must refresh.** A page hit never expires, and diigo pages are updated-desc, so every new or edited bookmark shifts the page boundaries: this week's bookmarks sit on page 1, cached last week. Proposal: refetch the head (pages from 1 until a fetched page's items are all already stored with the same `updated`), upserting into `items` (already keyed by `item_id`); the window/search reads of 1d run over `items`, not over cached page numbers. A fill or a cron alarm (`ctx.storage` alarm in the cache DO, not in madmode) can drive it. **Durable state**: additive only if `pages`/`page_members` keep their meaning; say so before changing them (capsite AGENTS.md). Needs a test: an item added to the fake webapp shows up after a refresh. Stopgap today: "invalidate the last N days" on the webkey page (`invalidateRecent`), then fill. - [ ] **1g. Policy: notes on public items are public too.** The public cache drops private *items* (`publicOnly`) but keeps public items whole, including annotations, comments, and sticky notes that diigo marks private (`pri_a_count`/`pri_c_count`/ `pri_sticky_count`: 2,525 of 14,645 stored public items, found 2026-10-08), and `/public/` JSON serves them. **Dan finds this acceptable as is**; this goal is to finish thinking it through, not to fix a leak. Open points: does diigo.com's own public view hide them (so readers are surprised)? the `comments` array may carry *other people's* words; which fields does potluck quote (annotations are the best commentary: see [the issues sketch](potluck/issues-sketch.md))? If the policy changes, stripping stored rows is a durable-state migration. - [ ] **2. madmode reads the public cache**: public data, so no capability, no cookie, no plugin sync job, no cron on madmode's side. madmode fetches `GET /public/dckc-madmode` with `Accept: application/json` (plus 1d's `in=`/`days=`/`q=`). - [ ] 2-i. Transport: a service binding to diigo-cache when both are deployed (no public hop, so no rate-limit exposure); in dev, a base URL (`http://localhost:8787`). - [ ] 2-ii. The JSON shape of the public page is a contract between two workers: pin it in a diigo-cache test (and madmode's reader tests against the same fixture). - [ ] 2-iii. Decide `DiigoBridge`'s fate: madmode doesn't need it (public reads need no cache id), so its "cache-id source" TODO (`src/index.ts`, README "RPC use") is either dropped with the entrypoint or kept for a private-reading caller. - [ ] **3. `/bookmarks` renders real data** — titles, tags, dates at http://localhost:4321/bookmarks, newest first. - [ ] **4. The weekly page**: `/bookmarks/2026-W41` → the cache's `in=2026-W41` (needs 1d and 1f). Decide (user's call): a raw week list, or a draft post the user edits before publishing (the PESOS practice is editorial; a tag like `#weekly` in `q` is one cheap form of discretion). - [ ] **5. Deployed end-to-end** (order matters): - [ ] 5-i. webapp: `cd webapp && npx wrangler deploy` - [ ] 5-ii. diigo-cache: rate-limit `/api` (and `/public/*`) first (diigo-cache README, "Deployment concern"); deploy; create against prod (ids → password manager; no worker secret); run the fill job. - [ ] 5-iii. madmode: deploy with the service binding; the weekly page renders against prod. - [ ] **6. Cookie expiry**: when diigo's session ends, head refreshes fail; the public page keeps serving what's stored (say so on the page, don't fail silently). Rotation is manual: re-mint via `/_connect`. Open question: can an existing cache be re-wired to a new webapp id (keeping its cache id and public username), or does re-mint mean a new cache? Prefer re-wire — durable ids. - [ ] **7. bib sync #195**: repeat the pattern for the next read-access need (webapp mints for any origin; the cache's item shape is diigo-specific): [the zotero goal tree](zotero-checklist.md). The social.coop account is next after that: [the social.coop goal tree](social-coop-checklist.md). - [ ] **webapp graduates from prototype** (optional; only if it earns it) - [ ] TODO(webapp): one canonical mint form — proxy-page (current) vs running webapp as the primary worker so its own port serves the form directly; revisit as the UI grows. - [ ] @endo/errors for structured error details (needs node_modules). - [ ] **toybox: typed pieces that fit together, peer to peer** (working name; tinker toys per the madmode.com home page). Kinds are typed interfaces, à la Genode session interfaces, but wired by introduction (E's Granovetter diagram), not a parent tree; named by petnames; each piece guards its own resources. Why: [the toybox user stories](toybox-stories.md). - [ ] **T1. Changed signal**: a reader can subscribe to "this cache changed" instead of polling (Genode's ROM-session sigh is the model: public cache ≈ ROM, `/_publish` ≈ report). Fires on 1f's head refresh when it upserts anything new. Shape TBD: a Cap'n Web callback stub the cache holds (live only), or a durable subscription it calls back via a webkey (survives eviction). - [ ] **T2. Compose with [Blockly](https://developers.google.com/blockly)**: connector checks (`setCheck(kind)`) are the shapes; a workspace is a set of introductions. **Blocks refer to capabilities by petname**, never by key: workspace JSON is shareable; the keys live with whatever runs it. - [ ] **T3. Run compositions in [Dynamic Workers](https://developers.cloudflare.com/dynamic-workers/)**: code from anywhere: Blockly's JS generator, code written by agents, by hand. Confinement makes codegen safe: the Worker Loader isolate's `env` holds only the stubs its petnames resolve to, with `globalOutbound: null`, so the code's authority is exactly what it was handed. (Ordinary Workers forbid `eval`; lifting that is the point of the Worker Loader.) ## How to run (continuation prompts) Dev (diigo-cache + webapp in one workerd): cd ~/oros-bots/capsite/diigo-cache pnpm dev # → http://localhost:8787 # webapp rides along (binding-only) First connect (or re-connect after cookie expiry) — browser flow, two steps: open http://localhost:8787/ # 1. "mint the diigo session at webapp" → webapp's own mint form # (diigo-prefilled, served on :8787); paste the Cookie header from # diigo.com devtools → mint (the cookie transits diigo-cache as an # opaque relay into webapp's DO only) # 2. the wire step: webapp id pre-filled, choose public|private → # cacheId; copy the ids somewhere you control (password manager), # paste the cacheId into the dump form # (no .dev.vars, no restarts: ids ride the form POSTs / your paste) Terminal flow (the same shape, scripted): wid=$(curl -s -X POST http://localhost:8787/_mint --data-urlencode \ 'url=https://www.diigo.com/interact_api/' --data-urlencode \ 'headers={"user-agent":"Mozilla/5.0","x-requested-with":"XMLHttpRequest","cookie":""}' \ --data-urlencode 'msPerCall=500' -D - -o /dev/null | grep -i x-webapp-id) curl -s -X POST http://localhost:8787/_wire -H 'content-type: application/json' \ -d "{\"webappId\":\"\",\"mode\":\"private\"}" Then: enter a page number, dump → JSON rows with a `cache: HIT/MISS` line (a repeat page is answered from DO sqlite storage). GET / is inert (dumps happen only on POST submits). Public pages: /public/?page=n (each page fills from diigo on first visit). A private-only cache can be published later from its created page (`POST /_publish`, cache id in the body). Fill and invalidate a cache: http://localhost:8787/cache/ (log in with the cache id from the password manager). Checks: `cd diigo-cache && pnpm lint && pnpm test`. Against a diigo RSS export: `node scripts/compare-export.ts `. madmode (EmDash site; separate worker, separate concerns): cd ~/oros-bots/capsite/madmode && pnpm dev # → :4321 # admin at /_emdash/admin; dev-bypass auth helper: # /_emdash/api/setup/dev-bypass?redirect=/_emdash/admin ## Working rules Agent working rules are in [AGENTS.md](AGENTS.md) — dev servers are the user's job; durable state (loud about breaking changes); secrets discipline; keep config types generated. They apply to every goal in this tree. ## Pointers - [the toybox user stories](toybox-stories.md) — what the pieces are for - [the zotero goal tree](zotero-checklist.md) and [the social.coop goal tree](social-coop-checklist.md) — the next caches - [the capsite notes](capsite-notes.md) — the original goal and the EmDash native-plugin wiring details - [docs/webkeys.md](docs/webkeys.md) — the Waterken URL-capability pattern, abstracted from the E-on-Java sources (bearer secrets, YURL32, secret≠address); the reason ids ride only fragments and POST bodies - [the mm-diigo diigo-bak scripts](https://github.com/dckc/mm-diigo) — provenance of the fluent client and dump generator (`projects/diigo-bak/`: WebApp.js, diigo-dump.mjs, diigo-interact.mjs; mm-diigo commit 7dbb367, Oct 6 2026) - [webapp/README.md](webapp/README.md) — HTTP + RPC surfaces of the factory worker; open TODO on the mint-form placement - [diigo-cache/README.md](diigo-cache/README.md) — the cache worker: create flow, the two DOs, run/deploy details