captoys/capsite-checklist.md

capsite goal tree

Nested checkboxes form the goal tree itself: each - [ ] is a goal; its indented children are the subgoals/steps that must pass before it (an item is only checked when all its children are checked — or, for leaf items, when it has passed verification itself). Working notes live under each goal; the capsite notes carry the background narrative.

• madmode publishes a weekly roundup of diigo bookmarks (#23); the diigo session stays inside webapp, so madmode holds no credential at all (background: the capsite notes; after that, bib sync #195)

  • Credential discipline: no worker secrets; the cookie lives only in webapp's DO storage, ids are held by the user (AGENTS.md, secrets section).

  • 1. Cookie-path dump worker: port of mm-diigo's diigo-dump.mjs (verified 2026-10-06: 200, total: 1674); evolved into 1c.

  • 1b. webapp DO factory (webapp/README.md); mint verified end-to-end in dev 2026-10-07 (via 1c). Prod mint is goal 5.

  • 1c. diigo-dump evolves into a bookmark cache (diigo-cache/CONTRIBUTING.md; design in the JSDoc of src/cache.ts, src/index.ts, src/pm-save.ts): - [x] 1c-i. Two independent page caches, private and public (PagedCache, DiigoCache, PublicCache). - [x] 1c-ii. Create flow on one port: /_connect → /_mint → /_wire (mintPage, proxyMint, wire). - [x] 1c-iii. Ids saved by explicit user action under the holder's petname (pmSave); pages show a fingerprint (nick). - [x] 1c-iv. Verified against live diigo in dev, 2026-10-07: create, dump, publish; /public/dckc-madmode?page=4 → cache: HIT. - [x] 1c-v. Tests: pnpm lint && pnpm test (the real worker + DOs against a fake webapp). - [x] 1c-vi. Complete: a filled cache holds everything in diigo's RSS export (16,744 of 16,744; differences are form only), 2026-10-07, via scripts/compare-export.ts. Getting there found diigo's page_num is 0-based (schema v2 migration).

  • 1d. Date windows and search on the public page, toward the weekly PESOS roundup of #23 (a week's permalink: in=2026-W41). One filter, two spellings: devs pass { days: 7, q: "#weekly" }, URLs the same fields; q takes gmail/Slack/GitHub operators. UTC, ISO weeks, half-open windows (diigo-cache/src/query.ts; tests/query.test.ts, tests/caches.test.ts "public page filters"). Filters read stored items only (the public page never fetches): results are as fresh as the cache, and say complete when every diigo page is stored. Keeping them fresh is 1f. 2026-10-09.

  • 1e. Fill job: a button that grabs every page into a cache from the webkey page /cache/#<key>, which trades the key for a capability over Cap'n Web (diigo-cache/src/api.ts; startFill/fillStatus/stopFill in src/cache.ts; tests/fill.test.ts). /cache/ without a key is a login form the password manager fills.

  • 1f. Cached pages go stale: the head must refresh. A page hit never expires, and diigo pages are updated-desc, so every new or edited bookmark shifts the page boundaries: this week's bookmarks sit on page 1, cached last week. Proposal: refetch the head (pages from 1 until a fetched page's items are all already stored with the same updated), upserting into items (already keyed by item_id); the window/search reads of 1d run over items, not over cached page numbers. A fill or a cron alarm (ctx.storage alarm in the cache DO, not in madmode) can drive it. Durable state: additive only if pages/page_members keep their meaning; say so before changing them (capsite AGENTS.md). Needs a test: an item added to the fake webapp shows up after a refresh. Stopgap today: "invalidate the last N days" on the webkey page (invalidateRecent), then fill.

  • 1g. Policy: notes on public items are public too. The public cache drops private items (publicOnly) but keeps public items whole, including annotations, comments, and sticky notes that diigo marks private (pri_a_count/pri_c_count/ pri_sticky_count: 2,525 of 14,645 stored public items, found 2026-10-08), and /public/<username> JSON serves them. Dan finds this acceptable as is; this goal is to finish thinking it through, not to fix a leak. Open points: does diigo.com's own public view hide them (so readers are surprised)? the comments array may carry other people's words; which fields does potluck quote (annotations are the best commentary: see the issues sketch)? If the policy changes, stripping stored rows is a durable-state migration.

  • 2. potluck reads the public cache: public data, so no capability, no cookie, no secret on potluck's side. The diigo-week plugin reads GET /public/dckc-madmode?in=<week>&by=created as microformats (an h-feed of h-entry bookmarks). Built 2026-10-09; not yet run in dev. - [x] 2-i. Transport: the DIIGO_CACHE service binding in potluck's wrangler.jsonc, in dev too: the plugin's ctx.http.fetch refuses localhost (EmDash's SSRF guard), and a binding names one worker, not a host. - [x] 2-ii. The page's microformats are the contract: diigo-cache's tests parse it with microformats-parser; diigo-week's tests read fixtures in the same markup. - [ ] 2-iii. Decide DiigoBridge's fate (postponed: nothing on this path uses it; it matters only for a caller that reads private items). - [ ] 2-iv. Run it in dev: diigo-cache and potluck both up; Plugins → Draft from diigo → a draft issue. Open question: whether the service binding resolves across the two dev servers (the dev registry).

  • 3. /bookmarks renders real data: superseded by potluck's issues (the madmode starter is gone).

  • 4. The weekly draft: decided (2026-10-09): a draft issue you edit before publishing, made by diigo-week. The slug comes from the title (week-2026-w40).

  • 5. Deployed end-to-end (order matters): - [ ] 5-i. webapp: cd webapp && npx wrangler deploy - [ ] 5-ii. diigo-cache: rate-limit /api (and /public/*) first (diigo-cache CONTRIBUTING, "Deployment concern"); deploy; create against prod (ids → password manager; no worker secret); run the fill job. - [ ] 5-iii. madmode: deploy with the service binding; the weekly page renders against prod.

  • 6. Cookie expiry: when diigo's session ends, head refreshes fail; the public page keeps serving what's stored (say so on the page, don't fail silently). Rotation is manual: re-mint via /_connect. Open question: can an existing cache be re-wired to a new webapp id (keeping its cache id and public username), or does re-mint mean a new cache? Prefer re-wire — durable ids.

  • 7. bib sync #195: repeat the pattern for the next read-access need (webapp mints for any origin; the cache's item shape is diigo-specific): the zotero goal tree. The social.coop account is next after that: the social.coop goal tree.

• webapp graduates from prototype (optional; only if it earns it)

  • TODO(webapp): one canonical mint form — proxy-page (current) vs running webapp as the primary worker so its own port serves the form directly; revisit as the UI grows.
  • @endo/errors for structured error details (needs node_modules).

• toybox: typed pieces that fit together, peer to peer (working name; tinker toys per the madmode.com home page). Kinds are typed interfaces, à la Genode session interfaces, but wired by introduction (E's Granovetter diagram), not a parent tree; named by petnames; each piece guards its own resources. Why: the toybox user stories.

  • T1. Changed signal: a reader can subscribe to "this cache changed" instead of polling (Genode's ROM-session sigh is the model: public cache ≈ ROM, /_publish ≈ report). Fires on 1f's head refresh when it upserts anything new. Shape TBD: a Cap'n Web callback stub the cache holds (live only), or a durable subscription it calls back via a webkey (survives eviction).
  • T2. Compose with Blockly: connector checks (setCheck(kind)) are the shapes; a workspace is a set of introductions. Blocks refer to capabilities by petname, never by key: workspace JSON is shareable; the keys live with whatever runs it.
  • T3. Run compositions in Dynamic Workers: code from anywhere: Blockly's JS generator, code written by agents, by hand. Confinement makes codegen safe: the Worker Loader isolate's env holds only the stubs its petnames resolve to, with globalOutbound: null, so the code's authority is exactly what it was handed. (Ordinary Workers forbid eval; lifting that is the point of the Worker Loader.)

How to run (continuation prompts)

Dev (diigo-cache + webapp in one workerd):

cd ~/oros-bots/capsite/diigo-cache
pnpm dev                        # → http://localhost:8787
                                # webapp rides along (binding-only)

First connect (or re-connect after cookie expiry) — browser flow, two steps:

open http://localhost:8787/
# 1. "mint the diigo session at webapp" → webapp's own mint form
#    (diigo-prefilled, served on :8787); paste the Cookie header from
#    diigo.com devtools → mint (the cookie transits diigo-cache as an
#    opaque relay into webapp's DO only)
# 2. the wire step: webapp id pre-filled, choose public|private →
#    cacheId; copy the ids somewhere you control (password manager),
#    paste the cacheId into the dump form
# (no .dev.vars, no restarts: ids ride the form POSTs / your paste)

Terminal flow (the same shape, scripted):

wid=$(curl -s -X POST http://localhost:8787/_mint --data-urlencode \
  'url=https://www.diigo.com/interact_api/' --data-urlencode \
  'headers={"user-agent":"Mozilla/5.0","x-requested-with":"XMLHttpRequest","cookie":"<paste>"}' \
  --data-urlencode 'msPerCall=500' -D - -o /dev/null | grep -i x-webapp-id)
curl -s -X POST http://localhost:8787/_wire -H 'content-type: application/json' \
  -d "{\"webappId\":\"<from $wid>\",\"mode\":\"private\"}"

Then: enter a page number, dump → JSON rows with a cache: HIT/MISS line (a repeat page is answered from DO sqlite storage). GET / is inert (dumps happen only on POST submits). Public pages: /public/?page=n (each page fills from diigo on first visit). A private-only cache can be published later from its created page (POST /_publish, cache id in the body).

Fill and invalidate a cache: http://localhost:8787/cache/ (log in with the cache id from the password manager).

Checks: cd diigo-cache && pnpm lint && pnpm test. Against a diigo RSS export: node scripts/compare-export.ts <cache.sqlite> <export.xml> <out-dir>.

potluck (EmDash site, fresh out of the box: pnpm create emdash@latest potluck --pm pnpm --install --yes → cloudflare:blog, 2026-10-08, per Create your first EmDash site; replaces the old madmode/ starter):

cd ~/oros-bots/capsite/potluck && pnpm dev   # → :4321
# first run: the setup wizard at /_emdash/admin (site title, email)
# dev-bypass auth helper:
# /_emdash/api/setup/dev-bypass?redirect=/_emdash/admin

Working rules

Agent working rules are in AGENTS.md — dev servers are the user's job; durable state (loud about breaking changes); secrets discipline; keep config types generated. They apply to every goal in this tree.

Pointers

• the toybox user stories — what the pieces are for
• the zotero goal tree and the social.coop goal tree — the next caches
• the capsite notes — the original goal and the EmDash native-plugin wiring details
• docs/webkeys.md — the Waterken URL-capability pattern, abstracted from the E-on-Java sources (bearer secrets, YURL32, secret≠address); the reason ids ride only fragments and POST bodies
• the mm-diigo diigo-bak scripts — provenance of the fluent client and dump generator (projects/diigo-bak/: WebApp.js, diigo-dump.mjs, diigo-interact.mjs; mm-diigo commit 7dbb367, Oct 6 2026)
• webapp/README.md — HTTP + RPC surfaces of the factory worker; open TODO on the mint-form placement
• diigo-cache/CONTRIBUTING.md — the cache worker: create flow, the two DOs, run/deploy details