captoys/capsite-checklist.md

# capsite goal tree

Nested checkboxes form the goal tree itself: each `- [ ]` is a goal; its
indented children are the subgoals/steps that must pass before it (an item
is only checked when **all** its children are checked — or, for leaf
items, when it has passed verification itself). Working notes live under
each goal; [the capsite notes](capsite-notes.md) carry the background
narrative.

- [ ] **madmode publishes a weekly roundup of diigo bookmarks**
      ([#23](https://github.com/dckc/madmode-blog/issues/23)); the diigo
      session stays inside webapp, so madmode holds no credential at all
      (background: [the capsite notes](capsite-notes.md); after that,
      [bib sync #195](https://github.com/dckc/madmode-blog/issues/195))

  - [x] Credential discipline: no worker secrets; the cookie lives only
        in webapp's DO storage, ids are held by the user
        ([AGENTS.md](AGENTS.md), secrets section).
  - [x] **1. Cookie-path dump worker**: port of mm-diigo's diigo-dump.mjs
        (verified 2026-10-06: 200, `total: 1674`); evolved into 1c.
  - [x] **1b. webapp DO factory** ([webapp/README.md](webapp/README.md));
        mint verified end-to-end in dev 2026-10-07 (via 1c). Prod mint is
        goal 5.

  - [x] **1c. diigo-dump evolves into a bookmark cache**
        ([diigo-cache/README.md](diigo-cache/README.md); design in the
        JSDoc of `src/cache.ts`, `src/index.ts`, `src/pm-save.ts`):
        - [x] 1c-i. Two independent page caches, private and public
              (`PagedCache`, `DiigoCache`, `PublicCache`).
        - [x] 1c-ii. Create flow on one port: `/_connect` → `/_mint`
              → `/_wire` (`mintPage`, `proxyMint`, `wire`).
        - [x] 1c-iii. Ids saved by explicit user action under the
              holder's petname (`pmSave`); pages show a fingerprint
              (`nick`).
        - [x] 1c-iv. Verified against live diigo in dev, 2026-10-07:
              create, dump, publish; `/public/dckc-madmode?page=4` →
              `cache: HIT`.
        - [x] 1c-v. Tests: `pnpm lint && pnpm test` (the real worker +
              DOs against [a fake webapp](diigo-cache/tests/fake-webapp.js)).
        - [x] 1c-vi. Complete: a filled cache holds everything in diigo's
              RSS export (16,744 of 16,744; differences are form only),
              2026-10-07, via `scripts/compare-export.ts`. Getting there
              found diigo's `page_num` is 0-based (schema v2 migration).

  - [ ] **1d. Date windows and search on the public page**, toward the
        weekly PESOS roundup of [#23](https://github.com/dckc/madmode-blog/issues/23)
        (a week's permalink: `in=2026-W41`). One filter, two spellings:
        devs pass `{ days: 7, q: "#weekly" }`, URLs the same fields;
        `q` takes gmail/Slack/GitHub operators. UTC, ISO weeks, half-open
        windows; a fetch budget per request. Design: the API in
        `diigo-cache/src/query.ts` and the skipped suites in
        `tests/query.test.ts` and `tests/caches.test.ts` ("public page
        filters"); done when they're unskipped and green.

  - [x] **1e. Fill job: a button that grabs every page** into a cache
        from the webkey page `/cache/#<key>`, which trades the key for a
        capability over Cap'n Web (`diigo-cache/src/api.ts`;
        `startFill`/`fillStatus`/`stopFill` in `src/cache.ts`;
        `tests/fill.test.ts`). `/cache/` without a key is a login form
        the password manager fills.

  - [ ] **1f. Cached pages go stale: the head must refresh.** A page hit
        never expires, and diigo pages are updated-desc, so every new or
        edited bookmark shifts the page boundaries: this week's bookmarks
        sit on page 1, cached last week. Proposal: refetch the head
        (pages from 1 until a fetched page's items are all already stored
        with the same `updated`), upserting into `items` (already keyed by
        `item_id`); the window/search reads of 1d run over `items`, not
        over cached page numbers. A fill or a cron alarm (`ctx.storage`
        alarm in the cache DO, not in madmode) can drive it.
        **Durable state**: additive only if `pages`/`page_members` keep
        their meaning; say so before changing them (capsite AGENTS.md).
        Needs a test: an item added to the fake webapp shows up after a
        refresh. Stopgap today: "invalidate the last N days" on the
        webkey page (`invalidateRecent`), then fill.

  - [ ] **2. madmode reads the public cache**: public data, so no
        capability, no cookie, no plugin sync job, no cron on madmode's
        side. madmode fetches `GET /public/dckc-madmode` with
        `Accept: application/json` (plus 1d's `in=`/`days=`/`q=`).
        - [ ] 2-i. Transport: a service binding to diigo-cache when both
              are deployed (no public hop, so no rate-limit exposure);
              in dev, a base URL (`http://localhost:8787`).
        - [ ] 2-ii. The JSON shape of the public page is a contract
              between two workers: pin it in a diigo-cache test (and
              madmode's reader tests against the same fixture).
  - [ ] **3. `/bookmarks` renders real data** — titles, tags, dates at
        http://localhost:4321/bookmarks, newest first.
  - [ ] **4. The weekly page**: `/bookmarks/2026-W41` → the cache's
        `in=2026-W41` (needs 1d and 1f). Decide (user's call): a raw
        week list, or a draft post the user edits before publishing
        (the PESOS practice is editorial; a tag like `#weekly` in `q`
        is one cheap form of discretion).
  - [ ] **5. Deployed end-to-end** (order matters):
        - [ ] 5-i. webapp: `cd webapp && npx wrangler deploy`
        - [ ] 5-ii. diigo-cache: rate-limit `/api` (and `/public/*`) first
              (diigo-cache README, "Deployment concern"); deploy; create
              against prod (ids → password manager; no worker secret);
              run the fill job.
        - [ ] 5-iii. madmode: deploy with the service binding; the
              weekly page renders against prod.
  - [ ] **6. Cookie expiry**: when diigo's session ends, head refreshes
        fail; the public page keeps serving what's stored (say so on the
        page, don't fail silently). Rotation is manual: re-mint via
        `/_connect`. Open question: can an existing cache be re-wired to
        a new webapp id (keeping its cache id and public username), or
        does re-mint mean a new cache? Prefer re-wire — durable ids.
  - [ ] **7. bib sync #195**: repeat the pattern for the next
        read-access need (webapp mints for any origin; the cache's
        item shape is diigo-specific).
- [ ] **webapp graduates from prototype** (optional; only if it earns it)
  - [ ] TODO(webapp): one canonical mint form — proxy-page (current) vs
        running webapp as the primary worker so its own port serves the
        form directly; revisit as the UI grows.
  - [ ] @endo/errors for structured error details (needs node_modules).

## How to run (continuation prompts)

Dev (diigo-cache + webapp in one workerd):

    cd ~/oros-bots/capsite/diigo-cache
    pnpm dev                        # → http://localhost:8787
                                    # webapp rides along (binding-only)

First connect (or re-connect after cookie expiry) — browser flow, two steps:

    open http://localhost:8787/
    # 1. "mint the diigo session at webapp" → webapp's own mint form
    #    (diigo-prefilled, served on :8787); paste the Cookie header from
    #    diigo.com devtools → mint (the cookie transits diigo-cache as an
    #    opaque relay into webapp's DO only)
    # 2. the wire step: webapp id pre-filled, choose public|private →
    #    cacheId; copy the ids somewhere you control (password manager),
    #    paste the cacheId into the dump form
    # (no .dev.vars, no restarts: ids ride the form POSTs / your paste)

Terminal flow (the same shape, scripted):

    wid=$(curl -s -X POST http://localhost:8787/_mint --data-urlencode \
      'url=https://www.diigo.com/interact_api/' --data-urlencode \
      'headers={"user-agent":"Mozilla/5.0","x-requested-with":"XMLHttpRequest","cookie":"<paste>"}' \
      --data-urlencode 'msPerCall=500' -D - -o /dev/null | grep -i x-webapp-id)
    curl -s -X POST http://localhost:8787/_wire -H 'content-type: application/json' \
      -d "{\"webappId\":\"<from $wid>\",\"mode\":\"private\"}"

Then: enter a page number, dump → JSON rows with a `cache: HIT/MISS`
line (a repeat page is answered from DO sqlite storage). GET / is inert
(dumps happen only on POST submits). Public pages:
/public/<username>?page=n (each page fills from diigo on first visit).
A private-only cache can be published later from its created page
(`POST /_publish`, cache id in the body).

Fill and invalidate a cache: http://localhost:8787/cache/ (log in with
the cache id from the password manager).

Checks: `cd diigo-cache && pnpm lint && pnpm test`. Against a diigo RSS
export: `node scripts/compare-export.ts <cache.sqlite> <export.xml> <out-dir>`.

madmode (EmDash site; separate worker, separate concerns):

    cd ~/oros-bots/capsite/madmode && pnpm dev   # → :4321
    # admin at /_emdash/admin; dev-bypass auth helper:
    # /_emdash/api/setup/dev-bypass?redirect=/_emdash/admin

## Working rules

Agent working rules are in [AGENTS.md](AGENTS.md) — dev servers are the
user's job; durable state (loud about breaking changes); secrets
discipline; keep config types generated. They apply
to every goal in this tree.

## Pointers

- [the capsite notes](capsite-notes.md) — the original goal and the EmDash
  native-plugin wiring details
- [docs/webkeys.md](docs/webkeys.md) — the Waterken URL-capability
  pattern, abstracted from the E-on-Java sources (bearer secrets,
  YURL32, secret≠address); the reason ids ride only fragments and POST
  bodies
- [the mm-diigo diigo-bak scripts](https://github.com/dckc/mm-diigo) —
  provenance of the fluent client and dump generator (`projects/diigo-bak/`:
  WebApp.js, diigo-dump.mjs, diigo-interact.mjs; mm-diigo commit 7dbb367,
  Oct 6 2026)
- [webapp/README.md](webapp/README.md) — HTTP + RPC surfaces of the
  factory worker; open TODO on the mint-form placement
- [diigo-cache/README.md](diigo-cache/README.md) — the cache worker:
  create flow, the two DOs, run/deploy details