dckc/captoys
1
0 Fork 0
Code Issues Pull Requests 4 Actions Packages Projects Releases Wiki Activity

docs: goal tree and rules track the 1c cache

Browse Source 
- checklist: 1c (diigo-dump evolves into a bookmark cache) with its
  create-flow, id-wiring, and verification subgoals
- notes: diigo-cache replaces diigo-dump in What-we-built
- AGENTS: no worker secret since 1c; durable state — be LOUD and
  conservative about breaking changes to stored DB state; pointer to
  the workspace skills for commit trailers

Generated-by: opencode (GLM)
Generated-by: Claude Opus 5.5
This commit is contained in:
dckc committed 2026-10-07 04:19:54 -05:00
1 parent 6f409fc83a
commit d0a9b899f2
3 files changed
+120 -58
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

capsite-checklist.md
+86 -51
View File
@@ -12,50 +12,69 @@ narrative.
[the capsite notes](capsite-notes.md); after step 2, expands to
[bib sync #195](https://github.com/dckc/madmode-blog/issues/195))
- [x] Credential discipline established: cookies/keys/secrets live in
Worker secrets or DO storage, never in source, config, or the
plugin. (`.dev.vars` gitignored: only `DIIGO_WEB_ID`;
cookie lives in the webapp DO's storage since the 1b split.)
- [x] **1. Cookie-path dump worker** (`diigo-dump/`): port of
mm-diigo's diigo-dump.mjs. GET inert, dump only on POST.
Verified 2026-10-06: 200, `total: 1674`. (Scrubbed: static
API-key ping worker 401'd — the cookie path was the one that
worked; see the API-key subgoal below for where that left us.)
- [x] **1b. webapp DO factory** (`webapp/`): credentialed authority
in a Durable Object; ids are bearer capabilities (not logged);
factory worker is site-agnostic; consumer holds only the id.
Dev wiring verified: `pnpm dev` boots both, home + /_connect
answer on :8787.
- [ ] 1b-i. Mint through the running dev server end-to-end:
POST /_mint (or /_connect page) with a fresh cookie →
id → dump page 1 through the DO → 200/items.
(Everything up to mint is verified; this closes the loop.)
- [ ] 1b-ii. Prod mint + handoff: deploy both, mint against
prod, `wrangler secret put DIIGO_WEB_ID`, dump a page.
- [x] Credential discipline: no worker secrets; the cookie lives only
in webapp's DO storage, ids are held by the user
([AGENTS.md](AGENTS.md), secrets section).
- [x] **1. Cookie-path dump worker**: port of mm-diigo's diigo-dump.mjs
(verified 2026-10-06: 200, `total: 1674`); evolved into 1c.
- [x] **1b. webapp DO factory** ([webapp/README.md](webapp/README.md));
mint verified end-to-end in dev 2026-10-07 (via 1c). Prod mint is
goal 5.
- [x] **1c. diigo-dump evolves into a bookmark cache**
([diigo-cache/README.md](diigo-cache/README.md); design in the
JSDoc of `src/cache.ts`, `src/index.ts`, `src/pm-save.ts`):
- [x] 1c-i. Two independent page caches, private and public
(`PagedCache`, `DiigoCache`, `PublicCache`).
- [x] 1c-ii. Create flow on one port: `/_connect` → `/_mint`
→ `/_wire` (`mintPage`, `proxyMint`, `wire`).
- [x] 1c-iii. Ids saved by explicit user action under the
holder's petname (`pmSave`); pages show a fingerprint
(`nick`).
- [x] 1c-iv. Verified against live diigo in dev, 2026-10-07:
create, dump, publish; `/public/dckc-madmode?page=4` →
`cache: HIT`.
- [x] 1c-v. Tests: `pnpm lint && pnpm test` (the real worker +
DOs against [a fake webapp](diigo-cache/tests/fake-webapp.js)).
- [ ] **1d. Date windows and search on the public page**, toward the
weekly PESOS roundup of [#23](https://github.com/dckc/madmode-blog/issues/23)
(a week's permalink: `in=2026-W41`). One filter, two spellings:
devs pass `{ days: 7, q: "#weekly" }`, URLs the same fields;
`q` takes gmail/Slack/GitHub operators. UTC, ISO weeks, half-open
windows; a fetch budget per request. Design: the API in
`diigo-cache/src/query.ts` and the skipped suites in
`tests/query.test.ts` and `tests/caches.test.ts` ("public page
filters"); done when they're unskipped and green.
- [ ] **1e. Fill job: a button that grabs every page** into a cache
(owner-only `POST /_fill`, cache id in the body; DO alarms, a
few pages per step; progress in storage; skips cached pages;
stop; fails loudly). Makes old `by=created` windows cheap. Design:
`startFill`/`fillStatus`/`stopFill` in `diigo-cache/src/cache.ts`
and the skipped suites in `tests/fill.test.ts`; done when they're
unskipped and green.
- [ ] **2. madmode collects real bookmarks via the webapp cookie path**
(API-key path is DEAD — verified in session
ses_eec320fe4ffehaMAn44U1nTKJU: both known real keys, all
documented schemes (key= param, key+user=, HTTP Basic), both
/api/v2/bookmarks and /api/v2/notes → 401 unconditionally, with
2024–2025 historical proof the same creds once downloaded 13.5k
bookmarks. Diigo deprecated/revoked v2 key auth; the session-
cookie /interact_api channel is the one that works.)
- [ ] 2-i. Plugin rides the webapp DO: plugin's sync calls
diigo-dump's DiigoBridge (or the instance id directly);
cookie + host gate stay host-side.
(the API-key path is dead: [evidence in the notes](capsite-notes.md#where-the-diigo-credentials-go))
- [ ] 2-i. Plugin rides the cache: plugin's sync calls
diigo-cache's DiigoBridge (cache id by wiring, not
secret); cookie + host gate stay host-side.
- [ ] 2-ii. Plugin "Sync now" succeeds: status 200, bookmarks
stored (admin at /_emdash/admin → Diigo Sync).
- [ ] 2-iii. Cookie-expiry story for cron: rotation is manual
(re-mint via /_connect) unless we automate it later.
(re-create via the home page form; the cache id rotates
with it) unless we automate it later.
- [ ] **3. `/bookmarks` renders real data** after a sync — titles/tags at
http://localhost:4321/bookmarks.
- [ ] **4. Cron sync** works locally without errors (`0 */6 * * *` in
`_emdash_cron_tasks`).
- [ ] **5. Deployed end-to-end** (order matters; webapp first):
- [ ] 5-i. webapp: `cd webapp && npx wrangler deploy`
- [ ] 5-ii. diigo-dump: deploy + prod mint + `wrangler secret
put DIIGO_WEB_ID` (id from x-diigo-web-id header)
- [ ] 5-ii. diigo-cache: deploy + first create against prod
(ids → password manager; no worker secret put needed);
rate-limit the public routes first (diigo-cache README,
"Deployment concern").
- [ ] 5-iii. madmode: `pnpm wrangler deploy`, secrets via
`wrangler secret put`; plugin "Sync now" against prod
- [ ] **6. bib sync #195**: repeat the pattern for the next
@@ -68,29 +87,41 @@ narrative.
## How to run (continuation prompts)
Dev (diigo-dump + webapp in one workerd):
Dev (diigo-cache + webapp in one workerd):
cd ~/oros-bots/capsite/diigo-dump
cd ~/oros-bots/capsite/diigo-cache
pnpm dev # → http://localhost:8787
# webapp rides along (binding-only)
First connect (or re-connect after cookie expiry) — browser flow, primary:
First connect (or re-connect after cookie expiry) — browser flow, two steps:
open http://localhost:8787/_connect
# paste the Cookie header from diigo.com devtools; mint
# success page shows a copy button for the id
# add DIIGO_WEB_ID="<id>" to diigo-dump/.dev.vars (only the id lives
# on disk; the cookie lives in the DO's storage)
# restart wrangler (pnpm dev) to pick it up
open http://localhost:8787/
# 1. "mint the diigo session at webapp" → webapp's own mint form
# (diigo-prefilled, served on :8787); paste the Cookie header from
# diigo.com devtools → mint (the cookie transits diigo-cache as an
# opaque relay into webapp's DO only)
# 2. the wire step: webapp id pre-filled, choose public|private →
# cacheId; copy the ids somewhere you control (password manager),
# paste the cacheId into the dump form
# (no .dev.vars, no restarts: ids ride the form POSTs / your paste)
Terminal flow (superseded by the page; needs a display):
Terminal flow (the same shape, scripted):
pnpm -s diigo-login >> .dev.vars # askpass prompt; appends the id line
# (.dev.vars: pnpm stdout noise can
# interleave; grep ^DIIGO_ to clean)
wid=$(curl -s -X POST http://localhost:8787/_mint --data-urlencode \
'url=https://www.diigo.com/interact_api/' --data-urlencode \
'headers={"user-agent":"Mozilla/5.0","x-requested-with":"XMLHttpRequest","cookie":"<paste>"}' \
--data-urlencode 'msPerCall=500' -D - -o /dev/null | grep -i x-webapp-id)
curl -s -X POST http://localhost:8787/_wire -H 'content-type: application/json' \
-d "{\"webappId\":\"<from $wid>\",\"mode\":\"private\"}"
Then: enter a page number on the home page, dump → JSON items. GET /
is inert (no accidental fetches; dumps happen only on POST submits).
Then: enter a page number, dump → JSON rows with a `cache: HIT/MISS`
line (a repeat page is answered from DO sqlite storage). GET / is inert
(dumps happen only on POST submits). Public pages:
/public/<username>?page=n (each page fills from diigo on first visit).
A private-only cache can be published later from its created page
(`POST /_publish`, cache id in the body).
Checks: `cd diigo-cache && pnpm lint && pnpm test`.
madmode (EmDash site; separate worker, separate concerns):
@@ -101,18 +132,22 @@ madmode (EmDash site; separate worker, separate concerns):
## Working rules
Agent working rules are in [AGENTS.md](AGENTS.md) — dev servers are the
user's job; secrets discipline; keep config types generated. They apply
user's job; durable state (loud about breaking changes); secrets
discipline; keep config types generated. They apply
to every goal in this tree.
## Pointers
- [the capsite notes](capsite-notes.md) — the original goal and the EmDash
native-plugin wiring details
- [docs/webkeys.md](docs/webkeys.md) — the Waterken URL-capability
pattern, abstracted from the E-on-Java sources (bearer secrets,
YURL32, secret≠address); the reason ids never ride URLs
- [the mm-diigo diigo-bak scripts](https://github.com/dckc/mm-diigo) —
provenance of the fluent client and dump generator (`projects/diigo-bak/`:
WebApp.js, diigo-dump.mjs, diigo-interact.mjs; mm-diigo commit 7dbb367,
Oct 6 2026)
- [webapp/README.md](webapp/README.md) — HTTP + RPC surfaces of the
factory worker; open TODO on the mint-form placement
- [diigo-dump/README.md](diigo-dump/README.md) — consumer wiring and
run/deploy details
- [diigo-cache/README.md](diigo-cache/README.md) — the cache worker:
create flow, the two DOs, run/deploy details