captoys/diigo-cache

diigo-cache

Cloudflare Worker cache over the webapp factory: session-cookie reads of diigo bookmarks, cached in Durable-Object SQLite storage. Evolved from the diigo-dump worker (provenance: mm-diigo projects/diigo-bak/); the static API-key path is dead (see the capsite checklist, goal 2 preamble).

What it holds

• The create flow, on one port: /_connect renders webapp's own mint form (diigo-prefilled, cookie blank) — wrangler multi-config dev doesn't serve aux workers' HTTP, so /_mint relays the form opaquely into the WEBAPP binding (the cookie crosses; nothing on this side opens or logs it). Mint → /_wire (webapp id pre-filled, mode public|private) → the created page, where the user saves the ids.
• Two independent page caches (src/cache.ts), one behavior: a page hit is served from storage with no diigo call; a miss asks the webapp instance in the cache's wiring and etches the page.
  • DiigoCache (binding CACHE_DO): every item. Addressed by its cache id.
  • PublicCache (binding PUBLIC_DO): shared items only — private rows are dropped before etching (publicOnly), so its storage holds nothing private. Same cookie webapp as the private cache. Public page n is diigo page n, filtered (may show fewer than PAGE_SIZE, 100: diigo's cap). /public/<username> resolves through the idFromName("index") registry.
• Pages: home's dump form (cache id + page number, POST) shows raw dump rows; /public/<username>?page=n shows an HTML list with a page form (JSON for Accept: application/json). Both say cache: HIT or MISS; every page read also logs {event: "cache", kind, page, outcome}.
• Publish later: POST /_publish (cache id in the body) wires a public companion to a private-only cache.
• The webkey page and Cap'n Web: /cache/#<cache id> is a skeleton page, the same for every key; its script reads the key from the fragment and trades it in-band for a capability over Cap'n Web at /api (CacheApi.cache(key) → a CacheCap facet: fill, status, the public companion; src/api.ts). The worker serves capnweb's browser build itself at /_capnweb.js.
• Fill job: walks every diigo page into a cache in the background — DO alarms, a few pages per step, progress stored in the DO (startFill/fillStatus/stopFill), driven from the webkey page; the public page shows progress. "Invalidate the last N days" there drops the cached pages holding anything updated that recently (invalidateRecent), so the next fill refetches them.
• The DiigoBridge entrypoint for other workers (its cache-id source is the goal-2 wiring TODO).

State surface

Ids are webkeys (the pattern): 256-bit bearer secrets that never ride a query string or path. The webkey page carries the cache id in the URL fragment (never sent to the server) and passes it in-band over Cap'n Web; forms carry ids in POST bodies; logs and pages show only a fingerprint (a prefix of the id's one-way hash).

• The cache id: the cache's read capability (bearer; not logged). Held by the user — saved to the password manager under a petname they choose (the username field; never sent to the server) — and passed per POST.
• The webapp instance id: the diigo-read capability; lives in each cache DO's wiring, arrives from the mint step.
• The cookie: lives only in webapp's DO storage; crosses the wire once, at mint time, as an opaque relay.
• The public cache: no capability — /public/<username> is public by design (a guessable URL; hiding the link doesn't hide the page).

Stored state is meant to be durable: see the breaking-change rule in capsite AGENTS.md.

Deployment concern: public pages spend your diigo session

Anyone who can reach /public/<username> can make the cache fetch from diigo through your cookie session: each request may spend up to FETCH_BUDGET (40) diigo fetches on uncached pages (goal 1d), paced at MS_PER_CALL. Cached pages cost nothing. Before exposing a deployment:

• rate-limit the public routes and /api (Cap'n Web pipelining makes calls cheap to send) — on Cloudflare, the Workers rate-limit binding (keyed by client and/or username); on other workerd hosts, a rate limit in the proxy in front;
• run the fill job (goal 1e) so most pages are cached before strangers arrive.

RPC use (from another worker, same account)

// wrangler.jsonc of the caller
"services": [{ "binding": "DIIGO", "service": "diigo-cache", "entrypoint": "DiigoBridge" }]

// cache ids ride goal-2's wiring; see the capsite checklist
const ndjson = await env.DIIGO.dumpNdjson(4); // 4 pages of PAGE_SIZE

Test

pnpm types  # regenerate worker-configuration.d.ts (after wrangler.jsonc changes)
pnpm lint   # fails if those types are stale; tsc over src/ and tests/
pnpm test   # vitest

Tests run the real worker and both DOs in the vitest pool, with WEBAPP bound to a fake webapp serving diigo-shaped items (wrangler.tests.jsonc).

Run, deploy

How to run / connect / deploy is maintained in the capsite checklist (single source; this README keeps only worker-level facts).

Status: create, dump, publish, and public pages exercised against live diigo in dev (2026-10-07); not yet deployed.