captoys/diigo-cache/README.md

# diigo-cache

Cloudflare Worker cache over [the webapp factory](../webapp/README.md):
session-cookie reads of diigo bookmarks, cached in Durable-Object SQLite
storage. Evolved from the diigo-dump worker (provenance:
[mm-diigo `projects/diigo-bak/`](https://github.com/dckc/mm-diigo)); the
static API-key path is dead (see [the capsite checklist](../capsite-checklist.md),
goal 2 preamble).

## What it holds

- **The create flow, on one port**: `/_connect` renders webapp's own
  mint form (diigo-prefilled, cookie blank) — wrangler multi-config dev
  doesn't serve aux workers' HTTP, so `/_mint` relays the form opaquely
  into the WEBAPP binding (the cookie crosses; nothing on this side opens
  or logs it). Mint → `/_wire` (webapp id pre-filled, mode
  public|private) → the created page, where the user saves the ids.
- **Two independent page caches** (`src/cache.ts`), one behavior: a
  page hit is served from storage with no diigo call; a miss asks the
  webapp instance in the cache's wiring and etches the page.
  - `DiigoCache` (binding `CACHE_DO`): every item. Addressed by its
    cache id.
  - `PublicCache` (binding `PUBLIC_DO`): shared items only — private
    rows are dropped *before* etching (`publicOnly`), so its storage
    holds nothing private. Same cookie webapp as the private cache.
    Public page n is diigo page n, filtered (may show fewer than
    `PAGE_SIZE`, 100: diigo's cap).
    `/public/<username>` resolves through the `idFromName("index")`
    registry, and reads through `viewer()`, a read-only facet: it shows
    stored pages and fill progress, and can't fetch from diigo or write.
- **Storage**: SQL lives in `src/cache-read.sql` (selects only) and
  `src/cache-write.sql`; `src/store.ts` turns them into a reader and a
  writer facet, so the cache logic holds no SQL text.
- **Pages**: home's dump form (cache id + page number, POST) shows raw
  dump rows; `/public/<username>?page=n` shows an HTML list with a page
  form (JSON for `Accept: application/json`). Both say `cache: HIT` or
  `MISS`; every page read also logs `{event: "cache", kind, page,
  outcome}`.
- **Publish later**: `POST /_publish` (cache id in the body) wires a
  public companion to a private-only cache.
- **The webkey page and Cap'n Web**: `/cache/#<cache id>` is a skeleton
  page, the same for every key; its script reads the key from the
  fragment and trades it in-band for a capability over
  [Cap'n Web](https://github.com/cloudflare/capnweb) at `/api`
  (`CacheApi.cache(key)` → a `CacheCap` facet: fill, status, the public
  companion; `src/api.ts`). The worker serves capnweb's browser build
  itself at `/_capnweb.js`.
- **Fill job**: walks every diigo page into a cache in the background —
  DO alarms, a few pages per step, progress stored in the DO
  (`startFill`/`fillStatus`/`stopFill`), driven from the webkey page;
  the public page shows progress. "Invalidate the last N days" there
  drops the cached pages holding anything updated that recently
  (`invalidateRecent`), so the next fill refetches them.
- The `DiigoBridge` entrypoint for other workers (its cache-id source is
  the goal-2 wiring TODO).

## State surface

Ids are **webkeys** ([the pattern](../docs/webkeys.md)): 256-bit bearer
secrets that never ride a query string or path. The webkey page carries
the cache id in the URL fragment (never sent to the server) and passes
it in-band over Cap'n Web; forms carry ids in POST bodies; logs and pages
show only a fingerprint (a prefix of the id's one-way hash).

- **The cache id**: the cache's read capability (bearer; not logged).
  Held by the user — saved to the password manager under a petname they
  choose (the username field; never sent to the server) — and passed
  per POST.
- **The webapp instance id**: the diigo-read capability; lives in each
  cache DO's wiring, arrives from the mint step.
- **The cookie**: lives **only in webapp's DO storage**; crosses the
  wire once, at mint time, as an opaque relay.
- **The public cache**: no capability — `/public/<username>` is public
  by design (a guessable URL; hiding the link doesn't hide the page).

Stored state is meant to be durable: see the breaking-change rule in
[capsite AGENTS.md](../AGENTS.md).

## Deployment concern: rate-limit `/api`

Public pages only read storage, so strangers can't make the cache spend
your diigo session. `/api` is keyed (a cache id is needed), but Cap'n
Web pipelining makes calls cheap to send. Before exposing a deployment:

- rate-limit `/api` (and `/public/*`, which reads storage) — on
  Cloudflare, the
  [Workers rate-limit binding](https://developers.cloudflare.com/workers/runtime-apis/bindings/rate-limit/)
  (keyed by client and/or username); on other workerd hosts, a rate
  limit in the proxy in front;
- run the fill job (goal 1e): public pages show only what's stored.

## RPC use (from another worker, same account)

```jsonc
// wrangler.jsonc of the caller
"services": [{ "binding": "DIIGO", "service": "diigo-cache", "entrypoint": "DiigoBridge" }]
```

```ts
// cache ids ride goal-2's wiring; see the capsite checklist
const ndjson = await env.DIIGO.dumpNdjson(4); // 4 pages of PAGE_SIZE
```

## Test

    pnpm types  # regenerate worker-configuration.d.ts (after wrangler.jsonc changes)
    pnpm lint   # fails if those types are stale; tsc over src/ and tests/
    pnpm test   # vitest

Tests run the real worker and both DOs in the vitest pool, with
`WEBAPP` bound to [a fake webapp](tests/fake-webapp.js) serving
diigo-shaped items (`wrangler.tests.jsonc`).

## Run, deploy

[How to run / connect / deploy is maintained in the capsite checklist](../capsite-checklist.md)
(single source; this README keeps only worker-level facts).

Status: create, dump, publish, and public pages exercised against live
diigo in dev (2026-10-07); not yet deployed.