From 6f409fc83a3e33a20e65ee42e13f0333880e64e0 Mon Sep 17 00:00:00 2001 From: Dan Connolly Date: Wed, 7 Oct 2026 03:30:24 -0500 Subject: [PATCH] diigo-cache: bookmark cache in Durable Objects (goal 1c) The dump worker becomes a page cache: a hit is served from DO SQLite storage with no diigo call; a miss asks the webapp instance in the cache's wiring and etches the page. - two independent caches, one behavior (PagedCache): DiigoCache keeps every item; PublicCache keeps shared items only, dropping private rows before they are stored; served at /public/ (HTML page form, or JSON) - create flow on one port: /_connect relays webapp's mint form, /_mint proxies the mint, /_wire wires a private or public cache; /_publish adds a public cache later - ids are webkeys: kept out of URLs (POST bodies), shown and logged only as a hash fingerprint; saved to the password manager under the holder's petname (never sent to the server) - items keyed on diigo's link_id; username from u_name; unkeyable items and every failure path are logged; each page read logs hit/miss - tests run the real worker + DOs in the vitest pool against a fake webapp (tests/fake-webapp.js) Generated-by: opencode (GLM) Generated-by: Claude Opus 5.5 --- diigo-cache/README.md | 118 ++++-- diigo-cache/package.json | 6 +- diigo-cache/src/cache.ts | 252 ++++++++++++ diigo-cache/src/index.ts | 642 ++++++++++++++++++++++-------- diigo-cache/src/items.ts | 66 +++ diigo-cache/src/pm-save.ts | 57 +++ diigo-cache/src/webkeys.ts | 204 ++++++++++ diigo-cache/tests/caches.test.ts | 122 ++++++ diigo-cache/tests/fake-webapp.js | 72 ++++ diigo-cache/tests/items.test.ts | 72 ++++ diigo-cache/tests/pm-save.test.ts | 62 +++ diigo-cache/tests/webkeys.test.ts | 139 +++++++ diigo-cache/vitest.config.ts | 28 ++ diigo-cache/wrangler.jsonc | 14 +- diigo-cache/wrangler.tests.jsonc | 21 + 15 files changed, 1664 insertions(+), 211 deletions(-) create mode 100644 diigo-cache/src/cache.ts create mode 100644 diigo-cache/src/items.ts create mode 100644 diigo-cache/src/pm-save.ts create mode 100644 diigo-cache/src/webkeys.ts create mode 100644 diigo-cache/tests/caches.test.ts create mode 100644 diigo-cache/tests/fake-webapp.js create mode 100644 diigo-cache/tests/items.test.ts create mode 100644 diigo-cache/tests/pm-save.test.ts create mode 100644 diigo-cache/tests/webkeys.test.ts create mode 100644 diigo-cache/vitest.config.ts create mode 100644 diigo-cache/wrangler.tests.jsonc diff --git a/diigo-cache/README.md b/diigo-cache/README.md index 621f3e0..2d189ca 100644 --- a/diigo-cache/README.md +++ b/diigo-cache/README.md @@ -1,57 +1,101 @@ -# diigo-dump +# diigo-cache -Cloudflare Worker consumer of [the webapp factory](../webapp/README.md): -the site-side of dumping [Diigo bookmarks from the mm-diigo provenance -scripts](https://github.com/dckc/mm-diigo) (`projects/diigo-bak/`) — -session-cookie access, most-recently-updated first. The static API-key -path is dead (both known keys, all documented schemes → 401; see -[the capsite checklist](../capsite-checklist.md), goal 2 preamble). +Cloudflare Worker cache over [the webapp factory](../webapp/README.md): +session-cookie reads of diigo bookmarks, cached in Durable-Object SQLite +storage. Evolved from the diigo-dump worker (provenance: +[mm-diigo `projects/diigo-bak/`](https://github.com/dckc/mm-diigo)); the +static API-key path is dead (see [the capsite checklist](../capsite-checklist.md), +goal 2 preamble). ## What it holds -**Only the diigo flavoring** — everything generic lives in webapp: - -- the mint call: `url` = `https://www.diigo.com/interact_api/`, seed - headers `{user-agent, x-requested-with}` (+ the cookie at mint time), - `msPerCall: 500` -- the dump page: page-number form → `WEBAPP.at(id)` → pipelined - `pathjoin("load_user_items").query({sort, count, page_num}).get()` -- the `DiigoBridge` entrypoint (`open()`, `dumpNdjson()`) for other - workers; `dumpBookmarks` (the generator from mm-diigo's - diigo-dump.mjs) is in `src/diigo.ts` - -GET `/` is inert; dumps happen only on POST submits. +- **The create flow, on one port**: `/_connect` renders webapp's own + mint form (diigo-prefilled, cookie blank) — wrangler multi-config dev + doesn't serve aux workers' HTTP, so `/_mint` relays the form opaquely + into the WEBAPP binding (the cookie crosses; nothing on this side opens + or logs it). Mint → `/_wire` (webapp id pre-filled, mode + public|private) → the created page, where the user saves the ids. +- **Two independent page caches** (`src/cache.ts`), one behavior: a + page hit is served from storage with no diigo call; a miss asks the + webapp instance in the cache's wiring and etches the page. + - `DiigoCache` (binding `CACHE_DO`): every item. Addressed by its + cache id. + - `PublicCache` (binding `PUBLIC_DO`): shared items only — private + rows are dropped *before* etching (`publicOnly`), so its storage + holds nothing private. Same cookie webapp as the private cache. + Public page n is diigo page n, filtered (may show fewer than 24). + `/public/` resolves through the `idFromName("index")` + registry. +- **Pages**: home's dump form (cache id + page number, POST) shows raw + dump rows; `/public/?page=n` shows an HTML list with a page + form (JSON for `Accept: application/json`). Both say `cache: HIT` or + `MISS`; every page read also logs `{event: "cache", kind, page, + outcome}`. +- **Publish later**: `POST /_publish` (cache id in the body) wires a + public companion to a private-only cache. +- The `DiigoBridge` entrypoint for other workers (its cache-id source is + the goal-2 wiring TODO). ## State surface -- The instance id (`DIIGO_WEB_ID`): the one secret (bearer capability; - possession is authority). Not logged. -- The cookie: lives **only in the webapp DO's storage**, crosses the wire - exactly once, at mint time. Nothing on disk holds it. -- `/_connect`: state-aware — pre-mint it's the paste-cookie form; - post-mint it health-checks the instance (count=1 probe) and offers - re-mint only for stale cookies. +Ids are **webkeys** ([the pattern](../docs/webkeys.md)): 256-bit bearer +secrets that never ride URLs — not query strings, not path segments. +Forms carry them in POST bodies; logs and pages show only a fingerprint +(a prefix of the id's one-way hash). + +- **The cache id**: the cache's read capability (bearer; not logged). + Held by the user — saved to the password manager under a petname they + choose (the username field; never sent to the server) — and passed + per POST. +- **The webapp instance id**: the diigo-read capability; lives in each + cache DO's wiring, arrives from the mint step. +- **The cookie**: lives **only in webapp's DO storage**; crosses the + wire once, at mint time, as an opaque relay. +- **The public cache**: no capability — `/public/` is public + by design (a guessable URL; hiding the link doesn't hide the page). + +Stored state is meant to be durable: see the breaking-change rule in +[capsite AGENTS.md](../AGENTS.md). + +## Deployment concern: public pages spend your diigo session + +Anyone who can reach `/public/` can make the cache fetch from +diigo through *your* cookie session: each request may spend up to +`FETCH_BUDGET` (40) diigo fetches on uncached pages (goal 1d), paced at +`MS_PER_CALL`. Cached pages cost nothing. Before exposing a deployment: + +- rate-limit the public routes — on Cloudflare, the + [Workers rate-limit binding](https://developers.cloudflare.com/workers/runtime-apis/bindings/rate-limit/) + (keyed by client and/or username); on other workerd hosts, a rate + limit in the proxy in front; +- run the fill job (goal 1e) so most pages are cached before strangers + arrive. ## RPC use (from another worker, same account) ```jsonc // wrangler.jsonc of the caller -"services": [{ "binding": "DIIGO", "service": "diigo-dump", "entrypoint": "DiigoBridge" }] +"services": [{ "binding": "DIIGO", "service": "diigo-cache", "entrypoint": "DiigoBridge" }] ``` ```ts -const stub = await env.DIIGO.open(); -// promise pipelining: whole chain = one round trip; cookie never leaves the DO -const first = JSON.parse(await stub.pathjoin("load_user_items") - .query({ sort: "updated", count: 24, page_num: 1 }).get()); +// cache ids ride goal-2's wiring; see the capsite checklist +const ndjson = await env.DIIGO.dumpNdjson(4, 24); ``` +## Test + + pnpm test + +Runs the real worker and both DOs in the vitest pool, with `WEBAPP` +bound to [a fake webapp](tests/fake-webapp.js) serving diigo-shaped +items (`wrangler.tests.jsonc`). vitest is borrowed from +`../diigo-bookmarks/node_modules` (see `vitest.config.ts`). + ## Run, deploy -[How to run / connect / deploy is maintained in the capsite checklist] -(../capsite-checklist.md) (single source; this README keeps only -worker-level facts). +[How to run / connect / deploy is maintained in the capsite checklist](../capsite-checklist.md) +(single source; this README keeps only worker-level facts). -Status: dev wiring verified (boots, pages answer); mint → dump through -the DO not yet exercised end-to-end (checklist 1b-i). Earlier flat -(non-DO) build passed manual test: 200, `total: 1674` (2026-10-06). \ No newline at end of file +Status: create, dump, publish, and public pages exercised against live +diigo in dev (2026-10-07); not yet deployed. diff --git a/diigo-cache/package.json b/diigo-cache/package.json index f5ebb8c..193f8aa 100644 --- a/diigo-cache/package.json +++ b/diigo-cache/package.json @@ -1,9 +1,11 @@ { - "name": "diigo-dump", + "name": "diigo-cache", "private": true, + "type": "module", "scripts": { "dev": "wrangler dev -c wrangler.jsonc -c ../webapp/wrangler.jsonc", - "diigo-login": "scripts/diigo-login" + "test": "vitest run", + "typecheck": "tsc -p tsconfig.check.json" }, "devDependencies": { "wrangler": "^4.148.0" diff --git a/diigo-cache/src/cache.ts b/diigo-cache/src/cache.ts new file mode 100644 index 0000000..bc7c2bf --- /dev/null +++ b/diigo-cache/src/cache.ts @@ -0,0 +1,252 @@ +/** + * @file the cache Durable Objects: `DiigoCache` (public+private items) and + * `PublicCache` (public-only items, widely shared). They are independent + * and work the same way ({@link PagedCache}): pages are the unit of + * caching; a hit is answered from `ctx.storage.sqlite` with no diigo call; + * a miss asks the webapp instance in the cache's wiring and etches the + * page. The only difference is what each keeps: the public cache drops + * private rows *before* etching ({@link publicOnly}). + */ + +import { DurableObject } from "cloudflare:workers"; +import type { PagedGet } from "./diigo.js"; +import { publicOnly, toRows } from "./items.js"; +import type { Item, ItemRow } from "./items.js"; + +export type { Item }; + +/** schema migration steps; the sqlite-backed class starts empty, so V1 is + * the initial shape. `page_members` keeps each page's fetch-time item + * order: a page-hit replays that order exactly, independent of later + * re-sorting as items shift ranks. */ +const SCHEMA_V1 = ` +create table if not exists items ( + item_id text primary key, + url text not null, + title text not null, + private integer not null, + json text not null, + updated text not null +); +create table if not exists pages ( + page_num integer primary key, + count integer not null, + total integer, + etched_at text not null +); +create table if not exists page_members ( + page_num integer not null, + pos integer not null, + item_id text not null, + primary key (page_num, pos) +); +`; + +/** one cached page, raw: `items` are the dump rows (JSON strings), ready + * to re-join with newlines */ +export type CachePage = { + total: number; + items: string[]; + /** true when answered from storage (no diigo call) */ + cached: boolean; +}; + +const MS_PER_CALL = 500; + +function sleep(ms: number): Promise { + return new Promise((r) => setTimeout(r, ms)); +} + +/** a cache's wiring: the webapp instance it asks on a miss, plus (private + * cache only) the public companion's id, recorded once published */ +type Wiring = { webappId: string; publicId?: string | undefined }; + +/** the diigo-cache DO's bindings (structural; `at` mirrors webapp's + * WebAppFactory) */ +export type CacheEnv = { + WEBAPP: { + at(id: string): Promise; + }; +}; + +/** + * the shared page-cache behavior. The webapp-instance id in the wiring + * authorizes diigo reads (the cookie rides in that DO's storage); the id + * this DO is addressed by authorizes cache reads. Neither is logged. + * `setup()` persists the wiring (webapp's configure pattern); `page(n)` is + * the read path; subclasses choose what to `keep` of a fetched page. + */ +abstract class PagedCache extends DurableObject { + #step = 0; + #pause?: Promise; + + /** one-shot wiring. Idempotent; different values are refused (mint a + * new cache instead). */ + async setup(init: Wiring): Promise { + const stored = await this.ctx.storage.get("init"); + if (stored) { + if (stored.webappId !== init.webappId || stored.publicId !== init.publicId) { + throw new Error("already wired to different ids; mint a new cache instead"); + } + return; + } + await this.ctx.storage.put("init", init); + } + + /** the stored wiring */ + async wiring(): Promise { + return this.ctx.storage.get("init"); + } + + /** which rows of a fetched page this cache stores */ + protected abstract keep(rows: ItemRow[]): ItemRow[]; + + /** which cache this is, for logs (never the id: that's the bearer key) */ + protected abstract readonly kind: "private" | "public"; + + /** + * read page `pageNum` (`count` per page, diigo's updated-desc order): + * storage first; on miss, fetch through the webapp instance (etching + * page 1 alongside — it carries `total`) and etch. Cache provenance + * rides in the result (`cached`). + */ + async page(pageNum: number, count: number): Promise { + const init = await this.wiring(); + if (!init) throw new Error("not wired: this cache id was never set up"); + this.ctx.storage.sql.exec(SCHEMA_V1); + + // one diigo call path at a time, min spacing MS_PER_CALL (webapp's + // Gate pattern; per activation) + let before; + do { + before = this.#step; + if (this.#pause) await this.#pause; + } while (before !== this.#step); + this.#step += 1; + this.#pause = sleep(MS_PER_CALL); + + const hit = await this.#pageFromStorage(pageNum, count); + if (hit) { + console.log({ event: "cache", kind: this.kind, page: pageNum, outcome: "hit", items: hit.items.length }); + return hit; + } + + const app = await this.env.WEBAPP.at(init.webappId); + const fetchPage = (n: number): Promise<{ total: number; items: Item[] }> => + app + .pathjoin("load_user_items") + .query({ sort: "updated", count, page_num: n }) + .get() + .then((t) => JSON.parse(t)) as Promise<{ total: number; items: Item[] }>; + + // a miss on page n>1 etches page 1 first (it carries `total`) + const first = await fetchPage(1); + const data = pageNum === 1 ? first : await fetchPage(pageNum); + const kept = this.keep(toRows(data.items, console)); + // fetched vs kept: the public cache's drop of private rows shows here + console.log({ event: "cache", kind: this.kind, page: pageNum, outcome: "miss", fetched: data.items.length, kept: kept.length }); + return this.#etch(pageNum, count, data.total, kept); + } + + /** etch a page (one transaction): upsert items, mark the page, keep + * the fetch-time order; returns the cache-page shape */ + #etch(pageNum: number, count: number, total: number, rows: ItemRow[]): CachePage { + this.ctx.storage.transactionSync(() => { + this.ctx.storage.sql.exec("delete from page_members where page_num = ?", pageNum); + let pos = 0; + for (const r of rows) { + pos += 1; + this.ctx.storage.sql.exec( + "insert into items (item_id, url, title, private, json, updated) values (?, ?, ?, ?, ?, ?) on conflict(item_id) do update set url = excluded.url, title = excluded.title, private = excluded.private, json = excluded.json, updated = excluded.updated", + r.itemId, r.url, r.title, r.private ? 1 : 0, r.json, r.updated, + ); + this.ctx.storage.sql.exec( + "insert into page_members (page_num, pos, item_id) values (?, ?, ?)", + pageNum, pos, r.itemId, + ); + } + this.ctx.storage.sql.exec( + "insert into pages (page_num, count, total, etched_at) values (?, ?, ?, ?) on conflict(page_num) do update set count = excluded.count, total = excluded.total, etched_at = excluded.etched_at", + pageNum, count, total, new Date().toISOString(), + ); + }); + return { total, items: rows.map((r) => r.json), cached: false }; + } + + /** storage-backed page read; null = cache miss. Replay order comes + * from `page_members`. */ + async #pageFromStorage(pageNum: number, count: number): Promise { + const page = this.ctx.storage.sql + .exec<{ total: number | null }>( + "select total from pages where page_num = ? and count = ?", + pageNum, count, + ) + .toArray(); + if (page.length === 0) return null; + const rows = this.ctx.storage.sql + .exec<{ json: string }>( + "select items.json from page_members join items on items.item_id = page_members.item_id where page_members.page_num = ? order by page_members.pos", + pageNum, + ) + .toArray(); + return { total: page[0].total ?? -1, items: rows.map((r) => r.json), cached: true }; + } +} + +/** + * the private cache (export name `DiigoCache`): public+private items, + * addressed by its cache id (a webkey, held by the user). + */ +export class DiigoCache extends PagedCache { + protected readonly kind = "private"; + + protected keep(rows: ItemRow[]): ItemRow[] { + return rows; + } + + /** one-way addition of the public companion (private → public + * publishable later; once set, immutable). The webapp id already in + * the wiring is what makes this work without re-minting. */ + async addPublic(publicId: string): Promise { + const stored = await this.ctx.storage.get("init"); + if (!stored) throw new Error("not wired"); + if (stored.publicId) { + if (stored.publicId !== publicId) { + throw new Error("already has a public cache"); + } + return; + } + await this.ctx.storage.put("init", { ...stored, publicId }); + } +} + +/** + * the public cache: shared items only — {@link publicOnly} drops private + * rows before they are stored, so this DO's storage holds nothing + * private. Widely shared: addressed by diigo username in the worker's + * route path (`/public/`). It does hold the webapp id (the same + * cookie session the private cache uses), so it *could* read private + * items; it keeps none. Public page n is diigo page n, filtered: it may + * hold fewer than `count` items. Doubles as the username→instance index + * (the idFromName("index") instance holds the kv mapping and is never + * wired). + */ +export class PublicCache extends PagedCache { + protected readonly kind = "public"; + + protected keep(rows: ItemRow[]): ItemRow[] { + return publicOnly(rows); + } + + /** register/replace the username → this-instance mapping (called on + * the idFromName("index") instance) */ + async register(username: string, publicId: string): Promise { + this.ctx.storage.kv.put(`user:${username}`, publicId); + } + + /** resolve a username to a public-cache instance id (index role); + * undefined = not registered */ + async lookup(username: string): Promise { + return this.ctx.storage.kv.get(`user:${username}`); + } +} diff --git a/diigo-cache/src/index.ts b/diigo-cache/src/index.ts index d4abbde..ea09df1 100644 --- a/diigo-cache/src/index.ts +++ b/diigo-cache/src/index.ts @@ -1,65 +1,55 @@ /** - * @file diigo-dump worker: cookie access to diigo's - * interact_api/load_user_items via a WebApp Durable Object instance from - * the webapp worker. The id is a bearer capability held in DIIGO_WEB_ID; - * the cookie lives only inside that DO's storage. + * @file diigo-cache worker: caches diigo bookmark dumps in Durable-Object + * SQLite storage. `PrivateCache` (public+private items) is addressed by its + * own unique id — minted by the create form alongside the webapp instance + * (the cookie lives in that DO's storage); `PublicCache` (public-only + * items) is widely shared, addressed by the diigo username in the path. + * Dump output stays raw; a repeat page load is answered from storage with + * no diigo call. * - * Connect flow (no askpass, no cookie-on-disk): the home page links to - * /_connect, which proxies webapp's mint form with the diigo prefills - * (cookie field blank) — paste the cookie, mint lands via the service - * binding, and the page shows the DIIGO_WEB_ID to put in .dev.vars. - * (scripts/diigo-login automates the same flow for terminals.) - * @see {@link DiigoBridge} + * The create form's ids reach the browser only via explicit user action + * (clipboard copy / file save — the poor-man's powerbox); a pasted id is + * how the home page's dump POST knows which cache to hit. */ import { WorkerEntrypoint } from "cloudflare:workers"; -import { dumpBookmarks, type PagedGet } from "./diigo.js"; +import { DiigoCache, PublicCache } from "./cache.js"; +import type { Item } from "./cache.js"; +import { usernameOf } from "./items.js"; +import { toWebkey, idFromWebkey, isWebkey, shashKey } from "./webkeys.js"; +import { pmSave } from "./pm-save.js"; -/** the WEBAPP service binding to webapp/src/index.ts:WebAppFactory; typed - * structurally here pending a shared types package */ -type WebAppFactoryShape = { - newWebApp(init: { - url: string; - headers?: Record; - msPerCall?: number; - }): Promise<{ id: string; app: PagedGet }>; - at(id: string): Promise; -}; +// the DO classes must be entrypoint exports (the exports/ binding targets +// the main module) +export { DiigoCache, PublicCache }; +/** worker bindings (structural until wrangler types regenerates) */ type Env = { - DIIGO_WEB_ID?: string; - WEBAPP: WebAppFactoryShape; + WEBAPP: { + fetch(request: Request): Promise; + }; + CACHE_DO: DurableObjectNamespace; + PUBLIC_DO: DurableObjectNamespace; + DIIGO_CACHE_ID?: string; }; const UA = "Mozilla/5.0 (X11; Linux x86_64) Gecko/20100101 Firefox/141.0"; const DIIGO_INTERACT = "https://www.diigo.com/interact_api/"; -/** prefill for webapp's mint form: everything but the cookie */ -const DIIGO_PREFILL = JSON.stringify({ - "user-agent": UA, - "x-requested-with": "XMLHttpRequest", -}); - -/** connect to the configured instance, or give the setup pointer */ -async function instance(env: Env): Promise { - const id = env.DIIGO_WEB_ID; - if (!id) { - throw new Error("no instance: use the connect flow on the home page, then set DIIGO_WEB_ID"); - } - return env.WEBAPP.at(id); -} +/** the diigo export (DiigoBridge lives on; the dumpNdjson cache-id source + * is TODO(2-i): ride the create form's handoff instead of a worker secret) */ export class DiigoBridge extends WorkerEntrypoint { - /** the diigo-attenuated handle: fluent chain over the instance */ - open(): Promise { - return instance(this.env as unknown as Env); - } - - /** bulk dump from the caller's address space */ + /** bulk dump from the caller's address space: `pages * count` bookmarks + * through the cache (cache-hits are storage reads) */ async dumpNdjson(pages: number, count = 24): Promise { - const diigoAPI = await instance(this.env as unknown as Env); + const ns = (this.env as unknown as Env).CACHE_DO; + const cacheId = (this.env as unknown as Env).DIIGO_CACHE_ID; + if (!cacheId) throw new Error("no cache id supplied (caller config error)"); + const cache = ns.get(ns.idFromString(idFromWebkey(cacheId))); const lines: string[] = []; - for await (const item of dumpBookmarks({ diigoAPI, pages, count })) { - lines.push(JSON.stringify(item)); + for (let p = 1; p <= pages; p += 1) { + const page = await cache.page(p, count); + lines.push(...(p === 1 ? page.items.slice(0, count) : page.items)); } return lines.join("\n"); } @@ -69,170 +59,482 @@ export default { async fetch(request: Request, env: Env): Promise { const url = new URL(request.url); - // the connect flow: proxy webapp's mint form (prefilled, cookie blank) - if (url.pathname === "/_connect") { - return env.DIIGO_WEB_ID ? connectedPage(env) : connectPage(); + // public cache: widely shared, username in the path + const pub = url.pathname.match(/^\/public\/([A-Za-z0-9_-]+)$/); + if (pub) return publicPage(request, env, pub[1]); + + // webapp's mint form, relayed onto our port (wrangler multi-config + // dev doesn't mount an aux HTTP server): diigo-cache supplies only + // the diigo prefill (base URL + UA headers) and the form HTML is + // webapp's own — its init params (url, headers, msPerCall) are + // editable there; the POST relays opaquely (a conduit: the cookie + // crosses, no code on this side opens/logs it). + if (url.pathname === "/_connect" && request.method === "GET") { + return mintPage(env); } - // webapp's form POSTs back through us: forward to the binding + + // the mint proxied: form fields (incl. the cookie) relay to + // WEBAPP.newWebApp; the response is the wire form, webapp id filled if (url.pathname === "/_mint" && request.method === "POST") { return proxyMint(request, env); } + + // the wire step: mint sent us back the webapp id (form-encoded from + // the mint form's hidden-continue, or JSON service-style). No cookie + // ever arrives here. + if (url.pathname === "/_wire" && request.method === "POST") { + return wire(request, env); + } + + // publish a cache that was wired private-only: adds the public + // companion (derived from the private cache's stored wiring — the + // webapp id rides along, no re-mint), registers the username read + // from its items, seeds the projection. The cache id is a webkey: it + // rides the POST body, never the path. + if (url.pathname === "/_publish" && request.method === "POST") { + return publishCache(request, env); + } + if (url.pathname !== "/") return new Response("not found", { status: 404 }); - if (request.method !== "POST") return home(null, undefined, Boolean(env.DIIGO_WEB_ID)); - - const form = await request.formData(); - const raw = form.get("page"); - const page = typeof raw === "string" ? Number.parseInt(raw, 10) : Number.NaN; - if (Number.isNaN(page) || page < 1 || page > 10000) { - return home("enter a page number (1 = most recently updated)", undefined, Boolean(env.DIIGO_WEB_ID)); + if (request.method === "POST") { + return dump(request, env); } - try { - const app = await instance(env); - const text = await app - .pathjoin("load_user_items") - .query({ sort: "updated", count: 24, page_num: page }) - .get(); - const data = JSON.parse(text) as { total?: number; items?: Record[] }; - return home( - `total: ${data.total}\nitems: ${data.items?.length ?? 0}\n\n${JSON.stringify(data.items ?? data, null, 2)}`, - undefined, - true, - ); - } catch (err) { - console.error({ event: "dump", page, error: String(err) }); - return home(null, String(err), Boolean(env.DIIGO_WEB_ID)); - } + // GET / is inert (dumps happen on POST submits). The cache id is a + // webkey: it never rides the URL (query strings leak into history/ + // Referer/logs — see docs/webkeys.md); the wired form POST is how + // it travels. + return home(env, { error: "GET is inert: dumps happen on POST submits" }, null); }, }; -/** the connect page: webapp's form with diigo defaults. The mint POST is - * proxied (/_mint) so one port serves the whole flow; the response - * page shows the id and the .dev.vars line to add. */ -async function connectPage(): Promise { - // proxy: render our own form with action="/_mint" so one port serves - // the whole flow. TODO(webapp): one canonical mint form; consider - // serving it from webapp directly. - const body = ` -connect diigo session - -

connect diigo session

-
    -
  1. copy the Cookie header from diigo.com devtools
    2. -
  2. paste below, mint, then add the DIIGO_WEB_ID line it -shows to .dev.vars (deployed: wrangler secret put DIIGO_WEB_ID)
    3. -
  3. restart wrangler, dump from the home page
    4. -
-
- - -

-

-
-`; - return new Response(body, { headers: { "content-type": "text/html;charset=utf-8" } }); -} -async function connectedPage(env: Env): Promise { - let health = ""; +/** the connect page: webapp's own mint form HTML (fetched over the + * service binding, action rewritten to our relay), carried with diigo's + * prefill only — the editable init params are webapp's, diigo-cache + * authors no form fields of its own */ +async function mintPage(env: Env): Promise { + const prefill = + `?render=1&title=${encodeURIComponent("diigo cache: mint the session")}` + + `&url=${encodeURIComponent(DIIGO_INTERACT)}` + + `&headers=${encodeURIComponent(JSON.stringify({ "user-agent": UA, "x-requested-with": "XMLHttpRequest" }))}` + + `&msPerCall=500`; + let html: string; try { - const app = await instance(env); - // a cheap call: one item, exercises the cookie + rate gate - const text = await app - .pathjoin("load_user_items") - .query({ sort: "updated", count: 1, page_num: 1 }) - .get(); - const data = JSON.parse(text) as { total?: number }; - health = `

instance is live: diigo reports ${esc(String(data.total ?? "?"))} bookmarks.

`; + // a plain fetch() from inside the worker goes to the internet, not + // to the service binding — binding calls need a Request handed to + // the stub (workerd's entrypoint-fetch protocol) + const res = await env.WEBAPP.fetch( + new Request(`https://form${prefill}`, { redirect: "manual" }), + ); + html = await res.text(); + html = html.replace('action="ACTION"', 'action="/_mint"'); } catch (err) { - console.error({ event: "connect-check", error: String(err) }); - health = `

instance is configured but the last check failed:

${esc(String(err))}

If diigo expired the cookie, re-mint below.

`; + console.error({ event: "mint-page", error: String(err) }); + return home(env, { error: "mint form unreachable (webapp worker down?)" }, null); } - const body = ` -diigo session - -

diigo session

-

connected: id ${esc(`${env.DIIGO_WEB_ID!.slice(0, 6)}…${env.DIIGO_WEB_ID!.slice(-4)}`)} (from .dev.vars).

-${health} -

replace the session

-

Only needed if the cookie is stale (diigo will say 401/403). Minting a -new instance invalidates nothing, but the id rotates and .dev.vars needs -the new DIIGO_WEB_ID.

-
- - -

-

-
-`; - return new Response(body, { headers: { "content-type": "text/html;charset=utf-8" } }); + return new Response(html, { headers: { "content-type": "text/html;charset=utf-8" } }); } -/** proxy the mint: merge the pasted cookie into the headers JSON, mint via - * the service binding, show the id (plus the .dev.vars line to copy) */ +/** the mint relay: the raw form body passes through untouched into the + * WEBAPP entrypoint's HTTP surface (webapp parses its own fields; the + * cookie crosses browser→here→binding→webapp's DO and no code on this + * side opens it) — a true conduit */ async function proxyMint(request: Request, env: Env): Promise { - const form = await request.formData(); - const cookie = String(form.get("cookie") || ""); - if (!cookie) { - return new Response("no cookie supplied", { status: 400 }); + try { + const res = await env.WEBAPP.fetch( + new Request("https://mint/", { + method: "POST", + headers: request.headers, + body: await request.arrayBuffer(), + }), + ); + if (!res.ok) { + const detail = (await res.text()).slice(0, 200); + return home(env, { error: `mint said ${res.status}: ${detail}` }, null); + } + // webapp answers {id} (+ x-webapp-id); the id rides the header, the + // wire page is ours + const id = res.headers.get("x-webapp-id"); + if (!id) return home(env, { error: "mint produced no id" }, null); + return await wirePage(id); + } catch (err) { + console.error({ event: "mint", error: String(err) }); + return home(env, { error: `mint failed: ${String(err)}` }, null); } - const headers = { - "user-agent": UA, - "x-requested-with": "XMLHttpRequest", - cookie, - }; - const { id } = await env.WEBAPP.newWebApp({ - url: DIIGO_INTERACT, - headers, - msPerCall: 500, - }); - return idPage(id); } -/** the id shown on the success page: a narrow text field (scrollable but - * too small to show most of the value) with a copy button for the full - * version. navigator.clipboard needs the page served from localhost/https - * — true in dev and at the deployed URL */ -function idPage(id: string): Response { +/** the credential-save helper, once per page (see ./pm-save.ts) */ +const PM_SAVE_JS = pmSave.toString(); + +/** the suggested petname gets the holder's local hh:mm (the worker only + * knows UTC), so two saves the same day are told apart at a glance */ +const PETNAME_TIME_JS = `{ + const p = document.getElementById("petname"); + const d = new Date(); + p.value += " " + String(d.getHours()).padStart(2, "0") + ":" + String(d.getMinutes()).padStart(2, "0"); +}`; + +/** the onclick that hands pmSave its browser powers */ +function pmSaveClick(value: string): string { + return `pmSave(this, ${JSON.stringify(value)}, { petname: document.getElementById("petname"), credentials: navigator.credentials, PasswordCredential: window.PasswordCredential, log: console })`; +} + +/** the wire step, with the freshly-minted webapp id pre-filled */ +async function wirePage(webappId: string): Promise { + const fingerprint = await nick(webappId); const body = ` -minted +diigo cache: step 2 — wire the cache -

minted

-

id: -

-

add to .dev.vars:

-
DIIGO_WEB_ID=
-

…click copy, then paste the full value after it. back to diigo dump (restart wrangler first).

+

step 2: wire the cache

+

minted: webapp id, fingerprint ${esc(fingerprint)} +

+

+(private to you: the password-manager username; never sent to the server)

+ +

+

choose the cache mode, then wire — the next page shows the +cache id (copy that too), after which you land back on home:

+
+ +

+
+

change your mind? mint again · home

+ `; return new Response(body, { headers: { "content-type": "text/html;charset=utf-8", - // for scripts: diigo-login reads this instead of scraping HTML - "x-diigo-web-id": id, + "x-webapp-id": webappId, }, }); } -function home(detail: string | null, error?: string, connected?: boolean): Response { - const form = `
`; - const result = error - ? `

status: error

${escapeHtml(error)}
` - : detail - ? `
${escapeHtml(detail)}
` +/** the wire step: webapp-instance id + the public/private choice. The + * cache is minted, wired to {webappId, publicId}, and page 1 is etched; + * in public mode the diigo username is read from the fetched items + * (self-authenticating: the cookie's account, never form input) and the + * public cache is wired to the same webapp id and registered. The two + * caches are independent from then on: each fills itself on a miss. */ +async function wire(request: Request, env: Env): Promise { + const ct = request.headers.get("content-type") ?? ""; + let webappId = ""; + let mode = "private"; + let wantsJson = false; + try { + if (ct.includes("application/json")) { + const body = (await request.json()) as { webappId?: string; mode?: string }; + webappId = String(body.webappId ?? ""); + mode = String(body.mode ?? "private"); + wantsJson = true; + } else { + const form = await request.formData(); + webappId = String(form.get("webappId") || ""); + mode = String(form.get("mode") || "private"); + // a browser form POST stays on the html path; only explicit JSON + // (service callers) gets the bare-JSON response + wantsJson = false; + } + } catch (err) { + return bad(`bad input: ${String(err)}`); + } + if (!/^[A-Za-z0-9-]+$/.test(webappId)) return bad("no webapp id supplied"); + + const publicId = mode === "public" ? toWebkey(env.PUBLIC_DO.newUniqueId()) : undefined; + const cacheId = toWebkey(env.CACHE_DO.newUniqueId()); + const cacheStub = env.CACHE_DO.get(env.CACHE_DO.idFromString(idFromWebkey(cacheId))); + await cacheStub.setup({ webappId, publicId }); + const first = await cacheStub.page(1, 24); + + if (publicId) { + const firstItem = first.items[0] ? (JSON.parse(first.items[0]) as Item) : undefined; + const username = usernameOf(firstItem); + if (!username) { + // diagnostic context without item contents (privacy: items may + // carry URL/titles; the count + cached flag are safe) + console.error({ + event: "wire", + stage: "username-derive", + // field names, not values: says which shape we got + itemFields: firstItem ? Object.keys(firstItem) : null, + itemsOnPage1: first.items.length, + total: first.total, + cached: first.cached, + }); + return bad("could not read your diigo username (u_name) from the fetched items"); + } + const index = env.PUBLIC_DO.get(env.PUBLIC_DO.idFromName("index")); + const pubStub = env.PUBLIC_DO.get(env.PUBLIC_DO.idFromString(idFromWebkey(publicId))); + await pubStub.setup({ webappId }); + await index.register(username, publicId); + if (wantsJson) return Response.json({ cacheId, publicId, username }); + return wiredPage(cacheId, `public cache live at /public/${esc(username)}`); + } + if (wantsJson) { + return Response.json({ + cacheId, + publishLater: { method: "POST", url: `${new URL(request.url).origin}/_publish`, body: { cacheId } }, + }); + } + return wiredPage( + cacheId, + `private-only for now — to add the public cache later: +
+ +
+(or POST /_publish with cacheId in the body)`, + ); +} + +/** the created-cache page: ids + save/copy buttons + the pointer home */ +async function wiredPage(cacheId: string, note: string): Promise { + const fingerprint = await nick(cacheId); + const body = ` +diigo cache: created + +

cache created

+

cache id, fingerprint ${esc(fingerprint)} +

+

+(private to you: the password-manager username; never sent to the server)

+ +

+

${note}

+

the cache id is the only way back to this cache — saved (above) it lives +with the rest of your credentials. it is a webkey: it never travels +in URLs (query strings leak to history/Referer/logs); the dump POST carries +it in its body instead. the webapp id rides inside the cache's wiring; +hold it separately only if you want another cache off the same session.

+← home (paste the cache id into the dump form) + +`; + return new Response(body, { headers: { "content-type": "text/html;charset=utf-8" } }); +} + +/** publish, later: the private cache's wiring already holds the webapp + * id, so the public companion needs no re-mint; the username comes from + * the items already in the cache */ +async function publishCache(request: Request, env: Env): Promise { + let cacheId = ""; + try { + const ct = request.headers.get("content-type") ?? ""; + if (ct.includes("application/json")) { + const body = (await request.json()) as { cacheId?: string }; + cacheId = String(body.cacheId ?? ""); + } else { + const form = await request.formData(); + cacheId = String(form.get("cacheId") || ""); + } + } catch (err) { + return bad(`bad input: ${String(err)}`); + } + if (!isWebkey(cacheId) && !/^[0-9a-fA-F]{64}$/.test(cacheId)) { + return bad("no cache id supplied (POST body: cacheId)"); + } + try { + const ns = env.CACHE_DO; + const cache = ns.get(ns.idFromString(idFromWebkey(cacheId))); + const wiring = await cache.wiring(); + if (!wiring) { + return Response.json({ error: "unknown cache id" }, { status: 404 }); + } + if (wiring.publicId) { + return Response.json({ cacheId, publicId: wiring.publicId, status: "already published" }); + } + // seed the projection *now*, while the cache is still private-only: + // only rows already cached and marked shared can enter the public + // side; the username rides the same rows + const firstPage = await cache.page(1, 24); + const firstItem = firstPage.items[0] ? (JSON.parse(firstPage.items[0]) as Item) : undefined; + const username = usernameOf(firstItem); + if (!username) { + console.error({ + event: "publish", + stage: "username-derive", + // field names, not values: says which shape we got + itemFields: firstItem ? Object.keys(firstItem) : null, + itemsOnPage1: firstPage.items.length, + total: firstPage.total, + cached: firstPage.cached, + }); + return Response.json( + { error: "could not read your diigo username (u_name) from the cached items" }, + { status: 400 }, + ); + } + const publicId = toWebkey(env.PUBLIC_DO.newUniqueId()); + const pubStub = env.PUBLIC_DO.get(env.PUBLIC_DO.idFromString(idFromWebkey(publicId))); + await pubStub.setup({ webappId: wiring.webappId }); + await cache.addPublic(publicId); + const index = env.PUBLIC_DO.get(env.PUBLIC_DO.idFromName("index")); + await index.register(username, publicId); + return Response.json({ cacheId, publicId, username, public: `/public/${username}` }); + } catch (err) { + // no cacheId in the log: it is the bearer secret + console.error({ event: "publish", error: String(err) }); + return Response.json({ error: `publish failed: ${String(err)}` }, { status: 400 }); + } +} + +function bad(message: string): Response { + return Response.json({ error: message }, { status: 400 }); +} + +/** the private dump POST: cache id from the (client-wired) form value */ +async function dump(request: Request, env: Env): Promise { + const form = await request.formData(); + const cacheId = String(form.get("cacheId") || ""); + const raw = form.get("page"); + const pageN = typeof raw === "string" ? Number.parseInt(raw, 10) : Number.NaN; + if (!isWebkey(cacheId) && !/^[0-9a-fA-F]{64}$/.test(cacheId)) { + return home(env, { error: "wire a cache id first (create flow, then copy/paste)" }, null); + } + if (Number.isNaN(pageN) || pageN < 1 || pageN > 10000) { + return home(env, { error: "enter a page number (1 = most recently updated)" }, null); + } + try { + const ns = env.CACHE_DO; + const cache = ns.get(ns.idFromString(idFromWebkey(cacheId))); + const page = await cache.page(pageN, 24); + const data = `total: ${page.total}\nitems: ${page.items.length}\ncache: ${page.cached ? "HIT (from storage)" : "MISS (etched now)"}` + + `\n\n${page.items.join("\n")}`; + return home(env, null, data, cacheId); + } catch (err) { + console.error({ event: "dump", page: pageN, error: String(err) }); + return home(env, null, String(err), cacheId); + } +} + +const PUBLIC_PER_PAGE = 24; + +/** the public page for a username: an HTML page with a page-number form + * (GET — nothing secret here, so `?page=` in the URL is fine, unlike the + * webkey routes), or the same page as JSON for `Accept: application/json` + * callers */ +async function publicPage(request: Request, env: Env, username: string): Promise { + const index = env.PUBLIC_DO.get(env.PUBLIC_DO.idFromName("index")); + const publicId = await index.lookup(username); + if (!publicId) return new Response("no such public cache", { status: 404 }); + const ns = env.PUBLIC_DO; + const cache = ns.get(ns.idFromString(idFromWebkey(publicId))); + + const raw = new URL(request.url).searchParams.get("page"); + const pageN = raw === null ? 1 : Number.parseInt(raw, 10); + const valid = !Number.isNaN(pageN) && pageN >= 1 && pageN <= 10000; + let page; + try { + // a hit is storage-only; a miss asks the webapp (private rows are + // dropped before etching) + page = await cache.page(valid ? pageN : 1, PUBLIC_PER_PAGE); + } catch (err) { + console.error({ event: "public-page", page: pageN, error: String(err) }); + return new Response(`could not load page ${valid ? pageN : 1}: ${String(err)}`, { status: 502 }); + } + // total and pages count all of diigo's items: public page n is diigo + // page n with the private rows dropped + const pages = Math.max(1, Math.ceil(page.total / PUBLIC_PER_PAGE)); + const items = page.items.map((j) => JSON.parse(j) as Item); + + const accept = request.headers.get("accept") ?? ""; + if (accept.includes("application/json") && !accept.includes("text/html")) { + return Response.json({ username, page: pageN, pages, total: page.total, items }); + } + + let notice = ""; + if (!valid || pageN > pages) { + console.log({ event: "notice", page: "public", text: "page out of range", pages }); + notice = `

no page ${esc(String(raw))}: pages run 1–${pages}

`; + } + const here = `/public/${encodeURIComponent(username)}`; + const nav = (n: number, label: string) => + n >= 1 && n <= pages ? `${label}` : `${label}`; + const list = notice + ? "" + : `
    +${items + .map((it) => { + const url = typeof it.url === "string" ? it.url : ""; + const title = typeof it.title === "string" && it.title ? it.title : url; + const desc = typeof it.description === "string" && it.description ? `
    ${esc(it.description)}` : ""; + const href = /^https?:\/\//i.test(url) ? url : "#"; + return `
  1. ${esc(title)}${desc}
    2. `; + }) + .join("\n")} +
`; + const body = ` +${esc(username)}'s shared bookmarks + +

${esc(username)}'s shared bookmarks

+

${pages} page${pages === 1 ? "" : "s"} (diigo's, ${PUBLIC_PER_PAGE} bookmarks each; private ones are left out, so a page may show fewer).

+

cache: ${page.cached ? "HIT (from storage)" : "MISS (fetched from diigo now)"} — page ${valid ? pageN : 1}, ${items.length} shared

+
+

${nav(pageN - 1, "← prev")} + + +${nav(pageN + 1, "next →")}

+
+${notice}${list} +`; + return new Response(body, { headers: { "content-type": "text/html;charset=utf-8" } }); +} + +/** home page: the connect link (webapp's cookie form, diigo-prefilled) + + * the wire step for an already-minted id + the dump form. `home` renders + * the error path too, so it takes env for the connect link. Notices (the + * inert-GET explainer, probe failures) log at info — exceptional errors + * log at error in their own handlers. */ +async function home(env: Env, opts: { error?: string } | null, data: string | null, cacheId?: string): Promise { + if (opts?.error) { + console.log({ event: "notice", page: "home", text: opts.error }); + } + const cacheState = cacheId + ? `

cache id, fingerprint ${esc(await nick(cacheId))}

` + : `

no cache id wired yet — use the create flow below (or paste one).

`; + const dumpForm = `
+${cacheId ? `` : `

+

`} +

+
`; + const result = data + ? `
${escapeHtml(data)}
` + : opts && opts.error + ? `

${escapeHtml(opts.error)}

` : ""; return new Response( ` -diigo dump +diigo cache -

diigo dump

-

${connected ? "diigo session status/replace…" : "connect diigo session…"}

-${form} +

diigo cache

+${cacheState} +

create (two steps)

+
    +
  1. mint the diigo session at webapp — webapp's +own mint form (diigo-prefilled); paste the Cookie header there. The +cookie transits this worker only as an opaque relay into webapp's DO; +nobody here opens it. Mint lands you on the wire step, id pre-filled.
    2. +
  2. wire the id into a cache (mode: public|private):
    3. +
+
+

+ +

+

+
+

the ids this mints are for you: copy them somewhere you control (password manager)

+${dumpForm} ${result} `, { headers: { "content-type": "text/html;charset=utf-8" } }, ); } +/** a short display name for a bearer id: a prefix of its swissHash + * (identity without authority), so labels and pages never show any + * glyphs of the key itself */ +async function nick(id: string): Promise { + return (await shashKey(id)).slice(0, 8); +} + function esc(s: string): string { return s .replaceAll("&", "&") diff --git a/diigo-cache/src/items.ts b/diigo-cache/src/items.ts new file mode 100644 index 0000000..c78cdcc --- /dev/null +++ b/diigo-cache/src/items.ts @@ -0,0 +1,66 @@ +/** + * @file diigo interact_api items: the field-reading helpers (pure, so + * tests can feed them rows shaped like diigo's). + */ + +/** one diigo item exactly as the interact_api returned it */ +export type Item = Record; + +/** the fields stored per item (the raw row rides in `json`) */ +export type ItemRow = { + itemId: string; + url: string; + title: string; + private: boolean; + /** the raw dump row, JSON-encoded */ + json: string; + updated: string; +}; + +/** diigo's id for a bookmark (`link_id`), as text; "" when absent */ +function linkIdOf(it: Item): string { + return typeof it.link_id === "number" || (typeof it.link_id === "string" && it.link_id !== "") + ? String(it.link_id) + : ""; +} + +/** + * normalize one page of interact_api items into storable rows. Items + * that can't be stored (no url, no link_id) are logged and skipped — + * never merged under a made-up key. Logs name the fields present, not + * their values (items may be private). + */ +export function toRows(items: Item[], log: Pick): ItemRow[] { + const rows: ItemRow[] = []; + for (const it of items) { + const itemId = linkIdOf(it); + const url = typeof it.url === "string" ? it.url : ""; + if (!itemId || !url) { + log.warn({ event: "item-skip", reason: !itemId ? "no link_id" : "no url", fields: Object.keys(it) }); + continue; + } + rows.push({ + itemId, + url, + title: String(it.title ?? url), + private: it.private !== undefined ? Boolean(it.private) : String(it.shared) !== "yes", + json: JSON.stringify(it), + updated: String(it.updated_at ?? it.created_at ?? ""), + }); + } + return rows; +} + +/** the diigo username an item belongs to (`u_name`; not `first_by`, + * who may be whoever bookmarked the URL first), or "" when the item + * carries none in the expected shape */ +export function usernameOf(item: Item | undefined): string { + const username = typeof item?.u_name === "string" ? item.u_name : ""; + return /^[A-Za-z0-9_-]{1,64}$/.test(username) ? username : ""; +} + +/** the public cache's filter: shared rows only. Applied before etching, + * so private rows never reach public storage. */ +export function publicOnly(rows: ItemRow[]): ItemRow[] { + return rows.filter((r) => !r.private); +} diff --git a/diigo-cache/src/pm-save.ts b/diigo-cache/src/pm-save.ts new file mode 100644 index 0000000..6d71131 --- /dev/null +++ b/diigo-cache/src/pm-save.ts @@ -0,0 +1,57 @@ +/** + * @file the password-manager save — ids reach the browser's storage only by + * this gesture-gated path (or copy), never localStorage/IndexedDB, which + * are ambient and XSS-reachable. It runs in the *browser*: pages embed it + * via `pmSave.toString()`, so it must be self-contained (no imports, no + * closure, no inner named functions — wrangler's keepNames would wrap + * those in an `__name` helper the browser doesn't have). Its powers + * (the petname field, the credential store, the console) are passed in, + * so tests can stand in for the browser. + */ + +/** the browser powers pmSave uses */ +export type PmPowers = { + petname: { value: string }; + credentials?: { store(credential: unknown): Promise }; + PasswordCredential?: new (init: { id: string; password: string }) => unknown; + log: Pick; +}; + +/** the button pmSave reports on */ +export type PmButton = { textContent: string | null; disabled: boolean }; + +/** + * gesture-gated save of `value` under the holder's petname (the PM + * username). + * + * @see https://developer.mozilla.org/en-US/docs/Web/API/CredentialsContainer/store + */ +export function pmSave(btn: PmButton, value: string, powers: PmPowers): Promise { + const label = powers.petname.value.trim(); + if (!label) { + btn.textContent = "give it a petname first"; + return Promise.resolve(); + } + btn.disabled = true; + if (!powers.PasswordCredential || !powers.credentials) { + powers.log.warn({ event: "pm-save", outcome: "unsupported", detail: "no PasswordCredential / navigator.credentials" }); + btn.textContent = "browser has no credential store — copy instead"; + btn.disabled = false; + return Promise.resolve(); + } + return powers.credentials + .store(new powers.PasswordCredential({ id: label, password: value })) + .then(() => { + // store() resolves with undefined per spec, whether the holder + // accepted the browser's prompt or not (or no prompt showed): + // we can only say it was offered — the PM's own UI is the truth + powers.log.info({ event: "pm-save", outcome: "offered", label }); + btn.textContent = "offered to the password manager ✓ (see its prompt)"; + btn.disabled = false; + }) + .catch((err) => { + powers.log.error({ event: "pm-save", error: String(err) }); + btn.textContent = "save failed — copy instead"; + btn.disabled = false; + }); +} diff --git a/diigo-cache/src/webkeys.ts b/diigo-cache/src/webkeys.ts new file mode 100644 index 0000000..1df46ba --- /dev/null +++ b/diigo-cache/src/webkeys.ts @@ -0,0 +1,204 @@ +/** + * @file webkeys: URL-borne bearer permissions, per Tyler Close's + * ["web-key: Mashing with permission"](https://waterken.sourceforge.net/web-key/) + * (IEEE W2SP 2008); the design abstract lives in + * [docs/webkeys.md](../../docs/webkeys.md), with the Waterken precursor + * code in `kpreid/e-on-java` as provenance (YURL32 = `org.waterken.uri. + * Base32` / `BigIntegerSugar.toYURL32`; bearer invariant = `EARL.java`; + * identity-without-authority = `SwissTable`'s swissHash). + * + * A webkey separates secret from address: + * - the **secret bits**: supplied directly by the caller as a Worker + * unique ID (e.g. `DurableObjectId` from `ns.newUniqueId()`), or raw + * bytes (`Uint8Array`). Per [OCap discipline] + * (.opencode/skills/ocap-discipline/SKILL.md), functions here are pure: + * no ambient RNG calls, no callback indirection. The caller holds the + * minting authority at the application boundary and supplies the result. + * - the **encoding**: YURL32 — lowercase `a–z` + `2–7`, 5 bits/char, + * URL-safe by construction (no `-`, no `_`, no case to misread). + * + * Cloudflare Worker `DurableObjectId`s are 256 bits (32 bytes), rendered + * as 64 hex characters by `id.toString()`. {@link toWebkey} encodes that + * ID into 52 YURL32 glyphs; {@link idFromWebkey} decodes it back to the + * 64-hex string expected by `ns.idFromString()`. + * + * {@link shashKey} gives the identity-without-authority twin (the + * swissHash discipline) for log-correlation: one-way, on SHA-256. + */ + +/** the YURL32 alphabet: `a–z` then `2–7` (32 glyphs, 5 bits each) */ +const YURL32_ALPHABET = "abcdefghijklmnopqrstuvwxyz234567" as const; + +/** encode bytes as YURL32 (`a–z2–7`, 5 bits/char; no padding, the tail + * byte's leftover bits shift-left, as `org.waterken.uri.Base32.encode` + * does) */ +export function yurl32(bytes: Uint8Array): string { + let out = ""; + let buffer = 0; + let bits = 0; + for (const byte of bytes) { + buffer = (buffer << 8) | byte; + bits += 8; + while (bits >= 5) { + out += YURL32_ALPHABET[(buffer >>> (bits - 5)) & 31]; + bits -= 5; + } + } + if (bits > 0) { + out += YURL32_ALPHABET[(buffer << (5 - bits)) & 31]; + } + return out; +} + +/** decode YURL32 back to bytes (inverse of {@link yurl32}); throws on + * any character outside the alphabet — a webkey is either a webkey or + * it is not */ +export function unyurl32(text: string): Uint8Array { + let buffer = 0; + let bits = 0; + const out: number[] = []; + for (const ch of text) { + const v = YURL32_ALPHABET.indexOf(ch); + if (v < 0) throw new Error(`not a webkey: ${JSON.stringify(ch)} is not YURL32`); + buffer = (buffer << 5) | v; + bits += 5; + if (bits >= 8) { + out.push((buffer >>> (bits - 8)) & 0xff); + bits -= 8; + } + } + // leftover bits (<8) are the encoder's zero-padding: validate, don't + // carry them + if (bits > 0 && (buffer & ((1 << bits) - 1)) !== 0) { + throw new Error("not a webkey: non-zero tail bits"); + } + return new Uint8Array(out); +} + +/** parse a hexadecimal string to raw bytes */ +export function hexToBytes(hex: string): Uint8Array { + const trimmed = hex.trim(); + if (trimmed.length % 2 !== 0) throw new Error("invalid hex length"); + const bytes = new Uint8Array(trimmed.length / 2); + for (let i = 0; i < bytes.length; i += 1) { + const val = Number.parseInt(trimmed.slice(i * 2, i * 2 + 2), 16); + if (Number.isNaN(val)) throw new Error(`invalid hex byte at offset ${i * 2}`); + bytes[i] = val; + } + return bytes; +} + +/** encode raw bytes to a lowercase hexadecimal string */ +export function bytesToHex(bytes: Uint8Array): string { + let hex = ""; + for (const b of bytes) { + hex += b.toString(16).padStart(2, "0"); + } + return hex; +} + +export const BITS_PER_GLYPH = 5; + +/** chars needed for n bits in YURL32 */ +export function glyphsFor(bits: number): number { + return Math.ceil(bits / BITS_PER_GLYPH); +} + +/** YURL32 glyphs → bytes recoverable via {@link unyurl32} */ +export function bytesInGlyphs(glyphs: number): number { + return Math.floor((glyphs * BITS_PER_GLYPH) / 8); +} + +/** + * Structural type for a Cloudflare Worker unique ID (such as `DurableObjectId`), + * matching `id.toString()` which returns the 64-hex string. + */ +export interface DurableObjectIdLike { + toString(): string; + equals?(other: unknown): boolean; + readonly name?: string; +} + +/** A Worker unique ID input: a `DurableObjectId`, 64-char hex string, or raw bytes. */ +export type WorkerUniqueId = DurableObjectIdLike | Uint8Array; + +/** + * Convert a Worker unique ID (or raw bytes) to a YURL32 webkey string. + * + * @param id A `DurableObjectId` (from `ns.newUniqueId()`), a 64-char hex ID, + * or raw bytes (`Uint8Array`). + * @returns A YURL32 webkey string (52 glyphs for 256-bit `DurableObjectId`, + * 26 glyphs for 128-bit raw bytes). + */ +export function toWebkey(id: WorkerUniqueId | string): string { + if (id instanceof Uint8Array) { + return yurl32(id); + } + const str = typeof id === "string" ? id.trim() : id.toString(); + if (/^[0-9a-fA-F]{64}$/.test(str)) { + return yurl32(hexToBytes(str)); + } + if (isWebkey(str)) { + return str; + } + throw new Error("cannot convert ID to webkey: unrecognized format"); +} + +/** + * Convert a YURL32 webkey string back to the 64-character hex ID string + * expected by `DurableObjectNamespace.idFromString(hex)`. + * + * If the input is already a 64-character hex ID, it is validated and returned + * in lowercase for backwards compatibility. + */ +export function idFromWebkey(webkey: string): string { + const trimmed = webkey.trim(); + if (/^[0-9a-fA-F]{64}$/.test(trimmed)) { + return trimmed.toLowerCase(); + } + const bytes = unyurl32(trimmed); + if (bytes.length !== 32) { + throw new Error( + `expected 32 bytes (256 bits) for Worker unique ID, got ${bytes.length}`, + ); + } + return bytesToHex(bytes); +} + +/** + * Is `text` a well-formed webkey of the expected size? + * + * When `bits` is omitted, accepts standard 256-bit Worker unique ID webkeys + * (52 glyphs), 128-bit webkeys (26 glyphs), or 64-bit webkeys (13 glyphs). + */ +export function isWebkey(text: string, bits?: number): boolean { + if (bits !== undefined) { + if (text.length !== glyphsFor(bits)) return false; + try { + return unyurl32(text).length === Math.floor(bits / 8); + } catch { + return false; + } + } + if (text.length === 52 || text.length === 26 || text.length === 13) { + try { + const b = unyurl32(text); + return b.length === 32 || b.length === 16 || b.length === 8; + } catch { + return false; + } + } + return false; +} + +/** + * The identity-without-authority twin (the swissHash discipline): + * SHA-256 hash of the webkey, YURL32-encoded (52 glyphs). + * + * Safe for logs/metrics/joins; never accepted as a credential. + */ +export async function shashKey(key: string | WorkerUniqueId): Promise { + const text = typeof key === "string" ? key : toWebkey(key); + const digest = await crypto.subtle.digest("SHA-256", new TextEncoder().encode(text)); + return yurl32(new Uint8Array(digest)); +} diff --git a/diigo-cache/tests/caches.test.ts b/diigo-cache/tests/caches.test.ts new file mode 100644 index 0000000..c65eabf --- /dev/null +++ b/diigo-cache/tests/caches.test.ts @@ -0,0 +1,122 @@ +import { env } from "cloudflare:workers"; +import { runInDurableObject } from "cloudflare:test"; +import { describe, expect, it, vi } from "vitest"; +import worker from "../src/index.js"; +import { toWebkey } from "../src/webkeys.js"; + +// matches tests/fake-webapp.js: 30 items, every third private +const WEBAPP_ID = "fake-webapp-id"; + +type TestEnv = { + CACHE_DO: DurableObjectNamespace; + PUBLIC_DO: DurableObjectNamespace; + WEBAPP: { calls(): Promise }; +}; +const testEnv = env as unknown as TestEnv; + +const parse = (items: string[]) => items.map((j) => JSON.parse(j) as Record); + +describe("PublicCache", () => { + it("on a miss asks the webapp and keeps only shared items; a hit makes no call", async () => { + const stub = testEnv.PUBLIC_DO.get(testEnv.PUBLIC_DO.newUniqueId()); + await stub.setup({ webappId: WEBAPP_ID }); + + const before = await testEnv.WEBAPP.calls(); + const miss = await stub.page(1, 24); + expect(miss.cached).toBe(false); + expect(miss.total).toBe(30); + // diigo page 1 = items 1..24, of which 8 are private + expect(parse(miss.items).map((i) => i.private)).toEqual(Array(16).fill(false)); + const afterMiss = await testEnv.WEBAPP.calls(); + expect(afterMiss).toBeGreaterThan(before); + + const hit = await stub.page(1, 24); + expect(hit.cached).toBe(true); + expect(hit.items).toEqual(miss.items); + expect(await testEnv.WEBAPP.calls()).toBe(afterMiss); + }); + + it("logs each page read as a hit or a miss (fetched vs kept), without the cache id", async () => { + const spy = vi.spyOn(console, "log"); + try { + const id = testEnv.PUBLIC_DO.newUniqueId(); + const stub = testEnv.PUBLIC_DO.get(id); + await stub.setup({ webappId: WEBAPP_ID }); + await stub.page(1, 24); + await stub.page(1, 24); + const cacheLogs = spy.mock.calls.map((c) => c[0]).filter((e) => e?.event === "cache"); + expect(cacheLogs).toEqual([ + { event: "cache", kind: "public", page: 1, outcome: "miss", fetched: 24, kept: 16 }, + { event: "cache", kind: "public", page: 1, outcome: "hit", items: 16 }, + ]); + expect(JSON.stringify(spy.mock.calls)).not.toContain(id.toString()); + } finally { + spy.mockRestore(); + } + }); + + it("stores no private rows at all", async () => { + const stub = testEnv.PUBLIC_DO.get(testEnv.PUBLIC_DO.newUniqueId()); + await stub.setup({ webappId: WEBAPP_ID }); + await stub.page(1, 24); + await stub.page(2, 24); + await runInDurableObject(stub, (_obj, state) => { + const rows = state.storage.sql.exec<{ n: number }>("select count(*) as n from items where private = 1").toArray(); + expect(rows[0].n).toBe(0); + }); + }); +}); + +describe("DiigoCache", () => { + it("keeps every item of a page, each under its own link_id", async () => { + const stub = testEnv.CACHE_DO.get(testEnv.CACHE_DO.newUniqueId()); + await stub.setup({ webappId: WEBAPP_ID }); + const page = await stub.page(1, 24); + const ids = parse(page.items).map((i) => i.link_id); + expect(ids).toHaveLength(24); + expect(new Set(ids).size).toBe(24); + expect(parse(page.items).filter((i) => i.private)).toHaveLength(8); + }); +}); + +describe("worker routes", () => { + const call = (path: string, init?: RequestInit) => + worker.fetch(new Request(`https://diigo-cache.test${path}`, init), env as never); + + it("wires a public cache whose page fills itself on a miss", async () => { + const wired = await call("/_wire", { + method: "POST", + headers: { "content-type": "application/json" }, + body: JSON.stringify({ webappId: WEBAPP_ID, mode: "public" }), + }); + expect(wired.status).toBe(200); + const { username } = (await wired.json()) as { username: string }; + expect(username).toBe("alice_99"); + + // page 2 was never fetched by anyone: the public cache asks the webapp + const html = await (await call(`/public/${username}?page=2`)).text(); + expect(html).toContain('
cache: MISS"); + const again = await (await call(`/public/${username}?page=2`)).text(); + expect(again).toContain("cache: HIT"); + expect(html).toContain("bookmark 25"); + expect(html).not.toContain("bookmark 27"); // private (27 % 3 === 0) + + const json = (await ( + await call(`/public/${username}?page=2`, { headers: { accept: "application/json" } }) + ).json()) as { pages: number; items: unknown[] }; + expect(json.pages).toBe(2); + expect(json.items).toHaveLength(4); // 25..30 minus 27, 30 + }); + + it("publishes a private cache later, without the cache id in the URL", async () => { + const cacheId = toWebkey(testEnv.CACHE_DO.newUniqueId()); + const res = await call("/_publish", { + method: "POST", + headers: { "content-type": "application/json" }, + body: JSON.stringify({ cacheId }), + }); + // never wired: refused, not crashed + expect(res.status).toBe(404); + }); +}); diff --git a/diigo-cache/tests/fake-webapp.js b/diigo-cache/tests/fake-webapp.js new file mode 100644 index 0000000..ab7af3f --- /dev/null +++ b/diigo-cache/tests/fake-webapp.js @@ -0,0 +1,72 @@ +/** + * @file a stand-in for webapp's `WebAppFactory` entrypoint, for tests: an + * auxiliary worker (plain JS — the vitest pool doesn't transform aux + * workers) serving diigo-shaped `load_user_items` pages. 30 items, every + * third private; one webapp id is known. Counts its diigo calls so tests + * can tell a cache hit (no call) from a miss. + */ +import { RpcTarget, WorkerEntrypoint } from "cloudflare:workers"; + +const KNOWN_WEBAPP = "fake-webapp-id"; +const TOTAL = 30; + +/** item n (1-based, updated-desc order): diigo's field names, made-up values */ +function item(n) { + return { + link_id: 1000 + n, + url: `https://example.com/${n}`, + title: `bookmark ${n}`, + description: "", + private: n % 3 === 0, + u_name: "alice_99", + first_by: "someone-else", + user_id: 4242, + tags: "", + created_at: 1700000000, + updated_at: 1700001000 - n, + }; +} + +let calls = 0; + +class FakeGet extends RpcTarget { + #path; + #params; + constructor(path, params) { + super(); + this.#path = path; + this.#params = params; + } + pathjoin(ref) { + return new FakeGet(ref, this.#params); + } + query(params) { + return new FakeGet(this.#path, { ...this.#params, ...params }); + } + async get() { + calls += 1; + if (this.#path !== "load_user_items") throw new Error(`fake webapp: no route ${this.#path}`); + const count = Number(this.#params.count); + const page = Number(this.#params.page_num); + const items = []; + for (let n = (page - 1) * count + 1; n <= Math.min(page * count, TOTAL); n += 1) items.push(item(n)); + return JSON.stringify({ total: TOTAL, items }); + } +} + +export class WebAppFactory extends WorkerEntrypoint { + async at(id) { + if (id !== KNOWN_WEBAPP) throw new Error("fake webapp: unknown id"); + return new FakeGet("", {}); + } + /** how many diigo calls so far (test hook) */ + async calls() { + return calls; + } +} + +export default { + fetch() { + return new Response("fake webapp", { status: 404 }); + }, +}; diff --git a/diigo-cache/tests/items.test.ts b/diigo-cache/tests/items.test.ts new file mode 100644 index 0000000..8ebc2db --- /dev/null +++ b/diigo-cache/tests/items.test.ts @@ -0,0 +1,72 @@ +import { describe, expect, it } from "vitest"; +import { publicOnly, toRows, usernameOf } from "../src/items.js"; +import type { Item } from "../src/items.js"; + +/** a load_user_items row: diigo's actual field names (observed + * 2026-10-07), made-up values */ +function diigoItem(linkId: number, url: string): Item { + return { + link_id: linkId, + url, + title: `title ${linkId}`, + description: "", + private: false, + u_name: "alice_99", + first_by: "bob-the-first", + user_id: 4242, + real_name: "Alice A.", + tags: "", + created_at: 1700000000, + updated_at: 1700000100 + linkId, + annotations: [], + comments: [], + }; +} + +function makeLog() { + const entries: { level: string; args: unknown[] }[] = []; + const at = (level: string) => (...args: unknown[]) => entries.push({ level, args }); + return { log: { warn: at("warn") }, entries }; +} + +describe("toRows", () => { + it("keys each item on diigo's link_id, so a page of distinct items stays distinct", () => { + const { log } = makeLog(); + const rows = toRows([diigoItem(11, "https://a.example/"), diigoItem(12, "https://b.example/")], log); + expect(rows.map((r) => r.itemId)).toEqual(["11", "12"]); + }); + + it("logs, rather than silently drops or merges, items it cannot key", () => { + const { log, entries } = makeLog(); + const noId = diigoItem(13, "https://c.example/"); + delete noId.link_id; + const noUrl = diigoItem(14, ""); + const rows = toRows([noId, noUrl, diigoItem(15, "https://d.example/")], log); + expect(rows.map((r) => r.itemId)).toEqual(["15"]); + expect(entries.map((e) => e.level)).toEqual(["warn", "warn"]); + // diagnostics name the fields present, never their values + expect(JSON.stringify(entries)).not.toContain("https://c.example/"); + }); +}); + +describe("usernameOf", () => { + it("reads the owner's username from diigo's u_name", () => { + expect(usernameOf(diigoItem(1, "https://a.example/"))).toBe("alice_99"); + }); + + it("answers empty for an item without one", () => { + expect(usernameOf(undefined)).toBe(""); + expect(usernameOf({ u_name: "has spaces" })).toBe(""); + }); +}); + +describe("publicOnly", () => { + it("keeps shared rows and drops private ones, before anything is stored", () => { + const { log } = makeLog(); + const shared = diigoItem(21, "https://shared.example/"); + const secret = { ...diigoItem(22, "https://secret.example/"), private: true }; + const rows = publicOnly(toRows([shared, secret], log)); + expect(rows.map((r) => r.itemId)).toEqual(["21"]); + expect(JSON.stringify(rows)).not.toContain("secret.example"); + }); +}); diff --git a/diigo-cache/tests/pm-save.test.ts b/diigo-cache/tests/pm-save.test.ts new file mode 100644 index 0000000..067553c --- /dev/null +++ b/diigo-cache/tests/pm-save.test.ts @@ -0,0 +1,62 @@ +import { describe, expect, it } from "vitest"; +import { pmSave } from "../src/pm-save.js"; +import type { PmPowers } from "../src/pm-save.js"; + +/** a fake console that records what was logged, by level */ +function makeLog() { + const entries: { level: string; args: unknown[] }[] = []; + const at = (level: string) => (...args: unknown[]) => entries.push({ level, args }); + return { log: { info: at("info"), warn: at("warn"), error: at("error") }, entries }; +} + +class FakeCredential { + constructor(readonly init: { id: string; password: string }) {} +} + +function setup(store: (c: unknown) => Promise, petname = "my cache") { + const { log, entries } = makeLog(); + const stored: unknown[] = []; + const powers: PmPowers = { + petname: { value: petname }, + credentials: { store: (c) => (stored.push(c), store(c)) }, + PasswordCredential: FakeCredential, + log, + }; + const btn = { textContent: "save to password manager", disabled: false }; + return { powers, btn, entries, stored }; +} + +describe("pmSave", () => { + it("stores under the petname, and reports a spec-conformant resolve (undefined) as offered, not failed", async () => { + const { powers, btn, entries, stored } = setup(() => Promise.resolve(undefined)); + await pmSave(btn, "the-key", powers); + expect(stored).toEqual([new FakeCredential({ id: "my cache", password: "the-key" })]); + expect(btn.textContent).not.toMatch(/not saved|fail/); + expect(btn.textContent).toMatch(/password manager/); + expect(btn.disabled).toBe(false); + expect(entries.map((e) => e.level)).toEqual(["info"]); + }); + + it("logs, never silently, when the browser has no credential store", async () => { + const { powers, btn, entries } = setup(() => Promise.resolve(undefined)); + delete powers.PasswordCredential; + await pmSave(btn, "the-key", powers); + expect(btn.textContent).toMatch(/no credential store/); + expect(entries.map((e) => e.level)).toEqual(["warn"]); + }); + + it("logs the error when the store rejects, without the key", async () => { + const { powers, btn, entries } = setup(() => Promise.reject(new Error("NotAllowedError"))); + await pmSave(btn, "the-key", powers); + expect(btn.textContent).toMatch(/save failed/); + expect(entries.map((e) => e.level)).toEqual(["error"]); + expect(JSON.stringify(entries)).not.toContain("the-key"); + }); + + it("asks for a petname rather than saving under an empty username", async () => { + const { powers, btn, stored } = setup(() => Promise.resolve(undefined), " "); + await pmSave(btn, "the-key", powers); + expect(stored).toEqual([]); + expect(btn.textContent).toMatch(/petname/); + }); +}); diff --git a/diigo-cache/tests/webkeys.test.ts b/diigo-cache/tests/webkeys.test.ts new file mode 100644 index 0000000..df36245 --- /dev/null +++ b/diigo-cache/tests/webkeys.test.ts @@ -0,0 +1,139 @@ +import { describe, expect, it } from "vitest"; +import { + yurl32, + unyurl32, + hexToBytes, + bytesToHex, + glyphsFor, + bytesInGlyphs, + toWebkey, + idFromWebkey, + isWebkey, + shashKey, + type DurableObjectIdLike, +} from "../src/webkeys.js"; + +// @ts-expect-error vite raw import +import webkeysSource from "../src/webkeys.ts?raw"; + +const sampleHex = "ab9d14667a05453602b28218c3dc60995d890e49c1502ee1fffa2d0815b367ee"; + +/** mock a Cloudflare DurableObjectId */ +function mockDurableObjectId(hex = sampleHex): DurableObjectIdLike { + return { + toString: () => hex, + equals: (other: unknown) => + typeof other === "object" && + other !== null && + "toString" in other && + (other as { toString(): string }).toString() === hex, + }; +} + +describe("yurl32 and hex codecs", () => { + it("matches the paper's alphabet sample", () => { + expect("mhbqcmmva5ja3").toMatch(/^[a-z2-7]{13}$/); + }); + + it("encodes 5 bits per glyph", () => { + expect(glyphsFor(64)).toBe(13); + expect(glyphsFor(128)).toBe(26); + expect(glyphsFor(160)).toBe(32); + expect(glyphsFor(256)).toBe(52); + expect(bytesInGlyphs(26)).toBe(16); + expect(bytesInGlyphs(52)).toBe(32); + }); + + it("round-trips hex bytes", () => { + const bytes = hexToBytes(sampleHex); + expect(bytes.length).toBe(32); + expect(bytesToHex(bytes)).toBe(sampleHex); + }); + + it("round-trips bytes through yurl32", () => { + const bytes = hexToBytes(sampleHex); + const encoded = yurl32(bytes); + expect(encoded).toMatch(/^[a-z2-7]+$/); + expect(encoded.length).toBe(52); + expect(unyurl32(encoded)).toEqual(bytes); + }); + + it("handles leftovers: 0xff encodes to '74'", () => { + expect(yurl32(new Uint8Array([0xff]))).toBe("74"); + }); + + it("refuses garbage", () => { + expect(() => unyurl32("0OIl")).toThrow(/not YURL32/); + expect(() => unyurl32("AAAA-")).toThrow(); + expect(() => hexToBytes("abc")).toThrow(/invalid hex length/); + expect(() => hexToBytes("zz")).toThrow(/invalid hex byte/); + }); +}); + +describe("toWebkey and idFromWebkey (aligning with Worker unique ID)", () => { + it("converts a DurableObjectId to a 52-glyph YURL32 webkey", () => { + const doId = mockDurableObjectId(); + const webkey = toWebkey(doId); + expect(webkey.length).toBe(52); + expect(webkey).toMatch(/^[a-z2-7]{52}$/); + expect(isWebkey(webkey)).toBe(true); + }); + + it("round-trips DurableObjectId -> webkey -> hex string", () => { + const doId = mockDurableObjectId(); + const webkey = toWebkey(doId); + const hex = idFromWebkey(webkey); + expect(hex).toBe(sampleHex); + }); + + it("accepts a raw 64-char hex string as input to toWebkey", () => { + const webkey = toWebkey(sampleHex); + expect(webkey.length).toBe(52); + expect(idFromWebkey(webkey)).toBe(sampleHex); + }); + + it("accepts an already-hex ID in idFromWebkey for backward compatibility", () => { + expect(idFromWebkey(sampleHex)).toBe(sampleHex); + expect(idFromWebkey(sampleHex.toUpperCase())).toBe(sampleHex); + }); + + it("encodes raw 16-byte (128-bit) entropy directly", () => { + const bytes = new Uint8Array(16).fill(42); + const webkey = toWebkey(bytes); + expect(webkey.length).toBe(26); + expect(isWebkey(webkey)).toBe(true); + expect(isWebkey(webkey, 128)).toBe(true); + expect(unyurl32(webkey)).toEqual(bytes); + }); + + it("refuses malformed or wrong-sized webkeys in idFromWebkey", () => { + expect(() => idFromWebkey("short")).toThrow(); + // 26-char (128-bit) webkey is valid YURL32, but not a 256-bit Worker ID: + const short128 = toWebkey(new Uint8Array(16)); + expect(() => idFromWebkey(short128)).toThrow(/expected 32 bytes/); + }); +}); + +describe("shashKey (swissHash discipline)", () => { + it("is one-way: identity, not authority", async () => { + const doId = mockDurableObjectId(); + const h = await shashKey(doId); + expect(h).toMatch(/^[a-z2-7]+$/); + expect(h.length).toBe(52); + expect(h).not.toBe(toWebkey(doId)); + }); + + it("is deterministic (joinable in logs)", async () => { + const doId = mockDurableObjectId(); + expect(await shashKey(doId)).toBe(await shashKey(doId)); + }); +}); + +describe("no ambient authority in this module", () => { + it("the module's functions take their effects as arguments", () => { + // strip comments before inspecting executable code + const code = webkeysSource.replace(/\/\*[\s\S]*?\*\/|\/\/.*/g, ""); + expect(code.includes("getRandomValues")).toBe(false); + expect(code.match(/crypto\.subtle/g)?.length ?? 0).toBe(1); // shashKey only + }); +}); \ No newline at end of file diff --git a/diigo-cache/vitest.config.ts b/diigo-cache/vitest.config.ts new file mode 100644 index 0000000..37b1516 --- /dev/null +++ b/diigo-cache/vitest.config.ts @@ -0,0 +1,28 @@ +import { defineProject } from "vitest/config"; +// the plugin resolves from diigo-bookmarks's store (the one vitest +// install on disk); direct-path import sidesteps pnpm's exports-map +// restriction *for the config file*. cloudflareTest (the plugin form, +// not the bare pool) is what provides the `cloudflare:test` module. +import { cloudflareTest } from "../diigo-bookmarks/node_modules/.pnpm/@cloudflare+vitest-plugin@1.3.6_@cloudflare+workers-types@5.20261005.1_@types+node@26.6.4_@vi_7mty7nznaeeyt2ohspxbzj4p6m/node_modules/@cloudflare/vitest-plugin/dist/pool/index.mjs"; + +export default defineProject({ + plugins: [ + cloudflareTest({ + // the real worker + DOs; WEBAPP is a fake (aux workers must be JS) + wrangler: { configPath: "./wrangler.tests.jsonc" }, + miniflare: { + workers: [ + { + name: "fake-webapp", + modules: true, + scriptPath: "./tests/fake-webapp.js", + compatibilityDate: "2026-09-29", + }, + ], + }, + }), + ], + test: { + include: ["tests/**/*.test.ts"], + }, +}); diff --git a/diigo-cache/wrangler.jsonc b/diigo-cache/wrangler.jsonc index d503bf8..798c1fd 100644 --- a/diigo-cache/wrangler.jsonc +++ b/diigo-cache/wrangler.jsonc @@ -1,6 +1,6 @@ { "$schema": "node_modules/wrangler/config-schema.json", - "name": "diigo-dump", + "name": "diigo-cache", "main": "src/index.ts", "compatibility_date": "2026-09-29", "observability": { "enabled": true }, @@ -10,5 +10,15 @@ "service": "webapp", "entrypoint": "WebAppFactory" } - ] + ], + "durable_objects": { + "bindings": [ + { "name": "CACHE_DO", "class_name": "DiigoCache" }, + { "name": "PUBLIC_DO", "class_name": "PublicCache" } + ] + }, + "exports": { + "DiigoCache": { "type": "durable-object", "storage": "sqlite" }, + "PublicCache": { "type": "durable-object", "storage": "sqlite" } + } } \ No newline at end of file diff --git a/diigo-cache/wrangler.tests.jsonc b/diigo-cache/wrangler.tests.jsonc new file mode 100644 index 0000000..49798e9 --- /dev/null +++ b/diigo-cache/wrangler.tests.jsonc @@ -0,0 +1,21 @@ +{ + "$schema": "node_modules/wrangler/config-schema.json", + // the real worker, with WEBAPP bound to tests/fake-webapp.js (an aux + // worker declared in vitest.config.ts) + "name": "diigo-cache-tests", + "main": "./src/index.ts", + "compatibility_date": "2026-09-29", + "services": [ + { "binding": "WEBAPP", "service": "fake-webapp", "entrypoint": "WebAppFactory" } + ], + "durable_objects": { + "bindings": [ + { "name": "CACHE_DO", "class_name": "DiigoCache" }, + { "name": "PUBLIC_DO", "class_name": "PublicCache" } + ] + }, + "exports": { + "DiigoCache": { "type": "durable-object", "storage": "sqlite" }, + "PublicCache": { "type": "durable-object", "storage": "sqlite" } + } +}