diigo-cache: bookmark cache in Durable Objects (goal 1c)

The dump worker becomes a page cache: a hit is served from DO SQLite
storage with no diigo call; a miss asks the webapp instance in the
cache's wiring and etches the page.

- two independent caches, one behavior (PagedCache): DiigoCache keeps
  every item; PublicCache keeps shared items only, dropping private
  rows before they are stored; served at /public/<username> (HTML page
  form, or JSON)
- create flow on one port: /_connect relays webapp's mint form, /_mint
  proxies the mint, /_wire wires a private or public cache; /_publish
  adds a public cache later
- ids are webkeys: kept out of URLs (POST bodies), shown and logged
  only as a hash fingerprint; saved to the password manager under the
  holder's petname (never sent to the server)
- items keyed on diigo's link_id; username from u_name; unkeyable
  items and every failure path are logged; each page read logs hit/miss
- tests run the real worker + DOs in the vitest pool against a fake
  webapp (tests/fake-webapp.js)

Generated-by: opencode (GLM)
Generated-by: Claude Opus 5.5
This commit is contained in:
dckc committed 2026-10-07 04:19:54 -05:00
1 parent c5e433cc9c
commit 6f409fc83a
15 files changed
+1663 -210

No files matched your search

+81 -37
View File
@@ -1,57 +1,101 @@
# diigo-dump # diigo-cache
Cloudflare Worker consumer of [the webapp factory](../webapp/README.md): Cloudflare Worker cache over [the webapp factory](../webapp/README.md):
the site-side of dumping [Diigo bookmarks from the mm-diigo provenance session-cookie reads of diigo bookmarks, cached in Durable-Object SQLite
scripts](https://github.com/dckc/mm-diigo) (`projects/diigo-bak/`) — storage. Evolved from the diigo-dump worker (provenance:
session-cookie access, most-recently-updated first. The static API-key [mm-diigo `projects/diigo-bak/`](https://github.com/dckc/mm-diigo)); the
path is dead (both known keys, all documented schemes → 401; see static API-key path is dead (see [the capsite checklist](../capsite-checklist.md),
[the capsite checklist](../capsite-checklist.md), goal 2 preamble). goal 2 preamble).
## What it holds ## What it holds
**Only the diigo flavoring** — everything generic lives in webapp: - **The create flow, on one port**: `/_connect` renders webapp's own
mint form (diigo-prefilled, cookie blank) — wrangler multi-config dev
- the mint call: `url` = `https://www.diigo.com/interact_api/`, seed doesn't serve aux workers' HTTP, so `/_mint` relays the form opaquely
headers `{user-agent, x-requested-with}` (+ the cookie at mint time), into the WEBAPP binding (the cookie crosses; nothing on this side opens
`msPerCall: 500` or logs it). Mint → `/_wire` (webapp id pre-filled, mode
- the dump page: page-number form → `WEBAPP.at(id)` → pipelined public|private) → the created page, where the user saves the ids.
`pathjoin("load_user_items").query({sort, count, page_num}).get()` - **Two independent page caches** (`src/cache.ts`), one behavior: a
- the `DiigoBridge` entrypoint (`open()`, `dumpNdjson()`) for other page hit is served from storage with no diigo call; a miss asks the
workers; `dumpBookmarks` (the generator from mm-diigo's webapp instance in the cache's wiring and etches the page.
diigo-dump.mjs) is in `src/diigo.ts` - `DiigoCache` (binding `CACHE_DO`): every item. Addressed by its
cache id.
GET `/` is inert; dumps happen only on POST submits. - `PublicCache` (binding `PUBLIC_DO`): shared items only — private
rows are dropped *before* etching (`publicOnly`), so its storage
holds nothing private. Same cookie webapp as the private cache.
Public page n is diigo page n, filtered (may show fewer than 24).
`/public/<username>` resolves through the `idFromName("index")`
registry.
- **Pages**: home's dump form (cache id + page number, POST) shows raw
dump rows; `/public/<username>?page=n` shows an HTML list with a page
form (JSON for `Accept: application/json`). Both say `cache: HIT` or
`MISS`; every page read also logs `{event: "cache", kind, page,
outcome}`.
- **Publish later**: `POST /_publish` (cache id in the body) wires a
public companion to a private-only cache.
- The `DiigoBridge` entrypoint for other workers (its cache-id source is
the goal-2 wiring TODO).
## State surface ## State surface
- The instance id (`DIIGO_WEB_ID`): the one secret (bearer capability; Ids are **webkeys** ([the pattern](../docs/webkeys.md)): 256-bit bearer
possession is authority). Not logged. secrets that never ride URLs — not query strings, not path segments.
- The cookie: lives **only in the webapp DO's storage**, crosses the wire Forms carry them in POST bodies; logs and pages show only a fingerprint
exactly once, at mint time. Nothing on disk holds it. (a prefix of the id's one-way hash).
- `/_connect`: state-aware — pre-mint it's the paste-cookie form;
post-mint it health-checks the instance (count=1 probe) and offers - **The cache id**: the cache's read capability (bearer; not logged).
re-mint only for stale cookies. Held by the user — saved to the password manager under a petname they
choose (the username field; never sent to the server) — and passed
per POST.
- **The webapp instance id**: the diigo-read capability; lives in each
cache DO's wiring, arrives from the mint step.
- **The cookie**: lives **only in webapp's DO storage**; crosses the
wire once, at mint time, as an opaque relay.
- **The public cache**: no capability — `/public/<username>` is public
by design (a guessable URL; hiding the link doesn't hide the page).
Stored state is meant to be durable: see the breaking-change rule in
[capsite AGENTS.md](../AGENTS.md).
## Deployment concern: public pages spend your diigo session
Anyone who can reach `/public/<username>` can make the cache fetch from
diigo through *your* cookie session: each request may spend up to
`FETCH_BUDGET` (40) diigo fetches on uncached pages (goal 1d), paced at
`MS_PER_CALL`. Cached pages cost nothing. Before exposing a deployment:
- rate-limit the public routes — on Cloudflare, the
[Workers rate-limit binding](https://developers.cloudflare.com/workers/runtime-apis/bindings/rate-limit/)
(keyed by client and/or username); on other workerd hosts, a rate
limit in the proxy in front;
- run the fill job (goal 1e) so most pages are cached before strangers
arrive.
## RPC use (from another worker, same account) ## RPC use (from another worker, same account)
```jsonc ```jsonc
// wrangler.jsonc of the caller // wrangler.jsonc of the caller
"services": [{ "binding": "DIIGO", "service": "diigo-dump", "entrypoint": "DiigoBridge" }] "services": [{ "binding": "DIIGO", "service": "diigo-cache", "entrypoint": "DiigoBridge" }]
``` ```
```ts ```ts
const stub = await env.DIIGO.open(); // cache ids ride goal-2's wiring; see the capsite checklist
// promise pipelining: whole chain = one round trip; cookie never leaves the DO const ndjson = await env.DIIGO.dumpNdjson(4, 24);
const first = JSON.parse(await stub.pathjoin("load_user_items")
.query({ sort: "updated", count: 24, page_num: 1 }).get());
``` ```
## Test
pnpm test
Runs the real worker and both DOs in the vitest pool, with `WEBAPP`
bound to [a fake webapp](tests/fake-webapp.js) serving diigo-shaped
items (`wrangler.tests.jsonc`). vitest is borrowed from
`../diigo-bookmarks/node_modules` (see `vitest.config.ts`).
## Run, deploy ## Run, deploy
[How to run / connect / deploy is maintained in the capsite checklist] [How to run / connect / deploy is maintained in the capsite checklist](../capsite-checklist.md)
(../capsite-checklist.md) (single source; this README keeps only (single source; this README keeps only worker-level facts).
worker-level facts).
Status: dev wiring verified (boots, pages answer); mint → dump through Status: create, dump, publish, and public pages exercised against live
the DO not yet exercised end-to-end (checklist 1b-i). Earlier flat diigo in dev (2026-10-07); not yet deployed.
(non-DO) build passed manual test: 200, `total: 1674` (2026-10-06).
+4 -2
View File
@@ -1,9 +1,11 @@
{ {
"name": "diigo-dump", "name": "diigo-cache",
"private": true, "private": true,
"type": "module",
"scripts": { "scripts": {
"dev": "wrangler dev -c wrangler.jsonc -c ../webapp/wrangler.jsonc", "dev": "wrangler dev -c wrangler.jsonc -c ../webapp/wrangler.jsonc",
"diigo-login": "scripts/diigo-login" "test": "vitest run",
"typecheck": "tsc -p tsconfig.check.json"
}, },
"devDependencies": { "devDependencies": {
"wrangler": "^4.148.0" "wrangler": "^4.148.0"
+252
View File
@@ -0,0 +1,252 @@
/**
* @file the cache Durable Objects: `DiigoCache` (public+private items) and
* `PublicCache` (public-only items, widely shared). They are independent
* and work the same way ({@link PagedCache}): pages are the unit of
* caching; a hit is answered from `ctx.storage.sqlite` with no diigo call;
* a miss asks the webapp instance in the cache's wiring and etches the
* page. The only difference is what each keeps: the public cache drops
* private rows *before* etching ({@link publicOnly}).
*/
import { DurableObject } from "cloudflare:workers";
import type { PagedGet } from "./diigo.js";
import { publicOnly, toRows } from "./items.js";
import type { Item, ItemRow } from "./items.js";
export type { Item };
/** schema migration steps; the sqlite-backed class starts empty, so V1 is
* the initial shape. `page_members` keeps each page's fetch-time item
* order: a page-hit replays that order exactly, independent of later
* re-sorting as items shift ranks. */
const SCHEMA_V1 = `
create table if not exists items (
item_id text primary key,
url text not null,
title text not null,
private integer not null,
json text not null,
updated text not null
);
create table if not exists pages (
page_num integer primary key,
count integer not null,
total integer,
etched_at text not null
);
create table if not exists page_members (
page_num integer not null,
pos integer not null,
item_id text not null,
primary key (page_num, pos)
);
`;
/** one cached page, raw: `items` are the dump rows (JSON strings), ready
* to re-join with newlines */
export type CachePage = {
total: number;
items: string[];
/** true when answered from storage (no diigo call) */
cached: boolean;
};
const MS_PER_CALL = 500;
function sleep(ms: number): Promise<void> {
return new Promise((r) => setTimeout(r, ms));
}
/** a cache's wiring: the webapp instance it asks on a miss, plus (private
* cache only) the public companion's id, recorded once published */
type Wiring = { webappId: string; publicId?: string | undefined };
/** the diigo-cache DO's bindings (structural; `at` mirrors webapp's
* WebAppFactory) */
export type CacheEnv = {
WEBAPP: {
at(id: string): Promise<PagedGet>;
};
};
/**
* the shared page-cache behavior. The webapp-instance id in the wiring
* authorizes diigo reads (the cookie rides in that DO's storage); the id
* this DO is addressed by authorizes cache reads. Neither is logged.
* `setup()` persists the wiring (webapp's configure pattern); `page(n)` is
* the read path; subclasses choose what to `keep` of a fetched page.
*/
abstract class PagedCache extends DurableObject<CacheEnv> {
#step = 0;
#pause?: Promise<void>;
/** one-shot wiring. Idempotent; different values are refused (mint a
* new cache instead). */
async setup(init: Wiring): Promise<void> {
const stored = await this.ctx.storage.get<Wiring>("init");
if (stored) {
if (stored.webappId !== init.webappId || stored.publicId !== init.publicId) {
throw new Error("already wired to different ids; mint a new cache instead");
}
return;
}
await this.ctx.storage.put("init", init);
}
/** the stored wiring */
async wiring(): Promise<Wiring | undefined> {
return this.ctx.storage.get<Wiring>("init");
}
/** which rows of a fetched page this cache stores */
protected abstract keep(rows: ItemRow[]): ItemRow[];
/** which cache this is, for logs (never the id: that's the bearer key) */
protected abstract readonly kind: "private" | "public";
/**
* read page `pageNum` (`count` per page, diigo's updated-desc order):
* storage first; on miss, fetch through the webapp instance (etching
* page 1 alongside — it carries `total`) and etch. Cache provenance
* rides in the result (`cached`).
*/
async page(pageNum: number, count: number): Promise<CachePage> {
const init = await this.wiring();
if (!init) throw new Error("not wired: this cache id was never set up");
this.ctx.storage.sql.exec(SCHEMA_V1);
// one diigo call path at a time, min spacing MS_PER_CALL (webapp's
// Gate pattern; per activation)
let before;
do {
before = this.#step;
if (this.#pause) await this.#pause;
} while (before !== this.#step);
this.#step += 1;
this.#pause = sleep(MS_PER_CALL);
const hit = await this.#pageFromStorage(pageNum, count);
if (hit) {
console.log({ event: "cache", kind: this.kind, page: pageNum, outcome: "hit", items: hit.items.length });
return hit;
}
const app = await this.env.WEBAPP.at(init.webappId);
const fetchPage = (n: number): Promise<{ total: number; items: Item[] }> =>
app
.pathjoin("load_user_items")
.query({ sort: "updated", count, page_num: n })
.get()
.then((t) => JSON.parse(t)) as Promise<{ total: number; items: Item[] }>;
// a miss on page n>1 etches page 1 first (it carries `total`)
const first = await fetchPage(1);
const data = pageNum === 1 ? first : await fetchPage(pageNum);
const kept = this.keep(toRows(data.items, console));
// fetched vs kept: the public cache's drop of private rows shows here
console.log({ event: "cache", kind: this.kind, page: pageNum, outcome: "miss", fetched: data.items.length, kept: kept.length });
return this.#etch(pageNum, count, data.total, kept);
}
/** etch a page (one transaction): upsert items, mark the page, keep
* the fetch-time order; returns the cache-page shape */
#etch(pageNum: number, count: number, total: number, rows: ItemRow[]): CachePage {
this.ctx.storage.transactionSync(() => {
this.ctx.storage.sql.exec("delete from page_members where page_num = ?", pageNum);
let pos = 0;
for (const r of rows) {
pos += 1;
this.ctx.storage.sql.exec(
"insert into items (item_id, url, title, private, json, updated) values (?, ?, ?, ?, ?, ?) on conflict(item_id) do update set url = excluded.url, title = excluded.title, private = excluded.private, json = excluded.json, updated = excluded.updated",
r.itemId, r.url, r.title, r.private ? 1 : 0, r.json, r.updated,
);
this.ctx.storage.sql.exec(
"insert into page_members (page_num, pos, item_id) values (?, ?, ?)",
pageNum, pos, r.itemId,
);
}
this.ctx.storage.sql.exec(
"insert into pages (page_num, count, total, etched_at) values (?, ?, ?, ?) on conflict(page_num) do update set count = excluded.count, total = excluded.total, etched_at = excluded.etched_at",
pageNum, count, total, new Date().toISOString(),
);
});
return { total, items: rows.map((r) => r.json), cached: false };
}
/** storage-backed page read; null = cache miss. Replay order comes
* from `page_members`. */
async #pageFromStorage(pageNum: number, count: number): Promise<CachePage | null> {
const page = this.ctx.storage.sql
.exec<{ total: number | null }>(
"select total from pages where page_num = ? and count = ?",
pageNum, count,
)
.toArray();
if (page.length === 0) return null;
const rows = this.ctx.storage.sql
.exec<{ json: string }>(
"select items.json from page_members join items on items.item_id = page_members.item_id where page_members.page_num = ? order by page_members.pos",
pageNum,
)
.toArray();
return { total: page[0].total ?? -1, items: rows.map((r) => r.json), cached: true };
}
}
/**
* the private cache (export name `DiigoCache`): public+private items,
* addressed by its cache id (a webkey, held by the user).
*/
export class DiigoCache extends PagedCache {
protected readonly kind = "private";
protected keep(rows: ItemRow[]): ItemRow[] {
return rows;
}
/** one-way addition of the public companion (private → public
* publishable later; once set, immutable). The webapp id already in
* the wiring is what makes this work without re-minting. */
async addPublic(publicId: string): Promise<void> {
const stored = await this.ctx.storage.get<Wiring>("init");
if (!stored) throw new Error("not wired");
if (stored.publicId) {
if (stored.publicId !== publicId) {
throw new Error("already has a public cache");
}
return;
}
await this.ctx.storage.put("init", { ...stored, publicId });
}
}
/**
* the public cache: shared items only — {@link publicOnly} drops private
* rows before they are stored, so this DO's storage holds nothing
* private. Widely shared: addressed by diigo username in the worker's
* route path (`/public/<username>`). It does hold the webapp id (the same
* cookie session the private cache uses), so it *could* read private
* items; it keeps none. Public page n is diigo page n, filtered: it may
* hold fewer than `count` items. Doubles as the username→instance index
* (the idFromName("index") instance holds the kv mapping and is never
* wired).
*/
export class PublicCache extends PagedCache {
protected readonly kind = "public";
protected keep(rows: ItemRow[]): ItemRow[] {
return publicOnly(rows);
}
/** register/replace the username → this-instance mapping (called on
* the idFromName("index") instance) */
async register(username: string, publicId: string): Promise<void> {
this.ctx.storage.kv.put(`user:${username}`, publicId);
}
/** resolve a username to a public-cache instance id (index role);
* undefined = not registered */
async lookup(username: string): Promise<string | undefined> {
return this.ctx.storage.kv.get<string>(`user:${username}`);
}
}
+472 -170
View File
@@ -1,65 +1,55 @@
/** /**
* @file diigo-dump worker: cookie access to diigo's * @file diigo-cache worker: caches diigo bookmark dumps in Durable-Object
* interact_api/load_user_items via a WebApp Durable Object instance from * SQLite storage. `PrivateCache` (public+private items) is addressed by its
* the webapp worker. The id is a bearer capability held in DIIGO_WEB_ID; * own unique id — minted by the create form alongside the webapp instance
* the cookie lives only inside that DO's storage. * (the cookie lives in that DO's storage); `PublicCache` (public-only
* items) is widely shared, addressed by the diigo username in the path.
* Dump output stays raw; a repeat page load is answered from storage with
* no diigo call.
* *
* Connect flow (no askpass, no cookie-on-disk): the home page links to * The create form's ids reach the browser only via explicit user action
* /_connect, which proxies webapp's mint form with the diigo prefills * (clipboard copy / file save — the poor-man's powerbox); a pasted id is
* (cookie field blank) — paste the cookie, mint lands via the service * how the home page's dump POST knows which cache to hit.
* binding, and the page shows the DIIGO_WEB_ID to put in .dev.vars.
* (scripts/diigo-login automates the same flow for terminals.)
* @see {@link DiigoBridge}
*/ */
import { WorkerEntrypoint } from "cloudflare:workers"; import { WorkerEntrypoint } from "cloudflare:workers";
import { dumpBookmarks, type PagedGet } from "./diigo.js"; import { DiigoCache, PublicCache } from "./cache.js";
import type { Item } from "./cache.js";
import { usernameOf } from "./items.js";
import { toWebkey, idFromWebkey, isWebkey, shashKey } from "./webkeys.js";
import { pmSave } from "./pm-save.js";
/** the WEBAPP service binding to webapp/src/index.ts:WebAppFactory; typed // the DO classes must be entrypoint exports (the exports/ binding targets
* structurally here pending a shared types package */ // the main module)
type WebAppFactoryShape = { export { DiigoCache, PublicCache };
newWebApp(init: {
url: string;
headers?: Record<string, string>;
msPerCall?: number;
}): Promise<{ id: string; app: PagedGet }>;
at(id: string): Promise<PagedGet>;
};
/** worker bindings (structural until wrangler types regenerates) */
type Env = { type Env = {
DIIGO_WEB_ID?: string; WEBAPP: {
WEBAPP: WebAppFactoryShape; fetch(request: Request): Promise<Response>;
};
CACHE_DO: DurableObjectNamespace<DiigoCache>;
PUBLIC_DO: DurableObjectNamespace<PublicCache>;
DIIGO_CACHE_ID?: string;
}; };
const UA = "Mozilla/5.0 (X11; Linux x86_64) Gecko/20100101 Firefox/141.0"; const UA = "Mozilla/5.0 (X11; Linux x86_64) Gecko/20100101 Firefox/141.0";
const DIIGO_INTERACT = "https://www.diigo.com/interact_api/"; const DIIGO_INTERACT = "https://www.diigo.com/interact_api/";
/** prefill for webapp's mint form: everything but the cookie */
const DIIGO_PREFILL = JSON.stringify({
"user-agent": UA,
"x-requested-with": "XMLHttpRequest",
});
/** connect to the configured instance, or give the setup pointer */
async function instance(env: Env): Promise<PagedGet> {
const id = env.DIIGO_WEB_ID;
if (!id) {
throw new Error("no instance: use the connect flow on the home page, then set DIIGO_WEB_ID");
}
return env.WEBAPP.at(id);
}
/** the diigo export (DiigoBridge lives on; the dumpNdjson cache-id source
* is TODO(2-i): ride the create form's handoff instead of a worker secret) */
export class DiigoBridge extends WorkerEntrypoint { export class DiigoBridge extends WorkerEntrypoint {
/** the diigo-attenuated handle: fluent chain over the instance */ /** bulk dump from the caller's address space: `pages * count` bookmarks
open(): Promise<PagedGet> { * through the cache (cache-hits are storage reads) */
return instance(this.env as unknown as Env);
}
/** bulk dump from the caller's address space */
async dumpNdjson(pages: number, count = 24): Promise<string> { async dumpNdjson(pages: number, count = 24): Promise<string> {
const diigoAPI = await instance(this.env as unknown as Env); const ns = (this.env as unknown as Env).CACHE_DO;
const cacheId = (this.env as unknown as Env).DIIGO_CACHE_ID;
if (!cacheId) throw new Error("no cache id supplied (caller config error)");
const cache = ns.get(ns.idFromString(idFromWebkey(cacheId)));
const lines: string[] = []; const lines: string[] = [];
for await (const item of dumpBookmarks({ diigoAPI, pages, count })) { for (let p = 1; p <= pages; p += 1) {
lines.push(JSON.stringify(item)); const page = await cache.page(p, count);
lines.push(...(p === 1 ? page.items.slice(0, count) : page.items));
} }
return lines.join("\n"); return lines.join("\n");
} }
@@ -69,170 +59,482 @@ export default {
async fetch(request: Request, env: Env): Promise<Response> { async fetch(request: Request, env: Env): Promise<Response> {
const url = new URL(request.url); const url = new URL(request.url);
// the connect flow: proxy webapp's mint form (prefilled, cookie blank) // public cache: widely shared, username in the path
if (url.pathname === "/_connect") { const pub = url.pathname.match(/^\/public\/([A-Za-z0-9_-]+)$/);
return env.DIIGO_WEB_ID ? connectedPage(env) : connectPage(); if (pub) return publicPage(request, env, pub[1]);
// webapp's mint form, relayed onto our port (wrangler multi-config
// dev doesn't mount an aux HTTP server): diigo-cache supplies only
// the diigo prefill (base URL + UA headers) and the form HTML is
// webapp's own — its init params (url, headers, msPerCall) are
// editable there; the POST relays opaquely (a conduit: the cookie
// crosses, no code on this side opens/logs it).
if (url.pathname === "/_connect" && request.method === "GET") {
return mintPage(env);
} }
// webapp's form POSTs back through us: forward to the binding
// the mint proxied: form fields (incl. the cookie) relay to
// WEBAPP.newWebApp; the response is the wire form, webapp id filled
if (url.pathname === "/_mint" && request.method === "POST") { if (url.pathname === "/_mint" && request.method === "POST") {
return proxyMint(request, env); return proxyMint(request, env);
} }
// the wire step: mint sent us back the webapp id (form-encoded from
// the mint form's hidden-continue, or JSON service-style). No cookie
// ever arrives here.
if (url.pathname === "/_wire" && request.method === "POST") {
return wire(request, env);
}
// publish a cache that was wired private-only: adds the public
// companion (derived from the private cache's stored wiring — the
// webapp id rides along, no re-mint), registers the username read
// from its items, seeds the projection. The cache id is a webkey: it
// rides the POST body, never the path.
if (url.pathname === "/_publish" && request.method === "POST") {
return publishCache(request, env);
}
if (url.pathname !== "/") return new Response("not found", { status: 404 }); if (url.pathname !== "/") return new Response("not found", { status: 404 });
if (request.method !== "POST") return home(null, undefined, Boolean(env.DIIGO_WEB_ID)); if (request.method === "POST") {
return dump(request, env);
const form = await request.formData();
const raw = form.get("page");
const page = typeof raw === "string" ? Number.parseInt(raw, 10) : Number.NaN;
if (Number.isNaN(page) || page < 1 || page > 10000) {
return home("enter a page number (1 = most recently updated)", undefined, Boolean(env.DIIGO_WEB_ID));
} }
try { // GET / is inert (dumps happen on POST submits). The cache id is a
const app = await instance(env); // webkey: it never rides the URL (query strings leak into history/
const text = await app // Referer/logs — see docs/webkeys.md); the wired form POST is how
.pathjoin("load_user_items") // it travels.
.query({ sort: "updated", count: 24, page_num: page }) return home(env, { error: "GET is inert: dumps happen on POST submits" }, null);
.get();
const data = JSON.parse(text) as { total?: number; items?: Record<string, unknown>[] };
return home(
`total: ${data.total}\nitems: ${data.items?.length ?? 0}\n\n${JSON.stringify(data.items ?? data, null, 2)}`,
undefined,
true,
);
} catch (err) {
console.error({ event: "dump", page, error: String(err) });
return home(null, String(err), Boolean(env.DIIGO_WEB_ID));
}
}, },
}; };
/** the connect page: webapp's form with diigo defaults. The mint POST is /** the connect page: webapp's own mint form HTML (fetched over the
* proxied (/_mint) so one port serves the whole flow; the response * service binding, action rewritten to our relay), carried with diigo's
* page shows the id and the .dev.vars line to add. */ * prefill only — the editable init params are webapp's, diigo-cache
async function connectPage(): Promise<Response> { * authors no form fields of its own */
// proxy: render our own form with action="/_mint" so one port serves async function mintPage(env: Env): Promise<Response> {
// the whole flow. TODO(webapp): one canonical mint form; consider const prefill =
// serving it from webapp directly. `?render=1&title=${encodeURIComponent("diigo cache: mint the session")}` +
const body = `<!doctype html> `&url=${encodeURIComponent(DIIGO_INTERACT)}` +
<html><head><meta charset="utf-8"><title>connect diigo session</title></head> `&headers=${encodeURIComponent(JSON.stringify({ "user-agent": UA, "x-requested-with": "XMLHttpRequest" }))}` +
<body> `&msPerCall=500`;
<h1>connect diigo session</h1> let html: string;
<ol>
<li>copy the <code>Cookie</code> header from diigo.com devtools</li>
<li>paste below, mint, then add the <code>DIIGO_WEB_ID</code> line it
shows to <code>.dev.vars</code> (deployed: <code>wrangler secret put DIIGO_WEB_ID</code>)</li>
<li>restart wrangler, dump from the <a href="/">home page</a></li>
</ol>
<form method="post" action="/_mint">
<input type="hidden" name="url" value="${esc(DIIGO_INTERACT)}">
<input type="hidden" name="headers" value="${esc(JSON.stringify({ "user-agent": UA, "x-requested-with": "XMLHttpRequest", cookie: "" }))}">
<p><label>cookie <input type="password" name="cookie" size="80" autocomplete="off"></label></p>
<p><button type="submit">mint</button></p>
</form>
</body></html>`;
return new Response(body, { headers: { "content-type": "text/html;charset=utf-8" } });
}
async function connectedPage(env: Env): Promise<Response> {
let health = "";
try { try {
const app = await instance(env); // a plain fetch() from inside the worker goes to the internet, not
// a cheap call: one item, exercises the cookie + rate gate // to the service binding — binding calls need a Request handed to
const text = await app // the stub (workerd's entrypoint-fetch protocol)
.pathjoin("load_user_items") const res = await env.WEBAPP.fetch(
.query({ sort: "updated", count: 1, page_num: 1 }) new Request(`https://form${prefill}`, { redirect: "manual" }),
.get(); );
const data = JSON.parse(text) as { total?: number }; html = await res.text();
health = `<p>instance is <b>live</b>: diigo reports <b>${esc(String(data.total ?? "?"))}</b> bookmarks.</p>`; html = html.replace('action="ACTION"', 'action="/_mint"');
} catch (err) { } catch (err) {
console.error({ event: "connect-check", error: String(err) }); console.error({ event: "mint-page", error: String(err) });
health = `<p>instance is configured but the last check <b>failed</b>:</p><pre>${esc(String(err))}</pre><p>If diigo expired the cookie, re-mint below.</p>`; return home(env, { error: "mint form unreachable (webapp worker down?)" }, null);
} }
const body = `<!doctype html> return new Response(html, { headers: { "content-type": "text/html;charset=utf-8" } });
<html><head><meta charset="utf-8"><title>diigo session</title></head>
<body>
<h1>diigo session</h1>
<p>connected: id <code>${esc(`${env.DIIGO_WEB_ID!.slice(0, 6)}…${env.DIIGO_WEB_ID!.slice(-4)}`)}</code> (from .dev.vars).</p>
${health}
<h2>replace the session</h2>
<p>Only needed if the cookie is stale (diigo will say 401/403). Minting a
new instance invalidates nothing, but the id rotates and .dev.vars needs
the new <code>DIIGO_WEB_ID</code>.</p>
<form method="post" action="/_mint">
<input type="hidden" name="url" value="${esc(DIIGO_INTERACT)}">
<input type="hidden" name="headers" value="${esc(JSON.stringify({ "user-agent": UA, "x-requested-with": "XMLHttpRequest", cookie: "" }))}">
<p><label>cookie <input type="password" name="cookie" size="80" autocomplete="off"></label></p>
<p><button type="submit">mint</button></p>
</form>
</body></html>`;
return new Response(body, { headers: { "content-type": "text/html;charset=utf-8" } });
} }
/** proxy the mint: merge the pasted cookie into the headers JSON, mint via /** the mint relay: the raw form body passes through untouched into the
* the service binding, show the id (plus the .dev.vars line to copy) */ * WEBAPP entrypoint's HTTP surface (webapp parses its own fields; the
* cookie crosses browser→here→binding→webapp's DO and no code on this
* side opens it) — a true conduit */
async function proxyMint(request: Request, env: Env): Promise<Response> { async function proxyMint(request: Request, env: Env): Promise<Response> {
const form = await request.formData(); try {
const cookie = String(form.get("cookie") || ""); const res = await env.WEBAPP.fetch(
if (!cookie) { new Request("https://mint/", {
return new Response("no cookie supplied", { status: 400 }); method: "POST",
headers: request.headers,
body: await request.arrayBuffer(),
}),
);
if (!res.ok) {
const detail = (await res.text()).slice(0, 200);
return home(env, { error: `mint said ${res.status}: ${detail}` }, null);
}
// webapp answers {id} (+ x-webapp-id); the id rides the header, the
// wire page is ours
const id = res.headers.get("x-webapp-id");
if (!id) return home(env, { error: "mint produced no id" }, null);
return await wirePage(id);
} catch (err) {
console.error({ event: "mint", error: String(err) });
return home(env, { error: `mint failed: ${String(err)}` }, null);
} }
const headers = {
"user-agent": UA,
"x-requested-with": "XMLHttpRequest",
cookie,
};
const { id } = await env.WEBAPP.newWebApp({
url: DIIGO_INTERACT,
headers,
msPerCall: 500,
});
return idPage(id);
} }
/** the id shown on the success page: a narrow text field (scrollable but /** the credential-save helper, once per page (see ./pm-save.ts) */
* too small to show most of the value) with a copy button for the full const PM_SAVE_JS = pmSave.toString();
* version. navigator.clipboard needs the page served from localhost/https
* — true in dev and at the deployed URL */ /** the suggested petname gets the holder's local hh:mm (the worker only
function idPage(id: string): Response { * knows UTC), so two saves the same day are told apart at a glance */
const PETNAME_TIME_JS = `{
const p = document.getElementById("petname");
const d = new Date();
p.value += " " + String(d.getHours()).padStart(2, "0") + ":" + String(d.getMinutes()).padStart(2, "0");
}`;
/** the onclick that hands pmSave its browser powers */
function pmSaveClick(value: string): string {
return `pmSave(this, ${JSON.stringify(value)}, { petname: document.getElementById("petname"), credentials: navigator.credentials, PasswordCredential: window.PasswordCredential, log: console })`;
}
/** the wire step, with the freshly-minted webapp id pre-filled */
async function wirePage(webappId: string): Promise<Response> {
const fingerprint = await nick(webappId);
const body = `<!doctype html> const body = `<!doctype html>
<html><head><meta charset="utf-8"><title>minted</title></head> <html><head><meta charset="utf-8"><title>diigo cache: step 2 — wire the cache</title></head>
<body> <body>
<h1>minted</h1> <h1>step 2: wire the cache</h1>
<p>id: <input id="wid" readonly size="4" value="${esc(id)}" onfocus="this.select()"> <p>minted: webapp id, fingerprint <code>${esc(fingerprint)}</code>
<button type="button" onclick='navigator.clipboard.writeText(${JSON.stringify(id)}).then(() => this.textContent = "copied ✓")'>copy</button></p> <button type="button" onclick='navigator.clipboard.writeText(${JSON.stringify(webappId)}).then(() => this.textContent = "copied ✓")'>copy</button></p>
<p>add to <code>.dev.vars</code>:</p> <p><label>your petname for it <input id="petname" autocomplete="off" size="30" value="${esc(`diigo webapp ${fingerprint}`)}"></label>
<pre>DIIGO_WEB_ID=</pre> <small>(private to you: the password-manager username; never sent to the server)</small></p>
<p>…click copy, then paste the full value after it. <a href="/">back to diigo dump</a> (restart wrangler first).</p> <script>${PETNAME_TIME_JS}</script>
<p><button type="button" onclick='${pmSaveClick(webappId)}'>save to password manager</button></p>
<p>choose the cache mode, then <b>wire</b> — the next page shows the
<b>cache id</b> (copy that too), after which you land back on home:</p>
<form method="post" action="/_wire">
<input type="hidden" name="webappId" value="${esc(webappId)}">
<p><label>mode <select name="mode"><option value="private">private</option><option value="public">public (serves /public/&lt;your-diigo-username&gt;)</option></select></label> <button type="submit">wire →</button></p>
</form>
<p>change your mind? <a href="/_connect">mint again</a> · <a href="/">home</a></p>
<script>${PM_SAVE_JS}</script>
</body></html>`; </body></html>`;
return new Response(body, { return new Response(body, {
headers: { headers: {
"content-type": "text/html;charset=utf-8", "content-type": "text/html;charset=utf-8",
// for scripts: diigo-login reads this instead of scraping HTML "x-webapp-id": webappId,
"x-diigo-web-id": id,
}, },
}); });
} }
function home(detail: string | null, error?: string, connected?: boolean): Response { /** the wire step: webapp-instance id + the public/private choice. The
const form = `<form method="post" action="/"><label>page <input type="number" name="page" min="1"></label> <button type="submit">dump</button></form>`; * cache is minted, wired to {webappId, publicId}, and page 1 is etched;
const result = error * in public mode the diigo username is read from the fetched items
? `<p>status: <b>error</b></p><pre readonly style="white-space:pre-wrap;border:1px solid #c00;padding:1rem">${escapeHtml(error)}</pre>` * (self-authenticating: the cookie's account, never form input) and the
: detail * public cache is wired to the same webapp id and registered. The two
? `<pre readonly style="white-space:pre-wrap;border:1px solid #ccc;padding:1rem;max-height:60vh;overflow:auto">${escapeHtml(detail)}</pre>` * caches are independent from then on: each fills itself on a miss. */
async function wire(request: Request, env: Env): Promise<Response> {
const ct = request.headers.get("content-type") ?? "";
let webappId = "";
let mode = "private";
let wantsJson = false;
try {
if (ct.includes("application/json")) {
const body = (await request.json()) as { webappId?: string; mode?: string };
webappId = String(body.webappId ?? "");
mode = String(body.mode ?? "private");
wantsJson = true;
} else {
const form = await request.formData();
webappId = String(form.get("webappId") || "");
mode = String(form.get("mode") || "private");
// a browser form POST stays on the html path; only explicit JSON
// (service callers) gets the bare-JSON response
wantsJson = false;
}
} catch (err) {
return bad(`bad input: ${String(err)}`);
}
if (!/^[A-Za-z0-9-]+$/.test(webappId)) return bad("no webapp id supplied");
const publicId = mode === "public" ? toWebkey(env.PUBLIC_DO.newUniqueId()) : undefined;
const cacheId = toWebkey(env.CACHE_DO.newUniqueId());
const cacheStub = env.CACHE_DO.get(env.CACHE_DO.idFromString(idFromWebkey(cacheId)));
await cacheStub.setup({ webappId, publicId });
const first = await cacheStub.page(1, 24);
if (publicId) {
const firstItem = first.items[0] ? (JSON.parse(first.items[0]) as Item) : undefined;
const username = usernameOf(firstItem);
if (!username) {
// diagnostic context without item contents (privacy: items may
// carry URL/titles; the count + cached flag are safe)
console.error({
event: "wire",
stage: "username-derive",
// field names, not values: says which shape we got
itemFields: firstItem ? Object.keys(firstItem) : null,
itemsOnPage1: first.items.length,
total: first.total,
cached: first.cached,
});
return bad("could not read your diigo username (u_name) from the fetched items");
}
const index = env.PUBLIC_DO.get(env.PUBLIC_DO.idFromName("index"));
const pubStub = env.PUBLIC_DO.get(env.PUBLIC_DO.idFromString(idFromWebkey(publicId)));
await pubStub.setup({ webappId });
await index.register(username, publicId);
if (wantsJson) return Response.json({ cacheId, publicId, username });
return wiredPage(cacheId, `public cache live at <a href="/public/${esc(encodeURIComponent(username))}">/public/${esc(username)}</a>`);
}
if (wantsJson) {
return Response.json({
cacheId,
publishLater: { method: "POST", url: `${new URL(request.url).origin}/_publish`, body: { cacheId } },
});
}
return wiredPage(
cacheId,
`private-only for now — to add the public cache later:
<form method="post" action="/_publish" style="display:inline">
<input type="hidden" name="cacheId" value="${esc(cacheId)}">
<button type="submit">publish</button></form>
(or <code>POST /_publish</code> with <code>cacheId</code> in the body)`,
);
}
/** the created-cache page: ids + save/copy buttons + the pointer home */
async function wiredPage(cacheId: string, note: string): Promise<Response> {
const fingerprint = await nick(cacheId);
const body = `<!doctype html>
<html><head><meta charset="utf-8"><title>diigo cache: created</title></head>
<body>
<h1>cache created</h1>
<p><b>cache id</b>, fingerprint <code>${esc(fingerprint)}</code>
<button type="button" onclick='navigator.clipboard.writeText(${JSON.stringify(cacheId)}).then(() => this.textContent = "copied ✓")'>copy</button></p>
<p><label>your petname for it <input id="petname" autocomplete="off" size="30" value="${esc(`diigo cache ${fingerprint}`)}"></label>
<small>(private to you: the password-manager username; never sent to the server)</small></p>
<script>${PETNAME_TIME_JS}</script>
<p><button type="button" onclick='${pmSaveClick(cacheId)}'>save to password manager</button></p>
<p>${note}</p>
<p>the cache id is the only way back to this cache — saved (above) it lives
with the rest of your credentials. it is a <b>webkey</b>: it never travels
in URLs (query strings leak to history/Referer/logs); the dump POST carries
it in its body instead. the webapp id rides inside the cache's wiring;
hold it separately only if you want another cache off the same session.</p>
<a href="/">← home (paste the cache id into the dump form)</a>
<script>${PM_SAVE_JS}</script>
</body></html>`;
return new Response(body, { headers: { "content-type": "text/html;charset=utf-8" } });
}
/** publish, later: the private cache's wiring already holds the webapp
* id, so the public companion needs no re-mint; the username comes from
* the items already in the cache */
async function publishCache(request: Request, env: Env): Promise<Response> {
let cacheId = "";
try {
const ct = request.headers.get("content-type") ?? "";
if (ct.includes("application/json")) {
const body = (await request.json()) as { cacheId?: string };
cacheId = String(body.cacheId ?? "");
} else {
const form = await request.formData();
cacheId = String(form.get("cacheId") || "");
}
} catch (err) {
return bad(`bad input: ${String(err)}`);
}
if (!isWebkey(cacheId) && !/^[0-9a-fA-F]{64}$/.test(cacheId)) {
return bad("no cache id supplied (POST body: cacheId)");
}
try {
const ns = env.CACHE_DO;
const cache = ns.get(ns.idFromString(idFromWebkey(cacheId)));
const wiring = await cache.wiring();
if (!wiring) {
return Response.json({ error: "unknown cache id" }, { status: 404 });
}
if (wiring.publicId) {
return Response.json({ cacheId, publicId: wiring.publicId, status: "already published" });
}
// seed the projection *now*, while the cache is still private-only:
// only rows already cached and marked shared can enter the public
// side; the username rides the same rows
const firstPage = await cache.page(1, 24);
const firstItem = firstPage.items[0] ? (JSON.parse(firstPage.items[0]) as Item) : undefined;
const username = usernameOf(firstItem);
if (!username) {
console.error({
event: "publish",
stage: "username-derive",
// field names, not values: says which shape we got
itemFields: firstItem ? Object.keys(firstItem) : null,
itemsOnPage1: firstPage.items.length,
total: firstPage.total,
cached: firstPage.cached,
});
return Response.json(
{ error: "could not read your diigo username (u_name) from the cached items" },
{ status: 400 },
);
}
const publicId = toWebkey(env.PUBLIC_DO.newUniqueId());
const pubStub = env.PUBLIC_DO.get(env.PUBLIC_DO.idFromString(idFromWebkey(publicId)));
await pubStub.setup({ webappId: wiring.webappId });
await cache.addPublic(publicId);
const index = env.PUBLIC_DO.get(env.PUBLIC_DO.idFromName("index"));
await index.register(username, publicId);
return Response.json({ cacheId, publicId, username, public: `/public/${username}` });
} catch (err) {
// no cacheId in the log: it is the bearer secret
console.error({ event: "publish", error: String(err) });
return Response.json({ error: `publish failed: ${String(err)}` }, { status: 400 });
}
}
function bad(message: string): Response {
return Response.json({ error: message }, { status: 400 });
}
/** the private dump POST: cache id from the (client-wired) form value */
async function dump(request: Request, env: Env): Promise<Response> {
const form = await request.formData();
const cacheId = String(form.get("cacheId") || "");
const raw = form.get("page");
const pageN = typeof raw === "string" ? Number.parseInt(raw, 10) : Number.NaN;
if (!isWebkey(cacheId) && !/^[0-9a-fA-F]{64}$/.test(cacheId)) {
return home(env, { error: "wire a cache id first (create flow, then copy/paste)" }, null);
}
if (Number.isNaN(pageN) || pageN < 1 || pageN > 10000) {
return home(env, { error: "enter a page number (1 = most recently updated)" }, null);
}
try {
const ns = env.CACHE_DO;
const cache = ns.get(ns.idFromString(idFromWebkey(cacheId)));
const page = await cache.page(pageN, 24);
const data = `total: ${page.total}\nitems: ${page.items.length}\ncache: ${page.cached ? "HIT (from storage)" : "MISS (etched now)"}` +
`\n\n${page.items.join("\n")}`;
return home(env, null, data, cacheId);
} catch (err) {
console.error({ event: "dump", page: pageN, error: String(err) });
return home(env, null, String(err), cacheId);
}
}
const PUBLIC_PER_PAGE = 24;
/** the public page for a username: an HTML page with a page-number form
* (GET — nothing secret here, so `?page=` in the URL is fine, unlike the
* webkey routes), or the same page as JSON for `Accept: application/json`
* callers */
async function publicPage(request: Request, env: Env, username: string): Promise<Response> {
const index = env.PUBLIC_DO.get(env.PUBLIC_DO.idFromName("index"));
const publicId = await index.lookup(username);
if (!publicId) return new Response("no such public cache", { status: 404 });
const ns = env.PUBLIC_DO;
const cache = ns.get(ns.idFromString(idFromWebkey(publicId)));
const raw = new URL(request.url).searchParams.get("page");
const pageN = raw === null ? 1 : Number.parseInt(raw, 10);
const valid = !Number.isNaN(pageN) && pageN >= 1 && pageN <= 10000;
let page;
try {
// a hit is storage-only; a miss asks the webapp (private rows are
// dropped before etching)
page = await cache.page(valid ? pageN : 1, PUBLIC_PER_PAGE);
} catch (err) {
console.error({ event: "public-page", page: pageN, error: String(err) });
return new Response(`could not load page ${valid ? pageN : 1}: ${String(err)}`, { status: 502 });
}
// total and pages count all of diigo's items: public page n is diigo
// page n with the private rows dropped
const pages = Math.max(1, Math.ceil(page.total / PUBLIC_PER_PAGE));
const items = page.items.map((j) => JSON.parse(j) as Item);
const accept = request.headers.get("accept") ?? "";
if (accept.includes("application/json") && !accept.includes("text/html")) {
return Response.json({ username, page: pageN, pages, total: page.total, items });
}
let notice = "";
if (!valid || pageN > pages) {
console.log({ event: "notice", page: "public", text: "page out of range", pages });
notice = `<p><em>no page ${esc(String(raw))}: pages run 1–${pages}</em></p>`;
}
const here = `/public/${encodeURIComponent(username)}`;
const nav = (n: number, label: string) =>
n >= 1 && n <= pages ? `<a href="${here}?page=${n}">${label}</a>` : `<span>${label}</span>`;
const list = notice
? ""
: `<ol>
${items
.map((it) => {
const url = typeof it.url === "string" ? it.url : "";
const title = typeof it.title === "string" && it.title ? it.title : url;
const desc = typeof it.description === "string" && it.description ? `<br><small>${esc(it.description)}</small>` : "";
const href = /^https?:\/\//i.test(url) ? url : "#";
return `<li><a href="${esc(href)}">${esc(title)}</a>${desc}</li>`;
})
.join("\n")}
</ol>`;
const body = `<!doctype html>
<html><head><meta charset="utf-8"><title>${esc(username)}'s shared bookmarks</title></head>
<body>
<h1>${esc(username)}'s shared bookmarks</h1>
<p>${pages} page${pages === 1 ? "" : "s"} (diigo's, ${PUBLIC_PER_PAGE} bookmarks each; private ones are left out, so a page may show fewer).</p>
<p><strong>cache: ${page.cached ? "HIT (from storage)" : "MISS (fetched from diigo now)"}</strong> — page ${valid ? pageN : 1}, ${items.length} shared</p>
<form method="get" action="${here}">
<p>${nav(pageN - 1, "← prev")}
<label>page <input type="number" name="page" min="1" value="${Number.isNaN(pageN) ? 1 : pageN}"></label>
<button type="submit">go</button>
${nav(pageN + 1, "next →")}</p>
</form>
${notice}${list}
</body></html>`;
return new Response(body, { headers: { "content-type": "text/html;charset=utf-8" } });
}
/** home page: the connect link (webapp's cookie form, diigo-prefilled) +
* the wire step for an already-minted id + the dump form. `home` renders
* the error path too, so it takes env for the connect link. Notices (the
* inert-GET explainer, probe failures) log at info — exceptional errors
* log at error in their own handlers. */
async function home(env: Env, opts: { error?: string } | null, data: string | null, cacheId?: string): Promise<Response> {
if (opts?.error) {
console.log({ event: "notice", page: "home", text: opts.error });
}
const cacheState = cacheId
? `<p>cache id, fingerprint <code>${esc(await nick(cacheId))}</code></p>`
: `<p>no cache id wired yet — use the create flow below (or paste one).</p>`;
const dumpForm = `<form method="post" action="/">
${cacheId ? `<input type="hidden" name="cacheId" value="${esc(cacheId)}">` : `<p><label>petname <input autocomplete="username" size="24"></label>
<label>cache id <input type="password" name="cacheId" autocomplete="current-password" size="24"></label></p>`}
<p><label>page <input type="number" name="page" min="1"></label> <button type="submit">dump</button></p>
</form>`;
const result = data
? `<pre readonly style="white-space:pre-wrap;border:1px solid #ccc;padding:1rem;max-height:60vh;overflow:auto">${escapeHtml(data)}</pre>`
: opts && opts.error
? `<p><em>${escapeHtml(opts.error)}</em></p>`
: ""; : "";
return new Response( return new Response(
`<!doctype html> `<!doctype html>
<html><head><meta charset="utf-8"><title>diigo dump</title></head> <html><head><meta charset="utf-8"><title>diigo cache</title></head>
<body> <body>
<h1>diigo dump</h1> <h1>diigo cache</h1>
<p><a href="/_connect">${connected ? "diigo session status/replace…" : "connect diigo session…"}</a></p> ${cacheState}
${form} <h2>create (two steps)</h2>
<ol>
<li><a href="/_connect">mint the diigo session at webapp</a> — webapp's
own mint form (diigo-prefilled); paste the Cookie header there. The
cookie transits this worker only as an opaque relay into webapp's DO;
nobody here opens it. Mint lands you on the wire step, id pre-filled.</li>
<li>wire the id into a cache (mode: public|private):</li>
</ol>
<form method="post" action="/_wire">
<p><label>petname <input autocomplete="username" size="24"></label>
<label>webapp id <input type="password" name="webappId" size="24" autocomplete="current-password"></label>
<label>mode <select name="mode"><option value="private">private</option><option value="public">public (serves /public/&lt;your-diigo-username&gt;)</option></select></label></p>
<p><button type="submit">wire</button></p>
</form>
<p>the ids this mints are for <b>you</b>: copy them somewhere you control (password manager)</p>
${dumpForm}
${result} ${result}
</body></html>`, </body></html>`,
{ headers: { "content-type": "text/html;charset=utf-8" } }, { headers: { "content-type": "text/html;charset=utf-8" } },
); );
} }
/** a short display name for a bearer id: a prefix of its swissHash
* (identity without authority), so labels and pages never show any
* glyphs of the key itself */
async function nick(id: string): Promise<string> {
return (await shashKey(id)).slice(0, 8);
}
function esc(s: string): string { function esc(s: string): string {
return s return s
.replaceAll("&", "&amp;") .replaceAll("&", "&amp;")
+66
View File
@@ -0,0 +1,66 @@
/**
* @file diigo interact_api items: the field-reading helpers (pure, so
* tests can feed them rows shaped like diigo's).
*/
/** one diigo item exactly as the interact_api returned it */
export type Item = Record<string, unknown>;
/** the fields stored per item (the raw row rides in `json`) */
export type ItemRow = {
itemId: string;
url: string;
title: string;
private: boolean;
/** the raw dump row, JSON-encoded */
json: string;
updated: string;
};
/** diigo's id for a bookmark (`link_id`), as text; "" when absent */
function linkIdOf(it: Item): string {
return typeof it.link_id === "number" || (typeof it.link_id === "string" && it.link_id !== "")
? String(it.link_id)
: "";
}
/**
* normalize one page of interact_api items into storable rows. Items
* that can't be stored (no url, no link_id) are logged and skipped —
* never merged under a made-up key. Logs name the fields present, not
* their values (items may be private).
*/
export function toRows(items: Item[], log: Pick<Console, "warn">): ItemRow[] {
const rows: ItemRow[] = [];
for (const it of items) {
const itemId = linkIdOf(it);
const url = typeof it.url === "string" ? it.url : "";
if (!itemId || !url) {
log.warn({ event: "item-skip", reason: !itemId ? "no link_id" : "no url", fields: Object.keys(it) });
continue;
}
rows.push({
itemId,
url,
title: String(it.title ?? url),
private: it.private !== undefined ? Boolean(it.private) : String(it.shared) !== "yes",
json: JSON.stringify(it),
updated: String(it.updated_at ?? it.created_at ?? ""),
});
}
return rows;
}
/** the diigo username an item belongs to (`u_name`; not `first_by`,
* who may be whoever bookmarked the URL first), or "" when the item
* carries none in the expected shape */
export function usernameOf(item: Item | undefined): string {
const username = typeof item?.u_name === "string" ? item.u_name : "";
return /^[A-Za-z0-9_-]{1,64}$/.test(username) ? username : "";
}
/** the public cache's filter: shared rows only. Applied before etching,
* so private rows never reach public storage. */
export function publicOnly(rows: ItemRow[]): ItemRow[] {
return rows.filter((r) => !r.private);
}
+57
View File
@@ -0,0 +1,57 @@
/**
* @file the password-manager save — ids reach the browser's storage only by
* this gesture-gated path (or copy), never localStorage/IndexedDB, which
* are ambient and XSS-reachable. It runs in the *browser*: pages embed it
* via `pmSave.toString()`, so it must be self-contained (no imports, no
* closure, no inner named functions — wrangler's keepNames would wrap
* those in an `__name` helper the browser doesn't have). Its powers
* (the petname field, the credential store, the console) are passed in,
* so tests can stand in for the browser.
*/
/** the browser powers pmSave uses */
export type PmPowers = {
petname: { value: string };
credentials?: { store(credential: unknown): Promise<unknown> };
PasswordCredential?: new (init: { id: string; password: string }) => unknown;
log: Pick<Console, "info" | "warn" | "error">;
};
/** the button pmSave reports on */
export type PmButton = { textContent: string | null; disabled: boolean };
/**
* gesture-gated save of `value` under the holder's petname (the PM
* username).
*
* @see https://developer.mozilla.org/en-US/docs/Web/API/CredentialsContainer/store
*/
export function pmSave(btn: PmButton, value: string, powers: PmPowers): Promise<void> {
const label = powers.petname.value.trim();
if (!label) {
btn.textContent = "give it a petname first";
return Promise.resolve();
}
btn.disabled = true;
if (!powers.PasswordCredential || !powers.credentials) {
powers.log.warn({ event: "pm-save", outcome: "unsupported", detail: "no PasswordCredential / navigator.credentials" });
btn.textContent = "browser has no credential store — copy instead";
btn.disabled = false;
return Promise.resolve();
}
return powers.credentials
.store(new powers.PasswordCredential({ id: label, password: value }))
.then(() => {
// store() resolves with undefined per spec, whether the holder
// accepted the browser's prompt or not (or no prompt showed):
// we can only say it was offered — the PM's own UI is the truth
powers.log.info({ event: "pm-save", outcome: "offered", label });
btn.textContent = "offered to the password manager ✓ (see its prompt)";
btn.disabled = false;
})
.catch((err) => {
powers.log.error({ event: "pm-save", error: String(err) });
btn.textContent = "save failed — copy instead";
btn.disabled = false;
});
}
+204
View File
@@ -0,0 +1,204 @@
/**
* @file webkeys: URL-borne bearer permissions, per Tyler Close's
* ["web-key: Mashing with permission"](https://waterken.sourceforge.net/web-key/)
* (IEEE W2SP 2008); the design abstract lives in
* [docs/webkeys.md](../../docs/webkeys.md), with the Waterken precursor
* code in `kpreid/e-on-java` as provenance (YURL32 = `org.waterken.uri.
* Base32` / `BigIntegerSugar.toYURL32`; bearer invariant = `EARL.java`;
* identity-without-authority = `SwissTable`'s swissHash).
*
* A webkey separates secret from address:
* - the **secret bits**: supplied directly by the caller as a Worker
* unique ID (e.g. `DurableObjectId` from `ns.newUniqueId()`), or raw
* bytes (`Uint8Array`). Per [OCap discipline]
* (.opencode/skills/ocap-discipline/SKILL.md), functions here are pure:
* no ambient RNG calls, no callback indirection. The caller holds the
* minting authority at the application boundary and supplies the result.
* - the **encoding**: YURL32 — lowercase `a–z` + `2–7`, 5 bits/char,
* URL-safe by construction (no `-`, no `_`, no case to misread).
*
* Cloudflare Worker `DurableObjectId`s are 256 bits (32 bytes), rendered
* as 64 hex characters by `id.toString()`. {@link toWebkey} encodes that
* ID into 52 YURL32 glyphs; {@link idFromWebkey} decodes it back to the
* 64-hex string expected by `ns.idFromString()`.
*
* {@link shashKey} gives the identity-without-authority twin (the
* swissHash discipline) for log-correlation: one-way, on SHA-256.
*/
/** the YURL32 alphabet: `a–z` then `2–7` (32 glyphs, 5 bits each) */
const YURL32_ALPHABET = "abcdefghijklmnopqrstuvwxyz234567" as const;
/** encode bytes as YURL32 (`a–z2–7`, 5 bits/char; no padding, the tail
* byte's leftover bits shift-left, as `org.waterken.uri.Base32.encode`
* does) */
export function yurl32(bytes: Uint8Array): string {
let out = "";
let buffer = 0;
let bits = 0;
for (const byte of bytes) {
buffer = (buffer << 8) | byte;
bits += 8;
while (bits >= 5) {
out += YURL32_ALPHABET[(buffer >>> (bits - 5)) & 31];
bits -= 5;
}
}
if (bits > 0) {
out += YURL32_ALPHABET[(buffer << (5 - bits)) & 31];
}
return out;
}
/** decode YURL32 back to bytes (inverse of {@link yurl32}); throws on
* any character outside the alphabet — a webkey is either a webkey or
* it is not */
export function unyurl32(text: string): Uint8Array {
let buffer = 0;
let bits = 0;
const out: number[] = [];
for (const ch of text) {
const v = YURL32_ALPHABET.indexOf(ch);
if (v < 0) throw new Error(`not a webkey: ${JSON.stringify(ch)} is not YURL32`);
buffer = (buffer << 5) | v;
bits += 5;
if (bits >= 8) {
out.push((buffer >>> (bits - 8)) & 0xff);
bits -= 8;
}
}
// leftover bits (<8) are the encoder's zero-padding: validate, don't
// carry them
if (bits > 0 && (buffer & ((1 << bits) - 1)) !== 0) {
throw new Error("not a webkey: non-zero tail bits");
}
return new Uint8Array(out);
}
/** parse a hexadecimal string to raw bytes */
export function hexToBytes(hex: string): Uint8Array {
const trimmed = hex.trim();
if (trimmed.length % 2 !== 0) throw new Error("invalid hex length");
const bytes = new Uint8Array(trimmed.length / 2);
for (let i = 0; i < bytes.length; i += 1) {
const val = Number.parseInt(trimmed.slice(i * 2, i * 2 + 2), 16);
if (Number.isNaN(val)) throw new Error(`invalid hex byte at offset ${i * 2}`);
bytes[i] = val;
}
return bytes;
}
/** encode raw bytes to a lowercase hexadecimal string */
export function bytesToHex(bytes: Uint8Array): string {
let hex = "";
for (const b of bytes) {
hex += b.toString(16).padStart(2, "0");
}
return hex;
}
export const BITS_PER_GLYPH = 5;
/** chars needed for n bits in YURL32 */
export function glyphsFor(bits: number): number {
return Math.ceil(bits / BITS_PER_GLYPH);
}
/** YURL32 glyphs → bytes recoverable via {@link unyurl32} */
export function bytesInGlyphs(glyphs: number): number {
return Math.floor((glyphs * BITS_PER_GLYPH) / 8);
}
/**
* Structural type for a Cloudflare Worker unique ID (such as `DurableObjectId`),
* matching `id.toString()` which returns the 64-hex string.
*/
export interface DurableObjectIdLike {
toString(): string;
equals?(other: unknown): boolean;
readonly name?: string;
}
/** A Worker unique ID input: a `DurableObjectId`, 64-char hex string, or raw bytes. */
export type WorkerUniqueId = DurableObjectIdLike | Uint8Array;
/**
* Convert a Worker unique ID (or raw bytes) to a YURL32 webkey string.
*
* @param id A `DurableObjectId` (from `ns.newUniqueId()`), a 64-char hex ID,
* or raw bytes (`Uint8Array`).
* @returns A YURL32 webkey string (52 glyphs for 256-bit `DurableObjectId`,
* 26 glyphs for 128-bit raw bytes).
*/
export function toWebkey(id: WorkerUniqueId | string): string {
if (id instanceof Uint8Array) {
return yurl32(id);
}
const str = typeof id === "string" ? id.trim() : id.toString();
if (/^[0-9a-fA-F]{64}$/.test(str)) {
return yurl32(hexToBytes(str));
}
if (isWebkey(str)) {
return str;
}
throw new Error("cannot convert ID to webkey: unrecognized format");
}
/**
* Convert a YURL32 webkey string back to the 64-character hex ID string
* expected by `DurableObjectNamespace.idFromString(hex)`.
*
* If the input is already a 64-character hex ID, it is validated and returned
* in lowercase for backwards compatibility.
*/
export function idFromWebkey(webkey: string): string {
const trimmed = webkey.trim();
if (/^[0-9a-fA-F]{64}$/.test(trimmed)) {
return trimmed.toLowerCase();
}
const bytes = unyurl32(trimmed);
if (bytes.length !== 32) {
throw new Error(
`expected 32 bytes (256 bits) for Worker unique ID, got ${bytes.length}`,
);
}
return bytesToHex(bytes);
}
/**
* Is `text` a well-formed webkey of the expected size?
*
* When `bits` is omitted, accepts standard 256-bit Worker unique ID webkeys
* (52 glyphs), 128-bit webkeys (26 glyphs), or 64-bit webkeys (13 glyphs).
*/
export function isWebkey(text: string, bits?: number): boolean {
if (bits !== undefined) {
if (text.length !== glyphsFor(bits)) return false;
try {
return unyurl32(text).length === Math.floor(bits / 8);
} catch {
return false;
}
}
if (text.length === 52 || text.length === 26 || text.length === 13) {
try {
const b = unyurl32(text);
return b.length === 32 || b.length === 16 || b.length === 8;
} catch {
return false;
}
}
return false;
}
/**
* The identity-without-authority twin (the swissHash discipline):
* SHA-256 hash of the webkey, YURL32-encoded (52 glyphs).
*
* Safe for logs/metrics/joins; never accepted as a credential.
*/
export async function shashKey(key: string | WorkerUniqueId): Promise<string> {
const text = typeof key === "string" ? key : toWebkey(key);
const digest = await crypto.subtle.digest("SHA-256", new TextEncoder().encode(text));
return yurl32(new Uint8Array(digest));
}
+122
View File
@@ -0,0 +1,122 @@
import { env } from "cloudflare:workers";
import { runInDurableObject } from "cloudflare:test";
import { describe, expect, it, vi } from "vitest";
import worker from "../src/index.js";
import { toWebkey } from "../src/webkeys.js";
// matches tests/fake-webapp.js: 30 items, every third private
const WEBAPP_ID = "fake-webapp-id";
type TestEnv = {
CACHE_DO: DurableObjectNamespace<import("../src/cache.js").DiigoCache>;
PUBLIC_DO: DurableObjectNamespace<import("../src/cache.js").PublicCache>;
WEBAPP: { calls(): Promise<number> };
};
const testEnv = env as unknown as TestEnv;
const parse = (items: string[]) => items.map((j) => JSON.parse(j) as Record<string, unknown>);
describe("PublicCache", () => {
it("on a miss asks the webapp and keeps only shared items; a hit makes no call", async () => {
const stub = testEnv.PUBLIC_DO.get(testEnv.PUBLIC_DO.newUniqueId());
await stub.setup({ webappId: WEBAPP_ID });
const before = await testEnv.WEBAPP.calls();
const miss = await stub.page(1, 24);
expect(miss.cached).toBe(false);
expect(miss.total).toBe(30);
// diigo page 1 = items 1..24, of which 8 are private
expect(parse(miss.items).map((i) => i.private)).toEqual(Array(16).fill(false));
const afterMiss = await testEnv.WEBAPP.calls();
expect(afterMiss).toBeGreaterThan(before);
const hit = await stub.page(1, 24);
expect(hit.cached).toBe(true);
expect(hit.items).toEqual(miss.items);
expect(await testEnv.WEBAPP.calls()).toBe(afterMiss);
});
it("logs each page read as a hit or a miss (fetched vs kept), without the cache id", async () => {
const spy = vi.spyOn(console, "log");
try {
const id = testEnv.PUBLIC_DO.newUniqueId();
const stub = testEnv.PUBLIC_DO.get(id);
await stub.setup({ webappId: WEBAPP_ID });
await stub.page(1, 24);
await stub.page(1, 24);
const cacheLogs = spy.mock.calls.map((c) => c[0]).filter((e) => e?.event === "cache");
expect(cacheLogs).toEqual([
{ event: "cache", kind: "public", page: 1, outcome: "miss", fetched: 24, kept: 16 },
{ event: "cache", kind: "public", page: 1, outcome: "hit", items: 16 },
]);
expect(JSON.stringify(spy.mock.calls)).not.toContain(id.toString());
} finally {
spy.mockRestore();
}
});
it("stores no private rows at all", async () => {
const stub = testEnv.PUBLIC_DO.get(testEnv.PUBLIC_DO.newUniqueId());
await stub.setup({ webappId: WEBAPP_ID });
await stub.page(1, 24);
await stub.page(2, 24);
await runInDurableObject(stub, (_obj, state) => {
const rows = state.storage.sql.exec<{ n: number }>("select count(*) as n from items where private = 1").toArray();
expect(rows[0].n).toBe(0);
});
});
});
describe("DiigoCache", () => {
it("keeps every item of a page, each under its own link_id", async () => {
const stub = testEnv.CACHE_DO.get(testEnv.CACHE_DO.newUniqueId());
await stub.setup({ webappId: WEBAPP_ID });
const page = await stub.page(1, 24);
const ids = parse(page.items).map((i) => i.link_id);
expect(ids).toHaveLength(24);
expect(new Set(ids).size).toBe(24);
expect(parse(page.items).filter((i) => i.private)).toHaveLength(8);
});
});
describe("worker routes", () => {
const call = (path: string, init?: RequestInit) =>
worker.fetch(new Request(`https://diigo-cache.test${path}`, init), env as never);
it("wires a public cache whose page fills itself on a miss", async () => {
const wired = await call("/_wire", {
method: "POST",
headers: { "content-type": "application/json" },
body: JSON.stringify({ webappId: WEBAPP_ID, mode: "public" }),
});
expect(wired.status).toBe(200);
const { username } = (await wired.json()) as { username: string };
expect(username).toBe("alice_99");
// page 2 was never fetched by anyone: the public cache asks the webapp
const html = await (await call(`/public/${username}?page=2`)).text();
expect(html).toContain('<form method="get"');
expect(html).toContain("<strong>cache: MISS");
const again = await (await call(`/public/${username}?page=2`)).text();
expect(again).toContain("<strong>cache: HIT");
expect(html).toContain("bookmark 25");
expect(html).not.toContain("bookmark 27"); // private (27 % 3 === 0)
const json = (await (
await call(`/public/${username}?page=2`, { headers: { accept: "application/json" } })
).json()) as { pages: number; items: unknown[] };
expect(json.pages).toBe(2);
expect(json.items).toHaveLength(4); // 25..30 minus 27, 30
});
it("publishes a private cache later, without the cache id in the URL", async () => {
const cacheId = toWebkey(testEnv.CACHE_DO.newUniqueId());
const res = await call("/_publish", {
method: "POST",
headers: { "content-type": "application/json" },
body: JSON.stringify({ cacheId }),
});
// never wired: refused, not crashed
expect(res.status).toBe(404);
});
});
+72
View File
@@ -0,0 +1,72 @@
/**
* @file a stand-in for webapp's `WebAppFactory` entrypoint, for tests: an
* auxiliary worker (plain JS — the vitest pool doesn't transform aux
* workers) serving diigo-shaped `load_user_items` pages. 30 items, every
* third private; one webapp id is known. Counts its diigo calls so tests
* can tell a cache hit (no call) from a miss.
*/
import { RpcTarget, WorkerEntrypoint } from "cloudflare:workers";
const KNOWN_WEBAPP = "fake-webapp-id";
const TOTAL = 30;
/** item n (1-based, updated-desc order): diigo's field names, made-up values */
function item(n) {
return {
link_id: 1000 + n,
url: `https://example.com/${n}`,
title: `bookmark ${n}`,
description: "",
private: n % 3 === 0,
u_name: "alice_99",
first_by: "someone-else",
user_id: 4242,
tags: "",
created_at: 1700000000,
updated_at: 1700001000 - n,
};
}
let calls = 0;
class FakeGet extends RpcTarget {
#path;
#params;
constructor(path, params) {
super();
this.#path = path;
this.#params = params;
}
pathjoin(ref) {
return new FakeGet(ref, this.#params);
}
query(params) {
return new FakeGet(this.#path, { ...this.#params, ...params });
}
async get() {
calls += 1;
if (this.#path !== "load_user_items") throw new Error(`fake webapp: no route ${this.#path}`);
const count = Number(this.#params.count);
const page = Number(this.#params.page_num);
const items = [];
for (let n = (page - 1) * count + 1; n <= Math.min(page * count, TOTAL); n += 1) items.push(item(n));
return JSON.stringify({ total: TOTAL, items });
}
}
export class WebAppFactory extends WorkerEntrypoint {
async at(id) {
if (id !== KNOWN_WEBAPP) throw new Error("fake webapp: unknown id");
return new FakeGet("", {});
}
/** how many diigo calls so far (test hook) */
async calls() {
return calls;
}
}
export default {
fetch() {
return new Response("fake webapp", { status: 404 });
},
};
+72
View File
@@ -0,0 +1,72 @@
import { describe, expect, it } from "vitest";
import { publicOnly, toRows, usernameOf } from "../src/items.js";
import type { Item } from "../src/items.js";
/** a load_user_items row: diigo's actual field names (observed
* 2026-10-07), made-up values */
function diigoItem(linkId: number, url: string): Item {
return {
link_id: linkId,
url,
title: `title ${linkId}`,
description: "",
private: false,
u_name: "alice_99",
first_by: "bob-the-first",
user_id: 4242,
real_name: "Alice A.",
tags: "",
created_at: 1700000000,
updated_at: 1700000100 + linkId,
annotations: [],
comments: [],
};
}
function makeLog() {
const entries: { level: string; args: unknown[] }[] = [];
const at = (level: string) => (...args: unknown[]) => entries.push({ level, args });
return { log: { warn: at("warn") }, entries };
}
describe("toRows", () => {
it("keys each item on diigo's link_id, so a page of distinct items stays distinct", () => {
const { log } = makeLog();
const rows = toRows([diigoItem(11, "https://a.example/"), diigoItem(12, "https://b.example/")], log);
expect(rows.map((r) => r.itemId)).toEqual(["11", "12"]);
});
it("logs, rather than silently drops or merges, items it cannot key", () => {
const { log, entries } = makeLog();
const noId = diigoItem(13, "https://c.example/");
delete noId.link_id;
const noUrl = diigoItem(14, "");
const rows = toRows([noId, noUrl, diigoItem(15, "https://d.example/")], log);
expect(rows.map((r) => r.itemId)).toEqual(["15"]);
expect(entries.map((e) => e.level)).toEqual(["warn", "warn"]);
// diagnostics name the fields present, never their values
expect(JSON.stringify(entries)).not.toContain("https://c.example/");
});
});
describe("usernameOf", () => {
it("reads the owner's username from diigo's u_name", () => {
expect(usernameOf(diigoItem(1, "https://a.example/"))).toBe("alice_99");
});
it("answers empty for an item without one", () => {
expect(usernameOf(undefined)).toBe("");
expect(usernameOf({ u_name: "has spaces" })).toBe("");
});
});
describe("publicOnly", () => {
it("keeps shared rows and drops private ones, before anything is stored", () => {
const { log } = makeLog();
const shared = diigoItem(21, "https://shared.example/");
const secret = { ...diigoItem(22, "https://secret.example/"), private: true };
const rows = publicOnly(toRows([shared, secret], log));
expect(rows.map((r) => r.itemId)).toEqual(["21"]);
expect(JSON.stringify(rows)).not.toContain("secret.example");
});
});
+62
View File
@@ -0,0 +1,62 @@
import { describe, expect, it } from "vitest";
import { pmSave } from "../src/pm-save.js";
import type { PmPowers } from "../src/pm-save.js";
/** a fake console that records what was logged, by level */
function makeLog() {
const entries: { level: string; args: unknown[] }[] = [];
const at = (level: string) => (...args: unknown[]) => entries.push({ level, args });
return { log: { info: at("info"), warn: at("warn"), error: at("error") }, entries };
}
class FakeCredential {
constructor(readonly init: { id: string; password: string }) {}
}
function setup(store: (c: unknown) => Promise<unknown>, petname = "my cache") {
const { log, entries } = makeLog();
const stored: unknown[] = [];
const powers: PmPowers = {
petname: { value: petname },
credentials: { store: (c) => (stored.push(c), store(c)) },
PasswordCredential: FakeCredential,
log,
};
const btn = { textContent: "save to password manager", disabled: false };
return { powers, btn, entries, stored };
}
describe("pmSave", () => {
it("stores under the petname, and reports a spec-conformant resolve (undefined) as offered, not failed", async () => {
const { powers, btn, entries, stored } = setup(() => Promise.resolve(undefined));
await pmSave(btn, "the-key", powers);
expect(stored).toEqual([new FakeCredential({ id: "my cache", password: "the-key" })]);
expect(btn.textContent).not.toMatch(/not saved|fail/);
expect(btn.textContent).toMatch(/password manager/);
expect(btn.disabled).toBe(false);
expect(entries.map((e) => e.level)).toEqual(["info"]);
});
it("logs, never silently, when the browser has no credential store", async () => {
const { powers, btn, entries } = setup(() => Promise.resolve(undefined));
delete powers.PasswordCredential;
await pmSave(btn, "the-key", powers);
expect(btn.textContent).toMatch(/no credential store/);
expect(entries.map((e) => e.level)).toEqual(["warn"]);
});
it("logs the error when the store rejects, without the key", async () => {
const { powers, btn, entries } = setup(() => Promise.reject(new Error("NotAllowedError")));
await pmSave(btn, "the-key", powers);
expect(btn.textContent).toMatch(/save failed/);
expect(entries.map((e) => e.level)).toEqual(["error"]);
expect(JSON.stringify(entries)).not.toContain("the-key");
});
it("asks for a petname rather than saving under an empty username", async () => {
const { powers, btn, stored } = setup(() => Promise.resolve(undefined), " ");
await pmSave(btn, "the-key", powers);
expect(stored).toEqual([]);
expect(btn.textContent).toMatch(/petname/);
});
});
+139
View File
@@ -0,0 +1,139 @@
import { describe, expect, it } from "vitest";
import {
yurl32,
unyurl32,
hexToBytes,
bytesToHex,
glyphsFor,
bytesInGlyphs,
toWebkey,
idFromWebkey,
isWebkey,
shashKey,
type DurableObjectIdLike,
} from "../src/webkeys.js";
// @ts-expect-error vite raw import
import webkeysSource from "../src/webkeys.ts?raw";
const sampleHex = "ab9d14667a05453602b28218c3dc60995d890e49c1502ee1fffa2d0815b367ee";
/** mock a Cloudflare DurableObjectId */
function mockDurableObjectId(hex = sampleHex): DurableObjectIdLike {
return {
toString: () => hex,
equals: (other: unknown) =>
typeof other === "object" &&
other !== null &&
"toString" in other &&
(other as { toString(): string }).toString() === hex,
};
}
describe("yurl32 and hex codecs", () => {
it("matches the paper's alphabet sample", () => {
expect("mhbqcmmva5ja3").toMatch(/^[a-z2-7]{13}$/);
});
it("encodes 5 bits per glyph", () => {
expect(glyphsFor(64)).toBe(13);
expect(glyphsFor(128)).toBe(26);
expect(glyphsFor(160)).toBe(32);
expect(glyphsFor(256)).toBe(52);
expect(bytesInGlyphs(26)).toBe(16);
expect(bytesInGlyphs(52)).toBe(32);
});
it("round-trips hex bytes", () => {
const bytes = hexToBytes(sampleHex);
expect(bytes.length).toBe(32);
expect(bytesToHex(bytes)).toBe(sampleHex);
});
it("round-trips bytes through yurl32", () => {
const bytes = hexToBytes(sampleHex);
const encoded = yurl32(bytes);
expect(encoded).toMatch(/^[a-z2-7]+$/);
expect(encoded.length).toBe(52);
expect(unyurl32(encoded)).toEqual(bytes);
});
it("handles leftovers: 0xff encodes to '74'", () => {
expect(yurl32(new Uint8Array([0xff]))).toBe("74");
});
it("refuses garbage", () => {
expect(() => unyurl32("0OIl")).toThrow(/not YURL32/);
expect(() => unyurl32("AAAA-")).toThrow();
expect(() => hexToBytes("abc")).toThrow(/invalid hex length/);
expect(() => hexToBytes("zz")).toThrow(/invalid hex byte/);
});
});
describe("toWebkey and idFromWebkey (aligning with Worker unique ID)", () => {
it("converts a DurableObjectId to a 52-glyph YURL32 webkey", () => {
const doId = mockDurableObjectId();
const webkey = toWebkey(doId);
expect(webkey.length).toBe(52);
expect(webkey).toMatch(/^[a-z2-7]{52}$/);
expect(isWebkey(webkey)).toBe(true);
});
it("round-trips DurableObjectId -> webkey -> hex string", () => {
const doId = mockDurableObjectId();
const webkey = toWebkey(doId);
const hex = idFromWebkey(webkey);
expect(hex).toBe(sampleHex);
});
it("accepts a raw 64-char hex string as input to toWebkey", () => {
const webkey = toWebkey(sampleHex);
expect(webkey.length).toBe(52);
expect(idFromWebkey(webkey)).toBe(sampleHex);
});
it("accepts an already-hex ID in idFromWebkey for backward compatibility", () => {
expect(idFromWebkey(sampleHex)).toBe(sampleHex);
expect(idFromWebkey(sampleHex.toUpperCase())).toBe(sampleHex);
});
it("encodes raw 16-byte (128-bit) entropy directly", () => {
const bytes = new Uint8Array(16).fill(42);
const webkey = toWebkey(bytes);
expect(webkey.length).toBe(26);
expect(isWebkey(webkey)).toBe(true);
expect(isWebkey(webkey, 128)).toBe(true);
expect(unyurl32(webkey)).toEqual(bytes);
});
it("refuses malformed or wrong-sized webkeys in idFromWebkey", () => {
expect(() => idFromWebkey("short")).toThrow();
// 26-char (128-bit) webkey is valid YURL32, but not a 256-bit Worker ID:
const short128 = toWebkey(new Uint8Array(16));
expect(() => idFromWebkey(short128)).toThrow(/expected 32 bytes/);
});
});
describe("shashKey (swissHash discipline)", () => {
it("is one-way: identity, not authority", async () => {
const doId = mockDurableObjectId();
const h = await shashKey(doId);
expect(h).toMatch(/^[a-z2-7]+$/);
expect(h.length).toBe(52);
expect(h).not.toBe(toWebkey(doId));
});
it("is deterministic (joinable in logs)", async () => {
const doId = mockDurableObjectId();
expect(await shashKey(doId)).toBe(await shashKey(doId));
});
});
describe("no ambient authority in this module", () => {
it("the module's functions take their effects as arguments", () => {
// strip comments before inspecting executable code
const code = webkeysSource.replace(/\/\*[\s\S]*?\*\/|\/\/.*/g, "");
expect(code.includes("getRandomValues")).toBe(false);
expect(code.match(/crypto\.subtle/g)?.length ?? 0).toBe(1); // shashKey only
});
});
+28
View File
@@ -0,0 +1,28 @@
import { defineProject } from "vitest/config";
// the plugin resolves from diigo-bookmarks's store (the one vitest
// install on disk); direct-path import sidesteps pnpm's exports-map
// restriction *for the config file*. cloudflareTest (the plugin form,
// not the bare pool) is what provides the `cloudflare:test` module.
import { cloudflareTest } from "../diigo-bookmarks/node_modules/.pnpm/@cloudflare+vitest-plugin@1.3.6_@cloudflare+workers-types@5.20261005.1_@types+node@26.6.4_@vi_7mty7nznaeeyt2ohspxbzj4p6m/node_modules/@cloudflare/vitest-plugin/dist/pool/index.mjs";
export default defineProject({
plugins: [
cloudflareTest({
// the real worker + DOs; WEBAPP is a fake (aux workers must be JS)
wrangler: { configPath: "./wrangler.tests.jsonc" },
miniflare: {
workers: [
{
name: "fake-webapp",
modules: true,
scriptPath: "./tests/fake-webapp.js",
compatibilityDate: "2026-09-29",
},
],
},
}),
],
test: {
include: ["tests/**/*.test.ts"],
},
});
+11 -1
View File
@@ -1,6 +1,6 @@
{ {
"$schema": "node_modules/wrangler/config-schema.json", "$schema": "node_modules/wrangler/config-schema.json",
"name": "diigo-dump", "name": "diigo-cache",
"main": "src/index.ts", "main": "src/index.ts",
"compatibility_date": "2026-09-29", "compatibility_date": "2026-09-29",
"observability": { "enabled": true }, "observability": { "enabled": true },
@@ -10,5 +10,15 @@
"service": "webapp", "service": "webapp",
"entrypoint": "WebAppFactory" "entrypoint": "WebAppFactory"
} }
],
"durable_objects": {
"bindings": [
{ "name": "CACHE_DO", "class_name": "DiigoCache" },
{ "name": "PUBLIC_DO", "class_name": "PublicCache" }
] ]
},
"exports": {
"DiigoCache": { "type": "durable-object", "storage": "sqlite" },
"PublicCache": { "type": "durable-object", "storage": "sqlite" }
}
} }
+21
View File
@@ -0,0 +1,21 @@
{
"$schema": "node_modules/wrangler/config-schema.json",
// the real worker, with WEBAPP bound to tests/fake-webapp.js (an aux
// worker declared in vitest.config.ts)
"name": "diigo-cache-tests",
"main": "./src/index.ts",
"compatibility_date": "2026-09-29",
"services": [
{ "binding": "WEBAPP", "service": "fake-webapp", "entrypoint": "WebAppFactory" }
],
"durable_objects": {
"bindings": [
{ "name": "CACHE_DO", "class_name": "DiigoCache" },
{ "name": "PUBLIC_DO", "class_name": "PublicCache" }
]
},
"exports": {
"DiigoCache": { "type": "durable-object", "storage": "sqlite" },
"PublicCache": { "type": "durable-object", "storage": "sqlite" }
}
}