diigo-cache: fill from a webkey page over Cap'n Web
Starting a fill no longer POSTs the cache id to /_fill. Instead: - the created page links the cache's webkey, /cache/#<cache id>: a skeleton page, the same for every key, that reads the key from the fragment (never sent to the server or in a Referer) - its script trades the key in-band for a capability over Cap'n Web at /api: CacheApi.cache(key) returns a CacheCap facet (fill, status, fingerprint, the public companion's fill) and never the DO's wiring methods; malformed and unknown keys get one refusal that never echoes the key, and each refusal is logged without it - then calls startFill(): trade and call pipeline into one round trip - the worker serves capnweb's browser build itself at /_capnweb.js (a wrangler Text rule), so no third-party script sits beside the key The /_fill route and its forms are gone. Tests run the real capnweb client against the worker (fetch routed to worker.fetch). Generated-by: Claude Opus 5.5
This commit is contained in:
1 parent
fbe2538345
commit
8c221abe6f
9 files changed
+397
-112
No files matched your search
+17
-9
@@ -33,20 +33,27 @@ goal 2 preamble).
|
|||||||
outcome}`.
|
outcome}`.
|
||||||
- **Publish later**: `POST /_publish` (cache id in the body) wires a
|
- **Publish later**: `POST /_publish` (cache id in the body) wires a
|
||||||
public companion to a private-only cache.
|
public companion to a private-only cache.
|
||||||
- **Fill job**: `POST /_fill` (cache id in the body; `target=private|public`,
|
- **The webkey page and Cap'n Web**: `/cache/#<cache id>` is a skeleton
|
||||||
`action=start|stop`) walks every diigo page into a cache in the
|
page, the same for every key; its script reads the key from the
|
||||||
background — DO alarms, a few pages per step, progress stored in the
|
fragment and trades it in-band for a capability over
|
||||||
DO (`startFill`/`fillStatus`/`stopFill`). The created page has the
|
[Cap'n Web](https://github.com/cloudflare/capnweb) at `/api`
|
||||||
buttons; the public page shows progress.
|
(`CacheApi.cache(key)` → a `CacheCap` facet: fill, status, the public
|
||||||
|
companion; `src/api.ts`). The worker serves capnweb's browser build
|
||||||
|
itself at `/_capnweb.js`.
|
||||||
|
- **Fill job**: walks every diigo page into a cache in the background —
|
||||||
|
DO alarms, a few pages per step, progress stored in the DO
|
||||||
|
(`startFill`/`fillStatus`/`stopFill`), driven from the webkey page;
|
||||||
|
the public page shows progress.
|
||||||
- The `DiigoBridge` entrypoint for other workers (its cache-id source is
|
- The `DiigoBridge` entrypoint for other workers (its cache-id source is
|
||||||
the goal-2 wiring TODO).
|
the goal-2 wiring TODO).
|
||||||
|
|
||||||
## State surface
|
## State surface
|
||||||
|
|
||||||
Ids are **webkeys** ([the pattern](../docs/webkeys.md)): 256-bit bearer
|
Ids are **webkeys** ([the pattern](../docs/webkeys.md)): 256-bit bearer
|
||||||
secrets that never ride URLs — not query strings, not path segments.
|
secrets that never ride a query string or path. The webkey page carries
|
||||||
Forms carry them in POST bodies; logs and pages show only a fingerprint
|
the cache id in the URL fragment (never sent to the server) and passes
|
||||||
(a prefix of the id's one-way hash).
|
it in-band over Cap'n Web; forms carry ids in POST bodies; logs and pages
|
||||||
|
show only a fingerprint (a prefix of the id's one-way hash).
|
||||||
|
|
||||||
- **The cache id**: the cache's read capability (bearer; not logged).
|
- **The cache id**: the cache's read capability (bearer; not logged).
|
||||||
Held by the user — saved to the password manager under a petname they
|
Held by the user — saved to the password manager under a petname they
|
||||||
@@ -69,7 +76,8 @@ diigo through *your* cookie session: each request may spend up to
|
|||||||
`FETCH_BUDGET` (40) diigo fetches on uncached pages (goal 1d), paced at
|
`FETCH_BUDGET` (40) diigo fetches on uncached pages (goal 1d), paced at
|
||||||
`MS_PER_CALL`. Cached pages cost nothing. Before exposing a deployment:
|
`MS_PER_CALL`. Cached pages cost nothing. Before exposing a deployment:
|
||||||
|
|
||||||
- rate-limit the public routes — on Cloudflare, the
|
- rate-limit the public routes and `/api` (Cap'n Web pipelining makes
|
||||||
|
calls cheap to send) — on Cloudflare, the
|
||||||
[Workers rate-limit binding](https://developers.cloudflare.com/workers/runtime-apis/bindings/rate-limit/)
|
[Workers rate-limit binding](https://developers.cloudflare.com/workers/runtime-apis/bindings/rate-limit/)
|
||||||
(keyed by client and/or username); on other workerd hosts, a rate
|
(keyed by client and/or username); on other workerd hosts, a rate
|
||||||
limit in the proxy in front;
|
limit in the proxy in front;
|
||||||
|
|||||||
@@ -13,5 +13,8 @@
|
|||||||
"typescript": "^5.9.0",
|
"typescript": "^5.9.0",
|
||||||
"vitest": "4.1.11",
|
"vitest": "4.1.11",
|
||||||
"wrangler": "^4.148.0"
|
"wrangler": "^4.148.0"
|
||||||
|
},
|
||||||
|
"dependencies": {
|
||||||
|
"capnweb": "^0.12.0"
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -0,0 +1,128 @@
|
|||||||
|
/**
|
||||||
|
* @file the Cap'n Web API at /api: trade a cache's webkey for a capability.
|
||||||
|
*
|
||||||
|
* The browser holds the key in a URL fragment (`/cache/#<key>`, see
|
||||||
|
* docs/webkeys.md), so it never reaches the server in a URL. The skeleton
|
||||||
|
* page passes it *in-band* — `api.cache(key)` — and gets back a stub for a
|
||||||
|
* {@link CacheCap}: a facet exposing only what the holder should do (fill,
|
||||||
|
* status, the public companion), never the DO's wiring methods.
|
||||||
|
* Authorization is in-band, so cross-origin calls are safe (capnweb's
|
||||||
|
* note on `newWorkersRpcResponse`); every argument is checked at runtime,
|
||||||
|
* since TypeScript types are erased on the wire.
|
||||||
|
*
|
||||||
|
* @see https://github.com/cloudflare/capnweb
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { RpcTarget } from "capnweb";
|
||||||
|
import type { DiigoCache, FillStatus, PublicCache } from "./cache.js";
|
||||||
|
import { idFromWebkey, isWebkey, shashKey } from "./webkeys.js";
|
||||||
|
|
||||||
|
/** the bindings the API reaches (structural) */
|
||||||
|
export type ApiEnv = {
|
||||||
|
CACHE_DO: DurableObjectNamespace<DiigoCache>;
|
||||||
|
PUBLIC_DO: DurableObjectNamespace<PublicCache>;
|
||||||
|
};
|
||||||
|
|
||||||
|
/** the fill-job surface of one cache (private or public) */
|
||||||
|
type Fillable = {
|
||||||
|
startFill(opts?: { pagesPerStep?: number }): Promise<FillStatus>;
|
||||||
|
fillStatus(): Promise<FillStatus | undefined>;
|
||||||
|
stopFill(): Promise<FillStatus>;
|
||||||
|
};
|
||||||
|
|
||||||
|
/** a cache's fill job, and nothing else */
|
||||||
|
export class FillCap extends RpcTarget {
|
||||||
|
#cache: Fillable;
|
||||||
|
|
||||||
|
constructor(cache: Fillable) {
|
||||||
|
super();
|
||||||
|
this.#cache = cache;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** start (or rejoin) the job; `pagesPerStep` 1–50 */
|
||||||
|
async startFill(opts: unknown = {}): Promise<FillStatus> {
|
||||||
|
const step = (opts as { pagesPerStep?: unknown } | null)?.pagesPerStep;
|
||||||
|
if (step !== undefined && typeof step !== "number") throw new Error("pagesPerStep: expected a number");
|
||||||
|
return this.#cache.startFill(step === undefined ? {} : { pagesPerStep: step });
|
||||||
|
}
|
||||||
|
|
||||||
|
/** the job's progress, or undefined if none was started */
|
||||||
|
async fillStatus(): Promise<FillStatus | undefined> {
|
||||||
|
return this.#cache.fillStatus();
|
||||||
|
}
|
||||||
|
|
||||||
|
/** stop the job; cached pages stay */
|
||||||
|
async stopFill(): Promise<FillStatus> {
|
||||||
|
return this.#cache.stopFill();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/** what the holder of a private cache's key may do */
|
||||||
|
export class CacheCap extends FillCap {
|
||||||
|
#key: string;
|
||||||
|
#publicId: string | undefined;
|
||||||
|
#env: ApiEnv;
|
||||||
|
|
||||||
|
constructor(env: ApiEnv, key: string, cache: Fillable, publicId: string | undefined) {
|
||||||
|
super(cache);
|
||||||
|
this.#env = env;
|
||||||
|
this.#key = key;
|
||||||
|
this.#publicId = publicId;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** a prefix of the key's one-way hash: names it without granting it */
|
||||||
|
async fingerprint(): Promise<string> {
|
||||||
|
return (await shashKey(this.#key)).slice(0, 8);
|
||||||
|
}
|
||||||
|
|
||||||
|
/** whether a public companion exists (see {@link publicCache}) */
|
||||||
|
hasPublic(): boolean {
|
||||||
|
return this.#publicId !== undefined;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** the public companion's fill job; refused if never published (a
|
||||||
|
* stub, not a maybe-null, so calls on it pipeline) */
|
||||||
|
publicCache(): FillCap {
|
||||||
|
if (!this.#publicId) throw new Error("no public cache: publish this cache first");
|
||||||
|
const ns = this.#env.PUBLIC_DO;
|
||||||
|
return new FillCap(ns.get(ns.idFromString(idFromWebkey(this.#publicId))));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/** the main interface at /api */
|
||||||
|
export class CacheApi extends RpcTarget {
|
||||||
|
#env: ApiEnv;
|
||||||
|
|
||||||
|
constructor(env: ApiEnv) {
|
||||||
|
super();
|
||||||
|
this.#env = env;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* trade a cache's webkey for its capability. Refused keys get one
|
||||||
|
* message for malformed and unknown alike, which never echoes the key.
|
||||||
|
*/
|
||||||
|
async cache(key: unknown): Promise<CacheCap> {
|
||||||
|
const refused = () => new Error("no such cache");
|
||||||
|
if (typeof key !== "string" || !isWebkey(key, 256)) {
|
||||||
|
console.log({ event: "api", refused: "cache", reason: "malformed key" });
|
||||||
|
throw refused();
|
||||||
|
}
|
||||||
|
const ns = this.#env.CACHE_DO;
|
||||||
|
let stub;
|
||||||
|
try {
|
||||||
|
// a well-formed id from another namespace throws here
|
||||||
|
stub = ns.get(ns.idFromString(idFromWebkey(key)));
|
||||||
|
} catch (err) {
|
||||||
|
// the key is never logged: it is the bearer secret
|
||||||
|
console.log({ event: "api", refused: "cache", reason: String(err) });
|
||||||
|
throw refused();
|
||||||
|
}
|
||||||
|
const wiring = await stub.wiring();
|
||||||
|
if (!wiring) {
|
||||||
|
console.log({ event: "api", refused: "cache", reason: "not wired" });
|
||||||
|
throw refused();
|
||||||
|
}
|
||||||
|
return new CacheCap(this.#env, key, stub, wiring.publicId);
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,94 @@
|
|||||||
|
/**
|
||||||
|
* @file the webkey page at /cache/ — a skeleton, identical for every key
|
||||||
|
* (docs/webkeys.md): the key rides the URL fragment, which browsers never
|
||||||
|
* send to the server or in a Referer. The inline module reads it, trades
|
||||||
|
* it for a capability (`api.cache(key)`, Cap'n Web over /api, in-band),
|
||||||
|
* and drives the fill job. Each round trip is its own HTTP batch session:
|
||||||
|
* calls made before an await travel together.
|
||||||
|
*/
|
||||||
|
|
||||||
|
export const CACHE_PAGE_HTML = `<!doctype html>
|
||||||
|
<html><head><meta charset="utf-8"><meta name="referrer" content="no-referrer">
|
||||||
|
<title>diigo cache</title></head>
|
||||||
|
<body>
|
||||||
|
<h1>diigo cache <code id="fingerprint"></code></h1>
|
||||||
|
<p id="problem" hidden></p>
|
||||||
|
<section id="private" hidden>
|
||||||
|
<h2>private cache</h2>
|
||||||
|
<p id="private-status">no fill job yet</p>
|
||||||
|
<button type="button" data-target="private" data-action="start">fill</button>
|
||||||
|
<button type="button" data-target="private" data-action="stop">stop</button>
|
||||||
|
</section>
|
||||||
|
<section id="public" hidden>
|
||||||
|
<h2>public cache</h2>
|
||||||
|
<p id="public-status">no fill job yet</p>
|
||||||
|
<button type="button" data-target="public" data-action="start">fill</button>
|
||||||
|
<button type="button" data-target="public" data-action="stop">stop</button>
|
||||||
|
</section>
|
||||||
|
<p><a href="/">← home</a></p>
|
||||||
|
<script type="module">
|
||||||
|
import { newHttpBatchRpcSession } from "/_capnweb.js";
|
||||||
|
|
||||||
|
const key = location.hash.slice(1);
|
||||||
|
const $ = (id) => document.getElementById(id);
|
||||||
|
const problem = (text) => {
|
||||||
|
$("problem").textContent = text;
|
||||||
|
$("problem").hidden = false;
|
||||||
|
};
|
||||||
|
/** a fresh batch session, traded for this key's capability */
|
||||||
|
const cap = () => newHttpBatchRpcSession("/api").cache(key);
|
||||||
|
/** the fill-job stub for a target */
|
||||||
|
const job = (target) => (target === "public" ? cap().publicCache() : cap());
|
||||||
|
|
||||||
|
const describe = (s) => {
|
||||||
|
if (!s) return "no fill job yet";
|
||||||
|
const of = Math.max(0, s.nextPage - 1) + " of " + (s.pages ?? "?") + " pages";
|
||||||
|
return {
|
||||||
|
running: "filling: " + of,
|
||||||
|
done: "filled: " + of,
|
||||||
|
stopped: "fill stopped: " + of,
|
||||||
|
failed: "fill failed at page " + s.nextPage + ": " + (s.error ?? "unknown error"),
|
||||||
|
}[s.state];
|
||||||
|
};
|
||||||
|
|
||||||
|
let polling = false;
|
||||||
|
async function refresh(hasPublic) {
|
||||||
|
const targets = hasPublic ? ["private", "public"] : ["private"];
|
||||||
|
const statuses = await Promise.all(targets.map((t) => job(t).fillStatus()));
|
||||||
|
targets.forEach((t, i) => ($(t + "-status").textContent = describe(statuses[i])));
|
||||||
|
const running = statuses.some((s) => s?.state === "running");
|
||||||
|
if (running && !polling) {
|
||||||
|
polling = true;
|
||||||
|
setTimeout(() => {
|
||||||
|
polling = false;
|
||||||
|
refresh(hasPublic).catch(report);
|
||||||
|
}, 3000);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
function report(err) {
|
||||||
|
console.error({ event: "cache-page", error: String(err) });
|
||||||
|
problem(String(err));
|
||||||
|
}
|
||||||
|
|
||||||
|
async function main() {
|
||||||
|
if (!key) return problem("no key: this page's address should end in #<cache id>");
|
||||||
|
const c = cap();
|
||||||
|
const [fingerprint, hasPublic] = await Promise.all([c.fingerprint(), c.hasPublic()]);
|
||||||
|
$("fingerprint").textContent = fingerprint;
|
||||||
|
$("private").hidden = false;
|
||||||
|
$("public").hidden = !hasPublic;
|
||||||
|
for (const button of document.querySelectorAll("button[data-target]")) {
|
||||||
|
button.addEventListener("click", () => {
|
||||||
|
const { target, action } = button.dataset;
|
||||||
|
const j = job(target);
|
||||||
|
(action === "start" ? j.startFill() : j.stopFill())
|
||||||
|
.then(() => refresh(hasPublic))
|
||||||
|
.catch(report);
|
||||||
|
});
|
||||||
|
}
|
||||||
|
await refresh(hasPublic);
|
||||||
|
}
|
||||||
|
main().catch(report);
|
||||||
|
</script>
|
||||||
|
</body></html>`;
|
||||||
@@ -0,0 +1,10 @@
|
|||||||
|
/**
|
||||||
|
* @file capnweb's browser build (a self-contained ES module), as text, for
|
||||||
|
* the worker to serve at /_capnweb.js; wrangler's Text rule for this path
|
||||||
|
* makes the import a string. Self-hosted: no third-party script on a page
|
||||||
|
* that holds a bearer key.
|
||||||
|
*/
|
||||||
|
// @ts-expect-error: a Text-rule import (wrangler.jsonc "rules"), not code
|
||||||
|
import source from "../node_modules/capnweb/dist/index.js";
|
||||||
|
|
||||||
|
export const CAPNWEB_BROWSER_JS: string = source;
|
||||||
+24
-71
@@ -13,6 +13,10 @@
|
|||||||
*/
|
*/
|
||||||
|
|
||||||
import { WorkerEntrypoint } from "cloudflare:workers";
|
import { WorkerEntrypoint } from "cloudflare:workers";
|
||||||
|
import { newWorkersRpcResponse } from "capnweb";
|
||||||
|
import { CacheApi } from "./api.js";
|
||||||
|
import { CAPNWEB_BROWSER_JS } from "./capnweb-browser.js";
|
||||||
|
import { CACHE_PAGE_HTML } from "./cache-page.js";
|
||||||
import { DiigoCache, PublicCache } from "./cache.js";
|
import { DiigoCache, PublicCache } from "./cache.js";
|
||||||
import type { FillStatus } from "./cache.js";
|
import type { FillStatus } from "./cache.js";
|
||||||
import type { Item } from "./cache.js";
|
import type { Item } from "./cache.js";
|
||||||
@@ -96,9 +100,17 @@ export default {
|
|||||||
return publishCache(request, env);
|
return publishCache(request, env);
|
||||||
}
|
}
|
||||||
|
|
||||||
// the fill job (goal 1e): owner-only, cache id in the body
|
// a cache's webkey page: the key rides the fragment (never sent
|
||||||
if (url.pathname === "/_fill" && request.method === "POST") {
|
// here); the page trades it for a capability over /api
|
||||||
return fillRoute(request, env);
|
if (url.pathname === "/cache/" && request.method === "GET") {
|
||||||
|
return new Response(CACHE_PAGE_HTML, { headers: { "content-type": "text/html;charset=utf-8" } });
|
||||||
|
}
|
||||||
|
if (url.pathname === "/_capnweb.js" && request.method === "GET") {
|
||||||
|
return new Response(CAPNWEB_BROWSER_JS, { headers: { "content-type": "text/javascript;charset=utf-8" } });
|
||||||
|
}
|
||||||
|
// Cap'n Web: in-band authorization (CacheApi.cache(key))
|
||||||
|
if (url.pathname === "/api") {
|
||||||
|
return newWorkersRpcResponse(request, new CacheApi(env));
|
||||||
}
|
}
|
||||||
|
|
||||||
if (url.pathname !== "/") return new Response("not found", { status: 404 });
|
if (url.pathname !== "/") return new Response("not found", { status: 404 });
|
||||||
@@ -108,9 +120,9 @@ export default {
|
|||||||
}
|
}
|
||||||
|
|
||||||
// GET / is inert (dumps happen on POST submits). The cache id is a
|
// GET / is inert (dumps happen on POST submits). The cache id is a
|
||||||
// webkey: it never rides the URL (query strings leak into history/
|
// webkey: it never rides a query string or path (they leak into
|
||||||
// Referer/logs — see docs/webkeys.md); the wired form POST is how
|
// history/Referer/logs — see docs/webkeys.md); the form POST is how
|
||||||
// it travels.
|
// it travels here.
|
||||||
return home(env, { error: "GET is inert: dumps happen on POST submits" }, null);
|
return home(env, { error: "GET is inert: dumps happen on POST submits" }, null);
|
||||||
},
|
},
|
||||||
};
|
};
|
||||||
@@ -293,18 +305,6 @@ async function wire(request: Request, env: Env): Promise<Response> {
|
|||||||
);
|
);
|
||||||
}
|
}
|
||||||
|
|
||||||
/** the fill job's buttons: one per cache the holder can fill (the id
|
|
||||||
* rides a hidden field, never the URL) */
|
|
||||||
function fillForms(cacheId: string, hasPublic: boolean): string {
|
|
||||||
const form = (target: string, label: string) => `<form method="post" action="/_fill">
|
|
||||||
<input type="hidden" name="cacheId" value="${esc(cacheId)}">
|
|
||||||
<input type="hidden" name="target" value="${target}">
|
|
||||||
<button type="submit">${label}</button></form>`;
|
|
||||||
return `<p>fetch every page now, in the background (a few minutes for thousands of bookmarks):</p>
|
|
||||||
${form("private", "fill the private cache")}
|
|
||||||
${hasPublic ? form("public", "fill the public cache") : ""}`;
|
|
||||||
}
|
|
||||||
|
|
||||||
/** the created-cache page: ids + save/copy buttons + the pointer home */
|
/** the created-cache page: ids + save/copy buttons + the pointer home */
|
||||||
async function wiredPage(cacheId: string, note: string, hasPublic: boolean): Promise<Response> {
|
async function wiredPage(cacheId: string, note: string, hasPublic: boolean): Promise<Response> {
|
||||||
const fingerprint = await nick(cacheId);
|
const fingerprint = await nick(cacheId);
|
||||||
@@ -319,11 +319,13 @@ async function wiredPage(cacheId: string, note: string, hasPublic: boolean): Pro
|
|||||||
<script>${PETNAME_TIME_JS}</script>
|
<script>${PETNAME_TIME_JS}</script>
|
||||||
<p><button type="button" onclick='${pmSaveClick(cacheId)}'>save to password manager</button></p>
|
<p><button type="button" onclick='${pmSaveClick(cacheId)}'>save to password manager</button></p>
|
||||||
<p>${note}</p>
|
<p>${note}</p>
|
||||||
${fillForms(cacheId, hasPublic)}
|
<p><a href="/cache/#${esc(cacheId)}">open this cache</a> — its <b>webkey</b>
|
||||||
|
(bookmark it, or save the id above): fill it, watch progress${hasPublic ? ", fill its public companion" : ""}.</p>
|
||||||
<p>the cache id is the only way back to this cache — saved (above) it lives
|
<p>the cache id is the only way back to this cache — saved (above) it lives
|
||||||
with the rest of your credentials. it is a <b>webkey</b>: it never travels
|
with the rest of your credentials. it is a <b>webkey</b>: it never rides a
|
||||||
in URLs (query strings leak to history/Referer/logs); the dump POST carries
|
query string or path (those leak to history/Referer/logs). the link above
|
||||||
it in its body instead. the webapp id rides inside the cache's wiring;
|
carries it in the fragment, which browsers never send; the dump POST
|
||||||
|
carries it in its body. the webapp id rides inside the cache's wiring;
|
||||||
hold it separately only if you want another cache off the same session.</p>
|
hold it separately only if you want another cache off the same session.</p>
|
||||||
<a href="/">← home (paste the cache id into the dump form)</a>
|
<a href="/">← home (paste the cache id into the dump form)</a>
|
||||||
<script>${PM_SAVE_JS}</script>
|
<script>${PM_SAVE_JS}</script>
|
||||||
@@ -343,55 +345,6 @@ function fillLine(fill: FillStatus): string {
|
|||||||
return `<p>${esc(said)}</p>`;
|
return `<p>${esc(said)}</p>`;
|
||||||
}
|
}
|
||||||
|
|
||||||
/** POST /_fill: start or stop the fill job of the holder's private cache
|
|
||||||
* (target=private, the default) or of its public companion
|
|
||||||
* (target=public). JSON in → JSON out; a form post gets a small page. */
|
|
||||||
async function fillRoute(request: Request, env: Env): Promise<Response> {
|
|
||||||
const ct = request.headers.get("content-type") ?? "";
|
|
||||||
const wantsJson = ct.includes("application/json");
|
|
||||||
let fields: Record<string, string>;
|
|
||||||
try {
|
|
||||||
fields = wantsJson
|
|
||||||
? Object.fromEntries(Object.entries((await request.json()) as Record<string, unknown>).map(([k, v]) => [k, String(v ?? "")]))
|
|
||||||
: Object.fromEntries([...(await request.formData()).entries()].map(([k, v]) => [k, String(v)]));
|
|
||||||
} catch (err) {
|
|
||||||
return bad(`bad input: ${String(err)}`);
|
|
||||||
}
|
|
||||||
const { cacheId = "", target = "private", action = "start" } = fields;
|
|
||||||
if (!isWebkey(cacheId)) return bad("no cache id supplied (POST body: cacheId)");
|
|
||||||
if (target !== "private" && target !== "public") return bad("target: expected private or public");
|
|
||||||
if (action !== "start" && action !== "stop") return bad("action: expected start or stop");
|
|
||||||
|
|
||||||
const cache = env.CACHE_DO.get(env.CACHE_DO.idFromString(idFromWebkey(cacheId)));
|
|
||||||
const wiring = await cache.wiring();
|
|
||||||
if (!wiring) return Response.json({ error: "unknown cache id" }, { status: 404 });
|
|
||||||
let stub: { startFill(): Promise<FillStatus>; stopFill(): Promise<FillStatus> } = cache;
|
|
||||||
if (target === "public") {
|
|
||||||
if (!wiring.publicId) return bad("this cache has no public companion; publish it first");
|
|
||||||
stub = env.PUBLIC_DO.get(env.PUBLIC_DO.idFromString(idFromWebkey(wiring.publicId)));
|
|
||||||
}
|
|
||||||
let status: FillStatus;
|
|
||||||
try {
|
|
||||||
status = action === "start" ? await stub.startFill() : await stub.stopFill();
|
|
||||||
} catch (err) {
|
|
||||||
console.error({ event: "fill", target, action, error: String(err) });
|
|
||||||
return bad(`could not ${action} the fill: ${String(err)}`);
|
|
||||||
}
|
|
||||||
const code = action === "start" ? 202 : 200;
|
|
||||||
if (wantsJson) return Response.json(status, { status: code });
|
|
||||||
return new Response(
|
|
||||||
`<!doctype html>
|
|
||||||
<html><head><meta charset="utf-8"><title>diigo cache: fill</title></head>
|
|
||||||
<body>
|
|
||||||
<h1>fill (${esc(target)} cache)</h1>
|
|
||||||
${fillLine(status)}
|
|
||||||
<p>it runs in the background; the progress shows on the public page, or post this form again to check.</p>
|
|
||||||
<a href="/">← home</a>
|
|
||||||
</body></html>`,
|
|
||||||
{ status: code, headers: { "content-type": "text/html;charset=utf-8" } },
|
|
||||||
);
|
|
||||||
}
|
|
||||||
|
|
||||||
/** publish, later: the private cache's wiring already holds the webapp
|
/** publish, later: the private cache's wiring already holds the webapp
|
||||||
* id, so the public companion needs no re-mint; the username comes from
|
* id, so the public companion needs no re-mint; the username comes from
|
||||||
* the items already in the cache */
|
* the items already in the cache */
|
||||||
|
|||||||
+117
-32
@@ -2,6 +2,8 @@ import { env } from "cloudflare:workers";
|
|||||||
import { evictDurableObject, runDurableObjectAlarm, runInDurableObject } from "cloudflare:test";
|
import { evictDurableObject, runDurableObjectAlarm, runInDurableObject } from "cloudflare:test";
|
||||||
import { describe, expect, it, vi } from "vitest";
|
import { describe, expect, it, vi } from "vitest";
|
||||||
import worker from "../src/index.js";
|
import worker from "../src/index.js";
|
||||||
|
import { newHttpBatchRpcSession } from "capnweb";
|
||||||
|
import type { CacheApi } from "../src/api.js";
|
||||||
import { toWebkey } from "../src/webkeys.js";
|
import { toWebkey } from "../src/webkeys.js";
|
||||||
|
|
||||||
// goal 1e (capsite checklist): a job that grabs every page. Fake
|
// goal 1e (capsite checklist): a job that grabs every page. Fake
|
||||||
@@ -117,56 +119,139 @@ describe("fill job (goal 1e)", () => {
|
|||||||
});
|
});
|
||||||
});
|
});
|
||||||
|
|
||||||
describe("fill routes (goal 1e)", () => {
|
describe("fill via webkey and Cap'n Web (goal 1e)", () => {
|
||||||
const call = (path: string, init?: RequestInit) =>
|
const ORIGIN = "https://diigo-cache.test";
|
||||||
worker.fetch(new Request(`https://diigo-cache.test${path}`, init), env as never);
|
const call = (path: string, init?: RequestInit) => worker.fetch(new Request(`${ORIGIN}${path}`, init), env as never);
|
||||||
const post = (body: Record<string, string>) =>
|
|
||||||
call("/_fill", { method: "POST", headers: { "content-type": "application/json" }, body: JSON.stringify(body) });
|
|
||||||
|
|
||||||
it("POST /_fill starts the owner's job (cache id in the body) and answers 202 with its status", async () => {
|
/** a Cap'n Web batch session whose fetch goes to the worker; counts round trips */
|
||||||
const id = testEnv.CACHE_DO.newUniqueId();
|
const session = () => {
|
||||||
await testEnv.CACHE_DO.get(id).setup({ webappId: "fake-webapp-id" });
|
const trips = { n: 0 };
|
||||||
const res = await post({ cacheId: toWebkey(id) });
|
const fetchSpy = vi.spyOn(globalThis, "fetch").mockImplementation((input, init) => {
|
||||||
expect(res.status).toBe(202);
|
trips.n += 1;
|
||||||
expect(await res.json()).toMatchObject({ state: "running" });
|
return worker.fetch(new Request(input as RequestInfo, init), env as never);
|
||||||
});
|
});
|
||||||
|
const api = newHttpBatchRpcSession<CacheApi>(`${ORIGIN}/api`);
|
||||||
|
return { api, trips, done: () => fetchSpy.mockRestore() };
|
||||||
|
};
|
||||||
|
|
||||||
it("target=public fills the cache's public companion; action=stop stops", async () => {
|
const wireJson = async (mode: "private" | "public") => {
|
||||||
const wiredRes = await call("/_wire", {
|
const res = await call("/_wire", {
|
||||||
method: "POST",
|
method: "POST",
|
||||||
headers: { "content-type": "application/json" },
|
headers: { "content-type": "application/json" },
|
||||||
body: JSON.stringify({ webappId: "fake-webapp-id", mode: "public" }),
|
body: JSON.stringify({ webappId: "fake-webapp-id", mode }),
|
||||||
});
|
});
|
||||||
const { cacheId } = (await wiredRes.json()) as { cacheId: string };
|
return (await res.json()) as { cacheId: string; username?: string };
|
||||||
expect((await post({ cacheId, target: "public" })).status).toBe(202);
|
};
|
||||||
const stopped = await post({ cacheId, target: "public", action: "stop" });
|
|
||||||
expect(await stopped.json()).toMatchObject({ state: "stopped" });
|
it("trading the key and starting the fill pipeline into one round trip", async () => {
|
||||||
|
const { cacheId } = await wireJson("private");
|
||||||
|
const { api, trips, done } = session();
|
||||||
|
try {
|
||||||
|
const status = await api.cache(cacheId).startFill();
|
||||||
|
expect(status).toMatchObject({ state: "running" });
|
||||||
|
expect(trips.n).toBe(1);
|
||||||
|
} finally {
|
||||||
|
done();
|
||||||
|
}
|
||||||
});
|
});
|
||||||
|
|
||||||
it("refuses an unknown cache id (404) and never takes the id from the URL", async () => {
|
it("the capability is a facet: fill and status, never the DO's wiring", async () => {
|
||||||
expect((await post({ cacheId: toWebkey(testEnv.CACHE_DO.newUniqueId()) })).status).toBe(404);
|
const { cacheId } = await wireJson("private");
|
||||||
expect((await call(`/_fill/${toWebkey(testEnv.CACHE_DO.newUniqueId())}`, { method: "POST" })).status).toBe(404);
|
const { api, done } = session();
|
||||||
|
try {
|
||||||
|
const cap = api.cache(cacheId) as unknown as Record<string, () => Promise<unknown>>;
|
||||||
|
await expect(cap.setup()).rejects.toThrow();
|
||||||
|
} finally {
|
||||||
|
done();
|
||||||
|
}
|
||||||
});
|
});
|
||||||
|
|
||||||
it("the created page offers a fetch-everything button, the id in a hidden field", async () => {
|
it("unknown and malformed keys get one refusal that never echoes the key", async () => {
|
||||||
|
const unknown = toWebkey(testEnv.CACHE_DO.newUniqueId());
|
||||||
|
for (const key of [unknown, "not-a-key", unknown.slice(0, 26)]) {
|
||||||
|
const { api, done } = session();
|
||||||
|
try {
|
||||||
|
const err = await api.cache(key).fillStatus().then(() => null, (e: unknown) => e);
|
||||||
|
expect(String(err)).toMatch(/no such cache/);
|
||||||
|
expect(String(err)).not.toContain(key);
|
||||||
|
} finally {
|
||||||
|
done();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
it("publicCache() reaches the public companion's fill; refused when never published", async () => {
|
||||||
|
const pub = await wireJson("public");
|
||||||
|
const s1 = session();
|
||||||
|
try {
|
||||||
|
expect(await s1.api.cache(pub.cacheId).hasPublic()).toBe(true);
|
||||||
|
} finally {
|
||||||
|
s1.done();
|
||||||
|
}
|
||||||
|
const s2 = session();
|
||||||
|
try {
|
||||||
|
expect(await s2.api.cache(pub.cacheId).publicCache().startFill()).toMatchObject({ state: "running" });
|
||||||
|
} finally {
|
||||||
|
s2.done();
|
||||||
|
}
|
||||||
|
const s3 = session();
|
||||||
|
try {
|
||||||
|
expect(await s3.api.cache(pub.cacheId).publicCache().stopFill()).toMatchObject({ state: "stopped" });
|
||||||
|
} finally {
|
||||||
|
s3.done();
|
||||||
|
}
|
||||||
|
const priv = await wireJson("private");
|
||||||
|
const s4 = session();
|
||||||
|
try {
|
||||||
|
expect(await s4.api.cache(priv.cacheId).hasPublic()).toBe(false);
|
||||||
|
} finally {
|
||||||
|
s4.done();
|
||||||
|
}
|
||||||
|
// a batch session ends after its round trip: a fresh one per await
|
||||||
|
const s5 = session();
|
||||||
|
try {
|
||||||
|
await expect(s5.api.cache(priv.cacheId).publicCache().fillStatus()).rejects.toThrow(/no public cache/);
|
||||||
|
} finally {
|
||||||
|
s5.done();
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
it("the webkey page is a skeleton: the same for every key, which it reads from the fragment", async () => {
|
||||||
|
const res = await call("/cache/");
|
||||||
|
expect(res.status).toBe(200);
|
||||||
|
const html = await res.text();
|
||||||
|
expect(html).toContain("location.hash");
|
||||||
|
expect(html).toContain('from "/_capnweb.js"');
|
||||||
|
expect(html).toContain('newHttpBatchRpcSession("/api")');
|
||||||
|
expect(html).toContain('<meta name="referrer" content="no-referrer">');
|
||||||
|
});
|
||||||
|
|
||||||
|
it("serves capnweb's browser build itself (no third-party script beside the key)", async () => {
|
||||||
|
const res = await call("/_capnweb.js");
|
||||||
|
expect(res.headers.get("content-type")).toMatch(/javascript/);
|
||||||
|
expect(await res.text()).toMatch(/export \{[^}]*newHttpBatchRpcSession/);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("the created page links its webkey (key in the fragment); no cache id rides a form to /_fill", async () => {
|
||||||
const res = await call("/_wire", {
|
const res = await call("/_wire", {
|
||||||
method: "POST",
|
method: "POST",
|
||||||
headers: { "content-type": "application/x-www-form-urlencoded" },
|
headers: { "content-type": "application/x-www-form-urlencoded" },
|
||||||
body: new URLSearchParams({ webappId: "fake-webapp-id", mode: "private" }),
|
body: new URLSearchParams({ webappId: "fake-webapp-id", mode: "private" }),
|
||||||
});
|
});
|
||||||
const html = await res.text();
|
const html = await res.text();
|
||||||
expect(html).toMatch(/<form method="post" action="\/_fill">/);
|
expect(html).toMatch(/<a href="\/cache\/#[a-z2-7]{52}">/);
|
||||||
expect(html).toMatch(/<input type="hidden" name="cacheId"/);
|
expect(html).not.toContain("/_fill");
|
||||||
|
expect((await call("/_fill", { method: "POST" })).status).toBe(404);
|
||||||
});
|
});
|
||||||
|
|
||||||
it("the public page shows a running fill's progress", async () => {
|
it("the public page shows a running fill's progress", async () => {
|
||||||
const wiredRes = await call("/_wire", {
|
const { cacheId, username } = await wireJson("public");
|
||||||
method: "POST",
|
const { api, done } = session();
|
||||||
headers: { "content-type": "application/json" },
|
try {
|
||||||
body: JSON.stringify({ webappId: "fake-webapp-id", mode: "public" }),
|
await api.cache(cacheId).publicCache().startFill();
|
||||||
});
|
} finally {
|
||||||
const { cacheId, username } = (await wiredRes.json()) as { cacheId: string; username: string };
|
done();
|
||||||
await post({ cacheId, target: "public" });
|
}
|
||||||
const html = await (await call(`/public/${username}`)).text();
|
const html = await (await call(`/public/${username}`)).text();
|
||||||
expect(html).toMatch(/fill(ed|ing)[^<]*\d+ of \d+ pages/);
|
expect(html).toMatch(/fill(ed|ing)[^<]*\d+ of \d+ pages/);
|
||||||
});
|
});
|
||||||
|
|||||||
@@ -4,6 +4,9 @@
|
|||||||
"main": "src/index.ts",
|
"main": "src/index.ts",
|
||||||
"compatibility_date": "2026-09-29",
|
"compatibility_date": "2026-09-29",
|
||||||
"observability": { "enabled": true },
|
"observability": { "enabled": true },
|
||||||
|
// capnweb's browser build, served as text at /_capnweb.js (the worker
|
||||||
|
// itself imports capnweb's workerd build as code)
|
||||||
|
"rules": [{ "type": "Text", "globs": ["**/capnweb/dist/index.js"], "fallthrough": true }],
|
||||||
"services": [
|
"services": [
|
||||||
{
|
{
|
||||||
"binding": "WEBAPP",
|
"binding": "WEBAPP",
|
||||||
|
|||||||
@@ -5,6 +5,7 @@
|
|||||||
"name": "diigo-cache-tests",
|
"name": "diigo-cache-tests",
|
||||||
"main": "./src/index.ts",
|
"main": "./src/index.ts",
|
||||||
"compatibility_date": "2026-09-29",
|
"compatibility_date": "2026-09-29",
|
||||||
|
"rules": [{ "type": "Text", "globs": ["**/capnweb/dist/index.js"], "fallthrough": true }],
|
||||||
"services": [
|
"services": [
|
||||||
{ "binding": "WEBAPP", "service": "fake-webapp", "entrypoint": "WebAppFactory" }
|
{ "binding": "WEBAPP", "service": "fake-webapp", "entrypoint": "WebAppFactory" }
|
||||||
],
|
],
|
||||||
|
|||||||
Reference in new issue
Block a user