dckc/captoys
1
0 Fork 0
Code Issues Pull Requests 4 Actions Packages Projects Releases Wiki Activity

feat: webapp DO factory worker; diigo-dump as its consumer

Browse Source 
webapp/ (new, site-agnostic, independently deployable):
- WebApp: fluent client (pathjoin/query/withHeaders/cookies/get/post)
  as RpcTarget; flower-style unions cite NuxiNL/flower v0.11 (Jan 2019);
  url/headers private (RpcTarget property reads are RPC); Error messages
  carry no header/param values; query-stripped URLs only
- WebAppDo: per-instance authority in DO storage (survives eviction);
  one-shot configure(), refuses different-url re-configure
- WebAppFactory: newWebApp(init) -> {id, app}, at(id); ids are bearer
  capabilities and are not logged
- HTTP surface: GET / mint form (secret header keys render blank),
  POST / mint (form or JSON), POST /at/<id> drive-by-capability
- types via wrangler types (worker-configuration.d.ts, gitignored)

diigo-dump/: now a webapp consumer. The diigo knowledge is only the
mint call: url=interact_api/, UA + x-requested-with, cookie, 500ms gate.
- src/webapp.ts, src/diigo-web.ts deleted (webapp's are canonical);
  diigo.ts slimmed to dumpBookmarks + PagedGet
- /_connect is state-aware: connect form pre-mint; health check
  (count=1 probe) + re-mint path post-mint; home-page link relabeled
- mint success page: narrow field + copy button (attribute-quoting
  fixed; id in x-diigo-web-id header for scripts)
- scripts/diigo-login: askpass flow, marked OBSOLETE (page flow is
  the primary); speaks the form protocol; writes only DIIGO_WEB_ID —
  the cookie lives in DO storage, never on disk
- package.json: dev = wrangler dev -c wrangler.jsonc -c ../webapp/...
  (verified: Ready on :8787, GET / 200, /_connect 200);
  wrangler ^4.148.0 devDependency after node_modules was cleaned

TODO(webapp) in webapp/README.md: proxy-page vs primary-worker mint
form; reconsider as the UI grows.

Generated-by: opencode (GLM)
This commit is contained in:
Connolly committed 2026-10-06 23:30:27 -05:00
1 parent 309f003fa3
commit 3f8e77eaa2
14 files changed
+1650 -329
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

diigo-dump/src/index.ts
+187 -65
View File
@@ -1,46 +1,62 @@
/**
* @file diigo-dump worker: one page-number field + button, plus the
* {@link DiigoService} RPC entrypoint. Browser-session-cookie access to
* diigo's interact_api/load_user_items — the API-key path is bkmkeep.py /
* the EmDash plugin; this one is the cookie path.
* @file diigo-dump worker: cookie access to diigo's
* interact_api/load_user_items via a WebApp Durable Object instance from
* the webapp worker. The id is a bearer capability held in DIIGO_WEB_ID;
* the cookie lives only inside that DO's storage.
*
* Secret: DIIGO_COOKIE (copy the "Cookie" header from diigo.com devtools).
* The fluent WebApp client lives in ./webapp.ts and ./diigo.ts; the cookie
* is attached only here, inside this worker's address space.
* Failures reach both surfaces: console.error for observability (wrapped
* workers never see exceptions the fetch handler renders itself), plus the
* rendered status/detail on the page.
* @see {@link DiigoService}
* Connect flow (no askpass, no cookie-on-disk): the home page links to
* /_connect, which proxies webapp's mint form with the diigo prefills
* (cookie field blank) — paste the cookie, mint lands via the service
* binding, and the page shows the DIIGO_WEB_ID to put in .dev.vars.
* (scripts/diigo-login automates the same flow for terminals.)
* @see {@link DiigoBridge}
*/
import { WorkerEntrypoint } from "cloudflare:workers";
import { makeDiigoAPI, dumpBookmarks, rateLimit } from "./diigo.js";
import { dumpBookmarks, type PagedGet } from "./diigo.js";
type Env = { DIIGO_COOKIE?: string };
/** the WEBAPP service binding to webapp/src/index.ts:WebAppFactory; typed
* structurally here pending a shared types package */
type WebAppFactoryShape = {
newWebApp(init: {
url: string;
headers?: Record<string, string>;
msPerCall?: number;
}): Promise<{ id: string; app: PagedGet }>;
at(id: string): Promise<PagedGet>;
};
const sleep = (ms: number) => new Promise<void>((r) => setTimeout(r, ms));
type Env = {
DIIGO_WEB_ID?: string;
WEBAPP: WebAppFactoryShape;
};
/**
* RPC entrypoint for other workers on the account. Public methods shape the
* ABI: `open()` hands out a rate-limited stub with the cookie attached
* server-side (the caller can pathjoin/query/get but never sees the cookie
* or fetch authority); `dumpNdjson` runs a whole dump here and returns ndjson.
*
* @see https://developers.cloudflare.com/workers/runtime-apis/rpc/ — stub
* lifecycle; use `using` / `Symbol.dispose` on the caller side
*/
export class DiigoService extends WorkerEntrypoint {
open(): ReturnType<typeof rateLimit> {
const { DIIGO_COOKIE: cookie } = this.env as Env;
if (!cookie) {
console.error({ event: "open", error: "Missing DIIGO_COOKIE in the diigo-dump worker." });
throw new Error("Missing DIIGO_COOKIE in the diigo-dump worker.");
}
return rateLimit(makeDiigoAPI().cookies(cookie), () => sleep(500));
const UA = "Mozilla/5.0 (X11; Linux x86_64) Gecko/20100101 Firefox/141.0";
const DIIGO_INTERACT = "https://www.diigo.com/interact_api/";
/** prefill for webapp's mint form: everything but the cookie */
const DIIGO_PREFILL = JSON.stringify({
"user-agent": UA,
"x-requested-with": "XMLHttpRequest",
});
/** connect to the configured instance, or give the setup pointer */
async function instance(env: Env): Promise<PagedGet> {
const id = env.DIIGO_WEB_ID;
if (!id) {
throw new Error("no instance: use the connect flow on the home page, then set DIIGO_WEB_ID");
}
return env.WEBAPP.at(id);
}
export class DiigoBridge extends WorkerEntrypoint {
/** the diigo-attenuated handle: fluent chain over the instance */
open(): Promise<PagedGet> {
return instance(this.env as unknown as Env);
}
/** bulk dump from the caller's address space */
async dumpNdjson(pages: number, count = 24): Promise<string> {
const diigoAPI = this.open();
const diigoAPI = await instance(this.env as unknown as Env);
const lines: string[] = [];
for await (const item of dumpBookmarks({ diigoAPI, pages, count })) {
lines.push(JSON.stringify(item));
@@ -52,60 +68,164 @@ export class DiigoService extends WorkerEntrypoint {
export default {
async fetch(request: Request, env: Env): Promise<Response> {
const url = new URL(request.url);
// the connect flow: proxy webapp's mint form (prefilled, cookie blank)
if (url.pathname === "/_connect") {
return env.DIIGO_WEB_ID ? connectedPage(env) : connectPage();
}
// webapp's form POSTs back through us: forward to the binding
if (url.pathname === "/_mint" && request.method === "POST") {
return proxyMint(request, env);
}
if (url.pathname !== "/") return new Response("not found", { status: 404 });
if (request.method !== "POST") return html(null);
if (request.method !== "POST") return home(null, undefined, Boolean(env.DIIGO_WEB_ID));
const form = await request.formData();
const raw = form.get("page");
const page = typeof raw === "string" ? Number.parseInt(raw, 10) : Number.NaN;
if (Number.isNaN(page) || page < 1 || page > 10000) {
return html(null, "enter a page number (1 = most recently updated)");
return home("enter a page number (1 = most recently updated)", undefined, Boolean(env.DIIGO_WEB_ID));
}
try {
const app = await instance(env);
const text = await app
.pathjoin("load_user_items")
.query({ sort: "updated", count: 24, page_num: page })
.get();
const data = JSON.parse(text) as { total?: number; items?: Record<string, unknown>[] };
return home(
`total: ${data.total}\nitems: ${data.items?.length ?? 0}\n\n${JSON.stringify(data.items ?? data, null, 2)}`,
undefined,
true,
);
} catch (err) {
console.error({ event: "dump", page, error: String(err) });
return home(null, String(err), Boolean(env.DIIGO_WEB_ID));
}
const result = await dumpPage(env, page);
return html(result.status, result.detail, String(page));
},
};
/** the HTTP form handler's dump path: one page via the same fluent chain an
* RPC caller would drive (see {@link DiigoService.open}); shows raw JSON. */
async function dumpPage(env: Env, page: number): Promise<{ status: string; detail: string }> {
const cookie = env.DIIGO_COOKIE;
if (!cookie) {
return {
status: "no-creds",
detail: "Missing DIIGO_COOKIE.\nSet it: npx wrangler secret put DIIGO_COOKIE (or .dev.vars locally).",
};
}
const diigoAPI = rateLimit(makeDiigoAPI().cookies(cookie), () => sleep(500));
/** the connect page: webapp's form with diigo defaults. The mint POST is
* proxied (/_mint) so one port serves the whole flow; the response
* page shows the id and the .dev.vars line to add. */
async function connectPage(): Promise<Response> {
// proxy: render our own form with action="/_mint" so one port serves
// the whole flow. TODO(webapp): one canonical mint form; consider
// serving it from webapp directly.
const body = `<!doctype html>
<html><head><meta charset="utf-8"><title>connect diigo session</title></head>
<body>
<h1>connect diigo session</h1>
<ol>
<li>copy the <code>Cookie</code> header from diigo.com devtools</li>
<li>paste below, mint, then add the <code>DIIGO_WEB_ID</code> line it
shows to <code>.dev.vars</code> (deployed: <code>wrangler secret put DIIGO_WEB_ID</code>)</li>
<li>restart wrangler, dump from the <a href="/">home page</a></li>
</ol>
<form method="post" action="/_mint">
<input type="hidden" name="url" value="${esc(DIIGO_INTERACT)}">
<input type="hidden" name="headers" value="${esc(JSON.stringify({ "user-agent": UA, "x-requested-with": "XMLHttpRequest", cookie: "" }))}">
<p><label>cookie <input type="password" name="cookie" size="80" autocomplete="off"></label></p>
<p><button type="submit">mint</button></p>
</form>
</body></html>`;
return new Response(body, { headers: { "content-type": "text/html;charset=utf-8" } });
}
async function connectedPage(env: Env): Promise<Response> {
let health = "";
try {
const text = await diigoAPI
const app = await instance(env);
// a cheap call: one item, exercises the cookie + rate gate
const text = await app
.pathjoin("load_user_items")
.query({ sort: "updated", count: 24, page_num: page })
.query({ sort: "updated", count: 1, page_num: 1 })
.get();
const data = JSON.parse(text) as { total?: number; items?: Record<string, unknown>[] };
return {
status: "200",
detail: `total: ${data.total}\nitems: ${data.items?.length ?? 0}\n\n${JSON.stringify(data.items ?? data, null, 2)}`,
};
const data = JSON.parse(text) as { total?: number };
health = `<p>instance is <b>live</b>: diigo reports <b>${esc(String(data.total ?? "?"))}</b> bookmarks.</p>`;
} catch (err) {
console.error({ event: "dump", page, error: String(err) });
return { status: "error", detail: String(err) };
console.error({ event: "connect-check", error: String(err) });
health = `<p>instance is configured but the last check <b>failed</b>:</p><pre>${esc(String(err))}</pre><p>If diigo expired the cookie, re-mint below.</p>`;
}
const body = `<!doctype html>
<html><head><meta charset="utf-8"><title>diigo session</title></head>
<body>
<h1>diigo session</h1>
<p>connected: id <code>${esc(`${env.DIIGO_WEB_ID!.slice(0, 6)}…${env.DIIGO_WEB_ID!.slice(-4)}`)}</code> (from .dev.vars).</p>
${health}
<h2>replace the session</h2>
<p>Only needed if the cookie is stale (diigo will say 401/403). Minting a
new instance invalidates nothing, but the id rotates and .dev.vars needs
the new <code>DIIGO_WEB_ID</code>.</p>
<form method="post" action="/_mint">
<input type="hidden" name="url" value="${esc(DIIGO_INTERACT)}">
<input type="hidden" name="headers" value="${esc(JSON.stringify({ "user-agent": UA, "x-requested-with": "XMLHttpRequest", cookie: "" }))}">
<p><label>cookie <input type="password" name="cookie" size="80" autocomplete="off"></label></p>
<p><button type="submit">mint</button></p>
</form>
</body></html>`;
return new Response(body, { headers: { "content-type": "text/html;charset=utf-8" } });
}
function html(status: string | null, detail?: string, page?: string): Response {
const form = `<form method="post" action="/"><label>page <input type="number" name="page" min="1" value="${page ?? 1}"></label> <button type="submit">dump</button></form>`;
const result =
status === null
? ""
: `<p>status: <b>${status}</b></p><pre readonly style="white-space:pre-wrap;border:1px solid #ccc;padding:1rem;max-height:60vh;overflow:auto">${escapeHtml(detail ?? "")}</pre>`;
/** proxy the mint: merge the pasted cookie into the headers JSON, mint via
* the service binding, show the id (plus the .dev.vars line to copy) */
async function proxyMint(request: Request, env: Env): Promise<Response> {
const form = await request.formData();
const cookie = String(form.get("cookie") || "");
if (!cookie) {
return new Response("no cookie supplied", { status: 400 });
}
const headers = {
"user-agent": UA,
"x-requested-with": "XMLHttpRequest",
cookie,
};
const { id } = await env.WEBAPP.newWebApp({
url: DIIGO_INTERACT,
headers,
msPerCall: 500,
});
return idPage(id);
}
/** the id shown on the success page: a narrow text field (scrollable but
* too small to show most of the value) with a copy button for the full
* version. navigator.clipboard needs the page served from localhost/https
* — true in dev and at the deployed URL */
function idPage(id: string): Response {
const body = `<!doctype html>
<html><head><meta charset="utf-8"><title>minted</title></head>
<body>
<h1>minted</h1>
<p>id: <input id="wid" readonly size="4" value="${esc(id)}" onfocus="this.select()">
<button type="button" onclick='navigator.clipboard.writeText(${JSON.stringify(id)}).then(() => this.textContent = "copied ✓")'>copy</button></p>
<p>add to <code>.dev.vars</code>:</p>
<pre>DIIGO_WEB_ID=</pre>
<p>…click copy, then paste the full value after it. <a href="/">back to diigo dump</a> (restart wrangler first).</p>
</body></html>`;
return new Response(body, {
headers: {
"content-type": "text/html;charset=utf-8",
// for scripts: diigo-login reads this instead of scraping HTML
"x-diigo-web-id": id,
},
});
}
function home(detail: string | null, error?: string, connected?: boolean): Response {
const form = `<form method="post" action="/"><label>page <input type="number" name="page" min="1"></label> <button type="submit">dump</button></form>`;
const result = error
? `<p>status: <b>error</b></p><pre readonly style="white-space:pre-wrap;border:1px solid #c00;padding:1rem">${escapeHtml(error)}</pre>`
: detail
? `<pre readonly style="white-space:pre-wrap;border:1px solid #ccc;padding:1rem;max-height:60vh;overflow:auto">${escapeHtml(detail)}</pre>`
: "";
return new Response(
`<!doctype html>
<html><head><meta charset="utf-8"><title>diigo dump</title></head>
<body>
<h1>diigo dump</h1>
<p><a href="/_connect">${connected ? "diigo session status/replace…" : "connect diigo session…"}</a></p>
${form}
${result}
</body></html>`,
@@ -113,10 +233,12 @@ ${result}
);
}
function escapeHtml(s: string): string {
function esc(s: string): string {
return s
.replaceAll("&", "&amp;")
.replaceAll("<", "&lt;")
.replaceAll(">", "&gt;")
.replaceAll('"', "&quot;");
}
}
const escapeHtml = esc;