docs: de-duplicate run instructions, READMEs track current code
diigo-dump/README was stale three ways: described the pre-DO connect flow (askpass + DIIGO_COOKIE, both retired), referenced deleted files (src/webapp.ts), and named the old entrypoint. Rewritten against current code (DiigoBridge, DO storage state surface, state-aware /_connect); run/connect/deploy instructions now live only in the capsite checklist, linked (intra-repo docs link rather than repeat). webapp/README Run section: port note corrected (two-config run makes diigo-dump the primary) and pointed at the checklist as the source. Generated-by: opencode (GLM)
This commit is contained in:
Connolly committed
1 parent
079415b961
commit
c587481761
2 files changed
+39
-82
No files matched your search
+35
-80
@@ -1,102 +1,57 @@
|
||||
# diigo-dump
|
||||
|
||||
Cloudflare Worker port of [`diigo-dump.mjs` in the mm-diigo repo](https://github.com/dckc/mm-diigo) (`projects/diigo-bak/diigo-dump.mjs`) — dump
|
||||
Diigo bookmarks, most-recently-updated first, via the browser session cookie
|
||||
rather than the static API key.
|
||||
Cloudflare Worker consumer of [the webapp factory](../webapp/README.md):
|
||||
the site-side of dumping [Diigo bookmarks from the mm-diigo provenance
|
||||
scripts](https://github.com/dckc/mm-diigo) (`projects/diigo-bak/`) —
|
||||
session-cookie access, most-recently-updated first. The static API-key
|
||||
path is dead (both known keys, all documented schemes → 401; see
|
||||
[the capsite checklist](../capsite-checklist.md), goal 2 preamble).
|
||||
|
||||
## What it does
|
||||
## What it holds
|
||||
|
||||
One page: a page-number field and a dump button. Submitting POSTs to the
|
||||
worker, which fetches
|
||||
**Only the diigo flavoring** — everything generic lives in webapp:
|
||||
|
||||
https://www.diigo.com/interact_api/load_user_items
|
||||
?sort=updated&count=24&page_num=<page>
|
||||
- the mint call: `url` = `https://www.diigo.com/interact_api/`, seed
|
||||
headers `{user-agent, x-requested-with}` (+ the cookie at mint time),
|
||||
`msPerCall: 500`
|
||||
- the dump page: page-number form → `WEBAPP.at(id)` → pipelined
|
||||
`pathjoin("load_user_items").query({sort, count, page_num}).get()`
|
||||
- the `DiigoBridge` entrypoint (`open()`, `dumpNdjson()`) for other
|
||||
workers; `dumpBookmarks` (the generator from mm-diigo's
|
||||
diigo-dump.mjs) is in `src/diigo.ts`
|
||||
|
||||
with the session cookie and shows status, `total`, and the JSON items in a
|
||||
read-only `<pre>`. GET `/` is inert — dumps happen only on explicit submits.
|
||||
GET `/` is inert; dumps happen only on POST submits.
|
||||
|
||||
## Structure: WebApp as a Durable Object (webapp worker)
|
||||
## State surface
|
||||
|
||||
The credentialed authority is a Durable Object instance from the separately
|
||||
deployable `../webapp/` worker (service binding `WEBAPP` → its
|
||||
`WebAppFactory` entrypoint; diigo knowledge = only the mint call in
|
||||
`src/index.ts`). The instance is configured once — diigo's URL + seed
|
||||
headers (cookie, UA, x-requested-with) + 500ms rate gate — and persists
|
||||
its init in DO storage: it survives restarts, and diigo-dump or any other
|
||||
worker can reconnect purely by id. The cookie crosses the wire exactly
|
||||
once, at mint time.
|
||||
|
||||
Setup (dev):
|
||||
|
||||
cd diigo-dump
|
||||
npx wrangler dev -c wrangler.jsonc -c ../webapp/wrangler.jsonc
|
||||
# → http://localhost:8787 (webapp is secondary, binding-only)
|
||||
|
||||
npm run diigo-login >> .dev.vars # askpass for the cookie, mints the
|
||||
# instance, writes DIIGO_COOKIE +
|
||||
# DIIGO_WEB_ID; then restart wrangler
|
||||
# or, no askpass: open http://localhost:8787/_connect and paste the
|
||||
# cookie into the mint form; it prints the DIIGO_WEB_ID line to add
|
||||
|
||||
Deployed:
|
||||
|
||||
cd ../webapp && npx wrangler deploy
|
||||
cd ../diigo-dump && npx wrangler deploy
|
||||
npx wrangler secret put DIIGO_COOKIE # re-run diigo-login only when
|
||||
npx wrangler secret put DIIGO_WEB_ID # the cookie itself expires
|
||||
|
||||
Re-login is needed only when diigo invalidates the cookie; re-minting
|
||||
(POST /_mint with the new cookie) yields a fresh id and no other changes.
|
||||
|
||||
Provenance notes: the `user-agent` / `x-requested-with` headers and the
|
||||
`interact_api` endpoint mirror the original script; the fluent client from
|
||||
`WebApp.js` is restored in `src/webapp.ts` with `WebApp` and the rate-limit
|
||||
wrapper as `RpcTarget` subclasses. `src/diigo.ts` holds `makeDiigoAPI` and
|
||||
the `dumpBookmarks` generator; `DiigoService.open()` hands an RPC caller a
|
||||
rate-limited stub with the cookie attached only on the server side, and
|
||||
promise pipelining makes a whole `pathjoin().query().get()` chain a single
|
||||
round trip. One deliberate divergence from the original: `url` is a private
|
||||
field, not a public property — on an `RpcTarget`, a public property is
|
||||
readable over RPC (a property read is an argument-less method call), and a
|
||||
query string can carry credentials, so a public `url` would leak them to any
|
||||
stub holder.
|
||||
- The instance id (`DIIGO_WEB_ID`): the one secret (bearer capability;
|
||||
possession is authority). Not logged.
|
||||
- The cookie: lives **only in the webapp DO's storage**, crosses the wire
|
||||
exactly once, at mint time. Nothing on disk holds it.
|
||||
- `/_connect`: state-aware — pre-mint it's the paste-cookie form;
|
||||
post-mint it health-checks the instance (count=1 probe) and offers
|
||||
re-mint only for stale cookies.
|
||||
|
||||
## RPC use (from another worker, same account)
|
||||
|
||||
```jsonc
|
||||
// wrangler.jsonc of the caller
|
||||
"services": [{ "binding": "DIIGO", "service": "diigo-dump", "entrypoint": "DiigoService" }]
|
||||
"services": [{ "binding": "DIIGO", "service": "diigo-dump", "entrypoint": "DiigoBridge" }]
|
||||
```
|
||||
|
||||
```ts
|
||||
import { dumpBookmarks } from "./diigo.js";
|
||||
|
||||
const stub = env.DIIGO.open(); // rate-limited, cookie stays server-side
|
||||
// pipeline: one round trip for the whole chain
|
||||
const stub = await env.DIIGO.open();
|
||||
// promise pipelining: whole chain = one round trip; cookie never leaves the DO
|
||||
const first = JSON.parse(await stub.pathjoin("load_user_items")
|
||||
.query({ sort: "updated", count: 24, page_num: 1 }).get());
|
||||
// or run the whole dump in this worker's address space:
|
||||
for await (const item of dumpBookmarks({ diigoAPI: stub, pages: 2 })) {
|
||||
console.log(item);
|
||||
}
|
||||
```
|
||||
|
||||
## Configuration
|
||||
## Run, deploy
|
||||
|
||||
- Secret `DIIGO_COOKIE`: the `Cookie` header from diigo.com devtools.
|
||||
- Local dev: `.dev.vars` (gitignored). Note wrangler reads it only at
|
||||
startup — restart after edits.
|
||||
- Deployed: `npx wrangler secret put DIIGO_COOKIE`.
|
||||
[How to run / connect / deploy is maintained in the capsite checklist]
|
||||
(../capsite-checklist.md) (single source; this README keeps only
|
||||
worker-level facts).
|
||||
|
||||
## Run
|
||||
|
||||
cd diigo-dump
|
||||
npx wrangler dev -c wrangler.jsonc -c ../webapp/wrangler.jsonc
|
||||
# → http://localhost:8787; webapp rides along, binding-only
|
||||
# (first run: npm run diigo-login >> .dev.vars, or the /_connect page;
|
||||
# restart wrangler after adding DIIGO_WEB_ID)
|
||||
|
||||
Status as of 2026-10-06: manual integration test passes — 200, `total: 1674`.
|
||||
See the project ladder in ../capsite-checklist.md; context in
|
||||
../capsite-notes.md. This is the cookie path; the API-key path
|
||||
(EmDash plugin) is still 401.
|
||||
Status: dev wiring verified (boots, pages answer); mint → dump through
|
||||
the DO not yet exercised end-to-end (checklist 1b-i). Earlier flat
|
||||
(non-DO) build passed manual test: 200, `total: 1674` (2026-10-06).
|
||||
Reference in new issue
Block a user