diigo-cache: README sells and gets to TADA; CONTRIBUTING updated
New README: two sentences, the dev run through connect, fill, and a filtered public page, then features. CONTRIBUTING gains the 1d filters and its own run and deploy sections instead of pointing at the checklist; links to the old README now point at CONTRIBUTING. Generated-by: Claude Opus 5.5
This commit is contained in:
1 parent
9719fd0aa8
commit
0af7a3ea6c
4 files changed
+70
-11
No files matched your search
@@ -23,7 +23,7 @@ narrative.
|
|||||||
goal 5.
|
goal 5.
|
||||||
|
|
||||||
- [x] **1c. diigo-dump evolves into a bookmark cache**
|
- [x] **1c. diigo-dump evolves into a bookmark cache**
|
||||||
([diigo-cache/README.md](diigo-cache/README.md); design in the
|
([diigo-cache/CONTRIBUTING.md](diigo-cache/CONTRIBUTING.md); design in the
|
||||||
JSDoc of `src/cache.ts`, `src/index.ts`, `src/pm-save.ts`):
|
JSDoc of `src/cache.ts`, `src/index.ts`, `src/pm-save.ts`):
|
||||||
- [x] 1c-i. Two independent page caches, private and public
|
- [x] 1c-i. Two independent page caches, private and public
|
||||||
(`PagedCache`, `DiigoCache`, `PublicCache`).
|
(`PagedCache`, `DiigoCache`, `PublicCache`).
|
||||||
@@ -113,7 +113,7 @@ narrative.
|
|||||||
- [ ] **5. Deployed end-to-end** (order matters):
|
- [ ] **5. Deployed end-to-end** (order matters):
|
||||||
- [ ] 5-i. webapp: `cd webapp && npx wrangler deploy`
|
- [ ] 5-i. webapp: `cd webapp && npx wrangler deploy`
|
||||||
- [ ] 5-ii. diigo-cache: rate-limit `/api` (and `/public/*`) first
|
- [ ] 5-ii. diigo-cache: rate-limit `/api` (and `/public/*`) first
|
||||||
(diigo-cache README, "Deployment concern"); deploy; create
|
(diigo-cache CONTRIBUTING, "Deployment concern"); deploy; create
|
||||||
against prod (ids → password manager; no worker secret);
|
against prod (ids → password manager; no worker secret);
|
||||||
run the fill job.
|
run the fill job.
|
||||||
- [ ] 5-iii. madmode: deploy with the service binding; the
|
- [ ] 5-iii. madmode: deploy with the service binding; the
|
||||||
@@ -235,5 +235,5 @@ to every goal in this tree.
|
|||||||
Oct 6 2026)
|
Oct 6 2026)
|
||||||
- [webapp/README.md](webapp/README.md) — HTTP + RPC surfaces of the
|
- [webapp/README.md](webapp/README.md) — HTTP + RPC surfaces of the
|
||||||
factory worker; open TODO on the mint-form placement
|
factory worker; open TODO on the mint-form placement
|
||||||
- [diigo-cache/README.md](diigo-cache/README.md) — the cache worker:
|
- [diigo-cache/CONTRIBUTING.md](diigo-cache/CONTRIBUTING.md) — the cache worker:
|
||||||
create flow, the two DOs, run/deploy details
|
create flow, the two DOs, run/deploy details
|
||||||
@@ -1,6 +1,6 @@
|
|||||||
# diigo-cache
|
# Contributing to diigo-cache
|
||||||
|
|
||||||
Cloudflare Worker cache over [the webapp factory](../webapp/README.md):
|
diigo-cache is a Cloudflare Worker cache over [the webapp factory](../webapp/README.md):
|
||||||
session-cookie reads of diigo bookmarks, cached in Durable-Object SQLite
|
session-cookie reads of diigo bookmarks, cached in Durable-Object SQLite
|
||||||
storage. Evolved from the diigo-dump worker (provenance:
|
storage. Evolved from the diigo-dump worker (provenance:
|
||||||
[mm-diigo `projects/diigo-bak/`](https://github.com/dckc/mm-diigo)); the
|
[mm-diigo `projects/diigo-bak/`](https://github.com/dckc/mm-diigo)); the
|
||||||
@@ -36,6 +36,14 @@ goal 2 preamble).
|
|||||||
form (JSON for `Accept: application/json`). Both say `cache: HIT` or
|
form (JSON for `Accept: application/json`). Both say `cache: HIT` or
|
||||||
`MISS`; every page read also logs `{event: "cache", kind, page,
|
`MISS`; every page read also logs `{event: "cache", kind, page,
|
||||||
outcome}`.
|
outcome}`.
|
||||||
|
- **Filters on the public page** (`src/query.ts`): `days`, `after`,
|
||||||
|
`before`, `in` (`2026`, `2026-03`, `2026-Q1`, `2026-W41`),
|
||||||
|
`by=updated|created`, and `q` (words, `"phrases"`, `#tag`, `-`
|
||||||
|
negation, gmail/Slack/GitHub date operators). UTC, half-open windows.
|
||||||
|
The same fields work as a dev-side spec (`toQuery({ days: 7 })`). A
|
||||||
|
filter queries the stored `items` table through `viewer()`, never
|
||||||
|
diigo: results are as fresh as the cache, and JSON says `complete`
|
||||||
|
when every diigo page is stored.
|
||||||
- **Publish later**: `POST /_publish` (cache id in the body) wires a
|
- **Publish later**: `POST /_publish` (cache id in the body) wires a
|
||||||
public companion to a private-only cache.
|
public companion to a private-only cache.
|
||||||
- **The webkey page and Cap'n Web**: `/cache/#<cache id>` is a skeleton
|
- **The webkey page and Cap'n Web**: `/cache/#<cache id>` is a skeleton
|
||||||
@@ -111,10 +119,24 @@ Tests run the real worker and both DOs in the vitest pool, with
|
|||||||
`WEBAPP` bound to [a fake webapp](tests/fake-webapp.js) serving
|
`WEBAPP` bound to [a fake webapp](tests/fake-webapp.js) serving
|
||||||
diigo-shaped items (`wrangler.tests.jsonc`).
|
diigo-shaped items (`wrangler.tests.jsonc`).
|
||||||
|
|
||||||
## Run, deploy
|
## Run
|
||||||
|
|
||||||
[How to run / connect / deploy is maintained in the capsite checklist](../capsite-checklist.md)
|
[The README](README.md) gets a cache going in dev. The same create flow
|
||||||
(single source; this README keeps only worker-level facts).
|
from a terminal:
|
||||||
|
|
||||||
Status: create, dump, publish, and public pages exercised against live
|
wid=$(curl -s -X POST http://localhost:8787/_mint --data-urlencode \
|
||||||
diigo in dev (2026-10-07); not yet deployed.
|
'url=https://www.diigo.com/interact_api/' --data-urlencode \
|
||||||
|
'headers={"user-agent":"Mozilla/5.0","x-requested-with":"XMLHttpRequest","cookie":"<paste>"}' \
|
||||||
|
--data-urlencode 'msPerCall=500' -D - -o /dev/null | grep -i x-webapp-id)
|
||||||
|
curl -s -X POST http://localhost:8787/_wire -H 'content-type: application/json' \
|
||||||
|
-d "{\"webappId\":\"<from $wid>\",\"mode\":\"private\"}"
|
||||||
|
|
||||||
|
Check a filled cache against a diigo RSS export:
|
||||||
|
`node scripts/compare-export.ts <cache.sqlite> <export.xml> <out-dir>`.
|
||||||
|
|
||||||
|
## Deploy
|
||||||
|
|
||||||
|
Not deployed yet; the order matters (goal 5 in
|
||||||
|
[the capsite checklist](../capsite-checklist.md)): webapp first
|
||||||
|
(`cd ../webapp && npx wrangler deploy`), then diigo-cache, after
|
||||||
|
rate-limiting `/api` (above).
|
||||||
@@ -0,0 +1,37 @@
|
|||||||
|
# diigo-cache
|
||||||
|
|
||||||
|
Your diigo bookmarks, cached on Cloudflare, with a public page you can
|
||||||
|
filter by week: `/public/<you>?in=2026-W41`. Your diigo session cookie
|
||||||
|
is stored in one Durable Object and nowhere else.
|
||||||
|
|
||||||
|
## Getting started
|
||||||
|
|
||||||
|
```sh
|
||||||
|
pnpm install
|
||||||
|
pnpm dev # → http://localhost:8787/
|
||||||
|
```
|
||||||
|
|
||||||
|
Then, in the browser:
|
||||||
|
|
||||||
|
1. **mint the diigo session at webapp**: paste the `Cookie` header from
|
||||||
|
diigo.com (devtools, any logged-in request).
|
||||||
|
2. **wire** it, mode **public**. Save the cache id it shows (the
|
||||||
|
password-manager button does this).
|
||||||
|
3. **open this cache** → **fill**, and watch it walk your bookmarks.
|
||||||
|
4. Visit `/public/<your diigo username>?days=7`.
|
||||||
|
|
||||||
|
## Bookmarks you hold, a page anyone can read
|
||||||
|
|
||||||
|
- **Filter by date and search:** `?days=7`, `?in=2026-W41` (ISO week,
|
||||||
|
UTC), `?after=2026-01-01&before=2026-02-01`, `?by=created`, and
|
||||||
|
`?q=` with words, `"phrases"`, `#tags`, `-negation`, and gmail-style
|
||||||
|
`newer_than:7d`. Add `Accept: application/json` for JSON.
|
||||||
|
- **Private stays private:** the public cache drops private bookmarks
|
||||||
|
before storing anything, so it never holds them.
|
||||||
|
- **Ids are capabilities:** the cache id is a bearer key you keep in
|
||||||
|
your password manager. It rides URL fragments and POST bodies, never a
|
||||||
|
path or query string, so it stays out of logs and history.
|
||||||
|
- **Cheap to read:** pages come from Durable Object SQLite storage, and
|
||||||
|
a fill job in the background fetches each diigo page once.
|
||||||
|
|
||||||
|
Design, tests, and deploying: [CONTRIBUTING](CONTRIBUTING.md).
|
||||||
+1
-1
@@ -3,7 +3,7 @@
|
|||||||
DO-backed fluent web client factory: site-agnostic. Deploy once; instances
|
DO-backed fluent web client factory: site-agnostic. Deploy once; instances
|
||||||
are attenuated handles whose **ids are bearer capabilities** — anyone
|
are attenuated handles whose **ids are bearer capabilities** — anyone
|
||||||
holding the id string can drive the instance; nobody without it can.
|
holding the id string can drive the instance; nobody without it can.
|
||||||
TODO: this proxy-page flow (see diigo-cache README) vs running webapp as
|
TODO: this proxy-page flow (see [diigo-cache CONTRIBUTING](../diigo-cache/CONTRIBUTING.md)) vs running webapp as
|
||||||
the primary worker (`wrangler dev -c ../webapp/wrangler.jsonc -c ../diigo-cache/wrangler.jsonc`)
|
the primary worker (`wrangler dev -c ../webapp/wrangler.jsonc -c ../diigo-cache/wrangler.jsonc`)
|
||||||
so its own port serves the mint form directly; reconsider when the UI
|
so its own port serves the mint form directly; reconsider when the UI
|
||||||
grows past a form or two.
|
grows past a form or two.
|
||||||
|
|||||||
Reference in new issue
Block a user