Commit Graph
62 Commits
Author SHA1 Message Date
dckc 4dd3fe6bde capsite/diigo-cache: the wire page hands the owner the store id
webapp's mint now answers a store id (x-webapp-store-id) beside the
instance id: authority to rotate or revoke the diigo cookie. The wire
page shows it apart from the instance id, with its own copy button,
petname, and password-manager save; the cache never gets it. A webapp
that sends none gets a note on the page and a warning in the log.

Generated-by: Claude Opus 5.5
2026-10-09 02:32:22 -05:00
dckc be796d9d0d diigo-cache: the public page is an h-feed of h-entry bookmarks
Microformats are the public page's API for other sites (potluck
first): each bookmark is an h-entry with u-bookmark-of, p-name,
p-summary, a p-category per tag (linked to a tag search),
dt-published (bookmarked) and dt-updated (last edited). Prev/next
links carry rel=prev/next, so a reader can follow a filter's pages.
Tests parse the page with microformats-parser to pin the contract.

Generated-by: Claude Opus 5.5
2026-10-09 01:49:40 -05:00
dckc c66aaea366 potluck, diigo-cache: docs one line per paragraph; potluck deploy caveat
Obsidian and GitHub render each newline as a line break, so the
hard-wrapped READMEs and CONTRIBUTINGs rendered ragged. Whitespace
only, except a note in potluck's CONTRIBUTING that other workerd
hosts are untested: they'd have to supply the D1 and R2 bindings.

Generated-by: Claude Opus 5.5
2026-10-09 01:30:56 -05:00
dckc ea3e256bb8 diigo-cache: README: workerd, not a vendor
Generated-by: Claude Opus 5.5
2026-10-09 01:28:50 -05:00
dckc 0af7a3ea6c diigo-cache: README sells and gets to TADA; CONTRIBUTING updated
New README: two sentences, the dev run through connect, fill, and a
filtered public page, then features. CONTRIBUTING gains the 1d
filters and its own run and deploy sections instead of pointing at
the checklist; links to the old README now point at CONTRIBUTING.

Generated-by: Claude Opus 5.5
2026-10-09 01:27:45 -05:00
dckc 9719fd0aa8 diigo-cache: README becomes CONTRIBUTING; AGENTS.md links to it
Generated-by: Claude Opus 5.5
2026-10-09 01:27:12 -05:00
dckc 002f08b456 diigo-cache: date windows and search on the public page (1d)
/public/<username> takes days, after, before, in (year, month,
quarter, ISO week), by=updated|created, and q (words, "phrases",
#tags, -negation, gmail/Slack/GitHub date operators). JSON answers
echo the parsed filter; a malformed one is a 400.

Filters query the stored items in sqlite through the read-only
viewer instead of walking diigo pages: the public page keeps no
fetch authority, and a filled cache already holds every item.
Results say `complete` when every diigo page is stored; freshness
is 1f's head refresh. So the fetch budget and the O(log pages)
walk test go, and the fake deep account shrinks to 300.

Generated-by: Claude Opus 5.5
2026-10-09 01:17:03 -05:00
dckc 279ddb4caa fixup! diigo-cache: the public page only reads
Comments written for the reader: no restating what the shebang says, no traces of the design conversation; the publish comment no longer mentions a projection that no longer exists.

Generated-by: Claude Opus 5.5
2026-10-07 09:46:21 -05:00
dckc 7c0d1d62c6 fixup! diigo-cache: RSS in diigo's export shape, from cached items
Comments written for the reader: no restating what the shebang says, no traces of the design conversation; the publish comment no longer mentions a projection that no longer exists.

Generated-by: Claude Opus 5.5
2026-10-07 09:46:21 -05:00
dckc 697fd95514 fixup! diigo-cache: RSS in diigo's export shape, from cached items
compare-export.ts: the manual is a HELP string at the top of the module
(a docstring, as a value) and main prints it on bad usage. No reading
its own source, no regex, one copy; @file points at it.

Generated-by: Claude Opus 5.5
2026-10-07 09:44:26 -05:00
dckc 2055e8c1c9 fixup! diigo-cache: RSS in diigo's export shape, from cached items
compare-export.ts: the usage message comes from the @file manual's
synopsis line (read from the script's own source, on the bad-usage path
only), docopt-style, so the synopsis lives in one place; if the header
changes shape, it says "see the @file comment".

Generated-by: Claude Opus 5.5
2026-10-07 09:42:31 -05:00
dckc 8400078656 fixup! diigo-cache: RSS in diigo's export shape, from cached items
compare-export.ts: the manual drops "exits 2 on bad usage"; it's the
convention, and BAD_USAGE documents it.

Generated-by: Claude Opus 5.5
2026-10-07 09:40:47 -05:00
dckc a40d2e924a fixup! diigo-cache: RSS in diigo's export shape, from cached items
compare-export.ts: the usage exit status is a named constant,
BAD_USAGE = 2, whose JSDoc cites the convention (Bash Reference Manual
§3.7.5); the manual just says "Exits 2 on bad usage".

Generated-by: Claude Opus 5.5
2026-10-07 09:40:10 -05:00
dckc 549ff5bf4b fixup! diigo-cache: RSS in diigo's export shape, from cached items
compare-export.ts: exit 2 on bad usage is an external convention, so its
@file cites the source (Bash Reference Manual §3.7.5, Exit Status).

Generated-by: Claude Opus 5.5
2026-10-07 09:39:00 -05:00
dckc 9c99f4f969 fixup! diigo-cache: RSS in diigo's export shape, from cached items
compare-export.ts: as a program script, its @file is its manual
(purpose, usage, outputs, exit code), per docs-files-api's exception;
main's JSDoc keeps only the edge and attenuation.

Generated-by: Claude Opus 5.5
2026-10-07 09:37:31 -05:00
dckc 2bb18a9e96 fixup! diigo-cache: diigo counts pages from 0; migrate stored pages (schema v2)
Docs per the docs-files-api skill: a 1–2 line @file; what a declaration is or does in JSDoc on it, not in // comments or the file header.

Generated-by: Claude Opus 5.5
2026-10-07 09:34:06 -05:00
dckc 2a7d0ae0cb fixup! diigo-cache: SQL in .sql files, behind a read-only facet
Docs per the docs-files-api skill: a 1–2 line @file; what a declaration is or does in JSDoc on it, not in // comments or the file header.

Generated-by: Claude Opus 5.5
2026-10-07 09:34:06 -05:00
dckc ab195b51c4 fixup! diigo-cache: date windows and search, designed as skipped specs (goal 1d)
Docs per the docs-files-api skill: a 1–2 line @file; what a declaration is or does in JSDoc on it, not in // comments or the file header.

Generated-by: Claude Opus 5.5
2026-10-07 09:34:06 -05:00
dckc 11c4583a17 fixup! diigo-cache: bookmark cache in Durable Objects (goal 1c)
Docs per the docs-files-api skill: a 1–2 line @file; what a declaration is or does in JSDoc on it, not in // comments or the file header.

Generated-by: Claude Opus 5.5
2026-10-07 09:34:06 -05:00
dckc 2fc5cfb9ae fixup! refactor: rename diigo-dump to diigo-cache
Docs per the docs-files-api skill: a 1–2 line @file; what a declaration is or does in JSDoc on it, not in // comments or the file header.

Generated-by: Claude Opus 5.5
2026-10-07 09:34:06 -05:00
dckc 0a5de621ec fixup! diigo-cache: fill from a webkey page over Cap'n Web
Docs per the docs-files-api skill: a 1–2 line @file; what a declaration is or does in JSDoc on it, not in // comments or the file header.

Generated-by: Claude Opus 5.5
2026-10-07 09:34:06 -05:00
dckc 164d3cb8cb fixup! diigo-cache: the public page only reads
Docs per the docs-files-api skill: a 1–2 line @file; what a declaration is or does in JSDoc on it, not in // comments or the file header.

Generated-by: Claude Opus 5.5
2026-10-07 09:34:06 -05:00
dckc 2a1942fc2a fixup! diigo-cache: invalidate the last N days of cached pages
Docs per the docs-files-api skill: a 1–2 line @file; what a declaration is or does in JSDoc on it, not in // comments or the file header.

Generated-by: Claude Opus 5.5
2026-10-07 09:34:06 -05:00
dckc 76b9ed1494 fixup! diigo-cache: RSS in diigo's export shape, from cached items
Docs per the docs-files-api skill: a 1–2 line @file; what a declaration is or does in JSDoc on it, not in // comments or the file header.

Generated-by: Claude Opus 5.5
2026-10-07 09:34:06 -05:00
dckc d7190abaf8 diigo-cache: the public page only reads
/public/<username> reads through the cache's viewer() facet
(PageViewer: stored pages, knownTotal, fill progress — the reader
alone), so it can't fetch from diigo or write storage.

Behavior change, agreed beforehand: an unfetched page no longer fills
itself on a miss; it says "not fetched yet" (JSON: stored: false), and
the owner fills via the webkey page. The HIT/MISS line becomes "stored
<when>". Strangers can no longer make the cache spend the owner's diigo
session; the README's deployment concern shrinks to /api.

Tests: viewing an unfetched page makes no webapp call; the page appears
once stored; fill progress shows after one fill step.

Generated-by: Claude Opus 5.5
2026-10-07 09:26:09 -05:00
dckc a17d287e1d diigo-cache: SQL in .sql files, behind a read-only facet
No SQL text in the cache logic any more. Statements live in two files,
parsed by sqlStatements (`-- name:` sections, as in finquick's
brcal/gcbook.js) and imported as text (wrangler Text rule for *.sql):

- cache-read.sql: selects only, checked when the module loads
- cache-write.sql: schema, the v2 renumbering, etch, drops

store.ts turns them into two frozen facets with domain methods:
makeReader(sql) runs only the read statements (page, knownTotal,
deepestPageSince, pagesThrough); makeWriter(storage) only the write
ones (migrate, etch, dropPagesThrough), each change one transaction.

The cache migrates in its constructor under blockConcurrencyWhile, as
Cloudflare recommends, so readers never meet missing tables; the
migration test plants v1 storage and evicts to get a fresh activation.
A viewer() facet (PageViewer: stored pages and fill progress, from the
reader alone) is added for the public page.

Pure refactor: stored state and behavior unchanged.

Generated-by: Claude Opus 5.5
2026-10-07 09:24:33 -05:00
dckc f3241e4985 fixup! diigo-cache: RSS in diigo's export shape, from cached items
compare-export.ts does its file I/O asynchronously (node:fs/promises;
main is async), so no sync-I/O precedent. The one exception says why:
node:sqlite has only DatabaseSync.

Generated-by: Claude Opus 5.5
2026-10-07 08:44:22 -05:00
dckc dc4f23ad04 fixup! diigo-cache: RSS in diigo's export shape, from cached items
compare-export.ts: Object.fromEntries(Object.entries(…).map(…)) as
objectMap(original, mapFn), with @endo/common's signature.

Generated-by: Claude Opus 5.5
2026-10-07 08:41:30 -05:00
dckc 21505a5b8f fixup! diigo-cache: RSS in diigo's export shape, from cached items
compare-export.ts runs main only under import.meta.main (Node's
`if __name__ == "__main__"`), so importing it — say, from a test —
runs nothing.

Generated-by: Claude Opus 5.5
2026-10-07 08:40:00 -05:00
dckc dfd9a3d1ef fixup! diigo-cache: RSS in diigo's export shape, from cached items
compare-export.ts per the ocap style guide (dckc/awesome-ocap,
style-guide/): main takes standard authorities under standard names as
default arguments — argv, fs, connect, now, stdout, stderr, exit (the
transitional form of _script_io) — and attenuates them itself:
connect(dbPath) to a read-only cacheItems, the output directory to an
outDir that writes only plain file names inside it. Counts go to
stdout (program output), usage to stderr.

Generated-by: Claude Opus 5.5
2026-10-07 08:36:57 -05:00
dckc d145841a2e fixup! diigo-cache: RSS in diigo's export shape, from cached items
compare-export.ts under ocap discipline: parseExport and compare are
pure; main is the program's edge and takes every power it uses (argv,
file reads and writes, the sqlite read, the clock, console, the exit
code) as default arguments. Nothing ambient at module level; the last
line just calls main().

Generated-by: Claude Opus 5.5
2026-10-07 08:34:02 -05:00
dckc 7be87d37dc chore: Update pnpm-lock.yaml
Generated-by: Claude Opus 5.5
2026-10-07 08:34:02 -05:00
dckc 0006c7c3a3 diigo-cache: lint typechecks the program scripts
scripts/tsconfig.json gives scripts/ node's types (@types/node, a new
devDependency) and allows the .ts import specifiers node's type
stripping needs; the worker's tsconfig still keeps node's types out.
pnpm lint runs it.

Generated-by: Claude Opus 5.5
2026-10-07 08:34:02 -05:00
dckc cbde0dfbd3 fixup! diigo-cache: RSS in diigo's export shape, from cached items
A diigo note item's body (note_content) renders first, as a Highlight:
line, as the export does: the 6 "export has notes, cache none" items
from the comparison were notes whose bodies the cache had all along.

Generated-by: Claude Opus 5.5
2026-10-07 08:31:07 -05:00
dckc 81f91a6747 fixup! diigo-cache: invalidate the last N days of cached pages
README: the invalidate control.

Generated-by: Claude Opus 5.5
2026-10-07 08:27:19 -05:00
dckc 904f620f06 diigo-cache: invalidate the last N days of cached pages
invalidateRecent(days) drops the cached pages holding anything diigo
updated within the last N days (on the cache's injected clock), and
every page before them, so the next read or fill refetches them; item
rows stay. Exposed on the fill capability (private and public) and on
the webkey page as "invalidate the last N days" per cache.

A user-requested refetch, not a change in what stored state means.

Generated-by: Claude Opus 5.5
2026-10-07 08:27:13 -05:00
dckc cffb1fa9b2 diigo-cache: diigo counts pages from 0; migrate stored pages (schema v2)
Comparing a cache with diigo's RSS export: the size-100 walk had missed
exactly the 100 newest bookmarks (everything updated since 2026-09-16).
diigo's page_num is 0-based, and we sent our 1-based page number
unchanged, so "page 1" was really page 2 and the newest page was never
fetched. diigo's total (16744) was right; the "total-overstated"
reading was a misdiagnosis.

- page() asks for page_num = n - 1
- BREAKING for stored state, agreed beforehand, migrated in place:
  schema v2 renumbers every stored page +1 (pages and page_members, via
  negation so no number collides), once, recorded as kv "schema" = 2;
  nothing is refetched, page 1 just becomes a miss
- fill jobs record their numbering; a job saved without it fails ("page
  numbering changed …; start the fill again") instead of resuming
- the fake pages from 0 like diigo (7 tests failed until the fix);
  tests feed v1-shaped storage and a v1-numbered job to the new code
- the next-page peek guard stays; its comments no longer claim diigo
  overstates its total
- the fixed test clock moves to 2099: pinned to "today" its alarms came
  due in real time and fired on their own

Generated-by: Claude Opus 5.5
2026-10-07 08:26:13 -05:00
dckc e6deb9fda0 diigo-cache: RSS in diigo's export shape, from cached items
src/rss.ts renders items as RSS 2.0 in the shape of diigo's RSS export:
title, link, and a description packing Tags:…;Description:…;
Annotations and Sticky notes:… (Highlight:/Sticky notes: lines). Unlike
the export it is well-formed (characters XML forbids are dropped; the
export carries some raw) and keeps whole titles (the export cuts at
255). Pure: dates come from the caller.

scripts/compare-export.ts (a program script: node, node:sqlite) loads a
cache's items, writes the RSS, and compares it with an export, keyed on
link, reporting counts by category.

Generated-by: Claude Opus 5.5
2026-10-07 05:22:12 -05:00
dckc 4034fba76b diigo-cache: a fill ends at the first page past the data
Seen live: "filled: 168 of 1 pages". diigo's total overstated by 100
(16744 vs 16644), so the walk went on to page 168, which diigo answered
empty with a total ≤ 100; the fill recomputed its page count from that,
ceil(total/100) = 1.

Now a page whose own total says the data ended before it ends the walk,
with the count set to the last page with items ("167 of 167"). Judged
by total, not by items, since the public cache may keep none of a page
that isn't the end. The known-pages estimate ignores zero totals. The
fake answers past-the-end pages as diigo does (no items, total 0).

Generated-by: Claude Opus 5.5
2026-10-07 05:08:12 -05:00
dckc fd16c7db53 diigo-cache: pace diigo calls only; cache hits don't wait
page() ran the 500 ms pacing gate before the cache check, so every hit
waited too: a fill rerun skipping 166 cached pages took ~80 s and
looked like a start from scratch. The gate now sits after the hit
check (re-checking storage once through it, in case a request ahead
had just etched the page).

Regression test: the fixed-clock test cache records its sleeps; a miss
sleeps once, three hits add none (failed before: 4 sleeps).

Generated-by: Claude Opus 5.5
2026-10-07 05:06:17 -05:00
dckc f5d57f73d3 diigo-cache: a short page before the stated end peeks at the next page
The real fill failed at "diigo returned 44 items for page 167 of 168":
diigo's `total` overstates what load_user_items returns, so the true
last page looked short and the guard took it for a capped page size.

A short page that isn't the last by `total` now fetches the next page
(paced like any diigo call): empty means the data really ends there —
accepted, with a "total-overstated" notice logging total vs actual;
anything else means items would be skipped — refused, the message now
giving both counts. The fake gains an account whose total claims 60
more than it has; a fill on it runs to done.

Generated-by: Claude Opus 5.5
2026-10-07 05:04:04 -05:00
dckc f4b68f1b59 fixup! diigo-cache: bookmark cache in Durable Objects (goal 1c)
README: PAGE_SIZE is 100, diigo's cap.

Generated-by: Claude Opus 5.5
2026-10-07 04:59:39 -05:00
dckc 563cb83995 diigo-cache: page size 100, diigo's cap; fill jobs record their page size
Asked for 128, diigo returned 100 (the short-page guard caught it on the
first real fill), so PAGE_SIZE = 100: ~168 pages for 16.7k bookmarks.

The same run showed a fill job outliving a page-size change: progress
counted in 24s ("page 25 of 698") resumed as if counted in 128s. Stored
state reinterpreted, so: FillStatus records pageSize (additive; jobs
saved without it counted in 24), and a job whose size differs from
PAGE_SIZE fails with "page size changed …; start the fill again"
instead of resuming; starting again replaces it. Tested with the old
shape in storage.

Generated-by: Claude Opus 5.5
2026-10-07 04:59:39 -05:00
dckc 3baffcab4c fixup! diigo-cache: bookmark cache in Durable Objects (goal 1c)
README: page size is PAGE_SIZE (128), not 24.

Generated-by: Claude Opus 5.5
2026-10-07 04:57:01 -05:00
dckc 826153e35f diigo-cache: fetch 128 items per diigo page; refuse short pages
PAGE_SIZE = 128 (was 24), one constant for the fill job, public pages,
dumps, and DiigoBridge, so a cache never mixes sizes: ~131 pages instead
of ~698 for 16.7k bookmarks.

Cache invalidation, agreed beforehand: pages are cached by (page number,
size), so pages cached at 24 become misses and refetch at 128. Items are
keyed by link_id and just re-upsert; nothing is lost.

Unknown whether diigo honors count=128, so a short page that isn't the
last is refused loudly (logged, thrown; a fill fails with the reason)
rather than silently skipping items. The fake gains a capped account to
test it; the plain account grows to 300 (3 pages), the deep one to 6000.

Generated-by: Claude Opus 5.5
2026-10-07 04:56:56 -05:00
dckc efaf74023d diigo-cache: webkey page logs in with the password manager
/cache/ without a fragment is now a login form: petname
(autocomplete=username) and cache id (type=password,
autocomplete=current-password), the shape password managers fill from
the entries the save button makes. Its inputs have no name, so a native
submit (script failed) sends nothing; the key stays in memory, out of
the address bar and history. Home links to it.

Generated-by: Claude Opus 5.5
2026-10-07 04:48:15 -05:00
dckc 961515d8db fixup! diigo-cache: the caches take their clock as a constructor argument
Past the transitional idiom: the cache logic (DiigoCacheLogic,
PublicCacheLogic in cache.ts) requires its Clock, with no default, so
cache.ts reaches no ambient time. The worker's entry module is the
edge: it makes the live clock and exports the runtime-constructed
DiigoCache/PublicCache as (ctx, env) subclasses that supply it (the
Cloudflare constraint: the runtime constructs DOs with ctx and env
only). Binding class names are unchanged, so stored state is too. The
tests' FixedClockCache extends the logic class.

Generated-by: Claude Opus 5.5
2026-10-07 04:39:29 -05:00
dckc cf901ded5f fixup! diigo-cache: the caches take their clock as a constructor argument
No module-level liveClock: the live clock is made in the default
argument expression, at the edge where the runtime constructs the DO,
so module scope holds no powerful object.

Generated-by: Claude Opus 5.5
2026-10-07 04:35:39 -05:00
dckc b6823867ff diigo-cache: the caches take their clock as a constructor argument
Ocap discipline: no ambient time in logic. PagedCache reads time and
waits only through a Clock { now, sleep }, a default constructor
argument (the live clock; the runtime constructs DOs with ctx and env
only). The fill job's timestamps and alarms, the etch time, and the
diigo-call pacing all go through it.

Tests interpose through the constructor: tests/test-worker.ts (now the
tests' main module) adds FixedClockCache, bound as CLOCKED_DO, and a
test pins startedAt, updatedAt, and the alarm time to the fixed clock
(an ambient new Date() fails it).

Generated-by: Claude Opus 5.5
2026-10-07 04:34:48 -05:00
dckc 3d0e3aea69 chore: Update pnpm-lock.yaml
Generated-by: Claude Opus 5.5
2026-10-07 04:32:44 -05:00