invalidateRecent(days) drops the cached pages holding anything diigo
updated within the last N days (on the cache's injected clock), and
every page before them, so the next read or fill refetches them; item
rows stay. Exposed on the fill capability (private and public) and on
the webkey page as "invalidate the last N days" per cache.
A user-requested refetch, not a change in what stored state means.
Generated-by: Claude Opus 5.5
Comparing a cache with diigo's RSS export: the size-100 walk had missed
exactly the 100 newest bookmarks (everything updated since 2026-09-16).
diigo's page_num is 0-based, and we sent our 1-based page number
unchanged, so "page 1" was really page 2 and the newest page was never
fetched. diigo's total (16744) was right; the "total-overstated"
reading was a misdiagnosis.
- page() asks for page_num = n - 1
- BREAKING for stored state, agreed beforehand, migrated in place:
schema v2 renumbers every stored page +1 (pages and page_members, via
negation so no number collides), once, recorded as kv "schema" = 2;
nothing is refetched, page 1 just becomes a miss
- fill jobs record their numbering; a job saved without it fails ("page
numbering changed …; start the fill again") instead of resuming
- the fake pages from 0 like diigo (7 tests failed until the fix);
tests feed v1-shaped storage and a v1-numbered job to the new code
- the next-page peek guard stays; its comments no longer claim diigo
overstates its total
- the fixed test clock moves to 2099: pinned to "today" its alarms came
due in real time and fired on their own
Generated-by: Claude Opus 5.5
Seen live: "filled: 168 of 1 pages". diigo's total overstated by 100
(16744 vs 16644), so the walk went on to page 168, which diigo answered
empty with a total ≤ 100; the fill recomputed its page count from that,
ceil(total/100) = 1.
Now a page whose own total says the data ended before it ends the walk,
with the count set to the last page with items ("167 of 167"). Judged
by total, not by items, since the public cache may keep none of a page
that isn't the end. The known-pages estimate ignores zero totals. The
fake answers past-the-end pages as diigo does (no items, total 0).
Generated-by: Claude Opus 5.5
page() ran the 500 ms pacing gate before the cache check, so every hit
waited too: a fill rerun skipping 166 cached pages took ~80 s and
looked like a start from scratch. The gate now sits after the hit
check (re-checking storage once through it, in case a request ahead
had just etched the page).
Regression test: the fixed-clock test cache records its sleeps; a miss
sleeps once, three hits add none (failed before: 4 sleeps).
Generated-by: Claude Opus 5.5
The real fill failed at "diigo returned 44 items for page 167 of 168":
diigo's `total` overstates what load_user_items returns, so the true
last page looked short and the guard took it for a capped page size.
A short page that isn't the last by `total` now fetches the next page
(paced like any diigo call): empty means the data really ends there —
accepted, with a "total-overstated" notice logging total vs actual;
anything else means items would be skipped — refused, the message now
giving both counts. The fake gains an account whose total claims 60
more than it has; a fill on it runs to done.
Generated-by: Claude Opus 5.5
Asked for 128, diigo returned 100 (the short-page guard caught it on the
first real fill), so PAGE_SIZE = 100: ~168 pages for 16.7k bookmarks.
The same run showed a fill job outliving a page-size change: progress
counted in 24s ("page 25 of 698") resumed as if counted in 128s. Stored
state reinterpreted, so: FillStatus records pageSize (additive; jobs
saved without it counted in 24), and a job whose size differs from
PAGE_SIZE fails with "page size changed …; start the fill again"
instead of resuming; starting again replaces it. Tested with the old
shape in storage.
Generated-by: Claude Opus 5.5
PAGE_SIZE = 128 (was 24), one constant for the fill job, public pages,
dumps, and DiigoBridge, so a cache never mixes sizes: ~131 pages instead
of ~698 for 16.7k bookmarks.
Cache invalidation, agreed beforehand: pages are cached by (page number,
size), so pages cached at 24 become misses and refetch at 128. Items are
keyed by link_id and just re-upsert; nothing is lost.
Unknown whether diigo honors count=128, so a short page that isn't the
last is refused loudly (logged, thrown; a fill fails with the reason)
rather than silently skipping items. The fake gains a capped account to
test it; the plain account grows to 300 (3 pages), the deep one to 6000.
Generated-by: Claude Opus 5.5
Past the transitional idiom: the cache logic (DiigoCacheLogic,
PublicCacheLogic in cache.ts) requires its Clock, with no default, so
cache.ts reaches no ambient time. The worker's entry module is the
edge: it makes the live clock and exports the runtime-constructed
DiigoCache/PublicCache as (ctx, env) subclasses that supply it (the
Cloudflare constraint: the runtime constructs DOs with ctx and env
only). Binding class names are unchanged, so stored state is too. The
tests' FixedClockCache extends the logic class.
Generated-by: Claude Opus 5.5
No module-level liveClock: the live clock is made in the default
argument expression, at the edge where the runtime constructs the DO,
so module scope holds no powerful object.
Generated-by: Claude Opus 5.5
Ocap discipline: no ambient time in logic. PagedCache reads time and
waits only through a Clock { now, sleep }, a default constructor
argument (the live clock; the runtime constructs DOs with ctx and env
only). The fill job's timestamps and alarms, the etch time, and the
diigo-call pacing all go through it.
Tests interpose through the constructor: tests/test-worker.ts (now the
tests' main module) adds FixedClockCache, bound as CLOCKED_DO, and a
test pins startedAt, updatedAt, and the alarm time to the fixed clock
(an ambient new Date() fails it).
Generated-by: Claude Opus 5.5
POST /_fill (cache id in the body; target private|public; action
start|stop) starts or stops a background walk of every diigo page into
the cache. DO alarms advance it pagesPerStep pages per step (default
20), a second apart; progress (FillStatus) is stored in the DO after
each page, so it survives eviction. Cached pages are skipped; starting
while running rejoins the job; a stop mid-step wins; a failure is
recorded with its reason and logged once. The created page gets fill
buttons; the public page shows "filling: n of m pages".
Fill state is new, additive DO storage (fill, fillStep): no existing
state changes meaning.
The 13 fill specs are unskipped and green, plus one: a fill makes one
diigo call per page fetched.
Generated-by: Claude Opus 5.5
Two bugs in page()'s miss path, found while building the fill job:
- the webapp stubs (app, pathjoin, query) were never disposed, which
kept the DO from being evicted (evictDurableObject hung after a single
miss); they are now disposed once the page is in hand
- a miss on page n > 1 also fetched page 1 and threw it away, doubling
diigo calls; every page carries `total`, so only page n is fetched
Regression test: a miss, then eviction, then a cache hit. tsconfig adds
the ESNext.Disposable lib for Symbol.dispose.
Generated-by: Claude Opus 5.5
A button that grabs every page into a cache: owner-only POST /_fill
(cache id in the body; target private|public; action start|stop),
advanced by DO alarms a few pages per step, progress stored in the DO
(new, additive state) so it survives eviction, cached pages skipped,
a rerun rejoins the running job, failures recorded and logged once.
The created page gets the button; the public page shows progress.
startFill/fillStatus/stopFill on PagedCache are JSDoc'd stubs; 13
specs are describe.skip until it lands (alarms driven by
runDurableObjectAlarm, eviction by evictDurableObject).
Generated-by: Claude Opus 5.5
The dump worker becomes a page cache: a hit is served from DO SQLite
storage with no diigo call; a miss asks the webapp instance in the
cache's wiring and etches the page.
- two independent caches, one behavior (PagedCache): DiigoCache keeps
every item; PublicCache keeps shared items only, dropping private
rows before they are stored; served at /public/<username> (HTML page
form, or JSON)
- create flow on one port: /_connect relays webapp's mint form, /_mint
proxies the mint, /_wire wires a private or public cache; /_publish
adds a public cache later
- ids are webkeys: kept out of URLs (POST bodies), shown and logged
only as a hash fingerprint; saved to the password manager under the
holder's petname (never sent to the server)
- items keyed on diigo's link_id; username from u_name; unkeyable
items and every failure path are logged; each page read logs hit/miss
- tests run the real worker + DOs in the vitest pool against a fake
webapp (tests/fake-webapp.js)
Generated-by: opencode (GLM)
Generated-by: Claude Opus 5.5
dckc