diigo-cache: /public/<username>/all, searched client-side

Every stored shared bookmark on one page, in the public page's
microformats. Without JS that's the whole list, and the search form
asks the server's filtered page. With JS, the newest 5 show and search
hides and shows entries in place.

The browser runs the server's own matcher: query.ts becomes query.js
(JSDoc types, checkJs), which the worker imports as code and serves as
text at /_query.js via a `?raw` import and a wrangler Text rule.
`shown()` picks what the page shows; the page's script reads items
back from each h-entry.

Tests: shown(); /all lists all 270 stored shared items, none hidden;
/_query.js is the module the worker runs; and client-side search over
/all's parsed microformats finds what the server's ?q= finds. The
script itself was checked once in headless Chromium (5 of 54 shown by
default; ?q=fragments shows 8), not in CI.

Generated-by: Claude Opus 5.5
This commit is contained in:
dckc committed 2026-10-10 16:45:41 -05:00
1 parent 1d8a220bbc
commit c3caaaeed1
12 files changed
+357 -86

No files matched your search

+3 -1
View File
@@ -10,7 +10,9 @@ diigo-cache is a Cloudflare Worker cache over [the webapp factory](../webapp/REA
- `PublicCache` (binding `PUBLIC_DO`): shared items only — private rows are dropped *before* etching (`publicOnly`), so its storage holds nothing private. Same cookie webapp as the private cache. Public page n is diigo page n, filtered (may show fewer than `PAGE_SIZE`, 100: diigo's cap). `/public/<username>` resolves through the `idFromName("index")` registry, and reads through `viewer()`, a read-only facet: it shows stored pages and fill progress, and can't fetch from diigo or write. - `PublicCache` (binding `PUBLIC_DO`): shared items only — private rows are dropped *before* etching (`publicOnly`), so its storage holds nothing private. Same cookie webapp as the private cache. Public page n is diigo page n, filtered (may show fewer than `PAGE_SIZE`, 100: diigo's cap). `/public/<username>` resolves through the `idFromName("index")` registry, and reads through `viewer()`, a read-only facet: it shows stored pages and fill progress, and can't fetch from diigo or write.
- **Storage**: SQL lives in `src/cache-read.sql` (selects only) and `src/cache-write.sql`; `src/store.ts` turns them into a reader and a writer facet, so the cache logic holds no SQL text. - **Storage**: SQL lives in `src/cache-read.sql` (selects only) and `src/cache-write.sql`; `src/store.ts` turns them into a reader and a writer facet, so the cache logic holds no SQL text.
- **Pages**: home's dump form (cache id + page number, POST) shows raw dump rows; `/public/<username>?page=n` shows an HTML list with a page form (JSON for `Accept: application/json`). Both say `cache: HIT` or `MISS`; every page read also logs `{event: "cache", kind, page, outcome}`. - **Pages**: home's dump form (cache id + page number, POST) shows raw dump rows; `/public/<username>?page=n` shows an HTML list with a page form (JSON for `Accept: application/json`). Both say `cache: HIT` or `MISS`; every page read also logs `{event: "cache", kind, page, outcome}`.
- **Filters on the public page** (`src/query.ts`): `days`, `after`, `before`, `in` (`2026`, `2026-03`, `2026-Q1`, `2026-W41`), `by=updated|created`, and `q` (words, `"phrases"`, `#tag`, `-` negation, gmail/Slack/GitHub date operators). UTC, half-open windows. The same fields work as a dev-side spec (`toQuery({ days: 7 })`). A filter queries the stored `items` table through `viewer()`, never diigo: results are as fresh as the cache, and JSON says `complete` when every diigo page is stored. - **Filters on the public page** (`src/query.js`): `days`, `after`, `before`, `in` (`2026`, `2026-03`, `2026-Q1`, `2026-W41`), `by=updated|created`, and `q` (words, `"phrases"`, `#tag`, `-` negation, gmail/Slack/GitHub date operators). UTC, half-open windows. The same fields work as a dev-side spec (`toQuery({ days: 7 })`). A filter queries the stored `items` table through `viewer()`, never diigo: results are as fresh as the cache, and JSON says `complete` when every diigo page is stored.
- **All bookmarks, searched client-side**: `/public/<username>/all` lists every stored shared bookmark in the same microformats. Without JS that's the whole list, and its search form asks the server. With JS (`src/all-page.ts`), the newest 5 show and search hides and shows entries in place, by the server's own matcher: `src/query.js` is plain JS with JSDoc types, so the worker imports it as code and also serves it, as text (`?raw`, a wrangler Text rule), at `/_query.js`.
- **All bookmarks, searched client-side**: `/public/<username>/all` lists every stored shared bookmark in the same microformats. Without JS that's the whole list, and its search form asks the server. With JS (`src/all-page.ts`), the newest 5 show and search hides and shows entries in place, by the server's own matcher: `src/query.js` is plain JS with JSDoc types, so the worker imports it as code and also serves it, as text (`?raw`, a wrangler Text rule), at `/_query.js`.
- **Publish later**: `POST /_publish` (cache id in the body) wires a public companion to a private-only cache. - **Publish later**: `POST /_publish` (cache id in the body) wires a public companion to a private-only cache.
- **The webkey page and Cap'n Web**: `/cache/#<cache id>` is a skeleton page, the same for every key; its script reads the key from the fragment and trades it in-band for a capability over [Cap'n Web](https://github.com/cloudflare/capnweb) at `/api` (`CacheApi.cache(key)` → a `CacheCap` facet: fill, status, the public companion; `src/api.ts`). The worker serves capnweb's browser build itself at `/_capnweb.js`. - **The webkey page and Cap'n Web**: `/cache/#<cache id>` is a skeleton page, the same for every key; its script reads the key from the fragment and trades it in-band for a capability over [Cap'n Web](https://github.com/cloudflare/capnweb) at `/api` (`CacheApi.cache(key)` → a `CacheCap` facet: fill, status, the public companion; `src/api.ts`). The worker serves capnweb's browser build itself at `/_capnweb.js`.
- **Fill job**: walks every diigo page into a cache in the background — DO alarms, a few pages per step, progress stored in the DO (`startFill`/`fillStatus`/`stopFill`), driven from the webkey page; the public page shows progress. "Invalidate the last N days" there drops the cached pages holding anything updated that recently (`invalidateRecent`), so the next fill refetches them. - **Fill job**: walks every diigo page into a cache in the background — DO alarms, a few pages per step, progress stored in the DO (`startFill`/`fillStatus`/`stopFill`), driven from the webkey page; the public page shows progress. "Invalidate the last N days" there drops the cached pages holding anything updated that recently (`invalidateRecent`), so the next fill refetches them.
+1
View File
@@ -19,6 +19,7 @@ Then, in the browser:
## Bookmarks you hold, a page anyone can read ## Bookmarks you hold, a page anyone can read
- **Filter by date and search:** `?days=7`, `?in=2026-W41` (ISO week, UTC), `?after=2026-01-01&before=2026-02-01`, `?by=created`, and `?q=` with words, `"phrases"`, `#tags`, `-negation`, and gmail-style `newer_than:7d`. Add `Accept: application/json` for JSON. - **Filter by date and search:** `?days=7`, `?in=2026-W41` (ISO week, UTC), `?after=2026-01-01&before=2026-02-01`, `?by=created`, and `?q=` with words, `"phrases"`, `#tags`, `-negation`, and gmail-style `newer_than:7d`. Add `Accept: application/json` for JSON.
- **Everything on one page:** `/public/<you>/all` lists every stored shared bookmark; with JS it shows the newest 5 and searches as you type, with the same syntax.
- **Private stays private:** the public cache drops private bookmarks before storing anything, so it never holds them. - **Private stays private:** the public cache drops private bookmarks before storing anything, so it never holds them.
- **Ids are capabilities:** the cache id is a bearer key you keep in your password manager. It rides URL fragments and POST bodies, never a path or query string, so it stays out of logs and history. - **Ids are capabilities:** the cache id is a bearer key you keep in your password manager. It rides URL fragments and POST bodies, never a path or query string, so it stays out of logs and history.
- **Cheap to read:** pages come from Durable Object SQLite storage, and a fill job in the background fetches each diigo page once. - **Cheap to read:** pages come from Durable Object SQLite storage, and a fill job in the background fetches each diigo page once.
+53
View File
@@ -0,0 +1,53 @@
/** @file the all-bookmarks page's script: client-side search over the page's own microformats. */
/**
* the script for /public/<username>/all. Without it, the page lists every
* stored shared bookmark, and its form searches on the server. With it,
* the page shows the newest few and searches as you type, using
* `shown()` from /_query.js: the server's own matcher, fed items read
* back from each `h-entry`.
*/
export const ALL_PAGE_SCRIPT = `
import { shown } from "/_query.js";
const entries = [...document.querySelectorAll(".h-feed > .h-entry")];
const text = (li, selector) => li.querySelector(selector)?.textContent ?? "";
const secs = (li, cls) => {
const time = li.querySelector("time." + cls);
return time ? Date.parse(time.dateTime) / 1000 : undefined;
};
const items = entries.map((li) => ({
title: text(li, ".p-name"),
url: li.querySelector(".u-bookmark-of")?.getAttribute("href") ?? "",
description: text(li, ".p-summary"),
tags: [...li.querySelectorAll(".p-category")].map((a) => a.textContent).join(","),
created_at: secs(li, "dt-published"),
updated_at: secs(li, "dt-updated"),
}));
const form = document.getElementById("search");
const input = form.elements.q;
const status = document.getElementById("all-status");
const apply = () => {
const result = shown(items, input.value, new Date());
if ("error" in result) {
status.textContent = result.error;
return;
}
entries.forEach((li, i) => {
li.hidden = !result.show[i];
});
const n = entries.length;
status.textContent =
result.matched === undefined
? \`the newest \${Math.min(5, n)} of \${n}; search to see more\`
: \`\${result.matched} of \${n} match\`;
};
form.addEventListener("submit", (event) => {
event.preventDefault();
apply();
});
input.addEventListener("input", apply);
input.value = new URLSearchParams(location.search).get("q") ?? "";
apply();
`;
+58 -5
View File
@@ -8,6 +8,7 @@ import { newWorkersRpcResponse } from "capnweb";
import { CacheApi } from "./api.js"; import { CacheApi } from "./api.js";
import { CAPNWEB_BROWSER_JS } from "./capnweb-browser.js"; import { CAPNWEB_BROWSER_JS } from "./capnweb-browser.js";
import { CACHE_PAGE_HTML } from "./cache-page.js"; import { CACHE_PAGE_HTML } from "./cache-page.js";
import { ALL_PAGE_SCRIPT } from "./all-page.js";
import { DiigoCacheLogic, PublicCacheLogic } from "./cache.js"; import { DiigoCacheLogic, PublicCacheLogic } from "./cache.js";
import { PAGE_SIZE } from "./cache.js"; import { PAGE_SIZE } from "./cache.js";
import type { CacheEnv, Clock, FillStatus, QueryPage } from "./cache.js"; import type { CacheEnv, Clock, FillStatus, QueryPage } from "./cache.js";
@@ -15,6 +16,7 @@ import type { Item } from "./cache.js";
import { usernameOf } from "./items.js"; import { usernameOf } from "./items.js";
import { dateOf, isEmpty, parseQuery } from "./query.js"; import { dateOf, isEmpty, parseQuery } from "./query.js";
import type { Query } from "./query.js"; import type { Query } from "./query.js";
import QUERY_JS from "./query.js?raw";
import { toWebkey, idFromWebkey, isWebkey, shashKey } from "./webkeys.js"; import { toWebkey, idFromWebkey, isWebkey, shashKey } from "./webkeys.js";
import { pmSave } from "./pm-save.js"; import { pmSave } from "./pm-save.js";
@@ -85,6 +87,8 @@ export class DiigoBridge extends WorkerEntrypoint {
* - `GET /cache/`: the webkey page ({@link CACHE_PAGE_HTML}); * - `GET /cache/`: the webkey page ({@link CACHE_PAGE_HTML});
* `/api`: its Cap'n Web API ({@link CacheApi}); `GET /_capnweb.js` * `/api`: its Cap'n Web API ({@link CacheApi}); `GET /_capnweb.js`
* - `GET /public/<username>`: public pages, read-only ({@link publicPage}) * - `GET /public/<username>`: public pages, read-only ({@link publicPage})
* - `GET /public/<username>/all`: every stored shared bookmark, searched
* client-side ({@link allPage}); `GET /_query.js`, its matcher
* *
* Cache ids are webkeys (docs/webkeys.md): they ride POST bodies and URL * Cache ids are webkeys (docs/webkeys.md): they ride POST bodies and URL
* fragments, never query strings or paths. * fragments, never query strings or paths.
@@ -95,6 +99,8 @@ export default {
const pub = url.pathname.match(/^\/public\/([A-Za-z0-9_-]+)$/); const pub = url.pathname.match(/^\/public\/([A-Za-z0-9_-]+)$/);
if (pub) return publicPage(request, env, pub[1]); if (pub) return publicPage(request, env, pub[1]);
const all = url.pathname.match(/^\/public\/([A-Za-z0-9_-]+)\/all$/);
if (all) return allPage(env, all[1]);
if (url.pathname === "/_connect" && request.method === "GET") { if (url.pathname === "/_connect" && request.method === "GET") {
return mintPage(env); return mintPage(env);
@@ -114,6 +120,9 @@ export default {
if (url.pathname === "/_capnweb.js" && request.method === "GET") { if (url.pathname === "/_capnweb.js" && request.method === "GET") {
return new Response(CAPNWEB_BROWSER_JS, { headers: { "content-type": "text/javascript;charset=utf-8" } }); return new Response(CAPNWEB_BROWSER_JS, { headers: { "content-type": "text/javascript;charset=utf-8" } });
} }
if (url.pathname === "/_query.js" && request.method === "GET") {
return new Response(QUERY_JS, { headers: { "content-type": "text/javascript;charset=utf-8" } });
}
if (url.pathname === "/api") { if (url.pathname === "/api") {
return newWorkersRpcResponse(request, new CacheApi(env)); return newWorkersRpcResponse(request, new CacheApi(env));
} }
@@ -458,11 +467,8 @@ const DAY_PRESETS = [7, 30, 90, 365];
* fine here: nothing on this page is secret. * fine here: nothing on this page is secret.
*/ */
async function publicPage(request: Request, env: Env, username: string): Promise<Response> { async function publicPage(request: Request, env: Env, username: string): Promise<Response> {
const index = env.PUBLIC_DO.get(env.PUBLIC_DO.idFromName("index")); const cache = await publicCacheOf(env, username);
const publicId = await index.lookup(username); if (!cache) return new Response("no such public cache", { status: 404 });
if (!publicId) return new Response("no such public cache", { status: 404 });
const ns = env.PUBLIC_DO;
const cache = ns.get(ns.idFromString(idFromWebkey(publicId)));
const params = new URL(request.url).searchParams; const params = new URL(request.url).searchParams;
const accept = request.headers.get("accept") ?? ""; const accept = request.headers.get("accept") ?? "";
@@ -534,6 +540,53 @@ ${notice}${list}`,
return new Response(body, { headers: { "content-type": "text/html;charset=utf-8" } }); return new Response(body, { headers: { "content-type": "text/html;charset=utf-8" } });
} }
/** the public cache registered for `username`, if any */
async function publicCacheOf(env: Env, username: string) {
const index = env.PUBLIC_DO.get(env.PUBLIC_DO.idFromName("index"));
const publicId = await index.lookup(username);
if (!publicId) return undefined;
return env.PUBLIC_DO.get(env.PUBLIC_DO.idFromString(idFromWebkey(publicId)));
}
/**
* every stored shared bookmark on one page, newest updated first, as the
* same microformats as the public page. Without JS, that's the whole
* list, and the search form asks the public page. With JS
* ({@link ALL_PAGE_SCRIPT}), the newest few show, and search hides and
* shows entries in place with the server's matcher.
*/
async function allPage(env: Env, username: string): Promise<Response> {
const cache = await publicCacheOf(env, username);
if (!cache) return new Response("no such public cache", { status: 404 });
const viewer = await cache.viewer();
let found, fill;
try {
found = await viewer.query({ by: "updated", words: [], tags: [] }, 1, Number.MAX_SAFE_INTEGER);
fill = await viewer.fillStatus();
} catch (err) {
console.error({ event: "all-page", error: String(err) });
return new Response(`could not load bookmarks: ${String(err)}`, { status: 502 });
} finally {
viewer[Symbol.dispose]();
}
const here = `/public/${encodeURIComponent(username)}`;
const items = found.items.map((j) => JSON.parse(j) as Item);
const caveat = found.complete ? "" : " — not every diigo page is stored yet, so some may be missing";
const body = publicFrame(
username,
`<p id="all-status">${items.length} shared bookmark${items.length === 1 ? "" : "s"}, newest first${caveat}</p>
${fill ? fillLine(fill) : ""}
<form id="search" method="get" action="${here}">
<p><label>search <input type="search" name="q" size="40"></label> <button type="submit">search</button></p>
<p><small>words, "phrases", #tags, -negation, and after:, before:, newer_than:7d, during:2026-Q1, created:2026 (UTC)</small></p>
</form>
<p><a href="${here}">by diigo page</a></p>
${itemList(items, "updated", here)}
<script type="module">${ALL_PAGE_SCRIPT}</script>`,
);
return new Response(body, { headers: { "content-type": "text/html;charset=utf-8" } });
}
/** a filter's matches, as the public page's HTML or JSON */ /** a filter's matches, as the public page's HTML or JSON */
function filteredPage( function filteredPage(
username: string, username: string,
@@ -1,10 +1,14 @@
/** /**
* @file the public page's filters: date windows and search over a cache's * @file the public page's filters: date windows and search over a cache's
* stored items (goal 1d; tests/query.test.ts). * stored items (goal 1d; tests/query.test.ts).
*
* Plain JS with JSDoc types, so the worker imports it as code and serves
* the same file to browsers at /_query.js: the all-bookmarks page searches
* client-side with exactly the server's matcher.
*
* @import { Item } from "./items.js"
*/ */
import type { Item } from "./items.js";
/** /**
* a filter as devs write it (`{ days: 7, q: "#weekly" }`); URL params map * a filter as devs write it (`{ days: 7, q: "#weekly" }`); URL params map
* onto these fields one-to-one (strings, with "" meaning absent). Dates * onto these fields one-to-one (strings, with "" meaning absent). Dates
@@ -12,49 +16,49 @@ import type { Item } from "./items.js";
* `2025-03`, `2025-Q1`, or ISO week `2025-W01`. Time is UTC throughout; * `2025-03`, `2025-Q1`, or ISO week `2025-W01`. Time is UTC throughout;
* ISO weeks start Monday, keeping a weekend together. * ISO weeks start Monday, keeping a weekend together.
*/ */
export type FilterSpec = { /**
/** the last N days, counted back from now (1–3650) */ * @typedef {object} FilterSpec
days?: number | string; * @property {number | string} [days] the last N days, counted back from now (1–3650)
/** inclusive lower bound, from 00:00Z */ * @property {string} [after] inclusive lower bound, from 00:00Z
after?: string; * @property {string} [before] exclusive upper bound, at 00:00Z
/** exclusive upper bound, at 00:00Z */ * @property {string} [in] a named period: shorthand for its after/before pair
before?: string; * @property {"updated" | "created" | ""} [by] which date the window, the listed date, and the sort use
/** a named period: shorthand for its after/before pair */ * @property {string} [q] words, phrases, tags, and operators
in?: string; */
/** which date the window, the listed date, and the sort use */
by?: "updated" | "created" | "";
/** words, phrases, tags, and operators */
q?: string;
};
/** one search term; `negated` for a leading `-` */ /**
export type Term = { text: string; negated: boolean }; * one search term; `negated` for a leading `-`
* @typedef {{ text: string; negated: boolean }} Term
*/
/** a parsed filter; an empty one (no window, no terms) filters nothing */ /**
export type Query = { * a parsed filter; an empty one (no window, no terms) filters nothing
/** which date the window, the listed date, and the sort use (default * @typedef {object} Query
* `updated`, diigo's own order) */ * @property {"updated" | "created"} by which date the window, the listed
by: "updated" | "created"; * date, and the sort use (default `updated`, diigo's own order)
/** inclusive lower bound */ * @property {Date} [after] inclusive lower bound
after?: Date; * @property {Date} [before] exclusive upper bound
/** exclusive upper bound */ * @property {Term[]} words words and phrases: each must appear in title, description, or url
before?: Date; * @property {Term[]} tags tags: exact, case-insensitive match against the item's tags
/** words and phrases: each must appear in title, description, or url */ */
words: Term[];
/** tags: exact, case-insensitive match against the item's tags */
tags: Term[];
};
/** a refused filter: the message is safe to show and log */ /**
export type QueryError = { error: string }; * a refused filter: the message is safe to show and log
* @typedef {{ error: string }} QueryError
*/
const DAY_MS = 24 * 60 * 60 * 1000; const DAY_MS = 24 * 60 * 60 * 1000;
/** thrown inside {@link toQuery}, returned as a {@link QueryError} */ /** thrown inside {@link toQuery}, returned as a {@link QueryError} */
class Refused extends Error {} class Refused extends Error {}
/** `yyyy-mm-dd` or `yyyy/mm/dd` as 00:00Z of that UTC day */ /**
function parseDay(text: string, what: string): Date { * `yyyy-mm-dd` or `yyyy/mm/dd` as 00:00Z of that UTC day
* @param {string} text
* @param {string} what
* @returns {Date}
*/
function parseDay(text, what) {
const m = /^(\d{4})([-/])(\d{2})\2(\d{2})$/.exec(text); const m = /^(\d{4})([-/])(\d{2})\2(\d{2})$/.exec(text);
const d = m && new Date(Date.UTC(Number(m[1]), Number(m[3]) - 1, Number(m[4]))); const d = m && new Date(Date.UTC(Number(m[1]), Number(m[3]) - 1, Number(m[4])));
if (!m || !d || d.getUTCMonth() !== Number(m[3]) - 1 || d.getUTCDate() !== Number(m[4])) { if (!m || !d || d.getUTCMonth() !== Number(m[3]) - 1 || d.getUTCDate() !== Number(m[4])) {
@@ -63,15 +67,24 @@ function parseDay(text: string, what: string): Date {
return d; return d;
} }
/** Monday 00:00Z of ISO week 1 of `year` (the week holding January 4) */ /**
function isoWeekOne(year: number): number { * Monday 00:00Z of ISO week 1 of `year` (the week holding January 4)
* @param {number} year
* @returns {number}
*/
function isoWeekOne(year) {
const jan4 = Date.UTC(year, 0, 4); const jan4 = Date.UTC(year, 0, 4);
const weekday = new Date(jan4).getUTCDay() || 7; const weekday = new Date(jan4).getUTCDay() || 7;
return jan4 - (weekday - 1) * DAY_MS; return jan4 - (weekday - 1) * DAY_MS;
} }
/** a named period as its half-open [after, before) window */ /**
function parsePeriod(text: string, what: string): [Date, Date] { * a named period as its half-open [after, before) window
* @param {string} text
* @param {string} what
* @returns {[Date, Date]}
*/
function parsePeriod(text, what) {
const refuse = () => new Refused(`${what}: expected a period as 2025, 2025-03, 2025-Q1, or 2025-W01`); const refuse = () => new Refused(`${what}: expected a period as 2025, 2025-03, 2025-Q1, or 2025-W01`);
let m; let m;
if ((m = /^(\d{4})$/.exec(text))) { if ((m = /^(\d{4})$/.exec(text))) {
@@ -97,9 +110,14 @@ function parsePeriod(text: string, what: string): [Date, Date] {
throw refuse(); throw refuse();
} }
/** gmail's `newer_than:` amount (`7d`, `2w`, `1m`, `1y`) counted back from /**
* `now`; months and years are calendar steps */ * gmail's `newer_than:` amount (`7d`, `2w`, `1m`, `1y`) counted back from
function parseNewerThan(text: string, now: Date): Date { * `now`; months and years are calendar steps
* @param {string} text
* @param {Date} now
* @returns {Date}
*/
function parseNewerThan(text, now) {
const m = /^(\d{1,4})([dwmy])$/.exec(text); const m = /^(\d{1,4})([dwmy])$/.exec(text);
if (!m || Number(m[1]) < 1) throw new Refused("newer_than: expected an amount like 7d, 2w, 1m, or 1y"); if (!m || Number(m[1]) < 1) throw new Refused("newer_than: expected an amount like 7d, 2w, 1m, or 1y");
const n = Number(m[1]); const n = Number(m[1]);
@@ -111,8 +129,12 @@ function parseNewerThan(text: string, now: Date): Date {
return d; return d;
} }
/** `days` as a whole number from 1 to 3650 */ /**
function parseDays(days: number | string): number { * `days` as a whole number from 1 to 3650
* @param {number | string} days
* @returns {number}
*/
function parseDays(days) {
const n = typeof days === "number" ? days : /^\d+$/.test(days) ? Number(days) : NaN; const n = typeof days === "number" ? days : /^\d+$/.test(days) ? Number(days) : NaN;
if (!Number.isInteger(n) || n < 1 || n > 3650) throw new Refused("days: expected a whole number from 1 to 3650"); if (!Number.isInteger(n) || n < 1 || n > 3650) throw new Refused("days: expected a whole number from 1 to 3650");
return n; return n;
@@ -133,10 +155,12 @@ function parseDays(days: number | string): number {
* @see https://slack.com/help/articles/202528808 * @see https://slack.com/help/articles/202528808
* @see https://docs.github.com/en/search-github/searching-on-github/searching-for-repositories * @see https://docs.github.com/en/search-github/searching-on-github/searching-for-repositories
* *
* @param now the clock `days`/`newer_than` count back from (passed in, * @param {FilterSpec} spec
* not read ambiently, so tests pin it) * @param {Date} now the clock `days`/`newer_than` count back from (passed
* in, not read ambiently, so tests pin it)
* @returns {Query | QueryError}
*/ */
export function toQuery(spec: FilterSpec, now: Date): Query | QueryError { export function toQuery(spec, now) {
try { try {
return resolve(spec, now); return resolve(spec, now);
} catch (err) { } catch (err) {
@@ -145,39 +169,54 @@ export function toQuery(spec: FilterSpec, now: Date): Query | QueryError {
} }
} }
function resolve(spec: FilterSpec, now: Date): Query { /**
const words: Term[] = []; * @param {FilterSpec} spec
const tags: Term[] = []; * @param {Date} now
let by: { value: Query["by"]; from: string } | undefined; * @returns {Query}
let after: { value: Date; from: string } | undefined; */
let before: { value: Date; from: string } | undefined; function resolve(spec, now) {
/** @type {Term[]} */
const words = [];
/** @type {Term[]} */
const tags = [];
/** @type {{ value: Query["by"]; from: string } | undefined} */
let by;
/** @type {{ value: Date; from: string } | undefined} */
let after;
/** @type {{ value: Date; from: string } | undefined} */
let before;
const setBy = (value: Query["by"], from: string) => { /** @type {(value: Query["by"], from: string) => void} */
const setBy = (value, from) => {
if (by && by.value !== value) throw new Refused(`${by.from} and ${from} name different dates`); if (by && by.value !== value) throw new Refused(`${by.from} and ${from} name different dates`);
by = { value, from }; by = { value, from };
}; };
const setAfter = (value: Date, from: string) => { /** @type {(value: Date, from: string) => void} */
const setAfter = (value, from) => {
if (after) throw new Refused(`${after.from} and ${from} both set where the window starts`); if (after) throw new Refused(`${after.from} and ${from} both set where the window starts`);
after = { value, from }; after = { value, from };
}; };
const setBefore = (value: Date, from: string) => { /** @type {(value: Date, from: string) => void} */
const setBefore = (value, from) => {
if (before) throw new Refused(`${before.from} and ${from} both set where the window ends`); if (before) throw new Refused(`${before.from} and ${from} both set where the window ends`);
before = { value, from }; before = { value, from };
}; };
const setPeriod = ([lo, hi]: [Date, Date], from: string) => { /** @type {(window: [Date, Date], from: string) => void} */
const setPeriod = ([lo, hi], from) => {
setAfter(lo, from); setAfter(lo, from);
setBefore(hi, from); setBefore(hi, from);
}; };
const present = (v: string | number | undefined) => v !== undefined && v !== ""; /** @type {(v: string | number | undefined) => boolean} */
const present = (v) => v !== undefined && v !== "";
if (present(spec.by)) { if (present(spec.by)) {
if (spec.by !== "updated" && spec.by !== "created") throw new Refused("by: expected updated or created"); if (spec.by !== "updated" && spec.by !== "created") throw new Refused("by: expected updated or created");
setBy(spec.by, `by=${spec.by}`); setBy(spec.by, `by=${spec.by}`);
} }
if (present(spec.days)) setAfter(new Date(now.getTime() - parseDays(spec.days!) * DAY_MS), "days"); if (spec.days !== undefined && present(spec.days)) setAfter(new Date(now.getTime() - parseDays(spec.days) * DAY_MS), "days");
if (present(spec.after)) setAfter(parseDay(spec.after!, "after"), "after"); if (spec.after) setAfter(parseDay(spec.after, "after"), "after");
if (present(spec.before)) setBefore(parseDay(spec.before!, "before"), "before"); if (spec.before) setBefore(parseDay(spec.before, "before"), "before");
if (present(spec.in)) setPeriod(parsePeriod(spec.in!, "in"), "in"); if (spec.in) setPeriod(parsePeriod(spec.in, "in"), "in");
for (const m of (spec.q ?? "").matchAll(/(-?)"([^"]*)"|(\S+)/g)) { for (const m of (spec.q ?? "").matchAll(/(-?)"([^"]*)"|(\S+)/g)) {
if (m[2] !== undefined) { if (m[2] !== undefined) {
@@ -207,15 +246,22 @@ function resolve(spec: FilterSpec, now: Date): Query {
if (after && before && after.value.getTime() >= before.value.getTime()) { if (after && before && after.value.getTime() >= before.value.getTime()) {
throw new Refused(`the window is empty: ${after.from} is not earlier than ${before.from}`); throw new Refused(`the window is empty: ${after.from} is not earlier than ${before.from}`);
} }
const query: Query = { by: by?.value ?? "updated", words, tags }; /** @type {Query} */
const query = { by: by?.value ?? "updated", words, tags };
if (after) query.after = after.value; if (after) query.after = after.value;
if (before) query.before = before.value; if (before) query.before = before.value;
return query; return query;
} }
/** {@link toQuery} over the public page's URL params */ /**
export function parseQuery(params: URLSearchParams, now: Date): Query | QueryError { * {@link toQuery} over the public page's URL params
const field = (name: string) => params.get(name) ?? ""; * @param {URLSearchParams} params
* @param {Date} now
* @returns {Query | QueryError}
*/
export function parseQuery(params, now) {
/** @type {(name: string) => string} */
const field = (name) => params.get(name) ?? "";
const by = field("by"); const by = field("by");
return toQuery( return toQuery(
{ {
@@ -223,34 +269,49 @@ export function parseQuery(params: URLSearchParams, now: Date): Query | QueryErr
after: field("after"), after: field("after"),
before: field("before"), before: field("before"),
in: field("in"), in: field("in"),
by: by as FilterSpec["by"], by: /** @type {FilterSpec["by"]} */ (by),
q: field("q"), q: field("q"),
}, },
now, now,
); );
} }
/** does the query filter anything? (an empty one lists diigo's pages as stored) */ /**
export function isEmpty(query: Query): boolean { * does the query filter anything? (an empty one lists diigo's pages as stored)
* @param {Query} query
* @returns {boolean}
*/
export function isEmpty(query) {
return !query.after && !query.before && !query.words.length && !query.tags.length && query.by === "updated"; return !query.after && !query.before && !query.words.length && !query.tags.length && query.by === "updated";
} }
/** the item's date that `by` names, in epoch seconds (diigo's unit), or /**
* undefined when it carries none */ * the item's date that `by` names, in epoch seconds (diigo's unit), or
export function dateOf(item: Item, by: Query["by"]): number | undefined { * undefined when it carries none
* @param {Item} item
* @param {Query["by"]} by
* @returns {number | undefined}
*/
export function dateOf(item, by) {
const secs = by === "created" ? item.created_at : item.updated_at; const secs = by === "created" ? item.created_at : item.updated_at;
return typeof secs === "number" && Number.isFinite(secs) ? secs : undefined; return typeof secs === "number" && Number.isFinite(secs) ? secs : undefined;
} }
/** does `item` fall in the window and satisfy every term? */ /**
export function matches(item: Item, query: Query): boolean { * does `item` fall in the window and satisfy every term?
* @param {Item} item
* @param {Query} query
* @returns {boolean}
*/
export function matches(item, query) {
if (query.after || query.before) { if (query.after || query.before) {
const secs = dateOf(item, query.by); const secs = dateOf(item, query.by);
if (secs === undefined) return false; if (secs === undefined) return false;
if (query.after && secs * 1000 < query.after.getTime()) return false; if (query.after && secs * 1000 < query.after.getTime()) return false;
if (query.before && secs * 1000 >= query.before.getTime()) return false; if (query.before && secs * 1000 >= query.before.getTime()) return false;
} }
const text = (v: unknown) => (typeof v === "string" ? v : ""); /** @type {(v: unknown) => string} */
const text = (v) => (typeof v === "string" ? v : "");
// fields joined by newline, so a phrase never spans two of them // fields joined by newline, so a phrase never spans two of them
const haystack = [item.title, item.description, item.url].map(text).join("\n").toLowerCase(); const haystack = [item.title, item.description, item.url].map(text).join("\n").toLowerCase();
if (!query.words.every((w) => haystack.includes(w.text) !== w.negated)) return false; if (!query.words.every((w) => haystack.includes(w.text) !== w.negated)) return false;
@@ -262,3 +323,22 @@ export function matches(item: Item, query: Query): boolean {
); );
return query.tags.every((t) => itemTags.has(t.text) !== t.negated); return query.tags.every((t) => itemTags.has(t.text) !== t.negated);
} }
/**
* which of a newest-first list of bookmarks the all-bookmarks page shows
* for search text `q`: the newest `latest` when `q` is blank, else every
* match. A refused search returns only its `error`, so the page can
* leave what it shows alone.
* @param {Item[]} items
* @param {string} q
* @param {Date} now
* @param {number} [latest]
* @returns {{ show: boolean[]; matched?: number } | QueryError}
*/
export function shown(items, q, now, latest = 5) {
if (!q.trim()) return { show: items.map((_, i) => i < latest) };
const query = toQuery({ q }, now);
if ("error" in query) return query;
const show = items.map((item) => matches(item, query));
return { show, matched: show.filter(Boolean).length };
}
+6
View File
@@ -3,3 +3,9 @@ declare module "*.sql" {
const text: string; const text: string;
export default text; export default text;
} }
/** `?raw` imports are a file's text (wrangler.jsonc "rules": Text) */
declare module "*?raw" {
const text: string;
export default text;
}
+50
View File
@@ -5,6 +5,8 @@ import worker from "../src/index.js";
import { idFromWebkey, toWebkey } from "../src/webkeys.js"; import { idFromWebkey, toWebkey } from "../src/webkeys.js";
import { PAGE_SIZE } from "../src/cache.js"; import { PAGE_SIZE } from "../src/cache.js";
import { mf2 } from "microformats-parser"; import { mf2 } from "microformats-parser";
import { shown } from "../src/query.js";
import QUERY_JS from "../src/query.js?raw";
// matches tests/fake-webapp.js: 30 items, every third private // matches tests/fake-webapp.js: 30 items, every third private
const WEBAPP_ID = "fake-webapp-id"; const WEBAPP_ID = "fake-webapp-id";
@@ -411,6 +413,54 @@ describe("public page filters (goal 1d)", () => {
expect(rels.next).toBeUndefined(); expect(rels.next).toBeUndefined();
}); });
it("/all lists every stored shared bookmark, none hidden, as h-entries", async () => {
const { page } = await wireStored("fake-deep-webapp", 3);
const html = await (await call(`${page}/all`)).text();
// 300, one a day, minus every 10th (private)
const want = Array.from({ length: 300 }, (_, k) => k + 1).filter((n) => n % 10 !== 0);
expect(titlesIn(html)).toEqual(dated(...want));
expect(html).not.toMatch(/<li[^>]* hidden/);
const { items } = mf2(html, { baseUrl: `https://diigo-cache.test${page}/all` });
expect(items.find((i) => i.type?.includes("h-feed"))?.children).toHaveLength(270);
// without JS, the search form asks the server
expect(html).toContain(`<form id="search" method="get" action="${page}">`);
expect(html).toContain('import { shown } from "/_query.js"');
});
it("/_query.js serves the worker's own matcher, as a browser module", async () => {
const res = await call("/_query.js");
expect(res.headers.get("content-type")).toMatch(/^text\/javascript/);
const js = await res.text();
expect(js).toBe(QUERY_JS);
expect(js).toMatch(/^export function shown\(/m);
expect(js).not.toMatch(/^import /m);
});
it("searching /all's microformats client-side finds what the server finds", async () => {
const page = await wireDated();
const all = await (await call(`${page}/all`)).text();
const entries = mf2(all, { baseUrl: `https://diigo-cache.test${page}/all` }).items.find((i) => i.type?.includes("h-feed"))?.children ?? [];
// as the page's script reads each h-entry
const secs = (v: unknown) => (v === undefined ? undefined : Date.parse(String(v)) / 1000);
const items = entries.map(({ properties: p }) => ({
title: p.name?.[0],
url: p["bookmark-of"]?.[0],
description: p.summary?.[0],
tags: (p.category ?? []).join(","),
created_at: secs(p.published?.[0]),
updated_at: secs(p.updated?.[0]),
}));
const names = items.map((i) => i.title);
for (const q of ["fragments", "#ocap -#weekly", "newer_than:7d", "created:2026 dated", '"dated 1"']) {
const server = titlesIn(await (await call(`${page}?q=${encodeURIComponent(q)}`)).text());
const result = shown(items, q, new Date());
if ("error" in result) throw new Error(result.error);
// the same matches; /all keeps its updated order, where created: sorts by created
expect(names.filter((_, i) => result.show[i]).sort(), q).toEqual(server.sort());
expect(server.length, q).toBeGreaterThan(0);
}
});
it("filtered pages link the next page with rel=next, keeping the filter", async () => { it("filtered pages link the next page with rel=next, keeping the filter", async () => {
const { page } = await wireStored("fake-deep-webapp", 3); const { page } = await wireStored("fake-deep-webapp", 3);
const html = await (await call(`${page}?days=300`)).text(); const html = await (await call(`${page}?days=300`)).text();
+24 -1
View File
@@ -1,5 +1,5 @@
import { describe, expect, it } from "vitest"; import { describe, expect, it } from "vitest";
import { matches, parseQuery, toQuery } from "../src/query.js"; import { matches, parseQuery, shown, toQuery } from "../src/query.js";
import type { FilterSpec, Query, QueryError } from "../src/query.js"; import type { FilterSpec, Query, QueryError } from "../src/query.js";
// goal 1d (capsite checklist) // goal 1d (capsite checklist)
@@ -204,3 +204,26 @@ describe("matches (goal 1d)", () => {
expect(matches(it1, query({ tags: [{ text: "js", negated: true }] }))).toBe(false); expect(matches(it1, query({ tags: [{ text: "js", negated: true }] }))).toBe(false);
}); });
}); });
describe("shown: the all-bookmarks page's client-side search", () => {
const items = Array.from({ length: 8 }, (_, k) => ({
title: `item ${k}`,
url: `https://example.com/${k}`,
tags: k % 2 ? "odd" : "even",
updated_at: (NOW.getTime() - k * DAY) / 1000,
}));
it("with no search, shows the newest 5", () => {
expect(shown(items, " ", NOW)).toEqual({ show: [true, true, true, true, true, false, false, false] });
});
it("a search shows every match, however old", () => {
expect(shown(items, "#odd", NOW)).toEqual({ show: [false, true, false, true, false, true, false, true], matched: 4 });
// the window's start is inclusive: item 2 is exactly 2 days old
expect(shown(items, "newer_than:2d -#odd", NOW)).toMatchObject({ matched: 2 });
});
it("a refused search returns only its error", () => {
expect(shown(items, "newer_than:soon", NOW)).toEqual({ error: expect.stringMatching(/newer_than/) });
});
});
+2 -1
View File
@@ -2,7 +2,8 @@
"extends": "../tsconfig.json", "extends": "../tsconfig.json",
"compilerOptions": { "compilerOptions": {
"types": ["@cloudflare/vitest-plugin/types"], "types": ["@cloudflare/vitest-plugin/types"],
"allowJs": true "allowJs": true,
"checkJs": false
}, },
"include": ["./**/*", "../worker-configuration.d.ts"] "include": ["./**/*", "../worker-configuration.d.ts"]
} }
+2
View File
@@ -9,6 +9,8 @@
"skipLibCheck": true, "skipLibCheck": true,
"lib": ["ES2022", "ESNext.Disposable"], "lib": ["ES2022", "ESNext.Disposable"],
"noEmit": true, "noEmit": true,
"allowJs": true,
"checkJs": true,
"types": [] "types": []
}, },
"include": ["src/**/*", "worker-configuration.d.ts"] "include": ["src/**/*", "worker-configuration.d.ts"]
+1 -1
View File
@@ -6,7 +6,7 @@
"observability": { "enabled": true }, "observability": { "enabled": true },
// capnweb's browser build, served as text at /_capnweb.js (the worker // capnweb's browser build, served as text at /_capnweb.js (the worker
// itself imports capnweb's workerd build as code) // itself imports capnweb's workerd build as code)
"rules": [{ "type": "Text", "globs": ["**/capnweb/dist/index.js", "**/*.sql"], "fallthrough": true }], "rules": [{ "type": "Text", "globs": ["**/capnweb/dist/index.js", "**/*.sql", "**/*.js?raw"], "fallthrough": true }],
"services": [ "services": [
{ {
"binding": "WEBAPP", "binding": "WEBAPP",
+1 -1
View File
@@ -6,7 +6,7 @@
"name": "diigo-cache-tests", "name": "diigo-cache-tests",
"main": "./tests/test-worker.ts", "main": "./tests/test-worker.ts",
"compatibility_date": "2026-09-29", "compatibility_date": "2026-09-29",
"rules": [{ "type": "Text", "globs": ["**/capnweb/dist/index.js", "**/*.sql"], "fallthrough": true }], "rules": [{ "type": "Text", "globs": ["**/capnweb/dist/index.js", "**/*.sql", "**/*.js?raw"], "fallthrough": true }],
"services": [ "services": [
{ "binding": "WEBAPP", "service": "fake-webapp", "entrypoint": "WebAppFactory" } { "binding": "WEBAPP", "service": "fake-webapp", "entrypoint": "WebAppFactory" }
], ],