WIP: diigo-cache: /public/<username>/all, searched client-side #2
No files matched your search
@@ -10,7 +10,9 @@ diigo-cache is a Cloudflare Worker cache over [the webapp factory](../webapp/REA
|
||||
- `PublicCache` (binding `PUBLIC_DO`): shared items only — private rows are dropped *before* etching (`publicOnly`), so its storage holds nothing private. Same cookie webapp as the private cache. Public page n is diigo page n, filtered (may show fewer than `PAGE_SIZE`, 100: diigo's cap). `/public/<username>` resolves through the `idFromName("index")` registry, and reads through `viewer()`, a read-only facet: it shows stored pages and fill progress, and can't fetch from diigo or write.
|
||||
- **Storage**: SQL lives in `src/cache-read.sql` (selects only) and `src/cache-write.sql`; `src/store.ts` turns them into a reader and a writer facet, so the cache logic holds no SQL text.
|
||||
- **Pages**: home's dump form (cache id + page number, POST) shows raw dump rows; `/public/<username>?page=n` shows an HTML list with a page form (JSON for `Accept: application/json`). Both say `cache: HIT` or `MISS`; every page read also logs `{event: "cache", kind, page, outcome}`.
|
||||
- **Filters on the public page** (`src/query.ts`): `days`, `after`, `before`, `in` (`2026`, `2026-03`, `2026-Q1`, `2026-W41`), `by=updated|created`, and `q` (words, `"phrases"`, `#tag`, `-` negation, gmail/Slack/GitHub date operators). UTC, half-open windows. The same fields work as a dev-side spec (`toQuery({ days: 7 })`). A filter queries the stored `items` table through `viewer()`, never diigo: results are as fresh as the cache, and JSON says `complete` when every diigo page is stored.
|
||||
- **Filters on the public page** (`src/query.js`): `days`, `after`, `before`, `in` (`2026`, `2026-03`, `2026-Q1`, `2026-W41`), `by=updated|created`, and `q` (words, `"phrases"`, `#tag`, `-` negation, gmail/Slack/GitHub date operators). UTC, half-open windows. The same fields work as a dev-side spec (`toQuery({ days: 7 })`). A filter queries the stored `items` table through `viewer()`, never diigo: results are as fresh as the cache, and JSON says `complete` when every diigo page is stored.
|
||||
- **All bookmarks, searched client-side**: `/public/<username>/all` lists every stored shared bookmark in the same microformats. Without JS that's the whole list, and its search form asks the server. With JS (`src/all-page.ts`), the newest 5 show and search hides and shows entries in place, by the server's own matcher: `src/query.js` is plain JS with JSDoc types, so the worker imports it as code and also serves it, as text (`?raw`, a wrangler Text rule), at `/_query.js`.
|
||||
- **All bookmarks, searched client-side**: `/public/<username>/all` lists every stored shared bookmark in the same microformats. Without JS that's the whole list, and its search form asks the server. With JS (`src/all-page.ts`), the newest 5 show and search hides and shows entries in place, by the server's own matcher: `src/query.js` is plain JS with JSDoc types, so the worker imports it as code and also serves it, as text (`?raw`, a wrangler Text rule), at `/_query.js`.
|
||||
- **Publish later**: `POST /_publish` (cache id in the body) wires a public companion to a private-only cache.
|
||||
- **The webkey page and Cap'n Web**: `/cache/#<cache id>` is a skeleton page, the same for every key; its script reads the key from the fragment and trades it in-band for a capability over [Cap'n Web](https://github.com/cloudflare/capnweb) at `/api` (`CacheApi.cache(key)` → a `CacheCap` facet: fill, status, the public companion; `src/api.ts`). The worker serves capnweb's browser build itself at `/_capnweb.js`.
|
||||
- **Fill job**: walks every diigo page into a cache in the background — DO alarms, a few pages per step, progress stored in the DO (`startFill`/`fillStatus`/`stopFill`), driven from the webkey page; the public page shows progress. "Invalidate the last N days" there drops the cached pages holding anything updated that recently (`invalidateRecent`), so the next fill refetches them.
|
||||
|
||||
@@ -19,6 +19,7 @@ Then, in the browser:
|
||||
## Bookmarks you hold, a page anyone can read
|
||||
|
||||
- **Filter by date and search:** `?days=7`, `?in=2026-W41` (ISO week, UTC), `?after=2026-01-01&before=2026-02-01`, `?by=created`, and `?q=` with words, `"phrases"`, `#tags`, `-negation`, and gmail-style `newer_than:7d`. Add `Accept: application/json` for JSON.
|
||||
- **Everything on one page:** `/public/<you>/all` lists every stored shared bookmark; with JS it shows the newest 5 and searches as you type, with the same syntax.
|
||||
- **Private stays private:** the public cache drops private bookmarks before storing anything, so it never holds them.
|
||||
- **Ids are capabilities:** the cache id is a bearer key you keep in your password manager. It rides URL fragments and POST bodies, never a path or query string, so it stays out of logs and history.
|
||||
- **Cheap to read:** pages come from Durable Object SQLite storage, and a fill job in the background fetches each diigo page once.
|
||||
|
||||
@@ -0,0 +1,53 @@
|
||||
/** @file the all-bookmarks page's script: client-side search over the page's own microformats. */
|
||||
|
||||
/**
|
||||
* the script for /public/<username>/all. Without it, the page lists every
|
||||
* stored shared bookmark, and its form searches on the server. With it,
|
||||
* the page shows the newest few and searches as you type, using
|
||||
* `shown()` from /_query.js: the server's own matcher, fed items read
|
||||
* back from each `h-entry`.
|
||||
*/
|
||||
export const ALL_PAGE_SCRIPT = `
|
||||
import { shown } from "/_query.js";
|
||||
|
||||
const entries = [...document.querySelectorAll(".h-feed > .h-entry")];
|
||||
const text = (li, selector) => li.querySelector(selector)?.textContent ?? "";
|
||||
const secs = (li, cls) => {
|
||||
const time = li.querySelector("time." + cls);
|
||||
return time ? Date.parse(time.dateTime) / 1000 : undefined;
|
||||
};
|
||||
const items = entries.map((li) => ({
|
||||
title: text(li, ".p-name"),
|
||||
url: li.querySelector(".u-bookmark-of")?.getAttribute("href") ?? "",
|
||||
description: text(li, ".p-summary"),
|
||||
tags: [...li.querySelectorAll(".p-category")].map((a) => a.textContent).join(","),
|
||||
created_at: secs(li, "dt-published"),
|
||||
updated_at: secs(li, "dt-updated"),
|
||||
}));
|
||||
|
||||
const form = document.getElementById("search");
|
||||
const input = form.elements.q;
|
||||
const status = document.getElementById("all-status");
|
||||
const apply = () => {
|
||||
const result = shown(items, input.value, new Date());
|
||||
if ("error" in result) {
|
||||
status.textContent = result.error;
|
||||
return;
|
||||
}
|
||||
entries.forEach((li, i) => {
|
||||
li.hidden = !result.show[i];
|
||||
});
|
||||
const n = entries.length;
|
||||
status.textContent =
|
||||
result.matched === undefined
|
||||
? \`the newest \${Math.min(5, n)} of \${n}; search to see more\`
|
||||
: \`\${result.matched} of \${n} match\`;
|
||||
};
|
||||
form.addEventListener("submit", (event) => {
|
||||
event.preventDefault();
|
||||
apply();
|
||||
});
|
||||
input.addEventListener("input", apply);
|
||||
input.value = new URLSearchParams(location.search).get("q") ?? "";
|
||||
apply();
|
||||
`;
|
||||
@@ -8,6 +8,7 @@ import { newWorkersRpcResponse } from "capnweb";
|
||||
import { CacheApi } from "./api.js";
|
||||
import { CAPNWEB_BROWSER_JS } from "./capnweb-browser.js";
|
||||
import { CACHE_PAGE_HTML } from "./cache-page.js";
|
||||
import { ALL_PAGE_SCRIPT } from "./all-page.js";
|
||||
import { DiigoCacheLogic, PublicCacheLogic } from "./cache.js";
|
||||
import { PAGE_SIZE } from "./cache.js";
|
||||
import type { CacheEnv, Clock, FillStatus, QueryPage } from "./cache.js";
|
||||
@@ -15,6 +16,7 @@ import type { Item } from "./cache.js";
|
||||
import { usernameOf } from "./items.js";
|
||||
import { dateOf, isEmpty, parseQuery } from "./query.js";
|
||||
import type { Query } from "./query.js";
|
||||
import QUERY_JS from "./query.js?raw";
|
||||
import { toWebkey, idFromWebkey, isWebkey, shashKey } from "./webkeys.js";
|
||||
import { pmSave } from "./pm-save.js";
|
||||
|
||||
@@ -85,6 +87,8 @@ export class DiigoBridge extends WorkerEntrypoint {
|
||||
* - `GET /cache/`: the webkey page ({@link CACHE_PAGE_HTML});
|
||||
* `/api`: its Cap'n Web API ({@link CacheApi}); `GET /_capnweb.js`
|
||||
* - `GET /public/<username>`: public pages, read-only ({@link publicPage})
|
||||
* - `GET /public/<username>/all`: every stored shared bookmark, searched
|
||||
* client-side ({@link allPage}); `GET /_query.js`, its matcher
|
||||
*
|
||||
* Cache ids are webkeys (docs/webkeys.md): they ride POST bodies and URL
|
||||
* fragments, never query strings or paths.
|
||||
@@ -95,6 +99,8 @@ export default {
|
||||
|
||||
const pub = url.pathname.match(/^\/public\/([A-Za-z0-9_-]+)$/);
|
||||
if (pub) return publicPage(request, env, pub[1]);
|
||||
const all = url.pathname.match(/^\/public\/([A-Za-z0-9_-]+)\/all$/);
|
||||
if (all) return allPage(env, all[1]);
|
||||
|
||||
if (url.pathname === "/_connect" && request.method === "GET") {
|
||||
return mintPage(env);
|
||||
@@ -114,6 +120,9 @@ export default {
|
||||
if (url.pathname === "/_capnweb.js" && request.method === "GET") {
|
||||
return new Response(CAPNWEB_BROWSER_JS, { headers: { "content-type": "text/javascript;charset=utf-8" } });
|
||||
}
|
||||
if (url.pathname === "/_query.js" && request.method === "GET") {
|
||||
return new Response(QUERY_JS, { headers: { "content-type": "text/javascript;charset=utf-8" } });
|
||||
}
|
||||
if (url.pathname === "/api") {
|
||||
return newWorkersRpcResponse(request, new CacheApi(env));
|
||||
}
|
||||
@@ -458,11 +467,8 @@ const DAY_PRESETS = [7, 30, 90, 365];
|
||||
* fine here: nothing on this page is secret.
|
||||
*/
|
||||
async function publicPage(request: Request, env: Env, username: string): Promise<Response> {
|
||||
const index = env.PUBLIC_DO.get(env.PUBLIC_DO.idFromName("index"));
|
||||
const publicId = await index.lookup(username);
|
||||
if (!publicId) return new Response("no such public cache", { status: 404 });
|
||||
const ns = env.PUBLIC_DO;
|
||||
const cache = ns.get(ns.idFromString(idFromWebkey(publicId)));
|
||||
const cache = await publicCacheOf(env, username);
|
||||
if (!cache) return new Response("no such public cache", { status: 404 });
|
||||
|
||||
const params = new URL(request.url).searchParams;
|
||||
const accept = request.headers.get("accept") ?? "";
|
||||
@@ -534,6 +540,53 @@ ${notice}${list}`,
|
||||
return new Response(body, { headers: { "content-type": "text/html;charset=utf-8" } });
|
||||
}
|
||||
|
||||
/** the public cache registered for `username`, if any */
|
||||
async function publicCacheOf(env: Env, username: string) {
|
||||
const index = env.PUBLIC_DO.get(env.PUBLIC_DO.idFromName("index"));
|
||||
const publicId = await index.lookup(username);
|
||||
if (!publicId) return undefined;
|
||||
return env.PUBLIC_DO.get(env.PUBLIC_DO.idFromString(idFromWebkey(publicId)));
|
||||
}
|
||||
|
||||
/**
|
||||
* every stored shared bookmark on one page, newest updated first, as the
|
||||
* same microformats as the public page. Without JS, that's the whole
|
||||
* list, and the search form asks the public page. With JS
|
||||
* ({@link ALL_PAGE_SCRIPT}), the newest few show, and search hides and
|
||||
* shows entries in place with the server's matcher.
|
||||
*/
|
||||
async function allPage(env: Env, username: string): Promise<Response> {
|
||||
const cache = await publicCacheOf(env, username);
|
||||
if (!cache) return new Response("no such public cache", { status: 404 });
|
||||
const viewer = await cache.viewer();
|
||||
let found, fill;
|
||||
try {
|
||||
found = await viewer.query({ by: "updated", words: [], tags: [] }, 1, Number.MAX_SAFE_INTEGER);
|
||||
fill = await viewer.fillStatus();
|
||||
} catch (err) {
|
||||
console.error({ event: "all-page", error: String(err) });
|
||||
return new Response(`could not load bookmarks: ${String(err)}`, { status: 502 });
|
||||
} finally {
|
||||
viewer[Symbol.dispose]();
|
||||
}
|
||||
const here = `/public/${encodeURIComponent(username)}`;
|
||||
const items = found.items.map((j) => JSON.parse(j) as Item);
|
||||
const caveat = found.complete ? "" : " — not every diigo page is stored yet, so some may be missing";
|
||||
const body = publicFrame(
|
||||
username,
|
||||
`<p id="all-status">${items.length} shared bookmark${items.length === 1 ? "" : "s"}, newest first${caveat}</p>
|
||||
${fill ? fillLine(fill) : ""}
|
||||
<form id="search" method="get" action="${here}">
|
||||
<p><label>search <input type="search" name="q" size="40"></label> <button type="submit">search</button></p>
|
||||
<p><small>words, "phrases", #tags, -negation, and after:, before:, newer_than:7d, during:2026-Q1, created:2026 (UTC)</small></p>
|
||||
</form>
|
||||
<p><a href="${here}">by diigo page</a></p>
|
||||
${itemList(items, "updated", here)}
|
||||
<script type="module">${ALL_PAGE_SCRIPT}</script>`,
|
||||
);
|
||||
return new Response(body, { headers: { "content-type": "text/html;charset=utf-8" } });
|
||||
}
|
||||
|
||||
/** a filter's matches, as the public page's HTML or JSON */
|
||||
function filteredPage(
|
||||
username: string,
|
||||
|
||||
@@ -1,10 +1,14 @@
|
||||
/**
|
||||
* @file the public page's filters: date windows and search over a cache's
|
||||
* stored items (goal 1d; tests/query.test.ts).
|
||||
*
|
||||
* Plain JS with JSDoc types, so the worker imports it as code and serves
|
||||
* the same file to browsers at /_query.js: the all-bookmarks page searches
|
||||
* client-side with exactly the server's matcher.
|
||||
*
|
||||
* @import { Item } from "./items.js"
|
||||
*/
|
||||
|
||||
import type { Item } from "./items.js";
|
||||
|
||||
/**
|
||||
* a filter as devs write it (`{ days: 7, q: "#weekly" }`); URL params map
|
||||
* onto these fields one-to-one (strings, with "" meaning absent). Dates
|
||||
@@ -12,49 +16,49 @@ import type { Item } from "./items.js";
|
||||
* `2025-03`, `2025-Q1`, or ISO week `2025-W01`. Time is UTC throughout;
|
||||
* ISO weeks start Monday, keeping a weekend together.
|
||||
*/
|
||||
export type FilterSpec = {
|
||||
/** the last N days, counted back from now (1–3650) */
|
||||
days?: number | string;
|
||||
/** inclusive lower bound, from 00:00Z */
|
||||
after?: string;
|
||||
/** exclusive upper bound, at 00:00Z */
|
||||
before?: string;
|
||||
/** a named period: shorthand for its after/before pair */
|
||||
in?: string;
|
||||
/** which date the window, the listed date, and the sort use */
|
||||
by?: "updated" | "created" | "";
|
||||
/** words, phrases, tags, and operators */
|
||||
q?: string;
|
||||
};
|
||||
/**
|
||||
* @typedef {object} FilterSpec
|
||||
* @property {number | string} [days] the last N days, counted back from now (1–3650)
|
||||
* @property {string} [after] inclusive lower bound, from 00:00Z
|
||||
* @property {string} [before] exclusive upper bound, at 00:00Z
|
||||
* @property {string} [in] a named period: shorthand for its after/before pair
|
||||
* @property {"updated" | "created" | ""} [by] which date the window, the listed date, and the sort use
|
||||
* @property {string} [q] words, phrases, tags, and operators
|
||||
*/
|
||||
|
||||
/** one search term; `negated` for a leading `-` */
|
||||
export type Term = { text: string; negated: boolean };
|
||||
/**
|
||||
* one search term; `negated` for a leading `-`
|
||||
* @typedef {{ text: string; negated: boolean }} Term
|
||||
*/
|
||||
|
||||
/** a parsed filter; an empty one (no window, no terms) filters nothing */
|
||||
export type Query = {
|
||||
/** which date the window, the listed date, and the sort use (default
|
||||
* `updated`, diigo's own order) */
|
||||
by: "updated" | "created";
|
||||
/** inclusive lower bound */
|
||||
after?: Date;
|
||||
/** exclusive upper bound */
|
||||
before?: Date;
|
||||
/** words and phrases: each must appear in title, description, or url */
|
||||
words: Term[];
|
||||
/** tags: exact, case-insensitive match against the item's tags */
|
||||
tags: Term[];
|
||||
};
|
||||
/**
|
||||
* a parsed filter; an empty one (no window, no terms) filters nothing
|
||||
* @typedef {object} Query
|
||||
* @property {"updated" | "created"} by which date the window, the listed
|
||||
* date, and the sort use (default `updated`, diigo's own order)
|
||||
* @property {Date} [after] inclusive lower bound
|
||||
* @property {Date} [before] exclusive upper bound
|
||||
* @property {Term[]} words words and phrases: each must appear in title, description, or url
|
||||
* @property {Term[]} tags tags: exact, case-insensitive match against the item's tags
|
||||
*/
|
||||
|
||||
/** a refused filter: the message is safe to show and log */
|
||||
export type QueryError = { error: string };
|
||||
/**
|
||||
* a refused filter: the message is safe to show and log
|
||||
* @typedef {{ error: string }} QueryError
|
||||
*/
|
||||
|
||||
const DAY_MS = 24 * 60 * 60 * 1000;
|
||||
|
||||
/** thrown inside {@link toQuery}, returned as a {@link QueryError} */
|
||||
class Refused extends Error {}
|
||||
|
||||
/** `yyyy-mm-dd` or `yyyy/mm/dd` as 00:00Z of that UTC day */
|
||||
function parseDay(text: string, what: string): Date {
|
||||
/**
|
||||
* `yyyy-mm-dd` or `yyyy/mm/dd` as 00:00Z of that UTC day
|
||||
* @param {string} text
|
||||
* @param {string} what
|
||||
* @returns {Date}
|
||||
*/
|
||||
function parseDay(text, what) {
|
||||
const m = /^(\d{4})([-/])(\d{2})\2(\d{2})$/.exec(text);
|
||||
const d = m && new Date(Date.UTC(Number(m[1]), Number(m[3]) - 1, Number(m[4])));
|
||||
if (!m || !d || d.getUTCMonth() !== Number(m[3]) - 1 || d.getUTCDate() !== Number(m[4])) {
|
||||
@@ -63,15 +67,24 @@ function parseDay(text: string, what: string): Date {
|
||||
return d;
|
||||
}
|
||||
|
||||
/** Monday 00:00Z of ISO week 1 of `year` (the week holding January 4) */
|
||||
function isoWeekOne(year: number): number {
|
||||
/**
|
||||
* Monday 00:00Z of ISO week 1 of `year` (the week holding January 4)
|
||||
* @param {number} year
|
||||
* @returns {number}
|
||||
*/
|
||||
function isoWeekOne(year) {
|
||||
const jan4 = Date.UTC(year, 0, 4);
|
||||
const weekday = new Date(jan4).getUTCDay() || 7;
|
||||
return jan4 - (weekday - 1) * DAY_MS;
|
||||
}
|
||||
|
||||
/** a named period as its half-open [after, before) window */
|
||||
function parsePeriod(text: string, what: string): [Date, Date] {
|
||||
/**
|
||||
* a named period as its half-open [after, before) window
|
||||
* @param {string} text
|
||||
* @param {string} what
|
||||
* @returns {[Date, Date]}
|
||||
*/
|
||||
function parsePeriod(text, what) {
|
||||
const refuse = () => new Refused(`${what}: expected a period as 2025, 2025-03, 2025-Q1, or 2025-W01`);
|
||||
let m;
|
||||
if ((m = /^(\d{4})$/.exec(text))) {
|
||||
@@ -97,9 +110,14 @@ function parsePeriod(text: string, what: string): [Date, Date] {
|
||||
throw refuse();
|
||||
}
|
||||
|
||||
/** gmail's `newer_than:` amount (`7d`, `2w`, `1m`, `1y`) counted back from
|
||||
* `now`; months and years are calendar steps */
|
||||
function parseNewerThan(text: string, now: Date): Date {
|
||||
/**
|
||||
* gmail's `newer_than:` amount (`7d`, `2w`, `1m`, `1y`) counted back from
|
||||
* `now`; months and years are calendar steps
|
||||
* @param {string} text
|
||||
* @param {Date} now
|
||||
* @returns {Date}
|
||||
*/
|
||||
function parseNewerThan(text, now) {
|
||||
const m = /^(\d{1,4})([dwmy])$/.exec(text);
|
||||
if (!m || Number(m[1]) < 1) throw new Refused("newer_than: expected an amount like 7d, 2w, 1m, or 1y");
|
||||
const n = Number(m[1]);
|
||||
@@ -111,8 +129,12 @@ function parseNewerThan(text: string, now: Date): Date {
|
||||
return d;
|
||||
}
|
||||
|
||||
/** `days` as a whole number from 1 to 3650 */
|
||||
function parseDays(days: number | string): number {
|
||||
/**
|
||||
* `days` as a whole number from 1 to 3650
|
||||
* @param {number | string} days
|
||||
* @returns {number}
|
||||
*/
|
||||
function parseDays(days) {
|
||||
const n = typeof days === "number" ? days : /^\d+$/.test(days) ? Number(days) : NaN;
|
||||
if (!Number.isInteger(n) || n < 1 || n > 3650) throw new Refused("days: expected a whole number from 1 to 3650");
|
||||
return n;
|
||||
@@ -133,10 +155,12 @@ function parseDays(days: number | string): number {
|
||||
* @see https://slack.com/help/articles/202528808
|
||||
* @see https://docs.github.com/en/search-github/searching-on-github/searching-for-repositories
|
||||
*
|
||||
* @param now the clock `days`/`newer_than` count back from (passed in,
|
||||
* not read ambiently, so tests pin it)
|
||||
* @param {FilterSpec} spec
|
||||
* @param {Date} now the clock `days`/`newer_than` count back from (passed
|
||||
* in, not read ambiently, so tests pin it)
|
||||
* @returns {Query | QueryError}
|
||||
*/
|
||||
export function toQuery(spec: FilterSpec, now: Date): Query | QueryError {
|
||||
export function toQuery(spec, now) {
|
||||
try {
|
||||
return resolve(spec, now);
|
||||
} catch (err) {
|
||||
@@ -145,39 +169,54 @@ export function toQuery(spec: FilterSpec, now: Date): Query | QueryError {
|
||||
}
|
||||
}
|
||||
|
||||
function resolve(spec: FilterSpec, now: Date): Query {
|
||||
const words: Term[] = [];
|
||||
const tags: Term[] = [];
|
||||
let by: { value: Query["by"]; from: string } | undefined;
|
||||
let after: { value: Date; from: string } | undefined;
|
||||
let before: { value: Date; from: string } | undefined;
|
||||
/**
|
||||
* @param {FilterSpec} spec
|
||||
* @param {Date} now
|
||||
* @returns {Query}
|
||||
*/
|
||||
function resolve(spec, now) {
|
||||
/** @type {Term[]} */
|
||||
const words = [];
|
||||
/** @type {Term[]} */
|
||||
const tags = [];
|
||||
/** @type {{ value: Query["by"]; from: string } | undefined} */
|
||||
let by;
|
||||
/** @type {{ value: Date; from: string } | undefined} */
|
||||
let after;
|
||||
/** @type {{ value: Date; from: string } | undefined} */
|
||||
let before;
|
||||
|
||||
const setBy = (value: Query["by"], from: string) => {
|
||||
/** @type {(value: Query["by"], from: string) => void} */
|
||||
const setBy = (value, from) => {
|
||||
if (by && by.value !== value) throw new Refused(`${by.from} and ${from} name different dates`);
|
||||
by = { value, from };
|
||||
};
|
||||
const setAfter = (value: Date, from: string) => {
|
||||
/** @type {(value: Date, from: string) => void} */
|
||||
const setAfter = (value, from) => {
|
||||
if (after) throw new Refused(`${after.from} and ${from} both set where the window starts`);
|
||||
after = { value, from };
|
||||
};
|
||||
const setBefore = (value: Date, from: string) => {
|
||||
/** @type {(value: Date, from: string) => void} */
|
||||
const setBefore = (value, from) => {
|
||||
if (before) throw new Refused(`${before.from} and ${from} both set where the window ends`);
|
||||
before = { value, from };
|
||||
};
|
||||
const setPeriod = ([lo, hi]: [Date, Date], from: string) => {
|
||||
/** @type {(window: [Date, Date], from: string) => void} */
|
||||
const setPeriod = ([lo, hi], from) => {
|
||||
setAfter(lo, from);
|
||||
setBefore(hi, from);
|
||||
};
|
||||
|
||||
const present = (v: string | number | undefined) => v !== undefined && v !== "";
|
||||
/** @type {(v: string | number | undefined) => boolean} */
|
||||
const present = (v) => v !== undefined && v !== "";
|
||||
if (present(spec.by)) {
|
||||
if (spec.by !== "updated" && spec.by !== "created") throw new Refused("by: expected updated or created");
|
||||
setBy(spec.by, `by=${spec.by}`);
|
||||
}
|
||||
if (present(spec.days)) setAfter(new Date(now.getTime() - parseDays(spec.days!) * DAY_MS), "days");
|
||||
if (present(spec.after)) setAfter(parseDay(spec.after!, "after"), "after");
|
||||
if (present(spec.before)) setBefore(parseDay(spec.before!, "before"), "before");
|
||||
if (present(spec.in)) setPeriod(parsePeriod(spec.in!, "in"), "in");
|
||||
if (spec.days !== undefined && present(spec.days)) setAfter(new Date(now.getTime() - parseDays(spec.days) * DAY_MS), "days");
|
||||
if (spec.after) setAfter(parseDay(spec.after, "after"), "after");
|
||||
if (spec.before) setBefore(parseDay(spec.before, "before"), "before");
|
||||
if (spec.in) setPeriod(parsePeriod(spec.in, "in"), "in");
|
||||
|
||||
for (const m of (spec.q ?? "").matchAll(/(-?)"([^"]*)"|(\S+)/g)) {
|
||||
if (m[2] !== undefined) {
|
||||
@@ -207,15 +246,22 @@ function resolve(spec: FilterSpec, now: Date): Query {
|
||||
if (after && before && after.value.getTime() >= before.value.getTime()) {
|
||||
throw new Refused(`the window is empty: ${after.from} is not earlier than ${before.from}`);
|
||||
}
|
||||
const query: Query = { by: by?.value ?? "updated", words, tags };
|
||||
/** @type {Query} */
|
||||
const query = { by: by?.value ?? "updated", words, tags };
|
||||
if (after) query.after = after.value;
|
||||
if (before) query.before = before.value;
|
||||
return query;
|
||||
}
|
||||
|
||||
/** {@link toQuery} over the public page's URL params */
|
||||
export function parseQuery(params: URLSearchParams, now: Date): Query | QueryError {
|
||||
const field = (name: string) => params.get(name) ?? "";
|
||||
/**
|
||||
* {@link toQuery} over the public page's URL params
|
||||
* @param {URLSearchParams} params
|
||||
* @param {Date} now
|
||||
* @returns {Query | QueryError}
|
||||
*/
|
||||
export function parseQuery(params, now) {
|
||||
/** @type {(name: string) => string} */
|
||||
const field = (name) => params.get(name) ?? "";
|
||||
const by = field("by");
|
||||
return toQuery(
|
||||
{
|
||||
@@ -223,34 +269,49 @@ export function parseQuery(params: URLSearchParams, now: Date): Query | QueryErr
|
||||
after: field("after"),
|
||||
before: field("before"),
|
||||
in: field("in"),
|
||||
by: by as FilterSpec["by"],
|
||||
by: /** @type {FilterSpec["by"]} */ (by),
|
||||
q: field("q"),
|
||||
},
|
||||
now,
|
||||
);
|
||||
}
|
||||
|
||||
/** does the query filter anything? (an empty one lists diigo's pages as stored) */
|
||||
export function isEmpty(query: Query): boolean {
|
||||
/**
|
||||
* does the query filter anything? (an empty one lists diigo's pages as stored)
|
||||
* @param {Query} query
|
||||
* @returns {boolean}
|
||||
*/
|
||||
export function isEmpty(query) {
|
||||
return !query.after && !query.before && !query.words.length && !query.tags.length && query.by === "updated";
|
||||
}
|
||||
|
||||
/** the item's date that `by` names, in epoch seconds (diigo's unit), or
|
||||
* undefined when it carries none */
|
||||
export function dateOf(item: Item, by: Query["by"]): number | undefined {
|
||||
/**
|
||||
* the item's date that `by` names, in epoch seconds (diigo's unit), or
|
||||
* undefined when it carries none
|
||||
* @param {Item} item
|
||||
* @param {Query["by"]} by
|
||||
* @returns {number | undefined}
|
||||
*/
|
||||
export function dateOf(item, by) {
|
||||
const secs = by === "created" ? item.created_at : item.updated_at;
|
||||
return typeof secs === "number" && Number.isFinite(secs) ? secs : undefined;
|
||||
}
|
||||
|
||||
/** does `item` fall in the window and satisfy every term? */
|
||||
export function matches(item: Item, query: Query): boolean {
|
||||
/**
|
||||
* does `item` fall in the window and satisfy every term?
|
||||
* @param {Item} item
|
||||
* @param {Query} query
|
||||
* @returns {boolean}
|
||||
*/
|
||||
export function matches(item, query) {
|
||||
if (query.after || query.before) {
|
||||
const secs = dateOf(item, query.by);
|
||||
if (secs === undefined) return false;
|
||||
if (query.after && secs * 1000 < query.after.getTime()) return false;
|
||||
if (query.before && secs * 1000 >= query.before.getTime()) return false;
|
||||
}
|
||||
const text = (v: unknown) => (typeof v === "string" ? v : "");
|
||||
/** @type {(v: unknown) => string} */
|
||||
const text = (v) => (typeof v === "string" ? v : "");
|
||||
// fields joined by newline, so a phrase never spans two of them
|
||||
const haystack = [item.title, item.description, item.url].map(text).join("\n").toLowerCase();
|
||||
if (!query.words.every((w) => haystack.includes(w.text) !== w.negated)) return false;
|
||||
@@ -262,3 +323,22 @@ export function matches(item: Item, query: Query): boolean {
|
||||
);
|
||||
return query.tags.every((t) => itemTags.has(t.text) !== t.negated);
|
||||
}
|
||||
|
||||
/**
|
||||
* which of a newest-first list of bookmarks the all-bookmarks page shows
|
||||
* for search text `q`: the newest `latest` when `q` is blank, else every
|
||||
* match. A refused search returns only its `error`, so the page can
|
||||
* leave what it shows alone.
|
||||
* @param {Item[]} items
|
||||
* @param {string} q
|
||||
* @param {Date} now
|
||||
* @param {number} [latest]
|
||||
* @returns {{ show: boolean[]; matched?: number } | QueryError}
|
||||
*/
|
||||
export function shown(items, q, now, latest = 5) {
|
||||
if (!q.trim()) return { show: items.map((_, i) => i < latest) };
|
||||
const query = toQuery({ q }, now);
|
||||
if ("error" in query) return query;
|
||||
const show = items.map((item) => matches(item, query));
|
||||
return { show, matched: show.filter(Boolean).length };
|
||||
}
|
||||
Vendored
+6
@@ -3,3 +3,9 @@ declare module "*.sql" {
|
||||
const text: string;
|
||||
export default text;
|
||||
}
|
||||
|
||||
/** `?raw` imports are a file's text (wrangler.jsonc "rules": Text) */
|
||||
declare module "*?raw" {
|
||||
const text: string;
|
||||
export default text;
|
||||
}
|
||||
@@ -5,6 +5,8 @@ import worker from "../src/index.js";
|
||||
import { idFromWebkey, toWebkey } from "../src/webkeys.js";
|
||||
import { PAGE_SIZE } from "../src/cache.js";
|
||||
import { mf2 } from "microformats-parser";
|
||||
import { shown } from "../src/query.js";
|
||||
import QUERY_JS from "../src/query.js?raw";
|
||||
|
||||
// matches tests/fake-webapp.js: 30 items, every third private
|
||||
const WEBAPP_ID = "fake-webapp-id";
|
||||
@@ -411,6 +413,54 @@ describe("public page filters (goal 1d)", () => {
|
||||
expect(rels.next).toBeUndefined();
|
||||
});
|
||||
|
||||
it("/all lists every stored shared bookmark, none hidden, as h-entries", async () => {
|
||||
const { page } = await wireStored("fake-deep-webapp", 3);
|
||||
const html = await (await call(`${page}/all`)).text();
|
||||
// 300, one a day, minus every 10th (private)
|
||||
const want = Array.from({ length: 300 }, (_, k) => k + 1).filter((n) => n % 10 !== 0);
|
||||
expect(titlesIn(html)).toEqual(dated(...want));
|
||||
expect(html).not.toMatch(/<li[^>]* hidden/);
|
||||
const { items } = mf2(html, { baseUrl: `https://diigo-cache.test${page}/all` });
|
||||
expect(items.find((i) => i.type?.includes("h-feed"))?.children).toHaveLength(270);
|
||||
// without JS, the search form asks the server
|
||||
expect(html).toContain(`<form id="search" method="get" action="${page}">`);
|
||||
expect(html).toContain('import { shown } from "/_query.js"');
|
||||
});
|
||||
|
||||
it("/_query.js serves the worker's own matcher, as a browser module", async () => {
|
||||
const res = await call("/_query.js");
|
||||
expect(res.headers.get("content-type")).toMatch(/^text\/javascript/);
|
||||
const js = await res.text();
|
||||
expect(js).toBe(QUERY_JS);
|
||||
expect(js).toMatch(/^export function shown\(/m);
|
||||
expect(js).not.toMatch(/^import /m);
|
||||
});
|
||||
|
||||
it("searching /all's microformats client-side finds what the server finds", async () => {
|
||||
const page = await wireDated();
|
||||
const all = await (await call(`${page}/all`)).text();
|
||||
const entries = mf2(all, { baseUrl: `https://diigo-cache.test${page}/all` }).items.find((i) => i.type?.includes("h-feed"))?.children ?? [];
|
||||
// as the page's script reads each h-entry
|
||||
const secs = (v: unknown) => (v === undefined ? undefined : Date.parse(String(v)) / 1000);
|
||||
const items = entries.map(({ properties: p }) => ({
|
||||
title: p.name?.[0],
|
||||
url: p["bookmark-of"]?.[0],
|
||||
description: p.summary?.[0],
|
||||
tags: (p.category ?? []).join(","),
|
||||
created_at: secs(p.published?.[0]),
|
||||
updated_at: secs(p.updated?.[0]),
|
||||
}));
|
||||
const names = items.map((i) => i.title);
|
||||
for (const q of ["fragments", "#ocap -#weekly", "newer_than:7d", "created:2026 dated", '"dated 1"']) {
|
||||
const server = titlesIn(await (await call(`${page}?q=${encodeURIComponent(q)}`)).text());
|
||||
const result = shown(items, q, new Date());
|
||||
if ("error" in result) throw new Error(result.error);
|
||||
// the same matches; /all keeps its updated order, where created: sorts by created
|
||||
expect(names.filter((_, i) => result.show[i]).sort(), q).toEqual(server.sort());
|
||||
expect(server.length, q).toBeGreaterThan(0);
|
||||
}
|
||||
});
|
||||
|
||||
it("filtered pages link the next page with rel=next, keeping the filter", async () => {
|
||||
const { page } = await wireStored("fake-deep-webapp", 3);
|
||||
const html = await (await call(`${page}?days=300`)).text();
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
import { describe, expect, it } from "vitest";
|
||||
import { matches, parseQuery, toQuery } from "../src/query.js";
|
||||
import { matches, parseQuery, shown, toQuery } from "../src/query.js";
|
||||
import type { FilterSpec, Query, QueryError } from "../src/query.js";
|
||||
|
||||
// goal 1d (capsite checklist)
|
||||
@@ -204,3 +204,26 @@ describe("matches (goal 1d)", () => {
|
||||
expect(matches(it1, query({ tags: [{ text: "js", negated: true }] }))).toBe(false);
|
||||
});
|
||||
});
|
||||
|
||||
describe("shown: the all-bookmarks page's client-side search", () => {
|
||||
const items = Array.from({ length: 8 }, (_, k) => ({
|
||||
title: `item ${k}`,
|
||||
url: `https://example.com/${k}`,
|
||||
tags: k % 2 ? "odd" : "even",
|
||||
updated_at: (NOW.getTime() - k * DAY) / 1000,
|
||||
}));
|
||||
|
||||
it("with no search, shows the newest 5", () => {
|
||||
expect(shown(items, " ", NOW)).toEqual({ show: [true, true, true, true, true, false, false, false] });
|
||||
});
|
||||
|
||||
it("a search shows every match, however old", () => {
|
||||
expect(shown(items, "#odd", NOW)).toEqual({ show: [false, true, false, true, false, true, false, true], matched: 4 });
|
||||
// the window's start is inclusive: item 2 is exactly 2 days old
|
||||
expect(shown(items, "newer_than:2d -#odd", NOW)).toMatchObject({ matched: 2 });
|
||||
});
|
||||
|
||||
it("a refused search returns only its error", () => {
|
||||
expect(shown(items, "newer_than:soon", NOW)).toEqual({ error: expect.stringMatching(/newer_than/) });
|
||||
});
|
||||
});
|
||||
@@ -2,7 +2,8 @@
|
||||
"extends": "../tsconfig.json",
|
||||
"compilerOptions": {
|
||||
"types": ["@cloudflare/vitest-plugin/types"],
|
||||
"allowJs": true
|
||||
"allowJs": true,
|
||||
"checkJs": false
|
||||
},
|
||||
"include": ["./**/*", "../worker-configuration.d.ts"]
|
||||
}
|
||||
@@ -9,6 +9,8 @@
|
||||
"skipLibCheck": true,
|
||||
"lib": ["ES2022", "ESNext.Disposable"],
|
||||
"noEmit": true,
|
||||
"allowJs": true,
|
||||
"checkJs": true,
|
||||
"types": []
|
||||
},
|
||||
"include": ["src/**/*", "worker-configuration.d.ts"]
|
||||
|
||||
@@ -6,7 +6,7 @@
|
||||
"observability": { "enabled": true },
|
||||
// capnweb's browser build, served as text at /_capnweb.js (the worker
|
||||
// itself imports capnweb's workerd build as code)
|
||||
"rules": [{ "type": "Text", "globs": ["**/capnweb/dist/index.js", "**/*.sql"], "fallthrough": true }],
|
||||
"rules": [{ "type": "Text", "globs": ["**/capnweb/dist/index.js", "**/*.sql", "**/*.js?raw"], "fallthrough": true }],
|
||||
"services": [
|
||||
{
|
||||
"binding": "WEBAPP",
|
||||
|
||||
@@ -6,7 +6,7 @@
|
||||
"name": "diigo-cache-tests",
|
||||
"main": "./tests/test-worker.ts",
|
||||
"compatibility_date": "2026-09-29",
|
||||
"rules": [{ "type": "Text", "globs": ["**/capnweb/dist/index.js", "**/*.sql"], "fallthrough": true }],
|
||||
"rules": [{ "type": "Text", "globs": ["**/capnweb/dist/index.js", "**/*.sql", "**/*.js?raw"], "fallthrough": true }],
|
||||
"services": [
|
||||
{ "binding": "WEBAPP", "service": "fake-webapp", "entrypoint": "WebAppFactory" }
|
||||
],
|
||||
|
||||
Reference in new issue
Block a user