From 9c3e6fa39675970e43a733c27f9f003795120c80 Mon Sep 17 00:00:00 2001 From: Dan Connolly Date: Wed, 7 Oct 2026 12:58:55 -0500 Subject: [PATCH] capsite: fedi-cache keeps the token; ActivityPub sources come later The existing export (mastodon-archive) reads favourites, bookmarks, and mentions, which ActivityPub doesn't carry. So the client API with a narrow read-only token (held in webapp) is the primary source; items are still keyed by ActivityPub uri, and outbox/follower-actor sources move to a later portability goal. Co-Authored-By: Claude Opus 5.5 (1M context) --- social-coop-checklist.md | 148 +++++++++++++++++++++------------------ 1 file changed, 78 insertions(+), 70 deletions(-) diff --git a/social-coop-checklist.md b/social-coop-checklist.md index 51658c7..90d1c1b 100644 --- a/social-coop-checklist.md +++ b/social-coop-checklist.md @@ -6,88 +6,91 @@ sit under each goal. - [ ] **madmode's weekly roundup includes my fediverse activity** ([#23](https://github.com/dckc/madmode-blog/issues/23), PESOS): - `fedi-cache` holds the posts and boosts of - [@dckc@social.coop](https://social.coop/@dckc), speaking - ActivityPub only — no server-specific API, so the cache works - against any server and survives an account move. social.coop - grants it nothing; madmode holds no credential + `fedi-cache` holds the posts, boosts, favourites, bookmarks, and + mentions of [@dckc@social.coop](https://social.coop/@dckc). The + access token stays inside webapp, so madmode holds no credential. + Mastodon's client API is one *source* behind the cache, not its + model: items are keyed by their ActivityPub `uri`, and what the + protocol can serve, it serves - [ ] **0. Decide the shape** (user's call before code): - [ ] 0-i. A new worker `fedi-cache/` beside `diigo-cache/`; extract shared pieces (date window/`q` query module, fill job, webkey page) once two caches work. - - [ ] 0-ii. Pull, push, or both? **Pull**: read the actor's - `outbox` (an `OrderedCollection` of `Create` and - `Announce`) — public posts only. **Push**: fedi-cache is - itself an actor that follows @dckc; social.coop delivers - `Create`/`Update`/`Delete`/`Announce` to its inbox, - including followers-only posts once the follow is - approved. Recommendation: both — outbox for backfill, - inbox for freshness. - - [ ] 0-iii. What's out of reach at the protocol level: likes - (`Like` goes to the post's author, not to followers, and - Mastodon serves no `liked` collection) and bookmarks (not - ActivityPub at all). Accept the gap, or take them from - the account archive (`likes.json`, `bookmarks.json`) as a - manual import. + - [ ] 0-ii. Which collections reach the public page? Posts and + boosts with `visibility: public`, surely. Favourites + (1,403 in the May 2025 archive) are the richest PESOS + vein but were never published by me; bookmarks are + private by nature. Opt-in per collection, or per item? - - [ ] **1. Authority**: fedi-cache's own actor keypair, generated in - its DO and never exported; HTTP Signatures on outgoing GETs - (servers in authorized-fetch mode require them) and on the - `Follow`. The authority it holds is exactly what @dckc grants by - approving the follow, and removing the follower revokes it. No - token, no cookie, so webapp isn't needed. Ids → password manager, - as with diigo. + - [ ] **1. Credential discipline**: a new token from social.coop → + Preferences → Development → New application, scopes + **read-only and narrow** (`read:statuses read:accounts + read:favourites read:bookmarks read:notifications`, for + mentions) — not mastodon-archive's token, whose scopes are + whatever that tool asked for. It rides as + `Authorization: Bearer`, which webapp's mint form already + blanks. Token → webapp DO storage only; ids → password + manager. Revoking the app on social.coop is the kill switch. - - [ ] **2. Pull: the outbox cache**: - - [ ] 2-i. Resolve the actor: WebFinger - (`/.well-known/webfinger?resource=acct:dckc@social.coop`) - → actor document → `outbox`. - - [ ] 2-ii. Storage: activities and objects keyed by their - `id` URI, objects with `updated`. **Durable state** from + - [ ] **2. The cache** (`fedi-cache/`, the 1c pattern from diigo): + - [ ] 2-i. Mint and wire against `https://social.coop/api/`; + `accounts/lookup?acct=dckc` gives the account id. + - [ ] 2-ii. Storage: one table per collection (statuses, + favourites, bookmarks, mentions), keyed by `uri`, holding + the status JSON and `edited_at`. **Durable state** from day one: a `SCHEMA_V1` step. - - [ ] 2-iii. Fill: follow the collection's `first`/`next` pages - with `Accept: application/activity+json`; politeness - delay per request; back off on 429. + - [ ] 2-iii. Fill: page backward following the `Link` header + (favourites and bookmarks paginate *only* by `Link`; + their cursors aren't status ids). Honor + `X-RateLimit-Remaining`/`-Reset`. - [ ] 2-iv. Tests: `pnpm lint && pnpm test`, the real worker - against a fake ActivityPub server (WebFinger, actor, - paged outbox, a 429). - - [ ] 2-v. Complete: a filled cache matches the account - archive's `outbox.json` — the archive is itself an - ActivityPub collection, so the comparison stays at the - protocol level (the analogue of diigo's - `scripts/compare-export.ts`). + against a fake Mastodon (`Link` paging, a rate-limit + response). + - [ ] 2-v. Complete: a filled cache holds everything in the + [mastodon-archive](https://github.com/kensanata/mastodon-archive) + output (`make sync` in `~/projects/office-admin/toots`; + `social.coop.user.dckc.json` has the same API shapes), + the analogue of diigo's `scripts/compare-export.ts`. - - [ ] **3. Push: fedi-cache as a follower**: - - [ ] 3-i. Serve its own actor: WebFinger, actor document with - `publicKey`, `inbox`. Needs a public HTTPS origin, so dev - wants a tunnel or a deployed test instance (and the user - runs it, per AGENTS.md). - - [ ] 3-ii. Send `Follow`; record `Accept`. - - [ ] 3-iii. Inbox: verify the HTTP Signature against the - sender's key, accept only activities whose `actor` is - @dckc, apply `Create`/`Update`/`Delete`/`Announce`/`Undo`. - - [ ] 3-iv. Tests: signed deliveries in; a forged one and one - from another actor rejected; an edit and a delete reflected. + - [ ] **3. Freshness**: new items are cheap (`min_id`, or the `Link` + `prev`, from the newest stored). Edits and deletions of older + posts aren't announced to an API poller: refetch the last N + days on each refresh (the window a roundup cares about). A + test: a post added, one edited, one deleted in the fake; the + weekly window reflects all three. A cron alarm in the cache DO + drives it. - [ ] **4. Read surfaces**: `/public/dckc` with diigo 1d's date window and `q` filter, HTML and JSON; JSON shape pinned by a - test. Only objects addressed to `as:Public` appear there: - followers-only posts (which only push brings in) never reach a - public page; a test says so. + test. Only `visibility: public` posts and opted-in collections + (0-ii) appear; followers-only and direct posts never do, and a + test says so. - [ ] **5. madmode's weekly page includes it**: diigo's goal 4 (`/bookmarks/2026-W41`) grows a section from this cache, or becomes a combined `/weekly/2026-W41`. Same editorial question as there: a raw list, or a draft the user edits. - - [ ] **6. Deployed**: rate-limit `/public/*`; the actor's origin is - its identity (moving it changes the follower), so choose the - hostname once. + - [ ] **6. Deployed**: after diigo's goal 5; rate-limit `/public/*`; + mint against prod; fill. - - [ ] **7. Revocation**: if the follow is removed or deliveries stop, - the public page keeps serving what's stored and says when it - last heard from @dckc; outbox polling still covers public posts. + - [ ] **7. Token revocation**: when social.coop rejects the token + (401), refreshes fail; the public page keeps serving what's + stored and says so. Re-mint is manual; same re-wire question + as diigo's goal 6. + + - [ ] **8. Protocol-level sources** (later; portability): + - [ ] 8-i. Public posts from the actor's ActivityPub `outbox` + (WebFinger → actor → `outbox`), no token; works against + any server, so an account move needs no new source. + - [ ] 8-ii. Push: fedi-cache as a follower actor (its own + keypair, HTTP Signatures), so `Update`/`Delete` arrive + instead of being polled for. Needs a public HTTPS origin + whose hostname is its identity; choose it once. + - [ ] 8-iii. Neither reaches favourites, bookmarks, or + mentions: those stay on the client API (or the server's + account archive). ## How to run @@ -102,12 +105,17 @@ config types. ## Pointers -- [ActivityPub](https://www.w3.org/TR/activitypub/) (outbox, inbox, - delivery) and [Activity Streams 2.0](https://www.w3.org/TR/activitystreams-core/) - (collections, paging) -- [WebFinger, RFC 7033](https://www.rfc-editor.org/rfc/rfc7033) -- [Mastodon's ActivityPub notes](https://docs.joinmastodon.org/spec/activitypub/) - and [its HTTP Signatures profile](https://docs.joinmastodon.org/spec/security/) - — what social.coop actually sends and requires (the server is - Mastodon; the cache doesn't depend on it) +- `~/projects/office-admin/toots/` — the current export: + [mastodon-archive](https://github.com/kensanata/mastodon-archive) via + `make sync` (its OAuth secrets sit there as `*.secret`, gitignored) +- Mastodon API: [accounts](https://docs.joinmastodon.org/methods/accounts/), + [favourites](https://docs.joinmastodon.org/methods/favourites/), + [bookmarks](https://docs.joinmastodon.org/methods/bookmarks/), + [OAuth scopes](https://docs.joinmastodon.org/api/oauth-scopes/), + [rate limits](https://docs.joinmastodon.org/api/rate-limits/), + [pagination](https://docs.joinmastodon.org/api/guidelines/#pagination) +- [ActivityPub](https://www.w3.org/TR/activitypub/), + [WebFinger, RFC 7033](https://www.rfc-editor.org/rfc/rfc7033), + [Mastodon's HTTP Signatures profile](https://docs.joinmastodon.org/spec/security/) + — for goal 8 - [#23](https://github.com/dckc/madmode-blog/issues/23) — the PESOS goal