Retrospective review: portal commits 394cccfb..e24c272e (landed 2026-10-20/21) #34

Closed
opened 2026-10-26 16:28:28 +00:00 by inference-bot · 2 comments
Owner

Context. The 7 commits in 394cccfb..e24c272e (app/main.py, +181/-52) were pushed straight to main without a human review pass and are live (image 20261026-4920190). A retrospective read of the full cumulative diff, plus exercising every endpoint it touches, is below. Two things worth changing; nothing that threatens member data.

Verified correct

  • fetch_all_member_nodes() — offset pagination driven by totalCount, 20-page cap, raises on any page failure so the sweep cannot act on a partial list. The failure mode is the safe direction (no mass deactivation). Right fix for members(limit:100) silently hiding everyone past page 1.
  • org-link INSERT — now binds 3 values to 3 placeholders (was 4→3, a 500 on every link). Reproduced in the diff and exercised live.
  • admin_org_links: or name → or org_slug — a genuine NameError fix.
  • LISTMONK_API_USER = os.environ.get(X) or default — correct root cause: an env var set empty beat the code default, so every sync authenticated with an empty user and 403'd silently. The or pattern is the right fix.
  • matrix_mxid in /admin/overview — SELECT and the unpack loop were updated together; live response includes the field.

Live checks (with the admin token): /admin/overview 200, /admin/org-links 200, /admin/newsletter/reconcile → {"removed":0,"emails":[]}, and /admin/overview without a token → 401.

Finding 1 — the newsletter reconcile deletes subscribers irreversibly

listmonk_reconcile() branches on whether the subscriber's email has a members row:

  • inactive member → PUT {"lists": []} — record kept as a suppression marker. Good.
  • no members row at all → DELETE /api/subscribers/{id} — the record is gone, including its consent/unsubscribe history.

The staging accounts that motivated the fix are real, but so is the failure mode: a person who subscribed while their members row is absent (deleted account, manual subscribe, email change) is dropped from the mailing list with nothing left behind. Irreversible, and silent — the only trace is one log line.

Proposal: treat the non-member case as PUT {"lists": []} as well, keeping the suppression record, or restrict DELETE to emails matching obvious test patterns (+test@, @example.com). Also consider dropping the address list from the Listmonk reconcile: removed … log line — the count is enough.

Finding 2 — the sweep's documented fail-safe is unreachable on a fetch error

reconcile_memberships() calls active = await fetch_members() unguarded, then comments:

FAIL-SAFE: if the OC fetch returned nothing … we must NOT deactivate everyone … Skip reconciliation entirely

Since fetch_members() now raises instead of returning [], an OC outage takes the exception path: the sweep 500s rather than returning {"status":"skipped","reason":"empty_member_list"}. Safety holds — the raise lands before any write, so nobody is deactivated — but the code does not do what its comment says. fetch_org_backers() is wrapped a few lines below; fetch_members() is not.

The two webhook calls (members = await fetch_members() in the credit-pack and membership branches) are unguarded too: an OC outage during a payment event 500s the webhook. OC retries, so it self-heals.

Not reproduced (would need an OC outage); read from the code paths.

Proposal: wrap the sweep's call and return the documented skipped dict, or amend the comment to say the sweep deliberately fails loudly. Either is defensible — they just have to agree.

Minor

listmonk_reconcile()'s subscriber pagination breaks after page 1 if listmonk omits total (len(subs) >= (data.get("total") or 0)). total is present in practice, so cosmetic.

**Context.** The 7 commits in `394cccfb..e24c272e` (`app/main.py`, +181/-52) were pushed straight to `main` without a human review pass and are live (image `20261026-4920190`). A retrospective read of the full cumulative diff, plus exercising every endpoint it touches, is below. Two things worth changing; nothing that threatens member data. ## Verified correct - **`fetch_all_member_nodes()`** — offset pagination driven by `totalCount`, 20-page cap, **raises on any page failure** so the sweep cannot act on a partial list. The failure mode is the safe direction (no mass deactivation). Right fix for `members(limit:100)` silently hiding everyone past page 1. - **org-link `INSERT`** — now binds 3 values to 3 placeholders (was 4→3, a 500 on every link). Reproduced in the diff and exercised live. - **`admin_org_links`: `or name` → `or org_slug`** — a genuine `NameError` fix. - **`LISTMONK_API_USER = os.environ.get(X) or default`** — correct root cause: an env var set *empty* beat the code default, so every sync authenticated with an empty user and 403'd silently. The `or` pattern is the right fix. - **`matrix_mxid` in `/admin/overview`** — `SELECT` and the unpack loop were updated together; live response includes the field. **Live checks (with the admin token):** `/admin/overview` 200, `/admin/org-links` 200, `/admin/newsletter/reconcile` → `{"removed":0,"emails":[]}`, and `/admin/overview` without a token → 401. ## Finding 1 — the newsletter reconcile deletes subscribers irreversibly `listmonk_reconcile()` branches on whether the subscriber's email has a `members` row: - inactive member → `PUT {"lists": []}` — record kept as a suppression marker. Good. - **no `members` row at all → `DELETE /api/subscribers/{id}`** — the record is gone, including its consent/unsubscribe history. The staging accounts that motivated the fix are real, but so is the failure mode: a person who subscribed while their `members` row is absent (deleted account, manual subscribe, email change) is dropped from the mailing list with nothing left behind. Irreversible, and silent — the only trace is one log line. **Proposal:** treat the non-member case as `PUT {"lists": []}` as well, keeping the suppression record, or restrict `DELETE` to emails matching obvious test patterns (`+test@`, `@example.com`). Also consider dropping the address list from the `Listmonk reconcile: removed …` log line — the count is enough. ## Finding 2 — the sweep's documented fail-safe is unreachable on a fetch error `reconcile_memberships()` calls `active = await fetch_members()` **unguarded**, then comments: > FAIL-SAFE: if the OC fetch returned nothing … we must NOT deactivate everyone … Skip reconciliation entirely Since `fetch_members()` now *raises* instead of returning `[]`, an OC outage takes the exception path: the sweep 500s rather than returning `{"status":"skipped","reason":"empty_member_list"}`. **Safety holds — the raise lands before any write, so nobody is deactivated — but the code does not do what its comment says.** `fetch_org_backers()` is wrapped a few lines below; `fetch_members()` is not. The two webhook calls (`members = await fetch_members()` in the credit-pack and membership branches) are unguarded too: an OC outage during a payment event 500s the webhook. OC retries, so it self-heals. Not reproduced (would need an OC outage); read from the code paths. **Proposal:** wrap the sweep's call and return the documented `skipped` dict, or amend the comment to say the sweep deliberately fails loudly. Either is defensible — they just have to agree. ## Minor `listmonk_reconcile()`'s subscriber pagination breaks after page 1 if listmonk omits `total` (`len(subs) >= (data.get("total") or 0)`). `total` is present in practice, so cosmetic.
Author
Owner

Fixes implemented and deployed as image 20261026-9495daa (commits 7bb676e, 9495daa; manifest pin 250ca6e).

Finding 1 — no more deletes. listmonk_reconcile() now takes both classes off the list the same way — PUT {"lists": []} — and keeps the record as a suppression marker. Nothing is purged automatically. The summary log line reports counts by reason instead of subscriber addresses.

Finding 2 — the fail-safe is reachable. reconcile_memberships() catches a member-fetch failure and returns {"status": "skipped", "reason": "member_fetch_failed"} instead of 500ing. The two webhook call sites are deliberately left to fail loudly: a payment webhook should error so OpenCollective retries it, and it self-heals.

A third bug surfaced while testing finding 1 — the original batch carried it too. PUT /api/subscribers/{id} returns 400 {"message":"Invalid email."} unless the body includes email. The portal sent only {"lists": []}, so the unsubscribe-on-deactivation path never worked — for inactive members either, which is this reconcile's headline job. Only the DELETE branch did anything, and its failure was a log warning nobody read. Established by probing the live API:

  • {"lists": []} → 400 Invalid email., subscriber stays on the list
  • {"email": ..., "lists": []} → 200, lists: []

Fixed in 9495daa.

Verification state. Deployed image confirmed running (health 200, /broker/usage 401, app manifest dockerImage = the new tag, repo pin committed). The accepted PUT shape is proven against the live API. The end-to-end reconcile run is held: the only remaining stale subscriber is info@inference.coop, which has no members row, so exercising the reconcile necessarily unsubscribes it — an operator call, raised separately. Census before the change: 67 subscribers, 66 active members all present, 0 active members missing.

Open inconsistency. The sweep honours MANUAL_MEMBERS (env) when deciding deactivations; listmonk_reconcile() computes its active set from the members table alone. Depending on the info@ decision, the reconcile should union MANUAL_MEMBERS in so the two agree.

Fixes implemented and deployed as image `20261026-9495daa` (commits `7bb676e`, `9495daa`; manifest pin `250ca6e`). **Finding 1 — no more deletes.** `listmonk_reconcile()` now takes both classes off the list the same way — `PUT {"lists": []}` — and keeps the record as a suppression marker. Nothing is purged automatically. The summary log line reports counts by reason instead of subscriber addresses. **Finding 2 — the fail-safe is reachable.** `reconcile_memberships()` catches a member-fetch failure and returns `{"status": "skipped", "reason": "member_fetch_failed"}` instead of 500ing. The two webhook call sites are deliberately left to fail loudly: a payment webhook should error so OpenCollective retries it, and it self-heals. **A third bug surfaced while testing finding 1 — the original batch carried it too.** `PUT /api/subscribers/{id}` returns `400 {"message":"Invalid email."}` unless the body includes `email`. The portal sent only `{"lists": []}`, so **the unsubscribe-on-deactivation path never worked** — for inactive members either, which is this reconcile's headline job. Only the `DELETE` branch did anything, and its failure was a log warning nobody read. Established by probing the live API: - `{"lists": []}` → 400 `Invalid email.`, subscriber stays on the list - `{"email": ..., "lists": []}` → 200, `lists: []` Fixed in `9495daa`. **Verification state.** Deployed image confirmed running (health 200, `/broker/usage` 401, app manifest `dockerImage` = the new tag, repo pin committed). The accepted PUT shape is proven against the live API. The end-to-end reconcile run is **held**: the only remaining stale subscriber is `info@inference.coop`, which has no `members` row, so exercising the reconcile necessarily unsubscribes it — an operator call, raised separately. Census before the change: 67 subscribers, 66 active members all present, 0 active members missing. **Open inconsistency.** The sweep honours `MANUAL_MEMBERS` (env) when deciding deactivations; `listmonk_reconcile()` computes its active set from the `members` table alone. Depending on the `info@` decision, the reconcile should union `MANUAL_MEMBERS` in so the two agree.
Author
Owner

Closing: both findings fixed, deployed and verified live. A third item raised after the review is also resolved.

Finding 1 — nothing is deleted any more

listmonk_reconcile() takes both classes off the list the same way (PUT {"lists": []}) and keeps the record as a suppression marker. Verified against the old build and the new one: a probe subscriber with no members row vanished on the old build (that is the test that failed) and now survives, off the list, with its consent record intact.

Live now: POST /admin/newsletter/reconcile → {"removed": 0, "emails": [], "operators_kept": 1}, and no log line carries a subscriber address any more (counts by reason only).

Finding 2 — the documented fail-safe is reachable

reconcile_memberships() catches a member-fetch failure and returns {"status": "skipped", "reason": "member_fetch_failed"} instead of letting the raise escape and 500 the sweep. The failure branch itself needs an Open Collective outage to trigger, so it is verified by reading the code path rather than by exercise; normal operation is confirmed — the sweep returns {"status": "reconciled", …} in ~9s.

Minor — subscriber pagination

Now also stops on a short/empty page, so a missing total cannot truncate the reconcile at page 1.

Also resolved here

The reconcile classified info@inference.coop as a stale subscriber (no members row). Operators are now a third class — neither members nor stale — drawn from OPERATOR_EMAILS, the same list the panel uses for its admins category, and the same rule the Loomio sync already applied via LOOMIO_ALWAYS_KEEP. Their subscription is left exactly as it is, and the response reports operators_kept. Verified: info@ still on the Members list, 67 subscribers.

Deliberately deferred

The sweep honours MANUAL_MEMBERS when deciding deactivations while the reconcile does not consult it. All ten addresses are real members rows, so it changes nothing today — noted rather than churned.

Live state at closing

Portal image registry.inference.coop/co-op/member-portal:20261026-e55d8a3; reconcile clean; sweep reconciled; repo and live config byte-identical (enforced now by start.sh, which syncs the shipped config over the persistent volume at boot).

Closing: both findings fixed, deployed and verified live. A third item raised after the review is also resolved. ## Finding 1 — nothing is deleted any more `listmonk_reconcile()` takes both classes off the list the same way (`PUT {"lists": []}`) and keeps the record as a suppression marker. Verified against the old build and the new one: a probe subscriber with no `members` row **vanished** on the old build (that is the test that failed) and now **survives, off the list**, with its consent record intact. Live now: `POST /admin/newsletter/reconcile` → `{"removed": 0, "emails": [], "operators_kept": 1}`, and no log line carries a subscriber address any more (counts by reason only). ## Finding 2 — the documented fail-safe is reachable `reconcile_memberships()` catches a member-fetch failure and returns `{"status": "skipped", "reason": "member_fetch_failed"}` instead of letting the raise escape and 500 the sweep. The failure branch itself needs an Open Collective outage to trigger, so it is verified by reading the code path rather than by exercise; normal operation is confirmed — the sweep returns `{"status": "reconciled", …}` in ~9s. ## Minor — subscriber pagination Now also stops on a short/empty page, so a missing `total` cannot truncate the reconcile at page 1. ## Also resolved here The reconcile classified `info@inference.coop` as a stale subscriber (no `members` row). Operators are now a third class — neither members nor stale — drawn from `OPERATOR_EMAILS`, the same list the panel uses for its **admins** category, and the same rule the Loomio sync already applied via `LOOMIO_ALWAYS_KEEP`. Their subscription is left exactly as it is, and the response reports `operators_kept`. Verified: `info@` still on the Members list, 67 subscribers. ## Deliberately deferred The sweep honours `MANUAL_MEMBERS` when deciding deactivations while the reconcile does not consult it. All ten addresses are real `members` rows, so it changes nothing today — noted rather than churned. ## Live state at closing Portal image `registry.inference.coop/co-op/member-portal:20261026-e55d8a3`; reconcile clean; sweep `reconciled`; repo and live config byte-identical (enforced now by `start.sh`, which syncs the shipped config over the persistent volume at boot).
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: co-op/support#34