loomio: enable by default; idempotent add (never re-invite) + targeted remove

The guard direction from 2026-11-04 is corrected: the co-op DOES use Loomio and
wants every new member added. Removing members/notifications is not the goal —
not duplicating them is. The feature is switched back ON and the sync is made
idempotent instead.

Enablement
  * LOOMIO_SYNC_ENABLED now defaults to ENABLED (kill switch: set =0 to pause).
    Restoring LOOMIO_API_KEY and deploying is all that is needed again.

Non-duplicating ADD (the real fix), two layers:
  * (a) live pre-check against Loomio group membership (b3 index, ordered and
    paginated stably). An address already in the group is skipped SILENTLY —
    it is never placed in an invite payload, so no API call about it can fire
    a notification or email.
  * (b) members.loomio_invited_at: an address we have handed over before is
    never sent again, even if it is not (yet) visible in the live roster.
  * FAIL CLOSED if the live roster cannot be read.
  * remove_absent is never sent.

Removal reconciliation preserved (deactivation must still propagate):
  * A member in the group who is no longer active is removed via a TARGETED
    b3 DELETE by membership id — it names only the member being removed, so it
    cannot re-invite or notify anyone staying. Operator allowlist
    (LOOMIO_ALWAYS_KEEP) is never removed. 404 (already gone) counts as
    success, so a re-run neither errors nor churns.
  * The keep-set is computed from the UNFILTERED active list: an active member
    whose Cloudron account is not yet activated is not removed (only their
    invite is deferred).

Tests: tests/test_loomio_guard.py (26/26) — add/dedupe, silent skip, record
guard, fail-closed, remove-once, allowlist-never-removed, targeted removal
(no present member in any payload), double-run no-op for both phases, 404
tolerance, kill switch.
This commit is contained in:
inference-bot committed 2026-11-04 19:25:45 +00:00
1 parent 17329c5d6f
commit f7bbec8a0a
2 files changed
+282 -145

No files matched your search

+142 -78
View File
@@ -309,12 +309,14 @@ LOOMIO_API_KEY = os.environ.get("LOOMIO_API_KEY", "")
# mangled auto-handle (email local-part + random suffix). The B3 users API
# lets us rename them to the member's Cloudron username.
LOOMIO_B3_KEY = os.environ.get("LOOMIO_B3_KEY", "")
# Master switch for the Loomio membership sync. OFF unless explicitly enabled.
# The sync's only effect is Loomio's invite notification, and a bad run emails
# the whole group (incident 2026-11-04: notification 744 delivered to ~83
# members). Restoring LOOMIO_API_KEY alone must NOT re-arm it — this flag has
# to be set deliberately by the operator, once Loomio's purpose is decided.
LOOMIO_SYNC_ENABLED = os.environ.get("LOOMIO_SYNC_ENABLED", "").strip().lower() in ("1", "true", "on", "yes")
# Master switch for the Loomio membership sync. It defaults to ENABLED: the
# co-op wants every new member added to Loomio, so restoring LOOMIO_API_KEY
# and deploying is all that is needed. It is expressed as a KILL SWITCH —
# set LOOMIO_SYNC_ENABLED=0 (or false/off/no) to pause the sync; any other
# value, or leaving it unset, keeps it ON. (Incident 2026-11-04 was fixed the
# right way by making the ADD path idempotent, NOT by switching the feature
# off — see sync_loomio_memberships for the guards.)
LOOMIO_SYNC_ENABLED = os.environ.get("LOOMIO_SYNC_ENABLED", "1").strip().lower() not in ("0", "false", "off", "no")
# Matrix homeserver (self-hosted Synapse at matrix.inference.coop). The bot
# account (admin) invites every active member to the co-op Space on sync.
@@ -1582,76 +1584,124 @@ async def matrix_member_mxids(client: httpx.AsyncClient):
yield mxid, mxid
async def fetch_loomio_member_emails(client: httpx.AsyncClient) -> set[str]:
"""Return the emails Loomio already has in the group (lowercased).
async def fetch_loomio_members(client: httpx.AsyncClient) -> dict[str, int]:
"""Return {email_lower: membership_id} for everyone in the Loomio group.
This is the 'has this member EVER been in that system' half of the positive
exclusion. It must succeed before any invite is sent — see the fail-closed
handling in sync_loomio_memberships.
Uses the Server API (b3) so we get membership ids in one pass (the id is
what a targeted removal needs) AND a STABLE read: the b3 index is ordered by
id and paginates cleanly, whereas the User API b2 index has no ORDER BY, so
paging it can repeat or skip rows — a skipped row would make the pre-check
wrongly think an existing member is new. Reads the group's ACTIVE
memberships (pending invitees included, already-removed members excluded).
This is the live-membership half of the positive exclusion, and the ONLY
layer that reflects reality rather than our own records. It must succeed
before any invite or removal (fail-closed in sync_loomio_memberships).
"""
r = await client.get(
f"{LOOMIO_BASE}/api/b2/memberships",
headers={"Authorization": f"Bearer {LOOMIO_API_KEY}"},
params={"group_id": int(LOOMIO_GROUP_ID)},
members: dict[str, int] = {}
offset = 0
while True:
r = await client.get(
f"{LOOMIO_BASE}/api/b3/memberships",
headers={"Authorization": f"Bearer {LOOMIO_B3_KEY}"},
params={"group_id": int(LOOMIO_GROUP_ID), "limit": 50, "offset": offset},
)
r.raise_for_status()
page = r.json().get("memberships", [])
for m in page:
email = (m.get("user_email") or "").lower()
if email:
members[email] = m.get("id")
if len(page) < 50:
break
offset += 50
return members
async def remove_loomio_membership(client: httpx.AsyncClient, membership_id: int) -> bool:
"""Revoke ONE Loomio membership by id, via the Server API (b3).
Targeted on purpose: the request names only the membership being removed, so
it carries no address of anyone who is staying and therefore cannot trigger
an invite or a group notification — unlike b2's `remove_absent`, which
requires re-sending the whole keep-list.
Idempotent: Loomio answers 404 for an id that is already gone (or never
existed); the end state we want is "not a member", so 404 counts as success
and a repeated run does not error or churn.
"""
r = await client.delete(
f"{LOOMIO_BASE}/api/b3/memberships/{membership_id}",
headers={"Authorization": f"Bearer {LOOMIO_B3_KEY}"},
)
r.raise_for_status()
data = r.json()
return {
(m.get("user_email") or "").lower()
for m in data.get("memberships", [])
if m.get("user_email")
}
if r.status_code in (200, 204, 404):
return True
logger.warning(
"Loomio remove of membership %s failed: %s %s",
membership_id, r.status_code, r.text[:200],
)
return False
async def sync_loomio_memberships() -> None:
"""Add members to the Loomio group — NEVER re-invite an existing member.
"""Reconcile the Loomio group with the current ACTIVE-member list.
SAFETY (incident 2026-11-04): a live run of this function emailed a Loomio
invite notification to essentially the WHOLE membership. The guards below
make a repeat impossible:
TWO jobs with DIFFERENT guards — a deactivation must still propagate:
1. LOOMIO_SYNC_ENABLED must be ON. The integration is OFF by default, so
restoring LOOMIO_API_KEY alone does NOT re-arm it; enabling it is a
deliberate operator action.
2. POSITIVE EXCLUSION — an address is only ever sent if it is neither
currently in the Loomio group NOR recorded as previously handed to
Loomio (members.loomio_invited_at). A member who is already in Loomio,
or was ever invited, can never be invited again.
3. FAIL CLOSED — if the current Loomio member list cannot be read, the sync
skips entirely rather than sending a list it cannot verify.
1. ADD (invite) — a member is invited only if they are BOTH not already in
the group (live pre-check) AND never handed to Loomio before
(members.loomio_invited_at). Exactly once, ever. This is the dedupe
guard, and it applies to INVITATIONS ONLY.
2. REMOVE — a member no longer active is revoked so a lapse propagates to
Loomio. The removal path is targeted (b3 DELETE by membership id) and
sends nothing for anyone who is staying, so it cannot re-invite or
notify a present member. Operator accounts (LOOMIO_ALWAYS_KEEP) are
never removed.
Deliberately does NOT pass `remove_absent`: this sync only ADDS. Removing a
lapsed member is a separate, explicit decision. Sending only the
never-invited subset (instead of the full list with remove_absent=1) also
removes the reconcile path that could evict a member on one run and then
re-add — and therefore re-invite — them on the next.
Guards (incident 2026-11-04: a live run emailed Loomio invites to the whole
membership):
* The invite payload contains ONLY addresses Loomio has never seen and we
have never handed over — an existing member is skipped SILENTLY, no API
call about them at all.
* FAIL CLOSED — if the live group roster cannot be read, NEITHER phase
runs. We never hand Loomio a list we could not check.
Loomio's b2/memberships endpoint has no "silent invite" flag, so the only
way to guarantee no invite is to send an address exactly once.
Loomio's b2 invite has no "silent" flag, so the only way to guarantee no
invite is to send an address exactly once, ever — hence the record.
"""
if not LOOMIO_SYNC_ENABLED:
logger.info("Loomio sync disabled (LOOMIO_SYNC_ENABLED is off); skipping")
logger.info("Loomio sync disabled (LOOMIO_SYNC_ENABLED=0); skipping")
return
if not LOOMIO_API_KEY or not LOOMIO_GROUP_ID:
if not LOOMIO_GROUP_ID:
logger.warning("Loomio not configured; skipping membership sync")
return
emails = await get_active_member_emails()
# Always keep operator accounts in the group.
for keep in LOOMIO_ALWAYS_KEEP:
if keep not in emails:
emails.append(keep)
# The set that decides who STAYS in the group: every active member plus the
# operator allowlist. Computed from the UNFILTERED active list on purpose —
# an active member whose Cloudron account is not yet activated is still a
# member and must NOT be removed from Loomio (only their invite is deferred).
keep_emails = {e.lower() for e in await get_active_member_emails()}
keep_emails |= {e.lower() for e in LOOMIO_ALWAYS_KEEP}
# Only members whose Cloudron account is already set up (inviteAccepted) —
# never email a Loomio invite to someone who hasn't created their account
# yet. Operator accounts bypass this filter (the bootstrap owner shows
# inviteAccepted: false forever despite being functional).
emails = await filter_loomio_activated(emails)
# Invite candidates are additionally filtered to activated accounts so a
# Loomio invite email never precedes the member's own account setup.
invite_emails = await get_active_member_emails()
for keep in LOOMIO_ALWAYS_KEEP:
if keep not in invite_emails:
invite_emails.append(keep)
invite_emails = await filter_loomio_activated(invite_emails)
if not LOOMIO_B3_KEY:
# The live roster (and the removal path) needs the Server API key. Without
# it we cannot verify reality, so we do nothing rather than risk a
# blind invite.
logger.warning("Loomio sync: LOOMIO_B3_KEY unset; cannot read group roster (fail-closed)")
return
async with httpx.AsyncClient() as client:
# FAIL CLOSED: never send a list we could not check against Loomio.
# FAIL CLOSED: never act on a roster we could not read.
try:
current = await fetch_loomio_member_emails(client)
current = await fetch_loomio_members(client)
except Exception as e:
logger.warning(
"Loomio sync: could not read current group members; skipping "
@@ -1659,35 +1709,49 @@ async def sync_loomio_memberships() -> None:
)
return
# POSITIVE EXCLUSION: only addresses Loomio has never seen, and that we
# have never handed over.
# --- REMOVE: reconcile the group to the current active set. ---
removed = 0
for email, mid in current.items():
if email in keep_emails or mid is None:
continue
if await remove_loomio_membership(client, mid):
removed += 1
logger.info("Loomio sync: removed lapsed membership %s", mid)
if removed:
logger.info("Loomio sync: %d lapsed membership(s) removed", removed)
# --- ADD: only addresses Loomio has never seen AND we never handed over. ---
fresh = [
e for e in emails
e for e in invite_emails
if e.lower() not in current and not get_loomio_invited_at(e)
]
if not fresh:
logger.info(
"Loomio sync: nobody new (all already in Loomio or already "
"invited); nothing to send"
"invited); nothing to invite"
)
return
r = await client.post(
f"{LOOMIO_BASE}/api/b2/memberships",
headers={"Authorization": f"Bearer {LOOMIO_API_KEY}"},
json={"group_id": int(LOOMIO_GROUP_ID), "emails": fresh},
)
if r.status_code != 200:
logger.warning("Loomio membership sync failed: %s %s", r.status_code, r.text[:200])
return
result = r.json()
added = result.get("added_emails", []) or []
# Record EVERY address we sent, so it is never sent again — even if
# Loomio didn't add it, or the membership is later revoked. A revoked
# member is NOT a never-invited member, and must not be re-invited.
for e in fresh:
mark_loomio_invited(e)
logger.info("Loomio sync: first-time invites sent (%d): %s", len(added), added)
elif not LOOMIO_API_KEY:
logger.warning(
"Loomio sync: %d member(s) to invite but LOOMIO_API_KEY unset; "
"skipping invites", len(fresh),
)
else:
r = await client.post(
f"{LOOMIO_BASE}/api/b2/memberships",
headers={"Authorization": f"Bearer {LOOMIO_API_KEY}"},
json={"group_id": int(LOOMIO_GROUP_ID), "emails": fresh},
)
if r.status_code != 200:
logger.warning("Loomio membership sync failed: %s %s", r.status_code, r.text[:200])
else:
added = r.json().get("added_emails", []) or []
# Record EVERY address we sent, so it is never sent again — even
# if Loomio didn't add it, or the membership is later revoked. A
# revoked member is NOT a never-invited member, and must not be
# re-invited.
for e in fresh:
mark_loomio_invited(e)
logger.info("Loomio sync: first-time invites sent (%d): %s", len(added), added)
# Fix auto-generated usernames (email+randomsuffix) → Cloudron username.
# Best-effort: no B3 key or an API failure just logs and moves on.