The guard direction from 2026-11-04 is corrected: the co-op DOES use Loomio and
wants every new member added. Removing members/notifications is not the goal —
not duplicating them is. The feature is switched back ON and the sync is made
idempotent instead.
Enablement
* LOOMIO_SYNC_ENABLED now defaults to ENABLED (kill switch: set =0 to pause).
Restoring LOOMIO_API_KEY and deploying is all that is needed again.
Non-duplicating ADD (the real fix), two layers:
* (a) live pre-check against Loomio group membership (b3 index, ordered and
paginated stably). An address already in the group is skipped SILENTLY —
it is never placed in an invite payload, so no API call about it can fire
a notification or email.
* (b) members.loomio_invited_at: an address we have handed over before is
never sent again, even if it is not (yet) visible in the live roster.
* FAIL CLOSED if the live roster cannot be read.
* remove_absent is never sent.
Removal reconciliation preserved (deactivation must still propagate):
* A member in the group who is no longer active is removed via a TARGETED
b3 DELETE by membership id — it names only the member being removed, so it
cannot re-invite or notify anyone staying. Operator allowlist
(LOOMIO_ALWAYS_KEEP) is never removed. 404 (already gone) counts as
success, so a re-run neither errors nor churns.
* The keep-set is computed from the UNFILTERED active list: an active member
whose Cloudron account is not yet activated is not removed (only their
invite is deferred).
Tests: tests/test_loomio_guard.py (26/26) — add/dedupe, silent skip, record
guard, fail-closed, remove-once, allowlist-never-removed, targeted removal
(no present member in any payload), double-run no-op for both phases, 404
tolerance, kill switch.
241 lines
9.6 KiB
Python
241 lines
9.6 KiB
Python
#!/usr/bin/env python3
|
|
"""Regression test: the Loomio sync is idempotent and non-duplicating.
|
|
|
|
WHY THIS EXISTS
|
|
On 2026-11-04 a live run of sync_loomio_memberships() emailed a Loomio invite
|
|
notification to (effectively) the whole membership. The guards that must hold
|
|
are proven here on a FIXTURE db with the network STUBBED:
|
|
|
|
ADD / dedupe
|
|
1. A member already in the Loomio group is skipped SILENTLY — no invite
|
|
payload ever contains their address.
|
|
2. members.loomio_invited_at is a second guard: an address we have handed
|
|
over before is never sent again, even if it is not (yet) visible in the
|
|
live roster.
|
|
3. `remove_absent` is never sent (no evict-then-re-add re-invite path).
|
|
4. FAIL CLOSED — if the live roster cannot be read, nothing is sent.
|
|
5. Running the sync twice adds nobody a second time.
|
|
|
|
REMOVE / reconciliation (a deactivation MUST still propagate to Loomio)
|
|
6. A member in the group who is no longer active is removed exactly once.
|
|
7. The operator allowlist is never removed, even when not active.
|
|
8. Removal is targeted — it never re-sends a present member's address, so it
|
|
cannot re-invite or notify anyone who is staying.
|
|
9. A second run removes nobody again (the roster no longer lists them).
|
|
10. remove_loomio_membership treats 404 as success (idempotent, no error).
|
|
|
|
The sync is ON by default (kill switch LOOMIO_SYNC_ENABLED=0 pauses it).
|
|
|
|
Runnable: python3 tests/test_loomio_guard.py (from the repo root)
|
|
"""
|
|
import os
|
|
import sys
|
|
import tempfile
|
|
import asyncio
|
|
|
|
os.environ["DB_PATH"] = os.path.join(tempfile.mkdtemp(), "fixture.db")
|
|
os.environ["LOOMIO_API_KEY"] = "test-user-key"
|
|
os.environ["LOOMIO_B3_KEY"] = "test-server-key"
|
|
os.environ["LOOMIO_GROUP_ID"] = "1"
|
|
# Leave LOOMIO_SYNC_ENABLED unset -> the sync must default to ENABLED.
|
|
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
|
|
import app.main as m # noqa: E402
|
|
|
|
m.DB_PATH = os.environ["DB_PATH"]
|
|
# Keep the membership assertions isolated: no extra keep accounts, and no
|
|
# username-fix pass (it makes its own /api/b3/users calls).
|
|
m.LOOMIO_ALWAYS_KEEP = ["keep@x.org"]
|
|
|
|
posts = [] # every b2/memberships POST payload
|
|
deletes = [] # every b3 membership DELETE url
|
|
ROSTER = {} # email -> membership id currently in the group
|
|
GET_FAILS = False # exercise fail-closed
|
|
DELETE_STATUS = 200 # toggle to prove 404 tolerance
|
|
|
|
|
|
class FakeResp:
|
|
def __init__(self, status=200, payload=None):
|
|
self.status_code = status
|
|
self._payload = payload or {}
|
|
self.text = str(payload or {})
|
|
|
|
def raise_for_status(self):
|
|
if self.status_code >= 400:
|
|
raise RuntimeError(f"HTTP {self.status_code}")
|
|
|
|
def json(self):
|
|
return self._payload
|
|
|
|
|
|
class FakeClient:
|
|
def __init__(self, *a, **k):
|
|
pass
|
|
|
|
async def __aenter__(self):
|
|
return self
|
|
|
|
async def __aexit__(self, *a):
|
|
return False
|
|
|
|
async def get(self, url, headers=None, params=None):
|
|
if GET_FAILS:
|
|
raise RuntimeError("simulated Loomio roster read failure")
|
|
offset = (params or {}).get("offset", 0)
|
|
limit = (params or {}).get("limit", 50)
|
|
rows = [{"id": i, "user_email": e} for e, i in ROSTER.items()]
|
|
page = rows[offset:offset + limit]
|
|
return FakeResp(200, {"memberships": page, "meta": {"total": len(rows)}})
|
|
|
|
async def post(self, url, headers=None, json=None):
|
|
posts.append({"url": url, "json": json or {}})
|
|
return FakeResp(200, {"added_emails": (json or {}).get("emails", [])})
|
|
|
|
async def delete(self, url, headers=None):
|
|
deletes.append(url)
|
|
return FakeResp(DELETE_STATUS)
|
|
|
|
async def patch(self, *a, **k):
|
|
return FakeResp(200, {})
|
|
|
|
|
|
async def fake_active():
|
|
# active members: new@ / old@ are NOT in the group; member@ IS.
|
|
return ["new@x.org", "old@x.org", "member@x.org"]
|
|
|
|
|
|
async def fake_activated(emails):
|
|
return emails
|
|
|
|
|
|
async def noop_fix(_client):
|
|
return {}
|
|
|
|
|
|
results = []
|
|
|
|
|
|
def check(name, ok):
|
|
results.append((name, ok))
|
|
|
|
|
|
async def main():
|
|
global GET_FAILS, ROSTER, DELETE_STATUS
|
|
m.get_db().close() # create + migrate the fixture schema
|
|
conn = m.get_db()
|
|
for e in ("new@x.org", "old@x.org", "member@x.org"):
|
|
conn.execute("INSERT OR REPLACE INTO members (email, active) VALUES (?, 1)", (e,))
|
|
conn.commit()
|
|
conn.close()
|
|
|
|
cols = [r[1] for r in m.get_db().execute("PRAGMA table_info(members)").fetchall()]
|
|
m.get_db().close()
|
|
check("migration: members.loomio_invited_at exists", "loomio_invited_at" in cols)
|
|
|
|
m.get_active_member_emails = fake_active
|
|
m.filter_loomio_activated = fake_activated
|
|
m.httpx.AsyncClient = FakeClient
|
|
m.loomio_fix_usernames = noop_fix
|
|
|
|
check("default: sync is ENABLED when the flag is unset", m.LOOMIO_SYNC_ENABLED is True)
|
|
|
|
# Loomio group currently holds: an active member (member@), and the operator
|
|
# allowlist account (keep@) which is NOT an active portal member.
|
|
ROSTER = {"member@x.org": 10, "keep@x.org": 2}
|
|
|
|
# --- 1. ADD path: only the never-seen address is invited. ---
|
|
m.mark_loomio_invited("old@x.org") # recorded -> excluded by layer (b)
|
|
posts.clear(); deletes.clear()
|
|
await m.sync_loomio_memberships()
|
|
body = posts[0]["json"] if posts else {}
|
|
check("add: exactly one invite call", len(posts) == 1)
|
|
check("add: only the never-seen address is invited", body.get("emails") == ["new@x.org"])
|
|
check("add: a member ALREADY in Loomio is skipped silently",
|
|
"member@x.org" not in (body.get("emails") or []))
|
|
check("add: the record guard excludes a previously-invited address",
|
|
"old@x.org" not in (body.get("emails") or []))
|
|
check("add: remove_absent is NEVER sent (no re-add path)", "remove_absent" not in body)
|
|
check("add: the newly-invited address is recorded",
|
|
m.get_loomio_invited_at("new@x.org") is not None)
|
|
check("add: nothing removed when nobody has lapsed", deletes == [])
|
|
|
|
# --- 2. Second run adds nobody again. ---
|
|
ROSTER = {"member@x.org": 10, "keep@x.org": 2, "new@x.org": 99} # Loomio now has new@
|
|
posts.clear(); deletes.clear()
|
|
await m.sync_loomio_memberships()
|
|
check("double-run: no second invite", posts == [])
|
|
check("double-run: no removals", deletes == [])
|
|
|
|
# --- 3. REMOVE path: a lapsed member is removed exactly once. ---
|
|
ROSTER = {"member@x.org": 10, "keep@x.org": 2, "new@x.org": 99, "lapsed@x.org": 77}
|
|
posts.clear(); deletes.clear()
|
|
await m.sync_loomio_memberships()
|
|
check("remove: the lapsed member is removed exactly once",
|
|
deletes == [f"{m.LOOMIO_BASE}/api/b3/memberships/77"])
|
|
check("remove: the live active member is NOT removed",
|
|
f"{m.LOOMIO_BASE}/api/b3/memberships/10" not in deletes)
|
|
check("remove: the operator allowlist is NEVER removed (even though not active)",
|
|
f"{m.LOOMIO_BASE}/api/b3/memberships/2" not in deletes)
|
|
check("remove: no re-invite is sent while removing", posts == [])
|
|
check("remove: it does not send present members to any endpoint",
|
|
all("keep@x.org" not in (p["json"].get("emails") or []) for p in posts))
|
|
|
|
# --- 4. Second run after the removal changes nothing. ---
|
|
ROSTER = {"member@x.org": 10, "keep@x.org": 2, "new@x.org": 99} # lapsed gone
|
|
posts.clear(); deletes.clear()
|
|
await m.sync_loomio_memberships()
|
|
check("remove double-run: nothing removed again", deletes == [])
|
|
check("remove double-run: nothing invited", posts == [])
|
|
|
|
# --- 5. A re-run against a STALE roster (member already gone) must not error:
|
|
# Loomio answers 404 for an absent membership, which is success. ---
|
|
ROSTER = {"member@x.org": 10, "keep@x.org": 2, "lapsed@x.org": 77}
|
|
DELETE_STATUS = 404
|
|
posts.clear(); deletes.clear()
|
|
err = None
|
|
try:
|
|
await m.sync_loomio_memberships()
|
|
except Exception as e: # pragma: no cover
|
|
err = e
|
|
check("idempotent remove: a 404 (already absent) is treated as success", err is None)
|
|
check("idempotent remove: still no re-invite", posts == [])
|
|
DELETE_STATUS = 200
|
|
|
|
# --- 6. FAIL CLOSED: an unreadable roster stops BOTH phases. ---
|
|
ROSTER = {"lapsed@x.org": 77}
|
|
GET_FAILS = True
|
|
posts.clear(); deletes.clear()
|
|
await m.sync_loomio_memberships()
|
|
check("fail-closed: no invite when the roster is unreadable", posts == [])
|
|
check("fail-closed: no removal when the roster is unreadable", deletes == [])
|
|
GET_FAILS = False
|
|
|
|
# --- 7. The kill switch really pauses everything. ---
|
|
m.LOOMIO_SYNC_ENABLED = False
|
|
posts.clear(); deletes.clear()
|
|
await m.sync_loomio_memberships()
|
|
check("kill switch: no calls at all when LOOMIO_SYNC_ENABLED=0",
|
|
posts == [] and deletes == [])
|
|
m.LOOMIO_SYNC_ENABLED = True
|
|
|
|
# --- 8. Recording is idempotent — an existing timestamp is never overwritten. ---
|
|
first = m.get_loomio_invited_at("old@x.org")
|
|
m.mark_loomio_invited("old@x.org")
|
|
check("mark_loomio_invited never overwrites an existing timestamp",
|
|
m.get_loomio_invited_at("old@x.org") == first)
|
|
|
|
# --- 9. remove_loomio_membership status handling (200/204/404 ok, 500 not). ---
|
|
async with FakeClient() as c:
|
|
DELETE_STATUS = 204
|
|
check("remove_loomio_membership: 204 is success", await m.remove_loomio_membership(c, 1))
|
|
DELETE_STATUS = 500
|
|
check("remove_loomio_membership: 500 is a failure", not await m.remove_loomio_membership(c, 1))
|
|
DELETE_STATUS = 200
|
|
|
|
|
|
asyncio.run(main())
|
|
fails = sum(not ok for _, ok in results)
|
|
for name, ok in results:
|
|
print(f" [{'PASS' if ok else 'FAIL'}] {name}")
|
|
print(f"\n{len(results) - fails}/{len(results)} passed")
|
|
sys.exit(1 if fails else 0)
|