activation hook: fire Loomio+Matrix syncs at first broker login
New members were skipped by the syncs until their Cloudron account was activated (deliberately - never invite a half-created account), but nothing fired the syncs at the moment they DID activate: the 3x/day sweep was the only pickup, so a newly-activated member could wait hours for forum/space access (limdingwen waited ~5h on 2026-11-06). Hook the broker identity path (_verify_broker_secret): after an active member resolves, if loomio_invited_at is still null, confirm their Cloudron account is activated (inviteAccepted, fail-CLOSED - an unreadable user list never fires) and run the same idempotent syncs the sweep runs, as a background task off the response. Established members pay one indexed SELECT; concurrent first-logins collapse into one run via an inflight flag; each service is independently best-effort so a Loomio outage cannot block the Matrix invite or the response. Tests: tests/test_first_login_sync.py (18 checks, fixture db + stubbed network). Also fix test_allowance_shortfall, whose expected exclusion reference still asserted the pre-a88ca86 placeholder. Sweep, provisioning and deactivation paths are untouched.
This commit is contained in:
1 parent
a88ca860d4
commit
ed66d73327
4 files changed
+348
-2
No files matched your search
@@ -107,6 +107,17 @@ Three jobs run inside the container via `CloudronManifest.json` → `addons.sche
|
|||||||
lapsed members, syncs Loomio + Matrix. Idempotent; a missed webhook
|
lapsed members, syncs Loomio + Matrix. Idempotent; a missed webhook
|
||||||
self-heals within a day. The same sweep also runs the payment-driven
|
self-heals within a day. The same sweep also runs the payment-driven
|
||||||
allowance reconciliation (below).
|
allowance reconciliation (below).
|
||||||
|
|
||||||
|
**First-login activation hook:** the sweep is also the backstop for service
|
||||||
|
invites, but it is no longer the only trigger. When a member first presents
|
||||||
|
themselves on the broker identity path (the dashboard's first login), the
|
||||||
|
portal checks — only if their Loomio invite is still outstanding
|
||||||
|
(`loomio_invited_at` is null) — that their Cloudron account is now activated
|
||||||
|
(`inviteAccepted`), and if so fires the same idempotent Loomio + Matrix syncs
|
||||||
|
as a background task. Established members cost one SELECT; the response never
|
||||||
|
waits on Loomio or Matrix; failures are logged and non-fatal; unactivated
|
||||||
|
accounts are never invited (the check fails closed). So a new member gets
|
||||||
|
forum + Space access at their first login instead of at the next sweep.
|
||||||
- **rag_refresh** (daily 3:30am UTC) — `rag_refresh.py` pulls this repo's
|
- **rag_refresh** (daily 3:30am UTC) — `rag_refresh.py` pulls this repo's
|
||||||
sibling docs from the Gitea raw API (public, no credentials), uploads changed
|
sibling docs from the Gitea raw API (public, no credentials), uploads changed
|
||||||
files to Open WebUI (`OWUI_API_KEY`), attaches them to the shared knowledge
|
files to Open WebUI (`OWUI_API_KEY`), attaches them to the shared knowledge
|
||||||
|
|||||||
+126
@@ -3891,6 +3891,127 @@ async def admin_model_usage(request: Request, days: int = 0):
|
|||||||
# own keys — never another member's, and never the chat key.
|
# own keys — never another member's, and never the chat key.
|
||||||
|
|
||||||
|
|
||||||
|
# --- First-login service sync (activation hook) ---
|
||||||
|
# A newly provisioned member is deliberately skipped by the Loomio/Matrix syncs
|
||||||
|
# until their Cloudron account is activated (so a half-created account is never
|
||||||
|
# invited), and the 3x/day sweep was the only thing that picked them up AFTER
|
||||||
|
# they did activate — a new member could wait hours for forum/space access.
|
||||||
|
# The broker identity path is where the member first presents themselves after
|
||||||
|
# setting up (the dashboard calls it on first login), so it is the natural
|
||||||
|
# trigger: when a broker-authenticated, active member resolves AND their Loomio
|
||||||
|
# invite is still outstanding, confirm their Cloudron account is now activated
|
||||||
|
# and, only then, fire the same idempotent syncs the sweep runs. The syncs run
|
||||||
|
# as a background task, so no response ever waits on Loomio or Matrix.
|
||||||
|
|
||||||
|
_first_login_sync_inflight = False
|
||||||
|
|
||||||
|
|
||||||
|
def _member_slug(email: str) -> str:
|
||||||
|
"""Member slug for log lines (slugs survive log redaction; emails don't)."""
|
||||||
|
try:
|
||||||
|
conn = get_db()
|
||||||
|
row = conn.execute(
|
||||||
|
"SELECT slug FROM members WHERE email = ?", (email,)
|
||||||
|
).fetchone()
|
||||||
|
conn.close()
|
||||||
|
return (row[0] if row and row[0] else "") or "<slug-unset>"
|
||||||
|
except Exception:
|
||||||
|
return "<slug-unset>"
|
||||||
|
|
||||||
|
|
||||||
|
async def _member_cloudron_activated(email: str) -> bool:
|
||||||
|
"""True only when the member's Cloudron account is CONFIRMED activated.
|
||||||
|
|
||||||
|
Fail-CLOSED on purpose: if the Cloudron user list cannot be read (or the
|
||||||
|
member is not on it), report not-activated and let the 3x/day sweep try
|
||||||
|
later. The hook must never be the path that invites a half-created account,
|
||||||
|
so unlike the sweep's fail-open roster filter, an unverifiable activation
|
||||||
|
state means "do not fire".
|
||||||
|
"""
|
||||||
|
async with httpx.AsyncClient(timeout=15.0) as client:
|
||||||
|
users = await cloudron_fetch_all_users(client)
|
||||||
|
if not users:
|
||||||
|
return False
|
||||||
|
for u in users:
|
||||||
|
if (u.get("email") or "").lower() == email.lower():
|
||||||
|
return bool(u.get("inviteAccepted"))
|
||||||
|
return False
|
||||||
|
|
||||||
|
|
||||||
|
async def _first_login_service_sync(email: str) -> None:
|
||||||
|
"""Fire the Loomio + Matrix Space syncs for a newly-activated member.
|
||||||
|
|
||||||
|
Runs as a background task off the broker request (a Loomio outage must
|
||||||
|
never block a dashboard). The syncs are the SAME idempotent functions the
|
||||||
|
sweep and provisioning run — guarded by members.loomio_invited_at and the
|
||||||
|
live-roster pre-check — so a re-fire cannot double-invite. Best-effort:
|
||||||
|
any failure is logged and swallowed; the sweep remains the backstop.
|
||||||
|
"""
|
||||||
|
global _first_login_sync_inflight
|
||||||
|
slug = _member_slug(email)
|
||||||
|
try:
|
||||||
|
if not await _member_cloudron_activated(email):
|
||||||
|
logger.info(
|
||||||
|
"first-login sync: %s Cloudron account not yet activated; not "
|
||||||
|
"firing (the sweep covers it later)", slug,
|
||||||
|
)
|
||||||
|
return
|
||||||
|
# Each service independently best-effort: a Loomio outage must never
|
||||||
|
# stop the Matrix invite (or vice versa), and neither may raise out of
|
||||||
|
# this background task.
|
||||||
|
for label, sync_fn in (
|
||||||
|
("Loomio", sync_loomio_memberships),
|
||||||
|
("Matrix", sync_matrix_space),
|
||||||
|
):
|
||||||
|
try:
|
||||||
|
logger.info("first-login sync: firing %s sync for %s", label, slug)
|
||||||
|
await sync_fn()
|
||||||
|
except Exception:
|
||||||
|
logger.warning(
|
||||||
|
"first-login sync: %s sync for %s failed (non-fatal; the "
|
||||||
|
"sweep covers later)", label, slug, exc_info=True,
|
||||||
|
)
|
||||||
|
logger.info(
|
||||||
|
"first-login sync: done for %s (loomio_invited_at=%s)",
|
||||||
|
slug, get_loomio_invited_at(email),
|
||||||
|
)
|
||||||
|
except Exception:
|
||||||
|
logger.warning(
|
||||||
|
"first-login sync for %s failed (non-fatal; sweep covers later)",
|
||||||
|
slug, exc_info=True,
|
||||||
|
)
|
||||||
|
finally:
|
||||||
|
_first_login_sync_inflight = False
|
||||||
|
|
||||||
|
|
||||||
|
def maybe_first_login_service_sync(email: str) -> None:
|
||||||
|
"""Cheap gate on the broker hot path; never raises.
|
||||||
|
|
||||||
|
Established members (loomio_invited_at set) cost one indexed SELECT and
|
||||||
|
return. A never-invited member schedules the sync as a background task.
|
||||||
|
While one first-login sync is in flight, further gate hits do nothing: the
|
||||||
|
syncs are full-roster reconciles, so the running pass already covers every
|
||||||
|
pending member.
|
||||||
|
"""
|
||||||
|
global _first_login_sync_inflight
|
||||||
|
try:
|
||||||
|
if get_loomio_invited_at(email):
|
||||||
|
return # established member — zero further work
|
||||||
|
if _first_login_sync_inflight:
|
||||||
|
return # a sync is already running; it covers every pending member
|
||||||
|
loop = asyncio.get_running_loop()
|
||||||
|
except Exception:
|
||||||
|
return # the gate must never break the broker response
|
||||||
|
_first_login_sync_inflight = True
|
||||||
|
try:
|
||||||
|
loop.create_task(_first_login_service_sync(email))
|
||||||
|
except Exception:
|
||||||
|
_first_login_sync_inflight = False
|
||||||
|
logger.warning(
|
||||||
|
"first-login sync could not be scheduled (non-fatal)", exc_info=True,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
def _verify_broker_secret(request: Request) -> str:
|
def _verify_broker_secret(request: Request) -> str:
|
||||||
"""Return the authenticated member's email, or 401.
|
"""Return the authenticated member's email, or 401.
|
||||||
|
|
||||||
@@ -3918,6 +4039,11 @@ def _verify_broker_secret(request: Request) -> str:
|
|||||||
# The member must be active in our DB before they can manage keys.
|
# The member must be active in our DB before they can manage keys.
|
||||||
if not get_member_key(email):
|
if not get_member_key(email):
|
||||||
raise HTTPException(403, "No active membership")
|
raise HTTPException(403, "No active membership")
|
||||||
|
# Activation hook: if this active member's service invites are still
|
||||||
|
# outstanding AND their Cloudron account is now activated, fire the Loomio
|
||||||
|
# + Matrix syncs (background task; established members cost one SELECT).
|
||||||
|
# Best-effort by construction — it must never affect this response.
|
||||||
|
maybe_first_login_service_sync(email)
|
||||||
return email
|
return email
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
@@ -109,9 +109,11 @@ check("reason names the cycle",
|
|||||||
"allowance shortfall, truncated paid cycle 2026-10-07→2026-11-02 (OC billing alignment)")
|
"allowance shortfall, truncated paid cycle 2026-10-07→2026-11-02 (OC billing alignment)")
|
||||||
|
|
||||||
# --- the manual-remediation exclusion is keyed on the charge ref alone
|
# --- the manual-remediation exclusion is keyed on the charge ref alone
|
||||||
# --- (no member PII in this public repo; identity/timeline live in ops docs)
|
# --- (no member PII in this public repo; identity/timeline live in ops docs;
|
||||||
|
# --- the OC tx id IS the reference key and was approved as such — see the
|
||||||
|
# --- commit that restored it to match the deployed image)
|
||||||
check("remediated charge excluded by reference",
|
check("remediated charge excluded by reference",
|
||||||
"[redacted member reference]" in SHORTFALL_REMEDIATED, True)
|
"tx_Kr6n3rW577OSjr2bIt5IC" in SHORTFALL_REMEDIATED, True)
|
||||||
|
|
||||||
print(f"\n{'FAILED' if failures else 'OK'} — {failures} failure(s)")
|
print(f"\n{'FAILED' if failures else 'OK'} — {failures} failure(s)")
|
||||||
sys.exit(1 if failures else 0)
|
sys.exit(1 if failures else 0)
|
||||||
@@ -0,0 +1,207 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""Regression test: the first-login service-sync hook (activation path).
|
||||||
|
|
||||||
|
WHY THIS EXISTS
|
||||||
|
New members were deliberately skipped by the Loomio/Matrix syncs until their
|
||||||
|
Cloudron account was activated (never invite a half-created account), but
|
||||||
|
nothing fired the syncs at the moment they DID activate — the 3x/day sweep
|
||||||
|
was the pickup, so a member could wait hours for forum/space access. The
|
||||||
|
hook on the broker identity path closes that gap. The guards that must hold
|
||||||
|
are proven here on a FIXTURE db with the network STUBBED:
|
||||||
|
|
||||||
|
CHEAP GATE
|
||||||
|
1. An established member (loomio_invited_at set) schedules NOTHING and
|
||||||
|
triggers no Cloudron call — one SELECT, zero further work.
|
||||||
|
ACTIVATION FILTER (fail-closed)
|
||||||
|
2. A never-invited member whose Cloudron account is NOT activated fires no
|
||||||
|
sync — never invite a half-created account.
|
||||||
|
3. A member Cloudron does not know fires no sync.
|
||||||
|
4. An unreadable Cloudron user list fires no sync (fail-closed — unlike
|
||||||
|
the sweep's fail-open roster filter, the hook decides whether to fire
|
||||||
|
at all, so it must be sure).
|
||||||
|
FIRES
|
||||||
|
5. A never-invited member whose Cloudron account IS activated fires BOTH
|
||||||
|
syncs (Loomio then Matrix) exactly once, as a background task.
|
||||||
|
SAFETY
|
||||||
|
6. A sync failure is swallowed and logged; the inflight flag clears so a
|
||||||
|
later gate hit can retry (sweep remains the backstop).
|
||||||
|
7. While a sync is in flight, further gate hits schedule nothing (the
|
||||||
|
full-roster reconcile already covers every pending member).
|
||||||
|
8. The gate NEVER raises — even when its own DB read fails, the broker
|
||||||
|
response must be untouched.
|
||||||
|
|
||||||
|
Runnable: python3 tests/test_first_login_sync.py (from the repo root)
|
||||||
|
"""
|
||||||
|
import os
|
||||||
|
import sys
|
||||||
|
import tempfile
|
||||||
|
import asyncio
|
||||||
|
|
||||||
|
os.environ["DB_PATH"] = os.path.join(tempfile.mkdtemp(), "fixture.db")
|
||||||
|
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
|
||||||
|
import app.main as m # noqa: E402
|
||||||
|
|
||||||
|
m.DB_PATH = os.environ["DB_PATH"]
|
||||||
|
|
||||||
|
results = []
|
||||||
|
|
||||||
|
|
||||||
|
def check(name, ok):
|
||||||
|
results.append((name, ok))
|
||||||
|
|
||||||
|
|
||||||
|
async def drain():
|
||||||
|
"""Await every task the gate scheduled, so assertions are deterministic."""
|
||||||
|
cur = asyncio.current_task()
|
||||||
|
pending = [t for t in asyncio.all_tasks() if t is not cur]
|
||||||
|
if pending:
|
||||||
|
await asyncio.gather(*pending, return_exceptions=True)
|
||||||
|
|
||||||
|
|
||||||
|
CLOUDRON_USERS = [] # what the (stubbed) Cloudron user list returns
|
||||||
|
cloudron_calls = 0
|
||||||
|
sync_calls = [] # ("loomio"|"matrix") in fire order
|
||||||
|
sync_gate = None # optional asyncio.Event: hold the sync in flight
|
||||||
|
|
||||||
|
|
||||||
|
async def fake_fetch_users(client=None):
|
||||||
|
global cloudron_calls
|
||||||
|
cloudron_calls += 1
|
||||||
|
if CLOUDRON_USERS is None:
|
||||||
|
return None
|
||||||
|
return CLOUDRON_USERS
|
||||||
|
|
||||||
|
|
||||||
|
async def fake_loomio_sync():
|
||||||
|
sync_calls.append("loomio")
|
||||||
|
if sync_gate is not None:
|
||||||
|
await sync_gate.wait()
|
||||||
|
|
||||||
|
|
||||||
|
async def fake_matrix_sync():
|
||||||
|
sync_calls.append("matrix")
|
||||||
|
|
||||||
|
|
||||||
|
async def boom_loomio_sync():
|
||||||
|
raise RuntimeError("simulated Loomio outage")
|
||||||
|
|
||||||
|
|
||||||
|
async def main():
|
||||||
|
global CLOUDRON_USERS, cloudron_calls, sync_calls, sync_gate
|
||||||
|
m.get_db().close() # create + migrate the fixture schema
|
||||||
|
conn = m.get_db()
|
||||||
|
for slug, email in (("est", "est@x.org"), ("new", "new@x.org"),
|
||||||
|
("pending", "pending@x.org"), ("ghost", "ghost@x.org")):
|
||||||
|
conn.execute(
|
||||||
|
"INSERT OR REPLACE INTO members (email, slug, active) VALUES (?, ?, 1)",
|
||||||
|
(email, slug),
|
||||||
|
)
|
||||||
|
conn.commit()
|
||||||
|
conn.close()
|
||||||
|
|
||||||
|
m.cloudron_fetch_all_users = fake_fetch_users
|
||||||
|
m.sync_loomio_memberships = fake_loomio_sync
|
||||||
|
m.sync_matrix_space = fake_matrix_sync
|
||||||
|
|
||||||
|
# --- 1. CHEAP GATE: an established member schedules nothing at all. ---
|
||||||
|
m.mark_loomio_invited("est@x.org")
|
||||||
|
cloudron_calls = 0
|
||||||
|
sync_calls = []
|
||||||
|
m.maybe_first_login_service_sync("est@x.org")
|
||||||
|
await drain()
|
||||||
|
check("established member: no sync scheduled", sync_calls == [])
|
||||||
|
check("established member: no Cloudron call", cloudron_calls == 0)
|
||||||
|
check("established member: inflight flag stays clear",
|
||||||
|
m._first_login_sync_inflight is False)
|
||||||
|
|
||||||
|
# --- 2. ACTIVATION FILTER: account not activated -> never fires. ---
|
||||||
|
CLOUDRON_USERS = [{"email": "new@x.org", "inviteAccepted": False}]
|
||||||
|
cloudron_calls = 0
|
||||||
|
sync_calls = []
|
||||||
|
m.maybe_first_login_service_sync("new@x.org")
|
||||||
|
await drain()
|
||||||
|
check("not-activated member: no sync fired", sync_calls == [])
|
||||||
|
check("not-activated member: activation WAS checked", cloudron_calls == 1)
|
||||||
|
check("not-activated member: inflight flag cleared",
|
||||||
|
m._first_login_sync_inflight is False)
|
||||||
|
|
||||||
|
# --- 3. Unknown to Cloudron -> never fires. ---
|
||||||
|
CLOUDRON_USERS = [{"email": "other@x.org", "inviteAccepted": True}]
|
||||||
|
sync_calls = []
|
||||||
|
m.maybe_first_login_service_sync("ghost@x.org")
|
||||||
|
await drain()
|
||||||
|
check("member unknown to Cloudron: no sync fired", sync_calls == [])
|
||||||
|
|
||||||
|
# --- 4. FAIL CLOSED: unreadable Cloudron user list -> never fires. ---
|
||||||
|
CLOUDRON_USERS = None
|
||||||
|
sync_calls = []
|
||||||
|
m.maybe_first_login_service_sync("new@x.org")
|
||||||
|
await drain()
|
||||||
|
check("cloudron outage: no sync fired (fail-closed)", sync_calls == [])
|
||||||
|
check("cloudron outage: inflight flag cleared",
|
||||||
|
m._first_login_sync_inflight is False)
|
||||||
|
|
||||||
|
# --- 5. FIRES: activated + outstanding invites -> both syncs, once each. ---
|
||||||
|
CLOUDRON_USERS = [{"email": "NEW@x.org", "inviteAccepted": True}] # case-insensitive
|
||||||
|
sync_calls = []
|
||||||
|
m.maybe_first_login_service_sync("new@x.org")
|
||||||
|
await drain()
|
||||||
|
check("activated member: Loomio sync fired first, then Matrix",
|
||||||
|
sync_calls == ["loomio", "matrix"])
|
||||||
|
check("activated member: inflight flag cleared after the run",
|
||||||
|
m._first_login_sync_inflight is False)
|
||||||
|
|
||||||
|
# --- 6. SAFETY: a sync failure is swallowed; the gate stays usable. ---
|
||||||
|
m.sync_loomio_memberships = boom_loomio_sync
|
||||||
|
sync_calls = []
|
||||||
|
m.maybe_first_login_service_sync("new@x.org")
|
||||||
|
err = None
|
||||||
|
try:
|
||||||
|
await drain()
|
||||||
|
except Exception as e: # pragma: no cover
|
||||||
|
err = e
|
||||||
|
check("loomio outage: nothing escapes the background task", err is None)
|
||||||
|
check("loomio outage: matrix still attempted", sync_calls == ["matrix"])
|
||||||
|
check("loomio outage: inflight flag cleared for the next attempt",
|
||||||
|
m._first_login_sync_inflight is False)
|
||||||
|
m.sync_loomio_memberships = fake_loomio_sync
|
||||||
|
|
||||||
|
# --- 7. INFLIGHT: a second gate hit while a sync runs schedules nothing. ---
|
||||||
|
sync_gate = asyncio.Event()
|
||||||
|
sync_calls = []
|
||||||
|
m.maybe_first_login_service_sync("new@x.org") # task starts, blocks in loomio stub
|
||||||
|
await asyncio.sleep(0) # let it reach the blocking stub
|
||||||
|
check("inflight: flag is set while the sync runs",
|
||||||
|
m._first_login_sync_inflight is True)
|
||||||
|
m.maybe_first_login_service_sync("pending@x.org") # second member, same moment
|
||||||
|
m.maybe_first_login_service_sync("new@x.org") # same member again
|
||||||
|
sync_gate.set()
|
||||||
|
await drain()
|
||||||
|
check("inflight: exactly ONE sync ran for three gate hits",
|
||||||
|
sync_calls == ["loomio", "matrix"])
|
||||||
|
sync_gate = None
|
||||||
|
|
||||||
|
# --- 8. The gate NEVER raises, even if its own DB read fails. ---
|
||||||
|
real_invited = m.get_loomio_invited_at
|
||||||
|
|
||||||
|
def broken_invited(_email):
|
||||||
|
raise RuntimeError("simulated DB failure")
|
||||||
|
|
||||||
|
m.get_loomio_invited_at = broken_invited
|
||||||
|
err = None
|
||||||
|
try:
|
||||||
|
m.maybe_first_login_service_sync("new@x.org")
|
||||||
|
await drain()
|
||||||
|
except Exception as e:
|
||||||
|
err = e
|
||||||
|
check("gate swallows its own DB failure (broker response untouched)", err is None)
|
||||||
|
check("gate DB failure schedules nothing", sync_calls == ["loomio", "matrix"])
|
||||||
|
m.get_loomio_invited_at = real_invited
|
||||||
|
|
||||||
|
|
||||||
|
asyncio.run(main())
|
||||||
|
fails = sum(not ok for _, ok in results)
|
||||||
|
for name, ok in results:
|
||||||
|
print(f" [{'PASS' if ok else 'FAIL'}] {name}")
|
||||||
|
print(f"\n{len(results) - fails}/{len(results)} passed")
|
||||||
|
sys.exit(1 if fails else 0)
|
||||||
Reference in new issue
Block a user