activation hook: fire Loomio+Matrix syncs at first broker login
New members were skipped by the syncs until their Cloudron account was activated (deliberately - never invite a half-created account), but nothing fired the syncs at the moment they DID activate: the 3x/day sweep was the only pickup, so a newly-activated member could wait hours for forum/space access (limdingwen waited ~5h on 2026-11-06). Hook the broker identity path (_verify_broker_secret): after an active member resolves, if loomio_invited_at is still null, confirm their Cloudron account is activated (inviteAccepted, fail-CLOSED - an unreadable user list never fires) and run the same idempotent syncs the sweep runs, as a background task off the response. Established members pay one indexed SELECT; concurrent first-logins collapse into one run via an inflight flag; each service is independently best-effort so a Loomio outage cannot block the Matrix invite or the response. Tests: tests/test_first_login_sync.py (18 checks, fixture db + stubbed network). Also fix test_allowance_shortfall, whose expected exclusion reference still asserted the pre-a88ca86 placeholder. Sweep, provisioning and deactivation paths are untouched.
This commit is contained in:
1 parent
a88ca860d4
commit
ed66d73327
4 files changed
+348
-2
No files matched your search
@@ -107,6 +107,17 @@ Three jobs run inside the container via `CloudronManifest.json` → `addons.sche
|
||||
lapsed members, syncs Loomio + Matrix. Idempotent; a missed webhook
|
||||
self-heals within a day. The same sweep also runs the payment-driven
|
||||
allowance reconciliation (below).
|
||||
|
||||
**First-login activation hook:** the sweep is also the backstop for service
|
||||
invites, but it is no longer the only trigger. When a member first presents
|
||||
themselves on the broker identity path (the dashboard's first login), the
|
||||
portal checks — only if their Loomio invite is still outstanding
|
||||
(`loomio_invited_at` is null) — that their Cloudron account is now activated
|
||||
(`inviteAccepted`), and if so fires the same idempotent Loomio + Matrix syncs
|
||||
as a background task. Established members cost one SELECT; the response never
|
||||
waits on Loomio or Matrix; failures are logged and non-fatal; unactivated
|
||||
accounts are never invited (the check fails closed). So a new member gets
|
||||
forum + Space access at their first login instead of at the next sweep.
|
||||
- **rag_refresh** (daily 3:30am UTC) — `rag_refresh.py` pulls this repo's
|
||||
sibling docs from the Gitea raw API (public, no credentials), uploads changed
|
||||
files to Open WebUI (`OWUI_API_KEY`), attaches them to the shared knowledge
|
||||
|
||||
+126
@@ -3891,6 +3891,127 @@ async def admin_model_usage(request: Request, days: int = 0):
|
||||
# own keys — never another member's, and never the chat key.
|
||||
|
||||
|
||||
# --- First-login service sync (activation hook) ---
|
||||
# A newly provisioned member is deliberately skipped by the Loomio/Matrix syncs
|
||||
# until their Cloudron account is activated (so a half-created account is never
|
||||
# invited), and the 3x/day sweep was the only thing that picked them up AFTER
|
||||
# they did activate — a new member could wait hours for forum/space access.
|
||||
# The broker identity path is where the member first presents themselves after
|
||||
# setting up (the dashboard calls it on first login), so it is the natural
|
||||
# trigger: when a broker-authenticated, active member resolves AND their Loomio
|
||||
# invite is still outstanding, confirm their Cloudron account is now activated
|
||||
# and, only then, fire the same idempotent syncs the sweep runs. The syncs run
|
||||
# as a background task, so no response ever waits on Loomio or Matrix.
|
||||
|
||||
_first_login_sync_inflight = False
|
||||
|
||||
|
||||
def _member_slug(email: str) -> str:
|
||||
"""Member slug for log lines (slugs survive log redaction; emails don't)."""
|
||||
try:
|
||||
conn = get_db()
|
||||
row = conn.execute(
|
||||
"SELECT slug FROM members WHERE email = ?", (email,)
|
||||
).fetchone()
|
||||
conn.close()
|
||||
return (row[0] if row and row[0] else "") or "<slug-unset>"
|
||||
except Exception:
|
||||
return "<slug-unset>"
|
||||
|
||||
|
||||
async def _member_cloudron_activated(email: str) -> bool:
|
||||
"""True only when the member's Cloudron account is CONFIRMED activated.
|
||||
|
||||
Fail-CLOSED on purpose: if the Cloudron user list cannot be read (or the
|
||||
member is not on it), report not-activated and let the 3x/day sweep try
|
||||
later. The hook must never be the path that invites a half-created account,
|
||||
so unlike the sweep's fail-open roster filter, an unverifiable activation
|
||||
state means "do not fire".
|
||||
"""
|
||||
async with httpx.AsyncClient(timeout=15.0) as client:
|
||||
users = await cloudron_fetch_all_users(client)
|
||||
if not users:
|
||||
return False
|
||||
for u in users:
|
||||
if (u.get("email") or "").lower() == email.lower():
|
||||
return bool(u.get("inviteAccepted"))
|
||||
return False
|
||||
|
||||
|
||||
async def _first_login_service_sync(email: str) -> None:
|
||||
"""Fire the Loomio + Matrix Space syncs for a newly-activated member.
|
||||
|
||||
Runs as a background task off the broker request (a Loomio outage must
|
||||
never block a dashboard). The syncs are the SAME idempotent functions the
|
||||
sweep and provisioning run — guarded by members.loomio_invited_at and the
|
||||
live-roster pre-check — so a re-fire cannot double-invite. Best-effort:
|
||||
any failure is logged and swallowed; the sweep remains the backstop.
|
||||
"""
|
||||
global _first_login_sync_inflight
|
||||
slug = _member_slug(email)
|
||||
try:
|
||||
if not await _member_cloudron_activated(email):
|
||||
logger.info(
|
||||
"first-login sync: %s Cloudron account not yet activated; not "
|
||||
"firing (the sweep covers it later)", slug,
|
||||
)
|
||||
return
|
||||
# Each service independently best-effort: a Loomio outage must never
|
||||
# stop the Matrix invite (or vice versa), and neither may raise out of
|
||||
# this background task.
|
||||
for label, sync_fn in (
|
||||
("Loomio", sync_loomio_memberships),
|
||||
("Matrix", sync_matrix_space),
|
||||
):
|
||||
try:
|
||||
logger.info("first-login sync: firing %s sync for %s", label, slug)
|
||||
await sync_fn()
|
||||
except Exception:
|
||||
logger.warning(
|
||||
"first-login sync: %s sync for %s failed (non-fatal; the "
|
||||
"sweep covers later)", label, slug, exc_info=True,
|
||||
)
|
||||
logger.info(
|
||||
"first-login sync: done for %s (loomio_invited_at=%s)",
|
||||
slug, get_loomio_invited_at(email),
|
||||
)
|
||||
except Exception:
|
||||
logger.warning(
|
||||
"first-login sync for %s failed (non-fatal; sweep covers later)",
|
||||
slug, exc_info=True,
|
||||
)
|
||||
finally:
|
||||
_first_login_sync_inflight = False
|
||||
|
||||
|
||||
def maybe_first_login_service_sync(email: str) -> None:
|
||||
"""Cheap gate on the broker hot path; never raises.
|
||||
|
||||
Established members (loomio_invited_at set) cost one indexed SELECT and
|
||||
return. A never-invited member schedules the sync as a background task.
|
||||
While one first-login sync is in flight, further gate hits do nothing: the
|
||||
syncs are full-roster reconciles, so the running pass already covers every
|
||||
pending member.
|
||||
"""
|
||||
global _first_login_sync_inflight
|
||||
try:
|
||||
if get_loomio_invited_at(email):
|
||||
return # established member — zero further work
|
||||
if _first_login_sync_inflight:
|
||||
return # a sync is already running; it covers every pending member
|
||||
loop = asyncio.get_running_loop()
|
||||
except Exception:
|
||||
return # the gate must never break the broker response
|
||||
_first_login_sync_inflight = True
|
||||
try:
|
||||
loop.create_task(_first_login_service_sync(email))
|
||||
except Exception:
|
||||
_first_login_sync_inflight = False
|
||||
logger.warning(
|
||||
"first-login sync could not be scheduled (non-fatal)", exc_info=True,
|
||||
)
|
||||
|
||||
|
||||
def _verify_broker_secret(request: Request) -> str:
|
||||
"""Return the authenticated member's email, or 401.
|
||||
|
||||
@@ -3918,6 +4039,11 @@ def _verify_broker_secret(request: Request) -> str:
|
||||
# The member must be active in our DB before they can manage keys.
|
||||
if not get_member_key(email):
|
||||
raise HTTPException(403, "No active membership")
|
||||
# Activation hook: if this active member's service invites are still
|
||||
# outstanding AND their Cloudron account is now activated, fire the Loomio
|
||||
# + Matrix syncs (background task; established members cost one SELECT).
|
||||
# Best-effort by construction — it must never affect this response.
|
||||
maybe_first_login_service_sync(email)
|
||||
return email
|
||||
|
||||
|
||||
|
||||
@@ -109,9 +109,11 @@ check("reason names the cycle",
|
||||
"allowance shortfall, truncated paid cycle 2026-10-07→2026-11-02 (OC billing alignment)")
|
||||
|
||||
# --- the manual-remediation exclusion is keyed on the charge ref alone
|
||||
# --- (no member PII in this public repo; identity/timeline live in ops docs)
|
||||
# --- (no member PII in this public repo; identity/timeline live in ops docs;
|
||||
# --- the OC tx id IS the reference key and was approved as such — see the
|
||||
# --- commit that restored it to match the deployed image)
|
||||
check("remediated charge excluded by reference",
|
||||
"[redacted member reference]" in SHORTFALL_REMEDIATED, True)
|
||||
"tx_Kr6n3rW577OSjr2bIt5IC" in SHORTFALL_REMEDIATED, True)
|
||||
|
||||
print(f"\n{'FAILED' if failures else 'OK'} — {failures} failure(s)")
|
||||
sys.exit(1 if failures else 0)
|
||||
@@ -0,0 +1,207 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Regression test: the first-login service-sync hook (activation path).
|
||||
|
||||
WHY THIS EXISTS
|
||||
New members were deliberately skipped by the Loomio/Matrix syncs until their
|
||||
Cloudron account was activated (never invite a half-created account), but
|
||||
nothing fired the syncs at the moment they DID activate — the 3x/day sweep
|
||||
was the pickup, so a member could wait hours for forum/space access. The
|
||||
hook on the broker identity path closes that gap. The guards that must hold
|
||||
are proven here on a FIXTURE db with the network STUBBED:
|
||||
|
||||
CHEAP GATE
|
||||
1. An established member (loomio_invited_at set) schedules NOTHING and
|
||||
triggers no Cloudron call — one SELECT, zero further work.
|
||||
ACTIVATION FILTER (fail-closed)
|
||||
2. A never-invited member whose Cloudron account is NOT activated fires no
|
||||
sync — never invite a half-created account.
|
||||
3. A member Cloudron does not know fires no sync.
|
||||
4. An unreadable Cloudron user list fires no sync (fail-closed — unlike
|
||||
the sweep's fail-open roster filter, the hook decides whether to fire
|
||||
at all, so it must be sure).
|
||||
FIRES
|
||||
5. A never-invited member whose Cloudron account IS activated fires BOTH
|
||||
syncs (Loomio then Matrix) exactly once, as a background task.
|
||||
SAFETY
|
||||
6. A sync failure is swallowed and logged; the inflight flag clears so a
|
||||
later gate hit can retry (sweep remains the backstop).
|
||||
7. While a sync is in flight, further gate hits schedule nothing (the
|
||||
full-roster reconcile already covers every pending member).
|
||||
8. The gate NEVER raises — even when its own DB read fails, the broker
|
||||
response must be untouched.
|
||||
|
||||
Runnable: python3 tests/test_first_login_sync.py (from the repo root)
|
||||
"""
|
||||
import os
|
||||
import sys
|
||||
import tempfile
|
||||
import asyncio
|
||||
|
||||
os.environ["DB_PATH"] = os.path.join(tempfile.mkdtemp(), "fixture.db")
|
||||
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
|
||||
import app.main as m # noqa: E402
|
||||
|
||||
m.DB_PATH = os.environ["DB_PATH"]
|
||||
|
||||
results = []
|
||||
|
||||
|
||||
def check(name, ok):
|
||||
results.append((name, ok))
|
||||
|
||||
|
||||
async def drain():
|
||||
"""Await every task the gate scheduled, so assertions are deterministic."""
|
||||
cur = asyncio.current_task()
|
||||
pending = [t for t in asyncio.all_tasks() if t is not cur]
|
||||
if pending:
|
||||
await asyncio.gather(*pending, return_exceptions=True)
|
||||
|
||||
|
||||
CLOUDRON_USERS = [] # what the (stubbed) Cloudron user list returns
|
||||
cloudron_calls = 0
|
||||
sync_calls = [] # ("loomio"|"matrix") in fire order
|
||||
sync_gate = None # optional asyncio.Event: hold the sync in flight
|
||||
|
||||
|
||||
async def fake_fetch_users(client=None):
|
||||
global cloudron_calls
|
||||
cloudron_calls += 1
|
||||
if CLOUDRON_USERS is None:
|
||||
return None
|
||||
return CLOUDRON_USERS
|
||||
|
||||
|
||||
async def fake_loomio_sync():
|
||||
sync_calls.append("loomio")
|
||||
if sync_gate is not None:
|
||||
await sync_gate.wait()
|
||||
|
||||
|
||||
async def fake_matrix_sync():
|
||||
sync_calls.append("matrix")
|
||||
|
||||
|
||||
async def boom_loomio_sync():
|
||||
raise RuntimeError("simulated Loomio outage")
|
||||
|
||||
|
||||
async def main():
|
||||
global CLOUDRON_USERS, cloudron_calls, sync_calls, sync_gate
|
||||
m.get_db().close() # create + migrate the fixture schema
|
||||
conn = m.get_db()
|
||||
for slug, email in (("est", "est@x.org"), ("new", "new@x.org"),
|
||||
("pending", "pending@x.org"), ("ghost", "ghost@x.org")):
|
||||
conn.execute(
|
||||
"INSERT OR REPLACE INTO members (email, slug, active) VALUES (?, ?, 1)",
|
||||
(email, slug),
|
||||
)
|
||||
conn.commit()
|
||||
conn.close()
|
||||
|
||||
m.cloudron_fetch_all_users = fake_fetch_users
|
||||
m.sync_loomio_memberships = fake_loomio_sync
|
||||
m.sync_matrix_space = fake_matrix_sync
|
||||
|
||||
# --- 1. CHEAP GATE: an established member schedules nothing at all. ---
|
||||
m.mark_loomio_invited("est@x.org")
|
||||
cloudron_calls = 0
|
||||
sync_calls = []
|
||||
m.maybe_first_login_service_sync("est@x.org")
|
||||
await drain()
|
||||
check("established member: no sync scheduled", sync_calls == [])
|
||||
check("established member: no Cloudron call", cloudron_calls == 0)
|
||||
check("established member: inflight flag stays clear",
|
||||
m._first_login_sync_inflight is False)
|
||||
|
||||
# --- 2. ACTIVATION FILTER: account not activated -> never fires. ---
|
||||
CLOUDRON_USERS = [{"email": "new@x.org", "inviteAccepted": False}]
|
||||
cloudron_calls = 0
|
||||
sync_calls = []
|
||||
m.maybe_first_login_service_sync("new@x.org")
|
||||
await drain()
|
||||
check("not-activated member: no sync fired", sync_calls == [])
|
||||
check("not-activated member: activation WAS checked", cloudron_calls == 1)
|
||||
check("not-activated member: inflight flag cleared",
|
||||
m._first_login_sync_inflight is False)
|
||||
|
||||
# --- 3. Unknown to Cloudron -> never fires. ---
|
||||
CLOUDRON_USERS = [{"email": "other@x.org", "inviteAccepted": True}]
|
||||
sync_calls = []
|
||||
m.maybe_first_login_service_sync("ghost@x.org")
|
||||
await drain()
|
||||
check("member unknown to Cloudron: no sync fired", sync_calls == [])
|
||||
|
||||
# --- 4. FAIL CLOSED: unreadable Cloudron user list -> never fires. ---
|
||||
CLOUDRON_USERS = None
|
||||
sync_calls = []
|
||||
m.maybe_first_login_service_sync("new@x.org")
|
||||
await drain()
|
||||
check("cloudron outage: no sync fired (fail-closed)", sync_calls == [])
|
||||
check("cloudron outage: inflight flag cleared",
|
||||
m._first_login_sync_inflight is False)
|
||||
|
||||
# --- 5. FIRES: activated + outstanding invites -> both syncs, once each. ---
|
||||
CLOUDRON_USERS = [{"email": "NEW@x.org", "inviteAccepted": True}] # case-insensitive
|
||||
sync_calls = []
|
||||
m.maybe_first_login_service_sync("new@x.org")
|
||||
await drain()
|
||||
check("activated member: Loomio sync fired first, then Matrix",
|
||||
sync_calls == ["loomio", "matrix"])
|
||||
check("activated member: inflight flag cleared after the run",
|
||||
m._first_login_sync_inflight is False)
|
||||
|
||||
# --- 6. SAFETY: a sync failure is swallowed; the gate stays usable. ---
|
||||
m.sync_loomio_memberships = boom_loomio_sync
|
||||
sync_calls = []
|
||||
m.maybe_first_login_service_sync("new@x.org")
|
||||
err = None
|
||||
try:
|
||||
await drain()
|
||||
except Exception as e: # pragma: no cover
|
||||
err = e
|
||||
check("loomio outage: nothing escapes the background task", err is None)
|
||||
check("loomio outage: matrix still attempted", sync_calls == ["matrix"])
|
||||
check("loomio outage: inflight flag cleared for the next attempt",
|
||||
m._first_login_sync_inflight is False)
|
||||
m.sync_loomio_memberships = fake_loomio_sync
|
||||
|
||||
# --- 7. INFLIGHT: a second gate hit while a sync runs schedules nothing. ---
|
||||
sync_gate = asyncio.Event()
|
||||
sync_calls = []
|
||||
m.maybe_first_login_service_sync("new@x.org") # task starts, blocks in loomio stub
|
||||
await asyncio.sleep(0) # let it reach the blocking stub
|
||||
check("inflight: flag is set while the sync runs",
|
||||
m._first_login_sync_inflight is True)
|
||||
m.maybe_first_login_service_sync("pending@x.org") # second member, same moment
|
||||
m.maybe_first_login_service_sync("new@x.org") # same member again
|
||||
sync_gate.set()
|
||||
await drain()
|
||||
check("inflight: exactly ONE sync ran for three gate hits",
|
||||
sync_calls == ["loomio", "matrix"])
|
||||
sync_gate = None
|
||||
|
||||
# --- 8. The gate NEVER raises, even if its own DB read fails. ---
|
||||
real_invited = m.get_loomio_invited_at
|
||||
|
||||
def broken_invited(_email):
|
||||
raise RuntimeError("simulated DB failure")
|
||||
|
||||
m.get_loomio_invited_at = broken_invited
|
||||
err = None
|
||||
try:
|
||||
m.maybe_first_login_service_sync("new@x.org")
|
||||
await drain()
|
||||
except Exception as e:
|
||||
err = e
|
||||
check("gate swallows its own DB failure (broker response untouched)", err is None)
|
||||
check("gate DB failure schedules nothing", sync_calls == ["loomio", "matrix"])
|
||||
m.get_loomio_invited_at = real_invited
|
||||
|
||||
|
||||
asyncio.run(main())
|
||||
fails = sum(not ok for _, ok in results)
|
||||
for name, ok in results:
|
||||
print(f" [{'PASS' if ok else 'FAIL'}] {name}")
|
||||
print(f"\n{len(results) - fails}/{len(results)} passed")
|
||||
sys.exit(1 if fails else 0)
|
||||
Reference in new issue
Block a user