This website requires JavaScript.
2c38f4d2c3
upgrade window 2026-11-09: executed (gateway 1.108.3, chat overlay 1.4.8) — record the outcome, the 400 MB task-limit OOM and its --no-backup remedy, and refresh the live image pins
main
inference-bot
2026-11-09 05:04:27 +00:00
52e3c9d502
runbook: re-check the upgrade window against live state — correct the rollback tag, mark the prebuilt candidate stale, extend the marker list to ten, and flag the failing backups as the open risk
inference-bot
2026-11-09 04:17:06 +00:00
5a4492e725
ops: document the member provisioning boundary (org-backed, credit-pack, manual, hidden-email) and the daily paid-but-unprovisioned watchdog
inference-bot
2026-11-07 00:27:15 +00:00
0f40dcc519
runbook: add the standing supply-chain IoC check to the gateway verify steps
inference-bot
2026-11-06 04:11:11 +00:00
816961fb2d
runbook: 2026-11-08 upgrade window — LiteLLM 1.103.2→1.108.0 + LibreChat overlay 1.4.5→1.4.8
inference-bot
2026-11-06 04:11:01 +00:00
fcd4ac6943
incident: portal's copied LiteLLM DB credential rotted after the 2026-11-05 gateway restore; all-time spend silently read $0 — timeline, fix, the digest spend-consistency guard, and the two-copies-of-one-secret dependency flagged for the next rotation
inference-bot
2026-11-05 19:10:26 +00:00
bb10d93412
incident: 2026-11-05 gateway addon loss during multi-app deploy — wiped LiteLLM DB, restored from the pre-deploy snapshot the same hour
inference-bot
2026-11-05 16:36:05 +00:00
4d3f539a84
ops: record the decision to keep the historical ledger test rows (audit L7 — net zero; rewriting the money ledger is worse than the noise)
inference-bot
2026-11-05 16:26:17 +00:00
3bf152b58a
ops: record the LibreChat key-feed design decision (audit M2) — kept with monitoring; per-member fetch rejected as the riskier path
inference-bot
2026-11-05 16:21:52 +00:00
b1a1c70e8a
ops: record the allowance-shortfall manual-remediation ledger
inference-bot
2026-11-05 15:59:13 +00:00
a9edaa3e4b
agent-fleet: record the Matrix homeserver accounts moved off the member docs
ops-bot
2026-11-04 21:29:29 +00:00
835aed2852
Record the DNS cutover as complete; flag the old box as an unresolved exposure (#5 )
inference-bot
2026-11-04 14:49:04 +00:00
6f2a6d01ed
docs: record the DNS cutover as complete; flag the old box as an unresolved internet exposure
docs/oldbox-exposure
inference-bot
2026-11-04 14:48:53 +00:00
6e777be842
Infrastructure audit: app inventory, backups, CI, provider pricing, agent fleet (#4 )
inference-bot
2026-11-04 14:47:18 +00:00
2eb88f55b4
docs: infrastructure audit — app inventory, backups, CI, provider pricing, agent fleet
docs/infra-audit-2026-11-04
inference-bot
2026-11-04 14:46:12 +00:00
8b33d823c1
Merge pull request 'operator-token-runbook: credential rotation policy and reminder (co-op/support#35)' (#3 ) from rotation-policy into main
inference-bot
2026-11-04 14:37:22 +00:00
43f8f8c3fe
operator-token-runbook: add the credential rotation policy and reminder
rotation-policy
inference-bot
2026-11-04 14:36:57 +00:00
c432edbae2
membership: record the allowance renewal and lapse policy (2026-11-04)
membership-allowance-policy
inference-bot
2026-11-04 05:20:45 +00:00
63d14156ec
operator-token-runbook: add a credential inventory
docs/credential-inventory
solo.ist
2026-11-04 01:18:39 +00:00
11b10e2b03
Consolidate co-op/runbooks into co-op/ops-docs
inference-bot
2026-11-03 23:34:03 +00:00
9307b81f09
Add README explaining this git-only operator docs repository
inference-bot
2026-11-03 23:30:28 +00:00
196bb11bc6
runbooks: record that the private twin is archived
inference-bot
2026-11-02 15:42:20 +00:00
098b0e9caa
runbooks: the migration is complete — mark it as history, not pending work
inference-bot
2026-11-02 15:37:28 +00:00
5de16b6537
migration: cutover complete — DNS switch verified, baseline recaptured against the new box, server.inference.coop resolved; all-clear email drafted
inference-bot
2026-11-02 07:53:40 +00:00
70c1208047
migration plan: state of play, for whoever runs the cutover
inference-bot
2026-11-02 03:33:57 +00:00
267db1fddd
baseline: recapture migration-before-state against the live box
inference-bot
2026-11-02 00:08:22 +00:00
48c4827957
dns change sheet: rebuild from the registrar's panel
inference-bot
2026-11-01 23:49:45 +00:00
049d8c59ad
migration plan: close the credential items; correct the mail-port claim
inference-bot
2026-11-01 23:07:34 +00:00
fec3826fd8
migration plan: port 25 inbound confirmed open; outbound 25 not needed
inference-bot
2026-11-01 21:58:41 +00:00
a525b1403b
migration plan: record the confirmed provider tier and the snapshot gap
inference-bot
2026-11-01 21:45:22 +00:00
12a0676d62
README: the runbooks are public — remove the 'Private' line and state the review intent
inference-bot
2026-11-01 16:38:54 +00:00
86e2afe5d0
dns sheet: drop the phantom docs.inference.coop rows; record that the zone is wildcarded
inference-bot
2026-11-01 00:46:20 +00:00
445d8b0c3e
provider pricing: measure our Regolo usage — $4.38/month, ~1.1% of inference spend
inference-bot
2026-11-01 00:33:09 +00:00
d7e057386d
provider pricing: Regolo's 2026-11-01 change, and the image model's EUR-as-USD bug
inference-bot
2026-11-01 00:31:04 +00:00
6dce468bf3
migration: install verified on the new box; correct two wrong claims and list the ports still shut
inference-bot
2026-10-30 23:22:40 +00:00
717c72c9bb
migration: refine the pre-install step — no --domain, so no ACME attempts at all
inference-bot
2026-10-30 22:59:27 +00:00
3cae100038
migration: record the off-box credential state; one custodian is a co-op risk
inference-bot
2026-10-30 22:49:52 +00:00
a5d5fb140d
migration: credentials status + the ready-to-post announcement
inference-bot
2026-10-30 21:26:35 +00:00
00b26b3e1e
migration: point §5 at verify-restore.py, the executable post-restore check
inference-bot
2026-10-30 21:23:48 +00:00
b313105148
migration: add the before-state snapshot; S3 secret is NOT off-box
inference-bot
2026-10-30 20:04:07 +00:00
144e7e77ca
migration: IPv4 firewall rules are missing; 25 split inbound/outbound
inference-bot
2026-10-30 18:22:17 +00:00
7d685120fa
migration: exact firewall port set; flag unverified mailbox backup coverage
inference-bot
2026-10-30 18:11:58 +00:00
deb3ab2af3
migration: resolve OS support, correct the version floor, split install from wizard
inference-bot
2026-10-30 18:01:10 +00:00
3a89c77cfc
migration: fill in the new box addresses, decide IPv6, add box specs
inference-bot
2026-10-30 17:58:48 +00:00
d1ff7fb0ff
dns: add the migration change sheet; correct AAAA scope, close the mount unknown
inference-bot
2026-10-30 17:48:22 +00:00
a2088e3976
migration runbook: fix two errors and five gaps found in a read-only review
inference-bot
2026-10-30 15:32:14 +00:00
14d36f8877
Merge #9 : incidents: add the 2026-10-02 gateway outage write-up
inference-bot
2026-10-30 15:03:53 +00:00
979bc8a984
Merge #9 : add the 2026-10-02 gateway outage write-up
inference-bot
2026-10-30 15:03:23 +00:00
eec9572a3f
Add Cloudron server migration runbook (Hostinger → Infomaniak)
inference-bot
2026-10-29 22:46:40 +00:00
7d6212ace8
runbook: how to confirm the webhook URL match, and what the watch job does and does not cover
inference-bot
2026-10-28 14:19:37 +00:00
340fbe9062
runbook: correct the WEBHOOK_TOKEN rotation state (rotated 2026-10-27; URL match unverified)
inference-bot
2026-10-28 14:03:56 +00:00
d7cee510cb
runbook: admin/webhook token split, metrics_snapshot task, correct sweep auth
inference-bot
2026-10-28 13:55:28 +00:00
891490d0d7
incidents: fix the 2026-10-02 byline date and the outage duration (review)
solo.ist and Claude Fable 5.1
2026-10-27 01:34:15 -04:00
317af55621
incidents: add the 2026-10-02 gateway outage write-up (from the unmerged incident branch)
solo.ist
2026-10-26 02:11:33 -04:00
8581983286
docs: chat task models — OWUI task model pinned to greenpt/glm-5.3-flash, suggestions state
Nathan Schneider
2026-10-19 14:53:20 -06:00
ed9e512260
librechat-sandbox: per-member image billing restored; system key is safety-net only
Nathan Schneider
2026-10-19 08:50:14 -06:00
af1e413e5b
librechat-sandbox: image gen bills to system key (no gate); fallback usage monitored in digest
Nathan Schneider
2026-10-19 08:30:40 -06:00
7c16c7cc2a
librechat-sandbox: homepage-canonical palette + green logo (flash-glitch fix)
Nathan Schneider
2026-10-18 23:50:54 -06:00
d5ee87d09c
librechat-sandbox: per-member key auto-load (portal feed + sync-keys.js), redaction gotcha
Nathan Schneider
2026-10-18 23:29:16 -06:00
aaf3091f0f
librechat-sandbox: manifest-pinned deploy flow, builder-cache + ECONNRESET pitfalls, branding overlay, keyless image path; current deployed tag
Nathan Schneider
2026-10-18 22:53:12 -06:00
b90af7aa64
LibreChat sandbox: accessible image generation via MCP tool; two MCP servers; new overlay tag
Nathan Schneider
2026-10-18 20:54:31 -06:00
b743f77737
LibreChat sandbox: rewrite maintenance doc for the volume-config + overlay-image architecture
Nathan Schneider
2026-10-18 20:31:03 -06:00
548d1e5a60
Add LibreChat sandbox maintenance documentation
Nathan Schneider
2026-10-18 15:32:23 -06:00
86663fd282
runbook: spaCy model in /app/data for EPUB parsing + restore caveat (issue #20 )
inference-bot
2026-10-14 10:27:24 -06:00
a6d824abef
docs: scheduled tasks in portal/gateway (self-maintaining apps) + operator verification steps
inference-bot
2026-10-12 19:32:32 -06:00
f9d88b7f63
Remove completed Cloudron 10 follow-ups section
inference-bot
2026-09-23 14:17:09 -06:00
a89b37e78f
Mark proxyAuth->OIDC migration done; only SnappyMail remains
inference-bot
2026-09-23 14:16:04 -06:00
ebae15cb3c
Track Cloudron 10 follow-ups and add SSO migration plan
inference-bot
2026-09-23 08:03:21 -06:00
6a3ef4d831
Correct OC runbook: email null was a missing email scope, not a regression
inference-bot
2026-09-22 15:50:27 -06:00
1bdc40a3a5
Document OC email-visibility regression and manual-provision workaround
inference-bot
2026-09-22 15:45:22 -06:00
0a96d2f73c
Add operator token runbook (two-token split, configure/env gotcha)
inference-bot
2026-09-19 17:21:08 -06:00