Wire-verified on the pinned v1.103.2 (2026-11-05): NO_DOCS/NO_REDOC/
NO_OPENAPI do disable /docs, /redoc and /openapi.json (live 404s), but
DISABLE_ADMIN_UI only disables the UI's SSO login-redirect routes in this
version — the static /ui/ bundle is mounted unconditionally and still
serves (login stays master-key-gated). README 'Admin access' and the
start.sh comment now say exactly that, with the build-patch path if the
bundle itself should ever be removed. There is no env/flag for it on
this version, so the README is corrected instead (M4).
CORS reality recorded: LITELLM_CORS_ORIGINS is set at the app level to *
by co-op decision (co-op/support#38); start.sh's chat-origin fallback
applies only if the app env omits it. PublicAI (removed 2026-10-16,
replaced by Regolo) dropped from the provider lists; chat is LibreChat,
not Open WebUI. The embedded first-boot fallback config cannot be served
(overwritten by the same boot's image sync) — noted inline.
Manifest contactEmail: botbot@hermes -> info@inference.coop (L6).
The pass-through row for /v1/systemone carried the correct spend but
prompt_tokens/completion_tokens/total_tokens were all 0: the generic
pass-through success handler hands the spend writer a
StandardPassThroughResponseObject with no usage, and the generic branch
builds no standard_logging_object usage, so anything summing tokens for
the route under-reported.
patch_passthrough_token_cost.py now also sets combined_usage_object (a
litellm.Usage carrying prompt_tokens=input_tokens, completion_tokens=0,
total_tokens=input_tokens) on the logging object -- the same override the
streaming handler uses for recovered partial usage, which the spend
writer reads before the response object. Spend is unchanged.
Defect 2 (a second zero-spend row with call_type=/v1/systemone) is NOT
duplication: it is upstream LiteLLM's failure row for a rejected
(missing/invalid key) request, written by
ProxyLogging._handle_logging_proxy_only_error on the pristine image. One
authenticated call produces exactly one priced row. Documented in the
README; no code change.
The /v1/systemone pass-through is an EndpointType.GENERIC route: LiteLLM
logs it, but with no model on the route the only cost it can apply is the
flat cost_per_request (a budget estimate, not an invoice). So decisions
were billed at a fixed 0.0001 instead of the real $0.12 per 1M input
tokens GreenPT reports back in usage.input_tokens.
Price the route from its own response body via a new build-time patch,
patch_passthrough_token_cost.py, in the house style of the existing
patch_*.py files: any route listed in
general_settings.coop_passthrough_cost_per_token (USD per input token,
keyed by path) is billed as usage.input_tokens x rate. cost_per_request
stays as the fallback for a listed route whose response carries no token
count, and every unlisted route is unchanged. Auth is untouched.
config.yaml: list /v1/systemone at 0.00000012 ($0.12 per 1M input
tokens); update the metering note.
Deploying the audio-pricing change proved the gap: the image built and the app
restarted, but /model/info still served the old prices — /app/data is a
persistent volume, so the Dockerfile's COPY config.yaml /app/data/config.yaml is
shadowed by the volume's file from the first install, and start.sh only wrote a
default when the file was missing. Config changes in this repo therefore never
reached a running gateway; the audit's byte-identical md5 check was a manual
sync.
Fix: ship a second copy at /app/code/config.yaml and have start.sh sync it over
the volume at boot whenever they differ. The repo stays the source of truth, the
legacy first-boot default remains, and the README documents the flow.
Tinfoil bills voxtral-tts $0.002 and whisper-large-v3-turbo $0.01 per REQUEST,
with $0 per token for both. Reading only the two token prices in its model list
made TTS look free and left STT billed at an older catalog's $0.05/1M tokens, so
neither matched what the co-op actually pays.
LiteLLM cannot express a per-request charge on the audio paths: per-request
pricing exists only on the search/rerank/MCP paths via input_cost_per_query, and
input_cost_per_request is read only by the proxy's cost-tracking settings, never
by the model cost calculator.
So patch_audio_request_cost.py, a build-time patch in the same shape as
patch_image_cost.py, makes response_cost_calculator honour a declared
input_cost_per_request. That function is the single funnel for every call type —
the proxy dispatches audio as `aspeech` and `transcription` and reaches it
through Logging._response_cost_calculator — so one insertion covers both. The
charge resolves from the per-deployment custom_pricing the logging layer builds
from our model_info, then the shared cost map, then call metadata, and the patch
is inert for any model that declares nothing.
config.yaml declares the real charges: $0.002/request for voxtral-tts, and
$0.01/request for whisper with its token price corrected to $0.
Verified before committing: the patch applies to the installed 1.103.2 anchors
and compiles; the helper's resolution order passes unit checks (all branches,
plus inert and garbage cases); the resulting config validates as YAML with all
12 entries intact. End-to-end spend is proven after deploy by a real TTS/STT call.
Refs code/litellm#5
The 2026-10-02 outage had three causes the runbook now prevents:
- floating tag drift (README now mandates exact-tag pinning)
- 256MB default cgroup cap OOM-killing the prisma migration engine
(manifest now sets memoryLimit: 2147483648)
- no _prisma_migrations ledger -> P3005 crash-loop on boot
(start.sh now exports LITELLM_MIGRATION_DIR=/app/data/migrations)
README 'Updating LiteLLM' section rewritten from the old (wrong)
floating-tag procedure into a runbook with rules + verify checklist.
inference-bot
Ubuntu