The per-request helper reads litellm.model_cost, which is built from model_prices_and_context_window.json — not from model_info. Router-side patching therefore never priced voxtral-tts speech, and every call logged $0. This build step writes the prices where the calculator looks, and fails the build if the file is missing instead of warning.
132 lines
7.3 KiB
Docker
132 lines
7.3 KiB
Docker
FROM ghcr.io/berriai/litellm:v1.108.3
|
|
|
|
# v1.84.0 is the fix line for CVE-2026-35029 (auth bypass on
|
|
# /config/update, fixed 1.83.0) and CVE-2026-59822 (MCP session auth
|
|
# bypass, fixed 1.84.0, CISA KEV catalog). Pin the exact tag — do NOT
|
|
# use a floating tag like main-v1.74.0-stable, which silently moved.
|
|
|
|
# LiteLLM's official image runs as root. Cloudron's sandboxing (read-only
|
|
# rootfs, AppArmor, dropped capabilities) still applies to root inside the
|
|
# container, so we keep root to match the base image and avoid /app/data
|
|
# volume-ownership issues.
|
|
|
|
# Create the data directory (Cloudron mounts the localstorage volume here)
|
|
RUN mkdir -p /app/data /app/code
|
|
|
|
# Create a startup script that configures LiteLLM with Cloudron addons
|
|
COPY start.sh /app/code/start.sh
|
|
RUN chmod +x /app/code/start.sh
|
|
|
|
# Default config — will be overridden by Cloudron env vars at runtime
|
|
COPY config.yaml /app/data/config.yaml
|
|
# A second copy inside the image: start.sh syncs it over the persistent
|
|
# /app/data volume at boot, so config changes in the repo actually deploy.
|
|
COPY config.yaml /app/code/config.yaml
|
|
|
|
# Fail-closed config validator (canonical copy lives in ops-docs/scripts/;
|
|
# this byte-identical twin ships with the image and is called from start.sh
|
|
# before the gateway boots). Mirrors validate_config.py's md5 — keep in step.
|
|
COPY validate_config.py /app/code/validate_config.py
|
|
|
|
# Append the co-op credit-pack hint to LiteLLM's hardcoded budget-exceeded
|
|
# messages. Done at BUILD time — the runtime rootfs is read-only under
|
|
# Cloudron, so a boot-time patch would crash the startup script (it did,
|
|
# 2026-10-06). patch_budget_message.py is non-fatal: it exits 0 even when the
|
|
# file/message can't be found, so a future LiteLLM version bump can't take the
|
|
# gateway down.
|
|
COPY patch_budget_message.py /app/code/patch_budget_message.py
|
|
RUN python3 /app/code/patch_budget_message.py
|
|
|
|
# Image-cost tracking: keep image-generation pricing on the shared cost-map
|
|
# key so successful image generations appear in SpendLogs (upstream bug
|
|
# BerriAI/litellm#10023/#12471 — closed not-planned). Non-fatal at build.
|
|
COPY patch_image_cost.py /app/code/patch_image_cost.py
|
|
RUN python3 /app/code/patch_image_cost.py
|
|
|
|
# Audio per-request pricing: Tinfoil bills voxtral-tts and whisper per request,
|
|
# which LiteLLM cannot express on the audio paths, so those two were billed at
|
|
# $0. The patch makes the cost calculator honour a declared
|
|
# input_cost_per_request. Check for the "[coop-patch] applied" line in the build
|
|
# log after upgrading LiteLLM: the script is non-fatal and warns instead.
|
|
COPY patch_audio_request_cost.py /app/code/patch_audio_request_cost.py
|
|
RUN python3 /app/code/patch_audio_request_cost.py
|
|
|
|
# The audio cost paths read pricing from the SHARED cost-map key, which the
|
|
# router strips per-deployment pricing from — so the lookup raised, the spend
|
|
# write was skipped, and audio produced no SpendLogs row at all. This re-adds the
|
|
# pricing (and input_cost_per_request) for the audio modes. Same mechanism as
|
|
# patch_image_cost.py above.
|
|
COPY patch_audio_router_pricing.py /app/code/patch_audio_router_pricing.py
|
|
RUN python3 /app/code/patch_audio_router_pricing.py
|
|
|
|
# Audio per-request prices must live in the SHARED cost map, which is built from
|
|
# model_prices_and_context_window.json — the per-request helper reads that map, not
|
|
# model_info, so a router-only patch left every speech call at $0 (issue #5).
|
|
# This patch fails the build rather than warning.
|
|
COPY patch_audio_cost_map.py /app/code/patch_audio_cost_map.py
|
|
RUN python3 /app/code/patch_audio_cost_map.py
|
|
|
|
# SECURITY (2026-10-27, member-disclosure finding 4): LiteLLM forwards every
|
|
# provider response header to callers as `llm_provider-*`, which leaked the
|
|
# co-op's remaining provider balance (x-credits-remaining) to any member API key.
|
|
# Verify the "[coop-patch] applied: provider response headers" line in the build
|
|
# output on every version bump — the patch is non-fatal by design.
|
|
COPY patch_strip_provider_headers.py /app/code/patch_strip_provider_headers.py
|
|
RUN python3 /app/code/patch_strip_provider_headers.py
|
|
|
|
# Expose the per-request cost headers to cross-origin browser clients (co-op/support#39).
|
|
COPY patch_expose_cost_headers.py /app/code/patch_expose_cost_headers.py
|
|
RUN python3 /app/code/patch_expose_cost_headers.py
|
|
|
|
# Member access to config-declared pass-through routes: LiteLLM gates auth=true
|
|
# pass-through endpoints behind `allowed_passthrough_routes`, an Enterprise-only
|
|
# field that cannot be set on this community-edition proxy. This patch lets any
|
|
# authenticated key call routes the operator lists in
|
|
# general_settings.coop_passthrough_open_to_keys (unauthenticated access is
|
|
# unaffected — still rejected). Non-fatal; check the "[coop-patch] applied" line
|
|
# in the build log on every LiteLLM upgrade.
|
|
COPY patch_passthrough_key_access.py /app/code/patch_passthrough_key_access.py
|
|
RUN python3 /app/code/patch_passthrough_key_access.py
|
|
|
|
# Exact per-input-token pricing for generic pass-through routes. A route whose
|
|
# target matches none of vertex/anthropic/openai/gemini resolves to
|
|
# EndpointType.GENERIC: it IS logged, but LiteLLM has no model to price it from,
|
|
# so the only cost it knows is the flat cost_per_request. Metis (/v1/systemone)
|
|
# is billed at $0.12 per 1M input tokens and returns usage.input_tokens, so this
|
|
# patch prices a route listed in general_settings.coop_passthrough_cost_per_token
|
|
# from that count and writes the exact figure to member spend. Non-fatal; check
|
|
# the "[coop-patch] applied" line in the build log on every LiteLLM upgrade.
|
|
COPY patch_passthrough_token_cost.py /app/code/patch_passthrough_token_cost.py
|
|
RUN python3 /app/code/patch_passthrough_token_cost.py
|
|
|
|
# Audio keep-alive: the in-container loop in start.sh pings tinfoil/voxtral-tts
|
|
# every 15 min so voice mode skips the ~24s cold-start (see audio_keepalive.py).
|
|
COPY audio_keepalive.py /app/code/audio_keepalive.py
|
|
RUN chmod +x /app/code/audio_keepalive.py
|
|
|
|
# Skip LiteLLM's forced verbose_json on transcription requests for opting-out
|
|
# models: Tinfoil's voxtral-small-24b rejects verbose_json with a 400, which
|
|
# broke every bare /v1/audio/transcriptions call to it (file + model — the
|
|
# shape members and dictation send). Its billing does not need the duration
|
|
# the injection exists for. Non-fatal; check the "[coop-patch] applied" line
|
|
# in the build log on every LiteLLM upgrade.
|
|
COPY patch_transcription_response_format.py /app/code/patch_transcription_response_format.py
|
|
RUN python3 /app/code/patch_transcription_response_format.py
|
|
|
|
# TTS default response format: LibreChat sends no response_format on its TTS
|
|
# request and Tinfoil's voxtral-tts then serves WAV (~8-17s warm measured) —
|
|
# while LibreChat labels the reply audio/mpeg regardless. Default to mp3
|
|
# (OpenAI's documented default, ~3s warm measured). Non-fatal; check the
|
|
# "[coop-patch] applied" line in the build log on every LiteLLM upgrade.
|
|
COPY patch_audio_default_mp3.py /app/code/patch_audio_default_mp3.py
|
|
RUN python3 /app/code/patch_audio_default_mp3.py
|
|
|
|
# Override the base image's ENTRYPOINT (which is `litellm`) so our
|
|
# startup script runs instead of being passed as an argument to litellm.
|
|
ENTRYPOINT ["/app/code/start.sh"]
|
|
|
|
# No CORS patch needed since v1.84.0: upstream replaced the hardcoded
|
|
# `origins = ["*"]` with a native LITELLM_CORS_ORIGINS env var (start.sh
|
|
# exports it). The old sed patch targeted a v1.74.0 file layout that no
|
|
# longer exists — the v1.84.0 rebuild self-skipped it correctly.
|