Files
litellm/Dockerfile
T
inference-bot 943701581c gateway: make /v1/audio/transcriptions work bare for voxtral-small-24b
LiteLLM's openai transcription transformation force-injects
response_format=verbose_json when the caller omits it (to obtain a duration
for per-second pricing). Tinfoil rejects verbose_json for
voxtral-small-24b with a 400, so every bare transcription call failed —
including the file+model shape members and Open WebUI dictation send.

patch_transcription_response_format.py (new build-time patch, non-fatal):
skip the injection for deployments whose model_info declares
coop_no_verbose_json, with an explicit model-name gate as the reliable
mechanism. Its billing (per-token via chat, per-request via the audio
patch) does not need the duration. whisper-large-v3-turbo is untouched:
Tinfoil serves verbose_json for it and dictation depends on the injection.
2026-11-06 22:40:53 +00:00

112 lines
6.2 KiB
Docker

FROM ghcr.io/berriai/litellm:v1.103.2
# v1.84.0 is the fix line for CVE-2026-35029 (auth bypass on
# /config/update, fixed 1.83.0) and CVE-2026-59822 (MCP session auth
# bypass, fixed 1.84.0, CISA KEV catalog). Pin the exact tag — do NOT
# use a floating tag like main-v1.74.0-stable, which silently moved.
# LiteLLM's official image runs as root. Cloudron's sandboxing (read-only
# rootfs, AppArmor, dropped capabilities) still applies to root inside the
# container, so we keep root to match the base image and avoid /app/data
# volume-ownership issues.
# Create the data directory (Cloudron mounts the localstorage volume here)
RUN mkdir -p /app/data /app/code
# Create a startup script that configures LiteLLM with Cloudron addons
COPY start.sh /app/code/start.sh
RUN chmod +x /app/code/start.sh
# Default config — will be overridden by Cloudron env vars at runtime
COPY config.yaml /app/data/config.yaml
# A second copy inside the image: start.sh syncs it over the persistent
# /app/data volume at boot, so config changes in the repo actually deploy.
COPY config.yaml /app/code/config.yaml
# Append the co-op credit-pack hint to LiteLLM's hardcoded budget-exceeded
# messages. Done at BUILD time — the runtime rootfs is read-only under
# Cloudron, so a boot-time patch would crash the startup script (it did,
# 2026-10-06). patch_budget_message.py is non-fatal: it exits 0 even when the
# file/message can't be found, so a future LiteLLM version bump can't take the
# gateway down.
COPY patch_budget_message.py /app/code/patch_budget_message.py
RUN python3 /app/code/patch_budget_message.py
# Image-cost tracking: keep image-generation pricing on the shared cost-map
# key so successful image generations appear in SpendLogs (upstream bug
# BerriAI/litellm#10023/#12471 — closed not-planned). Non-fatal at build.
COPY patch_image_cost.py /app/code/patch_image_cost.py
RUN python3 /app/code/patch_image_cost.py
# Audio per-request pricing: Tinfoil bills voxtral-tts and whisper per request,
# which LiteLLM cannot express on the audio paths, so those two were billed at
# $0. The patch makes the cost calculator honour a declared
# input_cost_per_request. Check for the "[coop-patch] applied" line in the build
# log after upgrading LiteLLM: the script is non-fatal and warns instead.
COPY patch_audio_request_cost.py /app/code/patch_audio_request_cost.py
RUN python3 /app/code/patch_audio_request_cost.py
# The audio cost paths read pricing from the SHARED cost-map key, which the
# router strips per-deployment pricing from — so the lookup raised, the spend
# write was skipped, and audio produced no SpendLogs row at all. This re-adds the
# pricing (and input_cost_per_request) for the audio modes. Same mechanism as
# patch_image_cost.py above.
COPY patch_audio_router_pricing.py /app/code/patch_audio_router_pricing.py
RUN python3 /app/code/patch_audio_router_pricing.py
# SECURITY (2026-10-27, member-disclosure finding 4): LiteLLM forwards every
# provider response header to callers as `llm_provider-*`, which leaked the
# co-op's remaining provider balance (x-credits-remaining) to any member API key.
# Verify the "[coop-patch] applied: provider response headers" line in the build
# output on every version bump — the patch is non-fatal by design.
COPY patch_strip_provider_headers.py /app/code/patch_strip_provider_headers.py
RUN python3 /app/code/patch_strip_provider_headers.py
# Expose the per-request cost headers to cross-origin browser clients (co-op/support#39).
COPY patch_expose_cost_headers.py /app/code/patch_expose_cost_headers.py
RUN python3 /app/code/patch_expose_cost_headers.py
# Member access to config-declared pass-through routes: LiteLLM gates auth=true
# pass-through endpoints behind `allowed_passthrough_routes`, an Enterprise-only
# field that cannot be set on this community-edition proxy. This patch lets any
# authenticated key call routes the operator lists in
# general_settings.coop_passthrough_open_to_keys (unauthenticated access is
# unaffected — still rejected). Non-fatal; check the "[coop-patch] applied" line
# in the build log on every LiteLLM upgrade.
COPY patch_passthrough_key_access.py /app/code/patch_passthrough_key_access.py
RUN python3 /app/code/patch_passthrough_key_access.py
# Exact per-input-token pricing for generic pass-through routes. A route whose
# target matches none of vertex/anthropic/openai/gemini resolves to
# EndpointType.GENERIC: it IS logged, but LiteLLM has no model to price it from,
# so the only cost it knows is the flat cost_per_request. Metis (/v1/systemone)
# is billed at $0.12 per 1M input tokens and returns usage.input_tokens, so this
# patch prices a route listed in general_settings.coop_passthrough_cost_per_token
# from that count and writes the exact figure to member spend. Non-fatal; check
# the "[coop-patch] applied" line in the build log on every LiteLLM upgrade.
COPY patch_passthrough_token_cost.py /app/code/patch_passthrough_token_cost.py
RUN python3 /app/code/patch_passthrough_token_cost.py
# Audio keep-alive: scheduled pings keep the Tinfoil TEE audio enclave
# warm so voice mode skips the ~24s cold-start (see audio_keepalive.py header).
COPY audio_keepalive.py /app/code/audio_keepalive.py
RUN chmod +x /app/code/audio_keepalive.py
# Skip LiteLLM's forced verbose_json on transcription requests for opting-out
# models: Tinfoil's voxtral-small-24b rejects verbose_json with a 400, which
# broke every bare /v1/audio/transcriptions call to it (file + model — the
# shape members and dictation send). Its billing does not need the duration
# the injection exists for. Non-fatal; check the "[coop-patch] applied" line
# in the build log on every LiteLLM upgrade.
COPY patch_transcription_response_format.py /app/code/patch_transcription_response_format.py
RUN python3 /app/code/patch_transcription_response_format.py
# Override the base image's ENTRYPOINT (which is `litellm`) so our
# startup script runs instead of being passed as an argument to litellm.
ENTRYPOINT ["/app/code/start.sh"]
# No CORS patch needed since v1.84.0: upstream replaced the hardcoded
# `origins = ["*"]` with a native LITELLM_CORS_ORIGINS env var (start.sh
# exports it). The old sed patch targeted a v1.74.0 file layout that no
# longer exists — the v1.84.0 rebuild self-skipped it correctly.