inference-bot
8ddd023786
LiteLLM copies every upstream response header onto its own response prefixed
'llm_provider-'. For our providers that hands callers operational detail about the
co-op's own account — verified live: llm_provider-x-credits-remaining carried the
remaining provider balance in every member's response.
Build-time patch makes _get_llm_provider_headers() return {}. The four OpenAI-style
x-ratelimit-* headers a provider may send are still forwarded: those describe the
caller's own request, not the co-op's account.
78 lines
4.0 KiB
Docker
78 lines
4.0 KiB
Docker
FROM ghcr.io/berriai/litellm:v1.103.2
|
|
|
|
# v1.84.0 is the fix line for CVE-2026-35029 (auth bypass on
|
|
# /config/update, fixed 1.83.0) and CVE-2026-59822 (MCP session auth
|
|
# bypass, fixed 1.84.0, CISA KEV catalog). Pin the exact tag — do NOT
|
|
# use a floating tag like main-v1.74.0-stable, which silently moved.
|
|
|
|
# LiteLLM's official image runs as root. Cloudron's sandboxing (read-only
|
|
# rootfs, AppArmor, dropped capabilities) still applies to root inside the
|
|
# container, so we keep root to match the base image and avoid /app/data
|
|
# volume-ownership issues.
|
|
|
|
# Create the data directory (Cloudron mounts the localstorage volume here)
|
|
RUN mkdir -p /app/data /app/code
|
|
|
|
# Create a startup script that configures LiteLLM with Cloudron addons
|
|
COPY start.sh /app/code/start.sh
|
|
RUN chmod +x /app/code/start.sh
|
|
|
|
# Default config — will be overridden by Cloudron env vars at runtime
|
|
COPY config.yaml /app/data/config.yaml
|
|
# A second copy inside the image: start.sh syncs it over the persistent
|
|
# /app/data volume at boot, so config changes in the repo actually deploy.
|
|
COPY config.yaml /app/code/config.yaml
|
|
|
|
# Append the co-op credit-pack hint to LiteLLM's hardcoded budget-exceeded
|
|
# messages. Done at BUILD time — the runtime rootfs is read-only under
|
|
# Cloudron, so a boot-time patch would crash the startup script (it did,
|
|
# 2026-10-06). patch_budget_message.py is non-fatal: it exits 0 even when the
|
|
# file/message can't be found, so a future LiteLLM version bump can't take the
|
|
# gateway down.
|
|
COPY patch_budget_message.py /app/code/patch_budget_message.py
|
|
RUN python3 /app/code/patch_budget_message.py
|
|
|
|
# Image-cost tracking: keep image-generation pricing on the shared cost-map
|
|
# key so successful image generations appear in SpendLogs (upstream bug
|
|
# BerriAI/litellm#10023/#12471 — closed not-planned). Non-fatal at build.
|
|
COPY patch_image_cost.py /app/code/patch_image_cost.py
|
|
RUN python3 /app/code/patch_image_cost.py
|
|
|
|
# Audio per-request pricing: Tinfoil bills voxtral-tts and whisper per request,
|
|
# which LiteLLM cannot express on the audio paths, so those two were billed at
|
|
# $0. The patch makes the cost calculator honour a declared
|
|
# input_cost_per_request. Check for the "[coop-patch] applied" line in the build
|
|
# log after upgrading LiteLLM: the script is non-fatal and warns instead.
|
|
COPY patch_audio_request_cost.py /app/code/patch_audio_request_cost.py
|
|
RUN python3 /app/code/patch_audio_request_cost.py
|
|
|
|
# The audio cost paths read pricing from the SHARED cost-map key, which the
|
|
# router strips per-deployment pricing from — so the lookup raised, the spend
|
|
# write was skipped, and audio produced no SpendLogs row at all. This re-adds the
|
|
# pricing (and input_cost_per_request) for the audio modes. Same mechanism as
|
|
# patch_image_cost.py above.
|
|
COPY patch_audio_router_pricing.py /app/code/patch_audio_router_pricing.py
|
|
RUN python3 /app/code/patch_audio_router_pricing.py
|
|
|
|
# SECURITY (2026-10-27, member-disclosure finding 4): LiteLLM forwards every
|
|
# provider response header to callers as `llm_provider-*`, which leaked the
|
|
# co-op's remaining provider balance (x-credits-remaining) to any member API key.
|
|
# Verify the "[coop-patch] applied: provider response headers" line in the build
|
|
# output on every version bump — the patch is non-fatal by design.
|
|
COPY patch_strip_provider_headers.py /app/code/patch_strip_provider_headers.py
|
|
RUN python3 /app/code/patch_strip_provider_headers.py
|
|
|
|
# Audio keep-alive: scheduled pings keep the Tinfoil TEE audio enclave
|
|
# warm so voice mode skips the ~24s cold-start (see audio_keepalive.py header).
|
|
COPY audio_keepalive.py /app/code/audio_keepalive.py
|
|
RUN chmod +x /app/code/audio_keepalive.py
|
|
|
|
# Override the base image's ENTRYPOINT (which is `litellm`) so our
|
|
# startup script runs instead of being passed as an argument to litellm.
|
|
ENTRYPOINT ["/app/code/start.sh"]
|
|
|
|
# No CORS patch needed since v1.84.0: upstream replaced the hardcoded
|
|
# `origins = ["*"]` with a native LITELLM_CORS_ORIGINS env var (start.sh
|
|
# exports it). The old sed patch targeted a v1.74.0 file layout that no
|
|
# longer exists — the v1.84.0 rebuild self-skipped it correctly.
|