Add inference-image MCP server (accessible image generation) + proper favicon sizes

- mcp/image_server.py: generate_image tool via the co-op gateway; per-member
  key injected through customUserVars (IC_API_KEY) -> env; verified standalone
  (handshake, tools/list, real 1024x1024 + 1536x1024 images, error paths)
- librechat.yaml: register inference-image with startup:false (per-user
  credentials), 120s timeout
- assets/: real favicon sizes (16/32/192), opaque apple-touch 180, padded
  maskable 512
This commit is contained in:
Nathan Schneider committed 2026-10-18 20:52:48 -06:00
1 parent 5169915641
commit 5e214e4d7e
8 files changed
+249 -14

No files matched your search

+19 -11
View File
@@ -17,7 +17,7 @@ from this directory.
| App config | `/app/data/librechat.yaml` (volume) | yes | `librechat.yaml` |
| Env vars (env-file wins!) | `/app/data/env` (volume, sourced LAST by start.sh) | yes | changed lines documented below |
| Env vars (Cloudron-managed) | `cloudron env` (process env) | yes | documented below |
| MCP docs server | `/app/data/mcp/server.py` (volume) | yes | `mcp/server.py` |
| MCP servers | `/app/data/mcp/server.py` + `/app/data/mcp/image_server.py` (volume) | yes | `mcp/` |
| Brand assets (logo, favicons) | baked into image overlay | no — rebuilt per image | `assets/` |
| CloudronManifest.json | repo only (used by `cloudron update`) | n/a | byte-identical copy of the OFFICIAL appstore manifest |
@@ -106,13 +106,21 @@ Set via `cloudron env set` (process env):
keys, 8 chat modelSpecs (emoji convention shared with models.md).
`regolo/qwen-image` is deliberately NOT in the chat picker (image models
are not chat models; chat-calling it was the source of the old 500s).
- **Image generation**: LibreChat exposes image gen through an **Agent with
the "OpenAI Image Tools" tool**, not a chat-picker button. Members create
an agent, add the tool, and paste their co-op key on first use.
- **MCP**: `inference-docs` (stdio, `/app/data/mcp/server.py`, tool
`search_docs`) is admin-configured; members may add their own via the MCP
Settings panel (the "bring your own tools" pattern). Permissions seeded via
`interface.mcpServers` (use/create true, share false).
- **Image generation — two paths**:
1. **MCP tool (easiest)**: `inference-image` server (tool `generate_image`)
— members select it in the composer's **+** palette and just ask. No
agent setup. First use: save your co-op key in the MCP panel
(customUserVars `IC_API_KEY`, injected per-user into the server process
env; `startup: false` because the key only exists per-user after they
configure it). Bills to the member's own key (~1–2¢/image at 1024²).
2. **Agent tool**: LibreChat's built-in "OpenAI Image Tools" (env
`IMAGE_GEN_OAI_BASEURL`/`IMAGE_GEN_OAI_MODEL` → gateway; no admin key,
members paste their own). Note LibreChat v0.8.8 renders the generated
image only inside the agent conversation.
- **MCP**: `inference-docs` (tool `search_docs`, starts at boot) and
`inference-image` (tool `generate_image`, per-user startup). Members may
add their own via the MCP Settings panel ("bring your own tools").
Permissions seeded via `interface.mcpServers` (use/create true, share false).
- **Speech**: TTS `tinfoil/voxtral-tts` (voice `casual_female`), STT
`tinfoil/whisper-large-v3-turbo`, both via the gateway with the capped key.
- **Web search**: SearXNG (self-hosted) + keyless Keenable scraper, no reranker.
@@ -124,10 +132,10 @@ Set via `cloudron env set` (process env):
## Known state / limitations
- `patches/`, `logo.png`, `logo-coop.png`, `logo.svg` (repo root) are legacy
leftovers from the abandoned 2026-10-18 hand-built-image attempt — unused,
pending cleanup.
- The `[schedules]` Redis guard error is resolved by
`SCHEDULES_SINGLE_PROCESS=true` (above).
- The `RAG API not reachable` warning is expected (no rag_api addon); it only
affects LibreChat's own file-RAG, which we do not use.
- The `inference-image` MCP server logs `Tools: undefined` at boot — expected:
with `startup: false` it registers but only spawns per-user after a member
saves their key and reinitializes in the MCP panel.
Binary file not shown.

Before

Width:  |  Height:  |  Size: 6.6 KiB

After

Width:  |  Height:  |  Size: 12 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 6.6 KiB

After

Width:  |  Height:  |  Size: 719 B

Binary file not shown.

Before

Width:  |  Height:  |  Size: 6.6 KiB

After

Width:  |  Height:  |  Size: 1.7 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 6.6 KiB

After

Width:  |  Height:  |  Size: 15 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 6.6 KiB

After

Width:  |  Height:  |  Size: 30 KiB

+20 -3
View File
@@ -160,11 +160,28 @@ webSearch:
scraperProvider: "keenable"
rerankerType: "none"
# MCP servers (admin-configured). The docs server is the same stdlib-only
# Python server Open WebUI uses; it lives on the persistent volume at
# /app/data/mcp/server.py so it survives image updates.
# MCP servers (admin-configured, on the persistent volume so they survive
# image updates). Members opt in per conversation via the composer's +
# palette; each user gets their own isolated server connection.
mcpServers:
inference-docs:
command: /app/code/venv/bin/python3
args: ["/app/data/mcp/server.py"]
timeout: 60000
# Accessible image generation: one click in the + palette, no agent setup.
# Each member's own co-op key is injected per-user via customUserVars →
# env, so images bill to the member's own balance (chat-endpoint model).
inference-image:
command: /app/code/venv/bin/python3
args: ["/app/data/mcp/image_server.py"]
timeout: 120000
# Per-user credentials aren't available at boot; each member initializes
# their own connection after saving their key in the MCP panel.
startup: false
customUserVars:
IC_API_KEY:
title: "Inference Co-op API key"
description: "Your co-op API key — create one in the <a href='https://dashboard.inference.coop' target='_blank'>Member Dashboard</a>. Used for image generation; bills to your own balance."
sensitive: true
env:
IC_API_KEY: "{{IC_API_KEY}}"
+210
View File
@@ -0,0 +1,210 @@
#!/usr/bin/env python3
"""Inference Cooperative image-generation MCP server (stdio, stdlib-only).
Exposes a `generate_image` tool that calls the co-op gateway
(`regolo/qwen-image` via POST /v1/images/generations). Each member's own
co-op API key is injected per-user by LibreChat as the IC_API_KEY env var
(`customUserVars` + `env` in librechat.yaml), so usage bills to the member's
own LiteLLM team budget — the same model as the chat endpoint.
Design notes (mirrors mcp/server.py, the docs server):
- stdio transport: newline-delimited JSON-RPC 2.0; logging to stderr only.
- No third-party dependencies. The image is returned inline as MCP image
content (base64 PNG), which LibreChat renders directly in the chat.
"""
import base64
import json
import os
import sys
import urllib.error
import urllib.request
GATEWAY_IMAGES_URL = "https://gateway.inference.coop/v1/images/generations"
MODEL = "regolo/qwen-image"
REQUEST_TIMEOUT = 90
ALLOWED_SIZES = ("1024x1024", "1536x1024", "1024x1536")
DEFAULT_SIZE = "1024x1024"
DASHBOARD_URL = "https://dashboard.inference.coop"
SERVER_NAME = "inference-cooperative-image"
SERVER_VERSION = "1.0.0"
def log(msg):
sys.stderr.write(f"[inference-image-mcp] {msg}\n")
sys.stderr.flush()
def tool_generate_image(args):
key = (os.environ.get("IC_API_KEY") or "").strip()
if not key:
return {
"content": [{
"type": "text",
"text": (
"No Inference Co-op API key configured yet. Open the MCP panel "
f"(inference-image) and save your key — create one at {DASHBOARD_URL}. "
"Your key is only visible to you, and image generation bills to your "
"own co-op balance."
),
}],
"isError": True,
}
prompt = str(args.get("prompt") or "").strip()
if not prompt:
return {"content": [{"type": "text", "text": "Provide a prompt for the image."}]}
size = str(args.get("size") or DEFAULT_SIZE).strip()
if size not in ALLOWED_SIZES:
return {
"content": [{
"type": "text",
"text": f"Unsupported size '{size}'. Use one of: {', '.join(ALLOWED_SIZES)}.",
}],
"isError": True,
}
body = json.dumps({"model": MODEL, "prompt": prompt, "n": 1, "size": size}).encode()
req = urllib.request.Request(
GATEWAY_IMAGES_URL,
data=body,
headers={"Authorization": f"Bearer {key}", "Content-Type": "application/json"},
)
try:
with urllib.request.urlopen(req, timeout=REQUEST_TIMEOUT) as resp:
data = json.loads(resp.read().decode("utf-8"))
except urllib.error.HTTPError as exc:
detail = exc.read().decode("utf-8", "replace")[:400]
log(f"gateway HTTP {exc.code}: {detail}")
hint = "Check that your co-op API key is valid and has balance."
if exc.code == 401:
hint = f"Your key was rejected — re-save it in the MCP panel (create one at {DASHBOARD_URL})."
elif exc.code == 429:
hint = "Budget or rate limit reached on your key."
return {
"content": [{"type": "text", "text": f"Image generation failed (HTTP {exc.code}). {hint}"}],
"isError": True,
}
except Exception as exc: # noqa: BLE001 - report transport errors in-band
log(f"gateway error: {exc}")
return {
"content": [{"type": "text", "text": f"Image generation failed: {exc}"}],
"isError": True,
}
items = (data or {}).get("data") or []
if not items:
return {"content": [{"type": "text", "text": "The gateway returned no image."}], "isError": True}
item = items[0]
content = []
if item.get("b64_json"):
content.append({"type": "image", "data": item["b64_json"], "mimeType": "image/png"})
notes = [f"Generated with {MODEL} ({size}) — billed to your co-op key."]
if item.get("revised_prompt"):
notes.append(f"Prompt: {item['revised_prompt']}")
if not item.get("b64_json") and item.get("url"):
notes.append(f"Image URL: {item['url']}")
content.append({"type": "text", "text": " ".join(notes)})
return {"content": content}
TOOLS = [
{
"name": "generate_image",
"description": (
"Generate an image from a text prompt (regolo/qwen-image via the Inference "
"Co-op gateway, EU renewable-powered). Costs roughly 1-2 cents per image, "
"billed to your own co-op API key."
),
"inputSchema": {
"type": "object",
"properties": {
"prompt": {"type": "string", "description": "What to draw"},
"size": {
"type": "string",
"enum": list(ALLOWED_SIZES),
"description": "Image size (default 1024x1024)",
},
},
"required": ["prompt"],
},
}
]
TOOLS_HANDLERS = {"generate_image": tool_generate_image}
def handle(msg):
method = msg.get("method")
msg_id = msg.get("id")
if method == "initialize":
return {
"jsonrpc": "2.0",
"id": msg_id,
"result": {
"protocolVersion": "2024-11-05",
"capabilities": {"tools": {}},
"serverInfo": {"name": SERVER_NAME, "version": SERVER_VERSION},
},
}
if method == "ping":
return {"jsonrpc": "2.0", "id": msg_id, "result": {}}
if method == "tools/list":
return {"jsonrpc": "2.0", "id": msg_id, "result": {"tools": TOOLS}}
if method == "tools/call":
params = msg.get("params") or {}
name = params.get("name")
handler = TOOLS_HANDLERS.get(name)
if not handler:
return {
"jsonrpc": "2.0",
"id": msg_id,
"error": {"code": -32602, "message": f"Unknown tool: {name}"},
}
try:
result = handler(params.get("arguments") or {})
return {"jsonrpc": "2.0", "id": msg_id, "result": result}
except Exception as exc: # noqa: BLE001 - report tool errors in-band
log(f"tool error: {exc}")
return {
"jsonrpc": "2.0",
"id": msg_id,
"result": {"content": [{"type": "text", "text": f"Tool failed: {exc}"}], "isError": True},
}
if method and method.startswith("notifications/"):
return None
if msg_id is not None:
return {"jsonrpc": "2.0", "id": msg_id, "error": {"code": -32601, "message": "Method not found"}}
return None
def main():
log("starting (stdio)")
for line in sys.stdin:
line = line.strip()
if not line:
continue
try:
msg = json.loads(line)
except json.JSONDecodeError as exc:
log(f"bad json: {exc}")
continue
try:
resp = handle(msg)
except Exception as exc: # noqa: BLE001 - never crash the protocol loop
log(f"handler crash: {exc}")
if msg.get("id") is not None:
resp = {"jsonrpc": "2.0", "id": msg["id"], "error": {"code": -32603, "message": "Internal error"}}
else:
resp = None
if resp is not None:
sys.stdout.write(json.dumps(resp) + "\n")
sys.stdout.flush()
if __name__ == "__main__":
main()