Files
inference-docs-mcp/README.md
T

1.2 KiB

inference-docs-mcp

The Inference Cooperative documentation as an MCP server — members opt in per conversation and can search the co-op docs (getting started, models, API access, charter, policies) from the chat.

Design

  • MCP over stdio, implemented with Python stdlib only (newline-delimited JSON-RPC 2.0) — no dependencies to install.
  • The corpus is fetched at runtime from the public co-op/docs repo (Gitea raw API), chunked by ## heading, and ranked with BM25. The index rebuilds lazily when older than 30 minutes, so doc edits show up without any redeploy.
  • Runs as a child process of the chat frontend (LibreChat spawns it via its mcpServers config with type: stdio).

Deploy (LibreChat sandbox)

cloudron push --app <chat-app-id> server.py /app/data/inference-docs-mcp/server.py
# librechat.yaml:
#   mcpServers:
#     inference-docs:
#       type: stdio
#       command: python3
#       args: ["/app/data/inference-docs-mcp/server.py"]

Restart the chat app; the tool (search_docs) registers on boot.

Tool

search_docs(query, limit?) → the most relevant doc sections with their doc title + heading context.