8 Commits
Author SHA1 Message Date
evanp c4bb02a7a2 0.2.1 2026-11-06 13:39:18 -05:00
evanp f295cb8535 doc: update README 2026-11-06 13:39:14 -05:00
evanp 55e2764480 0.2.0 2026-11-06 13:33:24 -05:00
evanp cb23c502aa doc: CHANGELOG v0.2.0 2026-11-06 13:33:16 -05:00
evanp 4eeace6ce3 feat: store API key in SecretStorage, add set/sign-out commands 2026-11-06 13:26:34 -05:00
evanp 713a1c1d75 fix: cache model list and fire change event to avoid startup race 2026-11-06 13:13:46 -05:00
evanp 76da3ac01e chore: package version 0.1.0 2026-11-06 13:05:44 -05:00
evanp ce03d6d902 feat: tool handling 2026-11-06 12:59:34 -05:00
5 changed files with 375 additions and 106 deletions

No files matched your search

+52 -12
View File
@@ -1,30 +1,70 @@
# Changelog
## [0.0.1] - 2026-11-06
<!-- markdownlint-configure-file { "MD024": { "siblings_only": true } } -->
Initial development release. Not yet functional for chat.
All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
and this project adheres to
[Semantic Versioning](https://semver.org/spec/v2.0.0.html).
## [Unreleased]
## [0.2.1] - 2026-11-06
### Fixed
- Correct instructions for setting the API key in README.
## [0.2.0] - 2026-11-06
### Added
- "Inference Coop: Set API Key" and "Inference Coop: Sign Out" commands.
- Model list caching with `onDidChangeLanguageModelChatInformation`, so the
picker refreshes when models change instead of requiring a restart.
### Changed
- **Breaking:** the API key is now stored in VS Code `SecretStorage` instead
of the `inferenceCoop.apiKey` setting. Run "Inference Coop: Set API Key"
once after upgrading; the gateway base URL is now the only setting.
### Fixed
- Models intermittently missing from the Copilot Chat model picker when the
provider's initial model fetch lost a race with chat activation.
## [0.1.0] - 2026-11-06
First release usable for chat.
### Added
- Streaming chat responses via `provideLanguageModelChatResponse`, with
text streamed to the UI as it arrives.
- Tool calling: tool schemas are sent to the gateway, streamed tool-call
fragments are reassembled, and tool results are translated back into the
conversation. Verified end to end in agent mode against
`tinfoil/gpt-oss-120b`.
- Extension scaffold with esbuild build pipeline and ESLint/tsc checks.
- Registration of a `LanguageModelChatProvider` under the `inference-coop`
vendor, discoverable via `contributes.languageModelChatProviders`.
- Model discovery: fetches `GET https://gateway.inference.coop/v1/models`
and filters to chat-capable models, excluding image-generation and
audio endpoints.
- `inferenceCoop.apiKey` setting (plain text, temporarily).
- "Inference Coop" output channel logging provider activation, model
enumeration, and HTTP status codes.
### Known issues
### Changed
- Models appear in the Language Models panel but not in the Copilot Chat
model picker. Under investigation; suspected upstream VS Code issue.
- `provideLanguageModelChatResponse` sends a streaming request but has not
been verified end to end from the chat UI.
- No tool calling, image input, or token counting (returns 0).
- API key is stored in settings rather than `SecretStorage`.
- Adopted the Apache License 2.0.
### Not yet published
## [0.0.1] - 2026-11-06
- No Marketplace publisher registered; installed from a local `.vsix`.
Initial development release. Not functional for chat.
[Unreleased]: https://git.inference.coop/evanp/inference-coop-copilot/compare/v0.2.1...HEAD
[0.2.1]: https://git.inference.coop/evanp/inference-coop-copilot/compare/v0.2.0...v0.2.1
[0.2.0]: https://git.inference.coop/evanp/inference-coop-copilot/compare/v0.1.0...v0.2.0
[0.1.0]: https://git.inference.coop/evanp/inference-coop-copilot/releases/tag/v0.1.0
+28 -5
View File
@@ -1,14 +1,37 @@
# Inference Coop (unofficial)
# Inference Coop for Copilot Chat (unofficial)
Community-built model provider for [inference.coop](https://inference.coop),
usable in GitHub Copilot Chat. Not affiliated with or endorsed by the
Inference Cooperative.
## Requirements
- VS Code 1.145 or newer
- GitHub Copilot Chat, installed and signed in
- A co-op membership and API key
Copilot Business/Enterprise: your administrator must leave "Bring Your Own
Language Model Key" enabled, or these models will not appear.
## Setup
1. Set `inferenceCoop.apiKey` in your settings to your co-op API key.
2. Open Copilot Chat and pick an `inference-coop/…` model.
1. Run **Inference Coop: Set API Key** from the Command Palette and paste
your co-op key. It is stored in VS Code's secret storage, not in settings.
2. Open Copilot Chat, click the model name, and pick an `inference-coop/…`
model. Tool calling works, so agent mode is usable.
## Status
To use a different gateway, set `inferenceCoop.baseUrl`. To sign out, run
**Inference Coop: Sign Out**.
Early development. Text chat only so far; tool calling is not implemented.
## Limitations
- No image input; models are advertised as text-and-tools only.
- Token counts are estimated rather than computed with a real tokenizer, so
context budgeting is approximate.
- No inline code completions. Copilot does not support third-party providers
for completions; this extension only adds chat models.
- Unofficial client. Report extension bugs here, not to the co-op.
## License
Apache-2.0. See [LICENSE](LICENSE) and [NOTICE](NOTICE).
+2 -2
View File
@@ -1,12 +1,12 @@
{
"name": "inference-coop-copilot",
"version": "0.0.1",
"version": "0.2.1",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "inference-coop-copilot",
"version": "0.0.1",
"version": "0.2.1",
"devDependencies": {
"@types/mocha": "^10.0.10",
"@types/node": "24.x",
+20 -12
View File
@@ -2,7 +2,7 @@
"name": "inference-coop-copilot",
"displayName": "inference-coop-copilot",
"description": "Inference.coop models for Copilot Chat (unofficial)",
"version": "0.0.1",
"version": "0.2.1",
"engines": {
"vscode": "^1.145.0"
},
@@ -28,8 +28,26 @@
{
"command": "inference-coop-copilot.helloWorld",
"title": "Hello World"
},
{
"command": "inferenceCoop.setKey",
"title": "Inference Coop: Set API Key"
},
{
"command": "inferenceCoop.clearKey",
"title": "Inference Coop: Sign Out"
}
],
"configuration": {
"title": "Inference Coop",
"properties": {
"inferenceCoop.baseUrl": {
"type": "string",
"default": "https://gateway.inference.coop/v1",
"description": "Gateway base URL. Leave default unless testing."
}
}
}
]
},
"scripts": {
"vscode:prepublish": "npm run package",
@@ -60,15 +78,5 @@
"overrides": {
"diff": "^8.0.4",
"serialize-javascript": "^7.0.6"
},
"configuration": {
"title": "Inference Coop",
"properties": {
"inferenceCoop.apiKey": {
"type": "string",
"default": "",
"description": "API key from the Inference Cooperative dashboard."
}
}
}
}
+273 -75
View File
@@ -1,139 +1,337 @@
import * as vscode from 'vscode';
import * as vscode from "vscode";
const BASE_URL = 'https://gateway.inference.coop/v1';
const PREFIX = 'inference-coop/';
const BASE_URL = "https://gateway.inference.coop/v1";
const PREFIX = "inference-coop/";
const SECRET_KEY = "inferenceCoop.apiKey";
type CoopModel = {
id: string;
name: string;
family: string;
version: string;
maxInputTokens: number;
maxOutputTokens: number;
capabilities: { toolCalling: boolean; imageInput: boolean };
};
export function activate(context: vscode.ExtensionContext) {
const out = vscode.window.createOutputChannel('Inference Coop');
const out = vscode.window.createOutputChannel("Inference Coop");
context.subscriptions.push(out);
out.appendLine('activate() called');
const provider: vscode.LanguageModelChatProvider = {
async provideLanguageModelChatInformation() {
out.appendLine('provideLanguageModelChatInformation: called');
const changed = new vscode.EventEmitter<void>();
context.subscriptions.push(changed);
const key = vscode.workspace
.getConfiguration('inferenceCoop')
.get<string>('apiKey');
out.appendLine(` key present: ${Boolean(key)}`);
if (!key) {return [];}
let cached: CoopModel[] = [];
let lastFetch = 0;
const res = await fetch(`${BASE_URL}/models`, {
headers: { Authorization: `Bearer ${key}` }
});
out.appendLine(` /models status: ${res.status}`);
if (!res.ok) {
out.appendLine(` body: ${await res.text()}`);
return [];
async function refresh(force = false): Promise<CoopModel[]> {
if (!force && Date.now() - lastFetch < 30_000 && cached.length) {
return cached;
}
const key = await context.secrets.get(SECRET_KEY);
if (!key) {
return cached;
}
try {
const res = await fetch(`${BASE_URL}/models`, {
headers: { Authorization: `Bearer ${key}` },
});
out.appendLine(`refresh: /models ${res.status}`);
if (!res.ok) {
return cached;
}
const body = (await res.json()) as {
data: Array<{ id: string; mode?: string }>;
};
out.appendLine(` raw models: ${body.data.length}`);
const models = body.data
.filter(m => !m.mode || m.mode === 'chat')
.map(m => ({
const next: CoopModel[] = body.data
.filter((m) => !m.mode || m.mode === "chat")
.map((m) => ({
id: `${PREFIX}${m.id}`,
name: m.id,
family: 'inference-coop',
version: '1.0.0',
family: "inference-coop",
version: "1.0.0",
maxInputTokens: 131072,
maxOutputTokens: 8192,
capabilities: { toolCalling: true, imageInput: false }
capabilities: { toolCalling: true, imageInput: false },
}));
lastFetch = Date.now();
if (JSON.stringify(next) !== JSON.stringify(cached)) {
cached = next;
changed.fire();
out.appendLine(`refresh: ${cached.length} models, fired change event`);
}
return cached;
} catch (e) {
out.appendLine(`refresh failed: ${e}`);
return cached;
}
}
out.appendLine(` returning ${models.length} models`);
return models;
const provider: vscode.LanguageModelChatProvider = {
onDidChangeLanguageModelChatInformation: changed.event,
async provideLanguageModelChatInformation() {
out.appendLine("provideLanguageModelChatInformation: called");
if (cached.length) {
void refresh(); // refresh in background, answer now
return cached;
}
return await refresh(true); // first call: must wait
},
async provideLanguageModelChatResponse(model, messages, _options, progress, token) {
async provideLanguageModelChatResponse(
model,
messages,
options,
progress,
token,
) {
out.appendLine(`provideLanguageModelChatResponse: ${model.id}`);
out.appendLine(` tools offered: ${options.tools?.length ?? 0}`);
const key = vscode.workspace
.getConfiguration('inferenceCoop')
.get<string>('apiKey');
const key = await context.secrets.get(SECRET_KEY);
const body = {
model: model.id.replace(PREFIX, ''),
const body: Record<string, unknown> = {
model: model.id.replace(PREFIX, ""),
stream: true,
messages: messages.map(m => ({
role:
m.role === vscode.LanguageModelChatMessageRole.Assistant
? 'assistant'
: 'user',
content: m.content
.filter(
(p): p is vscode.LanguageModelTextPart =>
p instanceof vscode.LanguageModelTextPart
)
.map(p => p.value)
.join('\n')
}))
messages: toOpenAIMessages(messages, out),
};
if (options.tools?.length) {
body.tools = options.tools.map((t) => ({
type: "function",
function: {
name: t.name,
description: t.description,
parameters: t.inputSchema ?? { type: "object", properties: {} },
},
}));
body.tool_choice = "auto";
}
const res = await fetch(`${BASE_URL}/chat/completions`, {
method: 'POST',
method: "POST",
headers: {
Authorization: `Bearer ${key}`,
'Content-Type': 'application/json'
"Content-Type": "application/json",
},
body: JSON.stringify(body),
signal: token.isCancellationRequested ? AbortSignal.abort() : undefined
signal: token.isCancellationRequested ? AbortSignal.abort() : undefined,
});
out.appendLine(` /chat/completions status: ${res.status}`);
if (!res.ok || !res.body) {
throw new Error(`Inference Coop: HTTP ${res.status} — ${await res.text()}`);
throw new Error(
`Inference Coop: HTTP ${res.status} — ${await res.text()}`,
);
}
const reader = res.body.getReader();
const decoder = new TextDecoder();
let buf = '';
let chunks = 0;
let buf = "";
let textChunks = 0;
const pending = new Map<
number,
{ id?: string; name: string; args: string }
>();
const flushTools = () => {
for (const call of pending.values()) {
let input: object = {};
try {
input = JSON.parse(call.args || "{}");
} catch {
out.appendLine(
` WARN bad tool args for ${call.name}: ${call.args}`,
);
}
out.appendLine(` tool call: ${call.name} ${JSON.stringify(input)}`);
progress.report(
new vscode.LanguageModelToolCallPart(
call.id ?? `call_${Math.random().toString(36).slice(2)}`,
call.name,
input,
),
);
}
pending.clear();
};
while (true) {
const { done, value } = await reader.read();
if (done) {break;}
if (done) {
break;
}
buf += decoder.decode(value, { stream: true });
const frames = buf.split('\n\n');
buf = frames.pop() ?? '';
const frames = buf.split("\n\n");
buf = frames.pop() ?? "";
for (const frame of frames) {
for (const line of frame.split('\n')) {
if (!line.startsWith('data:')) {continue;}
for (const line of frame.split("\n")) {
if (!line.startsWith("data:")) {
continue;
}
const data = line.slice(5).trim();
if (data === '[DONE]') {
out.appendLine(` done, ${chunks} text chunks`);
if (data === "[DONE]") {
if (pending.size) {
flushTools();
}
out.appendLine(` done, ${textChunks} text chunks`);
return;
}
let json: any;
try {
const delta = JSON.parse(data).choices?.[0]?.delta?.content;
if (delta) {
chunks++;
progress.report(new vscode.LanguageModelTextPart(delta));
}
json = JSON.parse(data);
} catch {
/* partial frame, ignore */
continue;
}
const choice = json.choices?.[0];
const delta = choice?.delta;
if (!delta) {
continue;
}
if (typeof delta.content === "string" && delta.content) {
textChunks++;
progress.report(new vscode.LanguageModelTextPart(delta.content));
}
for (const tc of delta.tool_calls ?? []) {
const i = tc.index ?? 0;
const acc = pending.get(i) ?? {
id: undefined,
name: "",
args: "",
};
acc.id = tc.id ?? acc.id;
acc.name = tc.function?.name ?? acc.name;
acc.args += tc.function?.arguments ?? "";
pending.set(i, acc);
}
if (choice?.finish_reason) {
out.appendLine(` finish_reason: ${choice.finish_reason}`);
if (pending.size) {
flushTools();
}
}
}
}
out.appendLine(` stream ended, ${chunks} text chunks`);
}
if (pending.size) {
flushTools();
}
out.appendLine(` stream ended, ${textChunks} text chunks`);
},
async provideTokenCount() {
return 0;
}
},
};
context.subscriptions.push(
vscode.lm.registerLanguageModelChatProvider('inference-coop', provider)
vscode.lm.registerLanguageModelChatProvider("inference-coop", provider),
);
context.subscriptions.push(
vscode.commands.registerCommand('inference-coop-copilot.helloWorld', () => {
vscode.window.showInformationMessage('Inference Coop extension loaded.');
})
vscode.commands.registerCommand("inference-coop-copilot.helloWorld", () => {
vscode.window.showInformationMessage("Inference Coop extension loaded.");
}),
);
context.subscriptions.push(
vscode.commands.registerCommand("inferenceCoop.setKey", async () => {
const key = await vscode.window.showInputBox({
title: "Inference Coop API key",
prompt: "Paste your key from the co-op dashboard.",
password: true,
ignoreFocusOut: true,
validateInput: (v) =>
v.trim().length < 16
? "That looks too short to be an API key."
: undefined,
});
if (!key) {return;}
await context.secrets.store(SECRET_KEY, key.trim());
cached = [];
lastFetch = 0;
await refresh(true);
vscode.window.showInformationMessage(
`Inference Coop: signed in, ${cached.length} models available.`,
);
}),
);
context.subscriptions.push(
vscode.commands.registerCommand("inferenceCoop.clearKey", async () => {
await context.secrets.delete(SECRET_KEY);
cached = [];
lastFetch = 0;
changed.fire();
vscode.window.showInformationMessage("Inference Coop: signed out.");
}),
);
}
function toOpenAIMessages(
messages: readonly vscode.LanguageModelChatRequestMessage[],
out: vscode.OutputChannel,
): object[] {
const result: object[] = [];
for (const msg of messages) {
const text: string[] = [];
const toolCalls: object[] = [];
const toolResults: object[] = [];
for (const part of msg.content ?? []) {
if (part instanceof vscode.LanguageModelTextPart) {
text.push(part.value);
} else if (part instanceof vscode.LanguageModelToolCallPart) {
toolCalls.push({
id: part.callId,
type: "function",
function: {
name: part.name,
arguments: JSON.stringify(part.input ?? {}),
},
});
} else if (part instanceof vscode.LanguageModelToolResultPart) {
toolResults.push({
role: "tool",
tool_call_id: part.callId,
content: part.content
.map((c: unknown) =>
c instanceof vscode.LanguageModelTextPart
? c.value
: JSON.stringify(c),
)
.join("\n"),
});
}
}
const role =
msg.role === vscode.LanguageModelChatMessageRole.Assistant
? "assistant"
: msg.role === vscode.LanguageModelChatMessageRole.User
? "user"
: "system";
if (text.length || toolCalls.length) {
const m: Record<string, unknown> = {
role,
content: text.length ? text.join("\n") : null,
};
if (toolCalls.length) {
m.tool_calls = toolCalls;
}
result.push(m);
}
result.push(...toolResults);
}
out.appendLine(` translated ${result.length} messages`);
return result;
}
export function deactivate() {}