Abort in-flight requests when the chat is cancelled #1

Merged
evanp merged 1 commits from code/inference-coop-copilot:fix/abort-on-cancel into main 2026-11-07 04:07:20 +00:00
Contributor

Hi Evan — this came out of the co-op's member-contribution review of the extension (the once-over projects get before they're featured; nothing serious turned up, and this was the one functional suggestion). I've written it up as a patch rather than just a note since it's easy to evaluate as a diff — please feel free to take it, adapt it, or set it aside; it's your project and any of those is fine.

The bug: cancellation is only checked at POST time (the old signal: line). Once the SSE stream is open, the reader loop never looks at the token again — so pressing Stop in Copilot Chat leaves the request generating server-side until the model finishes, billed to the member the whole way.

The fix (all in src/extension.ts):

  • An AbortController per request, aborted via token.onCancellationRequested, with its signal passed to fetch — this aborts the request whether cancellation lands before the headers arrive or mid-stream.
  • A token.isCancellationRequested check at the top of the reader loop, so the connection is closed as soon as a stop is noticed between chunks.
  • AbortError is swallowed (a stop isn't a failure — the chat UI just ends the turn); real HTTP errors still throw exactly as before, and the cancellation listener is disposed in a finally.

Verification:

  • npm ci, tsc --noEmit (strict), eslint src all clean; esbuild --production bundle contains the new logic; vsce package succeeds (with --baseContentUrl set for the Gitea URLs).
  • Behavioral test of the actual built bundle against a local SSE server over real sockets (fetch redirected to 127.0.0.1), with cancellation triggered after the first text chunk arrives:
    • before the fix: the server still delivered all 30/30 frames after cancel (~790 ms, same as an unccancelled run), and cancel-before-POST surfaced a raw AbortError to the caller.
    • after the fix: the socket closed at 1/30 frames and the response returned in 10 ms; cancel-before-POST sends nothing; cancel-during-headers aborts quietly; HTTP 500 still throws; no unhandled rejections. 7/7 checks pass.
  • Honest caveat: I couldn't run a live end-to-end pass through the real gateway — that needs a VS Code extension host and a gateway key, which I didn't have available. So the client-side behavior (connection closes on cancel) is verified above, but whether the gateway itself stops generating on client disconnect is inferred, not measured.

One commit, one file: fix/abort-on-cancel in the code/ fork. Happy to adjust anything — error handling, naming, or splitting it differently.

Hi Evan — this came out of the co-op's member-contribution review of the extension (the once-over projects get before they're featured; nothing serious turned up, and this was the one functional suggestion). I've written it up as a patch rather than just a note since it's easy to evaluate as a diff — please feel free to take it, adapt it, or set it aside; it's your project and any of those is fine. **The bug:** cancellation is only checked at POST time (the old `signal:` line). Once the SSE stream is open, the reader loop never looks at the token again — so pressing Stop in Copilot Chat leaves the request generating server-side until the model finishes, billed to the member the whole way. **The fix** (all in `src/extension.ts`): - An `AbortController` per request, aborted via `token.onCancellationRequested`, with its signal passed to `fetch` — this aborts the request whether cancellation lands before the headers arrive or mid-stream. - A `token.isCancellationRequested` check at the top of the reader loop, so the connection is closed as soon as a stop is noticed between chunks. - `AbortError` is swallowed (a stop isn't a failure — the chat UI just ends the turn); real HTTP errors still throw exactly as before, and the cancellation listener is disposed in a `finally`. **Verification:** - `npm ci`, `tsc --noEmit` (strict), `eslint src` all clean; `esbuild --production` bundle contains the new logic; `vsce package` succeeds (with `--baseContentUrl` set for the Gitea URLs). - Behavioral test of the actual built bundle against a local SSE server over real sockets (fetch redirected to 127.0.0.1), with cancellation triggered after the first text chunk arrives: - before the fix: the server still delivered all 30/30 frames after cancel (~790 ms, same as an unccancelled run), and cancel-before-POST surfaced a raw AbortError to the caller. - after the fix: the socket closed at 1/30 frames and the response returned in 10 ms; cancel-before-POST sends nothing; cancel-during-headers aborts quietly; HTTP 500 still throws; no unhandled rejections. 7/7 checks pass. - Honest caveat: I couldn't run a live end-to-end pass through the real gateway — that needs a VS Code extension host and a gateway key, which I didn't have available. So the client-side behavior (connection closes on cancel) is verified above, but whether the gateway itself stops generating on client disconnect is inferred, not measured. One commit, one file: `fix/abort-on-cancel` in the `code/` fork. Happy to adjust anything — error handling, naming, or splitting it differently.
inference-bot added 1 commit 2026-11-07 00:40:49 +00:00
Cancellation was only checked at POST time; once the SSE stream was
open the reader loop never looked at the token again, so a stopped
chat kept generating server-side (and billing) until the response
ended.

Wire an AbortController to token.onCancellationRequested, pass its
signal to fetch, and check token.isCancellationRequested at the top
of the reader loop so the connection closes as soon as the user
stops the request. AbortError is swallowed (a stop is not a
failure); real HTTP errors still surface, and the cancellation
listener is disposed in a finally.
Author
Contributor

Thanks! Moving this review/PR to the co-op fork for feedback — closing here.

Thanks! Moving this review/PR to the co-op fork for feedback — closing here.
inference-bot closed this pull request 2026-11-07 00:51:23 +00:00
evanp left a comment
Owner

Looks great! Thanks for adding this.

Looks great! Thanks for adding this.
evanp reopened this pull request 2026-11-07 04:07:03 +00:00
evanp merged commit 4fdaa0ca9d into main 2026-11-07 04:07:20 +00:00
Sign in to join this conversation.
No Reviewers
No labels
2 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: evanp/inference-coop-copilot#1