Abort in-flight chat requests on cancellation #1

Closed
inference-bot wants to merge 0 commits from fix/abort-on-cancel into main
pull from: fix/abort-on-cancel
Owner

Implements the MEDIUM finding from the 2026-11-06 member-contribution review of the upstream repo: cancelled chat requests were not aborted mid-stream — the AbortSignal was only set if the token was already cancelled at POST time (old code used AbortSignal.abort() there), and the reader loop never checked the token, so a cancelled chat kept generating server-side to end of response.

Fix (src/extension.ts only):

  • One AbortController per request, wired to token.onCancellationRequested -> abort() fed to fetch as signal.
  • Cancellation check at the top of the SSE reader loop: closes the connection and returns promptly instead of draining to end of response.
  • Pre-existing-cancel-at-POST-time sends nothing.
  • AbortError is swallowed (it is the cancellation); real errors still surface. The onCancellationRequested listener is disposed in finally.

Verification (on the built bundle):

  • npm ci, tsc --noEmit (strict) and eslint src clean; esbuild --production bundle contains the logic; vsce package OK with --baseContentUrl for Gitea URLs.
  • Behavioral test of the real built bundle against a local SSE server over real sockets (gateway URL redirected to 127.0.0.1, VS Code API stubbed), with the pre-fix bundle as a positive control:
    • fixed bundle 7/7: mid-stream cancel closes the socket at frame 1/30 and returns in ~12 ms (control delivered all 30/30 frames); cancel-before-POST sends nothing (control threw a raw AbortError); cancel during pending response headers aborts quietly; HTTP 500 still throws; no unhandled rejections.
    • control (pre-fix) bundle: 3/7 — reproduces the review's finding.
  • Live gateway test not run: minting a throwaway LiteLLM key requires infra access that is approval-gated, so "the gateway stops generating on client disconnect" is inferred from the closed socket, not measured. Spend: $0, no key minted.
Implements the MEDIUM finding from the 2026-11-06 member-contribution review of the upstream repo: cancelled chat requests were not aborted mid-stream — the `AbortSignal` was only set if the token was already cancelled at POST time (old code used `AbortSignal.abort()` there), and the reader loop never checked the token, so a cancelled chat kept generating server-side to end of response. **Fix** (`src/extension.ts` only): - One `AbortController` per request, wired to `token.onCancellationRequested` -> `abort()` fed to `fetch` as `signal`. - Cancellation check at the top of the SSE reader loop: closes the connection and returns promptly instead of draining to end of response. - Pre-existing-cancel-at-POST-time sends nothing. - `AbortError` is swallowed (it *is* the cancellation); real errors still surface. The `onCancellationRequested` listener is disposed in `finally`. **Verification** (on the built bundle): - `npm ci`, `tsc --noEmit` (strict) and `eslint src` clean; `esbuild --production` bundle contains the logic; `vsce package` OK with `--baseContentUrl` for Gitea URLs. - Behavioral test of the real built bundle against a local SSE server over real sockets (gateway URL redirected to 127.0.0.1, VS Code API stubbed), with the pre-fix bundle as a positive control: - fixed bundle 7/7: mid-stream cancel closes the socket at frame 1/30 and returns in ~12 ms (control delivered all 30/30 frames); cancel-before-POST sends nothing (control threw a raw AbortError); cancel during pending response headers aborts quietly; HTTP 500 still throws; no unhandled rejections. - control (pre-fix) bundle: 3/7 — reproduces the review's finding. - Live gateway test **not run**: minting a throwaway LiteLLM key requires infra access that is approval-gated, so "the gateway stops generating on client disconnect" is inferred from the closed socket, not measured. Spend: $0, no key minted.
evanp approved these changes 2026-11-07 04:10:04 +00:00
Dismissed
evanp left a comment
Collaborator

Looks good!

Looks good!
evanp approved these changes 2026-11-07 04:10:22 +00:00
evanp added 1 commit 2026-11-08 15:44:59 +00:00
Cancellation was only checked at POST time; once the SSE stream was
open the reader loop never looked at the token again, so a stopped
chat kept generating server-side (and billing) until the response
ended.

Wire an AbortController to token.onCancellationRequested, pass its
signal to fetch, and check token.isCancellationRequested at the top
of the reader loop so the connection closes as soon as the user
stops the request. AbortError is swallowed (a stop is not a
failure); real HTTP errors still surface, and the cancellation
listener is disposed in a finally.
evanp force-pushed fix/abort-on-cancel from 9798a41745 to 8f1d71d913 2026-11-08 15:44:59 +00:00 Compare
Collaborator

Merged!

Merged!
evanp closed this pull request 2026-11-08 15:50:07 +00:00

Pull request closed

Please reopen this pull request to perform a merge.
Sign in to join this conversation.
No Reviewers
No labels
2 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: code/inference-coop-copilot#1