Lars Grammel

fix: UI message stream end callbacks cannot distinguish failed responses from completed streams (#17578) ## Background UI stream consumers need an owner-declared lifecycle verdict because optional model finish metadata and individual error chunks cannot reliably distinguish completion, failure, abortion, and undeclared EOF. ## Root Cause UI message stream processing tracked abort chunks and model finish reasons but lacked an owner-level terminal verdict, while fatal source, execution, merge, metadata, conversion, ID-injection, and UI-state processing errors were not consistently promoted to that verdict. Reproduction and focused failure probes showed failed streams reporting unknown or completed. ## Summary Added and exported UIMessageStreamOutcome, included it in end callbacks, exposed composer declarations through UIMessageStreamWriterWithOutcome.setOutcome while preserving UIMessageStreamWriter compatibility, and tracked generation completion, aborts, execution, merge, source, metadata, conversion, ID-injection, and UI-state processing failures. Updated API documentation and the patch changeset. ## Testing Regression coverage now includes completed, failed, aborted, and unknown outcomes; explicit declarations; ambiguous error chunks; execution, merge, source, metadata, conversion, ID-injection, and reducer failures; terminal precedence; exactly-once callbacks; and writer compatibility. The focused Node and Edge suites passed 55 tests each, with full type checking, repository checks, and package build also passing. ## End-to-end Validation - Original reproduction replay — exited successfully without the original bug signal. - `pnpm -C packages/ai exec tsx -e '<immutable start-chunk probe>'` — undeclared and previously completed injection failures rejected and reported `failed` exactly once with the propagated error. ## Related Issues Fixes #17500 Closes #17518 Co-authored-by: bryan-hunter <26581405+bryan-hunter@users.noreply.github.com> --------- Co-authored-by: ai-sdk-factory <308175966+ai-sdk-factory@users.noreply.github.com> Co-authored-by: bryan-hunter <26581405+bryan-hunter@users.noreply.github.com> Co-authored-by: Gregor Martynus <39992+gr2m@users.noreply.github.com> Co-authored-by: ai-sdk-factory[bot] <305873210+ai-sdk-factory[bot]@users.noreply.github.com>

feat: add a per-step first-content timeout for streaming generations (#17561) ## Background Streaming transports can remain active through headers, metadata, keep-alives, empty deltas, or raw bytes without producing model content. Consumers need an SDK-level semantic deadline for the first content-bearing output of every streaming model-call step. ## Summary Adds optional timeout.firstChunkMs configuration and extraction. streamText arms a dedicated timeout for every model-call response stream, merges its abort signal into the provider call, and disarms it before forwarding the first non-empty text, reasoning, or tool-input delta, generated file, reasoning file, or tool call. Non-content activity does not affect firstChunkMs, and chunkMs now starts and resets only on semantic output. Step timers and abort listeners are cleaned up on completion, abort, setup failure, provider stream error, and cancellation, including through stitchable-stream lifecycle callbacks. ## Testing Adds Node and Edge runtime coverage for non-content activity, empty deltas, every qualifying output category, timeout identity, pre-forward disarming, multi-step re-arming, semantic chunkMs behavior, provider errors, cancellation, and timer cleanup. Also adds stitchable-stream lifecycle tests, timeout extraction tests, and compile-time public API coverage. ## End-to-end Validation - Ran the provider-independent firstChunkMs streamText example with delayed initial content; it completed successfully and emitted `The first content arrived before the deadline.` ## Documentation Updates timeout settings, the streamText reference, and ToolLoopAgent documentation with firstChunkMs semantics, qualifying output, ignored non-content activity, and content-based chunkMs behavior. ToolLoopAgent now explicitly identifies firstChunkMs and chunkMs as streaming-only. Adds a runnable mock-provider example and an ai patch changeset. ## Related Issues Fixes #17315 Co-authored-by: lgrammel <205036+lgrammel@users.noreply.github.com>

feat: add OpenAI Responses API computer tool support (#17290) ## Background Applications need typed access to OpenAI's GA computer-use tool so models can request UI actions and receive updated screenshots in multi-step workflows. ## Summary Added openai.tools.computer(), typed batched actions and safety checks, tool-choice serialization, streaming and non-streaming decoding, and computer_call_output round-tripping for image URLs and file IDs. ## Testing Added type and runtime tests covering every action variant, tool preparation and selection, streaming and non-streaming calls, safety checks, data URL and file ID screenshots, stored and stateless requests, and previousResponseId flows. ## End-to-end Validation - Ran the updated ai-functions example against live OpenAI GPT-5.4; it requested a screenshot action, received the screenshot output, and completed with an accurate screen description. ### Documentation Documented the computer tool API, action and safety-check types, execution loop, screenshot formats, persistence behavior, approvals, and security guidance in the OpenAI provider documentation. ## Related Issues Fixes #13730 Co-authored-by: realglyph123 <124026881+realglyph123@users.noreply.github.com>

feat: add persistent accessible light and dark theme selection to the DevTools viewer (#17576) ## Background The DevTools viewer previously provided only a hardcoded dark palette, preventing users from selecting and retaining a readable light appearance. ## Summary Adds an accessible light/dark theme toggle, dark-default initialization, localStorage persistence, document theme state, coordinated semantic palettes, contrast-compliant status colors, focus indicators, and timeline metadata. No exported SDK, telemetry, server, provider, or captured-run API changes. ## Testing Adds six theme unit tests covering defaults, stored preferences, storage failures, document application, switching, persistence, and toggle markup. Two production Chromium tests cover keyboard switching, reload persistence, generate and stream statuses, selected message/tool/error content, timeline metadata, 4.5:1 text contrast, and 3:1 focus-ring contrast. All 30 DevTools unit tests pass. ## End-to-end Validation - The Gateway DevTools theme example completed a live generation successfully. - `pnpm --filter @ai-sdk/devtools test:e2e` built the production viewer and passed both Chromium accessibility and persistence tests. ## Documentation Updates the DevTools guide and package README with theme selection, dark-default behavior, and browser persistence; includes a Gateway usage example and an `@ai-sdk/devtools` patch changeset. ## Related Issues Fixes #17568 Co-authored-by: lgrammel <205036+lgrammel@users.noreply.github.com>

fix: include parsed structured output in streamText end callbacks (#17717) ## Background The `streamText` structured-output replacement omitted the complete parsed output from `onEnd`/`onFinish`, preventing callback-based persistence workflows supported by `streamObject`. ## Root Cause `streamText` assembled end events exclusively from step results, while complete output parsing was only exposed through `result.output`. ## Summary - expose schema-inferred optional `output` in `onEnd`/`onFinish` - cache complete-output parsing so callbacks and `result.output` share one parse - keep `result.output` settlement independent from arbitrary callback completion - export `StreamTextEndEvent` and `StreamTextOnEndCallback` - keep telemetry events restricted to their existing shape - update docs, reproduction, focused tests, and patch changeset Callers that need to wait for asynchronous callback work can await `result.consumeStream()`. ## Testing - callback output and schema inference - reusable public callback/event types - single complete-output parse - parse failures return `undefined` to the callback while `result.output` rejects - `result.output` remains independent from active callbacks - callbacks can await `result.output` after asynchronous work - full `ai` Node and Edge suites - workspace type checking, linting, formatting, and package build - provider-independent reproduction ## Related Issues Fixes #11418 Closes #17701 Co-authored-by: pltoledo <57258959+pltoledo@users.noreply.github.com> --------- Co-authored-by: ai-sdk-factory <308175966+ai-sdk-factory@users.noreply.github.com> Co-authored-by: pltoledo <57258959+pltoledo@users.noreply.github.com> Co-authored-by: Gregor Martynus <39992+gr2m@users.noreply.github.com> Co-authored-by: ai-sdk-factory[bot] <305873210+ai-sdk-factory[bot]@users.noreply.github.com>

feat: add non-streaming Voxtral TTS to the Mistral provider (#17286) ## Background Users need first-class Mistral Voxtral text-to-speech support so they can generate audio with saved voices or one-off reference audio through generateSpeech. ## Summary Added a SpeechModelV4 implementation, speech and speechModel factories, public MistralSpeechModelId and MistralSpeechModelOptions types, supported output-format mapping, warnings, base64 response handling, and sensitive reference-audio redaction. ## Testing Added 20 focused speech-model tests covering factories, request mapping, voice cloning, formats, warnings, headers, custom configuration, abort signals, responses, errors, and reference-audio privacy in Node and Edge runtimes. ## End-to-end Validation - Ran the Mistral generateSpeech example against the live API and produced valid MP3 audio. - Live-validated one-off cloning using synthetic reference audio; generated valid MP3 audio without warnings. - Removed generated output artifacts after validation. ### Documentation Updated the Mistral provider documentation with speech factories, saved and preset voices, reference audio, formats, unsupported settings, privacy and consent guidance, and non-streaming constraints; also added Voxtral to the core speech model list. ## Related Issues Fixes #14245 Co-authored-by: sovetski <13520683+sovetski@users.noreply.github.com>

fix: prevent HTTP MCP mid-stream disconnects from crashing the process (#16608) ## Background Streamable HTTP MCP background SSE failures could escape as unhandled promise rejections after a mid-stream server disconnect, crashing Node processes instead of being contained by the transport error callback. ## Summary Updated HttpMCPTransport to observe background inbound SSE startup, reconnect, reader-loop, and close/cancel promise rejections and route non-abort errors through the existing onerror path. ## Testing Added regression coverage for an inbound SSE stream that errors and then rejects during reader cancellation; removed the temporary reproduction script and added a patch changeset. ## Related Issues Fixes #16541 Co-authored-by: jouve <1096799+jouve@users.noreply.github.com>

fix: Harden stream text processing and middleware against prototype pollution from stream part IDs (#16006) ## Background Provider stream part IDs come from upstream model responses and should not be trusted as plain object keys. `streamText` and the JSON/reasoning extraction middleware kept per-part state in `{}` objects, so a missing stream delta with an ID such as `__proto__` could resolve to `Object.prototype` and mutate shared prototype state while handling the chunk. ## Summary This PR hardens stream-part state tracking by using the existing null-prototype `createIdMap()` helper for provider-controlled IDs in: - `streamText` active text and reasoning content maps, including step resets - `extractJsonMiddleware` text block state - `extractReasoningMiddleware` reasoning extraction state It also adds regression tests for missing `__proto__` stream part IDs to verify that `Object.prototype` is not read or polluted, and includes a patch changeset for `ai`.

fix: transition client-denied tool approvals to output-denied (#17438) ## Background Client-denied tool approvals remained in `approval-responded`, causing persisted and rendered UI state to show a response instead of the terminal `output-denied` state. ## Root Cause `collectToolApprovals` discarded denied approvals whenever an existing tool result was present, including the synthetic `execution-denied` result produced by UI message conversion, preventing `streamText` from emitting `tool-output-denied`. ## Summary Denied approvals with existing `execution-denied` results are now collected for stream notification while retaining the existing result to prevent duplicate model-facing tool results. ## Testing Updated approval collection coverage and added generateText and streamText regressions verifying denial emission and execution-denied result deduplication. ## End-to-end Validation - `pnpm -C examples/ai-functions exec tsx src/reproduction/issue-17136-denied-tool-approval.ts` after rebuilding `ai` — emitted `tool-output-denied`, transitioned live and persisted state to `output-denied`, preserved the model-facing denial, and kept the approved control at `output-available`. ## Related Issues Fixes #17136 Closes #17427 Co-authored-by: Shanik1 <64074694+Shanik1@users.noreply.github.com>

fix: preserve Anthropic web search citations across multi-turn assistant message replay (#17398) ## Background Multi-turn Anthropic web-search conversations replayed previously cited assistant text without citations or encrypted indices, losing citation context. ## Root Cause Anthropic citation deltas were emitted only as source parts, which response history excludes, while assistant text serialization ignored citation metadata; this was confirmed by the failing replay reproduction and the converter and response-history code paths. ## Summary Stored web_search_result_location data on generated text metadata and serialized it back into Anthropic assistant text citation arrays. ## Testing Added inline-snapshot converter coverage and updated Anthropic generate and stream snapshots to verify citation metadata preservation. ## End-to-end Validation - `cd examples/ai-functions && pnpm tsx src/reproduction/anthropic-web-search-citation-replay.ts` — live two-turn Anthropic request passed and preserved `web_search_result_location` citations. ## Related Issues Fixes #17379 Closes #17392 Co-authored-by: flovouin <5155149+flovouin@users.noreply.github.com>