Bhavya U

Add E2E prompt snapshot tests for Copilot agent host (#328158) * Add E2E prompt snapshot tests for Copilot agent host Pins the prompt the bundled Copilot CLI assembles for each model: the system prompt, the tool definitions, and the turn messages with the context the CLI injects around them. Read off a replayed turn, so the tests are deterministic and need no token or network. Baselines keep as much real prompt text as possible. Only values that differ between two correct runs are elided - the session id, the clock, and the environment probe - each keeping its surrounding label so a change to the shape of those lines still fails. The repository instructions the CLI injects from .github/copilot-instructions.md and AGENTS.md are captured verbatim, so editing either file requires refreshing these baselines. POSIX-only: the Windows prompt carries PowerShell-specific sections rather than being a renaming of this one. See KNOWN_ISSUES.md. * Address review of the Copilot prompt snapshot tests Fail the turn on chat/error instead of treating it as completion, so a broken turn cannot be snapshotted as a good prompt. Require a committed baseline. assertSnapshot creates a missing one and passes, which would let a newly added model go green against a file nobody wrote or reviewed. Only accept a baseline from the replay-scoped update flag. The update-all flag also puts the harness in live-record mode, and the header comment already says a recording run must not be snapshotted because the live catalog and experiment assignment move the prompt. Skip the assertion entirely while recording rather than comparing a live prompt against a replay-derived baseline. Refresh the README, which still described the earlier one-model-per- dialect set and claimed the no-selection case was not pinned. * Make pinning a model opt-in by eliding the CLI's model catalog The CLI inlines the whole /models list into the Task tool's schema, as a count and a per-model listing, so adding one entry to capiStubs.ts rewrote all thirteen baselines - including those of models nobody snapshots. A new model release would have landed as a thirteen-file diff on an unrelated PR. Elide both, keeping their labels so the catalog changing shape or vanishing from the prompt still fails. Adding a model to the stub catalog is now inert; a model is pinned only once someone also adds it to SNAPSHOT_MODELS and commits its fixture and baseline. * Refuse to build a prompt baseline out of an empty request The system prompt and tools are read through dialect-specific field names, and a shape the formatter does not recognize yields empty rather than throwing. That is how the first version of these tests pinned a 12-character system prompt and no tools at all for a whole model family, green. Assert both are non-empty before formatting, so a renamed wire field fails loudly instead of silently rewriting the baseline to nothing. * Pin only explicitly selected models Sending no selection let the CLI rank the stub catalog itself, so the baseline recorded a property of this suite's fixture rather than the prompt, and would move whenever a higher-ranked model was added to or removed from the catalog. * Condense the prompt snapshot comments The same points were restated across the file header, the constants, and each function's doc block. Keep the reasoning that the code cannot show - why recording never produces a baseline, why an unrecognized wire shape must fail loudly, what is elided and why - and drop the rest. * Restart the shared host after a failed prompt snapshot test A failed test can leave a mid-turn session that wedges the shared Copilot process, cascading into the next model. Pass `failed` to `release` so it restarts, dump the runtime logs first, and move temp-directory cleanup into a `finally` so a throwing `release` cannot skip it - matching `agentHostE2ESuites.ts`. * Stabilize Copilot prompt snapshot tests Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Expand Copilot prompt snapshot coverage Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Cache explorer: conversation-level hit rate, exclude utility models, agent-type filter (#320902) Fixes #320765

Agent host: clean up prompt composition and extend model capability overrides (#325143) * Agent host: consolidate prompt composition and extend model capability overrides Cleanup + experimentation knobs for Copilot SDK agent-host sessions. Prompt composition cleanup: - Compose contributor section overrides over the default sections so a contributor only overrides what it names (default identity survives unless explicitly replaced); drop the duplicated identity re-statement in the Opus 4.8 override. - Apply the universal tool instructions, workspaceless scratch guidance, and file-link contract consistently for every prompt mode, including a full 'replace' prompt (appended after its content). Model capability overrides (experimental setting): - Add a '*' wildcard entry that merges field-by-field under a specific model-id entry via resolveModelCapabilityOverride(). - Extend chat.agentHost.modelCapabilityOverrides with per-model reasoningEffort, availableTools, and excludedTools; applied at session launch and validated defensively at each use site. Only affects Copilot SDK agent-host sessions; intended for experimentation. * Agent host: namespace the capability-overrides setting and gate prompts on filtered tools - Rename `chat.agentHost.modelCapabilityOverrides` to `chat.agentHost.copilot.modelCapabilityOverrides` (it only affects Copilot CLI sessions), along with the setting-id constant and nls keys. - `getToolFilterOverride` now strips a bare `'*'` pattern, which the SDK's `validateToolFilterList` throws on at session create, and treats an empty pattern list as unset. Failing open keeps a bad experiment value from breaking a session launch. - Add `filterClientToolNames` so the tool-specific prompt sections are gated on the post-filter tool set: the system message no longer advertises a client tool that `availableTools`/`excludedTools` disabled. Routing keeps the unfiltered set since the runtime is the enforcement point. * Agent host: apply capability overrides to the runtime process and resumed sessions The `family` alias only re-routed the agent host's own prompt registry, so an aliased preview model still got the runtime's default per-model config (its system-prompt parts, capabilities and reasoning-effort profile). The runtime resolves that from `service.agent.modelFamily`, which is process-scoped and reachable only through `COPILOT_MODEL_FAMILY` — there is no per-session SDK or protocol field for it. Lower the `*` entry's family (the only model-independent one) onto the CLI subprocess env, restarting the client when it changes, and validate its shape first: the setting is workspace-configurable, and a value carrying a NUL makes `spawn` throw, which would take down every session in the window. Also send a configured reasoning-effort override on session resume. The SDK forwards `reasoningEffort` on `session.resume` as well as `session.create`, but the launcher only set it for a create, so the setting did nothing for existing sessions until their model changed. Only a configured override is sent, so a session without one still keeps the effort the runtime persisted. Finally, let a session with no chosen model (server-side "Auto") match the `*` entry: it has no model-id entry to look up, and exempting it made the wildcard mean "every model except Auto" for both effort and tool filters. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Agent host: apply per-model capability overrides Route family aliases through the SDK model field, expose model capability and tool filters per model, and avoid process-wide family state. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Agent host: align capability override behavior Map tool-search filter names to the SDK runtime tool, retain tool-search guidance in replacement prompts, and use the canonical reasoning-effort schema. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Agent host: apply family aliases on model changes Keep mid-session model changes aligned with create and resume by sending the configured family alias and dropping picker tuning in favor of the family's runtime defaults. Also tighten the related prompt and capability override documentation.\n\nCo-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Agent host: remove global reasoning-effort override, retry resume without pinned model Fold the global chat.agentHost.copilot.reasoningEffortOverride setting into the per-model capability overrides (the '*' entry now covers the same case), and retry a failed resume without the pinned model id before surfacing the error, since the runtime falls back to its journaled model. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Refactor Copilot agent model handling and capability overrides * Agent host: update tool availability descriptions to clarify wildcard behavior * Agent host: clarify tool filter behavior and update documentation for capability overrides * Agent host: fix capability override edge cases Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Agent host: refine comments for clarity in capability override and tool filter functions * Agent host: refactor resolveModelCapabilityOverride to resolveModelCapabilityOverrideField for clarity and improved functionality * Agent host: enhance descriptions for tool availability and exclusion patterns; clean up unused code and improve test coverage for model family alias handling * Agent host: update session handling to use listLegacyChats and provisionSession for improved chat management --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Add tool search to Copilot agent host (#326213) * Add tool search to Copilot agent host * Address review: alias-aware tool-search gating, drop transient candidates, cover prompt branch - Compute the tool-search capability decision from the family-aliased model in both the launcher and CopilotAgentSession so an aliased preview model is no longer rejected and the two stay in agreement. - Strip the transient tool-search candidate corpus from the completed tool call's _meta so repeated searches don't bloat synchronized session state or persist across reconnects. - Add active/inactive + composition tests for the tool-search prompt line at both the unit and prompt-registry layers, plus a model-family alias regression test for tool-search gating. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Remove obsolete tool-search start barrier * Address tool-search review feedback * Use the extension's existing tool-embedding cache for Agent Host search * Address review: keep server tools eager, harden tool-search candidate corpus - Force server-provided SDK tools to defer:'never' so they stay eager and are not deferred behind tool_search. - Default missing candidate descriptions to '' when building the tool-search corpus, so the all-or-nothing metadata reader never drops a valid corpus. - Drop the unused inputSchema field from the tool-search candidate corpus (name + description are all the embeddings ranker consumes). - Use the client-facing tool name in the custom-tool auto-approve membership check so a deferred tool_search runtime name resolves correctly. - Remove a redundant conjunct in _clientToolName already implied by _isToolSearchActive(). * Potential fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * Potential fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * Temporarily disable GPT tool search --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

Report recursive Agent Host turn usage telemetry (#332043) * agentHost: report recursive turn usage telemetry Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * agentHost: preserve inclusive usage for unmapped children Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * agentHost: clean up completed subagent ownership Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * agentHost: preserve subagent ownership for legacy usage events Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * agentHost: trim comments to essentials Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * agentHost: report spawning tool call id on subagent turns Add parentToolCallId to agentHost.turnCompleted so a subagent turn joins exactly to the languageModelToolInvoked row for the task call that spawned it. Stable across resumed turns of the same subagent, undefined on roots. --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot CLI agent-host: experimentation overrides + config split (#324099) * Copilot CLI agent-host: experimentation overrides + config split Ports reasoningEffortOverride and modelCapabilityOverrides from the Copilot Chat extension to the Copilot CLI agent-host provider, and splits Copilot-CLI-specific root-config keys into a dedicated schema. - New settings chat.agentHost.reasoningEffortOverride and chat.agentHost.modelCapabilityOverrides (experimental/advanced), forwarded into the local agent host root config. - New copilotCliConfig module owns CLI-only keys; opus48Prompt, enableCustomTerminalTool, rubberDuck moved out of the shared schema (wire strings unchanged, so persisted config stays valid). - Reasoning-effort override applied at session create and mid-session model change; family alias applied for prompt routing only (wire model id untouched). - Shared AgentHostRootConfigForwarder de-duplicates the settings->root-config forwarding (schema gate, hydration retry, cross-window loop guard). * agentHost: consolidate CLI setting IDs, extract sandbox forwarder, trim comments - Move the chat.agentHost.* Copilot-CLI setting IDs into copilotCliConfig.ts beside their root-config keys; rename the prompt contribution to AgentHostCopilotCliSettingsContribution. - Extract the sandbox settings forwarding into AgentHostSandboxForwarder. - Trim verbose doc/inline comments across the changed agent-host files. * agentHost: revert comment-only changes in agentHostSandboxForwarder (keep import change only) * agentHost: clarify reasoning-effort override wording (recognized level vs model-supported)

Add semantic search to Copilot agent host sessions (#331836) * agentHost: add semantic search tool * Refactor agent host tool enablement logic and enhance semantic search documentation * agentHost: reserve semantic search tool names * agentHost: remove unused semantic search tool ID and update related logic * agentHost: fix semantic search tool hygiene * Refactor semantic search tool instructions for clarity and consolidation * agentHost: fix semantic search tool identity * agentHost: publish tools before starting turns * agentHost: trim semantic search scope * agentHost: fix semantic search execution * copilot: remove semantic search mode experiment

Surface Agent Host (Copilot CLI) sessions in the Chat Debug Logs panel (#321809) Adds a core-side IChatDebugLogProvider that reads each Agent Host Copilot CLI session's on-disk events.jsonl and converts the records into debug-panel events, reconstructing the user -> model-turn -> tool-call trajectory tree. Local and remote (remote-<authority>-copilotcli) agent-host sessions are made debug-eligible, historical local sessions are discovered for the home list, and the currently-viewed session's events.jsonl is watched for live refresh. Usage reporting: - Session-cumulative input/cache tokens and Copilot AIU are back-filled from the session.shutdown summary onto model-turn events so the Summary tiles sum exactly; in-progress sessions fall back to live AHP session-state usage. Lifecycle: - Add IChatDebugService.onDidEndSession so the provider can dispose its live file watcher when the session it follows is closed.

Add per-model system-prompt registry for Copilot agent-host sessions (#321864) * Add per-model system-prompt registry for Copilot agent-host sessions Introduces a prompt registry (mirroring the Copilot extension's PromptRegistry/IAgentPrompt) so Copilot CLI agent-host sessions resolve their SDK system message per model instead of a single hardcoded constant. First contributor: a Claude Opus 4.8 resolver applying customize-mode section overrides, opt-in via the new chat.agentHost.opus48Prompt.enabled setting (forwarded into the local agent host root config). Forwarding is registered for the VS Code workbench only, not the Agents window. * Address PR review: empty section overrides fall back to default; strongly type test schema helper - promptRegistry: treat an empty resolveSectionOverrides() result as 'no override' so the default identity customization is preserved instead of emitting customize-mode with empty sections. - agentHostCopilotPromptContribution.test: import ConfigPropertySchema and type makeRootStateWithSchema, removing the unsafe Record<string, never> cast. - agentHostPromptRegistry.test: add regression test for the empty-overrides fallback. - anthropicPrompt: lead the Opus 4.8 tone append with a newline so it doesn't run on from the SDK foundation tone.

Show Anthropic refusals correctly (#331315) * Fix Anthropic refusal error handling Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Handle refusal edge cases Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Simplify refusal error handling Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Avoid logging refusal explanations Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>