Aaron Munger

agentHost: track edit attribution telemetry (#327729) * agentHost: track edit attribution telemetry Attribute tool-driven edits in Agent Host, correlate workbench reloads, and coordinate aggregate telemetry without heavyweight reconciliation. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * agentHost: report attribution coverage gaps Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * address comments * agentHost: serialize attribution finalization Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * editTelemetry: provide text file service in test Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

agentHost: Emit standalone edit attribution stats (#329008) * agentHost: emit standalone edit attribution stats Emit honest standalone edit-source stats for Agent Host-only files, track external retained characters, and keep coordinated attribution exact-once. Add agentHostModifiedCount as a mutually exclusive stats category for both workbench and standalone windows.\n\nCo-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * agentHost: harden standalone attribution reconciliation Account for tracked edits invalidated by oversized files, make standalone coverage acknowledgements bounded and idempotent, and reconcile same-file sessions independently of flush order.\n\nCo-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * agentHost: bound attribution coordination Track coverage gaps by exact coordinated cutoffs, paginate standalone acknowledgements, scope GitHub telemetry to Copilot CLI, version the stats semantics, and enforce one wall-clock reconciliation deadline.\n\nCo-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

report edit arc in Agent host (#328027) * agentHost: report edit ARC telemetry Port ARC tracking to Agent Host and integrate it with edit attribution. * agentHost: harden ARC telemetry reporting Handle sampling failures, subagent attribution, and stable branch detection. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * agentHost: address ARC telemetry review feedback Ensure detailed diffs reconstruct EOL changes, keep best-effort ARC work off the tool completion path, and harden reporter lifecycle limits and races. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

agent host edit survival tracking (#321233) * use the existing editTracker to help emit edit survival metrics, tracked at full file level for now * attempt to grade survival per chunk, extracting chunk locations from the tool call parameters * build the file n-gram set once per chunked-survival call Previously computeChunkedFourGramSurvival called computeFractionPresentIn for each chunk, which rebuilt the file's 4-gram Set every time. Cost was O(|chunks| * |file|); a single chunked-scoring call against a 500 KB file with 50 chunks did ~25 MB of repeated Set builds, and that ran for every one of the 7 samples in the 15-min schedule. Factor out buildNGramSet + computeFractionPresentInSet and have the chunked path build the set exactly once. Adds a regression test that asserts 50 chunks against a 500 KB file scores in <1s. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * skip edit-survival tracking for files larger than 5 MB Reporter samples the file 7 times over 15 minutes and pins beforeText / afterText snapshots for the duration. Past a certain size the per-sample n-gram work and snapshot retention stop paying for themselves -- the file is almost certainly not something the AI meaningfully authored end-to-end, and survival math on it would be dominated by churn we don't care about. 5 MB still covers every realistic source file; chosen as a single threshold for both whole-file and chunked scoring modes because noRevert (the dominant per-sample cost) is identical between them. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * only treat FILE_NOT_FOUND as deletion in edit-survival reporter Transient readFile errors (permissions, FILE_TOO_LARGE, provider hiccups, etc.) were being recorded as 'file deleted' events, which both biased telemetry toward false reverts and stopped sampling early. Now we only mark the file as deleted when the underlying FileOperationResult is FILE_NOT_FOUND; other errors log a warning, skip the sample, and the reporter keeps running for subsequent samples. Addresses PR review feedback on #321233. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * move AI-chunk extraction into FileEditTracker The agent sessions (Claude observer + Copilot session) were each calling extractAiChunks() before takeCompletedEdit() and passing the result through. The tracker is the only consumer of the chunks (it forwards them to the survival reporter), so callers don't need to know about that detail. Move the extraction inside the tracker; callers now just pass the toolName + raw tool input they already have. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * tidy comments and document chunked-scoring coverage invariant - Drop stale historical references (Phase 8 prefix, 'production extension' pointers, 'previously / atm / deliberately' wording) and reword to describe current behavior only. - Soften the .ipynb skip comments to note we may revisit once we have a notebook-aware tracker, rather than ruling notebooks out. - Document in editChunkExtractor.ts that every tool the agent host currently treats as a file-edit tool has a matching case here, so the whole-file scoring path is a safety net for SDK shape drift / newly added tools rather than a regular code path. Cross-reference from IEditSurvivalReporterLaunchParams.aiChunks. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix tsgo override signature in RecordingTelemetryService NullTelemetryServiceShape.publicLog2 declares zero parameters; tsgo (stricter than tsc on override compatibility) rejects an override that adds required-shaped parameters. Match the base by declaring the override with optional params, and drop the now-unused gdprTypings imports. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * add modelId and aiCharCount to edit-survival telemetry Stamps each agentHost.trackEditSurvival event with the model that produced the edit and the AI-written character count, so dashboards can slice by model and weight survival by AI contribution (matching the chat ext's ghcopilot_committed_characters_hll_model_sku rollup). Claude reads the model per assistant message, which is naturally correct for subagents. Copilot tracks the session's last-seen model via setModel and onUsage -- best-effort for subagents but accurate for the parent agent. aiCharCount is 0 in the whole-file fallback (coverage invariant means this filters out ~0% of real traffic; dashboards should filter scoringMode='chunked'). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * warn when edit-survival modelId is missing modelId should always be populated in practice. An empty modelId in the emitted telemetry now indicates a bug worth investigating, and the tracker logs a warning at edit-completion time so we notice locally too. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * doc: correct modelId subagent-attribution claim The Copilot CLI SDK's Usage event fires per LLM call with the model that actually ran it -- including subagent calls. So _lastSeenModelId correctly attributes subagent edits, not just parent-agent edits. Verified against a live sonnet-parent / haiku-subagent run where the subagent's three tool calls correctly emitted modelId=claude-haiku-4.5. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * doc: trim _lastSeenModelId comment Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

agentHost: Attribute edits in focus windows (#329472) * agentHost: Attribute edits in focus windows Share Agent Host reload correlations across the long-term, 10-minute, and 20-minute edit trackers. Focus windows reclassify the observed reload without consuming Agent Host retained state or double-counting totals, with bounded late-marker fallback. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * agentHost: Harden focus edit correlation Reject malformed marker metadata, preserve per-edit request identity, and expire unresolved reload observations without discarding resolved results still used by active focus trackers. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * agentHost: Clarify marker source validation Use explicit rejection and a named type guard for untrusted Agent Host marker source metadata. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

agentHost: emit ask questions telemetry (#328460) * Add ask questions telemetry for agent host Track live ask-user input requests through completion and emit the existing telemetry event with Agent Host session identifiers. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Classify Claude elicitation requests in mapper Keep elicitation purpose ownership with the mapper so every generated request is classified consistently. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Clear ask questions telemetry on truncation Drop pending chat input telemetry when truncation removes the active turn, allowing reused request IDs to correlate with the new turn. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

agentHost: Track tool result size and model (#328866) * agentHost: Track tool result size Add a serialized result-size measurement to languageModelToolInvoked so Agent Host telemetry preserves tool response-size analysis without duplicating the legacy event schema. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * agentHost: Add model to tool telemetry Track the resolved usage model and turn identifier on Agent Host tool invocation events. Preserve Auto as the selection kind while applying the existing trusted, BYOK, and unknown model privacy treatment. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * agentHost: Wait for resolved tool model Defer tool invocation telemetry until usage supplies the resolved model, with turn-end fallback and explicit cleanup for chat, session, and subagent disposal paths. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

TTFT and total time events for agent host sessions (#320194) * TTFT events * tests * address comments

agentHost: separate agent selection and execution mode telemetry (#328843) * agentHost: separate agent and execution mode telemetry Keep chat.modeChange scoped to user picker and custom-agent selections, including the Agents Window, and report interactive, plan, and autopilot transitions through agentHost.executionModeChanged. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * sessions: scope mode picker telemetry to active chat Resolve the previous mode and request count from the picker surface's scoped session and active chat so multi-session and peer-chat selections are attributed correctly. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Add edit tracker characterization tests (#326062) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Validate workspace trust for notebook links (#328043) * Validate workspace trust for notebook links Route notebook webview resource links through the editor service trust gate while preserving selections and editor group targeting. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Preserve notebook link opener behavior Keep custom URI, folder, fragment selection, and existing editor handling intact while validating trust for out-of-workspace resources. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Fix notebook link opener options Rely on the opener URI fragment for selections instead of passing text editor options through the generic opener API. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

tool call details parity (#327873) * added classification comments for response events * Classify forwarded Copilot identifiers as pseudonymous Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * agentHost: Emit toolCallDetails telemetry Preserve the local per-turn tool-call aggregate on the Microsoft telemetry channel while retaining the restricted peer events. Include Agent Host correlation fields and report successful, cancelled, and failed turns once. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * agentHost: Report tool-call details before steering Reuse the normal turn completion path when steering preempts a turn so its telemetry aggregate is emitted before state resets. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

agentHost: track todo store telemetry (#328267) * agentHost: track todo store telemetry Classify successful Agent Host SQL access to todos and todo_deps without exposing raw queries. Add toolCallId correlation to generic tool telemetry while keeping the existing local todo-list event unchanged. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * agentHost: classify todo SQL table access Tokenize SQL before deriving todo telemetry so literals, comments, aliases, and unrelated writes cannot create false todo-store operations. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * telemetry: update todo store owner Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * agentHost: Adapt todo telemetry to referenced inputs Classify only inline todo SQL after the Agent Host protocol added referenced tool input, and update the model-attribution telemetry expectation for tool-call correlation. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * agentHost: scope todo telemetry to Copilot CLI Move todo SQL classification and emission into the Copilot provider path so other harnesses do not participate. Keep generic tool-call correlation in the shared tracker. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Add languageModelToolInvoked telemetry from Copilot agent host session (#317180) * Add languageModelToolInvoked telemetry from Copilot agent host session * Register NullTelemetryService in createTestAgentContext

agentHost: Fix edit harness attribution (#328519) * Fix Agent Host edit harness attribution Resolve AHP chat channel URIs to their owning session when deriving edit telemetry harnesses while preserving per-chat resource and conversation identity. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Clarify Agent Host conversation ID coverage Use explicit expected IDs so the harness attribution test documents the existing chat-channel conversation semantics without masking the shared encoded parent ID. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Fix Claude peer chat edit attribution Route Claude file edit tracking through the effective chat storage URI so peer chat edits remain independently recorded and flushed. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

chat tool approval parity (#327884) * added classification comments for response events * Classify forwarded Copilot identifiers as pseudonymous Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * agentHost: Emit toolCallDetails telemetry Preserve the local per-turn tool-call aggregate on the Microsoft telemetry channel while retaining the restricted peer events. Include Agent Host correlation fields and report successful, cancelled, and failed turns once. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * agentHost: Emit chat.toolApproval telemetry Emit the workbench-compatible tool-approval event from the agent host on the standard telemetry channel, correlating SDK permission events with each tool invocation. Fires once per tool call at either permission.completed (denied) or tool.execution_start (approved / no permission requested). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * agentHost: Wait for tool permission outcome Defer chat.toolApproval emission until permission.completed when a permission lifecycle exists. Tool calls without permission requests now emit at completion with no inferred auto-approval reason, preventing premature and contradictory rows. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

emit instructionsCollected event for agent host (#325012) * Add telemetry for collected instructions in CopilotAgentSession * polish * add AH specific properties * added tests * fix --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>

agentHost: report chat mode change telemetry (#328504) * agentHost: report chat mode change telemetry Emit chat.modeChange when the effective Agent Host execution mode changes while suppressing SDK echo duplicates. Include Agent Host session context and regression coverage for mode transitions and default resets. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * agentHost: preserve native execution mode names Keep interactive, plan, and autopilot verbatim in chat.modeChange rows so Agent Host execution modes are not conflated with workbench chat modes. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Relay Copilot SDK GitHub events (#324815) * Relay Copilot SDK GitHub telemetry events through ITelemetryService Update @github/copilot-sdk to 1.0.6-preview.0, which adds the onGitHubTelemetry connection-global callback. Wire it up via a dedicated CopilotGitHubTelemetryForwarder that owns ITelemetryService and re-emits each forwarded event through publicLog so they land in our cluster. Restricted events are only forwarded when restricted telemetry is enabled for the current Copilot token. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Route SDK telemetry through GitHub sinks Preserve native event names and payload fields, route restricted events to the enhanced GitHub telemetry sink, multiplex oversized properties, and add focused coverage for mapping and restricted telemetry gating. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Route SDK telemetry through VS Code 1DS Re-emit forwarded SDK events through ITelemetryService instead of posting them back to the GitHub telemetry endpoints. Preserve event metadata and continue gating restricted events on the account option. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

send telemetry for CLI tool calls (#316135) * send telemetry for CLI tool calls * :lipstick: * Address Copilot CLI telemetry review feedback * Fix Copilot CLI telemetry tool name fallback