Aaron Munger

agentHost: track edit attribution telemetry (#327729) * agentHost: track edit attribution telemetry Attribute tool-driven edits in Agent Host, correlate workbench reloads, and coordinate aggregate telemetry without heavyweight reconciliation. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * agentHost: report attribution coverage gaps Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * address comments * agentHost: serialize attribution finalization Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * editTelemetry: provide text file service in test Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

agentHost: Emit standalone edit attribution stats (#329008) * agentHost: emit standalone edit attribution stats Emit honest standalone edit-source stats for Agent Host-only files, track external retained characters, and keep coordinated attribution exact-once. Add agentHostModifiedCount as a mutually exclusive stats category for both workbench and standalone windows.\n\nCo-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * agentHost: harden standalone attribution reconciliation Account for tracked edits invalidated by oversized files, make standalone coverage acknowledgements bounded and idempotent, and reconcile same-file sessions independently of flush order.\n\nCo-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * agentHost: bound attribution coordination Track coverage gaps by exact coordinated cutoffs, paginate standalone acknowledgements, scope GitHub telemetry to Copilot CLI, version the stats semantics, and enforce one wall-clock reconciliation deadline.\n\nCo-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

report edit arc in Agent host (#328027) * agentHost: report edit ARC telemetry Port ARC tracking to Agent Host and integrate it with edit attribution. * agentHost: harden ARC telemetry reporting Handle sampling failures, subagent attribution, and stable branch detection. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * agentHost: address ARC telemetry review feedback Ensure detailed diffs reconstruct EOL changes, keep best-effort ARC work off the tool completion path, and harden reporter lifecycle limits and races. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

agent host edit survival tracking (#321233) * use the existing editTracker to help emit edit survival metrics, tracked at full file level for now * attempt to grade survival per chunk, extracting chunk locations from the tool call parameters * build the file n-gram set once per chunked-survival call Previously computeChunkedFourGramSurvival called computeFractionPresentIn for each chunk, which rebuilt the file's 4-gram Set every time. Cost was O(|chunks| * |file|); a single chunked-scoring call against a 500 KB file with 50 chunks did ~25 MB of repeated Set builds, and that ran for every one of the 7 samples in the 15-min schedule. Factor out buildNGramSet + computeFractionPresentInSet and have the chunked path build the set exactly once. Adds a regression test that asserts 50 chunks against a 500 KB file scores in <1s. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * skip edit-survival tracking for files larger than 5 MB Reporter samples the file 7 times over 15 minutes and pins beforeText / afterText snapshots for the duration. Past a certain size the per-sample n-gram work and snapshot retention stop paying for themselves -- the file is almost certainly not something the AI meaningfully authored end-to-end, and survival math on it would be dominated by churn we don't care about. 5 MB still covers every realistic source file; chosen as a single threshold for both whole-file and chunked scoring modes because noRevert (the dominant per-sample cost) is identical between them. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * only treat FILE_NOT_FOUND as deletion in edit-survival reporter Transient readFile errors (permissions, FILE_TOO_LARGE, provider hiccups, etc.) were being recorded as 'file deleted' events, which both biased telemetry toward false reverts and stopped sampling early. Now we only mark the file as deleted when the underlying FileOperationResult is FILE_NOT_FOUND; other errors log a warning, skip the sample, and the reporter keeps running for subsequent samples. Addresses PR review feedback on #321233. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * move AI-chunk extraction into FileEditTracker The agent sessions (Claude observer + Copilot session) were each calling extractAiChunks() before takeCompletedEdit() and passing the result through. The tracker is the only consumer of the chunks (it forwards them to the survival reporter), so callers don't need to know about that detail. Move the extraction inside the tracker; callers now just pass the toolName + raw tool input they already have. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * tidy comments and document chunked-scoring coverage invariant - Drop stale historical references (Phase 8 prefix, 'production extension' pointers, 'previously / atm / deliberately' wording) and reword to describe current behavior only. - Soften the .ipynb skip comments to note we may revisit once we have a notebook-aware tracker, rather than ruling notebooks out. - Document in editChunkExtractor.ts that every tool the agent host currently treats as a file-edit tool has a matching case here, so the whole-file scoring path is a safety net for SDK shape drift / newly added tools rather than a regular code path. Cross-reference from IEditSurvivalReporterLaunchParams.aiChunks. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix tsgo override signature in RecordingTelemetryService NullTelemetryServiceShape.publicLog2 declares zero parameters; tsgo (stricter than tsc on override compatibility) rejects an override that adds required-shaped parameters. Match the base by declaring the override with optional params, and drop the now-unused gdprTypings imports. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * add modelId and aiCharCount to edit-survival telemetry Stamps each agentHost.trackEditSurvival event with the model that produced the edit and the AI-written character count, so dashboards can slice by model and weight survival by AI contribution (matching the chat ext's ghcopilot_committed_characters_hll_model_sku rollup). Claude reads the model per assistant message, which is naturally correct for subagents. Copilot tracks the session's last-seen model via setModel and onUsage -- best-effort for subagents but accurate for the parent agent. aiCharCount is 0 in the whole-file fallback (coverage invariant means this filters out ~0% of real traffic; dashboards should filter scoringMode='chunked'). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * warn when edit-survival modelId is missing modelId should always be populated in practice. An empty modelId in the emitted telemetry now indicates a bug worth investigating, and the tracker logs a warning at edit-completion time so we notice locally too. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * doc: correct modelId subagent-attribution claim The Copilot CLI SDK's Usage event fires per LLM call with the model that actually ran it -- including subagent calls. So _lastSeenModelId correctly attributes subagent edits, not just parent-agent edits. Verified against a live sonnet-parent / haiku-subagent run where the subagent's three tool calls correctly emitted modelId=claude-haiku-4.5. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * doc: trim _lastSeenModelId comment Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

agentHost: Attribute edits in focus windows (#329472) * agentHost: Attribute edits in focus windows Share Agent Host reload correlations across the long-term, 10-minute, and 20-minute edit trackers. Focus windows reclassify the observed reload without consuming Agent Host retained state or double-counting totals, with bounded late-marker fallback. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * agentHost: Harden focus edit correlation Reject malformed marker metadata, preserve per-edit request identity, and expire unresolved reload observations without discarding resolved results still used by active focus trackers. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * agentHost: Clarify marker source validation Use explicit rejection and a named type guard for untrusted Agent Host marker source metadata. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

agentHost: emit ask questions telemetry (#328460) * Add ask questions telemetry for agent host Track live ask-user input requests through completion and emit the existing telemetry event with Agent Host session identifiers. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Classify Claude elicitation requests in mapper Keep elicitation purpose ownership with the mapper so every generated request is classified consistently. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Clear ask questions telemetry on truncation Drop pending chat input telemetry when truncation removes the active turn, allowing reused request IDs to correlate with the new turn. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

agentHost: classify Copilot SKU telemetry context (#333853) * agentHost: classify Copilot SKU telemetry context Declare the dynamically injected copilotSku property on each applicable Agent Host event so telemetry extraction generates ingestion schemas that accept it. Keep runtime population centralized through the common-property path. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * agentHost: clear stale Copilot SKU telemetry Allow common telemetry properties to be removed so account transitions clear the previous SKU before resolving the current account. Classify the same injected property on forwarded Copilot SDK event schemas. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * agentHost: compose shared telemetry context Name the common initiator and Copilot SKU schema composition and use event-specific aliases when enriching generic telemetry types. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * agentHost: complete Copilot SKU telemetry coverage Classify SKU context on Agent Host edit attribution and unhandled error events. Add regression coverage for authentication transitions, stale SKU resolution, and restricted envelope clearing. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

agentHost: report completed model calls per turn (#331551) * agentHost: report completed model calls per turn Count stable provider response boundaries across Copilot, Claude, and Codex so turn telemetry can analyze agent loop depth without counting streaming fragments or duplicate usage updates. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * agentHost: refine model call counting Document the Codex usage-based approximation and avoid overcounting split Copilot responses when call-level identifiers are unavailable. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

agentHost: Track tool result size and model (#328866) * agentHost: Track tool result size Add a serialized result-size measurement to languageModelToolInvoked so Agent Host telemetry preserves tool response-size analysis without duplicating the legacy event schema. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * agentHost: Add model to tool telemetry Track the resolved usage model and turn identifier on Agent Host tool invocation events. Preserve Auto as the selection kind while applying the existing trusted, BYOK, and unknown model privacy treatment. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * agentHost: Wait for resolved tool model Defer tool invocation telemetry until usage supplies the resolved model, with turn-end fallback and explicit cleanup for chat, session, and subagent disposal paths. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

sessions: protect provider telemetry identifiers (#330362) Replace connection-specific provider IDs with bounded categories, hash session correlation IDs, and migrate provider counters to the same safe dimensions. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>