Anthony Shew

fix: Discover native workspace metadata lazily (#14053) ### Description Closes [TURBO-6071](https://linear.app/vercel/issue/TURBO-6071/go-workspaces-defer-toolchain-discovery-until-go-packages-are-in-run). Use generic lazy native discovery across JavaScript, Cargo/Rust, Python/uv, and Go, while preserving each native toolchain's authoritative task and dependency semantics. - Collect a subprocess-free inventory containing only scope identities, manifest paths, and workspace roots. The inventory makes no claims about tasks, dependency edges, or execution contracts. - Use the existing package selector to narrow scope requests. When engine construction reaches unloaded metadata, request its owner and load that contributor's authoritative discovery output. Loading is monotonic, and final task selection happens only after those requests settle. - Load metadata reached through explicit task dependencies, topological dependencies, companion tasks, and root dependencies needed for hashing. Route this through contributor ownership rather than per-language filter checks. - Remove the previous planning-uncertainty framework, staged topology-reconciliation errors, and static Go task/build-constraint and replacement-graph reconstruction. Native Go discovery continues to resolve build tags and version-sensitive replacements. - Keep native resolution fallbacks scoped to their consumers so unrelated task hashes remain stable, and refresh watch snapshots when native metadata has been loaded. - Cover narrow missing-toolchain selections, qualified task arguments, cross-language dependencies, native catalogue queries, missing-task diagnostics, and hash stability with regression tests. ### Discovery contract Narrow JavaScript package or qualified-task selections do not load unrelated native contributors. Queries that reach native scopes or need repository-wide metadata may still require their toolchains, even if no native task ultimately executes. This includes unfiltered runs, dependency-wide or affected queries, strict task-entrypoint catalogue queries, and whole-repository consumers such as package listing and watch discovery. Task catalogues and dependency relationships come from native discovery rather than a partial approximation. Missing tools and invalid native workspaces retain their ordinary diagnostics; there are no new user-facing planning-proof or topology-reconciliation errors.

fix: Refactor `turbo watch` (#13423) ## Summary - replace the shared raw filesystem event broadcast with scoped, demand-driven subscriptions for package changes, hashing, outputs, discovery, cookies, devtools, and daemon root monitoring - model inherited and nested Git ignores, global excludes, tracked exceptions, dynamic Git controls, and explicit ignored input/output interests - prune ordinary ignored trees from Linux inotify coverage while preserving explicit coverage, and route non-mutating backends directly after readiness - add scoped recovery for backend rescans/errors and document the watcher architecture Closes #13402 ## Validation - `cargo test -p turborepo-filewatch --lib` focused and platform-independent coverage - Linux all-feature filewatch test binary in Docker: 68/68 passed - `cargo test -p turborepo-lib package_changes_watcher::test` - `cargo clippy -p turborepo-filewatch --all-targets --all-features -- -D warnings` - `cargo clippy -p turborepo-lib -p turborepo-daemon -p turborepo-devtools --all-targets -- -D warnings` - `cargo check -p turborepo-filewatch --target x86_64-pc-windows-gnu` - original Linux reproduction with three repeated 100,000-file bursts: zero package/hash lag, one package build each, and one web-server start ## Notes Native macOS FSEvents stress was limited by local stream exhaustion after repeated test runs; focused FSEvents path/rescan tests and macOS compilation pass.

perf: Skip work for disabled telemetry (#13972) <!-- startup-recovery-20260908 --> ### Review scope Independent optimization targeting `main`, with exactly one feature commit. No other optimization PR is a prerequisite. Recovered after the bundled #13970 merge was reverted by #13982. The separately merged parser optimization #13971 remains on `main`. This is not part of a cumulative optimization stack. The timing figures below are historical measurements from the original incremental benchmark sequence. They have not been remeasured on these newly independent branches and are not additive. <!-- /startup-recovery-20260908 --> ### Description Make explicitly disabled telemetry inert. Environment opt-out skips config loading; disabled telemetry creates no worker and avoids event UUID generation, formatting, hashing, and shutdown timers. Preserve enabled behavior and distinguish opt-out from missing initialization. ### Testing Instructions The startup investigation traced work performed even when telemetry was disabled. End-to-end run measurements did not establish a consistent standalone speedup for this layer; this removes avoidable opt-out work rather than claiming the combined benchmark gains. Enabled-mode measurements used a closed local proxy to exclude external network variability. #### Measurement limits - Measurements used optimized binaries, randomized paired before/after runs, warm OS/local caches, and synthetic npm fixtures. CPU/heap/syscall profiling was separate from wall-time timing. - Early batches had an unborn Git HEAD; later normalization and matcher-reuse batches used committed fixtures. Results from separate batches are not cumulative. - Windows runtime performance remains unmeasured. No worker-pool sizes or task concurrency defaults are reduced.

perf: Overhaul pnpm lockfile parsing and dependency closure computation (#13228) ## Why On large pnpm monorepos, the bulk of `turbo run` startup — the window between invocation and the first task starting — is spent parsing `pnpm-lock.yaml` and computing per-workspace transitive dependency closures. Profiling a 1191-package internal monorepo (43k tracked files, 5.4MB lockfile) showed the closure walk dominated by string allocation, hashing, and map probes rather than actual resolution work, and YAML parsing spending ~75% of its time inside a general-purpose scanner that pnpm's rigid machine-generated output doesn't need. ## What This PR has grown into the full pnpm startup overhaul: five stacked changes, each merged into this branch after independent review and verification. **1. Closure walk interning + FxHash.** Resolved packages are interned to dense `u32` ids behind the existing caches: resolve-cache hits become a refcount bump instead of cloning two `String`s per closure edge, visited sets hash a `u32` instead of two heap strings, and the deps cache is keyed by id. Cache maps use FxHash — lockfile content is developer tooling input, not a hash-flooding vector. **2. pnpm hash index** (#13229). The post-parse `dependency_index` becomes an `FxHashMap` carrying membership flags and versions, making `has_package`/`package_version`/`all_dependencies` O(1) probes instead of `BTreeMap` walks over long keys. Entry dependency maps stay `BTreeMap` so closure iteration order remains deterministic. Also fixes `subgraph()` returning pruned lockfiles with an empty index. **3. Fast pnpm parse on saphyr events** (#13230). An event-driven parser (pure Rust, ~2x libyaml scanner throughput) builds `PnpmLockfile` without serde on the hot path. Deliberately conservative: multi-document input, anchors/aliases, tags, duplicate keys, and exotic numeric forms bail to the unchanged serde path. Observable serde quirks (untagged variant order, `lockfileVersion` float formatting, plain-scalar typing, unknown-field dropping) are mirrored exactly and enforced differentially. **4. Structural line scanner** (#13238). pnpm-lock.yaml is line-oriented machine-generated YAML, so a `memchr`-driven scanner specialized to exactly that subset replaces the general YAML state machine as tier 1, emitting the same event stream the semantic layer from (3) consumes — quirk handling is shared, not duplicated. Tier order: scanner → saphyr → serde; the scanner also declines anything a general parser would *reject*, so it can never fabricate a lockfile from invalid input. Roughly halves parse time again on every corpus measured. **5. Shared bitset closure DP** (#13239). Closures were recomputed per workspace, so shared dependencies were re-walked once per workspace reaching them. When the lockfile can prove every transitive edge resolves identically across workspaces (pnpm importers can shadow transitive resolution, so this is proven per edge via importer equivalence classes, not assumed), closures are computed once globally: iterative Tarjan SCC condensation, then a bottom-up bitset DP where set unions are word-parallel `u64` ORs, processed in bounded-memory chunks. Any sensitive edge, non-pnpm lockfile, or single-workspace call falls back to the unchanged legacy walk. ## Results Interleaved A/B against current `main`, 4-core Linux, telemetry disabled, on the 1191-package monorepo. Time-to-first-task is measured in-process (invocation to first task dispatch in the chrome profile). | Benchmark | main | this PR | delta | |---|---|---|---| | time-to-first-task (real run) | 1117ms | 472ms | **−58%** | | `turbo run build --dry=json` end-to-end | 1481ms | 871ms | **−41%** | | lockfile parse (5.4MB, `from_bytes`) | ~285ms | 56ms | **−80%** | | peak RSS (dry) | — | −27MB vs branch base | bitsets beat per-workspace set churn | Parse deltas hold on public corpora too: next.js lockfile (1.29MB) 30.9→15.0ms, this repo's (707KB) 15.3→8.0ms for the scanner tier alone. ## How to verify - Full `--dry=json` output diffed against current main on the internal benchmark monorepo and this repo — byte-identical modulo run id and version string. - 20 differential parse tests assert scanner == saphyr == serde (`Eq`-identical lockfiles, byte-identical `encode()` round-trips) across v5/v6/v9 fixtures, folded/literal block scalars with chomping/blank-line edges, quoting, flow collections, catalogs, patches, and this repo's own `pnpm-lock.yaml` — with scanner acceptance *required* on mainline shapes so regressions can't silently fall through to slower tiers. Each unsupported construct has explicit fallback coverage. - The closure DP is differentially tested against the legacy walk (an independent oracle — the single-workspace entry point never uses the DP), including a crafted divergent-edge fixture proving the fallback triggers. 220 crate tests pass.

perf: Reuse compiled workspace matchers (#13970) ### Description Layer 11 of 11 in the startup-performance stack, split from #13970. Each PR is based on the previous layer so its diff contains only this optimization. The top layer preserves the original combined source tree. Reuse compiled workspace matchers between inference and local discovery. Retain one successful set keyed by exact ordered inclusion/exclusion vectors, including spelling and duplicates. Configuration is still reread; no discovered paths, mtimes, symlink resolutions, or containment results are cached. Clone outside the mutex, preserve caller isolation, and fall back to compilation if the lock is poisoned. ### Testing Instructions Ordinary 50/500-package fixtures were mostly neutral. On a committed stress fixture with 50 real packages, one matching workspace pattern, and 200 unmatched patterns, Linux cache-hit medians improved 25.10 → 21.26 ms (15.3%) and macOS 16.42 → 15.68 ms (4.5%). Repeated `--skip-infer` and large dry-run guardrails did not establish a regression. DHAT cumulative allocations fell approximately 0.9–1.1 MB, with a small live-memory increase from retaining a matcher; this is not an RSS reduction. Strace confirmed unchanged filesystem-check counts. The cache is bounded to one entry, not a fixed byte budget. #### Measurement limits - Measurements used optimized binaries, randomized paired before/after runs, warm OS/local caches, and synthetic npm fixtures. CPU/heap/syscall profiling was separate from wall-time timing. - Early batches had an unborn Git HEAD; later normalization and matcher-reuse batches used committed fixtures. Results from separate batches are not cumulative. - Windows runtime performance remains unmeasured. No worker-pool sizes or task concurrency defaults are reduced. This is the original PR retained as the top of the stack. Rejected byte-scanning, direct task-output arena, and Mach-O chained-fixup experiments remain excluded. Profiling artifacts and the original combined PR description are preserved locally.

fix: Harden cache archive restoration (#14095) ## Summary - Replace pathname-based cache restoration on Unix with descriptor-relative traversal and mutation. - Retain validated directory handles on Windows and reject intermediate reparse points while restoring. - Stage regular files before atomically replacing their destinations, avoiding writes through final symlinks or hardlinks. - Preserve archive-created internal symlinks by tracking their physical, root-relative locations. - Add regression coverage for intermediate swaps, nested symlinks, stale mappings, Windows directory locking, and long paths. ## Why Cache restoration validated intermediate path components before later reopening the full destination path. A concurrent directory-to-symlink replacement could therefore redirect restored contents outside the selected root. ## Restore flow ```text cache entry open and retain restore root resolve archive-created internal symlinks traverse or lock each real parent directory stage the restored file beside its destination atomically replace the final entry ```

feat: Rebuild the factory image on every merge to main (#13781) ## What Agents in `apps/factory` booted from `vercel/eve:latest` plus a shallow clone. The sandbox had no Rust toolchain, `protoc`, Cap'n Proto, Zig, LLD, pnpm 10.28.0, `node_modules`, or performance tooling, and its Eve `revalidationKey` was hardcoded to `"turborepo-main-v1"`, so the template never rotated. `.devcontainer/Dockerfile` had most of that toolchain but was unused and had drifted three Rust nightlies, two Node majors, and two pnpm majors behind. This adds the factory image: one specification for what an agent's sandbox contains, a snapshot rebuilt by the application on every merge to `main`, and both agent paths booting from it. ## One specification `apps/factory/agent/lib/factory-image.ts` owns the image. It pins versions to the values the repository and CI already use, emits idempotent provisioning phases, and fingerprints the whole thing so a changed pin rebuilds every image. | Tool | Version | Source of truth | | --- | --- | --- | | Rust | `nightly-2026-05-22` + `rustfmt`, `clippy` | `rust-toolchain.toml` | | Node.js | 24 | root `package.json` `engines` | | pnpm | 10.28.0 | root `package.json` `packageManager` | | protoc | 26.1 | `.github/actions/setup-protoc` | | Cap'n Proto | 1.1.0 | `.github/actions/setup-capnproto` | | Zig | 0.15.2 | `.github/actions/setup-zig` | Plus `build-essential`, `pkg-config`, LLD (`.cargo/config.toml` links with `-fuse-ld=lld`), OpenSSL headers, `jq`, `zstd`, the workspace `node_modules`, a warm Cargo registry, and `hyperfine`, `cargo-bloat`, and `twiggy` for the performance skill. A final phase verifies every required tool, warns about missing optional ones, and writes a version manifest. `.devcontainer/Dockerfile` was rewritten against the same pins, and a test fails when either drifts from `rust-toolchain.toml`, the root `package.json`, or the CI actions — so the local dev container and the agents' sandbox stay on one toolchain. ## Rebuilt on every merge, without GitHub Actions A push to `main` reaches `POST /api/github/push`, which verifies the GitHub HMAC signature and starts a Workflow run. The workflow creates a build sandbox, detaches the provisioning script inside it, polls the markers the script writes, snapshots the result, and publishes the snapshot id to a Blob-backed ledger. No step holds a function invocation open for the length of a build. When a published image already exists for the same toolchain the build boots from it, so a merge build only fast-forwards the checkout, refreshes dependencies, and recompiles. ## Rapid merges Resolved in the ledger rather than by racing: - Claiming a build cancels every build still in flight, records which build replaced it, stops its workflow run, and deletes its sandbox. - Every step re-reads the ledger before acting; a build that has lost can neither report progress nor publish a pointer. - Redelivered webhooks deduplicate onto the live build, a revision that is already published is skipped, and a build that stops reporting progress for 15 minutes is replaced instead of wedging its revision. ## Consumers - **Eve** (`agent/sandbox.ts`) provisions its template from the same phases and boots from the published snapshot when one matches. Eve freezes `revalidationKey` at build time, so the template rotates when the fingerprint changes or a newer image is published; each session then fast-forwards its checkout to the current `main`. - **Harness** (`agent/lib/harness-agent.ts`) uses the snapshot as its sandbox source instead of a stock `node24` runtime plus a clone, rotates its template with the image, and falls back to cloning when no image matches this deployment's toolchain. - The **operator page** shows the published snapshot, the toolchain fingerprint, warnings, and recent builds, and can rebuild from the current `main` head. ## Validation Every provisioning phase was run against a real `vercel/eve:latest` sandbox. All eleven phases pass through verification in about two minutes, with no warnings: ``` system-packages 10s · node 1s · pnpm 2s · rust 11s · protoc <1s · zig 5s checkout 2s · node-modules 12s · cargo-registry 7s · performance-tools 45s · verify 1s {"capnp":"Cap'n Proto version 1.1.0","node":"v24.17.0","pnpm":"10.28.0", "protoc":"libprotoc 26.1","rustc":"rustc 1.97.0-nightly (e96c36b6f 2026-05-21)", "zig":"0.15.2"} ``` That run also confirmed the image is Ubuntu 26.04 with `apt`, that commands run as root, and that `ld.lld`, `hyperfine`, `cargo-bloat`, and `twiggy` all land on `PATH`. 45 unit tests pass (`pnpm test`), covering the pinned versions against the files they mirror, the phase list and generated script, revision validation, progress parsing, the fingerprint, the ledger's supersede and publish rules, and webhook signature and event filtering. `eve build`, `next build`, `tsc --noEmit`, `oxlint --deny-warnings`, and `oxfmt --check` are clean. ## Deployment notes - Requires the private Vercel Blob store the run registry already uses. - Set `FACTORY_IMAGE_WEBHOOK_SECRET` (falls back to `GITHUB_WEBHOOK_SECRET`) and deliver `push` to `/api/github/push`. Deployment Protection covers that path, so append the automation bypass token as a query parameter; the HMAC signature authenticates the delivery. - The first build after a toolchain change provisions the Eve template during the Vercel build, because Eve prewarms templates there. The phases that compile Rust are wrapped in timeouts so one bad upstream release cannot hold a deployment build open, and only the merge webhook asks for the warm `cargo build`. - Performance-tool installation failures are recorded as warnings rather than failing a build, so a broken upstream crate cannot stop an otherwise complete image from publishing. The warnings surface on the operator page. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: vercel[bot] <35613825+vercel[bot]@users.noreply.github.com>

fix: Infer all co-located packages from cwd (#14074) ### Description Fixes [TURBO-6081](https://linear.app/vercel/issue/TURBO-6081). Distinct toolchain packages can legitimately share a directory. Cwd-based scope inference currently selects only the first package discovered at the deepest matching directory, silently omitting its co-located peers. ```diff Find the deepest package directory containing cwd. -Select its first discovered package. +Select every package at that directory. ``` For example, running `turbo run build` inside a directory containing JavaScript, Go, and Rust packages now selects all three instead of whichever package was discovered first. Invoking from a nested child package still selects only that child's package identities. - Expand inferred selectors per package while preserving explicit names, directory filters, dependency traversal, exclusions, and Git-range modifiers. - Reuse one change snapshot per complete Git range within a filter pass, avoiding repeated repository scans for expanded selectors. - Keep co-located tasks independent: no warnings or automatic deduplication. This change does not address the separate changed-file ownership or task-log collision issues.

fix: Tolerate transient input files (#13734) ## Summary - Tolerate `ENOENT` for task inputs and `globalDependencies` files that disappear between discovery and hashing. - Keep required resolution fallback files strict and report path-specific `HashFile` errors. - Respect configured inputs literally, including `.git` metadata and linked-worktree pointer files. - Classify regular files from opened handles where supported and skip non-regular discovery candidates consistently across Git and manual SCM. - Record vanished candidates, non-regular candidates, and hashing failures when verbosity is enabled. Closes #13732 ## Testing - `cargo test -p turborepo-scm` - `cargo test -p turborepo-task-hash` - `cargo clippy -p turborepo-scm -p turborepo-task-hash --all-targets -- -D warnings` - `cargo fmt --all -- --check` --------- Co-authored-by: vercel[bot] <35613825+vercel[bot]@users.noreply.github.com>

fix: Avoid duplicate interrupts during task shutdown (#14046) ## Summary - Track which captured descendants were targeted by successful initial SIGINT requests, including direct PID and process-group delivery. - Avoid interrupting those descendants again when a PTY wrapper exits, while preserving fallback signaling for initially untargeted survivors, waiting for cleanup, and force-kill behavior. - Add synchronized process-supervision regressions and native CLI coverage for SIGTERM, SIGHUP, and a second SIGINT after shutdown acknowledgment. Treat child-forwarded interrupts and output after terminal hangup separately from Turborepo's own guarantees. Related to #14043. This hardens signal delivery within Turborepo; it does not change an external package manager's decision to exit before Turborepo finishes shutting down.