turborepo — Engineering Performance

feat: Migrate TUI virtual terminals from vt100 to Ghostty (#13135) <!-- CURSOR_AGENT_PR_BODY_BEGIN --> ### Description Migrates Turborepo's TUI task output panes from the vendored `turborepo-vt100` crate to [Ghostty](https://ghostty.org)'s `libghostty-vt` terminal emulator. **Why Ghostty?** Ghostty's VT library is actively maintained, supports modern terminal features, and is the same engine powering the Ghostty terminal. This replaces the older in-repo vt100 implementation and removes the `tui-term` dependency. #### Architecture - **`turborepo-ghostty-sys`** — In-repo FFI against `libghostty-vt`. `build.rs` fetches a pinned Ghostty commit and compiles via Zig 0.15.2; `bindings.rs` is checked in for docs.rs. - **`turborepo-ghostty`** — Safe Rust wrappers (adapted from [libghostty-rs](https://github.com/Uzaaft/libghostty-rs)) plus Turborepo-specific `Parser` and ratatui `TerminalWidget` (adapted from [ratatui-ghostty](https://codeberg.org/jint/ratatui-ghostty)). - **`turborepo-ui`** — Task pane rendering, scroll, and selection updated to use the Ghostty-backed widget. TUI dependencies are gated behind a `tui` feature so lightweight consumers (e.g. `turborepo-telemetry`, `@turbo/repository`) don't pull in Ghostty/Zig. **Removed:** `crates/turborepo-vt100`, `tui-term` dependency. #### Notable fixes included - **Selection/copy** — Ghostty selection grid refs go stale as output streams; viewport endpoints are now persisted and refreshed before render/copy. Selected cells are highlighted during drag. - **`Send` for tokio** — Static Ghostty allocator objects are marked `Send + Sync` so the TUI `App` can be moved into `tokio::task::spawn`. - **CI / Zig** — `setup-zig` detects host arch (`aarch64` vs `x86_64`), uses the correct Windows `.zip`, and sets isolated `ZIG_GLOBAL_CACHE_DIR` to avoid stale linker failures on reused macOS runners. - **`@turbo/repository`** — Gating Ghostty behind `turborepo-ui`'s `tui` feature prevents the NAPI package from building `libghostty-vt` unnecessarily. #### Build requirements Zig 0.15.2+ is now required to build the `turbo` binary (documented in `CONTRIBUTING.md`). CI installs it via `.github/actions/setup-zig`. Optional env vars for local Ghostty development: `GHOSTTY_SOURCE_DIR`, `GHOSTTY_ZIG_SYSTEM_DIR`, `TURBOREPO_GHOSTTY_SYS_OPTIMIZE`. #### Attribution Vendored code is documented in the crate READMEs. Most safe wrappers and FFI patterns come from libghostty-rs; ratatui integration from ratatui-ghostty; `Parser` and Turborepo UI glue are original. ### Testing Instructions 1. Build and run the TUI: `cargo build` then `turbo run <tasks> --ui=tui` in a monorepo with long-running tasks. 2. **Selection/copy** — Click-drag across task output; verify highlight appears and copied text is correct after more output streams in. 3. **Scroll** — Mouse wheel and keyboard scrolling through task scrollback. 4. **Stdin** — Interactive tasks that read from stdin still work in the focused pane. 5. **Resize** — Resize the terminal while the TUI is open; panes should reflow cleanly. 6. **Non-TUI paths** — `turbo run` without `--ui=tui` and `@turbo/repository` tests should work without requiring a Zig build. <!-- CURSOR_AGENT_PR_BODY_END --> --------- Co-authored-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: Anthony Shew <anthonyshew@users.noreply.github.com>

fix: Refactor `turbo watch` (#13423) ## Summary - replace the shared raw filesystem event broadcast with scoped, demand-driven subscriptions for package changes, hashing, outputs, discovery, cookies, devtools, and daemon root monitoring - model inherited and nested Git ignores, global excludes, tracked exceptions, dynamic Git controls, and explicit ignored input/output interests - prune ordinary ignored trees from Linux inotify coverage while preserving explicit coverage, and route non-mutating backends directly after readiness - add scoped recovery for backend rescans/errors and document the watcher architecture Closes #13402 ## Validation - `cargo test -p turborepo-filewatch --lib` focused and platform-independent coverage - Linux all-feature filewatch test binary in Docker: 68/68 passed - `cargo test -p turborepo-lib package_changes_watcher::test` - `cargo clippy -p turborepo-filewatch --all-targets --all-features -- -D warnings` - `cargo clippy -p turborepo-lib -p turborepo-daemon -p turborepo-devtools --all-targets -- -D warnings` - `cargo check -p turborepo-filewatch --target x86_64-pc-windows-gnu` - original Linux reproduction with three repeated 100,000-file bursts: zero package/hash lag, one package build each, and one web-server start ## Notes Native macOS FSEvents stress was limited by local stream exhaustion after repeated test runs; focused FSEvents path/rescan tests and macOS compilation pass.

chore: Add `turborepo-log` crate to improve our logging situation (#12285) ## Summary - Introduces `turborepo-log`, a new crate for structured user-facing log events (warnings, errors, informational output), deliberately separate from `tracing` which remains for developer diagnostics. - The global `Logger` dispatches events to pluggable `LogSink` implementations. Ships with two built-in sinks: `CollectorSink` (in-memory buffer for post-run summaries) and `FileSink` (newline-delimited JSON with optional size limiting). - All user-supplied strings are sanitized against ANSI escape sequences and control characters to prevent terminal injection. ## Why Today, user-facing messages in `turbo run` are scattered across ad-hoc `println!`/`eprintln!` calls and tightly coupled to `turborepo-ui`'s terminal rendering. This makes it hard to capture warnings for post-run summaries, write them to structured log files, or route them to the TUI without touching rendering code. `turborepo-log` sits at the bottom of the dependency graph (no dependency on `turborepo-ui`) and provides a clean boundary: subsystems emit structured events, sinks decide how to present or store them. A future terminal sink in `turborepo-ui` can implement `LogSink` to bridge into the existing rendering pipeline. ## Testing ```sh cargo test -p turborepo-log ``` The crate includes unit tests for all public types and an integration test (`tests/global_init.rs`) that exercises the full global logger lifecycle including lazy handle resolution.

refactor: Split process child module (#13014) ## Why The process child module mixed public process management, platform-specific handle internals, IO, shutdown semantics, and tests in one large file. Splitting those responsibilities reduces context needed to review future process changes. ## What Moves child handle internals, IO helpers, shutdown types and processing, state manager/channel logic, PTY test guard, and tests into sibling modules under `crates/turborepo-process/src/child/` while preserving the public API from `child.rs`. ## How This is intended as a mechanical refactor. Verified with `cargo fmt -p turborepo-process`, targeted child tests, `cargo test -p turborepo-process -- --test-threads=1`, `git diff --check`, and the repository pre-push hook.

docs: layout redesign (#10178) ### Description Newly formatted pages for docs content. ### Testing Instructions There are a great many changes in here that probably aren't useful to review as file changes. You'll want to open the preview and make sure all the design holds together.

fix: Hardening for daemon IPC endpoints (#12742) - Prevent cross-user access to local daemon RPCs by moving daemon IPC into per-user paths and enforcing owner-only socket permissions. - Reject unauthorized Unix peers and validate Windows socket ownership before clients trust a daemon endpoint.

fix: Preserve graceful shutdown output (#12607) ## Summary - Keep signal-driven `turbo run` cleanup alive until task shutdown finishes so TUI and grouped output are not torn down early or truncated. - Route shutdown and cache-flush status through the logging pipeline, distinguish signal vs close shutdowns, and avoid signal-only force timers on normal completion. - Harden process shutdown and parent-death cleanup while adding regression coverage for TUI, process, and signal-handling edge cases.

perf: Overhaul pnpm lockfile parsing and dependency closure computation (#13228) ## Why On large pnpm monorepos, the bulk of `turbo run` startup — the window between invocation and the first task starting — is spent parsing `pnpm-lock.yaml` and computing per-workspace transitive dependency closures. Profiling a 1191-package internal monorepo (43k tracked files, 5.4MB lockfile) showed the closure walk dominated by string allocation, hashing, and map probes rather than actual resolution work, and YAML parsing spending ~75% of its time inside a general-purpose scanner that pnpm's rigid machine-generated output doesn't need. ## What This PR has grown into the full pnpm startup overhaul: five stacked changes, each merged into this branch after independent review and verification. **1. Closure walk interning + FxHash.** Resolved packages are interned to dense `u32` ids behind the existing caches: resolve-cache hits become a refcount bump instead of cloning two `String`s per closure edge, visited sets hash a `u32` instead of two heap strings, and the deps cache is keyed by id. Cache maps use FxHash — lockfile content is developer tooling input, not a hash-flooding vector. **2. pnpm hash index** (#13229). The post-parse `dependency_index` becomes an `FxHashMap` carrying membership flags and versions, making `has_package`/`package_version`/`all_dependencies` O(1) probes instead of `BTreeMap` walks over long keys. Entry dependency maps stay `BTreeMap` so closure iteration order remains deterministic. Also fixes `subgraph()` returning pruned lockfiles with an empty index. **3. Fast pnpm parse on saphyr events** (#13230). An event-driven parser (pure Rust, ~2x libyaml scanner throughput) builds `PnpmLockfile` without serde on the hot path. Deliberately conservative: multi-document input, anchors/aliases, tags, duplicate keys, and exotic numeric forms bail to the unchanged serde path. Observable serde quirks (untagged variant order, `lockfileVersion` float formatting, plain-scalar typing, unknown-field dropping) are mirrored exactly and enforced differentially. **4. Structural line scanner** (#13238). pnpm-lock.yaml is line-oriented machine-generated YAML, so a `memchr`-driven scanner specialized to exactly that subset replaces the general YAML state machine as tier 1, emitting the same event stream the semantic layer from (3) consumes — quirk handling is shared, not duplicated. Tier order: scanner → saphyr → serde; the scanner also declines anything a general parser would *reject*, so it can never fabricate a lockfile from invalid input. Roughly halves parse time again on every corpus measured. **5. Shared bitset closure DP** (#13239). Closures were recomputed per workspace, so shared dependencies were re-walked once per workspace reaching them. When the lockfile can prove every transitive edge resolves identically across workspaces (pnpm importers can shadow transitive resolution, so this is proven per edge via importer equivalence classes, not assumed), closures are computed once globally: iterative Tarjan SCC condensation, then a bottom-up bitset DP where set unions are word-parallel `u64` ORs, processed in bounded-memory chunks. Any sensitive edge, non-pnpm lockfile, or single-workspace call falls back to the unchanged legacy walk. ## Results Interleaved A/B against current `main`, 4-core Linux, telemetry disabled, on the 1191-package monorepo. Time-to-first-task is measured in-process (invocation to first task dispatch in the chrome profile). | Benchmark | main | this PR | delta | |---|---|---|---| | time-to-first-task (real run) | 1117ms | 472ms | **−58%** | | `turbo run build --dry=json` end-to-end | 1481ms | 871ms | **−41%** | | lockfile parse (5.4MB, `from_bytes`) | ~285ms | 56ms | **−80%** | | peak RSS (dry) | — | −27MB vs branch base | bitsets beat per-workspace set churn | Parse deltas hold on public corpora too: next.js lockfile (1.29MB) 30.9→15.0ms, this repo's (707KB) 15.3→8.0ms for the scanner tier alone. ## How to verify - Full `--dry=json` output diffed against current main on the internal benchmark monorepo and this repo — byte-identical modulo run id and version string. - 20 differential parse tests assert scanner == saphyr == serde (`Eq`-identical lockfiles, byte-identical `encode()` round-trips) across v5/v6/v9 fixtures, folded/literal block scalars with chomping/blank-line edges, quoting, flow collections, catalogs, patches, and this repo's own `pnpm-lock.yaml` — with scanner acceptance *required* on mainline shapes so regressions can't silently fall through to slower tiers. Each unsupported construct has explicit fallback coverage. - The closure DP is differentially tested against the legacy walk (an independent oracle — the single-workspace entry point never uses the DP), including a crafted divergent-edge fixture proving the fallback triggers. 220 crate tests pass.

feat: Rebuild the factory image on every merge to main (#13781) ## What Agents in `apps/factory` booted from `vercel/eve:latest` plus a shallow clone. The sandbox had no Rust toolchain, `protoc`, Cap'n Proto, Zig, LLD, pnpm 10.28.0, `node_modules`, or performance tooling, and its Eve `revalidationKey` was hardcoded to `"turborepo-main-v1"`, so the template never rotated. `.devcontainer/Dockerfile` had most of that toolchain but was unused and had drifted three Rust nightlies, two Node majors, and two pnpm majors behind. This adds the factory image: one specification for what an agent's sandbox contains, a snapshot rebuilt by the application on every merge to `main`, and both agent paths booting from it. ## One specification `apps/factory/agent/lib/factory-image.ts` owns the image. It pins versions to the values the repository and CI already use, emits idempotent provisioning phases, and fingerprints the whole thing so a changed pin rebuilds every image. | Tool | Version | Source of truth | | --- | --- | --- | | Rust | `nightly-2026-05-22` + `rustfmt`, `clippy` | `rust-toolchain.toml` | | Node.js | 24 | root `package.json` `engines` | | pnpm | 10.28.0 | root `package.json` `packageManager` | | protoc | 26.1 | `.github/actions/setup-protoc` | | Cap'n Proto | 1.1.0 | `.github/actions/setup-capnproto` | | Zig | 0.15.2 | `.github/actions/setup-zig` | Plus `build-essential`, `pkg-config`, LLD (`.cargo/config.toml` links with `-fuse-ld=lld`), OpenSSL headers, `jq`, `zstd`, the workspace `node_modules`, a warm Cargo registry, and `hyperfine`, `cargo-bloat`, and `twiggy` for the performance skill. A final phase verifies every required tool, warns about missing optional ones, and writes a version manifest. `.devcontainer/Dockerfile` was rewritten against the same pins, and a test fails when either drifts from `rust-toolchain.toml`, the root `package.json`, or the CI actions — so the local dev container and the agents' sandbox stay on one toolchain. ## Rebuilt on every merge, without GitHub Actions A push to `main` reaches `POST /api/github/push`, which verifies the GitHub HMAC signature and starts a Workflow run. The workflow creates a build sandbox, detaches the provisioning script inside it, polls the markers the script writes, snapshots the result, and publishes the snapshot id to a Blob-backed ledger. No step holds a function invocation open for the length of a build. When a published image already exists for the same toolchain the build boots from it, so a merge build only fast-forwards the checkout, refreshes dependencies, and recompiles. ## Rapid merges Resolved in the ledger rather than by racing: - Claiming a build cancels every build still in flight, records which build replaced it, stops its workflow run, and deletes its sandbox. - Every step re-reads the ledger before acting; a build that has lost can neither report progress nor publish a pointer. - Redelivered webhooks deduplicate onto the live build, a revision that is already published is skipped, and a build that stops reporting progress for 15 minutes is replaced instead of wedging its revision. ## Consumers - **Eve** (`agent/sandbox.ts`) provisions its template from the same phases and boots from the published snapshot when one matches. Eve freezes `revalidationKey` at build time, so the template rotates when the fingerprint changes or a newer image is published; each session then fast-forwards its checkout to the current `main`. - **Harness** (`agent/lib/harness-agent.ts`) uses the snapshot as its sandbox source instead of a stock `node24` runtime plus a clone, rotates its template with the image, and falls back to cloning when no image matches this deployment's toolchain. - The **operator page** shows the published snapshot, the toolchain fingerprint, warnings, and recent builds, and can rebuild from the current `main` head. ## Validation Every provisioning phase was run against a real `vercel/eve:latest` sandbox. All eleven phases pass through verification in about two minutes, with no warnings: ``` system-packages 10s · node 1s · pnpm 2s · rust 11s · protoc <1s · zig 5s checkout 2s · node-modules 12s · cargo-registry 7s · performance-tools 45s · verify 1s {"capnp":"Cap'n Proto version 1.1.0","node":"v24.17.0","pnpm":"10.28.0", "protoc":"libprotoc 26.1","rustc":"rustc 1.97.0-nightly (e96c36b6f 2026-05-21)", "zig":"0.15.2"} ``` That run also confirmed the image is Ubuntu 26.04 with `apt`, that commands run as root, and that `ld.lld`, `hyperfine`, `cargo-bloat`, and `twiggy` all land on `PATH`. 45 unit tests pass (`pnpm test`), covering the pinned versions against the files they mirror, the phase list and generated script, revision validation, progress parsing, the fingerprint, the ledger's supersede and publish rules, and webhook signature and event filtering. `eve build`, `next build`, `tsc --noEmit`, `oxlint --deny-warnings`, and `oxfmt --check` are clean. ## Deployment notes - Requires the private Vercel Blob store the run registry already uses. - Set `FACTORY_IMAGE_WEBHOOK_SECRET` (falls back to `GITHUB_WEBHOOK_SECRET`) and deliver `push` to `/api/github/push`. Deployment Protection covers that path, so append the automation bypass token as a query parameter; the HMAC signature authenticates the delivery. - The first build after a toolchain change provisions the Eve template during the Vercel build, because Eve prewarms templates there. The phases that compile Rust are wrapped in timeouts so one bad upstream release cannot hold a deployment build open, and only the merge webhook asks for the warm `cargo build`. - Performance-tool installation failures are recorded as warnings rather than failing a build, so a broken upstream crate cannot stop an otherwise complete image from publishing. The warnings surface on the operator page. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: vercel[bot] <35613825+vercel[bot]@users.noreply.github.com>

chore: Extract turbo_json module to turborepo-turbo-json crate (#11344) ## Summary Extract ~5,400 lines of turbo.json parsing, validation, and processing code from `turborepo-lib` into a new `turborepo-turbo-json` crate. This continues the modularization effort to make `turborepo-lib` a thin orchestration layer. ## What's extracted | File | Lines | Purpose | |------|-------|---------| | `lib.rs` | ~300 | Main `TurboJson` type, constants, core API | | `raw.rs` | ~320 | Raw parsed types (`RawTurboJson`, `RawTaskDefinition`, etc.) | | `parser.rs` | ~340 | Biome-based JSON parsing | | `processed.rs` | ~620 | Processed task definitions with DSL token handling | | `extend.rs` | ~610 | Task inheritance/extension logic | | `validator.rs` | ~250 | turbo.json validation rules | | `future_flags.rs` | ~40 | Feature flags | | `error.rs` | ~320 | Error types with miette diagnostics | ## What stays in turborepo-lib - `TurboJsonLoader` - Depends on MFE (microfrontends) and task_access - `TurboJsonReader` - Reading turbo.json from disk - `TaskDefinitionFromProcessed` trait - Creates `TaskDefinition` from processed types - Re-exports for backward compatibility ## Verification - ✅ All 339 `turborepo-lib` tests pass - ✅ All 57 `turborepo-turbo-json` tests pass - ✅ All 72 engine tests pass - ✅ Binary tested with `examples/basic` - ✅ **Zero behavioral changes** for CLI users (error messages, validation, parsing all identical) ## Test Plan ```bash cargo test -p turborepo-turbo-json cargo test -p turborepo-lib cargo build --bin turbo ```