Anthony Shew

chore: Add `turborepo-log` crate to improve our logging situation (#12285) ## Summary - Introduces `turborepo-log`, a new crate for structured user-facing log events (warnings, errors, informational output), deliberately separate from `tracing` which remains for developer diagnostics. - The global `Logger` dispatches events to pluggable `LogSink` implementations. Ships with two built-in sinks: `CollectorSink` (in-memory buffer for post-run summaries) and `FileSink` (newline-delimited JSON with optional size limiting). - All user-supplied strings are sanitized against ANSI escape sequences and control characters to prevent terminal injection. ## Why Today, user-facing messages in `turbo run` are scattered across ad-hoc `println!`/`eprintln!` calls and tightly coupled to `turborepo-ui`'s terminal rendering. This makes it hard to capture warnings for post-run summaries, write them to structured log files, or route them to the TUI without touching rendering code. `turborepo-log` sits at the bottom of the dependency graph (no dependency on `turborepo-ui`) and provides a clean boundary: subsystems emit structured events, sinks decide how to present or store them. A future terminal sink in `turborepo-ui` can implement `LogSink` to bridge into the existing rendering pipeline. ## Testing ```sh cargo test -p turborepo-log ``` The crate includes unit tests for all public types and an integration test (`tests/global_init.rs`) that exercises the full global logger lifecycle including lazy handle resolution.

fix: Hardening for daemon IPC endpoints (#12742) - Prevent cross-user access to local daemon RPCs by moving daemon IPC into per-user paths and enforcing owner-only socket permissions. - Reject unauthorized Unix peers and validate Windows socket ownership before clients trust a daemon endpoint.

fix: Preserve graceful shutdown output (#12607) ## Summary - Keep signal-driven `turbo run` cleanup alive until task shutdown finishes so TUI and grouped output are not torn down early or truncated. - Route shutdown and cache-flush status through the logging pipeline, distinguish signal vs close shutdowns, and avoid signal-only force timers on normal completion. - Harden process shutdown and parent-death cleanup while adding regression coverage for TUI, process, and signal-handling edge cases.

fix: Normalize CRLF line endings in file hashing to match git (#12572) ## Summary When `.gitattributes` marks files as `text` or `text=auto`, git normalizes CRLF→LF in blob objects. After `turbo prune` removes `.git/`, the manual hashing path was hashing raw bytes (with CRLF), producing different hashes than the git path. This caused cache misses when running `turbo run` in pruned output. This PR adds a `crlf` module to `turborepo-scm` that replicates git's CRLF→LF normalization so turbo's file hashes match git's regardless of whether the git or manual (no-git) code path is used. ## What changed - **New `crlf` module** (`crates/turborepo-scm/src/crlf.rs`): Parses `.gitattributes` via `gix-attributes`, resolves `text`/`text=auto`/`-text`/`binary` attributes per-file, and normalizes CRLF→LF during hashing. Uses a `BlobHasher` trait to write the normalization algorithm once across both the gix and sha1 hasher implementations. Single-pass for the common case (no CRLFs), two-pass only when normalization is actually needed. - **`turbo prune` preserves `.gitattributes`**: Added to `ADDITIONAL_FILES` so the manual hashing path in pruned output has the same normalization context. - **`.gitattributes` in global hash**: Changing normalization rules now invalidates all caches. - **Consolidated duplicate code**: Removed parallel `hash_file` (hash_object.rs) and `git_like_hash_file` (manual.rs) implementations. Both paths now route through the unified `BlobHasher` abstraction in `crlf.rs`, eliminating duplicate blob-header construction and file-type validation. Extracted `resolve_or_load` helper to replace 3x copy-pasted attrs-resolution boilerplate. - **CI comments updated**: Clarified that CRLF normalization is driven by `.gitattributes`, not `core.autocrlf`. ## Known limitations - Only root `.gitattributes` is loaded (nested per-directory files are not consulted) - `eol=` attribute is not handled - `core.autocrlf` from git config is not read; normalization is exclusively `.gitattributes`-driven ## Testing The test suite covers: - Ground-truth validation against `git hash-object --path` (with filters) - Cross-implementation agreement between gix and sha1 hashers - Chunk-boundary edge cases (CRLF split across 64KB buffer boundary) - Binary detection (NUL in first 8KB) - Simulated `turbo prune` → manual hash → verify hashes match git path - Dirty file re-hashing matches committed blob OID - `core.autocrlf=true` combined with `text=auto` - Package-scoped `.gitattributes` patterns Fixes #9616 Fixes #5081

fix: Prevent cache archive symlink reads (#12813) ## Summary - Prevent cache archive creation from reading through symlink/reparse-point races before archiving task outputs. - On Unix, resolve archive inputs with fd-relative traversal so symlinked parent directories and final-component symlink swaps are rejected before reads. - On Windows, open final files with reparse-point no-follow behavior and verify the opened handle still resolves under the cache anchor before reading. Linear: TURBO-5574

feat: Add experimental structured logging with `--json` and `--log-file` flags (#12405) ## Summary - Adds `--json` flag to stream NDJSON to stdout and `--log-file [path]` flag to write structured JSON logs to a file, both behind an experimental marker - Supports `TURBO_LOG_FILE` environment variable as an alternative to the CLI flag - Deliberately omits `logFile` from `turbo.json` — structured logging is a per-invocation concern better suited to CLI flags and env vars than checked-in config ## How to test ```bash # NDJSON to stdout turbo run build --json 2>/dev/null | head -5 # File output (default path) turbo run build --log-file cat .turbo/logs/*.json | python3 -m json.tool | head -20 # File output (custom path) turbo run build --log-file=my-log.json # Env var TURBO_LOG_FILE=1 turbo run build # Both modes simultaneously turbo run build --json --log-file=build.json ``` Each JSON entry has `timestamp`, `source`, `level`, and `text` fields. ANSI escape sequences are stripped in file output.

feat: Add `affectedUsingTaskInputs` future flag for task-level `--affected` detection (#12247) ## Summary - Adds `affectedUsingTaskInputs` future flag that changes `--affected` from package-level to task-level granularity - When enabled, only tasks whose declared `inputs` globs match changed files are selected (plus transitive dependents), instead of running all tasks in changed packages - Extracts shared glob matching into `turborepo-types/task_input_matching` for reuse between `turbo run --affected` and `turbo query { affectedTasks }` ## Design The task-level filtering is a post-build step on the single engine, not a separate code path. The normal `--affected` scope resolution determines affected packages and builds the engine as usual. If the future flag is enabled, `Engine::retain_affected_tasks` then filters the engine in place to only the tasks whose inputs match changed files plus their transitive dependents. This means: - One code path (no branching into a separate engine build) - One engine (mutated in place via `retain_affected_tasks`) - `--parallel` naturally works because the filter is re-applied after the parallel engine rebuild ## How it works 1. Normal `--affected` scope resolution determines affected packages 2. Engine is built with those packages (same as before) 3. SCM is queried for changed files between base and head refs 4. Each task's `inputs` globs are checked against changed files, with a compilation cache per unique (package, inputs) pair 5. `Engine::retain_affected_tasks` prunes the engine to only affected tasks + their transitive dependents via reversed DFS Global changes (root `package.json`, `turbo.json`, lockfile, or user-configured `globalDependencies`) short-circuit to running all tasks. ## Testing To verify, enable the flag in `turbo.json` and use tasks with specific `inputs` configurations: ```json { "tasks": { "build": { "dependsOn": ["^build"] }, "test": { "inputs": ["$TURBO_DEFAULT$", "!**/*.md"] }, "typecheck": { "inputs": ["$TURBO_DEFAULT$", "!**/*.md", "!**/*.test.ts"] } }, "futureFlags": { "affectedUsingTaskInputs": true } } ``` Changing a `.md` file should only trigger `build` (not `test` or `typecheck`). Changing a `.test.ts` should trigger `build` and `test` (not `typecheck`). Unit tests cover both `retain_affected_tasks` (8 tests) and `affected_task_ids` (10 tests). Integration tests cover the end-to-end `turbo run --affected --dry=json` path (5 tests). ## Known limitations - The `turbo query { affectedTasks }` path pre-filters files to the package directory, missing cross-package `$TURBO_ROOT$` inputs. TODO added in code for follow-up. --------- Co-authored-by: vercel[bot] <35613825+vercel[bot]@users.noreply.github.com>

test: Add turbo watch tests and refactor file change classification (#12119) ## Summary - Extract `classify_changed_files` as a pure, synchronous function from the async Subscriber polling loop, making the core file-classification logic directly unit-testable without tokio, broadcast channels, or file watchers - Add `MapperFailed` variant to `FileChangeAction` enum so callers can distinguish mapper errors from actual config changes and log at appropriate severity - Deduplicate the rediscovery+reinit pattern (previously copy-pasted 3x) via `rediscover_and_reinit()` + macro - Move `filter_to_watched` earlier in the watch pipeline (before `stop_impacted_tasks`) to avoid unnecessary task-graph traversal for unwatched packages - Add comprehensive unit tests for `classify_changed_files`, `ChangedPackages`, `handle_change_event`, and the full `PackageChangesWatcher` pipeline - Add integration tests for `turbo watch build` using marker files as side-effect detectors ## Why `turbo watch` had zero test coverage. The file-classification logic was embedded deep in an async loop, making it impossible to unit test without spinning up the full daemon infrastructure. The refactor separates decisions from effects, enabling thorough testing of all classification paths (config changes, gitignored files, .git directory filtering, package changes, mapper failures). ## Key changes for reviewers **Production code:** - `package_changes_watcher.rs`: The `classify_changed_files` function is the extracted pure core. `FileChangeAction::MapperFailed` replaces the previous behavior of silently returning `ConfigChanged` on mapper errors. The `expect()` on file watcher paths is replaced with graceful skip+log. `warn!("hashes are the same")` downgraded to `debug!` (expected behavior, not a warning). - `watch.rs`: `filter_to_watched` doc explains why `All` is unfiltered. Comment at call site explains the safety invariant (engine only contains watched-package tasks). **Test infrastructure:** - `ClassifyFixture` struct eliminates ~15 lines of repeated setup per classify test - `WatchGuard` RAII struct ensures child processes are cleaned up even on test panic - Negative assertion tests use a sentinel event pattern instead of timing-dependent 500ms timeouts - `nextest.toml` filter uses regex anchor `/^watch_/` to prevent accidental matches **Integration tests (`watch_test.rs`):** - `watch_initial_run_executes_tasks` — verifies both packages build on startup - `watch_file_change_reruns_affected_package` — verifies file change triggers rebuild (requires git commit because hash watcher uses git-based hashing) - `watch_clean_shutdown_on_sigint` — verifies process exits cleanly on SIGINT (unix only)

fix: Recover Vercel auth tokens across login flows (#12631) - Recover rejected or expired Vercel tokens across `turbo login`, `turbo sso`, `turbo link`, and remote-cache retry paths instead of surfacing raw auth failures or reusing bad tokens - Upgrade reusable legacy/config-backed Vercel tokens into Turbo OAuth state when possible, but fall back to a fresh login when Vercel rejects token exchange for login flows - Reduce duplicate recovery warnings and add regression coverage for refresh, backfill, exchange, and forbidden-response branches

feat: Add `global` configuration key behind `futureFlags.globalConfiguration` (#12399) ## Summary - Adds a `futureFlags.globalConfiguration` flag that moves scattered global settings (`globalDependencies`, `globalEnv`, `ui`, `envMode`, etc.) under a dedicated `global` key in `turbo.json` - Changes the semantics of global file dependencies: `global.inputs` are prepended to every task's `inputs` instead of being folded into the global hash, allowing tasks to opt out of specific global files via negation globs (e.g. `!$TURBO_ROOT$/config.txt`) - Includes validation that rejects mixing the new `global` key with legacy top-level global keys (and vice versa) ## Why Today, `globalDependencies` files affect the global hash, meaning a change to any global file invalidates **every** task's cache — even tasks that explicitly exclude the file via negation globs. This is a common pain point. By treating global files as per-task inputs instead, tasks gain the ability to exclude specific global files, giving users fine-grained control over cache invalidation without losing the "these files matter everywhere" default. ## How to test The integration tests in `crates/turborepo/tests/global_inputs_test.rs` demonstrate the key behavioral difference: 1. `test_global_dependencies_cannot_be_excluded_by_task_inputs` — Proves that with the old `globalDependencies`, changing a global file invalidates all tasks regardless of negation globs 2. `test_global_inputs_can_be_excluded_by_task_inputs` — Proves that with the new `global.inputs`, a task that excludes the file via `!$TURBO_ROOT$/config.txt` gets a cache hit even after the file changes 3. `test_global_inputs_preserves_default_package_file_hashing` — Proves that tasks without explicit `inputs` still hash their package files alongside global inputs Example `turbo.json` using the new format: ```json { "futureFlags": { "globalConfiguration": true }, "global": { "inputs": ["tsconfig.json"], "env": ["CI"], "ui": "tui" }, "tasks": { "build": {} } } ```

fix: Include transitive dependencies in engine graph pruning for affected paths using Task Graph (#12516) ## Summary - Engine graph pruning (`retain_affected_tasks` and `create_engine_for_subgraph`) now includes transitive **dependencies** in addition to transitive dependents. Previously, only downstream consumers were kept — upstream tasks needed for execution (e.g. `lib#build` providing cached outputs to `app#build` via `^build`) were incorrectly pruned. - Extracts shared `reachable_closure` (reverse + forward DFS) and `prune_to_reachable` helpers, eliminating duplication between the two pruning call sites. ## Why Fixes https://github.com/vercel/turborepo/issues/12512 On a cold cache or first run, `--affected` and watch mode could skip upstream dependency tasks whose inputs hadn't changed. The executor needs those tasks in the graph to restore cached outputs before downstream tasks consume them. Without them, builds fail because expected artifacts are missing. ## Testing - Unit tests in `turborepo-engine` updated to assert full dependency chains survive pruning (previously they only checked for dependents). - New regression test `affected_app_retains_lib_dependency` in both unit and integration suites covers the exact scenario from the issue. - Run `cargo test -p turborepo-engine` and `cargo test -p turborepo --test affected_test` to verify.

feat: Add incremental task caching (#12531) > [!NOTE] > This is an experiment and we're not documenting it yet. We want to learn about what this might look like. Use at your own risk. ## Summary Adds incremental cache partitions that persist tool-managed incremental state (e.g. `.tsbuildinfo`, Rust `target/incremental/`) across runs via remote cache. On cache misses, turbo restores prior incremental state before the tool runs, enabling faster re-execution. Gated behind `futureFlags.incrementalTasks`. - New `incremental` field on task definitions in `turbo.json` - `IncrementalTaskCache` in `turborepo-run-cache` handles fetch/upload lifecycle - Cache keys use SHA-256 with length-prefixed encoding (prevents separator collisions) - Sequential fetch (deterministic last-write-wins overlap), concurrent upload - 30s fetch / 60s upload timeouts prevent slow cache from blocking tasks - Graceful degradation: all failures are warnings, never task failures ### Notable design decisions - **Upload on every success**: This might be overdoing it, but we are going to try and find out to see what it looks like. ### How to test 1. Enable the feature flag: `"futureFlags": { "incrementalTasks": true }` in root `turbo.json` 2. Add `incremental` to a task: ```json "check-types": { "incremental": [{ "outputs": ["tsconfig.tsbuildinfo"] }] } ``` 3. Run the task twice — second run on a cache miss should log "incremental state restored" 4. Test with `--force` (skips fetch, still uploads), `--no-cache` (disables entirely), `--remote-only` (skips local file check)

fix: Preserve non-UTF-8 Git path boundaries (#12826) Why: - Valid Unix Git paths may contain non-UTF-8 bytes, and strict UTF-8 parsing at SCM boundaries could make unrelated packages fail to hash. - SCM parsing now keeps unsupported raw Git paths explicit, defers errors until a queried package actually includes one, and avoids falling back to manual hashing for unsupported Git paths.

test: Port 18 more prysk tests to Rust (other/ + lockfile-aware-caching/) (#12062) ## Summary - Ports all 13 remaining `other/` prysk tests and 5 of 6 `lockfile-aware-caching/` tests to Rust. - 61 new test functions across 14 files. Clears out the `other/` subdirectory entirely. - Only `new-package.t` (lockfile-aware-caching) and all 8 `find-turbo/` tests remain as prysk. Highlights: - `affected_test.rs` (9 tests) — the largest port, covering git branching, file/dependency/package.json changes, SCM base/head overrides, merge-base divergence, and `query`/`ls`/`run` affected modes. - `lockfile_aware_caching_test.rs` (5 tests) — parametrized test covering npm, yarn, pnpm, berry, and bun lockfile-aware cache invalidation using `patch` to bump dependencies. - `framework_inference_test.rs` (8 tests) — framework env var inference, TURBO_CI_VENDOR_ENV_KEY exclusion, turbo.json env/globalEnv exclusion patterns. - `config_layering_test.rs` (5 tests) — full config precedence chain: turbo.json < TURBO_ROOT_TURBO_JSON < --root-turbo-json < local config < env < CLI flag. - Adds `setup_lockfile_test` helper and makes `copy_dir_all` public in common/setup.rs. - `new-package.t` left as prysk since it requires running `pnpm install` to update the lockfile.

fix: Support pnpm per-workspace lockfiles in turbo prune (#12067) ## Summary - Fixes `turbo prune` failing with "no workspace found in lockfile" when pnpm is configured with `shared-workspace-lockfile=false` (#3541) - Adds a lockfile-test fixture (`pnpm-per-workspace-lockfile`) that reproduces the scenario ## Problem When pnpm's `.npmrc` has `shared-workspace-lockfile=false`, each workspace gets its own `pnpm-lock.yaml` instead of a single root-level lockfile. Turbo only looked for a root lockfile, so it either couldn't find it or found one that only contained the root's importers. ## Approach 1. **`.npmrc` parsing** — `NpmRc` now reads `shared-workspace-lockfile` 2. **Lockfile merging** — `PnpmLockfile::merge_per_workspace_lockfiles()` takes per-workspace lockfiles, re-keys their `"."` importers to the workspace's relative path (e.g. `apps/web`), and merges all packages/snapshots into a unified lockfile 3. **Discovery** — `PackageManager::read_lockfile()` checks `.npmrc` for pnpm, and when per-workspace lockfiles are detected, discovers workspace dirs via globs and merges their lockfiles 4. **Prune output cleanup** — The pruned output always uses a shared lockfile, so stale per-workspace lockfiles are removed from copied workspace directories and `.npmrc` is rewritten to `shared-workspace-lockfile=true` ## Testing To verify manually, build turbo and run against the new fixture: ``` cargo build # Copy fixture to a temp dir, git init, then: turbo prune web # Should succeed and produce a merged lockfile in out/ ``` Rust unit tests cover `.npmrc` parsing and the lockfile merge logic.

feat: Add `cacheMaxAge` and `cacheMaxSize` for local cache eviction (#12487) ## Summary - Adds `cacheMaxAge` (TTL) and `cacheMaxSize` (LRU) config options to `turbo.json` for automatic local filesystem cache eviction - Also configurable via `TURBO_CACHE_MAX_AGE` and `TURBO_CACHE_MAX_SIZE` environment variables - Both options are opt-in; no behavioral change for existing users ## How it works Eviction runs in a **background thread** at `turbo run` startup (fire-and-forget, doesn't block task scheduling). The two-phase algorithm: 1. **Phase 1 (TTL)**: Removes entries whose `.tar.zst` mtime exceeds `cacheMaxAge` 2. **Phase 2 (LRU)**: If remaining cache exceeds `cacheMaxSize`, evicts oldest entries first ## Config examples ```json { "cacheMaxAge": "7d", "cacheMaxSize": "10GB" } ``` Duration supports: `30s`, `5m`, `24h`, `7d`, `2w`. Size supports: `500MB`, `10GB`, `1TB` (case-insensitive, fractional values like `1.5GB` work). ## Safety - Parsers use checked arithmetic — overflow, NaN, negative, and Infinity inputs are rejected with clear error messages - `.tmp` file cleanup skips files younger than 1 hour to avoid racing concurrent `turbo run` processes - `SystemTime` subtraction uses `checked_sub` to prevent panics with extreme `max_age` values - Eviction is best-effort: individual file removal failures are silently skipped ## Testing To verify: set `TURBO_CACHE_MAX_AGE=1s` or `cacheMaxAge: "1s"` in turbo.json, run two builds with a gap, and confirm stale entries are removed. Eviction results are logged at `info` level.

fix: Resolve pnpm catalog: specifiers to workspace dependencies in task graph (#12185) ## Summary Closes #10785 - pnpm `catalog:` and `catalog:<name>` version specifiers that resolve to workspace packages (e.g., `workspace:*`) were not recognized as internal dependencies during package graph construction, causing missing edges in the task graph - The dependency splitter now reads catalog definitions from `pnpm-workspace.yaml` and resolves `catalog:` specifiers to their actual version strings before deciding whether a dependency is internal or external ## How to test Using the [reproduction repo](https://github.com/jeremymeng/my-turborepo): 1. `git clone https://github.com/jeremymeng/my-turborepo && cd my-turborepo && pnpm install` 2. `turbo build --graph --dry-run` 3. Verify the graph now shows `app-a#build -> pkg-b#build` (previously missing)

fix: Cap rayon thread pool and `block_in_place` in tokio to prevent deadlock potential (#12253) ## Summary - Fixes a deadlock where `turbo run` hangs indefinitely on 96+ core Linux machines due to a rayon global thread pool initialization race condition - Caps the rayon pool at 72 threads, initializes it explicitly before the tokio runtime, and wraps rayon blocking operations in `block_in_place()` so tokio can compensate Addresses #12251 but won't mark as closed until reporter confirms the issue is no longer observed. ## What changed **New crate: `turborepo-rayon-compat`** — shared leaf crate providing: - `MAX_RAYON_THREADS` (72) — conservative cap below the observed ~90-thread failure threshold - `scale_thread_count()` — clamps a CPU count to the safe range - `block_in_place()` — bridges rayon blocking work from tokio async contexts by detecting the runtime flavor and calling `tokio::task::block_in_place` only on multi-thread runtimes **`cli/mod.rs`** — `run()` is now synchronous. It initializes the rayon pool, then manually constructs the tokio runtime. The previous async body lives in `run_main()`. `RAYON_NUM_THREADS` is respected but clamped to the safe maximum so it can't re-enable the deadlock. **Call sites** — Six rayon entry points across `turborepo-lib`, `turborepo-repository`, and `turborepo-run-summary` are wrapped in `block_in_place()` so tokio workers aren't starved during parallel file hashing, package graph construction, or summary generation.

fix: Enforce cache filesystem boundaries (#12743) ## Summary - Keep cache restore and storage anchored to repository/restore boundaries so symlinks and output globs cannot redirect work outside the intended tree. - Preserve supported in-repository symlink cases while rejecting outside-repo outputs and workspace packages earlier. - Document that cached outputs must resolve inside the repository root. ## Testing - `cargo test -p turborepo-cache restore_ --lib` - `cargo test -p turborepo-run-cache save_outputs_ --lib` - `cargo test -p turborepo-lib can_cache_ --lib` - `cargo test -p turbopath strip_prefix --lib` - `cargo test -p turborepo-repository symlink_workspace_discovery --lib`

fix: Filter microfrontend proxy environments (#12732) - Prevent microfrontend proxy tasks from inheriting undeclared parent environment variables. - Cover both custom proxy scripts and `@vercel/microfrontends` proxy binaries with regressions for strict env filtering.