Luke Sandberg

Remove TraceRawVcs (#98953) ## What? Remove the repository-wide `TraceRawVcs` trait, implementations, bounds, and derive support. Persistent-to-transient task errors now report the parent and callee names without recursively inspecting task inputs. The existing field-assertion escape hatch is renamed from `trace_ignore` to `unsafe_ignore`. It remains available to `NonLocalValue` and `OperationValue`, making its remaining purpose explicit without retaining a connection to tracing. The stripped release `next-napi-bindings` binary decreases by 307,648 bytes (0.301%), from 102,325,280 to 102,017,632 bytes. ## Why? `TraceRawVcs` was introduced for a garbage-collection design that is no longer being pursued. Its only remaining runtime benefit was a more detailed version of an extremely rare diagnostic, which did not justify maintaining tracing implementations across the Rust workspace. ## How? The tracing module and proc macro are removed along with generated and manual implementations. APIs that previously required tracing now keep only the bounds they actively need. The backend diagnostic retains actionable task names while dropping its recursive cause graph, and the transient-collectible coverage is updated for the simplified message. ## Verification - `cargo fmt --all -- --check` - `cargo check --workspace --all-targets` - `cargo test -p turbo-tasks-backend --test transient_collectible` - `cargo test -p turbo-tasks-macros-tests` - `cargo test --doc -p turbo-tasks` <!-- NEXT_JS_LLM --> <!-- fleet 6de62cf9-27cd-4f08-8fce-7d495c71e7e8 --> Co-authored-by: vercel-fleet-prod[bot] <318278635+vercel-fleet-prod[bot]@users.noreply.github.com> Co-authored-by: Luke Sandberg <210140+lukesandberg@users.noreply.github.com>

turbo-tasks-backend: parent_count-driven garbage collection (#97282) Adds garbage collection, built on the `parent_count` reference counting ### How collection works * we scan all storage shards for collectible tasks to seed the sweep * For each collectible task we 1. mark it as deleted 2. remove all outgoing edges 3. queue new tasks for deletion if the edge removals triggered it For removing edges we rely on the existing `CleanupOldEdges` operation, though it is enhanced to collect the tasks that become collectible as edges are removed Deletion markers are transformed into tombstones by persistence and then dropped from memory by eviction. ### Coordinator changes GC mutates the graph, so we exclude other tasks from running while it works. When gc completes we hand off to persistence which switches to the existing copy on write mode. The subtle piece is `GcPhase::into_snapshot`: GC has to hand its exclusion directly to the snapshot that tombstones what it collected. Swapping the flags under one lock leaves no window where an operation could start and resurrect a just-collected task. ### Enabling it Off by default; `TURBO_ENGINE_GC` turns it on, also requires eviction to be enabled (otherwise deleted tasks persist in RAM) `prevent_gc()` becomes real, pinning the calling task through `transient_ref_count`.

Store code generation AST paths in a shared trie (#98581) ### What? Store the AST paths that code generation uses to locate nodes in a per-module trie, referenced by a 4-byte id, instead of a `Vec<AstParentKind>` per path. ### Why? Ast paths can share long prefixes leading to quadratic behavior (in pathological cases), even in non pathological cases there can be redundancy. Peak memory on a repro of 180 files, each with a 1200-term chain: | | peak | compile | |---|---|---| | before | 24.0 GB | 19.1 / 17.4 / 16.5s | | after | 4.0 GB | 13.7 / 14.2 / 14.8s | And on an ordinary app (`vercel-site`, 5,669 files; 5 cold runs + warmup): | | baseline | this branch | |---|---|---| | user time | 321.00 s ±0.8% | 321.30 s ±0.8% | | MaxRSS | 14,984 MB ±0.4% | 14,755 MB ±0.4% | ### How? The trie is built lazily, interning only the paths code generation asks for, so it stays sparse. A builder holds the child index during analysis and is frozen into an immutable trie stored beside the code gens. `ApplyVisitors` previously sorted the full paths and binary searched them at every level; it now descends the trie alongside the AST walk. Consumers that manipulated paths structurally walk it instead — `EsmBinding` steps to a parent rather than popping, and `removal.rs` and `code_gen::path_to` trim from the end without allocating. Interning was profiled separately. Sorting paths first, embedding the child map in each node, resuming interning from the previous path, and moving `Effect`'s paths into a second trie were all measured and rejected: each was slower, needed more memory, or was inside the noise. Interning is not a hot path. Ideally we would also intern the paths stored in the temporary `Effect` enum, but this is a pure time/memory tradeoff. We can massively reduce pressure on the arena in exchange for a 10% perf regression. Not worth it. <!-- NEXT_JS_LLM --> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>

[turbopack] Optimize the implementation of AutoMap/AutoSet (#95694) Optimize the implementation of AutoMap/AutoSet. Previously the `List` variant was backed by a `SmallVec` in inline mode, which needs at least 24 bytes of header (`len`, `cap`, `ptr` — all `usize`) even though the list never holds more than 32 elements. This replaces it with the `TinyVec` struct from turbo-tasks-backend which is now enhanced with support for an `inline` array. The length is a `NonZeroU8`, which reserves a niche the `AutoMap` enum folds its discriminant into — dropping `AutoMap`'s minimum size to 16 bytes. This in turn shrinks `TaskStorage` and the `LazyField` enum. Some inline structs in TaskStorage are increased in size so we keep the 128 byte footprint

turbo-tasks: explicit GC root anchoring + cross-session orphan reclamation (#96857) ### What? Reworks how the reference-counting GC decides what is a **root** (a task that must not be collected), and adds a durable path to reclaim cross-session orphaned roots Currently GC roots are persisted and the only way to get collected is for another session to revive them, and then for their ref-counts to drop. This provides a new mechanism. 1. **In-session** — Top level operations get reference counted so they can be dropped in session. 2. **Cross-session** — live roots are recorded along with a simple aging mechanism so they can still be collected even if a later session never restores it. ### Why? This closes the final GC gap. Top level operations are ambiguous currently and need tracking. With cross session aging we close a basic gap where if you delete a route between sessions we never clean up the tasks. Now it will eventually age out. ### How? **Roots are anchored explicitly**. Via pins and RAII guards like `GcRoot` **Persisted roots set + TTL age-out** A new Infra-keyspace `GcRoots` entry persists `Vec<(TaskId, ttl)>`. That allows us to detect when roots are stale and can be dropped. **Dispose transient root tasks on drop.** `RootTask::Drop` now disposes tasks and cleans up references allowing the GC to work

turbo-persistence: add key-value tombstones for MultiValue families (#96929) Add a new `tombstone` format to the persistence layer so we can delete key-value pairs out of MultiValued tables This is in service of the upcoming GC support, but also fills a basic API gap in the db. To delete a key-value-pair you need to call `value_delete` and currently the values are limited to only those that are able to be stored inline in key blocks. This is a non-trivial limitation but it fits our current usecase, and makes the compaction/query algorithms a bit simpler (don't need to 'resolve' values) One non-trivial complexity was maintaining the 'FixedLayout' block optimization for a mix of tombstones and values, so now we support a slightly different mode where all keys have the same length but possibly different types. Finally, this branch solves a problem with deleting tombstones. Tombstones 'shadow' older values and allow us to drop them during compaction. With GC getting ready to start writing tombstones the risk becomes 'when can we delete a tombstone! This is solved probabilistically using the amqf filters, during compaction we drop tombstones if they could not possibly match anything in an older SST. Without this, tombstones in the TaskCache table would fill up over time.

Reduce Turbopack cache size with per-family compression (#97714) ## What? Adds per-family compression configuration to `turbo-persistence` and configures the Turbopack filesystem cache according to each keyspace's access pattern: | Family | Compression | Reason | | --- | --- | --- | | Infra | LZ4 HC level 4 | Preserve LZ4 decode performance while improving write-time compression | | TaskMeta | LZ4 HC level 4 | Preserve latency-sensitive metadata reads | | TaskCache | LZ4 HC level 4 | Preserve latency-sensitive task-cache reads | | TaskData | zstd level 3 | Prioritize the dominant disk-size opportunity | The low-level configuration exposes only the three presets this database needs: LZ4, LZ4 HC4, and zstd3. Compression is applied consistently to SST blocks, blob values, and compaction output. Each database owns a `ThreadLocal` that lazily creates one lock-free zstd decompressor per participating thread; the linked zstd reports each context as about 96 KiB, and all contexts are released when the database drops. Each meta file stores its family's length-prefixed, bincode-encoded compression preset once. SST and blob headers remain unchanged, including the 12.5% minimum-savings fallback to uncompressed blocks. On normal database open, the stored marker must match the runtime `FamilyConfig`; a mismatch, invalid/truncated preset, or pre-marker meta file is rejected before reads are served. This intentionally requires a fresh/versioned cache after a codec change, consistent with `turbo-persistence`'s existing no-cross-version-compatibility policy. The marker also removes the duplicated keyspace-to-codec mapping from `sst_inspect`: it reads the codec directly from metadata. `turbo-persistence-tools` can inspect metadata without supplying a Turbopack-specific family configuration. ## Why? LZ4 decode performance is important for cache query latency, but using one codec for every keyspace leaves a substantial disk-size opportunity in TaskData. The families do not have the same tradeoff: metadata and task-cache reads should stay on the LZ4 decoder, while TaskData can spend modestly more CPU to reduce persistent cache size. The final three-workload A/B reduced fresh cache directories by **20.61% overall**, with no measured cold/write or warm/read regression (the observed -3.42% / -2.39% should be treated as noise, not a speed claim). ## How? ### Real Next.js cache A/B The complete A/B was re-run after the final fixed-preset, bincode-marker, and database-owned thread-local revision. Both the unchanged base worktree and final proposed working tree were bootstrapped with `pnpm install --frozen-lockfile` and `pnpm build-all`, using locally compiled native bindings on the same 8-vCPU Intel Xeon VM (16.3 GiB RAM, Linux 6.18.40, Node 24.14.1). Each workload used three fresh-cache builds and five warm-cache rebuilds; the table reports medians. ```sh TURBO_ENGINE_IGNORE_DIRTY=1 NEXT_TELEMETRY_DISABLED=1 \ node packages/next/dist/bin/next build <fixture> --turbopack ``` `test/e2e/filesystem-cache` additionally used `ENABLE_CACHING=1`. The other fixtures temporarily enabled `experimental.turbopackFileSystemCacheForBuild`; those fixture changes are not in this PR. | Workload | Cache size | Cold/write | Warm/read | | --- | ---: | ---:| ---: | | `test/e2e/filesystem-cache` | **-20.62%** | -2.24% | -3.00% | | `test/e2e/app-dir/app-rendering` | **-20.11%** | -6.04% | -1.47% | | `test/e2e/app-dir/client-reference-chunking` | **-21.02%** | -1.45% | -3.56% | | Combined medians | **-20.61%** (96.810 → 76.862 MiB) | **-3.42%** | **-2.39%** | Per-family inspection of a final filesystem-cache build, using the marker-derived codec: | Family | Total file size | Value-block savings | | --- | ---: | ---: | | Infra | 52 B | n/a | | TaskMeta | 6.31 MB | 29% | | TaskData | 18.17 MB | 66% | | TaskCache | 1.65 MB | n/a | The originally planned `app` and `app-static` fixtures could not run as standalone builds: `app` hit the existing Edge-runtime `process.cwd()` build failure, and `app-static` required an external data endpoint during page-data collection. They were replaced with the two successful repository fixtures shown above under the same base/proposed methodology. ### Temporary crate benchmark instrumentation The codec-specific Criterion cases were used locally to gather receipts, then removed from the delivered patch per review. | Codec | Synthetic write | vs LZ4 | Uncached get | Cached get | DB size | | --- | ---: | ---: | ---: | ---: | ---: | | LZ4 | 38.173 ms | baseline | 7.3004 µs | 10.735 µs | 21.59 MiB | | LZ4 HC4 | 256.31 ms | **+571.7%** | 3.4853 µs | 8.5467 µs | 21.57 MiB | | zstd3 | 57.790 ms | **+51.4%** | 6.2460 µs | 9.5323 µs | 21.49 MiB | The synthetic workload applies one codec to the entire database and barely separates codecs by size, so it is not representative of the real TaskData distribution. The all-HC write regression is real but confined to smaller families in the actual configuration. Cached reads do not decompress blocks, and the read samples contain substantial ordering/outlier noise. The real-cache A/B is the read-regression guard. ## Vercel Site ### Size 2.9G canary 2.2G this PR a 25% savings ### Performance Cold build (n=5), warm build (n=3), medians: | Scenario | Metric | Canary | Compression | Δ | |---|---|---|---|---| | **Cold** | wall (s) | 71.28 | 72.86 | +2.2% | | | user (s) | 555.21 | 570.23 | +2.7% | | | sys (s) | 94.49 | 95.00 | +0.5% | | | maxRSS (GB) | 17.95 | 17.98 | +0.2% | | **Warm** | wall (s) | 15.00 | 15.32 | +2.1% | | | user (s) | 7.59 | 9.98 | +31% | | | sys (s) | 12.79 | 12.87 | +0.6% | | | maxRSS (GB) | 3.78 | 3.86 | +2.0% | maxRSS converted at 1 GB = 2^30 bytes. Looking at tracing data i see in a cold build `persist` span went from 5.06s duration using 63s cpu time to 6.04s using 73s cpu time, a ~20% regression. Which explains the cpu regression and the time progression. Because the time regression is in the `persist` span during shutdown the added latency is somewhat hidden. Of course we also see a small wall/user time regression in warm builds due to the extra decompression costs for zstd. ### Verification - `cargo test -p turbo-persistence` (79 tests) - `cargo test -p turbo-tasks-backend` (99 unit tests plus integration suites) - `cargo test -p turbo-persistence-tools` - `cargo test -p turbo-persistence --all-targets --no-run` - `cargo fmt --all -- --check` - `cargo clippy -p turbo-persistence -p turbo-persistence-tools -p turbo-tasks-backend --all-targets -- -D warnings` - `cargo check -p turbo-persistence -p turbo-persistence-tools -p turbo-tasks-backend` - `pnpm lint-ast-grep` - `pnpm build-all` <!-- NEXT_JS_LLM --> <!-- fleet 621238c3-5a53-431b-8661-8c5b2920d55d --> --------- Co-authored-by: vercel-fleet-prod[bot] <318278635+vercel-fleet-prod[bot]@users.noreply.github.com> Co-authored-by: Luke Sandberg <210140+lukesandberg@users.noreply.github.com>

fix(watch): recover from slow initial hash instead of timing out (#13159) ## Problem `turbo watch` fails at startup with: ``` × Timed out waiting for the file watcher to become ready. Try running `turbo daemon clean` and retrying. ``` when a large **untracked** file lives in the repo. On macOS the time goes to git-hashing that file: `git status` lists the untracked file and `hash_objects` reads its full contents in the startup hash loop, blowing past the fixed 10s readiness deadline. The error was fatal, and the `turbo daemon clean` advice is misleading since watch mode runs the watcher in-process (no daemon). ## Fix - **Recover instead of failing.** Replace the one-shot 10s wait with a bounded retry loop (10s attempts up to a configurable `TURBO_WATCH_STARTUP_TIMEOUT`, default 120s). Warn on the first stall; only fail after the cap. - **Name the culprit.** A new `SlowestFiles` structure in `turborepo-scm` tracks the slowest-to-hash files **by time, including in-flight ones** (a file still being hashed is the likely cause of a hang). It's recorded inside the `hash_objects` rayon loop and surfaced via `HashWatcher::slowest_files()`. The startup warning/error now names the real file (project-relative, one per line) instead of guessing. - **Align the timeouts.** The package-changes subscriber's inner readiness wait was a hardcoded 5s — shorter than the outer cap, so it could abort before the outer loop reported why. Both now derive from a shared `startup_timeout_secs`. - **Fix the message.** Drop the `turbo daemon clean` advice. ## Real-repo verification Built and run against a real project with a large untracked Turbopack trace artifact: ``` • turbo 2.10.1-canary.1 • Packages in scope: frame, v0chat, web • Running dev in 3 packages • Remote caching disabled WARNING File watcher still initializing after 10s, likely a large file is slowing the initial hash. Slowest files to hash: chat/.next-profiles/trace-turbopack.bin (8.3s, still hashing) web/.next-profiles/trace-turbopack.bin (0.2s) frame/.next-profiles/trace-turbopack.bin (0.2s) Retrying... ``` ...and then startup **succeeded** instead of dying at 10s. The warning correctly pinpoints the blocking file (`trace-turbopack.bin`, still hashing at 8.3s) by hashing time — surfaced precisely because the recorder tracks live entries, not just completed ones. To force the fatal-after-cap path for testing: `TURBO_WATCH_STARTUP_TIMEOUT=1`. ## Tests - `SlowestFiles` unit tests (in-flight ordering, top-N bound, live-before-completed). - `turborepo-filewatch`: `test_large_file_recorded_as_slowest` — drops an 8 MiB **untracked** file into a fixture package, hashes it, asserts it appears in `slowest_files()`. (Committed files are read from the git tree and never go through `hash_objects`, so the file must be untracked — same as the real-world trigger.) - `slowest_files_hint` formatting test in watch.rs (one-per-line, in-flight flagging). - Full suites green: scm, filewatch, package_changes_watcher, watch; `clippy --workspace` clean. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Store keys in key order in SST blocks that omit hashes (#97480) ## What Key blocks for short keys (≤ 32 bytes) don't store a per-entry hash, but were still sorted by `(hash, key)`. That forced every binary-search probe to recompute the entry's xxh3 hash just to compare it — roughly ten hashes per lookup. These blocks now store their entries in **key order**, so the search compares key bytes directly and hashes nothing. All the families `turbo-tasks-backend` produces hit this case, so this is the common path in practice. ## Why this is safe Hash-based routing is unchanged. Files are still assigned by hash, the index block still routes to a key block by hash, and `min_hash`/`max_hash`, the AMQF, and compaction's coverage model all work exactly as before. Thus only the order *within* a hash-less block changes. This adds a small cost to writing, which now must re-sort blocks of keys and a double cost to compaction which now must re-sort blocks in hash order to iterate them, and then sort them back to key order when writing the new file. ## Benchmarks Measured against `canary`, baseline and comparison run back to back on the same machine. **Lookups ** All 16 `static_sorted_file_lookup` configurations improved | entries | hit/uncached | hit/cached | miss/uncached | miss/cached | | --- | --- | --- | --- | --- | | 1 Ki | -30.4% | -24.8% | -29.1% | -29.4% | | 10 Ki | -25.2% | -19.7% | -22.7% | -23.7% | | 100 Ki | -17.3% | -19.3% | -15.2% | -16.8% | | 1000 Ki | -7.0% | -25.7% | -7.0% | -10.9% | **Commits ** Short keys pay for the added per-block sort. | config | change | | --- | --- | | `key_8` 85Ki entries | +3.3% | | `key_8` 853Ki entries | +3.1% | | `key_8` 8.33Mi entries | +8.5% | | `key_32Ki` / `key_4` (large-key configs) | -2.6% to -19.1% | Six of thirteen write configurations came out as noise (p >= 0.05) and are omitted. **Compaction** `StaticSortedFileIter` must yield `(hash, key)` order because `MergeIter` merges on it, so it reorders each hash-less block back into hash order. Plus the additional cost of the commit (above) | config | change | | --- | --- | | 4Mi entries / 8 commits | +6.0% (reproduced at +10.2%) | | 16Mi entries / 8 commits | +7.5% | | 4Mi entries / 32 commits | +7.8% | | 16Mi entries / 32 commits | +5.2% | | 16Mi entries / 128 commits | -9.5% |

turbo-tasks: add scope_unbounded, a scoped execution primitive that allows more work to be discovered (#95974) ## What Adds `scope_unbounded `, a parallel scope to turbo-tasks: jobs run on a shared work-queue, any job may enqueue more, and the pool is drained by the runtime worker threads plus the calling thread until empty. ## Why The garbage collector (later in this stack) needs to process a pool of work that *discovers more work as it runs* — collecting a task cascades into collecting newly-parentless children. A fixed `parallel::for_each` can't express that. `scope_self_feeding` is the general primitive for "parallel pool where jobs spawn jobs". ## Design notes - **Deadlock-safe on thread-limited / contended runtimes.** Helper workers are a pure optimization: the calling thread always makes progress on the shared queue on its own, so the pool completes even when no worker threads are available. - Supports growing/shrinking the set of workers pulling from the queue based on the amount of work available. - Supports a simple mostly lock free way to aggregate data from the tasks - Supports a way for tasks to abort the whole queue, which is important for making GC interruptible ## Testing There are a fair number of new unit tests and i have done some runs under miri