Luke Sandberg

turbo-tasks: task-storage memory wins (#93720) ## Summary Four small, independent changes that shrink `TaskStorage` and the data it owns: Recommend reviewing commit-by-commit 1. **`Arc<CachedTaskType>` → `triomphe::Arc<CachedTaskType>`.** `triomphe::Arc` is already a workspace dep used in `ReadRef` / `SharedReference`. `CachedTaskType` never appears in a `Weak<...>`, so we can drop the weak count and the CAS in `drop_slow`. Saves one `usize` per allocation. Migrated via a `CachedTaskTypeArc` newtype so the bincode `Encode`/`Decode` impls don't need to cross the orphan rule. 2. **Niche-encode `CellDependency`.** The `cell_dependencies` / `cell_dependents` sets used to hold `(CellRef, Option<u64>)` tuples — `Option<u64>` cost a full 16 B (8 B discriminant + 8 B value, aligned), making each element 32 B. A `CellDependency` enum with two variants (`All(CellRef)` / `Hash(CellRef, u64)`) lets the layout algorithm reuse the niche on `ValueTypeId` (`NonZero<u16>`) inside `CellRef.cell.type_id` for the variant tag. Element size drops 32 → 24 B; `LazyField` from 56 → 48 B. The same enum backs both forward and reverse edges — for `cell_dependents` we re-point `CellRef.task` at the dependent task. Added `CellDependency::into_parts()` and use it in `iter_cell_dependents` / `iter_cell_dependencies` hot loops so the discriminant is checked once instead of twice via back-to-back `cell_ref()` + `key()` calls. 3. **`TaskStorage::lazy: Vec<LazyField>` → `TinyVec<LazyField>`.** The lazy vec only ever holds ~25 elements (one per declared lazy field in the schema). Swapping `Vec`'s 24 B `(ptr, len, cap)` header for `(ptr, len: u8, cap: u8)` + 6 B padding gives 16 B. Drops `size_of::<TaskStorage>()` from 136 → 128 B. `TinyVec` is hand-rolled so I added a push/iter micro-benchmark to confirm it doesn't lose performance vs std `Vec`. Results below. 4. **Rightsize collections** → Explore the `AutoSet`/`AutoMap` types in storage_schema and ensure each one is maximally sized for its natural alignment. ## Benchmark results ### `next build` on a representative app (15 runs each, M4 Pro, `caffeinate -dimsu nice -n -20`) Fresh same-day baseline against branch: | metric | canary | branch | Δ | 95% CI | significant? | |---|---:|---:|---:|---|:---:| | wall time | 40.83s | 41.12s | +0.7% | [−1.07s, +1.64s] | no | | user time | 282.27s | 283.21s | +0.3% | [−1.02s, +2.89s] | no | | sys time | 69.38s | 71.26s | +2.7% | [−1.54s, +5.32s] | no | | **MaxRSS** | **12.47 GB** | **12.04 GB** | **−3.4%** | **[−0.48 GB, −0.38 GB]** | **yes** | **MaxRSS is the headline.** −0.43 GB on a 12.5 GB working set, with t=−17.86 (every branch run lower than every canary run, CV ≤ 0.6% on both sides). Wall / user / sys are all within noise — this PR is a memory win with no measurable timing impact. ### `TinyVec` vs `Vec` micro-bench (`turbo-tasks/benches/tiny_vec.rs`, 200 samples each) | n | Vec push | TinyVec push | Δ% | Vec iter | TinyVec iter | Δ% | |---:|---:|---:|---:|---:|---:|---:| | 0 | 1.31ns | 894ps | **−31.8%** | 598ps | 596ps | −0.4% | | 1 | 16.92ns | 14.75ns | **−12.9%** | 964ps | 952ps | −1.2% | | 4 | 17.93ns | 15.93ns | **−11.1%** | 1.49ns | 1.50ns | +0.5% | | 8 | 63.13ns | 45.24ns | **−28.3%** | 1.97ns | 1.96ns | −0.2% | | 16 | 97.35ns | 79.91ns | **−17.9%** | 3.16ns | 3.14ns | −0.5% | | 24 | 137.41ns | 119.88ns | **−12.8%** | 4.30ns | 4.30ns | +0.0% | TinyVec push is 11–32% faster than Vec push across all realistic sizes; iter is identical. Run with `cargo bench -p turbo-tasks --bench tiny_vec`. ### `task_overhead/turbo` Criterion bench (M4 Pro, `--sample-size 200`) | variant | dur | canary | branch | Δ | significant? | |---|---:|---:|---:|---:|:---:| | turbo-uncached | 1µs | 9.77 µs | 9.68 µs | −1.0% | yes | | turbo-uncached | 1000µs | 1.01 ms | 1.01 ms | −0.1% | yes | | turbo-cached-same-keys | 1µs | 198.6 ns | 191.9 ns | −3.4% | yes | | turbo-cached-same-keys | 100µs | 226.5 ns | 208.1 ns | −8.1% | yes | | turbo-cached-different-keys | 1µs | 233.8 ns | 224.1 ns | −4.2% | yes | | turbo-cached-different-keys | 100µs | 305.3 ns | 246.9 ns | −19.1% | yes | | turbo-uncached-parallel | 10µs | 1.63 µs | 1.54 µs | −5.8% | yes | | turbo-uncached-parallel | 100µs | 8.41 µs | 7.88 µs | −6.3% | yes | <!-- NEXT_JS_LLM_PR -->

[turbopack] Optimize the implementation of AutoMap/AutoSet (#95694) Optimize the implementation of AutoMap/AutoSet. Previously the `List` variant was backed by a `SmallVec` in inline mode, which needs at least 24 bytes of header (`len`, `cap`, `ptr` — all `usize`) even though the list never holds more than 32 elements. This replaces it with the `TinyVec` struct from turbo-tasks-backend which is now enhanced with support for an `inline` array. The length is a `NonZeroU8`, which reserves a niche the `AutoMap` enum folds its discriminant into — dropping `AutoMap`'s minimum size to 16 bytes. This in turn shrinks `TaskStorage` and the `LazyField` enum. Some inline structs in TaskStorage are increased in size so we keep the 128 byte footprint

[turbopack] Enable Effects to be evicted (#94173) ### What? Enable Effects to be evicted. Currently `EffectInstance` transitively holds a `ReadRef<PersistentFileContent>` which means eviction cannot not free the memory for the outputs. Because the `ReadRefs` are captured by the task producing the `Effect` we cannot just drop them (otherwise we could need to somehow re-execute the write to recover them). So instead we need a new 'effect processing phase' Here is our new state machine: * Phase 1: `emit` effects. - At this point we capture the content as a `ResolvedVc` and the hash as a value. - In principle this means the hash and content can get out of sync, but only within an eventual consistency race which callers should be handling. * Phase 2: `take_effects`. - This will call `capture()` on all the effects which will conditionally read the content cell if the hash doesn't match the `EffectStateStorage` * Phase 3: `Effects::apply`. - This will apply the effects for all effects whose hashes don't match storage. - At this point we may discover that a hash in storage changed between `capture` and `apply` which will trigger a special `Retry` error and `invalidate` `Phase 2`. To facilitate the `Retry` logic a `read_strongly_consistent_and_apply_effects` helper function handles it. Which allows for a simple circuit breaker and some verbose warnings/errors when we retry too many times. Specifically: - Split the `Effect` trait into `Effect` (emit-side, returns a `Captured` payload) and `CapturedEffect` (apply-side). ` - Switch `Effects` to `cell = "new"`. Every producer re-execution allocates a fresh cell value; the prior cell (and its captured payload) drops on overwrite, cascade-releasing upstream `ReadRef<PersistedFileContent>` strong counts. The captured payload also drops naturally when the cell is evicted. - Push the per-key state machine (`Unapplied` / `InProgress` / `Applied { value_hash, result }`, `EventGuard` panic recovery, `Event` listener coordination) down into `EffectStateStorage::run_apply`. `Effects::apply` becomes a thin aggregator over `dyn_apply()` calls. - New `ApplyOutcome<E> { Failed(E), Retry }`. `CapturedEffect::apply` returns this; `Retry` signals "capture elided content but storage diverged before apply" — `Effects::apply` collects across the batch, fires the producer's invalidator once, and surfaces `EffectsError::Retry`. - New `EffectStateStorage::matches_applied(key, hash)`. `WriteEffect::capture` / `WriteLinkEffect::capture` consult this and skip the `ReadRef<PersistedFileContent>` / `ReadRef<LinkContent>` materialization when storage already holds `Applied { matching, Ok(()) }` — avoiding the disk-read + decompression hit on producer re-runs that don't actually change output. - 10 integration tests in `turbopack/crates/turbo-tasks-backend/tests/effects.rs` cover the new lifecycle: duplicate apply, sibling stomp re-apply, repeated apply with unchanged state dedupe, `cell = "new"` producing distinct cells per producer run, capture-skip on storage match, and the capture-skip-then-stomp race that exercises the `Retry` path. ### Why? A memory audit showed `EffectInstance` cells accumulating the majority of the ram in an eviction session #### Capture-time storage check (the perf story) `Effect::capture` may consult `EffectStateStorage::matches_applied` and elide the content materialization when storage already records the target hash. `CapturedWriteEffect.content` becomes `Option<ReadRef<PersistedFileContent>>` — `None` means "capture observed `Applied { matching }`; no `body` to pass to `run_apply`". The apply-side state machine then either dedup-hits (storage still matches → cached `Ok`) or enters the no-body branch and returns `ApplyOutcome::Retry`. Reading shared mutable `EffectStateStorage` from inside a turbo-tasks task is normally suspect (the state isn't a tracked input). It is sound here because the apply-time re-check fires the producer's invalidator on mismatch, turning the otherwise-untracked read into an explicitly-tracked one via the `Retry` pathway. The producer reruns, `capture` sees the new storage state, materializes content, and the next `apply` succeeds. #### `ApplyOutcome::Retry` and the invalidator `Effects::apply` collects `Retry` signals across the parallel batch via `AtomicBool`. `Failed(e)` errors fail-fast. After the batch, if any `Retry` fired (and no `Failed`), the producer's invalidator runs once and `EffectsError::Retry` propagates. Callers re-read the operation; the producer reruns; fresh `capture` either materializes content (storage diverged) or dedup-hits cleanly. #### `turbo-tasks-fs` `WriteEffect::capture` / `WriteLinkEffect::capture` call `matches_applied` first and conditionally `.await?` the content `ResolvedVc`. `CapturedWriteEffect::apply` / `CapturedWriteLinkEffect::apply` dispatch through `run_apply(key, hash, body)`, where `body` is `Some(closure)` iff content is `Some`. `apply_inner` takes the unwrapped `&ReadRef<…>` as a parameter so the option-unwrap is at the dispatch boundary (type-enforced). ### Risk: infinite retry loop **Failure mode.** Two sibling producers A and B contend on the same key with different hashes (e.g. two routes that both want to write `chunks/foo.js` with different content). Each `Retry` from A's apply invalidates A's producer; each `Retry` from B's apply invalidates B's producer. In the worst case, A and B perpetually re-capture, race to apply, stomp each other's storage state, and re-trigger Retry — invalidating themselves on each cycle. **Why we believe it terminates in practice.** A `Retry` only fires when (a) capture observed `Applied { matching }` and elided content, and (b) by apply time storage was stomped to a different hash. Each `Retry` invalidates only its own producer (no cross-producer invalidation), so without external input churn the system reaches a fixed point — whichever producer applies last wins, and subsequent re-runs of the loser's producer materialize content (because storage no longer matches) and apply cleanly. The pathological case requires a real, ongoing race fundamentally triggered by external invalidations (file edits). <!-- NEXT_JS_LLM_PR -->

Turbopack: fix error reporting with crashing webpack loaders (#93926) ### What? When a Turbopack webpack-loader subprocess crashes (e.g. a loader calls `process.exit()`, a native fatal error, or the IPC socket otherwise closes mid-message), the error users see today is: ``` - Execution of <WebpackLoadersProcessedAsset as Asset>::content failed - Execution of WebpackLoadersProcessedAsset::process failed - Execution of evaluate_webpack_loader failed - failed to receive message - reading packet length - unexpected end of file ``` After this PR, the same crash produces: ``` ⨯ ./data/crash.data Error evaluating Node.js code Error: Node.js subprocess crashed while evaluating loaders [/path/to/loaders/crash-loader.js]: failed to receive message Caused by: - Node.js process exited with exit status: 7 - reading packet length - unexpected end of file Debug info: - failed to receive message - Node.js process exited with exit status: 7 Recent process stderr: <whatever the loader wrote to stderr before exiting> - reading packet length - unexpected end of file ``` ### Why? The original message gave no actionable information: no exit code, no captured stdout/stderr, no indication of which loader was running. It also looked like an internal turbopack bug rather than a user-fixable error, and a transient pool failure could cascade into an unrelated "issue formatter crashed while reading the source for a code frame" failure on the way out. ### How? Four orthogonal fixes, plus a regression test: 1. **Capture stdout/stderr on subprocess crash.** `OutputStreamHandler` now keeps a bounded ring buffer (last 100 lines per stream) shared with the owning `NodeJsPoolProcess`. When `NodeJsPoolProcess::recv` fails, the buffers and the child's exit status are attached to the error via `anyhow::Error::context`. 2. **Recover from subprocess crash in `pull_operation`.** Instead of propagating the recv error up through `evaluate_webpack_loader` → `process()` → `Asset::content` (the cascade above), `pull_operation` catches it, synthesizes a `StructuredError` via `evaluate_context.emit_error(...)`, disables process reuse, and returns `Ok(None)`. This mirrors the existing in-band loader-error path, so the asset's existing `FileContent::NotFound` degradation kicks in naturally — `Asset::content` never errors. 3. **Include the loader chain in the error message and issue detail.** `WebpackLoaderContext` gained a `loader_names: Vec<RcStr>` field. A new optional `EvaluateContext::crash_context_prefix()` trait method lets webpack-loader evaluations describe what was being evaluated (\"loaders [a, b, c]\") in the synthesized crash message. `EvaluationIssue` also gained an optional `detail` field for the same chain, surfacing it in `--log-detail` output. PostCSS evaluations are labelled \"postcss\". 4. **Crash-proof the issue formatter.** `PlainSource::from_source` and `IssueSource::into_plain` previously propagated errors from `asset.content()` with `?`. They now degrade to `FileContent::NotFound` (and `range = None`) on read failure, so a future regression in some other code path can never cause the issue reporter itself to crash on top of whatever the user was debugging. ### Tests - Added `test/e2e/app-dir/webpack-loader-errors/loaders/crash-loader.js`: a loader that writes a marker to stderr and calls `process.exit(7)`. - Added an e2e test that fetches `/crash` and asserts the marker, the absence of the internal cascade, the loader name, and the resource name are all present in the CLI output. - All 11 tests in `webpack-loader-errors.test.ts` pass; the 5 Rust `turbopack-node` pool tests still pass. Some snapshot/golden tests for error formatting may need updating in CI since `EvaluationIssue` now emits a non-empty `detail`. <!-- NEXT_JS_LLM_PR -->

[turbopack] Optimize TaskInput implementations (#94202) # What Replace the `derive` macros for TaskInput with a proc macro `[turbo_tasks::task_input]` (and analogous support inside of the `turbo_tasks::value` macro. Introduce a hand written `CloneResolved` future type for TaskInput implementations that are always resolved. # Why This allows us to take advantage of the `NonLocalValue` marker trait. If a type is `NonLocalValue` then `TaskInput` can have trivial `is_resolved` and `resolve_input` implementations. This applies to the vast majority of TaskInputs but the current `derive` implementation always produces a recursive `async` `resolve_input` implementation but it is rarely needed and rustc/llvm do a bad job of optimizing it. ## Code-size impact `next-swc.darwin-arm64.node` — release profile, `pnpm swc-build-native --release`, macOS aarch64. | Metric | canary | this branch | Δ bytes | Δ % | | --- | ---: | ---: | ---: | ---: | | Raw `.node` | 124,494,592 | 124,217,280 | −277,312 | −0.22% | | Stripped (`strip -x`) | 84,294,000 | 84,013,136 | −280,864 | −0.33% | | Stripped + gzip -9 | 29,123,538 | 28,898,920 | −224,618 | −0.77% | ## Performance A similar measurement of performance was neutral | Metric | canary | temporary | Δ | % | |---|---|---|---|---| | wall time (s) | 42.659 ± 1.851 | 42.501 ± 2.028 | −0.158 | −0.4% | | user time (s) | 294.322 ± 1.834 | 293.207 ± 2.070 | −1.115 | −0.4% | | sys time (s) | 74.158 ± 4.197 | 73.543 ± 5.725 | −0.615 | −0.8% | | MaxRSS (MB) | 13650.1 ± 66.0 | 13610.1 ± 57.5 | −40.0 | −0.3% | All deltas are smaller than one standard deviation — statistically indistinguishable from noise on this benchmark, as expected for a small optimization.

fix(watch): recover from slow initial hash instead of timing out (#13159) ## Problem `turbo watch` fails at startup with: ``` × Timed out waiting for the file watcher to become ready. Try running `turbo daemon clean` and retrying. ``` when a large **untracked** file lives in the repo. On macOS the time goes to git-hashing that file: `git status` lists the untracked file and `hash_objects` reads its full contents in the startup hash loop, blowing past the fixed 10s readiness deadline. The error was fatal, and the `turbo daemon clean` advice is misleading since watch mode runs the watcher in-process (no daemon). ## Fix - **Recover instead of failing.** Replace the one-shot 10s wait with a bounded retry loop (10s attempts up to a configurable `TURBO_WATCH_STARTUP_TIMEOUT`, default 120s). Warn on the first stall; only fail after the cap. - **Name the culprit.** A new `SlowestFiles` structure in `turborepo-scm` tracks the slowest-to-hash files **by time, including in-flight ones** (a file still being hashed is the likely cause of a hang). It's recorded inside the `hash_objects` rayon loop and surfaced via `HashWatcher::slowest_files()`. The startup warning/error now names the real file (project-relative, one per line) instead of guessing. - **Align the timeouts.** The package-changes subscriber's inner readiness wait was a hardcoded 5s — shorter than the outer cap, so it could abort before the outer loop reported why. Both now derive from a shared `startup_timeout_secs`. - **Fix the message.** Drop the `turbo daemon clean` advice. ## Real-repo verification Built and run against a real project with a large untracked Turbopack trace artifact: ``` • turbo 2.10.1-canary.1 • Packages in scope: frame, v0chat, web • Running dev in 3 packages • Remote caching disabled WARNING File watcher still initializing after 10s, likely a large file is slowing the initial hash. Slowest files to hash: chat/.next-profiles/trace-turbopack.bin (8.3s, still hashing) web/.next-profiles/trace-turbopack.bin (0.2s) frame/.next-profiles/trace-turbopack.bin (0.2s) Retrying... ``` ...and then startup **succeeded** instead of dying at 10s. The warning correctly pinpoints the blocking file (`trace-turbopack.bin`, still hashing at 8.3s) by hashing time — surfaced precisely because the recorder tracks live entries, not just completed ones. To force the fatal-after-cap path for testing: `TURBO_WATCH_STARTUP_TIMEOUT=1`. ## Tests - `SlowestFiles` unit tests (in-flight ordering, top-N bound, live-before-completed). - `turborepo-filewatch`: `test_large_file_recorded_as_slowest` — drops an 8 MiB **untracked** file into a fixture package, hashes it, asserts it appears in `slowest_files()`. (Committed files are read from the git tree and never go through `hash_objects`, so the file must be untracked — same as the real-world trigger.) - `slowest_files_hint` formatting test in watch.rs (one-per-line, in-flight flagging). - Full suites green: scm, filewatch, package_changes_watcher, watch; `clippy --workspace` clean. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

[turbopack] Fix a potential deadlock in scope_and_block (#95695) ### What? Fix a potential deadlock in `scope_and_block` (the CPU fan-out primitive in `turbopack/crates/turbo-tasks/src/scope.rs`) by routing every job through a single shared work queue, so completion never depends on a spawned worker being scheduled. ### Why? `scope_and_block` runs a batch of jobs across the tokio runtime's worker threads while the calling thread blocks. Previously, jobs at indices `1..=WORKER_TASKS` were handed *exclusively* to freshly `handle.spawn`ed worker tasks and never placed on the shared queue — but the calling thread only drains the queue, so it could not run those jobs itself. Each spawned worker runs synchronous code and parks on a `parking_lot::Condvar` (no `.await`, no `block_in_place`), so once scheduled it holds its runtime core for the whole scope. When the runtime has fewer worker threads than host CPUs, or they are already occupied, those workers may never get a core. Their exclusively-assigned jobs then never run, `remaining_tasks` never reaches 0, and the caller blocks forever. ### How? - **Every job goes on one shared queue.** Spawned helpers are now a pure optimization that pull from the same queue; they are never assigned a dedicated job. The calling thread drains the whole queue itself in `end_and_help_complete`, so liveness never depends on a helper being scheduled. - **Runtime-accurate helper cap.** The helper count is `num_workers().min(number_of_tasks) - 1` (per-scope, from `Handle::current().metrics()`) instead of a process-global host-CPU constant. - **Close via a flag, not a sentinel.** The queue carries a `closed` bit guarded by the same lock as the jobs. `end_and_help_complete` sets it and `notify_all`s once; a drainer that finds the queue empty exits when closed or parks otherwise. This replaces the previous `End` token that had to be ping-ponged across drainers. - **Wakeup correctness/perf.** `pick_job_from_work_queue` hands off a surplus `notify_one` when work remains (parking_lot notifications are not latched), and the enqueue-time `notify_one` re-wakes a parked helper — the bootstrap wakeup the hand-off cannot provide from a fully-parked state, which matters most on thread-limited runtimes. - The `VecDeque` is presized to the job count so `push_back` never reallocates under the queue lock. Added a deterministic regression test (`test_scope_worker_threads_occupied`): it pins every runtime worker thread with a synchronous sleep, runs the scope on `spawn_blocking`, and asserts it completes well before the sleep releases. This fails (cleanly, via timeout) before the fix and passes after. Closes NEXT-

switch our test benchmark runs to release-with-assertions (#94538) In ci when testing our benchmarks use our test profile `release-with-assertions` This avoids an expensive `lto` step and ensures that our tests run with debug asserts. I noticed that our workflow for 'test cargo benches' was very slow and a lot of that time was the build presumably due to lto overheads across so many benchmark binaries. Compare: * cargo benches for this pr: https://github.com/vercel/next.js/runs/79974366863?pr=94538 - build=4m42s - test=1m46s * cargo benches before this pr: https://github.com/vercel/next.js/actions/runs/27097792020/job/79973082715#logs - build=8m23s - test=6m56s Of course doing this revealed a few things * a birthday paradox panic in one of the persistence tests * some 'top level read' errors to address in the benchmarks and in the turbopack-cli server used for some benchmarks. And finally, that the persistence tests are just extremely slow in test mode due to their initialization overhead (populating >1GB dbs), so that harness is rewritten to remove some variants and also to reduce max sizes in test mode

[turbopack] Strip leading BOM before parsing CSS (#96678) Fork PR #96379 by @lazerg, re-opened as a branch PR so the "when deployed" CI jobs can run — those require Vercel deployment secrets that GitHub does not expose to pull requests from forks, so they can never pass on the original. **The fix commits are unchanged and still authored by @lazerg.** This PR only adds tests on top. Please credit them; #96379 should be closed in favor of this one. ### What? A CSS file beginning with a UTF-8 BOM (`EF BB BF`) is mishandled by Turbopack. Lightning CSS does not skip the BOM, so it is tokenized as content and the first token is misparsed: ``` ./app/bom.css:1:2 Error: Parsing CSS source code failed Unexpected token AtKeyword("layer") ``` The user-visible symptom is broader than a failed build. Turbopack parses with `error_recovery: true`, and under that setting a leading BOM makes Lightning CSS return `Ok` with **zero rules** — so a BOM-prefixed stylesheet could silently drop all of its styles instead of erroring. dart-sass (compressed style) and PostCSS >= 8.5.24 both emit or round-trip such a BOM, so real projects hit this. Fixes #96374 ### How? Strip a leading `U+FEFF` in `parse_css_stylesheet` before handing the source to Lightning CSS, covering both `StyleSheet::parse` call sites while leaving `ParseCssResult.code` as the original bytes that code frames are rendered from. That split makes parser positions relative to the stripped copy while code frames still render the original line, so first-line positions need compensating. `source_pos_for_loc` adds the stripped character back for line 0 of BOM files. Only line 0 is affected, because the BOM contains no newline. ### Tests `test/e2e/app-dir/css-bom` — a BOM-prefixed stylesheet compiles and its rules reach the page. Verified failing without the fix with the exact error above, and passing with it, in dev-turbo, start-turbo and start-webpack. `test/development/app-dir/css-bom-code-frame` — covers the position correction. Two fixtures hold the same invalid `@media (min-width: {})` on line 1 and differ only by the leading BOM; the test asserts the BOM file's reported column is exactly one greater: | | `no-bom` | `bom` | | |---|---|---|---| | without `source_pos_for_loc` | 18 | 18 | fails | | with it | 18 | 19 | passes | Asserting the relationship rather than a literal column keeps this robust if Lightning CSS changes its absolute column convention. It is kept separate from the e2e suite because the fixtures are intentionally invalid CSS, and scoped to Turbopack in dev, where the warning reaches the CLI as the page is requested. --------- Co-authored-by: lazerg <lazerg2@gmail.com> Co-authored-by: vercel-gh-bot-3[bot] <282332853+vercel-gh-bot-3[bot]@users.noreply.github.com>

[turbopack] Don't evict when there is little memory to save (#95213) ### What? Adds a new `'auto'` mode to the experimental `turbopackMemoryEviction` config option and makes it the default. In `'auto'` mode, Turbopack only evicts in-memory cache after a snapshot once enough memory has been allocated since the last eviction to make the work worthwhile. The option now accepts three values: - `false`: never evict. - `'auto'` (new default): evict after a snapshot only once a memory threshold has been crossed. - `'full'`: evict all evictable data after every snapshot (the previous behavior). ### Why? When the persistent (FileSystem) cache is enabled, Turbopack snapshots its in-memory state to disk and can then evict those in-memory copies to reclaim memory, reloading them from disk on demand. Previously, with eviction enabled (`'full'`), we evicted after *every* snapshot. This is too aggressive: tasks get restored from disk and then immediately evicted again, cycle after cycle, wasting work for little memory benefit. `'auto'` mirrors the existing persistence-threshold model (we already skip a snapshot when too little compilation time has accumulated to justify its cost). Here the proxy is memory instead of time: it isn't worth paying the restore-then-re-evict churn to reclaim a small amount of memory. This was motivated by an example in v0.app where the client would poll the server every 5 seconds leading to a pathological behavior * client poll -> * next.js ensurePage -> * turbopack writeEndpointToDisk * recompute and restore settings for that endpoint (3-10ms of io work) * realize there is nothing to do * respond to client * 2 seconds later..... persist and evict everything saving 0.5M of ram (100ms of work! writes out 2 SSTs and then compacts them!) * 3 seconds after that restart the loop This is silly, #95137 will prevent the persistence loop from occurring , this PR will also just skip the 'restore and recompute' work using a similar strategy. ### How? We can't measure exactly how much memory a sweep would reclaim, so we use `TurboMalloc::memory_usage()` (process-global net live bytes) as a proxy. In `'auto'` mode an eviction sweep runs only once the net bytes allocated since the last eviction exceed a threshold (default 128 MiB, overridable via `TURBO_ENGINE_EVICT_MIN_BYTES`). The first eviction after startup always runs. The threshold scales down under OS memory pressure (`TurboMalloc::memory_pressure()`) so we evict more eagerly when memory is tight. **Backend (`turbo-tasks-backend`):** - `BackendOptions.evict_after_snapshot: bool` → `eviction_mode: EvictionMode` (`Off` / `Full` / `Auto`). - New `EvictionControl` type owns the policy: the mode plus the threshold bookkeeping. The background snapshot loop calls `should_evict(snapshot_had_new_data)` once per cycle and `record_eviction()` after a sweep, so the loop no longer branches on the mode. `'full'` and `false` behave exactly as before. <!-- NEXT_JS_LLM_PR -->

Simplify turbo-tasks-backend: collapse single-impl traits and remove redundant Arc layers (#93983) ### What? Simplifies `turbo-tasks-backend` by removing abstraction layers that each had a single implementation, plus several redundant `Arc` indirections that were no longer load-bearing. **Traits removed / collapsed (each had exactly one implementor):** - **`KeyValueDatabase`** — folded into inherent methods on its only impl, `TurboKeyValueDatabase`. - **`BackingStorage` + `BackingStorageSealed`** — folded into inherent methods on the renamed concrete type `TurboBackingStorage` (was the `KeyValueDatabaseBackingStorage<T>` generic + a type alias). - **`ConcurrentWriteBatch`** — folded into inherent methods on `TurboWriteBatch` (it existed to support an old backend implementation that's gone). - The **`B: BackingStorage` generic parameter** is dropped from `TurboTasksBackend`, `TurboTasksBackendInner`, `ExecuteContextImpl`, and `ChildExecuteContextImpl`. Every instantiation in the workspace already resolved to one concrete type. The only remaining trait in the crate's public role is `Backend` (defined in `turbo-tasks`), which stays — removing it would invert the `turbo-tasks` → backend dependency. **`Arc` indirections removed:** - `TurboKeyValueDatabase.db: Arc<TurboPersistence>` → `TurboPersistence` (the write batch only borrowed it; the clone in `new` was gratuitous). - The `self: &Arc<Self>` receivers and `self.clone()` calls in `run_backend_job`, `idle_start`, and `try_read_task_output` — the deferred/background work reaches the backend through the pinned `turbo_tasks` handle instead, whose `Arc` already keeps the backend alive. - With no remaining sharer, the backend's own `Arc<TurboTasksBackendInner>` and the `TurboTasksBackend` newtype were collapsed: the inner fields now live directly in `TurboTasksBackend`, stored inline in `TurboTasks` (which already shares it via its own `Arc<TurboTasks>`). **Other cleanups surfaced by the above:** - `noop_backing_storage()` is now an empty, read-only instance of the real `TurboPersistence` instead of a separate `NoopKvDb` impl — same concrete type as `turbo_backing_storage()`, so the `Either<…>` wrappers in napi and turbopack-cli go away. - `TurboTasksBackend::backing_storage()` accessor replaced with a focused `invalidate_storage(reason_code)` method, so the backend no longer leaks its storage object. - Dead code removed (`CellDependency::key`, `ExecuteContext::schedule` / `suspending_requested`, the non-stats `execute_with_stats` variant) and feature/test-only items gated behind their corresponding `cfg`. ### Why? The no-op storage work removed the last thing forcing these abstractions to be generic/dynamic, so the single-impl traits and the extra `Arc` layers were pure indirection. This is a self-contained simplification: behavior is unchanged. ### Tradeoff - One `Either`/match in every backing-storage method becomes one `is_empty` atomic load (`Relaxed` `AtomicBool`) in `should_restore` — essentially equivalent. - The backend is now stored inline in `TurboTasks` rather than behind a pointer: one fewer indirection on every backend access, at the cost of a larger `TurboTasks` struct. <!-- NEXT_JS_LLM_PR -->

Turbopack: don't strip async-module runtime from shared runtime chunks (#96599) ### What? Turbopack could emit a `[turbopack]_runtime.js` without the async-module (top-level await) machinery while chunks written next to it call `__turbopack_context__.a(...)`, failing production builds with `TypeError: __turbopack_context__.a is not a function`. Reported against 16.3 when loading a `postcss.config.js`. Intermittent, production only. ### Why? The runtime chunk is emitted to a fixed path, so every module graph using a chunking context writes the same file. Since #94376 the async-module machinery is dropped when a graph has no async modules — but that's decided per graph. The Next.js node execution context is shared by every build-time JS evaluation (postcss configs, webpack loaders, `next/font/google`), each with its own `ModuleGraph`. A graph with no async modules emits a runtime without `.a` and clobbers the variant the postcss loader needs. The winner is `assets.first()` in `emit_assets`, which depends on emission order — hence the intermittency. ### How? Add `shared_runtime_chunk` to the chunking contexts and set it where one runtime is shared by several graphs, so those always emit the complete runtime. This matches the carve-out that already exists for development: in both cases a single graph can't see everything sharing the runtime. Whole-app server contexts keep the optimization. The browser side tested `RuntimeType::Development` as a proxy for per-page graphs, which only works because `per_page_module_graph` currently *is* `mode == Development`. It now reads the real flag. Also renames `has_async_modules` to `include_async_module_runtime`, since it's now true whenever we can't tell. I audited the rest of `.next/build/` for the same hazard: the runtime chunk (and its `.map`) was the only fixed-path output. Everything else is content-hashed via `AssetIdent::output_name`. ### Testing `test/production/app-dir/turbopack-shared-runtime-async-module` uses a single synchronous loader — enough on its own to produce the stripped runtime. Verified to fail without the fix and pass with it. `cargo test -p turbopack-tests` passes with no snapshot churn. Follow-up, not in this PR: the conflict is silent. `EmitConflictIssue` is `IssueSeverity::Error` but never fired here, because the two runtimes weren't grouped into one `emit_assets` call.

fetch: respect HTTP Cache-Control headers with TTL-based invalidation (#91729) (#93228) ## Summary This is a re-application of the HTTP fetch part of #91729. Stacked on #93227. ### Fix fetch to respect HTTP `Cache-Control` headers Previously, `fetch` results were cached indefinitely, meaning results would never be refreshed (unless the cache was invalidated). Now they are `session_dependent` with a TTL to ensure we respect HTTP cache settings (e.g. Google Fonts with `max-age=86400`). New two-task pattern: - **`fetch_inner`** (NOT `session_dependent`): Performs the HTTP request, grabs an `Invalidator` for itself, and returns the response + invalidator + absolute deadline. Cached across sessions. - **`fetch`** (`network`, `session_dependent`): Reads the cached `fetch_inner` result and spawns a timer to invalidate when the TTL expires. On warm cache restore, `fetch` re-executes (session-dependent), reads the persisted deadline from `fetch_inner`'s cached result, computes remaining TTL, and spawns a timer — no HTTP request unless the TTL has already expired. Mid-session, the timer fires and triggers a re-fetch. Error handling: On fetch failure, `fetch_inner` takes a dependency on `Completion::session_dependent()` so transient errors (network down, DNS failure) are retried on the next session without busy-looping. ## Test Plan 3 new integration tests in `turbo-tasks-fetch/tests/fetch.rs`: - `ttl_invalidates_within_session` — mock server returns `max-age=1`, body changes, verifies re-fetch after TTL - `ttl_invalidates_on_session_restore` — fetches with TTL, stops TT, waits for expiry, warm restores with new TT, verifies re-fetch - `errors_retried_on_session_restore` — server returns 500, stops TT, fixes server, warm restores, verifies success - Existing 6 fetch tests continue to pass <!-- NEXT_JS_LLM_PR -->

fix bad socket file location (#2021) Co-authored-by: JJ Kasper <jj@jjsweb.site>

[turbopack] Remove turbotask functions from `trait ResolveOrigin` (#94324) ### What? Removes the `#[turbo_tasks::function]` annotations from the `ResolveOrigin` trait methods (`origin_path`, `asset_context`, `resolve_options`), turning them into plain synchronous trait methods. Callers obtain the value once via `Vc::into_trait_ref().await?` and then call the methods directly on the resulting `TraitRef`. This mirrors the earlier de-functionification of the `Issue` trait (#92623). ### Why? The `ResolveOrigin` implementations are all trivial — they return a stored field or delegate a single call. | Task (misses, before → after) | Misses removed | | --- | --- | | `ResolveOrigin::resolve_options` (dyn) | 92,608 → 0 | | `EcmascriptModuleAsset::origin_path` | 64,779 → 0 | | `ResolveOriginWithTransition::asset_context` | 52,292 → 0 | | `ResolveOriginWithTransition::origin_path` | 52,292 → 0 | | `EcmascriptModuleAsset::asset_context` | 41,164 → 0 | | `PlainResolveOrigin::{origin_path, asset_context}` | 105 + 105 → 0 | | `EcmascriptCssModule::{origin_path, asset_context}` | 32 + 32 → 0 | | `CssModule::{origin_path, asset_context}` | 19 + 19 → 0 | Net effect in that run: **~303k fewer cache misses** (total misses 6,197,068 → 5,893,545) — The ~1.35M eliminated cached-task *hits* additionally remove the per-call task lookup/scheduling overhead. The caching that genuinely matters is preserved one layer down — `AssetContext::resolve_options` (which `ResolveOrigin::resolve_options` delegated to) is still a cached task; its miss count is unchanged (39,668) since those distinct results were always computed, and it simply absorbs the call volume directly now (hit rate 57% → 92%). ## Build Performance: `ResolveOrigin` optimization Benchmark across 10 runs per branch. | Metric | Base | This branch | Δ | |---|---|---|---| | **Wall time** | 43.302 s | 42.692 s | **−0.61 s (−1.4%)** | | **User time** | 314.249 s | 310.919 s | **−3.33 s (−1.1%)** | | **Sys time** | 84.225 s | 84.618 s | +0.39 s (+0.5%) | | **MaxRSS** | 14106.1 MB | 13883.4 MB | **−222.8 MB (−1.6%)** | ### Notes on significance - **User time** is the cleanest signal: −1.1% with very tight variance (base σ 0.4%, branch σ 0.3%). The drop (~3.3 s) is well outside the noise — a real reduction in CPU work. - **MaxRSS** is down a solid 1.6% (~223 MB) with tight variance on both sides (σ ≈ 0.5–0.6%) — a clear, reproducible memory win. - **Wall time** improves 1.4%, but variance is large (base σ 6.6%, branch σ 4.5%, with one 50 s outlier in the base set), so treat this as directional. It tracks the user-time improvement. - **Sys time** is essentially flat (+0.5%, within its ~10% run-to-run noise) — no meaningful change, as expected for a CPU/allocation optimization. ### Invalidation tradeoff The main behavioral risk is invalidation granularity. Removing the per-method tasks removes their invalidation boundary: a consumer that previously depended only on the `origin_path()` (or `asset_context()`) task output now reads the value off the origin directly, so it is tied to the origin cell rather than to that one method's cached output. In practice this is narrow, because every `ResolveOrigin` implementor is an immutable, content-addressed `#[turbo_tasks::value]` (only `ResolvedVc`/owned fields) — a different field value means a different cell, so "reading the whole value vs. one field" does not by itself widen invalidation. <!-- NEXT_JS_LLM_PR -->

[turbo-tasks] Shrink RawVc to 8 bytes and CellId to 4 bytes (#94792) ### What? Shrink `RawVc` (16 → **8 bytes**) and `CellId` (6 → **4 bytes**), by hand-packing them into `NonZero` integers. ### Why? `RawVc` is the type-erased representation behind every `Vc` / `ResolvedVc` / `OperationVc`, and `CellId` keys every task cell. They live in huge numbers in the cache keys and task storage to track cells. Cutting `RawVc` in half and `CellId` by a third removes hundreds of megabytes of peak RSS on a real, large app. ### How? | Type | Before | After | Representation | |-----------|----------|----------|----------------| | `CellId` | 6 bytes | **4 bytes** | `NonZeroU32`: `ValueTypeId` in the top 10 bits, cell index in the low 22 bits | | `RawVc` | 16 bytes | **8 bytes** | `NonZeroU64`; bit 31 flags `LocalOutput`, and the two task variants are split by whether the high-32-bit `CellId` field is zero | Supporting changes: - **`TaskId` constrained to 31 bits** - **`ValueTypeId` capped at 1023** (10 bits) and **cell index capped at ~4.19M** (22 bits), enforced at the registry and cell-allocation sites. Together those restrictions enable us to preserve a bit to use as a discreminent in RawVc and pack `CellId` into a u32 ### Perf Building vercel-site, 5 runs each. `maxRSS` and `user CPU` are means; `wall` is the median | Condition | Branch | maxRSS | wall (s) | user CPU (s) | |---|---|---|---|---| | No persistence | canary | 13.64 GiB | 41.56 | 305.64 | | No persistence | **this PR** | **13.04 GiB** | 40.72 | 299.52 | | | | **−4.4%** | −2.0% | −2.0% | | Persistence | canary | 17.19 GiB | 55.20 | 466.54 | | Persistence | **this PR** | **16.30 GiB** | 55.12 | 462.33 | | | | **−5.2%** | −0.1% | −0.9% | <!-- NEXT_JS_LLM_PR -->

Stop pinning compiled chunk source on `EcmascriptBuildNodeChunkVersion`. (#93807) ## What Stop pinning compiled chunk source on `EcmascriptBuildNodeChunkVersion`. The struct previously held a `Vec<ReadRef<CodeAndIds>>` for every module in the chunk, even though the HMR update path only needs the bytes for the *changed* subset and the bytes are already on disk. That field was also forcing the upstream `CodeAndIds` / `BatchGroupCodeAndIds` tasks to stay `serialization = "skip"`, so warm restarts had to re-walk every module and re-hash its source to rebuild `entries_hashes`. This change mirrors the browser-side pattern: a new `EcmascriptBuildNodeChunkContentEntries` task lives on the chunk content and holds `ResolvedVc<Code>` + `ResolvedVc<u64>` per module. The version struct shrinks to `{ chunk_path, minify_type, entries_hashes }`, drops `serialization = "skip"`, and switches `chunk_path` from `String` to `RcStr`. `update_ecmascript_node_chunk_content` now resolves entries lazily, only when an added or modified module actually needs its code shipped. ## Why Two wins, both for dev sessions starting against a warm filesystem cache: - **Memory.** The version no longer transitively pins every module's compiled `Rope` in heap — those bytes can stay on disk until HMR actually needs them. - **Warm-restart CPU.** `entries_hashes` is sourced from the per-module `Code::source_code_hash()` task (already cached) and the version itself now round-trips through the persistent cache, so we don't re-hash anything on warm start. The HMR payload shape is unchanged. ## Perf This should speed up warm builds a bit but the major benefit is not recomputing node outputs and keeping them in ram measuring v0 after loading the main route Branch: Cold: 12.3G Warm: 7.34G Canary: Cold 12.3G Warm: 8.5G The trace file confirms the recomputations are gone and the heap measurements confirm we trimming ~1.1g of ram Using the devlow benchmarks i was able to confirm a possible small progression ``` # canary chat dev startup build=warm: root page = 23.96 s (from root page/start) chat dev startup build=warm: root page = 21.21 s (from root page/start) chat dev startup build=warm: root page = 22.70 s (from root page/start) # branch chat dev startup build=warm: root page = 20.94 s (from root page/start) chat dev startup build=warm: root page = 19.43 s (from root page/start) chat dev startup build=warm: root page = 23.49 s (from root page/start) ``` ## Tests I added a new integration test to ensure we don't accidentally regress here. Which confirms that 'clean warm builds' run nothing and 'clean warm dev sessions' just set up HMR session infra <!-- NEXT_JS_LLM_PR -->

Improve the NFT error message and ignore comment handling (#95144) Fixes #95125 in two ways The OP admits that they misread the message in a few ways, but we can fix all of them 1. Be specific about what function might need an annotation in the error message. A user with a nearby `path.join` might think that that is where the annotation belongs instead of on the function with the warning. 2. Be more flexible with how these `ignore` comments are interpreted `fs.readFileSync(path.join(process.cwd(), someVar))` shouldn't require 2 annotations. Handle this by checking for and detecting `turbopackIgnore` comments in all expressions! This will then bubble up through the linker and just cause us to ignore corresponding fs access apis.

[turbopack] Support undoable modifications for many field types on TaskStorage (#95180) ## What Allow some modifications that turn out to be no-ops to be undone on TaskStorage. This saves us some redundant check-then-mutate cycles. This doesn't apply to every field type * `Option<T>` - still read-check-modify * `CounterMap<T>` - updates are unconditional tracks and removals use the undo approach * `AutoSet`/`AutoMap` - mostly uses the new 'undo' approach * `take` as a bulk operation does not, taking an empty set is rare and cheap to check for * `AutoMap::insert` doesn't use the undo approach or even check, we always track a modification since all callers are only inserting unique values anyway A few other incidental cleanups were landed as well. ## Why Avoid redundant hash lookups. ## How Have `track_modification` return a new TrackOutcome that records what modifications were made and then in the undo case we can pass it to a new `undo_track_modification` function which can reverse it. This is a little brittle since both operations need to occur within a single lock transition I considered using a 'lambda' oriented approach but decided against it since it would be awkward for some of the mutators and as all the callers are generated by a macro (well _almost_ all callers) it isn't too risky.

make rcstrs on the heap/static slightly smaller (#93805) ### What? Splits the previously unified `PrehashedString` (which held a `Payload` enum of `String | &'static str`) into two separate types: `StaticPrehashedString { value: &'static str, hash: u64 }` for atoms produced by `rcstr!` / `make_const_prehashed_string`, and `DynamicPrehashedString { value: Box<str>, hash: u64 }` for atoms held in an `Arc`. The static and dynamic paths were already distinguished by the `STATIC_TAG` / `DYNAMIC_TAG` bits in `RcStr`, so the runtime branch on enum discriminant was redundant. ### Why? - **16 bytes saved per heap-allocated `RcStr` value .** Dynamic atoms drop the `String::capacity` field (which was always equal to `len` since the contents are immutable) and because they are stored in a `triomphe::Arc` this drops the Arc payload to 32 bytes instead of 40 which matches a mimalloc bucket (previously we were rounded up to a 48 byte bucket) so we save 16 bytes. - **8 bytes saved per static `RcStr` value.** The linker will optimally align our 24 byte struct. - Removes a layer of dispatch on the hot path (`as_str`, `==`, `Hash`) — typed deref to the correct variant instead of matching on `Payload`. (i.o.w. one 'descreminent' traversal instead of 2) ### How? - `dynamic.rs`: `Payload` enum removed; two structs replace `PrehashedString`. `deref_from` split into `deref_static` and `deref_dynamic`. `restore_arc` returns `Arc<DynamicPrehashedString>`. - `lib.rs`: `as_str` and `into_owned` dispatch on `tag()` (STATIC vs DYNAMIC vs INLINE) rather than `location()`. New `heap_hash_and_str` helper for `PartialEq` and `Hash` to share the static/dynamic branch. `into_owned`'s `try_unwrap` arm uses `String::from(Box<str>)` which reuses the box allocation (still O(1)). - `turbo-rcstr-macros`: emit `::turbo_rcstr::StaticPrehashedString` instead of `::turbo_rcstr::PrehashedString`. - Added a comment on `DynamicPrehashedString` noting the future move to `triomphe::ThinArc` to fold the two heap allocations (Arc header + boxed bytes) into one. Deferred because that change would make `RcStr::from(String)` copy the bytes, invalidating the documented cheap `String -> RcStr -> String` round-trip — wants a separate evaluation. <!-- NEXT_JS_LLM_PR -->