Tobias Koppers

[Turbopack] Add graph-based CSS chunking algorithm behind experimental.cssChunking: "graph" (#93606) ### What? Adds an alternative CSS chunking algorithm to Turbopack, opted into via: ```js // next.config.js module.exports = { experimental: { cssChunking: 'graph', // or, with explicit cost overrides: // cssChunking: { type: 'graph', requestCost: 20_000, moduleFactorCost: 1 }, }, } ``` The new algorithm is **off by default** — Turbopack still uses the existing "loose"/dependencies algorithm unless this flag is set, so this PR is a pure addition for users that don't opt in. While we were here, the `experimental.cssChunking` shape was also generalized so every existing string accepts an object form too: | Value | Bundler | Notes | |---|---|---| | `true` / `'loose'` / `{ type: 'loose' }` | both | default heuristic-based chunking | | `'strict'` / `{ type: 'strict' }` | webpack | unchanged | | `false` | webpack | unchanged (one chunk per CSS module) | | `'graph'` / `{ type: 'graph', requestCost?, moduleFactorCost? }` | Turbopack | new | Cross-bundler combinations are rejected at config-validation time: - `'graph'` with webpack throws. - `'strict'` and `false` with Turbopack throw. ### Why? The existing Turbopack CSS chunker (loose / dependencies) is good at preserving CSS ordering but doesn't share chunks across pages well — every page tends to load its own chunk per CSS module, which scales poorly for apps with many pages and shared component libraries. The new "graph" algorithm models the per-chunk-group CSS ordering as a weighted DAG over modules, then greedily merges adjacent runs in the global topological order whenever the merge reduces total cost. The cost model charges every CSS request and overshipped byte, with two tunable knobs (`requestCost` and `moduleFactorCost`). **Trade-off vs. the loose default.** With the default cost parameters (`requestCost: 20_000`, `moduleFactorCost: 1`) the graph algorithm typically ships **less CSS per chunk group at the cost of more requests** than the loose algorithm. The cost model is tuned to avoid overshipping unrelated CSS into pages that don't need it; on apps where the loose algorithm was already collapsing a lot into one big chunk that some pages didn't actually use, the graph algorithm will split it. Apps that prefer fewer requests can raise `requestCost`; apps that prefer less overshipping can raise `moduleFactorCost`. This is opt-in and Turbopack-only because: - The cost model is sensitive to per-app properties (number of pages, size distribution of CSS modules, …) — keeping it experimental gives us room to tune defaults from real usage. - Webpack already has its own `CssChunkingPlugin` and `'strict'` mode that cover the equivalent design space; we don't want to fork that. ### Performance Measured on `vercel.com` (the full graph algorithm spans `create_graph → make_acyclic → linearize → split_into_chunks → assemble`): - **~3s** end-to-end for the synchronous chunking pipeline on a realistic production input. Implementation choices that matter for that throughput: - Tarjan SCC uses `Vec<u32>` / `Vec<bool>` scratch arrays indexed by `NodeIndex` — no hashing on `indices` / `lowlinks` / `on_stack`. - `make_acyclic` batches multiple cuts per SCC pass by seeding successive short-cycle searches at the previous cut's target, only re-running Tarjan when no further cycle is reachable from the seed. - `find_short_cycle` is a bidirectional Dijkstra over a `BinaryHeap` with predecessor pointers (no path cloning) and skips its refinement loop for trivial 2-cycles. - `split_into_chunks` picks the next merge from a `BinaryHeap` keyed on the cost delta instead of an O(N) linear scan per merge. - `chunk_cost` reads a once-built `module_to_groups` inverse index instead of scanning every chunk group on every call; the GlobalStyle leakage check uses binary search on the inverse index rather than scanning each group's module list. ### How? #### Module layout (`turbopack/crates/turbopack-core/src/module_graph/`) The two algorithms are deliberately split so neither imports from the other: - `style_groups/` — algorithm-neutral output types (`StyleGroups`, `StyleItemInfo`, `make_style_groups`). Both algorithms produce these. - `style_groups_loose/` — the existing ("loose") algorithm plus the shared config types (`StyleGroupsAlgorithm`, `StyleGroupsConfig`, `F32TaskInput`). - `style_groups_graph/` — the new algorithm. Pure Rust, no `Vc`, with `petgraph::DiGraph` plus a thin `SubgraphView` wrapper and a small `ReadonlyGraph` trait that lets the same pipeline run against either a `&DiGraph` or a filtered view of one SCC. #### Algorithm ```text create_graph → make_acyclic → linearize → split_into_chunks → assemble batches ``` 1. **`create_graph`** — for each chunk group, every `(later, earlier)` pair inside the group's CSS-module list becomes an edge `later → earlier` (weight 1, accumulated). Heavy edges = strong co-occurrence. 2. **`make_acyclic`** — co-occurrence almost always introduces cycles; each multi-node SCC has its lowest-weight cycle edge cut until the graph is a DAG. 3. **`linearize`** — Kahn-style topological sort with a tie-break on edge weight, so strongly co-occurring modules end up adjacent in the global order. 4. **`split_into_chunks`** — greedy bottom-up merger over the global order. At every active split point we score the merge as `cost(merged) - cost(left) - cost(right)`, take the most-negative score from a min-heap, and repeat until no merge would reduce cost. `max_chunk_size` and "global CSS must not leak into unrelated chunk groups" are enforced as `+infinity` cost. The cost model is: ```text cost_per_group(chunk, group) = chunk_size + (chunk_size / group_total_size) * module_factor_cost + request_cost ``` summed over the chunk groups that load the chunk. #### Wiring - `StyleGroups::shared_chunk_items` is a `FxIndexMap<ChunkItemWithAsyncModuleInfo, StyleItemInfo>` where `StyleItemInfo { order: Option<u32>, batch: Option<…> }`. The graph algorithm fills `order` so `style_production.rs` can stable-sort chunks globally; the legacy algorithm leaves `order = None`, which makes the sort a no-op for it. `flatten_and_sort` returns the `StyleItemInfo` references alongside each chunk item so the per-item loop doesn't re-query the map. - A new `StyleGroupsAlgorithm` enum on `ChunkingConfig` selects the algorithm at chunking time; `ModuleGraph::style_groups` dispatches to either `compute_style_groups` (existing) or `compute_style_groups_graph` (new). - `next-core` exposes `NextConfig::css_chunking() -> Vc<CssChunkingAlgorithm>` resolving the JS `experimental.cssChunking` to the Rust enum, with cost defaults applied (`requestCost: 20_000`, `moduleFactorCost: 1`). All three chunking-context constructors (`next_client`, `next_edge`, `next_server`) thread it through. #### Configuration - `experimental.cssChunking` zod schema accepts the new shapes; cost params are `z.number().nonnegative().finite().optional()`. - `config-shared.ts` exports a `CssChunkingConfig` type alias and a `resolveCssChunkingMode(value)` helper that normalizes any input to one of `'off' | 'loose' | 'strict' | 'graph'`. Both `webpack-config.ts` (plugin wiring) and `config.ts` (bundler-compat validation) use the helper. - New `errors.json` entries for the three bundler-compatibility validation errors (E1193 graph-on-webpack, E1194 strict-on-Turbopack, E1195 false-on-Turbopack). #### Tests - 53 Rust unit tests in `style_groups_graph/tests.rs` cover `create_graph`, Tarjan SCC, `find_short_cycle` (bidirectional Dijkstra), `make_acyclic`, `linearize`, `split_into_chunks`, and end-to-end pipeline scenarios. - `test/e2e/app-dir/css-order/css-order.test.ts` is parametrised over `[label, value]` pairs. The Turbopack matrix now includes `'graph'` and an object-form `{ type: 'graph', requestCost: 1, moduleFactorCost: 1 }` in addition to the existing default. Per-page expectations grew a `requests` object encoding distinct request counts for `loose` and `graph` where they differ. - A new `sandwich` e2e fixture (`/sandwich/a`, `/sandwich/b`) exercises the case where two pages share a leading and trailing chunk around a unique middle stylesheet — including a global stylesheet that the algorithm must not leak into unrelated chunk groups. The graph algorithm hits the optimal 3 chunks per page on this fixture; loose mode falls short. #### Documentation - `ExperimentalConfig.cssChunking` JSDoc describes every accepted shape and what each cost knob does. - The `style_groups_graph` module-level docs describe the pipeline, cost model and constraints with diagrams. Closes NEXT- <!-- NEXT_JS_LLM_PR --> --------- Co-authored-by: v-work-app[bot] <262237222+v-work-app[bot]@users.noreply.github.com> Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: Tobias Koppers <sokra@users.noreply.github.com>

Add `next internal static-routes-info` CLI command (#93399) ### What? Adds a new internal CLI subcommand: ``` next internal static-routes-info [directory] [options] ``` It runs against an already-built Next.js app (`.next/` from `next build`), reads the manifests under `distDir`, and reports per-route bundle sizes split into six file categories. Supports markdown (default) or JSON output, sorting, limiting, and per-category file listings. ### Why? We want a **static, build-output-based** way to compare different chunking strategies for JS and CSS without running the user's app. Existing tooling either requires running the app (`@next/bundle-analyzer` is a webpack plugin), is bundler-specific, or aggregates per-bundle/per-asset rather than per-route. This tool answers the concrete question "how much JS / CSS does each route ship today?" purely by reading static manifests, so it can be diffed across builds, branches, or bundlers (Turbopack vs webpack) and used to evaluate chunking changes. It is namespaced under `next internal` because the output format and category boundaries are tied to internal manifest shapes; we don't intend to make it a stable public surface yet. ### How? #### Command surface ``` Usage: next internal static-routes-info [directory] [options] Options: --json Output as JSON instead of markdown. --limit <n> Only show the first N routes after sorting (totals always reflect all routes). --sort <key> Sort routes by: name (default, ascending), or one of client, client-js, client-css, client-map, server, server-bundled-js, server-unbundled, server-map, total (descending). --files Include the list of files (relative to the output directory) per category in the JSON output. Requires --json. -h, --help Displays this message. ``` `--limit` and `--sort` always consider every route for the totals; only the displayed table is trimmed/reordered. `--files` is only meaningful in JSON output and errors otherwise. Invalid `--sort` keys error with the valid set listed. #### Six categories per route Each file the tool sees is placed into **exactly one** of these six buckets, so totals are not double-counted: | Category | Description | | ------------------ | -------------------------------------------------------------------------------------------------- | | Client JS | `.js` chunks loaded by the browser | | Client CSS | `.css` files loaded by the browser | | Client Source Maps | `.map` files for client JS / CSS | | Server Bundled JS | `.js` chunks executed on the server (App Router, Pages SSR, route handlers, middleware) | | Server Unbundled | Files traced via `*.nft.json` outside `distDir` (typically `node_modules` deps for `serverExternalPackages` / Pages SSR) | | Server Source Maps | `.map` files for server JS, including `.map`s referenced from nft.json | Source maps are discovered three ways: `.map` extension matches, `//# sourceMappingURL=...` trailers in JS, and `/*# sourceMappingURL=...*/` trailers in CSS. Maps always go into the `Maps` category even when the manifest puts them next to their bundle, so they never inflate Bundled or CSS counts. `sourceMappingURL` reads are memoized per chunk so a chunk shared by N routes is opened once. #### Two-step measurement 1. **Capture per-route file sets** by reading manifests: - `pages-manifest.json` and `app-path-routes-manifest.json` for the route list. - `<entry>.nft.json` for server-bundled chunks plus traced node_modules deps. - `<entry>_client-reference-manifest.js` for App Router client JS/CSS (both `entryJSFiles` / `entryCSSFiles` on Turbopack and `clientModules.chunks` on webpack — the parser handles both layouts). - `build-manifest.json` for shared App Router root chunks (`rootMainFiles`) and Pages Router client chunks. - `middleware-manifest.json` for middleware and edge route handlers. 2. **Deduplicate inside each per-route category, then measure.** A global `lstat` cache stats every unique path once across the whole run; per-category sets dedupe via string equality. Files are routed by extension (`.map` → Maps, `.css` → CSS, `.js` → bundled / unbundled depending on whether the path stays inside `distDir`) at the point they enter a set, so e.g. an `.nft.json` referencing both bundle and `.map` paths places each in the right bucket. #### Shared metric For each route, every category also carries a `sharedAvg`: the average size of the *intersection* between this route and each peer route of the same type. Computed as ``` sharedAvg = (Σ over peers p: |files(this) ∩ files(p)|) / number_of_peers ``` with both file count and bytes reported. The metric is also expressed as a percentage of the route's own count and bytes (`percentCount`, `percentBytes`) to make sharing easy to interpret at a glance — e.g. `5.3 files (88%) / 424.12 KB (100%)` means "88% of this route's files, and effectively all of its bytes, are also shipped by an average peer". Routes with no peers (only one route of their type) get `null`. Note that percentages are NOT commutative across peers (they're divided by each route's own count/bytes) while raw intersection numbers are. #### Output Markdown (default), with three sections — `## Routes`, `## Shared (avg per other route of same type)`, `## Totals` — each rendered as a fixed-width aligned table. Empty cells render as `-` (and routes with no peers in the Shared section as `n/a`) so meaningful values stand out: ``` ## Routes | Route | Type | Client JS | Client CSS | Client Source Maps | Server Bundled JS | Server Unbundled | Server Source Maps | | ------------ | ---------- | ------------------- | --------------- | ------------------ | -------------------- | ------------------- | ------------------- | | / | app-page | 6 files / 424.40 KB | 2 files / 153 B | - | 16 files / 384.47 KB | 140 files / 1.37 MB | 16 files / 2.31 MB | | /api/edge | app-route | - | - | - | 9 files / 296.82 KB | - | 4 files / 1.53 MB | … ## Shared (avg per other route of same type) | Route | Type | Client JS | Client CSS | Client Source Maps | Server Bundled JS | Server Unbundled | Server Source Maps | | ------------ | -------- | ---------------------------------- | ---------------------------- | ------------------ | -------------------------------- | --------------------------------- | ------------------------------ | | / | app-page | 5.3 files (88%) / 424.12 KB (100%) | 1.3 files (63%) / 52 B (34%) | - | 11 files (69%) / 357.52 KB (93%) | 140 files (100%) / 1.37 MB (100%) | 11 files (69%) / 2.19 MB (95%) | … ``` JSON has the same per-category structure (count + bytes + sharedAvg + optional files list when `--files` is used) with identical category ordering: `clientJs`, `clientCss`, `clientMaps`, `serverBundled`, `serverUnbundled`, `serverMaps`. JSON values are exact (e.g. `0/0` is preserved as `{count:0, bytes:0}` rather than `-`) so machine consumers aren't affected by the markdown placeholder. Totals also expose dedup'd `files` arrays under `--files`. #### Route types Reported types: `app-page`, `app-route`, `pages`, `pages-static`, `pages-api`, `middleware`. App Router route handlers with `runtime: 'edge'` report as `app-route` (not a separate `edge-function`) so they're directly comparable with their Node-runtime peers. Middleware is a first-class type rather than being lumped under edge-function. #### Robust manifest parsing `_client-reference-manifest.js` is a JS module, not JSON. Both bundlers emit it but with different layouts: - Turbopack: multi-line, with a `for (const key in MANIFEST[entry].clientModules) MANIFEST[entry].clientModules[k] = val` suffix when a deployment ID is set. - Webpack: single-line, no whitespace around `=`. We extract the JSON body without evaluating the file. The implementation locates the `globalThis.__RSC_MANIFEST[` anchor, walks the JS string literal that holds the entry name (honoring `\\` escapes), then balance-walks the `{...}` body. This handles entry names that contain `]` characters, e.g. ```js globalThis.__RSC_MANIFEST["/(dashboard)/[teamSlug]/(team)/~/stores/(store-details)/blob/[storeId]/page"] = {...} ``` Any structural surprise (anchor missing, unterminated string/object, JSON parse failure) throws with the file path and offset — we never silently undercount client JS/CSS for a route. Only file-not-found stays as a `null` return — that's a normal case for server entries with no client-reference manifest (middleware, route handlers, etc.). ### Tests `test/production/static-routes-info/` is a real fixture covering every route type: - App Router: `/`, `/about`, `/no-client`, `/items/[itemId]` (a dynamic segment inside a `(group)` route group, which forces `]` to appear unescaped in the manifest entry name and exercises the parser), plus the auto-generated `/_not-found`. - App Router route handlers: `/api/node` (default Node runtime), `/api/edge` (`runtime: 'edge'`). - Pages Router: `/pages-ssr`, `/pages-ssr-2` (siblings sharing chunks), `/pages-static`, `/api/hello`. - Middleware: `middleware.ts`. - A shared lib (`lib/shared.ts`) imported by both pages-router siblings and `/`-`/about` to give the shared-avg metric something non-trivial to measure. - A `'use client'` `Counter` component imported by `/` and `/about` (but not `/no-client`), which itself imports `counter.module.css`. Routes that import Counter must ship strictly more client JS (and on Turbopack, more client CSS) than `/no-client` — this is asserted, and it's the cross-bundler regression check for the App Router client-JS collection on webpack via `clientModules.chunks` (without it, every webpack app-page reports `clientJs.count = 0`). The test file (`static-routes-info.test.ts`) covers all the above plus output formats, sort options, limit semantics, file-list integrity, totals dedup, shared-avg correctness against a hand-computed reference, markdown/JSON consistency, and the empty-cell `-` placeholder. The shared-avg metric is verified three independent ways: against a from-scratch reimplementation that walks the `--files` lists and recomputes every (route, category) cell; against a "sharedAvg.count == own.count IFF every peer is a strict superset" invariant that makes 100% values load-bearing; and against a hand-known case where one route ships a chunk no peer does, forcing strictly-below-100% sharing. 31 tests, passing on both Turbopack and webpack. The tool was also exercised against `bench/basic-app`, `bench/heavy-npm-deps`, `bench/nested-deps`, `bench/app-router-server`, and `bench/nested-deps-app-router` while developing. ### Notes for reviewers - New error codes added to `errors.json` for the manifest-parser throws and other invariant violations. - The command is registered under `next internal`; not advertised in user-facing docs by design. - Webpack quirk documented in the test: `flight-manifest-plugin.ts`'s `mergeManifest` merges every app-page's `entryCSSFiles` into every other route's CRM, so per-route CSS attribution on webpack is inherently fuzzy — the test asserts CSS attribution on Turbopack only, and the comment explains why. <!-- NEXT_JS_LLM_PR --> --------- Co-authored-by: v-work-app[bot] <262237222+v-work-app[bot]@users.noreply.github.com> Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: Tobias Koppers <sokra@users.noreply.github.com>

turbopack: reschedule stale tasks with correct invalidation priority (#92897) ### What? When an in-progress task is invalidated during execution, it transitions to a "stale" state. Previously, on completion it was directly re-executed in the same worker slot — inheriting the original schedule priority rather than the priority from the invalidation that made it stale. ### Why? A stale task that was invalidated at low priority was being re-executed at whatever high priority the original schedule had. This caused high-priority work to be unfairly blocked or deprioritized in the scheduler. ### How? **`backend.rs` trait:** `task_execution_completed` return type changed from `bool` (reschedule yes/no) to `Option<TaskPriority>` — `None` means done, `Some(priority)` means the task was stale and must be re-executed at this priority. **`backend/mod.rs`:** The three helper functions (`_prepare`, `_connect`, `_finish`) and the main `task_execution_completed` all propagate the invalidation priority on stale returns. In each stale path, the priority is read from `task.is_dirty().unwrap_or(TaskPriority::leaf())` before the task state is mutated. **`manager.rs`:** The executor no longer loops to directly re-execute stale tasks. Instead, if `task_execution_completed` returns `Some(stale_priority)`, the task is unconditionally re-scheduled through the priority runner at that priority, so all tasks execute in the correct priority order. <!-- NEXT_JS_LLM_PR -->

Turbopack: order CSS modules by chunk-group co-occurrence in linearize (#95579) ## Summary The graph-based CSS chunker's `linearize` step produces the global module order that `split_into_chunks` then cuts into chunks. It previously broke ties between ready modules using a weight-sorted stack. It now prefers the ready module that shares the most chunk groups with the previously placed module, so modules that are loaded together end up adjacent in the global order and split into better-aligned chunks. On a real world example this measurably lowers the modeled chunk-loading cost, with fewer requests and less over-fetched CSS. The shared-group count is read from edge weights (how many chunk groups two modules co-occur in). Since `make_acyclic` deletes edges to break cycles, it now returns the edges it cut so `linearize` can still count those conflicting-order co-occurrences — reconstructing the full co-occurrence without cloning the graph. ## Verification - `cargo test -p turbopack-core style_groups_graph::tests` (all pass) - Validated on a real world example that the resulting global order is deterministic and the modeled chunk-loading cost decreases. <!-- NEXT_JS_LLM_PR -->

Turbopack: lazy aggregation optimize via persistent pending flag (#93454) ### What? Make the aggregation `optimize_queue` in the Turbopack persistent backend bounded and lazy. Cap the in-memory queue size, persist an `optimization_pending` flag per task, drain the queue per `process()` call with a per-queue lifetime budget, and recover dropped optimizations opportunistically via the flag instead of via an unbounded scheduler-side queue. ### Why? The previous implementation pushed an `OptimizeJob` for every `push_optimize_task` call into a single in-memory queue with no upper bound. On large workloads (or pathological aggregation churn), this queue could grow very large and cause the thread that scheduled the optimization to do unbounded work, regressing latency for the operations that triggered the schedule. The goals of this change: - Bound the worst-case work any single `process()` call does for optimizations (per-queue budget). - Bound the in-memory queue size so memory use is predictable. - Avoid losing optimizations: anything we drop must be eventually recovered. - Keep the common fast path cheap — no extra `Meta`-category guard acquisitions when the optimization flows through normally. ### How? Persist a new `optimization_pending` flag on `TaskStorage` (`storage_schema.rs`) and use it to drive lazy recovery in `aggregation_update.rs`: - `push_optimize_task` only enqueues an in-memory `OptimizeJob` if the queue is under `MAX_OPTIMIZE_QUEUE_SIZE` (10000) and the per-queue lifetime budget `MAX_OPTIMIZATIONS_PER_QUEUE` (1000) hasn't been exhausted. If we can't enqueue, we set `optimization_pending = true` on the task so a future operation that visits this task will re-discover and re-enqueue the optimization. - Every `AggregationUpdateJob` handler calls `check_optimization_pending` on the primary task(s) it touches, which re-enqueues the optimize job if the flag is set (and the queue/budget allow). - `process()` drains the `optimize_queue` one job at a time (preserving the original "root first" ordering), counting against the per-queue budget. Once the budget is exhausted, further `OptimizeJob`s in the queue are dropped and the flag is left set on those tasks (so they recover later). - `optimize_task` clears `optimization_pending` at entry so the recovery loop eventually settles. - The flag is **only** written on the drop path — the common case (enqueue → process normally) does not touch `optimization_pending`, so no `Meta`-category guard contention is added on the hot path. - `OptimizeJob` carries a best-effort `flag_already_set` snapshot so that when a job is dropped at process time and the snapshot says the flag was already set, we skip the redundant write entirely. Most jobs originating from `check_optimization_pending` (the recovery path) and `optimize_task`'s self-re-enqueue carry this hint. - `try_enqueue_optimize_job` is `#[must_use]` so the contract \"if this returns false, set the flag\" is enforced at the type level. `lock_and_mark_optimization_pending` is shared between `push_optimize_task_by_id` and the budget-exhausted drop branch. - `optimizations_executed` is intentionally persisted with the queue so that suspending and resuming the queue cannot reset the per-queue budget. <!-- NEXT_JS_LLM_PR --> --------- Co-authored-by: v-work-app[bot] <262237222+v-work-app[bot]@users.noreply.github.com> Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: Tobias Koppers <sokra@users.noreply.github.com>

Turbopack: Fix unsound IntoIterator for ReadRef<T> (#94122) ### What? Replaces the unsound by-value `IntoIterator` impl for `ReadRef<T>` in `turbo-tasks` with a sound clone-free variant, and adapts the callers across `turbopack-*` and `next-api`. ### Why? The previous impl used `transmute_copy` to fabricate `&'static`-typed items so it could expose them through the standard `Iterator` trait. Those references were only really valid as long as the `ReadRef` inside the iterator stayed alive — but `Iterator::Item` is a fixed associated type, so once the items were stashed in futures, `Vec`s, or `serde_json` map keys, the lifetime was completely unenforced. This produced a latent use-after-free whenever something else (turbo-tasks cell eviction, an intermediate `Drop`, etc.) released the underlying storage between the iteration site and the next dereference. The observed symptom was a panic in `RcStr::as_str` during JSON serialization of `AssetHashesManifestAsset`'s manifest: ``` thread 'tokio-rt-worker' panicked at turbopack/crates/turbo-rcstr/src/lib.rs:132:52: range end index 13 out of range for slice of length 7 ``` The byte read at the inline-length position was junk left over from freed/reused memory — `len = 13` is unreachable for any legitimately-constructed inline `RcStr` (max inline length is 7 on 64-bit). The bug site was `crates/next-api/src/project_asset_hashes_manifest.rs`, which consumed an `OutputAssetsWithPaths` `ReadRef`, kept `&RcStr` references in `asset_paths` past the `try_join` that dropped the iterator, then serialized them. ### How? **`turbopack/crates/turbo-tasks/src/read_ref.rs`** — new by-value impl: ```rust pub struct ReadRefIter<T, I, J> where T: VcValueType, I: Copy + 'static, J: Iterator<Item = &'static I>, { iter: J, _read_ref: ReadRef<T>, } impl<T, I, J> Iterator for ReadRefIter<T, I, J> /* … */ { type Item = I; fn next(&mut self) -> Option<I> { self.iter.next().copied() } } impl<T, I, J> IntoIterator for ReadRef<T> where T: VcValueType, I: Copy + 'static, J: Iterator<Item = &'static I> + 'static, &'static VcReadTarget<T>: IntoIterator<Item = &'static I, IntoIter = J>, { type Item = I; type IntoIter = ReadRefIter<T, I, J>; fn into_iter(self) -> Self::IntoIter { let r: &VcReadTarget<T> = &self; // SAFETY: the fabricated `&'static` reference is only stored inside // `iter`, which lives inside the returned `ReadRefIter` alongside // the `ReadRef` that owns the data. `next()` only ever yields // `Copy`-ed-out values — no reference (with the fake `'static` // lifetime or otherwise) ever leaves the iterator. Struct-field drop // order (`iter` then `_read_ref`) drops the borrow before the // backing storage. let r = unsafe { std::mem::transmute::<&VcReadTarget<T>, &'static VcReadTarget<T>>(r) }; ReadRefIter { iter: r.into_iter(), _read_ref: self } } } ``` Key properties: - **No cloning.** Setup is one borrow + `transmute`; `next()` is `Option::copied()` (bitwise copy via the `Copy` bound), not `Clone::clone`. Nothing in the iterator clones the backing collection or its elements. - **Contained `unsafe`.** The fake `'static` reference never leaves `ReadRefIter`. `Iterator::next` yields `I` by value, so the lifetime never escapes into futures, `Vec`s, or other persistence outside the iterator. - **Drop order safe.** Struct fields drop in declaration order: `iter` (and any borrows it holds) drops before `_read_ref` (the backing `Arc`). - **`Copy` bound.** The impl is restricted to element types that are `Copy` — `ResolvedVc<_>`, integer ids, owned-tuple-of-`Copy`, etc. For non-`Copy` element types (`RcStr`, `FileSystemPath`, `PatternMatch`, `(String, _)`, `(ModuleId, ReadRef<_>)`, …) callers iterate by reference via the existing `IntoIterator for &'a ReadRef<T>` impl (`for x in &read_ref` or `read_ref.iter()`). The original buggy site in `project_asset_hashes_manifest.rs` now uses `output_assets.iter()` and keeps `&'a RcStr` references in the manifest struct. The borrow checker now enforces the lifetime that used to be faked via `transmute` — `output_assets` outlives the references because nothing consumes it, and there are no clones at the call site either. **Caller adjustments.** Touching the impl forced a sweep of all call sites that were implicitly leaning on the unsound shape (yielding `&'static`-typed items as a stand-in for owned items). The fixes fall into a small number of categories: - Drop redundant `.copied()` / `.cloned()` / `|&x| f(x)` patterns after `into_iter()` (items are owned `Copy` values now, no need to deref-and-copy). - Switch non-`Copy` element iteration to `&read_ref` / `read_ref.iter()` (e.g. `PatternMatches`, `CodeAndIds`, `UnresolvedUrlReferences`, `GraphEntries`, `Vec<RcStr>`). - Reshape `crates/next-api/src/paths.rs` helpers from `impl IntoIterator<Item = &ResolvedVc<_>>` to `impl IntoIterator<Item = ResolvedVc<_>>` — `ResolvedVc` is `Copy`, so by-value is the natural shape and it composes directly with the new by-value `ReadRef::into_iter`. Callers in `app.rs`, `pages.rs`, `middleware.rs`, `instrumentation.rs`, `font.rs` updated to match (either passing the `ReadRef`/`Vec` directly, or `.iter().copied()` for borrowed sources). - A few small follow-ups: `for (key, EndpointGroup { primary, .. }) in &entrypoint_groups` in `routes_hashes_manifest.rs` (with a borrowed `&'l str` key in the manifest); `compute_async_module_info_single(graph, result)` (no `*graph`, it's already `Copy`); `&(ty, batch)` → `(ty, batch)` destructures in `chunking/mod.rs`. ### Testing - `cac` clean across the workspace. - `ca clippy --all-targets` clean. - `ca test -p turbo-tasks-backend` — all unit + integration tests pass. - `ca test -p turbopack-tests --tests` — execution snapshot suite (218 passed, 0 failed, 1 ignored) and snapshot suite (87 passed, 0 failed). Closes NEXT- Fixes #

[turbopack-trace-server] Performance improvements for span event handling (#93179) ## What? Performance improvements for the turbopack-trace-server, optimizing how span events are stored, sorted, and retrieved. ## Why? When processing large traces, the trace server was spending significant time on: 1. Repeated sorting of events for each render in ExecutionOrder mode 2. Recomputing corrected self time on repeated lookups 3. Memory pressure from very large traces ## How? ### Lazy Sorting with LazySortedVec Introduces a new `LazySortedVec<T>` data structure that stores events unsorted and defers sorting until first read via `Deref`. Uses `UnsafeCell` + `Once` for thread-safe one-time sorting. This avoids repeated sorting overhead during trace ingestion. ### Pre-sorted Span Events by Start Time - `SpanEvent::Child` now stores its `start: Timestamp` alongside the index - `SpanEvent` implements `Ord` to sort by start time (with SelfTime before Child for equal timestamps) - The viewer's `ExecutionOrder` mode no longer needs to sort - events are already in order ### Cached Corrected Self Time - `SpanEvent::SelfTime` now wraps a `SpanEventSelfTime` struct containing an `OnceLock<Timestamp>` for `corrected_self_time` - The expensive tree lookup is performed once and cached for subsequent accesses - `SpanEventSelfTimeRef` provides access to the cached value via `corrected_self_time()` ### DROP_SPANS Environment Variable Adds `DROP_SPANS=<count>` env var to skip the first N spans when loading traces: - Useful for reducing memory usage with very large traces - Tracks dropped span IDs to also skip their subsequent events (End, SelfTime, etc.) - Stats output shows dropped span progress (e.g., "1000 spans, 500/1000 dropped") <!-- NEXT_JS_LLM_PR --> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Turbopack: fix lock-order inversion between Storage::map and Storage::snapshots (#93788) ### What? Fix a latent deadlock between `track_modification_internal` / `SnapshotShardIter::next` and `Storage::end_snapshot` in `turbo-tasks-backend`, and harden `dash_map_multi::RefMut` so that holding a `StorageWriteGuard` across an `.await` can no longer compile. ### Why? `Storage` has three sharded `DashMap`s — `task_cache`, `map`, and `snapshots`. The two call sites that touch both `map` and `snapshots` (`track_modification_internal` at `storage.rs:685/703` and `SnapshotShardIter::next` at `storage.rs:845/851`) hold a `map` shard write lock and then take a `snapshots` shard write lock — i.e. order `map → snapshots`. `end_snapshot` did the opposite: it called `parallel::for_each(self.snapshots.shards(), ...)`, took a `snapshots` shard write lock, drained it, and then called `self.map.get_mut(&key)` to promote flags — order `snapshots → map`. The existing comment argued this was safe because `track_modification` only inserts into `snapshots` when `snapshot_mode == true` and `end_snapshot` first stores `snapshot_mode = false`. That argument is incorrect. `track_modification_internal` loads `snapshot_mode` (line 616) and inserts into `snapshots` later (line 685 or 703); the two are not atomic. The interleaving: 1. T1 in `track_modification_internal`: holds `map[N]`, reads `snapshot_mode() → true`. 2. T2 in `end_snapshot`: stores `snapshot_mode = false`, takes `snapshots[N]` write lock. 3. T2: tries `map.get_mut(&key)` → blocks on `map[N]` (T1 holds it). 4. T1: tries `snapshots.insert(...)` → blocks on `snapshots[N]` (T2 holds it). deadlocks both threads. Separately, `dash_map_multi::RefMut` (which backs `StorageWriteGuard`) had `unsafe impl Send`. The wrapped `parking_lot` `RwLockWriteGuard` is intentionally `!Send` upstream (via `lock_api::GuardNoSend(*mut ())`); the manual override let callers compile code that holds a `StorageWriteGuard` across an `.await`. In that pattern the guard pins a shard write lock from a parked future and every other tokio worker piles up trying to take the same shard — a deadlock that the borrow checker would otherwise catch for free. ### How? **`Storage::end_snapshot`** (`storage.rs:383`): - Zip `map.shards()` with `snapshots.shards()` by index and lock each pair in the documented order: `map_shard.write()` first, then `snap_shard.write()`. - For every key drained from `snapshots[N]`, resolve it directly in the held `map_shard` guard via `RawTable::find`, rather than calling `self.map.get_mut` (which would re-enter the same shard). - `debug_assert_eq!` on the shard counts. - Update the obsolete "this is fine" comment to describe the prior race and the new pattern. **Shard pairing invariant** is now documented on the `map` and `snapshots` field declarations: both `DashMap`s are constructed with the same `shard_amount`, the same `TaskId` key type, and the same stateless `FxBuildHasher`, so shard `N` in `snapshots` corresponds exactly to shard `N` in `map`. Every key present in `snapshots.shards()[N]` (if present in `map`) lives in `map.shards()[N]`, so the zipped acquisition covers every drainable entry. **`dash_map_multi::RefMut`** (`dash_map_multi.rs:32`): - Removed the `unsafe impl<K: Eq + Hash + Sync, V: Sync> Send for RefMut<'_, K, V> {}`. - `RwLockWriteGuard` already contains `GuardNoSend(*mut ())`, so the auto-trait derivation correctly marks `RefMut` (and therefore `StorageWriteGuard`, `TaskGuardImpl`, `OccupiedEntry`, `VacantEntry`) `!Send`. Verified by compile-fail probes on all four guard types. - Kept `unsafe impl Sync` — sharing `&RefMut` between threads is still sound (the write guard provides exclusive access, `K: Sync + V: Sync`). - Replaced the old SAFETY comment with one that explains why `Send` is intentionally omitted. ### Audit summary After the fix, the full lock-order graph for the three `Storage` dashmaps is: ``` task_cache ─→ map ─→ snapshots ▲ └── (non-blocking try_lock_and_remove + defer in evict_after_snapshot) ``` Total order: `task_cache → map → snapshots`. The only reverse edge is `evict_after_snapshot`'s non-blocking `try_lock_and_remove` on `task_cache` while holding `map`, which defers on contention — no hold-and-wait possible. Within `map`, the only multi-guard path is `access_pair_mut`, whose retry loop in `get_multiple_mut` holds zero shard guards at every blocking acquire. Per-thread nested task guards are prevented at runtime in debug builds by `TaskLockCounter::acquire` (`operation/mod.rs:128–166`). ### Testing - `cargo check --all-targets` (turbopack workspace): clean. - `cargo test -p turbo-tasks-backend --lib storage::`: 4/4 pass, including `modify_during_snapshot_clears_live_modified_flags` and `modify_different_category_during_snapshot` which exercise the snapshot lifecycle this PR touches. - Compile-fail probes confirmed `StorageWriteGuard`, `TaskGuardImpl`, `OccupiedEntry`, and `VacantEntry` are all `!Send` post-change. <!-- NEXT_JS_LLM_PR -->

[turbopack] Enforce `root` attribute for strongly consistent reads and collectibles (#93114) ### What? Replaces the silent "make root node" promotion in the turbo-tasks backend with a panic that enforces tasks to already have the `root` attribute before performing strongly consistent reads, reading task collectibles, or removing collectibles. ### Why? Previously, the backend would silently promote any non-root task to a root node (aggregation number `u32::MAX`) when these operations were requested. This masked incorrect task configuration — tasks that needed root-level aggregation weren't explicitly declared as such, making it harder to reason about the aggregation graph and hiding potential performance issues from implicit promotions. ### How? **Backend enforcement (3 locations):** - **Strongly consistent read** (`backend/mod.rs`): Checks `NativeFunction.is_root` on the target task. If it's a persistent task without `root`, panics with both the target and reader task descriptions. - **Read task collectibles** (`backend/mod.rs`): Same check when reading collectibles from a task. - **Remove collectibles** (`operation/update_collectible.rs`): Same check when removing collectibles (count < 0). All three locations drop the task guard before panicking to avoid deadlocking with `debug_get_task_description`. **Macro support for `root` on methods:** - `value_impl_macro.rs`: Now reads the `root` attribute from `#[turbo_tasks::function(root)]` on inherent impl and trait impl methods (previously hardcoded to `false`). - `value_trait_macro.rs`: Same for trait default methods. **`root` attribute additions:** - All `#[turbo_tasks::function(operation)]` in test files, benchmarks, and fuzz code - Production operations in `crates/next-api/` that are read with strong consistency - Regular functions and methods in tests that use `.strongly_consistent()` - Trait methods in `value_impl` and `value_trait` blocks used with strong consistency **New test:** - `non_root_task_panic.rs`: Verifies the panic fires when attempting a strongly consistent read on a non-root operation task. Captures the panic from the worker thread via a panic hook since the panic propagates as a channel error to the test thread. <!-- NEXT_JS_LLM_PR --> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Turbopack: make available_modules an OperationVc and simplify merged module bookkeeping (#94055) ### What? Reworks how `chunk_group_content` exposes `available_modules` for downstream chunking, and consolidates the bookkeeping for merged modules in `ChunkGroup`. - Splits `ChunkGroupContent` into a `turbo_tasks::value` `ChunkGroupContentInner` (carrying `chunkable_items`, `batch_groups`, `async_modules`, `traced_modules`, and `available_modules`) plus a thin wrapper that pairs it with the resulting `AvailabilityInfo`. `chunk_group_content_operation` now runs entirely inside `turbo_tasks`, allowing `AvailableModules` to store its set as an `OperationVc<AvailableModulesSet>` and `.connect()` it on read. - Replaces the separate `included` set and `should_create_chunk_item_for` method on `MergedModuleInfo` with an `Option<ChunkableModule>` value type on the `replacements` map. Call sites now decide all three cases (replace / skip / keep) in one match via `should_replace_module`. ### Why? - `AvailableModules::snapshot` did not correctly derive its invalidation priority, which caused double invalidations: the snapshot was treated as a low-priority dependency even though it gated chunking work, so when its inputs changed turbo-tasks would invalidate it after dependent computations had already started, triggering them a second time. Exposing `available_modules` as an `OperationVc` makes `AvailableModules` inherit the correct invalidation priority from the operation that produces it, which removes the double-invalidation. - The previous `MergedModuleInfo` API had two parallel data structures (`replacements` and `included`) and a helper that combined them. Folding `included` into the value of `replacements` removes the duplication, makes the three states (replace, skip-because-merged, keep) explicit, and shrinks the call-site logic in `chunk_group.rs`. ### How? **`available_modules` as an `OperationVc`:** - New `ChunkGroupContentInner` is a `turbo_tasks::value` holding the chunkable items, batch groups, async/traced modules, and an `OperationVc<AvailableModulesSet>` for `available_modules`. - `ChunkGroupContent` becomes a plain struct wrapping `ResolvedVc<ChunkGroupContentInner>` together with the `AvailabilityInfo` computed for the group. - `chunk_group_content_operation` is an `OperationVc`-producing function so its inner `available_modules` can be returned as an `OperationVc<AvailableModulesSet>`. `AvailableModules` now stores that operation Vc and `.connect()`s it whenever the set is read, so consumers reach the set through the operation and inherit its priority. - Browser and Node.js chunking contexts are updated to thread the new shape through `make_chunk_group` and friends. **`MergedModuleInfo` simplification:** - `replacements` changes from `HashMap<ChunkableModule, ChunkableModule>` to `HashMap<ChunkableModule, Option<ChunkableModule>>`: `Some(m)` means "replace with `m`", `None` means "skip — already included in another merged module", absent means "keep as-is". - The old `included` set and `should_create_chunk_item_for` helper are removed; `should_replace_module` returns the three-way outcome and consumers match on it once. - Call sites in `chunk_group.rs` are restructured around the single match so the logic is no longer split across two lookups. ### Notes - No user-facing behavior change is intended — this is an internal refactor in `turbopack-core` (`chunk_group_content`, `available_modules`, `chunk_group`) and the browser/nodejs chunking contexts. - Existing chunking tests in `turbopack-core` exercise the new shape. <!-- NEXT_JS_LLM_PR -->

Report OS memory pressure in TurboMalloc and trace samples (#93333) ### What? Adds a new `TurboMalloc::memory_pressure()` method that returns a normalized OS-level memory pressure value in the range `0..=100` as `Option<u8>`. That value is attached to every memory sample in the tracing layer (`TraceRow::MemorySample`) and propagated through the trace-server so that span queries return a `memory_pressure_samples` vector next to the existing `memory_samples`. ### Why? Our current tracing only records the in-process allocator usage (`TurboMalloc::memory_usage()`), which does not tell us when the *operating system* is actually under memory pressure. We want that signal in the trace output to: 1. Surface real OS memory pressure in trace dashboards alongside our own allocation totals. 2. Eventually use it as input to task-eviction decisions in `turbo-tasks` (see branch description). This PR lands the plumbing; the eviction heuristic is not part of this change. ### How? **`TurboMalloc::memory_pressure() -> Option<u8>`** — new, in `turbopack/crates/turbo-tasks-malloc/`. Values are normalized so that `0` = no pressure, `100` = maximum pressure. Platform-specific backends: | Platform | Source | Notes | |---|---|---| | Linux | `/proc/pressure/memory` (`some` `avg10`), fallback to `(MemTotal - MemAvailable) / MemTotal` from `/proc/meminfo` | PSI is not available on all kernels (< 4.20, without `CONFIG_PSI`, restricted containers). The meminfo fallback keeps the signal meaningful on any standard Linux system and matches the semantics of Windows' `dwMemoryLoad`. | | macOS | `kern.memorystatus_level` sysctl (% free memory) | Pressure = `100 - level`, read via `libc::sysctlbyname`. | | Windows | `MEMORYSTATUSEX::dwMemoryLoad` via `GlobalMemoryStatusEx` (`windows-sys`) | Already a 0–100 percentage of physical memory in use. | | Other / wasm | — | Returns `None`. | All runtime failures (missing file, sysctl error, failed API call, unparseable content) silently yield `None` rather than panicking. **Wiring into tracing:** - `TraceRow::MemorySample` gains a `memory_pressure: u8` field. `0` is used when `memory_pressure()` returns `None` on unsupported platforms. - `RawTraceLayer::maybe_report_memory_sample` populates it on every sample (sampling cadence unchanged). - This is a breaking change to the postcard wire format of `MemorySample`; old trace files cannot be read by the new `turbopack-trace-server`. Given the dev-only nature of this data that seemed acceptable — let me know if a migration is desired. **Wiring into the trace-server:** - `Store::memory_samples` is now `Vec<(Timestamp, u64, u8)>` and `add_memory_sample(ts, memory, memory_pressure)`. - A new `Store::memory_pressure_samples_for_range(start, end) -> Vec<u8>` mirrors `memory_samples_for_range`: same `MAX_MEMORY_SAMPLES = 200` cap and same group-and-max downsampling, so both vectors align index-by-index for a given span query. - `ServerToClientMessage::QueryResult` gains a `memory_pressure_samples: Vec<u8>` field next to `memory_samples`. **Dependencies:** `libc` (macOS only, `cfg`-gated), `windows-sys` with the `Win32_System_SystemInformation` feature (Windows only, `cfg`-gated). No new deps on Linux. ### Verification - `cargo build -p turbo-tasks-malloc -p turbopack-trace-utils -p turbopack-trace-server` - `cargo clippy -p turbo-tasks-malloc -p turbopack-trace-utils -p turbopack-trace-server --all-targets -- -D warnings` - `cargo test -p turbo-tasks-malloc` — 7 tests pass, including: - `memory_pressure_is_in_range`: asserts `Some(_)` and `≤ 100` on Linux, macOS and Windows (via `cfg`-gated `.expect()`), and allows `None` elsewhere. - Parser tests for both PSI and `/proc/meminfo` code paths (typical content, malformed input, clamping). - Runtime sanity check on the Linux sandbox: `/proc/pressure/memory` is absent (kernel 5.10 without `CONFIG_PSI`); the `/proc/meminfo` fallback returned `Some(3)` as expected. <!-- NEXT_JS_LLM_PR --> --------- Co-authored-by: Tobias Koppers <sokra@users.noreply.github.com> Co-authored-by: Claude <noreply@anthropic.com>

Turbopack: import.meta.glob docs + edge case support (#92729) ## What? Adds documentation and TypeScript types for `import.meta.glob` in Turbopack, and implements additional edge cases for Vite compatibility. ### Documentation - New `## import.meta.glob` section in the [Turbopack reference page](/docs/app/api-reference/turbopack) covering: lazy/eager loading, named imports, query strings (including the object form), multiple patterns, negation, TypeScript types, and a full options table. - `import.meta.glob` row added to the Module Resolution feature table. - Migration note added to the [from-Vite guide](/docs/app/guides/migrating/from-vite) explaining that `import.meta.glob` works out of the box with Turbopack, with before/after for the deprecated `as` → `query` migration. ### TypeScript types - Added `ImportMetaGlobOptions` interface and overloaded `glob()` signatures to `ImportMeta` in `packages/next/types/global.d.ts`. - The overloads return `Record<string, unknown>` when `eager: true` is passed and `Record<string, () => Promise<unknown>>` otherwise. ### Implementation (edge cases) - **`import: '*'`** (namespace import): Treated the same as omitting the `import` option (returns the whole module namespace), matching Vite semantics. Previously would have generated broken `module["*"]` access. - **`query` as object literal**: `{ query: { bar: 'foo', raw: true } }` is now supported. Keys and values are URL-encoded and joined into a query string (`?bar=foo&raw=true`). - **Stricter option validation**: A non-object-literal second argument (e.g. `import.meta.glob('./*.js', 'eager')`) is now a compile-time error instead of a warning, since the options cannot be safely defaulted. ### Tests - Execution tests for new features: namespace `import: '*'`, query object, combining query with negation, explicit dotfile patterns. - New `import-meta-glob-errors` execution test with issue snapshots covering: too many arguments, non-string pattern, and non-object options argument. ## Why? 1. The feature shipped in #92640 but had no user-facing documentation and no TypeScript types. 2. Several Vite-compatible edge cases were not handled: namespace imports (`import: '*'`) and query objects. These gaps would cause subtle behavioral differences for users migrating from Vite. 3. Option validation was too lenient — invalid second arguments fell through to defaults silently. ## How? ### Docs & types - `docs/01-app/03-api-reference/08-turbopack.mdx` — new section, table row, TypeScript subsection - `docs/01-app/02-guides/migrating/from-vite.mdx` — migration note - `packages/next/types/global.d.ts` — `ImportMetaGlobOptions` + `ImportMeta.glob()` overloads ### Implementation - `turbopack/crates/turbopack-ecmascript/src/references/import_meta_glob.rs`: - `parse_import_meta_glob()`: `import: '*'` normalized to `None`; query object parsing via `urlencoding::encode()`; non-object options argument now emits an error and returns `None`. - `turbopack/crates/turbopack-ecmascript/Cargo.toml`: added `urlencoding` workspace dep. Note: dotfile/dot-directory filtering is not needed in `import.meta.glob` itself — the underlying `Glob` pattern engine and `read_glob_internal` already handle dotfile semantics correctly (explicit `./.foo/*.js` matches, wildcards include them). ### Tests - `turbopack-tests/tests/execution/turbopack/resolving/import-meta-glob/` — new cases for `import: '*'`, query object, query + negation, explicit dotfile pattern, wildcard including dotfiles - `turbopack-tests/tests/execution/turbopack/resolving/import-meta-glob-errors/` — new test suite for fatal parse errors with issue snapshots ### Checklist - [x] `pnpm prettier-fix` / `cargo fmt` run - [x] Documentation follows the [docs contribution guide](https://nextjs.org/docs/community/contribution-guide) - [x] Execution tests pass (`cargo test -p turbopack-tests --test execution -- "import_meta_glob"`) - [x] `cargo clippy` clean - [x] `pnpm --filter=next types` passes <!-- NEXT_JS_LLM_PR --> --------- Co-authored-by: Tobias Koppers <sokra@users.noreply.github.com> Co-authored-by: Claude <noreply@anthropic.com>

Turbopack: add TURBOPACK_DEBUG_CSS_CHUNKING env var (#95080) ### What? Adds a `TURBOPACK_DEBUG_CSS_CHUNKING` environment variable. When set to a truthy value, the graph-based CSS chunker (`experimental.cssChunking: "graph"`) writes a JSON snapshot of its inputs and outputs to the current working directory on each invocation. ### Why? We are investigating problems with the new graph-based CSS chunking algorithm. The pipeline (`create_graph` → `make_acyclic` → `linearize` → `split_into_chunks`) is internal to `turbopack-core` and doesn't surface enough information through normal build output to debug bad chunking decisions post-hoc. A side-channel dump lets us reproduce and reason about a problematic build without instrumenting Rust on a user's machine. ### How? In `turbopack/crates/turbopack-core/src/module_graph/style_groups_graph/mod.rs`, between `split_into_chunks` and the result assembly: - Whether the dump is enabled is cached in a `static DEBUG_DUMP_ENABLED: LazyLock<bool>` so the env var is read exactly once per process. Truthy = anything other than unset, empty, `0`, or `false` (case-insensitive). - When enabled, every call to `compute_style_groups_graph` resolves `ident_string()` for every CSS module in parallel via `try_join` and writes a pretty-printed JSON file `turbopack-css-chunking-debug-<unix_ms>-<seq>.json` in the current working directory. The timestamp + atomic counter suffix ensures concurrent or repeated calls don't clobber each other. - Failures while writing the dump are logged to stderr and otherwise swallowed — a debug toggle must never fail a build. The JSON document contains: - `chunk_groups`: `string[][]` — module idents per chunk group, in the order the algorithm sees them. - `global_order`: `string[]` — the flat global order produced by `linearize`. - `global_order_chunks`: `string[][]` — the same modules grouped by the merged segments produced by `split_into_chunks` (e.g. `[["a","b"], ["c"], ["d","e","f"]]`). - `modules`: `[{ ident, size, style_type }]` — per-module metadata where `size` is the chunk item size in bytes (the same value used by the cost model) and `style_type` is `"GlobalStyle"` or `"IsolatedStyle"`. Verification: `cargo clippy -p turbopack-core --no-deps` is clean and the existing `style_groups_graph` test suite passes (53 tests). No user-facing config change, no docs change — the env var is intentionally undocumented and only meant for triage. Closes NEXT- Fixes # --------- Co-authored-by: v-work-app[bot] <262237222+v-work-app[bot]@users.noreply.github.com> Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: Tobias Koppers <sokra@users.noreply.github.com>

Turbopack: replace cssChunking graph `moduleFactorCost` with `weightDistribution` (#95088) ## Summary Reworks the cost model of the experimental graph CSS chunking algorithm (`experimental.cssChunking: 'graph'`, Turbopack only) and retunes its defaults. - **Replaces the `moduleFactorCost` option with `weightDistribution`.** The old `moduleFactorCost` term penalized a chunk purely by `chunk_size / group_total_size`, which charged a chunk group even for CSS it fully needs and never actually measured overshipping. - **New per-group cost** is `chunk_group_weight * (chunk_size + request_cost)`, summed over the chunk groups that load a chunk, where `chunk_group_weight = group_total_size ^ (-weightDistribution)` is precomputed once per group. `weightDistribution = 0` weights every chunk group equally; higher values give smaller chunk groups a larger weight, so the algorithm overships less to small pages at the cost of more requests. The size weighting subsumes the explicit overship penalty, so the metric stays chunk-local and the greedy merger is unchanged. - **Retunes defaults:** `requestCost` `20000` → `100000` (bytes) and `weightDistribution` default `0.1`. Config shape (object form): `{ type: 'graph', requestCost?, weightDistribution? }`. The change is wired end to end through the config schema/types, `StyleGroupsAlgorithm::Graph`, and the chunking algorithm. ## Verification - `cargo test -p turbopack-core --lib style_groups_graph` (54 passed, incl. a new test asserting `weightDistribution` keeps an unneeded module out of a small group's chunk) - `pnpm test-start-turbo test/e2e/app-dir/css-order/css-order.test.ts` (243 passed, 12 todo) — confirms chunk shapes/request counts at the new defaults across the `graph` and `graph-object` modes <!-- NEXT_JS_LLM_PR -->

[turbopack] Store TaskDirtyCause in Dirtyness and pass to NativeFunction::span (#94057) ### What? Track the cause of task invalidation in turbo-tasks and surface it to the tracing span when a task executes, so traces can show *why* a task was re-run. A new `task_dirty_cause` Cargo feature gates the storage and parameter plumbing for this information. The existing `trace_task_dirty` feature now depends on it. ### Why? When investigating Turbopack performance and incremental-recompute behavior, traces show which tasks ran but not what invalidated them. Recording the `TaskDirtyCause` on the `turbo_tasks::function` span makes it possible to attribute task re-runs to their triggering change (cell update, output change, collectible change, etc.) directly from a trace. ### How? - Replace `Dirtyness::Dirty(TaskPriority)` with a struct variant `Dirty { parent_priority, cause: TaskDirtyCause }`. The `cause` field is only present when the `task_dirty_cause` feature is enabled. - Move `TaskDirtyCause` from `turbo-tasks-backend` into `turbo-tasks` so the span signature in `turbo-tasks` can reference it. - Replace `OutputChange { task_description }` with `OutputChange { function: FunctionId }`, and add a new `ResolveOutputChange { function: FunctionId }` variant used when the task's output is `OutputValue::Output(_)`. This avoids constructing a `String` for the cause at invalidation time. - `NativeFunction::span` now accepts a feature-gated `cause: Option<&TaskDirtyCause>` argument and records it as a `cause` field on the `turbo_tasks::function` span. The cause is read from the dirty state in `try_start_task_execution` before execution starts. - Split the `trace_task_dirty` feature: the new `task_dirty_cause` feature owns the storage/parameter plumbing, and `trace_task_dirty` now depends on it. This lets consumers opt into the storage cost without the full tracing overhead, and keeps the default build unaffected. Closes PACK- <!-- NEXT_JS_LLM_PR -->

Produce valid file URLs for `import.meta.url` on Windows in Turbopack (#94179) ### What? `import.meta.url` evaluated in a Turbopack-compiled module on Windows produced an invalid file URI: the path portion contained backslashes (`\`) and was not URL-encoded, e.g. ``` file://C:\Users\me\project\apps\web\app\page.tsx ``` This affects any code that hands the value to `URL`, `fileURLToPath`, source maps, dev tools, or anything else that expects a [valid file URI](https://en.wikipedia.org/wiki/File_URI_scheme). This PR makes `import.meta.url` return a correct, percent-encoded file URI on all platforms, e.g. `file:///C:/Users/me/project/apps/web/app/page.tsx`. ### Why? The existing codegen in `references/esm/meta.rs` emitted ```js `file://${__turbopack_context__.P("<rel/path>")}` ``` where `__turbopack_context__.P` is the runtime helper `resolveAbsolutePath`. On the Node-side runtime (`shared-node/node-externals-utils.ts`) that helper returns `path.join(ABSOLUTE_ROOT, modulePath)`, which is OS-native — so on Windows it returns a backslash-separated path with no percent-encoding. The result was then concatenated into a `file://...` template literal, producing an invalid URI. The initial hint that `turbopack-core/src/source_map/utils.rs` was the culprit turned out to be a false lead: that code path uses `FileSystemPath::try_join`, which already normalizes `\` to `/` on Windows. The bug is in the codegen + runtime helper for `import.meta.url`, not in source-map handling. ### How? - Introduce a new runtime function `__turbopack_resolve_file_url__` (shortcut `F`) whose contract is to return a complete `file://` URI for a given relative module path. - Node implementation (`shared-node/node-externals-utils.ts`): uses `url.pathToFileURL(resolveAbsolutePath(modulePath)).href`. `pathToFileURL` handles drive letters (`file:///C:/...`), normalizes slashes, and percent-encodes path segments. - Browser implementation (`browser/runtime/base/runtime-base.ts`): returns `` `file:///ROOT/${modulePath ?? ''}` `` — the browser runtime intentionally does not expose the real filesystem path, so it returns the same stable placeholder as before, just with a valid `file:///` prefix instead of having callers concatenate one. - Change the `import.meta.url` codegen in `references/esm/meta.rs` to call `__turbopack_resolve_file_url__($formatted)` directly instead of concatenating a `file://` prefix to the absolute path. The pre-existing `encode_path` is kept only to keep the embedded JS string literal safe (the runtime helper is responsible for the URI-level encoding). - Update `TurbopackBaseContext` and add a `ResolveFileUrl` type alias in `runtime-types.d.ts` so the new shortcut is typed. - Register the new shortcut in `TURBOPACK_RUNTIME_FUNCTION_SHORTCUTS` so user code can also reference `__turbopack_resolve_file_url__` (mirroring the existing `__turbopack_resolve_absolute_path__`). `P` is left in place and unchanged because it's still used elsewhere. - Regenerate the affected Turbopack snapshot fixtures (`import-meta/*`, `comptime/typeof`, `runtime/default_*_runtime`, `workers/*`, `debug-ids/*`, etc.). The functional change in the generated code is uniform: `` `file://${__turbopack_context__.P(<rel>)}` `` → `__turbopack_context__.F(<rel>)`. - Drop the `// TODO: These file URIs are wrong on turbopack+windows` comment and the `.replaceAll('\\', '/')` workarounds from `test/e2e/app-dir/non-root-project-monorepo/non-root-project-monorepo.test.ts`, so the assertion now verifies that the URI is correctly slashed and encoded on all platforms. ### Verification - `cargo clippy --workspace --all-targets` — clean - `cargo fmt --check` — clean - `cargo test -p turbopack-tests --test snapshot` — 87/87 pass - `cargo test -p turbopack-tests --test execution` — 219/219 pass (1 ignored) - `pnpm test-dev-turbo test/e2e/app-dir/non-root-project-monorepo/non-root-project-monorepo.test.ts` — 9/9 pass - `pnpm test-start-turbo test/e2e/app-dir/non-root-project-monorepo/non-root-project-monorepo.test.ts` — 6/6 pass - `pnpm test-dev-webpack test/e2e/app-dir/non-root-project-monorepo/non-root-project-monorepo.test.ts` — 9/9 pass (verifying webpack path still works) Windows itself can't be exercised in the sandbox, but the new code path no longer depends on `path.sep` or any OS-native string manipulation for the URI: `url.pathToFileURL` is the canonical Node API for producing a file URI from any OS-native path, and the test assertion has been tightened so that a regression on Windows would fail in CI rather than be papered over. <!-- NEXT_JS_LLM_PR --> --------- Co-authored-by: v-work-app[bot] <262237222+v-work-app[bot]@users.noreply.github.com> Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: Tobias Koppers <sokra@users.noreply.github.com>

Order loader tree imports by tree depth (#93537) ### What? Reorder the imports emitted by the app-page loader-tree builder so that they are grouped by their depth in the loader tree (shallow segments first, deep segments last) instead of the somewhat arbitrary order produced by recursive walking. ### Why? The loader tree is built by walking the app router tree recursively, and each segment contributes a few `require(...)` declarations (layout, error, loading, page, metadata, ...) to a shared list of imports that is later concatenated into the generated entry module. The order of that list is what ends up in the bundle, and walking the tree recursively interleaves segments at different depths in a way that depends on traversal order rather than on the structure of the route. For chunking and readability we want a more predictable order: *outer (shallower) layouts/segments before inner (deeper) ones*, with imports inside the same depth keeping their original relative order. This gives downstream chunking a more useful signal. ### How? In `crates/next-core/src/base_loader_tree.rs`: - `BaseLoaderTreeBuilder::imports` is now `Vec<(u32, RcStr)>` instead of `Vec<RcStr>`. The `u32` is a sort key (the depth at which the import was produced). - `create_module_tuple_code` takes a new `position: u32` argument and stores it alongside the generated `require` line. In `crates/next-core/src/app_page_loader_tree.rs`: - `walk_tree` takes a new `depth: u32` argument. The initial call from `build()` passes `0`. - `depth` is threaded through `write_modules_entry`, `write_metadata`, `write_metadata_items`, `write_metadata_item`, and `write_static_metadata_item`, so the three other places that push directly into `self.base.imports` (dynamic image metadata, static metadata items, and their alt-text companions) also tag their entries with the current depth. - When recursing into `parallel_routes`, depth is incremented only for the `"children"` slot. Named parallel routes (e.g. `@modal`) are sibling slots of the same segment and therefore stay at the same depth. - In `AppPageLoaderTreeBuilder::build`, the collected `(depth, import)` pairs are stable-sorted by depth (`sort_by_key`, which is stable in Rust, preserving the original relative order within each depth) and then stripped back to `Vec<RcStr>` before being placed on `AppPageLoaderTreeModule.imports`. The public type of `AppPageLoaderTreeModule.imports` (`Vec<RcStr>`) is unchanged, so consumers in `crates/next-core/src/next_app/app_page_entry.rs` need no adjustments. Co-authored-by: v-work-app[bot] <262237222+v-work-app[bot]@users.noreply.github.com> Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: Tobias Koppers <sokra@users.noreply.github.com>

Trace-server: Fix bottom up, and reduce allocations in turbopack-trace-server bottom-up grouping (#93460) ### What? Speeds up the bottom-up grouping pass in `turbopack-trace-server` by removing per-span `RcStr` allocations and fixes the hash map using the correct hasher for the grouping `HashMap`. ### Why? When loading large traces, building the bottom-up graph spent measurable time (1) allocating fresh `RcStr` values from `&str` keys for every span just to use them as `HashMap` keys, and (2) hashing those keys with the default randomized hasher. Both are avoidable: the underlying spans already own `RcStr`s, and the `StringTupleRef` equivalence-based lookup needs `FxHasher` anyway because `RcStr`'s `Hash` impl only matches `&str`'s `Hash` under `FxHasher`. ### How? - Change `nice_name`/`group_name`/`args` accessors on `SpanRef` and friends to return `&RcStr` instead of `&str`. The bottom-up grouping code can now clone the existing `RcStr` (a cheap ref-count bump) instead of allocating a new one from a `&str`. - Switch the `(RcStr, RcStr) -> SpanBottomUpBuilder` map in `bottom_up.rs` to use `FxBuildHasher`. This is required for the `StringTupleRef` equivalence lookup to produce matching hashes for the owned `(RcStr, RcStr)` keys, and it also removes the overhead of the default randomized hasher. - Update the `string_tuple_ref` test to use `FxBuildHasher` accordingly. No behavior changes; this is a pure refactor for performance inside the trace viewer tool. <!-- NEXT_JS_LLM_PR -->

[turbo-tasks-backend] Single-item optimization for lost follower operations (#93200) ## What? Adds performance optimizations for the common case of single-item lost follower operations in the turbo-tasks-backend aggregation update system, along with improved tracing capabilities for debugging. ## Why? When profiling turbopack operations, we observed that many aggregation update jobs involve only a single upper losing a single follower. The previous implementation always used vectors and iteration even for these single-item cases, adding unnecessary allocation overhead. Additionally, the existing `trace_aggregation_update` feature was incomplete (compilation errors) and lacked visibility into task data and operation statistics. ## How? ### Single-item optimization - Added a dedicated `InnerOfUpperLostFollower` job variant and `inner_of_upper_lost_follower` function to handle the single upper + single follower case without vector allocation - Optimized the plural variants (`InnerOfUppersLostFollowers`, `InnerOfUppersLostFollower`, `InnerOfUpperLostFollowers`) to delegate to the singular function when they contain only one item ### Tracing improvements - Added `trace_aggregation_update_stats` feature flag that enables counters for all aggregation update job types (new_followers, lost_followers, balance_edge, optimize_task, etc.) recorded in trace spans - Fixed `trace_aggregation_update` feature compilation errors by using `task.get_task_description()` instead of `ctx.get_task_description(task_id)` - Gated the `TaskDataCategory` used in aggregation update `ctx.task(...)` calls behind a const that resolves to `Meta` by default and `All` when tracing is enabled, so traces can see full task data without affecting default performance <!-- NEXT_JS_LLM_PR --> --------- Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>

Use Next.js version as Turbopack persistent cache versioning key (#93605) ### What? The Turbopack persistent cache directory under `.next/cache/turbopack/<version>/` is now versioned by the Next.js package version concatenated with the git short SHA, instead of `git describe`. Example: `v16.0.1-canary.13-94e9fa6` ### Why? The previous version key came from `VERGEN_GIT_DESCRIBE` (`git describe --match 'v[0-9]' --dirty`), which depends on having an annotated `v*` tag reachable from the build commit. That ties the cache version to local git tag state and breaks down for builds against forks, shallow clones, or branches without a recent matching tag. The Next.js package version from `packages/next/package.json` is the value users actually care about for cache compatibility, and it is already plumbed through to the Rust side as `ProjectOptions::next_version` (sourced from `process.env.__NEXT_VERSION`). Combining it with the git short SHA keeps per-commit cache uniqueness while removing the dependency on git tags. ### How? **Rust (`crates/next-napi-bindings`)** - `build.rs`: also emit `VERGEN_GIT_SHA` (short form) via `vergen_gitcl`. `VERGEN_GIT_DESCRIBE` is still emitted and is still used for the bug-report URL / panic log. - `src/next_api/turbopack_ctx.rs`: - New `cache_describe(next_version)` returns `format!("v{next_version}-{}", env!("VERGEN_GIT_SHA"))`. - `git_version_info` now takes the pre-built `describe` string instead of reading `VERGEN_GIT_DESCRIBE` itself. `dirty` handling is unchanged (still derived from `VERGEN_GIT_DIRTY`, still suppressed in CI), so `handle_db_versioning`'s dirty-repo behavior is preserved. - `create_turbo_tasks` takes `next_version: &str` and threads it through. - `src/next_api/project.rs`: - `project_new` passes `options.next_version` into `create_turbo_tasks`. No new field is needed on `NapiProjectOptions` — the `nextVersion` field already exists and is already populated by all three call sites. - `turbopack_database_compact` (the napi `databaseCompact` function used by `next-post-build`) gains a `next_version: String` parameter so post-build compaction targets the same versioned directory. **JS (`packages/next`)** - `src/build/swc/types.ts`, `generated-native.d.ts`, `index.ts`: update the `databaseCompact(path, nextVersion)` signature on both the native and WASM-fallback paths. - `src/cli/next-post-build.ts`: passes `process.env.__NEXT_VERSION` through to `bindings.turbo.databaseCompact`. `turbopack/crates/turbopack-cli` (the standalone Turbopack CLI) is intentionally left on `VERGEN_GIT_DESCRIBE` — it is not tied to a Next.js package version. --------- Co-authored-by: v-work-app[bot] <262237222+v-work-app[bot]@users.noreply.github.com> Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: Tobias Koppers <sokra@users.noreply.github.com>