Andrew Clark

Port React's @gate test directive to the e2e harness (#96228) The test suite has accumulated a bunch of patterns for disabling tests that are known to fail under some configuration: `it.skip`, `if (isNextDev) { test('skipped in dev mode', () => {}); return }`, whole describes toggled off by checking `process.env.__NEXT_CACHE_COMPONENTS`. These all have the same flaw: nothing tells you when the thing you skipped starts working. The test stays disabled forever, and the workaround it was guarding rots along with it. React solves this with the `@gate` pragma, and this PR ports it to the Next.js e2e harness: ```ts // Blocked on the optimization that marks a route as fully static when // no dynamic params are referenced in Server Components. // @gate !cacheComponents it('navigates to a page with a lazily-generated static param', async () => { // body unchanged }) ``` The test still runs. If the condition is false and the test fails, the failure is absorbed and the suite stays green. If it _passes_, the suite fails: the gate is stale, delete it. So instead of a skip that hides a fixed bug indefinitely, you get a CI failure the day the fix lands. When the condition is static, the inversion is Jest's own `test.failing` under the hood. A lazy condition isn't known until the fixture's resolved config is read inside the body, so those tests invert at runtime instead. `// @force-gate <condition>` skips for real — for tests that can't even be attempted (prefetching is disabled in dev, deploy has no local build output, the fixture won't build under the condition), and for tests of a new API, where the disabled state can only throw and running it proves nothing: ```ts // Prefetching is disabled in dev, so this suite has nothing to test. // @force-gate prefetching describe('segment cache prefetch scheduling', () => { // ... }) ``` There's no staleness check in that case, so this is a judgment call: prefer `@gate` when the off state fails for a meaningful reason — the flag changes behavior that already exists — and `@force-gate` when the body can only throw because the API doesn't exist. A static condition (mode, bundler) resolves at collection time into a normal Jest skip. A lazy condition resolves at runtime, and when a lazy force-gate on a describe is false, we skip the fixture build entirely — that's what makes it usable for suites whose fixtures are build-incompatible with the condition. (One caveat: Jest has no way to skip a test that's already running, so these report as passing with a warning in the log, not as skipped.) Conditions live in a hand-written registry. I considered deriving the lazy ones from the config schema automatically, but a gate is a claim about which dimension of the test matrix explains a failure, and I'd rather each of those claims be spelled out with a description. Referencing an undeclared name fails the suite at collection time, so a typo can't silently disable a gate. The important design decision for lazy conditions is that they read the fixture's _resolved_ config, never `process.env`. The env var isn't the truth: `__NEXT_CACHE_COMPONENTS=true` only applies when the fixture doesn't set `cacheComponents` itself, and config resolution implies flags the fixture never mentions (`cacheComponents: true` alone turns on `experimental.ppr`). Resolution happens in a child process, because in-process `loadConfig` would leak the fixture's `.env` files into the Jest worker. Suites with no lazy gate never pay for any of this. The condition expression is parsed using a small grammar (also ported from the React repo). An expression that doesn't parse fails the suite: ```ts // @gate mode === 'start' && !cacheComponents // @gate !(turbopack || rspack) ``` There's also a runtime version, mirroring React's `gate(flags => flags.enableFoo)`, for tests that run under both states but assert differently (and for `it.each`, where the pragma can't attach): ```ts import { gate } from 'next-test-utils' it('renders the fallback', async () => { if (await gate((conditions) => conditions.cacheComponents)) { // PPR: the fallback is part of the static shell } else { // fully dynamic: the fallback streams in } }) ``` It also accepts the pragma expression language as a string: `await gate('cacheComponents && !dev')`. Docs are in `test/lib/gate/README.md`; `test/unit/gate/` covers the transform, the expression language, and the runtime.

Attempt static prefetch before resorting to runtime (#96095) If we're reasonably confident that a segment can be prefetched statically without omitting data that would have been included during a runtime prefetch (e.g. cookies), the client should attempt prefetch the segment statically instead of going straight to a runtime request. The decision for whether to do this is based on the ShouldAttemptStaticPrefetch added in previous steps. If it turns out the static response is not sufficient, then it will fall back to a runtime request. This makes prefetching cheaper for pages that are fully statically renderable. We can make the optimization better in the future with more reliable per-segment computation of the ShouldAttemptStaticPrefetch, but the current approach should at least work for fully static pages, which is what's most important. This optimization applies during both the Shell phase and the Speculative phase of the prefetching algorithm.

[Flight] Server References for arbitrary object types (#37636) Added behind a new experimental flag, `enableFlightObjectReferences`. Extends Server References so that a reference can point to an object, not just a function. An object is registered with a new API, `registerServerObjectReference`, which tags it with its module id the same way `registerServerReference` tags a Server Function. The client receives an opaque handle: reading a property or calling it throws, similar to how Temporary References work inside Server Functions. The only thing the client can do with the handle is pass it back to the server via a Server Function, where it resolves through the server manifest. The motivating use case is letting a framework pass request-scoped values by reference, so it can omit them from the request body. For example, a framework could model `searchParams` as a module export whose value resolves from the current request. This avoids encoding the search params twice (they already appear in the request URL) but it also makes the responses more cacheable: if the search params don't appear elsewhere in the body, then a response cache can omit them from its cache key. This initial PR only exposes the new API in the Turbopack bindings. We will port it to the other bundler configs if the experiment advances.

[Flight] Add 'pending_weak' to Flight thenable protocol (#37154) Added behind a new experimental flag, `enableFlightWeakThenables`. Adds a new thenable status to the Flight protocol: `'pending_weak'`. Unlike a regular pending thenable, a weak thenable does not block the stream from closing. If it settles while the stream is still open, its value is emitted like a normal pending thenable. Otherwise its reference is left unfulfilled and on the client it stays forever pending, without erroring, even when the connection closes. It's up to the client to handle the unresolved promise in an appropriate way. The motivating use case is being able to encode metadata about a Flight stream into the response itself. For example, a framework might want to track whether a page varies by search params. It could represent this in the response as a `Promise<boolean>` that resolves to `true` as soon as the component being rendered in the stream accesses search params. If the thenable never resolves by the time the stream closes, then the client knows that no search params were ever accessed. In the future we could add a higher-level API for encoding this kind of information. For now, we intentionally start with the low-level primitive so frameworks can experiment in userspace without adding significantly to React's surface area. Internally, Flight already uses its own private thenable statuses, like `'resolved_model'`, and the protocol is designed to treat any status besides `'fulfilled'` and `'rejected'` as equivalent to `'pending'`, so `'pending_weak'` slots into the existing machinery. On the wire, a weak reference is encoded as `$w<id>`, next to `$@<id>` for regular promises, so the client knows its row may intentionally never arrive. On the client, a weak reference behaves like any other pending promise until the response closes; then, instead of erroring, it is left forever pending.

Unify full/partial navigation response types (#96439) Introduces a new type, TransportData, to replace FlightDataPath. The new format is the same regardless of whether the entire page is rendered by the server or if some segments are intentionally omitted. The type also unifies FlightRouterState and CacheNodeSeedData into a single tree that includes both route information and RSC data. Previously these were sent in two separate trees that were isomorphic by convention, requiring the consumer to walk both in parallel. These new types are intended to be transport formats only. The client already has its own, richer data structure called RouteTree. The transport format is converted into the client format at the network decoding boundary. This does not remove FlightRouterState from the codebase entirely; it's still used both for issuing requests to the server (Next-Router-State-Tree) and for tracking state in the browser's history object. A future refactor may replace it for the purpose of issuing requests, but it will likely survive as the format for tracking state in history. This PR does not yet update the server to produce TransportData directly; a temporary adapter layer is used to convert from the old data formats to the new format. This intermediate step will not land on its own; the next PR in the stack will both update the server and delete the temporary adapter layer.

Convert per-segment prefetches to NavigationFlightResponse format (#96877) This is the last of the changes to convert all RSC requests to a single unified response type, NavigationFlightResponse. The old format was a positional array whose ordering had to be kept in sync by hand between the server and the client. The new format is the same root-anchored tree used by every other response, so the dedicated client write path for per-segment responses is deleted and these responses go through the shared one. A few functions are renamed as their roles change: convertServerPatchToFullTree becomes createNavigationSeed (it now also accepts trees with no base to overlay), fetchSegmentsOnCacheMiss becomes fetchSegmentPrefetchesUsingStaticRequest (the counterpart of fetchSegmentPrefetchesUsingRuntimeRequest), and writeSeedDataIntoCache becomes writeTreeDataIntoCache (seed data no longer exists as a concept). During the unification, a few inconsistencies and oversights were caught and fixed: response data with no matching pending entry is written as a detached entry instead of dropped, and a response's byte size is spread across the entries it actually fulfilled rather than the old chain-based count. The fallback retry loop also marks itself pending before the first cache write instead of after, closing a window where the scheduler could spawn a duplicate revalidation. The second commit unifies cache keying for prefetched segments. There were two implementations of the logic that decides which cache key a prefetched segment is written to, one for per-segment responses and one for responses produced by a live render. Now there's one: if the server reported which params the segment depends on, key the entry using those; otherwise fall back to the keying implied by how the payload was fetched. A segment is only treated as param-independent when the server says so. Two precedence fixes were needed to merge them, and a few other inconsistencies found during the unification are fixed along the way; see the commit message for details.

Separate route structure comparison from param comparison (#98974) Changing a page’s search params doesn’t change which page it is, but it may change the data that page needs. The router currently mixes these concerns by appending search params to `__PAGE__` segments. Remove that encoding so we can compare which route is being rendered separately from the param values used to render it. Param values are now compared through the existing VaryPath data structure, while route structure is compared during the existing tree traversals. Each traversal can decide which comparisons matter for the work it’s doing. Client-side comparisons and history restoration still need the search params, so store them in a separate FlightRouterState slot for now. As before, previous-page search params are stripped from normal request headers. This is a temporary step that lets us migrate incrementally. Eventually, FlightRouterState will be replaced by a type that represents the route and its params more directly. This is mostly a refactor, though it fixes an accidental inconsistency in search-param handling compared with regular route params, avoiding some redundant prefetch work. Fully cached pages already track their vary params correctly. This separation prepares us to do the same for partially dynamic segments during navigation, so changing an unrelated param won’t require rendering their dynamic content again.

Store the head as its own CacheNode (#98973) Throughout most of the client router implementation, the head is treated as if it belongs to the page segment. For example, CacheNode has separate `rsc` and `headRsc` fields, and the page's CacheNode owns both. This had led to a bunch of special cases related to the head. The Segment Cache largely (though not entirely) avoids this by instead modeling the head as a special kind of segment. It has its own vary path, its own SegmentCacheEntry, etc. There are still a few places where treatment of the head is special, because it doesn't exist inside the normal route tree structure. But we treat it like segment wherever possible. This PR updates the navigation implementation to use the same model: treat the head as a segment that is a sibling to the route segment tree. This is a large diff but the changes are almost entirely mechanical. Although the added/removed balance is about even, that includes a new test case. The net number of implementation lines has gone down.

Fix prefetch loop when a predicted route redirects (#98998) A full or runtime prefetch (e.g. `router.prefetch(url, { kind: 'full' })` or `<Link prefetch={true}>`) of a URL that matched a learned route pattern but is redirected by the server (e.g. `/docs/changelog` → `/changelog` while `/docs/alpha` renders `[collection]/[...slug]`) caused the client to re-issue the same prefetch indefinitely, at whatever rate the network allowed, until the tab was closed. When the server contradicts a route prediction, the client is supposed to remember that the URL shape can't be predicted and resolve it on the server from then on. We were storing this information (`hasDynamicRewrite`) in the wrong place: on the predicted route cache entry, rather than the optimistic routing table that's used to generate predictions. The fix moves the `hasDynamicRewrite` flag to the optimistic routing table so it applies to all subsequent optimistic routing predictions. Fixes #98345. <!-- NEXT_JS_LLM -->

Track whether runtime data is accessed during prefetch (#95964) Adds a server-computed signal to static per-segment prefetch responses that tells the client whether a runtime prefetch request would return more content than the static response already contains. The signal is computed by tracking whether the prerender accessed any data source that hangs during a static prerender but would resolve during a runtime prerender. For example: cookies, headers, fallback params, and search params. The information is encoded two places: - As a prefetch hint called ShouldAttemptStaticPrefetch. This tells the client that it's worth attempting to do a static prefetch instead of a runtime one. It's only an optimization, though: if the static response ends up being insufficient, the client will follow it up with a runtime request. It's semantically OK if there are false positives. - Embedded in the static segment response. This tells the client whether the static response is missing data that would have been included in a runtime response. Unlike the prefetch hint, this value must never falsely claim that no runtime request is needed. A follow-up will update the client to skip the runtime request using this information. This commit only encodes the signal into the response.