Janka Uryga

[PP] Validate Shell prefetches (except gSP) (#95151) Implements instant validation for `partialPrefetching`. In this mode, `<Link>` prefetches an App Shell, which cannot access link data, and we need to warn for that. The changes in `instant-validation.tsx` are relatively simple: for an App Shell, we simply use `ShellRuntime` for all the new segments. We might also force them into `Runtime` for the purposes of discriminating dynamic holes. If a hole is present in `ShellRuntime` but disappears in `Runtime`, then we know it's caused by **link data** (as opposed to runtime or dynamic data). I've added some new error messages for this case. Note that the implementation here is incomplete: it uses the chunks from the dev render, which resolves static params in the `Static` stage. We use `ShellRuntime` for validating the App Shell, so as a result, static params are incorrectly included in it and don't trigger link data errors. This will be implemented in a follow-up. Note: It seems like we have some pre-existing bug in build validation where `fallbackParams` aren't populated, so params resolve statically when they shouldn't. I've marked two tests with `// TODO(app-shells): missing fallback params in build validation` so we can follow up and fix those.

[Instant] Point to first blocking await in validation (#96343) Closes NAR-854 ### TLDR New approach to #94949. We can work around the problem of debug info getting clamped when re-serializing by doing more work, which should be good enough unless we can figure out the correct way to adjust the clamping behavior in react. ### The problem Before this PR, instant validation works like this: 1. deserialize the full page payload (in stages) 2. reserialize each segment (in stages) 3. reassemble the segments into a combined payload, varying the stage per segment as needed Unfortunately, in step 2, we lose timing debug info that tells us when IO operations started/ended: https://github.com/facebook/react/blob/c235eb40f1c3121798d233bf992fd8f8c8558d06/packages/react-server/src/ReactFlightServer.js#L5550-L5553 This means that even if we provide an appropriate `endTime` during step 3, no IO info will be omitted by `filterDebugInfo`, because it was all re-timed to when the render occurred. In practice, this means that when we report a blocking dynamic hole, it might point to a different await than the one that was actually blocking in the stage that we rendered the segment in. ### The solution (h/t to @gnoff for coming up with this trick) We can work around this problem with no changes in React. We just have to apply an `endTime` when deserializing the payload for the first time, before reserializing each segments. However, this means that we can no longer disassemble the payload into segments in one pass -- e.g. if we trim the debug info to what it'd be in the `Static` stage, we can't get good data for the `Runtime` stage anymore. So we have to run the segment disassembly separately for each stage we care about. The debugInfo can now vary depending on the stage, we need to change the shape of `SegmentCacheItem`. Previously, it was like this: ``` '/foo': chunks: Static: [...] Runtime: [...] Dynamic: [...] debugChunks: [...] ``` But to account for the varying debug info, we now need to store the partial chunks, the complete chunks, and the debug chunks separately for each stage: ``` '/foo': Static: chunks: [...] allChunks: [...] debugChunks: [...] Runtime: chunks: [...] allChunks: [...] debugChunks: [...] Dynamic: ... ``` Most of the changes in the PR come from adjusting the code to - create separate entries for each segmentPath + stage in the new way outlined above, using an `endTime` based on the stage - consume the cache entry appropriately

[PP] Instant validation - error for unguarded static params (#94595) Follow-up to #95151, implementing support for validating awaits of static params (i.e. when using `generateStaticParams`) An app shell cannot contain any link data. This poses a challenge when we're trying to re-use the dev render for instant validation, because we have to make a choice: 1. either we resolve static params (from `generateStaticParams`) in the `Static` stage, and get an accurate *static HTML shell* (used for the initial navigation), but an incorrect app shell 2. or we resolve them in the `ShellRuntime` stage and we get an accurate App Shell, but cannot validate a static HTML shell (which would contain static params) We'll use order 2 in the main render whenever client-navigating to a page with `partialPrefetching` enabled. We still need a render with order 1 for Static Shell Validation, so we perform a second, partial render that aborts before the dynamic stage (because only instant navigation needs the dynamic stage). As an optimization, we can skip the secondary render if the page doesn't have static params, because in that case the two orders are equivalent. If we're performing an initial load, we'll use order 1 (to reflect the HTML shell). in this case, we'll do a full secondary render with order 2. Same as above, we can skip the secondary render if there's no static params. --- Where params resolve (order 1 vs order 2) is controlled by `requestStore.needsSessionShell`. We then end up with two sets of "validation inputs" (mainly rendered chunks) that we can feed into static and instant validation respectively. Also note that this PR doesn't touch `environmentName`, because that required changing too many tests. This will be addressed in a follow-up.

[PPF] Sync IO is only allowed in the dynamic stage (#95384) This PR makes Sync IO behavior more restrictive when `partialPrefetching` is on (either globally or for a page). - Previously, we allowed `await cookies(); Date.now()` in in segments that won't ever be runtime prefetched. This was achieved by separating segments into "early" and "late" stages and only erroring in the "early" ones. - After this PR, it will always be an error to do sync IO anywhere other than after `io()`/`connection()`/uncached IO (i.e. outside the Dynamic stage). We're doing this mainly because with `partialPrefetching`, `cookies()` resolves in App Shells, and sync IO causes a severe deopt in any prerender. `allow-runtime` also opts a route into `partialPrefetching`, which means that even the segments above the runtime prefetch boundary will need an App Shell and can't tolerate Sync IO. This PR removes all early/late stage separation, because we no longer need to vary the Sync IO behavior on the segment. Instead, we now pick whether a stageController should use - `SyncIOMode.AllowedInRuntimeOrDynamic` (legacy) - `SyncIOMode.AllowedInDynamic` (`partialPrefetching`) - `SyncIOMode.Untracked` if it needs to opt out I've removed the existing tests that asserted that sync IO is allowed in segments above the `allow-runtime` boundary. Instead, we check that sync io either errors (if partialPrefetching is on) or is allowed (if partialPrefetching is off) Closes NAR-855

fix: after(callback) called after response end (#94974) Fixes a bug where if `after(callback)`was called after the response ended, and no `after(callback)` calls occurred earlier, the callback would never run. Implementation-wise, what happens is: 1. when the first callback is scheduled, we do `runCallbacksOnClose()`, which waits until `onClose` fires and starts the callback queue 2. but if no `after(callback)` calls occurred before the response ended, then by the time we got to `runCallbacksOnClose()`, `onClose` has already fired, so we'd wait forever We now start listening for `onClose` when the context is initialized and track the state (`isRequestClosed`), which avoids this. I also did another drive-by fix here: if we only had `after(promise)` calls, we wouldn't switch the workUnitStore's phase to `'after'`, because we were only doing that in the `after(callback)` codepath. This is relevant in #94964 which fixes some bugs around `after(promise)`.

fix: build hangs if sync IO aborts before root chunk is flushed (#94365) Mitigates a regression introduced in https://github.com/vercel/next.js/pull/94044. We switched from prerender to render + cutting of chunks that are emitted after abort. This works well for the abort we trigger to finish prerendering, because we ensure that everything that was going to flush some output already did so before we abort. However, when Sync IO causes an abrupt abort in the middle of rendering, we also have to stop collecting output immediately (because e.g. async iterables are errored ~immediately, without `scheduleWork`), which means that partially-finished chunks may get omitted from the output. Essentially, when Sync IO happens, we might lose more content that we would've if we did a halt. In the the pages in the test suite added here, the root chunk (row 0) ends up being blocked. Before the fix introduced in this PR, this bad RSC payload would then flow into `collect-segment-data.ts` which attempts to deserialize it and hangs in an unexpected way (because with a halt, the root chunk was always there). I've also had to add a weird workaround to the streaming prerender codepaths -- it seems like in --debug-prerender, the stale time iterable keeps the stream open even after an abort, and we need to close it manually (which seems like a react bug). See NAR-810

fix: request APIs in promises passed to after() in actions/handlers (#94964) Fixes a bug where Route Handlers and Server Actions were prevented from calling `headers()` (and other request APIs) if - the call happened after the response is finished - the call *did not happen* inside an after-callback generally, this happens when you pass a promise to after. ```ts const longRunning = () => { // assume this resolves after the response await new Promise((resolve) => setTimeout(resolve, 100)) // response is done after(() => { await headers() // ❌ should be allowed, but throws }) } after(longRunning()) // or waitUntil, doesn't matter ``` The logic we used in the check was incorrect and assumed that an `afterTaskStore` was present, but we can only wrap callbacks in this ALS - we cannot change the async context of already-running promises. One noteworthy change is that i had to exit `actionAsyncStorage` when we do a render after the server action. Otherwise the checks in `isRequestApiAllowedInCurrentPhase` might believe that they're still in a server action when they aren't.

[App Shells] Allow root params in app shells (#94738) Server-side part of #94773. We remove previously added code that excludes root params from the ShellStatic stage and let them resolve there. We also add some special cases for `params` objects that consist entirely of root params - those are allowed to resolve as well.

Cancel a superseded Server Components HMR refresh's server-side work (#95486) When rapid edits overlap Server Components HMR refreshes in `next dev`, only the newest refresh can commit. The client already aborts a superseded refresh's fetch, which closes its response. We use that response-close to stop the server work the superseded refresh started, so the dev server no longer runs a render (and, under Cache Components, a validation) whose result is discarded. When the `serverComponentsHmrCancellation` flag is enabled and the request is an HMR refresh over a Node response, we derive an abort signal from `signalFromNodeResponse(ctx.res.originalResponse)` and thread it into the Flight render. This covers both dev RSC render paths: the Cache Components staged render in `generateDynamicFlightRenderResultWithStagesInDev` and the non-Cache-Components render in `generateDynamicFlightRenderResult`. The render is aborted through `renderToNodeFlightStream`, which calls `abort()` on the pipeable that `renderToPipeableStream` returns when the signal fires. React's Node Flight API has no `signal` option, unlike the Web `renderToReadableStream`, aborting the returned pipeable is the intended way to stop the render. This also removes the incorrect `signal` field from the `renderToPipeableStream` type declaration and derives `FlightRenderOptions` from `renderToReadableStream`'s options, so the render wrappers are no longer typed as `any`. Under Cache Components the superseded refresh additionally skips its detached validation, which only runs on the staged render, so it never validates a discarded tree, and any error thrown while its background renders tear down is swallowed. The aborted Flight renders surface as abort errors that `createReactServerErrorHandler` already ignores, so nothing reaches the error overlay or the CLI. The `hmr-rsc-cancellation` suite exercises both render paths, with Cache Components controlled by the `__NEXT_CACHE_COMPONENTS` test shard rather than the fixture config. A child component logs when React renders it, and the test asserts the superseded refresh never logs, since its render is aborted before reaching the child, while the committed refresh does. The detached-validation assertions run only when Cache Components is enabled.

[App Shells] App shells in runtime prefetches (#93801) Reimplements app shells in runtime prefetches in terms of a staged render. when `appShells` is on: - a `next-router-prefetch: 2` request (i.e. "including link data") will have a shell stage that only includes session data. we communicate the byte offset to the client so it can rewind the response to a session fallback - a `next-router-prefetch: 3` request (i.e. "session shell") stops rendering after the shell stages, i.e. link data is never resolved. this replaces the previous `forceOmitParams` implementation. Note that In order to produce a session shell, we have to resolve static `params` and rootParams in the Runtime stage (previously, they'd resolve in the static stage). This is because params are link data and cannot be part of the session shell. We don't really need to rewind a runtime prefetch to a static stage, so this divergence in behavior shouldn't be meaningfully observable. Notable call-out: root params getters can also be called inside caches. if a cache accesses root params, then the cache depends on link data and must be excluded from the shell. We rely on existing mechanisms for tracking root param access and delay the cache appropriately in the final render.

[Cache] Make caches error if called after prerender aborts (#96426) Closes #96339 Caches that started filling after a prerender was aborted would produce an empty stream, because an aborted `renderSignal` made [the signal we pass to `prerender()` inside `use-cache-wrapper`] be aborted immediately. So we produced an empty entry and saved it, essentially poisoning the cache. Fixed by removing `renderSignal` from the `AbortSignal.any(...)` we pass in and returning a rejected promise before we get to the cache filling codepath. This should not be observable to userspace because the prerender is already aborted.

[CC] add shell stages (#94438) Introduces new shell stages and adds relevant tasks where needed. We're not changing the timing of when anything resolves yet, so this should not have any behavioral changes.

[PP] Reveal after ShellRuntime when simulating a Shell Prefetch in dev (#95149) When `partialPrefetching` is on, the shell that we usually care about displaying is the App Shell, which is represented by the ShellRuntime stage. We should only show the cold cache indicator for caches that happen up until then, and release the client-side promise appropriately. This PR extends the existing `revealAfterStage` mechanics to account for this. > Note that this is incomplete: if we're navigating via `<Link prefetch={true}>` (or have `partialPrefetching: "unstable_eager"), we might want to use the Runtime stage instead. I'm leaving those for a follow up -- representing the common case (a shell prefetch) seems like the most useful thing. I've re-worked the `revealAfterStage/holdStreamUntilRevealed` setup into an object (`DevNavigationKind`) that represents the navigation that we're trying to simulate -- either an initial load or a client nav. This gets rid of the invalid `holdStreamUntilRevealed = true` + `revealAfterStage = RenderStage.Runtime/ShellRuntime` combination and lets us bring the logic closer to where the stream blocking tricks actually happen. I've also done some drive-by refactoring of `streamStagedRenderInDev` to dedupe some repetitive code, because every task was basically doing the same thing. I've also moved all `revealAfter.resolve()` calls into separate tasks -- we were inconsistent about this, and the `Static` `revealAfter.resolve()` was done together with the next stage, but the `RenderStage.Runtime` was done in a separate task. --- Strangely, despite the changes working as expected for the Cold Cache indicator, I can't actually get link data (`await searchParams`) to trigger a fallback when navigating, so there's a failing test for that in `cache-components-dev-streaming.test.ts`. I'm not sure why what's causing this, but I'm leaving that investigation for a follow-up as well.

[CC] Streaming prerender (#94044) We want to be able to recover a shell from static and runtime prerenders. This means that prerenders need to become staged renders, so that we can collect the outputs as we progress through the stages. This is a first step before #93801, where we'll actually implement the staging We do this with a regular `renderToReadableStream` so we can get the results streamingly. We run the render, abort it, and omit any chunks emitted after the abort (because in a render, pending chunks become errored instead of halting). This logic is encapsulated in `collectPrerenderChunksWeb`, which also tracks if the stream is still pending. In order to get debug info for hanging promises in `--debug-prerender`, we also need to still collect the debug chunks emitted after the abort and give them to the client just before aborting (so that the debug info can be read, but nothing new can render). We already do this during the dev staged render, so we can just re-use the existing `createNodeStreamWithLateRelease` to achieve this. I've also removed the unecessary microtasks from stale time and vary params tracking. we need to wait an immediate for them to actually get written into the stream, and the awaits weren't doing anything useful.

refactor: make stage advancing logic more generic (#94349) In #93801, we'll be adding 4 new stages. `StagedRenderingController` already contains a lot of duplication around advancing stages, and i want to avoid adding to that, so this PR does some cleanup. Previously, for a stage, we had - `resolve<name>Stage` - triggers listener callbacks and resolves the promise for the stage - `<name>StageListeners` - added via `onStage`, triggered when advancing to the stage but before resolving the promise - `<name>StagePromise` - resolved when we enter the stage (or rejected on abort/abandon) - sometimes, `<name>EndTime` these behaviors are all now packaged up into `StageTrigger`, which can be "fired" (when advancing to a stage) or "cancelled" (when aborting/abandoning) and tracks the time. In addition, we now have `RENDER_STAGE_ADVANCE_ORDER`, an array that defines the order in which advanceable stages (i.e. excluding Before/Abandoned) should happen. We can loop through this to advance stages without relying on a huge switch statement with fallthroughs. Re-working the logic into this shape will be handy in the future, when we want each route param to have its own stage, and the set of stages is no longer statically known.

test: allow-runtime breaks Link's Server component child detection (#95596) See NAR-876. We discovered this problem in #95384, when removing the Early/Late stage split, but it turns out that this was a preexisting race that occurs for runtime-prefetchable segments the problem existed before for runtime (which currently render in EarlyStatic, a task earlier than non-runtime-prefetchable segments) adds failing tests for `passHref` with allow-runtime (i.e. when the segment runs early), and passing tests for it in the dynamic stage (when the passHref is gated behind dynamic, which seems to delay it long enough for the debug channel to initialize)

[CC] refactor staged rendering codepaths in params/searchParams (#94718) The 'request' codepath in params/searchParams was forked `process.env.NODE_ENV === 'development'`, but that's a leftover from when dev was the only place where we did staged rendering. It didn't account for conditions like `--debug-prerender`, and now makes it generally confusing to reason about. This PR simplifies the flow so that renders that care about stages are clearly distinguished, and dev is just a minor variation within that (generally adding some extra warnings via proxies)

fix: this in edge sandbox setTimeout/setInterval (#94754) Our attempts at setting the correct `this` in our `setTimeout`/`setInterval` polyfills were slightly incorrect, because they used the wrong global object. (Thanks to [cyberjoker](https://hackerone.com/cyberjoker) for discovering this)

[App Shells] staged shell rendering during build (#94442) The final PR in the initial App Shell series. Here, we turn the build-time static prerender into a proper staged render, and recover a shell from it. The only difference between them is static link data, i.e. root params and static params. Those are delayed until the static stage analogously to the previous PRs. We collect the chunks emitted in the shell stages and emit them as a special `/_shell` segment marked with an `isPartial` byte.

[App Shells] staged shell rendering in cached navs (#94441) Continuing the app shell implementation work. This PR implements recovering *static* shells from staged dynamic requests, so that we can recover a shell from a navigation. This is done using a byte offset, analogous to existing recovery of the static stage. in #93801 we already implemented the session shell codepath, in `spawnRuntimePrefetchWithFilledCaches`, so we're not concerned with session shells here. This means that for a navigation, we can resolve static params in the static stage -- we don't care about recovering a session shell, so we don't have to delay link data until the runtime stage. One notable difference from the previous PR is that we don't set the byte length to `null` if the whole response is the same as the shell stage. We don't really expect this to happen -- if a page is fully static, it shouldn't be doing a navigation request anyway, so this doesn't seem worth optimizing. Highlight: changing the timing of when cookies/headers resolve affected some Sync IO tests -- params/searchParams were now unresolved at the abort point, and triggered a bunch of unhandled rejections. This led to the discovery that we our unhandled rejection filter is not applied to build validation, which is why we were seeing mysterious duplicated error logs in those tests. This is also fixed here since now it became more of a problem.