github.com-microsoft-FluidFramework

Switch to React's new JSX transform (#26631)

improvement(build-tools): add explicit return types and handle undefined from findConfigFile (#26186) ## Summary - Add explicit return type annotations to functions and methods across build-tools and bundle-size-tools packages to satisfy the `@typescript-eslint/explicit-function-return-type` rule - Handle the `undefined` case from `TscUtils.findConfigFile()` in repo policy check files, fixing potential runtime errors when config files cannot be found - Fix JSZip import in bundle-size-tools to use namespace import (`import * as JSZip`) instead of named import for compatibility ## Changes **Explicit Return Types** Added return type annotations to ~80 functions/methods in: - `gitRepo.ts` - Git command wrappers - `monoRepo.ts` - Monorepo utilities - `npmPackage.ts` - Package accessors - `timer.ts`, `logging.ts`, `utils.ts` - Common utilities - `buildGraph.ts`, `fluidRepo.ts`, `fluidRepoBuild.ts` - Build system - `leafTask.ts`, `tscTask.ts`, `webpackTask.ts` - Task implementations - `PrCommentsUtils.ts`, `getCommentForBundleDiff.ts` - ADO integration - Various other task and utility files **Undefined Handling** - `fluidBuildDatabase.ts` - Added undefined check for `findConfigFile` result - `fluidBuildTasks.ts` - Added undefined checks in two locations for `findConfigFile` result **API Changes** - `unzipStream` return type updated to use `ReturnType<typeof JSZip.loadAsync>` due to import change --------- Co-authored-by: Joshua Smithrud <54606601+Josmithr@users.noreply.github.com>

test(api-markdown-documenter): Add "deep" config end-to-end test cases (#23586) Adds a test case that leverages new hierarchy options to create a "deep" suite configuration where documents that generate folder hierarchy yield documents named "index" _inside_ their folder,

(pipelines): Add build performance observability pipeline (#26316) ## Description This PR updates the placeholder pipeline added in https://github.com/microsoft/FluidFramework/pull/26299. The new pipeline does the following: - Collects build metrics from ADO REST APIs for PR and internal builds - Generates an HTML dashboard (published as a pipeline artifact) which includes: - Summary metrics (total builds, avg duration, trend analysis) - Duration trend charts over time - Stage and task duration breakdown charts - Tables of recent and longest builds, including links to the source commits/PRs - Will run on a daily schedule Note: The pipeline must run in the public and internal projects to fetch PR build data and internal build data, respectively. ## Next Steps/Other Considerations - Seek feedback from the team on what other metrics could be useful. - It seems ADO only retains a limited number of PR builds. If possible, we should try to increase the limit. - Consider adding an external data store (especially if we cannot increase the public build limit). ## Misc [Spec](https://microsoft.sharepoint-df.com/:fl:/s/8b2be717-e115-43f1-9d10-bd09d33fd20b/IQAQCryKGC8vSo3_JIdYz-tnAW-5TjJglskciG2A7EQ3jN0?e=A68U9w&nav=cz0lMkZzaXRlcyUyRjhiMmJlNzE3LWUxMTUtNDNmMS05ZDEwLWJkMDlkMzNmZDIwYiZkPWIlMjF2TlJQUTAwMUprQ3BuVmxyYnpzMWoxeE9JU0NSOVFGT2dMOHk2RlpXTkk3YTZqZTc2bksxUWFoTExXaGpYQ3hzJmY9MDFTTE9WT1RRUUJLNklVR0JQRjVGSTM3WkVRNU1NNzIzSCZjPSUyRiZhPUxvb3BBcHAmcD0lNDBmbHVpZHglMkZsb29wLXBhZ2UtY29udGFpbmVy) [AB#55451](https://dev.azure.com/fluidframework/235294da-091d-4c29-84fc-cdfc3d90890b/_workitems/edit/55451) ## Example Screenshot 

build(client): generate eslint 9 configs for all packages (#25989) This change updates the shared eslint 9 config to be TypeScript and updates the script in scripts/generate-flat-eslint-configs.ts to support a `--typescript` flag to output typescript per-package configs instead of ESM. Also adds a `--finalize` flag that will do some additional cleanup once packages are fully migrated to eslint 9. Finally, there are lots of fixes to the script to handle ignore entries, inheritance from other configs, etc. The script now produces configs that work in my testing without any modifications. Even if that doesn't hold true once we are truly migrating (see #25932), merging this beforehand will simplify the review of those changes. All the configs were generated by the script using this command: ``` tsx scripts/generate-flat-eslint-configs.ts --typescript ```

feat(tree): staged allowed types (#25116) ## Description Adds `staged` API to `SchemaFactoryAlpha`. This creates an allowed type in the view schema that may or may not be included int he stored schema. Continuation of https://github.com/microsoft/FluidFramework/pull/24631, with a new PR from my fork so I can have push permissions. Major changes: - Adds SchemaFactoryAlpha.staged - There is no longer a single choice for how to derive a stored schema from a view schema: - Different use cases have been split into different APIs for example (toInitalSchema, toUpgradeSchema) - The underlying toStoredSchema now takes options to control how each staged schema is handled. - Unhydrated content always permits staged content (with the exception of clone): this ensures that export/import round tripping of staged content works. - testDocuments test suite has been expanded so existing round trip tests cover the above. Known issues/limitations: - Recursive types are not supported. Tracked by https://dev.azure.com/fluidframework/internal/_workitems/edit/45711 - Clone should produce nodes with a union of source context and staged types so that both unknown optional fields work, and new staged types can be inserted. Tracked by https://dev.azure.com/fluidframework/internal/_workitems/edit/45725 but also partially hidden by https://dev.azure.com/fluidframework/internal/_workitems/edit/45723 which covers how inserting the out of schema types does not error in unhydrated context currently. This is ok, as we catch them when inserting into hydrated documents and thus the two pugs combine to make the desired behavior, but itn't great, and could cause other issue due to violating internal invariants. Some of the new clone tests cover this. - Some places, mainly those not using alpha typer, can't accept annotated allowed types and thus don't accept staged types. Examples include the root of the tree view config, recursiveObject fields, and likely more. Many of these can be worked around using dedicated alpha APIs and/or wrapping the implicit field schema in an explicit one using SchemaFactoryAlpha.required. Some cases, like recursiveArray do not have a viable workaround, and can be addressed in future work, possibly after stabilizing annotated allowed types.

Add tree-agent package (#25114) This new package hosts the SharedTree semantic editing library. Much of the package config has been copied from other FF packages for consistency (e.g. from dds/tree). The library itself is in a usable state but will be iterated upon heavily in the coming weeks. --------- Co-authored-by: Taylor Williams <60717813+taylorsw04@users.noreply.github.com>

Automatic invalidation based on observation tracker for readers of SharedTree content (#25459) ## Description Adds automatic observation tracking for shared tree content. Adds react hooks to use this, along with strong types for use in those hooks to allow safe passing of nodes in props.

(tree): Incremental summarization of chunked forest (#24949) ## Description [AB#1263](https://dev.azure.com/fluidframework/235294da-091d-4c29-84fc-cdfc3d90890b/_workitems/edit/1263) has an overall description and link to a design document. This PR adds incremental summarization support to chunked forest. Currently, the chunked forest data is encoded and added to a single blob in the summary tree. This change breaks it down into mutiple subtrees where each subtree can be incrementally summarized, i.e., if such a subtree doesn't change between summaries, a summary handle will be added to the summary instead of the entire subtree. The schema will support marking properties as incrementally summarizable (TODO - not in this PR). Any chunks in the fields corresponding to this property will be the boundary at which incrementally summarization will work. Basically, if this property changes or anything within it changes (for objects and arrays), its chunks will be re-summarized, or else, a summary handle will be used. To achieve this, this PR makes the following changes: - Added a new encoded format called `EncodedIncrementalChunkShape` to `EncodedChunkShape` for chunks whose data will be incremental summarized. - Added a new shape called `IncrementalChunkShape` which represents chunks whose data can be incrementally summarized, aka, incremental chunks. - Each incremental chunk is assigned a `ChunkReferenceId` which uniquely identifies it under its parent field. - Added a new field encoder called `incrementalFieldEncoder` which encodes its chunks as `IncrementalChunkShape`. The `ChunkReferenceId`s of all chunks in a field are stored as `EncodedNestedArrayShape` in the encoded data. - Added a new decoder called `IncrementalChunkDecoder` which can decode an incremental chunk. - Added `IncrementalEncoder` and `IncrementalDecoder` interfaces which has properties that help with encoding and decoding of incremental chunks. These are added to the `FieldBatchEncodingContext`. - Added a `ForestIncrementalSummaryBuilder` which tracks chunks across summaries and generates summary tree or summary handles for them depending on whether they change between summaries. - The forest summary tree consists of a blob called `ForestTree` at the root that contains the data for the forest. If there are any incremental fields in the forest, an array of chunk reference ids will be encoded for its chunks in the data. - Fore each incremental chunk, a separate summary tree will be added with its reference id as the key for the tree. - The data for the chunk will be added to a blob called `contents` under the tree. If the chunk has any incremental fields, an array of chunk reference ids will be encoded for its chunks in the data. - The same format is repeated until all incremental fields have been summarized. - Note that currently, an incremental field will have exactly one chunk. This may change in the future where the data may be encoded into multiple chunks. [AB#14308](https://dev.azure.com/fluidframework/235294da-091d-4c29-84fc-cdfc3d90890b/_workitems/edit/14308) [AB#41861](https://dev.azure.com/fluidframework/235294da-091d-4c29-84fc-cdfc3d90890b/_workitems/edit/41861) [AB#41863](https://dev.azure.com/fluidframework/235294da-091d-4c29-84fc-cdfc3d90890b/_workitems/edit/41863) [AB#41864](https://dev.azure.com/fluidframework/235294da-091d-4c29-84fc-cdfc3d90890b/_workitems/edit/41864)

refactor(examples): Add explicit function return types (#26074) Part of a multi-PR effort in preparation for enabling global eslint enforcement.

refactor(api-markdown-documenter): Update Documentation Domain to be more restrictive (#24667) ### Background The Documentation Domain (`DocumentationNode` and its implementations) was created with a goal of being very flexible in the kinds of trees of content it could express. This was accomplished by, in most cases, allowing parent nodes to contain any kind of child content (with a couple of exceptions). This ultimately ended up creating problems in terms of Markdown-compatibility in particular (HTML is more expressive, so compat there was less of an issue). There are also some forms of hierarchy allowed by the system (and inadvertently used by the system) that don't make sense (e.g., parenting a `SectionNode` under a `ParagraphNode`. As this library has evolved, I have become more and more convinced that the Documentation Domain is unnecessary and should be removed. In this world, `ApiItem` transformations would output `mdast` (Markdown Syntax Trees) directly, instead of going through an intermediary domain. `mdast` has a vast support ecosystem, so it seems like a much better target than some bespoke intermediary domain. ### This PR This PR is an attempt to begin aligning the Documentation Domain with Markdown syntax requirements by restricting the kinds of contents that can appear under specific kinds of nodes. It also introduces a new pattern for domain extension, which itself aligns with how the broader `unist` ecosystem accomplishes extensibility. We require this extensibility internally, as our `fluidframework.com` website build actually does extend the Documentation Domain with a custom node type. In the future, this can be trivially mapped to a custom `mdast` node, but for now we need to support it. #### Immediate impact See the updated HTML test assets for the immediate impact on the output format. Note the reduction in unnecessary hierarchy.

feat(tree): Add Record node kind (#24908) Adds a new Record node type (constructed via `SchemaFactoryAlpha.record`) and updates various alpha-level domains to use it. - TableSchema now uses a Record in place of a Map for its `cells` (this was always the intended design) - JSonDomainSchema now uses a recursive Record in place of a recursive Map for its Object representation [AB#39284](https://dev.azure.com/fluidframework/235294da-091d-4c29-84fc-cdfc3d90890b/_workitems/edit/39284) --------- Co-authored-by: Noah Encke <78610362+noencke@users.noreply.github.com> Co-authored-by: Craig Macomber (Microsoft) <42876482+CraigMacomber@users.noreply.github.com> Co-authored-by: jzaffiro <110866475+jzaffiro@users.noreply.github.com>

feat(client): [internal] container extensions and `presence` use (#24399) ## ContainerRuntime path-based Signal addressing For Signals, support using a `/` prefixed address in `IEnvelope` (at ContainerRuntime). No change to Ops. - Only extension Signals are sent with `/` prefix. - Receiving is supported for `/ext/` (as well as `/runtime/` and `/channels/` for uniformity). Refactor signal submission so that application of `applyTrackingToBroadcastSignalEnvelope` is centralized under `sequenceAndSubmitSignal` and use `UnsequencedSignalEnvelope` to make `clientBroadcastSignalSequenceNumber` stamping (or lack of) more obvious. `ContainerRuntime.submitSignalFn` confirms envelopes coming through don't start with `/` that is reserved for path-based addressing. ## Signal message types promoted to generics `ISignalEnvelope` and `I*SignalMessage*` related types accept `type` and `content` pairs to form tight association between them. ## [internal] `ContainerExtension` API Refactor the prototype API from `presence` package to `container-runtime-definitions`. Extensions are registered with `ContainerRuntime` (via support from `FluidContainer` in `fluid-static` in declarative API) and currently may only send and receive Signals. Encapsulated support will require further work to define appropriate cross-layer interfaces. `fluid-static` support will likely change to register `presence` directly but does not currently. Build out extension message types to define outbound and inbound variants with inbound further divided into raw (unverified) and verified versions. Extensions may only submit messages conforming to the types they specify. The `ExtensionHost` (aka `ContainerRuntime` will deliver typed messages via `processSignal` but in unverified form. ## `core-interfaces` additions - `TypedMessage` added for Signal message generics constraint type. - [internal] `BrandedType` added to support distinguishing raw from verified messages. - [internal] `InternalUtilityTypes.FlattenIntersection` exposed and fixed up for extension message types. ## `presence` changes - Adapt internal `IEphemeralRuntime` to mostly match `ExtensionHost`. - Separate message types to `protocol.ts` and use more broadly. - Update test types (and some data) to conform to the stricter requirements - Improve incoming signal validation: - Check for exactly known type names - Allow for "optional" unrecognized signal; if a signal is not optional it must be recognized to avoid assert (throw) ## Support `getPresence(container: IFluidContainer)` - Update `Container` and `ContainerRuntime` to support extensions. - Replace `getPresenceViaDataObject` use in examples and tests --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Update how shared tree codec versions are generated (#25752) This change updates how the write version for the different codecs in tree are determined. Currently, there is a mapping of `SharedTreeFormatVersion` to a list of write versions for all codecs. The `SharedTreeFormatVersion` is one of the options that can be passed when creating a tree and the codecs version are selected from the mapping based on this value. With this change, `SharedTreeFormatVersion` will be removed. Instead, the write versions for the codecs will be selected based on the `minVersionForCollab` option that is also passed during tree creation. The `minVersionForCollab` will be passed down to each codec and it can decide what write version it wants based on this. This PR also standardizes how the various versions are maintained by the codecs (using an `as const` enum-like object and branded values, and a function that generates the right version based on the `minVersionForCollab`). The mapping from `SharedTreeFormatVersion` to `minVersionForCollab` is as follows: * `SharedTreeFormatVersion.v1`: no longer an option * `SharedTreeFormatVersion.v2`: no longer an option * `SharedTreeFormatVersion.v3`: `minVersionForCollab: FluidClientVersion.v2_0` * `SharedTreeFormatVersion.v4`: was never an option, still not an option * `SharedTreeFormatVersion.v5`: `minVersionForCollab: FluidClientVersion.v2_43` * `SharedTreeFormatVersion.vSharedBranches`: no longer an option. Use `SharedTreeOptions.enableSharedBranches` instead. Unchanged: specifying `minVersionForCollab: FluidClientVersion.v2_52` will enable `DetachedFieldIndexFormatVersion.v2`. [AB#51463](https://dev.azure.com/fluidframework/235294da-091d-4c29-84fc-cdfc3d90890b/_workitems/edit/51463). ## Breaking Changes `SharedTreeFormatOptions` (internal) no longer has a `formatVersion` field. --------- Co-authored-by: yann-achard-MS <97201204+yann-achard-MS@users.noreply.github.com>

refactor(container-runtime): Enable `no-explicit-any` eslint rule and fix violations (#23570) Enables `any`-related rules from the recommended eslint config and fixes violations. Specifically, - `@typescript-eslint/no-explicit-any` - `@typescript-eslint/no-unsafe-assignment` - `@typescript-eslint/no-unsafe-call` - `@typescript-eslint/no-unsafe-member-access` - `@typescript-eslint/no-unsafe-return` This PR is one piece of a multi-PR process to migrate the `container-runtime` package to our recommended eslint config. [AB#3027](https://dev.azure.com/fluidframework/235294da-091d-4c29-84fc-cdfc3d90890b/_workitems/edit/3027)

refactor(container-runtime): Enable `@typescript-eslint/explicit-module-boundary-types` eslint rule and fix violations (#23597) One in a series of PRs working towards migrating the package to our `recommended` config. Plus a couple of misc. comment syntax fixes/improvements. [AB#3027](https://dev.azure.com/fluidframework/235294da-091d-4c29-84fc-cdfc3d90890b/_workitems/edit/3027)

fix(container-runtime): block local submits during stashed-op apply (#27297) ## Summary - `PendingStateManager` exposes a one-way apply lifecycle (`applying` → `ended`, default `ended`). The window opens eagerly in the constructor when stashed state is present and closes the first time `applyStashedOpsAt` drains `initialMessages` successfully. An apply error leaves the lifecycle at `applying`. - `ContainerRuntime.isReadOnly()` aggregates `pendingStateManager.isApplyingStashedOps` and fans out via `notifyReadOnlyState`, so compliant DDSes skip submits during stashed-op replay. - `ContainerRuntime.submit()` always emits a `SubmitDuringStashedOpApply` error event on a bypass during the apply window. By default the runtime logs and continues; setting `Fluid.ContainerRuntime.EnableSubmitDuringStashedApplyThrow` opts in to the fatal `UsageError` + container close. Only `BlobAttach` is allowlisted (it is legitimate runtime-internal traffic during apply — `sharePendingBlobs` runs before the apply window closes); `IdAllocation` is *not* in the allowlist because assert `0x9a5` enforces that it is submitted directly to the outbox and never reaches `submit()`. Checking at submit time (rather than at `onFlushBatch`) closes the gap where a deferred flush would land after the apply window closes and mask the misuse. - `ContainerRuntime.replayPendingStates` asserts that the apply window is not open. The only real caller is `setConnectionStateCore`'s `canSendOps` edge, and the loader awaits `applyStashedOpsAt` before transitioning to a write-capable connection — the assert surfaces a contract violation rather than silently swallowing the edge. ## Background When a host calls `loadContainer` with `pendingLocalState`, the runtime replays stashed ops via `applyStashedOpsAt`. If a DDS submits a local op during that replay — typically a realize-time write that doesn't consult `readOnly` — the new op has no counterpart in the saved-op stream and the next saved-op replay throws "pending local message content mismatch". This change makes the runtime effectively readonly for the duration of the replay and turns the bypass case into an attributable telemetry event (default) or a fatal load failure (opt-in), with the opt-in flag as the production rollout knob. ## Risks & tradeoffs - **`baseLogger` reshape (incidental).** `ContainerRuntime.baseLogger` is now built via `createChildLogger` with always-on `error` properties (`inStagingMode`, `isApplyingStashedOps`, `isReadOnly`). Every error event flowing through `baseLogger` — including downstream consumers — now carries these tags. Behavior change for telemetry pipelines reading raw `baseLogger` output. - **Log-only by default; throw is opt-in.** `Fluid.ContainerRuntime.EnableSubmitDuringStashedApplyThrow` is off by default — a bypass surfaces a `SubmitDuringStashedOpApply` error event but does not close the container. Setting the flag opts in to the fatal `closeFn` + throw shape. The default favors observability over forcing a load failure in production for a DDS that quietly bypasses the readonly gate. ## Test plan - [x] PSM lifecycle tests cover eager open, no-op without stashed state, full-drain close, stay-open across partial drains, the close hook firing exactly once across repeated `applyStashedOpsAt` calls, and lifecycle staying `"applying"` when the final apply throws. - [x] ContainerRuntime submit-during-apply tests: default path logs the error event and does not throw or close; `EnableSubmitDuringStashedApplyThrow=true` throws + closes + logs; `BlobAttach` allowlisted (no throw); no throw outside the apply window. - [x] End-to-end test: with `EnableSubmitDuringStashedApplyThrow=true`, a `valueChanged` listener that does a cross-map edit during apply causes the load to reject with the expected `UsageError`. - [x] Existing PSM "has both pending messages and initial messages" test updated to push directly to the private queue (the prior flush-while-applying path is now a usage error). - [x] Container-runtime mocha suite passing locally, biome format clean, eslint clean. 🤖 Generated with [Claude Code](https://claude.com/claude-code) — signed: Claude (Opus 4.7) --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

refactor: Store data in unhydrated nodes directly instead of wrapping MapTree nodes (#24739) ## Description Implement map tree APIs with UnhydratedFlexTreeNode directly instead of a wrapping a MapTree. Store contextual default providers directly in a strongly typed way in the tree. Implicitly generate them using a global context if requested while unhydrated instead of special casing identifiers in the read code paths in multiple places. Provide contextual defaults as part of prepare for hydration pass instead of its own pass. Overall, this refactor should make the handling un unhydrated data simpler and more robust: - It is no longer possible for the unhydrated tree and map tree and/or the maps associating them to be out of sync. - The special case handling of identifiers has been implemented in a general way that will handle any contextual defaults, and no longer needs special logic in any code path reading from the tree. - There is no longer questionable ownership of ExclusiveMapTree nodes due to cache lookup. - Memory use should be slightly reduced by avoiding having the separate map tree objects and the caches to associate them with the unhydrated nodes. ## Breaking Changes Defaulted Identifiers show up when enumerating fields on an unhydrated node. This seems like an improvement/simplification (makes unhydrated and hydrated nodes more similar, and identifiers less special), but could theoretically break something.

arraynode event integration in text editor (#26901) ## Description Integrates the array node delta event payload into the text-editor app

Tag asserts and fail calls in tree (#23910) ## Description Tag asserts and fail calls in tree using `pnpm run policy-check:asserts` Updated build tools now tags fail calls in tree, causing the large change.