FluidFramework — Engineering Performance

refactor(markdown-magic): Replatform onto remark/mdast (#28098) This pattern unlocks a lot more flexibility in how we do generation / content copying. New functionality included in this PR: - Support for `.mdx` files (like we use in the website) - Automatic heading level detection when creating contents that include headings. Future improvements that will be possible: - Automatic file path link re-mapping (so you can safely embed contents that include links). Also updates the code to use TypeScript.

docs: Update Client Requirements template for 3.0 release (#27802) Updates the Client Requirements template with updates for 3.0. This content is embedded in the majority of our library package READMEs. [AB#78964](https://dev.azure.com/fluidframework/235294da-091d-4c29-84fc-cdfc3d90890b/_workitems/edit/78964)

refactor(examples): Add explicit function return types (#26074) Part of a multi-PR effort in preparation for enabling global eslint enforcement.

test(api-markdown-documenter): Add "deep" config end-to-end test cases (#23586) Adds a test case that leverages new hierarchy options to create a "deep" suite configuration where documents that generate folder hierarchy yield documents named "index" _inside_ their folder,

build(client): generate eslint 9 configs for all packages (#25989) This change updates the shared eslint 9 config to be TypeScript and updates the script in scripts/generate-flat-eslint-configs.ts to support a `--typescript` flag to output typescript per-package configs instead of ESM. Also adds a `--finalize` flag that will do some additional cleanup once packages are fully migrated to eslint 9. Finally, there are lots of fixes to the script to handle ignore entries, inheritance from other configs, etc. The script now produces configs that work in my testing without any modifications. Even if that doesn't hold true once we are truly migrating (see #25932), merging this beforehand will simplify the review of those changes. All the configs were generated by the script using this command: ``` tsx scripts/generate-flat-eslint-configs.ts --typescript ```

feat(tree): staged allowed types (#25116) ## Description Adds `staged` API to `SchemaFactoryAlpha`. This creates an allowed type in the view schema that may or may not be included int he stored schema. Continuation of https://github.com/microsoft/FluidFramework/pull/24631, with a new PR from my fork so I can have push permissions. Major changes: - Adds SchemaFactoryAlpha.staged - There is no longer a single choice for how to derive a stored schema from a view schema: - Different use cases have been split into different APIs for example (toInitalSchema, toUpgradeSchema) - The underlying toStoredSchema now takes options to control how each staged schema is handled. - Unhydrated content always permits staged content (with the exception of clone): this ensures that export/import round tripping of staged content works. - testDocuments test suite has been expanded so existing round trip tests cover the above. Known issues/limitations: - Recursive types are not supported. Tracked by https://dev.azure.com/fluidframework/internal/_workitems/edit/45711 - Clone should produce nodes with a union of source context and staged types so that both unknown optional fields work, and new staged types can be inserted. Tracked by https://dev.azure.com/fluidframework/internal/_workitems/edit/45725 but also partially hidden by https://dev.azure.com/fluidframework/internal/_workitems/edit/45723 which covers how inserting the out of schema types does not error in unhydrated context currently. This is ok, as we catch them when inserting into hydrated documents and thus the two pugs combine to make the desired behavior, but itn't great, and could cause other issue due to violating internal invariants. Some of the new clone tests cover this. - Some places, mainly those not using alpha typer, can't accept annotated allowed types and thus don't accept staged types. Examples include the root of the tree view config, recursiveObject fields, and likely more. Many of these can be worked around using dedicated alpha APIs and/or wrapping the implicit field schema in an explicit one using SchemaFactoryAlpha.required. Some cases, like recursiveArray do not have a viable workaround, and can be addressed in future work, possibly after stabilizing annotated allowed types.

feat(tree): add persisted commit metadata (#28064) ## Description Applications can now attach arbitrary, JSON-serializable metadata to the commit that a transaction produces, replicate it to collaborating clients, and persist it in the document. Write it via the new `customMetadata` field on `RunTransactionParamsAlpha`: ```typescript view.runTransaction( () => { view.root.insertAtEnd("new item"); }, { customMetadata: { author: "alice", intent: "add-item" } }, ); ``` Read it back while walking the branch's [history](https://fluidframework.com/docs/api/fluid-framework/treebranchhistory-interface), via the new `custom` property on `TreeBranchCommitMetadata`: ```typescript for ( let commit = view.branchHistory.getHead(); commit !== undefined; commit = commit.getParent() ) { const metadata = commit.custom; } ``` Because a commit may be produced by nested transactions, each of which may supply metadata, `custom` is the flattened combination of them all (outermost wins on conflicting keys). The structural view is available as `commit.customTree`, a `CustomMetadataTree` mirroring the transaction nesting — the same relationship `labels.tree` has to a change's label set. The metadata lives directly on the commit — on `GraphCommit.customMetadata` in memory and inline on the commits in the `EditManager` summary. That is what makes its lifetime automatically match the commit's: once the commit is trimmed from the trunk, the metadata goes with it, so there is no separate index to populate, reconcile, or prune. Both the op format and the summary format gain a `v7` (`MessageFormatVersion.v7` and `EditManagerFormatVersion.v7`), written only when `minVersionForCollab` is `2.117.0` or later (gated by `FluidClientVersion.v2_117`). That floor is a declaration rather than an enforcement mechanism: a client too old for v7 fails cleanly with an unsupported-version error when it *reaches* v7 data, so adopting this requires deploying v7-capable readers everywhere before raising the floor. The changeset spells out the rollout sequence. A revert may now be performed inside a transaction provided it is that transaction's only change, which lets the revert be given its own metadata. Attempting any other change in such a transaction throws an error. ## Reviewer Guidance The review process is outlined on [this wiki page](https://github.com/microsoft/FluidFramework/wiki/PR-Guidelines#guidelines). ### The `GraphCommit.customMetadata` property is required, not optional This is the load-bearing decision. Two places rebuild a commit from its parts rather than spreading it (`mintCommit` and `rebaseBranch`), and an optional property would let both silently drop the metadata. Declaring it required turns each into a compile error, so the type system enumerates every site that has to make a decision. Where a rebuilt commit is the *same logical commit* as its source, the property is propagated. `undefined` is used only where a genuinely new commit is minted — the inverse commit produced by reverting, rollback commits, the synthetic root/trunk-base commits, and edits from the editor that a transaction has not yet annotated. ### The metadata tree mirrors transaction nesting Each transaction in a nested stack contributes a node to a `CustomMetadataTree`. The root node is the outermost transaction, and nested transactions add child nodes. Aborted nested transactions remove their node again, so they never contribute. The single commit produced by the stack carries the tree, from which the flattened `custom` view is derived. ### Compatibility The `v7` codecs are only selected when `minVersionForCollab >= 2.117.0`, so nothing changes for existing clients by default. Evidence that this holds: regenerating the full snapshot corpus added new `v2_117` directories and **modified no existing snapshot file**, meaning the v3/v4/v6 output is byte-for-byte unchanged. If an application supplies metadata while configured below `2.117.0`, the value is kept in memory for the local session but is neither replicated nor persisted. There is a test for this. The version-specific typebox schemas exclude the `customMetadata` field for pre-v7 formats, so no pre-v7 encoder can write it. The two formats then differ in how strictly they validate, and deliberately so: the summary schemas were already `additionalProperties: false`, so a payload falsely claiming to be pre-v7 while carrying the field is rejected there. The op `Message` schema is left permissive, as it has always been — tightening the op envelope would risk rejecting ops over envelope properties unrelated to this feature, and belongs in its own PR — so such a payload is tolerated there instead. Both behaviors are covered by tests. ## Breaking Changes None. All new API surface is `@alpha` and additive. --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: a037b96b-478f-4a65-9555-d7a970e7855e Copilot-Session: 42b443d7-0621-42a1-b087-f4e4765046af

(pipelines): Add build performance observability pipeline (#26316) ## Description This PR updates the placeholder pipeline added in https://github.com/microsoft/FluidFramework/pull/26299. The new pipeline does the following: - Collects build metrics from ADO REST APIs for PR and internal builds - Generates an HTML dashboard (published as a pipeline artifact) which includes: - Summary metrics (total builds, avg duration, trend analysis) - Duration trend charts over time - Stage and task duration breakdown charts - Tables of recent and longest builds, including links to the source commits/PRs - Will run on a daily schedule Note: The pipeline must run in the public and internal projects to fetch PR build data and internal build data, respectively. ## Next Steps/Other Considerations - Seek feedback from the team on what other metrics could be useful. - It seems ADO only retains a limited number of PR builds. If possible, we should try to increase the limit. - Consider adding an external data store (especially if we cannot increase the public build limit). ## Misc [Spec](https://microsoft.sharepoint-df.com/:fl:/s/8b2be717-e115-43f1-9d10-bd09d33fd20b/IQAQCryKGC8vSo3_JIdYz-tnAW-5TjJglskciG2A7EQ3jN0?e=A68U9w&nav=cz0lMkZzaXRlcyUyRjhiMmJlNzE3LWUxMTUtNDNmMS05ZDEwLWJkMDlkMzNmZDIwYiZkPWIlMjF2TlJQUTAwMUprQ3BuVmxyYnpzMWoxeE9JU0NSOVFGT2dMOHk2RlpXTkk3YTZqZTc2bksxUWFoTExXaGpYQ3hzJmY9MDFTTE9WT1RRUUJLNklVR0JQRjVGSTM3WkVRNU1NNzIzSCZjPSUyRiZhPUxvb3BBcHAmcD0lNDBmbHVpZHglMkZsb29wLXBhZ2UtY29udGFpbmVy) [AB#55451](https://dev.azure.com/fluidframework/235294da-091d-4c29-84fc-cdfc3d90890b/_workitems/edit/55451) ## Example Screenshot 

feat(tree): Add Record node kind (#24908) Adds a new Record node type (constructed via `SchemaFactoryAlpha.record`) and updates various alpha-level domains to use it. - TableSchema now uses a Record in place of a Map for its `cells` (this was always the intended design) - JSonDomainSchema now uses a recursive Record in place of a recursive Map for its Object representation [AB#39284](https://dev.azure.com/fluidframework/235294da-091d-4c29-84fc-cdfc3d90890b/_workitems/edit/39284) --------- Co-authored-by: Noah Encke <78610362+noencke@users.noreply.github.com> Co-authored-by: Craig Macomber (Microsoft) <42876482+CraigMacomber@users.noreply.github.com> Co-authored-by: jzaffiro <110866475+jzaffiro@users.noreply.github.com>

feat(tree): Support blob handles in sandbox demo (#28299) ## Description Add Fluid handle support to the SharedTree sandbox demo so initialization and live edits can contain handles, and Guest code can resolve blob content without blocking tree synchronization. - Transport handles as session-local tokens. Restore Guest proxies on receipt and original Host handles on return, preserving identity for equivalent handles. - Resolve Guest `get()` calls through asynchronous blob requests and responses. Cache each proxy’s promise, propagate failures, and reject pending requests on disposal. Handles resolving to non-`ArrayBuffer` values are unsupported. - Use dedicated transport codecs with collision-safe escaping, preserving ordinary marker-shaped data and keys such as `__proto__`. - Restrict and normalize transport data before validation. Records use null prototypes; buffers become identity-checked placeholders during validation and are unwrapped only in validated blob responses. - Validate handle and blob messages with TypeBox, distinct branded identifiers, and runtime authorization and request-matching checks. - Document terminology, data flow, lifetime assumptions, and remaining production work. This change is limited to the sandbox example and tests; it does not change public APIs. ## Testing All 53 sandbox tests pass, including coverage for: - Initialization and bidirectional edits containing handles. - Proxy identity, concurrent resolution, failures, disposal, and deletion/undo/redo. - Marker-shaped user data, prototype-related keys, malformed messages, unauthorized tokens, and misplaced buffers. - Real `MessagePort` delivery and existing synchronization permutations. TypeScript compilation, ESLint, and formatting checks pass. ## Reviewer Guidance The review process is outlined in [the pull request guidelines](https://github.com/microsoft/FluidFramework/blob/main/docs/content/Contributing/PR-Guidelines.md#guidelines). Please focus on conversion/validation ordering, collision-safe escaping, and handle authorization and binding. Handles and proxies are retained for the owning session; per-handle reclamation is intentionally out of scope. This does **not** make the example a complete security boundary: full protocol and codec-position validation, resource limits, ID-compressor sharding, and isolated-iframe testing remain follow-up work. --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>