Kyle Into

Emit resolvable typing classes and TypeVars instead of opaque builtins Summary: When Pylance uses the Pyrefly TSP backend, hovering over special forms (`Literal`, `Final`, ...), `LiteralString`, `ParamSpec`, `Concatenate`, anonymous `TypedDict`, type-alias refs, and solver `TypeVar`s rendered as `Unknown`. The converter was stuffing these into `BuiltInType`, whose `name` is restricted to a fixed sentinel set — Pylance doesn't recognize `Literal`/etc. there and has no declaration node to chase, so it falls back to `Unknown` (microsoft/pylance-release#8062). Emit real, resolvable handles instead: special forms / `LiteralString` / `ParamSpec` / `Concatenate` / anonymous `TypedDict` / `TypeAlias::Ref` become a `typing.<name>` `ClassType` pointing at bundled `typing.pyi`; `Quantified` and the ParamSpec-internal placeholders (`Args`/`Kwargs`/`ElementOfTypeVarTuple`) become a TSP `TypeVar`. A shared `convert_type_alias_data` keeps the direct and `Forall`-wrapped alias arms in sync. This uses zero-range typeshed/synthesized declarations — enough for the consumer to render the correct name. A later diff upgrades these to precise source locations. ## Stack This stack fixes how Pyrefly's TSP backend converts internal types to the TSP wire format so Pylance renders real types instead of `Unknown` (microsoft/pylance-release#8062). Each diff is self-contained and passes the full `pyrefly_library` test suite on its own. - D107942955 [1/4] Emit the protocol-conformant lowercase `unknown` sentinel - D107943010 [2/4] Emit resolvable typing classes and TypeVars instead of opaque builtins - D107943088 [3/4] Carry specialized parameter and return types on functions - D107943323 [4/4] Resolve real source locations by reusing the query transaction This is [2/4] — the core #8062 fix. It makes the consumer render correct names via zero-range typeshed/synthesized declarations; [4/4] later upgrades those to precise source locations. Reviewed By: rchen152 Differential Revision: D107943010 fbshipit-source-id: 68ace1623f851b52873b560b62de55a4c762eb2b

extract connection logic from server.rs Summary: there's a lot of this and it might be better in another file Reviewed By: samwgoldman Differential Revision: D103911833 fbshipit-source-id: 315d5ce174ff26655f4e254c71d6c4879ac3f443

extract TypeErrorDisplayStatus into its own module Summary: The `TypeErrorDisplayStatus` types, negotiation logic, V2 response derivation, and tests were all living in `server.rs`, which has grown to nearly 5000 lines. This code forms a self-contained feature — the custom LSP request and its versioned wire protocol — with minimal coupling to the rest of the server implementation. Extracting it into a dedicated `type_error_display_status.rs` module improves code organization and makes the feature easier to maintain and test in isolation. The module exports all public types and functions that were previously part of `server.rs`, so no behavior changes — just a cleaner separation of concerns. Changes: - New file `pyrefly/lib/lsp/non_wasm/type_error_display_status.rs` contains all TypeErrorDisplayStatus types, helper functions, and tests - `server.rs` now imports from the new module instead of defining these types locally - `stdlib.rs` updated to import TypeErrorDisplayStatus from the new module - Removed unused imports that were only needed for the moved code Reviewed By: samwgoldman Differential Revision: D103907675 fbshipit-source-id: ed305b7d71a43cde2bb72de572c218f5dd831074

Fix exponential memory blowup in dict literal type inference Summary: Deeply-nested dict literals (e.g. `{"a": {"b": {"c": ...}}}`) caused exponential memory growth during type inference, with a depth-25 literal consuming ~7.7 GB and triggering OOM. The root cause was `AnonymousTypedDictInner` storing both the individual `fields` and a pre-computed `value_type` (the union of all field types). Since `value_type` was a clone of the field types, each nesting level doubled the size of the type tree on `Clone`, producing O(2^N) memory usage. The fix removes the redundant `value_type` field and instead computes it on demand via `compute_value_type()`. This ensures the type tree is only as deep as the actual nesting, reducing pyrefly's total memory for a depth-25 dict from ~7.7 GB to ~239 MB. fixes https://github.com/facebook/pyrefly/issues/3286 Reviewed By: stroxler Differential Revision: D103479382 fbshipit-source-id: 6734b427217ba54e5a3276291811bb77f98654f6

Implement multi-connection architecture (#3218) Summary: Pull Request resolved: https://github.com/facebook/pyrefly/pull/3218 Multi-connection architecture for the TSP (Type Server Protocol) The commit splits the single monolithic TspConnection into a three-tier structure: 1. TspServer<T> — shared core holding the Arc<T> LSP server, the snapshot counter, and a registry of extra connections. All connections share this via Arc. 2. TspMainConnection<T> — wraps a TspConnection (via Deref) and owns the main stdio event loop. Only this type can: - Process LSP lifecycle events (delegating to the inner server) - Handle typeServer/connection requests (open/close extra connections) - Broadcast snapshotChanged notifications to all connections 3. TspExtraConnection<T> — also wraps TspConnection (via Deref). Spawned on demand over IPC named pipes. Can handle TSP query requests (getComputedType, resolveImport, etc.) but explicitly rejects connection management and LSP methods. Key mechanics: - Extra connections are opened via a new ConnectionRequest with type: "open" and an IPC pipe name. Each gets its own reader thread + select loop, terminating on a close signal or pipe disconnect. - snapshotChanged notifications are broadcast from the main connection to all extra connections, so every client stays in sync. - The server advertises typeServerMultiConnection in its experimental capabilities. - All TSP request handlers now call self.inner() (method) instead of self.inner (field), since the inner server is accessed through the shared TspServer. The result: multiple IDE clients can query the same pyrefly process concurrently over IPC without spawning separate server instances. Current status: once we land this stack, Pylance with TSP 0.4.0 will no longer work. I will wait on this until 0.5.0 is stable. Reviewed By: maggiemoss Differential Revision: D102228875 fbshipit-source-id: 2e86d10104b3a8400790a5c05a2963eb1ee4d855

add criterion microbenchmark harness Reviewed By: stroxler Differential Revision: D112005638 fbshipit-source-id: ff75dec279e5a20befe3a00cdf5a28259637ef9b

Resolve real source locations by reusing the query transaction Summary: The zero-range typeshed/synthesized declarations emitted earlier render the right names but break goto-definition and typeshed hover docs, which need a real `Node { uri, range }`. Give the converter a third resolver, `ExportLocationResolver`, that maps `(module, name)` to a source path+range (following re-exports), and use it to upgrade special-form/`typing` classes, `Quantified` TypeVars (PEP 695 → typeparam, legacy → variable), and imported/special functions whose `FuncId` lacks a `def_index` (e.g. `typing.overload`) to precise locations. Resolving a location demands the target module's exports, which demands its `Stdlib`. Rather than spinning up a fresh cold transaction per lookup — which starts with an empty stdlib and would panic in `get_stdlib` unless warmed by an extra `Require::Exports` run — the converter now reuses the single transaction that already produced the type. Computing a type necessarily populated that transaction's stdlib, so the export lookups are warm by construction: no warm-up run, no cold-start guard, and one transaction per query instead of one per resolved symbol. Conversion moves server-side (`type_at_position`/`expected_type_at_position`) where that transaction lives, collapsing the previous five TSP interface methods into two. ## Stack This stack fixes how Pyrefly's TSP backend converts internal types to the TSP wire format so Pylance renders real types instead of `Unknown` (microsoft/pylance-release#8062). Each diff is self-contained and passes the full `pyrefly_library` test suite on its own. - D107942955 [1/4] Emit the protocol-conformant lowercase `unknown` sentinel - D107943010 [2/4] Emit resolvable typing classes and TypeVars instead of opaque builtins - D107943088 [3/4] Carry specialized parameter and return types on functions - D107943323 [4/4] Resolve real source locations by reusing the query transaction This is [4/4] — builds on [2/4], upgrading its zero-range declarations to precise source locations and refactoring the server to one transaction per query. Reviewed By: rchen152 Differential Revision: D107943323 fbshipit-source-id: 69c4b56dc0629faf02a25121eb3be58162e4e59f

Resolve real source locations by reusing the query transaction (#3829) Summary: Pull Request resolved: https://github.com/facebook/pyrefly/pull/3829 With the converter able to resolve export locations ([1/2]), the server must supply a resolver that actually finds them. Resolving a location demands the target module's exports, which demands its `Stdlib`. Spinning up a fresh transaction per lookup starts cold and would panic in `get_stdlib` unless warmed by an extra `Require::Exports` run. Instead the server now reuses the single transaction that already produced the type. Computing a type necessarily populated that transaction's `Stdlib`, so the export lookups are warm by construction — no warm-up run, no cold-start guard, and one transaction per query instead of one per resolved symbol. The target handle is reached via `import_handle` so it inherits the source file's `SysInfo`; re-deriving it from the module path would pick a config-default `SysInfo` whose `Stdlib` was never computed and panic once the transaction holds more than one `SysInfo`. To put the resolver where that transaction lives, type conversion moves server-side: `type_at_position`/`computed_type_at_range`/`expected_type_at_position` now return the TSP wire type directly, collapsing the previous five `TspInterface` methods into two and removing the per-call `convert_type` on the connection. `lookup_export_location` becomes `pub(crate)` so the converter seam can reach it. Reviewed By: samwgoldman Differential Revision: D108667579 fbshipit-source-id: f04c66f6e9e62a11c987a8e816e1180a35d5ecdf

Ignore Pyrefly client settings in TSP mode Summary: When Pyrefly backs Pylance as its type server (TSP mode) rather than running as its own language server, Pylance still forwards the whole `pyrefly.*` VS Code settings block over LSP. Those settings configure the standalone Pyrefly extension, which Pylance owns in TSP mode, so they must have no effect. We hit this during dogfooding: a stale `pyrefly.disableTypeErrors: true` left over from running the extension was still read by the language server and silently suppressed all type errors, even though Pylance had asked us to compute them. Fix: model how the server is driven with a `ServerMode` enum (`LanguageServer` / `TypeServer`) threaded into `apply_client_configuration`, and skip the forwarded `pyrefly.*` block entirely in `TypeServer` mode. `Server` holds the mode so the runtime `did_change_configuration` and `workspace/configuration` handlers can forward it, while the standalone language-server entry point passes `LanguageServer`. Reviewed By: yangdanny97 Differential Revision: D112866377 fbshipit-source-id: c4eab0828318e761b22f6ee53f0efd49a2a5d843

Add PyTorch cold-start benchmark Summary: Add cold_start_go_to_definition benchmark measuring cold-start latency to first cross-file go-to-definition against pinned PyTorch source. Existing micro benchmarks are single-threaded deterministic. We need real-world walltime signal covering indexing threading and I/O on multi-gigabyte project to catch cold-start regressions. Reviewed By: yangdanny97 Differential Revision: D112006986 fbshipit-source-id: 7c44998e286cf67d6fe5a13db5d7a55b775f8c54

Teach the type converter to resolve export locations Summary: The TSP converter emits zero-range/synthesized declarations for typing special forms, TypeVars, and imported/special functions, which renders names but breaks goto-definition and typeshed hover. The fix needs a way to map an exported symbol back to its real source location, but that capability is independent of where the location data comes from. This diff adds that capability to the converter in isolation: a third `ExportLocationResolver` callback that maps `(module, name)` to a `(ModulePath, Range)` following re-exports. The converter uses it in three new helpers — `convert_quantified` (PEP 695 -> typeparam, legacy `TypeVar` -> variable), `function_declaration` (functions whose `FuncId` lacks a `def_index`, e.g. `typing.overload`), and `typing_class_declaration` (special forms / `typing` classes). When no resolver is supplied the behavior is unchanged, so the sole existing call site simply passes `None`. This is [1/2]; [2/2] wires a real resolver in the server by reusing the query transaction. Reviewed By: samwgoldman Differential Revision: D108667578 fbshipit-source-id: 44eea460b0b6a4ffd30b447d262fb98e3d66d067

create expected_type_at API Summary: This new API is necessary for a TSP operation: `typeServer/getExpectedType`. Originally implemented [here](https://github.com/facebook/pyrefly/pull/3672), redone in this diff, we would like to have access to the solver's expected type at a position. It is also useful in simplifying features like literal completion (rest of stack). I implement this by tracing it during solving. This works pretty well and according to my benchmarks does not increase memory or check speed noticeably. Alternative approach: re-solve / redo this logic per-request. I think this alternative is worse because it has potential for the behavior to fork from the solver's behavior. Reviewed By: stroxler Differential Revision: D107577810 fbshipit-source-id: df87bf6c9b26782c7eb58a30992fa5d84222c80d

return real expected types from TSP getExpectedType Summary: The typeServer/getExpectedType request piggy-backed on get_type_at_position, so it returned the computed type at the cursor — identical to getComputedType and useless as a distinct query. Clients asking "what type does this position need to be?" got "what type is it?" instead. Route the request through a new TspInterface::get_expected_type_at_position that asks for the contextually expected type: the parameter type for a call argument, the declared type for an annotated assignment / return / attribute target, etc. When no expected-type context applies (e.g. a bare `x = 42`, or the LHS of an assignment), it falls back to the computed type so the result is still meaningful. This reuses expected_type_at, the same helper that powers expected-type-aware completion, which combines call-argument inference with the solver's recorded expected-type traces. Because those two sources never overlap and the computed-type fallback covers the rest, getExpectedType now returns the union/annotation a value is checked against (e.g. int | str for foo(4) or x: int | str = 42) rather than the narrowed literal. Reviewed By: yangdanny97 Differential Revision: D107587108 fbshipit-source-id: c55fffbabcc2c34f9ae2dbb74623242a0cd6eb66

complete dict keys in an empty subscript Summary: Extend dict-key completion to the empty subscript `d[`, before any key string is typed — the point where a user most wants to see the available keys. There is no string literal at the cursor (the slice parses as an empty recovered expression that the driver treats as an identifier in expression position), so this is detected separately: when the cursor sits in a subscript's slice, offer the base's known keys. Since there is no surrounding string, the completions insert a quoted key (`"k"`). The key-collection logic (binding facets plus TypedDict fields) is shared with the existing string-literal path. Reviewed By: yangdanny97 Differential Revision: D107668647 fbshipit-source-id: 891ad4f9743c09a3a6abec0c872a3bcbe552c492

unify module prefix so __unknown__ files count under --public-only (#4157) Summary: Pull Request resolved: https://github.com/facebook/pyrefly/pull/4157 Make coverage's `module_prefix()` unconditional so `__unknown__` fallback modules get the `__unknown__.` prefix like every other module. Why: under `--public-only`, these files silently reported "0 of N typable" — symbols were stored bare while the public-FQN filter expects the prefixed form, so all got dropped. Reviewed By: yangdanny97 Differential Revision: D112166629 fbshipit-source-id: 4e9fbb540a311a26708bb79cfbe27d56f046f638

Add basic Sphinx cross-reference support in hover tooltips Summary: Adds support for Sphinx cross-reference directives (`:meth:`, `:class:`, `:func:`, `:attr:`, etc.) in hover tooltips. When hovering over a symbol with a docstring containing `:meth:`farewell``, the reference becomes a clickable `[farewell](file://...)` link that navigates to the definition. Unresolvable targets fall back to inline code formatting. Resolution uses `find_attribute_definition_for_base_type` to look up same-class attributes. Cross-file and qualified name references are not supported — only unqualified names within the same class. On WASM, all targets render as inline code since `file://` URLs aren't supported. Fixes #3277 Reviewed By: ndmitchell Differential Revision: D103338874 fbshipit-source-id: 84b8b742dda612f072672f5d94b67459737d72ea

Encode Sentinel as a SentinelLiteral ClassType Summary: `Sentinel` types were emitted as `sentinel` `BuiltInType`. The protocol restricts `BuiltInType.name` to a fixed sentinel set that does not include `sentinel`, so consumers render the off-spec name as `Unknown` — while the protocol already has a dedicated `SentinelLiteral` (class name plus defining location) for exactly this case. This emits a `ClassType` carrying a `SentinelLiteral`, built entirely from the sentinel's own `QName`. Unlike the `None`/`bool`/`int` cases, no stdlib class is threaded in: the sentinel's declaration and literal location are its own definition, so goto-definition points at the real sentinel class. Reviewed By: yangdanny97 Differential Revision: D111366758 fbshipit-source-id: 17969fd5633cc48764cc4f337bdacdca1bac4190

Fix type[TypeVar] TSP conversion to preserve INSTANTIABLE flag Summary: `type[X]` always denotes an instantiable class object, but the `Type(inner)` arm only re-applied `INSTANTIABLE` when the inner type converted to a `TspType::Class`; every other shape (e.g. a TypeVar, which converts to `Var`) fell through an `other => other` catch-all and lost the flag. Mark the converted inner type instantiable generically via a new `mark_instantiable` helper that exhaustively covers all TSP variants, so future variants are caught at compile time rather than silently losing the flag. Updates the regression test from the previous commit to expect `INSTANTIABLE`. fixes #4069 Reviewed By: yangdanny97 Differential Revision: D110904620 fbshipit-source-id: 4a28ca4207ad7b9655ec887df661cbb838cf2faa

getComputedType: document gaps Summary: See last diff for context. reproduces https://github.com/facebook/pyrefly/issues/4042 with more examples Reviewed By: maggiemoss Differential Revision: D111155587 fbshipit-source-id: a461c136ac4b3a7dd6dd0c2c82281e67d8c6ea7a

Fix cross-file references lost when CRLF/LF line endings differ #3237 (#3359) Summary: Pull Request resolved: https://github.com/facebook/pyrefly/pull/3359 When files on disk use CRLF but the editor sends LF-normalized content via did_open, byte offsets diverge between the on-disk exports and the in-memory definition. The exact byte-range comparison in local_references_from_external_definition fails for any definition after a line break, dropping cross-file references. The fix adds a line-number fallback to the comparison. When the exact byte range doesn't match, it compares the line number of the export location against the definition's line number. Line numbers are invariant across CRLF/LF differences, so this correctly matches definitions even when byte offsets differ. The fallback preserves the exact-range fast path for the common case and only activates when content encoding differs. fixes #3237 Reviewed By: yangdanny97, jvansch1 Differential Revision: D104303086 fbshipit-source-id: cb986039339c7785ba4a26fe0a3074514aefd1a8