github.com-facebook-pyrefly

Port 9 TorchBench models with tensor shape annotations Summary: 9 new model ports with full shape tracking: tacotron2, llama, stargan, sam, tts_angular, drq, dlrm, deeprecommender, background_matting. Key improvements over initial ports: - sam: PromptEncoder[D, ES, MIC] fully typed, window partition/unpartition typed, ImageAttention generic over H/W, unified D across components - llama: RoPE chain typed (arange/outer DSL), KV cache typed with Optional start_pos, build_causal_mask typed - tacotron2: variadic Prenet[*Bs], full decoder forward with autoregressive loop - stargan: Generator and Discriminator generic over spatial dim S - drq: SquashedNormal[*EventShape] with typed distributions - dlrm: tensor fancy indexing verified with assert_type 11 type: ignore remaining (5× A1 algebraic, rest genuine limitations). Stub additions: torch.quantization, optim updates, F.interpolate symint. Reviewed By: stroxler Differential Revision: D97893786 fbshipit-source-id: 863984bee7092fdff92b5df46de35cb771c6ac9e

Add Tensor type and subtyping rules Summary: [4/N] of a stack implementing tensor shapes in Pyrefly. Adds foundational tensor shape type system support for torch.Tensor: Type System: - Type::Tensor(Box<TensorType>) for representing shaped tensors - TensorType, TensorShape for shape representation with literal and symbolic dimensions - tensor.rs with broadcast_shapes, parsing, and canonicalization Subtyping Rules: - Tensor rank and dimension matching in subset.rs - Shape compatibility with symbolic arithmetic expressions (N + M, N * 2, etc.) - Shapeless Tensor compatible with any shaped Tensor - TypeVarTuple support for variadic shapes (with arbitrary prefix and suffix) Core Features: - .shape property returns tuple of Literal[n] or Dim[N] types - Integer indexing, slice indexing being shape-aware - Arithmetic operations preserve shape via Self return type (TensorInstance) Minimal torch fixture with Tensor class, arithmetic, comparison, and unary operators. No constructors (torch.randn, etc.) or meta_shape inference - those come in subsequent commits. Reviewed By: samwgoldman Differential Revision: D92527702 fbshipit-source-id: c8716058d88513424678a885657e59c139f65550

Add github workflow for v1 issue ranking Summary: The full issue_ranker package, which is a 5-pass LLM pipeline for ranking pyrefly GitHub issues by priority: 1. GitHub Actions workflow (manual dispatch for now) 2. Issue collection which consists of: - code extraction from issue bodies - snippet repair via LLM - type checker execution (pyrefly/pyright/mypy) - status classification - relationship resolution, - dependency resolution 3. 5-pass LLM ranking pipeline: categorize (Haiku) → primer impact (deterministic error matching then LLM matching if that fails) → dependencies (Opus) → scoring (Sonnet) → final ranking (Opus, batched with retry) 4. Report generation: markdown + JSON output with V1 gap analysis 5. compare_typecheckers.py fixes — 10-min timeouts, pyrefly config cleanup, pyright path overrides (scipy takes too long because it has no config, so we override it) The ranking pipeline uses 8 weighted signals: false positive impact (x3), performance (x3), team priority labels (x2), false negatives/spec compliance (x2), actionability (x2), IDE/usability (x1.5), primer breadth (x1.5), and adoption ecosystem labels (x1.5). Team priority labels (P0/P1/P2) contribute ~20% of the weighting. On the full 391-issue run, the top 20 (critical tier) are dominated by type inference bugs and false positives that affect many primer projects — these would rank highly regardless of labels. Local Results: - Top 10 is 70% the same with or without labels - Labels boost V1 overlap from 29% → 38% (19 → 25 out of 65) with labels P2230410941 without P2230411326 The V1 overlap data from the comparison: WITH labels: 25/65 V1 issues in top 65 (38%) WITHOUT labels: 19/65 V1 issues in top 65 (29%) The 25 V1 issues captured (with labels): #105, #630, #1252, #1286, #1321, #1486, #1518, #1763, #1982, #2043, #2105, #2309, #2370, #2382, #2419, #2445, #2452, #2515, #2517, #2519, #2520, #2522, #2610, #2611, #2616 The 40 V1 issues missed (not in top 65): #43, #219, #317, #383, #385, #397, #793, #795, #881, #908, #920, #1078, #1089, #1159, #1336, #1340, #1422, #1440, #1474, #1728, #1956, #2039, #2064, #2065, #2132, #2139, #2184, #2199, #2207, #2219, #2227, #2303, #2304, #2319, #2386, #2454, #2508, #2535, #2558, #2621 So the ranker agrees with about 38% of the team's V1 picks. NOTE: I need to test this as an actual github workflow. I cannot be 100% certain that the workflow works end to end without kicking off a manual run there. Reviewed By: yangdanny97 Differential Revision: D95686786 fbshipit-source-id: 8de3332306be7d69875a4dfca9f8979e7d19c846

Port 7 TorchBenchmark models with tensor shape annotations Summary: Ports Demucs, DenseNet, LearningToPaint, SoftActorCritic, SpeechTransformer, Super-SloMo, and U-Net from TorchBenchmark with full tensor shape annotations. Each model demonstrates end-to-end shape inference through complex architectures including pooling, upsampling, LSTM, GLU, PixelShuffle, and None indexing. Also adds fixture stubs for torch.distributions (used by SoftActorCritic) and additional torch/__init__.pyi and nn/functional.pyi function stubs needed by the ported models. Reviewed By: yangdanny97 Differential Revision: D97134636 fbshipit-source-id: 672256498c414706845c51ab5c500d519a2b7328

Add attr and attrs stubs to test directory Summary: As discussed in D90205508, we will first add the stubs (attr and attrs) and then followup with testcases. Before adding the unittests, we will setup the plumbing so that the unittest site package path can access the stubs. We will also add a macro like the one for pydantic and django. Reviewed By: yangdanny97 Differential Revision: D90268993 fbshipit-source-id: 01a04045147a89d2ec285dc9e42c12f60bbb9ecd

TypeVar variant for arithmetic Summary: Add torch_shapes.TypeVar and TypeVarTuple classes that support arithmetic operators (N + 1, N * 2, etc.) at Python runtime, unlike typing.TypeVar which raises TypeError. These use __class__ = typing.TypeVar so that isinstance checks pass and Generic[N] works correctly. On the pyrefly side, extend the TypeVar/TypeVarTuple special export recognition to accept torch_shapes as a valid defining module, so torch_shapes.TypeVar is treated identically to typing.TypeVar by the type checker. Also add int_type_var model variants (nanogpt, gptfast) that use torch_shapes.TypeVar instead of PEP 695 syntax, with both type-checked and runnable versions, plus comprehensive runtime and type-checking tests. Reviewed By: stroxler Differential Revision: D95282862 fbshipit-source-id: ec07dbb9efc94b653d6afc4ec9bf6b3d821580f0

Add Size and Dim types for dimension tracking Summary: [3/N] of a stack implementing tensor shapes in Pyrefly. Introduce type system support for concrete and symbolic integers appearing in type positions. We will use these eventually to model dimensions in tensor shapes, but they also form a standalone feature. - Type::Size for dimension values (SizeExpr expressions). This is internal, and corresponds to int literals like 1, int symbols like N, and arithmetic expressions on ints like N + 1. - Type::Dim for symbolic integer annotations (Dim[N], Dim[N+1]). This is user-facing, and generalizes Literal[1]. - Arithmetic operations on Dim types (+, -, *, //). If n: Dim[N] then n + 1: Dim[N+1]. - Subtyping rules for Size and Dim types. Here we have a (semi-)decision procedure for equality of arithmetic expressions, which amounts to canonicalization via a set of arithmetic identities, followed by structural equality. Importantly, we never ask for satisfiability (i.e., does there exist a substitution that makes two expressions equal); rather, two expressions are equal only when they are universally equal (for all substitutions). - torch_shapes fixture with Dim class. Reviewed By: samwgoldman Differential Revision: D92527699 fbshipit-source-id: 4d593e1d9e6b9f1a4193d24f5dd464670f3740d2

Update TSP to version 0.2.0 (#1010) Summary: This PR updates TSP to match the newest internal version that Pylance is using. It eliminates the following requests: - `SearchForTypeAttributes` - `GetTypeAttributes` - `GetSymbolsForFile` - `GetSymbol` - `GetFunctionParts` It eliminates the `Attribute` class and replaces it with the already existing `Symbol` idea. It adds two new requests - `GetSymbolsForType` that can find the symbols for a type. - `GetSymbolsForNode` that can find the symbols scoped by a node. These two new requests handle the situations required by the 4 of the original functions. `GetFunctionParts` was eliminated entirely as it was printing out parameters and such and could be reimplemented with just the `GetRepr` request. This PR is a prerequisite for adding the handlers for these requests. Pull Request resolved: https://github.com/facebook/pyrefly/pull/1010 Reviewed By: stroxler Differential Revision: D81353800 Pulled By: kinto0 fbshipit-source-id: 656fab5331dba768ede9cf830e3787bd300f9ef6

port-model: add 7 typed port examples as eval baseline Summary: Adds seven Tensor[]-shaped ports under fbcode/pyrefly/test/tensor_shapes/models/: mobilenetv2, mixtral_moe, wavernn, openpose, apg, finalmlp, masknet. Each was selected as the highest-scoring round from the v7 multi-round eval batch (R1 blind → R2 self-reflection → optional R3) using the verify_port.sh metric (shaped-vs-bare ratio, receipt coverage, smoke tests). They serve two purposes: 1. Reference ports demonstrating the skill's intended output across architectures (vision, MoE transformer, RNN, pose estimation, recommendation MLPs). 2. A baseline corpus future skill iterations can be evaluated against without re-burning compute on the v7 multi-round flow. Also lifts three stub additions surfaced by the mobilenetv2 port into fixtures/torch/nn/__init__.pyi: Conv2d.bias, GroupNorm.weight, GroupNorm.bias. Without these, the init-loop pattern (`for m in self.modules(): if isinstance(m, nn.Conv2d): m.bias`) fails to typecheck. Reviewed By: stroxler Differential Revision: D102635653 fbshipit-source-id: 1508cc0bffd34726a22a3706c84e792890ad91e0

move server out of commands/ Summary: commands/lsp.rs is huge - we should split it. this is the first part: move the server logic away from commands. from here, we can start splitting the server pieces into respective parts. Reviewed By: grievejia Differential Revision: D78431426 fbshipit-source-id: c34f64502da29471437ff899d51534b7bab37310

Add Two-pass LLM classification with PR diff attribution for primer classification (#2539) Summary: This is another iteration on our mypy primer classifier work. There are a few bugs and improvements we can make. Specifically - The verdict contradicts the message. Solution: Separate the concerns. One pass for analyzing the diff and coming up with the message, and then a light weight pass to read the message and determine the verdict. - Include PR information to explain how they contributed to those changes - linkify and improve formatting for messages. Now we have a table that describes the errors per project, as well as a high level overall comment on next step suggestions Pull Request resolved: https://github.com/facebook/pyrefly/pull/2539 Reviewed By: yangdanny97 Differential Revision: D94280120 Pulled By: migeed-z fbshipit-source-id: e41a332980e4cf376122f58aec10ed511da57875

Split mod.rs into multiple files Summary: The `pysa/mod.rs` is getting quite large, since most of the implementation of the export to pysa is in that file. We plan to do even more work in the pysa export in follow-ups. To help with development, let's split that file into multiple files so it's easier to read and review. Reviewed By: tianhan0 Differential Revision: D83346445 fbshipit-source-id: b121a8665c67084db030f614f19e12843bfa735e

Replace UpgradeLock with lock-free atomics for module state Summary: Replace `UpgradeLock<Step, ModuleDataInner>` in `ModuleDataMut` with a lock-free scheme using `arc_swap::ArcSwapOption<T>` and atomics to eliminate read-side contention on the module data state lock during transactions. Like the existing `ModuleData`/`ModuleDataMut` split, moving to atomics requires splitting more data types into immutable and mutable representations, since the mutable form has a different representation (atomic types instead of plain values). `ModuleDataInner` is replaced by `ModuleState` (frozen, plain values stored in committed `ModuleData`) and `ModuleStateMut` (lock-free atomics used during transactions in `ModuleDataMut`). Similarly, `Steps` (frozen, `Option<Arc<T>>` fields) is paired with `StepsMut` (mutable, `ArcSwapOption<T>` fields). `Dirty`/`AtomicDirty`, `Epoch`/`AtomicEpoch`, and `Require`/`AtomicRequire` follow the same pattern. Since the lock-free design relies on atomic ordering for correctness, the public API of `module.rs` encapsulates the ordering protocol. `ComputeGuard` and `CleanGuard` provide RAII-scoped access for step computation and cleaning respectively, ensuring the release/acquire sequences are correct. `StepsMut` lives in `steps.rs` alongside its frozen counterpart and is also usable standalone for isolated computation (e.g., `report_timings`). Key correctness properties: - Writer ordering: store step data via ArcSwap, then release-store `current_step`. Readers acquire-load `current_step` before reading step data, forming a happens-before chain. - Clean ordering: reset step data and `current_step` with relaxed stores, then release-store `checked` epoch. Readers acquire-load `checked` first, so the relaxed writes are visible. - Key invariant: no epoch changes during reads. A new epoch is only started by `run_step()`, and all read paths (`lookup_answer`, `lookup_export`, `demand`) execute within a single step. Once a reader observes `checked == now`, this remains true for the duration of the read. - Dirty flags use atomic swap (`take_all`) during clean to atomically read and clear all flags in one operation. Any flag set after the swap survives for the next clean cycle, preventing a race where a flag set between check and clear is silently lost. - `try_mark_deps_dirty` uses a CAS to set the deps flag, then re-checks the `computed` epoch. If the module was computed between the initial check and the CAS, the flag is undone via `clear_deps`. This prevents spurious recomputation from a TOCTOU race. - `ExclusiveLock<Step>` retained for scheduling mutual exclusion (prevents duplicate computation), orthogonal to data access. Reviewed By: grievejia Differential Revision: D93615408 fbshipit-source-id: c07e6d195da9f3478190f5c53aea2e5e94da8a1c

Fix suppression matching for f/t-strings Summary: `# pyrefly: ignore` comments placed above or on the closing line of a multi-line f/t-string now suppress errors inside the string. Extracts f-string ranges from the AST at the `Errors` level and plumbs them into `ErrorCollector::collect_into` to check suppression at the f-string's start and end lines in addition to the error's own line. Now, if the error suppression comment applies to either the first or last line of the f-string, all errors in the f-string are suppressed. Reviewed By: stroxler Differential Revision: D95964858 fbshipit-source-id: 29c693b3010db66e1f90a17dec2e674590ab4579

Add type related messages to TSP for fetching types at nodes and declarations (#2730) Summary: This adds the next set of messages for TSP to Pyrefly. These are the type related messages. These are really the main piece of information in TSP Pull Request resolved: https://github.com/facebook/pyrefly/pull/2730 Test Plan: Added a number unit and integration tests. Also tested this using Pylance as it's pretty close to being usable :) Here's it working in Pylance: <img width="1048" height="227" alt="image" src="https://github.com/user-attachments/assets/929a7a4c-74d4-4b4a-88d4-dea500bdb994" /> Same Pylance, using a Pylance specific feature (restructured text support) but with Pyrefly as the backend: <img width="1088" height="332" alt="image" src="https://github.com/user-attachments/assets/eea5584e-df0a-4334-a2d1-66bb5a265020" /> /cc kinto0 Reviewed By: yangdanny97 Differential Revision: D95873469 Pulled By: kinto0 fbshipit-source-id: 6273f85a3b4239ac3ab2f272197bcf1eb0e81adf

Add scikit image stubs Summary: Adds scikit image stubs to Pyrefly bundle. Reviewed By: connernilsen Differential Revision: D88759422 fbshipit-source-id: 6e1b26af211576c5886ab7259c5a95898a3c2163

Add GleanPredicate trait Summary: Refactoring to include the function GLEAN_name in a trait and remove some duplicated code in D91475072 Reviewed By: phlalx Differential Revision: D91475076 fbshipit-source-id: c65f53ab53324d6c4bfa1962c24b9194cd600946

Add botocore stubs to Pyrefly bundle. Summary: We are also going to include botocore-stubs along with bundled Pyrefly. Reviewed By: kinto0 Differential Revision: D88660232 fbshipit-source-id: 7833d2448b256f7c38f8e1b2f6090ac2f93fed42

Rewrite model ports with generic blocks and recursive patterns Summary: Rewrite model ports to use class-level type parameters, ModuleList with recursive chain patterns, and overload for base/recursive cases: - bert: super() cleanup - dcgan: DCGAN config class, GenUpStage/DiscDownStage generic blocks, recursive _chain with overload and exponential return types - demucs: generic EncoderBlock/DecoderBlock, overload on center_trim, recursive encoder-decoder with skip connections - densenet: _dense_chain as free function with overload - gptfast: remove duplicate license, Optional cleanup - resnet: recursive _chain with overload, ShapePreservingActivation - soft_actor_critic: remove trailing whitespace - super_slomo: major rewrite with ModuleList + recursive patterns, separate encode/decode ModuleLists - unet: ModuleList recursive pattern with skip connections Also adds test_overload_narrowing.py and test_recursive_skip.py. Reviewed By: stroxler Differential Revision: D97425585 fbshipit-source-id: 2f5c0c7db4d68e42a027411308e3c543f78a9980

Add StrictInt stub definitions to testsuite Reviewed By: rchen152 Differential Revision: D84558282 fbshipit-source-id: 9359dd0be35b0453f2da98e76b24fc103852633c