pyrefly — Engineering Performance
32 engineers all time · Jan 2025 – Nov 2026 · built 2026-11-22 · GitHub
Performance snapshot
Today's rolling 90-day reading for pyrefly, compared with the start of the series. Pick a window to move that comparison point.
Avg. perf / dev / mo
+374.9%
1.60 → 7.58 ETV
Active engineers
±0%
16.0 → 16.0
Features
+0.1pp
30.1% → 30.2%
vs. Meta
2.2x
2.2x → 2.2x · +120% above
pyrefly vs. Meta
Per-engineer ETV for pyrefly against Meta as a whole. Both lines are 90-day rolling averages scaled to a 30-day month, so they share one axis and can be read against each other at any point. Pick a window to zoom the chart to it.
Performance Composition
Each month's output split by type of work: Features (new value), Maintenance (sustaining systems), Tests, Docs, and Fixes (rework). The yellow line is output per engineer, so when it rises each engineer is delivering more, whatever the team size did. Unit: Engineering Throughput Value (ETV).
Engineering capacity
Effective engineers behind pyrefly, against its pre-AI baseline. Each subject has its own: pyrefly's is 1.60 ETV / dev / mo, its first reading in Q1 2025. Per-engineer ETV divided by that gives a capacity multiple, and that multiple applied to the engineers active in the trailing 90 days turns it into engineer-equivalents. The line is the real headcount, so the gap between line and area is what the leverage is worth. Because each baseline is its own, every subject opens at 1.0x on its first day: multiples measure improvement and are not comparable between subjects.
Knowledge concentration
How dependent is this repo on a small number of engineers? Higher top-1 share = higher key-person risk.
Steven Troxler owns 25.0 % of commits.
Behind the numbers
Written summary of the work completed each month.
No monthly reports available yet.
Top engineers
Most impactful commits
Top 10 by ETV in the all-time window.
- 15.6ETVUpgrade typeshed Summary: Stack context: Upgrade pyrefly's bundled typeshed, and fix the pyrefly problems that the upgrade exposes. - This diff upgrades the bundled typeshed from `a7bcce8` (2026-07-07) to upstream `main` at `7030b26`, and updates the test expectations that depend on the stubs. - The bundled stubs are more than three months old. Upstream has stub fixes that users report as pyrefly bugs, for example the 3.15 `frozendict` fixes. The diffs below this one fix the pyrefly problems that the new stubs expose. - The test updates follow from the new stubs: moved line numbers in `builtins.pyi` and `typing.pyi`, the removed `builtins.ellipsis` alias, and the new `_SupportsReversed` protocol. The `unused-coroutine` test for a Protocol `Coroutine` now uses the bundled stubs directly, so it moves from a custom-typeshed test in `test/state.rs` to a `testcase!`. I ran `crates/pyrefly_bundled/update.py` with docify enabled. The script downloaded the archive URL directly, because the GitHub API rate limit blocks the default `main` lookup through fwdproxy. Reviewed By: stroxler Differential Revision: D126992287 fbshipit-source-id: 903c56ea456a8c65682a8eecd3c4de2998bb50d9Kyle Into · 75687253 · 2026-10-27
- 4.6ETVmigrate from lsp-types fork to gen-lsp-types 0.11.0 (#5032) Summary: Pull Request resolved: https://github.com/facebook/pyrefly/pull/5032 Mechanical conversion of `fbcode/pyrefly` off `lsp-types-yangdanny` onto vendored `gen-lsp-types 0.11.0` (`third-party/rust` update is a separate commit). `Cargo.toml`/`BUCK` deps use `lsp_types = { package = "gen-lsp-types", version = "0.11.0", features = ["url"] }` via `named_deps`. Notable API adaptations: `lsp_types::Url` -> `Uri`, flat request/notification types with `R::METHOD.as_str()`, PascalCase enum variants, flattened structs (`VersionedTextDocumentIdentifier`, `SymbolInformation`/`BaseSymbolInformation`), split response enums (`DefinitionResponse`/`TypeDefinitionResponse`/`ImplementationResponse`, single-item `CodeActionResponse`), `WorkspaceEdit.document_changes` as `Vec<DocumentChange>`, plus `SymbolInformation` Flat-golden key reorder (location now serializes first). Reviewed By: dtolnay, kinto0 Differential Revision: D122245430 fbshipit-source-id: ca62079b6315bcddbd49fc6cbb4c3e0affa4d589Sam Goldman · b3bfe98b · 2026-09-29
- 4.1ETVPort 9 TorchBench models with tensor shape annotations Summary: 9 new model ports with full shape tracking: tacotron2, llama, stargan, sam, tts_angular, drq, dlrm, deeprecommender, background_matting. Key improvements over initial ports: - sam: PromptEncoder[D, ES, MIC] fully typed, window partition/unpartition typed, ImageAttention generic over H/W, unified D across components - llama: RoPE chain typed (arange/outer DSL), KV cache typed with Optional start_pos, build_causal_mask typed - tacotron2: variadic Prenet[*Bs], full decoder forward with autoregressive loop - stargan: Generator and Discriminator generic over spatial dim S - drq: SquashedNormal[*EventShape] with typed distributions - dlrm: tensor fancy indexing verified with assert_type 11 type: ignore remaining (5× A1 algebraic, rest genuine limitations). Stub additions: torch.quantization, optim updates, F.interpolate symint. Reviewed By: stroxler Differential Revision: D97893786 fbshipit-source-id: 863984bee7092fdff92b5df46de35cb771c6ac9eAvik Chaudhuri · dc3b4870 · 2026-03-31
- 3.5ETVAdd github workflow for v1 issue ranking Summary: The full issue_ranker package, which is a 5-pass LLM pipeline for ranking pyrefly GitHub issues by priority: 1. GitHub Actions workflow (manual dispatch for now) 2. Issue collection which consists of: - code extraction from issue bodies - snippet repair via LLM - type checker execution (pyrefly/pyright/mypy) - status classification - relationship resolution, - dependency resolution 3. 5-pass LLM ranking pipeline: categorize (Haiku) → primer impact (deterministic error matching then LLM matching if that fails) → dependencies (Opus) → scoring (Sonnet) → final ranking (Opus, batched with retry) 4. Report generation: markdown + JSON output with V1 gap analysis 5. compare_typecheckers.py fixes — 10-min timeouts, pyrefly config cleanup, pyright path overrides (scipy takes too long because it has no config, so we override it) The ranking pipeline uses 8 weighted signals: false positive impact (x3), performance (x3), team priority labels (x2), false negatives/spec compliance (x2), actionability (x2), IDE/usability (x1.5), primer breadth (x1.5), and adoption ecosystem labels (x1.5). Team priority labels (P0/P1/P2) contribute ~20% of the weighting. On the full 391-issue run, the top 20 (critical tier) are dominated by type inference bugs and false positives that affect many primer projects — these would rank highly regardless of labels. Local Results: - Top 10 is 70% the same with or without labels - Labels boost V1 overlap from 29% → 38% (19 → 25 out of 65) with labels P2230410941 without P2230411326 The V1 overlap data from the comparison: WITH labels: 25/65 V1 issues in top 65 (38%) WITHOUT labels: 19/65 V1 issues in top 65 (29%) The 25 V1 issues captured (with labels): #105, #630, #1252, #1286, #1321, #1486, #1518, #1763, #1982, #2043, #2105, #2309, #2370, #2382, #2419, #2445, #2452, #2515, #2517, #2519, #2520, #2522, #2610, #2611, #2616 The 40 V1 issues missed (not in top 65): #43, #219, #317, #383, #385, #397, #793, #795, #881, #908, #920, #1078, #1089, #1159, #1336, #1340, #1422, #1440, #1474, #1728, #1956, #2039, #2064, #2065, #2132, #2139, #2184, #2199, #2207, #2219, #2227, #2303, #2304, #2319, #2386, #2454, #2508, #2535, #2558, #2621 So the ranker agrees with about 38% of the team's V1 picks. NOTE: I need to test this as an actual github workflow. I cannot be 100% certain that the workflow works end to end without kicking off a manual run there. Reviewed By: yangdanny97 Differential Revision: D95686786 fbshipit-source-id: 8de3332306be7d69875a4dfca9f8979e7d19c846Zeina Migeed · c7c846bc · 2026-03-11
- 3.2ETVAdd Tensor type and subtyping rules Summary: [4/N] of a stack implementing tensor shapes in Pyrefly. Adds foundational tensor shape type system support for torch.Tensor: Type System: - Type::Tensor(Box<TensorType>) for representing shaped tensors - TensorType, TensorShape for shape representation with literal and symbolic dimensions - tensor.rs with broadcast_shapes, parsing, and canonicalization Subtyping Rules: - Tensor rank and dimension matching in subset.rs - Shape compatibility with symbolic arithmetic expressions (N + M, N * 2, etc.) - Shapeless Tensor compatible with any shaped Tensor - TypeVarTuple support for variadic shapes (with arbitrary prefix and suffix) Core Features: - .shape property returns tuple of Literal[n] or Dim[N] types - Integer indexing, slice indexing being shape-aware - Arithmetic operations preserve shape via Self return type (TensorInstance) Minimal torch fixture with Tensor class, arithmetic, comparison, and unary operators. No constructors (torch.randn, etc.) or meta_shape inference - those come in subsequent commits. Reviewed By: samwgoldman Differential Revision: D92527702 fbshipit-source-id: c8716058d88513424678a885657e59c139f65550Avik Chaudhuri · 02dcdc40 · 2026-02-15
- 3.1ETVfix dunder all references #4344 Summary: Fixes part of https://github.com/meta/pyrefly/issues/4344 fix go to definition for `__all__` items. stack on https://github.com/meta/pyrefly/issues/5587 - [x] I am a human, and I fully understand the code I am submitting. X-link: https://github.com/meta/pyrefly/pull/5588 Reviewed By: stroxler Differential Revision: D130539715 Pulled By: kinto0 fbshipit-source-id: a77d1ff5d7555d308bce4c723bbb3ded1658c0edAsuka Minato · 9b387757 · 2026-11-18
- 3.1ETVAdd plumbing for literal-capturing Flag-bound type vars Summary: Pyrefly needs literal-preserving `Flag` type parameters, but adding a new restriction kind creates broad exhaustive-match fanout before source syntax should construct it. Model `Flag` as a passive `Restriction` with its small builtin domain beside `Restriction`, and mechanically teach display, simplification, solving, queries, and reporting to preserve or project that restriction. The end goal here is to allow us to pass flags to the type-level shape DSL with code that looks like this: ``` def f[B: Flag[bool]](b: B) -> Tensor[takes_a_bool(B)] ``` The stubs use this pattern extensively to avoid the need for a combinatorial number of overloads for all kinds of transforms that take flag values. This commit intentionally does not recognize `Flag` annotations or change inference; the following activation commit provides the single source construction point. Splitting passive representation from activation keeps the mechanical type-system fanout reviewable and behavior-neutral. Reviewed By: rchen152 Differential Revision: D114061238 fbshipit-source-id: c9a3acbff0ef9083a75bcadeb57fcaac1c8c1955Steven Troxler · 47ef3b51 · 2026-08-21
- 2.9ETVAdd attr and attrs stubs to test directory Summary: As discussed in D90205508, we will first add the stubs (attr and attrs) and then followup with testcases. Before adding the unittests, we will setup the plumbing so that the unittest site package path can access the stubs. We will also add a macro like the one for pydantic and django. Reviewed By: yangdanny97 Differential Revision: D90268993 fbshipit-source-id: 01a04045147a89d2ec285dc9e42c12f60bbb9ecdZeina Migeed · c3190f76 · 2026-01-08
- 2.9ETVPass a `TypeEnv` to union simplification Summary: Context: this diff is part of the work that makes `Class` a small `Copy` ID, so that cloning and dropping a type touches no shared reference count. Atomic reference counting on `Class` is the main bottleneck when checking with many threads: in an earlier prototype, making `Class` an ID cut CPU time by about 40% at 192 threads on a large codebase. An ID can't carry a class's name or location, so every piece of code that reads them has to go through an environment (`TypeEnv`) that looks the class up. Unions and intersections keep their members in a canonical order, which compares classes by name and then by where they are defined. Today that order is `Ord` on `Type`, which reads both from the `Class`. Once `Class` is an ID, the sort has to read them through an env, so union simplification needs one. `unions`, `unions_with_literals` and `intersect` now take a `TypeEnv`, and so does everything that calls them: type-argument substitution, the upper bound of a restriction, binding a method to an instance, and the solver's expansion, forcing and bound solving. `unions_with_literals` takes the `Stdlib` from the env instead of a separate argument. Inside the solver, `Subset` passes its `TypeOrder` and the `AnswersSolver` passes itself. `intersect` does not use its env yet; a later diff sorts intersection members with it. No behavior change: the env is only passed along, and the `Stdlib` it supplies is the one the callers used to pass. ___ landed-with-radar-review Differential Revision: D130053905 fbshipit-source-id: a2344e0bfdf2172b7aa2b21968aab80c4c2744c9Sam Goldman · e6baaeb0 · 2026-11-16
- 2.9ETVPort 7 TorchBenchmark models with tensor shape annotations Summary: Ports Demucs, DenseNet, LearningToPaint, SoftActorCritic, SpeechTransformer, Super-SloMo, and U-Net from TorchBenchmark with full tensor shape annotations. Each model demonstrates end-to-end shape inference through complex architectures including pooling, upsampling, LSTM, GLU, PixelShuffle, and None indexing. Also adds fixture stubs for torch.distributions (used by SoftActorCritic) and additional torch/__init__.pyi and nn/functional.pyi function stubs needed by the ported models. Reviewed By: yangdanny97 Differential Revision: D97134636 fbshipit-source-id: 672256498c414706845c51ab5c500d519a2b7328Avik Chaudhuri · 39ad2430 · 2026-03-21