github.com-facebook-buck2

Add profile options to all commands Summary: This adds a common set of flags to all commands that support enabling profiling based on regexes that match stringified forms of StarlarkEvalKind. There's more documentation on the flags on the --profile-patterns flag in CommonStarlarkOptions. But basically: --profile-patterns: accepts regex patterns of evaluations to enable profiling --profile-patterns-mode: profiling mode to use --profile-patterns-output: root path to write profiling data. every individual evaluation gets its profiling data written to a separate file. we write to a subdir of this with similar format to buck-out/log (i.e. datetime+uuid) we additionally write an all_keys.list file listing all the evaluation names we profiled. This works by (1) setting up profile configuration that StarlarkEvaluatorProvider uses to enable profiling as appropriate and then (2) adding a ProfileEventListener that gets profile_collected calls as evaluations are finished. We already consistently use StarlarkEvaluatorProvider to get the enabling of profiling. The remaining diffs in the stack ensure that we consistently finalize profiles for all evaluations. Reviewed By: JakobDegen Differential Revision: D64281578 fbshipit-source-id: b09824a94d1a596216fa9d30e761436492e63b39

Add errorformat lib Summary: rfc: https://fb.workplace.com/groups/buck2dev/permalink/4110348932586481/ This diff adds the errorformat parsing lib. Since I did this development first locally using git, it is hard for me to split the diffs. However, I added comprehensive tests, it should be ok. Reviewed By: Will-MingLun-Li Differential Revision: D82085318 fbshipit-source-id: 4a5f0cde6bb37a504e126bd70f5002de7d950167

fix: a lot of instances of clippy::needless_lifetimes (#944) Summary: This doesn't get all of them because: 1. there are so many of them 2. the lint is pretty jank Part of https://github.com/facebook/buck2/issues/943 Pull Request resolved: https://github.com/facebook/buck2/pull/944 Reviewed By: JakobDegen Differential Revision: D74271655 Pulled By: dtolnay fbshipit-source-id: c7e41c47248d4be1f5d33a14b676ab35fc5180e2

cli: Allow `--isolation-dir` globally Summary: Let's see what CI says at least? Reviewed By: IanChilds Differential Revision: D87866823 fbshipit-source-id: c04e23ac1dc764a35f58e50286bba436e51740c0

Add `--agent-context` CLI flag with buckconfig-driven validation Summary: RFC: https://fb.workplace.com/groups/buck2dev/permalink/4302138863407486/ Adds a new `--agent-context key=value` CLI flag (modeled after `--client-metadata`) that lets AI agents pass structured context like `intent=fix`, `attempt=2`, or `prior_error=missing_target`. On the server side, a buckconfig-driven schema (`[agent_context]` / `[agent_context#field]` sections) allows enforcing required fields, allowed values, and unknown-key rejection for specific client IDs, while non-enforced clients pass data through without validation. Reviewed By: jtbraun Differential Revision: D99735398 fbshipit-source-id: 1c3e7d0eacae1d92b18068ad5e8fd472fd4d2d39

errors: Remove test-related anyhow usage Summary: We still have a bunch of anyhow usage in test related code. This commit removes ~all of that This is overwhelmingly find-and-replace and other very simple stuff, nothing really interesting here Differential Revision: D88890953 fbshipit-source-id: 8d5299acf25590299314528925a3e46cc2a04560

Introduce pagable serialization framework Summary: This introduces the `pagable` serialization framework, which extends serde with the ability to preserve Arc sharing semantics across serialization boundaries. This is a foundational component of the "Infinite Graph" initiative to enable graph hydration for buck2. The framework allows buck2 to: 1. Serialize/deserialize its internal computation state while preserving the extensive Arc sharing in its data structures 2. Initialize daemon state from previously computed results (bulk graph hydration) 3. Load partial graphs on-demand to reduce memory usage 4. Share computation data across machines for daemon reuse Unlike standard serde which deep-copies Arc contents, pagable tracks Arc identity during serialization and restores the same sharing structure during deserialization. This is critical for buck2's memory efficiency and correctness. **pagable crate:** Core traits in `traits.rs`: - `PagableSerializer` / `PagableDeserializer` - context traits that implementations must provide, offering both serde serialization and pointer stashing - `PagableSerialize` / `PagableDeserialize` - traits for types that can be serialized/deserialized with the framework - `Pagable` - convenience trait combining both with Send + Sync + Debug + 'static Built-in implementations for: - Primitives (u8-u128, i8-i128, f32, f64, bool, String, Duration) - Standard types (Option, Box, PhantomData, tuples up to 5 elements) - Collections (Vec, Box<[T]>, BTreeMap, arrays up to 32 elements) - serde_json::Value Testing utilities: - `TestingSerializer` / `TestingDeserializer` - in-memory implementations with TypeId tracking to catch type mismatches during stash/unstash **pagable_derive crate:** Derive macros: - `#[derive(Pagable)]` - derives both PagableSerialize and PagableDeserialize - `#[derive(PagableSerialize)]` - derives just serialization - `#[derive(PagableDeserialize)]` - derives just deserialization Field attributes: - `#[pagable(flatten_serde)]` - serialize field using serde directly - `#[pagable(skip)]` - skip field during serialization - `#[pagable(discard = "expr")]` - skip serialization, use expr for deserialization - `#[pagable(bound = "...")]` - custom trait bounds for generics Reviewed By: christolliday Differential Revision: D88400546 fbshipit-source-id: 85e9baf8b313e6217a68efd47cf64ae599d9540a

update platform010 & platform010-aarch64 symlinks Summary: * `__rust_no_alloc_shim_is_unstable_v2` replaces `__rust_no_alloc_shim_is_unstable`. It's a no-op function rather than a static data member. * Fix clippy errors from if-let chains * Sprinkle `'_` in places to make ambiguous lifetimes lint happy * `nonnull_provenance` feature is stable * `result_flattening` feature is stable * `file_lock` feature is stable * `extract_if` now accepts a range as the first argument * `duration_constructors` split into `duration_constructors_lite` Reviewed By: diliop Differential Revision: D80307486 fbshipit-source-id: 6285ea8416d34d1a9fa7bff1d564176a7542c42c

feat: implement bytes type Summary: Implement the Starlark `bytes` type per the spec at https://github.com/bazelbuild/starlark/blob/master/spec.md#bytes. The `bytes` type is an immutable sequence of bytes (integers in range 0-255). It supports: - Literal syntax: `b"hello"`, `b'hello'`, `b"""hello"""`, `rb"\n"` (raw) - Constructor: `bytes(b"...")`, `bytes("str")` (UTF-8 encode), `bytes([int...])` (from iterable) - Indexing: `b"abc"[0]` returns `97` (integer, not 1-byte bytes) - Slicing: `b"hello"[1:3]` returns `b"el"` - Containment: `97 in b"abc"`, `b"bc" in b"abcd"` - Concatenation (`+`), repetition (`*`), comparison, hashing - `str(b"...")` decodes UTF-8 with U+FFFD replacement for invalid sequences - `repr(b"...")` returns byte literal syntax - `ord(b"A")` returns `65` - `.elems()` method returning iterable of 1-byte bytes objects - Not directly iterable (use `.elems()`) Reviewed By: JakobDegen Differential Revision: D105587378 fbshipit-source-id: 3b7d2444101c94e94444d428577213ddec8cc5c9

Require error tag for buck2_error! macro Summary: Last part of the code where `buck2_error` can be created. Have `buck2_error!` macro take in an ErrorTag instead of an array, making it possible to enforce an ErrorTag (Since ErrorTag can't be set as empty but we can't enforce a non-empty array at compile time unless there's something I'm not aware of). All usage of this macro have at most 1 ErrorTag anyways so this doesn't make a difference at the moment, and tags can be added as an array through the `.tag()` function Reviewed By: JakobDegen Differential Revision: D67998689 fbshipit-source-id: 397fbc272f05d97713ec66284c563035e516e46a

Add PagableArc and storage abstraction Summary: Add smart pointer types that support transparent paging of data to/from storage: - `PagableArc<T>`: Reference-counted pointer where data can be paged out to storage when memory pressure is high - `PinnedPagableArc<T>`: Reference-counted pointer that guarantees data stays in memory, with direct Deref access - `PinnedPagableArcBorrow<'a, T>`: Borrowed reference avoiding refcount manipulation for short-lived access The storage abstraction includes: - `DataKey`: 128-bit unique identifier for stored data - `PagableStorageHandle` trait: Interface for storage backends to persist and retrieve paged-out data - `PagableDeserializer::storage()`: Allow deserializers to access storage for nested paging Also removes unused dependencies from the BUCK file (bytemuck, dashmap, futures, fxhash, sled, libc). Differential Revision: D88924819 fbshipit-source-id: b4f31b2f0e8b4a330a481de638c642ac9d81f3f2

artifacts: Attach a `'v` lifetime to `DeclaredArtifact` Summary: Declared artifacts have shared mutability during an analysis evaluation. This isn't all that uncommon, but what is uncommon is that they achieve that by just storing an `Rc`: https://www.internalfb.com/code/fbsource/[72d70f64f6a9b1869a176e70a2ad2d6d95eb5729]/fbcode/buck2/app/buck2_artifact/src/artifact/artifact_type.rs?lines=341 We're not normally in the business of doing that. Shared mutability in starlark is generally represented by having some starlark value own a `RefCell`, allocating that starlark value, and then using references to that allocation to achieve sharing. That approach is generally a bit better because it cleanly ties the "duration" of the shared mutability to the starlark evaluation - once the starlark evaluation has completed, the shared mutable thing is gone - this helps with correctness a bit (important, given the standard implications of shared mutability) More concretely though, we have this thing: https://www.internalfb.com/code/fbsource/[72d70f64f6a9]/fbcode/buck2/app/buck2_core/src/unsafe_send_future.rs?lines=38-47 Used here, for example: https://www.internalfb.com/code/fbsource/[72d70f64f6a9]/fbcode/buck2/app/buck2_analysis/src/analysis/env.rs?lines=233 This justifying for Sendness is incorrect in cases like this. Even if you send an entire heap with all of its values across a thread, there's nothing guaranteeing that the `Rc` hasn't been cloned into something that isn't being sent across threads. A future diff does a better job of using the type system (instead of unsafe) to justify `Send` in a situation like that, and it turned up this problem. So the plan is to replace the `Rc` with an allocation that's correctly tied to the starlark heap. That requires putting a `'v` lifetime onto the `DeclaredArtifact` type, which in turn requires putting it onto a great many other places. This diff does that without yet using the `'v` lifetime for anything interesting, just a `PhantomData` for now. All changes in this diff are trivial(ish). The interesting stuff was pulled out earlier in the stack Differential Revision: D77119488 fbshipit-source-id: dbefb6b28a13fa166e0e88e4bda5fba73b329460

Make PagableSerialize/Deserialize traits object safe Summary: Making these traits object safe simplifies typetag implementation later in this stack, as well as more pagable implementations later on. This means incurring the cost of dynamic dispatch for each deserialize call, but let's not worry about that yet. (code mostly commandeered from cjhopman) Reviewed By: cjhopman Differential Revision: D90884770 fbshipit-source-id: 7b94e603fa9473222bd2ab838a91ca27246925dd

open source kosabi/common Summary: open sourcing kosabi/common is a dependency for stubsgen, so open sourcing it required for open sourcing stubsgen Reviewed By: IanChilds Differential Revision: D82041656 fbshipit-source-id: 527cd0467159b09d06f96f8187bdeab73b88e690

Require tag for all buck2_error_derive enum types Summary: Force an ErrorTag for all current and future enum types that are derived from buck2_error_derive. Can be as simple as `Input`, `Environment`, and `Tier0`. Note that I tagged all the current ones to the best of my knowledge and could be wrong. Please fix or let me know anyone finds anything that seems wrong Reviewed By: JakobDegen Differential Revision: D67954001 fbshipit-source-id: 17798faa4d7365e1d24c3eddac2bcc55d1e66b95

Integrate target_name_glob visibility patterns Summary: Expose `target_name_glob(name_globs, within = ...)` as a native Starlark value usable in `visibility` and `within_view` attribute lists. `name_globs` is a non-empty list of target-name globs (`*` is the only wildcard; a glob consisting solely of `*` is rejected in favor of `"PUBLIC"`); the optional `within` is a list of package or recursive package patterns that scope the match. A target matches when any glob matches its name and, when `within` is non-empty, its package falls inside any `within` scope. Coerced values are stored in the visibility pattern lists, serialize to a typed `__type`-tagged dict for `uquery`/BXL introspection, and render readable `target_name_glob(...)` display/JSON that round-trips back into the same expression. The coerced `within` scopes reuse `buck2_core`'s existing `PackagePattern` (package-or-recursive) rather than a bespoke type, so the scope matcher and `Display` live in one place. Review-driven refinements: `"PUBLIC"` is rejected as a `within` scope (it is a target name, not a package pattern) with a dedicated error; `name_globs` is positional-only and renders positionally so the repr round-trips (`target_name_glob(["*-test"], within = [...])`); `package()` accepts `Either<str, target_name_glob>` for type-checked unpacking; `Display` goes through `display_container::fmt_container` with the `StarlarkStr::repr` string escaper; and JSON serialization uses an infallible `TargetNameGlobRecord::to_json()` that builds the `Value` directly (no `to_value(...).expect()`) and which `Serialize` delegates to. Covered by unit and PACKAGE-integration tests, including PUBLIC-in-`within` rejection and positional-repr round-tripping. Reviewed By: jtbraun Differential Revision: D106104073 fbshipit-source-id: 7d0098bb5767235dcacc5253e0dae248dd40b778

lifetimes: Split up `avalue.rs` module Summary: This module currently contains all the `AValue` impls but that's honestly a bit of a shitty way to organize this. Split the module up into the actual distinct things Next diff will go and move some more related code with these impls Reviewed By: cjhopman Differential Revision: D90053892 fbshipit-source-id: c71b301ed85e5f357061a5ede345367b9b901b4d

Add Arc serialization with identity preservation to pagable Summary: This adds Arc support to the pagable serialization framework, enabling it to preserve Arc sharing semantics across serialization boundaries. This is essential for the "Infinite Graph" initiative, as buck2's internal data structures have extensive Arc sharing that must be maintained during graph hydration. **The problem:** Standard serde serializes Arc<T> by value, creating independent copies on deserialization. When serializing a structure like: ``` let shared = Arc::new(data); let obj = (shared.clone(), shared.clone()); ``` ...serde would deserialize into two separate allocations, breaking the sharing semantics and potentially causing memory bloat. **The solution:** The pagable framework now supports Arc serialization through new trait methods that allow serializer implementations to track Arc identity and preserve sharing. This ensures `Arc::ptr_eq(&restored[0], &restored[1])` holds after roundtrip. **New components:** `arc_erase.rs` - Type erasure for Arcs: - `ArcErase` trait for Arc types with serialize_inner/deserialize_inner methods - `ArcEraseDyn` for dynamic dispatch over type-erased Arcs - `WeakErase` / `WeakEraseDyn` for weak reference handling - Implementations for std::sync::Arc<T>, Arc<[T]>, triomphe::Arc<T>, triomphe::Arc<[T]>, and triomphe::ThinArc<H, T> `impls/arc.rs` - PagableSerialize/PagableDeserialize for Arc types: - Delegates to push_arc/pop_arc on the serializer/deserializer `traits.rs` updates: - Added `push_arc<T: ArcErase>` to PagableSerializer - Added `pop_arc<T: ArcErase>` to PagableDeserializer `testing.rs` updates: - TestingSerializer/TestingDeserializer updated to support Arc serialization with identity preservation for testing purposes Reviewed By: christolliday Differential Revision: D88774343 fbshipit-source-id: a2a07650532998136b4d2a83c93df6284e8bea1a

Fix library const val detection, annotation stripping, private member filtering, and internal supertypes Summary: Four fixes for K2 source-only ABI generation: 1. **Library const val detection**: For const vals loaded from bytecode (libraries), the FIR `initializer` is null because bytecode doesn't preserve expression trees. However, the constant value IS available at runtime via ConstantValue. Added `isFromLibrary` check to trust bytecode-loaded const vals have valid values. 2. **Class hierarchy reflection for annotations**: When stripping annotations with errors from FIR declarations, `getDeclaredField("annotations")` was failing for `FirDefaultPropertyBackingField` because the field is declared in the parent class `FirBackingFieldImpl`. Added `findFieldInHierarchy` helper to traverse the class hierarchy. 3. **Private member filtering**: The `removeNonPublicApi` function had incorrect logic that only removed private companion objects, not other private members. Fixed to properly strip private interface methods (e.g., `isVisualReplyToVisual`, `isUnseenRavens`) that were incorrectly exposed in the ABI JAR, causing KSP2 DI annotation processor errors. 4. **Internal supertype stripping**: When a public class implements an interface from an internal class (e.g., `class PublicClass : InternalHelper.Delegate`), the internal supertype must be stripped from the ABI. Otherwise, consumers get "cannot access 'InternalHelper.Delegate' which is a supertype of 'PublicClass'" errors. Added stripping in both IR supertypes list and FIR metadata sources. Also: - Improved source package detection to handle nested class imports - Added integration tests for private interface method stripping and internal supertype stripping - Updated whatsapp test expectations for correct K2 private nested class stripping Reviewed By: siaojiecai Differential Revision: D91892625 fbshipit-source-id: eed6c90c43037b0eb090ac3cc4d6caeda255873d

Extract FIR and IR sanitization into dedicated modules Summary: Move FIR metadata sanitization (annotation stripping, error expression handling, Throws cleanup, private declaration removal) into fir/FirMetadataSanitizer.kt, supertype pruning (private/internal supertype stripping, fake override conversion) into fir/FirSupertypePruner.kt, and IR non-ABI stripping (private declaration removal, body stubbing, supertype pruning) into ir/IrNonAbiStripper.kt. The main file now delegates to these modules, reducing from ~3300 to ~1600 lines. Differential Revision: D94354845 fbshipit-source-id: 28e4729f5ce85629d3736ae251b6d667b2e85c3e