github.com-google-zerocopy

[codegen] Test and document codegen under different layouts (#3085) * [codegen] Test and document codegen under different layouts This improves the robustness of our suite to different code paths, and illustrates in our documentation the relationship between type complexity and codegen complexity. gherrit-pr-id: G39e297d5b891f5115ef9607b090c5fbe95427d68 * [ci] Change zizmor persona from 'pedantic' to 'regular' * [ci] Temporarily disable `zizmor` job --------- Co-authored-by: Joshua Liebow-Feeser <joshlf@users.noreply.github.com>

[derive] Refactor into more modules (#2901) - Move most top-level derive code into modules under a new top-level `derive` module - Move some utilities into `util` - Leave the root module to primarily perform routing and scaffolding gherrit-pr-id: Gf40a4d73de475b2aae089e5e2d4cea53dd3c868c

[pointer] Support generic `TransmuteFrom` framework (#2408) This commit removes the `TransparentWrapper` trait and the `Ptr::transparent_wrapper_into_inner` method. It replaces them with a new family of transmutation traits which encode more generic transmutation (from any `T` to any `U`) and a set of `Ptr` methods which use those traits to bound transmutation operations. In particular: - `Dst: TransmuteFrom<Src>` denotes that a by-value transmutation is sound - `Dst: TryTransmuteFromPtr<Src>` denotes that a transmutation is sound so long as it can be guaranteed that the source is bit-valid for the destination; this is used by e.g. `Ptr::try_into_valid`, which performs runtime validation of bit validity - `Dst: TransmuteFromPtr<Src>` is equivalent to `TransmuteFrom<Src> + TryTransmuteFromPtr<Src>` Some type arguments are omitted in this summary. In particular, all three traits also take validity invariant parameters for both the source and destination types. Also, the `[Try]TransmuteFromPtr` traits take an aliasing parameter. In order to support these traits, we introduce a generalization of `Read` known as `MutationCompatible`. `T: MutationCompatible<U, A>` denotes that *either* `T: Read<A>` and `U: Read<A>` *or* `T` and `U` have the same interior mutation semantics (formally, it is sound for `&T` and `&U` to reference the same referent - safe code operating on these references cannot cause undefined behavior). This is a refinement of the "`UnsafeCell` agreement" concept that we have used before, but it supports types which store but don't actually use `UnsafeCell`s. For example, given a hypothetical `ReadOnly<T>`, the following bound holds: usize: MutationCompatible<ReadOnly<AtomicUsize>, Exclusive> This commit also takes a different approach from the one originally envisioned in #1945. In particular, it turns out that we don't need a full type-level mapping concept. Instead, we need a *predicate* over transitions to determine which ones are valid (e.g., it is valid to go from a `Valid` `MaybeUninit<T>` to an `Uninit` `MaybeUninit<T>`). By contrast, the invariant mapping concept suggests that each source validity has *exactly one* destination validity. This commit makes progress on #1940 by supporting unsized transmutations, but we don't yet support size shrinking or expanding transmutations. This commit obsoletes #1359, as that issue was predicated upon the existence of `TransparentWrapper`, which this commit removes. This commit closes #1226, which suggests supporting `UnsafeCell` agreement. Closes #1945 Closes #1359 Closes #2226 Closes #1226 Closes #1866 Makes progress on #1359 gherrit-pr-id: Iad14813bc6d933312bc8d7a1ddcf1aafc7126938 Co-authored-by: Jack Wrenn <jswrenn@amazon.com>

Add `SplitAt`, `derive(SplitAt)` (#2433) This derivable trait provides generalized splitting for slice DSTs. Splitting a DST `T` produces a pair `(T, [T::Elem])`, where `T::Elem` is the element type of `T`'s trailing slice. We expose this functionality as a derivable trait, so as to not violate the privacy of the trailing slice. Makes progress towards #1290.

[anneal][v2] Initial commit of `exocrate` (#3376) gherrit-pr-id: G34qjom3lz7cc6hd57tgp44t4pzlstdhj Co-authored-by: Mark Dittmer <markdittmer@google.com>

Account for alignment directives in `derive(IntoBytes)` (#3064) This fixes #3063 by threading alignment directives into the padding check macros, and fixes #3067 by unconditionally emitting the padding check even in the absence of fields (thus ensuring that there exists a padding check into which alignment directives can be threaded). Fixes #3063, #3067 gherrit-pr-id: Gf18655827d1c30e4421c83579b962bf7aa453969

[pointer] Add separate PtrInner (#1891) (#2380) `PtrInner` carries all invariants which are not controlled by type parameters. Since `PtrInner` does not promise to uphold aliasing, alignment, or validity, we can move some utility methods to `PtrInner` which previously were responsible for maintaining invariants orthogonal to their purpose. Makes progress on #1892 (still needs to be fixed on v0.8.x) Closes #1890 gherrit-pr-id: I1c2d4a54bc1d73d3079d043c2bc393a8967f44f6

Add code generation tests (#3042) Introduces a suite of code generation tests to the `benches` folder. Each tests consists of a pair of files: a Rust source containing a function named `codegen_test`, and a `.x86-64.mca` file containing the llvm-mca analysis of `codegen_test`. In practice, these tests operate similarly to UI tests; CI checks a fresh llvm-mca analysis against that in the file, and fails if they mismatch. Changes can be expressly blessed via `BLESS=1`. gherrit-pr-id: G5964d13c9ffb7a47ed4662892ef92c90ddc51e12

Separate `SplitAt`'s bounds checking and overlap checking (#2473) By doing so, dynamic overlap checking can be avoided entirely in many cases. Makes progress towards #1290.

`derive(IntoBytes)`: support `repr(C)` structs with explicit trailing slices (#2679) For example: #[repr(C)] struct Example { leading: u8, trailing: [Trailing] } Makes progress towards #1112

[codegen] Test and document codegen for `SplitAt` and `Split` (#3111) Makes progress towards #3079. gherrit-pr-id: G09ea59a47925616837b859b2c3cdf7f0f919232e

Use `ReadOnly` in `TryFromBytes::is_bit_valid` (#2873) Previously, `is_bit_valid` had the signature: ```rust fn is_bit_valid<A>(c: Ptr<'_, Self, (A, Unaligned, Initialized)>) -> bool ``` In this commit, we remove the `A` aliasing parameter and wrap `Self` in `ReadOnly`, yielding: ```rust fn is_bit_valid(c: Ptr<'_, ReadOnly<Self>, (Shared, Unaligned, Initialized)>) -> bool ``` This ensures that `is_bit_valid`'s argument is always uconditionally `Immutable` regardless of whether `Self: Immutable`. This solves a number of open problems: - Ensures that `is_bit_valid` can never mutate its referent (#1831), which is important for custom validators (#1330) - Makes it so that custom validators can be written without needing to be generic over aliasing, which in turn means we can support custom validators without exposing much of our `Ptr` machinery in our public API - Allows us to support `#[derive(TryFromBytes)]` on unions without requiring `Self: Immutable` (#1832) - Permits `T -> U` fallible transmutation in more cases (see #2336 for more details) Makes progress on #2336 gherrit-pr-id: G7691845b6b02e9f3d9578435d732bacfa6ca674f

Add `Self: IntoBytes` bound to `TryFromBytes::try_mut*` (#2331) Consider that `MaybeUninit<u8>` is `TryFromBytes`. If a `&mut [u8]` is cast into a `&mut MaybeUninit<u8>`, then uninit bytes are written, the shadowed `&mut [u8]`'s referent will no longer be valid. Makes progress towards #2226 and #1866. gherrit-pr-id: Ib233c4d0643e0690c53a37a08d9845e5fe43249c

[derive] Introduce Ctx type (#2890) Use the `Ctx` type to carry the `DeriveInput` and `zerocopy_crate: Path` values which are used by most functions. gherrit-pr-id: G0662505d5901f5c3cd21b16c314b9f3f8dc817a6

Introduce `HasField` trait (#2842) The `HasField` trait permits privacy-preserving field projections from structs, enums and unions. `HasField` preserves privacy by leveraging the [scoped trait pattern]; for each field `f`, `HasField` is implemented with the `Field` parameter set to a type with identical visibility as `f`. In many useful contexts, this `Field` parameter is inferable, and need not be actually reachable. Consequently, `HasField` can be used to index a field from only the name of a field. See `zerocopy-derive/tests/ui-nightly/privacy.rs` for examples. [scoped trait pattern]: https://jack.wrenn.fyi/blog/private-trait-impls/ gherrit-pr-id: G2238bc341570838db412d880017b3f0c25ac09fa

Publish `Ptr[Inner]` behind a `--cfg` (#3387) Release 0.8.49-alpha. gherrit-pr-id: Gblc7dfltwcey7wvtj3gjkseaqcltgwvf

Support unsized `try_transmute_{ref,mut}!` (#2944) Support: - Sized to sized - Sized to unsized - Unsized to unsized gherrit-pr-id: Gdf81e39c9f13236c2c2bad3bfbd2bb931e73b706

[codegen] Test and document codegen for `FromZeros` (#3114) Vendors `regex` to normalize labels. Makes progress towards #3079. gherrit-pr-id: Gea71a24b6b02a2d552b4af3e0980e71a50ab8f52

[hermes] Pre-build Lean libraries during `setup` (#3236) During `cargo hermes setup`, download and install the compiled Mathlib artifact (via `lake exe cache get`) and build the Aeneas Lean library, installing it in `~/.hermes`. This has a number of benefits: - Improves verification time since the Aeneas Lean library doesn't have to be built - Removes complexity from verification – currently, we rely on Lake's caching of built artifacts to speed up subsequent verifications - Cuts down on stdout/stderr noise during verification (caused by building Aeneas) In local development and CI infrastructure, make use of the pre-built Aeneas Lean library, allowing us to simplify by removing logic which was needed to cache this library when it was previously built during verification. While we're here, add support to `docker.sh` for using `podman` instead of `docker`. Closes #3233 gherrit-pr-id: Gimnjq2dz4vyuoa3jek7wvbwvdafgngq7

Support sized-to-unsized `transmute_{ref,mut}!` (#2943) Closes #2721 gherrit-pr-id: G73f67c103188ed404d0051bc140c4d0711fe0753