perfetto — Engineering Performance

bt-ui: convert bigtrace into a single-page application (#7127) Reworks the Bigtrace UI into a cohesive single-page application, replacing multi-page route navigation with a persistent, tabbed query workspace. This overhaul streamlines preset management, cleanly separates trace corpus selection from query execution settings, introduces support for trace UUID lists and experiment filters, and significantly reduces backend load through debounced fetching and request coalescing. Key Changes: 1. Single-Page Application Architecture: - Replaced multi-page routing (/home, /settings, /tables, /query) and the left sidebar with tabbed query workspaces. - Added QueryLauncher: an integrated launch surface shown for new queries and for configuring trace selections on existing queries. - Added LocalPresetStore to persist custom user presets in localStorage alongside the backend preset catalog. - Added a topbar connection button with real-time health checks to manage backend endpoints. 2. Decoupled Trace Selection & Query Settings: - Separated trace selection (which traces to query, corpus filters, and metadata columns) from query execution options (row limits, materialization, and trace fan-out caps). - Moved query execution controls into an Advanced Query Settings modal accessible via the toolbar gear button. - Implemented transactional editing sessions with Cancel/Apply semantics, ensuring in-progress edits can be discarded cleanly. 3. Trace UUID Selection & Fan-Out Capping: - Added a dedicated UUID mode allowing users to paste lists of trace UUIDs (comma or whitespace separated) directly into a query. - Elevated the trace fan-out limit (`traceLimit`) to a first-class request parameter with mode-specific defaults. 4. Experiment & Arm Filtering: - Added ExperimentFilterControl to search the /experiments catalog and filter queries to specific experiment arms (Treatment vs. Control). - Resolved experiment display metadata asynchronously with memoization while preserving numeric IDs on the wire and across stored presets. - Added client-side support for named, expiring persistent result tables (`tableName`, `tableTtlDays`, and /check_table_exists). 5. Performance & Network Optimizations: - Introduced FetchScheduler to debounce rapid scroll-triggered window requests (150ms) and abort superseded in-flight requests. - Decoupled 3-second status polling from result data fetching: the poll now monitors progress metrics only and avoids issuing full table scans while queries are still writing. - Removed the obsolete command palette / omnibox component (~600 lines) and added a direct theme toggle to the topbar. Testing: - Ran complete UI unit test suite: 155 test files passed, 2,659 tests passed. - Verified type checking (`ui/node ui/build.mjs --no-build --typecheck --bigtrace`). - Tested tab lifecycle, preset save/edit/delete flows, UUID mode switching, and experiment filtering

Copy Perfetto Java SDK Java and C++ (JNI) code from AOSP. Java and C++ classes are copied and refactored as follows: 1. SDK package renamed from 'os.android' to 'dev.perfetto'. 2. 'PerfettoTrace_register' function signature fixed to follow JNI ABI. 3. 'JNI_OnLoad' function used to register JNI functions. 4. Internal namespace renamed from 'tracing_perfetto' to 'perfetto'. 5. From java code deleted everything related to 'Ravenwood'. 6. From 'PerfettoTraceTest' class deleted platform related test. 'NativeAllocationRegistry' API is not supported outside of framework, so we intentionally don't free native memory accosted with Java objects, we will address this in future CLs. The leaking memory doesn't affect code correctness, all tests passes. We also do the following: 1. Copy 'nativehelper' header library from AOSP, because JNI code depends on it. 2. Change code formatting to be consistent with perfetto style. 3. Generate Android.bp and BUILD targets. PerfettoTrace.java copied from: https://source.corp.google.com/h/googleplex-android/platform/superproject/main/+/main:frameworks/base/core/java/android/os/PerfettoTrace.java;drc=029c7a04cc36484fe7c0d69c75b29bd9eff1e3a6 PerfettoTrackEventExtra.java copied from: https://source.corp.google.com/h/googleplex-android/platform/superproject/main/+/main:frameworks/base/core/java/android/os/PerfettoTrackEventExtra.java;drc=029c7a04cc36484fe7c0d69c75b29bd9eff1e3a6 com_google_perfetto_sdk_PerfettoTrace.cc copied from: https://source.corp.google.com/h/googleplex-android/platform/superproject/main/+/main:frameworks/base/core/jni/android_os_PerfettoTrace.cpp;drc=ea0f2553f513f05de29e2fe8d0769b13e93d912b com_google_perfetto_sdk_PerfettoTrackEventExtra.cc copied from: https://source.corp.google.com/h/googleplex-android/platform/superproject/main/+/main:frameworks/base/core/jni/android_os_PerfettoTrackEventExtra.cpp;drc=50adc08679d5b380dee7785111860cf3568b5a86 Tested: `atest perfetto_trace_instrumentation_test` and ``` tools/bazel test //:src_android_sdk_java_test_perfetto_trace_instrumentation_test --nocache_test_results --test_output=all ``` Bug: b/386353531 Change-Id: Ibf3b8d73f7cd85f72f65ad4f7290f9a5c32785fc

tracing: add the v2 shared-ring chunk protocol (#7324) Add the shared-memory ring that tracing v2 producers will write packet fragments into, with a writer and a reader class on top of it: - One 64-bit atomic holds both the write and the read position. Writers reserve positions with a lock-free CAS on it; the reader publishes read_pos once per drain pass. - Every chunk starts with a 32-bit state word. The five states (Free, BeingWritten, Complete, RewriteRequested, RewriteAcknowledged) change hands only through exact-value CAS, so whoever loses a race sees the word that won. - A Free word carries a 16-bit wrap count, so a delayed writer cannot claim a chunk for a position the reader already resolved. Only the reader ever writes Free. - Writers append fragments to their chunk and publish by moving the state word. The reader may take the committed prefix of a chunk that is still being written; only the just-ended fragment can require relocation into a fresh chunk. - Finishing a writer releases its cached chunk, so later fragments require a new reservation. This establishes the flush watermark boundary. - Ring exhaustion follows BufferExhaustedPolicy: kDrop, kStall and kStallThenDrop. On Linux and Android a stalled writer waits on read_pos with a futex, under the same 30-second ceiling as v1. - Tests cover the ABI encoding, reader/writer race outcomes, position and wrap-count rollover, and multi-writer stress against a concurrent reader.

protovm: initial implementation (#1275)

bt-ui: Implement and polish Bigtrace asynchronous queries (#5822) This PR introduces comprehensive support for executing asynchronous queries in the Bigtrace UI, along with a series of architectural improvements and UX polish. Key additions and improvements include: * Core Async Support: Introduces the ability to submit, track, and manage long-running asynchronous Bigtrace queries. * Architecture: Implements a centralized BigtraceQueryClient and QueryRunner to manage query lifecycles, separating state management from the UI components. * DataGrid Integration: Replaces traditional pagination with a scroll-based approach and integrates with server-side sorting (order_by) and filter encoding for Datagrid chips. * History & State: Adds persistent query history with background status polling, and fixes ephemeral tab state when switching pages. * UI/UX Polish: Introduces updated progress bars, detailed post-filter row counts, and improved duration formatting.

bt-ui: Revamp Bigtrace settings architecture and UI (#6205) Overhauls the Bigtrace settings infrastructure to consolidate storage, improve the UI, and simplify query execution. Key changes: - Trace Selection Grid: Introduces a dedicated, filterable DataGrid on the Settings page to select the exact traces a query will run over (the /trace_metadata wire contract), replacing the legacy panel. - Unified UI: Replaces the legacy trace-metadata panel with a cohesive settings strip and modal on the query page. - Storage & Mapping: Dedupes settings storage into SingleFieldStorage and centralizes the mapping logic for the /trace_metadata and /execute_* endpoints. - Component Polish: Makes boolean settings immediately editable (no enable/disable toggle), adds a "reset to default" affordance, and ensures default metadata columns are visible. - Layout & Rendering: Groups metadata result columns in a trailing block, places the link column first, and bounds the results grid for proper virtualization. - History & State: Restores history-entry hover states, parses camelCase setting IDs from snapshots to restore disabled settings, and defaults new queries to persistent execution.

tp: overhaul error handling and diagnostics for the shell and bundle (#6969) Issue: https://github.com/google/perfetto/issues/6927 `trace_processor bundle` (formerly `traceconv bundle`) gave cryptic or misleading feedback when things went wrong: - Traces with nothing to enrich (e.g. a pure atrace trace, or a function_graph trace recorded **with** `symbolize_ksyms`) failed with a bare `bundle: failed to create bundle.` and no explanation. - A function_graph trace recorded **without** `symbolize_ksyms` got no feedback at all, even though its kernel addresses can never be symbolized offline. - Unwritable output paths (missing parent dir, read-only, path-is-a-dir) crashed the whole process with a `PERFETTO_CHECK`. - Extra positional arguments were silently ignored/misparsed (e.g. passing each `--symbol-paths` directory as a separate argument). - Conversion failures produced double error messages (an internal `PERFETTO_ELOG` + a generic "conversion failed" wrapper), and several paths crashed with `PERFETTO_FATAL` on ordinary user error. ## Changes - **bundle** now always produces the bundle and explains what could not be enriched, with actionable hints. It detects function_graph traces recorded without `symbolize_ksyms`, fails gracefully (instead of crashing) on unwritable output paths, and reports reading/enrichment progress. - **CLI**: every fixed-arity subcommand (`bundle`, `convert`, `util`, `export`, `query`, `server`, `metrics`) now rejects extra positional arguments with a hint, instead of silently misinterpreting them. - **traceconv**: the conversion entry points (`TraceToJson`, `TraceToSystrace`, `TraceToText`, `TraceToProfile`, `TraceToFirefoxProfile`, `UnpackCompressedPackets`, `SymbolizeProfile`, `DeobfuscateProfile`) now return descriptive `base::Status` errors instead of int/bool returns paired with internal `PERFETTO_ELOG`/`FATAL`, so failures are reported once with the actual reason. - **TarWriter**: fails gracefully when the output path cannot be opened (also fixes the same crash in `util merge`). - **Parse errors**: no longer append a redundant "The trace file is corrupt." sentence; the UI error-dialog extraction is updated for the new message format.

tp: model trace types as importers, remove the TraceType enum (#6618) Today, all trace detection logic is centralized inside TP meaning it's impossible for plugins to register new formats and have them work without also changing the core code. As the number of formats has increased, this has gotten more and more unweildy Design it to instead be structured such that new trace types can be developed fully independently *without* making any changes to the core. This will be proved out in a followup change where perf text importer will be moved.

tp: decouple sorting from TraceProcessorContext (#2411) Allows parsing of multiple traces simultaneously without getting clashes between them from using the same parsers/tokenizers etc.

core: Introduce TraceBufferV2 (#3092) TraceBufferV2 is a major rewrite of the historical TraceBuffer. The main reasons for the redesign are: - Supporting ProtoVM and future interning improvements. - Reducing space wasted for internal fragmentation when a chunk contains only few bytes of trace data. - Making the trace output respect more the input ordering, rather than completely reordering packets by sequence. This CL contains a design doc that discusses various aspects and challenges of the new TraceBuffer architecture. Benchmark results: ``` Apple Macbook (M4) ------------------ BM_TraceBuffer_WR_SingleWriter<TraceBufferV1> bytes_per_second=9.77742G/s BM_TraceBuffer_WR_SingleWriter<TraceBufferV2> bytes_per_second=12.6395G/s BM_TraceBuffer_WR_MultipleWriters<TraceBufferV1> bytes_per_second=8.65385G/s BM_TraceBuffer_WR_MultipleWriters<TraceBufferV2> bytes_per_second=11.7582G/s BM_TraceBuffer_RD_MixedPackets<TraceBufferV1> bytes_per_second=4.27694G/s BM_TraceBuffer_RD_MixedPackets<TraceBufferV2> bytes_per_second=4.35475G/s Pixel 7 ------- BM_TraceBuffer_WR_SingleWriter<TraceBufferV1> bytes_per_second=4.4379G/s BM_TraceBuffer_WR_SingleWriter<TraceBufferV2> bytes_per_second=3.7931G/s BM_TraceBuffer_WR_MultipleWriters<TraceBufferV1> bytes_per_second=3.19148G/s BM_TraceBuffer_WR_MultipleWriters<TraceBufferV2> bytes_per_second=3.47354G/s BM_TraceBuffer_RD_MixedPackets<TraceBufferV1> bytes_per_second=1.26698G/s BM_TraceBuffer_RD_MixedPackets<TraceBufferV2> bytes_per_second=1.35394G/s ``` Bug: b/447426810 **Stack:** - [#3091](https://github.com/google/perfetto/pull/3091) (tb_extract) - **[This PR] (tbv2)**