github.com-google-perfetto

Copy Perfetto Java SDK Java and C++ (JNI) code from AOSP. Java and C++ classes are copied and refactored as follows: 1. SDK package renamed from 'os.android' to 'dev.perfetto'. 2. 'PerfettoTrace_register' function signature fixed to follow JNI ABI. 3. 'JNI_OnLoad' function used to register JNI functions. 4. Internal namespace renamed from 'tracing_perfetto' to 'perfetto'. 5. From java code deleted everything related to 'Ravenwood'. 6. From 'PerfettoTraceTest' class deleted platform related test. 'NativeAllocationRegistry' API is not supported outside of framework, so we intentionally don't free native memory accosted with Java objects, we will address this in future CLs. The leaking memory doesn't affect code correctness, all tests passes. We also do the following: 1. Copy 'nativehelper' header library from AOSP, because JNI code depends on it. 2. Change code formatting to be consistent with perfetto style. 3. Generate Android.bp and BUILD targets. PerfettoTrace.java copied from: https://source.corp.google.com/h/googleplex-android/platform/superproject/main/+/main:frameworks/base/core/java/android/os/PerfettoTrace.java;drc=029c7a04cc36484fe7c0d69c75b29bd9eff1e3a6 PerfettoTrackEventExtra.java copied from: https://source.corp.google.com/h/googleplex-android/platform/superproject/main/+/main:frameworks/base/core/java/android/os/PerfettoTrackEventExtra.java;drc=029c7a04cc36484fe7c0d69c75b29bd9eff1e3a6 com_google_perfetto_sdk_PerfettoTrace.cc copied from: https://source.corp.google.com/h/googleplex-android/platform/superproject/main/+/main:frameworks/base/core/jni/android_os_PerfettoTrace.cpp;drc=ea0f2553f513f05de29e2fe8d0769b13e93d912b com_google_perfetto_sdk_PerfettoTrackEventExtra.cc copied from: https://source.corp.google.com/h/googleplex-android/platform/superproject/main/+/main:frameworks/base/core/jni/android_os_PerfettoTrackEventExtra.cpp;drc=50adc08679d5b380dee7785111860cf3568b5a86 Tested: `atest perfetto_trace_instrumentation_test` and ``` tools/bazel test //:src_android_sdk_java_test_perfetto_trace_instrumentation_test --nocache_test_results --test_output=all ``` Bug: b/386353531 Change-Id: Ibf3b8d73f7cd85f72f65ad4f7290f9a5c32785fc

bt-ui: Revamp Bigtrace settings architecture and UI (#6205) Overhauls the Bigtrace settings infrastructure to consolidate storage, improve the UI, and simplify query execution. Key changes: - Trace Selection Grid: Introduces a dedicated, filterable DataGrid on the Settings page to select the exact traces a query will run over (the /trace_metadata wire contract), replacing the legacy panel. - Unified UI: Replaces the legacy trace-metadata panel with a cohesive settings strip and modal on the query page. - Storage & Mapping: Dedupes settings storage into SingleFieldStorage and centralizes the mapping logic for the /trace_metadata and /execute_* endpoints. - Component Polish: Makes boolean settings immediately editable (no enable/disable toggle), adds a "reset to default" affordance, and ensures default metadata columns are visible. - Layout & Rendering: Groups metadata result columns in a trailing block, places the link column first, and bounds the results grid for proper virtualization. - History & State: Restores history-entry hover states, parses camelCase setting IDs from snapshots to restore disabled settings, and defaults new queries to persistent execution.

protovm: initial implementation (#1275)

bt-ui: Implement and polish Bigtrace asynchronous queries (#5822) This PR introduces comprehensive support for executing asynchronous queries in the Bigtrace UI, along with a series of architectural improvements and UX polish. Key additions and improvements include: * Core Async Support: Introduces the ability to submit, track, and manage long-running asynchronous Bigtrace queries. * Architecture: Implements a centralized BigtraceQueryClient and QueryRunner to manage query lifecycles, separating state management from the UI components. * DataGrid Integration: Replaces traditional pagination with a scroll-based approach and integrates with server-side sorting (order_by) and filter encoding for Datagrid chips. * History & State: Adds persistent query history with background status polling, and fixes ephemeral tab state when switching pages. * UI/UX Polish: Introduces updated progress bars, detailed post-filter row counts, and improved duration formatting.

tp: overhaul error handling and diagnostics for the shell and bundle (#6969) Issue: https://github.com/google/perfetto/issues/6927 `trace_processor bundle` (formerly `traceconv bundle`) gave cryptic or misleading feedback when things went wrong: - Traces with nothing to enrich (e.g. a pure atrace trace, or a function_graph trace recorded **with** `symbolize_ksyms`) failed with a bare `bundle: failed to create bundle.` and no explanation. - A function_graph trace recorded **without** `symbolize_ksyms` got no feedback at all, even though its kernel addresses can never be symbolized offline. - Unwritable output paths (missing parent dir, read-only, path-is-a-dir) crashed the whole process with a `PERFETTO_CHECK`. - Extra positional arguments were silently ignored/misparsed (e.g. passing each `--symbol-paths` directory as a separate argument). - Conversion failures produced double error messages (an internal `PERFETTO_ELOG` + a generic "conversion failed" wrapper), and several paths crashed with `PERFETTO_FATAL` on ordinary user error. ## Changes - **bundle** now always produces the bundle and explains what could not be enriched, with actionable hints. It detects function_graph traces recorded without `symbolize_ksyms`, fails gracefully (instead of crashing) on unwritable output paths, and reports reading/enrichment progress. - **CLI**: every fixed-arity subcommand (`bundle`, `convert`, `util`, `export`, `query`, `server`, `metrics`) now rejects extra positional arguments with a hint, instead of silently misinterpreting them. - **traceconv**: the conversion entry points (`TraceToJson`, `TraceToSystrace`, `TraceToText`, `TraceToProfile`, `TraceToFirefoxProfile`, `UnpackCompressedPackets`, `SymbolizeProfile`, `DeobfuscateProfile`) now return descriptive `base::Status` errors instead of int/bool returns paired with internal `PERFETTO_ELOG`/`FATAL`, so failures are reported once with the actual reason. - **TarWriter**: fails gracefully when the output path cannot be opened (also fixes the same crash in `util merge`). - **Parse errors**: no longer append a redundant "The trace file is corrupt." sentence; the UI error-dialog extraction is updated for the new message format.

tp: model trace types as importers, remove the TraceType enum (#6618) Today, all trace detection logic is centralized inside TP meaning it's impossible for plugins to register new formats and have them work without also changing the core code. As the number of formats has increased, this has gotten more and more unweildy Design it to instead be structured such that new trace types can be developed fully independently *without* making any changes to the core. This will be proved out in a followup change where perf text importer will be moved.

core: Introduce TraceBufferV2 (#3092) TraceBufferV2 is a major rewrite of the historical TraceBuffer. The main reasons for the redesign are: - Supporting ProtoVM and future interning improvements. - Reducing space wasted for internal fragmentation when a chunk contains only few bytes of trace data. - Making the trace output respect more the input ordering, rather than completely reordering packets by sequence. This CL contains a design doc that discusses various aspects and challenges of the new TraceBuffer architecture. Benchmark results: ``` Apple Macbook (M4) ------------------ BM_TraceBuffer_WR_SingleWriter<TraceBufferV1> bytes_per_second=9.77742G/s BM_TraceBuffer_WR_SingleWriter<TraceBufferV2> bytes_per_second=12.6395G/s BM_TraceBuffer_WR_MultipleWriters<TraceBufferV1> bytes_per_second=8.65385G/s BM_TraceBuffer_WR_MultipleWriters<TraceBufferV2> bytes_per_second=11.7582G/s BM_TraceBuffer_RD_MixedPackets<TraceBufferV1> bytes_per_second=4.27694G/s BM_TraceBuffer_RD_MixedPackets<TraceBufferV2> bytes_per_second=4.35475G/s Pixel 7 ------- BM_TraceBuffer_WR_SingleWriter<TraceBufferV1> bytes_per_second=4.4379G/s BM_TraceBuffer_WR_SingleWriter<TraceBufferV2> bytes_per_second=3.7931G/s BM_TraceBuffer_WR_MultipleWriters<TraceBufferV1> bytes_per_second=3.19148G/s BM_TraceBuffer_WR_MultipleWriters<TraceBufferV2> bytes_per_second=3.47354G/s BM_TraceBuffer_RD_MixedPackets<TraceBufferV1> bytes_per_second=1.26698G/s BM_TraceBuffer_RD_MixedPackets<TraceBufferV2> bytes_per_second=1.35394G/s ``` Bug: b/447426810 **Stack:** - [#3091](https://github.com/google/perfetto/pull/3091) (tb_extract) - **[This PR] (tbv2)**

de: Add visualisation node (#4891) New node. - In terms of SQ it is passthrough + filters created from charts brushing -> a filter node - Only supports Histogram and Bar chart for now - Shows charts at the dataExplorer tab at the bottom

ui: Add Heapdump Explorer plugin (#5276) Add the com.android.HeapDumpExplorer plugin, inspired by Android's ahat tool, with navigation, shared components, queries, and all heap analysis views: overview dashboard, classes, objects, instances, dominators, bitmap gallery, and strings. Uses DataGrid with SQLDataSource for efficient sorting and filtering of large datasets.

tp: decouple sorting from TraceProcessorContext (#2411) Allows parsing of multiple traces simultaneously without getting clashes between them from using the same parsers/tokenizers etc.

Bigtrace UI V0 (#5243) This change introduces "BigTrace UI", a standalone bigtrace query interface within perfetto UI source code. BigTrace provides a focused environment for executing and managing complex SQL queries against traces. Key features and components: - Application Shell: A new entry point and lightweight application singleton (BigTraceApp) managing its own command and omnibox instances. - Multi-tab SQL Editor: A robust query interface supporting multiple persistent tabs, syntax-highlighted editors, and result limiting. - Result Visualization: Integration with DataGrid for high-performance rendering of SQL query results, including support for linkified columns and uuid-based row tracking. - Navigation & Layout: Custom routing (router.ts, routes.ts) and layout components (Sidebar, Topbar, Omnibox) tailored for the BigTrace experience. - Persistence: Local storage integration for saving editor tabs, query history, and user preferences across sessions. - Settings Management: A dedicated SettingsPage and storage service for configuring Brush-specific parameters, such as backend endpoints and default result limits. - Home & Help: A landing page with "getting started" guidance and a help modal detailing available keyboard shortcuts. - Styling: Dedicated SCSS for BigTrace to ensure a consistent and modern aesthetic. Note: we may have duplicated classes/interfaces between Bigtrace UI and Perfetto UI, because we wanted to **isolate** the changes and avoid exposing classes/interfaces from Perfetto UI. In a follow-up, we will expose the classes/interfaces suitable for reuse.

tp: de: Serializer responsiblity moved to Trace Processor (#4682) This change moves query materialization responsibility from the UI to Trace Processor (TP), establishing TP as the single source of truth for all materialization state. ## Changes **Public API (include/perfetto/trace_processor/):** - Added new experimental `Summarizer` interface for managing lazy materialization of structured queries - Added `CreateSummarizer()` method to `TraceProcessor` (returns ownership via out parameter) - Added `SummarizerUpdateSpecResult` with nested `QuerySyncInfo` for sync status - Added `SummarizerQueryResult` for query results - All Summarizer APIs marked as EXPERIMENTAL **Trace Processor (C++):** - Added `SummarizerImpl` class with smart optimizations: - Lazy materialization: tables created only when `Query()` is called - Proto-based change detection via hashing - Dependency propagation: if query A changes, all dependents (B→C→D) re-materialize - Table-source substitution: unchanged queries reference materialized tables directly - RPC layer manages Summarizer IDs (C++ TraceProcessor layer is ID-agnostic) - Summarizers automatically cleared when trace processor is reset **UI (TypeScript):** - Simplified `QueryExecutionService` by removing ~500 lines of complex state management - Removed UI-side query hash caching, materialization lifecycle, staleness detection - TP is now queried on-demand for table names and state - Updated `Engine` interface with new `createSummarizer()`, `updateSummarizerSpec()`, `querySummarizer()`, and `destroySummarizer()` methods

de: Add dashboard functionality (#5104) Add dashboard functionality to the Data Explorer, allowing users to create dashboard tabs that display charts from exported data sources. Users connect upstream nodes to "Export to Dashboard" nodes in graph tabs, then visualize the exported data on free-form dashboard canvases with drag-and-drop, resize, cross-chart brush filtering, and source switching.

de: Download from metric (#5212) This provides loading dowloading the .pbtxt file with TraceSummary or metric and show it as a graph. This is a big change, but relatively self contained - this whole file will disappear once we migrate proto based serialization.

tp: migrate all static table functions to dataframe (#1848) This completes the dataframe migration: * static tables * runtime tables * static table functions are all using dataframe. Follow up CLs will delete the legacy table implementation from the codebase once these set of changes sticks

ui: Simplify DataGrid schema models (#6710) Currently the schema is a record of tables keyed by name, like this: ```js { slice: { id: {}, ts: {}, dur: {}, track: {ref: 'track', ...}, // Ref to the foreign table }, track: { id: {}, name: {} }, parent: {ref: 'slice', ...}, // Recursive reference } ``` This works but is messy: - Can reference non-existent tables - must check for undefined everywhere. - The vast majority of schemas in practice are flat (single table) which means we have to nest every single schema inside a dummy table name when its totally redundant. - Field paths don't specify a 'root' table name so we must pass the 'root' (default) table name around as a string everywhere. ## The fix This patch replaces this nominal structure with a structured definition which solves all of these problems: E.g. ```js const sliceSchema = { id: {}, ts: {}, dur: {}, table: { schema: { // <-- Nested table schema id: {}, name: {} }, }, parent: { get schema() { // <-- recursive references can exist - you just have to use a getter. return sliceSchema; } }, } ``` Notes: - The recursive via getter trick is borrowed from [zod](https://zod.dev/api?id=recursive-objects#recursive-objects). - The sqlSchema (the separate schema passed to the SQLDataSource has been restructured in a similar way. - This is a big change due to the number of DataGrid call-sites, but it's mainly mechanical. Should be a no-op. Key changes: - SchemaRef.ref (string) → SchemaRef.schema (ColumnSchema), with get schema() for self-referential tables - SQLTableSchema.table → SQLTableSchema.tableOrSubquery, always wrapped in parens in generated SQL - DataGridAttrs.schema is now ColumnSchema (was SchemaRegistry), rootSchema removed - SQLDataSource config extends SQLTableSchema directly (was sqlSchema + rootSchemaName) - Add getQuery() to SQLDataSource and sub-engines for SQL inspection

ui: Split widgets page into subpages (#3465) Split widgets page into one widget per page with a nav menu.

ui: Refactor DataGrid SQL datasource (#4384) - Migrate to QuerySlots. - Break up SQL datasource implementation into multiple files. - Materialize perfetto table to make tree queries more efficient - fixes slow group collapsing/expanding and sorting when in pivot+tree mode. - Fix bug where a flash of stale data is shown while the new data loads when switching between datagrid modes like flat->pivot or pivot->drilldown.

tp: add Arrow support for dataframes (#6827) Provide an Arrow serializer and deserializer for Dataframes using a standard Arrow file containing one record batch. Primitive columns map to Arrow numeric arrays, strings map to Utf8, nullable columns carry validity bitmaps, and sparse storage uses Arrow's logical row layout. The serializer computes metadata and variable-width sizes before streaming column contents with bounded scratch memory. The deserializer validates file framing, record batch metadata, and buffer extents before rebuilding column storage, null state, implicit Id columns, and strings in the target StringPool.

ui: Memscope - Live dashboard page (#5785) This patch adds a new `dev.perfetto.Memscope` UI plugin: a live memory dashboard that connects to an Android (ADB / WDP) or Linux (websocket) target, starts a lightweight trace, and renders periodically refreshed snapshots across four tabs (Processes, System, File Cache, Pressure & Swap). NOTE: In order to keep the PRs as small as possible, this is just the first part of the memscope plugin which only features the dashboard - no profiling mode is implemented yet. ## Key bits - Live tracing session (sessions/live_session.ts): low-overhead monitoring config (process_stats, sys_stats, ftrace). Polls snapshots on an interval, parsing and extracting key counter information. - Connection view (views/connection.ts): Landing page when no session is currently running. Allows the user to select a device protocol, and, if relevant, presents a device list to allow device selection. - Dashboard view: (views/dashboard.ts): Shows up when the user connects to a device and starts recording a live tracing session. Displays a toolbar showing the status of the live session and buttons to control the session, and the four tabs with a tab switcher (SegmentedButton). - Tabs: - Processes tab (views/tabs/processes.ts): Shows a few header cards showing high level system memory usage, a chart showing the sum of all processes grouped by category (arbitrary list defined in process_categories.ts), and a table of all processes with various stats spread across various columns. - System tab (views/tabs/system.ts): Shows various system level memory stats across billboards and a chart. - Page cache tab (views/tabs/page_cache.ts): Show page cache stats including usage and activity charts. - Pressure, swap and LMK tab (views/tabs/pressure_swap.ts): Displays swap usage, memory pressure and LMK events. - Process categorization (process_categories.ts): maps Android process names → high-level categories (Framework, HAL, System UI, …). Colors are assigned positionally from the chart palette so adding categories doesn't require picking hex. ## Usage / testing: - Enable the dev.perfetto.Memscope plugin & restart the page. - Click the memscope sidebar entry. - Connect to your Android/Linux device of choice. - Verify stats look correct. Press pause and play again to ensure the snapshots stop updating when paused. - Switching tabs should stop the periodic updates (and period redraws). - Stop & Open Trace produces a working trace in the main UI. - Verify category chips and OOM-bucket chips read well in both light and dark themes