Mayzner

This commit introduces **interactive tag filtering** to the **standard library documentation page**, significantly enhancing user navigation and module discoverability. This **new feature** allows users to dynamically filter stdlib modules based on selected tags, supporting multiple tag selections with OR logic. The implementation involves adding new CSS for filter buttons, JavaScript logic (`filterByTags`, `toggleTag`) to manage module visibility and hide empty package sections, and modifications to the `gen_stdlib_docs_md.py` script to process tags and support a new `--tags` CLI argument for pre-filtering during generation. This provides a more organized and user-friendly experience for exploring the standard library.

This commit **refactors** and **reorganizes** the **UI sidebar navigation** to significantly **improve usability** and reduce visual clutter. It consolidates trace-related actions into a new **"Trace Files" section**, moves the "Widgets" page to "Settings", and collapses less-frequently-used sections like "Convert trace" and "Support" by default. Additionally, **platform-specific icons** (Android, Web) are introduced for example traces, and a duplicate "Info and Stats" menu item is removed. This **UI/UX improvement** provides a **cleaner, more focused interface** for users, making common workflows more discoverable. The changes affect various **Perfetto UI plugins** and the core `sidebar.ts` definitions.

This commit introduces significant **enhancements to the Query Builder UI**, primarily by adding **node categorization** and robust **computed column functionality**. The **Query Builder's node menu** now groups related nodes, such as `Modify Columns` and `Add Columns`, under a new "Columns" category for improved discoverability. Crucially, the **`Add Columns` node** has been refactored to support creating **computed columns** using custom SQL expressions, `SWITCH` statements, and `IF/THEN/ELSE` logic, replacing the previous "guided" vs "free" mode distinction with a unified, guided interface. This **new capability** empowers users to perform complex data transformations directly within the UI, while the `Modify Columns` node is now focused solely on reordering and renaming existing columns.

This commit provides a **bug fix** for the **Explore page query builder**, addressing issues with **query execution and result display** that arose after migrating to a SQL-backed datagrid. It **simplifies auto-execution logic** by unifying conditions for materialization reuse and auto-execution, and prevents premature clearing of query results by removing unnecessary `resetQueryState()` calls. Additionally, the `SqlSourceNode.onQueryExecuted()` method now uses `notifyNextNodes()` instead of `setOperationChanged()` to update downstream nodes, avoiding unintended re-execution and hash changes. These improvements enhance the reliability and user experience of the **Explore page's query builder**, ensuring consistent query analysis and correct result presentation.

This commit implements a **major refactoring** of the **Explore Page datagrid**, migrating its data fetching from in-memory processing to **server-side pagination** using `SQLDataSource`. Query results are now materialized into Perfetto tables and accessed on-demand, which significantly **reduces memory usage** and **improves performance** for large datasets and filtering/sorting operations. This **enhancement** introduces **new capabilities** such as displaying query statistics (row count, execution duration) and optimizes the materialization lifecycle with debouncing and query hash tracking. The changes primarily affect the **Explore Page's data handling** and UI, leading to a more efficient and responsive user experience.

This commit **refactors** the `ModifyColumnsNode` component within the **Explore Page's query builder** by extracting and utilizing **reusable UI widgets**. It introduces new generic components like `ColumnNameRow`, `CardWithHeader`, `ButtonGroup`, and `Section` in `widgets.ts`, along with their corresponding styles. This **enhancement** significantly improves code organization and establishes a foundational library of standardized UI elements. This will enable other query builder nodes to reuse these components, promoting a consistent user experience and accelerating future development.

This commit introduces a **new feature** and **documentation improvement** for the **PerfettoSQL standard library documentation**. It enhances the `infra/perfetto.dev/src/gen_stdlib_docs_md.py` script to generate interactive dependency graphs, displaying module includes and included-by relationships using Mermaid diagrams. Furthermore, the commit overhauls the documentation's presentation by making module sections collapsible, merging prelude modules into a single synthetic entry, and adding styling for improved navigability. This significantly clarifies module interdependencies and enhances the user experience when exploring the standard library.

This commit introduces comprehensive **validation for standard library module tags**, ensuring all modules with public artifacts are properly categorized and that all tags are valid and correctly structured. It defines a canonical set of `VALID_TAGS` in `stdlib_tags.py` and adds an automatic `_validate_tags()` function that runs on module import to check for tag validity and proper nesting. Furthermore, new functions like `check_tags()` and `check_orphaned_tags()` are added to `tools/check_sql_modules.py` with corresponding CLI flags, providing robust **developer and CI/CD tooling** for proactive tag verification. This **new capability** significantly improves the consistency and reliability of module metadata by enforcing strict tagging standards. Existing `MODULE_TAGS` entries are also updated to comply with the new validation rules.

This commit **fixes column validation** within the **trace summary generation** process to correctly handle structured query transformations. Previously, the validation in `summary.cc` would incorrectly apply to queries using `group_by` or `select_columns`, even though these operations fundamentally change the output schema, making the original `sql.column_names` field irrelevant for validation. The **bug fix** modifies the validation logic to **skip column validation** when such schema-altering transformations are present, ensuring `sql.column_names` is only validated when it accurately reflects the output. This improves the reliability of **trace processor's structured query handling** and is supported by new unit tests in `structured_query_generator_unittest.cc` and integration tests in `summary_integrationtest.cc`.

This commit introduces a **new capability** to categorize and filter standard library SQL modules using tags. It adds a new Python module `stdlib_tags.py` to define and retrieve tags for each `stdlib` module, which are then plumbed through the `format_docs` function in `stdlib_parser.py` during backend processing. On the frontend, the **Explore Page**'s query builder now supports tag filtering within the `TableList` component, with corresponding UI updates in `table_list.scss` and data model changes to the `SqlModule` interface. This feature allows users to filter SQL modules by selected tags, significantly enhancing the discoverability and usability of standard library modules.

This commit performs a significant **refactoring and maintenance** of the **Trace Processor's `slice` table schema** by **removing the unused `stack_id` and `parent_stack_id` columns** and their associated `ancestor_slice_by_stack` and `descendant_slice_by_stack` table functions. This change aims to reduce memory consumption and simplify the data model by eliminating legacy features. It constitutes a **breaking change** for existing queries that relied on these columns or functions, necessitating migration to `parent_id` and `ancestor_slice(id)` for hierarchical navigation. New migration helpers are provided in the **`slices.stack` stdlib module** for backward compatibility, though `stack_id` values will now be computed differently. Comprehensive **documentation** has been updated to guide users through this transition and its implications.

This commit introduces a **usability enhancement** to the **UI's query history feature**, ensuring that the most recently executed or re-run queries are always displayed at the top. Specifically, re-running a query now moves it to the top of its respective starred or unstarred section, eliminating the need for users to scroll to find frequently used queries. This **improves the user experience** by streamlining repeated query execution within the `QueryHistoryComponent` and `HistoryStorage` modules. This **feature enhancement** also includes comprehensive unit tests to validate the new reordering and display logic.

This commit **reverts** a previous rename, restoring the `check_sql_modules` tool to its original name `check_sql_modules.py`. This **maintenance** change addresses issues where multiple external systems depended on the tool having its Python extension. It updates the **build tools**, specifically the `tools/run_presubmit` script, to correctly reference `check_sql_modules.py` within the `CheckSqlModules` function. Additionally, the docstring for `format_full` in `python/generators/sql_processing/stdlib_parser.py` is updated to reflect the correct tool name. This ensures compatibility and prevents breakage for downstream dependencies relying on the `.py` extension.

This commit introduces a **new SQL-based stack table implementation** within the **Trace Processor's standard library (`stdlib`)**, specifically under `slices/stack.sql`. It provides a new view, `_slice_with_stack_id`, which computes `stack_id` and `parent_stack_id` on demand by hashing ancestor slice names, along with helper functions like `_ancestor_slice_by_stack()` and `_descendant_slice_by_stack()` for querying. This **architectural refactoring** prepares for the removal of these columns from the C++ slice table, simplifying the core data model. The change enhances the flexibility of stack analysis within SQL and is supported by new and updated diff tests.

This commit significantly **refactors** the **Perfetto SQL standard library parsing infrastructure** by centralizing shared logic into a new `stdlib_parser.py` module, improving code organization and maintainability. It introduces a new **`tools/check_sql_modules` validation tool** to consolidate and enhance checks for SQL modules, including documentation and dependency declarations, while also relaxing documentation requirements for internal artifacts. Furthermore, this update pushes module includes into the generated JSON output for future documentation improvements and **fixes a missing `INCLUDE` statement** in `src/trace_processor/perfetto_sql/stdlib/wattson/cpu/estimates.sql`. This work enhances the **robustness of SQL module validation** and streamlines the generation of stdlib documentation.

This commit introduces a **new capability** to the **CI/CD pipeline** by posting a direct link to the **UI build artifact** as a comment on pull requests. It **enhances the developer experience** by replacing the previous status check with a more accessible preview, allowing reviewers to easily access and test UI changes. Specifically, the `.github/workflows/ui-tests.yml` workflow is updated to upload the build to GCS and then comment on the PR. Concurrently, the `.github/workflows/analyze.yml` workflow receives necessary permission updates to enable writing to pull requests. This change **streamlines the review process** for UI-related contributions.

This commit provides a **bug fix** for the **CI system**, specifically addressing issues within the `check_stdlib_includes` tool. It updates the `extract_referenced_entities` function with a **new regex pattern** to correctly parse special syntax found in **'IN' clauses**. This change resolves previous CI failures caused by the tool's inability to properly identify referenced entities, ensuring the **CI pipeline** accurately validates standard library includes. The fix prevents incorrect flagging of valid code by correctly interpreting these specific syntax patterns.

This commit introduces a **new capability** to the **documentation generation tool** `gen_stdlib_docs_json.py`, allowing it to optionally include internal artifacts in its JSON output. A new `--with-internal` flag is added to the `gen_stdlib_docs_json.py` script, which modifies the behavior of the underlying `docs_parse.py` module to process and include elements typically prefixed with an underscore. Furthermore, all generated documentation entries now include a `visibility` field, explicitly marking each artifact as "public" or "private" based on its internal status. This enhancement provides greater control over the generated documentation, enabling users to produce more comprehensive outputs for internal analysis while maintaining a public-only default.

This commit **enhances the CI pipeline** by integrating a new presubmit check, `CheckStdlibIncludes`, into `tools/run_presubmit`. This **new capability** automatically validates **stdlib module dependencies**, ensuring that `INCLUDE PERFETTO MODULE` statements are correctly used and preventing "silent imports" within the standard library SQL files. The check runs automatically when stdlib SQL files or the tool itself are modified, significantly improving the **code quality and maintainability of the standard library** by enforcing explicit dependency declarations.

This commit performs a significant **refactoring** of the **Perfetto SQL standard library** by **cleaning up module includes** across numerous `.sql` files. It systematically **removes unused includes** and **adds necessary, previously indirect dependencies** to ensure explicit and correct module linkages. This impacts a wide range of **stdlib components**, including **Android performance metrics** (e.g., `android.cujs`, `android.startup`), **Chrome scroll jank analysis** (`chrome.scroll_jank`), **general utility modules** (`intervals`, `graphs`), and **power estimation tools** (`wattson`). The change is a **code hygiene improvement** aimed at enhancing the **maintainability and dependency clarity** of the stdlib, with **no functional impact** on existing queries or user-facing behavior.