skills — Engineering Performance
15 engineers all time · Jan 2026 – Sep 2026 · built 2026-09-08 · GitHub
Performance snapshot
Today's rolling 90-day reading for skills, compared with the start of the series. Pick a window to move that comparison point.
Avg. perf / dev / mo
−49.8%
0.83 → 0.41 ETV
Active engineers
+140.0%
5.0 → 12.0
Features
−4.9pp
19.4% → 14.5%
vs. UiPath
0.40x
0.79x → 0.40x · −60% below
skills vs. UiPath
Per-engineer ETV for skills against UiPath as a whole. Both lines are 90-day rolling averages scaled to a 30-day month, so they share one axis and can be read against each other at any point. Pick a window to zoom the chart to it.
Performance Composition
Each month's output split by type of work: Features (new value), Maintenance (sustaining systems), Tests, Docs, and Fixes (rework). The yellow line is output per engineer, so when it rises each engineer is delivering more, whatever the team size did. Unit: Engineering Throughput Value (ETV).
Engineering capacity
Effective engineers behind skills, against its pre-AI baseline. Each subject has its own: skills's is 0.83 ETV / dev / mo, its first reading in April 2026. Per-engineer ETV divided by that gives a capacity multiple, and that multiple applied to the engineers active in the trailing 90 days turns it into engineer-equivalents. The line is the real headcount, so the gap between line and area is what the leverage is worth. Because each baseline is its own, every subject opens at 1.0x on its first day: multiples measure improvement and are not comparable between subjects.
Knowledge concentration
How dependent is this repo on a small number of engineers? Higher top-1 share = higher key-person risk.
uipreliga owns 23.1 % of commits.
Behind the numbers
Written summary of the work completed each month.
No monthly reports available yet.
Most impactful commits
Top 10 by ETV in the all-time window.
- 1.4ETVfeat: add agent review (#1069) * feat: add agent review * fix: scaffold lowcode review fixtures in Python, not via uip CLI * fix: address comments * fix: flaky tests * fix: update agent review skill * fix: stabilize uipath-review agent tests for the unreleased review CLI The review-CLI verbs (`uip agent review` / `uip codedagent review`) are not in the deployed CLI yet, and the skill explicitly permits recording them as skipped — so asserting the agent runs them was non-deterministic and failed the smoke pipeline (~50% on low-code). - Drop the `uip agent review` command_executed criterion from the low-code judgment tests and `uip codedagent review` from the coded judgment tests; keep the one reliable instance in the coded delegation smoke. - Add an explicit "load the uipath-review skill" line to the low-code prompts so the agent doesn't select the built-in /review (PR-review) skill. - Remove the low-code delegation smoke — its only distinct assertion was the flaky `uip agent review`; the rest is covered by the low-code judgment tests. Keeps `uip agent validate` on low-code tests (a real, reliably-run command). All 8 remaining agent tests pass at score 1.000 under experiments/default.yaml. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>cristian-groza · e5872ae5 · 2026-06-09
- 1.3ETVSDD creation skill (#239) * Add solution desing skill to create projects from PDDs * refactor * refactor v2 * refactor v3 * changes * add tests * imrpove case management template * improvements * solution support * improvements more, hope doneRaduAna-Maria · 830eba22 · 2026-04-23
- 1.2ETVfeat(audit): /audit-verbs CLI-verb reachability pipeline + /lint-task axis (#833) * feat(lint): add CLI-verb reachability axis to /lint-task Verifies that `uip` verbs referenced in task YAMLs and skill docs exist in the catalog snapshot (`assets/uip-catalog-snapshot.json`), with retired verbs suggesting canonical replacements from `.claude/rules/cli-renames.md`. Catalog is regenerated nightly via the refresh workflow. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs(tests): document test authoring workflow and rules Restructure tests/README.md's authoring section into a numbered `/test-coverage` → `/generate-task` → `/lint-task` → run flow, add a top-level CLAUDE.md pointer, and introduce `.claude/rules/test-writing.md` with the must-do checklist and anti-patterns. Merge `.claude/rules/task-authoring.md` into test-writing.md so the sandbox `env_packages` rule and template pointer live next to the rest of the test-authoring guidance. CLAUDE.md now points to a single rules file instead of two. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(audit): /audit-verbs slash command with markdown report flags Add a one-shot command that runs both verb audits and writes their reports to tests/reports/, alongside the existing /test-coverage outputs. Tests and skills can be audited independently via `/audit-verbs tests` / `/audit-verbs skills`, or together with the default no-arg invocation. - Add `--report PATH` to scripts/check-cli-verbs.py and scripts/check-skill-verbs.py so reports are produced by the checkers themselves rather than by ad-hoc post-processing. - Add .claude/commands/audit-verbs.md describing the four-phase flow (ensure catalog → run audits → summarize → exit code). Reports remain gitignored under tests/reports/. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(audit): address code-review findings; add regression tests Three bugs surfaced by the multi-model code review of the CLI verb audit pipeline: - check-cli-verbs.py: enumerate_paths(allow_partial=True) returned mid-token literals when the regex went dynamic inside a verb token, causing patterns like `uip\s+solution\w+\s+list` to be classified as reachable via the prefix `solution`. Trim accumulated paths back to the last whitespace before returning a partial. - check-cli-verbs.py: extract_verb_paths emitted duplicate verb paths for `(uip|$UIP)`-style alternations, inflating the "Top unmatched verbs" report table. Dedup while preserving order. - build-uip-catalog.py: missing `uip` binary raised an unhandled FileNotFoundError. Wrap subprocess.run with a helpful sys.exit. Plus hygiene from the same review: - build-uip-catalog.py: move UNWALKABLE declaration above its first use, beside other module constants. - check-skill-verbs.py: drop redundant `tokens[0] in unwalkable` short-circuit in severity classification (equivalent to the n=1 iteration of the existing loop). - check-skill-verbs.py: log to stderr when read_text fails on a .md file instead of swallowing silently. - refresh-uip-catalog.yml: sanitize $CLI_VERSION before interpolating into the PR title/body/branch (defense in depth — input is trusted today but the regex was already used for the branch name only). Regression tests in tests/scripts/test_verb_checkers.py cover the three correctness fixes plus the redundant-OR simplification. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(audit): address second-round code review; pin behavior with tests Highs: - check-skill-verbs.py noise filter only fired on single-token tails, so "the uip CLI works" produced a Stale finding. Filter now matches the first token regardless of length (PROSE_NOISE set). - refresh workflow pinned @uipath npm scope to public registry, matching smoke-skills.yml. The internal GH Packages feed carries divergent 1.0.0-alpha.* prereleases and silently fails without auth. Mediums: - check-cli-verbs.py --report now exits 1 only on High, mirroring the contract in audit-verbs.md Phase 4 (was 1 on Medium too). - Word-boundary anchors (\b/\B) treated as dynamic; previously the AT branch skipped them silently and could yield false-reachable verdicts. - build-uip-catalog.py: ThreadPoolExecutor lifted outside the BFS while loop; install_all_tools logs stderr (npm errors) not stdout. - check-skill-verbs.py: extracted _aggregate() to dedupe Counter rollups between write_report and the text path. Docs: - Corrected build-uip-catalog.py docstring and audit-verbs.md guidance to reflect public-npm reality (no NPM_TOKEN needed). Three new regression tests pin the High/Medium fixes: - test_scan_suppresses_prose_noise - test_report_exit_zero_when_only_medium - test_extract_verb_paths_treats_word_boundary_as_dynamic Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(gate): per-skill CLI verb gate as required check Adds .github/workflows/verb-gate.yml — fails a PR when stale `uip <verb>` references exist in the markdown of any skill the PR touches. Mirrors the detect → matrix pattern from smoke-skills.yml / activation-gate.yml so only edited skills are checked, letting the existing doc-drift backlog be swept opportunistically per skill instead of all-at-once. Infrastructure edits (catalog snapshot, checker script, this workflow) fan out to every skill — same rationale as smoke-skills.yml's "test infrastructure changed → run everything" branch — so the nightly refresh-uip-catalog PR will surface CLI-wide drift the moment it lands. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(audit): kill verb-checker false positives + catalog rpa gap Multiple targeted fixes after the CLI Verb Gate's first run flagged ~700 findings, the vast majority of which were parser noise rather than stale verbs: scripts/check-skill-verbs.py - PLACEHOLDER regex accepts optional `.ext` after `<X>` / `[X]` so `validate <ProjectName>.flow` stops at the placeholder instead of treating `<ProjectName>.flow` as a verb token. - Inline `#` shell comments end verb extraction. Fixes `uip registry pull # refresh cache` swallowing the comment text. - Reject dot-bearing tokens (registry resource keys like `core.action.script`, version literals). No catalog verb contains a dot, so this is safe. - Reject leading non-alphanumeric tokens. Catches `uip — heading` patterns and other markdown-rendered non-commands. - Expand PROSE_NOISE with common prepositions, modal verbs, and the literal words "commands"/"version"/"tool" that appear in narrative ("Run uip commands against the platform"). scripts/build-uip-catalog.py + snapshot - PLATFORM_SPECIFIC_PREFIXES seeds `rpa` into UNWALKABLE when it's not exposed as a top-level subcommand. Linux-built catalogs lack the rpa tool (needs Studio Helm), so every `uip rpa <verb>` reference was being flagged as Stale with null prefix. With this fix consumers fall back to Uncertain — same treatment as codedagent/context-grounding. The current snapshot is patched in-place to unblock immediately; future refreshes regenerate it. .github/workflows/verb-gate.yml - Gate blocks only on Stale findings where `matched_prefix == null` (no valid catalog prefix at all). Findings with a valid prefix but a suspicious tail (positional argument values mis-parsed as sub-verbs) surface as ::warning annotations on the PR instead of failing the build. Hard-fail stays for the unambiguous case where the entire verb path is unknown to the catalog. tests/scripts/test_verb_checkers.py - Regression tests for the three new false-positive classes: `<X>.flow` placeholders, inline `#` comments, dot-separated argument values. Local projection after fixes: 4 skills still show 1 blocking finding each. Three are real stale verbs (`auth login` → `login`, `orchestrator folder list` → `or folders list`); one is a doc that literally documents `(uip orchestrator does not exist)` as a negative example and trips the parser. Down from 18/19 failing. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(refresh): auto-merge nightly catalog refresh when checks pass The nightly refresh-uip-catalog workflow opened a PR that sat waiting for a human to merge. Two failure modes: 1. Unreviewed PRs block subsequent refreshes (the existing "skip if open refresh PR" check short-circuits) — so a stalled PR freezes the catalog until someone gets to it. 2. Concurrent skill PRs gate against the stale catalog and may pass gates they'd fail against the latest CLI surface. Switch to PAT-based PR creation and arm auto-merge: - `GH_TOKEN: secrets.GH_PAT` (same PAT already used by activation-gate.yml). Required because PRs opened with the default GITHUB_TOKEN do not trigger `pull_request:` workflows — the verb gate and smoke tests would never run, and `--auto` would merge before any drift signal arrived. - `gh pr merge --auto --squash --delete-branch` — waits for all required status checks, then squashes and deletes the branch. If a check fails (real CLI drift breaking skill docs or task YAMLs), the PR stays open with the gate annotations attached, surfacing the affected references for a human sweep. Body text updated to explain the auto-merge contract: green = no drift, no review needed; red = real signal worth investigating. Repo settings prerequisite: 'Allow auto-merge' must be enabled (Settings → General → Pull Requests). If branch protection requires review approval, the bypass list must include whoever owns GH_PAT or the github-actions bot — otherwise the PR will sit waiting for a human approver even after checks pass. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor(refresh): scope auto-merge to refresh PRs only, not repo-wide Previous commit used `gh pr merge --auto` which depends on the repo's 'Allow auto-merge' setting — a repo-wide capability that any human or workflow could opt into for any PR. That's a convention, not a hard restriction. Replace with a dedicated `automerge-catalog-refresh.yml` workflow that: - Triggers on `check_suite: completed` (fires when any CI suite finishes). - Lists open PRs filtered by branch pattern `auto/refresh-uip-catalog-*` AND label `automated` — defense-in-depth re-verifies the branch pattern per-PR before merging. - Checks the full statusCheckRollup: skips if anything is still running, skips if anything failed, merges with `--squash --delete-branch` when every check is SUCCESS/SKIPPED/NEUTRAL. - Accepts `workflow_dispatch` with optional `pr_number` for emergency manual triggers. The repo-wide 'Allow auto-merge' setting can stay off. Auto-merge is now enforced in code by the branch-name filter, scoped strictly to the catalog refresh PR class. refresh-uip-catalog.yml drops the inline `gh pr merge --auto` call and updates the PR body to reference the dedicated automerger workflow. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore(catalog): refresh uip catalog to 1.2.0 Regenerated against uip 1.2.0 (was 1.0). Clears most of the apparent "doc drift" the verb checker had been flagging — top offenders like `agent migrate` (71 hits), `solution resource list` (57), `solution init` (38), `api-workflow registry stub` (17), and `agent guardrails list` (16) were catalog drift, not real stale references. New top-level groups now walkable: agenthub, tm, tools, traces, update, user, vss. `codedagent` is no longer wholesale unwalkable — broken into per-subcommand unwalkable entries so the checker walks the ones the runner exposes. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(audit): register 1.2.0 verb renames + fix latent classifier bugs Adds the two retired-verb entries that 1.2.0 introduced: - `flow` → `maestro flow` (100 leaf verbs moved under the `maestro` group) - `solution new` → `solution init` Wiring these surfaced two latent bugs in the cli-verb classifier that the empty rename table had been hiding: 1. `load_renames` kept markdown backticks in the dict keys, so entries like `| `solution new` |` produced `'`solution new`'` keys that no extracted verb could ever match. Strip backticks in the loader. 2. `classify` preferred any catalog longest-prefix over a more-specific renames entry, so `solution new` (retired) was getting shadowed by the still-walkable parent group `solution` and mis-classified as reachable. When the renames match is more specific than the catalog prefix, prefer the renames verdict. Both bugs are pinned with regression tests. After the fix, two task YAMLs that pattern-match `uip\s+solution\s+new` correctly surface as Medium with the canonical replacement suggested. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(audit): honor uip-check-skip + track cli/main via @alpha catalog Three reviewer items folded into the audit pipeline before merge: 1. `<!-- uip-check-skip -->` opt-out in check-skill-verbs.py. The maestro skills already ship `.maintenance/check-uip-commands.sh` that supports a per-line skip marker for intentional historical references — CLI version-comparison tables documenting removed prefixes, fallback-prefix sentences. Without honoring the marker here, the new gate hard-fails on lines that explicitly opted out (concrete usability regression flagged in review). Drops hard-fail findings from 35 → 25 across the 9 affected skills; pinned with regression test `test_skip_marker_suppresses_line`. 2. Refresh workflow tracks @alpha from GitHub Packages, not @latest from public npm. Public npm carries the cut-release `latest` line which lags `cli/main` by ~2 weeks; auditing against it would mean the gate audits skill docs against a CLI surface two weeks behind what developers actually run. Matches the install pattern already established in smoke-skills.yml. 3. Self-heal `automated` label in the refresh workflow. The label is the join key the auto-merger uses to identify refresh PRs, and it doesn't currently exist in the repo. Creating it idempotently with `gh label create --force` keeps the workflow self-contained — no one-time repo setup required after a fresh fork. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>uipreliga · a396797d · 2026-05-26
- 1.0ETVMove uip solution comands to uipath-solution skill (#821) * refact(skills): move task-list ownership from solution-design to planner Solution-design now produces architecture only — SDDs end with a Next Steps pointer at uipath-planner. The planner reads SDDs via the new ## Planner Handoff header, derives tasks per the project list, and emits live TaskCreate calls. Single source of truth for skill-routing prompts; clearer separation between architecture (solution-design) and orchestration (planner). Cross-skill contract: - ## Planner Handoff heading is the load-bearing detection marker - 6-field header (Execution autonomy, SDD scope, Project list section, Tasks file, Generated by, Generation date) - Pipeline: PDD → solution-design (SDD) → planner (tasks.md) → execution Solution-design changes: - Removed Implementation Plan section from all 6 SDD templates - Added ## Planner Handoff header + ## Next Steps to all 6 templates - SKILL.md: added When to Use, fleshed out Workflow, dropped rules 8/11, added planner redirect to description, listed package-selection-guide - sdd-generation-guide.md: collapsed Phase 1 Steps 4/4.25/4.3/4.5 into one orchestrator step linking to canonical level homes; deleted Phase 3 Steps 3 and 4 (no more TaskCreate / execute-implementation flows); rewrote Phase 3 Step 2 to write the Planner Handoff header - product-selection-guide.md: explicit canonical home for Levels 1, 1.75, 2.5 Part B, 3; clarified RPA-specific levels live in rpa-product-guide - rpa-product-guide.md: explicit canonical home for Levels 1.5, 2, 2.5 Part A and naming/REFramework guidance Planner changes: - SKILL.md slimmed from 385 to 163 lines; restructured around two lanes (PDD-driven, non-PDD) plus an Entry Guard - Description rewritten with PDD redirect; TaskCreate/TaskUpdate/TaskList added to allowed-tools - 4 new references extracted: pdd-driven-lane-guide.md, non-pdd-lane-guide.md, multi-skill-patterns-guide.md, plan-and-tasks-format.md - Question budget reframed to count AskUserQuestion calls (not individual questions); cap stays at 5 - Dropped Stop conditions section and test coverage question — testing is always thorough, specialists handle their own pause points - New regenerate-with-preservation logic: identity tuple matching preserves completed work across SDD revisions; archive footer for removed tasks Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * update tests * updates * resolve claude comments * add test * resolve comments * refactor * feat(skills): merge uipath-solution-design + uip solution ops into uipath-solution Merge the PDD->SDD authoring half of uipath-solution-design with the `uip solution` CLI lifecycle slice of uipath-platform into a single bifurcated skill at skills/uipath-solution/. Design half keeps the 15 critical rules and 3-phase workflow; Operate half lifts solution CLI guidance into 9 explicit rules plus the pre/post-rename probe. uipath-platform retains auth, Orchestrator, resources, IS, traces, and licensing; its description and Task Navigation drop solution tokens and redirect to uipath-solution. Retarget consumers (uipath-planner SKILL + 4 refs, uipath-rpa, uipath-review, uipath-maestro-case, uipath-maestro-flow). Rename activation dataset and gate-script baseline per R1b (omit, rebaseline after next full run). Bump plugin to 0.0.29. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * iteration 2 * resolve issues * add tests * resolve comment * do not update versions * solve tests --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>RaduAna-Maria · feca7721 · 2026-05-19
- 1.0ETVtest(uipath-agents): add 10 coded-agent tests for high-priority coverage gaps (#633) Closes the highest-priority gaps in the coded test-coverage report with 10 new e2e/integration tests. Each test has a user-facing prompt (the agent must consult the skill to figure out the fix) and either a check script asserting artifact shape or direct command_executed / file_contains criteria. - local_workspace_iterate — Studio Web Local Workspace iteration: enforces 4 of 5 Local-Workspace anti-patterns (no scaffold-new, no manual push, preserve project.uiproj, preserve UIPATH_PROJECT_ID) - push_pull_roundtrip — uip codedagent push + pull roundtrip against an existing Studio Web project - hitl_wait_task — interrupt(WaitTask) HITL pattern - hitl_create_task_with_app — interrupt(CreateTask) + Action Center app binding - coded_in_flow_published — Flow Integration Pattern 2: a published coded agent referenced from a separate Maestro Flow - antipattern_openai_agents_hitl — OpenAI Agents framework boundary: interrupt() does not work, skill must remove it - deploy_folder_and_rebump — uip codedagent deploy --folder plus patch-version bump on re-deploy - auth_one_shot_login — uip login one-shot form with both --organization and --tenant (Critical Rule 2) - existing_project_resume — existing-coded state with has_venv == false / has_entry_points == false; skill must prep venv + regenerate schema in place - llamaindex_rag — LlamaIndex Workflow + UiPath Context Grounding (ContextGroundingQueryEngine) for RAGRadu Mihai Gheorghe · 14316f47 · 2026-05-18
- 0.8ETVfix(uipath-review): reduce review verbosity and fix grade formula (#2471)Andrei Ancuța · c27677a8 · 2026-08-18
- 0.8ETVrefact(skills): move task-list ownership from solution-design to planner (#538) * refact(skills): move task-list ownership from solution-design to planner Solution-design now produces architecture only — SDDs end with a Next Steps pointer at uipath-planner. The planner reads SDDs via the new ## Planner Handoff header, derives tasks per the project list, and emits live TaskCreate calls. Single source of truth for skill-routing prompts; clearer separation between architecture (solution-design) and orchestration (planner). Cross-skill contract: - ## Planner Handoff heading is the load-bearing detection marker - 6-field header (Execution autonomy, SDD scope, Project list section, Tasks file, Generated by, Generation date) - Pipeline: PDD → solution-design (SDD) → planner (tasks.md) → execution Solution-design changes: - Removed Implementation Plan section from all 6 SDD templates - Added ## Planner Handoff header + ## Next Steps to all 6 templates - SKILL.md: added When to Use, fleshed out Workflow, dropped rules 8/11, added planner redirect to description, listed package-selection-guide - sdd-generation-guide.md: collapsed Phase 1 Steps 4/4.25/4.3/4.5 into one orchestrator step linking to canonical level homes; deleted Phase 3 Steps 3 and 4 (no more TaskCreate / execute-implementation flows); rewrote Phase 3 Step 2 to write the Planner Handoff header - product-selection-guide.md: explicit canonical home for Levels 1, 1.75, 2.5 Part B, 3; clarified RPA-specific levels live in rpa-product-guide - rpa-product-guide.md: explicit canonical home for Levels 1.5, 2, 2.5 Part A and naming/REFramework guidance Planner changes: - SKILL.md slimmed from 385 to 163 lines; restructured around two lanes (PDD-driven, non-PDD) plus an Entry Guard - Description rewritten with PDD redirect; TaskCreate/TaskUpdate/TaskList added to allowed-tools - 4 new references extracted: pdd-driven-lane-guide.md, non-pdd-lane-guide.md, multi-skill-patterns-guide.md, plan-and-tasks-format.md - Question budget reframed to count AskUserQuestion calls (not individual questions); cap stays at 5 - Dropped Stop conditions section and test coverage question — testing is always thorough, specialists handle their own pause points - New regenerate-with-preservation logic: identity tuple matching preserves completed work across SDD revisions; archive footer for removed tasks Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * update tests * updates * resolve claude comments * add test * resolve comments * refactor --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>RaduAna-Maria · 0b6d67ef · 2026-05-15
- 0.7ETVRPA: Merge coded-workflows + rpa-workflows into unified uipath-rpa skill (#160)Gabriela Vaduva · 833f595e · 2026-04-07
- 0.7ETVfix(ci): scope the skills.sh drift check to drift the PR introduces (#3043) * fix(ci): scope the skills.sh drift check to drift the PR introduces `Validate skills.sh.json against skills/` failed 68 times in the last three months — a 6.1% failure rate, the highest of any deterministic gate in the repo. 65 of those 68 failures were one incident. #2252 landed `skills/uipath-process-mining/` on 2026-08-04 without a `skills.sh.json` entry. The checker reads the whole tree, so from 15:52 that day until 17:13 on 2026-08-06 every PR touching any `skills/*/SKILL.md` went red for drift it did not cause — ~20 unrelated branches, 65 failed runs. The entry was finally added by an unrelated PR (#2498, a solution zip-filename fix) that happened to trip over it. The remaining 3 failures were skill-adding PRs failing legitimately. So 96% of this check's failures were false positives, which is also why it merged red 5 times: a check that cries wolf gets clicked through, and that is how the drift reached main in the first place. Fix: `--baseline-ref` compares findings against the base commit. Findings that also reproduce at the baseline print as `::warning::` and do not fail the run; only findings the change introduces exit 1. Whole-tree visibility is kept — the blast radius is not. An unreadable baseline (shallow clone, missing base) falls back to strict whole-tree behaviour, so the failure mode is the old behaviour rather than a silent pass. Replayed against the real incident: #2252 (adds process-mining) new=[uipath-process-mining] exit 1 ✓ still blocks #2436 (unrelated, maestro-case) pre-existing exit 0 ✓ no longer blocks #2498 (adds the entry) clean exit 0 ✓ Docs: `skills.sh.json` is not derived from disk, so a rename or removal leaves it stale with no symptom other than this check. CONTRIBUTING.md covered adding a skill, but it is not loaded into agent context — CLAUDE.md and `.claude/rules/` are, and neither mentioned the file. Added the add/rename/remove contract to CLAUDE.md (rule 5 + File Conventions), `.claude/rules/skill-structure.md` (new § skills.sh Grouping), `.claude/rules/pr-review.md` (both checklists), and a rename/removal subsection in CONTRIBUTING.md. Tests: `tests/scripts/test_check_skills_sh.py` (15 cases) pins the contract — pre-existing drift warns, introduced drift fails, a dirty baseline is no shield for fresh drift, renames fail on both halves, an unreadable baseline stays strict. Registered as a Test Helpers job. This also makes the check safe to add to the required set. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(ci): address review — stable finding identity, no paths filter, HEAD^1 baseline Review from @bai-uipath. All three findings reproduced and fixed. 1. Finding identity keyed on the formatted error string, which embeds grouping titles and list indices, so unrelated edits re-attributed pre-existing drift as new. Reproduced against the previous commit: retitle a grouping (duplicate drift already present) -> exit 1 insert a grouping at the top (schema drift present) -> exit 1 Findings now carry an explicit `kind` and a `subject`, and identity is `(kind, subject)`. `key`/`error` stay display-only, so output is unchanged. Coverage findings key on the skill name alone — retitling a grouping or moving a bad entry between groupings is not new drift. Schema findings key on the grouping title when it is valid, never the index. Both scenarios now exit 0, and five tests pin it. 2. Dropped the `paths:` filter. The reviewer is right that blast radius was not the only failure mode: the check never ran on #2252 at all — 19 checks reported on that head, and this was not among them — even though the workflow had existed since 2026-07-31 and the filter listed `skills/*/SKILL.md`. The guard was silent on the one PR it existed to catch. validate-version-sync.yml and smoke-rpa-skills.yml already document the required-check rationale verbatim; this workflow now matches. ~20s job, so it just always runs. A manual dispatch gets no baseline and validates the whole tree strictly. 3. Removed `fetch-depth: 0` from the new test job. Every test resolves HEAD or a deliberately missing ref, both fine on the default shallow clone. Also took both optional suggestions: - Baseline is now `HEAD^1`, the base parent of the merge commit actually under test, rather than the event payload's `base.sha`, which can lag the merge ref GitHub recomputed. Let the validate job's fetch-depth drop from 0 to 2. - CLAUDE.md rule 5 now points at the skill-structure rule for the full change→edit table instead of restating the `--fix` limits, per .claude/rules/token-optimization.md § 3 (one full statement + pointers). And the Claude review bot's finding: test_cli_json_output_carries_the_ preexisting_flag passed by vacuous truth — a clean tree emits no JSON, so the loop body never ran. Replaced with a fixture that produces one pre-existing and one introduced finding, asserting both `preexisting` values actually emit, plus a separate test that a clean tree stays silent. 21 tests, all passing. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>uipreliga · 302e5f24 · 2026-09-03
- 0.7ETVrefact: consolidate coded agents skills (#418)Radu Mihai Gheorghe · 1222f2b7 · 2026-05-04