You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Skills: split oversized skills into a core plus references - #1060
Follows #1050, which has merged. main is merged in, including #837's new winapp-performance skill. Skill names and descriptions don't change, except one wording fix to winapp-performance (see below).
What this does
The largest skills now keep a short core in SKILL.md and move detail into references that load only when needed. Every reference has a Load when row in its core. Consent and safety rules stay in the core.
Skill
Core before → after (~tokens)
Moved into references
winapp-ui-automation
9,413 → 4,216 (4,617 after merging main's additions)
keyboard and hover (input.md), drag, touch, pen, and scrolling (gestures.md), video recording (capture.md), reading state, setting values, and file dialogs (dialogs-and-state.md), troubleshooting, JSON envelope highlights (ui-json-envelope.md)
winapp-find-api
6,396 → 3,048
less common patterns and verb syntax, concepts, troubleshooting
winapp-setup
6,450 → 2,639
private NuGet feeds (nuget-feeds.md). winapp run project mode and single-file .cs apps moved to winapp-identity references
winapp-identity
2,863 → 3,116
receives run-project-mode.md and single-file-apps.md. Its production sparse packaging section moved to production-sparse.md
winui-ui-testing
5,585 → 2,027
batch script template (Step 2), pickers, flyouts, dialogs, and input, Sandbox persistence and file transfer
winui-code-review
1,638 → 1,744
quality-rules.md (3.2k) is split into performance, security, accessibility, code-quality, and globalization references, so a review reads only the topic it needs
winui-design
3,388 → 2,971
window sizing. brushes-and-icons.md is split into brushes.md and icons.md
Totals: unique core bodies drop from 65.5k to 49.5k tokens (−24%). The largest core is now 4.2k (was 9.4k). Always-loaded metadata doesn't change (WinApp 987, WinUI 594; winapp-performance from main adds 65).
Stale fixes
No wildcard Get-AppxPackage … | Remove-AppxPackage. winapp-package now looks up the exact package by identity name and removes it only after the user confirms.
Removed the contributor-only scripts\test-samples.ps1 section from winapp-maui. It now links to the MAUI sample on GitHub.
Removed the v0.3.1 envelope history from the winapp ui --json reference.
Cross-skill paths: winui-design, winui-code-review, winui-dev-workflow, and winui-wpf-migration now say "load the winui-packaging skill and read its sourcegen-patterns.md reference". Before, a bare references/… path didn't resolve from those skills. The C5 spike copied the file into four skills. This PR keeps one copy instead.
The WinUI agent said "chain dependent commands with &&". It now says to use ; in PowerShell, which matches winapp-ui-automation.
The spike's new content (Store reference, Fluent without porting, C++ WinUI guard, AutomationId authoring rule) is not in this PR.
Benchmark capability maps
No capability map changes. Every capability's content still sits in the core: commands, triggers, and the main workflow. References add detail to a capability, but no capability lives only in a reference. So skill-level credit stays correct, and adding file keys would only repeat the parent skill's capabilities. The existing winapp-current/winui-current maps apply to both sides.
winapp-ui-automation: the core tip now says win+… shortcuts need --via send-input --allow-system-keys. Under the default --via post-message transport, keys go only to the target window, so shell hotkeys never fire and the flag only prints a warning (checked against UiSendKeysCommand.cs). win+l and ctrl+alt+del stay blocked.
winapp-setup: the core again warns, next to init and restore, that both commands use the target directory's nuget.config and should run only on directories you trust.
winui-dev agent, both copies: "Use ; only for independent commands. For dependent steps, check $LASTEXITCODE before the next command."
Merging main
Main's newer skill facts are placed like the rest of the split:
winapp-ui-automation core: queries by --type/--class-name without a selector, exact-name matching for single-element commands, clicking or hovering at a screen point (--at), --wheel-horizontal in the input and exclusive-command lists, and a pointer to winapp-performance.
winapp-ui-automation references: the touch --angle and horizontal-wheel examples are in gestures.md. Restoring a minimized window before focus is in dialogs-and-state.md, with a one-line pointer in the core tips.
winapp-setup: main's updated winapp run output line is in winapp-identity's run-project-mode.md, where project mode now lives.
Main's two relative links to ../winapp-performance/SKILL.md (in winapp-ui-automation and winapp-troubleshoot) now name the skill, so no cross-skill relative paths remain.
winapp-performance is 1,385 tokens, so it isn't split.
benchmarks/agents -Lint had 1 error and benchmark Pester had 2 failures on main:
Capability map:winapp-performance maps to the existing winui.build capability ("build, run, and debug a WinUI 3 app"), because diagnosing a slow running app is debugging. winui.build now also lists ^perf commands and mentions slow layout or scrolling in its description. I didn't add a new capability, because that would also need at least two blind-written held-out scenarios. The winui.* prefix keeps loading winapp-performance in a non-WinUI scenario a failure, which matches its WinUI-only scope. A new winapp-current map covers the 13 skills. The previous 12-skill map stays as winapp-before-performance, so earlier results still score.
Command scoring: the benchmark's command extractor didn't know perf, so winapp perf … in an answer never counted. It now does, with tests for extraction and scoring.
Lint: the description said "not crash dumps", and a demand fixture is named crash-dump-summary.txt. User-visible change: the winapp-performance description now ends "…not crash analysis or UI correctness." (238 characters).
Spot check of winapp-ui-automation after the merge
main vs this branch, the 10 dev and held-out prompts whose primary capability is ui.automate, 3 models, 1 iteration, 768 credits:
main
this branch
Routing pass
28/30
27/30
Answer met
23/30
23/30
Skill text when winapp-ui-automation loaded (core + references)
12,815
8,232
The one routing flip is a Sonnet run that loaded no skill at all, which happens before any skill body loads.
Results
Both sides ran on the same harness, at #1050's head (408dc75d), and the baseline plugins come from that commit. Since then, main changed only winapp-find-ui and winapp-sandbox, and this PR doesn't touch either. Configuration both, no agent, 3 models, 2 iterations. Differences are paired (same scenario, model, and iteration). 95% CIs are bootstrapped over scenarios.
Outcomes and routing
Measure
Pairs
Base
Split
Δ [95% CI]
Better / worse
Held-out routing
674
0.737
0.731
−0.006 [−0.023, +0.012]
19 / 24
Held-out answer
521
0.835
0.840
+0.005 [−0.010, +0.019]
25 / 20
Demand outcome (two judges)
360
0.792
0.796
+0.004 [−0.019, +0.026]
40 / 36
By model:
Model
Held-out routing Δ
Held-out answer Δ
Demand outcome Δ
claude-sonnet-5.5
−0.024 [−0.069, +0.020]
+0.003 [−0.036, +0.042]
−0.004 [−0.054, +0.042]
claude-opus-5.5
+0.018 [−0.007, +0.046]
+0.014 [+0.000, +0.036]
−0.010 [−0.044, +0.023]
gpt-6.1-sol
−0.011 [−0.033, +0.009]
−0.003 [−0.034, +0.024]
+0.027 [−0.002, +0.058]
Every CI includes zero. Pairs where neither side loaded a split skill have identical skill text, so they show the noise level: demand +0.004 [−0.019, +0.027], held-out routing −0.001. Pairs where a split skill loaded: demand +0.007 [−0.071, +0.083], held-out answer −0.006 [−0.050, +0.029].
Tokens
Per run
Base
Split
Δ
Held-out, skill context + reference reads
3,287
2,679
−18%
Demand, skill context + reference reads
1,702
1,550
−9%
Runs that loaded a split skill, GPT
9,975
6,834
−31%
Runs that loaded a split skill, Opus
5,628
4,668
−17%
Runs that loaded a split skill, Sonnet
4,681
3,931
−16%
Credits per run are flat (held-out 10.5 → 10.3, demand 9.5 → 9.4), because Copilot's own prompt dominates input.
Reference reads
When a split skill loaded, the agent read one of its references in 34–52% of runs (GPT 34%, Opus 39%, Sonnet 52%). By skill:
Answers hold for every capability. The ui.test/ui.automate routing dips come from 8 flipped pairs. In 5 of them, Sonnet loaded no skill at all on one side. In the other 3, a different second skill was chosen after the first skill loaded, and one of those favored the split. Routing happens before the skill body loads, and descriptions didn't change, so these flips aren't caused by content moving into a reference. No content was pulled back into a core.
Ship rule
Outcomes and routing hold within noise, and loaded skill text drops. This passes.
Credits
Spot check after the merge: 768. Original measurement: 22,912 of the 23,000 cap: runs 20,969 (2,082 runs) and two-judge grading 1,943. Opus held-out iteration 2 stopped at its credit cap with 14 runs left. I finished 8 of them, enough to complete their pairs within the cap. The 3 wrenfall-category-browser-example prompts have no Opus iteration-2 pair; their iteration-1 pairs still count.
Validation
scripts\validate-plugin-package.ps1: 0 errors
scripts\validate-llm-docs.ps1 against a Debug CLI: command examples match the schema
validate-plugin-package.ps1 -CliSchemaPath with a fresh Debug CLI from this branch, and validate-llm-docs.ps1: pass
Test-WinuiUiTestingSandbox.ps1 and Test-SetupVersionDetection.ps1: pass
scripts\tests\Test-WinuiUiTestingSandbox.ps1 now checks the skill's core and references together, because the batch template moved into a reference. It passes.
Please check by hand
winapp-ui-automation: the core still reads well for a first-time winapp ui user. The new focus and --allow-system-keys lines in Tips restate rules that moved to references.
winui-ui-testing: Step 2 now points to references/batch-template.md. The core also restates the Sandbox shutdown consent rule.
winapp-package: the uninstall example (exact identity name, user confirmation) replaces the wildcard.
winapp-identity now owns winapp run project mode and .cs apps, and winapp-setup points there. Is that the split you want?
Whether to share one AOT reference across skills, as done here, or to copy it into each skill, as the spike did.
Coverage checklist: where every section of the changed skills went
Generated by matching each section's lines in main (4b681dd1, the unsplit skills with all of main's newer facts) against this branch. Sections marked "(edited)" changed some lines: stale fixes, link or pointer fixes, or a self-reference.
winapp-find-api/SKILL.md
Section
Now in
(intro)
winapp-find-api/SKILL.md
This is an agent-first command
winapp-find-api/SKILL.md
When to use
winapp-find-api/SKILL.md
Use it on compile errors, not just before writing code
… fix stale guidance
Same skill names and descriptions. Moves detail out of winapp-ui-automation, winapp-find-api,
winapp-setup (run/debug content now lives in winapp-identity references), winui-ui-testing,
winui-code-review (quality rules by topic), and winui-design (brushes, icons, window sizing).
Every reference has a 'Load when' row in its core; consent and safety rules stay in core.
Stale fixes: no wildcard Remove-AppxPackage; drop the contributor-only test-samples line and
the v0.3.1 envelope history; name the owning skill for the AOT reference instead of a
cross-skill relative path; the WinUI agent no longer advises chaining with &&.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Splits large WinApp and WinUI skills into concise cores with on-demand references, reducing loaded context while preserving workflows and safety guidance.
Changes:
Extracts detailed guidance into focused reference files with load conditions.
Moves run/identity guidance from setup into the identity skill.
Corrects stale uninstall, MAUI, cross-skill, and PowerShell guidance.
This newly duplicates the Debugging by framework section from winapp-frameworks/SKILL.md:117-134 verbatim. That adds always-loaded core text and creates two canonical copies that can diverge when framework commands change, contrary to this PR's core-size goal. Keep the matrix in winapp-frameworks and use a short cross-skill pointer here.
This issue also appears on line 192 of the same file.
Loading this reference for High Contrast currently leads to a dead end: references/theme-accessibility.md:38 still tells the reader to open brushes-and-icons.md, but this split removes that file in favor of brushes.md and icons.md. Update the stale pointer to brushes.md so the HC pairing table remains reachable.
Other suites are reflected in the overall validation status above.
✅ 10419 passed, 43 skipped out of 10462 tests in 2204.6s (-39.9s vs. baseline)
Test Coverage
✅ 87.2% line coverage, 81.3% branch coverage · ✅ no change vs. baseline
CLI Startup Time
55ms median (x64, winapp --version) · 📉 -11ms vs. baseline
Try This Build
Installs the MSIX for your architecture, replacing any previously installed build. Needs the GitHub CLI — the command offers to install it and sign you in if it is missing.
The reason will be displayed to describe this comment to others. Learn more.
🤖 AI-generated review (winappcli pr-review skill) — verify before acting.
The split looks good. Every moved section lands in a reference, every reference has a Load when row, and the deleted file names aren't referenced anywhere. validate-plugin-package.ps1 (with a CLI schema built from this branch), Test-WinuiUiTestingSandbox.ps1, and the benchmark Pester tests all pass. Please make these three small fixes before merging.
1. The Windows-key shortcut tip gives a command that does nothing
What is wrong: The new tip in the core says win+… shortcuts "are refused unless you pass --allow-system-keys". It leaves out --via send-input, which is the option that makes the shortcut fire. The example this PR removed had it.
Show me:winapp ui send-keys "win+r" -a myapp --allow-system-keys → prints a warning that "--allow-system-keys only applies to --via send-input", exits 0, and no Run dialog opens. Expected: winapp ui send-keys "win+r" -a myapp --via send-input --allow-system-keys. See UiSendKeysCommand.cs, around line 230. Under the default post-message transport nothing is refused either, so "are refused" is also wrong.
Why it matters: An agent that reads only the core runs a command that does nothing and reports success.
Smallest fix: Change the tip to say --via send-input --allow-system-keys.
2. The trusted-directory warning for init/restore left the setup core
What is wrong: The note "run these commands only against directories you trust" moved to references/nuget-feeds.md. Agents load that file only for a private feed or a 401/403 error, but the risk applies to every init and restore. This contradicts "consent and safety rules stay in the core".
Show me: Clone a repo whose nuget.config adds an unknown feed, then ask the agent to "set up this project with winapp". It loads only the core, sees no warning, and runs winapp restore against that feed.
Why it matters: Packages from an untrusted feed can run code during restore or build.
Smallest fix: Put one sentence back in the core next to init/restore: "init and restore use the target directory's nuget.config. Run them only on directories you trust."
Location:plugins/winapp/skills/winapp-setup/SKILL.md, near the Load when table (line ~161)
3. The ; chaining advice no longer separates dependent steps
What is wrong: "Chain dependent commands with &&" became "chain commands with ;". In PowerShell, ; runs the next command even if the previous one failed.
Show me:dotnet build ; winapp run . → the build fails, then the old binary launches anyway.
Why it matters: The agent tests stale output and reports success on code that didn't build.
Smallest fix: Add "Use ; only for independent commands. For dependent steps, check $LASTEXITCODE before the next command."
Location: line 24 of both plugins/winui/agents/winui-dev.agent.md and plugins/winui/agent-plugin/com.github.copilot/agents/winui-dev.agent.md
On your two questions: moving winapp run project mode and single-file apps to winapp-identity makes them easier to find, because that skill's description mentions running with identity and winapp-setup's doesn't. Keeping one shared sourcegen-patterns.md instead of four copies is the right call.
Places main's new winapp-ui-automation facts in the core (scoped queries without a
selector, exact-name matching, click or hover at a point, horizontal wheel in the
exclusive list, performance pointer) or in references (touch --angle and horizontal
wheel examples in gestures.md, restoring a minimized window in dialogs-and-state.md).
The winapp run output change lands in winapp-identity's run-project-mode.md.
Also fixes what #837 broke in the benchmark: winapp-performance maps to winui.build in
a new current map (the old map stays for earlier results), and its description says
'crash analysis' instead of 'crash dumps', which collided with a demand fixture name.
Cross-skill relative links to winapp-performance now name the skill.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…ng, dependent-step chaining
- winapp-ui-automation: win+ shortcuts need --via send-input --allow-system-keys; under the
default post-message transport they never fire, and the flag only warns.
- winapp-setup: init and restore use the target directory's nuget.config; the warning to run
them only on trusted directories is back in the core.
- winui-dev agent (both copies): use ; only for independent commands; check LASTEXITCODE
before a dependent step.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
winapp-ui-automation: the tip now says win+… shortcuts need --via send-input --allow-system-keys. Under the default post-message transport, keys go only to the target window, so shell hotkeys never fire and the flag only warns (checked against UiSendKeysCommand.cs).
winapp-setup: the core again warns, next to init and restore, that both use the target directory's nuget.config and should run only on directories you trust.
Both winui-dev agent copies, line 24: "Use ; only for independent commands. For dependent steps, check $LASTEXITCODE before the next command."
I also merged main in 8e3dea3, placed its new winapp-ui-automation facts in the core or the matching reference, and fixed the benchmark lint and Pester failures from #837. Details are in the PR description.
Get-WinappCommands didn't know 'perf', so answers and shell calls that ran
'winapp perf ...' never matched winui.build's ^perf pattern. Adds perf as a
top-level command group, with tests for extraction and command scoring.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
The reason will be displayed to describe this comment to others. Learn more.
🟢 Approval recommended
The reference moves preserve core workflows and safety guidance, and the benchmark changes include targeted regression tests with no unresolved blocking findings.
…enchmarks/ui (#1053)
## What and why
Adds `benchmarks/ui/`, two repeatable benchmarks for `winapp ui`. Rerun
them after a `winapp ui` change to see its effect.
- **`run.ps1` (agent benchmark).** A Copilot CLI agent does seven
natural-language tasks in inbox Calculator and a pinned WinUI 3 Gallery
(`v2.9.3`, built once by `setup-gallery.ps1`). The tasks include
real-world friction: the app isn't running, two instances are open, the
window is minimized, a dialog was left open, and a cold start. The
harness scores the final app state itself through UIA, the process count
and the window state. It never trusts the agent's answer, and it flags
collateral damage such as extra instances, the wrong window changed, or
an app closed. The report shows the pass rate, wall time, turns, tool vs
model time, tokens and credits. Methods: `winapp` (this repo's plugin
and CLI) and `none` (no plugin, `winapp` hidden).
- **`perf.ps1` (scripted, no agent, no credits).** It times single
`winapp ui` commands (20 runs after a warm-up; median, p95 and max). It
also runs fixed command sequences that mirror the agent tasks, scored
with the same checks.
Both benchmarks refuse to start when Calculator or the Gallery is
already open (unless you pass `-Force`), close only the processes they
launched (by PID), and restore your Calculator mode afterward. `run.ps1`
resumes when you rerun it into the same `-OutDir`. Other tools can be
benchmarked locally with `-Method <file>` or `-Adapter <file>`; the
README documents both formats.
`benchmarks/agents/lib/Benchmark.psm1` is reused, not copied. Its only
change is additive: `Read-SessionEvents` now also returns tool time, API
time and session duration, and `Format-Count` is exported.
## Before / after
New scripts; nothing in `winapp` itself changes. Observed output of
`perf.ps1` on current `main` (x64 Xeon, NativeAOT build of `901e4cde`,
Calculator only; excerpt):
```text
> .\benchmarks\ui\perf.ps1 -Operation startup,list-windows,search,inspect-calculator,get-property,get-value,invoke,wait-for,focus,screenshot -Sequence calc-add,calc-two-instances,calc-minimized-scientific
startup winapp median 91 ms p95 93 ms errors 0
list-windows winapp median 102 ms p95 105 ms errors 0
search winapp median 259 ms p95 267 ms errors 0
get-value winapp median 144 ms p95 158 ms errors 0
invoke winapp median 177 ms p95 183 ms errors 0
focus winapp median 176 ms p95 182 ms errors 0
[1/3] calc-add | winapp ... pass | 1,914 ms | 10 commands
[1/3] calc-two-instances | winapp ... pass | 1,383 ms | 7 commands
[1/3] calc-minimized-scientific | winapp ... pass | 1,993 ms | 10 commands
```
Agent benchmark, first full run on 2026-10-09: branch `93b2cbb8` on main
`a87a8304`, ARM64 Snapdragon X2 Elite, Copilot CLI 1.0.93,
`claude-sonnet-5.5`, 5 iterations. **70/70 passed with no collateral
damage**, in 32 minutes and 497.5 credits.
| Method | Median wall time | Median turns | Credits (35 runs) |
|---|---|---|---|
| `winapp` | 19.4 s | 6 | 315.7 |
| `none` | 21.8 s | 5 | 181.8 |
`winapp` was 20–30% faster on the Gallery tasks, about even on simple
Calculator tasks, and slower on the minimized-window task (36.9 s vs
15.0 s). It used about 1.7× the credits, mostly to load the
UI-automation skill.
## Not tested / risks
- The agent numbers above predate several `main` changes and were not
re-run: minimized-window handling in `winapp ui` (#1054), the
`winapp-performance` skill (#837), and selector changes (#1102, #1104).
The `gallery-dialog-left-open` task also changed since then. It now asks
to turn the switch off, because the switch starts on. The
minimized-window result in particular is likely better now, because
`perf.ps1`'s `calc-minimized-scientific` sequence, which failed at its
first step on 10/9, passes 3/3 on current `main`.
- I did not re-run the Gallery perf sequences here, because the Gallery
is not installed on this machine.
- Neither benchmark runs in CI: both drive the desktop, and the agent
benchmark costs credits. Only the Pester suites are suitable for
automation.
## Checklist
- [x] Tests added or updated
- [x] Tested locally on Windows
<details><summary>Notes for reviewers</summary>
Validated on the merged head:
- `Invoke-Pester benchmarks\ui\tests`: 135/135 passed.
- `Invoke-Pester benchmarks\tasks\tests`: 54/54 passed.
- `Invoke-Pester benchmarks\agents\tests`: 157 passed, 2 failed. `main`
has the same 2 failures from the `winapp-performance` skill, and #1060
fixes them.
- `benchmarks\agents\run.ps1 -Lint` and `-Plan` give the same output as
`main`: 1 error, which is the same known issue, and 1,584 sessions.
- `benchmarks\ui\run.ps1 -Plan`, `run.ps1 -Lint` (0 errors) and
`perf.ps1 -Plan` all succeed.
</details>
---------
Co-authored-by: Nikola Metulev <711864+nmetulev@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Zach Teutsch <88554871+zateutsch@users.noreply.github.com>
Copilot-Session: 0cf8e7b3-7171-42da-9482-89167c7668b8
…/reference splits
capabilities.json: winapp-current now maps all 14 skills, with winui-devtools ->
winui.inspect (from #943) and winapp-performance -> winui.build (from this branch),
hash e20195ae2f87. Main's current map had no winapp-performance entry, so its
hash didn't match the shipped skill set. Earlier results stay scoreable through
three legacy maps: 13 skills without performance (main's former current map),
13 skills without devtools, and the original 12.
Benchmark.psm1: the command extractor keeps main's config and devtools groups and
this branch's perf group. README: winapp-current hash, and both mapping notes.
The #943 edits to winapp-ui-automation (description, when-to-use, DevTools
prerequisite and consent rule, screenshot note) all land in the core.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
)
## What and why
The agent benchmark is broken on main since `winapp-performance` (#837)
and `winui-devtools` (#943) shipped:
- `benchmarks\agents\run.ps1 -Lint` reports 1 error, and benchmark
Pester has 2 failures.
- No capability map matches the current 14 winapp skills, so runs with
the current plugin can't be scored.
- `winapp perf ...` commands aren't recognized by the command extractor.
This PR:
- Adds a current `winapp` map for all 14 skills. `winapp-performance`
maps to no capability for now: no scenario measures runtime performance,
and crediting it as `winui.build` would let it pass unrelated build and
crash scenarios. `winapp perf` commands are recorded but count toward
nothing.
- Keeps the previous maps so older results still score: 13 skills
without `winapp-performance`, 13 without `winui-devtools`, and the
original 12.
- **User-visible:** changes one word in the `winapp-performance` skill
description, "not crash dumps" → "not crash analysis". A held-out
benchmark prompt mentions a "crash dump", so the old wording leaked into
routing and failed lint. The meaning is the same.
## Before / after
**Before (main)**
```
> .\benchmarks\agents\run.ps1 -Lint
error demand c04-winui-hot-reload-toolbar-av leak-bigram shares 'crash dump' with skill winapp-performance
Linted 258 prompts: 1 errors, 54 warnings.
> Invoke-Pester benchmarks\agents\tests, benchmarks\tasks\tests, benchmarks\ui\tests
Passed 346 Failed 2
```
**After**
```
Linted 258 prompts: 0 errors, 54 warnings.
Passed 350 Failed 0
```
## Not tested / risks
No live agent runs. Validation covers lint, Pester, `-Plan`, and the
plugin validator.
## Checklist
- [x] Tests added or updated (they fail without the fix)
- [x] Tested locally on Windows
- [x] Docs, samples, and `plugins/winapp/skills/` updated if user-facing
behavior changed
<details><summary>Notes for reviewers</summary>
Commands run on this branch:
- `.\benchmarks\agents\run.ps1 -Lint`: 0 errors
- `Invoke-Pester -Path
.\benchmarks\agents\tests,.\benchmarks\tasks\tests,.\benchmarks\ui\tests`:
350 passed, 0 failed
- `.\benchmarks\agents\run.ps1 -Plan`: runs (1710 sessions)
- `.\scripts\validate-plugin-package.ps1`: passed
Skill-set hashes (computed on main's skill names): current 14 =
`e20195ae2f87`; without performance = `ea21288d7949` (main's former
current map); without devtools = `93d52ca8d877`; without both =
`4a4abc8b6541`.
Giving `winapp-performance` its own capability would need at least two
held-out scenarios (a test requires that for every capability), so it's
left for a follow-up. Ported from the benchmark part of draft #1060
only; none of its skill splits are included.
</details>
---------
Co-authored-by: Nikola Metulev <711864+nmetulev@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 72865eb6-685a-4ede-8516-17cfe8abcc86
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follows #1050, which has merged.
mainis merged in, including #837's newwinapp-performanceskill. Skill names and descriptions don't change, except one wording fix towinapp-performance(see below).What this does
The largest skills now keep a short core in
SKILL.mdand move detail into references that load only when needed. Every reference has a Load when row in its core. Consent and safety rules stay in the core.winapp-ui-automationinput.md), drag, touch, pen, and scrolling (gestures.md), video recording (capture.md), reading state, setting values, and file dialogs (dialogs-and-state.md), troubleshooting, JSON envelope highlights (ui-json-envelope.md)winapp-find-apiwinapp-setupnuget-feeds.md).winapp runproject mode and single-file.csapps moved towinapp-identityreferenceswinapp-identityrun-project-mode.mdandsingle-file-apps.md. Its production sparse packaging section moved toproduction-sparse.mdwinui-ui-testingwinui-code-reviewquality-rules.md(3.2k) is split into performance, security, accessibility, code-quality, and globalization references, so a review reads only the topic it needswinui-designbrushes-and-icons.mdis split intobrushes.mdandicons.mdTotals: unique core bodies drop from 65.5k to 49.5k tokens (−24%). The largest core is now 4.2k (was 9.4k). Always-loaded metadata doesn't change (WinApp 987, WinUI 594;
winapp-performancefrommainadds 65).Stale fixes
Get-AppxPackage … | Remove-AppxPackage.winapp-packagenow looks up the exact package by identity name and removes it only after the user confirms.scripts\test-samples.ps1section fromwinapp-maui. It now links to the MAUI sample on GitHub.winapp ui --jsonreference.winui-design,winui-code-review,winui-dev-workflow, andwinui-wpf-migrationnow say "load thewinui-packagingskill and read itssourcegen-patterns.mdreference". Before, a barereferences/…path didn't resolve from those skills. The C5 spike copied the file into four skills. This PR keeps one copy instead.&&". It now says to use;in PowerShell, which matcheswinapp-ui-automation.The spike's new content (Store reference, Fluent without porting, C++ WinUI guard, AutomationId authoring rule) is not in this PR.
Benchmark capability maps
No capability map changes. Every capability's content still sits in the core: commands, triggers, and the main workflow. References add detail to a capability, but no capability lives only in a reference. So skill-level credit stays correct, and adding file keys would only repeat the parent skill's capabilities. The existing
winapp-current/winui-currentmaps apply to both sides.Changes since the first review
Review fixes (Zach Teutsch (@zateutsch))
winapp-ui-automation: the core tip now sayswin+…shortcuts need--via send-input --allow-system-keys. Under the default--via post-messagetransport, keys go only to the target window, so shell hotkeys never fire and the flag only prints a warning (checked againstUiSendKeysCommand.cs).win+landctrl+alt+delstay blocked.winapp-setup: the core again warns, next toinitandrestore, that both commands use the target directory'snuget.configand should run only on directories you trust.winui-devagent, both copies: "Use;only for independent commands. For dependent steps, check$LASTEXITCODEbefore the next command."Merging
mainMain's newer skill facts are placed like the rest of the split:
winapp-ui-automationcore: queries by--type/--class-namewithout a selector, exact-name matching for single-element commands, clicking or hovering at a screen point (--at),--wheel-horizontalin the input and exclusive-command lists, and a pointer towinapp-performance.winapp-ui-automationreferences: thetouch --angleand horizontal-wheel examples are ingestures.md. Restoring a minimized window beforefocusis indialogs-and-state.md, with a one-line pointer in the core tips.winapp-setup: main's updatedwinapp runoutput line is inwinapp-identity'srun-project-mode.md, where project mode now lives.../winapp-performance/SKILL.md(inwinapp-ui-automationandwinapp-troubleshoot) now name the skill, so no cross-skill relative paths remain.winapp-performanceis 1,385 tokens, so it isn't split.Benchmark fix for #837
benchmarks/agents -Linthad 1 error and benchmark Pester had 2 failures onmain:winapp-performancemaps to the existingwinui.buildcapability ("build, run, and debug a WinUI 3 app"), because diagnosing a slow running app is debugging.winui.buildnow also lists^perfcommands and mentions slow layout or scrolling in its description. I didn't add a new capability, because that would also need at least two blind-written held-out scenarios. Thewinui.*prefix keeps loadingwinapp-performancein a non-WinUI scenario a failure, which matches its WinUI-only scope. A newwinapp-currentmap covers the 13 skills. The previous 12-skill map stays aswinapp-before-performance, so earlier results still score.perf, sowinapp perf …in an answer never counted. It now does, with tests for extraction and scoring.crash-dump-summary.txt. User-visible change: thewinapp-performancedescription now ends "…not crash analysis or UI correctness." (238 characters).Spot check of
winapp-ui-automationafter the mergemainvs this branch, the 10 dev and held-out prompts whose primary capability isui.automate, 3 models, 1 iteration, 768 credits:winapp-ui-automationloaded (core + references)The one routing flip is a Sonnet run that loaded no skill at all, which happens before any skill body loads.
Results
Both sides ran on the same harness, at #1050's head (
408dc75d), and the baseline plugins come from that commit. Since then,mainchanged onlywinapp-find-uiandwinapp-sandbox, and this PR doesn't touch either. Configurationboth, no agent, 3 models, 2 iterations. Differences are paired (same scenario, model, and iteration). 95% CIs are bootstrapped over scenarios.Outcomes and routing
By model:
Every CI includes zero. Pairs where neither side loaded a split skill have identical skill text, so they show the noise level: demand +0.004 [−0.019, +0.027], held-out routing −0.001. Pairs where a split skill loaded: demand +0.007 [−0.071, +0.083], held-out answer −0.006 [−0.050, +0.029].
Tokens
Credits per run are flat (held-out 10.5 → 10.3, demand 9.5 → 9.4), because Copilot's own prompt dominates input.
Reference reads
When a split skill loaded, the agent read one of its references in 34–52% of runs (GPT 34%, Opus 39%, Sonnet 52%). By skill:
winui-ui-testingbatch-template.md)winapp-identityproduction-sparse.md)winapp-ui-automationui-json-envelope.md,dialogs-and-state.md)winui-code-reviewwinui-designwindow-sizing.md)winapp-find-apiwinapp-setupCapabilities whose content partly moved
Held-out + demand, 36 runs per side.
ui.automateui.testapi.lookupproject.setupproject.scaffoldidentitywinui.reviewwinui.designsandbox.runAnswers hold for every capability. The
ui.test/ui.automaterouting dips come from 8 flipped pairs. In 5 of them, Sonnet loaded no skill at all on one side. In the other 3, a different second skill was chosen after the first skill loaded, and one of those favored the split. Routing happens before the skill body loads, and descriptions didn't change, so these flips aren't caused by content moving into a reference. No content was pulled back into a core.Ship rule
Outcomes and routing hold within noise, and loaded skill text drops. This passes.
Credits
Spot check after the merge: 768. Original measurement: 22,912 of the 23,000 cap: runs 20,969 (2,082 runs) and two-judge grading 1,943. Opus held-out iteration 2 stopped at its credit cap with 14 runs left. I finished 8 of them, enough to complete their pairs within the cap. The 3
wrenfall-category-browser-exampleprompts have no Opus iteration-2 pair; their iteration-1 pairs still count.Validation
scripts\validate-plugin-package.ps1: 0 errorsscripts\validate-llm-docs.ps1against a Debug CLI: command examples match the schemarun.ps1 -Lint: 0 errors, 54 warnings (same as before feat: add WinUI ETW performance recording and analysis #837)validate-plugin-package.ps1 -CliSchemaPathwith a fresh Debug CLI from this branch, andvalidate-llm-docs.ps1: passTest-WinuiUiTestingSandbox.ps1andTest-SetupVersionDetection.ps1: passscripts\tests\Test-WinuiUiTestingSandbox.ps1now checks the skill's core and references together, because the batch template moved into a reference. It passes.Please check by hand
winapp-ui-automation: the core still reads well for a first-timewinapp uiuser. The new focus and--allow-system-keyslines in Tips restate rules that moved to references.winui-ui-testing: Step 2 now points toreferences/batch-template.md. The core also restates the Sandbox shutdown consent rule.winapp-package: the uninstall example (exact identity name, user confirmation) replaces the wildcard.winapp-identitynow ownswinapp runproject mode and.csapps, andwinapp-setuppoints there. Is that the split you want?Coverage checklist: where every section of the changed skills went
Generated by matching each section's lines in
main(4b681dd1, the unsplit skills with all of main's newer facts) against this branch. Sections marked "(edited)" changed some lines: stale fixes, link or pointer fixes, or a self-reference.winapp-find-api/SKILL.md
winapp-find-api/SKILL.mdwinapp-find-api/SKILL.mdwinapp-find-api/SKILL.mdwinapp-find-api/SKILL.mdwinapp-find-api/SKILL.md--filteron big member lists — not on enumswinapp-find-api/SKILL.mdwinapp-find-api/SKILL.mdwinapp-find-api/SKILL.mdwinapp-find-api/SKILL.mdwinapp-find-api/SKILL.mdwinapp-find-api/SKILL.mdwinapp-find-api/references/more-patterns.mdwinapp-find-api/references/more-patterns.mdwinapp-find-api/references/more-patterns.mdwinapp-find-api/references/more-patterns.mdwinapp-find-api/references/more-patterns.mdwinapp-find-api/references/concepts.mdwinapp-find-api/references/scopes-and-troubleshooting.mdwinapp-find-api/references/more-patterns.mdwinapp-identity/SKILL.md
winapp-identity/SKILL.mdwinapp-identity/SKILL.mdwinapp-identity/SKILL.mdwinapp-identity/SKILL.mdwinapp-identity/SKILL.mdwinapp-identity/SKILL.mdwinapp-identity/SKILL.mdwinapp-identity/SKILL.mdwinapp-identity/SKILL.mdwinapp-identity/SKILL.mdwinapp-identity/SKILL.mdwinapp runvscreate-debug-identitywinapp-identity/SKILL.mdwinapp-identity/SKILL.mdwinapp-identity/SKILL.mdinit --sparse/pack/embed-identity)winapp-identity/references/production-sparse.mdwinapp-identity/SKILL.mdwinapp-identity/SKILL.mdwinapp-maui/SKILL.md
winapp-maui/SKILL.mdwinapp-maui/SKILL.mdwinapp-maui/SKILL.mdwinapp-maui/SKILL.mdwinapp-maui/SKILL.mdwinapp-maui/SKILL.md--manifestat the resolved manifestwinapp-maui/SKILL.mdwinapp-maui/SKILL.mdwinapp-maui/SKILL.mdwinapp-maui/SKILL.mdwinapp-maui/SKILL.mdwinapp-maui/SKILL.mdwinapp-package/SKILL.md
winapp-package/SKILL.mdwinapp-package/SKILL.mdwinapp-package/SKILL.mdwinapp-package/SKILL.mdwinapp-package/SKILL.mdwinapp-package/SKILL.mdwinapp-package/SKILL.mdwinapp-package/SKILL.mdwinapp-package/SKILL.mdwinapp-package/SKILL.mdwinapp-package/SKILL.mdwinapp-package/SKILL.md(edited)winapp-package/SKILL.mdwinapp-package/SKILL.mdwinapp-package/SKILL.mdwinapp-package/SKILL.mdwinapp-package/SKILL.mdwinapp-package/SKILL.mdwinapp-package/SKILL.mdwinapp-package/SKILL.mdwinapp-identity/SKILL.mdwinapp-setup/SKILL.md
winapp-setup/SKILL.mdwinapp-setup/SKILL.mdwinapp-setup/SKILL.mdwinapp-setup/SKILL.mdwinapp-setup/SKILL.mdwinapp-setup/SKILL.mdwinapp-identity/references/production-sparse.mdwinapp-setup/SKILL.mdwinapp-setup/references/nuget-feeds.mdwinapp-setup/SKILL.mdwinapp-identity/references/run-project-mode.mdwinapp runon a.csproj(.NET / WinUI)winapp-identity/references/run-project-mode.md(edited)winapp runon a C++.vcxprojwinapp-identity/references/run-project-mode.mdwinapp runon a.csfile-based appwinapp-identity/references/single-file-apps.mdrunandcreate-debug-identitywinapp-identity/references/run-project-mode.mdwinapp-setup/SKILL.mdwinapp-setup/SKILL.mdwinapp-setup/SKILL.mdwinapp-identity/SKILL.mdwinapp-ui-automation/references/ui-json-envelope.md
winapp ui --jsonenvelopeswinapp-ui-automation/references/ui-json-envelope.md(edited)ui inspect --jsonwinapp-ui-automation/references/ui-json-envelope.md(edited)ui inspect --ancestors --jsonwinapp-ui-automation/references/ui-json-envelope.mdui inspect --interactivewinapp-ui-automation/references/ui-json-envelope.mdui get-focused --jsonwinapp-ui-automation/references/ui-json-envelope.md(edited)ui search --jsonwinapp-ui-automation/references/ui-json-envelope.mdui wait-for --jsonwinapp-ui-automation/references/ui-json-envelope.mdui get-property --jsonwinapp-ui-automation/references/ui-json-envelope.mdwinapp-ui-automation/references/ui-json-envelope.mdui status --jsonwinapp-ui-automation/references/ui-json-envelope.mdui invoke --jsonwinapp-ui-automation/references/ui-json-envelope.mdwinapp-ui-automation/references/ui-json-envelope.mdwinapp-ui-automation/references/ui-json-envelope.mdwinapp-ui-automation/SKILL.md
winapp-ui-automation/SKILL.mdwinapp-ui-automation/SKILL.mdwinapp-ui-automation/SKILL.md(edited)winapp-ui-automation/SKILL.mdwinapp-ui-automation/SKILL.mdwinapp-ui-automation/SKILL.mdwinapp-ui-automation/SKILL.mdwinapp-ui-automation/SKILL.mdwinapp-ui-automation/SKILL.mdwinapp-ui-automation/SKILL.mdwinapp-ui-automation/SKILL.mdwinapp-ui-automation/SKILL.mdwinapp-ui-automation/SKILL.mdwinapp-ui-automation/SKILL.mdwinapp-ui-automation/SKILL.mdwinapp-ui-automation/references/capture.mdwinapp-ui-automation/references/input.mdwinapp-ui-automation/SKILL.mdwinapp-ui-automation/references/input.mdwinapp-ui-automation/references/gestures.mdwinapp-ui-automation/references/gestures.mdwinapp-ui-automation/references/gestures.mdwinapp-ui-automation/references/dialogs-and-state.mdwinapp-ui-automation/references/dialogs-and-state.mdwinapp-ui-automation/references/gestures.mdwinapp-ui-automation/SKILL.mdwinapp-ui-automation/SKILL.md(edited);instead of&&winapp-ui-automation/SKILL.mdwinapp-ui-automation/references/dialogs-and-state.mdwinapp-ui-automation/references/ui-json-envelope.md(edited)winapp-ui-automation/SKILL.mdwinapp-ui-automation/references/troubleshooting.md(edited)winapp-identity/SKILL.mdwinui-code-review/references/quality-rules.md
winui-code-review/references/performance.mdwinui-code-review/references/performance.mdwinui-code-review/references/performance.mdwinui-code-review/references/performance.mdwinui-code-review/references/performance.mdwinui-code-review/references/performance.mdwinui-code-review/references/performance.mdwinui-code-review/references/security.mdwinui-code-review/references/security.mdwinui-code-review/references/security.mdwinui-code-review/references/security.mdwinui-code-review/references/security.mdwinui-code-review/references/security.mdwinui-code-review/references/accessibility.mdwinui-code-review/references/accessibility.mdwinui-code-review/references/accessibility.mdwinui-code-review/references/accessibility.mdwinui-code-review/references/accessibility.mdwinui-code-review/references/code-quality.mdwinui-code-review/references/code-quality.mdwinui-code-review/references/code-quality.mdwinui-code-review/references/code-quality.mdwinui-code-review/references/code-quality.mdwinui-code-review/references/globalization.mdwinui-code-review/references/globalization.mdwinui-code-review/references/globalization.mdwinui-code-review/references/globalization.mdwinui-code-review/references/globalization.mdwinui-code-review/references/globalization.mdwinui-code-review/references/globalization.mdwinui-code-review/references/globalization.mdwinui-code-review/SKILL.md
winui-code-review/SKILL.mdwinui-code-review/SKILL.mdwinui-code-review/SKILL.mdwinui-code-review/SKILL.mdwinui-code-review/SKILL.mdwinui-code-review/SKILL.md(edited)winui-code-review/SKILL.mdwinui-code-review/SKILL.mdwinui-code-review/SKILL.mdwinui-code-review/SKILL.mdwinui-code-review/SKILL.mdwinui-code-review/SKILL.mdwinui-design/references/brushes-and-icons.md
winui-design/references/brushes.md(edited)winui-design/references/brushes.mdwinui-design/references/brushes.mdwinui-design/references/brushes.mdwinui-design/references/brushes.mdwinui-design/references/brushes.mdwinui-design/references/brushes.mdwinui-design/references/brushes.mdwinui-design/references/brushes.mdwinui-design/references/icons.mdwinui-design/references/icons.mdwinui-design/references/icons.mdwinui-design/references/icons.mdwinui-design/references/icons.mdwinui-design/references/layout-review.md
winui-design/references/layout-review.mdwinui-design/references/layout-review.mdwinui-design/references/layout-review.mdwinui-design/references/layout-review.mdwinui-design/references/theme-accessibility.md
winui-design/references/theme-accessibility.mdwinui-design/references/theme-accessibility.mdwinui-design/references/theme-accessibility.mdwinui-design/references/theme-accessibility.md(edited)winui-design/references/theme-accessibility.mdBasedOndisciplinewinui-design/references/theme-accessibility.mdwinui-design/SKILL.md
winui-design/SKILL.mdwinui-design/SKILL.mdwinui-design/SKILL.mdwinui-design/SKILL.mdwinui-design/SKILL.mdwinui-design/references/window-sizing.mdx:Bindmodewinui-design/SKILL.md(edited)TextBoxtwo-way needsUpdateSourceTrigger=PropertyChangedwinui-design/SKILL.mdwinui-design/SKILL.mdConverter={x:Null}crashesx:Bindat runtimewinui-design/SKILL.mdx:Bindstatic functions overIValueConverterwinui-design/SKILL.mdThemeShadowrendering ruleswinui-design/SKILL.mdwinui-design/SKILL.mdwinui-design/SKILL.md(edited)winui-ui-testing/SKILL.md
winui-ui-testing/SKILL.mdwinui-ui-testing/SKILL.mdwinui-ui-testing/SKILL.mdwinui-ui-testing/SKILL.mdwinui-ui-testing/references/batch-template.md(edited)winui-ui-testing/SKILL.mdwinui-ui-testing/SKILL.mdwinui-ui-testing/references/advanced-interactions.mdwinui-ui-testing/references/advanced-interactions.mdwinui-ui-testing/references/advanced-interactions.mdwinui-ui-testing/references/sandbox-persistence.mdwinui-ui-testing/SKILL.md