Skip to content

[aw-compat] Daily AW cross-repo compatibility (2026-05-15): 3/20 still failing after fix — 3 codemod candidates #32326

Closed

Description

@github-actions

Summary

Daily cross-repo gh-aw compatibility sweep ran against the top 20 public repositories (by stars) found via GitHub code search for .lock.yml files containing gh-aw-metadata. Each repo's .md workflows were compiled against the latest local gh-aw build, then gh aw fix --write was applied and the workflows were recompiled.

  • gh-aw build under test: gh aw version 27f07dd
  • Repositories audited: 20
  • Clean compile, no fix needed: 12
  • Clean compile only after gh aw fix --write applied codemods: 4
  • Still failing after gh aw fix --write: 3
  • No .md source workflows present (only lock.yml committed): 1 (dotnet/dotnet)

Critical findings (failures that survived gh aw fix --write)

Three repositories still failed compile after codemods. The error shapes are deterministic and look like missing-codemod opportunities or user-facing pitfalls worth surfacing.

Repository ★ Root cause Suggested codemod
microsoft/aspire 5,937 Compiler regression-check fires: emitted lock.yml still contains ${{ github.token }} and ${{ env.* }} inside run: blocks (source workflow inlines them). auto-hoist-run-expressions — rewrite inline ${{ ... }} in run: text to per-step env: bindings with shell variable references.
PunGrumpy/logixlysia 242 engine.version: latest rejected by strict mode (supply chain risk). 2 workflows affected. pin-engine-version — replace version: latest with a pinned semver/tag from the engine catalog, or remove the line.
plengauer/Thoth 159 tools.github.toolsets: [..., issues] referenced without the matching permissions.issues: read. auto-add-toolset-permissions — when a toolset that maps to a required permission is referenced, ensure the matching permissions.<x>: read is present.
Detailed error output per failing repository

microsoft/aspire — milestone-changelog.md

error: compiler regression detected: GitHub Actions expressions found in run: shell scripts of compiled workflow

The compiler must rewrite expressions in run: blocks to env variables.
Use env: assignments and shell variable references instead of inline ${{ ... }} in run: scripts.

Examples found:
  - ${{ github.token }}
    in: echo "GH_TOKEN=${{ github.token }}" >> "$GITHUB_OUTPUT"

The source workflow legitimately uses inline ${{ github.token }}, ${{ github.repository }}, ${{ env.MILESTONE }} etc. inside run: scripts. The compiler self-check fires because the codemod that should hoist these never ran. The user's workflow is correct idiomatic Actions YAML.

PunGrumpy/logixlysia — assign-pr-reviewers.md, weekly-find-critical-bugs.md

strict mode: engine.version: latest is set – the engine CLI will be installed without a pinned version.
This is a supply chain security risk: unpinned 'latest' versions can change unexpectedly
and may introduce vulnerabilities or breaking changes.
Pin the engine version to a specific version for reproducibility and security.

Source uses:

engine:
  id: copilot
  version: latest

A codemod could replace this with the current pinned default.

plengauer/Thoth — autofix.md

error: Missing required permissions for GitHub toolsets:
  - issues: read (required by issues)

Source declares:

permissions:
  contents: read
  actions: read
tools:
  github:
    toolsets: [context, actions, issues]

The toolset → required-permission mapping is already known to the compiler (it produces the diagnostic). The fix is mechanical.

What gh aw fix --write recovered

The existing codemod registry is working — these repos compiled cleanly only because gh aw fix --write applied them:

Repository ★ Codemods that fired
clash-verge-rev/clash-verge-rev 118,247 Add checkout: false for pull_request_target
storybookjs/storybook 89,912 Migrate tools.serena to shared Serena import
Azure/azure-sdk-for-js 2,292 Add checkout: false for pull_request_target (6 files)
drasi-project/drasi-core 109 Transform activation outputs, convert expires int→day string, rename github-app.app-id → github-app.client-id, add checkout: false (9 files)
Full per-repository result table (all 20)
Repository ★ md compile-before fix changed compile-after
f/prompts.chat 162266 1 ✅ — ✅
clash-verge-rev/clash-verge-rev 118247 1 ❌ yes ✅
storybookjs/storybook 89912 4 ❌ yes (3) ✅
dotnet/aspnetcore 37933 6 ✅ — ✅
openvinotoolkit/openvino 10248 1 ✅ — ✅
Azure/azure-sdk-for-net 5986 2 ✅ — ✅
microsoft/aspire 5937 7 ❌ no ❌
Azure/azure-sdk-for-js 2292 10 ❌ yes (6) ✅
dotnet/skills 1705 7 ✅ — ✅
dotnet/dotnet 1116 0 n/a — no .md sources — n/a
dotnet/SqlClient 974 1 ✅ — ✅
rancher/dashboard 595 7 ✅ — ✅
globalbao/awesome-azure-policy 538 1 ✅ — ✅
fsprojects/FsHttp 498 1 ✅ — ✅
dotnet/docs-maui 274 1 ✅ — ✅
PunGrumpy/logixlysia 242 2 ❌ no ❌
plengauer/Thoth 159 4 ❌ no ❌
Azure/LogicAppsUX 111 3 ✅ — ✅
drasi-project/drasi-core 109 10 ❌ yes (9) ✅
microsoft/vscode-isort 107 3 ✅ — ✅

Recommended next steps

  1. Prioritize auto-hoist-run-expressions — the compiler already self-detects this case and refuses to emit, but cannot recover. Adding the codemod would unblock workflows that mirror idiomatic Actions YAML.
  2. Add a pin-engine-version codemod (or auto-warn) so first-time users who pick version: latest don't hit a hard fail in strict mode.
  3. Add auto-add-toolset-permissions — the toolset → permission map is already authoritative inside the compiler; this is purely synthesis work on top of existing metadata.
  4. Investigate why dotnet/dotnet has committed .lock.yml artifacts but no .md sources (orphaned files? sub-tree mirror?). Not a gh-aw bug per se, but worth a quick triage.

Discovery method

  • gh api search/code?q=gh-aw-metadata+filename:.lock.yml (4 pages, 200 results, 131 unique repos)
  • Star counts via gh api repos/<owner>/<name>
  • Top 20 selected by stargazers_count desc, public + non-archived + excluding github/gh-aw

References:

Generated by 🔧 Daily AW Cross-Repo Compile Check · ● 19.3M · ◷

  • expires on May 22, 2026, 10:03 AM UTC

Metadata

Metadata

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions