Repository navigation
Align CLI reference with command defaults and side effects - #76575
Conversation
Co-authored-by: gh-aw-bot <259018956+gh-aw-bot@users.noreply.github.com>
|
🧠 Matt Pocock Skills Reviewer has completed the skills-based review. ✅
|
|
✅ PR Code Quality Reviewer completed the code quality review. Caution agentic threat detected DetailsPotential security threats were detected in the agent output. Detected threat types: prompt injection. Review the workflow run logs for details. PR Code Quality ReviewerOutcome: The agent completed and reported one or more actions. The no-op output is a completion signal, not a claim that no action was taken. Agent messages are omitted from this comment. The category uses structured run status, not interpretation of agent prose.
|
|
✅ PR Specification Reviewer completed successfully! PR Specification ReviewerOutcome: The agent completed and reported no action. A no-op output alone does not verify that all required work was completed or that publication was possible. Agent messages are omitted from this comment. The category uses structured run status, not interpretation of agent prose.
|
|
✅ Test Quality Sentinel completed test quality analysis. Test Quality SentinelOutcome: The agent completed and reported no action. A no-op output alone does not verify that all required work was completed or that publication was possible. Agent messages are omitted from this comment. The category uses structured run status, not interpretation of agent prose.
|
|
✅ Ponytail Reviewer completed successfully! Ponytail ReviewerOutcome: The agent completed and reported no action. A no-op output alone does not verify that all required work was completed or that publication was possible. Agent messages are omitted from this comment. The category uses structured run status, not interpretation of agent prose.
|
|
✅ Design Decision Gate 🏗️ completed the design decision gate check. See the comment below for the result and any generated ADR draft. Design Decision Gate 🏗️Outcome: The agent completed and reported no action. A no-op output alone does not verify that all required work was completed or that publication was possible. Agent messages are omitted from this comment. The category uses structured run status, not interpretation of agent prose.
|
There was a problem hiding this comment.
🟡 Changes recommended
The documentation understates the files that fix --write can delete, omitting the .github/aw/ path and broader .github/agents/ scope.
1 open issue
What changed in this PR
This PR aligns the CLI reference with current command defaults and side effects while adding consistency checks to prevent documentation drift.
Changes:
- Documents
fix --writefile updates and cleanup operations, trial repository retention, and local versus remotelistbehavior. - Clarifies compact default
logsartifacts and adds agent/firewall artifact examples. - Adds focused help/reference consistency assertions for the four command sections.
| File | Description |
|---|---|
| pkg/cli/cli_consistency_help_test.go | Adds assertions that CLI help and reference documentation remain aligned for the fix, trial, list, and logs sections. |
| docs/src/content/docs/setup/cli.md | Updates CLI reference documentation for command side effects, repository retention, listing modes, and artifact selection. |
Review effort: Lite (auto)
| workflow-specific `.agent.md` files from `.github/agents/`. It also refreshes | ||
| `.github/skills/agentic-workflows/SKILL.md` and `.github/agents/agentic-workflows.md` |
|
Caution agentic threat detected DetailsPotential security threats were detected in the agent output. Detected threat types: prompt injection. Review the workflow run logs for details. Comment MemoryPeek at saved memory (pr-code-quality-reviewer)Note This comment is managed by comment memory.Expand the saved memory block to view or edit the persistent context for this thread.
|
There was a problem hiding this comment.
Caution
agentic threat detected
Threat detection flagged this output in warn mode. Manual review is REQUIRED before any follow-up automation.
Details
Potential security threats were detected in the agent output. Detected threat types: prompt injection.
Review the workflow run logs for details.
Found four actionable issues in the documentation and consistency test:
fix --writedocumentation omits legacy cleanup paths and its skip condition.--delete-host-repo-aftercan delete reused--host-reporepositories, but the docs imply only newly created hosts are affected.- The documented list path default ignores
GH_AW_WORKFLOWS_DIR. - New flag assertions can panic on a missing flag instead of reporting a test failure.
Review was static; no tests were run per review instructions.
🔎 Code quality review by PR Code Quality Reviewer · copilot · gpt50 · ~5 AIC · ⌖ ~2 AIC · ⊞ 1.6K
Comment /review to run again
| trialDocs := section("trial", "run") | ||
| trialHelp := NewTrialCommand(func(string) error { return nil }) | ||
| assert.Contains(t, trialHelp.Long, "retained by default") | ||
| assert.Contains(t, trialHelp.Flags().Lookup("delete-host-repo-after").Usage, "retained by default") |
There was a problem hiding this comment.
Flags().Lookup(...) can return nil, so dereferencing .Usage makes a renamed or missing flag panic instead of producing a useful assertion failure. Please assert the lookup is non-nil before reading Usage (the same issue occurs for stale and path below).
There was a problem hiding this comment.
Added nil checks before reading flag usage and asserted the trial, list, and logs flag defaults. Commit: 3a9f9ef7.
| `list` enumerates workflows locally without GitHub API queries. For enabled/disabled state and run information, use `status` instead. | ||
| Without `--repo`, `list` enumerates workflows from the local checkout without GitHub API | ||
| queries. With `--repo`, it lists workflows from that remote repository; `--path` selects | ||
| the remote workflows directory (default: `.github/workflows`). The `--stale` filter is |
There was a problem hiding this comment.
The --path default is constants.GetWorkflowDir(), which honors GH_AW_WORKFLOWS_DIR; .github/workflows is only the fallback. Please mention $GH_AW_WORKFLOWS_DIR here, otherwise this reference is inaccurate for configured repositories.
There was a problem hiding this comment.
Updated the remote --path default to document $GH_AW_WORKFLOWS_DIR, falling back to .github/workflows, and pinned the default in the consistency test. Commit: 3a9f9ef7.
|
|
||
| `--timeout` sets the execution timeout in integer minutes (default: `30`; `0` disables the timeout). For example, use `--timeout 10`, not `--timeout 10m`. | ||
|
|
||
| By default, a newly created private host repository is retained after the trial. Specify |
There was a problem hiding this comment.
This wording understates the destructive behavior of --delete-host-repo-after: the cleanup path is selected solely from the flag, so it also deletes a pre-existing repository supplied via --host-repo, not only a newly created trial repository. Please document that explicitly or change the cleanup logic to protect reused repositories (with a regression test).
There was a problem hiding this comment.
Clarified that --delete-host-repo-after deletes the host repository even when an existing repository was supplied via --host-repo. Commit: 3a9f9ef7.
|
|
||
| **Options:** `--dir/-d`, `--disable-codemod`, `--list-codemods`, `--write` | ||
|
|
||
| Without `--write`, no files are modified. With `--write`, `fix` writes the updated |
There was a problem hiding this comment.
The documented cleanup is incomplete and can mislead users about --write. The implementation also removes several legacy files under .github/agents/ and the singular .github/aw/upgrade-agentic-workflow.md, and these deletions are skipped when instruction-file updates are disabled. Please document the actual scope (or link to the command help), including the --skip-instruction-file-updates interaction.
There was a problem hiding this comment.
Expanded the help and reference to cover legacy .agent.md and .md files under .github/agents/ plus .github/aw/upgrade-agentic-workflow.md, and noted that cleanup/refresh are skipped when instruction-file updates are disabled. gh aw fix does not expose a --skip-instruction-file-updates flag, so the docs describe the behavior without advertising a nonexistent CLI flag. Commit: 3a9f9ef7.
There was a problem hiding this comment.
Skills-Based Review 🧠
Applied /grill-with-docs to the docs and /tdd to the new consistency test. Requesting changes on one incorrect example.
📋 Key Themes & Highlights
Key Themes
- Incorrect artifact example:
--artifactsreplaces its default instead of adding to it, sologs --artifacts agentdrops the usage artifact. The new docs line "Include detailed agent data" says otherwise. - Guard checks prose, not behaviour: the new test asserts substrings only. It does not pin flag defaults, which the PR title and test name promise.
Verified
- ✅
TestCLIDocsDocumentCommandSideEffectsAndDefaultspasses on the PR head. - ✅ Red check: reverting the docs hunks makes the new test fail.
- ✅
fix --writedeletion targets, thetrialretention default,list --stale(local only) andlist --pathmatch the code. - ✅
logsdefault ofusagematches the flag default.
Notes
- The
pr-triageagent was not invoked. Classification was done directly from the diff.
@copilot please address the review comments above.
🧠 Reviewed using Matt Pocock's skills by Matt Pocock Skills Reviewer · copilot · routed: haiku55 xhigh · ~4 AIC · ⌖ ~2 AIC · ⊞ 6.6K
Comment /matt to run again
|
|
||
| ```bash wrap | ||
| gh aw logs workflow # Download logs for workflow | ||
| gh aw logs workflow --artifacts agent # Include detailed agent data |
There was a problem hiding this comment.
[/grill-with-docs] --artifacts replaces its default rather than adding to it, so --artifacts agent downloads only agent data and drops the usage artifact. "Include detailed agent data" suggests the opposite. The command's own help already says "Download only agent logs" for this form.
💡 Suggested wording
gh aw logs workflow --artifacts agent # Download only detailed agent data
gh aw logs workflow --artifacts usage,agent # Keep usage data and add agent dataThis matches the flag's replace-the-default behavior and the existing help text.
@copilot please address this.
There was a problem hiding this comment.
Clarified that --artifacts replaces the default set: the agent and firewall examples download only those sets, and usage,agent preserves compact usage alongside agent data. Commit: 3a9f9ef7.
| assert.Equal(t, 2, strings.Count(text, "`--repos`, `--interactive/-i`, `--dry-run`")) | ||
| } | ||
|
|
||
| func TestCLIDocsDocumentCommandSideEffectsAndDefaults(t *testing.T) { |
There was a problem hiding this comment.
[/tdd] The test name promises "Defaults", but nothing asserts a flag default. The substring checks pin prose, so they still pass when the docs are wrong. The --artifacts agent example in cli.md is one case this test misses.
💡 Suggested assertions
assert.Equal(t, "[usage]", NewLogsCommand().Flags().Lookup("artifacts").DefValue)
assert.Equal(t, "false", NewTrialCommand(func(string) error { return nil }).Flags().Lookup("delete-host-repo-after").DefValue)
assert.Equal(t, ".github/workflows", NewListCommand().Flags().Lookup("path").DefValue)Checking DefValue pins the defaults the docs describe, so they cannot drift silently.
@copilot please address this.
There was a problem hiding this comment.
Added explicit default assertions for artifacts, trial cleanup, and list path, with nil checks before using flag metadata. Commit: 3a9f9ef7.
|
🍳 A little prep to help this PR along! Copilot assignment requested for the following outstanding work:
Keeping the changes focused; human review and CI get the final taste test. No merging or CI triggering. Structured data{
"completed_task": 0,
"head": "61e59fdeb0520689fb93100e05af0dc5b86c1dbf",
"kind": "requested",
"schema_version": "1",
"state": "8079f229407c4dc1b94be90f953eb609789a8bf74a8cc7f55882364d2afae20c",
"work": [
"098765d6bf5fc594b3de513c852ae9cdabf9f84a08770ab2a12fcfd2c505d7e0",
"21b73c503b6f269265cc9a0246d0ba5e76327382e43e1d5f6f3f1de41297f318",
"5a32863c94d237c6cf6034e8e5a4f71c6c501f524cb85112132f0c7428ff2151",
"8e6618d7e23e66ffeaea5c2cb9de8d790d01d04b7af32577b207558ae7e5ba4e",
"9dd36a2c47469e2a3cfcaf70990283249b3921109c0440d69632a86bfb85398a",
"c5db844f3700ef7443e236d5cb6861e205f6163935c08b8d9322476c35963b9a"
],
"workflow": "pr-sous-chef"
}
|
…cy-fix-sides-effects Co-authored-by: gh-aw-bot <259018956+gh-aw-bot@users.noreply.github.com>
Co-authored-by: gh-aw-bot <259018956+gh-aw-bot@users.noreply.github.com>
Co-authored-by: gh-aw-bot <259018956+gh-aw-bot@users.noreply.github.com>
|
🎉 This pull request is included in a new release. Release: |

The CLI reference omitted
fix --writefile changes, left trial repository retention unclear, and blurred local and remotelistbehavior. Its introductorylogsexample also implied detailed artifacts were downloaded by default.fix --writemay update or delete, clarify that trial host repositories are retained unless--delete-host-repo-afteris specified, and distinguish local and remotelistmodes, including the local-only--stalefilter.logsdefaults to compact usage data and show how to request agent or firewall data: