Skip to content

[cli-consistency] CLI Consistency Issues - 2026-09-08 #59468

Closed as not planned
Closed as not planned

Description

@github-actions

Summary

Automated inspection of the gh-aw CLI (all subcommand --help output, pre-collected in all-help.txt/help-output/*.txt) compared against docs/src/content/docs/setup/cli.md. Flag names, short/long pairings, and --no-* negation patterns are largely consistent across commands. Two concrete documentation gaps were found where real CLI flags/subcommands are completely absent from the docs.

Severity breakdown: 0 high · 2 medium · 0 low

Findings

[Medium] `gh aw logs` — two documented CLI flags missing from docs/src/content/docs/setup/cli.md

Affected command: gh aw logs

Exact CLI output (from gh aw logs --help):

      --max-github-api-rate-limit int   Maximum used GitHub core API requests before waiting for reset (positive = absolute, negative = reserve from API limit; e.g. 12000 or -2000)
      --max-storage int                 Maximum logs storage in MB after pruning non-essential cache data (0 = unlimited)

The help text also includes matching usage examples:

  gh aw logs --timeout 30 --max-github-api-rate-limit 12000 # Wait after 12000 core API requests are used
  gh aw logs --timeout 30 --max-github-api-rate-limit -2000 # Keep 2000 core API requests available
  gh aw logs --timeout 30 --max-storage 10240                # Prune cache data and stop downloads at 10 GB

Expected: These two flags (and the "Resource budgets" example block) should appear in the #### \logs`section ofdocs/src/content/docs/setup/cli.md`, at minimum in the Options: flag list (line ~504):

**Options:** `--after-run-id`, `--artifacts`, ..., `--last`, `--no-firewall`, ...

Actual: Neither --max-github-api-rate-limit nor --max-storage appears anywhere in cli.md (grep -c returns 0 hits for both strings in the whole file).

Suggested fix: Add both flags to the **Options:** list for logs, and add a short "Resource budgets" example / one-line explanation (mirroring the --cache-before treatment already present) describing how --max-github-api-rate-limit and --max-storage bound API/storage usage during long download runs.

Priority: Medium — these are real, user-facing flags for a commonly used command (logs) with no discoverability outside --help output.

[Medium] `gh aw graders operational-value report` — subcommand entirely undocumented

Affected command: gh aw graders operational-value report

Exact CLI output (from gh aw graders operational-value --help):

Available Commands:
  report Build a complete operational-value history report

Expected: The #### \graders`section indocs/src/content/docs/setup/cli.md(~line 574) documentsgraders runandgraders operational-value with an example and--evidence-atflag, but omits thereport` subcommand entirely — no usage example, no flag list, no mention that it exists.

Actual: Searching cli.md for "report" near the graders section yields no matches; the subcommand is invisible to a reader relying on the docs.

Suggested fix: Add a ##### \graders operational-value report`subsection (or at least one example line + an **Options:** list) describing what "Build a complete operational-value history report" produces and its flags (rungh aw graders operational-value report --helpto confirm the exact flag set, e.g.--repo/-r, --json/-j`, and any output/format flags).

Priority: Medium — a fully hidden subcommand reduces discoverability of an existing grading/reporting feature.

Inspection metadata

  • Commands inspected: all top-level and nested subcommands captured in the pre-collected help dump — add, add-wizard, audit, checks, compile, completion, deploy, disable, doctor, domains, edit, enable, env (get/update), experiments, fix, forecast, gh, graders (run/operational-value), hash-frontmatter, health, help, init, lint, list, logs, mcp (list/list-tools/inspect/add), mcp-server, models, new, outcomes (history), pr (transfer), project (new), remove, run, secrets (set/bootstrap), status, trial, update, upgrade, validate, version — 467 individual help snapshot files plus the combined all-help.txt (25,784 lines).
  • Date: 2026-09-08
  • Method: Read every --help output for each command/subcommand from the pre-collected dump, cross-referenced flag names, short/long flag pairings, --no-* negation naming, and example usage against the corresponding section of docs/src/content/docs/setup/cli.md. Checked for typos, grammar issues, and terminology drift; none of significance were found — help text is generally well-maintained and consistent (e.g. --dry-run, --force/-f, --json/-j, --repo/-r patterns are uniform across all commands that support them).
  • No other high/low severity issues identified. Flag capitalization, GitHub spelling, --no-X naming conventions, and cross-command flag consistency (e.g. --dir/-d, --engine/-e) were all verified consistent.

Generated by ✅ CLI Consistency Checker · copilot · auto · 121 AIC · ⌖ 5.42 AIC · ⊞ 7.1K · ◷

  • expires on Sep 10, 2026, 5:35 AM UTC-08:00

Activity

  1. github-actions commented on Sep 10, 2026

    @github-actions
    ContributorAuthor

    This issue was automatically closed because it expired on 2026-09-10T13:35:41.480Z.

    Closed by Workflow

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    automationclicookieIssue Monster Loves Cookies!documentationImprovements or additions to documentation

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions