Skip to content

[agentic-token-optimizer] Reduce Avenger AIC: skip duplicate CI fixes and cap guidance-test loops #76889

Closed as not planned

Description

@github-actions

Target workflow

Avenger (.github/workflows/avenger.md, hourly CI fixer). Selected as the highest-AIC workflow not optimized in the last 14 days: 96.2 AIC across 3 runs. Dead Code Removal Agent (107.9 AIC) was excluded because it was optimized on 2026-11-09.

Analysis period

  • Runs: 3 Avenger runs, 2026-11-11 12:30–14:26 UTC, all success.
  • The pre-aggregated sample covers only 11:53–14:38 UTC today. This is a short-window read, not a 7-day trend.

Cost profile

Metric Value
Total AIC 96.2
Avg AIC/run 32.1
Raw tokens (total / avg per run) 61,435 / 20,478
Model requests/run (turn proxy) 22.7 (10, 28, 30)
Tool calls/run 22.3 (9, 27, 31)
Cache efficiency 93.8% (2.97M cache-read of 3.17M input tokens)
Turns Not reported (grader value 0)

What drives the cost

Main-branch CI failed on 7 consecutive completed runs (08:58–14:09 UTC). The Avenger agent job ran in 22 of its last 24 hourly runs, and each run repeated the same work:

  • Byte-budget loop: test_debug_playbook_agent_consumption_budget fails because .github/aw/debug-playbook.md is 26,387 bytes against a 26,000-byte limit. Two runs made 14 and 15 bash calls re-checking size and re-running the guidance test. The third run made none.
  • Duplicate fixes: Two open issues propose the same trim, [avenger] Trim debug playbook to fit guidance size budget #76859 (12:39 UTC) and [avenger] Trim debug playbook under byte budget #76873 (13:36 UTC). Both PRs fell back to issues because they touched protected files.
  • Repeated filtered logs: get_job_logs was integrity-filtered twice per run (6 total), and the agent retried it.
  • Nonexistent tool names: The agent called github and safeoutputs as tool names, failing 1–2 times per run.

The cheapest run (43038561248, 18.0 AIC, 9 tool calls) classified the failure from job metadata as "release alignment, not code-fixable" and called noop. Early classification roughly halved its cost.

Ranked recommendations

  1. Skip when an open fix already exists (~20 AIC/run avg; up to ~30 on affected runs)

  2. Cap guidance-test iteration (~10 AIC/run avg; ~15 on loop runs)

    • Action: The source caps only make test-unit (Step 7). Add the same cap for make test-guidance and single-test reruns: at most 2 runs after a fix, then stop and report. Do not loop on wc -c size checks.
    • Evidence: 14 and 15 loop calls in 2 of 3 runs.
    • Overlaps with rejig docs #1, so do not add the two savings together.
  3. Stop retrying integrity-filtered logs (~3 AIC/run)

    • Action: On the first integrity denial from get_job_logs, classify the failure from job and step names, as run 43038561248 did. Do not retry.
    • Evidence: Integrity-filter friction cost was 2.6–3.6 AIC/run. All 6 filtered calls were get_job_logs.
  4. Use correct tool names (~1–2 AIC/run)

    • Action: State that GitHub MCP tools are called by their function name (for example actions_list), and that safe-output functions are called by their own names (for example create_pull_request, noop). Never invoke github or safeoutputs as tool names.
    • Evidence: "Tool 'github' does not exist" and "Tool 'safeoutputs' does not exist", 1–2 times per run.

Caveats

  • The sample is 3 runs from one 2.5-hour window, all with the same failure. That is below the 5-run minimum, so no tool-removal recommendation is made.
  • Savings are estimates from request-proportional AIC, not per-call measurements.
  • These changes lower cost per run but do not fix CI. The root cause (debug-playbook over budget) needs a human-reviewed change.
  • No sub-agent or setup-prefix change is recommended. The cost is in validation loops that need main-agent context.

References: §43029233076 · §43020983525 · §43038561248

Generated by Agentic Workflow AIC Usage Optimizer · copilot · routed: haiku55 xhigh · ~100 AIC · ⊞ 5.8K · ◷

  • expires on Nov 18, 2026, 7:25 AM UTC-08:00

Activity

  1. github-actions commented on Nov 12, 2026

    @github-actions
    ContributorAuthor

    This issue is being closed as outdated. A newer issue has been created: #77181

    View newer issue


    This action was performed automatically by the Agentic Workflow AIC Usage Optimizer workflow.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions