Skip to content

docs: describe AWF's on-by-default budget notices for token-steering and max-ai-credits - #77087

Merged
rabeckett merged 10 commits into
mainfrom
copilot/update-token-steering-max-ai-credits-docs
Nov 12, 2026
Merged

rabeckett merged 10 commits into
mainfrom
copilot/update-token-steering-max-ai-credits-docs

Conversation

Copilot AI commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

From the next AWF release after v0.29.11, budget notices are on unless the AWF config sets them to false (github/gh-aw-firewall#11486). gh-aw's docs still describe the old opt-in behaviour, and they wrongly say that token-steering preserves the configured provider and model. Only docs and comments change here; gh-aw writes the same AWF config as before.

token-steering

  • sandbox.md, glossary.md, the schema description, specs/awf-config-sources-spec.md and the agent-facing .github/aw/syntax-agentic.md now say the setting controls AWF's budget notices, which are on by default, and that token-steering: false turns them off. Steering never changes the provider or model.
  • sandbox.md notes that the setting doesn't reach threat-detection runs, whose notices follow the defaults.

What the notices do (AWF releases after v0.29.11)

  • In sandbox.md and the max-ai-credits section of frontmatter.md: each conversation in the run, sub-agents included, gets the latest notice once rather than every notice; the notice states how much budget or time is left, then says what to do; and a further notice comes when about five requests' worth of credits, or about five minutes, remain.
  • sandbox.md no longer mentions an effective-token budget, which AWF doesn't have, and notes that up to v0.29.11 notices are on only when gh-aw enables them, which it does for every workflow with a credit budget.

max-ai-credits: -1 and the hard cap

  • frontmatter.md, glossary.md, both schema descriptions, the .github/aw budget guides, specs/ai-credits-specification.md and the comment in awf_config_build.go now say a frontmatter -1 turns off only the budget. With AWF releases after v0.29.11, notices continue against the agent deadline and AWF's hard cap.
  • The docs now say the hard cap is 10,000 AI credits by default and that AWF clamps larger budgets to it, and the max-ai-credits examples stay under it (5K, 2K instead of 100M).
max-ai-credits: -1   # no credit budget; notices continue on AWF releases after v0.29.11
sandbox:
  agent:
    token-steering: false   # turns notices off

Runtime organization variables

  • Review found that GH_AW_DEFAULT_MAX_AI_CREDITS and GH_AW_DEFAULT_DETECTION_MAX_AI_CREDITS only accept plain whole numbers at run time: the compiled step resets -1 or a K/M value to the built-in default (1000 or 400). compiler-enterprise-environment-controls.md and threat-detection.md now describe that fallback and point to frontmatter -1 for removing the budget. The spec's normative requirement that a runtime -1 disables the budget is unchanged; making the code honour it is a separate fix.

Generated reference

  • frontmatter-full.md was regenerated with make generate-schema-docs, and only the sections from this change were kept. The generator also produces unrelated older changes (router model values text, a removed comment under labels); those are left out, so this file still differs from a fresh generator run.

Out of scope, for #77054: the audit, logs and session-file docs, and the smoke workflows.

…d max-ai-credits

Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>
Copilot AI changed the title [WIP] Update token-steering and max-ai-credits docs for budget notices docs: describe AWF's on-by-default budget notices for token-steering and max-ai-credits Nov 12, 2026
Copilot AI requested a review from rabeckett November 12, 2026 06:29
@rabeckett
rabeckett requested a max review from Copilot November 12, 2026 06:38

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Documentation still omits required AWF version qualifiers for budget notices and retains misleading token-steering guidance that could cause workflow-authoring agents to disable notices unnecessarily.

2 open issues
What changed in this PR

Updates gh-aw’s configuration documentation for AWF’s on-by-default budget notices after v0.29.11, addressing #77053 without changing runtime behavior or emitted AWF configuration.

Changes:

  • Clarifies that token-steering controls budget notices, not provider or model selection.
  • Describes per-conversation notices and additional warnings near budget or time exhaustion.
  • Explains that max-ai-credits: -1 removes the budget while newer AWF releases continue sending notices unless explicitly disabled.
File Description
specs/​awf-config-sources-spec.md Corrects the documented purpose of enableTokenSteering.
pkg/​workflow/​awf_config_build.go Updates comments about omitted settings and AWF defaults.
pkg/​parser/​schemas/​main_workflow_schema.json Revises steering and credit-budget descriptions.
docs/​src/​content/​docs/​reference/​sandbox.md Explains notice behavior, version differences, and opt-out configuration.
docs/​src/​content/​docs/​reference/​glossary.md Corrects the Token Steering definition.
docs/​src/​content/​docs/​reference/​frontmatter.md Updates credit-budget guidance and the disabling example.
docs/​src/​content/​docs/​reference/​frontmatter-full.md Refreshes the two affected schema-derived reference sections.

Review effort: Max

…29.11 and fix token-steering example

Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The documentation misstates runtime -1 handling: the digits-only guard rejects it and restores fallback credit budgets rather than using AWF’s hard cap.

1 open issue
2 resolved since last review

Review effort: Max

…DITS values

Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Human assessment is needed to evaluate documentation inaccuracies about budget caps, threshold-notice delivery, and whether token-steering settings apply to threat detection.

No open issues 🎉

1 resolved since last review

Review effort: Max

…cope

Co-authored-by: rabeckett <5445735+rabeckett@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@rabeckett
rabeckett marked this pull request as ready for review November 12, 2026 10:56
@github-actions

github-actions Bot commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

✅ PR Code Quality Reviewer completed the code quality review.

🔎 Code quality review by PR Code Quality Reviewer

@github-actions

github-actions Bot commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

✅ PR Specification Reviewer completed successfully!

Generated by PR Specification Reviewer for #77087

@github-actions

github-actions Bot commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

✅ Ponytail Reviewer completed successfully!

Ponytail Reviewer

Outcome: reported_no_action

The agent completed and reported no action. A no-op output alone does not verify that all required work was completed or that publication was possible.

Agent messages are omitted from this comment. The category uses structured run status, not interpretation of agent prose.

Generated from Ponytail Reviewer · routed: gpt61 medium · ~4 AIC · ⌖ ~1 AIC · ◇ ~1 AIC · ⊞ 181 · ◷

Generated by Ponytail Reviewer for #77087

@github-actions

github-actions Bot commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

✅ Design Decision Gate 🏗️ completed the design decision gate check. See the comment below for the result and any generated ADR draft.

Design Decision Gate 🏗️

Outcome: reported_no_action

The agent completed and reported no action. A no-op output alone does not verify that all required work was completed or that publication was possible.

Agent messages are omitted from this comment. The category uses structured run status, not interpretation of agent prose.

Generated from Design Decision Gate 🏗️ · routed: haiku55 xhigh · ~1 AIC · ⌖ ~2 AIC · ◇ ~1 AIC · ⊞ 8.5K · ◷

🏗️ ADR gate enforced by Design Decision Gate 🏗️

@github-actions

github-actions Bot commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

🧠 Matt Pocock Skills Reviewer has completed the skills-based review. ✅

🧠 Reviewed using Matt Pocock's skills by Matt Pocock Skills Reviewer

@github-actions

github-actions Bot commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

✅ Test Quality Sentinel completed test quality analysis.

Test Quality Sentinel

Outcome: reported_no_action

The agent completed and reported no action. A no-op output alone does not verify that all required work was completed or that publication was possible.

Agent messages are omitted from this comment. The category uses structured run status, not interpretation of agent prose.

Generated from Test Quality Sentinel · routed: haiku55 xhigh · ~2 AIC · ⌖ ~2 AIC · ◇ ~1 AIC · ⊞ 7.2K · ◷

🧪 Test quality analysis by Test Quality Sentinel

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed the changed lines in the documentation, schema descriptions, and AWF config comment. I found no actionable correctness, regression, or maintainability issues in the diff. The delegated specialist pass was unavailable due to tool-permission denial; this review is based on the independent static analysis.

🔎 Code quality review by PR Code Quality Reviewer · copilot · gpt50 · ~2 AIC · ⌖ ~1 AIC · ⊞ 1.6K
Comment /review to run again

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed PR head 709d43425c521d8a7a69782d39dae55d715f2a7e against docs/src/content/docs/specs/ai-credits-specification.md §10.5 and T-AIC-PR-003, plus the affected frontmatter/compiler documentation. The PR updates the explanatory text and correctly documents the current runtime fallback, but the normative contract still requires runtime -1 to disable the budget while buildMaxAICreditsExport rejects -1 and restores the default. See the inline finding on the changed §10.5 line. No specification synchronization was applied; the implementation or conformance coverage must be corrected rather than weakening the contract.

Generated by PR Specification Reviewer for #77087 · copilot · gpt50 · ~3 AIC · ⌖ ~1 AIC · ⊞ 1.6K
Comment /review-specs to run again

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Skills-Based Review 🧠

Applied /grill-with-docs, checking the new notice and budget wording against the code, the merged AWF change (AWF#11486), and the pinned AWF version (v0.29.11).

The core rewrite is accurate. Verified against code: detection runs don't inherit TokenSteering, and -1 and K/M values in the runtime variables fall back to the defaults as documented. Two inline comments cover the remaining issues.

📋 Remaining items (not inline, outside the diff)

Stale text not touched by this PR

  • docs/src/content/docs/reference/cost-management.md:218 still uses 100M as a max-ai-credits example. Line 224 still says gh-aw "emits steering warnings" and that you "disable budget enforcement explicitly", which conflicts with the new notice wording. Update both.
  • pkg/workflow/schemas/awf-config.schema.json:129 still describes enableTokenSteering as "effective token budget steering" that requires maxEffectiveTokens and defaults to false. AWF#11486 changed the default, so this is now wrong.
  • docs/public/editor/autocomplete-data.json:1161 is generated from the schema. Regenerate it with docs/scripts/generate-autocomplete-data.js after the schema change.

Things to confirm

  • The -1 claim in sandbox.md is version-dependent, as the inline comment notes. Make sure frontmatter.md:566 and main_workflow_schema.json say the same thing.
  • awf_config_build.go comment (lines 138-143) is accurate. The code omits enableTokenSteering when max-ai-credits is negative, which matches the "on AWF after v0.29.11" wording.

Positive highlights

  • ✅ The notice-cap explanation in sandbox.md (newest notice per conversation, hard cap 10,000 unless AWF_AI_CREDITS_HARD_CAP is set) matches the AWF release notes.
  • ✅ The token-steering explanation makes clear that steering never changes provider or model.
  • ✅ The threat-detection exception is documented in sandbox.md.

The author has already resolved the six earlier Copilot threads, so I didn't repeat them. Requesting changes on the two inline items: the -1 version qualifier in sandbox.md, and the spec §10.5 MUST that the code doesn't implement.

@copilot please address the review comments above.

🧠 Reviewed using Matt Pocock's skills by Matt Pocock Skills Reviewer · copilot · routed: haiku55 xhigh · ~12 AIC · ⌖ ~2 AIC · ⊞ 6.6K
Comment /matt to run again

@gh-aw-bot

Copy link
Copy Markdown
Collaborator

@copilot

🍳 A little prep to help this PR along! Copilot assignment requested for the following outstanding work:

  1. Update this branch with the latest main using make merge-main, resolving any conflicts and preserving the intended changes.
  2. Review (docs/src/content/docs/specs/ai-credits-specification.md:654): address unresolved feedback - Review comment
  3. Review (docs/src/content/docs/reference/sandbox.md:260): address unresolved feedback - Review comment
  4. Review (docs/src/content/docs/specs/ai-credits-specification.md:654): address unresolved feedback - Review comment
  5. Fix failing check Analyze (python) (FAILURE): Check run.

Keeping the changes focused; human review and CI get the final taste test. No merging or CI triggering.

Structured data
{
  "completed_task": 0,
  "head": "709d43425c521d8a7a69782d39dae55d715f2a7e",
  "kind": "requested",
  "schema_version": "1",
  "state": "91abc813a4e25b5c772e023f5d54e5acbe4b99fd11d22917ccbb29da59ed2e5c",
  "work": [
    "43282e7f5b3023b8c8cf9099eab79e7b99f650965f08b47d3df33cde5589879d",
    "5962a999b00e0b04cb7114ed3ee32ed7f60695c8b17d3a3e990f4d2ad5dd880b",
    "afbaada1f5c4b0f10b20b2fb645fc53dada4107da7a8aaed369fd52e8665148c",
    "bb7e9e468f1d826361e26438bf53a1712b92e55cc01f756a730c5387b6b2ff70"
  ],
  "workflow": "pr-sous-chef"
}

Generated by 👨🍳 PR Sous Chef · pi · routed: haiku55 xhigh · ~2 AIC · ⌖ ~2 AIC · ⊞ 6.9K · ◷
Comment /souschef to run again

Copilot AI and others added 3 commits November 12, 2026 11:39
…steering-max-ai-credits-docs

Co-authored-by: gh-aw-bot <259018956+gh-aw-bot@users.noreply.github.com>
Co-authored-by: gh-aw-bot <259018956+gh-aw-bot@users.noreply.github.com>
…tocomplete; link #77140

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@rabeckett

Copy link
Copy Markdown
Collaborator

Follow-up on the out-of-diff items from the skills review (674ae6d):

  • cost-management.md now uses a 5K example, mentions AWF's 10,000-credit hard cap, and says AWF sends the budget notices and that a negative max-ai-credits disables only enforcement.
  • autocomplete-data.json has the new token-steering description. A fresh generate-autocomplete-data.js run also picks up unrelated schema drift from main, so only this entry was applied.
  • pkg/workflow/schemas/awf-config.schema.json is the vendored AWF v0.29.11 schema, so its enableTokenSteering text is left as it is. It will change when gh-aw moves to the next AWF release.
  • The runtime -1 sentinel gap is filed as -1 and K/M values in GH_AW_DEFAULT_MAX_AI_CREDITS fall back to the default budget #77140 and linked from the spec.

@rabeckett
rabeckett merged commit eb38909 into main Nov 12, 2026
69 checks passed
@rabeckett
rabeckett deleted the copilot/update-token-steering-max-ai-credits-docs branch November 12, 2026 15:34
@github-actions

Copy link
Copy Markdown
Contributor

🎉 This pull request is included in a new release.

Release: v1.2.3

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Update the token-steering and max-ai-credits docs for AWF's on-by-default budget notices

5 participants