Skip to content

fix(auq-format): clarify label vs description split to prevent InputValidationError - #3222

Closed
Bhumika-1432006 wants to merge 1 commit into
garrytan:mainfrom
Bhumika-1432006:fix/auq-label-vs-description-clarity
Closed

Bhumika-1432006 wants to merge 1 commit into
garrytan:mainfrom
Bhumika-1432006:fix/auq-label-vs-description-clarity

Conversation

@Bhumika-1432006

Copy link
Copy Markdown
Contributor

Partially addresses #3219 (the AUQ option label issue; the report-rewrite issue is tracked separately).

Problem

In a 7-pass plan review, the model repeatedly wrote the long ✅/❌ description prose into the label field and omitted the short chip label entirely, producing InputValidationError round trips. The format template described option text as a single conceptual block without distinguishing the two required tool fields.

Fix

After the decision-brief format template in scripts/resolvers/preamble/generate-ask-user-format.ts, add an explicit AskUserQuestion tool fields — label vs description note:

  • label: short phrase (≤5 words), chip text; (recommended) goes here
  • description: the ✅/❌ reasoning prose

Also add a self-check bullet for plan-review skills:

Each option has both a short label (≤5 words) and a separate description (the ✅/❌ prose); never collapse both into label alone.

Files changed

  • scripts/resolvers/preamble/generate-ask-user-format.ts — source of truth
  • All generated SKILL.md files — regenerated with bun run gen:skill-docs --host claude

…alidationError (garrytan#3219)

The decision-brief format described option text as a single conceptual block,
causing the model to write long ✅/❌ prose as the label field and omit the
short chip label entirely. This produces InputValidationError round trips.

Add an explicit 'label vs description' note after the format template
explaining that:
- label: short phrase (≤5 words), chip text, (recommended) goes here
- description: the ✅/❌ prose
and add a self-check bullet for plan-review skills to verify both fields are
present before emitting the tool call.

Regenerate all SKILL.md files from updated template.
@trunk-io

trunk-io Bot commented Oct 16, 2026

Copy link
Copy Markdown

Merging to main in this repository is managed by Trunk.

  • To merge this pull request, check the box to the left or comment /trunk merge below.

After your PR is submitted to the merge queue, this comment will be automatically updated with its status. If the PR fails, failure details will also be posted here

@capy-ai

capy-ai Bot commented Oct 16, 2026

Copy link
Copy Markdown
Contributor

Triage (fix wave 5, GSTA-37): the problem (#3219 item 2) is real, but it's S3: the InputValidationError round trip corrects itself. This PR edits the shared preamble resolver, which regenerates every SKILL.md, including outputs owned by open PRs (#3190, #3209), and it needs a full paid eval pass over every skill. It's deferred to the next batched wave and will be rebased onto main then. The other half of #3219 (the report rewrite) is fixed in #3225.

@capy-ai

capy-ai Bot commented Oct 19, 2026

Copy link
Copy Markdown
Contributor

Thanks @Bhumika-1432006 for tracking this down and for the clear write-up. We're going to close this one, and here's why.

We tried the smallest version of it on current main: just your checklist bullet, which only renders into the four plan-review skills. Even that one line pushes plan-ceo-review, plan-eng-review and plan-devex-review over the size caps in test/parity-suite.test.ts. Those skills have no headroom left, and the full version adds about 250 lines across 41 SKILL.md files. The error itself also corrects on the next turn, because the tool schema already defines label and description and the validation message says what was wrong. So the cost in every skill's prompt outweighs the saved round trip.

If we do a trim pass on the plan-review preambles, we'll fold a one-line label/description note in then and credit this PR. Thank you for the contribution.

@capy-ai capy-ai Bot closed this Oct 19, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant