Skip to content

initiative-planner: Bob materializes a plan even when he has unresolved open questions — should gate, not guess #682

Closed

Description

@don-petry

Problem

The Initiative Planner ("Bob", .github/workflows/initiative-planner.yml) always materializes an epic + sub-issue DAG, even when the source idea still has unanswered, plan-blocking questions. When an idea is approved (or re-dispatched) before its scope is fully clarified, Bob plans anyway, records what he couldn't resolve in open_questions, and ships the whole inert epic. The human is then left to manually close the epic and every sub-issue and re-run planning once the questions are answered.

This has now happened twice in one day:

Each cleanup is 6–9 manual issue closures. The cost scales with story count and recurs every time scope isn't fully settled at approval time.

Root cause

By design Bob is non-interactive and "never blocks on prompts" (prompts/bmad/scrum-master.md). Unresolved items are routed to an open_questions array (scripts/initiative-planner/plan.schema.json) — but that array is treated as advisory output only:

  • apply-plan.sh materializes the epic + stories regardless of whether open_questions is empty, then appends the questions as an "Open questions for review" comment on the discussion (scripts/initiative-planner/apply-plan.sh:122).
  • Nothing inspects open_questions to decide whether to materialize at all.

So "I have open questions" and "here is a finished, ready-for-dev plan" are emitted simultaneously, which is contradictory and generates throwaway issues.

Desired behavior

Bob must not materialize a plan while he has blocking open questions. Specifically:

  1. If the emitted plan's open_questions is non-empty (or, better, contains any item flagged required/blocking), apply-plan.sh should not create the epic or stories.
  2. Instead it should post the open questions back to the source discussion (the same comment it posts today, minus the "epic created" framing) and exit cleanly — making clear the idea is not yet plannable and planning will resume once the questions are answered.
  3. Re-running after the questions are answered then produces the real epic/DAG. (Today's manual flow — close everything, answer, re-dispatch — becomes: answer, re-dispatch.)
  4. DRY_RUN semantics unchanged.

Open design questions for this fix

  • Blocking vs. non-blocking questions. Not every open_questions item should halt planning (some are "nice to confirm"). Options: (a) treat any open question as blocking; (b) add a per-question severity/blocking flag to the schema and only halt on blocking ones. (b) is more precise but a schema change.
  • Where to enforce — in apply-plan.sh (a hard gate, consistent with how it already enforces the no-initiative:auto rule), in the prompt (ask Bob to self-abort), or both. A script-level gate is the only one that can't be reasoned around by the model.
  • Signal back to the human — discussion comment only, or also a workflow annotation / non-zero-but-not-failing exit so the run is visibly "needs answers" rather than "succeeded."

Acceptance criteria

  • A plan with blocking open questions creates zero issues (verified in dry-run and live).
  • The open questions are posted back to the source discussion with clear "not yet planned — answer these to proceed" framing.
  • A plan with no blocking open questions materializes exactly as today.
  • The gate is covered by tests/test_initiative_planner.bats (the existing test home), mirroring how the initiative:auto invariant is tested.
  • AGENTS.md / scripts/initiative-planner/README.md document the new gate.

References

Activity

  1. don-petry commented on Jun 14, 2026

    @don-petry
    CollaboratorAuthor

    Scope addition — trigger Bob on discussion comments

    The gate above only closes half the loop (Bob stops when he has open questions). The other half is resuming automatically once the questions are answered. So this revision should also make the planner re-plan when a new comment is posted on an approved Ideas discussion.

    Target flow:

    1. Idea approved → Bob plans → blocking open questions → nothing materialized, questions posted back to the discussion (per the gate above).
    2. A maintainer answers in a discussion comment.
    3. That comment re-triggers Bob, who now plans cleanly and materializes the epic/DAG.

    Implementation notes / caveats (must be handled)

    • Add the trigger: on: discussion_comment: types: [created] in .github/workflows/initiative-planner.yml.
    • Reuse the redispatch bridge. claude-code-action aborts on discussion/discussion_comment event contexts ("Unsupported event type") — that's exactly why the current idea:approved path runs through redispatch.sh → workflow_dispatch. The comment trigger must go through the same bridge (resolve the discussion number from github.event.discussion.number, re-dispatch under workflow_dispatch). Requires GH_PAT_WORKFLOWS (a GITHUB_TOKEN-fired dispatch never starts a run).
    • Loop guard (critical). Bob posts comments on the discussion himself (the open-questions / plan-summary comment via apply-plan.sh). A naive comment trigger would make him re-plan in response to his own comment → infinite loop. Must ignore comments authored by the bot / github-actions[bot] (and ideally only the <!-- initiative-planner --> marker comments).
    • Gate the trigger. Only re-plan when the discussion is actually in scope: category Ideas and carries idea:approved (don't plan every comment on every random discussion). Consider also limiting to maintainer authors.
    • Concurrency. The existing concurrency group is keyed on discussion number and already prevents overlapping plans for the same discussion — good, but confirm a flurry of comments coalesces rather than queueing N plans.
    • Tests: extend tests/test_initiative_planner_redispatch.bats to cover the comment-event → redispatch path and the bot-author / unapproved-discussion skip cases.

    This is part of the same revision as the gate — they're two halves of one "answer-in-a-comment → auto re-plan" workflow and should land together.


    Generated by Claude Code

  2. don-petry commented on Jun 14, 2026

    @don-petry
    CollaboratorAuthor

    Dev-Lead Implementation Plan

    Issue: #682 — initiative-planner: Bob materializes a plan even when he has unresolved open questions — should gate, not guess

    Scope

    Add a script-level hard gate in apply-plan.sh so that when a plan contains any blocking open question, Bob creates zero issues, posts the questions back to the source discussion with "not yet planned" framing, and exits cleanly.

    Design decisions

    • Blocking vs non-blocking (issue option b): add an optional per-question blocking flag to the schema. Plain-string open_questions remain non-blocking (backward compatible — the existing test fixture and live plans keep materializing). Only items shaped {"question": "...", "blocking": true} halt planning. This preserves current behavior and avoids treating every advisory question as a stop.
    • Where to enforce: in apply-plan.sh (the only gate the model can't reason around), mirroring the existing initiative:auto invariant. The Bob prompt is also updated to make the flag intentional.
    • Signal back: discussion comment + GITHUB_STEP_SUMMARY + a ::warning:: annotation, with a clean exit 0 (DRY_RUN semantics unchanged).

    Implementation Checklist

    • scripts/initiative-planner/plan.schema.json — allow open_questions items to be a string OR {question, blocking} object
    • scripts/initiative-planner/apply-plan.sh — add blocking-gate before epic creation; normalize open_questions rendering in the existing summary path
    • scripts/initiative-planner/README.md — document the new gate (Invariants)
    • AGENTS.md — brief note on the gate
    • prompts/bmad/scrum-master.md — note the blocking flag semantics

    Tests to Write (tests/test_initiative_planner.bats)

    • validate-plan accepts open_questions with a blocking flag
    • apply-plan gates on a blocking open question: creates zero issues, exits 0
    • apply-plan blocking gate posts the open questions back to the discussion
    • apply-plan does NOT gate on a non-blocking (string / blocking:false) open question

    Test Command

    bats tests/test_initiative_planner.bats and bash .dev-lead/scripts/dev-lead-lint.sh

  3. don-petry commented on Jun 14, 2026

    @don-petry
    CollaboratorAuthor

    Dev-Lead: Implementation Complete

    Plan Execution

    • Added a per-question blocking flag to plan.schema.json open_questions (issue option b) — plain strings stay advisory/non-blocking, objects {"question":"...","blocking":true} are plan-blocking. Chosen over "treat any question as blocking" so the existing fixture/live plans keep materializing and only true blockers halt.
    • Added a hard, script-level gate in apply-plan.sh (before epic creation, mirroring the initiative:auto invariant): if any blocking question is present it creates zero issues, posts the questions back to the source discussion with "Not yet planned" framing, emits a ::warning:: + step summary, and exits 0. DRY_RUN semantics unchanged.
    • Fixed the existing open_questions rendering in the summary path to normalize string|object forms.
    • Documented the gate in README.md, AGENTS.md, and prompts/bmad/scrum-master.md.
    • Tests added mirroring the initiative:auto test.

    Test Results

    $ bats tests/test_initiative_planner.bats tests/test_initiative_planner_redispatch.bats
    1..25
    ok 2 validate-plan accepts open_questions carrying a blocking flag
    ok 7 apply-plan (DRY_RUN) creates epic + stories + edges and posts comment
    ok 12 apply-plan gates on a blocking open question: creates zero issues and exits 0
    ok 13 apply-plan blocking gate posts the open questions back to the discussion
    ok 14 apply-plan does NOT gate on a non-blocking open question
    ... (all 25 pass)
    
    $ bash .dev-lead/scripts/dev-lead-lint.sh
    [lint] shellcheck on 51 file(s)...
    [lint] validating 4 agent profile(s)...
    [lint] all checks passed
    

    Files Changed

    • scripts/initiative-planner/plan.schema.json: open_questions items now accept a string OR {question, blocking} object.
    • scripts/initiative-planner/apply-plan.sh: blocking open-questions gate (zero issues + discussion comment + warning/step-summary + clean exit); normalized open_questions rendering in the existing summary.
    • scripts/initiative-planner/README.md, AGENTS.md, prompts/bmad/scrum-master.md: document the gate + flag semantics.
    • tests/test_initiative_planner.bats: 4 new tests (schema accepts blocking flag; gate creates zero issues & exits 0; gate posts questions to discussion; non-blocking does not gate).

    Acceptance Criteria

    Notes

    • Backward-compatible: existing plans that emit plain-string open_questions (incl. the existing test fixture) are unaffected — only blocking:true objects gate.
    • The gate exits 0 (clean) rather than non-zero so the workflow step doesn't fail; visibility comes from the ::warning:: annotation + step summary.
    • No live GitHub calls were made; verification is via the DRY_RUN path (the only path tests exercise, per the suite design).
  4. don-petry commented on Jun 14, 2026

    @don-petry
    CollaboratorAuthor

    ✅ Design decision — resolves the "blocking vs. non-blocking questions" open item

    Adopt approach (b): classify every open question as required or optional, and gate only on required. This supersedes the corresponding "Open design question" and acceptance-criteria bullets above.

    Schema (scripts/initiative-planner/plan.schema.json)

    Replace the flat open_questions: string[] with objects carrying a severity:

    "open_questions": [
      { "question": "…", "required": true },
      { "question": "…", "required": false }
    ]

    A single array with a required boolean is preferred over two arrays (less churn, easier to render). For back-compat, a bare string item should be treated as optional. validate-plan.py must accept the new shape.

    Gate (scripts/initiative-planner/apply-plan.sh)

    • ≥1 unresolved required question → create ZERO issues. Post the required questions back to the source discussion as a clear "⛔ Not yet plannable — answer these to proceed" comment, and exit cleanly (non-failing). This is the hard gate.
    • No required questions → materialize exactly as today. Carry any optional questions through unchanged as the "Open questions for review" checklist on the inert epic / discussion — they are advisory and get confirmed by the human before initiative:auto.

    Classification guidance (prompts/bmad/scrum-master.md)

    Bob must tag each open question:

    • required — he cannot ground a correct plan without the answer: unresolved scope, a decision that changes the story set or the DAG, or an ambiguous deliverable.
    • optional — scope-boundary notes, "confirm X is tracked under epic #N", assumed-sane defaults, cross-repo follow-ups: things that belong in the activation review, not the build.

    Why this split (concrete evidence)

    Today's #666 re-plan converged on 5 open questions that were all optional (out-of-repo scope, "confirm parent epic #495", assumed admin release path). A blanket "any open question blocks" gate would have deadlocked a perfectly buildable plan. Required/optional is precisely what makes the gate usable instead of a permanent stall.

    Acceptance criteria (supersede the matching bullets above)

    • A plan with ≥1 required question creates zero issues (verified dry-run and live), and posts the required questions to the discussion.
    • A plan with only optional questions materializes as today, with the optional questions surfaced as the review checklist.
    • Schema + validate-plan.py accept the new object shape and back-compat bare strings (as optional).
    • tests/test_initiative_planner.bats covers both paths (required-blocks-zero-issues, optional-does-not-block).
    • AGENTS.md / scripts/initiative-planner/README.md document the required/optional gate.

    Generated by Claude Code

  5. added 2 commits that reference this issue on Jun 14, 2026
  6. added 3 commits that reference this issue on Jun 18, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    dev-leadFor dev-lead agent pickup

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions