Skip to content

[#1621 S1] Harness-only thread resolution: remove the model's resolveReviewThread path #1691

Closed

Description

@don-petry

Story

As a maintainer relying on required_review_thread_resolution as the real merge gate,
I want thread resolution to be performed only by the harness, never by the model,
so that the deterministic guards in dev-lead-fix-reviews.sh are authoritative instead of advisory.

Parent: #1621 (see its "Root cause" section). Story 1 of 2.

The bypass

prompts/dev-lead/fix-reviews.md § "Resolving a thread" hands the model the raw mutation:

gh api graphql -f query='mutation { resolveReviewThread(input: {threadId: "THREAD_NODE_ID"}) { thread { isResolved } } }'

The model has a shell and this snippet, so it can resolve any thread directly — bypassing resolve_addressed_bot_threads(), resolve_actor_outdated_threads(), review_thread_is_agent_authored (#1415) and the fail-closed re-read. The surrounding prose rules ("Never resolve a thread you did not fix") are instructions to a model, and on petry-projects/.github PR #1044 they were not followed.

Acceptance Criteria

  1. The resolveReviewThread mutation and its example are removed from prompts/dev-lead/fix-reviews.md. The prompt's contract becomes: reply with the addressed-marker; the harness resolves. The same removal is applied to any other dev-lead prompt that hands the model this mutation — check prompts/dev-lead/fix-bot-comment.md and prompts/dev-lead/review-changes.md, which also reference it.
  2. The reply instruction is unchanged and remains mandatory: name the file(s)/function(s) changed and how the change addresses the concern, ending with <!-- dev-lead:addressed -->, stamped only on a genuine addressed reply and never on a skip note.
  3. The prompt states explicitly that the model must not call resolveReviewThread (or unresolveReviewThread) under any circumstance, and that resolution is the harness's responsibility — so the rule is legible rather than merely absent.
  4. The existing marker-ownership rules survive verbatim in behaviour: marker-less human threads are still never resolved ([Phase 4] Close the maintainer review-gate gap — dev-lead PRs cannot be blocked by the owner (shared identity + agent-resolved threads) #1415); bot threads and outdated threads still resolve through the harness paths that already handle them.
  5. A prompt-coverage assertion fails if any prompts/dev-lead/*.md reintroduces a resolveReviewThread mutation. Wire it into the existing prompt-coverage suite (tests/dev-lead/integration/test_prompt_coverage.sh).
  6. A bats case asserts that after a fix-reviews pass the only resolutions observed come from the harness code path — i.e. no resolution occurs for a thread whose latest reply is not an our-account addressed-marker.
  7. shellcheck --severity=warning -x stays clean and the suites are registered in the existing lint/bats jobs.

Tasks / Subtasks

Dev Notes

References

Activity

  1. github-actions commented on Sep 7, 2026

    @github-actions
    Contributor

    Thanks for the detailed write-up and for tying this back to the specific incident on PR #1044 — the acceptance criteria are clear and the scope is well-bounded. Before work begins: beyond the three prompts called out (fix-reviews.md, fix-bot-comment.md, review-changes.md), have you confirmed via a full grep of prompts/dev-lead/ that no other files in that directory also expose the resolveReviewThread mutation, so the audit scope in AC #1 is complete?

  2. don-petry commented on Sep 7, 2026

    @don-petry
    CollaboratorAuthor

    Dev-Lead Implementation Plan

    Issue: #1691 — [#1621 S1] Harness-only thread resolution: remove the model's resolveReviewThread path

    Scope

    Remove the resolveReviewThread GraphQL mutation from the dev-lead review prompts and make the contract "reply with the addressed-marker; the harness resolves", so the deterministic guards in dev-lead-fix-reviews.sh are the sole resolution path. No change to scripts/dev-lead-fix-reviews.sh (its guards are already correct — Story 2 scope).

    Implementation Checklist

    • prompts/dev-lead/fix-reviews.md — remove the resolveReviewThread(input:…) mutation + example; add explicit "must not call resolveReviewThread/unresolveReviewThread — the harness resolves" statement; preserve the marker-ownership scope (bot vs marker-less human) as description of harness behaviour; keep the reply/addressed-marker instruction untouched
    • prompts/dev-lead/fix-bot-comment.md — same removal + prohibition
    • prompts/dev-lead/review-changes.md — same removal + prohibition
    • tests/dev-lead/integration/test_prompt_coverage.sh — stop requiring the resolve mutation; add an assertion that FAILS if any prompts/dev-lead/*.md reintroduces a resolveReviewThread(/unresolveReviewThread( mutation

    Tests to Write

    Test Command

    bats tests/dev-lead/unit/ && bash tests/dev-lead/integration/test_prompt_coverage.sh (plus bash .dev-lead/scripts/dev-lead-lint.sh)

  3. don-petry commented on Sep 7, 2026

    @don-petry
    CollaboratorAuthor

    Dev-Lead: Progress (in progress — not yet durable)

    Provisional. Nothing is pushed yet; the automation posts the authoritative
    completion record (with PR number + head SHA) after the push. Treat the notes
    below as intent, not delivered results.

    Plan Execution (intended)

    Local check output (provisional — not the pushed run)

    prompt-coverage: PASS
    bats tests/dev-lead/unit/: 884 tests, new resolution cases green
      (1 unrelated pre-existing flake in test_engine_fallback.bats — passes in isolation)
    dev-lead-lint.sh: all checks passed (shellcheck 132 files + agent frontmatter)
    shellcheck --severity=warning -x test_prompt_coverage.sh: clean
    

    Files touched (this workspace, not yet pushed)

    • prompts/dev-lead/fix-reviews.md, prompts/dev-lead/fix-bot-comment.md, prompts/dev-lead/review-changes.md
    • tests/dev-lead/integration/test_prompt_coverage.sh, tests/dev-lead/unit/test_fix_reviews.bats

    Notes

    • Per Dev Notes, scripts/dev-lead-fix-reviews.sh is intentionally unchanged (its guards are already authoritative — Story 2 scope). Both affected suites are already registered in test-dev-lead.yml (AC Workflow should skip no-op PRs in MAX_PRS count #7), so no workflow change was needed.
  4. don-petry commented on Sep 7, 2026

    @don-petry
    CollaboratorAuthor

    Dev-Lead: Implementation Complete — PR #1699

    The implementation for issue #1691 is durable: commits are pushed and a pull request is open. This record is backed by verifiable artifacts you can confirm in one click.

    Review happens on the PR. Acceptance criteria and test results are evidenced by the PR's diff and CI checks — not asserted here. If a later stage fails, this claim is retracted in place.

  5. added 2 commits that reference this issue on Sep 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugBug reportsdev-leadFor dev-lead agent pickupneeds-human-reviewFlagged by automated PR review agentsecuritySecurity-related PRs and issues

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions