Skip to content

The review Role reads the pull request's head, read-only #537

Closed

Description

@don-petry

Under the Maintainer's decision of 2026-11-04, the review Role reads the repo at the pull request's head, read-only.

Today the Harness runs the CLI with --tools "" in an empty folder (harness/run.py, harness/eval/run.py), so the Role sees only the diff and the issue. Most of the private replay's misses needed the code around the change.

Rulings so far

  • The Maintainer's tarball ruling, 2026-11-04: "Tarball, allow codeload". The head's tree is read as GET /repos/{r}/tarball/{sha}, and the transport follows one more named redirect, api.github.com → codeload.github.com, for that read alone.
  • The Factory Builder's rulings on the design, made from the design note:
    • the tools are Read, Grep and Glob, with no shell, network, writes or MCP;
    • the Harness makes the checkout, never the model;
    • an eval case carries a top-level "head", 40 lowercase hex characters;
    • the eval reads the public cases' repo with no new secret.

Parts

  • Trials P-59, the read tools held to their folder on a Linux runner, and P-60, a commit's tarball as one redirect to codeload: Trials P-59 and P-60: the read tools held to their folder, and a tarball's one redirect to codeload #532, which refs this issue.
  • Record 0138 and its build, which closes this issue.
    • Records: 0138 replaces 0050's two "no tool" rows and 0045's redirect line, and nothing else in either; both records are marked "replaced by 0138".
    • Engine: the review Intent alone gets --tools Read,Grep,Glob --allowedTools Read,Grep,Glob --permission-mode dontAsk --restricted --setting-sources "", and the CLI starts in the head's folder.
    • The two flags: --restricted is what confines the reads, and --setting-sources "" only keeps the head's settings and CLAUDE.md out (P-59). Each flag has its own test.
    • Transport: it follows the codeload redirect for the tarball read alone and refuses a second hop (P-60).
    • Unpacker: it checks each member by hand. It takes regular files and directories alone, with no links, devices or FIFOs, no absolute paths, no .., and every resolved path inside task/head/. It caps size and count. Each refusal has its own test and mutant.
    • Eval case: eval-case/1 requires "head", with an example and counter-examples. A head that cannot be read stops the case with a code.

Done when

  • P-59 and P-60 are on main.
  • 0138 is accepted and its build is on main. Both Verify commands and the four reads have passed.
  • Owed before a live Run reads a private repo's head: what codeload's Location holds for a private repo read with an App's token.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions