Skip to content

A test that a Run is given no part of a case's expected, and that its head holds no .git #614

Closed

Description

@don-petry

Why

What keeps a case's answer from the model is that the Task is built from the case's diff and head alone. Nothing tests
it. A change that passes the case whole, or unpacks a head with its history, would hand the model the answer.

The change

A test builds a Run's Task and workspace from a case whose expected holds marker strings, and asserts no marker
appears in the Task, the prompt or any file the read tools can reach, and that the unpacked head holds no .git.

Done when

  • The test, and a mutant that copies expected into the Task, which it kills.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions