You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
The eval scratch is held part by part, the heads folder is made fresh, and the canary's head is held apart #626
PR #624 (#623) unpacks each case's head apart from the eval scratch. Reads after its last fix round found four smaller gaps beside it:
scratch() reads containment by separator.harness/eval/__init__.py's scratch still checks with startswith(... + os.sep). A heldout linked to the root would be taken as the scratch, though the root holds the workspace. heads_place was moved to os.path.commonpath for this reason. This predates Each case's head is unpacked apart from the set's cases (#623) #624.
The heads folder is not made fresh.head_of makes heads with exist_ok=True, while plan makes heldout without it. A heads left by an earlier job on a reused runner would be accepted, with its own modes. GitHub's hosted runners are discarded after each job, so this cannot happen today.
Nothing holds the canary's head apart from its case.harness/eval/canary.py keeps them in separate temporary directories, but no test or mutant fails if they share one. The canary's case is the public example, so the harm is small.
Two of Each case's head is unpacked outside the folder that holds the set's case files #623's mutants are killed by a KeyError. In test_each_head_is_unpacked_apart_from_the_cases, "the fetch unpacks each head in the scratch" and "the run step looks for each head in the scratch" fail because the CLI is never asked, not on an assertion. An explicit assertIn("head", seen) would say why.
The change
scratch reads containment part by part, as heads_place does. A test links heldout to the root, and a mutant restores the separator form.
plan makes heads new, mode 0o700, beside heldout. A leftover one stops the step, as a leftover scratch does.
A test that the canary's head and case share no folder below its temporary directories, with a mutant.
Activity