Skip to content

P-59's trial tests bring their own timeout, and leave HOME empty on macOS - #545

Merged
don-petry merged 3 commits into
mainfrom
claude/544-fake-timeout
Nov 4, 2026
Merged

don-petry merged 3 commits into
mainfrom
claude/544-fake-timeout

Conversation

@don-petry

@don-petry don-petry commented Nov 4, 2026 •

Copy link
Copy Markdown
Contributor

Closes #544.

Since #532, three ReadConfinementTrial tests failed on macOS under PATH=/usr/bin:/bin. Two things caused it, and a third came from review:

  • No timeout on macOS's own PATH. tools/trial_p59_read_confinement.sh refuses to start without one. The stand-in folder now holds a bin/timeout that runs its command after the duration (exec after shift), as NarrowedToken's stand-in curl and openssl do, and environment(folder) puts that bin/ first on PATH.
  • Bytecode in the empty HOME. Once the script ran, macOS's own python3 cached the stand-in CLI's bytecode under HOME/Library/Caches/com.apple.python, which test_the_engine_configuration_is_judged_by_the_condition holds empty. The stand-in's claude wrapper now runs python3 -B.
  • The stand-in checks its duration, as GNU's does. On Linux CI the stand-in now replaces the real timeout in these tests, so it takes what GNU timeout takes (a number with an optional s, m, h or d) and refuses anything else, or no command, with 125. test_the_stand_in_timeout_runs_what_gnu_timeout_runs holds it on eleven durations, and runs the real timeout in /usr/bin or /bin beside it where there is one, so on CI the two must agree; on this Mac all eleven agree with Homebrew's coreutils 9.12. With line 212's 180 made 18O, test_each_configuration_is_given_the_read_tools_and_its_own_flags fails.

The script is unchanged: a real trial still needs a real timeout. Because the stand-in execs, the /proc/<parent>/environ a leaking call reads on Linux is now the CLI's own rather than a separate timeout's; both start with the same environment and marker, so no assertion changes.

No record, no rule changed: a test-only change.

Verify

  • PATH=/usr/bin:/bin python3 -m unittest tests.test_tools.ReadConfinementTrial: 5 tests OK (3 failed before).
  • PATH=/usr/bin:/bin SIM_SEEDS=40 make verify on an export of 5b01bd7: 1569 tests OK (8 expected failures).
  • The P-59 mutants (tools/mutants.py, the P59 block), run alone on b9f6ef5: 18 of 18 killed. 5b01bd7 changes only the stand-in and adds its test.

Reviews

Reviewer Head Unread parts Blocking Major Minor Info
astra 88c7e2e none 0 1 0 0
muse b9f6ef5 none 0 0 1 0
claude-review 88c7e2e none 0 0 1 1
claude-security 88c7e2e none 0 0 0 0
Finding Severity Criterion Where What Disposition
astra#1 major PR-3 tests/test_tools.py:8087 The timeout double hides invalid duration arguments Fixed in b9f6ef5: the stand-in refuses a malformed duration with 125; line 212's 180 made 18O now fails the tests
muse#1 minor PR-3 tests/test_tools.py:8088 Stand-in timeout's refusal rule is untested and rejects valid GNU durations Fixed in 5b01bd7: it takes GNU's durations (1.5, 90s, 3m) and a new test holds it, beside the real timeout where there is one
claude-review#1 minor PR-3 tests/test_tools.py:8087 The stand-in timeout accepts any first argument, so Linux CI no longer checks how the script calls timeout Fixed in b9f6ef5, as astra#1; the reviewer's 18O mutant is killed
claude-review#2 info none the change The PR body says the stand-in CLI reads no parent's environ, but on Linux it does when a call leaks Fixed in the PR body: the parent environ a leaking call reads is now the CLI's own, with the same environment

Bots: Codex reviewed b9f6ef5 with no findings; SonarCloud's gate passed on 5b01bd7; CodeRabbit skipped (auto reviews off).

Argon: not run (quota), owed; Astra read 88c7e2e as its stand-in.

🤖 Generated with Claude Code

…acOS

The ReadConfinementTrial tests ran tools/trial_p59_read_confinement.sh with
the host's PATH, and the script refuses to start without `timeout`, which
macOS keeps only in Homebrew. The stand-in folder now holds a bin/ with a
`timeout` that runs its command after the duration, as NarrowedToken's
stand-in curl and openssl do, and the tests put it first on PATH.

Once the script ran, macOS's own python3 cached the stand-in CLI's bytecode
under the empty HOME the test checks, so the stand-in runs with -B.

The script is unchanged: a real trial still needs a real `timeout`.

Closes #544

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@don-petry
don-petry requested a review from a team as a code owner November 4, 2026 18:12
@coderabbitai

coderabbitai Bot commented Nov 4, 2026 •

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

Run configuration

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • Trigger review
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 4, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-04T18:52:07.978108Z 5b01bd7 New commits
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

don-petry and others added 2 commits November 4, 2026 12:16
The stand-in ran whatever followed its first argument, so on Linux CI, where
it now stands first on PATH, a malformed duration in the script's calls to
`timeout` went unnoticed: with line 212's 180 made 18O, the tests passed.
It now exits 125, as GNU timeout does, for a duration that is not whole
seconds or no command, and that change to line 212 fails the tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…test holds it

The stand-in refused 1.5 and 90s, which GNU timeout runs, and no test
held what it refused. It now takes a number with an optional s, m, h or d
and refuses anything else, or no command, with 125. A new test runs it on
eleven durations and no command, and runs the real timeout in /usr/bin or
/bin beside it where there is one, as on Linux CI, so the two must agree.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@sonarqubecloud

sonarqubecloud Bot commented Nov 4, 2026

Copy link
Copy Markdown

@don-petry
don-petry added this pull request to the merge queue Nov 4, 2026
Merged via the queue into main with commit 716726f Nov 4, 2026
19 checks passed
@don-petry
don-petry deleted the claude/544-fake-timeout branch November 4, 2026 19:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

P-59 trial tests need a timeout on PATH, which macOS lacks; give them a fake one

1 participant