Repository navigation
test: deflake test_encode_and_write_as_completed_cancels_stray_writes_on_failure - #4409
Merged
d-v-b merged 2 commits intoSep 25, 2026
Merged
Conversation
…_on_failure The test assumed the slow write is always in flight when the failing write raises. When the failing chunk's encode lands first, its write raises while the loop still awaits the other encode; `gather` then runs the done callback of that already-failed child eagerly and returns an outer future that has already failed, so `await gather(...)` does not suspend, and the `finally` cancels the slow write before its first step. `write_started` is never set. That outcome is correct (no stray write), but it is not what the test checks. Make the failing write wait until the slow write has started before raising, so the in-flight case is exercised every time. Reproduced at 8/400 (py3.13) and 3/400 (py3.14) with 16 parallel workers; with the fix, 2000/2000 on both. It had failed in CI five times since the test landed, on Linux, macOS and Windows. Assisted-by: ClaudeCode:claude-opus-5-5
d-v-b
marked this pull request as ready for review
September 25, 2026 07:39
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
AI PR that makes a flaky test a bit more reliable.
🤖 AI text below 🤖
Summary
tests/test_fused_pipeline.py::test_encode_and_write_as_completed_cancels_stray_writes_on_failure(added in #4205) fails intermittently onassert write_started.is_set(). It has failed five times in CI since 2026-07-30:Cause
The test assumes the slow write is always in flight when the failing write raises. It isn't when the failing chunk's encode lands first (encodes run in the thread pool, so order varies):
_encode_and_write_as_completedis still awaiting the other encode.asyncio.gatherruns the done-callbacks of already-finished children synchronously (the "finished eagerly" optimization). Withreturn_exceptions=False, the failed child makes the outer future fail immediately.await gather(...)therefore doesn't suspend. Thefinallycancels the slow write before its first step, sowrite_startedis never set.Instrumented trace of a failing run:
The pipeline behaves correctly here: no write is left running, since it's cancelled before it starts. The test is what's wrong: it only means to check cancellation of an in-flight write.
Fix
Test-only. The failing write now waits (bounded, 5 s) until the slow write has started before raising, so every run exercises the in-flight case. The docstring explains why.
Verification
pytest-repeatand 16 parallel workers: 8/400 failed on py3.13, 3/400 on py3.14._cancel_and_draincall in_encode_and_write_as_completedreplaced bypass, the test fails onassert not write_finished.tests/test_fused_pipeline.pypasses.No changelog entry, since this is a test-only change.
🤖 Generated with Claude Code