Summary
TestSteer_ControlTargetsPromotedRun (internal/adapter/server/steer_handoff_test.go, landed with #570 steer-while-running) intermittently fails under -race with:
steer_handoff_test.go:441: promoted run was not registered after steer
Root cause
The test polls for an async multi-goroutine handoff (gRPC dial + stream + blocking-tool sync + atomic run-pointer swap) to complete against a hardcoded 5-second deadline. -race slows execution roughly 2-10x; on a loaded shared CI runner that's occasionally enough to blow the fixed budget for work that is completing correctly, just slowly. (The redis: connection pool: ... connection refused lines that appear near this failure in the CI log are unrelated noise from a different parallel subtest in the same package — not the cause.)
Where it was seen
PR #661, Test (race) job: https://github.com/stacklok/mecatl/actions/runs/32353522818/job/96377769838
Suggested fix
Either widen the deadline specifically under -race (testing.Short()/build-tag/env-based bump), or better, replace the fixed-deadline poll with a real synchronization signal from the handoff path (a channel/callback fired when the promoted run is actually registered) so the test has no timing dependency at all.
Confidence
High — root-caused from the actual assertion failure and the test's poll-loop code, not inferred. Not related to any change in the mecak8s Helm-convergence PR stack (#659/#661/#662/#663); the test file is untouched by that stack. Since this is a newly-landed test, it will likely keep tripping other PRs' CI at random until fixed.
Summary
TestSteer_ControlTargetsPromotedRun(internal/adapter/server/steer_handoff_test.go, landed with #570 steer-while-running) intermittently fails under-racewith:Root cause
The test polls for an async multi-goroutine handoff (gRPC dial + stream + blocking-tool sync + atomic run-pointer swap) to complete against a hardcoded 5-second deadline.
-raceslows execution roughly 2-10x; on a loaded shared CI runner that's occasionally enough to blow the fixed budget for work that is completing correctly, just slowly. (Theredis: connection pool: ... connection refusedlines that appear near this failure in the CI log are unrelated noise from a different parallel subtest in the same package — not the cause.)Where it was seen
PR #661,
Test (race)job: https://github.com/stacklok/mecatl/actions/runs/32353522818/job/96377769838Suggested fix
Either widen the deadline specifically under
-race(testing.Short()/build-tag/env-based bump), or better, replace the fixed-deadline poll with a real synchronization signal from the handoff path (a channel/callback fired when the promoted run is actually registered) so the test has no timing dependency at all.Confidence
High — root-caused from the actual assertion failure and the test's poll-loop code, not inferred. Not related to any change in the mecak8s Helm-convergence PR stack (#659/#661/#662/#663); the test file is untouched by that stack. Since this is a newly-landed test, it will likely keep tripping other PRs' CI at random until fixed.