Repository navigation
Conversation
…on a recorded image A deploy that changes the Talos image upgrades one node and then fails leaves that node on the new image. main cannot roll a node to another image at the same Talos version, so every later deploy fails the boot-image readback and no repair can land. Read a reviewed record that may accept one static node on one named image until a date. The node is reported as a warning and the result is never the plain pass; the Talos version, readiness, scheduling and node counts still bind it. A record that is not exactly the header and at most one well-formed row is refused before any node is read, and an expired row accepts nothing. Record the worker the failed deploy of #4197 left on its image. Part of #4665 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…l probe can make The runtime pull probe names its node, so the kubelet alone admits it. With a 16Mi memory request it was refused (OutOfmemory) by a worker that had 15 Mi left to request, the drain was refused, and the production deploy failed; the recovery deploy of main failed the same way on another worker. The probe only prints a version, so it now requests 1m CPU and 1Mi memory and keeps its limits. The fake kubectl rejects a probe that asks for more, so every probe test holds the manifest to that. Fixes #4664 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> (cherry picked from commit 3491077)
Tried as a user — at head The user of this change is the next production deploy. I ran the real check, read-only, against production at 19:38Z on 2026-10-08, with the record as committed here:
I also confirmed where that worker's image comes from: it is exactly the image #4197 defines, and The offline test covers what production cannot show safely: drift on a node the record does not name, the recorded node on another image, a second node on the recorded image, the recorded node while not ready, cordoned or on another Talos version, and eleven malformed records. Weakening the check in eight ways, one at a time, fails the test each time. For the commit carried from #4666: its Go tests pass here (one package, about six minutes) and the script is lint-clean. I did not exercise the probe on a full worker; that needs a deploy. Not reachable before merge: a real deploy passing both checks. That is this pull request's own queue deploy, and it is why it waits for the maintainer's approval first. |
There is now a root-repair alternative in #4669 at Local command regressions and exact-head review are complete. Hosted validation is still finishing, and protected deployment/readback remains required before this is called recovered. I have not changed this branch or treated the proposed exception as approved. |
@coderabbitai review |
✅ Action performedReview finished.
|
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configuration
⛔ Files ignored due to path filters (1)
📒 Files selected for processing (7)
Included review availability: This review used your included allowance. Your plan provides up to 1 included review per hour; 0 remain after this review. 📜 Recent review details🧰 Additional context used🧠 Learnings (1)📚 Learning: 2026-08-10T13:01:12.782ZApplied to files:
🔇 Additional comments (8)
📝 WalkthroughWalkthroughThe Talos readback script now supports a reviewed, dated exception for one static node on one specified non-current schematic. Tests cover accepted and rejected records, and CI runs the readback test when the record changes. Documentation describes exception handling and node recovery. The runtime pull probe’s CPU and memory requests are reduced; its limits are unchanged. Priority: ➖ Normal Merge Risk: ⚪ Minimal · up to The deploy uses the intended dated exception. No established issue blocks merge, though the required deployment checks remain pending. 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
Full details: Docstring CoverageExplanation Docstring coverage is 22.22% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 9 functions across 4 files. (3 skipped: 3 unsupported.)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Closing without merging: the state this was approved for no longer exists. The maintainer approved queueing this first, with a standing instruction to check it is still relevant before acting. Checked on 2026-10-09 at 12:27Z, read-only:
So nothing here should land as written, and it was not queued. The exception mechanism and its tests stay available on this pull request for the open criterion on #4665 (a reviewed way back when a failed deploy strands a node), if that is the design chosen for it. |
Why
No production deploy can finish. A deploy that failed on 7 October left one worker on a newer boot image, every deploy since fails the final check that all nodes run the expected image, and redeploying cannot fix the node. A second check also stops deploys while the workers are full. That keeps every repair, including the one for the secrets store, from landing.
What
The boot-image check can now be told, in one reviewed and dated record, that a single named node is known to be on a single named image. It then lets the deploy finish with a visible warning instead of failing, and goes back to strict the day after the date. The record added here names that one worker and lapses on 22 October.
This also carries the one-commit fix from #4666, which lets the registry-credential probe run on a worker that is almost full. Each pull request's own deploy has to pass both checks, so neither fix can land alone.
Part of #4665
Part of #4664
👉 After merge/promotion: the node still needs repairing; the record only lets deploys through meanwhile. Remove the record once every node is back on one image.
🤖 Generated with Claude Code