Skip to content

Keep guarded standby recovery live during status churn #4703

Closed

Description

@devantler

🤖 Generated by the Agentic Engineer

Why

The protected standby recovery stops safely when an operator status update invalidates the Cluster version test. The operator can update diagnostic status repeatedly while the protected recovery state is unchanged. An offline replay accepts the exact patch against its snapshot and rejects it after version-only churn. This explains a reproducible failure mechanism, not the specific failed predicate of an earlier live request.

What

Keep resource-version protection on every request. Add a small, separately authorized completed-join continuation that can re-prove its evidence and submit a fresh conditional pause or resume after a complete native Kubernetes rejection. Do not replace the version test with semantic-only concurrency control.

Acceptance:

  • No more than five attempts within a bounded deadline. Admit a further attempt only after a complete native 422 Status/Failure/Invalid response; stop for unknown, incomplete, timeout, conflict, throttled or ambiguous outcomes.
  • Freeze the original protected phase state. Re-read and prove current-main source, identities, full spec/generation/annotations, primary and health, backup, retained original storage, backing volume and operator leader before each request. Stop on protected-state drift; do not rebaseline it.
  • Keep the operator acknowledgment and all pre-deletion checks. Job and claim deletion remain one-shot with exact UID/version preconditions. The original backing volume is never deleted or reused.
  • The separately authorized diagnostic remains exactly one non-persisting request. Legacy recovery modes do not gain retries.
  • Regression tests exercise real patch application, benign status churn, protected-state changes, source/storage/leader drift, uncertain success, exhaustion and no premature destructive writes.

Part of #4673. Implementation and merge are not execution authority: the maintainer must specifically approve one new protected continuation after its review and merge. Consumed dispatches are never rerun.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

Fields

Priority

None yet

Effort

None yet

Projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions