Skip to content

Make the Longhorn storage classes retain deleted volumes #4439

Description

@devantler

🤖 Generated by the Agentic Engineer

Problem

Every Longhorn StorageClass in production has the Delete reclaim policy. Measured on 2026-10-03: longhorn-wffc (the default), longhorn and longhorn-static, with 14 volumes between them.

A volume takes its policy from its class when it is provisioned. So every new database or application volume is created to be destroyed together with its claim. The only thing in front of that today is the prune: disabled annotation that #3369 removes.

Proposal

This is step 1 of the rollout in docs/deletion-and-data-retention.md (the ADR for #3369). Each class keeps its name and is replaced in place:

  • longhorn-wffc: set reclaimPolicy: Retain in its manifest only after a separately reviewed and rehearsed bounded class replacement. The field is immutable and the layer now sets force: false; ordinary Flux apply rejects the change. Re-enabling layer-wide force is not an acceptable migration mechanism.
  • longhorn: set persistence.reclaimPolicy: Retain in the chart values. Longhorn recreates the class when the chart's StorageClass ConfigMap changes.
  • longhorn-static: Longhorn creates this class only when it is missing and sets no policy on it. Rehearse bounded replacement before declaring it in Git with Retain; ordinary apply cannot change the existing immutable policy.

This changes no existing volume (that is step 3) and removes no annotation.

Acceptance criteria

  • On the live cluster all three classes report Retain, and every other field is unchanged: provisioner, binding mode, parameters, default-class marker.
  • The next volume provisioned through a class that is in use reports Retain. A scratch claim delivered through Git is an acceptable way to observe this.
  • The existing claims and volumes are still bound, and their workloads were not restarted by the change.
  • The first nightly backup after the change leaves no Released volume behind. Velero's data mover sets Delete on its own temporary volumes; this confirms it on this cluster.
  • A test fails when a rendered Longhorn class is not Retain.
  • Rehearse exact-class replacement with concurrent provisioning fenced, all other class fields preserved, and no PVC/PV UID or binding changes. No layer-wide force is enabled.
  • Describe forward repair preserving Retain and a separately reviewed reverse replacement if necessary. A plain Git revert cannot replace immutable fields. Existing PV policies remain Retain.

Rough size: small.

Activity

  1. added theissue type on Oct 3, 2026
  2. devantler commented on Oct 5, 2026

    @devantler
    ContributorAuthor

    🤖 Generated by the Agentic Engineer

    Fresh bounded readback confirms the Longhorn class migration is still required. The class identity and its existing claims must be preserved; changing the immutable reclaim policy through ordinary apply is not a valid delivery. The reviewed layer keeps force disabled.

    The next concrete gate is an exact-class replacement rehearsal with provisioning fenced and before/after binding identities joined. Preserve all other class fields and measure the replacement interval. Neither a chart render nor a passing general health check completes the live acceptance. No replacement or storage mutation was performed in this round.

  3. moved this from 📥 Backlog to 🧊 Icebox in 🌊 Project Boardon Oct 8, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Fields

    Priority

    None yet

    Start date

    None yet

    Target date

    None yet

    Effort

    None yet

    Projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions