Skip to content

fix(longhorn): default to a WaitForFirstConsumer StorageClass - #2421

Merged
devantler merged 2 commits into
mainfrom
claude/longhorn-wffc-default-storageclass
Jul 3, 2026
Merged

devantler merged 2 commits into
mainfrom
claude/longhorn-wffc-default-storageclass

Conversation

@devantler

Copy link
Copy Markdown
Contributor

🤖 Generated by the Daily AI Assistant

Why

The default longhorn StorageClass uses volumeBindingMode: Immediate, so a PVC binds and its volume is provisioned before the pod is scheduled. Stateful pods then get dragged onto whatever node the volume landed on — including autoscaler overflow nodes — where they block scale-down and defeat the restrict-storage-to-baseline-workers intent. It also breaks the cluster-autoscaler’s scale-up simulation.

What

Adds a longhorn-wffc StorageClass (WaitForFirstConsumer, identical Longhorn parameters) and makes it the cluster default; demotes the chart’s longhorn class to non-default but keeps it. New default-SC PVCs now bind lazily so the scheduler places the pod first and the volume follows.

Safe rollout — no data loss (this is Phase 1 only)

  • Existing data is untouched. The longhorn class is kept (CNPG clusters and 14 live PVCs reference it by name); only its default annotation moves.
  • Phase 2 (separate, per-DB, not in this PR): migrate the stranded stateful workloads (umami-db, wedding-db, crossview-postgres, coroot/backstage CNPG) onto longhorn-wffc one at a time via CNPG replica-recreation / Longhorn backup→restore, verifying data each time. Never bulk-delete PVCs — reclaimPolicy: Delete destroys the volume.
  • After promotion, verify: longhorn SC still exists (non-default) and longhorn-wffc is the sole default before anything creates new PVCs.

Pairs with the autoscaler scale-down knobs (ksail#5756) and the cx33 removal (#2420) to complete node right-sizing.

The default longhorn StorageClass uses volumeBindingMode: Immediate, so a PVC
binds and its Longhorn volume is provisioned before the consuming pod is
scheduled. That drags stateful pods onto whatever node the volume landed on —
including autoscaler overflow nodes — which then can never scale down, and it
breaks the cluster-autoscaler's scale-up simulation.

Add a sibling longhorn-wffc StorageClass (WaitForFirstConsumer, identical
parameters) and make it the cluster default; demote the chart-managed longhorn
class to non-default but keep it, since existing PVCs and CNPG clusters name it
directly. Existing volumes are untouched; migrating them onto longhorn-wffc is a
deliberate per-workload follow-up (reclaimPolicy is Delete — no bulk PVC deletes).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Jul 3, 2026 •

Copy link
Copy Markdown
Contributor

Review Change Stack

📝 Walkthrough

Walkthrough

This PR changes the default Longhorn StorageClass setup on Hetzner. It adds a new longhorn-wffc StorageClass with WaitForFirstConsumer, marks it default, demotes the Helm-managed longhorn class, and includes the new manifest in kustomization.

Changes

Longhorn StorageClass Defaulting

Layer / File(s) Summary
New longhorn-wffc StorageClass definition
k8s/providers/hetzner/infrastructure/controllers/longhorn/storage-class-longhorn-wffc.yaml
Adds a new Longhorn-backed StorageClass marked as the cluster default, using WaitForFirstConsumer, Delete, volume expansion, and Longhorn parameter settings.
Demote Helm-managed StorageClass and wire new resource
k8s/providers/hetzner/infrastructure/controllers/longhorn/helm-release.yaml, k8s/providers/hetzner/infrastructure/controllers/longhorn/kustomization.yaml
Sets defaultSettings.defaultClass to false in the Helm-managed longhorn StorageClass with comments, and adds storage-class-longhorn-wffc.yaml to the kustomization resources list.

Estimated code review effort: 2 (Simple) | ~10 minutes

Related issues: None provided.

Related PRs: None provided.

Suggested labels: kubernetes, storage, hetzner

Suggested reviewers: devantler

🐰 A poem for this PR

A burrow needs its default den,
so longhorn steps aside just then.
Now wffc waits with patient care,
for pods to land before it's there.
Hop along, ye PVCs bound— 🐇
a wiser storage class is found!

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Title check ✅ Passed The title accurately summarizes the main change: switching the default Longhorn StorageClass to WaitForFirstConsumer.
Description check ✅ Passed The description matches the PR: it explains the new default WFFC StorageClass, the demoted legacy class, and the phased rollout.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch claude/longhorn-wffc-default-storageclass

Comment @coderabbitai help to get the list of available commands.

@devantler
devantler marked this pull request as ready for review July 3, 2026 19:16

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
k8s/providers/hetzner/infrastructure/controllers/longhorn/storage-class-longhorn-wffc.yaml (1)

27-28: 🗄️ Data Integrity & Integration | 🔵 Trivial | ⚡ Quick win

Avoid manual sync: reuse the templated replica-count variable.

numberOfReplicas: "3" is hardcoded here, while the sibling longhorn class in helm-release.yaml drives the same value via ${longhorn_replica_count:=3}. The comment on Line 27 acknowledges this requires manual sync — that's an avoidable drift risk if the variable is ever changed without updating this file.

♻️ Suggested fix using the shared variable
-  # Mirrors the chart default (longhorn_replica_count, default 3); keep in sync.
-  numberOfReplicas: "3"
+  numberOfReplicas: "${longhorn_replica_count:=3}"

If this directory isn't subject to the same variable-substitution pipeline as helm-release.yaml, disregard this suggestion.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In
`@k8s/providers/hetzner/infrastructure/controllers/longhorn/storage-class-longhorn-wffc.yaml`
around lines 27 - 28, The Longhorn storage class is hardcoding the replica count
instead of reusing the shared templated value, which risks drift from the
sibling configuration. Update the `numberOfReplicas` setting in the Longhorn
storage class to use the same `${longhorn_replica_count:=3}` substitution
pattern used by the `longhorn` class in `helm-release.yaml`, keeping the comment
and configuration in sync via the shared variable.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In
`@k8s/providers/hetzner/infrastructure/controllers/longhorn/storage-class-longhorn-wffc.yaml`:
- Around line 27-28: The Longhorn storage class is hardcoding the replica count
instead of reusing the shared templated value, which risks drift from the
sibling configuration. Update the `numberOfReplicas` setting in the Longhorn
storage class to use the same `${longhorn_replica_count:=3}` substitution
pattern used by the `longhorn` class in `helm-release.yaml`, keeping the comment
and configuration in sync via the shared variable.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: b42402d8-aba8-4f7f-9292-8d2c6ae8359a

📥 Commits

Reviewing files that changed from the base of the PR and between ac8ca46 and d0ba6bb.

📒 Files selected for processing (1)
  • k8s/providers/hetzner/infrastructure/controllers/longhorn/storage-class-longhorn-wffc.yaml
📜 Review details
🧰 Additional context used
📓 Path-based instructions (3)
k8s/providers/**/*.yaml

📄 CodeRabbit inference engine (AGENTS.md)

Use Kustomize patches: in provider overlays instead of directly editing base resources; provider overlays are the correct place for environment-specific changes.

Files:

  • k8s/providers/hetzner/infrastructure/controllers/longhorn/storage-class-longhorn-wffc.yaml
k8s/**/*.{yaml,yml}

📄 CodeRabbit inference engine (AGENTS.md)

Use one Kubernetes resource per file, except for the explicitly whitelisted vendored operator bundles.

Files:

  • k8s/providers/hetzner/infrastructure/controllers/longhorn/storage-class-longhorn-wffc.yaml
k8s/**/{controllers,infrastructure}/**/*.yaml

📄 CodeRabbit inference engine (AGENTS.md)

Use kebab-case directory names and resource-led filenames; CR folders may use intent-describing ‹verb›-‹purpose›.yaml names and must stay kebab-case.

Files:

  • k8s/providers/hetzner/infrastructure/controllers/longhorn/storage-class-longhorn-wffc.yaml
🧠 Learnings (1)
📚 Learning: 2026-07-01T21:13:36.950Z
Learnt from: devantler
Repo: devantler-tech/platform PR: 2359
File: k8s/bases/apps/actual-budget/helm-release.yaml:62-111
Timestamp: 2026-07-01T21:13:36.950Z
Learning: When reviewing Kustomize/Helm YAML in this repo, keep the base vs provider overlay split: `k8s/bases/apps/**` and `k8s/bases/infrastructure/**` should contain each app’s full, environment-agnostic configuration (including base-level postRenderer Kustomize patches such as deployment strategy, topology spread, probes, and env injection). `k8s/providers/{docker,hetzner}/**` should only add small provider-specific deltas (e.g., `interval`, `persistence.size`) via patch files (like `k8s/providers/<provider>/apps/<app>/patches/helm-release-patch.yaml`). If configuration is identical across providers (e.g., OIDC/OAuth env vars where `${domain}` is resolved per cluster via envsubst), it belongs in the base and must not be duplicated into provider overlays.

Applied to files:

  • k8s/providers/hetzner/infrastructure/controllers/longhorn/storage-class-longhorn-wffc.yaml
🔇 Additional comments (1)
k8s/providers/hetzner/infrastructure/controllers/longhorn/storage-class-longhorn-wffc.yaml (1)

1-26: LGTM!

Also applies to: 29-37

@devantler

Copy link
Copy Markdown
Contributor Author

🤖 Generated by the Daily AI Assistant

Re: CodeRabbit nitpick — numberOfReplicas templated-variable reuse (storage-class-longhorn-wffc.yaml:27-28)

Keeping the literal numberOfReplicas: "3" — declining the ${longhorn_replica_count:=3} substitution here, deliberately:

  • This is a raw StorageClass, where parameters is strictly typed map[string]string. kubeconform (Validate-Manifests) validates the value as-is. The templated form in this exact field already broke Validate-Manifests once (the substituted value serialized as a bare integer 3, which kubeconform rejects for a string-typed parameter) — the literal string is the fix that greened it.
  • The sibling longhorn class carries the variable because it lives inside a HelmRelease (.values), which kubeconform treats as free-form — not the same validation path as a StorageClass parameters map, so the two aren't equivalent.
  • Drift risk is real but minor and already mitigated by the explicit # keep in sync comment. For a fixed replica count that must be a literal string at validation time, robustness beats DRY here.

Leaving line 27-28 as-is.

@devantler
devantler added this pull request to the merge queue Jul 3, 2026
Merged via the queue into main with commit 5457cb6 Jul 3, 2026
15 checks passed
@devantler
devantler deleted the claude/longhorn-wffc-default-storageclass branch July 3, 2026 20:40
@github-project-automation github-project-automation Bot moved this from 🫴 Ready to ✅ Done in 🌊 Project Board Jul 3, 2026
@botantler-1

botantler-1 Bot commented Jul 3, 2026

Copy link
Copy Markdown
Contributor

🎉 This PR is included in version 1.96.3 🎉

The release is available on GitHub release

Your semantic-release bot 📦🚀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

Archived in project

Development

Successfully merging this pull request may close these issues.

1 participant