Skip to content

fix(crossview): stop auto-vpa OOM-killing the dashboard under load - #2472

Merged
devantler merged 1 commit into
mainfrom
claude/crossview-vpa-memory-burst-headroom
Jul 4, 2026
Merged

devantler merged 1 commit into
mainfrom
claude/crossview-vpa-memory-burst-headroom

Conversation

@devantler

Copy link
Copy Markdown
Contributor

🤖 Generated by the Daily AI Assistant

Why
The Crossview dashboard (crossview.platform.devantler.tech) shows "Error loading" on every Crossplane widget and looks like it has lost Kubernetes API access. It hasn't — the pod is repeatedly OOM-killed while a dashboard loads. Crossview sits near-empty at idle, but a dashboard load lists Crossplane resources cluster-wide and briefly needs ~17× that memory; the cluster's auto-VPA had right-sized its memory limit down toward the idle footprint, so the pod dies mid-request and every widget fails.

What
Give Crossview an explicit, generous memory limit so the auto-VPA keeps enough burst headroom — the same fix already applied to the homepage dashboard for this exact VPA behaviour. No new dependency and no behaviour change; requests are unchanged so it reserves no extra node memory at idle, and the VPA's 6Gi cap still guards against leaks.

Operational: own PR — needs a direct merge after promotion.

Crossview idles at ~14Mi but a dashboard load lists Crossplane resources
cluster-wide and spikes to ~240Mi+. The Kyverno auto-vpa VPA
(controlledValues: RequestsAndLimits) right-sizes the memory request to the
idle histogram and, preserving the authored limit:request ratio, drags the
limit down with it; at the chart's default 4:1 ratio a transient-low request
recommendation pins the limit near ~400Mi, below the burst, and the pod is
OOMKilled mid-request. Every dashboard widget then shows "Error loading" so
Crossview looks like it lost kube-API access (SA token, RBAC and network are
all fine).

Author resources explicitly with a wide 8:1 memory ratio (256Mi request /
2Gi limit), mirroring the homepage HelmRelease's fix for the same auto-vpa
footgun. VPA still scales the request to the working set, but the preserved
8:1 ratio keeps the limit generous at every request level, absorbing the
burst. The limit is a ceiling, not a reservation; auto-vpa maxAllowed (6Gi)
remains the leak guard.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Jul 4, 2026 •

Copy link
Copy Markdown
Contributor

Review Change Stack

📝 Walkthrough

Walkthrough

This change updates the Crossview HelmRelease configuration to explicitly define resource requests and limits for the application: requests of 250m CPU and 256Mi memory, and limits of 1 CPU and 2Gi memory. It also adds inline documentation explaining a memory sizing issue related to auto-VPA behavior and the reasoning behind the chosen request-to-limit ratio to avoid burst-time OOM kills.

Changes

File Change Summary
k8s/bases/apps/crossview/helm-release.yaml Added explicit resources block (requests/limits for CPU/memory) with explanatory comments on VPA/OOM sizing.

Sequence Diagram(s)

Not applicable — this change is a configuration/documentation update with no code execution flow to diagram.

Estimated code review effort: 1 (Low)

Related issues: None specified.

Related PRs: None specified.

Suggested labels: kubernetes, configuration

Suggested reviewers: None specified.

Poem

A rabbit hops through YAML fields so neat,
Setting limits so no memory pod will overheat,
Requests of two-fifty-six, limits of two Gi bright,
No more OOMKilled bursts in the night,
Hop, hop, hooray — resources set just right! 🐇

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Title check ✅ Passed The title clearly matches the change: adding explicit resources to prevent auto-VPA OOMKills under load.
Description check ✅ Passed The description is directly related to the Crossview memory-limit fix and explains the same OOMKilled behavior.

Comment @coderabbitai help to get the list of available commands.

@devantler
devantler marked this pull request as ready for review July 4, 2026 19:57
@devantler
devantler added this pull request to the merge queue Jul 4, 2026
Merged via the queue into main with commit 14f11a5 Jul 4, 2026
15 checks passed
@devantler
devantler deleted the claude/crossview-vpa-memory-burst-headroom branch July 4, 2026 20:24
@github-project-automation github-project-automation Bot moved this from 🫴 Ready to ✅ Done in 🌊 Project Board Jul 4, 2026
@botantler-1

botantler-1 Bot commented Jul 4, 2026

Copy link
Copy Markdown
Contributor

🎉 This PR is included in version 1.98.3 🎉

The release is available on GitHub release

Your semantic-release bot 📦🚀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

Archived in project

Development

Successfully merging this pull request may close these issues.

1 participant