Skip to content

fix(observability): complete OpenCost production retirement - #3822

Merged
devantler merged 4 commits into
mainfrom
codex/opencost-production-retirement-3810
Sep 14, 2026
Merged

devantler merged 4 commits into
mainfrom
codex/opencost-production-retirement-3810

Conversation

@devantler

@devantler devantler commented Sep 14, 2026 •

Copy link
Copy Markdown
Contributor

🤖 Generated by the Agentic Engineer

Why

PR #3818 removed OpenCost from the desired manifests, but production correctly retained its prune-protected release and namespace. OpenCost is therefore still running and the convergence is incomplete.

What

Add a default-off, protected production action that safely removes only the orphaned OpenCost installation and independently verifies it is gone. Merge-group and recovery deploys cannot activate the deletion.

Operational gate: after this merges, dispatch CD from main with retire-opencost enabled; then remove the one-shot tooling.

Part of #3810

Add a default-off manual main deployment path that safely deletes the prune-protected OpenCost HelmRelease and namespace after Flux convergence, then verifies the orphan detector is clean. Include behavioral and wiring regressions for ordering, ownership, PVC, and merge-group safety.\n\nRefs #3810
Reuse the existing required Coroot convergence entrypoint for the OpenCost retirement behavior and wiring tests, avoiding overlap with other active owners of the shared CI workflow.\n\nRefs #3810
@devantler

Copy link
Copy Markdown
Contributor Author

🤖 Generated by the Agentic Engineer

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Sep 14, 2026 •

Copy link
Copy Markdown
Contributor

@devantler I will review the changes in #3822.

⚠️ Action not completed

Review rate limited.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@coderabbitai

coderabbitai Bot commented Sep 14, 2026

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

Next included review available in 52 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Learn how review limits work.

Review configuration:

⚙️ Run configuration

Configuration used: Repository YAML (base), Organization UI (inherited)

Review profile: ASSERTIVE

Plan: Advanced

Run ID: 47412002-e29c-4874-b9aa-b787c18b7d27

📥 Commits

Reviewing files that changed from the base of the PR and between 70e2142 and ade8750.

📒 Files selected for processing (6)
  • .github/actions/deploy-prod/action.yml
  • .github/workflows/cd.yaml
  • scripts/retire-opencost.sh
  • scripts/tests/test-coroot-cost-convergence.sh
  • scripts/tests/test-opencost-retirement-wiring.sh
  • scripts/tests/test-opencost-retirement.sh

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@devantler

Copy link
Copy Markdown
Contributor Author

🤖 Generated by the Agentic Engineer

@codex review for the fail-closed OpenCost retirement path, GitHub Actions activation boundary, deletion ordering, and independent runtime verification

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 14, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-09-14T22:58:10.504336Z 40ff4f4 Manual request
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: ade8750579

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread scripts/retire-opencost.sh Resolved
Refuse stale manual dispatches before publication and execution, and register every retirement safety file in the authoritative k8s change filter.\n\nRefs #3810
@devantler

Copy link
Copy Markdown
Contributor Author

🤖 Generated by the Agentic Engineer

@codex review for the resolved CI path-selection and stale-main dispatch findings, plus the complete fail-closed retirement flow

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 9a5692c4d3

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread scripts/retire-opencost.sh Resolved
@devantler

Copy link
Copy Markdown
Contributor Author

🤖 Generated by the Agentic Engineer

@codex review for verify the Flux inventory fail-closed fix and the complete production-retirement safety path

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Breezy!

Reviewed commit: 40ff4f42d5

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@devantler

Copy link
Copy Markdown
Contributor Author

🤖 Generated by the Agentic Engineer

Readiness evidence

  • Exact PR head: 40ff4f42d504f37f0ded8b9d54c8162faa5f0aef; live main at closing rebind: 29ba6bc970beabe1a129919e83cc328e058167d8; GitHub reports the PR mergeable and clean.
  • TDD: the new missing-inventory case failed before the implementation change, then passed after the script began requiring .status.inventory.entries to be an array before interpreting absence as orphan proof.
  • Local behavior: retirement, wiring, aggregate Coroot convergence, prune-protected orphan alert, PVC-prune safety, ShellCheck, actionlint, and production-authorization unit checks pass.
  • CI: 29/29 checks are terminal with no failures (23 success, 6 expected skips).
  • Review: Codex completed an exact-head review on 40ff4f4 with no findings and no unresolved threads. CodeRabbit was explicitly unavailable under its recorded rate-limit window, so the configured fallback reviewer was used.
  • User evaluation: the fake-cluster acceptance test proves HelmRelease-before-Namespace ordering, current-main binding, default-off behavior, PVC/ownership/inventory refusal, cleanup verification, and no destructive command on every refusal path. The real production deletion remains deliberately gated to a manual main CD dispatch after merge.

Ready for the merge queue.

@devantler
devantler marked this pull request as ready for review September 14, 2026 23:10
@devantler
devantler added this pull request to the merge queue Sep 14, 2026
Merged via the queue into main with commit 07c04c5 Sep 14, 2026
30 checks passed
@devantler
devantler deleted the codex/opencost-production-retirement-3810 branch September 14, 2026 23:24
@github-project-automation github-project-automation Bot moved this from 🫴 Ready to ✅ Done in 🌊 Project Board Sep 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: ✅ Done

Development

Successfully merging this pull request may close these issues.

1 participant